2024-09-11T00:38:01.902 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T00:38:01.929 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T00:38:02.036 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899863' kernel: client: sha1: distro flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDrpktxf2I/1vIStXiVXkELgBtNaRq6j79Qbs9v1M3DN+MwKJOqCjz5E6Q4KjryNoQ5XPc+GGsaqoUqIE8EjVlM= smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDMuLsmRAoUa9CCEP59c0c2CAGowuOaf8MLvAOj/mnDl7oM1LwSbU8O3v+kiNLCaTsUYkP1itA7A7bTpoP0CDDg= smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6jjDpi3sjPOhQL6A2xTW9PpjIaLBK7BEaAZohso8K/5NvekmZFjOPXY5KtVrCsMgBvDgUQyUJ617/zI4I+JC8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T00:38:02.036 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-11T00:38:02.037 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-11T00:38:02.037 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T00:38:02.039 INFO:teuthology.task.internal:Checking packages... 2024-09-11T00:38:02.061 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-11T00:38:02.061 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T00:38:02.061 INFO:teuthology.packaging:ref: None 2024-09-11T00:38:02.061 INFO:teuthology.packaging:tag: None 2024-09-11T00:38:02.062 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-11T00:38:02.062 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:38:02.062 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-11T00:38:02.214 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854.g8f219c47 2024-09-11T00:38:02.216 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T00:38:02.223 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T00:38:02.223 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T00:38:02.365 INFO:teuthology.task.internal:Saving configuration 2024-09-11T00:38:02.386 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T00:38:02.429 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T00:38:02.463 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 00:33:03.625388', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDrpktxf2I/1vIStXiVXkELgBtNaRq6j79Qbs9v1M3DN+MwKJOqCjz5E6Q4KjryNoQ5XPc+GGsaqoUqIE8EjVlM='} 2024-09-11T00:38:02.491 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 00:33:03.621822', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDMuLsmRAoUa9CCEP59c0c2CAGowuOaf8MLvAOj/mnDl7oM1LwSbU8O3v+kiNLCaTsUYkP1itA7A7bTpoP0CDDg='} 2024-09-11T00:38:02.519 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 00:33:03.623806', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6jjDpi3sjPOhQL6A2xTW9PpjIaLBK7BEaAZohso8K/5NvekmZFjOPXY5KtVrCsMgBvDgUQyUJ617/zI4I+JC8='} 2024-09-11T00:38:02.519 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T00:38:02.530 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-11T00:38:02.531 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-11T00:38:02.531 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-11T00:38:02.531 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T00:38:02.682 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd5fff3fbe0>, signals=[15]) 2024-09-11T00:38:02.683 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T00:38:02.691 INFO:teuthology.task.internal:Opening connections... 2024-09-11T00:38:02.691 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T00:38:02.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:38:02.777 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-09-11T00:38:02.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:38:02.863 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-09-11T00:38:02.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:38:02.962 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T00:38:03.367 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-09-11T00:38:03.386 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-09-11T00:38:03.386 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-09-11T00:38:03.445 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream" 2024-09-11T00:38:03.445 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="9" 2024-09-11T00:38:03.445 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos" 2024-09-11T00:38:03.445 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora" 2024-09-11T00:38:03.445 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="9" 2024-09-11T00:38:03.445 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el9" 2024-09-11T00:38:03.445 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T00:38:03.445 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-09-11T00:38:03.445 INFO:teuthology.orchestra.run.smithi078.stdout:LOGO="fedora-logo-icon" 2024-09-11T00:38:03.446 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T00:38:03.446 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/" 2024-09-11T00:38:03.446 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T00:38:03.446 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T00:38:03.446 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T00:38:03.446 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-09-11T00:38:03.478 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-09-11T00:38:03.496 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-09-11T00:38:03.496 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-09-11T00:38:03.554 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-09-11T00:38:03.554 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-09-11T00:38:03.554 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-09-11T00:38:03.555 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-09-11T00:38:03.555 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-09-11T00:38:03.555 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-09-11T00:38:03.555 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T00:38:03.555 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-09-11T00:38:03.555 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-09-11T00:38:03.555 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T00:38:03.555 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-09-11T00:38:03.555 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T00:38:03.555 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T00:38:03.555 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T00:38:03.556 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-09-11T00:38:03.603 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-09-11T00:38:03.622 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-09-11T00:38:03.622 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-09-11T00:38:03.679 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-09-11T00:38:03.679 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-09-11T00:38:03.680 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-09-11T00:38:03.680 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-09-11T00:38:03.680 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-09-11T00:38:03.680 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-09-11T00:38:03.680 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T00:38:03.680 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-09-11T00:38:03.680 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-09-11T00:38:03.680 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T00:38:03.680 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-09-11T00:38:03.680 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T00:38:03.680 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T00:38:03.680 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T00:38:03.681 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-09-11T00:38:03.711 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T00:38:03.724 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T00:38:03.792 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T00:38:03.792 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-09-11T00:38:03.795 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-09-11T00:38:03.797 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-09-11T00:38:03.814 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T00:38:03.872 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T00:38:03.872 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-09-11T00:38:03.875 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-09-11T00:38:03.877 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-09-11T00:38:03.907 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T00:38:04.062 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T00:38:04.076 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-11T00:38:04.076 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-11T00:38:04.076 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-11T00:38:04.077 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T00:38:04.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:38:04.094 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-09-11T00:38:04.152 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-503.el9.x86_64 2024-09-11T00:38:04.152 INFO:teuthology.task.kernel:Running kernel on smithi078: 5.14.0-503.el9.x86_64 2024-09-11T00:38:04.152 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum install -y kernel 2024-09-11T00:38:08.136 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 3.6 MB/s | 8.2 MB 00:02 2024-09-11T00:38:11.870 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 20 MB 00:02 2024-09-11T00:38:20.646 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 6.5 MB 00:04 2024-09-11T00:38:22.155 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 116 kB/s | 19 kB 00:00 2024-09-11T00:38:23.159 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-11T00:38:28.540 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-11T00:38:30.538 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T00:38:30.538 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T00:38:30.626 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-09-11T00:38:30.637 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-09-11T00:38:30.637 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-09-11T00:38:30.637 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-09-11T00:38:30.637 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-09-11T00:38:30.637 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T00:38:30.637 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-09-11T00:38:30.637 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T00:38:30.637 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T00:38:30.637 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T00:38:30.637 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:38:30.637 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-09-11T00:38:30.637 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-09-11T00:38:30.638 INFO:teuthology.orchestra.run.smithi078.stdout:Install 4 Packages 2024-09-11T00:38:30.638 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:38:30.638 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 79 M 2024-09-11T00:38:30.638 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 126 M 2024-09-11T00:38:30.638 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-09-11T00:38:30.975 INFO:teuthology.orchestra.run.smithi078.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 275 kB/s | 23 kB 00:00 2024-09-11T00:38:34.417 INFO:teuthology.orchestra.run.smithi078.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 4.4 MB/s | 16 MB 00:03 2024-09-11T00:38:35.817 INFO:teuthology.orchestra.run.smithi078.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 5.9 MB/s | 29 MB 00:04 2024-09-11T00:38:38.258 INFO:teuthology.orchestra.run.smithi078.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 4.7 MB/s | 35 MB 00:07 2024-09-11T00:38:38.258 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-09-11T00:38:38.259 INFO:teuthology.orchestra.run.smithi078.stdout:Total 10 MB/s | 79 MB 00:07 2024-09-11T00:38:38.940 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-09-11T00:38:39.293 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-09-11T00:38:39.293 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-09-11T00:38:40.262 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-09-11T00:38:40.262 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-09-11T00:38:42.721 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-09-11T00:38:43.350 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T00:38:43.426 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T00:38:46.217 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T00:38:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T00:38:51.173 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T00:38:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T00:38:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T00:39:34.453 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T00:39:34.492 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T00:39:34.805 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T00:39:34.806 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T00:39:34.806 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T00:39:34.806 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T00:39:35.619 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T00:39:35.619 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:39:35.619 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-09-11T00:39:35.619 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T00:39:35.619 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T00:39:35.619 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T00:39:35.619 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T00:39:35.620 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:39:35.620 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-09-11T00:39:35.984 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T00:39:36.064 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T00:39:36.065 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T00:39:36.065 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T00:39:36.065 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T00:39:36.065 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T00:39:36.065 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-11T00:39:36.065 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi078.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T00:39:36.065 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T00:39:36.124 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T00:39:36.124 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T00:39:36.124 DEBUG:teuthology.orchestra.run.smithi078:> sudo rpm -qi grub2-tools 2024-09-11T00:39:36.159 INFO:teuthology.orchestra.run.smithi078.stdout:Name : grub2-tools 2024-09-11T00:39:36.159 INFO:teuthology.orchestra.run.smithi078.stdout:Epoch : 1 2024-09-11T00:39:36.159 INFO:teuthology.orchestra.run.smithi078.stdout:Version : 2.06 2024-09-11T00:39:36.159 INFO:teuthology.orchestra.run.smithi078.stdout:Release : 82.el9 2024-09-11T00:39:36.159 INFO:teuthology.orchestra.run.smithi078.stdout:Architecture: x86_64 2024-09-11T00:39:36.159 INFO:teuthology.orchestra.run.smithi078.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T00:39:36.159 INFO:teuthology.orchestra.run.smithi078.stdout:Group : Unspecified 2024-09-11T00:39:36.159 INFO:teuthology.orchestra.run.smithi078.stdout:Size : 8274670 2024-09-11T00:39:36.159 INFO:teuthology.orchestra.run.smithi078.stdout:License : GPLv3+ 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:Packager : builder@centos.org 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:Vendor : CentOS 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:Summary : Support tools for GRUB. 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:Description : 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T00:39:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:hardware devices. 2024-09-11T00:39:36.161 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:39:36.161 INFO:teuthology.orchestra.run.smithi078.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T00:39:36.161 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T00:39:36.162 INFO:teuthology.task.kernel:Updating grub on smithi078 to boot 5.14.0-505.el9.x86_64 2024-09-11T00:39:36.162 DEBUG:teuthology.orchestra.run.smithi078:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T00:39:36.865 INFO:teuthology.orchestra.run.smithi078.stderr:Generating grub configuration file ... 2024-09-11T00:39:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T00:39:38.200 INFO:teuthology.orchestra.run.smithi078.stderr:done 2024-09-11T00:39:38.202 DEBUG:teuthology.orchestra.run.smithi078:> mktemp 2024-09-11T00:39:38.217 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/tmp.8b0mglSkjq 2024-09-11T00:39:38.217 DEBUG:teuthology.orchestra.run.smithi078:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8b0mglSkjq 2024-09-11T00:39:38.286 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0666 /tmp/tmp.8b0mglSkjq 2024-09-11T00:39:38.423 DEBUG:teuthology.orchestra.remote:smithi078:/tmp/tmp.8b0mglSkjq is 6KB 2024-09-11T00:39:38.473 DEBUG:teuthology.orchestra.run.smithi078:> rm -fr /tmp/tmp.8b0mglSkjq 2024-09-11T00:39:38.488 DEBUG:teuthology.orchestra.run.smithi078:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T00:39:38.550 INFO:teuthology.orchestra.run.smithi078.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T00:39:38.550 INFO:teuthology.orchestra.run.smithi078.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T00:39:38.550 INFO:teuthology.orchestra.run.smithi078.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T00:39:38.550 INFO:teuthology.orchestra.run.smithi078.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T00:39:38.552 DEBUG:teuthology.orchestra.run.smithi078:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T00:39:38.697 DEBUG:teuthology.orchestra.run.smithi078:> sudo shutdown -r now 2024-09-11T00:40:08.700 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-11T00:40:08.701 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T00:40:08.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:40:27.201 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-09-11T00:40:36.203 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T00:40:36.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:40:39.293 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-09-11T00:40:51.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T00:40:51.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:41:09.697 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-09-11T00:41:24.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T00:41:24.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:41:25.008 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-09-11T00:41:25.428 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T00:41:25.428 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-11T00:41:25.429 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-09-11T00:41:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-505.el9.x86_64 2024-09-11T00:41:25.443 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T00:41:25.443 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T00:41:25.444 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T00:41:26.444 DEBUG:teuthology.parallel:result is None 2024-09-11T00:41:26.444 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T00:41:26.451 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T00:41:26.452 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T00:41:26.454 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T00:41:26.457 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T00:41:26.476 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T00:41:26.543 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T00:41:26.618 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T00:41:26.618 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T00:41:26.621 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T00:41:26.624 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T00:41:26.728 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T00:41:26.735 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T00:41:26.735 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T00:41:26.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:41:26.753 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T00:41:26.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:41:26.771 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T00:41:26.773 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T00:41:26.795 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T00:41:26.848 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T00:41:26.852 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T00:41:26.866 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T00:41:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T00:41:26.891 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T00:41:26.900 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T00:41:26.901 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T00:41:26.908 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T00:41:26.909 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T00:41:26.911 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T00:41:26.913 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T00:41:26.940 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T00:41:26.950 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T00:41:26.950 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T00:41:26.976 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T00:41:26.980 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T00:41:26.997 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T00:41:27.079 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T00:41:27.170 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T00:41:27.218 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T00:41:27.252 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:41:27.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T00:41:27.322 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T00:41:27.369 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T00:41:27.424 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T00:41:27.517 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T00:41:27.554 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T00:41:27.554 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T00:41:27.623 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T00:41:27.673 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T00:41:27.725 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T00:41:27.815 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T00:41:27.853 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-11T00:41:27.853 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T00:41:27.921 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-09-11T00:41:27.923 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-09-11T00:41:27.926 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-09-11T00:41:27.962 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T00:41:28.003 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T00:41:28.017 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T00:41:28.523 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T00:41:28.530 INFO:teuthology.task.internal:Starting timer... 2024-09-11T00:41:28.530 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T00:41:28.562 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T00:41:28.609 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-11T00:41:28.609 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-09-11T00:41:28.839 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-09-11T00:41:28.841 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2024-09-11T00:41:28.908 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2024-09-11T00:41:28.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-09-11T00:41:28.984 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-09-11T00:41:28.986 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T00:41:28.986 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-09-11T00:41:29.007 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-09-11T00:41:29.008 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2024-09-11T00:41:29.032 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2024-09-11T00:41:29.033 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-09-11T00:41:29.058 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-09-11T00:41:29.058 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi078.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi110.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi112.front.sepia.ceph.com': 'permissive'} 2024-09-11T00:41:29.058 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T00:41:29.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:41:29.085 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T00:41:29.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:41:29.114 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T00:41:29.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:41:29.142 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T00:41:29.142 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-09-11T00:41:29.231 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2024-09-11T00:41:29.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-09-11T00:41:29.294 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T00:41:29.304 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T00:41:29.307 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T00:41:29.424 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T00:41:29.435 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T00:41:29.436 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi078.front.sepia.ceph.com,smithi110.front.sepia.ceph.com,smithi112.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T00:45:28.571 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi110.front.sepia.ceph.com'), Remote(name='ubuntu@smithi112.front.sepia.ceph.com')] 2024-09-11T00:45:28.572 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T00:45:28.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:45:28.657 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-09-11T00:45:28.744 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T00:45:28.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T00:45:28.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:45:28.849 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-11T00:45:28.928 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T00:45:28.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-11T00:45:28.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:45:29.010 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-09-11T00:45:29.093 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-11T00:45:29.094 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T00:45:29.108 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T00:45:29.108 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T00:45:29.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T00:45:29.111 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T00:45:29.111 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T00:45:29.114 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T00:45:29.114 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T00:45:29.141 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T00:45:29.142 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T00:45:29.156 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T00:45:29.157 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T00:45:29.164 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T00:45:29.179 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2024-09-11T00:45:29.180 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T00:45:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-09-11T00:45:29.190 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2024-09-11T00:45:29.193 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-09-11T00:45:29.204 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T00:45:29.204 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-09-11T00:45:29.208 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T00:45:29.216 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-09-11T00:45:29.218 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T00:45:29.222 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T00:45:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T00:45:29.245 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T00:45:29.277 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-09-11T00:45:29.280 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T00:45:29.280 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-09-11T00:45:29.280 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:45:29.280 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:45:29.280 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:45:29.281 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:45:29.281 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-11T00:45:29.285 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T00:45:29.285 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-11T00:45:29.285 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:45:29.285 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:45:29.285 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:45:29.285 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:45:29.298 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-09-11T00:45:29.302 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T00:45:29.303 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-09-11T00:45:29.303 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:45:29.303 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:45:29.303 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:45:29.303 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:45:29.303 INFO:teuthology.run_tasks:Running task install... 2024-09-11T00:45:29.311 DEBUG:teuthology.task.install:project ceph 2024-09-11T00:45:29.311 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-11T00:45:29.311 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-11T00:45:29.311 INFO:teuthology.task.install:Using flavor: default 2024-09-11T00:45:29.316 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T00:45:29.316 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-11T00:45:29.317 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-11T00:45:29.317 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:45:29.319 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-11T00:45:29.320 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:45:29.321 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-11T00:45:29.321 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:45:29.469 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-11T00:45:29.470 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-11T00:45:29.484 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-11T00:45:29.484 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-11T00:45:29.485 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-11T00:45:29.485 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-11T00:45:29.592 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T00:45:29.592 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:45:29.592 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T00:45:29.604 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T00:45:29.604 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-11T00:45:29.604 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T00:45:29.612 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T00:45:29.612 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T00:45:29.612 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T00:45:29.618 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T00:45:29.619 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T00:45:29.630 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T00:45:29.631 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T00:45:29.639 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T00:45:29.639 DEBUG:teuthology.orchestra.run.smithi110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T00:45:29.684 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T00:45:29.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T00:45:29.704 DEBUG:teuthology.orchestra.run.smithi110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T00:45:29.759 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T00:45:29.771 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T00:45:29.779 DEBUG:teuthology.orchestra.run.smithi110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T00:45:29.823 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2024-09-11T00:45:29.825 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2024-09-11T00:45:29.836 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-09-11T00:45:29.837 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-09-11T00:45:29.843 INFO:teuthology.orchestra.run.smithi110.stdout:check_obsoletes = 1 2024-09-11T00:45:29.845 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum clean all 2024-09-11T00:45:30.059 INFO:teuthology.orchestra.run.smithi078.stdout:45 files removed 2024-09-11T00:45:30.060 INFO:teuthology.orchestra.run.smithi112.stdout:45 files removed 2024-09-11T00:45:30.069 INFO:teuthology.orchestra.run.smithi110.stdout:45 files removed 2024-09-11T00:45:30.086 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T00:45:30.087 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T00:45:30.096 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T00:45:30.628 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 318 kB/s | 84 kB 00:00 2024-09-11T00:45:30.632 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 323 kB/s | 84 kB 00:00 2024-09-11T00:45:30.705 INFO:teuthology.orchestra.run.smithi110.stdout:ceph packages for x86_64 252 kB/s | 84 kB 00:00 2024-09-11T00:45:30.917 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-11T00:45:30.921 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-11T00:45:30.989 INFO:teuthology.orchestra.run.smithi110.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-11T00:45:31.184 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-11T00:45:31.196 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-11T00:45:31.265 INFO:teuthology.orchestra.run.smithi110.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-11T00:45:32.015 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-11T00:45:33.067 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.2 MB 00:01 2024-09-11T00:45:33.204 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.2 MB 00:01 2024-09-11T00:45:34.868 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-11T00:45:35.617 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 8.1 MB/s | 20 MB 00:02 2024-09-11T00:45:39.006 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 4.3 MB/s | 20 MB 00:04 2024-09-11T00:45:40.503 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-11T00:45:41.903 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 98 kB/s | 19 kB 00:00 2024-09-11T00:45:42.051 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-11T00:45:43.183 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-11T00:45:43.390 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-11T00:45:43.794 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 19 kB 00:00 2024-09-11T00:45:44.773 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-11T00:45:45.115 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-11T00:45:46.345 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-11T00:45:48.630 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-11T00:45:50.220 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-11T00:45:51.007 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T00:45:51.008 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T00:45:51.019 INFO:teuthology.orchestra.run.smithi078.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T00:45:51.020 INFO:teuthology.orchestra.run.smithi078.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T00:45:51.021 INFO:teuthology.orchestra.run.smithi078.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T00:45:51.022 INFO:teuthology.orchestra.run.smithi078.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T00:45:51.024 INFO:teuthology.orchestra.run.smithi078.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T00:45:51.026 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T00:45:51.027 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T00:45:51.091 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-09-11T00:45:51.101 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-09-11T00:45:51.101 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-09-11T00:45:51.101 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-09-11T00:45:51.101 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-09-11T00:45:51.101 INFO:teuthology.orchestra.run.smithi078.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T00:45:51.101 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-11T00:45:51.101 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T00:45:51.101 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-11T00:45:51.101 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-11T00:45:51.102 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T00:45:51.103 INFO:teuthology.orchestra.run.smithi078.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T00:45:51.103 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T00:45:51.103 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-11T00:45:51.103 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-11T00:45:51.103 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-11T00:45:51.103 INFO:teuthology.orchestra.run.smithi078.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T00:45:51.103 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T00:45:51.103 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T00:45:51.103 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T00:45:51.104 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T00:45:51.104 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T00:45:51.104 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-11T00:45:51.104 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-11T00:45:51.104 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-11T00:45:51.104 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-11T00:45:51.105 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-11T00:45:51.105 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T00:45:51.105 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-11T00:45:51.105 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T00:45:51.105 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2024-09-11T00:45:51.105 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T00:45:51.105 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T00:45:51.105 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T00:45:51.105 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T00:45:51.105 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-11T00:45:51.105 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T00:45:51.105 INFO:teuthology.orchestra.run.smithi078.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-11T00:45:51.106 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T00:45:51.107 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T00:45:51.108 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T00:45:51.109 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T00:45:51.110 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T00:45:51.110 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T00:45:51.110 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T00:45:51.110 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T00:45:51.110 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T00:45:51.110 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T00:45:51.110 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T00:45:51.110 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T00:45:51.110 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T00:45:51.110 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T00:45:51.110 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T00:45:51.111 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T00:45:51.112 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T00:45:51.112 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T00:45:51.112 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T00:45:51.112 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T00:45:51.112 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T00:45:51.112 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T00:45:51.112 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T00:45:51.112 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T00:45:51.112 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T00:45:51.112 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T00:45:51.112 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T00:45:51.112 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T00:45:51.113 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout:Installing weak dependencies: 2024-09-11T00:45:51.114 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T00:45:51.115 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:45:51.115 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-09-11T00:45:51.115 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-09-11T00:45:51.115 INFO:teuthology.orchestra.run.smithi078.stdout:Install 149 Packages 2024-09-11T00:45:51.115 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 8 Packages 2024-09-11T00:45:51.115 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:45:51.115 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 240 M 2024-09-11T00:45:51.115 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-09-11T00:45:51.465 INFO:teuthology.orchestra.run.smithi078.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 50 kB/s | 6.6 kB 00:00 2024-09-11T00:45:51.657 INFO:teuthology.orchestra.run.smithi078.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 4.6 MB/s | 900 kB 00:00 2024-09-11T00:45:51.740 INFO:teuthology.orchestra.run.smithi078.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.7 MB/s | 145 kB 00:00 2024-09-11T00:45:51.759 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-11T00:45:51.816 INFO:teuthology.orchestra.run.smithi078.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-11T00:45:51.899 INFO:teuthology.orchestra.run.smithi078.stdout:(5/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 18 MB/s | 1.5 MB 00:00 2024-09-11T00:45:51.991 INFO:teuthology.orchestra.run.smithi078.stdout:(6/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 9.5 MB/s | 2.4 MB 00:00 2024-09-11T00:45:52.217 INFO:teuthology.orchestra.run.smithi078.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 15 MB/s | 4.8 MB 00:00 2024-09-11T00:45:52.509 INFO:teuthology.orchestra.run.smithi078.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 18 MB/s | 22 MB 00:01 2024-09-11T00:45:52.513 INFO:teuthology.orchestra.run.smithi110.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T00:45:52.514 INFO:teuthology.orchestra.run.smithi110.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T00:45:52.525 INFO:teuthology.orchestra.run.smithi110.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T00:45:52.526 INFO:teuthology.orchestra.run.smithi110.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T00:45:52.527 INFO:teuthology.orchestra.run.smithi110.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T00:45:52.528 INFO:teuthology.orchestra.run.smithi110.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T00:45:52.529 INFO:teuthology.orchestra.run.smithi110.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T00:45:52.532 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T00:45:52.532 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T00:45:52.559 INFO:teuthology.orchestra.run.smithi078.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 506 kB/s | 25 kB 00:00 2024-09-11T00:45:52.594 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-11T00:45:52.608 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-09-11T00:45:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-11T00:45:52.608 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-09-11T00:45:52.608 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-11T00:45:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T00:45:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-11T00:45:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T00:45:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-11T00:45:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-11T00:45:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-11T00:45:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-11T00:45:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-11T00:45:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-11T00:45:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-11T00:45:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T00:45:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrading: 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T00:45:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-11T00:45:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-11T00:45:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-11T00:45:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-11T00:45:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-11T00:45:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-11T00:45:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-11T00:45:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T00:45:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T00:45:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T00:45:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T00:45:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T00:45:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T00:45:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T00:45:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T00:45:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-11T00:45:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T00:45:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-11T00:45:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T00:45:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T00:45:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T00:45:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T00:45:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-11T00:45:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T00:45:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T00:45:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T00:45:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T00:45:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T00:45:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T00:45:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T00:45:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T00:45:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T00:45:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T00:45:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T00:45:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T00:45:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T00:45:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T00:45:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T00:45:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T00:45:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T00:45:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T00:45:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T00:45:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T00:45:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T00:45:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T00:45:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T00:45:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T00:45:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T00:45:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T00:45:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T00:45:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T00:45:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T00:45:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T00:45:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T00:45:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T00:45:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T00:45:52.622 INFO:teuthology.orchestra.run.smithi110.stdout:Installing weak dependencies: 2024-09-11T00:45:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T00:45:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:45:52.622 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-11T00:45:52.622 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-09-11T00:45:52.622 INFO:teuthology.orchestra.run.smithi110.stdout:Install 149 Packages 2024-09-11T00:45:52.622 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrade 8 Packages 2024-09-11T00:45:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:45:52.623 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 240 M 2024-09-11T00:45:52.623 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-11T00:45:53.059 INFO:teuthology.orchestra.run.smithi078.stdout:(10/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 16 MB/s | 17 MB 00:01 2024-09-11T00:45:53.109 INFO:teuthology.orchestra.run.smithi078.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 4.4 MB/s | 225 kB 00:00 2024-09-11T00:45:53.185 INFO:teuthology.orchestra.run.smithi078.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 361 kB/s | 27 kB 00:00 2024-09-11T00:45:53.251 INFO:teuthology.orchestra.run.smithi078.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 478 kB/s | 32 kB 00:00 2024-09-11T00:45:53.335 INFO:teuthology.orchestra.run.smithi078.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 8.9 MB/s | 755 kB 00:00 2024-09-11T00:45:53.339 INFO:teuthology.orchestra.run.smithi110.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-09-11T00:45:53.522 INFO:teuthology.orchestra.run.smithi078.stdout:(15/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 8.4 MB/s | 11 MB 00:01 2024-09-11T00:45:53.553 INFO:teuthology.orchestra.run.smithi078.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 744 kB/s | 162 kB 00:00 2024-09-11T00:45:53.581 INFO:teuthology.orchestra.run.smithi110.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 3.6 MB/s | 900 kB 00:00 2024-09-11T00:45:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 1.7 MB/s | 127 kB 00:00 2024-09-11T00:45:53.637 INFO:teuthology.orchestra.run.smithi078.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 5.9 MB/s | 507 kB 00:00 2024-09-11T00:45:53.728 INFO:teuthology.orchestra.run.smithi078.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 493 kB/s | 45 kB 00:00 2024-09-11T00:45:53.804 INFO:teuthology.orchestra.run.smithi078.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.9 MB/s | 143 kB 00:00 2024-09-11T00:45:53.879 INFO:teuthology.orchestra.run.smithi078.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 2.1 MB/s | 161 kB 00:00 2024-09-11T00:45:53.914 INFO:teuthology.orchestra.run.smithi110.stdout:(3/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 7.4 MB/s | 5.5 MB 00:00 2024-09-11T00:45:53.954 INFO:teuthology.orchestra.run.smithi078.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 4.2 MB/s | 321 kB 00:00 2024-09-11T00:45:54.037 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T00:45:54.038 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T00:45:54.046 INFO:teuthology.orchestra.run.smithi078.stdout:(23/157): python3-rbd-19.3.0-4854.g8f219c47.el9 3.3 MB/s | 306 kB 00:00 2024-09-11T00:45:54.051 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T00:45:54.052 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T00:45:54.053 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T00:45:54.054 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T00:45:54.056 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T00:45:54.059 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T00:45:54.059 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T00:45:54.121 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-11T00:45:54.130 INFO:teuthology.orchestra.run.smithi078.stdout:(24/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.2 MB/s | 99 kB 00:00 2024-09-11T00:45:54.138 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-11T00:45:54.138 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-11T00:45:54.138 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-11T00:45:54.138 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-11T00:45:54.138 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T00:45:54.139 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-11T00:45:54.139 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T00:45:54.139 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-11T00:45:54.139 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-11T00:45:54.139 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-11T00:45:54.139 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-11T00:45:54.139 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-11T00:45:54.139 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-11T00:45:54.140 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-11T00:45:54.140 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-11T00:45:54.140 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-11T00:45:54.140 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-11T00:45:54.140 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-11T00:45:54.140 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-11T00:45:54.140 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-11T00:45:54.140 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-11T00:45:54.140 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T00:45:54.140 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T00:45:54.140 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T00:45:54.141 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-11T00:45:54.141 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-11T00:45:54.141 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-11T00:45:54.141 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T00:45:54.141 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T00:45:54.141 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T00:45:54.141 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T00:45:54.141 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T00:45:54.141 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T00:45:54.141 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-11T00:45:54.141 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-11T00:45:54.142 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-11T00:45:54.142 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-11T00:45:54.142 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-11T00:45:54.142 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T00:45:54.142 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-11T00:45:54.142 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T00:45:54.142 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-09-11T00:45:54.142 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T00:45:54.142 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-11T00:45:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T00:45:54.144 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-11T00:45:54.145 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T00:45:54.145 INFO:teuthology.orchestra.run.smithi112.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-11T00:45:54.145 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T00:45:54.145 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T00:45:54.145 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T00:45:54.145 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T00:45:54.145 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-11T00:45:54.145 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T00:45:54.145 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T00:45:54.145 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T00:45:54.145 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T00:45:54.145 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T00:45:54.146 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T00:45:54.146 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T00:45:54.146 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T00:45:54.146 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T00:45:54.146 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T00:45:54.146 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T00:45:54.146 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T00:45:54.146 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T00:45:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T00:45:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T00:45:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T00:45:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T00:45:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T00:45:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T00:45:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-11T00:45:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-11T00:45:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T00:45:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T00:45:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T00:45:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T00:45:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T00:45:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T00:45:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T00:45:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T00:45:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T00:45:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T00:45:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T00:45:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T00:45:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T00:45:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T00:45:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T00:45:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T00:45:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T00:45:54.151 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T00:45:54.152 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T00:45:54.152 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T00:45:54.152 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T00:45:54.152 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T00:45:54.152 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T00:45:54.152 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T00:45:54.152 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T00:45:54.152 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T00:45:54.152 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T00:45:54.152 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-09-11T00:45:54.152 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T00:45:54.152 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:45:54.153 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-11T00:45:54.153 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-11T00:45:54.153 INFO:teuthology.orchestra.run.smithi112.stdout:Install 149 Packages 2024-09-11T00:45:54.153 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 8 Packages 2024-09-11T00:45:54.153 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:45:54.153 INFO:teuthology.orchestra.run.smithi110.stdout:(4/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 10 MB/s | 2.4 MB 00:00 2024-09-11T00:45:54.154 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 240 M 2024-09-11T00:45:54.154 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-11T00:45:54.173 INFO:teuthology.orchestra.run.smithi110.stdout:(5/157): ceph-immutable-object-cache-19.3.0-485 245 kB/s | 145 kB 00:00 2024-09-11T00:45:54.205 INFO:teuthology.orchestra.run.smithi078.stdout:(25/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.1 MB/s | 85 kB 00:00 2024-09-11T00:45:54.298 INFO:teuthology.orchestra.run.smithi110.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 10 MB/s | 1.5 MB 00:00 2024-09-11T00:45:54.481 INFO:teuthology.orchestra.run.smithi078.stdout:(26/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 12 MB/s | 3.2 MB 00:00 2024-09-11T00:45:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.9 MB/s | 171 kB 00:00 2024-09-11T00:45:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 687 kB/s | 34 kB 00:00 2024-09-11T00:45:54.657 INFO:teuthology.orchestra.run.smithi078.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 2.3 MB/s | 154 kB 00:00 2024-09-11T00:45:54.657 INFO:teuthology.orchestra.run.smithi110.stdout:(7/157): ceph-common-19.3.0-4854.g8f219c47.el9. 15 MB/s | 22 MB 00:01 2024-09-11T00:45:54.898 INFO:teuthology.orchestra.run.smithi078.stdout:(30/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 4.2 MB/s | 5.5 MB 00:01 2024-09-11T00:45:55.085 INFO:teuthology.orchestra.run.smithi112.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-09-11T00:45:55.233 INFO:teuthology.orchestra.run.smithi110.stdout:(8/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 4.5 MB/s | 4.8 MB 00:01 2024-09-11T00:45:55.291 INFO:teuthology.orchestra.run.smithi110.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 433 kB/s | 25 kB 00:00 2024-09-11T00:45:55.369 INFO:teuthology.orchestra.run.smithi112.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 3.1 MB/s | 900 kB 00:00 2024-09-11T00:45:55.424 INFO:teuthology.orchestra.run.smithi078.stdout:(31/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 18 MB/s | 50 MB 00:02 2024-09-11T00:45:55.452 INFO:teuthology.orchestra.run.smithi112.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.7 MB/s | 145 kB 00:00 2024-09-11T00:45:55.516 INFO:teuthology.orchestra.run.smithi078.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 3.1 MB/s | 291 kB 00:00 2024-09-11T00:45:55.583 INFO:teuthology.orchestra.run.smithi110.stdout:(10/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 12 MB/s | 11 MB 00:00 2024-09-11T00:45:55.616 INFO:teuthology.orchestra.run.smithi078.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 494 kB/s | 49 kB 00:00 2024-09-11T00:45:55.683 INFO:teuthology.orchestra.run.smithi110.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 2.2 MB/s | 225 kB 00:00 2024-09-11T00:45:55.719 INFO:teuthology.orchestra.run.smithi112.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 5.9 MB/s | 5.5 MB 00:00 2024-09-11T00:45:55.733 INFO:teuthology.orchestra.run.smithi110.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 542 kB/s | 27 kB 00:00 2024-09-11T00:45:55.808 INFO:teuthology.orchestra.run.smithi078.stdout:(34/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 12 MB/s | 14 MB 00:01 2024-09-11T00:45:55.819 INFO:teuthology.orchestra.run.smithi112.stdout:(5/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 15 MB/s | 1.5 MB 00:00 2024-09-11T00:45:55.833 INFO:teuthology.orchestra.run.smithi078.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 76 kB/s | 16 kB 00:00 2024-09-11T00:45:55.858 INFO:teuthology.orchestra.run.smithi078.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 5.6 MB/s | 285 kB 00:00 2024-09-11T00:45:55.900 INFO:teuthology.orchestra.run.smithi110.stdout:(13/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 11 MB/s | 17 MB 00:01 2024-09-11T00:45:55.925 INFO:teuthology.orchestra.run.smithi078.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 274 kB/s | 18 kB 00:00 2024-09-11T00:45:55.926 INFO:teuthology.orchestra.run.smithi110.stdout:(14/157): libcephfs-devel-19.3.0-4854.g8f219c47 165 kB/s | 32 kB 00:00 2024-09-11T00:45:55.959 INFO:teuthology.orchestra.run.smithi078.stdout:(38/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 6.1 MB/s | 783 kB 00:00 2024-09-11T00:45:56.001 INFO:teuthology.orchestra.run.smithi110.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.1 MB/s | 162 kB 00:00 2024-09-11T00:45:56.026 INFO:teuthology.orchestra.run.smithi110.stdout:(16/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 5.9 MB/s | 755 kB 00:00 2024-09-11T00:45:56.060 INFO:teuthology.orchestra.run.smithi110.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 2.1 MB/s | 127 kB 00:00 2024-09-11T00:45:56.061 INFO:teuthology.orchestra.run.smithi112.stdout:(6/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 3.9 MB/s | 2.4 MB 00:00 2024-09-11T00:45:56.085 INFO:teuthology.orchestra.run.smithi110.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 8.5 MB/s | 507 kB 00:00 2024-09-11T00:45:56.152 INFO:teuthology.orchestra.run.smithi110.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 680 kB/s | 45 kB 00:00 2024-09-11T00:45:56.186 INFO:teuthology.orchestra.run.smithi112.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 13 MB/s | 4.8 MB 00:00 2024-09-11T00:45:56.211 INFO:teuthology.orchestra.run.smithi110.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 2.4 MB/s | 143 kB 00:00 2024-09-11T00:45:56.294 INFO:teuthology.orchestra.run.smithi110.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 1.9 MB/s | 161 kB 00:00 2024-09-11T00:45:56.378 INFO:teuthology.orchestra.run.smithi110.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 3.8 MB/s | 321 kB 00:00 2024-09-11T00:45:56.470 INFO:teuthology.orchestra.run.smithi110.stdout:(23/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 13 MB/s | 5.5 MB 00:00 2024-09-11T00:45:56.492 INFO:teuthology.orchestra.run.smithi078.stdout:(39/157): ceph-mgr-diskprediction-local-19.3.0- 4.6 MB/s | 7.4 MB 00:01 2024-09-11T00:45:56.495 INFO:teuthology.orchestra.run.smithi110.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 2.6 MB/s | 306 kB 00:00 2024-09-11T00:45:56.537 INFO:teuthology.orchestra.run.smithi110.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.5 MB/s | 99 kB 00:00 2024-09-11T00:45:56.563 INFO:teuthology.orchestra.run.smithi110.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.2 MB/s | 85 kB 00:00 2024-09-11T00:45:56.593 INFO:teuthology.orchestra.run.smithi078.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 120 kB/s | 80 kB 00:00 2024-09-11T00:45:56.646 INFO:teuthology.orchestra.run.smithi110.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.0 MB/s | 171 kB 00:00 2024-09-11T00:45:56.730 INFO:teuthology.orchestra.run.smithi110.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 410 kB/s | 34 kB 00:00 2024-09-11T00:45:56.838 INFO:teuthology.orchestra.run.smithi110.stdout:(29/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 11 MB/s | 3.2 MB 00:00 2024-09-11T00:45:56.864 INFO:teuthology.orchestra.run.smithi110.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 1.1 MB/s | 154 kB 00:00 2024-09-11T00:45:56.960 INFO:teuthology.orchestra.run.smithi078.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 801 kB/s | 801 kB 00:01 2024-09-11T00:45:57.070 INFO:teuthology.orchestra.run.smithi112.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 9.5 MB/s | 22 MB 00:02 2024-09-11T00:45:57.195 INFO:teuthology.orchestra.run.smithi112.stdout:(9/157): ceph-radosgw-19.3.0-4854.g8f219c47.el9 11 MB/s | 11 MB 00:01 2024-09-11T00:45:57.220 INFO:teuthology.orchestra.run.smithi112.stdout:(10/157): ceph-selinux-19.3.0-4854.g8f219c47.el 168 kB/s | 25 kB 00:00 2024-09-11T00:45:57.260 INFO:teuthology.orchestra.run.smithi078.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 249 kB/s | 191 kB 00:00 2024-09-11T00:45:57.304 INFO:teuthology.orchestra.run.smithi112.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 2.6 MB/s | 225 kB 00:00 2024-09-11T00:45:57.380 INFO:teuthology.orchestra.run.smithi112.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 360 kB/s | 27 kB 00:00 2024-09-11T00:45:57.410 INFO:teuthology.orchestra.run.smithi078.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 310 kB/s | 253 kB 00:00 2024-09-11T00:45:57.446 INFO:teuthology.orchestra.run.smithi112.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 478 kB/s | 32 kB 00:00 2024-09-11T00:45:57.448 INFO:teuthology.orchestra.run.smithi110.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 13 MB/s | 7.4 MB 00:00 2024-09-11T00:45:57.555 INFO:teuthology.orchestra.run.smithi112.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 6.8 MB/s | 755 kB 00:00 2024-09-11T00:45:57.614 INFO:teuthology.orchestra.run.smithi112.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.7 MB/s | 162 kB 00:00 2024-09-11T00:45:57.623 INFO:teuthology.orchestra.run.smithi110.stdout:(32/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 18 MB/s | 14 MB 00:00 2024-09-11T00:45:57.648 INFO:teuthology.orchestra.run.smithi110.stdout:(33/157): ceph-mgr-modules-core-19.3.0-4854.g8f 1.4 MB/s | 291 kB 00:00 2024-09-11T00:45:57.672 INFO:teuthology.orchestra.run.smithi112.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 2.1 MB/s | 127 kB 00:00 2024-09-11T00:45:57.673 INFO:teuthology.orchestra.run.smithi110.stdout:(34/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 984 kB/s | 49 kB 00:00 2024-09-11T00:45:57.699 INFO:teuthology.orchestra.run.smithi110.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 325 kB/s | 16 kB 00:00 2024-09-11T00:45:57.741 INFO:teuthology.orchestra.run.smithi110.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 4.2 MB/s | 285 kB 00:00 2024-09-11T00:45:57.748 INFO:teuthology.orchestra.run.smithi112.stdout:(17/157): libradosstriper1-19.3.0-4854.g8f219c4 6.6 MB/s | 507 kB 00:00 2024-09-11T00:45:57.783 INFO:teuthology.orchestra.run.smithi110.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 439 kB/s | 18 kB 00:00 2024-09-11T00:45:57.808 INFO:teuthology.orchestra.run.smithi110.stdout:(38/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 7.0 MB/s | 783 kB 00:00 2024-09-11T00:45:57.911 INFO:teuthology.orchestra.run.smithi078.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-11T00:45:57.916 INFO:teuthology.orchestra.run.smithi110.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 598 kB/s | 80 kB 00:00 2024-09-11T00:45:57.952 INFO:teuthology.orchestra.run.smithi078.stdout:(45/157): python3-ply-3.11-14.el9.noarch.rpm 154 kB/s | 106 kB 00:00 2024-09-11T00:45:58.000 INFO:teuthology.orchestra.run.smithi110.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-11T00:45:58.057 INFO:teuthology.orchestra.run.smithi112.stdout:(18/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 18 MB/s | 5.5 MB 00:00 2024-09-11T00:45:58.115 INFO:teuthology.orchestra.run.smithi112.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 777 kB/s | 45 kB 00:00 2024-09-11T00:45:58.117 INFO:teuthology.orchestra.run.smithi110.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-11T00:45:58.128 INFO:teuthology.orchestra.run.smithi078.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 189 kB/s | 135 kB 00:00 2024-09-11T00:45:58.151 INFO:teuthology.orchestra.run.smithi110.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-11T00:45:58.182 INFO:teuthology.orchestra.run.smithi112.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 2.1 MB/s | 143 kB 00:00 2024-09-11T00:45:58.201 INFO:teuthology.orchestra.run.smithi110.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-11T00:45:58.276 INFO:teuthology.orchestra.run.smithi110.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-11T00:45:58.327 INFO:teuthology.orchestra.run.smithi110.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-11T00:45:58.357 INFO:teuthology.orchestra.run.smithi112.stdout:(21/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 7.5 MB/s | 17 MB 00:02 2024-09-11T00:45:58.382 INFO:teuthology.orchestra.run.smithi112.stdout:(22/157): python3-cephfs-19.3.0-4854.g8f219c47. 805 kB/s | 161 kB 00:00 2024-09-11T00:45:58.410 INFO:teuthology.orchestra.run.smithi110.stdout:(46/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-11T00:45:58.424 INFO:teuthology.orchestra.run.smithi112.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 4.7 MB/s | 321 kB 00:00 2024-09-11T00:45:58.453 INFO:teuthology.orchestra.run.smithi078.stdout:(47/157): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-09-11T00:45:58.458 INFO:teuthology.orchestra.run.smithi112.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 4.0 MB/s | 306 kB 00:00 2024-09-11T00:45:58.483 INFO:teuthology.orchestra.run.smithi112.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.7 MB/s | 99 kB 00:00 2024-09-11T00:45:58.494 INFO:teuthology.orchestra.run.smithi110.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 3.3 MB/s | 1.2 MB 00:00 2024-09-11T00:45:58.509 INFO:teuthology.orchestra.run.smithi112.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-11T00:45:58.559 INFO:teuthology.orchestra.run.smithi112.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 3.4 MB/s | 171 kB 00:00 2024-09-11T00:45:58.618 INFO:teuthology.orchestra.run.smithi112.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 588 kB/s | 34 kB 00:00 2024-09-11T00:45:58.636 INFO:teuthology.orchestra.run.smithi110.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-11T00:45:58.637 INFO:teuthology.orchestra.run.smithi078.stdout:(48/157): python3-requests-2.25.1-8.el9.noarch. 172 kB/s | 125 kB 00:00 2024-09-11T00:45:58.676 INFO:teuthology.orchestra.run.smithi112.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 2.6 MB/s | 154 kB 00:00 2024-09-11T00:45:58.686 INFO:teuthology.orchestra.run.smithi110.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-11T00:45:58.712 INFO:teuthology.orchestra.run.smithi078.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-09-11T00:45:58.737 INFO:teuthology.orchestra.run.smithi110.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-11T00:45:58.754 INFO:teuthology.orchestra.run.smithi078.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 523 kB/s | 22 kB 00:00 2024-09-11T00:45:58.770 INFO:teuthology.orchestra.run.smithi110.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-09-11T00:45:58.785 INFO:teuthology.orchestra.run.smithi112.stdout:(30/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 11 MB/s | 3.2 MB 00:00 2024-09-11T00:45:58.787 INFO:teuthology.orchestra.run.smithi078.stdout:(51/157): boost-program-options-1.75.0-8.el9.x8 711 kB/s | 107 kB 00:00 2024-09-11T00:45:58.812 INFO:teuthology.orchestra.run.smithi110.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 786 kB/s | 33 kB 00:00 2024-09-11T00:45:58.813 INFO:teuthology.orchestra.run.smithi078.stdout:(52/157): python3-urllib3-1.26.5-6.el9.noarch.r 250 kB/s | 215 kB 00:00 2024-09-11T00:45:58.830 INFO:teuthology.orchestra.run.smithi078.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 779 kB/s | 33 kB 00:00 2024-09-11T00:45:58.854 INFO:teuthology.orchestra.run.smithi110.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-11T00:45:58.872 INFO:teuthology.orchestra.run.smithi078.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-11T00:45:58.888 INFO:teuthology.orchestra.run.smithi078.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-11T00:45:58.896 INFO:teuthology.orchestra.run.smithi110.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-09-11T00:45:58.922 INFO:teuthology.orchestra.run.smithi078.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-11T00:45:58.938 INFO:teuthology.orchestra.run.smithi110.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-09-11T00:45:58.964 INFO:teuthology.orchestra.run.smithi078.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.2 MB/s | 171 kB 00:00 2024-09-11T00:45:59.014 INFO:teuthology.orchestra.run.smithi078.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-11T00:45:59.022 INFO:teuthology.orchestra.run.smithi110.stdout:(56/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2024-09-11T00:45:59.040 INFO:teuthology.orchestra.run.smithi078.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-09-11T00:45:59.105 INFO:teuthology.orchestra.run.smithi110.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-11T00:45:59.156 INFO:teuthology.orchestra.run.smithi110.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-11T00:45:59.192 INFO:teuthology.orchestra.run.smithi078.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-09-11T00:45:59.240 INFO:teuthology.orchestra.run.smithi078.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 491 kB/s | 24 kB 00:00 2024-09-11T00:45:59.291 INFO:teuthology.orchestra.run.smithi078.stdout:(62/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-11T00:45:59.302 INFO:teuthology.orchestra.run.smithi112.stdout:(31/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 23 MB/s | 14 MB 00:00 2024-09-11T00:45:59.324 INFO:teuthology.orchestra.run.smithi078.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 2.0 MB/s | 581 kB 00:00 2024-09-11T00:45:59.331 INFO:teuthology.orchestra.run.smithi110.stdout:(59/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-09-11T00:45:59.369 INFO:teuthology.orchestra.run.smithi112.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 4.3 MB/s | 291 kB 00:00 2024-09-11T00:45:59.408 INFO:teuthology.orchestra.run.smithi078.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-11T00:45:59.427 INFO:teuthology.orchestra.run.smithi112.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 851 kB/s | 49 kB 00:00 2024-09-11T00:45:59.433 INFO:teuthology.orchestra.run.smithi078.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-11T00:45:59.458 INFO:teuthology.orchestra.run.smithi078.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-11T00:45:59.486 INFO:teuthology.orchestra.run.smithi112.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 282 kB/s | 16 kB 00:00 2024-09-11T00:45:59.490 INFO:teuthology.orchestra.run.smithi110.stdout:(60/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.4 MB/s | 3.0 MB 00:00 2024-09-11T00:45:59.515 INFO:teuthology.orchestra.run.smithi110.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-09-11T00:45:59.541 INFO:teuthology.orchestra.run.smithi110.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-11T00:45:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 4.2 MB/s | 285 kB 00:00 2024-09-11T00:45:59.566 INFO:teuthology.orchestra.run.smithi110.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-11T00:45:59.624 INFO:teuthology.orchestra.run.smithi110.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-11T00:45:59.649 INFO:teuthology.orchestra.run.smithi110.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-11T00:45:59.675 INFO:teuthology.orchestra.run.smithi110.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-11T00:45:59.678 INFO:teuthology.orchestra.run.smithi112.stdout:(36/157): ceph-mgr-diskprediction-local-19.3.0- 8.3 MB/s | 7.4 MB 00:00 2024-09-11T00:45:59.700 INFO:teuthology.orchestra.run.smithi078.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.4 MB/s | 3.0 MB 00:00 2024-09-11T00:45:59.720 INFO:teuthology.orchestra.run.smithi112.stdout:(37/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 4.6 MB/s | 783 kB 00:00 2024-09-11T00:45:59.745 INFO:teuthology.orchestra.run.smithi112.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 273 kB/s | 18 kB 00:00 2024-09-11T00:46:00.042 INFO:teuthology.orchestra.run.smithi078.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-09-11T00:46:00.196 INFO:teuthology.orchestra.run.smithi112.stdout:(39/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 17 MB/s | 50 MB 00:02 2024-09-11T00:46:00.251 INFO:teuthology.orchestra.run.smithi078.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 4.3 MB/s | 910 kB 00:00 2024-09-11T00:46:00.343 INFO:teuthology.orchestra.run.smithi078.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2024-09-11T00:46:00.368 INFO:teuthology.orchestra.run.smithi078.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-11T00:46:00.417 INFO:teuthology.orchestra.run.smithi110.stdout:(67/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-09-11T00:46:00.546 INFO:teuthology.orchestra.run.smithi110.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 5.5 MB/s | 4.9 MB 00:00 2024-09-11T00:46:00.568 INFO:teuthology.orchestra.run.smithi078.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-09-11T00:46:00.627 INFO:teuthology.orchestra.run.smithi078.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-11T00:46:00.696 INFO:teuthology.orchestra.run.smithi110.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 5.7 MB/s | 1.6 MB 00:00 2024-09-11T00:46:00.738 INFO:teuthology.orchestra.run.smithi110.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-11T00:46:00.762 INFO:teuthology.orchestra.run.smithi112.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 77 kB/s | 80 kB 00:01 2024-09-11T00:46:00.819 INFO:teuthology.orchestra.run.smithi110.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-11T00:46:00.961 INFO:teuthology.orchestra.run.smithi078.stdout:(74/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.2 MB/s | 4.9 MB 00:01 2024-09-11T00:46:00.979 INFO:teuthology.orchestra.run.smithi112.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 244 kB/s | 191 kB 00:00 2024-09-11T00:46:00.989 INFO:teuthology.orchestra.run.smithi110.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 5.9 MB/s | 1.0 MB 00:00 2024-09-11T00:46:01.011 INFO:teuthology.orchestra.run.smithi078.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-11T00:46:01.039 INFO:teuthology.orchestra.run.smithi110.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-11T00:46:01.136 INFO:teuthology.orchestra.run.smithi078.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-11T00:46:01.253 INFO:teuthology.orchestra.run.smithi078.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-11T00:46:01.312 INFO:teuthology.orchestra.run.smithi078.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-09-11T00:46:01.420 INFO:teuthology.orchestra.run.smithi078.stdout:(79/157): postgresql-server-13.16-1.el9.x86_64. 5.4 MB/s | 5.8 MB 00:01 2024-09-11T00:46:01.446 INFO:teuthology.orchestra.run.smithi078.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-09-11T00:46:01.479 INFO:teuthology.orchestra.run.smithi078.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-11T00:46:01.505 INFO:teuthology.orchestra.run.smithi078.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-09-11T00:46:01.613 INFO:teuthology.orchestra.run.smithi078.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-11T00:46:01.614 INFO:teuthology.orchestra.run.smithi110.stdout:(74/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 7.9 MB/s | 50 MB 00:06 2024-09-11T00:46:01.621 INFO:teuthology.orchestra.run.smithi112.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 427 kB/s | 801 kB 00:01 2024-09-11T00:46:01.688 INFO:teuthology.orchestra.run.smithi112.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 274 kB/s | 253 kB 00:00 2024-09-11T00:46:01.722 INFO:teuthology.orchestra.run.smithi078.stdout:(84/157): python3-babel-2.9.1-2.el9.noarch.rpm 5.4 MB/s | 6.0 MB 00:01 2024-09-11T00:46:01.747 INFO:teuthology.orchestra.run.smithi078.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-09-11T00:46:01.748 INFO:teuthology.orchestra.run.smithi110.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 5.7 MB/s | 5.8 MB 00:01 2024-09-11T00:46:01.773 INFO:teuthology.orchestra.run.smithi078.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-11T00:46:01.773 INFO:teuthology.orchestra.run.smithi110.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 387 kB/s | 61 kB 00:00 2024-09-11T00:46:01.789 INFO:teuthology.orchestra.run.smithi078.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-11T00:46:01.815 INFO:teuthology.orchestra.run.smithi110.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-11T00:46:01.856 INFO:teuthology.orchestra.run.smithi078.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-09-11T00:46:01.868 INFO:teuthology.orchestra.run.smithi110.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-11T00:46:01.906 INFO:teuthology.orchestra.run.smithi078.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-11T00:46:01.916 INFO:teuthology.orchestra.run.smithi110.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-09-11T00:46:01.941 INFO:teuthology.orchestra.run.smithi110.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-09-11T00:46:01.957 INFO:teuthology.orchestra.run.smithi078.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-11T00:46:01.974 INFO:teuthology.orchestra.run.smithi078.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-09-11T00:46:01.992 INFO:teuthology.orchestra.run.smithi110.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-09-11T00:46:01.999 INFO:teuthology.orchestra.run.smithi078.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-11T00:46:02.041 INFO:teuthology.orchestra.run.smithi078.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-11T00:46:02.075 INFO:teuthology.orchestra.run.smithi110.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-11T00:46:02.083 INFO:teuthology.orchestra.run.smithi078.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-11T00:46:02.142 INFO:teuthology.orchestra.run.smithi078.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-11T00:46:02.184 INFO:teuthology.orchestra.run.smithi110.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-11T00:46:02.200 INFO:teuthology.orchestra.run.smithi078.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-09-11T00:46:02.259 INFO:teuthology.orchestra.run.smithi078.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-09-11T00:46:02.259 INFO:teuthology.orchestra.run.smithi110.stdout:(84/157): python3-babel-2.9.1-2.el9.noarch.rpm 4.9 MB/s | 6.0 MB 00:01 2024-09-11T00:46:02.293 INFO:teuthology.orchestra.run.smithi110.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-11T00:46:02.313 INFO:teuthology.orchestra.run.smithi112.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 154 kB/s | 106 kB 00:00 2024-09-11T00:46:02.318 INFO:teuthology.orchestra.run.smithi110.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-11T00:46:02.351 INFO:teuthology.orchestra.run.smithi078.stdout:(98/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-09-11T00:46:02.360 INFO:teuthology.orchestra.run.smithi110.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 42 kB 00:00 2024-09-11T00:46:02.410 INFO:teuthology.orchestra.run.smithi110.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-09-11T00:46:02.417 INFO:teuthology.orchestra.run.smithi078.stdout:(99/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.4 MB/s | 5.1 MB 00:00 2024-09-11T00:46:02.443 INFO:teuthology.orchestra.run.smithi078.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 628 kB/s | 58 kB 00:00 2024-09-11T00:46:02.464 INFO:teuthology.orchestra.run.smithi112.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 174 kB/s | 135 kB 00:00 2024-09-11T00:46:02.468 INFO:teuthology.orchestra.run.smithi078.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-11T00:46:02.494 INFO:teuthology.orchestra.run.smithi110.stdout:(89/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-09-11T00:46:02.498 INFO:teuthology.orchestra.run.smithi112.stdout:(46/157): python3-cryptography-36.0.1-4.el9.x86 831 kB/s | 1.2 MB 00:01 2024-09-11T00:46:02.502 INFO:teuthology.orchestra.run.smithi078.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-11T00:46:02.519 INFO:teuthology.orchestra.run.smithi110.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 999 kB/s | 159 kB 00:00 2024-09-11T00:46:02.527 INFO:teuthology.orchestra.run.smithi078.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 760 kB/s | 45 kB 00:00 2024-09-11T00:46:02.544 INFO:teuthology.orchestra.run.smithi110.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-09-11T00:46:02.569 INFO:teuthology.orchestra.run.smithi110.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-11T00:46:02.595 INFO:teuthology.orchestra.run.smithi110.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 54 kB 00:00 2024-09-11T00:46:02.620 INFO:teuthology.orchestra.run.smithi110.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 826 kB/s | 42 kB 00:00 2024-09-11T00:46:02.636 INFO:teuthology.orchestra.run.smithi078.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-09-11T00:46:02.646 INFO:teuthology.orchestra.run.smithi110.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-09-11T00:46:02.671 INFO:teuthology.orchestra.run.smithi110.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-11T00:46:02.688 INFO:teuthology.orchestra.run.smithi078.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-11T00:46:02.696 INFO:teuthology.orchestra.run.smithi110.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-11T00:46:02.711 INFO:teuthology.orchestra.run.smithi078.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-11T00:46:02.721 INFO:teuthology.orchestra.run.smithi110.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-11T00:46:02.737 INFO:teuthology.orchestra.run.smithi078.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 776 kB/s | 19 kB 00:00 2024-09-11T00:46:02.765 INFO:teuthology.orchestra.run.smithi112.stdout:(47/157): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-09-11T00:46:02.772 INFO:teuthology.orchestra.run.smithi110.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-11T00:46:02.804 INFO:teuthology.orchestra.run.smithi078.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-09-11T00:46:02.822 INFO:teuthology.orchestra.run.smithi110.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-11T00:46:02.847 INFO:teuthology.orchestra.run.smithi110.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-11T00:46:02.854 INFO:teuthology.orchestra.run.smithi078.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-09-11T00:46:02.873 INFO:teuthology.orchestra.run.smithi110.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-11T00:46:02.879 INFO:teuthology.orchestra.run.smithi078.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-11T00:46:02.898 INFO:teuthology.orchestra.run.smithi110.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-11T00:46:02.946 INFO:teuthology.orchestra.run.smithi078.stdout:(111/157): libarrow-9.0.0-11.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-09-11T00:46:02.956 INFO:teuthology.orchestra.run.smithi110.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-09-11T00:46:02.971 INFO:teuthology.orchestra.run.smithi078.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 732 kB/s | 67 kB 00:00 2024-09-11T00:46:02.996 INFO:teuthology.orchestra.run.smithi078.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-11T00:46:03.006 INFO:teuthology.orchestra.run.smithi110.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-11T00:46:03.047 INFO:teuthology.orchestra.run.smithi078.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-11T00:46:03.049 INFO:teuthology.orchestra.run.smithi110.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 7.2 MB/s | 309 kB 00:00 2024-09-11T00:46:03.072 INFO:teuthology.orchestra.run.smithi078.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T00:46:03.073 INFO:teuthology.orchestra.run.smithi112.stdout:(48/157): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-11T00:46:03.074 INFO:teuthology.orchestra.run.smithi110.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-11T00:46:03.097 INFO:teuthology.orchestra.run.smithi078.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-11T00:46:03.123 INFO:teuthology.orchestra.run.smithi078.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-11T00:46:03.149 INFO:teuthology.orchestra.run.smithi112.stdout:(49/157): python3-requests-2.25.1-8.el9.noarch. 150 kB/s | 125 kB 00:00 2024-09-11T00:46:03.149 INFO:teuthology.orchestra.run.smithi110.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-09-11T00:46:03.156 INFO:teuthology.orchestra.run.smithi078.stdout:(118/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.4 MB/s | 839 kB 00:00 2024-09-11T00:46:03.174 INFO:teuthology.orchestra.run.smithi112.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-09-11T00:46:03.181 INFO:teuthology.orchestra.run.smithi078.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-11T00:46:03.192 INFO:teuthology.orchestra.run.smithi110.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 627 kB/s | 26 kB 00:00 2024-09-11T00:46:03.215 INFO:teuthology.orchestra.run.smithi078.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-11T00:46:03.241 INFO:teuthology.orchestra.run.smithi078.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-11T00:46:03.250 INFO:teuthology.orchestra.run.smithi110.stdout:(110/157): libarrow-9.0.0-11.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-11T00:46:03.257 INFO:teuthology.orchestra.run.smithi112.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 201 kB/s | 22 kB 00:00 2024-09-11T00:46:03.266 INFO:teuthology.orchestra.run.smithi078.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-11T00:46:03.275 INFO:teuthology.orchestra.run.smithi110.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-09-11T00:46:03.301 INFO:teuthology.orchestra.run.smithi110.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-11T00:46:03.307 INFO:teuthology.orchestra.run.smithi112.stdout:(52/157): python3-urllib3-1.26.5-6.el9.noarch.r 255 kB/s | 215 kB 00:00 2024-09-11T00:46:03.308 INFO:teuthology.orchestra.run.smithi078.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-11T00:46:03.326 INFO:teuthology.orchestra.run.smithi110.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-11T00:46:03.333 INFO:teuthology.orchestra.run.smithi078.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 425 kB/s | 11 kB 00:00 2024-09-11T00:46:03.341 INFO:teuthology.orchestra.run.smithi112.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-09-11T00:46:03.358 INFO:teuthology.orchestra.run.smithi078.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-09-11T00:46:03.360 INFO:teuthology.orchestra.run.smithi110.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-11T00:46:03.366 INFO:teuthology.orchestra.run.smithi112.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 1.6 MB/s | 313 kB 00:00 2024-09-11T00:46:03.384 INFO:teuthology.orchestra.run.smithi078.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-11T00:46:03.385 INFO:teuthology.orchestra.run.smithi110.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-11T00:46:03.391 INFO:teuthology.orchestra.run.smithi112.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-11T00:46:03.409 INFO:teuthology.orchestra.run.smithi078.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 775 kB/s | 19 kB 00:00 2024-09-11T00:46:03.410 INFO:teuthology.orchestra.run.smithi110.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 866 kB/s | 43 kB 00:00 2024-09-11T00:46:03.416 INFO:teuthology.orchestra.run.smithi112.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-11T00:46:03.434 INFO:teuthology.orchestra.run.smithi078.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 796 kB/s | 20 kB 00:00 2024-09-11T00:46:03.435 INFO:teuthology.orchestra.run.smithi110.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-09-11T00:46:03.461 INFO:teuthology.orchestra.run.smithi110.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-11T00:46:03.485 INFO:teuthology.orchestra.run.smithi078.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-11T00:46:03.486 INFO:teuthology.orchestra.run.smithi110.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T00:46:03.504 INFO:teuthology.orchestra.run.smithi078.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-11T00:46:03.509 INFO:teuthology.orchestra.run.smithi112.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.4 MB/s | 171 kB 00:00 2024-09-11T00:46:03.511 INFO:teuthology.orchestra.run.smithi110.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-11T00:46:03.534 INFO:teuthology.orchestra.run.smithi112.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-11T00:46:03.535 INFO:teuthology.orchestra.run.smithi078.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-11T00:46:03.536 INFO:teuthology.orchestra.run.smithi110.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-11T00:46:03.553 INFO:teuthology.orchestra.run.smithi078.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-11T00:46:03.559 INFO:teuthology.orchestra.run.smithi112.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-09-11T00:46:03.570 INFO:teuthology.orchestra.run.smithi110.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-11T00:46:03.586 INFO:teuthology.orchestra.run.smithi078.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.7 MB/s | 1.0 MB 00:00 2024-09-11T00:46:03.603 INFO:teuthology.orchestra.run.smithi078.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 998 kB/s | 16 kB 00:00 2024-09-11T00:46:03.604 INFO:teuthology.orchestra.run.smithi110.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-11T00:46:03.610 INFO:teuthology.orchestra.run.smithi112.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-11T00:46:03.629 INFO:teuthology.orchestra.run.smithi110.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-11T00:46:03.637 INFO:teuthology.orchestra.run.smithi078.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-11T00:46:03.654 INFO:teuthology.orchestra.run.smithi110.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-11T00:46:03.677 INFO:teuthology.orchestra.run.smithi112.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 4.9 MB/s | 581 kB 00:00 2024-09-11T00:46:03.678 INFO:teuthology.orchestra.run.smithi078.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-11T00:46:03.680 INFO:teuthology.orchestra.run.smithi110.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-11T00:46:03.704 INFO:teuthology.orchestra.run.smithi078.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-09-11T00:46:03.708 INFO:teuthology.orchestra.run.smithi110.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 774 kB/s | 19 kB 00:00 2024-09-11T00:46:03.729 INFO:teuthology.orchestra.run.smithi078.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-11T00:46:03.730 INFO:teuthology.orchestra.run.smithi110.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-09-11T00:46:03.754 INFO:teuthology.orchestra.run.smithi078.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T00:46:03.773 INFO:teuthology.orchestra.run.smithi112.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 150 kB/s | 24 kB 00:00 2024-09-11T00:46:03.780 INFO:teuthology.orchestra.run.smithi078.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-11T00:46:03.781 INFO:teuthology.orchestra.run.smithi110.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-11T00:46:03.805 INFO:teuthology.orchestra.run.smithi078.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-11T00:46:03.806 INFO:teuthology.orchestra.run.smithi110.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T00:46:03.827 INFO:teuthology.orchestra.run.smithi112.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 316 kB/s | 47 kB 00:00 2024-09-11T00:46:03.830 INFO:teuthology.orchestra.run.smithi078.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-11T00:46:03.831 INFO:teuthology.orchestra.run.smithi110.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T00:46:03.856 INFO:teuthology.orchestra.run.smithi078.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-11T00:46:03.856 INFO:teuthology.orchestra.run.smithi110.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-11T00:46:03.881 INFO:teuthology.orchestra.run.smithi078.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-11T00:46:03.885 INFO:teuthology.orchestra.run.smithi112.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-11T00:46:03.891 INFO:teuthology.orchestra.run.smithi110.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-11T00:46:03.906 INFO:teuthology.orchestra.run.smithi078.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 801 kB/s | 20 kB 00:00 2024-09-11T00:46:03.916 INFO:teuthology.orchestra.run.smithi110.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-11T00:46:03.936 INFO:teuthology.orchestra.run.smithi112.stdout:(65/157): openblas-0.3.26-2.el9.x86_64.rpm 772 kB/s | 38 kB 00:00 2024-09-11T00:46:03.940 INFO:teuthology.orchestra.run.smithi078.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-11T00:46:03.941 INFO:teuthology.orchestra.run.smithi110.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-11T00:46:03.965 INFO:teuthology.orchestra.run.smithi078.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-11T00:46:03.966 INFO:teuthology.orchestra.run.smithi110.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-11T00:46:03.969 INFO:teuthology.orchestra.run.smithi112.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2024-09-11T00:46:03.983 INFO:teuthology.orchestra.run.smithi110.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 676 kB/s | 28 kB 00:00 2024-09-11T00:46:04.000 INFO:teuthology.orchestra.run.smithi110.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 913 kB/s | 31 kB 00:00 2024-09-11T00:46:04.016 INFO:teuthology.orchestra.run.smithi078.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-11T00:46:04.017 INFO:teuthology.orchestra.run.smithi110.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-11T00:46:04.043 INFO:teuthology.orchestra.run.smithi110.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-09-11T00:46:04.059 INFO:teuthology.orchestra.run.smithi110.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 855 kB/s | 36 kB 00:00 2024-09-11T00:46:04.085 INFO:teuthology.orchestra.run.smithi110.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-11T00:46:04.110 INFO:teuthology.orchestra.run.smithi110.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-11T00:46:04.136 INFO:teuthology.orchestra.run.smithi110.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-11T00:46:04.161 INFO:teuthology.orchestra.run.smithi110.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 804 kB/s | 20 kB 00:00 2024-09-11T00:46:04.186 INFO:teuthology.orchestra.run.smithi110.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-11T00:46:04.212 INFO:teuthology.orchestra.run.smithi110.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-11T00:46:04.258 INFO:teuthology.orchestra.run.smithi078.stdout:(149/157): librados2-19.3.0-4854.g8f219c47.el9. 12 MB/s | 3.5 MB 00:00 2024-09-11T00:46:04.262 INFO:teuthology.orchestra.run.smithi110.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-11T00:46:04.311 INFO:teuthology.orchestra.run.smithi112.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.0 MB/s | 3.0 MB 00:01 2024-09-11T00:46:04.441 INFO:teuthology.orchestra.run.smithi078.stdout:(150/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 7.6 MB/s | 3.2 MB 00:00 2024-09-11T00:46:04.545 INFO:teuthology.orchestra.run.smithi110.stdout:(149/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 11 MB/s | 3.2 MB 00:00 2024-09-11T00:46:04.596 INFO:teuthology.orchestra.run.smithi110.stdout:(150/157): librados2-19.3.0-4854.g8f219c47.el9. 9.0 MB/s | 3.5 MB 00:00 2024-09-11T00:46:04.863 INFO:teuthology.orchestra.run.smithi110.stdout:(151/157): NetworkManager-1.49.90-1.el9.x86_64. 7.3 MB/s | 2.3 MB 00:00 2024-09-11T00:46:05.038 INFO:teuthology.orchestra.run.smithi110.stdout:(152/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.5 MB/s | 16 MB 00:02 2024-09-11T00:46:05.063 INFO:teuthology.orchestra.run.smithi110.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 203 kB/s | 41 kB 00:00 2024-09-11T00:46:05.088 INFO:teuthology.orchestra.run.smithi110.stdout:(154/157): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-11T00:46:05.103 INFO:teuthology.orchestra.run.smithi112.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-09-11T00:46:05.183 INFO:teuthology.orchestra.run.smithi078.stdout:(151/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.1 MB/s | 16 MB 00:03 2024-09-11T00:46:05.213 INFO:teuthology.orchestra.run.smithi110.stdout:(155/157): NetworkManager-libnm-1.49.90-1.el9.x 2.9 MB/s | 1.8 MB 00:00 2024-09-11T00:46:05.306 INFO:teuthology.orchestra.run.smithi110.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 5.6 MB/s | 1.4 MB 00:00 2024-09-11T00:46:05.373 INFO:teuthology.orchestra.run.smithi110.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 8.4 MB/s | 2.4 MB 00:00 2024-09-11T00:46:05.377 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-11T00:46:05.377 INFO:teuthology.orchestra.run.smithi110.stdout:Total 19 MB/s | 240 MB 00:12 2024-09-11T00:46:05.604 INFO:teuthology.orchestra.run.smithi112.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.0 MB/s | 4.9 MB 00:01 2024-09-11T00:46:05.616 INFO:teuthology.orchestra.run.smithi078.stdout:(152/157): NetworkManager-1.49.90-1.el9.x86_64. 1.7 MB/s | 2.3 MB 00:01 2024-09-11T00:46:05.629 INFO:teuthology.orchestra.run.smithi112.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 1.7 MB/s | 910 kB 00:00 2024-09-11T00:46:05.687 INFO:teuthology.orchestra.run.smithi112.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-11T00:46:05.752 INFO:teuthology.orchestra.run.smithi078.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 72 kB/s | 41 kB 00:00 2024-09-11T00:46:05.762 INFO:teuthology.orchestra.run.smithi112.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-11T00:46:05.821 INFO:teuthology.orchestra.run.smithi112.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-11T00:46:05.996 INFO:teuthology.orchestra.run.smithi112.stdout:(74/157): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-09-11T00:46:06.047 INFO:teuthology.orchestra.run.smithi112.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-11T00:46:06.121 INFO:teuthology.orchestra.run.smithi112.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 3.2 MB/s | 245 kB 00:00 2024-09-11T00:46:06.179 INFO:teuthology.orchestra.run.smithi112.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-11T00:46:06.238 INFO:teuthology.orchestra.run.smithi112.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-09-11T00:46:06.289 INFO:teuthology.orchestra.run.smithi112.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-11T00:46:06.355 INFO:teuthology.orchestra.run.smithi112.stdout:(80/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 1.9 MB/s | 4.4 MB 00:02 2024-09-11T00:46:06.381 INFO:teuthology.orchestra.run.smithi112.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-11T00:46:06.406 INFO:teuthology.orchestra.run.smithi112.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-09-11T00:46:06.532 INFO:teuthology.orchestra.run.smithi112.stdout:(83/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-09-11T00:46:06.598 INFO:teuthology.orchestra.run.smithi112.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-09-11T00:46:06.624 INFO:teuthology.orchestra.run.smithi112.stdout:(85/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-09-11T00:46:06.625 INFO:teuthology.orchestra.run.smithi078.stdout:(154/157): NetworkManager-tui-1.49.90-1.el9.x86 247 kB/s | 249 kB 00:01 2024-09-11T00:46:06.627 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-11T00:46:06.649 INFO:teuthology.orchestra.run.smithi112.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-11T00:46:06.666 INFO:teuthology.orchestra.run.smithi112.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-11T00:46:06.723 INFO:teuthology.orchestra.run.smithi112.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 3.6 MB/s | 269 kB 00:00 2024-09-11T00:46:06.779 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-11T00:46:06.779 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-11T00:46:06.783 INFO:teuthology.orchestra.run.smithi112.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-11T00:46:06.809 INFO:teuthology.orchestra.run.smithi112.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-11T00:46:06.834 INFO:teuthology.orchestra.run.smithi112.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-11T00:46:06.859 INFO:teuthology.orchestra.run.smithi112.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-11T00:46:06.951 INFO:teuthology.orchestra.run.smithi112.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 454 kB/s | 42 kB 00:00 2024-09-11T00:46:07.026 INFO:teuthology.orchestra.run.smithi112.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 590 kB/s | 44 kB 00:00 2024-09-11T00:46:07.085 INFO:teuthology.orchestra.run.smithi112.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-11T00:46:07.144 INFO:teuthology.orchestra.run.smithi112.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-09-11T00:46:07.192 INFO:teuthology.orchestra.run.smithi078.stdout:(155/157): NetworkManager-libnm-1.49.90-1.el9.x 672 kB/s | 1.8 MB 00:02 2024-09-11T00:46:07.352 INFO:teuthology.orchestra.run.smithi112.stdout:(97/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-11T00:46:07.377 INFO:teuthology.orchestra.run.smithi112.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 203 kB/s | 47 kB 00:00 2024-09-11T00:46:07.411 INFO:teuthology.orchestra.run.smithi112.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-11T00:46:07.445 INFO:teuthology.orchestra.run.smithi112.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 863 kB/s | 58 kB 00:00 2024-09-11T00:46:07.478 INFO:teuthology.orchestra.run.smithi112.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-11T00:46:07.529 INFO:teuthology.orchestra.run.smithi112.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 761 kB/s | 64 kB 00:00 2024-09-11T00:46:07.554 INFO:teuthology.orchestra.run.smithi112.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-09-11T00:46:07.637 INFO:teuthology.orchestra.run.smithi112.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-09-11T00:46:07.688 INFO:teuthology.orchestra.run.smithi112.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-11T00:46:07.730 INFO:teuthology.orchestra.run.smithi112.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-09-11T00:46:07.755 INFO:teuthology.orchestra.run.smithi112.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-11T00:46:07.789 INFO:teuthology.orchestra.run.smithi112.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-09-11T00:46:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 523 kB/s | 26 kB 00:00 2024-09-11T00:46:07.864 INFO:teuthology.orchestra.run.smithi112.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-11T00:46:07.898 INFO:teuthology.orchestra.run.smithi112.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-11T00:46:07.931 INFO:teuthology.orchestra.run.smithi112.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-11T00:46:08.015 INFO:teuthology.orchestra.run.smithi112.stdout:(113/157): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-11T00:46:08.049 INFO:teuthology.orchestra.run.smithi112.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-11T00:46:08.074 INFO:teuthology.orchestra.run.smithi112.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T00:46:08.099 INFO:teuthology.orchestra.run.smithi112.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-11T00:46:08.125 INFO:teuthology.orchestra.run.smithi112.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-11T00:46:08.150 INFO:teuthology.orchestra.run.smithi112.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-09-11T00:46:08.175 INFO:teuthology.orchestra.run.smithi112.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T00:46:08.209 INFO:teuthology.orchestra.run.smithi112.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-11T00:46:08.242 INFO:teuthology.orchestra.run.smithi112.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-11T00:46:08.268 INFO:teuthology.orchestra.run.smithi112.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-11T00:46:08.293 INFO:teuthology.orchestra.run.smithi112.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-11T00:46:08.318 INFO:teuthology.orchestra.run.smithi112.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-11T00:46:08.343 INFO:teuthology.orchestra.run.smithi112.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-11T00:46:08.359 INFO:teuthology.orchestra.run.smithi078.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 531 kB/s | 1.4 MB 00:02 2024-09-11T00:46:08.369 INFO:teuthology.orchestra.run.smithi112.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-11T00:46:08.386 INFO:teuthology.orchestra.run.smithi112.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-11T00:46:08.403 INFO:teuthology.orchestra.run.smithi112.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-11T00:46:08.461 INFO:teuthology.orchestra.run.smithi112.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-09-11T00:46:08.478 INFO:teuthology.orchestra.run.smithi112.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-11T00:46:08.504 INFO:teuthology.orchestra.run.smithi112.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T00:46:08.520 INFO:teuthology.orchestra.run.smithi112.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-11T00:46:08.554 INFO:teuthology.orchestra.run.smithi112.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-11T00:46:08.579 INFO:teuthology.orchestra.run.smithi112.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-11T00:46:08.613 INFO:teuthology.orchestra.run.smithi112.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-11T00:46:08.638 INFO:teuthology.orchestra.run.smithi112.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-11T00:46:08.651 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-11T00:46:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-11T00:46:08.663 INFO:teuthology.orchestra.run.smithi112.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-11T00:46:08.667 INFO:teuthology.orchestra.run.smithi078.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 1.2 MB/s | 2.4 MB 00:02 2024-09-11T00:46:08.674 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-09-11T00:46:08.674 INFO:teuthology.orchestra.run.smithi078.stdout:Total 14 MB/s | 240 MB 00:17 2024-09-11T00:46:08.689 INFO:teuthology.orchestra.run.smithi112.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-11T00:46:08.714 INFO:teuthology.orchestra.run.smithi112.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T00:46:08.739 INFO:teuthology.orchestra.run.smithi112.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-11T00:46:08.765 INFO:teuthology.orchestra.run.smithi112.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-11T00:46:08.790 INFO:teuthology.orchestra.run.smithi112.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-11T00:46:08.815 INFO:teuthology.orchestra.run.smithi112.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-11T00:46:08.882 INFO:teuthology.orchestra.run.smithi112.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-11T00:46:08.908 INFO:teuthology.orchestra.run.smithi112.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 800 kB/s | 20 kB 00:00 2024-09-11T00:46:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-11T00:46:08.958 INFO:teuthology.orchestra.run.smithi112.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.0 MB/s | 427 kB 00:00 2024-09-11T00:46:09.008 INFO:teuthology.orchestra.run.smithi112.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-11T00:46:09.259 INFO:teuthology.orchestra.run.smithi112.stdout:(149/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 13 MB/s | 3.2 MB 00:00 2024-09-11T00:46:09.318 INFO:teuthology.orchestra.run.smithi112.stdout:(150/157): librados2-19.3.0-4854.g8f219c47.el9. 9.6 MB/s | 3.5 MB 00:00 2024-09-11T00:46:09.534 INFO:teuthology.orchestra.run.smithi112.stdout:(151/157): postgresql-server-13.16-1.el9.x86_64 1.5 MB/s | 5.8 MB 00:03 2024-09-11T00:46:09.922 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-09-11T00:46:10.073 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-09-11T00:46:10.073 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-09-11T00:46:10.109 INFO:teuthology.orchestra.run.smithi112.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 73 kB/s | 41 kB 00:00 2024-09-11T00:46:10.318 INFO:teuthology.orchestra.run.smithi112.stdout:(153/157): NetworkManager-libnm-1.49.90-1.el9.x 1.8 MB/s | 1.8 MB 00:01 2024-09-11T00:46:10.360 INFO:teuthology.orchestra.run.smithi112.stdout:(154/157): NetworkManager-1.49.90-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-09-11T00:46:10.927 INFO:teuthology.orchestra.run.smithi112.stdout:(155/157): NetworkManager-tui-1.49.90-1.el9.x86 304 kB/s | 249 kB 00:00 2024-09-11T00:46:11.335 INFO:teuthology.orchestra.run.smithi112.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 1.3 MB/s | 1.4 MB 00:01 2024-09-11T00:46:11.377 INFO:teuthology.orchestra.run.smithi112.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.4 MB/s | 2.4 MB 00:01 2024-09-11T00:46:11.384 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-11T00:46:11.384 INFO:teuthology.orchestra.run.smithi112.stdout:Total 14 MB/s | 240 MB 00:17 2024-09-11T00:46:12.388 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-09-11T00:46:12.388 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-09-11T00:46:12.562 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-11T00:46:12.717 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-11T00:46:12.717 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-11T00:46:14.590 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-11T00:46:14.591 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-11T00:46:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-11T00:46:15.260 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-11T00:46:15.298 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-11T00:46:15.360 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-11T00:46:15.394 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-11T00:46:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-11T00:46:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-11T00:46:15.786 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-11T00:46:15.904 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:46:16.026 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:46:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:46:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:46:16.185 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-11T00:46:16.214 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-11T00:46:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-11T00:46:16.415 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-11T00:46:16.444 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:46:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:46:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-11T00:46:16.632 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-11T00:46:16.654 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-11T00:46:16.681 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-11T00:46:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-11T00:46:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-11T00:46:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-11T00:46:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:46:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:46:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:46:17.648 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-11T00:46:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-11T00:46:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-11T00:46:17.783 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-11T00:46:17.829 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-11T00:46:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-11T00:46:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-11T00:46:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-11T00:46:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-11T00:46:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-11T00:46:18.103 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-11T00:46:18.133 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-11T00:46:18.169 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-11T00:46:18.246 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-11T00:46:18.280 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-11T00:46:18.340 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-11T00:46:18.369 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-11T00:46:18.441 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-11T00:46:18.483 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-11T00:46:18.558 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-11T00:46:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-11T00:46:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-11T00:46:18.910 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-09-11T00:46:18.997 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-11T00:46:19.051 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-11T00:46:19.106 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-11T00:46:19.139 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-11T00:46:19.164 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-11T00:46:19.226 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-11T00:46:19.272 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-11T00:46:19.352 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-11T00:46:19.381 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-11T00:46:19.424 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-11T00:46:19.463 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-11T00:46:19.504 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-11T00:46:19.580 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-11T00:46:19.656 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-11T00:46:19.700 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:46:19.716 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-11T00:46:19.808 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-11T00:46:19.829 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:46:19.859 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:46:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-11T00:46:19.937 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-11T00:46:19.941 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:46:19.976 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-11T00:46:19.980 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-11T00:46:20.011 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-11T00:46:20.015 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-11T00:46:20.059 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-11T00:46:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-11T00:46:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-11T00:46:20.181 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-11T00:46:20.191 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-11T00:46:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-11T00:46:20.272 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:46:20.318 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-11T00:46:20.357 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-11T00:46:20.377 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:46:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-11T00:46:20.406 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-11T00:46:20.469 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-11T00:46:20.499 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-11T00:46:20.507 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-11T00:46:20.526 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-11T00:46:20.547 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-11T00:46:20.586 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-11T00:46:20.612 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-11T00:46:20.637 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-11T00:46:20.713 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-11T00:46:20.751 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-11T00:46:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-11T00:46:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-11T00:46:21.071 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-11T00:46:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-11T00:46:21.156 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-11T00:46:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:46:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-11T00:46:21.190 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-11T00:46:21.194 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-11T00:46:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-11T00:46:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:46:21.257 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-11T00:46:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-11T00:46:21.290 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-11T00:46:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-11T00:46:21.316 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-11T00:46:21.324 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-11T00:46:21.358 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-11T00:46:21.359 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-11T00:46:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-11T00:46:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-11T00:46:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:46:21.468 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-11T00:46:21.555 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-11T00:46:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-11T00:46:21.607 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-11T00:46:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-11T00:46:21.638 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-11T00:46:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-11T00:46:21.685 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-11T00:46:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-11T00:46:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-11T00:46:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-11T00:46:21.789 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-11T00:46:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:46:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-11T00:46:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-11T00:46:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-11T00:46:21.904 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:46:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-11T00:46:21.934 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:46:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-11T00:46:22.016 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:46:22.046 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-11T00:46:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-11T00:46:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-11T00:46:22.084 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-11T00:46:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-11T00:46:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-11T00:46:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-11T00:46:22.255 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-11T00:46:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-11T00:46:22.302 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-11T00:46:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-11T00:46:22.331 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:46:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-11T00:46:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-11T00:46:22.433 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:46:22.456 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-11T00:46:22.501 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-11T00:46:22.509 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-11T00:46:22.518 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-11T00:46:22.541 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-11T00:46:22.554 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-11T00:46:22.560 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-11T00:46:22.587 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-11T00:46:22.642 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-11T00:46:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-11T00:46:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-11T00:46:22.734 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-11T00:46:22.750 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-11T00:46:22.779 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-11T00:46:22.867 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-11T00:46:22.880 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-11T00:46:22.907 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-11T00:46:23.013 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-11T00:46:23.090 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-11T00:46:23.108 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:46:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-11T00:46:23.182 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:46:23.198 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-11T00:46:23.225 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-11T00:46:23.316 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-11T00:46:23.362 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-11T00:46:23.397 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-11T00:46:23.400 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:46:23.435 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-11T00:46:23.477 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-11T00:46:23.507 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-11T00:46:23.536 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-11T00:46:23.587 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-11T00:46:23.642 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-11T00:46:23.669 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-11T00:46:23.705 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-11T00:46:23.712 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-11T00:46:23.745 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-11T00:46:23.789 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-11T00:46:23.795 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-11T00:46:23.821 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-11T00:46:23.843 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-11T00:46:23.857 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-11T00:46:23.860 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-11T00:46:23.925 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-11T00:46:23.932 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-11T00:46:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-11T00:46:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-11T00:46:23.979 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-11T00:46:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-11T00:46:24.010 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-11T00:46:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-11T00:46:24.064 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-11T00:46:24.100 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-11T00:46:24.121 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-11T00:46:24.143 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-11T00:46:24.157 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-11T00:46:24.233 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-11T00:46:24.367 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-11T00:46:24.428 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-11T00:46:24.540 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-11T00:46:24.580 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-11T00:46:24.597 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-11T00:46:24.621 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-11T00:46:24.643 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-11T00:46:24.716 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-11T00:46:24.722 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-11T00:46:24.755 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-11T00:46:24.795 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-11T00:46:24.836 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-11T00:46:24.851 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-11T00:46:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-11T00:46:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-11T00:46:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-11T00:46:25.112 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-11T00:46:25.160 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-11T00:46:25.180 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-11T00:46:25.197 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-11T00:46:25.238 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-11T00:46:25.276 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-11T00:46:25.278 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-11T00:46:25.304 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-11T00:46:25.310 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-11T00:46:25.337 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-11T00:46:25.340 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-11T00:46:25.378 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-11T00:46:25.411 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-11T00:46:25.424 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-11T00:46:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-11T00:46:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-11T00:46:25.509 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-11T00:46:25.540 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-11T00:46:25.574 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-11T00:46:25.582 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-11T00:46:25.617 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-11T00:46:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-11T00:46:25.655 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-11T00:46:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-11T00:46:25.665 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-11T00:46:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-11T00:46:25.768 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-11T00:46:25.791 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-11T00:46:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-11T00:46:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-11T00:46:25.824 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-11T00:46:25.858 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-11T00:46:25.890 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-11T00:46:25.981 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-11T00:46:26.046 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-11T00:46:26.119 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-11T00:46:26.150 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-11T00:46:26.186 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-11T00:46:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-11T00:46:26.347 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-11T00:46:26.403 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-11T00:46:26.475 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-11T00:46:26.505 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-11T00:46:26.512 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-11T00:46:26.539 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-11T00:46:26.565 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-11T00:46:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-11T00:46:26.615 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-11T00:46:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-11T00:46:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-11T00:46:26.754 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-11T00:46:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-11T00:46:26.798 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-11T00:46:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-11T00:46:26.848 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-11T00:46:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-11T00:46:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-11T00:46:26.948 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-11T00:46:26.981 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-11T00:46:27.029 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-11T00:46:27.063 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-11T00:46:27.088 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-11T00:46:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-11T00:46:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-11T00:46:27.162 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-11T00:46:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-11T00:46:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-11T00:46:27.328 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-11T00:46:27.356 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-11T00:46:27.390 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-11T00:46:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-11T00:46:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-11T00:46:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-11T00:46:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-11T00:46:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-11T00:46:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-11T00:46:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-11T00:46:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-11T00:46:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-11T00:46:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-11T00:46:27.897 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-11T00:46:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-11T00:46:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-11T00:46:28.021 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-11T00:46:28.682 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-11T00:46:28.736 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-11T00:46:28.769 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-11T00:46:28.797 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:46:28.835 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:46:28.868 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-11T00:46:28.916 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-11T00:46:28.969 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-11T00:46:29.056 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-11T00:46:29.089 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-11T00:46:29.273 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-11T00:46:29.274 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-11T00:46:29.303 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-11T00:46:29.360 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-11T00:46:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-11T00:46:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:46:29.388 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-11T00:46:29.421 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-11T00:46:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-11T00:46:29.514 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-11T00:46:29.514 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T00:46:29.514 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T00:46:29.514 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:46:29.548 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-11T00:46:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-11T00:46:29.801 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-11T00:46:29.900 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-11T00:46:29.930 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-11T00:46:29.967 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-11T00:46:30.018 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-11T00:46:30.126 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-11T00:46:30.457 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-11T00:46:30.488 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-11T00:46:30.558 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-11T00:46:30.589 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-11T00:46:31.338 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-11T00:46:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-11T00:46:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-11T00:46:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-11T00:46:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-11T00:46:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-11T00:46:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-11T00:46:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-11T00:46:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-11T00:46:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-11T00:46:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-11T00:46:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-11T00:46:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-11T00:46:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:46:32.910 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:46:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:46:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-11T00:46:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-11T00:46:33.596 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T00:46:33.596 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T00:46:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:46:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-11T00:46:34.979 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:46:35.015 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:46:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:46:35.611 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-11T00:46:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-11T00:46:35.705 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T00:46:35.705 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T00:46:35.705 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:46:35.753 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-11T00:46:41.759 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-11T00:46:41.760 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /sys 2024-09-11T00:46:41.760 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /proc 2024-09-11T00:46:41.760 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /mnt 2024-09-11T00:46:41.760 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /var/tmp 2024-09-11T00:46:41.760 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /home 2024-09-11T00:46:41.760 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /root 2024-09-11T00:46:41.760 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /tmp 2024-09-11T00:46:41.760 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:46:41.948 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-11T00:46:41.964 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-11T00:46:41.964 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:41.965 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T00:46:41.965 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:46:43.466 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-11T00:46:43.493 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-11T00:46:43.493 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:43.493 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T00:46:43.493 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T00:46:43.494 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T00:46:43.494 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:46:43.717 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-11T00:46:43.740 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-11T00:46:43.740 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:43.740 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T00:46:43.741 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T00:46:43.741 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T00:46:43.741 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:46:44.184 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-11T00:46:44.208 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-11T00:46:44.208 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:44.208 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T00:46:44.208 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T00:46:44.208 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T00:46:44.208 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:46:44.314 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-11T00:46:44.363 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-11T00:46:44.418 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-11T00:46:44.605 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-11T00:46:44.660 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-11T00:46:44.730 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-11T00:46:44.818 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-11T00:46:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-11T00:46:45.002 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-11T00:46:45.042 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-11T00:46:45.079 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-11T00:46:45.373 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-11T00:46:45.373 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2024-09-11T00:46:45.374 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2024-09-11T00:46:45.374 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2024-09-11T00:46:45.374 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2024-09-11T00:46:45.374 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2024-09-11T00:46:45.374 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /root 2024-09-11T00:46:45.374 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2024-09-11T00:46:45.374 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:46:45.543 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-11T00:46:45.569 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-11T00:46:45.570 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-11T00:46:45.589 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-11T00:46:45.589 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:45.589 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T00:46:45.589 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:46:45.601 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-11T00:46:45.667 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-11T00:46:45.692 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-11T00:46:47.098 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-11T00:46:47.124 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-11T00:46:47.125 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:47.125 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T00:46:47.125 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T00:46:47.125 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T00:46:47.125 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:46:47.348 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-11T00:46:47.371 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-11T00:46:47.372 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-09-11T00:46:47.372 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-09-11T00:46:47.372 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-09-11T00:46:47.372 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-09-11T00:46:47.372 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-09-11T00:46:47.372 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-09-11T00:46:47.372 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-09-11T00:46:47.372 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:46:47.372 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-11T00:46:47.373 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:47.373 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T00:46:47.373 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T00:46:47.373 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T00:46:47.373 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:46:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-11T00:46:47.562 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-11T00:46:47.562 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:47.562 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T00:46:47.562 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:46:47.816 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-11T00:46:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-11T00:46:47.839 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:47.839 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T00:46:47.839 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T00:46:47.840 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T00:46:47.840 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:46:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-11T00:46:48.002 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-11T00:46:48.058 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-11T00:46:48.251 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-11T00:46:48.325 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-11T00:46:48.412 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-11T00:46:48.499 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-11T00:46:48.639 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-11T00:46:48.684 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-11T00:46:48.723 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-11T00:46:48.751 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-11T00:46:48.861 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-11T00:46:48.916 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-11T00:46:49.066 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-11T00:46:49.092 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-11T00:46:49.092 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:49.093 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T00:46:49.093 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T00:46:49.093 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T00:46:49.093 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:46:49.240 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-11T00:46:49.275 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-11T00:46:49.307 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-11T00:46:49.317 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-11T00:46:49.339 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-11T00:46:49.340 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:49.340 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T00:46:49.340 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T00:46:49.340 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T00:46:49.340 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:46:49.373 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-11T00:46:49.406 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-11T00:46:49.784 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-11T00:46:49.807 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-11T00:46:49.807 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:49.807 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T00:46:49.807 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T00:46:49.807 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T00:46:49.807 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:46:49.870 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-11T00:46:49.899 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-11T00:46:49.931 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-11T00:46:49.979 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-11T00:46:50.013 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-11T00:46:50.034 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-11T00:46:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-11T00:46:50.201 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-11T00:46:50.217 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-11T00:46:50.225 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-11T00:46:50.225 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:50.225 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T00:46:50.225 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T00:46:50.225 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T00:46:50.225 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:46:50.268 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-11T00:46:50.277 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-11T00:46:50.289 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-11T00:46:50.338 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-11T00:46:50.395 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-11T00:46:50.424 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-11T00:46:50.442 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-11T00:46:50.557 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-11T00:46:50.593 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-11T00:46:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-11T00:46:50.669 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-11T00:46:51.115 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-11T00:46:51.143 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-11T00:46:51.175 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-11T00:46:51.241 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-11T00:46:51.274 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-11T00:46:51.394 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-11T00:46:51.430 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-11T00:46:51.453 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-11T00:46:51.454 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:51.454 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T00:46:51.454 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T00:46:51.454 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T00:46:51.454 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:46:51.497 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-11T00:46:51.521 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-11T00:46:51.521 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:51.521 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T00:46:51.521 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:46:51.564 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-11T00:46:51.588 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-11T00:46:51.588 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:51.589 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T00:46:51.589 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T00:46:51.589 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T00:46:51.589 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:46:51.898 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-11T00:46:51.922 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-11T00:46:51.923 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:51.923 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T00:46:51.923 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T00:46:51.923 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T00:46:51.923 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:46:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-11T00:46:52.646 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-11T00:46:53.608 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-11T00:46:53.638 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-11T00:46:53.754 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-11T00:46:53.917 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-11T00:46:53.940 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-11T00:46:53.964 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-11T00:46:53.964 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:53.964 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T00:46:53.964 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T00:46:53.964 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T00:46:53.964 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:46:54.015 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-11T00:46:54.028 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-11T00:46:54.133 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-11T00:46:54.192 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-11T00:46:54.437 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-11T00:46:54.497 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-11T00:46:55.140 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-11T00:46:55.168 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-11T00:46:55.192 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-11T00:46:55.192 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:55.192 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T00:46:55.193 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T00:46:55.193 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T00:46:55.193 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:46:55.244 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-11T00:46:55.267 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-11T00:46:55.267 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:55.267 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T00:46:55.267 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:46:55.319 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-11T00:46:55.343 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-11T00:46:55.343 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:55.343 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T00:46:55.343 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T00:46:55.343 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T00:46:55.343 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:46:55.453 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-11T00:46:55.480 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-11T00:46:55.593 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-11T00:46:55.661 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-11T00:46:55.685 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-11T00:46:55.685 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:55.685 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T00:46:55.685 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T00:46:55.685 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T00:46:55.686 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:46:55.758 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-11T00:46:55.791 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-11T00:46:55.814 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-11T00:46:55.814 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:55.815 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T00:46:55.815 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T00:46:55.815 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T00:46:55.815 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:46:55.858 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-11T00:46:55.869 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-11T00:46:55.967 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-11T00:46:56.014 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-11T00:46:56.570 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-11T00:46:56.610 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-11T00:46:56.651 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-11T00:46:56.684 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-11T00:46:56.702 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-11T00:46:56.886 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-11T00:46:56.933 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-11T00:46:56.964 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-11T00:46:56.970 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-11T00:46:56.993 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-11T00:46:57.002 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-11T00:46:57.025 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-11T00:46:57.025 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:57.025 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T00:46:57.025 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T00:46:57.026 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T00:46:57.026 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:46:57.031 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-11T00:46:57.070 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-11T00:46:57.092 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-11T00:46:57.092 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:57.093 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T00:46:57.093 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:46:57.137 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-11T00:46:57.159 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-11T00:46:57.159 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:57.159 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T00:46:57.159 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T00:46:57.160 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T00:46:57.160 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:46:57.462 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-11T00:46:57.485 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-11T00:46:57.485 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:46:57.485 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T00:46:57.486 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T00:46:57.486 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T00:46:57.486 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:46:57.655 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:46:57.696 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:46:57.717 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:46:58.557 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-11T00:46:58.643 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-11T00:46:58.714 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-11T00:46:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-11T00:46:58.838 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-11T00:46:58.864 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-11T00:46:58.889 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-11T00:46:58.889 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-11T00:46:58.974 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-11T00:46:58.974 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-11T00:46:59.058 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-11T00:46:59.083 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-11T00:46:59.089 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:46:59.089 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:46:59.268 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:46:59.268 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-11T00:46:59.352 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-11T00:46:59.415 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:47:00.407 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-11T00:47:00.456 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-11T00:47:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-11T00:47:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-11T00:47:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-11T00:47:00.763 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-11T00:47:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-11T00:47:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-11T00:47:00.872 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-11T00:47:00.926 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-11T00:47:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:47:01.607 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:47:01.627 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:47:02.141 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-11T00:47:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-11T00:47:02.231 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-11T00:47:02.287 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-11T00:47:02.324 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-11T00:47:02.453 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-11T00:47:02.503 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-11T00:47:02.537 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-11T00:47:02.545 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-11T00:47:02.576 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-11T00:47:02.585 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-11T00:47:02.605 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-11T00:47:02.643 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-11T00:47:02.676 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-11T00:47:02.706 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-11T00:47:02.732 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-11T00:47:02.766 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-11T00:47:02.766 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-11T00:47:02.859 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-11T00:47:02.859 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-11T00:47:02.943 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-11T00:47:02.968 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-11T00:47:02.973 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:47:02.973 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:47:03.153 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:47:03.153 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-11T00:47:03.237 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-11T00:47:03.238 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:47:03.274 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:47:03.294 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:47:03.309 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:47:04.115 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-11T00:47:04.212 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-11T00:47:04.252 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-11T00:47:04.333 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-11T00:47:04.365 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-11T00:47:04.390 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-11T00:47:04.424 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-11T00:47:04.424 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-11T00:47:04.517 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-11T00:47:04.517 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-11T00:47:04.601 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-11T00:47:04.626 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-11T00:47:04.631 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:47:04.631 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:47:04.811 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:47:04.811 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-11T00:47:04.895 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-11T00:47:04.958 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:47:05.722 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-11T00:47:05.723 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:47:05.724 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-11T00:47:05.724 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-11T00:47:05.724 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-11T00:47:05.724 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-11T00:47:05.724 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-11T00:47:05.724 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-11T00:47:05.724 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-11T00:47:05.724 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-11T00:47:05.724 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-11T00:47:05.724 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-11T00:47:05.724 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-11T00:47:05.725 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-11T00:47:05.725 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-11T00:47:05.725 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-11T00:47:05.725 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-11T00:47:05.725 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-11T00:47:05.725 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-11T00:47:05.738 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-11T00:47:05.738 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-11T00:47:05.738 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-11T00:47:05.738 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-11T00:47:05.738 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-11T00:47:05.738 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-11T00:47:05.738 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-11T00:47:05.738 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-11T00:47:05.738 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-11T00:47:05.739 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-11T00:47:05.740 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-11T00:47:05.740 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-11T00:47:05.740 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-11T00:47:05.740 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-11T00:47:05.740 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-11T00:47:05.740 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-11T00:47:05.740 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-11T00:47:05.740 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-11T00:47:05.740 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-11T00:47:05.740 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-11T00:47:05.740 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-11T00:47:05.740 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-11T00:47:05.741 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-11T00:47:05.741 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-11T00:47:05.741 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-11T00:47:05.741 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-11T00:47:05.741 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-11T00:47:05.741 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-11T00:47:05.741 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-11T00:47:05.741 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-11T00:47:05.741 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-11T00:47:05.741 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-11T00:47:05.741 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-11T00:47:05.741 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-11T00:47:05.742 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-11T00:47:05.743 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-11T00:47:05.743 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-11T00:47:05.743 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-11T00:47:05.743 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-11T00:47:05.743 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-11T00:47:05.743 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-11T00:47:05.743 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-11T00:47:05.743 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-11T00:47:05.743 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-11T00:47:05.743 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-11T00:47:05.743 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-11T00:47:05.743 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-11T00:47:05.744 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-11T00:47:05.744 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-11T00:47:05.744 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-11T00:47:05.744 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-11T00:47:05.744 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-11T00:47:05.744 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-11T00:47:05.744 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-11T00:47:05.744 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-11T00:47:05.744 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-11T00:47:05.745 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-11T00:47:05.745 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-11T00:47:05.745 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-11T00:47:05.745 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-11T00:47:05.745 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-11T00:47:05.745 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-11T00:47:05.745 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-11T00:47:05.745 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-11T00:47:05.746 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-11T00:47:05.747 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-11T00:47:05.747 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-11T00:47:05.747 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-11T00:47:05.747 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-11T00:47:05.747 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-11T00:47:05.747 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-11T00:47:05.747 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-11T00:47:05.747 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-11T00:47:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-11T00:47:05.749 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-11T00:47:05.749 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-11T00:47:05.749 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-11T00:47:05.749 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-11T00:47:05.749 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-11T00:47:08.232 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:47:08.232 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:47:08.232 INFO:teuthology.orchestra.run.smithi110.stdout:Upgraded: 2024-09-11T00:47:08.232 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T00:47:08.232 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T00:47:08.232 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.233 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.234 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T00:47:08.235 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T00:47:08.236 INFO:teuthology.orchestra.run.smithi110.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T00:47:08.236 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T00:47:08.236 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T00:47:08.236 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T00:47:08.236 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T00:47:08.236 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T00:47:08.236 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.236 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.236 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.236 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T00:47:08.236 INFO:teuthology.orchestra.run.smithi110.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T00:47:08.237 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T00:47:08.238 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T00:47:08.239 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T00:47:08.239 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T00:47:08.239 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T00:47:08.239 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.239 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.239 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.239 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T00:47:08.239 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T00:47:08.239 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T00:47:08.239 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T00:47:08.239 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T00:47:08.240 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T00:47:08.240 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T00:47:08.240 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T00:47:08.240 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T00:47:08.240 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T00:47:08.240 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T00:47:08.240 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T00:47:08.240 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T00:47:08.240 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T00:47:08.241 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T00:47:08.241 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T00:47:08.241 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T00:47:08.241 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T00:47:08.241 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T00:47:08.241 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T00:47:08.242 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T00:47:08.242 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T00:47:08.242 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T00:47:08.242 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T00:47:08.242 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T00:47:08.242 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T00:47:08.242 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T00:47:08.242 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T00:47:08.242 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T00:47:08.242 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T00:47:08.243 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T00:47:08.244 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T00:47:08.245 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T00:47:08.245 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T00:47:08.245 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.245 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.245 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:08.245 INFO:teuthology.orchestra.run.smithi110.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T00:47:08.245 INFO:teuthology.orchestra.run.smithi110.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T00:47:08.245 INFO:teuthology.orchestra.run.smithi110.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T00:47:08.245 INFO:teuthology.orchestra.run.smithi110.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T00:47:08.245 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T00:47:08.245 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T00:47:08.246 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T00:47:08.246 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:47:08.246 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-11T00:47:08.677 DEBUG:teuthology.parallel:result is None 2024-09-11T00:47:09.654 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:47:09.654 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-11T00:47:09.654 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-11T00:47:09.654 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-11T00:47:09.654 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-11T00:47:09.655 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-11T00:47:09.656 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-11T00:47:09.656 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-11T00:47:09.656 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-11T00:47:09.656 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-11T00:47:09.656 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-11T00:47:09.656 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-11T00:47:09.656 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-11T00:47:09.656 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-11T00:47:09.656 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-11T00:47:09.656 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-11T00:47:09.656 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-11T00:47:09.657 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-11T00:47:09.657 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-11T00:47:09.657 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-11T00:47:09.657 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-11T00:47:09.657 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-11T00:47:09.657 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-11T00:47:09.657 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-11T00:47:09.657 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-11T00:47:09.657 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-11T00:47:09.657 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-11T00:47:09.657 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-11T00:47:09.658 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-11T00:47:09.658 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-11T00:47:09.658 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-11T00:47:09.658 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-11T00:47:09.658 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-11T00:47:09.658 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-11T00:47:09.658 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-11T00:47:09.658 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-11T00:47:09.658 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-11T00:47:09.658 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-11T00:47:09.659 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-11T00:47:09.660 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-11T00:47:09.661 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-11T00:47:09.662 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-11T00:47:09.663 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-11T00:47:09.664 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-11T00:47:09.665 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-11T00:47:09.666 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-11T00:47:09.666 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-11T00:47:09.666 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-11T00:47:09.666 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-11T00:47:09.666 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-11T00:47:09.666 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-11T00:47:09.666 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-11T00:47:09.666 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-11T00:47:09.666 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-11T00:47:09.666 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-11T00:47:09.666 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-11T00:47:09.666 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-11T00:47:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:47:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-11T00:47:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-11T00:47:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-11T00:47:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-11T00:47:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-11T00:47:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-11T00:47:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-11T00:47:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-11T00:47:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-11T00:47:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-11T00:47:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-11T00:47:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-11T00:47:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-11T00:47:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-11T00:47:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-11T00:47:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-11T00:47:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-11T00:47:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-11T00:47:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-11T00:47:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-11T00:47:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-11T00:47:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-11T00:47:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-11T00:47:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-11T00:47:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-11T00:47:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-11T00:47:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-11T00:47:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-11T00:47:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-11T00:47:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-11T00:47:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-11T00:47:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-11T00:47:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-11T00:47:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-11T00:47:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-11T00:47:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-11T00:47:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-11T00:47:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-11T00:47:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-11T00:47:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-11T00:47:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-11T00:47:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-11T00:47:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-11T00:47:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-11T00:47:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-11T00:47:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-11T00:47:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-11T00:47:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-11T00:47:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-11T00:47:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-11T00:47:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-11T00:47:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-11T00:47:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-11T00:47:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-11T00:47:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-11T00:47:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-11T00:47:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-11T00:47:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-11T00:47:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-11T00:47:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-11T00:47:12.201 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:47:12.201 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:47:12.201 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2024-09-11T00:47:12.202 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T00:47:12.202 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T00:47:12.202 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T00:47:12.202 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T00:47:12.202 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.202 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.202 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T00:47:12.202 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T00:47:12.202 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-09-11T00:47:12.202 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T00:47:12.202 INFO:teuthology.orchestra.run.smithi078.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T00:47:12.202 INFO:teuthology.orchestra.run.smithi078.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:12.203 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:12.204 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:12.204 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.204 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.204 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:12.204 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.204 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.204 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.204 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:12.204 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:12.204 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.204 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.204 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.205 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T00:47:12.206 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T00:47:12.207 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T00:47:12.208 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T00:47:12.209 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T00:47:12.210 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T00:47:12.211 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T00:47:12.212 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:47:12.213 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-09-11T00:47:12.668 DEBUG:teuthology.parallel:result is None 2024-09-11T00:47:14.154 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:47:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:47:14.155 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-09-11T00:47:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T00:47:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T00:47:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T00:47:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T00:47:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.155 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.156 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.157 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T00:47:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.159 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T00:47:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T00:47:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T00:47:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T00:47:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T00:47:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T00:47:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T00:47:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T00:47:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T00:47:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T00:47:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T00:47:14.160 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T00:47:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T00:47:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T00:47:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T00:47:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T00:47:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T00:47:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T00:47:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T00:47:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T00:47:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T00:47:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T00:47:14.161 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T00:47:14.162 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T00:47:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T00:47:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T00:47:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T00:47:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T00:47:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T00:47:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T00:47:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T00:47:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T00:47:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T00:47:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T00:47:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T00:47:14.163 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T00:47:14.164 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T00:47:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T00:47:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T00:47:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T00:47:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T00:47:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T00:47:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T00:47:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.165 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T00:47:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T00:47:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T00:47:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T00:47:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T00:47:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T00:47:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T00:47:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T00:47:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T00:47:14.166 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T00:47:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T00:47:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T00:47:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T00:47:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T00:47:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T00:47:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T00:47:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T00:47:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T00:47:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:47:14.167 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T00:47:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T00:47:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T00:47:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T00:47:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T00:47:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T00:47:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T00:47:14.168 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:47:14.168 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-11T00:47:14.596 DEBUG:teuthology.parallel:result is None 2024-09-11T00:47:14.597 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:47:14.748 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T00:47:14.775 INFO:teuthology.orchestra.run.smithi078.stdout:19.3.0-4854.g8f219c47.el9 2024-09-11T00:47:14.775 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-11T00:47:14.776 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-11T00:47:14.778 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:47:14.923 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T00:47:14.949 INFO:teuthology.orchestra.run.smithi110.stdout:19.3.0-4854.g8f219c47.el9 2024-09-11T00:47:14.949 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-11T00:47:14.949 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-11T00:47:14.951 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:47:15.099 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T00:47:15.123 INFO:teuthology.orchestra.run.smithi112.stdout:19.3.0-4854.g8f219c47.el9 2024-09-11T00:47:15.123 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-11T00:47:15.123 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-11T00:47:15.125 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T00:47:15.126 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:47:15.126 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T00:47:15.157 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T00:47:15.158 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T00:47:15.190 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-11T00:47:15.190 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T00:47:15.219 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T00:47:15.219 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:47:15.219 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T00:47:15.247 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T00:47:15.315 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T00:47:15.315 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T00:47:15.345 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T00:47:15.412 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-11T00:47:15.413 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T00:47:15.440 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T00:47:15.507 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T00:47:15.507 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:47:15.507 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T00:47:15.534 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T00:47:15.600 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T00:47:15.600 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T00:47:15.628 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T00:47:15.696 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-11T00:47:15.696 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T00:47:15.726 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T00:47:15.796 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T00:47:15.796 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:47:15.796 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T00:47:15.822 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T00:47:15.888 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T00:47:15.888 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T00:47:15.916 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T00:47:15.984 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-11T00:47:15.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T00:47:16.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T00:47:16.082 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T00:47:16.182 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug'}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-11T00:47:16.183 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:47:16.183 INFO:tasks.cephadm:Cluster fsid is 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:47:16.183 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T00:47:16.184 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.78', 'mon.b': '172.21.15.110', 'mon.c': '172.21.15.112'} 2024-09-11T00:47:16.184 INFO:tasks.cephadm:First mon is mon.a on smithi078 2024-09-11T00:47:16.184 INFO:tasks.cephadm:First mgr is x 2024-09-11T00:47:16.184 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T00:47:16.184 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2024-09-11T00:47:16.211 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname $(hostname -s) 2024-09-11T00:47:16.241 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-09-11T00:47:16.270 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-11T00:47:16.270 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:47:16.422 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82938/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4854-g8f219c47', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4854.g8f219c47'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 09:27:13.215997', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-hemanth-testing-2024-09-10-0723', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-11T00:47:16.570 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-hemanth-testing-2024-09-10-0723, sha1 8f219c476cb4f26fc5719c661c0323719a45cb30 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:47:16.594 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-11T00:47:16.594 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-11T00:47:16.595 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T00:47:16.983 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 11 00:47 /home/ubuntu/cephtest/cephadm 2024-09-11T00:47:16.984 DEBUG:teuthology.orchestra.run.smithi110:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T00:47:17.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 11 00:47 /home/ubuntu/cephtest/cephadm 2024-09-11T00:47:17.373 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T00:47:17.754 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 11 00:47 /home/ubuntu/cephtest/cephadm 2024-09-11T00:47:17.754 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T00:47:17.774 DEBUG:teuthology.orchestra.run.smithi110:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T00:47:17.794 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T00:47:17.828 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 on all hosts... 2024-09-11T00:47:17.828 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-11T00:47:17.831 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-11T00:47:17.837 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-11T00:47:19.358 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-11T00:47:19.514 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-11T00:47:19.591 INFO:teuthology.orchestra.run.smithi110.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-11T00:47:47.783 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-11T00:47:47.783 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-11T00:47:47.783 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-11T00:47:47.783 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-09-11T00:47:47.784 INFO:teuthology.orchestra.run.smithi112.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-11T00:47:47.784 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-11T00:47:47.784 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-11T00:48:05.536 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:48:05.536 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-11T00:48:05.536 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-11T00:48:05.536 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2024-09-11T00:48:05.536 INFO:teuthology.orchestra.run.smithi078.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-11T00:48:05.536 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T00:48:05.536 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:48:10.005 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-11T00:48:10.005 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-11T00:48:10.005 INFO:teuthology.orchestra.run.smithi110.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-11T00:48:10.005 INFO:teuthology.orchestra.run.smithi110.stdout: "repo_digests": [ 2024-09-11T00:48:10.005 INFO:teuthology.orchestra.run.smithi110.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-11T00:48:10.005 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-11T00:48:10.006 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-11T00:48:10.031 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2024-09-11T00:48:10.064 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph 2024-09-11T00:48:10.091 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-09-11T00:48:10.125 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2024-09-11T00:48:10.153 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /etc/ceph 2024-09-11T00:48:10.178 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-09-11T00:48:10.207 INFO:tasks.cephadm:Writing seed config... 2024-09-11T00:48:10.208 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-11T00:48:10.208 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-11T00:48:10.208 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-11T00:48:10.208 INFO:tasks.cephadm: override: [mgr] client mount timeout = 30 2024-09-11T00:48:10.208 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-11T00:48:10.209 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T00:48:10.209 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T00:48:10.209 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T00:48:10.209 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T00:48:10.209 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T00:48:10.209 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-11T00:48:10.209 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-11T00:48:10.209 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-11T00:48:10.209 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-11T00:48:10.209 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-11T00:48:10.209 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-11T00:48:10.210 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T00:48:10.210 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T00:48:10.210 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-11T00:48:10.210 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-11T00:48:10.210 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-11T00:48:10.210 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-11T00:48:10.210 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-11T00:48:10.210 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-11T00:48:10.210 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T00:48:10.210 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-11T00:48:10.211 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:48:10.211 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T00:48:10.229 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 64384694-6fd7-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 client mount timeout = 30 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2024-09-11T00:48:10.229 DEBUG:teuthology.orchestra.run.smithi078:mon.a> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T00:48:10.272 DEBUG:teuthology.orchestra.run.smithi078:mgr.x> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T00:48:10.315 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T00:48:10.315 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 -v bootstrap --fsid 64384694-6fd7-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.78 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:48:10.520 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-09-11T00:48:10.521 INFO:teuthology.orchestra.run.smithi078.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', '-v', 'bootstrap', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.78', '--skip-admin-label'] 2024-09-11T00:48:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T00:48:10.521 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying podman|docker is present... 2024-09-11T00:48:10.545 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 5.2.2 2024-09-11T00:48:10.545 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying lvm2 is present... 2024-09-11T00:48:10.546 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying time synchronization is in place... 2024-09-11T00:48:10.553 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T00:48:10.553 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T00:48:10.559 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T00:48:10.559 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-09-11T00:48:10.566 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-09-11T00:48:10.572 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-09-11T00:48:10.572 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-09-11T00:48:10.572 INFO:teuthology.orchestra.run.smithi078.stdout:Repeating the final host check... 2024-09-11T00:48:10.595 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 5.2.2 2024-09-11T00:48:10.595 INFO:teuthology.orchestra.run.smithi078.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T00:48:10.595 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl is present 2024-09-11T00:48:10.595 INFO:teuthology.orchestra.run.smithi078.stdout:lvcreate is present 2024-09-11T00:48:10.602 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T00:48:10.602 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T00:48:10.608 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T00:48:10.608 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-09-11T00:48:10.614 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-09-11T00:48:10.621 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-09-11T00:48:10.621 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-09-11T00:48:10.621 INFO:teuthology.orchestra.run.smithi078.stdout:Host looks OK 2024-09-11T00:48:10.621 INFO:teuthology.orchestra.run.smithi078.stdout:Cluster fsid: 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:10.621 INFO:teuthology.orchestra.run.smithi078.stdout:Acquiring lock 140563252089712 on /run/cephadm/64384694-6fd7-11ef-bcea-c7b262605968.lock 2024-09-11T00:48:10.621 INFO:teuthology.orchestra.run.smithi078.stdout:Lock 140563252089712 acquired on /run/cephadm/64384694-6fd7-11ef-bcea-c7b262605968.lock 2024-09-11T00:48:10.621 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 3300 ... 2024-09-11T00:48:10.622 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 6789 ... 2024-09-11T00:48:10.622 INFO:teuthology.orchestra.run.smithi078.stdout:Base mon IP(s) is [172.21.15.78:3300, 172.21.15.78:6789], mon addrv is [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-09-11T00:48:10.625 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.78 metric 100 2024-09-11T00:48:10.625 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.78 metric 100 2024-09-11T00:48:10.628 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T00:48:10.628 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T00:48:10.628 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T00:48:10.630 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T00:48:10.630 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T00:48:10.630 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T00:48:10.630 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T00:48:10.630 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link noprefixroute 2024-09-11T00:48:10.630 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T00:48:10.631 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-09-11T00:48:10.631 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-09-11T00:48:10.631 INFO:teuthology.orchestra.run.smithi078.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T00:48:10.631 INFO:teuthology.orchestra.run.smithi078.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T00:48:10.632 INFO:teuthology.orchestra.run.smithi078.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-11T00:48:11.121 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-11T00:48:11.121 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-11T00:48:11.122 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T00:48:11.122 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:edd8cbe4deb828f809e8d76f308301dd93e971948e909d25cc03015fed1fd796 2024-09-11T00:48:11.122 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T00:48:11.122 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying config sha256:589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-11T00:48:11.122 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T00:48:12.058 INFO:teuthology.orchestra.run.smithi078.stdout:ceph: stdout ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-11T00:48:12.058 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph version: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-11T00:48:12.059 INFO:teuthology.orchestra.run.smithi078.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T00:48:13.030 INFO:teuthology.orchestra.run.smithi078.stdout:stat: stdout 167 167 2024-09-11T00:48:13.030 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial keys... 2024-09-11T00:48:14.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQDN6OBmk5sOFBAAiHU7All1fbZ+YQZYYQN/uQ== 2024-09-11T00:48:14.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQDO6OBm2OH4FhAAV194B4l5dGEqXVUOg2vb0Q== 2024-09-11T00:48:15.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQDP6OBmeEphFBAAxN0nxU4OoGuUUzExAEnRJw== 2024-09-11T00:48:15.907 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial monmap... 2024-09-11T00:48:16.729 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T00:48:16.729 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-11T00:48:16.729 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:16.729 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T00:48:16.729 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool for a [v2:172.21.15.78:3300,v1:172.21.15.78:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T00:48:16.729 INFO:teuthology.orchestra.run.smithi078.stdout:setting min_mon_release = quincy 2024-09-11T00:48:16.729 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: set fsid to 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:16.729 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T00:48:16.729 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:48:16.730 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mon... 2024-09-11T00:48:21.426 INFO:teuthology.orchestra.run.smithi078.stdout:create mon.a on 2024-09-11T00:48:22.073 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T00:48:22.563 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T00:48:23.094 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-64384694-6fd7-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-64384694-6fd7-11ef-bcea-c7b262605968.target. 2024-09-11T00:48:23.094 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-64384694-6fd7-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-64384694-6fd7-11ef-bcea-c7b262605968.target. 2024-09-11T00:48:24.212 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.a 2024-09-11T00:48:24.212 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T00:48:24.625 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-64384694-6fd7-11ef-bcea-c7b262605968.target.wants/ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-64384694-6fd7-11ef-bcea-c7b262605968@.service. 2024-09-11T00:48:24.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:24 smithi078 podman[30931]: 2024-09-11 00:48:24.761846452 +0000 UTC m=+0.015874298 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:48:25.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:25 smithi078 podman[30931]: 2024-09-11 00:48:25.70227462 +0000 UTC m=+0.956302451 container create d4d08c58261ffd727fb50b0201109802b0d615ec91923a52d94506618984e697 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T00:48:27.820 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T00:48:27.820 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-09-11T00:48:27.828 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T00:48:27.828 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-09-11T00:48:27.828 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-09-11T00:48:27.828 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T00:48:27.828 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon to start... 2024-09-11T00:48:27.829 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon... 2024-09-11T00:48:27.955 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:27 smithi078 podman[30931]: 2024-09-11 00:48:27.601830616 +0000 UTC m=+2.855858447 container init d4d08c58261ffd727fb50b0201109802b0d615ec91923a52d94506618984e697 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:48:27.955 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:27 smithi078 podman[30931]: 2024-09-11 00:48:27.606121937 +0000 UTC m=+2.860149785 container start d4d08c58261ffd727fb50b0201109802b0d615ec91923a52d94506618984e697 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T00:48:27.955 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:27 smithi078 ceph-mon[30950]: mkfs 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:27.955 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:27 smithi078 ceph-mon[30950]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T00:48:27.955 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:27 smithi078 bash[30931]: d4d08c58261ffd727fb50b0201109802b0d615ec91923a52d94506618984e697 2024-09-11T00:48:27.955 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:27 smithi078 systemd[1]: Started Ceph mon.a for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:48:28.944 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:28 smithi078 ceph-mon[30950]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T00:48:28.944 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:28 smithi078 ceph-mon[30950]: monmap epoch 1 2024-09-11T00:48:28.944 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:28 smithi078 ceph-mon[30950]: fsid 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:28.944 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:28 smithi078 ceph-mon[30950]: last_changed 2024-09-11T00:48:16.201306+0000 2024-09-11T00:48:28.944 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:28 smithi078 ceph-mon[30950]: created 2024-09-11T00:48:16.201306+0000 2024-09-11T00:48:28.944 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:28 smithi078 ceph-mon[30950]: min_mon_release 19 (squid) 2024-09-11T00:48:28.944 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:28 smithi078 ceph-mon[30950]: election_strategy: 1 2024-09-11T00:48:28.944 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:28 smithi078 ceph-mon[30950]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-11T00:48:28.944 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:28 smithi078 ceph-mon[30950]: fsmap 2024-09-11T00:48:28.944 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:28 smithi078 ceph-mon[30950]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T00:48:28.945 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:28 smithi078 ceph-mon[30950]: mgrmap e1: no daemons active 2024-09-11T00:48:28.945 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:28 smithi078 ceph-mon[30950]: from='client.? 172.21.15.78:0/1552742721' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-11T00:48:28.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T00:48:28.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout id: 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:28.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T00:48:28.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-11T00:48:28.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout services: 2024-09-11T00:48:28.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.695609s) 2024-09-11T00:48:28.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T00:48:28.948 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T00:48:28.948 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-11T00:48:28.948 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout data: 2024-09-11T00:48:28.949 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T00:48:28.949 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T00:48:28.949 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T00:48:28.949 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T00:48:28.949 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-11T00:48:28.949 INFO:teuthology.orchestra.run.smithi078.stdout:mon is available 2024-09-11T00:48:28.949 INFO:teuthology.orchestra.run.smithi078.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T00:48:29.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:29 smithi078 ceph-mon[30950]: from='client.? 172.21.15.78:0/3293330422' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-11T00:48:29.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:29 smithi078 ceph-mon[30950]: from='client.? 172.21.15.78:0/3293330422' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T00:48:30.003 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-11T00:48:30.003 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T00:48:30.003 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:30.003 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T00:48:30.003 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-09-11T00:48:30.003 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T00:48:30.004 INFO:teuthology.orchestra.run.smithi078.stdout:Generating new minimal ceph.conf... 2024-09-11T00:48:31.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:31 smithi078 ceph-mon[30950]: from='client.? 172.21.15.78:0/2600207412' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:48:31.441 INFO:teuthology.orchestra.run.smithi078.stdout:Restarting the monitor... 2024-09-11T00:48:31.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:31 smithi078 systemd[1]: Stopping Ceph mon.a for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:48:31.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:31 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a[30946]: 2024-09-11T00:48:31.731+0000 7f7167a6b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:48:31.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:31 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a[30946]: 2024-09-11T00:48:31.731+0000 7f7167a6b640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T00:48:31.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:31 smithi078 podman[31192]: 2024-09-11 00:48:31.87554799 +0000 UTC m=+0.270224427 container died d4d08c58261ffd727fb50b0201109802b0d615ec91923a52d94506618984e697 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T00:48:32.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:32 smithi078 podman[31192]: 2024-09-11 00:48:32.23201656 +0000 UTC m=+0.626692994 container cleanup d4d08c58261ffd727fb50b0201109802b0d615ec91923a52d94506618984e697 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T00:48:32.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:32 smithi078 bash[31192]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a 2024-09-11T00:48:32.857 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:32 smithi078 podman[31204]: 2024-09-11 00:48:32.499654154 +0000 UTC m=+0.621114602 container remove d4d08c58261ffd727fb50b0201109802b0d615ec91923a52d94506618984e697 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T00:48:32.857 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:32 smithi078 systemd[1]: ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T00:48:32.857 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:32 smithi078 systemd[1]: Stopped Ceph mon.a for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:48:32.857 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:32 smithi078 systemd[1]: Starting Ceph mon.a for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:48:33.111 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:32 smithi078 podman[31297]: 2024-09-11 00:48:32.759547068 +0000 UTC m=+0.015967032 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:48:33.111 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:32 smithi078 podman[31297]: 2024-09-11 00:48:32.879911289 +0000 UTC m=+0.136331245 container create ac3d670696112fa5f07d8702c5ee804bbd03a8f059c1b9ce8258465df86d2935 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T00:48:33.111 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 podman[31297]: 2024-09-11 00:48:33.085952142 +0000 UTC m=+0.342372101 container init ac3d670696112fa5f07d8702c5ee804bbd03a8f059c1b9ce8258465df86d2935 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:48:33.111 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 podman[31297]: 2024-09-11 00:48:33.089118393 +0000 UTC m=+0.345538352 container start ac3d670696112fa5f07d8702c5ee804bbd03a8f059c1b9ce8258465df86d2935 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:48:33.111 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T00:48:33.111 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-11T00:48:33.112 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: pidfile_write: ignore empty --pid-file 2024-09-11T00:48:33.367 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: load: jerasure load: lrc 2024-09-11T00:48:33.367 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: RocksDB version: 7.9.2 2024-09-11T00:48:33.367 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Git sha 0 2024-09-11T00:48:33.367 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-11T00:48:33.367 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: DB SUMMARY 2024-09-11T00:48:33.367 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: DB Session ID: 6CT0EH4OEZ1EFK6W7S4K 2024-09-11T00:48:33.367 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: CURRENT file: CURRENT 2024-09-11T00:48:33.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T00:48:33.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-11T00:48:33.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T00:48:33.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95684 ; 2024-09-11T00:48:33.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.error_if_exists: 0 2024-09-11T00:48:33.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.create_if_missing: 0 2024-09-11T00:48:33.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.paranoid_checks: 1 2024-09-11T00:48:33.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-11T00:48:33.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T00:48:33.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-11T00:48:33.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.env: 0x5573ca78b180 2024-09-11T00:48:33.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.fs: PosixFileSystem 2024-09-11T00:48:33.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.info_log: 0x5573cb09f700 2024-09-11T00:48:33.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T00:48:33.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.statistics: (nil) 2024-09-11T00:48:33.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.use_fsync: 0 2024-09-11T00:48:33.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_log_file_size: 0 2024-09-11T00:48:33.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T00:48:33.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T00:48:33.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T00:48:33.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T00:48:33.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.allow_fallocate: 1 2024-09-11T00:48:33.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T00:48:33.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T00:48:33.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.use_direct_reads: 0 2024-09-11T00:48:33.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T00:48:33.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T00:48:33.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.db_log_dir: 2024-09-11T00:48:33.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.wal_dir: 2024-09-11T00:48:33.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T00:48:33.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T00:48:33.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T00:48:33.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T00:48:33.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T00:48:33.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T00:48:33.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T00:48:33.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T00:48:33.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.write_buffer_manager: 0x5573cb0a3b80 2024-09-11T00:48:33.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T00:48:33.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T00:48:33.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T00:48:33.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T00:48:33.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T00:48:33.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T00:48:33.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T00:48:33.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T00:48:33.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.unordered_write: 0 2024-09-11T00:48:33.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T00:48:33.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T00:48:33.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T00:48:33.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T00:48:33.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.row_cache: None 2024-09-11T00:48:33.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.wal_filter: None 2024-09-11T00:48:33.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T00:48:33.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T00:48:33.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.two_write_queues: 0 2024-09-11T00:48:33.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T00:48:33.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.wal_compression: 0 2024-09-11T00:48:33.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.atomic_flush: 0 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.log_readahead_size: 0 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_background_jobs: 2 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_background_compactions: -1 2024-09-11T00:48:33.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_subcompactions: 1 2024-09-11T00:48:33.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T00:48:33.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T00:48:33.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T00:48:33.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T00:48:33.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T00:48:33.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T00:48:33.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T00:48:33.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T00:48:33.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_open_files: -1 2024-09-11T00:48:33.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T00:48:33.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_background_flushes: -1 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Compression algorithms supported: 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: kZSTD supported: 0 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: kXpressCompression supported: 0 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: kBZip2Compression supported: 0 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: kLZ4Compression supported: 1 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: kZlibCompression supported: 1 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: kSnappyCompression supported: 1 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T00:48:33.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.merge_operator: 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_filter: None 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5573cb09f320) 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks: 1 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T00:48:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_top_level_index_and_filter: 1 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_type: 0 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_index_type: 0 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_shortening: 1 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: checksum: 4 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: no_block_cache: 0 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache: 0x5573cb093610 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_name: BinnedLRUCache 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_options: 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: capacity : 536870912 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_shard_bits : 4 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: strict_capacity_limit : 0 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: high_pri_pool_ratio: 0.000 2024-09-11T00:48:33.379 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_compressed: (nil) 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: persistent_cache: (nil) 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size: 4096 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size_deviation: 10 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_restart_interval: 16 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_block_restart_interval: 1 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: metadata_block_size: 4096 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: partition_filters: 0 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: use_delta_encoding: 1 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: filter_policy: bloomfilter 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: whole_key_filtering: 1 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: verify_compression: 0 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: read_amp_bytes_per_bit: 0 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: format_version: 5 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_index_compression: 1 2024-09-11T00:48:33.380 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_align: 0 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout: max_auto_readahead_size: 262144 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout: prepopulate_block_cache: 0 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout: initial_auto_readahead_size: 8192 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_file_reads_for_auto_readahead: 2 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compression: NoCompression 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.num_levels: 7 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T00:48:33.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-11T00:48:33.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T00:48:33.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T00:48:33.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T00:48:33.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T00:48:33.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T00:48:33.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T00:48:33.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T00:48:33.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T00:48:33.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-11T00:48:33.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.arena_block_size: 1048576 2024-09-11T00:48:33.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T00:48:33.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T00:48:33.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T00:48:33.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.inplace_update_support: 0 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T00:48:33.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.bloom_locality: 0 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.max_successive_merges: 0 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.ttl: 2592000 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.enable_blob_files: false 2024-09-11T00:48:33.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.min_blob_size: 0 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.blob_file_starting_level: 0 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a8451cd4-99af-47d7-b470-1f6fa06d54ba 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726015713115338, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-11T00:48:33.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-11T00:48:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726015713116293, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91820, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 287, "table_properties": {"data_size": 89861, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11819, "raw_average_key_size": 48, "raw_value_size": 83182, "raw_average_value_size": 338, "num_data_blocks": 12, "num_entries": 246, "num_filter_entries": 246, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726015713, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a8451cd4-99af-47d7-b470-1f6fa06d54ba", "db_session_id": "6CT0EH4OEZ1EFK6W7S4K", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-11T00:48:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726015713116385, "job": 1, "event": "recovery_finished"} 2024-09-11T00:48:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-11T00:48:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T00:48:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5573cb0c2e00 2024-09-11T00:48:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: DB pointer 0x5573cb1c4000 2024-09-11T00:48:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-11T00:48:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-11T00:48:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** DB Stats ** 2024-09-11T00:48:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T00:48:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T00:48:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: L0 2/0 91.53 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: Sum 2/0 91.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-09-11T00:48:33.389 INFO:journalctl@ceph.mon.a.smithi078.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 137.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative compaction: 0.00 GB write, 29.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval compaction: 0.00 GB write, 29.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T00:48:33.390 INFO:journalctl@ceph.mon.a.smithi078.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache BinnedLRUCache@0x5573cb093610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: starting mon.a rank 0 at public addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] at bind addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: mon.a@-1(???) e1 preinit fsid 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: mon.a@-1(???).mds e1 new map 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: mon.a@-1(???).mds e1 print_map 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout: e1 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout: btime 2024-09-11T00:48:27:650665+0000 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T00:48:33.391 INFO:journalctl@ceph.mon.a.smithi078.stdout: legacy client fscid: -1 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout: No filesystems configured 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: monmap epoch 1 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: fsid 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: last_changed 2024-09-11T00:48:16.201306+0000 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: created 2024-09-11T00:48:16.201306+0000 2024-09-11T00:48:33.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: min_mon_release 19 (squid) 2024-09-11T00:48:33.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: election_strategy: 1 2024-09-11T00:48:33.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-11T00:48:33.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: fsmap 2024-09-11T00:48:33.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T00:48:33.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 ceph-mon[31315]: mgrmap e1: no daemons active 2024-09-11T00:48:33.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 bash[31297]: ac3d670696112fa5f07d8702c5ee804bbd03a8f059c1b9ce8258465df86d2935 2024-09-11T00:48:33.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:33 smithi078 systemd[1]: Started Ceph mon.a for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:48:33.393 INFO:teuthology.orchestra.run.smithi078.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T00:48:36.481 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T00:48:36.501 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:48:36.502 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mgr... 2024-09-11T00:48:36.502 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T00:48:36.502 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T00:48:36.741 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.x 2024-09-11T00:48:36.742 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-11T00:48:36.894 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-64384694-6fd7-11ef-bcea-c7b262605968.target.wants/ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-64384694-6fd7-11ef-bcea-c7b262605968@.service. 2024-09-11T00:48:37.124 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:36 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2957938376' entity='client.admin' 2024-09-11T00:48:37.125 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:36 smithi078 systemd[1]: Starting Ceph mgr.x for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:48:37.398 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:37 smithi078 podman[31537]: 2024-09-11 00:48:37.025340344 +0000 UTC m=+0.015612831 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:48:37.398 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:37 smithi078 podman[31537]: 2024-09-11 00:48:37.127974674 +0000 UTC m=+0.118247162 container create 41c1f890f76fda0403489d1a9933d9a43da4740931e4e909e455a433eee2af39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T00:48:37.399 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:37 smithi078 podman[31537]: 2024-09-11 00:48:37.338379793 +0000 UTC m=+0.328652281 container init 41c1f890f76fda0403489d1a9933d9a43da4740931e4e909e455a433eee2af39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-11T00:48:37.399 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:37 smithi078 podman[31537]: 2024-09-11 00:48:37.341194659 +0000 UTC m=+0.331467147 container start 41c1f890f76fda0403489d1a9933d9a43da4740931e4e909e455a433eee2af39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T00:48:37.399 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:37 smithi078 bash[31537]: 41c1f890f76fda0403489d1a9933d9a43da4740931e4e909e455a433eee2af39 2024-09-11T00:48:37.412 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T00:48:37.412 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-09-11T00:48:37.420 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T00:48:37.420 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-09-11T00:48:37.420 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-09-11T00:48:37.420 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T00:48:37.429 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T00:48:37.429 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-09-11T00:48:37.437 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T00:48:37.437 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-09-11T00:48:37.437 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-09-11T00:48:37.437 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T00:48:37.437 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr to start... 2024-09-11T00:48:37.437 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr... 2024-09-11T00:48:37.657 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:37 smithi078 systemd[1]: Started Ceph mgr.x for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:48:38.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:37 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2540654593' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-11T00:48:38.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-11T00:48:38.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:48:38.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:48:38.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T00:48:38.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T00:48:38.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T00:48:38.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T00:48:38.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:38.628 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T00:48:38.628 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T00:48:38.628 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-09-11T00:48:38.628 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:48:38.628 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T00:48:38.629 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T00:48:38.629 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:48:38.629 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-09-11T00:48:38.629 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T00:48:38.629 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:48:38.629 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T00:48:38.630 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T00:48:38.630 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:38.630 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T00:48:38.630 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:48:38.630 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T00:48:38.630 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T00:48:38.631 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T00:48:38.631 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T00:48:38.631 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T00:48:38.631 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T00:48:38.631 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:38.631 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T00:48:38.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T00:48:38.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T00:48:38.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T00:48:38.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T00:48:38.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T00:48:38.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T00:48:38.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T00:48:38.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T00:48:38.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:38.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T00:48:38.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:48:38.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T00:48:27:650665+0000", 2024-09-11T00:48:38.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T00:48:38.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T00:48:38.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:38.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T00:48:38.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T00:48:38.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T00:48:38.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T00:48:38.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T00:48:38.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T00:48:38.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T00:48:38.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:48:38.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T00:48:38.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:38.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T00:48:38.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:48:38.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T00:48:27.653040+0000", 2024-09-11T00:48:38.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T00:48:38.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:38.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T00:48:38.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:48:38.637 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (1/15)... 2024-09-11T00:48:41.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:41 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2177391020' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-11T00:48:41.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-11T00:48:41.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:48:41.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:48:41.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T00:48:41.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T00:48:41.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T00:48:41.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T00:48:41.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T00:48:41.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:41.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T00:48:41.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:48:41.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T00:48:41.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T00:48:41.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T00:48:41.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T00:48:41.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T00:48:41.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T00:48:41.905 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T00:48:27:650665+0000", 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:41.906 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T00:48:41.907 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:48:41.907 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T00:48:27.653040+0000", 2024-09-11T00:48:41.907 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T00:48:41.907 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:41.907 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T00:48:41.907 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:48:41.907 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (2/15)... 2024-09-11T00:48:42.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: Activating manager daemon x 2024-09-11T00:48:42.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: mgrmap e2: x(active, starting, since 0.00334113s) 2024-09-11T00:48:42.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: from='mgr.14100 172.21.15.78:0/1765869866' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-11T00:48:42.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: from='mgr.14100 172.21.15.78:0/1765869866' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-11T00:48:42.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: from='mgr.14100 172.21.15.78:0/1765869866' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-11T00:48:42.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: from='mgr.14100 172.21.15.78:0/1765869866' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:48:42.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: from='mgr.14100 172.21.15.78:0/1765869866' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-11T00:48:42.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: Manager daemon x is now available 2024-09-11T00:48:42.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: from='mgr.14100 172.21.15.78:0/1765869866' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-11T00:48:42.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: from='mgr.14100 172.21.15.78:0/1765869866' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-11T00:48:42.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: from='mgr.14100 172.21.15.78:0/1765869866' entity='mgr.x' 2024-09-11T00:48:42.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: from='mgr.14100 172.21.15.78:0/1765869866' entity='mgr.x' 2024-09-11T00:48:42.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:42 smithi078 ceph-mon[31315]: from='mgr.14100 172.21.15.78:0/1765869866' entity='mgr.x' 2024-09-11T00:48:44.558 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:44 smithi078 ceph-mon[31315]: mgrmap e3: x(active, since 1.00702s) 2024-09-11T00:48:45.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-11T00:48:45.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:48:45.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:48:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T00:48:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T00:48:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T00:48:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T00:48:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T00:48:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T00:48:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-09-11T00:48:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:48:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T00:48:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T00:48:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T00:48:45.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T00:48:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T00:48:27:650665+0000", 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:45.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T00:48:45.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:48:45.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T00:48:27.653040+0000", 2024-09-11T00:48:45.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T00:48:45.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:48:45.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T00:48:45.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:48:45.179 INFO:teuthology.orchestra.run.smithi078.stdout:mgr is available 2024-09-11T00:48:45.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:45 smithi078 ceph-mon[31315]: mgrmap e4: x(active, since 2s) 2024-09-11T00:48:45.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:45 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4271211333' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-11T00:48:46.390 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:46 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2327502037' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-11T00:48:46.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-11T00:48:46.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T00:48:46.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:46.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T00:48:46.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-09-11T00:48:46.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T00:48:46.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T00:48:46.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T00:48:46.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T00:48:46.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-11T00:48:46.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T00:48:46.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T00:48:46.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-11T00:48:46.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T00:48:46.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T00:48:46.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T00:48:46.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T00:48:46.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T00:48:46.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T00:48:46.395 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling cephadm module... 2024-09-11T00:48:47.645 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:47 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1199763874' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-11T00:48:47.645 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:47 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: ignoring --setuser ceph since I am not root 2024-09-11T00:48:47.646 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:47 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: ignoring --setgroup ceph since I am not root 2024-09-11T00:48:48.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:48 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1199763874' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T00:48:48.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:48 smithi078 ceph-mon[31315]: mgrmap e5: x(active, since 5s) 2024-09-11T00:48:49.113 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:48:49.113 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T00:48:49.113 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T00:48:49.113 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T00:48:49.113 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T00:48:49.113 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:48:49.114 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-09-11T00:48:49.114 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 5... 2024-09-11T00:48:49.437 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:49 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3270924219' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-11T00:48:52.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:52 smithi078 ceph-mon[31315]: Active manager daemon x restarted 2024-09-11T00:48:52.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:52 smithi078 ceph-mon[31315]: Activating manager daemon x 2024-09-11T00:48:52.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:52 smithi078 ceph-mon[31315]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T00:48:52.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:52 smithi078 ceph-mon[31315]: mgrmap e6: x(active, starting, since 0.00328379s) 2024-09-11T00:48:52.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:52 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:48:52.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:52 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-11T00:48:52.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:52 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-11T00:48:52.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:52 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-11T00:48:52.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:52 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-11T00:48:52.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:52 smithi078 ceph-mon[31315]: Manager daemon x is now available 2024-09-11T00:48:53.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:48:53.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T00:48:53.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T00:48:53.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:48:53.902 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 5 is available 2024-09-11T00:48:53.902 INFO:teuthology.orchestra.run.smithi078.stdout:Setting orchestrator backend to cephadm... 2024-09-11T00:48:54.285 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:48:54.285 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:48:54.285 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-mon[31315]: Found migration_current of "None". Setting to last migration. 2024-09-11T00:48:54.285 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:48:54.285 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:48:54.285 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:48:54.285 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:48:54.286 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-11T00:48:54.286 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-11T00:48:54.286 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-mon[31315]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T00:48:54.286 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-mon[31315]: mgrmap e7: x(active, since 1.00648s) 2024-09-11T00:48:54.286 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-mon[31315]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T00:48:54.747 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:48:54.379+0000 7f90b0369640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:00:48:54] ENGINE Error in HTTPServer.serve 2024-09-11T00:48:54.747 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: Traceback (most recent call last): 2024-09-11T00:48:54.747 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T00:48:54.747 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self._connections.run(self.expiration_interval) 2024-09-11T00:48:54.747 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T00:48:54.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self._run(expiration_interval) 2024-09-11T00:48:54.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T00:48:54.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T00:48:54.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T00:48:54.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T00:48:54.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T00:48:54.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: s = self.context.wrap_socket( 2024-09-11T00:48:54.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T00:48:54.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: return self.sslsocket_class._create( 2024-09-11T00:48:54.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T00:48:54.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self.do_handshake() 2024-09-11T00:48:54.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T00:48:54.749 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self._sslobj.do_handshake() 2024-09-11T00:48:54.749 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T00:48:54.749 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:48:55.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:55 smithi078 ceph-mon[31315]: [11/Sep/2024:00:48:54] ENGINE Bus STARTING 2024-09-11T00:48:55.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:55 smithi078 ceph-mon[31315]: [11/Sep/2024:00:48:54] ENGINE Serving on http://172.21.15.78:8765 2024-09-11T00:48:55.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:55 smithi078 ceph-mon[31315]: [11/Sep/2024:00:48:54] ENGINE Serving on https://172.21.15.78:7150 2024-09-11T00:48:55.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:55 smithi078 ceph-mon[31315]: [11/Sep/2024:00:48:54] ENGINE Bus STARTED 2024-09-11T00:48:55.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:55 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:55 smithi078 ceph-mon[31315]: [11/Sep/2024:00:48:54] ENGINE Error in HTTPServer.serve 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout: Traceback (most recent call last): 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._connections.run(self.expiration_interval) 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._run(expiration_interval) 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout: s = self.context.wrap_socket( 2024-09-11T00:48:55.417 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T00:48:55.418 INFO:journalctl@ceph.mon.a.smithi078.stdout: return self.sslsocket_class._create( 2024-09-11T00:48:55.418 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T00:48:55.418 INFO:journalctl@ceph.mon.a.smithi078.stdout: self.do_handshake() 2024-09-11T00:48:55.418 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T00:48:55.418 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._sslobj.do_handshake() 2024-09-11T00:48:55.418 INFO:journalctl@ceph.mon.a.smithi078.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T00:48:55.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:55 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:48:55.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:55 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:48:56.152 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-mon[31315]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:48:56.152 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-mon[31315]: mgrmap e8: x(active, since 2s) 2024-09-11T00:48:56.155 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T00:48:56.155 INFO:teuthology.orchestra.run.smithi078.stdout:Generating ssh key... 2024-09-11T00:48:57.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:57 smithi078 ceph-mon[31315]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:48:57.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:57 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:48:57.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:57 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:48:57.233 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: Generating public/private rsa key pair. 2024-09-11T00:48:57.233 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: Your identification has been saved in /tmp/tmpybyb1l51/key 2024-09-11T00:48:57.233 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: Your public key has been saved in /tmp/tmpybyb1l51/key.pub 2024-09-11T00:48:57.233 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: The key fingerprint is: 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: SHA256:FGcnDiiXFC+RXiZLN2OIMFcjbiyLUtheCSvctArj1EM ceph-64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: The key's randomart image is: 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: +---[RSA 3072]----+ 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: | +Eo+**o + . | 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: |.o=B+oO+B* o | 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: |=o==*=.Boo. | 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: |+*.=. oo | 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: |ooo S | 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: |. | 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: | | 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: | | 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: | | 2024-09-11T00:48:57.234 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:48:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: +----[SHA256]-----+ 2024-09-11T00:48:58.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:58 smithi078 ceph-mon[31315]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:48:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:58 smithi078 ceph-mon[31315]: Generating ssh key... 2024-09-11T00:48:58.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBpAPiYjVA/gO58KL2vp8F1U1gFCNe3NQdLH7vGfk3V3FacP/n3Tp6VyZ9a+oKB6BuQbAUE7hul14L9PEsLXEk6ClfbDJsVnhZ5eAHKneQD8SLK0QfTM5+O0ir496RYP3n+PQsLcsm+AmjhSP7JvNHCyHjT6mKVjQKAM36pql1jWkH3mdol8gArHH9g9STf7gsij8Bwb/dAtVoUQUPu0MUeTsa/f8/eHayQw6AVNCkLnaOYBUuNZJ6grCyc5WGjyXuOnRvJkARaCeeFtNiN8KDKYzDO8QcAUh6Q4/E7TYq3HLDEuPdNJqO8CQeaC763hlyCITh2jWnA2XqmVnhTbXIs3p9Qq6HqVnYDxGEVvkMD7aygn6vMgWVQt9wAgsg8PvSBdFh0RIuG/Z3MK0W1/l4E+mbCFCQM15Bk18hR3TO/xfQyoCvq+jq6vGipGJHwVhbPKytq7C7UCdbbEZlMaFek1lUYFcYntmEy1b2zitROGxBvHsC0TlJj0QPlCecmyk= ceph-64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:48:58.700 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T00:48:58.700 INFO:teuthology.orchestra.run.smithi078.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T00:48:58.705 INFO:teuthology.orchestra.run.smithi078.stdout:Adding host smithi078... 2024-09-11T00:48:59.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:48:59 smithi078 ceph-mon[31315]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:49:00.329 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:00 smithi078 ceph-mon[31315]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "addr": "172.21.15.78", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:49:01.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:01 smithi078 ceph-mon[31315]: Deploying cephadm binary to smithi078 2024-09-11T00:49:03.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Added host 'smithi078' with addr '172.21.15.78' 2024-09-11T00:49:03.050 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mon service... 2024-09-11T00:49:03.659 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:03 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:49:03.659 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:03 smithi078 ceph-mon[31315]: Added host smithi078 2024-09-11T00:49:03.659 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:03 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:49:04.628 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T00:49:04.628 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mgr service... 2024-09-11T00:49:04.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:04 smithi078 ceph-mon[31315]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:49:04.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:04 smithi078 ceph-mon[31315]: Saving service mon spec with placement count:5 2024-09-11T00:49:04.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:04 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:49:05.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T00:49:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:06 smithi078 ceph-mon[31315]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:49:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:06 smithi078 ceph-mon[31315]: Saving service mgr spec with placement count:2 2024-09-11T00:49:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:06 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:49:07.705 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:07 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1627057828' entity='client.admin' 2024-09-11T00:49:07.705 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:07 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:49:08.645 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling the dashboard module... 2024-09-11T00:49:09.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:08 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:49:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:08 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3859652858' entity='client.admin' 2024-09-11T00:49:10.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:09 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1144183990' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-11T00:49:10.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:09 smithi078 ceph-mon[31315]: from='mgr.14118 172.21.15.78:0/2510322346' entity='mgr.x' 2024-09-11T00:49:10.247 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:10 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: ignoring --setuser ceph since I am not root 2024-09-11T00:49:10.247 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:10 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: ignoring --setgroup ceph since I am not root 2024-09-11T00:49:11.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:10 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1144183990' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T00:49:11.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:10 smithi078 ceph-mon[31315]: mgrmap e9: x(active, since 17s) 2024-09-11T00:49:11.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:49:11.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T00:49:11.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T00:49:11.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T00:49:11.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T00:49:11.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:49:11.947 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-09-11T00:49:11.948 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 9... 2024-09-11T00:49:12.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:11 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3242196047' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-11T00:49:15.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:14 smithi078 ceph-mon[31315]: Active manager daemon x restarted 2024-09-11T00:49:15.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:14 smithi078 ceph-mon[31315]: Activating manager daemon x 2024-09-11T00:49:15.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:14 smithi078 ceph-mon[31315]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T00:49:15.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:14 smithi078 ceph-mon[31315]: mgrmap e10: x(active, starting, since 0.00580515s) 2024-09-11T00:49:15.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:14 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:49:15.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:14 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-11T00:49:15.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:14 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-11T00:49:15.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:14 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-11T00:49:15.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:14 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-11T00:49:15.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:14 smithi078 ceph-mon[31315]: Manager daemon x is now available 2024-09-11T00:49:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:15 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:49:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:15 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-11T00:49:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:15 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-11T00:49:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:15 smithi078 ceph-mon[31315]: mgrmap e11: x(active, since 1.00628s) 2024-09-11T00:49:16.619 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:49:16.336+0000 7fee47d15640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:00:49:16] ENGINE Error in HTTPServer.serve 2024-09-11T00:49:16.619 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: Traceback (most recent call last): 2024-09-11T00:49:16.619 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T00:49:16.619 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self._connections.run(self.expiration_interval) 2024-09-11T00:49:16.619 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T00:49:16.619 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self._run(expiration_interval) 2024-09-11T00:49:16.619 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T00:49:16.619 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T00:49:16.619 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T00:49:16.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T00:49:16.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T00:49:16.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: s = self.context.wrap_socket( 2024-09-11T00:49:16.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T00:49:16.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: return self.sslsocket_class._create( 2024-09-11T00:49:16.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T00:49:16.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self.do_handshake() 2024-09-11T00:49:16.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T00:49:16.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self._sslobj.do_handshake() 2024-09-11T00:49:16.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T00:49:16.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:49:16.621 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:49:16.622 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T00:49:16.622 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T00:49:16.622 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:49:16.622 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 9 is available 2024-09-11T00:49:16.622 INFO:teuthology.orchestra.run.smithi078.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-11T00:49:17.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-mon[31315]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T00:49:17.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:16 smithi078 ceph-mon[31315]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T00:49:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:17 smithi078 ceph-mon[31315]: [11/Sep/2024:00:49:16] ENGINE Bus STARTING 2024-09-11T00:49:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:17 smithi078 ceph-mon[31315]: [11/Sep/2024:00:49:16] ENGINE Serving on http://172.21.15.78:8765 2024-09-11T00:49:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:17 smithi078 ceph-mon[31315]: [11/Sep/2024:00:49:16] ENGINE Serving on https://172.21.15.78:7150 2024-09-11T00:49:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:17 smithi078 ceph-mon[31315]: [11/Sep/2024:00:49:16] ENGINE Bus STARTED 2024-09-11T00:49:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:17 smithi078 ceph-mon[31315]: [11/Sep/2024:00:49:16] ENGINE Error in HTTPServer.serve 2024-09-11T00:49:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout: Traceback (most recent call last): 2024-09-11T00:49:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T00:49:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._connections.run(self.expiration_interval) 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._run(expiration_interval) 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: s = self.context.wrap_socket( 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: return self.sslsocket_class._create( 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: self.do_handshake() 2024-09-11T00:49:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T00:49:18.249 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._sslobj.do_handshake() 2024-09-11T00:49:18.249 INFO:journalctl@ceph.mon.a.smithi078.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T00:49:18.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:17 smithi078 ceph-mon[31315]: mgrmap e12: x(active, since 2s) 2024-09-11T00:49:18.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:17 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:18.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:17 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:18.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFEDCCAvigAwIBAgIUXhGxVxFCFikO56BPF2kirYtdYhcwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxMTAwNDkxOFoXDTM0\nMDkxMjAwNDkxOFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1Ljc4MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEA+fSj1jUbj6BkpPsYSQAk+hShy1X16DS6zmkP\nQZ7Y62NyOxN94TYqm9nmidkLX7EDT7RcIA3Zy/qUBdiR+MuFDw/E5AepX0j9/eri\nuu8SpcWawLhx1KGJdpBMsdgXHK//R8nvdM1evhPNBJaYnWb+N+7xdQ8wHtKyX0t1\nE2YQhsSO/YJ2S4knJ5gMsiy6WUoOcNN139q0kMloHnWx66klcijUEkuLX5lSh8MG\nwEg3uC9hAE50azxB4pc9NpPCC4agdpzY+4FQ5/Vrt8KRms9ae/UhVcBfv9dKHlky\nM7suoPTSko35iIkbR6zPvN0nt0dtLXgSf16EBG2BFPQx733SPPWcayCO/G78TyaM\nXi97OsXCOWfBb1YR+Beq29/so630LWPfgO/idjvxCAYAN6OosJkbLDNwh3aGfqld\nEOi2c3/U78iuuk34WElAhw001QJGZFXhvZsbuNC5BvmzHsPHRlyrPN7NftaeaXnS\nscRHkL3zvBjVXAQPMTOPI0SrmObSFuj+vMQkIGQ0W8qmBrlbMINlJHJl1D6li1YW\noUBjFARHQdL6iiNvoMy0jL/DA6pt5hmeTPqqZec9NKp3xv+PvlnKLM3GwAQw8+IQ\n9xbNEO+z2qky7ff8o2h9K2QSD+Uul0yTU2hzx/Lqd6W05SlWZ0kOxyIvoWbKgXeN\nFQPM6tsCAwEAAaNUMFIwQgYDVR0RBDswOYIec21pdGhpMDc4LmZyb250LnNlcGlh\nLmNlcGguY29tghFkYXNoYm9hcmRfc2VydmVyc4cErBUPTjAMBgNVHRMBAf8EAjAA\nMA0GCSqGSIb3DQEBCwUAA4ICAQACXnCW7Waurom5Kn9TY6sriirMi3LgOolznv6M\n/fAdMzW4TPOOArdTT/PbCVkKSabTZJ1gqYgKnqzhDO+0zpZ7qY6ELq9j4u6eDBvJ\nFpyzv0FYjcUgztaJBWaHGl7HRURiKzAiJW0oaGsOb0FYL53x9NcjcPYgJC3k6aCW\nYjxjs+PA5TTlvsFGnzByKwIR8JfUcVxcyS4t2pNfCj19+5h8nTrfH3b0HQyACdPT\nLp76Oh0v2Up16xK9ttJv1KqE6WSabCZ8TIji68EBAcaWAiyRoifocFNb1BXmp/fN\neDUlAByvaLKqn25xfNbTtilLjovGlYsPrmr+TIIbZbwCJZNPkZSdYv07A/4s02ge\nAIc7HOQrhOk5y94zrp+9DKnp4AaIaEhsY/Hdu2rLQLmHv6w6GPitMeTJKwiyfw5i\nZgtSsdNA5RgLP/mjGW0JcCEls71BPAsJwRwN4okh93+LtVuojY45hJQq3GCyTADn\nBA/3h+j8oV5+BmIBNe3AjpZa70iFTTyN/hdeGOWGZDiXXfZwm7VPeRzcKh55lJgV\nJRlKh/sOKdlbbMybZYFxqcbc28cN3ntLTfuAslgE34FlDcB/E5UGnmsR7qwy9LKB\nPZm0mvyZECGzVWmYe1IHuv2OYaOY1iclPUX0jVnt06wLzkH9++82g7orUEpGmE4l\n3pC9lw==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEA+fSj1jUbj6BkpPsYSQAk+hShy1X16DS6zmkPQZ7Y62NyOxN9\n4TYqm9nmidkLX7EDT7RcIA3Zy/qUBdiR+MuFDw/E5AepX0j9/eriuu8SpcWawLhx\n1KGJdpBMsdgXHK//R8nvdM1evhPNBJaYnWb+N+7xdQ8wHtKyX0t1E2YQhsSO/YJ2\nS4knJ5gMsiy6WUoOcNN139q0kMloHnWx66klcijUEkuLX5lSh8MGwEg3uC9hAE50\nazxB4pc9NpPCC4agdpzY+4FQ5/Vrt8KRms9ae/UhVcBfv9dKHlkyM7suoPTSko35\niIkbR6zPvN0nt0dtLXgSf16EBG2BFPQx733SPPWcayCO/G78TyaMXi97OsXCOWfB\nb1YR+Beq29/so630LWPfgO/idjvxCAYAN6OosJkbLDNwh3aGfqldEOi2c3/U78iu\nuk34WElAhw001QJGZFXhvZsbuNC5BvmzHsPHRlyrPN7NftaeaXnSscRHkL3zvBjV\nXAQPMTOPI0SrmObSFuj+vMQkIGQ0W8qmBrlbMINlJHJl1D6li1YWoUBjFARHQdL6\niiNvoMy0jL/DA6pt5hmeTPqqZec9NKp3xv+PvlnKLM3GwAQw8+IQ9xbNEO+z2qky\n7ff8o2h9K2QSD+Uul0yTU2hzx/Lqd6W05SlWZ0kOxyIvoWbKgXeNFQPM6tsCAwEA\nAQKCAgBGDY3jaZP1X3cFlAf84aj7W8/ynH9kWm7mjaREeGAELKVlsDNN7K/paRsH\nFEjvQz2R15MgOrGJTQuyTb2vdF6w13MPk6Qg++SPhwaPbJSV2DlkfO0OMk3D6HJO\nP6lOlXbe6FYPrsU4UXVarVhNeQGTrIiIjb6nTNbIBW5lBuxn0OvgWfGMfv51xtP8\nFbQOdUZx0ZoOKI4FrCTdsI+yZAAGLDIvhHOl0eNDAGJCfpr7KHAvtU1EhqlSd5+k\nzamLMGqFg7T+XAJQWESdxk6njCioaA21KkrLieuQcOA0hmftnV/syhSe8wotvt2j\n6XmJkN+k6CuK/fLTCNuuAziBjgpfFIx4bFlPggTpjJIu+k+l9E6jNI6IbBVP3gzG\npXdiwrG71ky1GY0N0Xlw0WskQzLcPW6y6MsC+23JRqomH8Lk/XdOFVsvW9xTd21j\nifvq0gwttvtqY/DnRieB1SEEuj+zazqBCX0Gxw0GU5OowXVpVIV7OjTbwXQDh+6B\ncOqdGL/K3rvX63sesAYvK6fTovkJ6neodRdE2FVWmLr46H1RL94Y1KIO4jrTVA70\nCpnQcPX9sl4dLz5zKIY1ei5V0z5/kSI78s89AOSFb7jQvv/GgkyXzpJhOmst7YUw\nV21YL7o7wq6MBreqUjGs8n+CXGYbVJlSJ4aDopyxV6swPoGyAQKCAQEA/4HTOWJe\nXo7NBNufnVUmgh3kAphO7u01MtHkW1vN+kdGRUeCHten7dbkGE86Sao/uJYEwxj9\n4RND/O8Pr2Y0Cjb+sq9NsVL1urzMrSEhI2gocbjCkaRHqisDFYovm9W+420fftIR\nwXMimwdyVvGjdrMn3Fik4Z1Ux1nFGX9b21CvXN4SPO2I5q9keNOAlmH9+Ni0pJDc\n79ZdjNwEodwtBy/iQaW/SgXBE3JYfSEYmv2Is+uOKDjna0eL+/0EJjXbIP+WNgse\nTYPAgqF4UTZqyJeswZ9a08aIk6IGDpsx0qx/bDLzKN9w642nDOjo9wkcuEK3d6hn\nKMTE2p8AQzLGAQKCAQEA+nASzQYjNojRdwqZjzvop2ZDFA+gvwnHV4um5F2nC18K\nJPZucJBGVfko5jNPrTM1s1siW9Hg3oWDNnnwOJ+QSS/BwiNYeKnTmDXRJO0XTy9M\nd8WkRuusEHWqkeGIl46o+UgzsHGAlJU7jvdXhg8JMH6j1j1B+ilCjTntX0q732Mq\nQZJzeErcmuw05Dd02YiPz9wK8M/xQeoWI81IaI6kZ+YLj+zRM2nBMBgaophl8BgU\n3FTOCXekVto4/Ja74tTJGc+KvPU0LPjdGGRKLj7+ti3CnDS5jKse3uAmwP2koJGk\nEdKwlNOU4QxH8xl7AnRSDBn7roFNQGFZgt27wC2I2wKCAQAj5SI+gpJNaHRnBU+0\nM704NGjO8ITOTibWH+Sb7RbP3WmC3Bvsp1FhOXpL6/19Udyn2vOKUR2BnEpF43Pp\nqUsQUE+R3MzmZ4g8FsAOZoQYTE66Q0GaYmDrCCXjxckQdIPfwXnWQD7cVYd8DhzD\nGCllyJ+F9jIbvBWn9/WXsmu+MOSiAFTBmIFOHz/XNadv+Q5MAm3GXM+dJhl4a4R/\nFMaCpvh7ToTEKnDIgZzq31Zsl8e3txT+k5f3W1LQMOh/UbuaNC3vI7q7GxJVh8e2\naRWxXpWXkgKZk9+mFfIR5T2nojI4iiPdDx73CKx6RNJNukr9m+nYdqEfeh1JtWK9\nDgoBAoIBAAFB2MZQtpKKg0qusfCnI1bby5BpN6xd1zNYTyEsWy9SbdGbrxg/kSrT\nUmI9dbPv4uAgn/6IAjH7x5nZaNxP6MCFTdq0pLgOWjKUapVd31UVbrPXOSjl+/ol\nLRMIZ9Ai9jQyKZhEKZoVWbQxB65zmt80c4wayJ8B4ASnesqUrHbFn6uumYfMEGly\nsB1JLfv9lIM9noxw7swe6beKS0fDF44x+HLjhHVWnZKpPFD+pNxcspYckuA3Sorp\nE4opE45i9RN9GDw0muty4GFD959dz5eIiUaReTD8lu4Vf/xOCaqIsKMqRJqqqT+r\n+Rx+T0DG5raK1tmcao/dLT7b8BpZ4CsCggEBAOpoDLSR5EbjsNGxOl41iAGdmssf\nf1ub09jrGhCaGNRWO6J0O/u77iJoUYCxmOWGIRvePoeisivCuVMwqY+dFoi385j/\naqE5QgBKHpcW/kwNpMzqdSf8SQyX2q23A2NGWnIqQ/VI0QLh26L1kOKeg3Sz8b7X\nOc4Sv9IU1KgRi0sk5nLcTzY5KaHMj7z6LgCLE1IpAzzzlTMajwZ7bOyP8cAmN8tJ\nJ7jdWQa89NezMB8kHDPlYVdSTn3dWNuHGDUH0dGnMIrnMQmfEIlle+EiQ6Kqo16L\nuYsZkmc9cj4/7k3w2JuxIqjNaoavaNzUFQRwLE9tcqYrM1krPegaHAcfQ2Q=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-11T00:49:19.240 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:18 smithi078 ceph-mon[31315]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:49:20.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-11T00:49:20.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:20 smithi078 ceph-mon[31315]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:49:20.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:20 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:22.142 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-11T00:49:22.142 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial admin user... 2024-09-11T00:49:22.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:22 smithi078 ceph-mon[31315]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:49:22.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:22 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:23.573 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$EXui9o/TZSRLKWGwQoo7..q5mrSmuztGm.E/3dJRWKGTi0ACvpPd2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726015762, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T00:49:23.573 INFO:teuthology.orchestra.run.smithi078.stdout:Fetching dashboard port number... 2024-09-11T00:49:24.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:23 smithi078 ceph-mon[31315]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:49:24.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:23 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:24.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T00:49:24.756 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T00:49:24.756 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-09-11T00:49:24.763 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T00:49:24.763 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-09-11T00:49:24.763 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-09-11T00:49:24.763 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T00:49:24.764 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph Dashboard is now available at: 2024-09-11T00:49:24.764 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:49:24.765 INFO:teuthology.orchestra.run.smithi078.stdout: URL: https://smithi078.front.sepia.ceph.com:8443/ 2024-09-11T00:49:24.765 INFO:teuthology.orchestra.run.smithi078.stdout: User: admin 2024-09-11T00:49:24.765 INFO:teuthology.orchestra.run.smithi078.stdout: Password: vbxdjcl3pd 2024-09-11T00:49:24.765 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:49:24.765 INFO:teuthology.orchestra.run.smithi078.stdout:Saving cluster configuration to /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config directory 2024-09-11T00:49:25.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:24 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2847066944' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-11T00:49:26.045 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T00:49:26.045 INFO:teuthology.orchestra.run.smithi078.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T00:49:26.045 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:49:26.045 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:49:26.045 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:49:26.045 INFO:teuthology.orchestra.run.smithi078.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T00:49:26.045 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:49:26.045 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T00:49:26.046 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:49:26.046 INFO:teuthology.orchestra.run.smithi078.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T00:49:26.046 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:49:26.046 INFO:teuthology.orchestra.run.smithi078.stdout: ceph telemetry on 2024-09-11T00:49:26.046 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:49:26.046 INFO:teuthology.orchestra.run.smithi078.stdout:For more information see: 2024-09-11T00:49:26.046 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:49:26.046 INFO:teuthology.orchestra.run.smithi078.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-11T00:49:26.046 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:49:26.046 INFO:teuthology.orchestra.run.smithi078.stdout:Bootstrap complete. 2024-09-11T00:49:26.084 INFO:tasks.cephadm:Fetching config... 2024-09-11T00:49:26.084 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:49:26.084 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T00:49:26.100 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T00:49:26.100 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:49:26.100 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T00:49:26.156 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T00:49:26.156 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:49:26.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T00:49:26.220 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T00:49:26.220 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:49:26.220 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T00:49:26.275 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T00:49:26.276 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBpAPiYjVA/gO58KL2vp8F1U1gFCNe3NQdLH7vGfk3V3FacP/n3Tp6VyZ9a+oKB6BuQbAUE7hul14L9PEsLXEk6ClfbDJsVnhZ5eAHKneQD8SLK0QfTM5+O0ir496RYP3n+PQsLcsm+AmjhSP7JvNHCyHjT6mKVjQKAM36pql1jWkH3mdol8gArHH9g9STf7gsij8Bwb/dAtVoUQUPu0MUeTsa/f8/eHayQw6AVNCkLnaOYBUuNZJ6grCyc5WGjyXuOnRvJkARaCeeFtNiN8KDKYzDO8QcAUh6Q4/E7TYq3HLDEuPdNJqO8CQeaC763hlyCITh2jWnA2XqmVnhTbXIs3p9Qq6HqVnYDxGEVvkMD7aygn6vMgWVQt9wAgsg8PvSBdFh0RIuG/Z3MK0W1/l4E+mbCFCQM15Bk18hR3TO/xfQyoCvq+jq6vGipGJHwVhbPKytq7C7UCdbbEZlMaFek1lUYFcYntmEy1b2zitROGxBvHsC0TlJj0QPlCecmyk= ceph-64384694-6fd7-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T00:49:26.349 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBpAPiYjVA/gO58KL2vp8F1U1gFCNe3NQdLH7vGfk3V3FacP/n3Tp6VyZ9a+oKB6BuQbAUE7hul14L9PEsLXEk6ClfbDJsVnhZ5eAHKneQD8SLK0QfTM5+O0ir496RYP3n+PQsLcsm+AmjhSP7JvNHCyHjT6mKVjQKAM36pql1jWkH3mdol8gArHH9g9STf7gsij8Bwb/dAtVoUQUPu0MUeTsa/f8/eHayQw6AVNCkLnaOYBUuNZJ6grCyc5WGjyXuOnRvJkARaCeeFtNiN8KDKYzDO8QcAUh6Q4/E7TYq3HLDEuPdNJqO8CQeaC763hlyCITh2jWnA2XqmVnhTbXIs3p9Qq6HqVnYDxGEVvkMD7aygn6vMgWVQt9wAgsg8PvSBdFh0RIuG/Z3MK0W1/l4E+mbCFCQM15Bk18hR3TO/xfQyoCvq+jq6vGipGJHwVhbPKytq7C7UCdbbEZlMaFek1lUYFcYntmEy1b2zitROGxBvHsC0TlJj0QPlCecmyk= ceph-64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:49:26.368 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBpAPiYjVA/gO58KL2vp8F1U1gFCNe3NQdLH7vGfk3V3FacP/n3Tp6VyZ9a+oKB6BuQbAUE7hul14L9PEsLXEk6ClfbDJsVnhZ5eAHKneQD8SLK0QfTM5+O0ir496RYP3n+PQsLcsm+AmjhSP7JvNHCyHjT6mKVjQKAM36pql1jWkH3mdol8gArHH9g9STf7gsij8Bwb/dAtVoUQUPu0MUeTsa/f8/eHayQw6AVNCkLnaOYBUuNZJ6grCyc5WGjyXuOnRvJkARaCeeFtNiN8KDKYzDO8QcAUh6Q4/E7TYq3HLDEuPdNJqO8CQeaC763hlyCITh2jWnA2XqmVnhTbXIs3p9Qq6HqVnYDxGEVvkMD7aygn6vMgWVQt9wAgsg8PvSBdFh0RIuG/Z3MK0W1/l4E+mbCFCQM15Bk18hR3TO/xfQyoCvq+jq6vGipGJHwVhbPKytq7C7UCdbbEZlMaFek1lUYFcYntmEy1b2zitROGxBvHsC0TlJj0QPlCecmyk= ceph-64384694-6fd7-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T00:49:26.411 INFO:teuthology.orchestra.run.smithi110.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBpAPiYjVA/gO58KL2vp8F1U1gFCNe3NQdLH7vGfk3V3FacP/n3Tp6VyZ9a+oKB6BuQbAUE7hul14L9PEsLXEk6ClfbDJsVnhZ5eAHKneQD8SLK0QfTM5+O0ir496RYP3n+PQsLcsm+AmjhSP7JvNHCyHjT6mKVjQKAM36pql1jWkH3mdol8gArHH9g9STf7gsij8Bwb/dAtVoUQUPu0MUeTsa/f8/eHayQw6AVNCkLnaOYBUuNZJ6grCyc5WGjyXuOnRvJkARaCeeFtNiN8KDKYzDO8QcAUh6Q4/E7TYq3HLDEuPdNJqO8CQeaC763hlyCITh2jWnA2XqmVnhTbXIs3p9Qq6HqVnYDxGEVvkMD7aygn6vMgWVQt9wAgsg8PvSBdFh0RIuG/Z3MK0W1/l4E+mbCFCQM15Bk18hR3TO/xfQyoCvq+jq6vGipGJHwVhbPKytq7C7UCdbbEZlMaFek1lUYFcYntmEy1b2zitROGxBvHsC0TlJj0QPlCecmyk= ceph-64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:49:26.423 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBpAPiYjVA/gO58KL2vp8F1U1gFCNe3NQdLH7vGfk3V3FacP/n3Tp6VyZ9a+oKB6BuQbAUE7hul14L9PEsLXEk6ClfbDJsVnhZ5eAHKneQD8SLK0QfTM5+O0ir496RYP3n+PQsLcsm+AmjhSP7JvNHCyHjT6mKVjQKAM36pql1jWkH3mdol8gArHH9g9STf7gsij8Bwb/dAtVoUQUPu0MUeTsa/f8/eHayQw6AVNCkLnaOYBUuNZJ6grCyc5WGjyXuOnRvJkARaCeeFtNiN8KDKYzDO8QcAUh6Q4/E7TYq3HLDEuPdNJqO8CQeaC763hlyCITh2jWnA2XqmVnhTbXIs3p9Qq6HqVnYDxGEVvkMD7aygn6vMgWVQt9wAgsg8PvSBdFh0RIuG/Z3MK0W1/l4E+mbCFCQM15Bk18hR3TO/xfQyoCvq+jq6vGipGJHwVhbPKytq7C7UCdbbEZlMaFek1lUYFcYntmEy1b2zitROGxBvHsC0TlJj0QPlCecmyk= ceph-64384694-6fd7-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T00:49:26.466 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBpAPiYjVA/gO58KL2vp8F1U1gFCNe3NQdLH7vGfk3V3FacP/n3Tp6VyZ9a+oKB6BuQbAUE7hul14L9PEsLXEk6ClfbDJsVnhZ5eAHKneQD8SLK0QfTM5+O0ir496RYP3n+PQsLcsm+AmjhSP7JvNHCyHjT6mKVjQKAM36pql1jWkH3mdol8gArHH9g9STf7gsij8Bwb/dAtVoUQUPu0MUeTsa/f8/eHayQw6AVNCkLnaOYBUuNZJ6grCyc5WGjyXuOnRvJkARaCeeFtNiN8KDKYzDO8QcAUh6Q4/E7TYq3HLDEuPdNJqO8CQeaC763hlyCITh2jWnA2XqmVnhTbXIs3p9Qq6HqVnYDxGEVvkMD7aygn6vMgWVQt9wAgsg8PvSBdFh0RIuG/Z3MK0W1/l4E+mbCFCQM15Bk18hR3TO/xfQyoCvq+jq6vGipGJHwVhbPKytq7C7UCdbbEZlMaFek1lUYFcYntmEy1b2zitROGxBvHsC0TlJj0QPlCecmyk= ceph-64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:49:26.478 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T00:49:26.587 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:26 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3165643016' entity='client.admin' 2024-09-11T00:49:26.726 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:49:28.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:28 smithi078 ceph-mon[31315]: mgrmap e13: x(active, since 12s) 2024-09-11T00:49:28.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:28 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2071701342' entity='client.admin' 2024-09-11T00:49:28.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:28 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:28.657 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:28 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:28.658 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:28 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"} : dispatch 2024-09-11T00:49:28.658 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:28 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:28.658 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:28 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:49:28.658 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:28 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:28.684 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T00:49:28.684 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T00:49:28.926 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:49:30.282 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi110 2024-09-11T00:49:30.282 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T00:49:30.282 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.conf 2024-09-11T00:49:30.301 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T00:49:30.302 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:49:30.359 INFO:tasks.cephadm:Adding host smithi110 to orchestrator... 2024-09-11T00:49:30.359 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch host add smithi110 2024-09-11T00:49:30.604 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:49:30.724 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:30 smithi078 ceph-mon[31315]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:49:30.724 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:30 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:30.724 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:30 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:49:30.724 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:30 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:49:30.724 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:30 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:49:30.724 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:30 smithi078 ceph-mon[31315]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-11T00:49:30.724 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:30 smithi078 ceph-mon[31315]: Updating smithi078:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:49:30.725 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:30 smithi078 ceph-mon[31315]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:49:30.725 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:30 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:30.725 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:30 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:30.725 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:30 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:31.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:31 smithi078 ceph-mon[31315]: Updating smithi078:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:49:32.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:32 smithi078 ceph-mon[31315]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:49:33.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:33 smithi078 ceph-mon[31315]: Deploying cephadm binary to smithi110 2024-09-11T00:49:34.381 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi110' with addr '172.21.15.110' 2024-09-11T00:49:34.987 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T00:49:35.226 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:49:35.703 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:35 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:35.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:35 smithi078 ceph-mon[31315]: Added host smithi110 2024-09-11T00:49:35.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:35 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:49:35.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:35 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:35.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:35 smithi078 ceph-mon[31315]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:35.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:35 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:49:35.857 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}] 2024-09-11T00:49:36.444 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi112 2024-09-11T00:49:36.444 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-11T00:49:36.445 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/etc/ceph/ceph.conf 2024-09-11T00:49:36.463 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-11T00:49:36.463 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:49:36.521 INFO:tasks.cephadm:Adding host smithi112 to orchestrator... 2024-09-11T00:49:36.522 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch host add smithi112 2024-09-11T00:49:36.762 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:49:36.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:36 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:36.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:36 smithi078 ceph-mon[31315]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:49:37.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:37 smithi078 ceph-mon[31315]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:38.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:38 smithi078 ceph-mon[31315]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:49:38.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:38 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:38.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:38 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:38.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:38 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:38.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:38 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:38.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:38 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"} : dispatch 2024-09-11T00:49:38.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:38 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:49:38.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:38 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:49:39.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:39 smithi078 ceph-mon[31315]: Deploying cephadm binary to smithi112 2024-09-11T00:49:39.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:39 smithi078 ceph-mon[31315]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T00:49:39.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:39 smithi078 ceph-mon[31315]: Updating smithi110:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:49:39.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:39 smithi078 ceph-mon[31315]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:40.719 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi112' with addr '172.21.15.112' 2024-09-11T00:49:40.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:40 smithi078 ceph-mon[31315]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:49:40.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:40 smithi078 ceph-mon[31315]: Updating smithi110:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:49:40.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:40 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:40.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:40 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:40.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:40 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:40.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:40 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:40.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:40 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:49:41.376 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T00:49:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:49:42.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:41 smithi078 ceph-mon[31315]: Added host smithi112 2024-09-11T00:49:42.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:41 smithi078 ceph-mon[31315]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:42.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:41 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:42.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:41 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:49:42.261 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}, {"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}] 2024-09-11T00:49:42.884 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T00:49:42.885 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T00:49:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:49:43.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:43 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:43 smithi078 ceph-mon[31315]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:49:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:43 smithi078 ceph-mon[31315]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:44.069 INFO:teuthology.orchestra.run.smithi078.stderr:adjusted tunables profile to default 2024-09-11T00:49:44.361 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:44 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/858499104' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-11T00:49:44.640 INFO:tasks.cephadm:Adding mon.a on smithi078 2024-09-11T00:49:44.640 INFO:tasks.cephadm:Adding mon.b on smithi110 2024-09-11T00:49:44.641 INFO:tasks.cephadm:Adding mon.c on smithi112 2024-09-11T00:49:44.645 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi078:172.21.15.78=a;smithi110:172.21.15.110=b;smithi112:172.21.15.112=c' 2024-09-11T00:49:44.886 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T00:49:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T00:49:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:45 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/858499104' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T00:49:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:45 smithi078 ceph-mon[31315]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T00:49:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:45 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:45 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:45 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:45 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:45 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"} : dispatch 2024-09-11T00:49:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:45 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:49:45.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:45 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:49:45.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:45 smithi078 ceph-mon[31315]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-11T00:49:45.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:45 smithi078 ceph-mon[31315]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:45.649 INFO:teuthology.orchestra.run.smithi112.stdout:Scheduled mon update... 2024-09-11T00:49:46.321 DEBUG:teuthology.orchestra.run.smithi110:mon.b> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T00:49:46.325 DEBUG:teuthology.orchestra.run.smithi112:mon.c> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T00:49:46.327 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T00:49:46.327 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T00:49:46.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:46 smithi078 ceph-mon[31315]: Updating smithi112:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:49:46.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:46 smithi078 ceph-mon[31315]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:49:46.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:46 smithi078 ceph-mon[31315]: Updating smithi112:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:49:46.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:46 smithi078 ceph-mon[31315]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi110:172.21.15.110=b;smithi112:172.21.15.112=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:49:46.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:46 smithi078 ceph-mon[31315]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi110:172.21.15.110=b;smithi112:172.21.15.112=c;count:3 2024-09-11T00:49:46.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:46 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:46.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:46 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:46.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:46 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:46.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:46 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:46.999 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:46 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:49:46.999 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:46 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:49:46.999 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:46 smithi078 ceph-mon[31315]: Deploying daemon mon.c on smithi112 2024-09-11T00:49:47.370 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:49:47.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:47 smithi078 ceph-mon[31315]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:48.238 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:49:48.239 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":1,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","modified":"2024-09-11T00:48:16.201306Z","created":"2024-09-11T00:48:16.201306Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T00:49:48.239 INFO:teuthology.orchestra.run.smithi112.stderr:dumped monmap epoch 1 2024-09-11T00:49:48.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:48 smithi078 ceph-mon[31315]: from='client.? 172.21.15.112:0/3157694290' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-11T00:49:49.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:49 smithi078 ceph-mon[31315]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:50.277 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T00:49:50.277 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T00:49:50.790 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:49:56.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:49:55 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:49:56.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:49:56.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: mon.a calling monitor election 2024-09-11T00:49:56.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:49:56.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:56.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:49:56.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:49:56.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: mon.c calling monitor election 2024-09-11T00:49:56.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:56.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:49:56.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:49:56.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:56.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:49:56.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T00:49:56.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: monmap epoch 2 2024-09-11T00:49:56.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: fsid 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:49:56.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: last_changed 2024-09-11T00:49:50.732060+0000 2024-09-11T00:49:56.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: created 2024-09-11T00:48:16.201306+0000 2024-09-11T00:49:56.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: min_mon_release 19 (squid) 2024-09-11T00:49:56.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: election_strategy: 1 2024-09-11T00:49:56.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-11T00:49:56.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-11T00:49:56.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: fsmap 2024-09-11T00:49:56.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T00:49:56.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: mgrmap e13: x(active, since 40s) 2024-09-11T00:49:56.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: overall HEALTH_OK 2024-09-11T00:49:56.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:56.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:56.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:49:56.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:49:56.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:55 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:49:57.129 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:49:56 smithi112 ceph-mon[35384]: Deploying daemon mon.b on smithi110 2024-09-11T00:49:57.129 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:49:56 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:49:57.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:56 smithi078 ceph-mon[31315]: Deploying daemon mon.b on smithi110 2024-09-11T00:49:57.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:56 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:49:57.479 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:49:57.479 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","modified":"2024-09-11T00:49:50.732060Z","created":"2024-09-11T00:48:16.201306Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T00:49:57.479 INFO:teuthology.orchestra.run.smithi112.stderr:dumped monmap epoch 2 2024-09-11T00:49:58.125 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:49:57 smithi112 ceph-mon[35384]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:58.125 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:49:57 smithi112 ceph-mon[35384]: from='client.? 172.21.15.112:0/393682683' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-11T00:49:58.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:57 smithi078 ceph-mon[31315]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:49:58.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:49:57 smithi078 ceph-mon[31315]: from='client.? 172.21.15.112:0/393682683' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-11T00:49:59.154 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T00:49:59.154 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T00:49:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:49:59.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:49:59 smithi110 systemd[1]: Started Ceph mon.b for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:50:05.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:50:05.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: mon.a calling monitor election 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: mon.c calling monitor election 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: mon.b calling monitor election 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T00:50:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: monmap epoch 3 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: fsid 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: last_changed 2024-09-11T00:49:59.736502+0000 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: created 2024-09-11T00:48:16.201306+0000 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: min_mon_release 19 (squid) 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: election_strategy: 1 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: 2: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: fsmap 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: mgrmap e13: x(active, since 49s) 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: overall HEALTH_OK 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:05.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:05.193 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:05.193 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:04 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:05.213 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:50:05.213 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: mon.a calling monitor election 2024-09-11T00:50:05.213 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: mon.c calling monitor election 2024-09-11T00:50:05.213 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.213 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:50:05.213 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.213 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: mon.b calling monitor election 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: monmap epoch 3 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: fsid 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: last_changed 2024-09-11T00:49:59.736502+0000 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: created 2024-09-11T00:48:16.201306+0000 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: min_mon_release 19 (squid) 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: election_strategy: 1 2024-09-11T00:50:05.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-11T00:50:05.215 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-11T00:50:05.215 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: 2: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-11T00:50:05.215 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: fsmap 2024-09-11T00:50:05.215 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T00:50:05.215 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: mgrmap e13: x(active, since 49s) 2024-09-11T00:50:05.215 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: overall HEALTH_OK 2024-09-11T00:50:05.215 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:05.215 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:05.215 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:05.215 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:04 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:50:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: mon.a calling monitor election 2024-09-11T00:50:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: mon.c calling monitor election 2024-09-11T00:50:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:50:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: mon.b calling monitor election 2024-09-11T00:50:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: monmap epoch 3 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: fsid 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: last_changed 2024-09-11T00:49:59.736502+0000 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: created 2024-09-11T00:48:16.201306+0000 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: min_mon_release 19 (squid) 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: election_strategy: 1 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: 1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.c 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: 2: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: fsmap 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T00:50:05.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: mgrmap e13: x(active, since 49s) 2024-09-11T00:50:05.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: overall HEALTH_OK 2024-09-11T00:50:05.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:05.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:05.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:05.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:04 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.060 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:50:06.060 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","modified":"2024-09-11T00:49:59.736502Z","created":"2024-09-11T00:48:16.201306Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-11T00:50:06.060 INFO:teuthology.orchestra.run.smithi112.stderr:dumped monmap epoch 3 2024-09-11T00:50:06.072 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:05 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:06.072 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:05 smithi112 ceph-mon[35384]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:06.072 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:05 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.072 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:05 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:06.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:05 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:06.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:05 smithi110 ceph-mon[34712]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:06.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:05 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:05 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:06.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:05 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:06.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:05 smithi078 ceph-mon[31315]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:06.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:05 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:05 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:06.712 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T00:50:06.712 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T00:50:06.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:06.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:06.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='client.? 172.21.15.112:0/2110806561' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:06.976 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:50:06.976 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:50:06.976 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:06 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:07.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:07.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:07.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-11T00:50:07.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T00:50:07.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-11T00:50:07.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='client.? 172.21.15.112:0/2110806561' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-11T00:50:07.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:50:07.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:06 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='client.? 172.21.15.112:0/2110806561' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:07.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:50:07.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:50:07.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:06 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:08.020 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:08.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:07 smithi112 ceph-mon[35384]: Updating smithi110:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:08.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:07 smithi112 ceph-mon[35384]: Updating smithi112:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:08.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:07 smithi112 ceph-mon[35384]: Updating smithi078:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:08.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:07 smithi112 ceph-mon[35384]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T00:50:08.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:07 smithi112 ceph-mon[35384]: Reconfiguring daemon mon.a on smithi078 2024-09-11T00:50:08.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:07 smithi112 ceph-mon[35384]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:08.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:07 smithi078 ceph-mon[31315]: Updating smithi110:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:08.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:07 smithi078 ceph-mon[31315]: Updating smithi112:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:08.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:07 smithi078 ceph-mon[31315]: Updating smithi078:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:08.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:07 smithi078 ceph-mon[31315]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T00:50:08.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:07 smithi078 ceph-mon[31315]: Reconfiguring daemon mon.a on smithi078 2024-09-11T00:50:08.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:07 smithi078 ceph-mon[31315]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:08.341 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:07 smithi110 ceph-mon[34712]: Updating smithi110:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:08.341 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:07 smithi110 ceph-mon[34712]: Updating smithi112:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:08.341 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:07 smithi110 ceph-mon[34712]: Updating smithi078:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:08.341 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:07 smithi110 ceph-mon[34712]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T00:50:08.341 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:07 smithi110 ceph-mon[34712]: Reconfiguring daemon mon.a on smithi078 2024-09-11T00:50:08.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:07 smithi110 ceph-mon[34712]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:08.600 INFO:teuthology.orchestra.run.smithi078.stdout:# minimal ceph.conf for 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:50:08.601 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2024-09-11T00:50:08.601 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:50:08.601 INFO:teuthology.orchestra.run.smithi078.stdout: mon_host = [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] 2024-09-11T00:50:09.186 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T00:50:09.186 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:50:09.186 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T00:50:09.215 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:50:09.215 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:50:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:08 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:08 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:08 smithi078 ceph-mon[31315]: Reconfiguring mon.b (monmap changed)... 2024-09-11T00:50:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:08 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:50:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:08 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:50:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:08 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:08 smithi078 ceph-mon[31315]: Reconfiguring daemon mon.b on smithi110 2024-09-11T00:50:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:08 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3476038223' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:09.272 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T00:50:09.272 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T00:50:09.295 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:08 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:09.296 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:08 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:09.296 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:08 smithi110 ceph-mon[34712]: Reconfiguring mon.b (monmap changed)... 2024-09-11T00:50:09.296 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:08 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:50:09.296 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:08 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:50:09.296 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:08 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:09.296 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:08 smithi110 ceph-mon[34712]: Reconfiguring daemon mon.b on smithi110 2024-09-11T00:50:09.296 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:08 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3476038223' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:09.300 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T00:50:09.300 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:50:09.365 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-11T00:50:09.365 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T00:50:09.388 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:08 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:09.388 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:08 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:09.388 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:08 smithi112 ceph-mon[35384]: Reconfiguring mon.b (monmap changed)... 2024-09-11T00:50:09.388 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:08 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:50:09.388 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:08 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:50:09.388 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:08 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:09.388 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:08 smithi112 ceph-mon[35384]: Reconfiguring daemon mon.b on smithi110 2024-09-11T00:50:09.388 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:08 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3476038223' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:09.392 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-11T00:50:09.392 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:50:09.458 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-11T00:50:09.458 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-11T00:50:09.695 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:10.087 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:09 smithi078 ceph-mon[31315]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:10.087 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:09 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:10.088 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:09 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:10.088 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:09 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:50:10.088 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:09 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:50:10.088 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:09 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:10.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:09 smithi110 ceph-mon[34712]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:10.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:09 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:10.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:09 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:10.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:09 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:50:10.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:09 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:50:10.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:09 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:09 smithi112 ceph-mon[35384]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:09 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:09 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:09 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:50:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:09 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:50:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:09 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:11.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:10 smithi078 ceph-mon[31315]: Reconfiguring mon.c (monmap changed)... 2024-09-11T00:50:11.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:10 smithi078 ceph-mon[31315]: Reconfiguring daemon mon.c on smithi112 2024-09-11T00:50:11.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:10 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2185002012' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-11T00:50:11.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:10 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:11.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:10 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:11.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:10 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:11.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:10 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:11.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:10 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:11.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:10 smithi078 ceph-mon[31315]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:11.249 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:11 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: ignoring --setuser ceph since I am not root 2024-09-11T00:50:11.249 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:11 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: ignoring --setgroup ceph since I am not root 2024-09-11T00:50:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:10 smithi110 ceph-mon[34712]: Reconfiguring mon.c (monmap changed)... 2024-09-11T00:50:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:10 smithi110 ceph-mon[34712]: Reconfiguring daemon mon.c on smithi112 2024-09-11T00:50:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:10 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2185002012' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-11T00:50:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:10 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:11.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:10 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:11.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:10 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:11.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:10 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:11.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:10 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:11.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:10 smithi110 ceph-mon[34712]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:11.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:10 smithi112 ceph-mon[35384]: Reconfiguring mon.c (monmap changed)... 2024-09-11T00:50:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:10 smithi112 ceph-mon[35384]: Reconfiguring daemon mon.c on smithi112 2024-09-11T00:50:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:10 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2185002012' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-11T00:50:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:10 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:10 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:10 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:10 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:10 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:10 smithi112 ceph-mon[35384]: from='mgr.14150 172.21.15.78:0/3228631281' entity='mgr.x' 2024-09-11T00:50:11.764 INFO:tasks.cephadm:Adding mgr.x on smithi078 2024-09-11T00:50:11.764 INFO:tasks.cephadm:Adding mgr.y on smithi110 2024-09-11T00:50:11.764 INFO:tasks.cephadm:Adding mgr.z on smithi112 2024-09-11T00:50:11.765 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi078=x;smithi110=y;smithi112=z' 2024-09-11T00:50:12.016 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:50:12.135 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:11 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2185002012' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T00:50:12.135 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:11 smithi112 ceph-mon[35384]: mgrmap e14: x(active, since 56s) 2024-09-11T00:50:12.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:11 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2185002012' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T00:50:12.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:11 smithi078 ceph-mon[31315]: mgrmap e14: x(active, since 56s) 2024-09-11T00:50:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:11 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2185002012' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T00:50:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:11 smithi110 ceph-mon[34712]: mgrmap e14: x(active, since 56s) 2024-09-11T00:50:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:15 smithi078 ceph-mon[31315]: Active manager daemon x restarted 2024-09-11T00:50:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:15 smithi078 ceph-mon[31315]: Activating manager daemon x 2024-09-11T00:50:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:15 smithi078 ceph-mon[31315]: osdmap e5: 0 total, 0 up, 0 in 2024-09-11T00:50:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:15 smithi078 ceph-mon[31315]: mgrmap e15: x(active, starting, since 0.00439431s) 2024-09-11T00:50:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:50:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:50:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-11T00:50:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-11T00:50:16.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-11T00:50:16.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-11T00:50:16.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:15 smithi078 ceph-mon[31315]: Manager daemon x is now available 2024-09-11T00:50:16.317 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:15 smithi110 ceph-mon[34712]: Active manager daemon x restarted 2024-09-11T00:50:16.317 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:15 smithi110 ceph-mon[34712]: Activating manager daemon x 2024-09-11T00:50:16.317 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:15 smithi110 ceph-mon[34712]: osdmap e5: 0 total, 0 up, 0 in 2024-09-11T00:50:16.317 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:15 smithi110 ceph-mon[34712]: mgrmap e15: x(active, starting, since 0.00439431s) 2024-09-11T00:50:16.317 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:50:16.317 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:16.317 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:50:16.317 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-11T00:50:16.317 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-11T00:50:16.317 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-11T00:50:16.317 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-11T00:50:16.318 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:15 smithi110 ceph-mon[34712]: Manager daemon x is now available 2024-09-11T00:50:16.396 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:15 smithi112 ceph-mon[35384]: Active manager daemon x restarted 2024-09-11T00:50:16.396 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:15 smithi112 ceph-mon[35384]: Activating manager daemon x 2024-09-11T00:50:16.396 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:15 smithi112 ceph-mon[35384]: osdmap e5: 0 total, 0 up, 0 in 2024-09-11T00:50:16.396 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:15 smithi112 ceph-mon[35384]: mgrmap e15: x(active, starting, since 0.00439431s) 2024-09-11T00:50:16.396 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:50:16.396 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:50:16.396 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:50:16.396 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-11T00:50:16.396 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-11T00:50:16.397 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-11T00:50:16.397 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-11T00:50:16.397 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:15 smithi112 ceph-mon[35384]: Manager daemon x is now available 2024-09-11T00:50:16.943 INFO:teuthology.orchestra.run.smithi112.stdout:Scheduled mgr update... 2024-09-11T00:50:17.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:17.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-11T00:50:17.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-11T00:50:17.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:16 smithi112 ceph-mon[35384]: mgrmap e16: x(active, since 1.00838s) 2024-09-11T00:50:17.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:17.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:17.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-11T00:50:17.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-11T00:50:17.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:16 smithi078 ceph-mon[31315]: mgrmap e16: x(active, since 1.00838s) 2024-09-11T00:50:17.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:17.246 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:50:17.156+0000 7ff897b90640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:00:50:17] ENGINE Error in HTTPServer.serve 2024-09-11T00:50:17.246 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: Traceback (most recent call last): 2024-09-11T00:50:17.246 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T00:50:17.246 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self._connections.run(self.expiration_interval) 2024-09-11T00:50:17.246 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T00:50:17.247 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self._run(expiration_interval) 2024-09-11T00:50:17.247 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T00:50:17.247 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T00:50:17.247 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T00:50:17.247 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T00:50:17.248 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T00:50:17.248 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: s = self.context.wrap_socket( 2024-09-11T00:50:17.248 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T00:50:17.248 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: return self.sslsocket_class._create( 2024-09-11T00:50:17.248 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T00:50:17.249 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self.do_handshake() 2024-09-11T00:50:17.249 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T00:50:17.249 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self._sslobj.do_handshake() 2024-09-11T00:50:17.249 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T00:50:17.249 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:50:17.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:17.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-11T00:50:17.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-11T00:50:17.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:16 smithi110 ceph-mon[34712]: mgrmap e16: x(active, since 1.00838s) 2024-09-11T00:50:17.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.118 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:17 smithi112 ceph-mon[35384]: Saving service mgr spec with placement smithi078=x;smithi110=y;smithi112=z;count:3 2024-09-11T00:50:18.118 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:17 smithi112 ceph-mon[35384]: [11/Sep/2024:00:50:17] ENGINE Bus STARTING 2024-09-11T00:50:18.119 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.119 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.119 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.119 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.119 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.119 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.150 DEBUG:teuthology.orchestra.run.smithi110:mgr.y> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T00:50:18.153 DEBUG:teuthology.orchestra.run.smithi112:mgr.z> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.z.service 2024-09-11T00:50:18.155 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-11T00:50:18.155 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-09-11T00:50:18.157 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-11T00:50:18.157 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set client debug_client 20 2024-09-11T00:50:18.201 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-11T00:50:18.201 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set client debug_ms 1 2024-09-11T00:50:18.245 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-11T00:50:18.245 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-09-11T00:50:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-mon[31315]: Saving service mgr spec with placement smithi078=x;smithi110=y;smithi112=z;count:3 2024-09-11T00:50:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-mon[31315]: [11/Sep/2024:00:50:17] ENGINE Bus STARTING 2024-09-11T00:50:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.249 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-11T00:50:18.249 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-09-11T00:50:18.292 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-11T00:50:18.292 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds 20 2024-09-11T00:50:18.335 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-11T00:50:18.336 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-09-11T00:50:18.385 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-11T00:50:18.385 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds debug_ms 2 2024-09-11T00:50:18.429 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-11T00:50:18.429 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-11T00:50:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:17 smithi110 ceph-mon[34712]: Saving service mgr spec with placement smithi078=x;smithi110=y;smithi112=z;count:3 2024-09-11T00:50:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:17 smithi110 ceph-mon[34712]: [11/Sep/2024:00:50:17] ENGINE Bus STARTING 2024-09-11T00:50:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:18.476 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-11T00:50:18.476 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-09-11T00:50:18.521 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-11T00:50:18.522 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-09-11T00:50:18.572 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-11T00:50:18.573 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-09-11T00:50:18.607 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:18.612 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-11T00:50:18.612 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-09-11T00:50:18.656 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-11T00:50:18.656 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-09-11T00:50:18.687 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:18.694 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-11T00:50:18.694 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-09-11T00:50:18.753 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-11T00:50:18.754 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-09-11T00:50:18.808 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-11T00:50:18.808 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-09-11T00:50:18.857 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-11T00:50:18.857 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-09-11T00:50:18.909 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-11T00:50:18.910 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-09-11T00:50:18.966 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-11T00:50:18.966 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mgr debug_cephsqlite 20 2024-09-11T00:50:19.018 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-11T00:50:19.019 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-11T00:50:19.064 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-11T00:50:19.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-11T00:50:19.065 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:18 smithi078 ceph-mon[31315]: [11/Sep/2024:00:50:17] ENGINE Serving on https://172.21.15.78:7150 2024-09-11T00:50:19.065 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: [11/Sep/2024:00:50:17] ENGINE Error in HTTPServer.serve 2024-09-11T00:50:19.065 INFO:journalctl@ceph.mon.a.smithi078.stdout: Traceback (most recent call last): 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._connections.run(self.expiration_interval) 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._run(expiration_interval) 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: s = self.context.wrap_socket( 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: return self.sslsocket_class._create( 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T00:50:19.066 INFO:journalctl@ceph.mon.a.smithi078.stdout: self.do_handshake() 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._sslobj.do_handshake() 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: [11/Sep/2024:00:50:17] ENGINE Serving on http://172.21.15.78:8765 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: [11/Sep/2024:00:50:17] ENGINE Bus STARTED 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"} : dispatch 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"} : dispatch 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"} : dispatch 2024-09-11T00:50:19.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:19.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:19.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:19 smithi078 ceph-mon[31315]: mgrmap e17: x(active, since 2s) 2024-09-11T00:50:19.109 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-11T00:50:19.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-11T00:50:19.159 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-11T00:50:19.159 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-09-11T00:50:19.255 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-11T00:50:19.255 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-09-11T00:50:19.310 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-11T00:50:19.311 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-09-11T00:50:19.359 DEBUG:tasks.cephadm:set 26 configs 2024-09-11T00:50:19.359 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi078') 2024-09-11T00:50:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: [11/Sep/2024:00:50:17] ENGINE Serving on https://172.21.15.78:7150 2024-09-11T00:50:19.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: [11/Sep/2024:00:50:17] ENGINE Error in HTTPServer.serve 2024-09-11T00:50:19.441 INFO:journalctl@ceph.mon.b.smithi110.stdout: Traceback (most recent call last): 2024-09-11T00:50:19.441 INFO:journalctl@ceph.mon.b.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T00:50:19.441 INFO:journalctl@ceph.mon.b.smithi110.stdout: self._connections.run(self.expiration_interval) 2024-09-11T00:50:19.441 INFO:journalctl@ceph.mon.b.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T00:50:19.441 INFO:journalctl@ceph.mon.b.smithi110.stdout: self._run(expiration_interval) 2024-09-11T00:50:19.441 INFO:journalctl@ceph.mon.b.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T00:50:19.441 INFO:journalctl@ceph.mon.b.smithi110.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T00:50:19.441 INFO:journalctl@ceph.mon.b.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T00:50:19.442 INFO:journalctl@ceph.mon.b.smithi110.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T00:50:19.442 INFO:journalctl@ceph.mon.b.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T00:50:19.442 INFO:journalctl@ceph.mon.b.smithi110.stdout: s = self.context.wrap_socket( 2024-09-11T00:50:19.442 INFO:journalctl@ceph.mon.b.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T00:50:19.442 INFO:journalctl@ceph.mon.b.smithi110.stdout: return self.sslsocket_class._create( 2024-09-11T00:50:19.442 INFO:journalctl@ceph.mon.b.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T00:50:19.442 INFO:journalctl@ceph.mon.b.smithi110.stdout: self.do_handshake() 2024-09-11T00:50:19.442 INFO:journalctl@ceph.mon.b.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T00:50:19.442 INFO:journalctl@ceph.mon.b.smithi110.stdout: self._sslobj.do_handshake() 2024-09-11T00:50:19.442 INFO:journalctl@ceph.mon.b.smithi110.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T00:50:19.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: [11/Sep/2024:00:50:17] ENGINE Serving on http://172.21.15.78:8765 2024-09-11T00:50:19.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: [11/Sep/2024:00:50:17] ENGINE Bus STARTED 2024-09-11T00:50:19.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"} : dispatch 2024-09-11T00:50:19.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"} : dispatch 2024-09-11T00:50:19.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"} : dispatch 2024-09-11T00:50:19.444 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:19.444 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:19.444 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:18 smithi110 ceph-mon[34712]: mgrmap e17: x(active, since 2s) 2024-09-11T00:50:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: [11/Sep/2024:00:50:17] ENGINE Serving on https://172.21.15.78:7150 2024-09-11T00:50:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: [11/Sep/2024:00:50:17] ENGINE Error in HTTPServer.serve 2024-09-11T00:50:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout: Traceback (most recent call last): 2024-09-11T00:50:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T00:50:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout: self._connections.run(self.expiration_interval) 2024-09-11T00:50:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T00:50:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout: self._run(expiration_interval) 2024-09-11T00:50:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T00:50:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T00:50:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T00:50:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T00:50:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T00:50:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout: s = self.context.wrap_socket( 2024-09-11T00:50:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T00:50:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout: return self.sslsocket_class._create( 2024-09-11T00:50:19.476 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T00:50:19.476 INFO:journalctl@ceph.mon.c.smithi112.stdout: self.do_handshake() 2024-09-11T00:50:19.476 INFO:journalctl@ceph.mon.c.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T00:50:19.476 INFO:journalctl@ceph.mon.c.smithi112.stdout: self._sslobj.do_handshake() 2024-09-11T00:50:19.476 INFO:journalctl@ceph.mon.c.smithi112.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T00:50:19.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: [11/Sep/2024:00:50:17] ENGINE Serving on http://172.21.15.78:8765 2024-09-11T00:50:19.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: [11/Sep/2024:00:50:17] ENGINE Bus STARTED 2024-09-11T00:50:19.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"} : dispatch 2024-09-11T00:50:19.477 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.477 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.477 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.477 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"} : dispatch 2024-09-11T00:50:19.477 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:19.477 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"} : dispatch 2024-09-11T00:50:19.477 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:19.477 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:19.477 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:18 smithi112 ceph-mon[35384]: mgrmap e17: x(active, since 2s) 2024-09-11T00:50:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-11T00:50:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T00:50:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-11T00:50:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: Updating smithi110:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: Updating smithi112:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:50:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:50:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: Updating smithi078:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: Updating smithi110:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:50:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: Updating smithi112:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:50:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:20.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:20.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:20.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:20 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1956037415' entity='client.admin' 2024-09-11T00:50:20.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-11T00:50:20.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T00:50:20.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-11T00:50:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: Updating smithi110:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: Updating smithi112:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:50:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:50:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: Updating smithi078:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: Updating smithi110:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:50:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: Updating smithi112:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:50:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:20.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:20.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:20 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1956037415' entity='client.admin' 2024-09-11T00:50:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-11T00:50:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T00:50:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-11T00:50:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: Updating smithi110:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: Updating smithi112:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:50:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:50:20.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: Updating smithi078:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:50:20.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: Updating smithi110:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:50:20.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: Updating smithi112:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:50:20.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:20.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:20.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:20.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:20.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:20 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1956037415' entity='client.admin' 2024-09-11T00:50:20.664 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi078') 2024-09-11T00:50:20.863 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:20.887 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:20.911 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:20.923 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:21.081 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:21 smithi112 ceph-mon[35384]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:50:21.081 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:21 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1870560165' entity='client.admin' 2024-09-11T00:50:21.082 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:21.082 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:21.082 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:21.082 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:50:21.082 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T00:50:21.082 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:50:21.082 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:21.315 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:21 smithi078 ceph-mon[31315]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:50:21.316 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:21 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1870560165' entity='client.admin' 2024-09-11T00:50:21.316 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:21.316 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:21.316 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:21.316 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:50:21.316 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T00:50:21.316 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:50:21.316 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:21.330 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:21.333 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:21.333 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:21.361 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi078') 2024-09-11T00:50:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:21 smithi110 ceph-mon[34712]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:50:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:21 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1870560165' entity='client.admin' 2024-09-11T00:50:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:21.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:21.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:50:21.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T00:50:21.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:50:21.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:22.064 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:22.064 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:22.065 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:22.191 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:22 smithi078 ceph-mon[31315]: Updating smithi078:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:50:22.191 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:22 smithi078 ceph-mon[31315]: Deploying daemon mgr.z on smithi112 2024-09-11T00:50:22.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:22 smithi112 ceph-mon[35384]: Updating smithi078:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:50:22.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:22 smithi112 ceph-mon[35384]: Deploying daemon mgr.z on smithi112 2024-09-11T00:50:22.224 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 00:50:22 smithi112 systemd[1]: Starting Ceph mgr.z for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:50:22.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:22 smithi110 ceph-mon[34712]: Updating smithi078:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:50:22.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:22 smithi110 ceph-mon[34712]: Deploying daemon mgr.z on smithi112 2024-09-11T00:50:22.516 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 00:50:22 smithi112 podman[38000]: 2024-09-11 00:50:22.16210016 +0000 UTC m=+0.016006088 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:50:22.516 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 00:50:22 smithi112 podman[38000]: 2024-09-11 00:50:22.286022898 +0000 UTC m=+0.139928822 container create 8ece7eb0deefbe7926b6824a6ac82e0857220f424f20094882d541550c444a18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T00:50:22.516 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 00:50:22 smithi112 podman[38000]: 2024-09-11 00:50:22.488440076 +0000 UTC m=+0.342345999 container init 8ece7eb0deefbe7926b6824a6ac82e0857220f424f20094882d541550c444a18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T00:50:22.516 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 00:50:22 smithi112 podman[38000]: 2024-09-11 00:50:22.491078487 +0000 UTC m=+0.344984411 container start 8ece7eb0deefbe7926b6824a6ac82e0857220f424f20094882d541550c444a18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-z, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2) 2024-09-11T00:50:22.974 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 00:50:22 smithi112 ceph-mgr[38019]: -- 172.21.15.112:0/1811389209 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5621f25970e0 con 0x5621f2575400 2024-09-11T00:50:22.974 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 00:50:22 smithi112 bash[38000]: 8ece7eb0deefbe7926b6824a6ac82e0857220f424f20094882d541550c444a18 2024-09-11T00:50:22.974 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 00:50:22 smithi112 systemd[1]: Started Ceph mgr.z for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:50:23.374 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:23 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3325894267' entity='client.admin' 2024-09-11T00:50:23.374 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:23 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:23.374 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:23 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:23.375 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:23 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:23.375 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:23 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:50:23.375 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:23 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T00:50:23.375 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:23 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:50:23.375 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:23 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:23.375 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:23 smithi110 ceph-mon[34712]: Deploying daemon mgr.y on smithi110 2024-09-11T00:50:23.375 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:23 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3864736554' entity='client.admin' 2024-09-11T00:50:23.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:23 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3325894267' entity='client.admin' 2024-09-11T00:50:23.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:23 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:23.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:23 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:23.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:23 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:23.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:23 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:50:23.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:23 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T00:50:23.474 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:23 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:50:23.474 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:23 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:23.474 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:23 smithi078 ceph-mon[31315]: Deploying daemon mgr.y on smithi110 2024-09-11T00:50:23.474 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:23 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3864736554' entity='client.admin' 2024-09-11T00:50:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:23 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3325894267' entity='client.admin' 2024-09-11T00:50:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:23 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:23 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:23 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:23.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:23 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:50:23.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:23 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T00:50:23.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:23 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:50:23.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:23 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:23.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:23 smithi112 ceph-mon[35384]: Deploying daemon mgr.y on smithi110 2024-09-11T00:50:23.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:23 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3864736554' entity='client.admin' 2024-09-11T00:50:24.640 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 00:50:24 smithi110 systemd[1]: Starting Ceph mgr.y for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:50:24.825 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:24 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2763127700' entity='client.admin' 2024-09-11T00:50:24.825 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:24 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/320159351' entity='client.admin' 2024-09-11T00:50:24.825 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:24 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/690330273' entity='client.admin' 2024-09-11T00:50:24.829 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:24.898 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:24 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2763127700' entity='client.admin' 2024-09-11T00:50:24.898 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:24 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/320159351' entity='client.admin' 2024-09-11T00:50:24.898 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:24 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/690330273' entity='client.admin' 2024-09-11T00:50:24.898 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 00:50:24 smithi110 podman[37053]: 2024-09-11 00:50:24.644753205 +0000 UTC m=+0.105702866 container create c5d729218aea70209c6d0c38fa4d40c3ee5492e6326dae0f4deb1570df2f8aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-11T00:50:24.899 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 00:50:24 smithi110 podman[37053]: 2024-09-11 00:50:24.554787907 +0000 UTC m=+0.015737578 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:50:24.899 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 00:50:24 smithi110 podman[37053]: 2024-09-11 00:50:24.822436756 +0000 UTC m=+0.283386415 container init c5d729218aea70209c6d0c38fa4d40c3ee5492e6326dae0f4deb1570df2f8aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, ceph=True) 2024-09-11T00:50:24.899 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 00:50:24 smithi110 podman[37053]: 2024-09-11 00:50:24.825229753 +0000 UTC m=+0.286179430 container start c5d729218aea70209c6d0c38fa4d40c3ee5492e6326dae0f4deb1570df2f8aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T00:50:24.899 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 00:50:24 smithi110 ceph-mgr[37072]: -- 172.21.15.110:0/809162261 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b68ee390e0 con 0x55b68ee17000 2024-09-11T00:50:24.899 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 00:50:24 smithi110 bash[37053]: c5d729218aea70209c6d0c38fa4d40c3ee5492e6326dae0f4deb1570df2f8aa9 2024-09-11T00:50:24.927 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:24 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2763127700' entity='client.admin' 2024-09-11T00:50:24.927 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:24 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/320159351' entity='client.admin' 2024-09-11T00:50:24.927 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:24 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/690330273' entity='client.admin' 2024-09-11T00:50:25.190 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 00:50:24 smithi110 systemd[1]: Started Ceph mgr.y for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:50:26.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:25 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:26.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:25 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:26.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:25 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:26.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:25 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:26.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:25 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2671665364' entity='client.admin' 2024-09-11T00:50:26.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:25 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:26.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:25 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2329295035' entity='client.admin' 2024-09-11T00:50:26.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:25 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2557445745' entity='client.admin' 2024-09-11T00:50:26.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:25 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3604252854' entity='client.admin' 2024-09-11T00:50:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:25 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:25 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:25 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:25 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:25 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2671665364' entity='client.admin' 2024-09-11T00:50:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:25 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:25 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2329295035' entity='client.admin' 2024-09-11T00:50:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:25 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2557445745' entity='client.admin' 2024-09-11T00:50:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:25 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3604252854' entity='client.admin' 2024-09-11T00:50:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:25 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:25 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:25 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:25 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:25 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2671665364' entity='client.admin' 2024-09-11T00:50:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:25 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:25 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2329295035' entity='client.admin' 2024-09-11T00:50:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:25 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2557445745' entity='client.admin' 2024-09-11T00:50:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:25 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3604252854' entity='client.admin' 2024-09-11T00:50:27.398 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:27.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:27.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T00:50:27.430 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:50:27.430 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:50:27.430 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:27.430 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:27 smithi078 ceph-mon[31315]: Reconfiguring daemon mgr.x on smithi078 2024-09-11T00:50:27.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:27.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:27.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T00:50:27.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:50:27.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:50:27.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:27.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:27 smithi110 ceph-mon[34712]: Reconfiguring daemon mgr.x on smithi078 2024-09-11T00:50:27.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.479 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:27.480 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:27.480 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.480 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:27.480 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T00:50:27.480 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:50:27.480 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:50:27.480 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:27.481 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:27 smithi112 ceph-mon[35384]: Reconfiguring daemon mgr.x on smithi078 2024-09-11T00:50:28.317 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:28 smithi078 ceph-mon[31315]: from='mgr.? 172.21.15.112:0/2333463218' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-11T00:50:28.317 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:28 smithi078 ceph-mon[31315]: Standby manager daemon z started 2024-09-11T00:50:28.317 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:28 smithi078 ceph-mon[31315]: from='mgr.? 172.21.15.112:0/2333463218' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-11T00:50:28.317 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:28 smithi078 ceph-mon[31315]: from='mgr.? 172.21.15.112:0/2333463218' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-11T00:50:28.317 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:28 smithi078 ceph-mon[31315]: from='mgr.? 172.21.15.112:0/2333463218' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-11T00:50:28.317 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:28 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1433975767' entity='client.admin' 2024-09-11T00:50:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:28 smithi110 ceph-mon[34712]: from='mgr.? 172.21.15.112:0/2333463218' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-11T00:50:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:28 smithi110 ceph-mon[34712]: Standby manager daemon z started 2024-09-11T00:50:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:28 smithi110 ceph-mon[34712]: from='mgr.? 172.21.15.112:0/2333463218' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-11T00:50:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:28 smithi110 ceph-mon[34712]: from='mgr.? 172.21.15.112:0/2333463218' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-11T00:50:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:28 smithi110 ceph-mon[34712]: from='mgr.? 172.21.15.112:0/2333463218' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-11T00:50:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:28 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1433975767' entity='client.admin' 2024-09-11T00:50:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:28 smithi112 ceph-mon[35384]: from='mgr.? 172.21.15.112:0/2333463218' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-11T00:50:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:28 smithi112 ceph-mon[35384]: Standby manager daemon z started 2024-09-11T00:50:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:28 smithi112 ceph-mon[35384]: from='mgr.? 172.21.15.112:0/2333463218' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-11T00:50:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:28 smithi112 ceph-mon[35384]: from='mgr.? 172.21.15.112:0/2333463218' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-11T00:50:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:28 smithi112 ceph-mon[35384]: from='mgr.? 172.21.15.112:0/2333463218' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-11T00:50:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:28 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1433975767' entity='client.admin' 2024-09-11T00:50:28.867 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:28.867 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:28.867 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:28.870 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:28.871 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:28.871 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:28.873 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:28.873 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:28.878 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:29.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:29 smithi110 ceph-mon[34712]: mgrmap e18: x(active, since 12s), standbys: z 2024-09-11T00:50:29.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:29 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-11T00:50:29.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:29 smithi078 ceph-mon[31315]: mgrmap e18: x(active, since 12s), standbys: z 2024-09-11T00:50:29.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:29 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-11T00:50:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:29 smithi112 ceph-mon[35384]: mgrmap e18: x(active, since 12s), standbys: z 2024-09-11T00:50:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:29 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-11T00:50:30.301 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:30 smithi078 ceph-mon[31315]: Standby manager daemon y started 2024-09-11T00:50:30.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:30 smithi078 ceph-mon[31315]: from='mgr.? 172.21.15.110:0/901418995' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-11T00:50:30.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:30 smithi078 ceph-mon[31315]: from='mgr.? 172.21.15.110:0/901418995' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-11T00:50:30.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:30 smithi078 ceph-mon[31315]: from='mgr.? 172.21.15.110:0/901418995' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-11T00:50:30.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:30 smithi078 ceph-mon[31315]: from='mgr.? 172.21.15.110:0/901418995' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-11T00:50:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:30 smithi110 ceph-mon[34712]: Standby manager daemon y started 2024-09-11T00:50:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:30 smithi110 ceph-mon[34712]: from='mgr.? 172.21.15.110:0/901418995' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-11T00:50:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:30 smithi110 ceph-mon[34712]: from='mgr.? 172.21.15.110:0/901418995' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-11T00:50:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:30 smithi110 ceph-mon[34712]: from='mgr.? 172.21.15.110:0/901418995' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-11T00:50:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:30 smithi110 ceph-mon[34712]: from='mgr.? 172.21.15.110:0/901418995' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-11T00:50:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:30 smithi112 ceph-mon[35384]: Standby manager daemon y started 2024-09-11T00:50:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:30 smithi112 ceph-mon[35384]: from='mgr.? 172.21.15.110:0/901418995' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-11T00:50:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:30 smithi112 ceph-mon[35384]: from='mgr.? 172.21.15.110:0/901418995' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-11T00:50:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:30 smithi112 ceph-mon[35384]: from='mgr.? 172.21.15.110:0/901418995' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-11T00:50:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:30 smithi112 ceph-mon[35384]: from='mgr.? 172.21.15.110:0/901418995' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-11T00:50:31.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:31 smithi078 ceph-mon[31315]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-09-11T00:50:31.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:31 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-11T00:50:31.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:31 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2864596076' entity='client.admin' 2024-09-11T00:50:31.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:31 smithi110 ceph-mon[34712]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-09-11T00:50:31.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:31 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-11T00:50:31.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:31 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2864596076' entity='client.admin' 2024-09-11T00:50:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:31 smithi112 ceph-mon[35384]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-09-11T00:50:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:31 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-11T00:50:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:31 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2864596076' entity='client.admin' 2024-09-11T00:50:32.485 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:32 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/301667673' entity='client.admin' 2024-09-11T00:50:32.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:32 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/301667673' entity='client.admin' 2024-09-11T00:50:32.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:32 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/301667673' entity='client.admin' 2024-09-11T00:50:33.447 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:33 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3434219346' entity='client.admin' 2024-09-11T00:50:33.447 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:33 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2348241308' entity='client.admin' 2024-09-11T00:50:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:33 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3434219346' entity='client.admin' 2024-09-11T00:50:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:33 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2348241308' entity='client.admin' 2024-09-11T00:50:33.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:33 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3434219346' entity='client.admin' 2024-09-11T00:50:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:33 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2348241308' entity='client.admin' 2024-09-11T00:50:34.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:34 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1047979537' entity='client.admin' 2024-09-11T00:50:34.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:34 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1047979537' entity='client.admin' 2024-09-11T00:50:34.979 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:34 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1047979537' entity='client.admin' 2024-09-11T00:50:36.166 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:35 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/364576730' entity='client.admin' 2024-09-11T00:50:36.166 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:35 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1222713228' entity='client.admin' 2024-09-11T00:50:36.166 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:35 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1231767386' entity='client.admin' 2024-09-11T00:50:36.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:35 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/364576730' entity='client.admin' 2024-09-11T00:50:36.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:35 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1222713228' entity='client.admin' 2024-09-11T00:50:36.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:35 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1231767386' entity='client.admin' 2024-09-11T00:50:36.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:35 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/364576730' entity='client.admin' 2024-09-11T00:50:36.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:35 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1222713228' entity='client.admin' 2024-09-11T00:50:36.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:35 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1231767386' entity='client.admin' 2024-09-11T00:50:37.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:36 smithi112 ceph-mon[35384]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:37.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:36 smithi078 ceph-mon[31315]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:37.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:36 smithi110 ceph-mon[34712]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:38.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.934+0000 7f95183fc640 1 -- 172.21.15.78:0/3024813455 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94fc0b52a0 msgr2=0x7f94fc0b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:38.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.934+0000 7f95183fc640 1 --2- 172.21.15.78:0/3024813455 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94fc0b52a0 0x7f94fc0b7760 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f950400a990 tx=0x7f9504005c70 comp rx=0 tx=0).stop 2024-09-11T00:50:38.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.934+0000 7f95183fc640 1 -- 172.21.15.78:0/3024813455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101076a0 msgr2=0x7f951019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:38.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.934+0000 7f95183fc640 1 --2- 172.21.15.78:0/3024813455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101076a0 0x7f951019a500 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f950c005940 tx=0x7f950c003de0 comp rx=0 tx=0).stop 2024-09-11T00:50:38.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.934+0000 7f95183fc640 1 -- 172.21.15.78:0/3024813455 shutdown_connections 2024-09-11T00:50:38.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.934+0000 7f95183fc640 1 --2- 172.21.15.78:0/3024813455 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94fc0b52a0 0x7f94fc0b7760 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:38.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.934+0000 7f95183fc640 1 --2- 172.21.15.78:0/3024813455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9510108020 0x7f95101a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:38.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.934+0000 7f95183fc640 1 --2- 172.21.15.78:0/3024813455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101076a0 0x7f951019a500 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:38.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.934+0000 7f95183fc640 1 --2- 172.21.15.78:0/3024813455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9510101c90 0x7f9510199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:38.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.934+0000 7f95183fc640 1 -- 172.21.15.78:0/3024813455 >> 172.21.15.78:0/3024813455 conn(0x7f95100fd6e0 msgr2=0x7f95100ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:50:38.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.934+0000 7f95183fc640 1 -- 172.21.15.78:0/3024813455 shutdown_connections 2024-09-11T00:50:38.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.934+0000 7f95183fc640 1 -- 172.21.15.78:0/3024813455 wait complete. 2024-09-11T00:50:39.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:38 smithi078 ceph-mon[31315]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:39.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:38 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3024813455' entity='client.admin' 2024-09-11T00:50:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:38 smithi110 ceph-mon[34712]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:38 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3024813455' entity='client.admin' 2024-09-11T00:50:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:38 smithi112 ceph-mon[35384]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:38 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3024813455' entity='client.admin' 2024-09-11T00:50:40.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:40 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2004676821' entity='client.admin' 2024-09-11T00:50:40.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:40 smithi110 ceph-mon[34712]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:40.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:40 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2004676821' entity='client.admin' 2024-09-11T00:50:40.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:40 smithi112 ceph-mon[35384]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:40.483 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:40 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2004676821' entity='client.admin' 2024-09-11T00:50:40.483 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:40 smithi078 ceph-mon[31315]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:41.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.947+0000 7f932dcee640 1 -- 172.21.15.78:0/3220204947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93281045c0 msgr2=0x7f932810cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:41.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.947+0000 7f932dcee640 1 --2- 172.21.15.78:0/3220204947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93281045c0 0x7f932810cab0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f931c009900 tx=0x7f931c02a410 comp rx=0 tx=0).stop 2024-09-11T00:50:41.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.947+0000 7f932dcee640 1 -- 172.21.15.78:0/3220204947 shutdown_connections 2024-09-11T00:50:41.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.947+0000 7f932dcee640 1 --2- 172.21.15.78:0/3220204947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f932810d100 0x7f932810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:41.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.947+0000 7f932dcee640 1 --2- 172.21.15.78:0/3220204947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93281045c0 0x7f932810cab0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:41.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.947+0000 7f932dcee640 1 --2- 172.21.15.78:0/3220204947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328103ca0 0x7f9328104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:41.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.947+0000 7f932dcee640 1 -- 172.21.15.78:0/3220204947 >> 172.21.15.78:0/3220204947 conn(0x7f93280fd700 msgr2=0x7f93280ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:50:41.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.947+0000 7f932dcee640 1 -- 172.21.15.78:0/3220204947 shutdown_connections 2024-09-11T00:50:41.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.947+0000 7f932dcee640 1 -- 172.21.15.78:0/3220204947 wait complete. 2024-09-11T00:50:41.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.947+0000 7f932dcee640 1 Processor -- start 2024-09-11T00:50:41.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f932dcee640 1 -- start start 2024-09-11T00:50:41.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f932dcee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328103ca0 0x7f932819b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:41.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f932dcee640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93281045c0 0x7f932819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:41.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f932dcee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932810d100 0x7f93281a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:41.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f932dcee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9328112d70 con 0x7f932810d100 2024-09-11T00:50:41.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f932dcee640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9328112bf0 con 0x7f93281045c0 2024-09-11T00:50:41.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f932dcee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9328112ef0 con 0x7f9328103ca0 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f9327fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932810d100 0x7f93281a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f9327fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932810d100 0x7f93281a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33118/0 (socket says 172.21.15.78:33118) 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f9327fff640 1 -- 172.21.15.78:0/2004676821 learned_addr learned my addr 172.21.15.78:0/2004676821 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f9326ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93281045c0 0x7f932819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f93277fe640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328103ca0 0x7f932819b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f9327fff640 1 -- 172.21.15.78:0/2004676821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328103ca0 msgr2=0x7f932819b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f9327fff640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328103ca0 0x7f932819b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f9327fff640 1 -- 172.21.15.78:0/2004676821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93281045c0 msgr2=0x7f932819ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f9327fff640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93281045c0 0x7f932819ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f9327fff640 1 -- 172.21.15.78:0/2004676821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93281a15b0 con 0x7f932810d100 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.948+0000 7f9327fff640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932810d100 0x7f93281a0de0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f932000cad0 tx=0x7f932000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.949+0000 7f9324ff9640 1 -- 172.21.15.78:0/2004676821 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932000ae20 con 0x7f932810d100 2024-09-11T00:50:41.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.949+0000 7f9324ff9640 1 -- 172.21.15.78:0/2004676821 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9320005020 con 0x7f932810d100 2024-09-11T00:50:41.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.949+0000 7f932dcee640 1 -- 172.21.15.78:0/2004676821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93281a18a0 con 0x7f932810d100 2024-09-11T00:50:41.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.949+0000 7f9324ff9640 1 -- 172.21.15.78:0/2004676821 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932000f6b0 con 0x7f932810d100 2024-09-11T00:50:41.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.949+0000 7f932dcee640 1 -- 172.21.15.78:0/2004676821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93281a7f60 con 0x7f932810d100 2024-09-11T00:50:41.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.950+0000 7f932dcee640 1 -- 172.21.15.78:0/2004676821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9328104080 con 0x7f932810d100 2024-09-11T00:50:41.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.952+0000 7f9324ff9640 1 -- 172.21.15.78:0/2004676821 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f93200054f0 con 0x7f932810d100 2024-09-11T00:50:41.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.953+0000 7f9324ff9640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93100b52a0 0x7f93100b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:41.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.953+0000 7f93277fe640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93100b52a0 0x7f93100b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:41.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.953+0000 7f9324ff9640 1 -- 172.21.15.78:0/2004676821 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f93200e4370 con 0x7f932810d100 2024-09-11T00:50:41.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.953+0000 7f93277fe640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93100b52a0 0x7f93100b7760 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f93180058c0 tx=0x7f931800a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:50:41.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:38.958+0000 7f9324ff9640 1 -- 172.21.15.78:0/2004676821 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9320014030 con 0x7f932810d100 2024-09-11T00:50:41.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.141+0000 7f932dcee640 1 -- 172.21.15.78:0/2004676821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f932819c630 con 0x7f932810d100 2024-09-11T00:50:41.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.145+0000 7f9324ff9640 1 -- 172.21.15.78:0/2004676821 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v31)=0 v31) ==== 116+0+0 (secure 0 0 0) 0x7f932819c630 con 0x7f932810d100 2024-09-11T00:50:41.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.148+0000 7f932dcee640 1 -- 172.21.15.78:0/2004676821 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93100b52a0 msgr2=0x7f93100b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:41.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.148+0000 7f932dcee640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93100b52a0 0x7f93100b7760 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f93180058c0 tx=0x7f931800a2d0 comp rx=0 tx=0).stop 2024-09-11T00:50:41.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.148+0000 7f932dcee640 1 -- 172.21.15.78:0/2004676821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932810d100 msgr2=0x7f93281a0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:41.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.148+0000 7f932dcee640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932810d100 0x7f93281a0de0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f932000cad0 tx=0x7f932000cf90 comp rx=0 tx=0).stop 2024-09-11T00:50:41.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.148+0000 7f932dcee640 1 -- 172.21.15.78:0/2004676821 shutdown_connections 2024-09-11T00:50:41.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.148+0000 7f932dcee640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93100b52a0 0x7f93100b7760 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:41.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.148+0000 7f932dcee640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932810d100 0x7f93281a0de0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:41.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.148+0000 7f932dcee640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93281045c0 0x7f932819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:41.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.148+0000 7f932dcee640 1 --2- 172.21.15.78:0/2004676821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328103ca0 0x7f932819b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:41.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.148+0000 7f932dcee640 1 -- 172.21.15.78:0/2004676821 >> 172.21.15.78:0/2004676821 conn(0x7f93280fd700 msgr2=0x7f93280fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:50:41.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.148+0000 7f932dcee640 1 -- 172.21.15.78:0/2004676821 shutdown_connections 2024-09-11T00:50:41.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:39.148+0000 7f932dcee640 1 -- 172.21.15.78:0/2004676821 wait complete. 2024-09-11T00:50:41.301 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:41.302 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:41.476 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:43.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:43 smithi078 ceph-mon[31315]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:43.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.219+0000 7f75246f3640 1 -- 172.21.15.78:0/3313944470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c10d100 msgr2=0x7f751c10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:43.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.219+0000 7f75246f3640 1 --2- 172.21.15.78:0/3313944470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c10d100 0x7f751c10f560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7514009ca0 tx=0x7f751402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.219+0000 7f75246f3640 1 -- 172.21.15.78:0/3313944470 shutdown_connections 2024-09-11T00:50:43.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.219+0000 7f75246f3640 1 --2- 172.21.15.78:0/3313944470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c10d100 0x7f751c10f560 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.219+0000 7f75246f3640 1 --2- 172.21.15.78:0/3313944470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f751c1045c0 0x7f751c10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.219+0000 7f75246f3640 1 --2- 172.21.15.78:0/3313944470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f751c103ca0 0x7f751c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.219+0000 7f75246f3640 1 -- 172.21.15.78:0/3313944470 >> 172.21.15.78:0/3313944470 conn(0x7f751c0fd700 msgr2=0x7f751c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.219+0000 7f75246f3640 1 -- 172.21.15.78:0/3313944470 shutdown_connections 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.219+0000 7f75246f3640 1 -- 172.21.15.78:0/3313944470 wait complete. 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.219+0000 7f75246f3640 1 Processor -- start 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.219+0000 7f75246f3640 1 -- start start 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f75246f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f751c103ca0 0x7f751c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f75246f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f751c1045c0 0x7f751c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f75246f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c10d100 0x7f751c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f75246f3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f751c112d70 con 0x7f751c10d100 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f75246f3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f751c112bf0 con 0x7f751c1045c0 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f75246f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f751c112ef0 con 0x7f751c103ca0 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f7522c90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c10d100 0x7f751c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:43.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f7522c90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c10d100 0x7f751c19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33138/0 (socket says 172.21.15.78:33138) 2024-09-11T00:50:43.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f7522c90640 1 -- 172.21.15.78:0/3605925874 learned_addr learned my addr 172.21.15.78:0/3605925874 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:50:43.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f752248f640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f751c103ca0 0x7f751c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:43.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f7521c8e640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f751c1045c0 0x7f751c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:43.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f7522c90640 1 -- 172.21.15.78:0/3605925874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f751c103ca0 msgr2=0x7f751c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:43.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f7522c90640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f751c103ca0 0x7f751c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f7522c90640 1 -- 172.21.15.78:0/3605925874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f751c1045c0 msgr2=0x7f751c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:43.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f7522c90640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f751c1045c0 0x7f751c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f7522c90640 1 -- 172.21.15.78:0/3605925874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c19f310 con 0x7f751c10d100 2024-09-11T00:50:43.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.220+0000 7f7522c90640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c10d100 0x7f751c19eb40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7514005cc0 tx=0x7f751402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:50:43.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.221+0000 7f75137fe640 1 -- 172.21.15.78:0/3605925874 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514003bf0 con 0x7f751c10d100 2024-09-11T00:50:43.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.221+0000 7f75246f3640 1 -- 172.21.15.78:0/3605925874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751c19f5a0 con 0x7f751c10d100 2024-09-11T00:50:43.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.221+0000 7f75137fe640 1 -- 172.21.15.78:0/3605925874 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7514003d90 con 0x7f751c10d100 2024-09-11T00:50:43.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.221+0000 7f75137fe640 1 -- 172.21.15.78:0/3605925874 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751402fbc0 con 0x7f751c10d100 2024-09-11T00:50:43.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.221+0000 7f75246f3640 1 -- 172.21.15.78:0/3605925874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751c1a5cc0 con 0x7f751c10d100 2024-09-11T00:50:43.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.222+0000 7f75246f3640 1 -- 172.21.15.78:0/3605925874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f751c104080 con 0x7f751c10d100 2024-09-11T00:50:43.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.223+0000 7f75137fe640 1 -- 172.21.15.78:0/3605925874 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f7514004610 con 0x7f751c10d100 2024-09-11T00:50:43.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.223+0000 7f75137fe640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f75080b52a0 0x7f75080b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:43.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.224+0000 7f752248f640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f75080b52a0 0x7f75080b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:43.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.224+0000 7f75137fe640 1 -- 172.21.15.78:0/3605925874 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f75141055c0 con 0x7f751c10d100 2024-09-11T00:50:43.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.224+0000 7f752248f640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f75080b52a0 0x7f75080b7760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7518005fd0 tx=0x7f751800b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:50:43.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.227+0000 7f75137fe640 1 -- 172.21.15.78:0/3605925874 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f75140ce080 con 0x7f751c10d100 2024-09-11T00:50:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.409+0000 7f75246f3640 1 -- 172.21.15.78:0/3605925874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f751c1a6010 con 0x7f751c10d100 2024-09-11T00:50:43.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.413+0000 7f75137fe640 1 -- 172.21.15.78:0/3605925874 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v32)=0 v32) ==== 121+0+0 (secure 0 0 0) 0x7f75140cd980 con 0x7f751c10d100 2024-09-11T00:50:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.415+0000 7f75246f3640 1 -- 172.21.15.78:0/3605925874 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f75080b52a0 msgr2=0x7f75080b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.415+0000 7f75246f3640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f75080b52a0 0x7f75080b7760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7518005fd0 tx=0x7f751800b390 comp rx=0 tx=0).stop 2024-09-11T00:50:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.415+0000 7f75246f3640 1 -- 172.21.15.78:0/3605925874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c10d100 msgr2=0x7f751c19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.415+0000 7f75246f3640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c10d100 0x7f751c19eb40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7514005cc0 tx=0x7f751402f380 comp rx=0 tx=0).stop 2024-09-11T00:50:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.415+0000 7f75246f3640 1 -- 172.21.15.78:0/3605925874 shutdown_connections 2024-09-11T00:50:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.415+0000 7f75246f3640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f75080b52a0 0x7f75080b7760 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.415+0000 7f75246f3640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c10d100 0x7f751c19eb40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.415+0000 7f75246f3640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f751c1045c0 0x7f751c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.415+0000 7f75246f3640 1 --2- 172.21.15.78:0/3605925874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f751c103ca0 0x7f751c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.415+0000 7f75246f3640 1 -- 172.21.15.78:0/3605925874 >> 172.21.15.78:0/3605925874 conn(0x7f751c0fd700 msgr2=0x7f751c10f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:50:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.415+0000 7f75246f3640 1 -- 172.21.15.78:0/3605925874 shutdown_connections 2024-09-11T00:50:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.415+0000 7f75246f3640 1 -- 172.21.15.78:0/3605925874 wait complete. 2024-09-11T00:50:43.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:43 smithi110 ceph-mon[34712]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:43 smithi112 ceph-mon[35384]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:43.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.500+0000 7ff88b38e640 1 -- 172.21.15.78:0/838529123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8841076e0 msgr2=0x7ff884107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:43.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.500+0000 7ff88b38e640 1 --2- 172.21.15.78:0/838529123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8841076e0 0x7ff884107b20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff880009900 tx=0x7ff88002a410 comp rx=0 tx=0).stop 2024-09-11T00:50:43.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.500+0000 7ff88b38e640 1 -- 172.21.15.78:0/838529123 shutdown_connections 2024-09-11T00:50:43.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.500+0000 7ff88b38e640 1 --2- 172.21.15.78:0/838529123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff884108060 0x7ff88410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.500+0000 7ff88b38e640 1 --2- 172.21.15.78:0/838529123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8841076e0 0x7ff884107b20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.500+0000 7ff88b38e640 1 --2- 172.21.15.78:0/838529123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff884101cd0 0x7ff8841020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.500+0000 7ff88b38e640 1 -- 172.21.15.78:0/838529123 >> 172.21.15.78:0/838529123 conn(0x7ff8840fd700 msgr2=0x7ff8840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:50:43.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.500+0000 7ff88b38e640 1 -- 172.21.15.78:0/838529123 shutdown_connections 2024-09-11T00:50:43.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.500+0000 7ff88b38e640 1 -- 172.21.15.78:0/838529123 wait complete. 2024-09-11T00:50:43.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.500+0000 7ff88b38e640 1 Processor -- start 2024-09-11T00:50:43.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.500+0000 7ff88b38e640 1 -- start start 2024-09-11T00:50:43.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88b38e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff884101cd0 0x7ff8840fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:43.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88b38e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8841076e0 0x7ff8840fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:43.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88b38e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff884108060 0x7ff8840fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:43.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88b38e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff884111d60 con 0x7ff884101cd0 2024-09-11T00:50:43.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88b38e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff884111be0 con 0x7ff884108060 2024-09-11T00:50:43.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88b38e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff884111ee0 con 0x7ff8841076e0 2024-09-11T00:50:43.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88912a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff884101cd0 0x7ff8840fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:43.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88912a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff884101cd0 0x7ff8840fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50264/0 (socket says 172.21.15.78:50264) 2024-09-11T00:50:43.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88912a640 1 -- 172.21.15.78:0/1332946795 learned_addr learned my addr 172.21.15.78:0/1332946795 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:50:43.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff888929640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8841076e0 0x7ff8840fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:43.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88992b640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff884108060 0x7ff8840fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:43.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88912a640 1 -- 172.21.15.78:0/1332946795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8841076e0 msgr2=0x7ff8840fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:43.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88912a640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8841076e0 0x7ff8840fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88912a640 1 -- 172.21.15.78:0/1332946795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff884108060 msgr2=0x7ff8840fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:43.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88912a640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff884108060 0x7ff8840fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88912a640 1 -- 172.21.15.78:0/1332946795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8841a77f0 con 0x7ff884101cd0 2024-09-11T00:50:43.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88912a640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff884101cd0 0x7ff8840fdc00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff87800b4a0 tx=0x7ff87800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:50:43.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff8767fc640 1 -- 172.21.15.78:0/1332946795 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878014070 con 0x7ff884101cd0 2024-09-11T00:50:43.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff8767fc640 1 -- 172.21.15.78:0/1332946795 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff878010ac0 con 0x7ff884101cd0 2024-09-11T00:50:43.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88b38e640 1 -- 172.21.15.78:0/1332946795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8841a7ae0 con 0x7ff884101cd0 2024-09-11T00:50:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff8767fc640 1 -- 172.21.15.78:0/1332946795 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878017c30 con 0x7ff884101cd0 2024-09-11T00:50:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.501+0000 7ff88b38e640 1 -- 172.21.15.78:0/1332946795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8841a8070 con 0x7ff884101cd0 2024-09-11T00:50:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.502+0000 7ff88b38e640 1 -- 172.21.15.78:0/1332946795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff884102130 con 0x7ff884101cd0 2024-09-11T00:50:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.504+0000 7ff8767fc640 1 -- 172.21.15.78:0/1332946795 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7ff878010610 con 0x7ff884101cd0 2024-09-11T00:50:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.504+0000 7ff8767fc640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8700b5320 0x7ff8700b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.504+0000 7ff888929640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8700b5320 0x7ff8700b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.504+0000 7ff8767fc640 1 -- 172.21.15.78:0/1332946795 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff8780e38a0 con 0x7ff884101cd0 2024-09-11T00:50:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.505+0000 7ff888929640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8700b5320 0x7ff8700b77e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff880006010 tx=0x7ff880000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:50:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.507+0000 7ff8767fc640 1 -- 172.21.15.78:0/1332946795 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff8780ac360 con 0x7ff884101cd0 2024-09-11T00:50:43.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.690+0000 7ff88b38e640 1 -- 172.21.15.78:0/1332946795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7ff8840fef00 con 0x7ff884101cd0 2024-09-11T00:50:43.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.693+0000 7ff8767fc640 1 -- 172.21.15.78:0/1332946795 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v33)=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7ff8840fef00 con 0x7ff884101cd0 2024-09-11T00:50:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.696+0000 7ff88b38e640 1 -- 172.21.15.78:0/1332946795 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8700b5320 msgr2=0x7ff8700b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.696+0000 7ff88b38e640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8700b5320 0x7ff8700b77e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff880006010 tx=0x7ff880000bc0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.696+0000 7ff88b38e640 1 -- 172.21.15.78:0/1332946795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff884101cd0 msgr2=0x7ff8840fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.696+0000 7ff88b38e640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff884101cd0 0x7ff8840fdc00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff87800b4a0 tx=0x7ff87800b960 comp rx=0 tx=0).stop 2024-09-11T00:50:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.696+0000 7ff88b38e640 1 -- 172.21.15.78:0/1332946795 shutdown_connections 2024-09-11T00:50:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.696+0000 7ff88b38e640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8700b5320 0x7ff8700b77e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.696+0000 7ff88b38e640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff884108060 0x7ff8840fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.696+0000 7ff88b38e640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8841076e0 0x7ff8840fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.696+0000 7ff88b38e640 1 --2- 172.21.15.78:0/1332946795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff884101cd0 0x7ff8840fdc00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.696+0000 7ff88b38e640 1 -- 172.21.15.78:0/1332946795 >> 172.21.15.78:0/1332946795 conn(0x7ff8840fd700 msgr2=0x7ff884104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:50:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.696+0000 7ff88b38e640 1 -- 172.21.15.78:0/1332946795 shutdown_connections 2024-09-11T00:50:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:43.696+0000 7ff88b38e640 1 -- 172.21.15.78:0/1332946795 wait complete. 2024-09-11T00:50:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:44 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3605925874' entity='client.admin' 2024-09-11T00:50:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:44 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1332946795' entity='client.admin' 2024-09-11T00:50:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:44 smithi110 ceph-mon[34712]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:44.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:44 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3605925874' entity='client.admin' 2024-09-11T00:50:44.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:44 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1332946795' entity='client.admin' 2024-09-11T00:50:44.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:44 smithi112 ceph-mon[35384]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:44 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3605925874' entity='client.admin' 2024-09-11T00:50:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:44 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1332946795' entity='client.admin' 2024-09-11T00:50:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:44 smithi078 ceph-mon[31315]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:44.812 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi078') 2024-09-11T00:50:44.812 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi078') 2024-09-11T00:50:44.812 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi078') 2024-09-11T00:50:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.226+0000 7fcb5242f640 1 -- 172.21.15.78:0/378177167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c101cd0 msgr2=0x7fcb4c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.226+0000 7fcb5242f640 1 --2- 172.21.15.78:0/378177167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c101cd0 0x7fcb4c1020b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009900 tx=0x7fcb3c02a410 comp rx=0 tx=0).stop 2024-09-11T00:50:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.226+0000 7fcb5242f640 1 -- 172.21.15.78:0/378177167 shutdown_connections 2024-09-11T00:50:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.226+0000 7fcb5242f640 1 --2- 172.21.15.78:0/378177167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb4c108060 0x7fcb4c10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.226+0000 7fcb5242f640 1 --2- 172.21.15.78:0/378177167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c1076e0 0x7fcb4c107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.226+0000 7fcb5242f640 1 --2- 172.21.15.78:0/378177167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c101cd0 0x7fcb4c1020b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.226+0000 7fcb5242f640 1 -- 172.21.15.78:0/378177167 >> 172.21.15.78:0/378177167 conn(0x7fcb4c0fd700 msgr2=0x7fcb4c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:50:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.226+0000 7fcb5242f640 1 -- 172.21.15.78:0/378177167 shutdown_connections 2024-09-11T00:50:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.226+0000 7fcb5242f640 1 -- 172.21.15.78:0/378177167 wait complete. 2024-09-11T00:50:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.226+0000 7fcb5242f640 1 Processor -- start 2024-09-11T00:50:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.226+0000 7fcb5242f640 1 -- start start 2024-09-11T00:50:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb5242f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c101cd0 0x7fcb4c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb5242f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c1076e0 0x7fcb4c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb5242f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb4c108060 0x7fcb4c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb5242f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c111d20 con 0x7fcb4c1076e0 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb5242f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c111ba0 con 0x7fcb4c101cd0 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb5242f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c111ea0 con 0x7fcb4c108060 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c1076e0 0x7fcb4c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c1076e0 0x7fcb4c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50282/0 (socket says 172.21.15.78:50282) 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb43fff640 1 -- 172.21.15.78:0/1254577030 learned_addr learned my addr 172.21.15.78:0/1254577030 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb4bfff640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c101cd0 0x7fcb4c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb509cc640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb4c108060 0x7fcb4c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb43fff640 1 -- 172.21.15.78:0/1254577030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb4c108060 msgr2=0x7fcb4c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb43fff640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb4c108060 0x7fcb4c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb43fff640 1 -- 172.21.15.78:0/1254577030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c101cd0 msgr2=0x7fcb4c197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb43fff640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c101cd0 0x7fcb4c197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb43fff640 1 -- 172.21.15.78:0/1254577030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c1a7620 con 0x7fcb4c1076e0 2024-09-11T00:50:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb43fff640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c1076e0 0x7fcb4c198380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400c920 tx=0x7fcb3400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:50:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb49ffb640 1 -- 172.21.15.78:0/1254577030 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34007c70 con 0x7fcb4c1076e0 2024-09-11T00:50:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb49ffb640 1 -- 172.21.15.78:0/1254577030 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb34007e10 con 0x7fcb4c1076e0 2024-09-11T00:50:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb5242f640 1 -- 172.21.15.78:0/1254577030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4c1a7910 con 0x7fcb4c1076e0 2024-09-11T00:50:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb49ffb640 1 -- 172.21.15.78:0/1254577030 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34018820 con 0x7fcb4c1076e0 2024-09-11T00:50:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.227+0000 7fcb5242f640 1 -- 172.21.15.78:0/1254577030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4c1a7ea0 con 0x7fcb4c1076e0 2024-09-11T00:50:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.228+0000 7fcb5242f640 1 -- 172.21.15.78:0/1254577030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb4c1020b0 con 0x7fcb4c1076e0 2024-09-11T00:50:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.230+0000 7fcb49ffb640 1 -- 172.21.15.78:0/1254577030 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fcb3400f450 con 0x7fcb4c1076e0 2024-09-11T00:50:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.230+0000 7fcb49ffb640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcb380b5320 0x7fcb380b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.231+0000 7fcb4bfff640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcb380b5320 0x7fcb380b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.231+0000 7fcb49ffb640 1 -- 172.21.15.78:0/1254577030 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcb340e4320 con 0x7fcb4c1076e0 2024-09-11T00:50:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.231+0000 7fcb4bfff640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcb380b5320 0x7fcb380b77e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c031440 tx=0x7fcb3c005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:50:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.233+0000 7fcb49ffb640 1 -- 172.21.15.78:0/1254577030 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcb34014030 con 0x7fcb4c1076e0 2024-09-11T00:50:46.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:46 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:46.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:46 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:46.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:46 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:46.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:46 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:46.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:46 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:46.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:46 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:46.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:46 smithi078 ceph-mon[31315]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:46.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.411+0000 7fcb5242f640 1 -- 172.21.15.78:0/1254577030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fcb4c06b8a0 con 0x7fcb4c1076e0 2024-09-11T00:50:46.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.415+0000 7fcb49ffb640 1 -- 172.21.15.78:0/1254577030 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v34)=0 v34) ==== 108+0+0 (secure 0 0 0) 0x7fcb4c06b8a0 con 0x7fcb4c1076e0 2024-09-11T00:50:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.417+0000 7fcb5242f640 1 -- 172.21.15.78:0/1254577030 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcb380b5320 msgr2=0x7fcb380b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.417+0000 7fcb5242f640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcb380b5320 0x7fcb380b77e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c031440 tx=0x7fcb3c005e80 comp rx=0 tx=0).stop 2024-09-11T00:50:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.417+0000 7fcb5242f640 1 -- 172.21.15.78:0/1254577030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c1076e0 msgr2=0x7fcb4c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.417+0000 7fcb5242f640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c1076e0 0x7fcb4c198380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400c920 tx=0x7fcb3400cde0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.417+0000 7fcb5242f640 1 -- 172.21.15.78:0/1254577030 shutdown_connections 2024-09-11T00:50:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.417+0000 7fcb5242f640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcb380b5320 0x7fcb380b77e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.417+0000 7fcb5242f640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb4c108060 0x7fcb4c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.417+0000 7fcb5242f640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c1076e0 0x7fcb4c198380 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.417+0000 7fcb5242f640 1 --2- 172.21.15.78:0/1254577030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c101cd0 0x7fcb4c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.417+0000 7fcb5242f640 1 -- 172.21.15.78:0/1254577030 >> 172.21.15.78:0/1254577030 conn(0x7fcb4c0fd700 msgr2=0x7fcb4c0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:50:46.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.417+0000 7fcb5242f640 1 -- 172.21.15.78:0/1254577030 shutdown_connections 2024-09-11T00:50:46.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.417+0000 7fcb5242f640 1 -- 172.21.15.78:0/1254577030 wait complete. 2024-09-11T00:50:46.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.494+0000 7f4d6d088640 1 -- 172.21.15.78:0/2311016376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d681076e0 msgr2=0x7f4d68107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:46.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.494+0000 7f4d6d088640 1 --2- 172.21.15.78:0/2311016376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d681076e0 0x7f4d68107b20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009ca0 tx=0x7f4d5c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.494+0000 7f4d6d088640 1 -- 172.21.15.78:0/2311016376 shutdown_connections 2024-09-11T00:50:46.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.494+0000 7f4d6d088640 1 --2- 172.21.15.78:0/2311016376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d68108060 0x7f4d6810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.494+0000 7f4d6d088640 1 --2- 172.21.15.78:0/2311016376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d681076e0 0x7f4d68107b20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.494+0000 7f4d6d088640 1 --2- 172.21.15.78:0/2311016376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68101cd0 0x7f4d681020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.494+0000 7f4d6d088640 1 -- 172.21.15.78:0/2311016376 >> 172.21.15.78:0/2311016376 conn(0x7f4d680fd700 msgr2=0x7f4d680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:50:46.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.494+0000 7f4d6d088640 1 -- 172.21.15.78:0/2311016376 shutdown_connections 2024-09-11T00:50:46.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.494+0000 7f4d6d088640 1 -- 172.21.15.78:0/2311016376 wait complete. 2024-09-11T00:50:46.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.495+0000 7f4d6d088640 1 Processor -- start 2024-09-11T00:50:46.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.495+0000 7f4d6d088640 1 -- start start 2024-09-11T00:50:46.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.495+0000 7f4d6d088640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68101cd0 0x7f4d681992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:46.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.495+0000 7f4d6d088640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d681076e0 0x7f4d68199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:46.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.495+0000 7f4d6d088640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d68108060 0x7f4d6819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:46.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6d088640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68112e40 con 0x7f4d68108060 2024-09-11T00:50:46.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6d088640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68112cc0 con 0x7f4d681076e0 2024-09-11T00:50:46.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6d088640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68112fc0 con 0x7f4d68101cd0 2024-09-11T00:50:46.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d68108060 0x7f4d6819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:46.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d68108060 0x7f4d6819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50296/0 (socket says 172.21.15.78:50296) 2024-09-11T00:50:46.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6759e640 1 -- 172.21.15.78:0/2402308553 learned_addr learned my addr 172.21.15.78:0/2402308553 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:50:46.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d66d9d640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68101cd0 0x7f4d681992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:46.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6659c640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d681076e0 0x7f4d68199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:46.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6759e640 1 -- 172.21.15.78:0/2402308553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68101cd0 msgr2=0x7f4d681992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:46.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6759e640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68101cd0 0x7f4d681992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6759e640 1 -- 172.21.15.78:0/2402308553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d681076e0 msgr2=0x7f4d68199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:46.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6759e640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d681076e0 0x7f4d68199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6759e640 1 -- 172.21.15.78:0/2402308553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6819f3a0 con 0x7f4d68108060 2024-09-11T00:50:46.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6659c640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d681076e0 0x7f4d68199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:50:46.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6759e640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d68108060 0x7f4d6819ebd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000e970 tx=0x7f4d6000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:50:46.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d4ffff640 1 -- 172.21.15.78:0/2402308553 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000ccc0 con 0x7f4d68108060 2024-09-11T00:50:46.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d4ffff640 1 -- 172.21.15.78:0/2402308553 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d6000ce60 con 0x7f4d68108060 2024-09-11T00:50:46.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6d088640 1 -- 172.21.15.78:0/2402308553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6819f690 con 0x7f4d68108060 2024-09-11T00:50:46.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d4ffff640 1 -- 172.21.15.78:0/2402308553 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60010430 con 0x7f4d68108060 2024-09-11T00:50:46.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.496+0000 7f4d6d088640 1 -- 172.21.15.78:0/2402308553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d681a5dd0 con 0x7f4d68108060 2024-09-11T00:50:46.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.497+0000 7f4d6d088640 1 -- 172.21.15.78:0/2402308553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d681020b0 con 0x7f4d68108060 2024-09-11T00:50:46.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.498+0000 7f4d4ffff640 1 -- 172.21.15.78:0/2402308553 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f4d60004f20 con 0x7f4d68108060 2024-09-11T00:50:46.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.499+0000 7f4d4ffff640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4d500b5320 0x7f4d500b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:46.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.499+0000 7f4d66d9d640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4d500b5320 0x7f4d500b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:46.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.499+0000 7f4d4ffff640 1 -- 172.21.15.78:0/2402308553 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4d600e44e0 con 0x7f4d68108060 2024-09-11T00:50:46.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.500+0000 7f4d66d9d640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4d500b5320 0x7f4d500b77e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009990 tx=0x7f4d58009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:50:46.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.502+0000 7f4d4ffff640 1 -- 172.21.15.78:0/2402308553 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4d60056220 con 0x7f4d68108060 2024-09-11T00:50:46.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.682+0000 7f4d6d088640 1 -- 172.21.15.78:0/2402308553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f4d6819a420 con 0x7f4d68108060 2024-09-11T00:50:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.686+0000 7f4d4ffff640 1 -- 172.21.15.78:0/2402308553 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v35)=0 v35) ==== 127+0+0 (secure 0 0 0) 0x7f4d600acfa0 con 0x7f4d68108060 2024-09-11T00:50:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:46 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:46 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:46 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:46 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:46 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:46 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:46.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:46 smithi110 ceph-mon[34712]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:46.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.688+0000 7f4d6d088640 1 -- 172.21.15.78:0/2402308553 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4d500b5320 msgr2=0x7f4d500b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:46.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.688+0000 7f4d6d088640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4d500b5320 0x7f4d500b77e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009990 tx=0x7f4d58009110 comp rx=0 tx=0).stop 2024-09-11T00:50:46.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.688+0000 7f4d6d088640 1 -- 172.21.15.78:0/2402308553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d68108060 msgr2=0x7f4d6819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:46.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.688+0000 7f4d6d088640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d68108060 0x7f4d6819ebd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000e970 tx=0x7f4d6000ed70 comp rx=0 tx=0).stop 2024-09-11T00:50:46.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.688+0000 7f4d6d088640 1 -- 172.21.15.78:0/2402308553 shutdown_connections 2024-09-11T00:50:46.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.688+0000 7f4d6d088640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4d500b5320 0x7f4d500b77e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.688+0000 7f4d6d088640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d68108060 0x7f4d6819ebd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.688+0000 7f4d6d088640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d681076e0 0x7f4d68199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.688+0000 7f4d6d088640 1 --2- 172.21.15.78:0/2402308553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d68101cd0 0x7f4d681992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:46.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.688+0000 7f4d6d088640 1 -- 172.21.15.78:0/2402308553 >> 172.21.15.78:0/2402308553 conn(0x7f4d680fd700 msgr2=0x7f4d68104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:50:46.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.688+0000 7f4d6d088640 1 -- 172.21.15.78:0/2402308553 shutdown_connections 2024-09-11T00:50:46.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:46.688+0000 7f4d6d088640 1 -- 172.21.15.78:0/2402308553 wait complete. 2024-09-11T00:50:46.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:46 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:46.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:46 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:46.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:46 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:50:46.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:46 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:46.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:46 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:50:46.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:46 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:50:46.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:46 smithi112 ceph-mon[35384]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:47.264 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.264 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.264 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:47 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1254577030' entity='client.admin' 2024-09-11T00:50:47.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:47 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2402308553' entity='client.admin' 2024-09-11T00:50:47.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:47 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1254577030' entity='client.admin' 2024-09-11T00:50:47.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:47 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2402308553' entity='client.admin' 2024-09-11T00:50:47.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:47 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1254577030' entity='client.admin' 2024-09-11T00:50:47.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:47 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2402308553' entity='client.admin' 2024-09-11T00:50:47.777 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.777 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.777 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.778 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.778 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.778 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.778 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.778 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.778 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.778 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.778 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.779 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.779 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.779 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.779 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.779 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.779 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '64384694-6fd7-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi078') 2024-09-11T00:50:47.779 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T00:50:47.780 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:50:47.780 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T00:50:47.798 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T00:50:47.798 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-09-11T00:50:47.855 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T00:50:47.855 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:50:47.855 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-11T00:50:47.856 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:50:47.856 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:50:47.856 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-11 00:49:26.939193228 +0000 2024-09-11T00:50:47.856 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-11 00:44:50.565779124 +0000 2024-09-11T00:50:47.856 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-11 00:44:50.565779124 +0000 2024-09-11T00:50:47.856 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-11 00:44:50.565779124 +0000 2024-09-11T00:50:47.856 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T00:50:47.922 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T00:50:47.922 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T00:50:47.923 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.00024277 s, 2.1 MB/s 2024-09-11T00:50:47.924 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T00:50:47.982 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-09-11T00:50:48.039 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T00:50:48.039 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:50:48.039 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-11T00:50:48.039 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:50:48.039 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:50:48.039 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-11 00:49:26.967192803 +0000 2024-09-11T00:50:48.039 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-11 00:44:50.888774355 +0000 2024-09-11T00:50:48.039 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-11 00:44:50.888774355 +0000 2024-09-11T00:50:48.040 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-11 00:44:50.888774355 +0000 2024-09-11T00:50:48.040 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T00:50:48.103 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T00:50:48.104 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T00:50:48.104 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000204527 s, 2.5 MB/s 2024-09-11T00:50:48.105 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T00:50:48.162 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-09-11T00:50:48.218 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T00:50:48.218 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:50:48.218 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-11T00:50:48.218 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:50:48.218 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:50:48.218 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-11 00:49:26.994192392 +0000 2024-09-11T00:50:48.218 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-11 00:44:51.218769479 +0000 2024-09-11T00:50:48.218 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-11 00:44:51.218769479 +0000 2024-09-11T00:50:48.218 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-11 00:44:51.218769479 +0000 2024-09-11T00:50:48.219 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T00:50:48.281 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T00:50:48.282 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T00:50:48.282 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000189705 s, 2.7 MB/s 2024-09-11T00:50:48.283 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T00:50:48.340 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-09-11T00:50:48.395 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T00:50:48.395 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:50:48.395 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-11T00:50:48.396 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:50:48.396 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:50:48.396 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-11 00:49:27.021191982 +0000 2024-09-11T00:50:48.396 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-11 00:44:51.549764587 +0000 2024-09-11T00:50:48.396 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-11 00:44:51.549764587 +0000 2024-09-11T00:50:48.396 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-11 00:44:51.549764587 +0000 2024-09-11T00:50:48.396 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T00:50:48.458 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T00:50:48.458 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T00:50:48.458 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000198915 s, 2.6 MB/s 2024-09-11T00:50:48.460 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T00:50:48.516 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T00:50:48.516 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T00:50:48.536 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T00:50:48.536 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-09-11T00:50:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T00:50:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:50:48.596 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-11T00:50:48.596 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:50:48.596 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:50:48.596 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 00:50:24.383309430 +0000 2024-09-11T00:50:48.596 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 00:45:05.745573832 +0000 2024-09-11T00:50:48.596 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 00:45:05.745573832 +0000 2024-09-11T00:50:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 00:45:05.745573832 +0000 2024-09-11T00:50:48.597 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T00:50:48.665 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T00:50:48.665 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T00:50:48.665 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000102079 s, 5.0 MB/s 2024-09-11T00:50:48.667 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T00:50:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:48 smithi110 ceph-mon[34712]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:48.707 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-09-11T00:50:48.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:48 smithi112 ceph-mon[35384]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:48.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:48 smithi078 ceph-mon[31315]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:48.764 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T00:50:48.764 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:50:48.764 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-09-11T00:50:48.764 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:50:48.765 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:50:48.765 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 00:50:24.383309430 +0000 2024-09-11T00:50:48.765 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 00:45:06.051568314 +0000 2024-09-11T00:50:48.765 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 00:45:06.051568314 +0000 2024-09-11T00:50:48.765 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 00:45:06.051568314 +0000 2024-09-11T00:50:48.765 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T00:50:48.830 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T00:50:48.830 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T00:50:48.830 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000131519 s, 3.9 MB/s 2024-09-11T00:50:48.831 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T00:50:48.889 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-09-11T00:50:48.946 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T00:50:48.946 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:50:48.946 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-11T00:50:48.946 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:50:48.946 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:50:48.946 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 00:50:24.384309412 +0000 2024-09-11T00:50:48.946 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 00:45:06.373562508 +0000 2024-09-11T00:50:48.946 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 00:45:06.373562508 +0000 2024-09-11T00:50:48.946 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 00:45:06.373562508 +0000 2024-09-11T00:50:48.947 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T00:50:49.011 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T00:50:49.011 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T00:50:49.012 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000160862 s, 3.2 MB/s 2024-09-11T00:50:49.013 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T00:50:49.071 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-09-11T00:50:49.127 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T00:50:49.127 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:50:49.127 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-11T00:50:49.127 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:50:49.127 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:50:49.127 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 00:50:24.384309412 +0000 2024-09-11T00:50:49.127 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 00:45:06.687556845 +0000 2024-09-11T00:50:49.127 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 00:45:06.687556845 +0000 2024-09-11T00:50:49.127 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 00:45:06.687556845 +0000 2024-09-11T00:50:49.128 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T00:50:49.192 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T00:50:49.192 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T00:50:49.192 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000112346 s, 4.6 MB/s 2024-09-11T00:50:49.193 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T00:50:49.250 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-11T00:50:49.250 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T00:50:49.270 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T00:50:49.270 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-09-11T00:50:49.329 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T00:50:49.329 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:50:49.329 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-11T00:50:49.329 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:50:49.330 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:50:49.330 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-11 00:50:21.990953380 +0000 2024-09-11T00:50:49.330 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-11 00:45:10.258315583 +0000 2024-09-11T00:50:49.330 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-11 00:45:10.258315583 +0000 2024-09-11T00:50:49.330 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-11 00:45:10.258315583 +0000 2024-09-11T00:50:49.330 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T00:50:49.401 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-11T00:50:49.401 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-11T00:50:49.401 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000295546 s, 1.7 MB/s 2024-09-11T00:50:49.402 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T00:50:49.462 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-09-11T00:50:49.520 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T00:50:49.520 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:50:49.520 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-11T00:50:49.520 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:50:49.520 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:50:49.521 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-11 00:50:21.991953354 +0000 2024-09-11T00:50:49.521 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-11 00:45:10.579306976 +0000 2024-09-11T00:50:49.521 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-11 00:45:10.579306976 +0000 2024-09-11T00:50:49.521 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-11 00:45:10.579306976 +0000 2024-09-11T00:50:49.521 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T00:50:49.588 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-11T00:50:49.588 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-11T00:50:49.588 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000225701 s, 2.3 MB/s 2024-09-11T00:50:49.589 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T00:50:49.648 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-09-11T00:50:49.705 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T00:50:49.705 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:50:49.706 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-11T00:50:49.706 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:50:49.706 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:50:49.706 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-11 00:50:21.991953354 +0000 2024-09-11T00:50:49.706 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-11 00:45:10.926297671 +0000 2024-09-11T00:50:49.706 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-11 00:45:10.926297671 +0000 2024-09-11T00:50:49.706 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-11 00:45:10.926297671 +0000 2024-09-11T00:50:49.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T00:50:49.771 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-11T00:50:49.772 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-11T00:50:49.772 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000246661 s, 2.1 MB/s 2024-09-11T00:50:49.773 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T00:50:49.832 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-09-11T00:50:49.889 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T00:50:49.889 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:50:49.889 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-11T00:50:49.889 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:50:49.889 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:50:49.889 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-11 00:50:21.991953354 +0000 2024-09-11T00:50:49.890 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-11 00:45:11.273288367 +0000 2024-09-11T00:50:49.890 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-11 00:45:11.273288367 +0000 2024-09-11T00:50:49.890 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-11 00:45:11.273288367 +0000 2024-09-11T00:50:49.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T00:50:49.954 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-11T00:50:49.955 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-11T00:50:49.955 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000215305 s, 2.4 MB/s 2024-09-11T00:50:49.956 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T00:50:50.013 INFO:tasks.cephadm:Deploying osd.0 on smithi078 with /dev/vg_nvme/lv_4... 2024-09-11T00:50:50.014 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T00:50:50.264 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:51.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:51 smithi078 ceph-mon[31315]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:51 smithi110 ceph-mon[34712]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:51 smithi112 ceph-mon[35384]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:52.629 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:50:52.656 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_4 2024-09-11T00:50:52.900 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:50:53.289 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:53 smithi078 ceph-mon[31315]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:53.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.316+0000 7f0c26554640 1 -- 172.21.15.78:0/2453382226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c201087d0 msgr2=0x7f0c20108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:53.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.316+0000 7f0c26554640 1 --2- 172.21.15.78:0/2453382226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c201087d0 0x7f0c20108c10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009900 tx=0x7f0c1402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:50:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.316+0000 7f0c26554640 1 -- 172.21.15.78:0/2453382226 shutdown_connections 2024-09-11T00:50:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.316+0000 7f0c26554640 1 --2- 172.21.15.78:0/2453382226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c20109150 0x7f0c2010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.316+0000 7f0c26554640 1 --2- 172.21.15.78:0/2453382226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c201087d0 0x7f0c20108c10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.316+0000 7f0c26554640 1 --2- 172.21.15.78:0/2453382226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c20102dc0 0x7f0c201031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.316+0000 7f0c26554640 1 -- 172.21.15.78:0/2453382226 >> 172.21.15.78:0/2453382226 conn(0x7f0c200fe830 msgr2=0x7f0c20100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:50:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.316+0000 7f0c26554640 1 -- 172.21.15.78:0/2453382226 shutdown_connections 2024-09-11T00:50:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.316+0000 7f0c26554640 1 -- 172.21.15.78:0/2453382226 wait complete. 2024-09-11T00:50:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c26554640 1 Processor -- start 2024-09-11T00:50:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c26554640 1 -- start start 2024-09-11T00:50:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c26554640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102dc0 0x7f0c201991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c26554640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c201087d0 0x7f0c20199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c26554640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c20109150 0x7f0c2019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c26554640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c20112df0 con 0x7f0c20102dc0 2024-09-11T00:50:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c26554640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c20112c70 con 0x7f0c20109150 2024-09-11T00:50:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c26554640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c20112f70 con 0x7f0c201087d0 2024-09-11T00:50:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c1ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102dc0 0x7f0c201991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c1ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102dc0 0x7f0c201991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37538/0 (socket says 172.21.15.78:37538) 2024-09-11T00:50:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c1ffff640 1 -- 172.21.15.78:0/1644593050 learned_addr learned my addr 172.21.15.78:0/1644593050 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:50:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c1f7fe640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c201087d0 0x7f0c20199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c24af1640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c20109150 0x7f0c2019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c1ffff640 1 -- 172.21.15.78:0/1644593050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c201087d0 msgr2=0x7f0c20199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c1ffff640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c201087d0 0x7f0c20199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.317+0000 7f0c1ffff640 1 -- 172.21.15.78:0/1644593050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c20109150 msgr2=0x7f0c2019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:50:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.318+0000 7f0c1ffff640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c20109150 0x7f0c2019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:50:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.318+0000 7f0c1ffff640 1 -- 172.21.15.78:0/1644593050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2019f290 con 0x7f0c20102dc0 2024-09-11T00:50:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.318+0000 7f0c24af1640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c20109150 0x7f0c2019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:50:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.318+0000 7f0c1ffff640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102dc0 0x7f0c201991e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c920 tx=0x7f0c1000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:50:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.318+0000 7f0c1d7fa640 1 -- 172.21.15.78:0/1644593050 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10007c70 con 0x7f0c20102dc0 2024-09-11T00:50:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.318+0000 7f0c1d7fa640 1 -- 172.21.15.78:0/1644593050 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c10007e10 con 0x7f0c20102dc0 2024-09-11T00:50:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.318+0000 7f0c26554640 1 -- 172.21.15.78:0/1644593050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2019f580 con 0x7f0c20102dc0 2024-09-11T00:50:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.318+0000 7f0c1d7fa640 1 -- 172.21.15.78:0/1644593050 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10018820 con 0x7f0c20102dc0 2024-09-11T00:50:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.318+0000 7f0c26554640 1 -- 172.21.15.78:0/1644593050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c201a5c40 con 0x7f0c20102dc0 2024-09-11T00:50:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.319+0000 7f0c26554640 1 -- 172.21.15.78:0/1644593050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c201031a0 con 0x7f0c20102dc0 2024-09-11T00:50:53.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.320+0000 7f0c1d7fa640 1 -- 172.21.15.78:0/1644593050 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f0c1000f450 con 0x7f0c20102dc0 2024-09-11T00:50:53.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.321+0000 7f0c1d7fa640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c080b52a0 0x7f0c080b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:50:53.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.321+0000 7f0c1f7fe640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c080b52a0 0x7f0c080b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:50:53.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.321+0000 7f0c1d7fa640 1 -- 172.21.15.78:0/1644593050 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0c100e4160 con 0x7f0c20102dc0 2024-09-11T00:50:53.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.321+0000 7f0c1f7fe640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c080b52a0 0x7f0c080b7760 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c14005ab0 tx=0x7f0c14005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:50:53.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.325+0000 7f0c1d7fa640 1 -- 172.21.15.78:0/1644593050 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0c10014030 con 0x7f0c20102dc0 2024-09-11T00:50:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:53 smithi110 ceph-mon[34712]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:53 smithi112 ceph-mon[35384]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:50:53.509+0000 7f0c26554640 1 -- 172.21.15.78:0/1644593050 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0c20105db0 con 0x7f0c080b52a0 2024-09-11T00:50:54.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:50:54.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:50:54.207 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:50:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:50:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:50:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:50:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:50:55.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:55 smithi078 ceph-mon[31315]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:50:55.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:55 smithi078 ceph-mon[31315]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:55 smithi110 ceph-mon[34712]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:50:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:55 smithi110 ceph-mon[34712]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:55 smithi112 ceph-mon[35384]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:50:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:55 smithi112 ceph-mon[35384]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:56.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:56 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3006650387' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "beefc6d9-f74c-4200-96f2-ff697ad0d7dc"} : dispatch 2024-09-11T00:50:56.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:56 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3006650387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beefc6d9-f74c-4200-96f2-ff697ad0d7dc"}]': finished 2024-09-11T00:50:56.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:56 smithi078 ceph-mon[31315]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T00:50:56.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:56 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:50:56.341 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:56 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3006650387' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "beefc6d9-f74c-4200-96f2-ff697ad0d7dc"} : dispatch 2024-09-11T00:50:56.341 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:56 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3006650387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beefc6d9-f74c-4200-96f2-ff697ad0d7dc"}]': finished 2024-09-11T00:50:56.341 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:56 smithi110 ceph-mon[34712]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T00:50:56.341 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:56 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:50:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:56 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3006650387' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "beefc6d9-f74c-4200-96f2-ff697ad0d7dc"} : dispatch 2024-09-11T00:50:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:56 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3006650387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beefc6d9-f74c-4200-96f2-ff697ad0d7dc"}]': finished 2024-09-11T00:50:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:56 smithi112 ceph-mon[35384]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T00:50:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:56 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:50:57.375 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:57 smithi112 ceph-mon[35384]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:57.375 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:57 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/251242106' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:50:57.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:57 smithi110 ceph-mon[34712]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:57.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:57 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/251242106' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:50:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:57 smithi078 ceph-mon[31315]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:57 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/251242106' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:50:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:50:59 smithi110 ceph-mon[34712]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:50:59 smithi112 ceph-mon[35384]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:50:59.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:50:59 smithi078 ceph-mon[31315]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:01.118 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:01 smithi078 ceph-mon[31315]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:01 smithi110 ceph-mon[34712]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:01.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:01 smithi112 ceph-mon[35384]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:03.316 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:03 smithi078 ceph-mon[31315]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:03 smithi110 ceph-mon[34712]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:03 smithi112 ceph-mon[35384]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:04.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-11T00:51:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-11T00:51:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-11T00:51:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:05.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:05 smithi078 ceph-mon[31315]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:05.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:05 smithi078 ceph-mon[31315]: Deploying daemon osd.0 on smithi078 2024-09-11T00:51:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:05 smithi110 ceph-mon[34712]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:05 smithi110 ceph-mon[34712]: Deploying daemon osd.0 on smithi078 2024-09-11T00:51:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:05 smithi112 ceph-mon[35384]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:05 smithi112 ceph-mon[35384]: Deploying daemon osd.0 on smithi078 2024-09-11T00:51:07.295 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:07 smithi078 ceph-mon[31315]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:07 smithi110 ceph-mon[34712]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:07.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:07 smithi112 ceph-mon[35384]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:09.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:09 smithi078 ceph-mon[31315]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:09 smithi110 ceph-mon[34712]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:09.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:09 smithi112 ceph-mon[35384]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:11 smithi110 ceph-mon[34712]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:11 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:51:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:11 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:11 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:11 smithi112 ceph-mon[35384]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:11 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:51:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:11 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:11 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:11 smithi078 ceph-mon[31315]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:11 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:51:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:11 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:11 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:12 smithi110 ceph-mon[34712]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:12 smithi112 ceph-mon[35384]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:12.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:12 smithi078 ceph-mon[31315]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:14 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:14 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:14 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:14 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:51:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:14 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:14.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:14 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:14.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:14 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:14.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:14 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:14.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:14 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:51:14.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:14 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:14 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:14 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:14 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:14 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:51:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:14 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.633+0000 7f0c1d7fa640 1 -- 172.21.15.78:0/1644593050 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0c20105db0 con 0x7f0c080b52a0 2024-09-11T00:51:14.635 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 0 on host 'smithi078' 2024-09-11T00:51:14.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.636+0000 7f0c26554640 1 -- 172.21.15.78:0/1644593050 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c080b52a0 msgr2=0x7f0c080b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:51:14.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.636+0000 7f0c26554640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c080b52a0 0x7f0c080b7760 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c14005ab0 tx=0x7f0c14005a20 comp rx=0 tx=0).stop 2024-09-11T00:51:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.636+0000 7f0c26554640 1 -- 172.21.15.78:0/1644593050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102dc0 msgr2=0x7f0c201991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:51:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.636+0000 7f0c26554640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102dc0 0x7f0c201991e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c920 tx=0x7f0c1000cde0 comp rx=0 tx=0).stop 2024-09-11T00:51:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.636+0000 7f0c26554640 1 -- 172.21.15.78:0/1644593050 shutdown_connections 2024-09-11T00:51:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.636+0000 7f0c26554640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c080b52a0 0x7f0c080b7760 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.636+0000 7f0c26554640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c20109150 0x7f0c2019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.636+0000 7f0c26554640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c201087d0 0x7f0c20199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.636+0000 7f0c26554640 1 --2- 172.21.15.78:0/1644593050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102dc0 0x7f0c201991e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.636+0000 7f0c26554640 1 -- 172.21.15.78:0/1644593050 >> 172.21.15.78:0/1644593050 conn(0x7f0c200fe830 msgr2=0x7f0c20100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:51:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.636+0000 7f0c26554640 1 -- 172.21.15.78:0/1644593050 shutdown_connections 2024-09-11T00:51:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:51:14.636+0000 7f0c26554640 1 -- 172.21.15.78:0/1644593050 wait complete. 2024-09-11T00:51:15.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:15 smithi078 ceph-mon[31315]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:15.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:51:15.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:15.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:15.262 DEBUG:teuthology.orchestra.run.smithi078:osd.0> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T00:51:15.265 INFO:tasks.cephadm:Deploying osd.1 on smithi110 with /dev/vg_nvme/lv_4... 2024-09-11T00:51:15.265 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T00:51:15.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:15 smithi110 ceph-mon[34712]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:15.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:51:15.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:15.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:15.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:15 smithi112 ceph-mon[35384]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:15.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:51:15.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:15.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:15.567 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:51:16.577 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 00:51:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0[46957]: 2024-09-11T00:51:16.367+0000 7fe82729e640 -1 osd.0 0 log_to_monitors true 2024-09-11T00:51:17.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:17 smithi110 ceph-mon[34712]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:17.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:17 smithi110 ceph-mon[34712]: from='osd.0 [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-11T00:51:17.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:17 smithi110 ceph-mon[34712]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-11T00:51:17.301 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:17 smithi078 ceph-mon[31315]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:17.301 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:17 smithi078 ceph-mon[31315]: from='osd.0 [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-11T00:51:17.301 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:17 smithi078 ceph-mon[31315]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-11T00:51:17.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:17 smithi112 ceph-mon[35384]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:17.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:17 smithi112 ceph-mon[35384]: from='osd.0 [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-11T00:51:17.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:17 smithi112 ceph-mon[35384]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-11T00:51:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:51:18.117 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_4 2024-09-11T00:51:18.247 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0[46957]: 2024-09-11T00:51:18.064+0000 7fe823221640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T00:51:18.247 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0[46957]: 2024-09-11T00:51:18.068+0000 7fe81f048640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:51:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T00:51:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T00:51:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: from='osd.0 [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:51:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:51:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: Detected new or changed devices on smithi078 2024-09-11T00:51:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:51:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi078 to 17088M 2024-09-11T00:51:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:51:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:18 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:18.364 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:51:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T00:51:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T00:51:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: from='osd.0 [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:51:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:51:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: Detected new or changed devices on smithi078 2024-09-11T00:51:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:51:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi078 to 17088M 2024-09-11T00:51:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:18.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:18.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:51:18.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:18.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T00:51:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T00:51:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: from='osd.0 [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:51:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:51:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: Detected new or changed devices on smithi078 2024-09-11T00:51:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:51:18.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi078 to 17088M 2024-09-11T00:51:18.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:18.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:18.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:51:18.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:18.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.772+0000 7fbc24282640 1 -- 172.21.15.110:0/257127757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc1c075540 msgr2=0x7fbc1c075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:51:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.772+0000 7fbc24282640 1 --2- 172.21.15.110:0/257127757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc1c075540 0x7fbc1c075920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009900 tx=0x7fbc1802a410 comp rx=0 tx=0).stop 2024-09-11T00:51:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.772+0000 7fbc24282640 1 -- 172.21.15.110:0/257127757 shutdown_connections 2024-09-11T00:51:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.772+0000 7fbc24282640 1 --2- 172.21.15.110:0/257127757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc1c113b20 0x7fbc1c115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.772+0000 7fbc24282640 1 --2- 172.21.15.110:0/257127757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc1c075e60 0x7fbc1c1135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.772+0000 7fbc24282640 1 --2- 172.21.15.110:0/257127757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc1c075540 0x7fbc1c075920 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.772+0000 7fbc24282640 1 -- 172.21.15.110:0/257127757 >> 172.21.15.110:0/257127757 conn(0x7fbc1c0fe830 msgr2=0x7fbc1c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.772+0000 7fbc24282640 1 -- 172.21.15.110:0/257127757 shutdown_connections 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.772+0000 7fbc24282640 1 -- 172.21.15.110:0/257127757 wait complete. 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.772+0000 7fbc24282640 1 Processor -- start 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.772+0000 7fbc24282640 1 -- start start 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc24282640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc1c075540 0x7fbc1c19f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc24282640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc1c075e60 0x7fbc1c19fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc24282640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc1c113b20 0x7fbc1c1a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc24282640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1193a0 con 0x7fbc1c075540 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc24282640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c119220 con 0x7fbc1c113b20 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc24282640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c119520 con 0x7fbc1c075e60 2024-09-11T00:51:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2281f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc1c113b20 0x7fbc1c1a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2281f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc1c113b20 0x7fbc1c1a5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55538/0 (socket says 172.21.15.110:55538) 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2281f640 1 -- 172.21.15.110:0/2351058015 learned_addr learned my addr 172.21.15.110:0/2351058015 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2201e640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc1c075540 0x7fbc1c19f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2181d640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc1c075e60 0x7fbc1c19fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2281f640 1 -- 172.21.15.110:0/2351058015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc1c075e60 msgr2=0x7fbc1c19fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2281f640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc1c075e60 0x7fbc1c19fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2281f640 1 -- 172.21.15.110:0/2351058015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc1c075540 msgr2=0x7fbc1c19f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2281f640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc1c075540 0x7fbc1c19f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2281f640 1 -- 172.21.15.110:0/2351058015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c1a59e0 con 0x7fbc1c113b20 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2181d640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc1c075e60 0x7fbc1c19fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2201e640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc1c075540 0x7fbc1c19f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc2281f640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc1c113b20 0x7fbc1c1a5210 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400cab0 tx=0x7fbc1400cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:51:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc137fe640 1 -- 172.21.15.110:0/2351058015 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1400ae20 con 0x7fbc1c113b20 2024-09-11T00:51:18.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc137fe640 1 -- 172.21.15.110:0/2351058015 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc1400f900 con 0x7fbc1c113b20 2024-09-11T00:51:18.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc24282640 1 -- 172.21.15.110:0/2351058015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1c1a5cd0 con 0x7fbc1c113b20 2024-09-11T00:51:18.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc137fe640 1 -- 172.21.15.110:0/2351058015 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14018a50 con 0x7fbc1c113b20 2024-09-11T00:51:18.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.773+0000 7fbc24282640 1 -- 172.21.15.110:0/2351058015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1c1ac390 con 0x7fbc1c113b20 2024-09-11T00:51:18.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.774+0000 7fbc24282640 1 -- 172.21.15.110:0/2351058015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1c075920 con 0x7fbc1c113b20 2024-09-11T00:51:18.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.776+0000 7fbc137fe640 1 -- 172.21.15.110:0/2351058015 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fbc1400f450 con 0x7fbc1c113b20 2024-09-11T00:51:18.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.777+0000 7fbc137fe640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbc080b5320 0x7fbc080b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:51:18.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.777+0000 7fbc2201e640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbc080b5320 0x7fbc080b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:51:18.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.777+0000 7fbc137fe640 1 -- 172.21.15.110:0/2351058015 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1702+0+0 (secure 0 0 0) 0x7fbc140e49b0 con 0x7fbc1c113b20 2024-09-11T00:51:18.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.777+0000 7fbc2201e640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbc080b5320 0x7fbc080b77e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc18031440 tx=0x7fbc18005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:51:18.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.781+0000 7fbc137fe640 1 -- 172.21.15.110:0/2351058015 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbc14014030 con 0x7fbc1c113b20 2024-09-11T00:51:18.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:18.969+0000 7fbc24282640 1 -- 172.21.15.110:0/2351058015 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbc1c100b50 con 0x7fbc080b5320 2024-09-11T00:51:19.368 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:19 smithi110 ceph-mon[34712]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-11T00:51:19.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:19 smithi110 ceph-mon[34712]: osdmap e8: 1 total, 0 up, 1 in 2024-09-11T00:51:19.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:19.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:19.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:19 smithi110 ceph-mon[34712]: from='client.24254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:51:19.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:51:19.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:51:19.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:19 smithi112 ceph-mon[35384]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-11T00:51:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:19 smithi112 ceph-mon[35384]: osdmap e8: 1 total, 0 up, 1 in 2024-09-11T00:51:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:19 smithi112 ceph-mon[35384]: from='client.24254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:51:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:51:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:51:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:19.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:19 smithi078 ceph-mon[31315]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-11T00:51:19.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:19 smithi078 ceph-mon[31315]: osdmap e8: 1 total, 0 up, 1 in 2024-09-11T00:51:19.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:19.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:19.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:19 smithi078 ceph-mon[31315]: from='client.24254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:51:19.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:51:19.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:51:19.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:20.265 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:20 smithi110 ceph-mon[34712]: purged_snaps scrub starts 2024-09-11T00:51:20.266 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:20 smithi110 ceph-mon[34712]: purged_snaps scrub ok 2024-09-11T00:51:20.266 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:20 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:20.266 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:20 smithi110 ceph-mon[34712]: osd.0 [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053] boot 2024-09-11T00:51:20.266 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:20 smithi110 ceph-mon[34712]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T00:51:20.266 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:20 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:20.267 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:20 smithi110 ceph-mon[34712]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:20 smithi112 ceph-mon[35384]: purged_snaps scrub starts 2024-09-11T00:51:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:20 smithi112 ceph-mon[35384]: purged_snaps scrub ok 2024-09-11T00:51:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:20 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:20 smithi112 ceph-mon[35384]: osd.0 [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053] boot 2024-09-11T00:51:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:20 smithi112 ceph-mon[35384]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T00:51:20.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:20 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:20.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:20 smithi112 ceph-mon[35384]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:20 smithi078 ceph-mon[31315]: purged_snaps scrub starts 2024-09-11T00:51:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:20 smithi078 ceph-mon[31315]: purged_snaps scrub ok 2024-09-11T00:51:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:20 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:20 smithi078 ceph-mon[31315]: osd.0 [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053] boot 2024-09-11T00:51:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:20 smithi078 ceph-mon[31315]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T00:51:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:20 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:51:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:20 smithi078 ceph-mon[31315]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:51:21.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:21 smithi110 ceph-mon[34712]: osdmap e10: 1 total, 1 up, 1 in 2024-09-11T00:51:21.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:21 smithi112 ceph-mon[35384]: osdmap e10: 1 total, 1 up, 1 in 2024-09-11T00:51:21.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:21 smithi078 ceph-mon[31315]: osdmap e10: 1 total, 1 up, 1 in 2024-09-11T00:51:22.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:22 smithi110 ceph-mon[34712]: from='client.? 172.21.15.110:0/48988007' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d742e767-d673-4bb9-b624-8bc969f5ceb1"} : dispatch 2024-09-11T00:51:22.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:22 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d742e767-d673-4bb9-b624-8bc969f5ceb1"} : dispatch 2024-09-11T00:51:22.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:22 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d742e767-d673-4bb9-b624-8bc969f5ceb1"}]': finished 2024-09-11T00:51:22.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:22 smithi110 ceph-mon[34712]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T00:51:22.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:22 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:22.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:22 smithi110 ceph-mon[34712]: from='client.? 172.21.15.110:0/175183082' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:51:22.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:22 smithi110 ceph-mon[34712]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:22 smithi112 ceph-mon[35384]: from='client.? 172.21.15.110:0/48988007' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d742e767-d673-4bb9-b624-8bc969f5ceb1"} : dispatch 2024-09-11T00:51:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:22 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d742e767-d673-4bb9-b624-8bc969f5ceb1"} : dispatch 2024-09-11T00:51:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:22 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d742e767-d673-4bb9-b624-8bc969f5ceb1"}]': finished 2024-09-11T00:51:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:22 smithi112 ceph-mon[35384]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T00:51:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:22 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:22 smithi112 ceph-mon[35384]: from='client.? 172.21.15.110:0/175183082' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:51:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:22 smithi112 ceph-mon[35384]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:22 smithi078 ceph-mon[31315]: from='client.? 172.21.15.110:0/48988007' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d742e767-d673-4bb9-b624-8bc969f5ceb1"} : dispatch 2024-09-11T00:51:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:22 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d742e767-d673-4bb9-b624-8bc969f5ceb1"} : dispatch 2024-09-11T00:51:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:22 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d742e767-d673-4bb9-b624-8bc969f5ceb1"}]': finished 2024-09-11T00:51:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:22 smithi078 ceph-mon[31315]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T00:51:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:22 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:22 smithi078 ceph-mon[31315]: from='client.? 172.21.15.110:0/175183082' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:51:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:22 smithi078 ceph-mon[31315]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:25 smithi110 ceph-mon[34712]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:25.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:25 smithi112 ceph-mon[35384]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:25.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:25 smithi078 ceph-mon[31315]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:27.276 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:27 smithi112 ceph-mon[35384]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:27.326 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:27 smithi110 ceph-mon[34712]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:27.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:27 smithi078 ceph-mon[31315]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:29.220 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:29 smithi110 ceph-mon[34712]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:29 smithi112 ceph-mon[35384]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:29.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:29 smithi078 ceph-mon[31315]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-11T00:51:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:30.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-11T00:51:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:30.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-11T00:51:30.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:31.086 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:31 smithi110 ceph-mon[34712]: Deploying daemon osd.1 on smithi110 2024-09-11T00:51:31.086 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:31 smithi110 ceph-mon[34712]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:31 smithi112 ceph-mon[35384]: Deploying daemon osd.1 on smithi110 2024-09-11T00:51:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:31 smithi112 ceph-mon[35384]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:31.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:31 smithi078 ceph-mon[31315]: Deploying daemon osd.1 on smithi110 2024-09-11T00:51:31.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:31 smithi078 ceph-mon[31315]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:33.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:33 smithi110 ceph-mon[34712]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:33 smithi112 ceph-mon[35384]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:33.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:33 smithi078 ceph-mon[31315]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:35.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:35 smithi110 ceph-mon[34712]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:35.299 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:35 smithi078 ceph-mon[31315]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:35.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:35 smithi112 ceph-mon[35384]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:36.371 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:36 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:51:36.371 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:36 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:36.371 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:36 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:36.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:36 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:51:36.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:36 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:36.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:36 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:36.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:36 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:51:36.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:36 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:36.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:36 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:37.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:37 smithi110 ceph-mon[34712]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:37 smithi112 ceph-mon[35384]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:37.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:37 smithi078 ceph-mon[31315]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:39 smithi110 ceph-mon[34712]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:39 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:39 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:39 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:39 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:51:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:39 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:39 smithi112 ceph-mon[35384]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:39 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:39 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:39 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:39 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:51:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:39 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:39.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:39 smithi078 ceph-mon[31315]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:39.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:39 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:39.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:39 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:39.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:39 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:39.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:39 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:51:39.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:39 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:39.833 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 1 on host 'smithi110' 2024-09-11T00:51:39.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.829+0000 7fbc137fe640 1 -- 172.21.15.110:0/2351058015 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbc1c100b50 con 0x7fbc080b5320 2024-09-11T00:51:39.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.832+0000 7fbc24282640 1 -- 172.21.15.110:0/2351058015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbc080b5320 msgr2=0x7fbc080b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:51:39.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.832+0000 7fbc24282640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbc080b5320 0x7fbc080b77e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc18031440 tx=0x7fbc18005a20 comp rx=0 tx=0).stop 2024-09-11T00:51:39.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.832+0000 7fbc24282640 1 -- 172.21.15.110:0/2351058015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc1c113b20 msgr2=0x7fbc1c1a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:51:39.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.832+0000 7fbc24282640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc1c113b20 0x7fbc1c1a5210 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400cab0 tx=0x7fbc1400cf70 comp rx=0 tx=0).stop 2024-09-11T00:51:39.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.832+0000 7fbc24282640 1 -- 172.21.15.110:0/2351058015 shutdown_connections 2024-09-11T00:51:39.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.832+0000 7fbc24282640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbc080b5320 0x7fbc080b77e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:39.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.832+0000 7fbc24282640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc1c113b20 0x7fbc1c1a5210 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:39.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.832+0000 7fbc24282640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc1c075e60 0x7fbc1c19fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:39.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.832+0000 7fbc24282640 1 --2- 172.21.15.110:0/2351058015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc1c075540 0x7fbc1c19f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:39.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.832+0000 7fbc24282640 1 -- 172.21.15.110:0/2351058015 >> 172.21.15.110:0/2351058015 conn(0x7fbc1c0fe830 msgr2=0x7fbc1c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:51:39.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.832+0000 7fbc24282640 1 -- 172.21.15.110:0/2351058015 shutdown_connections 2024-09-11T00:51:39.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:51:39.832+0000 7fbc24282640 1 -- 172.21.15.110:0/2351058015 wait complete. 2024-09-11T00:51:40.121 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:40 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:51:40.121 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:40 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:40.121 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:40 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:40.414 DEBUG:teuthology.orchestra.run.smithi110:osd.1> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T00:51:40.416 INFO:tasks.cephadm:Deploying osd.2 on smithi112 with /dev/vg_nvme/lv_4... 2024-09-11T00:51:40.417 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T00:51:40.443 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:40 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:51:40.443 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:40 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:40.443 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:40 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:40.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:40 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:51:40.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:40 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:40.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:40 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:40.666 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:51:41.437 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:41 smithi112 ceph-mon[35384]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:41.437 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:41 smithi112 ceph-mon[35384]: from='osd.1 [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-11T00:51:41.437 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:41 smithi112 ceph-mon[35384]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-11T00:51:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:41 smithi110 ceph-mon[34712]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:41 smithi110 ceph-mon[34712]: from='osd.1 [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-11T00:51:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:41 smithi110 ceph-mon[34712]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-11T00:51:41.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:41 smithi078 ceph-mon[31315]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:41.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:41 smithi078 ceph-mon[31315]: from='osd.1 [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-11T00:51:41.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:41 smithi078 ceph-mon[31315]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-11T00:51:42.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:42 smithi110 ceph-mon[34712]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T00:51:42.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:42 smithi110 ceph-mon[34712]: from='osd.1 [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:51:42.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:42 smithi110 ceph-mon[34712]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T00:51:42.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:42.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:42 smithi110 ceph-mon[34712]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:51:42.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:42 smithi110 ceph-mon[34712]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T00:51:42.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:42 smithi110 ceph-mon[34712]: osdmap e13: 2 total, 1 up, 2 in 2024-09-11T00:51:42.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:42.375 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:42 smithi112 ceph-mon[35384]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T00:51:42.375 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:42 smithi112 ceph-mon[35384]: from='osd.1 [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:51:42.375 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:42 smithi112 ceph-mon[35384]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T00:51:42.375 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:42.375 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:42 smithi112 ceph-mon[35384]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:51:42.375 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:42 smithi112 ceph-mon[35384]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T00:51:42.376 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:42 smithi112 ceph-mon[35384]: osdmap e13: 2 total, 1 up, 2 in 2024-09-11T00:51:42.376 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:42.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:42 smithi078 ceph-mon[31315]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T00:51:42.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:42 smithi078 ceph-mon[31315]: from='osd.1 [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:51:42.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:42 smithi078 ceph-mon[31315]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T00:51:42.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:42.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:42 smithi078 ceph-mon[31315]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:51:42.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:42 smithi078 ceph-mon[31315]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T00:51:42.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:42 smithi078 ceph-mon[31315]: osdmap e13: 2 total, 1 up, 2 in 2024-09-11T00:51:42.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:42.941 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 00:51:42 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1[40292]: 2024-09-11T00:51:42.469+0000 7f13132c8640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T00:51:42.941 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 00:51:42 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1[40292]: 2024-09-11T00:51:42.473+0000 7f130a8d3640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:51:43.106 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:51:43.134 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-09-11T00:51:43.198 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:43 smithi112 ceph-mon[35384]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:43.198 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:43.198 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:43 smithi112 ceph-mon[35384]: from='osd.1 ' entity='osd.1' 2024-09-11T00:51:43.198 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:43.261 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:43 smithi110 ceph-mon[34712]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:43.261 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:43.261 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:43 smithi110 ceph-mon[34712]: from='osd.1 ' entity='osd.1' 2024-09-11T00:51:43.262 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:43.426 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:51:43.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:43 smithi078 ceph-mon[31315]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:43.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:43.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:43 smithi078 ceph-mon[31315]: from='osd.1 ' entity='osd.1' 2024-09-11T00:51:43.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:43.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.889+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/3750770089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c24103ca0 msgr2=0x7f6c24104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:51:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.889+0000 7f6c2c5b3640 1 --2- 172.21.15.112:0/3750770089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c24103ca0 0x7f6c24104080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009900 tx=0x7f6c2002a410 comp rx=0 tx=0).stop 2024-09-11T00:51:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.889+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/3750770089 shutdown_connections 2024-09-11T00:51:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.889+0000 7f6c2c5b3640 1 --2- 172.21.15.112:0/3750770089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c2410d100 0x7f6c2410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.889+0000 7f6c2c5b3640 1 --2- 172.21.15.112:0/3750770089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c241045c0 0x7f6c2410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.889+0000 7f6c2c5b3640 1 --2- 172.21.15.112:0/3750770089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c24103ca0 0x7f6c24104080 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.889+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/3750770089 >> 172.21.15.112:0/3750770089 conn(0x7f6c240fd700 msgr2=0x7f6c240ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:51:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.889+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/3750770089 shutdown_connections 2024-09-11T00:51:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.889+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/3750770089 wait complete. 2024-09-11T00:51:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.890+0000 7f6c2c5b3640 1 Processor -- start 2024-09-11T00:51:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.890+0000 7f6c2c5b3640 1 -- start start 2024-09-11T00:51:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.890+0000 7f6c2c5b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c24103ca0 0x7f6c24199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:51:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.890+0000 7f6c2c5b3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c241045c0 0x7f6c241997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:51:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.890+0000 7f6c2c5b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2410d100 0x7f6c2419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:51:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.890+0000 7f6c2c5b3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c24112d70 con 0x7f6c2410d100 2024-09-11T00:51:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.890+0000 7f6c2c5b3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c24112bf0 con 0x7f6c241045c0 2024-09-11T00:51:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.890+0000 7f6c2c5b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c24112ef0 con 0x7f6c24103ca0 2024-09-11T00:51:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.890+0000 7f6c2a34f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c24103ca0 0x7f6c24199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:51:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.890+0000 7f6c2a34f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c24103ca0 0x7f6c24199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41222/0 (socket says 172.21.15.112:41222) 2024-09-11T00:51:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.890+0000 7f6c2a34f640 1 -- 172.21.15.112:0/1049374975 learned_addr learned my addr 172.21.15.112:0/1049374975 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-11T00:51:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.891+0000 7f6c29b4e640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c241045c0 0x7f6c241997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:51:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.891+0000 7f6c2a34f640 1 -- 172.21.15.112:0/1049374975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c241045c0 msgr2=0x7f6c241997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:51:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.891+0000 7f6c2a34f640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c241045c0 0x7f6c241997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.891+0000 7f6c2a34f640 1 -- 172.21.15.112:0/1049374975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2410d100 msgr2=0x7f6c2419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:51:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.891+0000 7f6c2a34f640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2410d100 0x7f6c2419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:51:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.891+0000 7f6c2a34f640 1 -- 172.21.15.112:0/1049374975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2419f310 con 0x7f6c24103ca0 2024-09-11T00:51:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.891+0000 7f6c2a34f640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c24103ca0 0x7f6c24199260 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6c20005940 tx=0x7f6c200042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:51:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.891+0000 7f6c1b7fe640 1 -- 172.21.15.112:0/1049374975 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2003f030 con 0x7f6c24103ca0 2024-09-11T00:51:43.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.891+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/1049374975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2419f5a0 con 0x7f6c24103ca0 2024-09-11T00:51:43.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.891+0000 7f6c1b7fe640 1 -- 172.21.15.112:0/1049374975 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c20003cb0 con 0x7f6c24103ca0 2024-09-11T00:51:43.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.891+0000 7f6c1b7fe640 1 -- 172.21.15.112:0/1049374975 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20035070 con 0x7f6c24103ca0 2024-09-11T00:51:43.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.891+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/1049374975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c241a5cc0 con 0x7f6c24103ca0 2024-09-11T00:51:43.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.892+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/1049374975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c24104080 con 0x7f6c24103ca0 2024-09-11T00:51:43.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.894+0000 7f6c1b7fe640 1 -- 172.21.15.112:0/1049374975 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f6c2003bbf0 con 0x7f6c24103ca0 2024-09-11T00:51:43.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.894+0000 7f6c1b7fe640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6c100b52a0 0x7f6c100b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:51:43.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.894+0000 7f6c1b7fe640 1 -- 172.21.15.112:0/1049374975 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2524+0+0 (secure 0 0 0) 0x7f6c20105360 con 0x7f6c24103ca0 2024-09-11T00:51:43.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.895+0000 7f6c29b4e640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6c100b52a0 0x7f6c100b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:51:43.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.895+0000 7f6c29b4e640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6c100b52a0 0x7f6c100b7760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6c14005db0 tx=0x7f6c14005d40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:51:43.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:43.901+0000 7f6c1b7fe640 1 -- 172.21.15.112:0/1049374975 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6c200cd9a0 con 0x7f6c24103ca0 2024-09-11T00:51:44.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:51:44.087+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/1049374975 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6c240ffa20 con 0x7f6c100b52a0 2024-09-11T00:51:44.389 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: purged_snaps scrub starts 2024-09-11T00:51:44.389 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: purged_snaps scrub ok 2024-09-11T00:51:44.389 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: Detected new or changed devices on smithi110 2024-09-11T00:51:44.389 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:44.389 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:44.389 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:51:44.389 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi110 to 17088M 2024-09-11T00:51:44.390 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:44.390 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:44.390 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:51:44.390 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:44.390 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: osd.1 [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385] boot 2024-09-11T00:51:44.390 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T00:51:44.390 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:44.390 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:44.390 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:51:44.390 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:51:44.390 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: purged_snaps scrub starts 2024-09-11T00:51:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: purged_snaps scrub ok 2024-09-11T00:51:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: Detected new or changed devices on smithi110 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi110 to 17088M 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: osd.1 [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385] boot 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:51:44.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:51:44.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: purged_snaps scrub starts 2024-09-11T00:51:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: purged_snaps scrub ok 2024-09-11T00:51:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: Detected new or changed devices on smithi110 2024-09-11T00:51:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:51:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi110 to 17088M 2024-09-11T00:51:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:44.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:44.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:51:44.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:51:44.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: osd.1 [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385] boot 2024-09-11T00:51:44.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T00:51:44.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:51:44.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:51:44.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:51:44.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:51:44.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:45.700 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:45 smithi112 ceph-mon[35384]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:51:45.700 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:45 smithi112 ceph-mon[35384]: osdmap e15: 2 total, 2 up, 2 in 2024-09-11T00:51:45.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:45 smithi078 ceph-mon[31315]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:51:45.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:45 smithi078 ceph-mon[31315]: osdmap e15: 2 total, 2 up, 2 in 2024-09-11T00:51:45.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:45 smithi110 ceph-mon[34712]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:51:45.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:45 smithi110 ceph-mon[34712]: osdmap e15: 2 total, 2 up, 2 in 2024-09-11T00:51:46.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:46 smithi078 ceph-mon[31315]: pgmap v48: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:46.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:46 smithi078 ceph-mon[31315]: from='client.? 172.21.15.112:0/2470859315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b0106a5b-1465-4a50-96dd-934dbae49a4d"} : dispatch 2024-09-11T00:51:46.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:46 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b0106a5b-1465-4a50-96dd-934dbae49a4d"} : dispatch 2024-09-11T00:51:46.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:46 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0106a5b-1465-4a50-96dd-934dbae49a4d"}]': finished 2024-09-11T00:51:46.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:46 smithi078 ceph-mon[31315]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T00:51:46.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:46 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:51:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:46 smithi110 ceph-mon[34712]: pgmap v48: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:46 smithi110 ceph-mon[34712]: from='client.? 172.21.15.112:0/2470859315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b0106a5b-1465-4a50-96dd-934dbae49a4d"} : dispatch 2024-09-11T00:51:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:46 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b0106a5b-1465-4a50-96dd-934dbae49a4d"} : dispatch 2024-09-11T00:51:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:46 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0106a5b-1465-4a50-96dd-934dbae49a4d"}]': finished 2024-09-11T00:51:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:46 smithi110 ceph-mon[34712]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T00:51:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:46 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:51:46.968 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:46 smithi112 ceph-mon[35384]: pgmap v48: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:46.968 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:46 smithi112 ceph-mon[35384]: from='client.? 172.21.15.112:0/2470859315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b0106a5b-1465-4a50-96dd-934dbae49a4d"} : dispatch 2024-09-11T00:51:46.969 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:46 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b0106a5b-1465-4a50-96dd-934dbae49a4d"} : dispatch 2024-09-11T00:51:46.969 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:46 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0106a5b-1465-4a50-96dd-934dbae49a4d"}]': finished 2024-09-11T00:51:46.969 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:46 smithi112 ceph-mon[35384]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T00:51:46.969 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:46 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:51:47.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:47 smithi078 ceph-mon[31315]: from='client.? 172.21.15.112:0/1282913010' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:51:47.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:47 smithi110 ceph-mon[34712]: from='client.? 172.21.15.112:0/1282913010' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:51:47.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:47 smithi112 ceph-mon[35384]: from='client.? 172.21.15.112:0/1282913010' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:51:48.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:48 smithi078 ceph-mon[31315]: pgmap v50: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:48.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:48 smithi110 ceph-mon[34712]: pgmap v50: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:48.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:48 smithi112 ceph-mon[35384]: pgmap v50: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:51 smithi110 ceph-mon[34712]: pgmap v51: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:51 smithi112 ceph-mon[35384]: pgmap v51: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:51.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:51 smithi078 ceph-mon[31315]: pgmap v51: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:53.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:53 smithi112 ceph-mon[35384]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:53 smithi110 ceph-mon[34712]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:53 smithi078 ceph-mon[31315]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:55.355 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:55 smithi112 ceph-mon[35384]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:55.356 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:55 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-11T00:51:55.356 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:55 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:55 smithi110 ceph-mon[34712]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:55 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-11T00:51:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:55 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:55.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:55 smithi078 ceph-mon[31315]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:55.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:55 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-11T00:51:55.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:55 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:51:56.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:56 smithi112 ceph-mon[35384]: Deploying daemon osd.2 on smithi112 2024-09-11T00:51:56.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:56 smithi110 ceph-mon[34712]: Deploying daemon osd.2 on smithi112 2024-09-11T00:51:56.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:56 smithi078 ceph-mon[31315]: Deploying daemon osd.2 on smithi112 2024-09-11T00:51:57.398 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:57 smithi112 ceph-mon[35384]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:57.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:57 smithi110 ceph-mon[34712]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:57.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:57 smithi078 ceph-mon[31315]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:59.291 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:51:59 smithi112 ceph-mon[35384]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:51:59 smithi110 ceph-mon[34712]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:51:59.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:51:59 smithi078 ceph-mon[31315]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:01.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:01 smithi112 ceph-mon[35384]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:01 smithi110 ceph-mon[34712]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:01.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:01 smithi078 ceph-mon[31315]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:02.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:02 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:02.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:02 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:02.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:02 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:02 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:02 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:02 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:02 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:02 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:02 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:03.339 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:03 smithi112 ceph-mon[35384]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:03 smithi110 ceph-mon[34712]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:03 smithi078 ceph-mon[31315]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:04.786 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:04 smithi112 ceph-mon[35384]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:04.825 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 2 on host 'smithi112' 2024-09-11T00:52:04.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.823+0000 7f6c1b7fe640 1 -- 172.21.15.112:0/1049374975 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6c240ffa20 con 0x7f6c100b52a0 2024-09-11T00:52:04.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.825+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/1049374975 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6c100b52a0 msgr2=0x7f6c100b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:52:04.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.825+0000 7f6c2c5b3640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6c100b52a0 0x7f6c100b7760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6c14005db0 tx=0x7f6c14005d40 comp rx=0 tx=0).stop 2024-09-11T00:52:04.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.825+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/1049374975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c24103ca0 msgr2=0x7f6c24199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:52:04.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.825+0000 7f6c2c5b3640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c24103ca0 0x7f6c24199260 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6c20005940 tx=0x7f6c200042f0 comp rx=0 tx=0).stop 2024-09-11T00:52:04.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.825+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/1049374975 shutdown_connections 2024-09-11T00:52:04.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.825+0000 7f6c2c5b3640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6c100b52a0 0x7f6c100b7760 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:04.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.825+0000 7f6c2c5b3640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2410d100 0x7f6c2419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:04.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.825+0000 7f6c2c5b3640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c241045c0 0x7f6c241997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:04.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.825+0000 7f6c2c5b3640 1 --2- 172.21.15.112:0/1049374975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c24103ca0 0x7f6c24199260 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:04.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.825+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/1049374975 >> 172.21.15.112:0/1049374975 conn(0x7f6c240fd700 msgr2=0x7f6c240fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:52:04.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.826+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/1049374975 shutdown_connections 2024-09-11T00:52:04.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:52:04.826+0000 7f6c2c5b3640 1 -- 172.21.15.112:0/1049374975 wait complete. 2024-09-11T00:52:04.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:04.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:04.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:04.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:04.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:04.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:04 smithi078 ceph-mon[31315]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:05.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:05.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:05.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:05.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:04 smithi110 ceph-mon[34712]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:05.544 DEBUG:teuthology.orchestra.run.smithi112:osd.2> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T00:52:05.588 INFO:tasks.cephadm:Deploying osd.3 on smithi078 with /dev/vg_nvme/lv_3... 2024-09-11T00:52:05.588 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T00:52:05.777 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:05 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:05.777 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:05 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:05.777 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:05 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:05.843 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:52:05.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:05 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:05.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:05 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:05 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:06.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:05 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:06.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:05 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:06.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:05 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:06.929 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:06 smithi112 ceph-mon[35384]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:06.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:06 smithi078 ceph-mon[31315]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:07.078 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:06 smithi110 ceph-mon[34712]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:07.474 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 00:52:07 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2[41413]: 2024-09-11T00:52:07.177+0000 7f125b785640 -1 osd.2 0 log_to_monitors true 2024-09-11T00:52:07.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:07 smithi110 ceph-mon[34712]: from='osd.2 [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-11T00:52:07.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:07 smithi110 ceph-mon[34712]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-11T00:52:07.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:07 smithi112 ceph-mon[35384]: from='osd.2 [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-11T00:52:07.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:07 smithi112 ceph-mon[35384]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-11T00:52:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:07 smithi078 ceph-mon[31315]: from='osd.2 [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-11T00:52:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:07 smithi078 ceph-mon[31315]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-11T00:52:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:52:08.376 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_3 2024-09-11T00:52:08.623 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:52:08.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T00:52:08.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: from='osd.2 [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:52:08.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T00:52:08.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:08.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:52:08.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:08.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:08.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:08.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:52:08.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:08.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:08.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:08.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:09.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.082+0000 7fe762717640 1 -- 172.21.15.78:0/654995844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c1076e0 msgr2=0x7fe75c107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:52:09.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.082+0000 7fe762717640 1 --2- 172.21.15.78:0/654995844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c1076e0 0x7fe75c107b20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe744009900 tx=0x7fe74402a410 comp rx=0 tx=0).stop 2024-09-11T00:52:09.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.082+0000 7fe762717640 1 -- 172.21.15.78:0/654995844 shutdown_connections 2024-09-11T00:52:09.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.082+0000 7fe762717640 1 --2- 172.21.15.78:0/654995844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108060 0x7fe75c10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:09.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.082+0000 7fe762717640 1 --2- 172.21.15.78:0/654995844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c1076e0 0x7fe75c107b20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:09.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.082+0000 7fe762717640 1 --2- 172.21.15.78:0/654995844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75c101cd0 0x7fe75c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:09.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.082+0000 7fe762717640 1 -- 172.21.15.78:0/654995844 >> 172.21.15.78:0/654995844 conn(0x7fe75c0fd700 msgr2=0x7fe75c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:52:09.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.082+0000 7fe762717640 1 -- 172.21.15.78:0/654995844 shutdown_connections 2024-09-11T00:52:09.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.082+0000 7fe762717640 1 -- 172.21.15.78:0/654995844 wait complete. 2024-09-11T00:52:09.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.082+0000 7fe762717640 1 Processor -- start 2024-09-11T00:52:09.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.082+0000 7fe762717640 1 -- start start 2024-09-11T00:52:09.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe762717640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c101cd0 0x7fe75c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:52:09.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe762717640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75c1076e0 0x7fe75c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:52:09.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe762717640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108060 0x7fe75c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:52:09.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe762717640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c111cb0 con 0x7fe75c101cd0 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe762717640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c111b30 con 0x7fe75c1076e0 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe762717640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c111e30 con 0x7fe75c108060 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe75bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c101cd0 0x7fe75c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe75bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c101cd0 0x7fe75c1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40080/0 (socket says 172.21.15.78:40080) 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe75bfff640 1 -- 172.21.15.78:0/2643159560 learned_addr learned my addr 172.21.15.78:0/2643159560 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe760cb4640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108060 0x7fe75c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe75b7fe640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75c1076e0 0x7fe75c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe75bfff640 1 -- 172.21.15.78:0/2643159560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108060 msgr2=0x7fe75c19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe75bfff640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108060 0x7fe75c19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe75bfff640 1 -- 172.21.15.78:0/2643159560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75c1076e0 msgr2=0x7fe75c198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe75bfff640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75c1076e0 0x7fe75c198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe75bfff640 1 -- 172.21.15.78:0/2643159560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c19e170 con 0x7fe75c101cd0 2024-09-11T00:52:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.083+0000 7fe75bfff640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c101cd0 0x7fe75c1980c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00c920 tx=0x7fe74c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:52:09.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.084+0000 7fe7597fa640 1 -- 172.21.15.78:0/2643159560 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c007c70 con 0x7fe75c101cd0 2024-09-11T00:52:09.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.084+0000 7fe7597fa640 1 -- 172.21.15.78:0/2643159560 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe74c007e10 con 0x7fe75c101cd0 2024-09-11T00:52:09.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.084+0000 7fe762717640 1 -- 172.21.15.78:0/2643159560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75c19e460 con 0x7fe75c101cd0 2024-09-11T00:52:09.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.084+0000 7fe7597fa640 1 -- 172.21.15.78:0/2643159560 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c018820 con 0x7fe75c101cd0 2024-09-11T00:52:09.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.084+0000 7fe762717640 1 -- 172.21.15.78:0/2643159560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75c1a4cf0 con 0x7fe75c101cd0 2024-09-11T00:52:09.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.085+0000 7fe762717640 1 -- 172.21.15.78:0/2643159560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75c1020b0 con 0x7fe75c101cd0 2024-09-11T00:52:09.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.086+0000 7fe7597fa640 1 -- 172.21.15.78:0/2643159560 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fe74c00f450 con 0x7fe75c101cd0 2024-09-11T00:52:09.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.087+0000 7fe7597fa640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe7480b5320 0x7fe7480b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:52:09.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.087+0000 7fe75b7fe640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe7480b5320 0x7fe7480b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:52:09.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.087+0000 7fe7597fa640 1 -- 172.21.15.78:0/2643159560 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2786+0+0 (secure 0 0 0) 0x7fe74c0e4800 con 0x7fe75c101cd0 2024-09-11T00:52:09.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.087+0000 7fe75b7fe640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe7480b5320 0x7fe7480b77e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe744031440 tx=0x7fe744005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:52:09.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.090+0000 7fe7597fa640 1 -- 172.21.15.78:0/2643159560 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe74c014030 con 0x7fe75c101cd0 2024-09-11T00:52:09.167 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T00:52:09.167 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: from='osd.2 [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:52:09.167 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T00:52:09.167 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:09.167 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:52:09.167 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:09.167 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:09.168 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:09.168 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:52:09.168 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:09.168 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:09.168 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:09.168 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:09.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T00:52:09.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: from='osd.2 [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:52:09.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T00:52:09.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:09.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:52:09.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:09.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:09.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:09.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:52:09.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:09.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:09.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:09.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:09.279+0000 7fe762717640 1 -- 172.21.15.78:0/2643159560 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe75c104cc0 con 0x7fe7480b5320 2024-09-11T00:52:09.474 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2[41413]: 2024-09-11T00:52:09.164+0000 7f1257f1c640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T00:52:09.474 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2[41413]: 2024-09-11T00:52:09.168+0000 7f124f527640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:52:09.952 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:09 smithi078 ceph-mon[31315]: Detected new or changed devices on smithi112 2024-09-11T00:52:09.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:09 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi112 to 17088M 2024-09-11T00:52:09.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:09 smithi078 ceph-mon[31315]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-11T00:52:09.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:09 smithi078 ceph-mon[31315]: osdmap e18: 3 total, 2 up, 3 in 2024-09-11T00:52:09.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:09.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:09.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:09 smithi078 ceph-mon[31315]: from='osd.2 ' entity='osd.2' 2024-09-11T00:52:09.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:52:09.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:52:09.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:09.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:10.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:09 smithi110 ceph-mon[34712]: Detected new or changed devices on smithi112 2024-09-11T00:52:10.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:09 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi112 to 17088M 2024-09-11T00:52:10.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:09 smithi110 ceph-mon[34712]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-11T00:52:10.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:09 smithi110 ceph-mon[34712]: osdmap e18: 3 total, 2 up, 3 in 2024-09-11T00:52:10.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:10.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:10.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:09 smithi110 ceph-mon[34712]: from='osd.2 ' entity='osd.2' 2024-09-11T00:52:10.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:52:10.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:52:10.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:10.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:10.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-mon[35384]: Detected new or changed devices on smithi112 2024-09-11T00:52:10.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi112 to 17088M 2024-09-11T00:52:10.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-mon[35384]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-11T00:52:10.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-mon[35384]: osdmap e18: 3 total, 2 up, 3 in 2024-09-11T00:52:10.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:10.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:10.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-mon[35384]: from='osd.2 ' entity='osd.2' 2024-09-11T00:52:10.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:52:10.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:52:10.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:10.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:10.932 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:10 smithi078 ceph-mon[31315]: purged_snaps scrub starts 2024-09-11T00:52:10.932 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:10 smithi078 ceph-mon[31315]: purged_snaps scrub ok 2024-09-11T00:52:10.932 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:10 smithi078 ceph-mon[31315]: from='client.14472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:52:10.932 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:10 smithi078 ceph-mon[31315]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:10.932 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:10 smithi078 ceph-mon[31315]: osd.2 [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763] boot 2024-09-11T00:52:10.932 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:10 smithi078 ceph-mon[31315]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T00:52:10.932 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:10.932 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:52:11.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:10 smithi110 ceph-mon[34712]: purged_snaps scrub starts 2024-09-11T00:52:11.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:10 smithi110 ceph-mon[34712]: purged_snaps scrub ok 2024-09-11T00:52:11.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:10 smithi110 ceph-mon[34712]: from='client.14472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:52:11.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:10 smithi110 ceph-mon[34712]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:11.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:10 smithi110 ceph-mon[34712]: osd.2 [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763] boot 2024-09-11T00:52:11.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:10 smithi110 ceph-mon[34712]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T00:52:11.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:11.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:52:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:10 smithi112 ceph-mon[35384]: purged_snaps scrub starts 2024-09-11T00:52:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:10 smithi112 ceph-mon[35384]: purged_snaps scrub ok 2024-09-11T00:52:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:10 smithi112 ceph-mon[35384]: from='client.14472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:52:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:10 smithi112 ceph-mon[35384]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:52:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:10 smithi112 ceph-mon[35384]: osd.2 [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763] boot 2024-09-11T00:52:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:10 smithi112 ceph-mon[35384]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T00:52:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:52:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:52:11.950 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51799]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:52:11.951 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51799]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:52:11.951 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51799]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:52:11.951 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51799]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:52:11.951 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51802]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:52:11.951 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51802]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:52:11.951 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51802]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:52:11.951 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51802]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:52:11.951 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51785]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:52:11.951 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51785]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:52:11.951 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51785]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:52:11.951 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51785]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:52:11.951 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51790]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:52:11.952 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:52:11.952 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51790]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:52:11.952 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 00:52:11 smithi078 sudo[51790]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:52:12.173 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 00:52:11 smithi110 sudo[44116]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:52:12.173 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 00:52:11 smithi110 sudo[44116]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:52:12.173 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 00:52:11 smithi110 sudo[44116]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:52:12.173 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 00:52:11 smithi110 sudo[44116]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:52:12.173 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 00:52:11 smithi110 sudo[44119]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:52:12.173 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 00:52:11 smithi110 sudo[44119]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:52:12.173 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 00:52:11 smithi110 sudo[44119]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:52:12.173 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 00:52:11 smithi110 sudo[44119]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:52:12.174 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45340]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:52:12.174 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45340]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:52:12.174 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45340]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:52:12.174 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45340]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:52:12.174 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45343]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:52:12.174 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45343]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:52:12.174 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45343]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:52:12.175 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45343]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:52:12.175 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:11 smithi110 sudo[44122]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:52:12.175 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:11 smithi110 sudo[44122]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:52:12.175 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:11 smithi110 sudo[44122]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:52:12.175 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:11 smithi110 sudo[44122]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:52:12.175 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 sudo[44125]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:52:12.175 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 sudo[44125]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:52:12.175 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 sudo[44125]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:52:12.175 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 sudo[44125]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:52:12.175 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T00:52:12.175 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/20585531' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3270362b-3d61-414a-a3d9-f2416ff8b870"} : dispatch 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/20585531' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3270362b-3d61-414a-a3d9-f2416ff8b870"}]': finished 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:52:12.176 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:52:12.177 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:52:12.177 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45346]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:52:12.177 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45346]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:52:12.177 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45346]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:52:12.177 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45346]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:52:12.177 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45349]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:52:12.177 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45349]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:52:12.177 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45349]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:52:12.177 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:11 smithi112 sudo[45349]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:52:12.177 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T00:52:12.177 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T00:52:12.177 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:52:12.178 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/20585531' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3270362b-3d61-414a-a3d9-f2416ff8b870"} : dispatch 2024-09-11T00:52:12.178 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T00:52:12.178 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/20585531' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3270362b-3d61-414a-a3d9-f2416ff8b870"}]': finished 2024-09-11T00:52:12.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T00:52:12.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/20585531' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3270362b-3d61-414a-a3d9-f2416ff8b870"} : dispatch 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/20585531' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3270362b-3d61-414a-a3d9-f2416ff8b870"}]': finished 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:52:12.209 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:52:12.210 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:52:12.210 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:52:12.210 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:52:12.210 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:52:12.210 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:52:12.210 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: pgmap v67: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:12.210 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:52:12.211 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:52:12.211 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:52:12.211 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:52:12.211 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:52:12.211 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:12 smithi078 ceph-mon[31315]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:52:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:52:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:52:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: pgmap v67: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:52:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:52:12.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:52:12.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:52:12.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:52:12.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:12 smithi110 ceph-mon[34712]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:52:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T00:52:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:52:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:52:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:52:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:52:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:52:12.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:52:12.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:52:12.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:52:12.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:52:12.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: pgmap v67: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:12.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:52:12.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:52:12.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:52:12.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:52:12.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:52:12.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:12 smithi112 ceph-mon[35384]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:52:12.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:12.894+0000 7fe7597fa640 1 -- 172.21.15.78:0/2643159560 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fe74c0a89b0 con 0x7fe75c101cd0 2024-09-11T00:52:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:13 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1888925585' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:52:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:13 smithi110 ceph-mon[34712]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T00:52:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:13 smithi110 ceph-mon[34712]: mgrmap e20: x(active, since 116s), standbys: z, y 2024-09-11T00:52:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:13 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1888925585' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:52:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:13 smithi112 ceph-mon[35384]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T00:52:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:13 smithi112 ceph-mon[35384]: mgrmap e20: x(active, since 116s), standbys: z, y 2024-09-11T00:52:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:13 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1888925585' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:52:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:13 smithi078 ceph-mon[31315]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T00:52:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:13 smithi078 ceph-mon[31315]: mgrmap e20: x(active, since 116s), standbys: z, y 2024-09-11T00:52:14.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:14 smithi110 ceph-mon[34712]: pgmap v69: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:14.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:14 smithi112 ceph-mon[35384]: pgmap v69: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:14 smithi078 ceph-mon[31315]: pgmap v69: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:17.164 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:17 smithi078 ceph-mon[31315]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:17.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:17 smithi110 ceph-mon[34712]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:17.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:17 smithi112 ceph-mon[35384]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:19 smithi078 ceph-mon[31315]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:19 smithi110 ceph-mon[34712]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:19 smithi112 ceph-mon[35384]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:21.325 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:21 smithi078 ceph-mon[31315]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:21.326 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-11T00:52:21.326 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:21 smithi110 ceph-mon[34712]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-11T00:52:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:21.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:21 smithi112 ceph-mon[35384]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:21.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-11T00:52:21.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:22.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:22 smithi078 ceph-mon[31315]: Deploying daemon osd.3 on smithi078 2024-09-11T00:52:22.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:22 smithi110 ceph-mon[34712]: Deploying daemon osd.3 on smithi078 2024-09-11T00:52:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:22 smithi112 ceph-mon[35384]: Deploying daemon osd.3 on smithi078 2024-09-11T00:52:23.279 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:23 smithi078 ceph-mon[31315]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:23.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:23 smithi110 ceph-mon[34712]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:23 smithi112 ceph-mon[35384]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:25.214 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:25 smithi078 ceph-mon[31315]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:25 smithi110 ceph-mon[34712]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:25.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:25 smithi112 ceph-mon[35384]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:27.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:27 smithi078 ceph-mon[31315]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:27.230 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:27.230 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:27.230 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:27 smithi110 ceph-mon[34712]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:27 smithi112 ceph-mon[35384]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:29.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:29 smithi078 ceph-mon[31315]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:29.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:29 smithi110 ceph-mon[34712]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:29.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:29 smithi112 ceph-mon[35384]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:30.444 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:30.445 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:30.445 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:30.445 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:30.445 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:30.445 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:30 smithi078 ceph-mon[31315]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:30.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.482+0000 7fe7597fa640 1 -- 172.21.15.78:0/2643159560 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe75c104cc0 con 0x7fe7480b5320 2024-09-11T00:52:30.484 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 3 on host 'smithi078' 2024-09-11T00:52:30.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.484+0000 7fe762717640 1 -- 172.21.15.78:0/2643159560 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe7480b5320 msgr2=0x7fe7480b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:52:30.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.484+0000 7fe762717640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe7480b5320 0x7fe7480b77e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe744031440 tx=0x7fe744005b50 comp rx=0 tx=0).stop 2024-09-11T00:52:30.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.484+0000 7fe762717640 1 -- 172.21.15.78:0/2643159560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c101cd0 msgr2=0x7fe75c1980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:52:30.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.484+0000 7fe762717640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c101cd0 0x7fe75c1980c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00c920 tx=0x7fe74c00cde0 comp rx=0 tx=0).stop 2024-09-11T00:52:30.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.484+0000 7fe762717640 1 -- 172.21.15.78:0/2643159560 shutdown_connections 2024-09-11T00:52:30.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.484+0000 7fe762717640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe7480b5320 0x7fe7480b77e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:30.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.484+0000 7fe762717640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108060 0x7fe75c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:30.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.484+0000 7fe762717640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75c1076e0 0x7fe75c198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:30.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.484+0000 7fe762717640 1 --2- 172.21.15.78:0/2643159560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c101cd0 0x7fe75c1980c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:30.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.484+0000 7fe762717640 1 -- 172.21.15.78:0/2643159560 >> 172.21.15.78:0/2643159560 conn(0x7fe75c0fd700 msgr2=0x7fe75c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:52:30.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.484+0000 7fe762717640 1 -- 172.21.15.78:0/2643159560 shutdown_connections 2024-09-11T00:52:30.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:52:30.485+0000 7fe762717640 1 -- 172.21.15.78:0/2643159560 wait complete. 2024-09-11T00:52:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:30.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:30.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:30 smithi110 ceph-mon[34712]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:30 smithi112 ceph-mon[35384]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:31.183 DEBUG:teuthology.orchestra.run.smithi078:osd.3> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T00:52:31.185 INFO:tasks.cephadm:Deploying osd.4 on smithi110 with /dev/vg_nvme/lv_3... 2024-09-11T00:52:31.186 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T00:52:31.443 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:52:31.458 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:31 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:31.458 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:31 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:31.458 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:31 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:31.563 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:31 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:31.563 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:31 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:31.563 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:31 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:31 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:31 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:31 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:32.481 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:32 smithi078 ceph-mon[31315]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:32.591 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:32 smithi110 ceph-mon[34712]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:32.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:32 smithi112 ceph-mon[35384]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:32.747 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 00:52:32 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3[53815]: 2024-09-11T00:52:32.568+0000 7f51134da640 -1 osd.3 0 log_to_monitors true 2024-09-11T00:52:33.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:33 smithi078 ceph-mon[31315]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-11T00:52:33.660 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:33 smithi110 ceph-mon[34712]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-11T00:52:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:33 smithi112 ceph-mon[35384]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-11T00:52:33.952 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:52:33.978 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_3 2024-09-11T00:52:34.218 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:52:34.511 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T00:52:34.511 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T00:52:34.511 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:52:34.512 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:34.512 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: Detected new or changed devices on smithi078 2024-09-11T00:52:34.512 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:34.512 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:34.512 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:52:34.512 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:52:34.512 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi078 to 8544M 2024-09-11T00:52:34.512 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:34.512 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:34.512 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:34.512 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:34.512 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:34 smithi110 ceph-mon[34712]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T00:52:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T00:52:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:52:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: Detected new or changed devices on smithi078 2024-09-11T00:52:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:34.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:52:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:52:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi078 to 8544M 2024-09-11T00:52:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:34.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-mon[31315]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:34.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.682+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/4009907902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab0101cd0 msgr2=0x7f5ab01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:52:34.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.682+0000 7f5ab4eb3640 1 --2- 172.21.15.110:0/4009907902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab0101cd0 0x7f5ab01020b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009900 tx=0x7f5aa002a410 comp rx=0 tx=0).stop 2024-09-11T00:52:34.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.682+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/4009907902 shutdown_connections 2024-09-11T00:52:34.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.682+0000 7f5ab4eb3640 1 --2- 172.21.15.110:0/4009907902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ab0108060 0x7f5ab010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:34.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.682+0000 7f5ab4eb3640 1 --2- 172.21.15.110:0/4009907902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ab01076e0 0x7f5ab0107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:34.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.682+0000 7f5ab4eb3640 1 --2- 172.21.15.110:0/4009907902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab0101cd0 0x7f5ab01020b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:34.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.682+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/4009907902 >> 172.21.15.110:0/4009907902 conn(0x7f5ab00fd700 msgr2=0x7f5ab00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:52:34.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.682+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/4009907902 shutdown_connections 2024-09-11T00:52:34.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.682+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/4009907902 wait complete. 2024-09-11T00:52:34.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.683+0000 7f5ab4eb3640 1 Processor -- start 2024-09-11T00:52:34.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.683+0000 7f5ab4eb3640 1 -- start start 2024-09-11T00:52:34.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.683+0000 7f5ab4eb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ab0101cd0 0x7f5ab0199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:52:34.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.683+0000 7f5ab4eb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ab01076e0 0x7f5ab0199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:52:34.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.683+0000 7f5ab4eb3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab0108060 0x7f5ab019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:52:34.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.683+0000 7f5ab4eb3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0111d60 con 0x7f5ab01076e0 2024-09-11T00:52:34.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.683+0000 7f5ab4eb3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0111be0 con 0x7f5ab0108060 2024-09-11T00:52:34.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.683+0000 7f5ab4eb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0111ee0 con 0x7f5ab0101cd0 2024-09-11T00:52:34.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.683+0000 7f5aaed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab0108060 0x7f5ab019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:52:34.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.683+0000 7f5aaed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab0108060 0x7f5ab019ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49438/0 (socket says 172.21.15.110:49438) 2024-09-11T00:52:34.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.683+0000 7f5aaed9d640 1 -- 172.21.15.110:0/2519931547 learned_addr learned my addr 172.21.15.110:0/2519931547 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-11T00:52:34.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.683+0000 7f5aadd9b640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ab01076e0 0x7f5ab0199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:52:34.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5aae59c640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ab0101cd0 0x7f5ab0199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:52:34.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5aaed9d640 1 -- 172.21.15.110:0/2519931547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ab0101cd0 msgr2=0x7f5ab0199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:52:34.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5aaed9d640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ab0101cd0 0x7f5ab0199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:34.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5aaed9d640 1 -- 172.21.15.110:0/2519931547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ab01076e0 msgr2=0x7f5ab0199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:52:34.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5aaed9d640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ab01076e0 0x7f5ab0199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:34.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5aaed9d640 1 -- 172.21.15.110:0/2519931547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab019f4e0 con 0x7f5ab0108060 2024-09-11T00:52:34.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5aadd9b640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ab01076e0 0x7f5ab0199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:52:34.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5aaed9d640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab0108060 0x7f5ab019ed10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800b970 tx=0x7f5aa800be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:52:34.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5a9f7fe640 1 -- 172.21.15.110:0/2519931547 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8019070 con 0x7f5ab0108060 2024-09-11T00:52:34.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/2519931547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab019f7d0 con 0x7f5ab0108060 2024-09-11T00:52:34.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5a9f7fe640 1 -- 172.21.15.110:0/2519931547 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa8014440 con 0x7f5ab0108060 2024-09-11T00:52:34.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5a9f7fe640 1 -- 172.21.15.110:0/2519931547 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80075d0 con 0x7f5ab0108060 2024-09-11T00:52:34.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.684+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/2519931547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab01a5f10 con 0x7f5ab0108060 2024-09-11T00:52:34.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.685+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/2519931547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab0102130 con 0x7f5ab0108060 2024-09-11T00:52:34.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.686+0000 7f5a9f7fe640 1 -- 172.21.15.110:0/2519931547 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f5aa800cb40 con 0x7f5ab0108060 2024-09-11T00:52:34.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.687+0000 7f5a9f7fe640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a980b5330 0x7f5a980b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:52:34.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.687+0000 7f5a9f7fe640 1 -- 172.21.15.110:0/2519931547 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3604+0+0 (secure 0 0 0) 0x7f5aa80e4200 con 0x7f5ab0108060 2024-09-11T00:52:34.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.687+0000 7f5aae59c640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a980b5330 0x7f5a980b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:52:34.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.688+0000 7f5aae59c640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a980b5330 0x7f5a980b77f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0031440 tx=0x7f5aa00095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:52:34.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.691+0000 7f5a9f7fe640 1 -- 172.21.15.110:0/2519931547 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5aa80ac320 con 0x7f5ab0108060 2024-09-11T00:52:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T00:52:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T00:52:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:52:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: Detected new or changed devices on smithi078 2024-09-11T00:52:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:34.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:52:34.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:52:34.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi078 to 8544M 2024-09-11T00:52:34.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:34.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:34.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:34.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:34.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:34 smithi112 ceph-mon[35384]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:52:34.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:34.878+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/2519931547 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5ab0104ca0 con 0x7f5a980b5330 2024-09-11T00:52:34.997 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3[53815]: 2024-09-11T00:52:34.634+0000 7f510f45d640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T00:52:34.997 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 00:52:34 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3[53815]: 2024-09-11T00:52:34.638+0000 7f510727c640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:52:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:35 smithi110 ceph-mon[34712]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-11T00:52:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:35 smithi110 ceph-mon[34712]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T00:52:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:35 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:35 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:35 smithi110 ceph-mon[34712]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' 2024-09-11T00:52:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:35 smithi110 ceph-mon[34712]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:52:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:35 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:52:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:35 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:52:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:35 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:35 smithi110 ceph-mon[34712]: osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464] boot 2024-09-11T00:52:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:35 smithi110 ceph-mon[34712]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T00:52:35.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:35 smithi112 ceph-mon[35384]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-11T00:52:35.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:35 smithi112 ceph-mon[35384]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T00:52:35.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:35 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:35.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:35 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:35.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:35 smithi112 ceph-mon[35384]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' 2024-09-11T00:52:35.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:35 smithi112 ceph-mon[35384]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:52:35.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:35 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:52:35.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:35 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:52:35.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:35 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:35.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:35 smithi112 ceph-mon[35384]: osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464] boot 2024-09-11T00:52:35.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:35 smithi112 ceph-mon[35384]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T00:52:35.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:35 smithi078 ceph-mon[31315]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-11T00:52:35.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:35 smithi078 ceph-mon[31315]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T00:52:35.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:35 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:35.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:35 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:35.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:35 smithi078 ceph-mon[31315]: from='osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464]' entity='osd.3' 2024-09-11T00:52:35.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:35 smithi078 ceph-mon[31315]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:52:35.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:35 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:52:35.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:35 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:52:35.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:35 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:35.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:35 smithi078 ceph-mon[31315]: osd.3 [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464] boot 2024-09-11T00:52:35.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:35 smithi078 ceph-mon[31315]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T00:52:36.586 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:36 smithi110 ceph-mon[34712]: purged_snaps scrub starts 2024-09-11T00:52:36.586 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:36 smithi110 ceph-mon[34712]: purged_snaps scrub ok 2024-09-11T00:52:36.586 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:36 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:36.586 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:36 smithi110 ceph-mon[34712]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:36 smithi112 ceph-mon[35384]: purged_snaps scrub starts 2024-09-11T00:52:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:36 smithi112 ceph-mon[35384]: purged_snaps scrub ok 2024-09-11T00:52:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:36 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:36 smithi112 ceph-mon[35384]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:36 smithi078 ceph-mon[31315]: purged_snaps scrub starts 2024-09-11T00:52:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:36 smithi078 ceph-mon[31315]: purged_snaps scrub ok 2024-09-11T00:52:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:36 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:52:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:36 smithi078 ceph-mon[31315]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:37.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:37 smithi078 ceph-mon[31315]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T00:52:37.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:37 smithi078 ceph-mon[31315]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T00:52:37.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:37 smithi110 ceph-mon[34712]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T00:52:37.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:37 smithi110 ceph-mon[34712]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T00:52:37.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:37 smithi112 ceph-mon[35384]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T00:52:37.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:37 smithi112 ceph-mon[35384]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T00:52:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:38 smithi110 ceph-mon[34712]: from='client.? 172.21.15.110:0/3231070845' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e47746ec-9443-4004-abfa-17c1ed73b982"} : dispatch 2024-09-11T00:52:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:38 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e47746ec-9443-4004-abfa-17c1ed73b982"} : dispatch 2024-09-11T00:52:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:38 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e47746ec-9443-4004-abfa-17c1ed73b982"}]': finished 2024-09-11T00:52:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:38 smithi110 ceph-mon[34712]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T00:52:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:38 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:52:38.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:38 smithi110 ceph-mon[34712]: from='client.? 172.21.15.110:0/2457002176' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:52:38.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:38 smithi110 ceph-mon[34712]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:38 smithi112 ceph-mon[35384]: from='client.? 172.21.15.110:0/3231070845' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e47746ec-9443-4004-abfa-17c1ed73b982"} : dispatch 2024-09-11T00:52:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:38 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e47746ec-9443-4004-abfa-17c1ed73b982"} : dispatch 2024-09-11T00:52:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:38 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e47746ec-9443-4004-abfa-17c1ed73b982"}]': finished 2024-09-11T00:52:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:38 smithi112 ceph-mon[35384]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T00:52:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:38 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:52:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:38 smithi112 ceph-mon[35384]: from='client.? 172.21.15.110:0/2457002176' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:52:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:38 smithi112 ceph-mon[35384]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:38 smithi078 ceph-mon[31315]: from='client.? 172.21.15.110:0/3231070845' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e47746ec-9443-4004-abfa-17c1ed73b982"} : dispatch 2024-09-11T00:52:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:38 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e47746ec-9443-4004-abfa-17c1ed73b982"} : dispatch 2024-09-11T00:52:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:38 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e47746ec-9443-4004-abfa-17c1ed73b982"}]': finished 2024-09-11T00:52:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:38 smithi078 ceph-mon[31315]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T00:52:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:38 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:52:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:38 smithi078 ceph-mon[31315]: from='client.? 172.21.15.110:0/2457002176' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:52:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:38 smithi078 ceph-mon[31315]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:41 smithi110 ceph-mon[34712]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-09-11T00:52:41.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:41 smithi112 ceph-mon[35384]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-09-11T00:52:41.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:41 smithi078 ceph-mon[31315]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-09-11T00:52:43.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:43 smithi110 ceph-mon[34712]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:52:43.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:43 smithi112 ceph-mon[35384]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:52:43.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:43 smithi078 ceph-mon[31315]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:52:45.373 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:45 smithi110 ceph-mon[34712]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-11T00:52:45.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:45 smithi112 ceph-mon[35384]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-11T00:52:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:45 smithi078 ceph-mon[31315]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-11T00:52:46.376 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:46 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-11T00:52:46.376 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:46 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:46 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-11T00:52:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:46 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:46.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:46 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-11T00:52:46.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:46 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:47.326 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:47 smithi110 ceph-mon[34712]: Deploying daemon osd.4 on smithi110 2024-09-11T00:52:47.327 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:47 smithi110 ceph-mon[34712]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-11T00:52:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:47 smithi112 ceph-mon[35384]: Deploying daemon osd.4 on smithi110 2024-09-11T00:52:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:47 smithi112 ceph-mon[35384]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-11T00:52:47.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:47 smithi078 ceph-mon[31315]: Deploying daemon osd.4 on smithi110 2024-09-11T00:52:47.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:47 smithi078 ceph-mon[31315]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-11T00:52:49.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:49 smithi110 ceph-mon[34712]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-11T00:52:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:49 smithi112 ceph-mon[35384]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-11T00:52:49.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:49 smithi078 ceph-mon[31315]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-11T00:52:51.412 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:51 smithi110 ceph-mon[34712]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:52:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:51 smithi112 ceph-mon[35384]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:52:51.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:51 smithi078 ceph-mon[31315]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:52:53.319 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:53 smithi110 ceph-mon[34712]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:53.319 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:53 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:53.319 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:53 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:53.319 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:53 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:53 smithi112 ceph-mon[35384]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:53 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:53 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:53 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:53 smithi078 ceph-mon[31315]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:53 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:53 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:53 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:55.424 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:55 smithi110 ceph-mon[34712]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:55.425 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:55 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:55.425 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:55 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:55 smithi112 ceph-mon[35384]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:55 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:55 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:55.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:55 smithi078 ceph-mon[31315]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:55.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:55 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:55.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:55 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:56.117 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:56 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:56.118 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:56 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:56.118 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:56 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:56.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.155+0000 7f5a9f7fe640 1 -- 172.21.15.110:0/2519931547 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5ab0104ca0 con 0x7f5a980b5330 2024-09-11T00:52:56.157 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 4 on host 'smithi110' 2024-09-11T00:52:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.157+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/2519931547 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a980b5330 msgr2=0x7f5a980b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:52:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.157+0000 7f5ab4eb3640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a980b5330 0x7f5a980b77f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0031440 tx=0x7f5aa00095f0 comp rx=0 tx=0).stop 2024-09-11T00:52:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.157+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/2519931547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab0108060 msgr2=0x7f5ab019ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:52:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.157+0000 7f5ab4eb3640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab0108060 0x7f5ab019ed10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800b970 tx=0x7f5aa800be30 comp rx=0 tx=0).stop 2024-09-11T00:52:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.157+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/2519931547 shutdown_connections 2024-09-11T00:52:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.157+0000 7f5ab4eb3640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a980b5330 0x7f5a980b77f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.157+0000 7f5ab4eb3640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab0108060 0x7f5ab019ed10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.157+0000 7f5ab4eb3640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ab01076e0 0x7f5ab0199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.157+0000 7f5ab4eb3640 1 --2- 172.21.15.110:0/2519931547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ab0101cd0 0x7f5ab0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:52:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.157+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/2519931547 >> 172.21.15.110:0/2519931547 conn(0x7f5ab00fd700 msgr2=0x7f5ab006b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:52:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.157+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/2519931547 shutdown_connections 2024-09-11T00:52:56.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:52:56.157+0000 7f5ab4eb3640 1 -- 172.21.15.110:0/2519931547 wait complete. 2024-09-11T00:52:56.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:56 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:56 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:56 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:56.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:56 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:52:56.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:56 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:52:56.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:56 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:56.765 DEBUG:teuthology.orchestra.run.smithi110:osd.4> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T00:52:56.767 INFO:tasks.cephadm:Deploying osd.5 on smithi112 with /dev/vg_nvme/lv_3... 2024-09-11T00:52:56.767 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T00:52:57.020 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:52:57.308 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:57 smithi112 ceph-mon[35384]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:57.308 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:57 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:57.308 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:57 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:57.308 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:57 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:57.308 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:57 smithi110 ceph-mon[34712]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:57.309 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:57 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:57.309 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:57 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:57.309 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:57 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:57 smithi078 ceph-mon[31315]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:57 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:52:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:57 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:57 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:52:57.690 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 00:52:57 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4[46754]: 2024-09-11T00:52:57.402+0000 7f43b4326640 -1 osd.4 0 log_to_monitors true 2024-09-11T00:52:58.282 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:58 smithi110 ceph-mon[34712]: from='osd.4 [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:52:58.282 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:58 smithi110 ceph-mon[34712]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:52:58.412 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:58 smithi112 ceph-mon[35384]: from='osd.4 [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:52:58.412 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:58 smithi112 ceph-mon[35384]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:52:58.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:58 smithi078 ceph-mon[31315]: from='osd.4 [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:52:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:58 smithi078 ceph-mon[31315]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:52:59.267 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 00:52:59 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4[46754]: 2024-09-11T00:52:59.069+0000 7f43b02a9640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T00:52:59.267 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 00:52:59 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4[46754]: 2024-09-11T00:52:59.076+0000 7f43ac0d0640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:52:59.267 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:59 smithi110 ceph-mon[34712]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:59.267 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:59 smithi110 ceph-mon[34712]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T00:52:59.268 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:59 smithi110 ceph-mon[34712]: from='osd.4 [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:52:59.268 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:59 smithi110 ceph-mon[34712]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T00:52:59.268 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:52:59.268 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:52:59 smithi110 ceph-mon[34712]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:52:59.431 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:59 smithi112 ceph-mon[35384]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:59.432 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:59 smithi112 ceph-mon[35384]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T00:52:59.432 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:59 smithi112 ceph-mon[35384]: from='osd.4 [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:52:59.432 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:59 smithi112 ceph-mon[35384]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T00:52:59.432 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:52:59.432 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:52:59 smithi112 ceph-mon[35384]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:52:59.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:59 smithi078 ceph-mon[31315]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:52:59.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:59 smithi078 ceph-mon[31315]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T00:52:59.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:59 smithi078 ceph-mon[31315]: from='osd.4 [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:52:59.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:59 smithi078 ceph-mon[31315]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T00:52:59.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:52:59.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:52:59 smithi078 ceph-mon[31315]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:52:59.702 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:52:59.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-09-11T00:52:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:53:00.361 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:00 smithi112 ceph-mon[35384]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T00:53:00.362 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:00 smithi112 ceph-mon[35384]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T00:53:00.362 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:53:00.362 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:53:00.362 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:00.362 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:00.362 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:00.362 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:00.362 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:00.362 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:00.362 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:00.363 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:00 smithi110 ceph-mon[34712]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T00:53:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:00 smithi110 ceph-mon[34712]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T00:53:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:53:00.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:53:00.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:00.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:00.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:00.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:00.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:00.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:00.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:00.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.441+0000 7f685b269640 1 -- 172.21.15.112:0/3658353270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6854103b70 msgr2=0x7f6854103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.441+0000 7f685b269640 1 --2- 172.21.15.112:0/3658353270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6854103b70 0x7f6854103f50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6848009900 tx=0x7f684802a410 comp rx=0 tx=0).stop 2024-09-11T00:53:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.441+0000 7f685b269640 1 -- 172.21.15.112:0/3658353270 shutdown_connections 2024-09-11T00:53:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.441+0000 7f685b269640 1 --2- 172.21.15.112:0/3658353270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f685410d120 0x7f685410f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.441+0000 7f685b269640 1 --2- 172.21.15.112:0/3658353270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6854104570 0x7f685410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.441+0000 7f685b269640 1 --2- 172.21.15.112:0/3658353270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6854103b70 0x7f6854103f50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.441+0000 7f685b269640 1 -- 172.21.15.112:0/3658353270 >> 172.21.15.112:0/3658353270 conn(0x7f68540fd700 msgr2=0x7f68540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.441+0000 7f685b269640 1 -- 172.21.15.112:0/3658353270 shutdown_connections 2024-09-11T00:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.441+0000 7f685b269640 1 -- 172.21.15.112:0/3658353270 wait complete. 2024-09-11T00:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f685b269640 1 Processor -- start 2024-09-11T00:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f685b269640 1 -- start start 2024-09-11T00:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f685b269640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6854103b70 0x7f6854199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f685b269640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6854104570 0x7f68541997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f685b269640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685410d120 0x7f685419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f685b269640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6854112da0 con 0x7f6854104570 2024-09-11T00:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f685b269640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6854112c20 con 0x7f6854103b70 2024-09-11T00:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f685b269640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6854112f20 con 0x7f685410d120 2024-09-11T00:53:00.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f6859806640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685410d120 0x7f685419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:53:00.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f6859806640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685410d120 0x7f685419eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42850/0 (socket says 172.21.15.112:42850) 2024-09-11T00:53:00.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f6859806640 1 -- 172.21.15.112:0/2655923623 learned_addr learned my addr 172.21.15.112:0/2655923623 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-11T00:53:00.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f6858804640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6854104570 0x7f68541997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:53:00.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f6859005640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6854103b70 0x7f6854199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:53:00.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f6859806640 1 -- 172.21.15.112:0/2655923623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6854103b70 msgr2=0x7f6854199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:00.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f6859806640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6854103b70 0x7f6854199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:00.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f6859806640 1 -- 172.21.15.112:0/2655923623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6854104570 msgr2=0x7f68541997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:00.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f6859806640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6854104570 0x7f68541997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:00.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.442+0000 7f6859806640 1 -- 172.21.15.112:0/2655923623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685419f320 con 0x7f685410d120 2024-09-11T00:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.443+0000 7f6859806640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685410d120 0x7f685419eb50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f684c00eb10 tx=0x7f684c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.443+0000 7f68467fc640 1 -- 172.21.15.112:0/2655923623 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c004b20 con 0x7f685410d120 2024-09-11T00:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.443+0000 7f68467fc640 1 -- 172.21.15.112:0/2655923623 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f684c004cc0 con 0x7f685410d120 2024-09-11T00:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.443+0000 7f685b269640 1 -- 172.21.15.112:0/2655923623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685419f610 con 0x7f685410d120 2024-09-11T00:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.443+0000 7f68467fc640 1 -- 172.21.15.112:0/2655923623 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c010670 con 0x7f685410d120 2024-09-11T00:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.443+0000 7f685b269640 1 -- 172.21.15.112:0/2655923623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68541a5cd0 con 0x7f685410d120 2024-09-11T00:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.444+0000 7f685b269640 1 -- 172.21.15.112:0/2655923623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6854103f50 con 0x7f685410d120 2024-09-11T00:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.445+0000 7f68467fc640 1 -- 172.21.15.112:0/2655923623 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f684c005020 con 0x7f685410d120 2024-09-11T00:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.446+0000 7f68467fc640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68400b5330 0x7f68400b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.446+0000 7f68467fc640 1 -- 172.21.15.112:0/2655923623 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4316+0+0 (secure 0 0 0) 0x7f684c0e5280 con 0x7f685410d120 2024-09-11T00:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.446+0000 7f6859005640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68400b5330 0x7f68400b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:53:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.446+0000 7f6859005640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68400b5330 0x7f68400b77f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6848031440 tx=0x7f6848005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:53:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.450+0000 7f68467fc640 1 -- 172.21.15.112:0/2655923623 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f684c0ad0e0 con 0x7f685410d120 2024-09-11T00:53:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:00 smithi078 ceph-mon[31315]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T00:53:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:00 smithi078 ceph-mon[31315]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T00:53:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:53:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:53:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:00.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:00.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:00.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:00.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:00.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:00.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:00.634+0000 7f685b269640 1 -- 172.21.15.112:0/2655923623 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f68540ffa20 con 0x7f68400b5330 2024-09-11T00:53:01.381 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:01 smithi112 ceph-mon[35384]: purged_snaps scrub starts 2024-09-11T00:53:01.381 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:01 smithi112 ceph-mon[35384]: purged_snaps scrub ok 2024-09-11T00:53:01.381 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:01 smithi112 ceph-mon[35384]: Detected new or changed devices on smithi110 2024-09-11T00:53:01.381 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:01 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi110 to 8544M 2024-09-11T00:53:01.381 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:01 smithi112 ceph-mon[35384]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:53:01.381 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:01 smithi112 ceph-mon[35384]: osd.4 [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435] boot 2024-09-11T00:53:01.381 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:01 smithi112 ceph-mon[35384]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T00:53:01.381 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:01 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:53:01.381 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:01 smithi112 ceph-mon[35384]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:53:01.381 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:01 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:53:01.382 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:01 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:53:01.382 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:01 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:01 smithi110 ceph-mon[34712]: purged_snaps scrub starts 2024-09-11T00:53:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:01 smithi110 ceph-mon[34712]: purged_snaps scrub ok 2024-09-11T00:53:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:01 smithi110 ceph-mon[34712]: Detected new or changed devices on smithi110 2024-09-11T00:53:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:01 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi110 to 8544M 2024-09-11T00:53:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:01 smithi110 ceph-mon[34712]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:53:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:01 smithi110 ceph-mon[34712]: osd.4 [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435] boot 2024-09-11T00:53:01.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:01 smithi110 ceph-mon[34712]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T00:53:01.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:01 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:53:01.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:01 smithi110 ceph-mon[34712]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:53:01.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:01 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:53:01.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:01 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:53:01.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:01 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:01 smithi078 ceph-mon[31315]: purged_snaps scrub starts 2024-09-11T00:53:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:01 smithi078 ceph-mon[31315]: purged_snaps scrub ok 2024-09-11T00:53:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:01 smithi078 ceph-mon[31315]: Detected new or changed devices on smithi110 2024-09-11T00:53:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:01 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi110 to 8544M 2024-09-11T00:53:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:01 smithi078 ceph-mon[31315]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:53:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:01 smithi078 ceph-mon[31315]: osd.4 [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435] boot 2024-09-11T00:53:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:01 smithi078 ceph-mon[31315]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T00:53:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:01 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:53:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:01 smithi078 ceph-mon[31315]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:53:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:01 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:53:01.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:01 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:53:01.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:01 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:02.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:02 smithi112 ceph-mon[35384]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T00:53:02.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:02 smithi112 ceph-mon[35384]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:02 smithi110 ceph-mon[34712]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T00:53:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:02 smithi110 ceph-mon[34712]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:02 smithi078 ceph-mon[31315]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T00:53:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:02 smithi078 ceph-mon[31315]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:03 smithi110 ceph-mon[34712]: from='client.? 172.21.15.112:0/2028751222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f8cedb01-310d-442d-a588-33197d32e208"} : dispatch 2024-09-11T00:53:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:03 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f8cedb01-310d-442d-a588-33197d32e208"} : dispatch 2024-09-11T00:53:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:03 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8cedb01-310d-442d-a588-33197d32e208"}]': finished 2024-09-11T00:53:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:03 smithi110 ceph-mon[34712]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T00:53:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:03 smithi112 ceph-mon[35384]: from='client.? 172.21.15.112:0/2028751222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f8cedb01-310d-442d-a588-33197d32e208"} : dispatch 2024-09-11T00:53:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:03 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f8cedb01-310d-442d-a588-33197d32e208"} : dispatch 2024-09-11T00:53:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:03 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8cedb01-310d-442d-a588-33197d32e208"}]': finished 2024-09-11T00:53:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:03 smithi112 ceph-mon[35384]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T00:53:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:03.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:03 smithi078 ceph-mon[31315]: from='client.? 172.21.15.112:0/2028751222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f8cedb01-310d-442d-a588-33197d32e208"} : dispatch 2024-09-11T00:53:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:03 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f8cedb01-310d-442d-a588-33197d32e208"} : dispatch 2024-09-11T00:53:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:03 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8cedb01-310d-442d-a588-33197d32e208"}]': finished 2024-09-11T00:53:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:03 smithi078 ceph-mon[31315]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T00:53:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:04 smithi110 ceph-mon[34712]: from='client.? 172.21.15.112:0/286225794' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:53:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:04 smithi110 ceph-mon[34712]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:04 smithi112 ceph-mon[35384]: from='client.? 172.21.15.112:0/286225794' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:53:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:04 smithi112 ceph-mon[35384]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:04 smithi078 ceph-mon[31315]: from='client.? 172.21.15.112:0/286225794' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:53:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:04 smithi078 ceph-mon[31315]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:07 smithi110 ceph-mon[34712]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:07.467 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:07 smithi078 ceph-mon[31315]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:07.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:07 smithi112 ceph-mon[35384]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:09.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:09 smithi112 ceph-mon[35384]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:09 smithi110 ceph-mon[34712]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:09.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:09 smithi078 ceph-mon[31315]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:11.284 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:11 smithi112 ceph-mon[35384]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:11 smithi110 ceph-mon[34712]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:11 smithi078 ceph-mon[31315]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-11T00:53:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:12.446 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-11T00:53:12.446 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:12.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-11T00:53:12.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:13.309 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:13 smithi112 ceph-mon[35384]: Deploying daemon osd.5 on smithi112 2024-09-11T00:53:13.309 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:13 smithi112 ceph-mon[35384]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:13 smithi110 ceph-mon[34712]: Deploying daemon osd.5 on smithi112 2024-09-11T00:53:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:13 smithi110 ceph-mon[34712]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:13 smithi078 ceph-mon[31315]: Deploying daemon osd.5 on smithi112 2024-09-11T00:53:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:13 smithi078 ceph-mon[31315]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:15.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:15 smithi110 ceph-mon[34712]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:15.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:15 smithi112 ceph-mon[35384]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:15.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:15 smithi078 ceph-mon[31315]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:17.271 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:17 smithi112 ceph-mon[35384]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:17.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:17 smithi110 ceph-mon[34712]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:17.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:17 smithi078 ceph-mon[31315]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:18.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:18 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:53:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:18 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:18 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:18.435 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:53:18.435 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:18.435 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:53:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:19.182 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:19 smithi112 ceph-mon[35384]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:19 smithi110 ceph-mon[34712]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:19.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:19 smithi078 ceph-mon[31315]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:21.209 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:21 smithi112 ceph-mon[35384]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:21.209 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:21.210 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:21.210 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:21.210 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:21.210 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:21 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:21 smithi110 ceph-mon[34712]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:21.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:21 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:21.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:21 smithi078 ceph-mon[31315]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:21.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:21.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:21.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:21.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:21.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:21 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.549+0000 7f68467fc640 1 -- 172.21.15.112:0/2655923623 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f68540ffa20 con 0x7f68400b5330 2024-09-11T00:53:21.551 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 5 on host 'smithi112' 2024-09-11T00:53:21.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.551+0000 7f685b269640 1 -- 172.21.15.112:0/2655923623 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68400b5330 msgr2=0x7f68400b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:21.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.551+0000 7f685b269640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68400b5330 0x7f68400b77f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6848031440 tx=0x7f6848005b50 comp rx=0 tx=0).stop 2024-09-11T00:53:21.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.551+0000 7f685b269640 1 -- 172.21.15.112:0/2655923623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685410d120 msgr2=0x7f685419eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:21.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.551+0000 7f685b269640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685410d120 0x7f685419eb50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f684c00eb10 tx=0x7f684c00c2d0 comp rx=0 tx=0).stop 2024-09-11T00:53:21.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.551+0000 7f685b269640 1 -- 172.21.15.112:0/2655923623 shutdown_connections 2024-09-11T00:53:21.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.551+0000 7f685b269640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68400b5330 0x7f68400b77f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:21.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.551+0000 7f685b269640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685410d120 0x7f685419eb50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:21.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.551+0000 7f685b269640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6854104570 0x7f68541997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:21.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.551+0000 7f685b269640 1 --2- 172.21.15.112:0/2655923623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6854103b70 0x7f6854199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:21.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.551+0000 7f685b269640 1 -- 172.21.15.112:0/2655923623 >> 172.21.15.112:0/2655923623 conn(0x7f68540fd700 msgr2=0x7f68540fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:53:21.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.551+0000 7f685b269640 1 -- 172.21.15.112:0/2655923623 shutdown_connections 2024-09-11T00:53:21.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:53:21.551+0000 7f685b269640 1 -- 172.21.15.112:0/2655923623 wait complete. 2024-09-11T00:53:22.155 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:22 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:53:22.155 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:22 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:22.155 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:22 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:22.182 DEBUG:teuthology.orchestra.run.smithi112:osd.5> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T00:53:22.187 INFO:tasks.cephadm:Deploying osd.6 on smithi078 with /dev/vg_nvme/lv_2... 2024-09-11T00:53:22.187 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T00:53:22.446 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:22 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:53:22.446 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:22 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:22.446 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:22 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:53:22.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:22 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:53:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:22 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:22 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:23.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:23 smithi078 ceph-mon[31315]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:23.353 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:23 smithi112 ceph-mon[35384]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:23.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:23 smithi110 ceph-mon[34712]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:23.974 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 00:53:23 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T00:53:23.716+0000 7f20f5f8a640 -1 osd.5 0 log_to_monitors true 2024-09-11T00:53:24.332 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:24 smithi112 ceph-mon[35384]: from='osd.5 [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:53:24.333 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:24 smithi112 ceph-mon[35384]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:53:24.337 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:24 smithi078 ceph-mon[31315]: from='osd.5 [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:53:24.337 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:24 smithi078 ceph-mon[31315]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:53:24.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:24 smithi110 ceph-mon[34712]: from='osd.5 [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:53:24.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:24 smithi110 ceph-mon[34712]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:53:25.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:25 smithi078 ceph-mon[31315]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:25.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:25 smithi078 ceph-mon[31315]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T00:53:25.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:25 smithi078 ceph-mon[31315]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T00:53:25.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:25 smithi078 ceph-mon[31315]: from='osd.5 [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:53:25.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:25 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:25.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:25 smithi078 ceph-mon[31315]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:53:25.244 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:53:25.273 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_2 2024-09-11T00:53:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:25 smithi110 ceph-mon[34712]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:25 smithi110 ceph-mon[34712]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T00:53:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:25 smithi110 ceph-mon[34712]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T00:53:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:25 smithi110 ceph-mon[34712]: from='osd.5 [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:53:25.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:25 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:25.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:25 smithi110 ceph-mon[34712]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:53:25.454 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:25 smithi112 ceph-mon[35384]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:25.454 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:25 smithi112 ceph-mon[35384]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T00:53:25.454 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:25 smithi112 ceph-mon[35384]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T00:53:25.454 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:25 smithi112 ceph-mon[35384]: from='osd.5 [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:53:25.454 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:25 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:25.454 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:25 smithi112 ceph-mon[35384]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:53:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:53:25.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 00:53:25 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T00:53:25.451+0000 7f20f2721640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T00:53:25.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 00:53:25 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T00:53:25.459+0000 7f20e9d2c640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:53:26.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.052+0000 7f20aa95a640 1 -- 172.21.15.78:0/251279192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4101cb0 msgr2=0x7f20a4102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:26.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.052+0000 7f20aa95a640 1 --2- 172.21.15.78:0/251279192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4101cb0 0x7f20a4102090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2094009900 tx=0x7f209402a410 comp rx=0 tx=0).stop 2024-09-11T00:53:26.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.052+0000 7f20aa95a640 1 -- 172.21.15.78:0/251279192 shutdown_connections 2024-09-11T00:53:26.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.052+0000 7f20aa95a640 1 --2- 172.21.15.78:0/251279192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4108040 0x7f20a410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:26.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.052+0000 7f20aa95a640 1 --2- 172.21.15.78:0/251279192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20a41076c0 0x7f20a4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:26.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.052+0000 7f20aa95a640 1 --2- 172.21.15.78:0/251279192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4101cb0 0x7f20a4102090 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:26.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.052+0000 7f20aa95a640 1 -- 172.21.15.78:0/251279192 >> 172.21.15.78:0/251279192 conn(0x7f20a40fd700 msgr2=0x7f20a40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:53:26.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.052+0000 7f20aa95a640 1 -- 172.21.15.78:0/251279192 shutdown_connections 2024-09-11T00:53:26.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.052+0000 7f20aa95a640 1 -- 172.21.15.78:0/251279192 wait complete. 2024-09-11T00:53:26.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.053+0000 7f20aa95a640 1 Processor -- start 2024-09-11T00:53:26.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.053+0000 7f20aa95a640 1 -- start start 2024-09-11T00:53:26.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.053+0000 7f20aa95a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4101cb0 0x7f20a419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:53:26.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.053+0000 7f20aa95a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20a41076c0 0x7f20a419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.053+0000 7f20aa95a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4108040 0x7f20a419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.053+0000 7f20aa95a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4111d00 con 0x7f20a4108040 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.053+0000 7f20aa95a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4111b80 con 0x7f20a41076c0 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.053+0000 7f20aa95a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4111e80 con 0x7f20a4101cb0 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.053+0000 7f20a8ef7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4108040 0x7f20a419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.053+0000 7f20a8ef7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4108040 0x7f20a419ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43008/0 (socket says 172.21.15.78:43008) 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.053+0000 7f20a8ef7640 1 -- 172.21.15.78:0/1035647035 learned_addr learned my addr 172.21.15.78:0/1035647035 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.053+0000 7f20a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4101cb0 0x7f20a419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.054+0000 7f20a37fe640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20a41076c0 0x7f20a419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.054+0000 7f20a8ef7640 1 -- 172.21.15.78:0/1035647035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4101cb0 msgr2=0x7f20a419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.054+0000 7f20a8ef7640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4101cb0 0x7f20a419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.054+0000 7f20a8ef7640 1 -- 172.21.15.78:0/1035647035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20a41076c0 msgr2=0x7f20a419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:26.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.054+0000 7f20a8ef7640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20a41076c0 0x7f20a419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:26.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.054+0000 7f20a8ef7640 1 -- 172.21.15.78:0/1035647035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a41a5460 con 0x7f20a4108040 2024-09-11T00:53:26.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.054+0000 7f20a8ef7640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4108040 0x7f20a419ab20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f209c00b970 tx=0x7f209c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:53:26.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.054+0000 7f20a17fa640 1 -- 172.21.15.78:0/1035647035 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c019070 con 0x7f20a4108040 2024-09-11T00:53:26.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.054+0000 7f20a17fa640 1 -- 172.21.15.78:0/1035647035 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f209c014440 con 0x7f20a4108040 2024-09-11T00:53:26.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.054+0000 7f20aa95a640 1 -- 172.21.15.78:0/1035647035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a41a5750 con 0x7f20a4108040 2024-09-11T00:53:26.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.054+0000 7f20a17fa640 1 -- 172.21.15.78:0/1035647035 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c0075d0 con 0x7f20a4108040 2024-09-11T00:53:26.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.054+0000 7f20aa95a640 1 -- 172.21.15.78:0/1035647035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a41a5d10 con 0x7f20a4108040 2024-09-11T00:53:26.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.055+0000 7f20aa95a640 1 -- 172.21.15.78:0/1035647035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a4102090 con 0x7f20a4108040 2024-09-11T00:53:26.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.056+0000 7f20a17fa640 1 -- 172.21.15.78:0/1035647035 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f209c00cb40 con 0x7f20a4108040 2024-09-11T00:53:26.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.057+0000 7f20a17fa640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f208c0b53b0 0x7f208c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:53:26.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.057+0000 7f20a3fff640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f208c0b53b0 0x7f208c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:53:26.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.057+0000 7f20a17fa640 1 -- 172.21.15.78:0/1035647035 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4468+0+0 (secure 0 0 0) 0x7f209c0e4470 con 0x7f20a4108040 2024-09-11T00:53:26.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.057+0000 7f20a3fff640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f208c0b53b0 0x7f208c0b7870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2094031440 tx=0x7f2094005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:53:26.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.062+0000 7f20a17fa640 1 -- 172.21.15.78:0/1035647035 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f209c0abbe0 con 0x7f20a4108040 2024-09-11T00:53:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-11T00:53:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T00:53:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: Detected new or changed devices on smithi112 2024-09-11T00:53:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi112 to 8544M 2024-09-11T00:53:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='osd.5 ' entity='osd.5' 2024-09-11T00:53:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: osd.5 [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096] boot 2024-09-11T00:53:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T00:53:26.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:26.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:26.251+0000 7f20aa95a640 1 -- 172.21.15.78:0/1035647035 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f20a4063590 con 0x7f208c0b53b0 2024-09-11T00:53:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: Detected new or changed devices on smithi112 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi112 to 8544M 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:26.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:26.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='osd.5 ' entity='osd.5' 2024-09-11T00:53:26.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:26.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: osd.5 [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096] boot 2024-09-11T00:53:26.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T00:53:26.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-11T00:53:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T00:53:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: Detected new or changed devices on smithi112 2024-09-11T00:53:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:26.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:26.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi112 to 8544M 2024-09-11T00:53:26.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:26.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:26.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:26.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:26.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='osd.5 ' entity='osd.5' 2024-09-11T00:53:26.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:53:26.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: osd.5 [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096] boot 2024-09-11T00:53:26.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T00:53:26.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:53:27.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:27 smithi078 ceph-mon[31315]: purged_snaps scrub starts 2024-09-11T00:53:27.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:27 smithi078 ceph-mon[31315]: purged_snaps scrub ok 2024-09-11T00:53:27.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:27 smithi078 ceph-mon[31315]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:53:27.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:53:27.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:53:27.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:27.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:27 smithi078 ceph-mon[31315]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T00:53:27.341 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:27 smithi110 ceph-mon[34712]: purged_snaps scrub starts 2024-09-11T00:53:27.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:27 smithi110 ceph-mon[34712]: purged_snaps scrub ok 2024-09-11T00:53:27.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:27 smithi110 ceph-mon[34712]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:53:27.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:53:27.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:53:27.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:27.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:27 smithi110 ceph-mon[34712]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T00:53:27.378 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:27 smithi112 ceph-mon[35384]: purged_snaps scrub starts 2024-09-11T00:53:27.378 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:27 smithi112 ceph-mon[35384]: purged_snaps scrub ok 2024-09-11T00:53:27.378 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:27 smithi112 ceph-mon[35384]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:53:27.378 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:53:27.378 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:53:27.378 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:27.378 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:27 smithi112 ceph-mon[35384]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T00:53:29.177 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:29 smithi078 ceph-mon[31315]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:29.178 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:29 smithi078 ceph-mon[31315]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T00:53:29.178 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:29 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2732888772' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d508a037-8f56-436b-b7bf-9a1c69f6d9e2"} : dispatch 2024-09-11T00:53:29.178 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:29 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2732888772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d508a037-8f56-436b-b7bf-9a1c69f6d9e2"}]': finished 2024-09-11T00:53:29.178 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:29 smithi078 ceph-mon[31315]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T00:53:29.178 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:29 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:29.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:29 smithi110 ceph-mon[34712]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:29.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:29 smithi110 ceph-mon[34712]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T00:53:29.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:29 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2732888772' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d508a037-8f56-436b-b7bf-9a1c69f6d9e2"} : dispatch 2024-09-11T00:53:29.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:29 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2732888772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d508a037-8f56-436b-b7bf-9a1c69f6d9e2"}]': finished 2024-09-11T00:53:29.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:29 smithi110 ceph-mon[34712]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T00:53:29.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:29 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:29 smithi112 ceph-mon[35384]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:29 smithi112 ceph-mon[35384]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T00:53:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:29 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2732888772' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d508a037-8f56-436b-b7bf-9a1c69f6d9e2"} : dispatch 2024-09-11T00:53:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:29 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2732888772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d508a037-8f56-436b-b7bf-9a1c69f6d9e2"}]': finished 2024-09-11T00:53:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:29 smithi112 ceph-mon[35384]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T00:53:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:29 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:30 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/659705020' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:53:30.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:30 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/659705020' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:53:30.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:30 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/659705020' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:53:31.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:31 smithi110 ceph-mon[34712]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T00:53:31.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:31 smithi112 ceph-mon[35384]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T00:53:31.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:31 smithi078 ceph-mon[31315]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T00:53:32.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:32 smithi110 ceph-mon[34712]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-11T00:53:32.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:32 smithi112 ceph-mon[35384]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-11T00:53:32.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:32 smithi078 ceph-mon[31315]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-11T00:53:35.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:35 smithi078 ceph-mon[31315]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-11T00:53:35.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:35 smithi110 ceph-mon[34712]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-11T00:53:35.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:35 smithi112 ceph-mon[35384]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-11T00:53:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:37 smithi078 ceph-mon[31315]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:53:37.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:37 smithi110 ceph-mon[34712]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:53:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:37 smithi112 ceph-mon[35384]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:53:38.181 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:38 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-11T00:53:38.181 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:38 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:38 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-11T00:53:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:38 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:38.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:38 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-11T00:53:38.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:38 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:39.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:39 smithi078 ceph-mon[31315]: Deploying daemon osd.6 on smithi078 2024-09-11T00:53:39.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:39 smithi078 ceph-mon[31315]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:53:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:39 smithi110 ceph-mon[34712]: Deploying daemon osd.6 on smithi078 2024-09-11T00:53:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:39 smithi110 ceph-mon[34712]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:53:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:39 smithi112 ceph-mon[35384]: Deploying daemon osd.6 on smithi078 2024-09-11T00:53:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:39 smithi112 ceph-mon[35384]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:53:41.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:41 smithi078 ceph-mon[31315]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-11T00:53:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:41 smithi110 ceph-mon[34712]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-11T00:53:41.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:41 smithi112 ceph-mon[35384]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-11T00:53:43.334 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:43 smithi078 ceph-mon[31315]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:43.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:43 smithi110 ceph-mon[34712]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:43 smithi112 ceph-mon[35384]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:44.089 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:53:44.089 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:44.089 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:44.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:53:44.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:44.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:53:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:45 smithi110 ceph-mon[34712]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:45.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:45 smithi112 ceph-mon[35384]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:45.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:45 smithi078 ceph-mon[31315]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:47.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:47 smithi078 ceph-mon[31315]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:47.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:47.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:47.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:47.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:47.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:47.269 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 6 on host 'smithi078' 2024-09-11T00:53:47.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.268+0000 7f20a17fa640 1 -- 172.21.15.78:0/1035647035 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f20a4063590 con 0x7f208c0b53b0 2024-09-11T00:53:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.270+0000 7f20aa95a640 1 -- 172.21.15.78:0/1035647035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f208c0b53b0 msgr2=0x7f208c0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.270+0000 7f20aa95a640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f208c0b53b0 0x7f208c0b7870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2094031440 tx=0x7f2094005e80 comp rx=0 tx=0).stop 2024-09-11T00:53:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.270+0000 7f20aa95a640 1 -- 172.21.15.78:0/1035647035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4108040 msgr2=0x7f20a419ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.270+0000 7f20aa95a640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4108040 0x7f20a419ab20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f209c00b970 tx=0x7f209c00be30 comp rx=0 tx=0).stop 2024-09-11T00:53:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.270+0000 7f20aa95a640 1 -- 172.21.15.78:0/1035647035 shutdown_connections 2024-09-11T00:53:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.270+0000 7f20aa95a640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f208c0b53b0 0x7f208c0b7870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.270+0000 7f20aa95a640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4108040 0x7f20a419ab20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.270+0000 7f20aa95a640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20a41076c0 0x7f20a419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.270+0000 7f20aa95a640 1 --2- 172.21.15.78:0/1035647035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a4101cb0 0x7f20a419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:47.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.270+0000 7f20aa95a640 1 -- 172.21.15.78:0/1035647035 >> 172.21.15.78:0/1035647035 conn(0x7f20a40fd700 msgr2=0x7f20a40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:53:47.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.270+0000 7f20aa95a640 1 -- 172.21.15.78:0/1035647035 shutdown_connections 2024-09-11T00:53:47.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:53:47.270+0000 7f20aa95a640 1 -- 172.21.15.78:0/1035647035 wait complete. 2024-09-11T00:53:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:47 smithi110 ceph-mon[34712]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:47.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:47.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:47.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:47.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:47.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:47 smithi112 ceph-mon[35384]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:48.090 DEBUG:teuthology.orchestra.run.smithi078:osd.6> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T00:53:48.097 INFO:tasks.cephadm:Deploying osd.7 on smithi110 with /dev/vg_nvme/lv_2... 2024-09-11T00:53:48.098 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T00:53:48.356 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:53:48.390 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:48 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:53:48.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:48 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:48.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:48 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:48 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:53:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:48 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:48 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:48.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:48 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:53:48.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:48 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:48.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:48 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:49.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:49 smithi078 ceph-mon[31315]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:49 smithi110 ceph-mon[34712]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:49 smithi112 ceph-mon[35384]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:49.649 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 00:53:49 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T00:53:49.428+0000 7fce3576b640 -1 osd.6 0 log_to_monitors true 2024-09-11T00:53:50.301 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:50 smithi110 ceph-mon[34712]: from='osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-11T00:53:50.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:50 smithi078 ceph-mon[31315]: from='osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-11T00:53:50.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:50 smithi112 ceph-mon[35384]: from='osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-11T00:53:51.113 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:53:51.140 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_2 2024-09-11T00:53:51.403 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:53:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:51 smithi110 ceph-mon[34712]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:51 smithi110 ceph-mon[34712]: from='osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T00:53:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:51 smithi110 ceph-mon[34712]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T00:53:51.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:51 smithi110 ceph-mon[34712]: from='osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:53:51.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:51 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:51 smithi112 ceph-mon[35384]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:51 smithi112 ceph-mon[35384]: from='osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T00:53:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:51 smithi112 ceph-mon[35384]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T00:53:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:51 smithi112 ceph-mon[35384]: from='osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:53:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:51 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:51.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:51 smithi078 ceph-mon[31315]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:51.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:51 smithi078 ceph-mon[31315]: from='osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T00:53:51.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:51 smithi078 ceph-mon[31315]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T00:53:51.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:51 smithi078 ceph-mon[31315]: from='osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:53:51.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:51 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:51.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 00:53:51 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T00:53:51.075+0000 7fce316ee640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T00:53:51.498 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 00:53:51 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T00:53:51.082+0000 7fce2d515640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:53:51.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.879+0000 7fed64762640 1 -- 172.21.15.110:0/3560580533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed5c1087b0 msgr2=0x7fed5c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:51.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.879+0000 7fed64762640 1 --2- 172.21.15.110:0/3560580533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed5c1087b0 0x7fed5c108bf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fed58009900 tx=0x7fed5802a410 comp rx=0 tx=0).stop 2024-09-11T00:53:51.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.879+0000 7fed64762640 1 -- 172.21.15.110:0/3560580533 shutdown_connections 2024-09-11T00:53:51.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.879+0000 7fed64762640 1 --2- 172.21.15.110:0/3560580533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed5c109130 0x7fed5c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:51.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.879+0000 7fed64762640 1 --2- 172.21.15.110:0/3560580533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed5c1087b0 0x7fed5c108bf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:51.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.879+0000 7fed64762640 1 --2- 172.21.15.110:0/3560580533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5c102da0 0x7fed5c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:51.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.879+0000 7fed64762640 1 -- 172.21.15.110:0/3560580533 >> 172.21.15.110:0/3560580533 conn(0x7fed5c0fe810 msgr2=0x7fed5c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:53:51.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.879+0000 7fed64762640 1 -- 172.21.15.110:0/3560580533 shutdown_connections 2024-09-11T00:53:51.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.879+0000 7fed64762640 1 -- 172.21.15.110:0/3560580533 wait complete. 2024-09-11T00:53:51.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.880+0000 7fed64762640 1 Processor -- start 2024-09-11T00:53:51.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.880+0000 7fed64762640 1 -- start start 2024-09-11T00:53:51.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.880+0000 7fed64762640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed5c102da0 0x7fed5c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:53:51.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.880+0000 7fed64762640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5c1087b0 0x7fed5c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:53:51.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.880+0000 7fed64762640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed5c109130 0x7fed5c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:53:51.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.880+0000 7fed64762640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c112df0 con 0x7fed5c102da0 2024-09-11T00:53:51.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.880+0000 7fed64762640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c112c70 con 0x7fed5c109130 2024-09-11T00:53:51.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.880+0000 7fed64762640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c112f70 con 0x7fed5c1087b0 2024-09-11T00:53:51.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.880+0000 7fed62cff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed5c109130 0x7fed5c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:53:51.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.880+0000 7fed62cff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed5c109130 0x7fed5c075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48182/0 (socket says 172.21.15.110:48182) 2024-09-11T00:53:51.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.880+0000 7fed62cff640 1 -- 172.21.15.110:0/1484378719 learned_addr learned my addr 172.21.15.110:0/1484378719 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-11T00:53:51.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.880+0000 7fed624fe640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed5c102da0 0x7fed5c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:53:51.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed61cfd640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5c1087b0 0x7fed5c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:53:51.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed62cff640 1 -- 172.21.15.110:0/1484378719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5c1087b0 msgr2=0x7fed5c0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:51.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed62cff640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5c1087b0 0x7fed5c0759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:51.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed62cff640 1 -- 172.21.15.110:0/1484378719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed5c102da0 msgr2=0x7fed5c0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:53:51.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed62cff640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed5c102da0 0x7fed5c0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:53:51.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed62cff640 1 -- 172.21.15.110:0/1484378719 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c1a7700 con 0x7fed5c109130 2024-09-11T00:53:51.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed624fe640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed5c102da0 0x7fed5c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:53:51.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed62cff640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed5c109130 0x7fed5c075f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed54009fd0 tx=0x7fed5400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:53:51.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed537fe640 1 -- 172.21.15.110:0/1484378719 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54017070 con 0x7fed5c109130 2024-09-11T00:53:51.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed64762640 1 -- 172.21.15.110:0/1484378719 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c1a79f0 con 0x7fed5c109130 2024-09-11T00:53:51.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed64762640 1 -- 172.21.15.110:0/1484378719 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c1a7f30 con 0x7fed5c109130 2024-09-11T00:53:51.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed537fe640 1 -- 172.21.15.110:0/1484378719 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed54003e20 con 0x7fed5c109130 2024-09-11T00:53:51.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.881+0000 7fed537fe640 1 -- 172.21.15.110:0/1484378719 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54010460 con 0x7fed5c109130 2024-09-11T00:53:51.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.882+0000 7fed64762640 1 -- 172.21.15.110:0/1484378719 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed400052c0 con 0x7fed5c109130 2024-09-11T00:53:51.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.883+0000 7fed537fe640 1 -- 172.21.15.110:0/1484378719 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fed54003970 con 0x7fed5c109130 2024-09-11T00:53:51.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.884+0000 7fed537fe640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fed480b53b0 0x7fed480b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:53:51.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.884+0000 7fed537fe640 1 -- 172.21.15.110:0/1484378719 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4900+0+0 (secure 0 0 0) 0x7fed540e4ad0 con 0x7fed5c109130 2024-09-11T00:53:51.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.884+0000 7fed624fe640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fed480b53b0 0x7fed480b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:53:51.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.885+0000 7fed624fe640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fed480b53b0 0x7fed480b7870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009930 tx=0x7fed4c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:53:51.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:51.888+0000 7fed537fe640 1 -- 172.21.15.110:0/1484378719 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fed540ac6e0 con 0x7fed5c109130 2024-09-11T00:53:52.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:53:52.073+0000 7fed64762640 1 -- 172.21.15.110:0/1484378719 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fed40000bf0 con 0x7fed480b53b0 2024-09-11T00:53:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: from='osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-11T00:53:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T00:53:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:52.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:52.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:52.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:52.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:52.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:52.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:52.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:52.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:52.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:52.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:52 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: from='osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-11T00:53:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T00:53:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:52.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:52.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:52.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:52 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: from='osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-11T00:53:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T00:53:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:53:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:53:52.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:52 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:53:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:53 smithi110 ceph-mon[34712]: purged_snaps scrub starts 2024-09-11T00:53:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:53 smithi110 ceph-mon[34712]: purged_snaps scrub ok 2024-09-11T00:53:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:53 smithi110 ceph-mon[34712]: Detected new or changed devices on smithi078 2024-09-11T00:53:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:53 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi078 to 5696M 2024-09-11T00:53:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:53 smithi110 ceph-mon[34712]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:53 smithi110 ceph-mon[34712]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:53:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:53 smithi110 ceph-mon[34712]: osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775] boot 2024-09-11T00:53:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:53 smithi110 ceph-mon[34712]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T00:53:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:53 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:53 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:53:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:53 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:53:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:53 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:53 smithi112 ceph-mon[35384]: purged_snaps scrub starts 2024-09-11T00:53:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:53 smithi112 ceph-mon[35384]: purged_snaps scrub ok 2024-09-11T00:53:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:53 smithi112 ceph-mon[35384]: Detected new or changed devices on smithi078 2024-09-11T00:53:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:53 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi078 to 5696M 2024-09-11T00:53:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:53 smithi112 ceph-mon[35384]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:53 smithi112 ceph-mon[35384]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:53:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:53 smithi112 ceph-mon[35384]: osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775] boot 2024-09-11T00:53:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:53 smithi112 ceph-mon[35384]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T00:53:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:53 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:53.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:53 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:53:53.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:53 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:53:53.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:53 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:53 smithi078 ceph-mon[31315]: purged_snaps scrub starts 2024-09-11T00:53:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:53 smithi078 ceph-mon[31315]: purged_snaps scrub ok 2024-09-11T00:53:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:53 smithi078 ceph-mon[31315]: Detected new or changed devices on smithi078 2024-09-11T00:53:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:53 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi078 to 5696M 2024-09-11T00:53:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:53 smithi078 ceph-mon[31315]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:53:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:53 smithi078 ceph-mon[31315]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:53:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:53 smithi078 ceph-mon[31315]: osd.6 [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775] boot 2024-09-11T00:53:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:53 smithi078 ceph-mon[31315]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T00:53:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:53 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:53:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:53 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:53:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:53 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:53:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:53 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:53:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:54 smithi110 ceph-mon[34712]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T00:53:54.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:54 smithi110 ceph-mon[34712]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:53:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:54 smithi112 ceph-mon[35384]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T00:53:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:54 smithi112 ceph-mon[35384]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:53:54.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:54 smithi078 ceph-mon[31315]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T00:53:54.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:54 smithi078 ceph-mon[31315]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:53:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:55 smithi110 ceph-mon[34712]: from='client.? 172.21.15.110:0/2117089857' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e8842047-09a6-48b8-8d0a-2d54e7f89063"} : dispatch 2024-09-11T00:53:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:55 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e8842047-09a6-48b8-8d0a-2d54e7f89063"} : dispatch 2024-09-11T00:53:55.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:55 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8842047-09a6-48b8-8d0a-2d54e7f89063"}]': finished 2024-09-11T00:53:55.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:55 smithi110 ceph-mon[34712]: osdmap e44: 8 total, 7 up, 8 in 2024-09-11T00:53:55.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:55 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:53:55.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:55 smithi110 ceph-mon[34712]: from='client.? 172.21.15.110:0/1788018897' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:53:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:55 smithi112 ceph-mon[35384]: from='client.? 172.21.15.110:0/2117089857' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e8842047-09a6-48b8-8d0a-2d54e7f89063"} : dispatch 2024-09-11T00:53:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:55 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e8842047-09a6-48b8-8d0a-2d54e7f89063"} : dispatch 2024-09-11T00:53:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:55 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8842047-09a6-48b8-8d0a-2d54e7f89063"}]': finished 2024-09-11T00:53:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:55 smithi112 ceph-mon[35384]: osdmap e44: 8 total, 7 up, 8 in 2024-09-11T00:53:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:55 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:53:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:55 smithi112 ceph-mon[35384]: from='client.? 172.21.15.110:0/1788018897' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:53:55.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:55 smithi078 ceph-mon[31315]: from='client.? 172.21.15.110:0/2117089857' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e8842047-09a6-48b8-8d0a-2d54e7f89063"} : dispatch 2024-09-11T00:53:55.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:55 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e8842047-09a6-48b8-8d0a-2d54e7f89063"} : dispatch 2024-09-11T00:53:55.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:55 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8842047-09a6-48b8-8d0a-2d54e7f89063"}]': finished 2024-09-11T00:53:55.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:55 smithi078 ceph-mon[31315]: osdmap e44: 8 total, 7 up, 8 in 2024-09-11T00:53:55.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:55 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:53:55.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:55 smithi078 ceph-mon[31315]: from='client.? 172.21.15.110:0/1788018897' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:53:56.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:56 smithi110 ceph-mon[34712]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:53:56.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:56 smithi112 ceph-mon[35384]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:53:56.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:56 smithi078 ceph-mon[31315]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:53:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:53:59 smithi110 ceph-mon[34712]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:53:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:53:59 smithi112 ceph-mon[35384]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:53:59.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:53:59 smithi078 ceph-mon[31315]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:01.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:01 smithi110 ceph-mon[34712]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:01.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:01 smithi112 ceph-mon[35384]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:01 smithi078 ceph-mon[31315]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:03.313 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:03 smithi110 ceph-mon[34712]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:03.314 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-11T00:54:03.314 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:03 smithi112 ceph-mon[35384]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-11T00:54:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:03 smithi078 ceph-mon[31315]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-11T00:54:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:04.289 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:04 smithi110 ceph-mon[34712]: Deploying daemon osd.7 on smithi110 2024-09-11T00:54:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:04 smithi112 ceph-mon[35384]: Deploying daemon osd.7 on smithi110 2024-09-11T00:54:04.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:04 smithi078 ceph-mon[31315]: Deploying daemon osd.7 on smithi110 2024-09-11T00:54:05.395 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:05 smithi110 ceph-mon[34712]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:05 smithi112 ceph-mon[35384]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:05.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:05 smithi078 ceph-mon[31315]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:07 smithi110 ceph-mon[34712]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:07.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:07 smithi112 ceph-mon[35384]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:07.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:07 smithi078 ceph-mon[31315]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:09 smithi110 ceph-mon[34712]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:09.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:09 smithi112 ceph-mon[35384]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:09.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:09 smithi078 ceph-mon[31315]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:10.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:54:10.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:10.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:54:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:10.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:54:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:11.162 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:11 smithi110 ceph-mon[34712]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:11 smithi112 ceph-mon[35384]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:11.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:11 smithi078 ceph-mon[31315]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:12.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.509+0000 7fed537fe640 1 -- 172.21.15.110:0/1484378719 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fed40000bf0 con 0x7fed480b53b0 2024-09-11T00:54:12.511 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 7 on host 'smithi110' 2024-09-11T00:54:12.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.512+0000 7fed64762640 1 -- 172.21.15.110:0/1484378719 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fed480b53b0 msgr2=0x7fed480b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:54:12.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.512+0000 7fed64762640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fed480b53b0 0x7fed480b7870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009930 tx=0x7fed4c008040 comp rx=0 tx=0).stop 2024-09-11T00:54:12.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.512+0000 7fed64762640 1 -- 172.21.15.110:0/1484378719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed5c109130 msgr2=0x7fed5c075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:54:12.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.512+0000 7fed64762640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed5c109130 0x7fed5c075f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed54009fd0 tx=0x7fed5400b260 comp rx=0 tx=0).stop 2024-09-11T00:54:12.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.512+0000 7fed64762640 1 -- 172.21.15.110:0/1484378719 shutdown_connections 2024-09-11T00:54:12.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.512+0000 7fed64762640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fed480b53b0 0x7fed480b7870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:12.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.512+0000 7fed64762640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed5c109130 0x7fed5c075f20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:12.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.512+0000 7fed64762640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5c1087b0 0x7fed5c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:12.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.512+0000 7fed64762640 1 --2- 172.21.15.110:0/1484378719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed5c102da0 0x7fed5c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:12.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.512+0000 7fed64762640 1 -- 172.21.15.110:0/1484378719 >> 172.21.15.110:0/1484378719 conn(0x7fed5c0fe810 msgr2=0x7fed5c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:54:12.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.512+0000 7fed64762640 1 -- 172.21.15.110:0/1484378719 shutdown_connections 2024-09-11T00:54:12.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:54:12.512+0000 7fed64762640 1 -- 172.21.15.110:0/1484378719 wait complete. 2024-09-11T00:54:13.091 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:13 smithi110 ceph-mon[34712]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:13.091 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:13.091 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:13.126 DEBUG:teuthology.orchestra.run.smithi110:osd.7> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T00:54:13.128 INFO:tasks.cephadm:Deploying osd.8 on smithi112 with /dev/vg_nvme/lv_2... 2024-09-11T00:54:13.129 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T00:54:13.386 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:54:13.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:13 smithi112 ceph-mon[35384]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:13 smithi078 ceph-mon[31315]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:14.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:14 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:14.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:14 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:14.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:14 smithi110 ceph-mon[34712]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:14 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:14 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:14 smithi112 ceph-mon[35384]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:14 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:14 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:14 smithi078 ceph-mon[31315]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:15.251 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 00:54:15 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T00:54:15.188+0000 7f796f6f4640 -1 osd.7 0 log_to_monitors true 2024-09-11T00:54:15.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:15 smithi112 ceph-mon[35384]: from='osd.7 [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:54:15.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:15 smithi112 ceph-mon[35384]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:54:15.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:15 smithi110 ceph-mon[34712]: from='osd.7 [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:54:15.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:15 smithi110 ceph-mon[34712]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:54:15.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:15 smithi078 ceph-mon[31315]: from='osd.7 [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:54:15.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:15 smithi078 ceph-mon[31315]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:54:16.021 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:54:16.046 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-09-11T00:54:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:54:16.416 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:16 smithi112 ceph-mon[35384]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T00:54:16.416 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:16 smithi112 ceph-mon[35384]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T00:54:16.416 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:16 smithi112 ceph-mon[35384]: from='osd.7 [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:54:16.416 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:54:16.416 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:16 smithi112 ceph-mon[35384]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:54:16.416 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:16 smithi112 ceph-mon[35384]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:16.416 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:16 smithi112 ceph-mon[35384]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T00:54:16.616 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:16 smithi110 ceph-mon[34712]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T00:54:16.616 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:16 smithi110 ceph-mon[34712]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T00:54:16.616 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:16 smithi110 ceph-mon[34712]: from='osd.7 [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:54:16.616 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:54:16.616 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:16 smithi110 ceph-mon[34712]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:54:16.616 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:16 smithi110 ceph-mon[34712]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:16.616 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:16 smithi110 ceph-mon[34712]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T00:54:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:16 smithi078 ceph-mon[31315]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T00:54:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:16 smithi078 ceph-mon[31315]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T00:54:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:16 smithi078 ceph-mon[31315]: from='osd.7 [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:54:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:54:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:16 smithi078 ceph-mon[31315]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:54:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:16 smithi078 ceph-mon[31315]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:16 smithi078 ceph-mon[31315]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T00:54:16.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.813+0000 7f92f93fe640 1 -- 172.21.15.112:0/999221770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4102dc0 msgr2=0x7f92f41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:54:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.813+0000 7f92f93fe640 1 --2- 172.21.15.112:0/999221770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4102dc0 0x7f92f41031a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009900 tx=0x7f92e402a410 comp rx=0 tx=0).stop 2024-09-11T00:54:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.813+0000 7f92f93fe640 1 -- 172.21.15.112:0/999221770 shutdown_connections 2024-09-11T00:54:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.813+0000 7f92f93fe640 1 --2- 172.21.15.112:0/999221770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f4109150 0x7f92f410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.813+0000 7f92f93fe640 1 --2- 172.21.15.112:0/999221770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92f41087d0 0x7f92f4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.813+0000 7f92f93fe640 1 --2- 172.21.15.112:0/999221770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4102dc0 0x7f92f41031a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.813+0000 7f92f93fe640 1 -- 172.21.15.112:0/999221770 >> 172.21.15.112:0/999221770 conn(0x7f92f40fe830 msgr2=0x7f92f4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:54:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.813+0000 7f92f93fe640 1 -- 172.21.15.112:0/999221770 shutdown_connections 2024-09-11T00:54:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.813+0000 7f92f93fe640 1 -- 172.21.15.112:0/999221770 wait complete. 2024-09-11T00:54:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f93fe640 1 Processor -- start 2024-09-11T00:54:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f93fe640 1 -- start start 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f93fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4102dc0 0x7f92f41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f93fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f41087d0 0x7f92f4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f93fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92f4109150 0x7f92f419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f93fe640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4112e10 con 0x7f92f41087d0 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f93fe640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4112c90 con 0x7f92f4109150 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f93fe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4112f90 con 0x7f92f4102dc0 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4102dc0 0x7f92f41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4102dc0 0x7f92f41992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52914/0 (socket says 172.21.15.112:52914) 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f2ffd640 1 -- 172.21.15.112:0/2525234908 learned_addr learned my addr 172.21.15.112:0/2525234908 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92ebfff640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f41087d0 0x7f92f4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f37fe640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92f4109150 0x7f92f419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f2ffd640 1 -- 172.21.15.112:0/2525234908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92f4109150 msgr2=0x7f92f419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:54:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f2ffd640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92f4109150 0x7f92f419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f2ffd640 1 -- 172.21.15.112:0/2525234908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f41087d0 msgr2=0x7f92f4199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f2ffd640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f41087d0 0x7f92f4199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f2ffd640 1 -- 172.21.15.112:0/2525234908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f419f370 con 0x7f92f4102dc0 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92ebfff640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f41087d0 0x7f92f4199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.814+0000 7f92f37fe640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92f4109150 0x7f92f419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.815+0000 7f92f2ffd640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4102dc0 0x7f92f41992c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f92e4005940 tx=0x7f92e40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.815+0000 7f92f0ff9640 1 -- 172.21.15.112:0/2525234908 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e403f030 con 0x7f92f4102dc0 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.815+0000 7f92f93fe640 1 -- 172.21.15.112:0/2525234908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f419f600 con 0x7f92f4102dc0 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.815+0000 7f92f0ff9640 1 -- 172.21.15.112:0/2525234908 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e4003cb0 con 0x7f92f4102dc0 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.815+0000 7f92f0ff9640 1 -- 172.21.15.112:0/2525234908 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4035070 con 0x7f92f4102dc0 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.815+0000 7f92f93fe640 1 -- 172.21.15.112:0/2525234908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f41a5d20 con 0x7f92f4102dc0 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.816+0000 7f92f93fe640 1 -- 172.21.15.112:0/2525234908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f41031a0 con 0x7f92f4102dc0 2024-09-11T00:54:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.817+0000 7f92f0ff9640 1 -- 172.21.15.112:0/2525234908 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f92e403bbf0 con 0x7f92f4102dc0 2024-09-11T00:54:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.818+0000 7f92f0ff9640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f92e00b5330 0x7f92e00b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:54:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.818+0000 7f92f0ff9640 1 -- 172.21.15.112:0/2525234908 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5332+0+0 (secure 0 0 0) 0x7f92e4105f30 con 0x7f92f4102dc0 2024-09-11T00:54:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.818+0000 7f92ebfff640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f92e00b5330 0x7f92e00b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:54:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.818+0000 7f92ebfff640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f92e00b5330 0x7f92e00b77f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0058c0 tx=0x7f92dc005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:54:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:16.821+0000 7f92f0ff9640 1 -- 172.21.15.112:0/2525234908 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f92e40cd990 con 0x7f92f4102dc0 2024-09-11T00:54:16.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 00:54:16 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T00:54:16.643+0000 7f796b677640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T00:54:16.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 00:54:16 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T00:54:16.651+0000 7f7963496640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:54:17.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:17.012+0000 7f92f93fe640 1 -- 172.21.15.112:0/2525234908 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f92f4063590 con 0x7f92e00b5330 2024-09-11T00:54:17.265 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T00:54:17.265 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:54:17.265 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:54:17.265 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: Detected new or changed devices on smithi110 2024-09-11T00:54:17.266 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='osd.7 ' entity='osd.7' 2024-09-11T00:54:17.266 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.266 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.550 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:17.550 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:17.550 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:17.550 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi110 to 5696M 2024-09-11T00:54:17.550 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.550 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:17.550 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:54:17.551 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.551 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:54:17.551 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:17.551 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:54:17.551 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.551 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:54:17.551 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:54:17.551 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:54:17.551 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:17.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T00:54:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:54:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:54:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: Detected new or changed devices on smithi110 2024-09-11T00:54:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='osd.7 ' entity='osd.7' 2024-09-11T00:54:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi110 to 5696M 2024-09-11T00:54:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:54:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:54:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:54:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:54:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:54:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:54:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T00:54:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:54:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:54:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: Detected new or changed devices on smithi110 2024-09-11T00:54:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='osd.7 ' entity='osd.7' 2024-09-11T00:54:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi110 to 5696M 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:54:17.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:54:17.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:54:17.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:18.429 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:18 smithi112 ceph-mon[35384]: purged_snaps scrub starts 2024-09-11T00:54:18.429 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:18 smithi112 ceph-mon[35384]: purged_snaps scrub ok 2024-09-11T00:54:18.429 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:18 smithi112 ceph-mon[35384]: osd.7 [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594] boot 2024-09-11T00:54:18.429 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:18 smithi112 ceph-mon[35384]: osdmap e47: 8 total, 8 up, 8 in 2024-09-11T00:54:18.429 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:18 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:54:18.429 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:18 smithi112 ceph-mon[35384]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:18.429 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:18 smithi112 ceph-mon[35384]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T00:54:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:18 smithi110 ceph-mon[34712]: purged_snaps scrub starts 2024-09-11T00:54:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:18 smithi110 ceph-mon[34712]: purged_snaps scrub ok 2024-09-11T00:54:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:18 smithi110 ceph-mon[34712]: osd.7 [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594] boot 2024-09-11T00:54:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:18 smithi110 ceph-mon[34712]: osdmap e47: 8 total, 8 up, 8 in 2024-09-11T00:54:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:18 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:54:18.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:18 smithi110 ceph-mon[34712]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:18.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:18 smithi110 ceph-mon[34712]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T00:54:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:18 smithi078 ceph-mon[31315]: purged_snaps scrub starts 2024-09-11T00:54:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:18 smithi078 ceph-mon[31315]: purged_snaps scrub ok 2024-09-11T00:54:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:18 smithi078 ceph-mon[31315]: osd.7 [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594] boot 2024-09-11T00:54:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:18 smithi078 ceph-mon[31315]: osdmap e47: 8 total, 8 up, 8 in 2024-09-11T00:54:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:18 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:54:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:18 smithi078 ceph-mon[31315]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:54:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:18 smithi078 ceph-mon[31315]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T00:54:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:20 smithi110 ceph-mon[34712]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T00:54:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:20 smithi110 ceph-mon[34712]: from='client.? 172.21.15.112:0/4151907659' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a3367da8-375a-4a46-a952-867d4b72ae2e"} : dispatch 2024-09-11T00:54:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:20 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a3367da8-375a-4a46-a952-867d4b72ae2e"} : dispatch 2024-09-11T00:54:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:20 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3367da8-375a-4a46-a952-867d4b72ae2e"}]': finished 2024-09-11T00:54:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:20 smithi110 ceph-mon[34712]: osdmap e50: 9 total, 8 up, 9 in 2024-09-11T00:54:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:20 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:20 smithi110 ceph-mon[34712]: pgmap v160: 1 pgs: 1 peering; 0 B data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:20 smithi110 ceph-mon[34712]: from='client.? 172.21.15.112:0/2670878133' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:54:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:20 smithi112 ceph-mon[35384]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T00:54:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:20 smithi112 ceph-mon[35384]: from='client.? 172.21.15.112:0/4151907659' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a3367da8-375a-4a46-a952-867d4b72ae2e"} : dispatch 2024-09-11T00:54:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:20 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a3367da8-375a-4a46-a952-867d4b72ae2e"} : dispatch 2024-09-11T00:54:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:20 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3367da8-375a-4a46-a952-867d4b72ae2e"}]': finished 2024-09-11T00:54:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:20 smithi112 ceph-mon[35384]: osdmap e50: 9 total, 8 up, 9 in 2024-09-11T00:54:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:20 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:20 smithi112 ceph-mon[35384]: pgmap v160: 1 pgs: 1 peering; 0 B data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:20.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:20 smithi112 ceph-mon[35384]: from='client.? 172.21.15.112:0/2670878133' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:54:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:20 smithi078 ceph-mon[31315]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T00:54:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:20 smithi078 ceph-mon[31315]: from='client.? 172.21.15.112:0/4151907659' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a3367da8-375a-4a46-a952-867d4b72ae2e"} : dispatch 2024-09-11T00:54:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:20 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a3367da8-375a-4a46-a952-867d4b72ae2e"} : dispatch 2024-09-11T00:54:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:20 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3367da8-375a-4a46-a952-867d4b72ae2e"}]': finished 2024-09-11T00:54:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:20 smithi078 ceph-mon[31315]: osdmap e50: 9 total, 8 up, 9 in 2024-09-11T00:54:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:20 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:20 smithi078 ceph-mon[31315]: pgmap v160: 1 pgs: 1 peering; 0 B data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:20 smithi078 ceph-mon[31315]: from='client.? 172.21.15.112:0/2670878133' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:54:23.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:23 smithi110 ceph-mon[34712]: pgmap v161: 1 pgs: 1 peering; 0 B data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:23.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:23 smithi112 ceph-mon[35384]: pgmap v161: 1 pgs: 1 peering; 0 B data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:23.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:23 smithi078 ceph-mon[31315]: pgmap v161: 1 pgs: 1 peering; 0 B data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:25 smithi110 ceph-mon[34712]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:25.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:25 smithi112 ceph-mon[35384]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:25.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:25 smithi078 ceph-mon[31315]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:27 smithi110 ceph-mon[34712]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:27 smithi112 ceph-mon[35384]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:27.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:27 smithi078 ceph-mon[31315]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:28.386 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:28 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-11T00:54:28.387 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:28 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:28 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-11T00:54:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:28 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:28 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-11T00:54:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:28 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:29.303 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:29 smithi112 ceph-mon[35384]: Deploying daemon osd.8 on smithi112 2024-09-11T00:54:29.303 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:29 smithi112 ceph-mon[35384]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:29.341 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:29 smithi110 ceph-mon[34712]: Deploying daemon osd.8 on smithi112 2024-09-11T00:54:29.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:29 smithi110 ceph-mon[34712]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:29.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:29 smithi078 ceph-mon[31315]: Deploying daemon osd.8 on smithi112 2024-09-11T00:54:29.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:29 smithi078 ceph-mon[31315]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:31.214 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:31 smithi112 ceph-mon[35384]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:31.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:31 smithi110 ceph-mon[34712]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:31.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:31 smithi078 ceph-mon[31315]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:33 smithi110 ceph-mon[34712]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:33 smithi112 ceph-mon[35384]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:33.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:33 smithi078 ceph-mon[31315]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:35.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:35 smithi112 ceph-mon[35384]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:35.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:35 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:54:35.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:35 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:35.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:35 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:35.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:35 smithi110 ceph-mon[34712]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:35.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:35 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:54:35.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:35 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:35.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:35 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:35.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:35 smithi078 ceph-mon[31315]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:35.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:35 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:54:35.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:35 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:35.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:35 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:37.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:37 smithi110 ceph-mon[34712]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:37.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:37 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:37.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:37 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:37.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:37 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:37.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:37 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:54:37.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:37 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:37 smithi112 ceph-mon[35384]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:37 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:37 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:37 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:37 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:54:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:37 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:37.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:37 smithi078 ceph-mon[31315]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:37.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:37 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:37.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:37 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:37.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:37 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:37.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:37 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:54:37.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:37 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:37.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.954+0000 7f92f0ff9640 1 -- 172.21.15.112:0/2525234908 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f92f4063590 con 0x7f92e00b5330 2024-09-11T00:54:37.956 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 8 on host 'smithi112' 2024-09-11T00:54:37.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.956+0000 7f92f93fe640 1 -- 172.21.15.112:0/2525234908 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f92e00b5330 msgr2=0x7f92e00b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:54:37.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.956+0000 7f92f93fe640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f92e00b5330 0x7f92e00b77f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0058c0 tx=0x7f92dc005850 comp rx=0 tx=0).stop 2024-09-11T00:54:37.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.956+0000 7f92f93fe640 1 -- 172.21.15.112:0/2525234908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4102dc0 msgr2=0x7f92f41992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:54:37.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.956+0000 7f92f93fe640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4102dc0 0x7f92f41992c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f92e4005940 tx=0x7f92e40042f0 comp rx=0 tx=0).stop 2024-09-11T00:54:37.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.956+0000 7f92f93fe640 1 -- 172.21.15.112:0/2525234908 shutdown_connections 2024-09-11T00:54:37.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.956+0000 7f92f93fe640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f92e00b5330 0x7f92e00b77f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:37.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.956+0000 7f92f93fe640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92f4109150 0x7f92f419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:37.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.956+0000 7f92f93fe640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f41087d0 0x7f92f4199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:37.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.956+0000 7f92f93fe640 1 --2- 172.21.15.112:0/2525234908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4102dc0 0x7f92f41992c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:37.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.956+0000 7f92f93fe640 1 -- 172.21.15.112:0/2525234908 >> 172.21.15.112:0/2525234908 conn(0x7f92f40fe830 msgr2=0x7f92f4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:54:37.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.956+0000 7f92f93fe640 1 -- 172.21.15.112:0/2525234908 shutdown_connections 2024-09-11T00:54:37.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:54:37.956+0000 7f92f93fe640 1 -- 172.21.15.112:0/2525234908 wait complete. 2024-09-11T00:54:38.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:38 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:54:38.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:38 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:38.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:38 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:38 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:54:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:38 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:38 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:38.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:38 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:54:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:38 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:38 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:38.578 DEBUG:teuthology.orchestra.run.smithi112:osd.8> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.8.service 2024-09-11T00:54:38.580 INFO:tasks.cephadm:Deploying osd.9 on smithi078 with /dev/vg_nvme/lv_1... 2024-09-11T00:54:38.580 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T00:54:38.829 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:54:39.204 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:39 smithi078 ceph-mon[31315]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:39 smithi110 ceph-mon[34712]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:39 smithi112 ceph-mon[35384]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:40.083 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 00:54:39 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T00:54:39.924+0000 7f2a105aa640 -1 osd.8 0 log_to_monitors true 2024-09-11T00:54:40.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:40 smithi078 ceph-mon[31315]: from='osd.8 [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:54:40.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:40 smithi078 ceph-mon[31315]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:54:40.353 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:40 smithi112 ceph-mon[35384]: from='osd.8 [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:54:40.353 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:40 smithi112 ceph-mon[35384]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:54:40.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:40 smithi110 ceph-mon[34712]: from='osd.8 [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:54:40.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:40 smithi110 ceph-mon[34712]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:54:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:41 smithi110 ceph-mon[34712]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:41 smithi110 ceph-mon[34712]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T00:54:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:41 smithi110 ceph-mon[34712]: from='osd.8 [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:54:41.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:41 smithi110 ceph-mon[34712]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T00:54:41.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:41 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:41.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:41 smithi110 ceph-mon[34712]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:54:41.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:41 smithi112 ceph-mon[35384]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:41.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:41 smithi112 ceph-mon[35384]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T00:54:41.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:41 smithi112 ceph-mon[35384]: from='osd.8 [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:54:41.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:41 smithi112 ceph-mon[35384]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T00:54:41.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:41 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:41.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:41 smithi112 ceph-mon[35384]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:54:41.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:41 smithi078 ceph-mon[31315]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:41.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:41 smithi078 ceph-mon[31315]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T00:54:41.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:41 smithi078 ceph-mon[31315]: from='osd.8 [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:54:41.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:41 smithi078 ceph-mon[31315]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T00:54:41.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:41 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:41.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:41 smithi078 ceph-mon[31315]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:54:41.633 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:54:41.659 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_1 2024-09-11T00:54:41.904 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:54:41.973 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 00:54:41 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T00:54:41.500+0000 7f2a0c52d640 -1 osd.8 0 waiting for initial osdmap 2024-09-11T00:54:41.974 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 00:54:41 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T00:54:41.511+0000 7f2a0434c640 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:54:42.301 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-11T00:54:42.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T00:54:42.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:42.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:42.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='osd.8 ' entity='osd.8' 2024-09-11T00:54:42.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: Detected new or changed devices on smithi112 2024-09-11T00:54:42.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:42.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:42.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:42.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:42.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:42.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi112 to 5696M 2024-09-11T00:54:42.302 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:42.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:42.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:54:42.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:42.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:42 smithi078 ceph-mon[31315]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:42.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.383+0000 7f638ba66640 1 -- 172.21.15.78:0/2171866819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6384075e60 msgr2=0x7f63841135e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:54:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.383+0000 7f638ba66640 1 --2- 172.21.15.78:0/2171866819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6384075e60 0x7f63841135e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6374009900 tx=0x7f637402a410 comp rx=0 tx=0).stop 2024-09-11T00:54:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.383+0000 7f638ba66640 1 -- 172.21.15.78:0/2171866819 shutdown_connections 2024-09-11T00:54:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.383+0000 7f638ba66640 1 --2- 172.21.15.78:0/2171866819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6384113b20 0x7f6384115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.383+0000 7f638ba66640 1 --2- 172.21.15.78:0/2171866819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6384075e60 0x7f63841135e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.383+0000 7f638ba66640 1 --2- 172.21.15.78:0/2171866819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6384075540 0x7f6384075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.383+0000 7f638ba66640 1 -- 172.21.15.78:0/2171866819 >> 172.21.15.78:0/2171866819 conn(0x7f63840fe830 msgr2=0x7f6384100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:54:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.383+0000 7f638ba66640 1 -- 172.21.15.78:0/2171866819 shutdown_connections 2024-09-11T00:54:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.383+0000 7f638ba66640 1 -- 172.21.15.78:0/2171866819 wait complete. 2024-09-11T00:54:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.383+0000 7f638ba66640 1 Processor -- start 2024-09-11T00:54:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.383+0000 7f638ba66640 1 -- start start 2024-09-11T00:54:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f638ba66640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6384075540 0x7f638419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:54:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f638ba66640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6384075e60 0x7f638419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:54:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f638ba66640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6384113b20 0x7f63841a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:54:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f638ba66640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63841193f0 con 0x7f6384075540 2024-09-11T00:54:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f638ba66640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6384119270 con 0x7f6384113b20 2024-09-11T00:54:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f638ba66640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6384119570 con 0x7f6384075e60 2024-09-11T00:54:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f6389802640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6384075540 0x7f638419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:54:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f6389802640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6384075540 0x7f638419f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53150/0 (socket says 172.21.15.78:53150) 2024-09-11T00:54:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f6389802640 1 -- 172.21.15.78:0/576343707 learned_addr learned my addr 172.21.15.78:0/576343707 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:54:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f6389001640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6384075e60 0x7f638419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:54:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f638a003640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6384113b20 0x7f63841a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:54:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f6389802640 1 -- 172.21.15.78:0/576343707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6384075e60 msgr2=0x7f638419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:54:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f6389802640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6384075e60 0x7f638419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f6389802640 1 -- 172.21.15.78:0/576343707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6384113b20 msgr2=0x7f63841a51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:54:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f6389802640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6384113b20 0x7f63841a51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:54:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f6389802640 1 -- 172.21.15.78:0/576343707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63841a59b0 con 0x7f6384075540 2024-09-11T00:54:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f638a003640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6384113b20 0x7f63841a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:54:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f6389802640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6384075540 0x7f638419f900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f638000d950 tx=0x7f638000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:54:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f637affd640 1 -- 172.21.15.78:0/576343707 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380017070 con 0x7f6384075540 2024-09-11T00:54:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f637affd640 1 -- 172.21.15.78:0/576343707 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6380009c30 con 0x7f6384075540 2024-09-11T00:54:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f638ba66640 1 -- 172.21.15.78:0/576343707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63841a5ca0 con 0x7f6384075540 2024-09-11T00:54:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f637affd640 1 -- 172.21.15.78:0/576343707 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638000f620 con 0x7f6384075540 2024-09-11T00:54:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.384+0000 7f638ba66640 1 -- 172.21.15.78:0/576343707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63841ac360 con 0x7f6384075540 2024-09-11T00:54:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.385+0000 7f638ba66640 1 -- 172.21.15.78:0/576343707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6384075920 con 0x7f6384075540 2024-09-11T00:54:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.387+0000 7f637affd640 1 -- 172.21.15.78:0/576343707 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f6380005030 con 0x7f6384075540 2024-09-11T00:54:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.388+0000 7f637affd640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63700b5330 0x7f63700b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:54:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.388+0000 7f6389001640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63700b5330 0x7f63700b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:54:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.388+0000 7f637affd640 1 -- 172.21.15.78:0/576343707 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5764+0+0 (secure 0 0 0) 0x7f63800e4f70 con 0x7f6384075540 2024-09-11T00:54:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.388+0000 7f6389001640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63700b5330 0x7f63700b77f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6374000c00 tx=0x7f6374031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:54:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.391+0000 7f637affd640 1 -- 172.21.15.78:0/576343707 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f63800ac780 con 0x7f6384075540 2024-09-11T00:54:42.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-11T00:54:42.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T00:54:42.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:42.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:42.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='osd.8 ' entity='osd.8' 2024-09-11T00:54:42.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: Detected new or changed devices on smithi112 2024-09-11T00:54:42.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:42.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:42.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:42.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:42.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:42.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi112 to 5696M 2024-09-11T00:54:42.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:42.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:42.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:54:42.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:42.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:42 smithi110 ceph-mon[34712]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-11T00:54:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T00:54:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='osd.8 ' entity='osd.8' 2024-09-11T00:54:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: Detected new or changed devices on smithi112 2024-09-11T00:54:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:42.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:54:42.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi112 to 5696M 2024-09-11T00:54:42.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:42.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:42.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:54:42.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:54:42.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:42 smithi112 ceph-mon[35384]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:54:42.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:54:42.580+0000 7f638ba66640 1 -- 172.21.15.78:0/576343707 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6384100b50 con 0x7f63700b5330 2024-09-11T00:54:43.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:43 smithi078 ceph-mon[31315]: purged_snaps scrub starts 2024-09-11T00:54:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:43 smithi078 ceph-mon[31315]: purged_snaps scrub ok 2024-09-11T00:54:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:43 smithi078 ceph-mon[31315]: osd.8 [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321] boot 2024-09-11T00:54:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:43 smithi078 ceph-mon[31315]: osdmap e53: 9 total, 9 up, 9 in 2024-09-11T00:54:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:43 smithi078 ceph-mon[31315]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:54:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:54:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:54:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:43.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:43 smithi110 ceph-mon[34712]: purged_snaps scrub starts 2024-09-11T00:54:43.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:43 smithi110 ceph-mon[34712]: purged_snaps scrub ok 2024-09-11T00:54:43.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:43.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:43 smithi110 ceph-mon[34712]: osd.8 [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321] boot 2024-09-11T00:54:43.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:43 smithi110 ceph-mon[34712]: osdmap e53: 9 total, 9 up, 9 in 2024-09-11T00:54:43.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:43.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:43 smithi110 ceph-mon[34712]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:54:43.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:54:43.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:54:43.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:43 smithi112 ceph-mon[35384]: purged_snaps scrub starts 2024-09-11T00:54:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:43 smithi112 ceph-mon[35384]: purged_snaps scrub ok 2024-09-11T00:54:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:43 smithi112 ceph-mon[35384]: osd.8 [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321] boot 2024-09-11T00:54:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:43 smithi112 ceph-mon[35384]: osdmap e53: 9 total, 9 up, 9 in 2024-09-11T00:54:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:54:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:43 smithi112 ceph-mon[35384]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:54:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:54:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:54:43.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:44.310 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:44 smithi078 ceph-mon[31315]: osdmap e54: 9 total, 9 up, 9 in 2024-09-11T00:54:44.311 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:44 smithi078 ceph-mon[31315]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:54:44.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:44 smithi112 ceph-mon[35384]: osdmap e54: 9 total, 9 up, 9 in 2024-09-11T00:54:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:44 smithi112 ceph-mon[35384]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:54:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:44 smithi110 ceph-mon[34712]: osdmap e54: 9 total, 9 up, 9 in 2024-09-11T00:54:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:44 smithi110 ceph-mon[34712]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:54:45.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:45 smithi112 ceph-mon[35384]: osdmap e55: 9 total, 9 up, 9 in 2024-09-11T00:54:45.475 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:45 smithi078 ceph-mon[31315]: osdmap e55: 9 total, 9 up, 9 in 2024-09-11T00:54:45.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:45 smithi110 ceph-mon[34712]: osdmap e55: 9 total, 9 up, 9 in 2024-09-11T00:54:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:46 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3978427468' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d1825764-cefc-495b-99da-8ba429df0937"} : dispatch 2024-09-11T00:54:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:46 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3978427468' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1825764-cefc-495b-99da-8ba429df0937"}]': finished 2024-09-11T00:54:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:46 smithi112 ceph-mon[35384]: osdmap e56: 10 total, 9 up, 10 in 2024-09-11T00:54:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:46 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:54:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:46 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/63595247' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:54:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:46 smithi112 ceph-mon[35384]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:54:46.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:46 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3978427468' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d1825764-cefc-495b-99da-8ba429df0937"} : dispatch 2024-09-11T00:54:46.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:46 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3978427468' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1825764-cefc-495b-99da-8ba429df0937"}]': finished 2024-09-11T00:54:46.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:46 smithi078 ceph-mon[31315]: osdmap e56: 10 total, 9 up, 10 in 2024-09-11T00:54:46.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:46 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:54:46.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:46 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/63595247' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:54:46.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:46 smithi078 ceph-mon[31315]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:54:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:46 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3978427468' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d1825764-cefc-495b-99da-8ba429df0937"} : dispatch 2024-09-11T00:54:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:46 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3978427468' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1825764-cefc-495b-99da-8ba429df0937"}]': finished 2024-09-11T00:54:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:46 smithi110 ceph-mon[34712]: osdmap e56: 10 total, 9 up, 10 in 2024-09-11T00:54:46.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:46 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:54:46.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:46 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/63595247' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:54:46.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:46 smithi110 ceph-mon[34712]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:54:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:49 smithi110 ceph-mon[34712]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:54:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:49 smithi112 ceph-mon[35384]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:54:49.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:49 smithi078 ceph-mon[31315]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:54:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:51 smithi110 ceph-mon[34712]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-11T00:54:51.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:51 smithi112 ceph-mon[35384]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-11T00:54:51.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:51 smithi078 ceph-mon[31315]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-11T00:54:53.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:53 smithi078 ceph-mon[31315]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:54:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:53 smithi110 ceph-mon[34712]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:54:53.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:53 smithi112 ceph-mon[35384]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:54:55.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:55 smithi078 ceph-mon[31315]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T00:54:55.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:55 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-11T00:54:55.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:55 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:55 smithi110 ceph-mon[34712]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T00:54:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:55 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-11T00:54:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:55 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:55 smithi112 ceph-mon[35384]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T00:54:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:55 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-11T00:54:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:55 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:54:56.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:56 smithi110 ceph-mon[34712]: Deploying daemon osd.9 on smithi078 2024-09-11T00:54:56.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:56 smithi078 ceph-mon[31315]: Deploying daemon osd.9 on smithi078 2024-09-11T00:54:56.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:56 smithi112 ceph-mon[35384]: Deploying daemon osd.9 on smithi078 2024-09-11T00:54:57.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:57 smithi110 ceph-mon[34712]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-11T00:54:57.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:57 smithi112 ceph-mon[35384]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-11T00:54:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:57 smithi078 ceph-mon[31315]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-11T00:54:59.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:54:59 smithi078 ceph-mon[31315]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:54:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:54:59 smithi110 ceph-mon[34712]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:54:59.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:54:59 smithi112 ceph-mon[35384]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:55:00.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:00.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:00.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:00.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:00.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:01 smithi110 ceph-mon[34712]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:55:01.456 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:01 smithi078 ceph-mon[31315]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:55:01.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:01 smithi112 ceph-mon[35384]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:55:03.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.010+0000 7f637affd640 1 -- 172.21.15.78:0/576343707 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6384100b50 con 0x7f63700b5330 2024-09-11T00:55:03.012 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 9 on host 'smithi078' 2024-09-11T00:55:03.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.012+0000 7f638ba66640 1 -- 172.21.15.78:0/576343707 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63700b5330 msgr2=0x7f63700b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.012+0000 7f638ba66640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63700b5330 0x7f63700b77f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6374000c00 tx=0x7f6374031400 comp rx=0 tx=0).stop 2024-09-11T00:55:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.012+0000 7f638ba66640 1 -- 172.21.15.78:0/576343707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6384075540 msgr2=0x7f638419f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.012+0000 7f638ba66640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6384075540 0x7f638419f900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f638000d950 tx=0x7f638000de10 comp rx=0 tx=0).stop 2024-09-11T00:55:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.012+0000 7f638ba66640 1 -- 172.21.15.78:0/576343707 shutdown_connections 2024-09-11T00:55:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.012+0000 7f638ba66640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63700b5330 0x7f63700b77f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.012+0000 7f638ba66640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6384113b20 0x7f63841a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.012+0000 7f638ba66640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6384075e60 0x7f638419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.012+0000 7f638ba66640 1 --2- 172.21.15.78:0/576343707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6384075540 0x7f638419f900 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.012+0000 7f638ba66640 1 -- 172.21.15.78:0/576343707 >> 172.21.15.78:0/576343707 conn(0x7f63840fe830 msgr2=0x7f63840ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:55:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.012+0000 7f638ba66640 1 -- 172.21.15.78:0/576343707 shutdown_connections 2024-09-11T00:55:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:03.012+0000 7f638ba66640 1 -- 172.21.15.78:0/576343707 wait complete. 2024-09-11T00:55:03.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:03 smithi078 ceph-mon[31315]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:55:03.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:03.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:03 smithi110 ceph-mon[34712]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:55:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:03.478 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:03 smithi112 ceph-mon[35384]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:55:03.478 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:03.478 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:03.985 DEBUG:teuthology.orchestra.run.smithi078:osd.9> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.9.service 2024-09-11T00:55:03.988 INFO:tasks.cephadm:Deploying osd.10 on smithi110 with /dev/vg_nvme/lv_1... 2024-09-11T00:55:03.988 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T00:55:04.250 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:55:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:05 smithi078 ceph-mon[31315]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:55:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:05 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:05 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:05 smithi110 ceph-mon[34712]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:55:05.677 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:05 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:05.677 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:05 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:05.678 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:05 smithi112 ceph-mon[35384]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:55:05.678 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:05 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:05.678 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:05 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:06.086 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 00:55:05 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T00:55:05.829+0000 7f6878748640 -1 osd.9 0 log_to_monitors true 2024-09-11T00:55:06.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:06 smithi110 ceph-mon[34712]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:55:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:06 smithi112 ceph-mon[35384]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:55:06.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:06 smithi078 ceph-mon[31315]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:55:07.026 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T00:55:07.052 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_1 2024-09-11T00:55:07.307 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:55:07.427 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:07 smithi110 ceph-mon[34712]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:55:07.427 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:07 smithi110 ceph-mon[34712]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T00:55:07.428 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:07 smithi110 ceph-mon[34712]: osdmap e57: 10 total, 9 up, 10 in 2024-09-11T00:55:07.428 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:07 smithi110 ceph-mon[34712]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:55:07.428 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:07.456 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:07 smithi078 ceph-mon[31315]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:55:07.456 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:07 smithi078 ceph-mon[31315]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T00:55:07.456 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:07 smithi078 ceph-mon[31315]: osdmap e57: 10 total, 9 up, 10 in 2024-09-11T00:55:07.457 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:07 smithi078 ceph-mon[31315]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:55:07.457 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:07 smithi112 ceph-mon[35384]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:55:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:07 smithi112 ceph-mon[35384]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T00:55:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:07 smithi112 ceph-mon[35384]: osdmap e57: 10 total, 9 up, 10 in 2024-09-11T00:55:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:07 smithi112 ceph-mon[35384]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]} : dispatch 2024-09-11T00:55:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:07.746 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 00:55:07 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T00:55:07.503+0000 7f68746cb640 -1 osd.9 0 waiting for initial osdmap 2024-09-11T00:55:07.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 00:55:07 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T00:55:07.514+0000 7f686c4ea640 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:55:07.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.851+0000 7f62843b1640 1 -- 172.21.15.110:0/1512657499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f627c107710 msgr2=0x7f627c107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:07.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.851+0000 7f62843b1640 1 --2- 172.21.15.110:0/1512657499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f627c107710 0x7f627c107b50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6278009900 tx=0x7f627802a410 comp rx=0 tx=0).stop 2024-09-11T00:55:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.851+0000 7f62843b1640 1 -- 172.21.15.110:0/1512657499 shutdown_connections 2024-09-11T00:55:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.851+0000 7f62843b1640 1 --2- 172.21.15.110:0/1512657499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c108090 0x7f627c10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.851+0000 7f62843b1640 1 --2- 172.21.15.110:0/1512657499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f627c107710 0x7f627c107b50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.851+0000 7f62843b1640 1 --2- 172.21.15.110:0/1512657499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c102920 0x7f627c102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.851+0000 7f62843b1640 1 -- 172.21.15.110:0/1512657499 >> 172.21.15.110:0/1512657499 conn(0x7f627c076f20 msgr2=0x7f627c077330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:55:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.851+0000 7f62843b1640 1 -- 172.21.15.110:0/1512657499 shutdown_connections 2024-09-11T00:55:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.851+0000 7f62843b1640 1 -- 172.21.15.110:0/1512657499 wait complete. 2024-09-11T00:55:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f62843b1640 1 Processor -- start 2024-09-11T00:55:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f62843b1640 1 -- start start 2024-09-11T00:55:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f62843b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c102920 0x7f627c199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f62843b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c107710 0x7f627c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f62843b1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f627c108090 0x7f627c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f62843b1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f627c112e70 con 0x7f627c107710 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f62843b1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f627c112cf0 con 0x7f627c108090 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f62843b1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f627c112ff0 con 0x7f627c102920 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f628294e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f627c108090 0x7f627c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f628294e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f627c108090 0x7f627c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56530/0 (socket says 172.21.15.110:56530) 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f628294e640 1 -- 172.21.15.110:0/3512879920 learned_addr learned my addr 172.21.15.110:0/3512879920 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f628214d640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c102920 0x7f627c199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f628194c640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c107710 0x7f627c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f628294e640 1 -- 172.21.15.110:0/3512879920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c102920 msgr2=0x7f627c199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f628294e640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c102920 0x7f627c199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f628294e640 1 -- 172.21.15.110:0/3512879920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c107710 msgr2=0x7f627c1999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f628294e640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c107710 0x7f627c1999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f628294e640 1 -- 172.21.15.110:0/3512879920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627c19f530 con 0x7f627c108090 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.852+0000 7f628294e640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f627c108090 0x7f627c19ed60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f627400b970 tx=0x7f627400bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.853+0000 7f626f7fe640 1 -- 172.21.15.110:0/3512879920 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274019070 con 0x7f627c108090 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.853+0000 7f626f7fe640 1 -- 172.21.15.110:0/3512879920 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6274014440 con 0x7f627c108090 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.853+0000 7f62843b1640 1 -- 172.21.15.110:0/3512879920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f627c19f820 con 0x7f627c108090 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.853+0000 7f626f7fe640 1 -- 172.21.15.110:0/3512879920 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62740075d0 con 0x7f627c108090 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.853+0000 7f62843b1640 1 -- 172.21.15.110:0/3512879920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f627c1a5ee0 con 0x7f627c108090 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.854+0000 7f62843b1640 1 -- 172.21.15.110:0/3512879920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f627c102d80 con 0x7f627c108090 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.855+0000 7f626f7fe640 1 -- 172.21.15.110:0/3512879920 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f62740145f0 con 0x7f627c108090 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.855+0000 7f626f7fe640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62680b5330 0x7f62680b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.856+0000 7f626f7fe640 1 -- 172.21.15.110:0/3512879920 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6196+0+0 (secure 0 0 0) 0x7f62740e4f90 con 0x7f627c108090 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.856+0000 7f628214d640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62680b5330 0x7f62680b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.856+0000 7f628214d640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62680b5330 0x7f62680b77f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6270009990 tx=0x7f6270009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:55:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:07.858+0000 7f626f7fe640 1 -- 172.21.15.110:0/3512879920 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f62740ac040 con 0x7f627c108090 2024-09-11T00:55:08.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:08.044+0000 7f62843b1640 1 -- 172.21.15.110:0/3512879920 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f627c063590 con 0x7f62680b5330 2024-09-11T00:55:08.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-11T00:55:08.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: osdmap e58: 10 total, 9 up, 10 in 2024-09-11T00:55:08.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:08.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:08.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: Detected new or changed devices on smithi078 2024-09-11T00:55:08.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:08.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:08.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi078 to 4272M 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:55:08.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:55:08.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:55:08.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-11T00:55:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: osdmap e58: 10 total, 9 up, 10 in 2024-09-11T00:55:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: Detected new or changed devices on smithi078 2024-09-11T00:55:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi078 to 4272M 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:08.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:55:08.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:55:08.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:55:08.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:55:08.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-11T00:55:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: osdmap e58: 10 total, 9 up, 10 in 2024-09-11T00:55:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: Detected new or changed devices on smithi078 2024-09-11T00:55:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi078 to 4272M 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350]' entity='osd.9' 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:08.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:08.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:55:08.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:55:08.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:55:08.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:55:08.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:09 smithi078 ceph-mon[31315]: purged_snaps scrub starts 2024-09-11T00:55:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:09 smithi078 ceph-mon[31315]: purged_snaps scrub ok 2024-09-11T00:55:09.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:09.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:09 smithi078 ceph-mon[31315]: osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350] boot 2024-09-11T00:55:09.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:09 smithi078 ceph-mon[31315]: osdmap e59: 10 total, 10 up, 10 in 2024-09-11T00:55:09.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:09 smithi110 ceph-mon[34712]: purged_snaps scrub starts 2024-09-11T00:55:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:09 smithi110 ceph-mon[34712]: purged_snaps scrub ok 2024-09-11T00:55:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:09 smithi110 ceph-mon[34712]: osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350] boot 2024-09-11T00:55:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:09 smithi110 ceph-mon[34712]: osdmap e59: 10 total, 10 up, 10 in 2024-09-11T00:55:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:09.488 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:09 smithi112 ceph-mon[35384]: purged_snaps scrub starts 2024-09-11T00:55:09.488 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:09 smithi112 ceph-mon[35384]: purged_snaps scrub ok 2024-09-11T00:55:09.488 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:09.488 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:09 smithi112 ceph-mon[35384]: osd.9 [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350] boot 2024-09-11T00:55:09.488 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:09 smithi112 ceph-mon[35384]: osdmap e59: 10 total, 10 up, 10 in 2024-09-11T00:55:09.488 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:55:10.914 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:10 smithi110 ceph-mon[34712]: osdmap e60: 10 total, 10 up, 10 in 2024-09-11T00:55:10.914 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:10 smithi110 ceph-mon[34712]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:10.914 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:10 smithi110 ceph-mon[34712]: from='client.? 172.21.15.110:0/1399043928' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e1cfff7a-8916-4845-a482-d79fa275da30"} : dispatch 2024-09-11T00:55:10.914 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:10 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e1cfff7a-8916-4845-a482-d79fa275da30"} : dispatch 2024-09-11T00:55:10.915 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:10 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1cfff7a-8916-4845-a482-d79fa275da30"}]': finished 2024-09-11T00:55:10.915 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:10 smithi110 ceph-mon[34712]: osdmap e61: 11 total, 10 up, 11 in 2024-09-11T00:55:10.915 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:10.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:10 smithi112 ceph-mon[35384]: osdmap e60: 10 total, 10 up, 10 in 2024-09-11T00:55:10.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:10 smithi112 ceph-mon[35384]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:10.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:10 smithi112 ceph-mon[35384]: from='client.? 172.21.15.110:0/1399043928' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e1cfff7a-8916-4845-a482-d79fa275da30"} : dispatch 2024-09-11T00:55:10.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:10 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e1cfff7a-8916-4845-a482-d79fa275da30"} : dispatch 2024-09-11T00:55:10.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:10 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1cfff7a-8916-4845-a482-d79fa275da30"}]': finished 2024-09-11T00:55:10.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:10 smithi112 ceph-mon[35384]: osdmap e61: 11 total, 10 up, 11 in 2024-09-11T00:55:10.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:10.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:10 smithi078 ceph-mon[31315]: osdmap e60: 10 total, 10 up, 10 in 2024-09-11T00:55:10.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:10 smithi078 ceph-mon[31315]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:10.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:10 smithi078 ceph-mon[31315]: from='client.? 172.21.15.110:0/1399043928' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e1cfff7a-8916-4845-a482-d79fa275da30"} : dispatch 2024-09-11T00:55:10.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:10 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e1cfff7a-8916-4845-a482-d79fa275da30"} : dispatch 2024-09-11T00:55:10.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:10 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1cfff7a-8916-4845-a482-d79fa275da30"}]': finished 2024-09-11T00:55:10.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:10 smithi078 ceph-mon[31315]: osdmap e61: 11 total, 10 up, 11 in 2024-09-11T00:55:10.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:11.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:11 smithi110 ceph-mon[34712]: from='client.? 172.21.15.110:0/261517885' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:55:11.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:11 smithi112 ceph-mon[35384]: from='client.? 172.21.15.110:0/261517885' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:55:11.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:11 smithi078 ceph-mon[31315]: from='client.? 172.21.15.110:0/261517885' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:55:12.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:12 smithi110 ceph-mon[34712]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:12.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:12 smithi112 ceph-mon[35384]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:12.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:12 smithi078 ceph-mon[31315]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:15.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:15 smithi110 ceph-mon[34712]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:15.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:15 smithi112 ceph-mon[35384]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:15.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:15 smithi078 ceph-mon[31315]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:17.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:17 smithi110 ceph-mon[34712]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:17.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:17 smithi112 ceph-mon[35384]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:17.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:17 smithi078 ceph-mon[31315]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:19.394 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:19 smithi078 ceph-mon[31315]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:19.394 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-11T00:55:19.394 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:19 smithi110 ceph-mon[34712]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-11T00:55:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:19 smithi112 ceph-mon[35384]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-11T00:55:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:20 smithi078 ceph-mon[31315]: Deploying daemon osd.10 on smithi110 2024-09-11T00:55:20.428 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:20 smithi110 ceph-mon[34712]: Deploying daemon osd.10 on smithi110 2024-09-11T00:55:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:20 smithi112 ceph-mon[35384]: Deploying daemon osd.10 on smithi110 2024-09-11T00:55:21.425 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:21 smithi110 ceph-mon[34712]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:21.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:21 smithi112 ceph-mon[35384]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:21.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:21 smithi078 ceph-mon[31315]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:23.365 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:23 smithi110 ceph-mon[34712]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:23 smithi112 ceph-mon[35384]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:23 smithi078 ceph-mon[31315]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:25 smithi110 ceph-mon[34712]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:25.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:25 smithi112 ceph-mon[35384]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:25.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:25 smithi078 ceph-mon[31315]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:26.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:26.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:26.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:27.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:27 smithi110 ceph-mon[34712]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:27 smithi112 ceph-mon[35384]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:27.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:27 smithi078 ceph-mon[31315]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:28.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.201+0000 7f626f7fe640 1 -- 172.21.15.110:0/3512879920 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f627c063590 con 0x7f62680b5330 2024-09-11T00:55:28.203 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 10 on host 'smithi110' 2024-09-11T00:55:28.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.203+0000 7f62843b1640 1 -- 172.21.15.110:0/3512879920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62680b5330 msgr2=0x7f62680b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:28.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.203+0000 7f62843b1640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62680b5330 0x7f62680b77f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6270009990 tx=0x7f6270009110 comp rx=0 tx=0).stop 2024-09-11T00:55:28.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.203+0000 7f62843b1640 1 -- 172.21.15.110:0/3512879920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f627c108090 msgr2=0x7f627c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:28.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.203+0000 7f62843b1640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f627c108090 0x7f627c19ed60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f627400b970 tx=0x7f627400bd70 comp rx=0 tx=0).stop 2024-09-11T00:55:28.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.203+0000 7f62843b1640 1 -- 172.21.15.110:0/3512879920 shutdown_connections 2024-09-11T00:55:28.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.203+0000 7f62843b1640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62680b5330 0x7f62680b77f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:28.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.203+0000 7f62843b1640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f627c108090 0x7f627c19ed60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:28.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.203+0000 7f62843b1640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c107710 0x7f627c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:28.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.203+0000 7f62843b1640 1 --2- 172.21.15.110:0/3512879920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f627c102920 0x7f627c199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:28.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.203+0000 7f62843b1640 1 -- 172.21.15.110:0/3512879920 >> 172.21.15.110:0/3512879920 conn(0x7f627c076f20 msgr2=0x7f627c100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:55:28.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.203+0000 7f62843b1640 1 -- 172.21.15.110:0/3512879920 shutdown_connections 2024-09-11T00:55:28.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-11T00:55:28.203+0000 7f62843b1640 1 -- 172.21.15.110:0/3512879920 wait complete. 2024-09-11T00:55:29.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:29 smithi110 ceph-mon[34712]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:29.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:29 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:29.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:29 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:29.227 DEBUG:teuthology.orchestra.run.smithi110:osd.10> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.10.service 2024-09-11T00:55:29.229 INFO:tasks.cephadm:Deploying osd.11 on smithi112 with /dev/vg_nvme/lv_1... 2024-09-11T00:55:29.229 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T00:55:29.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:29 smithi112 ceph-mon[35384]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:29 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:29 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:29.486 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:55:29.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:29 smithi078 ceph-mon[31315]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:29.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:29 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:29.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:29 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:30.380 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:30.380 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:30.380 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:30 smithi110 ceph-mon[34712]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:30.626 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:30.626 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:30.626 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:30 smithi112 ceph-mon[35384]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:30.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:30.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:30.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:30 smithi078 ceph-mon[31315]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:31.510 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:31 smithi112 ceph-mon[35384]: from='osd.10 [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:55:31.510 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:31 smithi112 ceph-mon[35384]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:55:31.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:31 smithi110 ceph-mon[34712]: from='osd.10 [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:55:31.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:31 smithi110 ceph-mon[34712]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:55:31.691 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 00:55:31 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T00:55:31.397+0000 7fc6f22a5640 -1 osd.10 0 log_to_monitors true 2024-09-11T00:55:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:31 smithi078 ceph-mon[31315]: from='osd.10 [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:55:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:31 smithi078 ceph-mon[31315]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:55:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-11T00:55:32.369 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_1 2024-09-11T00:55:32.619 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:55:32.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:32 smithi112 ceph-mon[35384]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T00:55:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:32 smithi112 ceph-mon[35384]: from='osd.10 [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:55:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:32 smithi112 ceph-mon[35384]: osdmap e62: 11 total, 10 up, 11 in 2024-09-11T00:55:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:32 smithi112 ceph-mon[35384]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:55:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:32 smithi112 ceph-mon[35384]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:32.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:32 smithi078 ceph-mon[31315]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T00:55:32.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:32 smithi078 ceph-mon[31315]: from='osd.10 [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:55:32.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:32 smithi078 ceph-mon[31315]: osdmap e62: 11 total, 10 up, 11 in 2024-09-11T00:55:32.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:32.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:32 smithi078 ceph-mon[31315]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:55:32.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:32 smithi078 ceph-mon[31315]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:32.877 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 00:55:32 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T00:55:32.470+0000 7fc6ee228640 -1 osd.10 0 waiting for initial osdmap 2024-09-11T00:55:32.878 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 00:55:32 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T00:55:32.480+0000 7fc6ea04f640 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:55:32.878 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:32 smithi110 ceph-mon[34712]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T00:55:32.878 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:32 smithi110 ceph-mon[34712]: from='osd.10 [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:55:32.878 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:32 smithi110 ceph-mon[34712]: osdmap e62: 11 total, 10 up, 11 in 2024-09-11T00:55:32.878 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:32.878 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:32 smithi110 ceph-mon[34712]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-11T00:55:32.878 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:32 smithi110 ceph-mon[34712]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:33.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.113+0000 7ffb588bf640 1 -- 172.21.15.112:0/2887929748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb54108020 msgr2=0x7ffb5410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:33.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.113+0000 7ffb588bf640 1 --2- 172.21.15.112:0/2887929748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb54108020 0x7ffb5410e820 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009ca0 tx=0x7ffb4c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:55:33.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.113+0000 7ffb588bf640 1 -- 172.21.15.112:0/2887929748 shutdown_connections 2024-09-11T00:55:33.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.113+0000 7ffb588bf640 1 --2- 172.21.15.112:0/2887929748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb54108020 0x7ffb5410e820 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.113+0000 7ffb588bf640 1 --2- 172.21.15.112:0/2887929748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb541076a0 0x7ffb54107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.113+0000 7ffb588bf640 1 --2- 172.21.15.112:0/2887929748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb54101c90 0x7ffb54102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.113+0000 7ffb588bf640 1 -- 172.21.15.112:0/2887929748 >> 172.21.15.112:0/2887929748 conn(0x7ffb540fd700 msgr2=0x7ffb540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.113+0000 7ffb588bf640 1 -- 172.21.15.112:0/2887929748 shutdown_connections 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.113+0000 7ffb588bf640 1 -- 172.21.15.112:0/2887929748 wait complete. 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb588bf640 1 Processor -- start 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb588bf640 1 -- start start 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb588bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb54101c90 0x7ffb54199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb588bf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb541076a0 0x7ffb541997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb588bf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb54108020 0x7ffb5419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb588bf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffb54111ca0 con 0x7ffb54101c90 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb588bf640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffb54111b20 con 0x7ffb54108020 2024-09-11T00:55:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb588bf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb54111e20 con 0x7ffb541076a0 2024-09-11T00:55:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb541076a0 0x7ffb541997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb541076a0 0x7ffb541997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44308/0 (socket says 172.21.15.112:44308) 2024-09-11T00:55:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb4bfff640 1 -- 172.21.15.112:0/1389201275 learned_addr learned my addr 172.21.15.112:0/1389201275 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-11T00:55:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb5259c640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb54101c90 0x7ffb54199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb52d9d640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb54108020 0x7ffb5419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb4bfff640 1 -- 172.21.15.112:0/1389201275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb54108020 msgr2=0x7ffb5419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb4bfff640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb54108020 0x7ffb5419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb4bfff640 1 -- 172.21.15.112:0/1389201275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb54101c90 msgr2=0x7ffb54199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb4bfff640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb54101c90 0x7ffb54199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb4bfff640 1 -- 172.21.15.112:0/1389201275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5419f320 con 0x7ffb541076a0 2024-09-11T00:55:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.114+0000 7ffb52d9d640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb54108020 0x7ffb5419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:55:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.115+0000 7ffb4bfff640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb541076a0 0x7ffb541997b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00e9f0 tx=0x7ffb3c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:55:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.115+0000 7ffb4b7fe640 1 -- 172.21.15.112:0/1389201275 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00cd80 con 0x7ffb541076a0 2024-09-11T00:55:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.115+0000 7ffb4b7fe640 1 -- 172.21.15.112:0/1389201275 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb3c0054d0 con 0x7ffb541076a0 2024-09-11T00:55:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.115+0000 7ffb4b7fe640 1 -- 172.21.15.112:0/1389201275 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c010690 con 0x7ffb541076a0 2024-09-11T00:55:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.115+0000 7ffb588bf640 1 -- 172.21.15.112:0/1389201275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5419f610 con 0x7ffb541076a0 2024-09-11T00:55:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.115+0000 7ffb588bf640 1 -- 172.21.15.112:0/1389201275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb541a5cd0 con 0x7ffb541076a0 2024-09-11T00:55:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.116+0000 7ffb588bf640 1 -- 172.21.15.112:0/1389201275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb54102070 con 0x7ffb541076a0 2024-09-11T00:55:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.117+0000 7ffb4b7fe640 1 -- 172.21.15.112:0/1389201275 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7ffb3c004e30 con 0x7ffb541076a0 2024-09-11T00:55:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.118+0000 7ffb4b7fe640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ffb400b5330 0x7ffb400b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.118+0000 7ffb4b7fe640 1 -- 172.21.15.112:0/1389201275 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6629+0+0 (secure 0 0 0) 0x7ffb3c0e5c30 con 0x7ffb541076a0 2024-09-11T00:55:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.118+0000 7ffb5259c640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ffb400b5330 0x7ffb400b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.119+0000 7ffb5259c640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ffb400b5330 0x7ffb400b77f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400bbe0 tx=0x7ffb4400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:55:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.121+0000 7ffb4b7fe640 1 -- 172.21.15.112:0/1389201275 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ffb3c056220 con 0x7ffb541076a0 2024-09-11T00:55:33.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:33.309+0000 7ffb588bf640 1 -- 172.21.15.112:0/1389201275 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ffb54106ce0 con 0x7ffb400b5330 2024-09-11T00:55:33.718 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T00:55:33.718 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: osdmap e63: 11 total, 10 up, 11 in 2024-09-11T00:55:33.718 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:33.718 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:33.718 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: Detected new or changed devices on smithi110 2024-09-11T00:55:33.718 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.718 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.718 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:33.718 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:33.718 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:33.718 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi110 to 4272M 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:33.719 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T00:55:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: osdmap e63: 11 total, 10 up, 11 in 2024-09-11T00:55:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: Detected new or changed devices on smithi110 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi110 to 4272M 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:33.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:33.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:33.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:55:33.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:55:33.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:33.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:33.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T00:55:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: osdmap e63: 11 total, 10 up, 11 in 2024-09-11T00:55:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: Detected new or changed devices on smithi110 2024-09-11T00:55:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:33.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:33.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:33.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi110 to 4272M 2024-09-11T00:55:33.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:33.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:33.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:33.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:33.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:33.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:33.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:55:33.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:55:33.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:33.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:34 smithi112 ceph-mon[35384]: purged_snaps scrub starts 2024-09-11T00:55:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:34 smithi112 ceph-mon[35384]: purged_snaps scrub ok 2024-09-11T00:55:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:34 smithi112 ceph-mon[35384]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:55:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:34 smithi112 ceph-mon[35384]: osd.10 [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168] boot 2024-09-11T00:55:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:34 smithi112 ceph-mon[35384]: osdmap e64: 11 total, 11 up, 11 in 2024-09-11T00:55:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:34 smithi112 ceph-mon[35384]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:34.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:34 smithi078 ceph-mon[31315]: purged_snaps scrub starts 2024-09-11T00:55:34.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:34 smithi078 ceph-mon[31315]: purged_snaps scrub ok 2024-09-11T00:55:34.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:34 smithi078 ceph-mon[31315]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:55:34.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:34 smithi078 ceph-mon[31315]: osd.10 [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168] boot 2024-09-11T00:55:34.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:34 smithi078 ceph-mon[31315]: osdmap e64: 11 total, 11 up, 11 in 2024-09-11T00:55:34.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:34.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:34 smithi078 ceph-mon[31315]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:34.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:34 smithi110 ceph-mon[34712]: purged_snaps scrub starts 2024-09-11T00:55:34.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:34 smithi110 ceph-mon[34712]: purged_snaps scrub ok 2024-09-11T00:55:34.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:34 smithi110 ceph-mon[34712]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:55:34.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:34 smithi110 ceph-mon[34712]: osd.10 [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168] boot 2024-09-11T00:55:34.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:34 smithi110 ceph-mon[34712]: osdmap e64: 11 total, 11 up, 11 in 2024-09-11T00:55:34.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:55:34.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:34 smithi110 ceph-mon[34712]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:55:35.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:35 smithi112 ceph-mon[35384]: osdmap e65: 11 total, 11 up, 11 in 2024-09-11T00:55:35.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:35 smithi078 ceph-mon[31315]: osdmap e65: 11 total, 11 up, 11 in 2024-09-11T00:55:35.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:35 smithi110 ceph-mon[34712]: osdmap e65: 11 total, 11 up, 11 in 2024-09-11T00:55:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:36 smithi112 ceph-mon[35384]: from='client.? 172.21.15.112:0/3960707493' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68"} : dispatch 2024-09-11T00:55:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:36 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68"} : dispatch 2024-09-11T00:55:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:36 smithi112 ceph-mon[35384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68"}]': finished 2024-09-11T00:55:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:36 smithi112 ceph-mon[35384]: osdmap e66: 12 total, 11 up, 12 in 2024-09-11T00:55:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:36 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:55:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:36 smithi112 ceph-mon[35384]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:36 smithi112 ceph-mon[35384]: from='client.? 172.21.15.112:0/503549533' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:55:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:36 smithi078 ceph-mon[31315]: from='client.? 172.21.15.112:0/3960707493' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68"} : dispatch 2024-09-11T00:55:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:36 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68"} : dispatch 2024-09-11T00:55:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:36 smithi078 ceph-mon[31315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68"}]': finished 2024-09-11T00:55:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:36 smithi078 ceph-mon[31315]: osdmap e66: 12 total, 11 up, 12 in 2024-09-11T00:55:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:36 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:55:36.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:36 smithi078 ceph-mon[31315]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:36.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:36 smithi078 ceph-mon[31315]: from='client.? 172.21.15.112:0/503549533' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:55:36.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:36 smithi110 ceph-mon[34712]: from='client.? 172.21.15.112:0/3960707493' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68"} : dispatch 2024-09-11T00:55:36.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:36 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68"} : dispatch 2024-09-11T00:55:36.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:36 smithi110 ceph-mon[34712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68"}]': finished 2024-09-11T00:55:36.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:36 smithi110 ceph-mon[34712]: osdmap e66: 12 total, 11 up, 12 in 2024-09-11T00:55:36.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:36 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:55:36.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:36 smithi110 ceph-mon[34712]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:36.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:36 smithi110 ceph-mon[34712]: from='client.? 172.21.15.112:0/503549533' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:55:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:39 smithi110 ceph-mon[34712]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:39.450 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:39 smithi078 ceph-mon[31315]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:39 smithi112 ceph-mon[35384]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:41.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:41 smithi112 ceph-mon[35384]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-11T00:55:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:41 smithi110 ceph-mon[34712]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-11T00:55:41.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:41 smithi078 ceph-mon[31315]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-11T00:55:43.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:43 smithi112 ceph-mon[35384]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:55:43.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:43 smithi110 ceph-mon[34712]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:55:43.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:43 smithi078 ceph-mon[31315]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:55:45.316 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:45 smithi112 ceph-mon[35384]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T00:55:45.316 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-11T00:55:45.316 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:45 smithi110 ceph-mon[34712]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T00:55:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-11T00:55:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:45 smithi078 ceph-mon[31315]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T00:55:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-11T00:55:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:46.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:46 smithi110 ceph-mon[34712]: Deploying daemon osd.11 on smithi112 2024-09-11T00:55:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:46 smithi112 ceph-mon[35384]: Deploying daemon osd.11 on smithi112 2024-09-11T00:55:46.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:46 smithi078 ceph-mon[31315]: Deploying daemon osd.11 on smithi112 2024-09-11T00:55:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:47 smithi110 ceph-mon[34712]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-11T00:55:47.458 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:47 smithi112 ceph-mon[35384]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-11T00:55:47.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:47 smithi078 ceph-mon[31315]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-11T00:55:49.363 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:49 smithi112 ceph-mon[35384]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:55:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:49 smithi110 ceph-mon[34712]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:55:49.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:49 smithi078 ceph-mon[31315]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:55:51.414 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:51 smithi112 ceph-mon[35384]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:55:51.414 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:51 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:51.414 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:51 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:51.415 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:51 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:51 smithi110 ceph-mon[34712]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:55:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:51 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:51 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:51 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:51.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:51 smithi078 ceph-mon[31315]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:55:51.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:51 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:51.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:51 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:51.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:51 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:53.398 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:53 smithi112 ceph-mon[35384]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:53 smithi110 ceph-mon[34712]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:53 smithi078 ceph-mon[31315]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:54.065 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 11 on host 'smithi112' 2024-09-11T00:55:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.063+0000 7ffb4b7fe640 1 -- 172.21.15.112:0/1389201275 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ffb54106ce0 con 0x7ffb400b5330 2024-09-11T00:55:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.065+0000 7ffb588bf640 1 -- 172.21.15.112:0/1389201275 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ffb400b5330 msgr2=0x7ffb400b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.065+0000 7ffb588bf640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ffb400b5330 0x7ffb400b77f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400bbe0 tx=0x7ffb4400b390 comp rx=0 tx=0).stop 2024-09-11T00:55:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.065+0000 7ffb588bf640 1 -- 172.21.15.112:0/1389201275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb541076a0 msgr2=0x7ffb541997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.065+0000 7ffb588bf640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb541076a0 0x7ffb541997b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00e9f0 tx=0x7ffb3c00edf0 comp rx=0 tx=0).stop 2024-09-11T00:55:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.066+0000 7ffb588bf640 1 -- 172.21.15.112:0/1389201275 shutdown_connections 2024-09-11T00:55:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.066+0000 7ffb588bf640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ffb400b5330 0x7ffb400b77f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.066+0000 7ffb588bf640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb54108020 0x7ffb5419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.066+0000 7ffb588bf640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb541076a0 0x7ffb541997b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.066+0000 7ffb588bf640 1 --2- 172.21.15.112:0/1389201275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb54101c90 0x7ffb54199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.066+0000 7ffb588bf640 1 -- 172.21.15.112:0/1389201275 >> 172.21.15.112:0/1389201275 conn(0x7ffb540fd700 msgr2=0x7ffb5406aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:55:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.066+0000 7ffb588bf640 1 -- 172.21.15.112:0/1389201275 shutdown_connections 2024-09-11T00:55:54.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:55:54.066+0000 7ffb588bf640 1 -- 172.21.15.112:0/1389201275 wait complete. 2024-09-11T00:55:54.783 DEBUG:teuthology.orchestra.run.smithi112:osd.11> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.11.service 2024-09-11T00:55:54.786 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-11T00:55:54.786 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T00:55:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:55:55.171 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:55 smithi078 ceph-mon[31315]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:55.172 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:55 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:55.172 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:55 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:55.172 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:55 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:55.172 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:55 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:55.422 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:55 smithi112 ceph-mon[35384]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:55.423 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:55 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:55.423 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:55 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:55.423 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:55 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:55.423 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:55 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:55 smithi110 ceph-mon[34712]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:55 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:55 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:55 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:55 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:55.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.799+0000 7fdb81e5f640 1 -- 172.21.15.78:0/3536654630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb7c102de0 msgr2=0x7fdb7c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:55.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.799+0000 7fdb81e5f640 1 --2- 172.21.15.78:0/3536654630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb7c102de0 0x7fdb7c1031c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009900 tx=0x7fdb6c02a410 comp rx=0 tx=0).stop 2024-09-11T00:55:55.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.799+0000 7fdb81e5f640 1 -- 172.21.15.78:0/3536654630 shutdown_connections 2024-09-11T00:55:55.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.799+0000 7fdb81e5f640 1 --2- 172.21.15.78:0/3536654630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7c109170 0x7fdb7c10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.799+0000 7fdb81e5f640 1 --2- 172.21.15.78:0/3536654630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb7c1087f0 0x7fdb7c108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.799+0000 7fdb81e5f640 1 --2- 172.21.15.78:0/3536654630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb7c102de0 0x7fdb7c1031c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.799+0000 7fdb81e5f640 1 -- 172.21.15.78:0/3536654630 >> 172.21.15.78:0/3536654630 conn(0x7fdb7c0fe830 msgr2=0x7fdb7c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:55:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.799+0000 7fdb81e5f640 1 -- 172.21.15.78:0/3536654630 shutdown_connections 2024-09-11T00:55:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.799+0000 7fdb81e5f640 1 -- 172.21.15.78:0/3536654630 wait complete. 2024-09-11T00:55:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.799+0000 7fdb81e5f640 1 Processor -- start 2024-09-11T00:55:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.799+0000 7fdb81e5f640 1 -- start start 2024-09-11T00:55:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb81e5f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7c102de0 0x7fdb7c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb81e5f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb7c1087f0 0x7fdb7c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb81e5f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb7c109170 0x7fdb7c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb81e5f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c112e30 con 0x7fdb7c1087f0 2024-09-11T00:55:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb81e5f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c112cb0 con 0x7fdb7c102de0 2024-09-11T00:55:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb81e5f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c112fb0 con 0x7fdb7c109170 2024-09-11T00:55:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb7affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb7c1087f0 0x7fdb7c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb7affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb7c1087f0 0x7fdb7c199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53386/0 (socket says 172.21.15.78:53386) 2024-09-11T00:55:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb7affd640 1 -- 172.21.15.78:0/2419016119 learned_addr learned my addr 172.21.15.78:0/2419016119 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:55:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb7bfff640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb7c109170 0x7fdb7c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb7b7fe640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7c102de0 0x7fdb7c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb7affd640 1 -- 172.21.15.78:0/2419016119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb7c109170 msgr2=0x7fdb7c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb7affd640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb7c109170 0x7fdb7c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb7affd640 1 -- 172.21.15.78:0/2419016119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7c102de0 msgr2=0x7fdb7c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb7affd640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7c102de0 0x7fdb7c199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb7affd640 1 -- 172.21.15.78:0/2419016119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c19f3c0 con 0x7fdb7c1087f0 2024-09-11T00:55:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.800+0000 7fdb7affd640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb7c1087f0 0x7fdb7c199850 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400d770 tx=0x7fdb6400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:55:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.801+0000 7fdb78ff9640 1 -- 172.21.15.78:0/2419016119 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64017070 con 0x7fdb7c1087f0 2024-09-11T00:55:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.801+0000 7fdb81e5f640 1 -- 172.21.15.78:0/2419016119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7c19f6b0 con 0x7fdb7c1087f0 2024-09-11T00:55:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.801+0000 7fdb78ff9640 1 -- 172.21.15.78:0/2419016119 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb640054d0 con 0x7fdb7c1087f0 2024-09-11T00:55:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.801+0000 7fdb78ff9640 1 -- 172.21.15.78:0/2419016119 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64004c30 con 0x7fdb7c1087f0 2024-09-11T00:55:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.801+0000 7fdb81e5f640 1 -- 172.21.15.78:0/2419016119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7c1a5df0 con 0x7fdb7c1087f0 2024-09-11T00:55:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.802+0000 7fdb81e5f640 1 -- 172.21.15.78:0/2419016119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb7c1031c0 con 0x7fdb7c1087f0 2024-09-11T00:55:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.804+0000 7fdb78ff9640 1 -- 172.21.15.78:0/2419016119 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fdb64004dd0 con 0x7fdb7c1087f0 2024-09-11T00:55:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.805+0000 7fdb78ff9640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdb680b5330 0x7fdb680b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.805+0000 7fdb7b7fe640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdb680b5330 0x7fdb680b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.805+0000 7fdb78ff9640 1 -- 172.21.15.78:0/2419016119 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7024+0+0 (secure 0 0 0) 0x7fdb640e5560 con 0x7fdb7c1087f0 2024-09-11T00:55:55.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.805+0000 7fdb7b7fe640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdb680b5330 0x7fdb680b77f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c031440 tx=0x7fdb6c005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:55:55.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.807+0000 7fdb78ff9640 1 -- 172.21.15.78:0/2419016119 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdb640ac2e0 con 0x7fdb7c1087f0 2024-09-11T00:55:55.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.993+0000 7fdb81e5f640 1 -- 172.21.15.78:0/2419016119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fdb7c10a9c0 con 0x7fdb7c1087f0 2024-09-11T00:55:55.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.993+0000 7fdb78ff9640 1 -- 172.21.15.78:0/2419016119 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7fdb640b0d10 con 0x7fdb7c1087f0 2024-09-11T00:55:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:55:55.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.995+0000 7fdb81e5f640 1 -- 172.21.15.78:0/2419016119 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdb680b5330 msgr2=0x7fdb680b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:55.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.995+0000 7fdb81e5f640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdb680b5330 0x7fdb680b77f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c031440 tx=0x7fdb6c005b50 comp rx=0 tx=0).stop 2024-09-11T00:55:55.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.995+0000 7fdb81e5f640 1 -- 172.21.15.78:0/2419016119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb7c1087f0 msgr2=0x7fdb7c199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:55.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.995+0000 7fdb81e5f640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb7c1087f0 0x7fdb7c199850 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400d770 tx=0x7fdb6400dc30 comp rx=0 tx=0).stop 2024-09-11T00:55:55.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.996+0000 7fdb81e5f640 1 -- 172.21.15.78:0/2419016119 shutdown_connections 2024-09-11T00:55:55.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.996+0000 7fdb81e5f640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdb680b5330 0x7fdb680b77f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:55.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.996+0000 7fdb81e5f640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb7c109170 0x7fdb7c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:55.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.996+0000 7fdb81e5f640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb7c1087f0 0x7fdb7c199850 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:55.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.996+0000 7fdb81e5f640 1 --2- 172.21.15.78:0/2419016119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7c102de0 0x7fdb7c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:55.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.996+0000 7fdb81e5f640 1 -- 172.21.15.78:0/2419016119 >> 172.21.15.78:0/2419016119 conn(0x7fdb7c0fe830 msgr2=0x7fdb7c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:55:55.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.996+0000 7fdb81e5f640 1 -- 172.21.15.78:0/2419016119 shutdown_connections 2024-09-11T00:55:55.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:55.996+0000 7fdb81e5f640 1 -- 172.21.15.78:0/2419016119 wait complete. 2024-09-11T00:55:56.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:56 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2419016119' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:55:56.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:56 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2419016119' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:55:56.461 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:56 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2419016119' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:55:56.784 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726016133,"num_in_osds":12,"osd_in_since":1726016135,"num_remapped_pgs":0} 2024-09-11T00:55:57.067 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 00:55:56 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T00:55:56.787+0000 7f246d5c9640 -1 osd.11 0 log_to_monitors true 2024-09-11T00:55:57.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:57 smithi110 ceph-mon[34712]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:57.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:57 smithi110 ceph-mon[34712]: from='osd.11 [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:55:57.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:57 smithi110 ceph-mon[34712]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:55:57.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:57 smithi112 ceph-mon[35384]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:57.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:57 smithi112 ceph-mon[35384]: from='osd.11 [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:55:57.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:57 smithi112 ceph-mon[35384]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:55:57.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:57 smithi078 ceph-mon[31315]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:57 smithi078 ceph-mon[31315]: from='osd.11 [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:55:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:57 smithi078 ceph-mon[31315]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:55:57.785 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T00:55:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:55:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:58 smithi112 ceph-mon[35384]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T00:55:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:58 smithi112 ceph-mon[35384]: osdmap e67: 12 total, 11 up, 12 in 2024-09-11T00:55:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:58 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:55:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:58 smithi112 ceph-mon[35384]: from='osd.11 [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:55:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:58 smithi112 ceph-mon[35384]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:55:58.224 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 00:55:58 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T00:55:58.074+0000 7f246954c640 -1 osd.11 0 waiting for initial osdmap 2024-09-11T00:55:58.225 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 00:55:58 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T00:55:58.088+0000 7f2465373640 -1 osd.11 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:55:58.340 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:58 smithi078 ceph-mon[31315]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T00:55:58.340 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:58 smithi078 ceph-mon[31315]: osdmap e67: 12 total, 11 up, 12 in 2024-09-11T00:55:58.340 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:58 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:55:58.340 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:58 smithi078 ceph-mon[31315]: from='osd.11 [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:55:58.340 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:58 smithi078 ceph-mon[31315]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:55:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:58 smithi110 ceph-mon[34712]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T00:55:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:58 smithi110 ceph-mon[34712]: osdmap e67: 12 total, 11 up, 12 in 2024-09-11T00:55:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:58 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:55:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:58 smithi110 ceph-mon[34712]: from='osd.11 [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:55:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:58 smithi110 ceph-mon[34712]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-11T00:55:58.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.550+0000 7f12a9b2c640 1 -- 172.21.15.78:0/3371645881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a4103ca0 msgr2=0x7f12a4104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:58.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.550+0000 7f12a9b2c640 1 --2- 172.21.15.78:0/3371645881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a4103ca0 0x7f12a4104080 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1294009900 tx=0x7f129402a410 comp rx=0 tx=0).stop 2024-09-11T00:55:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.550+0000 7f12a9b2c640 1 -- 172.21.15.78:0/3371645881 shutdown_connections 2024-09-11T00:55:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.550+0000 7f12a9b2c640 1 --2- 172.21.15.78:0/3371645881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12a410d100 0x7f12a410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.550+0000 7f12a9b2c640 1 --2- 172.21.15.78:0/3371645881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12a41045c0 0x7f12a410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.550+0000 7f12a9b2c640 1 --2- 172.21.15.78:0/3371645881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a4103ca0 0x7f12a4104080 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.550+0000 7f12a9b2c640 1 -- 172.21.15.78:0/3371645881 >> 172.21.15.78:0/3371645881 conn(0x7f12a40fd700 msgr2=0x7f12a40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:55:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.550+0000 7f12a9b2c640 1 -- 172.21.15.78:0/3371645881 shutdown_connections 2024-09-11T00:55:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.550+0000 7f12a9b2c640 1 -- 172.21.15.78:0/3371645881 wait complete. 2024-09-11T00:55:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.550+0000 7f12a9b2c640 1 Processor -- start 2024-09-11T00:55:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.550+0000 7f12a9b2c640 1 -- start start 2024-09-11T00:55:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a9b2c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12a4103ca0 0x7f12a4101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a9b2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12a41045c0 0x7f12a4101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a9b2c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a410d100 0x7f12a41023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a9b2c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4112d70 con 0x7f12a410d100 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a9b2c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4112bf0 con 0x7f12a4103ca0 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a9b2c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4112ef0 con 0x7f12a41045c0 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a410d100 0x7f12a41023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a410d100 0x7f12a41023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53412/0 (socket says 172.21.15.78:53412) 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a3fff640 1 -- 172.21.15.78:0/1946654901 learned_addr learned my addr 172.21.15.78:0/1946654901 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a37fe640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12a4103ca0 0x7f12a4101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f129bfff640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12a41045c0 0x7f12a4101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a3fff640 1 -- 172.21.15.78:0/1946654901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12a41045c0 msgr2=0x7f12a4101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a3fff640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12a41045c0 0x7f12a4101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a3fff640 1 -- 172.21.15.78:0/1946654901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12a4103ca0 msgr2=0x7f12a4101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a3fff640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12a4103ca0 0x7f12a4101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a3fff640 1 -- 172.21.15.78:0/1946654901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a41a76e0 con 0x7f12a410d100 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a3fff640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a410d100 0x7f12a41023d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f129c00eb10 tx=0x7f129c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a17fa640 1 -- 172.21.15.78:0/1946654901 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c004b20 con 0x7f12a410d100 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a17fa640 1 -- 172.21.15.78:0/1946654901 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f129c004cc0 con 0x7f12a410d100 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a9b2c640 1 -- 172.21.15.78:0/1946654901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a41a79d0 con 0x7f12a410d100 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a17fa640 1 -- 172.21.15.78:0/1946654901 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c010670 con 0x7f12a410d100 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.551+0000 7f12a9b2c640 1 -- 172.21.15.78:0/1946654901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a41a7f10 con 0x7f12a410d100 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.553+0000 7f12a9b2c640 1 -- 172.21.15.78:0/1946654901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a4104080 con 0x7f12a410d100 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.554+0000 7f12a17fa640 1 -- 172.21.15.78:0/1946654901 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f129c005020 con 0x7f12a410d100 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.554+0000 7f12a17fa640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f12900b53b0 0x7f12900b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.554+0000 7f12a37fe640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f12900b53b0 0x7f12900b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.555+0000 7f12a17fa640 1 -- 172.21.15.78:0/1946654901 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7062+0+0 (secure 0 0 0) 0x7f129c0e5d40 con 0x7f12a410d100 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.555+0000 7f12a37fe640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f12900b53b0 0x7f12900b7870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1294031440 tx=0x7f1294005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:55:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.558+0000 7f12a17fa640 1 -- 172.21.15.78:0/1946654901 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f129c0ad0e0 con 0x7f12a410d100 2024-09-11T00:55:58.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.733+0000 7f12a9b2c640 1 -- 172.21.15.78:0/1946654901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f12a402e5c0 con 0x7f12a410d100 2024-09-11T00:55:58.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.734+0000 7f12a17fa640 1 -- 172.21.15.78:0/1946654901 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f129c0ac9e0 con 0x7f12a410d100 2024-09-11T00:55:58.736 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:55:58.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.736+0000 7f12a9b2c640 1 -- 172.21.15.78:0/1946654901 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f12900b53b0 msgr2=0x7f12900b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:58.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.736+0000 7f12a9b2c640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f12900b53b0 0x7f12900b7870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1294031440 tx=0x7f1294005b50 comp rx=0 tx=0).stop 2024-09-11T00:55:58.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.736+0000 7f12a9b2c640 1 -- 172.21.15.78:0/1946654901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a410d100 msgr2=0x7f12a41023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:55:58.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.736+0000 7f12a9b2c640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a410d100 0x7f12a41023d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f129c00eb10 tx=0x7f129c00c2d0 comp rx=0 tx=0).stop 2024-09-11T00:55:58.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.736+0000 7f12a9b2c640 1 -- 172.21.15.78:0/1946654901 shutdown_connections 2024-09-11T00:55:58.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.736+0000 7f12a9b2c640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f12900b53b0 0x7f12900b7870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:58.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.736+0000 7f12a9b2c640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a410d100 0x7f12a41023d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:58.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.736+0000 7f12a9b2c640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12a41045c0 0x7f12a4101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:58.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.736+0000 7f12a9b2c640 1 --2- 172.21.15.78:0/1946654901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12a4103ca0 0x7f12a4101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:55:58.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.736+0000 7f12a9b2c640 1 -- 172.21.15.78:0/1946654901 >> 172.21.15.78:0/1946654901 conn(0x7f12a40fd700 msgr2=0x7f12a40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:55:58.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.736+0000 7f12a9b2c640 1 -- 172.21.15.78:0/1946654901 shutdown_connections 2024-09-11T00:55:58.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:55:58.736+0000 7f12a9b2c640 1 -- 172.21.15.78:0/1946654901 wait complete. 2024-09-11T00:55:59.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:59.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-11T00:55:59.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: osdmap e68: 12 total, 11 up, 12 in 2024-09-11T00:55:59.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:55:59.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:55:59.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.342 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.343 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:55:59 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1946654901' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:55:59.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:59.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-11T00:55:59.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: osdmap e68: 12 total, 11 up, 12 in 2024-09-11T00:55:59.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:55:59.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:55:59.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:59.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:59.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:59.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:59.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:59.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:59.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:59.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:59.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:59.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:55:59 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1946654901' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:55:59.456 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726016133,"num_in_osds":12,"osd_in_since":1726016135,"num_remapped_pgs":0} 2024-09-11T00:55:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:55:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-11T00:55:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: osdmap e68: 12 total, 11 up, 12 in 2024-09-11T00:55:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:55:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:55:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:55:59.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:55:59 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1946654901' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:56:00.362 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:00 smithi110 ceph-mon[34712]: purged_snaps scrub starts 2024-09-11T00:56:00.362 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:00 smithi110 ceph-mon[34712]: purged_snaps scrub ok 2024-09-11T00:56:00.362 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:00 smithi110 ceph-mon[34712]: Detected new or changed devices on smithi112 2024-09-11T00:56:00.363 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:00 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi112 to 4272M 2024-09-11T00:56:00.363 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:00 smithi110 ceph-mon[34712]: osd.11 [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654] boot 2024-09-11T00:56:00.363 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:00 smithi110 ceph-mon[34712]: osdmap e69: 12 total, 12 up, 12 in 2024-09-11T00:56:00.363 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:00 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:56:00.457 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T00:56:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:00 smithi112 ceph-mon[35384]: purged_snaps scrub starts 2024-09-11T00:56:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:00 smithi112 ceph-mon[35384]: purged_snaps scrub ok 2024-09-11T00:56:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:00 smithi112 ceph-mon[35384]: Detected new or changed devices on smithi112 2024-09-11T00:56:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:00 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi112 to 4272M 2024-09-11T00:56:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:00 smithi112 ceph-mon[35384]: osd.11 [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654] boot 2024-09-11T00:56:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:00 smithi112 ceph-mon[35384]: osdmap e69: 12 total, 12 up, 12 in 2024-09-11T00:56:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:00 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:56:00.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:00 smithi078 ceph-mon[31315]: purged_snaps scrub starts 2024-09-11T00:56:00.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:00 smithi078 ceph-mon[31315]: purged_snaps scrub ok 2024-09-11T00:56:00.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:00 smithi078 ceph-mon[31315]: Detected new or changed devices on smithi112 2024-09-11T00:56:00.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:00 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi112 to 4272M 2024-09-11T00:56:00.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:00 smithi078 ceph-mon[31315]: osd.11 [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654] boot 2024-09-11T00:56:00.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:00 smithi078 ceph-mon[31315]: osdmap e69: 12 total, 12 up, 12 in 2024-09-11T00:56:00.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:00 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:56:00.713 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:01.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.214+0000 7f9d1a424640 1 -- 172.21.15.78:0/872981779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d1410d140 msgr2=0x7f9d1410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:01.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.214+0000 7f9d1a424640 1 --2- 172.21.15.78:0/872981779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d1410d140 0x7f9d1410f5a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009ca0 tx=0x7f9d0c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:56:01.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.214+0000 7f9d1a424640 1 -- 172.21.15.78:0/872981779 shutdown_connections 2024-09-11T00:56:01.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.214+0000 7f9d1a424640 1 --2- 172.21.15.78:0/872981779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d1410d140 0x7f9d1410f5a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.214+0000 7f9d1a424640 1 --2- 172.21.15.78:0/872981779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d14104600 0x7f9d1410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.214+0000 7f9d1a424640 1 --2- 172.21.15.78:0/872981779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d14103ce0 0x7f9d141040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.214+0000 7f9d1a424640 1 -- 172.21.15.78:0/872981779 >> 172.21.15.78:0/872981779 conn(0x7f9d140fd700 msgr2=0x7f9d140ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.214+0000 7f9d1a424640 1 -- 172.21.15.78:0/872981779 shutdown_connections 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.214+0000 7f9d1a424640 1 -- 172.21.15.78:0/872981779 wait complete. 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.214+0000 7f9d1a424640 1 Processor -- start 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.214+0000 7f9d1a424640 1 -- start start 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d1a424640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d14103ce0 0x7f9d14199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d1a424640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d14104600 0x7f9d141997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d1a424640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d1410d140 0x7f9d1419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d1a424640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14112db0 con 0x7f9d1410d140 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d1a424640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14112c30 con 0x7f9d14104600 2024-09-11T00:56:01.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d1a424640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14112f30 con 0x7f9d14103ce0 2024-09-11T00:56:01.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d189c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d1410d140 0x7f9d1419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:01.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d189c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d1410d140 0x7f9d1419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53420/0 (socket says 172.21.15.78:53420) 2024-09-11T00:56:01.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d189c1640 1 -- 172.21.15.78:0/4038294999 learned_addr learned my addr 172.21.15.78:0/4038294999 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:01.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d137fe640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d14104600 0x7f9d141997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:01.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d13fff640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d14103ce0 0x7f9d14199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:01.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d189c1640 1 -- 172.21.15.78:0/4038294999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d14103ce0 msgr2=0x7f9d14199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:01.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d189c1640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d14103ce0 0x7f9d14199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:01.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d189c1640 1 -- 172.21.15.78:0/4038294999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d14104600 msgr2=0x7f9d141997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:01.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d189c1640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d14104600 0x7f9d141997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:01.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d189c1640 1 -- 172.21.15.78:0/4038294999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1419f330 con 0x7f9d1410d140 2024-09-11T00:56:01.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d137fe640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d14104600 0x7f9d141997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:56:01.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.215+0000 7f9d189c1640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d1410d140 0x7f9d1419eb60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0098d0 tx=0x7f9d0c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:01.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.216+0000 7f9d117fa640 1 -- 172.21.15.78:0/4038294999 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c003b70 con 0x7f9d1410d140 2024-09-11T00:56:01.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.216+0000 7f9d117fa640 1 -- 172.21.15.78:0/4038294999 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0c003d10 con 0x7f9d1410d140 2024-09-11T00:56:01.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.216+0000 7f9d1a424640 1 -- 172.21.15.78:0/4038294999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1419f5c0 con 0x7f9d1410d140 2024-09-11T00:56:01.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.216+0000 7f9d117fa640 1 -- 172.21.15.78:0/4038294999 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c02fa20 con 0x7f9d1410d140 2024-09-11T00:56:01.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.216+0000 7f9d1a424640 1 -- 172.21.15.78:0/4038294999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d141a5ce0 con 0x7f9d1410d140 2024-09-11T00:56:01.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.217+0000 7f9d1a424640 1 -- 172.21.15.78:0/4038294999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d141040c0 con 0x7f9d1410d140 2024-09-11T00:56:01.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.218+0000 7f9d117fa640 1 -- 172.21.15.78:0/4038294999 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f9d0c02fbe0 con 0x7f9d1410d140 2024-09-11T00:56:01.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.219+0000 7f9d117fa640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9cfc0b5330 0x7f9cfc0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:01.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.219+0000 7f9d13fff640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9cfc0b5330 0x7f9cfc0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:01.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.219+0000 7f9d117fa640 1 -- 172.21.15.78:0/4038294999 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f9d0c106c50 con 0x7f9d1410d140 2024-09-11T00:56:01.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.219+0000 7f9d13fff640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9cfc0b5330 0x7f9cfc0b77f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009bd0 tx=0x7f9d08005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:01.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.222+0000 7f9d117fa640 1 -- 172.21.15.78:0/4038294999 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9d0c0cdee0 con 0x7f9d1410d140 2024-09-11T00:56:01.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.398+0000 7f9d1a424640 1 -- 172.21.15.78:0/4038294999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9d14107bd0 con 0x7f9d1410d140 2024-09-11T00:56:01.400 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:01 smithi078 ceph-mon[31315]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:56:01.400 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:01 smithi078 ceph-mon[31315]: osdmap e70: 12 total, 12 up, 12 in 2024-09-11T00:56:01.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.398+0000 7f9d117fa640 1 -- 172.21.15.78:0/4038294999 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f9d0c0cd7e0 con 0x7f9d1410d140 2024-09-11T00:56:01.401 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:01.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.400+0000 7f9d1a424640 1 -- 172.21.15.78:0/4038294999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9cfc0b5330 msgr2=0x7f9cfc0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:01.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.400+0000 7f9d1a424640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9cfc0b5330 0x7f9cfc0b77f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009bd0 tx=0x7f9d08005f90 comp rx=0 tx=0).stop 2024-09-11T00:56:01.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.400+0000 7f9d1a424640 1 -- 172.21.15.78:0/4038294999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d1410d140 msgr2=0x7f9d1419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:01.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.400+0000 7f9d1a424640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d1410d140 0x7f9d1419eb60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0098d0 tx=0x7f9d0c0317b0 comp rx=0 tx=0).stop 2024-09-11T00:56:01.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.400+0000 7f9d1a424640 1 -- 172.21.15.78:0/4038294999 shutdown_connections 2024-09-11T00:56:01.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.400+0000 7f9d1a424640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9cfc0b5330 0x7f9cfc0b77f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:01.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.400+0000 7f9d1a424640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d1410d140 0x7f9d1419eb60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:01.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.400+0000 7f9d1a424640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d14104600 0x7f9d141997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:01.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.400+0000 7f9d1a424640 1 --2- 172.21.15.78:0/4038294999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d14103ce0 0x7f9d14199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:01.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.400+0000 7f9d1a424640 1 -- 172.21.15.78:0/4038294999 >> 172.21.15.78:0/4038294999 conn(0x7f9d140fd700 msgr2=0x7f9d140fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:01.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.401+0000 7f9d1a424640 1 -- 172.21.15.78:0/4038294999 shutdown_connections 2024-09-11T00:56:01.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:01.401+0000 7f9d1a424640 1 -- 172.21.15.78:0/4038294999 wait complete. 2024-09-11T00:56:01.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:01 smithi110 ceph-mon[34712]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:56:01.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:01 smithi110 ceph-mon[34712]: osdmap e70: 12 total, 12 up, 12 in 2024-09-11T00:56:01.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:01 smithi112 ceph-mon[35384]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:56:01.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:01 smithi112 ceph-mon[35384]: osdmap e70: 12 total, 12 up, 12 in 2024-09-11T00:56:02.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:02 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4038294999' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:56:02.255 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1726016159,"num_in_osds":12,"osd_in_since":1726016135,"num_remapped_pgs":0} 2024-09-11T00:56:02.256 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:56:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:02 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/4038294999' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:56:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:02 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/4038294999' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:03.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.997+0000 7f65f3f89640 1 -- 172.21.15.78:0/450828222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65ec109150 msgr2=0x7f65ec10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:03.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.997+0000 7f65f3f89640 1 --2- 172.21.15.78:0/450828222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65ec109150 0x7f65ec10f950 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009ca0 tx=0x7f65e402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:56:03.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.997+0000 7f65f3f89640 1 -- 172.21.15.78:0/450828222 shutdown_connections 2024-09-11T00:56:03.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.997+0000 7f65f3f89640 1 --2- 172.21.15.78:0/450828222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65ec109150 0x7f65ec10f950 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:03.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.997+0000 7f65f3f89640 1 --2- 172.21.15.78:0/450828222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec1087d0 0x7f65ec108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:03.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.997+0000 7f65f3f89640 1 --2- 172.21.15.78:0/450828222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65ec102dc0 0x7f65ec1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:03.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.997+0000 7f65f3f89640 1 -- 172.21.15.78:0/450828222 >> 172.21.15.78:0/450828222 conn(0x7f65ec0fe830 msgr2=0x7f65ec100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:03.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f3f89640 1 -- 172.21.15.78:0/450828222 shutdown_connections 2024-09-11T00:56:03.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f3f89640 1 -- 172.21.15.78:0/450828222 wait complete. 2024-09-11T00:56:03.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f3f89640 1 Processor -- start 2024-09-11T00:56:03.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f3f89640 1 -- start start 2024-09-11T00:56:03.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f3f89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec102dc0 0x7f65ec1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:03.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f3f89640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65ec1087d0 0x7f65ec1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:03.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f3f89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65ec109150 0x7f65ec19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:03.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f3f89640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec112e50 con 0x7f65ec109150 2024-09-11T00:56:03.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f3f89640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec112cd0 con 0x7f65ec1087d0 2024-09-11T00:56:03.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f3f89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec112fd0 con 0x7f65ec102dc0 2024-09-11T00:56:03.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f2526640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65ec109150 0x7f65ec19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:03.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f2526640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65ec109150 0x7f65ec19ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53434/0 (socket says 172.21.15.78:53434) 2024-09-11T00:56:03.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.998+0000 7f65f2526640 1 -- 172.21.15.78:0/870358017 learned_addr learned my addr 172.21.15.78:0/870358017 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:03.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65f1524640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65ec1087d0 0x7f65ec1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:03.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65f2526640 1 -- 172.21.15.78:0/870358017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec102dc0 msgr2=0x7f65ec1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:03.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65f1d25640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec102dc0 0x7f65ec1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:03.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65f2526640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec102dc0 0x7f65ec1993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:03.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65f2526640 1 -- 172.21.15.78:0/870358017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65ec1087d0 msgr2=0x7f65ec1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65f2526640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65ec1087d0 0x7f65ec1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65f2526640 1 -- 172.21.15.78:0/870358017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec19f450 con 0x7f65ec109150 2024-09-11T00:56:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65f1d25640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec102dc0 0x7f65ec1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:56:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65f2526640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65ec109150 0x7f65ec19ec80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f65e4005cc0 tx=0x7f65e402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65e2ffd640 1 -- 172.21.15.78:0/870358017 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4003cf0 con 0x7f65ec109150 2024-09-11T00:56:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65e2ffd640 1 -- 172.21.15.78:0/870358017 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e40044f0 con 0x7f65ec109150 2024-09-11T00:56:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65f3f89640 1 -- 172.21.15.78:0/870358017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec19f6e0 con 0x7f65ec109150 2024-09-11T00:56:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65e2ffd640 1 -- 172.21.15.78:0/870358017 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e402fd10 con 0x7f65ec109150 2024-09-11T00:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:02.999+0000 7f65f3f89640 1 -- 172.21.15.78:0/870358017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec1a5e80 con 0x7f65ec109150 2024-09-11T00:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.000+0000 7f65f3f89640 1 -- 172.21.15.78:0/870358017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ec103220 con 0x7f65ec109150 2024-09-11T00:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.001+0000 7f65e2ffd640 1 -- 172.21.15.78:0/870358017 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f65e4004040 con 0x7f65ec109150 2024-09-11T00:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.002+0000 7f65e2ffd640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f65d80b53b0 0x7f65d80b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.002+0000 7f65f1d25640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f65d80b53b0 0x7f65d80b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.003+0000 7f65e2ffd640 1 -- 172.21.15.78:0/870358017 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f65e4107090 con 0x7f65ec109150 2024-09-11T00:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.003+0000 7f65f1d25640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f65d80b53b0 0x7f65d80b7870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f65e8006fd0 tx=0x7f65e8008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:03.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.006+0000 7f65e2ffd640 1 -- 172.21.15.78:0/870358017 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f65e40ce280 con 0x7f65ec109150 2024-09-11T00:56:03.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.183+0000 7f65f3f89640 1 -- 172.21.15.78:0/870358017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65ec02e5c0 con 0x7f65ec109150 2024-09-11T00:56:03.186 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:03 smithi078 ceph-mon[31315]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:03.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.184+0000 7f65e2ffd640 1 -- 172.21.15.78:0/870358017 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18760 (secure 0 0 0) 0x7f65e40cdb80 con 0x7f65ec109150 2024-09-11T00:56:03.187 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:03.187 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":70,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","created":"2024-09-11T00:48:27.652512+0000","modified":"2024-09-11T00:56:00.077878+0000","last_up_change":"2024-09-11T00:55:59.071507+0000","last_in_change":"2024-09-11T00:55:35.815150+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:52:10.229887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beefc6d9-f74c-4200-96f2-ff697ad0d7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6803","nonce":1424639053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6805","nonce":1424639053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6809","nonce":1424639053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6807","nonce":1424639053}]},"public_addr":"172.21.15.78:6803/1424639053","cluster_addr":"172.21.15.78:6805/1424639053","heartbeat_back_addr":"172.21.15.78:6809/1424639053","heartbeat_front_addr":"172.21.15.78:6807/1424639053","state":["exists","up"]},{"osd":1,"uuid":"d742e767-d673-4bb9-b624-8bc969f5ceb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6801","nonce":580939385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6803","nonce":580939385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6807","nonce":580939385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6805","nonce":580939385}]},"public_addr":"172.21.15.110:6801/580939385","cluster_addr":"172.21.15.110:6803/580939385","heartbeat_back_addr":"172.21.15.110:6807/580939385","heartbeat_front_addr":"172.21.15.110:6805/580939385","state":["exists","up"]},{"osd":2,"uuid":"b0106a5b-1465-4a50-96dd-934dbae49a4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6801","nonce":2983155763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6803","nonce":2983155763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6807","nonce":2983155763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6805","nonce":2983155763}]},"public_addr":"172.21.15.112:6801/2983155763","cluster_addr":"172.21.15.112:6803/2983155763","heartbeat_back_addr":"172.21.15.112:6807/2983155763","heartbeat_front_addr":"172.21.15.112:6805/2983155763","state":["exists","up"]},{"osd":3,"uuid":"3270362b-3d61-414a-a3d9-f2416ff8b870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6811","nonce":2906086464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6813","nonce":2906086464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6817","nonce":2906086464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6815","nonce":2906086464}]},"public_addr":"172.21.15.78:6811/2906086464","cluster_addr":"172.21.15.78:6813/2906086464","heartbeat_back_addr":"172.21.15.78:6817/2906086464","heartbeat_front_addr":"172.21.15.78:6815/2906086464","state":["exists","up"]},{"osd":4,"uuid":"e47746ec-9443-4004-abfa-17c1ed73b982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6809","nonce":477287435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6811","nonce":477287435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6815","nonce":477287435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6813","nonce":477287435}]},"public_addr":"172.21.15.110:6809/477287435","cluster_addr":"172.21.15.110:6811/477287435","heartbeat_back_addr":"172.21.15.110:6815/477287435","heartbeat_front_addr":"172.21.15.110:6813/477287435","state":["exists","up"]},{"osd":5,"uuid":"f8cedb01-310d-442d-a588-33197d32e208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6809","nonce":3557095096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6811","nonce":3557095096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6815","nonce":3557095096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6813","nonce":3557095096}]},"public_addr":"172.21.15.112:6809/3557095096","cluster_addr":"172.21.15.112:6811/3557095096","heartbeat_back_addr":"172.21.15.112:6815/3557095096","heartbeat_front_addr":"172.21.15.112:6813/3557095096","state":["exists","up"]},{"osd":6,"uuid":"d508a037-8f56-436b-b7bf-9a1c69f6d9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6819","nonce":1267910775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6821","nonce":1267910775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6825","nonce":1267910775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6823","nonce":1267910775}]},"public_addr":"172.21.15.78:6819/1267910775","cluster_addr":"172.21.15.78:6821/1267910775","heartbeat_back_addr":"172.21.15.78:6825/1267910775","heartbeat_front_addr":"172.21.15.78:6823/1267910775","state":["exists","up"]},{"osd":7,"uuid":"e8842047-09a6-48b8-8d0a-2d54e7f89063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6817","nonce":1556494594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6819","nonce":1556494594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6823","nonce":1556494594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6821","nonce":1556494594}]},"public_addr":"172.21.15.110:6817/1556494594","cluster_addr":"172.21.15.110:6819/1556494594","heartbeat_back_addr":"172.21.15.110:6823/1556494594","heartbeat_front_addr":"172.21.15.110:6821/1556494594","state":["exists","up"]},{"osd":8,"uuid":"a3367da8-375a-4a46-a952-867d4b72ae2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6817","nonce":2481321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6819","nonce":2481321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6823","nonce":2481321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6821","nonce":2481321}]},"public_addr":"172.21.15.112:6817/2481321","cluster_addr":"172.21.15.112:6819/2481321","heartbeat_back_addr":"172.21.15.112:6823/2481321","heartbeat_front_addr":"172.21.15.112:6821/2481321","state":["exists","up"]},{"osd":9,"uuid":"d1825764-cefc-495b-99da-8ba429df0937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6827","nonce":197048350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6829","nonce":197048350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6833","nonce":197048350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6831","nonce":197048350}]},"public_addr":"172.21.15.78:6827/197048350","cluster_addr":"172.21.15.78:6829/197048350","heartbeat_back_addr":"172.21.15.78:6833/197048350","heartbeat_front_addr":"172.21.15.78:6831/197048350","state":["exists","up"]},{"osd":10,"uuid":"e1cfff7a-8916-4845-a482-d79fa275da30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6825","nonce":2101123168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6827","nonce":2101123168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6831","nonce":2101123168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6829","nonce":2101123168}]},"public_addr":"172.21.15.110:6825/2101123168","cluster_addr":"172.21.15.110:6827/2101123168","heartbeat_back_addr":"172.21.15.110:6831/2101123168","heartbeat_front_addr":"172.21.15.110:6829/2101123168","state":["exists","up"]},{"osd":11,"uuid":"fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6825","nonce":1526310654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6827","nonce":1526310654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6831","nonce":1526310654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6829","nonce":1526310654}]},"public_addr":"172.21.15.112:6825/1526310654","cluster_addr":"172.21.15.112:6827/1526310654","heartbeat_back_addr":"172.21.15.112:6831/1526310654","heartbeat_front_addr":"172.21.15.112:6829/1526310654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:17.334199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:41.400872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:08.205144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:33.550527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:58.394116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:24.699607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:50.450193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:16.223363+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:40.947727+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:06.845451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:32.410035+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:57.744364+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/2390785703":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:0/1038951931":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/391908936":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2706514734":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/2357025243":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6800/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2795778473":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2747741101":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6801/1289343426":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/3558896007":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/438261739":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/1289343426":"2024-09-12T00:50:15.928079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:56:03.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.187+0000 7f65f3f89640 1 -- 172.21.15.78:0/870358017 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f65d80b53b0 msgr2=0x7f65d80b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:03.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.187+0000 7f65f3f89640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f65d80b53b0 0x7f65d80b7870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f65e8006fd0 tx=0x7f65e8008040 comp rx=0 tx=0).stop 2024-09-11T00:56:03.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.187+0000 7f65f3f89640 1 -- 172.21.15.78:0/870358017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65ec109150 msgr2=0x7f65ec19ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:03.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.187+0000 7f65f3f89640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65ec109150 0x7f65ec19ec80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f65e4005cc0 tx=0x7f65e402f470 comp rx=0 tx=0).stop 2024-09-11T00:56:03.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.187+0000 7f65f3f89640 1 -- 172.21.15.78:0/870358017 shutdown_connections 2024-09-11T00:56:03.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.187+0000 7f65f3f89640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f65d80b53b0 0x7f65d80b7870 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:03.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.187+0000 7f65f3f89640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65ec109150 0x7f65ec19ec80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:03.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.187+0000 7f65f3f89640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65ec1087d0 0x7f65ec1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:03.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.187+0000 7f65f3f89640 1 --2- 172.21.15.78:0/870358017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec102dc0 0x7f65ec1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:03.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.187+0000 7f65f3f89640 1 -- 172.21.15.78:0/870358017 >> 172.21.15.78:0/870358017 conn(0x7f65ec0fe830 msgr2=0x7f65ec100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:03.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.187+0000 7f65f3f89640 1 -- 172.21.15.78:0/870358017 shutdown_connections 2024-09-11T00:56:03.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:03.187+0000 7f65f3f89640 1 -- 172.21.15.78:0/870358017 wait complete. 2024-09-11T00:56:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:03 smithi110 ceph-mon[34712]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:03.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:03 smithi112 ceph-mon[35384]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:03.972 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T00:52:10.229887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T00:56:03.972 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T00:56:04.221 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:04.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:04 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/870358017' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:56:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:04 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/870358017' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:56:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:04 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/870358017' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:56:04.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.700+0000 7f1909cac640 1 -- 172.21.15.78:0/517746773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1904109150 msgr2=0x7f190410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.700+0000 7f1909cac640 1 --2- 172.21.15.78:0/517746773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1904109150 0x7f190410f950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009ca0 tx=0x7f18fc02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:56:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.700+0000 7f1909cac640 1 -- 172.21.15.78:0/517746773 shutdown_connections 2024-09-11T00:56:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.700+0000 7f1909cac640 1 --2- 172.21.15.78:0/517746773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1904109150 0x7f190410f950 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.700+0000 7f1909cac640 1 --2- 172.21.15.78:0/517746773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19041087d0 0x7f1904108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.700+0000 7f1909cac640 1 --2- 172.21.15.78:0/517746773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1904102dc0 0x7f19041031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.700+0000 7f1909cac640 1 -- 172.21.15.78:0/517746773 >> 172.21.15.78:0/517746773 conn(0x7f19040fe830 msgr2=0x7f1904100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.700+0000 7f1909cac640 1 -- 172.21.15.78:0/517746773 shutdown_connections 2024-09-11T00:56:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.700+0000 7f1909cac640 1 -- 172.21.15.78:0/517746773 wait complete. 2024-09-11T00:56:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f1909cac640 1 Processor -- start 2024-09-11T00:56:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f1909cac640 1 -- start start 2024-09-11T00:56:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f1909cac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1904102dc0 0x7f1904199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f1909cac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19041087d0 0x7f19041997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f1909cac640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1904109150 0x7f190419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f1909cac640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1904112dd0 con 0x7f19041087d0 2024-09-11T00:56:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f1909cac640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1904112c50 con 0x7f1904109150 2024-09-11T00:56:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f1909cac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1904112f50 con 0x7f1904102dc0 2024-09-11T00:56:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f1902ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19041087d0 0x7f19041997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f1902ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19041087d0 0x7f19041997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57138/0 (socket says 172.21.15.78:57138) 2024-09-11T00:56:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f1902ffd640 1 -- 172.21.15.78:0/572174049 learned_addr learned my addr 172.21.15.78:0/572174049 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f1903fff640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1904109150 0x7f190419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.701+0000 7f19037fe640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1904102dc0 0x7f1904199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f1902ffd640 1 -- 172.21.15.78:0/572174049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1904102dc0 msgr2=0x7f1904199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f1902ffd640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1904102dc0 0x7f1904199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f1902ffd640 1 -- 172.21.15.78:0/572174049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1904109150 msgr2=0x7f190419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f1902ffd640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1904109150 0x7f190419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f1902ffd640 1 -- 172.21.15.78:0/572174049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190419f310 con 0x7f19041087d0 2024-09-11T00:56:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f19037fe640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1904102dc0 0x7f1904199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:56:04.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f1903fff640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1904109150 0x7f190419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:56:04.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f1902ffd640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19041087d0 0x7f19041997a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f18f800d770 tx=0x7f18f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:04.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f1900ff9640 1 -- 172.21.15.78:0/572174049 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8017070 con 0x7f19041087d0 2024-09-11T00:56:04.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f1900ff9640 1 -- 172.21.15.78:0/572174049 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f80054d0 con 0x7f19041087d0 2024-09-11T00:56:04.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f1909cac640 1 -- 172.21.15.78:0/572174049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190419f600 con 0x7f19041087d0 2024-09-11T00:56:04.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f1900ff9640 1 -- 172.21.15.78:0/572174049 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8004c30 con 0x7f19041087d0 2024-09-11T00:56:04.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.702+0000 7f1909cac640 1 -- 172.21.15.78:0/572174049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19041a5cc0 con 0x7f19041087d0 2024-09-11T00:56:04.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.703+0000 7f1909cac640 1 -- 172.21.15.78:0/572174049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19041031a0 con 0x7f19041087d0 2024-09-11T00:56:04.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.705+0000 7f1900ff9640 1 -- 172.21.15.78:0/572174049 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f18f8004dd0 con 0x7f19041087d0 2024-09-11T00:56:04.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.705+0000 7f1900ff9640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18ec0b53b0 0x7f18ec0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:04.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.705+0000 7f19037fe640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18ec0b53b0 0x7f18ec0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:04.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.705+0000 7f1900ff9640 1 -- 172.21.15.78:0/572174049 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f18f80e5790 con 0x7f19041087d0 2024-09-11T00:56:04.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.706+0000 7f19037fe640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18ec0b53b0 0x7f18ec0b7870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f18f40058c0 tx=0x7f18f4005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:04.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.708+0000 7f1900ff9640 1 -- 172.21.15.78:0/572174049 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f18f80aca80 con 0x7f19041087d0 2024-09-11T00:56:04.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.883+0000 7f1909cac640 1 -- 172.21.15.78:0/572174049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f190402e5c0 con 0x7f19041087d0 2024-09-11T00:56:04.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.884+0000 7f1900ff9640 1 -- 172.21.15.78:0/572174049 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f18f80ac380 con 0x7f19041087d0 2024-09-11T00:56:04.886 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T00:56:04.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.886+0000 7f1909cac640 1 -- 172.21.15.78:0/572174049 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18ec0b53b0 msgr2=0x7f18ec0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:04.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.886+0000 7f1909cac640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18ec0b53b0 0x7f18ec0b7870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f18f40058c0 tx=0x7f18f4005850 comp rx=0 tx=0).stop 2024-09-11T00:56:04.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.886+0000 7f1909cac640 1 -- 172.21.15.78:0/572174049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19041087d0 msgr2=0x7f19041997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:04.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.886+0000 7f1909cac640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19041087d0 0x7f19041997a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f18f800d770 tx=0x7f18f800dc30 comp rx=0 tx=0).stop 2024-09-11T00:56:04.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.886+0000 7f1909cac640 1 -- 172.21.15.78:0/572174049 shutdown_connections 2024-09-11T00:56:04.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.886+0000 7f1909cac640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18ec0b53b0 0x7f18ec0b7870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.886+0000 7f1909cac640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1904109150 0x7f190419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.886+0000 7f1909cac640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19041087d0 0x7f19041997a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.886+0000 7f1909cac640 1 --2- 172.21.15.78:0/572174049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1904102dc0 0x7f1904199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.886+0000 7f1909cac640 1 -- 172.21.15.78:0/572174049 >> 172.21.15.78:0/572174049 conn(0x7f19040fe830 msgr2=0x7f1904100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.886+0000 7f1909cac640 1 -- 172.21.15.78:0/572174049 shutdown_connections 2024-09-11T00:56:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:04.887+0000 7f1909cac640 1 -- 172.21.15.78:0/572174049 wait complete. 2024-09-11T00:56:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:05 smithi078 ceph-mon[31315]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:05 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/572174049' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:56:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:05 smithi110 ceph-mon[34712]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:05 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/572174049' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:56:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:05 smithi112 ceph-mon[35384]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:05 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/572174049' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:56:05.722 INFO:tasks.cephadm:Adding mds.a on smithi078 2024-09-11T00:56:05.722 INFO:tasks.cephadm:Adding mds.d on smithi078 2024-09-11T00:56:05.722 INFO:tasks.cephadm:Adding mds.g on smithi078 2024-09-11T00:56:05.722 INFO:tasks.cephadm:Adding mds.j on smithi078 2024-09-11T00:56:05.723 INFO:tasks.cephadm:Adding mds.b on smithi110 2024-09-11T00:56:05.723 INFO:tasks.cephadm:Adding mds.e on smithi110 2024-09-11T00:56:05.723 INFO:tasks.cephadm:Adding mds.h on smithi110 2024-09-11T00:56:05.723 INFO:tasks.cephadm:Adding mds.k on smithi110 2024-09-11T00:56:05.723 INFO:tasks.cephadm:Adding mds.c on smithi112 2024-09-11T00:56:05.723 INFO:tasks.cephadm:Adding mds.f on smithi112 2024-09-11T00:56:05.723 INFO:tasks.cephadm:Adding mds.i on smithi112 2024-09-11T00:56:05.723 INFO:tasks.cephadm:Adding mds.l on smithi112 2024-09-11T00:56:05.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l' 2024-09-11T00:56:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:56:06.417 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:06 smithi112 ceph-mon[35384]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:06.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:06 smithi110 ceph-mon[34712]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:06.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:06 smithi078 ceph-mon[31315]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:06.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.498+0000 7f552ee64640 1 -- 172.21.15.112:0/1234478018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5528075de0 msgr2=0x7f552810ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:06.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.498+0000 7f552ee64640 1 --2- 172.21.15.112:0/1234478018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5528075de0 0x7f552810ee90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5518009900 tx=0x7f551802a410 comp rx=0 tx=0).stop 2024-09-11T00:56:06.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.498+0000 7f552ee64640 1 -- 172.21.15.112:0/1234478018 shutdown_connections 2024-09-11T00:56:06.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.498+0000 7f552ee64640 1 --2- 172.21.15.112:0/1234478018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f552810f3d0 0x7f55281117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:06.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.498+0000 7f552ee64640 1 --2- 172.21.15.112:0/1234478018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5528075de0 0x7f552810ee90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:06.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.498+0000 7f552ee64640 1 --2- 172.21.15.112:0/1234478018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55280754c0 0x7f55280758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:06.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.498+0000 7f552ee64640 1 -- 172.21.15.112:0/1234478018 >> 172.21.15.112:0/1234478018 conn(0x7f55280fe830 msgr2=0x7f5528100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:06.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.498+0000 7f552ee64640 1 -- 172.21.15.112:0/1234478018 shutdown_connections 2024-09-11T00:56:06.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.498+0000 7f552ee64640 1 -- 172.21.15.112:0/1234478018 wait complete. 2024-09-11T00:56:06.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.498+0000 7f552ee64640 1 Processor -- start 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.498+0000 7f552ee64640 1 -- start start 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552ee64640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55280754c0 0x7f552819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552ee64640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528075de0 0x7f552819b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552ee64640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f552810f3d0 0x7f55281a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552ee64640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5528114fb0 con 0x7f5528075de0 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552ee64640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5528114e30 con 0x7f552810f3d0 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552ee64640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5528115130 con 0x7f55280754c0 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552cc00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55280754c0 0x7f552819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552cc00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55280754c0 0x7f552819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46786/0 (socket says 172.21.15.112:46786) 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552cc00640 1 -- 172.21.15.112:0/494727920 learned_addr learned my addr 172.21.15.112:0/494727920 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f551ffff640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528075de0 0x7f552819b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552d401640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f552810f3d0 0x7f55281a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552cc00640 1 -- 172.21.15.112:0/494727920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f552810f3d0 msgr2=0x7f55281a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:06.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552cc00640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f552810f3d0 0x7f55281a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:06.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552cc00640 1 -- 172.21.15.112:0/494727920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528075de0 msgr2=0x7f552819b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:06.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552cc00640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528075de0 0x7f552819b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:06.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552cc00640 1 -- 172.21.15.112:0/494727920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55281a1510 con 0x7f55280754c0 2024-09-11T00:56:06.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552cc00640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55280754c0 0x7f552819b460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f552400d950 tx=0x7f552400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:06.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f551dffb640 1 -- 172.21.15.112:0/494727920 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524017030 con 0x7f55280754c0 2024-09-11T00:56:06.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f552ee64640 1 -- 172.21.15.112:0/494727920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55281a1800 con 0x7f55280754c0 2024-09-11T00:56:06.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f551dffb640 1 -- 172.21.15.112:0/494727920 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5524004020 con 0x7f55280754c0 2024-09-11T00:56:06.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.499+0000 7f551dffb640 1 -- 172.21.15.112:0/494727920 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55240047a0 con 0x7f55280754c0 2024-09-11T00:56:06.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.500+0000 7f552ee64640 1 -- 172.21.15.112:0/494727920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55281a8090 con 0x7f55280754c0 2024-09-11T00:56:06.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.501+0000 7f552ee64640 1 -- 172.21.15.112:0/494727920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55280769d0 con 0x7f55280754c0 2024-09-11T00:56:06.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.502+0000 7f551dffb640 1 -- 172.21.15.112:0/494727920 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f5524009a30 con 0x7f55280754c0 2024-09-11T00:56:06.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.503+0000 7f551dffb640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55100b5330 0x7f55100b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:06.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.503+0000 7f551dffb640 1 -- 172.21.15.112:0/494727920 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f55240e5290 con 0x7f55280754c0 2024-09-11T00:56:06.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.503+0000 7f551ffff640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55100b5330 0x7f55100b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:06.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.503+0000 7f551ffff640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55100b5330 0x7f55100b77f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5518031440 tx=0x7f5518005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:06.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.507+0000 7f551dffb640 1 -- 172.21.15.112:0/494727920 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f55240ac520 con 0x7f55280754c0 2024-09-11T00:56:06.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.690+0000 7f552ee64640 1 -- 172.21.15.112:0/494727920 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l", "target": ["mon-mgr", ""]}) -- 0x7f5528100540 con 0x7f55100b5330 2024-09-11T00:56:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.699+0000 7f551dffb640 1 -- 172.21.15.112:0/494727920 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f5528100540 con 0x7f55100b5330 2024-09-11T00:56:06.702 INFO:teuthology.orchestra.run.smithi112.stdout:Scheduled mds.all update... 2024-09-11T00:56:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.701+0000 7f552ee64640 1 -- 172.21.15.112:0/494727920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55100b5330 msgr2=0x7f55100b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.701+0000 7f552ee64640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55100b5330 0x7f55100b77f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5518031440 tx=0x7f5518005b50 comp rx=0 tx=0).stop 2024-09-11T00:56:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.701+0000 7f552ee64640 1 -- 172.21.15.112:0/494727920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55280754c0 msgr2=0x7f552819b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.701+0000 7f552ee64640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55280754c0 0x7f552819b460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f552400d950 tx=0x7f552400de10 comp rx=0 tx=0).stop 2024-09-11T00:56:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.701+0000 7f552ee64640 1 -- 172.21.15.112:0/494727920 shutdown_connections 2024-09-11T00:56:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.701+0000 7f552ee64640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55100b5330 0x7f55100b77f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.701+0000 7f552ee64640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f552810f3d0 0x7f55281a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.701+0000 7f552ee64640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528075de0 0x7f552819b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.701+0000 7f552ee64640 1 --2- 172.21.15.112:0/494727920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55280754c0 0x7f552819b460 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.701+0000 7f552ee64640 1 -- 172.21.15.112:0/494727920 >> 172.21.15.112:0/494727920 conn(0x7f55280fe830 msgr2=0x7f5528111580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.701+0000 7f552ee64640 1 -- 172.21.15.112:0/494727920 shutdown_connections 2024-09-11T00:56:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-11T00:56:06.701+0000 7f552ee64640 1 -- 172.21.15.112:0/494727920 wait complete. 2024-09-11T00:56:07.399 DEBUG:teuthology.orchestra.run.smithi078:mds.a> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.a.service 2024-09-11T00:56:07.403 DEBUG:teuthology.orchestra.run.smithi078:mds.d> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.d.service 2024-09-11T00:56:07.446 DEBUG:teuthology.orchestra.run.smithi078:mds.g> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.g.service 2024-09-11T00:56:07.489 DEBUG:teuthology.orchestra.run.smithi078:mds.j> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.j.service 2024-09-11T00:56:07.532 DEBUG:teuthology.orchestra.run.smithi110:mds.b> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.b.service 2024-09-11T00:56:07.536 DEBUG:teuthology.orchestra.run.smithi110:mds.e> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.e.service 2024-09-11T00:56:07.578 DEBUG:teuthology.orchestra.run.smithi110:mds.h> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.h.service 2024-09-11T00:56:07.623 DEBUG:teuthology.orchestra.run.smithi110:mds.k> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.k.service 2024-09-11T00:56:07.667 DEBUG:teuthology.orchestra.run.smithi112:mds.c> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.c.service 2024-09-11T00:56:07.670 DEBUG:teuthology.orchestra.run.smithi112:mds.f> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.f.service 2024-09-11T00:56:07.712 DEBUG:teuthology.orchestra.run.smithi112:mds.i> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.i.service 2024-09-11T00:56:07.755 DEBUG:teuthology.orchestra.run.smithi112:mds.l> sudo journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.l.service 2024-09-11T00:56:07.798 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-11T00:56:07.799 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:56:07.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:07 smithi110 ceph-mon[34712]: from='client.24517 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:56:07.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:07 smithi110 ceph-mon[34712]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l;count:12 2024-09-11T00:56:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:56:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:56:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:07 smithi110 ceph-mon[34712]: Deploying daemon mds.l on smithi112 2024-09-11T00:56:07.959 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:07 smithi112 ceph-mon[35384]: from='client.24517 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:56:07.959 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:07 smithi112 ceph-mon[35384]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l;count:12 2024-09-11T00:56:07.959 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:07.959 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:56:07.959 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:07.960 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:56:07.960 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:07.960 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:07.960 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:07.960 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:07.960 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:07 smithi112 ceph-mon[35384]: Deploying daemon mds.l on smithi112 2024-09-11T00:56:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:07 smithi078 ceph-mon[31315]: from='client.24517 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:56:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:07 smithi078 ceph-mon[31315]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l;count:12 2024-09-11T00:56:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:56:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:56:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:07 smithi078 ceph-mon[31315]: Deploying daemon mds.l on smithi112 2024-09-11T00:56:08.043 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.523+0000 7fc36b779640 1 -- 172.21.15.78:0/1017585220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3641076e0 msgr2=0x7fc364107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 --2- 172.21.15.78:0/1017585220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3641076e0 0x7fc364107b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc358009900 tx=0x7fc35802a410 comp rx=0 tx=0).stop 2024-09-11T00:56:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 -- 172.21.15.78:0/1017585220 shutdown_connections 2024-09-11T00:56:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 --2- 172.21.15.78:0/1017585220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc364108060 0x7fc36410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 --2- 172.21.15.78:0/1017585220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3641076e0 0x7fc364107b20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 --2- 172.21.15.78:0/1017585220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc364101cd0 0x7fc3641020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 -- 172.21.15.78:0/1017585220 >> 172.21.15.78:0/1017585220 conn(0x7fc3640fd700 msgr2=0x7fc3640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 -- 172.21.15.78:0/1017585220 shutdown_connections 2024-09-11T00:56:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 -- 172.21.15.78:0/1017585220 wait complete. 2024-09-11T00:56:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 Processor -- start 2024-09-11T00:56:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 -- start start 2024-09-11T00:56:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364101cd0 0x7fc3640fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3641076e0 0x7fc3640fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.524+0000 7fc36b779640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc364108060 0x7fc3640fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc36b779640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc364111d20 con 0x7fc364101cd0 2024-09-11T00:56:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc36b779640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc364111ba0 con 0x7fc364108060 2024-09-11T00:56:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc36b779640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc364111ea0 con 0x7fc3641076e0 2024-09-11T00:56:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc369515640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364101cd0 0x7fc3640fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc369515640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364101cd0 0x7fc3640fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57154/0 (socket says 172.21.15.78:57154) 2024-09-11T00:56:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc369515640 1 -- 172.21.15.78:0/2616828810 learned_addr learned my addr 172.21.15.78:0/2616828810 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc368d14640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3641076e0 0x7fc3640fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc369d16640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc364108060 0x7fc3640fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc369515640 1 -- 172.21.15.78:0/2616828810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3641076e0 msgr2=0x7fc3640fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc369515640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3641076e0 0x7fc3640fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc369515640 1 -- 172.21.15.78:0/2616828810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc364108060 msgr2=0x7fc3640fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc369515640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc364108060 0x7fc3640fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc369515640 1 -- 172.21.15.78:0/2616828810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3641a77c0 con 0x7fc364101cd0 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc369d16640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc364108060 0x7fc3640fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc369515640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364101cd0 0x7fc3640fdc00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00c890 tx=0x7fc35c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc3627fc640 1 -- 172.21.15.78:0/2616828810 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c004220 con 0x7fc364101cd0 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc36b779640 1 -- 172.21.15.78:0/2616828810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3641a7ab0 con 0x7fc364101cd0 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc3627fc640 1 -- 172.21.15.78:0/2616828810 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc35c0043c0 con 0x7fc364101cd0 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc3627fc640 1 -- 172.21.15.78:0/2616828810 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c004a00 con 0x7fc364101cd0 2024-09-11T00:56:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.525+0000 7fc36b779640 1 -- 172.21.15.78:0/2616828810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3641a7fc0 con 0x7fc364101cd0 2024-09-11T00:56:08.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.526+0000 7fc36b779640 1 -- 172.21.15.78:0/2616828810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3641020b0 con 0x7fc364101cd0 2024-09-11T00:56:08.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.528+0000 7fc3627fc640 1 -- 172.21.15.78:0/2616828810 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fc35c0079b0 con 0x7fc364101cd0 2024-09-11T00:56:08.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.529+0000 7fc3627fc640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc34c0b5330 0x7fc34c0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:08.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.529+0000 7fc368d14640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc34c0b5330 0x7fc34c0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:08.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.529+0000 7fc3627fc640 1 -- 172.21.15.78:0/2616828810 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fc35c0e59a0 con 0x7fc364101cd0 2024-09-11T00:56:08.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.529+0000 7fc368d14640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc34c0b5330 0x7fc34c0b77f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc358031440 tx=0x7fc358005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:08.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.532+0000 7fc3627fc640 1 -- 172.21.15.78:0/2616828810 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc35c014030 con 0x7fc364101cd0 2024-09-11T00:56:08.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.745+0000 7fc36b779640 1 -- 172.21.15.78:0/2616828810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3640fef00 con 0x7fc364101cd0 2024-09-11T00:56:08.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.745+0000 7fc3627fc640 1 -- 172.21.15.78:0/2616828810 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fc35c0acc30 con 0x7fc364101cd0 2024-09-11T00:56:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:08.747 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"btime":"2024-09-11T00:48:27:650665+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-11T00:56:08.748 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 1 2024-09-11T00:56:08.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.747+0000 7fc36b779640 1 -- 172.21.15.78:0/2616828810 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc34c0b5330 msgr2=0x7fc34c0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:08.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.747+0000 7fc36b779640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc34c0b5330 0x7fc34c0b77f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc358031440 tx=0x7fc358005b50 comp rx=0 tx=0).stop 2024-09-11T00:56:08.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.747+0000 7fc36b779640 1 -- 172.21.15.78:0/2616828810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364101cd0 msgr2=0x7fc3640fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:08.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.747+0000 7fc36b779640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364101cd0 0x7fc3640fdc00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00c890 tx=0x7fc35c00cd50 comp rx=0 tx=0).stop 2024-09-11T00:56:08.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.747+0000 7fc36b779640 1 -- 172.21.15.78:0/2616828810 shutdown_connections 2024-09-11T00:56:08.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.747+0000 7fc36b779640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc34c0b5330 0x7fc34c0b77f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:08.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.747+0000 7fc36b779640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc364108060 0x7fc3640fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:08.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.747+0000 7fc36b779640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3641076e0 0x7fc3640fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:08.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.748+0000 7fc36b779640 1 --2- 172.21.15.78:0/2616828810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364101cd0 0x7fc3640fdc00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:08.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.748+0000 7fc36b779640 1 -- 172.21.15.78:0/2616828810 >> 172.21.15.78:0/2616828810 conn(0x7fc3640fd700 msgr2=0x7fc36406b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:08.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.748+0000 7fc36b779640 1 -- 172.21.15.78:0/2616828810 shutdown_connections 2024-09-11T00:56:08.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:08.748+0000 7fc36b779640 1 -- 172.21.15.78:0/2616828810 wait complete. 2024-09-11T00:56:08.924 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:08 smithi112 ceph-mon[35384]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:08.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:08 smithi078 ceph-mon[31315]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:09.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:08 smithi110 ceph-mon[34712]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:09.224 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 00:56:09 smithi112 systemd[1]: Starting Ceph mds.l for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:56:09.578 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 00:56:09 smithi112 podman[65860]: 2024-09-11 00:56:09.212811512 +0000 UTC m=+0.015664430 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:56:09.578 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 00:56:09 smithi112 podman[65860]: 2024-09-11 00:56:09.332204615 +0000 UTC m=+0.135057526 container create 04aca3bc67a0e69d361b6fc7ce38d810234368f354c7220399bf97ade30eebfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T00:56:09.578 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 00:56:09 smithi112 podman[65860]: 2024-09-11 00:56:09.576614331 +0000 UTC m=+0.379467254 container init 04aca3bc67a0e69d361b6fc7ce38d810234368f354c7220399bf97ade30eebfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-11T00:56:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:09 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2616828810' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:09 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.112:6832/1732756287,v1:172.21.15.112:6833/1732756287] up:boot 2024-09-11T00:56:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:09 smithi112 ceph-mon[35384]: fsmap 1 up:standby 2024-09-11T00:56:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T00:56:09.974 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 00:56:09 smithi112 podman[65860]: 2024-09-11 00:56:09.579476443 +0000 UTC m=+0.382329363 container start 04aca3bc67a0e69d361b6fc7ce38d810234368f354c7220399bf97ade30eebfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T00:56:09.974 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 00:56:09 smithi112 ceph-mds[65879]: -- 172.21.15.112:0/3968375412 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c9d849860 con 0x556c9d827400 2024-09-11T00:56:09.974 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 00:56:09 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l[65875]: starting mds.l at 2024-09-11T00:56:09.975 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 00:56:09 smithi112 bash[65860]: 04aca3bc67a0e69d361b6fc7ce38d810234368f354c7220399bf97ade30eebfe 2024-09-11T00:56:09.975 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 00:56:09 smithi112 systemd[1]: Started Ceph mds.l for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:56:09.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:09 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2616828810' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:09.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:09 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.112:6832/1732756287,v1:172.21.15.112:6833/1732756287] up:boot 2024-09-11T00:56:09.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:09 smithi078 ceph-mon[31315]: fsmap 1 up:standby 2024-09-11T00:56:09.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T00:56:10.168 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:09 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2616828810' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:10.168 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:09 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.112:6832/1732756287,v1:172.21.15.112:6833/1732756287] up:boot 2024-09-11T00:56:10.168 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:09 smithi110 ceph-mon[34712]: fsmap 1 up:standby 2024-09-11T00:56:10.168 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T00:56:10.854 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:10.854 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:10.854 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:10.854 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:10.855 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:10.855 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:10.855 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:10 smithi078 ceph-mon[31315]: Deploying daemon mds.k on smithi110 2024-09-11T00:56:10.855 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:10 smithi078 ceph-mon[31315]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:10.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:10.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:10.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:10.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:10.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:10.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:10.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:10 smithi110 ceph-mon[34712]: Deploying daemon mds.k on smithi110 2024-09-11T00:56:10.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:10 smithi110 ceph-mon[34712]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:10 smithi112 ceph-mon[35384]: Deploying daemon mds.k on smithi110 2024-09-11T00:56:11.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:10 smithi112 ceph-mon[35384]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:11.586 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:56:11.843 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:12.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.370+0000 7f03d6395640 1 -- 172.21.15.78:0/2201161116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d0103ca0 msgr2=0x7f03d0104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:12.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.370+0000 7f03d6395640 1 --2- 172.21.15.78:0/2201161116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d0103ca0 0x7f03d0104080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009900 tx=0x7f03c402a410 comp rx=0 tx=0).stop 2024-09-11T00:56:12.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.370+0000 7f03d6395640 1 -- 172.21.15.78:0/2201161116 shutdown_connections 2024-09-11T00:56:12.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.370+0000 7f03d6395640 1 --2- 172.21.15.78:0/2201161116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03d010d100 0x7f03d010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.370+0000 7f03d6395640 1 --2- 172.21.15.78:0/2201161116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03d01045c0 0x7f03d010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.370+0000 7f03d6395640 1 --2- 172.21.15.78:0/2201161116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d0103ca0 0x7f03d0104080 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.370+0000 7f03d6395640 1 -- 172.21.15.78:0/2201161116 >> 172.21.15.78:0/2201161116 conn(0x7f03d00fd700 msgr2=0x7f03d00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.370+0000 7f03d6395640 1 -- 172.21.15.78:0/2201161116 shutdown_connections 2024-09-11T00:56:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.370+0000 7f03d6395640 1 -- 172.21.15.78:0/2201161116 wait complete. 2024-09-11T00:56:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.371+0000 7f03d6395640 1 Processor -- start 2024-09-11T00:56:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.371+0000 7f03d6395640 1 -- start start 2024-09-11T00:56:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.371+0000 7f03d6395640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03d0103ca0 0x7f03d0199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.371+0000 7f03d6395640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03d01045c0 0x7f03d0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.371+0000 7f03d6395640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d010d100 0x7f03d019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.371+0000 7f03d6395640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03d0112d70 con 0x7f03d010d100 2024-09-11T00:56:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.371+0000 7f03d6395640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03d0112bf0 con 0x7f03d0103ca0 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.371+0000 7f03d6395640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03d0112ef0 con 0x7f03d01045c0 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.371+0000 7f03d4932640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d010d100 0x7f03d019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.371+0000 7f03d4932640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d010d100 0x7f03d019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57174/0 (socket says 172.21.15.78:57174) 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.371+0000 7f03d4932640 1 -- 172.21.15.78:0/4228656552 learned_addr learned my addr 172.21.15.78:0/4228656552 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.371+0000 7f03cffff640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03d0103ca0 0x7f03d0199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.372+0000 7f03cf7fe640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03d01045c0 0x7f03d0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.372+0000 7f03d4932640 1 -- 172.21.15.78:0/4228656552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03d01045c0 msgr2=0x7f03d0199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.372+0000 7f03d4932640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03d01045c0 0x7f03d0199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.372+0000 7f03d4932640 1 -- 172.21.15.78:0/4228656552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03d0103ca0 msgr2=0x7f03d0199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.372+0000 7f03d4932640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03d0103ca0 0x7f03d0199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.372+0000 7f03d4932640 1 -- 172.21.15.78:0/4228656552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d019f2b0 con 0x7f03d010d100 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.372+0000 7f03d4932640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d010d100 0x7f03d019eae0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f03c800cce0 tx=0x7f03c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:12.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.372+0000 7f03cd7fa640 1 -- 172.21.15.78:0/4228656552 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c800f450 con 0x7f03d010d100 2024-09-11T00:56:12.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.372+0000 7f03d6395640 1 -- 172.21.15.78:0/4228656552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d019f5a0 con 0x7f03d010d100 2024-09-11T00:56:12.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.372+0000 7f03cd7fa640 1 -- 172.21.15.78:0/4228656552 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c800faa0 con 0x7f03d010d100 2024-09-11T00:56:12.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.372+0000 7f03cd7fa640 1 -- 172.21.15.78:0/4228656552 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8018b70 con 0x7f03d010d100 2024-09-11T00:56:12.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.372+0000 7f03d6395640 1 -- 172.21.15.78:0/4228656552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d01a5c60 con 0x7f03d010d100 2024-09-11T00:56:12.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.373+0000 7f03d6395640 1 -- 172.21.15.78:0/4228656552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d0104080 con 0x7f03d010d100 2024-09-11T00:56:12.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.375+0000 7f03cd7fa640 1 -- 172.21.15.78:0/4228656552 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f03c800f5f0 con 0x7f03d010d100 2024-09-11T00:56:12.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.376+0000 7f03cd7fa640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f03b80b5330 0x7f03b80b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:12.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.376+0000 7f03cffff640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f03b80b5330 0x7f03b80b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:12.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.376+0000 7f03cd7fa640 1 -- 172.21.15.78:0/4228656552 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f03c80e6050 con 0x7f03d010d100 2024-09-11T00:56:12.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.376+0000 7f03cffff640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f03b80b5330 0x7f03b80b77f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03c4031440 tx=0x7f03c40095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:12.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.380+0000 7f03cd7fa640 1 -- 172.21.15.78:0/4228656552 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f03c8014030 con 0x7f03d010d100 2024-09-11T00:56:12.440 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 00:56:12 smithi110 systemd[1]: Started Ceph mds.k for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:56:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.602+0000 7f03d6395640 1 -- 172.21.15.78:0/4228656552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03d01a5fd0 con 0x7f03d010d100 2024-09-11T00:56:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.603+0000 7f03cd7fa640 1 -- 172.21.15.78:0/4228656552 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f03c80ad2e0 con 0x7f03d010d100 2024-09-11T00:56:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:12.606 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":2,"btime":"2024-09-11T00:56:09:608075+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24529,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T00:56:12.606 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 2 2024-09-11T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.605+0000 7f03d6395640 1 -- 172.21.15.78:0/4228656552 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f03b80b5330 msgr2=0x7f03b80b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.605+0000 7f03d6395640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f03b80b5330 0x7f03b80b77f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03c4031440 tx=0x7f03c40095f0 comp rx=0 tx=0).stop 2024-09-11T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.605+0000 7f03d6395640 1 -- 172.21.15.78:0/4228656552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d010d100 msgr2=0x7f03d019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.605+0000 7f03d6395640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d010d100 0x7f03d019eae0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f03c800cce0 tx=0x7f03c800a620 comp rx=0 tx=0).stop 2024-09-11T00:56:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.605+0000 7f03d6395640 1 -- 172.21.15.78:0/4228656552 shutdown_connections 2024-09-11T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.605+0000 7f03d6395640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f03b80b5330 0x7f03b80b77f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.605+0000 7f03d6395640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d010d100 0x7f03d019eae0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.605+0000 7f03d6395640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03d01045c0 0x7f03d0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.605+0000 7f03d6395640 1 --2- 172.21.15.78:0/4228656552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03d0103ca0 0x7f03d0199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.606+0000 7f03d6395640 1 -- 172.21.15.78:0/4228656552 >> 172.21.15.78:0/4228656552 conn(0x7f03d00fd700 msgr2=0x7f03d010f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.606+0000 7f03d6395640 1 -- 172.21.15.78:0/4228656552 shutdown_connections 2024-09-11T00:56:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:12.606+0000 7f03d6395640 1 -- 172.21.15.78:0/4228656552 wait complete. 2024-09-11T00:56:13.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:13 smithi078 ceph-mon[31315]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:13.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:13.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:13.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:13 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4228656552' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:13 smithi110 ceph-mon[34712]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:13 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/4228656552' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:13 smithi112 ceph-mon[35384]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:13 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/4228656552' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:14.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:14 smithi078 ceph-mon[31315]: Deploying daemon mds.j on smithi078 2024-09-11T00:56:14.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:14 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.110:6832/2579452587,v1:172.21.15.110:6833/2579452587] up:boot 2024-09-11T00:56:14.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:14 smithi078 ceph-mon[31315]: fsmap 2 up:standby 2024-09-11T00:56:14.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:14 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T00:56:14.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:14 smithi110 ceph-mon[34712]: Deploying daemon mds.j on smithi078 2024-09-11T00:56:14.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:14 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.110:6832/2579452587,v1:172.21.15.110:6833/2579452587] up:boot 2024-09-11T00:56:14.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:14 smithi110 ceph-mon[34712]: fsmap 2 up:standby 2024-09-11T00:56:14.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:14 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T00:56:14.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:14 smithi112 ceph-mon[35384]: Deploying daemon mds.j on smithi078 2024-09-11T00:56:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:14 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.110:6832/2579452587,v1:172.21.15.110:6833/2579452587] up:boot 2024-09-11T00:56:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:14 smithi112 ceph-mon[35384]: fsmap 2 up:standby 2024-09-11T00:56:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:14 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T00:56:15.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:15 smithi110 ceph-mon[34712]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:15.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:15 smithi112 ceph-mon[35384]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:15.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:15 smithi078 ceph-mon[31315]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:15.497 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 00:56:15 smithi078 systemd[1]: Starting Ceph mds.j for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:56:15.768 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 00:56:15 smithi078 podman[73133]: 2024-09-11 00:56:15.400315805 +0000 UTC m=+0.015824613 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:56:15.768 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 00:56:15 smithi078 podman[73133]: 2024-09-11 00:56:15.514457801 +0000 UTC m=+0.129966599 container create a0258905a31c9195b4f3ec31502597ec8a7677f1260fd84963517494e7f0f541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-j, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-09-11T00:56:15.768 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 00:56:15 smithi078 podman[73133]: 2024-09-11 00:56:15.767285452 +0000 UTC m=+0.382794245 container init a0258905a31c9195b4f3ec31502597ec8a7677f1260fd84963517494e7f0f541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True) 2024-09-11T00:56:16.243 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:56:16.247 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 00:56:15 smithi078 podman[73133]: 2024-09-11 00:56:15.770224744 +0000 UTC m=+0.385733542 container start a0258905a31c9195b4f3ec31502597ec8a7677f1260fd84963517494e7f0f541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T00:56:16.247 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 00:56:15 smithi078 ceph-mds[73153]: -- 172.21.15.78:0/3173198600 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5631102a7860 con 0x563110284800 2024-09-11T00:56:16.247 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 00:56:15 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-j[73149]: starting mds.j at 2024-09-11T00:56:16.247 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 00:56:15 smithi078 bash[73133]: a0258905a31c9195b4f3ec31502597ec8a7677f1260fd84963517494e7f0f541 2024-09-11T00:56:16.247 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 00:56:15 smithi078 systemd[1]: Started Ceph mds.j for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:56:16.934 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:17.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:17.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:17.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:17.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:17.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:17.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:17.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:16 smithi110 ceph-mon[34712]: Deploying daemon mds.d on smithi078 2024-09-11T00:56:17.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:16 smithi110 ceph-mon[34712]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:17.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:16 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.78:6834/494285934,v1:172.21.15.78:6835/494285934] up:boot 2024-09-11T00:56:17.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:16 smithi110 ceph-mon[34712]: fsmap 3 up:standby 2024-09-11T00:56:17.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-11T00:56:17.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:17.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:17.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:17.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:17.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:17.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:17.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:16 smithi112 ceph-mon[35384]: Deploying daemon mds.d on smithi078 2024-09-11T00:56:17.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:16 smithi112 ceph-mon[35384]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:17.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:16 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.78:6834/494285934,v1:172.21.15.78:6835/494285934] up:boot 2024-09-11T00:56:17.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:16 smithi112 ceph-mon[35384]: fsmap 3 up:standby 2024-09-11T00:56:17.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-11T00:56:17.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:17.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:17.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:17.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:17.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:17.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:17.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:16 smithi078 ceph-mon[31315]: Deploying daemon mds.d on smithi078 2024-09-11T00:56:17.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:16 smithi078 ceph-mon[31315]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:17.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:16 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.78:6834/494285934,v1:172.21.15.78:6835/494285934] up:boot 2024-09-11T00:56:17.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:16 smithi078 ceph-mon[31315]: fsmap 3 up:standby 2024-09-11T00:56:17.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-11T00:56:18.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.186+0000 7f8d213d1640 1 -- 172.21.15.78:0/3078492151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c10d100 msgr2=0x7f8d1c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:18.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.186+0000 7f8d213d1640 1 --2- 172.21.15.78:0/3078492151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c10d100 0x7f8d1c10f550 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009ca0 tx=0x7f8d1402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:56:18.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.186+0000 7f8d213d1640 1 -- 172.21.15.78:0/3078492151 shutdown_connections 2024-09-11T00:56:18.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.186+0000 7f8d213d1640 1 --2- 172.21.15.78:0/3078492151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c10d100 0x7f8d1c10f550 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:18.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.186+0000 7f8d213d1640 1 --2- 172.21.15.78:0/3078492151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d1c104550 0x7f8d1c10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:18.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.186+0000 7f8d213d1640 1 --2- 172.21.15.78:0/3078492151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d1c103b50 0x7f8d1c103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:18.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.186+0000 7f8d213d1640 1 -- 172.21.15.78:0/3078492151 >> 172.21.15.78:0/3078492151 conn(0x7f8d1c0fd700 msgr2=0x7f8d1c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:18.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.186+0000 7f8d213d1640 1 -- 172.21.15.78:0/3078492151 shutdown_connections 2024-09-11T00:56:18.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.186+0000 7f8d213d1640 1 -- 172.21.15.78:0/3078492151 wait complete. 2024-09-11T00:56:18.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.186+0000 7f8d213d1640 1 Processor -- start 2024-09-11T00:56:18.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.186+0000 7f8d213d1640 1 -- start start 2024-09-11T00:56:18.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d213d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d1c103b50 0x7f8d1c1019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:18.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d213d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c104550 0x7f8d1c101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:18.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d213d1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d1c10d100 0x7f8d1c102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:18.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d213d1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c112da0 con 0x7f8d1c104550 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d213d1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c112c20 con 0x7f8d1c10d100 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d213d1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c112f20 con 0x7f8d1c103b50 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d1a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c104550 0x7f8d1c101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d1a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c104550 0x7f8d1c101f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45698/0 (socket says 172.21.15.78:45698) 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d1a7fc640 1 -- 172.21.15.78:0/257752234 learned_addr learned my addr 172.21.15.78:0/257752234 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d1affd640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d1c103b50 0x7f8d1c1019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d1b7fe640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d1c10d100 0x7f8d1c102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d1a7fc640 1 -- 172.21.15.78:0/257752234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d1c103b50 msgr2=0x7f8d1c1019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d1a7fc640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d1c103b50 0x7f8d1c1019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d1a7fc640 1 -- 172.21.15.78:0/257752234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d1c10d100 msgr2=0x7f8d1c102450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d1a7fc640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d1c10d100 0x7f8d1c102450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d1a7fc640 1 -- 172.21.15.78:0/257752234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c1a75f0 con 0x7f8d1c104550 2024-09-11T00:56:18.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d1b7fe640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d1c10d100 0x7f8d1c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d1a7fc640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c104550 0x7f8d1c101f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000b4a0 tx=0x7f8d1000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d03fff640 1 -- 172.21.15.78:0/257752234 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10015020 con 0x7f8d1c104550 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d213d1640 1 -- 172.21.15.78:0/257752234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1c1a78e0 con 0x7f8d1c104550 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d03fff640 1 -- 172.21.15.78:0/257752234 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d10011ce0 con 0x7f8d1c104550 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.187+0000 7f8d03fff640 1 -- 172.21.15.78:0/257752234 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10010dc0 con 0x7f8d1c104550 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.188+0000 7f8d213d1640 1 -- 172.21.15.78:0/257752234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1c1a7ef0 con 0x7f8d1c104550 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.188+0000 7f8d213d1640 1 -- 172.21.15.78:0/257752234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d1c103f30 con 0x7f8d1c104550 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.190+0000 7f8d03fff640 1 -- 172.21.15.78:0/257752234 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f8d10011830 con 0x7f8d1c104550 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.190+0000 7f8d03fff640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d040b5330 0x7f8d040b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.191+0000 7f8d1affd640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d040b5330 0x7f8d040b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.191+0000 7f8d03fff640 1 -- 172.21.15.78:0/257752234 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f8d100e5840 con 0x7f8d1c104550 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.191+0000 7f8d1affd640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d040b5330 0x7f8d040b77f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00abe0 tx=0x7f8d0c00a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:18.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.193+0000 7f8d03fff640 1 -- 172.21.15.78:0/257752234 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8d1001f020 con 0x7f8d1c104550 2024-09-11T00:56:18.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.418+0000 7f8d213d1640 1 -- 172.21.15.78:0/257752234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d1c102cd0 con 0x7f8d1c104550 2024-09-11T00:56:18.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.419+0000 7f8d03fff640 1 -- 172.21.15.78:0/257752234 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f8d1c102cd0 con 0x7f8d1c104550 2024-09-11T00:56:18.422 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":4,"btime":"2024-09-11T00:56:16:073083+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24529,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T00:56:18.422 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 4 2024-09-11T00:56:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.421+0000 7f8d213d1640 1 -- 172.21.15.78:0/257752234 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d040b5330 msgr2=0x7f8d040b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.421+0000 7f8d213d1640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d040b5330 0x7f8d040b77f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00abe0 tx=0x7f8d0c00a2d0 comp rx=0 tx=0).stop 2024-09-11T00:56:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.421+0000 7f8d213d1640 1 -- 172.21.15.78:0/257752234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c104550 msgr2=0x7f8d1c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.421+0000 7f8d213d1640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c104550 0x7f8d1c101f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000b4a0 tx=0x7f8d1000b8a0 comp rx=0 tx=0).stop 2024-09-11T00:56:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.421+0000 7f8d213d1640 1 -- 172.21.15.78:0/257752234 shutdown_connections 2024-09-11T00:56:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.421+0000 7f8d213d1640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d040b5330 0x7f8d040b77f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.421+0000 7f8d213d1640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d1c10d100 0x7f8d1c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.421+0000 7f8d213d1640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c104550 0x7f8d1c101f10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.421+0000 7f8d213d1640 1 --2- 172.21.15.78:0/257752234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d1c103b50 0x7f8d1c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.421+0000 7f8d213d1640 1 -- 172.21.15.78:0/257752234 >> 172.21.15.78:0/257752234 conn(0x7f8d1c0fd700 msgr2=0x7f8d1c10f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.421+0000 7f8d213d1640 1 -- 172.21.15.78:0/257752234 shutdown_connections 2024-09-11T00:56:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:18.421+0000 7f8d213d1640 1 -- 172.21.15.78:0/257752234 wait complete. 2024-09-11T00:56:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:19 smithi078 ceph-mon[31315]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:19 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/257752234' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:19 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:19 smithi110 ceph-mon[34712]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:19 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/257752234' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:19.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:19.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:19 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:19 smithi112 ceph-mon[35384]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:19 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/257752234' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:19.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:19 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:20.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:20 smithi110 ceph-mon[34712]: Deploying daemon mds.e on smithi110 2024-09-11T00:56:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:20 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.78:6836/1108876555,v1:172.21.15.78:6837/1108876555] up:boot 2024-09-11T00:56:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:20 smithi110 ceph-mon[34712]: fsmap 4 up:standby 2024-09-11T00:56:20.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:20 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-11T00:56:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:20 smithi112 ceph-mon[35384]: Deploying daemon mds.e on smithi110 2024-09-11T00:56:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:20 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.78:6836/1108876555,v1:172.21.15.78:6837/1108876555] up:boot 2024-09-11T00:56:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:20 smithi112 ceph-mon[35384]: fsmap 4 up:standby 2024-09-11T00:56:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:20 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-11T00:56:20.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:20 smithi078 ceph-mon[31315]: Deploying daemon mds.e on smithi110 2024-09-11T00:56:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:20 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.78:6836/1108876555,v1:172.21.15.78:6837/1108876555] up:boot 2024-09-11T00:56:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:20 smithi078 ceph-mon[31315]: fsmap 4 up:standby 2024-09-11T00:56:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:20 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-11T00:56:21.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:21 smithi110 ceph-mon[34712]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:21.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:56:21.458 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:21 smithi078 ceph-mon[31315]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:21.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:21 smithi112 ceph-mon[35384]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:21.689 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:21.940 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 00:56:21 smithi110 systemd[1]: Started Ceph mds.e for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:56:22.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.180+0000 7f669fc78640 1 -- 172.21.15.78:0/1125360778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6698102dc0 msgr2=0x7f66981031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.180+0000 7f669fc78640 1 --2- 172.21.15.78:0/1125360778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6698102dc0 0x7f66981031a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6688009900 tx=0x7f668802a410 comp rx=0 tx=0).stop 2024-09-11T00:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.180+0000 7f669fc78640 1 -- 172.21.15.78:0/1125360778 shutdown_connections 2024-09-11T00:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.180+0000 7f669fc78640 1 --2- 172.21.15.78:0/1125360778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6698109150 0x7f669810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.180+0000 7f669fc78640 1 --2- 172.21.15.78:0/1125360778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66981087d0 0x7f6698108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.180+0000 7f669fc78640 1 --2- 172.21.15.78:0/1125360778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6698102dc0 0x7f66981031a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.181+0000 7f669fc78640 1 -- 172.21.15.78:0/1125360778 >> 172.21.15.78:0/1125360778 conn(0x7f66980fe830 msgr2=0x7f6698100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.181+0000 7f669fc78640 1 -- 172.21.15.78:0/1125360778 shutdown_connections 2024-09-11T00:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.181+0000 7f669fc78640 1 -- 172.21.15.78:0/1125360778 wait complete. 2024-09-11T00:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.181+0000 7f669fc78640 1 Processor -- start 2024-09-11T00:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.181+0000 7f669fc78640 1 -- start start 2024-09-11T00:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.181+0000 7f669fc78640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6698102dc0 0x7f6698199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.181+0000 7f669fc78640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66981087d0 0x7f6698199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.181+0000 7f669fc78640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6698109150 0x7f669819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.181+0000 7f669fc78640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6698112e10 con 0x7f6698109150 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.181+0000 7f669fc78640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6698112c90 con 0x7f6698102dc0 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.181+0000 7f669fc78640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6698112f90 con 0x7f66981087d0 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669e215640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6698109150 0x7f669819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669da14640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6698102dc0 0x7f6698199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669e215640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6698109150 0x7f669819ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45724/0 (socket says 172.21.15.78:45724) 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669e215640 1 -- 172.21.15.78:0/4278140324 learned_addr learned my addr 172.21.15.78:0/4278140324 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669d213640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66981087d0 0x7f6698199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669e215640 1 -- 172.21.15.78:0/4278140324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66981087d0 msgr2=0x7f6698199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669e215640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66981087d0 0x7f6698199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669e215640 1 -- 172.21.15.78:0/4278140324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6698102dc0 msgr2=0x7f6698199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669e215640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6698102dc0 0x7f6698199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669e215640 1 -- 172.21.15.78:0/4278140324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669819f3f0 con 0x7f6698109150 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669da14640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6698102dc0 0x7f6698199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669e215640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6698109150 0x7f669819ec20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f669000b970 tx=0x7f669000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f668effd640 1 -- 172.21.15.78:0/4278140324 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690019070 con 0x7f6698109150 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669fc78640 1 -- 172.21.15.78:0/4278140324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669819f6e0 con 0x7f6698109150 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f668effd640 1 -- 172.21.15.78:0/4278140324 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6690014440 con 0x7f6698109150 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f668effd640 1 -- 172.21.15.78:0/4278140324 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66900075d0 con 0x7f6698109150 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.182+0000 7f669fc78640 1 -- 172.21.15.78:0/4278140324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66981a5e20 con 0x7f6698109150 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.183+0000 7f669fc78640 1 -- 172.21.15.78:0/4278140324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66981031a0 con 0x7f6698109150 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.185+0000 7f668effd640 1 -- 172.21.15.78:0/4278140324 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f669000cb40 con 0x7f6698109150 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.185+0000 7f668effd640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f66840b5330 0x7f66840b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.185+0000 7f669da14640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f66840b5330 0x7f66840b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.185+0000 7f668effd640 1 -- 172.21.15.78:0/4278140324 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f66900e50a0 con 0x7f6698109150 2024-09-11T00:56:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.186+0000 7f669da14640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f66840b5330 0x7f66840b77f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6688031440 tx=0x7f6688005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:22.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.189+0000 7f668effd640 1 -- 172.21.15.78:0/4278140324 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f66900abc40 con 0x7f6698109150 2024-09-11T00:56:22.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.399+0000 7f669fc78640 1 -- 172.21.15.78:0/4278140324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f669802e5c0 con 0x7f6698109150 2024-09-11T00:56:22.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.399+0000 7f668effd640 1 -- 172.21.15.78:0/4278140324 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4741 (secure 0 0 0) 0x7f66900b0670 con 0x7f6698109150 2024-09-11T00:56:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:22.402 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":6,"btime":"2024-09-11T00:56:22:086608+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24529,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24545,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-11T00:56:22.403 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 6 2024-09-11T00:56:22.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.402+0000 7f669fc78640 1 -- 172.21.15.78:0/4278140324 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f66840b5330 msgr2=0x7f66840b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:22.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.402+0000 7f669fc78640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f66840b5330 0x7f66840b77f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6688031440 tx=0x7f6688005b50 comp rx=0 tx=0).stop 2024-09-11T00:56:22.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.402+0000 7f669fc78640 1 -- 172.21.15.78:0/4278140324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6698109150 msgr2=0x7f669819ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:22.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.402+0000 7f669fc78640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6698109150 0x7f669819ec20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f669000b970 tx=0x7f669000be30 comp rx=0 tx=0).stop 2024-09-11T00:56:22.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.402+0000 7f669fc78640 1 -- 172.21.15.78:0/4278140324 shutdown_connections 2024-09-11T00:56:22.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.402+0000 7f669fc78640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f66840b5330 0x7f66840b77f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:22.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.402+0000 7f669fc78640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6698109150 0x7f669819ec20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:22.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.402+0000 7f669fc78640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66981087d0 0x7f6698199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:22.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.402+0000 7f669fc78640 1 --2- 172.21.15.78:0/4278140324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6698102dc0 0x7f6698199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:22.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.402+0000 7f669fc78640 1 -- 172.21.15.78:0/4278140324 >> 172.21.15.78:0/4278140324 conn(0x7f66980fe830 msgr2=0x7f66980ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:22.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.402+0000 7f669fc78640 1 -- 172.21.15.78:0/4278140324 shutdown_connections 2024-09-11T00:56:22.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:22.402+0000 7f669fc78640 1 -- 172.21.15.78:0/4278140324 wait complete. 2024-09-11T00:56:22.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:22 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:22.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:22 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:22.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:22 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:22.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:22 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:22.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:22 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:22.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:22 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:22.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:22 smithi112 ceph-mon[35384]: Deploying daemon mds.f on smithi112 2024-09-11T00:56:22.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:22 smithi112 ceph-mon[35384]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:22.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:22 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.110:6834/3549318748,v1:172.21.15.110:6835/3549318748] up:boot 2024-09-11T00:56:22.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:22 smithi112 ceph-mon[35384]: fsmap 5 up:standby 2024-09-11T00:56:22.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:22 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:56:22.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:22 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/4278140324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:22.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:22 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:22.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:22 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:22.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:22 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:22.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:22 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:22.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:22 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:22.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:22 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:22.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:22 smithi078 ceph-mon[31315]: Deploying daemon mds.f on smithi112 2024-09-11T00:56:22.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:22 smithi078 ceph-mon[31315]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:22.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:22 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.110:6834/3549318748,v1:172.21.15.110:6835/3549318748] up:boot 2024-09-11T00:56:22.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:22 smithi078 ceph-mon[31315]: fsmap 5 up:standby 2024-09-11T00:56:22.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:22 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:56:22.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:22 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4278140324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:23.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:22 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:23.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:22 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:23.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:22 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:23.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:22 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:23.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:22 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:23.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:22 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:23.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:22 smithi110 ceph-mon[34712]: Deploying daemon mds.f on smithi112 2024-09-11T00:56:23.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:22 smithi110 ceph-mon[34712]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:23.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:22 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.110:6834/3549318748,v1:172.21.15.110:6835/3549318748] up:boot 2024-09-11T00:56:23.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:22 smithi110 ceph-mon[34712]: fsmap 5 up:standby 2024-09-11T00:56:23.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:22 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:56:23.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:22 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/4278140324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:23.888 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 00:56:23 smithi112 systemd[1]: Starting Ceph mds.f for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:56:24.175 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 00:56:23 smithi112 podman[66161]: 2024-09-11 00:56:23.789582478 +0000 UTC m=+0.016150787 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:56:24.175 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 00:56:23 smithi112 podman[66161]: 2024-09-11 00:56:23.912842633 +0000 UTC m=+0.139410931 container create 8aa434e7ed0f390dcd56f3f9e6d9de2480f111fee57af7a24fc7ae68ab0ec6b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T00:56:24.175 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 00:56:24 smithi112 podman[66161]: 2024-09-11 00:56:24.173992583 +0000 UTC m=+0.400560882 container init 8aa434e7ed0f390dcd56f3f9e6d9de2480f111fee57af7a24fc7ae68ab0ec6b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T00:56:24.474 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 00:56:24 smithi112 podman[66161]: 2024-09-11 00:56:24.17698672 +0000 UTC m=+0.403555027 container start 8aa434e7ed0f390dcd56f3f9e6d9de2480f111fee57af7a24fc7ae68ab0ec6b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-f, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T00:56:24.474 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 00:56:24 smithi112 ceph-mds[66180]: -- 172.21.15.112:0/809507650 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b86307c20 con 0x564b862e4800 2024-09-11T00:56:24.474 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 00:56:24 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-f[66176]: starting mds.f at 2024-09-11T00:56:24.474 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 00:56:24 smithi112 bash[66161]: 8aa434e7ed0f390dcd56f3f9e6d9de2480f111fee57af7a24fc7ae68ab0ec6b9 2024-09-11T00:56:24.474 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 00:56:24 smithi112 systemd[1]: Started Ceph mds.f for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:56:25.134 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:56:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:25 smithi110 ceph-mon[34712]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:25.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:25 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:25.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:25 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:25.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:25 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:25.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:25 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:25.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:25 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:25.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:25 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:25.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:25 smithi112 ceph-mon[35384]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:25.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:25 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:25.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:25 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:25.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:25 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:25.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:25 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:25.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:25 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:25.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:25 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:25.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:25 smithi078 ceph-mon[31315]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:25.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:25 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:25.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:25 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:25.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:25 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:25.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:25 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:25.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:25 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:25.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:25 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:25.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.882+0000 7f4787e14640 1 -- 172.21.15.78:0/3806047191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4780109150 msgr2=0x7f478010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:25.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.882+0000 7f4787e14640 1 --2- 172.21.15.78:0/3806047191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4780109150 0x7f478010f950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4778009ca0 tx=0x7f477802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:56:25.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.882+0000 7f4787e14640 1 -- 172.21.15.78:0/3806047191 shutdown_connections 2024-09-11T00:56:25.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.882+0000 7f4787e14640 1 --2- 172.21.15.78:0/3806047191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4780109150 0x7f478010f950 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:25.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.882+0000 7f4787e14640 1 --2- 172.21.15.78:0/3806047191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47801087d0 0x7f4780108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:25.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.882+0000 7f4787e14640 1 --2- 172.21.15.78:0/3806047191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4780102dc0 0x7f47801031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:25.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.882+0000 7f4787e14640 1 -- 172.21.15.78:0/3806047191 >> 172.21.15.78:0/3806047191 conn(0x7f47800fe830 msgr2=0x7f4780100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:25.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.882+0000 7f4787e14640 1 -- 172.21.15.78:0/3806047191 shutdown_connections 2024-09-11T00:56:25.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.882+0000 7f4787e14640 1 -- 172.21.15.78:0/3806047191 wait complete. 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.882+0000 7f4787e14640 1 Processor -- start 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.882+0000 7f4787e14640 1 -- start start 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4787e14640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4780102dc0 0x7f4780199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4787e14640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47801087d0 0x7f4780199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4787e14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4780109150 0x7f478019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4787e14640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4780112e10 con 0x7f4780102dc0 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4787e14640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4780112c90 con 0x7f47801087d0 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4787e14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4780112f90 con 0x7f4780109150 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4785bb0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4780102dc0 0x7f4780199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4785bb0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4780102dc0 0x7f4780199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51030/0 (socket says 172.21.15.78:51030) 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4785bb0640 1 -- 172.21.15.78:0/3149460135 learned_addr learned my addr 172.21.15.78:0/3149460135 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f47863b1640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4780109150 0x7f478019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:25.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f47853af640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47801087d0 0x7f4780199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4785bb0640 1 -- 172.21.15.78:0/3149460135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4780109150 msgr2=0x7f478019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4785bb0640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4780109150 0x7f478019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4785bb0640 1 -- 172.21.15.78:0/3149460135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47801087d0 msgr2=0x7f4780199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4785bb0640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47801087d0 0x7f4780199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4785bb0640 1 -- 172.21.15.78:0/3149460135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478019f3b0 con 0x7f4780102dc0 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f47853af640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47801087d0 0x7f4780199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4785bb0640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4780102dc0 0x7f4780199300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f477000b4a0 tx=0x7f477000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4776ffd640 1 -- 172.21.15.78:0/3149460135 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770014070 con 0x7f4780102dc0 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4787e14640 1 -- 172.21.15.78:0/3149460135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478019f6a0 con 0x7f4780102dc0 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4776ffd640 1 -- 172.21.15.78:0/3149460135 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4770010ac0 con 0x7f4780102dc0 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4776ffd640 1 -- 172.21.15.78:0/3149460135 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770017c30 con 0x7f4780102dc0 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.883+0000 7f4787e14640 1 -- 172.21.15.78:0/3149460135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47801a5f30 con 0x7f4780102dc0 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.885+0000 7f4787e14640 1 -- 172.21.15.78:0/3149460135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47801031a0 con 0x7f4780102dc0 2024-09-11T00:56:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.887+0000 7f4776ffd640 1 -- 172.21.15.78:0/3149460135 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f4770010610 con 0x7f4780102dc0 2024-09-11T00:56:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.887+0000 7f4776ffd640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f476c0b5330 0x7f476c0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.887+0000 7f47853af640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f476c0b5330 0x7f476c0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.888+0000 7f4776ffd640 1 -- 172.21.15.78:0/3149460135 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f47700e5130 con 0x7f4780102dc0 2024-09-11T00:56:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.888+0000 7f47853af640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f476c0b5330 0x7f476c0b77f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f47801051a0 tx=0x7f477c005e50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:25.890+0000 7f4776ffd640 1 -- 172.21.15.78:0/3149460135 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f47700ac3c0 con 0x7f4780102dc0 2024-09-11T00:56:26.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.104+0000 7f4787e14640 1 -- 172.21.15.78:0/3149460135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f478002e5c0 con 0x7f4780102dc0 2024-09-11T00:56:26.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.105+0000 7f4776ffd640 1 -- 172.21.15.78:0/3149460135 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5566 (secure 0 0 0) 0x7f47700abcc0 con 0x7f4780102dc0 2024-09-11T00:56:26.107 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:26.108 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":7,"btime":"2024-09-11T00:56:25:066066+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24529,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24545,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-11T00:56:26.108 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 7 2024-09-11T00:56:26.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.107+0000 7f4787e14640 1 -- 172.21.15.78:0/3149460135 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f476c0b5330 msgr2=0x7f476c0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:26.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.107+0000 7f4787e14640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f476c0b5330 0x7f476c0b77f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f47801051a0 tx=0x7f477c005e50 comp rx=0 tx=0).stop 2024-09-11T00:56:26.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.107+0000 7f4787e14640 1 -- 172.21.15.78:0/3149460135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4780102dc0 msgr2=0x7f4780199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:26.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.107+0000 7f4787e14640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4780102dc0 0x7f4780199300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f477000b4a0 tx=0x7f477000b960 comp rx=0 tx=0).stop 2024-09-11T00:56:26.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.107+0000 7f4787e14640 1 -- 172.21.15.78:0/3149460135 shutdown_connections 2024-09-11T00:56:26.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.107+0000 7f4787e14640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f476c0b5330 0x7f476c0b77f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:26.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.107+0000 7f4787e14640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4780109150 0x7f478019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:26.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.107+0000 7f4787e14640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47801087d0 0x7f4780199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:26.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.107+0000 7f4787e14640 1 --2- 172.21.15.78:0/3149460135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4780102dc0 0x7f4780199300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:26.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.107+0000 7f4787e14640 1 -- 172.21.15.78:0/3149460135 >> 172.21.15.78:0/3149460135 conn(0x7f47800fe830 msgr2=0x7f4780105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:26.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.107+0000 7f4787e14640 1 -- 172.21.15.78:0/3149460135 shutdown_connections 2024-09-11T00:56:26.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:26.107+0000 7f4787e14640 1 -- 172.21.15.78:0/3149460135 wait complete. 2024-09-11T00:56:26.394 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:26 smithi078 ceph-mon[31315]: Deploying daemon mds.c on smithi112 2024-09-11T00:56:26.394 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:26 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.112:6834/303956922,v1:172.21.15.112:6835/303956922] up:boot 2024-09-11T00:56:26.394 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:26 smithi078 ceph-mon[31315]: fsmap 6 up:standby 2024-09-11T00:56:26.394 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:26 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:56:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:26 smithi110 ceph-mon[34712]: Deploying daemon mds.c on smithi112 2024-09-11T00:56:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:26 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.112:6834/303956922,v1:172.21.15.112:6835/303956922] up:boot 2024-09-11T00:56:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:26 smithi110 ceph-mon[34712]: fsmap 6 up:standby 2024-09-11T00:56:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:26 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:56:26.460 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:26 smithi112 ceph-mon[35384]: Deploying daemon mds.c on smithi112 2024-09-11T00:56:26.460 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:26 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.112:6834/303956922,v1:172.21.15.112:6835/303956922] up:boot 2024-09-11T00:56:26.460 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:26 smithi112 ceph-mon[35384]: fsmap 6 up:standby 2024-09-11T00:56:26.461 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:26 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:56:26.461 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 00:56:26 smithi112 systemd[1]: Starting Ceph mds.c for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:56:26.723 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 00:56:26 smithi112 podman[66461]: 2024-09-11 00:56:26.361855302 +0000 UTC m=+0.016381601 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:56:26.724 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 00:56:26 smithi112 podman[66461]: 2024-09-11 00:56:26.49389878 +0000 UTC m=+0.148425058 container create 3af06f9e3ebf3423504e94d40d33cb46410215f91b66fe15319086565d3f4d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:56:27.080 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 00:56:26 smithi112 podman[66461]: 2024-09-11 00:56:26.771655005 +0000 UTC m=+0.426181284 container init 3af06f9e3ebf3423504e94d40d33cb46410215f91b66fe15319086565d3f4d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T00:56:27.080 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 00:56:26 smithi112 podman[66461]: 2024-09-11 00:56:26.774367546 +0000 UTC m=+0.428893826 container start 3af06f9e3ebf3423504e94d40d33cb46410215f91b66fe15319086565d3f4d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T00:56:27.080 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 00:56:26 smithi112 ceph-mds[66480]: -- 172.21.15.112:0/1982402443 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d40c05860 con 0x563d40be3000 2024-09-11T00:56:27.080 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 00:56:26 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c[66476]: starting mds.c at 2024-09-11T00:56:27.080 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 00:56:26 smithi112 bash[66461]: 3af06f9e3ebf3423504e94d40d33cb46410215f91b66fe15319086565d3f4d7f 2024-09-11T00:56:27.081 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 00:56:26 smithi112 systemd[1]: Started Ceph mds.c for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:56:27.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:27 smithi110 ceph-mon[34712]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:27.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:27 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3149460135' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:27.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:27.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:27.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:27.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:27.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:27.369 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:27 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:27.380 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:27 smithi078 ceph-mon[31315]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:27.380 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:27 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3149460135' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:27.380 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:27.380 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:27.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:27.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:27.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:27.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:27 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:27 smithi112 ceph-mon[35384]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:27 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3149460135' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:27 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:28 smithi110 ceph-mon[34712]: Deploying daemon mds.h on smithi110 2024-09-11T00:56:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:28 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] up:boot 2024-09-11T00:56:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:28 smithi110 ceph-mon[34712]: fsmap 7 up:standby 2024-09-11T00:56:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:28 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:56:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:28 smithi112 ceph-mon[35384]: Deploying daemon mds.h on smithi110 2024-09-11T00:56:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:28 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] up:boot 2024-09-11T00:56:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:28 smithi112 ceph-mon[35384]: fsmap 7 up:standby 2024-09-11T00:56:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:28 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:56:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:28 smithi078 ceph-mon[31315]: Deploying daemon mds.h on smithi110 2024-09-11T00:56:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:28 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] up:boot 2024-09-11T00:56:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:28 smithi078 ceph-mon[31315]: fsmap 7 up:standby 2024-09-11T00:56:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:28 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:56:28.779 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:56:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:29.093 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 00:56:28 smithi110 systemd[1]: Starting Ceph mds.h for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:56:29.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:29 smithi078 ceph-mon[31315]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:29.348 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:29 smithi110 ceph-mon[34712]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:29.349 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 00:56:29 smithi110 podman[65287]: 2024-09-11 00:56:29.062154504 +0000 UTC m=+0.015925092 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:56:29.349 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 00:56:29 smithi110 podman[65287]: 2024-09-11 00:56:29.199376581 +0000 UTC m=+0.153147153 container create 7242af13f12682cb1a3cbd4e503eeb3ecbd6774f90460f19f038badaca9242d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T00:56:29.375 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:29 smithi112 ceph-mon[35384]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:29.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.638+0000 7f6e01025640 1 -- 172.21.15.78:0/2614553798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dfc1076a0 msgr2=0x7f6dfc107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:29.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.638+0000 7f6e01025640 1 --2- 172.21.15.78:0/2614553798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dfc1076a0 0x7f6dfc107ae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009900 tx=0x7f6de402a410 comp rx=0 tx=0).stop 2024-09-11T00:56:29.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.638+0000 7f6e01025640 1 -- 172.21.15.78:0/2614553798 shutdown_connections 2024-09-11T00:56:29.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.638+0000 7f6e01025640 1 --2- 172.21.15.78:0/2614553798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dfc108020 0x7f6dfc10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:29.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.638+0000 7f6e01025640 1 --2- 172.21.15.78:0/2614553798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dfc1076a0 0x7f6dfc107ae0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.638+0000 7f6e01025640 1 --2- 172.21.15.78:0/2614553798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dfc101c90 0x7f6dfc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.638+0000 7f6e01025640 1 -- 172.21.15.78:0/2614553798 >> 172.21.15.78:0/2614553798 conn(0x7f6dfc0fd700 msgr2=0x7f6dfc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.638+0000 7f6e01025640 1 -- 172.21.15.78:0/2614553798 shutdown_connections 2024-09-11T00:56:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.638+0000 7f6e01025640 1 -- 172.21.15.78:0/2614553798 wait complete. 2024-09-11T00:56:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6e01025640 1 Processor -- start 2024-09-11T00:56:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6e01025640 1 -- start start 2024-09-11T00:56:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6e01025640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dfc101c90 0x7f6dfc1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6e01025640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dfc1076a0 0x7f6dfc199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6e01025640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dfc108020 0x7f6dfc19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6e01025640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc111ce0 con 0x7f6dfc101c90 2024-09-11T00:56:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6e01025640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc111b60 con 0x7f6dfc1076a0 2024-09-11T00:56:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6e01025640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc111e60 con 0x7f6dfc108020 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6dfad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dfc101c90 0x7f6dfc1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6dfad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dfc101c90 0x7f6dfc1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51040/0 (socket says 172.21.15.78:51040) 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6dfad9d640 1 -- 172.21.15.78:0/3928502022 learned_addr learned my addr 172.21.15.78:0/3928502022 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6dfa59c640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dfc1076a0 0x7f6dfc199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6dfb59e640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dfc108020 0x7f6dfc19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6dfad9d640 1 -- 172.21.15.78:0/3928502022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dfc108020 msgr2=0x7f6dfc19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.639+0000 7f6dfad9d640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dfc108020 0x7f6dfc19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.640+0000 7f6dfad9d640 1 -- 172.21.15.78:0/3928502022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dfc1076a0 msgr2=0x7f6dfc199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.640+0000 7f6dfad9d640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dfc1076a0 0x7f6dfc199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.640+0000 7f6dfad9d640 1 -- 172.21.15.78:0/3928502022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc19f380 con 0x7f6dfc101c90 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.640+0000 7f6dfad9d640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dfc101c90 0x7f6dfc1992d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6df400e9f0 tx=0x7f6df400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.642+0000 7f6df3fff640 1 -- 172.21.15.78:0/3928502022 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df400cd80 con 0x7f6dfc101c90 2024-09-11T00:56:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.642+0000 7f6e01025640 1 -- 172.21.15.78:0/3928502022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dfc19f670 con 0x7f6dfc101c90 2024-09-11T00:56:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.642+0000 7f6df3fff640 1 -- 172.21.15.78:0/3928502022 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df40108e0 con 0x7f6dfc101c90 2024-09-11T00:56:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.642+0000 7f6df3fff640 1 -- 172.21.15.78:0/3928502022 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4018980 con 0x7f6dfc101c90 2024-09-11T00:56:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.642+0000 7f6e01025640 1 -- 172.21.15.78:0/3928502022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dfc1a5f00 con 0x7f6dfc101c90 2024-09-11T00:56:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.643+0000 7f6e01025640 1 -- 172.21.15.78:0/3928502022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dfc102f40 con 0x7f6dfc101c90 2024-09-11T00:56:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.644+0000 7f6df3fff640 1 -- 172.21.15.78:0/3928502022 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f6df4010430 con 0x7f6dfc101c90 2024-09-11T00:56:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.645+0000 7f6df3fff640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6de80b53b0 0x7f6de80b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.645+0000 7f6dfa59c640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6de80b53b0 0x7f6de80b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.645+0000 7f6df3fff640 1 -- 172.21.15.78:0/3928502022 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f6df40e5ac0 con 0x7f6dfc101c90 2024-09-11T00:56:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.646+0000 7f6dfa59c640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6de80b53b0 0x7f6de80b7870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6de4031440 tx=0x7f6de4005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.649+0000 7f6df3fff640 1 -- 172.21.15.78:0/3928502022 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6df40ac700 con 0x7f6dfc101c90 2024-09-11T00:56:29.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.861+0000 7f6e01025640 1 -- 172.21.15.78:0/3928502022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dfc10ae10 con 0x7f6dfc101c90 2024-09-11T00:56:29.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.862+0000 7f6df3fff640 1 -- 172.21.15.78:0/3928502022 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6391 (secure 0 0 0) 0x7f6df40b1130 con 0x7f6dfc101c90 2024-09-11T00:56:29.864 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:29.864 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"btime":"2024-09-11T00:56:27:078127+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24529,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24545,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-11T00:56:29.864 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-09-11T00:56:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.864+0000 7f6e01025640 1 -- 172.21.15.78:0/3928502022 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6de80b53b0 msgr2=0x7f6de80b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.864+0000 7f6e01025640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6de80b53b0 0x7f6de80b7870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6de4031440 tx=0x7f6de4005b50 comp rx=0 tx=0).stop 2024-09-11T00:56:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.864+0000 7f6e01025640 1 -- 172.21.15.78:0/3928502022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dfc101c90 msgr2=0x7f6dfc1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.864+0000 7f6e01025640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dfc101c90 0x7f6dfc1992d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6df400e9f0 tx=0x7f6df400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:56:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.864+0000 7f6e01025640 1 -- 172.21.15.78:0/3928502022 shutdown_connections 2024-09-11T00:56:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.864+0000 7f6e01025640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6de80b53b0 0x7f6de80b7870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.864+0000 7f6e01025640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dfc108020 0x7f6dfc19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.864+0000 7f6e01025640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dfc1076a0 0x7f6dfc199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.864+0000 7f6e01025640 1 --2- 172.21.15.78:0/3928502022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dfc101c90 0x7f6dfc1992d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.864+0000 7f6e01025640 1 -- 172.21.15.78:0/3928502022 >> 172.21.15.78:0/3928502022 conn(0x7f6dfc0fd700 msgr2=0x7f6dfc06ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:29.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.864+0000 7f6e01025640 1 -- 172.21.15.78:0/3928502022 shutdown_connections 2024-09-11T00:56:29.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:29.864+0000 7f6e01025640 1 -- 172.21.15.78:0/3928502022 wait complete. 2024-09-11T00:56:29.940 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 00:56:29 smithi110 podman[65287]: 2024-09-11 00:56:29.460539022 +0000 UTC m=+0.414309602 container init 7242af13f12682cb1a3cbd4e503eeb3ecbd6774f90460f19f038badaca9242d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T00:56:29.940 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 00:56:29 smithi110 podman[65287]: 2024-09-11 00:56:29.463242809 +0000 UTC m=+0.417013382 container start 7242af13f12682cb1a3cbd4e503eeb3ecbd6774f90460f19f038badaca9242d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T00:56:29.940 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 00:56:29 smithi110 ceph-mds[65306]: -- 172.21.15.110:0/3148965893 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557807539860 con 0x557807517400 2024-09-11T00:56:29.940 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 00:56:29 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h[65302]: starting mds.h at 2024-09-11T00:56:29.940 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 00:56:29 smithi110 bash[65287]: 7242af13f12682cb1a3cbd4e503eeb3ecbd6774f90460f19f038badaca9242d7 2024-09-11T00:56:29.940 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 00:56:29 smithi110 systemd[1]: Started Ceph mds.h for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:56:30.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:30.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:30.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:30.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:30.897 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:30.898 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:30.898 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:30 smithi110 ceph-mon[34712]: Deploying daemon mds.i on smithi112 2024-09-11T00:56:30.898 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:30 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3928502022' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:30.898 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:30 smithi110 ceph-mon[34712]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:30.898 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:30 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] up:boot 2024-09-11T00:56:30.898 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:30 smithi110 ceph-mon[34712]: fsmap 8 up:standby 2024-09-11T00:56:30.898 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:56:30.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:30.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:30.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:30.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:30.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:30.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:30.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:30 smithi112 ceph-mon[35384]: Deploying daemon mds.i on smithi112 2024-09-11T00:56:30.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:30 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3928502022' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:30.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:30 smithi112 ceph-mon[35384]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:30.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:30 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] up:boot 2024-09-11T00:56:30.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:30 smithi112 ceph-mon[35384]: fsmap 8 up:standby 2024-09-11T00:56:30.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:56:30.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:30.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:30.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:30.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:30.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:30.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:30.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:30 smithi078 ceph-mon[31315]: Deploying daemon mds.i on smithi112 2024-09-11T00:56:30.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:30 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3928502022' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:30.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:30 smithi078 ceph-mon[31315]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:30.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:30 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] up:boot 2024-09-11T00:56:30.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:30 smithi078 ceph-mon[31315]: fsmap 8 up:standby 2024-09-11T00:56:30.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:56:31.724 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 00:56:31 smithi112 systemd[1]: Starting Ceph mds.i for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:56:32.140 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 00:56:31 smithi112 podman[66775]: 2024-09-11 00:56:31.728263241 +0000 UTC m=+0.015794072 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:56:32.140 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 00:56:31 smithi112 podman[66775]: 2024-09-11 00:56:31.860637352 +0000 UTC m=+0.148168176 container create be7bba0306b443a94351301c676f99124e0617d9c6412b5d7d80e870f8495a79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-i, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T00:56:32.140 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 00:56:32 smithi112 podman[66775]: 2024-09-11 00:56:32.139052806 +0000 UTC m=+0.426583624 container init be7bba0306b443a94351301c676f99124e0617d9c6412b5d7d80e870f8495a79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T00:56:32.474 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 00:56:32 smithi112 podman[66775]: 2024-09-11 00:56:32.142170765 +0000 UTC m=+0.429701590 container start be7bba0306b443a94351301c676f99124e0617d9c6412b5d7d80e870f8495a79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:56:32.474 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 00:56:32 smithi112 ceph-mds[66794]: -- 172.21.15.112:0/3621807687 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4269e7c20 con 0x55a4269c5000 2024-09-11T00:56:32.474 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 00:56:32 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-i[66790]: starting mds.i at 2024-09-11T00:56:32.474 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 00:56:32 smithi112 bash[66775]: be7bba0306b443a94351301c676f99124e0617d9c6412b5d7d80e870f8495a79 2024-09-11T00:56:32.474 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 00:56:32 smithi112 systemd[1]: Started Ceph mds.i for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:56:32.565 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:56:33.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:33 smithi078 ceph-mon[31315]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:33.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:33.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:33.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:33.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:33.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:33.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:33 smithi110 ceph-mon[34712]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:33.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:33.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:33 smithi112 ceph-mon[35384]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:33.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:33.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:33.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:33.788 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:34.333 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:34 smithi078 ceph-mon[31315]: Deploying daemon mds.g on smithi078 2024-09-11T00:56:34.334 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:34 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] up:boot 2024-09-11T00:56:34.334 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:34 smithi078 ceph-mon[31315]: fsmap 9 up:standby 2024-09-11T00:56:34.334 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:56:34.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:34 smithi110 ceph-mon[34712]: Deploying daemon mds.g on smithi078 2024-09-11T00:56:34.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:34 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] up:boot 2024-09-11T00:56:34.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:34 smithi110 ceph-mon[34712]: fsmap 9 up:standby 2024-09-11T00:56:34.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:56:34.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.412+0000 7fdb4077a640 1 -- 172.21.15.78:0/123388945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb381076e0 msgr2=0x7fdb38107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:34.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.412+0000 7fdb4077a640 1 --2- 172.21.15.78:0/123388945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb381076e0 0x7fdb38107b20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009900 tx=0x7fdb2802a410 comp rx=0 tx=0).stop 2024-09-11T00:56:34.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.412+0000 7fdb4077a640 1 -- 172.21.15.78:0/123388945 shutdown_connections 2024-09-11T00:56:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.412+0000 7fdb4077a640 1 --2- 172.21.15.78:0/123388945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb38108060 0x7fdb3810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.412+0000 7fdb4077a640 1 --2- 172.21.15.78:0/123388945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb381076e0 0x7fdb38107b20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.412+0000 7fdb4077a640 1 --2- 172.21.15.78:0/123388945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38101cd0 0x7fdb381020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.412+0000 7fdb4077a640 1 -- 172.21.15.78:0/123388945 >> 172.21.15.78:0/123388945 conn(0x7fdb380fd700 msgr2=0x7fdb380ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.412+0000 7fdb4077a640 1 -- 172.21.15.78:0/123388945 shutdown_connections 2024-09-11T00:56:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.412+0000 7fdb4077a640 1 -- 172.21.15.78:0/123388945 wait complete. 2024-09-11T00:56:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb4077a640 1 Processor -- start 2024-09-11T00:56:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb4077a640 1 -- start start 2024-09-11T00:56:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb4077a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb38101cd0 0x7fdb380fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb4077a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb381076e0 0x7fdb380fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb4077a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38108060 0x7fdb380fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb4077a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38111d20 con 0x7fdb381076e0 2024-09-11T00:56:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb4077a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38111ba0 con 0x7fdb38108060 2024-09-11T00:56:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb4077a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38111ea0 con 0x7fdb38101cd0 2024-09-11T00:56:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb3dd15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb381076e0 0x7fdb380fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb3dd15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb381076e0 0x7fdb380fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41530/0 (socket says 172.21.15.78:41530) 2024-09-11T00:56:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb3dd15640 1 -- 172.21.15.78:0/3788168945 learned_addr learned my addr 172.21.15.78:0/3788168945 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb3ed17640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38108060 0x7fdb380fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.413+0000 7fdb3e516640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb38101cd0 0x7fdb380fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.414+0000 7fdb3dd15640 1 -- 172.21.15.78:0/3788168945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb38101cd0 msgr2=0x7fdb380fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.414+0000 7fdb3dd15640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb38101cd0 0x7fdb380fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.414+0000 7fdb3dd15640 1 -- 172.21.15.78:0/3788168945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38108060 msgr2=0x7fdb380fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.414+0000 7fdb3dd15640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38108060 0x7fdb380fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.414+0000 7fdb3dd15640 1 -- 172.21.15.78:0/3788168945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb381a7680 con 0x7fdb381076e0 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.414+0000 7fdb3dd15640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb381076e0 0x7fdb380fe140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb28005940 tx=0x7fdb280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.414+0000 7fdb2f7fe640 1 -- 172.21.15.78:0/3788168945 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2803f030 con 0x7fdb381076e0 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.414+0000 7fdb2f7fe640 1 -- 172.21.15.78:0/3788168945 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb28003cb0 con 0x7fdb381076e0 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.414+0000 7fdb4077a640 1 -- 172.21.15.78:0/3788168945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb381a7910 con 0x7fdb381076e0 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.414+0000 7fdb2f7fe640 1 -- 172.21.15.78:0/3788168945 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28035070 con 0x7fdb381076e0 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.414+0000 7fdb4077a640 1 -- 172.21.15.78:0/3788168945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb381a7df0 con 0x7fdb381076e0 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.415+0000 7fdb4077a640 1 -- 172.21.15.78:0/3788168945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb381020b0 con 0x7fdb381076e0 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.416+0000 7fdb2f7fe640 1 -- 172.21.15.78:0/3788168945 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fdb2803bbf0 con 0x7fdb381076e0 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.417+0000 7fdb2f7fe640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdb240b5330 0x7fdb240b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.417+0000 7fdb3e516640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdb240b5330 0x7fdb240b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.417+0000 7fdb2f7fe640 1 -- 172.21.15.78:0/3788168945 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fdb281065b0 con 0x7fdb381076e0 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.417+0000 7fdb3e516640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdb240b5330 0x7fdb240b77f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400bbe0 tx=0x7fdb3400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.420+0000 7fdb2f7fe640 1 -- 172.21.15.78:0/3788168945 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdb280cd1f0 con 0x7fdb381076e0 2024-09-11T00:56:34.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:34 smithi112 ceph-mon[35384]: Deploying daemon mds.g on smithi078 2024-09-11T00:56:34.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:34 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] up:boot 2024-09-11T00:56:34.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:34 smithi112 ceph-mon[35384]: fsmap 9 up:standby 2024-09-11T00:56:34.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:56:34.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.651+0000 7fdb4077a640 1 -- 172.21.15.78:0/3788168945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb381a81b0 con 0x7fdb381076e0 2024-09-11T00:56:34.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.651+0000 7fdb2f7fe640 1 -- 172.21.15.78:0/3788168945 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8049 (secure 0 0 0) 0x7fdb280d1c20 con 0x7fdb381076e0 2024-09-11T00:56:34.653 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:34.653 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":10,"btime":"2024-09-11T00:56:33:067567+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24529,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24545,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24563,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24580,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T00:56:34.654 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 10 2024-09-11T00:56:34.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.653+0000 7fdb4077a640 1 -- 172.21.15.78:0/3788168945 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdb240b5330 msgr2=0x7fdb240b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:34.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.653+0000 7fdb4077a640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdb240b5330 0x7fdb240b77f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400bbe0 tx=0x7fdb3400b390 comp rx=0 tx=0).stop 2024-09-11T00:56:34.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.653+0000 7fdb4077a640 1 -- 172.21.15.78:0/3788168945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb381076e0 msgr2=0x7fdb380fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:34.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.653+0000 7fdb4077a640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb381076e0 0x7fdb380fe140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb28005940 tx=0x7fdb280042f0 comp rx=0 tx=0).stop 2024-09-11T00:56:34.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.654+0000 7fdb4077a640 1 -- 172.21.15.78:0/3788168945 shutdown_connections 2024-09-11T00:56:34.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.654+0000 7fdb4077a640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdb240b5330 0x7fdb240b77f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:34.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.654+0000 7fdb4077a640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38108060 0x7fdb380fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:34.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.654+0000 7fdb4077a640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb381076e0 0x7fdb380fe140 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:34.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.654+0000 7fdb4077a640 1 --2- 172.21.15.78:0/3788168945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb38101cd0 0x7fdb380fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:34.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.654+0000 7fdb4077a640 1 -- 172.21.15.78:0/3788168945 >> 172.21.15.78:0/3788168945 conn(0x7fdb380fd700 msgr2=0x7fdb3806b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:34.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.654+0000 7fdb4077a640 1 -- 172.21.15.78:0/3788168945 shutdown_connections 2024-09-11T00:56:34.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:34.654+0000 7fdb4077a640 1 -- 172.21.15.78:0/3788168945 wait complete. 2024-09-11T00:56:35.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:35 smithi110 ceph-mon[34712]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:35.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:35 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3788168945' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:35.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:35 smithi112 ceph-mon[35384]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:35.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:35 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3788168945' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:35.490 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:35 smithi078 ceph-mon[31315]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:35.490 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:35 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3788168945' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:35.747 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 00:56:35 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-g[74328]: starting mds.g at 2024-09-11T00:56:36.247 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 00:56:35 smithi078 bash[74300]: 33a1b47b8cfbe6b2a2ac9211a684bf8ee316e11e1e021ca40a7c75f03500307f 2024-09-11T00:56:36.247 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 00:56:35 smithi078 systemd[1]: Started Ceph mds.g for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:56:37.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:36 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:37.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:36 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:37.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:36 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:37.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:36 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:37.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:36 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:37.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:36 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:37.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:36 smithi110 ceph-mon[34712]: Deploying daemon mds.a on smithi078 2024-09-11T00:56:37.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:36 smithi110 ceph-mon[34712]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:37.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:36 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.78:6838/3902249887,v1:172.21.15.78:6839/3902249887] up:boot 2024-09-11T00:56:37.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:36 smithi110 ceph-mon[34712]: fsmap 10 up:standby 2024-09-11T00:56:37.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:36 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T00:56:37.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:36 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:37.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:36 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:37.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:36 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:37.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:36 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:37.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:36 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:37.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:36 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:37.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:36 smithi112 ceph-mon[35384]: Deploying daemon mds.a on smithi078 2024-09-11T00:56:37.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:36 smithi112 ceph-mon[35384]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:37.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:36 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.78:6838/3902249887,v1:172.21.15.78:6839/3902249887] up:boot 2024-09-11T00:56:37.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:36 smithi112 ceph-mon[35384]: fsmap 10 up:standby 2024-09-11T00:56:37.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:36 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T00:56:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:36 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:36 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:36 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:36 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:36 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:36 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:36 smithi078 ceph-mon[31315]: Deploying daemon mds.a on smithi078 2024-09-11T00:56:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:36 smithi078 ceph-mon[31315]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:36 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.78:6838/3902249887,v1:172.21.15.78:6839/3902249887] up:boot 2024-09-11T00:56:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:36 smithi078 ceph-mon[31315]: fsmap 10 up:standby 2024-09-11T00:56:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:36 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T00:56:37.521 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:56:37.800 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:37.923 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 00:56:37 smithi078 systemd[1]: Starting Ceph mds.a for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:56:38.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.309+0000 7f8081d9a640 1 -- 172.21.15.78:0/3151502772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f807c10d100 msgr2=0x7f807c10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:38.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.309+0000 7f8081d9a640 1 --2- 172.21.15.78:0/3151502772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f807c10d100 0x7f807c10f560 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8074009ca0 tx=0x7f807402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:56:38.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.309+0000 7f8081d9a640 1 -- 172.21.15.78:0/3151502772 shutdown_connections 2024-09-11T00:56:38.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.309+0000 7f8081d9a640 1 --2- 172.21.15.78:0/3151502772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f807c10d100 0x7f807c10f560 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:38.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.309+0000 7f8081d9a640 1 --2- 172.21.15.78:0/3151502772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807c1045c0 0x7f807c10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:38.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.309+0000 7f8081d9a640 1 --2- 172.21.15.78:0/3151502772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f807c103ca0 0x7f807c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:38.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.309+0000 7f8081d9a640 1 -- 172.21.15.78:0/3151502772 >> 172.21.15.78:0/3151502772 conn(0x7f807c0fd700 msgr2=0x7f807c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:38.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.309+0000 7f8081d9a640 1 -- 172.21.15.78:0/3151502772 shutdown_connections 2024-09-11T00:56:38.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.309+0000 7f8081d9a640 1 -- 172.21.15.78:0/3151502772 wait complete. 2024-09-11T00:56:38.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.309+0000 7f8081d9a640 1 Processor -- start 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.309+0000 7f8081d9a640 1 -- start start 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8081d9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807c103ca0 0x7f807c101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8081d9a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f807c1045c0 0x7f807c101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8081d9a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f807c10d100 0x7f807c1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8081d9a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f807c112db0 con 0x7f807c1045c0 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8081d9a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f807c112c30 con 0x7f807c10d100 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8081d9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f807c112f30 con 0x7f807c103ca0 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8072ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f807c1045c0 0x7f807c101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8072ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f807c1045c0 0x7f807c101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41556/0 (socket says 172.21.15.78:41556) 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8072ffd640 1 -- 172.21.15.78:0/2461795980 learned_addr learned my addr 172.21.15.78:0/2461795980 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f807b7fe640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807c103ca0 0x7f807c101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f807bfff640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f807c10d100 0x7f807c1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8072ffd640 1 -- 172.21.15.78:0/2461795980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807c103ca0 msgr2=0x7f807c101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8072ffd640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807c103ca0 0x7f807c101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8072ffd640 1 -- 172.21.15.78:0/2461795980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f807c10d100 msgr2=0x7f807c1023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8072ffd640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f807c10d100 0x7f807c1023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8072ffd640 1 -- 172.21.15.78:0/2461795980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c1a7640 con 0x7f807c1045c0 2024-09-11T00:56:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f807bfff640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f807c10d100 0x7f807c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:56:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8072ffd640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f807c1045c0 0x7f807c101e90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f806400c970 tx=0x7f806400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f80797fa640 1 -- 172.21.15.78:0/2461795980 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064007bb0 con 0x7f807c1045c0 2024-09-11T00:56:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f80797fa640 1 -- 172.21.15.78:0/2461795980 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8064007d50 con 0x7f807c1045c0 2024-09-11T00:56:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8081d9a640 1 -- 172.21.15.78:0/2461795980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807c1a7930 con 0x7f807c1045c0 2024-09-11T00:56:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f80797fa640 1 -- 172.21.15.78:0/2461795980 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806400f6b0 con 0x7f807c1045c0 2024-09-11T00:56:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.310+0000 7f8081d9a640 1 -- 172.21.15.78:0/2461795980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807c1a7f40 con 0x7f807c1045c0 2024-09-11T00:56:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.311+0000 7f8081d9a640 1 -- 172.21.15.78:0/2461795980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f807c104080 con 0x7f807c1045c0 2024-09-11T00:56:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.313+0000 7f80797fa640 1 -- 172.21.15.78:0/2461795980 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f8064004020 con 0x7f807c1045c0 2024-09-11T00:56:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.313+0000 7f80797fa640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80680b53b0 0x7f80680b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.313+0000 7f807b7fe640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80680b53b0 0x7f80680b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.314+0000 7f80797fa640 1 -- 172.21.15.78:0/2461795980 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f80640e5cd0 con 0x7f807c1045c0 2024-09-11T00:56:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.314+0000 7f807b7fe640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80680b53b0 0x7f80680b7870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f806c009990 tx=0x7f806c009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.316+0000 7f80797fa640 1 -- 172.21.15.78:0/2461795980 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f80640ad080 con 0x7f807c1045c0 2024-09-11T00:56:38.497 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 00:56:38 smithi078 podman[74688]: 2024-09-11 00:56:38.36780967 +0000 UTC m=+0.245366430 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:56:38.497 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 00:56:38 smithi078 podman[74688]: 2024-09-11 00:56:38.472054143 +0000 UTC m=+0.349610891 container create 40e21add21e03d1029c7203203ddcade61efcc2663f3307e242cef47f6e5bb59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T00:56:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.532+0000 7f8081d9a640 1 -- 172.21.15.78:0/2461795980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f807c102c50 con 0x7f807c1045c0 2024-09-11T00:56:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.532+0000 7f80797fa640 1 -- 172.21.15.78:0/2461795980 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8875 (secure 0 0 0) 0x7f807c102c50 con 0x7f807c1045c0 2024-09-11T00:56:38.534 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:38.534 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":11,"btime":"2024-09-11T00:56:36:085123+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24529,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24545,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24563,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24580,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T00:56:38.534 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 11 2024-09-11T00:56:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.534+0000 7f8081d9a640 1 -- 172.21.15.78:0/2461795980 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80680b53b0 msgr2=0x7f80680b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.534+0000 7f8081d9a640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80680b53b0 0x7f80680b7870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f806c009990 tx=0x7f806c009110 comp rx=0 tx=0).stop 2024-09-11T00:56:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.535+0000 7f8081d9a640 1 -- 172.21.15.78:0/2461795980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f807c1045c0 msgr2=0x7f807c101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.535+0000 7f8081d9a640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f807c1045c0 0x7f807c101e90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f806400c970 tx=0x7f806400cd70 comp rx=0 tx=0).stop 2024-09-11T00:56:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.535+0000 7f8081d9a640 1 -- 172.21.15.78:0/2461795980 shutdown_connections 2024-09-11T00:56:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.535+0000 7f8081d9a640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80680b53b0 0x7f80680b7870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.535+0000 7f8081d9a640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f807c10d100 0x7f807c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.535+0000 7f8081d9a640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f807c1045c0 0x7f807c101e90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.535+0000 7f8081d9a640 1 --2- 172.21.15.78:0/2461795980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807c103ca0 0x7f807c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.535+0000 7f8081d9a640 1 -- 172.21.15.78:0/2461795980 >> 172.21.15.78:0/2461795980 conn(0x7f807c0fd700 msgr2=0x7f807c10f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.535+0000 7f8081d9a640 1 -- 172.21.15.78:0/2461795980 shutdown_connections 2024-09-11T00:56:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:38.535+0000 7f8081d9a640 1 -- 172.21.15.78:0/2461795980 wait complete. 2024-09-11T00:56:39.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:39 smithi078 ceph-mon[31315]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:39.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:39 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2461795980' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:39 smithi110 ceph-mon[34712]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:39 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2461795980' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:39.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:39 smithi112 ceph-mon[35384]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:39 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2461795980' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:39.747 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 00:56:39 smithi078 podman[74688]: 2024-09-11 00:56:39.363171659 +0000 UTC m=+1.240728412 container init 40e21add21e03d1029c7203203ddcade61efcc2663f3307e242cef47f6e5bb59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-11T00:56:39.747 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 00:56:39 smithi078 podman[74688]: 2024-09-11 00:56:39.366204675 +0000 UTC m=+1.243761424 container start 40e21add21e03d1029c7203203ddcade61efcc2663f3307e242cef47f6e5bb59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T00:56:39.747 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 00:56:39 smithi078 ceph-mds[74744]: -- 172.21.15.78:0/2997939042 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56535532b860 con 0x565355308800 2024-09-11T00:56:39.747 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 00:56:39 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-a[74736]: starting mds.a at 2024-09-11T00:56:39.747 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 00:56:39 smithi078 bash[74688]: 40e21add21e03d1029c7203203ddcade61efcc2663f3307e242cef47f6e5bb59 2024-09-11T00:56:39.747 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 00:56:39 smithi078 systemd[1]: Started Ceph mds.a for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:56:40.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:40 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:40.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:40 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:40.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:40 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:40.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:40 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:40.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:40 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:40.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:40 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:40.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:40 smithi110 ceph-mon[34712]: Deploying daemon mds.b on smithi110 2024-09-11T00:56:40.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:40 smithi110 ceph-mon[34712]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:40.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:40 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] up:boot 2024-09-11T00:56:40.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:40 smithi110 ceph-mon[34712]: fsmap 11 up:standby 2024-09-11T00:56:40.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:40 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T00:56:40.781 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:40 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:40.781 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:40 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:40.781 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:40 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:40.781 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:40 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:40.781 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:40 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:40.781 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:40 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:40.781 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:40 smithi078 ceph-mon[31315]: Deploying daemon mds.b on smithi110 2024-09-11T00:56:40.782 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:40 smithi078 ceph-mon[31315]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:40.782 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:40 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] up:boot 2024-09-11T00:56:40.782 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:40 smithi078 ceph-mon[31315]: fsmap 11 up:standby 2024-09-11T00:56:40.782 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:40 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T00:56:40.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:40 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:40.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:40 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:40.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:40 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:40.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:40 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:56:40.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:40 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:56:40.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:40 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:40.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:40 smithi112 ceph-mon[35384]: Deploying daemon mds.b on smithi110 2024-09-11T00:56:40.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:40 smithi112 ceph-mon[35384]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:40.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:40 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] up:boot 2024-09-11T00:56:40.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:40 smithi112 ceph-mon[35384]: fsmap 11 up:standby 2024-09-11T00:56:40.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:40 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T00:56:41.191 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:56:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:41.690 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 00:56:41 smithi110 systemd[1]: Starting Ceph mds.b for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T00:56:41.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.955+0000 7fcdac8ed640 1 -- 172.21.15.78:0/1741494927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcda8105590 msgr2=0x7fcda81059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:41.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.955+0000 7fcdac8ed640 1 --2- 172.21.15.78:0/1741494927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcda8105590 0x7fcda81059d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcd98009900 tx=0x7fcd9802a410 comp rx=0 tx=0).stop 2024-09-11T00:56:41.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.955+0000 7fcdac8ed640 1 -- 172.21.15.78:0/1741494927 shutdown_connections 2024-09-11T00:56:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.955+0000 7fcdac8ed640 1 --2- 172.21.15.78:0/1741494927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda8105f10 0x7fcda810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.955+0000 7fcdac8ed640 1 --2- 172.21.15.78:0/1741494927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcda8105590 0x7fcda81059d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.955+0000 7fcdac8ed640 1 --2- 172.21.15.78:0/1741494927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcda81011e0 0x7fcda81015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.955+0000 7fcdac8ed640 1 -- 172.21.15.78:0/1741494927 >> 172.21.15.78:0/1741494927 conn(0x7fcda8078060 msgr2=0x7fcda80ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.955+0000 7fcdac8ed640 1 -- 172.21.15.78:0/1741494927 shutdown_connections 2024-09-11T00:56:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.955+0000 7fcdac8ed640 1 -- 172.21.15.78:0/1741494927 wait complete. 2024-09-11T00:56:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.956+0000 7fcdac8ed640 1 Processor -- start 2024-09-11T00:56:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.956+0000 7fcdac8ed640 1 -- start start 2024-09-11T00:56:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.956+0000 7fcdac8ed640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcda81011e0 0x7fcda81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.956+0000 7fcdac8ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcda8105590 0x7fcda8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.956+0000 7fcdac8ed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda8105f10 0x7fcda819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.956+0000 7fcdac8ed640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcda8112e10 con 0x7fcda8105590 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.956+0000 7fcdac8ed640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcda8112c90 con 0x7fcda81011e0 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.956+0000 7fcdac8ed640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcda8112f90 con 0x7fcda8105f10 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcda5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcda8105590 0x7fcda8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcda5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcda8105590 0x7fcda8199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41592/0 (socket says 172.21.15.78:41592) 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcda5d9b640 1 -- 172.21.15.78:0/3996032526 learned_addr learned my addr 172.21.15.78:0/3996032526 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcda6d9d640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda8105f10 0x7fcda819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcda5d9b640 1 -- 172.21.15.78:0/3996032526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda8105f10 msgr2=0x7fcda819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcda5d9b640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda8105f10 0x7fcda819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcda5d9b640 1 -- 172.21.15.78:0/3996032526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcda81011e0 msgr2=0x7fcda81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcda5d9b640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcda81011e0 0x7fcda81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcda5d9b640 1 -- 172.21.15.78:0/3996032526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda819f390 con 0x7fcda8105590 2024-09-11T00:56:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcda5d9b640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcda8105590 0x7fcda8199820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcd98005940 tx=0x7fcd98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcd9f7fe640 1 -- 172.21.15.78:0/3996032526 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9803f030 con 0x7fcda8105590 2024-09-11T00:56:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcd9f7fe640 1 -- 172.21.15.78:0/3996032526 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd98003aa0 con 0x7fcda8105590 2024-09-11T00:56:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcdac8ed640 1 -- 172.21.15.78:0/3996032526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda819f620 con 0x7fcda8105590 2024-09-11T00:56:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcd9f7fe640 1 -- 172.21.15.78:0/3996032526 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98035070 con 0x7fcda8105590 2024-09-11T00:56:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.957+0000 7fcdac8ed640 1 -- 172.21.15.78:0/3996032526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda81a5d40 con 0x7fcda8105590 2024-09-11T00:56:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.958+0000 7fcdac8ed640 1 -- 172.21.15.78:0/3996032526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda81015c0 con 0x7fcda8105590 2024-09-11T00:56:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.960+0000 7fcd9f7fe640 1 -- 172.21.15.78:0/3996032526 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fcd9803bce0 con 0x7fcda8105590 2024-09-11T00:56:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.960+0000 7fcd9f7fe640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcd940b53b0 0x7fcd940b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.960+0000 7fcda659c640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcd940b53b0 0x7fcd940b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.960+0000 7fcd9f7fe640 1 -- 172.21.15.78:0/3996032526 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fcd981062b0 con 0x7fcda8105590 2024-09-11T00:56:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.961+0000 7fcda659c640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcd940b53b0 0x7fcd940b7870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcd900058c0 tx=0x7fcd90005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:41.964+0000 7fcd9f7fe640 1 -- 172.21.15.78:0/3996032526 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcd980cd540 con 0x7fcda8105590 2024-09-11T00:56:42.061 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 00:56:41 smithi110 podman[65601]: 2024-09-11 00:56:41.67500912 +0000 UTC m=+0.015955087 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:56:42.061 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 00:56:41 smithi110 podman[65601]: 2024-09-11 00:56:41.81947646 +0000 UTC m=+0.160422417 container create 7019b796cf94467945c0bc98a0eec9567e0a29528c2a00aa4fd99d3503d9181e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T00:56:42.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.176+0000 7fcdac8ed640 1 -- 172.21.15.78:0/3996032526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcda819a410 con 0x7fcda8105590 2024-09-11T00:56:42.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.177+0000 7fcd9f7fe640 1 -- 172.21.15.78:0/3996032526 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9701 (secure 0 0 0) 0x7fcd980cce40 con 0x7fcda8105590 2024-09-11T00:56:42.179 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:42.179 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":12,"btime":"2024-09-11T00:56:40:070220+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24529,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24545,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24563,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24580,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T00:56:42.180 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 12 2024-09-11T00:56:42.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.180+0000 7fcdac8ed640 1 -- 172.21.15.78:0/3996032526 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcd940b53b0 msgr2=0x7fcd940b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:42.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.180+0000 7fcdac8ed640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcd940b53b0 0x7fcd940b7870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcd900058c0 tx=0x7fcd90005850 comp rx=0 tx=0).stop 2024-09-11T00:56:42.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.180+0000 7fcdac8ed640 1 -- 172.21.15.78:0/3996032526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcda8105590 msgr2=0x7fcda8199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:42.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.180+0000 7fcdac8ed640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcda8105590 0x7fcda8199820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcd98005940 tx=0x7fcd98004270 comp rx=0 tx=0).stop 2024-09-11T00:56:42.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.180+0000 7fcdac8ed640 1 -- 172.21.15.78:0/3996032526 shutdown_connections 2024-09-11T00:56:42.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.180+0000 7fcdac8ed640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcd940b53b0 0x7fcd940b7870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:42.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.180+0000 7fcdac8ed640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcda8105f10 0x7fcda819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:42.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.180+0000 7fcdac8ed640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcda8105590 0x7fcda8199820 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:42.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.180+0000 7fcdac8ed640 1 --2- 172.21.15.78:0/3996032526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcda81011e0 0x7fcda81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:42.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.180+0000 7fcdac8ed640 1 -- 172.21.15.78:0/3996032526 >> 172.21.15.78:0/3996032526 conn(0x7fcda8078060 msgr2=0x7fcda8108f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:42.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.180+0000 7fcdac8ed640 1 -- 172.21.15.78:0/3996032526 shutdown_connections 2024-09-11T00:56:42.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:42.180+0000 7fcdac8ed640 1 -- 172.21.15.78:0/3996032526 wait complete. 2024-09-11T00:56:42.440 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 00:56:42 smithi110 podman[65601]: 2024-09-11 00:56:42.060178348 +0000 UTC m=+0.401124305 container init 7019b796cf94467945c0bc98a0eec9567e0a29528c2a00aa4fd99d3503d9181e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:56:42.440 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 00:56:42 smithi110 podman[65601]: 2024-09-11 00:56:42.063293025 +0000 UTC m=+0.404238983 container start 7019b796cf94467945c0bc98a0eec9567e0a29528c2a00aa4fd99d3503d9181e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:56:42.440 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 00:56:42 smithi110 ceph-mds[65621]: -- 172.21.15.110:0/810595966 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56335fc8b860 con 0x56335fc69400 2024-09-11T00:56:42.441 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 00:56:42 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b[65617]: starting mds.b at 2024-09-11T00:56:42.441 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 00:56:42 smithi110 bash[65601]: 7019b796cf94467945c0bc98a0eec9567e0a29528c2a00aa4fd99d3503d9181e 2024-09-11T00:56:42.441 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 00:56:42 smithi110 systemd[1]: Started Ceph mds.b for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T00:56:43.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:43 smithi112 ceph-mon[35384]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:43.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:43 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3996032526' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:43.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:56:43.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:43 smithi078 ceph-mon[31315]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:43.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:43 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3996032526' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:56:43.407 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:43 smithi110 ceph-mon[34712]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:43.407 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:43 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3996032526' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:43.407 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.407 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.408 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.408 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.408 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:43.408 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:56:44.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:44 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] up:boot 2024-09-11T00:56:44.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:44 smithi110 ceph-mon[34712]: fsmap 12 up:standby 2024-09-11T00:56:44.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:56:44.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:44.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:44.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:44.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:44 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] up:boot 2024-09-11T00:56:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:44 smithi112 ceph-mon[35384]: fsmap 12 up:standby 2024-09-11T00:56:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:56:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:44.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:44 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] up:boot 2024-09-11T00:56:44.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:44 smithi078 ceph-mon[31315]: fsmap 12 up:standby 2024-09-11T00:56:44.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:56:44.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:44.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:44.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:44.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:44.901 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:56:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:45 smithi078 ceph-mon[31315]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:56:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:56:45.357 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:45 smithi110 ceph-mon[34712]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:45.358 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:45.358 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:45.358 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:45.358 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:56:45.358 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:45.358 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:56:45.369 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:45 smithi112 ceph-mon[35384]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:45.369 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:45.369 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:45.369 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:45.369 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:56:45.369 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:45.369 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:56:45.805 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:46.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.319+0000 7f718321b640 1 -- 172.21.15.78:0/643014591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f717c108020 msgr2=0x7f717c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:46.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.319+0000 7f718321b640 1 --2- 172.21.15.78:0/643014591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f717c108020 0x7f717c10e820 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7174009900 tx=0x7f717402a410 comp rx=0 tx=0).stop 2024-09-11T00:56:46.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.319+0000 7f718321b640 1 -- 172.21.15.78:0/643014591 shutdown_connections 2024-09-11T00:56:46.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.319+0000 7f718321b640 1 --2- 172.21.15.78:0/643014591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f717c108020 0x7f717c10e820 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:46.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.319+0000 7f718321b640 1 --2- 172.21.15.78:0/643014591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c1076a0 0x7f717c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:46.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.319+0000 7f718321b640 1 --2- 172.21.15.78:0/643014591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f717c101c90 0x7f717c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:46.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.319+0000 7f718321b640 1 -- 172.21.15.78:0/643014591 >> 172.21.15.78:0/643014591 conn(0x7f717c0fd700 msgr2=0x7f717c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:46.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.319+0000 7f718321b640 1 -- 172.21.15.78:0/643014591 shutdown_connections 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.319+0000 7f718321b640 1 -- 172.21.15.78:0/643014591 wait complete. 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.319+0000 7f718321b640 1 Processor -- start 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.319+0000 7f718321b640 1 -- start start 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f718321b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c101c90 0x7f717c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f718321b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f717c1076a0 0x7f717c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f718321b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f717c108020 0x7f717c19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f718321b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f717c112e00 con 0x7f717c1076a0 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f718321b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f717c112c80 con 0x7f717c101c90 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f718321b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f717c112f80 con 0x7f717c108020 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f717bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f717c1076a0 0x7f717c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f717bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f717c1076a0 0x7f717c199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60692/0 (socket says 172.21.15.78:60692) 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f717bfff640 1 -- 172.21.15.78:0/3548951441 learned_addr learned my addr 172.21.15.78:0/3548951441 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f71817b8640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f717c108020 0x7f717c19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f7180fb7640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c101c90 0x7f717c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f717bfff640 1 -- 172.21.15.78:0/3548951441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f717c108020 msgr2=0x7f717c19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f717bfff640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f717c108020 0x7f717c19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f717bfff640 1 -- 172.21.15.78:0/3548951441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c101c90 msgr2=0x7f717c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f717bfff640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c101c90 0x7f717c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f717bfff640 1 -- 172.21.15.78:0/3548951441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c19f280 con 0x7f717c1076a0 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f717bfff640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f717c1076a0 0x7f717c199710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f717000cce0 tx=0x7f717000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f7179ffb640 1 -- 172.21.15.78:0/3548951441 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717000f650 con 0x7f717c1076a0 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f7179ffb640 1 -- 172.21.15.78:0/3548951441 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f717000fca0 con 0x7f717c1076a0 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f7179ffb640 1 -- 172.21.15.78:0/3548951441 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170018b40 con 0x7f717c1076a0 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f718321b640 1 -- 172.21.15.78:0/3548951441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c19f570 con 0x7f717c1076a0 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.320+0000 7f718321b640 1 -- 172.21.15.78:0/3548951441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c1a5e00 con 0x7f717c1076a0 2024-09-11T00:56:46.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.321+0000 7f718321b640 1 -- 172.21.15.78:0/3548951441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f717c102070 con 0x7f717c1076a0 2024-09-11T00:56:46.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.323+0000 7f7179ffb640 1 -- 172.21.15.78:0/3548951441 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f717000f7f0 con 0x7f717c1076a0 2024-09-11T00:56:46.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.323+0000 7f7179ffb640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71640b53b0 0x7f71640b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:46.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.323+0000 7f7180fb7640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71640b53b0 0x7f71640b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:46.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.323+0000 7f7179ffb640 1 -- 172.21.15.78:0/3548951441 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f71700e5c10 con 0x7f717c1076a0 2024-09-11T00:56:46.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.324+0000 7f7180fb7640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71640b53b0 0x7f71640b7870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f716c009bd0 tx=0x7f716c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:46.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.327+0000 7f7179ffb640 1 -- 172.21.15.78:0/3548951441 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7170014030 con 0x7f717c1076a0 2024-09-11T00:56:46.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.600+0000 7f718321b640 1 -- 172.21.15.78:0/3548951441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f717c1a5fd0 con 0x7f717c1076a0 2024-09-11T00:56:46.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.601+0000 7f7179ffb640 1 -- 172.21.15.78:0/3548951441 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10530 (secure 0 0 0) 0x7f71700acea0 con 0x7f717c1076a0 2024-09-11T00:56:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:46.603 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":13,"btime":"2024-09-11T00:56:43:069197+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24529,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24545,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24553,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24563,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24580,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-11T00:56:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 13 2024-09-11T00:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.604+0000 7f718321b640 1 -- 172.21.15.78:0/3548951441 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71640b53b0 msgr2=0x7f71640b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.604+0000 7f718321b640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71640b53b0 0x7f71640b7870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f716c009bd0 tx=0x7f716c005f90 comp rx=0 tx=0).stop 2024-09-11T00:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.604+0000 7f718321b640 1 -- 172.21.15.78:0/3548951441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f717c1076a0 msgr2=0x7f717c199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.604+0000 7f718321b640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f717c1076a0 0x7f717c199710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f717000cce0 tx=0x7f717000a620 comp rx=0 tx=0).stop 2024-09-11T00:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.604+0000 7f718321b640 1 -- 172.21.15.78:0/3548951441 shutdown_connections 2024-09-11T00:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.604+0000 7f718321b640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71640b53b0 0x7f71640b7870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.604+0000 7f718321b640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f717c108020 0x7f717c19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.604+0000 7f718321b640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f717c1076a0 0x7f717c199710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.604+0000 7f718321b640 1 --2- 172.21.15.78:0/3548951441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c101c90 0x7f717c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.604+0000 7f718321b640 1 -- 172.21.15.78:0/3548951441 >> 172.21.15.78:0/3548951441 conn(0x7f717c0fd700 msgr2=0x7f717c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.604+0000 7f718321b640 1 -- 172.21.15.78:0/3548951441 shutdown_connections 2024-09-11T00:56:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:46.604+0000 7f718321b640 1 -- 172.21.15.78:0/3548951441 wait complete. 2024-09-11T00:56:47.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:47 smithi078 ceph-mon[31315]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:47.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:47 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3548951441' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:47.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:47.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:56:47.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.414 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-11T00:56:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:47 smithi110 ceph-mon[34712]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:47 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3548951441' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:47.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:47.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:56:47.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:47 smithi112 ceph-mon[35384]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:47 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3548951441' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:56:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:56:47.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:56:47.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:56:47.660 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.132+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/2753554166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a68109150 msgr2=0x7f7a6810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.132+0000 7f7a6cf5f640 1 --2- 172.21.15.78:0/2753554166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a68109150 0x7f7a6810f950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009ca0 tx=0x7f7a6002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.132+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/2753554166 shutdown_connections 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.132+0000 7f7a6cf5f640 1 --2- 172.21.15.78:0/2753554166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a68109150 0x7f7a6810f950 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.132+0000 7f7a6cf5f640 1 --2- 172.21.15.78:0/2753554166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a681087d0 0x7f7a68108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.132+0000 7f7a6cf5f640 1 --2- 172.21.15.78:0/2753554166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a68102dc0 0x7f7a681031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.132+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/2753554166 >> 172.21.15.78:0/2753554166 conn(0x7f7a680fe830 msgr2=0x7f7a68100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.132+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/2753554166 shutdown_connections 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.132+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/2753554166 wait complete. 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a6cf5f640 1 Processor -- start 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a6cf5f640 1 -- start start 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a6cf5f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a68102dc0 0x7f7a68199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a6cf5f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a681087d0 0x7f7a681998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a6cf5f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a68109150 0x7f7a6819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a6cf5f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68112e50 con 0x7f7a681087d0 2024-09-11T00:56:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a6cf5f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68112cd0 con 0x7f7a68109150 2024-09-11T00:56:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a6cf5f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68112fd0 con 0x7f7a68102dc0 2024-09-11T00:56:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a5ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a681087d0 0x7f7a681998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a5ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a681087d0 0x7f7a681998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60700/0 (socket says 172.21.15.78:60700) 2024-09-11T00:56:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a5ffff640 1 -- 172.21.15.78:0/109058001 learned_addr learned my addr 172.21.15.78:0/109058001 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a66d9d640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a68109150 0x7f7a6819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a5ffff640 1 -- 172.21.15.78:0/109058001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a68102dc0 msgr2=0x7f7a68199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a5ffff640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a68102dc0 0x7f7a68199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a5ffff640 1 -- 172.21.15.78:0/109058001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a68109150 msgr2=0x7f7a6819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a5ffff640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a68109150 0x7f7a6819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a5ffff640 1 -- 172.21.15.78:0/109058001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6819f430 con 0x7f7a681087d0 2024-09-11T00:56:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.133+0000 7f7a5ffff640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a681087d0 0x7f7a681998c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000b4a0 tx=0x7f7a5000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.134+0000 7f7a5f7fe640 1 -- 172.21.15.78:0/109058001 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50014070 con 0x7f7a681087d0 2024-09-11T00:56:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.134+0000 7f7a5f7fe640 1 -- 172.21.15.78:0/109058001 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a50010ac0 con 0x7f7a681087d0 2024-09-11T00:56:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.134+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/109058001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6819f720 con 0x7f7a681087d0 2024-09-11T00:56:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.134+0000 7f7a5f7fe640 1 -- 172.21.15.78:0/109058001 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50017c30 con 0x7f7a681087d0 2024-09-11T00:56:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.134+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/109058001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a681a5e60 con 0x7f7a681087d0 2024-09-11T00:56:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.135+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/109058001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a68103220 con 0x7f7a681087d0 2024-09-11T00:56:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.136+0000 7f7a5f7fe640 1 -- 172.21.15.78:0/109058001 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f7a50010610 con 0x7f7a681087d0 2024-09-11T00:56:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.137+0000 7f7a5f7fe640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7a540b5330 0x7f7a540b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.137+0000 7f7a6659c640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7a540b5330 0x7f7a540b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.137+0000 7f7a5f7fe640 1 -- 172.21.15.78:0/109058001 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f7a500e5220 con 0x7f7a681087d0 2024-09-11T00:56:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.137+0000 7f7a6659c640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7a540b5330 0x7f7a540b77f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009710 tx=0x7f7a58006c90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.140+0000 7f7a5f7fe640 1 -- 172.21.15.78:0/109058001 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7a500ac410 con 0x7f7a681087d0 2024-09-11T00:56:48.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.318+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/109058001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7a6819a4b0 con 0x7f7a681087d0 2024-09-11T00:56:48.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.319+0000 7f7a5f7fe640 1 -- 172.21.15.78:0/109058001 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7f7a500abd10 con 0x7f7a681087d0 2024-09-11T00:56:48.320 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T00:56:48.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T00:56:48.321 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T00:56:48.321 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T00:56:48.321 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T00:56:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T00:56:48.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.320+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/109058001 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7a540b5330 msgr2=0x7f7a540b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:48.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.321+0000 7f7a6cf5f640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7a540b5330 0x7f7a540b77f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009710 tx=0x7f7a58006c90 comp rx=0 tx=0).stop 2024-09-11T00:56:48.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.321+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/109058001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a681087d0 msgr2=0x7f7a681998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:48.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.321+0000 7f7a6cf5f640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a681087d0 0x7f7a681998c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000b4a0 tx=0x7f7a5000b960 comp rx=0 tx=0).stop 2024-09-11T00:56:48.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.321+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/109058001 shutdown_connections 2024-09-11T00:56:48.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.321+0000 7f7a6cf5f640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7a540b5330 0x7f7a540b77f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:48.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.321+0000 7f7a6cf5f640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a68109150 0x7f7a6819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:48.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.321+0000 7f7a6cf5f640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a681087d0 0x7f7a681998c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:48.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.321+0000 7f7a6cf5f640 1 --2- 172.21.15.78:0/109058001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a68102dc0 0x7f7a68199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:48.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.321+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/109058001 >> 172.21.15.78:0/109058001 conn(0x7f7a680fe830 msgr2=0x7f7a68100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:48.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.321+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/109058001 shutdown_connections 2024-09-11T00:56:48.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:48.321+0000 7f7a6cf5f640 1 -- 172.21.15.78:0/109058001 wait complete. 2024-09-11T00:56:49.010 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T00:56:49.011 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T00:56:49.260 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:49.380 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:49 smithi078 ceph-mon[31315]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:49.380 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:49 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/109058001' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-11T00:56:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:49 smithi110 ceph-mon[34712]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:49 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/109058001' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-11T00:56:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:49 smithi112 ceph-mon[35384]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:49 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/109058001' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-11T00:56:49.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.751+0000 7fb6c3956640 1 -- 172.21.15.78:0/1886461994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6bc109150 msgr2=0x7fb6bc10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:49.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.751+0000 7fb6c3956640 1 --2- 172.21.15.78:0/1886461994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6bc109150 0x7fb6bc10f950 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009ca0 tx=0x7fb6b402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:56:49.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.751+0000 7fb6c3956640 1 -- 172.21.15.78:0/1886461994 shutdown_connections 2024-09-11T00:56:49.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.751+0000 7fb6c3956640 1 --2- 172.21.15.78:0/1886461994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6bc109150 0x7fb6bc10f950 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:49.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.751+0000 7fb6c3956640 1 --2- 172.21.15.78:0/1886461994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6bc1087d0 0x7fb6bc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:49.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.751+0000 7fb6c3956640 1 --2- 172.21.15.78:0/1886461994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6bc102dc0 0x7fb6bc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:49.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.751+0000 7fb6c3956640 1 -- 172.21.15.78:0/1886461994 >> 172.21.15.78:0/1886461994 conn(0x7fb6bc0fe830 msgr2=0x7fb6bc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:49.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.751+0000 7fb6c3956640 1 -- 172.21.15.78:0/1886461994 shutdown_connections 2024-09-11T00:56:49.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.751+0000 7fb6c3956640 1 -- 172.21.15.78:0/1886461994 wait complete. 2024-09-11T00:56:49.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c3956640 1 Processor -- start 2024-09-11T00:56:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c3956640 1 -- start start 2024-09-11T00:56:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c3956640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6bc102dc0 0x7fb6bc199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c3956640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6bc1087d0 0x7fb6bc199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c3956640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6bc109150 0x7fb6bc19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c3956640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc112e10 con 0x7fb6bc102dc0 2024-09-11T00:56:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c3956640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc112c90 con 0x7fb6bc109150 2024-09-11T00:56:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c3956640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc112f90 con 0x7fb6bc1087d0 2024-09-11T00:56:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c16f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6bc102dc0 0x7fb6bc199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c16f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6bc102dc0 0x7fb6bc199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60720/0 (socket says 172.21.15.78:60720) 2024-09-11T00:56:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c16f2640 1 -- 172.21.15.78:0/1064760158 learned_addr learned my addr 172.21.15.78:0/1064760158 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c1ef3640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6bc109150 0x7fb6bc19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c16f2640 1 -- 172.21.15.78:0/1064760158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6bc1087d0 msgr2=0x7fb6bc199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c16f2640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6bc1087d0 0x7fb6bc199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c16f2640 1 -- 172.21.15.78:0/1064760158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6bc109150 msgr2=0x7fb6bc19ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c16f2640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6bc109150 0x7fb6bc19ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c16f2640 1 -- 172.21.15.78:0/1064760158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6bc19f1f0 con 0x7fb6bc102dc0 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6c16f2640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6bc102dc0 0x7fb6bc199140 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800c920 tx=0x7fb6b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6b27fc640 1 -- 172.21.15.78:0/1064760158 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8007c70 con 0x7fb6bc102dc0 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.752+0000 7fb6b27fc640 1 -- 172.21.15.78:0/1064760158 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6b8007e10 con 0x7fb6bc102dc0 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.753+0000 7fb6c3956640 1 -- 172.21.15.78:0/1064760158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6bc19f4e0 con 0x7fb6bc102dc0 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.753+0000 7fb6b27fc640 1 -- 172.21.15.78:0/1064760158 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8018820 con 0x7fb6bc102dc0 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.753+0000 7fb6c3956640 1 -- 172.21.15.78:0/1064760158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6bc1a5ba0 con 0x7fb6bc102dc0 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.753+0000 7fb6c3956640 1 -- 172.21.15.78:0/1064760158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6bc1031a0 con 0x7fb6bc102dc0 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.755+0000 7fb6b27fc640 1 -- 172.21.15.78:0/1064760158 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fb6b800f450 con 0x7fb6bc102dc0 2024-09-11T00:56:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.756+0000 7fb6b27fc640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb6a80b5330 0x7fb6a80b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:49.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.756+0000 7fb6c0ef1640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb6a80b5330 0x7fb6a80b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:49.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.756+0000 7fb6b27fc640 1 -- 172.21.15.78:0/1064760158 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fb6b80e5ce0 con 0x7fb6bc102dc0 2024-09-11T00:56:49.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.756+0000 7fb6c0ef1640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb6a80b5330 0x7fb6a80b77f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc100f90 tx=0x7fb6ac005c70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:49.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.758+0000 7fb6b27fc640 1 -- 172.21.15.78:0/1064760158 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb6b8014030 con 0x7fb6bc102dc0 2024-09-11T00:56:49.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:49.934+0000 7fb6c3956640 1 -- 172.21.15.78:0/1064760158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb6bc1a6000 con 0x7fb6bc102dc0 2024-09-11T00:56:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.083+0000 7fb6b27fc640 1 -- 172.21.15.78:0/1064760158 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7fb6b80ad000 con 0x7fb6bc102dc0 2024-09-11T00:56:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T00:56:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.086+0000 7fb6c3956640 1 -- 172.21.15.78:0/1064760158 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb6a80b5330 msgr2=0x7fb6a80b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.086+0000 7fb6c3956640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb6a80b5330 0x7fb6a80b77f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc100f90 tx=0x7fb6ac005c70 comp rx=0 tx=0).stop 2024-09-11T00:56:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.086+0000 7fb6c3956640 1 -- 172.21.15.78:0/1064760158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6bc102dc0 msgr2=0x7fb6bc199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.086+0000 7fb6c3956640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6bc102dc0 0x7fb6bc199140 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800c920 tx=0x7fb6b800cde0 comp rx=0 tx=0).stop 2024-09-11T00:56:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.086+0000 7fb6c3956640 1 -- 172.21.15.78:0/1064760158 shutdown_connections 2024-09-11T00:56:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.086+0000 7fb6c3956640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb6a80b5330 0x7fb6a80b77f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.086+0000 7fb6c3956640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6bc109150 0x7fb6bc19ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.086+0000 7fb6c3956640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6bc1087d0 0x7fb6bc199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.086+0000 7fb6c3956640 1 --2- 172.21.15.78:0/1064760158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6bc102dc0 0x7fb6bc199140 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.086+0000 7fb6c3956640 1 -- 172.21.15.78:0/1064760158 >> 172.21.15.78:0/1064760158 conn(0x7fb6bc0fe830 msgr2=0x7fb6bc101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.086+0000 7fb6c3956640 1 -- 172.21.15.78:0/1064760158 shutdown_connections 2024-09-11T00:56:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:50.086+0000 7fb6c3956640 1 -- 172.21.15.78:0/1064760158 wait complete. 2024-09-11T00:56:50.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:50 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1064760158' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:56:50.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:50 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1064760158' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:56:50.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:50 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1064760158' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:56:50.788 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T00:56:51.033 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:51.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:51 smithi078 ceph-mon[31315]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:51.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:51 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1064760158' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T00:56:51.430 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:51 smithi078 ceph-mon[31315]: osdmap e71: 12 total, 12 up, 12 in 2024-09-11T00:56:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:51 smithi110 ceph-mon[34712]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:51 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1064760158' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T00:56:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:51 smithi110 ceph-mon[34712]: osdmap e71: 12 total, 12 up, 12 in 2024-09-11T00:56:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:51 smithi112 ceph-mon[35384]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:51 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1064760158' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T00:56:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:51 smithi112 ceph-mon[35384]: osdmap e71: 12 total, 12 up, 12 in 2024-09-11T00:56:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.514+0000 7f2d833a2640 1 -- 172.21.15.78:0/2166890927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c107510 msgr2=0x7f2d7c107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.514+0000 7f2d833a2640 1 --2- 172.21.15.78:0/2166890927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c107510 0x7f2d7c107950 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009900 tx=0x7f2d7002a410 comp rx=0 tx=0).stop 2024-09-11T00:56:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.514+0000 7f2d833a2640 1 -- 172.21.15.78:0/2166890927 shutdown_connections 2024-09-11T00:56:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.514+0000 7f2d833a2640 1 --2- 172.21.15.78:0/2166890927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c107e90 0x7f2d7c10e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.514+0000 7f2d833a2640 1 --2- 172.21.15.78:0/2166890927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c107510 0x7f2d7c107950 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.514+0000 7f2d833a2640 1 --2- 172.21.15.78:0/2166890927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d7c101b00 0x7f2d7c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.514+0000 7f2d833a2640 1 -- 172.21.15.78:0/2166890927 >> 172.21.15.78:0/2166890927 conn(0x7f2d7c0fd5b0 msgr2=0x7f2d7c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.514+0000 7f2d833a2640 1 -- 172.21.15.78:0/2166890927 shutdown_connections 2024-09-11T00:56:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.514+0000 7f2d833a2640 1 -- 172.21.15.78:0/2166890927 wait complete. 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.514+0000 7f2d833a2640 1 Processor -- start 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.514+0000 7f2d833a2640 1 -- start start 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d833a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c101b00 0x7f2d7c199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d833a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c107510 0x7f2d7c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d833a2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d7c107e90 0x7f2d7c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d833a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c112c70 con 0x7f2d7c101b00 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d833a2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c112af0 con 0x7f2d7c107e90 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d833a2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c112df0 con 0x7f2d7c107510 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d823a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c101b00 0x7f2d7c199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d823a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c101b00 0x7f2d7c199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60736/0 (socket says 172.21.15.78:60736) 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d823a0640 1 -- 172.21.15.78:0/4049617227 learned_addr learned my addr 172.21.15.78:0/4049617227 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d81b9f640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c107510 0x7f2d7c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d82ba1640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d7c107e90 0x7f2d7c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d823a0640 1 -- 172.21.15.78:0/4049617227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c107510 msgr2=0x7f2d7c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d823a0640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c107510 0x7f2d7c1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d823a0640 1 -- 172.21.15.78:0/4049617227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d7c107e90 msgr2=0x7f2d7c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d823a0640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d7c107e90 0x7f2d7c19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d823a0640 1 -- 172.21.15.78:0/4049617227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c19f110 con 0x7f2d7c101b00 2024-09-11T00:56:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d81b9f640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c107510 0x7f2d7c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:56:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d823a0640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c101b00 0x7f2d7c199060 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00d950 tx=0x7f2d6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d7b7fe640 1 -- 172.21.15.78:0/4049617227 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c017070 con 0x7f2d7c101b00 2024-09-11T00:56:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d833a2640 1 -- 172.21.15.78:0/4049617227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7c19f400 con 0x7f2d7c101b00 2024-09-11T00:56:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d7b7fe640 1 -- 172.21.15.78:0/4049617227 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d6c009c30 con 0x7f2d7c101b00 2024-09-11T00:56:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d7b7fe640 1 -- 172.21.15.78:0/4049617227 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c00f620 con 0x7f2d7c101b00 2024-09-11T00:56:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.515+0000 7f2d833a2640 1 -- 172.21.15.78:0/4049617227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7c1a5c90 con 0x7f2d7c101b00 2024-09-11T00:56:51.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.516+0000 7f2d833a2640 1 -- 172.21.15.78:0/4049617227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d7c102db0 con 0x7f2d7c101b00 2024-09-11T00:56:51.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.518+0000 7f2d7b7fe640 1 -- 172.21.15.78:0/4049617227 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f2d6c010460 con 0x7f2d7c101b00 2024-09-11T00:56:51.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.518+0000 7f2d7b7fe640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2d640b5460 0x7f2d640b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:51.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.519+0000 7f2d81b9f640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2d640b5460 0x7f2d640b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:51.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.519+0000 7f2d7b7fe640 1 -- 172.21.15.78:0/4049617227 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7712+0+0 (secure 0 0 0) 0x7f2d6c0e5640 con 0x7f2d7c101b00 2024-09-11T00:56:51.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.519+0000 7f2d81b9f640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2d640b5460 0x7f2d640b7920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d70031440 tx=0x7f2d70005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:51.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.522+0000 7f2d7b7fe640 1 -- 172.21.15.78:0/4049617227 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2d6c0ac7e0 con 0x7f2d7c101b00 2024-09-11T00:56:51.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:51.698+0000 7f2d833a2640 1 -- 172.21.15.78:0/4049617227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2d7c101ee0 con 0x7f2d7c101b00 2024-09-11T00:56:52.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.097+0000 7f2d7b7fe640 1 -- 172.21.15.78:0/4049617227 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7f2d7c101ee0 con 0x7f2d7c101b00 2024-09-11T00:56:52.099 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T00:56:52.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.100+0000 7f2d833a2640 1 -- 172.21.15.78:0/4049617227 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2d640b5460 msgr2=0x7f2d640b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:52.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.100+0000 7f2d833a2640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2d640b5460 0x7f2d640b7920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d70031440 tx=0x7f2d70005a20 comp rx=0 tx=0).stop 2024-09-11T00:56:52.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.100+0000 7f2d833a2640 1 -- 172.21.15.78:0/4049617227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c101b00 msgr2=0x7f2d7c199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:52.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.100+0000 7f2d833a2640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c101b00 0x7f2d7c199060 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00d950 tx=0x7f2d6c00de10 comp rx=0 tx=0).stop 2024-09-11T00:56:52.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.101+0000 7f2d833a2640 1 -- 172.21.15.78:0/4049617227 shutdown_connections 2024-09-11T00:56:52.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.101+0000 7f2d833a2640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2d640b5460 0x7f2d640b7920 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:52.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.101+0000 7f2d833a2640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d7c107e90 0x7f2d7c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:52.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.101+0000 7f2d833a2640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c107510 0x7f2d7c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:52.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.101+0000 7f2d833a2640 1 --2- 172.21.15.78:0/4049617227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c101b00 0x7f2d7c199060 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00d950 tx=0x7f2d6c00de10 comp rx=0 tx=0).stop 2024-09-11T00:56:52.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.101+0000 7f2d833a2640 1 -- 172.21.15.78:0/4049617227 >> 172.21.15.78:0/4049617227 conn(0x7f2d7c0fd5b0 msgr2=0x7f2d7c0ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:52.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.101+0000 7f2d833a2640 1 -- 172.21.15.78:0/4049617227 shutdown_connections 2024-09-11T00:56:52.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:52.103+0000 7f2d833a2640 1 -- 172.21.15.78:0/4049617227 wait complete. 2024-09-11T00:56:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:52 smithi110 ceph-mon[34712]: osdmap e72: 12 total, 12 up, 12 in 2024-09-11T00:56:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:52 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/4049617227' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:56:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:52 smithi112 ceph-mon[35384]: osdmap e72: 12 total, 12 up, 12 in 2024-09-11T00:56:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:52 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/4049617227' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:56:52.483 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:52 smithi078 ceph-mon[31315]: osdmap e72: 12 total, 12 up, 12 in 2024-09-11T00:56:52.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:52 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4049617227' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:56:52.829 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T00:56:53.072 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:53 smithi110 ceph-mon[34712]: pgmap v258: 33 pgs: 5 creating+activating, 8 creating+peering, 19 unknown, 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:53 smithi110 ceph-mon[34712]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:56:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:53 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/4049617227' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T00:56:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:53 smithi110 ceph-mon[34712]: osdmap e73: 12 total, 12 up, 12 in 2024-09-11T00:56:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:53 smithi112 ceph-mon[35384]: pgmap v258: 33 pgs: 5 creating+activating, 8 creating+peering, 19 unknown, 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:53 smithi112 ceph-mon[35384]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:56:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:53 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/4049617227' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T00:56:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:53 smithi112 ceph-mon[35384]: osdmap e73: 12 total, 12 up, 12 in 2024-09-11T00:56:53.490 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:53 smithi078 ceph-mon[31315]: pgmap v258: 33 pgs: 5 creating+activating, 8 creating+peering, 19 unknown, 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:53.490 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:53 smithi078 ceph-mon[31315]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:56:53.490 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:53 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4049617227' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T00:56:53.490 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:53 smithi078 ceph-mon[31315]: osdmap e73: 12 total, 12 up, 12 in 2024-09-11T00:56:53.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.570+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3021174024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb41076c0 msgr2=0x7f2fb4107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:53.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.570+0000 7f2fbc71f640 1 --2- 172.21.15.78:0/3021174024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb41076c0 0x7f2fb4107b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009900 tx=0x7f2fa402a410 comp rx=0 tx=0).stop 2024-09-11T00:56:53.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.570+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3021174024 shutdown_connections 2024-09-11T00:56:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.570+0000 7f2fbc71f640 1 --2- 172.21.15.78:0/3021174024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4108040 0x7f2fb410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.570+0000 7f2fbc71f640 1 --2- 172.21.15.78:0/3021174024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb41076c0 0x7f2fb4107b00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.570+0000 7f2fbc71f640 1 --2- 172.21.15.78:0/3021174024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fb4101cb0 0x7f2fb4102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.570+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3021174024 >> 172.21.15.78:0/3021174024 conn(0x7f2fb40fd700 msgr2=0x7f2fb40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.570+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3021174024 shutdown_connections 2024-09-11T00:56:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.570+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3021174024 wait complete. 2024-09-11T00:56:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.571+0000 7f2fbc71f640 1 Processor -- start 2024-09-11T00:56:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.571+0000 7f2fbc71f640 1 -- start start 2024-09-11T00:56:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.571+0000 7f2fbc71f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4101cb0 0x7f2fb41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.571+0000 7f2fbc71f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb41076c0 0x7f2fb4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:53.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.571+0000 7f2fbc71f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fb4108040 0x7f2fb419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:53.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.571+0000 7f2fbc71f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4111d40 con 0x7f2fb41076c0 2024-09-11T00:56:53.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.571+0000 7f2fbc71f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4111bc0 con 0x7f2fb4108040 2024-09-11T00:56:53.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.571+0000 7f2fbc71f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4111ec0 con 0x7f2fb4101cb0 2024-09-11T00:56:53.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.571+0000 7f2fb9cba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb41076c0 0x7f2fb4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:53.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.571+0000 7f2fb9cba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb41076c0 0x7f2fb4198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38956/0 (socket says 172.21.15.78:38956) 2024-09-11T00:56:53.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fb9cba640 1 -- 172.21.15.78:0/3746330712 learned_addr learned my addr 172.21.15.78:0/3746330712 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:53.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fbacbc640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fb4108040 0x7f2fb419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:53.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fba4bb640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4101cb0 0x7f2fb41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:53.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fb9cba640 1 -- 172.21.15.78:0/3746330712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4101cb0 msgr2=0x7f2fb41982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fb9cba640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4101cb0 0x7f2fb41982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fb9cba640 1 -- 172.21.15.78:0/3746330712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fb4108040 msgr2=0x7f2fb419dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fb9cba640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fb4108040 0x7f2fb419dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fb9cba640 1 -- 172.21.15.78:0/3746330712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb419e3a0 con 0x7f2fb41076c0 2024-09-11T00:56:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fba4bb640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4101cb0 0x7f2fb41982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:56:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fbacbc640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fb4108040 0x7f2fb419dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:56:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fb9cba640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb41076c0 0x7f2fb4198830 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4006010 tx=0x7f2fa4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fab7fe640 1 -- 172.21.15.78:0/3746330712 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4035050 con 0x7f2fb41076c0 2024-09-11T00:56:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3746330712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb419e630 con 0x7f2fb41076c0 2024-09-11T00:56:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fab7fe640 1 -- 172.21.15.78:0/3746330712 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa40035f0 con 0x7f2fb41076c0 2024-09-11T00:56:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fab7fe640 1 -- 172.21.15.78:0/3746330712 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4032260 con 0x7f2fb41076c0 2024-09-11T00:56:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.572+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3746330712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb406b490 con 0x7f2fb41076c0 2024-09-11T00:56:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.573+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3746330712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb4102110 con 0x7f2fb41076c0 2024-09-11T00:56:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.575+0000 7f2fab7fe640 1 -- 172.21.15.78:0/3746330712 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f2fa4003aa0 con 0x7f2fb41076c0 2024-09-11T00:56:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.575+0000 7f2fab7fe640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2fa00b5330 0x7f2fa00b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.575+0000 7f2fba4bb640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2fa00b5330 0x7f2fa00b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.576+0000 7f2fab7fe640 1 -- 172.21.15.78:0/3746330712 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8094+0+0 (secure 0 0 0) 0x7f2fa4107d40 con 0x7f2fb41076c0 2024-09-11T00:56:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.576+0000 7f2fba4bb640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2fa00b5330 0x7f2fa00b77f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006fd0 tx=0x7f2fb0008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.578+0000 7f2fab7fe640 1 -- 172.21.15.78:0/3746330712 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2fa40ced70 con 0x7f2fb41076c0 2024-09-11T00:56:53.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:53.791+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3746330712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2fb406bfe0 con 0x7f2fb41076c0 2024-09-11T00:56:54.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.112+0000 7f2fab7fe640 1 -- 172.21.15.78:0/3746330712 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f2fa40ce670 con 0x7f2fb41076c0 2024-09-11T00:56:54.114 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 2 and data pool 3 2024-09-11T00:56:54.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.115+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3746330712 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2fa00b5330 msgr2=0x7f2fa00b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.115+0000 7f2fbc71f640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2fa00b5330 0x7f2fa00b77f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006fd0 tx=0x7f2fb0008040 comp rx=0 tx=0).stop 2024-09-11T00:56:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.115+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3746330712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb41076c0 msgr2=0x7f2fb4198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.115+0000 7f2fbc71f640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb41076c0 0x7f2fb4198830 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4006010 tx=0x7f2fa4004270 comp rx=0 tx=0).stop 2024-09-11T00:56:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.115+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3746330712 shutdown_connections 2024-09-11T00:56:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.115+0000 7f2fbc71f640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2fa00b5330 0x7f2fa00b77f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.115+0000 7f2fbc71f640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fb4108040 0x7f2fb419dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.115+0000 7f2fbc71f640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb41076c0 0x7f2fb4198830 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.115+0000 7f2fbc71f640 1 --2- 172.21.15.78:0/3746330712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4101cb0 0x7f2fb41982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.115+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3746330712 >> 172.21.15.78:0/3746330712 conn(0x7f2fb40fd700 msgr2=0x7f2fb40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.115+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3746330712 shutdown_connections 2024-09-11T00:56:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:54.115+0000 7f2fbc71f640 1 -- 172.21.15.78:0/3746330712 wait complete. 2024-09-11T00:56:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:54 smithi110 ceph-mon[34712]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T00:56:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:54 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3746330712' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-11T00:56:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:54 smithi110 ceph-mon[34712]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T00:56:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:54 smithi110 ceph-mon[34712]: pgmap v261: 97 pgs: 5 creating+activating, 18 creating+peering, 53 unknown, 21 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:54 smithi112 ceph-mon[35384]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T00:56:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:54 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3746330712' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-11T00:56:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:54 smithi112 ceph-mon[35384]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T00:56:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:54 smithi112 ceph-mon[35384]: pgmap v261: 97 pgs: 5 creating+activating, 18 creating+peering, 53 unknown, 21 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:54.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:54 smithi078 ceph-mon[31315]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T00:56:54.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:54 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3746330712' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-11T00:56:54.495 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:54 smithi078 ceph-mon[31315]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T00:56:54.495 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:54 smithi078 ceph-mon[31315]: pgmap v261: 97 pgs: 5 creating+activating, 18 creating+peering, 53 unknown, 21 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:56:54.847 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:56:55.093 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:55.380 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:55 smithi078 ceph-mon[31315]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T00:56:55.380 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:55 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3746330712' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T00:56:55.380 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:55 smithi078 ceph-mon[31315]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T00:56:55.380 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:55 smithi078 ceph-mon[31315]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T00:56:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:55 smithi110 ceph-mon[34712]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T00:56:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:55 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3746330712' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T00:56:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:55 smithi110 ceph-mon[34712]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T00:56:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:55 smithi110 ceph-mon[34712]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T00:56:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:55 smithi112 ceph-mon[35384]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T00:56:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:55 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3746330712' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T00:56:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:55 smithi112 ceph-mon[35384]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T00:56:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:55 smithi112 ceph-mon[35384]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T00:56:55.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.593+0000 7fee4f994640 1 -- 172.21.15.78:0/3171605527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee48101c90 msgr2=0x7fee48102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:55.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.593+0000 7fee4f994640 1 --2- 172.21.15.78:0/3171605527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee48101c90 0x7fee48102070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fee38009900 tx=0x7fee3802a410 comp rx=0 tx=0).stop 2024-09-11T00:56:55.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.593+0000 7fee4f994640 1 -- 172.21.15.78:0/3171605527 shutdown_connections 2024-09-11T00:56:55.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.593+0000 7fee4f994640 1 --2- 172.21.15.78:0/3171605527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee48108020 0x7fee4810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:55.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.593+0000 7fee4f994640 1 --2- 172.21.15.78:0/3171605527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee481076a0 0x7fee48107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.593+0000 7fee4f994640 1 --2- 172.21.15.78:0/3171605527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee48101c90 0x7fee48102070 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.593+0000 7fee4f994640 1 -- 172.21.15.78:0/3171605527 >> 172.21.15.78:0/3171605527 conn(0x7fee480fd700 msgr2=0x7fee480ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4f994640 1 -- 172.21.15.78:0/3171605527 shutdown_connections 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4f994640 1 -- 172.21.15.78:0/3171605527 wait complete. 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4f994640 1 Processor -- start 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4f994640 1 -- start start 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4f994640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee48101c90 0x7fee4819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4f994640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee481076a0 0x7fee4819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4f994640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee48108020 0x7fee4819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4f994640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee48111ce0 con 0x7fee48108020 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4f994640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee48111b60 con 0x7fee48101c90 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4f994640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee48111e60 con 0x7fee481076a0 2024-09-11T00:56:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4df31640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee48108020 0x7fee4819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4df31640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee48108020 0x7fee4819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38962/0 (socket says 172.21.15.78:38962) 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.594+0000 7fee4df31640 1 -- 172.21.15.78:0/3946145504 learned_addr learned my addr 172.21.15.78:0/3946145504 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee4cf2f640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee481076a0 0x7fee4819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee4d730640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee48101c90 0x7fee4819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee4df31640 1 -- 172.21.15.78:0/3946145504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee481076a0 msgr2=0x7fee4819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee4df31640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee481076a0 0x7fee4819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee4df31640 1 -- 172.21.15.78:0/3946145504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee48101c90 msgr2=0x7fee4819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee4df31640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee48101c90 0x7fee4819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee4df31640 1 -- 172.21.15.78:0/3946145504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee481a54f0 con 0x7fee48108020 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee4cf2f640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee481076a0 0x7fee4819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee4d730640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee48101c90 0x7fee4819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee4df31640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee48108020 0x7fee4819abb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fee4000b970 tx=0x7fee4000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee3e7fc640 1 -- 172.21.15.78:0/3946145504 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40019070 con 0x7fee48108020 2024-09-11T00:56:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee4f994640 1 -- 172.21.15.78:0/3946145504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee481a57e0 con 0x7fee48108020 2024-09-11T00:56:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee3e7fc640 1 -- 172.21.15.78:0/3946145504 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee40014440 con 0x7fee48108020 2024-09-11T00:56:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee3e7fc640 1 -- 172.21.15.78:0/3946145504 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee400075d0 con 0x7fee48108020 2024-09-11T00:56:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.595+0000 7fee4f994640 1 -- 172.21.15.78:0/3946145504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee481a5da0 con 0x7fee48108020 2024-09-11T00:56:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.596+0000 7fee4f994640 1 -- 172.21.15.78:0/3946145504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee48102070 con 0x7fee48108020 2024-09-11T00:56:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.597+0000 7fee3e7fc640 1 -- 172.21.15.78:0/3946145504 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fee4000cb40 con 0x7fee48108020 2024-09-11T00:56:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.598+0000 7fee3e7fc640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee340b5330 0x7fee340b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.598+0000 7fee4d730640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee340b5330 0x7fee340b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.598+0000 7fee3e7fc640 1 -- 172.21.15.78:0/3946145504 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fee400e5480 con 0x7fee48108020 2024-09-11T00:56:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.598+0000 7fee4d730640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee340b5330 0x7fee340b77f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fee38031440 tx=0x7fee38005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.602+0000 7fee3e7fc640 1 -- 172.21.15.78:0/3946145504 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fee400ac360 con 0x7fee48108020 2024-09-11T00:56:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.780+0000 7fee4f994640 1 -- 172.21.15.78:0/3946145504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee4806aee0 con 0x7fee48108020 2024-09-11T00:56:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.781+0000 7fee3e7fc640 1 -- 172.21.15.78:0/3946145504 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22571 (secure 0 0 0) 0x7fee400abc60 con 0x7fee48108020 2024-09-11T00:56:55.783 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:55.784 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":75,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","created":"2024-09-11T00:48:27.652512+0000","modified":"2024-09-11T00:56:53.792209+0000","last_up_change":"2024-09-11T00:55:59.071507+0000","last_in_change":"2024-09-11T00:55:35.815150+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:52:10.229887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:56:49.939136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:56:51.703446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beefc6d9-f74c-4200-96f2-ff697ad0d7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6803","nonce":1424639053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6805","nonce":1424639053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6809","nonce":1424639053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6807","nonce":1424639053}]},"public_addr":"172.21.15.78:6803/1424639053","cluster_addr":"172.21.15.78:6805/1424639053","heartbeat_back_addr":"172.21.15.78:6809/1424639053","heartbeat_front_addr":"172.21.15.78:6807/1424639053","state":["exists","up"]},{"osd":1,"uuid":"d742e767-d673-4bb9-b624-8bc969f5ceb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6801","nonce":580939385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6803","nonce":580939385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6807","nonce":580939385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6805","nonce":580939385}]},"public_addr":"172.21.15.110:6801/580939385","cluster_addr":"172.21.15.110:6803/580939385","heartbeat_back_addr":"172.21.15.110:6807/580939385","heartbeat_front_addr":"172.21.15.110:6805/580939385","state":["exists","up"]},{"osd":2,"uuid":"b0106a5b-1465-4a50-96dd-934dbae49a4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6801","nonce":2983155763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6803","nonce":2983155763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6807","nonce":2983155763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6805","nonce":2983155763}]},"public_addr":"172.21.15.112:6801/2983155763","cluster_addr":"172.21.15.112:6803/2983155763","heartbeat_back_addr":"172.21.15.112:6807/2983155763","heartbeat_front_addr":"172.21.15.112:6805/2983155763","state":["exists","up"]},{"osd":3,"uuid":"3270362b-3d61-414a-a3d9-f2416ff8b870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6811","nonce":2906086464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6813","nonce":2906086464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6817","nonce":2906086464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6815","nonce":2906086464}]},"public_addr":"172.21.15.78:6811/2906086464","cluster_addr":"172.21.15.78:6813/2906086464","heartbeat_back_addr":"172.21.15.78:6817/2906086464","heartbeat_front_addr":"172.21.15.78:6815/2906086464","state":["exists","up"]},{"osd":4,"uuid":"e47746ec-9443-4004-abfa-17c1ed73b982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6809","nonce":477287435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6811","nonce":477287435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6815","nonce":477287435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6813","nonce":477287435}]},"public_addr":"172.21.15.110:6809/477287435","cluster_addr":"172.21.15.110:6811/477287435","heartbeat_back_addr":"172.21.15.110:6815/477287435","heartbeat_front_addr":"172.21.15.110:6813/477287435","state":["exists","up"]},{"osd":5,"uuid":"f8cedb01-310d-442d-a588-33197d32e208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6809","nonce":3557095096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6811","nonce":3557095096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6815","nonce":3557095096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6813","nonce":3557095096}]},"public_addr":"172.21.15.112:6809/3557095096","cluster_addr":"172.21.15.112:6811/3557095096","heartbeat_back_addr":"172.21.15.112:6815/3557095096","heartbeat_front_addr":"172.21.15.112:6813/3557095096","state":["exists","up"]},{"osd":6,"uuid":"d508a037-8f56-436b-b7bf-9a1c69f6d9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6819","nonce":1267910775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6821","nonce":1267910775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6825","nonce":1267910775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6823","nonce":1267910775}]},"public_addr":"172.21.15.78:6819/1267910775","cluster_addr":"172.21.15.78:6821/1267910775","heartbeat_back_addr":"172.21.15.78:6825/1267910775","heartbeat_front_addr":"172.21.15.78:6823/1267910775","state":["exists","up"]},{"osd":7,"uuid":"e8842047-09a6-48b8-8d0a-2d54e7f89063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6817","nonce":1556494594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6819","nonce":1556494594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6823","nonce":1556494594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6821","nonce":1556494594}]},"public_addr":"172.21.15.110:6817/1556494594","cluster_addr":"172.21.15.110:6819/1556494594","heartbeat_back_addr":"172.21.15.110:6823/1556494594","heartbeat_front_addr":"172.21.15.110:6821/1556494594","state":["exists","up"]},{"osd":8,"uuid":"a3367da8-375a-4a46-a952-867d4b72ae2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6817","nonce":2481321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6819","nonce":2481321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6823","nonce":2481321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6821","nonce":2481321}]},"public_addr":"172.21.15.112:6817/2481321","cluster_addr":"172.21.15.112:6819/2481321","heartbeat_back_addr":"172.21.15.112:6823/2481321","heartbeat_front_addr":"172.21.15.112:6821/2481321","state":["exists","up"]},{"osd":9,"uuid":"d1825764-cefc-495b-99da-8ba429df0937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6827","nonce":197048350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6829","nonce":197048350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6833","nonce":197048350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6831","nonce":197048350}]},"public_addr":"172.21.15.78:6827/197048350","cluster_addr":"172.21.15.78:6829/197048350","heartbeat_back_addr":"172.21.15.78:6833/197048350","heartbeat_front_addr":"172.21.15.78:6831/197048350","state":["exists","up"]},{"osd":10,"uuid":"e1cfff7a-8916-4845-a482-d79fa275da30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6825","nonce":2101123168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6827","nonce":2101123168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6831","nonce":2101123168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6829","nonce":2101123168}]},"public_addr":"172.21.15.110:6825/2101123168","cluster_addr":"172.21.15.110:6827/2101123168","heartbeat_back_addr":"172.21.15.110:6831/2101123168","heartbeat_front_addr":"172.21.15.110:6829/2101123168","state":["exists","up"]},{"osd":11,"uuid":"fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6825","nonce":1526310654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6827","nonce":1526310654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6831","nonce":1526310654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6829","nonce":1526310654}]},"public_addr":"172.21.15.112:6825/1526310654","cluster_addr":"172.21.15.112:6827/1526310654","heartbeat_back_addr":"172.21.15.112:6831/1526310654","heartbeat_front_addr":"172.21.15.112:6829/1526310654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:17.334199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:41.400872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:08.205144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:33.550527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:58.394116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:24.699607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:50.450193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:16.223363+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:40.947727+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:06.845451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:32.410035+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:57.744364+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/2390785703":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:0/1038951931":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/391908936":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2706514734":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/2357025243":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6800/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2795778473":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2747741101":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6801/1289343426":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/3558896007":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/438261739":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/1289343426":"2024-09-12T00:50:15.928079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:56:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.784+0000 7fee4f994640 1 -- 172.21.15.78:0/3946145504 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee340b5330 msgr2=0x7fee340b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.784+0000 7fee4f994640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee340b5330 0x7fee340b77f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fee38031440 tx=0x7fee38005a20 comp rx=0 tx=0).stop 2024-09-11T00:56:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.784+0000 7fee4f994640 1 -- 172.21.15.78:0/3946145504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee48108020 msgr2=0x7fee4819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.784+0000 7fee4f994640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee48108020 0x7fee4819abb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fee4000b970 tx=0x7fee4000be30 comp rx=0 tx=0).stop 2024-09-11T00:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.784+0000 7fee4f994640 1 -- 172.21.15.78:0/3946145504 shutdown_connections 2024-09-11T00:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.784+0000 7fee4f994640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee340b5330 0x7fee340b77f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.784+0000 7fee4f994640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee48108020 0x7fee4819abb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.784+0000 7fee4f994640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee481076a0 0x7fee4819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.784+0000 7fee4f994640 1 --2- 172.21.15.78:0/3946145504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee48101c90 0x7fee4819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.784+0000 7fee4f994640 1 -- 172.21.15.78:0/3946145504 >> 172.21.15.78:0/3946145504 conn(0x7fee480fd700 msgr2=0x7fee480fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.784+0000 7fee4f994640 1 -- 172.21.15.78:0/3946145504 shutdown_connections 2024-09-11T00:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:55.784+0000 7fee4f994640 1 -- 172.21.15.78:0/3946145504 wait complete. 2024-09-11T00:56:56.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:56 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] up:active 2024-09-11T00:56:56.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:56 smithi110 ceph-mon[34712]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:56:56.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:56 smithi110 ceph-mon[34712]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:56:56.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:56 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3946145504' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:56:56.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:56 smithi110 ceph-mon[34712]: pgmap v263: 97 pgs: 5 creating+activating, 13 creating+peering, 7 unknown, 72 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 1 op/s 2024-09-11T00:56:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:56 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] up:active 2024-09-11T00:56:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:56 smithi112 ceph-mon[35384]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:56:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:56 smithi112 ceph-mon[35384]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:56:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:56 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3946145504' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:56:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:56 smithi112 ceph-mon[35384]: pgmap v263: 97 pgs: 5 creating+activating, 13 creating+peering, 7 unknown, 72 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 1 op/s 2024-09-11T00:56:56.479 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:56 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] up:active 2024-09-11T00:56:56.479 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:56 smithi078 ceph-mon[31315]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:56:56.479 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:56 smithi078 ceph-mon[31315]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:56:56.479 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:56 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3946145504' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:56:56.479 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:56 smithi078 ceph-mon[31315]: pgmap v263: 97 pgs: 5 creating+activating, 13 creating+peering, 7 unknown, 72 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 1 op/s 2024-09-11T00:56:56.509 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:56:56.764 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.239+0000 7f145c232640 1 -- 172.21.15.78:0/1725748087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1454104550 msgr2=0x7f145410cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.239+0000 7f145c232640 1 --2- 172.21.15.78:0/1725748087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1454104550 0x7f145410cb30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1444009900 tx=0x7f144402a410 comp rx=0 tx=0).stop 2024-09-11T00:56:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.239+0000 7f145c232640 1 -- 172.21.15.78:0/1725748087 shutdown_connections 2024-09-11T00:56:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.239+0000 7f145c232640 1 --2- 172.21.15.78:0/1725748087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f145410d100 0x7f145410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.239+0000 7f145c232640 1 --2- 172.21.15.78:0/1725748087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1454104550 0x7f145410cb30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.239+0000 7f145c232640 1 --2- 172.21.15.78:0/1725748087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1454103b50 0x7f1454103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.239+0000 7f145c232640 1 -- 172.21.15.78:0/1725748087 >> 172.21.15.78:0/1725748087 conn(0x7f14540fd700 msgr2=0x7f14540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.239+0000 7f145c232640 1 -- 172.21.15.78:0/1725748087 shutdown_connections 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.239+0000 7f145c232640 1 -- 172.21.15.78:0/1725748087 wait complete. 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145c232640 1 Processor -- start 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145c232640 1 -- start start 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145c232640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1454103b50 0x7f1454199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145c232640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1454104550 0x7f14541997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145c232640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f145410d100 0x7f145419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145c232640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1454112d60 con 0x7f145410d100 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145c232640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1454112be0 con 0x7f1454104550 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145c232640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1454112ee0 con 0x7f1454103b50 2024-09-11T00:56:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145a7cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f145410d100 0x7f145419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145a7cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f145410d100 0x7f145419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38974/0 (socket says 172.21.15.78:38974) 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145a7cf640 1 -- 172.21.15.78:0/3756043426 learned_addr learned my addr 172.21.15.78:0/3756043426 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f14597cd640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1454104550 0x7f14541997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145a7cf640 1 -- 172.21.15.78:0/3756043426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1454103b50 msgr2=0x7f1454199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f1459fce640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1454103b50 0x7f1454199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145a7cf640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1454103b50 0x7f1454199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145a7cf640 1 -- 172.21.15.78:0/3756043426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1454104550 msgr2=0x7f14541997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145a7cf640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1454104550 0x7f14541997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145a7cf640 1 -- 172.21.15.78:0/3756043426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145419f340 con 0x7f145410d100 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f14597cd640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1454104550 0x7f14541997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f1459fce640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1454103b50 0x7f1454199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.240+0000 7f145a7cf640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f145410d100 0x7f145419eb70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f144c00eaa0 tx=0x7f144c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.241+0000 7f144affd640 1 -- 172.21.15.78:0/3756043426 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c00cdb0 con 0x7f145410d100 2024-09-11T00:56:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.241+0000 7f144affd640 1 -- 172.21.15.78:0/3756043426 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f144c005020 con 0x7f145410d100 2024-09-11T00:56:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.241+0000 7f145c232640 1 -- 172.21.15.78:0/3756043426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145419f630 con 0x7f145410d100 2024-09-11T00:56:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.241+0000 7f144affd640 1 -- 172.21.15.78:0/3756043426 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c0116d0 con 0x7f145410d100 2024-09-11T00:56:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.241+0000 7f145c232640 1 -- 172.21.15.78:0/3756043426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14541a5cf0 con 0x7f145410d100 2024-09-11T00:56:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.242+0000 7f145c232640 1 -- 172.21.15.78:0/3756043426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1454103f30 con 0x7f145410d100 2024-09-11T00:56:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.243+0000 7f144affd640 1 -- 172.21.15.78:0/3756043426 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f144c00f040 con 0x7f145410d100 2024-09-11T00:56:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.244+0000 7f144affd640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14400b5330 0x7f14400b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.244+0000 7f1459fce640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14400b5330 0x7f14400b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.244+0000 7f144affd640 1 -- 172.21.15.78:0/3756043426 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f144c0e5f40 con 0x7f145410d100 2024-09-11T00:56:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.244+0000 7f1459fce640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14400b5330 0x7f14400b77f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1450009b40 tx=0x7f1450005e50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.247+0000 7f144affd640 1 -- 172.21.15.78:0/3756043426 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f144c016030 con 0x7f145410d100 2024-09-11T00:56:57.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.430+0000 7f145c232640 1 -- 172.21.15.78:0/3756043426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f145402e5c0 con 0x7f145410d100 2024-09-11T00:56:57.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:57 smithi078 ceph-mon[31315]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T00:56:57.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:57 smithi078 ceph-mon[31315]: Cluster is now healthy 2024-09-11T00:56:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.432+0000 7f144affd640 1 -- 172.21.15.78:0/3756043426 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22571 (secure 0 0 0) 0x7f144c0a6e20 con 0x7f145410d100 2024-09-11T00:56:57.434 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:56:57.434 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":75,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","created":"2024-09-11T00:48:27.652512+0000","modified":"2024-09-11T00:56:53.792209+0000","last_up_change":"2024-09-11T00:55:59.071507+0000","last_in_change":"2024-09-11T00:55:35.815150+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:52:10.229887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:56:49.939136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:56:51.703446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beefc6d9-f74c-4200-96f2-ff697ad0d7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6803","nonce":1424639053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6805","nonce":1424639053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6809","nonce":1424639053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6807","nonce":1424639053}]},"public_addr":"172.21.15.78:6803/1424639053","cluster_addr":"172.21.15.78:6805/1424639053","heartbeat_back_addr":"172.21.15.78:6809/1424639053","heartbeat_front_addr":"172.21.15.78:6807/1424639053","state":["exists","up"]},{"osd":1,"uuid":"d742e767-d673-4bb9-b624-8bc969f5ceb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6801","nonce":580939385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6803","nonce":580939385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6807","nonce":580939385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6805","nonce":580939385}]},"public_addr":"172.21.15.110:6801/580939385","cluster_addr":"172.21.15.110:6803/580939385","heartbeat_back_addr":"172.21.15.110:6807/580939385","heartbeat_front_addr":"172.21.15.110:6805/580939385","state":["exists","up"]},{"osd":2,"uuid":"b0106a5b-1465-4a50-96dd-934dbae49a4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6801","nonce":2983155763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6803","nonce":2983155763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6807","nonce":2983155763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6805","nonce":2983155763}]},"public_addr":"172.21.15.112:6801/2983155763","cluster_addr":"172.21.15.112:6803/2983155763","heartbeat_back_addr":"172.21.15.112:6807/2983155763","heartbeat_front_addr":"172.21.15.112:6805/2983155763","state":["exists","up"]},{"osd":3,"uuid":"3270362b-3d61-414a-a3d9-f2416ff8b870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6811","nonce":2906086464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6813","nonce":2906086464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6817","nonce":2906086464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6815","nonce":2906086464}]},"public_addr":"172.21.15.78:6811/2906086464","cluster_addr":"172.21.15.78:6813/2906086464","heartbeat_back_addr":"172.21.15.78:6817/2906086464","heartbeat_front_addr":"172.21.15.78:6815/2906086464","state":["exists","up"]},{"osd":4,"uuid":"e47746ec-9443-4004-abfa-17c1ed73b982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6809","nonce":477287435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6811","nonce":477287435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6815","nonce":477287435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6813","nonce":477287435}]},"public_addr":"172.21.15.110:6809/477287435","cluster_addr":"172.21.15.110:6811/477287435","heartbeat_back_addr":"172.21.15.110:6815/477287435","heartbeat_front_addr":"172.21.15.110:6813/477287435","state":["exists","up"]},{"osd":5,"uuid":"f8cedb01-310d-442d-a588-33197d32e208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6809","nonce":3557095096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6811","nonce":3557095096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6815","nonce":3557095096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6813","nonce":3557095096}]},"public_addr":"172.21.15.112:6809/3557095096","cluster_addr":"172.21.15.112:6811/3557095096","heartbeat_back_addr":"172.21.15.112:6815/3557095096","heartbeat_front_addr":"172.21.15.112:6813/3557095096","state":["exists","up"]},{"osd":6,"uuid":"d508a037-8f56-436b-b7bf-9a1c69f6d9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6819","nonce":1267910775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6821","nonce":1267910775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6825","nonce":1267910775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6823","nonce":1267910775}]},"public_addr":"172.21.15.78:6819/1267910775","cluster_addr":"172.21.15.78:6821/1267910775","heartbeat_back_addr":"172.21.15.78:6825/1267910775","heartbeat_front_addr":"172.21.15.78:6823/1267910775","state":["exists","up"]},{"osd":7,"uuid":"e8842047-09a6-48b8-8d0a-2d54e7f89063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6817","nonce":1556494594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6819","nonce":1556494594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6823","nonce":1556494594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6821","nonce":1556494594}]},"public_addr":"172.21.15.110:6817/1556494594","cluster_addr":"172.21.15.110:6819/1556494594","heartbeat_back_addr":"172.21.15.110:6823/1556494594","heartbeat_front_addr":"172.21.15.110:6821/1556494594","state":["exists","up"]},{"osd":8,"uuid":"a3367da8-375a-4a46-a952-867d4b72ae2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6817","nonce":2481321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6819","nonce":2481321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6823","nonce":2481321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6821","nonce":2481321}]},"public_addr":"172.21.15.112:6817/2481321","cluster_addr":"172.21.15.112:6819/2481321","heartbeat_back_addr":"172.21.15.112:6823/2481321","heartbeat_front_addr":"172.21.15.112:6821/2481321","state":["exists","up"]},{"osd":9,"uuid":"d1825764-cefc-495b-99da-8ba429df0937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6827","nonce":197048350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6829","nonce":197048350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6833","nonce":197048350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6831","nonce":197048350}]},"public_addr":"172.21.15.78:6827/197048350","cluster_addr":"172.21.15.78:6829/197048350","heartbeat_back_addr":"172.21.15.78:6833/197048350","heartbeat_front_addr":"172.21.15.78:6831/197048350","state":["exists","up"]},{"osd":10,"uuid":"e1cfff7a-8916-4845-a482-d79fa275da30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6825","nonce":2101123168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6827","nonce":2101123168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6831","nonce":2101123168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6829","nonce":2101123168}]},"public_addr":"172.21.15.110:6825/2101123168","cluster_addr":"172.21.15.110:6827/2101123168","heartbeat_back_addr":"172.21.15.110:6831/2101123168","heartbeat_front_addr":"172.21.15.110:6829/2101123168","state":["exists","up"]},{"osd":11,"uuid":"fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6825","nonce":1526310654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6827","nonce":1526310654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6831","nonce":1526310654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6829","nonce":1526310654}]},"public_addr":"172.21.15.112:6825/1526310654","cluster_addr":"172.21.15.112:6827/1526310654","heartbeat_back_addr":"172.21.15.112:6831/1526310654","heartbeat_front_addr":"172.21.15.112:6829/1526310654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:17.334199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:41.400872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:08.205144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:33.550527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:58.394116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:24.699607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:50.450193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:16.223363+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:40.947727+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:06.845451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:32.410035+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:57.744364+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/2390785703":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:0/1038951931":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/391908936":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2706514734":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/2357025243":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6800/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2795778473":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2747741101":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6801/1289343426":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/3558896007":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/438261739":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/1289343426":"2024-09-12T00:50:15.928079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:56:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.434+0000 7f145c232640 1 -- 172.21.15.78:0/3756043426 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14400b5330 msgr2=0x7f14400b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.434+0000 7f145c232640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14400b5330 0x7f14400b77f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1450009b40 tx=0x7f1450005e50 comp rx=0 tx=0).stop 2024-09-11T00:56:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.434+0000 7f145c232640 1 -- 172.21.15.78:0/3756043426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f145410d100 msgr2=0x7f145419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.434+0000 7f145c232640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f145410d100 0x7f145419eb70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f144c00eaa0 tx=0x7f144c00ef60 comp rx=0 tx=0).stop 2024-09-11T00:56:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.434+0000 7f145c232640 1 -- 172.21.15.78:0/3756043426 shutdown_connections 2024-09-11T00:56:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.434+0000 7f145c232640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14400b5330 0x7f14400b77f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.434+0000 7f145c232640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f145410d100 0x7f145419eb70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.434+0000 7f145c232640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1454104550 0x7f14541997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.434+0000 7f145c232640 1 --2- 172.21.15.78:0/3756043426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1454103b50 0x7f1454199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.434+0000 7f145c232640 1 -- 172.21.15.78:0/3756043426 >> 172.21.15.78:0/3756043426 conn(0x7f14540fd700 msgr2=0x7f14540fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.434+0000 7f145c232640 1 -- 172.21.15.78:0/3756043426 shutdown_connections 2024-09-11T00:56:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:57.434+0000 7f145c232640 1 -- 172.21.15.78:0/3756043426 wait complete. 2024-09-11T00:56:57.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:57 smithi110 ceph-mon[34712]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T00:56:57.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:57 smithi110 ceph-mon[34712]: Cluster is now healthy 2024-09-11T00:56:57.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:57 smithi112 ceph-mon[35384]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T00:56:57.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:57 smithi112 ceph-mon[35384]: Cluster is now healthy 2024-09-11T00:56:58.153 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-11T00:56:58.401 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:56:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:58 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3756043426' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:56:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:58 smithi110 ceph-mon[34712]: pgmap v264: 97 pgs: 10 creating+peering, 87 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-09-11T00:56:58.440 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 00:56:58 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b[65617]: 2024-09-11T00:56:58.084+0000 7fb47cd03640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T00:56:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:58 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3756043426' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:56:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:58 smithi112 ceph-mon[35384]: pgmap v264: 97 pgs: 10 creating+peering, 87 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-09-11T00:56:58.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:58 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3756043426' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:56:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:58 smithi078 ceph-mon[31315]: pgmap v264: 97 pgs: 10 creating+peering, 87 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-09-11T00:56:58.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.866+0000 7f6b93768640 1 -- 172.21.15.78:0/639943887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c109150 msgr2=0x7f6b8c10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:58.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.866+0000 7f6b93768640 1 --2- 172.21.15.78:0/639943887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c109150 0x7f6b8c10f950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009ca0 tx=0x7f6b8402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:56:58.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.866+0000 7f6b93768640 1 -- 172.21.15.78:0/639943887 shutdown_connections 2024-09-11T00:56:58.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.866+0000 7f6b93768640 1 --2- 172.21.15.78:0/639943887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c109150 0x7f6b8c10f950 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:58.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.866+0000 7f6b93768640 1 --2- 172.21.15.78:0/639943887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b8c1087d0 0x7f6b8c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:58.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.866+0000 7f6b93768640 1 --2- 172.21.15.78:0/639943887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c102dc0 0x7f6b8c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:58.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.866+0000 7f6b93768640 1 -- 172.21.15.78:0/639943887 >> 172.21.15.78:0/639943887 conn(0x7f6b8c0fe830 msgr2=0x7f6b8c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:58.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.866+0000 7f6b93768640 1 -- 172.21.15.78:0/639943887 shutdown_connections 2024-09-11T00:56:58.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.866+0000 7f6b93768640 1 -- 172.21.15.78:0/639943887 wait complete. 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b93768640 1 Processor -- start 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b93768640 1 -- start start 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b93768640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c102dc0 0x7f6b8c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b93768640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c1087d0 0x7f6b8c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b93768640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b8c109150 0x7f6b8c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b93768640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c112e10 con 0x7f6b8c102dc0 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b93768640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c112c90 con 0x7f6b8c109150 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b93768640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c112f90 con 0x7f6b8c1087d0 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b91504640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c102dc0 0x7f6b8c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b91504640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c102dc0 0x7f6b8c1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38978/0 (socket says 172.21.15.78:38978) 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b91504640 1 -- 172.21.15.78:0/3032604019 learned_addr learned my addr 172.21.15.78:0/3032604019 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b90d03640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c1087d0 0x7f6b8c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b91d05640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b8c109150 0x7f6b8c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b91504640 1 -- 172.21.15.78:0/3032604019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c1087d0 msgr2=0x7f6b8c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b91504640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c1087d0 0x7f6b8c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b91504640 1 -- 172.21.15.78:0/3032604019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b8c109150 msgr2=0x7f6b8c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b91504640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b8c109150 0x7f6b8c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b91504640 1 -- 172.21.15.78:0/3032604019 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c19f370 con 0x7f6b8c102dc0 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b90d03640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c1087d0 0x7f6b8c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b91d05640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b8c109150 0x7f6b8c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.867+0000 7f6b91504640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c102dc0 0x7f6b8c1992c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800c890 tx=0x7f6b8800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.868+0000 7f6b827fc640 1 -- 172.21.15.78:0/3032604019 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88004220 con 0x7f6b8c102dc0 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.868+0000 7f6b827fc640 1 -- 172.21.15.78:0/3032604019 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b880043c0 con 0x7f6b8c102dc0 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.868+0000 7f6b93768640 1 -- 172.21.15.78:0/3032604019 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c19f660 con 0x7f6b8c102dc0 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.868+0000 7f6b827fc640 1 -- 172.21.15.78:0/3032604019 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88004a00 con 0x7f6b8c102dc0 2024-09-11T00:56:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.868+0000 7f6b93768640 1 -- 172.21.15.78:0/3032604019 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c1a5ef0 con 0x7f6b8c102dc0 2024-09-11T00:56:58.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.869+0000 7f6b93768640 1 -- 172.21.15.78:0/3032604019 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b8c1031a0 con 0x7f6b8c102dc0 2024-09-11T00:56:58.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.870+0000 7f6b827fc640 1 -- 172.21.15.78:0/3032604019 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f6b880079b0 con 0x7f6b8c102dc0 2024-09-11T00:56:58.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.871+0000 7f6b827fc640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6b780b53b0 0x7f6b780b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:56:58.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.871+0000 7f6b90d03640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6b780b53b0 0x7f6b780b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:56:58.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.871+0000 7f6b827fc640 1 -- 172.21.15.78:0/3032604019 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f6b880e5de0 con 0x7f6b8c102dc0 2024-09-11T00:56:58.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.871+0000 7f6b90d03640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6b780b53b0 0x7f6b780b7870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c005fd0 tx=0x7f6b7c0094d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:56:58.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:58.874+0000 7f6b827fc640 1 -- 172.21.15.78:0/3032604019 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6b88014030 con 0x7f6b8c102dc0 2024-09-11T00:56:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.091+0000 7f6b93768640 1 -- 172.21.15.78:0/3032604019 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6b8c19a3f0 con 0x7f6b8c102dc0 2024-09-11T00:56:59.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.139+0000 7f6b827fc640 1 -- 172.21.15.78:0/3032604019 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f6b880acd40 con 0x7f6b8c102dc0 2024-09-11T00:56:59.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.141+0000 7f6b93768640 1 -- 172.21.15.78:0/3032604019 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6b780b53b0 msgr2=0x7f6b780b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:59.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.141+0000 7f6b93768640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6b780b53b0 0x7f6b780b7870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c005fd0 tx=0x7f6b7c0094d0 comp rx=0 tx=0).stop 2024-09-11T00:56:59.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.141+0000 7f6b93768640 1 -- 172.21.15.78:0/3032604019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c102dc0 msgr2=0x7f6b8c1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:56:59.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.141+0000 7f6b93768640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c102dc0 0x7f6b8c1992c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800c890 tx=0x7f6b8800cd50 comp rx=0 tx=0).stop 2024-09-11T00:56:59.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.141+0000 7f6b93768640 1 -- 172.21.15.78:0/3032604019 shutdown_connections 2024-09-11T00:56:59.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.141+0000 7f6b93768640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6b780b53b0 0x7f6b780b7870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.141+0000 7f6b93768640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b8c109150 0x7f6b8c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.141+0000 7f6b93768640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c1087d0 0x7f6b8c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.141+0000 7f6b93768640 1 --2- 172.21.15.78:0/3032604019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c102dc0 0x7f6b8c1992c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:56:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.141+0000 7f6b93768640 1 -- 172.21.15.78:0/3032604019 >> 172.21.15.78:0/3032604019 conn(0x7f6b8c0fe830 msgr2=0x7f6b8c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:56:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.142+0000 7f6b93768640 1 -- 172.21.15.78:0/3032604019 shutdown_connections 2024-09-11T00:56:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:56:59.142+0000 7f6b93768640 1 -- 172.21.15.78:0/3032604019 wait complete. 2024-09-11T00:56:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:56:59 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3032604019' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-11T00:56:59.463 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:56:59 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3032604019' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-11T00:56:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:56:59 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3032604019' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-11T00:56:59.826 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-11T00:56:59.827 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-11T00:57:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:00 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3032604019' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T00:57:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:00 smithi110 ceph-mon[34712]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:57:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:00 smithi110 ceph-mon[34712]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-11T00:57:00.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:00 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3032604019' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T00:57:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:00 smithi112 ceph-mon[35384]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:57:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:00 smithi112 ceph-mon[35384]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-11T00:57:00.487 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:00 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3032604019' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T00:57:00.487 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:00 smithi078 ceph-mon[31315]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:57:00.487 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:00 smithi078 ceph-mon[31315]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-11T00:57:00.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.567+0000 7fcd0fe89640 1 -- 172.21.15.78:0/4084510879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd08109150 msgr2=0x7fcd0810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:00.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.567+0000 7fcd0fe89640 1 --2- 172.21.15.78:0/4084510879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd08109150 0x7fcd0810f950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009ca0 tx=0x7fcd0002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:00.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.567+0000 7fcd0fe89640 1 -- 172.21.15.78:0/4084510879 shutdown_connections 2024-09-11T00:57:00.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.567+0000 7fcd0fe89640 1 --2- 172.21.15.78:0/4084510879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd08109150 0x7fcd0810f950 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:00.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.567+0000 7fcd0fe89640 1 --2- 172.21.15.78:0/4084510879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd081087d0 0x7fcd08108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:00.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.567+0000 7fcd0fe89640 1 --2- 172.21.15.78:0/4084510879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd08102dc0 0x7fcd081031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:00.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.567+0000 7fcd0fe89640 1 -- 172.21.15.78:0/4084510879 >> 172.21.15.78:0/4084510879 conn(0x7fcd080fe830 msgr2=0x7fcd08100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:00.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.567+0000 7fcd0fe89640 1 -- 172.21.15.78:0/4084510879 shutdown_connections 2024-09-11T00:57:00.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.567+0000 7fcd0fe89640 1 -- 172.21.15.78:0/4084510879 wait complete. 2024-09-11T00:57:00.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0fe89640 1 Processor -- start 2024-09-11T00:57:00.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0fe89640 1 -- start start 2024-09-11T00:57:00.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0fe89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd08102dc0 0x7fcd08199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:00.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0fe89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd081087d0 0x7fcd081997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:00.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0fe89640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd08109150 0x7fcd0819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:00.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0fe89640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08112dd0 con 0x7fcd081087d0 2024-09-11T00:57:00.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0fe89640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08112c50 con 0x7fcd08109150 2024-09-11T00:57:00.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0fe89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08112f50 con 0x7fcd08102dc0 2024-09-11T00:57:00.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0d424640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd081087d0 0x7fcd081997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:00.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0dc25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd08102dc0 0x7fcd08199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:00.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0d424640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd081087d0 0x7fcd081997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38998/0 (socket says 172.21.15.78:38998) 2024-09-11T00:57:00.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0d424640 1 -- 172.21.15.78:0/338097402 learned_addr learned my addr 172.21.15.78:0/338097402 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:00.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0e426640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd08109150 0x7fcd0819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:00.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0d424640 1 -- 172.21.15.78:0/338097402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd08102dc0 msgr2=0x7fcd08199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:00.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0d424640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd08102dc0 0x7fcd08199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:00.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0d424640 1 -- 172.21.15.78:0/338097402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd08109150 msgr2=0x7fcd0819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:00.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0d424640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd08109150 0x7fcd0819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:00.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0d424640 1 -- 172.21.15.78:0/338097402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0819f310 con 0x7fcd081087d0 2024-09-11T00:57:00.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.568+0000 7fcd0e426640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd08109150 0x7fcd0819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:00.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.569+0000 7fcd0d424640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd081087d0 0x7fcd081997a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fccf800d950 tx=0x7fccf800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:00.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.569+0000 7fccfeffd640 1 -- 172.21.15.78:0/338097402 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8017070 con 0x7fcd081087d0 2024-09-11T00:57:00.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.569+0000 7fcd0fe89640 1 -- 172.21.15.78:0/338097402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0819f600 con 0x7fcd081087d0 2024-09-11T00:57:00.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.569+0000 7fccfeffd640 1 -- 172.21.15.78:0/338097402 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccf8009c30 con 0x7fcd081087d0 2024-09-11T00:57:00.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.569+0000 7fccfeffd640 1 -- 172.21.15.78:0/338097402 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf800f620 con 0x7fcd081087d0 2024-09-11T00:57:00.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.569+0000 7fcd0fe89640 1 -- 172.21.15.78:0/338097402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd081a5e90 con 0x7fcd081087d0 2024-09-11T00:57:00.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.570+0000 7fcd0fe89640 1 -- 172.21.15.78:0/338097402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd081031a0 con 0x7fcd081087d0 2024-09-11T00:57:00.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.571+0000 7fccfeffd640 1 -- 172.21.15.78:0/338097402 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fccf8010460 con 0x7fcd081087d0 2024-09-11T00:57:00.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.572+0000 7fccfeffd640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fccf40b5330 0x7fccf40b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:00.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.572+0000 7fcd0dc25640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fccf40b5330 0x7fccf40b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:00.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.572+0000 7fccfeffd640 1 -- 172.21.15.78:0/338097402 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fccf80e5850 con 0x7fcd081087d0 2024-09-11T00:57:00.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.572+0000 7fcd0dc25640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fccf40b5330 0x7fccf40b77f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400bbe0 tx=0x7fcd0400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:00.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.575+0000 7fccfeffd640 1 -- 172.21.15.78:0/338097402 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fccf80ac730 con 0x7fcd081087d0 2024-09-11T00:57:00.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:00.787+0000 7fcd0fe89640 1 -- 172.21.15.78:0/338097402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fcd0810af30 con 0x7fcd081087d0 2024-09-11T00:57:01.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.151+0000 7fccfeffd640 1 -- 172.21.15.78:0/338097402 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fccf8009de0 con 0x7fcd081087d0 2024-09-11T00:57:01.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.153+0000 7fcd0fe89640 1 -- 172.21.15.78:0/338097402 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fccf40b5330 msgr2=0x7fccf40b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:01.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.153+0000 7fcd0fe89640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fccf40b5330 0x7fccf40b77f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400bbe0 tx=0x7fcd0400b390 comp rx=0 tx=0).stop 2024-09-11T00:57:01.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.153+0000 7fcd0fe89640 1 -- 172.21.15.78:0/338097402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd081087d0 msgr2=0x7fcd081997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.153+0000 7fcd0fe89640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd081087d0 0x7fcd081997a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fccf800d950 tx=0x7fccf800de10 comp rx=0 tx=0).stop 2024-09-11T00:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.153+0000 7fcd0fe89640 1 -- 172.21.15.78:0/338097402 shutdown_connections 2024-09-11T00:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.153+0000 7fcd0fe89640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fccf40b5330 0x7fccf40b77f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.153+0000 7fcd0fe89640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd08109150 0x7fcd0819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.153+0000 7fcd0fe89640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd081087d0 0x7fcd081997a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.153+0000 7fcd0fe89640 1 --2- 172.21.15.78:0/338097402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd08102dc0 0x7fcd08199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.153+0000 7fcd0fe89640 1 -- 172.21.15.78:0/338097402 >> 172.21.15.78:0/338097402 conn(0x7fcd080fe830 msgr2=0x7fcd080ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.153+0000 7fcd0fe89640 1 -- 172.21.15.78:0/338097402 shutdown_connections 2024-09-11T00:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:01.153+0000 7fcd0fe89640 1 -- 172.21.15.78:0/338097402 wait complete. 2024-09-11T00:57:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:01 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/338097402' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-11T00:57:01.474 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:01 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/338097402' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-11T00:57:01.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:01 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/338097402' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-11T00:57:01.835 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-11T00:57:02.081 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:02 smithi110 ceph-mon[34712]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T00:57:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:02 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/338097402' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T00:57:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:02 smithi110 ceph-mon[34712]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T00:57:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:02 smithi110 ceph-mon[34712]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:57:02.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:02 smithi110 ceph-mon[34712]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T00:57:02.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:02 smithi110 ceph-mon[34712]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T00:57:02.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:02 smithi110 ceph-mon[34712]: pgmap v266: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-11T00:57:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:02 smithi112 ceph-mon[35384]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T00:57:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:02 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/338097402' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T00:57:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:02 smithi112 ceph-mon[35384]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T00:57:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:02 smithi112 ceph-mon[35384]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:57:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:02 smithi112 ceph-mon[35384]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T00:57:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:02 smithi112 ceph-mon[35384]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T00:57:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:02 smithi112 ceph-mon[35384]: pgmap v266: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-11T00:57:02.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:02 smithi078 ceph-mon[31315]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T00:57:02.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:02 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/338097402' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T00:57:02.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:02 smithi078 ceph-mon[31315]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T00:57:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:02 smithi078 ceph-mon[31315]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:57:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:02 smithi078 ceph-mon[31315]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T00:57:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:02 smithi078 ceph-mon[31315]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T00:57:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:02 smithi078 ceph-mon[31315]: pgmap v266: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-11T00:57:02.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.576+0000 7fab9fa9f640 1 -- 172.21.15.78:0/3867930598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab98103ce0 msgr2=0x7fab981040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:02.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.577+0000 7fab9fa9f640 1 --2- 172.21.15.78:0/3867930598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab98103ce0 0x7fab981040c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fab94009900 tx=0x7fab9402a410 comp rx=0 tx=0).stop 2024-09-11T00:57:02.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.577+0000 7fab9fa9f640 1 -- 172.21.15.78:0/3867930598 shutdown_connections 2024-09-11T00:57:02.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.577+0000 7fab9fa9f640 1 --2- 172.21.15.78:0/3867930598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab9810d140 0x7fab9810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:02.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.577+0000 7fab9fa9f640 1 --2- 172.21.15.78:0/3867930598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab98104600 0x7fab9810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:02.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.577+0000 7fab9fa9f640 1 --2- 172.21.15.78:0/3867930598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab98103ce0 0x7fab981040c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:02.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.577+0000 7fab9fa9f640 1 -- 172.21.15.78:0/3867930598 >> 172.21.15.78:0/3867930598 conn(0x7fab980fd700 msgr2=0x7fab980ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:02.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.577+0000 7fab9fa9f640 1 -- 172.21.15.78:0/3867930598 shutdown_connections 2024-09-11T00:57:02.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.577+0000 7fab9fa9f640 1 -- 172.21.15.78:0/3867930598 wait complete. 2024-09-11T00:57:02.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.577+0000 7fab9fa9f640 1 Processor -- start 2024-09-11T00:57:02.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.577+0000 7fab9fa9f640 1 -- start start 2024-09-11T00:57:02.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9fa9f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab98103ce0 0x7fab98101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:02.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9fa9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab98104600 0x7fab98101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:02.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9fa9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab9810d140 0x7fab98102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:02.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9fa9f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab98112db0 con 0x7fab9810d140 2024-09-11T00:57:02.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9fa9f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab98112c30 con 0x7fab98103ce0 2024-09-11T00:57:02.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9fa9f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab98112f30 con 0x7fab98104600 2024-09-11T00:57:02.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9e03c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab9810d140 0x7fab98102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:02.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9e03c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab9810d140 0x7fab98102410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39018/0 (socket says 172.21.15.78:39018) 2024-09-11T00:57:02.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9e03c640 1 -- 172.21.15.78:0/4158219345 learned_addr learned my addr 172.21.15.78:0/4158219345 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:02.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9d03a640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab98104600 0x7fab98101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:02.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9d83b640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab98103ce0 0x7fab98101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:02.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9e03c640 1 -- 172.21.15.78:0/4158219345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab98104600 msgr2=0x7fab98101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:02.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9e03c640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab98104600 0x7fab98101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:02.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9e03c640 1 -- 172.21.15.78:0/4158219345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab98103ce0 msgr2=0x7fab98101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:02.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9e03c640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab98103ce0 0x7fab98101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:02.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9e03c640 1 -- 172.21.15.78:0/4158219345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab981a76a0 con 0x7fab9810d140 2024-09-11T00:57:02.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9d83b640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab98103ce0 0x7fab98101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:02.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9e03c640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab9810d140 0x7fab98102410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fab9000cce0 tx=0x7fab9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:02.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab8effd640 1 -- 172.21.15.78:0/4158219345 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9000f450 con 0x7fab9810d140 2024-09-11T00:57:02.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab8effd640 1 -- 172.21.15.78:0/4158219345 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9000faa0 con 0x7fab9810d140 2024-09-11T00:57:02.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9fa9f640 1 -- 172.21.15.78:0/4158219345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab981a7990 con 0x7fab9810d140 2024-09-11T00:57:02.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab8effd640 1 -- 172.21.15.78:0/4158219345 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90018b70 con 0x7fab9810d140 2024-09-11T00:57:02.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.578+0000 7fab9fa9f640 1 -- 172.21.15.78:0/4158219345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab981a7e80 con 0x7fab9810d140 2024-09-11T00:57:02.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.579+0000 7fab9fa9f640 1 -- 172.21.15.78:0/4158219345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab981040c0 con 0x7fab9810d140 2024-09-11T00:57:02.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.581+0000 7fab8effd640 1 -- 172.21.15.78:0/4158219345 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fab9000f5f0 con 0x7fab9810d140 2024-09-11T00:57:02.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.582+0000 7fab8effd640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab840b5330 0x7fab840b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:02.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.582+0000 7fab9d83b640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab840b5330 0x7fab840b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:02.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.582+0000 7fab8effd640 1 -- 172.21.15.78:0/4158219345 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fab900e6340 con 0x7fab9810d140 2024-09-11T00:57:02.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.582+0000 7fab9d83b640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab840b5330 0x7fab840b77f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab94031440 tx=0x7fab94005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:02.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.586+0000 7fab8effd640 1 -- 172.21.15.78:0/4158219345 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fab90014030 con 0x7fab9810d140 2024-09-11T00:57:02.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:02.797+0000 7fab9fa9f640 1 -- 172.21.15.78:0/4158219345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fab981a82b0 con 0x7fab9810d140 2024-09-11T00:57:03.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.174+0000 7fab8effd640 1 -- 172.21.15.78:0/4158219345 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fab900ad220 con 0x7fab9810d140 2024-09-11T00:57:03.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.176+0000 7fab9fa9f640 1 -- 172.21.15.78:0/4158219345 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab840b5330 msgr2=0x7fab840b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:03.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.176+0000 7fab9fa9f640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab840b5330 0x7fab840b77f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab94031440 tx=0x7fab94005a20 comp rx=0 tx=0).stop 2024-09-11T00:57:03.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.176+0000 7fab9fa9f640 1 -- 172.21.15.78:0/4158219345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab9810d140 msgr2=0x7fab98102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:03.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.176+0000 7fab9fa9f640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab9810d140 0x7fab98102410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fab9000cce0 tx=0x7fab9000a620 comp rx=0 tx=0).stop 2024-09-11T00:57:03.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.176+0000 7fab9fa9f640 1 -- 172.21.15.78:0/4158219345 shutdown_connections 2024-09-11T00:57:03.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.176+0000 7fab9fa9f640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab840b5330 0x7fab840b77f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:03.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.176+0000 7fab9fa9f640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab9810d140 0x7fab98102410 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:03.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.176+0000 7fab9fa9f640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab98104600 0x7fab98101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:03.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.176+0000 7fab9fa9f640 1 --2- 172.21.15.78:0/4158219345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab98103ce0 0x7fab98101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:03.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.176+0000 7fab9fa9f640 1 -- 172.21.15.78:0/4158219345 >> 172.21.15.78:0/4158219345 conn(0x7fab980fd700 msgr2=0x7fab980fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:03.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.176+0000 7fab9fa9f640 1 -- 172.21.15.78:0/4158219345 shutdown_connections 2024-09-11T00:57:03.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:03.176+0000 7fab9fa9f640 1 -- 172.21.15.78:0/4158219345 wait complete. 2024-09-11T00:57:03.440 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 00:57:03 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b[65617]: 2024-09-11T00:57:03.084+0000 7fb47cd03640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-11T00:57:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:03 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] up:active 2024-09-11T00:57:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:03 smithi110 ceph-mon[34712]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T00:57:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:03 smithi110 ceph-mon[34712]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T00:57:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:03 smithi110 ceph-mon[34712]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-11T00:57:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:03 smithi110 ceph-mon[34712]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-11T00:57:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:03 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/4158219345' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-11T00:57:03.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:03 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] up:active 2024-09-11T00:57:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:03 smithi112 ceph-mon[35384]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T00:57:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:03 smithi112 ceph-mon[35384]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T00:57:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:03 smithi112 ceph-mon[35384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-11T00:57:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:03 smithi112 ceph-mon[35384]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-11T00:57:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:03 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/4158219345' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-11T00:57:03.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:03 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] up:active 2024-09-11T00:57:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:03 smithi078 ceph-mon[31315]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T00:57:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:03 smithi078 ceph-mon[31315]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T00:57:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:03 smithi078 ceph-mon[31315]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-11T00:57:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:03 smithi078 ceph-mon[31315]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-11T00:57:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:03 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4158219345' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-11T00:57:03.848 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-11T00:57:04.092 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/4158219345' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] up:active 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] up:active 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: Cluster is now healthy 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-11T00:57:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:04 smithi110 ceph-mon[34712]: pgmap v267: 97 pgs: 97 active+clean; 580 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-11T00:57:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/4158219345' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T00:57:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] up:active 2024-09-11T00:57:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T00:57:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T00:57:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-11T00:57:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-11T00:57:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] up:active 2024-09-11T00:57:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T00:57:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T00:57:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: Cluster is now healthy 2024-09-11T00:57:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2024-09-11T00:57:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-11T00:57:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-11T00:57:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:04 smithi112 ceph-mon[35384]: pgmap v267: 97 pgs: 97 active+clean; 580 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-11T00:57:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4158219345' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T00:57:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] up:active 2024-09-11T00:57:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T00:57:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T00:57:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-11T00:57:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-11T00:57:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] up:active 2024-09-11T00:57:04.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T00:57:04.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T00:57:04.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: Cluster is now healthy 2024-09-11T00:57:04.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2024-09-11T00:57:04.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-11T00:57:04.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-11T00:57:04.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:04 smithi078 ceph-mon[31315]: pgmap v267: 97 pgs: 97 active+clean; 580 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-11T00:57:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.599+0000 7f4bd6533640 1 -- 172.21.15.78:0/2745135358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd0108020 msgr2=0x7f4bd010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.599+0000 7f4bd6533640 1 --2- 172.21.15.78:0/2745135358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd0108020 0x7f4bd010e820 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009ca0 tx=0x7f4bc802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.599+0000 7f4bd6533640 1 -- 172.21.15.78:0/2745135358 shutdown_connections 2024-09-11T00:57:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.599+0000 7f4bd6533640 1 --2- 172.21.15.78:0/2745135358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd0108020 0x7f4bd010e820 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.599+0000 7f4bd6533640 1 --2- 172.21.15.78:0/2745135358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd01076a0 0x7f4bd0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.599+0000 7f4bd6533640 1 --2- 172.21.15.78:0/2745135358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd0101c90 0x7f4bd0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.599+0000 7f4bd6533640 1 -- 172.21.15.78:0/2745135358 >> 172.21.15.78:0/2745135358 conn(0x7f4bd00fd700 msgr2=0x7f4bd00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.599+0000 7f4bd6533640 1 -- 172.21.15.78:0/2745135358 shutdown_connections 2024-09-11T00:57:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.599+0000 7f4bd6533640 1 -- 172.21.15.78:0/2745135358 wait complete. 2024-09-11T00:57:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.599+0000 7f4bd6533640 1 Processor -- start 2024-09-11T00:57:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.599+0000 7f4bd6533640 1 -- start start 2024-09-11T00:57:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd6533640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd0101c90 0x7f4bd010f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd6533640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd01076a0 0x7f4bd010f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd6533640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd0108020 0x7f4bd010fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd6533640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0112e00 con 0x7f4bd0108020 2024-09-11T00:57:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd6533640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0112c80 con 0x7f4bd0101c90 2024-09-11T00:57:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd6533640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0112f80 con 0x7f4bd01076a0 2024-09-11T00:57:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd4ad0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd0108020 0x7f4bd010fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd4ad0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd0108020 0x7f4bd010fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53012/0 (socket says 172.21.15.78:53012) 2024-09-11T00:57:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd4ad0640 1 -- 172.21.15.78:0/202968697 learned_addr learned my addr 172.21.15.78:0/202968697 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bcffff640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd0101c90 0x7f4bd010f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bc7fff640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd01076a0 0x7f4bd010f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd4ad0640 1 -- 172.21.15.78:0/202968697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd01076a0 msgr2=0x7f4bd010f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd4ad0640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd01076a0 0x7f4bd010f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd4ad0640 1 -- 172.21.15.78:0/202968697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd0101c90 msgr2=0x7f4bd010f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd4ad0640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd0101c90 0x7f4bd010f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd4ad0640 1 -- 172.21.15.78:0/202968697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd01a7650 con 0x7f4bd0108020 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bcffff640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd0101c90 0x7f4bd010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd4ad0640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd0108020 0x7f4bd010fdb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8005cc0 tx=0x7f4bc802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bcdffb640 1 -- 172.21.15.78:0/202968697 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8003cf0 con 0x7f4bd0108020 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd6533640 1 -- 172.21.15.78:0/202968697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd01a78e0 con 0x7f4bd0108020 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bcdffb640 1 -- 172.21.15.78:0/202968697 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bc80044f0 con 0x7f4bd0108020 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bcdffb640 1 -- 172.21.15.78:0/202968697 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc802fd10 con 0x7f4bd0108020 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.600+0000 7f4bd6533640 1 -- 172.21.15.78:0/202968697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd01a7ef0 con 0x7f4bd0108020 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.602+0000 7f4bd6533640 1 -- 172.21.15.78:0/202968697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd0102070 con 0x7f4bd0108020 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.603+0000 7f4bcdffb640 1 -- 172.21.15.78:0/202968697 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f4bc8004040 con 0x7f4bd0108020 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.603+0000 7f4bcdffb640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4bbc0b5330 0x7f4bbc0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.603+0000 7f4bcffff640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4bbc0b5330 0x7f4bbc0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.604+0000 7f4bcdffb640 1 -- 172.21.15.78:0/202968697 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f4bc81073f0 con 0x7f4bd0108020 2024-09-11T00:57:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.604+0000 7f4bcffff640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4bbc0b5330 0x7f4bbc0b77f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000abe0 tx=0x7f4bc000a5c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.607+0000 7f4bcdffb640 1 -- 172.21.15.78:0/202968697 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4bc80ce2d0 con 0x7f4bd0108020 2024-09-11T00:57:04.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:04.819+0000 7f4bd6533640 1 -- 172.21.15.78:0/202968697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f4bd002e5c0 con 0x7f4bd0108020 2024-09-11T00:57:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.239+0000 7f4bcdffb640 1 -- 172.21.15.78:0/202968697 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f4bc80cdbd0 con 0x7f4bd0108020 2024-09-11T00:57:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.241+0000 7f4bd6533640 1 -- 172.21.15.78:0/202968697 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4bbc0b5330 msgr2=0x7f4bbc0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.241+0000 7f4bd6533640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4bbc0b5330 0x7f4bbc0b77f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000abe0 tx=0x7f4bc000a5c0 comp rx=0 tx=0).stop 2024-09-11T00:57:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.241+0000 7f4bd6533640 1 -- 172.21.15.78:0/202968697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd0108020 msgr2=0x7f4bd010fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.241+0000 7f4bd6533640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd0108020 0x7f4bd010fdb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8005cc0 tx=0x7f4bc802f470 comp rx=0 tx=0).stop 2024-09-11T00:57:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.241+0000 7f4bd6533640 1 -- 172.21.15.78:0/202968697 shutdown_connections 2024-09-11T00:57:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.241+0000 7f4bd6533640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4bbc0b5330 0x7f4bbc0b77f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.241+0000 7f4bd6533640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd0108020 0x7f4bd010fdb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.241+0000 7f4bd6533640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd01076a0 0x7f4bd010f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.241+0000 7f4bd6533640 1 --2- 172.21.15.78:0/202968697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd0101c90 0x7f4bd010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.241+0000 7f4bd6533640 1 -- 172.21.15.78:0/202968697 >> 172.21.15.78:0/202968697 conn(0x7f4bd00fd700 msgr2=0x7f4bd00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.241+0000 7f4bd6533640 1 -- 172.21.15.78:0/202968697 shutdown_connections 2024-09-11T00:57:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:05.241+0000 7f4bd6533640 1 -- 172.21.15.78:0/202968697 wait complete. 2024-09-11T00:57:05.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:05 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] up:active 2024-09-11T00:57:05.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:05 smithi078 ceph-mon[31315]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T00:57:05.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:05 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:57:05.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:05 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/202968697' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-11T00:57:05.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:05 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] up:active 2024-09-11T00:57:05.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:05 smithi110 ceph-mon[34712]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T00:57:05.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:05 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:57:05.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:05 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/202968697' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-11T00:57:05.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:05 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] up:active 2024-09-11T00:57:05.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:05 smithi112 ceph-mon[35384]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T00:57:05.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:05 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:57:05.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:05 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/202968697' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-11T00:57:05.936 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-11T00:57:05.937 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-11T00:57:06.189 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:06 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/202968697' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T00:57:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:06 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:57:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:06 smithi078 ceph-mon[31315]: pgmap v268: 97 pgs: 97 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T00:57:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.664+0000 7f5af99f3640 1 -- 172.21.15.78:0/2951940771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5af4101cd0 msgr2=0x7f5af41020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.664+0000 7f5af99f3640 1 --2- 172.21.15.78:0/2951940771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5af4101cd0 0x7f5af41020b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009900 tx=0x7f5ae402a410 comp rx=0 tx=0).stop 2024-09-11T00:57:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.664+0000 7f5af99f3640 1 -- 172.21.15.78:0/2951940771 shutdown_connections 2024-09-11T00:57:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.664+0000 7f5af99f3640 1 --2- 172.21.15.78:0/2951940771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af4108060 0x7f5af410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.664+0000 7f5af99f3640 1 --2- 172.21.15.78:0/2951940771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5af41076e0 0x7f5af4107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.664+0000 7f5af99f3640 1 --2- 172.21.15.78:0/2951940771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5af4101cd0 0x7f5af41020b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.664+0000 7f5af99f3640 1 -- 172.21.15.78:0/2951940771 >> 172.21.15.78:0/2951940771 conn(0x7f5af40fd700 msgr2=0x7f5af40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.664+0000 7f5af99f3640 1 -- 172.21.15.78:0/2951940771 shutdown_connections 2024-09-11T00:57:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.664+0000 7f5af99f3640 1 -- 172.21.15.78:0/2951940771 wait complete. 2024-09-11T00:57:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5af99f3640 1 Processor -- start 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5af99f3640 1 -- start start 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5af99f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5af4101cd0 0x7f5af419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5af99f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5af41076e0 0x7f5af419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5af99f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af4108060 0x7f5af419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5af99f3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4111d20 con 0x7f5af41076e0 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5af99f3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4111ba0 con 0x7f5af4108060 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5af99f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4111ea0 con 0x7f5af4101cd0 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5aea7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5af41076e0 0x7f5af419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5aea7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5af41076e0 0x7f5af419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53034/0 (socket says 172.21.15.78:53034) 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5aea7fc640 1 -- 172.21.15.78:0/2898434008 learned_addr learned my addr 172.21.15.78:0/2898434008 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5af2ffd640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5af4101cd0 0x7f5af419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5af37fe640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af4108060 0x7f5af419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5aea7fc640 1 -- 172.21.15.78:0/2898434008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5af4101cd0 msgr2=0x7f5af419a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5aea7fc640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5af4101cd0 0x7f5af419a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5aea7fc640 1 -- 172.21.15.78:0/2898434008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af4108060 msgr2=0x7f5af419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5aea7fc640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af4108060 0x7f5af419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.665+0000 7f5aea7fc640 1 -- 172.21.15.78:0/2898434008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af41a54b0 con 0x7f5af41076e0 2024-09-11T00:57:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.666+0000 7f5aea7fc640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5af41076e0 0x7f5af419a5f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00e9f0 tx=0x7f5adc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.666+0000 7f5af89f1640 1 -- 172.21.15.78:0/2898434008 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc00cd80 con 0x7f5af41076e0 2024-09-11T00:57:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.666+0000 7f5af89f1640 1 -- 172.21.15.78:0/2898434008 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5adc0054d0 con 0x7f5af41076e0 2024-09-11T00:57:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.666+0000 7f5af99f3640 1 -- 172.21.15.78:0/2898434008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af41a57a0 con 0x7f5af41076e0 2024-09-11T00:57:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.666+0000 7f5af89f1640 1 -- 172.21.15.78:0/2898434008 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc010690 con 0x7f5af41076e0 2024-09-11T00:57:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.666+0000 7f5af99f3640 1 -- 172.21.15.78:0/2898434008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af41a5ce0 con 0x7f5af41076e0 2024-09-11T00:57:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.667+0000 7f5af99f3640 1 -- 172.21.15.78:0/2898434008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af41020b0 con 0x7f5af41076e0 2024-09-11T00:57:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.668+0000 7f5af89f1640 1 -- 172.21.15.78:0/2898434008 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f5adc004e90 con 0x7f5af41076e0 2024-09-11T00:57:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.669+0000 7f5af89f1640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5ae00b53b0 0x7f5ae00b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.669+0000 7f5af2ffd640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5ae00b53b0 0x7f5ae00b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.669+0000 7f5af89f1640 1 -- 172.21.15.78:0/2898434008 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f5adc0e60a0 con 0x7f5af41076e0 2024-09-11T00:57:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.669+0000 7f5af2ffd640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5ae00b53b0 0x7f5ae00b7870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4031440 tx=0x7f5ae4005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.673+0000 7f5af89f1640 1 -- 172.21.15.78:0/2898434008 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5adc0acf80 con 0x7f5af41076e0 2024-09-11T00:57:06.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:06 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/202968697' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T00:57:06.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:06 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:57:06.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:06 smithi110 ceph-mon[34712]: pgmap v268: 97 pgs: 97 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T00:57:06.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:06 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/202968697' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T00:57:06.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:06 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:57:06.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:06 smithi112 ceph-mon[35384]: pgmap v268: 97 pgs: 97 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T00:57:06.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.871+0000 7f5af99f3640 1 -- 172.21.15.78:0/2898434008 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5af4063590 con 0x7f5ae00b53b0 2024-09-11T00:57:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.895+0000 7f5af89f1640 1 -- 172.21.15.78:0/2898434008 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5af4063590 con 0x7f5ae00b53b0 2024-09-11T00:57:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.897+0000 7f5af99f3640 1 -- 172.21.15.78:0/2898434008 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5ae00b53b0 msgr2=0x7f5ae00b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.897+0000 7f5af99f3640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5ae00b53b0 0x7f5ae00b7870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4031440 tx=0x7f5ae4005b50 comp rx=0 tx=0).stop 2024-09-11T00:57:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.897+0000 7f5af99f3640 1 -- 172.21.15.78:0/2898434008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5af41076e0 msgr2=0x7f5af419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.897+0000 7f5af99f3640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5af41076e0 0x7f5af419a5f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00e9f0 tx=0x7f5adc00edf0 comp rx=0 tx=0).stop 2024-09-11T00:57:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.898+0000 7f5af99f3640 1 -- 172.21.15.78:0/2898434008 shutdown_connections 2024-09-11T00:57:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.898+0000 7f5af99f3640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5ae00b53b0 0x7f5ae00b7870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.898+0000 7f5af99f3640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af4108060 0x7f5af419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:06.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.898+0000 7f5af99f3640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5af41076e0 0x7f5af419a5f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:06.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.898+0000 7f5af99f3640 1 --2- 172.21.15.78:0/2898434008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5af4101cd0 0x7f5af419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:06.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.898+0000 7f5af99f3640 1 -- 172.21.15.78:0/2898434008 >> 172.21.15.78:0/2898434008 conn(0x7f5af40fd700 msgr2=0x7f5af40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:06.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.898+0000 7f5af99f3640 1 -- 172.21.15.78:0/2898434008 shutdown_connections 2024-09-11T00:57:06.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:06.898+0000 7f5af99f3640 1 -- 172.21.15.78:0/2898434008 wait complete. 2024-09-11T00:57:07.247 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:57:06 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:57:06.876+0000 7ff89f599640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:57:07.247 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:57:06 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:57:06.876+0000 7ff89f599640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:57:07.247 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:57:06 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:57:06.876+0000 7ff89f599640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:57:07.247 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:57:06 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:57:06.876+0000 7ff89f599640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:57:07.247 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:57:06 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:57:06.876+0000 7ff89f599640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:57:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:07 smithi078 ceph-mon[31315]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:07.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:07 smithi110 ceph-mon[34712]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:07.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:07 smithi112 ceph-mon[35384]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:07.728 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-11T00:57:07.981 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:08.401 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:08 smithi078 ceph-mon[31315]: pgmap v269: 97 pgs: 97 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3.6 KiB/s wr, 17 op/s 2024-09-11T00:57:08.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.483+0000 7f212a054640 1 -- 172.21.15.78:0/2918773731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2124102dc0 msgr2=0x7f21241031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:08.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.483+0000 7f212a054640 1 --2- 172.21.15.78:0/2918773731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2124102dc0 0x7f21241031a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2114009900 tx=0x7f211402a410 comp rx=0 tx=0).stop 2024-09-11T00:57:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.483+0000 7f212a054640 1 -- 172.21.15.78:0/2918773731 shutdown_connections 2024-09-11T00:57:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.483+0000 7f212a054640 1 --2- 172.21.15.78:0/2918773731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2124109150 0x7f212410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.483+0000 7f212a054640 1 --2- 172.21.15.78:0/2918773731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21241087d0 0x7f2124108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.483+0000 7f212a054640 1 --2- 172.21.15.78:0/2918773731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2124102dc0 0x7f21241031a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.483+0000 7f212a054640 1 -- 172.21.15.78:0/2918773731 >> 172.21.15.78:0/2918773731 conn(0x7f21240fe830 msgr2=0x7f2124100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.483+0000 7f212a054640 1 -- 172.21.15.78:0/2918773731 shutdown_connections 2024-09-11T00:57:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.483+0000 7f212a054640 1 -- 172.21.15.78:0/2918773731 wait complete. 2024-09-11T00:57:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.484+0000 7f212a054640 1 Processor -- start 2024-09-11T00:57:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.484+0000 7f212a054640 1 -- start start 2024-09-11T00:57:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f212a054640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2124102dc0 0x7f21241992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f212a054640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21241087d0 0x7f2124199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f212a054640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2124109150 0x7f212419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f212a054640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2124112e10 con 0x7f2124109150 2024-09-11T00:57:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f212a054640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2124112c90 con 0x7f2124102dc0 2024-09-11T00:57:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f212a054640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2124112f90 con 0x7f21241087d0 2024-09-11T00:57:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2123fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2124109150 0x7f212419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2123fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2124109150 0x7f212419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53084/0 (socket says 172.21.15.78:53084) 2024-09-11T00:57:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2123fff640 1 -- 172.21.15.78:0/2394435038 learned_addr learned my addr 172.21.15.78:0/2394435038 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2122ffd640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21241087d0 0x7f2124199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f21237fe640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2124102dc0 0x7f21241992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2123fff640 1 -- 172.21.15.78:0/2394435038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21241087d0 msgr2=0x7f2124199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2123fff640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21241087d0 0x7f2124199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2123fff640 1 -- 172.21.15.78:0/2394435038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2124102dc0 msgr2=0x7f21241992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2123fff640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2124102dc0 0x7f21241992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2123fff640 1 -- 172.21.15.78:0/2394435038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212419f390 con 0x7f2124109150 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f21237fe640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2124102dc0 0x7f21241992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2123fff640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2124109150 0x7f212419ebc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f211c00eaa0 tx=0x7f211c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2120ff9640 1 -- 172.21.15.78:0/2394435038 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c00cdb0 con 0x7f2124109150 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2120ff9640 1 -- 172.21.15.78:0/2394435038 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f211c005020 con 0x7f2124109150 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f212a054640 1 -- 172.21.15.78:0/2394435038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212419f680 con 0x7f2124109150 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f2120ff9640 1 -- 172.21.15.78:0/2394435038 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c0116d0 con 0x7f2124109150 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.485+0000 7f212a054640 1 -- 172.21.15.78:0/2394435038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21241a5d40 con 0x7f2124109150 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.486+0000 7f212a054640 1 -- 172.21.15.78:0/2394435038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21241031a0 con 0x7f2124109150 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.488+0000 7f2120ff9640 1 -- 172.21.15.78:0/2394435038 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f211c00f040 con 0x7f2124109150 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.488+0000 7f2120ff9640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f210c0b5440 0x7f210c0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.488+0000 7f21237fe640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f210c0b5440 0x7f210c0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.488+0000 7f2120ff9640 1 -- 172.21.15.78:0/2394435038 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f211c0e5bc0 con 0x7f2124109150 2024-09-11T00:57:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.489+0000 7f21237fe640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f210c0b5440 0x7f210c0b7900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2114031440 tx=0x7f2114005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.492+0000 7f2120ff9640 1 -- 172.21.15.78:0/2394435038 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f211c016030 con 0x7f2124109150 2024-09-11T00:57:08.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:08 smithi110 ceph-mon[34712]: pgmap v269: 97 pgs: 97 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3.6 KiB/s wr, 17 op/s 2024-09-11T00:57:08.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.694+0000 7f212a054640 1 -- 172.21.15.78:0/2394435038 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2124063590 con 0x7f210c0b5440 2024-09-11T00:57:08.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.718+0000 7f2120ff9640 1 -- 172.21.15.78:0/2394435038 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2124063590 con 0x7f210c0b5440 2024-09-11T00:57:08.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.720+0000 7f212a054640 1 -- 172.21.15.78:0/2394435038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f210c0b5440 msgr2=0x7f210c0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:08.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.720+0000 7f212a054640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f210c0b5440 0x7f210c0b7900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2114031440 tx=0x7f2114005a20 comp rx=0 tx=0).stop 2024-09-11T00:57:08.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.720+0000 7f212a054640 1 -- 172.21.15.78:0/2394435038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2124109150 msgr2=0x7f212419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:08.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.720+0000 7f212a054640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2124109150 0x7f212419ebc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f211c00eaa0 tx=0x7f211c00ef60 comp rx=0 tx=0).stop 2024-09-11T00:57:08.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.720+0000 7f212a054640 1 -- 172.21.15.78:0/2394435038 shutdown_connections 2024-09-11T00:57:08.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.720+0000 7f212a054640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f210c0b5440 0x7f210c0b7900 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:08.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.720+0000 7f212a054640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2124109150 0x7f212419ebc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:08.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.720+0000 7f212a054640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21241087d0 0x7f2124199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:08.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.720+0000 7f212a054640 1 --2- 172.21.15.78:0/2394435038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2124102dc0 0x7f21241992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:08.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.720+0000 7f212a054640 1 -- 172.21.15.78:0/2394435038 >> 172.21.15.78:0/2394435038 conn(0x7f21240fe830 msgr2=0x7f2124100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:08.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.720+0000 7f212a054640 1 -- 172.21.15.78:0/2394435038 shutdown_connections 2024-09-11T00:57:08.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:08.720+0000 7f212a054640 1 -- 172.21.15.78:0/2394435038 wait complete. 2024-09-11T00:57:08.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:08 smithi112 ceph-mon[35384]: pgmap v269: 97 pgs: 97 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3.6 KiB/s wr, 17 op/s 2024-09-11T00:57:09.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:09 smithi078 ceph-mon[31315]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-11T00:57:09.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:09 smithi078 ceph-mon[31315]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:09.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-11T00:57:09.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:09 smithi110 ceph-mon[34712]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-11T00:57:09.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:09 smithi110 ceph-mon[34712]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:09.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:09 smithi112 ceph-mon[35384]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-11T00:57:09.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:09 smithi112 ceph-mon[35384]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:09.747 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:10.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.255+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2994056655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d281087f0 msgr2=0x7f4d28108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:10.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.255+0000 7f4d2fb22640 1 --2- 172.21.15.78:0/2994056655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d281087f0 0x7f4d28108c30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009900 tx=0x7f4d1802a410 comp rx=0 tx=0).stop 2024-09-11T00:57:10.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.255+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2994056655 shutdown_connections 2024-09-11T00:57:10.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.255+0000 7f4d2fb22640 1 --2- 172.21.15.78:0/2994056655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d28109170 0x7f4d2810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:10.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.255+0000 7f4d2fb22640 1 --2- 172.21.15.78:0/2994056655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d281087f0 0x7f4d28108c30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:10.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.255+0000 7f4d2fb22640 1 --2- 172.21.15.78:0/2994056655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d28102de0 0x7f4d281031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:10.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.255+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2994056655 >> 172.21.15.78:0/2994056655 conn(0x7f4d280fe830 msgr2=0x7f4d28100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:10.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.255+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2994056655 shutdown_connections 2024-09-11T00:57:10.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.255+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2994056655 wait complete. 2024-09-11T00:57:10.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.256+0000 7f4d2fb22640 1 Processor -- start 2024-09-11T00:57:10.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.256+0000 7f4d2fb22640 1 -- start start 2024-09-11T00:57:10.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.256+0000 7f4d2fb22640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d28102de0 0x7f4d28199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.256+0000 7f4d2fb22640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d281087f0 0x7f4d28199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.256+0000 7f4d2fb22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d28109170 0x7f4d2819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.256+0000 7f4d2fb22640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28112df0 con 0x7f4d281087f0 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.256+0000 7f4d2fb22640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28112c70 con 0x7f4d28102de0 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.256+0000 7f4d2fb22640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28112f70 con 0x7f4d28109170 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.256+0000 7f4d2d0bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d281087f0 0x7f4d28199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.257+0000 7f4d2d0bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d281087f0 0x7f4d28199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53108/0 (socket says 172.21.15.78:53108) 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.257+0000 7f4d2d0bd640 1 -- 172.21.15.78:0/2442501965 learned_addr learned my addr 172.21.15.78:0/2442501965 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.257+0000 7f4d2d8be640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d28102de0 0x7f4d28199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.257+0000 7f4d2e0bf640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d28109170 0x7f4d2819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.257+0000 7f4d2d0bd640 1 -- 172.21.15.78:0/2442501965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d28109170 msgr2=0x7f4d2819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.257+0000 7f4d2d0bd640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d28109170 0x7f4d2819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.257+0000 7f4d2d0bd640 1 -- 172.21.15.78:0/2442501965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d28102de0 msgr2=0x7f4d28199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.257+0000 7f4d2d0bd640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d28102de0 0x7f4d28199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.257+0000 7f4d2d0bd640 1 -- 172.21.15.78:0/2442501965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2819f3d0 con 0x7f4d281087f0 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.257+0000 7f4d2d0bd640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d281087f0 0x7f4d28199860 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005940 tx=0x7f4d180040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.258+0000 7f4d1effd640 1 -- 172.21.15.78:0/2442501965 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1803f030 con 0x7f4d281087f0 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.258+0000 7f4d1effd640 1 -- 172.21.15.78:0/2442501965 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d180044c0 con 0x7f4d281087f0 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.258+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2442501965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2819f660 con 0x7f4d281087f0 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.258+0000 7f4d1effd640 1 -- 172.21.15.78:0/2442501965 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18035070 con 0x7f4d281087f0 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.258+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2442501965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d281a5d80 con 0x7f4d281087f0 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.259+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2442501965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d281031c0 con 0x7f4d281087f0 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.261+0000 7f4d1effd640 1 -- 172.21.15.78:0/2442501965 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f4d1803ba20 con 0x7f4d281087f0 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.262+0000 7f4d1effd640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4d140b5440 0x7f4d140b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.262+0000 7f4d2d8be640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4d140b5440 0x7f4d140b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.262+0000 7f4d1effd640 1 -- 172.21.15.78:0/2442501965 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f4d18106380 con 0x7f4d281087f0 2024-09-11T00:57:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.262+0000 7f4d2d8be640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4d140b5440 0x7f4d140b7900 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009bd0 tx=0x7f4d24005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:10.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.271+0000 7f4d1effd640 1 -- 172.21.15.78:0/2442501965 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4d180cca50 con 0x7f4d281087f0 2024-09-11T00:57:10.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:10 smithi110 ceph-mon[34712]: pgmap v270: 97 pgs: 97 active+clean; 605 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 23 op/s 2024-09-11T00:57:10.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:10 smithi112 ceph-mon[35384]: pgmap v270: 97 pgs: 97 active+clean; 605 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 23 op/s 2024-09-11T00:57:10.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.483+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2442501965 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4d28063590 con 0x7f4d140b5440 2024-09-11T00:57:10.485 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:10 smithi078 ceph-mon[31315]: pgmap v270: 97 pgs: 97 active+clean; 605 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 23 op/s 2024-09-11T00:57:10.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.488+0000 7f4d1effd640 1 -- 172.21.15.78:0/2442501965 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4d28063590 con 0x7f4d140b5440 2024-09-11T00:57:10.490 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/qa/sv_0/6b207f1b-fd5f-48d8-9f10-9b47fe23b8b6 2024-09-11T00:57:10.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.491+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2442501965 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4d140b5440 msgr2=0x7f4d140b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:10.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.491+0000 7f4d2fb22640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4d140b5440 0x7f4d140b7900 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009bd0 tx=0x7f4d24005f90 comp rx=0 tx=0).stop 2024-09-11T00:57:10.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.491+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2442501965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d281087f0 msgr2=0x7f4d28199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:10.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.491+0000 7f4d2fb22640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d281087f0 0x7f4d28199860 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005940 tx=0x7f4d180040c0 comp rx=0 tx=0).stop 2024-09-11T00:57:10.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.491+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2442501965 shutdown_connections 2024-09-11T00:57:10.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.491+0000 7f4d2fb22640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4d140b5440 0x7f4d140b7900 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:10.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.491+0000 7f4d2fb22640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d28109170 0x7f4d2819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:10.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.491+0000 7f4d2fb22640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d281087f0 0x7f4d28199860 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:10.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.491+0000 7f4d2fb22640 1 --2- 172.21.15.78:0/2442501965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d28102de0 0x7f4d28199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:10.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.491+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2442501965 >> 172.21.15.78:0/2442501965 conn(0x7f4d280fe830 msgr2=0x7f4d28100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:10.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.491+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2442501965 shutdown_connections 2024-09-11T00:57:10.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:10.491+0000 7f4d2fb22640 1 -- 172.21.15.78:0/2442501965 wait complete. 2024-09-11T00:57:11.187 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-11T00:57:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:11 smithi078 ceph-mon[31315]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:11.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:11 smithi110 ceph-mon[34712]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:11.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:11 smithi112 ceph-mon[35384]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:11.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.912+0000 7f97ef2af640 1 -- 172.21.15.78:0/4011522536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97e81076a0 msgr2=0x7f97e8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:11.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.912+0000 7f97ef2af640 1 --2- 172.21.15.78:0/4011522536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97e81076a0 0x7f97e8107ae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009900 tx=0x7f97d802a410 comp rx=0 tx=0).stop 2024-09-11T00:57:11.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.912+0000 7f97ef2af640 1 -- 172.21.15.78:0/4011522536 shutdown_connections 2024-09-11T00:57:11.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.912+0000 7f97ef2af640 1 --2- 172.21.15.78:0/4011522536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e8108020 0x7f97e810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:11.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.912+0000 7f97ef2af640 1 --2- 172.21.15.78:0/4011522536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97e81076a0 0x7f97e8107ae0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:11.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.912+0000 7f97ef2af640 1 --2- 172.21.15.78:0/4011522536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97e8101c90 0x7f97e8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:11.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.912+0000 7f97ef2af640 1 -- 172.21.15.78:0/4011522536 >> 172.21.15.78:0/4011522536 conn(0x7f97e80fd6c0 msgr2=0x7f97e80ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:11.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.912+0000 7f97ef2af640 1 -- 172.21.15.78:0/4011522536 shutdown_connections 2024-09-11T00:57:11.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.912+0000 7f97ef2af640 1 -- 172.21.15.78:0/4011522536 wait complete. 2024-09-11T00:57:11.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.913+0000 7f97ef2af640 1 Processor -- start 2024-09-11T00:57:11.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.913+0000 7f97ef2af640 1 -- start start 2024-09-11T00:57:11.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.913+0000 7f97ef2af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97e8101c90 0x7f97e80fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:11.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.913+0000 7f97ef2af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e81076a0 0x7f97e80fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:11.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.913+0000 7f97ef2af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97e8108020 0x7f97e80fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:11.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.913+0000 7f97ef2af640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8111d30 con 0x7f97e8108020 2024-09-11T00:57:11.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.913+0000 7f97ef2af640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8111bb0 con 0x7f97e8101c90 2024-09-11T00:57:11.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.913+0000 7f97ef2af640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8111eb0 con 0x7f97e81076a0 2024-09-11T00:57:11.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.913+0000 7f97ed84c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97e8108020 0x7f97e80fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:11.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.913+0000 7f97ed84c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97e8108020 0x7f97e80fe640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53138/0 (socket says 172.21.15.78:53138) 2024-09-11T00:57:11.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.913+0000 7f97ed84c640 1 -- 172.21.15.78:0/2185450136 learned_addr learned my addr 172.21.15.78:0/2185450136 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97ed04b640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97e8101c90 0x7f97e80fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97ec84a640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e81076a0 0x7f97e80fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97ed84c640 1 -- 172.21.15.78:0/2185450136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e81076a0 msgr2=0x7f97e80fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97ed84c640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e81076a0 0x7f97e80fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97ed84c640 1 -- 172.21.15.78:0/2185450136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97e8101c90 msgr2=0x7f97e80fdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97ed84c640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97e8101c90 0x7f97e80fdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97ed84c640 1 -- 172.21.15.78:0/2185450136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e81a77b0 con 0x7f97e8108020 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97ed84c640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97e8108020 0x7f97e80fe640 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f97e000b970 tx=0x7f97e000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97de7fc640 1 -- 172.21.15.78:0/2185450136 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0019070 con 0x7f97e8108020 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97de7fc640 1 -- 172.21.15.78:0/2185450136 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e0014440 con 0x7f97e8108020 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97ef2af640 1 -- 172.21.15.78:0/2185450136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e81a7aa0 con 0x7f97e8108020 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97de7fc640 1 -- 172.21.15.78:0/2185450136 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e00075d0 con 0x7f97e8108020 2024-09-11T00:57:11.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.914+0000 7f97ef2af640 1 -- 172.21.15.78:0/2185450136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e81a7f60 con 0x7f97e8108020 2024-09-11T00:57:11.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.915+0000 7f97ef2af640 1 -- 172.21.15.78:0/2185450136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e8102070 con 0x7f97e8108020 2024-09-11T00:57:11.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.916+0000 7f97de7fc640 1 -- 172.21.15.78:0/2185450136 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f97e000cb40 con 0x7f97e8108020 2024-09-11T00:57:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.917+0000 7f97de7fc640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f97d40b53c0 0x7f97d40b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.917+0000 7f97ed04b640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f97d40b53c0 0x7f97d40b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.917+0000 7f97de7fc640 1 -- 172.21.15.78:0/2185450136 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f97e00e5540 con 0x7f97e8108020 2024-09-11T00:57:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.917+0000 7f97ed04b640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f97d40b53c0 0x7f97d40b7880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f97e40058c0 tx=0x7f97e400a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:11.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:11.921+0000 7f97de7fc640 1 -- 172.21.15.78:0/2185450136 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f97e00abc70 con 0x7f97e8108020 2024-09-11T00:57:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.123+0000 7f97ef2af640 1 -- 172.21.15.78:0/2185450136 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f97e8100290 con 0x7f97d40b53c0 2024-09-11T00:57:12.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.149+0000 7f97de7fc640 1 -- 172.21.15.78:0/2185450136 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f97e8100290 con 0x7f97d40b53c0 2024-09-11T00:57:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.151+0000 7f97ef2af640 1 -- 172.21.15.78:0/2185450136 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f97d40b53c0 msgr2=0x7f97d40b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.151+0000 7f97ef2af640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f97d40b53c0 0x7f97d40b7880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f97e40058c0 tx=0x7f97e400a2d0 comp rx=0 tx=0).stop 2024-09-11T00:57:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.151+0000 7f97ef2af640 1 -- 172.21.15.78:0/2185450136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97e8108020 msgr2=0x7f97e80fe640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.151+0000 7f97ef2af640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97e8108020 0x7f97e80fe640 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f97e000b970 tx=0x7f97e000be30 comp rx=0 tx=0).stop 2024-09-11T00:57:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.151+0000 7f97ef2af640 1 -- 172.21.15.78:0/2185450136 shutdown_connections 2024-09-11T00:57:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.151+0000 7f97ef2af640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f97d40b53c0 0x7f97d40b7880 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.151+0000 7f97ef2af640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97e8108020 0x7f97e80fe640 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.151+0000 7f97ef2af640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e81076a0 0x7f97e80fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.151+0000 7f97ef2af640 1 --2- 172.21.15.78:0/2185450136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97e8101c90 0x7f97e80fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.151+0000 7f97ef2af640 1 -- 172.21.15.78:0/2185450136 >> 172.21.15.78:0/2185450136 conn(0x7f97e80fd6c0 msgr2=0x7f97e806b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.151+0000 7f97ef2af640 1 -- 172.21.15.78:0/2185450136 shutdown_connections 2024-09-11T00:57:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:12.151+0000 7f97ef2af640 1 -- 172.21.15.78:0/2185450136 wait complete. 2024-09-11T00:57:12.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:12 smithi078 ceph-mon[31315]: pgmap v271: 97 pgs: 97 active+clean; 605 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.1 KiB/s wr, 22 op/s 2024-09-11T00:57:12.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:12 smithi110 ceph-mon[34712]: pgmap v271: 97 pgs: 97 active+clean; 605 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.1 KiB/s wr, 22 op/s 2024-09-11T00:57:12.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:12 smithi112 ceph-mon[35384]: pgmap v271: 97 pgs: 97 active+clean; 605 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.1 KiB/s wr, 22 op/s 2024-09-11T00:57:12.860 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-11T00:57:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:13 smithi078 ceph-mon[31315]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:13.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.602+0000 7f8d670dc640 1 -- 172.21.15.78:0/3179109029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d60102dc0 msgr2=0x7f8d601031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:13.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.602+0000 7f8d670dc640 1 --2- 172.21.15.78:0/3179109029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d60102dc0 0x7f8d601031a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009900 tx=0x7f8d5402a410 comp rx=0 tx=0).stop 2024-09-11T00:57:13.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.602+0000 7f8d670dc640 1 -- 172.21.15.78:0/3179109029 shutdown_connections 2024-09-11T00:57:13.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.602+0000 7f8d670dc640 1 --2- 172.21.15.78:0/3179109029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d60109150 0x7f8d6010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.602+0000 7f8d670dc640 1 --2- 172.21.15.78:0/3179109029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d601087d0 0x7f8d60108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.602+0000 7f8d670dc640 1 --2- 172.21.15.78:0/3179109029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d60102dc0 0x7f8d601031a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.602+0000 7f8d670dc640 1 -- 172.21.15.78:0/3179109029 >> 172.21.15.78:0/3179109029 conn(0x7f8d600fe830 msgr2=0x7f8d60100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.602+0000 7f8d670dc640 1 -- 172.21.15.78:0/3179109029 shutdown_connections 2024-09-11T00:57:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.602+0000 7f8d670dc640 1 -- 172.21.15.78:0/3179109029 wait complete. 2024-09-11T00:57:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.603+0000 7f8d670dc640 1 Processor -- start 2024-09-11T00:57:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.603+0000 7f8d670dc640 1 -- start start 2024-09-11T00:57:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.603+0000 7f8d670dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d60102dc0 0x7f8d60199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.603+0000 7f8d670dc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d601087d0 0x7f8d60199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.603+0000 7f8d670dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d60109150 0x7f8d6019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.603+0000 7f8d670dc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d60112e50 con 0x7f8d60109150 2024-09-11T00:57:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.603+0000 7f8d670dc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d60112cd0 con 0x7f8d601087d0 2024-09-11T00:57:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.603+0000 7f8d670dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d60112fd0 con 0x7f8d60102dc0 2024-09-11T00:57:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.603+0000 7f8d65679640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d60109150 0x7f8d6019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.603+0000 7f8d65679640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d60109150 0x7f8d6019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40074/0 (socket says 172.21.15.78:40074) 2024-09-11T00:57:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.603+0000 7f8d65679640 1 -- 172.21.15.78:0/2326627520 learned_addr learned my addr 172.21.15.78:0/2326627520 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d64e78640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d60102dc0 0x7f8d60199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d5ffff640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d601087d0 0x7f8d60199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d65679640 1 -- 172.21.15.78:0/2326627520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d60102dc0 msgr2=0x7f8d60199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d65679640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d60102dc0 0x7f8d60199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d65679640 1 -- 172.21.15.78:0/2326627520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d601087d0 msgr2=0x7f8d60199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d65679640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d601087d0 0x7f8d60199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d65679640 1 -- 172.21.15.78:0/2326627520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6019f4f0 con 0x7f8d60109150 2024-09-11T00:57:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d5ffff640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d601087d0 0x7f8d60199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:57:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d65679640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d60109150 0x7f8d6019ed20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800b970 tx=0x7f8d5800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d5dffb640 1 -- 172.21.15.78:0/2326627520 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58019070 con 0x7f8d60109150 2024-09-11T00:57:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d670dc640 1 -- 172.21.15.78:0/2326627520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6019f7e0 con 0x7f8d60109150 2024-09-11T00:57:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d5dffb640 1 -- 172.21.15.78:0/2326627520 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d58014440 con 0x7f8d60109150 2024-09-11T00:57:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d5dffb640 1 -- 172.21.15.78:0/2326627520 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d580075d0 con 0x7f8d60109150 2024-09-11T00:57:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.604+0000 7f8d670dc640 1 -- 172.21.15.78:0/2326627520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d601a5ea0 con 0x7f8d60109150 2024-09-11T00:57:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.605+0000 7f8d670dc640 1 -- 172.21.15.78:0/2326627520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d60103220 con 0x7f8d60109150 2024-09-11T00:57:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.606+0000 7f8d5dffb640 1 -- 172.21.15.78:0/2326627520 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f8d5800cb40 con 0x7f8d60109150 2024-09-11T00:57:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.607+0000 7f8d5dffb640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d500b53c0 0x7f8d500b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.607+0000 7f8d5dffb640 1 -- 172.21.15.78:0/2326627520 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f8d580e5a70 con 0x7f8d60109150 2024-09-11T00:57:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.607+0000 7f8d64e78640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d500b53c0 0x7f8d500b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:13.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.608+0000 7f8d64e78640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d500b53c0 0x7f8d500b7880 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8d54006010 tx=0x7f8d54000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:13.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.610+0000 7f8d5dffb640 1 -- 172.21.15.78:0/2326627520 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8d580ac7f0 con 0x7f8d60109150 2024-09-11T00:57:13.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:13 smithi110 ceph-mon[34712]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:13.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:13 smithi112 ceph-mon[35384]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.813+0000 7f8d670dc640 1 -- 172.21.15.78:0/2326627520 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8d6010af90 con 0x7f8d500b53c0 2024-09-11T00:57:13.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.818+0000 7f8d5dffb640 1 -- 172.21.15.78:0/2326627520 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8d6010af90 con 0x7f8d500b53c0 2024-09-11T00:57:13.819 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/qa/sv_1/6ce94f05-2660-4611-9762-7179453c5d71 2024-09-11T00:57:13.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.820+0000 7f8d670dc640 1 -- 172.21.15.78:0/2326627520 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d500b53c0 msgr2=0x7f8d500b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:13.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.820+0000 7f8d670dc640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d500b53c0 0x7f8d500b7880 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8d54006010 tx=0x7f8d54000bc0 comp rx=0 tx=0).stop 2024-09-11T00:57:13.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.820+0000 7f8d670dc640 1 -- 172.21.15.78:0/2326627520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d60109150 msgr2=0x7f8d6019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:13.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.820+0000 7f8d670dc640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d60109150 0x7f8d6019ed20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800b970 tx=0x7f8d5800be30 comp rx=0 tx=0).stop 2024-09-11T00:57:13.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.820+0000 7f8d670dc640 1 -- 172.21.15.78:0/2326627520 shutdown_connections 2024-09-11T00:57:13.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.820+0000 7f8d670dc640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d500b53c0 0x7f8d500b7880 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:13.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.820+0000 7f8d670dc640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d60109150 0x7f8d6019ed20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:13.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.820+0000 7f8d670dc640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d601087d0 0x7f8d60199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:13.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.820+0000 7f8d670dc640 1 --2- 172.21.15.78:0/2326627520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d60102dc0 0x7f8d60199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:13.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.820+0000 7f8d670dc640 1 -- 172.21.15.78:0/2326627520 >> 172.21.15.78:0/2326627520 conn(0x7f8d600fe830 msgr2=0x7f8d60100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:13.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.820+0000 7f8d670dc640 1 -- 172.21.15.78:0/2326627520 shutdown_connections 2024-09-11T00:57:13.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:13.820+0000 7f8d670dc640 1 -- 172.21.15.78:0/2326627520 wait complete. 2024-09-11T00:57:14.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:14 smithi078 ceph-mon[31315]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:14.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:14 smithi078 ceph-mon[31315]: pgmap v272: 97 pgs: 97 active+clean; 605 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 23 op/s 2024-09-11T00:57:14.520 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:57:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:14 smithi110 ceph-mon[34712]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:14 smithi110 ceph-mon[34712]: pgmap v272: 97 pgs: 97 active+clean; 605 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 23 op/s 2024-09-11T00:57:14.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:14 smithi112 ceph-mon[35384]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:57:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:14 smithi112 ceph-mon[35384]: pgmap v272: 97 pgs: 97 active+clean; 605 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 23 op/s 2024-09-11T00:57:14.765 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:15.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.289+0000 7f9f91845640 1 -- 172.21.15.78:0/2659199978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8c108030 msgr2=0x7f9f8c10e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:15.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.289+0000 7f9f91845640 1 --2- 172.21.15.78:0/2659199978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8c108030 0x7f9f8c10e830 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009ca0 tx=0x7f9f8402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:15.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.289+0000 7f9f91845640 1 -- 172.21.15.78:0/2659199978 shutdown_connections 2024-09-11T00:57:15.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.289+0000 7f9f91845640 1 --2- 172.21.15.78:0/2659199978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8c108030 0x7f9f8c10e830 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:15.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.289+0000 7f9f91845640 1 --2- 172.21.15.78:0/2659199978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c1076b0 0x7f9f8c107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:15.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.289+0000 7f9f91845640 1 --2- 172.21.15.78:0/2659199978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f8c101ca0 0x7f9f8c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:15.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.289+0000 7f9f91845640 1 -- 172.21.15.78:0/2659199978 >> 172.21.15.78:0/2659199978 conn(0x7f9f8c0fd710 msgr2=0x7f9f8c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:15.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.289+0000 7f9f91845640 1 -- 172.21.15.78:0/2659199978 shutdown_connections 2024-09-11T00:57:15.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.289+0000 7f9f91845640 1 -- 172.21.15.78:0/2659199978 wait complete. 2024-09-11T00:57:15.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f91845640 1 Processor -- start 2024-09-11T00:57:15.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f91845640 1 -- start start 2024-09-11T00:57:15.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f91845640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c101ca0 0x7f9f8c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:15.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f91845640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8c1076b0 0x7f9f8c19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:15.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f91845640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f8c108030 0x7f9f8c1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:15.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f91845640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c112df0 con 0x7f9f8c1076b0 2024-09-11T00:57:15.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f91845640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c112c70 con 0x7f9f8c108030 2024-09-11T00:57:15.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f91845640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c112f70 con 0x7f9f8c101ca0 2024-09-11T00:57:15.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f8a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8c1076b0 0x7f9f8c19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:15.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f8a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8c1076b0 0x7f9f8c19b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40090/0 (socket says 172.21.15.78:40090) 2024-09-11T00:57:15.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f8a7fc640 1 -- 172.21.15.78:0/1388358111 learned_addr learned my addr 172.21.15.78:0/1388358111 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:15.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f8affd640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c101ca0 0x7f9f8c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:15.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f8a7fc640 1 -- 172.21.15.78:0/1388358111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c101ca0 msgr2=0x7f9f8c19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:15.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f8a7fc640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c101ca0 0x7f9f8c19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:15.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f8a7fc640 1 -- 172.21.15.78:0/1388358111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f8c108030 msgr2=0x7f9f8c1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:15.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.290+0000 7f9f8a7fc640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f8c108030 0x7f9f8c1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:15.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.291+0000 7f9f8a7fc640 1 -- 172.21.15.78:0/1388358111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c1a14f0 con 0x7f9f8c1076b0 2024-09-11T00:57:15.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.291+0000 7f9f8a7fc640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8c1076b0 0x7f9f8c19b980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400b4a0 tx=0x7f9f7400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:15.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.291+0000 7f9f90843640 1 -- 172.21.15.78:0/1388358111 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74014070 con 0x7f9f8c1076b0 2024-09-11T00:57:15.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.291+0000 7f9f90843640 1 -- 172.21.15.78:0/1388358111 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f74010ac0 con 0x7f9f8c1076b0 2024-09-11T00:57:15.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.291+0000 7f9f90843640 1 -- 172.21.15.78:0/1388358111 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74017c30 con 0x7f9f8c1076b0 2024-09-11T00:57:15.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.291+0000 7f9f91845640 1 -- 172.21.15.78:0/1388358111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c1a17e0 con 0x7f9f8c1076b0 2024-09-11T00:57:15.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.291+0000 7f9f91845640 1 -- 172.21.15.78:0/1388358111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c1a7f20 con 0x7f9f8c1076b0 2024-09-11T00:57:15.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.292+0000 7f9f91845640 1 -- 172.21.15.78:0/1388358111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8c102080 con 0x7f9f8c1076b0 2024-09-11T00:57:15.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.293+0000 7f9f90843640 1 -- 172.21.15.78:0/1388358111 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f9f74010610 con 0x7f9f8c1076b0 2024-09-11T00:57:15.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.294+0000 7f9f90843640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9f780b53c0 0x7f9f780b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:15.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.294+0000 7f9f8affd640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9f780b53c0 0x7f9f780b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:15.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.295+0000 7f9f90843640 1 -- 172.21.15.78:0/1388358111 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f9f740e57c0 con 0x7f9f8c1076b0 2024-09-11T00:57:15.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.295+0000 7f9f8affd640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9f780b53c0 0x7f9f780b7880 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c005e10 tx=0x7f9f7c005da0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:15.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.297+0000 7f9f90843640 1 -- 172.21.15.78:0/1388358111 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9f740ac540 con 0x7f9f8c1076b0 2024-09-11T00:57:15.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.510+0000 7f9f91845640 1 -- 172.21.15.78:0/1388358111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f8c19c570 con 0x7f9f8c1076b0 2024-09-11T00:57:15.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.511+0000 7f9f90843640 1 -- 172.21.15.78:0/1388358111 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11979 (secure 0 0 0) 0x7f9f740abe40 con 0x7f9f8c1076b0 2024-09-11T00:57:15.514 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:57:15.514 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-09-11T00:57:05:238610+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T00:57:04.821148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T00:57:15.514 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-11T00:57:15.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.513+0000 7f9f91845640 1 -- 172.21.15.78:0/1388358111 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9f780b53c0 msgr2=0x7f9f780b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:15.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.513+0000 7f9f91845640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9f780b53c0 0x7f9f780b7880 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c005e10 tx=0x7f9f7c005da0 comp rx=0 tx=0).stop 2024-09-11T00:57:15.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.513+0000 7f9f91845640 1 -- 172.21.15.78:0/1388358111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8c1076b0 msgr2=0x7f9f8c19b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:15.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.513+0000 7f9f91845640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8c1076b0 0x7f9f8c19b980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400b4a0 tx=0x7f9f7400b960 comp rx=0 tx=0).stop 2024-09-11T00:57:15.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.513+0000 7f9f91845640 1 -- 172.21.15.78:0/1388358111 shutdown_connections 2024-09-11T00:57:15.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.513+0000 7f9f91845640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9f780b53c0 0x7f9f780b7880 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:15.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.513+0000 7f9f91845640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f8c108030 0x7f9f8c1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:15.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.513+0000 7f9f91845640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8c1076b0 0x7f9f8c19b980 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:15.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.513+0000 7f9f91845640 1 --2- 172.21.15.78:0/1388358111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f8c101ca0 0x7f9f8c19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:15.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.514+0000 7f9f91845640 1 -- 172.21.15.78:0/1388358111 >> 172.21.15.78:0/1388358111 conn(0x7f9f8c0fd710 msgr2=0x7f9f8c104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:15.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.514+0000 7f9f91845640 1 -- 172.21.15.78:0/1388358111 shutdown_connections 2024-09-11T00:57:15.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:15.514+0000 7f9f91845640 1 -- 172.21.15.78:0/1388358111 wait complete. 2024-09-11T00:57:15.892 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:15 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1388358111' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:57:15.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:15 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1388358111' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:57:15.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:15 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1388358111' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:57:16.482 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:57:16.738 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:16.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-11T00:57:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-11T00:57:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-11T00:57:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-11T00:57:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]} : dispatch 2024-09-11T00:57:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]} : dispatch 2024-09-11T00:57:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:16 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-09-11T00:57:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:16 smithi078 ceph-mon[31315]: pgmap v273: 97 pgs: 97 active+clean; 625 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 24 op/s 2024-09-11T00:57:16.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-11T00:57:16.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-11T00:57:16.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-11T00:57:16.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-11T00:57:16.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]} : dispatch 2024-09-11T00:57:16.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]} : dispatch 2024-09-11T00:57:16.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:16 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-09-11T00:57:16.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:16 smithi110 ceph-mon[34712]: pgmap v273: 97 pgs: 97 active+clean; 625 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 24 op/s 2024-09-11T00:57:16.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-11T00:57:16.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-11T00:57:16.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-11T00:57:16.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-11T00:57:16.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]} : dispatch 2024-09-11T00:57:16.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]} : dispatch 2024-09-11T00:57:16.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:16 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-09-11T00:57:16.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:16 smithi112 ceph-mon[35384]: pgmap v273: 97 pgs: 97 active+clean; 625 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 24 op/s 2024-09-11T00:57:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.209+0000 7f388e362640 1 -- 172.21.15.78:0/3276580204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f388810d120 msgr2=0x7f388810f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.209+0000 7f388e362640 1 --2- 172.21.15.78:0/3276580204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f388810d120 0x7f388810f580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3880009ca0 tx=0x7f388002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.209+0000 7f388e362640 1 -- 172.21.15.78:0/3276580204 shutdown_connections 2024-09-11T00:57:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.209+0000 7f388e362640 1 --2- 172.21.15.78:0/3276580204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f388810d120 0x7f388810f580 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.209+0000 7f388e362640 1 --2- 172.21.15.78:0/3276580204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38881045e0 0x7f388810cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.209+0000 7f388e362640 1 --2- 172.21.15.78:0/3276580204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3888103cc0 0x7f38881040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.209+0000 7f388e362640 1 -- 172.21.15.78:0/3276580204 >> 172.21.15.78:0/3276580204 conn(0x7f38880fd700 msgr2=0x7f38880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.209+0000 7f388e362640 1 -- 172.21.15.78:0/3276580204 shutdown_connections 2024-09-11T00:57:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.209+0000 7f388e362640 1 -- 172.21.15.78:0/3276580204 wait complete. 2024-09-11T00:57:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.209+0000 7f388e362640 1 Processor -- start 2024-09-11T00:57:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.209+0000 7f388e362640 1 -- start start 2024-09-11T00:57:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f388e362640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3888103cc0 0x7f3888199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f388e362640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38881045e0 0x7f3888199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f388e362640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f388810d120 0x7f388819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f388e362640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3888112e70 con 0x7f3888103cc0 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f388e362640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3888112cf0 con 0x7f388810d120 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f388e362640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3888112ff0 con 0x7f38881045e0 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f3887fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3888103cc0 0x7f3888199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f3887fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3888103cc0 0x7f3888199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40114/0 (socket says 172.21.15.78:40114) 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f3887fff640 1 -- 172.21.15.78:0/3118254597 learned_addr learned my addr 172.21.15.78:0/3118254597 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f387ffff640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38881045e0 0x7f3888199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f388c8ff640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f388810d120 0x7f388819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f3887fff640 1 -- 172.21.15.78:0/3118254597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38881045e0 msgr2=0x7f3888199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f3887fff640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38881045e0 0x7f3888199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f3887fff640 1 -- 172.21.15.78:0/3118254597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f388810d120 msgr2=0x7f388819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f3887fff640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f388810d120 0x7f388819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f3887fff640 1 -- 172.21.15.78:0/3118254597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388819f4b0 con 0x7f3888103cc0 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f3887fff640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3888103cc0 0x7f3888199400 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f387800b4a0 tx=0x7f387800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f3885ffb640 1 -- 172.21.15.78:0/3118254597 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878014070 con 0x7f3888103cc0 2024-09-11T00:57:17.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f388e362640 1 -- 172.21.15.78:0/3118254597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388819f7a0 con 0x7f3888103cc0 2024-09-11T00:57:17.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.210+0000 7f3885ffb640 1 -- 172.21.15.78:0/3118254597 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3878010ac0 con 0x7f3888103cc0 2024-09-11T00:57:17.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.211+0000 7f3885ffb640 1 -- 172.21.15.78:0/3118254597 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878017c30 con 0x7f3888103cc0 2024-09-11T00:57:17.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.211+0000 7f388e362640 1 -- 172.21.15.78:0/3118254597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38881a5ee0 con 0x7f3888103cc0 2024-09-11T00:57:17.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.211+0000 7f388e362640 1 -- 172.21.15.78:0/3118254597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3888104120 con 0x7f3888103cc0 2024-09-11T00:57:17.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.213+0000 7f3885ffb640 1 -- 172.21.15.78:0/3118254597 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f3878010610 con 0x7f3888103cc0 2024-09-11T00:57:17.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.214+0000 7f3885ffb640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f38740b53c0 0x7f38740b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:17.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.214+0000 7f387ffff640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f38740b53c0 0x7f38740b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:17.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.214+0000 7f3885ffb640 1 -- 172.21.15.78:0/3118254597 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8405+0+0 (secure 0 0 0) 0x7f38780e57d0 con 0x7f3888103cc0 2024-09-11T00:57:17.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.214+0000 7f387ffff640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f38740b53c0 0x7f38740b7880 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f388810f880 tx=0x7f3870009060 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:17.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.216+0000 7f3885ffb640 1 -- 172.21.15.78:0/3118254597 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f38780ac480 con 0x7f3888103cc0 2024-09-11T00:57:17.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.396+0000 7f388e362640 1 -- 172.21.15.78:0/3118254597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f388819a530 con 0x7f3888103cc0 2024-09-11T00:57:17.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.397+0000 7f3885ffb640 1 -- 172.21.15.78:0/3118254597 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22975 (secure 0 0 0) 0x7f38780abd80 con 0x7f3888103cc0 2024-09-11T00:57:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:57:17.400 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":76,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","created":"2024-09-11T00:48:27.652512+0000","modified":"2024-09-11T00:57:16.564890+0000","last_up_change":"2024-09-11T00:55:59.071507+0000","last_in_change":"2024-09-11T00:55:35.815150+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:52:10.229887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:56:49.939136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:56:51.703446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beefc6d9-f74c-4200-96f2-ff697ad0d7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6803","nonce":1424639053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6805","nonce":1424639053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6809","nonce":1424639053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6807","nonce":1424639053}]},"public_addr":"172.21.15.78:6803/1424639053","cluster_addr":"172.21.15.78:6805/1424639053","heartbeat_back_addr":"172.21.15.78:6809/1424639053","heartbeat_front_addr":"172.21.15.78:6807/1424639053","state":["exists","up"]},{"osd":1,"uuid":"d742e767-d673-4bb9-b624-8bc969f5ceb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6801","nonce":580939385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6803","nonce":580939385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6807","nonce":580939385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6805","nonce":580939385}]},"public_addr":"172.21.15.110:6801/580939385","cluster_addr":"172.21.15.110:6803/580939385","heartbeat_back_addr":"172.21.15.110:6807/580939385","heartbeat_front_addr":"172.21.15.110:6805/580939385","state":["exists","up"]},{"osd":2,"uuid":"b0106a5b-1465-4a50-96dd-934dbae49a4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6801","nonce":2983155763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6803","nonce":2983155763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6807","nonce":2983155763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6805","nonce":2983155763}]},"public_addr":"172.21.15.112:6801/2983155763","cluster_addr":"172.21.15.112:6803/2983155763","heartbeat_back_addr":"172.21.15.112:6807/2983155763","heartbeat_front_addr":"172.21.15.112:6805/2983155763","state":["exists","up"]},{"osd":3,"uuid":"3270362b-3d61-414a-a3d9-f2416ff8b870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6811","nonce":2906086464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6813","nonce":2906086464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6817","nonce":2906086464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6815","nonce":2906086464}]},"public_addr":"172.21.15.78:6811/2906086464","cluster_addr":"172.21.15.78:6813/2906086464","heartbeat_back_addr":"172.21.15.78:6817/2906086464","heartbeat_front_addr":"172.21.15.78:6815/2906086464","state":["exists","up"]},{"osd":4,"uuid":"e47746ec-9443-4004-abfa-17c1ed73b982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6809","nonce":477287435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6811","nonce":477287435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6815","nonce":477287435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6813","nonce":477287435}]},"public_addr":"172.21.15.110:6809/477287435","cluster_addr":"172.21.15.110:6811/477287435","heartbeat_back_addr":"172.21.15.110:6815/477287435","heartbeat_front_addr":"172.21.15.110:6813/477287435","state":["exists","up"]},{"osd":5,"uuid":"f8cedb01-310d-442d-a588-33197d32e208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6809","nonce":3557095096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6811","nonce":3557095096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6815","nonce":3557095096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6813","nonce":3557095096}]},"public_addr":"172.21.15.112:6809/3557095096","cluster_addr":"172.21.15.112:6811/3557095096","heartbeat_back_addr":"172.21.15.112:6815/3557095096","heartbeat_front_addr":"172.21.15.112:6813/3557095096","state":["exists","up"]},{"osd":6,"uuid":"d508a037-8f56-436b-b7bf-9a1c69f6d9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6819","nonce":1267910775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6821","nonce":1267910775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6825","nonce":1267910775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6823","nonce":1267910775}]},"public_addr":"172.21.15.78:6819/1267910775","cluster_addr":"172.21.15.78:6821/1267910775","heartbeat_back_addr":"172.21.15.78:6825/1267910775","heartbeat_front_addr":"172.21.15.78:6823/1267910775","state":["exists","up"]},{"osd":7,"uuid":"e8842047-09a6-48b8-8d0a-2d54e7f89063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6817","nonce":1556494594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6819","nonce":1556494594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6823","nonce":1556494594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6821","nonce":1556494594}]},"public_addr":"172.21.15.110:6817/1556494594","cluster_addr":"172.21.15.110:6819/1556494594","heartbeat_back_addr":"172.21.15.110:6823/1556494594","heartbeat_front_addr":"172.21.15.110:6821/1556494594","state":["exists","up"]},{"osd":8,"uuid":"a3367da8-375a-4a46-a952-867d4b72ae2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6817","nonce":2481321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6819","nonce":2481321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6823","nonce":2481321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6821","nonce":2481321}]},"public_addr":"172.21.15.112:6817/2481321","cluster_addr":"172.21.15.112:6819/2481321","heartbeat_back_addr":"172.21.15.112:6823/2481321","heartbeat_front_addr":"172.21.15.112:6821/2481321","state":["exists","up"]},{"osd":9,"uuid":"d1825764-cefc-495b-99da-8ba429df0937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6827","nonce":197048350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6829","nonce":197048350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6833","nonce":197048350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6831","nonce":197048350}]},"public_addr":"172.21.15.78:6827/197048350","cluster_addr":"172.21.15.78:6829/197048350","heartbeat_back_addr":"172.21.15.78:6833/197048350","heartbeat_front_addr":"172.21.15.78:6831/197048350","state":["exists","up"]},{"osd":10,"uuid":"e1cfff7a-8916-4845-a482-d79fa275da30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6825","nonce":2101123168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6827","nonce":2101123168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6831","nonce":2101123168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6829","nonce":2101123168}]},"public_addr":"172.21.15.110:6825/2101123168","cluster_addr":"172.21.15.110:6827/2101123168","heartbeat_back_addr":"172.21.15.110:6831/2101123168","heartbeat_front_addr":"172.21.15.110:6829/2101123168","state":["exists","up"]},{"osd":11,"uuid":"fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6825","nonce":1526310654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6827","nonce":1526310654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6831","nonce":1526310654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6829","nonce":1526310654}]},"public_addr":"172.21.15.112:6825/1526310654","cluster_addr":"172.21.15.112:6827/1526310654","heartbeat_back_addr":"172.21.15.112:6831/1526310654","heartbeat_front_addr":"172.21.15.112:6829/1526310654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:17.334199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:41.400872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:08.205144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:33.550527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:58.394116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:24.699607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:50.450193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:16.223363+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:40.947727+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:06.845451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:32.410035+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:57.744364+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/2390785703":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:0/1038951931":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/391908936":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2706514734":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/2357025243":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6800/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2795778473":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2747741101":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6801/1289343426":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/3558896007":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/438261739":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/1289343426":"2024-09-12T00:50:15.928079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:57:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.400+0000 7f388e362640 1 -- 172.21.15.78:0/3118254597 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f38740b53c0 msgr2=0x7f38740b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.400+0000 7f388e362640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f38740b53c0 0x7f38740b7880 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f388810f880 tx=0x7f3870009060 comp rx=0 tx=0).stop 2024-09-11T00:57:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.400+0000 7f388e362640 1 -- 172.21.15.78:0/3118254597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3888103cc0 msgr2=0x7f3888199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.400+0000 7f388e362640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3888103cc0 0x7f3888199400 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f387800b4a0 tx=0x7f387800b960 comp rx=0 tx=0).stop 2024-09-11T00:57:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.400+0000 7f388e362640 1 -- 172.21.15.78:0/3118254597 shutdown_connections 2024-09-11T00:57:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.400+0000 7f388e362640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f38740b53c0 0x7f38740b7880 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.400+0000 7f388e362640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f388810d120 0x7f388819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.400+0000 7f388e362640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38881045e0 0x7f3888199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.400+0000 7f388e362640 1 --2- 172.21.15.78:0/3118254597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3888103cc0 0x7f3888199400 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.400+0000 7f388e362640 1 -- 172.21.15.78:0/3118254597 >> 172.21.15.78:0/3118254597 conn(0x7f38880fd700 msgr2=0x7f38880fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.400+0000 7f388e362640 1 -- 172.21.15.78:0/3118254597 shutdown_connections 2024-09-11T00:57:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:17.400+0000 7f388e362640 1 -- 172.21.15.78:0/3118254597 wait complete. 2024-09-11T00:57:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-11T00:57:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-11T00:57:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-11T00:57:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-11T00:57:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-11T00:57:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]': finished 2024-09-11T00:57:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:17 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-11T00:57:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:17 smithi078 ceph-mon[31315]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T00:57:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:17 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3118254597' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:57:17.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-11T00:57:17.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-11T00:57:17.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-11T00:57:17.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-11T00:57:17.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-11T00:57:17.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]': finished 2024-09-11T00:57:17.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:17 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-11T00:57:17.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:17 smithi110 ceph-mon[34712]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T00:57:17.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:17 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3118254597' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:57:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-11T00:57:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-11T00:57:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-11T00:57:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-11T00:57:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-11T00:57:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]': finished 2024-09-11T00:57:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:17 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-11T00:57:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:17 smithi112 ceph-mon[35384]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T00:57:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:17 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3118254597' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:57:18.109 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T00:57:18.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:57:18.359 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:18 smithi078 ceph-mon[31315]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T00:57:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:18 smithi078 ceph-mon[31315]: pgmap v276: 97 pgs: 97 active+clean; 625 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-09-11T00:57:18.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.825+0000 7faae8435640 1 -- 172.21.15.78:0/424167146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faae0103b30 msgr2=0x7faae0103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:18.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.825+0000 7faae8435640 1 --2- 172.21.15.78:0/424167146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faae0103b30 0x7faae0103f10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faadc009900 tx=0x7faadc02a410 comp rx=0 tx=0).stop 2024-09-11T00:57:18.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.825+0000 7faae8435640 1 -- 172.21.15.78:0/424167146 shutdown_connections 2024-09-11T00:57:18.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.825+0000 7faae8435640 1 --2- 172.21.15.78:0/424167146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae010d0e0 0x7faae010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:18.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.825+0000 7faae8435640 1 --2- 172.21.15.78:0/424167146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faae0104530 0x7faae010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:18.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.825+0000 7faae8435640 1 --2- 172.21.15.78:0/424167146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faae0103b30 0x7faae0103f10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:18.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.825+0000 7faae8435640 1 -- 172.21.15.78:0/424167146 >> 172.21.15.78:0/424167146 conn(0x7faae00fd700 msgr2=0x7faae00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:18.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.825+0000 7faae8435640 1 -- 172.21.15.78:0/424167146 shutdown_connections 2024-09-11T00:57:18.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.825+0000 7faae8435640 1 -- 172.21.15.78:0/424167146 wait complete. 2024-09-11T00:57:18.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.825+0000 7faae8435640 1 Processor -- start 2024-09-11T00:57:18.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.825+0000 7faae8435640 1 -- start start 2024-09-11T00:57:18.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.825+0000 7faae8435640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faae0103b30 0x7faae01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:18.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae8435640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faae0104530 0x7faae01997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:18.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae8435640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae010d0e0 0x7faae019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:18.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae8435640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faae0112d60 con 0x7faae0103b30 2024-09-11T00:57:18.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae8435640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faae0112be0 con 0x7faae0104530 2024-09-11T00:57:18.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae8435640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faae0112ee0 con 0x7faae010d0e0 2024-09-11T00:57:18.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae61d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faae0103b30 0x7faae01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:18.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae61d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faae0103b30 0x7faae01992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40134/0 (socket says 172.21.15.78:40134) 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae61d1640 1 -- 172.21.15.78:0/991760514 learned_addr learned my addr 172.21.15.78:0/991760514 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae69d2640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae010d0e0 0x7faae019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae59d0640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faae0104530 0x7faae01997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae61d1640 1 -- 172.21.15.78:0/991760514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae010d0e0 msgr2=0x7faae019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae61d1640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae010d0e0 0x7faae019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae61d1640 1 -- 172.21.15.78:0/991760514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faae0104530 msgr2=0x7faae01997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae61d1640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faae0104530 0x7faae01997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae61d1640 1 -- 172.21.15.78:0/991760514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae019f360 con 0x7faae0103b30 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae69d2640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae010d0e0 0x7faae019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae59d0640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faae0104530 0x7faae01997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae61d1640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faae0103b30 0x7faae01992b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faadc005940 tx=0x7faadc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faad77fe640 1 -- 172.21.15.78:0/991760514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc03f030 con 0x7faae0103b30 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faad77fe640 1 -- 172.21.15.78:0/991760514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faadc003cb0 con 0x7faae0103b30 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae8435640 1 -- 172.21.15.78:0/991760514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae019f5f0 con 0x7faae0103b30 2024-09-11T00:57:18.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faad77fe640 1 -- 172.21.15.78:0/991760514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc035070 con 0x7faae0103b30 2024-09-11T00:57:18.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.826+0000 7faae8435640 1 -- 172.21.15.78:0/991760514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae01a5d10 con 0x7faae0103b30 2024-09-11T00:57:18.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.827+0000 7faae8435640 1 -- 172.21.15.78:0/991760514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae0103f10 con 0x7faae0103b30 2024-09-11T00:57:18.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.829+0000 7faad77fe640 1 -- 172.21.15.78:0/991760514 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7faadc03bbf0 con 0x7faae0103b30 2024-09-11T00:57:18.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.829+0000 7faad77fe640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faacc0b5440 0x7faacc0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:18.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.830+0000 7faae59d0640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faacc0b5440 0x7faacc0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:18.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.830+0000 7faad77fe640 1 -- 172.21.15.78:0/991760514 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7faadc106cf0 con 0x7faae0103b30 2024-09-11T00:57:18.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.830+0000 7faae59d0640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faacc0b5440 0x7faacc0b7900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faae010f840 tx=0x7faad0005d30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:18.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:18.832+0000 7faad77fe640 1 -- 172.21.15.78:0/991760514 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7faadc0cd9a0 con 0x7faae0103b30 2024-09-11T00:57:18.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:18 smithi110 ceph-mon[34712]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T00:57:18.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:18 smithi110 ceph-mon[34712]: pgmap v276: 97 pgs: 97 active+clean; 625 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-09-11T00:57:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:18 smithi112 ceph-mon[35384]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T00:57:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:18 smithi112 ceph-mon[35384]: pgmap v276: 97 pgs: 97 active+clean; 625 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-09-11T00:57:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.006+0000 7faae8435640 1 -- 172.21.15.78:0/991760514 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faae0063590 con 0x7faacc0b5440 2024-09-11T00:57:19.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.013+0000 7faad77fe640 1 -- 172.21.15.78:0/991760514 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258895 (secure 0 0 0) 0x7faae0063590 con 0x7faacc0b5440 2024-09-11T00:57:19.018 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:57:19.019 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T00:57:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.016+0000 7faae8435640 1 -- 172.21.15.78:0/991760514 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faacc0b5440 msgr2=0x7faacc0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.016+0000 7faae8435640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faacc0b5440 0x7faacc0b7900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faae010f840 tx=0x7faad0005d30 comp rx=0 tx=0).stop 2024-09-11T00:57:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.016+0000 7faae8435640 1 -- 172.21.15.78:0/991760514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faae0103b30 msgr2=0x7faae01992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.016+0000 7faae8435640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faae0103b30 0x7faae01992b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faadc005940 tx=0x7faadc0042f0 comp rx=0 tx=0).stop 2024-09-11T00:57:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.016+0000 7faae8435640 1 -- 172.21.15.78:0/991760514 shutdown_connections 2024-09-11T00:57:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.016+0000 7faae8435640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faacc0b5440 0x7faacc0b7900 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.016+0000 7faae8435640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae010d0e0 0x7faae019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.016+0000 7faae8435640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faae0104530 0x7faae01997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.016+0000 7faae8435640 1 --2- 172.21.15.78:0/991760514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faae0103b30 0x7faae01992b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.016+0000 7faae8435640 1 -- 172.21.15.78:0/991760514 >> 172.21.15.78:0/991760514 conn(0x7faae00fd700 msgr2=0x7faae00fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.016+0000 7faae8435640 1 -- 172.21.15.78:0/991760514 shutdown_connections 2024-09-11T00:57:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:19.016+0000 7faae8435640 1 -- 172.21.15.78:0/991760514 wait complete. 2024-09-11T00:57:19.800 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-09-11T00:57:18.025515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639616,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":234,"num_read_kb":199,"num_write":353,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":292,"ondisk_log_size":292,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371856,"kb_used_data":10128,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569968,"statfs":{"total":1151940427776,"available":1151559647232,"internally_reserved":0,"allocated":10371072,"data_stored":6693024,"data_compressed":109425,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20379,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":67,"num_read_kb":46,"num_write":15,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001852"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123827+0000","last_change":"2024-09-11T00:56:53.123622+0000","last_active":"2024-09-11T00:56:54.123827+0000","last_peered":"2024-09-11T00:56:54.123827+0000","last_clean":"2024-09-11T00:56:54.123827+0000","last_became_active":"2024-09-11T00:56:53.121460+0000","last_became_peered":"2024-09-11T00:56:53.121460+0000","last_unstale":"2024-09-11T00:56:54.123827+0000","last_undegraded":"2024-09-11T00:56:54.123827+0000","last_fullsized":"2024-09-11T00:56:54.123827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:30.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122552+0000","last_change":"2024-09-11T00:56:53.120189+0000","last_active":"2024-09-11T00:56:54.122552+0000","last_peered":"2024-09-11T00:56:54.122552+0000","last_clean":"2024-09-11T00:56:54.122552+0000","last_became_active":"2024-09-11T00:56:53.119853+0000","last_became_peered":"2024-09-11T00:56:53.119853+0000","last_unstale":"2024-09-11T00:56:54.122552+0000","last_undegraded":"2024-09-11T00:56:54.122552+0000","last_fullsized":"2024-09-11T00:56:54.122552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:36.927113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123054+0000","last_change":"2024-09-11T00:56:53.117832+0000","last_active":"2024-09-11T00:56:54.123054+0000","last_peered":"2024-09-11T00:56:54.123054+0000","last_clean":"2024-09-11T00:56:54.123054+0000","last_became_active":"2024-09-11T00:56:53.117508+0000","last_became_peered":"2024-09-11T00:56:53.117508+0000","last_unstale":"2024-09-11T00:56:54.123054+0000","last_undegraded":"2024-09-11T00:56:54.123054+0000","last_fullsized":"2024-09-11T00:56:54.123054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:00.628673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110735+0000","last_change":"2024-09-11T00:56:53.128302+0000","last_active":"2024-09-11T00:56:54.110735+0000","last_peered":"2024-09-11T00:56:54.110735+0000","last_clean":"2024-09-11T00:56:54.110735+0000","last_became_active":"2024-09-11T00:56:53.127584+0000","last_became_peered":"2024-09-11T00:56:53.127584+0000","last_unstale":"2024-09-11T00:56:54.110735+0000","last_undegraded":"2024-09-11T00:56:54.110735+0000","last_fullsized":"2024-09-11T00:56:54.110735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:41.963656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.111122+0000","last_change":"2024-09-11T00:56:53.116249+0000","last_active":"2024-09-11T00:56:54.111122+0000","last_peered":"2024-09-11T00:56:54.111122+0000","last_clean":"2024-09-11T00:56:54.111122+0000","last_became_active":"2024-09-11T00:56:53.115881+0000","last_became_peered":"2024-09-11T00:56:53.115881+0000","last_unstale":"2024-09-11T00:56:54.111122+0000","last_undegraded":"2024-09-11T00:56:54.111122+0000","last_fullsized":"2024-09-11T00:56:54.111122+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:47.432607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.121744+0000","last_change":"2024-09-11T00:56:53.116992+0000","last_active":"2024-09-11T00:56:54.121744+0000","last_peered":"2024-09-11T00:56:54.121744+0000","last_clean":"2024-09-11T00:56:54.121744+0000","last_became_active":"2024-09-11T00:56:53.116629+0000","last_became_peered":"2024-09-11T00:56:53.116629+0000","last_unstale":"2024-09-11T00:56:54.121744+0000","last_undegraded":"2024-09-11T00:56:54.121744+0000","last_fullsized":"2024-09-11T00:56:54.121744+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:25.410377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.124414+0000","last_change":"2024-09-11T00:56:53.127873+0000","last_active":"2024-09-11T00:56:54.124414+0000","last_peered":"2024-09-11T00:56:54.124414+0000","last_clean":"2024-09-11T00:56:54.124414+0000","last_became_active":"2024-09-11T00:56:53.127575+0000","last_became_peered":"2024-09-11T00:56:53.127575+0000","last_unstale":"2024-09-11T00:56:54.124414+0000","last_undegraded":"2024-09-11T00:56:54.124414+0000","last_fullsized":"2024-09-11T00:56:54.124414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:12.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.109690+0000","last_change":"2024-09-11T00:56:53.117727+0000","last_active":"2024-09-11T00:56:54.109690+0000","last_peered":"2024-09-11T00:56:54.109690+0000","last_clean":"2024-09-11T00:56:54.109690+0000","last_became_active":"2024-09-11T00:56:53.117444+0000","last_became_peered":"2024-09-11T00:56:53.117444+0000","last_unstale":"2024-09-11T00:56:54.109690+0000","last_undegraded":"2024-09-11T00:56:54.109690+0000","last_fullsized":"2024-09-11T00:56:54.109690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:28.625231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.126289+0000","last_change":"2024-09-11T00:56:53.128974+0000","last_active":"2024-09-11T00:56:54.126289+0000","last_peered":"2024-09-11T00:56:54.126289+0000","last_clean":"2024-09-11T00:56:54.126289+0000","last_became_active":"2024-09-11T00:56:53.128650+0000","last_became_peered":"2024-09-11T00:56:53.128650+0000","last_unstale":"2024-09-11T00:56:54.126289+0000","last_undegraded":"2024-09-11T00:56:54.126289+0000","last_fullsized":"2024-09-11T00:56:54.126289+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:07.251373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122432+0000","last_change":"2024-09-11T00:56:53.117816+0000","last_active":"2024-09-11T00:56:54.122432+0000","last_peered":"2024-09-11T00:56:54.122432+0000","last_clean":"2024-09-11T00:56:54.122432+0000","last_became_active":"2024-09-11T00:56:53.117449+0000","last_became_peered":"2024-09-11T00:56:53.117449+0000","last_unstale":"2024-09-11T00:56:54.122432+0000","last_undegraded":"2024-09-11T00:56:54.122432+0000","last_fullsized":"2024-09-11T00:56:54.122432+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:09.058869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.124756+0000","last_change":"2024-09-11T00:56:53.124616+0000","last_active":"2024-09-11T00:56:54.124756+0000","last_peered":"2024-09-11T00:56:54.124756+0000","last_clean":"2024-09-11T00:56:54.124756+0000","last_became_active":"2024-09-11T00:56:53.124309+0000","last_became_peered":"2024-09-11T00:56:53.124309+0000","last_unstale":"2024-09-11T00:56:54.124756+0000","last_undegraded":"2024-09-11T00:56:54.124756+0000","last_fullsized":"2024-09-11T00:56:54.124756+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:14.469909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110739+0000","last_change":"2024-09-11T00:56:53.126263+0000","last_active":"2024-09-11T00:56:54.110739+0000","last_peered":"2024-09-11T00:56:54.110739+0000","last_clean":"2024-09-11T00:56:54.110739+0000","last_became_active":"2024-09-11T00:56:53.125923+0000","last_became_peered":"2024-09-11T00:56:53.125923+0000","last_unstale":"2024-09-11T00:56:54.110739+0000","last_undegraded":"2024-09-11T00:56:54.110739+0000","last_fullsized":"2024-09-11T00:56:54.110739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:12:33.627372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122565+0000","last_change":"2024-09-11T00:56:53.105292+0000","last_active":"2024-09-11T00:56:54.122565+0000","last_peered":"2024-09-11T00:56:54.122565+0000","last_clean":"2024-09-11T00:56:54.122565+0000","last_became_active":"2024-09-11T00:56:53.104830+0000","last_became_peered":"2024-09-11T00:56:53.104830+0000","last_unstale":"2024-09-11T00:56:54.122565+0000","last_undegraded":"2024-09-11T00:56:54.122565+0000","last_fullsized":"2024-09-11T00:56:54.122565+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:14.136952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122346+0000","last_change":"2024-09-11T00:56:53.120463+0000","last_active":"2024-09-11T00:56:54.122346+0000","last_peered":"2024-09-11T00:56:54.122346+0000","last_clean":"2024-09-11T00:56:54.122346+0000","last_became_active":"2024-09-11T00:56:53.120167+0000","last_became_peered":"2024-09-11T00:56:53.120167+0000","last_unstale":"2024-09-11T00:56:54.122346+0000","last_undegraded":"2024-09-11T00:56:54.122346+0000","last_fullsized":"2024-09-11T00:56:54.122346+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:49.783452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122647+0000","last_change":"2024-09-11T00:56:53.123312+0000","last_active":"2024-09-11T00:56:54.122647+0000","last_peered":"2024-09-11T00:56:54.122647+0000","last_clean":"2024-09-11T00:56:54.122647+0000","last_became_active":"2024-09-11T00:56:53.123015+0000","last_became_peered":"2024-09-11T00:56:53.123015+0000","last_unstale":"2024-09-11T00:56:54.122647+0000","last_undegraded":"2024-09-11T00:56:54.122647+0000","last_fullsized":"2024-09-11T00:56:54.122647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:22.069629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.125983+0000","last_change":"2024-09-11T00:56:53.125701+0000","last_active":"2024-09-11T00:56:54.125983+0000","last_peered":"2024-09-11T00:56:54.125983+0000","last_clean":"2024-09-11T00:56:54.125983+0000","last_became_active":"2024-09-11T00:56:53.125362+0000","last_became_peered":"2024-09-11T00:56:53.125362+0000","last_unstale":"2024-09-11T00:56:54.125983+0000","last_undegraded":"2024-09-11T00:56:54.125983+0000","last_fullsized":"2024-09-11T00:56:54.125983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:14.594324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123785+0000","last_change":"2024-09-11T00:56:53.124275+0000","last_active":"2024-09-11T00:56:54.123785+0000","last_peered":"2024-09-11T00:56:54.123785+0000","last_clean":"2024-09-11T00:56:54.123785+0000","last_became_active":"2024-09-11T00:56:53.122695+0000","last_became_peered":"2024-09-11T00:56:53.122695+0000","last_unstale":"2024-09-11T00:56:54.123785+0000","last_undegraded":"2024-09-11T00:56:54.123785+0000","last_fullsized":"2024-09-11T00:56:54.123785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:23.955266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.120998+0000","last_change":"2024-09-11T00:56:53.119077+0000","last_active":"2024-09-11T00:56:54.120998+0000","last_peered":"2024-09-11T00:56:54.120998+0000","last_clean":"2024-09-11T00:56:54.120998+0000","last_became_active":"2024-09-11T00:56:53.118693+0000","last_became_peered":"2024-09-11T00:56:53.118693+0000","last_unstale":"2024-09-11T00:56:54.120998+0000","last_undegraded":"2024-09-11T00:56:54.120998+0000","last_fullsized":"2024-09-11T00:56:54.120998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:39.135391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.124616+0000","last_change":"2024-09-11T00:56:53.115430+0000","last_active":"2024-09-11T00:56:54.124616+0000","last_peered":"2024-09-11T00:56:54.124616+0000","last_clean":"2024-09-11T00:56:54.124616+0000","last_became_active":"2024-09-11T00:56:53.115111+0000","last_became_peered":"2024-09-11T00:56:53.115111+0000","last_unstale":"2024-09-11T00:56:54.124616+0000","last_undegraded":"2024-09-11T00:56:54.124616+0000","last_fullsized":"2024-09-11T00:56:54.124616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:11.306661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123180+0000","last_change":"2024-09-11T00:56:53.124827+0000","last_active":"2024-09-11T00:56:54.123180+0000","last_peered":"2024-09-11T00:56:54.123180+0000","last_clean":"2024-09-11T00:56:54.123180+0000","last_became_active":"2024-09-11T00:56:53.123726+0000","last_became_peered":"2024-09-11T00:56:53.123726+0000","last_unstale":"2024-09-11T00:56:54.123180+0000","last_undegraded":"2024-09-11T00:56:54.123180+0000","last_fullsized":"2024-09-11T00:56:54.123180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:38.073354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122753+0000","last_change":"2024-09-11T00:56:53.121303+0000","last_active":"2024-09-11T00:56:54.122753+0000","last_peered":"2024-09-11T00:56:54.122753+0000","last_clean":"2024-09-11T00:56:54.122753+0000","last_became_active":"2024-09-11T00:56:53.121004+0000","last_became_peered":"2024-09-11T00:56:53.121004+0000","last_unstale":"2024-09-11T00:56:54.122753+0000","last_undegraded":"2024-09-11T00:56:54.122753+0000","last_fullsized":"2024-09-11T00:56:54.122753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:36.899870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.126504+0000","last_change":"2024-09-11T00:56:53.125042+0000","last_active":"2024-09-11T00:56:54.126504+0000","last_peered":"2024-09-11T00:56:54.126504+0000","last_clean":"2024-09-11T00:56:54.126504+0000","last_became_active":"2024-09-11T00:56:53.124661+0000","last_became_peered":"2024-09-11T00:56:53.124661+0000","last_unstale":"2024-09-11T00:56:54.126504+0000","last_undegraded":"2024-09-11T00:56:54.126504+0000","last_fullsized":"2024-09-11T00:56:54.126504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:34.392172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.109697+0000","last_change":"2024-09-11T00:56:53.116544+0000","last_active":"2024-09-11T00:56:54.109697+0000","last_peered":"2024-09-11T00:56:54.109697+0000","last_clean":"2024-09-11T00:56:54.109697+0000","last_became_active":"2024-09-11T00:56:53.116262+0000","last_became_peered":"2024-09-11T00:56:53.116262+0000","last_unstale":"2024-09-11T00:56:54.109697+0000","last_undegraded":"2024-09-11T00:56:54.109697+0000","last_fullsized":"2024-09-11T00:56:54.109697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:34.500669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.126456+0000","last_change":"2024-09-11T00:56:53.116967+0000","last_active":"2024-09-11T00:56:54.126456+0000","last_peered":"2024-09-11T00:56:54.126456+0000","last_clean":"2024-09-11T00:56:54.126456+0000","last_became_active":"2024-09-11T00:56:53.116405+0000","last_became_peered":"2024-09-11T00:56:53.116405+0000","last_unstale":"2024-09-11T00:56:54.126456+0000","last_undegraded":"2024-09-11T00:56:54.126456+0000","last_fullsized":"2024-09-11T00:56:54.126456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:33.521964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110122+0000","last_change":"2024-09-11T00:56:53.116727+0000","last_active":"2024-09-11T00:56:54.110122+0000","last_peered":"2024-09-11T00:56:54.110122+0000","last_clean":"2024-09-11T00:56:54.110122+0000","last_became_active":"2024-09-11T00:56:53.116416+0000","last_became_peered":"2024-09-11T00:56:53.116416+0000","last_unstale":"2024-09-11T00:56:54.110122+0000","last_undegraded":"2024-09-11T00:56:54.110122+0000","last_fullsized":"2024-09-11T00:56:54.110122+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:28.277909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.124869+0000","last_change":"2024-09-11T00:56:53.123141+0000","last_active":"2024-09-11T00:56:54.124869+0000","last_peered":"2024-09-11T00:56:54.124869+0000","last_clean":"2024-09-11T00:56:54.124869+0000","last_became_active":"2024-09-11T00:56:53.122811+0000","last_became_peered":"2024-09-11T00:56:53.122811+0000","last_unstale":"2024-09-11T00:56:54.124869+0000","last_undegraded":"2024-09-11T00:56:54.124869+0000","last_fullsized":"2024-09-11T00:56:54.124869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:47.175120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123872+0000","last_change":"2024-09-11T00:56:53.124213+0000","last_active":"2024-09-11T00:56:54.123872+0000","last_peered":"2024-09-11T00:56:54.123872+0000","last_clean":"2024-09-11T00:56:54.123872+0000","last_became_active":"2024-09-11T00:56:53.123757+0000","last_became_peered":"2024-09-11T00:56:53.123757+0000","last_unstale":"2024-09-11T00:56:54.123872+0000","last_undegraded":"2024-09-11T00:56:54.123872+0000","last_fullsized":"2024-09-11T00:56:54.123872+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:54.263476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123062+0000","last_change":"2024-09-11T00:56:53.114016+0000","last_active":"2024-09-11T00:56:54.123062+0000","last_peered":"2024-09-11T00:56:54.123062+0000","last_clean":"2024-09-11T00:56:54.123062+0000","last_became_active":"2024-09-11T00:56:53.113719+0000","last_became_peered":"2024-09-11T00:56:53.113719+0000","last_unstale":"2024-09-11T00:56:54.123062+0000","last_undegraded":"2024-09-11T00:56:54.123062+0000","last_fullsized":"2024-09-11T00:56:54.123062+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:48:06.623810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122977+0000","last_change":"2024-09-11T00:56:53.124798+0000","last_active":"2024-09-11T00:56:54.122977+0000","last_peered":"2024-09-11T00:56:54.122977+0000","last_clean":"2024-09-11T00:56:54.122977+0000","last_became_active":"2024-09-11T00:56:53.124381+0000","last_became_peered":"2024-09-11T00:56:53.124381+0000","last_unstale":"2024-09-11T00:56:54.122977+0000","last_undegraded":"2024-09-11T00:56:54.122977+0000","last_fullsized":"2024-09-11T00:56:54.122977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:37.823112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122041+0000","last_change":"2024-09-11T00:56:53.115297+0000","last_active":"2024-09-11T00:56:54.122041+0000","last_peered":"2024-09-11T00:56:54.122041+0000","last_clean":"2024-09-11T00:56:54.122041+0000","last_became_active":"2024-09-11T00:56:53.114958+0000","last_became_peered":"2024-09-11T00:56:53.114958+0000","last_unstale":"2024-09-11T00:56:54.122041+0000","last_undegraded":"2024-09-11T00:56:54.122041+0000","last_fullsized":"2024-09-11T00:56:54.122041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:47.612610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.127915+0000","last_change":"2024-09-11T00:56:53.126717+0000","last_active":"2024-09-11T00:56:54.127915+0000","last_peered":"2024-09-11T00:56:54.127915+0000","last_clean":"2024-09-11T00:56:54.127915+0000","last_became_active":"2024-09-11T00:56:53.126387+0000","last_became_peered":"2024-09-11T00:56:53.126387+0000","last_unstale":"2024-09-11T00:56:54.127915+0000","last_undegraded":"2024-09-11T00:56:54.127915+0000","last_fullsized":"2024-09-11T00:56:54.127915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:52.429127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123170+0000","last_change":"2024-09-11T00:56:53.126257+0000","last_active":"2024-09-11T00:56:54.123170+0000","last_peered":"2024-09-11T00:56:54.123170+0000","last_clean":"2024-09-11T00:56:54.123170+0000","last_became_active":"2024-09-11T00:56:53.125146+0000","last_became_peered":"2024-09-11T00:56:53.125146+0000","last_unstale":"2024-09-11T00:56:54.123170+0000","last_undegraded":"2024-09-11T00:56:54.123170+0000","last_fullsized":"2024-09-11T00:56:54.123170+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110712+0000","last_change":"2024-09-11T00:56:53.119187+0000","last_active":"2024-09-11T00:56:54.110712+0000","last_peered":"2024-09-11T00:56:54.110712+0000","last_clean":"2024-09-11T00:56:54.110712+0000","last_became_active":"2024-09-11T00:56:53.118850+0000","last_became_peered":"2024-09-11T00:56:53.118850+0000","last_unstale":"2024-09-11T00:56:54.110712+0000","last_undegraded":"2024-09-11T00:56:54.110712+0000","last_fullsized":"2024-09-11T00:56:54.110712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:05.403876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:14.250920+0000","last_change":"2024-09-11T00:56:51.107140+0000","last_active":"2024-09-11T00:57:14.250920+0000","last_peered":"2024-09-11T00:57:14.250920+0000","last_clean":"2024-09-11T00:57:14.250920+0000","last_became_active":"2024-09-11T00:56:51.106748+0000","last_became_peered":"2024-09-11T00:56:51.106748+0000","last_unstale":"2024-09-11T00:57:14.250920+0000","last_undegraded":"2024-09-11T00:57:14.250920+0000","last_fullsized":"2024-09-11T00:57:14.250920+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:16.228197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.126288+0000","last_change":"2024-09-11T00:56:53.118049+0000","last_active":"2024-09-11T00:56:54.126288+0000","last_peered":"2024-09-11T00:56:54.126288+0000","last_clean":"2024-09-11T00:56:54.126288+0000","last_became_active":"2024-09-11T00:56:53.117685+0000","last_became_peered":"2024-09-11T00:56:53.117685+0000","last_unstale":"2024-09-11T00:56:54.126288+0000","last_undegraded":"2024-09-11T00:56:54.126288+0000","last_fullsized":"2024-09-11T00:56:54.126288+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:17.017594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.192341+0000","last_change":"2024-09-11T00:56:51.101690+0000","last_active":"2024-09-11T00:57:03.192341+0000","last_peered":"2024-09-11T00:57:03.192341+0000","last_clean":"2024-09-11T00:57:03.192341+0000","last_became_active":"2024-09-11T00:56:51.101055+0000","last_became_peered":"2024-09-11T00:56:51.101055+0000","last_unstale":"2024-09-11T00:57:03.192341+0000","last_undegraded":"2024-09-11T00:57:03.192341+0000","last_fullsized":"2024-09-11T00:57:03.192341+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:28.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:02.180369+0000","last_change":"2024-09-11T00:56:51.101700+0000","last_active":"2024-09-11T00:57:02.180369+0000","last_peered":"2024-09-11T00:57:02.180369+0000","last_clean":"2024-09-11T00:57:02.180369+0000","last_became_active":"2024-09-11T00:56:51.101044+0000","last_became_peered":"2024-09-11T00:56:51.101044+0000","last_unstale":"2024-09-11T00:57:02.180369+0000","last_undegraded":"2024-09-11T00:57:02.180369+0000","last_fullsized":"2024-09-11T00:57:02.180369+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:12.121126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.109707+0000","last_change":"2024-09-11T00:56:53.121652+0000","last_active":"2024-09-11T00:56:54.109707+0000","last_peered":"2024-09-11T00:56:54.109707+0000","last_clean":"2024-09-11T00:56:54.109707+0000","last_became_active":"2024-09-11T00:56:53.121375+0000","last_became_peered":"2024-09-11T00:56:53.121375+0000","last_unstale":"2024-09-11T00:56:54.109707+0000","last_undegraded":"2024-09-11T00:56:54.109707+0000","last_fullsized":"2024-09-11T00:56:54.109707+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:39.154985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.232842+0000","last_change":"2024-09-11T00:56:51.104367+0000","last_active":"2024-09-11T00:57:03.232842+0000","last_peered":"2024-09-11T00:57:03.232842+0000","last_clean":"2024-09-11T00:57:03.232842+0000","last_became_active":"2024-09-11T00:56:51.104045+0000","last_became_peered":"2024-09-11T00:56:51.104045+0000","last_unstale":"2024-09-11T00:57:03.232842+0000","last_undegraded":"2024-09-11T00:57:03.232842+0000","last_fullsized":"2024-09-11T00:57:03.232842+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:03.989437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.121035+0000","last_change":"2024-09-11T00:56:53.116167+0000","last_active":"2024-09-11T00:56:54.121035+0000","last_peered":"2024-09-11T00:56:54.121035+0000","last_clean":"2024-09-11T00:56:54.121035+0000","last_became_active":"2024-09-11T00:56:53.115857+0000","last_became_peered":"2024-09-11T00:56:53.115857+0000","last_unstale":"2024-09-11T00:56:54.121035+0000","last_undegraded":"2024-09-11T00:56:54.121035+0000","last_fullsized":"2024-09-11T00:56:54.121035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:04.476094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:14.250274+0000","last_change":"2024-09-11T00:56:51.103287+0000","last_active":"2024-09-11T00:57:14.250274+0000","last_peered":"2024-09-11T00:57:14.250274+0000","last_clean":"2024-09-11T00:57:14.250274+0000","last_became_active":"2024-09-11T00:56:51.102953+0000","last_became_peered":"2024-09-11T00:56:51.102953+0000","last_unstale":"2024-09-11T00:57:14.250274+0000","last_undegraded":"2024-09-11T00:57:14.250274+0000","last_fullsized":"2024-09-11T00:57:14.250274+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:49.701612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.126633+0000","last_change":"2024-09-11T00:56:53.125130+0000","last_active":"2024-09-11T00:56:54.126633+0000","last_peered":"2024-09-11T00:56:54.126633+0000","last_clean":"2024-09-11T00:56:54.126633+0000","last_became_active":"2024-09-11T00:56:53.124442+0000","last_became_peered":"2024-09-11T00:56:53.124442+0000","last_unstale":"2024-09-11T00:56:54.126633+0000","last_undegraded":"2024-09-11T00:56:54.126633+0000","last_fullsized":"2024-09-11T00:56:54.126633+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:24.439430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.238000+0000","last_change":"2024-09-11T00:56:51.104802+0000","last_active":"2024-09-11T00:57:03.238000+0000","last_peered":"2024-09-11T00:57:03.238000+0000","last_clean":"2024-09-11T00:57:03.238000+0000","last_became_active":"2024-09-11T00:56:51.104335+0000","last_became_peered":"2024-09-11T00:56:51.104335+0000","last_unstale":"2024-09-11T00:57:03.238000+0000","last_undegraded":"2024-09-11T00:57:03.238000+0000","last_fullsized":"2024-09-11T00:57:03.238000+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:26.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123298+0000","last_change":"2024-09-11T00:56:53.124846+0000","last_active":"2024-09-11T00:56:54.123298+0000","last_peered":"2024-09-11T00:56:54.123298+0000","last_clean":"2024-09-11T00:56:54.123298+0000","last_became_active":"2024-09-11T00:56:53.124369+0000","last_became_peered":"2024-09-11T00:56:53.124369+0000","last_unstale":"2024-09-11T00:56:54.123298+0000","last_undegraded":"2024-09-11T00:56:54.123298+0000","last_fullsized":"2024-09-11T00:56:54.123298+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:49.012164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.191917+0000","last_change":"2024-09-11T00:56:51.154544+0000","last_active":"2024-09-11T00:57:03.191917+0000","last_peered":"2024-09-11T00:57:03.191917+0000","last_clean":"2024-09-11T00:57:03.191917+0000","last_became_active":"2024-09-11T00:56:51.154195+0000","last_became_peered":"2024-09-11T00:56:51.154195+0000","last_unstale":"2024-09-11T00:57:03.191917+0000","last_undegraded":"2024-09-11T00:57:03.191917+0000","last_fullsized":"2024-09-11T00:57:03.191917+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:57:24.530947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.126016+0000","last_change":"2024-09-11T00:56:53.127728+0000","last_active":"2024-09-11T00:56:54.126016+0000","last_peered":"2024-09-11T00:56:54.126016+0000","last_clean":"2024-09-11T00:56:54.126016+0000","last_became_active":"2024-09-11T00:56:53.127423+0000","last_became_peered":"2024-09-11T00:56:53.127423+0000","last_unstale":"2024-09-11T00:56:54.126016+0000","last_undegraded":"2024-09-11T00:56:54.126016+0000","last_fullsized":"2024-09-11T00:56:54.126016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:55.717123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'24","reported_seq":72,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:14.254951+0000","last_change":"2024-09-11T00:56:51.100774+0000","last_active":"2024-09-11T00:57:14.254951+0000","last_peered":"2024-09-11T00:57:14.254951+0000","last_clean":"2024-09-11T00:57:14.254951+0000","last_became_active":"2024-09-11T00:56:51.100398+0000","last_became_peered":"2024-09-11T00:56:51.100398+0000","last_unstale":"2024-09-11T00:57:14.254951+0000","last_undegraded":"2024-09-11T00:57:14.254951+0000","last_fullsized":"2024-09-11T00:57:14.254951+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:51.975236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42201,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":55,"num_write":26,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.111212+0000","last_change":"2024-09-11T00:56:53.120113+0000","last_active":"2024-09-11T00:56:54.111212+0000","last_peered":"2024-09-11T00:56:54.111212+0000","last_clean":"2024-09-11T00:56:54.111212+0000","last_became_active":"2024-09-11T00:56:53.119733+0000","last_became_peered":"2024-09-11T00:56:53.119733+0000","last_unstale":"2024-09-11T00:56:54.111212+0000","last_undegraded":"2024-09-11T00:56:54.111212+0000","last_fullsized":"2024-09-11T00:56:54.111212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:59.541710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.239133+0000","last_change":"2024-09-11T00:56:51.104234+0000","last_active":"2024-09-11T00:57:03.239133+0000","last_peered":"2024-09-11T00:57:03.239133+0000","last_clean":"2024-09-11T00:57:03.239133+0000","last_became_active":"2024-09-11T00:56:51.103631+0000","last_became_peered":"2024-09-11T00:56:51.103631+0000","last_unstale":"2024-09-11T00:57:03.239133+0000","last_undegraded":"2024-09-11T00:57:03.239133+0000","last_fullsized":"2024-09-11T00:57:03.239133+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:47.378135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.121985+0000","last_change":"2024-09-11T00:56:53.122231+0000","last_active":"2024-09-11T00:56:54.121985+0000","last_peered":"2024-09-11T00:56:54.121985+0000","last_clean":"2024-09-11T00:56:54.121985+0000","last_became_active":"2024-09-11T00:56:53.121903+0000","last_became_peered":"2024-09-11T00:56:53.121903+0000","last_unstale":"2024-09-11T00:56:54.121985+0000","last_undegraded":"2024-09-11T00:56:54.121985+0000","last_fullsized":"2024-09-11T00:56:54.121985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:44.246919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.237744+0000","last_change":"2024-09-11T00:56:51.105625+0000","last_active":"2024-09-11T00:57:03.237744+0000","last_peered":"2024-09-11T00:57:03.237744+0000","last_clean":"2024-09-11T00:57:03.237744+0000","last_became_active":"2024-09-11T00:56:51.105291+0000","last_became_peered":"2024-09-11T00:56:51.105291+0000","last_unstale":"2024-09-11T00:57:03.237744+0000","last_undegraded":"2024-09-11T00:57:03.237744+0000","last_fullsized":"2024-09-11T00:57:03.237744+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:24.237174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.125238+0000","last_change":"2024-09-11T00:56:53.126272+0000","last_active":"2024-09-11T00:56:54.125238+0000","last_peered":"2024-09-11T00:56:54.125238+0000","last_clean":"2024-09-11T00:56:54.125238+0000","last_became_active":"2024-09-11T00:56:53.125964+0000","last_became_peered":"2024-09-11T00:56:53.125964+0000","last_unstale":"2024-09-11T00:56:54.125238+0000","last_undegraded":"2024-09-11T00:56:54.125238+0000","last_fullsized":"2024-09-11T00:56:54.125238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:36.331220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:14.251287+0000","last_change":"2024-09-11T00:56:51.101053+0000","last_active":"2024-09-11T00:57:14.251287+0000","last_peered":"2024-09-11T00:57:14.251287+0000","last_clean":"2024-09-11T00:57:14.251287+0000","last_became_active":"2024-09-11T00:56:51.100714+0000","last_became_peered":"2024-09-11T00:56:51.100714+0000","last_unstale":"2024-09-11T00:57:14.251287+0000","last_undegraded":"2024-09-11T00:57:14.251287+0000","last_fullsized":"2024-09-11T00:57:14.251287+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:57.709126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.124292+0000","last_change":"2024-09-11T00:56:53.119973+0000","last_active":"2024-09-11T00:56:54.124292+0000","last_peered":"2024-09-11T00:56:54.124292+0000","last_clean":"2024-09-11T00:56:54.124292+0000","last_became_active":"2024-09-11T00:56:53.118972+0000","last_became_peered":"2024-09-11T00:56:53.118972+0000","last_unstale":"2024-09-11T00:56:54.124292+0000","last_undegraded":"2024-09-11T00:56:54.124292+0000","last_fullsized":"2024-09-11T00:56:54.124292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:35.912920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.228815+0000","last_change":"2024-09-11T00:56:51.153453+0000","last_active":"2024-09-11T00:57:03.228815+0000","last_peered":"2024-09-11T00:57:03.228815+0000","last_clean":"2024-09-11T00:57:03.228815+0000","last_became_active":"2024-09-11T00:56:51.153112+0000","last_became_peered":"2024-09-11T00:56:51.153112+0000","last_unstale":"2024-09-11T00:57:03.228815+0000","last_undegraded":"2024-09-11T00:57:03.228815+0000","last_fullsized":"2024-09-11T00:57:03.228815+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:41:32.001080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.125791+0000","last_change":"2024-09-11T00:56:53.120972+0000","last_active":"2024-09-11T00:56:54.125791+0000","last_peered":"2024-09-11T00:56:54.125791+0000","last_clean":"2024-09-11T00:56:54.125791+0000","last_became_active":"2024-09-11T00:56:53.120568+0000","last_became_peered":"2024-09-11T00:56:53.120568+0000","last_unstale":"2024-09-11T00:56:54.125791+0000","last_undegraded":"2024-09-11T00:56:54.125791+0000","last_fullsized":"2024-09-11T00:56:54.125791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:19.696201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.229283+0000","last_change":"2024-09-11T00:56:51.103756+0000","last_active":"2024-09-11T00:57:03.229283+0000","last_peered":"2024-09-11T00:57:03.229283+0000","last_clean":"2024-09-11T00:57:03.229283+0000","last_became_active":"2024-09-11T00:56:51.103406+0000","last_became_peered":"2024-09-11T00:56:51.103406+0000","last_unstale":"2024-09-11T00:57:03.229283+0000","last_undegraded":"2024-09-11T00:57:03.229283+0000","last_fullsized":"2024-09-11T00:57:03.229283+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:28.022346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123457+0000","last_change":"2024-09-11T00:56:53.128628+0000","last_active":"2024-09-11T00:56:54.123457+0000","last_peered":"2024-09-11T00:56:54.123457+0000","last_clean":"2024-09-11T00:56:54.123457+0000","last_became_active":"2024-09-11T00:56:53.127197+0000","last_became_peered":"2024-09-11T00:56:53.127197+0000","last_unstale":"2024-09-11T00:56:54.123457+0000","last_undegraded":"2024-09-11T00:56:54.123457+0000","last_fullsized":"2024-09-11T00:56:54.123457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:38.753676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.236116+0000","last_change":"2024-09-11T00:56:51.101398+0000","last_active":"2024-09-11T00:57:03.236116+0000","last_peered":"2024-09-11T00:57:03.236116+0000","last_clean":"2024-09-11T00:57:03.236116+0000","last_became_active":"2024-09-11T00:56:51.101034+0000","last_became_peered":"2024-09-11T00:56:51.101034+0000","last_unstale":"2024-09-11T00:57:03.236116+0000","last_undegraded":"2024-09-11T00:57:03.236116+0000","last_fullsized":"2024-09-11T00:57:03.236116+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:40.165368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.125819+0000","last_change":"2024-09-11T00:56:53.127047+0000","last_active":"2024-09-11T00:56:54.125819+0000","last_peered":"2024-09-11T00:56:54.125819+0000","last_clean":"2024-09-11T00:56:54.125819+0000","last_became_active":"2024-09-11T00:56:53.126719+0000","last_became_peered":"2024-09-11T00:56:53.126719+0000","last_unstale":"2024-09-11T00:56:54.125819+0000","last_undegraded":"2024-09-11T00:56:54.125819+0000","last_fullsized":"2024-09-11T00:56:54.125819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.686604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.228693+0000","last_change":"2024-09-11T00:56:51.101665+0000","last_active":"2024-09-11T00:57:03.228693+0000","last_peered":"2024-09-11T00:57:03.228693+0000","last_clean":"2024-09-11T00:57:03.228693+0000","last_became_active":"2024-09-11T00:56:51.101332+0000","last_became_peered":"2024-09-11T00:56:51.101332+0000","last_unstale":"2024-09-11T00:57:03.228693+0000","last_undegraded":"2024-09-11T00:57:03.228693+0000","last_fullsized":"2024-09-11T00:57:03.228693+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:47.639877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.124859+0000","last_change":"2024-09-11T00:56:53.128736+0000","last_active":"2024-09-11T00:56:54.124859+0000","last_peered":"2024-09-11T00:56:54.124859+0000","last_clean":"2024-09-11T00:56:54.124859+0000","last_became_active":"2024-09-11T00:56:53.128417+0000","last_became_peered":"2024-09-11T00:56:53.128417+0000","last_unstale":"2024-09-11T00:56:54.124859+0000","last_undegraded":"2024-09-11T00:56:54.124859+0000","last_fullsized":"2024-09-11T00:56:54.124859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:52.331343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:14.250940+0000","last_change":"2024-09-11T00:56:51.154733+0000","last_active":"2024-09-11T00:57:14.250940+0000","last_peered":"2024-09-11T00:57:14.250940+0000","last_clean":"2024-09-11T00:57:14.250940+0000","last_became_active":"2024-09-11T00:56:51.153613+0000","last_became_peered":"2024-09-11T00:56:51.153613+0000","last_unstale":"2024-09-11T00:57:14.250940+0000","last_undegraded":"2024-09-11T00:57:14.250940+0000","last_fullsized":"2024-09-11T00:57:14.250940+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:24.749186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.124793+0000","last_change":"2024-09-11T00:56:53.124684+0000","last_active":"2024-09-11T00:56:54.124793+0000","last_peered":"2024-09-11T00:56:54.124793+0000","last_clean":"2024-09-11T00:56:54.124793+0000","last_became_active":"2024-09-11T00:56:53.124147+0000","last_became_peered":"2024-09-11T00:56:53.124147+0000","last_unstale":"2024-09-11T00:56:54.124793+0000","last_undegraded":"2024-09-11T00:56:54.124793+0000","last_fullsized":"2024-09-11T00:56:54.124793+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:14.280465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":92,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.124980+0000","last_change":"2024-09-11T00:55:34.500373+0000","last_active":"2024-09-11T00:56:54.124980+0000","last_peered":"2024-09-11T00:56:54.124980+0000","last_clean":"2024-09-11T00:56:54.124980+0000","last_became_active":"2024-09-11T00:55:34.495540+0000","last_became_peered":"2024-09-11T00:55:34.495540+0000","last_unstale":"2024-09-11T00:56:54.124980+0000","last_undegraded":"2024-09-11T00:56:54.124980+0000","last_fullsized":"2024-09-11T00:56:54.124980+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_clean_scrub_stamp":"2024-09-11T00:52:11.168966+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:37.681466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.237524+0000","last_change":"2024-09-11T00:56:51.101524+0000","last_active":"2024-09-11T00:57:03.237524+0000","last_peered":"2024-09-11T00:57:03.237524+0000","last_clean":"2024-09-11T00:57:03.237524+0000","last_became_active":"2024-09-11T00:56:51.101180+0000","last_became_peered":"2024-09-11T00:56:51.101180+0000","last_unstale":"2024-09-11T00:57:03.237524+0000","last_undegraded":"2024-09-11T00:57:03.237524+0000","last_fullsized":"2024-09-11T00:57:03.237524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:27.293494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.109947+0000","last_change":"2024-09-11T00:56:53.118503+0000","last_active":"2024-09-11T00:56:54.109947+0000","last_peered":"2024-09-11T00:56:54.109947+0000","last_clean":"2024-09-11T00:56:54.109947+0000","last_became_active":"2024-09-11T00:56:53.118174+0000","last_became_peered":"2024-09-11T00:56:53.118174+0000","last_unstale":"2024-09-11T00:56:54.109947+0000","last_undegraded":"2024-09-11T00:56:54.109947+0000","last_fullsized":"2024-09-11T00:56:54.109947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:13.524153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:02.181617+0000","last_change":"2024-09-11T00:56:51.104222+0000","last_active":"2024-09-11T00:57:02.181617+0000","last_peered":"2024-09-11T00:57:02.181617+0000","last_clean":"2024-09-11T00:57:02.181617+0000","last_became_active":"2024-09-11T00:56:51.103913+0000","last_became_peered":"2024-09-11T00:56:51.103913+0000","last_unstale":"2024-09-11T00:57:02.181617+0000","last_undegraded":"2024-09-11T00:57:02.181617+0000","last_fullsized":"2024-09-11T00:57:02.181617+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:57.467659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.121882+0000","last_change":"2024-09-11T00:56:53.115130+0000","last_active":"2024-09-11T00:56:54.121882+0000","last_peered":"2024-09-11T00:56:54.121882+0000","last_clean":"2024-09-11T00:56:54.121882+0000","last_became_active":"2024-09-11T00:56:53.114813+0000","last_became_peered":"2024-09-11T00:56:53.114813+0000","last_unstale":"2024-09-11T00:56:54.121882+0000","last_undegraded":"2024-09-11T00:56:54.121882+0000","last_fullsized":"2024-09-11T00:56:54.121882+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:13.265173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:11.245990+0000","last_change":"2024-09-11T00:56:51.152892+0000","last_active":"2024-09-11T00:57:11.245990+0000","last_peered":"2024-09-11T00:57:11.245990+0000","last_clean":"2024-09-11T00:57:11.245990+0000","last_became_active":"2024-09-11T00:56:51.152572+0000","last_became_peered":"2024-09-11T00:56:51.152572+0000","last_unstale":"2024-09-11T00:57:11.245990+0000","last_undegraded":"2024-09-11T00:57:11.245990+0000","last_fullsized":"2024-09-11T00:57:11.245990+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:03.774593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.126674+0000","last_change":"2024-09-11T00:56:53.117416+0000","last_active":"2024-09-11T00:56:54.126674+0000","last_peered":"2024-09-11T00:56:54.126674+0000","last_clean":"2024-09-11T00:56:54.126674+0000","last_became_active":"2024-09-11T00:56:53.117016+0000","last_became_peered":"2024-09-11T00:56:53.117016+0000","last_unstale":"2024-09-11T00:56:54.126674+0000","last_undegraded":"2024-09-11T00:56:54.126674+0000","last_fullsized":"2024-09-11T00:56:54.126674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:07.607993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:12.254258+0000","last_change":"2024-09-11T00:56:51.102879+0000","last_active":"2024-09-11T00:57:12.254258+0000","last_peered":"2024-09-11T00:57:12.254258+0000","last_clean":"2024-09-11T00:57:12.254258+0000","last_became_active":"2024-09-11T00:56:51.102498+0000","last_became_peered":"2024-09-11T00:56:51.102498+0000","last_unstale":"2024-09-11T00:57:12.254258+0000","last_undegraded":"2024-09-11T00:57:12.254258+0000","last_fullsized":"2024-09-11T00:57:12.254258+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:41.360908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.121897+0000","last_change":"2024-09-11T00:56:53.119804+0000","last_active":"2024-09-11T00:56:54.121897+0000","last_peered":"2024-09-11T00:56:54.121897+0000","last_clean":"2024-09-11T00:56:54.121897+0000","last_became_active":"2024-09-11T00:56:53.119473+0000","last_became_peered":"2024-09-11T00:56:53.119473+0000","last_unstale":"2024-09-11T00:56:54.121897+0000","last_undegraded":"2024-09-11T00:56:54.121897+0000","last_fullsized":"2024-09-11T00:56:54.121897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:35.021789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.120870+0000","last_change":"2024-09-11T00:56:51.154604+0000","last_active":"2024-09-11T00:56:54.120870+0000","last_peered":"2024-09-11T00:56:54.120870+0000","last_clean":"2024-09-11T00:56:54.120870+0000","last_became_active":"2024-09-11T00:56:51.154189+0000","last_became_peered":"2024-09-11T00:56:51.154189+0000","last_unstale":"2024-09-11T00:56:54.120870+0000","last_undegraded":"2024-09-11T00:56:54.120870+0000","last_fullsized":"2024-09-11T00:56:54.120870+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:51.655321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.480056+0000","last_change":"2024-09-11T00:56:53.112550+0000","last_active":"2024-09-11T00:56:54.480056+0000","last_peered":"2024-09-11T00:56:54.480056+0000","last_clean":"2024-09-11T00:56:54.480056+0000","last_became_active":"2024-09-11T00:56:53.112179+0000","last_became_peered":"2024-09-11T00:56:53.112179+0000","last_unstale":"2024-09-11T00:56:54.480056+0000","last_undegraded":"2024-09-11T00:56:54.480056+0000","last_fullsized":"2024-09-11T00:56:54.480056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:03.339839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.229730+0000","last_change":"2024-09-11T00:56:51.152957+0000","last_active":"2024-09-11T00:57:03.229730+0000","last_peered":"2024-09-11T00:57:03.229730+0000","last_clean":"2024-09-11T00:57:03.229730+0000","last_became_active":"2024-09-11T00:56:51.152616+0000","last_became_peered":"2024-09-11T00:56:51.152616+0000","last_unstale":"2024-09-11T00:57:03.229730+0000","last_undegraded":"2024-09-11T00:57:03.229730+0000","last_fullsized":"2024-09-11T00:57:03.229730+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:02.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123265+0000","last_change":"2024-09-11T00:56:53.115155+0000","last_active":"2024-09-11T00:56:54.123265+0000","last_peered":"2024-09-11T00:56:54.123265+0000","last_clean":"2024-09-11T00:56:54.123265+0000","last_became_active":"2024-09-11T00:56:53.114865+0000","last_became_peered":"2024-09-11T00:56:53.114865+0000","last_unstale":"2024-09-11T00:56:54.123265+0000","last_undegraded":"2024-09-11T00:56:54.123265+0000","last_fullsized":"2024-09-11T00:56:54.123265+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:23.968888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'11","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:12.121212+0000","last_change":"2024-09-11T00:56:51.100887+0000","last_active":"2024-09-11T00:57:12.121212+0000","last_peered":"2024-09-11T00:57:12.121212+0000","last_clean":"2024-09-11T00:57:12.121212+0000","last_became_active":"2024-09-11T00:56:51.100474+0000","last_became_peered":"2024-09-11T00:56:51.100474+0000","last_unstale":"2024-09-11T00:57:12.121212+0000","last_undegraded":"2024-09-11T00:57:12.121212+0000","last_fullsized":"2024-09-11T00:57:12.121212+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:40.884871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110178+0000","last_change":"2024-09-11T00:56:53.122060+0000","last_active":"2024-09-11T00:56:54.110178+0000","last_peered":"2024-09-11T00:56:54.110178+0000","last_clean":"2024-09-11T00:56:54.110178+0000","last_became_active":"2024-09-11T00:56:53.121423+0000","last_became_peered":"2024-09-11T00:56:53.121423+0000","last_unstale":"2024-09-11T00:56:54.110178+0000","last_undegraded":"2024-09-11T00:56:54.110178+0000","last_fullsized":"2024-09-11T00:56:54.110178+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:24.356006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.237796+0000","last_change":"2024-09-11T00:56:51.102031+0000","last_active":"2024-09-11T00:57:03.237796+0000","last_peered":"2024-09-11T00:57:03.237796+0000","last_clean":"2024-09-11T00:57:03.237796+0000","last_became_active":"2024-09-11T00:56:51.100614+0000","last_became_peered":"2024-09-11T00:56:51.100614+0000","last_unstale":"2024-09-11T00:57:03.237796+0000","last_undegraded":"2024-09-11T00:57:03.237796+0000","last_fullsized":"2024-09-11T00:57:03.237796+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:52.611261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122728+0000","last_change":"2024-09-11T00:56:53.124806+0000","last_active":"2024-09-11T00:56:54.122728+0000","last_peered":"2024-09-11T00:56:54.122728+0000","last_clean":"2024-09-11T00:56:54.122728+0000","last_became_active":"2024-09-11T00:56:53.124452+0000","last_became_peered":"2024-09-11T00:56:53.124452+0000","last_unstale":"2024-09-11T00:56:54.122728+0000","last_undegraded":"2024-09-11T00:56:54.122728+0000","last_fullsized":"2024-09-11T00:56:54.122728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:57.934808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:01.167037+0000","last_change":"2024-09-11T00:56:51.153621+0000","last_active":"2024-09-11T00:57:01.167037+0000","last_peered":"2024-09-11T00:57:01.167037+0000","last_clean":"2024-09-11T00:57:01.167037+0000","last_became_active":"2024-09-11T00:56:51.153315+0000","last_became_peered":"2024-09-11T00:56:51.153315+0000","last_unstale":"2024-09-11T00:57:01.167037+0000","last_undegraded":"2024-09-11T00:57:01.167037+0000","last_fullsized":"2024-09-11T00:57:01.167037+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:10.058295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110771+0000","last_change":"2024-09-11T00:56:53.114463+0000","last_active":"2024-09-11T00:56:54.110771+0000","last_peered":"2024-09-11T00:56:54.110771+0000","last_clean":"2024-09-11T00:56:54.110771+0000","last_became_active":"2024-09-11T00:56:53.114099+0000","last_became_peered":"2024-09-11T00:56:53.114099+0000","last_unstale":"2024-09-11T00:56:54.110771+0000","last_undegraded":"2024-09-11T00:56:54.110771+0000","last_fullsized":"2024-09-11T00:56:54.110771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:58.981787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.235580+0000","last_change":"2024-09-11T00:56:51.096942+0000","last_active":"2024-09-11T00:57:03.235580+0000","last_peered":"2024-09-11T00:57:03.235580+0000","last_clean":"2024-09-11T00:57:03.235580+0000","last_became_active":"2024-09-11T00:56:51.096584+0000","last_became_peered":"2024-09-11T00:56:51.096584+0000","last_unstale":"2024-09-11T00:57:03.235580+0000","last_undegraded":"2024-09-11T00:57:03.235580+0000","last_fullsized":"2024-09-11T00:57:03.235580+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:03.484123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123153+0000","last_change":"2024-09-11T00:56:53.126804+0000","last_active":"2024-09-11T00:56:54.123153+0000","last_peered":"2024-09-11T00:56:54.123153+0000","last_clean":"2024-09-11T00:56:54.123153+0000","last_became_active":"2024-09-11T00:56:53.125408+0000","last_became_peered":"2024-09-11T00:56:53.125408+0000","last_unstale":"2024-09-11T00:56:54.123153+0000","last_undegraded":"2024-09-11T00:56:54.123153+0000","last_fullsized":"2024-09-11T00:56:54.123153+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:46.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.190572+0000","last_change":"2024-09-11T00:56:51.103755+0000","last_active":"2024-09-11T00:57:03.190572+0000","last_peered":"2024-09-11T00:57:03.190572+0000","last_clean":"2024-09-11T00:57:03.190572+0000","last_became_active":"2024-09-11T00:56:51.103395+0000","last_became_peered":"2024-09-11T00:56:51.103395+0000","last_unstale":"2024-09-11T00:57:03.190572+0000","last_undegraded":"2024-09-11T00:57:03.190572+0000","last_fullsized":"2024-09-11T00:57:03.190572+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:16.782660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110136+0000","last_change":"2024-09-11T00:56:53.121926+0000","last_active":"2024-09-11T00:56:54.110136+0000","last_peered":"2024-09-11T00:56:54.110136+0000","last_clean":"2024-09-11T00:56:54.110136+0000","last_became_active":"2024-09-11T00:56:53.121603+0000","last_became_peered":"2024-09-11T00:56:53.121603+0000","last_unstale":"2024-09-11T00:56:54.110136+0000","last_undegraded":"2024-09-11T00:56:54.110136+0000","last_fullsized":"2024-09-11T00:56:54.110136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:31.021030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:01.166384+0000","last_change":"2024-09-11T00:56:51.107221+0000","last_active":"2024-09-11T00:57:01.166384+0000","last_peered":"2024-09-11T00:57:01.166384+0000","last_clean":"2024-09-11T00:57:01.166384+0000","last_became_active":"2024-09-11T00:56:51.106849+0000","last_became_peered":"2024-09-11T00:56:51.106849+0000","last_unstale":"2024-09-11T00:57:01.166384+0000","last_undegraded":"2024-09-11T00:57:01.166384+0000","last_fullsized":"2024-09-11T00:57:01.166384+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:46.497230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.124482+0000","last_change":"2024-09-11T00:56:53.120525+0000","last_active":"2024-09-11T00:56:54.124482+0000","last_peered":"2024-09-11T00:56:54.124482+0000","last_clean":"2024-09-11T00:56:54.124482+0000","last_became_active":"2024-09-11T00:56:53.120228+0000","last_became_peered":"2024-09-11T00:56:53.120228+0000","last_unstale":"2024-09-11T00:56:54.124482+0000","last_undegraded":"2024-09-11T00:56:54.124482+0000","last_fullsized":"2024-09-11T00:56:54.124482+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:52.047687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.126929+0000","last_change":"2024-09-11T00:56:53.120311+0000","last_active":"2024-09-11T00:56:54.126929+0000","last_peered":"2024-09-11T00:56:54.126929+0000","last_clean":"2024-09-11T00:56:54.126929+0000","last_became_active":"2024-09-11T00:56:53.119648+0000","last_became_peered":"2024-09-11T00:56:53.119648+0000","last_unstale":"2024-09-11T00:56:54.126929+0000","last_undegraded":"2024-09-11T00:56:54.126929+0000","last_fullsized":"2024-09-11T00:56:54.126929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:21.187784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.238619+0000","last_change":"2024-09-11T00:56:51.103631+0000","last_active":"2024-09-11T00:57:03.238619+0000","last_peered":"2024-09-11T00:57:03.238619+0000","last_clean":"2024-09-11T00:57:03.238619+0000","last_became_active":"2024-09-11T00:56:51.103281+0000","last_became_peered":"2024-09-11T00:56:51.103281+0000","last_unstale":"2024-09-11T00:57:03.238619+0000","last_undegraded":"2024-09-11T00:57:03.238619+0000","last_fullsized":"2024-09-11T00:57:03.238619+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:16.910956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.125888+0000","last_change":"2024-09-11T00:56:53.124752+0000","last_active":"2024-09-11T00:56:54.125888+0000","last_peered":"2024-09-11T00:56:54.125888+0000","last_clean":"2024-09-11T00:56:54.125888+0000","last_became_active":"2024-09-11T00:56:53.124245+0000","last_became_peered":"2024-09-11T00:56:53.124245+0000","last_unstale":"2024-09-11T00:56:54.125888+0000","last_undegraded":"2024-09-11T00:56:54.125888+0000","last_fullsized":"2024-09-11T00:56:54.125888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:10.277903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.239515+0000","last_change":"2024-09-11T00:56:51.104624+0000","last_active":"2024-09-11T00:57:03.239515+0000","last_peered":"2024-09-11T00:57:03.239515+0000","last_clean":"2024-09-11T00:57:03.239515+0000","last_became_active":"2024-09-11T00:56:51.104041+0000","last_became_peered":"2024-09-11T00:56:51.104041+0000","last_unstale":"2024-09-11T00:57:03.239515+0000","last_undegraded":"2024-09-11T00:57:03.239515+0000","last_fullsized":"2024-09-11T00:57:03.239515+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:10.397455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123171+0000","last_change":"2024-09-11T00:56:53.118547+0000","last_active":"2024-09-11T00:56:54.123171+0000","last_peered":"2024-09-11T00:56:54.123171+0000","last_clean":"2024-09-11T00:56:54.123171+0000","last_became_active":"2024-09-11T00:56:53.118291+0000","last_became_peered":"2024-09-11T00:56:53.118291+0000","last_unstale":"2024-09-11T00:56:54.123171+0000","last_undegraded":"2024-09-11T00:56:54.123171+0000","last_fullsized":"2024-09-11T00:56:54.123171+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:42.012173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.125382+0000","last_change":"2024-09-11T00:56:51.153047+0000","last_active":"2024-09-11T00:56:54.125382+0000","last_peered":"2024-09-11T00:56:54.125382+0000","last_clean":"2024-09-11T00:56:54.125382+0000","last_became_active":"2024-09-11T00:56:51.152564+0000","last_became_peered":"2024-09-11T00:56:51.152564+0000","last_unstale":"2024-09-11T00:56:54.125382+0000","last_undegraded":"2024-09-11T00:56:54.125382+0000","last_fullsized":"2024-09-11T00:56:54.125382+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:16.849087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110141+0000","last_change":"2024-09-11T00:56:53.119926+0000","last_active":"2024-09-11T00:56:54.110141+0000","last_peered":"2024-09-11T00:56:54.110141+0000","last_clean":"2024-09-11T00:56:54.110141+0000","last_became_active":"2024-09-11T00:56:53.119643+0000","last_became_peered":"2024-09-11T00:56:53.119643+0000","last_unstale":"2024-09-11T00:56:54.110141+0000","last_undegraded":"2024-09-11T00:56:54.110141+0000","last_fullsized":"2024-09-11T00:56:54.110141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:51.838958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:12.253767+0000","last_change":"2024-09-11T00:56:51.100837+0000","last_active":"2024-09-11T00:57:12.253767+0000","last_peered":"2024-09-11T00:57:12.253767+0000","last_clean":"2024-09-11T00:57:12.253767+0000","last_became_active":"2024-09-11T00:56:51.100494+0000","last_became_peered":"2024-09-11T00:56:51.100494+0000","last_unstale":"2024-09-11T00:57:12.253767+0000","last_undegraded":"2024-09-11T00:57:12.253767+0000","last_fullsized":"2024-09-11T00:57:12.253767+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:39.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49014,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":153,"num_read_kb":130,"num_write":228,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":147042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":198,"ondisk_log_size":198,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286865,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743441,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":399040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1032270,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070492,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":398970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266720,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":399348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":991866,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626474,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":827392,"data_stored":398687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":991544,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":827392,"data_stored":398548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":398512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378688,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":442412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":441801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705738,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":400026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":43953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":43937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:57:19.805 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:57:19.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:19 smithi110 ceph-mon[34712]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:57:19.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:19 smithi112 ceph-mon[35384]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:57:19.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:19 smithi078 ceph-mon[31315]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:57:20.053 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:20.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.559+0000 7f78a8eec640 1 -- 172.21.15.78:0/3196867189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78a4108040 msgr2=0x7f78a410e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:20.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.559+0000 7f78a8eec640 1 --2- 172.21.15.78:0/3196867189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78a4108040 0x7f78a410e840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f789c009ca0 tx=0x7f789c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:20.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.559+0000 7f78a8eec640 1 -- 172.21.15.78:0/3196867189 shutdown_connections 2024-09-11T00:57:20.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.559+0000 7f78a8eec640 1 --2- 172.21.15.78:0/3196867189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78a4108040 0x7f78a410e840 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:20.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.559+0000 7f78a8eec640 1 --2- 172.21.15.78:0/3196867189 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41076c0 0x7f78a4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:20.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.559+0000 7f78a8eec640 1 --2- 172.21.15.78:0/3196867189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78a4101cb0 0x7f78a4102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:20.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.559+0000 7f78a8eec640 1 -- 172.21.15.78:0/3196867189 >> 172.21.15.78:0/3196867189 conn(0x7f78a40fd700 msgr2=0x7f78a40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:20.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.559+0000 7f78a8eec640 1 -- 172.21.15.78:0/3196867189 shutdown_connections 2024-09-11T00:57:20.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.559+0000 7f78a8eec640 1 -- 172.21.15.78:0/3196867189 wait complete. 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.560+0000 7f78a8eec640 1 Processor -- start 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.560+0000 7f78a8eec640 1 -- start start 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.560+0000 7f78a8eec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78a4101cb0 0x7f78a41992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.560+0000 7f78a8eec640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41076c0 0x7f78a4199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.560+0000 7f78a8eec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78a4108040 0x7f78a419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.560+0000 7f78a8eec640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4111d00 con 0x7f78a4108040 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.560+0000 7f78a8eec640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4111b80 con 0x7f78a41076c0 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.560+0000 7f78a8eec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4111e80 con 0x7f78a4101cb0 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.560+0000 7f78a2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78a4108040 0x7f78a419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.560+0000 7f78a2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78a4108040 0x7f78a419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40158/0 (socket says 172.21.15.78:40158) 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.560+0000 7f78a2d9d640 1 -- 172.21.15.78:0/1676327187 learned_addr learned my addr 172.21.15.78:0/1676327187 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f78a259c640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78a4101cb0 0x7f78a41992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f789bfff640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41076c0 0x7f78a4199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f78a2d9d640 1 -- 172.21.15.78:0/1676327187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78a4101cb0 msgr2=0x7f78a41992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f78a2d9d640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78a4101cb0 0x7f78a41992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f78a2d9d640 1 -- 172.21.15.78:0/1676327187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41076c0 msgr2=0x7f78a4199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f78a2d9d640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41076c0 0x7f78a4199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f78a2d9d640 1 -- 172.21.15.78:0/1676327187 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a419f390 con 0x7f78a4108040 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f78a2d9d640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78a4108040 0x7f78a419ebc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f789c005cc0 tx=0x7f789c02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f789b7fe640 1 -- 172.21.15.78:0/1676327187 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c003cf0 con 0x7f78a4108040 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f789b7fe640 1 -- 172.21.15.78:0/1676327187 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f789c0044f0 con 0x7f78a4108040 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f789b7fe640 1 -- 172.21.15.78:0/1676327187 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c02fd10 con 0x7f78a4108040 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f78a8eec640 1 -- 172.21.15.78:0/1676327187 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a419f620 con 0x7f78a4108040 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.561+0000 7f78a8eec640 1 -- 172.21.15.78:0/1676327187 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a41a5d40 con 0x7f78a4108040 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.562+0000 7f78a8eec640 1 -- 172.21.15.78:0/1676327187 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a4102090 con 0x7f78a4108040 2024-09-11T00:57:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.563+0000 7f789b7fe640 1 -- 172.21.15.78:0/1676327187 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f789c004040 con 0x7f78a4108040 2024-09-11T00:57:20.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.564+0000 7f789b7fe640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78900b53c0 0x7f78900b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:20.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.564+0000 7f78a259c640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78900b53c0 0x7f78900b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:20.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.564+0000 7f789b7fe640 1 -- 172.21.15.78:0/1676327187 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f789c107630 con 0x7f78a4108040 2024-09-11T00:57:20.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.565+0000 7f78a259c640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78900b53c0 0x7f78900b7880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f78940058c0 tx=0x7f789400a250 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:20.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.567+0000 7f789b7fe640 1 -- 172.21.15.78:0/1676327187 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f789c0ce2e0 con 0x7f78a4108040 2024-09-11T00:57:20.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.740+0000 7f78a8eec640 1 -- 172.21.15.78:0/1676327187 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78a410aea0 con 0x7f78900b53c0 2024-09-11T00:57:20.743 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:20 smithi078 ceph-mon[31315]: pgmap v277: 97 pgs: 97 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 13 op/s 2024-09-11T00:57:20.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.744+0000 7f789b7fe640 1 -- 172.21.15.78:0/1676327187 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258939 (secure 0 0 0) 0x7f789400fb60 con 0x7f78900b53c0 2024-09-11T00:57:20.750 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:57:20.751 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T00:57:20.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.747+0000 7f78a8eec640 1 -- 172.21.15.78:0/1676327187 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78900b53c0 msgr2=0x7f78900b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:20.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.747+0000 7f78a8eec640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78900b53c0 0x7f78900b7880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f78940058c0 tx=0x7f789400a250 comp rx=0 tx=0).stop 2024-09-11T00:57:20.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.747+0000 7f78a8eec640 1 -- 172.21.15.78:0/1676327187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78a4108040 msgr2=0x7f78a419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:20.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.747+0000 7f78a8eec640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78a4108040 0x7f78a419ebc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f789c005cc0 tx=0x7f789c02f470 comp rx=0 tx=0).stop 2024-09-11T00:57:20.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.747+0000 7f78a8eec640 1 -- 172.21.15.78:0/1676327187 shutdown_connections 2024-09-11T00:57:20.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.747+0000 7f78a8eec640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78900b53c0 0x7f78900b7880 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:20.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.747+0000 7f78a8eec640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78a4108040 0x7f78a419ebc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:20.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.747+0000 7f78a8eec640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a41076c0 0x7f78a4199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:20.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.747+0000 7f78a8eec640 1 --2- 172.21.15.78:0/1676327187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78a4101cb0 0x7f78a41992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:20.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.747+0000 7f78a8eec640 1 -- 172.21.15.78:0/1676327187 >> 172.21.15.78:0/1676327187 conn(0x7f78a40fd700 msgr2=0x7f78a406b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:20.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.747+0000 7f78a8eec640 1 -- 172.21.15.78:0/1676327187 shutdown_connections 2024-09-11T00:57:20.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:20.747+0000 7f78a8eec640 1 -- 172.21.15.78:0/1676327187 wait complete. 2024-09-11T00:57:20.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:20 smithi110 ceph-mon[34712]: pgmap v277: 97 pgs: 97 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 13 op/s 2024-09-11T00:57:20.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:20 smithi112 ceph-mon[35384]: pgmap v277: 97 pgs: 97 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 13 op/s 2024-09-11T00:57:21.617 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:21 smithi078 ceph-mon[31315]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:57:21.618 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-11T00:57:20.026013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652367,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":267,"num_read_kb":235,"num_write":364,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":294,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372040,"kb_used_data":10276,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569784,"statfs":{"total":1151940427776,"available":1151559458816,"internally_reserved":0,"allocated":10522624,"data_stored":6814535,"data_compressed":141481,"data_compressed_allocated":180224,"data_compressed_original":1859584,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32739,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":84,"num_read_kb":81,"num_write":25,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001885"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123827+0000","last_change":"2024-09-11T00:56:53.123622+0000","last_active":"2024-09-11T00:56:54.123827+0000","last_peered":"2024-09-11T00:56:54.123827+0000","last_clean":"2024-09-11T00:56:54.123827+0000","last_became_active":"2024-09-11T00:56:53.121460+0000","last_became_peered":"2024-09-11T00:56:53.121460+0000","last_unstale":"2024-09-11T00:56:54.123827+0000","last_undegraded":"2024-09-11T00:56:54.123827+0000","last_fullsized":"2024-09-11T00:56:54.123827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:30.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122552+0000","last_change":"2024-09-11T00:56:53.120189+0000","last_active":"2024-09-11T00:56:54.122552+0000","last_peered":"2024-09-11T00:56:54.122552+0000","last_clean":"2024-09-11T00:56:54.122552+0000","last_became_active":"2024-09-11T00:56:53.119853+0000","last_became_peered":"2024-09-11T00:56:53.119853+0000","last_unstale":"2024-09-11T00:56:54.122552+0000","last_undegraded":"2024-09-11T00:56:54.122552+0000","last_fullsized":"2024-09-11T00:56:54.122552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:36.927113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123054+0000","last_change":"2024-09-11T00:56:53.117832+0000","last_active":"2024-09-11T00:56:54.123054+0000","last_peered":"2024-09-11T00:56:54.123054+0000","last_clean":"2024-09-11T00:56:54.123054+0000","last_became_active":"2024-09-11T00:56:53.117508+0000","last_became_peered":"2024-09-11T00:56:53.117508+0000","last_unstale":"2024-09-11T00:56:54.123054+0000","last_undegraded":"2024-09-11T00:56:54.123054+0000","last_fullsized":"2024-09-11T00:56:54.123054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:00.628673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110735+0000","last_change":"2024-09-11T00:56:53.128302+0000","last_active":"2024-09-11T00:56:54.110735+0000","last_peered":"2024-09-11T00:56:54.110735+0000","last_clean":"2024-09-11T00:56:54.110735+0000","last_became_active":"2024-09-11T00:56:53.127584+0000","last_became_peered":"2024-09-11T00:56:53.127584+0000","last_unstale":"2024-09-11T00:56:54.110735+0000","last_undegraded":"2024-09-11T00:56:54.110735+0000","last_fullsized":"2024-09-11T00:56:54.110735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:41.963656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580317+0000","last_change":"2024-09-11T00:56:53.116249+0000","last_active":"2024-09-11T00:57:17.580317+0000","last_peered":"2024-09-11T00:57:17.580317+0000","last_clean":"2024-09-11T00:57:17.580317+0000","last_became_active":"2024-09-11T00:56:53.115881+0000","last_became_peered":"2024-09-11T00:56:53.115881+0000","last_unstale":"2024-09-11T00:57:17.580317+0000","last_undegraded":"2024-09-11T00:57:17.580317+0000","last_fullsized":"2024-09-11T00:57:17.580317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:47.432607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578616+0000","last_change":"2024-09-11T00:56:53.116992+0000","last_active":"2024-09-11T00:57:17.578616+0000","last_peered":"2024-09-11T00:57:17.578616+0000","last_clean":"2024-09-11T00:57:17.578616+0000","last_became_active":"2024-09-11T00:56:53.116629+0000","last_became_peered":"2024-09-11T00:56:53.116629+0000","last_unstale":"2024-09-11T00:57:17.578616+0000","last_undegraded":"2024-09-11T00:57:17.578616+0000","last_fullsized":"2024-09-11T00:57:17.578616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:25.410377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586216+0000","last_change":"2024-09-11T00:56:53.127873+0000","last_active":"2024-09-11T00:57:17.586216+0000","last_peered":"2024-09-11T00:57:17.586216+0000","last_clean":"2024-09-11T00:57:17.586216+0000","last_became_active":"2024-09-11T00:56:53.127575+0000","last_became_peered":"2024-09-11T00:56:53.127575+0000","last_unstale":"2024-09-11T00:57:17.586216+0000","last_undegraded":"2024-09-11T00:57:17.586216+0000","last_fullsized":"2024-09-11T00:57:17.586216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:12.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.109690+0000","last_change":"2024-09-11T00:56:53.117727+0000","last_active":"2024-09-11T00:56:54.109690+0000","last_peered":"2024-09-11T00:56:54.109690+0000","last_clean":"2024-09-11T00:56:54.109690+0000","last_became_active":"2024-09-11T00:56:53.117444+0000","last_became_peered":"2024-09-11T00:56:53.117444+0000","last_unstale":"2024-09-11T00:56:54.109690+0000","last_undegraded":"2024-09-11T00:56:54.109690+0000","last_fullsized":"2024-09-11T00:56:54.109690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:28.625231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160844+0000","last_change":"2024-09-11T00:56:53.128974+0000","last_active":"2024-09-11T00:57:18.160844+0000","last_peered":"2024-09-11T00:57:18.160844+0000","last_clean":"2024-09-11T00:57:18.160844+0000","last_became_active":"2024-09-11T00:56:53.128650+0000","last_became_peered":"2024-09-11T00:56:53.128650+0000","last_unstale":"2024-09-11T00:57:18.160844+0000","last_undegraded":"2024-09-11T00:57:18.160844+0000","last_fullsized":"2024-09-11T00:57:18.160844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:07.251373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.579078+0000","last_change":"2024-09-11T00:56:53.117816+0000","last_active":"2024-09-11T00:57:17.579078+0000","last_peered":"2024-09-11T00:57:17.579078+0000","last_clean":"2024-09-11T00:57:17.579078+0000","last_became_active":"2024-09-11T00:56:53.117449+0000","last_became_peered":"2024-09-11T00:56:53.117449+0000","last_unstale":"2024-09-11T00:57:17.579078+0000","last_undegraded":"2024-09-11T00:57:17.579078+0000","last_fullsized":"2024-09-11T00:57:17.579078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:09.058869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586597+0000","last_change":"2024-09-11T00:56:53.124616+0000","last_active":"2024-09-11T00:57:17.586597+0000","last_peered":"2024-09-11T00:57:17.586597+0000","last_clean":"2024-09-11T00:57:17.586597+0000","last_became_active":"2024-09-11T00:56:53.124309+0000","last_became_peered":"2024-09-11T00:56:53.124309+0000","last_unstale":"2024-09-11T00:57:17.586597+0000","last_undegraded":"2024-09-11T00:57:17.586597+0000","last_fullsized":"2024-09-11T00:57:17.586597+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:14.469909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110739+0000","last_change":"2024-09-11T00:56:53.126263+0000","last_active":"2024-09-11T00:56:54.110739+0000","last_peered":"2024-09-11T00:56:54.110739+0000","last_clean":"2024-09-11T00:56:54.110739+0000","last_became_active":"2024-09-11T00:56:53.125923+0000","last_became_peered":"2024-09-11T00:56:53.125923+0000","last_unstale":"2024-09-11T00:56:54.110739+0000","last_undegraded":"2024-09-11T00:56:54.110739+0000","last_fullsized":"2024-09-11T00:56:54.110739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:12:33.627372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576397+0000","last_change":"2024-09-11T00:56:53.105292+0000","last_active":"2024-09-11T00:57:17.576397+0000","last_peered":"2024-09-11T00:57:17.576397+0000","last_clean":"2024-09-11T00:57:17.576397+0000","last_became_active":"2024-09-11T00:56:53.104830+0000","last_became_peered":"2024-09-11T00:56:53.104830+0000","last_unstale":"2024-09-11T00:57:17.576397+0000","last_undegraded":"2024-09-11T00:57:17.576397+0000","last_fullsized":"2024-09-11T00:57:17.576397+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:14.136952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122346+0000","last_change":"2024-09-11T00:56:53.120463+0000","last_active":"2024-09-11T00:56:54.122346+0000","last_peered":"2024-09-11T00:56:54.122346+0000","last_clean":"2024-09-11T00:56:54.122346+0000","last_became_active":"2024-09-11T00:56:53.120167+0000","last_became_peered":"2024-09-11T00:56:53.120167+0000","last_unstale":"2024-09-11T00:56:54.122346+0000","last_undegraded":"2024-09-11T00:56:54.122346+0000","last_fullsized":"2024-09-11T00:56:54.122346+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:49.783452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122647+0000","last_change":"2024-09-11T00:56:53.123312+0000","last_active":"2024-09-11T00:56:54.122647+0000","last_peered":"2024-09-11T00:56:54.122647+0000","last_clean":"2024-09-11T00:56:54.122647+0000","last_became_active":"2024-09-11T00:56:53.123015+0000","last_became_peered":"2024-09-11T00:56:53.123015+0000","last_unstale":"2024-09-11T00:56:54.122647+0000","last_undegraded":"2024-09-11T00:56:54.122647+0000","last_fullsized":"2024-09-11T00:56:54.122647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:22.069629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160379+0000","last_change":"2024-09-11T00:56:53.125701+0000","last_active":"2024-09-11T00:57:18.160379+0000","last_peered":"2024-09-11T00:57:18.160379+0000","last_clean":"2024-09-11T00:57:18.160379+0000","last_became_active":"2024-09-11T00:56:53.125362+0000","last_became_peered":"2024-09-11T00:56:53.125362+0000","last_unstale":"2024-09-11T00:57:18.160379+0000","last_undegraded":"2024-09-11T00:57:18.160379+0000","last_fullsized":"2024-09-11T00:57:18.160379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:14.594324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123785+0000","last_change":"2024-09-11T00:56:53.124275+0000","last_active":"2024-09-11T00:56:54.123785+0000","last_peered":"2024-09-11T00:56:54.123785+0000","last_clean":"2024-09-11T00:56:54.123785+0000","last_became_active":"2024-09-11T00:56:53.122695+0000","last_became_peered":"2024-09-11T00:56:53.122695+0000","last_unstale":"2024-09-11T00:56:54.123785+0000","last_undegraded":"2024-09-11T00:56:54.123785+0000","last_fullsized":"2024-09-11T00:56:54.123785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:23.955266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580487+0000","last_change":"2024-09-11T00:56:53.119077+0000","last_active":"2024-09-11T00:57:17.580487+0000","last_peered":"2024-09-11T00:57:17.580487+0000","last_clean":"2024-09-11T00:57:17.580487+0000","last_became_active":"2024-09-11T00:56:53.118693+0000","last_became_peered":"2024-09-11T00:56:53.118693+0000","last_unstale":"2024-09-11T00:57:17.580487+0000","last_undegraded":"2024-09-11T00:57:17.580487+0000","last_fullsized":"2024-09-11T00:57:17.580487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:39.135391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.124616+0000","last_change":"2024-09-11T00:56:53.115430+0000","last_active":"2024-09-11T00:56:54.124616+0000","last_peered":"2024-09-11T00:56:54.124616+0000","last_clean":"2024-09-11T00:56:54.124616+0000","last_became_active":"2024-09-11T00:56:53.115111+0000","last_became_peered":"2024-09-11T00:56:53.115111+0000","last_unstale":"2024-09-11T00:56:54.124616+0000","last_undegraded":"2024-09-11T00:56:54.124616+0000","last_fullsized":"2024-09-11T00:56:54.124616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:11.306661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123180+0000","last_change":"2024-09-11T00:56:53.124827+0000","last_active":"2024-09-11T00:56:54.123180+0000","last_peered":"2024-09-11T00:56:54.123180+0000","last_clean":"2024-09-11T00:56:54.123180+0000","last_became_active":"2024-09-11T00:56:53.123726+0000","last_became_peered":"2024-09-11T00:56:53.123726+0000","last_unstale":"2024-09-11T00:56:54.123180+0000","last_undegraded":"2024-09-11T00:56:54.123180+0000","last_fullsized":"2024-09-11T00:56:54.123180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:38.073354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576404+0000","last_change":"2024-09-11T00:56:53.121303+0000","last_active":"2024-09-11T00:57:17.576404+0000","last_peered":"2024-09-11T00:57:17.576404+0000","last_clean":"2024-09-11T00:57:17.576404+0000","last_became_active":"2024-09-11T00:56:53.121004+0000","last_became_peered":"2024-09-11T00:56:53.121004+0000","last_unstale":"2024-09-11T00:57:17.576404+0000","last_undegraded":"2024-09-11T00:57:17.576404+0000","last_fullsized":"2024-09-11T00:57:17.576404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:36.899870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160496+0000","last_change":"2024-09-11T00:56:53.125042+0000","last_active":"2024-09-11T00:57:18.160496+0000","last_peered":"2024-09-11T00:57:18.160496+0000","last_clean":"2024-09-11T00:57:18.160496+0000","last_became_active":"2024-09-11T00:56:53.124661+0000","last_became_peered":"2024-09-11T00:56:53.124661+0000","last_unstale":"2024-09-11T00:57:18.160496+0000","last_undegraded":"2024-09-11T00:57:18.160496+0000","last_fullsized":"2024-09-11T00:57:18.160496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:34.392172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.109697+0000","last_change":"2024-09-11T00:56:53.116544+0000","last_active":"2024-09-11T00:56:54.109697+0000","last_peered":"2024-09-11T00:56:54.109697+0000","last_clean":"2024-09-11T00:56:54.109697+0000","last_became_active":"2024-09-11T00:56:53.116262+0000","last_became_peered":"2024-09-11T00:56:53.116262+0000","last_unstale":"2024-09-11T00:56:54.109697+0000","last_undegraded":"2024-09-11T00:56:54.109697+0000","last_fullsized":"2024-09-11T00:56:54.109697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:34.500669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.126456+0000","last_change":"2024-09-11T00:56:53.116967+0000","last_active":"2024-09-11T00:56:54.126456+0000","last_peered":"2024-09-11T00:56:54.126456+0000","last_clean":"2024-09-11T00:56:54.126456+0000","last_became_active":"2024-09-11T00:56:53.116405+0000","last_became_peered":"2024-09-11T00:56:53.116405+0000","last_unstale":"2024-09-11T00:56:54.126456+0000","last_undegraded":"2024-09-11T00:56:54.126456+0000","last_fullsized":"2024-09-11T00:56:54.126456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:33.521964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110122+0000","last_change":"2024-09-11T00:56:53.116727+0000","last_active":"2024-09-11T00:56:54.110122+0000","last_peered":"2024-09-11T00:56:54.110122+0000","last_clean":"2024-09-11T00:56:54.110122+0000","last_became_active":"2024-09-11T00:56:53.116416+0000","last_became_peered":"2024-09-11T00:56:53.116416+0000","last_unstale":"2024-09-11T00:56:54.110122+0000","last_undegraded":"2024-09-11T00:56:54.110122+0000","last_fullsized":"2024-09-11T00:56:54.110122+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:28.277909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.124869+0000","last_change":"2024-09-11T00:56:53.123141+0000","last_active":"2024-09-11T00:56:54.124869+0000","last_peered":"2024-09-11T00:56:54.124869+0000","last_clean":"2024-09-11T00:56:54.124869+0000","last_became_active":"2024-09-11T00:56:53.122811+0000","last_became_peered":"2024-09-11T00:56:53.122811+0000","last_unstale":"2024-09-11T00:56:54.124869+0000","last_undegraded":"2024-09-11T00:56:54.124869+0000","last_fullsized":"2024-09-11T00:56:54.124869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:47.175120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123872+0000","last_change":"2024-09-11T00:56:53.124213+0000","last_active":"2024-09-11T00:56:54.123872+0000","last_peered":"2024-09-11T00:56:54.123872+0000","last_clean":"2024-09-11T00:56:54.123872+0000","last_became_active":"2024-09-11T00:56:53.123757+0000","last_became_peered":"2024-09-11T00:56:53.123757+0000","last_unstale":"2024-09-11T00:56:54.123872+0000","last_undegraded":"2024-09-11T00:56:54.123872+0000","last_fullsized":"2024-09-11T00:56:54.123872+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:54.263476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576385+0000","last_change":"2024-09-11T00:56:53.114016+0000","last_active":"2024-09-11T00:57:17.576385+0000","last_peered":"2024-09-11T00:57:17.576385+0000","last_clean":"2024-09-11T00:57:17.576385+0000","last_became_active":"2024-09-11T00:56:53.113719+0000","last_became_peered":"2024-09-11T00:56:53.113719+0000","last_unstale":"2024-09-11T00:57:17.576385+0000","last_undegraded":"2024-09-11T00:57:17.576385+0000","last_fullsized":"2024-09-11T00:57:17.576385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:48:06.623810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122977+0000","last_change":"2024-09-11T00:56:53.124798+0000","last_active":"2024-09-11T00:56:54.122977+0000","last_peered":"2024-09-11T00:56:54.122977+0000","last_clean":"2024-09-11T00:56:54.122977+0000","last_became_active":"2024-09-11T00:56:53.124381+0000","last_became_peered":"2024-09-11T00:56:53.124381+0000","last_unstale":"2024-09-11T00:56:54.122977+0000","last_undegraded":"2024-09-11T00:56:54.122977+0000","last_fullsized":"2024-09-11T00:56:54.122977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:37.823112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122041+0000","last_change":"2024-09-11T00:56:53.115297+0000","last_active":"2024-09-11T00:56:54.122041+0000","last_peered":"2024-09-11T00:56:54.122041+0000","last_clean":"2024-09-11T00:56:54.122041+0000","last_became_active":"2024-09-11T00:56:53.114958+0000","last_became_peered":"2024-09-11T00:56:53.114958+0000","last_unstale":"2024-09-11T00:56:54.122041+0000","last_undegraded":"2024-09-11T00:56:54.122041+0000","last_fullsized":"2024-09-11T00:56:54.122041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:47.612610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.127915+0000","last_change":"2024-09-11T00:56:53.126717+0000","last_active":"2024-09-11T00:56:54.127915+0000","last_peered":"2024-09-11T00:56:54.127915+0000","last_clean":"2024-09-11T00:56:54.127915+0000","last_became_active":"2024-09-11T00:56:53.126387+0000","last_became_peered":"2024-09-11T00:56:53.126387+0000","last_unstale":"2024-09-11T00:56:54.127915+0000","last_undegraded":"2024-09-11T00:56:54.127915+0000","last_fullsized":"2024-09-11T00:56:54.127915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:52.429127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123170+0000","last_change":"2024-09-11T00:56:53.126257+0000","last_active":"2024-09-11T00:56:54.123170+0000","last_peered":"2024-09-11T00:56:54.123170+0000","last_clean":"2024-09-11T00:56:54.123170+0000","last_became_active":"2024-09-11T00:56:53.125146+0000","last_became_peered":"2024-09-11T00:56:53.125146+0000","last_unstale":"2024-09-11T00:56:54.123170+0000","last_undegraded":"2024-09-11T00:56:54.123170+0000","last_fullsized":"2024-09-11T00:56:54.123170+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581244+0000","last_change":"2024-09-11T00:56:53.119187+0000","last_active":"2024-09-11T00:57:17.581244+0000","last_peered":"2024-09-11T00:57:17.581244+0000","last_clean":"2024-09-11T00:57:17.581244+0000","last_became_active":"2024-09-11T00:56:53.118850+0000","last_became_peered":"2024-09-11T00:56:53.118850+0000","last_unstale":"2024-09-11T00:57:17.581244+0000","last_undegraded":"2024-09-11T00:57:17.581244+0000","last_fullsized":"2024-09-11T00:57:17.581244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:05.403876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:14.250920+0000","last_change":"2024-09-11T00:56:51.107140+0000","last_active":"2024-09-11T00:57:14.250920+0000","last_peered":"2024-09-11T00:57:14.250920+0000","last_clean":"2024-09-11T00:57:14.250920+0000","last_became_active":"2024-09-11T00:56:51.106748+0000","last_became_peered":"2024-09-11T00:56:51.106748+0000","last_unstale":"2024-09-11T00:57:14.250920+0000","last_undegraded":"2024-09-11T00:57:14.250920+0000","last_fullsized":"2024-09-11T00:57:14.250920+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:16.228197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.126288+0000","last_change":"2024-09-11T00:56:53.118049+0000","last_active":"2024-09-11T00:56:54.126288+0000","last_peered":"2024-09-11T00:56:54.126288+0000","last_clean":"2024-09-11T00:56:54.126288+0000","last_became_active":"2024-09-11T00:56:53.117685+0000","last_became_peered":"2024-09-11T00:56:53.117685+0000","last_unstale":"2024-09-11T00:56:54.126288+0000","last_undegraded":"2024-09-11T00:56:54.126288+0000","last_fullsized":"2024-09-11T00:56:54.126288+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:17.017594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578639+0000","last_change":"2024-09-11T00:56:51.101690+0000","last_active":"2024-09-11T00:57:17.578639+0000","last_peered":"2024-09-11T00:57:17.578639+0000","last_clean":"2024-09-11T00:57:17.578639+0000","last_became_active":"2024-09-11T00:56:51.101055+0000","last_became_peered":"2024-09-11T00:56:51.101055+0000","last_unstale":"2024-09-11T00:57:17.578639+0000","last_undegraded":"2024-09-11T00:57:17.578639+0000","last_fullsized":"2024-09-11T00:57:17.578639+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:28.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576176+0000","last_change":"2024-09-11T00:56:51.101700+0000","last_active":"2024-09-11T00:57:17.576176+0000","last_peered":"2024-09-11T00:57:17.576176+0000","last_clean":"2024-09-11T00:57:17.576176+0000","last_became_active":"2024-09-11T00:56:51.101044+0000","last_became_peered":"2024-09-11T00:56:51.101044+0000","last_unstale":"2024-09-11T00:57:17.576176+0000","last_undegraded":"2024-09-11T00:57:17.576176+0000","last_fullsized":"2024-09-11T00:57:17.576176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:12.121126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.109707+0000","last_change":"2024-09-11T00:56:53.121652+0000","last_active":"2024-09-11T00:56:54.109707+0000","last_peered":"2024-09-11T00:56:54.109707+0000","last_clean":"2024-09-11T00:56:54.109707+0000","last_became_active":"2024-09-11T00:56:53.121375+0000","last_became_peered":"2024-09-11T00:56:53.121375+0000","last_unstale":"2024-09-11T00:56:54.109707+0000","last_undegraded":"2024-09-11T00:56:54.109707+0000","last_fullsized":"2024-09-11T00:56:54.109707+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:39.154985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.232842+0000","last_change":"2024-09-11T00:56:51.104367+0000","last_active":"2024-09-11T00:57:03.232842+0000","last_peered":"2024-09-11T00:57:03.232842+0000","last_clean":"2024-09-11T00:57:03.232842+0000","last_became_active":"2024-09-11T00:56:51.104045+0000","last_became_peered":"2024-09-11T00:56:51.104045+0000","last_unstale":"2024-09-11T00:57:03.232842+0000","last_undegraded":"2024-09-11T00:57:03.232842+0000","last_fullsized":"2024-09-11T00:57:03.232842+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:03.989437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580225+0000","last_change":"2024-09-11T00:56:53.116167+0000","last_active":"2024-09-11T00:57:17.580225+0000","last_peered":"2024-09-11T00:57:17.580225+0000","last_clean":"2024-09-11T00:57:17.580225+0000","last_became_active":"2024-09-11T00:56:53.115857+0000","last_became_peered":"2024-09-11T00:56:53.115857+0000","last_unstale":"2024-09-11T00:57:17.580225+0000","last_undegraded":"2024-09-11T00:57:17.580225+0000","last_fullsized":"2024-09-11T00:57:17.580225+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:04.476094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.590345+0000","last_change":"2024-09-11T00:56:51.103287+0000","last_active":"2024-09-11T00:57:18.590345+0000","last_peered":"2024-09-11T00:57:18.590345+0000","last_clean":"2024-09-11T00:57:18.590345+0000","last_became_active":"2024-09-11T00:56:51.102953+0000","last_became_peered":"2024-09-11T00:56:51.102953+0000","last_unstale":"2024-09-11T00:57:18.590345+0000","last_undegraded":"2024-09-11T00:57:18.590345+0000","last_fullsized":"2024-09-11T00:57:18.590345+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:49.701612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160775+0000","last_change":"2024-09-11T00:56:53.125130+0000","last_active":"2024-09-11T00:57:18.160775+0000","last_peered":"2024-09-11T00:57:18.160775+0000","last_clean":"2024-09-11T00:57:18.160775+0000","last_became_active":"2024-09-11T00:56:53.124442+0000","last_became_peered":"2024-09-11T00:56:53.124442+0000","last_unstale":"2024-09-11T00:57:18.160775+0000","last_undegraded":"2024-09-11T00:57:18.160775+0000","last_fullsized":"2024-09-11T00:57:18.160775+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:24.439430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586976+0000","last_change":"2024-09-11T00:56:51.104802+0000","last_active":"2024-09-11T00:57:17.586976+0000","last_peered":"2024-09-11T00:57:17.586976+0000","last_clean":"2024-09-11T00:57:17.586976+0000","last_became_active":"2024-09-11T00:56:51.104335+0000","last_became_peered":"2024-09-11T00:56:51.104335+0000","last_unstale":"2024-09-11T00:57:17.586976+0000","last_undegraded":"2024-09-11T00:57:17.586976+0000","last_fullsized":"2024-09-11T00:57:17.586976+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:26.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123298+0000","last_change":"2024-09-11T00:56:53.124846+0000","last_active":"2024-09-11T00:56:54.123298+0000","last_peered":"2024-09-11T00:56:54.123298+0000","last_clean":"2024-09-11T00:56:54.123298+0000","last_became_active":"2024-09-11T00:56:53.124369+0000","last_became_peered":"2024-09-11T00:56:53.124369+0000","last_unstale":"2024-09-11T00:56:54.123298+0000","last_undegraded":"2024-09-11T00:56:54.123298+0000","last_fullsized":"2024-09-11T00:56:54.123298+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:49.012164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.191917+0000","last_change":"2024-09-11T00:56:51.154544+0000","last_active":"2024-09-11T00:57:03.191917+0000","last_peered":"2024-09-11T00:57:03.191917+0000","last_clean":"2024-09-11T00:57:03.191917+0000","last_became_active":"2024-09-11T00:56:51.154195+0000","last_became_peered":"2024-09-11T00:56:51.154195+0000","last_unstale":"2024-09-11T00:57:03.191917+0000","last_undegraded":"2024-09-11T00:57:03.191917+0000","last_fullsized":"2024-09-11T00:57:03.191917+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:57:24.530947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160506+0000","last_change":"2024-09-11T00:56:53.127728+0000","last_active":"2024-09-11T00:57:18.160506+0000","last_peered":"2024-09-11T00:57:18.160506+0000","last_clean":"2024-09-11T00:57:18.160506+0000","last_became_active":"2024-09-11T00:56:53.127423+0000","last_became_peered":"2024-09-11T00:56:53.127423+0000","last_unstale":"2024-09-11T00:57:18.160506+0000","last_undegraded":"2024-09-11T00:57:18.160506+0000","last_fullsized":"2024-09-11T00:57:18.160506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:55.717123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'27","reported_seq":88,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.590809+0000","last_change":"2024-09-11T00:56:51.100774+0000","last_active":"2024-09-11T00:57:18.590809+0000","last_peered":"2024-09-11T00:57:18.590809+0000","last_clean":"2024-09-11T00:57:18.590809+0000","last_became_active":"2024-09-11T00:56:51.100398+0000","last_became_peered":"2024-09-11T00:56:51.100398+0000","last_unstale":"2024-09-11T00:57:18.590809+0000","last_undegraded":"2024-09-11T00:57:18.590809+0000","last_fullsized":"2024-09-11T00:57:18.590809+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:51.975236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54952,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":72,"num_write":29,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581567+0000","last_change":"2024-09-11T00:56:53.120113+0000","last_active":"2024-09-11T00:57:17.581567+0000","last_peered":"2024-09-11T00:57:17.581567+0000","last_clean":"2024-09-11T00:57:17.581567+0000","last_became_active":"2024-09-11T00:56:53.119733+0000","last_became_peered":"2024-09-11T00:56:53.119733+0000","last_unstale":"2024-09-11T00:57:17.581567+0000","last_undegraded":"2024-09-11T00:57:17.581567+0000","last_fullsized":"2024-09-11T00:57:17.581567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:59.541710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.239133+0000","last_change":"2024-09-11T00:56:51.104234+0000","last_active":"2024-09-11T00:57:03.239133+0000","last_peered":"2024-09-11T00:57:03.239133+0000","last_clean":"2024-09-11T00:57:03.239133+0000","last_became_active":"2024-09-11T00:56:51.103631+0000","last_became_peered":"2024-09-11T00:56:51.103631+0000","last_unstale":"2024-09-11T00:57:03.239133+0000","last_undegraded":"2024-09-11T00:57:03.239133+0000","last_fullsized":"2024-09-11T00:57:03.239133+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:47.378135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576221+0000","last_change":"2024-09-11T00:56:53.122231+0000","last_active":"2024-09-11T00:57:17.576221+0000","last_peered":"2024-09-11T00:57:17.576221+0000","last_clean":"2024-09-11T00:57:17.576221+0000","last_became_active":"2024-09-11T00:56:53.121903+0000","last_became_peered":"2024-09-11T00:56:53.121903+0000","last_unstale":"2024-09-11T00:57:17.576221+0000","last_undegraded":"2024-09-11T00:57:17.576221+0000","last_fullsized":"2024-09-11T00:57:17.576221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:44.246919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.237744+0000","last_change":"2024-09-11T00:56:51.105625+0000","last_active":"2024-09-11T00:57:03.237744+0000","last_peered":"2024-09-11T00:57:03.237744+0000","last_clean":"2024-09-11T00:57:03.237744+0000","last_became_active":"2024-09-11T00:56:51.105291+0000","last_became_peered":"2024-09-11T00:56:51.105291+0000","last_unstale":"2024-09-11T00:57:03.237744+0000","last_undegraded":"2024-09-11T00:57:03.237744+0000","last_fullsized":"2024-09-11T00:57:03.237744+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:24.237174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586551+0000","last_change":"2024-09-11T00:56:53.126272+0000","last_active":"2024-09-11T00:57:17.586551+0000","last_peered":"2024-09-11T00:57:17.586551+0000","last_clean":"2024-09-11T00:57:17.586551+0000","last_became_active":"2024-09-11T00:56:53.125964+0000","last_became_peered":"2024-09-11T00:56:53.125964+0000","last_unstale":"2024-09-11T00:57:17.586551+0000","last_undegraded":"2024-09-11T00:57:17.586551+0000","last_fullsized":"2024-09-11T00:57:17.586551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:36.331220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:14.251287+0000","last_change":"2024-09-11T00:56:51.101053+0000","last_active":"2024-09-11T00:57:14.251287+0000","last_peered":"2024-09-11T00:57:14.251287+0000","last_clean":"2024-09-11T00:57:14.251287+0000","last_became_active":"2024-09-11T00:56:51.100714+0000","last_became_peered":"2024-09-11T00:56:51.100714+0000","last_unstale":"2024-09-11T00:57:14.251287+0000","last_undegraded":"2024-09-11T00:57:14.251287+0000","last_fullsized":"2024-09-11T00:57:14.251287+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:57.709126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586208+0000","last_change":"2024-09-11T00:56:53.119973+0000","last_active":"2024-09-11T00:57:17.586208+0000","last_peered":"2024-09-11T00:57:17.586208+0000","last_clean":"2024-09-11T00:57:17.586208+0000","last_became_active":"2024-09-11T00:56:53.118972+0000","last_became_peered":"2024-09-11T00:56:53.118972+0000","last_unstale":"2024-09-11T00:57:17.586208+0000","last_undegraded":"2024-09-11T00:57:17.586208+0000","last_fullsized":"2024-09-11T00:57:17.586208+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:35.912920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.228815+0000","last_change":"2024-09-11T00:56:51.153453+0000","last_active":"2024-09-11T00:57:03.228815+0000","last_peered":"2024-09-11T00:57:03.228815+0000","last_clean":"2024-09-11T00:57:03.228815+0000","last_became_active":"2024-09-11T00:56:51.153112+0000","last_became_peered":"2024-09-11T00:56:51.153112+0000","last_unstale":"2024-09-11T00:57:03.228815+0000","last_undegraded":"2024-09-11T00:57:03.228815+0000","last_fullsized":"2024-09-11T00:57:03.228815+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:41:32.001080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.125791+0000","last_change":"2024-09-11T00:56:53.120972+0000","last_active":"2024-09-11T00:56:54.125791+0000","last_peered":"2024-09-11T00:56:54.125791+0000","last_clean":"2024-09-11T00:56:54.125791+0000","last_became_active":"2024-09-11T00:56:53.120568+0000","last_became_peered":"2024-09-11T00:56:53.120568+0000","last_unstale":"2024-09-11T00:56:54.125791+0000","last_undegraded":"2024-09-11T00:56:54.125791+0000","last_fullsized":"2024-09-11T00:56:54.125791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:19.696201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581943+0000","last_change":"2024-09-11T00:56:51.103756+0000","last_active":"2024-09-11T00:57:17.581943+0000","last_peered":"2024-09-11T00:57:17.581943+0000","last_clean":"2024-09-11T00:57:17.581943+0000","last_became_active":"2024-09-11T00:56:51.103406+0000","last_became_peered":"2024-09-11T00:56:51.103406+0000","last_unstale":"2024-09-11T00:57:17.581943+0000","last_undegraded":"2024-09-11T00:57:17.581943+0000","last_fullsized":"2024-09-11T00:57:17.581943+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:28.022346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123457+0000","last_change":"2024-09-11T00:56:53.128628+0000","last_active":"2024-09-11T00:56:54.123457+0000","last_peered":"2024-09-11T00:56:54.123457+0000","last_clean":"2024-09-11T00:56:54.123457+0000","last_became_active":"2024-09-11T00:56:53.127197+0000","last_became_peered":"2024-09-11T00:56:53.127197+0000","last_unstale":"2024-09-11T00:56:54.123457+0000","last_undegraded":"2024-09-11T00:56:54.123457+0000","last_fullsized":"2024-09-11T00:56:54.123457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:38.753676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576498+0000","last_change":"2024-09-11T00:56:51.101398+0000","last_active":"2024-09-11T00:57:17.576498+0000","last_peered":"2024-09-11T00:57:17.576498+0000","last_clean":"2024-09-11T00:57:17.576498+0000","last_became_active":"2024-09-11T00:56:51.101034+0000","last_became_peered":"2024-09-11T00:56:51.101034+0000","last_unstale":"2024-09-11T00:57:17.576498+0000","last_undegraded":"2024-09-11T00:57:17.576498+0000","last_fullsized":"2024-09-11T00:57:17.576498+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:40.165368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586583+0000","last_change":"2024-09-11T00:56:53.127047+0000","last_active":"2024-09-11T00:57:17.586583+0000","last_peered":"2024-09-11T00:57:17.586583+0000","last_clean":"2024-09-11T00:57:17.586583+0000","last_became_active":"2024-09-11T00:56:53.126719+0000","last_became_peered":"2024-09-11T00:56:53.126719+0000","last_unstale":"2024-09-11T00:57:17.586583+0000","last_undegraded":"2024-09-11T00:57:17.586583+0000","last_fullsized":"2024-09-11T00:57:17.586583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.686604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.228693+0000","last_change":"2024-09-11T00:56:51.101665+0000","last_active":"2024-09-11T00:57:03.228693+0000","last_peered":"2024-09-11T00:57:03.228693+0000","last_clean":"2024-09-11T00:57:03.228693+0000","last_became_active":"2024-09-11T00:56:51.101332+0000","last_became_peered":"2024-09-11T00:56:51.101332+0000","last_unstale":"2024-09-11T00:57:03.228693+0000","last_undegraded":"2024-09-11T00:57:03.228693+0000","last_fullsized":"2024-09-11T00:57:03.228693+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:47.639877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586452+0000","last_change":"2024-09-11T00:56:53.128736+0000","last_active":"2024-09-11T00:57:17.586452+0000","last_peered":"2024-09-11T00:57:17.586452+0000","last_clean":"2024-09-11T00:57:17.586452+0000","last_became_active":"2024-09-11T00:56:53.128417+0000","last_became_peered":"2024-09-11T00:56:53.128417+0000","last_unstale":"2024-09-11T00:57:17.586452+0000","last_undegraded":"2024-09-11T00:57:17.586452+0000","last_fullsized":"2024-09-11T00:57:17.586452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:52.331343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:14.250940+0000","last_change":"2024-09-11T00:56:51.154733+0000","last_active":"2024-09-11T00:57:14.250940+0000","last_peered":"2024-09-11T00:57:14.250940+0000","last_clean":"2024-09-11T00:57:14.250940+0000","last_became_active":"2024-09-11T00:56:51.153613+0000","last_became_peered":"2024-09-11T00:56:51.153613+0000","last_unstale":"2024-09-11T00:57:14.250940+0000","last_undegraded":"2024-09-11T00:57:14.250940+0000","last_fullsized":"2024-09-11T00:57:14.250940+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:24.749186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586818+0000","last_change":"2024-09-11T00:56:53.124684+0000","last_active":"2024-09-11T00:57:17.586818+0000","last_peered":"2024-09-11T00:57:17.586818+0000","last_clean":"2024-09-11T00:57:17.586818+0000","last_became_active":"2024-09-11T00:56:53.124147+0000","last_became_peered":"2024-09-11T00:56:53.124147+0000","last_unstale":"2024-09-11T00:57:17.586818+0000","last_undegraded":"2024-09-11T00:57:17.586818+0000","last_fullsized":"2024-09-11T00:57:17.586818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:14.280465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586464+0000","last_change":"2024-09-11T00:55:34.500373+0000","last_active":"2024-09-11T00:57:17.586464+0000","last_peered":"2024-09-11T00:57:17.586464+0000","last_clean":"2024-09-11T00:57:17.586464+0000","last_became_active":"2024-09-11T00:55:34.495540+0000","last_became_peered":"2024-09-11T00:55:34.495540+0000","last_unstale":"2024-09-11T00:57:17.586464+0000","last_undegraded":"2024-09-11T00:57:17.586464+0000","last_fullsized":"2024-09-11T00:57:17.586464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_clean_scrub_stamp":"2024-09-11T00:52:11.168966+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:37.681466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.237524+0000","last_change":"2024-09-11T00:56:51.101524+0000","last_active":"2024-09-11T00:57:03.237524+0000","last_peered":"2024-09-11T00:57:03.237524+0000","last_clean":"2024-09-11T00:57:03.237524+0000","last_became_active":"2024-09-11T00:56:51.101180+0000","last_became_peered":"2024-09-11T00:56:51.101180+0000","last_unstale":"2024-09-11T00:57:03.237524+0000","last_undegraded":"2024-09-11T00:57:03.237524+0000","last_fullsized":"2024-09-11T00:57:03.237524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:27.293494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.109947+0000","last_change":"2024-09-11T00:56:53.118503+0000","last_active":"2024-09-11T00:56:54.109947+0000","last_peered":"2024-09-11T00:56:54.109947+0000","last_clean":"2024-09-11T00:56:54.109947+0000","last_became_active":"2024-09-11T00:56:53.118174+0000","last_became_peered":"2024-09-11T00:56:53.118174+0000","last_unstale":"2024-09-11T00:56:54.109947+0000","last_undegraded":"2024-09-11T00:56:54.109947+0000","last_fullsized":"2024-09-11T00:56:54.109947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:13.524153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586788+0000","last_change":"2024-09-11T00:56:51.104222+0000","last_active":"2024-09-11T00:57:17.586788+0000","last_peered":"2024-09-11T00:57:17.586788+0000","last_clean":"2024-09-11T00:57:17.586788+0000","last_became_active":"2024-09-11T00:56:51.103913+0000","last_became_peered":"2024-09-11T00:56:51.103913+0000","last_unstale":"2024-09-11T00:57:17.586788+0000","last_undegraded":"2024-09-11T00:57:17.586788+0000","last_fullsized":"2024-09-11T00:57:17.586788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:57.467659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576162+0000","last_change":"2024-09-11T00:56:53.115130+0000","last_active":"2024-09-11T00:57:17.576162+0000","last_peered":"2024-09-11T00:57:17.576162+0000","last_clean":"2024-09-11T00:57:17.576162+0000","last_became_active":"2024-09-11T00:56:53.114813+0000","last_became_peered":"2024-09-11T00:56:53.114813+0000","last_unstale":"2024-09-11T00:57:17.576162+0000","last_undegraded":"2024-09-11T00:57:17.576162+0000","last_fullsized":"2024-09-11T00:57:17.576162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:13.265173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576273+0000","last_change":"2024-09-11T00:56:51.152892+0000","last_active":"2024-09-11T00:57:17.576273+0000","last_peered":"2024-09-11T00:57:17.576273+0000","last_clean":"2024-09-11T00:57:17.576273+0000","last_became_active":"2024-09-11T00:56:51.152572+0000","last_became_peered":"2024-09-11T00:56:51.152572+0000","last_unstale":"2024-09-11T00:57:17.576273+0000","last_undegraded":"2024-09-11T00:57:17.576273+0000","last_fullsized":"2024-09-11T00:57:17.576273+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:03.774593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.126674+0000","last_change":"2024-09-11T00:56:53.117416+0000","last_active":"2024-09-11T00:56:54.126674+0000","last_peered":"2024-09-11T00:56:54.126674+0000","last_clean":"2024-09-11T00:56:54.126674+0000","last_became_active":"2024-09-11T00:56:53.117016+0000","last_became_peered":"2024-09-11T00:56:53.117016+0000","last_unstale":"2024-09-11T00:56:54.126674+0000","last_undegraded":"2024-09-11T00:56:54.126674+0000","last_fullsized":"2024-09-11T00:56:54.126674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:07.607993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586732+0000","last_change":"2024-09-11T00:56:51.102879+0000","last_active":"2024-09-11T00:57:17.586732+0000","last_peered":"2024-09-11T00:57:17.586732+0000","last_clean":"2024-09-11T00:57:17.586732+0000","last_became_active":"2024-09-11T00:56:51.102498+0000","last_became_peered":"2024-09-11T00:56:51.102498+0000","last_unstale":"2024-09-11T00:57:17.586732+0000","last_undegraded":"2024-09-11T00:57:17.586732+0000","last_fullsized":"2024-09-11T00:57:17.586732+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:41.360908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581368+0000","last_change":"2024-09-11T00:57:17.581261+0000","last_active":"2024-09-11T00:57:17.581368+0000","last_peered":"2024-09-11T00:57:17.581368+0000","last_clean":"2024-09-11T00:57:17.581368+0000","last_became_active":"2024-09-11T00:57:17.580941+0000","last_became_peered":"2024-09-11T00:57:17.580941+0000","last_unstale":"2024-09-11T00:57:17.581368+0000","last_undegraded":"2024-09-11T00:57:17.581368+0000","last_fullsized":"2024-09-11T00:57:17.581368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:58.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.584416+0000","last_change":"2024-09-11T00:57:17.584311+0000","last_active":"2024-09-11T00:57:17.584416+0000","last_peered":"2024-09-11T00:57:17.584416+0000","last_clean":"2024-09-11T00:57:17.584416+0000","last_became_active":"2024-09-11T00:57:17.583964+0000","last_became_peered":"2024-09-11T00:57:17.583964+0000","last_unstale":"2024-09-11T00:57:17.584416+0000","last_undegraded":"2024-09-11T00:57:17.584416+0000","last_fullsized":"2024-09-11T00:57:17.584416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:15.515947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578614+0000","last_change":"2024-09-11T00:56:53.112550+0000","last_active":"2024-09-11T00:57:17.578614+0000","last_peered":"2024-09-11T00:57:17.578614+0000","last_clean":"2024-09-11T00:57:17.578614+0000","last_became_active":"2024-09-11T00:56:53.112179+0000","last_became_peered":"2024-09-11T00:56:53.112179+0000","last_unstale":"2024-09-11T00:57:17.578614+0000","last_undegraded":"2024-09-11T00:57:17.578614+0000","last_fullsized":"2024-09-11T00:57:17.578614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:03.339839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581699+0000","last_change":"2024-09-11T00:56:51.152957+0000","last_active":"2024-09-11T00:57:17.581699+0000","last_peered":"2024-09-11T00:57:17.581699+0000","last_clean":"2024-09-11T00:57:17.581699+0000","last_became_active":"2024-09-11T00:56:51.152616+0000","last_became_peered":"2024-09-11T00:56:51.152616+0000","last_unstale":"2024-09-11T00:57:17.581699+0000","last_undegraded":"2024-09-11T00:57:17.581699+0000","last_fullsized":"2024-09-11T00:57:17.581699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:02.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576410+0000","last_change":"2024-09-11T00:56:53.115155+0000","last_active":"2024-09-11T00:57:17.576410+0000","last_peered":"2024-09-11T00:57:17.576410+0000","last_clean":"2024-09-11T00:57:17.576410+0000","last_became_active":"2024-09-11T00:56:53.114865+0000","last_became_peered":"2024-09-11T00:56:53.114865+0000","last_unstale":"2024-09-11T00:57:17.576410+0000","last_undegraded":"2024-09-11T00:57:17.576410+0000","last_fullsized":"2024-09-11T00:57:17.576410+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:23.968888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'15","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580547+0000","last_change":"2024-09-11T00:56:51.100887+0000","last_active":"2024-09-11T00:57:17.580547+0000","last_peered":"2024-09-11T00:57:17.580547+0000","last_clean":"2024-09-11T00:57:17.580547+0000","last_became_active":"2024-09-11T00:56:51.100474+0000","last_became_peered":"2024-09-11T00:56:51.100474+0000","last_unstale":"2024-09-11T00:57:17.580547+0000","last_undegraded":"2024-09-11T00:57:17.580547+0000","last_fullsized":"2024-09-11T00:57:17.580547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:40.884871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.588425+0000","last_change":"2024-09-11T00:57:17.588086+0000","last_active":"2024-09-11T00:57:17.588425+0000","last_peered":"2024-09-11T00:57:17.588425+0000","last_clean":"2024-09-11T00:57:17.588425+0000","last_became_active":"2024-09-11T00:57:17.587785+0000","last_became_peered":"2024-09-11T00:57:17.587785+0000","last_unstale":"2024-09-11T00:57:17.588425+0000","last_undegraded":"2024-09-11T00:57:17.588425+0000","last_fullsized":"2024-09-11T00:57:17.588425+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:49.494588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.237796+0000","last_change":"2024-09-11T00:56:51.102031+0000","last_active":"2024-09-11T00:57:03.237796+0000","last_peered":"2024-09-11T00:57:03.237796+0000","last_clean":"2024-09-11T00:57:03.237796+0000","last_became_active":"2024-09-11T00:56:51.100614+0000","last_became_peered":"2024-09-11T00:56:51.100614+0000","last_unstale":"2024-09-11T00:57:03.237796+0000","last_undegraded":"2024-09-11T00:57:03.237796+0000","last_fullsized":"2024-09-11T00:57:03.237796+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:52.611261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.122728+0000","last_change":"2024-09-11T00:56:53.124806+0000","last_active":"2024-09-11T00:56:54.122728+0000","last_peered":"2024-09-11T00:56:54.122728+0000","last_clean":"2024-09-11T00:56:54.122728+0000","last_became_active":"2024-09-11T00:56:53.124452+0000","last_became_peered":"2024-09-11T00:56:53.124452+0000","last_unstale":"2024-09-11T00:56:54.122728+0000","last_undegraded":"2024-09-11T00:56:54.122728+0000","last_fullsized":"2024-09-11T00:56:54.122728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:57.934808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160509+0000","last_change":"2024-09-11T00:56:51.153621+0000","last_active":"2024-09-11T00:57:18.160509+0000","last_peered":"2024-09-11T00:57:18.160509+0000","last_clean":"2024-09-11T00:57:18.160509+0000","last_became_active":"2024-09-11T00:56:51.153315+0000","last_became_peered":"2024-09-11T00:56:51.153315+0000","last_unstale":"2024-09-11T00:57:18.160509+0000","last_undegraded":"2024-09-11T00:57:18.160509+0000","last_fullsized":"2024-09-11T00:57:18.160509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:10.058295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580192+0000","last_change":"2024-09-11T00:56:53.114463+0000","last_active":"2024-09-11T00:57:17.580192+0000","last_peered":"2024-09-11T00:57:17.580192+0000","last_clean":"2024-09-11T00:57:17.580192+0000","last_became_active":"2024-09-11T00:56:53.114099+0000","last_became_peered":"2024-09-11T00:56:53.114099+0000","last_unstale":"2024-09-11T00:57:17.580192+0000","last_undegraded":"2024-09-11T00:57:17.580192+0000","last_fullsized":"2024-09-11T00:57:17.580192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:58.981787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.235580+0000","last_change":"2024-09-11T00:56:51.096942+0000","last_active":"2024-09-11T00:57:03.235580+0000","last_peered":"2024-09-11T00:57:03.235580+0000","last_clean":"2024-09-11T00:57:03.235580+0000","last_became_active":"2024-09-11T00:56:51.096584+0000","last_became_peered":"2024-09-11T00:56:51.096584+0000","last_unstale":"2024-09-11T00:57:03.235580+0000","last_undegraded":"2024-09-11T00:57:03.235580+0000","last_fullsized":"2024-09-11T00:57:03.235580+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:03.484123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123153+0000","last_change":"2024-09-11T00:56:53.126804+0000","last_active":"2024-09-11T00:56:54.123153+0000","last_peered":"2024-09-11T00:56:54.123153+0000","last_clean":"2024-09-11T00:56:54.123153+0000","last_became_active":"2024-09-11T00:56:53.125408+0000","last_became_peered":"2024-09-11T00:56:53.125408+0000","last_unstale":"2024-09-11T00:56:54.123153+0000","last_undegraded":"2024-09-11T00:56:54.123153+0000","last_fullsized":"2024-09-11T00:56:54.123153+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:46.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:57:03.190572+0000","last_change":"2024-09-11T00:56:51.103755+0000","last_active":"2024-09-11T00:57:03.190572+0000","last_peered":"2024-09-11T00:57:03.190572+0000","last_clean":"2024-09-11T00:57:03.190572+0000","last_became_active":"2024-09-11T00:56:51.103395+0000","last_became_peered":"2024-09-11T00:56:51.103395+0000","last_unstale":"2024-09-11T00:57:03.190572+0000","last_undegraded":"2024-09-11T00:57:03.190572+0000","last_fullsized":"2024-09-11T00:57:03.190572+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:16.782660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110136+0000","last_change":"2024-09-11T00:56:53.121926+0000","last_active":"2024-09-11T00:56:54.110136+0000","last_peered":"2024-09-11T00:56:54.110136+0000","last_clean":"2024-09-11T00:56:54.110136+0000","last_became_active":"2024-09-11T00:56:53.121603+0000","last_became_peered":"2024-09-11T00:56:53.121603+0000","last_unstale":"2024-09-11T00:56:54.110136+0000","last_undegraded":"2024-09-11T00:56:54.110136+0000","last_fullsized":"2024-09-11T00:56:54.110136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:31.021030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580437+0000","last_change":"2024-09-11T00:56:51.107221+0000","last_active":"2024-09-11T00:57:17.580437+0000","last_peered":"2024-09-11T00:57:17.580437+0000","last_clean":"2024-09-11T00:57:17.580437+0000","last_became_active":"2024-09-11T00:56:51.106849+0000","last_became_peered":"2024-09-11T00:56:51.106849+0000","last_unstale":"2024-09-11T00:57:17.580437+0000","last_undegraded":"2024-09-11T00:57:17.580437+0000","last_fullsized":"2024-09-11T00:57:17.580437+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:46.497230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586475+0000","last_change":"2024-09-11T00:56:53.120525+0000","last_active":"2024-09-11T00:57:17.586475+0000","last_peered":"2024-09-11T00:57:17.586475+0000","last_clean":"2024-09-11T00:57:17.586475+0000","last_became_active":"2024-09-11T00:56:53.120228+0000","last_became_peered":"2024-09-11T00:56:53.120228+0000","last_unstale":"2024-09-11T00:57:17.586475+0000","last_undegraded":"2024-09-11T00:57:17.586475+0000","last_fullsized":"2024-09-11T00:57:17.586475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:52.047687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.126929+0000","last_change":"2024-09-11T00:56:53.120311+0000","last_active":"2024-09-11T00:56:54.126929+0000","last_peered":"2024-09-11T00:56:54.126929+0000","last_clean":"2024-09-11T00:56:54.126929+0000","last_became_active":"2024-09-11T00:56:53.119648+0000","last_became_peered":"2024-09-11T00:56:53.119648+0000","last_unstale":"2024-09-11T00:56:54.126929+0000","last_undegraded":"2024-09-11T00:56:54.126929+0000","last_fullsized":"2024-09-11T00:56:54.126929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:21.187784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581118+0000","last_change":"2024-09-11T00:56:51.103631+0000","last_active":"2024-09-11T00:57:17.581118+0000","last_peered":"2024-09-11T00:57:17.581118+0000","last_clean":"2024-09-11T00:57:17.581118+0000","last_became_active":"2024-09-11T00:56:51.103281+0000","last_became_peered":"2024-09-11T00:56:51.103281+0000","last_unstale":"2024-09-11T00:57:17.581118+0000","last_undegraded":"2024-09-11T00:57:17.581118+0000","last_fullsized":"2024-09-11T00:57:17.581118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:16.910956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160481+0000","last_change":"2024-09-11T00:56:53.124752+0000","last_active":"2024-09-11T00:57:18.160481+0000","last_peered":"2024-09-11T00:57:18.160481+0000","last_clean":"2024-09-11T00:57:18.160481+0000","last_became_active":"2024-09-11T00:56:53.124245+0000","last_became_peered":"2024-09-11T00:56:53.124245+0000","last_unstale":"2024-09-11T00:57:18.160481+0000","last_undegraded":"2024-09-11T00:57:18.160481+0000","last_fullsized":"2024-09-11T00:57:18.160481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:10.277903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581704+0000","last_change":"2024-09-11T00:56:51.104624+0000","last_active":"2024-09-11T00:57:17.581704+0000","last_peered":"2024-09-11T00:57:17.581704+0000","last_clean":"2024-09-11T00:57:17.581704+0000","last_became_active":"2024-09-11T00:56:51.104041+0000","last_became_peered":"2024-09-11T00:56:51.104041+0000","last_unstale":"2024-09-11T00:57:17.581704+0000","last_undegraded":"2024-09-11T00:57:17.581704+0000","last_fullsized":"2024-09-11T00:57:17.581704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:10.397455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.123171+0000","last_change":"2024-09-11T00:56:53.118547+0000","last_active":"2024-09-11T00:56:54.123171+0000","last_peered":"2024-09-11T00:56:54.123171+0000","last_clean":"2024-09-11T00:56:54.123171+0000","last_became_active":"2024-09-11T00:56:53.118291+0000","last_became_peered":"2024-09-11T00:56:53.118291+0000","last_unstale":"2024-09-11T00:56:54.123171+0000","last_undegraded":"2024-09-11T00:56:54.123171+0000","last_fullsized":"2024-09-11T00:56:54.123171+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:42.012173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.125382+0000","last_change":"2024-09-11T00:56:51.153047+0000","last_active":"2024-09-11T00:56:54.125382+0000","last_peered":"2024-09-11T00:56:54.125382+0000","last_clean":"2024-09-11T00:56:54.125382+0000","last_became_active":"2024-09-11T00:56:51.152564+0000","last_became_peered":"2024-09-11T00:56:51.152564+0000","last_unstale":"2024-09-11T00:56:54.125382+0000","last_undegraded":"2024-09-11T00:56:54.125382+0000","last_fullsized":"2024-09-11T00:56:54.125382+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:16.849087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-11T00:56:54.110141+0000","last_change":"2024-09-11T00:56:53.119926+0000","last_active":"2024-09-11T00:56:54.110141+0000","last_peered":"2024-09-11T00:56:54.110141+0000","last_clean":"2024-09-11T00:56:54.110141+0000","last_became_active":"2024-09-11T00:56:53.119643+0000","last_became_peered":"2024-09-11T00:56:53.119643+0000","last_unstale":"2024-09-11T00:56:54.110141+0000","last_undegraded":"2024-09-11T00:56:54.110141+0000","last_fullsized":"2024-09-11T00:56:54.110141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:51.838958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":52,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580658+0000","last_change":"2024-09-11T00:56:51.100837+0000","last_active":"2024-09-11T00:57:17.580658+0000","last_peered":"2024-09-11T00:57:17.580658+0000","last_clean":"2024-09-11T00:57:17.580658+0000","last_became_active":"2024-09-11T00:56:51.100494+0000","last_became_peered":"2024-09-11T00:56:51.100494+0000","last_unstale":"2024-09-11T00:57:17.580658+0000","last_undegraded":"2024-09-11T00:57:17.580658+0000","last_fullsized":"2024-09-11T00:57:17.580658+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:39.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61765,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":186,"num_read_kb":166,"num_write":239,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":172432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286865,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743441,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":399040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":1062482,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070493,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":868352,"data_stored":416431,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266721,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":856064,"data_stored":416697,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":917504,"data_stored":1009327,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626475,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":407503,"data_compressed":2911,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":991544,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":827392,"data_stored":398548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":398512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378688,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":442412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542214,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":909312,"data_stored":472013,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705738,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":400026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:57:21.622 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T00:57:21.622 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T00:57:21.623 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T00:57:21.872 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:21.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:21 smithi110 ceph-mon[34712]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:57:21.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:21 smithi112 ceph-mon[35384]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:57:22.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.378+0000 7fd795d47640 1 -- 172.21.15.78:0/191362027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd790102dc0 msgr2=0x7fd7901031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:22.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.378+0000 7fd795d47640 1 --2- 172.21.15.78:0/191362027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd790102dc0 0x7fd7901031a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd780009900 tx=0x7fd78002a410 comp rx=0 tx=0).stop 2024-09-11T00:57:22.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.378+0000 7fd795d47640 1 -- 172.21.15.78:0/191362027 shutdown_connections 2024-09-11T00:57:22.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.378+0000 7fd795d47640 1 --2- 172.21.15.78:0/191362027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd790109150 0x7fd79010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.378+0000 7fd795d47640 1 --2- 172.21.15.78:0/191362027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7901087d0 0x7fd790108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.378+0000 7fd795d47640 1 --2- 172.21.15.78:0/191362027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd790102dc0 0x7fd7901031a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.378+0000 7fd795d47640 1 -- 172.21.15.78:0/191362027 >> 172.21.15.78:0/191362027 conn(0x7fd7900fe830 msgr2=0x7fd790100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.378+0000 7fd795d47640 1 -- 172.21.15.78:0/191362027 shutdown_connections 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.378+0000 7fd795d47640 1 -- 172.21.15.78:0/191362027 wait complete. 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd795d47640 1 Processor -- start 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd795d47640 1 -- start start 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd795d47640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd790102dc0 0x7fd790199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd795d47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7901087d0 0x7fd790199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd795d47640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd790109150 0x7fd79019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd795d47640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd790112e50 con 0x7fd7901087d0 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd795d47640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd790112cd0 con 0x7fd790102dc0 2024-09-11T00:57:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd795d47640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd790112fd0 con 0x7fd790109150 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd78effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7901087d0 0x7fd790199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd78effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7901087d0 0x7fd790199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40176/0 (socket says 172.21.15.78:40176) 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd78effd640 1 -- 172.21.15.78:0/669208146 learned_addr learned my addr 172.21.15.78:0/669208146 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd78f7fe640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd790102dc0 0x7fd790199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd78ffff640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd790109150 0x7fd79019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd78effd640 1 -- 172.21.15.78:0/669208146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd790109150 msgr2=0x7fd79019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd78effd640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd790109150 0x7fd79019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd78effd640 1 -- 172.21.15.78:0/669208146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd790102dc0 msgr2=0x7fd790199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd78effd640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd790102dc0 0x7fd790199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.379+0000 7fd78effd640 1 -- 172.21.15.78:0/669208146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79019f4f0 con 0x7fd7901087d0 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.380+0000 7fd78f7fe640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd790102dc0 0x7fd790199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.380+0000 7fd78effd640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7901087d0 0x7fd790199980 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd78400b4a0 tx=0x7fd78400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.380+0000 7fd78cff9640 1 -- 172.21.15.78:0/669208146 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784014070 con 0x7fd7901087d0 2024-09-11T00:57:22.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.380+0000 7fd78cff9640 1 -- 172.21.15.78:0/669208146 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd784010ac0 con 0x7fd7901087d0 2024-09-11T00:57:22.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.380+0000 7fd78cff9640 1 -- 172.21.15.78:0/669208146 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784017c30 con 0x7fd7901087d0 2024-09-11T00:57:22.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.380+0000 7fd795d47640 1 -- 172.21.15.78:0/669208146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79019f7e0 con 0x7fd7901087d0 2024-09-11T00:57:22.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.380+0000 7fd795d47640 1 -- 172.21.15.78:0/669208146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7901a5ea0 con 0x7fd7901087d0 2024-09-11T00:57:22.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.381+0000 7fd795d47640 1 -- 172.21.15.78:0/669208146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd790103220 con 0x7fd7901087d0 2024-09-11T00:57:22.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.382+0000 7fd78cff9640 1 -- 172.21.15.78:0/669208146 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fd784010610 con 0x7fd7901087d0 2024-09-11T00:57:22.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.383+0000 7fd78cff9640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd7780b5440 0x7fd7780b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:22.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.383+0000 7fd78f7fe640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd7780b5440 0x7fd7780b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:22.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.383+0000 7fd78cff9640 1 -- 172.21.15.78:0/669208146 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd7840e5830 con 0x7fd7901087d0 2024-09-11T00:57:22.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.383+0000 7fd78f7fe640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd7780b5440 0x7fd7780b7900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd780031440 tx=0x7fd7800095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:22.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.386+0000 7fd78cff9640 1 -- 172.21.15.78:0/669208146 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd7840ac4e0 con 0x7fd7901087d0 2024-09-11T00:57:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.629+0000 7fd795d47640 1 -- 172.21.15.78:0/669208146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd79019a570 con 0x7fd7901087d0 2024-09-11T00:57:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.631+0000 7fd78cff9640 1 -- 172.21.15.78:0/669208146 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd79019a570 con 0x7fd7901087d0 2024-09-11T00:57:22.634 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T00:57:22.634 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQDy6uBmKhqsJRAAMbXiE6m2Ln6uegfpyEaJDw== 2024-09-11T00:57:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.634+0000 7fd795d47640 1 -- 172.21.15.78:0/669208146 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd7780b5440 msgr2=0x7fd7780b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.634+0000 7fd795d47640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd7780b5440 0x7fd7780b7900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd780031440 tx=0x7fd7800095f0 comp rx=0 tx=0).stop 2024-09-11T00:57:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.634+0000 7fd795d47640 1 -- 172.21.15.78:0/669208146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7901087d0 msgr2=0x7fd790199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.634+0000 7fd795d47640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7901087d0 0x7fd790199980 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd78400b4a0 tx=0x7fd78400b960 comp rx=0 tx=0).stop 2024-09-11T00:57:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.634+0000 7fd795d47640 1 -- 172.21.15.78:0/669208146 shutdown_connections 2024-09-11T00:57:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.634+0000 7fd795d47640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd7780b5440 0x7fd7780b7900 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.634+0000 7fd795d47640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd790109150 0x7fd79019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.634+0000 7fd795d47640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7901087d0 0x7fd790199980 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.634+0000 7fd795d47640 1 --2- 172.21.15.78:0/669208146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd790102dc0 0x7fd790199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.634+0000 7fd795d47640 1 -- 172.21.15.78:0/669208146 >> 172.21.15.78:0/669208146 conn(0x7fd7900fe830 msgr2=0x7fd790100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.634+0000 7fd795d47640 1 -- 172.21.15.78:0/669208146 shutdown_connections 2024-09-11T00:57:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:22.634+0000 7fd795d47640 1 -- 172.21.15.78:0/669208146 wait complete. 2024-09-11T00:57:22.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:22 smithi110 ceph-mon[34712]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.0 KiB/s wr, 13 op/s; 140 B/s, 1 objects/s recovering 2024-09-11T00:57:22.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:22 smithi112 ceph-mon[35384]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.0 KiB/s wr, 13 op/s; 140 B/s, 1 objects/s recovering 2024-09-11T00:57:22.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:22 smithi078 ceph-mon[31315]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.0 KiB/s wr, 13 op/s; 140 B/s, 1 objects/s recovering 2024-09-11T00:57:23.481 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:57:23.481 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T00:57:23.482 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T00:57:23.521 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph config log 1 --format=json 2024-09-11T00:57:23.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:23 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/669208146' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-11T00:57:23.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:23 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/669208146' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T00:57:23.807 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:23.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:23 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/669208146' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-11T00:57:23.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:23 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/669208146' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T00:57:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:23 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/669208146' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-11T00:57:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:23 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/669208146' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T00:57:24.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.322+0000 7f6d4da50640 1 -- 172.21.15.78:0/1237191841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d48103b70 msgr2=0x7f6d48103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:24.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.322+0000 7f6d4da50640 1 --2- 172.21.15.78:0/1237191841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d48103b70 0x7f6d48103f50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009ca0 tx=0x7f6d3802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:24.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.322+0000 7f6d4da50640 1 -- 172.21.15.78:0/1237191841 shutdown_connections 2024-09-11T00:57:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.322+0000 7f6d4da50640 1 --2- 172.21.15.78:0/1237191841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d4810d120 0x7f6d4810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.323+0000 7f6d4da50640 1 --2- 172.21.15.78:0/1237191841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d48104570 0x7f6d4810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.323+0000 7f6d4da50640 1 --2- 172.21.15.78:0/1237191841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d48103b70 0x7f6d48103f50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.323+0000 7f6d4da50640 1 -- 172.21.15.78:0/1237191841 >> 172.21.15.78:0/1237191841 conn(0x7f6d480fd700 msgr2=0x7f6d480ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.323+0000 7f6d4da50640 1 -- 172.21.15.78:0/1237191841 shutdown_connections 2024-09-11T00:57:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.323+0000 7f6d4da50640 1 -- 172.21.15.78:0/1237191841 wait complete. 2024-09-11T00:57:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.323+0000 7f6d4da50640 1 Processor -- start 2024-09-11T00:57:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.323+0000 7f6d4da50640 1 -- start start 2024-09-11T00:57:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.323+0000 7f6d4da50640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d48103b70 0x7f6d48199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.323+0000 7f6d4da50640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d48104570 0x7f6d481997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d4da50640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d4810d120 0x7f6d4819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d4da50640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48112d60 con 0x7f6d48103b70 2024-09-11T00:57:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d4da50640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48112be0 con 0x7f6d4810d120 2024-09-11T00:57:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d4da50640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48112ee0 con 0x7f6d48104570 2024-09-11T00:57:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d48103b70 0x7f6d48199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d48103b70 0x7f6d48199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56096/0 (socket says 172.21.15.78:56096) 2024-09-11T00:57:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d46ffd640 1 -- 172.21.15.78:0/3893392435 learned_addr learned my addr 172.21.15.78:0/3893392435 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d467fc640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d48104570 0x7f6d481997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d477fe640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d4810d120 0x7f6d4819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d46ffd640 1 -- 172.21.15.78:0/3893392435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d48104570 msgr2=0x7f6d481997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d46ffd640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d48104570 0x7f6d481997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d46ffd640 1 -- 172.21.15.78:0/3893392435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d4810d120 msgr2=0x7f6d4819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d46ffd640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d4810d120 0x7f6d4819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d46ffd640 1 -- 172.21.15.78:0/3893392435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4819f340 con 0x7f6d48103b70 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d477fe640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d4810d120 0x7f6d4819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d46ffd640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d48103b70 0x7f6d48199290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6d38006010 tx=0x7f6d380319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d4ca4e640 1 -- 172.21.15.78:0/3893392435 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38003bf0 con 0x7f6d48103b70 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d4ca4e640 1 -- 172.21.15.78:0/3893392435 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d38003d90 con 0x7f6d48103b70 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d4ca4e640 1 -- 172.21.15.78:0/3893392435 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3802fc20 con 0x7f6d48103b70 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d4da50640 1 -- 172.21.15.78:0/3893392435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4819f5d0 con 0x7f6d48103b70 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.324+0000 7f6d4da50640 1 -- 172.21.15.78:0/3893392435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d481a5cf0 con 0x7f6d48103b70 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.325+0000 7f6d4da50640 1 -- 172.21.15.78:0/3893392435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d48103f50 con 0x7f6d48103b70 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.327+0000 7f6d4ca4e640 1 -- 172.21.15.78:0/3893392435 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f6d380046a0 con 0x7f6d48103b70 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.327+0000 7f6d4ca4e640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d340b53c0 0x7f6d340b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.327+0000 7f6d467fc640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d340b53c0 0x7f6d340b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.327+0000 7f6d4ca4e640 1 -- 172.21.15.78:0/3893392435 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6d381073d0 con 0x7f6d48103b70 2024-09-11T00:57:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.328+0000 7f6d467fc640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d340b53c0 0x7f6d340b7880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d30005fd0 tx=0x7f6d30005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.330+0000 7f6d4ca4e640 1 -- 172.21.15.78:0/3893392435 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6d380ce080 con 0x7f6d48103b70 2024-09-11T00:57:24.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.507+0000 7f6d4da50640 1 -- 172.21.15.78:0/3893392435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f6d48107c00 con 0x7f6d48103b70 2024-09-11T00:57:24.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.508+0000 7f6d4ca4e640 1 -- 172.21.15.78:0/3893392435 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v55) ==== 86+0+133 (secure 0 0 0) 0x7f6d380cd980 con 0x7f6d48103b70 2024-09-11T00:57:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:57:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.510+0000 7f6d4da50640 1 -- 172.21.15.78:0/3893392435 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d340b53c0 msgr2=0x7f6d340b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.510+0000 7f6d4da50640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d340b53c0 0x7f6d340b7880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d30005fd0 tx=0x7f6d30005850 comp rx=0 tx=0).stop 2024-09-11T00:57:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.510+0000 7f6d4da50640 1 -- 172.21.15.78:0/3893392435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d48103b70 msgr2=0x7f6d48199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.510+0000 7f6d4da50640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d48103b70 0x7f6d48199290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6d38006010 tx=0x7f6d380319f0 comp rx=0 tx=0).stop 2024-09-11T00:57:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.510+0000 7f6d4da50640 1 -- 172.21.15.78:0/3893392435 shutdown_connections 2024-09-11T00:57:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.510+0000 7f6d4da50640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d340b53c0 0x7f6d340b7880 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.510+0000 7f6d4da50640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d4810d120 0x7f6d4819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.510+0000 7f6d4da50640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d48104570 0x7f6d481997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.510+0000 7f6d4da50640 1 --2- 172.21.15.78:0/3893392435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d48103b70 0x7f6d48199290 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.510+0000 7f6d4da50640 1 -- 172.21.15.78:0/3893392435 >> 172.21.15.78:0/3893392435 conn(0x7f6d480fd700 msgr2=0x7f6d480fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.510+0000 7f6d4da50640 1 -- 172.21.15.78:0/3893392435 shutdown_connections 2024-09-11T00:57:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:24.510+0000 7f6d4da50640 1 -- 172.21.15.78:0/3893392435 wait complete. 2024-09-11T00:57:24.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:24 smithi078 ceph-mon[31315]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 140 B/s, 1 objects/s recovering 2024-09-11T00:57:24.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:24 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3893392435' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-11T00:57:24.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:24 smithi110 ceph-mon[34712]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 140 B/s, 1 objects/s recovering 2024-09-11T00:57:24.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:24 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3893392435' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-11T00:57:24.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:24 smithi112 ceph-mon[35384]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 140 B/s, 1 objects/s recovering 2024-09-11T00:57:24.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:24 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3893392435' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-11T00:57:25.266 INFO:teuthology.orchestra.run.smithi078.stdout:[{"version":55,"timestamp":"2024-09-11T00:56:42.217565+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-11T00:57:25.267 INFO:tasks.ceph_manager:config epoch is 55 2024-09-11T00:57:25.267 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T00:57:25.268 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T00:57:25.268 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T00:57:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:26.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.979+0000 7f8654ba4640 1 -- 172.21.15.78:0/4280560761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8650108040 msgr2=0x7f865010e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:26.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.979+0000 7f8654ba4640 1 --2- 172.21.15.78:0/4280560761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8650108040 0x7f865010e840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8640009900 tx=0x7f864002a410 comp rx=0 tx=0).stop 2024-09-11T00:57:26.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.979+0000 7f8654ba4640 1 -- 172.21.15.78:0/4280560761 shutdown_connections 2024-09-11T00:57:26.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.979+0000 7f8654ba4640 1 --2- 172.21.15.78:0/4280560761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8650108040 0x7f865010e840 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:26.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.979+0000 7f8654ba4640 1 --2- 172.21.15.78:0/4280560761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86501076c0 0x7f8650107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:26.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.979+0000 7f8654ba4640 1 --2- 172.21.15.78:0/4280560761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8650101cb0 0x7f8650102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:26.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.979+0000 7f8654ba4640 1 -- 172.21.15.78:0/4280560761 >> 172.21.15.78:0/4280560761 conn(0x7f86500fd700 msgr2=0x7f86500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:26.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.979+0000 7f8654ba4640 1 -- 172.21.15.78:0/4280560761 shutdown_connections 2024-09-11T00:57:26.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.979+0000 7f8654ba4640 1 -- 172.21.15.78:0/4280560761 wait complete. 2024-09-11T00:57:26.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.980+0000 7f8654ba4640 1 Processor -- start 2024-09-11T00:57:26.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.980+0000 7f8654ba4640 1 -- start start 2024-09-11T00:57:26.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.980+0000 7f8654ba4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8650101cb0 0x7f86501992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:26.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.980+0000 7f8654ba4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86501076c0 0x7f86501997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:26.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.980+0000 7f8654ba4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8650108040 0x7f865019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:26.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.980+0000 7f8654ba4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8650112e20 con 0x7f8650101cb0 2024-09-11T00:57:26.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.980+0000 7f8654ba4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8650112ca0 con 0x7f8650108040 2024-09-11T00:57:26.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.980+0000 7f8654ba4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8650112fa0 con 0x7f86501076c0 2024-09-11T00:57:26.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.980+0000 7f864e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8650101cb0 0x7f86501992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:26.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.980+0000 7f864e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8650101cb0 0x7f86501992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56116/0 (socket says 172.21.15.78:56116) 2024-09-11T00:57:26.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.980+0000 7f864e59c640 1 -- 172.21.15.78:0/3907767197 learned_addr learned my addr 172.21.15.78:0/3907767197 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:26.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f864dd9b640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86501076c0 0x7f86501997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:26.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f864ed9d640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8650108040 0x7f865019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:26.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f864e59c640 1 -- 172.21.15.78:0/3907767197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86501076c0 msgr2=0x7f86501997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:26.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f864e59c640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86501076c0 0x7f86501997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:26.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f864e59c640 1 -- 172.21.15.78:0/3907767197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8650108040 msgr2=0x7f865019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f864e59c640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8650108040 0x7f865019eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f864e59c640 1 -- 172.21.15.78:0/3907767197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865019f360 con 0x7f8650101cb0 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f864dd9b640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86501076c0 0x7f86501997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f864e59c640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8650101cb0 0x7f86501992b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f864800eab0 tx=0x7f864800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f86477fe640 1 -- 172.21.15.78:0/3907767197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864800ce40 con 0x7f8650101cb0 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f86477fe640 1 -- 172.21.15.78:0/3907767197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8648011920 con 0x7f8650101cb0 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f86477fe640 1 -- 172.21.15.78:0/3907767197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648010a30 con 0x7f8650101cb0 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f8654ba4640 1 -- 172.21.15.78:0/3907767197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865019f650 con 0x7f8650101cb0 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.981+0000 7f8654ba4640 1 -- 172.21.15.78:0/3907767197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86501a5ee0 con 0x7f8650101cb0 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.982+0000 7f8654ba4640 1 -- 172.21.15.78:0/3907767197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8650102090 con 0x7f8650101cb0 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.983+0000 7f86477fe640 1 -- 172.21.15.78:0/3907767197 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f864800f040 con 0x7f8650101cb0 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.984+0000 7f86477fe640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f863c0b5440 0x7f863c0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.985+0000 7f864dd9b640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f863c0b5440 0x7f863c0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:26.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.985+0000 7f86477fe640 1 -- 172.21.15.78:0/3907767197 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f86480e61e0 con 0x7f8650101cb0 2024-09-11T00:57:26.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.985+0000 7f864dd9b640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f863c0b5440 0x7f863c0b7900 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8650106e00 tx=0x7f8638005cb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:26.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:25.988+0000 7f86477fe640 1 -- 172.21.15.78:0/3907767197 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8648016030 con 0x7f8650101cb0 2024-09-11T00:57:26.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.214+0000 7f8654ba4640 1 -- 172.21.15.78:0/3907767197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f865002e5c0 con 0x7f8650101cb0 2024-09-11T00:57:26.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.219+0000 7f86477fe640 1 -- 172.21.15.78:0/3907767197 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+294232 (secure 0 0 0) 0x7f86480aceb0 con 0x7f8650101cb0 2024-09-11T00:57:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:57:26.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.222+0000 7f8654ba4640 1 -- 172.21.15.78:0/3907767197 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f863c0b5440 msgr2=0x7f863c0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:26.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.222+0000 7f8654ba4640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f863c0b5440 0x7f863c0b7900 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8650106e00 tx=0x7f8638005cb0 comp rx=0 tx=0).stop 2024-09-11T00:57:26.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.222+0000 7f8654ba4640 1 -- 172.21.15.78:0/3907767197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8650101cb0 msgr2=0x7f86501992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:26.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.222+0000 7f8654ba4640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8650101cb0 0x7f86501992b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f864800eab0 tx=0x7f864800ef70 comp rx=0 tx=0).stop 2024-09-11T00:57:26.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.222+0000 7f8654ba4640 1 -- 172.21.15.78:0/3907767197 shutdown_connections 2024-09-11T00:57:26.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.222+0000 7f8654ba4640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f863c0b5440 0x7f863c0b7900 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:26.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.222+0000 7f8654ba4640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8650108040 0x7f865019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:26.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.222+0000 7f8654ba4640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86501076c0 0x7f86501997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:26.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.222+0000 7f8654ba4640 1 --2- 172.21.15.78:0/3907767197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8650101cb0 0x7f86501992b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:26.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.222+0000 7f8654ba4640 1 -- 172.21.15.78:0/3907767197 >> 172.21.15.78:0/3907767197 conn(0x7f86500fd700 msgr2=0x7f86500fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:26.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.222+0000 7f8654ba4640 1 -- 172.21.15.78:0/3907767197 shutdown_connections 2024-09-11T00:57:26.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:26.222+0000 7f8654ba4640 1 -- 172.21.15.78:0/3907767197 wait complete. 2024-09-11T00:57:26.938 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":21,"flags":0,"active_gid":14217,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":1082762376},{"type":"v1","addr":"172.21.15.78:6801","nonce":1082762376}]},"active_addr":"172.21.15.78:6801/1082762376","active_change":"2024-09-11T00:50:15.928190+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24148,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24170,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.78:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":4150137614}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":1183783243}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2925630667}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2743194845}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2994300835}]}]} 2024-09-11T00:57:26.948 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T00:57:26.948 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T00:57:26.949 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:57:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:27.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:27 smithi078 ceph-mon[31315]: pgmap v280: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.3 KiB/s wr, 10 op/s; 118 B/s, 1 objects/s recovering 2024-09-11T00:57:27.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:27 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3907767197' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-11T00:57:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:27 smithi110 ceph-mon[34712]: pgmap v280: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.3 KiB/s wr, 10 op/s; 118 B/s, 1 objects/s recovering 2024-09-11T00:57:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:27 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3907767197' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-11T00:57:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:27 smithi112 ceph-mon[35384]: pgmap v280: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.3 KiB/s wr, 10 op/s; 118 B/s, 1 objects/s recovering 2024-09-11T00:57:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:27 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3907767197' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-11T00:57:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.698+0000 7f5b53a98640 1 -- 172.21.15.78:0/576680363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c1076b0 msgr2=0x7f5b4c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.698+0000 7f5b53a98640 1 --2- 172.21.15.78:0/576680363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c1076b0 0x7f5b4c107af0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009900 tx=0x7f5b3c02a410 comp rx=0 tx=0).stop 2024-09-11T00:57:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 -- 172.21.15.78:0/576680363 shutdown_connections 2024-09-11T00:57:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 --2- 172.21.15.78:0/576680363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b4c108030 0x7f5b4c10e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 --2- 172.21.15.78:0/576680363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c1076b0 0x7f5b4c107af0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 --2- 172.21.15.78:0/576680363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b4c101ca0 0x7f5b4c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 -- 172.21.15.78:0/576680363 >> 172.21.15.78:0/576680363 conn(0x7f5b4c0fd6f0 msgr2=0x7f5b4c0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 -- 172.21.15.78:0/576680363 shutdown_connections 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 -- 172.21.15.78:0/576680363 wait complete. 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 Processor -- start 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 -- start start 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c101ca0 0x7f5b4c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b4c1076b0 0x7f5b4c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b4c108030 0x7f5b4c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c111d40 con 0x7f5b4c101ca0 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c111bc0 con 0x7f5b4c1076b0 2024-09-11T00:57:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.699+0000 7f5b53a98640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c111ec0 con 0x7f5b4c108030 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b51834640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c101ca0 0x7f5b4c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b51834640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c101ca0 0x7f5b4c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56132/0 (socket says 172.21.15.78:56132) 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b51834640 1 -- 172.21.15.78:0/2001692055 learned_addr learned my addr 172.21.15.78:0/2001692055 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b52035640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b4c108030 0x7f5b4c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b51834640 1 -- 172.21.15.78:0/2001692055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b4c108030 msgr2=0x7f5b4c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b51834640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b4c108030 0x7f5b4c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b51834640 1 -- 172.21.15.78:0/2001692055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b4c1076b0 msgr2=0x7f5b4c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b51834640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b4c1076b0 0x7f5b4c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b51834640 1 -- 172.21.15.78:0/2001692055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c19f320 con 0x7f5b4c101ca0 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b51834640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c101ca0 0x7f5b4c199270 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800e9f0 tx=0x7f5b4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b42ffd640 1 -- 172.21.15.78:0/2001692055 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800cd80 con 0x7f5b4c101ca0 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b53a98640 1 -- 172.21.15.78:0/2001692055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4c19f610 con 0x7f5b4c101ca0 2024-09-11T00:57:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b42ffd640 1 -- 172.21.15.78:0/2001692055 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b480108e0 con 0x7f5b4c101ca0 2024-09-11T00:57:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b42ffd640 1 -- 172.21.15.78:0/2001692055 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48018980 con 0x7f5b4c101ca0 2024-09-11T00:57:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.700+0000 7f5b53a98640 1 -- 172.21.15.78:0/2001692055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4c1a5cd0 con 0x7f5b4c101ca0 2024-09-11T00:57:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.701+0000 7f5b53a98640 1 -- 172.21.15.78:0/2001692055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b4c102080 con 0x7f5b4c101ca0 2024-09-11T00:57:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.703+0000 7f5b42ffd640 1 -- 172.21.15.78:0/2001692055 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f5b48010430 con 0x7f5b4c101ca0 2024-09-11T00:57:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.704+0000 7f5b42ffd640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5b380b53c0 0x7f5b380b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.704+0000 7f5b51033640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5b380b53c0 0x7f5b380b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.704+0000 7f5b42ffd640 1 -- 172.21.15.78:0/2001692055 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5b480e67d0 con 0x7f5b4c101ca0 2024-09-11T00:57:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.704+0000 7f5b51033640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5b380b53c0 0x7f5b380b7880 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c02a930 tx=0x7f5b3c031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.707+0000 7f5b42ffd640 1 -- 172.21.15.78:0/2001692055 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5b480ad480 con 0x7f5b4c101ca0 2024-09-11T00:57:27.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.885+0000 7f5b53a98640 1 -- 172.21.15.78:0/2001692055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b4c02e5c0 con 0x7f5b4c101ca0 2024-09-11T00:57:27.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.887+0000 7f5b42ffd640 1 -- 172.21.15.78:0/2001692055 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22975 (secure 0 0 0) 0x7f5b480acd80 con 0x7f5b4c101ca0 2024-09-11T00:57:27.890 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:57:27.890 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","created":"2024-09-11T00:48:27.652512+0000","modified":"2024-09-11T00:57:17.572848+0000","last_up_change":"2024-09-11T00:55:59.071507+0000","last_in_change":"2024-09-11T00:55:35.815150+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:52:10.229887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:56:49.939136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:56:51.703446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beefc6d9-f74c-4200-96f2-ff697ad0d7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6803","nonce":1424639053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6805","nonce":1424639053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6809","nonce":1424639053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6807","nonce":1424639053}]},"public_addr":"172.21.15.78:6803/1424639053","cluster_addr":"172.21.15.78:6805/1424639053","heartbeat_back_addr":"172.21.15.78:6809/1424639053","heartbeat_front_addr":"172.21.15.78:6807/1424639053","state":["exists","up"]},{"osd":1,"uuid":"d742e767-d673-4bb9-b624-8bc969f5ceb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6801","nonce":580939385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6803","nonce":580939385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6807","nonce":580939385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6805","nonce":580939385}]},"public_addr":"172.21.15.110:6801/580939385","cluster_addr":"172.21.15.110:6803/580939385","heartbeat_back_addr":"172.21.15.110:6807/580939385","heartbeat_front_addr":"172.21.15.110:6805/580939385","state":["exists","up"]},{"osd":2,"uuid":"b0106a5b-1465-4a50-96dd-934dbae49a4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6801","nonce":2983155763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6803","nonce":2983155763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6807","nonce":2983155763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6805","nonce":2983155763}]},"public_addr":"172.21.15.112:6801/2983155763","cluster_addr":"172.21.15.112:6803/2983155763","heartbeat_back_addr":"172.21.15.112:6807/2983155763","heartbeat_front_addr":"172.21.15.112:6805/2983155763","state":["exists","up"]},{"osd":3,"uuid":"3270362b-3d61-414a-a3d9-f2416ff8b870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6811","nonce":2906086464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6813","nonce":2906086464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6817","nonce":2906086464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6815","nonce":2906086464}]},"public_addr":"172.21.15.78:6811/2906086464","cluster_addr":"172.21.15.78:6813/2906086464","heartbeat_back_addr":"172.21.15.78:6817/2906086464","heartbeat_front_addr":"172.21.15.78:6815/2906086464","state":["exists","up"]},{"osd":4,"uuid":"e47746ec-9443-4004-abfa-17c1ed73b982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6809","nonce":477287435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6811","nonce":477287435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6815","nonce":477287435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6813","nonce":477287435}]},"public_addr":"172.21.15.110:6809/477287435","cluster_addr":"172.21.15.110:6811/477287435","heartbeat_back_addr":"172.21.15.110:6815/477287435","heartbeat_front_addr":"172.21.15.110:6813/477287435","state":["exists","up"]},{"osd":5,"uuid":"f8cedb01-310d-442d-a588-33197d32e208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6809","nonce":3557095096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6811","nonce":3557095096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6815","nonce":3557095096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6813","nonce":3557095096}]},"public_addr":"172.21.15.112:6809/3557095096","cluster_addr":"172.21.15.112:6811/3557095096","heartbeat_back_addr":"172.21.15.112:6815/3557095096","heartbeat_front_addr":"172.21.15.112:6813/3557095096","state":["exists","up"]},{"osd":6,"uuid":"d508a037-8f56-436b-b7bf-9a1c69f6d9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6819","nonce":1267910775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6821","nonce":1267910775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6825","nonce":1267910775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6823","nonce":1267910775}]},"public_addr":"172.21.15.78:6819/1267910775","cluster_addr":"172.21.15.78:6821/1267910775","heartbeat_back_addr":"172.21.15.78:6825/1267910775","heartbeat_front_addr":"172.21.15.78:6823/1267910775","state":["exists","up"]},{"osd":7,"uuid":"e8842047-09a6-48b8-8d0a-2d54e7f89063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6817","nonce":1556494594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6819","nonce":1556494594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6823","nonce":1556494594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6821","nonce":1556494594}]},"public_addr":"172.21.15.110:6817/1556494594","cluster_addr":"172.21.15.110:6819/1556494594","heartbeat_back_addr":"172.21.15.110:6823/1556494594","heartbeat_front_addr":"172.21.15.110:6821/1556494594","state":["exists","up"]},{"osd":8,"uuid":"a3367da8-375a-4a46-a952-867d4b72ae2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6817","nonce":2481321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6819","nonce":2481321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6823","nonce":2481321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6821","nonce":2481321}]},"public_addr":"172.21.15.112:6817/2481321","cluster_addr":"172.21.15.112:6819/2481321","heartbeat_back_addr":"172.21.15.112:6823/2481321","heartbeat_front_addr":"172.21.15.112:6821/2481321","state":["exists","up"]},{"osd":9,"uuid":"d1825764-cefc-495b-99da-8ba429df0937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6827","nonce":197048350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6829","nonce":197048350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6833","nonce":197048350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6831","nonce":197048350}]},"public_addr":"172.21.15.78:6827/197048350","cluster_addr":"172.21.15.78:6829/197048350","heartbeat_back_addr":"172.21.15.78:6833/197048350","heartbeat_front_addr":"172.21.15.78:6831/197048350","state":["exists","up"]},{"osd":10,"uuid":"e1cfff7a-8916-4845-a482-d79fa275da30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6825","nonce":2101123168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6827","nonce":2101123168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6831","nonce":2101123168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6829","nonce":2101123168}]},"public_addr":"172.21.15.110:6825/2101123168","cluster_addr":"172.21.15.110:6827/2101123168","heartbeat_back_addr":"172.21.15.110:6831/2101123168","heartbeat_front_addr":"172.21.15.110:6829/2101123168","state":["exists","up"]},{"osd":11,"uuid":"fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6825","nonce":1526310654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6827","nonce":1526310654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6831","nonce":1526310654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6829","nonce":1526310654}]},"public_addr":"172.21.15.112:6825/1526310654","cluster_addr":"172.21.15.112:6827/1526310654","heartbeat_back_addr":"172.21.15.112:6831/1526310654","heartbeat_front_addr":"172.21.15.112:6829/1526310654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:17.334199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:41.400872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:08.205144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:33.550527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:58.394116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:24.699607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:50.450193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:16.223363+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:40.947727+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:06.845451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:32.410035+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:57.744364+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/2390785703":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:0/1038951931":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/391908936":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2706514734":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/2357025243":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6800/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2795778473":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2747741101":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6801/1289343426":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/3558896007":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/438261739":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/1289343426":"2024-09-12T00:50:15.928079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:57:27.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.889+0000 7f5b53a98640 1 -- 172.21.15.78:0/2001692055 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5b380b53c0 msgr2=0x7f5b380b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:27.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.889+0000 7f5b53a98640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5b380b53c0 0x7f5b380b7880 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c02a930 tx=0x7f5b3c031400 comp rx=0 tx=0).stop 2024-09-11T00:57:27.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.889+0000 7f5b53a98640 1 -- 172.21.15.78:0/2001692055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c101ca0 msgr2=0x7f5b4c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:27.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.889+0000 7f5b53a98640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c101ca0 0x7f5b4c199270 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800e9f0 tx=0x7f5b4800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:57:27.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.889+0000 7f5b53a98640 1 -- 172.21.15.78:0/2001692055 shutdown_connections 2024-09-11T00:57:27.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.889+0000 7f5b53a98640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5b380b53c0 0x7f5b380b7880 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:27.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.889+0000 7f5b53a98640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b4c108030 0x7f5b4c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:27.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.889+0000 7f5b53a98640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b4c1076b0 0x7f5b4c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:27.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.889+0000 7f5b53a98640 1 --2- 172.21.15.78:0/2001692055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c101ca0 0x7f5b4c199270 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:27.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.889+0000 7f5b53a98640 1 -- 172.21.15.78:0/2001692055 >> 172.21.15.78:0/2001692055 conn(0x7f5b4c0fd6f0 msgr2=0x7f5b4c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:27.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.889+0000 7f5b53a98640 1 -- 172.21.15.78:0/2001692055 shutdown_connections 2024-09-11T00:57:27.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:27.889+0000 7f5b53a98640 1 -- 172.21.15.78:0/2001692055 wait complete. 2024-09-11T00:57:28.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:28 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2001692055' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:57:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:28 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2001692055' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:57:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:28 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2001692055' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:57:28.646 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T00:57:28.646 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:57:28.896 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:29.313 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:29 smithi078 ceph-mon[31315]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.1 KiB/s wr, 10 op/s; 107 B/s, 0 objects/s recovering 2024-09-11T00:57:29.357 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:29 smithi110 ceph-mon[34712]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.1 KiB/s wr, 10 op/s; 107 B/s, 0 objects/s recovering 2024-09-11T00:57:29.375 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:29 smithi112 ceph-mon[35384]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.1 KiB/s wr, 10 op/s; 107 B/s, 0 objects/s recovering 2024-09-11T00:57:29.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.388+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3459339668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb548102dc0 msgr2=0x7fb5481031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:29.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.388+0000 7fb54ea3d640 1 --2- 172.21.15.78:0/3459339668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb548102dc0 0x7fb5481031a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb538009900 tx=0x7fb53802a410 comp rx=0 tx=0).stop 2024-09-11T00:57:29.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.388+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3459339668 shutdown_connections 2024-09-11T00:57:29.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.388+0000 7fb54ea3d640 1 --2- 172.21.15.78:0/3459339668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb548109150 0x7fb54810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:29.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.388+0000 7fb54ea3d640 1 --2- 172.21.15.78:0/3459339668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5481087d0 0x7fb548108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:29.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.388+0000 7fb54ea3d640 1 --2- 172.21.15.78:0/3459339668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb548102dc0 0x7fb5481031a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:29.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.388+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3459339668 >> 172.21.15.78:0/3459339668 conn(0x7fb5480fe830 msgr2=0x7fb548100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:29.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.388+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3459339668 shutdown_connections 2024-09-11T00:57:29.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.388+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3459339668 wait complete. 2024-09-11T00:57:29.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb54ea3d640 1 Processor -- start 2024-09-11T00:57:29.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb54ea3d640 1 -- start start 2024-09-11T00:57:29.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb54ea3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb548102dc0 0x7fb548199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:29.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb54ea3d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5481087d0 0x7fb548199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:29.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb54ea3d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb548109150 0x7fb54819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:29.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb54ea3d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb548112e10 con 0x7fb548102dc0 2024-09-11T00:57:29.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb54ea3d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb548112c90 con 0x7fb548109150 2024-09-11T00:57:29.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb54ea3d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb548112f90 con 0x7fb5481087d0 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb547fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb548102dc0 0x7fb548199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb547fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb548102dc0 0x7fb548199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56154/0 (socket says 172.21.15.78:56154) 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb547fff640 1 -- 172.21.15.78:0/3565361708 learned_addr learned my addr 172.21.15.78:0/3565361708 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb54cfda640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb548109150 0x7fb54819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.389+0000 7fb5477fe640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5481087d0 0x7fb548199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.390+0000 7fb547fff640 1 -- 172.21.15.78:0/3565361708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5481087d0 msgr2=0x7fb548199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.390+0000 7fb547fff640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5481087d0 0x7fb548199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.390+0000 7fb547fff640 1 -- 172.21.15.78:0/3565361708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb548109150 msgr2=0x7fb54819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.390+0000 7fb547fff640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb548109150 0x7fb54819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.390+0000 7fb547fff640 1 -- 172.21.15.78:0/3565361708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54819f3d0 con 0x7fb548102dc0 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.390+0000 7fb547fff640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb548102dc0 0x7fb548199320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb538006010 tx=0x7fb5380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.390+0000 7fb5457fa640 1 -- 172.21.15.78:0/3565361708 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53803f030 con 0x7fb548102dc0 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.390+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3565361708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54819f660 con 0x7fb548102dc0 2024-09-11T00:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.390+0000 7fb5457fa640 1 -- 172.21.15.78:0/3565361708 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb538003800 con 0x7fb548102dc0 2024-09-11T00:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.390+0000 7fb5457fa640 1 -- 172.21.15.78:0/3565361708 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538035070 con 0x7fb548102dc0 2024-09-11T00:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.390+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3565361708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5481a5d80 con 0x7fb548102dc0 2024-09-11T00:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.391+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3565361708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5481031a0 con 0x7fb548102dc0 2024-09-11T00:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.392+0000 7fb5457fa640 1 -- 172.21.15.78:0/3565361708 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fb538031b10 con 0x7fb548102dc0 2024-09-11T00:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.393+0000 7fb5457fa640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5300b53c0 0x7fb5300b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.393+0000 7fb5477fe640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5300b53c0 0x7fb5300b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.393+0000 7fb5457fa640 1 -- 172.21.15.78:0/3565361708 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb538106950 con 0x7fb548102dc0 2024-09-11T00:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.394+0000 7fb5477fe640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5300b53c0 0x7fb5300b7880 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb5481020b0 tx=0x7fb53c009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.397+0000 7fb5457fa640 1 -- 172.21.15.78:0/3565361708 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb5380cd5a0 con 0x7fb548102dc0 2024-09-11T00:57:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.574+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3565361708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb54810af30 con 0x7fb548102dc0 2024-09-11T00:57:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.575+0000 7fb5457fa640 1 -- 172.21.15.78:0/3565361708 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22975 (secure 0 0 0) 0x7fb5380ccea0 con 0x7fb548102dc0 2024-09-11T00:57:29.578 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:57:29.578 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","created":"2024-09-11T00:48:27.652512+0000","modified":"2024-09-11T00:57:17.572848+0000","last_up_change":"2024-09-11T00:55:59.071507+0000","last_in_change":"2024-09-11T00:55:35.815150+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:52:10.229887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:56:49.939136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:56:51.703446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beefc6d9-f74c-4200-96f2-ff697ad0d7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6803","nonce":1424639053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6805","nonce":1424639053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6809","nonce":1424639053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6807","nonce":1424639053}]},"public_addr":"172.21.15.78:6803/1424639053","cluster_addr":"172.21.15.78:6805/1424639053","heartbeat_back_addr":"172.21.15.78:6809/1424639053","heartbeat_front_addr":"172.21.15.78:6807/1424639053","state":["exists","up"]},{"osd":1,"uuid":"d742e767-d673-4bb9-b624-8bc969f5ceb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6801","nonce":580939385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6803","nonce":580939385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6807","nonce":580939385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6805","nonce":580939385}]},"public_addr":"172.21.15.110:6801/580939385","cluster_addr":"172.21.15.110:6803/580939385","heartbeat_back_addr":"172.21.15.110:6807/580939385","heartbeat_front_addr":"172.21.15.110:6805/580939385","state":["exists","up"]},{"osd":2,"uuid":"b0106a5b-1465-4a50-96dd-934dbae49a4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6801","nonce":2983155763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6803","nonce":2983155763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6807","nonce":2983155763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6805","nonce":2983155763}]},"public_addr":"172.21.15.112:6801/2983155763","cluster_addr":"172.21.15.112:6803/2983155763","heartbeat_back_addr":"172.21.15.112:6807/2983155763","heartbeat_front_addr":"172.21.15.112:6805/2983155763","state":["exists","up"]},{"osd":3,"uuid":"3270362b-3d61-414a-a3d9-f2416ff8b870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6811","nonce":2906086464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6813","nonce":2906086464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6817","nonce":2906086464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6815","nonce":2906086464}]},"public_addr":"172.21.15.78:6811/2906086464","cluster_addr":"172.21.15.78:6813/2906086464","heartbeat_back_addr":"172.21.15.78:6817/2906086464","heartbeat_front_addr":"172.21.15.78:6815/2906086464","state":["exists","up"]},{"osd":4,"uuid":"e47746ec-9443-4004-abfa-17c1ed73b982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6809","nonce":477287435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6811","nonce":477287435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6815","nonce":477287435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6813","nonce":477287435}]},"public_addr":"172.21.15.110:6809/477287435","cluster_addr":"172.21.15.110:6811/477287435","heartbeat_back_addr":"172.21.15.110:6815/477287435","heartbeat_front_addr":"172.21.15.110:6813/477287435","state":["exists","up"]},{"osd":5,"uuid":"f8cedb01-310d-442d-a588-33197d32e208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6809","nonce":3557095096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6811","nonce":3557095096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6815","nonce":3557095096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6813","nonce":3557095096}]},"public_addr":"172.21.15.112:6809/3557095096","cluster_addr":"172.21.15.112:6811/3557095096","heartbeat_back_addr":"172.21.15.112:6815/3557095096","heartbeat_front_addr":"172.21.15.112:6813/3557095096","state":["exists","up"]},{"osd":6,"uuid":"d508a037-8f56-436b-b7bf-9a1c69f6d9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6819","nonce":1267910775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6821","nonce":1267910775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6825","nonce":1267910775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6823","nonce":1267910775}]},"public_addr":"172.21.15.78:6819/1267910775","cluster_addr":"172.21.15.78:6821/1267910775","heartbeat_back_addr":"172.21.15.78:6825/1267910775","heartbeat_front_addr":"172.21.15.78:6823/1267910775","state":["exists","up"]},{"osd":7,"uuid":"e8842047-09a6-48b8-8d0a-2d54e7f89063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6817","nonce":1556494594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6819","nonce":1556494594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6823","nonce":1556494594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6821","nonce":1556494594}]},"public_addr":"172.21.15.110:6817/1556494594","cluster_addr":"172.21.15.110:6819/1556494594","heartbeat_back_addr":"172.21.15.110:6823/1556494594","heartbeat_front_addr":"172.21.15.110:6821/1556494594","state":["exists","up"]},{"osd":8,"uuid":"a3367da8-375a-4a46-a952-867d4b72ae2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6817","nonce":2481321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6819","nonce":2481321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6823","nonce":2481321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6821","nonce":2481321}]},"public_addr":"172.21.15.112:6817/2481321","cluster_addr":"172.21.15.112:6819/2481321","heartbeat_back_addr":"172.21.15.112:6823/2481321","heartbeat_front_addr":"172.21.15.112:6821/2481321","state":["exists","up"]},{"osd":9,"uuid":"d1825764-cefc-495b-99da-8ba429df0937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6827","nonce":197048350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6829","nonce":197048350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6833","nonce":197048350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6831","nonce":197048350}]},"public_addr":"172.21.15.78:6827/197048350","cluster_addr":"172.21.15.78:6829/197048350","heartbeat_back_addr":"172.21.15.78:6833/197048350","heartbeat_front_addr":"172.21.15.78:6831/197048350","state":["exists","up"]},{"osd":10,"uuid":"e1cfff7a-8916-4845-a482-d79fa275da30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6825","nonce":2101123168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6827","nonce":2101123168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6831","nonce":2101123168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6829","nonce":2101123168}]},"public_addr":"172.21.15.110:6825/2101123168","cluster_addr":"172.21.15.110:6827/2101123168","heartbeat_back_addr":"172.21.15.110:6831/2101123168","heartbeat_front_addr":"172.21.15.110:6829/2101123168","state":["exists","up"]},{"osd":11,"uuid":"fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6825","nonce":1526310654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6827","nonce":1526310654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6831","nonce":1526310654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6829","nonce":1526310654}]},"public_addr":"172.21.15.112:6825/1526310654","cluster_addr":"172.21.15.112:6827/1526310654","heartbeat_back_addr":"172.21.15.112:6831/1526310654","heartbeat_front_addr":"172.21.15.112:6829/1526310654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:17.334199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:41.400872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:08.205144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:33.550527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:58.394116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:24.699607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:50.450193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:16.223363+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:40.947727+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:06.845451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:32.410035+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:57.744364+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/2390785703":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:0/1038951931":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/391908936":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2706514734":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/2357025243":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6800/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2795778473":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2747741101":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6801/1289343426":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/3558896007":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/438261739":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/1289343426":"2024-09-12T00:50:15.928079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:57:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.577+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3565361708 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5300b53c0 msgr2=0x7fb5300b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.577+0000 7fb54ea3d640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5300b53c0 0x7fb5300b7880 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb5481020b0 tx=0x7fb53c009110 comp rx=0 tx=0).stop 2024-09-11T00:57:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.577+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3565361708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb548102dc0 msgr2=0x7fb548199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.577+0000 7fb54ea3d640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb548102dc0 0x7fb548199320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb538006010 tx=0x7fb5380042f0 comp rx=0 tx=0).stop 2024-09-11T00:57:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.577+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3565361708 shutdown_connections 2024-09-11T00:57:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.577+0000 7fb54ea3d640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5300b53c0 0x7fb5300b7880 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.577+0000 7fb54ea3d640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb548109150 0x7fb54819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.577+0000 7fb54ea3d640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5481087d0 0x7fb548199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:29.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.577+0000 7fb54ea3d640 1 --2- 172.21.15.78:0/3565361708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb548102dc0 0x7fb548199320 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:29.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.577+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3565361708 >> 172.21.15.78:0/3565361708 conn(0x7fb5480fe830 msgr2=0x7fb548100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:29.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.577+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3565361708 shutdown_connections 2024-09-11T00:57:29.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:29.577+0000 7fb54ea3d640 1 -- 172.21.15.78:0/3565361708 wait complete. 2024-09-11T00:57:30.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:30 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3565361708' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:57:30.372 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T00:57:30.372 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T00:57:30.373 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T00:57:30.373 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T00:57:30.373 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T00:57:30.374 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T00:57:30.374 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T00:57:30.375 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T00:57:30.375 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-11T00:57:30.376 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-11T00:57:30.376 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-11T00:57:30.377 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-11T00:57:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:30 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3565361708' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:57:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:30 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3565361708' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:57:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:31.358 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:31 smithi078 ceph-mon[31315]: pgmap v282: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s; 93 B/s, 0 objects/s recovering 2024-09-11T00:57:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:31.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:31 smithi110 ceph-mon[34712]: pgmap v282: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s; 93 B/s, 0 objects/s recovering 2024-09-11T00:57:31.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:31 smithi112 ceph-mon[35384]: pgmap v282: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s; 93 B/s, 0 objects/s recovering 2024-09-11T00:57:31.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.576+0000 7fab47577640 1 -- 172.21.15.78:0/586223345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab40109150 msgr2=0x7fab4010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:31.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.576+0000 7fab47577640 1 --2- 172.21.15.78:0/586223345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab40109150 0x7fab4010f950 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fab38009900 tx=0x7fab3802a410 comp rx=0 tx=0).stop 2024-09-11T00:57:31.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.576+0000 7fab47577640 1 -- 172.21.15.78:0/586223345 shutdown_connections 2024-09-11T00:57:31.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.576+0000 7fab47577640 1 --2- 172.21.15.78:0/586223345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab40109150 0x7fab4010f950 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:31.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.576+0000 7fab47577640 1 --2- 172.21.15.78:0/586223345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab401087d0 0x7fab40108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:31.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.576+0000 7fab47577640 1 --2- 172.21.15.78:0/586223345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab40102dc0 0x7fab401031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:31.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.576+0000 7fab47577640 1 -- 172.21.15.78:0/586223345 >> 172.21.15.78:0/586223345 conn(0x7fab400fe830 msgr2=0x7fab40100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:31.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.576+0000 7fab47577640 1 -- 172.21.15.78:0/586223345 shutdown_connections 2024-09-11T00:57:31.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.576+0000 7fab47577640 1 -- 172.21.15.78:0/586223345 wait complete. 2024-09-11T00:57:31.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab47577640 1 Processor -- start 2024-09-11T00:57:31.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab47577640 1 -- start start 2024-09-11T00:57:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab47577640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab40102dc0 0x7fab40199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab47577640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab401087d0 0x7fab401997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab47577640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab40109150 0x7fab4019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab47577640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab40112e10 con 0x7fab401087d0 2024-09-11T00:57:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab47577640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab40112c90 con 0x7fab40109150 2024-09-11T00:57:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab47577640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab40112f90 con 0x7fab40102dc0 2024-09-11T00:57:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab44b12640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab401087d0 0x7fab401997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab44b12640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab401087d0 0x7fab401997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56172/0 (socket says 172.21.15.78:56172) 2024-09-11T00:57:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab44b12640 1 -- 172.21.15.78:0/4173297724 learned_addr learned my addr 172.21.15.78:0/4173297724 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab45313640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab40102dc0 0x7fab40199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab44b12640 1 -- 172.21.15.78:0/4173297724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab40102dc0 msgr2=0x7fab40199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab44b12640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab40102dc0 0x7fab40199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab44b12640 1 -- 172.21.15.78:0/4173297724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab40109150 msgr2=0x7fab4019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab45b14640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab40109150 0x7fab4019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab44b12640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab40109150 0x7fab4019eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab44b12640 1 -- 172.21.15.78:0/4173297724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4019f330 con 0x7fab401087d0 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.577+0000 7fab45313640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab40102dc0 0x7fab40199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.578+0000 7fab44b12640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab401087d0 0x7fab401997c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab3000d950 tx=0x7fab3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.578+0000 7fab367fc640 1 -- 172.21.15.78:0/4173297724 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30017070 con 0x7fab401087d0 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.578+0000 7fab47577640 1 -- 172.21.15.78:0/4173297724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4019f620 con 0x7fab401087d0 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.578+0000 7fab367fc640 1 -- 172.21.15.78:0/4173297724 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab30009c30 con 0x7fab401087d0 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.578+0000 7fab367fc640 1 -- 172.21.15.78:0/4173297724 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3000f620 con 0x7fab401087d0 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.578+0000 7fab47577640 1 -- 172.21.15.78:0/4173297724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab401a5ce0 con 0x7fab401087d0 2024-09-11T00:57:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.579+0000 7fab47577640 1 -- 172.21.15.78:0/4173297724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab400638e0 con 0x7fab401087d0 2024-09-11T00:57:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.580+0000 7fab367fc640 1 -- 172.21.15.78:0/4173297724 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fab30010460 con 0x7fab401087d0 2024-09-11T00:57:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.581+0000 7fab367fc640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab2c0b53c0 0x7fab2c0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.581+0000 7fab45313640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab2c0b53c0 0x7fab2c0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.581+0000 7fab367fc640 1 -- 172.21.15.78:0/4173297724 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fab300e6010 con 0x7fab401087d0 2024-09-11T00:57:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.581+0000 7fab367fc640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464] conn(0x7fab2c0bf060 0x7fab2c0c1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.581+0000 7fab367fc640 1 -- 172.21.15.78:0/4173297724 --> [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab30022e70 con 0x7fab2c0bf060 2024-09-11T00:57:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.581+0000 7fab367fc640 1 -- 172.21.15.78:0/4173297724 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fab300e6460 con 0x7fab401087d0 2024-09-11T00:57:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.581+0000 7fab45b14640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464] conn(0x7fab2c0bf060 0x7fab2c0c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.581+0000 7fab45313640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab2c0b53c0 0x7fab2c0b7880 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fab3c005fd0 tx=0x7fab3c0093e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.581+0000 7fab45b14640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464] conn(0x7fab2c0bf060 0x7fab2c0c1500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.582+0000 7fab367fc640 1 -- 172.21.15.78:0/4173297724 <== osd.3 v2:172.21.15.78:6810/2906086464 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab30022e70 con 0x7fab2c0bf060 2024-09-11T00:57:31.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.599+0000 7fab47577640 1 -- 172.21.15.78:0/4173297724 --> [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fab40063ce0 con 0x7fab2c0bf060 2024-09-11T00:57:31.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.599+0000 7fab367fc640 1 -- 172.21.15.78:0/4173297724 <== osd.3 v2:172.21.15.78:6810/2906086464 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fab40063ce0 con 0x7fab2c0bf060 2024-09-11T00:57:31.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.600+0000 7fab47577640 1 -- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464] conn(0x7fab2c0bf060 msgr2=0x7fab2c0c1500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:31.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.600+0000 7fab47577640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464] conn(0x7fab2c0bf060 0x7fab2c0c1500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:31.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.600+0000 7fab47577640 1 -- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab2c0b53c0 msgr2=0x7fab2c0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:31.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.600+0000 7fab47577640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab2c0b53c0 0x7fab2c0b7880 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fab3c005fd0 tx=0x7fab3c0093e0 comp rx=0 tx=0).stop 2024-09-11T00:57:31.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.600+0000 7fab47577640 1 -- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab401087d0 msgr2=0x7fab401997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:31.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.600+0000 7fab47577640 1 --2- 172.21.15.78:0/4173297724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab401087d0 0x7fab401997c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab3000d950 tx=0x7fab3000de10 comp rx=0 tx=0).stop 2024-09-11T00:57:31.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.600+0000 7fab45b14640 1 -- 172.21.15.78:0/4173297724 reap_dead start 2024-09-11T00:57:31.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.600+0000 7fab47577640 1 -- 172.21.15.78:0/4173297724 shutdown_connections 2024-09-11T00:57:31.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.600+0000 7fab47577640 1 -- 172.21.15.78:0/4173297724 >> 172.21.15.78:0/4173297724 conn(0x7fab400fe830 msgr2=0x7fab40100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:31.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.600+0000 7fab47577640 1 -- 172.21.15.78:0/4173297724 shutdown_connections 2024-09-11T00:57:31.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:31.600+0000 7fab47577640 1 -- 172.21.15.78:0/4173297724 wait complete. 2024-09-11T00:57:32.500 INFO:teuthology.orchestra.run.smithi078.stdout:107374182462 2024-09-11T00:57:32.501 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T00:57:32.523 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:32.530 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:32.531 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:32.531 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:32.534 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:32.548 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:33.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.308+0000 7f7150e2a640 1 -- 172.21.15.78:0/4175961767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c1076c0 msgr2=0x7f714c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:33.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.308+0000 7f7150e2a640 1 --2- 172.21.15.78:0/4175961767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c1076c0 0x7f714c107b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7140009900 tx=0x7f714002a410 comp rx=0 tx=0).stop 2024-09-11T00:57:33.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.308+0000 7f7150e2a640 1 -- 172.21.15.78:0/4175961767 shutdown_connections 2024-09-11T00:57:33.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.308+0000 7f7150e2a640 1 --2- 172.21.15.78:0/4175961767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f714c108040 0x7f714c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:33.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.308+0000 7f7150e2a640 1 --2- 172.21.15.78:0/4175961767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c1076c0 0x7f714c107b00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:33.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.308+0000 7f7150e2a640 1 --2- 172.21.15.78:0/4175961767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c101cb0 0x7f714c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:33.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.308+0000 7f7150e2a640 1 -- 172.21.15.78:0/4175961767 >> 172.21.15.78:0/4175961767 conn(0x7f714c0fd700 msgr2=0x7f714c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:33.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.308+0000 7f7150e2a640 1 -- 172.21.15.78:0/4175961767 shutdown_connections 2024-09-11T00:57:33.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.308+0000 7f7150e2a640 1 -- 172.21.15.78:0/4175961767 wait complete. 2024-09-11T00:57:33.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f7150e2a640 1 Processor -- start 2024-09-11T00:57:33.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f7150e2a640 1 -- start start 2024-09-11T00:57:33.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f7150e2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c101cb0 0x7f714c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:33.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f7150e2a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f714c1076c0 0x7f714c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f7150e2a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c108040 0x7f714c19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f7150e2a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f714c111d00 con 0x7f714c101cb0 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f7150e2a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f714c111b80 con 0x7f714c108040 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f7150e2a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f714c111e80 con 0x7f714c1076c0 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f714a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c101cb0 0x7f714c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f714a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c101cb0 0x7f714c199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45656/0 (socket says 172.21.15.78:45656) 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f714a59c640 1 -- 172.21.15.78:0/250217783 learned_addr learned my addr 172.21.15.78:0/250217783 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f714a59c640 1 -- 172.21.15.78:0/250217783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f714c1076c0 msgr2=0x7f714c1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f714a59c640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f714c1076c0 0x7f714c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f714a59c640 1 -- 172.21.15.78:0/250217783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c108040 msgr2=0x7f714c19ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f714ad9d640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c108040 0x7f714c19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f714a59c640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c108040 0x7f714c19ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f714a59c640 1 -- 172.21.15.78:0/250217783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c19f410 con 0x7f714c101cb0 2024-09-11T00:57:33.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f714ad9d640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c108040 0x7f714c19ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:57:33.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.309+0000 7f714a59c640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c101cb0 0x7f714c199360 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f713c00c890 tx=0x7f713c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:33.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.310+0000 7f713b7fe640 1 -- 172.21.15.78:0/250217783 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c004220 con 0x7f714c101cb0 2024-09-11T00:57:33.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.310+0000 7f713b7fe640 1 -- 172.21.15.78:0/250217783 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f713c0043c0 con 0x7f714c101cb0 2024-09-11T00:57:33.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.310+0000 7f7150e2a640 1 -- 172.21.15.78:0/250217783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714c19f700 con 0x7f714c101cb0 2024-09-11T00:57:33.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.310+0000 7f713b7fe640 1 -- 172.21.15.78:0/250217783 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c004a00 con 0x7f714c101cb0 2024-09-11T00:57:33.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.310+0000 7f7150e2a640 1 -- 172.21.15.78:0/250217783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714c1a5f90 con 0x7f714c101cb0 2024-09-11T00:57:33.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.311+0000 7f7150e2a640 1 -- 172.21.15.78:0/250217783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f714c1047d0 con 0x7f714c101cb0 2024-09-11T00:57:33.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.312+0000 7f713b7fe640 1 -- 172.21.15.78:0/250217783 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f713c0079b0 con 0x7f714c101cb0 2024-09-11T00:57:33.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.313+0000 7f713b7fe640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71340b5440 0x7f71340b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:33.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.313+0000 7f7149d9b640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71340b5440 0x7f71340b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:33.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.313+0000 7f713b7fe640 1 -- 172.21.15.78:0/250217783 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f713c0e5fe0 con 0x7f714c101cb0 2024-09-11T00:57:33.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.313+0000 7f713b7fe640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053] conn(0x7f71340bf0e0 0x7f71340c1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:33.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.313+0000 7f713b7fe640 1 -- 172.21.15.78:0/250217783 --> [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71340c1c50 con 0x7f71340bf0e0 2024-09-11T00:57:33.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.313+0000 7f713b7fe640 1 -- 172.21.15.78:0/250217783 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f713c0e6430 con 0x7f714c101cb0 2024-09-11T00:57:33.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.313+0000 7f714ad9d640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053] conn(0x7f71340bf0e0 0x7f71340c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:33.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.313+0000 7f7149d9b640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71340b5440 0x7f71340b7900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f71400096c0 tx=0x7f7140031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:33.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.314+0000 7f714ad9d640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053] conn(0x7f71340bf0e0 0x7f71340c1580 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:33.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.314+0000 7f713b7fe640 1 -- 172.21.15.78:0/250217783 <== osd.0 v2:172.21.15.78:6802/1424639053 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71340c1c50 con 0x7f71340bf0e0 2024-09-11T00:57:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.327+0000 7f7150e2a640 1 -- 172.21.15.78:0/250217783 --> [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f714c102f60 con 0x7f71340bf0e0 2024-09-11T00:57:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.328+0000 7f713b7fe640 1 -- 172.21.15.78:0/250217783 <== osd.0 v2:172.21.15.78:6802/1424639053 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f714c102f60 con 0x7f71340bf0e0 2024-09-11T00:57:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.328+0000 7f7150e2a640 1 -- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053] conn(0x7f71340bf0e0 msgr2=0x7f71340c1580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.328+0000 7f7150e2a640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053] conn(0x7f71340bf0e0 0x7f71340c1580 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.328+0000 7f7150e2a640 1 -- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71340b5440 msgr2=0x7f71340b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.328+0000 7f7150e2a640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71340b5440 0x7f71340b7900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f71400096c0 tx=0x7f7140031400 comp rx=0 tx=0).stop 2024-09-11T00:57:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.328+0000 7f7150e2a640 1 -- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c101cb0 msgr2=0x7f714c199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.328+0000 7f7150e2a640 1 --2- 172.21.15.78:0/250217783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c101cb0 0x7f714c199360 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f713c00c890 tx=0x7f713c00cd50 comp rx=0 tx=0).stop 2024-09-11T00:57:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.328+0000 7f714ad9d640 1 -- 172.21.15.78:0/250217783 reap_dead start 2024-09-11T00:57:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.328+0000 7f7150e2a640 1 -- 172.21.15.78:0/250217783 shutdown_connections 2024-09-11T00:57:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.328+0000 7f7150e2a640 1 -- 172.21.15.78:0/250217783 >> 172.21.15.78:0/250217783 conn(0x7f714c0fd700 msgr2=0x7f714c06b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.328+0000 7f7150e2a640 1 -- 172.21.15.78:0/250217783 shutdown_connections 2024-09-11T00:57:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.328+0000 7f7150e2a640 1 -- 172.21.15.78:0/250217783 wait complete. 2024-09-11T00:57:33.421 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:33 smithi078 ceph-mon[31315]: pgmap v283: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s; 93 B/s, 0 objects/s recovering 2024-09-11T00:57:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:33 smithi110 ceph-mon[34712]: pgmap v283: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s; 93 B/s, 0 objects/s recovering 2024-09-11T00:57:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:33 smithi112 ceph-mon[35384]: pgmap v283: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s; 93 B/s, 0 objects/s recovering 2024-09-11T00:57:33.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.762+0000 7f4317055640 1 -- 172.21.15.78:0/907359523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310109150 msgr2=0x7f431010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:33.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.762+0000 7f4317055640 1 --2- 172.21.15.78:0/907359523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310109150 0x7f431010f950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4308009900 tx=0x7f430802a410 comp rx=0 tx=0).stop 2024-09-11T00:57:33.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.762+0000 7f4317055640 1 -- 172.21.15.78:0/907359523 shutdown_connections 2024-09-11T00:57:33.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.762+0000 7f4317055640 1 --2- 172.21.15.78:0/907359523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310109150 0x7f431010f950 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:33.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.762+0000 7f4317055640 1 --2- 172.21.15.78:0/907359523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43101087d0 0x7f4310108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:33.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.762+0000 7f4317055640 1 --2- 172.21.15.78:0/907359523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4310102dc0 0x7f43101031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:33.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.762+0000 7f4317055640 1 -- 172.21.15.78:0/907359523 >> 172.21.15.78:0/907359523 conn(0x7f43100fe830 msgr2=0x7f4310100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:33.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.762+0000 7f4317055640 1 -- 172.21.15.78:0/907359523 shutdown_connections 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.762+0000 7f4317055640 1 -- 172.21.15.78:0/907359523 wait complete. 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f4317055640 1 Processor -- start 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f4317055640 1 -- start start 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f4317055640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4310102dc0 0x7f43101993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f4317055640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43101087d0 0x7f4310199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f4317055640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310109150 0x7f431019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f4317055640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4310112e50 con 0x7f4310109150 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f4317055640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4310112cd0 con 0x7f4310102dc0 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f4317055640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4310112fd0 con 0x7f43101087d0 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f43155f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310109150 0x7f431019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f43155f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310109150 0x7f431019ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45678/0 (socket says 172.21.15.78:45678) 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f43155f2640 1 -- 172.21.15.78:0/4080814809 learned_addr learned my addr 172.21.15.78:0/4080814809 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:33.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f4314df1640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4310102dc0 0x7f43101993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:33.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f4307fff640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43101087d0 0x7f4310199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:33.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f43155f2640 1 -- 172.21.15.78:0/4080814809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43101087d0 msgr2=0x7f4310199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:33.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f43155f2640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43101087d0 0x7f4310199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:33.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f43155f2640 1 -- 172.21.15.78:0/4080814809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4310102dc0 msgr2=0x7f43101993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:33.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f43155f2640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4310102dc0 0x7f43101993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:33.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.763+0000 7f43155f2640 1 -- 172.21.15.78:0/4080814809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431019f490 con 0x7f4310109150 2024-09-11T00:57:33.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.764+0000 7f43155f2640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310109150 0x7f431019ecc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4308009a20 tx=0x7f4308004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:33.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.764+0000 7f4305ffb640 1 -- 172.21.15.78:0/4080814809 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430803f030 con 0x7f4310109150 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.764+0000 7f4317055640 1 -- 172.21.15.78:0/4080814809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431019f720 con 0x7f4310109150 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.764+0000 7f4305ffb640 1 -- 172.21.15.78:0/4080814809 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4308003aa0 con 0x7f4310109150 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.764+0000 7f4305ffb640 1 -- 172.21.15.78:0/4080814809 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308035070 con 0x7f4310109150 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.764+0000 7f4317055640 1 -- 172.21.15.78:0/4080814809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43101a5e40 con 0x7f4310109150 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.765+0000 7f4317055640 1 -- 172.21.15.78:0/4080814809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43100638e0 con 0x7f4310109150 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.766+0000 7f4305ffb640 1 -- 172.21.15.78:0/4080814809 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f43080035f0 con 0x7f4310109150 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.767+0000 7f4305ffb640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f42f80b53c0 0x7f42f80b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.767+0000 7f4314df1640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f42f80b53c0 0x7f42f80b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.767+0000 7f4305ffb640 1 -- 172.21.15.78:0/4080814809 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4308106fd0 con 0x7f4310109150 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.767+0000 7f4305ffb640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168] conn(0x7f42f80bf060 0x7f42f80c1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.767+0000 7f4305ffb640 1 -- 172.21.15.78:0/4080814809 --> [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f430802bec0 con 0x7f42f80bf060 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.767+0000 7f4305ffb640 1 -- 172.21.15.78:0/4080814809 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f4308107420 con 0x7f4310109150 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.767+0000 7f4314df1640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f42f80b53c0 0x7f42f80b7880 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f430c00a400 tx=0x7f430c00d040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:33.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.767+0000 7f4307fff640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168] conn(0x7f42f80bf060 0x7f42f80c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.768+0000 7f4307fff640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168] conn(0x7f42f80bf060 0x7f42f80c1500 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.769+0000 7f4305ffb640 1 -- 172.21.15.78:0/4080814809 <== osd.10 v2:172.21.15.110:6824/2101123168 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f430802bec0 con 0x7f42f80bf060 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.782+0000 7f4317055640 1 -- 172.21.15.78:0/4080814809 --> [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4310063ce0 con 0x7f42f80bf060 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.783+0000 7f4305ffb640 1 -- 172.21.15.78:0/4080814809 <== osd.10 v2:172.21.15.110:6824/2101123168 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4310063ce0 con 0x7f42f80bf060 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.783+0000 7f4317055640 1 -- 172.21.15.78:0/4080814809 >> [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168] conn(0x7f42f80bf060 msgr2=0x7f42f80c1500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.783+0000 7f4317055640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168] conn(0x7f42f80bf060 0x7f42f80c1500 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.783+0000 7f4317055640 1 -- 172.21.15.78:0/4080814809 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f42f80b53c0 msgr2=0x7f42f80b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.783+0000 7f4317055640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f42f80b53c0 0x7f42f80b7880 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f430c00a400 tx=0x7f430c00d040 comp rx=0 tx=0).stop 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.783+0000 7f4317055640 1 -- 172.21.15.78:0/4080814809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310109150 msgr2=0x7f431019ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.783+0000 7f4317055640 1 --2- 172.21.15.78:0/4080814809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310109150 0x7f431019ecc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4308009a20 tx=0x7f4308004270 comp rx=0 tx=0).stop 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.783+0000 7f43155f2640 1 -- 172.21.15.78:0/4080814809 reap_dead start 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.783+0000 7f4317055640 1 -- 172.21.15.78:0/4080814809 shutdown_connections 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.783+0000 7f4317055640 1 -- 172.21.15.78:0/4080814809 >> 172.21.15.78:0/4080814809 conn(0x7f43100fe830 msgr2=0x7f4310100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:33.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.783+0000 7f4317055640 1 -- 172.21.15.78:0/4080814809 shutdown_connections 2024-09-11T00:57:33.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:33.783+0000 7f4317055640 1 -- 172.21.15.78:0/4080814809 wait complete. 2024-09-11T00:57:34.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.710+0000 7f54ee04c640 1 -- 172.21.15.78:0/3795291369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e8104550 msgr2=0x7f54e810cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:34.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.710+0000 7f54ee04c640 1 --2- 172.21.15.78:0/3795291369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e8104550 0x7f54e810cb30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009900 tx=0x7f54d002a410 comp rx=0 tx=0).stop 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.710+0000 7f54ee04c640 1 -- 172.21.15.78:0/3795291369 shutdown_connections 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.710+0000 7f54ee04c640 1 --2- 172.21.15.78:0/3795291369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e810d100 0x7f54e810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.710+0000 7f54ee04c640 1 --2- 172.21.15.78:0/3795291369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e8104550 0x7f54e810cb30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.710+0000 7f54ee04c640 1 --2- 172.21.15.78:0/3795291369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54e8103b50 0x7f54e8103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.710+0000 7f54ee04c640 1 -- 172.21.15.78:0/3795291369 >> 172.21.15.78:0/3795291369 conn(0x7f54e80fd700 msgr2=0x7f54e80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.710+0000 7f54ee04c640 1 -- 172.21.15.78:0/3795291369 shutdown_connections 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.710+0000 7f54ee04c640 1 -- 172.21.15.78:0/3795291369 wait complete. 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.710+0000 7f54ee04c640 1 Processor -- start 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54ee04c640 1 -- start start 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54ee04c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e8103b50 0x7f54e81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54ee04c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e8104550 0x7f54e8199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54ee04c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54e810d100 0x7f54e819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:34.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54ee04c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8112db0 con 0x7f54e8103b50 2024-09-11T00:57:34.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54ee04c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8112c30 con 0x7f54e810d100 2024-09-11T00:57:34.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54ee04c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8112f30 con 0x7f54e8104550 2024-09-11T00:57:34.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54e77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e8103b50 0x7f54e81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:34.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54e77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e8103b50 0x7f54e81991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45702/0 (socket says 172.21.15.78:45702) 2024-09-11T00:57:34.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54e77fe640 1 -- 172.21.15.78:0/3662127038 learned_addr learned my addr 172.21.15.78:0/3662127038 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:34.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54e77fe640 1 -- 172.21.15.78:0/3662127038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e8104550 msgr2=0x7f54e8199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:34.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54e77fe640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e8104550 0x7f54e8199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:34.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54e77fe640 1 -- 172.21.15.78:0/3662127038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54e810d100 msgr2=0x7f54e819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:34.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54e7fff640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54e810d100 0x7f54e819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:34.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54e77fe640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54e810d100 0x7f54e819eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:34.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54e77fe640 1 -- 172.21.15.78:0/3662127038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e819f270 con 0x7f54e8103b50 2024-09-11T00:57:34.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54e77fe640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e8103b50 0x7f54e81991c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f54d800c920 tx=0x7f54d800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:34.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54e57fa640 1 -- 172.21.15.78:0/3662127038 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8007c70 con 0x7f54e8103b50 2024-09-11T00:57:34.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.711+0000 7f54ee04c640 1 -- 172.21.15.78:0/3662127038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e819f560 con 0x7f54e8103b50 2024-09-11T00:57:34.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.712+0000 7f54e57fa640 1 -- 172.21.15.78:0/3662127038 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d8007e10 con 0x7f54e8103b50 2024-09-11T00:57:34.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.712+0000 7f54e57fa640 1 -- 172.21.15.78:0/3662127038 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8018820 con 0x7f54e8103b50 2024-09-11T00:57:34.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.712+0000 7f54ee04c640 1 -- 172.21.15.78:0/3662127038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e81a5df0 con 0x7f54e8103b50 2024-09-11T00:57:34.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.712+0000 7f54ee04c640 1 -- 172.21.15.78:0/3662127038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54e80ffd70 con 0x7f54e8103b50 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.714+0000 7f54e57fa640 1 -- 172.21.15.78:0/3662127038 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f54d800f450 con 0x7f54e8103b50 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.714+0000 7f54e57fa640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54d40b53c0 0x7f54d40b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.714+0000 7f54dffff640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54d40b53c0 0x7f54d40b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.715+0000 7f54e57fa640 1 -- 172.21.15.78:0/3662127038 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f54d80e6460 con 0x7f54e8103b50 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.715+0000 7f54e57fa640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385] conn(0x7f54d40bf060 0x7f54d40c1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.715+0000 7f54e57fa640 1 -- 172.21.15.78:0/3662127038 --> [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54d40c1bd0 con 0x7f54d40bf060 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.715+0000 7f54e57fa640 1 -- 172.21.15.78:0/3662127038 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f54d80e68b0 con 0x7f54e8103b50 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.715+0000 7f54dffff640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54d40b53c0 0x7f54d40b7880 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f54d002a930 tx=0x7f54d0031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.715+0000 7f54e7fff640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385] conn(0x7f54d40bf060 0x7f54d40c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.715+0000 7f54e7fff640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385] conn(0x7f54d40bf060 0x7f54d40c1500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.716+0000 7f54e57fa640 1 -- 172.21.15.78:0/3662127038 <== osd.1 v2:172.21.15.110:6800/580939385 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f54d40c1bd0 con 0x7f54d40bf060 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.729+0000 7f54ee04c640 1 -- 172.21.15.78:0/3662127038 --> [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f54e806b130 con 0x7f54d40bf060 2024-09-11T00:57:34.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.730+0000 7f54e57fa640 1 -- 172.21.15.78:0/3662127038 <== osd.1 v2:172.21.15.110:6800/580939385 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f54e806b130 con 0x7f54d40bf060 2024-09-11T00:57:34.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.730+0000 7f54ee04c640 1 -- 172.21.15.78:0/3662127038 >> [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385] conn(0x7f54d40bf060 msgr2=0x7f54d40c1500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:34.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.730+0000 7f54ee04c640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385] conn(0x7f54d40bf060 0x7f54d40c1500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:34.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.730+0000 7f54ee04c640 1 -- 172.21.15.78:0/3662127038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54d40b53c0 msgr2=0x7f54d40b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:34.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.730+0000 7f54ee04c640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54d40b53c0 0x7f54d40b7880 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f54d002a930 tx=0x7f54d0031400 comp rx=0 tx=0).stop 2024-09-11T00:57:34.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.730+0000 7f54ee04c640 1 -- 172.21.15.78:0/3662127038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e8103b50 msgr2=0x7f54e81991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:34.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.730+0000 7f54ee04c640 1 --2- 172.21.15.78:0/3662127038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e8103b50 0x7f54e81991c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f54d800c920 tx=0x7f54d800cde0 comp rx=0 tx=0).stop 2024-09-11T00:57:34.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.730+0000 7f54e7fff640 1 -- 172.21.15.78:0/3662127038 reap_dead start 2024-09-11T00:57:34.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.730+0000 7f54ee04c640 1 -- 172.21.15.78:0/3662127038 shutdown_connections 2024-09-11T00:57:34.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.730+0000 7f54ee04c640 1 -- 172.21.15.78:0/3662127038 >> 172.21.15.78:0/3662127038 conn(0x7f54e80fd700 msgr2=0x7f54e80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:34.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.730+0000 7f54ee04c640 1 -- 172.21.15.78:0/3662127038 shutdown_connections 2024-09-11T00:57:34.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:34.730+0000 7f54ee04c640 1 -- 172.21.15.78:0/3662127038 wait complete. 2024-09-11T00:57:35.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:35 smithi078 ceph-mon[31315]: pgmap v284: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T00:57:35.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:35 smithi110 ceph-mon[34712]: pgmap v284: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T00:57:35.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:35 smithi112 ceph-mon[35384]: pgmap v284: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T00:57:35.689 INFO:teuthology.orchestra.run.smithi078.stdout:38654705742 2024-09-11T00:57:35.689 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T00:57:36.291 INFO:teuthology.orchestra.run.smithi078.stdout:274877906970 2024-09-11T00:57:36.292 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-11T00:57:36.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.512+0000 7fb77a236640 1 -- 172.21.15.78:0/196797653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb774101c90 msgr2=0x7fb774102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.512+0000 7fb77a236640 1 --2- 172.21.15.78:0/196797653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb774101c90 0x7fb774102070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb768009900 tx=0x7fb76802a410 comp rx=0 tx=0).stop 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.512+0000 7fb77a236640 1 -- 172.21.15.78:0/196797653 shutdown_connections 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.512+0000 7fb77a236640 1 --2- 172.21.15.78:0/196797653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb774108020 0x7fb77410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.512+0000 7fb77a236640 1 --2- 172.21.15.78:0/196797653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7741076a0 0x7fb774107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.512+0000 7fb77a236640 1 --2- 172.21.15.78:0/196797653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb774101c90 0x7fb774102070 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.512+0000 7fb77a236640 1 -- 172.21.15.78:0/196797653 >> 172.21.15.78:0/196797653 conn(0x7fb7740fd700 msgr2=0x7fb7740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.512+0000 7fb77a236640 1 -- 172.21.15.78:0/196797653 shutdown_connections 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.512+0000 7fb77a236640 1 -- 172.21.15.78:0/196797653 wait complete. 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.513+0000 7fb77a236640 1 Processor -- start 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.513+0000 7fb77a236640 1 -- start start 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.513+0000 7fb77a236640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb774101c90 0x7fb774199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.513+0000 7fb77a236640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7741076a0 0x7fb774199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:36.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.513+0000 7fb77a236640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb774108020 0x7fb77419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.513+0000 7fb77a236640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb774111ce0 con 0x7fb774101c90 2024-09-11T00:57:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.513+0000 7fb77a236640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb774111b60 con 0x7fb7741076a0 2024-09-11T00:57:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.513+0000 7fb77a236640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb774111e60 con 0x7fb774108020 2024-09-11T00:57:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.513+0000 7fb7737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb774101c90 0x7fb774199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.513+0000 7fb7737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb774101c90 0x7fb774199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45742/0 (socket says 172.21.15.78:45742) 2024-09-11T00:57:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb7737fe640 1 -- 172.21.15.78:0/3057623383 learned_addr learned my addr 172.21.15.78:0/3057623383 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb772ffd640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7741076a0 0x7fb774199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb773fff640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb774108020 0x7fb77419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb7737fe640 1 -- 172.21.15.78:0/3057623383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb774108020 msgr2=0x7fb77419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb7737fe640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb774108020 0x7fb77419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb7737fe640 1 -- 172.21.15.78:0/3057623383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7741076a0 msgr2=0x7fb774199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb7737fe640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7741076a0 0x7fb774199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb7737fe640 1 -- 172.21.15.78:0/3057623383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77419f3d0 con 0x7fb774101c90 2024-09-11T00:57:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb772ffd640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7741076a0 0x7fb774199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb7737fe640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb774101c90 0x7fb774199320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb768006010 tx=0x7fb7680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb770ff9640 1 -- 172.21.15.78:0/3057623383 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76803f030 con 0x7fb774101c90 2024-09-11T00:57:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb770ff9640 1 -- 172.21.15.78:0/3057623383 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb768003800 con 0x7fb774101c90 2024-09-11T00:57:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb77a236640 1 -- 172.21.15.78:0/3057623383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77419f660 con 0x7fb774101c90 2024-09-11T00:57:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb770ff9640 1 -- 172.21.15.78:0/3057623383 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768035070 con 0x7fb774101c90 2024-09-11T00:57:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.514+0000 7fb77a236640 1 -- 172.21.15.78:0/3057623383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7741a5d80 con 0x7fb774101c90 2024-09-11T00:57:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.515+0000 7fb77a236640 1 -- 172.21.15.78:0/3057623383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb774104f70 con 0x7fb774101c90 2024-09-11T00:57:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.516+0000 7fb770ff9640 1 -- 172.21.15.78:0/3057623383 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fb768031b10 con 0x7fb774101c90 2024-09-11T00:57:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.517+0000 7fb770ff9640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb75c0b53c0 0x7fb75c0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.517+0000 7fb772ffd640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb75c0b53c0 0x7fb75c0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.517+0000 7fb770ff9640 1 -- 172.21.15.78:0/3057623383 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb768106f10 con 0x7fb774101c90 2024-09-11T00:57:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.517+0000 7fb770ff9640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775] conn(0x7fb75c0bf060 0x7fb75c0c1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.517+0000 7fb770ff9640 1 -- 172.21.15.78:0/3057623383 --> [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb75c0c1bd0 con 0x7fb75c0bf060 2024-09-11T00:57:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.517+0000 7fb770ff9640 1 -- 172.21.15.78:0/3057623383 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fb768107360 con 0x7fb774101c90 2024-09-11T00:57:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.518+0000 7fb773fff640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775] conn(0x7fb75c0bf060 0x7fb75c0c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.518+0000 7fb772ffd640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb75c0b53c0 0x7fb75c0b7880 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb764005fd0 tx=0x7fb764005ea0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.518+0000 7fb773fff640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775] conn(0x7fb75c0bf060 0x7fb75c0c1500 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.518+0000 7fb770ff9640 1 -- 172.21.15.78:0/3057623383 <== osd.6 v2:172.21.15.78:6818/1267910775 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb75c0c1bd0 con 0x7fb75c0bf060 2024-09-11T00:57:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.532+0000 7fb77a236640 1 -- 172.21.15.78:0/3057623383 --> [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb774102f40 con 0x7fb75c0bf060 2024-09-11T00:57:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.532+0000 7fb770ff9640 1 -- 172.21.15.78:0/3057623383 <== osd.6 v2:172.21.15.78:6818/1267910775 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb774102f40 con 0x7fb75c0bf060 2024-09-11T00:57:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.532+0000 7fb77a236640 1 -- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775] conn(0x7fb75c0bf060 msgr2=0x7fb75c0c1500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.532+0000 7fb77a236640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775] conn(0x7fb75c0bf060 0x7fb75c0c1500 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.532+0000 7fb77a236640 1 -- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb75c0b53c0 msgr2=0x7fb75c0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.532+0000 7fb77a236640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb75c0b53c0 0x7fb75c0b7880 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb764005fd0 tx=0x7fb764005ea0 comp rx=0 tx=0).stop 2024-09-11T00:57:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.532+0000 7fb77a236640 1 -- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb774101c90 msgr2=0x7fb774199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.532+0000 7fb77a236640 1 --2- 172.21.15.78:0/3057623383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb774101c90 0x7fb774199320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb768006010 tx=0x7fb7680042f0 comp rx=0 tx=0).stop 2024-09-11T00:57:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.532+0000 7fb773fff640 1 -- 172.21.15.78:0/3057623383 reap_dead start 2024-09-11T00:57:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.532+0000 7fb77a236640 1 -- 172.21.15.78:0/3057623383 shutdown_connections 2024-09-11T00:57:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.532+0000 7fb77a236640 1 -- 172.21.15.78:0/3057623383 >> 172.21.15.78:0/3057623383 conn(0x7fb7740fd700 msgr2=0x7fb77406aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.533+0000 7fb77a236640 1 -- 172.21.15.78:0/3057623383 shutdown_connections 2024-09-11T00:57:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:36.533+0000 7fb77a236640 1 -- 172.21.15.78:0/3057623383 wait complete. 2024-09-11T00:57:37.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:37 smithi078 ceph-mon[31315]: pgmap v285: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T00:57:37.264 INFO:teuthology.orchestra.run.smithi078.stdout:180388626479 2024-09-11T00:57:37.264 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T00:57:37.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:37 smithi110 ceph-mon[34712]: pgmap v285: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T00:57:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:37 smithi112 ceph-mon[35384]: pgmap v285: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T00:57:37.941 INFO:teuthology.orchestra.run.smithi078.stdout:60129542218 2024-09-11T00:57:37.942 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T00:57:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.872+0000 7f1dcecf3640 1 -- 172.21.15.78:0/580611131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dc8108020 msgr2=0x7f1dc810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.872+0000 7f1dcecf3640 1 --2- 172.21.15.78:0/580611131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dc8108020 0x7f1dc810e820 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009ca0 tx=0x7f1dc002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.872+0000 7f1dcecf3640 1 -- 172.21.15.78:0/580611131 shutdown_connections 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.872+0000 7f1dcecf3640 1 --2- 172.21.15.78:0/580611131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dc8108020 0x7f1dc810e820 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.872+0000 7f1dcecf3640 1 --2- 172.21.15.78:0/580611131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dc81076a0 0x7f1dc8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.872+0000 7f1dcecf3640 1 --2- 172.21.15.78:0/580611131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dc8101c90 0x7f1dc8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.872+0000 7f1dcecf3640 1 -- 172.21.15.78:0/580611131 >> 172.21.15.78:0/580611131 conn(0x7f1dc80fd700 msgr2=0x7f1dc80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.872+0000 7f1dcecf3640 1 -- 172.21.15.78:0/580611131 shutdown_connections 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.872+0000 7f1dcecf3640 1 -- 172.21.15.78:0/580611131 wait complete. 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.872+0000 7f1dcecf3640 1 Processor -- start 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.872+0000 7f1dcecf3640 1 -- start start 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcecf3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dc8101c90 0x7f1dc8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcecf3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dc81076a0 0x7f1dc819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcecf3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dc8108020 0x7f1dc819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcecf3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc8111ce0 con 0x7f1dc8108020 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcecf3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc8111b60 con 0x7f1dc8101c90 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcecf3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc8111e60 con 0x7f1dc81076a0 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcd290640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dc8108020 0x7f1dc819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcd290640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dc8108020 0x7f1dc819aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45718/0 (socket says 172.21.15.78:45718) 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcd290640 1 -- 172.21.15.78:0/425345649 learned_addr learned my addr 172.21.15.78:0/425345649 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcca8f640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dc8101c90 0x7f1dc8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dc7fff640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dc81076a0 0x7f1dc819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcd290640 1 -- 172.21.15.78:0/425345649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dc81076a0 msgr2=0x7f1dc819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcd290640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dc81076a0 0x7f1dc819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcd290640 1 -- 172.21.15.78:0/425345649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dc8101c90 msgr2=0x7f1dc8199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcd290640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dc8101c90 0x7f1dc8199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcd290640 1 -- 172.21.15.78:0/425345649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc81a53d0 con 0x7f1dc8108020 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dcd290640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dc8108020 0x7f1dc819aa90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0005cc0 tx=0x7f1dc002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.873+0000 7f1dc5ffb640 1 -- 172.21.15.78:0/425345649 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0003bf0 con 0x7f1dc8108020 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.874+0000 7f1dcecf3640 1 -- 172.21.15.78:0/425345649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc81a5660 con 0x7f1dc8108020 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.874+0000 7f1dc5ffb640 1 -- 172.21.15.78:0/425345649 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc0003d90 con 0x7f1dc8108020 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.874+0000 7f1dc5ffb640 1 -- 172.21.15.78:0/425345649 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc002fbc0 con 0x7f1dc8108020 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.874+0000 7f1dcecf3640 1 -- 172.21.15.78:0/425345649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc81a5ba0 con 0x7f1dc8108020 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.875+0000 7f1dcecf3640 1 -- 172.21.15.78:0/425345649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1dc8104fb0 con 0x7f1dc8108020 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.876+0000 7f1dc5ffb640 1 -- 172.21.15.78:0/425345649 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f1dc0004610 con 0x7f1dc8108020 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.877+0000 7f1dc5ffb640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1db80b53c0 0x7f1db80b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.877+0000 7f1dcca8f640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1db80b53c0 0x7f1db80b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.877+0000 7f1dc5ffb640 1 -- 172.21.15.78:0/425345649 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1dc0107600 con 0x7f1dc8108020 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.877+0000 7f1dc5ffb640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594] conn(0x7f1db80bf060 0x7f1db80c1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.877+0000 7f1dc5ffb640 1 -- 172.21.15.78:0/425345649 --> [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1db80c1bd0 con 0x7f1db80bf060 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.877+0000 7f1dc5ffb640 1 -- 172.21.15.78:0/425345649 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f1dc0107a50 con 0x7f1dc8108020 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.877+0000 7f1dcca8f640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1db80b53c0 0x7f1db80b7880 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc005fd0 tx=0x7f1dbc005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.878+0000 7f1dc7fff640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594] conn(0x7f1db80bf060 0x7f1db80c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.878+0000 7f1dc7fff640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594] conn(0x7f1db80bf060 0x7f1db80c1500 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.879+0000 7f1dc5ffb640 1 -- 172.21.15.78:0/425345649 <== osd.7 v2:172.21.15.110:6816/1556494594 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1db80c1bd0 con 0x7f1db80bf060 2024-09-11T00:57:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.892+0000 7f1dcecf3640 1 -- 172.21.15.78:0/425345649 --> [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1dc8102f40 con 0x7f1db80bf060 2024-09-11T00:57:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.893+0000 7f1dc5ffb640 1 -- 172.21.15.78:0/425345649 <== osd.7 v2:172.21.15.110:6816/1556494594 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1dc8102f40 con 0x7f1db80bf060 2024-09-11T00:57:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.893+0000 7f1dcecf3640 1 -- 172.21.15.78:0/425345649 >> [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594] conn(0x7f1db80bf060 msgr2=0x7f1db80c1500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.893+0000 7f1dcecf3640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594] conn(0x7f1db80bf060 0x7f1db80c1500 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.893+0000 7f1dcecf3640 1 -- 172.21.15.78:0/425345649 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1db80b53c0 msgr2=0x7f1db80b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.893+0000 7f1dcecf3640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1db80b53c0 0x7f1db80b7880 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc005fd0 tx=0x7f1dbc005850 comp rx=0 tx=0).stop 2024-09-11T00:57:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.893+0000 7f1dcecf3640 1 -- 172.21.15.78:0/425345649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dc8108020 msgr2=0x7f1dc819aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.893+0000 7f1dcecf3640 1 --2- 172.21.15.78:0/425345649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dc8108020 0x7f1dc819aa90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0005cc0 tx=0x7f1dc002f380 comp rx=0 tx=0).stop 2024-09-11T00:57:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.893+0000 7f1dcd290640 1 -- 172.21.15.78:0/425345649 reap_dead start 2024-09-11T00:57:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.893+0000 7f1dcecf3640 1 -- 172.21.15.78:0/425345649 shutdown_connections 2024-09-11T00:57:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.893+0000 7f1dcecf3640 1 -- 172.21.15.78:0/425345649 >> 172.21.15.78:0/425345649 conn(0x7f1dc80fd700 msgr2=0x7f1dc80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.893+0000 7f1dcecf3640 1 -- 172.21.15.78:0/425345649 shutdown_connections 2024-09-11T00:57:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:35.894+0000 7f1dcecf3640 1 -- 172.21.15.78:0/425345649 wait complete. 2024-09-11T00:57:38.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.700+0000 7f4f8c597640 1 -- 172.21.15.78:0/1423412162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8410d120 msgr2=0x7f4f8410f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:38.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.700+0000 7f4f8c597640 1 --2- 172.21.15.78:0/1423412162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8410d120 0x7f4f8410f590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009ca0 tx=0x7f4f7c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.700+0000 7f4f8c597640 1 -- 172.21.15.78:0/1423412162 shutdown_connections 2024-09-11T00:57:38.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.700+0000 7f4f8c597640 1 --2- 172.21.15.78:0/1423412162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8410d120 0x7f4f8410f590 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.700+0000 7f4f8c597640 1 --2- 172.21.15.78:0/1423412162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f84104570 0x7f4f8410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.700+0000 7f4f8c597640 1 --2- 172.21.15.78:0/1423412162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f84103b70 0x7f4f84103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.700+0000 7f4f8c597640 1 -- 172.21.15.78:0/1423412162 >> 172.21.15.78:0/1423412162 conn(0x7f4f840fd700 msgr2=0x7f4f840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.700+0000 7f4f8c597640 1 -- 172.21.15.78:0/1423412162 shutdown_connections 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.700+0000 7f4f8c597640 1 -- 172.21.15.78:0/1423412162 wait complete. 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f8c597640 1 Processor -- start 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f8c597640 1 -- start start 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f8c597640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f84103b70 0x7f4f841991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f8c597640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f84104570 0x7f4f84199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f8c597640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8410d120 0x7f4f8419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f8c597640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84112de0 con 0x7f4f84104570 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f8c597640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84112c60 con 0x7f4f84103b70 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f8c597640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84112f60 con 0x7f4f8410d120 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f89b32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f84104570 0x7f4f84199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f89b32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f84104570 0x7f4f84199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45750/0 (socket says 172.21.15.78:45750) 2024-09-11T00:57:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f89b32640 1 -- 172.21.15.78:0/3138482173 learned_addr learned my addr 172.21.15.78:0/3138482173 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f8a333640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f84103b70 0x7f4f841991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f8ab34640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8410d120 0x7f4f8419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f89b32640 1 -- 172.21.15.78:0/3138482173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8410d120 msgr2=0x7f4f8419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f89b32640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8410d120 0x7f4f8419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f89b32640 1 -- 172.21.15.78:0/3138482173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f84103b70 msgr2=0x7f4f841991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f89b32640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f84103b70 0x7f4f841991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f89b32640 1 -- 172.21.15.78:0/3138482173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8419f2a0 con 0x7f4f84104570 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.701+0000 7f4f8a333640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f84103b70 0x7f4f841991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.702+0000 7f4f89b32640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f84104570 0x7f4f84199730 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400b4a0 tx=0x7f4f7400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.702+0000 7f4f7b7fe640 1 -- 172.21.15.78:0/3138482173 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74014070 con 0x7f4f84104570 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.702+0000 7f4f7b7fe640 1 -- 172.21.15.78:0/3138482173 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f74010ac0 con 0x7f4f84104570 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.702+0000 7f4f7b7fe640 1 -- 172.21.15.78:0/3138482173 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74017c30 con 0x7f4f84104570 2024-09-11T00:57:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.702+0000 7f4f8c597640 1 -- 172.21.15.78:0/3138482173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8419f590 con 0x7f4f84104570 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.702+0000 7f4f8c597640 1 -- 172.21.15.78:0/3138482173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f841a5cd0 con 0x7f4f84104570 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.703+0000 7f4f8c597640 1 -- 172.21.15.78:0/3138482173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f840ff660 con 0x7f4f84104570 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.704+0000 7f4f7b7fe640 1 -- 172.21.15.78:0/3138482173 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f4f74010610 con 0x7f4f84104570 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.705+0000 7f4f7b7fe640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4f700b53c0 0x7f4f700b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.705+0000 7f4f8a333640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4f700b53c0 0x7f4f700b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.705+0000 7f4f7b7fe640 1 -- 172.21.15.78:0/3138482173 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4f740e5890 con 0x7f4f84104570 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.705+0000 7f4f7b7fe640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321] conn(0x7f4f700bf060 0x7f4f700c1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.705+0000 7f4f7b7fe640 1 -- 172.21.15.78:0/3138482173 --> [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f700c1bd0 con 0x7f4f700bf060 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.705+0000 7f4f7b7fe640 1 -- 172.21.15.78:0/3138482173 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f4f740e5ce0 con 0x7f4f84104570 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.705+0000 7f4f8a333640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4f700b53c0 0x7f4f700b7880 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000abe0 tx=0x7f4f8000a5c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.706+0000 7f4f8ab34640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321] conn(0x7f4f700bf060 0x7f4f700c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.706+0000 7f4f8ab34640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321] conn(0x7f4f700bf060 0x7f4f700c1500 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.707+0000 7f4f7b7fe640 1 -- 172.21.15.78:0/3138482173 <== osd.8 v2:172.21.15.112:6816/2481321 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f700c1bd0 con 0x7f4f700bf060 2024-09-11T00:57:38.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.721+0000 7f4f8c597640 1 -- 172.21.15.78:0/3138482173 --> [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f8406bb30 con 0x7f4f700bf060 2024-09-11T00:57:38.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.721+0000 7f4f7b7fe640 1 -- 172.21.15.78:0/3138482173 <== osd.8 v2:172.21.15.112:6816/2481321 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4f8406bb30 con 0x7f4f700bf060 2024-09-11T00:57:38.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.722+0000 7f4f8c597640 1 -- 172.21.15.78:0/3138482173 >> [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321] conn(0x7f4f700bf060 msgr2=0x7f4f700c1500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:38.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.722+0000 7f4f8c597640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321] conn(0x7f4f700bf060 0x7f4f700c1500 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.722+0000 7f4f8c597640 1 -- 172.21.15.78:0/3138482173 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4f700b53c0 msgr2=0x7f4f700b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:38.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.722+0000 7f4f8c597640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4f700b53c0 0x7f4f700b7880 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000abe0 tx=0x7f4f8000a5c0 comp rx=0 tx=0).stop 2024-09-11T00:57:38.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.722+0000 7f4f8c597640 1 -- 172.21.15.78:0/3138482173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f84104570 msgr2=0x7f4f84199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:38.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.722+0000 7f4f8c597640 1 --2- 172.21.15.78:0/3138482173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f84104570 0x7f4f84199730 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400b4a0 tx=0x7f4f7400b960 comp rx=0 tx=0).stop 2024-09-11T00:57:38.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.722+0000 7f4f8ab34640 1 -- 172.21.15.78:0/3138482173 reap_dead start 2024-09-11T00:57:38.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.722+0000 7f4f8c597640 1 -- 172.21.15.78:0/3138482173 shutdown_connections 2024-09-11T00:57:38.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.722+0000 7f4f8c597640 1 -- 172.21.15.78:0/3138482173 >> 172.21.15.78:0/3138482173 conn(0x7f4f840fd700 msgr2=0x7f4f8410f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:38.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.722+0000 7f4f8c597640 1 -- 172.21.15.78:0/3138482173 shutdown_connections 2024-09-11T00:57:38.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:38.722+0000 7f4f8c597640 1 -- 172.21.15.78:0/3138482173 wait complete. 2024-09-11T00:57:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.120+0000 7f4bdd621640 1 -- 172.21.15.78:0/4119594634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd81045c0 msgr2=0x7f4bd810cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.120+0000 7f4bdd621640 1 --2- 172.21.15.78:0/4119594634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd81045c0 0x7f4bd810cab0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009900 tx=0x7f4bcc02a410 comp rx=0 tx=0).stop 2024-09-11T00:57:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.120+0000 7f4bdd621640 1 -- 172.21.15.78:0/4119594634 shutdown_connections 2024-09-11T00:57:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.120+0000 7f4bdd621640 1 --2- 172.21.15.78:0/4119594634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd810d100 0x7f4bd810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.120+0000 7f4bdd621640 1 --2- 172.21.15.78:0/4119594634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd81045c0 0x7f4bd810cab0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.120+0000 7f4bdd621640 1 --2- 172.21.15.78:0/4119594634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd8103ca0 0x7f4bd8104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.120+0000 7f4bdd621640 1 -- 172.21.15.78:0/4119594634 >> 172.21.15.78:0/4119594634 conn(0x7f4bd80fd700 msgr2=0x7f4bd80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.120+0000 7f4bdd621640 1 -- 172.21.15.78:0/4119594634 shutdown_connections 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.120+0000 7f4bdd621640 1 -- 172.21.15.78:0/4119594634 wait complete. 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.121+0000 7f4bdd621640 1 Processor -- start 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.121+0000 7f4bdd621640 1 -- start start 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.121+0000 7f4bdd621640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd8103ca0 0x7f4bd8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.121+0000 7f4bdd621640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd81045c0 0x7f4bd8199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.121+0000 7f4bdd621640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd810d100 0x7f4bd819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.121+0000 7f4bdd621640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd8112db0 con 0x7f4bd8103ca0 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.121+0000 7f4bdd621640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd8112c30 con 0x7f4bd810d100 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.121+0000 7f4bdd621640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd8112f30 con 0x7f4bd81045c0 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.121+0000 7f4bd6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd8103ca0 0x7f4bd8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.121+0000 7f4bd6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd8103ca0 0x7f4bd8199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45768/0 (socket says 172.21.15.78:45768) 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.121+0000 7f4bd6ffd640 1 -- 172.21.15.78:0/960489398 learned_addr learned my addr 172.21.15.78:0/960489398 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bd67fc640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd81045c0 0x7f4bd8199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bd6ffd640 1 -- 172.21.15.78:0/960489398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd81045c0 msgr2=0x7f4bd8199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bd77fe640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd810d100 0x7f4bd819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bd6ffd640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd81045c0 0x7f4bd8199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bd6ffd640 1 -- 172.21.15.78:0/960489398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd810d100 msgr2=0x7f4bd819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bd6ffd640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd810d100 0x7f4bd819ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bd6ffd640 1 -- 172.21.15.78:0/960489398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd819f3d0 con 0x7f4bd8103ca0 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bd67fc640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd81045c0 0x7f4bd8199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bd6ffd640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd8103ca0 0x7f4bd8199320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c890 tx=0x7f4bc800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bbffff640 1 -- 172.21.15.78:0/960489398 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8004220 con 0x7f4bd8103ca0 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bbffff640 1 -- 172.21.15.78:0/960489398 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bc80043c0 con 0x7f4bd8103ca0 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bdd621640 1 -- 172.21.15.78:0/960489398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd819f6c0 con 0x7f4bd8103ca0 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bbffff640 1 -- 172.21.15.78:0/960489398 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8004a00 con 0x7f4bd8103ca0 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.122+0000 7f4bdd621640 1 -- 172.21.15.78:0/960489398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd81a5e00 con 0x7f4bd8103ca0 2024-09-11T00:57:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.123+0000 7f4bdd621640 1 -- 172.21.15.78:0/960489398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4bd80ff620 con 0x7f4bd8103ca0 2024-09-11T00:57:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.124+0000 7f4bbffff640 1 -- 172.21.15.78:0/960489398 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f4bc80079b0 con 0x7f4bd8103ca0 2024-09-11T00:57:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.125+0000 7f4bbffff640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4bc00b53c0 0x7f4bc00b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.125+0000 7f4bd67fc640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4bc00b53c0 0x7f4bc00b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.125+0000 7f4bbffff640 1 -- 172.21.15.78:0/960489398 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4bc80e60a0 con 0x7f4bd8103ca0 2024-09-11T00:57:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.125+0000 7f4bbffff640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096] conn(0x7f4bc00bf060 0x7f4bc00c1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.125+0000 7f4bbffff640 1 -- 172.21.15.78:0/960489398 --> [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4bc801ae50 con 0x7f4bc00bf060 2024-09-11T00:57:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.126+0000 7f4bbffff640 1 -- 172.21.15.78:0/960489398 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f4bc80e64f0 con 0x7f4bd8103ca0 2024-09-11T00:57:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.126+0000 7f4bd67fc640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4bc00b53c0 0x7f4bc00b7880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc000c00 tx=0x7f4bcc031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.126+0000 7f4bd77fe640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096] conn(0x7f4bc00bf060 0x7f4bc00c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.126+0000 7f4bd77fe640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096] conn(0x7f4bc00bf060 0x7f4bc00c1500 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.127+0000 7f4bbffff640 1 -- 172.21.15.78:0/960489398 <== osd.5 v2:172.21.15.112:6808/3557095096 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc801ae50 con 0x7f4bc00bf060 2024-09-11T00:57:39.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.140+0000 7f4bdd621640 1 -- 172.21.15.78:0/960489398 --> [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4bd806b1b0 con 0x7f4bc00bf060 2024-09-11T00:57:39.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.141+0000 7f4bbffff640 1 -- 172.21.15.78:0/960489398 <== osd.5 v2:172.21.15.112:6808/3557095096 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4bd806b1b0 con 0x7f4bc00bf060 2024-09-11T00:57:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.141+0000 7f4bdd621640 1 -- 172.21.15.78:0/960489398 >> [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096] conn(0x7f4bc00bf060 msgr2=0x7f4bc00c1500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.141+0000 7f4bdd621640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096] conn(0x7f4bc00bf060 0x7f4bc00c1500 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.142+0000 7f4bdd621640 1 -- 172.21.15.78:0/960489398 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4bc00b53c0 msgr2=0x7f4bc00b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.142+0000 7f4bdd621640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4bc00b53c0 0x7f4bc00b7880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc000c00 tx=0x7f4bcc031400 comp rx=0 tx=0).stop 2024-09-11T00:57:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.142+0000 7f4bdd621640 1 -- 172.21.15.78:0/960489398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd8103ca0 msgr2=0x7f4bd8199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.142+0000 7f4bdd621640 1 --2- 172.21.15.78:0/960489398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd8103ca0 0x7f4bd8199320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c890 tx=0x7f4bc800cd50 comp rx=0 tx=0).stop 2024-09-11T00:57:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.142+0000 7f4bd77fe640 1 -- 172.21.15.78:0/960489398 reap_dead start 2024-09-11T00:57:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.142+0000 7f4bdd621640 1 -- 172.21.15.78:0/960489398 shutdown_connections 2024-09-11T00:57:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.142+0000 7f4bdd621640 1 -- 172.21.15.78:0/960489398 >> 172.21.15.78:0/960489398 conn(0x7f4bd80fd700 msgr2=0x7f4bd810f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:39.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.142+0000 7f4bdd621640 1 -- 172.21.15.78:0/960489398 shutdown_connections 2024-09-11T00:57:39.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:39.142+0000 7f4bdd621640 1 -- 172.21.15.78:0/960489398 wait complete. 2024-09-11T00:57:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:39 smithi110 ceph-mon[34712]: pgmap v286: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-11T00:57:39.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:39 smithi112 ceph-mon[35384]: pgmap v286: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-11T00:57:39.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:39 smithi078 ceph-mon[31315]: pgmap v286: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-11T00:57:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.411+0000 7f5f4f345640 1 -- 172.21.15.78:0/2018203211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f48108040 msgr2=0x7f5f4810e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.411+0000 7f5f4f345640 1 --2- 172.21.15.78:0/2018203211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f48108040 0x7f5f4810e840 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009ca0 tx=0x7f5f4002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.411+0000 7f5f4f345640 1 -- 172.21.15.78:0/2018203211 shutdown_connections 2024-09-11T00:57:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.411+0000 7f5f4f345640 1 --2- 172.21.15.78:0/2018203211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f48108040 0x7f5f4810e840 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.411+0000 7f5f4f345640 1 --2- 172.21.15.78:0/2018203211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f481076c0 0x7f5f48107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.411+0000 7f5f4f345640 1 --2- 172.21.15.78:0/2018203211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f48101cb0 0x7f5f48102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.411+0000 7f5f4f345640 1 -- 172.21.15.78:0/2018203211 >> 172.21.15.78:0/2018203211 conn(0x7f5f480fd700 msgr2=0x7f5f480ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.411+0000 7f5f4f345640 1 -- 172.21.15.78:0/2018203211 shutdown_connections 2024-09-11T00:57:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.411+0000 7f5f4f345640 1 -- 172.21.15.78:0/2018203211 wait complete. 2024-09-11T00:57:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.411+0000 7f5f4f345640 1 Processor -- start 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.411+0000 7f5f4f345640 1 -- start start 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4f345640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f48101cb0 0x7f5f48197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4f345640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481076c0 0x7f5f481982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4f345640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f48108040 0x7f5f48198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4f345640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48111d00 con 0x7f5f481076c0 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4f345640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48111b80 con 0x7f5f48108040 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4f345640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48111e80 con 0x7f5f48101cb0 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4c8e0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481076c0 0x7f5f481982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4c8e0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481076c0 0x7f5f481982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45802/0 (socket says 172.21.15.78:45802) 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4c8e0640 1 -- 172.21.15.78:0/4228553565 learned_addr learned my addr 172.21.15.78:0/4228553565 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4d0e1640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f48101cb0 0x7f5f48197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4d8e2640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f48108040 0x7f5f48198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4d0e1640 1 -- 172.21.15.78:0/4228553565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f48108040 msgr2=0x7f5f48198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4d0e1640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f48108040 0x7f5f48198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4d0e1640 1 -- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481076c0 msgr2=0x7f5f481982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4d0e1640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481076c0 0x7f5f481982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4d0e1640 1 -- 172.21.15.78:0/4228553565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f481a7580 con 0x7f5f48101cb0 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4c8e0640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481076c0 0x7f5f481982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.412+0000 7f5f4d0e1640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f48101cb0 0x7f5f48197da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800e9f0 tx=0x7f5f3800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.413+0000 7f5f3e7fc640 1 -- 172.21.15.78:0/4228553565 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800ccc0 con 0x7f5f48101cb0 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.413+0000 7f5f4f345640 1 -- 172.21.15.78:0/4228553565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f481a7870 con 0x7f5f48101cb0 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.413+0000 7f5f4f345640 1 -- 172.21.15.78:0/4228553565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f481a7db0 con 0x7f5f48101cb0 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.413+0000 7f5f3e7fc640 1 -- 172.21.15.78:0/4228553565 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f380044d0 con 0x7f5f48101cb0 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.413+0000 7f5f3e7fc640 1 -- 172.21.15.78:0/4228553565 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3801c070 con 0x7f5f48101cb0 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.414+0000 7f5f4f345640 1 -- 172.21.15.78:0/4228553565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5f2c001060 con 0x7f5f48101cb0 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.416+0000 7f5f3e7fc640 1 -- 172.21.15.78:0/4228553565 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f5f38005020 con 0x7f5f48101cb0 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.417+0000 7f5f3e7fc640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5f340b53c0 0x7f5f340b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.417+0000 7f5f4c8e0640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5f340b53c0 0x7f5f340b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:40.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.417+0000 7f5f3e7fc640 1 -- 172.21.15.78:0/4228553565 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5f380e5fe0 con 0x7f5f48101cb0 2024-09-11T00:57:40.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.417+0000 7f5f3e7fc640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350] conn(0x7f5f340bf060 0x7f5f340c1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:40.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.417+0000 7f5f3e7fc640 1 -- 172.21.15.78:0/4228553565 --> [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f38023ea0 con 0x7f5f340bf060 2024-09-11T00:57:40.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.417+0000 7f5f3e7fc640 1 -- 172.21.15.78:0/4228553565 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f5f380e6430 con 0x7f5f48101cb0 2024-09-11T00:57:40.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.417+0000 7f5f4d8e2640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350] conn(0x7f5f340bf060 0x7f5f340c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:40.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.418+0000 7f5f4c8e0640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5f340b53c0 0x7f5f340b7880 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5f48106e90 tx=0x7f5f4400a380 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:40.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.418+0000 7f5f4d8e2640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350] conn(0x7f5f340bf060 0x7f5f340c1500 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:40.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.418+0000 7f5f3e7fc640 1 -- 172.21.15.78:0/4228553565 <== osd.9 v2:172.21.15.78:6826/197048350 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f38023ea0 con 0x7f5f340bf060 2024-09-11T00:57:40.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.432+0000 7f5f4f345640 1 -- 172.21.15.78:0/4228553565 --> [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5f2c002e00 con 0x7f5f340bf060 2024-09-11T00:57:40.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.432+0000 7f5f3e7fc640 1 -- 172.21.15.78:0/4228553565 <== osd.9 v2:172.21.15.78:6826/197048350 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5f2c002e00 con 0x7f5f340bf060 2024-09-11T00:57:40.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.433+0000 7f5f4f345640 1 -- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350] conn(0x7f5f340bf060 msgr2=0x7f5f340c1500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:40.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.433+0000 7f5f4f345640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350] conn(0x7f5f340bf060 0x7f5f340c1500 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:40.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.433+0000 7f5f4f345640 1 -- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5f340b53c0 msgr2=0x7f5f340b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:40.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.433+0000 7f5f4f345640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5f340b53c0 0x7f5f340b7880 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5f48106e90 tx=0x7f5f4400a380 comp rx=0 tx=0).stop 2024-09-11T00:57:40.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.433+0000 7f5f4f345640 1 -- 172.21.15.78:0/4228553565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f48101cb0 msgr2=0x7f5f48197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:40.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.433+0000 7f5f4f345640 1 --2- 172.21.15.78:0/4228553565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f48101cb0 0x7f5f48197da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800e9f0 tx=0x7f5f3800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:57:40.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.433+0000 7f5f4d8e2640 1 -- 172.21.15.78:0/4228553565 reap_dead start 2024-09-11T00:57:40.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.433+0000 7f5f4f345640 1 -- 172.21.15.78:0/4228553565 shutdown_connections 2024-09-11T00:57:40.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.433+0000 7f5f4f345640 1 -- 172.21.15.78:0/4228553565 >> 172.21.15.78:0/4228553565 conn(0x7f5f480fd700 msgr2=0x7f5f480ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:40.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.433+0000 7f5f4f345640 1 -- 172.21.15.78:0/4228553565 shutdown_connections 2024-09-11T00:57:40.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.433+0000 7f5f4f345640 1 -- 172.21.15.78:0/4228553565 wait complete. 2024-09-11T00:57:40.899 INFO:teuthology.orchestra.run.smithi078.stdout:154618822709 2024-09-11T00:57:40.900 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T00:57:41.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:41 smithi078 ceph-mon[31315]: pgmap v287: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:57:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:41 smithi110 ceph-mon[34712]: pgmap v287: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:57:41.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:41 smithi112 ceph-mon[35384]: pgmap v287: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:57:41.518 INFO:teuthology.orchestra.run.smithi078.stdout:201863462955 2024-09-11T00:57:41.518 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T00:57:41.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.290+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/2950400862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc481076a0 msgr2=0x7fcc48107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:41.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.290+0000 7fcc4ccf8640 1 --2- 172.21.15.78:0/2950400862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc481076a0 0x7fcc48107ae0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009900 tx=0x7fcc3c02a410 comp rx=0 tx=0).stop 2024-09-11T00:57:41.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.290+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/2950400862 shutdown_connections 2024-09-11T00:57:41.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.290+0000 7fcc4ccf8640 1 --2- 172.21.15.78:0/2950400862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc48108020 0x7fcc4810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:41.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.290+0000 7fcc4ccf8640 1 --2- 172.21.15.78:0/2950400862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc481076a0 0x7fcc48107ae0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:41.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.290+0000 7fcc4ccf8640 1 --2- 172.21.15.78:0/2950400862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc48101c90 0x7fcc48102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:41.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.290+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/2950400862 >> 172.21.15.78:0/2950400862 conn(0x7fcc480fd700 msgr2=0x7fcc480ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:41.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.290+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/2950400862 shutdown_connections 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.290+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/2950400862 wait complete. 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4ccf8640 1 Processor -- start 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4ccf8640 1 -- start start 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4ccf8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc48101c90 0x7fcc481981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4ccf8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc481076a0 0x7fcc48198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4ccf8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc48108020 0x7fcc4819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4ccf8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc48111c70 con 0x7fcc48101c90 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4ccf8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcc48111af0 con 0x7fcc481076a0 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4ccf8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc48111df0 con 0x7fcc48108020 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc48101c90 0x7fcc481981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc48101c90 0x7fcc481981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45792/0 (socket says 172.21.15.78:45792) 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4659c640 1 -- 172.21.15.78:0/1388941989 learned_addr learned my addr 172.21.15.78:0/1388941989 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc45d9b640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc481076a0 0x7fcc48198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4659c640 1 -- 172.21.15.78:0/1388941989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc48108020 msgr2=0x7fcc4819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc46d9d640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc48108020 0x7fcc4819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4659c640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc48108020 0x7fcc4819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4659c640 1 -- 172.21.15.78:0/1388941989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc481076a0 msgr2=0x7fcc48198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4659c640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc481076a0 0x7fcc48198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4659c640 1 -- 172.21.15.78:0/1388941989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4819e2a0 con 0x7fcc48101c90 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc45d9b640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc481076a0 0x7fcc48198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc46d9d640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc48108020 0x7fcc4819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.291+0000 7fcc4659c640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc48101c90 0x7fcc481981f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009900 tx=0x7fcc38009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.292+0000 7fcc377fe640 1 -- 172.21.15.78:0/1388941989 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3800d030 con 0x7fcc48101c90 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.292+0000 7fcc377fe640 1 -- 172.21.15.78:0/1388941989 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3800d1d0 con 0x7fcc48101c90 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.292+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/1388941989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4819e590 con 0x7fcc48101c90 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.292+0000 7fcc377fe640 1 -- 172.21.15.78:0/1388941989 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc38005280 con 0x7fcc48101c90 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.292+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/1388941989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4806ac80 con 0x7fcc48101c90 2024-09-11T00:57:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.292+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/1388941989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc4806b290 con 0x7fcc48101c90 2024-09-11T00:57:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.294+0000 7fcc377fe640 1 -- 172.21.15.78:0/1388941989 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fcc38005420 con 0x7fcc48101c90 2024-09-11T00:57:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.295+0000 7fcc377fe640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcc300b5440 0x7fcc300b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.295+0000 7fcc45d9b640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcc300b5440 0x7fcc300b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.295+0000 7fcc377fe640 1 -- 172.21.15.78:0/1388941989 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fcc380e7840 con 0x7fcc48101c90 2024-09-11T00:57:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.295+0000 7fcc377fe640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435] conn(0x7fcc300bf0e0 0x7fcc300c1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.295+0000 7fcc377fe640 1 -- 172.21.15.78:0/1388941989 --> [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc3800fe50 con 0x7fcc300bf0e0 2024-09-11T00:57:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.295+0000 7fcc377fe640 1 -- 172.21.15.78:0/1388941989 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fcc380e7c90 con 0x7fcc48101c90 2024-09-11T00:57:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.295+0000 7fcc45d9b640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcc300b5440 0x7fcc300b7900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c031440 tx=0x7fcc3c005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.295+0000 7fcc46d9d640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435] conn(0x7fcc300bf0e0 0x7fcc300c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.296+0000 7fcc46d9d640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435] conn(0x7fcc300bf0e0 0x7fcc300c1580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.296+0000 7fcc377fe640 1 -- 172.21.15.78:0/1388941989 <== osd.4 v2:172.21.15.110:6808/477287435 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc3800fe50 con 0x7fcc300bf0e0 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.310+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/1388941989 --> [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc4806b710 con 0x7fcc300bf0e0 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.310+0000 7fcc377fe640 1 -- 172.21.15.78:0/1388941989 <== osd.4 v2:172.21.15.110:6808/477287435 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcc4806b710 con 0x7fcc300bf0e0 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.311+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/1388941989 >> [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435] conn(0x7fcc300bf0e0 msgr2=0x7fcc300c1580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.311+0000 7fcc4ccf8640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435] conn(0x7fcc300bf0e0 0x7fcc300c1580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.311+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/1388941989 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcc300b5440 msgr2=0x7fcc300b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.311+0000 7fcc4ccf8640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcc300b5440 0x7fcc300b7900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c031440 tx=0x7fcc3c005a20 comp rx=0 tx=0).stop 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.311+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/1388941989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc48101c90 msgr2=0x7fcc481981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.311+0000 7fcc4ccf8640 1 --2- 172.21.15.78:0/1388941989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc48101c90 0x7fcc481981f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009900 tx=0x7fcc38009dc0 comp rx=0 tx=0).stop 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.311+0000 7fcc46d9d640 1 -- 172.21.15.78:0/1388941989 reap_dead start 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.311+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/1388941989 shutdown_connections 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.311+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/1388941989 >> 172.21.15.78:0/1388941989 conn(0x7fcc480fd700 msgr2=0x7fcc480ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.311+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/1388941989 shutdown_connections 2024-09-11T00:57:41.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:40.311+0000 7fcc4ccf8640 1 -- 172.21.15.78:0/1388941989 wait complete. 2024-09-11T00:57:42.598 INFO:teuthology.orchestra.run.smithi078.stdout:133143986235 2024-09-11T00:57:42.598 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T00:57:42.680 INFO:teuthology.orchestra.run.smithi078.stdout:227633266726 2024-09-11T00:57:42.680 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-11T00:57:43.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.129+0000 7fc22f5e5640 1 -- 172.21.15.78:0/1788828995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc228075540 msgr2=0x7fc228075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:43.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.129+0000 7fc22f5e5640 1 --2- 172.21.15.78:0/1788828995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc228075540 0x7fc228075920 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc224009900 tx=0x7fc22402a410 comp rx=0 tx=0).stop 2024-09-11T00:57:43.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.129+0000 7fc22f5e5640 1 -- 172.21.15.78:0/1788828995 shutdown_connections 2024-09-11T00:57:43.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.129+0000 7fc22f5e5640 1 --2- 172.21.15.78:0/1788828995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc228113b20 0x7fc228115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:43.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.129+0000 7fc22f5e5640 1 --2- 172.21.15.78:0/1788828995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc228075e60 0x7fc2281135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:43.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.129+0000 7fc22f5e5640 1 --2- 172.21.15.78:0/1788828995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc228075540 0x7fc228075920 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:43.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.129+0000 7fc22f5e5640 1 -- 172.21.15.78:0/1788828995 >> 172.21.15.78:0/1788828995 conn(0x7fc2280fe830 msgr2=0x7fc228100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:43.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.129+0000 7fc22f5e5640 1 -- 172.21.15.78:0/1788828995 shutdown_connections 2024-09-11T00:57:43.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.129+0000 7fc22f5e5640 1 -- 172.21.15.78:0/1788828995 wait complete. 2024-09-11T00:57:43.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22f5e5640 1 Processor -- start 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22f5e5640 1 -- start start 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22f5e5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc228075540 0x7fc22819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22f5e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc228075e60 0x7fc22819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22f5e5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc228113b20 0x7fc2281a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22f5e5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc2281193c0 con 0x7fc228075540 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22f5e5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc228119240 con 0x7fc228113b20 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22f5e5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc228119540 con 0x7fc228075e60 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22d381640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc228075540 0x7fc22819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22d381640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc228075540 0x7fc22819f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45810/0 (socket says 172.21.15.78:45810) 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22d381640 1 -- 172.21.15.78:0/3307807658 learned_addr learned my addr 172.21.15.78:0/3307807658 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22db82640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc228113b20 0x7fc2281a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.130+0000 7fc22cb80640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc228075e60 0x7fc22819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.131+0000 7fc22d381640 1 -- 172.21.15.78:0/3307807658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc228075e60 msgr2=0x7fc22819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:43.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.131+0000 7fc22d381640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc228075e60 0x7fc22819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:43.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.131+0000 7fc22d381640 1 -- 172.21.15.78:0/3307807658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc228113b20 msgr2=0x7fc2281a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:43.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.131+0000 7fc22d381640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc228113b20 0x7fc2281a51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:43.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.131+0000 7fc22d381640 1 -- 172.21.15.78:0/3307807658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2281a59c0 con 0x7fc228075540 2024-09-11T00:57:43.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.131+0000 7fc22cb80640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc228075e60 0x7fc22819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:57:43.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.131+0000 7fc22d381640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc228075540 0x7fc22819f910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc224005940 tx=0x7fc2240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:43.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.131+0000 7fc21e7fc640 1 -- 172.21.15.78:0/3307807658 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22403f030 con 0x7fc228075540 2024-09-11T00:57:43.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.131+0000 7fc21e7fc640 1 -- 172.21.15.78:0/3307807658 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc224003cb0 con 0x7fc228075540 2024-09-11T00:57:43.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.131+0000 7fc22f5e5640 1 -- 172.21.15.78:0/3307807658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2281a5c50 con 0x7fc228075540 2024-09-11T00:57:43.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.131+0000 7fc21e7fc640 1 -- 172.21.15.78:0/3307807658 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224035070 con 0x7fc228075540 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.131+0000 7fc22f5e5640 1 -- 172.21.15.78:0/3307807658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2281ac370 con 0x7fc228075540 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.132+0000 7fc22f5e5640 1 -- 172.21.15.78:0/3307807658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc228100ea0 con 0x7fc228075540 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.133+0000 7fc21e7fc640 1 -- 172.21.15.78:0/3307807658 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fc22403bb50 con 0x7fc228075540 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.134+0000 7fc21e7fc640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc2140b5440 0x7fc2140b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.134+0000 7fc22cb80640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc2140b5440 0x7fc2140b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.134+0000 7fc21e7fc640 1 -- 172.21.15.78:0/3307807658 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc224106c60 con 0x7fc228075540 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.134+0000 7fc21e7fc640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763] conn(0x7fc2140bf0e0 0x7fc2140c1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.134+0000 7fc21e7fc640 1 -- 172.21.15.78:0/3307807658 --> [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc22403ce50 con 0x7fc2140bf0e0 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.134+0000 7fc21e7fc640 1 -- 172.21.15.78:0/3307807658 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fc2241070b0 con 0x7fc228075540 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.134+0000 7fc22cb80640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc2140b5440 0x7fc2140b7900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc228116130 tx=0x7fc218005e50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.135+0000 7fc22db82640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763] conn(0x7fc2140bf0e0 0x7fc2140c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.135+0000 7fc22db82640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763] conn(0x7fc2140bf0e0 0x7fc2140c1580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.136+0000 7fc21e7fc640 1 -- 172.21.15.78:0/3307807658 <== osd.2 v2:172.21.15.112:6800/2983155763 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc22403ce50 con 0x7fc2140bf0e0 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.149+0000 7fc22f5e5640 1 -- 172.21.15.78:0/3307807658 --> [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc228076a50 con 0x7fc2140bf0e0 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.150+0000 7fc21e7fc640 1 -- 172.21.15.78:0/3307807658 <== osd.2 v2:172.21.15.112:6800/2983155763 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc228076a50 con 0x7fc2140bf0e0 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.150+0000 7fc22f5e5640 1 -- 172.21.15.78:0/3307807658 >> [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763] conn(0x7fc2140bf0e0 msgr2=0x7fc2140c1580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.150+0000 7fc22f5e5640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763] conn(0x7fc2140bf0e0 0x7fc2140c1580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.150+0000 7fc22f5e5640 1 -- 172.21.15.78:0/3307807658 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc2140b5440 msgr2=0x7fc2140b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.150+0000 7fc22f5e5640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc2140b5440 0x7fc2140b7900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc228116130 tx=0x7fc218005e50 comp rx=0 tx=0).stop 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.150+0000 7fc22f5e5640 1 -- 172.21.15.78:0/3307807658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc228075540 msgr2=0x7fc22819f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.150+0000 7fc22f5e5640 1 --2- 172.21.15.78:0/3307807658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc228075540 0x7fc22819f910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc224005940 tx=0x7fc2240042f0 comp rx=0 tx=0).stop 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.150+0000 7fc22db82640 1 -- 172.21.15.78:0/3307807658 reap_dead start 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.150+0000 7fc22f5e5640 1 -- 172.21.15.78:0/3307807658 shutdown_connections 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.150+0000 7fc22f5e5640 1 -- 172.21.15.78:0/3307807658 >> 172.21.15.78:0/3307807658 conn(0x7fc2280fe830 msgr2=0x7fc2280ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.150+0000 7fc22f5e5640 1 -- 172.21.15.78:0/3307807658 shutdown_connections 2024-09-11T00:57:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:43.150+0000 7fc22f5e5640 1 -- 172.21.15.78:0/3307807658 wait complete. 2024-09-11T00:57:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:43 smithi078 ceph-mon[31315]: pgmap v288: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-11T00:57:43.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:43 smithi110 ceph-mon[34712]: pgmap v288: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-11T00:57:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:43 smithi112 ceph-mon[35384]: pgmap v288: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-11T00:57:43.844 INFO:teuthology.orchestra.run.smithi078.stdout:253403070498 2024-09-11T00:57:43.844 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-11T00:57:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.132+0000 7f396bb25640 1 -- 172.21.15.78:0/780125703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964104530 msgr2=0x7f396410cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.132+0000 7f396bb25640 1 --2- 172.21.15.78:0/780125703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964104530 0x7f396410cb10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3958009900 tx=0x7f395802a410 comp rx=0 tx=0).stop 2024-09-11T00:57:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.132+0000 7f396bb25640 1 -- 172.21.15.78:0/780125703 shutdown_connections 2024-09-11T00:57:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.132+0000 7f396bb25640 1 --2- 172.21.15.78:0/780125703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396410d0e0 0x7f396410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.132+0000 7f396bb25640 1 --2- 172.21.15.78:0/780125703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964104530 0x7f396410cb10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.132+0000 7f396bb25640 1 --2- 172.21.15.78:0/780125703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3964103b30 0x7f3964103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.132+0000 7f396bb25640 1 -- 172.21.15.78:0/780125703 >> 172.21.15.78:0/780125703 conn(0x7f39640fd700 msgr2=0x7f39640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.133+0000 7f396bb25640 1 -- 172.21.15.78:0/780125703 shutdown_connections 2024-09-11T00:57:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.133+0000 7f396bb25640 1 -- 172.21.15.78:0/780125703 wait complete. 2024-09-11T00:57:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.133+0000 7f396bb25640 1 Processor -- start 2024-09-11T00:57:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.133+0000 7f396bb25640 1 -- start start 2024-09-11T00:57:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f396bb25640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964103b30 0x7f3964199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f396bb25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3964104530 0x7f39641998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f396bb25640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396410d0e0 0x7f396419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f396bb25640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3964112e20 con 0x7f3964103b30 2024-09-11T00:57:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f396bb25640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3964112ca0 con 0x7f396410d0e0 2024-09-11T00:57:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f396bb25640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3964112fa0 con 0x7f3964104530 2024-09-11T00:57:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f39698c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964103b30 0x7f3964199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f39698c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964103b30 0x7f3964199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35086/0 (socket says 172.21.15.78:35086) 2024-09-11T00:57:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f39698c1640 1 -- 172.21.15.78:0/3557803074 learned_addr learned my addr 172.21.15.78:0/3557803074 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f39690c0640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3964104530 0x7f39641998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f39698c1640 1 -- 172.21.15.78:0/3557803074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3964104530 msgr2=0x7f39641998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f39698c1640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3964104530 0x7f39641998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f396a0c2640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396410d0e0 0x7f396419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f39698c1640 1 -- 172.21.15.78:0/3557803074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396410d0e0 msgr2=0x7f396419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f39698c1640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396410d0e0 0x7f396419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f39698c1640 1 -- 172.21.15.78:0/3557803074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396419f440 con 0x7f3964103b30 2024-09-11T00:57:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f396a0c2640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396410d0e0 0x7f396419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:57:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f39698c1640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964103b30 0x7f3964199390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f395400ba50 tx=0x7f395400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f3962ffd640 1 -- 172.21.15.78:0/3557803074 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954019070 con 0x7f3964103b30 2024-09-11T00:57:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f396bb25640 1 -- 172.21.15.78:0/3557803074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396419f730 con 0x7f3964103b30 2024-09-11T00:57:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f396bb25640 1 -- 172.21.15.78:0/3557803074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39641a5e70 con 0x7f3964103b30 2024-09-11T00:57:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.134+0000 7f3962ffd640 1 -- 172.21.15.78:0/3557803074 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f395400cbe0 con 0x7f3964103b30 2024-09-11T00:57:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.135+0000 7f3962ffd640 1 -- 172.21.15.78:0/3557803074 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39540075d0 con 0x7f3964103b30 2024-09-11T00:57:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.135+0000 7f396bb25640 1 -- 172.21.15.78:0/3557803074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3950001060 con 0x7f3964103b30 2024-09-11T00:57:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.137+0000 7f3962ffd640 1 -- 172.21.15.78:0/3557803074 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f3954026050 con 0x7f3964103b30 2024-09-11T00:57:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.138+0000 7f3962ffd640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f394c0b5440 0x7f394c0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.138+0000 7f39690c0640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f394c0b5440 0x7f394c0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.138+0000 7f3962ffd640 1 -- 172.21.15.78:0/3557803074 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f39540e5a30 con 0x7f3964103b30 2024-09-11T00:57:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.138+0000 7f3962ffd640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654] conn(0x7f394c0bf0e0 0x7f394c0c1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.138+0000 7f3962ffd640 1 -- 172.21.15.78:0/3557803074 --> [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3954024e50 con 0x7f394c0bf0e0 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.138+0000 7f3962ffd640 1 -- 172.21.15.78:0/3557803074 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f39540e5e80 con 0x7f3964103b30 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.138+0000 7f39690c0640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f394c0b5440 0x7f394c0b7900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3958006010 tx=0x7f3958000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.138+0000 7f396a0c2640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654] conn(0x7f394c0bf0e0 0x7f394c0c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.139+0000 7f396a0c2640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654] conn(0x7f394c0bf0e0 0x7f394c0c1580 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.139+0000 7f3962ffd640 1 -- 172.21.15.78:0/3557803074 <== osd.11 v2:172.21.15.112:6824/1526310654 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3954024e50 con 0x7f394c0bf0e0 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.153+0000 7f396bb25640 1 -- 172.21.15.78:0/3557803074 --> [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3950002e00 con 0x7f394c0bf0e0 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.154+0000 7f3962ffd640 1 -- 172.21.15.78:0/3557803074 <== osd.11 v2:172.21.15.112:6824/1526310654 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3950002e00 con 0x7f394c0bf0e0 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.154+0000 7f396bb25640 1 -- 172.21.15.78:0/3557803074 >> [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654] conn(0x7f394c0bf0e0 msgr2=0x7f394c0c1580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.154+0000 7f396bb25640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654] conn(0x7f394c0bf0e0 0x7f394c0c1580 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.154+0000 7f396bb25640 1 -- 172.21.15.78:0/3557803074 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f394c0b5440 msgr2=0x7f394c0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.154+0000 7f396bb25640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f394c0b5440 0x7f394c0b7900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3958006010 tx=0x7f3958000bc0 comp rx=0 tx=0).stop 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.154+0000 7f396bb25640 1 -- 172.21.15.78:0/3557803074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964103b30 msgr2=0x7f3964199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.154+0000 7f396bb25640 1 --2- 172.21.15.78:0/3557803074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964103b30 0x7f3964199390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f395400ba50 tx=0x7f395400bf10 comp rx=0 tx=0).stop 2024-09-11T00:57:44.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.154+0000 7f396a0c2640 1 -- 172.21.15.78:0/3557803074 reap_dead start 2024-09-11T00:57:44.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.154+0000 7f396bb25640 1 -- 172.21.15.78:0/3557803074 shutdown_connections 2024-09-11T00:57:44.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.154+0000 7f396bb25640 1 -- 172.21.15.78:0/3557803074 >> 172.21.15.78:0/3557803074 conn(0x7f39640fd700 msgr2=0x7f39640fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:44.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.154+0000 7f396bb25640 1 -- 172.21.15.78:0/3557803074 shutdown_connections 2024-09-11T00:57:44.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:44.154+0000 7f396bb25640 1 -- 172.21.15.78:0/3557803074 wait complete. 2024-09-11T00:57:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:44.653 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:45.042 INFO:teuthology.orchestra.run.smithi078.stdout:296352743447 2024-09-11T00:57:45.042 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-11T00:57:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:45 smithi110 ceph-mon[34712]: pgmap v289: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T00:57:45.453 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:45 smithi078 ceph-mon[31315]: pgmap v289: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T00:57:45.462 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:45.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:45 smithi112 ceph-mon[35384]: pgmap v289: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T00:57:45.510 INFO:teuthology.orchestra.run.smithi078.stdout:81604378694 2024-09-11T00:57:45.511 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T00:57:45.513 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:45.557 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:45.557 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:45.559 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:46.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.439+0000 7fa72036c640 1 -- 172.21.15.78:0/3005867541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa718109130 msgr2=0x7fa71810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:46.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.439+0000 7fa72036c640 1 --2- 172.21.15.78:0/3005867541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa718109130 0x7fa71810f930 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa710009900 tx=0x7fa71002a410 comp rx=0 tx=0).stop 2024-09-11T00:57:46.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.439+0000 7fa72036c640 1 -- 172.21.15.78:0/3005867541 shutdown_connections 2024-09-11T00:57:46.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.439+0000 7fa72036c640 1 --2- 172.21.15.78:0/3005867541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa718109130 0x7fa71810f930 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:46.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.439+0000 7fa72036c640 1 --2- 172.21.15.78:0/3005867541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7181087b0 0x7fa718108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.439+0000 7fa72036c640 1 --2- 172.21.15.78:0/3005867541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa718102da0 0x7fa718103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.439+0000 7fa72036c640 1 -- 172.21.15.78:0/3005867541 >> 172.21.15.78:0/3005867541 conn(0x7fa7180fe850 msgr2=0x7fa718100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.439+0000 7fa72036c640 1 -- 172.21.15.78:0/3005867541 shutdown_connections 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.439+0000 7fa72036c640 1 -- 172.21.15.78:0/3005867541 wait complete. 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa72036c640 1 Processor -- start 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa72036c640 1 -- start start 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa72036c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa718102da0 0x7fa718199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa72036c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181087b0 0x7fa7181997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa72036c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa718109130 0x7fa71819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa72036c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa718112db0 con 0x7fa7181087b0 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa72036c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa718112c30 con 0x7fa718102da0 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa72036c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa718112f30 con 0x7fa718109130 2024-09-11T00:57:46.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa71d907640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181087b0 0x7fa7181997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa71d907640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181087b0 0x7fa7181997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35104/0 (socket says 172.21.15.78:35104) 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa71d907640 1 -- 172.21.15.78:0/1135189554 learned_addr learned my addr 172.21.15.78:0/1135189554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa71e108640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa718102da0 0x7fa718199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa71e909640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa718109130 0x7fa71819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa71d907640 1 -- 172.21.15.78:0/1135189554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa718109130 msgr2=0x7fa71819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa71d907640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa718109130 0x7fa71819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa71d907640 1 -- 172.21.15.78:0/1135189554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa718102da0 msgr2=0x7fa718199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa71d907640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa718102da0 0x7fa718199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa71d907640 1 -- 172.21.15.78:0/1135189554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71819f310 con 0x7fa7181087b0 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.440+0000 7fa71d907640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181087b0 0x7fa7181997a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa70800d950 tx=0x7fa70800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.441+0000 7fa70f7fe640 1 -- 172.21.15.78:0/1135189554 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708017070 con 0x7fa7181087b0 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.441+0000 7fa72036c640 1 -- 172.21.15.78:0/1135189554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71819f600 con 0x7fa7181087b0 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.441+0000 7fa70f7fe640 1 -- 172.21.15.78:0/1135189554 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa708009c30 con 0x7fa7181087b0 2024-09-11T00:57:46.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.441+0000 7fa70f7fe640 1 -- 172.21.15.78:0/1135189554 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70800f620 con 0x7fa7181087b0 2024-09-11T00:57:46.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.441+0000 7fa72036c640 1 -- 172.21.15.78:0/1135189554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7181a5e90 con 0x7fa7181087b0 2024-09-11T00:57:46.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.442+0000 7fa72036c640 1 -- 172.21.15.78:0/1135189554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa718104050 con 0x7fa7181087b0 2024-09-11T00:57:46.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.443+0000 7fa70f7fe640 1 -- 172.21.15.78:0/1135189554 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fa708010460 con 0x7fa7181087b0 2024-09-11T00:57:46.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.444+0000 7fa70f7fe640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa7040b5440 0x7fa7040b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:46.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.444+0000 7fa71e108640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa7040b5440 0x7fa7040b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:46.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.444+0000 7fa70f7fe640 1 -- 172.21.15.78:0/1135189554 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fa7080e5a30 con 0x7fa7181087b0 2024-09-11T00:57:46.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.444+0000 7fa71e108640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa7040b5440 0x7fa7040b7900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa714005c40 tx=0x7fa71400a590 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:46.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.447+0000 7fa70f7fe640 1 -- 172.21.15.78:0/1135189554 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa7080ac760 con 0x7fa7181087b0 2024-09-11T00:57:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.626+0000 7fa72036c640 1 -- 172.21.15.78:0/1135189554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa71810af10 con 0x7fa7181087b0 2024-09-11T00:57:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.626+0000 7fa70f7fe640 1 -- 172.21.15.78:0/1135189554 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa708009de0 con 0x7fa7181087b0 2024-09-11T00:57:46.628 INFO:teuthology.orchestra.run.smithi078.stdout:180388626481 2024-09-11T00:57:46.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.628+0000 7fa72036c640 1 -- 172.21.15.78:0/1135189554 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa7040b5440 msgr2=0x7fa7040b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:46.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.628+0000 7fa72036c640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa7040b5440 0x7fa7040b7900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa714005c40 tx=0x7fa71400a590 comp rx=0 tx=0).stop 2024-09-11T00:57:46.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.628+0000 7fa72036c640 1 -- 172.21.15.78:0/1135189554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181087b0 msgr2=0x7fa7181997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:46.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.628+0000 7fa72036c640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181087b0 0x7fa7181997a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa70800d950 tx=0x7fa70800de10 comp rx=0 tx=0).stop 2024-09-11T00:57:46.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.629+0000 7fa72036c640 1 -- 172.21.15.78:0/1135189554 shutdown_connections 2024-09-11T00:57:46.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.629+0000 7fa72036c640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa7040b5440 0x7fa7040b7900 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:46.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.629+0000 7fa72036c640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa718109130 0x7fa71819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:46.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.629+0000 7fa72036c640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181087b0 0x7fa7181997a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:46.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.629+0000 7fa72036c640 1 --2- 172.21.15.78:0/1135189554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa718102da0 0x7fa718199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:46.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.629+0000 7fa72036c640 1 -- 172.21.15.78:0/1135189554 >> 172.21.15.78:0/1135189554 conn(0x7fa7180fe850 msgr2=0x7fa7180ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:46.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.629+0000 7fa72036c640 1 -- 172.21.15.78:0/1135189554 shutdown_connections 2024-09-11T00:57:46.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.629+0000 7fa72036c640 1 -- 172.21.15.78:0/1135189554 wait complete. 2024-09-11T00:57:47.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.086+0000 7fad52b7d640 1 -- 172.21.15.78:0/278999987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c1045c0 msgr2=0x7fad4c10cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:47.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.086+0000 7fad52b7d640 1 --2- 172.21.15.78:0/278999987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c1045c0 0x7fad4c10cab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fad38009900 tx=0x7fad3802a410 comp rx=0 tx=0).stop 2024-09-11T00:57:47.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 -- 172.21.15.78:0/278999987 shutdown_connections 2024-09-11T00:57:47.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 --2- 172.21.15.78:0/278999987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad4c10d100 0x7fad4c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 --2- 172.21.15.78:0/278999987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c1045c0 0x7fad4c10cab0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 --2- 172.21.15.78:0/278999987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad4c103ca0 0x7fad4c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 -- 172.21.15.78:0/278999987 >> 172.21.15.78:0/278999987 conn(0x7fad4c0fd700 msgr2=0x7fad4c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:47.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 -- 172.21.15.78:0/278999987 shutdown_connections 2024-09-11T00:57:47.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 -- 172.21.15.78:0/278999987 wait complete. 2024-09-11T00:57:47.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 Processor -- start 2024-09-11T00:57:47.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 -- start start 2024-09-11T00:57:47.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c103ca0 0x7fad4c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:47.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad4c1045c0 0x7fad4c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:47.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad4c10d100 0x7fad4c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:47.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c112d70 con 0x7fad4c103ca0 2024-09-11T00:57:47.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c112bf0 con 0x7fad4c1045c0 2024-09-11T00:57:47.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.087+0000 7fad52b7d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c112ef0 con 0x7fad4c10d100 2024-09-11T00:57:47.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad50919640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c103ca0 0x7fad4c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:47.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad50919640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c103ca0 0x7fad4c1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35138/0 (socket says 172.21.15.78:35138) 2024-09-11T00:57:47.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad50919640 1 -- 172.21.15.78:0/2113872863 learned_addr learned my addr 172.21.15.78:0/2113872863 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:47.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad4bfff640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad4c1045c0 0x7fad4c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:47.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad5111a640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad4c10d100 0x7fad4c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:47.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad50919640 1 -- 172.21.15.78:0/2113872863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad4c10d100 msgr2=0x7fad4c19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:47.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad50919640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad4c10d100 0x7fad4c19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad50919640 1 -- 172.21.15.78:0/2113872863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad4c1045c0 msgr2=0x7fad4c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:47.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad50919640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad4c1045c0 0x7fad4c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad50919640 1 -- 172.21.15.78:0/2113872863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c19f350 con 0x7fad4c103ca0 2024-09-11T00:57:47.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad50919640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c103ca0 0x7fad4c1992a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fad4000e9f0 tx=0x7fad4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:47.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad49ffb640 1 -- 172.21.15.78:0/2113872863 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4000cd80 con 0x7fad4c103ca0 2024-09-11T00:57:47.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad49ffb640 1 -- 172.21.15.78:0/2113872863 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad400108e0 con 0x7fad4c103ca0 2024-09-11T00:57:47.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad49ffb640 1 -- 172.21.15.78:0/2113872863 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40018980 con 0x7fad4c103ca0 2024-09-11T00:57:47.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.088+0000 7fad52b7d640 1 -- 172.21.15.78:0/2113872863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c19f640 con 0x7fad4c103ca0 2024-09-11T00:57:47.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.089+0000 7fad52b7d640 1 -- 172.21.15.78:0/2113872863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c1a5ed0 con 0x7fad4c103ca0 2024-09-11T00:57:47.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.090+0000 7fad52b7d640 1 -- 172.21.15.78:0/2113872863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c06b1b0 con 0x7fad4c103ca0 2024-09-11T00:57:47.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.092+0000 7fad49ffb640 1 -- 172.21.15.78:0/2113872863 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fad40010430 con 0x7fad4c103ca0 2024-09-11T00:57:47.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.092+0000 7fad49ffb640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fad3c0b5440 0x7fad3c0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:47.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.093+0000 7fad4bfff640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fad3c0b5440 0x7fad3c0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:47.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.093+0000 7fad49ffb640 1 -- 172.21.15.78:0/2113872863 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fad400e65c0 con 0x7fad4c103ca0 2024-09-11T00:57:47.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.093+0000 7fad4bfff640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fad3c0b5440 0x7fad3c0b7900 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fad38000c00 tx=0x7fad38031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:47.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.095+0000 7fad49ffb640 1 -- 172.21.15.78:0/2113872863 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fad400acc20 con 0x7fad4c103ca0 2024-09-11T00:57:47.167 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:47 smithi078 ceph-mon[31315]: pgmap v290: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T00:57:47.167 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:47 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1135189554' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-11T00:57:47.167 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:57:47.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.283+0000 7fad52b7d640 1 -- 172.21.15.78:0/2113872863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fad4c104080 con 0x7fad4c103ca0 2024-09-11T00:57:47.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.284+0000 7fad49ffb640 1 -- 172.21.15.78:0/2113872863 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fad4c104080 con 0x7fad4c103ca0 2024-09-11T00:57:47.285 INFO:teuthology.orchestra.run.smithi078.stdout:227633266727 2024-09-11T00:57:47.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.286+0000 7fad52b7d640 1 -- 172.21.15.78:0/2113872863 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fad3c0b5440 msgr2=0x7fad3c0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:47.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.286+0000 7fad52b7d640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fad3c0b5440 0x7fad3c0b7900 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fad38000c00 tx=0x7fad38031400 comp rx=0 tx=0).stop 2024-09-11T00:57:47.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.286+0000 7fad52b7d640 1 -- 172.21.15.78:0/2113872863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c103ca0 msgr2=0x7fad4c1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.286+0000 7fad52b7d640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c103ca0 0x7fad4c1992a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fad4000e9f0 tx=0x7fad4000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.286+0000 7fad52b7d640 1 -- 172.21.15.78:0/2113872863 shutdown_connections 2024-09-11T00:57:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.286+0000 7fad52b7d640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fad3c0b5440 0x7fad3c0b7900 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.286+0000 7fad52b7d640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad4c10d100 0x7fad4c19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.286+0000 7fad52b7d640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad4c1045c0 0x7fad4c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.286+0000 7fad52b7d640 1 --2- 172.21.15.78:0/2113872863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c103ca0 0x7fad4c1992a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.286+0000 7fad52b7d640 1 -- 172.21.15.78:0/2113872863 >> 172.21.15.78:0/2113872863 conn(0x7fad4c0fd700 msgr2=0x7fad4c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.286+0000 7fad52b7d640 1 -- 172.21.15.78:0/2113872863 shutdown_connections 2024-09-11T00:57:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.286+0000 7fad52b7d640 1 -- 172.21.15.78:0/2113872863 wait complete. 2024-09-11T00:57:47.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.960+0000 7f7a7562e640 1 -- 172.21.15.78:0/3896825899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70104550 msgr2=0x7f7a7010cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:47.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.960+0000 7f7a7562e640 1 --2- 172.21.15.78:0/3896825899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70104550 0x7f7a7010cb30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009900 tx=0x7f7a5802a410 comp rx=0 tx=0).stop 2024-09-11T00:57:47.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.961+0000 7f7a7562e640 1 -- 172.21.15.78:0/3896825899 shutdown_connections 2024-09-11T00:57:47.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.961+0000 7f7a7562e640 1 --2- 172.21.15.78:0/3896825899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a7010d100 0x7f7a7010f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.961+0000 7f7a7562e640 1 --2- 172.21.15.78:0/3896825899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70104550 0x7f7a7010cb30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.961+0000 7f7a7562e640 1 --2- 172.21.15.78:0/3896825899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a70103b50 0x7f7a70103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.961+0000 7f7a7562e640 1 -- 172.21.15.78:0/3896825899 >> 172.21.15.78:0/3896825899 conn(0x7f7a700fd700 msgr2=0x7f7a700ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.961+0000 7f7a7562e640 1 -- 172.21.15.78:0/3896825899 shutdown_connections 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.961+0000 7f7a7562e640 1 -- 172.21.15.78:0/3896825899 wait complete. 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.961+0000 7f7a7562e640 1 Processor -- start 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a7562e640 1 -- start start 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a7562e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70103b50 0x7f7a701019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a7562e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a70104550 0x7f7a70101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a7562e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a7010d100 0x7f7a70102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a7562e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70112d80 con 0x7f7a70103b50 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a7562e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70112c00 con 0x7f7a70104550 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a7562e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70112f00 con 0x7f7a7010d100 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a6effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70103b50 0x7f7a701019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a6effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70103b50 0x7f7a701019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35122/0 (socket says 172.21.15.78:35122) 2024-09-11T00:57:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a6effd640 1 -- 172.21.15.78:0/4143405898 learned_addr learned my addr 172.21.15.78:0/4143405898 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:47.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a6e7fc640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a70104550 0x7f7a70101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:47.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a6f7fe640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a7010d100 0x7f7a70102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:47.352 INFO:teuthology.orchestra.run.smithi078.stdout:154618822710 2024-09-11T00:57:47.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a6effd640 1 -- 172.21.15.78:0/4143405898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a7010d100 msgr2=0x7f7a70102450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:47.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a6effd640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a7010d100 0x7f7a70102450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a6effd640 1 -- 172.21.15.78:0/4143405898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a70104550 msgr2=0x7f7a70101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a6effd640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a70104550 0x7f7a70101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.962+0000 7f7a6effd640 1 -- 172.21.15.78:0/4143405898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a701a76f0 con 0x7f7a70103b50 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.963+0000 7f7a6effd640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70103b50 0x7f7a701019d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000d950 tx=0x7f7a6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.963+0000 7f7a67fff640 1 -- 172.21.15.78:0/4143405898 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60017070 con 0x7f7a70103b50 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.963+0000 7f7a7562e640 1 -- 172.21.15.78:0/4143405898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a701a79e0 con 0x7f7a70103b50 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.963+0000 7f7a67fff640 1 -- 172.21.15.78:0/4143405898 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a60009c30 con 0x7f7a70103b50 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.963+0000 7f7a67fff640 1 -- 172.21.15.78:0/4143405898 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000f620 con 0x7f7a70103b50 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.963+0000 7f7a7562e640 1 -- 172.21.15.78:0/4143405898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a701a7ef0 con 0x7f7a70103b50 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.964+0000 7f7a7562e640 1 -- 172.21.15.78:0/4143405898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a70103f30 con 0x7f7a70103b50 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.966+0000 7f7a67fff640 1 -- 172.21.15.78:0/4143405898 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f7a60010460 con 0x7f7a70103b50 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.967+0000 7f7a67fff640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7a5c0b53c0 0x7f7a5c0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.967+0000 7f7a6e7fc640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7a5c0b53c0 0x7f7a5c0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.967+0000 7f7a67fff640 1 -- 172.21.15.78:0/4143405898 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7a600e5a00 con 0x7f7a70103b50 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.967+0000 7f7a6e7fc640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7a5c0b53c0 0x7f7a5c0b7880 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7a58031440 tx=0x7f7a58005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:46.970+0000 7f7a67fff640 1 -- 172.21.15.78:0/4143405898 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7a60009de0 con 0x7f7a70103b50 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.165+0000 7f7a7562e640 1 -- 172.21.15.78:0/4143405898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7a70102cd0 con 0x7f7a70103b50 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.166+0000 7f7a67fff640 1 -- 172.21.15.78:0/4143405898 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7a600ac060 con 0x7f7a70103b50 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.168+0000 7f7a7562e640 1 -- 172.21.15.78:0/4143405898 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7a5c0b53c0 msgr2=0x7f7a5c0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.168+0000 7f7a7562e640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7a5c0b53c0 0x7f7a5c0b7880 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7a58031440 tx=0x7f7a58005b50 comp rx=0 tx=0).stop 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.168+0000 7f7a7562e640 1 -- 172.21.15.78:0/4143405898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70103b50 msgr2=0x7f7a701019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.168+0000 7f7a7562e640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70103b50 0x7f7a701019d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000d950 tx=0x7f7a6000de10 comp rx=0 tx=0).stop 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.168+0000 7f7a7562e640 1 -- 172.21.15.78:0/4143405898 shutdown_connections 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.168+0000 7f7a7562e640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7a5c0b53c0 0x7f7a5c0b7880 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.168+0000 7f7a7562e640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a7010d100 0x7f7a70102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.168+0000 7f7a7562e640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a70104550 0x7f7a70101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.168+0000 7f7a7562e640 1 --2- 172.21.15.78:0/4143405898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70103b50 0x7f7a701019d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:47.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.168+0000 7f7a7562e640 1 -- 172.21.15.78:0/4143405898 >> 172.21.15.78:0/4143405898 conn(0x7f7a700fd700 msgr2=0x7f7a700fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:47.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.168+0000 7f7a7562e640 1 -- 172.21.15.78:0/4143405898 shutdown_connections 2024-09-11T00:57:47.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:47.168+0000 7f7a7562e640 1 -- 172.21.15.78:0/4143405898 wait complete. 2024-09-11T00:57:47.382 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:47 smithi110 ceph-mon[34712]: pgmap v290: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T00:57:47.382 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:47 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1135189554' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-11T00:57:47.382 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:57:47.403 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:47 smithi112 ceph-mon[35384]: pgmap v290: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T00:57:47.403 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:47 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1135189554' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-11T00:57:47.403 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:57:48.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:48 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4143405898' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-11T00:57:48.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:48 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2113872863' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-11T00:57:48.236 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626479 got 180388626481 for osd.6 2024-09-11T00:57:48.236 DEBUG:teuthology.parallel:result is None 2024-09-11T00:57:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:48 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/4143405898' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-11T00:57:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:48 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2113872863' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-11T00:57:48.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:48 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/4143405898' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-11T00:57:48.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:48 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2113872863' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-11T00:57:48.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.865+0000 7f419d018640 1 -- 172.21.15.78:0/3984777024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4198103bb0 msgr2=0x7f4198103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.865+0000 7f419d018640 1 --2- 172.21.15.78:0/3984777024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4198103bb0 0x7f4198103f90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f418c009ca0 tx=0x7f418c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.865+0000 7f419d018640 1 -- 172.21.15.78:0/3984777024 shutdown_connections 2024-09-11T00:57:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.865+0000 7f419d018640 1 --2- 172.21.15.78:0/3984777024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f419810d160 0x7f419810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.865+0000 7f419d018640 1 --2- 172.21.15.78:0/3984777024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41981045b0 0x7f419810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.865+0000 7f419d018640 1 --2- 172.21.15.78:0/3984777024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4198103bb0 0x7f4198103f90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.865+0000 7f419d018640 1 -- 172.21.15.78:0/3984777024 >> 172.21.15.78:0/3984777024 conn(0x7f41980fd700 msgr2=0x7f41980ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.865+0000 7f419d018640 1 -- 172.21.15.78:0/3984777024 shutdown_connections 2024-09-11T00:57:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.865+0000 7f419d018640 1 -- 172.21.15.78:0/3984777024 wait complete. 2024-09-11T00:57:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.866+0000 7f419d018640 1 Processor -- start 2024-09-11T00:57:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.866+0000 7f419d018640 1 -- start start 2024-09-11T00:57:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419d018640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4198103bb0 0x7f4198101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419d018640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41981045b0 0x7f4198101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419d018640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f419810d160 0x7f41981024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419d018640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4198112da0 con 0x7f41981045b0 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419d018640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4198112c20 con 0x7f4198103bb0 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419d018640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4198112f20 con 0x7f419810d160 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41981045b0 0x7f4198101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f4196d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4198103bb0 0x7f4198101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41981045b0 0x7f4198101f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35166/0 (socket says 172.21.15.78:35166) 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f4196d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4198103bb0 0x7f4198101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:53484/0 (socket says 172.21.15.78:53484) 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419659c640 1 -- 172.21.15.78:0/1633818059 learned_addr learned my addr 172.21.15.78:0/1633818059 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419759e640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f419810d160 0x7f41981024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419659c640 1 -- 172.21.15.78:0/1633818059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f419810d160 msgr2=0x7f41981024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419659c640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f419810d160 0x7f41981024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419659c640 1 -- 172.21.15.78:0/1633818059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4198103bb0 msgr2=0x7f4198101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419659c640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4198103bb0 0x7f4198101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419659c640 1 -- 172.21.15.78:0/1633818059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41981a7670 con 0x7f41981045b0 2024-09-11T00:57:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f4196d9d640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4198103bb0 0x7f4198101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:57:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.867+0000 7f419659c640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41981045b0 0x7f4198101f70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f418800d950 tx=0x7f418800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.868+0000 7f417ffff640 1 -- 172.21.15.78:0/1633818059 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188017070 con 0x7f41981045b0 2024-09-11T00:57:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.868+0000 7f419d018640 1 -- 172.21.15.78:0/1633818059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41981a7960 con 0x7f41981045b0 2024-09-11T00:57:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.868+0000 7f417ffff640 1 -- 172.21.15.78:0/1633818059 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4188009c30 con 0x7f41981045b0 2024-09-11T00:57:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.868+0000 7f417ffff640 1 -- 172.21.15.78:0/1633818059 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418800f620 con 0x7f41981045b0 2024-09-11T00:57:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.868+0000 7f419d018640 1 -- 172.21.15.78:0/1633818059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41981a7ef0 con 0x7f41981045b0 2024-09-11T00:57:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.869+0000 7f419d018640 1 -- 172.21.15.78:0/1633818059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4198103f90 con 0x7f41981045b0 2024-09-11T00:57:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.870+0000 7f417ffff640 1 -- 172.21.15.78:0/1633818059 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f4188010460 con 0x7f41981045b0 2024-09-11T00:57:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.871+0000 7f417ffff640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f41800b53c0 0x7f41800b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:48.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.871+0000 7f4196d9d640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f41800b53c0 0x7f41800b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:48.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.871+0000 7f417ffff640 1 -- 172.21.15.78:0/1633818059 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f41880e5a30 con 0x7f41981045b0 2024-09-11T00:57:48.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.871+0000 7f4196d9d640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f41800b53c0 0x7f41800b7880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f418c009430 tx=0x7f418c009360 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:48.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:48.875+0000 7f417ffff640 1 -- 172.21.15.78:0/1633818059 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4188009de0 con 0x7f41981045b0 2024-09-11T00:57:49.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.055+0000 7f419d018640 1 -- 172.21.15.78:0/1633818059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f41981a81f0 con 0x7f41981045b0 2024-09-11T00:57:49.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.055+0000 7f417ffff640 1 -- 172.21.15.78:0/1633818059 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f41880ac090 con 0x7f41981045b0 2024-09-11T00:57:49.057 INFO:teuthology.orchestra.run.smithi078.stdout:107374182465 2024-09-11T00:57:49.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.058+0000 7f419d018640 1 -- 172.21.15.78:0/1633818059 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f41800b53c0 msgr2=0x7f41800b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:49.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.058+0000 7f419d018640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f41800b53c0 0x7f41800b7880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f418c009430 tx=0x7f418c009360 comp rx=0 tx=0).stop 2024-09-11T00:57:49.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.058+0000 7f419d018640 1 -- 172.21.15.78:0/1633818059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41981045b0 msgr2=0x7f4198101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:49.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.058+0000 7f419d018640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41981045b0 0x7f4198101f70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f418800d950 tx=0x7f418800de10 comp rx=0 tx=0).stop 2024-09-11T00:57:49.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.058+0000 7f419d018640 1 -- 172.21.15.78:0/1633818059 shutdown_connections 2024-09-11T00:57:49.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.058+0000 7f419d018640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f41800b53c0 0x7f41800b7880 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.058+0000 7f419d018640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f419810d160 0x7f41981024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.058+0000 7f419d018640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41981045b0 0x7f4198101f70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.058+0000 7f419d018640 1 --2- 172.21.15.78:0/1633818059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4198103bb0 0x7f4198101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.058+0000 7f419d018640 1 -- 172.21.15.78:0/1633818059 >> 172.21.15.78:0/1633818059 conn(0x7f41980fd700 msgr2=0x7f419810f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:49.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.058+0000 7f419d018640 1 -- 172.21.15.78:0/1633818059 shutdown_connections 2024-09-11T00:57:49.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.058+0000 7f419d018640 1 -- 172.21.15.78:0/1633818059 wait complete. 2024-09-11T00:57:49.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.088+0000 7fd6651d1640 1 -- 172.21.15.78:0/612979943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6601080c0 msgr2=0x7fd66010e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:49.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.088+0000 7fd6651d1640 1 --2- 172.21.15.78:0/612979943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6601080c0 0x7fd66010e8c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd658009ca0 tx=0x7fd65802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.088+0000 7fd6651d1640 1 -- 172.21.15.78:0/612979943 shutdown_connections 2024-09-11T00:57:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.088+0000 7fd6651d1640 1 --2- 172.21.15.78:0/612979943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6601080c0 0x7fd66010e8c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.088+0000 7fd6651d1640 1 --2- 172.21.15.78:0/612979943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd660107740 0x7fd660107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.088+0000 7fd6651d1640 1 --2- 172.21.15.78:0/612979943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd660101d30 0x7fd660102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.088+0000 7fd6651d1640 1 -- 172.21.15.78:0/612979943 >> 172.21.15.78:0/612979943 conn(0x7fd6600fd7e0 msgr2=0x7fd6600ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.088+0000 7fd6651d1640 1 -- 172.21.15.78:0/612979943 shutdown_connections 2024-09-11T00:57:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.088+0000 7fd6651d1640 1 -- 172.21.15.78:0/612979943 wait complete. 2024-09-11T00:57:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.089+0000 7fd6651d1640 1 Processor -- start 2024-09-11T00:57:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.089+0000 7fd6651d1640 1 -- start start 2024-09-11T00:57:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd6651d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd660101d30 0x7fd660198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd6651d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd660107740 0x7fd6601986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd6651d1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6601080c0 0x7fd66019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd6651d1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd660111d40 con 0x7fd660101d30 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd6651d1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd660111bc0 con 0x7fd6601080c0 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd6651d1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd660111ec0 con 0x7fd660107740 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd65ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd660101d30 0x7fd660198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd65ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd660101d30 0x7fd660198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35184/0 (socket says 172.21.15.78:35184) 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd65ed9d640 1 -- 172.21.15.78:0/337891031 learned_addr learned my addr 172.21.15.78:0/337891031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd65e59c640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd660107740 0x7fd6601986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd65f59e640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6601080c0 0x7fd66019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd65ed9d640 1 -- 172.21.15.78:0/337891031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd660107740 msgr2=0x7fd6601986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd65ed9d640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd660107740 0x7fd6601986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd65ed9d640 1 -- 172.21.15.78:0/337891031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6601080c0 msgr2=0x7fd66019da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd65ed9d640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6601080c0 0x7fd66019da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd65ed9d640 1 -- 172.21.15.78:0/337891031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66019e240 con 0x7fd660101d30 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd65f59e640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6601080c0 0x7fd66019da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd65ed9d640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd660101d30 0x7fd660198190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd65000c920 tx=0x7fd65000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd647fff640 1 -- 172.21.15.78:0/337891031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650007c70 con 0x7fd660101d30 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd647fff640 1 -- 172.21.15.78:0/337891031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd650007e10 con 0x7fd660101d30 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.090+0000 7fd6651d1640 1 -- 172.21.15.78:0/337891031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66019e530 con 0x7fd660101d30 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.091+0000 7fd647fff640 1 -- 172.21.15.78:0/337891031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650018820 con 0x7fd660101d30 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.091+0000 7fd6651d1640 1 -- 172.21.15.78:0/337891031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6601a4dc0 con 0x7fd660101d30 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.092+0000 7fd6651d1640 1 -- 172.21.15.78:0/337891031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd660102110 con 0x7fd660101d30 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.093+0000 7fd647fff640 1 -- 172.21.15.78:0/337891031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fd65000f450 con 0x7fd660101d30 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.093+0000 7fd647fff640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd6480b5440 0x7fd6480b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.094+0000 7fd65e59c640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd6480b5440 0x7fd6480b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.094+0000 7fd647fff640 1 -- 172.21.15.78:0/337891031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd6500e5f30 con 0x7fd660101d30 2024-09-11T00:57:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.094+0000 7fd65e59c640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd6480b5440 0x7fd6480b7900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd660100210 tx=0x7fd65400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:49.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.097+0000 7fd647fff640 1 -- 172.21.15.78:0/337891031 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd650014030 con 0x7fd660101d30 2024-09-11T00:57:49.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.283+0000 7fd6651d1640 1 -- 172.21.15.78:0/337891031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd6601992c0 con 0x7fd660101d30 2024-09-11T00:57:49.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.284+0000 7fd647fff640 1 -- 172.21.15.78:0/337891031 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd6500acbe0 con 0x7fd660101d30 2024-09-11T00:57:49.286 INFO:teuthology.orchestra.run.smithi078.stdout:296352743448 2024-09-11T00:57:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.286+0000 7fd6651d1640 1 -- 172.21.15.78:0/337891031 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd6480b5440 msgr2=0x7fd6480b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.286+0000 7fd6651d1640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd6480b5440 0x7fd6480b7900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd660100210 tx=0x7fd65400b410 comp rx=0 tx=0).stop 2024-09-11T00:57:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.286+0000 7fd6651d1640 1 -- 172.21.15.78:0/337891031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd660101d30 msgr2=0x7fd660198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.286+0000 7fd6651d1640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd660101d30 0x7fd660198190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd65000c920 tx=0x7fd65000cde0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.286+0000 7fd6651d1640 1 -- 172.21.15.78:0/337891031 shutdown_connections 2024-09-11T00:57:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.286+0000 7fd6651d1640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd6480b5440 0x7fd6480b7900 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.286+0000 7fd6651d1640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6601080c0 0x7fd66019da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.286+0000 7fd6651d1640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd660107740 0x7fd6601986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.286+0000 7fd6651d1640 1 --2- 172.21.15.78:0/337891031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd660101d30 0x7fd660198190 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:49.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.286+0000 7fd6651d1640 1 -- 172.21.15.78:0/337891031 >> 172.21.15.78:0/337891031 conn(0x7fd6600fd7e0 msgr2=0x7fd6600fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:49.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.286+0000 7fd6651d1640 1 -- 172.21.15.78:0/337891031 shutdown_connections 2024-09-11T00:57:49.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:49.286+0000 7fd6651d1640 1 -- 172.21.15.78:0/337891031 wait complete. 2024-09-11T00:57:49.351 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:49 smithi078 ceph-mon[31315]: pgmap v291: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:57:49.352 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:49 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:57:49.352 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:49 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:57:49.352 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:49 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:57:49.352 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:49 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:57:49.352 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:49 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1633818059' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-11T00:57:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:49 smithi110 ceph-mon[34712]: pgmap v291: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:57:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:49 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:57:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:49 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:57:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:49 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:57:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:49 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:57:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:49 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1633818059' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-11T00:57:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:49 smithi112 ceph-mon[35384]: pgmap v291: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:57:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:49 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:57:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:49 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:57:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:49 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:57:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:49 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:57:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:49 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1633818059' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-11T00:57:50.026 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182465 for osd.3 2024-09-11T00:57:50.026 DEBUG:teuthology.parallel:result is None 2024-09-11T00:57:50.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:50 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/337891031' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-11T00:57:50.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:50 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/337891031' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-11T00:57:50.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:50 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/337891031' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-11T00:57:50.688 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266726 got 227633266727 for osd.8 2024-09-11T00:57:50.688 DEBUG:teuthology.parallel:result is None 2024-09-11T00:57:51.196 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:51 smithi078 ceph-mon[31315]: pgmap v292: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-11T00:57:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:51 smithi110 ceph-mon[34712]: pgmap v292: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-11T00:57:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:51 smithi112 ceph-mon[35384]: pgmap v292: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-11T00:57:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.635+0000 7fb414e9a640 1 -- 172.21.15.78:0/51382232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb410103ca0 msgr2=0x7fb410104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.635+0000 7fb414e9a640 1 --2- 172.21.15.78:0/51382232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb410103ca0 0x7fb410104080 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb400009900 tx=0x7fb40002a410 comp rx=0 tx=0).stop 2024-09-11T00:57:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.635+0000 7fb414e9a640 1 -- 172.21.15.78:0/51382232 shutdown_connections 2024-09-11T00:57:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.635+0000 7fb414e9a640 1 --2- 172.21.15.78:0/51382232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb41010d100 0x7fb41010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.635+0000 7fb414e9a640 1 --2- 172.21.15.78:0/51382232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4101045c0 0x7fb41010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.635+0000 7fb414e9a640 1 --2- 172.21.15.78:0/51382232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb410103ca0 0x7fb410104080 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.635+0000 7fb414e9a640 1 -- 172.21.15.78:0/51382232 >> 172.21.15.78:0/51382232 conn(0x7fb4100fd700 msgr2=0x7fb4100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.635+0000 7fb414e9a640 1 -- 172.21.15.78:0/51382232 shutdown_connections 2024-09-11T00:57:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.635+0000 7fb414e9a640 1 -- 172.21.15.78:0/51382232 wait complete. 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.636+0000 7fb414e9a640 1 Processor -- start 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.636+0000 7fb414e9a640 1 -- start start 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.636+0000 7fb414e9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb410103ca0 0x7fb410199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.636+0000 7fb414e9a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4101045c0 0x7fb4101997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.636+0000 7fb414e9a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb41010d100 0x7fb41019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.636+0000 7fb414e9a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb410112d70 con 0x7fb4101045c0 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.636+0000 7fb414e9a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb410112bf0 con 0x7fb41010d100 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.636+0000 7fb414e9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb410112ef0 con 0x7fb410103ca0 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb40dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4101045c0 0x7fb4101997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb40dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4101045c0 0x7fb4101997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35206/0 (socket says 172.21.15.78:35206) 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb40dd9b640 1 -- 172.21.15.78:0/927343599 learned_addr learned my addr 172.21.15.78:0/927343599 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb40e59c640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb410103ca0 0x7fb410199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb40ed9d640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb41010d100 0x7fb41019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb40dd9b640 1 -- 172.21.15.78:0/927343599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb410103ca0 msgr2=0x7fb410199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb40dd9b640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb410103ca0 0x7fb410199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb40dd9b640 1 -- 172.21.15.78:0/927343599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb41010d100 msgr2=0x7fb41019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb40dd9b640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb41010d100 0x7fb41019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb40dd9b640 1 -- 172.21.15.78:0/927343599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41019f330 con 0x7fb4101045c0 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb40ed9d640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb41010d100 0x7fb41019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb40dd9b640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4101045c0 0x7fb4101997c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb40400d740 tx=0x7fb40400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb3ff7fe640 1 -- 172.21.15.78:0/927343599 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404017020 con 0x7fb4101045c0 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb414e9a640 1 -- 172.21.15.78:0/927343599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41019f620 con 0x7fb4101045c0 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb3ff7fe640 1 -- 172.21.15.78:0/927343599 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40400de50 con 0x7fb4101045c0 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb3ff7fe640 1 -- 172.21.15.78:0/927343599 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40400f620 con 0x7fb4101045c0 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.637+0000 7fb414e9a640 1 -- 172.21.15.78:0/927343599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101a5eb0 con 0x7fb4101045c0 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.638+0000 7fb414e9a640 1 -- 172.21.15.78:0/927343599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb410104080 con 0x7fb4101045c0 2024-09-11T00:57:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.640+0000 7fb3ff7fe640 1 -- 172.21.15.78:0/927343599 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fb404010460 con 0x7fb4101045c0 2024-09-11T00:57:51.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.640+0000 7fb3ff7fe640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb3f80b53c0 0x7fb3f80b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:51.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.640+0000 7fb40e59c640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb3f80b53c0 0x7fb3f80b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:51.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.640+0000 7fb3ff7fe640 1 -- 172.21.15.78:0/927343599 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb4040e60a0 con 0x7fb4101045c0 2024-09-11T00:57:51.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.641+0000 7fb40e59c640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb3f80b53c0 0x7fb3f80b7880 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb400031440 tx=0x7fb4000095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:51.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.645+0000 7fb3ff7fe640 1 -- 172.21.15.78:0/927343599 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb404021020 con 0x7fb4101045c0 2024-09-11T00:57:51.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.829+0000 7fb414e9a640 1 -- 172.21.15.78:0/927343599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb4101a60b0 con 0x7fb4101045c0 2024-09-11T00:57:51.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.830+0000 7fb3ff7fe640 1 -- 172.21.15.78:0/927343599 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb4040acd50 con 0x7fb4101045c0 2024-09-11T00:57:51.831 INFO:teuthology.orchestra.run.smithi078.stdout:38654705745 2024-09-11T00:57:51.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.832+0000 7fb414e9a640 1 -- 172.21.15.78:0/927343599 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb3f80b53c0 msgr2=0x7fb3f80b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:51.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.832+0000 7fb414e9a640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb3f80b53c0 0x7fb3f80b7880 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb400031440 tx=0x7fb4000095f0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.832+0000 7fb414e9a640 1 -- 172.21.15.78:0/927343599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4101045c0 msgr2=0x7fb4101997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.832+0000 7fb414e9a640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4101045c0 0x7fb4101997c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb40400d740 tx=0x7fb40400dc00 comp rx=0 tx=0).stop 2024-09-11T00:57:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.832+0000 7fb414e9a640 1 -- 172.21.15.78:0/927343599 shutdown_connections 2024-09-11T00:57:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.832+0000 7fb414e9a640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb3f80b53c0 0x7fb3f80b7880 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.832+0000 7fb414e9a640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb41010d100 0x7fb41019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.832+0000 7fb414e9a640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4101045c0 0x7fb4101997c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.832+0000 7fb414e9a640 1 --2- 172.21.15.78:0/927343599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb410103ca0 0x7fb410199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.832+0000 7fb414e9a640 1 -- 172.21.15.78:0/927343599 >> 172.21.15.78:0/927343599 conn(0x7fb4100fd700 msgr2=0x7fb4100fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.832+0000 7fb414e9a640 1 -- 172.21.15.78:0/927343599 shutdown_connections 2024-09-11T00:57:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.832+0000 7fb414e9a640 1 -- 172.21.15.78:0/927343599 wait complete. 2024-09-11T00:57:51.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.902+0000 7f66166e2640 1 -- 172.21.15.78:0/2146275352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6610108020 msgr2=0x7f661010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:51.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.902+0000 7f66166e2640 1 --2- 172.21.15.78:0/2146275352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6610108020 0x7f661010e820 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6608009ca0 tx=0x7f660802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.902+0000 7f66166e2640 1 -- 172.21.15.78:0/2146275352 shutdown_connections 2024-09-11T00:57:51.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.902+0000 7f66166e2640 1 --2- 172.21.15.78:0/2146275352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6610108020 0x7f661010e820 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.902+0000 7f66166e2640 1 --2- 172.21.15.78:0/2146275352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66101076a0 0x7f6610107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.902+0000 7f66166e2640 1 --2- 172.21.15.78:0/2146275352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6610101c90 0x7f6610102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.902+0000 7f66166e2640 1 -- 172.21.15.78:0/2146275352 >> 172.21.15.78:0/2146275352 conn(0x7f66100fd700 msgr2=0x7f66100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:51.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.902+0000 7f66166e2640 1 -- 172.21.15.78:0/2146275352 shutdown_connections 2024-09-11T00:57:51.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.902+0000 7f66166e2640 1 -- 172.21.15.78:0/2146275352 wait complete. 2024-09-11T00:57:51.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.902+0000 7f66166e2640 1 Processor -- start 2024-09-11T00:57:51.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.902+0000 7f66166e2640 1 -- start start 2024-09-11T00:57:51.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f66166e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6610101c90 0x7f6610198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:51.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f66166e2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66101076a0 0x7f66101986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:51.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f66166e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6610108020 0x7f661019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:51.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f66166e2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6610111ca0 con 0x7f6610108020 2024-09-11T00:57:51.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f66166e2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6610111b20 con 0x7f66101076a0 2024-09-11T00:57:51.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f66166e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6610111e20 con 0x7f6610101c90 2024-09-11T00:57:51.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f6614c7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6610108020 0x7f661019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:51.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f6614c7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6610108020 0x7f661019da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35214/0 (socket says 172.21.15.78:35214) 2024-09-11T00:57:51.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f6614c7f640 1 -- 172.21.15.78:0/2815705679 learned_addr learned my addr 172.21.15.78:0/2815705679 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:51.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f660ffff640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6610101c90 0x7f6610198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:51.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f6607fff640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66101076a0 0x7f66101986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:51.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f6614c7f640 1 -- 172.21.15.78:0/2815705679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6610101c90 msgr2=0x7f6610198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:51.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f6614c7f640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6610101c90 0x7f6610198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f6614c7f640 1 -- 172.21.15.78:0/2815705679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66101076a0 msgr2=0x7f66101986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:51.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f6614c7f640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66101076a0 0x7f66101986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:51.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f6614c7f640 1 -- 172.21.15.78:0/2815705679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661019e220 con 0x7f6610108020 2024-09-11T00:57:51.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f660ffff640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6610101c90 0x7f6610198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f6607fff640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66101076a0 0x7f66101986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f6614c7f640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6610108020 0x7f661019da50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6608005cc0 tx=0x7f660802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f660dffb640 1 -- 172.21.15.78:0/2815705679 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608003bf0 con 0x7f6610108020 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f660dffb640 1 -- 172.21.15.78:0/2815705679 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6608003d90 con 0x7f6610108020 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f66166e2640 1 -- 172.21.15.78:0/2815705679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661019e4b0 con 0x7f6610108020 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f660dffb640 1 -- 172.21.15.78:0/2815705679 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660802fbc0 con 0x7f6610108020 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.903+0000 7f66166e2640 1 -- 172.21.15.78:0/2815705679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661006aab0 con 0x7f6610108020 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.904+0000 7f66166e2640 1 -- 172.21.15.78:0/2815705679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6610102070 con 0x7f6610108020 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.906+0000 7f660dffb640 1 -- 172.21.15.78:0/2815705679 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f6608004610 con 0x7f6610108020 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.907+0000 7f660dffb640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f65fc0b53c0 0x7f65fc0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.907+0000 7f660ffff640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f65fc0b53c0 0x7f65fc0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.907+0000 7f660dffb640 1 -- 172.21.15.78:0/2815705679 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f66081075c0 con 0x7f6610108020 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.907+0000 7f660ffff640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f65fc0b53c0 0x7f65fc0b7880 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f66000058c0 tx=0x7f6600005ec0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:51.910+0000 7f660dffb640 1 -- 172.21.15.78:0/2815705679 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f66080ce270 con 0x7f6610108020 2024-09-11T00:57:52.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.092+0000 7f66166e2640 1 -- 172.21.15.78:0/2815705679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f661006b500 con 0x7f6610108020 2024-09-11T00:57:52.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.092+0000 7f660dffb640 1 -- 172.21.15.78:0/2815705679 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f66080cdb70 con 0x7f6610108020 2024-09-11T00:57:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:274877906974 2024-09-11T00:57:52.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.094+0000 7f66166e2640 1 -- 172.21.15.78:0/2815705679 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f65fc0b53c0 msgr2=0x7f65fc0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:52.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.094+0000 7f66166e2640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f65fc0b53c0 0x7f65fc0b7880 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f66000058c0 tx=0x7f6600005ec0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.094+0000 7f66166e2640 1 -- 172.21.15.78:0/2815705679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6610108020 msgr2=0x7f661019da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:52.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.094+0000 7f66166e2640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6610108020 0x7f661019da50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6608005cc0 tx=0x7f660802f380 comp rx=0 tx=0).stop 2024-09-11T00:57:52.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.094+0000 7f66166e2640 1 -- 172.21.15.78:0/2815705679 shutdown_connections 2024-09-11T00:57:52.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.094+0000 7f66166e2640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f65fc0b53c0 0x7f65fc0b7880 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.094+0000 7f66166e2640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6610108020 0x7f661019da50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.094+0000 7f66166e2640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66101076a0 0x7f66101986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.094+0000 7f66166e2640 1 --2- 172.21.15.78:0/2815705679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6610101c90 0x7f6610198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.095+0000 7f66166e2640 1 -- 172.21.15.78:0/2815705679 >> 172.21.15.78:0/2815705679 conn(0x7f66100fd700 msgr2=0x7f66100ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:52.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.095+0000 7f66166e2640 1 -- 172.21.15.78:0/2815705679 shutdown_connections 2024-09-11T00:57:52.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.095+0000 7f66166e2640 1 -- 172.21.15.78:0/2815705679 wait complete. 2024-09-11T00:57:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:52 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/927343599' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-11T00:57:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:52 smithi110 ceph-mon[34712]: pgmap v293: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:57:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:52 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2815705679' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-11T00:57:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:52 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/927343599' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-11T00:57:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:52 smithi112 ceph-mon[35384]: pgmap v293: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:57:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:52 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2815705679' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-11T00:57:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:52 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/927343599' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-11T00:57:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:52 smithi078 ceph-mon[31315]: pgmap v293: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:57:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:52 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2815705679' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-11T00:57:52.584 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705742 got 38654705745 for osd.0 2024-09-11T00:57:52.584 DEBUG:teuthology.parallel:result is None 2024-09-11T00:57:52.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.769+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/2251189845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ef0109130 msgr2=0x7f1ef010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:52.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.769+0000 7f1ef6bb6640 1 --2- 172.21.15.78:0/2251189845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ef0109130 0x7f1ef010f930 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009900 tx=0x7f1ee002a410 comp rx=0 tx=0).stop 2024-09-11T00:57:52.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.769+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/2251189845 shutdown_connections 2024-09-11T00:57:52.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.769+0000 7f1ef6bb6640 1 --2- 172.21.15.78:0/2251189845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ef0109130 0x7f1ef010f930 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.769+0000 7f1ef6bb6640 1 --2- 172.21.15.78:0/2251189845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ef01087b0 0x7f1ef0108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.769+0000 7f1ef6bb6640 1 --2- 172.21.15.78:0/2251189845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef0102da0 0x7f1ef0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.769+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/2251189845 >> 172.21.15.78:0/2251189845 conn(0x7f1ef00fe850 msgr2=0x7f1ef0100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.769+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/2251189845 shutdown_connections 2024-09-11T00:57:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.770+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/2251189845 wait complete. 2024-09-11T00:57:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.770+0000 7f1ef6bb6640 1 Processor -- start 2024-09-11T00:57:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.770+0000 7f1ef6bb6640 1 -- start start 2024-09-11T00:57:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.770+0000 7f1ef6bb6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ef0102da0 0x7f1ef00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.770+0000 7f1ef6bb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef01087b0 0x7f1ef00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.770+0000 7f1ef6bb6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ef0109130 0x7f1ef0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.770+0000 7f1ef6bb6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef0112e30 con 0x7f1ef0109130 2024-09-11T00:57:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.770+0000 7f1ef6bb6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef0112cb0 con 0x7f1ef0102da0 2024-09-11T00:57:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.770+0000 7f1ef6bb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef0112fb0 con 0x7f1ef01087b0 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.770+0000 7f1ef5153640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ef0109130 0x7f1ef0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.770+0000 7f1ef5153640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ef0109130 0x7f1ef0075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35236/0 (socket says 172.21.15.78:35236) 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.770+0000 7f1ef5153640 1 -- 172.21.15.78:0/4051433942 learned_addr learned my addr 172.21.15.78:0/4051433942 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1ef4952640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ef0102da0 0x7f1ef00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1eeffff640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef01087b0 0x7f1ef00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1ef5153640 1 -- 172.21.15.78:0/4051433942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef01087b0 msgr2=0x7f1ef00759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1ef5153640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef01087b0 0x7f1ef00759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1ef5153640 1 -- 172.21.15.78:0/4051433942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ef0102da0 msgr2=0x7f1ef00754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1ef5153640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ef0102da0 0x7f1ef00754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1ef5153640 1 -- 172.21.15.78:0/4051433942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef01a7840 con 0x7f1ef0109130 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1ef4952640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ef0102da0 0x7f1ef00754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1eeffff640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef01087b0 0x7f1ef00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:52.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1ef5153640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ef0109130 0x7f1ef0075f20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009a20 tx=0x7f1ee0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:52.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1eedffb640 1 -- 172.21.15.78:0/4051433942 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee003f030 con 0x7f1ef0109130 2024-09-11T00:57:52.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1eedffb640 1 -- 172.21.15.78:0/4051433942 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ee0003aa0 con 0x7f1ef0109130 2024-09-11T00:57:52.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/4051433942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef01a7ad0 con 0x7f1ef0109130 2024-09-11T00:57:52.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1eedffb640 1 -- 172.21.15.78:0/4051433942 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0035070 con 0x7f1ef0109130 2024-09-11T00:57:52.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.771+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/4051433942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef01a7fb0 con 0x7f1ef0109130 2024-09-11T00:57:52.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.772+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/4051433942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef0103200 con 0x7f1ef0109130 2024-09-11T00:57:52.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.774+0000 7f1eedffb640 1 -- 172.21.15.78:0/4051433942 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f1ee00035f0 con 0x7f1ef0109130 2024-09-11T00:57:52.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.774+0000 7f1eedffb640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1edc0b53c0 0x7f1edc0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:52.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.774+0000 7f1ef4952640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1edc0b53c0 0x7f1edc0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:52.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.775+0000 7f1eedffb640 1 -- 172.21.15.78:0/4051433942 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1ee0106c70 con 0x7f1ef0109130 2024-09-11T00:57:52.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.775+0000 7f1ef4952640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1edc0b53c0 0x7f1edc0b7880 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009930 tx=0x7f1ee8008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:52.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.777+0000 7f1eedffb640 1 -- 172.21.15.78:0/4051433942 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1ee00cd920 con 0x7f1ef0109130 2024-09-11T00:57:52.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.963+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/4051433942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1ef010a980 con 0x7f1ef0109130 2024-09-11T00:57:52.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.963+0000 7f1eedffb640 1 -- 172.21.15.78:0/4051433942 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1ee00cd220 con 0x7f1ef0109130 2024-09-11T00:57:52.965 INFO:teuthology.orchestra.run.smithi078.stdout:201863462958 2024-09-11T00:57:52.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.965+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/4051433942 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1edc0b53c0 msgr2=0x7f1edc0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:52.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.965+0000 7f1ef6bb6640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1edc0b53c0 0x7f1edc0b7880 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009930 tx=0x7f1ee8008040 comp rx=0 tx=0).stop 2024-09-11T00:57:52.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.965+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/4051433942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ef0109130 msgr2=0x7f1ef0075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:52.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.965+0000 7f1ef6bb6640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ef0109130 0x7f1ef0075f20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009a20 tx=0x7f1ee0004270 comp rx=0 tx=0).stop 2024-09-11T00:57:52.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.965+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/4051433942 shutdown_connections 2024-09-11T00:57:52.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.965+0000 7f1ef6bb6640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1edc0b53c0 0x7f1edc0b7880 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.965+0000 7f1ef6bb6640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ef0109130 0x7f1ef0075f20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.965+0000 7f1ef6bb6640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ef01087b0 0x7f1ef00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.965+0000 7f1ef6bb6640 1 --2- 172.21.15.78:0/4051433942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ef0102da0 0x7f1ef00754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:52.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.965+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/4051433942 >> 172.21.15.78:0/4051433942 conn(0x7f1ef00fe850 msgr2=0x7f1ef0100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:52.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.965+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/4051433942 shutdown_connections 2024-09-11T00:57:52.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:52.965+0000 7f1ef6bb6640 1 -- 172.21.15.78:0/4051433942 wait complete. 2024-09-11T00:57:53.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:53 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4051433942' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-11T00:57:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:53 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/4051433942' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-11T00:57:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:53 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/4051433942' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-11T00:57:53.741 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462955 got 201863462958 for osd.7 2024-09-11T00:57:53.741 DEBUG:teuthology.parallel:result is None 2024-09-11T00:57:54.202 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822709 got 154618822710 for osd.5 2024-09-11T00:57:54.203 DEBUG:teuthology.parallel:result is None 2024-09-11T00:57:54.216 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:54.217 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:54 smithi110 ceph-mon[34712]: pgmap v294: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-11T00:57:54.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:54 smithi112 ceph-mon[35384]: pgmap v294: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-11T00:57:54.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:54 smithi078 ceph-mon[31315]: pgmap v294: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-11T00:57:54.684 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743447 got 296352743448 for osd.11 2024-09-11T00:57:54.684 DEBUG:teuthology.parallel:result is None 2024-09-11T00:57:55.349 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906970 got 274877906974 for osd.10 2024-09-11T00:57:55.349 DEBUG:teuthology.parallel:result is None 2024-09-11T00:57:55.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.533+0000 7fa8f4284640 1 -- 172.21.15.78:0/1769435468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ec101cb0 msgr2=0x7fa8ec102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:55.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.533+0000 7fa8f4284640 1 --2- 172.21.15.78:0/1769435468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ec101cb0 0x7fa8ec102090 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009900 tx=0x7fa8e802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:57:55.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.533+0000 7fa8f4284640 1 -- 172.21.15.78:0/1769435468 shutdown_connections 2024-09-11T00:57:55.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.533+0000 7fa8f4284640 1 --2- 172.21.15.78:0/1769435468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ec108040 0x7fa8ec10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:55.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.533+0000 7fa8f4284640 1 --2- 172.21.15.78:0/1769435468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ec1076c0 0x7fa8ec107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:55.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.533+0000 7fa8f4284640 1 --2- 172.21.15.78:0/1769435468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ec101cb0 0x7fa8ec102090 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:55.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.533+0000 7fa8f4284640 1 -- 172.21.15.78:0/1769435468 >> 172.21.15.78:0/1769435468 conn(0x7fa8ec0fd700 msgr2=0x7fa8ec0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:55.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f4284640 1 -- 172.21.15.78:0/1769435468 shutdown_connections 2024-09-11T00:57:55.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f4284640 1 -- 172.21.15.78:0/1769435468 wait complete. 2024-09-11T00:57:55.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f4284640 1 Processor -- start 2024-09-11T00:57:55.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f4284640 1 -- start start 2024-09-11T00:57:55.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f4284640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ec101cb0 0x7fa8ec0fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:55.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f4284640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ec1076c0 0x7fa8ec0fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:55.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f4284640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ec108040 0x7fa8ec0fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:55.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f4284640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec111cc0 con 0x7fa8ec101cb0 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f4284640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec111b40 con 0x7fa8ec108040 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f4284640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec111e40 con 0x7fa8ec1076c0 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f2020640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ec101cb0 0x7fa8ec0fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f2020640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ec101cb0 0x7fa8ec0fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44140/0 (socket says 172.21.15.78:44140) 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.534+0000 7fa8f2020640 1 -- 172.21.15.78:0/1314737062 learned_addr learned my addr 172.21.15.78:0/1314737062 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.535+0000 7fa8f2020640 1 -- 172.21.15.78:0/1314737062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ec1076c0 msgr2=0x7fa8ec0fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.535+0000 7fa8f2020640 1 --2- 172.21.15.78:0/1314737062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ec1076c0 0x7fa8ec0fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.535+0000 7fa8f2020640 1 -- 172.21.15.78:0/1314737062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ec108040 msgr2=0x7fa8ec0fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.535+0000 7fa8f2020640 1 --2- 172.21.15.78:0/1314737062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ec108040 0x7fa8ec0fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.535+0000 7fa8f2020640 1 -- 172.21.15.78:0/1314737062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec1a7620 con 0x7fa8ec101cb0 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.535+0000 7fa8f2020640 1 --2- 172.21.15.78:0/1314737062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ec101cb0 0x7fa8ec0fdc00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8006010 tx=0x7fa8e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.535+0000 7fa8e37fe640 1 -- 172.21.15.78:0/1314737062 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e803f030 con 0x7fa8ec101cb0 2024-09-11T00:57:55.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.535+0000 7fa8e37fe640 1 -- 172.21.15.78:0/1314737062 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e8003800 con 0x7fa8ec101cb0 2024-09-11T00:57:55.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.535+0000 7fa8f4284640 1 -- 172.21.15.78:0/1314737062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ec1a78b0 con 0x7fa8ec101cb0 2024-09-11T00:57:55.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.535+0000 7fa8e37fe640 1 -- 172.21.15.78:0/1314737062 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8035070 con 0x7fa8ec101cb0 2024-09-11T00:57:55.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.535+0000 7fa8f4284640 1 -- 172.21.15.78:0/1314737062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ec1a7d90 con 0x7fa8ec101cb0 2024-09-11T00:57:55.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.536+0000 7fa8f4284640 1 -- 172.21.15.78:0/1314737062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8ec102090 con 0x7fa8ec101cb0 2024-09-11T00:57:55.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.537+0000 7fa8e37fe640 1 -- 172.21.15.78:0/1314737062 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fa8e8031b10 con 0x7fa8ec101cb0 2024-09-11T00:57:55.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.538+0000 7fa8e37fe640 1 --2- 172.21.15.78:0/1314737062 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa8d80b53c0 0x7fa8d80b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:55.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.538+0000 7fa8f181f640 1 --2- 172.21.15.78:0/1314737062 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa8d80b53c0 0x7fa8d80b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:55.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.538+0000 7fa8e37fe640 1 -- 172.21.15.78:0/1314737062 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fa8e8106af0 con 0x7fa8ec101cb0 2024-09-11T00:57:55.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.539+0000 7fa8f181f640 1 --2- 172.21.15.78:0/1314737062 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa8d80b53c0 0x7fa8d80b7880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0ff6f0 tx=0x7fa8dc005c70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:55.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.541+0000 7fa8e37fe640 1 -- 172.21.15.78:0/1314737062 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa8e80cd7a0 con 0x7fa8ec101cb0 2024-09-11T00:57:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:57:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.735+0000 7fa8f4284640 1 -- 172.21.15.78:0/1314737062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa8ec1a81a0 con 0x7fa8ec101cb0 2024-09-11T00:57:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.735+0000 7fa8e37fe640 1 -- 172.21.15.78:0/1314737062 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa8e80cd0a0 con 0x7fa8ec101cb0 2024-09-11T00:57:55.737 INFO:teuthology.orchestra.run.smithi078.stdout:60129542221 2024-09-11T00:57:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.738+0000 7fa8f4284640 1 -- 172.21.15.78:0/1314737062 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa8d80b53c0 msgr2=0x7fa8d80b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.738+0000 7fa8f4284640 1 --2- 172.21.15.78:0/1314737062 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa8d80b53c0 0x7fa8d80b7880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0ff6f0 tx=0x7fa8dc005c70 comp rx=0 tx=0).stop 2024-09-11T00:57:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.738+0000 7fa8f4284640 1 -- 172.21.15.78:0/1314737062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ec101cb0 msgr2=0x7fa8ec0fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.738+0000 7fa8f4284640 1 --2- 172.21.15.78:0/1314737062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ec101cb0 0x7fa8ec0fdc00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8006010 tx=0x7fa8e80042f0 comp rx=0 tx=0).stop 2024-09-11T00:57:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.738+0000 7fa8f4284640 1 -- 172.21.15.78:0/1314737062 shutdown_connections 2024-09-11T00:57:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.738+0000 7fa8f4284640 1 --2- 172.21.15.78:0/1314737062 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa8d80b53c0 0x7fa8d80b7880 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.738+0000 7fa8f4284640 1 --2- 172.21.15.78:0/1314737062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ec108040 0x7fa8ec0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.738+0000 7fa8f4284640 1 --2- 172.21.15.78:0/1314737062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ec1076c0 0x7fa8ec0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.738+0000 7fa8f4284640 1 --2- 172.21.15.78:0/1314737062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ec101cb0 0x7fa8ec0fdc00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.738+0000 7fa8f4284640 1 -- 172.21.15.78:0/1314737062 >> 172.21.15.78:0/1314737062 conn(0x7fa8ec0fd700 msgr2=0x7fa8ec06b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.738+0000 7fa8f4284640 1 -- 172.21.15.78:0/1314737062 shutdown_connections 2024-09-11T00:57:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:55.738+0000 7fa8f4284640 1 -- 172.21.15.78:0/1314737062 wait complete. 2024-09-11T00:57:55.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:55 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1314737062' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-11T00:57:56.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:55 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1314737062' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-11T00:57:56.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:55 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1314737062' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-11T00:57:56.572 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542218 got 60129542221 for osd.1 2024-09-11T00:57:56.573 DEBUG:teuthology.parallel:result is None 2024-09-11T00:57:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.812+0000 7f0994f22640 1 -- 172.21.15.78:0/615511278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f099010d140 msgr2=0x7f099010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.812+0000 7f0994f22640 1 --2- 172.21.15.78:0/615511278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f099010d140 0x7f099010f5a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0988009ca0 tx=0x7f098802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.812+0000 7f0994f22640 1 -- 172.21.15.78:0/615511278 shutdown_connections 2024-09-11T00:57:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.812+0000 7f0994f22640 1 --2- 172.21.15.78:0/615511278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f099010d140 0x7f099010f5a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.812+0000 7f0994f22640 1 --2- 172.21.15.78:0/615511278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990104600 0x7f099010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.812+0000 7f0994f22640 1 --2- 172.21.15.78:0/615511278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0990103ce0 0x7f09901040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.812+0000 7f0994f22640 1 -- 172.21.15.78:0/615511278 >> 172.21.15.78:0/615511278 conn(0x7f09900fd700 msgr2=0x7f09900ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.812+0000 7f0994f22640 1 -- 172.21.15.78:0/615511278 shutdown_connections 2024-09-11T00:57:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.812+0000 7f0994f22640 1 -- 172.21.15.78:0/615511278 wait complete. 2024-09-11T00:57:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.813+0000 7f0994f22640 1 Processor -- start 2024-09-11T00:57:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.813+0000 7f0994f22640 1 -- start start 2024-09-11T00:57:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.813+0000 7f0994f22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103ce0 0x7f09901993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.813+0000 7f0994f22640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0990104600 0x7f09901998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.813+0000 7f0994f22640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f099010d140 0x7f099019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.813+0000 7f0994f22640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0990112e90 con 0x7f0990104600 2024-09-11T00:57:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.813+0000 7f0994f22640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0990112d10 con 0x7f099010d140 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.813+0000 7f0994f22640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0990113010 con 0x7f0990103ce0 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.813+0000 7f098dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0990104600 0x7f09901998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.813+0000 7f098dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0990104600 0x7f09901998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44156/0 (socket says 172.21.15.78:44156) 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.813+0000 7f098dd9b640 1 -- 172.21.15.78:0/35768546 learned_addr learned my addr 172.21.15.78:0/35768546 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.813+0000 7f098ed9d640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f099010d140 0x7f099019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f098dd9b640 1 -- 172.21.15.78:0/35768546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103ce0 msgr2=0x7f09901993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f098e59c640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103ce0 0x7f09901993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f098dd9b640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103ce0 0x7f09901993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f098dd9b640 1 -- 172.21.15.78:0/35768546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f099010d140 msgr2=0x7f099019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f098dd9b640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f099010d140 0x7f099019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f098dd9b640 1 -- 172.21.15.78:0/35768546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099019f450 con 0x7f0990104600 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f098ed9d640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f099010d140 0x7f099019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f098dd9b640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0990104600 0x7f09901998e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f098400b3c0 tx=0x7f098400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f097f7fe640 1 -- 172.21.15.78:0/35768546 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984011830 con 0x7f0990104600 2024-09-11T00:57:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f097f7fe640 1 -- 172.21.15.78:0/35768546 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0984016070 con 0x7f0990104600 2024-09-11T00:57:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f0994f22640 1 -- 172.21.15.78:0/35768546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099019f740 con 0x7f0990104600 2024-09-11T00:57:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f097f7fe640 1 -- 172.21.15.78:0/35768546 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984010de0 con 0x7f0990104600 2024-09-11T00:57:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.814+0000 7f0994f22640 1 -- 172.21.15.78:0/35768546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09901a5e80 con 0x7f0990104600 2024-09-11T00:57:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.815+0000 7f0994f22640 1 -- 172.21.15.78:0/35768546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0990104140 con 0x7f0990104600 2024-09-11T00:57:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.816+0000 7f097f7fe640 1 -- 172.21.15.78:0/35768546 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f09840119d0 con 0x7f0990104600 2024-09-11T00:57:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.817+0000 7f097f7fe640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f09780b53c0 0x7f09780b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.818+0000 7f098e59c640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f09780b53c0 0x7f09780b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.818+0000 7f097f7fe640 1 -- 172.21.15.78:0/35768546 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f09840e5b30 con 0x7f0990104600 2024-09-11T00:57:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.818+0000 7f098e59c640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f09780b53c0 0x7f09780b7880 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f098000a810 tx=0x7f0980008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:56.821+0000 7f097f7fe640 1 -- 172.21.15.78:0/35768546 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f09840ac7e0 con 0x7f0990104600 2024-09-11T00:57:57.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.008+0000 7f0994f22640 1 -- 172.21.15.78:0/35768546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f099002e5c0 con 0x7f0990104600 2024-09-11T00:57:57.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.008+0000 7f097f7fe640 1 -- 172.21.15.78:0/35768546 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f09840ac0e0 con 0x7f0990104600 2024-09-11T00:57:57.010 INFO:teuthology.orchestra.run.smithi078.stdout:133143986238 2024-09-11T00:57:57.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.010+0000 7f0994f22640 1 -- 172.21.15.78:0/35768546 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f09780b53c0 msgr2=0x7f09780b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:57.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.010+0000 7f0994f22640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f09780b53c0 0x7f09780b7880 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f098000a810 tx=0x7f0980008040 comp rx=0 tx=0).stop 2024-09-11T00:57:57.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.010+0000 7f0994f22640 1 -- 172.21.15.78:0/35768546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0990104600 msgr2=0x7f09901998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:57.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.010+0000 7f0994f22640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0990104600 0x7f09901998e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f098400b3c0 tx=0x7f098400b880 comp rx=0 tx=0).stop 2024-09-11T00:57:57.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.010+0000 7f0994f22640 1 -- 172.21.15.78:0/35768546 shutdown_connections 2024-09-11T00:57:57.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.010+0000 7f0994f22640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f09780b53c0 0x7f09780b7880 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.010+0000 7f0994f22640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f099010d140 0x7f099019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.010+0000 7f0994f22640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0990104600 0x7f09901998e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.010+0000 7f0994f22640 1 --2- 172.21.15.78:0/35768546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0990103ce0 0x7f09901993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.010+0000 7f0994f22640 1 -- 172.21.15.78:0/35768546 >> 172.21.15.78:0/35768546 conn(0x7f09900fd700 msgr2=0x7f09900fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:57.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.010+0000 7f0994f22640 1 -- 172.21.15.78:0/35768546 shutdown_connections 2024-09-11T00:57:57.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.010+0000 7f0994f22640 1 -- 172.21.15.78:0/35768546 wait complete. 2024-09-11T00:57:57.101 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:56 smithi078 ceph-mon[31315]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-11T00:57:57.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:56 smithi110 ceph-mon[34712]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-11T00:57:57.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:56 smithi112 ceph-mon[35384]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-11T00:57:57.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.568+0000 7f8f85f56640 1 -- 172.21.15.78:0/2261418844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f80108020 msgr2=0x7f8f8010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:57.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.568+0000 7f8f85f56640 1 --2- 172.21.15.78:0/2261418844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f80108020 0x7f8f8010e820 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009ca0 tx=0x7f8f7802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.568+0000 7f8f85f56640 1 -- 172.21.15.78:0/2261418844 shutdown_connections 2024-09-11T00:57:57.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.568+0000 7f8f85f56640 1 --2- 172.21.15.78:0/2261418844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f80108020 0x7f8f8010e820 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.568+0000 7f8f85f56640 1 --2- 172.21.15.78:0/2261418844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f801076a0 0x7f8f80107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.568+0000 7f8f85f56640 1 --2- 172.21.15.78:0/2261418844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f80101c90 0x7f8f80102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.568+0000 7f8f85f56640 1 -- 172.21.15.78:0/2261418844 >> 172.21.15.78:0/2261418844 conn(0x7f8f800fd700 msgr2=0x7f8f800ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:57.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.568+0000 7f8f85f56640 1 -- 172.21.15.78:0/2261418844 shutdown_connections 2024-09-11T00:57:57.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.568+0000 7f8f85f56640 1 -- 172.21.15.78:0/2261418844 wait complete. 2024-09-11T00:57:57.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.569+0000 7f8f85f56640 1 Processor -- start 2024-09-11T00:57:57.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.569+0000 7f8f85f56640 1 -- start start 2024-09-11T00:57:57.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.569+0000 7f8f85f56640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f80101c90 0x7f8f801982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:57.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f85f56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f801076a0 0x7f8f801987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:57.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f85f56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f80108020 0x7f8f8019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f85f56640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f80111d20 con 0x7f8f801076a0 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f85f56640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f80111ba0 con 0x7f8f80101c90 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f85f56640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f80111ea0 con 0x7f8f80108020 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f801076a0 0x7f8f801987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f801076a0 0x7f8f801987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44182/0 (socket says 172.21.15.78:44182) 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7effd640 1 -- 172.21.15.78:0/1465628307 learned_addr learned my addr 172.21.15.78:0/1465628307 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7ffff640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f80108020 0x7f8f8019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7f7fe640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f80101c90 0x7f8f801982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7effd640 1 -- 172.21.15.78:0/1465628307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f80108020 msgr2=0x7f8f8019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7effd640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f80108020 0x7f8f8019db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7effd640 1 -- 172.21.15.78:0/1465628307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f80101c90 msgr2=0x7f8f801982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7effd640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f80101c90 0x7f8f801982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7effd640 1 -- 172.21.15.78:0/1465628307 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8019e360 con 0x7f8f801076a0 2024-09-11T00:57:57.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7f7fe640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f80101c90 0x7f8f801982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7effd640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f801076a0 0x7f8f801987f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400b4a0 tx=0x7f8f7400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7cff9640 1 -- 172.21.15.78:0/1465628307 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74014070 con 0x7f8f801076a0 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7cff9640 1 -- 172.21.15.78:0/1465628307 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f74010ac0 con 0x7f8f801076a0 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f85f56640 1 -- 172.21.15.78:0/1465628307 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8019e650 con 0x7f8f801076a0 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f7cff9640 1 -- 172.21.15.78:0/1465628307 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74017c30 con 0x7f8f801076a0 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.570+0000 7f8f85f56640 1 -- 172.21.15.78:0/1465628307 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8006ab30 con 0x7f8f801076a0 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.572+0000 7f8f85f56640 1 -- 172.21.15.78:0/1465628307 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f801020f0 con 0x7f8f801076a0 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.573+0000 7f8f7cff9640 1 -- 172.21.15.78:0/1465628307 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f8f74010610 con 0x7f8f801076a0 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.574+0000 7f8f7cff9640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8f680b53c0 0x7f8f680b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.574+0000 7f8f7f7fe640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8f680b53c0 0x7f8f680b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.574+0000 7f8f7cff9640 1 -- 172.21.15.78:0/1465628307 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f8f740e5430 con 0x7f8f801076a0 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.574+0000 7f8f7f7fe640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8f680b53c0 0x7f8f680b7880 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009930 tx=0x7f8f70008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:57.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.578+0000 7f8f7cff9640 1 -- 172.21.15.78:0/1465628307 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8f740ac0e0 con 0x7f8f801076a0 2024-09-11T00:57:57.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.768+0000 7f8f85f56640 1 -- 172.21.15.78:0/1465628307 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8f8006b550 con 0x7f8f801076a0 2024-09-11T00:57:57.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.769+0000 7f8f7cff9640 1 -- 172.21.15.78:0/1465628307 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8f740ab9e0 con 0x7f8f801076a0 2024-09-11T00:57:57.771 INFO:teuthology.orchestra.run.smithi078.stdout:81604378697 2024-09-11T00:57:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.771+0000 7f8f85f56640 1 -- 172.21.15.78:0/1465628307 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8f680b53c0 msgr2=0x7f8f680b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.771+0000 7f8f85f56640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8f680b53c0 0x7f8f680b7880 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009930 tx=0x7f8f70008040 comp rx=0 tx=0).stop 2024-09-11T00:57:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.771+0000 7f8f85f56640 1 -- 172.21.15.78:0/1465628307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f801076a0 msgr2=0x7f8f801987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.771+0000 7f8f85f56640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f801076a0 0x7f8f801987f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400b4a0 tx=0x7f8f7400b960 comp rx=0 tx=0).stop 2024-09-11T00:57:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.771+0000 7f8f85f56640 1 -- 172.21.15.78:0/1465628307 shutdown_connections 2024-09-11T00:57:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.771+0000 7f8f85f56640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8f680b53c0 0x7f8f680b7880 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.771+0000 7f8f85f56640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f80108020 0x7f8f8019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.771+0000 7f8f85f56640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f801076a0 0x7f8f801987f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.771+0000 7f8f85f56640 1 --2- 172.21.15.78:0/1465628307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f80101c90 0x7f8f801982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.771+0000 7f8f85f56640 1 -- 172.21.15.78:0/1465628307 >> 172.21.15.78:0/1465628307 conn(0x7f8f800fd700 msgr2=0x7f8f800ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.771+0000 7f8f85f56640 1 -- 172.21.15.78:0/1465628307 shutdown_connections 2024-09-11T00:57:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:57.771+0000 7f8f85f56640 1 -- 172.21.15.78:0/1465628307 wait complete. 2024-09-11T00:57:58.033 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:57 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/35768546' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-11T00:57:58.033 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:57 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1465628307' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-11T00:57:58.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:57 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/35768546' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-11T00:57:58.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:57 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1465628307' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-11T00:57:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:57 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/35768546' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-11T00:57:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:57 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1465628307' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-11T00:57:58.606 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986235 got 133143986238 for osd.4 2024-09-11T00:57:58.606 DEBUG:teuthology.parallel:result is None 2024-09-11T00:57:58.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.783+0000 7f1880663640 1 -- 172.21.15.78:0/331902668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1878108060 msgr2=0x7f187810e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.783+0000 7f1880663640 1 --2- 172.21.15.78:0/331902668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1878108060 0x7f187810e860 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1870009ca0 tx=0x7f187002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:57:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.783+0000 7f1880663640 1 -- 172.21.15.78:0/331902668 shutdown_connections 2024-09-11T00:57:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.783+0000 7f1880663640 1 --2- 172.21.15.78:0/331902668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1878108060 0x7f187810e860 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.783+0000 7f1880663640 1 --2- 172.21.15.78:0/331902668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18781076e0 0x7f1878107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.783+0000 7f1880663640 1 --2- 172.21.15.78:0/331902668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1878101cd0 0x7f18781020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.783+0000 7f1880663640 1 -- 172.21.15.78:0/331902668 >> 172.21.15.78:0/331902668 conn(0x7f18780fd700 msgr2=0x7f18780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.783+0000 7f1880663640 1 -- 172.21.15.78:0/331902668 shutdown_connections 2024-09-11T00:57:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.783+0000 7f1880663640 1 -- 172.21.15.78:0/331902668 wait complete. 2024-09-11T00:57:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f1880663640 1 Processor -- start 2024-09-11T00:57:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f1880663640 1 -- start start 2024-09-11T00:57:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f1880663640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1878101cd0 0x7f1878199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f1880663640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18781076e0 0x7f1878199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f1880663640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1878108060 0x7f187819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f1880663640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1878111ce0 con 0x7f18781076e0 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f1880663640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1878111b60 con 0x7f1878101cd0 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f1880663640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1878111e60 con 0x7f1878108060 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f187dbfe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18781076e0 0x7f1878199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f187dbfe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18781076e0 0x7f1878199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44206/0 (socket says 172.21.15.78:44206) 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f187dbfe640 1 -- 172.21.15.78:0/2367158115 learned_addr learned my addr 172.21.15.78:0/2367158115 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f187e3ff640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1878101cd0 0x7f1878199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.784+0000 7f187dbfe640 1 -- 172.21.15.78:0/2367158115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1878108060 msgr2=0x7f187819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.785+0000 7f187dbfe640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1878108060 0x7f187819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.785+0000 7f187dbfe640 1 -- 172.21.15.78:0/2367158115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1878101cd0 msgr2=0x7f1878199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.785+0000 7f187dbfe640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1878101cd0 0x7f1878199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.785+0000 7f187dbfe640 1 -- 172.21.15.78:0/2367158115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187819f3d0 con 0x7f18781076e0 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.785+0000 7f187e3ff640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1878101cd0 0x7f1878199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.785+0000 7f187dbfe640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18781076e0 0x7f1878199860 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f186800d770 tx=0x7f186800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.785+0000 7f186f7fe640 1 -- 172.21.15.78:0/2367158115 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868017070 con 0x7f18781076e0 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.785+0000 7f1880663640 1 -- 172.21.15.78:0/2367158115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187819f6c0 con 0x7f18781076e0 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.785+0000 7f186f7fe640 1 -- 172.21.15.78:0/2367158115 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18680054d0 con 0x7f18781076e0 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.785+0000 7f186f7fe640 1 -- 172.21.15.78:0/2367158115 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868004c30 con 0x7f18781076e0 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.785+0000 7f1880663640 1 -- 172.21.15.78:0/2367158115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18781a5d80 con 0x7f18781076e0 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.786+0000 7f1880663640 1 -- 172.21.15.78:0/2367158115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1878102f80 con 0x7f18781076e0 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.789+0000 7f186f7fe640 1 -- 172.21.15.78:0/2367158115 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f1868004dd0 con 0x7f18781076e0 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.789+0000 7f186f7fe640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18640b53c0 0x7f18640b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.789+0000 7f187e3ff640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18640b53c0 0x7f18640b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.789+0000 7f186f7fe640 1 -- 172.21.15.78:0/2367158115 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f18680e5e00 con 0x7f18781076e0 2024-09-11T00:57:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.790+0000 7f187e3ff640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18640b53c0 0x7f18640b7880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f187400baa0 tx=0x7f1874005d90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:57:58.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.792+0000 7f186f7fe640 1 -- 172.21.15.78:0/2367158115 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f18680acab0 con 0x7f18781076e0 2024-09-11T00:57:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.973+0000 7f1880663640 1 -- 172.21.15.78:0/2367158115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f187810ae80 con 0x7f18781076e0 2024-09-11T00:57:58.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:58 smithi078 ceph-mon[31315]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:57:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.974+0000 7f186f7fe640 1 -- 172.21.15.78:0/2367158115 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f18680ac3b0 con 0x7f18781076e0 2024-09-11T00:57:58.976 INFO:teuthology.orchestra.run.smithi078.stdout:253403070501 2024-09-11T00:57:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.976+0000 7f1880663640 1 -- 172.21.15.78:0/2367158115 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18640b53c0 msgr2=0x7f18640b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.976+0000 7f1880663640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18640b53c0 0x7f18640b7880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f187400baa0 tx=0x7f1874005d90 comp rx=0 tx=0).stop 2024-09-11T00:57:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.976+0000 7f1880663640 1 -- 172.21.15.78:0/2367158115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18781076e0 msgr2=0x7f1878199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:57:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.976+0000 7f1880663640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18781076e0 0x7f1878199860 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f186800d770 tx=0x7f186800dc30 comp rx=0 tx=0).stop 2024-09-11T00:57:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.976+0000 7f1880663640 1 -- 172.21.15.78:0/2367158115 shutdown_connections 2024-09-11T00:57:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.976+0000 7f1880663640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18640b53c0 0x7f18640b7880 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.976+0000 7f1880663640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1878108060 0x7f187819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.976+0000 7f1880663640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18781076e0 0x7f1878199860 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.976+0000 7f1880663640 1 --2- 172.21.15.78:0/2367158115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1878101cd0 0x7f1878199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:57:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.976+0000 7f1880663640 1 -- 172.21.15.78:0/2367158115 >> 172.21.15.78:0/2367158115 conn(0x7f18780fd700 msgr2=0x7f187806b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:57:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.976+0000 7f1880663640 1 -- 172.21.15.78:0/2367158115 shutdown_connections 2024-09-11T00:57:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:57:58.976+0000 7f1880663640 1 -- 172.21.15.78:0/2367158115 wait complete. 2024-09-11T00:57:59.092 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:58 smithi110 ceph-mon[34712]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:57:59.125 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:58 smithi112 ceph-mon[35384]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:57:59.688 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070498 got 253403070501 for osd.9 2024-09-11T00:57:59.688 DEBUG:teuthology.parallel:result is None 2024-09-11T00:57:59.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:57:59 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2367158115' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-11T00:58:00.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:57:59 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2367158115' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-11T00:58:00.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:57:59 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2367158115' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-11T00:58:00.630 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378694 got 81604378697 for osd.2 2024-09-11T00:58:00.631 DEBUG:teuthology.parallel:result is None 2024-09-11T00:58:00.631 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T00:58:00.631 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:58:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:00.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:00 smithi078 ceph-mon[31315]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:58:01.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:00 smithi110 ceph-mon[34712]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:58:01.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:00 smithi112 ceph-mon[35384]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:58:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.438+0000 7fbcc6993640 1 -- 172.21.15.78:0/1746787748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcc01087d0 msgr2=0x7fbcc0108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.438+0000 7fbcc6993640 1 --2- 172.21.15.78:0/1746787748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcc01087d0 0x7fbcc0108c10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009900 tx=0x7fbcb402a410 comp rx=0 tx=0).stop 2024-09-11T00:58:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.438+0000 7fbcc6993640 1 -- 172.21.15.78:0/1746787748 shutdown_connections 2024-09-11T00:58:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.438+0000 7fbcc6993640 1 --2- 172.21.15.78:0/1746787748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc0109150 0x7fbcc010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.438+0000 7fbcc6993640 1 --2- 172.21.15.78:0/1746787748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcc01087d0 0x7fbcc0108c10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.438+0000 7fbcc6993640 1 --2- 172.21.15.78:0/1746787748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcc0102dc0 0x7fbcc01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.438+0000 7fbcc6993640 1 -- 172.21.15.78:0/1746787748 >> 172.21.15.78:0/1746787748 conn(0x7fbcc00fe830 msgr2=0x7fbcc0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.438+0000 7fbcc6993640 1 -- 172.21.15.78:0/1746787748 shutdown_connections 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.438+0000 7fbcc6993640 1 -- 172.21.15.78:0/1746787748 wait complete. 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.439+0000 7fbcc6993640 1 Processor -- start 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.439+0000 7fbcc6993640 1 -- start start 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcc6993640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcc0102dc0 0x7fbcc0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcc6993640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcc01087d0 0x7fbcc01997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcc6993640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc0109150 0x7fbcc019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcc6993640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0112da0 con 0x7fbcc01087d0 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcc6993640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0112c20 con 0x7fbcc0102dc0 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcc6993640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0112f20 con 0x7fbcc0109150 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcc01087d0 0x7fbcc01997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcc01087d0 0x7fbcc01997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44224/0 (socket says 172.21.15.78:44224) 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbf7fe640 1 -- 172.21.15.78:0/3263556776 learned_addr learned my addr 172.21.15.78:0/3263556776 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbffff640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcc0102dc0 0x7fbcc0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcc4f30640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc0109150 0x7fbcc019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbf7fe640 1 -- 172.21.15.78:0/3263556776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc0109150 msgr2=0x7fbcc019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbf7fe640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc0109150 0x7fbcc019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbf7fe640 1 -- 172.21.15.78:0/3263556776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcc0102dc0 msgr2=0x7fbcc0199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbf7fe640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcc0102dc0 0x7fbcc0199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbf7fe640 1 -- 172.21.15.78:0/3263556776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc019f320 con 0x7fbcc01087d0 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbf7fe640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcc01087d0 0x7fbcc01997b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4005940 tx=0x7fbcb40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbd7fa640 1 -- 172.21.15.78:0/3263556776 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb403f030 con 0x7fbcc01087d0 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbd7fa640 1 -- 172.21.15.78:0/3263556776 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcb4003cb0 con 0x7fbcc01087d0 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcc6993640 1 -- 172.21.15.78:0/3263556776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc019f5b0 con 0x7fbcc01087d0 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcbd7fa640 1 -- 172.21.15.78:0/3263556776 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4035070 con 0x7fbcc01087d0 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.440+0000 7fbcc6993640 1 -- 172.21.15.78:0/3263556776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc01a5cd0 con 0x7fbcc01087d0 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.441+0000 7fbcc6993640 1 -- 172.21.15.78:0/3263556776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc01031a0 con 0x7fbcc01087d0 2024-09-11T00:58:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.443+0000 7fbcbd7fa640 1 -- 172.21.15.78:0/3263556776 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fbcb403bbf0 con 0x7fbcc01087d0 2024-09-11T00:58:01.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.443+0000 7fbcbd7fa640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbca80b53c0 0x7fbca80b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:01.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.443+0000 7fbcbffff640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbca80b53c0 0x7fbca80b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:01.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.443+0000 7fbcbd7fa640 1 -- 172.21.15.78:0/3263556776 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbcb4106c20 con 0x7fbcc01087d0 2024-09-11T00:58:01.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.444+0000 7fbcbffff640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbca80b53c0 0x7fbca80b7880 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000bbe0 tx=0x7fbcb000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:01.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.446+0000 7fbcbd7fa640 1 -- 172.21.15.78:0/3263556776 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbcb40cd220 con 0x7fbcc01087d0 2024-09-11T00:58:01.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.620+0000 7fbcc6993640 1 -- 172.21.15.78:0/3263556776 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbcc0063590 con 0x7fbca80b53c0 2024-09-11T00:58:01.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.625+0000 7fbcbd7fa640 1 -- 172.21.15.78:0/3263556776 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259007 (secure 0 0 0) 0x7fbcc0063590 con 0x7fbca80b53c0 2024-09-11T00:58:01.630 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:01.631 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T00:58:01.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.628+0000 7fbcc6993640 1 -- 172.21.15.78:0/3263556776 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbca80b53c0 msgr2=0x7fbca80b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:01.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.628+0000 7fbcc6993640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbca80b53c0 0x7fbca80b7880 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000bbe0 tx=0x7fbcb000b390 comp rx=0 tx=0).stop 2024-09-11T00:58:01.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.628+0000 7fbcc6993640 1 -- 172.21.15.78:0/3263556776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcc01087d0 msgr2=0x7fbcc01997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:01.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.628+0000 7fbcc6993640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcc01087d0 0x7fbcc01997b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4005940 tx=0x7fbcb40042f0 comp rx=0 tx=0).stop 2024-09-11T00:58:01.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.628+0000 7fbcc6993640 1 -- 172.21.15.78:0/3263556776 shutdown_connections 2024-09-11T00:58:01.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.628+0000 7fbcc6993640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbca80b53c0 0x7fbca80b7880 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:01.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.628+0000 7fbcc6993640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc0109150 0x7fbcc019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:01.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.628+0000 7fbcc6993640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcc01087d0 0x7fbcc01997b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:01.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.628+0000 7fbcc6993640 1 --2- 172.21.15.78:0/3263556776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcc0102dc0 0x7fbcc0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:01.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.628+0000 7fbcc6993640 1 -- 172.21.15.78:0/3263556776 >> 172.21.15.78:0/3263556776 conn(0x7fbcc00fe830 msgr2=0x7fbcc0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:01.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.628+0000 7fbcc6993640 1 -- 172.21.15.78:0/3263556776 shutdown_connections 2024-09-11T00:58:01.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:01.628+0000 7fbcc6993640 1 -- 172.21.15.78:0/3263556776 wait complete. 2024-09-11T00:58:02.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:01 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:58:02.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:01 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:58:02.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:01 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:58:02.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:01 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:58:02.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:01 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:58:02.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:01 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:58:02.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:01 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:58:02.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:01 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:58:02.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:01 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:58:02.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:01 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:58:02.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:01 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:58:02.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:01 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:58:02.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:01 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:58:02.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:01 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:58:02.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:01 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:58:02.297 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-11T00:58:00.035914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657762,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":664,"num_read_kb":440,"num_write":368,"num_write_kb":2285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2952965,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376408,"kb_used_data":10464,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565416,"statfs":{"total":1151940427776,"available":1151554985984,"internally_reserved":0,"allocated":10715136,"data_stored":6956994,"data_compressed":179373,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002963"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578191+0000","last_change":"2024-09-11T00:56:53.123622+0000","last_active":"2024-09-11T00:57:17.578191+0000","last_peered":"2024-09-11T00:57:17.578191+0000","last_clean":"2024-09-11T00:57:17.578191+0000","last_became_active":"2024-09-11T00:56:53.121460+0000","last_became_peered":"2024-09-11T00:56:53.121460+0000","last_unstale":"2024-09-11T00:57:17.578191+0000","last_undegraded":"2024-09-11T00:57:17.578191+0000","last_fullsized":"2024-09-11T00:57:17.578191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:30.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578612+0000","last_change":"2024-09-11T00:56:53.120189+0000","last_active":"2024-09-11T00:57:17.578612+0000","last_peered":"2024-09-11T00:57:17.578612+0000","last_clean":"2024-09-11T00:57:17.578612+0000","last_became_active":"2024-09-11T00:56:53.119853+0000","last_became_peered":"2024-09-11T00:56:53.119853+0000","last_unstale":"2024-09-11T00:57:17.578612+0000","last_undegraded":"2024-09-11T00:57:17.578612+0000","last_fullsized":"2024-09-11T00:57:17.578612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:36.927113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578824+0000","last_change":"2024-09-11T00:56:53.117832+0000","last_active":"2024-09-11T00:57:17.578824+0000","last_peered":"2024-09-11T00:57:17.578824+0000","last_clean":"2024-09-11T00:57:17.578824+0000","last_became_active":"2024-09-11T00:56:53.117508+0000","last_became_peered":"2024-09-11T00:56:53.117508+0000","last_unstale":"2024-09-11T00:57:17.578824+0000","last_undegraded":"2024-09-11T00:57:17.578824+0000","last_fullsized":"2024-09-11T00:57:17.578824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:00.628673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576984+0000","last_change":"2024-09-11T00:56:53.128302+0000","last_active":"2024-09-11T00:57:17.576984+0000","last_peered":"2024-09-11T00:57:17.576984+0000","last_clean":"2024-09-11T00:57:17.576984+0000","last_became_active":"2024-09-11T00:56:53.127584+0000","last_became_peered":"2024-09-11T00:56:53.127584+0000","last_unstale":"2024-09-11T00:57:17.576984+0000","last_undegraded":"2024-09-11T00:57:17.576984+0000","last_fullsized":"2024-09-11T00:57:17.576984+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:41.963656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580317+0000","last_change":"2024-09-11T00:56:53.116249+0000","last_active":"2024-09-11T00:57:17.580317+0000","last_peered":"2024-09-11T00:57:17.580317+0000","last_clean":"2024-09-11T00:57:17.580317+0000","last_became_active":"2024-09-11T00:56:53.115881+0000","last_became_peered":"2024-09-11T00:56:53.115881+0000","last_unstale":"2024-09-11T00:57:17.580317+0000","last_undegraded":"2024-09-11T00:57:17.580317+0000","last_fullsized":"2024-09-11T00:57:17.580317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:47.432607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578616+0000","last_change":"2024-09-11T00:56:53.116992+0000","last_active":"2024-09-11T00:57:17.578616+0000","last_peered":"2024-09-11T00:57:17.578616+0000","last_clean":"2024-09-11T00:57:17.578616+0000","last_became_active":"2024-09-11T00:56:53.116629+0000","last_became_peered":"2024-09-11T00:56:53.116629+0000","last_unstale":"2024-09-11T00:57:17.578616+0000","last_undegraded":"2024-09-11T00:57:17.578616+0000","last_fullsized":"2024-09-11T00:57:17.578616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:25.410377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586216+0000","last_change":"2024-09-11T00:56:53.127873+0000","last_active":"2024-09-11T00:57:17.586216+0000","last_peered":"2024-09-11T00:57:17.586216+0000","last_clean":"2024-09-11T00:57:17.586216+0000","last_became_active":"2024-09-11T00:56:53.127575+0000","last_became_peered":"2024-09-11T00:56:53.127575+0000","last_unstale":"2024-09-11T00:57:17.586216+0000","last_undegraded":"2024-09-11T00:57:17.586216+0000","last_fullsized":"2024-09-11T00:57:17.586216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:12.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771230+0000","last_change":"2024-09-11T00:56:53.117727+0000","last_active":"2024-09-11T00:57:17.771230+0000","last_peered":"2024-09-11T00:57:17.771230+0000","last_clean":"2024-09-11T00:57:17.771230+0000","last_became_active":"2024-09-11T00:56:53.117444+0000","last_became_peered":"2024-09-11T00:56:53.117444+0000","last_unstale":"2024-09-11T00:57:17.771230+0000","last_undegraded":"2024-09-11T00:57:17.771230+0000","last_fullsized":"2024-09-11T00:57:17.771230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:28.625231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160844+0000","last_change":"2024-09-11T00:56:53.128974+0000","last_active":"2024-09-11T00:57:18.160844+0000","last_peered":"2024-09-11T00:57:18.160844+0000","last_clean":"2024-09-11T00:57:18.160844+0000","last_became_active":"2024-09-11T00:56:53.128650+0000","last_became_peered":"2024-09-11T00:56:53.128650+0000","last_unstale":"2024-09-11T00:57:18.160844+0000","last_undegraded":"2024-09-11T00:57:18.160844+0000","last_fullsized":"2024-09-11T00:57:18.160844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:07.251373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.579078+0000","last_change":"2024-09-11T00:56:53.117816+0000","last_active":"2024-09-11T00:57:17.579078+0000","last_peered":"2024-09-11T00:57:17.579078+0000","last_clean":"2024-09-11T00:57:17.579078+0000","last_became_active":"2024-09-11T00:56:53.117449+0000","last_became_peered":"2024-09-11T00:56:53.117449+0000","last_unstale":"2024-09-11T00:57:17.579078+0000","last_undegraded":"2024-09-11T00:57:17.579078+0000","last_fullsized":"2024-09-11T00:57:17.579078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:09.058869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586597+0000","last_change":"2024-09-11T00:56:53.124616+0000","last_active":"2024-09-11T00:57:17.586597+0000","last_peered":"2024-09-11T00:57:17.586597+0000","last_clean":"2024-09-11T00:57:17.586597+0000","last_became_active":"2024-09-11T00:56:53.124309+0000","last_became_peered":"2024-09-11T00:56:53.124309+0000","last_unstale":"2024-09-11T00:57:17.586597+0000","last_undegraded":"2024-09-11T00:57:17.586597+0000","last_fullsized":"2024-09-11T00:57:17.586597+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:14.469909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577149+0000","last_change":"2024-09-11T00:56:53.126263+0000","last_active":"2024-09-11T00:57:17.577149+0000","last_peered":"2024-09-11T00:57:17.577149+0000","last_clean":"2024-09-11T00:57:17.577149+0000","last_became_active":"2024-09-11T00:56:53.125923+0000","last_became_peered":"2024-09-11T00:56:53.125923+0000","last_unstale":"2024-09-11T00:57:17.577149+0000","last_undegraded":"2024-09-11T00:57:17.577149+0000","last_fullsized":"2024-09-11T00:57:17.577149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:12:33.627372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576397+0000","last_change":"2024-09-11T00:56:53.105292+0000","last_active":"2024-09-11T00:57:17.576397+0000","last_peered":"2024-09-11T00:57:17.576397+0000","last_clean":"2024-09-11T00:57:17.576397+0000","last_became_active":"2024-09-11T00:56:53.104830+0000","last_became_peered":"2024-09-11T00:56:53.104830+0000","last_unstale":"2024-09-11T00:57:17.576397+0000","last_undegraded":"2024-09-11T00:57:17.576397+0000","last_fullsized":"2024-09-11T00:57:17.576397+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:14.136952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578749+0000","last_change":"2024-09-11T00:56:53.120463+0000","last_active":"2024-09-11T00:57:17.578749+0000","last_peered":"2024-09-11T00:57:17.578749+0000","last_clean":"2024-09-11T00:57:17.578749+0000","last_became_active":"2024-09-11T00:56:53.120167+0000","last_became_peered":"2024-09-11T00:56:53.120167+0000","last_unstale":"2024-09-11T00:57:17.578749+0000","last_undegraded":"2024-09-11T00:57:17.578749+0000","last_fullsized":"2024-09-11T00:57:17.578749+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:49.783452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578610+0000","last_change":"2024-09-11T00:56:53.123312+0000","last_active":"2024-09-11T00:57:17.578610+0000","last_peered":"2024-09-11T00:57:17.578610+0000","last_clean":"2024-09-11T00:57:17.578610+0000","last_became_active":"2024-09-11T00:56:53.123015+0000","last_became_peered":"2024-09-11T00:56:53.123015+0000","last_unstale":"2024-09-11T00:57:17.578610+0000","last_undegraded":"2024-09-11T00:57:17.578610+0000","last_fullsized":"2024-09-11T00:57:17.578610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:22.069629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160379+0000","last_change":"2024-09-11T00:56:53.125701+0000","last_active":"2024-09-11T00:57:18.160379+0000","last_peered":"2024-09-11T00:57:18.160379+0000","last_clean":"2024-09-11T00:57:18.160379+0000","last_became_active":"2024-09-11T00:56:53.125362+0000","last_became_peered":"2024-09-11T00:56:53.125362+0000","last_unstale":"2024-09-11T00:57:18.160379+0000","last_undegraded":"2024-09-11T00:57:18.160379+0000","last_fullsized":"2024-09-11T00:57:18.160379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:14.594324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578272+0000","last_change":"2024-09-11T00:56:53.124275+0000","last_active":"2024-09-11T00:57:17.578272+0000","last_peered":"2024-09-11T00:57:17.578272+0000","last_clean":"2024-09-11T00:57:17.578272+0000","last_became_active":"2024-09-11T00:56:53.122695+0000","last_became_peered":"2024-09-11T00:56:53.122695+0000","last_unstale":"2024-09-11T00:57:17.578272+0000","last_undegraded":"2024-09-11T00:57:17.578272+0000","last_fullsized":"2024-09-11T00:57:17.578272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:23.955266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580487+0000","last_change":"2024-09-11T00:56:53.119077+0000","last_active":"2024-09-11T00:57:17.580487+0000","last_peered":"2024-09-11T00:57:17.580487+0000","last_clean":"2024-09-11T00:57:17.580487+0000","last_became_active":"2024-09-11T00:56:53.118693+0000","last_became_peered":"2024-09-11T00:56:53.118693+0000","last_unstale":"2024-09-11T00:57:17.580487+0000","last_undegraded":"2024-09-11T00:57:17.580487+0000","last_fullsized":"2024-09-11T00:57:17.580487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:39.135391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.579844+0000","last_change":"2024-09-11T00:56:53.115430+0000","last_active":"2024-09-11T00:57:17.579844+0000","last_peered":"2024-09-11T00:57:17.579844+0000","last_clean":"2024-09-11T00:57:17.579844+0000","last_became_active":"2024-09-11T00:56:53.115111+0000","last_became_peered":"2024-09-11T00:56:53.115111+0000","last_unstale":"2024-09-11T00:57:17.579844+0000","last_undegraded":"2024-09-11T00:57:17.579844+0000","last_fullsized":"2024-09-11T00:57:17.579844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:11.306661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583359+0000","last_change":"2024-09-11T00:57:17.583260+0000","last_active":"2024-09-11T00:57:17.583359+0000","last_peered":"2024-09-11T00:57:17.583359+0000","last_clean":"2024-09-11T00:57:17.583359+0000","last_became_active":"2024-09-11T00:57:17.582909+0000","last_became_peered":"2024-09-11T00:57:17.582909+0000","last_unstale":"2024-09-11T00:57:17.583359+0000","last_undegraded":"2024-09-11T00:57:17.583359+0000","last_fullsized":"2024-09-11T00:57:17.583359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:42.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576404+0000","last_change":"2024-09-11T00:56:53.121303+0000","last_active":"2024-09-11T00:57:17.576404+0000","last_peered":"2024-09-11T00:57:17.576404+0000","last_clean":"2024-09-11T00:57:17.576404+0000","last_became_active":"2024-09-11T00:56:53.121004+0000","last_became_peered":"2024-09-11T00:56:53.121004+0000","last_unstale":"2024-09-11T00:57:17.576404+0000","last_undegraded":"2024-09-11T00:57:17.576404+0000","last_fullsized":"2024-09-11T00:57:17.576404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:36.899870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160496+0000","last_change":"2024-09-11T00:56:53.125042+0000","last_active":"2024-09-11T00:57:18.160496+0000","last_peered":"2024-09-11T00:57:18.160496+0000","last_clean":"2024-09-11T00:57:18.160496+0000","last_became_active":"2024-09-11T00:56:53.124661+0000","last_became_peered":"2024-09-11T00:56:53.124661+0000","last_unstale":"2024-09-11T00:57:18.160496+0000","last_undegraded":"2024-09-11T00:57:18.160496+0000","last_fullsized":"2024-09-11T00:57:18.160496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:34.392172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771270+0000","last_change":"2024-09-11T00:56:53.116544+0000","last_active":"2024-09-11T00:57:17.771270+0000","last_peered":"2024-09-11T00:57:17.771270+0000","last_clean":"2024-09-11T00:57:17.771270+0000","last_became_active":"2024-09-11T00:56:53.116262+0000","last_became_peered":"2024-09-11T00:56:53.116262+0000","last_unstale":"2024-09-11T00:57:17.771270+0000","last_undegraded":"2024-09-11T00:57:17.771270+0000","last_fullsized":"2024-09-11T00:57:17.771270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:34.500669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583244+0000","last_change":"2024-09-11T00:56:53.116967+0000","last_active":"2024-09-11T00:57:17.583244+0000","last_peered":"2024-09-11T00:57:17.583244+0000","last_clean":"2024-09-11T00:57:17.583244+0000","last_became_active":"2024-09-11T00:56:53.116405+0000","last_became_peered":"2024-09-11T00:56:53.116405+0000","last_unstale":"2024-09-11T00:57:17.583244+0000","last_undegraded":"2024-09-11T00:57:17.583244+0000","last_fullsized":"2024-09-11T00:57:17.583244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:33.521964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771594+0000","last_change":"2024-09-11T00:56:53.116727+0000","last_active":"2024-09-11T00:57:17.771594+0000","last_peered":"2024-09-11T00:57:17.771594+0000","last_clean":"2024-09-11T00:57:17.771594+0000","last_became_active":"2024-09-11T00:56:53.116416+0000","last_became_peered":"2024-09-11T00:56:53.116416+0000","last_unstale":"2024-09-11T00:57:17.771594+0000","last_undegraded":"2024-09-11T00:57:17.771594+0000","last_fullsized":"2024-09-11T00:57:17.771594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:28.277909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580269+0000","last_change":"2024-09-11T00:56:53.123141+0000","last_active":"2024-09-11T00:57:17.580269+0000","last_peered":"2024-09-11T00:57:17.580269+0000","last_clean":"2024-09-11T00:57:17.580269+0000","last_became_active":"2024-09-11T00:56:53.122811+0000","last_became_peered":"2024-09-11T00:56:53.122811+0000","last_unstale":"2024-09-11T00:57:17.580269+0000","last_undegraded":"2024-09-11T00:57:17.580269+0000","last_fullsized":"2024-09-11T00:57:17.580269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:47.175120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577673+0000","last_change":"2024-09-11T00:56:53.124213+0000","last_active":"2024-09-11T00:57:17.577673+0000","last_peered":"2024-09-11T00:57:17.577673+0000","last_clean":"2024-09-11T00:57:17.577673+0000","last_became_active":"2024-09-11T00:56:53.123757+0000","last_became_peered":"2024-09-11T00:56:53.123757+0000","last_unstale":"2024-09-11T00:57:17.577673+0000","last_undegraded":"2024-09-11T00:57:17.577673+0000","last_fullsized":"2024-09-11T00:57:17.577673+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:54.263476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576385+0000","last_change":"2024-09-11T00:56:53.114016+0000","last_active":"2024-09-11T00:57:17.576385+0000","last_peered":"2024-09-11T00:57:17.576385+0000","last_clean":"2024-09-11T00:57:17.576385+0000","last_became_active":"2024-09-11T00:56:53.113719+0000","last_became_peered":"2024-09-11T00:56:53.113719+0000","last_unstale":"2024-09-11T00:57:17.576385+0000","last_undegraded":"2024-09-11T00:57:17.576385+0000","last_fullsized":"2024-09-11T00:57:17.576385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:48:06.623810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578935+0000","last_change":"2024-09-11T00:56:53.124798+0000","last_active":"2024-09-11T00:57:17.578935+0000","last_peered":"2024-09-11T00:57:17.578935+0000","last_clean":"2024-09-11T00:57:17.578935+0000","last_became_active":"2024-09-11T00:56:53.124381+0000","last_became_peered":"2024-09-11T00:56:53.124381+0000","last_unstale":"2024-09-11T00:57:17.578935+0000","last_undegraded":"2024-09-11T00:57:17.578935+0000","last_fullsized":"2024-09-11T00:57:17.578935+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:37.823112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580396+0000","last_change":"2024-09-11T00:56:53.115297+0000","last_active":"2024-09-11T00:57:17.580396+0000","last_peered":"2024-09-11T00:57:17.580396+0000","last_clean":"2024-09-11T00:57:17.580396+0000","last_became_active":"2024-09-11T00:56:53.114958+0000","last_became_peered":"2024-09-11T00:56:53.114958+0000","last_unstale":"2024-09-11T00:57:17.580396+0000","last_undegraded":"2024-09-11T00:57:17.580396+0000","last_fullsized":"2024-09-11T00:57:17.580396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:47.612610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.579885+0000","last_change":"2024-09-11T00:56:53.126717+0000","last_active":"2024-09-11T00:57:17.579885+0000","last_peered":"2024-09-11T00:57:17.579885+0000","last_clean":"2024-09-11T00:57:17.579885+0000","last_became_active":"2024-09-11T00:56:53.126387+0000","last_became_peered":"2024-09-11T00:56:53.126387+0000","last_unstale":"2024-09-11T00:57:17.579885+0000","last_undegraded":"2024-09-11T00:57:17.579885+0000","last_fullsized":"2024-09-11T00:57:17.579885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:52.429127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577554+0000","last_change":"2024-09-11T00:56:53.126257+0000","last_active":"2024-09-11T00:57:17.577554+0000","last_peered":"2024-09-11T00:57:17.577554+0000","last_clean":"2024-09-11T00:57:17.577554+0000","last_became_active":"2024-09-11T00:56:53.125146+0000","last_became_peered":"2024-09-11T00:56:53.125146+0000","last_unstale":"2024-09-11T00:57:17.577554+0000","last_undegraded":"2024-09-11T00:57:17.577554+0000","last_fullsized":"2024-09-11T00:57:17.577554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581244+0000","last_change":"2024-09-11T00:56:53.119187+0000","last_active":"2024-09-11T00:57:17.581244+0000","last_peered":"2024-09-11T00:57:17.581244+0000","last_clean":"2024-09-11T00:57:17.581244+0000","last_became_active":"2024-09-11T00:56:53.118850+0000","last_became_peered":"2024-09-11T00:56:53.118850+0000","last_unstale":"2024-09-11T00:57:17.581244+0000","last_undegraded":"2024-09-11T00:57:17.581244+0000","last_fullsized":"2024-09-11T00:57:17.581244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:05.403876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":98,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:54.636547+0000","last_change":"2024-09-11T00:57:17.591689+0000","last_active":"2024-09-11T00:57:54.636547+0000","last_peered":"2024-09-11T00:57:54.636547+0000","last_clean":"2024-09-11T00:57:54.636547+0000","last_became_active":"2024-09-11T00:57:17.587389+0000","last_became_peered":"2024-09-11T00:57:17.587389+0000","last_unstale":"2024-09-11T00:57:54.636547+0000","last_undegraded":"2024-09-11T00:57:54.636547+0000","last_fullsized":"2024-09-11T00:57:54.636547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:25.162270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583905+0000","last_change":"2024-09-11T00:56:53.118049+0000","last_active":"2024-09-11T00:57:17.583905+0000","last_peered":"2024-09-11T00:57:17.583905+0000","last_clean":"2024-09-11T00:57:17.583905+0000","last_became_active":"2024-09-11T00:56:53.117685+0000","last_became_peered":"2024-09-11T00:56:53.117685+0000","last_unstale":"2024-09-11T00:57:17.583905+0000","last_undegraded":"2024-09-11T00:57:17.583905+0000","last_fullsized":"2024-09-11T00:57:17.583905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:17.017594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578639+0000","last_change":"2024-09-11T00:56:51.101690+0000","last_active":"2024-09-11T00:57:17.578639+0000","last_peered":"2024-09-11T00:57:17.578639+0000","last_clean":"2024-09-11T00:57:17.578639+0000","last_became_active":"2024-09-11T00:56:51.101055+0000","last_became_peered":"2024-09-11T00:56:51.101055+0000","last_unstale":"2024-09-11T00:57:17.578639+0000","last_undegraded":"2024-09-11T00:57:17.578639+0000","last_fullsized":"2024-09-11T00:57:17.578639+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:28.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576176+0000","last_change":"2024-09-11T00:56:51.101700+0000","last_active":"2024-09-11T00:57:17.576176+0000","last_peered":"2024-09-11T00:57:17.576176+0000","last_clean":"2024-09-11T00:57:17.576176+0000","last_became_active":"2024-09-11T00:56:51.101044+0000","last_became_peered":"2024-09-11T00:56:51.101044+0000","last_unstale":"2024-09-11T00:57:17.576176+0000","last_undegraded":"2024-09-11T00:57:17.576176+0000","last_fullsized":"2024-09-11T00:57:17.576176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:12.121126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771297+0000","last_change":"2024-09-11T00:56:53.121652+0000","last_active":"2024-09-11T00:57:17.771297+0000","last_peered":"2024-09-11T00:57:17.771297+0000","last_clean":"2024-09-11T00:57:17.771297+0000","last_became_active":"2024-09-11T00:56:53.121375+0000","last_became_peered":"2024-09-11T00:56:53.121375+0000","last_unstale":"2024-09-11T00:57:17.771297+0000","last_undegraded":"2024-09-11T00:57:17.771297+0000","last_fullsized":"2024-09-11T00:57:17.771297+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:39.154985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583515+0000","last_change":"2024-09-11T00:56:51.104367+0000","last_active":"2024-09-11T00:57:17.583515+0000","last_peered":"2024-09-11T00:57:17.583515+0000","last_clean":"2024-09-11T00:57:17.583515+0000","last_became_active":"2024-09-11T00:56:51.104045+0000","last_became_peered":"2024-09-11T00:56:51.104045+0000","last_unstale":"2024-09-11T00:57:17.583515+0000","last_undegraded":"2024-09-11T00:57:17.583515+0000","last_fullsized":"2024-09-11T00:57:17.583515+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:03.989437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580225+0000","last_change":"2024-09-11T00:56:53.116167+0000","last_active":"2024-09-11T00:57:17.580225+0000","last_peered":"2024-09-11T00:57:17.580225+0000","last_clean":"2024-09-11T00:57:17.580225+0000","last_became_active":"2024-09-11T00:56:53.115857+0000","last_became_peered":"2024-09-11T00:56:53.115857+0000","last_unstale":"2024-09-11T00:57:17.580225+0000","last_undegraded":"2024-09-11T00:57:17.580225+0000","last_fullsized":"2024-09-11T00:57:17.580225+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:04.476094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:58.641245+0000","last_change":"2024-09-11T00:56:51.103287+0000","last_active":"2024-09-11T00:57:58.641245+0000","last_peered":"2024-09-11T00:57:58.641245+0000","last_clean":"2024-09-11T00:57:58.641245+0000","last_became_active":"2024-09-11T00:56:51.102953+0000","last_became_peered":"2024-09-11T00:56:51.102953+0000","last_unstale":"2024-09-11T00:57:58.641245+0000","last_undegraded":"2024-09-11T00:57:58.641245+0000","last_fullsized":"2024-09-11T00:57:58.641245+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:49.701612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160775+0000","last_change":"2024-09-11T00:56:53.125130+0000","last_active":"2024-09-11T00:57:18.160775+0000","last_peered":"2024-09-11T00:57:18.160775+0000","last_clean":"2024-09-11T00:57:18.160775+0000","last_became_active":"2024-09-11T00:56:53.124442+0000","last_became_peered":"2024-09-11T00:56:53.124442+0000","last_unstale":"2024-09-11T00:57:18.160775+0000","last_undegraded":"2024-09-11T00:57:18.160775+0000","last_fullsized":"2024-09-11T00:57:18.160775+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:24.439430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586976+0000","last_change":"2024-09-11T00:56:51.104802+0000","last_active":"2024-09-11T00:57:17.586976+0000","last_peered":"2024-09-11T00:57:17.586976+0000","last_clean":"2024-09-11T00:57:17.586976+0000","last_became_active":"2024-09-11T00:56:51.104335+0000","last_became_peered":"2024-09-11T00:56:51.104335+0000","last_unstale":"2024-09-11T00:57:17.586976+0000","last_undegraded":"2024-09-11T00:57:17.586976+0000","last_fullsized":"2024-09-11T00:57:17.586976+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:26.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.585064+0000","last_change":"2024-09-11T00:57:17.584979+0000","last_active":"2024-09-11T00:57:17.585064+0000","last_peered":"2024-09-11T00:57:17.585064+0000","last_clean":"2024-09-11T00:57:17.585064+0000","last_became_active":"2024-09-11T00:57:17.584518+0000","last_became_peered":"2024-09-11T00:57:17.584518+0000","last_unstale":"2024-09-11T00:57:17.585064+0000","last_undegraded":"2024-09-11T00:57:17.585064+0000","last_fullsized":"2024-09-11T00:57:17.585064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:44.762621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583377+0000","last_change":"2024-09-11T00:56:51.154544+0000","last_active":"2024-09-11T00:57:17.583377+0000","last_peered":"2024-09-11T00:57:17.583377+0000","last_clean":"2024-09-11T00:57:17.583377+0000","last_became_active":"2024-09-11T00:56:51.154195+0000","last_became_peered":"2024-09-11T00:56:51.154195+0000","last_unstale":"2024-09-11T00:57:17.583377+0000","last_undegraded":"2024-09-11T00:57:17.583377+0000","last_fullsized":"2024-09-11T00:57:17.583377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:57:24.530947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160506+0000","last_change":"2024-09-11T00:56:53.127728+0000","last_active":"2024-09-11T00:57:18.160506+0000","last_peered":"2024-09-11T00:57:18.160506+0000","last_clean":"2024-09-11T00:57:18.160506+0000","last_became_active":"2024-09-11T00:56:53.127423+0000","last_became_peered":"2024-09-11T00:56:53.127423+0000","last_unstale":"2024-09-11T00:57:18.160506+0000","last_undegraded":"2024-09-11T00:57:18.160506+0000","last_fullsized":"2024-09-11T00:57:18.160506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:55.717123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'29","reported_seq":171,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:58.641345+0000","last_change":"2024-09-11T00:56:51.100774+0000","last_active":"2024-09-11T00:57:58.641345+0000","last_peered":"2024-09-11T00:57:58.641345+0000","last_clean":"2024-09-11T00:57:58.641345+0000","last_became_active":"2024-09-11T00:56:51.100398+0000","last_became_peered":"2024-09-11T00:56:51.100398+0000","last_unstale":"2024-09-11T00:57:58.641345+0000","last_undegraded":"2024-09-11T00:57:58.641345+0000","last_fullsized":"2024-09-11T00:57:58.641345+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:51.975236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":119,"num_read_kb":118,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581567+0000","last_change":"2024-09-11T00:56:53.120113+0000","last_active":"2024-09-11T00:57:17.581567+0000","last_peered":"2024-09-11T00:57:17.581567+0000","last_clean":"2024-09-11T00:57:17.581567+0000","last_became_active":"2024-09-11T00:56:53.119733+0000","last_became_peered":"2024-09-11T00:56:53.119733+0000","last_unstale":"2024-09-11T00:57:17.581567+0000","last_undegraded":"2024-09-11T00:57:17.581567+0000","last_fullsized":"2024-09-11T00:57:17.581567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:59.541710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580361+0000","last_change":"2024-09-11T00:56:51.104234+0000","last_active":"2024-09-11T00:57:17.580361+0000","last_peered":"2024-09-11T00:57:17.580361+0000","last_clean":"2024-09-11T00:57:17.580361+0000","last_became_active":"2024-09-11T00:56:51.103631+0000","last_became_peered":"2024-09-11T00:56:51.103631+0000","last_unstale":"2024-09-11T00:57:17.580361+0000","last_undegraded":"2024-09-11T00:57:17.580361+0000","last_fullsized":"2024-09-11T00:57:17.580361+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:47.378135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576221+0000","last_change":"2024-09-11T00:56:53.122231+0000","last_active":"2024-09-11T00:57:17.576221+0000","last_peered":"2024-09-11T00:57:17.576221+0000","last_clean":"2024-09-11T00:57:17.576221+0000","last_became_active":"2024-09-11T00:56:53.121903+0000","last_became_peered":"2024-09-11T00:56:53.121903+0000","last_unstale":"2024-09-11T00:57:17.576221+0000","last_undegraded":"2024-09-11T00:57:17.576221+0000","last_fullsized":"2024-09-11T00:57:17.576221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:44.246919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.582884+0000","last_change":"2024-09-11T00:56:51.105625+0000","last_active":"2024-09-11T00:57:17.582884+0000","last_peered":"2024-09-11T00:57:17.582884+0000","last_clean":"2024-09-11T00:57:17.582884+0000","last_became_active":"2024-09-11T00:56:51.105291+0000","last_became_peered":"2024-09-11T00:56:51.105291+0000","last_unstale":"2024-09-11T00:57:17.582884+0000","last_undegraded":"2024-09-11T00:57:17.582884+0000","last_fullsized":"2024-09-11T00:57:17.582884+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:24.237174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586551+0000","last_change":"2024-09-11T00:56:53.126272+0000","last_active":"2024-09-11T00:57:17.586551+0000","last_peered":"2024-09-11T00:57:17.586551+0000","last_clean":"2024-09-11T00:57:17.586551+0000","last_became_active":"2024-09-11T00:56:53.125964+0000","last_became_peered":"2024-09-11T00:56:53.125964+0000","last_unstale":"2024-09-11T00:57:17.586551+0000","last_undegraded":"2024-09-11T00:57:17.586551+0000","last_fullsized":"2024-09-11T00:57:17.586551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:36.331220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":83,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:54.324454+0000","last_change":"2024-09-11T00:56:51.101053+0000","last_active":"2024-09-11T00:57:54.324454+0000","last_peered":"2024-09-11T00:57:54.324454+0000","last_clean":"2024-09-11T00:57:54.324454+0000","last_became_active":"2024-09-11T00:56:51.100714+0000","last_became_peered":"2024-09-11T00:56:51.100714+0000","last_unstale":"2024-09-11T00:57:54.324454+0000","last_undegraded":"2024-09-11T00:57:54.324454+0000","last_fullsized":"2024-09-11T00:57:54.324454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:57.709126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586208+0000","last_change":"2024-09-11T00:56:53.119973+0000","last_active":"2024-09-11T00:57:17.586208+0000","last_peered":"2024-09-11T00:57:17.586208+0000","last_clean":"2024-09-11T00:57:17.586208+0000","last_became_active":"2024-09-11T00:56:53.118972+0000","last_became_peered":"2024-09-11T00:56:53.118972+0000","last_unstale":"2024-09-11T00:57:17.586208+0000","last_undegraded":"2024-09-11T00:57:17.586208+0000","last_fullsized":"2024-09-11T00:57:17.586208+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:35.912920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.598033+0000","last_change":"2024-09-11T00:57:17.597537+0000","last_active":"2024-09-11T00:57:17.598033+0000","last_peered":"2024-09-11T00:57:17.598033+0000","last_clean":"2024-09-11T00:57:17.598033+0000","last_became_active":"2024-09-11T00:57:17.581631+0000","last_became_peered":"2024-09-11T00:57:17.581631+0000","last_unstale":"2024-09-11T00:57:17.598033+0000","last_undegraded":"2024-09-11T00:57:17.598033+0000","last_fullsized":"2024-09-11T00:57:17.598033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:34.278845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.582618+0000","last_change":"2024-09-11T00:56:53.120972+0000","last_active":"2024-09-11T00:57:17.582618+0000","last_peered":"2024-09-11T00:57:17.582618+0000","last_clean":"2024-09-11T00:57:17.582618+0000","last_became_active":"2024-09-11T00:56:53.120568+0000","last_became_peered":"2024-09-11T00:56:53.120568+0000","last_unstale":"2024-09-11T00:57:17.582618+0000","last_undegraded":"2024-09-11T00:57:17.582618+0000","last_fullsized":"2024-09-11T00:57:17.582618+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:19.696201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581943+0000","last_change":"2024-09-11T00:56:51.103756+0000","last_active":"2024-09-11T00:57:17.581943+0000","last_peered":"2024-09-11T00:57:17.581943+0000","last_clean":"2024-09-11T00:57:17.581943+0000","last_became_active":"2024-09-11T00:56:51.103406+0000","last_became_peered":"2024-09-11T00:56:51.103406+0000","last_unstale":"2024-09-11T00:57:17.581943+0000","last_undegraded":"2024-09-11T00:57:17.581943+0000","last_fullsized":"2024-09-11T00:57:17.581943+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:28.022346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577747+0000","last_change":"2024-09-11T00:56:53.128628+0000","last_active":"2024-09-11T00:57:17.577747+0000","last_peered":"2024-09-11T00:57:17.577747+0000","last_clean":"2024-09-11T00:57:17.577747+0000","last_became_active":"2024-09-11T00:56:53.127197+0000","last_became_peered":"2024-09-11T00:56:53.127197+0000","last_unstale":"2024-09-11T00:57:17.577747+0000","last_undegraded":"2024-09-11T00:57:17.577747+0000","last_fullsized":"2024-09-11T00:57:17.577747+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:38.753676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576498+0000","last_change":"2024-09-11T00:56:51.101398+0000","last_active":"2024-09-11T00:57:17.576498+0000","last_peered":"2024-09-11T00:57:17.576498+0000","last_clean":"2024-09-11T00:57:17.576498+0000","last_became_active":"2024-09-11T00:56:51.101034+0000","last_became_peered":"2024-09-11T00:56:51.101034+0000","last_unstale":"2024-09-11T00:57:17.576498+0000","last_undegraded":"2024-09-11T00:57:17.576498+0000","last_fullsized":"2024-09-11T00:57:17.576498+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:40.165368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586583+0000","last_change":"2024-09-11T00:56:53.127047+0000","last_active":"2024-09-11T00:57:17.586583+0000","last_peered":"2024-09-11T00:57:17.586583+0000","last_clean":"2024-09-11T00:57:17.586583+0000","last_became_active":"2024-09-11T00:56:53.126719+0000","last_became_peered":"2024-09-11T00:56:53.126719+0000","last_unstale":"2024-09-11T00:57:17.586583+0000","last_undegraded":"2024-09-11T00:57:17.586583+0000","last_fullsized":"2024-09-11T00:57:17.586583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.686604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583494+0000","last_change":"2024-09-11T00:56:51.101665+0000","last_active":"2024-09-11T00:57:17.583494+0000","last_peered":"2024-09-11T00:57:17.583494+0000","last_clean":"2024-09-11T00:57:17.583494+0000","last_became_active":"2024-09-11T00:56:51.101332+0000","last_became_peered":"2024-09-11T00:56:51.101332+0000","last_unstale":"2024-09-11T00:57:17.583494+0000","last_undegraded":"2024-09-11T00:57:17.583494+0000","last_fullsized":"2024-09-11T00:57:17.583494+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:47.639877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586452+0000","last_change":"2024-09-11T00:56:53.128736+0000","last_active":"2024-09-11T00:57:17.586452+0000","last_peered":"2024-09-11T00:57:17.586452+0000","last_clean":"2024-09-11T00:57:17.586452+0000","last_became_active":"2024-09-11T00:56:53.128417+0000","last_became_peered":"2024-09-11T00:56:53.128417+0000","last_unstale":"2024-09-11T00:57:17.586452+0000","last_undegraded":"2024-09-11T00:57:17.586452+0000","last_fullsized":"2024-09-11T00:57:17.586452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:52.331343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":83,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:54.307336+0000","last_change":"2024-09-11T00:56:51.154733+0000","last_active":"2024-09-11T00:57:54.307336+0000","last_peered":"2024-09-11T00:57:54.307336+0000","last_clean":"2024-09-11T00:57:54.307336+0000","last_became_active":"2024-09-11T00:56:51.153613+0000","last_became_peered":"2024-09-11T00:56:51.153613+0000","last_unstale":"2024-09-11T00:57:54.307336+0000","last_undegraded":"2024-09-11T00:57:54.307336+0000","last_fullsized":"2024-09-11T00:57:54.307336+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:24.749186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586818+0000","last_change":"2024-09-11T00:56:53.124684+0000","last_active":"2024-09-11T00:57:17.586818+0000","last_peered":"2024-09-11T00:57:17.586818+0000","last_clean":"2024-09-11T00:57:17.586818+0000","last_became_active":"2024-09-11T00:56:53.124147+0000","last_became_peered":"2024-09-11T00:56:53.124147+0000","last_unstale":"2024-09-11T00:57:17.586818+0000","last_undegraded":"2024-09-11T00:57:17.586818+0000","last_fullsized":"2024-09-11T00:57:17.586818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:14.280465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586464+0000","last_change":"2024-09-11T00:55:34.500373+0000","last_active":"2024-09-11T00:57:17.586464+0000","last_peered":"2024-09-11T00:57:17.586464+0000","last_clean":"2024-09-11T00:57:17.586464+0000","last_became_active":"2024-09-11T00:55:34.495540+0000","last_became_peered":"2024-09-11T00:55:34.495540+0000","last_unstale":"2024-09-11T00:57:17.586464+0000","last_undegraded":"2024-09-11T00:57:17.586464+0000","last_fullsized":"2024-09-11T00:57:17.586464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_clean_scrub_stamp":"2024-09-11T00:52:11.168966+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:37.681466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577058+0000","last_change":"2024-09-11T00:56:51.101524+0000","last_active":"2024-09-11T00:57:17.577058+0000","last_peered":"2024-09-11T00:57:17.577058+0000","last_clean":"2024-09-11T00:57:17.577058+0000","last_became_active":"2024-09-11T00:56:51.101180+0000","last_became_peered":"2024-09-11T00:56:51.101180+0000","last_unstale":"2024-09-11T00:57:17.577058+0000","last_undegraded":"2024-09-11T00:57:17.577058+0000","last_fullsized":"2024-09-11T00:57:17.577058+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:27.293494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771467+0000","last_change":"2024-09-11T00:56:53.118503+0000","last_active":"2024-09-11T00:57:17.771467+0000","last_peered":"2024-09-11T00:57:17.771467+0000","last_clean":"2024-09-11T00:57:17.771467+0000","last_became_active":"2024-09-11T00:56:53.118174+0000","last_became_peered":"2024-09-11T00:56:53.118174+0000","last_unstale":"2024-09-11T00:57:17.771467+0000","last_undegraded":"2024-09-11T00:57:17.771467+0000","last_fullsized":"2024-09-11T00:57:17.771467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:13.524153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586788+0000","last_change":"2024-09-11T00:56:51.104222+0000","last_active":"2024-09-11T00:57:17.586788+0000","last_peered":"2024-09-11T00:57:17.586788+0000","last_clean":"2024-09-11T00:57:17.586788+0000","last_became_active":"2024-09-11T00:56:51.103913+0000","last_became_peered":"2024-09-11T00:56:51.103913+0000","last_unstale":"2024-09-11T00:57:17.586788+0000","last_undegraded":"2024-09-11T00:57:17.586788+0000","last_fullsized":"2024-09-11T00:57:17.586788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:57.467659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576162+0000","last_change":"2024-09-11T00:56:53.115130+0000","last_active":"2024-09-11T00:57:17.576162+0000","last_peered":"2024-09-11T00:57:17.576162+0000","last_clean":"2024-09-11T00:57:17.576162+0000","last_became_active":"2024-09-11T00:56:53.114813+0000","last_became_peered":"2024-09-11T00:56:53.114813+0000","last_unstale":"2024-09-11T00:57:17.576162+0000","last_undegraded":"2024-09-11T00:57:17.576162+0000","last_fullsized":"2024-09-11T00:57:17.576162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:13.265173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:56.327610+0000","last_change":"2024-09-11T00:56:51.152892+0000","last_active":"2024-09-11T00:57:56.327610+0000","last_peered":"2024-09-11T00:57:56.327610+0000","last_clean":"2024-09-11T00:57:56.327610+0000","last_became_active":"2024-09-11T00:56:51.152572+0000","last_became_peered":"2024-09-11T00:56:51.152572+0000","last_unstale":"2024-09-11T00:57:56.327610+0000","last_undegraded":"2024-09-11T00:57:56.327610+0000","last_fullsized":"2024-09-11T00:57:56.327610+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:03.774593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583356+0000","last_change":"2024-09-11T00:56:53.117416+0000","last_active":"2024-09-11T00:57:17.583356+0000","last_peered":"2024-09-11T00:57:17.583356+0000","last_clean":"2024-09-11T00:57:17.583356+0000","last_became_active":"2024-09-11T00:56:53.117016+0000","last_became_peered":"2024-09-11T00:56:53.117016+0000","last_unstale":"2024-09-11T00:57:17.583356+0000","last_undegraded":"2024-09-11T00:57:17.583356+0000","last_fullsized":"2024-09-11T00:57:17.583356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:07.607993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":83,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:56.309355+0000","last_change":"2024-09-11T00:56:51.102879+0000","last_active":"2024-09-11T00:57:56.309355+0000","last_peered":"2024-09-11T00:57:56.309355+0000","last_clean":"2024-09-11T00:57:56.309355+0000","last_became_active":"2024-09-11T00:56:51.102498+0000","last_became_peered":"2024-09-11T00:56:51.102498+0000","last_unstale":"2024-09-11T00:57:56.309355+0000","last_undegraded":"2024-09-11T00:57:56.309355+0000","last_fullsized":"2024-09-11T00:57:56.309355+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:41.360908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581368+0000","last_change":"2024-09-11T00:57:17.581261+0000","last_active":"2024-09-11T00:57:17.581368+0000","last_peered":"2024-09-11T00:57:17.581368+0000","last_clean":"2024-09-11T00:57:17.581368+0000","last_became_active":"2024-09-11T00:57:17.580941+0000","last_became_peered":"2024-09-11T00:57:17.580941+0000","last_unstale":"2024-09-11T00:57:17.581368+0000","last_undegraded":"2024-09-11T00:57:17.581368+0000","last_fullsized":"2024-09-11T00:57:17.581368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:58.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.584416+0000","last_change":"2024-09-11T00:57:17.584311+0000","last_active":"2024-09-11T00:57:17.584416+0000","last_peered":"2024-09-11T00:57:17.584416+0000","last_clean":"2024-09-11T00:57:17.584416+0000","last_became_active":"2024-09-11T00:57:17.583964+0000","last_became_peered":"2024-09-11T00:57:17.583964+0000","last_unstale":"2024-09-11T00:57:17.584416+0000","last_undegraded":"2024-09-11T00:57:17.584416+0000","last_fullsized":"2024-09-11T00:57:17.584416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:15.515947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578614+0000","last_change":"2024-09-11T00:56:53.112550+0000","last_active":"2024-09-11T00:57:17.578614+0000","last_peered":"2024-09-11T00:57:17.578614+0000","last_clean":"2024-09-11T00:57:17.578614+0000","last_became_active":"2024-09-11T00:56:53.112179+0000","last_became_peered":"2024-09-11T00:56:53.112179+0000","last_unstale":"2024-09-11T00:57:17.578614+0000","last_undegraded":"2024-09-11T00:57:17.578614+0000","last_fullsized":"2024-09-11T00:57:17.578614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:03.339839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581699+0000","last_change":"2024-09-11T00:56:51.152957+0000","last_active":"2024-09-11T00:57:17.581699+0000","last_peered":"2024-09-11T00:57:17.581699+0000","last_clean":"2024-09-11T00:57:17.581699+0000","last_became_active":"2024-09-11T00:56:51.152616+0000","last_became_peered":"2024-09-11T00:56:51.152616+0000","last_unstale":"2024-09-11T00:57:17.581699+0000","last_undegraded":"2024-09-11T00:57:17.581699+0000","last_fullsized":"2024-09-11T00:57:17.581699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:02.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576410+0000","last_change":"2024-09-11T00:56:53.115155+0000","last_active":"2024-09-11T00:57:17.576410+0000","last_peered":"2024-09-11T00:57:17.576410+0000","last_clean":"2024-09-11T00:57:17.576410+0000","last_became_active":"2024-09-11T00:56:53.114865+0000","last_became_peered":"2024-09-11T00:56:53.114865+0000","last_unstale":"2024-09-11T00:57:17.576410+0000","last_undegraded":"2024-09-11T00:57:17.576410+0000","last_fullsized":"2024-09-11T00:57:17.576410+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:23.968888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'16","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.122556+0000","last_change":"2024-09-11T00:56:51.100887+0000","last_active":"2024-09-11T00:57:18.122556+0000","last_peered":"2024-09-11T00:57:18.122556+0000","last_clean":"2024-09-11T00:57:18.122556+0000","last_became_active":"2024-09-11T00:56:51.100474+0000","last_became_peered":"2024-09-11T00:56:51.100474+0000","last_unstale":"2024-09-11T00:57:18.122556+0000","last_undegraded":"2024-09-11T00:57:18.122556+0000","last_fullsized":"2024-09-11T00:57:18.122556+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:40.884871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.588425+0000","last_change":"2024-09-11T00:57:17.588086+0000","last_active":"2024-09-11T00:57:17.588425+0000","last_peered":"2024-09-11T00:57:17.588425+0000","last_clean":"2024-09-11T00:57:17.588425+0000","last_became_active":"2024-09-11T00:57:17.587785+0000","last_became_peered":"2024-09-11T00:57:17.587785+0000","last_unstale":"2024-09-11T00:57:17.588425+0000","last_undegraded":"2024-09-11T00:57:17.588425+0000","last_fullsized":"2024-09-11T00:57:17.588425+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:49.494588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577254+0000","last_change":"2024-09-11T00:56:51.102031+0000","last_active":"2024-09-11T00:57:17.577254+0000","last_peered":"2024-09-11T00:57:17.577254+0000","last_clean":"2024-09-11T00:57:17.577254+0000","last_became_active":"2024-09-11T00:56:51.100614+0000","last_became_peered":"2024-09-11T00:56:51.100614+0000","last_unstale":"2024-09-11T00:57:17.577254+0000","last_undegraded":"2024-09-11T00:57:17.577254+0000","last_fullsized":"2024-09-11T00:57:17.577254+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:52.611261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578722+0000","last_change":"2024-09-11T00:56:53.124806+0000","last_active":"2024-09-11T00:57:17.578722+0000","last_peered":"2024-09-11T00:57:17.578722+0000","last_clean":"2024-09-11T00:57:17.578722+0000","last_became_active":"2024-09-11T00:56:53.124452+0000","last_became_peered":"2024-09-11T00:56:53.124452+0000","last_unstale":"2024-09-11T00:57:17.578722+0000","last_undegraded":"2024-09-11T00:57:17.578722+0000","last_fullsized":"2024-09-11T00:57:17.578722+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:57.934808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160509+0000","last_change":"2024-09-11T00:56:51.153621+0000","last_active":"2024-09-11T00:57:18.160509+0000","last_peered":"2024-09-11T00:57:18.160509+0000","last_clean":"2024-09-11T00:57:18.160509+0000","last_became_active":"2024-09-11T00:56:51.153315+0000","last_became_peered":"2024-09-11T00:56:51.153315+0000","last_unstale":"2024-09-11T00:57:18.160509+0000","last_undegraded":"2024-09-11T00:57:18.160509+0000","last_fullsized":"2024-09-11T00:57:18.160509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:10.058295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580192+0000","last_change":"2024-09-11T00:56:53.114463+0000","last_active":"2024-09-11T00:57:17.580192+0000","last_peered":"2024-09-11T00:57:17.580192+0000","last_clean":"2024-09-11T00:57:17.580192+0000","last_became_active":"2024-09-11T00:56:53.114099+0000","last_became_peered":"2024-09-11T00:56:53.114099+0000","last_unstale":"2024-09-11T00:57:17.580192+0000","last_undegraded":"2024-09-11T00:57:17.580192+0000","last_fullsized":"2024-09-11T00:57:17.580192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:58.981787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771377+0000","last_change":"2024-09-11T00:56:51.096942+0000","last_active":"2024-09-11T00:57:17.771377+0000","last_peered":"2024-09-11T00:57:17.771377+0000","last_clean":"2024-09-11T00:57:17.771377+0000","last_became_active":"2024-09-11T00:56:51.096584+0000","last_became_peered":"2024-09-11T00:56:51.096584+0000","last_unstale":"2024-09-11T00:57:17.771377+0000","last_undegraded":"2024-09-11T00:57:17.771377+0000","last_fullsized":"2024-09-11T00:57:17.771377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:03.484123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577668+0000","last_change":"2024-09-11T00:56:53.126804+0000","last_active":"2024-09-11T00:57:17.577668+0000","last_peered":"2024-09-11T00:57:17.577668+0000","last_clean":"2024-09-11T00:57:17.577668+0000","last_became_active":"2024-09-11T00:56:53.125408+0000","last_became_peered":"2024-09-11T00:56:53.125408+0000","last_unstale":"2024-09-11T00:57:17.577668+0000","last_undegraded":"2024-09-11T00:57:17.577668+0000","last_fullsized":"2024-09-11T00:57:17.577668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:46.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580130+0000","last_change":"2024-09-11T00:56:51.103755+0000","last_active":"2024-09-11T00:57:17.580130+0000","last_peered":"2024-09-11T00:57:17.580130+0000","last_clean":"2024-09-11T00:57:17.580130+0000","last_became_active":"2024-09-11T00:56:51.103395+0000","last_became_peered":"2024-09-11T00:56:51.103395+0000","last_unstale":"2024-09-11T00:57:17.580130+0000","last_undegraded":"2024-09-11T00:57:17.580130+0000","last_fullsized":"2024-09-11T00:57:17.580130+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:16.782660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771488+0000","last_change":"2024-09-11T00:56:53.121926+0000","last_active":"2024-09-11T00:57:17.771488+0000","last_peered":"2024-09-11T00:57:17.771488+0000","last_clean":"2024-09-11T00:57:17.771488+0000","last_became_active":"2024-09-11T00:56:53.121603+0000","last_became_peered":"2024-09-11T00:56:53.121603+0000","last_unstale":"2024-09-11T00:57:17.771488+0000","last_undegraded":"2024-09-11T00:57:17.771488+0000","last_fullsized":"2024-09-11T00:57:17.771488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:31.021030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580437+0000","last_change":"2024-09-11T00:56:51.107221+0000","last_active":"2024-09-11T00:57:17.580437+0000","last_peered":"2024-09-11T00:57:17.580437+0000","last_clean":"2024-09-11T00:57:17.580437+0000","last_became_active":"2024-09-11T00:56:51.106849+0000","last_became_peered":"2024-09-11T00:56:51.106849+0000","last_unstale":"2024-09-11T00:57:17.580437+0000","last_undegraded":"2024-09-11T00:57:17.580437+0000","last_fullsized":"2024-09-11T00:57:17.580437+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:46.497230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586475+0000","last_change":"2024-09-11T00:56:53.120525+0000","last_active":"2024-09-11T00:57:17.586475+0000","last_peered":"2024-09-11T00:57:17.586475+0000","last_clean":"2024-09-11T00:57:17.586475+0000","last_became_active":"2024-09-11T00:56:53.120228+0000","last_became_peered":"2024-09-11T00:56:53.120228+0000","last_unstale":"2024-09-11T00:57:17.586475+0000","last_undegraded":"2024-09-11T00:57:17.586475+0000","last_fullsized":"2024-09-11T00:57:17.586475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:52.047687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583268+0000","last_change":"2024-09-11T00:56:53.120311+0000","last_active":"2024-09-11T00:57:17.583268+0000","last_peered":"2024-09-11T00:57:17.583268+0000","last_clean":"2024-09-11T00:57:17.583268+0000","last_became_active":"2024-09-11T00:56:53.119648+0000","last_became_peered":"2024-09-11T00:56:53.119648+0000","last_unstale":"2024-09-11T00:57:17.583268+0000","last_undegraded":"2024-09-11T00:57:17.583268+0000","last_fullsized":"2024-09-11T00:57:17.583268+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:21.187784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581118+0000","last_change":"2024-09-11T00:56:51.103631+0000","last_active":"2024-09-11T00:57:17.581118+0000","last_peered":"2024-09-11T00:57:17.581118+0000","last_clean":"2024-09-11T00:57:17.581118+0000","last_became_active":"2024-09-11T00:56:51.103281+0000","last_became_peered":"2024-09-11T00:56:51.103281+0000","last_unstale":"2024-09-11T00:57:17.581118+0000","last_undegraded":"2024-09-11T00:57:17.581118+0000","last_fullsized":"2024-09-11T00:57:17.581118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:16.910956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160481+0000","last_change":"2024-09-11T00:56:53.124752+0000","last_active":"2024-09-11T00:57:18.160481+0000","last_peered":"2024-09-11T00:57:18.160481+0000","last_clean":"2024-09-11T00:57:18.160481+0000","last_became_active":"2024-09-11T00:56:53.124245+0000","last_became_peered":"2024-09-11T00:56:53.124245+0000","last_unstale":"2024-09-11T00:57:18.160481+0000","last_undegraded":"2024-09-11T00:57:18.160481+0000","last_fullsized":"2024-09-11T00:57:18.160481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:10.277903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581704+0000","last_change":"2024-09-11T00:56:51.104624+0000","last_active":"2024-09-11T00:57:17.581704+0000","last_peered":"2024-09-11T00:57:17.581704+0000","last_clean":"2024-09-11T00:57:17.581704+0000","last_became_active":"2024-09-11T00:56:51.104041+0000","last_became_peered":"2024-09-11T00:56:51.104041+0000","last_unstale":"2024-09-11T00:57:17.581704+0000","last_undegraded":"2024-09-11T00:57:17.581704+0000","last_fullsized":"2024-09-11T00:57:17.581704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:10.397455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.579287+0000","last_change":"2024-09-11T00:56:53.118547+0000","last_active":"2024-09-11T00:57:17.579287+0000","last_peered":"2024-09-11T00:57:17.579287+0000","last_clean":"2024-09-11T00:57:17.579287+0000","last_became_active":"2024-09-11T00:56:53.118291+0000","last_became_peered":"2024-09-11T00:56:53.118291+0000","last_unstale":"2024-09-11T00:57:17.579287+0000","last_undegraded":"2024-09-11T00:57:17.579287+0000","last_fullsized":"2024-09-11T00:57:17.579287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:42.012173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771739+0000","last_change":"2024-09-11T00:56:51.153047+0000","last_active":"2024-09-11T00:57:17.771739+0000","last_peered":"2024-09-11T00:57:17.771739+0000","last_clean":"2024-09-11T00:57:17.771739+0000","last_became_active":"2024-09-11T00:56:51.152564+0000","last_became_peered":"2024-09-11T00:56:51.152564+0000","last_unstale":"2024-09-11T00:57:17.771739+0000","last_undegraded":"2024-09-11T00:57:17.771739+0000","last_fullsized":"2024-09-11T00:57:17.771739+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:16.849087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771630+0000","last_change":"2024-09-11T00:56:53.119926+0000","last_active":"2024-09-11T00:57:17.771630+0000","last_peered":"2024-09-11T00:57:17.771630+0000","last_clean":"2024-09-11T00:57:17.771630+0000","last_became_active":"2024-09-11T00:56:53.119643+0000","last_became_peered":"2024-09-11T00:56:53.119643+0000","last_unstale":"2024-09-11T00:57:17.771630+0000","last_undegraded":"2024-09-11T00:57:17.771630+0000","last_fullsized":"2024-09-11T00:57:17.771630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:51.838958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":92,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:57.310232+0000","last_change":"2024-09-11T00:56:51.100837+0000","last_active":"2024-09-11T00:57:57.310232+0000","last_peered":"2024-09-11T00:57:57.310232+0000","last_clean":"2024-09-11T00:57:57.310232+0000","last_became_active":"2024-09-11T00:56:51.100494+0000","last_became_peered":"2024-09-11T00:56:51.100494+0000","last_unstale":"2024-09-11T00:57:57.310232+0000","last_undegraded":"2024-09-11T00:57:57.310232+0000","last_fullsized":"2024-09-11T00:57:57.310232+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:39.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67160,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":583,"num_read_kb":371,"num_write":243,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286865,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743450,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":851968,"data_stored":416613,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906976,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1067877,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070502,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":868352,"data_stored":416431,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266730,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":856064,"data_stored":416697,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":917504,"data_stored":1009327,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626484,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":416148,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822713,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":1009005,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":860160,"data_stored":417022,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":868352,"data_stored":415973,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":477006,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":913408,"data_stored":477408,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705747,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":417487,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:58:02.352 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:58:02.598 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:02.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:02 smithi078 ceph-mon[31315]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:02.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:02 smithi078 ceph-mon[31315]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.075+0000 7fb8e54cc640 1 -- 172.21.15.78:0/265855331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e01087d0 msgr2=0x7fb8e0108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.075+0000 7fb8e54cc640 1 --2- 172.21.15.78:0/265855331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e01087d0 0x7fb8e0108c10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009900 tx=0x7fb8c802a410 comp rx=0 tx=0).stop 2024-09-11T00:58:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.075+0000 7fb8e54cc640 1 -- 172.21.15.78:0/265855331 shutdown_connections 2024-09-11T00:58:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.075+0000 7fb8e54cc640 1 --2- 172.21.15.78:0/265855331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e0109150 0x7fb8e010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.075+0000 7fb8e54cc640 1 --2- 172.21.15.78:0/265855331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e01087d0 0x7fb8e0108c10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.075+0000 7fb8e54cc640 1 --2- 172.21.15.78:0/265855331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8e0102dc0 0x7fb8e01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.075+0000 7fb8e54cc640 1 -- 172.21.15.78:0/265855331 >> 172.21.15.78:0/265855331 conn(0x7fb8e00fe830 msgr2=0x7fb8e0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.075+0000 7fb8e54cc640 1 -- 172.21.15.78:0/265855331 shutdown_connections 2024-09-11T00:58:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.075+0000 7fb8e54cc640 1 -- 172.21.15.78:0/265855331 wait complete. 2024-09-11T00:58:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.075+0000 7fb8e54cc640 1 Processor -- start 2024-09-11T00:58:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.075+0000 7fb8e54cc640 1 -- start start 2024-09-11T00:58:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8e54cc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e0102dc0 0x7fb8e01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8e54cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8e01087d0 0x7fb8e0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8e54cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e0109150 0x7fb8e019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8e54cc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e0112e10 con 0x7fb8e0102dc0 2024-09-11T00:58:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8e54cc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e0112c90 con 0x7fb8e01087d0 2024-09-11T00:58:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8e54cc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e0112f90 con 0x7fb8e0109150 2024-09-11T00:58:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8deffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e0102dc0 0x7fb8e01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8deffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e0102dc0 0x7fb8e01992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44236/0 (socket says 172.21.15.78:44236) 2024-09-11T00:58:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8deffd640 1 -- 172.21.15.78:0/1130245484 learned_addr learned my addr 172.21.15.78:0/1130245484 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8de7fc640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8e01087d0 0x7fb8e0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8df7fe640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e0109150 0x7fb8e019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8deffd640 1 -- 172.21.15.78:0/1130245484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e0109150 msgr2=0x7fb8e019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8deffd640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e0109150 0x7fb8e019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8deffd640 1 -- 172.21.15.78:0/1130245484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8e01087d0 msgr2=0x7fb8e0199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8deffd640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8e01087d0 0x7fb8e0199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8deffd640 1 -- 172.21.15.78:0/1130245484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e019f390 con 0x7fb8e0102dc0 2024-09-11T00:58:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8deffd640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e0102dc0 0x7fb8e01992e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800eab0 tx=0x7fb8d800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8d7fff640 1 -- 172.21.15.78:0/1130245484 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800ce40 con 0x7fb8e0102dc0 2024-09-11T00:58:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8d7fff640 1 -- 172.21.15.78:0/1130245484 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d8011920 con 0x7fb8e0102dc0 2024-09-11T00:58:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8e54cc640 1 -- 172.21.15.78:0/1130245484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e019f680 con 0x7fb8e0102dc0 2024-09-11T00:58:03.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.076+0000 7fb8d7fff640 1 -- 172.21.15.78:0/1130245484 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8010a30 con 0x7fb8e0102dc0 2024-09-11T00:58:03.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.077+0000 7fb8e54cc640 1 -- 172.21.15.78:0/1130245484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e01a5dc0 con 0x7fb8e0102dc0 2024-09-11T00:58:03.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.078+0000 7fb8e54cc640 1 -- 172.21.15.78:0/1130245484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e01031a0 con 0x7fb8e0102dc0 2024-09-11T00:58:03.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.079+0000 7fb8d7fff640 1 -- 172.21.15.78:0/1130245484 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fb8d800f040 con 0x7fb8e0102dc0 2024-09-11T00:58:03.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.080+0000 7fb8d7fff640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb8cc0b53c0 0x7fb8cc0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:03.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.080+0000 7fb8de7fc640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb8cc0b53c0 0x7fb8cc0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:03.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.080+0000 7fb8d7fff640 1 -- 172.21.15.78:0/1130245484 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb8d80e61a0 con 0x7fb8e0102dc0 2024-09-11T00:58:03.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.080+0000 7fb8de7fc640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb8cc0b53c0 0x7fb8cc0b7880 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8031440 tx=0x7fb8c80095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:03.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.082+0000 7fb8d7fff640 1 -- 172.21.15.78:0/1130245484 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb8d8016030 con 0x7fb8e0102dc0 2024-09-11T00:58:03.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:02 smithi112 ceph-mon[35384]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:03.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:02 smithi112 ceph-mon[35384]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:03.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.256+0000 7fb8e54cc640 1 -- 172.21.15.78:0/1130245484 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8e0063590 con 0x7fb8cc0b53c0 2024-09-11T00:58:03.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.263+0000 7fb8d7fff640 1 -- 172.21.15.78:0/1130245484 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259008 (secure 0 0 0) 0x7fb8e0063590 con 0x7fb8cc0b53c0 2024-09-11T00:58:03.267 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:03.268 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T00:58:03.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.265+0000 7fb8e54cc640 1 -- 172.21.15.78:0/1130245484 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb8cc0b53c0 msgr2=0x7fb8cc0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:03.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.265+0000 7fb8e54cc640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb8cc0b53c0 0x7fb8cc0b7880 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8031440 tx=0x7fb8c80095f0 comp rx=0 tx=0).stop 2024-09-11T00:58:03.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.265+0000 7fb8e54cc640 1 -- 172.21.15.78:0/1130245484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e0102dc0 msgr2=0x7fb8e01992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:03.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.265+0000 7fb8e54cc640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e0102dc0 0x7fb8e01992e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800eab0 tx=0x7fb8d800ef70 comp rx=0 tx=0).stop 2024-09-11T00:58:03.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.265+0000 7fb8e54cc640 1 -- 172.21.15.78:0/1130245484 shutdown_connections 2024-09-11T00:58:03.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.265+0000 7fb8e54cc640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb8cc0b53c0 0x7fb8cc0b7880 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:03.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.265+0000 7fb8e54cc640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e0109150 0x7fb8e019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:03.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.265+0000 7fb8e54cc640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8e01087d0 0x7fb8e0199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:03.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.265+0000 7fb8e54cc640 1 --2- 172.21.15.78:0/1130245484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e0102dc0 0x7fb8e01992e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:03.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.265+0000 7fb8e54cc640 1 -- 172.21.15.78:0/1130245484 >> 172.21.15.78:0/1130245484 conn(0x7fb8e00fe830 msgr2=0x7fb8e0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:03.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.266+0000 7fb8e54cc640 1 -- 172.21.15.78:0/1130245484 shutdown_connections 2024-09-11T00:58:03.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:03.266+0000 7fb8e54cc640 1 -- 172.21.15.78:0/1130245484 wait complete. 2024-09-11T00:58:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:02 smithi110 ceph-mon[34712]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:02 smithi110 ceph-mon[34712]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-11T00:58:02.036437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657762,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":679,"num_read_kb":440,"num_write":368,"num_write_kb":2285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2952965,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376408,"kb_used_data":10464,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565416,"statfs":{"total":1151940427776,"available":1151554985984,"internally_reserved":0,"allocated":10715136,"data_stored":6956994,"data_compressed":179373,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003004"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578191+0000","last_change":"2024-09-11T00:56:53.123622+0000","last_active":"2024-09-11T00:57:17.578191+0000","last_peered":"2024-09-11T00:57:17.578191+0000","last_clean":"2024-09-11T00:57:17.578191+0000","last_became_active":"2024-09-11T00:56:53.121460+0000","last_became_peered":"2024-09-11T00:56:53.121460+0000","last_unstale":"2024-09-11T00:57:17.578191+0000","last_undegraded":"2024-09-11T00:57:17.578191+0000","last_fullsized":"2024-09-11T00:57:17.578191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:30.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578612+0000","last_change":"2024-09-11T00:56:53.120189+0000","last_active":"2024-09-11T00:57:17.578612+0000","last_peered":"2024-09-11T00:57:17.578612+0000","last_clean":"2024-09-11T00:57:17.578612+0000","last_became_active":"2024-09-11T00:56:53.119853+0000","last_became_peered":"2024-09-11T00:56:53.119853+0000","last_unstale":"2024-09-11T00:57:17.578612+0000","last_undegraded":"2024-09-11T00:57:17.578612+0000","last_fullsized":"2024-09-11T00:57:17.578612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:36.927113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578824+0000","last_change":"2024-09-11T00:56:53.117832+0000","last_active":"2024-09-11T00:57:17.578824+0000","last_peered":"2024-09-11T00:57:17.578824+0000","last_clean":"2024-09-11T00:57:17.578824+0000","last_became_active":"2024-09-11T00:56:53.117508+0000","last_became_peered":"2024-09-11T00:56:53.117508+0000","last_unstale":"2024-09-11T00:57:17.578824+0000","last_undegraded":"2024-09-11T00:57:17.578824+0000","last_fullsized":"2024-09-11T00:57:17.578824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:00.628673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576984+0000","last_change":"2024-09-11T00:56:53.128302+0000","last_active":"2024-09-11T00:57:17.576984+0000","last_peered":"2024-09-11T00:57:17.576984+0000","last_clean":"2024-09-11T00:57:17.576984+0000","last_became_active":"2024-09-11T00:56:53.127584+0000","last_became_peered":"2024-09-11T00:56:53.127584+0000","last_unstale":"2024-09-11T00:57:17.576984+0000","last_undegraded":"2024-09-11T00:57:17.576984+0000","last_fullsized":"2024-09-11T00:57:17.576984+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:41.963656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580317+0000","last_change":"2024-09-11T00:56:53.116249+0000","last_active":"2024-09-11T00:57:17.580317+0000","last_peered":"2024-09-11T00:57:17.580317+0000","last_clean":"2024-09-11T00:57:17.580317+0000","last_became_active":"2024-09-11T00:56:53.115881+0000","last_became_peered":"2024-09-11T00:56:53.115881+0000","last_unstale":"2024-09-11T00:57:17.580317+0000","last_undegraded":"2024-09-11T00:57:17.580317+0000","last_fullsized":"2024-09-11T00:57:17.580317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:47.432607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578616+0000","last_change":"2024-09-11T00:56:53.116992+0000","last_active":"2024-09-11T00:57:17.578616+0000","last_peered":"2024-09-11T00:57:17.578616+0000","last_clean":"2024-09-11T00:57:17.578616+0000","last_became_active":"2024-09-11T00:56:53.116629+0000","last_became_peered":"2024-09-11T00:56:53.116629+0000","last_unstale":"2024-09-11T00:57:17.578616+0000","last_undegraded":"2024-09-11T00:57:17.578616+0000","last_fullsized":"2024-09-11T00:57:17.578616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:25.410377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586216+0000","last_change":"2024-09-11T00:56:53.127873+0000","last_active":"2024-09-11T00:57:17.586216+0000","last_peered":"2024-09-11T00:57:17.586216+0000","last_clean":"2024-09-11T00:57:17.586216+0000","last_became_active":"2024-09-11T00:56:53.127575+0000","last_became_peered":"2024-09-11T00:56:53.127575+0000","last_unstale":"2024-09-11T00:57:17.586216+0000","last_undegraded":"2024-09-11T00:57:17.586216+0000","last_fullsized":"2024-09-11T00:57:17.586216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:12.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771230+0000","last_change":"2024-09-11T00:56:53.117727+0000","last_active":"2024-09-11T00:57:17.771230+0000","last_peered":"2024-09-11T00:57:17.771230+0000","last_clean":"2024-09-11T00:57:17.771230+0000","last_became_active":"2024-09-11T00:56:53.117444+0000","last_became_peered":"2024-09-11T00:56:53.117444+0000","last_unstale":"2024-09-11T00:57:17.771230+0000","last_undegraded":"2024-09-11T00:57:17.771230+0000","last_fullsized":"2024-09-11T00:57:17.771230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:28.625231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160844+0000","last_change":"2024-09-11T00:56:53.128974+0000","last_active":"2024-09-11T00:57:18.160844+0000","last_peered":"2024-09-11T00:57:18.160844+0000","last_clean":"2024-09-11T00:57:18.160844+0000","last_became_active":"2024-09-11T00:56:53.128650+0000","last_became_peered":"2024-09-11T00:56:53.128650+0000","last_unstale":"2024-09-11T00:57:18.160844+0000","last_undegraded":"2024-09-11T00:57:18.160844+0000","last_fullsized":"2024-09-11T00:57:18.160844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:07.251373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.579078+0000","last_change":"2024-09-11T00:56:53.117816+0000","last_active":"2024-09-11T00:57:17.579078+0000","last_peered":"2024-09-11T00:57:17.579078+0000","last_clean":"2024-09-11T00:57:17.579078+0000","last_became_active":"2024-09-11T00:56:53.117449+0000","last_became_peered":"2024-09-11T00:56:53.117449+0000","last_unstale":"2024-09-11T00:57:17.579078+0000","last_undegraded":"2024-09-11T00:57:17.579078+0000","last_fullsized":"2024-09-11T00:57:17.579078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:09.058869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586597+0000","last_change":"2024-09-11T00:56:53.124616+0000","last_active":"2024-09-11T00:57:17.586597+0000","last_peered":"2024-09-11T00:57:17.586597+0000","last_clean":"2024-09-11T00:57:17.586597+0000","last_became_active":"2024-09-11T00:56:53.124309+0000","last_became_peered":"2024-09-11T00:56:53.124309+0000","last_unstale":"2024-09-11T00:57:17.586597+0000","last_undegraded":"2024-09-11T00:57:17.586597+0000","last_fullsized":"2024-09-11T00:57:17.586597+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:14.469909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577149+0000","last_change":"2024-09-11T00:56:53.126263+0000","last_active":"2024-09-11T00:57:17.577149+0000","last_peered":"2024-09-11T00:57:17.577149+0000","last_clean":"2024-09-11T00:57:17.577149+0000","last_became_active":"2024-09-11T00:56:53.125923+0000","last_became_peered":"2024-09-11T00:56:53.125923+0000","last_unstale":"2024-09-11T00:57:17.577149+0000","last_undegraded":"2024-09-11T00:57:17.577149+0000","last_fullsized":"2024-09-11T00:57:17.577149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:12:33.627372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576397+0000","last_change":"2024-09-11T00:56:53.105292+0000","last_active":"2024-09-11T00:57:17.576397+0000","last_peered":"2024-09-11T00:57:17.576397+0000","last_clean":"2024-09-11T00:57:17.576397+0000","last_became_active":"2024-09-11T00:56:53.104830+0000","last_became_peered":"2024-09-11T00:56:53.104830+0000","last_unstale":"2024-09-11T00:57:17.576397+0000","last_undegraded":"2024-09-11T00:57:17.576397+0000","last_fullsized":"2024-09-11T00:57:17.576397+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:14.136952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578749+0000","last_change":"2024-09-11T00:56:53.120463+0000","last_active":"2024-09-11T00:57:17.578749+0000","last_peered":"2024-09-11T00:57:17.578749+0000","last_clean":"2024-09-11T00:57:17.578749+0000","last_became_active":"2024-09-11T00:56:53.120167+0000","last_became_peered":"2024-09-11T00:56:53.120167+0000","last_unstale":"2024-09-11T00:57:17.578749+0000","last_undegraded":"2024-09-11T00:57:17.578749+0000","last_fullsized":"2024-09-11T00:57:17.578749+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:49.783452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578610+0000","last_change":"2024-09-11T00:56:53.123312+0000","last_active":"2024-09-11T00:57:17.578610+0000","last_peered":"2024-09-11T00:57:17.578610+0000","last_clean":"2024-09-11T00:57:17.578610+0000","last_became_active":"2024-09-11T00:56:53.123015+0000","last_became_peered":"2024-09-11T00:56:53.123015+0000","last_unstale":"2024-09-11T00:57:17.578610+0000","last_undegraded":"2024-09-11T00:57:17.578610+0000","last_fullsized":"2024-09-11T00:57:17.578610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:22.069629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160379+0000","last_change":"2024-09-11T00:56:53.125701+0000","last_active":"2024-09-11T00:57:18.160379+0000","last_peered":"2024-09-11T00:57:18.160379+0000","last_clean":"2024-09-11T00:57:18.160379+0000","last_became_active":"2024-09-11T00:56:53.125362+0000","last_became_peered":"2024-09-11T00:56:53.125362+0000","last_unstale":"2024-09-11T00:57:18.160379+0000","last_undegraded":"2024-09-11T00:57:18.160379+0000","last_fullsized":"2024-09-11T00:57:18.160379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:14.594324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578272+0000","last_change":"2024-09-11T00:56:53.124275+0000","last_active":"2024-09-11T00:57:17.578272+0000","last_peered":"2024-09-11T00:57:17.578272+0000","last_clean":"2024-09-11T00:57:17.578272+0000","last_became_active":"2024-09-11T00:56:53.122695+0000","last_became_peered":"2024-09-11T00:56:53.122695+0000","last_unstale":"2024-09-11T00:57:17.578272+0000","last_undegraded":"2024-09-11T00:57:17.578272+0000","last_fullsized":"2024-09-11T00:57:17.578272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:23.955266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580487+0000","last_change":"2024-09-11T00:56:53.119077+0000","last_active":"2024-09-11T00:57:17.580487+0000","last_peered":"2024-09-11T00:57:17.580487+0000","last_clean":"2024-09-11T00:57:17.580487+0000","last_became_active":"2024-09-11T00:56:53.118693+0000","last_became_peered":"2024-09-11T00:56:53.118693+0000","last_unstale":"2024-09-11T00:57:17.580487+0000","last_undegraded":"2024-09-11T00:57:17.580487+0000","last_fullsized":"2024-09-11T00:57:17.580487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:39.135391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.579844+0000","last_change":"2024-09-11T00:56:53.115430+0000","last_active":"2024-09-11T00:57:17.579844+0000","last_peered":"2024-09-11T00:57:17.579844+0000","last_clean":"2024-09-11T00:57:17.579844+0000","last_became_active":"2024-09-11T00:56:53.115111+0000","last_became_peered":"2024-09-11T00:56:53.115111+0000","last_unstale":"2024-09-11T00:57:17.579844+0000","last_undegraded":"2024-09-11T00:57:17.579844+0000","last_fullsized":"2024-09-11T00:57:17.579844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:11.306661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583359+0000","last_change":"2024-09-11T00:57:17.583260+0000","last_active":"2024-09-11T00:57:17.583359+0000","last_peered":"2024-09-11T00:57:17.583359+0000","last_clean":"2024-09-11T00:57:17.583359+0000","last_became_active":"2024-09-11T00:57:17.582909+0000","last_became_peered":"2024-09-11T00:57:17.582909+0000","last_unstale":"2024-09-11T00:57:17.583359+0000","last_undegraded":"2024-09-11T00:57:17.583359+0000","last_fullsized":"2024-09-11T00:57:17.583359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:42.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576404+0000","last_change":"2024-09-11T00:56:53.121303+0000","last_active":"2024-09-11T00:57:17.576404+0000","last_peered":"2024-09-11T00:57:17.576404+0000","last_clean":"2024-09-11T00:57:17.576404+0000","last_became_active":"2024-09-11T00:56:53.121004+0000","last_became_peered":"2024-09-11T00:56:53.121004+0000","last_unstale":"2024-09-11T00:57:17.576404+0000","last_undegraded":"2024-09-11T00:57:17.576404+0000","last_fullsized":"2024-09-11T00:57:17.576404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:36.899870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160496+0000","last_change":"2024-09-11T00:56:53.125042+0000","last_active":"2024-09-11T00:57:18.160496+0000","last_peered":"2024-09-11T00:57:18.160496+0000","last_clean":"2024-09-11T00:57:18.160496+0000","last_became_active":"2024-09-11T00:56:53.124661+0000","last_became_peered":"2024-09-11T00:56:53.124661+0000","last_unstale":"2024-09-11T00:57:18.160496+0000","last_undegraded":"2024-09-11T00:57:18.160496+0000","last_fullsized":"2024-09-11T00:57:18.160496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:34.392172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771270+0000","last_change":"2024-09-11T00:56:53.116544+0000","last_active":"2024-09-11T00:57:17.771270+0000","last_peered":"2024-09-11T00:57:17.771270+0000","last_clean":"2024-09-11T00:57:17.771270+0000","last_became_active":"2024-09-11T00:56:53.116262+0000","last_became_peered":"2024-09-11T00:56:53.116262+0000","last_unstale":"2024-09-11T00:57:17.771270+0000","last_undegraded":"2024-09-11T00:57:17.771270+0000","last_fullsized":"2024-09-11T00:57:17.771270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:34.500669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583244+0000","last_change":"2024-09-11T00:56:53.116967+0000","last_active":"2024-09-11T00:57:17.583244+0000","last_peered":"2024-09-11T00:57:17.583244+0000","last_clean":"2024-09-11T00:57:17.583244+0000","last_became_active":"2024-09-11T00:56:53.116405+0000","last_became_peered":"2024-09-11T00:56:53.116405+0000","last_unstale":"2024-09-11T00:57:17.583244+0000","last_undegraded":"2024-09-11T00:57:17.583244+0000","last_fullsized":"2024-09-11T00:57:17.583244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:33.521964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771594+0000","last_change":"2024-09-11T00:56:53.116727+0000","last_active":"2024-09-11T00:57:17.771594+0000","last_peered":"2024-09-11T00:57:17.771594+0000","last_clean":"2024-09-11T00:57:17.771594+0000","last_became_active":"2024-09-11T00:56:53.116416+0000","last_became_peered":"2024-09-11T00:56:53.116416+0000","last_unstale":"2024-09-11T00:57:17.771594+0000","last_undegraded":"2024-09-11T00:57:17.771594+0000","last_fullsized":"2024-09-11T00:57:17.771594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:28.277909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580269+0000","last_change":"2024-09-11T00:56:53.123141+0000","last_active":"2024-09-11T00:57:17.580269+0000","last_peered":"2024-09-11T00:57:17.580269+0000","last_clean":"2024-09-11T00:57:17.580269+0000","last_became_active":"2024-09-11T00:56:53.122811+0000","last_became_peered":"2024-09-11T00:56:53.122811+0000","last_unstale":"2024-09-11T00:57:17.580269+0000","last_undegraded":"2024-09-11T00:57:17.580269+0000","last_fullsized":"2024-09-11T00:57:17.580269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:47.175120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577673+0000","last_change":"2024-09-11T00:56:53.124213+0000","last_active":"2024-09-11T00:57:17.577673+0000","last_peered":"2024-09-11T00:57:17.577673+0000","last_clean":"2024-09-11T00:57:17.577673+0000","last_became_active":"2024-09-11T00:56:53.123757+0000","last_became_peered":"2024-09-11T00:56:53.123757+0000","last_unstale":"2024-09-11T00:57:17.577673+0000","last_undegraded":"2024-09-11T00:57:17.577673+0000","last_fullsized":"2024-09-11T00:57:17.577673+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:54.263476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576385+0000","last_change":"2024-09-11T00:56:53.114016+0000","last_active":"2024-09-11T00:57:17.576385+0000","last_peered":"2024-09-11T00:57:17.576385+0000","last_clean":"2024-09-11T00:57:17.576385+0000","last_became_active":"2024-09-11T00:56:53.113719+0000","last_became_peered":"2024-09-11T00:56:53.113719+0000","last_unstale":"2024-09-11T00:57:17.576385+0000","last_undegraded":"2024-09-11T00:57:17.576385+0000","last_fullsized":"2024-09-11T00:57:17.576385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:48:06.623810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578935+0000","last_change":"2024-09-11T00:56:53.124798+0000","last_active":"2024-09-11T00:57:17.578935+0000","last_peered":"2024-09-11T00:57:17.578935+0000","last_clean":"2024-09-11T00:57:17.578935+0000","last_became_active":"2024-09-11T00:56:53.124381+0000","last_became_peered":"2024-09-11T00:56:53.124381+0000","last_unstale":"2024-09-11T00:57:17.578935+0000","last_undegraded":"2024-09-11T00:57:17.578935+0000","last_fullsized":"2024-09-11T00:57:17.578935+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:37.823112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580396+0000","last_change":"2024-09-11T00:56:53.115297+0000","last_active":"2024-09-11T00:57:17.580396+0000","last_peered":"2024-09-11T00:57:17.580396+0000","last_clean":"2024-09-11T00:57:17.580396+0000","last_became_active":"2024-09-11T00:56:53.114958+0000","last_became_peered":"2024-09-11T00:56:53.114958+0000","last_unstale":"2024-09-11T00:57:17.580396+0000","last_undegraded":"2024-09-11T00:57:17.580396+0000","last_fullsized":"2024-09-11T00:57:17.580396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:47.612610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.579885+0000","last_change":"2024-09-11T00:56:53.126717+0000","last_active":"2024-09-11T00:57:17.579885+0000","last_peered":"2024-09-11T00:57:17.579885+0000","last_clean":"2024-09-11T00:57:17.579885+0000","last_became_active":"2024-09-11T00:56:53.126387+0000","last_became_peered":"2024-09-11T00:56:53.126387+0000","last_unstale":"2024-09-11T00:57:17.579885+0000","last_undegraded":"2024-09-11T00:57:17.579885+0000","last_fullsized":"2024-09-11T00:57:17.579885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:52.429127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577554+0000","last_change":"2024-09-11T00:56:53.126257+0000","last_active":"2024-09-11T00:57:17.577554+0000","last_peered":"2024-09-11T00:57:17.577554+0000","last_clean":"2024-09-11T00:57:17.577554+0000","last_became_active":"2024-09-11T00:56:53.125146+0000","last_became_peered":"2024-09-11T00:56:53.125146+0000","last_unstale":"2024-09-11T00:57:17.577554+0000","last_undegraded":"2024-09-11T00:57:17.577554+0000","last_fullsized":"2024-09-11T00:57:17.577554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581244+0000","last_change":"2024-09-11T00:56:53.119187+0000","last_active":"2024-09-11T00:57:17.581244+0000","last_peered":"2024-09-11T00:57:17.581244+0000","last_clean":"2024-09-11T00:57:17.581244+0000","last_became_active":"2024-09-11T00:56:53.118850+0000","last_became_peered":"2024-09-11T00:56:53.118850+0000","last_unstale":"2024-09-11T00:57:17.581244+0000","last_undegraded":"2024-09-11T00:57:17.581244+0000","last_fullsized":"2024-09-11T00:57:17.581244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:05.403876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":103,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:59.643030+0000","last_change":"2024-09-11T00:57:17.591689+0000","last_active":"2024-09-11T00:57:59.643030+0000","last_peered":"2024-09-11T00:57:59.643030+0000","last_clean":"2024-09-11T00:57:59.643030+0000","last_became_active":"2024-09-11T00:57:17.587389+0000","last_became_peered":"2024-09-11T00:57:17.587389+0000","last_unstale":"2024-09-11T00:57:59.643030+0000","last_undegraded":"2024-09-11T00:57:59.643030+0000","last_fullsized":"2024-09-11T00:57:59.643030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:25.162270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583905+0000","last_change":"2024-09-11T00:56:53.118049+0000","last_active":"2024-09-11T00:57:17.583905+0000","last_peered":"2024-09-11T00:57:17.583905+0000","last_clean":"2024-09-11T00:57:17.583905+0000","last_became_active":"2024-09-11T00:56:53.117685+0000","last_became_peered":"2024-09-11T00:56:53.117685+0000","last_unstale":"2024-09-11T00:57:17.583905+0000","last_undegraded":"2024-09-11T00:57:17.583905+0000","last_fullsized":"2024-09-11T00:57:17.583905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:17.017594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578639+0000","last_change":"2024-09-11T00:56:51.101690+0000","last_active":"2024-09-11T00:57:17.578639+0000","last_peered":"2024-09-11T00:57:17.578639+0000","last_clean":"2024-09-11T00:57:17.578639+0000","last_became_active":"2024-09-11T00:56:51.101055+0000","last_became_peered":"2024-09-11T00:56:51.101055+0000","last_unstale":"2024-09-11T00:57:17.578639+0000","last_undegraded":"2024-09-11T00:57:17.578639+0000","last_fullsized":"2024-09-11T00:57:17.578639+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:28.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576176+0000","last_change":"2024-09-11T00:56:51.101700+0000","last_active":"2024-09-11T00:57:17.576176+0000","last_peered":"2024-09-11T00:57:17.576176+0000","last_clean":"2024-09-11T00:57:17.576176+0000","last_became_active":"2024-09-11T00:56:51.101044+0000","last_became_peered":"2024-09-11T00:56:51.101044+0000","last_unstale":"2024-09-11T00:57:17.576176+0000","last_undegraded":"2024-09-11T00:57:17.576176+0000","last_fullsized":"2024-09-11T00:57:17.576176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:12.121126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771297+0000","last_change":"2024-09-11T00:56:53.121652+0000","last_active":"2024-09-11T00:57:17.771297+0000","last_peered":"2024-09-11T00:57:17.771297+0000","last_clean":"2024-09-11T00:57:17.771297+0000","last_became_active":"2024-09-11T00:56:53.121375+0000","last_became_peered":"2024-09-11T00:56:53.121375+0000","last_unstale":"2024-09-11T00:57:17.771297+0000","last_undegraded":"2024-09-11T00:57:17.771297+0000","last_fullsized":"2024-09-11T00:57:17.771297+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:39.154985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583515+0000","last_change":"2024-09-11T00:56:51.104367+0000","last_active":"2024-09-11T00:57:17.583515+0000","last_peered":"2024-09-11T00:57:17.583515+0000","last_clean":"2024-09-11T00:57:17.583515+0000","last_became_active":"2024-09-11T00:56:51.104045+0000","last_became_peered":"2024-09-11T00:56:51.104045+0000","last_unstale":"2024-09-11T00:57:17.583515+0000","last_undegraded":"2024-09-11T00:57:17.583515+0000","last_fullsized":"2024-09-11T00:57:17.583515+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:03.989437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580225+0000","last_change":"2024-09-11T00:56:53.116167+0000","last_active":"2024-09-11T00:57:17.580225+0000","last_peered":"2024-09-11T00:57:17.580225+0000","last_clean":"2024-09-11T00:57:17.580225+0000","last_became_active":"2024-09-11T00:56:53.115857+0000","last_became_peered":"2024-09-11T00:56:53.115857+0000","last_unstale":"2024-09-11T00:57:17.580225+0000","last_undegraded":"2024-09-11T00:57:17.580225+0000","last_fullsized":"2024-09-11T00:57:17.580225+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:04.476094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:58.641245+0000","last_change":"2024-09-11T00:56:51.103287+0000","last_active":"2024-09-11T00:57:58.641245+0000","last_peered":"2024-09-11T00:57:58.641245+0000","last_clean":"2024-09-11T00:57:58.641245+0000","last_became_active":"2024-09-11T00:56:51.102953+0000","last_became_peered":"2024-09-11T00:56:51.102953+0000","last_unstale":"2024-09-11T00:57:58.641245+0000","last_undegraded":"2024-09-11T00:57:58.641245+0000","last_fullsized":"2024-09-11T00:57:58.641245+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:49.701612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160775+0000","last_change":"2024-09-11T00:56:53.125130+0000","last_active":"2024-09-11T00:57:18.160775+0000","last_peered":"2024-09-11T00:57:18.160775+0000","last_clean":"2024-09-11T00:57:18.160775+0000","last_became_active":"2024-09-11T00:56:53.124442+0000","last_became_peered":"2024-09-11T00:56:53.124442+0000","last_unstale":"2024-09-11T00:57:18.160775+0000","last_undegraded":"2024-09-11T00:57:18.160775+0000","last_fullsized":"2024-09-11T00:57:18.160775+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:24.439430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586976+0000","last_change":"2024-09-11T00:56:51.104802+0000","last_active":"2024-09-11T00:57:17.586976+0000","last_peered":"2024-09-11T00:57:17.586976+0000","last_clean":"2024-09-11T00:57:17.586976+0000","last_became_active":"2024-09-11T00:56:51.104335+0000","last_became_peered":"2024-09-11T00:56:51.104335+0000","last_unstale":"2024-09-11T00:57:17.586976+0000","last_undegraded":"2024-09-11T00:57:17.586976+0000","last_fullsized":"2024-09-11T00:57:17.586976+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:26.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.585064+0000","last_change":"2024-09-11T00:57:17.584979+0000","last_active":"2024-09-11T00:57:17.585064+0000","last_peered":"2024-09-11T00:57:17.585064+0000","last_clean":"2024-09-11T00:57:17.585064+0000","last_became_active":"2024-09-11T00:57:17.584518+0000","last_became_peered":"2024-09-11T00:57:17.584518+0000","last_unstale":"2024-09-11T00:57:17.585064+0000","last_undegraded":"2024-09-11T00:57:17.585064+0000","last_fullsized":"2024-09-11T00:57:17.585064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:44.762621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583377+0000","last_change":"2024-09-11T00:56:51.154544+0000","last_active":"2024-09-11T00:57:17.583377+0000","last_peered":"2024-09-11T00:57:17.583377+0000","last_clean":"2024-09-11T00:57:17.583377+0000","last_became_active":"2024-09-11T00:56:51.154195+0000","last_became_peered":"2024-09-11T00:56:51.154195+0000","last_unstale":"2024-09-11T00:57:17.583377+0000","last_undegraded":"2024-09-11T00:57:17.583377+0000","last_fullsized":"2024-09-11T00:57:17.583377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:57:24.530947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160506+0000","last_change":"2024-09-11T00:56:53.127728+0000","last_active":"2024-09-11T00:57:18.160506+0000","last_peered":"2024-09-11T00:57:18.160506+0000","last_clean":"2024-09-11T00:57:18.160506+0000","last_became_active":"2024-09-11T00:56:53.127423+0000","last_became_peered":"2024-09-11T00:56:53.127423+0000","last_unstale":"2024-09-11T00:57:18.160506+0000","last_undegraded":"2024-09-11T00:57:18.160506+0000","last_fullsized":"2024-09-11T00:57:18.160506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:55.717123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'29","reported_seq":171,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:58.641345+0000","last_change":"2024-09-11T00:56:51.100774+0000","last_active":"2024-09-11T00:57:58.641345+0000","last_peered":"2024-09-11T00:57:58.641345+0000","last_clean":"2024-09-11T00:57:58.641345+0000","last_became_active":"2024-09-11T00:56:51.100398+0000","last_became_peered":"2024-09-11T00:56:51.100398+0000","last_unstale":"2024-09-11T00:57:58.641345+0000","last_undegraded":"2024-09-11T00:57:58.641345+0000","last_fullsized":"2024-09-11T00:57:58.641345+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:51.975236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":119,"num_read_kb":118,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581567+0000","last_change":"2024-09-11T00:56:53.120113+0000","last_active":"2024-09-11T00:57:17.581567+0000","last_peered":"2024-09-11T00:57:17.581567+0000","last_clean":"2024-09-11T00:57:17.581567+0000","last_became_active":"2024-09-11T00:56:53.119733+0000","last_became_peered":"2024-09-11T00:56:53.119733+0000","last_unstale":"2024-09-11T00:57:17.581567+0000","last_undegraded":"2024-09-11T00:57:17.581567+0000","last_fullsized":"2024-09-11T00:57:17.581567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:59.541710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580361+0000","last_change":"2024-09-11T00:56:51.104234+0000","last_active":"2024-09-11T00:57:17.580361+0000","last_peered":"2024-09-11T00:57:17.580361+0000","last_clean":"2024-09-11T00:57:17.580361+0000","last_became_active":"2024-09-11T00:56:51.103631+0000","last_became_peered":"2024-09-11T00:56:51.103631+0000","last_unstale":"2024-09-11T00:57:17.580361+0000","last_undegraded":"2024-09-11T00:57:17.580361+0000","last_fullsized":"2024-09-11T00:57:17.580361+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:47.378135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576221+0000","last_change":"2024-09-11T00:56:53.122231+0000","last_active":"2024-09-11T00:57:17.576221+0000","last_peered":"2024-09-11T00:57:17.576221+0000","last_clean":"2024-09-11T00:57:17.576221+0000","last_became_active":"2024-09-11T00:56:53.121903+0000","last_became_peered":"2024-09-11T00:56:53.121903+0000","last_unstale":"2024-09-11T00:57:17.576221+0000","last_undegraded":"2024-09-11T00:57:17.576221+0000","last_fullsized":"2024-09-11T00:57:17.576221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:44.246919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.582884+0000","last_change":"2024-09-11T00:56:51.105625+0000","last_active":"2024-09-11T00:57:17.582884+0000","last_peered":"2024-09-11T00:57:17.582884+0000","last_clean":"2024-09-11T00:57:17.582884+0000","last_became_active":"2024-09-11T00:56:51.105291+0000","last_became_peered":"2024-09-11T00:56:51.105291+0000","last_unstale":"2024-09-11T00:57:17.582884+0000","last_undegraded":"2024-09-11T00:57:17.582884+0000","last_fullsized":"2024-09-11T00:57:17.582884+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:24.237174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586551+0000","last_change":"2024-09-11T00:56:53.126272+0000","last_active":"2024-09-11T00:57:17.586551+0000","last_peered":"2024-09-11T00:57:17.586551+0000","last_clean":"2024-09-11T00:57:17.586551+0000","last_became_active":"2024-09-11T00:56:53.125964+0000","last_became_peered":"2024-09-11T00:56:53.125964+0000","last_unstale":"2024-09-11T00:57:17.586551+0000","last_undegraded":"2024-09-11T00:57:17.586551+0000","last_fullsized":"2024-09-11T00:57:17.586551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:36.331220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":88,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:59.334389+0000","last_change":"2024-09-11T00:56:51.101053+0000","last_active":"2024-09-11T00:57:59.334389+0000","last_peered":"2024-09-11T00:57:59.334389+0000","last_clean":"2024-09-11T00:57:59.334389+0000","last_became_active":"2024-09-11T00:56:51.100714+0000","last_became_peered":"2024-09-11T00:56:51.100714+0000","last_unstale":"2024-09-11T00:57:59.334389+0000","last_undegraded":"2024-09-11T00:57:59.334389+0000","last_fullsized":"2024-09-11T00:57:59.334389+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:57.709126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586208+0000","last_change":"2024-09-11T00:56:53.119973+0000","last_active":"2024-09-11T00:57:17.586208+0000","last_peered":"2024-09-11T00:57:17.586208+0000","last_clean":"2024-09-11T00:57:17.586208+0000","last_became_active":"2024-09-11T00:56:53.118972+0000","last_became_peered":"2024-09-11T00:56:53.118972+0000","last_unstale":"2024-09-11T00:57:17.586208+0000","last_undegraded":"2024-09-11T00:57:17.586208+0000","last_fullsized":"2024-09-11T00:57:17.586208+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:35.912920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.598033+0000","last_change":"2024-09-11T00:57:17.597537+0000","last_active":"2024-09-11T00:57:17.598033+0000","last_peered":"2024-09-11T00:57:17.598033+0000","last_clean":"2024-09-11T00:57:17.598033+0000","last_became_active":"2024-09-11T00:57:17.581631+0000","last_became_peered":"2024-09-11T00:57:17.581631+0000","last_unstale":"2024-09-11T00:57:17.598033+0000","last_undegraded":"2024-09-11T00:57:17.598033+0000","last_fullsized":"2024-09-11T00:57:17.598033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:34.278845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.582618+0000","last_change":"2024-09-11T00:56:53.120972+0000","last_active":"2024-09-11T00:57:17.582618+0000","last_peered":"2024-09-11T00:57:17.582618+0000","last_clean":"2024-09-11T00:57:17.582618+0000","last_became_active":"2024-09-11T00:56:53.120568+0000","last_became_peered":"2024-09-11T00:56:53.120568+0000","last_unstale":"2024-09-11T00:57:17.582618+0000","last_undegraded":"2024-09-11T00:57:17.582618+0000","last_fullsized":"2024-09-11T00:57:17.582618+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:19.696201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581943+0000","last_change":"2024-09-11T00:56:51.103756+0000","last_active":"2024-09-11T00:57:17.581943+0000","last_peered":"2024-09-11T00:57:17.581943+0000","last_clean":"2024-09-11T00:57:17.581943+0000","last_became_active":"2024-09-11T00:56:51.103406+0000","last_became_peered":"2024-09-11T00:56:51.103406+0000","last_unstale":"2024-09-11T00:57:17.581943+0000","last_undegraded":"2024-09-11T00:57:17.581943+0000","last_fullsized":"2024-09-11T00:57:17.581943+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:28.022346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577747+0000","last_change":"2024-09-11T00:56:53.128628+0000","last_active":"2024-09-11T00:57:17.577747+0000","last_peered":"2024-09-11T00:57:17.577747+0000","last_clean":"2024-09-11T00:57:17.577747+0000","last_became_active":"2024-09-11T00:56:53.127197+0000","last_became_peered":"2024-09-11T00:56:53.127197+0000","last_unstale":"2024-09-11T00:57:17.577747+0000","last_undegraded":"2024-09-11T00:57:17.577747+0000","last_fullsized":"2024-09-11T00:57:17.577747+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:38.753676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576498+0000","last_change":"2024-09-11T00:56:51.101398+0000","last_active":"2024-09-11T00:57:17.576498+0000","last_peered":"2024-09-11T00:57:17.576498+0000","last_clean":"2024-09-11T00:57:17.576498+0000","last_became_active":"2024-09-11T00:56:51.101034+0000","last_became_peered":"2024-09-11T00:56:51.101034+0000","last_unstale":"2024-09-11T00:57:17.576498+0000","last_undegraded":"2024-09-11T00:57:17.576498+0000","last_fullsized":"2024-09-11T00:57:17.576498+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:40.165368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586583+0000","last_change":"2024-09-11T00:56:53.127047+0000","last_active":"2024-09-11T00:57:17.586583+0000","last_peered":"2024-09-11T00:57:17.586583+0000","last_clean":"2024-09-11T00:57:17.586583+0000","last_became_active":"2024-09-11T00:56:53.126719+0000","last_became_peered":"2024-09-11T00:56:53.126719+0000","last_unstale":"2024-09-11T00:57:17.586583+0000","last_undegraded":"2024-09-11T00:57:17.586583+0000","last_fullsized":"2024-09-11T00:57:17.586583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.686604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583494+0000","last_change":"2024-09-11T00:56:51.101665+0000","last_active":"2024-09-11T00:57:17.583494+0000","last_peered":"2024-09-11T00:57:17.583494+0000","last_clean":"2024-09-11T00:57:17.583494+0000","last_became_active":"2024-09-11T00:56:51.101332+0000","last_became_peered":"2024-09-11T00:56:51.101332+0000","last_unstale":"2024-09-11T00:57:17.583494+0000","last_undegraded":"2024-09-11T00:57:17.583494+0000","last_fullsized":"2024-09-11T00:57:17.583494+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:47.639877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586452+0000","last_change":"2024-09-11T00:56:53.128736+0000","last_active":"2024-09-11T00:57:17.586452+0000","last_peered":"2024-09-11T00:57:17.586452+0000","last_clean":"2024-09-11T00:57:17.586452+0000","last_became_active":"2024-09-11T00:56:53.128417+0000","last_became_peered":"2024-09-11T00:56:53.128417+0000","last_unstale":"2024-09-11T00:57:17.586452+0000","last_undegraded":"2024-09-11T00:57:17.586452+0000","last_fullsized":"2024-09-11T00:57:17.586452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:52.331343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":88,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:59.314430+0000","last_change":"2024-09-11T00:56:51.154733+0000","last_active":"2024-09-11T00:57:59.314430+0000","last_peered":"2024-09-11T00:57:59.314430+0000","last_clean":"2024-09-11T00:57:59.314430+0000","last_became_active":"2024-09-11T00:56:51.153613+0000","last_became_peered":"2024-09-11T00:56:51.153613+0000","last_unstale":"2024-09-11T00:57:59.314430+0000","last_undegraded":"2024-09-11T00:57:59.314430+0000","last_fullsized":"2024-09-11T00:57:59.314430+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:24.749186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":60,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586818+0000","last_change":"2024-09-11T00:56:53.124684+0000","last_active":"2024-09-11T00:57:17.586818+0000","last_peered":"2024-09-11T00:57:17.586818+0000","last_clean":"2024-09-11T00:57:17.586818+0000","last_became_active":"2024-09-11T00:56:53.124147+0000","last_became_peered":"2024-09-11T00:56:53.124147+0000","last_unstale":"2024-09-11T00:57:17.586818+0000","last_undegraded":"2024-09-11T00:57:17.586818+0000","last_fullsized":"2024-09-11T00:57:17.586818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:14.280465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586464+0000","last_change":"2024-09-11T00:55:34.500373+0000","last_active":"2024-09-11T00:57:17.586464+0000","last_peered":"2024-09-11T00:57:17.586464+0000","last_clean":"2024-09-11T00:57:17.586464+0000","last_became_active":"2024-09-11T00:55:34.495540+0000","last_became_peered":"2024-09-11T00:55:34.495540+0000","last_unstale":"2024-09-11T00:57:17.586464+0000","last_undegraded":"2024-09-11T00:57:17.586464+0000","last_fullsized":"2024-09-11T00:57:17.586464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_clean_scrub_stamp":"2024-09-11T00:52:11.168966+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:37.681466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577058+0000","last_change":"2024-09-11T00:56:51.101524+0000","last_active":"2024-09-11T00:57:17.577058+0000","last_peered":"2024-09-11T00:57:17.577058+0000","last_clean":"2024-09-11T00:57:17.577058+0000","last_became_active":"2024-09-11T00:56:51.101180+0000","last_became_peered":"2024-09-11T00:56:51.101180+0000","last_unstale":"2024-09-11T00:57:17.577058+0000","last_undegraded":"2024-09-11T00:57:17.577058+0000","last_fullsized":"2024-09-11T00:57:17.577058+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:27.293494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771467+0000","last_change":"2024-09-11T00:56:53.118503+0000","last_active":"2024-09-11T00:57:17.771467+0000","last_peered":"2024-09-11T00:57:17.771467+0000","last_clean":"2024-09-11T00:57:17.771467+0000","last_became_active":"2024-09-11T00:56:53.118174+0000","last_became_peered":"2024-09-11T00:56:53.118174+0000","last_unstale":"2024-09-11T00:57:17.771467+0000","last_undegraded":"2024-09-11T00:57:17.771467+0000","last_fullsized":"2024-09-11T00:57:17.771467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:13.524153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586788+0000","last_change":"2024-09-11T00:56:51.104222+0000","last_active":"2024-09-11T00:57:17.586788+0000","last_peered":"2024-09-11T00:57:17.586788+0000","last_clean":"2024-09-11T00:57:17.586788+0000","last_became_active":"2024-09-11T00:56:51.103913+0000","last_became_peered":"2024-09-11T00:56:51.103913+0000","last_unstale":"2024-09-11T00:57:17.586788+0000","last_undegraded":"2024-09-11T00:57:17.586788+0000","last_fullsized":"2024-09-11T00:57:17.586788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:57.467659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576162+0000","last_change":"2024-09-11T00:56:53.115130+0000","last_active":"2024-09-11T00:57:17.576162+0000","last_peered":"2024-09-11T00:57:17.576162+0000","last_clean":"2024-09-11T00:57:17.576162+0000","last_became_active":"2024-09-11T00:56:53.114813+0000","last_became_peered":"2024-09-11T00:56:53.114813+0000","last_unstale":"2024-09-11T00:57:17.576162+0000","last_undegraded":"2024-09-11T00:57:17.576162+0000","last_fullsized":"2024-09-11T00:57:17.576162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:13.265173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:56.327610+0000","last_change":"2024-09-11T00:56:51.152892+0000","last_active":"2024-09-11T00:57:56.327610+0000","last_peered":"2024-09-11T00:57:56.327610+0000","last_clean":"2024-09-11T00:57:56.327610+0000","last_became_active":"2024-09-11T00:56:51.152572+0000","last_became_peered":"2024-09-11T00:56:51.152572+0000","last_unstale":"2024-09-11T00:57:56.327610+0000","last_undegraded":"2024-09-11T00:57:56.327610+0000","last_fullsized":"2024-09-11T00:57:56.327610+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:03.774593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583356+0000","last_change":"2024-09-11T00:56:53.117416+0000","last_active":"2024-09-11T00:57:17.583356+0000","last_peered":"2024-09-11T00:57:17.583356+0000","last_clean":"2024-09-11T00:57:17.583356+0000","last_became_active":"2024-09-11T00:56:53.117016+0000","last_became_peered":"2024-09-11T00:56:53.117016+0000","last_unstale":"2024-09-11T00:57:17.583356+0000","last_undegraded":"2024-09-11T00:57:17.583356+0000","last_fullsized":"2024-09-11T00:57:17.583356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:07.607993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":83,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:56.309355+0000","last_change":"2024-09-11T00:56:51.102879+0000","last_active":"2024-09-11T00:57:56.309355+0000","last_peered":"2024-09-11T00:57:56.309355+0000","last_clean":"2024-09-11T00:57:56.309355+0000","last_became_active":"2024-09-11T00:56:51.102498+0000","last_became_peered":"2024-09-11T00:56:51.102498+0000","last_unstale":"2024-09-11T00:57:56.309355+0000","last_undegraded":"2024-09-11T00:57:56.309355+0000","last_fullsized":"2024-09-11T00:57:56.309355+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:41.360908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581368+0000","last_change":"2024-09-11T00:57:17.581261+0000","last_active":"2024-09-11T00:57:17.581368+0000","last_peered":"2024-09-11T00:57:17.581368+0000","last_clean":"2024-09-11T00:57:17.581368+0000","last_became_active":"2024-09-11T00:57:17.580941+0000","last_became_peered":"2024-09-11T00:57:17.580941+0000","last_unstale":"2024-09-11T00:57:17.581368+0000","last_undegraded":"2024-09-11T00:57:17.581368+0000","last_fullsized":"2024-09-11T00:57:17.581368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:58.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.584416+0000","last_change":"2024-09-11T00:57:17.584311+0000","last_active":"2024-09-11T00:57:17.584416+0000","last_peered":"2024-09-11T00:57:17.584416+0000","last_clean":"2024-09-11T00:57:17.584416+0000","last_became_active":"2024-09-11T00:57:17.583964+0000","last_became_peered":"2024-09-11T00:57:17.583964+0000","last_unstale":"2024-09-11T00:57:17.584416+0000","last_undegraded":"2024-09-11T00:57:17.584416+0000","last_fullsized":"2024-09-11T00:57:17.584416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:15.515947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578614+0000","last_change":"2024-09-11T00:56:53.112550+0000","last_active":"2024-09-11T00:57:17.578614+0000","last_peered":"2024-09-11T00:57:17.578614+0000","last_clean":"2024-09-11T00:57:17.578614+0000","last_became_active":"2024-09-11T00:56:53.112179+0000","last_became_peered":"2024-09-11T00:56:53.112179+0000","last_unstale":"2024-09-11T00:57:17.578614+0000","last_undegraded":"2024-09-11T00:57:17.578614+0000","last_fullsized":"2024-09-11T00:57:17.578614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:03.339839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581699+0000","last_change":"2024-09-11T00:56:51.152957+0000","last_active":"2024-09-11T00:57:17.581699+0000","last_peered":"2024-09-11T00:57:17.581699+0000","last_clean":"2024-09-11T00:57:17.581699+0000","last_became_active":"2024-09-11T00:56:51.152616+0000","last_became_peered":"2024-09-11T00:56:51.152616+0000","last_unstale":"2024-09-11T00:57:17.581699+0000","last_undegraded":"2024-09-11T00:57:17.581699+0000","last_fullsized":"2024-09-11T00:57:17.581699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:02.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576410+0000","last_change":"2024-09-11T00:56:53.115155+0000","last_active":"2024-09-11T00:57:17.576410+0000","last_peered":"2024-09-11T00:57:17.576410+0000","last_clean":"2024-09-11T00:57:17.576410+0000","last_became_active":"2024-09-11T00:56:53.114865+0000","last_became_peered":"2024-09-11T00:56:53.114865+0000","last_unstale":"2024-09-11T00:57:17.576410+0000","last_undegraded":"2024-09-11T00:57:17.576410+0000","last_fullsized":"2024-09-11T00:57:17.576410+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:23.968888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'16","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.122556+0000","last_change":"2024-09-11T00:56:51.100887+0000","last_active":"2024-09-11T00:57:18.122556+0000","last_peered":"2024-09-11T00:57:18.122556+0000","last_clean":"2024-09-11T00:57:18.122556+0000","last_became_active":"2024-09-11T00:56:51.100474+0000","last_became_peered":"2024-09-11T00:56:51.100474+0000","last_unstale":"2024-09-11T00:57:18.122556+0000","last_undegraded":"2024-09-11T00:57:18.122556+0000","last_fullsized":"2024-09-11T00:57:18.122556+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:40.884871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.588425+0000","last_change":"2024-09-11T00:57:17.588086+0000","last_active":"2024-09-11T00:57:17.588425+0000","last_peered":"2024-09-11T00:57:17.588425+0000","last_clean":"2024-09-11T00:57:17.588425+0000","last_became_active":"2024-09-11T00:57:17.587785+0000","last_became_peered":"2024-09-11T00:57:17.587785+0000","last_unstale":"2024-09-11T00:57:17.588425+0000","last_undegraded":"2024-09-11T00:57:17.588425+0000","last_fullsized":"2024-09-11T00:57:17.588425+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:49.494588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577254+0000","last_change":"2024-09-11T00:56:51.102031+0000","last_active":"2024-09-11T00:57:17.577254+0000","last_peered":"2024-09-11T00:57:17.577254+0000","last_clean":"2024-09-11T00:57:17.577254+0000","last_became_active":"2024-09-11T00:56:51.100614+0000","last_became_peered":"2024-09-11T00:56:51.100614+0000","last_unstale":"2024-09-11T00:57:17.577254+0000","last_undegraded":"2024-09-11T00:57:17.577254+0000","last_fullsized":"2024-09-11T00:57:17.577254+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:52.611261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578722+0000","last_change":"2024-09-11T00:56:53.124806+0000","last_active":"2024-09-11T00:57:17.578722+0000","last_peered":"2024-09-11T00:57:17.578722+0000","last_clean":"2024-09-11T00:57:17.578722+0000","last_became_active":"2024-09-11T00:56:53.124452+0000","last_became_peered":"2024-09-11T00:56:53.124452+0000","last_unstale":"2024-09-11T00:57:17.578722+0000","last_undegraded":"2024-09-11T00:57:17.578722+0000","last_fullsized":"2024-09-11T00:57:17.578722+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:57.934808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160509+0000","last_change":"2024-09-11T00:56:51.153621+0000","last_active":"2024-09-11T00:57:18.160509+0000","last_peered":"2024-09-11T00:57:18.160509+0000","last_clean":"2024-09-11T00:57:18.160509+0000","last_became_active":"2024-09-11T00:56:51.153315+0000","last_became_peered":"2024-09-11T00:56:51.153315+0000","last_unstale":"2024-09-11T00:57:18.160509+0000","last_undegraded":"2024-09-11T00:57:18.160509+0000","last_fullsized":"2024-09-11T00:57:18.160509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:10.058295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580192+0000","last_change":"2024-09-11T00:56:53.114463+0000","last_active":"2024-09-11T00:57:17.580192+0000","last_peered":"2024-09-11T00:57:17.580192+0000","last_clean":"2024-09-11T00:57:17.580192+0000","last_became_active":"2024-09-11T00:56:53.114099+0000","last_became_peered":"2024-09-11T00:56:53.114099+0000","last_unstale":"2024-09-11T00:57:17.580192+0000","last_undegraded":"2024-09-11T00:57:17.580192+0000","last_fullsized":"2024-09-11T00:57:17.580192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:58.981787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771377+0000","last_change":"2024-09-11T00:56:51.096942+0000","last_active":"2024-09-11T00:57:17.771377+0000","last_peered":"2024-09-11T00:57:17.771377+0000","last_clean":"2024-09-11T00:57:17.771377+0000","last_became_active":"2024-09-11T00:56:51.096584+0000","last_became_peered":"2024-09-11T00:56:51.096584+0000","last_unstale":"2024-09-11T00:57:17.771377+0000","last_undegraded":"2024-09-11T00:57:17.771377+0000","last_fullsized":"2024-09-11T00:57:17.771377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:03.484123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577668+0000","last_change":"2024-09-11T00:56:53.126804+0000","last_active":"2024-09-11T00:57:17.577668+0000","last_peered":"2024-09-11T00:57:17.577668+0000","last_clean":"2024-09-11T00:57:17.577668+0000","last_became_active":"2024-09-11T00:56:53.125408+0000","last_became_peered":"2024-09-11T00:56:53.125408+0000","last_unstale":"2024-09-11T00:57:17.577668+0000","last_undegraded":"2024-09-11T00:57:17.577668+0000","last_fullsized":"2024-09-11T00:57:17.577668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:46.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580130+0000","last_change":"2024-09-11T00:56:51.103755+0000","last_active":"2024-09-11T00:57:17.580130+0000","last_peered":"2024-09-11T00:57:17.580130+0000","last_clean":"2024-09-11T00:57:17.580130+0000","last_became_active":"2024-09-11T00:56:51.103395+0000","last_became_peered":"2024-09-11T00:56:51.103395+0000","last_unstale":"2024-09-11T00:57:17.580130+0000","last_undegraded":"2024-09-11T00:57:17.580130+0000","last_fullsized":"2024-09-11T00:57:17.580130+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:16.782660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771488+0000","last_change":"2024-09-11T00:56:53.121926+0000","last_active":"2024-09-11T00:57:17.771488+0000","last_peered":"2024-09-11T00:57:17.771488+0000","last_clean":"2024-09-11T00:57:17.771488+0000","last_became_active":"2024-09-11T00:56:53.121603+0000","last_became_peered":"2024-09-11T00:56:53.121603+0000","last_unstale":"2024-09-11T00:57:17.771488+0000","last_undegraded":"2024-09-11T00:57:17.771488+0000","last_fullsized":"2024-09-11T00:57:17.771488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:31.021030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580437+0000","last_change":"2024-09-11T00:56:51.107221+0000","last_active":"2024-09-11T00:57:17.580437+0000","last_peered":"2024-09-11T00:57:17.580437+0000","last_clean":"2024-09-11T00:57:17.580437+0000","last_became_active":"2024-09-11T00:56:51.106849+0000","last_became_peered":"2024-09-11T00:56:51.106849+0000","last_unstale":"2024-09-11T00:57:17.580437+0000","last_undegraded":"2024-09-11T00:57:17.580437+0000","last_fullsized":"2024-09-11T00:57:17.580437+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:46.497230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586475+0000","last_change":"2024-09-11T00:56:53.120525+0000","last_active":"2024-09-11T00:57:17.586475+0000","last_peered":"2024-09-11T00:57:17.586475+0000","last_clean":"2024-09-11T00:57:17.586475+0000","last_became_active":"2024-09-11T00:56:53.120228+0000","last_became_peered":"2024-09-11T00:56:53.120228+0000","last_unstale":"2024-09-11T00:57:17.586475+0000","last_undegraded":"2024-09-11T00:57:17.586475+0000","last_fullsized":"2024-09-11T00:57:17.586475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:52.047687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583268+0000","last_change":"2024-09-11T00:56:53.120311+0000","last_active":"2024-09-11T00:57:17.583268+0000","last_peered":"2024-09-11T00:57:17.583268+0000","last_clean":"2024-09-11T00:57:17.583268+0000","last_became_active":"2024-09-11T00:56:53.119648+0000","last_became_peered":"2024-09-11T00:56:53.119648+0000","last_unstale":"2024-09-11T00:57:17.583268+0000","last_undegraded":"2024-09-11T00:57:17.583268+0000","last_fullsized":"2024-09-11T00:57:17.583268+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:21.187784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581118+0000","last_change":"2024-09-11T00:56:51.103631+0000","last_active":"2024-09-11T00:57:17.581118+0000","last_peered":"2024-09-11T00:57:17.581118+0000","last_clean":"2024-09-11T00:57:17.581118+0000","last_became_active":"2024-09-11T00:56:51.103281+0000","last_became_peered":"2024-09-11T00:56:51.103281+0000","last_unstale":"2024-09-11T00:57:17.581118+0000","last_undegraded":"2024-09-11T00:57:17.581118+0000","last_fullsized":"2024-09-11T00:57:17.581118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:16.910956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160481+0000","last_change":"2024-09-11T00:56:53.124752+0000","last_active":"2024-09-11T00:57:18.160481+0000","last_peered":"2024-09-11T00:57:18.160481+0000","last_clean":"2024-09-11T00:57:18.160481+0000","last_became_active":"2024-09-11T00:56:53.124245+0000","last_became_peered":"2024-09-11T00:56:53.124245+0000","last_unstale":"2024-09-11T00:57:18.160481+0000","last_undegraded":"2024-09-11T00:57:18.160481+0000","last_fullsized":"2024-09-11T00:57:18.160481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:10.277903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581704+0000","last_change":"2024-09-11T00:56:51.104624+0000","last_active":"2024-09-11T00:57:17.581704+0000","last_peered":"2024-09-11T00:57:17.581704+0000","last_clean":"2024-09-11T00:57:17.581704+0000","last_became_active":"2024-09-11T00:56:51.104041+0000","last_became_peered":"2024-09-11T00:56:51.104041+0000","last_unstale":"2024-09-11T00:57:17.581704+0000","last_undegraded":"2024-09-11T00:57:17.581704+0000","last_fullsized":"2024-09-11T00:57:17.581704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:10.397455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.579287+0000","last_change":"2024-09-11T00:56:53.118547+0000","last_active":"2024-09-11T00:57:17.579287+0000","last_peered":"2024-09-11T00:57:17.579287+0000","last_clean":"2024-09-11T00:57:17.579287+0000","last_became_active":"2024-09-11T00:56:53.118291+0000","last_became_peered":"2024-09-11T00:56:53.118291+0000","last_unstale":"2024-09-11T00:57:17.579287+0000","last_undegraded":"2024-09-11T00:57:17.579287+0000","last_fullsized":"2024-09-11T00:57:17.579287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:42.012173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771739+0000","last_change":"2024-09-11T00:56:51.153047+0000","last_active":"2024-09-11T00:57:17.771739+0000","last_peered":"2024-09-11T00:57:17.771739+0000","last_clean":"2024-09-11T00:57:17.771739+0000","last_became_active":"2024-09-11T00:56:51.152564+0000","last_became_peered":"2024-09-11T00:56:51.152564+0000","last_unstale":"2024-09-11T00:57:17.771739+0000","last_undegraded":"2024-09-11T00:57:17.771739+0000","last_fullsized":"2024-09-11T00:57:17.771739+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:16.849087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771630+0000","last_change":"2024-09-11T00:56:53.119926+0000","last_active":"2024-09-11T00:57:17.771630+0000","last_peered":"2024-09-11T00:57:17.771630+0000","last_clean":"2024-09-11T00:57:17.771630+0000","last_became_active":"2024-09-11T00:56:53.119643+0000","last_became_peered":"2024-09-11T00:56:53.119643+0000","last_unstale":"2024-09-11T00:57:17.771630+0000","last_undegraded":"2024-09-11T00:57:17.771630+0000","last_fullsized":"2024-09-11T00:57:17.771630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:51.838958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":92,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:57.310232+0000","last_change":"2024-09-11T00:56:51.100837+0000","last_active":"2024-09-11T00:57:57.310232+0000","last_peered":"2024-09-11T00:57:57.310232+0000","last_clean":"2024-09-11T00:57:57.310232+0000","last_became_active":"2024-09-11T00:56:51.100494+0000","last_became_peered":"2024-09-11T00:56:51.100494+0000","last_unstale":"2024-09-11T00:57:57.310232+0000","last_undegraded":"2024-09-11T00:57:57.310232+0000","last_fullsized":"2024-09-11T00:57:57.310232+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:39.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67160,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":598,"num_read_kb":371,"num_write":243,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286865,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743451,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":851968,"data_stored":416613,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906976,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1067877,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070502,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":868352,"data_stored":416431,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266730,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":856064,"data_stored":416697,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":917504,"data_stored":1009327,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626484,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":416148,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":1009005,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986240,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":860160,"data_stored":417022,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":868352,"data_stored":415973,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":477006,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":913408,"data_stored":477408,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":417487,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:58:04.015 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T00:58:04.015 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T00:58:04.015 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T00:58:04.015 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T00:58:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:04.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.759+0000 7f67b2189640 1 -- 172.21.15.78:0/4130058317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67ac101c90 msgr2=0x7f67ac102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:04.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.759+0000 7f67b2189640 1 --2- 172.21.15.78:0/4130058317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67ac101c90 0x7f67ac102070 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f679c009900 tx=0x7f679c02a410 comp rx=0 tx=0).stop 2024-09-11T00:58:04.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.759+0000 7f67b2189640 1 -- 172.21.15.78:0/4130058317 shutdown_connections 2024-09-11T00:58:04.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.759+0000 7f67b2189640 1 --2- 172.21.15.78:0/4130058317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ac108020 0x7f67ac10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:04.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.759+0000 7f67b2189640 1 --2- 172.21.15.78:0/4130058317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac1076a0 0x7f67ac107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:04.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.759+0000 7f67b2189640 1 --2- 172.21.15.78:0/4130058317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67ac101c90 0x7f67ac102070 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:04.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.759+0000 7f67b2189640 1 -- 172.21.15.78:0/4130058317 >> 172.21.15.78:0/4130058317 conn(0x7f67ac0fd700 msgr2=0x7f67ac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:04.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.759+0000 7f67b2189640 1 -- 172.21.15.78:0/4130058317 shutdown_connections 2024-09-11T00:58:04.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.759+0000 7f67b2189640 1 -- 172.21.15.78:0/4130058317 wait complete. 2024-09-11T00:58:04.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.760+0000 7f67b2189640 1 Processor -- start 2024-09-11T00:58:04.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.760+0000 7f67b2189640 1 -- start start 2024-09-11T00:58:04.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.760+0000 7f67b2189640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ac101c90 0x7f67ac0fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:04.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.760+0000 7f67b2189640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac1076a0 0x7f67ac0fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:04.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.760+0000 7f67b2189640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67ac108020 0x7f67ac0fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:04.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.760+0000 7f67b2189640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac111ca0 con 0x7f67ac108020 2024-09-11T00:58:04.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.760+0000 7f67b2189640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac111b20 con 0x7f67ac1076a0 2024-09-11T00:58:04.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.760+0000 7f67b2189640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac111e20 con 0x7f67ac101c90 2024-09-11T00:58:04.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.760+0000 7f67abfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67ac108020 0x7f67ac0fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.760+0000 7f67abfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67ac108020 0x7f67ac0fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51724/0 (socket says 172.21.15.78:51724) 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.760+0000 7f67abfff640 1 -- 172.21.15.78:0/73542114 learned_addr learned my addr 172.21.15.78:0/73542114 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67ab7fe640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ac101c90 0x7f67ac0fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67abfff640 1 -- 172.21.15.78:0/73542114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ac101c90 msgr2=0x7f67ac0fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67a2ffd640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac1076a0 0x7f67ac0fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67abfff640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ac101c90 0x7f67ac0fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67abfff640 1 -- 172.21.15.78:0/73542114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac1076a0 msgr2=0x7f67ac0fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67abfff640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac1076a0 0x7f67ac0fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67abfff640 1 -- 172.21.15.78:0/73542114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ac1a7690 con 0x7f67ac108020 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67ab7fe640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ac101c90 0x7f67ac0fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67a2ffd640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac1076a0 0x7f67ac0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67abfff640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67ac108020 0x7f67ac0fe680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f67a400cad0 tx=0x7f67a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67a97fa640 1 -- 172.21.15.78:0/73542114 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a400ae20 con 0x7f67ac108020 2024-09-11T00:58:04.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67a97fa640 1 -- 172.21.15.78:0/73542114 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a4005020 con 0x7f67ac108020 2024-09-11T00:58:04.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67b2189640 1 -- 172.21.15.78:0/73542114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ac1a7980 con 0x7f67ac108020 2024-09-11T00:58:04.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67a97fa640 1 -- 172.21.15.78:0/73542114 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a400f6b0 con 0x7f67ac108020 2024-09-11T00:58:04.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.761+0000 7f67b2189640 1 -- 172.21.15.78:0/73542114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ac1a7e70 con 0x7f67ac108020 2024-09-11T00:58:04.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.762+0000 7f67b2189640 1 -- 172.21.15.78:0/73542114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67ac102070 con 0x7f67ac108020 2024-09-11T00:58:04.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.763+0000 7f67a97fa640 1 -- 172.21.15.78:0/73542114 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f67a40054f0 con 0x7f67ac108020 2024-09-11T00:58:04.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.764+0000 7f67a97fa640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f67980b53c0 0x7f67980b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:04.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.764+0000 7f67ab7fe640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f67980b53c0 0x7f67980b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:04.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.764+0000 7f67a97fa640 1 -- 172.21.15.78:0/73542114 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f67a40e6340 con 0x7f67ac108020 2024-09-11T00:58:04.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.765+0000 7f67ab7fe640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f67980b53c0 0x7f67980b7880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f679c031440 tx=0x7f679c0095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:04.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.767+0000 7f67a97fa640 1 -- 172.21.15.78:0/73542114 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f67a4014030 con 0x7f67ac108020 2024-09-11T00:58:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.980+0000 7f67b2189640 1 -- 172.21.15.78:0/73542114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f67ac02e5c0 con 0x7f67ac108020 2024-09-11T00:58:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.980+0000 7f67a97fa640 1 -- 172.21.15.78:0/73542114 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f67a40acff0 con 0x7f67ac108020 2024-09-11T00:58:04.982 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:04.982 INFO:teuthology.orchestra.run.smithi078.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T00:58:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.982+0000 7f67b2189640 1 -- 172.21.15.78:0/73542114 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f67980b53c0 msgr2=0x7f67980b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.982+0000 7f67b2189640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f67980b53c0 0x7f67980b7880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f679c031440 tx=0x7f679c0095f0 comp rx=0 tx=0).stop 2024-09-11T00:58:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.983+0000 7f67b2189640 1 -- 172.21.15.78:0/73542114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67ac108020 msgr2=0x7f67ac0fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.983+0000 7f67b2189640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67ac108020 0x7f67ac0fe680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f67a400cad0 tx=0x7f67a400cf90 comp rx=0 tx=0).stop 2024-09-11T00:58:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.983+0000 7f67b2189640 1 -- 172.21.15.78:0/73542114 shutdown_connections 2024-09-11T00:58:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.983+0000 7f67b2189640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f67980b53c0 0x7f67980b7880 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.983+0000 7f67b2189640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67ac108020 0x7f67ac0fe680 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.983+0000 7f67b2189640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac1076a0 0x7f67ac0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.983+0000 7f67b2189640 1 --2- 172.21.15.78:0/73542114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67ac101c90 0x7f67ac0fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.983+0000 7f67b2189640 1 -- 172.21.15.78:0/73542114 >> 172.21.15.78:0/73542114 conn(0x7f67ac0fd700 msgr2=0x7f67ac06ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.983+0000 7f67b2189640 1 -- 172.21.15.78:0/73542114 shutdown_connections 2024-09-11T00:58:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:04.983+0000 7f67b2189640 1 -- 172.21.15.78:0/73542114 wait complete. 2024-09-11T00:58:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:05 smithi078 ceph-mon[31315]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:05 smithi078 ceph-mon[31315]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:58:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:05 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/73542114' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-11T00:58:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:05 smithi110 ceph-mon[34712]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:05 smithi110 ceph-mon[34712]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:58:05.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:05 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/73542114' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-11T00:58:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:05 smithi112 ceph-mon[35384]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:05 smithi112 ceph-mon[35384]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:58:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:05 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/73542114' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-11T00:58:05.674 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T00:58:05.674 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:58:05.918 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.437+0000 7f2d690af640 1 -- 172.21.15.78:0/1769146604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d64108030 msgr2=0x7f2d6410e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.437+0000 7f2d690af640 1 --2- 172.21.15.78:0/1769146604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d64108030 0x7f2d6410e830 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009900 tx=0x7f2d5c02a410 comp rx=0 tx=0).stop 2024-09-11T00:58:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.437+0000 7f2d690af640 1 -- 172.21.15.78:0/1769146604 shutdown_connections 2024-09-11T00:58:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.437+0000 7f2d690af640 1 --2- 172.21.15.78:0/1769146604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d64108030 0x7f2d6410e830 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.437+0000 7f2d690af640 1 --2- 172.21.15.78:0/1769146604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d641076b0 0x7f2d64107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.437+0000 7f2d690af640 1 --2- 172.21.15.78:0/1769146604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d64101ca0 0x7f2d64102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.437+0000 7f2d690af640 1 -- 172.21.15.78:0/1769146604 >> 172.21.15.78:0/1769146604 conn(0x7f2d640fd6d0 msgr2=0x7f2d640ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.437+0000 7f2d690af640 1 -- 172.21.15.78:0/1769146604 shutdown_connections 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.437+0000 7f2d690af640 1 -- 172.21.15.78:0/1769146604 wait complete. 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.438+0000 7f2d690af640 1 Processor -- start 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.438+0000 7f2d690af640 1 -- start start 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.438+0000 7f2d690af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d64101ca0 0x7f2d641993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.438+0000 7f2d690af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d641076b0 0x7f2d641998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.438+0000 7f2d690af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d64108030 0x7f2d6419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.438+0000 7f2d690af640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64111d30 con 0x7f2d64101ca0 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.438+0000 7f2d690af640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64111bb0 con 0x7f2d64108030 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.438+0000 7f2d690af640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64111eb0 con 0x7f2d641076b0 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.438+0000 7f2d62d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d64101ca0 0x7f2d641993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d62d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d64101ca0 0x7f2d641993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51738/0 (socket says 172.21.15.78:51738) 2024-09-11T00:58:06.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d62d9d640 1 -- 172.21.15.78:0/1556209353 learned_addr learned my addr 172.21.15.78:0/1556209353 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d6359e640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d64108030 0x7f2d6419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d6259c640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d641076b0 0x7f2d641998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d62d9d640 1 -- 172.21.15.78:0/1556209353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d641076b0 msgr2=0x7f2d641998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d62d9d640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d641076b0 0x7f2d641998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d62d9d640 1 -- 172.21.15.78:0/1556209353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d64108030 msgr2=0x7f2d6419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d62d9d640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d64108030 0x7f2d6419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d62d9d640 1 -- 172.21.15.78:0/1556209353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6419f450 con 0x7f2d64101ca0 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d6359e640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d64108030 0x7f2d6419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d62d9d640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d64101ca0 0x7f2d641993a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800b4a0 tx=0x7f2d5800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d4bfff640 1 -- 172.21.15.78:0/1556209353 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58014070 con 0x7f2d64101ca0 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d690af640 1 -- 172.21.15.78:0/1556209353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6419f740 con 0x7f2d64101ca0 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d4bfff640 1 -- 172.21.15.78:0/1556209353 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d58010ac0 con 0x7f2d64101ca0 2024-09-11T00:58:06.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d4bfff640 1 -- 172.21.15.78:0/1556209353 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58017c30 con 0x7f2d64101ca0 2024-09-11T00:58:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.439+0000 7f2d690af640 1 -- 172.21.15.78:0/1556209353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d641a5e80 con 0x7f2d64101ca0 2024-09-11T00:58:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.440+0000 7f2d690af640 1 -- 172.21.15.78:0/1556209353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d64102100 con 0x7f2d64101ca0 2024-09-11T00:58:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.442+0000 7f2d4bfff640 1 -- 172.21.15.78:0/1556209353 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f2d58010610 con 0x7f2d64101ca0 2024-09-11T00:58:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.442+0000 7f2d4bfff640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2d4c0b53c0 0x7f2d4c0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.442+0000 7f2d6259c640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2d4c0b53c0 0x7f2d4c0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.442+0000 7f2d4bfff640 1 -- 172.21.15.78:0/1556209353 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f2d580e57f0 con 0x7f2d64101ca0 2024-09-11T00:58:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.443+0000 7f2d6259c640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2d4c0b53c0 0x7f2d4c0b7880 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2d64104320 tx=0x7f2d540072f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.445+0000 7f2d4bfff640 1 -- 172.21.15.78:0/1556209353 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2d580ac440 con 0x7f2d64101ca0 2024-09-11T00:58:06.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.657+0000 7f2d690af640 1 -- 172.21.15.78:0/1556209353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d6419a4d0 con 0x7f2d64101ca0 2024-09-11T00:58:06.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.658+0000 7f2d4bfff640 1 -- 172.21.15.78:0/1556209353 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11979 (secure 0 0 0) 0x7f2d580abd40 con 0x7f2d64101ca0 2024-09-11T00:58:06.660 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:06.660 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-09-11T00:57:05:238610+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T00:57:04.821148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T00:58:06.660 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-11T00:58:06.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.660+0000 7f2d690af640 1 -- 172.21.15.78:0/1556209353 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2d4c0b53c0 msgr2=0x7f2d4c0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:06.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.660+0000 7f2d690af640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2d4c0b53c0 0x7f2d4c0b7880 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2d64104320 tx=0x7f2d540072f0 comp rx=0 tx=0).stop 2024-09-11T00:58:06.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.660+0000 7f2d690af640 1 -- 172.21.15.78:0/1556209353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d64101ca0 msgr2=0x7f2d641993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:06.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.660+0000 7f2d690af640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d64101ca0 0x7f2d641993a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800b4a0 tx=0x7f2d5800b960 comp rx=0 tx=0).stop 2024-09-11T00:58:06.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.661+0000 7f2d690af640 1 -- 172.21.15.78:0/1556209353 shutdown_connections 2024-09-11T00:58:06.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.661+0000 7f2d690af640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2d4c0b53c0 0x7f2d4c0b7880 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:06.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.661+0000 7f2d690af640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d64108030 0x7f2d6419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:06.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.661+0000 7f2d690af640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d641076b0 0x7f2d641998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:06.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.661+0000 7f2d690af640 1 --2- 172.21.15.78:0/1556209353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d64101ca0 0x7f2d641993a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:06.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.661+0000 7f2d690af640 1 -- 172.21.15.78:0/1556209353 >> 172.21.15.78:0/1556209353 conn(0x7f2d640fd6d0 msgr2=0x7f2d6406b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:06.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.661+0000 7f2d690af640 1 -- 172.21.15.78:0/1556209353 shutdown_connections 2024-09-11T00:58:06.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:06.661+0000 7f2d690af640 1 -- 172.21.15.78:0/1556209353 wait complete. 2024-09-11T00:58:07.340 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:07 smithi078 ceph-mon[31315]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:58:07.340 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:07 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1556209353' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:07.370 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:58:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:07 smithi110 ceph-mon[34712]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:58:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:07 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1556209353' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:07.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:07 smithi112 ceph-mon[35384]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:58:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:07 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1556209353' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:07.614 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:08.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.104+0000 7f49dbeba640 1 -- 172.21.15.78:0/3713758045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49d41045e0 msgr2=0x7f49d410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:08.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.104+0000 7f49dbeba640 1 --2- 172.21.15.78:0/3713758045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49d41045e0 0x7f49d410e020 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009ca0 tx=0x7f49c402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:08.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.104+0000 7f49dbeba640 1 -- 172.21.15.78:0/3713758045 shutdown_connections 2024-09-11T00:58:08.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.104+0000 7f49dbeba640 1 --2- 172.21.15.78:0/3713758045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49d41045e0 0x7f49d410e020 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:08.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.104+0000 7f49dbeba640 1 --2- 172.21.15.78:0/3713758045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49d4103c40 0x7f49d41040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:08.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.104+0000 7f49dbeba640 1 --2- 172.21.15.78:0/3713758045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d4101810 0x7f49d4101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:08.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.104+0000 7f49dbeba640 1 -- 172.21.15.78:0/3713758045 >> 172.21.15.78:0/3713758045 conn(0x7f49d4076f20 msgr2=0x7f49d4077330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:08.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.104+0000 7f49dbeba640 1 -- 172.21.15.78:0/3713758045 shutdown_connections 2024-09-11T00:58:08.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.104+0000 7f49dbeba640 1 -- 172.21.15.78:0/3713758045 wait complete. 2024-09-11T00:58:08.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49dbeba640 1 Processor -- start 2024-09-11T00:58:08.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49dbeba640 1 -- start start 2024-09-11T00:58:08.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49dbeba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49d4101810 0x7f49d4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:08.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49dbeba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49d4103c40 0x7f49d4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:08.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49dbeba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d41045e0 0x7f49d419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:08.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49dbeba640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4110c20 con 0x7f49d4103c40 2024-09-11T00:58:08.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49dbeba640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4110aa0 con 0x7f49d4101810 2024-09-11T00:58:08.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49dbeba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4110da0 con 0x7f49d41045e0 2024-09-11T00:58:08.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49d9455640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49d4103c40 0x7f49d4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:08.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49d9455640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49d4103c40 0x7f49d4199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51766/0 (socket says 172.21.15.78:51766) 2024-09-11T00:58:08.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49d9455640 1 -- 172.21.15.78:0/3561948620 learned_addr learned my addr 172.21.15.78:0/3561948620 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:08.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49da457640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d41045e0 0x7f49d419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:08.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49d9455640 1 -- 172.21.15.78:0/3561948620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d41045e0 msgr2=0x7f49d419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:08.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49d9c56640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49d4101810 0x7f49d4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:08.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49d9455640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d41045e0 0x7f49d419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:08.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49d9455640 1 -- 172.21.15.78:0/3561948620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49d4101810 msgr2=0x7f49d4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:08.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49d9455640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49d4101810 0x7f49d4199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:08.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49d9455640 1 -- 172.21.15.78:0/3561948620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d419f2f0 con 0x7f49d4103c40 2024-09-11T00:58:08.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.105+0000 7f49d9455640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49d4103c40 0x7f49d4199780 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f49c800c920 tx=0x7f49c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:08.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.106+0000 7f49d2ffd640 1 -- 172.21.15.78:0/3561948620 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8007c70 con 0x7f49d4103c40 2024-09-11T00:58:08.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.106+0000 7f49d2ffd640 1 -- 172.21.15.78:0/3561948620 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49c8007e10 con 0x7f49d4103c40 2024-09-11T00:58:08.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.106+0000 7f49dbeba640 1 -- 172.21.15.78:0/3561948620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d419f5e0 con 0x7f49d4103c40 2024-09-11T00:58:08.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.106+0000 7f49d2ffd640 1 -- 172.21.15.78:0/3561948620 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8018820 con 0x7f49d4103c40 2024-09-11T00:58:08.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.106+0000 7f49dbeba640 1 -- 172.21.15.78:0/3561948620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d41a5ca0 con 0x7f49d4103c40 2024-09-11T00:58:08.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.107+0000 7f49dbeba640 1 -- 172.21.15.78:0/3561948620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d4101bf0 con 0x7f49d4103c40 2024-09-11T00:58:08.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.108+0000 7f49d2ffd640 1 -- 172.21.15.78:0/3561948620 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f49c800f450 con 0x7f49d4103c40 2024-09-11T00:58:08.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.109+0000 7f49d2ffd640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f49bc0b53c0 0x7f49bc0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:08.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.109+0000 7f49d9c56640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f49bc0b53c0 0x7f49bc0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:08.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.109+0000 7f49d2ffd640 1 -- 172.21.15.78:0/3561948620 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f49c80e62f0 con 0x7f49d4103c40 2024-09-11T00:58:08.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.109+0000 7f49d9c56640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f49bc0b53c0 0x7f49bc0b7880 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f49cc005fd0 tx=0x7f49cc005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:08.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.111+0000 7f49d2ffd640 1 -- 172.21.15.78:0/3561948620 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f49c8014030 con 0x7f49d4103c40 2024-09-11T00:58:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.326+0000 7f49dbeba640 1 -- 172.21.15.78:0/3561948620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49d419a370 con 0x7f49d4103c40 2024-09-11T00:58:08.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.327+0000 7f49d2ffd640 1 -- 172.21.15.78:0/3561948620 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11979 (secure 0 0 0) 0x7f49c80ad030 con 0x7f49d4103c40 2024-09-11T00:58:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:08.329 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-09-11T00:57:05:238610+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T00:57:04.821148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T00:58:08.329 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-11T00:58:08.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.330+0000 7f49dbeba640 1 -- 172.21.15.78:0/3561948620 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f49bc0b53c0 msgr2=0x7f49bc0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.330+0000 7f49dbeba640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f49bc0b53c0 0x7f49bc0b7880 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f49cc005fd0 tx=0x7f49cc005850 comp rx=0 tx=0).stop 2024-09-11T00:58:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.330+0000 7f49dbeba640 1 -- 172.21.15.78:0/3561948620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49d4103c40 msgr2=0x7f49d4199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.330+0000 7f49dbeba640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49d4103c40 0x7f49d4199780 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f49c800c920 tx=0x7f49c800cde0 comp rx=0 tx=0).stop 2024-09-11T00:58:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.330+0000 7f49dbeba640 1 -- 172.21.15.78:0/3561948620 shutdown_connections 2024-09-11T00:58:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.330+0000 7f49dbeba640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f49bc0b53c0 0x7f49bc0b7880 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.330+0000 7f49dbeba640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d41045e0 0x7f49d419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.330+0000 7f49dbeba640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49d4103c40 0x7f49d4199780 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.330+0000 7f49dbeba640 1 --2- 172.21.15.78:0/3561948620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49d4101810 0x7f49d4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.330+0000 7f49dbeba640 1 -- 172.21.15.78:0/3561948620 >> 172.21.15.78:0/3561948620 conn(0x7f49d4076f20 msgr2=0x7f49d4100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.330+0000 7f49dbeba640 1 -- 172.21.15.78:0/3561948620 shutdown_connections 2024-09-11T00:58:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:08.330+0000 7f49dbeba640 1 -- 172.21.15.78:0/3561948620 wait complete. 2024-09-11T00:58:08.985 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:58:09.234 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:09.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:09 smithi078 ceph-mon[31315]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:09.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:09 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3561948620' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:09 smithi110 ceph-mon[34712]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:09 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3561948620' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:09 smithi112 ceph-mon[35384]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:09 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3561948620' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.731+0000 7f0b3eef5640 1 -- 172.21.15.78:0/4072217427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b381076a0 msgr2=0x7f0b38107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.731+0000 7f0b3eef5640 1 --2- 172.21.15.78:0/4072217427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b381076a0 0x7f0b38107ae0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009900 tx=0x7f0b2402a410 comp rx=0 tx=0).stop 2024-09-11T00:58:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.731+0000 7f0b3eef5640 1 -- 172.21.15.78:0/4072217427 shutdown_connections 2024-09-11T00:58:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.731+0000 7f0b3eef5640 1 --2- 172.21.15.78:0/4072217427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38108020 0x7f0b3810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.731+0000 7f0b3eef5640 1 --2- 172.21.15.78:0/4072217427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b381076a0 0x7f0b38107ae0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.731+0000 7f0b3eef5640 1 --2- 172.21.15.78:0/4072217427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b38101c90 0x7f0b38102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.731+0000 7f0b3eef5640 1 -- 172.21.15.78:0/4072217427 >> 172.21.15.78:0/4072217427 conn(0x7f0b380fd6c0 msgr2=0x7f0b380ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.731+0000 7f0b3eef5640 1 -- 172.21.15.78:0/4072217427 shutdown_connections 2024-09-11T00:58:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.731+0000 7f0b3eef5640 1 -- 172.21.15.78:0/4072217427 wait complete. 2024-09-11T00:58:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b3eef5640 1 Processor -- start 2024-09-11T00:58:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b3eef5640 1 -- start start 2024-09-11T00:58:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b3eef5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b38101c90 0x7f0b381981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b3eef5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b381076a0 0x7f0b38198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b3eef5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38108020 0x7f0b3819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b3eef5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0b38111ce0 con 0x7f0b381076a0 2024-09-11T00:58:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b3eef5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b38111b60 con 0x7f0b38108020 2024-09-11T00:58:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b3eef5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b38111e60 con 0x7f0b38101c90 2024-09-11T00:58:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b37fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b381076a0 0x7f0b38198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b37fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b381076a0 0x7f0b38198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51782/0 (socket says 172.21.15.78:51782) 2024-09-11T00:58:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b37fff640 1 -- 172.21.15.78:0/3002374757 learned_addr learned my addr 172.21.15.78:0/3002374757 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b37fff640 1 -- 172.21.15.78:0/3002374757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b38101c90 msgr2=0x7f0b381981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b37fff640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b38101c90 0x7f0b381981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b37fff640 1 -- 172.21.15.78:0/3002374757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38108020 msgr2=0x7f0b3819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b3d492640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38108020 0x7f0b3819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b37fff640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38108020 0x7f0b3819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b37fff640 1 -- 172.21.15.78:0/3002374757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3819e280 con 0x7f0b381076a0 2024-09-11T00:58:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.732+0000 7f0b3d492640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38108020 0x7f0b3819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.733+0000 7f0b37fff640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b381076a0 0x7f0b38198710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0b24006010 tx=0x7f0b240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.733+0000 7f0b35ffb640 1 -- 172.21.15.78:0/3002374757 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2403f030 con 0x7f0b381076a0 2024-09-11T00:58:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.733+0000 7f0b3eef5640 1 -- 172.21.15.78:0/3002374757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3819e510 con 0x7f0b381076a0 2024-09-11T00:58:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.733+0000 7f0b35ffb640 1 -- 172.21.15.78:0/3002374757 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b24003800 con 0x7f0b381076a0 2024-09-11T00:58:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.733+0000 7f0b35ffb640 1 -- 172.21.15.78:0/3002374757 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24035070 con 0x7f0b381076a0 2024-09-11T00:58:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.733+0000 7f0b3eef5640 1 -- 172.21.15.78:0/3002374757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b381a4c30 con 0x7f0b381076a0 2024-09-11T00:58:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.734+0000 7f0b3eef5640 1 -- 172.21.15.78:0/3002374757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b38102070 con 0x7f0b381076a0 2024-09-11T00:58:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.735+0000 7f0b35ffb640 1 -- 172.21.15.78:0/3002374757 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f0b24031b10 con 0x7f0b381076a0 2024-09-11T00:58:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.736+0000 7f0b35ffb640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0b280b53c0 0x7f0b280b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.736+0000 7f0b3cc91640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0b280b53c0 0x7f0b280b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.736+0000 7f0b35ffb640 1 -- 172.21.15.78:0/3002374757 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0b24106af0 con 0x7f0b381076a0 2024-09-11T00:58:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.736+0000 7f0b3cc91640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0b280b53c0 0x7f0b280b7880 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009710 tx=0x7f0b2c006c90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.739+0000 7f0b35ffb640 1 -- 172.21.15.78:0/3002374757 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0b240cd7a0 con 0x7f0b381076a0 2024-09-11T00:58:09.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.915+0000 7f0b3eef5640 1 -- 172.21.15.78:0/3002374757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b38199300 con 0x7f0b381076a0 2024-09-11T00:58:09.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.916+0000 7f0b35ffb640 1 -- 172.21.15.78:0/3002374757 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22975 (secure 0 0 0) 0x7f0b240cd0a0 con 0x7f0b381076a0 2024-09-11T00:58:09.918 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:09.918 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","created":"2024-09-11T00:48:27.652512+0000","modified":"2024-09-11T00:57:17.572848+0000","last_up_change":"2024-09-11T00:55:59.071507+0000","last_in_change":"2024-09-11T00:55:35.815150+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:52:10.229887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:56:49.939136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:56:51.703446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beefc6d9-f74c-4200-96f2-ff697ad0d7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6803","nonce":1424639053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6805","nonce":1424639053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6809","nonce":1424639053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6807","nonce":1424639053}]},"public_addr":"172.21.15.78:6803/1424639053","cluster_addr":"172.21.15.78:6805/1424639053","heartbeat_back_addr":"172.21.15.78:6809/1424639053","heartbeat_front_addr":"172.21.15.78:6807/1424639053","state":["exists","up"]},{"osd":1,"uuid":"d742e767-d673-4bb9-b624-8bc969f5ceb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6801","nonce":580939385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6803","nonce":580939385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6807","nonce":580939385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6805","nonce":580939385}]},"public_addr":"172.21.15.110:6801/580939385","cluster_addr":"172.21.15.110:6803/580939385","heartbeat_back_addr":"172.21.15.110:6807/580939385","heartbeat_front_addr":"172.21.15.110:6805/580939385","state":["exists","up"]},{"osd":2,"uuid":"b0106a5b-1465-4a50-96dd-934dbae49a4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6801","nonce":2983155763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6803","nonce":2983155763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6807","nonce":2983155763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6805","nonce":2983155763}]},"public_addr":"172.21.15.112:6801/2983155763","cluster_addr":"172.21.15.112:6803/2983155763","heartbeat_back_addr":"172.21.15.112:6807/2983155763","heartbeat_front_addr":"172.21.15.112:6805/2983155763","state":["exists","up"]},{"osd":3,"uuid":"3270362b-3d61-414a-a3d9-f2416ff8b870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6811","nonce":2906086464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6813","nonce":2906086464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6817","nonce":2906086464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6815","nonce":2906086464}]},"public_addr":"172.21.15.78:6811/2906086464","cluster_addr":"172.21.15.78:6813/2906086464","heartbeat_back_addr":"172.21.15.78:6817/2906086464","heartbeat_front_addr":"172.21.15.78:6815/2906086464","state":["exists","up"]},{"osd":4,"uuid":"e47746ec-9443-4004-abfa-17c1ed73b982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6809","nonce":477287435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6811","nonce":477287435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6815","nonce":477287435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6813","nonce":477287435}]},"public_addr":"172.21.15.110:6809/477287435","cluster_addr":"172.21.15.110:6811/477287435","heartbeat_back_addr":"172.21.15.110:6815/477287435","heartbeat_front_addr":"172.21.15.110:6813/477287435","state":["exists","up"]},{"osd":5,"uuid":"f8cedb01-310d-442d-a588-33197d32e208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6809","nonce":3557095096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6811","nonce":3557095096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6815","nonce":3557095096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6813","nonce":3557095096}]},"public_addr":"172.21.15.112:6809/3557095096","cluster_addr":"172.21.15.112:6811/3557095096","heartbeat_back_addr":"172.21.15.112:6815/3557095096","heartbeat_front_addr":"172.21.15.112:6813/3557095096","state":["exists","up"]},{"osd":6,"uuid":"d508a037-8f56-436b-b7bf-9a1c69f6d9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6819","nonce":1267910775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6821","nonce":1267910775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6825","nonce":1267910775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6823","nonce":1267910775}]},"public_addr":"172.21.15.78:6819/1267910775","cluster_addr":"172.21.15.78:6821/1267910775","heartbeat_back_addr":"172.21.15.78:6825/1267910775","heartbeat_front_addr":"172.21.15.78:6823/1267910775","state":["exists","up"]},{"osd":7,"uuid":"e8842047-09a6-48b8-8d0a-2d54e7f89063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6817","nonce":1556494594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6819","nonce":1556494594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6823","nonce":1556494594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6821","nonce":1556494594}]},"public_addr":"172.21.15.110:6817/1556494594","cluster_addr":"172.21.15.110:6819/1556494594","heartbeat_back_addr":"172.21.15.110:6823/1556494594","heartbeat_front_addr":"172.21.15.110:6821/1556494594","state":["exists","up"]},{"osd":8,"uuid":"a3367da8-375a-4a46-a952-867d4b72ae2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6817","nonce":2481321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6819","nonce":2481321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6823","nonce":2481321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6821","nonce":2481321}]},"public_addr":"172.21.15.112:6817/2481321","cluster_addr":"172.21.15.112:6819/2481321","heartbeat_back_addr":"172.21.15.112:6823/2481321","heartbeat_front_addr":"172.21.15.112:6821/2481321","state":["exists","up"]},{"osd":9,"uuid":"d1825764-cefc-495b-99da-8ba429df0937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6827","nonce":197048350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6829","nonce":197048350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6833","nonce":197048350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6831","nonce":197048350}]},"public_addr":"172.21.15.78:6827/197048350","cluster_addr":"172.21.15.78:6829/197048350","heartbeat_back_addr":"172.21.15.78:6833/197048350","heartbeat_front_addr":"172.21.15.78:6831/197048350","state":["exists","up"]},{"osd":10,"uuid":"e1cfff7a-8916-4845-a482-d79fa275da30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6825","nonce":2101123168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6827","nonce":2101123168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6831","nonce":2101123168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6829","nonce":2101123168}]},"public_addr":"172.21.15.110:6825/2101123168","cluster_addr":"172.21.15.110:6827/2101123168","heartbeat_back_addr":"172.21.15.110:6831/2101123168","heartbeat_front_addr":"172.21.15.110:6829/2101123168","state":["exists","up"]},{"osd":11,"uuid":"fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6825","nonce":1526310654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6827","nonce":1526310654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6831","nonce":1526310654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6829","nonce":1526310654}]},"public_addr":"172.21.15.112:6825/1526310654","cluster_addr":"172.21.15.112:6827/1526310654","heartbeat_back_addr":"172.21.15.112:6831/1526310654","heartbeat_front_addr":"172.21.15.112:6829/1526310654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:17.334199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:41.400872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:08.205144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:33.550527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:58.394116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:24.699607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:50.450193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:16.223363+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:40.947727+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:06.845451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:32.410035+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:57.744364+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/2390785703":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:0/1038951931":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/391908936":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2706514734":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/2357025243":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6800/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2795778473":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2747741101":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6801/1289343426":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/3558896007":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/438261739":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/1289343426":"2024-09-12T00:50:15.928079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:58:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.918+0000 7f0b3eef5640 1 -- 172.21.15.78:0/3002374757 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0b280b53c0 msgr2=0x7f0b280b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.918+0000 7f0b3eef5640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0b280b53c0 0x7f0b280b7880 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009710 tx=0x7f0b2c006c90 comp rx=0 tx=0).stop 2024-09-11T00:58:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.918+0000 7f0b3eef5640 1 -- 172.21.15.78:0/3002374757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b381076a0 msgr2=0x7f0b38198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.918+0000 7f0b3eef5640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b381076a0 0x7f0b38198710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0b24006010 tx=0x7f0b240042f0 comp rx=0 tx=0).stop 2024-09-11T00:58:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.918+0000 7f0b3eef5640 1 -- 172.21.15.78:0/3002374757 shutdown_connections 2024-09-11T00:58:09.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.918+0000 7f0b3eef5640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0b280b53c0 0x7f0b280b7880 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:09.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.918+0000 7f0b3eef5640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38108020 0x7f0b3819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:09.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.918+0000 7f0b3eef5640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b381076a0 0x7f0b38198710 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:09.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.918+0000 7f0b3eef5640 1 --2- 172.21.15.78:0/3002374757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b38101c90 0x7f0b381981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:09.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.918+0000 7f0b3eef5640 1 -- 172.21.15.78:0/3002374757 >> 172.21.15.78:0/3002374757 conn(0x7f0b380fd6c0 msgr2=0x7f0b380ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:09.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.919+0000 7f0b3eef5640 1 -- 172.21.15.78:0/3002374757 shutdown_connections 2024-09-11T00:58:09.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:09.919+0000 7f0b3eef5640 1 -- 172.21.15.78:0/3002374757 wait complete. 2024-09-11T00:58:10.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:10 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3002374757' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:58:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:10 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3002374757' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:58:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:10 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3002374757' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:58:10.654 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:58:10.909 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:11.188 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:11 smithi078 ceph-mon[31315]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:58:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.412+0000 7f5a75808640 1 -- 172.21.15.78:0/673123196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a7010d100 msgr2=0x7f5a7010f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.412+0000 7f5a75808640 1 --2- 172.21.15.78:0/673123196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a7010d100 0x7f5a7010f560 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009ca0 tx=0x7f5a6802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.412+0000 7f5a75808640 1 -- 172.21.15.78:0/673123196 shutdown_connections 2024-09-11T00:58:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.412+0000 7f5a75808640 1 --2- 172.21.15.78:0/673123196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a7010d100 0x7f5a7010f560 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.412+0000 7f5a75808640 1 --2- 172.21.15.78:0/673123196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a701045c0 0x7f5a7010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.412+0000 7f5a75808640 1 --2- 172.21.15.78:0/673123196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a70103ca0 0x7f5a70104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.413+0000 7f5a75808640 1 -- 172.21.15.78:0/673123196 >> 172.21.15.78:0/673123196 conn(0x7f5a700fd700 msgr2=0x7f5a700ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.413+0000 7f5a75808640 1 -- 172.21.15.78:0/673123196 shutdown_connections 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.413+0000 7f5a75808640 1 -- 172.21.15.78:0/673123196 wait complete. 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.413+0000 7f5a75808640 1 Processor -- start 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.413+0000 7f5a75808640 1 -- start start 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.413+0000 7f5a75808640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a70103ca0 0x7f5a70101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.413+0000 7f5a75808640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a701045c0 0x7f5a70101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a75808640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a7010d100 0x7f5a701023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a75808640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70112d70 con 0x7f5a7010d100 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a75808640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70112bf0 con 0x7f5a70103ca0 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a75808640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70112ef0 con 0x7f5a701045c0 2024-09-11T00:58:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a7010d100 0x7f5a701023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a7010d100 0x7f5a701023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51788/0 (socket says 172.21.15.78:51788) 2024-09-11T00:58:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a6f7fe640 1 -- 172.21.15.78:0/1888325015 learned_addr learned my addr 172.21.15.78:0/1888325015 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a6effd640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a70103ca0 0x7f5a70101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a6e7fc640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a701045c0 0x7f5a70101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a6f7fe640 1 -- 172.21.15.78:0/1888325015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a701045c0 msgr2=0x7f5a70101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a6f7fe640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a701045c0 0x7f5a70101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a6f7fe640 1 -- 172.21.15.78:0/1888325015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a70103ca0 msgr2=0x7f5a70101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a6f7fe640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a70103ca0 0x7f5a70101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a6f7fe640 1 -- 172.21.15.78:0/1888325015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a701a7680 con 0x7f5a7010d100 2024-09-11T00:58:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a6effd640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a70103ca0 0x7f5a70101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a6f7fe640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a7010d100 0x7f5a701023d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a68005cc0 tx=0x7f5a6802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a74806640 1 -- 172.21.15.78:0/1888325015 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68003bf0 con 0x7f5a7010d100 2024-09-11T00:58:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a75808640 1 -- 172.21.15.78:0/1888325015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a701a7910 con 0x7f5a7010d100 2024-09-11T00:58:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a74806640 1 -- 172.21.15.78:0/1888325015 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a68003d90 con 0x7f5a7010d100 2024-09-11T00:58:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.414+0000 7f5a74806640 1 -- 172.21.15.78:0/1888325015 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6802fbc0 con 0x7f5a7010d100 2024-09-11T00:58:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.415+0000 7f5a75808640 1 -- 172.21.15.78:0/1888325015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a701a7ea0 con 0x7f5a7010d100 2024-09-11T00:58:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.416+0000 7f5a75808640 1 -- 172.21.15.78:0/1888325015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a70104080 con 0x7f5a7010d100 2024-09-11T00:58:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.417+0000 7f5a74806640 1 -- 172.21.15.78:0/1888325015 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f5a68004610 con 0x7f5a7010d100 2024-09-11T00:58:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.418+0000 7f5a74806640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a580b5440 0x7f5a580b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.418+0000 7f5a6effd640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a580b5440 0x7f5a580b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.418+0000 7f5a74806640 1 -- 172.21.15.78:0/1888325015 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5a681073b0 con 0x7f5a7010d100 2024-09-11T00:58:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.418+0000 7f5a6effd640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a580b5440 0x7f5a580b7900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a600058c0 tx=0x7f5a60005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.422+0000 7f5a74806640 1 -- 172.21.15.78:0/1888325015 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5a680ce060 con 0x7f5a7010d100 2024-09-11T00:58:11.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:11 smithi110 ceph-mon[34712]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:58:11.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:11 smithi112 ceph-mon[35384]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:58:11.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.644+0000 7f5a75808640 1 -- 172.21.15.78:0/1888325015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a701a8230 con 0x7f5a7010d100 2024-09-11T00:58:11.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.644+0000 7f5a74806640 1 -- 172.21.15.78:0/1888325015 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11979 (secure 0 0 0) 0x7f5a680098b0 con 0x7f5a7010d100 2024-09-11T00:58:11.646 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:11.646 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-09-11T00:57:05:238610+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T00:57:04.821148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T00:58:11.647 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-11T00:58:11.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.646+0000 7f5a75808640 1 -- 172.21.15.78:0/1888325015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a580b5440 msgr2=0x7f5a580b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:11.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.646+0000 7f5a75808640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a580b5440 0x7f5a580b7900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a600058c0 tx=0x7f5a60005850 comp rx=0 tx=0).stop 2024-09-11T00:58:11.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.646+0000 7f5a75808640 1 -- 172.21.15.78:0/1888325015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a7010d100 msgr2=0x7f5a701023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.646+0000 7f5a75808640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a7010d100 0x7f5a701023d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a68005cc0 tx=0x7f5a6802f380 comp rx=0 tx=0).stop 2024-09-11T00:58:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.647+0000 7f5a75808640 1 -- 172.21.15.78:0/1888325015 shutdown_connections 2024-09-11T00:58:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.647+0000 7f5a75808640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a580b5440 0x7f5a580b7900 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.647+0000 7f5a75808640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a7010d100 0x7f5a701023d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.647+0000 7f5a75808640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a701045c0 0x7f5a70101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.647+0000 7f5a75808640 1 --2- 172.21.15.78:0/1888325015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a70103ca0 0x7f5a70101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.647+0000 7f5a75808640 1 -- 172.21.15.78:0/1888325015 >> 172.21.15.78:0/1888325015 conn(0x7f5a700fd700 msgr2=0x7f5a700fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.647+0000 7f5a75808640 1 -- 172.21.15.78:0/1888325015 shutdown_connections 2024-09-11T00:58:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:11.647+0000 7f5a75808640 1 -- 172.21.15.78:0/1888325015 wait complete. 2024-09-11T00:58:12.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:12 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1888325015' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:12.372 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T00:56:53.792170+0000', 'modified': '2024-09-11T00:57:04.821148+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24599, 'mds_1': 24580, 'mds_2': 24565, 'mds_3': 24563, 'mds_4': 14808}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14796': {'gid': 14796, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6839/3902249887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 3902249887}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 3902249887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14808': {'gid': 14808, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.78:6841/3713123708', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6840', 'nonce': 3713123708}, {'type': 'v1', 'addr': '172.21.15.78:6841', 'nonce': 3713123708}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6833/2579452587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6832', 'nonce': 2579452587}, {'type': 'v1', 'addr': '172.21.15.110:6833', 'nonce': 2579452587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6833/1732756287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6832', 'nonce': 1732756287}, {'type': 'v1', 'addr': '172.21.15.112:6833', 'nonce': 1732756287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6835/3549318748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 3549318748}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 3549318748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24553': {'gid': 24553, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6835/303956922', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 303956922}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 303956922}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24563': {'gid': 24563, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.110:6837/1873031527', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 1873031527}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 1873031527}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24565': {'gid': 24565, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.112:6837/854577265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6836', 'nonce': 854577265}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 854577265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24580': {'gid': 24580, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.112:6839/2141614771', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 2141614771}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 2141614771}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24599': {'gid': 24599, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.110:6839/3180315245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 3180315245}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 3180315245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24599, 'qdb_cluster': [14808, 24563, 24565, 24580, 24599]} 2024-09-11T00:58:12.373 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T00:58:12.373 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-11T00:58:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:12 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1888325015' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:12 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1888325015' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:12.622 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.112+0000 7f3c7359e640 1 -- 172.21.15.78:0/2321174503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c74103b10 msgr2=0x7f3c74103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.112+0000 7f3c7359e640 1 --2- 172.21.15.78:0/2321174503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c74103b10 0x7f3c74103ef0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009900 tx=0x7f3c6402a410 comp rx=0 tx=0).stop 2024-09-11T00:58:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.112+0000 7f3c7359e640 1 -- 172.21.15.78:0/2321174503 shutdown_connections 2024-09-11T00:58:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.112+0000 7f3c7359e640 1 --2- 172.21.15.78:0/2321174503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c7410cf70 0x7f3c7410f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.112+0000 7f3c7359e640 1 --2- 172.21.15.78:0/2321174503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c74104430 0x7f3c7410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.112+0000 7f3c7359e640 1 --2- 172.21.15.78:0/2321174503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c74103b10 0x7f3c74103ef0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.112+0000 7f3c7359e640 1 -- 172.21.15.78:0/2321174503 >> 172.21.15.78:0/2321174503 conn(0x7f3c740fd5b0 msgr2=0x7f3c740ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.112+0000 7f3c7359e640 1 -- 172.21.15.78:0/2321174503 shutdown_connections 2024-09-11T00:58:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.112+0000 7f3c7359e640 1 -- 172.21.15.78:0/2321174503 wait complete. 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.113+0000 7f3c7359e640 1 Processor -- start 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.113+0000 7f3c7359e640 1 -- start start 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c7359e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c74103b10 0x7f3c74199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c7359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c74104430 0x7f3c741995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c7359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c7410cf70 0x7f3c7419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c7359e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c74112c40 con 0x7f3c7410cf70 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c7359e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3c74112ac0 con 0x7f3c74103b10 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c7359e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c74112dc0 con 0x7f3c74104430 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c72d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c7410cf70 0x7f3c7419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c72d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c7410cf70 0x7f3c7419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51804/0 (socket says 172.21.15.78:51804) 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c72d9d640 1 -- 172.21.15.78:0/1501092102 learned_addr learned my addr 172.21.15.78:0/1501092102 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c74104430 0x7f3c741995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c7259c640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c74103b10 0x7f3c74199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c72d9d640 1 -- 172.21.15.78:0/1501092102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c74104430 msgr2=0x7f3c741995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c72d9d640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c74104430 0x7f3c741995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c72d9d640 1 -- 172.21.15.78:0/1501092102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c74103b10 msgr2=0x7f3c74199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c72d9d640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c74103b10 0x7f3c74199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.114+0000 7f3c72d9d640 1 -- 172.21.15.78:0/1501092102 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7419f130 con 0x7f3c7410cf70 2024-09-11T00:58:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.115+0000 7f3c72d9d640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c7410cf70 0x7f3c7419e960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00cad0 tx=0x7f3c6c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.118+0000 7f3c6bfff640 1 -- 172.21.15.78:0/1501092102 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c00ae20 con 0x7f3c7410cf70 2024-09-11T00:58:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.118+0000 7f3c7359e640 1 -- 172.21.15.78:0/1501092102 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7419f420 con 0x7f3c7410cf70 2024-09-11T00:58:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.118+0000 7f3c6bfff640 1 -- 172.21.15.78:0/1501092102 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c6c005020 con 0x7f3c7410cf70 2024-09-11T00:58:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.118+0000 7f3c7359e640 1 -- 172.21.15.78:0/1501092102 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c741a5b60 con 0x7f3c7410cf70 2024-09-11T00:58:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.118+0000 7f3c6bfff640 1 -- 172.21.15.78:0/1501092102 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c00f6b0 con 0x7f3c7410cf70 2024-09-11T00:58:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.121+0000 7f3c6bfff640 1 -- 172.21.15.78:0/1501092102 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f3c6c0054f0 con 0x7f3c7410cf70 2024-09-11T00:58:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.121+0000 7f3c7afd6740 1 Processor -- start 2024-09-11T00:58:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.121+0000 7f3c7afd6740 1 -- start start 2024-09-11T00:58:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.121+0000 7f3c7afd6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 0x56296cfda2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.121+0000 7f3c7afd6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfda800 0x56296cfe6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.121+0000 7f3c7afd6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfe6790 0x56296cfe8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.121+0000 7f3c7afd6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56296cf51c10 con 0x56296cfe6790 2024-09-11T00:58:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.121+0000 7f3c7afd6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56296cf41040 con 0x56296cfda800 2024-09-11T00:58:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.121+0000 7f3c7afd6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56296cf44e50 con 0x56296cfd9ee0 2024-09-11T00:58:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c69d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 0x56296cfda2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c69d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 0x56296cfda2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54650/0 (socket says 172.21.15.78:54650) 2024-09-11T00:58:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c69d9b640 1 -- 172.21.15.78:0/60585792 learned_addr learned my addr 172.21.15.78:0/60585792 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c7259c640 1 --2- 172.21.15.78:0/60585792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfda800 0x56296cfe6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/60585792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfe6790 0x56296cfe8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c6bfff640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3c600b98f0 0x7f3c600bbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c7259c640 1 -- 172.21.15.78:0/60585792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 msgr2=0x56296cfda2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c7259c640 1 --2- 172.21.15.78:0/60585792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 0x56296cfda2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c7259c640 1 -- 172.21.15.78:0/60585792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfe6790 msgr2=0x56296cfe8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c7259c640 1 --2- 172.21.15.78:0/60585792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfe6790 0x56296cfe8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c6bfff640 1 -- 172.21.15.78:0/1501092102 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3c6c0e5db0 con 0x7f3c7410cf70 2024-09-11T00:58:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c7259c640 1 -- 172.21.15.78:0/60585792 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56296cc9f540 con 0x56296cfda800 2024-09-11T00:58:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c72d9d640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3c600b98f0 0x7f3c600bbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/60585792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 0x56296cfda2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:58:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.122+0000 7f3c7259c640 1 --2- 172.21.15.78:0/60585792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfda800 0x56296cfe6250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009230 tx=0x7f3c64031de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=485d0b7407c2fc5b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:13.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.123+0000 7f3c72d9d640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3c600b98f0 0x7f3c600bbdb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0a8b70 tx=0x7f3c6c0a8a00 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:13.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.123+0000 7f3c68d99640 1 -- 172.21.15.78:0/60585792 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64040070 con 0x56296cfda800 2024-09-11T00:58:13.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.123+0000 7f3c68d99640 1 -- 172.21.15.78:0/60585792 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c640316a0 con 0x56296cfda800 2024-09-11T00:58:13.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.123+0000 7f3c68d99640 1 -- 172.21.15.78:0/60585792 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64035040 con 0x56296cfda800 2024-09-11T00:58:13.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.123+0000 7f3c7afd6740 1 -- 172.21.15.78:0/60585792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfda800 msgr2=0x56296cfe6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:13.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.123+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/60585792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfda800 0x56296cfe6250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009230 tx=0x7f3c64031de0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 -- 172.21.15.78:0/60585792 shutdown_connections 2024-09-11T00:58:13.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/60585792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfe6790 0x56296cfe8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/60585792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfda800 0x56296cfe6250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/60585792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 0x56296cfda2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 -- 172.21.15.78:0/60585792 >> 172.21.15.78:0/60585792 conn(0x56296cfd7d10 msgr2=0x56296cfd8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:13.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 -- 172.21.15.78:0/60585792 shutdown_connections 2024-09-11T00:58:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 -- 172.21.15.78:0/60585792 wait complete. 2024-09-11T00:58:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 -- start start 2024-09-11T00:58:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 Processor -- start 2024-09-11T00:58:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 0x56296cfe3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfda800 0x56296cfe3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 0x56296cfe4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56296cda25f0 con 0x56296cfda800 2024-09-11T00:58:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56296cedc010 con 0x56296cfe6790 2024-09-11T00:58:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56296cfe93e0 con 0x56296cfd9ee0 2024-09-11T00:58:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfda800 0x56296cfe3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7afd6740 10 client.0 _ll_register_callbacks cb 0x56296cf3b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c69d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 0x56296cfe3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfda800 0x56296cfe3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51826/0 (socket says 172.21.15.78:51826) 2024-09-11T00:58:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.124+0000 7f3c7259c640 1 -- 172.21.15.78:0/2328201640 learned_addr learned my addr 172.21.15.78:0/2328201640 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 0x56296cfe4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c7259c640 1 -- 172.21.15.78:0/2328201640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 msgr2=0x56296cfe3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c7259c640 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 0x56296cfe3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c7259c640 1 -- 172.21.15.78:0/2328201640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 msgr2=0x56296cfe4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c7259c640 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 0x56296cfe4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c7259c640 1 -- 172.21.15.78:0/2328201640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56296cf09a00 con 0x56296cfda800 2024-09-11T00:58:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 0x56296cfe3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:58:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c7259c640 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfda800 0x56296cfe3cc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3c640058d0 tx=0x7f3c64005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10faab7d2e69610 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c59ffb640 1 -- 172.21.15.78:0/2328201640 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6404c070 con 0x56296cfda800 2024-09-11T00:58:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c59ffb640 1 -- 172.21.15.78:0/2328201640 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3c6403c440 con 0x56296cfda800 2024-09-11T00:58:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c7afd6740 1 -- 172.21.15.78:0/2328201640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c640316a0 con 0x56296cfda800 2024-09-11T00:58:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.125+0000 7f3c59ffb640 1 -- 172.21.15.78:0/2328201640 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c64058050 con 0x56296cfda800 2024-09-11T00:58:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c7afd6740 10 client.15174 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c7afd6740 1 -- 172.21.15.78:0/2328201640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56296cff4fc0 con 0x56296cfda800 2024-09-11T00:58:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c59ffb640 1 -- 172.21.15.78:0/2328201640 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c640316a0 con 0x56296cfda800 2024-09-11T00:58:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c59ffb640 1 -- 172.21.15.78:0/2328201640 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3c640446a0 con 0x56296cfda800 2024-09-11T00:58:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c7afd6740 10 client.15174 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c7afd6740 10 client.15174 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T00:58:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c7afd6740 20 client.15174 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56296cff71f0 0x56296cff8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c7afd6740 4 client.15174 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c7afd6740 1 -- 172.21.15.78:0/2328201640 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56296cf53b60 con 0x56296cff71f0 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56296cff71f0 0x56296cff8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c59ffb640 10 client.15174 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.126+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56296cff71f0 0x56296cff8660 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4eb4e53d519a139c server_cookie=989435f5f276e329 in_seq=0 out_seq=0 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c59ffb640 1 -- 172.21.15.78:0/2328201640 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56296cf53b60 con 0x56296cff71f0 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c59ffb640 10 client.15174 handle_command_reply: tid=0 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c7afd6740 1 client.15174 shutdown 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c7afd6740 1 -- 172.21.15.78:0/2328201640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfda800 msgr2=0x56296cfe3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfda800 0x56296cfe3cc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3c640058d0 tx=0x7f3c64005900 comp rx=0 tx=0).stop 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c7afd6740 1 -- 172.21.15.78:0/2328201640 shutdown_connections 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56296cff71f0 0x56296cff8660 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 0x56296cfe4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfda800 0x56296cfe3cc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c59ffb640 0 client.15174 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T00:58:13.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/2328201640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfd9ee0 0x56296cfe3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c7afd6740 1 -- 172.21.15.78:0/2328201640 >> 172.21.15.78:0/2328201640 conn(0x56296cfd7d10 msgr2=0x56296cfdd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:13.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c7afd6740 1 -- 172.21.15.78:0/2328201640 shutdown_connections 2024-09-11T00:58:13.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c7afd6740 1 -- 172.21.15.78:0/2328201640 wait complete. 2024-09-11T00:58:13.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.127+0000 7f3c7afd6740 20 client.15174 trim_cache size 0 max 16384 2024-09-11T00:58:13.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c7afd6740 1 Processor -- start 2024-09-11T00:58:13.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c7afd6740 1 -- start start 2024-09-11T00:58:13.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c7afd6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 0x56296cfe4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c7afd6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfda800 0x56296cfddf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c7afd6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 0x56296cfe54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c7afd6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56296cff69a0 con 0x56296cfd9ee0 2024-09-11T00:58:13.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c7afd6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56296cff7070 con 0x56296cfe6790 2024-09-11T00:58:13.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c7afd6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56296cff7740 con 0x56296cfda800 2024-09-11T00:58:13.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c69d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 0x56296cfe4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c69d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 0x56296cfe4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51832/0 (socket says 172.21.15.78:51832) 2024-09-11T00:58:13.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c69d9b640 1 -- 172.21.15.78:0/1607885967 learned_addr learned my addr 172.21.15.78:0/1607885967 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:13.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c7259c640 1 --2- 172.21.15.78:0/1607885967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfda800 0x56296cfddf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c69d9b640 1 -- 172.21.15.78:0/1607885967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfda800 msgr2=0x56296cfddf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:13.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/1607885967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfda800 0x56296cfddf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.146+0000 7f3c69d9b640 1 -- 172.21.15.78:0/1607885967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 msgr2=0x56296cfe54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.147+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/1607885967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 0x56296cfe54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.147+0000 7f3c69d9b640 1 -- 172.21.15.78:0/1607885967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56296cfd8790 con 0x56296cfd9ee0 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.147+0000 7f3c7259c640 1 --2- 172.21.15.78:0/1607885967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfda800 0x56296cfddf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.147+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/1607885967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 0x56296cfe4540 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c017700 tx=0x7f3c5c0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6eaa9cc54e1256 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.147+0000 7f3c5bfff640 1 -- 172.21.15.78:0/1607885967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c01a0b0 con 0x56296cfd9ee0 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.147+0000 7f3c5bfff640 1 -- 172.21.15.78:0/1607885967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c5c01a250 con 0x56296cfd9ee0 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.147+0000 7f3c5bfff640 1 -- 172.21.15.78:0/1607885967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c018620 con 0x56296cfd9ee0 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1607885967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 msgr2=0x56296cfe4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/1607885967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 0x56296cfe4540 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c017700 tx=0x7f3c5c0094c0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1607885967 shutdown_connections 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/1607885967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 0x56296cfe54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/1607885967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfda800 0x56296cfddf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/1607885967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 0x56296cfe4540 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1607885967 >> 172.21.15.78:0/1607885967 conn(0x56296cfd7d10 msgr2=0x56296cfe4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1607885967 shutdown_connections 2024-09-11T00:58:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1607885967 wait complete. 2024-09-11T00:58:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 -- start start 2024-09-11T00:58:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 Processor -- start 2024-09-11T00:58:13.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 0x56296cffacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfda800 0x56296cffb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 0x56296d000480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56296cff84e0 con 0x56296cfd9ee0 2024-09-11T00:58:13.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56296cff7e10 con 0x56296cfe6790 2024-09-11T00:58:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56296cff8bb0 con 0x56296cfda800 2024-09-11T00:58:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c7afd6740 10 client.0 _ll_register_callbacks cb 0x7f3c5c009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c69d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 0x56296cffacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.148+0000 7f3c69d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 0x56296cffacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51834/0 (socket says 172.21.15.78:51834) 2024-09-11T00:58:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c69d9b640 1 -- 172.21.15.78:0/1367296236 learned_addr learned my addr 172.21.15.78:0/1367296236 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 0x56296d000480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c7259c640 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfda800 0x56296cffb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c69d9b640 1 -- 172.21.15.78:0/1367296236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfda800 msgr2=0x56296cffb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfda800 0x56296cffb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c69d9b640 1 -- 172.21.15.78:0/1367296236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 msgr2=0x56296d000480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 0x56296d000480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c69d9b640 1 -- 172.21.15.78:0/1367296236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56296d0009c0 con 0x56296cfd9ee0 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c69d9b640 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 0x56296cffacf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c01a220 tx=0x7f3c5c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5336206c403e40d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c59ffb640 1 -- 172.21.15.78:0/1367296236 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c01c310 con 0x56296cfd9ee0 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c59ffb640 1 -- 172.21.15.78:0/1367296236 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3c5c01ca30 con 0x56296cfd9ee0 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1367296236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c5c01a250 con 0x56296cfd9ee0 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c59ffb640 1 -- 172.21.15.78:0/1367296236 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c5c03d050 con 0x56296cfd9ee0 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c7afd6740 10 client.15180 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1367296236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56296d064280 con 0x56296cfd9ee0 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.149+0000 7f3c59ffb640 1 -- 172.21.15.78:0/1367296236 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c5c01a250 con 0x56296cfd9ee0 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c59ffb640 1 -- 172.21.15.78:0/1367296236 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3c5c02b020 con 0x56296cfd9ee0 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 10 client.15180 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 10 client.15180 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 20 client.15180 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56296d068a50 0x56296d067760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 4 client.15180 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1367296236 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "status"}) -- 0x56296cf4f610 con 0x56296d068a50 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7259c640 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56296d068a50 0x56296d067760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c59ffb640 10 client.15180 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7259c640 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56296d068a50 0x56296d067760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fb9ea8e9ec5065e0 server_cookie=289fce8584e1751 in_seq=0 out_seq=0 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c59ffb640 1 -- 172.21.15.78:0/1367296236 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56296cf4f610 con 0x56296d068a50 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c59ffb640 10 client.15180 handle_command_reply: tid=0 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 1 client.15180 shutdown 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1367296236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 msgr2=0x56296cffacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 0x56296cffacf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c01a220 tx=0x7f3c5c01a050 comp rx=0 tx=0).stop 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1367296236 shutdown_connections 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56296d068a50 0x56296d067760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c59ffb640 0 client.15180 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T00:58:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56296cfe6790 0x56296d000480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56296cfda800 0x56296cffb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 1 --2- 172.21.15.78:0/1367296236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296cfd9ee0 0x56296cffacf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.150+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1367296236 >> 172.21.15.78:0/1367296236 conn(0x56296cfd7d10 msgr2=0x56296cfe55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1367296236 shutdown_connections 2024-09-11T00:58:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7afd6740 1 -- 172.21.15.78:0/1367296236 wait complete. 2024-09-11T00:58:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7afd6740 20 client.15180 trim_cache size 0 max 16384 2024-09-11T00:58:13.159 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:58:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:58:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 4, 2024-09-11T00:58:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 14808, 2024-09-11T00:58:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T00:58:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T00:58:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T00:58:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 69.921429412999998, 2024-09-11T00:58:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-11T00:58:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-09-11T00:58:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-09-11T00:58:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 93.759253190999999 2024-09-11T00:58:13.160 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:58:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7359e640 1 -- 172.21.15.78:0/1501092102 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3c600b98f0 msgr2=0x7f3c600bbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7359e640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3c600b98f0 0x7f3c600bbdb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0a8b70 tx=0x7f3c6c0a8a00 comp rx=0 tx=0).stop 2024-09-11T00:58:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7359e640 1 -- 172.21.15.78:0/1501092102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c7410cf70 msgr2=0x7f3c7419e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7359e640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c7410cf70 0x7f3c7419e960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00cad0 tx=0x7f3c6c00cf90 comp rx=0 tx=0).stop 2024-09-11T00:58:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7359e640 1 -- 172.21.15.78:0/1501092102 shutdown_connections 2024-09-11T00:58:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7359e640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3c600b98f0 0x7f3c600bbdb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7359e640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c7410cf70 0x7f3c7419e960 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7359e640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c74104430 0x7f3c741995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7359e640 1 --2- 172.21.15.78:0/1501092102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c74103b10 0x7f3c74199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7359e640 1 -- 172.21.15.78:0/1501092102 >> 172.21.15.78:0/1501092102 conn(0x7f3c740fd5b0 msgr2=0x7f3c740fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:13.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7359e640 1 -- 172.21.15.78:0/1501092102 shutdown_connections 2024-09-11T00:58:13.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:13.151+0000 7f3c7359e640 1 -- 172.21.15.78:0/1501092102 wait complete. 2024-09-11T00:58:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:13 smithi110 ceph-mon[34712]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:13.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:13 smithi112 ceph-mon[35384]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:13.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:13 smithi078 ceph-mon[31315]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:13.932 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T00:58:14.180 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:14.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.679+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/764806378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdd0109150 msgr2=0x7fbdd010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:14.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.679+0000 7fbdd4a7f640 1 --2- 172.21.15.78:0/764806378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdd0109150 0x7fbdd010f950 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009ca0 tx=0x7fbdc802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.679+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/764806378 shutdown_connections 2024-09-11T00:58:14.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.679+0000 7fbdd4a7f640 1 --2- 172.21.15.78:0/764806378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdd0109150 0x7fbdd010f950 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.679+0000 7fbdd4a7f640 1 --2- 172.21.15.78:0/764806378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdd01087d0 0x7fbdd0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.679+0000 7fbdd4a7f640 1 --2- 172.21.15.78:0/764806378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd0102dc0 0x7fbdd01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.679+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/764806378 >> 172.21.15.78:0/764806378 conn(0x7fbdd00fe830 msgr2=0x7fbdd0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:14.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.679+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/764806378 shutdown_connections 2024-09-11T00:58:14.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.679+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/764806378 wait complete. 2024-09-11T00:58:14.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.679+0000 7fbdd4a7f640 1 Processor -- start 2024-09-11T00:58:14.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.679+0000 7fbdd4a7f640 1 -- start start 2024-09-11T00:58:14.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdd4a7f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd0102dc0 0x7fbdd01991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdd4a7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdd01087d0 0x7fbdd0199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdd4a7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdd0109150 0x7fbdd019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdd4a7f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0112e10 con 0x7fbdd01087d0 2024-09-11T00:58:14.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdd4a7f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0112c90 con 0x7fbdd0102dc0 2024-09-11T00:58:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdd4a7f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0112f90 con 0x7fbdd0109150 2024-09-11T00:58:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdcdd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdd01087d0 0x7fbdd0199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdcdd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdd01087d0 0x7fbdd0199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37844/0 (socket says 172.21.15.78:37844) 2024-09-11T00:58:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdcdd9b640 1 -- 172.21.15.78:0/283198206 learned_addr learned my addr 172.21.15.78:0/283198206 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdced9d640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdd0109150 0x7fbdd019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdce59c640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd0102dc0 0x7fbdd01991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdcdd9b640 1 -- 172.21.15.78:0/283198206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdd0109150 msgr2=0x7fbdd019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdd0109150 0x7fbdd019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdcdd9b640 1 -- 172.21.15.78:0/283198206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd0102dc0 msgr2=0x7fbdd01991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd0102dc0 0x7fbdd01991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdcdd9b640 1 -- 172.21.15.78:0/283198206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd019f270 con 0x7fbdd01087d0 2024-09-11T00:58:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdce59c640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd0102dc0 0x7fbdd01991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdd01087d0 0x7fbdd0199700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400c920 tx=0x7fbdc400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdbf7fe640 1 -- 172.21.15.78:0/283198206 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4007c70 con 0x7fbdd01087d0 2024-09-11T00:58:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdbf7fe640 1 -- 172.21.15.78:0/283198206 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc4007e10 con 0x7fbdd01087d0 2024-09-11T00:58:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/283198206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd019f560 con 0x7fbdd01087d0 2024-09-11T00:58:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdbf7fe640 1 -- 172.21.15.78:0/283198206 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4018820 con 0x7fbdd01087d0 2024-09-11T00:58:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.680+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/283198206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd01a5c20 con 0x7fbdd01087d0 2024-09-11T00:58:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.683+0000 7fbdbf7fe640 1 -- 172.21.15.78:0/283198206 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fbdc400f450 con 0x7fbdd01087d0 2024-09-11T00:58:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.683+0000 7fbdd6e54740 1 Processor -- start 2024-09-11T00:58:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.683+0000 7fbdd6e54740 1 -- start start 2024-09-11T00:58:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.683+0000 7fbdd6e54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4140 0x5601959e4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.683+0000 7fbdd6e54740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 0x5601959f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.683+0000 7fbdd6e54740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959f09f0 0x5601959f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.683+0000 7fbdd6e54740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560195957c10 con 0x5601959e4a60 2024-09-11T00:58:14.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.683+0000 7fbdd6e54740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560195947040 con 0x5601959f09f0 2024-09-11T00:58:14.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.683+0000 7fbdd6e54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56019594ae50 con 0x5601959e4140 2024-09-11T00:58:14.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.683+0000 7fbdce59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 0x5601959f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdce59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 0x5601959f04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37846/0 (socket says 172.21.15.78:37846) 2024-09-11T00:58:14.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdce59c640 1 -- 172.21.15.78:0/4291949631 learned_addr learned my addr 172.21.15.78:0/4291949631 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:14.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdbf7fe640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbdb80b53c0 0x7fbdb80b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdced9d640 1 --2- 172.21.15.78:0/4291949631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4140 0x5601959e4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/4291949631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959f09f0 0x5601959f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdce59c640 1 -- 172.21.15.78:0/4291949631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4140 msgr2=0x5601959e4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:14.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdce59c640 1 --2- 172.21.15.78:0/4291949631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4140 0x5601959e4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdce59c640 1 -- 172.21.15.78:0/4291949631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959f09f0 msgr2=0x5601959f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdced9d640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbdb80b53c0 0x7fbdb80b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdbf7fe640 1 -- 172.21.15.78:0/283198206 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbdc40e5ff0 con 0x7fbdd01087d0 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdce59c640 1 --2- 172.21.15.78:0/4291949631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959f09f0 0x5601959f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdce59c640 1 -- 172.21.15.78:0/4291949631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601956a5540 con 0x5601959e4a60 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/4291949631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959f09f0 0x5601959f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdce59c640 1 --2- 172.21.15.78:0/4291949631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 0x5601959f04b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009c30 tx=0x7fbdc0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6170241c0ce67049 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdced9d640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbdb80b53c0 0x7fbdb80b7880 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbdc802f500 tx=0x7fbdc802a8c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdbcff9640 1 -- 172.21.15.78:0/4291949631 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0004030 con 0x5601959e4a60 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdbcff9640 1 -- 172.21.15.78:0/4291949631 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc00041d0 con 0x5601959e4a60 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.684+0000 7fbdbcff9640 1 -- 172.21.15.78:0/4291949631 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00126b0 con 0x5601959e4a60 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.685+0000 7fbdd6e54740 1 -- 172.21.15.78:0/4291949631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 msgr2=0x5601959f04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:14.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.685+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/4291949631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 0x5601959f04b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009c30 tx=0x7fbdc0010740 comp rx=0 tx=0).stop 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.685+0000 7fbdd6e54740 1 -- 172.21.15.78:0/4291949631 shutdown_connections 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.685+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/4291949631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959f09f0 0x5601959f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.685+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/4291949631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 0x5601959f04b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.685+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/4291949631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4140 0x5601959e4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.685+0000 7fbdd6e54740 1 -- 172.21.15.78:0/4291949631 >> 172.21.15.78:0/4291949631 conn(0x5601959e1f70 msgr2=0x5601959e3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.685+0000 7fbdd6e54740 1 -- 172.21.15.78:0/4291949631 shutdown_connections 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.685+0000 7fbdd6e54740 1 -- 172.21.15.78:0/4291949631 wait complete. 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdd6e54740 1 -- start start 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdd6e54740 1 Processor -- start 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdd6e54740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959e4140 0x5601959eda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdd6e54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4a60 0x5601959edf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdd6e54740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959f09f0 0x5601959ee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdd6e54740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5601957a85f0 con 0x5601959f09f0 2024-09-11T00:58:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdd6e54740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5601958e2010 con 0x5601959e4140 2024-09-11T00:58:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdd6e54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5601959f3670 con 0x5601959e4a60 2024-09-11T00:58:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdd6e54740 10 client.0 _ll_register_callbacks cb 0x560195941340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdced9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959f09f0 0x5601959ee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdced9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959f09f0 0x5601959ee490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37862/0 (socket says 172.21.15.78:37862) 2024-09-11T00:58:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdced9d640 1 -- 172.21.15.78:0/3670790663 learned_addr learned my addr 172.21.15.78:0/3670790663 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4a60 0x5601959edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdced9d640 1 -- 172.21.15.78:0/3670790663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4a60 msgr2=0x5601959edf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdced9d640 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4a60 0x5601959edf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdce59c640 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959e4140 0x5601959eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdced9d640 1 -- 172.21.15.78:0/3670790663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959e4140 msgr2=0x5601959eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdced9d640 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959e4140 0x5601959eda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.686+0000 7fbdced9d640 1 -- 172.21.15.78:0/3670790663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56019590fa00 con 0x5601959f09f0 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdced9d640 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959f09f0 0x5601959ee490 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8035000 tx=0x7fbdc80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cd9c39a2619455 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdb5ffb640 1 -- 172.21.15.78:0/3670790663 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80046c0 con 0x5601959f09f0 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdb5ffb640 1 -- 172.21.15.78:0/3670790663 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbdc802f660 con 0x5601959f09f0 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdd6e54740 1 -- 172.21.15.78:0/3670790663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc00041d0 con 0x5601959f09f0 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdb5ffb640 1 -- 172.21.15.78:0/3670790663 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbdc805c050 con 0x5601959f09f0 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdd6e54740 10 client.15192 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdd6e54740 1 -- 172.21.15.78:0/3670790663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5601959ff1f0 con 0x5601959f09f0 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdb5ffb640 1 -- 172.21.15.78:0/3670790663 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc00041d0 con 0x5601959f09f0 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdb5ffb640 1 -- 172.21.15.78:0/3670790663 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbdc804a050 con 0x5601959f09f0 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdd6e54740 10 client.15192 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:14.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdd6e54740 10 client.15192 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdd6e54740 20 client.15192 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x560195a018f0 0x560195a02d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdd6e54740 4 client.15192 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.687+0000 7fbdd6e54740 1 -- 172.21.15.78:0/3670790663 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560195959b60 con 0x560195a018f0 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.688+0000 7fbdce59c640 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x560195a018f0 0x560195a02d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.688+0000 7fbdce59c640 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x560195a018f0 0x560195a02d60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3bc088d7cbaed707 server_cookie=cab558a064c92050 in_seq=0 out_seq=0 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.688+0000 7fbdb5ffb640 10 client.15192 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdb5ffb640 1 -- 172.21.15.78:0/3670790663 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560195959b60 con 0x560195a018f0 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdb5ffb640 10 client.15192 handle_command_reply: tid=0 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdd6e54740 1 client.15192 shutdown 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdd6e54740 1 -- 172.21.15.78:0/3670790663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959f09f0 msgr2=0x5601959ee490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959f09f0 0x5601959ee490 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8035000 tx=0x7fbdc80095c0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdd6e54740 1 -- 172.21.15.78:0/3670790663 shutdown_connections 2024-09-11T00:58:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x560195a018f0 0x560195a02d60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdb5ffb640 0 client.15192 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T00:58:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959f09f0 0x5601959ee490 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4a60 0x5601959edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/3670790663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959e4140 0x5601959eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdd6e54740 1 -- 172.21.15.78:0/3670790663 >> 172.21.15.78:0/3670790663 conn(0x5601959e1f70 msgr2=0x5601959dcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdd6e54740 1 -- 172.21.15.78:0/3670790663 shutdown_connections 2024-09-11T00:58:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdd6e54740 1 -- 172.21.15.78:0/3670790663 wait complete. 2024-09-11T00:58:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.689+0000 7fbdd6e54740 20 client.15192 trim_cache size 0 max 16384 2024-09-11T00:58:14.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdd6e54740 1 Processor -- start 2024-09-11T00:58:14.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdd6e54740 1 -- start start 2024-09-11T00:58:14.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdd6e54740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959e4140 0x5601959fd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdd6e54740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 0x5601959e8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdd6e54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959f09f0 0x5601959e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdd6e54740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560195a010a0 con 0x5601959e4a60 2024-09-11T00:58:14.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdd6e54740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560195a01770 con 0x5601959e4140 2024-09-11T00:58:14.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdd6e54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560195a01e40 con 0x5601959f09f0 2024-09-11T00:58:14.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdcdd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 0x5601959e8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdcdd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 0x5601959e8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37876/0 (socket says 172.21.15.78:37876) 2024-09-11T00:58:14.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdcdd9b640 1 -- 172.21.15.78:0/724555359 learned_addr learned my addr 172.21.15.78:0/724555359 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:14.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdce59c640 1 --2- 172.21.15.78:0/724555359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959e4140 0x5601959fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdcdd9b640 1 -- 172.21.15.78:0/724555359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959f09f0 msgr2=0x5601959e91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:14.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.707+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/724555359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959f09f0 0x5601959e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdcdd9b640 1 -- 172.21.15.78:0/724555359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959e4140 msgr2=0x5601959fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:14.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/724555359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959e4140 0x5601959fd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdcdd9b640 1 -- 172.21.15.78:0/724555359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601959e29f0 con 0x5601959e4a60 2024-09-11T00:58:14.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdce59c640 1 --2- 172.21.15.78:0/724555359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959e4140 0x5601959fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:14.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/724555359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 0x5601959e8c90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40a9810 tx=0x7fbdc401cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eede39e8c3e27ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdb7fff640 1 -- 172.21.15.78:0/724555359 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc40ac5a0 con 0x5601959e4a60 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdb7fff640 1 -- 172.21.15.78:0/724555359 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc40b1080 con 0x5601959e4a60 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdb7fff640 1 -- 172.21.15.78:0/724555359 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc40a89a0 con 0x5601959e4a60 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdd6e54740 1 -- 172.21.15.78:0/724555359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 msgr2=0x5601959e8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/724555359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 0x5601959e8c90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40a9810 tx=0x7fbdc401cf90 comp rx=0 tx=0).stop 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdd6e54740 1 -- 172.21.15.78:0/724555359 shutdown_connections 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/724555359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959f09f0 0x5601959e91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/724555359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4a60 0x5601959e8c90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/724555359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959e4140 0x5601959fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdd6e54740 1 -- 172.21.15.78:0/724555359 >> 172.21.15.78:0/724555359 conn(0x5601959e1f70 msgr2=0x5601959d8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.708+0000 7fbdd6e54740 1 -- 172.21.15.78:0/724555359 shutdown_connections 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdd6e54740 1 -- 172.21.15.78:0/724555359 wait complete. 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdd6e54740 1 -- start start 2024-09-11T00:58:14.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdd6e54740 1 Processor -- start 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdd6e54740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4140 0x560195a05480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdd6e54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4a60 0x560195a059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdd6e54740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959f09f0 0x560195a0abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdd6e54740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560195a02be0 con 0x5601959e4140 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdd6e54740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560195a02510 con 0x5601959f09f0 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdd6e54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560195a032b0 con 0x5601959e4a60 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdd6e54740 10 client.0 _ll_register_callbacks cb 0x7fbdc8005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdce59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4140 0x560195a05480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdce59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4140 0x560195a05480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37882/0 (socket says 172.21.15.78:37882) 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdce59c640 1 -- 172.21.15.78:0/469261701 learned_addr learned my addr 172.21.15.78:0/469261701 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4a60 0x560195a059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdced9d640 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959f09f0 0x560195a0abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdce59c640 1 -- 172.21.15.78:0/469261701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4a60 msgr2=0x560195a059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:14.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdce59c640 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4a60 0x560195a059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdce59c640 1 -- 172.21.15.78:0/469261701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959f09f0 msgr2=0x560195a0abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.709+0000 7fbdce59c640 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959f09f0 0x560195a0abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdce59c640 1 -- 172.21.15.78:0/469261701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601959e8780 con 0x5601959e4140 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdced9d640 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959f09f0 0x560195a0abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4a60 0x560195a059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdce59c640 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4140 0x560195a05480 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11f04987254d4fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdb5ffb640 1 -- 172.21.15.78:0/469261701 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0005270 con 0x5601959e4140 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdb5ffb640 1 -- 172.21.15.78:0/469261701 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbdc0005410 con 0x5601959e4140 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdd6e54740 1 -- 172.21.15.78:0/469261701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc40b1080 con 0x5601959e4140 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdb5ffb640 1 -- 172.21.15.78:0/469261701 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbdc0009830 con 0x5601959e4140 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdd6e54740 10 client.15198 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:14.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdd6e54740 1 -- 172.21.15.78:0/469261701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560195a6e7e0 con 0x5601959e4140 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdb5ffb640 1 -- 172.21.15.78:0/469261701 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc40b1080 con 0x5601959e4140 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdb5ffb640 1 -- 172.21.15.78:0/469261701 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbdc0036020 con 0x5601959e4140 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdd6e54740 10 client.15198 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdd6e54740 10 client.15198 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdd6e54740 20 client.15198 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x560195a018f0 0x560195a71ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdd6e54740 4 client.15198 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.710+0000 7fbdd6e54740 1 -- 172.21.15.78:0/469261701 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "status"}) -- 0x560195955610 con 0x560195a018f0 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.711+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x560195a018f0 0x560195a71ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.711+0000 7fbdcdd9b640 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x560195a018f0 0x560195a71ef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a6ea72733858a6a5 server_cookie=2c21e5ab9a53d6a4 in_seq=0 out_seq=0 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.711+0000 7fbdb5ffb640 10 client.15198 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.711+0000 7fbdb5ffb640 1 -- 172.21.15.78:0/469261701 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560195955610 con 0x560195a018f0 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdb5ffb640 10 client.15198 handle_command_reply: tid=0 2024-09-11T00:58:14.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd6e54740 1 client.15198 shutdown 2024-09-11T00:58:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd6e54740 1 -- 172.21.15.78:0/469261701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4140 msgr2=0x560195a05480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4140 0x560195a05480 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc0005850 comp rx=0 tx=0).stop 2024-09-11T00:58:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd6e54740 1 -- 172.21.15.78:0/469261701 shutdown_connections 2024-09-11T00:58:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x560195a018f0 0x560195a71ef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdb5ffb640 0 client.15198 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T00:58:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601959f09f0 0x560195a0abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601959e4a60 0x560195a059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd6e54740 1 --2- 172.21.15.78:0/469261701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601959e4140 0x560195a05480 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd6e54740 1 -- 172.21.15.78:0/469261701 >> 172.21.15.78:0/469261701 conn(0x5601959e1f70 msgr2=0x5601959e8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd6e54740 1 -- 172.21.15.78:0/469261701 shutdown_connections 2024-09-11T00:58:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd6e54740 1 -- 172.21.15.78:0/469261701 wait complete. 2024-09-11T00:58:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd6e54740 20 client.15198 trim_cache size 0 max 16384 2024-09-11T00:58:14.723 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:58:14.723 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:58:14.723 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 3, 2024-09-11T00:58:14.723 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24563, 2024-09-11T00:58:14.723 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T00:58:14.723 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T00:58:14.723 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T00:58:14.723 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 71.530172081000003, 2024-09-11T00:58:14.723 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-11T00:58:14.723 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-09-11T00:58:14.723 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-09-11T00:58:14.723 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 105.223888104 2024-09-11T00:58:14.724 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:58:14.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/283198206 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbdb80b53c0 msgr2=0x7fbdb80b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:14.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd4a7f640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbdb80b53c0 0x7fbdb80b7880 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbdc802f500 tx=0x7fbdc802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/283198206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdd01087d0 msgr2=0x7fbdd0199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:14.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd4a7f640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdd01087d0 0x7fbdd0199700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400c920 tx=0x7fbdc400cde0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/283198206 shutdown_connections 2024-09-11T00:58:14.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd4a7f640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbdb80b53c0 0x7fbdb80b7880 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd4a7f640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdd0109150 0x7fbdd019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd4a7f640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdd01087d0 0x7fbdd0199700 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd4a7f640 1 --2- 172.21.15.78:0/283198206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd0102dc0 0x7fbdd01991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:14.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.712+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/283198206 >> 172.21.15.78:0/283198206 conn(0x7fbdd00fe830 msgr2=0x7fbdd0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:14.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.713+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/283198206 shutdown_connections 2024-09-11T00:58:14.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:14.713+0000 7fbdd4a7f640 1 -- 172.21.15.78:0/283198206 wait complete. 2024-09-11T00:58:15.406 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:15 smithi078 ceph-mon[31315]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:58:15.434 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T00:58:15.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:15 smithi110 ceph-mon[34712]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:58:15.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:15 smithi112 ceph-mon[35384]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:58:15.680 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.190+0000 7fc241442640 1 -- 172.21.15.78:0/1614800386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc23c102dc0 msgr2=0x7fc23c1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.190+0000 7fc241442640 1 --2- 172.21.15.78:0/1614800386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc23c102dc0 0x7fc23c1031a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009900 tx=0x7fc22c02a410 comp rx=0 tx=0).stop 2024-09-11T00:58:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.190+0000 7fc241442640 1 -- 172.21.15.78:0/1614800386 shutdown_connections 2024-09-11T00:58:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.190+0000 7fc241442640 1 --2- 172.21.15.78:0/1614800386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc23c109150 0x7fc23c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.190+0000 7fc241442640 1 --2- 172.21.15.78:0/1614800386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23c1087d0 0x7fc23c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.190+0000 7fc241442640 1 --2- 172.21.15.78:0/1614800386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc23c102dc0 0x7fc23c1031a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.190+0000 7fc241442640 1 -- 172.21.15.78:0/1614800386 >> 172.21.15.78:0/1614800386 conn(0x7fc23c0fe830 msgr2=0x7fc23c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.190+0000 7fc241442640 1 -- 172.21.15.78:0/1614800386 shutdown_connections 2024-09-11T00:58:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.190+0000 7fc241442640 1 -- 172.21.15.78:0/1614800386 wait complete. 2024-09-11T00:58:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.190+0000 7fc241442640 1 Processor -- start 2024-09-11T00:58:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.190+0000 7fc241442640 1 -- start start 2024-09-11T00:58:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc241442640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc23c102dc0 0x7fc23c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc241442640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23c1087d0 0x7fc23c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc241442640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc23c109150 0x7fc23c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc241442640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c112e50 con 0x7fc23c102dc0 2024-09-11T00:58:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc241442640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c112cd0 con 0x7fc23c109150 2024-09-11T00:58:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc241442640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c112fd0 con 0x7fc23c1087d0 2024-09-11T00:58:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc23affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc23c102dc0 0x7fc23c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc23affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc23c102dc0 0x7fc23c0754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37898/0 (socket says 172.21.15.78:37898) 2024-09-11T00:58:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc23affd640 1 -- 172.21.15.78:0/3505843247 learned_addr learned my addr 172.21.15.78:0/3505843247 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc2327fc640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23c1087d0 0x7fc23c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc23c109150 0x7fc23c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc23affd640 1 -- 172.21.15.78:0/3505843247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23c1087d0 msgr2=0x7fc23c0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc23affd640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23c1087d0 0x7fc23c0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc23affd640 1 -- 172.21.15.78:0/3505843247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc23c109150 msgr2=0x7fc23c075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc23affd640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc23c109150 0x7fc23c075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc23affd640 1 -- 172.21.15.78:0/3505843247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c1a7840 con 0x7fc23c102dc0 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc23c109150 0x7fc23c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc2327fc640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23c1087d0 0x7fc23c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc23affd640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc23c102dc0 0x7fc23c0754a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc22c006010 tx=0x7fc22c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc238ff9640 1 -- 172.21.15.78:0/3505843247 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c03f030 con 0x7fc23c102dc0 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc238ff9640 1 -- 172.21.15.78:0/3505843247 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc22c003800 con 0x7fc23c102dc0 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc241442640 1 -- 172.21.15.78:0/3505843247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c1a7ad0 con 0x7fc23c102dc0 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc238ff9640 1 -- 172.21.15.78:0/3505843247 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c035070 con 0x7fc23c102dc0 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.191+0000 7fc241442640 1 -- 172.21.15.78:0/3505843247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c1a7fb0 con 0x7fc23c102dc0 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.194+0000 7fc238ff9640 1 -- 172.21.15.78:0/3505843247 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fc22c04c060 con 0x7fc23c102dc0 2024-09-11T00:58:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.194+0000 7fc243817740 1 Processor -- start 2024-09-11T00:58:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.194+0000 7fc243817740 1 -- start start 2024-09-11T00:58:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.194+0000 7fc243817740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1ed30 0x564f60a1f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.194+0000 7fc243817740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1f650 0x564f60a2b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.194+0000 7fc243817740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a2b5e0 0x564f60a2d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.194+0000 7fc243817740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f60996c10 con 0x564f60a1ed30 2024-09-11T00:58:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.194+0000 7fc243817740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564f60986040 con 0x564f60a1f650 2024-09-11T00:58:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.194+0000 7fc243817740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f60989e50 con 0x564f60a2b5e0 2024-09-11T00:58:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.194+0000 7fc23b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1ed30 0x564f60a1f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.194+0000 7fc23b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1ed30 0x564f60a1f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37914/0 (socket says 172.21.15.78:37914) 2024-09-11T00:58:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.194+0000 7fc23b7fe640 1 -- 172.21.15.78:0/1611465954 learned_addr learned my addr 172.21.15.78:0/1611465954 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc2327fc640 1 --2- 172.21.15.78:0/1611465954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a2b5e0 0x564f60a2d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc23affd640 1 --2- 172.21.15.78:0/1611465954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1f650 0x564f60a2b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc23b7fe640 1 -- 172.21.15.78:0/1611465954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a2b5e0 msgr2=0x564f60a2d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/1611465954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a2b5e0 0x564f60a2d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc23b7fe640 1 -- 172.21.15.78:0/1611465954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1f650 msgr2=0x564f60a2b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/1611465954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1f650 0x564f60a2b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc23b7fe640 1 -- 172.21.15.78:0/1611465954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f606e4540 con 0x564f60a1ed30 2024-09-11T00:58:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc2327fc640 1 --2- 172.21.15.78:0/1611465954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a2b5e0 0x564f60a2d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc238ff9640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc21c0b53c0 0x7fc21c0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc23affd640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc21c0b53c0 0x7fc21c0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/1611465954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1ed30 0x564f60a1f110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc234007f30 tx=0x7fc23400c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587556f59d6c24cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc238ff9640 1 -- 172.21.15.78:0/3505843247 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc22c106250 con 0x7fc23c102dc0 2024-09-11T00:58:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc2317fa640 1 -- 172.21.15.78:0/1611465954 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234014070 con 0x564f60a1ed30 2024-09-11T00:58:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc2317fa640 1 -- 172.21.15.78:0/1611465954 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23400fd00 con 0x564f60a1ed30 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.195+0000 7fc2317fa640 1 -- 172.21.15.78:0/1611465954 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23400ec40 con 0x564f60a1ed30 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.196+0000 7fc243817740 1 -- 172.21.15.78:0/1611465954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1ed30 msgr2=0x564f60a1f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.196+0000 7fc243817740 1 --2- 172.21.15.78:0/1611465954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1ed30 0x564f60a1f110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc234007f30 tx=0x7fc23400c750 comp rx=0 tx=0).stop 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.196+0000 7fc23affd640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc21c0b53c0 0x7fc21c0b7880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x564f60a21eb0 tx=0x7fc22c02a410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.196+0000 7fc243817740 1 -- 172.21.15.78:0/1611465954 shutdown_connections 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.196+0000 7fc243817740 1 --2- 172.21.15.78:0/1611465954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a2b5e0 0x564f60a2d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.196+0000 7fc243817740 1 --2- 172.21.15.78:0/1611465954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1f650 0x564f60a2b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.196+0000 7fc243817740 1 --2- 172.21.15.78:0/1611465954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1ed30 0x564f60a1f110 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.196+0000 7fc243817740 1 -- 172.21.15.78:0/1611465954 >> 172.21.15.78:0/1611465954 conn(0x564f60a1cb60 msgr2=0x564f60a1dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.196+0000 7fc243817740 1 -- 172.21.15.78:0/1611465954 shutdown_connections 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.196+0000 7fc243817740 1 -- 172.21.15.78:0/1611465954 wait complete. 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.196+0000 7fc243817740 1 -- start start 2024-09-11T00:58:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc243817740 1 Processor -- start 2024-09-11T00:58:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc243817740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1ed30 0x564f60a28590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc243817740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1f650 0x564f60a28ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc243817740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a2b5e0 0x564f60a29010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc243817740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f607e75f0 con 0x564f60a1f650 2024-09-11T00:58:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc243817740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564f60921010 con 0x564f60a2b5e0 2024-09-11T00:58:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc243817740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f60a2e290 con 0x564f60a1ed30 2024-09-11T00:58:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc2327fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1f650 0x564f60a28ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc243817740 10 client.0 _ll_register_callbacks cb 0x564f60980340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc2327fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1f650 0x564f60a28ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37922/0 (socket says 172.21.15.78:37922) 2024-09-11T00:58:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc2327fc640 1 -- 172.21.15.78:0/925160213 learned_addr learned my addr 172.21.15.78:0/925160213 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1ed30 0x564f60a28590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc2327fc640 1 -- 172.21.15.78:0/925160213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1ed30 msgr2=0x564f60a28590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc2327fc640 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1ed30 0x564f60a28590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc2327fc640 1 -- 172.21.15.78:0/925160213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a2b5e0 msgr2=0x564f60a29010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a2b5e0 0x564f60a29010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc2327fc640 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a2b5e0 0x564f60a29010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc2327fc640 1 -- 172.21.15.78:0/925160213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f6094ea00 con 0x564f60a1f650 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a2b5e0 0x564f60a29010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc2327fc640 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1f650 0x564f60a28ad0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc224007fd0 tx=0x7fc22400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6790a9cb76b5e9cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.197+0000 7fc22a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc22a7fc640 1 -- 172.21.15.78:0/925160213 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224018020 con 0x564f60a1f650 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc22a7fc640 1 -- 172.21.15.78:0/925160213 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc224014870 con 0x564f60a1f650 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc243817740 1 -- 172.21.15.78:0/925160213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc23400fd00 con 0x564f60a1f650 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc22a7fc640 1 -- 172.21.15.78:0/925160213 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc224023050 con 0x564f60a1f650 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc243817740 10 client.15210 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc243817740 1 -- 172.21.15.78:0/925160213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f60a39e90 con 0x564f60a1f650 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc22a7fc640 1 -- 172.21.15.78:0/925160213 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23400fd00 con 0x564f60a1f650 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc22a7fc640 1 -- 172.21.15.78:0/925160213 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc224013a60 con 0x564f60a1f650 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc243817740 10 client.15210 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc243817740 10 client.15210 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc243817740 20 client.15210 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc243817740 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564f60a3c530 0x564f60a3d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc243817740 4 client.15210 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.198+0000 7fc243817740 1 -- 172.21.15.78:0/925160213 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f60998b60 con 0x564f60a3c530 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.199+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564f60a3c530 0x564f60a3d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.199+0000 7fc22a7fc640 10 client.15210 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.199+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564f60a3c530 0x564f60a3d9a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4fa87239392587c8 server_cookie=d93268d39439b55 in_seq=0 out_seq=0 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc22a7fc640 1 -- 172.21.15.78:0/925160213 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564f60998b60 con 0x564f60a3c530 2024-09-11T00:58:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc22a7fc640 10 client.15210 handle_command_reply: tid=0 2024-09-11T00:58:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc243817740 1 client.15210 shutdown 2024-09-11T00:58:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc243817740 1 -- 172.21.15.78:0/925160213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1f650 msgr2=0x564f60a28ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc243817740 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1f650 0x564f60a28ad0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc224007fd0 tx=0x7fc22400f670 comp rx=0 tx=0).stop 2024-09-11T00:58:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc243817740 1 -- 172.21.15.78:0/925160213 shutdown_connections 2024-09-11T00:58:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc243817740 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564f60a3c530 0x564f60a3d9a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc243817740 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a2b5e0 0x564f60a29010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc243817740 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a1f650 0x564f60a28ad0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc22a7fc640 0 client.15210 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T00:58:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc243817740 1 --2- 172.21.15.78:0/925160213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1ed30 0x564f60a28590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc243817740 1 -- 172.21.15.78:0/925160213 >> 172.21.15.78:0/925160213 conn(0x564f60a1cb60 msgr2=0x564f60a22700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc243817740 1 -- 172.21.15.78:0/925160213 shutdown_connections 2024-09-11T00:58:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc243817740 1 -- 172.21.15.78:0/925160213 wait complete. 2024-09-11T00:58:16.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.200+0000 7fc243817740 20 client.15210 trim_cache size 0 max 16384 2024-09-11T00:58:16.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.219+0000 7fc243817740 1 Processor -- start 2024-09-11T00:58:16.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.219+0000 7fc243817740 1 -- start start 2024-09-11T00:58:16.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.219+0000 7fc243817740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1ed30 0x564f60a22ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.219+0000 7fc243817740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1f650 0x564f60a237e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.219+0000 7fc243817740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 0x564f60a23d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.219+0000 7fc243817740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f60a3bce0 con 0x564f60a2b5e0 2024-09-11T00:58:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.219+0000 7fc243817740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564f60a3c3b0 con 0x564f60a1f650 2024-09-11T00:58:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.219+0000 7fc243817740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f60a3ca80 con 0x564f60a1ed30 2024-09-11T00:58:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc23b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 0x564f60a23d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc23b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 0x564f60a23d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37932/0 (socket says 172.21.15.78:37932) 2024-09-11T00:58:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc23b7fe640 1 -- 172.21.15.78:0/531321947 learned_addr learned my addr 172.21.15.78:0/531321947 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/531321947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1ed30 0x564f60a22ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc2327fc640 1 --2- 172.21.15.78:0/531321947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1f650 0x564f60a237e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc23b7fe640 1 -- 172.21.15.78:0/531321947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1ed30 msgr2=0x564f60a22ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/531321947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1ed30 0x564f60a22ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc23b7fe640 1 -- 172.21.15.78:0/531321947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1f650 msgr2=0x564f60a237e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/531321947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1f650 0x564f60a237e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc23b7fe640 1 -- 172.21.15.78:0/531321947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f60a1d5e0 con 0x564f60a2b5e0 2024-09-11T00:58:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/531321947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 0x564f60a23d20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc2340073b0 tx=0x7fc23401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2c3e5c08b6ef65 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc230ff9640 1 -- 172.21.15.78:0/531321947 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234014040 con 0x564f60a2b5e0 2024-09-11T00:58:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc230ff9640 1 -- 172.21.15.78:0/531321947 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23400e3a0 con 0x564f60a2b5e0 2024-09-11T00:58:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.220+0000 7fc230ff9640 1 -- 172.21.15.78:0/531321947 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234019450 con 0x564f60a2b5e0 2024-09-11T00:58:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.221+0000 7fc243817740 1 -- 172.21.15.78:0/531321947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 msgr2=0x564f60a23d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.221+0000 7fc243817740 1 --2- 172.21.15.78:0/531321947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 0x564f60a23d20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc2340073b0 tx=0x7fc23401b040 comp rx=0 tx=0).stop 2024-09-11T00:58:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.221+0000 7fc243817740 1 -- 172.21.15.78:0/531321947 shutdown_connections 2024-09-11T00:58:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.221+0000 7fc243817740 1 --2- 172.21.15.78:0/531321947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 0x564f60a23d20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.221+0000 7fc243817740 1 --2- 172.21.15.78:0/531321947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1f650 0x564f60a237e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.221+0000 7fc243817740 1 --2- 172.21.15.78:0/531321947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1ed30 0x564f60a22ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.221+0000 7fc243817740 1 -- 172.21.15.78:0/531321947 >> 172.21.15.78:0/531321947 conn(0x564f60a1cb60 msgr2=0x564f60a1dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:16.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.221+0000 7fc243817740 1 -- 172.21.15.78:0/531321947 shutdown_connections 2024-09-11T00:58:16.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.221+0000 7fc243817740 1 -- 172.21.15.78:0/531321947 wait complete. 2024-09-11T00:58:16.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc243817740 1 -- start start 2024-09-11T00:58:16.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc243817740 1 Processor -- start 2024-09-11T00:58:16.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc243817740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1ed30 0x564f60a46080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc243817740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1f650 0x564f60a46600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc243817740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 0x564f60a40740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc243817740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f60a3d820 con 0x564f60a2b5e0 2024-09-11T00:58:16.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc243817740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564f60a3d150 con 0x564f60a1ed30 2024-09-11T00:58:16.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc243817740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f60a3def0 con 0x564f60a1f650 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc243817740 10 client.0 _ll_register_callbacks cb 0x7fc2240072c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc23b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 0x564f60a40740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc23b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 0x564f60a40740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37948/0 (socket says 172.21.15.78:37948) 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc23b7fe640 1 -- 172.21.15.78:0/850777510 learned_addr learned my addr 172.21.15.78:0/850777510 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1ed30 0x564f60a46080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc2327fc640 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1f650 0x564f60a46600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc23b7fe640 1 -- 172.21.15.78:0/850777510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1f650 msgr2=0x564f60a46600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1f650 0x564f60a46600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc23b7fe640 1 -- 172.21.15.78:0/850777510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1ed30 msgr2=0x564f60a46080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1ed30 0x564f60a46080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.222+0000 7fc23b7fe640 1 -- 172.21.15.78:0/850777510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f60a40f70 con 0x564f60a2b5e0 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.223+0000 7fc23b7fe640 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 0x564f60a40740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc234004a80 tx=0x7fc23400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f624709bcdd622 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:16.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.223+0000 7fc22a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.223+0000 7fc22a7fc640 1 -- 172.21.15.78:0/850777510 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234033040 con 0x564f60a2b5e0 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.223+0000 7fc243817740 1 -- 172.21.15.78:0/850777510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc23400e3a0 con 0x564f60a2b5e0 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.223+0000 7fc22a7fc640 1 -- 172.21.15.78:0/850777510 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc23400f650 con 0x564f60a2b5e0 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.223+0000 7fc22a7fc640 1 -- 172.21.15.78:0/850777510 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc234014040 con 0x564f60a2b5e0 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.223+0000 7fc243817740 10 client.15216 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.223+0000 7fc243817740 1 -- 172.21.15.78:0/850777510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f60aa9b40 con 0x564f60a2b5e0 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.224+0000 7fc22a7fc640 1 -- 172.21.15.78:0/850777510 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23400f850 con 0x564f60a2b5e0 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.225+0000 7fc22a7fc640 1 -- 172.21.15.78:0/850777510 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc234044030 con 0x564f60a2b5e0 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.226+0000 7fc243817740 10 client.15216 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.226+0000 7fc243817740 10 client.15216 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.226+0000 7fc243817740 20 client.15216 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.226+0000 7fc243817740 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564f60a3c530 0x564f60aacb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.226+0000 7fc243817740 4 client.15216 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:58:16.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.226+0000 7fc243817740 1 -- 172.21.15.78:0/850777510 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "status"}) -- 0x7fc23400e3a0 con 0x564f60a3c530 2024-09-11T00:58:16.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.234+0000 7fc2327fc640 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564f60a3c530 0x564f60aacb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:16.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.246+0000 7fc2327fc640 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564f60a3c530 0x564f60aacb50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f31f45d9c7cc8a89 server_cookie=c90b178b4a444de3 in_seq=0 out_seq=0 2024-09-11T00:58:16.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.246+0000 7fc22a7fc640 10 client.15216 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T00:58:16.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.246+0000 7fc22a7fc640 1 -- 172.21.15.78:0/850777510 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x7fc23400e3a0 con 0x564f60a3c530 2024-09-11T00:58:16.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.246+0000 7fc22a7fc640 10 client.15216 handle_command_reply: tid=0 2024-09-11T00:58:16.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.247+0000 7fc243817740 1 client.15216 shutdown 2024-09-11T00:58:16.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.247+0000 7fc243817740 1 -- 172.21.15.78:0/850777510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 msgr2=0x564f60a40740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:16.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.247+0000 7fc243817740 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 0x564f60a40740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc234004a80 tx=0x7fc23400cf20 comp rx=0 tx=0).stop 2024-09-11T00:58:16.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.247+0000 7fc243817740 1 -- 172.21.15.78:0/850777510 shutdown_connections 2024-09-11T00:58:16.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.247+0000 7fc243817740 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f60a2b5e0 0x564f60a40740 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.248+0000 7fc243817740 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f60a1f650 0x564f60a46600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.248+0000 7fc243817740 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564f60a3c530 0x564f60aacb50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.248+0000 7fc243817740 1 --2- 172.21.15.78:0/850777510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f60a1ed30 0x564f60a46080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.248+0000 7fc22a7fc640 0 client.15216 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T00:58:16.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.248+0000 7fc243817740 1 -- 172.21.15.78:0/850777510 >> 172.21.15.78:0/850777510 conn(0x564f60a1cb60 msgr2=0x564f60a237e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:16.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.248+0000 7fc243817740 1 -- 172.21.15.78:0/850777510 shutdown_connections 2024-09-11T00:58:16.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.248+0000 7fc243817740 1 -- 172.21.15.78:0/850777510 wait complete. 2024-09-11T00:58:16.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.248+0000 7fc243817740 20 client.15216 trim_cache size 0 max 16384 2024-09-11T00:58:16.251 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:58:16.251 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:58:16.251 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-09-11T00:58:16.251 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24565, 2024-09-11T00:58:16.251 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T00:58:16.251 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T00:58:16.251 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T00:58:16.251 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 74.075655212000001, 2024-09-11T00:58:16.251 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-11T00:58:16.251 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-09-11T00:58:16.252 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-09-11T00:58:16.252 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 109.446562187 2024-09-11T00:58:16.252 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:58:16.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.253+0000 7fc241442640 1 -- 172.21.15.78:0/3505843247 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc21c0b53c0 msgr2=0x7fc21c0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:16.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.253+0000 7fc241442640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc21c0b53c0 0x7fc21c0b7880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x564f60a21eb0 tx=0x7fc22c02a410 comp rx=0 tx=0).stop 2024-09-11T00:58:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.253+0000 7fc241442640 1 -- 172.21.15.78:0/3505843247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc23c102dc0 msgr2=0x7fc23c0754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.253+0000 7fc241442640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc23c102dc0 0x7fc23c0754a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc22c006010 tx=0x7fc22c0042f0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.253+0000 7fc241442640 1 -- 172.21.15.78:0/3505843247 shutdown_connections 2024-09-11T00:58:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.253+0000 7fc241442640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc21c0b53c0 0x7fc21c0b7880 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.253+0000 7fc241442640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc23c109150 0x7fc23c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.253+0000 7fc241442640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23c1087d0 0x7fc23c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.253+0000 7fc241442640 1 --2- 172.21.15.78:0/3505843247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc23c102dc0 0x7fc23c0754a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.253+0000 7fc241442640 1 -- 172.21.15.78:0/3505843247 >> 172.21.15.78:0/3505843247 conn(0x7fc23c0fe830 msgr2=0x7fc23c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.254+0000 7fc241442640 1 -- 172.21.15.78:0/3505843247 shutdown_connections 2024-09-11T00:58:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:16.254+0000 7fc241442640 1 -- 172.21.15.78:0/3505843247 wait complete. 2024-09-11T00:58:16.976 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T00:58:17.222 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:17.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:17 smithi078 ceph-mon[31315]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:58:17.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:17 smithi110 ceph-mon[34712]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:58:17.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:17 smithi112 ceph-mon[35384]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:58:17.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.690+0000 7efd631c0640 1 -- 172.21.15.78:0/831263922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5c101810 msgr2=0x7efd5c101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:17.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.690+0000 7efd631c0640 1 --2- 172.21.15.78:0/831263922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5c101810 0x7efd5c101bf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efd50009900 tx=0x7efd5002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.690+0000 7efd631c0640 1 -- 172.21.15.78:0/831263922 shutdown_connections 2024-09-11T00:58:17.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.690+0000 7efd631c0640 1 --2- 172.21.15.78:0/831263922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c10d250 0x7efd5c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.690+0000 7efd631c0640 1 --2- 172.21.15.78:0/831263922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd5c10a920 0x7efd5c10cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.690+0000 7efd631c0640 1 --2- 172.21.15.78:0/831263922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5c101810 0x7efd5c101bf0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.690+0000 7efd631c0640 1 -- 172.21.15.78:0/831263922 >> 172.21.15.78:0/831263922 conn(0x7efd5c076f50 msgr2=0x7efd5c077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:17.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.690+0000 7efd631c0640 1 -- 172.21.15.78:0/831263922 shutdown_connections 2024-09-11T00:58:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.690+0000 7efd631c0640 1 -- 172.21.15.78:0/831263922 wait complete. 2024-09-11T00:58:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.691+0000 7efd631c0640 1 Processor -- start 2024-09-11T00:58:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.691+0000 7efd631c0640 1 -- start start 2024-09-11T00:58:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.691+0000 7efd631c0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd5c101810 0x7efd5c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.691+0000 7efd631c0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5c10a920 0x7efd5c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.691+0000 7efd631c0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c10d250 0x7efd5c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.691+0000 7efd631c0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c112d90 con 0x7efd5c10a920 2024-09-11T00:58:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.691+0000 7efd631c0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c112c10 con 0x7efd5c101810 2024-09-11T00:58:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.691+0000 7efd631c0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c112f10 con 0x7efd5c10d250 2024-09-11T00:58:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.691+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5c10a920 0x7efd5c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.691+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5c10a920 0x7efd5c1997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37960/0 (socket says 172.21.15.78:37960) 2024-09-11T00:58:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.691+0000 7efd5bfff640 1 -- 172.21.15.78:0/2363748836 learned_addr learned my addr 172.21.15.78:0/2363748836 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd60f5c640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd5c101810 0x7efd5c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd6175d640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c10d250 0x7efd5c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd5bfff640 1 -- 172.21.15.78:0/2363748836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c10d250 msgr2=0x7efd5c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd5bfff640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c10d250 0x7efd5c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd5bfff640 1 -- 172.21.15.78:0/2363748836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd5c101810 msgr2=0x7efd5c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd5bfff640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd5c101810 0x7efd5c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd5bfff640 1 -- 172.21.15.78:0/2363748836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c19f330 con 0x7efd5c10a920 2024-09-11T00:58:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd60f5c640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd5c101810 0x7efd5c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd6175d640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c10d250 0x7efd5c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd5bfff640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5c10a920 0x7efd5c1997c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efd4800c920 tx=0x7efd4800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd59ffb640 1 -- 172.21.15.78:0/2363748836 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48007c70 con 0x7efd5c10a920 2024-09-11T00:58:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd59ffb640 1 -- 172.21.15.78:0/2363748836 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd48007e10 con 0x7efd5c10a920 2024-09-11T00:58:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd631c0640 1 -- 172.21.15.78:0/2363748836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5c19f620 con 0x7efd5c10a920 2024-09-11T00:58:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd59ffb640 1 -- 172.21.15.78:0/2363748836 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48018820 con 0x7efd5c10a920 2024-09-11T00:58:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.692+0000 7efd631c0640 1 -- 172.21.15.78:0/2363748836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5c1a5ce0 con 0x7efd5c10a920 2024-09-11T00:58:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.695+0000 7efd59ffb640 1 -- 172.21.15.78:0/2363748836 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7efd4800f450 con 0x7efd5c10a920 2024-09-11T00:58:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.695+0000 7efd65595740 1 Processor -- start 2024-09-11T00:58:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.695+0000 7efd65595740 1 -- start start 2024-09-11T00:58:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd65595740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c9286fd30 0x558c92870110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd65595740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 0x558c928804b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd65595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c928809f0 0x558c92882de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd65595740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c927e7c10 con 0x558c9286fd30 2024-09-11T00:58:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd65595740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558c927d7040 con 0x558c92870650 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd65595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558c927dae50 con 0x558c928809f0 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd6175d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c9286fd30 0x558c92870110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd6175d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c9286fd30 0x558c92870110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37970/0 (socket says 172.21.15.78:37970) 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd6175d640 1 -- 172.21.15.78:0/808115025 learned_addr learned my addr 172.21.15.78:0/808115025 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd5bfff640 1 --2- 172.21.15.78:0/808115025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c928809f0 0x558c92882de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd60f5c640 1 --2- 172.21.15.78:0/808115025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 0x558c928804b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd6175d640 1 -- 172.21.15.78:0/808115025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c928809f0 msgr2=0x558c92882de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd6175d640 1 --2- 172.21.15.78:0/808115025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c928809f0 0x558c92882de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd6175d640 1 -- 172.21.15.78:0/808115025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 msgr2=0x558c928804b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd59ffb640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efd4c0b53c0 0x7efd4c0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd6175d640 1 --2- 172.21.15.78:0/808115025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 0x558c928804b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd6175d640 1 -- 172.21.15.78:0/808115025 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c92535540 con 0x558c9286fd30 2024-09-11T00:58:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd60f5c640 1 --2- 172.21.15.78:0/808115025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 0x558c928804b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd59ffb640 1 -- 172.21.15.78:0/2363748836 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efd480e60a0 con 0x7efd5c10a920 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd6175d640 1 --2- 172.21.15.78:0/808115025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c9286fd30 0x558c92870110 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efd54008de0 tx=0x7efd5400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438ef0db2c92fc33 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.696+0000 7efd6175d640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efd4c0b53c0 0x7efd4c0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd46ffd640 1 -- 172.21.15.78:0/808115025 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5401f070 con 0x558c9286fd30 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd46ffd640 1 -- 172.21.15.78:0/808115025 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5400e3b0 con 0x558c9286fd30 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd46ffd640 1 -- 172.21.15.78:0/808115025 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5400b5e0 con 0x558c9286fd30 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd65595740 1 -- 172.21.15.78:0/808115025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c9286fd30 msgr2=0x558c92870110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd6175d640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efd4c0b53c0 0x7efd4c0b7880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efd5401c000 tx=0x7efd5401f000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd65595740 1 --2- 172.21.15.78:0/808115025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c9286fd30 0x558c92870110 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efd54008de0 tx=0x7efd5400d710 comp rx=0 tx=0).stop 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd65595740 1 -- 172.21.15.78:0/808115025 shutdown_connections 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd65595740 1 --2- 172.21.15.78:0/808115025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c928809f0 0x558c92882de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd65595740 1 --2- 172.21.15.78:0/808115025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 0x558c928804b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd65595740 1 --2- 172.21.15.78:0/808115025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c9286fd30 0x558c92870110 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd65595740 1 -- 172.21.15.78:0/808115025 >> 172.21.15.78:0/808115025 conn(0x558c9286db60 msgr2=0x558c9286ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd65595740 1 -- 172.21.15.78:0/808115025 shutdown_connections 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.697+0000 7efd65595740 1 -- 172.21.15.78:0/808115025 wait complete. 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd65595740 1 -- start start 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd65595740 1 Processor -- start 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd65595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c9286fd30 0x558c9287d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd65595740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c92870650 0x558c9287dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd65595740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c928809f0 0x558c9287e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd65595740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c926385f0 con 0x558c92870650 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd65595740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558c92772010 con 0x558c928809f0 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd65595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558c928836a0 con 0x558c9286fd30 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd65595740 10 client.0 _ll_register_callbacks cb 0x558c927d1340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c92870650 0x558c9287dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c92870650 0x558c9287dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37980/0 (socket says 172.21.15.78:37980) 2024-09-11T00:58:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd5bfff640 1 -- 172.21.15.78:0/696990925 learned_addr learned my addr 172.21.15.78:0/696990925 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.698+0000 7efd60f5c640 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c9286fd30 0x558c9287d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd5bfff640 1 -- 172.21.15.78:0/696990925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c9286fd30 msgr2=0x558c9287d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd5bfff640 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c9286fd30 0x558c9287d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd5bfff640 1 -- 172.21.15.78:0/696990925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c928809f0 msgr2=0x558c9287e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd5bfff640 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c928809f0 0x558c9287e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd5bfff640 1 -- 172.21.15.78:0/696990925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c9279fa00 con 0x558c92870650 2024-09-11T00:58:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd60f5c640 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c9286fd30 0x558c9287d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd5bfff640 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c92870650 0x558c9287dea0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efd480a8c20 tx=0x7efd4800fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186315e9c0de408 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd3bfff640 1 -- 172.21.15.78:0/696990925 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd480acab0 con 0x558c92870650 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd65595740 1 -- 172.21.15.78:0/696990925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5400e3b0 con 0x558c92870650 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd3bfff640 1 -- 172.21.15.78:0/696990925 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efd480b5750 con 0x558c92870650 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd3bfff640 1 -- 172.21.15.78:0/696990925 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd480e9900 con 0x558c92870650 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd65595740 10 client.15228 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd65595740 1 -- 172.21.15.78:0/696990925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c9288f250 con 0x558c92870650 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd3bfff640 1 -- 172.21.15.78:0/696990925 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5400e3b0 con 0x558c92870650 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd3bfff640 1 -- 172.21.15.78:0/696990925 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efd480e80f0 con 0x558c92870650 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd65595740 10 client.15228 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd65595740 10 client.15228 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd65595740 20 client.15228 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd65595740 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x558c928917e0 0x558c92892c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd65595740 4 client.15228 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.699+0000 7efd65595740 1 -- 172.21.15.78:0/696990925 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c927e9b60 con 0x558c928917e0 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.700+0000 7efd60f5c640 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x558c928917e0 0x558c92892c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.700+0000 7efd3bfff640 10 client.15228 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.700+0000 7efd60f5c640 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x558c928917e0 0x558c92892c50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=367c42c056148855 server_cookie=484db69caaea2f80 in_seq=0 out_seq=0 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd3bfff640 1 -- 172.21.15.78:0/696990925 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c927e9b60 con 0x558c928917e0 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd3bfff640 10 client.15228 handle_command_reply: tid=0 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd65595740 1 client.15228 shutdown 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd65595740 1 -- 172.21.15.78:0/696990925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c92870650 msgr2=0x558c9287dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd65595740 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c92870650 0x558c9287dea0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efd480a8c20 tx=0x7efd4800fcc0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd65595740 1 -- 172.21.15.78:0/696990925 shutdown_connections 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd65595740 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c928809f0 0x558c9287e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd65595740 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x558c928917e0 0x558c92892c50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd65595740 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c92870650 0x558c9287dea0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd3bfff640 0 client.15228 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd65595740 1 --2- 172.21.15.78:0/696990925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c9286fd30 0x558c9287d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd65595740 1 -- 172.21.15.78:0/696990925 >> 172.21.15.78:0/696990925 conn(0x558c9286db60 msgr2=0x558c92873690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd65595740 1 -- 172.21.15.78:0/696990925 shutdown_connections 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd65595740 1 -- 172.21.15.78:0/696990925 wait complete. 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.701+0000 7efd65595740 20 client.15228 trim_cache size 0 max 16384 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.719+0000 7efd65595740 1 Processor -- start 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.719+0000 7efd65595740 1 -- start start 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd65595740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c9286fd30 0x558c92892c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd65595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c92870650 0x558c92881a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd65595740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c928809f0 0x558c92881fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd65595740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c92890f90 con 0x558c928809f0 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd65595740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558c92891660 con 0x558c9286fd30 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd65595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558c92891d30 con 0x558c92870650 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd6175d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c928809f0 0x558c92881fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd6175d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c928809f0 0x558c92881fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37996/0 (socket says 172.21.15.78:37996) 2024-09-11T00:58:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd6175d640 1 -- 172.21.15.78:0/614030136 learned_addr learned my addr 172.21.15.78:0/614030136 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd60f5c640 1 --2- 172.21.15.78:0/614030136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c9286fd30 0x558c92892c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd6175d640 1 -- 172.21.15.78:0/614030136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c92870650 msgr2=0x558c92881a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd6175d640 1 --2- 172.21.15.78:0/614030136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c92870650 0x558c92881a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd6175d640 1 -- 172.21.15.78:0/614030136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c9286fd30 msgr2=0x558c92892c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd6175d640 1 --2- 172.21.15.78:0/614030136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c9286fd30 0x558c92892c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd6175d640 1 -- 172.21.15.78:0/614030136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c9287b8c0 con 0x558c928809f0 2024-09-11T00:58:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd60f5c640 1 --2- 172.21.15.78:0/614030136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c9286fd30 0x558c92892c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd6175d640 1 --2- 172.21.15.78:0/614030136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c928809f0 0x558c92881fb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efd5400f630 tx=0x7efd54022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c1417ddc63df74 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd467fc640 1 -- 172.21.15.78:0/614030136 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5400f9d0 con 0x558c928809f0 2024-09-11T00:58:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd467fc640 1 -- 172.21.15.78:0/614030136 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd54024400 con 0x558c928809f0 2024-09-11T00:58:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.720+0000 7efd467fc640 1 -- 172.21.15.78:0/614030136 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5400b600 con 0x558c928809f0 2024-09-11T00:58:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.721+0000 7efd65595740 1 -- 172.21.15.78:0/614030136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c928809f0 msgr2=0x558c92881fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.721+0000 7efd65595740 1 --2- 172.21.15.78:0/614030136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c928809f0 0x558c92881fb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efd5400f630 tx=0x7efd54022000 comp rx=0 tx=0).stop 2024-09-11T00:58:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.721+0000 7efd65595740 1 -- 172.21.15.78:0/614030136 shutdown_connections 2024-09-11T00:58:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.721+0000 7efd65595740 1 --2- 172.21.15.78:0/614030136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c928809f0 0x558c92881fb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.721+0000 7efd65595740 1 --2- 172.21.15.78:0/614030136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c92870650 0x558c92881a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.721+0000 7efd65595740 1 --2- 172.21.15.78:0/614030136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c9286fd30 0x558c92892c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.721+0000 7efd65595740 1 -- 172.21.15.78:0/614030136 >> 172.21.15.78:0/614030136 conn(0x558c9286db60 msgr2=0x558c92882a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.721+0000 7efd65595740 1 -- 172.21.15.78:0/614030136 shutdown_connections 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.721+0000 7efd65595740 1 -- 172.21.15.78:0/614030136 wait complete. 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd65595740 1 -- start start 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd65595740 1 Processor -- start 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd65595740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c9286fd30 0x558c9287eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd65595740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 0x558c9287f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd65595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c928809f0 0x558c9289ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd65595740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c92892ad0 con 0x558c9286fd30 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd65595740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558c92892400 con 0x558c92870650 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd65595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558c928931a0 con 0x558c928809f0 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd65595740 10 client.0 _ll_register_callbacks cb 0x7efd4800cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 0x558c9287f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 0x558c9287f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:42482/0 (socket says 172.21.15.78:42482) 2024-09-11T00:58:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd5bfff640 1 -- 172.21.15.78:0/2515273652 learned_addr learned my addr 172.21.15.78:0/2515273652 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.722+0000 7efd6175d640 1 --2- 172.21.15.78:0/2515273652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c928809f0 0x558c9289ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd5bfff640 1 -- 172.21.15.78:0/2515273652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c928809f0 msgr2=0x558c9289ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd5bfff640 1 --2- 172.21.15.78:0/2515273652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c928809f0 0x558c9289ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd5bfff640 1 -- 172.21.15.78:0/2515273652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c9286fd30 msgr2=0x558c9287eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd5bfff640 1 --2- 172.21.15.78:0/2515273652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c9286fd30 0x558c9287eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd5bfff640 1 -- 172.21.15.78:0/2515273652 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c9289b260 con 0x558c92870650 2024-09-11T00:58:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd5bfff640 1 --2- 172.21.15.78:0/2515273652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 0x558c9287f530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd4801f000 tx=0x7efd4800a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8370e7e5c54fcb35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T00:58:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd3bfff640 1 -- 172.21.15.78:0/2515273652 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd480abfa0 con 0x558c92870650 2024-09-11T00:58:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd65595740 1 -- 172.21.15.78:0/2515273652 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd54024400 con 0x558c92870650 2024-09-11T00:58:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd3bfff640 1 -- 172.21.15.78:0/2515273652 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efd480ebc10 con 0x558c92870650 2024-09-11T00:58:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd3bfff640 1 -- 172.21.15.78:0/2515273652 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd480a8ca0 con 0x558c92870650 2024-09-11T00:58:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd65595740 10 client.24794 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.723+0000 7efd65595740 1 -- 172.21.15.78:0/2515273652 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c928fe770 con 0x558c92870650 2024-09-11T00:58:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.724+0000 7efd3bfff640 1 -- 172.21.15.78:0/2515273652 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd54024400 con 0x558c92870650 2024-09-11T00:58:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.724+0000 7efd3bfff640 1 -- 172.21.15.78:0/2515273652 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efd480e9b20 con 0x558c92870650 2024-09-11T00:58:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.724+0000 7efd65595740 10 client.24794 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.724+0000 7efd65595740 10 client.24794 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:58:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.724+0000 7efd65595740 20 client.24794 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.724+0000 7efd65595740 1 --2- 172.21.15.78:0/2515273652 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x558c92903310 0x558c92902020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.724+0000 7efd65595740 4 client.24794 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:58:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.724+0000 7efd65595740 1 -- 172.21.15.78:0/2515273652 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "status"}) -- 0x558c927e5610 con 0x558c92903310 2024-09-11T00:58:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.724+0000 7efd60f5c640 1 --2- 172.21.15.78:0/2515273652 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x558c92903310 0x558c92902020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd60f5c640 1 --2- 172.21.15.78:0/2515273652 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x558c92903310 0x558c92902020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dac794cd5efe489 server_cookie=3281d40e890b6ecc in_seq=0 out_seq=0 2024-09-11T00:58:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd3bfff640 10 client.24794 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T00:58:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd3bfff640 1 -- 172.21.15.78:0/2515273652 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558c927e5610 con 0x558c92903310 2024-09-11T00:58:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd3bfff640 10 client.24794 handle_command_reply: tid=0 2024-09-11T00:58:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd65595740 1 client.24794 shutdown 2024-09-11T00:58:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd65595740 1 -- 172.21.15.78:0/2515273652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 msgr2=0x558c9287f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd65595740 1 --2- 172.21.15.78:0/2515273652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 0x558c9287f530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd4801f000 tx=0x7efd4800a630 comp rx=0 tx=0).stop 2024-09-11T00:58:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd65595740 1 -- 172.21.15.78:0/2515273652 shutdown_connections 2024-09-11T00:58:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd65595740 1 --2- 172.21.15.78:0/2515273652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c928809f0 0x558c9289ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd65595740 1 --2- 172.21.15.78:0/2515273652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c92870650 0x558c9287f530 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd65595740 1 --2- 172.21.15.78:0/2515273652 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x558c92903310 0x558c92902020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd65595740 1 --2- 172.21.15.78:0/2515273652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c9286fd30 0x558c9287eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd3bfff640 0 client.24794 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T00:58:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd65595740 1 -- 172.21.15.78:0/2515273652 >> 172.21.15.78:0/2515273652 conn(0x558c9286db60 msgr2=0x558c92881a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd65595740 1 -- 172.21.15.78:0/2515273652 shutdown_connections 2024-09-11T00:58:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd65595740 1 -- 172.21.15.78:0/2515273652 wait complete. 2024-09-11T00:58:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.725+0000 7efd65595740 20 client.24794 trim_cache size 0 max 16384 2024-09-11T00:58:17.743 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:58:17.743 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:58:17.743 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-09-11T00:58:17.743 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24580, 2024-09-11T00:58:17.743 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T00:58:17.743 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T00:58:17.744 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T00:58:17.744 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 76.566714447999999, 2024-09-11T00:58:17.744 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-11T00:58:17.744 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-09-11T00:58:17.744 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-09-11T00:58:17.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 105.557800696 2024-09-11T00:58:17.744 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:58:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.726+0000 7efd631c0640 1 -- 172.21.15.78:0/2363748836 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efd4c0b53c0 msgr2=0x7efd4c0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.726+0000 7efd631c0640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efd4c0b53c0 0x7efd4c0b7880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efd5401c000 tx=0x7efd5401f000 comp rx=0 tx=0).stop 2024-09-11T00:58:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.726+0000 7efd631c0640 1 -- 172.21.15.78:0/2363748836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5c10a920 msgr2=0x7efd5c1997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.726+0000 7efd631c0640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5c10a920 0x7efd5c1997c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efd4800c920 tx=0x7efd4800cde0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.726+0000 7efd631c0640 1 -- 172.21.15.78:0/2363748836 shutdown_connections 2024-09-11T00:58:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.726+0000 7efd631c0640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efd4c0b53c0 0x7efd4c0b7880 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.726+0000 7efd631c0640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c10d250 0x7efd5c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.726+0000 7efd631c0640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5c10a920 0x7efd5c1997c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.726+0000 7efd631c0640 1 --2- 172.21.15.78:0/2363748836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd5c101810 0x7efd5c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.726+0000 7efd631c0640 1 -- 172.21.15.78:0/2363748836 >> 172.21.15.78:0/2363748836 conn(0x7efd5c076f50 msgr2=0x7efd5c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.726+0000 7efd631c0640 1 -- 172.21.15.78:0/2363748836 shutdown_connections 2024-09-11T00:58:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:17.726+0000 7efd631c0640 1 -- 172.21.15.78:0/2363748836 wait complete. 2024-09-11T00:58:18.622 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T00:58:18.868 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:19 smithi078 ceph-mon[31315]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:19 smithi078 ceph-mon[31315]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-11T00:58:19.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.367+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2869440458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b81076b0 msgr2=0x7f41b8107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:19.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.367+0000 7f41bd6c5640 1 --2- 172.21.15.78:0/2869440458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b81076b0 0x7f41b8107af0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009900 tx=0x7f41a002a410 comp rx=0 tx=0).stop 2024-09-11T00:58:19.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.367+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2869440458 shutdown_connections 2024-09-11T00:58:19.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.367+0000 7f41bd6c5640 1 --2- 172.21.15.78:0/2869440458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b8108030 0x7f41b810e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.367+0000 7f41bd6c5640 1 --2- 172.21.15.78:0/2869440458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b81076b0 0x7f41b8107af0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.367+0000 7f41bd6c5640 1 --2- 172.21.15.78:0/2869440458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8101ca0 0x7f41b8102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.367+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2869440458 >> 172.21.15.78:0/2869440458 conn(0x7f41b80fd730 msgr2=0x7f41b80ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:19.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.367+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2869440458 shutdown_connections 2024-09-11T00:58:19.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.367+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2869440458 wait complete. 2024-09-11T00:58:19.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.367+0000 7f41bd6c5640 1 Processor -- start 2024-09-11T00:58:19.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.367+0000 7f41bd6c5640 1 -- start start 2024-09-11T00:58:19.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41bd6c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8101ca0 0x7f41b8199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41bd6c5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b81076b0 0x7f41b8199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41bd6c5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b8108030 0x7f41b819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41bd6c5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8111d80 con 0x7f41b8108030 2024-09-11T00:58:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41bd6c5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8111c00 con 0x7f41b81076b0 2024-09-11T00:58:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41bd6c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8111f00 con 0x7f41b8101ca0 2024-09-11T00:58:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b8108030 0x7f41b819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b8108030 0x7f41b819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38026/0 (socket says 172.21.15.78:38026) 2024-09-11T00:58:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41b77fe640 1 -- 172.21.15.78:0/2143186373 learned_addr learned my addr 172.21.15.78:0/2143186373 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41b67fc640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b81076b0 0x7f41b8199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8101ca0 0x7f41b8199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41b77fe640 1 -- 172.21.15.78:0/2143186373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8101ca0 msgr2=0x7f41b8199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41b77fe640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8101ca0 0x7f41b8199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41b77fe640 1 -- 172.21.15.78:0/2143186373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b81076b0 msgr2=0x7f41b8199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41b77fe640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b81076b0 0x7f41b8199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41b77fe640 1 -- 172.21.15.78:0/2143186373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b819f4d0 con 0x7f41b8108030 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8101ca0 0x7f41b8199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41b77fe640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b8108030 0x7f41b819ed00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f41b000b970 tx=0x7f41b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41affff640 1 -- 172.21.15.78:0/2143186373 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0019070 con 0x7f41b8108030 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2143186373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b819f7c0 con 0x7f41b8108030 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41affff640 1 -- 172.21.15.78:0/2143186373 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41b0014440 con 0x7f41b8108030 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.368+0000 7f41affff640 1 -- 172.21.15.78:0/2143186373 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b00075d0 con 0x7f41b8108030 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.369+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2143186373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b81a5f00 con 0x7f41b8108030 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.371+0000 7f41bfa9a740 1 Processor -- start 2024-09-11T00:58:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41affff640 1 -- 172.21.15.78:0/2143186373 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f41b000cb40 con 0x7f41b8108030 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41bfa9a740 1 -- start start 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41bfa9a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b2d30 0x5650954b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41bfa9a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b3650 0x5650954bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41bfa9a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954bf5e0 0x5650954c19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41bfa9a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56509542ac10 con 0x5650954b3650 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41bfa9a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56509541a040 con 0x5650954b2d30 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41bfa9a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56509541de50 con 0x5650954bf5e0 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41b6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b3650 0x5650954bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41b6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b3650 0x5650954bf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38030/0 (socket says 172.21.15.78:38030) 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41b6ffd640 1 -- 172.21.15.78:0/2613021950 learned_addr learned my addr 172.21.15.78:0/2613021950 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41b67fc640 1 --2- 172.21.15.78:0/2613021950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b2d30 0x5650954b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41b67fc640 1 --2- 172.21.15.78:0/2613021950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954bf5e0 0x5650954c19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41b6ffd640 1 -- 172.21.15.78:0/2613021950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954bf5e0 msgr2=0x5650954c19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/2613021950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954bf5e0 0x5650954c19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41b6ffd640 1 -- 172.21.15.78:0/2613021950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b2d30 msgr2=0x5650954b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/2613021950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b2d30 0x5650954b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41b6ffd640 1 -- 172.21.15.78:0/2613021950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565095178540 con 0x5650954b3650 2024-09-11T00:58:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.372+0000 7f41b67fc640 1 --2- 172.21.15.78:0/2613021950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b2d30 0x5650954b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.373+0000 7f41affff640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f41a40b5330 0x7f41a40b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.373+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/2613021950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b3650 0x5650954bf0a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f41a800aa40 tx=0x7f41a800af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822bb62dfede845d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.373+0000 7f41b67fc640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f41a40b5330 0x7f41a40b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.373+0000 7f41ad7fa640 1 -- 172.21.15.78:0/2613021950 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8015070 con 0x5650954b3650 2024-09-11T00:58:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.373+0000 7f41affff640 1 -- 172.21.15.78:0/2143186373 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f41b00e54c0 con 0x7f41b8108030 2024-09-11T00:58:19.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.373+0000 7f41ad7fa640 1 -- 172.21.15.78:0/2613021950 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a800ccc0 con 0x5650954b3650 2024-09-11T00:58:19.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.373+0000 7f41ad7fa640 1 -- 172.21.15.78:0/2613021950 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a80105b0 con 0x5650954b3650 2024-09-11T00:58:19.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2613021950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b3650 msgr2=0x5650954bf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:19.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41b67fc640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f41a40b5330 0x7f41a40b77f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f41a0005ab0 tx=0x7f41a0000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:19.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2613021950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b3650 0x5650954bf0a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f41a800aa40 tx=0x7f41a800af00 comp rx=0 tx=0).stop 2024-09-11T00:58:19.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2613021950 shutdown_connections 2024-09-11T00:58:19.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2613021950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954bf5e0 0x5650954c19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2613021950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b3650 0x5650954bf0a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2613021950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b2d30 0x5650954b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2613021950 >> 172.21.15.78:0/2613021950 conn(0x5650954b0b60 msgr2=0x5650954b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:19.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2613021950 shutdown_connections 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2613021950 wait complete. 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 -- start start 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 Processor -- start 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b2d30 0x5650954bc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b3650 0x5650954bca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 0x5650954bcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56509527b5f0 con 0x5650954bf5e0 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5650953b5010 con 0x5650954b3650 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5650954c2290 con 0x5650954b2d30 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.374+0000 7f41bfa9a740 10 client.0 _ll_register_callbacks cb 0x565095414340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 0x5650954bcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 0x5650954bcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38044/0 (socket says 172.21.15.78:38044) 2024-09-11T00:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41b77fe640 1 -- 172.21.15.78:0/2358719106 learned_addr learned my addr 172.21.15.78:0/2358719106 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b2d30 0x5650954bc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41b67fc640 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b3650 0x5650954bca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41b77fe640 1 -- 172.21.15.78:0/2358719106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b2d30 msgr2=0x5650954bc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41b77fe640 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b2d30 0x5650954bc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41b77fe640 1 -- 172.21.15.78:0/2358719106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b3650 msgr2=0x5650954bca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41b77fe640 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b3650 0x5650954bca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41b77fe640 1 -- 172.21.15.78:0/2358719106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650953e2a00 con 0x5650954bf5e0 2024-09-11T00:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41b77fe640 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 0x5650954bcfb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f41b0014ba0 tx=0x7f41b00ab4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c66c462cc35d997 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f419e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f419e7fc640 1 -- 172.21.15.78:0/2358719106 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b00b4ed0 con 0x5650954bf5e0 2024-09-11T00:58:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f419e7fc640 1 -- 172.21.15.78:0/2358719106 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f41b00b01b0 con 0x5650954bf5e0 2024-09-11T00:58:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2358719106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a800ccc0 con 0x5650954bf5e0 2024-09-11T00:58:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f419e7fc640 1 -- 172.21.15.78:0/2358719106 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41b00abc80 con 0x5650954bf5e0 2024-09-11T00:58:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41bfa9a740 10 client.15243 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.375+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2358719106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5650954cdda0 con 0x5650954bf5e0 2024-09-11T00:58:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.376+0000 7f419e7fc640 1 -- 172.21.15.78:0/2358719106 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a800ccc0 con 0x5650954bf5e0 2024-09-11T00:58:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.376+0000 7f419e7fc640 1 -- 172.21.15.78:0/2358719106 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f41b00e5020 con 0x5650954bf5e0 2024-09-11T00:58:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.376+0000 7f41bfa9a740 10 client.15243 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.376+0000 7f41bfa9a740 10 client.15243 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:58:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.376+0000 7f41bfa9a740 20 client.15243 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.376+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5650954d03f0 0x5650954d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.376+0000 7f41bfa9a740 4 client.15243 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:58:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.376+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2358719106 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56509542cb60 con 0x5650954d03f0 2024-09-11T00:58:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.376+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5650954d03f0 0x5650954d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.376+0000 7f419e7fc640 10 client.15243 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.376+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5650954d03f0 0x5650954d1860 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f6060bc838ac55 server_cookie=738e4773b5b2c24f in_seq=0 out_seq=0 2024-09-11T00:58:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f419e7fc640 1 -- 172.21.15.78:0/2358719106 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56509542cb60 con 0x5650954d03f0 2024-09-11T00:58:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f419e7fc640 10 client.15243 handle_command_reply: tid=0 2024-09-11T00:58:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f41bfa9a740 1 client.15243 shutdown 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2358719106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 msgr2=0x5650954bcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 0x5650954bcfb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f41b0014ba0 tx=0x7f41b00ab4b0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2358719106 shutdown_connections 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5650954d03f0 0x5650954d1860 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 0x5650954bcfb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b3650 0x5650954bca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2358719106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b2d30 0x5650954bc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f419e7fc640 0 client.15243 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2358719106 >> 172.21.15.78:0/2358719106 conn(0x5650954b0b60 msgr2=0x5650954b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2358719106 shutdown_connections 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.377+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2358719106 wait complete. 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.378+0000 7f41bfa9a740 20 client.15243 trim_cache size 0 max 16384 2024-09-11T00:58:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41bfa9a740 1 Processor -- start 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41bfa9a740 1 -- start start 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41bfa9a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b2d30 0x5650954bba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41bfa9a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b3650 0x5650954d9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41bfa9a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954bf5e0 0x5650954da470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41bfa9a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5650954cfba0 con 0x5650954b2d30 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41bfa9a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5650954d0270 con 0x5650954bf5e0 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41bfa9a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5650954d0940 con 0x5650954b3650 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41b6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b2d30 0x5650954bba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41b6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b2d30 0x5650954bba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38050/0 (socket says 172.21.15.78:38050) 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41b6ffd640 1 -- 172.21.15.78:0/2307432367 learned_addr learned my addr 172.21.15.78:0/2307432367 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41b77fe640 1 --2- 172.21.15.78:0/2307432367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954bf5e0 0x5650954da470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41b67fc640 1 --2- 172.21.15.78:0/2307432367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b3650 0x5650954d9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41b6ffd640 1 -- 172.21.15.78:0/2307432367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b3650 msgr2=0x5650954d9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/2307432367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b3650 0x5650954d9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41b6ffd640 1 -- 172.21.15.78:0/2307432367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954bf5e0 msgr2=0x5650954da470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/2307432367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954bf5e0 0x5650954da470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.396+0000 7f41b6ffd640 1 -- 172.21.15.78:0/2307432367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650954b15e0 con 0x5650954b2d30 2024-09-11T00:58:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.397+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/2307432367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b2d30 0x5650954bba20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f41a8007fe0 tx=0x7f41a801c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcdfa11448306d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.397+0000 7f41acff9640 1 -- 172.21.15.78:0/2307432367 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8032070 con 0x5650954b2d30 2024-09-11T00:58:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.397+0000 7f41acff9640 1 -- 172.21.15.78:0/2307432367 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a800cc60 con 0x5650954b2d30 2024-09-11T00:58:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.397+0000 7f41acff9640 1 -- 172.21.15.78:0/2307432367 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8015040 con 0x5650954b2d30 2024-09-11T00:58:19.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2307432367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b2d30 msgr2=0x5650954bba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:19.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2307432367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b2d30 0x5650954bba20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f41a8007fe0 tx=0x7f41a801c040 comp rx=0 tx=0).stop 2024-09-11T00:58:19.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2307432367 shutdown_connections 2024-09-11T00:58:19.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2307432367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954bf5e0 0x5650954da470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2307432367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b3650 0x5650954d9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/2307432367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954b2d30 0x5650954bba20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2307432367 >> 172.21.15.78:0/2307432367 conn(0x5650954b0b60 msgr2=0x5650954b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:19.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2307432367 shutdown_connections 2024-09-11T00:58:19.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 -- 172.21.15.78:0/2307432367 wait complete. 2024-09-11T00:58:19.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 -- start start 2024-09-11T00:58:19.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 Processor -- start 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b2d30 0x5650954d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b3650 0x5650954d2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 0x56509553fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5650954d16e0 con 0x5650954bf5e0 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5650954d1010 con 0x5650954b2d30 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5650954d1db0 con 0x5650954b3650 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.398+0000 7f41bfa9a740 10 client.0 _ll_register_callbacks cb 0x7f41b00abac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 0x56509553fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 0x56509553fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38066/0 (socket says 172.21.15.78:38066) 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41b77fe640 1 -- 172.21.15.78:0/3433982081 learned_addr learned my addr 172.21.15.78:0/3433982081 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41b67fc640 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b3650 0x5650954d2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b2d30 0x5650954d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41b77fe640 1 -- 172.21.15.78:0/3433982081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b3650 msgr2=0x5650954d2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41b77fe640 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b3650 0x5650954d2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41b77fe640 1 -- 172.21.15.78:0/3433982081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b2d30 msgr2=0x5650954d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41b77fe640 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b2d30 0x5650954d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41b77fe640 1 -- 172.21.15.78:0/3433982081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650955402a0 con 0x5650954bf5e0 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41b77fe640 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 0x56509553fd60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f41b00ab980 tx=0x7f41b00a7a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262e942e5f4b09c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f419e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f419e7fc640 1 -- 172.21.15.78:0/3433982081 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b00c5070 con 0x5650954bf5e0 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f419e7fc640 1 -- 172.21.15.78:0/3433982081 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f41b00b4ed0 con 0x5650954bf5e0 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41bfa9a740 1 -- 172.21.15.78:0/3433982081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a800cc60 con 0x5650954bf5e0 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f419e7fc640 1 -- 172.21.15.78:0/3433982081 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41b00d2040 con 0x5650954bf5e0 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41bfa9a740 10 client.15249 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f41bfa9a740 1 -- 172.21.15.78:0/3433982081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565095541390 con 0x5650954bf5e0 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.399+0000 7f419e7fc640 1 -- 172.21.15.78:0/3433982081 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a800cc60 con 0x5650954bf5e0 2024-09-11T00:58:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.400+0000 7f419e7fc640 1 -- 172.21.15.78:0/3433982081 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f41b00aa950 con 0x5650954bf5e0 2024-09-11T00:58:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.400+0000 7f41bfa9a740 10 client.15249 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.400+0000 7f41bfa9a740 10 client.15249 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:58:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.400+0000 7f41bfa9a740 20 client.15249 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.400+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x565095545d40 0x565095544a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.400+0000 7f41bfa9a740 4 client.15249 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:58:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.400+0000 7f41bfa9a740 1 -- 172.21.15.78:0/3433982081 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "status"}) -- 0x565095428610 con 0x565095545d40 2024-09-11T00:58:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.400+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x565095545d40 0x565095544a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.400+0000 7f419e7fc640 10 client.15249 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.400+0000 7f41b6ffd640 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x565095545d40 0x565095544a50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=913794346aea152d server_cookie=dd7036d26e8103fa in_seq=0 out_seq=0 2024-09-11T00:58:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.400+0000 7f419e7fc640 1 -- 172.21.15.78:0/3433982081 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x565095428610 con 0x565095545d40 2024-09-11T00:58:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.400+0000 7f419e7fc640 10 client.15249 handle_command_reply: tid=0 2024-09-11T00:58:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bfa9a740 1 client.15249 shutdown 2024-09-11T00:58:19.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bfa9a740 1 -- 172.21.15.78:0/3433982081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 msgr2=0x56509553fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:19.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 0x56509553fd60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f41b00ab980 tx=0x7f41b00a7a80 comp rx=0 tx=0).stop 2024-09-11T00:58:19.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bfa9a740 1 -- 172.21.15.78:0/3433982081 shutdown_connections 2024-09-11T00:58:19.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x565095545d40 0x565095544a50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650954bf5e0 0x56509553fd60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650954b3650 0x5650954d2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bfa9a740 1 --2- 172.21.15.78:0/3433982081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650954b2d30 0x5650954d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bfa9a740 1 -- 172.21.15.78:0/3433982081 >> 172.21.15.78:0/3433982081 conn(0x5650954b0b60 msgr2=0x5650954b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f419e7fc640 0 client.15249 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bfa9a740 1 -- 172.21.15.78:0/3433982081 shutdown_connections 2024-09-11T00:58:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bfa9a740 1 -- 172.21.15.78:0/3433982081 wait complete. 2024-09-11T00:58:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bfa9a740 20 client.15249 trim_cache size 0 max 16384 2024-09-11T00:58:19.426 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:58:19.426 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:58:19.426 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T00:58:19.426 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24599, 2024-09-11T00:58:19.426 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T00:58:19.426 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T00:58:19.426 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T00:58:19.426 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 85.287840532000004, 2024-09-11T00:58:19.427 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-11T00:58:19.427 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-09-11T00:58:19.427 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-09-11T00:58:19.427 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 97.312889006999995 2024-09-11T00:58:19.427 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:58:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2143186373 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f41a40b5330 msgr2=0x7f41a40b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bd6c5640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f41a40b5330 0x7f41a40b77f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f41a0005ab0 tx=0x7f41a0000bc0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2143186373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b8108030 msgr2=0x7f41b819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.401+0000 7f41bd6c5640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b8108030 0x7f41b819ed00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f41b000b970 tx=0x7f41b000be30 comp rx=0 tx=0).stop 2024-09-11T00:58:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.402+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2143186373 shutdown_connections 2024-09-11T00:58:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.402+0000 7f41bd6c5640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f41a40b5330 0x7f41a40b77f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.402+0000 7f41bd6c5640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b8108030 0x7f41b819ed00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.402+0000 7f41bd6c5640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b81076b0 0x7f41b8199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.402+0000 7f41bd6c5640 1 --2- 172.21.15.78:0/2143186373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b8101ca0 0x7f41b8199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.402+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2143186373 >> 172.21.15.78:0/2143186373 conn(0x7f41b80fd730 msgr2=0x7f41b806b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.402+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2143186373 shutdown_connections 2024-09-11T00:58:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:19.402+0000 7f41bd6c5640 1 -- 172.21.15.78:0/2143186373 wait complete. 2024-09-11T00:58:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:19 smithi110 ceph-mon[34712]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:19 smithi110 ceph-mon[34712]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-11T00:58:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:19 smithi112 ceph-mon[35384]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:19 smithi112 ceph-mon[35384]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-11T00:58:20.158 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T00:58:20.158 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T00:58:20.170 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T00:58:20.170 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T00:58:20.417 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.934+0000 7f29f3b31640 1 -- 172.21.15.78:0/1348490626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec101cd0 msgr2=0x7f29ec1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.934+0000 7f29f3b31640 1 --2- 172.21.15.78:0/1348490626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec101cd0 0x7f29ec1020b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009900 tx=0x7f29e802a410 comp rx=0 tx=0).stop 2024-09-11T00:58:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.934+0000 7f29f3b31640 1 -- 172.21.15.78:0/1348490626 shutdown_connections 2024-09-11T00:58:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.934+0000 7f29f3b31640 1 --2- 172.21.15.78:0/1348490626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29ec108060 0x7f29ec10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.934+0000 7f29f3b31640 1 --2- 172.21.15.78:0/1348490626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec1076e0 0x7f29ec107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.934+0000 7f29f3b31640 1 --2- 172.21.15.78:0/1348490626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec101cd0 0x7f29ec1020b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.934+0000 7f29f3b31640 1 -- 172.21.15.78:0/1348490626 >> 172.21.15.78:0/1348490626 conn(0x7f29ec0fd700 msgr2=0x7f29ec0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.934+0000 7f29f3b31640 1 -- 172.21.15.78:0/1348490626 shutdown_connections 2024-09-11T00:58:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.934+0000 7f29f3b31640 1 -- 172.21.15.78:0/1348490626 wait complete. 2024-09-11T00:58:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f3b31640 1 Processor -- start 2024-09-11T00:58:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f3b31640 1 -- start start 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f3b31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec101cd0 0x7f29ec1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f3b31640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec1076e0 0x7f29ec199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f3b31640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29ec108060 0x7f29ec19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f3b31640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec112e40 con 0x7f29ec1076e0 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f3b31640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec112cc0 con 0x7f29ec108060 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f3b31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec112fc0 con 0x7f29ec101cd0 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f10cc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec1076e0 0x7f29ec199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f10cc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec1076e0 0x7f29ec199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38086/0 (socket says 172.21.15.78:38086) 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f10cc640 1 -- 172.21.15.78:0/2962794780 learned_addr learned my addr 172.21.15.78:0/2962794780 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f18cd640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec101cd0 0x7f29ec1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f20ce640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29ec108060 0x7f29ec19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f10cc640 1 -- 172.21.15.78:0/2962794780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec101cd0 msgr2=0x7f29ec1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f10cc640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec101cd0 0x7f29ec1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f10cc640 1 -- 172.21.15.78:0/2962794780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29ec108060 msgr2=0x7f29ec19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f10cc640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29ec108060 0x7f29ec19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f10cc640 1 -- 172.21.15.78:0/2962794780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ec19f3a0 con 0x7f29ec1076e0 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.935+0000 7f29f20ce640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29ec108060 0x7f29ec19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.936+0000 7f29f10cc640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec1076e0 0x7f29ec199830 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00c890 tx=0x7f29dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.936+0000 7f29e2ffd640 1 -- 172.21.15.78:0/2962794780 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc004220 con 0x7f29ec1076e0 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.936+0000 7f29e2ffd640 1 -- 172.21.15.78:0/2962794780 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29dc0043c0 con 0x7f29ec1076e0 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.936+0000 7f29f3b31640 1 -- 172.21.15.78:0/2962794780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ec19f690 con 0x7f29ec1076e0 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.936+0000 7f29e2ffd640 1 -- 172.21.15.78:0/2962794780 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc004a00 con 0x7f29ec1076e0 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.936+0000 7f29f3b31640 1 -- 172.21.15.78:0/2962794780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ec1a5dd0 con 0x7f29ec1076e0 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.937+0000 7f29f3b31640 1 -- 172.21.15.78:0/2962794780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29ec1020b0 con 0x7f29ec1076e0 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.938+0000 7f29e2ffd640 1 -- 172.21.15.78:0/2962794780 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f29dc0079b0 con 0x7f29ec1076e0 2024-09-11T00:58:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.939+0000 7f29e2ffd640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f29d80b5330 0x7f29d80b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.939+0000 7f29f18cd640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f29d80b5330 0x7f29d80b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.939+0000 7f29e2ffd640 1 -- 172.21.15.78:0/2962794780 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f29dc0e5e30 con 0x7f29ec1076e0 2024-09-11T00:58:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.939+0000 7f29f18cd640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f29d80b5330 0x7f29d80b77f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29e8031440 tx=0x7f29e8005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:20.942+0000 7f29e2ffd640 1 -- 172.21.15.78:0/2962794780 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f29dc014030 con 0x7f29ec1076e0 2024-09-11T00:58:21.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.137+0000 7f29f3b31640 1 -- 172.21.15.78:0/2962794780 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f29ec063590 con 0x7f29d80b5330 2024-09-11T00:58:21.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.139+0000 7f29e2ffd640 1 -- 172.21.15.78:0/2962794780 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f29ec063590 con 0x7f29d80b5330 2024-09-11T00:58:21.141 INFO:teuthology.orchestra.run.smithi078.stdout:Backend: cephadm 2024-09-11T00:58:21.141 INFO:teuthology.orchestra.run.smithi078.stdout:Available: Yes 2024-09-11T00:58:21.141 INFO:teuthology.orchestra.run.smithi078.stdout:Paused: No 2024-09-11T00:58:21.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.141+0000 7f29f3b31640 1 -- 172.21.15.78:0/2962794780 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f29d80b5330 msgr2=0x7f29d80b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:21.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.141+0000 7f29f3b31640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f29d80b5330 0x7f29d80b77f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29e8031440 tx=0x7f29e8005e80 comp rx=0 tx=0).stop 2024-09-11T00:58:21.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.141+0000 7f29f3b31640 1 -- 172.21.15.78:0/2962794780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec1076e0 msgr2=0x7f29ec199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:21.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.141+0000 7f29f3b31640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec1076e0 0x7f29ec199830 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00c890 tx=0x7f29dc00cd50 comp rx=0 tx=0).stop 2024-09-11T00:58:21.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.141+0000 7f29f3b31640 1 -- 172.21.15.78:0/2962794780 shutdown_connections 2024-09-11T00:58:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.141+0000 7f29f3b31640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f29d80b5330 0x7f29d80b77f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.141+0000 7f29f3b31640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29ec108060 0x7f29ec19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.141+0000 7f29f3b31640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec1076e0 0x7f29ec199830 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.141+0000 7f29f3b31640 1 --2- 172.21.15.78:0/2962794780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec101cd0 0x7f29ec1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.141+0000 7f29f3b31640 1 -- 172.21.15.78:0/2962794780 >> 172.21.15.78:0/2962794780 conn(0x7f29ec0fd700 msgr2=0x7f29ec0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.141+0000 7f29f3b31640 1 -- 172.21.15.78:0/2962794780 shutdown_connections 2024-09-11T00:58:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:21.141+0000 7f29f3b31640 1 -- 172.21.15.78:0/2962794780 wait complete. 2024-09-11T00:58:21.430 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:21 smithi078 ceph-mon[31315]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-11T00:58:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:21 smithi110 ceph-mon[34712]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-11T00:58:21.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:21 smithi112 ceph-mon[35384]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-11T00:58:21.865 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T00:58:22.115 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:22.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.673+0000 7f2c48953640 1 -- 172.21.15.78:0/3313003992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c441076c0 msgr2=0x7f2c44107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:22.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.673+0000 7f2c48953640 1 --2- 172.21.15.78:0/3313003992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c441076c0 0x7f2c44107b00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009900 tx=0x7f2c3802a410 comp rx=0 tx=0).stop 2024-09-11T00:58:22.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.673+0000 7f2c48953640 1 -- 172.21.15.78:0/3313003992 shutdown_connections 2024-09-11T00:58:22.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.673+0000 7f2c48953640 1 --2- 172.21.15.78:0/3313003992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c44108040 0x7f2c4410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:22.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.673+0000 7f2c48953640 1 --2- 172.21.15.78:0/3313003992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c441076c0 0x7f2c44107b00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:22.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.673+0000 7f2c48953640 1 --2- 172.21.15.78:0/3313003992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c44101cb0 0x7f2c44102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:22.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.673+0000 7f2c48953640 1 -- 172.21.15.78:0/3313003992 >> 172.21.15.78:0/3313003992 conn(0x7f2c440fd700 msgr2=0x7f2c440ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:22.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.673+0000 7f2c48953640 1 -- 172.21.15.78:0/3313003992 shutdown_connections 2024-09-11T00:58:22.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.673+0000 7f2c48953640 1 -- 172.21.15.78:0/3313003992 wait complete. 2024-09-11T00:58:22.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.674+0000 7f2c48953640 1 Processor -- start 2024-09-11T00:58:22.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.674+0000 7f2c48953640 1 -- start start 2024-09-11T00:58:22.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.674+0000 7f2c48953640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c44101cb0 0x7f2c441982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:22.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c48953640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c441076c0 0x7f2c44198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:22.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c48953640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c44108040 0x7f2c4419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:22.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c48953640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44111d40 con 0x7f2c44101cb0 2024-09-11T00:58:22.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c48953640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44111bc0 con 0x7f2c441076c0 2024-09-11T00:58:22.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c48953640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44111ec0 con 0x7f2c44108040 2024-09-11T00:58:22.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c4259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c44101cb0 0x7f2c441982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:22.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c4259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c44101cb0 0x7f2c441982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38114/0 (socket says 172.21.15.78:38114) 2024-09-11T00:58:22.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c4259c640 1 -- 172.21.15.78:0/3205954225 learned_addr learned my addr 172.21.15.78:0/3205954225 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:22.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c41d9b640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c441076c0 0x7f2c44198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:22.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c42d9d640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c44108040 0x7f2c4419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:22.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c4259c640 1 -- 172.21.15.78:0/3205954225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c44108040 msgr2=0x7f2c4419dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:22.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c4259c640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c44108040 0x7f2c4419dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:22.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c4259c640 1 -- 172.21.15.78:0/3205954225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c441076c0 msgr2=0x7f2c44198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:22.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c4259c640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c441076c0 0x7f2c44198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:22.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c4259c640 1 -- 172.21.15.78:0/3205954225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4419e3a0 con 0x7f2c44101cb0 2024-09-11T00:58:22.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c41d9b640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c441076c0 0x7f2c44198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:22.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c4259c640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c44101cb0 0x7f2c441982f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400d990 tx=0x7f2c3400de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:22.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c337fe640 1 -- 172.21.15.78:0/3205954225 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3401b070 con 0x7f2c44101cb0 2024-09-11T00:58:22.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c337fe640 1 -- 172.21.15.78:0/3205954225 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c3400edf0 con 0x7f2c44101cb0 2024-09-11T00:58:22.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c48953640 1 -- 172.21.15.78:0/3205954225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4419e690 con 0x7f2c44101cb0 2024-09-11T00:58:22.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c337fe640 1 -- 172.21.15.78:0/3205954225 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34014800 con 0x7f2c44101cb0 2024-09-11T00:58:22.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.675+0000 7f2c48953640 1 -- 172.21.15.78:0/3205954225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4406b510 con 0x7f2c44101cb0 2024-09-11T00:58:22.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.676+0000 7f2c48953640 1 -- 172.21.15.78:0/3205954225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c44102110 con 0x7f2c44101cb0 2024-09-11T00:58:22.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.678+0000 7f2c337fe640 1 -- 172.21.15.78:0/3205954225 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f2c3400e040 con 0x7f2c44101cb0 2024-09-11T00:58:22.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.678+0000 7f2c337fe640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2c2c0b5330 0x7f2c2c0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:22.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.678+0000 7f2c41d9b640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2c2c0b5330 0x7f2c2c0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:22.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.678+0000 7f2c337fe640 1 -- 172.21.15.78:0/3205954225 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f2c340e7c10 con 0x7f2c44101cb0 2024-09-11T00:58:22.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.679+0000 7f2c41d9b640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2c2c0b5330 0x7f2c2c0b77f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2c38031440 tx=0x7f2c380095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:22.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.681+0000 7f2c337fe640 1 -- 172.21.15.78:0/3205954225 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2c34025070 con 0x7f2c44101cb0 2024-09-11T00:58:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.864+0000 7f2c48953640 1 -- 172.21.15.78:0/3205954225 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2c4406c0a0 con 0x7f2c2c0b5330 2024-09-11T00:58:22.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.877+0000 7f2c337fe640 1 -- 172.21.15.78:0/3205954225 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f2c4406c0a0 con 0x7f2c2c0b5330 2024-09-11T00:58:22.880 INFO:teuthology.orchestra.run.smithi078.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T00:58:22.880 INFO:teuthology.orchestra.run.smithi078.stdout:mds.a smithi078 running (104s) 21s ago 103s 18.5M - 19.3.0-4854-g8f219c47 589dd18248e4 40e21add21e0 2024-09-11T00:58:22.880 INFO:teuthology.orchestra.run.smithi078.stdout:mds.b smithi110 running (101s) 34s ago 100s 26.3M - 19.3.0-4854-g8f219c47 589dd18248e4 7019b796cf94 2024-09-11T00:58:22.880 INFO:teuthology.orchestra.run.smithi078.stdout:mds.c smithi112 running (116s) 34s ago 115s 20.7M - 19.3.0-4854-g8f219c47 589dd18248e4 3af06f9e3ebf 2024-09-11T00:58:22.880 INFO:teuthology.orchestra.run.smithi078.stdout:mds.d smithi078 running (2m) 21s ago 2m 13.6M - 19.3.0-4854-g8f219c47 589dd18248e4 4e4330e32fad 2024-09-11T00:58:22.880 INFO:teuthology.orchestra.run.smithi078.stdout:mds.e smithi110 running (2m) 34s ago 2m 17.3M - 19.3.0-4854-g8f219c47 589dd18248e4 da3ad5bad4b1 2024-09-11T00:58:22.880 INFO:teuthology.orchestra.run.smithi078.stdout:mds.f smithi112 running (119s) 34s ago 118s 17.3M - 19.3.0-4854-g8f219c47 589dd18248e4 8aa434e7ed0f 2024-09-11T00:58:22.880 INFO:teuthology.orchestra.run.smithi078.stdout:mds.g smithi078 running (108s) 21s ago 107s 17.5M - 19.3.0-4854-g8f219c47 589dd18248e4 33a1b47b8cfb 2024-09-11T00:58:22.880 INFO:teuthology.orchestra.run.smithi078.stdout:mds.h smithi110 running (113s) 34s ago 113s 18.4M - 19.3.0-4854-g8f219c47 589dd18248e4 7242af13f126 2024-09-11T00:58:22.881 INFO:teuthology.orchestra.run.smithi078.stdout:mds.i smithi112 running (111s) 34s ago 110s 21.0M - 19.3.0-4854-g8f219c47 589dd18248e4 be7bba0306b4 2024-09-11T00:58:22.881 INFO:teuthology.orchestra.run.smithi078.stdout:mds.j smithi078 running (2m) 21s ago 2m 16.1M - 19.3.0-4854-g8f219c47 589dd18248e4 a0258905a31c 2024-09-11T00:58:22.881 INFO:teuthology.orchestra.run.smithi078.stdout:mds.k smithi110 running (2m) 34s ago 2m 17.2M - 19.3.0-4854-g8f219c47 589dd18248e4 10ae6b85e67a 2024-09-11T00:58:22.881 INFO:teuthology.orchestra.run.smithi078.stdout:mds.l smithi112 running (2m) 34s ago 2m 17.1M - 19.3.0-4854-g8f219c47 589dd18248e4 04aca3bc67a0 2024-09-11T00:58:22.881 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.x smithi078 *:9283,8765 running (9m) 21s ago 9m 516M - 19.3.0-4854-g8f219c47 589dd18248e4 41c1f890f76f 2024-09-11T00:58:22.881 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.y smithi110 *:8443,8765 running (7m) 34s ago 7m 456M - 19.3.0-4854-g8f219c47 589dd18248e4 c5d729218aea 2024-09-11T00:58:22.881 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.z smithi112 *:8443,8765 running (8m) 34s ago 8m 457M - 19.3.0-4854-g8f219c47 589dd18248e4 8ece7eb0deef 2024-09-11T00:58:22.881 INFO:teuthology.orchestra.run.smithi078.stdout:mon.a smithi078 running (9m) 21s ago 9m 52.9M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 ac3d67069611 2024-09-11T00:58:22.881 INFO:teuthology.orchestra.run.smithi078.stdout:mon.b smithi110 running (8m) 34s ago 8m 41.9M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 befced7d7ecb 2024-09-11T00:58:22.881 INFO:teuthology.orchestra.run.smithi078.stdout:mon.c smithi112 running (8m) 34s ago 8m 43.4M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 4f047f3ebd3e 2024-09-11T00:58:22.881 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 smithi078 running (7m) 21s ago 7m 47.1M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 00a7ca0c9ebb 2024-09-11T00:58:22.882 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 smithi110 running (6m) 34s ago 6m 54.2M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 4eae4a626bdf 2024-09-11T00:58:22.882 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 smithi112 running (6m) 34s ago 6m 56.5M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 7feeff381784 2024-09-11T00:58:22.882 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 smithi078 running (5m) 21s ago 5m 52.3M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 4d2a6e3898ba 2024-09-11T00:58:22.882 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 smithi110 running (5m) 34s ago 5m 43.3M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 17d77cbbf603 2024-09-11T00:58:22.882 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 smithi112 running (5m) 34s ago 5m 52.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 58ebd9a95a3d 2024-09-11T00:58:22.882 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 smithi078 running (4m) 21s ago 4m 44.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 5e8aebc0c442 2024-09-11T00:58:22.882 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 smithi110 running (4m) 34s ago 4m 54.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 477616b7556c 2024-09-11T00:58:22.882 INFO:teuthology.orchestra.run.smithi078.stdout:osd.8 smithi112 running (3m) 34s ago 3m 57.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 c75d07ecd2d2 2024-09-11T00:58:22.882 INFO:teuthology.orchestra.run.smithi078.stdout:osd.9 smithi078 running (3m) 21s ago 3m 51.7M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 6181ab3466d8 2024-09-11T00:58:22.882 INFO:teuthology.orchestra.run.smithi078.stdout:osd.10 smithi110 running (2m) 34s ago 2m 44.8M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 f19988511860 2024-09-11T00:58:22.882 INFO:teuthology.orchestra.run.smithi078.stdout:osd.11 smithi112 running (2m) 34s ago 2m 43.7M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 766ddd613566 2024-09-11T00:58:22.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.879+0000 7f2c48953640 1 -- 172.21.15.78:0/3205954225 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2c2c0b5330 msgr2=0x7f2c2c0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:22.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.879+0000 7f2c48953640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2c2c0b5330 0x7f2c2c0b77f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2c38031440 tx=0x7f2c380095f0 comp rx=0 tx=0).stop 2024-09-11T00:58:22.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.879+0000 7f2c48953640 1 -- 172.21.15.78:0/3205954225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c44101cb0 msgr2=0x7f2c441982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:22.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.879+0000 7f2c48953640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c44101cb0 0x7f2c441982f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400d990 tx=0x7f2c3400de50 comp rx=0 tx=0).stop 2024-09-11T00:58:22.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.879+0000 7f2c48953640 1 -- 172.21.15.78:0/3205954225 shutdown_connections 2024-09-11T00:58:22.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.879+0000 7f2c48953640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2c2c0b5330 0x7f2c2c0b77f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:22.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.879+0000 7f2c48953640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c44108040 0x7f2c4419dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:22.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.879+0000 7f2c48953640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c441076c0 0x7f2c44198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:22.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.879+0000 7f2c48953640 1 --2- 172.21.15.78:0/3205954225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c44101cb0 0x7f2c441982f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:22.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.879+0000 7f2c48953640 1 -- 172.21.15.78:0/3205954225 >> 172.21.15.78:0/3205954225 conn(0x7f2c440fd700 msgr2=0x7f2c440ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:22.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.879+0000 7f2c48953640 1 -- 172.21.15.78:0/3205954225 shutdown_connections 2024-09-11T00:58:22.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:22.879+0000 7f2c48953640 1 -- 172.21.15.78:0/3205954225 wait complete. 2024-09-11T00:58:23.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:23 smithi110 ceph-mon[34712]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:23.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:23 smithi110 ceph-mon[34712]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-11T00:58:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:23 smithi112 ceph-mon[35384]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:23 smithi112 ceph-mon[35384]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-11T00:58:23.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:23 smithi078 ceph-mon[31315]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:23.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:23 smithi078 ceph-mon[31315]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-11T00:58:23.719 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T00:58:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:24.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:24 smithi078 ceph-mon[31315]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:24.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:24 smithi110 ceph-mon[34712]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:24.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:24 smithi112 ceph-mon[35384]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:24.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.464+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/1849270505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a4101cb0 msgr2=0x7fd0a4102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.464+0000 7fd0a8bf1640 1 --2- 172.21.15.78:0/1849270505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a4101cb0 0x7fd0a4102090 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd094009900 tx=0x7fd09402a410 comp rx=0 tx=0).stop 2024-09-11T00:58:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.464+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/1849270505 shutdown_connections 2024-09-11T00:58:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.464+0000 7fd0a8bf1640 1 --2- 172.21.15.78:0/1849270505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0a4108040 0x7fd0a410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.464+0000 7fd0a8bf1640 1 --2- 172.21.15.78:0/1849270505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a41076c0 0x7fd0a4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.464+0000 7fd0a8bf1640 1 --2- 172.21.15.78:0/1849270505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a4101cb0 0x7fd0a4102090 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.464+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/1849270505 >> 172.21.15.78:0/1849270505 conn(0x7fd0a40fd700 msgr2=0x7fd0a40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.464+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/1849270505 shutdown_connections 2024-09-11T00:58:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.464+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/1849270505 wait complete. 2024-09-11T00:58:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd0a8bf1640 1 Processor -- start 2024-09-11T00:58:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd0a8bf1640 1 -- start start 2024-09-11T00:58:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd0a8bf1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0a4101cb0 0x7fd0a4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd0a8bf1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a41076c0 0x7fd0a41997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd0a8bf1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a4108040 0x7fd0a419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd0a8bf1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4111d00 con 0x7fd0a41076c0 2024-09-11T00:58:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd0a8bf1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4111b80 con 0x7fd0a4101cb0 2024-09-11T00:58:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd0a8bf1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4111e80 con 0x7fd0a4108040 2024-09-11T00:58:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd09bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a41076c0 0x7fd0a41997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd09bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a41076c0 0x7fd0a41997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58990/0 (socket says 172.21.15.78:58990) 2024-09-11T00:58:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd09bfff640 1 -- 172.21.15.78:0/2673802182 learned_addr learned my addr 172.21.15.78:0/2673802182 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd0a259c640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0a4101cb0 0x7fd0a4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd09bfff640 1 -- 172.21.15.78:0/2673802182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a4108040 msgr2=0x7fd0a419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd09bfff640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a4108040 0x7fd0a419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd09bfff640 1 -- 172.21.15.78:0/2673802182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0a4101cb0 msgr2=0x7fd0a4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd09bfff640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0a4101cb0 0x7fd0a4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.465+0000 7fd09bfff640 1 -- 172.21.15.78:0/2673802182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a419f330 con 0x7fd0a41076c0 2024-09-11T00:58:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.466+0000 7fd09bfff640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a41076c0 0x7fd0a41997c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00c890 tx=0x7fd08c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.466+0000 7fd09b7fe640 1 -- 172.21.15.78:0/2673802182 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c004220 con 0x7fd0a41076c0 2024-09-11T00:58:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.466+0000 7fd09b7fe640 1 -- 172.21.15.78:0/2673802182 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd08c0043c0 con 0x7fd0a41076c0 2024-09-11T00:58:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.466+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/2673802182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a419f620 con 0x7fd0a41076c0 2024-09-11T00:58:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.466+0000 7fd09b7fe640 1 -- 172.21.15.78:0/2673802182 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c004a00 con 0x7fd0a41076c0 2024-09-11T00:58:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.466+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/2673802182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a41a5ce0 con 0x7fd0a41076c0 2024-09-11T00:58:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.467+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/2673802182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a4102090 con 0x7fd0a41076c0 2024-09-11T00:58:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.468+0000 7fd09b7fe640 1 -- 172.21.15.78:0/2673802182 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fd08c0079b0 con 0x7fd0a41076c0 2024-09-11T00:58:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.469+0000 7fd09b7fe640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd0900b5330 0x7fd0900b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.469+0000 7fd0a259c640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd0900b5330 0x7fd0900b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.469+0000 7fd09b7fe640 1 -- 172.21.15.78:0/2673802182 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd08c0e5e30 con 0x7fd0a41076c0 2024-09-11T00:58:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.469+0000 7fd0a259c640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd0900b5330 0x7fd0900b77f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd094031440 tx=0x7fd094005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.472+0000 7fd09b7fe640 1 -- 172.21.15.78:0/2673802182 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd08c014030 con 0x7fd0a41076c0 2024-09-11T00:58:24.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.657+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/2673802182 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fd0a406b540 con 0x7fd0900b5330 2024-09-11T00:58:24.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.663+0000 7fd09b7fe640 1 -- 172.21.15.78:0/2673802182 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fd0a406b540 con 0x7fd0900b5330 2024-09-11T00:58:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T00:58:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:mds.all 12/12 36s ago 2m smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi112=c;smithi112=f;smithi112=i;smithi112=l;count:12 2024-09-11T00:58:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:mgr 3/3 36s ago 8m smithi078=x;smithi110=y;smithi112=z;count:3 2024-09-11T00:58:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:mon 3/3 36s ago 8m smithi078:172.21.15.78=a;smithi110:172.21.15.110=b;smithi112:172.21.15.112=c;count:3 2024-09-11T00:58:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:osd 12 36s ago - 2024-09-11T00:58:24.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.665+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/2673802182 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd0900b5330 msgr2=0x7fd0900b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:24.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.665+0000 7fd0a8bf1640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd0900b5330 0x7fd0900b77f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd094031440 tx=0x7fd094005e80 comp rx=0 tx=0).stop 2024-09-11T00:58:24.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.665+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/2673802182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a41076c0 msgr2=0x7fd0a41997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.665+0000 7fd0a8bf1640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a41076c0 0x7fd0a41997c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00c890 tx=0x7fd08c00cd50 comp rx=0 tx=0).stop 2024-09-11T00:58:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.665+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/2673802182 shutdown_connections 2024-09-11T00:58:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.665+0000 7fd0a8bf1640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd0900b5330 0x7fd0900b77f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.665+0000 7fd0a8bf1640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a4108040 0x7fd0a419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.665+0000 7fd0a8bf1640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a41076c0 0x7fd0a41997c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.665+0000 7fd0a8bf1640 1 --2- 172.21.15.78:0/2673802182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0a4101cb0 0x7fd0a4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.665+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/2673802182 >> 172.21.15.78:0/2673802182 conn(0x7fd0a40fd700 msgr2=0x7fd0a40ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.665+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/2673802182 shutdown_connections 2024-09-11T00:58:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:24.666+0000 7fd0a8bf1640 1 -- 172.21.15.78:0/2673802182 wait complete. 2024-09-11T00:58:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:25 smithi110 ceph-mon[34712]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T00:58:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:25 smithi110 ceph-mon[34712]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:25.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:25 smithi078 ceph-mon[31315]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T00:58:25.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:25 smithi078 ceph-mon[31315]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:25.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:25 smithi112 ceph-mon[35384]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T00:58:25.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:25 smithi112 ceph-mon[35384]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:25.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T00:58:25.753 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.244+0000 7fcdf0917640 1 -- 172.21.15.78:0/3034006016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdec10d1a0 msgr2=0x7fcdec10f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.244+0000 7fcdf0917640 1 --2- 172.21.15.78:0/3034006016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdec10d1a0 0x7fcdec10f600 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009ca0 tx=0x7fcde402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.244+0000 7fcdf0917640 1 -- 172.21.15.78:0/3034006016 shutdown_connections 2024-09-11T00:58:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.244+0000 7fcdf0917640 1 --2- 172.21.15.78:0/3034006016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdec10d1a0 0x7fcdec10f600 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.244+0000 7fcdf0917640 1 --2- 172.21.15.78:0/3034006016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec104660 0x7fcdec10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.244+0000 7fcdf0917640 1 --2- 172.21.15.78:0/3034006016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdec103d40 0x7fcdec104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.244+0000 7fcdf0917640 1 -- 172.21.15.78:0/3034006016 >> 172.21.15.78:0/3034006016 conn(0x7fcdec0fd7e0 msgr2=0x7fcdec0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.244+0000 7fcdf0917640 1 -- 172.21.15.78:0/3034006016 shutdown_connections 2024-09-11T00:58:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.244+0000 7fcdf0917640 1 -- 172.21.15.78:0/3034006016 wait complete. 2024-09-11T00:58:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdf0917640 1 Processor -- start 2024-09-11T00:58:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdf0917640 1 -- start start 2024-09-11T00:58:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdf0917640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdec103d40 0x7fcdec199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdf0917640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec104660 0x7fcdec1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdf0917640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdec10d1a0 0x7fcdec19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdf0917640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec112e90 con 0x7fcdec103d40 2024-09-11T00:58:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdf0917640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec112d10 con 0x7fcdec104660 2024-09-11T00:58:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdf0917640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec113010 con 0x7fcdec10d1a0 2024-09-11T00:58:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdea59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdec103d40 0x7fcdec199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdea59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdec103d40 0x7fcdec199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59008/0 (socket says 172.21.15.78:59008) 2024-09-11T00:58:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdea59c640 1 -- 172.21.15.78:0/2660577308 learned_addr learned my addr 172.21.15.78:0/2660577308 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcde9d9b640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec104660 0x7fcdec1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdead9d640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdec10d1a0 0x7fcdec19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdea59c640 1 -- 172.21.15.78:0/2660577308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdec10d1a0 msgr2=0x7fcdec19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdea59c640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdec10d1a0 0x7fcdec19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdea59c640 1 -- 172.21.15.78:0/2660577308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec104660 msgr2=0x7fcdec1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdea59c640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec104660 0x7fcdec1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdea59c640 1 -- 172.21.15.78:0/2660577308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec19f430 con 0x7fcdec103d40 2024-09-11T00:58:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcde9d9b640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec104660 0x7fcdec1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:26.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.245+0000 7fcdea59c640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdec103d40 0x7fcdec199380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400c970 tx=0x7fcdd400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:26.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.246+0000 7fcde37fe640 1 -- 172.21.15.78:0/2660577308 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4007bb0 con 0x7fcdec103d40 2024-09-11T00:58:26.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.246+0000 7fcde37fe640 1 -- 172.21.15.78:0/2660577308 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd4007d50 con 0x7fcdec103d40 2024-09-11T00:58:26.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.246+0000 7fcdf0917640 1 -- 172.21.15.78:0/2660577308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec19f720 con 0x7fcdec103d40 2024-09-11T00:58:26.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.246+0000 7fcde37fe640 1 -- 172.21.15.78:0/2660577308 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd400f450 con 0x7fcdec103d40 2024-09-11T00:58:26.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.246+0000 7fcdf0917640 1 -- 172.21.15.78:0/2660577308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec1018d0 con 0x7fcdec103d40 2024-09-11T00:58:26.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.247+0000 7fcdf0917640 1 -- 172.21.15.78:0/2660577308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec104120 con 0x7fcdec103d40 2024-09-11T00:58:26.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.248+0000 7fcde37fe640 1 -- 172.21.15.78:0/2660577308 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fcdd4004f00 con 0x7fcdec103d40 2024-09-11T00:58:26.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.249+0000 7fcde37fe640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdd80b5330 0x7fcdd80b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:26.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.249+0000 7fcde9d9b640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdd80b5330 0x7fcdd80b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:26.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.249+0000 7fcde37fe640 1 -- 172.21.15.78:0/2660577308 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fcdd40e5e80 con 0x7fcdec103d40 2024-09-11T00:58:26.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.250+0000 7fcde9d9b640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdd80b5330 0x7fcdd80b77f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006fd0 tx=0x7fcddc008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:26.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.256+0000 7fcde37fe640 1 -- 172.21.15.78:0/2660577308 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcdd4014030 con 0x7fcdec103d40 2024-09-11T00:58:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:26 smithi078 ceph-mon[31315]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T00:58:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:26 smithi110 ceph-mon[34712]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T00:58:26.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.445+0000 7fcdf0917640 1 -- 172.21.15.78:0/2660577308 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fcdec0ffb00 con 0x7fcdd80b5330 2024-09-11T00:58:26.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.447+0000 7fcde37fe640 1 -- 172.21.15.78:0/2660577308 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fcdec0ffb00 con 0x7fcdd80b5330 2024-09-11T00:58:26.449 INFO:teuthology.orchestra.run.smithi078.stdout:HOST ADDR LABELS STATUS 2024-09-11T00:58:26.450 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 172.21.15.78 2024-09-11T00:58:26.450 INFO:teuthology.orchestra.run.smithi078.stdout:smithi110 172.21.15.110 2024-09-11T00:58:26.450 INFO:teuthology.orchestra.run.smithi078.stdout:smithi112 172.21.15.112 2024-09-11T00:58:26.450 INFO:teuthology.orchestra.run.smithi078.stdout:3 hosts in cluster 2024-09-11T00:58:26.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.449+0000 7fcdf0917640 1 -- 172.21.15.78:0/2660577308 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdd80b5330 msgr2=0x7fcdd80b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:26.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.449+0000 7fcdf0917640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdd80b5330 0x7fcdd80b77f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006fd0 tx=0x7fcddc008040 comp rx=0 tx=0).stop 2024-09-11T00:58:26.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.449+0000 7fcdf0917640 1 -- 172.21.15.78:0/2660577308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdec103d40 msgr2=0x7fcdec199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:26.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.449+0000 7fcdf0917640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdec103d40 0x7fcdec199380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400c970 tx=0x7fcdd400ce30 comp rx=0 tx=0).stop 2024-09-11T00:58:26.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.449+0000 7fcdf0917640 1 -- 172.21.15.78:0/2660577308 shutdown_connections 2024-09-11T00:58:26.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.449+0000 7fcdf0917640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdd80b5330 0x7fcdd80b77f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:26.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.449+0000 7fcdf0917640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdec10d1a0 0x7fcdec19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:26.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.449+0000 7fcdf0917640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec104660 0x7fcdec1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:26.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.449+0000 7fcdf0917640 1 --2- 172.21.15.78:0/2660577308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdec103d40 0x7fcdec199380 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:26.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.449+0000 7fcdf0917640 1 -- 172.21.15.78:0/2660577308 >> 172.21.15.78:0/2660577308 conn(0x7fcdec0fd7e0 msgr2=0x7fcdec0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:26.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.449+0000 7fcdf0917640 1 -- 172.21.15.78:0/2660577308 shutdown_connections 2024-09-11T00:58:26.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:26.450+0000 7fcdf0917640 1 -- 172.21.15.78:0/2660577308 wait complete. 2024-09-11T00:58:26.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:26 smithi112 ceph-mon[35384]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T00:58:27.180 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T00:58:27.405 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:27 smithi078 ceph-mon[31315]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:27 smithi110 ceph-mon[34712]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:27 smithi112 ceph-mon[35384]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.896+0000 7fc197424640 1 -- 172.21.15.78:0/4259932663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc190109150 msgr2=0x7fc19010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.896+0000 7fc197424640 1 --2- 172.21.15.78:0/4259932663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc190109150 0x7fc19010f950 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc188009ca0 tx=0x7fc18802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.896+0000 7fc197424640 1 -- 172.21.15.78:0/4259932663 shutdown_connections 2024-09-11T00:58:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.896+0000 7fc197424640 1 --2- 172.21.15.78:0/4259932663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc190109150 0x7fc19010f950 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.896+0000 7fc197424640 1 --2- 172.21.15.78:0/4259932663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1901087d0 0x7fc190108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.896+0000 7fc197424640 1 --2- 172.21.15.78:0/4259932663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc190102dc0 0x7fc1901031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.896+0000 7fc197424640 1 -- 172.21.15.78:0/4259932663 >> 172.21.15.78:0/4259932663 conn(0x7fc1900fe830 msgr2=0x7fc190100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.896+0000 7fc197424640 1 -- 172.21.15.78:0/4259932663 shutdown_connections 2024-09-11T00:58:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.896+0000 7fc197424640 1 -- 172.21.15.78:0/4259932663 wait complete. 2024-09-11T00:58:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc197424640 1 Processor -- start 2024-09-11T00:58:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc197424640 1 -- start start 2024-09-11T00:58:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc197424640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc190102dc0 0x7fc190199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc197424640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1901087d0 0x7fc190199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc197424640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc190109150 0x7fc19019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc197424640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc190112e10 con 0x7fc190102dc0 2024-09-11T00:58:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc197424640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc190112c90 con 0x7fc190109150 2024-09-11T00:58:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc197424640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc190112f90 con 0x7fc1901087d0 2024-09-11T00:58:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc1951c0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc190102dc0 0x7fc190199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc1951c0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc190102dc0 0x7fc190199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59026/0 (socket says 172.21.15.78:59026) 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc1951c0640 1 -- 172.21.15.78:0/2289268353 learned_addr learned my addr 172.21.15.78:0/2289268353 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc1951c0640 1 -- 172.21.15.78:0/2289268353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1901087d0 msgr2=0x7fc190199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc1951c0640 1 --2- 172.21.15.78:0/2289268353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1901087d0 0x7fc190199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc1951c0640 1 -- 172.21.15.78:0/2289268353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc190109150 msgr2=0x7fc19019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc1951c0640 1 --2- 172.21.15.78:0/2289268353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc190109150 0x7fc19019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc1951c0640 1 -- 172.21.15.78:0/2289268353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19019f3f0 con 0x7fc190102dc0 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.897+0000 7fc1951c0640 1 --2- 172.21.15.78:0/2289268353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc190102dc0 0x7fc190199340 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00b4a0 tx=0x7fc18c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.898+0000 7fc1867fc640 1 -- 172.21.15.78:0/2289268353 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c014070 con 0x7fc190102dc0 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.898+0000 7fc1867fc640 1 -- 172.21.15.78:0/2289268353 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18c010ac0 con 0x7fc190102dc0 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.898+0000 7fc197424640 1 -- 172.21.15.78:0/2289268353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19019f6e0 con 0x7fc190102dc0 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.898+0000 7fc1867fc640 1 -- 172.21.15.78:0/2289268353 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c017c30 con 0x7fc190102dc0 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.898+0000 7fc197424640 1 -- 172.21.15.78:0/2289268353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901a5da0 con 0x7fc190102dc0 2024-09-11T00:58:27.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.899+0000 7fc197424640 1 -- 172.21.15.78:0/2289268353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1901031a0 con 0x7fc190102dc0 2024-09-11T00:58:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.900+0000 7fc1867fc640 1 -- 172.21.15.78:0/2289268353 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fc18c010610 con 0x7fc190102dc0 2024-09-11T00:58:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.901+0000 7fc1867fc640 1 --2- 172.21.15.78:0/2289268353 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc17c0b53b0 0x7fc17c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.901+0000 7fc1949bf640 1 --2- 172.21.15.78:0/2289268353 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc17c0b53b0 0x7fc17c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.901+0000 7fc1867fc640 1 -- 172.21.15.78:0/2289268353 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc18c0e52d0 con 0x7fc190102dc0 2024-09-11T00:58:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.901+0000 7fc1949bf640 1 --2- 172.21.15.78:0/2289268353 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc17c0b53b0 0x7fc17c0b7870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc190107210 tx=0x7fc180005bf0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:27.905+0000 7fc1867fc640 1 -- 172.21.15.78:0/2289268353 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc18c0ac0e0 con 0x7fc190102dc0 2024-09-11T00:58:28.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.090+0000 7fc197424640 1 -- 172.21.15.78:0/2289268353 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc190063590 con 0x7fc17c0b53b0 2024-09-11T00:58:28.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.091+0000 7fc1867fc640 1 -- 172.21.15.78:0/2289268353 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc190063590 con 0x7fc17c0b53b0 2024-09-11T00:58:28.094 INFO:teuthology.orchestra.run.smithi078.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T00:58:28.094 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-11T00:58:28.095 INFO:teuthology.orchestra.run.smithi078.stdout:smithi110 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002T480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-11T00:58:28.095 INFO:teuthology.orchestra.run.smithi078.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T00:58:28.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.094+0000 7fc197424640 1 -- 172.21.15.78:0/2289268353 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc17c0b53b0 msgr2=0x7fc17c0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:28.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.094+0000 7fc197424640 1 --2- 172.21.15.78:0/2289268353 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc17c0b53b0 0x7fc17c0b7870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc190107210 tx=0x7fc180005bf0 comp rx=0 tx=0).stop 2024-09-11T00:58:28.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.094+0000 7fc197424640 1 -- 172.21.15.78:0/2289268353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc190102dc0 msgr2=0x7fc190199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:28.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.094+0000 7fc197424640 1 --2- 172.21.15.78:0/2289268353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc190102dc0 0x7fc190199340 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00b4a0 tx=0x7fc18c00b960 comp rx=0 tx=0).stop 2024-09-11T00:58:28.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.094+0000 7fc197424640 1 -- 172.21.15.78:0/2289268353 shutdown_connections 2024-09-11T00:58:28.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.094+0000 7fc197424640 1 --2- 172.21.15.78:0/2289268353 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc17c0b53b0 0x7fc17c0b7870 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:28.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.094+0000 7fc197424640 1 --2- 172.21.15.78:0/2289268353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc190109150 0x7fc19019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:28.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.094+0000 7fc197424640 1 --2- 172.21.15.78:0/2289268353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1901087d0 0x7fc190199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:28.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.094+0000 7fc197424640 1 --2- 172.21.15.78:0/2289268353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc190102dc0 0x7fc190199340 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:28.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.094+0000 7fc197424640 1 -- 172.21.15.78:0/2289268353 >> 172.21.15.78:0/2289268353 conn(0x7fc1900fe830 msgr2=0x7fc190100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:28.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.094+0000 7fc197424640 1 -- 172.21.15.78:0/2289268353 shutdown_connections 2024-09-11T00:58:28.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:28.094+0000 7fc197424640 1 -- 172.21.15.78:0/2289268353 wait complete. 2024-09-11T00:58:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:28 smithi110 ceph-mon[34712]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T00:58:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:28 smithi110 ceph-mon[34712]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:28.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:28 smithi112 ceph-mon[35384]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T00:58:28.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:28 smithi112 ceph-mon[35384]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:28.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:28 smithi078 ceph-mon[31315]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T00:58:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:28 smithi078 ceph-mon[31315]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:28.877 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T00:58:28.887 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T00:58:28.887 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-11T00:58:29.137 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:29.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.621+0000 7ff1d794d640 1 -- 172.21.15.78:0/1666856494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0101c80 msgr2=0x7ff1d0102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.621+0000 7ff1d794d640 1 --2- 172.21.15.78:0/1666856494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0101c80 0x7ff1d0102060 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009900 tx=0x7ff1c402a410 comp rx=0 tx=0).stop 2024-09-11T00:58:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.621+0000 7ff1d794d640 1 -- 172.21.15.78:0/1666856494 shutdown_connections 2024-09-11T00:58:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.621+0000 7ff1d794d640 1 --2- 172.21.15.78:0/1666856494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0108010 0x7ff1d010e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.621+0000 7ff1d794d640 1 --2- 172.21.15.78:0/1666856494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1d0107690 0x7ff1d0107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.621+0000 7ff1d794d640 1 --2- 172.21.15.78:0/1666856494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0101c80 0x7ff1d0102060 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.621+0000 7ff1d794d640 1 -- 172.21.15.78:0/1666856494 >> 172.21.15.78:0/1666856494 conn(0x7ff1d00fd6d0 msgr2=0x7ff1d00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.621+0000 7ff1d794d640 1 -- 172.21.15.78:0/1666856494 shutdown_connections 2024-09-11T00:58:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.621+0000 7ff1d794d640 1 -- 172.21.15.78:0/1666856494 wait complete. 2024-09-11T00:58:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.621+0000 7ff1d794d640 1 Processor -- start 2024-09-11T00:58:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.621+0000 7ff1d794d640 1 -- start start 2024-09-11T00:58:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d794d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1d0101c80 0x7ff1d0199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d794d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0107690 0x7ff1d0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d794d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0108010 0x7ff1d019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d794d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d0112df0 con 0x7ff1d0107690 2024-09-11T00:58:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d794d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d0112c70 con 0x7ff1d0108010 2024-09-11T00:58:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d794d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d0112f70 con 0x7ff1d0101c80 2024-09-11T00:58:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d4ee8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0107690 0x7ff1d0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d4ee8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0107690 0x7ff1d0199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59044/0 (socket says 172.21.15.78:59044) 2024-09-11T00:58:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d4ee8640 1 -- 172.21.15.78:0/26140668 learned_addr learned my addr 172.21.15.78:0/26140668 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d56e9640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1d0101c80 0x7ff1d0199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d5eea640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0108010 0x7ff1d019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d4ee8640 1 -- 172.21.15.78:0/26140668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1d0101c80 msgr2=0x7ff1d0199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d4ee8640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1d0101c80 0x7ff1d0199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d4ee8640 1 -- 172.21.15.78:0/26140668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0108010 msgr2=0x7ff1d019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d4ee8640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0108010 0x7ff1d019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d4ee8640 1 -- 172.21.15.78:0/26140668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d019f2f0 con 0x7ff1d0107690 2024-09-11T00:58:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d4ee8640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0107690 0x7ff1d0199780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00d950 tx=0x7ff1cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1c27fc640 1 -- 172.21.15.78:0/26140668 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc017070 con 0x7ff1d0107690 2024-09-11T00:58:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1c27fc640 1 -- 172.21.15.78:0/26140668 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1cc009c30 con 0x7ff1d0107690 2024-09-11T00:58:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1c27fc640 1 -- 172.21.15.78:0/26140668 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc00f620 con 0x7ff1d0107690 2024-09-11T00:58:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d794d640 1 -- 172.21.15.78:0/26140668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d019f5e0 con 0x7ff1d0107690 2024-09-11T00:58:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.622+0000 7ff1d794d640 1 -- 172.21.15.78:0/26140668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d01a5ca0 con 0x7ff1d0107690 2024-09-11T00:58:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.623+0000 7ff1d794d640 1 -- 172.21.15.78:0/26140668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d0102060 con 0x7ff1d0107690 2024-09-11T00:58:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.625+0000 7ff1c27fc640 1 -- 172.21.15.78:0/26140668 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7ff1cc010460 con 0x7ff1d0107690 2024-09-11T00:58:29.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.625+0000 7ff1c27fc640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff1bc0b5330 0x7ff1bc0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:29.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.626+0000 7ff1d56e9640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff1bc0b5330 0x7ff1bc0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:29.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.626+0000 7ff1c27fc640 1 -- 172.21.15.78:0/26140668 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff1cc0e58d0 con 0x7ff1d0107690 2024-09-11T00:58:29.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.626+0000 7ff1d56e9640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff1bc0b5330 0x7ff1bc0b77f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4031440 tx=0x7ff1c4005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:29.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.629+0000 7ff1c27fc640 1 -- 172.21.15.78:0/26140668 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff1cc009de0 con 0x7ff1d0107690 2024-09-11T00:58:29.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.842+0000 7ff1d794d640 1 -- 172.21.15.78:0/26140668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff1d002e5c0 con 0x7ff1d0107690 2024-09-11T00:58:29.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.843+0000 7ff1c27fc640 1 -- 172.21.15.78:0/26140668 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3132 (secure 0 0 0) 0x7ff1cc0ac030 con 0x7ff1d0107690 2024-09-11T00:58:29.846 INFO:teuthology.orchestra.run.smithi078.stdout:e28 2024-09-11T00:58:29.846 INFO:teuthology.orchestra.run.smithi078.stdout:btime 2024-09-11T00:57:05:238610+0000 2024-09-11T00:58:29.846 INFO:teuthology.orchestra.run.smithi078.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T00:58:29.846 INFO:teuthology.orchestra.run.smithi078.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T00:58:29.846 INFO:teuthology.orchestra.run.smithi078.stdout:legacy client fscid: 1 2024-09-11T00:58:29.846 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:29.846 INFO:teuthology.orchestra.run.smithi078.stdout:Filesystem 'cephfs' (1) 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:fs_name cephfs 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 28 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-09-11T00:56:53.792170+0000 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:modified 2024-09-11T00:57:04.821148+0000 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:tableserver 0 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:root 0 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:session_timeout 300 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:session_autoclose 300 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:max_file_size 1099511627776 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:max_xattr_size 65536 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:required_client_features {} 2024-09-11T00:58:29.847 INFO:teuthology.orchestra.run.smithi078.stdout:last_failure 0 2024-09-11T00:58:29.848 INFO:teuthology.orchestra.run.smithi078.stdout:last_failure_osd_epoch 0 2024-09-11T00:58:29.848 INFO:teuthology.orchestra.run.smithi078.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T00:58:29.848 INFO:teuthology.orchestra.run.smithi078.stdout:max_mds 5 2024-09-11T00:58:29.848 INFO:teuthology.orchestra.run.smithi078.stdout:in 0,1,2,3,4 2024-09-11T00:58:29.848 INFO:teuthology.orchestra.run.smithi078.stdout:up {0=24599,1=24580,2=24565,3=24563,4=14808} 2024-09-11T00:58:29.848 INFO:teuthology.orchestra.run.smithi078.stdout:failed 2024-09-11T00:58:29.848 INFO:teuthology.orchestra.run.smithi078.stdout:damaged 2024-09-11T00:58:29.848 INFO:teuthology.orchestra.run.smithi078.stdout:stopped 2024-09-11T00:58:29.848 INFO:teuthology.orchestra.run.smithi078.stdout:data_pools [3] 2024-09-11T00:58:29.848 INFO:teuthology.orchestra.run.smithi078.stdout:metadata_pool 2 2024-09-11T00:58:29.848 INFO:teuthology.orchestra.run.smithi078.stdout:inline_data disabled 2024-09-11T00:58:29.848 INFO:teuthology.orchestra.run.smithi078.stdout:balancer 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:bal_rank_mask -1 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:standby_count_wanted 0 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:qdb_cluster leader: 24599 members: 14808,24563,24565,24580,24599 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b{0:24599} state up:active seq 5 addr [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.f{0:24553} state up:standby-replay seq 1 addr [v2:172.21.15.112:6834/303956922,v1:172.21.15.112:6835/303956922] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.i{1:24580} state up:active seq 9 addr [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.e{1:24545} state up:standby-replay seq 1 addr [v2:172.21.15.110:6834/3549318748,v1:172.21.15.110:6835/3549318748] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c{2:24565} state up:active seq 10 addr [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.l{2:24529} state up:standby-replay seq 1 addr [v2:172.21.15.112:6832/1732756287,v1:172.21.15.112:6833/1732756287] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.h{3:24563} state up:active seq 10 addr [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.k{3:24521} state up:standby-replay seq 1 addr [v2:172.21.15.110:6832/2579452587,v1:172.21.15.110:6833/2579452587] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a{4:14808} state up:active seq 7 addr [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:58:29.849 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.g{4:14796} state up:standby-replay seq 1 addr [v2:172.21.15.78:6838/3902249887,v1:172.21.15.78:6839/3902249887] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:58:29.850 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:29.850 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:29.850 INFO:teuthology.orchestra.run.smithi078.stdout:Standby daemons: 2024-09-11T00:58:29.850 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:29.850 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.j{-1:14730} state up:standby seq 1 addr [v2:172.21.15.78:6834/494285934,v1:172.21.15.78:6835/494285934] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:58:29.850 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d{-1:14742} state up:standby seq 1 addr [v2:172.21.15.78:6836/1108876555,v1:172.21.15.78:6837/1108876555] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:58:29.850 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-11T00:58:29.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.845+0000 7ff1d794d640 1 -- 172.21.15.78:0/26140668 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff1bc0b5330 msgr2=0x7ff1bc0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:29.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.845+0000 7ff1d794d640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff1bc0b5330 0x7ff1bc0b77f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4031440 tx=0x7ff1c4005b50 comp rx=0 tx=0).stop 2024-09-11T00:58:29.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.845+0000 7ff1d794d640 1 -- 172.21.15.78:0/26140668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0107690 msgr2=0x7ff1d0199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:29.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.845+0000 7ff1d794d640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0107690 0x7ff1d0199780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00d950 tx=0x7ff1cc00de10 comp rx=0 tx=0).stop 2024-09-11T00:58:29.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.845+0000 7ff1d794d640 1 -- 172.21.15.78:0/26140668 shutdown_connections 2024-09-11T00:58:29.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.845+0000 7ff1d794d640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff1bc0b5330 0x7ff1bc0b77f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:29.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.845+0000 7ff1d794d640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0108010 0x7ff1d019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:29.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.845+0000 7ff1d794d640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0107690 0x7ff1d0199780 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:29.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.845+0000 7ff1d794d640 1 --2- 172.21.15.78:0/26140668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1d0101c80 0x7ff1d0199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:29.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.845+0000 7ff1d794d640 1 -- 172.21.15.78:0/26140668 >> 172.21.15.78:0/26140668 conn(0x7ff1d00fd6d0 msgr2=0x7ff1d00ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:29.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.845+0000 7ff1d794d640 1 -- 172.21.15.78:0/26140668 shutdown_connections 2024-09-11T00:58:29.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:29.845+0000 7ff1d794d640 1 -- 172.21.15.78:0/26140668 wait complete. 2024-09-11T00:58:30.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:29 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/26140668' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-11T00:58:30.191 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:29 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/26140668' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-11T00:58:30.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:29 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/26140668' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-11T00:58:30.510 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-11T00:58:30.754 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:31.178 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:30 smithi078 ceph-mon[31315]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T00:58:31.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:30 smithi110 ceph-mon[34712]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T00:58:31.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:30 smithi112 ceph-mon[35384]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T00:58:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.256+0000 7f031ba29640 1 -- 172.21.15.78:0/1573222598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0314101c90 msgr2=0x7f0314102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.256+0000 7f031ba29640 1 --2- 172.21.15.78:0/1573222598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0314101c90 0x7f0314102070 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0310009900 tx=0x7f031002a410 comp rx=0 tx=0).stop 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 -- 172.21.15.78:0/1573222598 shutdown_connections 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 --2- 172.21.15.78:0/1573222598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0314108020 0x7f031410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 --2- 172.21.15.78:0/1573222598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03141076a0 0x7f0314107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 --2- 172.21.15.78:0/1573222598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0314101c90 0x7f0314102070 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 -- 172.21.15.78:0/1573222598 >> 172.21.15.78:0/1573222598 conn(0x7f03140fd700 msgr2=0x7f03140ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 -- 172.21.15.78:0/1573222598 shutdown_connections 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 -- 172.21.15.78:0/1573222598 wait complete. 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 Processor -- start 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 -- start start 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0314101c90 0x7f0314103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03141076a0 0x7f0314103a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0314108020 0x7f0314103fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0314111ce0 con 0x7f03141076a0 2024-09-11T00:58:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0314111b60 con 0x7f0314108020 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.257+0000 7f031ba29640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0314111e60 con 0x7f0314101c90 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f0318fc4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03141076a0 0x7f0314103a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f0318fc4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03141076a0 0x7f0314103a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59068/0 (socket says 172.21.15.78:59068) 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f0318fc4640 1 -- 172.21.15.78:0/2771739785 learned_addr learned my addr 172.21.15.78:0/2771739785 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f03197c5640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0314101c90 0x7f0314103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f0319fc6640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0314108020 0x7f0314103fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f0318fc4640 1 -- 172.21.15.78:0/2771739785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0314101c90 msgr2=0x7f0314103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f0318fc4640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0314101c90 0x7f0314103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f0318fc4640 1 -- 172.21.15.78:0/2771739785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0314108020 msgr2=0x7f0314103fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f0318fc4640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0314108020 0x7f0314103fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f0318fc4640 1 -- 172.21.15.78:0/2771739785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03141a7600 con 0x7f03141076a0 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f0318fc4640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03141076a0 0x7f0314103a90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f030800c920 tx=0x7f030800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f03067fc640 1 -- 172.21.15.78:0/2771739785 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308007c70 con 0x7f03141076a0 2024-09-11T00:58:31.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f03067fc640 1 -- 172.21.15.78:0/2771739785 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0308007e10 con 0x7f03141076a0 2024-09-11T00:58:31.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f031ba29640 1 -- 172.21.15.78:0/2771739785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03141a78f0 con 0x7f03141076a0 2024-09-11T00:58:31.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f03067fc640 1 -- 172.21.15.78:0/2771739785 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308018820 con 0x7f03141076a0 2024-09-11T00:58:31.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.258+0000 7f031ba29640 1 -- 172.21.15.78:0/2771739785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03141a7e80 con 0x7f03141076a0 2024-09-11T00:58:31.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.259+0000 7f031ba29640 1 -- 172.21.15.78:0/2771739785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0314102070 con 0x7f03141076a0 2024-09-11T00:58:31.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.261+0000 7f03067fc640 1 -- 172.21.15.78:0/2771739785 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f030800f450 con 0x7f03141076a0 2024-09-11T00:58:31.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.261+0000 7f03067fc640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f03000b53b0 0x7f03000b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:31.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.261+0000 7f03197c5640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f03000b53b0 0x7f03000b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:31.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.261+0000 7f03067fc640 1 -- 172.21.15.78:0/2771739785 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f03080e60b0 con 0x7f03141076a0 2024-09-11T00:58:31.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.262+0000 7f03197c5640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f03000b53b0 0x7f03000b7870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0310006010 tx=0x7f0310000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:31.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.264+0000 7f03067fc640 1 -- 172.21.15.78:0/2771739785 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0308014030 con 0x7f03141076a0 2024-09-11T00:58:31.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.444+0000 7f031ba29640 1 -- 172.21.15.78:0/2771739785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f0314104850 con 0x7f03141076a0 2024-09-11T00:58:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.445+0000 7f03067fc640 1 -- 172.21.15.78:0/2771739785 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v77) ==== 56+0+5635 (secure 0 0 0) 0x7f03080acfd0 con 0x7f03141076a0 2024-09-11T00:58:31.447 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 77 2024-09-11T00:58:31.448 INFO:teuthology.orchestra.run.smithi078.stdout:fsid 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T00:58:31.448 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-09-11T00:48:27.652512+0000 2024-09-11T00:58:31.448 INFO:teuthology.orchestra.run.smithi078.stdout:modified 2024-09-11T00:57:17.572848+0000 2024-09-11T00:58:31.448 INFO:teuthology.orchestra.run.smithi078.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T00:58:31.449 INFO:teuthology.orchestra.run.smithi078.stdout:crush_version 26 2024-09-11T00:58:31.449 INFO:teuthology.orchestra.run.smithi078.stdout:full_ratio 0.95 2024-09-11T00:58:31.449 INFO:teuthology.orchestra.run.smithi078.stdout:backfillfull_ratio 0.9 2024-09-11T00:58:31.449 INFO:teuthology.orchestra.run.smithi078.stdout:nearfull_ratio 0.85 2024-09-11T00:58:31.449 INFO:teuthology.orchestra.run.smithi078.stdout:require_min_compat_client luminous 2024-09-11T00:58:31.449 INFO:teuthology.orchestra.run.smithi078.stdout:min_compat_client luminous 2024-09-11T00:58:31.449 INFO:teuthology.orchestra.run.smithi078.stdout:require_osd_release squid 2024-09-11T00:58:31.449 INFO:teuthology.orchestra.run.smithi078.stdout:stretch_mode_enabled false 2024-09-11T00:58:31.449 INFO:teuthology.orchestra.run.smithi078.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-11T00:58:31.449 INFO:teuthology.orchestra.run.smithi078.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:max_osd 12 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 up in weight 1 up_from 9 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6802/1424639053,v1:172.21.15.78:6803/1424639053] [v2:172.21.15.78:6804/1424639053,v1:172.21.15.78:6805/1424639053] exists,up beefc6d9-f74c-4200-96f2-ff697ad0d7dc 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 up in weight 1 up_from 14 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6800/580939385,v1:172.21.15.110:6801/580939385] [v2:172.21.15.110:6802/580939385,v1:172.21.15.110:6803/580939385] exists,up d742e767-d673-4bb9-b624-8bc969f5ceb1 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 up in weight 1 up_from 19 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6800/2983155763,v1:172.21.15.112:6801/2983155763] [v2:172.21.15.112:6802/2983155763,v1:172.21.15.112:6803/2983155763] exists,up b0106a5b-1465-4a50-96dd-934dbae49a4d 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 up in weight 1 up_from 25 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6810/2906086464,v1:172.21.15.78:6811/2906086464] [v2:172.21.15.78:6812/2906086464,v1:172.21.15.78:6813/2906086464] exists,up 3270362b-3d61-414a-a3d9-f2416ff8b870 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 up in weight 1 up_from 31 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6808/477287435,v1:172.21.15.110:6809/477287435] [v2:172.21.15.110:6810/477287435,v1:172.21.15.110:6811/477287435] exists,up e47746ec-9443-4004-abfa-17c1ed73b982 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 up in weight 1 up_from 36 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6808/3557095096,v1:172.21.15.112:6809/3557095096] [v2:172.21.15.112:6810/3557095096,v1:172.21.15.112:6811/3557095096] exists,up f8cedb01-310d-442d-a588-33197d32e208 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 up in weight 1 up_from 42 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6818/1267910775,v1:172.21.15.78:6819/1267910775] [v2:172.21.15.78:6820/1267910775,v1:172.21.15.78:6821/1267910775] exists,up d508a037-8f56-436b-b7bf-9a1c69f6d9e2 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 up in weight 1 up_from 47 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6816/1556494594,v1:172.21.15.110:6817/1556494594] [v2:172.21.15.110:6818/1556494594,v1:172.21.15.110:6819/1556494594] exists,up e8842047-09a6-48b8-8d0a-2d54e7f89063 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:osd.8 up in weight 1 up_from 53 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6816/2481321,v1:172.21.15.112:6817/2481321] [v2:172.21.15.112:6818/2481321,v1:172.21.15.112:6819/2481321] exists,up a3367da8-375a-4a46-a952-867d4b72ae2e 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:osd.9 up in weight 1 up_from 59 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6826/197048350,v1:172.21.15.78:6827/197048350] [v2:172.21.15.78:6828/197048350,v1:172.21.15.78:6829/197048350] exists,up d1825764-cefc-495b-99da-8ba429df0937 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:osd.10 up in weight 1 up_from 64 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6824/2101123168,v1:172.21.15.110:6825/2101123168] [v2:172.21.15.110:6826/2101123168,v1:172.21.15.110:6827/2101123168] exists,up e1cfff7a-8916-4845-a482-d79fa275da30 2024-09-11T00:58:31.450 INFO:teuthology.orchestra.run.smithi078.stdout:osd.11 up in weight 1 up_from 69 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6824/1526310654,v1:172.21.15.112:6825/1526310654] [v2:172.21.15.112:6826/1526310654,v1:172.21.15.112:6827/1526310654] exists,up fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68 2024-09-11T00:58:31.451 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.4 [2,11] 2024-09-11T00:58:31.451 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.10 [8,11] 2024-09-11T00:58:31.451 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.1b [2,4] 2024-09-11T00:58:31.451 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.3 [4,9] 2024-09-11T00:58:31.451 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.7 [3,9] 2024-09-11T00:58:31.451 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.17 [4,8,5,6] 2024-09-11T00:58:31.451 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.2c [4,9] 2024-09-11T00:58:31.451 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2390785703 expires 2024-09-12T00:48:52.336916+0000 2024-09-11T00:58:31.451 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/2984952017 expires 2024-09-12T00:48:52.336916+0000 2024-09-11T00:58:31.451 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/2984952017 expires 2024-09-12T00:48:52.336916+0000 2024-09-11T00:58:31.451 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/1038951931 expires 2024-09-12T00:50:15.928079+0000 2024-09-11T00:58:31.452 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/391908936 expires 2024-09-12T00:49:14.935550+0000 2024-09-11T00:58:31.452 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2706514734 expires 2024-09-12T00:50:15.928079+0000 2024-09-11T00:58:31.452 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2357025243 expires 2024-09-12T00:48:52.336916+0000 2024-09-11T00:58:31.452 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/1743270552 expires 2024-09-12T00:49:14.935550+0000 2024-09-11T00:58:31.452 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/1743270552 expires 2024-09-12T00:49:14.935550+0000 2024-09-11T00:58:31.452 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2795778473 expires 2024-09-12T00:49:14.935550+0000 2024-09-11T00:58:31.452 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2747741101 expires 2024-09-12T00:49:14.935550+0000 2024-09-11T00:58:31.452 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/1289343426 expires 2024-09-12T00:50:15.928079+0000 2024-09-11T00:58:31.452 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/3558896007 expires 2024-09-12T00:50:15.928079+0000 2024-09-11T00:58:31.452 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/438261739 expires 2024-09-12T00:48:52.336916+0000 2024-09-11T00:58:31.452 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/1289343426 expires 2024-09-12T00:50:15.928079+0000 2024-09-11T00:58:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.447+0000 7f031ba29640 1 -- 172.21.15.78:0/2771739785 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f03000b53b0 msgr2=0x7f03000b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.447+0000 7f031ba29640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f03000b53b0 0x7f03000b7870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0310006010 tx=0x7f0310000bc0 comp rx=0 tx=0).stop 2024-09-11T00:58:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.447+0000 7f031ba29640 1 -- 172.21.15.78:0/2771739785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03141076a0 msgr2=0x7f0314103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.447+0000 7f031ba29640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03141076a0 0x7f0314103a90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f030800c920 tx=0x7f030800cde0 comp rx=0 tx=0).stop 2024-09-11T00:58:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.447+0000 7f031ba29640 1 -- 172.21.15.78:0/2771739785 shutdown_connections 2024-09-11T00:58:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.447+0000 7f031ba29640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f03000b53b0 0x7f03000b7870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.447+0000 7f031ba29640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0314108020 0x7f0314103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.447+0000 7f031ba29640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03141076a0 0x7f0314103a90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.447+0000 7f031ba29640 1 --2- 172.21.15.78:0/2771739785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0314101c90 0x7f0314103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.447+0000 7f031ba29640 1 -- 172.21.15.78:0/2771739785 >> 172.21.15.78:0/2771739785 conn(0x7f03140fd700 msgr2=0x7f03140ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.447+0000 7f031ba29640 1 -- 172.21.15.78:0/2771739785 shutdown_connections 2024-09-11T00:58:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:31.447+0000 7f031ba29640 1 -- 172.21.15.78:0/2771739785 wait complete. 2024-09-11T00:58:32.042 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:31 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2771739785' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-11T00:58:32.070 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-11T00:58:32.084 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:58:32.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:31 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2771739785' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-11T00:58:32.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:31 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2771739785' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-11T00:58:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:32.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.831+0000 7f04d6aa3640 1 -- 172.21.15.78:0/2338142185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d01076d0 msgr2=0x7f04d0107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:32.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.831+0000 7f04d6aa3640 1 --2- 172.21.15.78:0/2338142185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d01076d0 0x7f04d0107b10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009900 tx=0x7f04bc02a410 comp rx=0 tx=0).stop 2024-09-11T00:58:32.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.831+0000 7f04d6aa3640 1 -- 172.21.15.78:0/2338142185 shutdown_connections 2024-09-11T00:58:32.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.831+0000 7f04d6aa3640 1 --2- 172.21.15.78:0/2338142185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04d0108050 0x7f04d010e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:32.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.831+0000 7f04d6aa3640 1 --2- 172.21.15.78:0/2338142185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d01076d0 0x7f04d0107b10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:32.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.831+0000 7f04d6aa3640 1 --2- 172.21.15.78:0/2338142185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d0101cc0 0x7f04d01020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:32.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.831+0000 7f04d6aa3640 1 -- 172.21.15.78:0/2338142185 >> 172.21.15.78:0/2338142185 conn(0x7f04d00fd750 msgr2=0x7f04d00ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:32.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.831+0000 7f04d6aa3640 1 -- 172.21.15.78:0/2338142185 shutdown_connections 2024-09-11T00:58:32.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.831+0000 7f04d6aa3640 1 -- 172.21.15.78:0/2338142185 wait complete. 2024-09-11T00:58:32.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.832+0000 7f04d6aa3640 1 Processor -- start 2024-09-11T00:58:32.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.832+0000 7f04d6aa3640 1 -- start start 2024-09-11T00:58:32.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.832+0000 7f04d6aa3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04d0101cc0 0x7f04d019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.832+0000 7f04d6aa3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d01076d0 0x7f04d019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.832+0000 7f04d6aa3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d0108050 0x7f04d019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.832+0000 7f04d6aa3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04d0111d10 con 0x7f04d0108050 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.832+0000 7f04d6aa3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04d0111b90 con 0x7f04d0101cc0 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.832+0000 7f04d6aa3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04d0111e90 con 0x7f04d01076d0 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.832+0000 7f04d5040640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d0108050 0x7f04d019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.832+0000 7f04d5040640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d0108050 0x7f04d019abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59084/0 (socket says 172.21.15.78:59084) 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.832+0000 7f04d5040640 1 -- 172.21.15.78:0/1552771214 learned_addr learned my addr 172.21.15.78:0/1552771214 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04cffff640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d01076d0 0x7f04d019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04d5040640 1 -- 172.21.15.78:0/1552771214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d01076d0 msgr2=0x7f04d019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04d5040640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d01076d0 0x7f04d019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04d5040640 1 -- 172.21.15.78:0/1552771214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04d0101cc0 msgr2=0x7f04d019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:32.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04d483f640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04d0101cc0 0x7f04d019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04d5040640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04d0101cc0 0x7f04d019a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04d5040640 1 -- 172.21.15.78:0/1552771214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d01a5500 con 0x7f04d0108050 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04cffff640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d01076d0 0x7f04d019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04d5040640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d0108050 0x7f04d019abc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009fd0 tx=0x7f04c800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04cdffb640 1 -- 172.21.15.78:0/1552771214 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8017070 con 0x7f04d0108050 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04d6aa3640 1 -- 172.21.15.78:0/1552771214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d01a57f0 con 0x7f04d0108050 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04cdffb640 1 -- 172.21.15.78:0/1552771214 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c8003e20 con 0x7f04d0108050 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04cdffb640 1 -- 172.21.15.78:0/1552771214 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8010460 con 0x7f04d0108050 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.833+0000 7f04d6aa3640 1 -- 172.21.15.78:0/1552771214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d01a5d30 con 0x7f04d0108050 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.834+0000 7f04d6aa3640 1 -- 172.21.15.78:0/1552771214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d01020a0 con 0x7f04d0108050 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.835+0000 7f04cdffb640 1 -- 172.21.15.78:0/1552771214 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f04c8003970 con 0x7f04d0108050 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.836+0000 7f04cdffb640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f04c00b5330 0x7f04c00b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.836+0000 7f04d483f640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f04c00b5330 0x7f04c00b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.836+0000 7f04cdffb640 1 -- 172.21.15.78:0/1552771214 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f04c80e5bc0 con 0x7f04d0108050 2024-09-11T00:58:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.836+0000 7f04d483f640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f04c00b5330 0x7f04c00b77f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f04c400a810 tx=0x7f04c4008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:32.840+0000 7f04cdffb640 1 -- 172.21.15.78:0/1552771214 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f04c80ac9d0 con 0x7f04d0108050 2024-09-11T00:58:33.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.051+0000 7f04d6aa3640 1 -- 172.21.15.78:0/1552771214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04d002e5c0 con 0x7f04d0108050 2024-09-11T00:58:33.054 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:32 smithi078 ceph-mon[31315]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-11T00:58:33.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.052+0000 7f04cdffb640 1 -- 172.21.15.78:0/1552771214 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11979 (secure 0 0 0) 0x7f04c80ac2d0 con 0x7f04d0108050 2024-09-11T00:58:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:33.056 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-09-11T00:57:05:238610+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T00:57:04.821148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T00:58:33.056 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-11T00:58:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.054+0000 7f04d6aa3640 1 -- 172.21.15.78:0/1552771214 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f04c00b5330 msgr2=0x7f04c00b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.054+0000 7f04d6aa3640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f04c00b5330 0x7f04c00b77f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f04c400a810 tx=0x7f04c4008040 comp rx=0 tx=0).stop 2024-09-11T00:58:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.054+0000 7f04d6aa3640 1 -- 172.21.15.78:0/1552771214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d0108050 msgr2=0x7f04d019abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.054+0000 7f04d6aa3640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d0108050 0x7f04d019abc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009fd0 tx=0x7f04c800b260 comp rx=0 tx=0).stop 2024-09-11T00:58:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.054+0000 7f04d6aa3640 1 -- 172.21.15.78:0/1552771214 shutdown_connections 2024-09-11T00:58:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.054+0000 7f04d6aa3640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f04c00b5330 0x7f04c00b77f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.054+0000 7f04d6aa3640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d0108050 0x7f04d019abc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.054+0000 7f04d6aa3640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d01076d0 0x7f04d019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.054+0000 7f04d6aa3640 1 --2- 172.21.15.78:0/1552771214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04d0101cc0 0x7f04d019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.054+0000 7f04d6aa3640 1 -- 172.21.15.78:0/1552771214 >> 172.21.15.78:0/1552771214 conn(0x7f04d00fd750 msgr2=0x7f04d00ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.054+0000 7f04d6aa3640 1 -- 172.21.15.78:0/1552771214 shutdown_connections 2024-09-11T00:58:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:33.054+0000 7f04d6aa3640 1 -- 172.21.15.78:0/1552771214 wait complete. 2024-09-11T00:58:33.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:32 smithi110 ceph-mon[34712]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-11T00:58:33.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:32 smithi112 ceph-mon[35384]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-11T00:58:33.883 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:58:34.133 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:34.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:33 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1552771214' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:34.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:33 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1552771214' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:34.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:33 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1552771214' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:34.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.616+0000 7f0c45d7a640 1 -- 172.21.15.78:0/186240992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c40101c90 msgr2=0x7f0c40102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:34.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.616+0000 7f0c45d7a640 1 --2- 172.21.15.78:0/186240992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c40101c90 0x7f0c40102070 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009900 tx=0x7f0c3002a410 comp rx=0 tx=0).stop 2024-09-11T00:58:34.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 -- 172.21.15.78:0/186240992 shutdown_connections 2024-09-11T00:58:34.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 --2- 172.21.15.78:0/186240992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c40108020 0x7f0c4010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:34.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 --2- 172.21.15.78:0/186240992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c401076a0 0x7f0c40107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:34.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 --2- 172.21.15.78:0/186240992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c40101c90 0x7f0c40102070 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:34.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 -- 172.21.15.78:0/186240992 >> 172.21.15.78:0/186240992 conn(0x7f0c400fd700 msgr2=0x7f0c400ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:34.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 -- 172.21.15.78:0/186240992 shutdown_connections 2024-09-11T00:58:34.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 -- 172.21.15.78:0/186240992 wait complete. 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 Processor -- start 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 -- start start 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c40101c90 0x7f0c400fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c401076a0 0x7f0c400fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40108020 0x7f0c400fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40111d70 con 0x7f0c40101c90 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40111bf0 con 0x7f0c401076a0 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c45d7a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40111ef0 con 0x7f0c40108020 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c3f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c40101c90 0x7f0c400fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.617+0000 7f0c3f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c40101c90 0x7f0c400fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57312/0 (socket says 172.21.15.78:57312) 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c3f7fe640 1 -- 172.21.15.78:0/2029373265 learned_addr learned my addr 172.21.15.78:0/2029373265 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c37fff640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c401076a0 0x7f0c400fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c3ffff640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40108020 0x7f0c400fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:34.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c3f7fe640 1 -- 172.21.15.78:0/2029373265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40108020 msgr2=0x7f0c400fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c3f7fe640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40108020 0x7f0c400fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c3f7fe640 1 -- 172.21.15.78:0/2029373265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c401076a0 msgr2=0x7f0c400fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c3f7fe640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c401076a0 0x7f0c400fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c3f7fe640 1 -- 172.21.15.78:0/2029373265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c401a7830 con 0x7f0c40101c90 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c3f7fe640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c40101c90 0x7f0c400fdc00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0c30005ab0 tx=0x7f0c30003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c3d7fa640 1 -- 172.21.15.78:0/2029373265 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3003f070 con 0x7f0c40101c90 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c3d7fa640 1 -- 172.21.15.78:0/2029373265 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c30031d60 con 0x7f0c40101c90 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c3d7fa640 1 -- 172.21.15.78:0/2029373265 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30035030 con 0x7f0c40101c90 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c45d7a640 1 -- 172.21.15.78:0/2029373265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c401a7ac0 con 0x7f0c40101c90 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.618+0000 7f0c45d7a640 1 -- 172.21.15.78:0/2029373265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c401a7ff0 con 0x7f0c40101c90 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.619+0000 7f0c45d7a640 1 -- 172.21.15.78:0/2029373265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c401020f0 con 0x7f0c40101c90 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.620+0000 7f0c3d7fa640 1 -- 172.21.15.78:0/2029373265 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f0c3003b4f0 con 0x7f0c40101c90 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.621+0000 7f0c3d7fa640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c2c0b53b0 0x7f0c2c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.621+0000 7f0c37fff640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c2c0b53b0 0x7f0c2c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:34.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.621+0000 7f0c3d7fa640 1 -- 172.21.15.78:0/2029373265 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0c30106fe0 con 0x7f0c40101c90 2024-09-11T00:58:34.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.622+0000 7f0c37fff640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c2c0b53b0 0x7f0c2c0b7870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c400ff6f0 tx=0x7f0c28009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:34.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.624+0000 7f0c3d7fa640 1 -- 172.21.15.78:0/2029373265 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0c300cddf0 con 0x7f0c40101c90 2024-09-11T00:58:34.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.835+0000 7f0c45d7a640 1 -- 172.21.15.78:0/2029373265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c400fef00 con 0x7f0c40101c90 2024-09-11T00:58:34.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.836+0000 7f0c3d7fa640 1 -- 172.21.15.78:0/2029373265 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11979 (secure 0 0 0) 0x7f0c400fef00 con 0x7f0c40101c90 2024-09-11T00:58:34.838 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:34.838 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-09-11T00:57:05:238610+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T00:57:04.821148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T00:58:34.838 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-11T00:58:34.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.838+0000 7f0c45d7a640 1 -- 172.21.15.78:0/2029373265 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c2c0b53b0 msgr2=0x7f0c2c0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:34.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.838+0000 7f0c45d7a640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c2c0b53b0 0x7f0c2c0b7870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c400ff6f0 tx=0x7f0c28009110 comp rx=0 tx=0).stop 2024-09-11T00:58:34.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.838+0000 7f0c45d7a640 1 -- 172.21.15.78:0/2029373265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c40101c90 msgr2=0x7f0c400fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:34.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.838+0000 7f0c45d7a640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c40101c90 0x7f0c400fdc00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0c30005ab0 tx=0x7f0c30003d50 comp rx=0 tx=0).stop 2024-09-11T00:58:34.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.838+0000 7f0c45d7a640 1 -- 172.21.15.78:0/2029373265 shutdown_connections 2024-09-11T00:58:34.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.838+0000 7f0c45d7a640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c2c0b53b0 0x7f0c2c0b7870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:34.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.838+0000 7f0c45d7a640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40108020 0x7f0c400fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:34.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.838+0000 7f0c45d7a640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c401076a0 0x7f0c400fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:34.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.838+0000 7f0c45d7a640 1 --2- 172.21.15.78:0/2029373265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c40101c90 0x7f0c400fdc00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:34.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.838+0000 7f0c45d7a640 1 -- 172.21.15.78:0/2029373265 >> 172.21.15.78:0/2029373265 conn(0x7f0c400fd700 msgr2=0x7f0c4006adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:34.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.838+0000 7f0c45d7a640 1 -- 172.21.15.78:0/2029373265 shutdown_connections 2024-09-11T00:58:34.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:34.839+0000 7f0c45d7a640 1 -- 172.21.15.78:0/2029373265 wait complete. 2024-09-11T00:58:35.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:34 smithi110 ceph-mon[34712]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:58:35.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:34 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2029373265' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:35.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:34 smithi112 ceph-mon[35384]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:58:35.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:34 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2029373265' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:35.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:34 smithi078 ceph-mon[31315]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:58:35.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:34 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2029373265' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:35.692 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:58:35.938 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.429+0000 7fd47d262640 1 -- 172.21.15.78:0/1305169920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd478101880 msgr2=0x7fd478101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.429+0000 7fd47d262640 1 --2- 172.21.15.78:0/1305169920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd478101880 0x7fd478101c60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd468009900 tx=0x7fd46802a410 comp rx=0 tx=0).stop 2024-09-11T00:58:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.429+0000 7fd47d262640 1 -- 172.21.15.78:0/1305169920 shutdown_connections 2024-09-11T00:58:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.429+0000 7fd47d262640 1 --2- 172.21.15.78:0/1305169920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4781055e0 0x7fd47810dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.429+0000 7fd47d262640 1 --2- 172.21.15.78:0/1305169920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd478104c60 0x7fd4781050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.429+0000 7fd47d262640 1 --2- 172.21.15.78:0/1305169920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd478101880 0x7fd478101c60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.429+0000 7fd47d262640 1 -- 172.21.15.78:0/1305169920 >> 172.21.15.78:0/1305169920 conn(0x7fd478076e60 msgr2=0x7fd478077270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.429+0000 7fd47d262640 1 -- 172.21.15.78:0/1305169920 shutdown_connections 2024-09-11T00:58:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.429+0000 7fd47d262640 1 -- 172.21.15.78:0/1305169920 wait complete. 2024-09-11T00:58:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.429+0000 7fd47d262640 1 Processor -- start 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd47d262640 1 -- start start 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd47d262640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd478101880 0x7fd4781992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd47d262640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd478104c60 0x7fd478199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd47d262640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781055e0 0x7fd47819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd47d262640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd478112e00 con 0x7fd4781055e0 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd47d262640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd478112c80 con 0x7fd478101880 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd47d262640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd478112f80 con 0x7fd478104c60 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd4777fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781055e0 0x7fd47819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd4777fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781055e0 0x7fd47819ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57336/0 (socket says 172.21.15.78:57336) 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd4777fe640 1 -- 172.21.15.78:0/1404561197 learned_addr learned my addr 172.21.15.78:0/1404561197 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd476ffd640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd478101880 0x7fd4781992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd46e7fc640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd478104c60 0x7fd478199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd4777fe640 1 -- 172.21.15.78:0/1404561197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd478104c60 msgr2=0x7fd478199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd4777fe640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd478104c60 0x7fd478199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd4777fe640 1 -- 172.21.15.78:0/1404561197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd478101880 msgr2=0x7fd4781992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd4777fe640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd478101880 0x7fd4781992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd4777fe640 1 -- 172.21.15.78:0/1404561197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47819f380 con 0x7fd4781055e0 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.430+0000 7fd46e7fc640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd478104c60 0x7fd478199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.431+0000 7fd4777fe640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781055e0 0x7fd47819ebb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd47000b970 tx=0x7fd47000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.431+0000 7fd474ff9640 1 -- 172.21.15.78:0/1404561197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470019070 con 0x7fd4781055e0 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.431+0000 7fd474ff9640 1 -- 172.21.15.78:0/1404561197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd470014440 con 0x7fd4781055e0 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.431+0000 7fd47d262640 1 -- 172.21.15.78:0/1404561197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47819f670 con 0x7fd4781055e0 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.431+0000 7fd474ff9640 1 -- 172.21.15.78:0/1404561197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4700075d0 con 0x7fd4781055e0 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.431+0000 7fd47d262640 1 -- 172.21.15.78:0/1404561197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4781a5db0 con 0x7fd4781055e0 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.432+0000 7fd47d262640 1 -- 172.21.15.78:0/1404561197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd478101c60 con 0x7fd4781055e0 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.433+0000 7fd474ff9640 1 -- 172.21.15.78:0/1404561197 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fd47000cb40 con 0x7fd4781055e0 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.434+0000 7fd474ff9640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4640b5330 0x7fd4640b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.434+0000 7fd476ffd640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4640b5330 0x7fd4640b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.434+0000 7fd474ff9640 1 -- 172.21.15.78:0/1404561197 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd4700e5960 con 0x7fd4781055e0 2024-09-11T00:58:36.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.434+0000 7fd476ffd640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4640b5330 0x7fd4640b77f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd468031440 tx=0x7fd468005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:36.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.437+0000 7fd474ff9640 1 -- 172.21.15.78:0/1404561197 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd4700ac120 con 0x7fd4781055e0 2024-09-11T00:58:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.614+0000 7fd47d262640 1 -- 172.21.15.78:0/1404561197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd47819a400 con 0x7fd4781055e0 2024-09-11T00:58:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.616+0000 7fd474ff9640 1 -- 172.21.15.78:0/1404561197 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22975 (secure 0 0 0) 0x7fd47819a400 con 0x7fd4781055e0 2024-09-11T00:58:36.618 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:36.618 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","created":"2024-09-11T00:48:27.652512+0000","modified":"2024-09-11T00:57:17.572848+0000","last_up_change":"2024-09-11T00:55:59.071507+0000","last_in_change":"2024-09-11T00:55:35.815150+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:52:10.229887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:56:49.939136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:56:51.703446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beefc6d9-f74c-4200-96f2-ff697ad0d7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6803","nonce":1424639053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6805","nonce":1424639053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6809","nonce":1424639053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6807","nonce":1424639053}]},"public_addr":"172.21.15.78:6803/1424639053","cluster_addr":"172.21.15.78:6805/1424639053","heartbeat_back_addr":"172.21.15.78:6809/1424639053","heartbeat_front_addr":"172.21.15.78:6807/1424639053","state":["exists","up"]},{"osd":1,"uuid":"d742e767-d673-4bb9-b624-8bc969f5ceb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6801","nonce":580939385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6803","nonce":580939385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6807","nonce":580939385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6805","nonce":580939385}]},"public_addr":"172.21.15.110:6801/580939385","cluster_addr":"172.21.15.110:6803/580939385","heartbeat_back_addr":"172.21.15.110:6807/580939385","heartbeat_front_addr":"172.21.15.110:6805/580939385","state":["exists","up"]},{"osd":2,"uuid":"b0106a5b-1465-4a50-96dd-934dbae49a4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6801","nonce":2983155763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6803","nonce":2983155763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6807","nonce":2983155763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6805","nonce":2983155763}]},"public_addr":"172.21.15.112:6801/2983155763","cluster_addr":"172.21.15.112:6803/2983155763","heartbeat_back_addr":"172.21.15.112:6807/2983155763","heartbeat_front_addr":"172.21.15.112:6805/2983155763","state":["exists","up"]},{"osd":3,"uuid":"3270362b-3d61-414a-a3d9-f2416ff8b870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6811","nonce":2906086464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6813","nonce":2906086464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6817","nonce":2906086464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6815","nonce":2906086464}]},"public_addr":"172.21.15.78:6811/2906086464","cluster_addr":"172.21.15.78:6813/2906086464","heartbeat_back_addr":"172.21.15.78:6817/2906086464","heartbeat_front_addr":"172.21.15.78:6815/2906086464","state":["exists","up"]},{"osd":4,"uuid":"e47746ec-9443-4004-abfa-17c1ed73b982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6809","nonce":477287435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6811","nonce":477287435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6815","nonce":477287435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6813","nonce":477287435}]},"public_addr":"172.21.15.110:6809/477287435","cluster_addr":"172.21.15.110:6811/477287435","heartbeat_back_addr":"172.21.15.110:6815/477287435","heartbeat_front_addr":"172.21.15.110:6813/477287435","state":["exists","up"]},{"osd":5,"uuid":"f8cedb01-310d-442d-a588-33197d32e208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6809","nonce":3557095096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6811","nonce":3557095096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6815","nonce":3557095096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6813","nonce":3557095096}]},"public_addr":"172.21.15.112:6809/3557095096","cluster_addr":"172.21.15.112:6811/3557095096","heartbeat_back_addr":"172.21.15.112:6815/3557095096","heartbeat_front_addr":"172.21.15.112:6813/3557095096","state":["exists","up"]},{"osd":6,"uuid":"d508a037-8f56-436b-b7bf-9a1c69f6d9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6819","nonce":1267910775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6821","nonce":1267910775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6825","nonce":1267910775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6823","nonce":1267910775}]},"public_addr":"172.21.15.78:6819/1267910775","cluster_addr":"172.21.15.78:6821/1267910775","heartbeat_back_addr":"172.21.15.78:6825/1267910775","heartbeat_front_addr":"172.21.15.78:6823/1267910775","state":["exists","up"]},{"osd":7,"uuid":"e8842047-09a6-48b8-8d0a-2d54e7f89063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6817","nonce":1556494594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6819","nonce":1556494594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6823","nonce":1556494594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6821","nonce":1556494594}]},"public_addr":"172.21.15.110:6817/1556494594","cluster_addr":"172.21.15.110:6819/1556494594","heartbeat_back_addr":"172.21.15.110:6823/1556494594","heartbeat_front_addr":"172.21.15.110:6821/1556494594","state":["exists","up"]},{"osd":8,"uuid":"a3367da8-375a-4a46-a952-867d4b72ae2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6817","nonce":2481321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6819","nonce":2481321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6823","nonce":2481321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6821","nonce":2481321}]},"public_addr":"172.21.15.112:6817/2481321","cluster_addr":"172.21.15.112:6819/2481321","heartbeat_back_addr":"172.21.15.112:6823/2481321","heartbeat_front_addr":"172.21.15.112:6821/2481321","state":["exists","up"]},{"osd":9,"uuid":"d1825764-cefc-495b-99da-8ba429df0937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6827","nonce":197048350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6829","nonce":197048350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6833","nonce":197048350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6831","nonce":197048350}]},"public_addr":"172.21.15.78:6827/197048350","cluster_addr":"172.21.15.78:6829/197048350","heartbeat_back_addr":"172.21.15.78:6833/197048350","heartbeat_front_addr":"172.21.15.78:6831/197048350","state":["exists","up"]},{"osd":10,"uuid":"e1cfff7a-8916-4845-a482-d79fa275da30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6825","nonce":2101123168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6827","nonce":2101123168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6831","nonce":2101123168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6829","nonce":2101123168}]},"public_addr":"172.21.15.110:6825/2101123168","cluster_addr":"172.21.15.110:6827/2101123168","heartbeat_back_addr":"172.21.15.110:6831/2101123168","heartbeat_front_addr":"172.21.15.110:6829/2101123168","state":["exists","up"]},{"osd":11,"uuid":"fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6825","nonce":1526310654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6827","nonce":1526310654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6831","nonce":1526310654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6829","nonce":1526310654}]},"public_addr":"172.21.15.112:6825/1526310654","cluster_addr":"172.21.15.112:6827/1526310654","heartbeat_back_addr":"172.21.15.112:6831/1526310654","heartbeat_front_addr":"172.21.15.112:6829/1526310654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:17.334199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:41.400872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:08.205144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:33.550527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:58.394116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:24.699607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:50.450193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:16.223363+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:40.947727+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:06.845451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:32.410035+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:57.744364+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/2390785703":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:0/1038951931":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/391908936":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2706514734":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/2357025243":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6800/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2795778473":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2747741101":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6801/1289343426":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/3558896007":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/438261739":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/1289343426":"2024-09-12T00:50:15.928079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:58:36.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.619+0000 7fd47d262640 1 -- 172.21.15.78:0/1404561197 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4640b5330 msgr2=0x7fd4640b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:36.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.619+0000 7fd47d262640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4640b5330 0x7fd4640b77f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd468031440 tx=0x7fd468005b50 comp rx=0 tx=0).stop 2024-09-11T00:58:36.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.619+0000 7fd47d262640 1 -- 172.21.15.78:0/1404561197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781055e0 msgr2=0x7fd47819ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:36.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.619+0000 7fd47d262640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781055e0 0x7fd47819ebb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd47000b970 tx=0x7fd47000be30 comp rx=0 tx=0).stop 2024-09-11T00:58:36.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.619+0000 7fd47d262640 1 -- 172.21.15.78:0/1404561197 shutdown_connections 2024-09-11T00:58:36.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.619+0000 7fd47d262640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4640b5330 0x7fd4640b77f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:36.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.619+0000 7fd47d262640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781055e0 0x7fd47819ebb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:36.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.619+0000 7fd47d262640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd478104c60 0x7fd478199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:36.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.619+0000 7fd47d262640 1 --2- 172.21.15.78:0/1404561197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd478101880 0x7fd4781992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:36.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.619+0000 7fd47d262640 1 -- 172.21.15.78:0/1404561197 >> 172.21.15.78:0/1404561197 conn(0x7fd478076e60 msgr2=0x7fd478102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:36.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.619+0000 7fd47d262640 1 -- 172.21.15.78:0/1404561197 shutdown_connections 2024-09-11T00:58:36.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:36.619+0000 7fd47d262640 1 -- 172.21.15.78:0/1404561197 wait complete. 2024-09-11T00:58:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:37 smithi078 ceph-mon[31315]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:58:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:37 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1404561197' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:58:37.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:37 smithi110 ceph-mon[34712]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:58:37.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:37 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1404561197' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:58:37.459 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T00:56:53.792170+0000', 'modified': '2024-09-11T00:57:04.821148+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24599, 'mds_1': 24580, 'mds_2': 24565, 'mds_3': 24563, 'mds_4': 14808}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14796': {'gid': 14796, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6839/3902249887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 3902249887}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 3902249887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14808': {'gid': 14808, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.78:6841/3713123708', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6840', 'nonce': 3713123708}, {'type': 'v1', 'addr': '172.21.15.78:6841', 'nonce': 3713123708}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6833/2579452587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6832', 'nonce': 2579452587}, {'type': 'v1', 'addr': '172.21.15.110:6833', 'nonce': 2579452587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6833/1732756287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6832', 'nonce': 1732756287}, {'type': 'v1', 'addr': '172.21.15.112:6833', 'nonce': 1732756287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6835/3549318748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 3549318748}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 3549318748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24553': {'gid': 24553, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6835/303956922', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 303956922}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 303956922}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24563': {'gid': 24563, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.110:6837/1873031527', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 1873031527}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 1873031527}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24565': {'gid': 24565, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.112:6837/854577265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6836', 'nonce': 854577265}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 854577265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24580': {'gid': 24580, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.112:6839/2141614771', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 2141614771}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 2141614771}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24599': {'gid': 24599, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.110:6839/3180315245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 3180315245}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 3180315245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24599, 'qdb_cluster': [14808, 24563, 24565, 24580, 24599]} 2024-09-11T00:58:37.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T00:58:37.460 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-11T00:58:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:37 smithi112 ceph-mon[35384]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:58:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:37 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1404561197' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:58:37.708 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:38.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.241+0000 7fd0967e5640 1 -- 172.21.15.78:0/3793004360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd09010d250 msgr2=0x7fd09010f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.241+0000 7fd0967e5640 1 --2- 172.21.15.78:0/3793004360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd09010d250 0x7fd09010f640 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd088009ca0 tx=0x7fd08802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.241+0000 7fd0967e5640 1 -- 172.21.15.78:0/3793004360 shutdown_connections 2024-09-11T00:58:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.241+0000 7fd0967e5640 1 --2- 172.21.15.78:0/3793004360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd09010d250 0x7fd09010f640 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.241+0000 7fd0967e5640 1 --2- 172.21.15.78:0/3793004360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd09010a920 0x7fd09010cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.241+0000 7fd0967e5640 1 --2- 172.21.15.78:0/3793004360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd090101810 0x7fd090101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.241+0000 7fd0967e5640 1 -- 172.21.15.78:0/3793004360 >> 172.21.15.78:0/3793004360 conn(0x7fd090076f50 msgr2=0x7fd090077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.241+0000 7fd0967e5640 1 -- 172.21.15.78:0/3793004360 shutdown_connections 2024-09-11T00:58:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.241+0000 7fd0967e5640 1 -- 172.21.15.78:0/3793004360 wait complete. 2024-09-11T00:58:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd0967e5640 1 Processor -- start 2024-09-11T00:58:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd0967e5640 1 -- start start 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd0967e5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd090101810 0x7fd0901027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd0967e5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09010a920 0x7fd090102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd0967e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd09010d250 0x7fd090103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd0967e5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd090112db0 con 0x7fd090101810 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd0967e5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd090112c30 con 0x7fd09010a920 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd0967e5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd090112f30 con 0x7fd09010d250 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd08ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd090101810 0x7fd0901027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd08ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd090101810 0x7fd0901027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57362/0 (socket says 172.21.15.78:57362) 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd08ffff640 1 -- 172.21.15.78:0/3178540421 learned_addr learned my addr 172.21.15.78:0/3178540421 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd087fff640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09010a920 0x7fd090102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd08ffff640 1 -- 172.21.15.78:0/3178540421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd09010d250 msgr2=0x7fd090103260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd08ffff640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd09010d250 0x7fd090103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd08ffff640 1 -- 172.21.15.78:0/3178540421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09010a920 msgr2=0x7fd090102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd08ffff640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09010a920 0x7fd090102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd08ffff640 1 -- 172.21.15.78:0/3178540421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0901a7700 con 0x7fd090101810 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.242+0000 7fd08ffff640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd090101810 0x7fd0901027e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd08000d950 tx=0x7fd08000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.243+0000 7fd08dffb640 1 -- 172.21.15.78:0/3178540421 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080017070 con 0x7fd090101810 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.243+0000 7fd0967e5640 1 -- 172.21.15.78:0/3178540421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0901a79f0 con 0x7fd090101810 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.243+0000 7fd0967e5640 1 -- 172.21.15.78:0/3178540421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0901a7f80 con 0x7fd090101810 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.243+0000 7fd08dffb640 1 -- 172.21.15.78:0/3178540421 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd080009c30 con 0x7fd090101810 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.243+0000 7fd08dffb640 1 -- 172.21.15.78:0/3178540421 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080020060 con 0x7fd090101810 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.245+0000 7fd098bba740 1 Processor -- start 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.245+0000 7fd098bba740 1 -- start start 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.245+0000 7fd08dffb640 1 -- 172.21.15.78:0/3178540421 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fd08000f620 con 0x7fd090101810 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.245+0000 7fd098bba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 0x5622ffbc2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.245+0000 7fd098bba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.245+0000 7fd098bba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbce5e0 0x5622ffbd09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.245+0000 7fd098bba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5622ffb39c10 con 0x5622ffbc1d30 2024-09-11T00:58:38.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.245+0000 7fd098bba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5622ffb29040 con 0x5622ffbc2650 2024-09-11T00:58:38.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.245+0000 7fd098bba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5622ffb2ce50 con 0x5622ffbce5e0 2024-09-11T00:58:38.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd094d82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 0x5622ffbc2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd094d82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 0x5622ffbc2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57370/0 (socket says 172.21.15.78:57370) 2024-09-11T00:58:38.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd094d82640 1 -- 172.21.15.78:0/753604080 learned_addr learned my addr 172.21.15.78:0/753604080 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:38.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd087fff640 1 --2- 172.21.15.78:0/753604080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbce5e0 0x5622ffbd09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd08ffff640 1 --2- 172.21.15.78:0/753604080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd094d82640 1 -- 172.21.15.78:0/753604080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbce5e0 msgr2=0x5622ffbd09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:38.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd094d82640 1 --2- 172.21.15.78:0/753604080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbce5e0 0x5622ffbd09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd094d82640 1 -- 172.21.15.78:0/753604080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 msgr2=0x5622ffbce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd094d82640 1 --2- 172.21.15.78:0/753604080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd094d82640 1 -- 172.21.15.78:0/753604080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ff887540 con 0x5622ffbc1d30 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd094d82640 1 --2- 172.21.15.78:0/753604080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 0x5622ffbc2110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd088005fd0 tx=0x7fd0880093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fe6974c9826aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd0867fc640 1 -- 172.21.15.78:0/753604080 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088031cf0 con 0x5622ffbc1d30 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd0867fc640 1 -- 172.21.15.78:0/753604080 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd08803f070 con 0x5622ffbc1d30 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.246+0000 7fd0867fc640 1 -- 172.21.15.78:0/753604080 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0880042a0 con 0x5622ffbc1d30 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd08dffb640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd07c0b5330 0x7fd07c0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd08ffff640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd07c0b5330 0x7fd07c0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd098bba740 1 -- 172.21.15.78:0/753604080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 msgr2=0x5622ffbc2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd098bba740 1 --2- 172.21.15.78:0/753604080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 0x5622ffbc2110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd088005fd0 tx=0x7fd0880093c0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd098bba740 1 -- 172.21.15.78:0/753604080 shutdown_connections 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd098bba740 1 --2- 172.21.15.78:0/753604080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbce5e0 0x5622ffbd09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd098bba740 1 --2- 172.21.15.78:0/753604080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd098bba740 1 --2- 172.21.15.78:0/753604080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 0x5622ffbc2110 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd098bba740 1 -- 172.21.15.78:0/753604080 >> 172.21.15.78:0/753604080 conn(0x5622ffbbfb60 msgr2=0x5622ffbc0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd08dffb640 1 -- 172.21.15.78:0/3178540421 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd0800e57f0 con 0x7fd090101810 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd098bba740 1 -- 172.21.15.78:0/753604080 shutdown_connections 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd098bba740 1 -- 172.21.15.78:0/753604080 wait complete. 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd08ffff640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd07c0b5330 0x7fd07c0b77f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd0800e35b0 tx=0x7fd08000aaf0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.247+0000 7fd098bba740 1 -- start start 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd098bba740 1 Processor -- start 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd098bba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 0x5622ffbcb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd098bba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbcbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd098bba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbce5e0 0x5622ffbcc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd098bba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5622ff98a5f0 con 0x5622ffbc1d30 2024-09-11T00:58:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd098bba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5622ffac4010 con 0x5622ffbc2650 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd098bba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5622ffbd1230 con 0x5622ffbce5e0 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd098bba740 10 client.0 _ll_register_callbacks cb 0x5622ffb23340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd094d82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 0x5622ffbcb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd094d82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 0x5622ffbcb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57374/0 (socket says 172.21.15.78:57374) 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd094d82640 1 -- 172.21.15.78:0/1880001031 learned_addr learned my addr 172.21.15.78:0/1880001031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd087fff640 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbcbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd094d82640 1 -- 172.21.15.78:0/1880001031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbce5e0 msgr2=0x5622ffbcc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd094d82640 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbce5e0 0x5622ffbcc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd094d82640 1 -- 172.21.15.78:0/1880001031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 msgr2=0x5622ffbcbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd094d82640 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbcbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd094d82640 1 -- 172.21.15.78:0/1880001031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622ffaf1a00 con 0x5622ffbc1d30 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd087fff640 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbcbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd094d82640 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 0x5622ffbcb5a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd08802ade0 tx=0x7fd088003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75afd9c70147897c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.248+0000 7fd06f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd06f7fe640 1 -- 172.21.15.78:0/1880001031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0880318e0 con 0x5622ffbc1d30 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd06f7fe640 1 -- 172.21.15.78:0/1880001031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd08802f660 con 0x5622ffbc1d30 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd098bba740 1 -- 172.21.15.78:0/1880001031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd08803f070 con 0x5622ffbc1d30 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd06f7fe640 1 -- 172.21.15.78:0/1880001031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd08802f860 con 0x5622ffbc1d30 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd098bba740 10 client.15321 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd098bba740 1 -- 172.21.15.78:0/1880001031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622ffbdce80 con 0x5622ffbc1d30 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd06f7fe640 1 -- 172.21.15.78:0/1880001031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd08803f070 con 0x5622ffbc1d30 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd06f7fe640 1 -- 172.21.15.78:0/1880001031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd088030d50 con 0x5622ffbc1d30 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd098bba740 10 client.15321 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd098bba740 10 client.15321 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd098bba740 20 client.15321 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd098bba740 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5622ffbdf040 0x5622ffbe04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd098bba740 4 client.15321 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:58:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd098bba740 1 -- 172.21.15.78:0/1880001031 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622ffb3bb60 con 0x5622ffbdf040 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.249+0000 7fd087fff640 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5622ffbdf040 0x5622ffbe04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd06f7fe640 10 client.15321 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd087fff640 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5622ffbdf040 0x5622ffbe04b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1dfbb2d47744cf55 server_cookie=990239735e0d1e61 in_seq=0 out_seq=0 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd06f7fe640 1 -- 172.21.15.78:0/1880001031 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5622ffb3bb60 con 0x5622ffbdf040 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd06f7fe640 10 client.15321 handle_command_reply: tid=0 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd098bba740 1 client.15321 shutdown 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd098bba740 1 -- 172.21.15.78:0/1880001031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 msgr2=0x5622ffbcb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd098bba740 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 0x5622ffbcb5a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd08802ade0 tx=0x7fd088003ba0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd098bba740 1 -- 172.21.15.78:0/1880001031 shutdown_connections 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd098bba740 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5622ffbdf040 0x5622ffbe04b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd098bba740 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbce5e0 0x5622ffbcc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd098bba740 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbcbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd098bba740 1 --2- 172.21.15.78:0/1880001031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbc1d30 0x5622ffbcb5a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd098bba740 1 -- 172.21.15.78:0/1880001031 >> 172.21.15.78:0/1880001031 conn(0x5622ffbbfb60 msgr2=0x5622ffbc5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.250+0000 7fd06f7fe640 0 client.15321 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T00:58:38.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.251+0000 7fd098bba740 1 -- 172.21.15.78:0/1880001031 shutdown_connections 2024-09-11T00:58:38.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.251+0000 7fd098bba740 1 -- 172.21.15.78:0/1880001031 wait complete. 2024-09-11T00:58:38.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.251+0000 7fd098bba740 20 client.15321 trim_cache size 0 max 16384 2024-09-11T00:58:38.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd098bba740 1 Processor -- start 2024-09-11T00:58:38.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd098bba740 1 -- start start 2024-09-11T00:58:38.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd098bba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbc1d30 0x5622ffbdfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd098bba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbcaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd098bba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 0x5622ffbc0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd098bba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5622ffbde7f0 con 0x5622ffbce5e0 2024-09-11T00:58:38.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd098bba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5622ffbdeec0 con 0x5622ffbc2650 2024-09-11T00:58:38.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd098bba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5622ffbdf590 con 0x5622ffbc1d30 2024-09-11T00:58:38.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd094d82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 0x5622ffbc0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd094d82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 0x5622ffbc0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57384/0 (socket says 172.21.15.78:57384) 2024-09-11T00:58:38.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd094d82640 1 -- 172.21.15.78:0/2576328468 learned_addr learned my addr 172.21.15.78:0/2576328468 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:38.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd087fff640 1 --2- 172.21.15.78:0/2576328468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbcaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd094d82640 1 --2- 172.21.15.78:0/2576328468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbc1d30 0x5622ffbdfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd094d82640 1 -- 172.21.15.78:0/2576328468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbc1d30 msgr2=0x5622ffbdfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:38.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd094d82640 1 --2- 172.21.15.78:0/2576328468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbc1d30 0x5622ffbdfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd094d82640 1 -- 172.21.15.78:0/2576328468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 msgr2=0x5622ffbcaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:38.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.269+0000 7fd094d82640 1 --2- 172.21.15.78:0/2576328468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbcaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.270+0000 7fd094d82640 1 -- 172.21.15.78:0/2576328468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ffbc05e0 con 0x5622ffbce5e0 2024-09-11T00:58:38.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.270+0000 7fd094d82640 1 --2- 172.21.15.78:0/2576328468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 0x5622ffbc0930 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd08802f780 tx=0x7fd08802bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef55816e6ff6bd13 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:38.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.270+0000 7fd085ffb640 1 -- 172.21.15.78:0/2576328468 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088003c40 con 0x5622ffbce5e0 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.270+0000 7fd085ffb640 1 -- 172.21.15.78:0/2576328468 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd08803f040 con 0x5622ffbce5e0 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.270+0000 7fd085ffb640 1 -- 172.21.15.78:0/2576328468 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088004580 con 0x5622ffbce5e0 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.271+0000 7fd098bba740 1 -- 172.21.15.78:0/2576328468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 msgr2=0x5622ffbc0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.271+0000 7fd098bba740 1 --2- 172.21.15.78:0/2576328468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 0x5622ffbc0930 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd08802f780 tx=0x7fd08802bfd0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.271+0000 7fd098bba740 1 -- 172.21.15.78:0/2576328468 shutdown_connections 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.271+0000 7fd098bba740 1 --2- 172.21.15.78:0/2576328468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 0x5622ffbc0930 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.271+0000 7fd098bba740 1 --2- 172.21.15.78:0/2576328468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc2650 0x5622ffbcaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.271+0000 7fd098bba740 1 --2- 172.21.15.78:0/2576328468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbc1d30 0x5622ffbdfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.271+0000 7fd098bba740 1 -- 172.21.15.78:0/2576328468 >> 172.21.15.78:0/2576328468 conn(0x5622ffbbfb60 msgr2=0x5622ffbc9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.271+0000 7fd098bba740 1 -- 172.21.15.78:0/2576328468 shutdown_connections 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.271+0000 7fd098bba740 1 -- 172.21.15.78:0/2576328468 wait complete. 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.271+0000 7fd098bba740 1 -- start start 2024-09-11T00:58:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.271+0000 7fd098bba740 1 Processor -- start 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd098bba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc1d30 0x5622ffbe91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd098bba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbc2650 0x5622ffbe5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd098bba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 0x5622ffbe63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd098bba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5622ffbe0330 con 0x5622ffbce5e0 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd098bba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5622ffbdfc60 con 0x5622ffbc1d30 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd098bba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5622ffbe0a00 con 0x5622ffbc2650 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd098bba740 10 client.0 _ll_register_callbacks cb 0x7fd088005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd094d82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 0x5622ffbe63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd094d82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 0x5622ffbe63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57392/0 (socket says 172.21.15.78:57392) 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd094d82640 1 -- 172.21.15.78:0/3981733403 learned_addr learned my addr 172.21.15.78:0/3981733403 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd094d82640 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc1d30 0x5622ffbe91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd087fff640 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbc2650 0x5622ffbe5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd094d82640 1 -- 172.21.15.78:0/3981733403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbc2650 msgr2=0x5622ffbe5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd094d82640 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbc2650 0x5622ffbe5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd094d82640 1 -- 172.21.15.78:0/3981733403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc1d30 msgr2=0x5622ffbe91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd094d82640 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc1d30 0x5622ffbe91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd094d82640 1 -- 172.21.15.78:0/3981733403 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622ffbe4bf0 con 0x5622ffbce5e0 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd094d82640 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 0x5622ffbe63b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd088000f80 tx=0x7fd088005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa139b4a525c333 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd06f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd06f7fe640 1 -- 172.21.15.78:0/3981733403 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088004820 con 0x5622ffbce5e0 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd06f7fe640 1 -- 172.21.15.78:0/3981733403 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd08802fa00 con 0x5622ffbce5e0 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd098bba740 1 -- 172.21.15.78:0/3981733403 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd08803f040 con 0x5622ffbce5e0 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd06f7fe640 1 -- 172.21.15.78:0/3981733403 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd08802fc00 con 0x5622ffbce5e0 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd098bba740 10 client.15327 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.272+0000 7fd098bba740 1 -- 172.21.15.78:0/3981733403 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622ffc4bc20 con 0x5622ffbce5e0 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd06f7fe640 1 -- 172.21.15.78:0/3981733403 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd08803f040 con 0x5622ffbce5e0 2024-09-11T00:58:38.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd06f7fe640 1 -- 172.21.15.78:0/3981733403 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd088044470 con 0x5622ffbce5e0 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd098bba740 10 client.15327 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd098bba740 10 client.15327 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd098bba740 20 client.15327 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd098bba740 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5622ffc50500 0x5622ffc4f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd098bba740 4 client.15327 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd098bba740 1 -- 172.21.15.78:0/3981733403 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "status"}) -- 0x5622ffb37610 con 0x5622ffc50500 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd087fff640 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5622ffc50500 0x5622ffc4f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd06f7fe640 10 client.15327 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd087fff640 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5622ffc50500 0x5622ffc4f210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=67bc29234573c81 server_cookie=992ce4c0c00b4b74 in_seq=0 out_seq=0 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd06f7fe640 1 -- 172.21.15.78:0/3981733403 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5622ffb37610 con 0x5622ffc50500 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd06f7fe640 10 client.15327 handle_command_reply: tid=0 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.273+0000 7fd098bba740 1 client.15327 shutdown 2024-09-11T00:58:38.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd098bba740 1 -- 172.21.15.78:0/3981733403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 msgr2=0x5622ffbe63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd098bba740 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 0x5622ffbe63b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd088000f80 tx=0x7fd088005df0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd098bba740 1 -- 172.21.15.78:0/3981733403 shutdown_connections 2024-09-11T00:58:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd098bba740 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5622ffc50500 0x5622ffc4f210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd098bba740 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffbce5e0 0x5622ffbe63b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd098bba740 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622ffbc2650 0x5622ffbe5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd098bba740 1 --2- 172.21.15.78:0/3981733403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622ffbc1d30 0x5622ffbe91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd098bba740 1 -- 172.21.15.78:0/3981733403 >> 172.21.15.78:0/3981733403 conn(0x5622ffbbfb60 msgr2=0x5622ffbd9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd06f7fe640 0 client.15327 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T00:58:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd098bba740 1 -- 172.21.15.78:0/3981733403 shutdown_connections 2024-09-11T00:58:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd098bba740 1 -- 172.21.15.78:0/3981733403 wait complete. 2024-09-11T00:58:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd098bba740 20 client.15327 trim_cache size 0 max 16384 2024-09-11T00:58:38.286 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:58:38.286 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:58:38.286 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 4, 2024-09-11T00:58:38.286 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 14808, 2024-09-11T00:58:38.286 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T00:58:38.286 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T00:58:38.286 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T00:58:38.286 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 95.044649853999999, 2024-09-11T00:58:38.286 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-11T00:58:38.287 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-09-11T00:58:38.287 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-09-11T00:58:38.287 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 118.882473563 2024-09-11T00:58:38.287 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:58:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd0967e5640 1 -- 172.21.15.78:0/3178540421 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd07c0b5330 msgr2=0x7fd07c0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd0967e5640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd07c0b5330 0x7fd07c0b77f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd0800e35b0 tx=0x7fd08000aaf0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd0967e5640 1 -- 172.21.15.78:0/3178540421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd090101810 msgr2=0x7fd0901027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd0967e5640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd090101810 0x7fd0901027e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd08000d950 tx=0x7fd08000de10 comp rx=0 tx=0).stop 2024-09-11T00:58:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.274+0000 7fd0967e5640 1 -- 172.21.15.78:0/3178540421 shutdown_connections 2024-09-11T00:58:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.275+0000 7fd0967e5640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd07c0b5330 0x7fd07c0b77f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.275+0000 7fd0967e5640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd09010d250 0x7fd090103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.275+0000 7fd0967e5640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09010a920 0x7fd090102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.275+0000 7fd0967e5640 1 --2- 172.21.15.78:0/3178540421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd090101810 0x7fd0901027e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.275+0000 7fd0967e5640 1 -- 172.21.15.78:0/3178540421 >> 172.21.15.78:0/3178540421 conn(0x7fd090076f50 msgr2=0x7fd0901005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.275+0000 7fd0967e5640 1 -- 172.21.15.78:0/3178540421 shutdown_connections 2024-09-11T00:58:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:38.275+0000 7fd0967e5640 1 -- 172.21.15.78:0/3178540421 wait complete. 2024-09-11T00:58:38.997 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T00:58:39.245 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:39.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:39 smithi078 ceph-mon[31315]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:39 smithi110 ceph-mon[34712]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:39 smithi112 ceph-mon[35384]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.775+0000 7f8fe7344640 1 -- 172.21.15.78:0/3956475336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe0108020 msgr2=0x7f8fe010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.775+0000 7f8fe7344640 1 --2- 172.21.15.78:0/3956475336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe0108020 0x7f8fe010e820 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009ca0 tx=0x7f8fd802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 -- 172.21.15.78:0/3956475336 shutdown_connections 2024-09-11T00:58:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 --2- 172.21.15.78:0/3956475336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe0108020 0x7f8fe010e820 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 --2- 172.21.15.78:0/3956475336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe01076a0 0x7f8fe0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 --2- 172.21.15.78:0/3956475336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8fe0101c90 0x7f8fe0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 -- 172.21.15.78:0/3956475336 >> 172.21.15.78:0/3956475336 conn(0x7f8fe00fd700 msgr2=0x7f8fe00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 -- 172.21.15.78:0/3956475336 shutdown_connections 2024-09-11T00:58:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 -- 172.21.15.78:0/3956475336 wait complete. 2024-09-11T00:58:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 Processor -- start 2024-09-11T00:58:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 -- start start 2024-09-11T00:58:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe0101c90 0x7f8fe0199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe01076a0 0x7f8fe0199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8fe0108020 0x7f8fe019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0111ce0 con 0x7f8fe0101c90 2024-09-11T00:58:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0111b60 con 0x7f8fe0108020 2024-09-11T00:58:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe7344640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0111e60 con 0x7f8fe01076a0 2024-09-11T00:58:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.776+0000 7f8fe50e0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe0101c90 0x7f8fe0199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fe50e0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe0101c90 0x7f8fe0199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57408/0 (socket says 172.21.15.78:57408) 2024-09-11T00:58:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fe50e0640 1 -- 172.21.15.78:0/2035209942 learned_addr learned my addr 172.21.15.78:0/2035209942 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fe48df640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe01076a0 0x7f8fe0199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fe50e0640 1 -- 172.21.15.78:0/2035209942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe01076a0 msgr2=0x7f8fe0199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fe50e0640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe01076a0 0x7f8fe0199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fe50e0640 1 -- 172.21.15.78:0/2035209942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8fe0108020 msgr2=0x7f8fe019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fe50e0640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8fe0108020 0x7f8fe019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fe50e0640 1 -- 172.21.15.78:0/2035209942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe019f3c0 con 0x7f8fe0101c90 2024-09-11T00:58:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fe48df640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe01076a0 0x7f8fe0199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fe50e0640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe0101c90 0x7f8fe0199310 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00d950 tx=0x7f8fdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fd67fc640 1 -- 172.21.15.78:0/2035209942 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc017070 con 0x7f8fe0101c90 2024-09-11T00:58:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fd67fc640 1 -- 172.21.15.78:0/2035209942 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fdc009c30 con 0x7f8fe0101c90 2024-09-11T00:58:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fe7344640 1 -- 172.21.15.78:0/2035209942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe019f6b0 con 0x7f8fe0101c90 2024-09-11T00:58:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fd67fc640 1 -- 172.21.15.78:0/2035209942 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc00f620 con 0x7f8fe0101c90 2024-09-11T00:58:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.777+0000 7f8fe7344640 1 -- 172.21.15.78:0/2035209942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe01a5df0 con 0x7f8fe0101c90 2024-09-11T00:58:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.780+0000 7f8fe9719740 1 Processor -- start 2024-09-11T00:58:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.780+0000 7f8fe9719740 1 -- start start 2024-09-11T00:58:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.780+0000 7f8fd67fc640 1 -- 172.21.15.78:0/2035209942 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f8fdc010460 con 0x7f8fe0101c90 2024-09-11T00:58:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe9719740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 0x5556e3ccc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe9719740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3ccc650 0x5556e3cdc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe9719740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3cdc9f0 0x5556e3cdede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe9719740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5556e3c43c10 con 0x5556e3ccbd30 2024-09-11T00:58:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe9719740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5556e3c33040 con 0x5556e3cdc9f0 2024-09-11T00:58:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe9719740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5556e3c36e50 con 0x5556e3ccc650 2024-09-11T00:58:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe58e1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 0x5556e3ccc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe58e1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 0x5556e3ccc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57412/0 (socket says 172.21.15.78:57412) 2024-09-11T00:58:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe58e1640 1 -- 172.21.15.78:0/859718166 learned_addr learned my addr 172.21.15.78:0/859718166 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe50e0640 1 --2- 172.21.15.78:0/859718166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3ccc650 0x5556e3cdc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe48df640 1 --2- 172.21.15.78:0/859718166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3cdc9f0 0x5556e3cdede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe58e1640 1 -- 172.21.15.78:0/859718166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3ccc650 msgr2=0x5556e3cdc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe58e1640 1 --2- 172.21.15.78:0/859718166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3ccc650 0x5556e3cdc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe58e1640 1 -- 172.21.15.78:0/859718166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3cdc9f0 msgr2=0x5556e3cdede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe58e1640 1 --2- 172.21.15.78:0/859718166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3cdc9f0 0x5556e3cdede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe58e1640 1 -- 172.21.15.78:0/859718166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556e3991540 con 0x5556e3ccbd30 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe48df640 1 --2- 172.21.15.78:0/859718166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3cdc9f0 0x5556e3cdede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fe58e1640 1 --2- 172.21.15.78:0/859718166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 0x5556e3ccc110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80097b0 tx=0x7f8fd8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49062dd2453d0634 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fd67fc640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8fcc0b5330 0x7f8fcc0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.781+0000 7f8fcb7fe640 1 -- 172.21.15.78:0/859718166 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8031cf0 con 0x5556e3ccbd30 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.782+0000 7f8fcb7fe640 1 -- 172.21.15.78:0/859718166 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd803f070 con 0x5556e3ccbd30 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.782+0000 7f8fcb7fe640 1 -- 172.21.15.78:0/859718166 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8004040 con 0x5556e3ccbd30 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.782+0000 7f8fe50e0640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8fcc0b5330 0x7f8fcc0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.782+0000 7f8fd67fc640 1 -- 172.21.15.78:0/2035209942 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f8fdc0e5a30 con 0x7f8fe0101c90 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.782+0000 7f8fe9719740 1 -- 172.21.15.78:0/859718166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 msgr2=0x5556e3ccc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.782+0000 7f8fe50e0640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8fcc0b5330 0x7f8fcc0b77f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0e37f0 tx=0x7f8fdc093040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.782+0000 7f8fe9719740 1 --2- 172.21.15.78:0/859718166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 0x5556e3ccc110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80097b0 tx=0x7f8fd8005fa0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.782+0000 7f8fe9719740 1 -- 172.21.15.78:0/859718166 shutdown_connections 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.782+0000 7f8fe9719740 1 --2- 172.21.15.78:0/859718166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3cdc9f0 0x5556e3cdede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.782+0000 7f8fe9719740 1 --2- 172.21.15.78:0/859718166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3ccc650 0x5556e3cdc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.782+0000 7f8fe9719740 1 --2- 172.21.15.78:0/859718166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 0x5556e3ccc110 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.782+0000 7f8fe9719740 1 -- 172.21.15.78:0/859718166 >> 172.21.15.78:0/859718166 conn(0x5556e3cc9b60 msgr2=0x5556e3ccad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe9719740 1 -- 172.21.15.78:0/859718166 shutdown_connections 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe9719740 1 -- 172.21.15.78:0/859718166 wait complete. 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe9719740 1 -- start start 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe9719740 1 Processor -- start 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe9719740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccbd30 0x5556e3cd9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe9719740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3ccc650 0x5556e3cd9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe9719740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3cdc9f0 0x5556e3cda410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe9719740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5556e3a945f0 con 0x5556e3cdc9f0 2024-09-11T00:58:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe9719740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5556e3bce010 con 0x5556e3ccbd30 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe9719740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5556e3cdf640 con 0x5556e3ccc650 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe9719740 10 client.0 _ll_register_callbacks cb 0x5556e3c2d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe58e1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3cdc9f0 0x5556e3cda410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.783+0000 7f8fe58e1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3cdc9f0 0x5556e3cda410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57426/0 (socket says 172.21.15.78:57426) 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fe58e1640 1 -- 172.21.15.78:0/3013520548 learned_addr learned my addr 172.21.15.78:0/3013520548 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fe48df640 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3ccc650 0x5556e3cd9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fe58e1640 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccbd30 0x5556e3cd9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fe58e1640 1 -- 172.21.15.78:0/3013520548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3ccc650 msgr2=0x5556e3cd9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fe58e1640 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3ccc650 0x5556e3cd9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fe58e1640 1 -- 172.21.15.78:0/3013520548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccbd30 msgr2=0x5556e3cd9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fe58e1640 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccbd30 0x5556e3cd9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fe58e1640 1 -- 172.21.15.78:0/3013520548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556e3bfba00 con 0x5556e3cdc9f0 2024-09-11T00:58:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fe58e1640 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3cdc9f0 0x5556e3cda410 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8005cc0 tx=0x7f8fd8005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f4bddd81eaa46e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fc8ff9640 1 -- 172.21.15.78:0/3013520548 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8003a70 con 0x5556e3cdc9f0 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fe9719740 1 -- 172.21.15.78:0/3013520548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fd803f070 con 0x5556e3cdc9f0 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fc8ff9640 1 -- 172.21.15.78:0/3013520548 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f8fd8031660 con 0x5556e3cdc9f0 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fc8ff9640 1 -- 172.21.15.78:0/3013520548 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8fd8031860 con 0x5556e3cdc9f0 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fe9719740 10 client.15339 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fe9719740 1 -- 172.21.15.78:0/3013520548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556e3ceb200 con 0x5556e3cdc9f0 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.784+0000 7f8fc8ff9640 1 -- 172.21.15.78:0/3013520548 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd803f070 con 0x5556e3cdc9f0 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.785+0000 7f8fc8ff9640 1 -- 172.21.15.78:0/3013520548 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8fd802fa30 con 0x5556e3cdc9f0 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.785+0000 7f8fe9719740 10 client.15339 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.785+0000 7f8fe9719740 10 client.15339 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.785+0000 7f8fe9719740 20 client.15339 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.785+0000 7f8fe9719740 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5556e3cec550 0x5556e3cee700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.785+0000 7f8fe9719740 4 client.15339 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.785+0000 7f8fe9719740 1 -- 172.21.15.78:0/3013520548 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556e3c45b60 con 0x5556e3cec550 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.785+0000 7f8fe48df640 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5556e3cec550 0x5556e3cee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.785+0000 7f8fe48df640 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5556e3cec550 0x5556e3cee700 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1eec9195fee238bb server_cookie=2eb3d5878640c2df in_seq=0 out_seq=0 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.785+0000 7f8fc8ff9640 10 client.15339 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.786+0000 7f8fc8ff9640 1 -- 172.21.15.78:0/3013520548 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5556e3c45b60 con 0x5556e3cec550 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.786+0000 7f8fc8ff9640 10 client.15339 handle_command_reply: tid=0 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.786+0000 7f8fe9719740 1 client.15339 shutdown 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.786+0000 7f8fe9719740 1 -- 172.21.15.78:0/3013520548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3cdc9f0 msgr2=0x5556e3cda410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.786+0000 7f8fe9719740 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3cdc9f0 0x5556e3cda410 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8005cc0 tx=0x7f8fd8005c30 comp rx=0 tx=0).stop 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.786+0000 7f8fe9719740 1 -- 172.21.15.78:0/3013520548 shutdown_connections 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.786+0000 7f8fe9719740 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5556e3cec550 0x5556e3cee700 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.786+0000 7f8fe9719740 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3cdc9f0 0x5556e3cda410 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.786+0000 7f8fe9719740 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3ccc650 0x5556e3cd9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.786+0000 7f8fe9719740 1 --2- 172.21.15.78:0/3013520548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccbd30 0x5556e3cd9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.786+0000 7f8fe9719740 1 -- 172.21.15.78:0/3013520548 >> 172.21.15.78:0/3013520548 conn(0x5556e3cc9b60 msgr2=0x5556e3cc8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.786+0000 7f8fc8ff9640 0 client.15339 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T00:58:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.787+0000 7f8fe9719740 1 -- 172.21.15.78:0/3013520548 shutdown_connections 2024-09-11T00:58:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.787+0000 7f8fe9719740 1 -- 172.21.15.78:0/3013520548 wait complete. 2024-09-11T00:58:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.787+0000 7f8fe9719740 20 client.15339 trim_cache size 0 max 16384 2024-09-11T00:58:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.804+0000 7f8fe9719740 1 Processor -- start 2024-09-11T00:58:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.804+0000 7f8fe9719740 1 -- start start 2024-09-11T00:58:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.804+0000 7f8fe9719740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccbd30 0x5556e3ccb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.804+0000 7f8fe9719740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccc650 0x5556e3cd6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe9719740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3cdc9f0 0x5556e3cd7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe9719740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5556e3ceca40 con 0x5556e3ccc650 2024-09-11T00:58:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe9719740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5556e3ced110 con 0x5556e3ccbd30 2024-09-11T00:58:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe9719740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5556e3ced7e0 con 0x5556e3cdc9f0 2024-09-11T00:58:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe48df640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccc650 0x5556e3cd6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe48df640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccc650 0x5556e3cd6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57440/0 (socket says 172.21.15.78:57440) 2024-09-11T00:58:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe48df640 1 -- 172.21.15.78:0/3415512195 learned_addr learned my addr 172.21.15.78:0/3415512195 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe58e1640 1 --2- 172.21.15.78:0/3415512195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3cdc9f0 0x5556e3cd7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe58e1640 1 --2- 172.21.15.78:0/3415512195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccbd30 0x5556e3ccb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe48df640 1 -- 172.21.15.78:0/3415512195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3cdc9f0 msgr2=0x5556e3cd7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe48df640 1 --2- 172.21.15.78:0/3415512195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3cdc9f0 0x5556e3cd7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe48df640 1 -- 172.21.15.78:0/3415512195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccbd30 msgr2=0x5556e3ccb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe48df640 1 --2- 172.21.15.78:0/3415512195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccbd30 0x5556e3ccb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe48df640 1 -- 172.21.15.78:0/3415512195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556e3ccb570 con 0x5556e3ccc650 2024-09-11T00:58:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fe48df640 1 --2- 172.21.15.78:0/3415512195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccc650 0x5556e3cd6fe0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0007a10 tx=0x7f8fd0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df758543bb2dc75c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fcaffd640 1 -- 172.21.15.78:0/3415512195 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0016a60 con 0x5556e3ccc650 2024-09-11T00:58:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fcaffd640 1 -- 172.21.15.78:0/3415512195 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd001f030 con 0x5556e3ccc650 2024-09-11T00:58:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.805+0000 7f8fcaffd640 1 -- 172.21.15.78:0/3415512195 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd00143b0 con 0x5556e3ccc650 2024-09-11T00:58:39.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.806+0000 7f8fe9719740 1 -- 172.21.15.78:0/3415512195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccc650 msgr2=0x5556e3cd6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:39.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.806+0000 7f8fe9719740 1 --2- 172.21.15.78:0/3415512195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccc650 0x5556e3cd6fe0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0007a10 tx=0x7f8fd0008040 comp rx=0 tx=0).stop 2024-09-11T00:58:39.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.806+0000 7f8fe9719740 1 -- 172.21.15.78:0/3415512195 shutdown_connections 2024-09-11T00:58:39.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.806+0000 7f8fe9719740 1 --2- 172.21.15.78:0/3415512195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3cdc9f0 0x5556e3cd7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.806+0000 7f8fe9719740 1 --2- 172.21.15.78:0/3415512195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccc650 0x5556e3cd6fe0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.806+0000 7f8fe9719740 1 --2- 172.21.15.78:0/3415512195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccbd30 0x5556e3ccb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.806+0000 7f8fe9719740 1 -- 172.21.15.78:0/3415512195 >> 172.21.15.78:0/3415512195 conn(0x5556e3cc9b60 msgr2=0x5556e3cca920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:39.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.806+0000 7f8fe9719740 1 -- 172.21.15.78:0/3415512195 shutdown_connections 2024-09-11T00:58:39.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.806+0000 7f8fe9719740 1 -- 172.21.15.78:0/3415512195 wait complete. 2024-09-11T00:58:39.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.806+0000 7f8fe9719740 1 -- start start 2024-09-11T00:58:39.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.806+0000 7f8fe9719740 1 Processor -- start 2024-09-11T00:58:39.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe9719740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 0x5556e3cd74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe9719740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccc650 0x5556e3cf6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe9719740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3cdc9f0 0x5556e3cf70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe9719740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5556e3cee580 con 0x5556e3ccbd30 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe9719740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5556e3cedeb0 con 0x5556e3ccc650 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe9719740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5556e3ceec50 con 0x5556e3cdc9f0 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe9719740 10 client.0 _ll_register_callbacks cb 0x7f8fd8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe58e1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 0x5556e3cd74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe58e1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 0x5556e3cd74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57442/0 (socket says 172.21.15.78:57442) 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe58e1640 1 -- 172.21.15.78:0/638284087 learned_addr learned my addr 172.21.15.78:0/638284087 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe48df640 1 --2- 172.21.15.78:0/638284087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccc650 0x5556e3cf6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe58e1640 1 -- 172.21.15.78:0/638284087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3cdc9f0 msgr2=0x5556e3cf70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe58e1640 1 --2- 172.21.15.78:0/638284087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3cdc9f0 0x5556e3cf70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe58e1640 1 -- 172.21.15.78:0/638284087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccc650 msgr2=0x5556e3cf6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe58e1640 1 --2- 172.21.15.78:0/638284087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccc650 0x5556e3cf6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe58e1640 1 -- 172.21.15.78:0/638284087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556e3cd4990 con 0x5556e3ccbd30 2024-09-11T00:58:39.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fe58e1640 1 --2- 172.21.15.78:0/638284087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 0x5556e3cd74a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8fd802fe50 tx=0x7f8fd8003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa01913271c38bac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.807+0000 7f8fc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fc8ff9640 1 -- 172.21.15.78:0/638284087 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8031660 con 0x5556e3ccbd30 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fc8ff9640 1 -- 172.21.15.78:0/638284087 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f8fd8052040 con 0x5556e3ccbd30 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fe9719740 1 -- 172.21.15.78:0/638284087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fd001f030 con 0x5556e3ccbd30 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fc8ff9640 1 -- 172.21.15.78:0/638284087 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8fd80309e0 con 0x5556e3ccbd30 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fe9719740 10 client.15345 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fe9719740 1 -- 172.21.15.78:0/638284087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556e3d5e5f0 con 0x5556e3ccbd30 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fc8ff9640 1 -- 172.21.15.78:0/638284087 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd001f030 con 0x5556e3ccbd30 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fc8ff9640 1 -- 172.21.15.78:0/638284087 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8fd8004000 con 0x5556e3ccbd30 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fe9719740 10 client.15345 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fe9719740 10 client.15345 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fe9719740 20 client.15345 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:39.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fe9719740 1 --2- 172.21.15.78:0/638284087 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5556e3d62fc0 0x5556e3d61cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:39.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fe9719740 4 client.15345 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:58:39.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fe9719740 1 -- 172.21.15.78:0/638284087 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "status"}) -- 0x5556e3c41610 con 0x5556e3d62fc0 2024-09-11T00:58:39.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.808+0000 7f8fe48df640 1 --2- 172.21.15.78:0/638284087 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5556e3d62fc0 0x5556e3d61cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:39.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fe48df640 1 --2- 172.21.15.78:0/638284087 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5556e3d62fc0 0x5556e3d61cd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7032d29af077ed23 server_cookie=21c51b061fce87ac in_seq=0 out_seq=0 2024-09-11T00:58:39.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fc8ff9640 10 client.15345 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T00:58:39.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fc8ff9640 1 -- 172.21.15.78:0/638284087 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5556e3c41610 con 0x5556e3d62fc0 2024-09-11T00:58:39.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fc8ff9640 10 client.15345 handle_command_reply: tid=0 2024-09-11T00:58:39.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fe9719740 1 client.15345 shutdown 2024-09-11T00:58:39.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fe9719740 1 -- 172.21.15.78:0/638284087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 msgr2=0x5556e3cd74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:39.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fe9719740 1 --2- 172.21.15.78:0/638284087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 0x5556e3cd74a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8fd802fe50 tx=0x7f8fd8003d10 comp rx=0 tx=0).stop 2024-09-11T00:58:39.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fe9719740 1 -- 172.21.15.78:0/638284087 shutdown_connections 2024-09-11T00:58:39.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fe9719740 1 --2- 172.21.15.78:0/638284087 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5556e3d62fc0 0x5556e3d61cd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fe9719740 1 --2- 172.21.15.78:0/638284087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e3cdc9f0 0x5556e3cf70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fe9719740 1 --2- 172.21.15.78:0/638284087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556e3ccc650 0x5556e3cf6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fe9719740 1 --2- 172.21.15.78:0/638284087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556e3ccbd30 0x5556e3cd74a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fe9719740 1 -- 172.21.15.78:0/638284087 >> 172.21.15.78:0/638284087 conn(0x5556e3cc9b60 msgr2=0x5556e3cdafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:39.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.809+0000 7f8fc8ff9640 0 client.15345 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T00:58:39.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe9719740 1 -- 172.21.15.78:0/638284087 shutdown_connections 2024-09-11T00:58:39.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe9719740 1 -- 172.21.15.78:0/638284087 wait complete. 2024-09-11T00:58:39.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe9719740 20 client.15345 trim_cache size 0 max 16384 2024-09-11T00:58:39.818 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:58:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:58:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 3, 2024-09-11T00:58:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24563, 2024-09-11T00:58:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T00:58:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T00:58:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T00:58:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 96.627878566999996, 2024-09-11T00:58:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-11T00:58:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-09-11T00:58:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-09-11T00:58:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 130.32159482200001 2024-09-11T00:58:39.819 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:58:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe7344640 1 -- 172.21.15.78:0/2035209942 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8fcc0b5330 msgr2=0x7f8fcc0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe7344640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8fcc0b5330 0x7f8fcc0b77f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0e37f0 tx=0x7f8fdc093040 comp rx=0 tx=0).stop 2024-09-11T00:58:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe7344640 1 -- 172.21.15.78:0/2035209942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe0101c90 msgr2=0x7f8fe0199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe7344640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe0101c90 0x7f8fe0199310 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00d950 tx=0x7f8fdc00de10 comp rx=0 tx=0).stop 2024-09-11T00:58:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe7344640 1 -- 172.21.15.78:0/2035209942 shutdown_connections 2024-09-11T00:58:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe7344640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8fcc0b5330 0x7f8fcc0b77f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe7344640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8fe0108020 0x7f8fe019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe7344640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fe01076a0 0x7f8fe0199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe7344640 1 --2- 172.21.15.78:0/2035209942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe0101c90 0x7f8fe0199310 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.810+0000 7f8fe7344640 1 -- 172.21.15.78:0/2035209942 >> 172.21.15.78:0/2035209942 conn(0x7f8fe00fd700 msgr2=0x7f8fe01005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.811+0000 7f8fe7344640 1 -- 172.21.15.78:0/2035209942 shutdown_connections 2024-09-11T00:58:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:39.811+0000 7f8fe7344640 1 -- 172.21.15.78:0/2035209942 wait complete. 2024-09-11T00:58:40.634 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T00:58:40.888 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:41.163 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:41 smithi078 ceph-mon[31315]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:58:41.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.407+0000 7fe59194b640 1 -- 172.21.15.78:0/1468633655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe58c1076a0 msgr2=0x7fe58c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:41.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.407+0000 7fe59194b640 1 --2- 172.21.15.78:0/1468633655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe58c1076a0 0x7fe58c107ae0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe574009900 tx=0x7fe57402a410 comp rx=0 tx=0).stop 2024-09-11T00:58:41.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.407+0000 7fe59194b640 1 -- 172.21.15.78:0/1468633655 shutdown_connections 2024-09-11T00:58:41.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.407+0000 7fe59194b640 1 --2- 172.21.15.78:0/1468633655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe58c108020 0x7fe58c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.407+0000 7fe59194b640 1 --2- 172.21.15.78:0/1468633655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe58c1076a0 0x7fe58c107ae0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.407+0000 7fe59194b640 1 --2- 172.21.15.78:0/1468633655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe58c101c90 0x7fe58c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.407+0000 7fe59194b640 1 -- 172.21.15.78:0/1468633655 >> 172.21.15.78:0/1468633655 conn(0x7fe58c0fd700 msgr2=0x7fe58c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:41.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.407+0000 7fe59194b640 1 -- 172.21.15.78:0/1468633655 shutdown_connections 2024-09-11T00:58:41.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.407+0000 7fe59194b640 1 -- 172.21.15.78:0/1468633655 wait complete. 2024-09-11T00:58:41.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe59194b640 1 Processor -- start 2024-09-11T00:58:41.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe59194b640 1 -- start start 2024-09-11T00:58:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe59194b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe58c101c90 0x7fe58c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe59194b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe58c1076a0 0x7fe58c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe59194b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe58c108020 0x7fe58c19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe59194b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c111ce0 con 0x7fe58c1076a0 2024-09-11T00:58:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe59194b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c111b60 con 0x7fe58c101c90 2024-09-11T00:58:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe59194b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c111e60 con 0x7fe58c108020 2024-09-11T00:58:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe58a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe58c1076a0 0x7fe58c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe58a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe58c1076a0 0x7fe58c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57450/0 (socket says 172.21.15.78:57450) 2024-09-11T00:58:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe58a7fc640 1 -- 172.21.15.78:0/1157734941 learned_addr learned my addr 172.21.15.78:0/1157734941 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe58affd640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe58c101c90 0x7fe58c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe58b7fe640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe58c108020 0x7fe58c19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe58a7fc640 1 -- 172.21.15.78:0/1157734941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe58c108020 msgr2=0x7fe58c19da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe58a7fc640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe58c108020 0x7fe58c19da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe58a7fc640 1 -- 172.21.15.78:0/1157734941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe58c101c90 msgr2=0x7fe58c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe58a7fc640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe58c101c90 0x7fe58c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.408+0000 7fe58a7fc640 1 -- 172.21.15.78:0/1157734941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c19e250 con 0x7fe58c1076a0 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.409+0000 7fe58a7fc640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe58c1076a0 0x7fe58c1986e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe574005940 tx=0x7fe5740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.409+0000 7fe590949640 1 -- 172.21.15.78:0/1157734941 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57403f030 con 0x7fe58c1076a0 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.409+0000 7fe59194b640 1 -- 172.21.15.78:0/1157734941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58c19e4e0 con 0x7fe58c1076a0 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.409+0000 7fe590949640 1 -- 172.21.15.78:0/1157734941 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe574003cb0 con 0x7fe58c1076a0 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.409+0000 7fe590949640 1 -- 172.21.15.78:0/1157734941 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574035070 con 0x7fe58c1076a0 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.409+0000 7fe59194b640 1 -- 172.21.15.78:0/1157734941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe58c06aab0 con 0x7fe58c1076a0 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.411+0000 7fe593d20740 1 Processor -- start 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe593d20740 1 -- start start 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe593d20740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9dd30 0x55a6fac9e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe593d20740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 0x55a6facaa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe590949640 1 -- 172.21.15.78:0/1157734941 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fe57403bbf0 con 0x7fe58c1076a0 2024-09-11T00:58:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe593d20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6facb0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe593d20740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a6fac15c10 con 0x55a6fac9e650 2024-09-11T00:58:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe593d20740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a6fac05040 con 0x55a6fac9dd30 2024-09-11T00:58:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe593d20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a6fac08e50 con 0x55a6facae9f0 2024-09-11T00:58:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe58affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 0x55a6facaa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe58affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 0x55a6facaa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57462/0 (socket says 172.21.15.78:57462) 2024-09-11T00:58:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe58affd640 1 -- 172.21.15.78:0/1075296721 learned_addr learned my addr 172.21.15.78:0/1075296721 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe58b7fe640 1 --2- 172.21.15.78:0/1075296721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9dd30 0x55a6fac9e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe58a7fc640 1 --2- 172.21.15.78:0/1075296721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6facb0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe58affd640 1 -- 172.21.15.78:0/1075296721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 msgr2=0x55a6facb0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe58affd640 1 --2- 172.21.15.78:0/1075296721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6facb0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe58affd640 1 -- 172.21.15.78:0/1075296721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9dd30 msgr2=0x55a6fac9e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe58affd640 1 --2- 172.21.15.78:0/1075296721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9dd30 0x55a6fac9e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe58affd640 1 -- 172.21.15.78:0/1075296721 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6fa963540 con 0x55a6fac9e650 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe58a7fc640 1 --2- 172.21.15.78:0/1075296721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6facb0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.412+0000 7fe58b7fe640 1 --2- 172.21.15.78:0/1075296721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9dd30 0x55a6fac9e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.413+0000 7fe58affd640 1 --2- 172.21.15.78:0/1075296721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 0x55a6facaa0a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00b790 tx=0x7fe57c00bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7167e4c1f21922a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.413+0000 7fe590949640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe5780b5330 0x7fe5780b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.413+0000 7fe581ffb640 1 -- 172.21.15.78:0/1075296721 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c003dd0 con 0x55a6fac9e650 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.413+0000 7fe58b7fe640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe5780b5330 0x7fe5780b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.413+0000 7fe590949640 1 -- 172.21.15.78:0/1157734941 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe574106810 con 0x7fe58c1076a0 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.413+0000 7fe581ffb640 1 -- 172.21.15.78:0/1075296721 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe57c026070 con 0x55a6fac9e650 2024-09-11T00:58:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.413+0000 7fe581ffb640 1 -- 172.21.15.78:0/1075296721 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c022450 con 0x55a6fac9e650 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.414+0000 7fe593d20740 1 -- 172.21.15.78:0/1075296721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 msgr2=0x55a6facaa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.414+0000 7fe58b7fe640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe5780b5330 0x7fe5780b77f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe584005fd0 tx=0x7fe584009600 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.414+0000 7fe593d20740 1 --2- 172.21.15.78:0/1075296721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 0x55a6facaa0a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00b790 tx=0x7fe57c00bc50 comp rx=0 tx=0).stop 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.414+0000 7fe593d20740 1 -- 172.21.15.78:0/1075296721 shutdown_connections 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.414+0000 7fe593d20740 1 --2- 172.21.15.78:0/1075296721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6facb0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.414+0000 7fe593d20740 1 --2- 172.21.15.78:0/1075296721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 0x55a6facaa0a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.414+0000 7fe593d20740 1 --2- 172.21.15.78:0/1075296721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9dd30 0x55a6fac9e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.414+0000 7fe593d20740 1 -- 172.21.15.78:0/1075296721 >> 172.21.15.78:0/1075296721 conn(0x55a6fac9bb60 msgr2=0x55a6fac9cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.414+0000 7fe593d20740 1 -- 172.21.15.78:0/1075296721 shutdown_connections 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.414+0000 7fe593d20740 1 -- 172.21.15.78:0/1075296721 wait complete. 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.414+0000 7fe593d20740 1 -- start start 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe593d20740 1 Processor -- start 2024-09-11T00:58:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe593d20740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 0x55a6faca5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe593d20740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9e650 0x55a6faca6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe593d20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6facafe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe593d20740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a6faa665f0 con 0x55a6fac9dd30 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe593d20740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a6faba0010 con 0x55a6fac9e650 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe593d20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a6facaa960 con 0x55a6facae9f0 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe593d20740 10 client.0 _ll_register_callbacks cb 0x55a6fabff340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe58affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 0x55a6faca5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe58affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 0x55a6faca5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57464/0 (socket says 172.21.15.78:57464) 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe58affd640 1 -- 172.21.15.78:0/3228953337 learned_addr learned my addr 172.21.15.78:0/3228953337 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe58a7fc640 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9e650 0x55a6faca6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe58b7fe640 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6facafe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe58affd640 1 -- 172.21.15.78:0/3228953337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 msgr2=0x55a6facafe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe58affd640 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6facafe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe58affd640 1 -- 172.21.15.78:0/3228953337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9e650 msgr2=0x55a6faca6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe58affd640 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9e650 0x55a6faca6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.415+0000 7fe58affd640 1 -- 172.21.15.78:0/3228953337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6fabcda00 con 0x55a6fac9dd30 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe58affd640 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 0x55a6faca5f90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00b760 tx=0x7fe57c004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4569db41bbcd333b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe56affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe56affd640 1 -- 172.21.15.78:0/3228953337 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c039060 con 0x55a6fac9dd30 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe56affd640 1 -- 172.21.15.78:0/3228953337 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe57c034460 con 0x55a6fac9dd30 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe593d20740 1 -- 172.21.15.78:0/3228953337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57c026070 con 0x55a6fac9dd30 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe56affd640 1 -- 172.21.15.78:0/3228953337 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe57c034660 con 0x55a6fac9dd30 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe593d20740 10 client.15357 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe593d20740 1 -- 172.21.15.78:0/3228953337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6facb8e50 con 0x55a6fac9dd30 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe56affd640 1 -- 172.21.15.78:0/3228953337 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe57c026070 con 0x55a6fac9dd30 2024-09-11T00:58:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe56affd640 1 -- 172.21.15.78:0/3228953337 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe57c03c020 con 0x55a6fac9dd30 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe593d20740 10 client.15357 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe593d20740 10 client.15357 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe593d20740 20 client.15357 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe593d20740 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a6facbb0a0 0x55a6facbc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe593d20740 4 client.15357 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.416+0000 7fe593d20740 1 -- 172.21.15.78:0/3228953337 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6fac17b60 con 0x55a6facbb0a0 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.417+0000 7fe58a7fc640 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a6facbb0a0 0x55a6facbc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.417+0000 7fe58a7fc640 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a6facbb0a0 0x55a6facbc510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eed13aea99a4ce63 server_cookie=4e1488121391e1e4 in_seq=0 out_seq=0 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.417+0000 7fe56affd640 10 client.15357 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe56affd640 1 -- 172.21.15.78:0/3228953337 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6fac17b60 con 0x55a6facbb0a0 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe56affd640 10 client.15357 handle_command_reply: tid=0 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe593d20740 1 client.15357 shutdown 2024-09-11T00:58:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe593d20740 1 -- 172.21.15.78:0/3228953337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 msgr2=0x55a6faca5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe593d20740 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 0x55a6faca5f90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00b760 tx=0x7fe57c004420 comp rx=0 tx=0).stop 2024-09-11T00:58:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe593d20740 1 -- 172.21.15.78:0/3228953337 shutdown_connections 2024-09-11T00:58:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe593d20740 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6facafe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe593d20740 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a6facbb0a0 0x55a6facbc510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe593d20740 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9e650 0x55a6faca6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe56affd640 0 client.15357 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T00:58:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe593d20740 1 --2- 172.21.15.78:0/3228953337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 0x55a6faca5f90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe593d20740 1 -- 172.21.15.78:0/3228953337 >> 172.21.15.78:0/3228953337 conn(0x55a6fac9bb60 msgr2=0x55a6faca1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe593d20740 1 -- 172.21.15.78:0/3228953337 shutdown_connections 2024-09-11T00:58:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe593d20740 1 -- 172.21.15.78:0/3228953337 wait complete. 2024-09-11T00:58:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.418+0000 7fe593d20740 20 client.15357 trim_cache size 0 max 16384 2024-09-11T00:58:41.443 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:41 smithi110 ceph-mon[34712]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:58:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe593d20740 1 Processor -- start 2024-09-11T00:58:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe593d20740 1 -- start start 2024-09-11T00:58:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe593d20740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 0x55a6facb6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe593d20740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9e650 0x55a6facafde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe593d20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6faca5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe593d20740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a6facba850 con 0x55a6fac9dd30 2024-09-11T00:58:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe593d20740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a6facbaf20 con 0x55a6fac9e650 2024-09-11T00:58:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe593d20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a6facbb5f0 con 0x55a6facae9f0 2024-09-11T00:58:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe58affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 0x55a6facb6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe58affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 0x55a6facb6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57476/0 (socket says 172.21.15.78:57476) 2024-09-11T00:58:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe58affd640 1 -- 172.21.15.78:0/2139234108 learned_addr learned my addr 172.21.15.78:0/2139234108 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe58a7fc640 1 --2- 172.21.15.78:0/2139234108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9e650 0x55a6facafde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe58b7fe640 1 --2- 172.21.15.78:0/2139234108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6faca5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe58affd640 1 -- 172.21.15.78:0/2139234108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 msgr2=0x55a6faca5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe58affd640 1 --2- 172.21.15.78:0/2139234108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6faca5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe58affd640 1 -- 172.21.15.78:0/2139234108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9e650 msgr2=0x55a6facafde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe58affd640 1 --2- 172.21.15.78:0/2139234108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9e650 0x55a6facafde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe58affd640 1 -- 172.21.15.78:0/2139234108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6fac9e480 con 0x55a6fac9dd30 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.441+0000 7fe58affd640 1 --2- 172.21.15.78:0/2139234108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 0x55a6facb6d70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe57c003c20 tx=0x7fe57c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73dc09151678bd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.442+0000 7fe5817fa640 1 -- 172.21.15.78:0/2139234108 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c039070 con 0x55a6fac9dd30 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.442+0000 7fe5817fa640 1 -- 172.21.15.78:0/2139234108 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe57c026070 con 0x55a6fac9dd30 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.442+0000 7fe5817fa640 1 -- 172.21.15.78:0/2139234108 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c034800 con 0x55a6fac9dd30 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.442+0000 7fe593d20740 1 -- 172.21.15.78:0/2139234108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 msgr2=0x55a6facb6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.442+0000 7fe593d20740 1 --2- 172.21.15.78:0/2139234108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 0x55a6facb6d70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe57c003c20 tx=0x7fe57c005f30 comp rx=0 tx=0).stop 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.442+0000 7fe593d20740 1 -- 172.21.15.78:0/2139234108 shutdown_connections 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.442+0000 7fe593d20740 1 --2- 172.21.15.78:0/2139234108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6facae9f0 0x55a6faca5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.442+0000 7fe593d20740 1 --2- 172.21.15.78:0/2139234108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6fac9e650 0x55a6facafde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.442+0000 7fe593d20740 1 --2- 172.21.15.78:0/2139234108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9dd30 0x55a6facb6d70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.442+0000 7fe593d20740 1 -- 172.21.15.78:0/2139234108 >> 172.21.15.78:0/2139234108 conn(0x55a6fac9bb60 msgr2=0x55a6facada10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.442+0000 7fe593d20740 1 -- 172.21.15.78:0/2139234108 shutdown_connections 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.443+0000 7fe593d20740 1 -- 172.21.15.78:0/2139234108 wait complete. 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.443+0000 7fe593d20740 1 -- start start 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.443+0000 7fe593d20740 1 Processor -- start 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.443+0000 7fe593d20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6fac9dd30 0x55a6facb2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.443+0000 7fe593d20740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 0x55a6facb31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.443+0000 7fe593d20740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6facae9f0 0x55a6facb3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.443+0000 7fe593d20740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a6facbc390 con 0x55a6fac9e650 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.443+0000 7fe593d20740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a6facbbcc0 con 0x55a6facae9f0 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.443+0000 7fe593d20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a6facbca60 con 0x55a6fac9dd30 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.443+0000 7fe58a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 0x55a6facb31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.443+0000 7fe58a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 0x55a6facb31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57480/0 (socket says 172.21.15.78:57480) 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.443+0000 7fe58a7fc640 1 -- 172.21.15.78:0/293576590 learned_addr learned my addr 172.21.15.78:0/293576590 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe593d20740 10 client.0 _ll_register_callbacks cb 0x7fe584009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe58affd640 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6fac9dd30 0x55a6facb2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe58b7fe640 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6facae9f0 0x55a6facb3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe58a7fc640 1 -- 172.21.15.78:0/293576590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6fac9dd30 msgr2=0x55a6facb2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe58a7fc640 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6fac9dd30 0x55a6facb2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe58a7fc640 1 -- 172.21.15.78:0/293576590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6facae9f0 msgr2=0x55a6facb3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe58a7fc640 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6facae9f0 0x55a6facb3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe58a7fc640 1 -- 172.21.15.78:0/293576590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6fad26cb0 con 0x55a6fac9e650 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe58affd640 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6fac9dd30 0x55a6facb2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe58a7fc640 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 0x55a6facb31a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe5740ca240 tx=0x7fe57402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9590fa606c0a265b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe56affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe56affd640 1 -- 172.21.15.78:0/293576590 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574049040 con 0x55a6fac9e650 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe56affd640 1 -- 172.21.15.78:0/293576590 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe5740c9240 con 0x55a6fac9e650 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe593d20740 1 -- 172.21.15.78:0/293576590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57c026070 con 0x55a6fac9e650 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe56affd640 1 -- 172.21.15.78:0/293576590 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe574005d50 con 0x55a6fac9e650 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe593d20740 10 client.15363 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe593d20740 1 -- 172.21.15.78:0/293576590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6fad27fb0 con 0x55a6fac9e650 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.444+0000 7fe56affd640 1 -- 172.21.15.78:0/293576590 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5740ccae0 con 0x55a6fac9e650 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.445+0000 7fe56affd640 1 -- 172.21.15.78:0/293576590 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe5740d0070 con 0x55a6fac9e650 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.445+0000 7fe593d20740 10 client.15363 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.445+0000 7fe593d20740 10 client.15363 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T00:58:41.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.445+0000 7fe593d20740 20 client.15363 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.445+0000 7fe593d20740 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a6fad2cc00 0x55a6fad2b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.445+0000 7fe593d20740 4 client.15363 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.445+0000 7fe593d20740 1 -- 172.21.15.78:0/293576590 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "status"}) -- 0x7fe57c026070 con 0x55a6fad2cc00 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.445+0000 7fe58affd640 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a6fad2cc00 0x55a6fad2b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.445+0000 7fe56affd640 10 client.15363 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.445+0000 7fe58affd640 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a6fad2cc00 0x55a6fad2b910 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aef60af07abafbad server_cookie=fc2039c65ee51ba2 in_seq=0 out_seq=0 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe56affd640 1 -- 172.21.15.78:0/293576590 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x7fe57c026070 con 0x55a6fad2cc00 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe56affd640 10 client.15363 handle_command_reply: tid=0 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe593d20740 1 client.15363 shutdown 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe593d20740 1 -- 172.21.15.78:0/293576590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 msgr2=0x55a6facb31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe593d20740 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 0x55a6facb31a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe5740ca240 tx=0x7fe57402a410 comp rx=0 tx=0).stop 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe593d20740 1 -- 172.21.15.78:0/293576590 shutdown_connections 2024-09-11T00:58:41.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe593d20740 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a6fad2cc00 0x55a6fad2b910 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe593d20740 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a6facae9f0 0x55a6facb3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe593d20740 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6fac9e650 0x55a6facb31a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe593d20740 1 --2- 172.21.15.78:0/293576590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a6fac9dd30 0x55a6facb2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe593d20740 1 -- 172.21.15.78:0/293576590 >> 172.21.15.78:0/293576590 conn(0x55a6fac9bb60 msgr2=0x55a6faca5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:41.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe56affd640 0 client.15363 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T00:58:41.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe593d20740 1 -- 172.21.15.78:0/293576590 shutdown_connections 2024-09-11T00:58:41.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe593d20740 1 -- 172.21.15.78:0/293576590 wait complete. 2024-09-11T00:58:41.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.446+0000 7fe593d20740 20 client.15363 trim_cache size 0 max 16384 2024-09-11T00:58:41.454 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:58:41.454 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:58:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-09-11T00:58:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24565, 2024-09-11T00:58:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T00:58:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T00:58:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T00:58:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 99.275066925999994, 2024-09-11T00:58:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-11T00:58:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-09-11T00:58:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-09-11T00:58:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 134.64597135099999 2024-09-11T00:58:41.455 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:58:41.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.447+0000 7fe59194b640 1 -- 172.21.15.78:0/1157734941 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe5780b5330 msgr2=0x7fe5780b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:41.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.447+0000 7fe59194b640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe5780b5330 0x7fe5780b77f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe584005fd0 tx=0x7fe584009600 comp rx=0 tx=0).stop 2024-09-11T00:58:41.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.447+0000 7fe59194b640 1 -- 172.21.15.78:0/1157734941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe58c1076a0 msgr2=0x7fe58c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:41.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.447+0000 7fe59194b640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe58c1076a0 0x7fe58c1986e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe574005940 tx=0x7fe5740042f0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.447+0000 7fe59194b640 1 -- 172.21.15.78:0/1157734941 shutdown_connections 2024-09-11T00:58:41.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.447+0000 7fe59194b640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe5780b5330 0x7fe5780b77f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.447+0000 7fe59194b640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe58c108020 0x7fe58c19da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.447+0000 7fe59194b640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe58c1076a0 0x7fe58c1986e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.447+0000 7fe59194b640 1 --2- 172.21.15.78:0/1157734941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe58c101c90 0x7fe58c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:41.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.447+0000 7fe59194b640 1 -- 172.21.15.78:0/1157734941 >> 172.21.15.78:0/1157734941 conn(0x7fe58c0fd700 msgr2=0x7fe58c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:41.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.447+0000 7fe59194b640 1 -- 172.21.15.78:0/1157734941 shutdown_connections 2024-09-11T00:58:41.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:41.447+0000 7fe59194b640 1 -- 172.21.15.78:0/1157734941 wait complete. 2024-09-11T00:58:41.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:41 smithi112 ceph-mon[35384]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:58:42.334 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T00:58:42.581 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.097+0000 7f605ce78640 1 -- 172.21.15.78:0/1985787838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6058101c90 msgr2=0x7f6058102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.097+0000 7f605ce78640 1 --2- 172.21.15.78:0/1985787838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6058101c90 0x7f6058102070 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6048009900 tx=0x7f604802a410 comp rx=0 tx=0).stop 2024-09-11T00:58:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.097+0000 7f605ce78640 1 -- 172.21.15.78:0/1985787838 shutdown_connections 2024-09-11T00:58:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.097+0000 7f605ce78640 1 --2- 172.21.15.78:0/1985787838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6058108020 0x7f605810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.097+0000 7f605ce78640 1 --2- 172.21.15.78:0/1985787838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60581076a0 0x7f6058107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.097+0000 7f605ce78640 1 --2- 172.21.15.78:0/1985787838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6058101c90 0x7f6058102070 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.097+0000 7f605ce78640 1 -- 172.21.15.78:0/1985787838 >> 172.21.15.78:0/1985787838 conn(0x7f60580fd6e0 msgr2=0x7f60580ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.097+0000 7f605ce78640 1 -- 172.21.15.78:0/1985787838 shutdown_connections 2024-09-11T00:58:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.097+0000 7f605ce78640 1 -- 172.21.15.78:0/1985787838 wait complete. 2024-09-11T00:58:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.098+0000 7f605ce78640 1 Processor -- start 2024-09-11T00:58:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.098+0000 7f605ce78640 1 -- start start 2024-09-11T00:58:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.098+0000 7f605ce78640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6058101c90 0x7f60581982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.098+0000 7f605ce78640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60581076a0 0x7f60581987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.098+0000 7f605ce78640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6058108020 0x7f605819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.098+0000 7f605ce78640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6058111d20 con 0x7f60581076a0 2024-09-11T00:58:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.098+0000 7f605ce78640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6058111ba0 con 0x7f6058108020 2024-09-11T00:58:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.098+0000 7f605ce78640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6058111ea0 con 0x7f6058101c90 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.098+0000 7f604ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60581076a0 0x7f60581987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.098+0000 7f604ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60581076a0 0x7f60581987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57488/0 (socket says 172.21.15.78:57488) 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.098+0000 7f604ffff640 1 -- 172.21.15.78:0/3884558724 learned_addr learned my addr 172.21.15.78:0/3884558724 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f6056d9d640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6058108020 0x7f605819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f605659c640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6058101c90 0x7f60581982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f604ffff640 1 -- 172.21.15.78:0/3884558724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6058101c90 msgr2=0x7f60581982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f604ffff640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6058101c90 0x7f60581982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f604ffff640 1 -- 172.21.15.78:0/3884558724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6058108020 msgr2=0x7f605819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f604ffff640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6058108020 0x7f605819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f604ffff640 1 -- 172.21.15.78:0/3884558724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605819e350 con 0x7f60581076a0 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f605659c640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6058101c90 0x7f60581982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f604ffff640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60581076a0 0x7f60581987e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f604000b4a0 tx=0x7f604000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f604f7fe640 1 -- 172.21.15.78:0/3884558724 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040014070 con 0x7f60581076a0 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f605ce78640 1 -- 172.21.15.78:0/3884558724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605819e640 con 0x7f60581076a0 2024-09-11T00:58:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f604f7fe640 1 -- 172.21.15.78:0/3884558724 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6040010ac0 con 0x7f60581076a0 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f605ce78640 1 -- 172.21.15.78:0/3884558724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605806b700 con 0x7f60581076a0 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.099+0000 7f604f7fe640 1 -- 172.21.15.78:0/3884558724 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604001e060 con 0x7f60581076a0 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.101+0000 7f604f7fe640 1 -- 172.21.15.78:0/3884558724 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f6040017bd0 con 0x7f60581076a0 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.102+0000 7f605f24d740 1 Processor -- start 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.102+0000 7f605f24d740 1 -- start start 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.102+0000 7f605f24d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819ad30 0x55fdc819b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.102+0000 7f605f24d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 0x55fdc81a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.102+0000 7f605f24d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 0x55fdc81a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.102+0000 7f605f24d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fdc8112c10 con 0x55fdc81a75e0 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.102+0000 7f605f24d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fdc8102040 con 0x55fdc819ad30 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.102+0000 7f605f24d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fdc8105e50 con 0x55fdc819b650 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f604f7fe640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f60440b5330 0x7f60440b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f605659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 0x55fdc81a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f605659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 0x55fdc81a70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:41018/0 (socket says 172.21.15.78:41018) 2024-09-11T00:58:43.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f605659c640 1 -- 172.21.15.78:0/2281971289 learned_addr learned my addr 172.21.15.78:0/2281971289 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:43.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f6056d9d640 1 --2- 172.21.15.78:0/2281971289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819ad30 0x55fdc819b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f6056d9d640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f60440b5330 0x7f60440b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f604f7fe640 1 -- 172.21.15.78:0/3884558724 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f60400e5690 con 0x7f60581076a0 2024-09-11T00:58:43.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f605659c640 1 -- 172.21.15.78:0/2281971289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819ad30 msgr2=0x55fdc819b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:43.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f605659c640 1 --2- 172.21.15.78:0/2281971289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819ad30 0x55fdc819b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f605659c640 1 -- 172.21.15.78:0/2281971289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 msgr2=0x55fdc81a99d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:58:43.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f604ffff640 1 --2- 172.21.15.78:0/2281971289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 0x55fdc81a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f605659c640 1 --2- 172.21.15.78:0/2281971289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 0x55fdc81a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f605659c640 1 -- 172.21.15.78:0/2281971289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc7e60540 con 0x55fdc819b650 2024-09-11T00:58:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f6056d9d640 1 --2- 172.21.15.78:0/2281971289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819ad30 0x55fdc819b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:58:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.103+0000 7f604ffff640 1 --2- 172.21.15.78:0/2281971289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 0x55fdc81a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.104+0000 7f6056d9d640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f60440b5330 0x7f60440b77f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f60500070b0 tx=0x7f60500093d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.104+0000 7f605659c640 1 --2- 172.21.15.78:0/2281971289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 0x55fdc81a70a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6048032e90 tx=0x7f6048032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5b84ba75600631a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.104+0000 7f604cff9640 1 -- 172.21.15.78:0/2281971289 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048035030 con 0x55fdc819b650 2024-09-11T00:58:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.104+0000 7f604cff9640 1 -- 172.21.15.78:0/2281971289 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6048031760 con 0x55fdc819b650 2024-09-11T00:58:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.104+0000 7f604cff9640 1 -- 172.21.15.78:0/2281971289 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048005650 con 0x55fdc819b650 2024-09-11T00:58:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.105+0000 7f605f24d740 1 -- 172.21.15.78:0/2281971289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 msgr2=0x55fdc81a70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.105+0000 7f605f24d740 1 --2- 172.21.15.78:0/2281971289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 0x55fdc81a70a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6048032e90 tx=0x7f6048032ec0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.105+0000 7f605f24d740 1 -- 172.21.15.78:0/2281971289 shutdown_connections 2024-09-11T00:58:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.105+0000 7f605f24d740 1 --2- 172.21.15.78:0/2281971289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 0x55fdc81a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.105+0000 7f605f24d740 1 --2- 172.21.15.78:0/2281971289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 0x55fdc81a70a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.105+0000 7f605f24d740 1 --2- 172.21.15.78:0/2281971289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819ad30 0x55fdc819b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.105+0000 7f605f24d740 1 -- 172.21.15.78:0/2281971289 >> 172.21.15.78:0/2281971289 conn(0x55fdc8198b60 msgr2=0x55fdc8199d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.105+0000 7f605f24d740 1 -- 172.21.15.78:0/2281971289 shutdown_connections 2024-09-11T00:58:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.105+0000 7f605f24d740 1 -- 172.21.15.78:0/2281971289 wait complete. 2024-09-11T00:58:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.105+0000 7f605f24d740 1 -- start start 2024-09-11T00:58:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.105+0000 7f605f24d740 1 Processor -- start 2024-09-11T00:58:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.105+0000 7f605f24d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 0x55fdc81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605f24d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819b650 0x55fdc81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605f24d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc81a75e0 0x55fdc81a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605f24d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fdc7f635f0 con 0x55fdc819ad30 2024-09-11T00:58:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605f24d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fdc809d010 con 0x55fdc819b650 2024-09-11T00:58:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605f24d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fdc81aa0f0 con 0x55fdc81a75e0 2024-09-11T00:58:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605f24d740 10 client.0 _ll_register_callbacks cb 0x55fdc80fc340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 0x55fdc81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 0x55fdc81a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57514/0 (socket says 172.21.15.78:57514) 2024-09-11T00:58:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605659c640 1 -- 172.21.15.78:0/1713357635 learned_addr learned my addr 172.21.15.78:0/1713357635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f604ffff640 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819b650 0x55fdc81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f6056d9d640 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc81a75e0 0x55fdc81a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605659c640 1 -- 172.21.15.78:0/1713357635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc81a75e0 msgr2=0x55fdc81a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605659c640 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc81a75e0 0x55fdc81a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605659c640 1 -- 172.21.15.78:0/1713357635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819b650 msgr2=0x55fdc81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605659c640 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819b650 0x55fdc81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605659c640 1 -- 172.21.15.78:0/1713357635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc80caa00 con 0x55fdc819ad30 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f604ffff640 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819b650 0x55fdc81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f605659c640 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 0x55fdc81a0650 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6048033000 tx=0x7f604802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be14ac600631c3ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.106+0000 7f603dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.111+0000 7f603dffb640 1 -- 172.21.15.78:0/1713357635 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604804b020 con 0x55fdc819ad30 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.111+0000 7f603dffb640 1 -- 172.21.15.78:0/1713357635 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f60480055f0 con 0x55fdc819ad30 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.111+0000 7f605f24d740 1 -- 172.21.15.78:0/1713357635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6048031760 con 0x55fdc819ad30 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.111+0000 7f603dffb640 1 -- 172.21.15.78:0/1713357635 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6048056050 con 0x55fdc819ad30 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.111+0000 7f605f24d740 10 client.15372 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.111+0000 7f605f24d740 1 -- 172.21.15.78:0/1713357635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fdc81be570 con 0x55fdc819ad30 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.111+0000 7f603dffb640 1 -- 172.21.15.78:0/1713357635 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6048031760 con 0x55fdc819ad30 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.112+0000 7f603dffb640 1 -- 172.21.15.78:0/1713357635 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f60480035f0 con 0x55fdc819ad30 2024-09-11T00:58:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.112+0000 7f605f24d740 10 client.15372 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.112+0000 7f605f24d740 10 client.15372 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.112+0000 7f605f24d740 20 client.15372 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.112+0000 7f605f24d740 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55fdc81c07a0 0x55fdc81c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.112+0000 7f605f24d740 4 client.15372 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.112+0000 7f605f24d740 1 -- 172.21.15.78:0/1713357635 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdc8114b60 con 0x55fdc81c07a0 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.112+0000 7f604ffff640 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55fdc81c07a0 0x55fdc81c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.113+0000 7f604ffff640 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55fdc81c07a0 0x55fdc81c1c10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a306c52a16dc308e server_cookie=c464db1d614a8752 in_seq=0 out_seq=0 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.113+0000 7f603dffb640 10 client.15372 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.113+0000 7f603dffb640 1 -- 172.21.15.78:0/1713357635 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fdc8114b60 con 0x55fdc81c07a0 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.113+0000 7f603dffb640 10 client.15372 handle_command_reply: tid=0 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.113+0000 7f605f24d740 1 client.15372 shutdown 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.113+0000 7f605f24d740 1 -- 172.21.15.78:0/1713357635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 msgr2=0x55fdc81a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.113+0000 7f605f24d740 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 0x55fdc81a0650 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6048033000 tx=0x7f604802a410 comp rx=0 tx=0).stop 2024-09-11T00:58:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.113+0000 7f605f24d740 1 -- 172.21.15.78:0/1713357635 shutdown_connections 2024-09-11T00:58:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.114+0000 7f605f24d740 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc81a75e0 0x55fdc81a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.114+0000 7f605f24d740 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819b650 0x55fdc81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.114+0000 7f605f24d740 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55fdc81c07a0 0x55fdc81c1c10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.114+0000 7f605f24d740 1 --2- 172.21.15.78:0/1713357635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 0x55fdc81a0650 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.114+0000 7f603dffb640 0 client.15372 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T00:58:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.114+0000 7f605f24d740 1 -- 172.21.15.78:0/1713357635 >> 172.21.15.78:0/1713357635 conn(0x55fdc8198b60 msgr2=0x55fdc819e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.114+0000 7f605f24d740 1 -- 172.21.15.78:0/1713357635 shutdown_connections 2024-09-11T00:58:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.114+0000 7f605f24d740 1 -- 172.21.15.78:0/1713357635 wait complete. 2024-09-11T00:58:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.114+0000 7f605f24d740 20 client.15372 trim_cache size 0 max 16384 2024-09-11T00:58:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f605f24d740 1 Processor -- start 2024-09-11T00:58:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f605f24d740 1 -- start start 2024-09-11T00:58:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f605f24d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819ad30 0x55fdc8199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f605f24d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 0x55fdc819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f605f24d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 0x55fdc81c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f605f24d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fdc81bff50 con 0x55fdc81a75e0 2024-09-11T00:58:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f605f24d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fdc81c0620 con 0x55fdc819ad30 2024-09-11T00:58:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f605f24d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fdc81c0cf0 con 0x55fdc819b650 2024-09-11T00:58:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f6056d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 0x55fdc81c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f6056d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 0x55fdc81c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57518/0 (socket says 172.21.15.78:57518) 2024-09-11T00:58:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f6056d9d640 1 -- 172.21.15.78:0/2161712769 learned_addr learned my addr 172.21.15.78:0/2161712769 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f604ffff640 1 --2- 172.21.15.78:0/2161712769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 0x55fdc819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f605659c640 1 --2- 172.21.15.78:0/2161712769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819ad30 0x55fdc8199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f6056d9d640 1 -- 172.21.15.78:0/2161712769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 msgr2=0x55fdc819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f6056d9d640 1 --2- 172.21.15.78:0/2161712769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 0x55fdc819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f6056d9d640 1 -- 172.21.15.78:0/2161712769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819ad30 msgr2=0x55fdc8199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f6056d9d640 1 --2- 172.21.15.78:0/2161712769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819ad30 0x55fdc8199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f6056d9d640 1 -- 172.21.15.78:0/2161712769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc8198990 con 0x55fdc81a75e0 2024-09-11T00:58:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f604ffff640 1 --2- 172.21.15.78:0/2161712769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 0x55fdc819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.132+0000 7f6056d9d640 1 --2- 172.21.15.78:0/2161712769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 0x55fdc81c9ef0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6050009570 tx=0x7f6050009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e672264b076e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.133+0000 7f603ffff640 1 -- 172.21.15.78:0/2161712769 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605002a070 con 0x55fdc81a75e0 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.133+0000 7f603ffff640 1 -- 172.21.15.78:0/2161712769 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6050019d20 con 0x55fdc81a75e0 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.133+0000 7f603ffff640 1 -- 172.21.15.78:0/2161712769 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050013040 con 0x55fdc81a75e0 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.133+0000 7f605f24d740 1 -- 172.21.15.78:0/2161712769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 msgr2=0x55fdc81c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.133+0000 7f605f24d740 1 --2- 172.21.15.78:0/2161712769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 0x55fdc81c9ef0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6050009570 tx=0x7f6050009700 comp rx=0 tx=0).stop 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.133+0000 7f605f24d740 1 -- 172.21.15.78:0/2161712769 shutdown_connections 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.133+0000 7f605f24d740 1 --2- 172.21.15.78:0/2161712769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc81a75e0 0x55fdc81c9ef0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.133+0000 7f605f24d740 1 --2- 172.21.15.78:0/2161712769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc819b650 0x55fdc819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.133+0000 7f605f24d740 1 --2- 172.21.15.78:0/2161712769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819ad30 0x55fdc8199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.133+0000 7f605f24d740 1 -- 172.21.15.78:0/2161712769 >> 172.21.15.78:0/2161712769 conn(0x55fdc8198b60 msgr2=0x55fdc81aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.133+0000 7f605f24d740 1 -- 172.21.15.78:0/2161712769 shutdown_connections 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.133+0000 7f605f24d740 1 -- 172.21.15.78:0/2161712769 wait complete. 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605f24d740 1 -- start start 2024-09-11T00:58:43.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605f24d740 1 Processor -- start 2024-09-11T00:58:43.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605f24d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 0x55fdc81ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605f24d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819b650 0x55fdc81ba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605f24d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc81a75e0 0x55fdc822c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605f24d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fdc81c1a90 con 0x55fdc819ad30 2024-09-11T00:58:43.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605f24d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fdc81c13c0 con 0x55fdc819b650 2024-09-11T00:58:43.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605f24d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fdc81c2160 con 0x55fdc81a75e0 2024-09-11T00:58:43.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605f24d740 10 client.0 _ll_register_callbacks cb 0x7f6050009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:43.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 0x55fdc81ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 0x55fdc81ba0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57520/0 (socket says 172.21.15.78:57520) 2024-09-11T00:58:43.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605659c640 1 -- 172.21.15.78:0/1722607864 learned_addr learned my addr 172.21.15.78:0/1722607864 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:43.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f604ffff640 1 --2- 172.21.15.78:0/1722607864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819b650 0x55fdc81ba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605659c640 1 -- 172.21.15.78:0/1722607864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc81a75e0 msgr2=0x55fdc822c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:43.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605659c640 1 --2- 172.21.15.78:0/1722607864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc81a75e0 0x55fdc822c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605659c640 1 -- 172.21.15.78:0/1722607864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819b650 msgr2=0x55fdc81ba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605659c640 1 --2- 172.21.15.78:0/1722607864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819b650 0x55fdc81ba5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605659c640 1 -- 172.21.15.78:0/1722607864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc81995e0 con 0x55fdc819ad30 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f605659c640 1 --2- 172.21.15.78:0/1722607864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 0x55fdc81ba0a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6048005ab0 tx=0x7f6048005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eea89b0b4422400 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.134+0000 7f603dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f603dffb640 1 -- 172.21.15.78:0/1722607864 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604804b040 con 0x55fdc819ad30 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f603dffb640 1 -- 172.21.15.78:0/1722607864 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6048003c80 con 0x55fdc819ad30 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f605f24d740 1 -- 172.21.15.78:0/1722607864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6050019d20 con 0x55fdc819ad30 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f603dffb640 1 -- 172.21.15.78:0/1722607864 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6048035040 con 0x55fdc819ad30 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f605f24d740 10 client.15378 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f605f24d740 1 -- 172.21.15.78:0/1722607864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fdc822d590 con 0x55fdc819ad30 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f603dffb640 1 -- 172.21.15.78:0/1722607864 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6050019d20 con 0x55fdc819ad30 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f603dffb640 1 -- 172.21.15.78:0/1722607864 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6048004b70 con 0x55fdc819ad30 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f605f24d740 10 client.15378 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f605f24d740 10 client.15378 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f605f24d740 20 client.15378 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f605f24d740 1 --2- 172.21.15.78:0/1722607864 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55fdc8232190 0x55fdc8230ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f605f24d740 4 client.15378 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f605f24d740 1 -- 172.21.15.78:0/1722607864 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "status"}) -- 0x55fdc8110610 con 0x55fdc8232190 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.135+0000 7f604ffff640 1 --2- 172.21.15.78:0/1722607864 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55fdc8232190 0x55fdc8230ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f603dffb640 10 client.15378 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f604ffff640 1 --2- 172.21.15.78:0/1722607864 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55fdc8232190 0x55fdc8230ea0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c490fb3b028151c server_cookie=b66c0d8d062dda11 in_seq=0 out_seq=0 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f603dffb640 1 -- 172.21.15.78:0/1722607864 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55fdc8110610 con 0x55fdc8232190 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f603dffb640 10 client.15378 handle_command_reply: tid=0 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f605f24d740 1 client.15378 shutdown 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f605f24d740 1 -- 172.21.15.78:0/1722607864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 msgr2=0x55fdc81ba0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f605f24d740 1 --2- 172.21.15.78:0/1722607864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 0x55fdc81ba0a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6048005ab0 tx=0x7f6048005260 comp rx=0 tx=0).stop 2024-09-11T00:58:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f605f24d740 1 -- 172.21.15.78:0/1722607864 shutdown_connections 2024-09-11T00:58:43.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f605f24d740 1 --2- 172.21.15.78:0/1722607864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdc81a75e0 0x55fdc822c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f605f24d740 1 --2- 172.21.15.78:0/1722607864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fdc819b650 0x55fdc81ba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f605f24d740 1 --2- 172.21.15.78:0/1722607864 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55fdc8232190 0x55fdc8230ea0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f605f24d740 1 --2- 172.21.15.78:0/1722607864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fdc819ad30 0x55fdc81ba0a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f603dffb640 0 client.15378 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T00:58:43.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f605f24d740 1 -- 172.21.15.78:0/1722607864 >> 172.21.15.78:0/1722607864 conn(0x55fdc8198b60 msgr2=0x55fdc8199bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:43.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f605f24d740 1 -- 172.21.15.78:0/1722607864 shutdown_connections 2024-09-11T00:58:43.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f605f24d740 1 -- 172.21.15.78:0/1722607864 wait complete. 2024-09-11T00:58:43.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.136+0000 7f605f24d740 20 client.15378 trim_cache size 0 max 16384 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24580, 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 101.977898811, 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 130.968986747 2024-09-11T00:58:43.145 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:58:43.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.137+0000 7f605ce78640 1 -- 172.21.15.78:0/3884558724 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f60440b5330 msgr2=0x7f60440b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:43.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.137+0000 7f605ce78640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f60440b5330 0x7f60440b77f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f60500070b0 tx=0x7f60500093d0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.137+0000 7f605ce78640 1 -- 172.21.15.78:0/3884558724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60581076a0 msgr2=0x7f60581987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:43.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.137+0000 7f605ce78640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60581076a0 0x7f60581987e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f604000b4a0 tx=0x7f604000b960 comp rx=0 tx=0).stop 2024-09-11T00:58:43.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.137+0000 7f605ce78640 1 -- 172.21.15.78:0/3884558724 shutdown_connections 2024-09-11T00:58:43.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.137+0000 7f605ce78640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f60440b5330 0x7f60440b77f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.137+0000 7f605ce78640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6058108020 0x7f605819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.137+0000 7f605ce78640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60581076a0 0x7f60581987e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.137+0000 7f605ce78640 1 --2- 172.21.15.78:0/3884558724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6058101c90 0x7f60581982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.137+0000 7f605ce78640 1 -- 172.21.15.78:0/3884558724 >> 172.21.15.78:0/3884558724 conn(0x7f60580fd6e0 msgr2=0x7f60580ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.137+0000 7f605ce78640 1 -- 172.21.15.78:0/3884558724 shutdown_connections 2024-09-11T00:58:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:43.137+0000 7f605ce78640 1 -- 172.21.15.78:0/3884558724 wait complete. 2024-09-11T00:58:43.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:43 smithi110 ceph-mon[34712]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:43 smithi112 ceph-mon[35384]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:43.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:43 smithi078 ceph-mon[31315]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:43.961 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T00:58:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.742+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/3826250715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38101810 msgr2=0x7f3e38101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.742+0000 7f3e3f1a1640 1 --2- 172.21.15.78:0/3826250715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38101810 0x7f3e38101bf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009900 tx=0x7f3e3002a410 comp rx=0 tx=0).stop 2024-09-11T00:58:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.742+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/3826250715 shutdown_connections 2024-09-11T00:58:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.742+0000 7f3e3f1a1640 1 --2- 172.21.15.78:0/3826250715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3810d250 0x7f3e3810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.742+0000 7f3e3f1a1640 1 --2- 172.21.15.78:0/3826250715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e3810a920 0x7f3e3810cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.742+0000 7f3e3f1a1640 1 --2- 172.21.15.78:0/3826250715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38101810 0x7f3e38101bf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.742+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/3826250715 >> 172.21.15.78:0/3826250715 conn(0x7f3e38076f50 msgr2=0x7f3e38077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.742+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/3826250715 shutdown_connections 2024-09-11T00:58:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.742+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/3826250715 wait complete. 2024-09-11T00:58:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.742+0000 7f3e3f1a1640 1 Processor -- start 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3f1a1640 1 -- start start 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3f1a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38101810 0x7f3e38199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3f1a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3810a920 0x7f3e38199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3f1a1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e3810d250 0x7f3e3819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3f1a1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e38112d50 con 0x7f3e38101810 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3f1a1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3e38112bd0 con 0x7f3e3810d250 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3f1a1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e38112ed0 con 0x7f3e3810a920 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3cf3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38101810 0x7f3e38199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3cf3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38101810 0x7f3e38199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55266/0 (socket says 172.21.15.78:55266) 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3cf3d640 1 -- 172.21.15.78:0/2661349411 learned_addr learned my addr 172.21.15.78:0/2661349411 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e37fff640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3810a920 0x7f3e38199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3cf3d640 1 -- 172.21.15.78:0/2661349411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3810a920 msgr2=0x7f3e38199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3810a920 0x7f3e38199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3cf3d640 1 -- 172.21.15.78:0/2661349411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e3810d250 msgr2=0x7f3e3819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e3810d250 0x7f3e3819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3cf3d640 1 -- 172.21.15.78:0/2661349411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3819f3d0 con 0x7f3e38101810 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e37fff640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3810a920 0x7f3e38199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.743+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38101810 0x7f3e38199320 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3e30006010 tx=0x7f3e300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.744+0000 7f3e35ffb640 1 -- 172.21.15.78:0/2661349411 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3003f030 con 0x7f3e38101810 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.744+0000 7f3e35ffb640 1 -- 172.21.15.78:0/2661349411 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e30003800 con 0x7f3e38101810 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.744+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/2661349411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3819f660 con 0x7f3e38101810 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.744+0000 7f3e35ffb640 1 -- 172.21.15.78:0/2661349411 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30035070 con 0x7f3e38101810 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.744+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/2661349411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e381a5d80 con 0x7f3e38101810 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.746+0000 7f3e41576740 1 Processor -- start 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.746+0000 7f3e41576740 1 -- start start 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.746+0000 7f3e35ffb640 1 -- 172.21.15.78:0/2661349411 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f3e30031b10 con 0x7f3e38101810 2024-09-11T00:58:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e41576740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446747dd30 0x56446747e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e41576740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 0x56446748e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e41576740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446748e9f0 0x564467490de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e41576740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5644673f5c10 con 0x56446747e650 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e41576740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5644673e5040 con 0x56446748e9f0 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e41576740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5644673e8e50 con 0x56446747dd30 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e3cf3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 0x56446748e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e3cf3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 0x56446748e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55272/0 (socket says 172.21.15.78:55272) 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e3cf3d640 1 -- 172.21.15.78:0/2873072191 learned_addr learned my addr 172.21.15.78:0/2873072191 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e3d73e640 1 --2- 172.21.15.78:0/2873072191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446747dd30 0x56446747e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e37fff640 1 --2- 172.21.15.78:0/2873072191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446748e9f0 0x564467490de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e3cf3d640 1 -- 172.21.15.78:0/2873072191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446747dd30 msgr2=0x56446747e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/2873072191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446747dd30 0x56446747e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e3cf3d640 1 -- 172.21.15.78:0/2873072191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446748e9f0 msgr2=0x564467490de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/2873072191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446748e9f0 0x564467490de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e3cf3d640 1 -- 172.21.15.78:0/2873072191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564467143540 con 0x56446747e650 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e35ffb640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3e240b5330 0x7f3e240b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.747+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/2873072191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 0x56446748e4b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3e3010cc40 tx=0x7f3e3010cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47444f3d0e4c6e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e3d73e640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3e240b5330 0x7f3e240b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e2effd640 1 -- 172.21.15.78:0/2873072191 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300c9440 con 0x56446747e650 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e2effd640 1 -- 172.21.15.78:0/2873072191 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e300cc900 con 0x56446747e650 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e2effd640 1 -- 172.21.15.78:0/2873072191 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300ef890 con 0x56446747e650 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e35ffb640 1 -- 172.21.15.78:0/2661349411 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3e301066e0 con 0x7f3e38101810 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e41576740 1 -- 172.21.15.78:0/2873072191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 msgr2=0x56446748e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e3d73e640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3e240b5330 0x7f3e240b77f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3e20005fd0 tx=0x7f3e20005ec0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e41576740 1 --2- 172.21.15.78:0/2873072191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 0x56446748e4b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3e3010cc40 tx=0x7f3e3010cd30 comp rx=0 tx=0).stop 2024-09-11T00:58:44.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e41576740 1 -- 172.21.15.78:0/2873072191 shutdown_connections 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e41576740 1 --2- 172.21.15.78:0/2873072191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446748e9f0 0x564467490de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e41576740 1 --2- 172.21.15.78:0/2873072191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 0x56446748e4b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e41576740 1 --2- 172.21.15.78:0/2873072191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446747dd30 0x56446747e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.748+0000 7f3e41576740 1 -- 172.21.15.78:0/2873072191 >> 172.21.15.78:0/2873072191 conn(0x56446747bb60 msgr2=0x56446747cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e41576740 1 -- 172.21.15.78:0/2873072191 shutdown_connections 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e41576740 1 -- 172.21.15.78:0/2873072191 wait complete. 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e41576740 1 -- start start 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e41576740 1 Processor -- start 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e41576740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747dd30 0x56446748e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e41576740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747e650 0x56446748b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e41576740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446748e9f0 0x56446748c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e41576740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5644672465f0 con 0x56446747dd30 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e41576740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564467380010 con 0x56446747e650 2024-09-11T00:58:44.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e41576740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564467491460 con 0x56446748e9f0 2024-09-11T00:58:44.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e41576740 10 client.0 _ll_register_callbacks cb 0x5644673df340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:44.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e3cf3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747dd30 0x56446748e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e3cf3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747dd30 0x56446748e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55274/0 (socket says 172.21.15.78:55274) 2024-09-11T00:58:44.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.749+0000 7f3e3cf3d640 1 -- 172.21.15.78:0/126560668 learned_addr learned my addr 172.21.15.78:0/126560668 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:44.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e37fff640 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747e650 0x56446748b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e3d73e640 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446748e9f0 0x56446748c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e3cf3d640 1 -- 172.21.15.78:0/126560668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446748e9f0 msgr2=0x56446748c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:44.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446748e9f0 0x56446748c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e3cf3d640 1 -- 172.21.15.78:0/126560668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747e650 msgr2=0x56446748b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:44.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747e650 0x56446748b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e3cf3d640 1 -- 172.21.15.78:0/126560668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644673ada00 con 0x56446747dd30 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e3d73e640 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446748e9f0 0x56446748c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747dd30 0x56446748e4a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3e300ca060 tx=0x7f3e300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ee518dfea0efe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e17fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e17fff640 1 -- 172.21.15.78:0/126560668 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30049070 con 0x56446747dd30 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e17fff640 1 -- 172.21.15.78:0/126560668 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3e3010c790 con 0x56446747dd30 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e41576740 1 -- 172.21.15.78:0/126560668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e300cc900 con 0x56446747dd30 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e17fff640 1 -- 172.21.15.78:0/126560668 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e300efa60 con 0x56446747dd30 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e41576740 10 client.15390 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e41576740 1 -- 172.21.15.78:0/126560668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56446749cef0 con 0x56446747dd30 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.750+0000 7f3e17fff640 1 -- 172.21.15.78:0/126560668 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e300cc900 con 0x56446747dd30 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.751+0000 7f3e17fff640 1 -- 172.21.15.78:0/126560668 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3e300b4070 con 0x56446747dd30 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.751+0000 7f3e41576740 10 client.15390 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:44.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.751+0000 7f3e41576740 10 client.15390 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:58:44.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.751+0000 7f3e41576740 20 client.15390 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:44.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.751+0000 7f3e41576740 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56446749f160 0x5644674a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.751+0000 7f3e41576740 4 client.15390 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:58:44.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.751+0000 7f3e41576740 1 -- 172.21.15.78:0/126560668 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644673f7b60 con 0x56446749f160 2024-09-11T00:58:44.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.751+0000 7f3e37fff640 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56446749f160 0x5644674a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.751+0000 7f3e17fff640 10 client.15390 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:44.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.751+0000 7f3e37fff640 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56446749f160 0x5644674a05d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d8d74a10d93d228 server_cookie=4455b1409a7a8ad6 in_seq=0 out_seq=0 2024-09-11T00:58:44.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e17fff640 1 -- 172.21.15.78:0/126560668 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5644673f7b60 con 0x56446749f160 2024-09-11T00:58:44.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e17fff640 10 client.15390 handle_command_reply: tid=0 2024-09-11T00:58:44.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e41576740 1 client.15390 shutdown 2024-09-11T00:58:44.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e41576740 1 -- 172.21.15.78:0/126560668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747dd30 msgr2=0x56446748e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:44.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e41576740 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747dd30 0x56446748e4a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3e300ca060 tx=0x7f3e300057b0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e41576740 1 -- 172.21.15.78:0/126560668 shutdown_connections 2024-09-11T00:58:44.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e41576740 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446748e9f0 0x56446748c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e41576740 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747e650 0x56446748b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e41576740 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56446749f160 0x5644674a05d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e41576740 1 --2- 172.21.15.78:0/126560668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747dd30 0x56446748e4a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e17fff640 0 client.15390 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:44.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e41576740 1 -- 172.21.15.78:0/126560668 >> 172.21.15.78:0/126560668 conn(0x56446747bb60 msgr2=0x56446747ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:44.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e41576740 1 -- 172.21.15.78:0/126560668 shutdown_connections 2024-09-11T00:58:44.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e41576740 1 -- 172.21.15.78:0/126560668 wait complete. 2024-09-11T00:58:44.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.752+0000 7f3e41576740 20 client.15390 trim_cache size 0 max 16384 2024-09-11T00:58:44.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.770+0000 7f3e41576740 1 Processor -- start 2024-09-11T00:58:44.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.770+0000 7f3e41576740 1 -- start start 2024-09-11T00:58:44.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.770+0000 7f3e41576740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446747dd30 0x56446749f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.799 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:58:44.799 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "64384694-6fd7-11ef-bcea-c7b262605968", 2024-09-11T00:58:44.799 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T00:58:44.799 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24599, 2024-09-11T00:58:44.799 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T00:58:44.799 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T00:58:44.799 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T00:58:44.800 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 110.660784701, 2024-09-11T00:58:44.800 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-11T00:58:44.800 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-09-11T00:58:44.800 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-09-11T00:58:44.800 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 122.68583178999999 2024-09-11T00:58:44.800 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:58:44.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.770+0000 7f3e41576740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747e650 0x56446747b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.770+0000 7f3e41576740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446748e9f0 0x56446749e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.770+0000 7f3e41576740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56446749e910 con 0x56446748e9f0 2024-09-11T00:58:44.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.770+0000 7f3e41576740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56446749efe0 con 0x56446747e650 2024-09-11T00:58:44.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.770+0000 7f3e41576740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56446749f6b0 con 0x56446747dd30 2024-09-11T00:58:44.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.770+0000 7f3e3d73e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446748e9f0 0x56446749e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.770+0000 7f3e3d73e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446748e9f0 0x56446749e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55288/0 (socket says 172.21.15.78:55288) 2024-09-11T00:58:44.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.770+0000 7f3e3d73e640 1 -- 172.21.15.78:0/274290258 learned_addr learned my addr 172.21.15.78:0/274290258 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:44.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.770+0000 7f3e37fff640 1 --2- 172.21.15.78:0/274290258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747e650 0x56446747b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.771+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/274290258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446747dd30 0x56446749f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.771+0000 7f3e3d73e640 1 -- 172.21.15.78:0/274290258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446747dd30 msgr2=0x56446749f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:44.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.771+0000 7f3e3d73e640 1 --2- 172.21.15.78:0/274290258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446747dd30 0x56446749f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.771+0000 7f3e3d73e640 1 -- 172.21.15.78:0/274290258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747e650 msgr2=0x56446747b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:44.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.771+0000 7f3e3d73e640 1 --2- 172.21.15.78:0/274290258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747e650 0x56446747b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.771+0000 7f3e3d73e640 1 -- 172.21.15.78:0/274290258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56446748e7d0 con 0x56446748e9f0 2024-09-11T00:58:44.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.771+0000 7f3e37fff640 1 --2- 172.21.15.78:0/274290258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747e650 0x56446747b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:44.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.771+0000 7f3e3d73e640 1 --2- 172.21.15.78:0/274290258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446748e9f0 0x56446749e070 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000fda0 tx=0x7f3e20012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898c7cc42676b54a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:44.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.771+0000 7f3e2e7fc640 1 -- 172.21.15.78:0/274290258 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20016070 con 0x56446748e9f0 2024-09-11T00:58:44.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.771+0000 7f3e2e7fc640 1 -- 172.21.15.78:0/274290258 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e20010dc0 con 0x56446748e9f0 2024-09-11T00:58:44.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.771+0000 7f3e2e7fc640 1 -- 172.21.15.78:0/274290258 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2001a670 con 0x56446748e9f0 2024-09-11T00:58:44.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 -- 172.21.15.78:0/274290258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446748e9f0 msgr2=0x56446749e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:44.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 --2- 172.21.15.78:0/274290258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446748e9f0 0x56446749e070 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000fda0 tx=0x7f3e20012000 comp rx=0 tx=0).stop 2024-09-11T00:58:44.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 -- 172.21.15.78:0/274290258 shutdown_connections 2024-09-11T00:58:44.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 --2- 172.21.15.78:0/274290258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446748e9f0 0x56446749e070 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 --2- 172.21.15.78:0/274290258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747e650 0x56446747b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 --2- 172.21.15.78:0/274290258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446747dd30 0x56446749f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 -- 172.21.15.78:0/274290258 >> 172.21.15.78:0/274290258 conn(0x56446747bb60 msgr2=0x56446748b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:44.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 -- 172.21.15.78:0/274290258 shutdown_connections 2024-09-11T00:58:44.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 -- 172.21.15.78:0/274290258 wait complete. 2024-09-11T00:58:44.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 -- start start 2024-09-11T00:58:44.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 Processor -- start 2024-09-11T00:58:44.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747dd30 0x56446747ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 0x56446747f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446748e9f0 0x56446747fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5644674a0450 con 0x56446747e650 2024-09-11T00:58:44.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56446749fd80 con 0x56446747dd30 2024-09-11T00:58:44.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5644674a0b20 con 0x56446748e9f0 2024-09-11T00:58:44.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.772+0000 7f3e41576740 10 client.0 _ll_register_callbacks cb 0x7f3e2800a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:44.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e37fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 0x56446747f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e37fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 0x56446747f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55296/0 (socket says 172.21.15.78:55296) 2024-09-11T00:58:44.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e37fff640 1 -- 172.21.15.78:0/2705481822 learned_addr learned my addr 172.21.15.78:0/2705481822 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:44.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e3d73e640 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446748e9f0 0x56446747fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747dd30 0x56446747ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e37fff640 1 -- 172.21.15.78:0/2705481822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446748e9f0 msgr2=0x56446747fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:44.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e37fff640 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446748e9f0 0x56446747fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e37fff640 1 -- 172.21.15.78:0/2705481822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747dd30 msgr2=0x56446747ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:44.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e37fff640 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747dd30 0x56446747ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e37fff640 1 -- 172.21.15.78:0/2705481822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564467480350 con 0x56446747e650 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e37fff640 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 0x56446747f4a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5644674a0dd0 tx=0x7f3e28012d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39140bb5c1ca4275 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e17fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e17fff640 1 -- 172.21.15.78:0/2705481822 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28021040 con 0x56446747e650 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e17fff640 1 -- 172.21.15.78:0/2705481822 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3e28004d20 con 0x56446747e650 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e41576740 1 -- 172.21.15.78:0/2705481822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e20010dc0 con 0x56446747e650 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e17fff640 1 -- 172.21.15.78:0/2705481822 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e28003970 con 0x56446747e650 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e41576740 10 client.15396 fetch_fsmap learned FSMap version 28 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e41576740 1 -- 172.21.15.78:0/2705481822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56446750c5d0 con 0x56446747e650 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.773+0000 7f3e17fff640 1 -- 172.21.15.78:0/2705481822 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e20010dc0 con 0x56446747e650 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e17fff640 1 -- 172.21.15.78:0/2705481822 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3e28004020 con 0x56446747e650 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e41576740 10 client.15396 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e41576740 10 client.15396 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:58:44.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e41576740 20 client.15396 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e41576740 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564467511230 0x56446750ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e41576740 4 client.15396 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e41576740 1 -- 172.21.15.78:0/2705481822 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "status"}) -- 0x5644673f3610 con 0x564467511230 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564467511230 0x56446750ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e17fff640 10 client.15396 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e3cf3d640 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564467511230 0x56446750ff40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71c978887cec6deb server_cookie=99e7c31695d5516a in_seq=0 out_seq=0 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e17fff640 1 -- 172.21.15.78:0/2705481822 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5644673f3610 con 0x564467511230 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e17fff640 10 client.15396 handle_command_reply: tid=0 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.774+0000 7f3e41576740 1 client.15396 shutdown 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.775+0000 7f3e41576740 1 -- 172.21.15.78:0/2705481822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 msgr2=0x56446747f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.775+0000 7f3e41576740 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 0x56446747f4a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5644674a0dd0 tx=0x7f3e28012d30 comp rx=0 tx=0).stop 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.775+0000 7f3e41576740 1 -- 172.21.15.78:0/2705481822 shutdown_connections 2024-09-11T00:58:44.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.775+0000 7f3e41576740 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56446748e9f0 0x56446747fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.775+0000 7f3e41576740 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564467511230 0x56446750ff40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.775+0000 7f3e41576740 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56446747e650 0x56446747f4a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.775+0000 7f3e41576740 1 --2- 172.21.15.78:0/2705481822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56446747dd30 0x56446747ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.775+0000 7f3e41576740 1 -- 172.21.15.78:0/2705481822 >> 172.21.15.78:0/2705481822 conn(0x56446747bb60 msgr2=0x56446749e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:44.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.775+0000 7f3e17fff640 0 client.15396 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:44.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.775+0000 7f3e41576740 1 -- 172.21.15.78:0/2705481822 shutdown_connections 2024-09-11T00:58:44.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.775+0000 7f3e41576740 1 -- 172.21.15.78:0/2705481822 wait complete. 2024-09-11T00:58:44.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.775+0000 7f3e41576740 20 client.15396 trim_cache size 0 max 16384 2024-09-11T00:58:44.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.776+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/2661349411 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3e240b5330 msgr2=0x7f3e240b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:44.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.776+0000 7f3e3f1a1640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3e240b5330 0x7f3e240b77f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3e20005fd0 tx=0x7f3e20005ec0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.776+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/2661349411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38101810 msgr2=0x7f3e38199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:44.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.776+0000 7f3e3f1a1640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38101810 0x7f3e38199320 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3e30006010 tx=0x7f3e300042f0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.776+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/2661349411 shutdown_connections 2024-09-11T00:58:44.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.776+0000 7f3e3f1a1640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3e240b5330 0x7f3e240b77f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.776+0000 7f3e3f1a1640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e3810d250 0x7f3e3819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.776+0000 7f3e3f1a1640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3810a920 0x7f3e38199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.776+0000 7f3e3f1a1640 1 --2- 172.21.15.78:0/2661349411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38101810 0x7f3e38199320 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:44.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.776+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/2661349411 >> 172.21.15.78:0/2661349411 conn(0x7f3e38076f50 msgr2=0x7f3e381005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:44.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.776+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/2661349411 shutdown_connections 2024-09-11T00:58:44.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:44.776+0000 7f3e3f1a1640 1 -- 172.21.15.78:0/2661349411 wait complete. 2024-09-11T00:58:45.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:45 smithi078 ceph-mon[31315]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:58:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:45 smithi110 ceph-mon[34712]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:58:45.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:45 smithi112 ceph-mon[35384]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:58:45.581 INFO:teuthology.run_tasks:Running task kclient... 2024-09-11T00:58:45.607 INFO:tasks.kclient:Mounting kernel clients... 2024-09-11T00:58:45.608 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-11T00:58:45.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:58:45.608 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T00:58:45.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:58:45.624 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-09-11T00:58:45.692 INFO:teuthology.orchestra.run.smithi078.stderr:Cannot find device "ceph-brx" 2024-09-11T00:58:45.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:58:45.693 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-11T00:58:45.694 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T00:58:45.694 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T00:58:45.694 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T00:58:45.977 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.518+0000 7fb9249af640 1 -- 172.21.15.78:0/224102503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920108040 msgr2=0x7fb92010e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.518+0000 7fb9249af640 1 --2- 172.21.15.78:0/224102503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920108040 0x7fb92010e840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb918009ca0 tx=0x7fb91802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.518+0000 7fb9249af640 1 -- 172.21.15.78:0/224102503 shutdown_connections 2024-09-11T00:58:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.518+0000 7fb9249af640 1 --2- 172.21.15.78:0/224102503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920108040 0x7fb92010e840 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.518+0000 7fb9249af640 1 --2- 172.21.15.78:0/224102503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9201076c0 0x7fb920107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.518+0000 7fb9249af640 1 --2- 172.21.15.78:0/224102503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb920101cb0 0x7fb920102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.518+0000 7fb9249af640 1 -- 172.21.15.78:0/224102503 >> 172.21.15.78:0/224102503 conn(0x7fb9200fd700 msgr2=0x7fb9200ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.518+0000 7fb9249af640 1 -- 172.21.15.78:0/224102503 shutdown_connections 2024-09-11T00:58:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.518+0000 7fb9249af640 1 -- 172.21.15.78:0/224102503 wait complete. 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.518+0000 7fb9249af640 1 Processor -- start 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.518+0000 7fb9249af640 1 -- start start 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb9249af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920101cb0 0x7fb92019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb9249af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9201076c0 0x7fb92019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb9249af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb920108040 0x7fb92019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb9249af640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb920111cc0 con 0x7fb920101cb0 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb9249af640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb920111b40 con 0x7fb9201076c0 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb9249af640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb920111e40 con 0x7fb920108040 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb91e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920101cb0 0x7fb92019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb91e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920101cb0 0x7fb92019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55308/0 (socket says 172.21.15.78:55308) 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb91e59c640 1 -- 172.21.15.78:0/3772325999 learned_addr learned my addr 172.21.15.78:0/3772325999 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb91dd9b640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9201076c0 0x7fb92019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb91ed9d640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb920108040 0x7fb92019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb91e59c640 1 -- 172.21.15.78:0/3772325999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb920108040 msgr2=0x7fb92019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb91e59c640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb920108040 0x7fb92019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb91e59c640 1 -- 172.21.15.78:0/3772325999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9201076c0 msgr2=0x7fb92019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb91e59c640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9201076c0 0x7fb92019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb91e59c640 1 -- 172.21.15.78:0/3772325999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9201a5470 con 0x7fb920101cb0 2024-09-11T00:58:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb91e59c640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920101cb0 0x7fb92019a090 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb90800d950 tx=0x7fb90800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb9177fe640 1 -- 172.21.15.78:0/3772325999 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908017030 con 0x7fb920101cb0 2024-09-11T00:58:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb9177fe640 1 -- 172.21.15.78:0/3772325999 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb908004020 con 0x7fb920101cb0 2024-09-11T00:58:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb9177fe640 1 -- 172.21.15.78:0/3772325999 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9080047a0 con 0x7fb920101cb0 2024-09-11T00:58:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb9249af640 1 -- 172.21.15.78:0/3772325999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9201a5760 con 0x7fb920101cb0 2024-09-11T00:58:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.519+0000 7fb9249af640 1 -- 172.21.15.78:0/3772325999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9201a5cf0 con 0x7fb920101cb0 2024-09-11T00:58:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.520+0000 7fb9249af640 1 -- 172.21.15.78:0/3772325999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb920102090 con 0x7fb920101cb0 2024-09-11T00:58:46.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.522+0000 7fb9177fe640 1 -- 172.21.15.78:0/3772325999 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fb908009a30 con 0x7fb920101cb0 2024-09-11T00:58:46.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.523+0000 7fb9177fe640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb90c0b53b0 0x7fb90c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:46.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.523+0000 7fb91dd9b640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb90c0b53b0 0x7fb90c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:46.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.523+0000 7fb9177fe640 1 -- 172.21.15.78:0/3772325999 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb9080e5dc0 con 0x7fb920101cb0 2024-09-11T00:58:46.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.523+0000 7fb91dd9b640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb90c0b53b0 0x7fb90c0b7870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb92006c090 tx=0x7fb910005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:46.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.525+0000 7fb9177fe640 1 -- 172.21.15.78:0/3772325999 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb9080acc50 con 0x7fb920101cb0 2024-09-11T00:58:46.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.742+0000 7fb9249af640 1 -- 172.21.15.78:0/3772325999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb92002e5c0 con 0x7fb920101cb0 2024-09-11T00:58:46.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.742+0000 7fb9177fe640 1 -- 172.21.15.78:0/3772325999 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7fb9080ac550 con 0x7fb920101cb0 2024-09-11T00:58:46.744 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T00:58:46.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.744+0000 7fb9249af640 1 -- 172.21.15.78:0/3772325999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb90c0b53b0 msgr2=0x7fb90c0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:46.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.744+0000 7fb9249af640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb90c0b53b0 0x7fb90c0b7870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb92006c090 tx=0x7fb910005f90 comp rx=0 tx=0).stop 2024-09-11T00:58:46.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.744+0000 7fb9249af640 1 -- 172.21.15.78:0/3772325999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920101cb0 msgr2=0x7fb92019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:46.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.744+0000 7fb9249af640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920101cb0 0x7fb92019a090 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb90800d950 tx=0x7fb90800de10 comp rx=0 tx=0).stop 2024-09-11T00:58:46.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.744+0000 7fb9249af640 1 -- 172.21.15.78:0/3772325999 shutdown_connections 2024-09-11T00:58:46.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.744+0000 7fb9249af640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb90c0b53b0 0x7fb90c0b7870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:46.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.744+0000 7fb9249af640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb920108040 0x7fb92019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:46.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.744+0000 7fb9249af640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9201076c0 0x7fb92019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:46.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.744+0000 7fb9249af640 1 --2- 172.21.15.78:0/3772325999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920101cb0 0x7fb92019a090 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:46.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.744+0000 7fb9249af640 1 -- 172.21.15.78:0/3772325999 >> 172.21.15.78:0/3772325999 conn(0x7fb9200fd700 msgr2=0x7fb9200ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:46.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.744+0000 7fb9249af640 1 -- 172.21.15.78:0/3772325999 shutdown_connections 2024-09-11T00:58:46.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:46.744+0000 7fb9249af640 1 -- 172.21.15.78:0/3772325999 wait complete. 2024-09-11T00:58:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:47 smithi110 ceph-mon[34712]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:58:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:47 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3772325999' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:58:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:47 smithi112 ceph-mon[35384]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:58:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:47 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3772325999' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:58:47.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:47 smithi078 ceph-mon[31315]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:58:47.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:47 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3772325999' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:58:47.541 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T00:58:47.542 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T00:58:47.542 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T00:58:47.542 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T00:58:47.542 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T00:58:47.542 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T00:58:47.542 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T00:58:47.542 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T00:58:47.542 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T00:58:47.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:58:47.542 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T00:58:47.559 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T00:58:47.559 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T00:58:47.559 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T00:58:47.559 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:58:47.559 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T00:58:47.559 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:58:47.559 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:58:47.559 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T00:58:47.560 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T00:58:47.560 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:58:47.560 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T00:58:47.560 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T00:58:47.560 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:58:47.560 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T00:58:47.560 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T00:58:47.560 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T00:58:47.560 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-11T00:58:47.560 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42152sec preferred_lft 42152sec 2024-09-11T00:58:47.560 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link noprefixroute 2024-09-11T00:58:47.560 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:58:47.561 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T00:58:47.561 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:58:47.561 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T00:58:47.561 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add name ceph-brx type bridge 2024-09-11T00:58:47.561 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr flush dev ceph-brx 2024-09-11T00:58:47.561 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set ceph-brx up 2024-09-11T00:58:47.561 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T00:58:47.561 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T00:58:47.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:58:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:58:47.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:58:47.762 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T00:58:47.786 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T00:58:47.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:58:47.787 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T00:58:47.852 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T00:58:47.852 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T00:58:47.852 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T00:58:47.852 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T00:58:47.853 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T00:58:47.853 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:58:47.853 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T00:58:47.853 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T00:58:47.853 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T00:58:47.853 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T00:58:47.853 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T00:58:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:58:48.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:58:48.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:58:48.236 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T00:58:48.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:58:48.252 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2024-09-11T00:58:48.307 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:58:48.308 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T00:58:48.308 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T00:58:48.308 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-11T00:58:48.308 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T00:58:48.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:58:48.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:58:48.424 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-11T00:58:48.425 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:58:48.425 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T00:58:48.425 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-11T00:58:48.425 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T00:58:48.425 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-11T00:58:48.425 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-11T00:58:48.425 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-11T00:58:48.426 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T00:58:48.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:58:48.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:58:48.557 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:58:48.557 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T00:58:48.557 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T00:58:48.557 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T00:58:48.557 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T00:58:48.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:58:48.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:58:48.633 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T00:58:48.633 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T00:58:48.648 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T00:58:48.649 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T00:58:48.649 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T00:58:48.704 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-11T00:58:48.704 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-11T00:58:48.704 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/6ce94f05-2660-4611-9762-7179453c5d71 2024-09-11T00:58:48.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:58:48.705 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/6ce94f05-2660-4611-9762-7179453c5d71 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-11T00:58:48.800 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-11T00:58:48.801 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-11T00:58:48.801 INFO:teuthology.orchestra.run.smithi078.stdout:invalid new device string format 2024-09-11T00:58:48.933 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-11T00:58:48.933 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-11T00:58:48.933 INFO:teuthology.orchestra.run.smithi078.stdout:invalid new device string format 2024-09-11T00:58:48.933 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.110:3300,172.21.15.112:3300" 2024-09-11T00:58:48.933 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.78:3300,172.21.15.110:3300,172.21.15.112:3300:/volumes/qa/sv_1/6ce94f05-2660-4611-9762-7179453c5d71 2024-09-11T00:58:48.934 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=64384694-6fd7-11ef-bcea-c7b262605968" will pass to kernel 2024-09-11T00:58:48.934 INFO:teuthology.orchestra.run.smithi078.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-11T00:58:48.934 INFO:teuthology.orchestra.run.smithi078.stdout: You just mounted a file system that supports labels which does not 2024-09-11T00:58:48.934 INFO:teuthology.orchestra.run.smithi078.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-11T00:58:48.934 INFO:teuthology.orchestra.run.smithi078.stdout: applications will generate AVC messages and not be allowed access to 2024-09-11T00:58:48.934 INFO:teuthology.orchestra.run.smithi078.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-11T00:58:48.936 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T00:58:48.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:58:48.936 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T00:58:48.962 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T00:58:49.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T00:58:49.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:58:49.058 INFO:teuthology.orchestra.run.smithi078.stdout:client24899 2024-09-11T00:58:49.059 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:58:49.059 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T00:58:49.076 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T00:58:49.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T00:58:49.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:58:49.172 INFO:teuthology.orchestra.run.smithi078.stdout:client24899 2024-09-11T00:58:49.172 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T00:58:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T00:58:49.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:58:49.229 INFO:teuthology.orchestra.run.smithi078.stdout:client24899 2024-09-11T00:58:49.229 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/64384694-6fd7-11ef-bcea-c7b262605968.client24899/status') 2024-09-11T00:58:49.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:58:49.287 INFO:teuthology.orchestra.run.smithi078.stdout:instance: client.24899 (3)172.21.15.78:0/2958689368 2024-09-11T00:58:49.288 INFO:teuthology.orchestra.run.smithi078.stdout:blocklisted: false 2024-09-11T00:58:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records in 2024-09-11T00:58:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records out 2024-09-11T00:58:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:71 bytes copied, 2.186e-05 s, 3.2 MB/s 2024-09-11T00:58:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:58:49.295 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T00:58:49.307 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T00:58:49.307 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T00:58:49.307 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-11T00:58:49.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.407+0000 7ff79ee65640 1 -- 172.21.15.78:0/373994157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798104d70 msgr2=0x7ff79810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:49.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.407+0000 7ff79ee65640 1 --2- 172.21.15.78:0/373994157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798104d70 0x7ff79810e770 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff790009ca0 tx=0x7ff79002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:49.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.407+0000 7ff79ee65640 1 -- 172.21.15.78:0/373994157 shutdown_connections 2024-09-11T00:58:49.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.407+0000 7ff79ee65640 1 --2- 172.21.15.78:0/373994157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798104d70 0x7ff79810e770 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:49.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.407+0000 7ff79ee65640 1 --2- 172.21.15.78:0/373994157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7981043d0 0x7ff798104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:49.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.407+0000 7ff79ee65640 1 --2- 172.21.15.78:0/373994157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff798102010 0x7ff7981023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:49.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.407+0000 7ff79ee65640 1 -- 172.21.15.78:0/373994157 >> 172.21.15.78:0/373994157 conn(0x7ff798077720 msgr2=0x7ff798077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:49.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.407+0000 7ff79ee65640 1 -- 172.21.15.78:0/373994157 shutdown_connections 2024-09-11T00:58:49.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.407+0000 7ff79ee65640 1 -- 172.21.15.78:0/373994157 wait complete. 2024-09-11T00:58:49.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79ee65640 1 Processor -- start 2024-09-11T00:58:49.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79ee65640 1 -- start start 2024-09-11T00:58:49.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79ee65640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff798102010 0x7ff798199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:49.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79ee65640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7981043d0 0x7ff7981996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:49.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79ee65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798104d70 0x7ff79819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:49.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79ee65640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff798111c80 con 0x7ff798104d70 2024-09-11T00:58:49.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79ee65640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff798111b00 con 0x7ff798102010 2024-09-11T00:58:49.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79ee65640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff798111e00 con 0x7ff7981043d0 2024-09-11T00:58:49.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79d402640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798104d70 0x7ff79819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:49.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79d402640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798104d70 0x7ff79819ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55326/0 (socket says 172.21.15.78:55326) 2024-09-11T00:58:49.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79d402640 1 -- 172.21.15.78:0/4220692088 learned_addr learned my addr 172.21.15.78:0/4220692088 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:49.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79cc01640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff798102010 0x7ff798199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff797fff640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7981043d0 0x7ff7981996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79d402640 1 -- 172.21.15.78:0/4220692088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7981043d0 msgr2=0x7ff7981996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79d402640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7981043d0 0x7ff7981996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79d402640 1 -- 172.21.15.78:0/4220692088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff798102010 msgr2=0x7ff798199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79d402640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff798102010 0x7ff798199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79d402640 1 -- 172.21.15.78:0/4220692088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79819f230 con 0x7ff798104d70 2024-09-11T00:58:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79cc01640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff798102010 0x7ff798199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff797fff640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7981043d0 0x7ff7981996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.408+0000 7ff79d402640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798104d70 0x7ff79819ea60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff790009430 tx=0x7ff7900319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.409+0000 7ff795ffb640 1 -- 172.21.15.78:0/4220692088 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790003c30 con 0x7ff798104d70 2024-09-11T00:58:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.409+0000 7ff795ffb640 1 -- 172.21.15.78:0/4220692088 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff790003dd0 con 0x7ff798104d70 2024-09-11T00:58:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.409+0000 7ff795ffb640 1 -- 172.21.15.78:0/4220692088 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79002faa0 con 0x7ff798104d70 2024-09-11T00:58:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.409+0000 7ff79ee65640 1 -- 172.21.15.78:0/4220692088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79819f4c0 con 0x7ff798104d70 2024-09-11T00:58:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.409+0000 7ff79ee65640 1 -- 172.21.15.78:0/4220692088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7981a5be0 con 0x7ff798104d70 2024-09-11T00:58:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.410+0000 7ff79ee65640 1 -- 172.21.15.78:0/4220692088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7981023f0 con 0x7ff798104d70 2024-09-11T00:58:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.411+0000 7ff795ffb640 1 -- 172.21.15.78:0/4220692088 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7ff79002fc40 con 0x7ff798104d70 2024-09-11T00:58:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.412+0000 7ff795ffb640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff7800b5330 0x7ff7800b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.412+0000 7ff79cc01640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff7800b5330 0x7ff7800b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.412+0000 7ff795ffb640 1 -- 172.21.15.78:0/4220692088 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff790107290 con 0x7ff798104d70 2024-09-11T00:58:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.412+0000 7ff79cc01640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff7800b5330 0x7ff7800b77f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7880058c0 tx=0x7ff788005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.415+0000 7ff795ffb640 1 -- 172.21.15.78:0/4220692088 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff7900ce0a0 con 0x7ff798104d70 2024-09-11T00:58:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:49 smithi110 ceph-mon[34712]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:49 smithi112 ceph-mon[35384]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:49.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:49 smithi078 ceph-mon[31315]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:58:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:49.632+0000 7ff79ee65640 1 -- 172.21.15.78:0/4220692088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7ff7981a5fb0 con 0x7ff798104d70 2024-09-11T00:58:50.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.128+0000 7ff795ffb640 1 -- 172.21.15.78:0/4220692088 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7ff7900cd9a0 con 0x7ff798104d70 2024-09-11T00:58:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.131+0000 7ff79ee65640 1 -- 172.21.15.78:0/4220692088 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff7800b5330 msgr2=0x7ff7800b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.131+0000 7ff79ee65640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff7800b5330 0x7ff7800b77f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7880058c0 tx=0x7ff788005850 comp rx=0 tx=0).stop 2024-09-11T00:58:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.131+0000 7ff79ee65640 1 -- 172.21.15.78:0/4220692088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798104d70 msgr2=0x7ff79819ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.131+0000 7ff79ee65640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798104d70 0x7ff79819ea60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff790009430 tx=0x7ff7900319f0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.131+0000 7ff79ee65640 1 -- 172.21.15.78:0/4220692088 shutdown_connections 2024-09-11T00:58:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.131+0000 7ff79ee65640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff7800b5330 0x7ff7800b77f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.131+0000 7ff79ee65640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff798104d70 0x7ff79819ea60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.131+0000 7ff79ee65640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7981043d0 0x7ff7981996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.131+0000 7ff79ee65640 1 --2- 172.21.15.78:0/4220692088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff798102010 0x7ff798199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.131+0000 7ff79ee65640 1 -- 172.21.15.78:0/4220692088 >> 172.21.15.78:0/4220692088 conn(0x7ff798077720 msgr2=0x7ff79806a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.131+0000 7ff79ee65640 1 -- 172.21.15.78:0/4220692088 shutdown_connections 2024-09-11T00:58:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.131+0000 7ff79ee65640 1 -- 172.21.15.78:0/4220692088 wait complete. 2024-09-11T00:58:50.145 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-11T00:58:50.167 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-11T00:58:50.180 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T00:58:50.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.280+0000 7f7c147b4640 1 -- 172.21.15.78:0/2854002238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c1075c0 msgr2=0x7f7c0c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.280+0000 7f7c147b4640 1 --2- 172.21.15.78:0/2854002238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c1075c0 0x7f7c0c107a00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009900 tx=0x7f7c0002a410 comp rx=0 tx=0).stop 2024-09-11T00:58:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 -- 172.21.15.78:0/2854002238 shutdown_connections 2024-09-11T00:58:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 --2- 172.21.15.78:0/2854002238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c0c107f40 0x7f7c0c10e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 --2- 172.21.15.78:0/2854002238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c1075c0 0x7f7c0c107a00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 --2- 172.21.15.78:0/2854002238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c0c101fe0 0x7f7c0c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 -- 172.21.15.78:0/2854002238 >> 172.21.15.78:0/2854002238 conn(0x7f7c0c077670 msgr2=0x7f7c0c077a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 -- 172.21.15.78:0/2854002238 shutdown_connections 2024-09-11T00:58:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 -- 172.21.15.78:0/2854002238 wait complete. 2024-09-11T00:58:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 Processor -- start 2024-09-11T00:58:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 -- start start 2024-09-11T00:58:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c0c101fe0 0x7f7c0c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c0c1075c0 0x7f7c0c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c107f40 0x7f7c0c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c111c50 con 0x7f7c0c107f40 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c111ad0 con 0x7f7c0c101fe0 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c147b4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c111dd0 con 0x7f7c0c1075c0 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.281+0000 7f7c12d51640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c107f40 0x7f7c0c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c12d51640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c107f40 0x7f7c0c19d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55348/0 (socket says 172.21.15.78:55348) 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c12d51640 1 -- 172.21.15.78:0/963504965 learned_addr learned my addr 172.21.15.78:0/963504965 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c11d4f640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c0c1075c0 0x7f7c0c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c12550640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c0c101fe0 0x7f7c0c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c12d51640 1 -- 172.21.15.78:0/963504965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c0c1075c0 msgr2=0x7f7c0c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c12d51640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c0c1075c0 0x7f7c0c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c12d51640 1 -- 172.21.15.78:0/963504965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c0c101fe0 msgr2=0x7f7c0c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c12d51640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c0c101fe0 0x7f7c0c198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c12d51640 1 -- 172.21.15.78:0/963504965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c19e1b0 con 0x7f7c0c107f40 2024-09-11T00:58:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c12d51640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c107f40 0x7f7c0c19d9e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400d750 tx=0x7f7c0400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c0b7fe640 1 -- 172.21.15.78:0/963504965 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04017070 con 0x7f7c0c107f40 2024-09-11T00:58:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c0b7fe640 1 -- 172.21.15.78:0/963504965 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c040054d0 con 0x7f7c0c107f40 2024-09-11T00:58:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c147b4640 1 -- 172.21.15.78:0/963504965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0c19e4a0 con 0x7f7c0c107f40 2024-09-11T00:58:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c0b7fe640 1 -- 172.21.15.78:0/963504965 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04004c60 con 0x7f7c0c107f40 2024-09-11T00:58:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.282+0000 7f7c147b4640 1 -- 172.21.15.78:0/963504965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0c1a4b60 con 0x7f7c0c107f40 2024-09-11T00:58:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.283+0000 7f7c147b4640 1 -- 172.21.15.78:0/963504965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0c1023c0 con 0x7f7c0c107f40 2024-09-11T00:58:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.284+0000 7f7c0b7fe640 1 -- 172.21.15.78:0/963504965 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f7c04004e00 con 0x7f7c0c107f40 2024-09-11T00:58:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.285+0000 7f7c0b7fe640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7bf40b53b0 0x7f7bf40b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.285+0000 7f7c12550640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7bf40b53b0 0x7f7bf40b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.285+0000 7f7c0b7fe640 1 -- 172.21.15.78:0/963504965 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7c040e5b40 con 0x7f7c0c107f40 2024-09-11T00:58:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.286+0000 7f7c12550640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7bf40b53b0 0x7f7bf40b7870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc005c40 tx=0x7f7bfc008230 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:50.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.288+0000 7f7c0b7fe640 1 -- 172.21.15.78:0/963504965 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7c040ac9d0 con 0x7f7c0c107f40 2024-09-11T00:58:50.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:50 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/4220692088' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-11T00:58:50.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.465+0000 7f7c147b4640 1 -- 172.21.15.78:0/963504965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c0c1a4ef0 con 0x7f7c0c107f40 2024-09-11T00:58:50.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.467+0000 7f7c0b7fe640 1 -- 172.21.15.78:0/963504965 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22975 (secure 0 0 0) 0x7f7c040ac2d0 con 0x7f7c0c107f40 2024-09-11T00:58:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:50.469 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","created":"2024-09-11T00:48:27.652512+0000","modified":"2024-09-11T00:57:17.572848+0000","last_up_change":"2024-09-11T00:55:59.071507+0000","last_in_change":"2024-09-11T00:55:35.815150+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:52:10.229887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:56:49.939136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:56:51.703446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beefc6d9-f74c-4200-96f2-ff697ad0d7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6803","nonce":1424639053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6805","nonce":1424639053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6809","nonce":1424639053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6807","nonce":1424639053}]},"public_addr":"172.21.15.78:6803/1424639053","cluster_addr":"172.21.15.78:6805/1424639053","heartbeat_back_addr":"172.21.15.78:6809/1424639053","heartbeat_front_addr":"172.21.15.78:6807/1424639053","state":["exists","up"]},{"osd":1,"uuid":"d742e767-d673-4bb9-b624-8bc969f5ceb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6801","nonce":580939385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6803","nonce":580939385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6807","nonce":580939385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6805","nonce":580939385}]},"public_addr":"172.21.15.110:6801/580939385","cluster_addr":"172.21.15.110:6803/580939385","heartbeat_back_addr":"172.21.15.110:6807/580939385","heartbeat_front_addr":"172.21.15.110:6805/580939385","state":["exists","up"]},{"osd":2,"uuid":"b0106a5b-1465-4a50-96dd-934dbae49a4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6801","nonce":2983155763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6803","nonce":2983155763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6807","nonce":2983155763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6805","nonce":2983155763}]},"public_addr":"172.21.15.112:6801/2983155763","cluster_addr":"172.21.15.112:6803/2983155763","heartbeat_back_addr":"172.21.15.112:6807/2983155763","heartbeat_front_addr":"172.21.15.112:6805/2983155763","state":["exists","up"]},{"osd":3,"uuid":"3270362b-3d61-414a-a3d9-f2416ff8b870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6811","nonce":2906086464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6813","nonce":2906086464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6817","nonce":2906086464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6815","nonce":2906086464}]},"public_addr":"172.21.15.78:6811/2906086464","cluster_addr":"172.21.15.78:6813/2906086464","heartbeat_back_addr":"172.21.15.78:6817/2906086464","heartbeat_front_addr":"172.21.15.78:6815/2906086464","state":["exists","up"]},{"osd":4,"uuid":"e47746ec-9443-4004-abfa-17c1ed73b982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6809","nonce":477287435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6811","nonce":477287435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6815","nonce":477287435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6813","nonce":477287435}]},"public_addr":"172.21.15.110:6809/477287435","cluster_addr":"172.21.15.110:6811/477287435","heartbeat_back_addr":"172.21.15.110:6815/477287435","heartbeat_front_addr":"172.21.15.110:6813/477287435","state":["exists","up"]},{"osd":5,"uuid":"f8cedb01-310d-442d-a588-33197d32e208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6809","nonce":3557095096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6811","nonce":3557095096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6815","nonce":3557095096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6813","nonce":3557095096}]},"public_addr":"172.21.15.112:6809/3557095096","cluster_addr":"172.21.15.112:6811/3557095096","heartbeat_back_addr":"172.21.15.112:6815/3557095096","heartbeat_front_addr":"172.21.15.112:6813/3557095096","state":["exists","up"]},{"osd":6,"uuid":"d508a037-8f56-436b-b7bf-9a1c69f6d9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6819","nonce":1267910775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6821","nonce":1267910775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6825","nonce":1267910775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6823","nonce":1267910775}]},"public_addr":"172.21.15.78:6819/1267910775","cluster_addr":"172.21.15.78:6821/1267910775","heartbeat_back_addr":"172.21.15.78:6825/1267910775","heartbeat_front_addr":"172.21.15.78:6823/1267910775","state":["exists","up"]},{"osd":7,"uuid":"e8842047-09a6-48b8-8d0a-2d54e7f89063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6817","nonce":1556494594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6819","nonce":1556494594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6823","nonce":1556494594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6821","nonce":1556494594}]},"public_addr":"172.21.15.110:6817/1556494594","cluster_addr":"172.21.15.110:6819/1556494594","heartbeat_back_addr":"172.21.15.110:6823/1556494594","heartbeat_front_addr":"172.21.15.110:6821/1556494594","state":["exists","up"]},{"osd":8,"uuid":"a3367da8-375a-4a46-a952-867d4b72ae2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6817","nonce":2481321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6819","nonce":2481321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6823","nonce":2481321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6821","nonce":2481321}]},"public_addr":"172.21.15.112:6817/2481321","cluster_addr":"172.21.15.112:6819/2481321","heartbeat_back_addr":"172.21.15.112:6823/2481321","heartbeat_front_addr":"172.21.15.112:6821/2481321","state":["exists","up"]},{"osd":9,"uuid":"d1825764-cefc-495b-99da-8ba429df0937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6827","nonce":197048350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6829","nonce":197048350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6833","nonce":197048350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6831","nonce":197048350}]},"public_addr":"172.21.15.78:6827/197048350","cluster_addr":"172.21.15.78:6829/197048350","heartbeat_back_addr":"172.21.15.78:6833/197048350","heartbeat_front_addr":"172.21.15.78:6831/197048350","state":["exists","up"]},{"osd":10,"uuid":"e1cfff7a-8916-4845-a482-d79fa275da30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6825","nonce":2101123168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6827","nonce":2101123168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6831","nonce":2101123168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6829","nonce":2101123168}]},"public_addr":"172.21.15.110:6825/2101123168","cluster_addr":"172.21.15.110:6827/2101123168","heartbeat_back_addr":"172.21.15.110:6831/2101123168","heartbeat_front_addr":"172.21.15.110:6829/2101123168","state":["exists","up"]},{"osd":11,"uuid":"fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6825","nonce":1526310654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6827","nonce":1526310654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6831","nonce":1526310654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6829","nonce":1526310654}]},"public_addr":"172.21.15.112:6825/1526310654","cluster_addr":"172.21.15.112:6827/1526310654","heartbeat_back_addr":"172.21.15.112:6831/1526310654","heartbeat_front_addr":"172.21.15.112:6829/1526310654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:17.334199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:41.400872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:08.205144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:33.550527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:58.394116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:24.699607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:50.450193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:16.223363+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:40.947727+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:06.845451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:32.410035+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:57.744364+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/2390785703":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:0/1038951931":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/391908936":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2706514734":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/2357025243":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6800/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2795778473":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2747741101":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6801/1289343426":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/3558896007":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/438261739":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/1289343426":"2024-09-12T00:50:15.928079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:58:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.469+0000 7f7c147b4640 1 -- 172.21.15.78:0/963504965 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7bf40b53b0 msgr2=0x7f7bf40b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.469+0000 7f7c147b4640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7bf40b53b0 0x7f7bf40b7870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc005c40 tx=0x7f7bfc008230 comp rx=0 tx=0).stop 2024-09-11T00:58:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.469+0000 7f7c147b4640 1 -- 172.21.15.78:0/963504965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c107f40 msgr2=0x7f7c0c19d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.469+0000 7f7c147b4640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c107f40 0x7f7c0c19d9e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400d750 tx=0x7f7c0400dc10 comp rx=0 tx=0).stop 2024-09-11T00:58:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.469+0000 7f7c147b4640 1 -- 172.21.15.78:0/963504965 shutdown_connections 2024-09-11T00:58:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.469+0000 7f7c147b4640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7bf40b53b0 0x7f7bf40b7870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.469+0000 7f7c147b4640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c107f40 0x7f7c0c19d9e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.469+0000 7f7c147b4640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c0c1075c0 0x7f7c0c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.469+0000 7f7c147b4640 1 --2- 172.21.15.78:0/963504965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c0c101fe0 0x7f7c0c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.469+0000 7f7c147b4640 1 -- 172.21.15.78:0/963504965 >> 172.21.15.78:0/963504965 conn(0x7f7c0c077670 msgr2=0x7f7c0c103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:50.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.469+0000 7f7c147b4640 1 -- 172.21.15.78:0/963504965 shutdown_connections 2024-09-11T00:58:50.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.469+0000 7f7c147b4640 1 -- 172.21.15.78:0/963504965 wait complete. 2024-09-11T00:58:50.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:50 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/4220692088' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-11T00:58:50.481 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:50 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4220692088' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-11T00:58:50.482 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T00:52:10.229887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T00:56:49.939136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T00:56:51.703446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T00:58:50.483 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T00:58:50.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.584+0000 7f3967633640 1 -- 172.21.15.78:0/1723757111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960108700 msgr2=0x7f3960108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.584+0000 7f3967633640 1 --2- 172.21.15.78:0/1723757111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960108700 0x7f3960108b40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3954009900 tx=0x7f395402a410 comp rx=0 tx=0).stop 2024-09-11T00:58:50.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.584+0000 7f3967633640 1 -- 172.21.15.78:0/1723757111 shutdown_connections 2024-09-11T00:58:50.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.584+0000 7f3967633640 1 --2- 172.21.15.78:0/1723757111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3960109080 0x7f396010f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.584+0000 7f3967633640 1 --2- 172.21.15.78:0/1723757111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960108700 0x7f3960108b40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.584+0000 7f3967633640 1 --2- 172.21.15.78:0/1723757111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3960102d00 0x7f39601030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.584+0000 7f3967633640 1 -- 172.21.15.78:0/1723757111 >> 172.21.15.78:0/1723757111 conn(0x7f39600fe740 msgr2=0x7f3960100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.584+0000 7f3967633640 1 -- 172.21.15.78:0/1723757111 shutdown_connections 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f3967633640 1 -- 172.21.15.78:0/1723757111 wait complete. 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f3967633640 1 Processor -- start 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f3967633640 1 -- start start 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f3967633640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960102d00 0x7f3960199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f3967633640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3960108700 0x7f39601997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f3967633640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3960109080 0x7f396019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f3967633640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3960112d40 con 0x7f3960102d00 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f3967633640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3960112bc0 con 0x7f3960108700 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f3967633640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3960112ec0 con 0x7f3960109080 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f39653cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960102d00 0x7f3960199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f39653cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960102d00 0x7f3960199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55372/0 (socket says 172.21.15.78:55372) 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f39653cf640 1 -- 172.21.15.78:0/1156013237 learned_addr learned my addr 172.21.15.78:0/1156013237 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.585+0000 7f3965bd0640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3960109080 0x7f396019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f39653cf640 1 -- 172.21.15.78:0/1156013237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3960109080 msgr2=0x7f396019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f39653cf640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3960109080 0x7f396019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f39653cf640 1 -- 172.21.15.78:0/1156013237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3960108700 msgr2=0x7f39601997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f3964bce640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3960108700 0x7f39601997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f39653cf640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3960108700 0x7f39601997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f39653cf640 1 -- 172.21.15.78:0/1156013237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396019f300 con 0x7f3960102d00 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f3965bd0640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3960109080 0x7f396019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f39653cf640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960102d00 0x7f3960199260 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f395000ea80 tx=0x7f395000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f395e7fc640 1 -- 172.21.15.78:0/1156013237 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395000ce40 con 0x7f3960102d00 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f3967633640 1 -- 172.21.15.78:0/1156013237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396019f5f0 con 0x7f3960102d00 2024-09-11T00:58:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f395e7fc640 1 -- 172.21.15.78:0/1156013237 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3950005020 con 0x7f3960102d00 2024-09-11T00:58:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f395e7fc640 1 -- 172.21.15.78:0/1156013237 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39500116d0 con 0x7f3960102d00 2024-09-11T00:58:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.586+0000 7f3967633640 1 -- 172.21.15.78:0/1156013237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39601a5cb0 con 0x7f3960102d00 2024-09-11T00:58:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.587+0000 7f3967633640 1 -- 172.21.15.78:0/1156013237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39601030e0 con 0x7f3960102d00 2024-09-11T00:58:50.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.588+0000 7f395e7fc640 1 -- 172.21.15.78:0/1156013237 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f395000f040 con 0x7f3960102d00 2024-09-11T00:58:50.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.589+0000 7f395e7fc640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f39480b5330 0x7f39480b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:50.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.589+0000 7f3964bce640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f39480b5330 0x7f39480b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:50.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.589+0000 7f395e7fc640 1 -- 172.21.15.78:0/1156013237 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f39500e60e0 con 0x7f3960102d00 2024-09-11T00:58:50.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.590+0000 7f3964bce640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f39480b5330 0x7f39480b77f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3954006010 tx=0x7f3954000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:50.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.592+0000 7f395e7fc640 1 -- 172.21.15.78:0/1156013237 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3950016030 con 0x7f3960102d00 2024-09-11T00:58:50.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.768+0000 7f3967633640 1 -- 172.21.15.78:0/1156013237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f396002e5c0 con 0x7f3960102d00 2024-09-11T00:58:50.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.769+0000 7f395e7fc640 1 -- 172.21.15.78:0/1156013237 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f39500acef0 con 0x7f3960102d00 2024-09-11T00:58:50.770 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T00:58:50.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.771+0000 7f3967633640 1 -- 172.21.15.78:0/1156013237 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f39480b5330 msgr2=0x7f39480b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.771+0000 7f3967633640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f39480b5330 0x7f39480b77f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3954006010 tx=0x7f3954000bc0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.771+0000 7f3967633640 1 -- 172.21.15.78:0/1156013237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960102d00 msgr2=0x7f3960199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.771+0000 7f3967633640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960102d00 0x7f3960199260 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f395000ea80 tx=0x7f395000ef40 comp rx=0 tx=0).stop 2024-09-11T00:58:50.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.771+0000 7f3967633640 1 -- 172.21.15.78:0/1156013237 shutdown_connections 2024-09-11T00:58:50.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.771+0000 7f3967633640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f39480b5330 0x7f39480b77f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.771+0000 7f3967633640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3960109080 0x7f396019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.771+0000 7f3967633640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3960108700 0x7f39601997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.771+0000 7f3967633640 1 --2- 172.21.15.78:0/1156013237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960102d00 0x7f3960199260 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.771+0000 7f3967633640 1 -- 172.21.15.78:0/1156013237 >> 172.21.15.78:0/1156013237 conn(0x7f39600fe740 msgr2=0x7f3960100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:50.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.771+0000 7f3967633640 1 -- 172.21.15.78:0/1156013237 shutdown_connections 2024-09-11T00:58:50.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.771+0000 7f3967633640 1 -- 172.21.15.78:0/1156013237 wait complete. 2024-09-11T00:58:50.783 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T00:58:50.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.885+0000 7f2721247640 1 -- 172.21.15.78:0/2575795386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c109050 msgr2=0x7f271c10f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.885+0000 7f2721247640 1 --2- 172.21.15.78:0/2575795386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c109050 0x7f271c10f850 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2714009ca0 tx=0x7f271402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.885+0000 7f2721247640 1 -- 172.21.15.78:0/2575795386 shutdown_connections 2024-09-11T00:58:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.885+0000 7f2721247640 1 --2- 172.21.15.78:0/2575795386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c109050 0x7f271c10f850 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.885+0000 7f2721247640 1 --2- 172.21.15.78:0/2575795386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f271c1086d0 0x7f271c108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.885+0000 7f2721247640 1 --2- 172.21.15.78:0/2575795386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c102cd0 0x7f271c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.885+0000 7f2721247640 1 -- 172.21.15.78:0/2575795386 >> 172.21.15.78:0/2575795386 conn(0x7f271c0fe710 msgr2=0x7f271c100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.885+0000 7f2721247640 1 -- 172.21.15.78:0/2575795386 shutdown_connections 2024-09-11T00:58:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.885+0000 7f2721247640 1 -- 172.21.15.78:0/2575795386 wait complete. 2024-09-11T00:58:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f2721247640 1 Processor -- start 2024-09-11T00:58:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f2721247640 1 -- start start 2024-09-11T00:58:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f2721247640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c102cd0 0x7f271c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f2721247640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f271c1086d0 0x7f271c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f2721247640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c109050 0x7f271c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f2721247640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f271c112d10 con 0x7f271c102cd0 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f2721247640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f271c112b90 con 0x7f271c1086d0 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f2721247640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f271c112e90 con 0x7f271c109050 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f271ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c102cd0 0x7f271c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f271ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c102cd0 0x7f271c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55400/0 (socket says 172.21.15.78:55400) 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f271ad9d640 1 -- 172.21.15.78:0/4275547088 learned_addr learned my addr 172.21.15.78:0/4275547088 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f271b59e640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c109050 0x7f271c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f271259c640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f271c1086d0 0x7f271c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f271ad9d640 1 -- 172.21.15.78:0/4275547088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c109050 msgr2=0x7f271c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f271ad9d640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c109050 0x7f271c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f271ad9d640 1 -- 172.21.15.78:0/4275547088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f271c1086d0 msgr2=0x7f271c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f271ad9d640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f271c1086d0 0x7f271c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.886+0000 7f271ad9d640 1 -- 172.21.15.78:0/4275547088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c19f2b0 con 0x7f271c102cd0 2024-09-11T00:58:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.887+0000 7f271ad9d640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c102cd0 0x7f271c199210 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f270c00c970 tx=0x7f270c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.887+0000 7f2718d99640 1 -- 172.21.15.78:0/4275547088 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c007bb0 con 0x7f271c102cd0 2024-09-11T00:58:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.887+0000 7f2721247640 1 -- 172.21.15.78:0/4275547088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c19f5a0 con 0x7f271c102cd0 2024-09-11T00:58:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.887+0000 7f2718d99640 1 -- 172.21.15.78:0/4275547088 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f270c007d50 con 0x7f271c102cd0 2024-09-11T00:58:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.887+0000 7f2718d99640 1 -- 172.21.15.78:0/4275547088 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c00f450 con 0x7f271c102cd0 2024-09-11T00:58:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.887+0000 7f2721247640 1 -- 172.21.15.78:0/4275547088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c1a5c60 con 0x7f271c102cd0 2024-09-11T00:58:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.888+0000 7f2721247640 1 -- 172.21.15.78:0/4275547088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271c1030b0 con 0x7f271c102cd0 2024-09-11T00:58:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.890+0000 7f2718d99640 1 -- 172.21.15.78:0/4275547088 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f270c004f00 con 0x7f271c102cd0 2024-09-11T00:58:50.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.890+0000 7f2718d99640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27080b5330 0x7f27080b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:50.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.891+0000 7f271259c640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27080b5330 0x7f27080b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:50.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.891+0000 7f2718d99640 1 -- 172.21.15.78:0/4275547088 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f270c0e5e80 con 0x7f271c102cd0 2024-09-11T00:58:50.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.891+0000 7f271259c640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27080b5330 0x7f27080b77f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f271c106c60 tx=0x7f270400a4c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:50.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:50.893+0000 7f2718d99640 1 -- 172.21.15.78:0/4275547088 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f270c014030 con 0x7f271c102cd0 2024-09-11T00:58:51.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.068+0000 7f2721247640 1 -- 172.21.15.78:0/4275547088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f271c02e5c0 con 0x7f271c102cd0 2024-09-11T00:58:51.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.069+0000 7f2718d99640 1 -- 172.21.15.78:0/4275547088 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f270c0acc90 con 0x7f271c102cd0 2024-09-11T00:58:51.070 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-09-11T00:58:51.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.071+0000 7f2721247640 1 -- 172.21.15.78:0/4275547088 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27080b5330 msgr2=0x7f27080b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:51.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.071+0000 7f2721247640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27080b5330 0x7f27080b77f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f271c106c60 tx=0x7f270400a4c0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.071+0000 7f2721247640 1 -- 172.21.15.78:0/4275547088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c102cd0 msgr2=0x7f271c199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:51.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.071+0000 7f2721247640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c102cd0 0x7f271c199210 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f270c00c970 tx=0x7f270c00ce30 comp rx=0 tx=0).stop 2024-09-11T00:58:51.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.071+0000 7f2721247640 1 -- 172.21.15.78:0/4275547088 shutdown_connections 2024-09-11T00:58:51.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.071+0000 7f2721247640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27080b5330 0x7f27080b77f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.071+0000 7f2721247640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c109050 0x7f271c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.071+0000 7f2721247640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f271c1086d0 0x7f271c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.071+0000 7f2721247640 1 --2- 172.21.15.78:0/4275547088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c102cd0 0x7f271c199210 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.071+0000 7f2721247640 1 -- 172.21.15.78:0/4275547088 >> 172.21.15.78:0/4275547088 conn(0x7f271c0fe710 msgr2=0x7f271c100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:51.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.071+0000 7f2721247640 1 -- 172.21.15.78:0/4275547088 shutdown_connections 2024-09-11T00:58:51.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.071+0000 7f2721247640 1 -- 172.21.15.78:0/4275547088 wait complete. 2024-09-11T00:58:51.083 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T00:58:51.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.187+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1500164505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76cc108700 msgr2=0x7f76cc108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:51.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.187+0000 7f76d0d3a640 1 --2- 172.21.15.78:0/1500164505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76cc108700 0x7f76cc108b40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009900 tx=0x7f76b402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.187+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1500164505 shutdown_connections 2024-09-11T00:58:51.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.187+0000 7f76d0d3a640 1 --2- 172.21.15.78:0/1500164505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76cc109080 0x7f76cc10f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.187+0000 7f76d0d3a640 1 --2- 172.21.15.78:0/1500164505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76cc108700 0x7f76cc108b40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.187+0000 7f76d0d3a640 1 --2- 172.21.15.78:0/1500164505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc102d00 0x7f76cc1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.187+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1500164505 >> 172.21.15.78:0/1500164505 conn(0x7f76cc0fe740 msgr2=0x7f76cc100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:51.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.187+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1500164505 shutdown_connections 2024-09-11T00:58:51.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.187+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1500164505 wait complete. 2024-09-11T00:58:51.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.188+0000 7f76d0d3a640 1 Processor -- start 2024-09-11T00:58:51.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.188+0000 7f76d0d3a640 1 -- start start 2024-09-11T00:58:51.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.188+0000 7f76d0d3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76cc102d00 0x7f76cc1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:51.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.188+0000 7f76d0d3a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc108700 0x7f76cc199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:51.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.188+0000 7f76d0d3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76cc109080 0x7f76cc19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:51.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.188+0000 7f76d0d3a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc112d00 con 0x7f76cc109080 2024-09-11T00:58:51.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.188+0000 7f76d0d3a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc112b80 con 0x7f76cc108700 2024-09-11T00:58:51.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.188+0000 7f76d0d3a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc112e80 con 0x7f76cc102d00 2024-09-11T00:58:51.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.188+0000 7f76cad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76cc109080 0x7f76cc19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:51.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.188+0000 7f76cad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76cc109080 0x7f76cc19ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55428/0 (socket says 172.21.15.78:55428) 2024-09-11T00:58:51.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.188+0000 7f76cad9d640 1 -- 172.21.15.78:0/1363366268 learned_addr learned my addr 172.21.15.78:0/1363366268 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:51.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76c1d9b640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc108700 0x7f76cc199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:51.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76ca59c640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76cc102d00 0x7f76cc1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76cad9d640 1 -- 172.21.15.78:0/1363366268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76cc102d00 msgr2=0x7f76cc1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76cad9d640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76cc102d00 0x7f76cc1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76cad9d640 1 -- 172.21.15.78:0/1363366268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc108700 msgr2=0x7f76cc199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76cad9d640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc108700 0x7f76cc199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76cad9d640 1 -- 172.21.15.78:0/1363366268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76cc19f260 con 0x7f76cc109080 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76ca59c640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76cc102d00 0x7f76cc1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76cad9d640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76cc109080 0x7f76cc19ea90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f76c400eaa0 tx=0x7f76c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76c3fff640 1 -- 172.21.15.78:0/1363366268 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c400cdb0 con 0x7f76cc109080 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76c3fff640 1 -- 172.21.15.78:0/1363366268 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c4005020 con 0x7f76cc109080 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1363366268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76cc19f550 con 0x7f76cc109080 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76c3fff640 1 -- 172.21.15.78:0/1363366268 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c40116d0 con 0x7f76cc109080 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.189+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1363366268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76cc1a5c10 con 0x7f76cc109080 2024-09-11T00:58:51.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.190+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1363366268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76cc1030e0 con 0x7f76cc109080 2024-09-11T00:58:51.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.191+0000 7f76c3fff640 1 -- 172.21.15.78:0/1363366268 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f76c400f040 con 0x7f76cc109080 2024-09-11T00:58:51.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.192+0000 7f76c3fff640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f76b80b5330 0x7f76b80b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:51.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.192+0000 7f76ca59c640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f76b80b5330 0x7f76b80b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:51.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.192+0000 7f76c3fff640 1 -- 172.21.15.78:0/1363366268 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f76c40e5fb0 con 0x7f76cc109080 2024-09-11T00:58:51.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.192+0000 7f76ca59c640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f76b80b5330 0x7f76b80b77f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00baa0 tx=0x7f76bc005d90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:51.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.195+0000 7f76c3fff640 1 -- 172.21.15.78:0/1363366268 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f76c4016030 con 0x7f76cc109080 2024-09-11T00:58:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.370+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1363366268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f76cc02e5c0 con 0x7f76cc109080 2024-09-11T00:58:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.371+0000 7f76c3fff640 1 -- 172.21.15.78:0/1363366268 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f76c40acdc0 con 0x7f76cc109080 2024-09-11T00:58:51.372 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-09-11T00:58:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.373+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1363366268 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f76b80b5330 msgr2=0x7f76b80b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.373+0000 7f76d0d3a640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f76b80b5330 0x7f76b80b77f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00baa0 tx=0x7f76bc005d90 comp rx=0 tx=0).stop 2024-09-11T00:58:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.373+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1363366268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76cc109080 msgr2=0x7f76cc19ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.373+0000 7f76d0d3a640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76cc109080 0x7f76cc19ea90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f76c400eaa0 tx=0x7f76c400ef60 comp rx=0 tx=0).stop 2024-09-11T00:58:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.373+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1363366268 shutdown_connections 2024-09-11T00:58:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.373+0000 7f76d0d3a640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f76b80b5330 0x7f76b80b77f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.373+0000 7f76d0d3a640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76cc109080 0x7f76cc19ea90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.373+0000 7f76d0d3a640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc108700 0x7f76cc199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.373+0000 7f76d0d3a640 1 --2- 172.21.15.78:0/1363366268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76cc102d00 0x7f76cc1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.373+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1363366268 >> 172.21.15.78:0/1363366268 conn(0x7f76cc0fe740 msgr2=0x7f76cc100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.373+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1363366268 shutdown_connections 2024-09-11T00:58:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:51.373+0000 7f76d0d3a640 1 -- 172.21.15.78:0/1363366268 wait complete. 2024-09-11T00:58:51.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:51 smithi078 ceph-mon[31315]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T00:58:51.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:51 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4220692088' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T00:58:51.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:51 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:58:51.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:51 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/963504965' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:58:51.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:51 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1156013237' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:58:51.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:51 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/4275547088' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-11T00:58:51.387 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-11T00:58:51.387 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:58:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:51 smithi110 ceph-mon[34712]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T00:58:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:51 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/4220692088' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T00:58:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:51 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:58:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:51 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/963504965' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:58:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:51 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1156013237' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:58:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:51 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/4275547088' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-11T00:58:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:51 smithi112 ceph-mon[35384]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T00:58:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:51 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/4220692088' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T00:58:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:51 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:58:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:51 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/963504965' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:58:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:51 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1156013237' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:58:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:51 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/4275547088' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-11T00:58:51.633 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:52.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.118+0000 7fd920654640 1 -- 172.21.15.78:0/705043313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9181076a0 msgr2=0x7fd918107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.118+0000 7fd920654640 1 --2- 172.21.15.78:0/705043313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9181076a0 0x7fd918107ae0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009900 tx=0x7fd90c02a410 comp rx=0 tx=0).stop 2024-09-11T00:58:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.118+0000 7fd920654640 1 -- 172.21.15.78:0/705043313 shutdown_connections 2024-09-11T00:58:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.118+0000 7fd920654640 1 --2- 172.21.15.78:0/705043313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd918108020 0x7fd91810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.118+0000 7fd920654640 1 --2- 172.21.15.78:0/705043313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9181076a0 0x7fd918107ae0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.118+0000 7fd920654640 1 --2- 172.21.15.78:0/705043313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd918101c90 0x7fd918102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.118+0000 7fd920654640 1 -- 172.21.15.78:0/705043313 >> 172.21.15.78:0/705043313 conn(0x7fd9180fd700 msgr2=0x7fd9180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.118+0000 7fd920654640 1 -- 172.21.15.78:0/705043313 shutdown_connections 2024-09-11T00:58:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.118+0000 7fd920654640 1 -- 172.21.15.78:0/705043313 wait complete. 2024-09-11T00:58:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd920654640 1 Processor -- start 2024-09-11T00:58:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd920654640 1 -- start start 2024-09-11T00:58:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd920654640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd918101c90 0x7fd91819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd920654640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181076a0 0x7fd91819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd920654640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd918108020 0x7fd91819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd920654640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd918111ce0 con 0x7fd918101c90 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd920654640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd918111b60 con 0x7fd918108020 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd920654640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd918111e60 con 0x7fd9181076a0 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd91e3f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd918101c90 0x7fd91819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd91e3f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd918101c90 0x7fd91819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55444/0 (socket says 172.21.15.78:55444) 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd91e3f0640 1 -- 172.21.15.78:0/3855293555 learned_addr learned my addr 172.21.15.78:0/3855293555 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd91ebf1640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd918108020 0x7fd91819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd91dbef640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181076a0 0x7fd91819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd91e3f0640 1 -- 172.21.15.78:0/3855293555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181076a0 msgr2=0x7fd91819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd91e3f0640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181076a0 0x7fd91819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd91e3f0640 1 -- 172.21.15.78:0/3855293555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd918108020 msgr2=0x7fd91819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd91e3f0640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd918108020 0x7fd91819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd91e3f0640 1 -- 172.21.15.78:0/3855293555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9181a5450 con 0x7fd918101c90 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd91ebf1640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd918108020 0x7fd91819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.119+0000 7fd91dbef640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181076a0 0x7fd91819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.120+0000 7fd91e3f0640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd918101c90 0x7fd91819a070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd91000eab0 tx=0x7fd91000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.120+0000 7fd9177fe640 1 -- 172.21.15.78:0/3855293555 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91000ce40 con 0x7fd918101c90 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.120+0000 7fd9177fe640 1 -- 172.21.15.78:0/3855293555 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd910011920 con 0x7fd918101c90 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.120+0000 7fd9177fe640 1 -- 172.21.15.78:0/3855293555 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910010a30 con 0x7fd918101c90 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.120+0000 7fd920654640 1 -- 172.21.15.78:0/3855293555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9181a5740 con 0x7fd918101c90 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.120+0000 7fd920654640 1 -- 172.21.15.78:0/3855293555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9181a5c80 con 0x7fd918101c90 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.121+0000 7fd920654640 1 -- 172.21.15.78:0/3855293555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd918102070 con 0x7fd918101c90 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.122+0000 7fd9177fe640 1 -- 172.21.15.78:0/3855293555 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fd91000f040 con 0x7fd918101c90 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.123+0000 7fd9177fe640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd9000b5330 0x7fd9000b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.123+0000 7fd91dbef640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd9000b5330 0x7fd9000b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.123+0000 7fd9177fe640 1 -- 172.21.15.78:0/3855293555 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd9100e6090 con 0x7fd918101c90 2024-09-11T00:58:52.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.123+0000 7fd91dbef640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd9000b5330 0x7fd9000b77f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000c00 tx=0x7fd90c031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:52.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.126+0000 7fd9177fe640 1 -- 172.21.15.78:0/3855293555 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd910016030 con 0x7fd918101c90 2024-09-11T00:58:52.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.342+0000 7fd920654640 1 -- 172.21.15.78:0/3855293555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd91806aec0 con 0x7fd918101c90 2024-09-11T00:58:52.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:52 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1363366268' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-11T00:58:52.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.342+0000 7fd9177fe640 1 -- 172.21.15.78:0/3855293555 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11979 (secure 0 0 0) 0x7fd9100acea0 con 0x7fd918101c90 2024-09-11T00:58:52.344 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:52.344 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"btime":"2024-09-11T00:58:50:118172+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T00:58:49.633286+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T00:58:52.345 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-09-11T00:58:52.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.345+0000 7fd920654640 1 -- 172.21.15.78:0/3855293555 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd9000b5330 msgr2=0x7fd9000b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:52.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.345+0000 7fd920654640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd9000b5330 0x7fd9000b77f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000c00 tx=0x7fd90c031400 comp rx=0 tx=0).stop 2024-09-11T00:58:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.345+0000 7fd920654640 1 -- 172.21.15.78:0/3855293555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd918101c90 msgr2=0x7fd91819a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.345+0000 7fd920654640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd918101c90 0x7fd91819a070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd91000eab0 tx=0x7fd91000ef70 comp rx=0 tx=0).stop 2024-09-11T00:58:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.345+0000 7fd920654640 1 -- 172.21.15.78:0/3855293555 shutdown_connections 2024-09-11T00:58:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.345+0000 7fd920654640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd9000b5330 0x7fd9000b77f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.345+0000 7fd920654640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd918108020 0x7fd91819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.345+0000 7fd920654640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181076a0 0x7fd91819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.345+0000 7fd920654640 1 --2- 172.21.15.78:0/3855293555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd918101c90 0x7fd91819a070 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.345+0000 7fd920654640 1 -- 172.21.15.78:0/3855293555 >> 172.21.15.78:0/3855293555 conn(0x7fd9180fd700 msgr2=0x7fd9180ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.345+0000 7fd920654640 1 -- 172.21.15.78:0/3855293555 shutdown_connections 2024-09-11T00:58:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:52.345+0000 7fd920654640 1 -- 172.21.15.78:0/3855293555 wait complete. 2024-09-11T00:58:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:52 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1363366268' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-11T00:58:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:52 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1363366268' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-11T00:58:53.136 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-11T00:58:53.136 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-11T00:58:53.136 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T00:58:53.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.238+0000 7f6f85bc6640 1 -- 172.21.15.78:0/810307679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f80108720 msgr2=0x7f6f80108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.238+0000 7f6f85bc6640 1 --2- 172.21.15.78:0/810307679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f80108720 0x7f6f80108b60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009900 tx=0x7f6f6802a410 comp rx=0 tx=0).stop 2024-09-11T00:58:53.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.238+0000 7f6f85bc6640 1 -- 172.21.15.78:0/810307679 shutdown_connections 2024-09-11T00:58:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.238+0000 7f6f85bc6640 1 --2- 172.21.15.78:0/810307679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f801090a0 0x7f6f8010f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.238+0000 7f6f85bc6640 1 --2- 172.21.15.78:0/810307679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f80108720 0x7f6f80108b60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.238+0000 7f6f85bc6640 1 --2- 172.21.15.78:0/810307679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f80102d20 0x7f6f80103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.238+0000 7f6f85bc6640 1 -- 172.21.15.78:0/810307679 >> 172.21.15.78:0/810307679 conn(0x7f6f800fe780 msgr2=0x7f6f80100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.238+0000 7f6f85bc6640 1 -- 172.21.15.78:0/810307679 shutdown_connections 2024-09-11T00:58:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.238+0000 7f6f85bc6640 1 -- 172.21.15.78:0/810307679 wait complete. 2024-09-11T00:58:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.238+0000 7f6f85bc6640 1 Processor -- start 2024-09-11T00:58:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.238+0000 7f6f85bc6640 1 -- start start 2024-09-11T00:58:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f85bc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f80102d20 0x7f6f80199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f85bc6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f80108720 0x7f6f80199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f85bc6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f801090a0 0x7f6f8019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f85bc6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80112db0 con 0x7f6f801090a0 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f85bc6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80112c30 con 0x7f6f80108720 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f85bc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80112f30 con 0x7f6f80102d20 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f7ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f801090a0 0x7f6f8019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f7ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f801090a0 0x7f6f8019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55458/0 (socket says 172.21.15.78:55458) 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f7ffff640 1 -- 172.21.15.78:0/3390162227 learned_addr learned my addr 172.21.15.78:0/3390162227 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f7f7fe640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f80102d20 0x7f6f80199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f76ffd640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f80108720 0x7f6f80199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f7ffff640 1 -- 172.21.15.78:0/3390162227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f80102d20 msgr2=0x7f6f80199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f7ffff640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f80102d20 0x7f6f80199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f7ffff640 1 -- 172.21.15.78:0/3390162227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f80108720 msgr2=0x7f6f80199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f7ffff640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f80108720 0x7f6f80199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f7ffff640 1 -- 172.21.15.78:0/3390162227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8019f2b0 con 0x7f6f801090a0 2024-09-11T00:58:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.239+0000 7f6f7ffff640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f801090a0 0x7f6f8019eae0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800eaa0 tx=0x7f6f7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:53.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.240+0000 7f6f7d7fa640 1 -- 172.21.15.78:0/3390162227 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7800cdb0 con 0x7f6f801090a0 2024-09-11T00:58:53.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.240+0000 7f6f7d7fa640 1 -- 172.21.15.78:0/3390162227 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f78005020 con 0x7f6f801090a0 2024-09-11T00:58:53.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.240+0000 7f6f85bc6640 1 -- 172.21.15.78:0/3390162227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8019f5a0 con 0x7f6f801090a0 2024-09-11T00:58:53.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.240+0000 7f6f7d7fa640 1 -- 172.21.15.78:0/3390162227 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f780116d0 con 0x7f6f801090a0 2024-09-11T00:58:53.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.240+0000 7f6f85bc6640 1 -- 172.21.15.78:0/3390162227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f801a5e40 con 0x7f6f801090a0 2024-09-11T00:58:53.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.241+0000 7f6f85bc6640 1 -- 172.21.15.78:0/3390162227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f80103100 con 0x7f6f801090a0 2024-09-11T00:58:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.243+0000 7f6f7d7fa640 1 -- 172.21.15.78:0/3390162227 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f6f7800f040 con 0x7f6f801090a0 2024-09-11T00:58:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.244+0000 7f6f7d7fa640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6f6c0b53b0 0x7f6f6c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.244+0000 7f6f7f7fe640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6f6c0b53b0 0x7f6f6c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.244+0000 7f6f7d7fa640 1 -- 172.21.15.78:0/3390162227 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6f780e5e10 con 0x7f6f801090a0 2024-09-11T00:58:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.244+0000 7f6f7f7fe640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6f6c0b53b0 0x7f6f6c0b7870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000bbe0 tx=0x7f6f7000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.246+0000 7f6f7d7fa640 1 -- 172.21.15.78:0/3390162227 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6f78016030 con 0x7f6f801090a0 2024-09-11T00:58:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.423+0000 7f6f85bc6640 1 -- 172.21.15.78:0/3390162227 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f80105ce0 con 0x7f6f6c0b53b0 2024-09-11T00:58:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.427+0000 7f6f7d7fa640 1 -- 172.21.15.78:0/3390162227 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259130 (secure 0 0 0) 0x7f6f80105ce0 con 0x7f6f6c0b53b0 2024-09-11T00:58:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T00:58:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.431+0000 7f6f85bc6640 1 -- 172.21.15.78:0/3390162227 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6f6c0b53b0 msgr2=0x7f6f6c0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.431+0000 7f6f85bc6640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6f6c0b53b0 0x7f6f6c0b7870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000bbe0 tx=0x7f6f7000b390 comp rx=0 tx=0).stop 2024-09-11T00:58:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.431+0000 7f6f85bc6640 1 -- 172.21.15.78:0/3390162227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f801090a0 msgr2=0x7f6f8019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.431+0000 7f6f85bc6640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f801090a0 0x7f6f8019eae0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800eaa0 tx=0x7f6f7800ef60 comp rx=0 tx=0).stop 2024-09-11T00:58:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.431+0000 7f6f85bc6640 1 -- 172.21.15.78:0/3390162227 shutdown_connections 2024-09-11T00:58:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.431+0000 7f6f85bc6640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6f6c0b53b0 0x7f6f6c0b7870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.431+0000 7f6f85bc6640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f801090a0 0x7f6f8019eae0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.431+0000 7f6f85bc6640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f80108720 0x7f6f80199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.431+0000 7f6f85bc6640 1 --2- 172.21.15.78:0/3390162227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f80102d20 0x7f6f80199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.431+0000 7f6f85bc6640 1 -- 172.21.15.78:0/3390162227 >> 172.21.15.78:0/3390162227 conn(0x7f6f800fe780 msgr2=0x7f6f80100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.431+0000 7f6f85bc6640 1 -- 172.21.15.78:0/3390162227 shutdown_connections 2024-09-11T00:58:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.431+0000 7f6f85bc6640 1 -- 172.21.15.78:0/3390162227 wait complete. 2024-09-11T00:58:53.445 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:53 smithi110 ceph-mon[34712]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-11T00:58:53.445 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:53 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3855293555' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:53.445 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:53 smithi078 ceph-mon[31315]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-11T00:58:53.446 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:53 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3855293555' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:53.447 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-11T00:58:52.048680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658768,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1182,"num_read_kb":693,"num_write":379,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2952965,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":812,"ondisk_log_size":812,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376352,"kb_used_data":10464,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565472,"statfs":{"total":1151940427776,"available":1151555043328,"internally_reserved":0,"allocated":10715136,"data_stored":6959704,"data_compressed":179373,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":51,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002996"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266212+0000","last_change":"2024-09-11T00:56:53.123622+0000","last_active":"2024-09-11T00:58:16.266212+0000","last_peered":"2024-09-11T00:58:16.266212+0000","last_clean":"2024-09-11T00:58:16.266212+0000","last_became_active":"2024-09-11T00:56:53.121460+0000","last_became_peered":"2024-09-11T00:56:53.121460+0000","last_unstale":"2024-09-11T00:58:16.266212+0000","last_undegraded":"2024-09-11T00:58:16.266212+0000","last_fullsized":"2024-09-11T00:58:16.266212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:30.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264953+0000","last_change":"2024-09-11T00:56:53.120189+0000","last_active":"2024-09-11T00:58:16.264953+0000","last_peered":"2024-09-11T00:58:16.264953+0000","last_clean":"2024-09-11T00:58:16.264953+0000","last_became_active":"2024-09-11T00:56:53.119853+0000","last_became_peered":"2024-09-11T00:56:53.119853+0000","last_unstale":"2024-09-11T00:58:16.264953+0000","last_undegraded":"2024-09-11T00:58:16.264953+0000","last_fullsized":"2024-09-11T00:58:16.264953+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:36.927113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.272647+0000","last_change":"2024-09-11T00:56:53.117832+0000","last_active":"2024-09-11T00:58:16.272647+0000","last_peered":"2024-09-11T00:58:16.272647+0000","last_clean":"2024-09-11T00:58:16.272647+0000","last_became_active":"2024-09-11T00:56:53.117508+0000","last_became_peered":"2024-09-11T00:56:53.117508+0000","last_unstale":"2024-09-11T00:58:16.272647+0000","last_undegraded":"2024-09-11T00:58:16.272647+0000","last_fullsized":"2024-09-11T00:58:16.272647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:00.628673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.271210+0000","last_change":"2024-09-11T00:56:53.128302+0000","last_active":"2024-09-11T00:58:16.271210+0000","last_peered":"2024-09-11T00:58:16.271210+0000","last_clean":"2024-09-11T00:58:16.271210+0000","last_became_active":"2024-09-11T00:56:53.127584+0000","last_became_peered":"2024-09-11T00:56:53.127584+0000","last_unstale":"2024-09-11T00:58:16.271210+0000","last_undegraded":"2024-09-11T00:58:16.271210+0000","last_fullsized":"2024-09-11T00:58:16.271210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:41.963656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.248383+0000","last_change":"2024-09-11T00:56:53.116249+0000","last_active":"2024-09-11T00:58:16.248383+0000","last_peered":"2024-09-11T00:58:16.248383+0000","last_clean":"2024-09-11T00:58:16.248383+0000","last_became_active":"2024-09-11T00:56:53.115881+0000","last_became_peered":"2024-09-11T00:56:53.115881+0000","last_unstale":"2024-09-11T00:58:16.248383+0000","last_undegraded":"2024-09-11T00:58:16.248383+0000","last_fullsized":"2024-09-11T00:58:16.248383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:47.432607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.262557+0000","last_change":"2024-09-11T00:56:53.116992+0000","last_active":"2024-09-11T00:58:16.262557+0000","last_peered":"2024-09-11T00:58:16.262557+0000","last_clean":"2024-09-11T00:58:16.262557+0000","last_became_active":"2024-09-11T00:56:53.116629+0000","last_became_peered":"2024-09-11T00:56:53.116629+0000","last_unstale":"2024-09-11T00:58:16.262557+0000","last_undegraded":"2024-09-11T00:58:16.262557+0000","last_fullsized":"2024-09-11T00:58:16.262557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:25.410377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266888+0000","last_change":"2024-09-11T00:56:53.127873+0000","last_active":"2024-09-11T00:58:16.266888+0000","last_peered":"2024-09-11T00:58:16.266888+0000","last_clean":"2024-09-11T00:58:16.266888+0000","last_became_active":"2024-09-11T00:56:53.127575+0000","last_became_peered":"2024-09-11T00:56:53.127575+0000","last_unstale":"2024-09-11T00:58:16.266888+0000","last_undegraded":"2024-09-11T00:58:16.266888+0000","last_fullsized":"2024-09-11T00:58:16.266888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:12.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.261154+0000","last_change":"2024-09-11T00:56:53.117727+0000","last_active":"2024-09-11T00:58:16.261154+0000","last_peered":"2024-09-11T00:58:16.261154+0000","last_clean":"2024-09-11T00:58:16.261154+0000","last_became_active":"2024-09-11T00:56:53.117444+0000","last_became_peered":"2024-09-11T00:56:53.117444+0000","last_unstale":"2024-09-11T00:58:16.261154+0000","last_undegraded":"2024-09-11T00:58:16.261154+0000","last_fullsized":"2024-09-11T00:58:16.261154+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:28.625231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268315+0000","last_change":"2024-09-11T00:56:53.128974+0000","last_active":"2024-09-11T00:58:16.268315+0000","last_peered":"2024-09-11T00:58:16.268315+0000","last_clean":"2024-09-11T00:58:16.268315+0000","last_became_active":"2024-09-11T00:56:53.128650+0000","last_became_peered":"2024-09-11T00:56:53.128650+0000","last_unstale":"2024-09-11T00:58:16.268315+0000","last_undegraded":"2024-09-11T00:58:16.268315+0000","last_fullsized":"2024-09-11T00:58:16.268315+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:07.251373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.262269+0000","last_change":"2024-09-11T00:56:53.117816+0000","last_active":"2024-09-11T00:58:16.262269+0000","last_peered":"2024-09-11T00:58:16.262269+0000","last_clean":"2024-09-11T00:58:16.262269+0000","last_became_active":"2024-09-11T00:56:53.117449+0000","last_became_peered":"2024-09-11T00:56:53.117449+0000","last_unstale":"2024-09-11T00:58:16.262269+0000","last_undegraded":"2024-09-11T00:58:16.262269+0000","last_fullsized":"2024-09-11T00:58:16.262269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:09.058869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265129+0000","last_change":"2024-09-11T00:56:53.124616+0000","last_active":"2024-09-11T00:58:16.265129+0000","last_peered":"2024-09-11T00:58:16.265129+0000","last_clean":"2024-09-11T00:58:16.265129+0000","last_became_active":"2024-09-11T00:56:53.124309+0000","last_became_peered":"2024-09-11T00:56:53.124309+0000","last_unstale":"2024-09-11T00:58:16.265129+0000","last_undegraded":"2024-09-11T00:58:16.265129+0000","last_fullsized":"2024-09-11T00:58:16.265129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:14.469909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.257580+0000","last_change":"2024-09-11T00:56:53.126263+0000","last_active":"2024-09-11T00:58:16.257580+0000","last_peered":"2024-09-11T00:58:16.257580+0000","last_clean":"2024-09-11T00:58:16.257580+0000","last_became_active":"2024-09-11T00:56:53.125923+0000","last_became_peered":"2024-09-11T00:56:53.125923+0000","last_unstale":"2024-09-11T00:58:16.257580+0000","last_undegraded":"2024-09-11T00:58:16.257580+0000","last_fullsized":"2024-09-11T00:58:16.257580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:12:33.627372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.258526+0000","last_change":"2024-09-11T00:56:53.105292+0000","last_active":"2024-09-11T00:58:16.258526+0000","last_peered":"2024-09-11T00:58:16.258526+0000","last_clean":"2024-09-11T00:58:16.258526+0000","last_became_active":"2024-09-11T00:56:53.104830+0000","last_became_peered":"2024-09-11T00:56:53.104830+0000","last_unstale":"2024-09-11T00:58:16.258526+0000","last_undegraded":"2024-09-11T00:58:16.258526+0000","last_fullsized":"2024-09-11T00:58:16.258526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:14.136952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264895+0000","last_change":"2024-09-11T00:56:53.120463+0000","last_active":"2024-09-11T00:58:16.264895+0000","last_peered":"2024-09-11T00:58:16.264895+0000","last_clean":"2024-09-11T00:58:16.264895+0000","last_became_active":"2024-09-11T00:56:53.120167+0000","last_became_peered":"2024-09-11T00:56:53.120167+0000","last_unstale":"2024-09-11T00:58:16.264895+0000","last_undegraded":"2024-09-11T00:58:16.264895+0000","last_fullsized":"2024-09-11T00:58:16.264895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:49.783452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265508+0000","last_change":"2024-09-11T00:56:53.123312+0000","last_active":"2024-09-11T00:58:16.265508+0000","last_peered":"2024-09-11T00:58:16.265508+0000","last_clean":"2024-09-11T00:58:16.265508+0000","last_became_active":"2024-09-11T00:56:53.123015+0000","last_became_peered":"2024-09-11T00:56:53.123015+0000","last_unstale":"2024-09-11T00:58:16.265508+0000","last_undegraded":"2024-09-11T00:58:16.265508+0000","last_fullsized":"2024-09-11T00:58:16.265508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:22.069629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.272553+0000","last_change":"2024-09-11T00:56:53.125701+0000","last_active":"2024-09-11T00:58:16.272553+0000","last_peered":"2024-09-11T00:58:16.272553+0000","last_clean":"2024-09-11T00:58:16.272553+0000","last_became_active":"2024-09-11T00:56:53.125362+0000","last_became_peered":"2024-09-11T00:56:53.125362+0000","last_unstale":"2024-09-11T00:58:16.272553+0000","last_undegraded":"2024-09-11T00:58:16.272553+0000","last_fullsized":"2024-09-11T00:58:16.272553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:14.594324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266425+0000","last_change":"2024-09-11T00:56:53.124275+0000","last_active":"2024-09-11T00:58:16.266425+0000","last_peered":"2024-09-11T00:58:16.266425+0000","last_clean":"2024-09-11T00:58:16.266425+0000","last_became_active":"2024-09-11T00:56:53.122695+0000","last_became_peered":"2024-09-11T00:56:53.122695+0000","last_unstale":"2024-09-11T00:58:16.266425+0000","last_undegraded":"2024-09-11T00:58:16.266425+0000","last_fullsized":"2024-09-11T00:58:16.266425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:23.955266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264639+0000","last_change":"2024-09-11T00:56:53.119077+0000","last_active":"2024-09-11T00:58:16.264639+0000","last_peered":"2024-09-11T00:58:16.264639+0000","last_clean":"2024-09-11T00:58:16.264639+0000","last_became_active":"2024-09-11T00:56:53.118693+0000","last_became_peered":"2024-09-11T00:56:53.118693+0000","last_unstale":"2024-09-11T00:58:16.264639+0000","last_undegraded":"2024-09-11T00:58:16.264639+0000","last_fullsized":"2024-09-11T00:58:16.264639+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:39.135391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265439+0000","last_change":"2024-09-11T00:56:53.115430+0000","last_active":"2024-09-11T00:58:16.265439+0000","last_peered":"2024-09-11T00:58:16.265439+0000","last_clean":"2024-09-11T00:58:16.265439+0000","last_became_active":"2024-09-11T00:56:53.115111+0000","last_became_peered":"2024-09-11T00:56:53.115111+0000","last_unstale":"2024-09-11T00:58:16.265439+0000","last_undegraded":"2024-09-11T00:58:16.265439+0000","last_fullsized":"2024-09-11T00:58:16.265439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:11.306661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264277+0000","last_change":"2024-09-11T00:57:17.583260+0000","last_active":"2024-09-11T00:58:16.264277+0000","last_peered":"2024-09-11T00:58:16.264277+0000","last_clean":"2024-09-11T00:58:16.264277+0000","last_became_active":"2024-09-11T00:57:17.582909+0000","last_became_peered":"2024-09-11T00:57:17.582909+0000","last_unstale":"2024-09-11T00:58:16.264277+0000","last_undegraded":"2024-09-11T00:58:16.264277+0000","last_fullsized":"2024-09-11T00:58:16.264277+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:42.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268531+0000","last_change":"2024-09-11T00:56:53.121303+0000","last_active":"2024-09-11T00:58:16.268531+0000","last_peered":"2024-09-11T00:58:16.268531+0000","last_clean":"2024-09-11T00:58:16.268531+0000","last_became_active":"2024-09-11T00:56:53.121004+0000","last_became_peered":"2024-09-11T00:56:53.121004+0000","last_unstale":"2024-09-11T00:58:16.268531+0000","last_undegraded":"2024-09-11T00:58:16.268531+0000","last_fullsized":"2024-09-11T00:58:16.268531+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:36.899870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266931+0000","last_change":"2024-09-11T00:56:53.125042+0000","last_active":"2024-09-11T00:58:16.266931+0000","last_peered":"2024-09-11T00:58:16.266931+0000","last_clean":"2024-09-11T00:58:16.266931+0000","last_became_active":"2024-09-11T00:56:53.124661+0000","last_became_peered":"2024-09-11T00:56:53.124661+0000","last_unstale":"2024-09-11T00:58:16.266931+0000","last_undegraded":"2024-09-11T00:58:16.266931+0000","last_fullsized":"2024-09-11T00:58:16.266931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:34.392172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268728+0000","last_change":"2024-09-11T00:56:53.116544+0000","last_active":"2024-09-11T00:58:16.268728+0000","last_peered":"2024-09-11T00:58:16.268728+0000","last_clean":"2024-09-11T00:58:16.268728+0000","last_became_active":"2024-09-11T00:56:53.116262+0000","last_became_peered":"2024-09-11T00:56:53.116262+0000","last_unstale":"2024-09-11T00:58:16.268728+0000","last_undegraded":"2024-09-11T00:58:16.268728+0000","last_fullsized":"2024-09-11T00:58:16.268728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:34.500669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.263134+0000","last_change":"2024-09-11T00:56:53.116967+0000","last_active":"2024-09-11T00:58:16.263134+0000","last_peered":"2024-09-11T00:58:16.263134+0000","last_clean":"2024-09-11T00:58:16.263134+0000","last_became_active":"2024-09-11T00:56:53.116405+0000","last_became_peered":"2024-09-11T00:56:53.116405+0000","last_unstale":"2024-09-11T00:58:16.263134+0000","last_undegraded":"2024-09-11T00:58:16.263134+0000","last_fullsized":"2024-09-11T00:58:16.263134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:33.521964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268124+0000","last_change":"2024-09-11T00:56:53.116727+0000","last_active":"2024-09-11T00:58:16.268124+0000","last_peered":"2024-09-11T00:58:16.268124+0000","last_clean":"2024-09-11T00:58:16.268124+0000","last_became_active":"2024-09-11T00:56:53.116416+0000","last_became_peered":"2024-09-11T00:56:53.116416+0000","last_unstale":"2024-09-11T00:58:16.268124+0000","last_undegraded":"2024-09-11T00:58:16.268124+0000","last_fullsized":"2024-09-11T00:58:16.268124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:28.277909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265487+0000","last_change":"2024-09-11T00:56:53.123141+0000","last_active":"2024-09-11T00:58:16.265487+0000","last_peered":"2024-09-11T00:58:16.265487+0000","last_clean":"2024-09-11T00:58:16.265487+0000","last_became_active":"2024-09-11T00:56:53.122811+0000","last_became_peered":"2024-09-11T00:56:53.122811+0000","last_unstale":"2024-09-11T00:58:16.265487+0000","last_undegraded":"2024-09-11T00:58:16.265487+0000","last_fullsized":"2024-09-11T00:58:16.265487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:47.175120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.260452+0000","last_change":"2024-09-11T00:56:53.124213+0000","last_active":"2024-09-11T00:58:16.260452+0000","last_peered":"2024-09-11T00:58:16.260452+0000","last_clean":"2024-09-11T00:58:16.260452+0000","last_became_active":"2024-09-11T00:56:53.123757+0000","last_became_peered":"2024-09-11T00:56:53.123757+0000","last_unstale":"2024-09-11T00:58:16.260452+0000","last_undegraded":"2024-09-11T00:58:16.260452+0000","last_fullsized":"2024-09-11T00:58:16.260452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:54.263476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.260255+0000","last_change":"2024-09-11T00:56:53.114016+0000","last_active":"2024-09-11T00:58:16.260255+0000","last_peered":"2024-09-11T00:58:16.260255+0000","last_clean":"2024-09-11T00:58:16.260255+0000","last_became_active":"2024-09-11T00:56:53.113719+0000","last_became_peered":"2024-09-11T00:56:53.113719+0000","last_unstale":"2024-09-11T00:58:16.260255+0000","last_undegraded":"2024-09-11T00:58:16.260255+0000","last_fullsized":"2024-09-11T00:58:16.260255+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:48:06.623810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.258203+0000","last_change":"2024-09-11T00:56:53.124798+0000","last_active":"2024-09-11T00:58:16.258203+0000","last_peered":"2024-09-11T00:58:16.258203+0000","last_clean":"2024-09-11T00:58:16.258203+0000","last_became_active":"2024-09-11T00:56:53.124381+0000","last_became_peered":"2024-09-11T00:56:53.124381+0000","last_unstale":"2024-09-11T00:58:16.258203+0000","last_undegraded":"2024-09-11T00:58:16.258203+0000","last_fullsized":"2024-09-11T00:58:16.258203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:37.823112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265603+0000","last_change":"2024-09-11T00:56:53.115297+0000","last_active":"2024-09-11T00:58:16.265603+0000","last_peered":"2024-09-11T00:58:16.265603+0000","last_clean":"2024-09-11T00:58:16.265603+0000","last_became_active":"2024-09-11T00:56:53.114958+0000","last_became_peered":"2024-09-11T00:56:53.114958+0000","last_unstale":"2024-09-11T00:58:16.265603+0000","last_undegraded":"2024-09-11T00:58:16.265603+0000","last_fullsized":"2024-09-11T00:58:16.265603+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:47.612610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268541+0000","last_change":"2024-09-11T00:56:53.126717+0000","last_active":"2024-09-11T00:58:16.268541+0000","last_peered":"2024-09-11T00:58:16.268541+0000","last_clean":"2024-09-11T00:58:16.268541+0000","last_became_active":"2024-09-11T00:56:53.126387+0000","last_became_peered":"2024-09-11T00:56:53.126387+0000","last_unstale":"2024-09-11T00:58:16.268541+0000","last_undegraded":"2024-09-11T00:58:16.268541+0000","last_fullsized":"2024-09-11T00:58:16.268541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:52.429127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.260388+0000","last_change":"2024-09-11T00:56:53.126257+0000","last_active":"2024-09-11T00:58:16.260388+0000","last_peered":"2024-09-11T00:58:16.260388+0000","last_clean":"2024-09-11T00:58:16.260388+0000","last_became_active":"2024-09-11T00:56:53.125146+0000","last_became_peered":"2024-09-11T00:56:53.125146+0000","last_unstale":"2024-09-11T00:58:16.260388+0000","last_undegraded":"2024-09-11T00:58:16.260388+0000","last_fullsized":"2024-09-11T00:58:16.260388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.259691+0000","last_change":"2024-09-11T00:56:53.119187+0000","last_active":"2024-09-11T00:58:16.259691+0000","last_peered":"2024-09-11T00:58:16.259691+0000","last_clean":"2024-09-11T00:58:16.259691+0000","last_became_active":"2024-09-11T00:56:53.118850+0000","last_became_peered":"2024-09-11T00:56:53.118850+0000","last_unstale":"2024-09-11T00:58:16.259691+0000","last_undegraded":"2024-09-11T00:58:16.259691+0000","last_fullsized":"2024-09-11T00:58:16.259691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:05.403876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'7","reported_seq":155,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:49.714727+0000","last_change":"2024-09-11T00:57:17.591689+0000","last_active":"2024-09-11T00:58:49.714727+0000","last_peered":"2024-09-11T00:58:49.714727+0000","last_clean":"2024-09-11T00:58:49.714727+0000","last_became_active":"2024-09-11T00:57:17.587389+0000","last_became_peered":"2024-09-11T00:57:17.587389+0000","last_unstale":"2024-09-11T00:58:49.714727+0000","last_undegraded":"2024-09-11T00:58:49.714727+0000","last_fullsized":"2024-09-11T00:58:49.714727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:25.162270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266423+0000","last_change":"2024-09-11T00:56:53.118049+0000","last_active":"2024-09-11T00:58:16.266423+0000","last_peered":"2024-09-11T00:58:16.266423+0000","last_clean":"2024-09-11T00:58:16.266423+0000","last_became_active":"2024-09-11T00:56:53.117685+0000","last_became_peered":"2024-09-11T00:56:53.117685+0000","last_unstale":"2024-09-11T00:58:16.266423+0000","last_undegraded":"2024-09-11T00:58:16.266423+0000","last_fullsized":"2024-09-11T00:58:16.266423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:17.017594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578639+0000","last_change":"2024-09-11T00:56:51.101690+0000","last_active":"2024-09-11T00:57:17.578639+0000","last_peered":"2024-09-11T00:57:17.578639+0000","last_clean":"2024-09-11T00:57:17.578639+0000","last_became_active":"2024-09-11T00:56:51.101055+0000","last_became_peered":"2024-09-11T00:56:51.101055+0000","last_unstale":"2024-09-11T00:57:17.578639+0000","last_undegraded":"2024-09-11T00:57:17.578639+0000","last_fullsized":"2024-09-11T00:57:17.578639+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:28.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576176+0000","last_change":"2024-09-11T00:56:51.101700+0000","last_active":"2024-09-11T00:57:17.576176+0000","last_peered":"2024-09-11T00:57:17.576176+0000","last_clean":"2024-09-11T00:57:17.576176+0000","last_became_active":"2024-09-11T00:56:51.101044+0000","last_became_peered":"2024-09-11T00:56:51.101044+0000","last_unstale":"2024-09-11T00:57:17.576176+0000","last_undegraded":"2024-09-11T00:57:17.576176+0000","last_fullsized":"2024-09-11T00:57:17.576176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:12.121126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.261909+0000","last_change":"2024-09-11T00:56:53.121652+0000","last_active":"2024-09-11T00:58:16.261909+0000","last_peered":"2024-09-11T00:58:16.261909+0000","last_clean":"2024-09-11T00:58:16.261909+0000","last_became_active":"2024-09-11T00:56:53.121375+0000","last_became_peered":"2024-09-11T00:56:53.121375+0000","last_unstale":"2024-09-11T00:58:16.261909+0000","last_undegraded":"2024-09-11T00:58:16.261909+0000","last_fullsized":"2024-09-11T00:58:16.261909+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:39.154985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583515+0000","last_change":"2024-09-11T00:56:51.104367+0000","last_active":"2024-09-11T00:57:17.583515+0000","last_peered":"2024-09-11T00:57:17.583515+0000","last_clean":"2024-09-11T00:57:17.583515+0000","last_became_active":"2024-09-11T00:56:51.104045+0000","last_became_peered":"2024-09-11T00:56:51.104045+0000","last_unstale":"2024-09-11T00:57:17.583515+0000","last_undegraded":"2024-09-11T00:57:17.583515+0000","last_fullsized":"2024-09-11T00:57:17.583515+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:03.989437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264374+0000","last_change":"2024-09-11T00:56:53.116167+0000","last_active":"2024-09-11T00:58:16.264374+0000","last_peered":"2024-09-11T00:58:16.264374+0000","last_clean":"2024-09-11T00:58:16.264374+0000","last_became_active":"2024-09-11T00:56:53.115857+0000","last_became_peered":"2024-09-11T00:56:53.115857+0000","last_unstale":"2024-09-11T00:58:16.264374+0000","last_undegraded":"2024-09-11T00:58:16.264374+0000","last_fullsized":"2024-09-11T00:58:16.264374+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:04.476094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":242,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:48.927958+0000","last_change":"2024-09-11T00:56:51.103287+0000","last_active":"2024-09-11T00:58:48.927958+0000","last_peered":"2024-09-11T00:58:48.927958+0000","last_clean":"2024-09-11T00:58:48.927958+0000","last_became_active":"2024-09-11T00:56:51.102953+0000","last_became_peered":"2024-09-11T00:56:51.102953+0000","last_unstale":"2024-09-11T00:58:48.927958+0000","last_undegraded":"2024-09-11T00:58:48.927958+0000","last_fullsized":"2024-09-11T00:58:48.927958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:49.701612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":213,"num_read_kb":213,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.273793+0000","last_change":"2024-09-11T00:56:53.125130+0000","last_active":"2024-09-11T00:58:16.273793+0000","last_peered":"2024-09-11T00:58:16.273793+0000","last_clean":"2024-09-11T00:58:16.273793+0000","last_became_active":"2024-09-11T00:56:53.124442+0000","last_became_peered":"2024-09-11T00:56:53.124442+0000","last_unstale":"2024-09-11T00:58:16.273793+0000","last_undegraded":"2024-09-11T00:58:16.273793+0000","last_fullsized":"2024-09-11T00:58:16.273793+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:24.439430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586976+0000","last_change":"2024-09-11T00:56:51.104802+0000","last_active":"2024-09-11T00:57:17.586976+0000","last_peered":"2024-09-11T00:57:17.586976+0000","last_clean":"2024-09-11T00:57:17.586976+0000","last_became_active":"2024-09-11T00:56:51.104335+0000","last_became_peered":"2024-09-11T00:56:51.104335+0000","last_unstale":"2024-09-11T00:57:17.586976+0000","last_undegraded":"2024-09-11T00:57:17.586976+0000","last_fullsized":"2024-09-11T00:57:17.586976+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:26.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.263190+0000","last_change":"2024-09-11T00:57:17.584979+0000","last_active":"2024-09-11T00:58:16.263190+0000","last_peered":"2024-09-11T00:58:16.263190+0000","last_clean":"2024-09-11T00:58:16.263190+0000","last_became_active":"2024-09-11T00:57:17.584518+0000","last_became_peered":"2024-09-11T00:57:17.584518+0000","last_unstale":"2024-09-11T00:58:16.263190+0000","last_undegraded":"2024-09-11T00:58:16.263190+0000","last_fullsized":"2024-09-11T00:58:16.263190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:44.762621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583377+0000","last_change":"2024-09-11T00:56:51.154544+0000","last_active":"2024-09-11T00:57:17.583377+0000","last_peered":"2024-09-11T00:57:17.583377+0000","last_clean":"2024-09-11T00:57:17.583377+0000","last_became_active":"2024-09-11T00:56:51.154195+0000","last_became_peered":"2024-09-11T00:56:51.154195+0000","last_unstale":"2024-09-11T00:57:17.583377+0000","last_undegraded":"2024-09-11T00:57:17.583377+0000","last_fullsized":"2024-09-11T00:57:17.583377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:57:24.530947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.274114+0000","last_change":"2024-09-11T00:56:53.127728+0000","last_active":"2024-09-11T00:58:16.274114+0000","last_peered":"2024-09-11T00:58:16.274114+0000","last_clean":"2024-09-11T00:58:16.274114+0000","last_became_active":"2024-09-11T00:56:53.127423+0000","last_became_peered":"2024-09-11T00:56:53.127423+0000","last_unstale":"2024-09-11T00:58:16.274114+0000","last_undegraded":"2024-09-11T00:58:16.274114+0000","last_fullsized":"2024-09-11T00:58:16.274114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:55.717123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'34","reported_seq":277,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:48.931068+0000","last_change":"2024-09-11T00:56:51.100774+0000","last_active":"2024-09-11T00:58:48.931068+0000","last_peered":"2024-09-11T00:58:48.931068+0000","last_clean":"2024-09-11T00:58:48.931068+0000","last_became_active":"2024-09-11T00:56:51.100398+0000","last_became_peered":"2024-09-11T00:56:51.100398+0000","last_unstale":"2024-09-11T00:58:48.931068+0000","last_undegraded":"2024-09-11T00:58:48.931068+0000","last_fullsized":"2024-09-11T00:58:48.931068+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:51.975236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60904,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":169,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.267905+0000","last_change":"2024-09-11T00:56:53.120113+0000","last_active":"2024-09-11T00:58:16.267905+0000","last_peered":"2024-09-11T00:58:16.267905+0000","last_clean":"2024-09-11T00:58:16.267905+0000","last_became_active":"2024-09-11T00:56:53.119733+0000","last_became_peered":"2024-09-11T00:56:53.119733+0000","last_unstale":"2024-09-11T00:58:16.267905+0000","last_undegraded":"2024-09-11T00:58:16.267905+0000","last_fullsized":"2024-09-11T00:58:16.267905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:59.541710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580361+0000","last_change":"2024-09-11T00:56:51.104234+0000","last_active":"2024-09-11T00:57:17.580361+0000","last_peered":"2024-09-11T00:57:17.580361+0000","last_clean":"2024-09-11T00:57:17.580361+0000","last_became_active":"2024-09-11T00:56:51.103631+0000","last_became_peered":"2024-09-11T00:56:51.103631+0000","last_unstale":"2024-09-11T00:57:17.580361+0000","last_undegraded":"2024-09-11T00:57:17.580361+0000","last_fullsized":"2024-09-11T00:57:17.580361+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:47.378135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.267835+0000","last_change":"2024-09-11T00:56:53.122231+0000","last_active":"2024-09-11T00:58:16.267835+0000","last_peered":"2024-09-11T00:58:16.267835+0000","last_clean":"2024-09-11T00:58:16.267835+0000","last_became_active":"2024-09-11T00:56:53.121903+0000","last_became_peered":"2024-09-11T00:56:53.121903+0000","last_unstale":"2024-09-11T00:58:16.267835+0000","last_undegraded":"2024-09-11T00:58:16.267835+0000","last_fullsized":"2024-09-11T00:58:16.267835+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:44.246919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.582884+0000","last_change":"2024-09-11T00:56:51.105625+0000","last_active":"2024-09-11T00:57:17.582884+0000","last_peered":"2024-09-11T00:57:17.582884+0000","last_clean":"2024-09-11T00:57:17.582884+0000","last_became_active":"2024-09-11T00:56:51.105291+0000","last_became_peered":"2024-09-11T00:56:51.105291+0000","last_unstale":"2024-09-11T00:57:17.582884+0000","last_undegraded":"2024-09-11T00:57:17.582884+0000","last_fullsized":"2024-09-11T00:57:17.582884+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:24.237174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.269475+0000","last_change":"2024-09-11T00:56:53.126272+0000","last_active":"2024-09-11T00:58:16.269475+0000","last_peered":"2024-09-11T00:58:16.269475+0000","last_clean":"2024-09-11T00:58:16.269475+0000","last_became_active":"2024-09-11T00:56:53.125964+0000","last_became_peered":"2024-09-11T00:56:53.125964+0000","last_unstale":"2024-09-11T00:58:16.269475+0000","last_undegraded":"2024-09-11T00:58:16.269475+0000","last_fullsized":"2024-09-11T00:58:16.269475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:36.331220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:49.423686+0000","last_change":"2024-09-11T00:56:51.101053+0000","last_active":"2024-09-11T00:58:49.423686+0000","last_peered":"2024-09-11T00:58:49.423686+0000","last_clean":"2024-09-11T00:58:49.423686+0000","last_became_active":"2024-09-11T00:56:51.100714+0000","last_became_peered":"2024-09-11T00:56:51.100714+0000","last_unstale":"2024-09-11T00:58:49.423686+0000","last_undegraded":"2024-09-11T00:58:49.423686+0000","last_fullsized":"2024-09-11T00:58:49.423686+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:57.709126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266408+0000","last_change":"2024-09-11T00:56:53.119973+0000","last_active":"2024-09-11T00:58:16.266408+0000","last_peered":"2024-09-11T00:58:16.266408+0000","last_clean":"2024-09-11T00:58:16.266408+0000","last_became_active":"2024-09-11T00:56:53.118972+0000","last_became_peered":"2024-09-11T00:56:53.118972+0000","last_unstale":"2024-09-11T00:58:16.266408+0000","last_undegraded":"2024-09-11T00:58:16.266408+0000","last_fullsized":"2024-09-11T00:58:16.266408+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:35.912920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.598033+0000","last_change":"2024-09-11T00:57:17.597537+0000","last_active":"2024-09-11T00:57:17.598033+0000","last_peered":"2024-09-11T00:57:17.598033+0000","last_clean":"2024-09-11T00:57:17.598033+0000","last_became_active":"2024-09-11T00:57:17.581631+0000","last_became_peered":"2024-09-11T00:57:17.581631+0000","last_unstale":"2024-09-11T00:57:17.598033+0000","last_undegraded":"2024-09-11T00:57:17.598033+0000","last_fullsized":"2024-09-11T00:57:17.598033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:34.278845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264649+0000","last_change":"2024-09-11T00:56:53.120972+0000","last_active":"2024-09-11T00:58:16.264649+0000","last_peered":"2024-09-11T00:58:16.264649+0000","last_clean":"2024-09-11T00:58:16.264649+0000","last_became_active":"2024-09-11T00:56:53.120568+0000","last_became_peered":"2024-09-11T00:56:53.120568+0000","last_unstale":"2024-09-11T00:58:16.264649+0000","last_undegraded":"2024-09-11T00:58:16.264649+0000","last_fullsized":"2024-09-11T00:58:16.264649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:19.696201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581943+0000","last_change":"2024-09-11T00:56:51.103756+0000","last_active":"2024-09-11T00:57:17.581943+0000","last_peered":"2024-09-11T00:57:17.581943+0000","last_clean":"2024-09-11T00:57:17.581943+0000","last_became_active":"2024-09-11T00:56:51.103406+0000","last_became_peered":"2024-09-11T00:56:51.103406+0000","last_unstale":"2024-09-11T00:57:17.581943+0000","last_undegraded":"2024-09-11T00:57:17.581943+0000","last_fullsized":"2024-09-11T00:57:17.581943+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:28.022346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.257373+0000","last_change":"2024-09-11T00:56:53.128628+0000","last_active":"2024-09-11T00:58:16.257373+0000","last_peered":"2024-09-11T00:58:16.257373+0000","last_clean":"2024-09-11T00:58:16.257373+0000","last_became_active":"2024-09-11T00:56:53.127197+0000","last_became_peered":"2024-09-11T00:56:53.127197+0000","last_unstale":"2024-09-11T00:58:16.257373+0000","last_undegraded":"2024-09-11T00:58:16.257373+0000","last_fullsized":"2024-09-11T00:58:16.257373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:38.753676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576498+0000","last_change":"2024-09-11T00:56:51.101398+0000","last_active":"2024-09-11T00:57:17.576498+0000","last_peered":"2024-09-11T00:57:17.576498+0000","last_clean":"2024-09-11T00:57:17.576498+0000","last_became_active":"2024-09-11T00:56:51.101034+0000","last_became_peered":"2024-09-11T00:56:51.101034+0000","last_unstale":"2024-09-11T00:57:17.576498+0000","last_undegraded":"2024-09-11T00:57:17.576498+0000","last_fullsized":"2024-09-11T00:57:17.576498+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:40.165368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265346+0000","last_change":"2024-09-11T00:56:53.127047+0000","last_active":"2024-09-11T00:58:16.265346+0000","last_peered":"2024-09-11T00:58:16.265346+0000","last_clean":"2024-09-11T00:58:16.265346+0000","last_became_active":"2024-09-11T00:56:53.126719+0000","last_became_peered":"2024-09-11T00:56:53.126719+0000","last_unstale":"2024-09-11T00:58:16.265346+0000","last_undegraded":"2024-09-11T00:58:16.265346+0000","last_fullsized":"2024-09-11T00:58:16.265346+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.686604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583494+0000","last_change":"2024-09-11T00:56:51.101665+0000","last_active":"2024-09-11T00:57:17.583494+0000","last_peered":"2024-09-11T00:57:17.583494+0000","last_clean":"2024-09-11T00:57:17.583494+0000","last_became_active":"2024-09-11T00:56:51.101332+0000","last_became_peered":"2024-09-11T00:56:51.101332+0000","last_unstale":"2024-09-11T00:57:17.583494+0000","last_undegraded":"2024-09-11T00:57:17.583494+0000","last_fullsized":"2024-09-11T00:57:17.583494+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:47.639877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.267267+0000","last_change":"2024-09-11T00:56:53.128736+0000","last_active":"2024-09-11T00:58:16.267267+0000","last_peered":"2024-09-11T00:58:16.267267+0000","last_clean":"2024-09-11T00:58:16.267267+0000","last_became_active":"2024-09-11T00:56:53.128417+0000","last_became_peered":"2024-09-11T00:56:53.128417+0000","last_unstale":"2024-09-11T00:58:16.267267+0000","last_undegraded":"2024-09-11T00:58:16.267267+0000","last_fullsized":"2024-09-11T00:58:16.267267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:52.331343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'6","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:49.383863+0000","last_change":"2024-09-11T00:56:51.154733+0000","last_active":"2024-09-11T00:58:49.383863+0000","last_peered":"2024-09-11T00:58:49.383863+0000","last_clean":"2024-09-11T00:58:49.383863+0000","last_became_active":"2024-09-11T00:56:51.153613+0000","last_became_peered":"2024-09-11T00:56:51.153613+0000","last_unstale":"2024-09-11T00:58:49.383863+0000","last_undegraded":"2024-09-11T00:58:49.383863+0000","last_fullsized":"2024-09-11T00:58:49.383863+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:24.749186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1478,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.260212+0000","last_change":"2024-09-11T00:56:53.124684+0000","last_active":"2024-09-11T00:58:16.260212+0000","last_peered":"2024-09-11T00:58:16.260212+0000","last_clean":"2024-09-11T00:58:16.260212+0000","last_became_active":"2024-09-11T00:56:53.124147+0000","last_became_peered":"2024-09-11T00:56:53.124147+0000","last_unstale":"2024-09-11T00:58:16.260212+0000","last_undegraded":"2024-09-11T00:58:16.260212+0000","last_fullsized":"2024-09-11T00:58:16.260212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:14.280465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586515+0000","last_change":"2024-09-11T00:55:34.500373+0000","last_active":"2024-09-11T00:57:17.586515+0000","last_peered":"2024-09-11T00:57:17.586515+0000","last_clean":"2024-09-11T00:57:17.586515+0000","last_became_active":"2024-09-11T00:55:34.495540+0000","last_became_peered":"2024-09-11T00:55:34.495540+0000","last_unstale":"2024-09-11T00:57:17.586515+0000","last_undegraded":"2024-09-11T00:57:17.586515+0000","last_fullsized":"2024-09-11T00:57:17.586515+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_clean_scrub_stamp":"2024-09-11T00:52:11.168966+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:37.681466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577058+0000","last_change":"2024-09-11T00:56:51.101524+0000","last_active":"2024-09-11T00:57:17.577058+0000","last_peered":"2024-09-11T00:57:17.577058+0000","last_clean":"2024-09-11T00:57:17.577058+0000","last_became_active":"2024-09-11T00:56:51.101180+0000","last_became_peered":"2024-09-11T00:56:51.101180+0000","last_unstale":"2024-09-11T00:57:17.577058+0000","last_undegraded":"2024-09-11T00:57:17.577058+0000","last_fullsized":"2024-09-11T00:57:17.577058+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:27.293494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.259285+0000","last_change":"2024-09-11T00:56:53.118503+0000","last_active":"2024-09-11T00:58:16.259285+0000","last_peered":"2024-09-11T00:58:16.259285+0000","last_clean":"2024-09-11T00:58:16.259285+0000","last_became_active":"2024-09-11T00:56:53.118174+0000","last_became_peered":"2024-09-11T00:56:53.118174+0000","last_unstale":"2024-09-11T00:58:16.259285+0000","last_undegraded":"2024-09-11T00:58:16.259285+0000","last_fullsized":"2024-09-11T00:58:16.259285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:13.524153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586788+0000","last_change":"2024-09-11T00:56:51.104222+0000","last_active":"2024-09-11T00:57:17.586788+0000","last_peered":"2024-09-11T00:57:17.586788+0000","last_clean":"2024-09-11T00:57:17.586788+0000","last_became_active":"2024-09-11T00:56:51.103913+0000","last_became_peered":"2024-09-11T00:56:51.103913+0000","last_unstale":"2024-09-11T00:57:17.586788+0000","last_undegraded":"2024-09-11T00:57:17.586788+0000","last_fullsized":"2024-09-11T00:57:17.586788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:57.467659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.261731+0000","last_change":"2024-09-11T00:56:53.115130+0000","last_active":"2024-09-11T00:58:16.261731+0000","last_peered":"2024-09-11T00:58:16.261731+0000","last_clean":"2024-09-11T00:58:16.261731+0000","last_became_active":"2024-09-11T00:56:53.114813+0000","last_became_peered":"2024-09-11T00:56:53.114813+0000","last_unstale":"2024-09-11T00:58:16.261731+0000","last_undegraded":"2024-09-11T00:58:16.261731+0000","last_fullsized":"2024-09-11T00:58:16.261731+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:13.265173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":134,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:46.417780+0000","last_change":"2024-09-11T00:56:51.152892+0000","last_active":"2024-09-11T00:58:46.417780+0000","last_peered":"2024-09-11T00:58:46.417780+0000","last_clean":"2024-09-11T00:58:46.417780+0000","last_became_active":"2024-09-11T00:56:51.152572+0000","last_became_peered":"2024-09-11T00:56:51.152572+0000","last_unstale":"2024-09-11T00:58:46.417780+0000","last_undegraded":"2024-09-11T00:58:46.417780+0000","last_fullsized":"2024-09-11T00:58:46.417780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:03.774593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.267390+0000","last_change":"2024-09-11T00:56:53.117416+0000","last_active":"2024-09-11T00:58:16.267390+0000","last_peered":"2024-09-11T00:58:16.267390+0000","last_clean":"2024-09-11T00:58:16.267390+0000","last_became_active":"2024-09-11T00:56:53.117016+0000","last_became_peered":"2024-09-11T00:56:53.117016+0000","last_unstale":"2024-09-11T00:58:16.267390+0000","last_undegraded":"2024-09-11T00:58:16.267390+0000","last_fullsized":"2024-09-11T00:58:16.267390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:07.607993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:46.377435+0000","last_change":"2024-09-11T00:56:51.102879+0000","last_active":"2024-09-11T00:58:46.377435+0000","last_peered":"2024-09-11T00:58:46.377435+0000","last_clean":"2024-09-11T00:58:46.377435+0000","last_became_active":"2024-09-11T00:56:51.102498+0000","last_became_peered":"2024-09-11T00:56:51.102498+0000","last_unstale":"2024-09-11T00:58:46.377435+0000","last_undegraded":"2024-09-11T00:58:46.377435+0000","last_fullsized":"2024-09-11T00:58:46.377435+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:41.360908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265698+0000","last_change":"2024-09-11T00:57:17.581261+0000","last_active":"2024-09-11T00:58:16.265698+0000","last_peered":"2024-09-11T00:58:16.265698+0000","last_clean":"2024-09-11T00:58:16.265698+0000","last_became_active":"2024-09-11T00:57:17.580941+0000","last_became_peered":"2024-09-11T00:57:17.580941+0000","last_unstale":"2024-09-11T00:58:16.265698+0000","last_undegraded":"2024-09-11T00:58:16.265698+0000","last_fullsized":"2024-09-11T00:58:16.265698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:58.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.584416+0000","last_change":"2024-09-11T00:57:17.584311+0000","last_active":"2024-09-11T00:57:17.584416+0000","last_peered":"2024-09-11T00:57:17.584416+0000","last_clean":"2024-09-11T00:57:17.584416+0000","last_became_active":"2024-09-11T00:57:17.583964+0000","last_became_peered":"2024-09-11T00:57:17.583964+0000","last_unstale":"2024-09-11T00:57:17.584416+0000","last_undegraded":"2024-09-11T00:57:17.584416+0000","last_fullsized":"2024-09-11T00:57:17.584416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:15.515947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.254129+0000","last_change":"2024-09-11T00:56:53.112550+0000","last_active":"2024-09-11T00:58:16.254129+0000","last_peered":"2024-09-11T00:58:16.254129+0000","last_clean":"2024-09-11T00:58:16.254129+0000","last_became_active":"2024-09-11T00:56:53.112179+0000","last_became_peered":"2024-09-11T00:56:53.112179+0000","last_unstale":"2024-09-11T00:58:16.254129+0000","last_undegraded":"2024-09-11T00:58:16.254129+0000","last_fullsized":"2024-09-11T00:58:16.254129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:03.339839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581699+0000","last_change":"2024-09-11T00:56:51.152957+0000","last_active":"2024-09-11T00:57:17.581699+0000","last_peered":"2024-09-11T00:57:17.581699+0000","last_clean":"2024-09-11T00:57:17.581699+0000","last_became_active":"2024-09-11T00:56:51.152616+0000","last_became_peered":"2024-09-11T00:56:51.152616+0000","last_unstale":"2024-09-11T00:57:17.581699+0000","last_undegraded":"2024-09-11T00:57:17.581699+0000","last_fullsized":"2024-09-11T00:57:17.581699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:02.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.262966+0000","last_change":"2024-09-11T00:56:53.115155+0000","last_active":"2024-09-11T00:58:16.262966+0000","last_peered":"2024-09-11T00:58:16.262966+0000","last_clean":"2024-09-11T00:58:16.262966+0000","last_became_active":"2024-09-11T00:56:53.114865+0000","last_became_peered":"2024-09-11T00:56:53.114865+0000","last_unstale":"2024-09-11T00:58:16.262966+0000","last_undegraded":"2024-09-11T00:58:16.262966+0000","last_fullsized":"2024-09-11T00:58:16.262966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:23.968888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'17","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:19.123128+0000","last_change":"2024-09-11T00:56:51.100887+0000","last_active":"2024-09-11T00:57:19.123128+0000","last_peered":"2024-09-11T00:57:19.123128+0000","last_clean":"2024-09-11T00:57:19.123128+0000","last_became_active":"2024-09-11T00:56:51.100474+0000","last_became_peered":"2024-09-11T00:56:51.100474+0000","last_unstale":"2024-09-11T00:57:19.123128+0000","last_undegraded":"2024-09-11T00:57:19.123128+0000","last_fullsized":"2024-09-11T00:57:19.123128+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:40.884871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.257501+0000","last_change":"2024-09-11T00:57:17.588086+0000","last_active":"2024-09-11T00:58:16.257501+0000","last_peered":"2024-09-11T00:58:16.257501+0000","last_clean":"2024-09-11T00:58:16.257501+0000","last_became_active":"2024-09-11T00:57:17.587785+0000","last_became_peered":"2024-09-11T00:57:17.587785+0000","last_unstale":"2024-09-11T00:58:16.257501+0000","last_undegraded":"2024-09-11T00:58:16.257501+0000","last_fullsized":"2024-09-11T00:58:16.257501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:49.494588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577254+0000","last_change":"2024-09-11T00:56:51.102031+0000","last_active":"2024-09-11T00:57:17.577254+0000","last_peered":"2024-09-11T00:57:17.577254+0000","last_clean":"2024-09-11T00:57:17.577254+0000","last_became_active":"2024-09-11T00:56:51.100614+0000","last_became_peered":"2024-09-11T00:56:51.100614+0000","last_unstale":"2024-09-11T00:57:17.577254+0000","last_undegraded":"2024-09-11T00:57:17.577254+0000","last_fullsized":"2024-09-11T00:57:17.577254+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:52.611261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265773+0000","last_change":"2024-09-11T00:56:53.124806+0000","last_active":"2024-09-11T00:58:16.265773+0000","last_peered":"2024-09-11T00:58:16.265773+0000","last_clean":"2024-09-11T00:58:16.265773+0000","last_became_active":"2024-09-11T00:56:53.124452+0000","last_became_peered":"2024-09-11T00:56:53.124452+0000","last_unstale":"2024-09-11T00:58:16.265773+0000","last_undegraded":"2024-09-11T00:58:16.265773+0000","last_fullsized":"2024-09-11T00:58:16.265773+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:57.934808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160509+0000","last_change":"2024-09-11T00:56:51.153621+0000","last_active":"2024-09-11T00:57:18.160509+0000","last_peered":"2024-09-11T00:57:18.160509+0000","last_clean":"2024-09-11T00:57:18.160509+0000","last_became_active":"2024-09-11T00:56:51.153315+0000","last_became_peered":"2024-09-11T00:56:51.153315+0000","last_unstale":"2024-09-11T00:57:18.160509+0000","last_undegraded":"2024-09-11T00:57:18.160509+0000","last_fullsized":"2024-09-11T00:57:18.160509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:10.058295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.226023+0000","last_change":"2024-09-11T00:56:53.114463+0000","last_active":"2024-09-11T00:58:16.226023+0000","last_peered":"2024-09-11T00:58:16.226023+0000","last_clean":"2024-09-11T00:58:16.226023+0000","last_became_active":"2024-09-11T00:56:53.114099+0000","last_became_peered":"2024-09-11T00:56:53.114099+0000","last_unstale":"2024-09-11T00:58:16.226023+0000","last_undegraded":"2024-09-11T00:58:16.226023+0000","last_fullsized":"2024-09-11T00:58:16.226023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:58.981787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771377+0000","last_change":"2024-09-11T00:56:51.096942+0000","last_active":"2024-09-11T00:57:17.771377+0000","last_peered":"2024-09-11T00:57:17.771377+0000","last_clean":"2024-09-11T00:57:17.771377+0000","last_became_active":"2024-09-11T00:56:51.096584+0000","last_became_peered":"2024-09-11T00:56:51.096584+0000","last_unstale":"2024-09-11T00:57:17.771377+0000","last_undegraded":"2024-09-11T00:57:17.771377+0000","last_fullsized":"2024-09-11T00:57:17.771377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:03.484123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265844+0000","last_change":"2024-09-11T00:56:53.126804+0000","last_active":"2024-09-11T00:58:16.265844+0000","last_peered":"2024-09-11T00:58:16.265844+0000","last_clean":"2024-09-11T00:58:16.265844+0000","last_became_active":"2024-09-11T00:56:53.125408+0000","last_became_peered":"2024-09-11T00:56:53.125408+0000","last_unstale":"2024-09-11T00:58:16.265844+0000","last_undegraded":"2024-09-11T00:58:16.265844+0000","last_fullsized":"2024-09-11T00:58:16.265844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:46.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580130+0000","last_change":"2024-09-11T00:56:51.103755+0000","last_active":"2024-09-11T00:57:17.580130+0000","last_peered":"2024-09-11T00:57:17.580130+0000","last_clean":"2024-09-11T00:57:17.580130+0000","last_became_active":"2024-09-11T00:56:51.103395+0000","last_became_peered":"2024-09-11T00:56:51.103395+0000","last_unstale":"2024-09-11T00:57:17.580130+0000","last_undegraded":"2024-09-11T00:57:17.580130+0000","last_fullsized":"2024-09-11T00:57:17.580130+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:16.782660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.257207+0000","last_change":"2024-09-11T00:56:53.121926+0000","last_active":"2024-09-11T00:58:16.257207+0000","last_peered":"2024-09-11T00:58:16.257207+0000","last_clean":"2024-09-11T00:58:16.257207+0000","last_became_active":"2024-09-11T00:56:53.121603+0000","last_became_peered":"2024-09-11T00:56:53.121603+0000","last_unstale":"2024-09-11T00:58:16.257207+0000","last_undegraded":"2024-09-11T00:58:16.257207+0000","last_fullsized":"2024-09-11T00:58:16.257207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:31.021030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580437+0000","last_change":"2024-09-11T00:56:51.107221+0000","last_active":"2024-09-11T00:57:17.580437+0000","last_peered":"2024-09-11T00:57:17.580437+0000","last_clean":"2024-09-11T00:57:17.580437+0000","last_became_active":"2024-09-11T00:56:51.106849+0000","last_became_peered":"2024-09-11T00:56:51.106849+0000","last_unstale":"2024-09-11T00:57:17.580437+0000","last_undegraded":"2024-09-11T00:57:17.580437+0000","last_fullsized":"2024-09-11T00:57:17.580437+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:46.497230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.269124+0000","last_change":"2024-09-11T00:56:53.120525+0000","last_active":"2024-09-11T00:58:16.269124+0000","last_peered":"2024-09-11T00:58:16.269124+0000","last_clean":"2024-09-11T00:58:16.269124+0000","last_became_active":"2024-09-11T00:56:53.120228+0000","last_became_peered":"2024-09-11T00:56:53.120228+0000","last_unstale":"2024-09-11T00:58:16.269124+0000","last_undegraded":"2024-09-11T00:58:16.269124+0000","last_fullsized":"2024-09-11T00:58:16.269124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:52.047687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266504+0000","last_change":"2024-09-11T00:56:53.120311+0000","last_active":"2024-09-11T00:58:16.266504+0000","last_peered":"2024-09-11T00:58:16.266504+0000","last_clean":"2024-09-11T00:58:16.266504+0000","last_became_active":"2024-09-11T00:56:53.119648+0000","last_became_peered":"2024-09-11T00:56:53.119648+0000","last_unstale":"2024-09-11T00:58:16.266504+0000","last_undegraded":"2024-09-11T00:58:16.266504+0000","last_fullsized":"2024-09-11T00:58:16.266504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:21.187784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581118+0000","last_change":"2024-09-11T00:56:51.103631+0000","last_active":"2024-09-11T00:57:17.581118+0000","last_peered":"2024-09-11T00:57:17.581118+0000","last_clean":"2024-09-11T00:57:17.581118+0000","last_became_active":"2024-09-11T00:56:51.103281+0000","last_became_peered":"2024-09-11T00:56:51.103281+0000","last_unstale":"2024-09-11T00:57:17.581118+0000","last_undegraded":"2024-09-11T00:57:17.581118+0000","last_fullsized":"2024-09-11T00:57:17.581118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:16.910956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.274004+0000","last_change":"2024-09-11T00:56:53.124752+0000","last_active":"2024-09-11T00:58:16.274004+0000","last_peered":"2024-09-11T00:58:16.274004+0000","last_clean":"2024-09-11T00:58:16.274004+0000","last_became_active":"2024-09-11T00:56:53.124245+0000","last_became_peered":"2024-09-11T00:56:53.124245+0000","last_unstale":"2024-09-11T00:58:16.274004+0000","last_undegraded":"2024-09-11T00:58:16.274004+0000","last_fullsized":"2024-09-11T00:58:16.274004+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:10.277903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581704+0000","last_change":"2024-09-11T00:56:51.104624+0000","last_active":"2024-09-11T00:57:17.581704+0000","last_peered":"2024-09-11T00:57:17.581704+0000","last_clean":"2024-09-11T00:57:17.581704+0000","last_became_active":"2024-09-11T00:56:51.104041+0000","last_became_peered":"2024-09-11T00:56:51.104041+0000","last_unstale":"2024-09-11T00:57:17.581704+0000","last_undegraded":"2024-09-11T00:57:17.581704+0000","last_fullsized":"2024-09-11T00:57:17.581704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:10.397455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.272637+0000","last_change":"2024-09-11T00:56:53.118547+0000","last_active":"2024-09-11T00:58:16.272637+0000","last_peered":"2024-09-11T00:58:16.272637+0000","last_clean":"2024-09-11T00:58:16.272637+0000","last_became_active":"2024-09-11T00:56:53.118291+0000","last_became_peered":"2024-09-11T00:56:53.118291+0000","last_unstale":"2024-09-11T00:58:16.272637+0000","last_undegraded":"2024-09-11T00:58:16.272637+0000","last_fullsized":"2024-09-11T00:58:16.272637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:42.012173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771774+0000","last_change":"2024-09-11T00:56:51.153047+0000","last_active":"2024-09-11T00:57:17.771774+0000","last_peered":"2024-09-11T00:57:17.771774+0000","last_clean":"2024-09-11T00:57:17.771774+0000","last_became_active":"2024-09-11T00:56:51.152564+0000","last_became_peered":"2024-09-11T00:56:51.152564+0000","last_unstale":"2024-09-11T00:57:17.771774+0000","last_undegraded":"2024-09-11T00:57:17.771774+0000","last_fullsized":"2024-09-11T00:57:17.771774+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:16.849087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.269357+0000","last_change":"2024-09-11T00:56:53.119926+0000","last_active":"2024-09-11T00:58:16.269357+0000","last_peered":"2024-09-11T00:58:16.269357+0000","last_clean":"2024-09-11T00:58:16.269357+0000","last_became_active":"2024-09-11T00:56:53.119643+0000","last_became_peered":"2024-09-11T00:56:53.119643+0000","last_unstale":"2024-09-11T00:58:16.269357+0000","last_undegraded":"2024-09-11T00:58:16.269357+0000","last_fullsized":"2024-09-11T00:58:16.269357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:51.838958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:47.378123+0000","last_change":"2024-09-11T00:56:51.100837+0000","last_active":"2024-09-11T00:58:47.378123+0000","last_peered":"2024-09-11T00:58:47.378123+0000","last_clean":"2024-09-11T00:58:47.378123+0000","last_became_active":"2024-09-11T00:56:51.100494+0000","last_became_peered":"2024-09-11T00:56:51.100494+0000","last_unstale":"2024-09-11T00:58:47.378123+0000","last_undegraded":"2024-09-11T00:58:47.378123+0000","last_fullsized":"2024-09-11T00:58:47.378123+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:39.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68166,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1101,"num_read_kb":624,"num_write":254,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":204498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286865,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":416921,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906986,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1068434,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":416431,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266740,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":416697,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":1009468,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626494,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":416148,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822724,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1009146,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986250,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":417163,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":415973,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378708,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":477871,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542233,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":913408,"data_stored":477657,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":417795,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:58:53.455 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T00:58:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:53 smithi112 ceph-mon[35384]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-11T00:58:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:53 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3855293555' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.585+0000 7f0703b9b640 1 -- 172.21.15.78:0/2231686893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06fc107630 msgr2=0x7f06fc107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.585+0000 7f0703b9b640 1 --2- 172.21.15.78:0/2231686893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06fc107630 0x7f06fc107a10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009ca0 tx=0x7f06ec02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.585+0000 7f0703b9b640 1 -- 172.21.15.78:0/2231686893 shutdown_connections 2024-09-11T00:58:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.585+0000 7f0703b9b640 1 --2- 172.21.15.78:0/2231686893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06fc113940 0x7f06fc115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.585+0000 7f0703b9b640 1 --2- 172.21.15.78:0/2231686893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06fc107f50 0x7f06fc113400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.585+0000 7f0703b9b640 1 --2- 172.21.15.78:0/2231686893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06fc107630 0x7f06fc107a10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.585+0000 7f0703b9b640 1 -- 172.21.15.78:0/2231686893 >> 172.21.15.78:0/2231686893 conn(0x7f06fc076e90 msgr2=0x7f06fc0772a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.585+0000 7f0703b9b640 1 -- 172.21.15.78:0/2231686893 shutdown_connections 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.585+0000 7f0703b9b640 1 -- 172.21.15.78:0/2231686893 wait complete. 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0703b9b640 1 Processor -- start 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0703b9b640 1 -- start start 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0703b9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06fc107630 0x7f06fc19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0703b9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06fc107f50 0x7f06fc19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0703b9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06fc113940 0x7f06fc1a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0703b9b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc119330 con 0x7f06fc107630 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0703b9b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1191b0 con 0x7f06fc113940 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0703b9b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1194b0 con 0x7f06fc107f50 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0701937640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06fc107630 0x7f06fc19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0701937640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06fc107630 0x7f06fc19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42712/0 (socket says 172.21.15.78:42712) 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0701937640 1 -- 172.21.15.78:0/3341385481 learned_addr learned my addr 172.21.15.78:0/3341385481 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0701937640 1 -- 172.21.15.78:0/3341385481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06fc107f50 msgr2=0x7f06fc19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.586+0000 7f0701136640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06fc107f50 0x7f06fc19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.587+0000 7f0701937640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06fc107f50 0x7f06fc19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.587+0000 7f0701937640 1 -- 172.21.15.78:0/3341385481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06fc113940 msgr2=0x7f06fc1a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.587+0000 7f0702138640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06fc113940 0x7f06fc1a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.587+0000 7f0701937640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06fc113940 0x7f06fc1a5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.587+0000 7f0701937640 1 -- 172.21.15.78:0/3341385481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc1a58d0 con 0x7f06fc107630 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.587+0000 7f0701136640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06fc107f50 0x7f06fc19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.587+0000 7f0701937640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06fc107630 0x7f06fc19f830 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f06ec02ade0 tx=0x7f06ec003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.587+0000 7f06faffd640 1 -- 172.21.15.78:0/3341385481 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec003d60 con 0x7f06fc107630 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.587+0000 7f06faffd640 1 -- 172.21.15.78:0/3341385481 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06ec0044f0 con 0x7f06fc107630 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.587+0000 7f0703b9b640 1 -- 172.21.15.78:0/3341385481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc1a5b60 con 0x7f06fc107630 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.587+0000 7f06faffd640 1 -- 172.21.15.78:0/3341385481 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec02fd10 con 0x7f06fc107630 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.587+0000 7f0703b9b640 1 -- 172.21.15.78:0/3341385481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc1ac300 con 0x7f06fc107630 2024-09-11T00:58:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.588+0000 7f0703b9b640 1 -- 172.21.15.78:0/3341385481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06fc107a10 con 0x7f06fc107630 2024-09-11T00:58:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.589+0000 7f06faffd640 1 -- 172.21.15.78:0/3341385481 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f06ec004040 con 0x7f06fc107630 2024-09-11T00:58:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.590+0000 7f06faffd640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f06e40b5330 0x7f06e40b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.590+0000 7f0701136640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f06e40b5330 0x7f06e40b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.590+0000 7f06faffd640 1 -- 172.21.15.78:0/3341385481 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f06ec107400 con 0x7f06fc107630 2024-09-11T00:58:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.591+0000 7f0701136640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f06e40b5330 0x7f06e40b77f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f06f0006fd0 tx=0x7f06f0008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.594+0000 7f06faffd640 1 -- 172.21.15.78:0/3341385481 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f06ec0ce210 con 0x7f06fc107630 2024-09-11T00:58:53.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.766+0000 7f0703b9b640 1 -- 172.21.15.78:0/3341385481 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06fc114430 con 0x7f06e40b5330 2024-09-11T00:58:53.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.770+0000 7f06faffd640 1 -- 172.21.15.78:0/3341385481 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259130 (secure 0 0 0) 0x7f06fc114430 con 0x7f06e40b5330 2024-09-11T00:58:53.775 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:53.777 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T00:58:53.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.773+0000 7f0703b9b640 1 -- 172.21.15.78:0/3341385481 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f06e40b5330 msgr2=0x7f06e40b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.773+0000 7f0703b9b640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f06e40b5330 0x7f06e40b77f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f06f0006fd0 tx=0x7f06f0008040 comp rx=0 tx=0).stop 2024-09-11T00:58:53.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.773+0000 7f0703b9b640 1 -- 172.21.15.78:0/3341385481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06fc107630 msgr2=0x7f06fc19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.773+0000 7f0703b9b640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06fc107630 0x7f06fc19f830 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f06ec02ade0 tx=0x7f06ec003970 comp rx=0 tx=0).stop 2024-09-11T00:58:53.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.773+0000 7f0703b9b640 1 -- 172.21.15.78:0/3341385481 shutdown_connections 2024-09-11T00:58:53.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.773+0000 7f0703b9b640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f06e40b5330 0x7f06e40b77f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.773+0000 7f0703b9b640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06fc113940 0x7f06fc1a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.773+0000 7f0703b9b640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06fc107f50 0x7f06fc19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.773+0000 7f0703b9b640 1 --2- 172.21.15.78:0/3341385481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06fc107630 0x7f06fc19f830 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.773+0000 7f0703b9b640 1 -- 172.21.15.78:0/3341385481 >> 172.21.15.78:0/3341385481 conn(0x7f06fc076e90 msgr2=0x7f06fc113d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:53.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.773+0000 7f0703b9b640 1 -- 172.21.15.78:0/3341385481 shutdown_connections 2024-09-11T00:58:53.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.773+0000 7f0703b9b640 1 -- 172.21.15.78:0/3341385481 wait complete. 2024-09-11T00:58:53.786 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-11T00:58:52.048680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658768,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1182,"num_read_kb":693,"num_write":379,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2952965,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":812,"ondisk_log_size":812,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376352,"kb_used_data":10464,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565472,"statfs":{"total":1151940427776,"available":1151555043328,"internally_reserved":0,"allocated":10715136,"data_stored":6959704,"data_compressed":179373,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":51,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002996"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266212+0000","last_change":"2024-09-11T00:56:53.123622+0000","last_active":"2024-09-11T00:58:16.266212+0000","last_peered":"2024-09-11T00:58:16.266212+0000","last_clean":"2024-09-11T00:58:16.266212+0000","last_became_active":"2024-09-11T00:56:53.121460+0000","last_became_peered":"2024-09-11T00:56:53.121460+0000","last_unstale":"2024-09-11T00:58:16.266212+0000","last_undegraded":"2024-09-11T00:58:16.266212+0000","last_fullsized":"2024-09-11T00:58:16.266212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:30.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264953+0000","last_change":"2024-09-11T00:56:53.120189+0000","last_active":"2024-09-11T00:58:16.264953+0000","last_peered":"2024-09-11T00:58:16.264953+0000","last_clean":"2024-09-11T00:58:16.264953+0000","last_became_active":"2024-09-11T00:56:53.119853+0000","last_became_peered":"2024-09-11T00:56:53.119853+0000","last_unstale":"2024-09-11T00:58:16.264953+0000","last_undegraded":"2024-09-11T00:58:16.264953+0000","last_fullsized":"2024-09-11T00:58:16.264953+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:36.927113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.272647+0000","last_change":"2024-09-11T00:56:53.117832+0000","last_active":"2024-09-11T00:58:16.272647+0000","last_peered":"2024-09-11T00:58:16.272647+0000","last_clean":"2024-09-11T00:58:16.272647+0000","last_became_active":"2024-09-11T00:56:53.117508+0000","last_became_peered":"2024-09-11T00:56:53.117508+0000","last_unstale":"2024-09-11T00:58:16.272647+0000","last_undegraded":"2024-09-11T00:58:16.272647+0000","last_fullsized":"2024-09-11T00:58:16.272647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:00.628673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.271210+0000","last_change":"2024-09-11T00:56:53.128302+0000","last_active":"2024-09-11T00:58:16.271210+0000","last_peered":"2024-09-11T00:58:16.271210+0000","last_clean":"2024-09-11T00:58:16.271210+0000","last_became_active":"2024-09-11T00:56:53.127584+0000","last_became_peered":"2024-09-11T00:56:53.127584+0000","last_unstale":"2024-09-11T00:58:16.271210+0000","last_undegraded":"2024-09-11T00:58:16.271210+0000","last_fullsized":"2024-09-11T00:58:16.271210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:41.963656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.248383+0000","last_change":"2024-09-11T00:56:53.116249+0000","last_active":"2024-09-11T00:58:16.248383+0000","last_peered":"2024-09-11T00:58:16.248383+0000","last_clean":"2024-09-11T00:58:16.248383+0000","last_became_active":"2024-09-11T00:56:53.115881+0000","last_became_peered":"2024-09-11T00:56:53.115881+0000","last_unstale":"2024-09-11T00:58:16.248383+0000","last_undegraded":"2024-09-11T00:58:16.248383+0000","last_fullsized":"2024-09-11T00:58:16.248383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:47.432607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.262557+0000","last_change":"2024-09-11T00:56:53.116992+0000","last_active":"2024-09-11T00:58:16.262557+0000","last_peered":"2024-09-11T00:58:16.262557+0000","last_clean":"2024-09-11T00:58:16.262557+0000","last_became_active":"2024-09-11T00:56:53.116629+0000","last_became_peered":"2024-09-11T00:56:53.116629+0000","last_unstale":"2024-09-11T00:58:16.262557+0000","last_undegraded":"2024-09-11T00:58:16.262557+0000","last_fullsized":"2024-09-11T00:58:16.262557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:25.410377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266888+0000","last_change":"2024-09-11T00:56:53.127873+0000","last_active":"2024-09-11T00:58:16.266888+0000","last_peered":"2024-09-11T00:58:16.266888+0000","last_clean":"2024-09-11T00:58:16.266888+0000","last_became_active":"2024-09-11T00:56:53.127575+0000","last_became_peered":"2024-09-11T00:56:53.127575+0000","last_unstale":"2024-09-11T00:58:16.266888+0000","last_undegraded":"2024-09-11T00:58:16.266888+0000","last_fullsized":"2024-09-11T00:58:16.266888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:12.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.261154+0000","last_change":"2024-09-11T00:56:53.117727+0000","last_active":"2024-09-11T00:58:16.261154+0000","last_peered":"2024-09-11T00:58:16.261154+0000","last_clean":"2024-09-11T00:58:16.261154+0000","last_became_active":"2024-09-11T00:56:53.117444+0000","last_became_peered":"2024-09-11T00:56:53.117444+0000","last_unstale":"2024-09-11T00:58:16.261154+0000","last_undegraded":"2024-09-11T00:58:16.261154+0000","last_fullsized":"2024-09-11T00:58:16.261154+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:28.625231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268315+0000","last_change":"2024-09-11T00:56:53.128974+0000","last_active":"2024-09-11T00:58:16.268315+0000","last_peered":"2024-09-11T00:58:16.268315+0000","last_clean":"2024-09-11T00:58:16.268315+0000","last_became_active":"2024-09-11T00:56:53.128650+0000","last_became_peered":"2024-09-11T00:56:53.128650+0000","last_unstale":"2024-09-11T00:58:16.268315+0000","last_undegraded":"2024-09-11T00:58:16.268315+0000","last_fullsized":"2024-09-11T00:58:16.268315+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:07.251373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.262269+0000","last_change":"2024-09-11T00:56:53.117816+0000","last_active":"2024-09-11T00:58:16.262269+0000","last_peered":"2024-09-11T00:58:16.262269+0000","last_clean":"2024-09-11T00:58:16.262269+0000","last_became_active":"2024-09-11T00:56:53.117449+0000","last_became_peered":"2024-09-11T00:56:53.117449+0000","last_unstale":"2024-09-11T00:58:16.262269+0000","last_undegraded":"2024-09-11T00:58:16.262269+0000","last_fullsized":"2024-09-11T00:58:16.262269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:09.058869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265129+0000","last_change":"2024-09-11T00:56:53.124616+0000","last_active":"2024-09-11T00:58:16.265129+0000","last_peered":"2024-09-11T00:58:16.265129+0000","last_clean":"2024-09-11T00:58:16.265129+0000","last_became_active":"2024-09-11T00:56:53.124309+0000","last_became_peered":"2024-09-11T00:56:53.124309+0000","last_unstale":"2024-09-11T00:58:16.265129+0000","last_undegraded":"2024-09-11T00:58:16.265129+0000","last_fullsized":"2024-09-11T00:58:16.265129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:14.469909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.257580+0000","last_change":"2024-09-11T00:56:53.126263+0000","last_active":"2024-09-11T00:58:16.257580+0000","last_peered":"2024-09-11T00:58:16.257580+0000","last_clean":"2024-09-11T00:58:16.257580+0000","last_became_active":"2024-09-11T00:56:53.125923+0000","last_became_peered":"2024-09-11T00:56:53.125923+0000","last_unstale":"2024-09-11T00:58:16.257580+0000","last_undegraded":"2024-09-11T00:58:16.257580+0000","last_fullsized":"2024-09-11T00:58:16.257580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:12:33.627372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.258526+0000","last_change":"2024-09-11T00:56:53.105292+0000","last_active":"2024-09-11T00:58:16.258526+0000","last_peered":"2024-09-11T00:58:16.258526+0000","last_clean":"2024-09-11T00:58:16.258526+0000","last_became_active":"2024-09-11T00:56:53.104830+0000","last_became_peered":"2024-09-11T00:56:53.104830+0000","last_unstale":"2024-09-11T00:58:16.258526+0000","last_undegraded":"2024-09-11T00:58:16.258526+0000","last_fullsized":"2024-09-11T00:58:16.258526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:14.136952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264895+0000","last_change":"2024-09-11T00:56:53.120463+0000","last_active":"2024-09-11T00:58:16.264895+0000","last_peered":"2024-09-11T00:58:16.264895+0000","last_clean":"2024-09-11T00:58:16.264895+0000","last_became_active":"2024-09-11T00:56:53.120167+0000","last_became_peered":"2024-09-11T00:56:53.120167+0000","last_unstale":"2024-09-11T00:58:16.264895+0000","last_undegraded":"2024-09-11T00:58:16.264895+0000","last_fullsized":"2024-09-11T00:58:16.264895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:49.783452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265508+0000","last_change":"2024-09-11T00:56:53.123312+0000","last_active":"2024-09-11T00:58:16.265508+0000","last_peered":"2024-09-11T00:58:16.265508+0000","last_clean":"2024-09-11T00:58:16.265508+0000","last_became_active":"2024-09-11T00:56:53.123015+0000","last_became_peered":"2024-09-11T00:56:53.123015+0000","last_unstale":"2024-09-11T00:58:16.265508+0000","last_undegraded":"2024-09-11T00:58:16.265508+0000","last_fullsized":"2024-09-11T00:58:16.265508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:22.069629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.272553+0000","last_change":"2024-09-11T00:56:53.125701+0000","last_active":"2024-09-11T00:58:16.272553+0000","last_peered":"2024-09-11T00:58:16.272553+0000","last_clean":"2024-09-11T00:58:16.272553+0000","last_became_active":"2024-09-11T00:56:53.125362+0000","last_became_peered":"2024-09-11T00:56:53.125362+0000","last_unstale":"2024-09-11T00:58:16.272553+0000","last_undegraded":"2024-09-11T00:58:16.272553+0000","last_fullsized":"2024-09-11T00:58:16.272553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:14.594324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266425+0000","last_change":"2024-09-11T00:56:53.124275+0000","last_active":"2024-09-11T00:58:16.266425+0000","last_peered":"2024-09-11T00:58:16.266425+0000","last_clean":"2024-09-11T00:58:16.266425+0000","last_became_active":"2024-09-11T00:56:53.122695+0000","last_became_peered":"2024-09-11T00:56:53.122695+0000","last_unstale":"2024-09-11T00:58:16.266425+0000","last_undegraded":"2024-09-11T00:58:16.266425+0000","last_fullsized":"2024-09-11T00:58:16.266425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:23.955266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264639+0000","last_change":"2024-09-11T00:56:53.119077+0000","last_active":"2024-09-11T00:58:16.264639+0000","last_peered":"2024-09-11T00:58:16.264639+0000","last_clean":"2024-09-11T00:58:16.264639+0000","last_became_active":"2024-09-11T00:56:53.118693+0000","last_became_peered":"2024-09-11T00:56:53.118693+0000","last_unstale":"2024-09-11T00:58:16.264639+0000","last_undegraded":"2024-09-11T00:58:16.264639+0000","last_fullsized":"2024-09-11T00:58:16.264639+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:39.135391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265439+0000","last_change":"2024-09-11T00:56:53.115430+0000","last_active":"2024-09-11T00:58:16.265439+0000","last_peered":"2024-09-11T00:58:16.265439+0000","last_clean":"2024-09-11T00:58:16.265439+0000","last_became_active":"2024-09-11T00:56:53.115111+0000","last_became_peered":"2024-09-11T00:56:53.115111+0000","last_unstale":"2024-09-11T00:58:16.265439+0000","last_undegraded":"2024-09-11T00:58:16.265439+0000","last_fullsized":"2024-09-11T00:58:16.265439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:11.306661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264277+0000","last_change":"2024-09-11T00:57:17.583260+0000","last_active":"2024-09-11T00:58:16.264277+0000","last_peered":"2024-09-11T00:58:16.264277+0000","last_clean":"2024-09-11T00:58:16.264277+0000","last_became_active":"2024-09-11T00:57:17.582909+0000","last_became_peered":"2024-09-11T00:57:17.582909+0000","last_unstale":"2024-09-11T00:58:16.264277+0000","last_undegraded":"2024-09-11T00:58:16.264277+0000","last_fullsized":"2024-09-11T00:58:16.264277+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:42.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268531+0000","last_change":"2024-09-11T00:56:53.121303+0000","last_active":"2024-09-11T00:58:16.268531+0000","last_peered":"2024-09-11T00:58:16.268531+0000","last_clean":"2024-09-11T00:58:16.268531+0000","last_became_active":"2024-09-11T00:56:53.121004+0000","last_became_peered":"2024-09-11T00:56:53.121004+0000","last_unstale":"2024-09-11T00:58:16.268531+0000","last_undegraded":"2024-09-11T00:58:16.268531+0000","last_fullsized":"2024-09-11T00:58:16.268531+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:36.899870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266931+0000","last_change":"2024-09-11T00:56:53.125042+0000","last_active":"2024-09-11T00:58:16.266931+0000","last_peered":"2024-09-11T00:58:16.266931+0000","last_clean":"2024-09-11T00:58:16.266931+0000","last_became_active":"2024-09-11T00:56:53.124661+0000","last_became_peered":"2024-09-11T00:56:53.124661+0000","last_unstale":"2024-09-11T00:58:16.266931+0000","last_undegraded":"2024-09-11T00:58:16.266931+0000","last_fullsized":"2024-09-11T00:58:16.266931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:34.392172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268728+0000","last_change":"2024-09-11T00:56:53.116544+0000","last_active":"2024-09-11T00:58:16.268728+0000","last_peered":"2024-09-11T00:58:16.268728+0000","last_clean":"2024-09-11T00:58:16.268728+0000","last_became_active":"2024-09-11T00:56:53.116262+0000","last_became_peered":"2024-09-11T00:56:53.116262+0000","last_unstale":"2024-09-11T00:58:16.268728+0000","last_undegraded":"2024-09-11T00:58:16.268728+0000","last_fullsized":"2024-09-11T00:58:16.268728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:34.500669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.263134+0000","last_change":"2024-09-11T00:56:53.116967+0000","last_active":"2024-09-11T00:58:16.263134+0000","last_peered":"2024-09-11T00:58:16.263134+0000","last_clean":"2024-09-11T00:58:16.263134+0000","last_became_active":"2024-09-11T00:56:53.116405+0000","last_became_peered":"2024-09-11T00:56:53.116405+0000","last_unstale":"2024-09-11T00:58:16.263134+0000","last_undegraded":"2024-09-11T00:58:16.263134+0000","last_fullsized":"2024-09-11T00:58:16.263134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:33.521964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268124+0000","last_change":"2024-09-11T00:56:53.116727+0000","last_active":"2024-09-11T00:58:16.268124+0000","last_peered":"2024-09-11T00:58:16.268124+0000","last_clean":"2024-09-11T00:58:16.268124+0000","last_became_active":"2024-09-11T00:56:53.116416+0000","last_became_peered":"2024-09-11T00:56:53.116416+0000","last_unstale":"2024-09-11T00:58:16.268124+0000","last_undegraded":"2024-09-11T00:58:16.268124+0000","last_fullsized":"2024-09-11T00:58:16.268124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:28.277909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265487+0000","last_change":"2024-09-11T00:56:53.123141+0000","last_active":"2024-09-11T00:58:16.265487+0000","last_peered":"2024-09-11T00:58:16.265487+0000","last_clean":"2024-09-11T00:58:16.265487+0000","last_became_active":"2024-09-11T00:56:53.122811+0000","last_became_peered":"2024-09-11T00:56:53.122811+0000","last_unstale":"2024-09-11T00:58:16.265487+0000","last_undegraded":"2024-09-11T00:58:16.265487+0000","last_fullsized":"2024-09-11T00:58:16.265487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:47.175120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.260452+0000","last_change":"2024-09-11T00:56:53.124213+0000","last_active":"2024-09-11T00:58:16.260452+0000","last_peered":"2024-09-11T00:58:16.260452+0000","last_clean":"2024-09-11T00:58:16.260452+0000","last_became_active":"2024-09-11T00:56:53.123757+0000","last_became_peered":"2024-09-11T00:56:53.123757+0000","last_unstale":"2024-09-11T00:58:16.260452+0000","last_undegraded":"2024-09-11T00:58:16.260452+0000","last_fullsized":"2024-09-11T00:58:16.260452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:54.263476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.260255+0000","last_change":"2024-09-11T00:56:53.114016+0000","last_active":"2024-09-11T00:58:16.260255+0000","last_peered":"2024-09-11T00:58:16.260255+0000","last_clean":"2024-09-11T00:58:16.260255+0000","last_became_active":"2024-09-11T00:56:53.113719+0000","last_became_peered":"2024-09-11T00:56:53.113719+0000","last_unstale":"2024-09-11T00:58:16.260255+0000","last_undegraded":"2024-09-11T00:58:16.260255+0000","last_fullsized":"2024-09-11T00:58:16.260255+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:48:06.623810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.258203+0000","last_change":"2024-09-11T00:56:53.124798+0000","last_active":"2024-09-11T00:58:16.258203+0000","last_peered":"2024-09-11T00:58:16.258203+0000","last_clean":"2024-09-11T00:58:16.258203+0000","last_became_active":"2024-09-11T00:56:53.124381+0000","last_became_peered":"2024-09-11T00:56:53.124381+0000","last_unstale":"2024-09-11T00:58:16.258203+0000","last_undegraded":"2024-09-11T00:58:16.258203+0000","last_fullsized":"2024-09-11T00:58:16.258203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:37.823112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265603+0000","last_change":"2024-09-11T00:56:53.115297+0000","last_active":"2024-09-11T00:58:16.265603+0000","last_peered":"2024-09-11T00:58:16.265603+0000","last_clean":"2024-09-11T00:58:16.265603+0000","last_became_active":"2024-09-11T00:56:53.114958+0000","last_became_peered":"2024-09-11T00:56:53.114958+0000","last_unstale":"2024-09-11T00:58:16.265603+0000","last_undegraded":"2024-09-11T00:58:16.265603+0000","last_fullsized":"2024-09-11T00:58:16.265603+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:47.612610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268541+0000","last_change":"2024-09-11T00:56:53.126717+0000","last_active":"2024-09-11T00:58:16.268541+0000","last_peered":"2024-09-11T00:58:16.268541+0000","last_clean":"2024-09-11T00:58:16.268541+0000","last_became_active":"2024-09-11T00:56:53.126387+0000","last_became_peered":"2024-09-11T00:56:53.126387+0000","last_unstale":"2024-09-11T00:58:16.268541+0000","last_undegraded":"2024-09-11T00:58:16.268541+0000","last_fullsized":"2024-09-11T00:58:16.268541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:52.429127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.260388+0000","last_change":"2024-09-11T00:56:53.126257+0000","last_active":"2024-09-11T00:58:16.260388+0000","last_peered":"2024-09-11T00:58:16.260388+0000","last_clean":"2024-09-11T00:58:16.260388+0000","last_became_active":"2024-09-11T00:56:53.125146+0000","last_became_peered":"2024-09-11T00:56:53.125146+0000","last_unstale":"2024-09-11T00:58:16.260388+0000","last_undegraded":"2024-09-11T00:58:16.260388+0000","last_fullsized":"2024-09-11T00:58:16.260388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.259691+0000","last_change":"2024-09-11T00:56:53.119187+0000","last_active":"2024-09-11T00:58:16.259691+0000","last_peered":"2024-09-11T00:58:16.259691+0000","last_clean":"2024-09-11T00:58:16.259691+0000","last_became_active":"2024-09-11T00:56:53.118850+0000","last_became_peered":"2024-09-11T00:56:53.118850+0000","last_unstale":"2024-09-11T00:58:16.259691+0000","last_undegraded":"2024-09-11T00:58:16.259691+0000","last_fullsized":"2024-09-11T00:58:16.259691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:05.403876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'7","reported_seq":155,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:49.714727+0000","last_change":"2024-09-11T00:57:17.591689+0000","last_active":"2024-09-11T00:58:49.714727+0000","last_peered":"2024-09-11T00:58:49.714727+0000","last_clean":"2024-09-11T00:58:49.714727+0000","last_became_active":"2024-09-11T00:57:17.587389+0000","last_became_peered":"2024-09-11T00:57:17.587389+0000","last_unstale":"2024-09-11T00:58:49.714727+0000","last_undegraded":"2024-09-11T00:58:49.714727+0000","last_fullsized":"2024-09-11T00:58:49.714727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:25.162270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266423+0000","last_change":"2024-09-11T00:56:53.118049+0000","last_active":"2024-09-11T00:58:16.266423+0000","last_peered":"2024-09-11T00:58:16.266423+0000","last_clean":"2024-09-11T00:58:16.266423+0000","last_became_active":"2024-09-11T00:56:53.117685+0000","last_became_peered":"2024-09-11T00:56:53.117685+0000","last_unstale":"2024-09-11T00:58:16.266423+0000","last_undegraded":"2024-09-11T00:58:16.266423+0000","last_fullsized":"2024-09-11T00:58:16.266423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:17.017594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578639+0000","last_change":"2024-09-11T00:56:51.101690+0000","last_active":"2024-09-11T00:57:17.578639+0000","last_peered":"2024-09-11T00:57:17.578639+0000","last_clean":"2024-09-11T00:57:17.578639+0000","last_became_active":"2024-09-11T00:56:51.101055+0000","last_became_peered":"2024-09-11T00:56:51.101055+0000","last_unstale":"2024-09-11T00:57:17.578639+0000","last_undegraded":"2024-09-11T00:57:17.578639+0000","last_fullsized":"2024-09-11T00:57:17.578639+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:28.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576176+0000","last_change":"2024-09-11T00:56:51.101700+0000","last_active":"2024-09-11T00:57:17.576176+0000","last_peered":"2024-09-11T00:57:17.576176+0000","last_clean":"2024-09-11T00:57:17.576176+0000","last_became_active":"2024-09-11T00:56:51.101044+0000","last_became_peered":"2024-09-11T00:56:51.101044+0000","last_unstale":"2024-09-11T00:57:17.576176+0000","last_undegraded":"2024-09-11T00:57:17.576176+0000","last_fullsized":"2024-09-11T00:57:17.576176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:12.121126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.261909+0000","last_change":"2024-09-11T00:56:53.121652+0000","last_active":"2024-09-11T00:58:16.261909+0000","last_peered":"2024-09-11T00:58:16.261909+0000","last_clean":"2024-09-11T00:58:16.261909+0000","last_became_active":"2024-09-11T00:56:53.121375+0000","last_became_peered":"2024-09-11T00:56:53.121375+0000","last_unstale":"2024-09-11T00:58:16.261909+0000","last_undegraded":"2024-09-11T00:58:16.261909+0000","last_fullsized":"2024-09-11T00:58:16.261909+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:39.154985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583515+0000","last_change":"2024-09-11T00:56:51.104367+0000","last_active":"2024-09-11T00:57:17.583515+0000","last_peered":"2024-09-11T00:57:17.583515+0000","last_clean":"2024-09-11T00:57:17.583515+0000","last_became_active":"2024-09-11T00:56:51.104045+0000","last_became_peered":"2024-09-11T00:56:51.104045+0000","last_unstale":"2024-09-11T00:57:17.583515+0000","last_undegraded":"2024-09-11T00:57:17.583515+0000","last_fullsized":"2024-09-11T00:57:17.583515+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:03.989437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264374+0000","last_change":"2024-09-11T00:56:53.116167+0000","last_active":"2024-09-11T00:58:16.264374+0000","last_peered":"2024-09-11T00:58:16.264374+0000","last_clean":"2024-09-11T00:58:16.264374+0000","last_became_active":"2024-09-11T00:56:53.115857+0000","last_became_peered":"2024-09-11T00:56:53.115857+0000","last_unstale":"2024-09-11T00:58:16.264374+0000","last_undegraded":"2024-09-11T00:58:16.264374+0000","last_fullsized":"2024-09-11T00:58:16.264374+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:04.476094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":242,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:48.927958+0000","last_change":"2024-09-11T00:56:51.103287+0000","last_active":"2024-09-11T00:58:48.927958+0000","last_peered":"2024-09-11T00:58:48.927958+0000","last_clean":"2024-09-11T00:58:48.927958+0000","last_became_active":"2024-09-11T00:56:51.102953+0000","last_became_peered":"2024-09-11T00:56:51.102953+0000","last_unstale":"2024-09-11T00:58:48.927958+0000","last_undegraded":"2024-09-11T00:58:48.927958+0000","last_fullsized":"2024-09-11T00:58:48.927958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:49.701612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":213,"num_read_kb":213,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.273793+0000","last_change":"2024-09-11T00:56:53.125130+0000","last_active":"2024-09-11T00:58:16.273793+0000","last_peered":"2024-09-11T00:58:16.273793+0000","last_clean":"2024-09-11T00:58:16.273793+0000","last_became_active":"2024-09-11T00:56:53.124442+0000","last_became_peered":"2024-09-11T00:56:53.124442+0000","last_unstale":"2024-09-11T00:58:16.273793+0000","last_undegraded":"2024-09-11T00:58:16.273793+0000","last_fullsized":"2024-09-11T00:58:16.273793+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:24.439430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586976+0000","last_change":"2024-09-11T00:56:51.104802+0000","last_active":"2024-09-11T00:57:17.586976+0000","last_peered":"2024-09-11T00:57:17.586976+0000","last_clean":"2024-09-11T00:57:17.586976+0000","last_became_active":"2024-09-11T00:56:51.104335+0000","last_became_peered":"2024-09-11T00:56:51.104335+0000","last_unstale":"2024-09-11T00:57:17.586976+0000","last_undegraded":"2024-09-11T00:57:17.586976+0000","last_fullsized":"2024-09-11T00:57:17.586976+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:26.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.263190+0000","last_change":"2024-09-11T00:57:17.584979+0000","last_active":"2024-09-11T00:58:16.263190+0000","last_peered":"2024-09-11T00:58:16.263190+0000","last_clean":"2024-09-11T00:58:16.263190+0000","last_became_active":"2024-09-11T00:57:17.584518+0000","last_became_peered":"2024-09-11T00:57:17.584518+0000","last_unstale":"2024-09-11T00:58:16.263190+0000","last_undegraded":"2024-09-11T00:58:16.263190+0000","last_fullsized":"2024-09-11T00:58:16.263190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:44.762621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583377+0000","last_change":"2024-09-11T00:56:51.154544+0000","last_active":"2024-09-11T00:57:17.583377+0000","last_peered":"2024-09-11T00:57:17.583377+0000","last_clean":"2024-09-11T00:57:17.583377+0000","last_became_active":"2024-09-11T00:56:51.154195+0000","last_became_peered":"2024-09-11T00:56:51.154195+0000","last_unstale":"2024-09-11T00:57:17.583377+0000","last_undegraded":"2024-09-11T00:57:17.583377+0000","last_fullsized":"2024-09-11T00:57:17.583377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:57:24.530947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.274114+0000","last_change":"2024-09-11T00:56:53.127728+0000","last_active":"2024-09-11T00:58:16.274114+0000","last_peered":"2024-09-11T00:58:16.274114+0000","last_clean":"2024-09-11T00:58:16.274114+0000","last_became_active":"2024-09-11T00:56:53.127423+0000","last_became_peered":"2024-09-11T00:56:53.127423+0000","last_unstale":"2024-09-11T00:58:16.274114+0000","last_undegraded":"2024-09-11T00:58:16.274114+0000","last_fullsized":"2024-09-11T00:58:16.274114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:55.717123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'34","reported_seq":277,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:48.931068+0000","last_change":"2024-09-11T00:56:51.100774+0000","last_active":"2024-09-11T00:58:48.931068+0000","last_peered":"2024-09-11T00:58:48.931068+0000","last_clean":"2024-09-11T00:58:48.931068+0000","last_became_active":"2024-09-11T00:56:51.100398+0000","last_became_peered":"2024-09-11T00:56:51.100398+0000","last_unstale":"2024-09-11T00:58:48.931068+0000","last_undegraded":"2024-09-11T00:58:48.931068+0000","last_fullsized":"2024-09-11T00:58:48.931068+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:51.975236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60904,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":169,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.267905+0000","last_change":"2024-09-11T00:56:53.120113+0000","last_active":"2024-09-11T00:58:16.267905+0000","last_peered":"2024-09-11T00:58:16.267905+0000","last_clean":"2024-09-11T00:58:16.267905+0000","last_became_active":"2024-09-11T00:56:53.119733+0000","last_became_peered":"2024-09-11T00:56:53.119733+0000","last_unstale":"2024-09-11T00:58:16.267905+0000","last_undegraded":"2024-09-11T00:58:16.267905+0000","last_fullsized":"2024-09-11T00:58:16.267905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:59.541710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580361+0000","last_change":"2024-09-11T00:56:51.104234+0000","last_active":"2024-09-11T00:57:17.580361+0000","last_peered":"2024-09-11T00:57:17.580361+0000","last_clean":"2024-09-11T00:57:17.580361+0000","last_became_active":"2024-09-11T00:56:51.103631+0000","last_became_peered":"2024-09-11T00:56:51.103631+0000","last_unstale":"2024-09-11T00:57:17.580361+0000","last_undegraded":"2024-09-11T00:57:17.580361+0000","last_fullsized":"2024-09-11T00:57:17.580361+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:47.378135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.267835+0000","last_change":"2024-09-11T00:56:53.122231+0000","last_active":"2024-09-11T00:58:16.267835+0000","last_peered":"2024-09-11T00:58:16.267835+0000","last_clean":"2024-09-11T00:58:16.267835+0000","last_became_active":"2024-09-11T00:56:53.121903+0000","last_became_peered":"2024-09-11T00:56:53.121903+0000","last_unstale":"2024-09-11T00:58:16.267835+0000","last_undegraded":"2024-09-11T00:58:16.267835+0000","last_fullsized":"2024-09-11T00:58:16.267835+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:44.246919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.582884+0000","last_change":"2024-09-11T00:56:51.105625+0000","last_active":"2024-09-11T00:57:17.582884+0000","last_peered":"2024-09-11T00:57:17.582884+0000","last_clean":"2024-09-11T00:57:17.582884+0000","last_became_active":"2024-09-11T00:56:51.105291+0000","last_became_peered":"2024-09-11T00:56:51.105291+0000","last_unstale":"2024-09-11T00:57:17.582884+0000","last_undegraded":"2024-09-11T00:57:17.582884+0000","last_fullsized":"2024-09-11T00:57:17.582884+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:24.237174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.269475+0000","last_change":"2024-09-11T00:56:53.126272+0000","last_active":"2024-09-11T00:58:16.269475+0000","last_peered":"2024-09-11T00:58:16.269475+0000","last_clean":"2024-09-11T00:58:16.269475+0000","last_became_active":"2024-09-11T00:56:53.125964+0000","last_became_peered":"2024-09-11T00:56:53.125964+0000","last_unstale":"2024-09-11T00:58:16.269475+0000","last_undegraded":"2024-09-11T00:58:16.269475+0000","last_fullsized":"2024-09-11T00:58:16.269475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:36.331220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:49.423686+0000","last_change":"2024-09-11T00:56:51.101053+0000","last_active":"2024-09-11T00:58:49.423686+0000","last_peered":"2024-09-11T00:58:49.423686+0000","last_clean":"2024-09-11T00:58:49.423686+0000","last_became_active":"2024-09-11T00:56:51.100714+0000","last_became_peered":"2024-09-11T00:56:51.100714+0000","last_unstale":"2024-09-11T00:58:49.423686+0000","last_undegraded":"2024-09-11T00:58:49.423686+0000","last_fullsized":"2024-09-11T00:58:49.423686+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:57.709126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266408+0000","last_change":"2024-09-11T00:56:53.119973+0000","last_active":"2024-09-11T00:58:16.266408+0000","last_peered":"2024-09-11T00:58:16.266408+0000","last_clean":"2024-09-11T00:58:16.266408+0000","last_became_active":"2024-09-11T00:56:53.118972+0000","last_became_peered":"2024-09-11T00:56:53.118972+0000","last_unstale":"2024-09-11T00:58:16.266408+0000","last_undegraded":"2024-09-11T00:58:16.266408+0000","last_fullsized":"2024-09-11T00:58:16.266408+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:35.912920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.598033+0000","last_change":"2024-09-11T00:57:17.597537+0000","last_active":"2024-09-11T00:57:17.598033+0000","last_peered":"2024-09-11T00:57:17.598033+0000","last_clean":"2024-09-11T00:57:17.598033+0000","last_became_active":"2024-09-11T00:57:17.581631+0000","last_became_peered":"2024-09-11T00:57:17.581631+0000","last_unstale":"2024-09-11T00:57:17.598033+0000","last_undegraded":"2024-09-11T00:57:17.598033+0000","last_fullsized":"2024-09-11T00:57:17.598033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:34.278845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264649+0000","last_change":"2024-09-11T00:56:53.120972+0000","last_active":"2024-09-11T00:58:16.264649+0000","last_peered":"2024-09-11T00:58:16.264649+0000","last_clean":"2024-09-11T00:58:16.264649+0000","last_became_active":"2024-09-11T00:56:53.120568+0000","last_became_peered":"2024-09-11T00:56:53.120568+0000","last_unstale":"2024-09-11T00:58:16.264649+0000","last_undegraded":"2024-09-11T00:58:16.264649+0000","last_fullsized":"2024-09-11T00:58:16.264649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:19.696201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581943+0000","last_change":"2024-09-11T00:56:51.103756+0000","last_active":"2024-09-11T00:57:17.581943+0000","last_peered":"2024-09-11T00:57:17.581943+0000","last_clean":"2024-09-11T00:57:17.581943+0000","last_became_active":"2024-09-11T00:56:51.103406+0000","last_became_peered":"2024-09-11T00:56:51.103406+0000","last_unstale":"2024-09-11T00:57:17.581943+0000","last_undegraded":"2024-09-11T00:57:17.581943+0000","last_fullsized":"2024-09-11T00:57:17.581943+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:28.022346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.257373+0000","last_change":"2024-09-11T00:56:53.128628+0000","last_active":"2024-09-11T00:58:16.257373+0000","last_peered":"2024-09-11T00:58:16.257373+0000","last_clean":"2024-09-11T00:58:16.257373+0000","last_became_active":"2024-09-11T00:56:53.127197+0000","last_became_peered":"2024-09-11T00:56:53.127197+0000","last_unstale":"2024-09-11T00:58:16.257373+0000","last_undegraded":"2024-09-11T00:58:16.257373+0000","last_fullsized":"2024-09-11T00:58:16.257373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:38.753676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576498+0000","last_change":"2024-09-11T00:56:51.101398+0000","last_active":"2024-09-11T00:57:17.576498+0000","last_peered":"2024-09-11T00:57:17.576498+0000","last_clean":"2024-09-11T00:57:17.576498+0000","last_became_active":"2024-09-11T00:56:51.101034+0000","last_became_peered":"2024-09-11T00:56:51.101034+0000","last_unstale":"2024-09-11T00:57:17.576498+0000","last_undegraded":"2024-09-11T00:57:17.576498+0000","last_fullsized":"2024-09-11T00:57:17.576498+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:40.165368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265346+0000","last_change":"2024-09-11T00:56:53.127047+0000","last_active":"2024-09-11T00:58:16.265346+0000","last_peered":"2024-09-11T00:58:16.265346+0000","last_clean":"2024-09-11T00:58:16.265346+0000","last_became_active":"2024-09-11T00:56:53.126719+0000","last_became_peered":"2024-09-11T00:56:53.126719+0000","last_unstale":"2024-09-11T00:58:16.265346+0000","last_undegraded":"2024-09-11T00:58:16.265346+0000","last_fullsized":"2024-09-11T00:58:16.265346+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.686604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583494+0000","last_change":"2024-09-11T00:56:51.101665+0000","last_active":"2024-09-11T00:57:17.583494+0000","last_peered":"2024-09-11T00:57:17.583494+0000","last_clean":"2024-09-11T00:57:17.583494+0000","last_became_active":"2024-09-11T00:56:51.101332+0000","last_became_peered":"2024-09-11T00:56:51.101332+0000","last_unstale":"2024-09-11T00:57:17.583494+0000","last_undegraded":"2024-09-11T00:57:17.583494+0000","last_fullsized":"2024-09-11T00:57:17.583494+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:47.639877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.267267+0000","last_change":"2024-09-11T00:56:53.128736+0000","last_active":"2024-09-11T00:58:16.267267+0000","last_peered":"2024-09-11T00:58:16.267267+0000","last_clean":"2024-09-11T00:58:16.267267+0000","last_became_active":"2024-09-11T00:56:53.128417+0000","last_became_peered":"2024-09-11T00:56:53.128417+0000","last_unstale":"2024-09-11T00:58:16.267267+0000","last_undegraded":"2024-09-11T00:58:16.267267+0000","last_fullsized":"2024-09-11T00:58:16.267267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:52.331343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'6","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:49.383863+0000","last_change":"2024-09-11T00:56:51.154733+0000","last_active":"2024-09-11T00:58:49.383863+0000","last_peered":"2024-09-11T00:58:49.383863+0000","last_clean":"2024-09-11T00:58:49.383863+0000","last_became_active":"2024-09-11T00:56:51.153613+0000","last_became_peered":"2024-09-11T00:56:51.153613+0000","last_unstale":"2024-09-11T00:58:49.383863+0000","last_undegraded":"2024-09-11T00:58:49.383863+0000","last_fullsized":"2024-09-11T00:58:49.383863+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:24.749186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1478,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.260212+0000","last_change":"2024-09-11T00:56:53.124684+0000","last_active":"2024-09-11T00:58:16.260212+0000","last_peered":"2024-09-11T00:58:16.260212+0000","last_clean":"2024-09-11T00:58:16.260212+0000","last_became_active":"2024-09-11T00:56:53.124147+0000","last_became_peered":"2024-09-11T00:56:53.124147+0000","last_unstale":"2024-09-11T00:58:16.260212+0000","last_undegraded":"2024-09-11T00:58:16.260212+0000","last_fullsized":"2024-09-11T00:58:16.260212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:14.280465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586515+0000","last_change":"2024-09-11T00:55:34.500373+0000","last_active":"2024-09-11T00:57:17.586515+0000","last_peered":"2024-09-11T00:57:17.586515+0000","last_clean":"2024-09-11T00:57:17.586515+0000","last_became_active":"2024-09-11T00:55:34.495540+0000","last_became_peered":"2024-09-11T00:55:34.495540+0000","last_unstale":"2024-09-11T00:57:17.586515+0000","last_undegraded":"2024-09-11T00:57:17.586515+0000","last_fullsized":"2024-09-11T00:57:17.586515+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_clean_scrub_stamp":"2024-09-11T00:52:11.168966+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:37.681466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577058+0000","last_change":"2024-09-11T00:56:51.101524+0000","last_active":"2024-09-11T00:57:17.577058+0000","last_peered":"2024-09-11T00:57:17.577058+0000","last_clean":"2024-09-11T00:57:17.577058+0000","last_became_active":"2024-09-11T00:56:51.101180+0000","last_became_peered":"2024-09-11T00:56:51.101180+0000","last_unstale":"2024-09-11T00:57:17.577058+0000","last_undegraded":"2024-09-11T00:57:17.577058+0000","last_fullsized":"2024-09-11T00:57:17.577058+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:27.293494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.259285+0000","last_change":"2024-09-11T00:56:53.118503+0000","last_active":"2024-09-11T00:58:16.259285+0000","last_peered":"2024-09-11T00:58:16.259285+0000","last_clean":"2024-09-11T00:58:16.259285+0000","last_became_active":"2024-09-11T00:56:53.118174+0000","last_became_peered":"2024-09-11T00:56:53.118174+0000","last_unstale":"2024-09-11T00:58:16.259285+0000","last_undegraded":"2024-09-11T00:58:16.259285+0000","last_fullsized":"2024-09-11T00:58:16.259285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:13.524153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586788+0000","last_change":"2024-09-11T00:56:51.104222+0000","last_active":"2024-09-11T00:57:17.586788+0000","last_peered":"2024-09-11T00:57:17.586788+0000","last_clean":"2024-09-11T00:57:17.586788+0000","last_became_active":"2024-09-11T00:56:51.103913+0000","last_became_peered":"2024-09-11T00:56:51.103913+0000","last_unstale":"2024-09-11T00:57:17.586788+0000","last_undegraded":"2024-09-11T00:57:17.586788+0000","last_fullsized":"2024-09-11T00:57:17.586788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:57.467659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.261731+0000","last_change":"2024-09-11T00:56:53.115130+0000","last_active":"2024-09-11T00:58:16.261731+0000","last_peered":"2024-09-11T00:58:16.261731+0000","last_clean":"2024-09-11T00:58:16.261731+0000","last_became_active":"2024-09-11T00:56:53.114813+0000","last_became_peered":"2024-09-11T00:56:53.114813+0000","last_unstale":"2024-09-11T00:58:16.261731+0000","last_undegraded":"2024-09-11T00:58:16.261731+0000","last_fullsized":"2024-09-11T00:58:16.261731+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:13.265173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":134,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:46.417780+0000","last_change":"2024-09-11T00:56:51.152892+0000","last_active":"2024-09-11T00:58:46.417780+0000","last_peered":"2024-09-11T00:58:46.417780+0000","last_clean":"2024-09-11T00:58:46.417780+0000","last_became_active":"2024-09-11T00:56:51.152572+0000","last_became_peered":"2024-09-11T00:56:51.152572+0000","last_unstale":"2024-09-11T00:58:46.417780+0000","last_undegraded":"2024-09-11T00:58:46.417780+0000","last_fullsized":"2024-09-11T00:58:46.417780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:03.774593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.267390+0000","last_change":"2024-09-11T00:56:53.117416+0000","last_active":"2024-09-11T00:58:16.267390+0000","last_peered":"2024-09-11T00:58:16.267390+0000","last_clean":"2024-09-11T00:58:16.267390+0000","last_became_active":"2024-09-11T00:56:53.117016+0000","last_became_peered":"2024-09-11T00:56:53.117016+0000","last_unstale":"2024-09-11T00:58:16.267390+0000","last_undegraded":"2024-09-11T00:58:16.267390+0000","last_fullsized":"2024-09-11T00:58:16.267390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:07.607993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:46.377435+0000","last_change":"2024-09-11T00:56:51.102879+0000","last_active":"2024-09-11T00:58:46.377435+0000","last_peered":"2024-09-11T00:58:46.377435+0000","last_clean":"2024-09-11T00:58:46.377435+0000","last_became_active":"2024-09-11T00:56:51.102498+0000","last_became_peered":"2024-09-11T00:56:51.102498+0000","last_unstale":"2024-09-11T00:58:46.377435+0000","last_undegraded":"2024-09-11T00:58:46.377435+0000","last_fullsized":"2024-09-11T00:58:46.377435+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:41.360908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265698+0000","last_change":"2024-09-11T00:57:17.581261+0000","last_active":"2024-09-11T00:58:16.265698+0000","last_peered":"2024-09-11T00:58:16.265698+0000","last_clean":"2024-09-11T00:58:16.265698+0000","last_became_active":"2024-09-11T00:57:17.580941+0000","last_became_peered":"2024-09-11T00:57:17.580941+0000","last_unstale":"2024-09-11T00:58:16.265698+0000","last_undegraded":"2024-09-11T00:58:16.265698+0000","last_fullsized":"2024-09-11T00:58:16.265698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:58.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.584416+0000","last_change":"2024-09-11T00:57:17.584311+0000","last_active":"2024-09-11T00:57:17.584416+0000","last_peered":"2024-09-11T00:57:17.584416+0000","last_clean":"2024-09-11T00:57:17.584416+0000","last_became_active":"2024-09-11T00:57:17.583964+0000","last_became_peered":"2024-09-11T00:57:17.583964+0000","last_unstale":"2024-09-11T00:57:17.584416+0000","last_undegraded":"2024-09-11T00:57:17.584416+0000","last_fullsized":"2024-09-11T00:57:17.584416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:15.515947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.254129+0000","last_change":"2024-09-11T00:56:53.112550+0000","last_active":"2024-09-11T00:58:16.254129+0000","last_peered":"2024-09-11T00:58:16.254129+0000","last_clean":"2024-09-11T00:58:16.254129+0000","last_became_active":"2024-09-11T00:56:53.112179+0000","last_became_peered":"2024-09-11T00:56:53.112179+0000","last_unstale":"2024-09-11T00:58:16.254129+0000","last_undegraded":"2024-09-11T00:58:16.254129+0000","last_fullsized":"2024-09-11T00:58:16.254129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:03.339839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581699+0000","last_change":"2024-09-11T00:56:51.152957+0000","last_active":"2024-09-11T00:57:17.581699+0000","last_peered":"2024-09-11T00:57:17.581699+0000","last_clean":"2024-09-11T00:57:17.581699+0000","last_became_active":"2024-09-11T00:56:51.152616+0000","last_became_peered":"2024-09-11T00:56:51.152616+0000","last_unstale":"2024-09-11T00:57:17.581699+0000","last_undegraded":"2024-09-11T00:57:17.581699+0000","last_fullsized":"2024-09-11T00:57:17.581699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:02.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.262966+0000","last_change":"2024-09-11T00:56:53.115155+0000","last_active":"2024-09-11T00:58:16.262966+0000","last_peered":"2024-09-11T00:58:16.262966+0000","last_clean":"2024-09-11T00:58:16.262966+0000","last_became_active":"2024-09-11T00:56:53.114865+0000","last_became_peered":"2024-09-11T00:56:53.114865+0000","last_unstale":"2024-09-11T00:58:16.262966+0000","last_undegraded":"2024-09-11T00:58:16.262966+0000","last_fullsized":"2024-09-11T00:58:16.262966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:23.968888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'17","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:19.123128+0000","last_change":"2024-09-11T00:56:51.100887+0000","last_active":"2024-09-11T00:57:19.123128+0000","last_peered":"2024-09-11T00:57:19.123128+0000","last_clean":"2024-09-11T00:57:19.123128+0000","last_became_active":"2024-09-11T00:56:51.100474+0000","last_became_peered":"2024-09-11T00:56:51.100474+0000","last_unstale":"2024-09-11T00:57:19.123128+0000","last_undegraded":"2024-09-11T00:57:19.123128+0000","last_fullsized":"2024-09-11T00:57:19.123128+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:40.884871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.257501+0000","last_change":"2024-09-11T00:57:17.588086+0000","last_active":"2024-09-11T00:58:16.257501+0000","last_peered":"2024-09-11T00:58:16.257501+0000","last_clean":"2024-09-11T00:58:16.257501+0000","last_became_active":"2024-09-11T00:57:17.587785+0000","last_became_peered":"2024-09-11T00:57:17.587785+0000","last_unstale":"2024-09-11T00:58:16.257501+0000","last_undegraded":"2024-09-11T00:58:16.257501+0000","last_fullsized":"2024-09-11T00:58:16.257501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:49.494588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577254+0000","last_change":"2024-09-11T00:56:51.102031+0000","last_active":"2024-09-11T00:57:17.577254+0000","last_peered":"2024-09-11T00:57:17.577254+0000","last_clean":"2024-09-11T00:57:17.577254+0000","last_became_active":"2024-09-11T00:56:51.100614+0000","last_became_peered":"2024-09-11T00:56:51.100614+0000","last_unstale":"2024-09-11T00:57:17.577254+0000","last_undegraded":"2024-09-11T00:57:17.577254+0000","last_fullsized":"2024-09-11T00:57:17.577254+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:52.611261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265773+0000","last_change":"2024-09-11T00:56:53.124806+0000","last_active":"2024-09-11T00:58:16.265773+0000","last_peered":"2024-09-11T00:58:16.265773+0000","last_clean":"2024-09-11T00:58:16.265773+0000","last_became_active":"2024-09-11T00:56:53.124452+0000","last_became_peered":"2024-09-11T00:56:53.124452+0000","last_unstale":"2024-09-11T00:58:16.265773+0000","last_undegraded":"2024-09-11T00:58:16.265773+0000","last_fullsized":"2024-09-11T00:58:16.265773+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:57.934808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160509+0000","last_change":"2024-09-11T00:56:51.153621+0000","last_active":"2024-09-11T00:57:18.160509+0000","last_peered":"2024-09-11T00:57:18.160509+0000","last_clean":"2024-09-11T00:57:18.160509+0000","last_became_active":"2024-09-11T00:56:51.153315+0000","last_became_peered":"2024-09-11T00:56:51.153315+0000","last_unstale":"2024-09-11T00:57:18.160509+0000","last_undegraded":"2024-09-11T00:57:18.160509+0000","last_fullsized":"2024-09-11T00:57:18.160509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:10.058295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.226023+0000","last_change":"2024-09-11T00:56:53.114463+0000","last_active":"2024-09-11T00:58:16.226023+0000","last_peered":"2024-09-11T00:58:16.226023+0000","last_clean":"2024-09-11T00:58:16.226023+0000","last_became_active":"2024-09-11T00:56:53.114099+0000","last_became_peered":"2024-09-11T00:56:53.114099+0000","last_unstale":"2024-09-11T00:58:16.226023+0000","last_undegraded":"2024-09-11T00:58:16.226023+0000","last_fullsized":"2024-09-11T00:58:16.226023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:58.981787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771377+0000","last_change":"2024-09-11T00:56:51.096942+0000","last_active":"2024-09-11T00:57:17.771377+0000","last_peered":"2024-09-11T00:57:17.771377+0000","last_clean":"2024-09-11T00:57:17.771377+0000","last_became_active":"2024-09-11T00:56:51.096584+0000","last_became_peered":"2024-09-11T00:56:51.096584+0000","last_unstale":"2024-09-11T00:57:17.771377+0000","last_undegraded":"2024-09-11T00:57:17.771377+0000","last_fullsized":"2024-09-11T00:57:17.771377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:03.484123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265844+0000","last_change":"2024-09-11T00:56:53.126804+0000","last_active":"2024-09-11T00:58:16.265844+0000","last_peered":"2024-09-11T00:58:16.265844+0000","last_clean":"2024-09-11T00:58:16.265844+0000","last_became_active":"2024-09-11T00:56:53.125408+0000","last_became_peered":"2024-09-11T00:56:53.125408+0000","last_unstale":"2024-09-11T00:58:16.265844+0000","last_undegraded":"2024-09-11T00:58:16.265844+0000","last_fullsized":"2024-09-11T00:58:16.265844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:46.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580130+0000","last_change":"2024-09-11T00:56:51.103755+0000","last_active":"2024-09-11T00:57:17.580130+0000","last_peered":"2024-09-11T00:57:17.580130+0000","last_clean":"2024-09-11T00:57:17.580130+0000","last_became_active":"2024-09-11T00:56:51.103395+0000","last_became_peered":"2024-09-11T00:56:51.103395+0000","last_unstale":"2024-09-11T00:57:17.580130+0000","last_undegraded":"2024-09-11T00:57:17.580130+0000","last_fullsized":"2024-09-11T00:57:17.580130+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:16.782660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.257207+0000","last_change":"2024-09-11T00:56:53.121926+0000","last_active":"2024-09-11T00:58:16.257207+0000","last_peered":"2024-09-11T00:58:16.257207+0000","last_clean":"2024-09-11T00:58:16.257207+0000","last_became_active":"2024-09-11T00:56:53.121603+0000","last_became_peered":"2024-09-11T00:56:53.121603+0000","last_unstale":"2024-09-11T00:58:16.257207+0000","last_undegraded":"2024-09-11T00:58:16.257207+0000","last_fullsized":"2024-09-11T00:58:16.257207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:31.021030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580437+0000","last_change":"2024-09-11T00:56:51.107221+0000","last_active":"2024-09-11T00:57:17.580437+0000","last_peered":"2024-09-11T00:57:17.580437+0000","last_clean":"2024-09-11T00:57:17.580437+0000","last_became_active":"2024-09-11T00:56:51.106849+0000","last_became_peered":"2024-09-11T00:56:51.106849+0000","last_unstale":"2024-09-11T00:57:17.580437+0000","last_undegraded":"2024-09-11T00:57:17.580437+0000","last_fullsized":"2024-09-11T00:57:17.580437+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:46.497230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.269124+0000","last_change":"2024-09-11T00:56:53.120525+0000","last_active":"2024-09-11T00:58:16.269124+0000","last_peered":"2024-09-11T00:58:16.269124+0000","last_clean":"2024-09-11T00:58:16.269124+0000","last_became_active":"2024-09-11T00:56:53.120228+0000","last_became_peered":"2024-09-11T00:56:53.120228+0000","last_unstale":"2024-09-11T00:58:16.269124+0000","last_undegraded":"2024-09-11T00:58:16.269124+0000","last_fullsized":"2024-09-11T00:58:16.269124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:52.047687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266504+0000","last_change":"2024-09-11T00:56:53.120311+0000","last_active":"2024-09-11T00:58:16.266504+0000","last_peered":"2024-09-11T00:58:16.266504+0000","last_clean":"2024-09-11T00:58:16.266504+0000","last_became_active":"2024-09-11T00:56:53.119648+0000","last_became_peered":"2024-09-11T00:56:53.119648+0000","last_unstale":"2024-09-11T00:58:16.266504+0000","last_undegraded":"2024-09-11T00:58:16.266504+0000","last_fullsized":"2024-09-11T00:58:16.266504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:21.187784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581118+0000","last_change":"2024-09-11T00:56:51.103631+0000","last_active":"2024-09-11T00:57:17.581118+0000","last_peered":"2024-09-11T00:57:17.581118+0000","last_clean":"2024-09-11T00:57:17.581118+0000","last_became_active":"2024-09-11T00:56:51.103281+0000","last_became_peered":"2024-09-11T00:56:51.103281+0000","last_unstale":"2024-09-11T00:57:17.581118+0000","last_undegraded":"2024-09-11T00:57:17.581118+0000","last_fullsized":"2024-09-11T00:57:17.581118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:16.910956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.274004+0000","last_change":"2024-09-11T00:56:53.124752+0000","last_active":"2024-09-11T00:58:16.274004+0000","last_peered":"2024-09-11T00:58:16.274004+0000","last_clean":"2024-09-11T00:58:16.274004+0000","last_became_active":"2024-09-11T00:56:53.124245+0000","last_became_peered":"2024-09-11T00:56:53.124245+0000","last_unstale":"2024-09-11T00:58:16.274004+0000","last_undegraded":"2024-09-11T00:58:16.274004+0000","last_fullsized":"2024-09-11T00:58:16.274004+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:10.277903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581704+0000","last_change":"2024-09-11T00:56:51.104624+0000","last_active":"2024-09-11T00:57:17.581704+0000","last_peered":"2024-09-11T00:57:17.581704+0000","last_clean":"2024-09-11T00:57:17.581704+0000","last_became_active":"2024-09-11T00:56:51.104041+0000","last_became_peered":"2024-09-11T00:56:51.104041+0000","last_unstale":"2024-09-11T00:57:17.581704+0000","last_undegraded":"2024-09-11T00:57:17.581704+0000","last_fullsized":"2024-09-11T00:57:17.581704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:10.397455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.272637+0000","last_change":"2024-09-11T00:56:53.118547+0000","last_active":"2024-09-11T00:58:16.272637+0000","last_peered":"2024-09-11T00:58:16.272637+0000","last_clean":"2024-09-11T00:58:16.272637+0000","last_became_active":"2024-09-11T00:56:53.118291+0000","last_became_peered":"2024-09-11T00:56:53.118291+0000","last_unstale":"2024-09-11T00:58:16.272637+0000","last_undegraded":"2024-09-11T00:58:16.272637+0000","last_fullsized":"2024-09-11T00:58:16.272637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:42.012173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771774+0000","last_change":"2024-09-11T00:56:51.153047+0000","last_active":"2024-09-11T00:57:17.771774+0000","last_peered":"2024-09-11T00:57:17.771774+0000","last_clean":"2024-09-11T00:57:17.771774+0000","last_became_active":"2024-09-11T00:56:51.152564+0000","last_became_peered":"2024-09-11T00:56:51.152564+0000","last_unstale":"2024-09-11T00:57:17.771774+0000","last_undegraded":"2024-09-11T00:57:17.771774+0000","last_fullsized":"2024-09-11T00:57:17.771774+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:16.849087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.269357+0000","last_change":"2024-09-11T00:56:53.119926+0000","last_active":"2024-09-11T00:58:16.269357+0000","last_peered":"2024-09-11T00:58:16.269357+0000","last_clean":"2024-09-11T00:58:16.269357+0000","last_became_active":"2024-09-11T00:56:53.119643+0000","last_became_peered":"2024-09-11T00:56:53.119643+0000","last_unstale":"2024-09-11T00:58:16.269357+0000","last_undegraded":"2024-09-11T00:58:16.269357+0000","last_fullsized":"2024-09-11T00:58:16.269357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:51.838958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:47.378123+0000","last_change":"2024-09-11T00:56:51.100837+0000","last_active":"2024-09-11T00:58:47.378123+0000","last_peered":"2024-09-11T00:58:47.378123+0000","last_clean":"2024-09-11T00:58:47.378123+0000","last_became_active":"2024-09-11T00:56:51.100494+0000","last_became_peered":"2024-09-11T00:56:51.100494+0000","last_unstale":"2024-09-11T00:58:47.378123+0000","last_undegraded":"2024-09-11T00:58:47.378123+0000","last_fullsized":"2024-09-11T00:58:47.378123+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:39.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68166,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1101,"num_read_kb":624,"num_write":254,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":204498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286865,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":416921,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906986,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1068434,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":416431,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266740,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":416697,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":1009468,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626494,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":416148,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822724,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1009146,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986250,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":417163,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":415973,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378708,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":477871,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542233,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":913408,"data_stored":477657,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":417795,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:58:53.790 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-11T00:58:53.790 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T00:58:53.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.925+0000 7f1ace564640 1 -- 172.21.15.78:0/2007387387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8109070 msgr2=0x7f1ac810f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.925+0000 7f1ace564640 1 --2- 172.21.15.78:0/2007387387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8109070 0x7f1ac810f870 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009ca0 tx=0x7f1ac002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.925+0000 7f1ace564640 1 -- 172.21.15.78:0/2007387387 shutdown_connections 2024-09-11T00:58:53.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.925+0000 7f1ace564640 1 --2- 172.21.15.78:0/2007387387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8109070 0x7f1ac810f870 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.925+0000 7f1ace564640 1 --2- 172.21.15.78:0/2007387387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ac81086f0 0x7f1ac8108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.925+0000 7f1ace564640 1 --2- 172.21.15.78:0/2007387387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac8102cf0 0x7f1ac81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.925+0000 7f1ace564640 1 -- 172.21.15.78:0/2007387387 >> 172.21.15.78:0/2007387387 conn(0x7f1ac80fe750 msgr2=0x7f1ac8100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.925+0000 7f1ace564640 1 -- 172.21.15.78:0/2007387387 shutdown_connections 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.925+0000 7f1ace564640 1 -- 172.21.15.78:0/2007387387 wait complete. 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.926+0000 7f1ace564640 1 Processor -- start 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.926+0000 7f1ace564640 1 -- start start 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.926+0000 7f1ace564640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ac8102cf0 0x7f1ac80754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.926+0000 7f1ace564640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac81086f0 0x7f1ac80759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.926+0000 7f1ace564640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8109070 0x7f1ac8075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.926+0000 7f1ace564640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8112d30 con 0x7f1ac8109070 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.926+0000 7f1ace564640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8112bb0 con 0x7f1ac8102cf0 2024-09-11T00:58:53.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.926+0000 7f1ace564640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8112eb0 con 0x7f1ac81086f0 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.926+0000 7f1accb01640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8109070 0x7f1ac8075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.926+0000 7f1accb01640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8109070 0x7f1ac8075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42726/0 (socket says 172.21.15.78:42726) 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.926+0000 7f1accb01640 1 -- 172.21.15.78:0/3797454677 learned_addr learned my addr 172.21.15.78:0/3797454677 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1ac7fff640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ac8102cf0 0x7f1ac80754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1ac77fe640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac81086f0 0x7f1ac80759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1accb01640 1 -- 172.21.15.78:0/3797454677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac81086f0 msgr2=0x7f1ac80759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1accb01640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac81086f0 0x7f1ac80759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1accb01640 1 -- 172.21.15.78:0/3797454677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ac8102cf0 msgr2=0x7f1ac80754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1accb01640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ac8102cf0 0x7f1ac80754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1accb01640 1 -- 172.21.15.78:0/3797454677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac81a75a0 con 0x7f1ac8109070 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1ac77fe640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac81086f0 0x7f1ac80759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1accb01640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8109070 0x7f1ac8075f20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00098d0 tx=0x7f1ac00317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1ac57fa640 1 -- 172.21.15.78:0/3797454677 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0003b70 con 0x7f1ac8109070 2024-09-11T00:58:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1ac57fa640 1 -- 172.21.15.78:0/3797454677 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac0003d10 con 0x7f1ac8109070 2024-09-11T00:58:53.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1ace564640 1 -- 172.21.15.78:0/3797454677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac81a7830 con 0x7f1ac8109070 2024-09-11T00:58:53.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1ac57fa640 1 -- 172.21.15.78:0/3797454677 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac002fa20 con 0x7f1ac8109070 2024-09-11T00:58:53.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.927+0000 7f1ace564640 1 -- 172.21.15.78:0/3797454677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac81a7d80 con 0x7f1ac8109070 2024-09-11T00:58:53.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.928+0000 7f1ace564640 1 -- 172.21.15.78:0/3797454677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac81030d0 con 0x7f1ac8109070 2024-09-11T00:58:53.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.930+0000 7f1ac57fa640 1 -- 172.21.15.78:0/3797454677 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f1ac002fbe0 con 0x7f1ac8109070 2024-09-11T00:58:53.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.930+0000 7f1ac57fa640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1ab00b5330 0x7f1ab00b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:53.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.930+0000 7f1ac7fff640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1ab00b5330 0x7f1ab00b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:53.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.930+0000 7f1ac57fa640 1 -- 172.21.15.78:0/3797454677 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1ac0107250 con 0x7f1ac8109070 2024-09-11T00:58:53.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.931+0000 7f1ac7fff640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1ab00b5330 0x7f1ab00b77f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005fd0 tx=0x7f1ab8005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:53.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:53.933+0000 7f1ac57fa640 1 -- 172.21.15.78:0/3797454677 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1ac00ce060 con 0x7f1ac8109070 2024-09-11T00:58:54.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.105+0000 7f1ace564640 1 -- 172.21.15.78:0/3797454677 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ac8063590 con 0x7f1ab00b5330 2024-09-11T00:58:54.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.109+0000 7f1ac57fa640 1 -- 172.21.15.78:0/3797454677 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259130 (secure 0 0 0) 0x7f1ac8063590 con 0x7f1ab00b5330 2024-09-11T00:58:54.114 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:54.115 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T00:58:54.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.112+0000 7f1ace564640 1 -- 172.21.15.78:0/3797454677 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1ab00b5330 msgr2=0x7f1ab00b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:54.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.112+0000 7f1ace564640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1ab00b5330 0x7f1ab00b77f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005fd0 tx=0x7f1ab8005850 comp rx=0 tx=0).stop 2024-09-11T00:58:54.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.112+0000 7f1ace564640 1 -- 172.21.15.78:0/3797454677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8109070 msgr2=0x7f1ac8075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:54.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.112+0000 7f1ace564640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8109070 0x7f1ac8075f20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00098d0 tx=0x7f1ac00317b0 comp rx=0 tx=0).stop 2024-09-11T00:58:54.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.112+0000 7f1ace564640 1 -- 172.21.15.78:0/3797454677 shutdown_connections 2024-09-11T00:58:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.112+0000 7f1ace564640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1ab00b5330 0x7f1ab00b77f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.112+0000 7f1ace564640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8109070 0x7f1ac8075f20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.112+0000 7f1ace564640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac81086f0 0x7f1ac80759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.112+0000 7f1ace564640 1 --2- 172.21.15.78:0/3797454677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ac8102cf0 0x7f1ac80754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.112+0000 7f1ace564640 1 -- 172.21.15.78:0/3797454677 >> 172.21.15.78:0/3797454677 conn(0x7f1ac80fe750 msgr2=0x7f1ac8100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.113+0000 7f1ace564640 1 -- 172.21.15.78:0/3797454677 shutdown_connections 2024-09-11T00:58:54.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.113+0000 7f1ace564640 1 -- 172.21.15.78:0/3797454677 wait complete. 2024-09-11T00:58:54.125 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-11T00:58:54.049134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658768,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1192,"num_read_kb":698,"num_write":379,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2952965,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":812,"ondisk_log_size":812,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376352,"kb_used_data":10464,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565472,"statfs":{"total":1151940427776,"available":1151555043328,"internally_reserved":0,"allocated":10715136,"data_stored":6959704,"data_compressed":179373,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":56,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002926"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266212+0000","last_change":"2024-09-11T00:56:53.123622+0000","last_active":"2024-09-11T00:58:16.266212+0000","last_peered":"2024-09-11T00:58:16.266212+0000","last_clean":"2024-09-11T00:58:16.266212+0000","last_became_active":"2024-09-11T00:56:53.121460+0000","last_became_peered":"2024-09-11T00:56:53.121460+0000","last_unstale":"2024-09-11T00:58:16.266212+0000","last_undegraded":"2024-09-11T00:58:16.266212+0000","last_fullsized":"2024-09-11T00:58:16.266212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:30.942093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264953+0000","last_change":"2024-09-11T00:56:53.120189+0000","last_active":"2024-09-11T00:58:16.264953+0000","last_peered":"2024-09-11T00:58:16.264953+0000","last_clean":"2024-09-11T00:58:16.264953+0000","last_became_active":"2024-09-11T00:56:53.119853+0000","last_became_peered":"2024-09-11T00:56:53.119853+0000","last_unstale":"2024-09-11T00:58:16.264953+0000","last_undegraded":"2024-09-11T00:58:16.264953+0000","last_fullsized":"2024-09-11T00:58:16.264953+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:36.927113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.272647+0000","last_change":"2024-09-11T00:56:53.117832+0000","last_active":"2024-09-11T00:58:16.272647+0000","last_peered":"2024-09-11T00:58:16.272647+0000","last_clean":"2024-09-11T00:58:16.272647+0000","last_became_active":"2024-09-11T00:56:53.117508+0000","last_became_peered":"2024-09-11T00:56:53.117508+0000","last_unstale":"2024-09-11T00:58:16.272647+0000","last_undegraded":"2024-09-11T00:58:16.272647+0000","last_fullsized":"2024-09-11T00:58:16.272647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:00.628673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.271210+0000","last_change":"2024-09-11T00:56:53.128302+0000","last_active":"2024-09-11T00:58:16.271210+0000","last_peered":"2024-09-11T00:58:16.271210+0000","last_clean":"2024-09-11T00:58:16.271210+0000","last_became_active":"2024-09-11T00:56:53.127584+0000","last_became_peered":"2024-09-11T00:56:53.127584+0000","last_unstale":"2024-09-11T00:58:16.271210+0000","last_undegraded":"2024-09-11T00:58:16.271210+0000","last_fullsized":"2024-09-11T00:58:16.271210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:41.963656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.248383+0000","last_change":"2024-09-11T00:56:53.116249+0000","last_active":"2024-09-11T00:58:16.248383+0000","last_peered":"2024-09-11T00:58:16.248383+0000","last_clean":"2024-09-11T00:58:16.248383+0000","last_became_active":"2024-09-11T00:56:53.115881+0000","last_became_peered":"2024-09-11T00:56:53.115881+0000","last_unstale":"2024-09-11T00:58:16.248383+0000","last_undegraded":"2024-09-11T00:58:16.248383+0000","last_fullsized":"2024-09-11T00:58:16.248383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:47.432607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.262557+0000","last_change":"2024-09-11T00:56:53.116992+0000","last_active":"2024-09-11T00:58:16.262557+0000","last_peered":"2024-09-11T00:58:16.262557+0000","last_clean":"2024-09-11T00:58:16.262557+0000","last_became_active":"2024-09-11T00:56:53.116629+0000","last_became_peered":"2024-09-11T00:56:53.116629+0000","last_unstale":"2024-09-11T00:58:16.262557+0000","last_undegraded":"2024-09-11T00:58:16.262557+0000","last_fullsized":"2024-09-11T00:58:16.262557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:25.410377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266888+0000","last_change":"2024-09-11T00:56:53.127873+0000","last_active":"2024-09-11T00:58:16.266888+0000","last_peered":"2024-09-11T00:58:16.266888+0000","last_clean":"2024-09-11T00:58:16.266888+0000","last_became_active":"2024-09-11T00:56:53.127575+0000","last_became_peered":"2024-09-11T00:56:53.127575+0000","last_unstale":"2024-09-11T00:58:16.266888+0000","last_undegraded":"2024-09-11T00:58:16.266888+0000","last_fullsized":"2024-09-11T00:58:16.266888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:12.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.261154+0000","last_change":"2024-09-11T00:56:53.117727+0000","last_active":"2024-09-11T00:58:16.261154+0000","last_peered":"2024-09-11T00:58:16.261154+0000","last_clean":"2024-09-11T00:58:16.261154+0000","last_became_active":"2024-09-11T00:56:53.117444+0000","last_became_peered":"2024-09-11T00:56:53.117444+0000","last_unstale":"2024-09-11T00:58:16.261154+0000","last_undegraded":"2024-09-11T00:58:16.261154+0000","last_fullsized":"2024-09-11T00:58:16.261154+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:28.625231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268315+0000","last_change":"2024-09-11T00:56:53.128974+0000","last_active":"2024-09-11T00:58:16.268315+0000","last_peered":"2024-09-11T00:58:16.268315+0000","last_clean":"2024-09-11T00:58:16.268315+0000","last_became_active":"2024-09-11T00:56:53.128650+0000","last_became_peered":"2024-09-11T00:56:53.128650+0000","last_unstale":"2024-09-11T00:58:16.268315+0000","last_undegraded":"2024-09-11T00:58:16.268315+0000","last_fullsized":"2024-09-11T00:58:16.268315+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:07.251373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.262269+0000","last_change":"2024-09-11T00:56:53.117816+0000","last_active":"2024-09-11T00:58:16.262269+0000","last_peered":"2024-09-11T00:58:16.262269+0000","last_clean":"2024-09-11T00:58:16.262269+0000","last_became_active":"2024-09-11T00:56:53.117449+0000","last_became_peered":"2024-09-11T00:56:53.117449+0000","last_unstale":"2024-09-11T00:58:16.262269+0000","last_undegraded":"2024-09-11T00:58:16.262269+0000","last_fullsized":"2024-09-11T00:58:16.262269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:09.058869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265129+0000","last_change":"2024-09-11T00:56:53.124616+0000","last_active":"2024-09-11T00:58:16.265129+0000","last_peered":"2024-09-11T00:58:16.265129+0000","last_clean":"2024-09-11T00:58:16.265129+0000","last_became_active":"2024-09-11T00:56:53.124309+0000","last_became_peered":"2024-09-11T00:56:53.124309+0000","last_unstale":"2024-09-11T00:58:16.265129+0000","last_undegraded":"2024-09-11T00:58:16.265129+0000","last_fullsized":"2024-09-11T00:58:16.265129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:14.469909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.257580+0000","last_change":"2024-09-11T00:56:53.126263+0000","last_active":"2024-09-11T00:58:16.257580+0000","last_peered":"2024-09-11T00:58:16.257580+0000","last_clean":"2024-09-11T00:58:16.257580+0000","last_became_active":"2024-09-11T00:56:53.125923+0000","last_became_peered":"2024-09-11T00:56:53.125923+0000","last_unstale":"2024-09-11T00:58:16.257580+0000","last_undegraded":"2024-09-11T00:58:16.257580+0000","last_fullsized":"2024-09-11T00:58:16.257580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:12:33.627372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.258526+0000","last_change":"2024-09-11T00:56:53.105292+0000","last_active":"2024-09-11T00:58:16.258526+0000","last_peered":"2024-09-11T00:58:16.258526+0000","last_clean":"2024-09-11T00:58:16.258526+0000","last_became_active":"2024-09-11T00:56:53.104830+0000","last_became_peered":"2024-09-11T00:56:53.104830+0000","last_unstale":"2024-09-11T00:58:16.258526+0000","last_undegraded":"2024-09-11T00:58:16.258526+0000","last_fullsized":"2024-09-11T00:58:16.258526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:14.136952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264895+0000","last_change":"2024-09-11T00:56:53.120463+0000","last_active":"2024-09-11T00:58:16.264895+0000","last_peered":"2024-09-11T00:58:16.264895+0000","last_clean":"2024-09-11T00:58:16.264895+0000","last_became_active":"2024-09-11T00:56:53.120167+0000","last_became_peered":"2024-09-11T00:56:53.120167+0000","last_unstale":"2024-09-11T00:58:16.264895+0000","last_undegraded":"2024-09-11T00:58:16.264895+0000","last_fullsized":"2024-09-11T00:58:16.264895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:49.783452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265508+0000","last_change":"2024-09-11T00:56:53.123312+0000","last_active":"2024-09-11T00:58:16.265508+0000","last_peered":"2024-09-11T00:58:16.265508+0000","last_clean":"2024-09-11T00:58:16.265508+0000","last_became_active":"2024-09-11T00:56:53.123015+0000","last_became_peered":"2024-09-11T00:56:53.123015+0000","last_unstale":"2024-09-11T00:58:16.265508+0000","last_undegraded":"2024-09-11T00:58:16.265508+0000","last_fullsized":"2024-09-11T00:58:16.265508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:22.069629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.272553+0000","last_change":"2024-09-11T00:56:53.125701+0000","last_active":"2024-09-11T00:58:16.272553+0000","last_peered":"2024-09-11T00:58:16.272553+0000","last_clean":"2024-09-11T00:58:16.272553+0000","last_became_active":"2024-09-11T00:56:53.125362+0000","last_became_peered":"2024-09-11T00:56:53.125362+0000","last_unstale":"2024-09-11T00:58:16.272553+0000","last_undegraded":"2024-09-11T00:58:16.272553+0000","last_fullsized":"2024-09-11T00:58:16.272553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:14.594324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266425+0000","last_change":"2024-09-11T00:56:53.124275+0000","last_active":"2024-09-11T00:58:16.266425+0000","last_peered":"2024-09-11T00:58:16.266425+0000","last_clean":"2024-09-11T00:58:16.266425+0000","last_became_active":"2024-09-11T00:56:53.122695+0000","last_became_peered":"2024-09-11T00:56:53.122695+0000","last_unstale":"2024-09-11T00:58:16.266425+0000","last_undegraded":"2024-09-11T00:58:16.266425+0000","last_fullsized":"2024-09-11T00:58:16.266425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:23.955266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264639+0000","last_change":"2024-09-11T00:56:53.119077+0000","last_active":"2024-09-11T00:58:16.264639+0000","last_peered":"2024-09-11T00:58:16.264639+0000","last_clean":"2024-09-11T00:58:16.264639+0000","last_became_active":"2024-09-11T00:56:53.118693+0000","last_became_peered":"2024-09-11T00:56:53.118693+0000","last_unstale":"2024-09-11T00:58:16.264639+0000","last_undegraded":"2024-09-11T00:58:16.264639+0000","last_fullsized":"2024-09-11T00:58:16.264639+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:39.135391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265439+0000","last_change":"2024-09-11T00:56:53.115430+0000","last_active":"2024-09-11T00:58:16.265439+0000","last_peered":"2024-09-11T00:58:16.265439+0000","last_clean":"2024-09-11T00:58:16.265439+0000","last_became_active":"2024-09-11T00:56:53.115111+0000","last_became_peered":"2024-09-11T00:56:53.115111+0000","last_unstale":"2024-09-11T00:58:16.265439+0000","last_undegraded":"2024-09-11T00:58:16.265439+0000","last_fullsized":"2024-09-11T00:58:16.265439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:11.306661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264277+0000","last_change":"2024-09-11T00:57:17.583260+0000","last_active":"2024-09-11T00:58:16.264277+0000","last_peered":"2024-09-11T00:58:16.264277+0000","last_clean":"2024-09-11T00:58:16.264277+0000","last_became_active":"2024-09-11T00:57:17.582909+0000","last_became_peered":"2024-09-11T00:57:17.582909+0000","last_unstale":"2024-09-11T00:58:16.264277+0000","last_undegraded":"2024-09-11T00:58:16.264277+0000","last_fullsized":"2024-09-11T00:58:16.264277+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:42.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268531+0000","last_change":"2024-09-11T00:56:53.121303+0000","last_active":"2024-09-11T00:58:16.268531+0000","last_peered":"2024-09-11T00:58:16.268531+0000","last_clean":"2024-09-11T00:58:16.268531+0000","last_became_active":"2024-09-11T00:56:53.121004+0000","last_became_peered":"2024-09-11T00:56:53.121004+0000","last_unstale":"2024-09-11T00:58:16.268531+0000","last_undegraded":"2024-09-11T00:58:16.268531+0000","last_fullsized":"2024-09-11T00:58:16.268531+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:36.899870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266931+0000","last_change":"2024-09-11T00:56:53.125042+0000","last_active":"2024-09-11T00:58:16.266931+0000","last_peered":"2024-09-11T00:58:16.266931+0000","last_clean":"2024-09-11T00:58:16.266931+0000","last_became_active":"2024-09-11T00:56:53.124661+0000","last_became_peered":"2024-09-11T00:56:53.124661+0000","last_unstale":"2024-09-11T00:58:16.266931+0000","last_undegraded":"2024-09-11T00:58:16.266931+0000","last_fullsized":"2024-09-11T00:58:16.266931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:34.392172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268728+0000","last_change":"2024-09-11T00:56:53.116544+0000","last_active":"2024-09-11T00:58:16.268728+0000","last_peered":"2024-09-11T00:58:16.268728+0000","last_clean":"2024-09-11T00:58:16.268728+0000","last_became_active":"2024-09-11T00:56:53.116262+0000","last_became_peered":"2024-09-11T00:56:53.116262+0000","last_unstale":"2024-09-11T00:58:16.268728+0000","last_undegraded":"2024-09-11T00:58:16.268728+0000","last_fullsized":"2024-09-11T00:58:16.268728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:34.500669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.263134+0000","last_change":"2024-09-11T00:56:53.116967+0000","last_active":"2024-09-11T00:58:16.263134+0000","last_peered":"2024-09-11T00:58:16.263134+0000","last_clean":"2024-09-11T00:58:16.263134+0000","last_became_active":"2024-09-11T00:56:53.116405+0000","last_became_peered":"2024-09-11T00:56:53.116405+0000","last_unstale":"2024-09-11T00:58:16.263134+0000","last_undegraded":"2024-09-11T00:58:16.263134+0000","last_fullsized":"2024-09-11T00:58:16.263134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:33.521964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268124+0000","last_change":"2024-09-11T00:56:53.116727+0000","last_active":"2024-09-11T00:58:16.268124+0000","last_peered":"2024-09-11T00:58:16.268124+0000","last_clean":"2024-09-11T00:58:16.268124+0000","last_became_active":"2024-09-11T00:56:53.116416+0000","last_became_peered":"2024-09-11T00:56:53.116416+0000","last_unstale":"2024-09-11T00:58:16.268124+0000","last_undegraded":"2024-09-11T00:58:16.268124+0000","last_fullsized":"2024-09-11T00:58:16.268124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:28.277909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265487+0000","last_change":"2024-09-11T00:56:53.123141+0000","last_active":"2024-09-11T00:58:16.265487+0000","last_peered":"2024-09-11T00:58:16.265487+0000","last_clean":"2024-09-11T00:58:16.265487+0000","last_became_active":"2024-09-11T00:56:53.122811+0000","last_became_peered":"2024-09-11T00:56:53.122811+0000","last_unstale":"2024-09-11T00:58:16.265487+0000","last_undegraded":"2024-09-11T00:58:16.265487+0000","last_fullsized":"2024-09-11T00:58:16.265487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:47.175120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.260452+0000","last_change":"2024-09-11T00:56:53.124213+0000","last_active":"2024-09-11T00:58:16.260452+0000","last_peered":"2024-09-11T00:58:16.260452+0000","last_clean":"2024-09-11T00:58:16.260452+0000","last_became_active":"2024-09-11T00:56:53.123757+0000","last_became_peered":"2024-09-11T00:56:53.123757+0000","last_unstale":"2024-09-11T00:58:16.260452+0000","last_undegraded":"2024-09-11T00:58:16.260452+0000","last_fullsized":"2024-09-11T00:58:16.260452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:54.263476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.260255+0000","last_change":"2024-09-11T00:56:53.114016+0000","last_active":"2024-09-11T00:58:16.260255+0000","last_peered":"2024-09-11T00:58:16.260255+0000","last_clean":"2024-09-11T00:58:16.260255+0000","last_became_active":"2024-09-11T00:56:53.113719+0000","last_became_peered":"2024-09-11T00:56:53.113719+0000","last_unstale":"2024-09-11T00:58:16.260255+0000","last_undegraded":"2024-09-11T00:58:16.260255+0000","last_fullsized":"2024-09-11T00:58:16.260255+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:48:06.623810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.258203+0000","last_change":"2024-09-11T00:56:53.124798+0000","last_active":"2024-09-11T00:58:16.258203+0000","last_peered":"2024-09-11T00:58:16.258203+0000","last_clean":"2024-09-11T00:58:16.258203+0000","last_became_active":"2024-09-11T00:56:53.124381+0000","last_became_peered":"2024-09-11T00:56:53.124381+0000","last_unstale":"2024-09-11T00:58:16.258203+0000","last_undegraded":"2024-09-11T00:58:16.258203+0000","last_fullsized":"2024-09-11T00:58:16.258203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:37.823112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265603+0000","last_change":"2024-09-11T00:56:53.115297+0000","last_active":"2024-09-11T00:58:16.265603+0000","last_peered":"2024-09-11T00:58:16.265603+0000","last_clean":"2024-09-11T00:58:16.265603+0000","last_became_active":"2024-09-11T00:56:53.114958+0000","last_became_peered":"2024-09-11T00:56:53.114958+0000","last_unstale":"2024-09-11T00:58:16.265603+0000","last_undegraded":"2024-09-11T00:58:16.265603+0000","last_fullsized":"2024-09-11T00:58:16.265603+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:47.612610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.268541+0000","last_change":"2024-09-11T00:56:53.126717+0000","last_active":"2024-09-11T00:58:16.268541+0000","last_peered":"2024-09-11T00:58:16.268541+0000","last_clean":"2024-09-11T00:58:16.268541+0000","last_became_active":"2024-09-11T00:56:53.126387+0000","last_became_peered":"2024-09-11T00:56:53.126387+0000","last_unstale":"2024-09-11T00:58:16.268541+0000","last_undegraded":"2024-09-11T00:58:16.268541+0000","last_fullsized":"2024-09-11T00:58:16.268541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:52.429127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.260388+0000","last_change":"2024-09-11T00:56:53.126257+0000","last_active":"2024-09-11T00:58:16.260388+0000","last_peered":"2024-09-11T00:58:16.260388+0000","last_clean":"2024-09-11T00:58:16.260388+0000","last_became_active":"2024-09-11T00:56:53.125146+0000","last_became_peered":"2024-09-11T00:56:53.125146+0000","last_unstale":"2024-09-11T00:58:16.260388+0000","last_undegraded":"2024-09-11T00:58:16.260388+0000","last_fullsized":"2024-09-11T00:58:16.260388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:59.046126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.259691+0000","last_change":"2024-09-11T00:56:53.119187+0000","last_active":"2024-09-11T00:58:16.259691+0000","last_peered":"2024-09-11T00:58:16.259691+0000","last_clean":"2024-09-11T00:58:16.259691+0000","last_became_active":"2024-09-11T00:56:53.118850+0000","last_became_peered":"2024-09-11T00:56:53.118850+0000","last_unstale":"2024-09-11T00:58:16.259691+0000","last_undegraded":"2024-09-11T00:58:16.259691+0000","last_fullsized":"2024-09-11T00:58:16.259691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:05.403876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'7","reported_seq":155,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:49.714727+0000","last_change":"2024-09-11T00:57:17.591689+0000","last_active":"2024-09-11T00:58:49.714727+0000","last_peered":"2024-09-11T00:58:49.714727+0000","last_clean":"2024-09-11T00:58:49.714727+0000","last_became_active":"2024-09-11T00:57:17.587389+0000","last_became_peered":"2024-09-11T00:57:17.587389+0000","last_unstale":"2024-09-11T00:58:49.714727+0000","last_undegraded":"2024-09-11T00:58:49.714727+0000","last_fullsized":"2024-09-11T00:58:49.714727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:25.162270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266423+0000","last_change":"2024-09-11T00:56:53.118049+0000","last_active":"2024-09-11T00:58:16.266423+0000","last_peered":"2024-09-11T00:58:16.266423+0000","last_clean":"2024-09-11T00:58:16.266423+0000","last_became_active":"2024-09-11T00:56:53.117685+0000","last_became_peered":"2024-09-11T00:56:53.117685+0000","last_unstale":"2024-09-11T00:58:16.266423+0000","last_undegraded":"2024-09-11T00:58:16.266423+0000","last_fullsized":"2024-09-11T00:58:16.266423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:17.017594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.578639+0000","last_change":"2024-09-11T00:56:51.101690+0000","last_active":"2024-09-11T00:57:17.578639+0000","last_peered":"2024-09-11T00:57:17.578639+0000","last_clean":"2024-09-11T00:57:17.578639+0000","last_became_active":"2024-09-11T00:56:51.101055+0000","last_became_peered":"2024-09-11T00:56:51.101055+0000","last_unstale":"2024-09-11T00:57:17.578639+0000","last_undegraded":"2024-09-11T00:57:17.578639+0000","last_fullsized":"2024-09-11T00:57:17.578639+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:28.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576176+0000","last_change":"2024-09-11T00:56:51.101700+0000","last_active":"2024-09-11T00:57:17.576176+0000","last_peered":"2024-09-11T00:57:17.576176+0000","last_clean":"2024-09-11T00:57:17.576176+0000","last_became_active":"2024-09-11T00:56:51.101044+0000","last_became_peered":"2024-09-11T00:56:51.101044+0000","last_unstale":"2024-09-11T00:57:17.576176+0000","last_undegraded":"2024-09-11T00:57:17.576176+0000","last_fullsized":"2024-09-11T00:57:17.576176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:12.121126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.261909+0000","last_change":"2024-09-11T00:56:53.121652+0000","last_active":"2024-09-11T00:58:16.261909+0000","last_peered":"2024-09-11T00:58:16.261909+0000","last_clean":"2024-09-11T00:58:16.261909+0000","last_became_active":"2024-09-11T00:56:53.121375+0000","last_became_peered":"2024-09-11T00:56:53.121375+0000","last_unstale":"2024-09-11T00:58:16.261909+0000","last_undegraded":"2024-09-11T00:58:16.261909+0000","last_fullsized":"2024-09-11T00:58:16.261909+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:39.154985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583515+0000","last_change":"2024-09-11T00:56:51.104367+0000","last_active":"2024-09-11T00:57:17.583515+0000","last_peered":"2024-09-11T00:57:17.583515+0000","last_clean":"2024-09-11T00:57:17.583515+0000","last_became_active":"2024-09-11T00:56:51.104045+0000","last_became_peered":"2024-09-11T00:56:51.104045+0000","last_unstale":"2024-09-11T00:57:17.583515+0000","last_undegraded":"2024-09-11T00:57:17.583515+0000","last_fullsized":"2024-09-11T00:57:17.583515+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:03.989437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264374+0000","last_change":"2024-09-11T00:56:53.116167+0000","last_active":"2024-09-11T00:58:16.264374+0000","last_peered":"2024-09-11T00:58:16.264374+0000","last_clean":"2024-09-11T00:58:16.264374+0000","last_became_active":"2024-09-11T00:56:53.115857+0000","last_became_peered":"2024-09-11T00:56:53.115857+0000","last_unstale":"2024-09-11T00:58:16.264374+0000","last_undegraded":"2024-09-11T00:58:16.264374+0000","last_fullsized":"2024-09-11T00:58:16.264374+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:04.476094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":242,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:48.927958+0000","last_change":"2024-09-11T00:56:51.103287+0000","last_active":"2024-09-11T00:58:48.927958+0000","last_peered":"2024-09-11T00:58:48.927958+0000","last_clean":"2024-09-11T00:58:48.927958+0000","last_became_active":"2024-09-11T00:56:51.102953+0000","last_became_peered":"2024-09-11T00:56:51.102953+0000","last_unstale":"2024-09-11T00:58:48.927958+0000","last_undegraded":"2024-09-11T00:58:48.927958+0000","last_fullsized":"2024-09-11T00:58:48.927958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:49.701612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":213,"num_read_kb":213,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.273793+0000","last_change":"2024-09-11T00:56:53.125130+0000","last_active":"2024-09-11T00:58:16.273793+0000","last_peered":"2024-09-11T00:58:16.273793+0000","last_clean":"2024-09-11T00:58:16.273793+0000","last_became_active":"2024-09-11T00:56:53.124442+0000","last_became_peered":"2024-09-11T00:56:53.124442+0000","last_unstale":"2024-09-11T00:58:16.273793+0000","last_undegraded":"2024-09-11T00:58:16.273793+0000","last_fullsized":"2024-09-11T00:58:16.273793+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:24.439430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586976+0000","last_change":"2024-09-11T00:56:51.104802+0000","last_active":"2024-09-11T00:57:17.586976+0000","last_peered":"2024-09-11T00:57:17.586976+0000","last_clean":"2024-09-11T00:57:17.586976+0000","last_became_active":"2024-09-11T00:56:51.104335+0000","last_became_peered":"2024-09-11T00:56:51.104335+0000","last_unstale":"2024-09-11T00:57:17.586976+0000","last_undegraded":"2024-09-11T00:57:17.586976+0000","last_fullsized":"2024-09-11T00:57:17.586976+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:26.894084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.263190+0000","last_change":"2024-09-11T00:57:17.584979+0000","last_active":"2024-09-11T00:58:16.263190+0000","last_peered":"2024-09-11T00:58:16.263190+0000","last_clean":"2024-09-11T00:58:16.263190+0000","last_became_active":"2024-09-11T00:57:17.584518+0000","last_became_peered":"2024-09-11T00:57:17.584518+0000","last_unstale":"2024-09-11T00:58:16.263190+0000","last_undegraded":"2024-09-11T00:58:16.263190+0000","last_fullsized":"2024-09-11T00:58:16.263190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:44.762621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583377+0000","last_change":"2024-09-11T00:56:51.154544+0000","last_active":"2024-09-11T00:57:17.583377+0000","last_peered":"2024-09-11T00:57:17.583377+0000","last_clean":"2024-09-11T00:57:17.583377+0000","last_became_active":"2024-09-11T00:56:51.154195+0000","last_became_peered":"2024-09-11T00:56:51.154195+0000","last_unstale":"2024-09-11T00:57:17.583377+0000","last_undegraded":"2024-09-11T00:57:17.583377+0000","last_fullsized":"2024-09-11T00:57:17.583377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:57:24.530947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.274114+0000","last_change":"2024-09-11T00:56:53.127728+0000","last_active":"2024-09-11T00:58:16.274114+0000","last_peered":"2024-09-11T00:58:16.274114+0000","last_clean":"2024-09-11T00:58:16.274114+0000","last_became_active":"2024-09-11T00:56:53.127423+0000","last_became_peered":"2024-09-11T00:56:53.127423+0000","last_unstale":"2024-09-11T00:58:16.274114+0000","last_undegraded":"2024-09-11T00:58:16.274114+0000","last_fullsized":"2024-09-11T00:58:16.274114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:55.717123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'34","reported_seq":277,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:48.931068+0000","last_change":"2024-09-11T00:56:51.100774+0000","last_active":"2024-09-11T00:58:48.931068+0000","last_peered":"2024-09-11T00:58:48.931068+0000","last_clean":"2024-09-11T00:58:48.931068+0000","last_became_active":"2024-09-11T00:56:51.100398+0000","last_became_peered":"2024-09-11T00:56:51.100398+0000","last_unstale":"2024-09-11T00:58:48.931068+0000","last_undegraded":"2024-09-11T00:58:48.931068+0000","last_fullsized":"2024-09-11T00:58:48.931068+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:51.975236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60904,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":169,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.267905+0000","last_change":"2024-09-11T00:56:53.120113+0000","last_active":"2024-09-11T00:58:16.267905+0000","last_peered":"2024-09-11T00:58:16.267905+0000","last_clean":"2024-09-11T00:58:16.267905+0000","last_became_active":"2024-09-11T00:56:53.119733+0000","last_became_peered":"2024-09-11T00:56:53.119733+0000","last_unstale":"2024-09-11T00:58:16.267905+0000","last_undegraded":"2024-09-11T00:58:16.267905+0000","last_fullsized":"2024-09-11T00:58:16.267905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:59.541710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580361+0000","last_change":"2024-09-11T00:56:51.104234+0000","last_active":"2024-09-11T00:57:17.580361+0000","last_peered":"2024-09-11T00:57:17.580361+0000","last_clean":"2024-09-11T00:57:17.580361+0000","last_became_active":"2024-09-11T00:56:51.103631+0000","last_became_peered":"2024-09-11T00:56:51.103631+0000","last_unstale":"2024-09-11T00:57:17.580361+0000","last_undegraded":"2024-09-11T00:57:17.580361+0000","last_fullsized":"2024-09-11T00:57:17.580361+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:47.378135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.267835+0000","last_change":"2024-09-11T00:56:53.122231+0000","last_active":"2024-09-11T00:58:16.267835+0000","last_peered":"2024-09-11T00:58:16.267835+0000","last_clean":"2024-09-11T00:58:16.267835+0000","last_became_active":"2024-09-11T00:56:53.121903+0000","last_became_peered":"2024-09-11T00:56:53.121903+0000","last_unstale":"2024-09-11T00:58:16.267835+0000","last_undegraded":"2024-09-11T00:58:16.267835+0000","last_fullsized":"2024-09-11T00:58:16.267835+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:44.246919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.582884+0000","last_change":"2024-09-11T00:56:51.105625+0000","last_active":"2024-09-11T00:57:17.582884+0000","last_peered":"2024-09-11T00:57:17.582884+0000","last_clean":"2024-09-11T00:57:17.582884+0000","last_became_active":"2024-09-11T00:56:51.105291+0000","last_became_peered":"2024-09-11T00:56:51.105291+0000","last_unstale":"2024-09-11T00:57:17.582884+0000","last_undegraded":"2024-09-11T00:57:17.582884+0000","last_fullsized":"2024-09-11T00:57:17.582884+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:24.237174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.269475+0000","last_change":"2024-09-11T00:56:53.126272+0000","last_active":"2024-09-11T00:58:16.269475+0000","last_peered":"2024-09-11T00:58:16.269475+0000","last_clean":"2024-09-11T00:58:16.269475+0000","last_became_active":"2024-09-11T00:56:53.125964+0000","last_became_peered":"2024-09-11T00:56:53.125964+0000","last_unstale":"2024-09-11T00:58:16.269475+0000","last_undegraded":"2024-09-11T00:58:16.269475+0000","last_fullsized":"2024-09-11T00:58:16.269475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:36.331220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:49.423686+0000","last_change":"2024-09-11T00:56:51.101053+0000","last_active":"2024-09-11T00:58:49.423686+0000","last_peered":"2024-09-11T00:58:49.423686+0000","last_clean":"2024-09-11T00:58:49.423686+0000","last_became_active":"2024-09-11T00:56:51.100714+0000","last_became_peered":"2024-09-11T00:56:51.100714+0000","last_unstale":"2024-09-11T00:58:49.423686+0000","last_undegraded":"2024-09-11T00:58:49.423686+0000","last_fullsized":"2024-09-11T00:58:49.423686+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:57.709126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266408+0000","last_change":"2024-09-11T00:56:53.119973+0000","last_active":"2024-09-11T00:58:16.266408+0000","last_peered":"2024-09-11T00:58:16.266408+0000","last_clean":"2024-09-11T00:58:16.266408+0000","last_became_active":"2024-09-11T00:56:53.118972+0000","last_became_peered":"2024-09-11T00:56:53.118972+0000","last_unstale":"2024-09-11T00:58:16.266408+0000","last_undegraded":"2024-09-11T00:58:16.266408+0000","last_fullsized":"2024-09-11T00:58:16.266408+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:35.912920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.598033+0000","last_change":"2024-09-11T00:57:17.597537+0000","last_active":"2024-09-11T00:57:17.598033+0000","last_peered":"2024-09-11T00:57:17.598033+0000","last_clean":"2024-09-11T00:57:17.598033+0000","last_became_active":"2024-09-11T00:57:17.581631+0000","last_became_peered":"2024-09-11T00:57:17.581631+0000","last_unstale":"2024-09-11T00:57:17.598033+0000","last_undegraded":"2024-09-11T00:57:17.598033+0000","last_fullsized":"2024-09-11T00:57:17.598033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:34.278845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.264649+0000","last_change":"2024-09-11T00:56:53.120972+0000","last_active":"2024-09-11T00:58:16.264649+0000","last_peered":"2024-09-11T00:58:16.264649+0000","last_clean":"2024-09-11T00:58:16.264649+0000","last_became_active":"2024-09-11T00:56:53.120568+0000","last_became_peered":"2024-09-11T00:56:53.120568+0000","last_unstale":"2024-09-11T00:58:16.264649+0000","last_undegraded":"2024-09-11T00:58:16.264649+0000","last_fullsized":"2024-09-11T00:58:16.264649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:19.696201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581943+0000","last_change":"2024-09-11T00:56:51.103756+0000","last_active":"2024-09-11T00:57:17.581943+0000","last_peered":"2024-09-11T00:57:17.581943+0000","last_clean":"2024-09-11T00:57:17.581943+0000","last_became_active":"2024-09-11T00:56:51.103406+0000","last_became_peered":"2024-09-11T00:56:51.103406+0000","last_unstale":"2024-09-11T00:57:17.581943+0000","last_undegraded":"2024-09-11T00:57:17.581943+0000","last_fullsized":"2024-09-11T00:57:17.581943+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:28.022346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.257373+0000","last_change":"2024-09-11T00:56:53.128628+0000","last_active":"2024-09-11T00:58:16.257373+0000","last_peered":"2024-09-11T00:58:16.257373+0000","last_clean":"2024-09-11T00:58:16.257373+0000","last_became_active":"2024-09-11T00:56:53.127197+0000","last_became_peered":"2024-09-11T00:56:53.127197+0000","last_unstale":"2024-09-11T00:58:16.257373+0000","last_undegraded":"2024-09-11T00:58:16.257373+0000","last_fullsized":"2024-09-11T00:58:16.257373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:38.753676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.576498+0000","last_change":"2024-09-11T00:56:51.101398+0000","last_active":"2024-09-11T00:57:17.576498+0000","last_peered":"2024-09-11T00:57:17.576498+0000","last_clean":"2024-09-11T00:57:17.576498+0000","last_became_active":"2024-09-11T00:56:51.101034+0000","last_became_peered":"2024-09-11T00:56:51.101034+0000","last_unstale":"2024-09-11T00:57:17.576498+0000","last_undegraded":"2024-09-11T00:57:17.576498+0000","last_fullsized":"2024-09-11T00:57:17.576498+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:40.165368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265346+0000","last_change":"2024-09-11T00:56:53.127047+0000","last_active":"2024-09-11T00:58:16.265346+0000","last_peered":"2024-09-11T00:58:16.265346+0000","last_clean":"2024-09-11T00:58:16.265346+0000","last_became_active":"2024-09-11T00:56:53.126719+0000","last_became_peered":"2024-09-11T00:56:53.126719+0000","last_unstale":"2024-09-11T00:58:16.265346+0000","last_undegraded":"2024-09-11T00:58:16.265346+0000","last_fullsized":"2024-09-11T00:58:16.265346+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.686604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.583494+0000","last_change":"2024-09-11T00:56:51.101665+0000","last_active":"2024-09-11T00:57:17.583494+0000","last_peered":"2024-09-11T00:57:17.583494+0000","last_clean":"2024-09-11T00:57:17.583494+0000","last_became_active":"2024-09-11T00:56:51.101332+0000","last_became_peered":"2024-09-11T00:56:51.101332+0000","last_unstale":"2024-09-11T00:57:17.583494+0000","last_undegraded":"2024-09-11T00:57:17.583494+0000","last_fullsized":"2024-09-11T00:57:17.583494+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:47.639877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.267267+0000","last_change":"2024-09-11T00:56:53.128736+0000","last_active":"2024-09-11T00:58:16.267267+0000","last_peered":"2024-09-11T00:58:16.267267+0000","last_clean":"2024-09-11T00:58:16.267267+0000","last_became_active":"2024-09-11T00:56:53.128417+0000","last_became_peered":"2024-09-11T00:56:53.128417+0000","last_unstale":"2024-09-11T00:58:16.267267+0000","last_undegraded":"2024-09-11T00:58:16.267267+0000","last_fullsized":"2024-09-11T00:58:16.267267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:52.331343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'6","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:49.383863+0000","last_change":"2024-09-11T00:56:51.154733+0000","last_active":"2024-09-11T00:58:49.383863+0000","last_peered":"2024-09-11T00:58:49.383863+0000","last_clean":"2024-09-11T00:58:49.383863+0000","last_became_active":"2024-09-11T00:56:51.153613+0000","last_became_peered":"2024-09-11T00:56:51.153613+0000","last_unstale":"2024-09-11T00:58:49.383863+0000","last_undegraded":"2024-09-11T00:58:49.383863+0000","last_fullsized":"2024-09-11T00:58:49.383863+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:24.749186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1478,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.260212+0000","last_change":"2024-09-11T00:56:53.124684+0000","last_active":"2024-09-11T00:58:16.260212+0000","last_peered":"2024-09-11T00:58:16.260212+0000","last_clean":"2024-09-11T00:58:16.260212+0000","last_became_active":"2024-09-11T00:56:53.124147+0000","last_became_peered":"2024-09-11T00:56:53.124147+0000","last_unstale":"2024-09-11T00:58:16.260212+0000","last_undegraded":"2024-09-11T00:58:16.260212+0000","last_fullsized":"2024-09-11T00:58:16.260212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:14.280465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586515+0000","last_change":"2024-09-11T00:55:34.500373+0000","last_active":"2024-09-11T00:57:17.586515+0000","last_peered":"2024-09-11T00:57:17.586515+0000","last_clean":"2024-09-11T00:57:17.586515+0000","last_became_active":"2024-09-11T00:55:34.495540+0000","last_became_peered":"2024-09-11T00:55:34.495540+0000","last_unstale":"2024-09-11T00:57:17.586515+0000","last_undegraded":"2024-09-11T00:57:17.586515+0000","last_fullsized":"2024-09-11T00:57:17.586515+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:52:11.168966+0000","last_clean_scrub_stamp":"2024-09-11T00:52:11.168966+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:37.681466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577058+0000","last_change":"2024-09-11T00:56:51.101524+0000","last_active":"2024-09-11T00:57:17.577058+0000","last_peered":"2024-09-11T00:57:17.577058+0000","last_clean":"2024-09-11T00:57:17.577058+0000","last_became_active":"2024-09-11T00:56:51.101180+0000","last_became_peered":"2024-09-11T00:56:51.101180+0000","last_unstale":"2024-09-11T00:57:17.577058+0000","last_undegraded":"2024-09-11T00:57:17.577058+0000","last_fullsized":"2024-09-11T00:57:17.577058+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:27.293494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.259285+0000","last_change":"2024-09-11T00:56:53.118503+0000","last_active":"2024-09-11T00:58:16.259285+0000","last_peered":"2024-09-11T00:58:16.259285+0000","last_clean":"2024-09-11T00:58:16.259285+0000","last_became_active":"2024-09-11T00:56:53.118174+0000","last_became_peered":"2024-09-11T00:56:53.118174+0000","last_unstale":"2024-09-11T00:58:16.259285+0000","last_undegraded":"2024-09-11T00:58:16.259285+0000","last_fullsized":"2024-09-11T00:58:16.259285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:13.524153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.586850+0000","last_change":"2024-09-11T00:56:51.104222+0000","last_active":"2024-09-11T00:57:17.586850+0000","last_peered":"2024-09-11T00:57:17.586850+0000","last_clean":"2024-09-11T00:57:17.586850+0000","last_became_active":"2024-09-11T00:56:51.103913+0000","last_became_peered":"2024-09-11T00:56:51.103913+0000","last_unstale":"2024-09-11T00:57:17.586850+0000","last_undegraded":"2024-09-11T00:57:17.586850+0000","last_fullsized":"2024-09-11T00:57:17.586850+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:57.467659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.261731+0000","last_change":"2024-09-11T00:56:53.115130+0000","last_active":"2024-09-11T00:58:16.261731+0000","last_peered":"2024-09-11T00:58:16.261731+0000","last_clean":"2024-09-11T00:58:16.261731+0000","last_became_active":"2024-09-11T00:56:53.114813+0000","last_became_peered":"2024-09-11T00:56:53.114813+0000","last_unstale":"2024-09-11T00:58:16.261731+0000","last_undegraded":"2024-09-11T00:58:16.261731+0000","last_fullsized":"2024-09-11T00:58:16.261731+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:13.265173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":139,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:51.426261+0000","last_change":"2024-09-11T00:56:51.152892+0000","last_active":"2024-09-11T00:58:51.426261+0000","last_peered":"2024-09-11T00:58:51.426261+0000","last_clean":"2024-09-11T00:58:51.426261+0000","last_became_active":"2024-09-11T00:56:51.152572+0000","last_became_peered":"2024-09-11T00:56:51.152572+0000","last_unstale":"2024-09-11T00:58:51.426261+0000","last_undegraded":"2024-09-11T00:58:51.426261+0000","last_fullsized":"2024-09-11T00:58:51.426261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:03.774593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.267390+0000","last_change":"2024-09-11T00:56:53.117416+0000","last_active":"2024-09-11T00:58:16.267390+0000","last_peered":"2024-09-11T00:58:16.267390+0000","last_clean":"2024-09-11T00:58:16.267390+0000","last_became_active":"2024-09-11T00:56:53.117016+0000","last_became_peered":"2024-09-11T00:56:53.117016+0000","last_unstale":"2024-09-11T00:58:16.267390+0000","last_undegraded":"2024-09-11T00:58:16.267390+0000","last_fullsized":"2024-09-11T00:58:16.267390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:07.607993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:51.384317+0000","last_change":"2024-09-11T00:56:51.102879+0000","last_active":"2024-09-11T00:58:51.384317+0000","last_peered":"2024-09-11T00:58:51.384317+0000","last_clean":"2024-09-11T00:58:51.384317+0000","last_became_active":"2024-09-11T00:56:51.102498+0000","last_became_peered":"2024-09-11T00:56:51.102498+0000","last_unstale":"2024-09-11T00:58:51.384317+0000","last_undegraded":"2024-09-11T00:58:51.384317+0000","last_fullsized":"2024-09-11T00:58:51.384317+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:41.360908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265698+0000","last_change":"2024-09-11T00:57:17.581261+0000","last_active":"2024-09-11T00:58:16.265698+0000","last_peered":"2024-09-11T00:58:16.265698+0000","last_clean":"2024-09-11T00:58:16.265698+0000","last_became_active":"2024-09-11T00:57:17.580941+0000","last_became_peered":"2024-09-11T00:57:17.580941+0000","last_unstale":"2024-09-11T00:58:16.265698+0000","last_undegraded":"2024-09-11T00:58:16.265698+0000","last_fullsized":"2024-09-11T00:58:16.265698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:58.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.584416+0000","last_change":"2024-09-11T00:57:17.584311+0000","last_active":"2024-09-11T00:57:17.584416+0000","last_peered":"2024-09-11T00:57:17.584416+0000","last_clean":"2024-09-11T00:57:17.584416+0000","last_became_active":"2024-09-11T00:57:17.583964+0000","last_became_peered":"2024-09-11T00:57:17.583964+0000","last_unstale":"2024-09-11T00:57:17.584416+0000","last_undegraded":"2024-09-11T00:57:17.584416+0000","last_fullsized":"2024-09-11T00:57:17.584416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:15.515947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.254129+0000","last_change":"2024-09-11T00:56:53.112550+0000","last_active":"2024-09-11T00:58:16.254129+0000","last_peered":"2024-09-11T00:58:16.254129+0000","last_clean":"2024-09-11T00:58:16.254129+0000","last_became_active":"2024-09-11T00:56:53.112179+0000","last_became_peered":"2024-09-11T00:56:53.112179+0000","last_unstale":"2024-09-11T00:58:16.254129+0000","last_undegraded":"2024-09-11T00:58:16.254129+0000","last_fullsized":"2024-09-11T00:58:16.254129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:03.339839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581699+0000","last_change":"2024-09-11T00:56:51.152957+0000","last_active":"2024-09-11T00:57:17.581699+0000","last_peered":"2024-09-11T00:57:17.581699+0000","last_clean":"2024-09-11T00:57:17.581699+0000","last_became_active":"2024-09-11T00:56:51.152616+0000","last_became_peered":"2024-09-11T00:56:51.152616+0000","last_unstale":"2024-09-11T00:57:17.581699+0000","last_undegraded":"2024-09-11T00:57:17.581699+0000","last_fullsized":"2024-09-11T00:57:17.581699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:02.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.262966+0000","last_change":"2024-09-11T00:56:53.115155+0000","last_active":"2024-09-11T00:58:16.262966+0000","last_peered":"2024-09-11T00:58:16.262966+0000","last_clean":"2024-09-11T00:58:16.262966+0000","last_became_active":"2024-09-11T00:56:53.114865+0000","last_became_peered":"2024-09-11T00:56:53.114865+0000","last_unstale":"2024-09-11T00:58:16.262966+0000","last_undegraded":"2024-09-11T00:58:16.262966+0000","last_fullsized":"2024-09-11T00:58:16.262966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:23.968888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'17","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:19.123128+0000","last_change":"2024-09-11T00:56:51.100887+0000","last_active":"2024-09-11T00:57:19.123128+0000","last_peered":"2024-09-11T00:57:19.123128+0000","last_clean":"2024-09-11T00:57:19.123128+0000","last_became_active":"2024-09-11T00:56:51.100474+0000","last_became_peered":"2024-09-11T00:56:51.100474+0000","last_unstale":"2024-09-11T00:57:19.123128+0000","last_undegraded":"2024-09-11T00:57:19.123128+0000","last_fullsized":"2024-09-11T00:57:19.123128+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:40.884871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.257501+0000","last_change":"2024-09-11T00:57:17.588086+0000","last_active":"2024-09-11T00:58:16.257501+0000","last_peered":"2024-09-11T00:58:16.257501+0000","last_clean":"2024-09-11T00:58:16.257501+0000","last_became_active":"2024-09-11T00:57:17.587785+0000","last_became_peered":"2024-09-11T00:57:17.587785+0000","last_unstale":"2024-09-11T00:58:16.257501+0000","last_undegraded":"2024-09-11T00:58:16.257501+0000","last_fullsized":"2024-09-11T00:58:16.257501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:49.494588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.577254+0000","last_change":"2024-09-11T00:56:51.102031+0000","last_active":"2024-09-11T00:57:17.577254+0000","last_peered":"2024-09-11T00:57:17.577254+0000","last_clean":"2024-09-11T00:57:17.577254+0000","last_became_active":"2024-09-11T00:56:51.100614+0000","last_became_peered":"2024-09-11T00:56:51.100614+0000","last_unstale":"2024-09-11T00:57:17.577254+0000","last_undegraded":"2024-09-11T00:57:17.577254+0000","last_fullsized":"2024-09-11T00:57:17.577254+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:52.611261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265773+0000","last_change":"2024-09-11T00:56:53.124806+0000","last_active":"2024-09-11T00:58:16.265773+0000","last_peered":"2024-09-11T00:58:16.265773+0000","last_clean":"2024-09-11T00:58:16.265773+0000","last_became_active":"2024-09-11T00:56:53.124452+0000","last_became_peered":"2024-09-11T00:56:53.124452+0000","last_unstale":"2024-09-11T00:58:16.265773+0000","last_undegraded":"2024-09-11T00:58:16.265773+0000","last_fullsized":"2024-09-11T00:58:16.265773+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:57.934808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:18.160509+0000","last_change":"2024-09-11T00:56:51.153621+0000","last_active":"2024-09-11T00:57:18.160509+0000","last_peered":"2024-09-11T00:57:18.160509+0000","last_clean":"2024-09-11T00:57:18.160509+0000","last_became_active":"2024-09-11T00:56:51.153315+0000","last_became_peered":"2024-09-11T00:56:51.153315+0000","last_unstale":"2024-09-11T00:57:18.160509+0000","last_undegraded":"2024-09-11T00:57:18.160509+0000","last_fullsized":"2024-09-11T00:57:18.160509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:10.058295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.226023+0000","last_change":"2024-09-11T00:56:53.114463+0000","last_active":"2024-09-11T00:58:16.226023+0000","last_peered":"2024-09-11T00:58:16.226023+0000","last_clean":"2024-09-11T00:58:16.226023+0000","last_became_active":"2024-09-11T00:56:53.114099+0000","last_became_peered":"2024-09-11T00:56:53.114099+0000","last_unstale":"2024-09-11T00:58:16.226023+0000","last_undegraded":"2024-09-11T00:58:16.226023+0000","last_fullsized":"2024-09-11T00:58:16.226023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:58.981787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771377+0000","last_change":"2024-09-11T00:56:51.096942+0000","last_active":"2024-09-11T00:57:17.771377+0000","last_peered":"2024-09-11T00:57:17.771377+0000","last_clean":"2024-09-11T00:57:17.771377+0000","last_became_active":"2024-09-11T00:56:51.096584+0000","last_became_peered":"2024-09-11T00:56:51.096584+0000","last_unstale":"2024-09-11T00:57:17.771377+0000","last_undegraded":"2024-09-11T00:57:17.771377+0000","last_fullsized":"2024-09-11T00:57:17.771377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:03.484123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.265844+0000","last_change":"2024-09-11T00:56:53.126804+0000","last_active":"2024-09-11T00:58:16.265844+0000","last_peered":"2024-09-11T00:58:16.265844+0000","last_clean":"2024-09-11T00:58:16.265844+0000","last_became_active":"2024-09-11T00:56:53.125408+0000","last_became_peered":"2024-09-11T00:56:53.125408+0000","last_unstale":"2024-09-11T00:58:16.265844+0000","last_undegraded":"2024-09-11T00:58:16.265844+0000","last_fullsized":"2024-09-11T00:58:16.265844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:46.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580130+0000","last_change":"2024-09-11T00:56:51.103755+0000","last_active":"2024-09-11T00:57:17.580130+0000","last_peered":"2024-09-11T00:57:17.580130+0000","last_clean":"2024-09-11T00:57:17.580130+0000","last_became_active":"2024-09-11T00:56:51.103395+0000","last_became_peered":"2024-09-11T00:56:51.103395+0000","last_unstale":"2024-09-11T00:57:17.580130+0000","last_undegraded":"2024-09-11T00:57:17.580130+0000","last_fullsized":"2024-09-11T00:57:17.580130+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:16.782660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.257207+0000","last_change":"2024-09-11T00:56:53.121926+0000","last_active":"2024-09-11T00:58:16.257207+0000","last_peered":"2024-09-11T00:58:16.257207+0000","last_clean":"2024-09-11T00:58:16.257207+0000","last_became_active":"2024-09-11T00:56:53.121603+0000","last_became_peered":"2024-09-11T00:56:53.121603+0000","last_unstale":"2024-09-11T00:58:16.257207+0000","last_undegraded":"2024-09-11T00:58:16.257207+0000","last_fullsized":"2024-09-11T00:58:16.257207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:31.021030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.580437+0000","last_change":"2024-09-11T00:56:51.107221+0000","last_active":"2024-09-11T00:57:17.580437+0000","last_peered":"2024-09-11T00:57:17.580437+0000","last_clean":"2024-09-11T00:57:17.580437+0000","last_became_active":"2024-09-11T00:56:51.106849+0000","last_became_peered":"2024-09-11T00:56:51.106849+0000","last_unstale":"2024-09-11T00:57:17.580437+0000","last_undegraded":"2024-09-11T00:57:17.580437+0000","last_fullsized":"2024-09-11T00:57:17.580437+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:46.497230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.269124+0000","last_change":"2024-09-11T00:56:53.120525+0000","last_active":"2024-09-11T00:58:16.269124+0000","last_peered":"2024-09-11T00:58:16.269124+0000","last_clean":"2024-09-11T00:58:16.269124+0000","last_became_active":"2024-09-11T00:56:53.120228+0000","last_became_peered":"2024-09-11T00:56:53.120228+0000","last_unstale":"2024-09-11T00:58:16.269124+0000","last_undegraded":"2024-09-11T00:58:16.269124+0000","last_fullsized":"2024-09-11T00:58:16.269124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:52.047687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.266504+0000","last_change":"2024-09-11T00:56:53.120311+0000","last_active":"2024-09-11T00:58:16.266504+0000","last_peered":"2024-09-11T00:58:16.266504+0000","last_clean":"2024-09-11T00:58:16.266504+0000","last_became_active":"2024-09-11T00:56:53.119648+0000","last_became_peered":"2024-09-11T00:56:53.119648+0000","last_unstale":"2024-09-11T00:58:16.266504+0000","last_undegraded":"2024-09-11T00:58:16.266504+0000","last_fullsized":"2024-09-11T00:58:16.266504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:21.187784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581118+0000","last_change":"2024-09-11T00:56:51.103631+0000","last_active":"2024-09-11T00:57:17.581118+0000","last_peered":"2024-09-11T00:57:17.581118+0000","last_clean":"2024-09-11T00:57:17.581118+0000","last_became_active":"2024-09-11T00:56:51.103281+0000","last_became_peered":"2024-09-11T00:56:51.103281+0000","last_unstale":"2024-09-11T00:57:17.581118+0000","last_undegraded":"2024-09-11T00:57:17.581118+0000","last_fullsized":"2024-09-11T00:57:17.581118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:16.910956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.274004+0000","last_change":"2024-09-11T00:56:53.124752+0000","last_active":"2024-09-11T00:58:16.274004+0000","last_peered":"2024-09-11T00:58:16.274004+0000","last_clean":"2024-09-11T00:58:16.274004+0000","last_became_active":"2024-09-11T00:56:53.124245+0000","last_became_peered":"2024-09-11T00:56:53.124245+0000","last_unstale":"2024-09-11T00:58:16.274004+0000","last_undegraded":"2024-09-11T00:58:16.274004+0000","last_fullsized":"2024-09-11T00:58:16.274004+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:10.277903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.581704+0000","last_change":"2024-09-11T00:56:51.104624+0000","last_active":"2024-09-11T00:57:17.581704+0000","last_peered":"2024-09-11T00:57:17.581704+0000","last_clean":"2024-09-11T00:57:17.581704+0000","last_became_active":"2024-09-11T00:56:51.104041+0000","last_became_peered":"2024-09-11T00:56:51.104041+0000","last_unstale":"2024-09-11T00:57:17.581704+0000","last_undegraded":"2024-09-11T00:57:17.581704+0000","last_fullsized":"2024-09-11T00:57:17.581704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:10.397455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.272637+0000","last_change":"2024-09-11T00:56:53.118547+0000","last_active":"2024-09-11T00:58:16.272637+0000","last_peered":"2024-09-11T00:58:16.272637+0000","last_clean":"2024-09-11T00:58:16.272637+0000","last_became_active":"2024-09-11T00:56:53.118291+0000","last_became_peered":"2024-09-11T00:56:53.118291+0000","last_unstale":"2024-09-11T00:58:16.272637+0000","last_undegraded":"2024-09-11T00:58:16.272637+0000","last_fullsized":"2024-09-11T00:58:16.272637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:42.012173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:57:17.771774+0000","last_change":"2024-09-11T00:56:51.153047+0000","last_active":"2024-09-11T00:57:17.771774+0000","last_peered":"2024-09-11T00:57:17.771774+0000","last_clean":"2024-09-11T00:57:17.771774+0000","last_became_active":"2024-09-11T00:56:51.152564+0000","last_became_peered":"2024-09-11T00:56:51.152564+0000","last_unstale":"2024-09-11T00:57:17.771774+0000","last_undegraded":"2024-09-11T00:57:17.771774+0000","last_fullsized":"2024-09-11T00:57:17.771774+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:16.849087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:16.269357+0000","last_change":"2024-09-11T00:56:53.119926+0000","last_active":"2024-09-11T00:58:16.269357+0000","last_peered":"2024-09-11T00:58:16.269357+0000","last_clean":"2024-09-11T00:58:16.269357+0000","last_became_active":"2024-09-11T00:56:53.119643+0000","last_became_peered":"2024-09-11T00:56:53.119643+0000","last_unstale":"2024-09-11T00:58:16.269357+0000","last_undegraded":"2024-09-11T00:58:16.269357+0000","last_fullsized":"2024-09-11T00:58:16.269357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:52.091196+0000","last_clean_scrub_stamp":"2024-09-11T00:56:52.091196+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:51.838958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T00:58:47.378123+0000","last_change":"2024-09-11T00:56:51.100837+0000","last_active":"2024-09-11T00:58:47.378123+0000","last_peered":"2024-09-11T00:58:47.378123+0000","last_clean":"2024-09-11T00:58:47.378123+0000","last_became_active":"2024-09-11T00:56:51.100494+0000","last_became_peered":"2024-09-11T00:56:51.100494+0000","last_unstale":"2024-09-11T00:58:47.378123+0000","last_undegraded":"2024-09-11T00:58:47.378123+0000","last_fullsized":"2024-09-11T00:58:47.378123+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:56:50.079321+0000","last_clean_scrub_stamp":"2024-09-11T00:56:50.079321+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:39.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68166,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1111,"num_read_kb":629,"num_write":254,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1125,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":204498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286865,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":416921,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906986,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1068434,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":416431,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266741,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":416697,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462971,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":1009468,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626495,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":416148,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822724,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1009146,"data_compressed":42304,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986250,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":417163,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":415973,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378708,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":477871,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542233,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":913408,"data_stored":477657,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":417795,"data_compressed":5829,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34830,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36475,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:58:54.130 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:58:54.415 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:54 smithi110 ceph-mon[34712]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:54 smithi110 ceph-mon[34712]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:54 smithi110 ceph-mon[34712]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-11T00:58:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:54 smithi110 ceph-mon[34712]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:54.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:54 smithi112 ceph-mon[35384]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:54 smithi112 ceph-mon[35384]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:54 smithi112 ceph-mon[35384]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-11T00:58:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:54 smithi112 ceph-mon[35384]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:54.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:54 smithi078 ceph-mon[31315]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:54.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:54 smithi078 ceph-mon[31315]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:54.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:54 smithi078 ceph-mon[31315]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-11T00:58:54.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:54 smithi078 ceph-mon[31315]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:58:54.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.875+0000 7f9576524640 1 -- 172.21.15.78:0/274595728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f957010d250 msgr2=0x7f957010f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:54.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.875+0000 7f9576524640 1 --2- 172.21.15.78:0/274595728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f957010d250 0x7f957010f640 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9568009ca0 tx=0x7f956802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:54.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.875+0000 7f9576524640 1 -- 172.21.15.78:0/274595728 shutdown_connections 2024-09-11T00:58:54.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.875+0000 7f9576524640 1 --2- 172.21.15.78:0/274595728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f957010d250 0x7f957010f640 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:54.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.875+0000 7f9576524640 1 --2- 172.21.15.78:0/274595728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f957010a920 0x7f957010cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:54.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.875+0000 7f9576524640 1 --2- 172.21.15.78:0/274595728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9570101810 0x7f9570101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:54.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.875+0000 7f9576524640 1 -- 172.21.15.78:0/274595728 >> 172.21.15.78:0/274595728 conn(0x7f9570076f50 msgr2=0x7f9570077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:54.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.875+0000 7f9576524640 1 -- 172.21.15.78:0/274595728 shutdown_connections 2024-09-11T00:58:54.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.875+0000 7f9576524640 1 -- 172.21.15.78:0/274595728 wait complete. 2024-09-11T00:58:54.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.876+0000 7f9576524640 1 Processor -- start 2024-09-11T00:58:54.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.876+0000 7f9576524640 1 -- start start 2024-09-11T00:58:54.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.876+0000 7f9576524640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9570101810 0x7f9570199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:54.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.876+0000 7f9576524640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957010a920 0x7f9570199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:54.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.876+0000 7f9576524640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f957010d250 0x7f957019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:54.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.876+0000 7f9576524640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9570112e30 con 0x7f957010d250 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.876+0000 7f9576524640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9570112cb0 con 0x7f957010a920 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.876+0000 7f9576524640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9570112fb0 con 0x7f9570101810 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.876+0000 7f9574ac1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f957010d250 0x7f957019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f9574ac1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f957010d250 0x7f957019ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42748/0 (socket says 172.21.15.78:42748) 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f9574ac1640 1 -- 172.21.15.78:0/37815179 learned_addr learned my addr 172.21.15.78:0/37815179 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f956f7fe640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957010a920 0x7f9570199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f956ffff640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9570101810 0x7f9570199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f9574ac1640 1 -- 172.21.15.78:0/37815179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9570101810 msgr2=0x7f9570199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f9574ac1640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9570101810 0x7f9570199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f9574ac1640 1 -- 172.21.15.78:0/37815179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957010a920 msgr2=0x7f9570199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f9574ac1640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957010a920 0x7f9570199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f9574ac1640 1 -- 172.21.15.78:0/37815179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957019f3f0 con 0x7f957010d250 2024-09-11T00:58:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f9574ac1640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f957010d250 0x7f957019ec20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9568005cc0 tx=0x7f956802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f956d7fa640 1 -- 172.21.15.78:0/37815179 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568003cf0 con 0x7f957010d250 2024-09-11T00:58:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f956d7fa640 1 -- 172.21.15.78:0/37815179 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95680044f0 con 0x7f957010d250 2024-09-11T00:58:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f956d7fa640 1 -- 172.21.15.78:0/37815179 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956802fc30 con 0x7f957010d250 2024-09-11T00:58:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f9576524640 1 -- 172.21.15.78:0/37815179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957019f680 con 0x7f957010d250 2024-09-11T00:58:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.877+0000 7f9576524640 1 -- 172.21.15.78:0/37815179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95701a5e20 con 0x7f957010d250 2024-09-11T00:58:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.878+0000 7f9576524640 1 -- 172.21.15.78:0/37815179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9570101c70 con 0x7f957010d250 2024-09-11T00:58:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.880+0000 7f956d7fa640 1 -- 172.21.15.78:0/37815179 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f9568004040 con 0x7f957010d250 2024-09-11T00:58:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.880+0000 7f956d7fa640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f95580b5330 0x7f95580b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.880+0000 7f956ffff640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f95580b5330 0x7f95580b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.880+0000 7f956d7fa640 1 -- 172.21.15.78:0/37815179 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f95681071d0 con 0x7f957010d250 2024-09-11T00:58:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.881+0000 7f956ffff640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f95580b5330 0x7f95580b77f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9560009890 tx=0x7f9560008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:54.884+0000 7f956d7fa640 1 -- 172.21.15.78:0/37815179 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f95680cdfe0 con 0x7f957010d250 2024-09-11T00:58:55.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.094+0000 7f9576524640 1 -- 172.21.15.78:0/37815179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95701a60c0 con 0x7f957010d250 2024-09-11T00:58:55.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.095+0000 7f956d7fa640 1 -- 172.21.15.78:0/37815179 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11979 (secure 0 0 0) 0x7f95680cd8e0 con 0x7f957010d250 2024-09-11T00:58:55.096 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:55.097 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"btime":"2024-09-11T00:58:50:118172+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T00:58:49.633286+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T00:58:55.097 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-09-11T00:58:55.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.097+0000 7f9576524640 1 -- 172.21.15.78:0/37815179 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f95580b5330 msgr2=0x7f95580b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:55.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.097+0000 7f9576524640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f95580b5330 0x7f95580b77f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9560009890 tx=0x7f9560008040 comp rx=0 tx=0).stop 2024-09-11T00:58:55.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.097+0000 7f9576524640 1 -- 172.21.15.78:0/37815179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f957010d250 msgr2=0x7f957019ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:55.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.097+0000 7f9576524640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f957010d250 0x7f957019ec20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9568005cc0 tx=0x7f956802f470 comp rx=0 tx=0).stop 2024-09-11T00:58:55.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.097+0000 7f9576524640 1 -- 172.21.15.78:0/37815179 shutdown_connections 2024-09-11T00:58:55.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.097+0000 7f9576524640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f95580b5330 0x7f95580b77f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:55.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.097+0000 7f9576524640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f957010d250 0x7f957019ec20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:55.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.097+0000 7f9576524640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957010a920 0x7f9570199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:55.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.097+0000 7f9576524640 1 --2- 172.21.15.78:0/37815179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9570101810 0x7f9570199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:55.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.097+0000 7f9576524640 1 -- 172.21.15.78:0/37815179 >> 172.21.15.78:0/37815179 conn(0x7f9570076f50 msgr2=0x7f95701005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:55.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.098+0000 7f9576524640 1 -- 172.21.15.78:0/37815179 shutdown_connections 2024-09-11T00:58:55.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:55.098+0000 7f9576524640 1 -- 172.21.15.78:0/37815179 wait complete. 2024-09-11T00:58:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:55 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/37815179' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:55.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:55 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/37815179' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:55.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:55 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/37815179' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:58:55.851 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:58:56.099 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:56.402 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:56 smithi078 ceph-mon[31315]: pgmap v325: 97 pgs: 97 active+clean; 649 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:58:56.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:56 smithi110 ceph-mon[34712]: pgmap v325: 97 pgs: 97 active+clean; 649 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:58:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:56 smithi112 ceph-mon[35384]: pgmap v325: 97 pgs: 97 active+clean; 649 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:58:56.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.627+0000 7f759728c640 1 -- 172.21.15.78:0/3908984035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f759010d0e0 msgr2=0x7f759010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:56.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.627+0000 7f759728c640 1 --2- 172.21.15.78:0/3908984035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f759010d0e0 0x7f759010f550 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7580009ca0 tx=0x7f758002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.627+0000 7f759728c640 1 -- 172.21.15.78:0/3908984035 shutdown_connections 2024-09-11T00:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.627+0000 7f759728c640 1 --2- 172.21.15.78:0/3908984035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f759010d0e0 0x7f759010f550 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.627+0000 7f759728c640 1 --2- 172.21.15.78:0/3908984035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590104530 0x7f759010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.627+0000 7f759728c640 1 --2- 172.21.15.78:0/3908984035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7590103b30 0x7f7590103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.627+0000 7f759728c640 1 -- 172.21.15.78:0/3908984035 >> 172.21.15.78:0/3908984035 conn(0x7f75900fd700 msgr2=0x7f75900ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.627+0000 7f759728c640 1 -- 172.21.15.78:0/3908984035 shutdown_connections 2024-09-11T00:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.627+0000 7f759728c640 1 -- 172.21.15.78:0/3908984035 wait complete. 2024-09-11T00:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f759728c640 1 Processor -- start 2024-09-11T00:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f759728c640 1 -- start start 2024-09-11T00:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f759728c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7590103b30 0x7f75901019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f759728c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7590104530 0x7f7590101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f759728c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f759010d0e0 0x7f7590102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f759728c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7590112d60 con 0x7f7590103b30 2024-09-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f759728c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7590112be0 con 0x7f7590104530 2024-09-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f759728c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7590112ee0 con 0x7f759010d0e0 2024-09-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f7595028640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7590103b30 0x7f75901019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f7595028640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7590103b30 0x7f75901019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42764/0 (socket says 172.21.15.78:42764) 2024-09-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f7595028640 1 -- 172.21.15.78:0/2478522925 learned_addr learned my addr 172.21.15.78:0/2478522925 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f7594827640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7590104530 0x7f7590101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f7595829640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f759010d0e0 0x7f7590102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f7595028640 1 -- 172.21.15.78:0/2478522925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f759010d0e0 msgr2=0x7f7590102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f7595028640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f759010d0e0 0x7f7590102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f7595028640 1 -- 172.21.15.78:0/2478522925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7590104530 msgr2=0x7f7590101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f7595028640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7590104530 0x7f7590101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f7595028640 1 -- 172.21.15.78:0/2478522925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75901a7610 con 0x7f7590103b30 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f7595028640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7590103b30 0x7f75901019b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f758800c890 tx=0x7f758800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.628+0000 7f758e7fc640 1 -- 172.21.15.78:0/2478522925 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588004220 con 0x7f7590103b30 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.629+0000 7f758e7fc640 1 -- 172.21.15.78:0/2478522925 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75880043c0 con 0x7f7590103b30 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.629+0000 7f759728c640 1 -- 172.21.15.78:0/2478522925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75901a7900 con 0x7f7590103b30 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.629+0000 7f758e7fc640 1 -- 172.21.15.78:0/2478522925 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588004a00 con 0x7f7590103b30 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.629+0000 7f759728c640 1 -- 172.21.15.78:0/2478522925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75901a7e40 con 0x7f7590103b30 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.630+0000 7f759728c640 1 -- 172.21.15.78:0/2478522925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7590103f10 con 0x7f7590103b30 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.632+0000 7f758e7fc640 1 -- 172.21.15.78:0/2478522925 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f75880079b0 con 0x7f7590103b30 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.632+0000 7f758e7fc640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f75780b5330 0x7f75780b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.633+0000 7f7594827640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f75780b5330 0x7f75780b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.633+0000 7f758e7fc640 1 -- 172.21.15.78:0/2478522925 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f75880e5ee0 con 0x7f7590103b30 2024-09-11T00:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.633+0000 7f7594827640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f75780b5330 0x7f75780b77f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7584005fd0 tx=0x7f7584005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:56.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.636+0000 7f758e7fc640 1 -- 172.21.15.78:0/2478522925 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7588014030 con 0x7f7590103b30 2024-09-11T00:58:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.813+0000 7f759728c640 1 -- 172.21.15.78:0/2478522925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75901a82b0 con 0x7f7590103b30 2024-09-11T00:58:56.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.814+0000 7f758e7fc640 1 -- 172.21.15.78:0/2478522925 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22975 (secure 0 0 0) 0x7f75880accf0 con 0x7f7590103b30 2024-09-11T00:58:56.816 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:58:56.816 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","created":"2024-09-11T00:48:27.652512+0000","modified":"2024-09-11T00:57:17.572848+0000","last_up_change":"2024-09-11T00:55:59.071507+0000","last_in_change":"2024-09-11T00:55:35.815150+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:52:10.229887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:56:49.939136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:56:51.703446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beefc6d9-f74c-4200-96f2-ff697ad0d7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6803","nonce":1424639053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6805","nonce":1424639053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6809","nonce":1424639053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6807","nonce":1424639053}]},"public_addr":"172.21.15.78:6803/1424639053","cluster_addr":"172.21.15.78:6805/1424639053","heartbeat_back_addr":"172.21.15.78:6809/1424639053","heartbeat_front_addr":"172.21.15.78:6807/1424639053","state":["exists","up"]},{"osd":1,"uuid":"d742e767-d673-4bb9-b624-8bc969f5ceb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6801","nonce":580939385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6803","nonce":580939385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6807","nonce":580939385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6805","nonce":580939385}]},"public_addr":"172.21.15.110:6801/580939385","cluster_addr":"172.21.15.110:6803/580939385","heartbeat_back_addr":"172.21.15.110:6807/580939385","heartbeat_front_addr":"172.21.15.110:6805/580939385","state":["exists","up"]},{"osd":2,"uuid":"b0106a5b-1465-4a50-96dd-934dbae49a4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6801","nonce":2983155763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6803","nonce":2983155763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6807","nonce":2983155763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6805","nonce":2983155763}]},"public_addr":"172.21.15.112:6801/2983155763","cluster_addr":"172.21.15.112:6803/2983155763","heartbeat_back_addr":"172.21.15.112:6807/2983155763","heartbeat_front_addr":"172.21.15.112:6805/2983155763","state":["exists","up"]},{"osd":3,"uuid":"3270362b-3d61-414a-a3d9-f2416ff8b870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6811","nonce":2906086464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6813","nonce":2906086464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6817","nonce":2906086464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6815","nonce":2906086464}]},"public_addr":"172.21.15.78:6811/2906086464","cluster_addr":"172.21.15.78:6813/2906086464","heartbeat_back_addr":"172.21.15.78:6817/2906086464","heartbeat_front_addr":"172.21.15.78:6815/2906086464","state":["exists","up"]},{"osd":4,"uuid":"e47746ec-9443-4004-abfa-17c1ed73b982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6809","nonce":477287435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6811","nonce":477287435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6815","nonce":477287435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6813","nonce":477287435}]},"public_addr":"172.21.15.110:6809/477287435","cluster_addr":"172.21.15.110:6811/477287435","heartbeat_back_addr":"172.21.15.110:6815/477287435","heartbeat_front_addr":"172.21.15.110:6813/477287435","state":["exists","up"]},{"osd":5,"uuid":"f8cedb01-310d-442d-a588-33197d32e208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6809","nonce":3557095096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6811","nonce":3557095096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6815","nonce":3557095096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6813","nonce":3557095096}]},"public_addr":"172.21.15.112:6809/3557095096","cluster_addr":"172.21.15.112:6811/3557095096","heartbeat_back_addr":"172.21.15.112:6815/3557095096","heartbeat_front_addr":"172.21.15.112:6813/3557095096","state":["exists","up"]},{"osd":6,"uuid":"d508a037-8f56-436b-b7bf-9a1c69f6d9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6819","nonce":1267910775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6821","nonce":1267910775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6825","nonce":1267910775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6823","nonce":1267910775}]},"public_addr":"172.21.15.78:6819/1267910775","cluster_addr":"172.21.15.78:6821/1267910775","heartbeat_back_addr":"172.21.15.78:6825/1267910775","heartbeat_front_addr":"172.21.15.78:6823/1267910775","state":["exists","up"]},{"osd":7,"uuid":"e8842047-09a6-48b8-8d0a-2d54e7f89063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6817","nonce":1556494594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6819","nonce":1556494594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6823","nonce":1556494594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6821","nonce":1556494594}]},"public_addr":"172.21.15.110:6817/1556494594","cluster_addr":"172.21.15.110:6819/1556494594","heartbeat_back_addr":"172.21.15.110:6823/1556494594","heartbeat_front_addr":"172.21.15.110:6821/1556494594","state":["exists","up"]},{"osd":8,"uuid":"a3367da8-375a-4a46-a952-867d4b72ae2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6817","nonce":2481321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6819","nonce":2481321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6823","nonce":2481321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6821","nonce":2481321}]},"public_addr":"172.21.15.112:6817/2481321","cluster_addr":"172.21.15.112:6819/2481321","heartbeat_back_addr":"172.21.15.112:6823/2481321","heartbeat_front_addr":"172.21.15.112:6821/2481321","state":["exists","up"]},{"osd":9,"uuid":"d1825764-cefc-495b-99da-8ba429df0937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6827","nonce":197048350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6829","nonce":197048350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6833","nonce":197048350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6831","nonce":197048350}]},"public_addr":"172.21.15.78:6827/197048350","cluster_addr":"172.21.15.78:6829/197048350","heartbeat_back_addr":"172.21.15.78:6833/197048350","heartbeat_front_addr":"172.21.15.78:6831/197048350","state":["exists","up"]},{"osd":10,"uuid":"e1cfff7a-8916-4845-a482-d79fa275da30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6825","nonce":2101123168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6827","nonce":2101123168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6831","nonce":2101123168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6829","nonce":2101123168}]},"public_addr":"172.21.15.110:6825/2101123168","cluster_addr":"172.21.15.110:6827/2101123168","heartbeat_back_addr":"172.21.15.110:6831/2101123168","heartbeat_front_addr":"172.21.15.110:6829/2101123168","state":["exists","up"]},{"osd":11,"uuid":"fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6825","nonce":1526310654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6827","nonce":1526310654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6831","nonce":1526310654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6829","nonce":1526310654}]},"public_addr":"172.21.15.112:6825/1526310654","cluster_addr":"172.21.15.112:6827/1526310654","heartbeat_back_addr":"172.21.15.112:6831/1526310654","heartbeat_front_addr":"172.21.15.112:6829/1526310654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:17.334199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:41.400872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:08.205144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:33.550527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:58.394116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:24.699607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:50.450193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:16.223363+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:40.947727+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:06.845451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:32.410035+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:57.744364+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/2390785703":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:0/1038951931":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/391908936":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2706514734":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/2357025243":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6800/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2795778473":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2747741101":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6801/1289343426":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/3558896007":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/438261739":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/1289343426":"2024-09-12T00:50:15.928079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:58:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.816+0000 7f759728c640 1 -- 172.21.15.78:0/2478522925 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f75780b5330 msgr2=0x7f75780b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.816+0000 7f759728c640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f75780b5330 0x7f75780b77f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7584005fd0 tx=0x7f7584005850 comp rx=0 tx=0).stop 2024-09-11T00:58:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.816+0000 7f759728c640 1 -- 172.21.15.78:0/2478522925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7590103b30 msgr2=0x7f75901019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.816+0000 7f759728c640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7590103b30 0x7f75901019b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f758800c890 tx=0x7f758800cd50 comp rx=0 tx=0).stop 2024-09-11T00:58:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.817+0000 7f759728c640 1 -- 172.21.15.78:0/2478522925 shutdown_connections 2024-09-11T00:58:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.817+0000 7f759728c640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f75780b5330 0x7f75780b77f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.817+0000 7f759728c640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f759010d0e0 0x7f7590102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.817+0000 7f759728c640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7590104530 0x7f7590101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.817+0000 7f759728c640 1 --2- 172.21.15.78:0/2478522925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7590103b30 0x7f75901019b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.817+0000 7f759728c640 1 -- 172.21.15.78:0/2478522925 >> 172.21.15.78:0/2478522925 conn(0x7f75900fd700 msgr2=0x7f75900fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.817+0000 7f759728c640 1 -- 172.21.15.78:0/2478522925 shutdown_connections 2024-09-11T00:58:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:56.817+0000 7f759728c640 1 -- 172.21.15.78:0/2478522925 wait complete. 2024-09-11T00:58:57.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:57 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2478522925' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:58:57.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:57 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2478522925' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:58:57.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:57 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2478522925' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:58:57.554 DEBUG:tasks.fwd_scrub:Yielding 2024-09-11T00:58:57.555 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T00:58:57.564 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T00:58:57.564 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T00:58:57.565 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-11T00:58:57.565 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-11T00:58:57.565 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T00:58:57.565 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T00:58:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.670+0000 7f8abc310640 1 -- 172.21.15.78:0/1970172546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ab41054b0 msgr2=0x7f8ab41058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.670+0000 7f8abc310640 1 --2- 172.21.15.78:0/1970172546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ab41054b0 0x7f8ab41058f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009900 tx=0x7f8aa402a410 comp rx=0 tx=0).stop 2024-09-11T00:58:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.670+0000 7f8abc310640 1 -- 172.21.15.78:0/1970172546 shutdown_connections 2024-09-11T00:58:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.670+0000 7f8abc310640 1 --2- 172.21.15.78:0/1970172546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ab4105e30 0x7f8ab410f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.670+0000 7f8abc310640 1 --2- 172.21.15.78:0/1970172546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ab41054b0 0x7f8ab41058f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.670+0000 7f8abc310640 1 --2- 172.21.15.78:0/1970172546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ab406aaa0 0x7f8ab406ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.670+0000 7f8abc310640 1 -- 172.21.15.78:0/1970172546 >> 172.21.15.78:0/1970172546 conn(0x7f8ab4077720 msgr2=0x7f8ab4077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.670+0000 7f8abc310640 1 -- 172.21.15.78:0/1970172546 shutdown_connections 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.670+0000 7f8abc310640 1 -- 172.21.15.78:0/1970172546 wait complete. 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.671+0000 7f8abc310640 1 Processor -- start 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.671+0000 7f8abc310640 1 -- start start 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.671+0000 7f8abc310640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ab406aaa0 0x7f8ab41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.671+0000 7f8abc310640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ab41054b0 0x7f8ab4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.671+0000 7f8abc310640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ab4105e30 0x7f8ab419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.671+0000 7f8abc310640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4112d80 con 0x7f8ab406aaa0 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.671+0000 7f8abc310640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4112c00 con 0x7f8ab4105e30 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.671+0000 7f8abc310640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4112f00 con 0x7f8ab41054b0 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.671+0000 7f8aba0ac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ab406aaa0 0x7f8ab41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.671+0000 7f8aba0ac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ab406aaa0 0x7f8ab41992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42776/0 (socket says 172.21.15.78:42776) 2024-09-11T00:58:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.671+0000 7f8aba0ac640 1 -- 172.21.15.78:0/190010173 learned_addr learned my addr 172.21.15.78:0/190010173 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.671+0000 7f8ab98ab640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ab41054b0 0x7f8ab4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.672+0000 7f8aba8ad640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ab4105e30 0x7f8ab419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.672+0000 7f8aba0ac640 1 -- 172.21.15.78:0/190010173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ab41054b0 msgr2=0x7f8ab4199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.672+0000 7f8aba0ac640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ab41054b0 0x7f8ab4199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.672+0000 7f8aba0ac640 1 -- 172.21.15.78:0/190010173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ab4105e30 msgr2=0x7f8ab419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.672+0000 7f8aba0ac640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ab4105e30 0x7f8ab419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.672+0000 7f8aba0ac640 1 -- 172.21.15.78:0/190010173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab419f360 con 0x7f8ab406aaa0 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.672+0000 7f8aba0ac640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ab406aaa0 0x7f8ab41992c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000d770 tx=0x7f8ab000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.672+0000 7f8aab7fe640 1 -- 172.21.15.78:0/190010173 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0017070 con 0x7f8ab406aaa0 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.672+0000 7f8aab7fe640 1 -- 172.21.15.78:0/190010173 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab00054d0 con 0x7f8ab406aaa0 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.672+0000 7f8aab7fe640 1 -- 172.21.15.78:0/190010173 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0004c30 con 0x7f8ab406aaa0 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.672+0000 7f8abc310640 1 -- 172.21.15.78:0/190010173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab419f650 con 0x7f8ab406aaa0 2024-09-11T00:58:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.672+0000 7f8abc310640 1 -- 172.21.15.78:0/190010173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab41a5d10 con 0x7f8ab406aaa0 2024-09-11T00:58:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.673+0000 7f8abc310640 1 -- 172.21.15.78:0/190010173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab406ae80 con 0x7f8ab406aaa0 2024-09-11T00:58:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.675+0000 7f8aab7fe640 1 -- 172.21.15.78:0/190010173 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f8ab0004e10 con 0x7f8ab406aaa0 2024-09-11T00:58:57.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.676+0000 7f8aab7fe640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8aa00b53b0 0x7f8aa00b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:57.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.676+0000 7f8ab98ab640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8aa00b53b0 0x7f8aa00b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:57.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.676+0000 7f8aab7fe640 1 -- 172.21.15.78:0/190010173 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f8ab00e5810 con 0x7f8ab406aaa0 2024-09-11T00:58:57.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.677+0000 7f8ab98ab640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8aa00b53b0 0x7f8aa00b7870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8aa402a930 tx=0x7f8aa4005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:57.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.681+0000 7f8aab7fe640 1 -- 172.21.15.78:0/190010173 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8ab00ac620 con 0x7f8ab406aaa0 2024-09-11T00:58:57.831 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:57.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:57.927+0000 7f8abc310640 1 -- 172.21.15.78:0/190010173 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f8ab4063590 con 0x7f8aa00b53b0 2024-09-11T00:58:58.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.089+0000 7f8aab7fe640 1 -- 172.21.15.78:0/190010173 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8ab4063590 con 0x7f8aa00b53b0 2024-09-11T00:58:58.091 INFO:teuthology.orchestra.run.smithi078.stdout:Schedule set for path / 2024-09-11T00:58:58.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.091+0000 7f8abc310640 1 -- 172.21.15.78:0/190010173 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8aa00b53b0 msgr2=0x7f8aa00b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.091+0000 7f8abc310640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8aa00b53b0 0x7f8aa00b7870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8aa402a930 tx=0x7f8aa4005fd0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.092+0000 7f8abc310640 1 -- 172.21.15.78:0/190010173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ab406aaa0 msgr2=0x7f8ab41992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.092+0000 7f8abc310640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ab406aaa0 0x7f8ab41992c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000d770 tx=0x7f8ab000dc30 comp rx=0 tx=0).stop 2024-09-11T00:58:58.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.092+0000 7f8abc310640 1 -- 172.21.15.78:0/190010173 shutdown_connections 2024-09-11T00:58:58.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.092+0000 7f8abc310640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8aa00b53b0 0x7f8aa00b7870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.092+0000 7f8abc310640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ab4105e30 0x7f8ab419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.092+0000 7f8abc310640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ab41054b0 0x7f8ab4199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.092+0000 7f8abc310640 1 --2- 172.21.15.78:0/190010173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ab406aaa0 0x7f8ab41992c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.092+0000 7f8abc310640 1 -- 172.21.15.78:0/190010173 >> 172.21.15.78:0/190010173 conn(0x7f8ab4077720 msgr2=0x7f8ab40698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:58.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.092+0000 7f8abc310640 1 -- 172.21.15.78:0/190010173 shutdown_connections 2024-09-11T00:58:58.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.092+0000 7f8abc310640 1 -- 172.21.15.78:0/190010173 wait complete. 2024-09-11T00:58:58.107 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-11T00:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.216+0000 7ff30f0dd640 1 -- 172.21.15.78:0/3567680700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3081043d0 msgr2=0x7ff308104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.216+0000 7ff30f0dd640 1 --2- 172.21.15.78:0/3567680700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3081043d0 0x7ff308104830 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009900 tx=0x7ff2f402a410 comp rx=0 tx=0).stop 2024-09-11T00:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.216+0000 7ff30f0dd640 1 -- 172.21.15.78:0/3567680700 shutdown_connections 2024-09-11T00:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.216+0000 7ff30f0dd640 1 --2- 172.21.15.78:0/3567680700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff308104d70 0x7ff30810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.216+0000 7ff30f0dd640 1 --2- 172.21.15.78:0/3567680700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3081043d0 0x7ff308104830 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.216+0000 7ff30f0dd640 1 --2- 172.21.15.78:0/3567680700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff308102010 0x7ff3081023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.216+0000 7ff30f0dd640 1 -- 172.21.15.78:0/3567680700 >> 172.21.15.78:0/3567680700 conn(0x7ff308077720 msgr2=0x7ff308077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.216+0000 7ff30f0dd640 1 -- 172.21.15.78:0/3567680700 shutdown_connections 2024-09-11T00:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.216+0000 7ff30f0dd640 1 -- 172.21.15.78:0/3567680700 wait complete. 2024-09-11T00:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.216+0000 7ff30f0dd640 1 Processor -- start 2024-09-11T00:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.216+0000 7ff30f0dd640 1 -- start start 2024-09-11T00:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30f0dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff308102010 0x7ff308199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30f0dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3081043d0 0x7ff308199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30f0dd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff308104d70 0x7ff30819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30f0dd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff308111c30 con 0x7ff308102010 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30f0dd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff308111ab0 con 0x7ff308104d70 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30f0dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff308111db0 con 0x7ff3081043d0 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30ce79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff308102010 0x7ff308199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30ce79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff308102010 0x7ff308199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42814/0 (socket says 172.21.15.78:42814) 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30ce79640 1 -- 172.21.15.78:0/1027403105 learned_addr learned my addr 172.21.15.78:0/1027403105 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30d67a640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff308104d70 0x7ff30819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff307fff640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3081043d0 0x7ff308199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30ce79640 1 -- 172.21.15.78:0/1027403105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3081043d0 msgr2=0x7ff308199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30ce79640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3081043d0 0x7ff308199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30ce79640 1 -- 172.21.15.78:0/1027403105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff308104d70 msgr2=0x7ff30819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30ce79640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff308104d70 0x7ff30819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30ce79640 1 -- 172.21.15.78:0/1027403105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30819f2c0 con 0x7ff308102010 2024-09-11T00:58:58.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30d67a640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff308104d70 0x7ff30819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:58:58.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff30ce79640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff308102010 0x7ff308199220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00e9f0 tx=0x7ff2fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:58.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.217+0000 7ff305ffb640 1 -- 172.21.15.78:0/1027403105 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00cd80 con 0x7ff308102010 2024-09-11T00:58:58.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.218+0000 7ff30f0dd640 1 -- 172.21.15.78:0/1027403105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30819f5b0 con 0x7ff308102010 2024-09-11T00:58:58.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.218+0000 7ff305ffb640 1 -- 172.21.15.78:0/1027403105 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2fc0108e0 con 0x7ff308102010 2024-09-11T00:58:58.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.218+0000 7ff305ffb640 1 -- 172.21.15.78:0/1027403105 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc018980 con 0x7ff308102010 2024-09-11T00:58:58.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.218+0000 7ff30f0dd640 1 -- 172.21.15.78:0/1027403105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3081a5e50 con 0x7ff308102010 2024-09-11T00:58:58.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.219+0000 7ff30f0dd640 1 -- 172.21.15.78:0/1027403105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff308102fc0 con 0x7ff308102010 2024-09-11T00:58:58.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.220+0000 7ff305ffb640 1 -- 172.21.15.78:0/1027403105 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7ff2fc010430 con 0x7ff308102010 2024-09-11T00:58:58.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.221+0000 7ff305ffb640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff2f80b53c0 0x7ff2f80b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.221+0000 7ff307fff640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff2f80b53c0 0x7ff2f80b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.221+0000 7ff305ffb640 1 -- 172.21.15.78:0/1027403105 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff2fc0e6660 con 0x7ff308102010 2024-09-11T00:58:58.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.222+0000 7ff307fff640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff2f80b53c0 0x7ff2f80b7880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4031440 tx=0x7ff2f4005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:58.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.224+0000 7ff305ffb640 1 -- 172.21.15.78:0/1027403105 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff2fc0acc60 con 0x7ff308102010 2024-09-11T00:58:58.333 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:58 smithi078 ceph-mon[31315]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:58.333 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:58 smithi078 ceph-mon[31315]: pgmap v326: 97 pgs: 97 active+clean; 649 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:58:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.324+0000 7f0bcb970640 1 -- 172.21.15.78:0/1383690125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc4101c90 msgr2=0x7f0bc4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.324+0000 7f0bcb970640 1 --2- 172.21.15.78:0/1383690125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc4101c90 0x7f0bc4102070 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009900 tx=0x7f0bc002a410 comp rx=0 tx=0).stop 2024-09-11T00:58:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.324+0000 7f0bcb970640 1 -- 172.21.15.78:0/1383690125 shutdown_connections 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.324+0000 7f0bcb970640 1 --2- 172.21.15.78:0/1383690125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bc4108020 0x7f0bc410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.324+0000 7f0bcb970640 1 --2- 172.21.15.78:0/1383690125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc41076a0 0x7f0bc4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.324+0000 7f0bcb970640 1 --2- 172.21.15.78:0/1383690125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc4101c90 0x7f0bc4102070 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.324+0000 7f0bcb970640 1 -- 172.21.15.78:0/1383690125 >> 172.21.15.78:0/1383690125 conn(0x7f0bc40fd700 msgr2=0x7f0bc40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.325+0000 7f0bcb970640 1 -- 172.21.15.78:0/1383690125 shutdown_connections 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.325+0000 7f0bcb970640 1 -- 172.21.15.78:0/1383690125 wait complete. 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.325+0000 7f0bcb970640 1 Processor -- start 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.325+0000 7f0bcb970640 1 -- start start 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.325+0000 7f0bcb970640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc4101c90 0x7f0bc4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.325+0000 7f0bcb970640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc41076a0 0x7f0bc4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.325+0000 7f0bcb970640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bc4108020 0x7f0bc419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.325+0000 7f0bcb970640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4111ca0 con 0x7f0bc41076a0 2024-09-11T00:58:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.325+0000 7f0bcb970640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4111b20 con 0x7f0bc4108020 2024-09-11T00:58:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.325+0000 7f0bcb970640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4111e20 con 0x7f0bc4101c90 2024-09-11T00:58:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.325+0000 7f0bc8f0b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc41076a0 0x7f0bc4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.325+0000 7f0bc8f0b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc41076a0 0x7f0bc4199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42832/0 (socket says 172.21.15.78:42832) 2024-09-11T00:58:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bc8f0b640 1 -- 172.21.15.78:0/922287421 learned_addr learned my addr 172.21.15.78:0/922287421 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bc970c640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc4101c90 0x7f0bc4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bc8f0b640 1 -- 172.21.15.78:0/922287421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc4101c90 msgr2=0x7f0bc4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bc8f0b640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc4101c90 0x7f0bc4199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bc8f0b640 1 -- 172.21.15.78:0/922287421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bc4108020 msgr2=0x7f0bc419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bc8f0b640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bc4108020 0x7f0bc419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bc8f0b640 1 -- 172.21.15.78:0/922287421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc419f2f0 con 0x7f0bc41076a0 2024-09-11T00:58:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bc970c640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc4101c90 0x7f0bc4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bc8f0b640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc41076a0 0x7f0bc4199780 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400d950 tx=0x7f0bb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bba7fc640 1 -- 172.21.15.78:0/922287421 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4017030 con 0x7f0bc41076a0 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bba7fc640 1 -- 172.21.15.78:0/922287421 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb4004020 con 0x7f0bc41076a0 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bcb970640 1 -- 172.21.15.78:0/922287421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc419f5e0 con 0x7f0bc41076a0 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bba7fc640 1 -- 172.21.15.78:0/922287421 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40047a0 con 0x7f0bc41076a0 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.326+0000 7f0bcb970640 1 -- 172.21.15.78:0/922287421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc41a5ca0 con 0x7f0bc41076a0 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.329+0000 7f0bba7fc640 1 -- 172.21.15.78:0/922287421 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f0bb4009a30 con 0x7f0bc41076a0 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.330+0000 7f0bcdd45740 1 Processor -- start 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.330+0000 7f0bcdd45740 1 -- start start 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.330+0000 7f0bba7fc640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0bb00b53c0 0x7f0bb00b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.330+0000 7f0bcdd45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 0x561836fba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.330+0000 7f0bcdd45740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 0x561836fc6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.330+0000 7f0bcdd45740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fc6790 0x561836fc8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.330+0000 7f0bcdd45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561836f30e60 con 0x561836fb9ee0 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.330+0000 7f0bcdd45740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561836f20290 con 0x561836fba800 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.330+0000 7f0bcdd45740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561836f240a0 con 0x561836fc6790 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.330+0000 7f0bc9f0d640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0bb00b53c0 0x7f0bb00b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.331+0000 7f0bba7fc640 1 -- 172.21.15.78:0/922287421 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0bb40e5ab0 con 0x7f0bc41076a0 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.331+0000 7f0bc970c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 0x561836fba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.331+0000 7f0bc970c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 0x561836fba2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42842/0 (socket says 172.21.15.78:42842) 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.331+0000 7f0bc970c640 1 -- 172.21.15.78:0/1235608498 learned_addr learned my addr 172.21.15.78:0/1235608498 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.331+0000 7f0bc8f0b640 1 --2- 172.21.15.78:0/1235608498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 0x561836fc6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.331+0000 7f0bc970c640 1 -- 172.21.15.78:0/1235608498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fc6790 msgr2=0x561836fc8b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.331+0000 7f0bc970c640 1 --2- 172.21.15.78:0/1235608498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fc6790 0x561836fc8b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.331+0000 7f0bc970c640 1 -- 172.21.15.78:0/1235608498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 msgr2=0x561836fc6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.331+0000 7f0bc970c640 1 --2- 172.21.15.78:0/1235608498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 0x561836fc6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.331+0000 7f0bc970c640 1 -- 172.21.15.78:0/1235608498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561836c7e930 con 0x561836fb9ee0 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.331+0000 7f0bc8f0b640 1 --2- 172.21.15.78:0/1235608498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 0x561836fc6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.331+0000 7f0bc9f0d640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0bb00b53c0 0x7f0bb00b7880 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc005fd0 tx=0x7f0bbc005f60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.332+0000 7f0bc970c640 1 --2- 172.21.15.78:0/1235608498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 0x561836fba2c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009900 tx=0x7f0bc0032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c231d1a7a0b2a89a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.332+0000 7f0baf7fe640 1 -- 172.21.15.78:0/1235608498 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0035030 con 0x561836fb9ee0 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.332+0000 7f0baf7fe640 1 -- 172.21.15.78:0/1235608498 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bc0031760 con 0x561836fb9ee0 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.332+0000 7f0baf7fe640 1 -- 172.21.15.78:0/1235608498 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0035030 con 0x561836fb9ee0 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.333+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1235608498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 msgr2=0x561836fba2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.333+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/1235608498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 0x561836fba2c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009900 tx=0x7f0bc0032e70 comp rx=0 tx=0).stop 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.333+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1235608498 shutdown_connections 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.333+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/1235608498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fc6790 0x561836fc8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.333+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/1235608498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 0x561836fc6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.333+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/1235608498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 0x561836fba2c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.333+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1235608498 >> 172.21.15.78:0/1235608498 conn(0x561836fb7d10 msgr2=0x561836fb8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.333+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1235608498 shutdown_connections 2024-09-11T00:58:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.333+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1235608498 wait complete. 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.333+0000 7f0bcdd45740 1 -- start start 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bcdd45740 1 Processor -- start 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bcdd45740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fb9ee0 0x561836fc3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bcdd45740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 0x561836fc3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bcdd45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 0x561836fc41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bcdd45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561836d81890 con 0x561836fc6790 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bcdd45740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561836ebad40 con 0x561836fba800 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bcdd45740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561836fcd7f0 con 0x561836fb9ee0 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bcdd45740 10 client.0 _ll_register_callbacks cb 0x561836f1ec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bc9f0d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 0x561836fc41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bc8f0b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 0x561836fc3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bc8f0b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 0x561836fc3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:33960/0 (socket says 172.21.15.78:33960) 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bc9f0d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 0x561836fc41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42848/0 (socket says 172.21.15.78:42848) 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bc8f0b640 1 -- 172.21.15.78:0/3660555821 learned_addr learned my addr 172.21.15.78:0/3660555821 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.334+0000 7f0bc970c640 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fb9ee0 0x561836fc3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.335+0000 7f0bc9f0d640 1 -- 172.21.15.78:0/3660555821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fb9ee0 msgr2=0x561836fc3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.335+0000 7f0bc9f0d640 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fb9ee0 0x561836fc3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.335+0000 7f0bc9f0d640 1 -- 172.21.15.78:0/3660555821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 msgr2=0x561836fc3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.335+0000 7f0bc9f0d640 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 0x561836fc3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.335+0000 7f0bc9f0d640 1 -- 172.21.15.78:0/3660555821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561836fd6bf0 con 0x561836fc6790 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.335+0000 7f0bc8f0b640 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 0x561836fc3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.335+0000 7f0bc970c640 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fb9ee0 0x561836fc3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.335+0000 7f0bc9f0d640 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 0x561836fc41f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00c760 tx=0x7f0bbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dbacfe0309bd70 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.335+0000 7f0bacff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.335+0000 7f0bacff9640 1 -- 172.21.15.78:0/3660555821 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00f860 con 0x561836fc6790 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.335+0000 7f0bacff9640 1 -- 172.21.15.78:0/3660555821 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0bbc00fa00 con 0x561836fc6790 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.335+0000 7f0bcdd45740 1 -- 172.21.15.78:0/3660555821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc0031760 con 0x561836fc6790 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.336+0000 7f0bacff9640 1 -- 172.21.15.78:0/3660555821 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0bbc00fc00 con 0x561836fc6790 2024-09-11T00:58:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.336+0000 7f0bcdd45740 10 client.15498 fetch_fsmap learned FSMap version 29 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.336+0000 7f0bcdd45740 1 -- 172.21.15.78:0/3660555821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561836fd9570 con 0x561836fc6790 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.336+0000 7f0bacff9640 1 -- 172.21.15.78:0/3660555821 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bc0031760 con 0x561836fc6790 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.336+0000 7f0bacff9640 1 -- 172.21.15.78:0/3660555821 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0bbc0263e0 con 0x561836fc6790 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.336+0000 7f0bcdd45740 10 client.15498 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.336+0000 7f0bcdd45740 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.336+0000 7f0bcdd45740 20 client.15498 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.336+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561836fdaad0 0x561836fdcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.336+0000 7f0bcdd45740 4 client.15498 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.336+0000 7f0bcdd45740 1 -- 172.21.15.78:0/3660555821 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561836f32db0 con 0x561836fdaad0 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.337+0000 7f0bc970c640 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561836fdaad0 0x561836fdcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.337+0000 7f0bc970c640 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561836fdaad0 0x561836fdcec0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a2627ef5577d8ad server_cookie=3f1956bc34b7945 in_seq=0 out_seq=0 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.337+0000 7f0bacff9640 10 client.15498 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bacff9640 1 -- 172.21.15.78:0/3660555821 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561836f32db0 con 0x561836fdaad0 2024-09-11T00:58:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bacff9640 10 client.15498 handle_command_reply: tid=0 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bcdd45740 1 client.15498 shutdown 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bcdd45740 1 -- 172.21.15.78:0/3660555821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 msgr2=0x561836fc41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 0x561836fc41f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00c760 tx=0x7f0bbc004a40 comp rx=0 tx=0).stop 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bcdd45740 1 -- 172.21.15.78:0/3660555821 shutdown_connections 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561836fdaad0 0x561836fdcec0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 0x561836fc41f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fba800 0x561836fc3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/3660555821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fb9ee0 0x561836fc3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bcdd45740 1 -- 172.21.15.78:0/3660555821 >> 172.21.15.78:0/3660555821 conn(0x561836fb7d10 msgr2=0x561836fb6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bacff9640 0 client.15498 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bcdd45740 1 -- 172.21.15.78:0/3660555821 shutdown_connections 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bcdd45740 1 -- 172.21.15.78:0/3660555821 wait complete. 2024-09-11T00:58:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.338+0000 7f0bcdd45740 20 client.15498 trim_cache size 0 max 16384 2024-09-11T00:58:58.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.372+0000 7f0bcdd45740 1 Processor -- start 2024-09-11T00:58:58.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.372+0000 7f0bcdd45740 1 -- start start 2024-09-11T00:58:58.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.372+0000 7f0bcdd45740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fb9ee0 0x561836fc54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.372+0000 7f0bcdd45740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fba800 0x561836fe2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.372+0000 7f0bcdd45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 0x561836fe2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bcdd45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41ac310 con 0x561836fc6790 2024-09-11T00:58:58.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bcdd45740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41ade50 con 0x561836fb9ee0 2024-09-11T00:58:58.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bcdd45740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41ac9e0 con 0x561836fba800 2024-09-11T00:58:58.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bc9f0d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 0x561836fe2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bc9f0d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 0x561836fe2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42864/0 (socket says 172.21.15.78:42864) 2024-09-11T00:58:58.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bc9f0d640 1 -- 172.21.15.78:0/2208469475 learned_addr learned my addr 172.21.15.78:0/2208469475 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:58.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bc970c640 1 --2- 172.21.15.78:0/2208469475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fb9ee0 0x561836fc54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bc8f0b640 1 --2- 172.21.15.78:0/2208469475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fba800 0x561836fe2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bc970c640 1 -- 172.21.15.78:0/2208469475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fba800 msgr2=0x561836fe2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:58.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bc970c640 1 --2- 172.21.15.78:0/2208469475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fba800 0x561836fe2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bc970c640 1 -- 172.21.15.78:0/2208469475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 msgr2=0x561836fe2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:58.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bc970c640 1 --2- 172.21.15.78:0/2208469475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 0x561836fe2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bc970c640 1 -- 172.21.15.78:0/2208469475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561836fb8790 con 0x561836fb9ee0 2024-09-11T00:58:58.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bc9f0d640 1 --2- 172.21.15.78:0/2208469475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 0x561836fe2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:58:58.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.373+0000 7f0bc970c640 1 --2- 172.21.15.78:0/2208469475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fb9ee0 0x561836fc54b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0bc002a930 tx=0x7f0bc0005320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5df44bad2b95d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:58.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.374+0000 7f0baeffd640 1 -- 172.21.15.78:0/2208469475 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0035040 con 0x561836fb9ee0 2024-09-11T00:58:58.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.374+0000 7f0baeffd640 1 -- 172.21.15.78:0/2208469475 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bc00055c0 con 0x561836fb9ee0 2024-09-11T00:58:58.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.374+0000 7f0baeffd640 1 -- 172.21.15.78:0/2208469475 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc00046c0 con 0x561836fb9ee0 2024-09-11T00:58:58.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.375+0000 7f0bcdd45740 1 -- 172.21.15.78:0/2208469475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fb9ee0 msgr2=0x561836fc54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.375+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/2208469475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fb9ee0 0x561836fc54b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0bc002a930 tx=0x7f0bc0005320 comp rx=0 tx=0).stop 2024-09-11T00:58:58.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.375+0000 7f0bcdd45740 1 -- 172.21.15.78:0/2208469475 shutdown_connections 2024-09-11T00:58:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.375+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/2208469475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fc6790 0x561836fe2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.375+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/2208469475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fba800 0x561836fe2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.375+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/2208469475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fb9ee0 0x561836fc54b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.375+0000 7f0bcdd45740 1 -- 172.21.15.78:0/2208469475 >> 172.21.15.78:0/2208469475 conn(0x561836fb7d10 msgr2=0x561836fc4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.375+0000 7f0bcdd45740 1 -- 172.21.15.78:0/2208469475 shutdown_connections 2024-09-11T00:58:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.375+0000 7f0bcdd45740 1 -- 172.21.15.78:0/2208469475 wait complete. 2024-09-11T00:58:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.375+0000 7f0bcdd45740 1 -- start start 2024-09-11T00:58:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.375+0000 7f0bcdd45740 1 Processor -- start 2024-09-11T00:58:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bcdd45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 0x561836fcf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bcdd45740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fba800 0x561836fcf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bcdd45740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fc6790 0x561837046990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bcdd45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41ad780 con 0x561836fb9ee0 2024-09-11T00:58:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bcdd45740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41ad0b0 con 0x561836fc6790 2024-09-11T00:58:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bcdd45740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41aa310 con 0x561836fba800 2024-09-11T00:58:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bcdd45740 10 client.0 _ll_register_callbacks cb 0x7f0bbc014040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bc970c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 0x561836fcf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bc970c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 0x561836fcf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42878/0 (socket says 172.21.15.78:42878) 2024-09-11T00:58:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bc970c640 1 -- 172.21.15.78:0/1367509801 learned_addr learned my addr 172.21.15.78:0/1367509801 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bc8f0b640 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fba800 0x561836fcf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bc9f0d640 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fc6790 0x561837046990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bc8f0b640 1 -- 172.21.15.78:0/1367509801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fc6790 msgr2=0x561837046990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bc8f0b640 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fc6790 0x561837046990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bc8f0b640 1 -- 172.21.15.78:0/1367509801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 msgr2=0x561836fcf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bc8f0b640 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 0x561836fcf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bc8f0b640 1 -- 172.21.15.78:0/1367509801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561837046f00 con 0x561836fba800 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bc970c640 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 0x561836fcf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bc8f0b640 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fba800 0x561836fcf620 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0bb401b000 tx=0x7f0bb4004400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41b4b159f4fd5a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bacff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.376+0000 7f0bacff9640 1 -- 172.21.15.78:0/1367509801 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40b3e60 con 0x561836fba800 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bacff9640 1 -- 172.21.15.78:0/1367509801 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0bb40b5100 con 0x561836fba800 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1367509801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc00055c0 con 0x561836fba800 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bacff9640 1 -- 172.21.15.78:0/1367509801 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0bb40ac000 con 0x561836fba800 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bcdd45740 10 client.24964 fetch_fsmap learned FSMap version 29 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1367509801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561837048240 con 0x561836fba800 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bacff9640 1 -- 172.21.15.78:0/1367509801 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb40b0ae0 con 0x561836fba800 2024-09-11T00:58:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bacff9640 1 -- 172.21.15.78:0/1367509801 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0bb40a7ae0 con 0x561836fba800 2024-09-11T00:58:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bcdd45740 10 client.24964 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:58:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bcdd45740 10 client.24964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T00:58:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bcdd45740 20 client.24964 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5618370489c0 0x56183704ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bcdd45740 4 client.24964 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T00:58:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.377+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1367509801 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f0bc00055c0 con 0x5618370489c0 2024-09-11T00:58:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.378+0000 7f0bc970c640 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5618370489c0 0x56183704ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.378+0000 7f0bacff9640 10 client.24964 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.378+0000 7f0bc970c640 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5618370489c0 0x56183704ade0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1752a525886646ea server_cookie=bb5b114e4230f990 in_seq=0 out_seq=0 2024-09-11T00:58:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.379+0000 7f0bacff9640 1 -- 172.21.15.78:0/1367509801 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f0bc00055c0 con 0x5618370489c0 2024-09-11T00:58:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.379+0000 7f0bacff9640 10 client.24964 handle_command_reply: tid=0 2024-09-11T00:58:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.379+0000 7f0bcdd45740 1 client.24964 shutdown 2024-09-11T00:58:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.380+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1367509801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fba800 msgr2=0x561836fcf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.380+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fba800 0x561836fcf620 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0bb401b000 tx=0x7f0bb4004400 comp rx=0 tx=0).stop 2024-09-11T00:58:58.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.380+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1367509801 shutdown_connections 2024-09-11T00:58:58.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.380+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561836fc6790 0x561837046990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.380+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561836fba800 0x561836fcf620 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0bb401b000 tx=0x7f0bb4004400 comp rx=0 tx=0).stop 2024-09-11T00:58:58.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.380+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5618370489c0 0x56183704ade0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.380+0000 7f0bcdd45740 1 --2- 172.21.15.78:0/1367509801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561836fb9ee0 0x561836fcf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.380+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1367509801 >> 172.21.15.78:0/1367509801 conn(0x561836fb7d10 msgr2=0x561836fc55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:58.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.380+0000 7f0bacff9640 0 client.24964 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:58.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.380+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1367509801 shutdown_connections 2024-09-11T00:58:58.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.380+0000 7f0bcdd45740 1 -- 172.21.15.78:0/1367509801 wait complete. 2024-09-11T00:58:58.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.380+0000 7f0bcdd45740 20 client.24964 trim_cache size 0 max 16384 2024-09-11T00:58:58.386 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:58:58.386 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T00:58:58.387 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "89389c97-2d94-44a5-9660-dc74c67b6658", 2024-09-11T00:58:58.387 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T00:58:58.387 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:58:58.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.381+0000 7f0bcb970640 1 -- 172.21.15.78:0/922287421 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0bb00b53c0 msgr2=0x7f0bb00b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.381+0000 7f0bcb970640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0bb00b53c0 0x7f0bb00b7880 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc005fd0 tx=0x7f0bbc005f60 comp rx=0 tx=0).stop 2024-09-11T00:58:58.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.381+0000 7f0bcb970640 1 -- 172.21.15.78:0/922287421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc41076a0 msgr2=0x7f0bc4199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.381+0000 7f0bcb970640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc41076a0 0x7f0bc4199780 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400d950 tx=0x7f0bb400de10 comp rx=0 tx=0).stop 2024-09-11T00:58:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.381+0000 7f0bcb970640 1 -- 172.21.15.78:0/922287421 shutdown_connections 2024-09-11T00:58:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.381+0000 7f0bcb970640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0bb00b53c0 0x7f0bb00b7880 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.381+0000 7f0bcb970640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bc4108020 0x7f0bc419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.381+0000 7f0bcb970640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0bc41076a0 0x7f0bc4199780 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.381+0000 7f0bcb970640 1 --2- 172.21.15.78:0/922287421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bc4101c90 0x7f0bc4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.381+0000 7f0bcb970640 1 -- 172.21.15.78:0/922287421 >> 172.21.15.78:0/922287421 conn(0x7f0bc40fd700 msgr2=0x7f0bc406ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.381+0000 7f0bcb970640 1 -- 172.21.15.78:0/922287421 shutdown_connections 2024-09-11T00:58:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.381+0000 7f0bcb970640 1 -- 172.21.15.78:0/922287421 wait complete. 2024-09-11T00:58:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:58 smithi110 ceph-mon[34712]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:58 smithi110 ceph-mon[34712]: pgmap v326: 97 pgs: 97 active+clean; 649 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:58:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.449+0000 7ff30f0dd640 1 -- 172.21.15.78:0/1027403105 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7ff30806a830 con 0x7ff2f80b53c0 2024-09-11T00:58:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:58 smithi112 ceph-mon[35384]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:58 smithi112 ceph-mon[35384]: pgmap v326: 97 pgs: 97 active+clean; 649 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:58:58.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.508+0000 7ff305ffb640 1 -- 172.21.15.78:0/1027403105 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7ff30806a830 con 0x7ff2f80b53c0 2024-09-11T00:58:58.510 INFO:teuthology.orchestra.run.smithi078.stdout:Retention added to path / 2024-09-11T00:58:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.510+0000 7ff30f0dd640 1 -- 172.21.15.78:0/1027403105 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff2f80b53c0 msgr2=0x7ff2f80b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.510+0000 7ff30f0dd640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff2f80b53c0 0x7ff2f80b7880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4031440 tx=0x7ff2f4005b50 comp rx=0 tx=0).stop 2024-09-11T00:58:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.510+0000 7ff30f0dd640 1 -- 172.21.15.78:0/1027403105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff308102010 msgr2=0x7ff308199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.510+0000 7ff30f0dd640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff308102010 0x7ff308199220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00e9f0 tx=0x7ff2fc00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.510+0000 7ff30f0dd640 1 -- 172.21.15.78:0/1027403105 shutdown_connections 2024-09-11T00:58:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.510+0000 7ff30f0dd640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff2f80b53c0 0x7ff2f80b7880 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.510+0000 7ff30f0dd640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff308104d70 0x7ff30819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.510+0000 7ff30f0dd640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3081043d0 0x7ff308199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.510+0000 7ff30f0dd640 1 --2- 172.21.15.78:0/1027403105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff308102010 0x7ff308199220 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.510+0000 7ff30f0dd640 1 -- 172.21.15.78:0/1027403105 >> 172.21.15.78:0/1027403105 conn(0x7ff308077720 msgr2=0x7ff308101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.510+0000 7ff30f0dd640 1 -- 172.21.15.78:0/1027403105 shutdown_connections 2024-09-11T00:58:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.510+0000 7ff30f0dd640 1 -- 172.21.15.78:0/1027403105 wait complete. 2024-09-11T00:58:58.522 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T00:58:58.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.625+0000 7f6865fe8640 1 -- 172.21.15.78:0/3196887289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6860102010 msgr2=0x7f68601023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.625+0000 7f6865fe8640 1 --2- 172.21.15.78:0/3196887289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6860102010 0x7f68601023f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6850009900 tx=0x7f685002a410 comp rx=0 tx=0).stop 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.625+0000 7f6865fe8640 1 -- 172.21.15.78:0/3196887289 shutdown_connections 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.625+0000 7f6865fe8640 1 --2- 172.21.15.78:0/3196887289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6860104d70 0x7f686010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.625+0000 7f6865fe8640 1 --2- 172.21.15.78:0/3196887289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68601043d0 0x7f6860104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.625+0000 7f6865fe8640 1 --2- 172.21.15.78:0/3196887289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6860102010 0x7f68601023f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.625+0000 7f6865fe8640 1 -- 172.21.15.78:0/3196887289 >> 172.21.15.78:0/3196887289 conn(0x7f6860077720 msgr2=0x7f6860077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.625+0000 7f6865fe8640 1 -- 172.21.15.78:0/3196887289 shutdown_connections 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.625+0000 7f6865fe8640 1 -- 172.21.15.78:0/3196887289 wait complete. 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.625+0000 7f6865fe8640 1 Processor -- start 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.625+0000 7f6865fe8640 1 -- start start 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f6865fe8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6860102010 0x7f68600feea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f6865fe8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68601043d0 0x7f68600ff3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f6865fe8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6860104d70 0x7f68600ff920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f6865fe8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6860111c00 con 0x7f68601043d0 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f6865fe8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6860111a80 con 0x7f6860102010 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f6865fe8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6860111d80 con 0x7f6860104d70 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f685effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68601043d0 0x7f68600ff3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f685effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68601043d0 0x7f68600ff3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42894/0 (socket says 172.21.15.78:42894) 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f685effd640 1 -- 172.21.15.78:0/3204946194 learned_addr learned my addr 172.21.15.78:0/3204946194 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f685ffff640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6860104d70 0x7f68600ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f685f7fe640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6860102010 0x7f68600feea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f685effd640 1 -- 172.21.15.78:0/3204946194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6860104d70 msgr2=0x7f68600ff920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f685effd640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6860104d70 0x7f68600ff920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f685effd640 1 -- 172.21.15.78:0/3204946194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6860102010 msgr2=0x7f68600feea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f685effd640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6860102010 0x7f68600feea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f685effd640 1 -- 172.21.15.78:0/3204946194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68601a7680 con 0x7f68601043d0 2024-09-11T00:58:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f685effd640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68601043d0 0x7f68600ff3e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f685400d950 tx=0x7f685400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.626+0000 7f685cff9640 1 -- 172.21.15.78:0/3204946194 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854017070 con 0x7f68601043d0 2024-09-11T00:58:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.627+0000 7f685cff9640 1 -- 172.21.15.78:0/3204946194 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6854009c30 con 0x7f68601043d0 2024-09-11T00:58:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.627+0000 7f685cff9640 1 -- 172.21.15.78:0/3204946194 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685400f620 con 0x7f68601043d0 2024-09-11T00:58:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.627+0000 7f6865fe8640 1 -- 172.21.15.78:0/3204946194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68601a7970 con 0x7f68601043d0 2024-09-11T00:58:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.627+0000 7f6865fe8640 1 -- 172.21.15.78:0/3204946194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68601a7ec0 con 0x7f68601043d0 2024-09-11T00:58:58.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.628+0000 7f6865fe8640 1 -- 172.21.15.78:0/3204946194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68601023f0 con 0x7f68601043d0 2024-09-11T00:58:58.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.629+0000 7f685cff9640 1 -- 172.21.15.78:0/3204946194 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f6854010460 con 0x7f68601043d0 2024-09-11T00:58:58.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.630+0000 7f685cff9640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68480b53c0 0x7f68480b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.630+0000 7f685f7fe640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68480b53c0 0x7f68480b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.630+0000 7f685cff9640 1 -- 172.21.15.78:0/3204946194 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f68540e5a60 con 0x7f68601043d0 2024-09-11T00:58:58.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.630+0000 7f685f7fe640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68480b53c0 0x7f68480b7880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6850031440 tx=0x7f6850005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:58.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.634+0000 7f685cff9640 1 -- 172.21.15.78:0/3204946194 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6854009de0 con 0x7f68601043d0 2024-09-11T00:58:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.858+0000 7f6865fe8640 1 -- 172.21.15.78:0/3204946194 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f68601074d0 con 0x7f68480b53c0 2024-09-11T00:58:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.864+0000 7f685cff9640 1 -- 172.21.15.78:0/3204946194 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f68601074d0 con 0x7f68480b53c0 2024-09-11T00:58:58.866 INFO:teuthology.orchestra.run.smithi078.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T00:58:57", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-11T00:58:58.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.867+0000 7f6865fe8640 1 -- 172.21.15.78:0/3204946194 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68480b53c0 msgr2=0x7f68480b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.867+0000 7f6865fe8640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68480b53c0 0x7f68480b7880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6850031440 tx=0x7f6850005b50 comp rx=0 tx=0).stop 2024-09-11T00:58:58.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.867+0000 7f6865fe8640 1 -- 172.21.15.78:0/3204946194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68601043d0 msgr2=0x7f68600ff3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.867+0000 7f6865fe8640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68601043d0 0x7f68600ff3e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f685400d950 tx=0x7f685400de10 comp rx=0 tx=0).stop 2024-09-11T00:58:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.867+0000 7f6865fe8640 1 -- 172.21.15.78:0/3204946194 shutdown_connections 2024-09-11T00:58:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.867+0000 7f6865fe8640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68480b53c0 0x7f68480b7880 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.867+0000 7f6865fe8640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6860104d70 0x7f68600ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.867+0000 7f6865fe8640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68601043d0 0x7f68600ff3e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.867+0000 7f6865fe8640 1 --2- 172.21.15.78:0/3204946194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6860102010 0x7f68600feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.867+0000 7f6865fe8640 1 -- 172.21.15.78:0/3204946194 >> 172.21.15.78:0/3204946194 conn(0x7f6860077720 msgr2=0x7f686006aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.867+0000 7f6865fe8640 1 -- 172.21.15.78:0/3204946194 shutdown_connections 2024-09-11T00:58:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.867+0000 7f6865fe8640 1 -- 172.21.15.78:0/3204946194 wait complete. 2024-09-11T00:58:58.879 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T00:58:58.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.981+0000 7f299a265640 1 -- 172.21.15.78:0/3455837283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29941043d0 msgr2=0x7f2994104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.981+0000 7f299a265640 1 --2- 172.21.15.78:0/3455837283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29941043d0 0x7f2994104830 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f297c009900 tx=0x7f297c02a410 comp rx=0 tx=0).stop 2024-09-11T00:58:58.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.981+0000 7f299a265640 1 -- 172.21.15.78:0/3455837283 shutdown_connections 2024-09-11T00:58:58.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.981+0000 7f299a265640 1 --2- 172.21.15.78:0/3455837283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2994104d70 0x7f299410e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.981+0000 7f299a265640 1 --2- 172.21.15.78:0/3455837283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29941043d0 0x7f2994104830 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.981+0000 7f299a265640 1 --2- 172.21.15.78:0/3455837283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994102010 0x7f29941023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.981+0000 7f299a265640 1 -- 172.21.15.78:0/3455837283 >> 172.21.15.78:0/3455837283 conn(0x7f2994077720 msgr2=0x7f2994077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:58.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.981+0000 7f299a265640 1 -- 172.21.15.78:0/3455837283 shutdown_connections 2024-09-11T00:58:58.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.981+0000 7f299a265640 1 -- 172.21.15.78:0/3455837283 wait complete. 2024-09-11T00:58:58.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.982+0000 7f299a265640 1 Processor -- start 2024-09-11T00:58:58.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.982+0000 7f299a265640 1 -- start start 2024-09-11T00:58:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.982+0000 7f299a265640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2994102010 0x7f29941991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.982+0000 7f299a265640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29941043d0 0x7f29941996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.982+0000 7f299a265640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2994104d70 0x7f299419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.982+0000 7f299a265640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2994112d50 con 0x7f2994102010 2024-09-11T00:58:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.982+0000 7f299a265640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2994112bd0 con 0x7f2994104d70 2024-09-11T00:58:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.982+0000 7f299a265640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2994112ed0 con 0x7f29941043d0 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f2993fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2994102010 0x7f29941991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f2993fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2994102010 0x7f29941991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42920/0 (socket says 172.21.15.78:42920) 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f2993fff640 1 -- 172.21.15.78:0/2450689646 learned_addr learned my addr 172.21.15.78:0/2450689646 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f2998802640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2994104d70 0x7f299419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f29937fe640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29941043d0 0x7f29941996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f2993fff640 1 -- 172.21.15.78:0/2450689646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29941043d0 msgr2=0x7f29941996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f2993fff640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29941043d0 0x7f29941996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f2993fff640 1 -- 172.21.15.78:0/2450689646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2994104d70 msgr2=0x7f299419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f2993fff640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2994104d70 0x7f299419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f2993fff640 1 -- 172.21.15.78:0/2450689646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299419f250 con 0x7f2994102010 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f2998802640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2994104d70 0x7f299419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f2993fff640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2994102010 0x7f29941991b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f298400c920 tx=0x7f298400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f29917fa640 1 -- 172.21.15.78:0/2450689646 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984007c70 con 0x7f2994102010 2024-09-11T00:58:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f29917fa640 1 -- 172.21.15.78:0/2450689646 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2984007e10 con 0x7f2994102010 2024-09-11T00:58:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f299a265640 1 -- 172.21.15.78:0/2450689646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299419f540 con 0x7f2994102010 2024-09-11T00:58:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f29917fa640 1 -- 172.21.15.78:0/2450689646 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984018820 con 0x7f2994102010 2024-09-11T00:58:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.983+0000 7f299a265640 1 -- 172.21.15.78:0/2450689646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29941a5de0 con 0x7f2994102010 2024-09-11T00:58:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.984+0000 7f299a265640 1 -- 172.21.15.78:0/2450689646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29941023f0 con 0x7f2994102010 2024-09-11T00:58:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.985+0000 7f29917fa640 1 -- 172.21.15.78:0/2450689646 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f298400f450 con 0x7f2994102010 2024-09-11T00:58:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.986+0000 7f29917fa640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f29800b5440 0x7f29800b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.986+0000 7f29937fe640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f29800b5440 0x7f29800b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.986+0000 7f29917fa640 1 -- 172.21.15.78:0/2450689646 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f29840e6010 con 0x7f2994102010 2024-09-11T00:58:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.987+0000 7f29937fe640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f29800b5440 0x7f29800b7900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f297c006010 tx=0x7f297c000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:58.990+0000 7f29917fa640 1 -- 172.21.15.78:0/2450689646 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2984014030 con 0x7f2994102010 2024-09-11T00:58:59.146 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T00:58:59.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.196+0000 7f299a265640 1 -- 172.21.15.78:0/2450689646 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2994069f20 con 0x7f29800b5440 2024-09-11T00:58:59.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.203+0000 7f29917fa640 1 -- 172.21.15.78:0/2450689646 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2994069f20 con 0x7f29800b5440 2024-09-11T00:58:59.204 INFO:teuthology.orchestra.run.smithi078.stdout:/ 1m 6m3h 2024-09-11T00:58:59.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.205+0000 7f299a265640 1 -- 172.21.15.78:0/2450689646 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f29800b5440 msgr2=0x7f29800b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:59.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.205+0000 7f299a265640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f29800b5440 0x7f29800b7900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f297c006010 tx=0x7f297c000bc0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.205+0000 7f299a265640 1 -- 172.21.15.78:0/2450689646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2994102010 msgr2=0x7f29941991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:59.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.205+0000 7f299a265640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2994102010 0x7f29941991b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f298400c920 tx=0x7f298400cde0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.205+0000 7f299a265640 1 -- 172.21.15.78:0/2450689646 shutdown_connections 2024-09-11T00:58:59.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.205+0000 7f299a265640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f29800b5440 0x7f29800b7900 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.205+0000 7f299a265640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2994104d70 0x7f299419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.205+0000 7f299a265640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29941043d0 0x7f29941996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.205+0000 7f299a265640 1 --2- 172.21.15.78:0/2450689646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2994102010 0x7f29941991b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.205+0000 7f299a265640 1 -- 172.21.15.78:0/2450689646 >> 172.21.15.78:0/2450689646 conn(0x7f2994077720 msgr2=0x7f2994069810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:59.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.205+0000 7f299a265640 1 -- 172.21.15.78:0/2450689646 shutdown_connections 2024-09-11T00:58:59.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.205+0000 7f299a265640 1 -- 172.21.15.78:0/2450689646 wait complete. 2024-09-11T00:58:59.217 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-11T00:58:59.243 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-11T00:58:59.256 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T00:58:59.283 INFO:tasks.workunit:Pulling workunits from ref 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:58:59.284 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T00:58:59.284 INFO:tasks.workunit:timeout=3h 2024-09-11T00:58:59.284 INFO:tasks.workunit:cleanup=True 2024-09-11T00:58:59.284 DEBUG:teuthology.orchestra.run.smithi078:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T00:58:59.300 INFO:teuthology.orchestra.run.smithi078.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-11T00:58:59.300 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-11T00:58:59.300 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 54h/84d Inode: 1099511628283 Links: 2 2024-09-11T00:58:59.300 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:58:59.300 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-11T00:58:59.300 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-11 00:57:12.128678661 +0000 2024-09-11T00:58:59.300 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-11 00:57:12.128678661 +0000 2024-09-11T00:58:59.300 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-11 00:58:48.958519486 +0000 2024-09-11T00:58:59.300 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-11 00:57:12.128678661 +0000 2024-09-11T00:58:59.301 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-11T00:58:59.301 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-11T00:58:59.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:59 smithi078 ceph-mon[31315]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-11T00:58:59.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:59 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T00:58:59.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:59 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T00:58:59.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:59 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T00:58:59.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:59 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T00:58:59.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:59 smithi078 ceph-mon[31315]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:59.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:58:59 smithi078 ceph-mon[31315]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:59.376 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:58:59.403 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:58:59.418 INFO:tasks.workunit.client.0.smithi078.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T00:58:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:59 smithi110 ceph-mon[34712]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-11T00:58:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:59 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T00:58:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:59 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T00:58:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:59 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T00:58:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:59 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T00:58:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:59 smithi110 ceph-mon[34712]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:58:59 smithi110 ceph-mon[34712]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:59 smithi112 ceph-mon[35384]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-11T00:58:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:59 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T00:58:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:59 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T00:58:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:59 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T00:58:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:59 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T00:58:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:59 smithi112 ceph-mon[35384]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:59.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:58:59 smithi112 ceph-mon[35384]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:58:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.887+0000 7f60c580d640 1 -- 172.21.15.78:0/1529292750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c0102dc0 msgr2=0x7f60c01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.887+0000 7f60c580d640 1 --2- 172.21.15.78:0/1529292750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c0102dc0 0x7f60c01031a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009900 tx=0x7f60b002a410 comp rx=0 tx=0).stop 2024-09-11T00:58:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.887+0000 7f60c580d640 1 -- 172.21.15.78:0/1529292750 shutdown_connections 2024-09-11T00:58:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.887+0000 7f60c580d640 1 --2- 172.21.15.78:0/1529292750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60c0109150 0x7f60c010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.887+0000 7f60c580d640 1 --2- 172.21.15.78:0/1529292750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c01087d0 0x7f60c0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.887+0000 7f60c580d640 1 --2- 172.21.15.78:0/1529292750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c0102dc0 0x7f60c01031a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.887+0000 7f60c580d640 1 -- 172.21.15.78:0/1529292750 >> 172.21.15.78:0/1529292750 conn(0x7f60c00fe830 msgr2=0x7f60c0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.887+0000 7f60c580d640 1 -- 172.21.15.78:0/1529292750 shutdown_connections 2024-09-11T00:58:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.887+0000 7f60c580d640 1 -- 172.21.15.78:0/1529292750 wait complete. 2024-09-11T00:58:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.888+0000 7f60c580d640 1 Processor -- start 2024-09-11T00:58:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.888+0000 7f60c580d640 1 -- start start 2024-09-11T00:58:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60c580d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c0102dc0 0x7f60c0199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60c580d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c01087d0 0x7f60c0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60c580d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60c0109150 0x7f60c019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60c580d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60c0112e50 con 0x7f60c01087d0 2024-09-11T00:58:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60c580d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60c0112cd0 con 0x7f60c0109150 2024-09-11T00:58:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60c580d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60c0112fd0 con 0x7f60c0102dc0 2024-09-11T00:58:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c01087d0 0x7f60c0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c01087d0 0x7f60c0199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42942/0 (socket says 172.21.15.78:42942) 2024-09-11T00:58:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60be7fc640 1 -- 172.21.15.78:0/2730330505 learned_addr learned my addr 172.21.15.78:0/2730330505 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60beffd640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c0102dc0 0x7f60c0199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60bf7fe640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60c0109150 0x7f60c019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60be7fc640 1 -- 172.21.15.78:0/2730330505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c0102dc0 msgr2=0x7f60c0199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60be7fc640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c0102dc0 0x7f60c0199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60be7fc640 1 -- 172.21.15.78:0/2730330505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60c0109150 msgr2=0x7f60c019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60be7fc640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60c0109150 0x7f60c019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60be7fc640 1 -- 172.21.15.78:0/2730330505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c019f4f0 con 0x7f60c01087d0 2024-09-11T00:58:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60bf7fe640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60c0109150 0x7f60c019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:58:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60be7fc640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c01087d0 0x7f60c0199980 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f60b400b4a0 tx=0x7f60b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60c480b640 1 -- 172.21.15.78:0/2730330505 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4014070 con 0x7f60c01087d0 2024-09-11T00:58:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60c480b640 1 -- 172.21.15.78:0/2730330505 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b4010ac0 con 0x7f60c01087d0 2024-09-11T00:58:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60c580d640 1 -- 172.21.15.78:0/2730330505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c019f7e0 con 0x7f60c01087d0 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60c480b640 1 -- 172.21.15.78:0/2730330505 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4017c30 con 0x7f60c01087d0 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.889+0000 7f60c580d640 1 -- 172.21.15.78:0/2730330505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c01a5ea0 con 0x7f60c01087d0 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60c7be2740 1 Processor -- start 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60c7be2740 1 -- start start 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60c480b640 1 -- 172.21.15.78:0/2730330505 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f60b4010610 con 0x7f60c01087d0 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60c7be2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790600a0 0x55bd79060480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60c7be2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790609c0 0x55bd79070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60c7be2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 0x55bd79073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60c7be2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bd78fd6e50 con 0x55bd79070d60 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60c7be2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55bd78fc6280 con 0x55bd790609c0 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60c7be2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bd78fca090 con 0x55bd790600a0 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 0x55bd79073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 0x55bd79073150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42950/0 (socket says 172.21.15.78:42950) 2024-09-11T00:58:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60be7fc640 1 -- 172.21.15.78:0/2016667213 learned_addr learned my addr 172.21.15.78:0/2016667213 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60beffd640 1 --2- 172.21.15.78:0/2016667213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790609c0 0x55bd79070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60bf7fe640 1 --2- 172.21.15.78:0/2016667213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790600a0 0x55bd79060480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.892+0000 7f60be7fc640 1 -- 172.21.15.78:0/2016667213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790600a0 msgr2=0x55bd79060480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.893+0000 7f60be7fc640 1 --2- 172.21.15.78:0/2016667213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790600a0 0x55bd79060480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.893+0000 7f60be7fc640 1 -- 172.21.15.78:0/2016667213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790609c0 msgr2=0x55bd79070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.893+0000 7f60be7fc640 1 --2- 172.21.15.78:0/2016667213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790609c0 0x55bd79070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.893+0000 7f60be7fc640 1 -- 172.21.15.78:0/2016667213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd78d24650 con 0x55bd79070d60 2024-09-11T00:58:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.893+0000 7f60be7fc640 1 --2- 172.21.15.78:0/2016667213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 0x55bd79073150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f60b40eb600 tx=0x7f60b40ebac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5d7fb541622f48 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.893+0000 7f60a5ffb640 1 -- 172.21.15.78:0/2016667213 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b40f8070 con 0x55bd79070d60 2024-09-11T00:58:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.893+0000 7f60a5ffb640 1 -- 172.21.15.78:0/2016667213 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b40e5ca0 con 0x55bd79070d60 2024-09-11T00:58:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.893+0000 7f60a5ffb640 1 -- 172.21.15.78:0/2016667213 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b40ed920 con 0x55bd79070d60 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.893+0000 7f60c480b640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f60a80b53c0 0x7f60a80b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.893+0000 7f60bf7fe640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f60a80b53c0 0x7f60a80b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.893+0000 7f60c480b640 1 -- 172.21.15.78:0/2730330505 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f60b40e5420 con 0x7f60c01087d0 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.894+0000 7f60c7be2740 1 -- 172.21.15.78:0/2016667213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 msgr2=0x55bd79073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.894+0000 7f60bf7fe640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f60a80b53c0 0x7f60a80b7880 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f60b800a400 tx=0x7f60b8009610 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.894+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2016667213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 0x55bd79073150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f60b40eb600 tx=0x7f60b40ebac0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.894+0000 7f60c7be2740 1 -- 172.21.15.78:0/2016667213 shutdown_connections 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.894+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2016667213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 0x55bd79073150 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.894+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2016667213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790609c0 0x55bd79070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.894+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2016667213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790600a0 0x55bd79060480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.894+0000 7f60c7be2740 1 -- 172.21.15.78:0/2016667213 >> 172.21.15.78:0/2016667213 conn(0x55bd7905ded0 msgr2=0x55bd7905f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.894+0000 7f60c7be2740 1 -- 172.21.15.78:0/2016667213 shutdown_connections 2024-09-11T00:58:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.894+0000 7f60c7be2740 1 -- 172.21.15.78:0/2016667213 wait complete. 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.894+0000 7f60c7be2740 1 -- start start 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.894+0000 7f60c7be2740 1 Processor -- start 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60c7be2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790600a0 0x55bd79070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60c7be2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790609c0 0x55bd7906dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60c7be2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 0x55bd7906e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60c7be2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bd78e27700 con 0x55bd79070d60 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60c7be2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55bd78f61110 con 0x55bd790600a0 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60c7be2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bd79073a10 con 0x55bd790609c0 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60c7be2740 10 client.0 _ll_register_callbacks cb 0x55bd78f7a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60bf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 0x55bd7906e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60be7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790609c0 0x55bd7906dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60bf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 0x55bd7906e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42952/0 (socket says 172.21.15.78:42952) 2024-09-11T00:58:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60beffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790600a0 0x55bd79070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60bf7fe640 1 -- 172.21.15.78:0/3430553996 learned_addr learned my addr 172.21.15.78:0/3430553996 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60bf7fe640 1 -- 172.21.15.78:0/3430553996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790609c0 msgr2=0x55bd7906dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60bf7fe640 1 --2- 172.21.15.78:0/3430553996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790609c0 0x55bd7906dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60bf7fe640 1 -- 172.21.15.78:0/3430553996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790600a0 msgr2=0x55bd79070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60bf7fe640 1 --2- 172.21.15.78:0/3430553996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790600a0 0x55bd79070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60bf7fe640 1 -- 172.21.15.78:0/3430553996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd78f8ec40 con 0x55bd79070d60 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60bf7fe640 1 --2- 172.21.15.78:0/3430553996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 0x55bd7906e160 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f60b801ae00 tx=0x7f60b8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed46a0a8baa46fba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f609effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f609effd640 1 -- 172.21.15.78:0/3430553996 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8028060 con 0x55bd79070d60 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f60c7be2740 1 -- 172.21.15.78:0/3430553996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60b40e5ca0 con 0x55bd79070d60 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.895+0000 7f609effd640 1 -- 172.21.15.78:0/3430553996 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f60b80178a0 con 0x55bd79070d60 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.896+0000 7f609effd640 1 -- 172.21.15.78:0/3430553996 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60b801ae30 con 0x55bd79070d60 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.896+0000 7f60c7be2740 10 client.15528 fetch_fsmap learned FSMap version 29 2024-09-11T00:58:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.896+0000 7f60c7be2740 1 -- 172.21.15.78:0/3430553996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd7907f430 con 0x55bd79070d60 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.896+0000 7f609effd640 1 -- 172.21.15.78:0/3430553996 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b40e5ca0 con 0x55bd79070d60 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.896+0000 7f609effd640 1 -- 172.21.15.78:0/3430553996 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f60b802b020 con 0x55bd79070d60 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.896+0000 7f60c7be2740 10 client.15528 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.896+0000 7f60c7be2740 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.896+0000 7f60c7be2740 20 client.15528 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.896+0000 7f60c7be2740 1 --2- 172.21.15.78:0/3430553996 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55bd79080890 0x55bd79082d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.896+0000 7f60c7be2740 4 client.15528 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.896+0000 7f60c7be2740 1 -- 172.21.15.78:0/3430553996 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd78fd8da0 con 0x55bd79080890 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.896+0000 7f60beffd640 1 --2- 172.21.15.78:0/3430553996 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55bd79080890 0x55bd79082d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.897+0000 7f609effd640 10 client.15528 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.897+0000 7f60beffd640 1 --2- 172.21.15.78:0/3430553996 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55bd79080890 0x55bd79082d10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=312217e226144306 server_cookie=1c3c6c0f94c3cb3e in_seq=0 out_seq=0 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.897+0000 7f609effd640 1 -- 172.21.15.78:0/3430553996 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd78fd8da0 con 0x55bd79080890 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.897+0000 7f609effd640 10 client.15528 handle_command_reply: tid=0 2024-09-11T00:58:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f60c7be2740 1 client.15528 shutdown 2024-09-11T00:58:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f60c7be2740 1 -- 172.21.15.78:0/3430553996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 msgr2=0x55bd7906e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f60c7be2740 1 --2- 172.21.15.78:0/3430553996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 0x55bd7906e160 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f60b801ae00 tx=0x7f60b8004b50 comp rx=0 tx=0).stop 2024-09-11T00:58:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f60c7be2740 1 -- 172.21.15.78:0/3430553996 shutdown_connections 2024-09-11T00:58:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f60c7be2740 1 --2- 172.21.15.78:0/3430553996 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55bd79080890 0x55bd79082d10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f60c7be2740 1 --2- 172.21.15.78:0/3430553996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd79070d60 0x55bd7906e160 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f609effd640 0 client.15528 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f60c7be2740 1 --2- 172.21.15.78:0/3430553996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790609c0 0x55bd7906dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f60c7be2740 1 --2- 172.21.15.78:0/3430553996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790600a0 0x55bd79070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f60c7be2740 1 -- 172.21.15.78:0/3430553996 >> 172.21.15.78:0/3430553996 conn(0x55bd7905ded0 msgr2=0x55bd79063a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f60c7be2740 1 -- 172.21.15.78:0/3430553996 shutdown_connections 2024-09-11T00:58:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f60c7be2740 1 -- 172.21.15.78:0/3430553996 wait complete. 2024-09-11T00:58:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.898+0000 7f60c7be2740 20 client.15528 trim_cache size 0 max 16384 2024-09-11T00:58:59.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.915+0000 7f60c7be2740 1 Processor -- start 2024-09-11T00:58:59.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.915+0000 7f60c7be2740 1 -- start start 2024-09-11T00:58:59.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.915+0000 7f60c7be2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790600a0 0x55bd7905f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.915+0000 7f60c7be2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790609c0 0x55bd7907cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.915+0000 7f60c7be2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd79070d60 0x55bd7906f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.915+0000 7f60c7be2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60c01ac500 con 0x55bd790600a0 2024-09-11T00:58:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.915+0000 7f60c7be2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60c01ae040 con 0x55bd79070d60 2024-09-11T00:58:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.915+0000 7f60c7be2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60c01acbd0 con 0x55bd790609c0 2024-09-11T00:58:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.915+0000 7f60beffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790600a0 0x55bd7905f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.915+0000 7f60beffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790600a0 0x55bd7905f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42960/0 (socket says 172.21.15.78:42960) 2024-09-11T00:58:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.915+0000 7f60beffd640 1 -- 172.21.15.78:0/2407240904 learned_addr learned my addr 172.21.15.78:0/2407240904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.916+0000 7f60be7fc640 1 --2- 172.21.15.78:0/2407240904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790609c0 0x55bd7907cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.916+0000 7f60bf7fe640 1 --2- 172.21.15.78:0/2407240904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd79070d60 0x55bd7906f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.916+0000 7f60beffd640 1 -- 172.21.15.78:0/2407240904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790609c0 msgr2=0x55bd7907cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.916+0000 7f60beffd640 1 --2- 172.21.15.78:0/2407240904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790609c0 0x55bd7907cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.916+0000 7f60beffd640 1 -- 172.21.15.78:0/2407240904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd79070d60 msgr2=0x55bd7906f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.916+0000 7f60beffd640 1 --2- 172.21.15.78:0/2407240904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd79070d60 0x55bd7906f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.916+0000 7f60beffd640 1 -- 172.21.15.78:0/2407240904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd7905e950 con 0x55bd790600a0 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.916+0000 7f60be7fc640 1 --2- 172.21.15.78:0/2407240904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790609c0 0x55bd7907cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.916+0000 7f60beffd640 1 --2- 172.21.15.78:0/2407240904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790600a0 0x55bd7905f3d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f60b002a930 tx=0x7f60b0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190e0e24ef4d610c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.916+0000 7f60a57fa640 1 -- 172.21.15.78:0/2407240904 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b004a070 con 0x55bd790600a0 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.916+0000 7f60a57fa640 1 -- 172.21.15.78:0/2407240904 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b00054d0 con 0x55bd790600a0 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.916+0000 7f60a57fa640 1 -- 172.21.15.78:0/2407240904 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0035040 con 0x55bd790600a0 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.917+0000 7f60c7be2740 1 -- 172.21.15.78:0/2407240904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790600a0 msgr2=0x55bd7905f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.917+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2407240904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790600a0 0x55bd7905f3d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f60b002a930 tx=0x7f60b0005e80 comp rx=0 tx=0).stop 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.917+0000 7f60c7be2740 1 -- 172.21.15.78:0/2407240904 shutdown_connections 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.917+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2407240904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd79070d60 0x55bd7906f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.917+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2407240904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd790609c0 0x55bd7907cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.917+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2407240904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790600a0 0x55bd7905f3d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.917+0000 7f60c7be2740 1 -- 172.21.15.78:0/2407240904 >> 172.21.15.78:0/2407240904 conn(0x55bd7905ded0 msgr2=0x55bd7906f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.917+0000 7f60c7be2740 1 -- 172.21.15.78:0/2407240904 shutdown_connections 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.917+0000 7f60c7be2740 1 -- 172.21.15.78:0/2407240904 wait complete. 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.917+0000 7f60c7be2740 1 -- start start 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60c7be2740 1 Processor -- start 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60c7be2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790600a0 0x55bd79078ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60c7be2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790609c0 0x55bd790791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60c7be2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd79070d60 0x55bd79068f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60c7be2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60c01ad970 con 0x55bd790609c0 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60c7be2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60c01ad2a0 con 0x55bd790600a0 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60c7be2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60c01aa500 con 0x55bd79070d60 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60c7be2740 10 client.0 _ll_register_callbacks cb 0x7f60b80071b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790609c0 0x55bd790791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790609c0 0x55bd790791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42964/0 (socket says 172.21.15.78:42964) 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60be7fc640 1 -- 172.21.15.78:0/2865497206 learned_addr learned my addr 172.21.15.78:0/2865497206 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60beffd640 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790600a0 0x55bd79078ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60bf7fe640 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd79070d60 0x55bd79068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60be7fc640 1 -- 172.21.15.78:0/2865497206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd79070d60 msgr2=0x55bd79068f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60be7fc640 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd79070d60 0x55bd79068f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60be7fc640 1 -- 172.21.15.78:0/2865497206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790600a0 msgr2=0x55bd79078ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60be7fc640 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790600a0 0x55bd79078ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60be7fc640 1 -- 172.21.15.78:0/2865497206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd79079a30 con 0x55bd790609c0 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f60be7fc640 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790609c0 0x55bd790791e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f60b40ab6d0 tx=0x7f60b40e2c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa7b4d03803c78d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.918+0000 7f609effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f609effd640 1 -- 172.21.15.78:0/2865497206 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b401e070 con 0x55bd790609c0 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f609effd640 1 -- 172.21.15.78:0/2865497206 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f60b40b01b0 con 0x55bd790609c0 2024-09-11T00:58:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f60c7be2740 1 -- 172.21.15.78:0/2865497206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60b00054d0 con 0x55bd790609c0 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f609effd640 1 -- 172.21.15.78:0/2865497206 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60b40a8020 con 0x55bd790609c0 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f60c7be2740 10 client.15534 fetch_fsmap learned FSMap version 29 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f60c7be2740 1 -- 172.21.15.78:0/2865497206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd790f2590 con 0x55bd790609c0 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f609effd640 1 -- 172.21.15.78:0/2865497206 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b40e8d40 con 0x55bd790609c0 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f609effd640 1 -- 172.21.15.78:0/2865497206 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f60b40a7c40 con 0x55bd790609c0 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f60c7be2740 10 client.15534 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f60c7be2740 10 client.15534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f60c7be2740 20 client.15534 populate_metadata read hostname 'smithi078' 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55bd790f28d0 0x55bd790f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f60c7be2740 4 client.15534 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.919+0000 7f60c7be2740 1 -- 172.21.15.78:0/2865497206 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f60b00054d0 con 0x55bd790f28d0 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.920+0000 7f60beffd640 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55bd790f28d0 0x55bd790f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.920+0000 7f60beffd640 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55bd790f28d0 0x55bd790f4cc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bff214d45cbbe47 server_cookie=ccf405861d608d74 in_seq=0 out_seq=0 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.920+0000 7f609effd640 10 client.15534 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.920+0000 7f609effd640 1 -- 172.21.15.78:0/2865497206 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x7f60b00054d0 con 0x55bd790f28d0 2024-09-11T00:58:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.920+0000 7f609effd640 10 client.15534 handle_command_reply: tid=0 2024-09-11T00:58:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.920+0000 7f60c7be2740 1 client.15534 shutdown 2024-09-11T00:58:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c7be2740 1 -- 172.21.15.78:0/2865497206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790609c0 msgr2=0x55bd790791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790609c0 0x55bd790791e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f60b40ab6d0 tx=0x7f60b40e2c60 comp rx=0 tx=0).stop 2024-09-11T00:58:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c7be2740 1 -- 172.21.15.78:0/2865497206 shutdown_connections 2024-09-11T00:58:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd79070d60 0x55bd79068f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55bd790f28d0 0x55bd790f4cc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd790609c0 0x55bd790791e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c7be2740 1 --2- 172.21.15.78:0/2865497206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd790600a0 0x55bd79078ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f609effd640 0 client.15534 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:58:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c7be2740 1 -- 172.21.15.78:0/2865497206 >> 172.21.15.78:0/2865497206 conn(0x55bd7905ded0 msgr2=0x55bd7906f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c7be2740 1 -- 172.21.15.78:0/2865497206 shutdown_connections 2024-09-11T00:58:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c7be2740 1 -- 172.21.15.78:0/2865497206 wait complete. 2024-09-11T00:58:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c7be2740 20 client.15534 trim_cache size 0 max 16384 2024-09-11T00:58:59.935 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:58:59.935 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T00:58:59.935 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T00:58:59.935 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T00:58:59.935 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T00:58:59.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "89389c97-2d94-44a5-9660-dc74c67b6658", 2024-09-11T00:58:59.935 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T00:58:59.936 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T00:58:59.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T00:58:59.936 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T00:58:59.936 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T00:58:59.936 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T00:58:59.936 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T00:58:59.936 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T00:58:59.936 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T00:58:59.936 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T00:58:59.936 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T00:58:59.936 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T00:58:59.937 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T00:58:59.937 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:58:59.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c580d640 1 -- 172.21.15.78:0/2730330505 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f60a80b53c0 msgr2=0x7f60a80b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:59.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c580d640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f60a80b53c0 0x7f60a80b7880 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f60b800a400 tx=0x7f60b8009610 comp rx=0 tx=0).stop 2024-09-11T00:58:59.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c580d640 1 -- 172.21.15.78:0/2730330505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c01087d0 msgr2=0x7f60c0199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:58:59.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c580d640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c01087d0 0x7f60c0199980 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f60b400b4a0 tx=0x7f60b400b960 comp rx=0 tx=0).stop 2024-09-11T00:58:59.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c580d640 1 -- 172.21.15.78:0/2730330505 shutdown_connections 2024-09-11T00:58:59.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c580d640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f60a80b53c0 0x7f60a80b7880 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c580d640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60c0109150 0x7f60c019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c580d640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c01087d0 0x7f60c0199980 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c580d640 1 --2- 172.21.15.78:0/2730330505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c0102dc0 0x7f60c0199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:58:59.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.921+0000 7f60c580d640 1 -- 172.21.15.78:0/2730330505 >> 172.21.15.78:0/2730330505 conn(0x7f60c00fe830 msgr2=0x7f60c0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:58:59.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.922+0000 7f60c580d640 1 -- 172.21.15.78:0/2730330505 shutdown_connections 2024-09-11T00:58:59.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:58:59.922+0000 7f60c580d640 1 -- 172.21.15.78:0/2730330505 wait complete. 2024-09-11T00:59:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:00 smithi110 ceph-mon[34712]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:59:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:00 smithi110 ceph-mon[34712]: pgmap v327: 97 pgs: 97 active+clean; 1.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 103 KiB/s wr, 49 op/s 2024-09-11T00:59:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:00 smithi112 ceph-mon[35384]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:59:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:00 smithi112 ceph-mon[35384]: pgmap v327: 97 pgs: 97 active+clean; 1.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 103 KiB/s wr, 49 op/s 2024-09-11T00:59:00.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:00 smithi078 ceph-mon[31315]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:59:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:00 smithi078 ceph-mon[31315]: pgmap v327: 97 pgs: 97 active+clean; 1.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 103 KiB/s wr, 49 op/s 2024-09-11T00:59:00.682 INFO:tasks.cephfs.filesystem:scrub status for tag:89389c97-2d94-44a5-9660-dc74c67b6658 - {'path': '/', 'tag': '89389c97-2d94-44a5-9660-dc74c67b6658', 'options': 'recursive,force'} 2024-09-11T00:59:00.997 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:59:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:59:00.518+0000 7ff88496a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:59:00.997 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:59:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:59:00.518+0000 7ff88496a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:59:00.997 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:59:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:59:00.518+0000 7ff88496a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:59:00.997 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:59:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:59:00.518+0000 7ff88496a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:59:00.997 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 00:59:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T00:59:00.518+0000 7ff88496a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:59:01.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:01 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:59:01.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:01 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:59:01.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:01 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:59:02.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:02 smithi110 ceph-mon[34712]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 187 KiB/s wr, 57 op/s 2024-09-11T00:59:02.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:02 smithi112 ceph-mon[35384]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 187 KiB/s wr, 57 op/s 2024-09-11T00:59:02.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:02 smithi078 ceph-mon[31315]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 187 KiB/s wr, 57 op/s 2024-09-11T00:59:03.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:03 smithi110 ceph-mon[34712]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-11T00:59:03.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:59:03.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:59:03.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:03 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:03 smithi112 ceph-mon[35384]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-11T00:59:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:59:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:59:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:03 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:03 smithi078 ceph-mon[31315]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-11T00:59:03.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:03.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:59:03.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:59:03.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:03 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T00:59:04.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:04 smithi110 ceph-mon[34712]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 187 KiB/s wr, 57 op/s 2024-09-11T00:59:04.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:04 smithi112 ceph-mon[35384]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 187 KiB/s wr, 57 op/s 2024-09-11T00:59:04.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:04 smithi078 ceph-mon[31315]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 187 KiB/s wr, 57 op/s 2024-09-11T00:59:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:07 smithi110 ceph-mon[34712]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 198 KiB/s wr, 74 op/s 2024-09-11T00:59:07.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:07 smithi112 ceph-mon[35384]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 198 KiB/s wr, 74 op/s 2024-09-11T00:59:07.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:07 smithi078 ceph-mon[31315]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 198 KiB/s wr, 74 op/s 2024-09-11T00:59:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:09 smithi110 ceph-mon[34712]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 198 KiB/s wr, 72 op/s 2024-09-11T00:59:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:09 smithi112 ceph-mon[35384]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 198 KiB/s wr, 72 op/s 2024-09-11T00:59:09.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:09 smithi078 ceph-mon[31315]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 198 KiB/s wr, 72 op/s 2024-09-11T00:59:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:11 smithi110 ceph-mon[34712]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 198 KiB/s wr, 74 op/s 2024-09-11T00:59:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:11 smithi112 ceph-mon[35384]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 198 KiB/s wr, 74 op/s 2024-09-11T00:59:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:11 smithi078 ceph-mon[31315]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 198 KiB/s wr, 74 op/s 2024-09-11T00:59:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:12 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T00:59:12.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:12 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T00:59:12.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:12 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T00:59:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:13 smithi110 ceph-mon[34712]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 96 KiB/s wr, 35 op/s 2024-09-11T00:59:13.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:13 smithi112 ceph-mon[35384]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 96 KiB/s wr, 35 op/s 2024-09-11T00:59:13.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:13 smithi078 ceph-mon[31315]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 96 KiB/s wr, 35 op/s 2024-09-11T00:59:15.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:15 smithi110 ceph-mon[34712]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 24 op/s 2024-09-11T00:59:15.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:15 smithi112 ceph-mon[35384]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 24 op/s 2024-09-11T00:59:15.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:15 smithi078 ceph-mon[31315]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 24 op/s 2024-09-11T00:59:17.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:17 smithi110 ceph-mon[34712]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 26 op/s 2024-09-11T00:59:17.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:17 smithi112 ceph-mon[35384]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 26 op/s 2024-09-11T00:59:17.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:17 smithi078 ceph-mon[31315]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 26 op/s 2024-09-11T00:59:19.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:19 smithi110 ceph-mon[34712]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T00:59:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:19 smithi112 ceph-mon[35384]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T00:59:19.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:19 smithi078 ceph-mon[31315]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T00:59:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:21 smithi110 ceph-mon[34712]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-11T00:59:21.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:21 smithi112 ceph-mon[35384]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-11T00:59:21.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:21 smithi078 ceph-mon[31315]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-11T00:59:23.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:23 smithi110 ceph-mon[34712]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:59:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:23 smithi112 ceph-mon[35384]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:59:23.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:23 smithi078 ceph-mon[31315]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:59:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:25 smithi110 ceph-mon[34712]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:59:25.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:25 smithi112 ceph-mon[35384]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:59:25.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:25 smithi078 ceph-mon[31315]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:59:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:27 smithi110 ceph-mon[34712]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:59:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:27 smithi112 ceph-mon[35384]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:59:27.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:27 smithi078 ceph-mon[31315]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T00:59:29.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:29 smithi110 ceph-mon[34712]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:59:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:29 smithi112 ceph-mon[35384]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:59:29.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:29 smithi078 ceph-mon[31315]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:59:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:30 smithi112 ceph-mon[35384]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:59:30.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:30 smithi078 ceph-mon[31315]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:59:30.684 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T00:59:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:30 smithi110 ceph-mon[34712]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T00:59:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:59:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.576+0000 7f821c130640 1 -- 172.21.15.78:0/1463823417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8214108060 msgr2=0x7f821410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.576+0000 7f821c130640 1 --2- 172.21.15.78:0/1463823417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8214108060 0x7f821410e860 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f820c009ca0 tx=0x7f820c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.576+0000 7f821c130640 1 -- 172.21.15.78:0/1463823417 shutdown_connections 2024-09-11T00:59:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.576+0000 7f821c130640 1 --2- 172.21.15.78:0/1463823417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8214108060 0x7f821410e860 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.576+0000 7f821c130640 1 --2- 172.21.15.78:0/1463823417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82141076e0 0x7f8214107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.576+0000 7f821c130640 1 --2- 172.21.15.78:0/1463823417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8214101cd0 0x7f82141020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.576+0000 7f821c130640 1 -- 172.21.15.78:0/1463823417 >> 172.21.15.78:0/1463823417 conn(0x7f82140fd700 msgr2=0x7f82140ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.577+0000 7f821c130640 1 -- 172.21.15.78:0/1463823417 shutdown_connections 2024-09-11T00:59:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.577+0000 7f821c130640 1 -- 172.21.15.78:0/1463823417 wait complete. 2024-09-11T00:59:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.577+0000 7f821c130640 1 Processor -- start 2024-09-11T00:59:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.577+0000 7f821c130640 1 -- start start 2024-09-11T00:59:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f821c130640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8214101cd0 0x7f821419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f821c130640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82141076e0 0x7f821419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f821c130640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8214108060 0x7f821419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f821c130640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8214111ce0 con 0x7f82141076e0 2024-09-11T00:59:31.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f821c130640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8214111b60 con 0x7f8214101cd0 2024-09-11T00:59:31.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f821c130640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8214111e60 con 0x7f8214108060 2024-09-11T00:59:31.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f82196cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82141076e0 0x7f821419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f8219ecc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8214101cd0 0x7f821419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f82196cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82141076e0 0x7f821419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39282/0 (socket says 172.21.15.78:39282) 2024-09-11T00:59:31.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f82196cb640 1 -- 172.21.15.78:0/799086214 learned_addr learned my addr 172.21.15.78:0/799086214 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:31.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f821a6cd640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8214108060 0x7f821419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f82196cb640 1 -- 172.21.15.78:0/799086214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8214108060 msgr2=0x7f821419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:31.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f82196cb640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8214108060 0x7f821419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f82196cb640 1 -- 172.21.15.78:0/799086214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8214101cd0 msgr2=0x7f821419a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:31.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f82196cb640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8214101cd0 0x7f821419a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f82196cb640 1 -- 172.21.15.78:0/799086214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82141a5410 con 0x7f82141076e0 2024-09-11T00:59:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f82196cb640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82141076e0 0x7f821419a550 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f820400c920 tx=0x7f820400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.578+0000 7f820affd640 1 -- 172.21.15.78:0/799086214 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204007bb0 con 0x7f82141076e0 2024-09-11T00:59:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.579+0000 7f821c130640 1 -- 172.21.15.78:0/799086214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82141a5700 con 0x7f82141076e0 2024-09-11T00:59:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.579+0000 7f821c130640 1 -- 172.21.15.78:0/799086214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82141a5c90 con 0x7f82141076e0 2024-09-11T00:59:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.579+0000 7f820affd640 1 -- 172.21.15.78:0/799086214 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8204007d50 con 0x7f82141076e0 2024-09-11T00:59:31.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.579+0000 7f820affd640 1 -- 172.21.15.78:0/799086214 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204018730 con 0x7f82141076e0 2024-09-11T00:59:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.581+0000 7f820affd640 1 -- 172.21.15.78:0/799086214 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f820400f450 con 0x7f82141076e0 2024-09-11T00:59:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f820affd640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f82000b54d0 0x7f82000b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f821e505740 1 Processor -- start 2024-09-11T00:59:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f8219ecc640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f82000b54d0 0x7f82000b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f820affd640 1 -- 172.21.15.78:0/799086214 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f82040e5fc0 con 0x7f82141076e0 2024-09-11T00:59:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f821e505740 1 -- start start 2024-09-11T00:59:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f821e505740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d70a0 0x55b6a59d7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f821e505740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 0x55b6a59e3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f821e505740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59e3950 0x55b6a59e5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f821e505740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b6a594de50 con 0x55b6a59d79c0 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f821e505740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b6a593d280 con 0x55b6a59d70a0 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f821e505740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b6a5941090 con 0x55b6a59e3950 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f8219ecc640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f82000b54d0 0x7f82000b7990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f821000abe0 tx=0x7f821000a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f82196cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 0x55b6a59e3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f82196cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 0x55b6a59e3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39296/0 (socket says 172.21.15.78:39296) 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.582+0000 7f82196cb640 1 -- 172.21.15.78:0/3192777265 learned_addr learned my addr 172.21.15.78:0/3192777265 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.583+0000 7f8219ecc640 1 --2- 172.21.15.78:0/3192777265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d70a0 0x55b6a59d7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.583+0000 7f821a6cd640 1 --2- 172.21.15.78:0/3192777265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59e3950 0x55b6a59e5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.583+0000 7f82196cb640 1 -- 172.21.15.78:0/3192777265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59e3950 msgr2=0x55b6a59e5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.583+0000 7f82196cb640 1 --2- 172.21.15.78:0/3192777265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59e3950 0x55b6a59e5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.583+0000 7f82196cb640 1 -- 172.21.15.78:0/3192777265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d70a0 msgr2=0x55b6a59d7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.583+0000 7f82196cb640 1 --2- 172.21.15.78:0/3192777265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d70a0 0x55b6a59d7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.583+0000 7f82196cb640 1 -- 172.21.15.78:0/3192777265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a569b650 con 0x55b6a59d79c0 2024-09-11T00:59:31.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.583+0000 7f82196cb640 1 --2- 172.21.15.78:0/3192777265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 0x55b6a59e3410 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f82040ac3e0 tx=0x7f82040a8420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c249c3f3e8e94b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:31.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.583+0000 7f81fffff640 1 -- 172.21.15.78:0/3192777265 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82040acb50 con 0x55b6a59d79c0 2024-09-11T00:59:31.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.583+0000 7f81fffff640 1 -- 172.21.15.78:0/3192777265 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82040ac450 con 0x55b6a59d79c0 2024-09-11T00:59:31.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.583+0000 7f81fffff640 1 -- 172.21.15.78:0/3192777265 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82040e3b30 con 0x55b6a59d79c0 2024-09-11T00:59:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.584+0000 7f821e505740 1 -- 172.21.15.78:0/3192777265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 msgr2=0x55b6a59e3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.584+0000 7f821e505740 1 --2- 172.21.15.78:0/3192777265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 0x55b6a59e3410 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f82040ac3e0 tx=0x7f82040a8420 comp rx=0 tx=0).stop 2024-09-11T00:59:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.584+0000 7f821e505740 1 -- 172.21.15.78:0/3192777265 shutdown_connections 2024-09-11T00:59:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.584+0000 7f821e505740 1 --2- 172.21.15.78:0/3192777265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59e3950 0x55b6a59e5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.584+0000 7f821e505740 1 --2- 172.21.15.78:0/3192777265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 0x55b6a59e3410 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.584+0000 7f821e505740 1 --2- 172.21.15.78:0/3192777265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d70a0 0x55b6a59d7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.584+0000 7f821e505740 1 -- 172.21.15.78:0/3192777265 >> 172.21.15.78:0/3192777265 conn(0x55b6a59d4ed0 msgr2=0x55b6a59d6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.584+0000 7f821e505740 1 -- 172.21.15.78:0/3192777265 shutdown_connections 2024-09-11T00:59:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.584+0000 7f821e505740 1 -- 172.21.15.78:0/3192777265 wait complete. 2024-09-11T00:59:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.585+0000 7f821e505740 1 -- start start 2024-09-11T00:59:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.585+0000 7f821e505740 1 Processor -- start 2024-09-11T00:59:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.586+0000 7f821e505740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d70a0 0x55b6a59e3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.586+0000 7f821e505740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59d79c0 0x55b6a59e06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.586+0000 7f821e505740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59e3950 0x55b6a59e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.586+0000 7f821a6cd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59e3950 0x55b6a59e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.586+0000 7f8219ecc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d70a0 0x55b6a59e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.586+0000 7f821a6cd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59e3950 0x55b6a59e0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39302/0 (socket says 172.21.15.78:39302) 2024-09-11T00:59:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.586+0000 7f8219ecc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d70a0 0x55b6a59e3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:58034/0 (socket says 172.21.15.78:58034) 2024-09-11T00:59:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.586+0000 7f821a6cd640 1 -- 172.21.15.78:0/1476803035 learned_addr learned my addr 172.21.15.78:0/1476803035 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.586+0000 7f82196cb640 1 --2- 172.21.15.78:0/1476803035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59d79c0 0x55b6a59e06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.586+0000 7f821e505740 1 -- 172.21.15.78:0/1476803035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b6a579e700 con 0x55b6a59e3950 2024-09-11T00:59:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.586+0000 7f821e505740 1 -- 172.21.15.78:0/1476803035 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b6a58d8110 con 0x55b6a59d70a0 2024-09-11T00:59:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.586+0000 7f821e505740 1 -- 172.21.15.78:0/1476803035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b6a59e6570 con 0x55b6a59d79c0 2024-09-11T00:59:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f821a6cd640 1 -- 172.21.15.78:0/1476803035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59d79c0 msgr2=0x55b6a59e06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f821a6cd640 1 --2- 172.21.15.78:0/1476803035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59d79c0 0x55b6a59e06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f821a6cd640 1 -- 172.21.15.78:0/1476803035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d70a0 msgr2=0x55b6a59e3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f821a6cd640 1 --2- 172.21.15.78:0/1476803035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d70a0 0x55b6a59e3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f821a6cd640 1 -- 172.21.15.78:0/1476803035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a5905c40 con 0x55b6a59e3950 2024-09-11T00:59:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f821a6cd640 1 --2- 172.21.15.78:0/1476803035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59e3950 0x55b6a59e0ce0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f820c02f500 tx=0x7f820c031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178f1fbb8c9c7884 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f81fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f821e505740 10 client.0 _ll_register_callbacks cb 0x55b6a58f1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f81fd7fa640 1 -- 172.21.15.78:0/1476803035 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c003970 con 0x55b6a59e3950 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f821e505740 1 -- 172.21.15.78:0/1476803035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82040ac450 con 0x55b6a59e3950 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f81fd7fa640 1 -- 172.21.15.78:0/1476803035 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f820c003b10 con 0x55b6a59e3950 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f81fd7fa640 1 -- 172.21.15.78:0/1476803035 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f820c003d10 con 0x55b6a59e3950 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f821e505740 10 client.15552 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.587+0000 7f821e505740 1 -- 172.21.15.78:0/1476803035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55b6a59f1d30 con 0x55b6a59e3950 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.588+0000 7f81fd7fa640 1 -- 172.21.15.78:0/1476803035 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f820c004b40 con 0x55b6a59e3950 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.588+0000 7f821e505740 10 client.15552 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.588+0000 7f821e505740 10 client.15552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.588+0000 7f821e505740 20 client.15552 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.588+0000 7f821e505740 1 --2- 172.21.15.78:0/1476803035 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b6a59f30e0 0x55b6a59f5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.588+0000 7f821e505740 4 client.15552 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:59:31.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.588+0000 7f821e505740 1 -- 172.21.15.78:0/1476803035 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6a594fda0 con 0x55b6a59f30e0 2024-09-11T00:59:31.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.588+0000 7f8219ecc640 1 --2- 172.21.15.78:0/1476803035 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b6a59f30e0 0x55b6a59f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.588+0000 7f8219ecc640 1 --2- 172.21.15.78:0/1476803035 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b6a59f30e0 0x55b6a59f5530 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc61c98b1ea022d server_cookie=934b14c9ff59c305 in_seq=0 out_seq=0 2024-09-11T00:59:31.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.588+0000 7f81fd7fa640 10 client.15552 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:31.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.589+0000 7f81fd7fa640 1 -- 172.21.15.78:0/1476803035 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6a594fda0 con 0x55b6a59f30e0 2024-09-11T00:59:31.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.589+0000 7f81fd7fa640 10 client.15552 handle_command_reply: tid=0 2024-09-11T00:59:31.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.589+0000 7f821e505740 1 client.15552 shutdown 2024-09-11T00:59:31.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.589+0000 7f821e505740 1 -- 172.21.15.78:0/1476803035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59e3950 msgr2=0x55b6a59e0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:31.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.589+0000 7f821e505740 1 --2- 172.21.15.78:0/1476803035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59e3950 0x55b6a59e0ce0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f820c02f500 tx=0x7f820c031680 comp rx=0 tx=0).stop 2024-09-11T00:59:31.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.589+0000 7f821e505740 1 -- 172.21.15.78:0/1476803035 shutdown_connections 2024-09-11T00:59:31.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.589+0000 7f821e505740 1 --2- 172.21.15.78:0/1476803035 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b6a59f30e0 0x55b6a59f5530 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.589+0000 7f821e505740 1 --2- 172.21.15.78:0/1476803035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59e3950 0x55b6a59e0ce0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.589+0000 7f821e505740 1 --2- 172.21.15.78:0/1476803035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59d79c0 0x55b6a59e06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.589+0000 7f821e505740 1 --2- 172.21.15.78:0/1476803035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d70a0 0x55b6a59e3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.589+0000 7f81fd7fa640 0 client.15552 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:31.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.589+0000 7f821e505740 1 -- 172.21.15.78:0/1476803035 >> 172.21.15.78:0/1476803035 conn(0x55b6a59d4ed0 msgr2=0x55b6a59daa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:31.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.590+0000 7f821e505740 1 -- 172.21.15.78:0/1476803035 shutdown_connections 2024-09-11T00:59:31.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.590+0000 7f821e505740 1 -- 172.21.15.78:0/1476803035 wait complete. 2024-09-11T00:59:31.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.590+0000 7f821e505740 20 client.15552 trim_cache size 0 max 16384 2024-09-11T00:59:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f821e505740 1 Processor -- start 2024-09-11T00:59:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f821e505740 1 -- start start 2024-09-11T00:59:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f821e505740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59d70a0 0x55b6a59ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f821e505740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 0x55b6a59e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f821e505740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59e3950 0x55b6a59e5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f821e505740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f82141ae390 con 0x55b6a59d79c0 2024-09-11T00:59:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f821e505740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f82141afed0 con 0x55b6a59e3950 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f821e505740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82141aea60 con 0x55b6a59d70a0 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f82196cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 0x55b6a59e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f82196cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 0x55b6a59e4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39304/0 (socket says 172.21.15.78:39304) 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f82196cb640 1 -- 172.21.15.78:0/3932439163 learned_addr learned my addr 172.21.15.78:0/3932439163 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f821a6cd640 1 --2- 172.21.15.78:0/3932439163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59e3950 0x55b6a59e5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.607+0000 7f8219ecc640 1 --2- 172.21.15.78:0/3932439163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59d70a0 0x55b6a59ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.608+0000 7f82196cb640 1 -- 172.21.15.78:0/3932439163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59d70a0 msgr2=0x55b6a59ef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.608+0000 7f82196cb640 1 --2- 172.21.15.78:0/3932439163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59d70a0 0x55b6a59ef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.608+0000 7f82196cb640 1 -- 172.21.15.78:0/3932439163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59e3950 msgr2=0x55b6a59e5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.608+0000 7f82196cb640 1 --2- 172.21.15.78:0/3932439163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59e3950 0x55b6a59e5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.608+0000 7f82196cb640 1 -- 172.21.15.78:0/3932439163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a59e35d0 con 0x55b6a59d79c0 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.608+0000 7f8219ecc640 1 --2- 172.21.15.78:0/3932439163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59d70a0 0x55b6a59ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:59:31.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.608+0000 7f82196cb640 1 --2- 172.21.15.78:0/3932439163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 0x55b6a59e4d40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f820401f000 tx=0x7f82040a82f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a419ebf6333fcca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.608+0000 7f81ff7fe640 1 -- 172.21.15.78:0/3932439163 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82040acb50 con 0x55b6a59d79c0 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.608+0000 7f81ff7fe640 1 -- 172.21.15.78:0/3932439163 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82040b59e0 con 0x55b6a59d79c0 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.608+0000 7f81ff7fe640 1 -- 172.21.15.78:0/3932439163 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82040b57b0 con 0x55b6a59d79c0 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.609+0000 7f821e505740 1 -- 172.21.15.78:0/3932439163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 msgr2=0x55b6a59e4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.609+0000 7f821e505740 1 --2- 172.21.15.78:0/3932439163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 0x55b6a59e4d40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f820401f000 tx=0x7f82040a82f0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.609+0000 7f821e505740 1 -- 172.21.15.78:0/3932439163 shutdown_connections 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.609+0000 7f821e505740 1 --2- 172.21.15.78:0/3932439163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59e3950 0x55b6a59e5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.609+0000 7f821e505740 1 --2- 172.21.15.78:0/3932439163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d79c0 0x55b6a59e4d40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.609+0000 7f821e505740 1 --2- 172.21.15.78:0/3932439163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59d70a0 0x55b6a59ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.609+0000 7f821e505740 1 -- 172.21.15.78:0/3932439163 >> 172.21.15.78:0/3932439163 conn(0x55b6a59d4ed0 msgr2=0x55b6a59def30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.609+0000 7f821e505740 1 -- 172.21.15.78:0/3932439163 shutdown_connections 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.609+0000 7f821e505740 1 -- 172.21.15.78:0/3932439163 wait complete. 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.609+0000 7f821e505740 1 -- start start 2024-09-11T00:59:31.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.609+0000 7f821e505740 1 Processor -- start 2024-09-11T00:59:31.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f821e505740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d70a0 0x55b6a59eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f821e505740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d79c0 0x55b6a59ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f821e505740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59e3950 0x55b6a59e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f821e505740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f82141af800 con 0x55b6a59d70a0 2024-09-11T00:59:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f821e505740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f82141af130 con 0x55b6a59d79c0 2024-09-11T00:59:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f821e505740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82141ac370 con 0x55b6a59e3950 2024-09-11T00:59:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f821e505740 10 client.0 _ll_register_callbacks cb 0x7f820c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f8219ecc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d70a0 0x55b6a59eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f8219ecc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d70a0 0x55b6a59eb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39308/0 (socket says 172.21.15.78:39308) 2024-09-11T00:59:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f8219ecc640 1 -- 172.21.15.78:0/938239811 learned_addr learned my addr 172.21.15.78:0/938239811 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f82196cb640 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d79c0 0x55b6a59ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f821a6cd640 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59e3950 0x55b6a59e67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f8219ecc640 1 -- 172.21.15.78:0/938239811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59e3950 msgr2=0x55b6a59e67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f8219ecc640 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59e3950 0x55b6a59e67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f8219ecc640 1 -- 172.21.15.78:0/938239811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d79c0 msgr2=0x55b6a59ebf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f8219ecc640 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d79c0 0x55b6a59ebf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f8219ecc640 1 -- 172.21.15.78:0/938239811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a59e6dd0 con 0x55b6a59d70a0 2024-09-11T00:59:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.610+0000 7f82196cb640 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d79c0 0x55b6a59ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:59:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f8219ecc640 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d70a0 0x55b6a59eb9f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f821000a9c0 tx=0x7f8210012070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9330105224411039 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f81fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:59:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f81fd7fa640 1 -- 172.21.15.78:0/938239811 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210035060 con 0x55b6a59d70a0 2024-09-11T00:59:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f821e505740 1 -- 172.21.15.78:0/938239811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82040b59e0 con 0x55b6a59d70a0 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f81fd7fa640 1 -- 172.21.15.78:0/938239811 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f821002a640 con 0x55b6a59d70a0 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f81fd7fa640 1 -- 172.21.15.78:0/938239811 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f821002c030 con 0x55b6a59d70a0 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f821e505740 10 client.15558 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f821e505740 1 -- 172.21.15.78:0/938239811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6a5a609b0 con 0x55b6a59d70a0 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f81fd7fa640 1 -- 172.21.15.78:0/938239811 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82040b59e0 con 0x55b6a59d70a0 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f81fd7fa640 1 -- 172.21.15.78:0/938239811 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8210038020 con 0x55b6a59d70a0 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f821e505740 10 client.15558 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f821e505740 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f821e505740 20 client.15558 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f821e505740 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b6a5a610b0 0x55b6a5a634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f821e505740 4 client.15558 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.611+0000 7f821e505740 1 -- 172.21.15.78:0/938239811 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6a594b850 con 0x55b6a5a610b0 2024-09-11T00:59:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f82196cb640 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b6a5a610b0 0x55b6a5a634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f82196cb640 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b6a5a610b0 0x55b6a5a634d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7039ea8b93c4ea2f server_cookie=6cd4f4a8eec2ebbb in_seq=0 out_seq=0 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f81fd7fa640 10 client.15558 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f81fd7fa640 1 -- 172.21.15.78:0/938239811 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55b6a594b850 con 0x55b6a5a610b0 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f81fd7fa640 10 client.15558 handle_command_reply: tid=0 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f821e505740 1 client.15558 shutdown 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f821e505740 1 -- 172.21.15.78:0/938239811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d70a0 msgr2=0x55b6a59eb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f821e505740 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d70a0 0x55b6a59eb9f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f821000a9c0 tx=0x7f8210012070 comp rx=0 tx=0).stop 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f821e505740 1 -- 172.21.15.78:0/938239811 shutdown_connections 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f821e505740 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6a59e3950 0x55b6a59e67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f821e505740 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6a59d79c0 0x55b6a59ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f821e505740 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b6a5a610b0 0x55b6a5a634d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.612+0000 7f821e505740 1 --2- 172.21.15.78:0/938239811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6a59d70a0 0x55b6a59eb9f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f81fd7fa640 0 client.15558 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:31.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821e505740 1 -- 172.21.15.78:0/938239811 >> 172.21.15.78:0/938239811 conn(0x55b6a59d4ed0 msgr2=0x55b6a59e1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:31.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821e505740 1 -- 172.21.15.78:0/938239811 shutdown_connections 2024-09-11T00:59:31.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821e505740 1 -- 172.21.15.78:0/938239811 wait complete. 2024-09-11T00:59:31.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821e505740 20 client.15558 trim_cache size 0 max 16384 2024-09-11T00:59:31.622 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:59:31.622 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T00:59:31.622 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T00:59:31.622 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T00:59:31.622 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T00:59:31.622 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T00:59:31.623 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T00:59:31.623 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T00:59:31.623 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T00:59:31.623 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T00:59:31.623 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T00:59:31.623 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T00:59:31.623 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T00:59:31.623 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:59:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821c130640 1 -- 172.21.15.78:0/799086214 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f82000b54d0 msgr2=0x7f82000b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821c130640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f82000b54d0 0x7f82000b7990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f821000abe0 tx=0x7f821000a2d0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821c130640 1 -- 172.21.15.78:0/799086214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82141076e0 msgr2=0x7f821419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821c130640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82141076e0 0x7f821419a550 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f820400c920 tx=0x7f820400cd20 comp rx=0 tx=0).stop 2024-09-11T00:59:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821c130640 1 -- 172.21.15.78:0/799086214 shutdown_connections 2024-09-11T00:59:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821c130640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f82000b54d0 0x7f82000b7990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821c130640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8214108060 0x7f821419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821c130640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82141076e0 0x7f821419a550 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821c130640 1 --2- 172.21.15.78:0/799086214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8214101cd0 0x7f821419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:31.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821c130640 1 -- 172.21.15.78:0/799086214 >> 172.21.15.78:0/799086214 conn(0x7f82140fd700 msgr2=0x7f82140fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:31.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821c130640 1 -- 172.21.15.78:0/799086214 shutdown_connections 2024-09-11T00:59:31.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:31.613+0000 7f821c130640 1 -- 172.21.15.78:0/799086214 wait complete. 2024-09-11T00:59:32.345 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T00:59:32.345 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:59:32.741 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:59:33.137 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:33 smithi078 ceph-mon[31315]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:59:33.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.322+0000 7f9b7a99b640 1 -- 172.21.15.78:0/2318211083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b74101c90 msgr2=0x7f9b74102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:33.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.322+0000 7f9b7a99b640 1 --2- 172.21.15.78:0/2318211083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b74101c90 0x7f9b74102070 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009900 tx=0x7f9b6402a410 comp rx=0 tx=0).stop 2024-09-11T00:59:33.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.322+0000 7f9b7a99b640 1 -- 172.21.15.78:0/2318211083 shutdown_connections 2024-09-11T00:59:33.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.322+0000 7f9b7a99b640 1 --2- 172.21.15.78:0/2318211083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b74108020 0x7f9b7410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:33.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.322+0000 7f9b7a99b640 1 --2- 172.21.15.78:0/2318211083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b741076a0 0x7f9b74107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:33.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.322+0000 7f9b7a99b640 1 --2- 172.21.15.78:0/2318211083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b74101c90 0x7f9b74102070 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:33.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.322+0000 7f9b7a99b640 1 -- 172.21.15.78:0/2318211083 >> 172.21.15.78:0/2318211083 conn(0x7f9b740fd700 msgr2=0x7f9b740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:33.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.322+0000 7f9b7a99b640 1 -- 172.21.15.78:0/2318211083 shutdown_connections 2024-09-11T00:59:33.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.323+0000 7f9b7a99b640 1 -- 172.21.15.78:0/2318211083 wait complete. 2024-09-11T00:59:33.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.323+0000 7f9b7a99b640 1 Processor -- start 2024-09-11T00:59:33.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.323+0000 7f9b7a99b640 1 -- start start 2024-09-11T00:59:33.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b7a99b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b74101c90 0x7f9b74199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:33.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b7a99b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b741076a0 0x7f9b74199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b7a99b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74108020 0x7f9b7419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b7a99b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b74111ce0 con 0x7f9b74101c90 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b7a99b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9b74111b60 con 0x7f9b74108020 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b7a99b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b74111e60 con 0x7f9b741076a0 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b73fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b74101c90 0x7f9b74199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b73fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b74101c90 0x7f9b74199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32880/0 (socket says 172.21.15.78:32880) 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b73fff640 1 -- 172.21.15.78:0/1664517317 learned_addr learned my addr 172.21.15.78:0/1664517317 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b737fe640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b741076a0 0x7f9b74199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b78f38640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74108020 0x7f9b7419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b73fff640 1 -- 172.21.15.78:0/1664517317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b741076a0 msgr2=0x7f9b74199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b73fff640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b741076a0 0x7f9b74199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b73fff640 1 -- 172.21.15.78:0/1664517317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74108020 msgr2=0x7f9b7419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b73fff640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74108020 0x7f9b7419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b73fff640 1 -- 172.21.15.78:0/1664517317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7419f3e0 con 0x7f9b74101c90 2024-09-11T00:59:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.324+0000 7f9b73fff640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b74101c90 0x7f9b74199330 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9b64005940 tx=0x7f9b640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.325+0000 7f9b717fa640 1 -- 172.21.15.78:0/1664517317 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6403f030 con 0x7f9b74101c90 2024-09-11T00:59:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.325+0000 7f9b7a99b640 1 -- 172.21.15.78:0/1664517317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7419f670 con 0x7f9b74101c90 2024-09-11T00:59:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.325+0000 7f9b717fa640 1 -- 172.21.15.78:0/1664517317 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b64003cb0 con 0x7f9b74101c90 2024-09-11T00:59:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.325+0000 7f9b7a99b640 1 -- 172.21.15.78:0/1664517317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b741a5d90 con 0x7f9b74101c90 2024-09-11T00:59:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.325+0000 7f9b717fa640 1 -- 172.21.15.78:0/1664517317 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b64049060 con 0x7f9b74101c90 2024-09-11T00:59:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.326+0000 7f9b7a99b640 1 -- 172.21.15.78:0/1664517317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b74102070 con 0x7f9b74101c90 2024-09-11T00:59:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.327+0000 7f9b717fa640 1 -- 172.21.15.78:0/1664517317 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9b6403bb90 con 0x7f9b74101c90 2024-09-11T00:59:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.328+0000 7f9b717fa640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9b5c0b5450 0x7f9b5c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.329+0000 7f9b717fa640 1 -- 172.21.15.78:0/1664517317 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f9b64106c80 con 0x7f9b74101c90 2024-09-11T00:59:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.331+0000 7f9b737fe640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9b5c0b5450 0x7f9b5c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.331+0000 7f9b737fe640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9b5c0b5450 0x7f9b5c0b7910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9b741070a0 tx=0x7f9b6800b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:33.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.334+0000 7f9b717fa640 1 -- 172.21.15.78:0/1664517317 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9b640cd220 con 0x7f9b74101c90 2024-09-11T00:59:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:33 smithi110 ceph-mon[34712]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:59:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:33 smithi112 ceph-mon[35384]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:59:33.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.613+0000 7f9b7a99b640 1 -- 172.21.15.78:0/1664517317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b7419a460 con 0x7f9b74101c90 2024-09-11T00:59:33.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.614+0000 7f9b717fa640 1 -- 172.21.15.78:0/1664517317 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11979 (secure 0 0 0) 0x7f9b7419a460 con 0x7f9b74101c90 2024-09-11T00:59:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T00:59:33.617 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"btime":"2024-09-11T00:58:50:118172+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T00:58:49.633286+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T00:59:33.617 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-09-11T00:59:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.617+0000 7f9b7a99b640 1 -- 172.21.15.78:0/1664517317 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9b5c0b5450 msgr2=0x7f9b5c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.617+0000 7f9b7a99b640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9b5c0b5450 0x7f9b5c0b7910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9b741070a0 tx=0x7f9b6800b390 comp rx=0 tx=0).stop 2024-09-11T00:59:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.617+0000 7f9b7a99b640 1 -- 172.21.15.78:0/1664517317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b74101c90 msgr2=0x7f9b74199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.617+0000 7f9b7a99b640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b74101c90 0x7f9b74199330 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9b64005940 tx=0x7f9b640042f0 comp rx=0 tx=0).stop 2024-09-11T00:59:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.617+0000 7f9b7a99b640 1 -- 172.21.15.78:0/1664517317 shutdown_connections 2024-09-11T00:59:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.617+0000 7f9b7a99b640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9b5c0b5450 0x7f9b5c0b7910 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.617+0000 7f9b7a99b640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74108020 0x7f9b7419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:33.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.617+0000 7f9b7a99b640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b741076a0 0x7f9b74199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:33.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.617+0000 7f9b7a99b640 1 --2- 172.21.15.78:0/1664517317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b74101c90 0x7f9b74199330 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:33.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.617+0000 7f9b7a99b640 1 -- 172.21.15.78:0/1664517317 >> 172.21.15.78:0/1664517317 conn(0x7f9b740fd700 msgr2=0x7f9b74103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:33.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.617+0000 7f9b7a99b640 1 -- 172.21.15.78:0/1664517317 shutdown_connections 2024-09-11T00:59:33.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:33.618+0000 7f9b7a99b640 1 -- 172.21.15.78:0/1664517317 wait complete. 2024-09-11T00:59:34.346 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:34 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1664517317' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:59:34.386 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T00:59:34.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:34 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1664517317' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:59:34.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:34 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1664517317' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:59:34.669 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:59:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.283+0000 7f9f5e683640 1 -- 172.21.15.78:0/1784183052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f581018c0 msgr2=0x7f9f58101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.283+0000 7f9f5e683640 1 --2- 172.21.15.78:0/1784183052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f581018c0 0x7f9f58101ca0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800ac70 tx=0x7f9f4802b9a0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.283+0000 7f9f5e683640 1 -- 172.21.15.78:0/1784183052 shutdown_connections 2024-09-11T00:59:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.283+0000 7f9f5e683640 1 --2- 172.21.15.78:0/1784183052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f581045a0 0x7f9f5810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.283+0000 7f9f5e683640 1 --2- 172.21.15.78:0/1784183052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f58103c20 0x7f9f58104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.283+0000 7f9f5e683640 1 --2- 172.21.15.78:0/1784183052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f581018c0 0x7f9f58101ca0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.283+0000 7f9f5e683640 1 -- 172.21.15.78:0/1784183052 >> 172.21.15.78:0/1784183052 conn(0x7f9f58076e50 msgr2=0x7f9f58077260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.283+0000 7f9f5e683640 1 -- 172.21.15.78:0/1784183052 shutdown_connections 2024-09-11T00:59:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.283+0000 7f9f5e683640 1 -- 172.21.15.78:0/1784183052 wait complete. 2024-09-11T00:59:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.285+0000 7f9f5e683640 1 Processor -- start 2024-09-11T00:59:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.285+0000 7f9f5e683640 1 -- start start 2024-09-11T00:59:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.285+0000 7f9f5e683640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f581018c0 0x7f9f581990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.285+0000 7f9f5e683640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58103c20 0x7f9f58199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.285+0000 7f9f5e683640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f581045a0 0x7f9f5819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.285+0000 7f9f5e683640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58110c20 con 0x7f9f58103c20 2024-09-11T00:59:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.285+0000 7f9f5e683640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58110aa0 con 0x7f9f581018c0 2024-09-11T00:59:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.285+0000 7f9f5e683640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58110da0 con 0x7f9f581045a0 2024-09-11T00:59:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.285+0000 7f9f577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58103c20 0x7f9f58199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.285+0000 7f9f577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58103c20 0x7f9f58199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32904/0 (socket says 172.21.15.78:32904) 2024-09-11T00:59:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.285+0000 7f9f577fe640 1 -- 172.21.15.78:0/1276811659 learned_addr learned my addr 172.21.15.78:0/1276811659 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.285+0000 7f9f57fff640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f581018c0 0x7f9f581990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f5cc20640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f581045a0 0x7f9f5819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f577fe640 1 -- 172.21.15.78:0/1276811659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f581045a0 msgr2=0x7f9f5819e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f577fe640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f581045a0 0x7f9f5819e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f577fe640 1 -- 172.21.15.78:0/1276811659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f581018c0 msgr2=0x7f9f581990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f577fe640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f581018c0 0x7f9f581990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f577fe640 1 -- 172.21.15.78:0/1276811659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5819f190 con 0x7f9f58103c20 2024-09-11T00:59:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f57fff640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f581018c0 0x7f9f581990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:59:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f577fe640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58103c20 0x7f9f58199620 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00bd30 tx=0x7f9f4c009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f557fa640 1 -- 172.21.15.78:0/1276811659 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c004020 con 0x7f9f58103c20 2024-09-11T00:59:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f5e683640 1 -- 172.21.15.78:0/1276811659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5819f480 con 0x7f9f58103c20 2024-09-11T00:59:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f557fa640 1 -- 172.21.15.78:0/1276811659 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4c0054d0 con 0x7f9f58103c20 2024-09-11T00:59:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f557fa640 1 -- 172.21.15.78:0/1276811659 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c004bc0 con 0x7f9f58103c20 2024-09-11T00:59:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.286+0000 7f9f5e683640 1 -- 172.21.15.78:0/1276811659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f58196d90 con 0x7f9f58103c20 2024-09-11T00:59:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.290+0000 7f9f557fa640 1 -- 172.21.15.78:0/1276811659 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9f4c0110b0 con 0x7f9f58103c20 2024-09-11T00:59:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.291+0000 7f9f60a58740 1 Processor -- start 2024-09-11T00:59:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.291+0000 7f9f60a58740 1 -- start start 2024-09-11T00:59:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.291+0000 7f9f60a58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae94d0 0x55c34cae98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.291+0000 7f9f60a58740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34caf5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.291+0000 7f9f60a58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34caf5d80 0x55c34caf8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.291+0000 7f9f60a58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c34ca5fea0 con 0x55c34cae94d0 2024-09-11T00:59:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.291+0000 7f9f60a58740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c34ca4f2d0 con 0x55c34cae9df0 2024-09-11T00:59:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.291+0000 7f9f60a58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c34ca530e0 con 0x55c34caf5d80 2024-09-11T00:59:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.291+0000 7f9f5cc20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae94d0 0x55c34cae98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.291+0000 7f9f5cc20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae94d0 0x55c34cae98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32916/0 (socket says 172.21.15.78:32916) 2024-09-11T00:59:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.291+0000 7f9f5cc20640 1 -- 172.21.15.78:0/366720239 learned_addr learned my addr 172.21.15.78:0/366720239 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.292+0000 7f9f577fe640 1 --2- 172.21.15.78:0/366720239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34caf5d80 0x55c34caf8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.292+0000 7f9f57fff640 1 --2- 172.21.15.78:0/366720239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34caf5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.292+0000 7f9f5cc20640 1 -- 172.21.15.78:0/366720239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34caf5d80 msgr2=0x55c34caf8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.292+0000 7f9f5cc20640 1 --2- 172.21.15.78:0/366720239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34caf5d80 0x55c34caf8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.292+0000 7f9f5cc20640 1 -- 172.21.15.78:0/366720239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 msgr2=0x55c34caf5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.292+0000 7f9f5cc20640 1 --2- 172.21.15.78:0/366720239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34caf5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.292+0000 7f9f5cc20640 1 -- 172.21.15.78:0/366720239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c34c7ad650 con 0x55c34cae94d0 2024-09-11T00:59:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.292+0000 7f9f57fff640 1 --2- 172.21.15.78:0/366720239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34caf5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:59:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.292+0000 7f9f577fe640 1 --2- 172.21.15.78:0/366720239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34caf5d80 0x55c34caf8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:59:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.293+0000 7f9f557fa640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9f400b5450 0x7f9f400b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.293+0000 7f9f5cc20640 1 --2- 172.21.15.78:0/366720239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae94d0 0x55c34cae98b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009f50 tx=0x7f9f500118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480352ff84700cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.293+0000 7f9f57fff640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9f400b5450 0x7f9f400b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.293+0000 7f9f557fa640 1 -- 172.21.15.78:0/1276811659 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f9f4c0e5590 con 0x7f9f58103c20 2024-09-11T00:59:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.293+0000 7f9f3e7fc640 1 -- 172.21.15.78:0/366720239 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50004d20 con 0x55c34cae94d0 2024-09-11T00:59:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.293+0000 7f9f3e7fc640 1 -- 172.21.15.78:0/366720239 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f50004050 con 0x55c34cae94d0 2024-09-11T00:59:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.293+0000 7f9f3e7fc640 1 -- 172.21.15.78:0/366720239 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50005280 con 0x55c34cae94d0 2024-09-11T00:59:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.294+0000 7f9f60a58740 1 -- 172.21.15.78:0/366720239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae94d0 msgr2=0x55c34cae98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.294+0000 7f9f60a58740 1 --2- 172.21.15.78:0/366720239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae94d0 0x55c34cae98b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009f50 tx=0x7f9f500118d0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.294+0000 7f9f57fff640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9f400b5450 0x7f9f400b7910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55c34cae81e0 tx=0x7f9f480303f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.294+0000 7f9f60a58740 1 -- 172.21.15.78:0/366720239 shutdown_connections 2024-09-11T00:59:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.295+0000 7f9f60a58740 1 --2- 172.21.15.78:0/366720239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34caf5d80 0x55c34caf8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.295+0000 7f9f60a58740 1 --2- 172.21.15.78:0/366720239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34caf5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.295+0000 7f9f60a58740 1 --2- 172.21.15.78:0/366720239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae94d0 0x55c34cae98b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.295+0000 7f9f60a58740 1 -- 172.21.15.78:0/366720239 >> 172.21.15.78:0/366720239 conn(0x55c34cae7300 msgr2=0x55c34cae84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.295+0000 7f9f60a58740 1 -- 172.21.15.78:0/366720239 shutdown_connections 2024-09-11T00:59:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.295+0000 7f9f60a58740 1 -- 172.21.15.78:0/366720239 wait complete. 2024-09-11T00:59:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.295+0000 7f9f60a58740 1 -- start start 2024-09-11T00:59:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.295+0000 7f9f60a58740 1 Processor -- start 2024-09-11T00:59:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f60a58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34cae94d0 0x55c34caf5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f60a58740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34caf2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f60a58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 0x55c34caf3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f60a58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c34c8b0700 con 0x55c34caf5d80 2024-09-11T00:59:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f60a58740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c34c9ea160 con 0x55c34cae9df0 2024-09-11T00:59:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f60a58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c34caf8a00 con 0x55c34cae94d0 2024-09-11T00:59:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f60a58740 10 client.0 _ll_register_callbacks cb 0x55c34ca039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f5cc20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 0x55c34caf3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f577fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34caf2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f5cc20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 0x55c34caf3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32932/0 (socket says 172.21.15.78:32932) 2024-09-11T00:59:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f5cc20640 1 -- 172.21.15.78:0/3938106949 learned_addr learned my addr 172.21.15.78:0/3938106949 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f577fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34caf2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38190/0 (socket says 172.21.15.78:38190) 2024-09-11T00:59:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f57fff640 1 --2- 172.21.15.78:0/3938106949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34cae94d0 0x55c34caf5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f577fe640 1 -- 172.21.15.78:0/3938106949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34cae94d0 msgr2=0x55c34caf5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f577fe640 1 --2- 172.21.15.78:0/3938106949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34cae94d0 0x55c34caf5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f577fe640 1 -- 172.21.15.78:0/3938106949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 msgr2=0x55c34caf3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f577fe640 1 --2- 172.21.15.78:0/3938106949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 0x55c34caf3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.296+0000 7f9f577fe640 1 -- 172.21.15.78:0/3938106949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c34ca17c90 con 0x55c34cae9df0 2024-09-11T00:59:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.297+0000 7f9f577fe640 1 --2- 172.21.15.78:0/3938106949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34caf2ba0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0e2d50 tx=0x7f9f4c0a79d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80331de0371291cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.297+0000 7f9f377fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T00:59:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.297+0000 7f9f377fe640 1 -- 172.21.15.78:0/3938106949 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0ab2a0 con 0x55c34cae9df0 2024-09-11T00:59:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.297+0000 7f9f377fe640 1 -- 172.21.15.78:0/3938106949 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f9f4c0113d0 con 0x55c34cae9df0 2024-09-11T00:59:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.297+0000 7f9f60a58740 1 -- 172.21.15.78:0/3938106949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f50004050 con 0x55c34cae9df0 2024-09-11T00:59:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.297+0000 7f9f377fe640 1 -- 172.21.15.78:0/3938106949 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c0a72a0 con 0x55c34cae9df0 2024-09-11T00:59:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.297+0000 7f9f60a58740 10 client.25025 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.297+0000 7f9f60a58740 1 -- 172.21.15.78:0/3938106949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c34cb04450 con 0x55c34cae9df0 2024-09-11T00:59:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.297+0000 7f9f377fe640 1 -- 172.21.15.78:0/3938106949 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4c0ab640 con 0x55c34cae9df0 2024-09-11T00:59:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.298+0000 7f9f377fe640 1 -- 172.21.15.78:0/3938106949 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9f4c0e7b80 con 0x55c34cae9df0 2024-09-11T00:59:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.298+0000 7f9f60a58740 10 client.25025 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.298+0000 7f9f60a58740 10 client.25025 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T00:59:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.298+0000 7f9f60a58740 20 client.25025 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.298+0000 7f9f60a58740 1 --2- 172.21.15.78:0/3938106949 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c34cb05910 0x55c34cb07d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.298+0000 7f9f60a58740 4 client.25025 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:59:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.298+0000 7f9f60a58740 1 -- 172.21.15.78:0/3938106949 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9f50004050 con 0x55c34cb05910 2024-09-11T00:59:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.298+0000 7f9f57fff640 1 --2- 172.21.15.78:0/3938106949 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c34cb05910 0x55c34cb07d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.299+0000 7f9f57fff640 1 --2- 172.21.15.78:0/3938106949 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c34cb05910 0x55c34cb07d00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f64ace2b7923a9d8 server_cookie=baa1e46f4ab3e35c in_seq=0 out_seq=0 2024-09-11T00:59:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.299+0000 7f9f377fe640 10 client.25025 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T00:59:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.299+0000 7f9f377fe640 1 -- 172.21.15.78:0/3938106949 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f9f50004050 con 0x55c34cb05910 2024-09-11T00:59:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.299+0000 7f9f377fe640 10 client.25025 handle_command_reply: tid=0 2024-09-11T00:59:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.300+0000 7f9f60a58740 1 client.25025 shutdown 2024-09-11T00:59:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.300+0000 7f9f60a58740 1 -- 172.21.15.78:0/3938106949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 msgr2=0x55c34caf2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.300+0000 7f9f60a58740 1 --2- 172.21.15.78:0/3938106949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34caf2ba0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0e2d50 tx=0x7f9f4c0a79d0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.300+0000 7f9f60a58740 1 -- 172.21.15.78:0/3938106949 shutdown_connections 2024-09-11T00:59:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.300+0000 7f9f60a58740 1 --2- 172.21.15.78:0/3938106949 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c34cb05910 0x55c34cb07d00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.300+0000 7f9f377fe640 0 client.25025 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T00:59:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.300+0000 7f9f60a58740 1 --2- 172.21.15.78:0/3938106949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 0x55c34caf3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.300+0000 7f9f60a58740 1 --2- 172.21.15.78:0/3938106949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34caf2ba0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.300+0000 7f9f60a58740 1 --2- 172.21.15.78:0/3938106949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34cae94d0 0x55c34caf5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.301+0000 7f9f60a58740 1 -- 172.21.15.78:0/3938106949 >> 172.21.15.78:0/3938106949 conn(0x55c34cae7300 msgr2=0x55c34caece70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.301+0000 7f9f60a58740 1 -- 172.21.15.78:0/3938106949 shutdown_connections 2024-09-11T00:59:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.301+0000 7f9f60a58740 1 -- 172.21.15.78:0/3938106949 wait complete. 2024-09-11T00:59:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.301+0000 7f9f60a58740 20 client.25025 trim_cache size 0 max 16384 2024-09-11T00:59:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.326+0000 7f9f60a58740 1 Processor -- start 2024-09-11T00:59:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.326+0000 7f9f60a58740 1 -- start start 2024-09-11T00:59:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.327+0000 7f9f60a58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34cae94d0 0x55c34cb01c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.327+0000 7f9f60a58740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34cae83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.327+0000 7f9f60a58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 0x55c34cae88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.327+0000 7f9f5cc20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 0x55c34cae88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.327+0000 7f9f5cc20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 0x55c34cae88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32944/0 (socket says 172.21.15.78:32944) 2024-09-11T00:59:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.327+0000 7f9f577fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34cae83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.327+0000 7f9f5cc20640 1 -- 172.21.15.78:0/1510893166 learned_addr learned my addr 172.21.15.78:0/1510893166 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.327+0000 7f9f60a58740 1 -- 172.21.15.78:0/1510893166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b4c60 con 0x55c34caf5d80 2024-09-11T00:59:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.327+0000 7f9f60a58740 1 -- 172.21.15.78:0/1510893166 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b67a0 con 0x55c34cae9df0 2024-09-11T00:59:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.327+0000 7f9f57fff640 1 --2- 172.21.15.78:0/1510893166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34cae94d0 0x55c34cb01c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.328+0000 7f9f60a58740 1 -- 172.21.15.78:0/1510893166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b5330 con 0x55c34cae94d0 2024-09-11T00:59:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.328+0000 7f9f5cc20640 1 -- 172.21.15.78:0/1510893166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34cae94d0 msgr2=0x55c34cb01c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:35.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.328+0000 7f9f5cc20640 1 --2- 172.21.15.78:0/1510893166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34cae94d0 0x55c34cb01c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.328+0000 7f9f5cc20640 1 -- 172.21.15.78:0/1510893166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 msgr2=0x55c34cae83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.328+0000 7f9f5cc20640 1 --2- 172.21.15.78:0/1510893166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34cae83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.328+0000 7f9f5cc20640 1 -- 172.21.15.78:0/1510893166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c34cae7d80 con 0x55c34caf5d80 2024-09-11T00:59:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.328+0000 7f9f577fe640 1 --2- 172.21.15.78:0/1510893166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34cae83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:59:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.328+0000 7f9f5cc20640 1 --2- 172.21.15.78:0/1510893166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 0x55c34cae88e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009ef0 tx=0x7f9f50003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31738c8eec54845c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.329+0000 7f9f3dffb640 1 -- 172.21.15.78:0/1510893166 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50003da0 con 0x55c34caf5d80 2024-09-11T00:59:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.329+0000 7f9f3dffb640 1 -- 172.21.15.78:0/1510893166 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f50004500 con 0x55c34caf5d80 2024-09-11T00:59:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.329+0000 7f9f3dffb640 1 -- 172.21.15.78:0/1510893166 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50005290 con 0x55c34caf5d80 2024-09-11T00:59:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.330+0000 7f9f60a58740 1 -- 172.21.15.78:0/1510893166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 msgr2=0x55c34cae88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.330+0000 7f9f60a58740 1 --2- 172.21.15.78:0/1510893166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 0x55c34cae88e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009ef0 tx=0x7f9f50003a30 comp rx=0 tx=0).stop 2024-09-11T00:59:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.330+0000 7f9f60a58740 1 -- 172.21.15.78:0/1510893166 shutdown_connections 2024-09-11T00:59:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.330+0000 7f9f60a58740 1 --2- 172.21.15.78:0/1510893166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34caf5d80 0x55c34cae88e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.330+0000 7f9f60a58740 1 --2- 172.21.15.78:0/1510893166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae9df0 0x55c34cae83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.330+0000 7f9f60a58740 1 --2- 172.21.15.78:0/1510893166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34cae94d0 0x55c34cb01c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.330+0000 7f9f60a58740 1 -- 172.21.15.78:0/1510893166 >> 172.21.15.78:0/1510893166 conn(0x55c34cae7300 msgr2=0x55c34caeddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.330+0000 7f9f60a58740 1 -- 172.21.15.78:0/1510893166 shutdown_connections 2024-09-11T00:59:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.330+0000 7f9f60a58740 1 -- 172.21.15.78:0/1510893166 wait complete. 2024-09-11T00:59:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.330+0000 7f9f60a58740 1 -- start start 2024-09-11T00:59:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.331+0000 7f9f60a58740 1 Processor -- start 2024-09-11T00:59:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.331+0000 7f9f60a58740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae94d0 0x55c34caf8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.331+0000 7f9f60a58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae9df0 0x55c34caf91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.331+0000 7f9f57fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae94d0 0x55c34caf8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.331+0000 7f9f577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae9df0 0x55c34caf91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.331+0000 7f9f57fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae94d0 0x55c34caf8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38220/0 (socket says 172.21.15.78:38220) 2024-09-11T00:59:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.331+0000 7f9f57fff640 1 -- 172.21.15.78:0/4169662924 learned_addr learned my addr 172.21.15.78:0/4169662924 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.331+0000 7f9f60a58740 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34caf5d80 0x55c34caf9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.331+0000 7f9f60a58740 1 -- 172.21.15.78:0/4169662924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b60d0 con 0x55c34cae9df0 2024-09-11T00:59:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.331+0000 7f9f60a58740 1 -- 172.21.15.78:0/4169662924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b5a00 con 0x55c34cae94d0 2024-09-11T00:59:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.331+0000 7f9f60a58740 1 -- 172.21.15.78:0/4169662924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b2c40 con 0x55c34caf5d80 2024-09-11T00:59:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.332+0000 7f9f5cc20640 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34caf5d80 0x55c34caf9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.332+0000 7f9f577fe640 1 -- 172.21.15.78:0/4169662924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34caf5d80 msgr2=0x55c34caf9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.332+0000 7f9f577fe640 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34caf5d80 0x55c34caf9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.332+0000 7f9f577fe640 1 -- 172.21.15.78:0/4169662924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae94d0 msgr2=0x55c34caf8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.332+0000 7f9f577fe640 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae94d0 0x55c34caf8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.332+0000 7f9f577fe640 1 -- 172.21.15.78:0/4169662924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c34cae2990 con 0x55c34cae9df0 2024-09-11T00:59:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.332+0000 7f9f60a58740 10 client.0 _ll_register_callbacks cb 0x7f9f4c011ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.332+0000 7f9f60a58740 1 -- 172.21.15.78:0/4169662924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f50004500 con 0x55c34cae9df0 2024-09-11T00:59:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.332+0000 7f9f577fe640 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae9df0 0x55c34caf91f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0a8da0 tx=0x7f9f4c011600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ca20c7e9353bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.332+0000 7f9f377fe640 10 client.15582 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:59:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.333+0000 7f9f377fe640 1 -- 172.21.15.78:0/4169662924 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c05b020 con 0x55c34cae9df0 2024-09-11T00:59:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.333+0000 7f9f377fe640 1 -- 172.21.15.78:0/4169662924 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f9f4c0e50d0 con 0x55c34cae9df0 2024-09-11T00:59:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.333+0000 7f9f377fe640 1 -- 172.21.15.78:0/4169662924 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c0ab2a0 con 0x55c34cae9df0 2024-09-11T00:59:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.333+0000 7f9f60a58740 10 client.15582 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.333+0000 7f9f60a58740 1 -- 172.21.15.78:0/4169662924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55c34cb738a0 con 0x55c34cae9df0 2024-09-11T00:59:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.333+0000 7f9f377fe640 1 -- 172.21.15.78:0/4169662924 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9f4c0a7480 con 0x55c34cae9df0 2024-09-11T00:59:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.333+0000 7f9f60a58740 10 client.15582 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.333+0000 7f9f60a58740 10 client.15582 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T00:59:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.334+0000 7f9f60a58740 20 client.15582 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.334+0000 7f9f60a58740 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c34cb73d40 0x55c34cb76160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.334+0000 7f9f60a58740 4 client.15582 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T00:59:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.334+0000 7f9f60a58740 1 -- 172.21.15.78:0/4169662924 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9f50004500 con 0x55c34cb73d40 2024-09-11T00:59:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.334+0000 7f9f57fff640 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c34cb73d40 0x55c34cb76160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.334+0000 7f9f377fe640 10 client.15582 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T00:59:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.334+0000 7f9f57fff640 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c34cb73d40 0x55c34cb76160 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=51a5ddc77a73e20 server_cookie=e44dfab2e5d78b6a in_seq=0 out_seq=0 2024-09-11T00:59:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f377fe640 1 -- 172.21.15.78:0/4169662924 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9f50004500 con 0x55c34cb73d40 2024-09-11T00:59:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f377fe640 10 client.15582 handle_command_reply: tid=0 2024-09-11T00:59:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f60a58740 1 client.15582 shutdown 2024-09-11T00:59:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f60a58740 1 -- 172.21.15.78:0/4169662924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae9df0 msgr2=0x55c34caf91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f60a58740 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae9df0 0x55c34caf91f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0a8da0 tx=0x7f9f4c011600 comp rx=0 tx=0).stop 2024-09-11T00:59:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f60a58740 1 -- 172.21.15.78:0/4169662924 shutdown_connections 2024-09-11T00:59:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f60a58740 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c34cb73d40 0x55c34cb76160 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f60a58740 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c34caf5d80 0x55c34caf9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f60a58740 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c34cae9df0 0x55c34caf91f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f60a58740 1 --2- 172.21.15.78:0/4169662924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c34cae94d0 0x55c34caf8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.344 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T00:59:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f60a58740 1 -- 172.21.15.78:0/4169662924 >> 172.21.15.78:0/4169662924 conn(0x55c34cae7300 msgr2=0x55c34cae83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f377fe640 0 client.15582 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T00:59:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f60a58740 1 -- 172.21.15.78:0/4169662924 shutdown_connections 2024-09-11T00:59:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f60a58740 1 -- 172.21.15.78:0/4169662924 wait complete. 2024-09-11T00:59:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.335+0000 7f9f60a58740 20 client.15582 trim_cache size 0 max 16384 2024-09-11T00:59:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.336+0000 7f9f5e683640 1 -- 172.21.15.78:0/1276811659 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9f400b5450 msgr2=0x7f9f400b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.336+0000 7f9f5e683640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9f400b5450 0x7f9f400b7910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55c34cae81e0 tx=0x7f9f480303f0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.336+0000 7f9f5e683640 1 -- 172.21.15.78:0/1276811659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58103c20 msgr2=0x7f9f58199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:35.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.336+0000 7f9f5e683640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58103c20 0x7f9f58199620 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00bd30 tx=0x7f9f4c009610 comp rx=0 tx=0).stop 2024-09-11T00:59:35.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.337+0000 7f9f5e683640 1 -- 172.21.15.78:0/1276811659 shutdown_connections 2024-09-11T00:59:35.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.337+0000 7f9f5e683640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9f400b5450 0x7f9f400b7910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.337+0000 7f9f5e683640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f581045a0 0x7f9f5819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.337+0000 7f9f5e683640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58103c20 0x7f9f58199620 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.337+0000 7f9f5e683640 1 --2- 172.21.15.78:0/1276811659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f581018c0 0x7f9f581990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:35.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.337+0000 7f9f5e683640 1 -- 172.21.15.78:0/1276811659 >> 172.21.15.78:0/1276811659 conn(0x7f9f58076e50 msgr2=0x7f9f58100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:35.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.337+0000 7f9f5e683640 1 -- 172.21.15.78:0/1276811659 shutdown_connections 2024-09-11T00:59:35.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:35.337+0000 7f9f5e683640 1 -- 172.21.15.78:0/1276811659 wait complete. 2024-09-11T00:59:35.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:35 smithi110 ceph-mon[34712]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:59:35.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:35 smithi112 ceph-mon[35384]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:59:35.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:35 smithi078 ceph-mon[31315]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T00:59:36.036 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T00:59:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:59:37.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.028+0000 7f0a88993640 1 -- 172.21.15.78:0/2934896444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a841076a0 msgr2=0x7f0a84107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:37.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.028+0000 7f0a88993640 1 --2- 172.21.15.78:0/2934896444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a841076a0 0x7f0a84107ae0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009900 tx=0x7f0a7802a410 comp rx=0 tx=0).stop 2024-09-11T00:59:37.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.028+0000 7f0a88993640 1 -- 172.21.15.78:0/2934896444 shutdown_connections 2024-09-11T00:59:37.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.028+0000 7f0a88993640 1 --2- 172.21.15.78:0/2934896444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a84108020 0x7f0a8410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.028+0000 7f0a88993640 1 --2- 172.21.15.78:0/2934896444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a841076a0 0x7f0a84107ae0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.028+0000 7f0a88993640 1 --2- 172.21.15.78:0/2934896444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a84101c90 0x7f0a84102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.028+0000 7f0a88993640 1 -- 172.21.15.78:0/2934896444 >> 172.21.15.78:0/2934896444 conn(0x7f0a840fd700 msgr2=0x7f0a840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:37.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.028+0000 7f0a88993640 1 -- 172.21.15.78:0/2934896444 shutdown_connections 2024-09-11T00:59:37.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.028+0000 7f0a88993640 1 -- 172.21.15.78:0/2934896444 wait complete. 2024-09-11T00:59:37.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.029+0000 7f0a88993640 1 Processor -- start 2024-09-11T00:59:37.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.029+0000 7f0a88993640 1 -- start start 2024-09-11T00:59:37.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.030+0000 7f0a88993640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a84101c90 0x7f0a8419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.030+0000 7f0a8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a84101c90 0x7f0a8419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.030+0000 7f0a8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a84101c90 0x7f0a8419b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32980/0 (socket says 172.21.15.78:32980) 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.030+0000 7f0a88993640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a841076a0 0x7f0a8419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.030+0000 7f0a8259c640 1 -- 172.21.15.78:0/1090766569 learned_addr learned my addr 172.21.15.78:0/1090766569 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.030+0000 7f0a88993640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a84108020 0x7f0a841a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.030+0000 7f0a88993640 1 -- 172.21.15.78:0/1090766569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a84111ca0 con 0x7f0a84101c90 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.030+0000 7f0a88993640 1 -- 172.21.15.78:0/1090766569 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a84111b20 con 0x7f0a84108020 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.030+0000 7f0a88993640 1 -- 172.21.15.78:0/1090766569 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a84111e20 con 0x7f0a841076a0 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.030+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a841076a0 0x7f0a8419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.030+0000 7f0a82d9d640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a84108020 0x7f0a841a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.031+0000 7f0a8259c640 1 -- 172.21.15.78:0/1090766569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a841076a0 msgr2=0x7f0a8419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.031+0000 7f0a8259c640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a841076a0 0x7f0a8419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.031+0000 7f0a8259c640 1 -- 172.21.15.78:0/1090766569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a84108020 msgr2=0x7f0a841a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:37.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.031+0000 7f0a8259c640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a84108020 0x7f0a841a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.031+0000 7f0a8259c640 1 -- 172.21.15.78:0/1090766569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a841a15e0 con 0x7f0a84101c90 2024-09-11T00:59:37.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.031+0000 7f0a8259c640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a84101c90 0x7f0a8419b6f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400e9f0 tx=0x7f0a7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:37.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.031+0000 7f0a737fe640 1 -- 172.21.15.78:0/1090766569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74024070 con 0x7f0a84101c90 2024-09-11T00:59:37.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.031+0000 7f0a737fe640 1 -- 172.21.15.78:0/1090766569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a7401f440 con 0x7f0a84101c90 2024-09-11T00:59:37.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.031+0000 7f0a88993640 1 -- 172.21.15.78:0/1090766569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a841a18d0 con 0x7f0a84101c90 2024-09-11T00:59:37.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.031+0000 7f0a737fe640 1 -- 172.21.15.78:0/1090766569 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7401d560 con 0x7f0a84101c90 2024-09-11T00:59:37.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.032+0000 7f0a88993640 1 -- 172.21.15.78:0/1090766569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a841972e0 con 0x7f0a84101c90 2024-09-11T00:59:37.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.035+0000 7f0a8ad68740 1 Processor -- start 2024-09-11T00:59:37.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.035+0000 7f0a8ad68740 1 -- start start 2024-09-11T00:59:37.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a8ad68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 0x561d8a72b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a8ad68740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a72bdf0 0x561d8a737840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a8ad68740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a73a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a8ad68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561d8a6a1ea0 con 0x561d8a72b4d0 2024-09-11T00:59:37.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a8ad68740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561d8a6912d0 con 0x561d8a737d80 2024-09-11T00:59:37.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a8ad68740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561d8a6950e0 con 0x561d8a72bdf0 2024-09-11T00:59:37.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a82d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 0x561d8a72b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a82d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 0x561d8a72b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32996/0 (socket says 172.21.15.78:32996) 2024-09-11T00:59:37.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a82d9d640 1 -- 172.21.15.78:0/230480325 learned_addr learned my addr 172.21.15.78:0/230480325 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:37.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/230480325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a73a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a82d9d640 1 -- 172.21.15.78:0/230480325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a72bdf0 msgr2=0x561d8a737840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a82d9d640 1 --2- 172.21.15.78:0/230480325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a72bdf0 0x561d8a737840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a82d9d640 1 -- 172.21.15.78:0/230480325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 msgr2=0x561d8a73a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a82d9d640 1 --2- 172.21.15.78:0/230480325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a73a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.036+0000 7f0a82d9d640 1 -- 172.21.15.78:0/230480325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d8a3ef650 con 0x561d8a72b4d0 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.037+0000 7f0a82d9d640 1 --2- 172.21.15.78:0/230480325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 0x561d8a72b8b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009f50 tx=0x7f0a7c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df27c283da30ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.037+0000 7f0a70ff9640 1 -- 172.21.15.78:0/230480325 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c004b20 con 0x561d8a72b4d0 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.037+0000 7f0a70ff9640 1 -- 172.21.15.78:0/230480325 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a7c004cc0 con 0x561d8a72b4d0 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.037+0000 7f0a737fe640 1 -- 172.21.15.78:0/1090766569 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0a7400cab0 con 0x7f0a84101c90 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.038+0000 7f0a70ff9640 1 -- 172.21.15.78:0/230480325 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0053a0 con 0x561d8a72b4d0 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.038+0000 7f0a8ad68740 1 -- 172.21.15.78:0/230480325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 msgr2=0x561d8a72b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.038+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/230480325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 0x561d8a72b8b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009f50 tx=0x7f0a7c0118d0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.039+0000 7f0a8ad68740 1 -- 172.21.15.78:0/230480325 shutdown_connections 2024-09-11T00:59:37.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.039+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/230480325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a73a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.039+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/230480325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a72bdf0 0x561d8a737840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.039+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/230480325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 0x561d8a72b8b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.039+0000 7f0a8ad68740 1 -- 172.21.15.78:0/230480325 >> 172.21.15.78:0/230480325 conn(0x561d8a729300 msgr2=0x561d8a72a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:37.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.039+0000 7f0a737fe640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a6c0b54d0 0x7f0a6c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.039+0000 7f0a737fe640 1 -- 172.21.15.78:0/1090766569 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0a740ee5c0 con 0x7f0a84101c90 2024-09-11T00:59:37.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.039+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a6c0b54d0 0x7f0a6c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.039+0000 7f0a8ad68740 1 -- 172.21.15.78:0/230480325 shutdown_connections 2024-09-11T00:59:37.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.040+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a6c0b54d0 0x7f0a6c0b7990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a78000c00 tx=0x7f0a78005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:37.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.040+0000 7f0a8ad68740 1 -- 172.21.15.78:0/230480325 wait complete. 2024-09-11T00:59:37.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.040+0000 7f0a8ad68740 1 -- start start 2024-09-11T00:59:37.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.040+0000 7f0a8ad68740 1 Processor -- start 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8ad68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 0x561d8a737550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8ad68740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a72bdf0 0x561d8a7349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8ad68740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a734ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8ad68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561d8a4f2700 con 0x561d8a72b4d0 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8ad68740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561d8a62c160 con 0x561d8a737d80 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8ad68740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561d8a73aa50 con 0x561d8a72bdf0 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 0x561d8a737550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8ad68740 10 client.0 _ll_register_callbacks cb 0x561d8a6459e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 0x561d8a737550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33000/0 (socket says 172.21.15.78:33000) 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8259c640 1 -- 172.21.15.78:0/1401805314 learned_addr learned my addr 172.21.15.78:0/1401805314 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a82d9d640 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a734ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a72bdf0 0x561d8a7349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8259c640 1 -- 172.21.15.78:0/1401805314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a72bdf0 msgr2=0x561d8a7349a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:37.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8259c640 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a72bdf0 0x561d8a7349a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8259c640 1 -- 172.21.15.78:0/1401805314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 msgr2=0x561d8a734ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8259c640 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a734ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.041+0000 7f0a8259c640 1 -- 172.21.15.78:0/1401805314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d8a659c90 con 0x561d8a72b4d0 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.042+0000 7f0a8259c640 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 0x561d8a737550 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400c6a0 tx=0x7f0a740b49f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7e297f6bb5f4e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.042+0000 7f0a69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.042+0000 7f0a69ffb640 1 -- 172.21.15.78:0/1401805314 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a740b4670 con 0x561d8a72b4d0 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.042+0000 7f0a69ffb640 1 -- 172.21.15.78:0/1401805314 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0a740b0dd0 con 0x561d8a72b4d0 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.042+0000 7f0a8ad68740 1 -- 172.21.15.78:0/1401805314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a7c004cc0 con 0x561d8a72b4d0 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.042+0000 7f0a69ffb640 1 -- 172.21.15.78:0/1401805314 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a740b4b60 con 0x561d8a72b4d0 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.042+0000 7f0a8ad68740 10 client.15594 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.042+0000 7f0a8ad68740 1 -- 172.21.15.78:0/1401805314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561d8a746300 con 0x561d8a72b4d0 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.042+0000 7f0a69ffb640 1 -- 172.21.15.78:0/1401805314 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a7c004cc0 con 0x561d8a72b4d0 2024-09-11T00:59:37.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.042+0000 7f0a69ffb640 1 -- 172.21.15.78:0/1401805314 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0a7409e080 con 0x561d8a72b4d0 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.042+0000 7f0a8ad68740 10 client.15594 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.042+0000 7f0a8ad68740 10 client.15594 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.043+0000 7f0a8ad68740 20 client.15594 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.043+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561d8a747810 0x561d8a749c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.043+0000 7f0a8ad68740 4 client.15594 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.043+0000 7f0a8ad68740 1 -- 172.21.15.78:0/1401805314 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d8a6a3df0 con 0x561d8a747810 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.043+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561d8a747810 0x561d8a749c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.043+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561d8a747810 0x561d8a749c20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=161e35995a296336 server_cookie=b4157fc80e9d0db9 in_seq=0 out_seq=0 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.043+0000 7f0a69ffb640 10 client.15594 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a69ffb640 1 -- 172.21.15.78:0/1401805314 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561d8a6a3df0 con 0x561d8a747810 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a69ffb640 10 client.15594 handle_command_reply: tid=0 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a8ad68740 1 client.15594 shutdown 2024-09-11T00:59:37.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a8ad68740 1 -- 172.21.15.78:0/1401805314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 msgr2=0x561d8a737550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 0x561d8a737550 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400c6a0 tx=0x7f0a740b49f0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a8ad68740 1 -- 172.21.15.78:0/1401805314 shutdown_connections 2024-09-11T00:59:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561d8a747810 0x561d8a749c20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a734ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a72bdf0 0x561d8a7349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a69ffb640 0 client.15594 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T00:59:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/1401805314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72b4d0 0x561d8a737550 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a8ad68740 1 -- 172.21.15.78:0/1401805314 >> 172.21.15.78:0/1401805314 conn(0x561d8a729300 msgr2=0x561d8a72ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a8ad68740 1 -- 172.21.15.78:0/1401805314 shutdown_connections 2024-09-11T00:59:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.044+0000 7f0a8ad68740 1 -- 172.21.15.78:0/1401805314 wait complete. 2024-09-11T00:59:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.045+0000 7f0a8ad68740 20 client.15594 trim_cache size 0 max 16384 2024-09-11T00:59:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.070+0000 7f0a8ad68740 1 Processor -- start 2024-09-11T00:59:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.070+0000 7f0a8ad68740 1 -- start start 2024-09-11T00:59:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.070+0000 7f0a8ad68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 0x561d8a728c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.070+0000 7f0a8ad68740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a7366e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.070+0000 7f0a8ad68740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a747810 0x561d8a731bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.070+0000 7f0a8ad68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8406ef10 con 0x561d8a72bdf0 2024-09-11T00:59:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.070+0000 7f0a8ad68740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a84070a50 con 0x561d8a737d80 2024-09-11T00:59:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.070+0000 7f0a8ad68740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8406f5e0 con 0x561d8a747810 2024-09-11T00:59:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.071+0000 7f0a8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 0x561d8a728c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.071+0000 7f0a8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 0x561d8a728c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33012/0 (socket says 172.21.15.78:33012) 2024-09-11T00:59:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.071+0000 7f0a8259c640 1 -- 172.21.15.78:0/2299390410 learned_addr learned my addr 172.21.15.78:0/2299390410 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.071+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/2299390410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a7366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.071+0000 7f0a81d9b640 1 -- 172.21.15.78:0/2299390410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a747810 msgr2=0x561d8a731bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:37.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.071+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/2299390410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a747810 0x561d8a731bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.071+0000 7f0a81d9b640 1 -- 172.21.15.78:0/2299390410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 msgr2=0x561d8a728c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:37.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.071+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/2299390410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 0x561d8a728c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.071+0000 7f0a81d9b640 1 -- 172.21.15.78:0/2299390410 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d8a729b50 con 0x561d8a737d80 2024-09-11T00:59:37.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.071+0000 7f0a8259c640 1 --2- 172.21.15.78:0/2299390410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 0x561d8a728c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:59:37.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.071+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/2299390410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a7366e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009230 tx=0x7f0a78045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb2cf4129becf9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.071+0000 7f0a6bfff640 1 -- 172.21.15.78:0/2299390410 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78069020 con 0x561d8a737d80 2024-09-11T00:59:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.072+0000 7f0a6bfff640 1 -- 172.21.15.78:0/2299390410 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a78004340 con 0x561d8a737d80 2024-09-11T00:59:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.072+0000 7f0a6bfff640 1 -- 172.21.15.78:0/2299390410 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78035020 con 0x561d8a737d80 2024-09-11T00:59:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.072+0000 7f0a8ad68740 1 -- 172.21.15.78:0/2299390410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 msgr2=0x561d8a7366e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.072+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/2299390410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a7366e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009230 tx=0x7f0a78045000 comp rx=0 tx=0).stop 2024-09-11T00:59:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.072+0000 7f0a8ad68740 1 -- 172.21.15.78:0/2299390410 shutdown_connections 2024-09-11T00:59:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.072+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/2299390410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a747810 0x561d8a731bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.072+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/2299390410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a737d80 0x561d8a7366e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.072+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/2299390410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 0x561d8a728c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.072+0000 7f0a8ad68740 1 -- 172.21.15.78:0/2299390410 >> 172.21.15.78:0/2299390410 conn(0x561d8a729300 msgr2=0x561d8a735fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.072+0000 7f0a8ad68740 1 -- 172.21.15.78:0/2299390410 shutdown_connections 2024-09-11T00:59:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.072+0000 7f0a8ad68740 1 -- 172.21.15.78:0/2299390410 wait complete. 2024-09-11T00:59:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a8ad68740 1 -- start start 2024-09-11T00:59:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a8ad68740 1 Processor -- start 2024-09-11T00:59:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a8ad68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 0x561d8a7402d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a8ad68740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a737d80 0x561d8a73c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a8ad68740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a747810 0x561d8a73ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a8ad68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a84070380 con 0x561d8a72bdf0 2024-09-11T00:59:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a8ad68740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8406fcb0 con 0x561d8a747810 2024-09-11T00:59:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a8ad68740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8406cf70 con 0x561d8a737d80 2024-09-11T00:59:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a8ad68740 10 client.0 _ll_register_callbacks cb 0x7f0a7400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:37.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 0x561d8a7402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 0x561d8a7402d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33016/0 (socket says 172.21.15.78:33016) 2024-09-11T00:59:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a8259c640 1 -- 172.21.15.78:0/191145183 learned_addr learned my addr 172.21.15.78:0/191145183 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a737d80 0x561d8a73c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.073+0000 7f0a82d9d640 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a747810 0x561d8a73ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.074+0000 7f0a81d9b640 1 -- 172.21.15.78:0/191145183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a747810 msgr2=0x561d8a73ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.074+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a747810 0x561d8a73ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.074+0000 7f0a81d9b640 1 -- 172.21.15.78:0/191145183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 msgr2=0x561d8a7402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.074+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 0x561d8a7402d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.074+0000 7f0a81d9b640 1 -- 172.21.15.78:0/191145183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d8a73d330 con 0x561d8a737d80 2024-09-11T00:59:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.074+0000 7f0a81d9b640 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a737d80 0x561d8a73c7b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0a78005520 tx=0x7f0a780044f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2441d11ba9082cf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.074+0000 7f0a69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T00:59:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.074+0000 7f0a69ffb640 1 -- 172.21.15.78:0/191145183 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78072070 con 0x561d8a737d80 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.074+0000 7f0a8ad68740 1 -- 172.21.15.78:0/191145183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a78004340 con 0x561d8a737d80 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.074+0000 7f0a69ffb640 1 -- 172.21.15.78:0/191145183 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0a78031b50 con 0x561d8a737d80 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.075+0000 7f0a69ffb640 1 -- 172.21.15.78:0/191145183 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a78035050 con 0x561d8a737d80 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.075+0000 7f0a8ad68740 10 client.25030 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.075+0000 7f0a8ad68740 1 -- 172.21.15.78:0/191145183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561d8a7b55a0 con 0x561d8a737d80 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.075+0000 7f0a69ffb640 1 -- 172.21.15.78:0/191145183 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a78004340 con 0x561d8a737d80 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.075+0000 7f0a69ffb640 1 -- 172.21.15.78:0/191145183 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0a7803a040 con 0x561d8a737d80 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a8ad68740 10 client.25030 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a8ad68740 10 client.25030 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a8ad68740 20 client.25030 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561d8a7b5bf0 0x561d8a7b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a8ad68740 4 client.25030 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T00:59:37.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a8ad68740 1 -- 172.21.15.78:0/191145183 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d8a69f8a0 con 0x561d8a7b5bf0 2024-09-11T00:59:37.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a8259c640 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561d8a7b5bf0 0x561d8a7b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:37.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a69ffb640 10 client.25030 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T00:59:37.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a8259c640 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561d8a7b5bf0 0x561d8a7b8010 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=305548189c8c1575 server_cookie=7afcedad7abe430d in_seq=0 out_seq=0 2024-09-11T00:59:37.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a69ffb640 1 -- 172.21.15.78:0/191145183 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d8a69f8a0 con 0x561d8a7b5bf0 2024-09-11T00:59:37.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a69ffb640 10 client.25030 handle_command_reply: tid=0 2024-09-11T00:59:37.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a8ad68740 1 client.25030 shutdown 2024-09-11T00:59:37.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a8ad68740 1 -- 172.21.15.78:0/191145183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a737d80 msgr2=0x561d8a73c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:37.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.076+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a737d80 0x561d8a73c7b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0a78005520 tx=0x7f0a780044f0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.083 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T00:59:37.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a8ad68740 1 -- 172.21.15.78:0/191145183 shutdown_connections 2024-09-11T00:59:37.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561d8a7b5bf0 0x561d8a7b8010 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a69ffb640 0 client.25030 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T00:59:37.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561d8a747810 0x561d8a73ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d8a737d80 0x561d8a73c7b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a8ad68740 1 --2- 172.21.15.78:0/191145183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d8a72bdf0 0x561d8a7402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a8ad68740 1 -- 172.21.15.78:0/191145183 >> 172.21.15.78:0/191145183 conn(0x561d8a729300 msgr2=0x561d8a7367f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:37.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a8ad68740 1 -- 172.21.15.78:0/191145183 shutdown_connections 2024-09-11T00:59:37.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a8ad68740 1 -- 172.21.15.78:0/191145183 wait complete. 2024-09-11T00:59:37.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a8ad68740 20 client.25030 trim_cache size 0 max 16384 2024-09-11T00:59:37.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a88993640 1 -- 172.21.15.78:0/1090766569 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a6c0b54d0 msgr2=0x7f0a6c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:37.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a88993640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a6c0b54d0 0x7f0a6c0b7990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a78000c00 tx=0x7f0a78005d50 comp rx=0 tx=0).stop 2024-09-11T00:59:37.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a88993640 1 -- 172.21.15.78:0/1090766569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a84101c90 msgr2=0x7f0a8419b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:37.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a88993640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a84101c90 0x7f0a8419b6f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400e9f0 tx=0x7f0a7400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a88993640 1 -- 172.21.15.78:0/1090766569 shutdown_connections 2024-09-11T00:59:37.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a88993640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a6c0b54d0 0x7f0a6c0b7990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a88993640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a84108020 0x7f0a841a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a88993640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a841076a0 0x7f0a8419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a88993640 1 --2- 172.21.15.78:0/1090766569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a84101c90 0x7f0a8419b6f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:37.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.077+0000 7f0a88993640 1 -- 172.21.15.78:0/1090766569 >> 172.21.15.78:0/1090766569 conn(0x7f0a840fd700 msgr2=0x7f0a84106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:37.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.078+0000 7f0a88993640 1 -- 172.21.15.78:0/1090766569 shutdown_connections 2024-09-11T00:59:37.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:37.078+0000 7f0a88993640 1 -- 172.21.15.78:0/1090766569 wait complete. 2024-09-11T00:59:37.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:37 smithi078 ceph-mon[31315]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:59:37.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:37 smithi110 ceph-mon[34712]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:59:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:37 smithi112 ceph-mon[35384]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:59:37.825 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T00:59:38.174 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:59:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.728+0000 7f92d4b84640 1 -- 172.21.15.78:0/2299773416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92d0101cb0 msgr2=0x7f92d0102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.728+0000 7f92d4b84640 1 --2- 172.21.15.78:0/2299773416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92d0101cb0 0x7f92d0102090 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009900 tx=0x7f92c002a410 comp rx=0 tx=0).stop 2024-09-11T00:59:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.728+0000 7f92d4b84640 1 -- 172.21.15.78:0/2299773416 shutdown_connections 2024-09-11T00:59:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.728+0000 7f92d4b84640 1 --2- 172.21.15.78:0/2299773416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d0108040 0x7f92d010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.728+0000 7f92d4b84640 1 --2- 172.21.15.78:0/2299773416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92d01076c0 0x7f92d0107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.728+0000 7f92d4b84640 1 --2- 172.21.15.78:0/2299773416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92d0101cb0 0x7f92d0102090 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.728+0000 7f92d4b84640 1 -- 172.21.15.78:0/2299773416 >> 172.21.15.78:0/2299773416 conn(0x7f92d00fd740 msgr2=0x7f92d00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.729+0000 7f92d4b84640 1 -- 172.21.15.78:0/2299773416 shutdown_connections 2024-09-11T00:59:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.729+0000 7f92d4b84640 1 -- 172.21.15.78:0/2299773416 wait complete. 2024-09-11T00:59:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.729+0000 7f92d4b84640 1 Processor -- start 2024-09-11T00:59:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92d4b84640 1 -- start start 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92d4b84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92d0101cb0 0x7f92d0199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92d4b84640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d01076c0 0x7f92d0199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92d4b84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92d0108040 0x7f92d019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92d4b84640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0111d00 con 0x7f92d0108040 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92d4b84640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0111b80 con 0x7f92d01076c0 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92d4b84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0111e80 con 0x7f92d0101cb0 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92ced9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92d0108040 0x7f92d019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92ced9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92d0108040 0x7f92d019ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33034/0 (socket says 172.21.15.78:33034) 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92ced9d640 1 -- 172.21.15.78:0/941929827 learned_addr learned my addr 172.21.15.78:0/941929827 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92c5d9b640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d01076c0 0x7f92d0199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92ce59c640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92d0101cb0 0x7f92d0199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92ced9d640 1 -- 172.21.15.78:0/941929827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92d0101cb0 msgr2=0x7f92d0199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92ced9d640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92d0101cb0 0x7f92d0199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92ced9d640 1 -- 172.21.15.78:0/941929827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d01076c0 msgr2=0x7f92d0199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92ced9d640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d01076c0 0x7f92d0199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92ced9d640 1 -- 172.21.15.78:0/941929827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d019f3f0 con 0x7f92d0108040 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92ce59c640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92d0101cb0 0x7f92d0199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.730+0000 7f92c5d9b640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d01076c0 0x7f92d0199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.731+0000 7f92ced9d640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92d0108040 0x7f92d019ec20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f92c800cad0 tx=0x7f92c800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.731+0000 7f92c7fff640 1 -- 172.21.15.78:0/941929827 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c800ae20 con 0x7f92d0108040 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.731+0000 7f92d4b84640 1 -- 172.21.15.78:0/941929827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d019f6e0 con 0x7f92d0108040 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.731+0000 7f92c7fff640 1 -- 172.21.15.78:0/941929827 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c8005020 con 0x7f92d0108040 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.731+0000 7f92c7fff640 1 -- 172.21.15.78:0/941929827 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c800f6b0 con 0x7f92d0108040 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.731+0000 7f92d4b84640 1 -- 172.21.15.78:0/941929827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d01a5da0 con 0x7f92d0108040 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.734+0000 7f92d6f59740 1 Processor -- start 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.735+0000 7f92c7fff640 1 -- 172.21.15.78:0/941929827 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f92c80054f0 con 0x7f92d0108040 2024-09-11T00:59:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.735+0000 7f92d6f59740 1 -- start start 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.735+0000 7f92d6f59740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341c4d0 0x555a7341c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.735+0000 7f92d6f59740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a73428840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.735+0000 7f92d6f59740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a73428d80 0x555a7342b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.735+0000 7f92d6f59740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a73392ea0 con 0x555a7341c4d0 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.735+0000 7f92d6f59740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555a733822d0 con 0x555a73428d80 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.735+0000 7f92d6f59740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555a733860e0 con 0x555a7341cdf0 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.735+0000 7f92c5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341c4d0 0x555a7341c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.735+0000 7f92c5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341c4d0 0x555a7341c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33050/0 (socket says 172.21.15.78:33050) 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.735+0000 7f92c5d9b640 1 -- 172.21.15.78:0/444389388 learned_addr learned my addr 172.21.15.78:0/444389388 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.736+0000 7f92ce59c640 1 --2- 172.21.15.78:0/444389388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a73428840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.736+0000 7f92c5d9b640 1 --2- 172.21.15.78:0/444389388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a73428d80 0x555a7342b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.736+0000 7f92c5d9b640 1 -- 172.21.15.78:0/444389388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 msgr2=0x555a73428840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.736+0000 7f92c5d9b640 1 --2- 172.21.15.78:0/444389388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a73428840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.736+0000 7f92c5d9b640 1 -- 172.21.15.78:0/444389388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a73428d80 msgr2=0x555a7342b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.736+0000 7f92c5d9b640 1 --2- 172.21.15.78:0/444389388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a73428d80 0x555a7342b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.736+0000 7f92c5d9b640 1 -- 172.21.15.78:0/444389388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a730e0650 con 0x555a7341c4d0 2024-09-11T00:59:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.736+0000 7f92c7fff640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f92bc0b5450 0x7f92bc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.736+0000 7f92ce59c640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f92bc0b5450 0x7f92bc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.736+0000 7f92c7fff640 1 -- 172.21.15.78:0/941929827 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f92c80e6000 con 0x7f92d0108040 2024-09-11T00:59:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.736+0000 7f92c5d9b640 1 --2- 172.21.15.78:0/444389388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341c4d0 0x555a7341c8b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f92b800bd60 tx=0x7f92b800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e43b4c6e5168468 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.737+0000 7f92c4d99640 1 -- 172.21.15.78:0/444389388 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8011740 con 0x555a7341c4d0 2024-09-11T00:59:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.737+0000 7f92ce59c640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f92bc0b5450 0x7f92bc0b7910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f92c0031440 tx=0x7f92c00058b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.737+0000 7f92c4d99640 1 -- 172.21.15.78:0/444389388 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92b8020070 con 0x555a7341c4d0 2024-09-11T00:59:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.737+0000 7f92c4d99640 1 -- 172.21.15.78:0/444389388 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b801baf0 con 0x555a7341c4d0 2024-09-11T00:59:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.737+0000 7f92d6f59740 1 -- 172.21.15.78:0/444389388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341c4d0 msgr2=0x555a7341c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.737+0000 7f92d6f59740 1 --2- 172.21.15.78:0/444389388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341c4d0 0x555a7341c8b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f92b800bd60 tx=0x7f92b800b730 comp rx=0 tx=0).stop 2024-09-11T00:59:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 -- 172.21.15.78:0/444389388 shutdown_connections 2024-09-11T00:59:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 --2- 172.21.15.78:0/444389388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a73428d80 0x555a7342b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 --2- 172.21.15.78:0/444389388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a73428840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 --2- 172.21.15.78:0/444389388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341c4d0 0x555a7341c8b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 -- 172.21.15.78:0/444389388 >> 172.21.15.78:0/444389388 conn(0x555a7341a300 msgr2=0x555a7341b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 -- 172.21.15.78:0/444389388 shutdown_connections 2024-09-11T00:59:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 -- 172.21.15.78:0/444389388 wait complete. 2024-09-11T00:59:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 -- start start 2024-09-11T00:59:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 Processor -- start 2024-09-11T00:59:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a7341c4d0 0x555a73428750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a73425ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 0x555a73426290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a731e3700 con 0x555a73428d80 2024-09-11T00:59:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.738+0000 7f92d6f59740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555a7331d160 con 0x555a7341c4d0 2024-09-11T00:59:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92d6f59740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555a7342ba00 con 0x555a7341cdf0 2024-09-11T00:59:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92d6f59740 10 client.0 _ll_register_callbacks cb 0x555a733369e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92ced9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 0x555a73426290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92ced9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 0x555a73426290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33054/0 (socket says 172.21.15.78:33054) 2024-09-11T00:59:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92ced9d640 1 -- 172.21.15.78:0/356829537 learned_addr learned my addr 172.21.15.78:0/356829537 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92ce59c640 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a7341c4d0 0x555a73428750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92c5d9b640 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a73425ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92ced9d640 1 -- 172.21.15.78:0/356829537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 msgr2=0x555a73425ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92ced9d640 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a73425ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92ced9d640 1 -- 172.21.15.78:0/356829537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a7341c4d0 msgr2=0x555a73428750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92ced9d640 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a7341c4d0 0x555a73428750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92ced9d640 1 -- 172.21.15.78:0/356829537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a7334ac90 con 0x555a73428d80 2024-09-11T00:59:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92c5d9b640 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a73425ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:59:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92ced9d640 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 0x555a73426290 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f92c80ac480 tx=0x7f92c80abd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a5c35deae50102 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:59:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92b5ffb640 1 -- 172.21.15.78:0/356829537 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c801d030 con 0x555a73428d80 2024-09-11T00:59:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.739+0000 7f92b5ffb640 1 -- 172.21.15.78:0/356829537 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f92c80ab950 con 0x555a73428d80 2024-09-11T00:59:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.740+0000 7f92d6f59740 1 -- 172.21.15.78:0/356829537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92b8020070 con 0x555a73428d80 2024-09-11T00:59:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.740+0000 7f92b5ffb640 1 -- 172.21.15.78:0/356829537 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92c80b0fd0 con 0x555a73428d80 2024-09-11T00:59:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.740+0000 7f92d6f59740 10 client.15612 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.740+0000 7f92d6f59740 1 -- 172.21.15.78:0/356829537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555a7343fc30 con 0x555a73428d80 2024-09-11T00:59:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.740+0000 7f92b5ffb640 1 -- 172.21.15.78:0/356829537 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92b8020070 con 0x555a73428d80 2024-09-11T00:59:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.740+0000 7f92b5ffb640 1 -- 172.21.15.78:0/356829537 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92c80d30d0 con 0x555a73428d80 2024-09-11T00:59:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.740+0000 7f92d6f59740 10 client.15612 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.740+0000 7f92d6f59740 10 client.15612 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T00:59:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.740+0000 7f92d6f59740 20 client.15612 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.740+0000 7f92d6f59740 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555a73441090 0x555a73443480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.740+0000 7f92d6f59740 4 client.15612 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:59:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.740+0000 7f92d6f59740 1 -- 172.21.15.78:0/356829537 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a73394df0 con 0x555a73441090 2024-09-11T00:59:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.741+0000 7f92ce59c640 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555a73441090 0x555a73443480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.741+0000 7f92ce59c640 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555a73441090 0x555a73443480 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2225f24ec9f6dc84 server_cookie=f6f43d785e123dc8 in_seq=0 out_seq=0 2024-09-11T00:59:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.741+0000 7f92b5ffb640 10 client.15612 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T00:59:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92b5ffb640 1 -- 172.21.15.78:0/356829537 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a73394df0 con 0x555a73441090 2024-09-11T00:59:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92b5ffb640 10 client.15612 handle_command_reply: tid=0 2024-09-11T00:59:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92d6f59740 1 client.15612 shutdown 2024-09-11T00:59:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92d6f59740 1 -- 172.21.15.78:0/356829537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 msgr2=0x555a73426290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92d6f59740 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 0x555a73426290 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f92c80ac480 tx=0x7f92c80abd80 comp rx=0 tx=0).stop 2024-09-11T00:59:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92d6f59740 1 -- 172.21.15.78:0/356829537 shutdown_connections 2024-09-11T00:59:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92d6f59740 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 0x555a73426290 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92d6f59740 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a73425ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92d6f59740 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555a73441090 0x555a73443480 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92d6f59740 1 --2- 172.21.15.78:0/356829537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a7341c4d0 0x555a73428750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92b5ffb640 0 client.15612 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T00:59:38.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92d6f59740 1 -- 172.21.15.78:0/356829537 >> 172.21.15.78:0/356829537 conn(0x555a7341a300 msgr2=0x555a7341fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:38.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92d6f59740 1 -- 172.21.15.78:0/356829537 shutdown_connections 2024-09-11T00:59:38.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.742+0000 7f92d6f59740 1 -- 172.21.15.78:0/356829537 wait complete. 2024-09-11T00:59:38.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.743+0000 7f92d6f59740 20 client.15612 trim_cache size 0 max 16384 2024-09-11T00:59:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92d6f59740 1 Processor -- start 2024-09-11T00:59:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92d6f59740 1 -- start start 2024-09-11T00:59:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92d6f59740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a7341c4d0 0x555a7343d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92d6f59740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a7342a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92d6f59740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 0x555a7342a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92d6f59740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01ac400 con 0x555a73428d80 2024-09-11T00:59:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92d6f59740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01adf40 con 0x555a7341c4d0 2024-09-11T00:59:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92d6f59740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01acad0 con 0x555a7341cdf0 2024-09-11T00:59:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92ced9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 0x555a7342a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92ced9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 0x555a7342a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33068/0 (socket says 172.21.15.78:33068) 2024-09-11T00:59:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92ced9d640 1 -- 172.21.15.78:0/1064213279 learned_addr learned my addr 172.21.15.78:0/1064213279 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92ce59c640 1 --2- 172.21.15.78:0/1064213279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a7341c4d0 0x555a7343d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.756+0000 7f92c5d9b640 1 --2- 172.21.15.78:0/1064213279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a7342a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.757+0000 7f92ce59c640 1 -- 172.21.15.78:0/1064213279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 msgr2=0x555a7342a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.757+0000 7f92ce59c640 1 --2- 172.21.15.78:0/1064213279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a7342a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.757+0000 7f92ce59c640 1 -- 172.21.15.78:0/1064213279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 msgr2=0x555a7342a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.757+0000 7f92ce59c640 1 --2- 172.21.15.78:0/1064213279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 0x555a7342a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.757+0000 7f92ce59c640 1 -- 172.21.15.78:0/1064213279 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a7341cc20 con 0x555a7341c4d0 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.757+0000 7f92ced9d640 1 --2- 172.21.15.78:0/1064213279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 0x555a7342a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.757+0000 7f92ce59c640 1 --2- 172.21.15.78:0/1064213279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a7341c4d0 0x555a7343d4c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009230 tx=0x7f92c003e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25542241d7da1505 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.757+0000 7f92b7fff640 1 -- 172.21.15.78:0/1064213279 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0056070 con 0x555a7341c4d0 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.757+0000 7f92b7fff640 1 -- 172.21.15.78:0/1064213279 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c0004380 con 0x555a7341c4d0 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.757+0000 7f92b7fff640 1 -- 172.21.15.78:0/1064213279 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0035040 con 0x555a7341c4d0 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.758+0000 7f92d6f59740 1 -- 172.21.15.78:0/1064213279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a7341c4d0 msgr2=0x555a7343d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.758+0000 7f92d6f59740 1 --2- 172.21.15.78:0/1064213279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a7341c4d0 0x555a7343d4c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009230 tx=0x7f92c003e000 comp rx=0 tx=0).stop 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.758+0000 7f92d6f59740 1 -- 172.21.15.78:0/1064213279 shutdown_connections 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.758+0000 7f92d6f59740 1 --2- 172.21.15.78:0/1064213279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a73428d80 0x555a7342a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.758+0000 7f92d6f59740 1 --2- 172.21.15.78:0/1064213279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341cdf0 0x555a7342a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.758+0000 7f92d6f59740 1 --2- 172.21.15.78:0/1064213279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a7341c4d0 0x555a7343d4c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.758+0000 7f92d6f59740 1 -- 172.21.15.78:0/1064213279 >> 172.21.15.78:0/1064213279 conn(0x555a7341a300 msgr2=0x555a7341b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.758+0000 7f92d6f59740 1 -- 172.21.15.78:0/1064213279 shutdown_connections 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.758+0000 7f92d6f59740 1 -- 172.21.15.78:0/1064213279 wait complete. 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.758+0000 7f92d6f59740 1 -- start start 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.758+0000 7f92d6f59740 1 Processor -- start 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92d6f59740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341c4d0 0x555a73439710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92d6f59740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341cdf0 0x555a73439c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92d6f59740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a73428d80 0x555a734344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92d6f59740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01ad870 con 0x555a7341cdf0 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92d6f59740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01ad1a0 con 0x555a73428d80 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92d6f59740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01aa3e0 con 0x555a7341c4d0 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92d6f59740 10 client.0 _ll_register_callbacks cb 0x7f92c80a8bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92c5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341cdf0 0x555a73439c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92ce59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341c4d0 0x555a73439710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92c5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341cdf0 0x555a73439c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33078/0 (socket says 172.21.15.78:33078) 2024-09-11T00:59:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92ce59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341c4d0 0x555a73439710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:39026/0 (socket says 172.21.15.78:39026) 2024-09-11T00:59:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92c5d9b640 1 -- 172.21.15.78:0/454052685 learned_addr learned my addr 172.21.15.78:0/454052685 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92ced9d640 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a73428d80 0x555a734344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92c5d9b640 1 -- 172.21.15.78:0/454052685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341c4d0 msgr2=0x555a73439710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92c5d9b640 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341c4d0 0x555a73439710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92c5d9b640 1 -- 172.21.15.78:0/454052685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a73428d80 msgr2=0x555a734344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92c5d9b640 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a73428d80 0x555a734344e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92c5d9b640 1 -- 172.21.15.78:0/454052685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a73434bd0 con 0x555a7341cdf0 2024-09-11T00:59:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.759+0000 7f92ce59c640 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341c4d0 0x555a73439710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:59:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.760+0000 7f92c5d9b640 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341cdf0 0x555a73439c50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f92b800b630 tx=0x7f92b800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4246fb9b7b84655 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.760+0000 7f92b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:59:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.760+0000 7f92b5ffb640 1 -- 172.21.15.78:0/454052685 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8013240 con 0x555a7341cdf0 2024-09-11T00:59:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.760+0000 7f92b5ffb640 1 -- 172.21.15.78:0/454052685 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f92b8005030 con 0x555a7341cdf0 2024-09-11T00:59:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.760+0000 7f92d6f59740 1 -- 172.21.15.78:0/454052685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c0004380 con 0x555a7341cdf0 2024-09-11T00:59:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.760+0000 7f92b5ffb640 1 -- 172.21.15.78:0/454052685 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92b80133e0 con 0x555a7341cdf0 2024-09-11T00:59:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.760+0000 7f92d6f59740 10 client.15618 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.760+0000 7f92d6f59740 1 -- 172.21.15.78:0/454052685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555a734ae930 con 0x555a7341cdf0 2024-09-11T00:59:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.761+0000 7f92b5ffb640 1 -- 172.21.15.78:0/454052685 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92b8020040 con 0x555a7341cdf0 2024-09-11T00:59:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.761+0000 7f92b5ffb640 1 -- 172.21.15.78:0/454052685 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92b8004680 con 0x555a7341cdf0 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.761+0000 7f92d6f59740 10 client.15618 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.761+0000 7f92d6f59740 10 client.15618 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.761+0000 7f92d6f59740 20 client.15618 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.761+0000 7f92d6f59740 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555a734aee70 0x555a734b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.761+0000 7f92d6f59740 4 client.15618 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.761+0000 7f92d6f59740 1 -- 172.21.15.78:0/454052685 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f92c0004380 con 0x555a734aee70 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.761+0000 7f92ce59c640 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555a734aee70 0x555a734b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.762+0000 7f92ce59c640 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555a734aee70 0x555a734b1290 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aaa2cc2ca18c8695 server_cookie=c0abf82d31b81a21 in_seq=0 out_seq=0 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.762+0000 7f92b5ffb640 10 client.15618 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.762+0000 7f92b5ffb640 1 -- 172.21.15.78:0/454052685 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f92c0004380 con 0x555a734aee70 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.762+0000 7f92b5ffb640 10 client.15618 handle_command_reply: tid=0 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.762+0000 7f92d6f59740 1 client.15618 shutdown 2024-09-11T00:59:38.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.763+0000 7f92d6f59740 1 -- 172.21.15.78:0/454052685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341cdf0 msgr2=0x555a73439c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:38.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.763+0000 7f92d6f59740 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341cdf0 0x555a73439c50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f92b800b630 tx=0x7f92b800be00 comp rx=0 tx=0).stop 2024-09-11T00:59:38.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.763+0000 7f92d6f59740 1 -- 172.21.15.78:0/454052685 shutdown_connections 2024-09-11T00:59:38.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.763+0000 7f92d6f59740 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555a734aee70 0x555a734b1290 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.763+0000 7f92d6f59740 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a73428d80 0x555a734344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.763+0000 7f92d6f59740 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a7341cdf0 0x555a73439c50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.763+0000 7f92d6f59740 1 --2- 172.21.15.78:0/454052685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555a7341c4d0 0x555a73439710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.763+0000 7f92d6f59740 1 -- 172.21.15.78:0/454052685 >> 172.21.15.78:0/454052685 conn(0x555a7341a300 msgr2=0x555a734271d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:38.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.763+0000 7f92b5ffb640 0 client.15618 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T00:59:38.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.763+0000 7f92d6f59740 1 -- 172.21.15.78:0/454052685 shutdown_connections 2024-09-11T00:59:38.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.763+0000 7f92d6f59740 1 -- 172.21.15.78:0/454052685 wait complete. 2024-09-11T00:59:38.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.763+0000 7f92d6f59740 20 client.15618 trim_cache size 0 max 16384 2024-09-11T00:59:38.776 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T00:59:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.764+0000 7f92d4b84640 1 -- 172.21.15.78:0/941929827 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f92bc0b5450 msgr2=0x7f92bc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.764+0000 7f92d4b84640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f92bc0b5450 0x7f92bc0b7910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f92c0031440 tx=0x7f92c00058b0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.764+0000 7f92d4b84640 1 -- 172.21.15.78:0/941929827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92d0108040 msgr2=0x7f92d019ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.764+0000 7f92d4b84640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92d0108040 0x7f92d019ec20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f92c800cad0 tx=0x7f92c800cf90 comp rx=0 tx=0).stop 2024-09-11T00:59:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.764+0000 7f92d4b84640 1 -- 172.21.15.78:0/941929827 shutdown_connections 2024-09-11T00:59:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.764+0000 7f92d4b84640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f92bc0b5450 0x7f92bc0b7910 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.764+0000 7f92d4b84640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92d0108040 0x7f92d019ec20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.764+0000 7f92d4b84640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d01076c0 0x7f92d0199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.764+0000 7f92d4b84640 1 --2- 172.21.15.78:0/941929827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92d0101cb0 0x7f92d0199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.764+0000 7f92d4b84640 1 -- 172.21.15.78:0/941929827 >> 172.21.15.78:0/941929827 conn(0x7f92d00fd740 msgr2=0x7f92d006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.764+0000 7f92d4b84640 1 -- 172.21.15.78:0/941929827 shutdown_connections 2024-09-11T00:59:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:38.764+0000 7f92d4b84640 1 -- 172.21.15.78:0/941929827 wait complete. 2024-09-11T00:59:39.359 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:39 smithi078 ceph-mon[31315]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:59:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:39 smithi110 ceph-mon[34712]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:59:39.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:39 smithi112 ceph-mon[35384]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:59:39.626 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T00:59:39.942 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:59:40.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.503+0000 7fdf9bd66640 1 -- 172.21.15.78:0/1036236731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf94109150 msgr2=0x7fdf9410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:40.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.503+0000 7fdf9bd66640 1 --2- 172.21.15.78:0/1036236731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf94109150 0x7fdf9410f950 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009ca0 tx=0x7fdf8c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.504+0000 7fdf9bd66640 1 -- 172.21.15.78:0/1036236731 shutdown_connections 2024-09-11T00:59:40.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.504+0000 7fdf9bd66640 1 --2- 172.21.15.78:0/1036236731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf94109150 0x7fdf9410f950 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.504+0000 7fdf9bd66640 1 --2- 172.21.15.78:0/1036236731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf941087d0 0x7fdf94108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.504+0000 7fdf9bd66640 1 --2- 172.21.15.78:0/1036236731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf94102dc0 0x7fdf941031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.504+0000 7fdf9bd66640 1 -- 172.21.15.78:0/1036236731 >> 172.21.15.78:0/1036236731 conn(0x7fdf940fe870 msgr2=0x7fdf94100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:40.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.504+0000 7fdf9bd66640 1 -- 172.21.15.78:0/1036236731 shutdown_connections 2024-09-11T00:59:40.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.504+0000 7fdf9bd66640 1 -- 172.21.15.78:0/1036236731 wait complete. 2024-09-11T00:59:40.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.505+0000 7fdf9bd66640 1 Processor -- start 2024-09-11T00:59:40.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.505+0000 7fdf9bd66640 1 -- start start 2024-09-11T00:59:40.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.505+0000 7fdf9bd66640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf94102dc0 0x7fdf940754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.505+0000 7fdf9bd66640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf941087d0 0x7fdf940759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.505+0000 7fdf9bd66640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf94109150 0x7fdf94075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.505+0000 7fdf9bd66640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94112e10 con 0x7fdf94102dc0 2024-09-11T00:59:40.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.505+0000 7fdf9bd66640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94112c90 con 0x7fdf941087d0 2024-09-11T00:59:40.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.505+0000 7fdf9bd66640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94112f90 con 0x7fdf94109150 2024-09-11T00:59:40.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.505+0000 7fdf99b02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf94102dc0 0x7fdf940754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.505+0000 7fdf99b02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf94102dc0 0x7fdf940754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33100/0 (socket says 172.21.15.78:33100) 2024-09-11T00:59:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.505+0000 7fdf99b02640 1 -- 172.21.15.78:0/2474362359 learned_addr learned my addr 172.21.15.78:0/2474362359 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf99301640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf941087d0 0x7fdf940759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf99b02640 1 -- 172.21.15.78:0/2474362359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf94109150 msgr2=0x7fdf94075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf9a303640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf94109150 0x7fdf94075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf99b02640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf94109150 0x7fdf94075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf99b02640 1 -- 172.21.15.78:0/2474362359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf941087d0 msgr2=0x7fdf940759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf99b02640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf941087d0 0x7fdf940759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf99b02640 1 -- 172.21.15.78:0/2474362359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf941a75c0 con 0x7fdf94102dc0 2024-09-11T00:59:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf99301640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf941087d0 0x7fdf940759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:59:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf9a303640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf94109150 0x7fdf94075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:59:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf99b02640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf94102dc0 0x7fdf940754a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000c970 tx=0x7fdf9000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf8affd640 1 -- 172.21.15.78:0/2474362359 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90007bb0 con 0x7fdf94102dc0 2024-09-11T00:59:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf9bd66640 1 -- 172.21.15.78:0/2474362359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf941a78b0 con 0x7fdf94102dc0 2024-09-11T00:59:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf8affd640 1 -- 172.21.15.78:0/2474362359 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf90007d50 con 0x7fdf94102dc0 2024-09-11T00:59:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf8affd640 1 -- 172.21.15.78:0/2474362359 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000f450 con 0x7fdf94102dc0 2024-09-11T00:59:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.506+0000 7fdf9bd66640 1 -- 172.21.15.78:0/2474362359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf941a7df0 con 0x7fdf94102dc0 2024-09-11T00:59:40.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf9e13b740 1 Processor -- start 2024-09-11T00:59:40.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf9e13b740 1 -- start start 2024-09-11T00:59:40.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf8affd640 1 -- 172.21.15.78:0/2474362359 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fdf90004f60 con 0x7fdf94102dc0 2024-09-11T00:59:40.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf9e13b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c584d0 0x55cfa4c588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf9e13b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c64840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf9e13b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c64d80 0x55cfa4c67170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf9e13b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cfa4bceea0 con 0x55cfa4c58df0 2024-09-11T00:59:40.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf9e13b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cfa4bbe2d0 con 0x55cfa4c64d80 2024-09-11T00:59:40.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf9e13b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cfa4bc20e0 con 0x55cfa4c584d0 2024-09-11T00:59:40.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf99b02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c64840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf99b02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c64840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33116/0 (socket says 172.21.15.78:33116) 2024-09-11T00:59:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf99b02640 1 -- 172.21.15.78:0/3339095150 learned_addr learned my addr 172.21.15.78:0/3339095150 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.510+0000 7fdf9a303640 1 --2- 172.21.15.78:0/3339095150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c584d0 0x55cfa4c588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.511+0000 7fdf99301640 1 --2- 172.21.15.78:0/3339095150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c64d80 0x55cfa4c67170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.511+0000 7fdf99b02640 1 -- 172.21.15.78:0/3339095150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c584d0 msgr2=0x55cfa4c588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.511+0000 7fdf99b02640 1 --2- 172.21.15.78:0/3339095150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c584d0 0x55cfa4c588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.511+0000 7fdf99b02640 1 -- 172.21.15.78:0/3339095150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c64d80 msgr2=0x55cfa4c67170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.511+0000 7fdf99b02640 1 --2- 172.21.15.78:0/3339095150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c64d80 0x55cfa4c67170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.511+0000 7fdf99b02640 1 -- 172.21.15.78:0/3339095150 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa491c650 con 0x55cfa4c58df0 2024-09-11T00:59:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.511+0000 7fdf99301640 1 --2- 172.21.15.78:0/3339095150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c64d80 0x55cfa4c67170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:59:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.511+0000 7fdf99b02640 1 --2- 172.21.15.78:0/3339095150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c64840 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdf900e3430 tx=0x7fdf900ec880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809aadc153a24c3f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.511+0000 7fdf8affd640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdf800b54d0 0x7fdf800b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.511+0000 7fdf7ffff640 1 -- 172.21.15.78:0/3339095150 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900fb070 con 0x55cfa4c58df0 2024-09-11T00:59:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.511+0000 7fdf7ffff640 1 -- 172.21.15.78:0/3339095150 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf900abd00 con 0x55cfa4c58df0 2024-09-11T00:59:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.511+0000 7fdf7ffff640 1 -- 172.21.15.78:0/3339095150 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900ab960 con 0x55cfa4c58df0 2024-09-11T00:59:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.512+0000 7fdf9a303640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdf800b54d0 0x7fdf800b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.512+0000 7fdf8affd640 1 -- 172.21.15.78:0/2474362359 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdf900e5d00 con 0x7fdf94102dc0 2024-09-11T00:59:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.512+0000 7fdf9e13b740 1 -- 172.21.15.78:0/3339095150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 msgr2=0x55cfa4c64840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.512+0000 7fdf9a303640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdf800b54d0 0x7fdf800b7990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009430 tx=0x7fdf8c009380 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.512+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/3339095150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c64840 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdf900e3430 tx=0x7fdf900ec880 comp rx=0 tx=0).stop 2024-09-11T00:59:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.512+0000 7fdf9e13b740 1 -- 172.21.15.78:0/3339095150 shutdown_connections 2024-09-11T00:59:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.512+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/3339095150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c64d80 0x55cfa4c67170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.512+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/3339095150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c64840 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.512+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/3339095150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c584d0 0x55cfa4c588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.512+0000 7fdf9e13b740 1 -- 172.21.15.78:0/3339095150 >> 172.21.15.78:0/3339095150 conn(0x55cfa4c56300 msgr2=0x55cfa4c574a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.513+0000 7fdf9e13b740 1 -- 172.21.15.78:0/3339095150 shutdown_connections 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.513+0000 7fdf9e13b740 1 -- 172.21.15.78:0/3339095150 wait complete. 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.513+0000 7fdf9e13b740 1 -- start start 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.513+0000 7fdf9e13b740 1 Processor -- start 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.514+0000 7fdf9e13b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c584d0 0x55cfa4c645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.514+0000 7fdf9e13b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c58df0 0x55cfa4c61a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.514+0000 7fdf9e13b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c64d80 0x55cfa4c61fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.514+0000 7fdf99b02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c584d0 0x55cfa4c645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.514+0000 7fdf99b02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c584d0 0x55cfa4c645f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33130/0 (socket says 172.21.15.78:33130) 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.514+0000 7fdf99b02640 1 -- 172.21.15.78:0/2361307708 learned_addr learned my addr 172.21.15.78:0/2361307708 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.514+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2361307708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cfa4a1f700 con 0x55cfa4c584d0 2024-09-11T00:59:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.514+0000 7fdf99301640 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c58df0 0x55cfa4c61a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.514+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2361307708 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cfa4b59160 con 0x55cfa4c58df0 2024-09-11T00:59:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.514+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2361307708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cfa4c67a00 con 0x55cfa4c64d80 2024-09-11T00:59:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.514+0000 7fdf9a303640 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c64d80 0x55cfa4c61fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.514+0000 7fdf9e13b740 10 client.0 _ll_register_callbacks cb 0x55cfa4b729e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf9a303640 1 -- 172.21.15.78:0/2361307708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c58df0 msgr2=0x55cfa4c61a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf9a303640 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c58df0 0x55cfa4c61a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf9a303640 1 -- 172.21.15.78:0/2361307708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c584d0 msgr2=0x55cfa4c645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf9a303640 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c584d0 0x55cfa4c645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf9a303640 1 -- 172.21.15.78:0/2361307708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa4b86c90 con 0x55cfa4c64d80 2024-09-11T00:59:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf99301640 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c58df0 0x55cfa4c61a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:59:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2361307708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf900abd00 con 0x55cfa4c64d80 2024-09-11T00:59:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf9a303640 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c64d80 0x55cfa4c61fc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdf94077560 tx=0x7fdf8c02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1216935547ebab6c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf7d7fa640 10 client.25054 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf7d7fa640 1 -- 172.21.15.78:0/2361307708 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c004310 con 0x55cfa4c64d80 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf7d7fa640 1 -- 172.21.15.78:0/2361307708 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdf8c0044b0 con 0x55cfa4c64d80 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf7d7fa640 1 -- 172.21.15.78:0/2361307708 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf8c0046d0 con 0x55cfa4c64d80 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf9e13b740 10 client.25054 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.515+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2361307708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55cfa4c731c0 con 0x55cfa4c64d80 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.516+0000 7fdf7d7fa640 1 -- 172.21.15.78:0/2361307708 <== mon.1 v2:172.21.15.112:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf8c050070 con 0x55cfa4c64d80 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.516+0000 7fdf9e13b740 10 client.25054 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.516+0000 7fdf9e13b740 10 client.25054 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.516+0000 7fdf9e13b740 20 client.25054 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.516+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cfa4c748f0 0x55cfa4c76db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.516+0000 7fdf9e13b740 4 client.25054 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.516+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2361307708 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfa4bd0df0 con 0x55cfa4c748f0 2024-09-11T00:59:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.516+0000 7fdf99b02640 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cfa4c748f0 0x55cfa4c76db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.516+0000 7fdf7d7fa640 10 client.25054 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T00:59:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.516+0000 7fdf99b02640 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cfa4c748f0 0x55cfa4c76db0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bad3ee237421cd97 server_cookie=69a1b752498fa84c in_seq=0 out_seq=0 2024-09-11T00:59:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.517+0000 7fdf7d7fa640 1 -- 172.21.15.78:0/2361307708 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cfa4bd0df0 con 0x55cfa4c748f0 2024-09-11T00:59:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.517+0000 7fdf7d7fa640 10 client.25054 handle_command_reply: tid=0 2024-09-11T00:59:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.517+0000 7fdf9e13b740 1 client.25054 shutdown 2024-09-11T00:59:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.517+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2361307708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c64d80 msgr2=0x55cfa4c61fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.517+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c64d80 0x55cfa4c61fc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdf94077560 tx=0x7fdf8c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.517+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2361307708 shutdown_connections 2024-09-11T00:59:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.517+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c64d80 0x55cfa4c61fc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.517+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c58df0 0x55cfa4c61a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.517+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cfa4c748f0 0x55cfa4c76db0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.517+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/2361307708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c584d0 0x55cfa4c645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.517+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2361307708 >> 172.21.15.78:0/2361307708 conn(0x55cfa4c56300 msgr2=0x55cfa4c5bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.517+0000 7fdf7d7fa640 0 client.25054 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T00:59:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.518+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2361307708 shutdown_connections 2024-09-11T00:59:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.518+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2361307708 wait complete. 2024-09-11T00:59:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.518+0000 7fdf9e13b740 20 client.25054 trim_cache size 0 max 16384 2024-09-11T00:59:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf9e13b740 1 Processor -- start 2024-09-11T00:59:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf9e13b740 1 -- start start 2024-09-11T00:59:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf9e13b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c584d0 0x55cfa4c70c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf9e13b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c55310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf9e13b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c64d80 0x55cfa4c5ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf9e13b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941b0690 con 0x55cfa4c58df0 2024-09-11T00:59:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf9e13b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941b21d0 con 0x55cfa4c584d0 2024-09-11T00:59:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf9e13b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941b0d60 con 0x55cfa4c64d80 2024-09-11T00:59:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf99301640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c55310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf99301640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c55310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33146/0 (socket says 172.21.15.78:33146) 2024-09-11T00:59:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf99301640 1 -- 172.21.15.78:0/1989602652 learned_addr learned my addr 172.21.15.78:0/1989602652 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:40.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf99b02640 1 --2- 172.21.15.78:0/1989602652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c584d0 0x55cfa4c70c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.532+0000 7fdf99301640 1 -- 172.21.15.78:0/1989602652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c64d80 msgr2=0x55cfa4c5ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:40.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.531+0000 7fdf9a303640 1 --2- 172.21.15.78:0/1989602652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c64d80 0x55cfa4c5ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.532+0000 7fdf99301640 1 --2- 172.21.15.78:0/1989602652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c64d80 0x55cfa4c5ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.532+0000 7fdf99301640 1 -- 172.21.15.78:0/1989602652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c584d0 msgr2=0x55cfa4c70c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:40.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.532+0000 7fdf99301640 1 --2- 172.21.15.78:0/1989602652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c584d0 0x55cfa4c70c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.532+0000 7fdf99301640 1 -- 172.21.15.78:0/1989602652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa4c64960 con 0x55cfa4c58df0 2024-09-11T00:59:40.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.532+0000 7fdf99b02640 1 --2- 172.21.15.78:0/1989602652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c584d0 0x55cfa4c70c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:59:40.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.532+0000 7fdf9a303640 1 --2- 172.21.15.78:0/1989602652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c64d80 0x55cfa4c5ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:59:40.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.532+0000 7fdf99301640 1 --2- 172.21.15.78:0/1989602652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c55310 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdf84011470 tx=0x7fdf84011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55104a1fecd97b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:40.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.532+0000 7fdf7f7fe640 1 -- 172.21.15.78:0/1989602652 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84005020 con 0x55cfa4c58df0 2024-09-11T00:59:40.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.532+0000 7fdf7f7fe640 1 -- 172.21.15.78:0/1989602652 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf840047a0 con 0x55cfa4c58df0 2024-09-11T00:59:40.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.532+0000 7fdf7f7fe640 1 -- 172.21.15.78:0/1989602652 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84010a50 con 0x55cfa4c58df0 2024-09-11T00:59:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.533+0000 7fdf9e13b740 1 -- 172.21.15.78:0/1989602652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 msgr2=0x55cfa4c55310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.533+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/1989602652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c55310 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdf84011470 tx=0x7fdf84011930 comp rx=0 tx=0).stop 2024-09-11T00:59:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.533+0000 7fdf9e13b740 1 -- 172.21.15.78:0/1989602652 shutdown_connections 2024-09-11T00:59:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.533+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/1989602652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c64d80 0x55cfa4c5ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.533+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/1989602652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c55310 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.533+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/1989602652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c584d0 0x55cfa4c70c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.533+0000 7fdf9e13b740 1 -- 172.21.15.78:0/1989602652 >> 172.21.15.78:0/1989602652 conn(0x55cfa4c56300 msgr2=0x55cfa4c54c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.533+0000 7fdf9e13b740 1 -- 172.21.15.78:0/1989602652 shutdown_connections 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.533+0000 7fdf9e13b740 1 -- 172.21.15.78:0/1989602652 wait complete. 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.534+0000 7fdf9e13b740 1 -- start start 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.534+0000 7fdf9e13b740 1 Processor -- start 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.534+0000 7fdf9e13b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c584d0 0x55cfa4c6d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.534+0000 7fdf9e13b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c68a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.534+0000 7fdf9e13b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c64d80 0x55cfa4c69160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.534+0000 7fdf9e13b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941b1b00 con 0x55cfa4c58df0 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.534+0000 7fdf9e13b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941b1430 con 0x55cfa4c64d80 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.534+0000 7fdf9e13b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941ae6c0 con 0x55cfa4c584d0 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.534+0000 7fdf9e13b740 10 client.0 _ll_register_callbacks cb 0x7fdf8c031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.534+0000 7fdf99301640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c68a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.534+0000 7fdf99301640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c68a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33156/0 (socket says 172.21.15.78:33156) 2024-09-11T00:59:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.534+0000 7fdf99301640 1 -- 172.21.15.78:0/2895880312 learned_addr learned my addr 172.21.15.78:0/2895880312 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.535+0000 7fdf99b02640 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c584d0 0x55cfa4c6d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.535+0000 7fdf9a303640 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c64d80 0x55cfa4c69160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.535+0000 7fdf99301640 1 -- 172.21.15.78:0/2895880312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c584d0 msgr2=0x55cfa4c6d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.535+0000 7fdf99301640 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c584d0 0x55cfa4c6d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.535+0000 7fdf99301640 1 -- 172.21.15.78:0/2895880312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c64d80 msgr2=0x55cfa4c69160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.535+0000 7fdf99301640 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c64d80 0x55cfa4c69160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.535+0000 7fdf99301640 1 -- 172.21.15.78:0/2895880312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa4ce08c0 con 0x55cfa4c58df0 2024-09-11T00:59:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.535+0000 7fdf99301640 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c68a90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdf84000c00 tx=0x7fdf84005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3060228a2d774ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.535+0000 7fdf7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:59:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.535+0000 7fdf7d7fa640 1 -- 172.21.15.78:0/2895880312 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84028020 con 0x55cfa4c58df0 2024-09-11T00:59:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.535+0000 7fdf7d7fa640 1 -- 172.21.15.78:0/2895880312 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdf84010760 con 0x55cfa4c58df0 2024-09-11T00:59:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.535+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2895880312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf840047a0 con 0x55cfa4c58df0 2024-09-11T00:59:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.536+0000 7fdf7d7fa640 1 -- 172.21.15.78:0/2895880312 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf84033050 con 0x55cfa4c58df0 2024-09-11T00:59:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.536+0000 7fdf9e13b740 10 client.15636 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.536+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2895880312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cfa4ce19f0 con 0x55cfa4c58df0 2024-09-11T00:59:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.536+0000 7fdf7d7fa640 1 -- 172.21.15.78:0/2895880312 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf840047a0 con 0x55cfa4c58df0 2024-09-11T00:59:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.536+0000 7fdf7d7fa640 1 -- 172.21.15.78:0/2895880312 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf840248c0 con 0x55cfa4c58df0 2024-09-11T00:59:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.536+0000 7fdf9e13b740 10 client.15636 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.536+0000 7fdf9e13b740 10 client.15636 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T00:59:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.536+0000 7fdf9e13b740 20 client.15636 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.536+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cfa4ce2050 0x55cfa4ce4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.536+0000 7fdf9e13b740 4 client.15636 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T00:59:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.536+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2895880312 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfa4bcc8a0 con 0x55cfa4ce2050 2024-09-11T00:59:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.537+0000 7fdf99b02640 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cfa4ce2050 0x55cfa4ce4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.537+0000 7fdf7d7fa640 10 client.15636 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T00:59:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.537+0000 7fdf99b02640 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cfa4ce2050 0x55cfa4ce4470 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d0ef3610cdd9e25 server_cookie=fe7989f140bc4b79 in_seq=0 out_seq=0 2024-09-11T00:59:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.537+0000 7fdf7d7fa640 1 -- 172.21.15.78:0/2895880312 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfa4bcc8a0 con 0x55cfa4ce2050 2024-09-11T00:59:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.537+0000 7fdf7d7fa640 10 client.15636 handle_command_reply: tid=0 2024-09-11T00:59:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.537+0000 7fdf9e13b740 1 client.15636 shutdown 2024-09-11T00:59:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.538+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2895880312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 msgr2=0x55cfa4c68a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:40.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.538+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c68a90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdf84000c00 tx=0x7fdf84005250 comp rx=0 tx=0).stop 2024-09-11T00:59:40.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.538+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2895880312 shutdown_connections 2024-09-11T00:59:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.538+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfa4c64d80 0x55cfa4c69160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.538+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cfa4ce2050 0x55cfa4ce4470 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.538+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfa4c58df0 0x55cfa4c68a90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.538+0000 7fdf9e13b740 1 --2- 172.21.15.78:0/2895880312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfa4c584d0 0x55cfa4c6d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.538+0000 7fdf7d7fa640 0 client.15636 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T00:59:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.538+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2895880312 >> 172.21.15.78:0/2895880312 conn(0x55cfa4c56300 msgr2=0x55cfa4c63890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.538+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2895880312 shutdown_connections 2024-09-11T00:59:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.538+0000 7fdf9e13b740 1 -- 172.21.15.78:0/2895880312 wait complete. 2024-09-11T00:59:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.538+0000 7fdf9e13b740 20 client.15636 trim_cache size 0 max 16384 2024-09-11T00:59:40.545 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T00:59:40.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.539+0000 7fdf9bd66640 1 -- 172.21.15.78:0/2474362359 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdf800b54d0 msgr2=0x7fdf800b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:40.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.539+0000 7fdf9bd66640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdf800b54d0 0x7fdf800b7990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009430 tx=0x7fdf8c009380 comp rx=0 tx=0).stop 2024-09-11T00:59:40.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.539+0000 7fdf9bd66640 1 -- 172.21.15.78:0/2474362359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf94102dc0 msgr2=0x7fdf940754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:40.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.539+0000 7fdf9bd66640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf94102dc0 0x7fdf940754a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000c970 tx=0x7fdf9000ce30 comp rx=0 tx=0).stop 2024-09-11T00:59:40.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.539+0000 7fdf9bd66640 1 -- 172.21.15.78:0/2474362359 shutdown_connections 2024-09-11T00:59:40.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.539+0000 7fdf9bd66640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdf800b54d0 0x7fdf800b7990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.539+0000 7fdf9bd66640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf94109150 0x7fdf94075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.539+0000 7fdf9bd66640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf941087d0 0x7fdf940759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.539+0000 7fdf9bd66640 1 --2- 172.21.15.78:0/2474362359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf94102dc0 0x7fdf940754a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:40.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.539+0000 7fdf9bd66640 1 -- 172.21.15.78:0/2474362359 >> 172.21.15.78:0/2474362359 conn(0x7fdf940fe870 msgr2=0x7fdf94100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:40.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.540+0000 7fdf9bd66640 1 -- 172.21.15.78:0/2474362359 shutdown_connections 2024-09-11T00:59:40.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:40.540+0000 7fdf9bd66640 1 -- 172.21.15.78:0/2474362359 wait complete. 2024-09-11T00:59:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:41 smithi110 ceph-mon[34712]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:59:41.459 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:41 smithi078 ceph-mon[31315]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:59:41.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:41 smithi112 ceph-mon[35384]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:59:41.488 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T00:59:41.800 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:59:42.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.386+0000 7f2759b35640 1 -- 172.21.15.78:0/1097939785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27541045c0 msgr2=0x7f275410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:42.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.386+0000 7f2759b35640 1 --2- 172.21.15.78:0/1097939785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27541045c0 0x7f275410cab0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2748009900 tx=0x7f274802a410 comp rx=0 tx=0).stop 2024-09-11T00:59:42.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.387+0000 7f2759b35640 1 -- 172.21.15.78:0/1097939785 shutdown_connections 2024-09-11T00:59:42.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.387+0000 7f2759b35640 1 --2- 172.21.15.78:0/1097939785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f275410d100 0x7f275410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.387+0000 7f2759b35640 1 --2- 172.21.15.78:0/1097939785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27541045c0 0x7f275410cab0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.387+0000 7f2759b35640 1 --2- 172.21.15.78:0/1097939785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2754103ca0 0x7f2754104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.387+0000 7f2759b35640 1 -- 172.21.15.78:0/1097939785 >> 172.21.15.78:0/1097939785 conn(0x7f27540fd700 msgr2=0x7f27540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:42.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.387+0000 7f2759b35640 1 -- 172.21.15.78:0/1097939785 shutdown_connections 2024-09-11T00:59:42.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.387+0000 7f2759b35640 1 -- 172.21.15.78:0/1097939785 wait complete. 2024-09-11T00:59:42.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.388+0000 7f2759b35640 1 Processor -- start 2024-09-11T00:59:42.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.388+0000 7f2759b35640 1 -- start start 2024-09-11T00:59:42.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.388+0000 7f2759b35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2754103ca0 0x7f27541992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.388+0000 7f2759b35640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27541045c0 0x7f2754199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.388+0000 7f2759b35640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f275410d100 0x7f275419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.388+0000 7f27537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2754103ca0 0x7f27541992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.388+0000 7f2759b35640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2754112dc0 con 0x7f2754103ca0 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.388+0000 7f2759b35640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2754112c40 con 0x7f27541045c0 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.388+0000 7f2759b35640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2754112f40 con 0x7f275410d100 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.389+0000 7f27537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2754103ca0 0x7f27541992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33178/0 (socket says 172.21.15.78:33178) 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.389+0000 7f27537fe640 1 -- 172.21.15.78:0/3043405244 learned_addr learned my addr 172.21.15.78:0/3043405244 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.389+0000 7f2752ffd640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27541045c0 0x7f2754199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.389+0000 7f2753fff640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f275410d100 0x7f275419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.389+0000 7f27537fe640 1 -- 172.21.15.78:0/3043405244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f275410d100 msgr2=0x7f275419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.389+0000 7f27537fe640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f275410d100 0x7f275419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.389+0000 7f27537fe640 1 -- 172.21.15.78:0/3043405244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27541045c0 msgr2=0x7f2754199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.389+0000 7f27537fe640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27541045c0 0x7f2754199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.389+0000 7f27537fe640 1 -- 172.21.15.78:0/3043405244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275419f380 con 0x7f2754103ca0 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.389+0000 7f2752ffd640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27541045c0 0x7f2754199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:59:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.390+0000 7f27537fe640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2754103ca0 0x7f27541992d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f274400d950 tx=0x7f274400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:42.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.390+0000 7f2750ff9640 1 -- 172.21.15.78:0/3043405244 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744017070 con 0x7f2754103ca0 2024-09-11T00:59:42.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.390+0000 7f2750ff9640 1 -- 172.21.15.78:0/3043405244 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2744009c30 con 0x7f2754103ca0 2024-09-11T00:59:42.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.390+0000 7f2750ff9640 1 -- 172.21.15.78:0/3043405244 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274400f620 con 0x7f2754103ca0 2024-09-11T00:59:42.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.390+0000 7f2759b35640 1 -- 172.21.15.78:0/3043405244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275419f670 con 0x7f2754103ca0 2024-09-11T00:59:42.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.390+0000 7f2759b35640 1 -- 172.21.15.78:0/3043405244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2754101830 con 0x7f2754103ca0 2024-09-11T00:59:42.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.395+0000 7f2750ff9640 1 -- 172.21.15.78:0/3043405244 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2744010460 con 0x7f2754103ca0 2024-09-11T00:59:42.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.396+0000 7f275bf0a740 1 Processor -- start 2024-09-11T00:59:42.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.396+0000 7f275bf0a740 1 -- start start 2024-09-11T00:59:42.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.396+0000 7f2750ff9640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f273c0b5500 0x7f273c0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.396+0000 7f275bf0a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d5d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f2753fff640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f273c0b5500 0x7f273c0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f2750ff9640 1 -- 172.21.15.78:0/3043405244 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f27440e5890 con 0x7f2754103ca0 2024-09-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f27537fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d5d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f27537fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d5d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38390/0 (socket says 172.21.15.78:38390) 2024-09-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f27537fe640 1 -- 172.21.15.78:0/2026211044 learned_addr learned my addr 172.21.15.78:0/2026211044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2026211044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d69460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f2753fff640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f273c0b5500 0x7f273c0b79c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f274c005fd0 tx=0x7f274c009600 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2026211044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d6bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f275bf0a740 1 -- 172.21.15.78:0/2026211044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c524cd3ea0 con 0x55c524d699a0 2024-09-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f275bf0a740 1 -- 172.21.15.78:0/2026211044 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c524cc32d0 con 0x55c524d5d0f0 2024-09-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f275bf0a740 1 -- 172.21.15.78:0/2026211044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c524cc70e0 con 0x55c524d5da10 2024-09-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f2753fff640 1 --2- 172.21.15.78:0/2026211044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d6bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.397+0000 7f2752ffd640 1 --2- 172.21.15.78:0/2026211044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d69460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.398+0000 7f2753fff640 1 -- 172.21.15.78:0/2026211044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 msgr2=0x55c524d69460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:42.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.398+0000 7f2753fff640 1 --2- 172.21.15.78:0/2026211044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d69460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.398+0000 7f2753fff640 1 -- 172.21.15.78:0/2026211044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 msgr2=0x55c524d5d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:42.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.398+0000 7f2753fff640 1 --2- 172.21.15.78:0/2026211044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d5d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.398+0000 7f2753fff640 1 -- 172.21.15.78:0/2026211044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c524a21650 con 0x55c524d699a0 2024-09-11T00:59:42.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.398+0000 7f2753fff640 1 --2- 172.21.15.78:0/2026211044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d6bd90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f274c010bd0 tx=0x7f274c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280142773560c7bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.398+0000 7f2739ffb640 1 -- 172.21.15.78:0/2026211044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c023070 con 0x55c524d699a0 2024-09-11T00:59:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.399+0000 7f2739ffb640 1 -- 172.21.15.78:0/2026211044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f274c011b50 con 0x55c524d699a0 2024-09-11T00:59:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.399+0000 7f2739ffb640 1 -- 172.21.15.78:0/2026211044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c01fca0 con 0x55c524d699a0 2024-09-11T00:59:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.399+0000 7f275bf0a740 1 -- 172.21.15.78:0/2026211044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 msgr2=0x55c524d6bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.399+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2026211044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d6bd90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f274c010bd0 tx=0x7f274c013000 comp rx=0 tx=0).stop 2024-09-11T00:59:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.400+0000 7f275bf0a740 1 -- 172.21.15.78:0/2026211044 shutdown_connections 2024-09-11T00:59:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.400+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2026211044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d6bd90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.400+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2026211044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d69460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.400+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2026211044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d5d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.400+0000 7f275bf0a740 1 -- 172.21.15.78:0/2026211044 >> 172.21.15.78:0/2026211044 conn(0x55c524d5af20 msgr2=0x55c524d5c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.400+0000 7f275bf0a740 1 -- 172.21.15.78:0/2026211044 shutdown_connections 2024-09-11T00:59:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.400+0000 7f275bf0a740 1 -- 172.21.15.78:0/2026211044 wait complete. 2024-09-11T00:59:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.400+0000 7f275bf0a740 1 -- start start 2024-09-11T00:59:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.400+0000 7f275bf0a740 1 Processor -- start 2024-09-11T00:59:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f275bf0a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d69310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f275bf0a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d66760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f275bf0a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d66de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f275bf0a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c524b24700 con 0x55c524d699a0 2024-09-11T00:59:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f275bf0a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c524c5e160 con 0x55c524d5d0f0 2024-09-11T00:59:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f275bf0a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c524d6c620 con 0x55c524d5da10 2024-09-11T00:59:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f275bf0a740 10 client.0 _ll_register_callbacks cb 0x55c524c779e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f27537fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d69310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f2753fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d66de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f27537fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d69310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38394/0 (socket says 172.21.15.78:38394) 2024-09-11T00:59:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f27537fe640 1 -- 172.21.15.78:0/961036294 learned_addr learned my addr 172.21.15.78:0/961036294 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f2752ffd640 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d66760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f2752ffd640 1 -- 172.21.15.78:0/961036294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 msgr2=0x55c524d69310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f2752ffd640 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d69310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.401+0000 7f2752ffd640 1 -- 172.21.15.78:0/961036294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 msgr2=0x55c524d66de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.402+0000 7f2752ffd640 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d66de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.402+0000 7f2752ffd640 1 -- 172.21.15.78:0/961036294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c524c8bc90 con 0x55c524d5da10 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.402+0000 7f2753fff640 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d66de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.402+0000 7f2752ffd640 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d66760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2748009850 tx=0x7f274803d6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bf1068a0a053243 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.402+0000 7f2732ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.402+0000 7f2732ffd640 1 -- 172.21.15.78:0/961036294 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748052060 con 0x55c524d5da10 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.402+0000 7f275bf0a740 1 -- 172.21.15.78:0/961036294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f274c011b50 con 0x55c524d5da10 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.402+0000 7f2732ffd640 1 -- 172.21.15.78:0/961036294 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f2748005550 con 0x55c524d5da10 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.403+0000 7f2732ffd640 1 -- 172.21.15.78:0/961036294 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2748035040 con 0x55c524d5da10 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.403+0000 7f275bf0a740 10 client.25069 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.403+0000 7f275bf0a740 1 -- 172.21.15.78:0/961036294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c524d78020 con 0x55c524d5da10 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.403+0000 7f2732ffd640 1 -- 172.21.15.78:0/961036294 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f274c011b50 con 0x55c524d5da10 2024-09-11T00:59:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.403+0000 7f2732ffd640 1 -- 172.21.15.78:0/961036294 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2748055020 con 0x55c524d5da10 2024-09-11T00:59:42.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.403+0000 7f275bf0a740 10 client.25069 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:42.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.403+0000 7f275bf0a740 10 client.25069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T00:59:42.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.403+0000 7f275bf0a740 20 client.25069 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:42.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.403+0000 7f275bf0a740 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c524d79900 0x55c524d7bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.403+0000 7f275bf0a740 4 client.25069 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:59:42.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.403+0000 7f275bf0a740 1 -- 172.21.15.78:0/961036294 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c524cd5df0 con 0x55c524d79900 2024-09-11T00:59:42.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.404+0000 7f27537fe640 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c524d79900 0x55c524d7bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.404+0000 7f2732ffd640 10 client.25069 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:42.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.405+0000 7f27537fe640 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c524d79900 0x55c524d7bd20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cdb28b95d330e26 server_cookie=1dbb024c4a3c2fc3 in_seq=0 out_seq=0 2024-09-11T00:59:42.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.405+0000 7f2732ffd640 1 -- 172.21.15.78:0/961036294 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c524cd5df0 con 0x55c524d79900 2024-09-11T00:59:42.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.405+0000 7f2732ffd640 10 client.25069 handle_command_reply: tid=0 2024-09-11T00:59:42.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.405+0000 7f275bf0a740 1 client.25069 shutdown 2024-09-11T00:59:42.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.406+0000 7f275bf0a740 1 -- 172.21.15.78:0/961036294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 msgr2=0x55c524d66760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:42.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.406+0000 7f275bf0a740 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d66760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2748009850 tx=0x7f274803d6f0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.406+0000 7f275bf0a740 1 -- 172.21.15.78:0/961036294 shutdown_connections 2024-09-11T00:59:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.406+0000 7f275bf0a740 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c524d79900 0x55c524d7bd20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.406+0000 7f275bf0a740 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d66de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.406+0000 7f275bf0a740 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d66760 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.406+0000 7f275bf0a740 1 --2- 172.21.15.78:0/961036294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d69310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.406+0000 7f275bf0a740 1 -- 172.21.15.78:0/961036294 >> 172.21.15.78:0/961036294 conn(0x55c524d5af20 msgr2=0x55c524d60ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.406+0000 7f2732ffd640 0 client.25069 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.406+0000 7f275bf0a740 1 -- 172.21.15.78:0/961036294 shutdown_connections 2024-09-11T00:59:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.406+0000 7f275bf0a740 1 -- 172.21.15.78:0/961036294 wait complete. 2024-09-11T00:59:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.406+0000 7f275bf0a740 20 client.25069 trim_cache size 0 max 16384 2024-09-11T00:59:42.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.419+0000 7f275bf0a740 1 Processor -- start 2024-09-11T00:59:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.419+0000 7f275bf0a740 1 -- start start 2024-09-11T00:59:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f275bf0a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d75850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f275bf0a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d5da10 0x55c524d65c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f275bf0a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d699a0 0x55c524d61970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f275bf0a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27541ac3c0 con 0x55c524d5da10 2024-09-11T00:59:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f275bf0a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27541adf00 con 0x55c524d5d0f0 2024-09-11T00:59:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f275bf0a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27541aca90 con 0x55c524d699a0 2024-09-11T00:59:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f2752ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d5da10 0x55c524d65c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f2752ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d5da10 0x55c524d65c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33224/0 (socket says 172.21.15.78:33224) 2024-09-11T00:59:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f2752ffd640 1 -- 172.21.15.78:0/2036819390 learned_addr learned my addr 172.21.15.78:0/2036819390 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f2753fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d699a0 0x55c524d61970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f27537fe640 1 --2- 172.21.15.78:0/2036819390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d75850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f2753fff640 1 -- 172.21.15.78:0/2036819390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 msgr2=0x55c524d75850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f2753fff640 1 --2- 172.21.15.78:0/2036819390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d75850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f2753fff640 1 -- 172.21.15.78:0/2036819390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d5da10 msgr2=0x55c524d65c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f2753fff640 1 --2- 172.21.15.78:0/2036819390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d5da10 0x55c524d65c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.420+0000 7f2753fff640 1 -- 172.21.15.78:0/2036819390 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c524d7c3a0 con 0x55c524d699a0 2024-09-11T00:59:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.421+0000 7f2753fff640 1 --2- 172.21.15.78:0/2036819390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d699a0 0x55c524d61970 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f274c00a010 tx=0x7f274c009d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69d4734c4e97cd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.421+0000 7f27397fa640 1 -- 172.21.15.78:0/2036819390 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c023040 con 0x55c524d699a0 2024-09-11T00:59:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.421+0000 7f27397fa640 1 -- 172.21.15.78:0/2036819390 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f274c01f450 con 0x55c524d699a0 2024-09-11T00:59:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.421+0000 7f27397fa640 1 -- 172.21.15.78:0/2036819390 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c0056a0 con 0x55c524d699a0 2024-09-11T00:59:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.422+0000 7f275bf0a740 1 -- 172.21.15.78:0/2036819390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d699a0 msgr2=0x55c524d61970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.422+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2036819390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d699a0 0x55c524d61970 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f274c00a010 tx=0x7f274c009d50 comp rx=0 tx=0).stop 2024-09-11T00:59:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.422+0000 7f275bf0a740 1 -- 172.21.15.78:0/2036819390 shutdown_connections 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.422+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2036819390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d699a0 0x55c524d61970 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.422+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2036819390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d5da10 0x55c524d65c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.422+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2036819390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d75850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.422+0000 7f275bf0a740 1 -- 172.21.15.78:0/2036819390 >> 172.21.15.78:0/2036819390 conn(0x55c524d5af20 msgr2=0x55c524d5c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.422+0000 7f275bf0a740 1 -- 172.21.15.78:0/2036819390 shutdown_connections 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.422+0000 7f275bf0a740 1 -- 172.21.15.78:0/2036819390 wait complete. 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.422+0000 7f275bf0a740 1 -- start start 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.422+0000 7f275bf0a740 1 Processor -- start 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f275bf0a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f275bf0a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d71e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f275bf0a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d6c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f275bf0a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27541ad830 con 0x55c524d699a0 2024-09-11T00:59:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f2753fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d6c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f2752ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d71e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f2753fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d6c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33240/0 (socket says 172.21.15.78:33240) 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f2752ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d71e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:39158/0 (socket says 172.21.15.78:39158) 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f2752ffd640 1 -- 172.21.15.78:0/2410298063 learned_addr learned my addr 172.21.15.78:0/2410298063 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f27537fe640 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f275bf0a740 1 -- 172.21.15.78:0/2410298063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27541ad160 con 0x55c524d5d0f0 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f275bf0a740 1 -- 172.21.15.78:0/2410298063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27541aa3a0 con 0x55c524d5da10 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f275bf0a740 10 client.0 _ll_register_callbacks cb 0x7f2748009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f27537fe640 1 -- 172.21.15.78:0/2410298063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 msgr2=0x55c524d71e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f27537fe640 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d71e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f27537fe640 1 -- 172.21.15.78:0/2410298063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 msgr2=0x55c524d6c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f27537fe640 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d6c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f27537fe640 1 -- 172.21.15.78:0/2410298063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c524d723f0 con 0x55c524d5d0f0 2024-09-11T00:59:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f2753fff640 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d6c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:59:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.423+0000 7f2752ffd640 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d71e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:59:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.425+0000 7f27537fe640 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d718c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55c524d7c5d0 tx=0x7f274400d8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4915af5339303f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.425+0000 7f2732ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T00:59:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.425+0000 7f2732ffd640 1 -- 172.21.15.78:0/2410298063 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27440e3470 con 0x55c524d5d0f0 2024-09-11T00:59:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.425+0000 7f275bf0a740 1 -- 172.21.15.78:0/2410298063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f274c01f450 con 0x55c524d5d0f0 2024-09-11T00:59:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.425+0000 7f2732ffd640 1 -- 172.21.15.78:0/2410298063 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f27440b3b80 con 0x55c524d5d0f0 2024-09-11T00:59:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.426+0000 7f2732ffd640 1 -- 172.21.15.78:0/2410298063 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27440abd20 con 0x55c524d5d0f0 2024-09-11T00:59:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.426+0000 7f275bf0a740 10 client.25094 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.426+0000 7f275bf0a740 1 -- 172.21.15.78:0/2410298063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c524de6c80 con 0x55c524d5d0f0 2024-09-11T00:59:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.426+0000 7f2732ffd640 1 -- 172.21.15.78:0/2410298063 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27440b0800 con 0x55c524d5d0f0 2024-09-11T00:59:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.427+0000 7f2732ffd640 1 -- 172.21.15.78:0/2410298063 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2744010780 con 0x55c524d5d0f0 2024-09-11T00:59:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.427+0000 7f275bf0a740 10 client.25094 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.427+0000 7f275bf0a740 10 client.25094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T00:59:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.427+0000 7f275bf0a740 20 client.25094 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.427+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c524de7380 0x55c524de97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.427+0000 7f275bf0a740 4 client.25094 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T00:59:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.427+0000 7f275bf0a740 1 -- 172.21.15.78:0/2410298063 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f274c01f450 con 0x55c524de7380 2024-09-11T00:59:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.427+0000 7f2752ffd640 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c524de7380 0x55c524de97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:42.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.427+0000 7f2752ffd640 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c524de7380 0x55c524de97a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeca70d19e439822 server_cookie=b4929a7c377123e7 in_seq=0 out_seq=0 2024-09-11T00:59:42.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.427+0000 7f2732ffd640 10 client.25094 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:42.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f2732ffd640 1 -- 172.21.15.78:0/2410298063 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f274c01f450 con 0x55c524de7380 2024-09-11T00:59:42.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f2732ffd640 10 client.25094 handle_command_reply: tid=0 2024-09-11T00:59:42.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f275bf0a740 1 client.25094 shutdown 2024-09-11T00:59:42.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f275bf0a740 1 -- 172.21.15.78:0/2410298063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 msgr2=0x55c524d718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:42.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d718c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55c524d7c5d0 tx=0x7f274400d8d0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f275bf0a740 1 -- 172.21.15.78:0/2410298063 shutdown_connections 2024-09-11T00:59:42.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c524de7380 0x55c524de97a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c524d699a0 0x55c524d6c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c524d5da10 0x55c524d71e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f275bf0a740 1 --2- 172.21.15.78:0/2410298063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c524d5d0f0 0x55c524d718c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f275bf0a740 1 -- 172.21.15.78:0/2410298063 >> 172.21.15.78:0/2410298063 conn(0x55c524d5af20 msgr2=0x55c524d61970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:42.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f2732ffd640 0 client.25094 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:42.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f275bf0a740 1 -- 172.21.15.78:0/2410298063 shutdown_connections 2024-09-11T00:59:42.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f275bf0a740 1 -- 172.21.15.78:0/2410298063 wait complete. 2024-09-11T00:59:42.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.428+0000 7f275bf0a740 20 client.25094 trim_cache size 0 max 16384 2024-09-11T00:59:42.438 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T00:59:42.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.429+0000 7f2759b35640 1 -- 172.21.15.78:0/3043405244 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f273c0b5500 msgr2=0x7f273c0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:42.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.429+0000 7f2759b35640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f273c0b5500 0x7f273c0b79c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f274c005fd0 tx=0x7f274c009600 comp rx=0 tx=0).stop 2024-09-11T00:59:42.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.429+0000 7f2759b35640 1 -- 172.21.15.78:0/3043405244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2754103ca0 msgr2=0x7f27541992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:42.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.429+0000 7f2759b35640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2754103ca0 0x7f27541992d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f274400d950 tx=0x7f274400de10 comp rx=0 tx=0).stop 2024-09-11T00:59:42.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.429+0000 7f2759b35640 1 -- 172.21.15.78:0/3043405244 shutdown_connections 2024-09-11T00:59:42.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.429+0000 7f2759b35640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f273c0b5500 0x7f273c0b79c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.429+0000 7f2759b35640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f275410d100 0x7f275419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.429+0000 7f2759b35640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27541045c0 0x7f2754199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.429+0000 7f2759b35640 1 --2- 172.21.15.78:0/3043405244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2754103ca0 0x7f27541992d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:42.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.429+0000 7f2759b35640 1 -- 172.21.15.78:0/3043405244 >> 172.21.15.78:0/3043405244 conn(0x7f27540fd700 msgr2=0x7f27540fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:42.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.429+0000 7f2759b35640 1 -- 172.21.15.78:0/3043405244 shutdown_connections 2024-09-11T00:59:42.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:42.430+0000 7f2759b35640 1 -- 172.21.15.78:0/3043405244 wait complete. 2024-09-11T00:59:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:43 smithi078 ceph-mon[31315]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:59:43.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:43 smithi110 ceph-mon[34712]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:59:43.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:43 smithi112 ceph-mon[35384]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:59:44.414 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T00:59:44.415 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T00:59:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:59:45.138 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:45 smithi078 ceph-mon[31315]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:59:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.268+0000 7f83fb689640 1 -- 172.21.15.78:0/1337480102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83f41076a0 msgr2=0x7f83f4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.268+0000 7f83fb689640 1 --2- 172.21.15.78:0/1337480102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83f41076a0 0x7f83f4107ae0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009900 tx=0x7f83e402a410 comp rx=0 tx=0).stop 2024-09-11T00:59:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.268+0000 7f83fb689640 1 -- 172.21.15.78:0/1337480102 shutdown_connections 2024-09-11T00:59:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.268+0000 7f83fb689640 1 --2- 172.21.15.78:0/1337480102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f4108020 0x7f83f410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.268+0000 7f83fb689640 1 --2- 172.21.15.78:0/1337480102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83f41076a0 0x7f83f4107ae0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.268+0000 7f83fb689640 1 --2- 172.21.15.78:0/1337480102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83f4101c90 0x7f83f4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.268+0000 7f83fb689640 1 -- 172.21.15.78:0/1337480102 >> 172.21.15.78:0/1337480102 conn(0x7f83f40fd700 msgr2=0x7f83f40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.268+0000 7f83fb689640 1 -- 172.21.15.78:0/1337480102 shutdown_connections 2024-09-11T00:59:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.268+0000 7f83fb689640 1 -- 172.21.15.78:0/1337480102 wait complete. 2024-09-11T00:59:45.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83fb689640 1 Processor -- start 2024-09-11T00:59:45.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83fb689640 1 -- start start 2024-09-11T00:59:45.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83fb689640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83f4101c90 0x7f83f4198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83fb689640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83f41076a0 0x7f83f41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83fb689640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f4108020 0x7f83f419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83fb689640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4111c70 con 0x7f83f41076a0 2024-09-11T00:59:45.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83fb689640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4111af0 con 0x7f83f4101c90 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83fb689640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4111df0 con 0x7f83f4108020 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83f8c24640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83f41076a0 0x7f83f41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83f8c24640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83f41076a0 0x7f83f41986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59014/0 (socket says 172.21.15.78:59014) 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83f8c24640 1 -- 172.21.15.78:0/1241861269 learned_addr learned my addr 172.21.15.78:0/1241861269 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83f9425640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83f4101c90 0x7f83f4198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83f9c26640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f4108020 0x7f83f419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83f8c24640 1 -- 172.21.15.78:0/1241861269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f4108020 msgr2=0x7f83f419da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83f8c24640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f4108020 0x7f83f419da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83f8c24640 1 -- 172.21.15.78:0/1241861269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83f4101c90 msgr2=0x7f83f4198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83f8c24640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83f4101c90 0x7f83f4198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.269+0000 7f83f8c24640 1 -- 172.21.15.78:0/1241861269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f419e210 con 0x7f83f41076a0 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.270+0000 7f83f8c24640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83f41076a0 0x7f83f41986a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f83e4005940 tx=0x7f83e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.270+0000 7f83ea7fc640 1 -- 172.21.15.78:0/1241861269 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e403f030 con 0x7f83f41076a0 2024-09-11T00:59:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.270+0000 7f83ea7fc640 1 -- 172.21.15.78:0/1241861269 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83e4003cb0 con 0x7f83f41076a0 2024-09-11T00:59:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.270+0000 7f83fb689640 1 -- 172.21.15.78:0/1241861269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f419e4a0 con 0x7f83f41076a0 2024-09-11T00:59:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.270+0000 7f83ea7fc640 1 -- 172.21.15.78:0/1241861269 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4035070 con 0x7f83f41076a0 2024-09-11T00:59:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.270+0000 7f83fb689640 1 -- 172.21.15.78:0/1241861269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f406aab0 con 0x7f83f41076a0 2024-09-11T00:59:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.273+0000 7f83ea7fc640 1 -- 172.21.15.78:0/1241861269 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f83e403bbf0 con 0x7f83f41076a0 2024-09-11T00:59:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83ea7fc640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f83e00b54d0 0x7f83e00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83fda5e740 1 Processor -- start 2024-09-11T00:59:45.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83ea7fc640 1 -- 172.21.15.78:0/1241861269 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f83e41066a0 con 0x7f83f41076a0 2024-09-11T00:59:45.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83f9425640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f83e00b54d0 0x7f83e00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83fda5e740 1 -- start start 2024-09-11T00:59:45.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83fda5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 0x55ab0609fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83fda5e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060af280 0x55ab060af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83fda5e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060afc00 0x55ab060b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83fda5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ab06018e60 con 0x55ab0609f960 2024-09-11T00:59:45.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83fda5e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ab06008290 con 0x55ab060afc00 2024-09-11T00:59:45.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83fda5e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ab0600c0a0 con 0x55ab060af280 2024-09-11T00:59:45.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83f9425640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 0x55ab0609fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83f9425640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 0x55ab0609fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59026/0 (socket says 172.21.15.78:59026) 2024-09-11T00:59:45.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83f9425640 1 -- 172.21.15.78:0/1099544833 learned_addr learned my addr 172.21.15.78:0/1099544833 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.274+0000 7f83f9425640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f83e00b54d0 0x7f83e00b7990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f83f0005fd0 tx=0x7f83f0009680 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.275+0000 7f83f8c24640 1 --2- 172.21.15.78:0/1099544833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060af280 0x55ab060af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.275+0000 7f83f9c26640 1 --2- 172.21.15.78:0/1099544833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060afc00 0x55ab060b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.275+0000 7f83f9425640 1 -- 172.21.15.78:0/1099544833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060af280 msgr2=0x55ab060af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.275+0000 7f83f9425640 1 --2- 172.21.15.78:0/1099544833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060af280 0x55ab060af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.275+0000 7f83f9425640 1 -- 172.21.15.78:0/1099544833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060afc00 msgr2=0x55ab060b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.275+0000 7f83f9425640 1 --2- 172.21.15.78:0/1099544833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060afc00 0x55ab060b95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.275+0000 7f83f9425640 1 -- 172.21.15.78:0/1099544833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab05d66930 con 0x55ab0609f960 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.275+0000 7f83f9c26640 1 --2- 172.21.15.78:0/1099544833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060afc00 0x55ab060b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.275+0000 7f83f9425640 1 --2- 172.21.15.78:0/1099544833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 0x55ab0609fd40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f83f0004990 tx=0x7f83f0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf4ed9b3e66117d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.275+0000 7f83df7fe640 1 -- 172.21.15.78:0/1099544833 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0023070 con 0x55ab0609f960 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.275+0000 7f83df7fe640 1 -- 172.21.15.78:0/1099544833 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83f001fb00 con 0x55ab0609f960 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.275+0000 7f83df7fe640 1 -- 172.21.15.78:0/1099544833 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f001ead0 con 0x55ab0609f960 2024-09-11T00:59:45.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.276+0000 7f83fda5e740 1 -- 172.21.15.78:0/1099544833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 msgr2=0x55ab0609fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.276+0000 7f83fda5e740 1 --2- 172.21.15.78:0/1099544833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 0x55ab0609fd40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f83f0004990 tx=0x7f83f0013000 comp rx=0 tx=0).stop 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.276+0000 7f83fda5e740 1 -- 172.21.15.78:0/1099544833 shutdown_connections 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.276+0000 7f83fda5e740 1 --2- 172.21.15.78:0/1099544833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060afc00 0x55ab060b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.276+0000 7f83fda5e740 1 --2- 172.21.15.78:0/1099544833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060af280 0x55ab060af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.276+0000 7f83fda5e740 1 --2- 172.21.15.78:0/1099544833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 0x55ab0609fd40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.276+0000 7f83fda5e740 1 -- 172.21.15.78:0/1099544833 >> 172.21.15.78:0/1099544833 conn(0x55ab060a1e80 msgr2=0x55ab060a2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.276+0000 7f83fda5e740 1 -- 172.21.15.78:0/1099544833 shutdown_connections 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.276+0000 7f83fda5e740 1 -- 172.21.15.78:0/1099544833 wait complete. 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83fda5e740 1 -- start start 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83fda5e740 1 Processor -- start 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83fda5e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab0609f960 0x55ab060b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83fda5e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060af280 0x55ab060b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83fda5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 0x55ab060b7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83fda5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ab05e69890 con 0x55ab060afc00 2024-09-11T00:59:45.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83fda5e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ab05fa2d40 con 0x55ab060af280 2024-09-11T00:59:45.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83fda5e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ab060cad40 con 0x55ab0609f960 2024-09-11T00:59:45.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83fda5e740 10 client.0 _ll_register_callbacks cb 0x55ab06006c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:45.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83f9c26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 0x55ab060b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83f9c26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 0x55ab060b7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59036/0 (socket says 172.21.15.78:59036) 2024-09-11T00:59:45.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83f9c26640 1 -- 172.21.15.78:0/2637914150 learned_addr learned my addr 172.21.15.78:0/2637914150 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:45.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.277+0000 7f83f9425640 1 --2- 172.21.15.78:0/2637914150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab0609f960 0x55ab060b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83f9c26640 1 -- 172.21.15.78:0/2637914150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab0609f960 msgr2=0x55ab060b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:45.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83f9c26640 1 --2- 172.21.15.78:0/2637914150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab0609f960 0x55ab060b95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83f9c26640 1 -- 172.21.15.78:0/2637914150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060af280 msgr2=0x55ab060b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:45.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83f9c26640 1 --2- 172.21.15.78:0/2637914150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060af280 0x55ab060b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83f9c26640 1 -- 172.21.15.78:0/2637914150 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab060d4030 con 0x55ab060afc00 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83f9c26640 1 --2- 172.21.15.78:0/2637914150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 0x55ab060b7260 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009580 tx=0x7f83ec009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790a1900a7b881b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83dcff9640 1 -- 172.21.15.78:0/2637914150 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00c800 con 0x55ab060afc00 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83dcff9640 1 -- 172.21.15.78:0/2637914150 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f83ec00c9a0 con 0x55ab060afc00 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83fda5e740 1 -- 172.21.15.78:0/2637914150 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f001fb00 con 0x55ab060afc00 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83dcff9640 1 -- 172.21.15.78:0/2637914150 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f83ec00cba0 con 0x55ab060afc00 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83fda5e740 10 client.15666 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.278+0000 7f83fda5e740 1 -- 172.21.15.78:0/2637914150 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ab060d69b0 con 0x55ab060afc00 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.279+0000 7f83dcff9640 1 -- 172.21.15.78:0/2637914150 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83f001fb00 con 0x55ab060afc00 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.279+0000 7f83dcff9640 1 -- 172.21.15.78:0/2637914150 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f83ec016070 con 0x55ab060afc00 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.279+0000 7f83fda5e740 10 client.15666 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.279+0000 7f83fda5e740 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T00:59:45.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.279+0000 7f83fda5e740 20 client.15666 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:45.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.279+0000 7f83fda5e740 1 --2- 172.21.15.78:0/2637914150 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab060d7f10 0x55ab060da390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.279+0000 7f83fda5e740 4 client.15666 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:59:45.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.279+0000 7f83fda5e740 1 -- 172.21.15.78:0/2637914150 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab0601adb0 con 0x55ab060d7f10 2024-09-11T00:59:45.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.279+0000 7f83f9425640 1 --2- 172.21.15.78:0/2637914150 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab060d7f10 0x55ab060da390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.280+0000 7f83f9425640 1 --2- 172.21.15.78:0/2637914150 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab060d7f10 0x55ab060da390 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=270837befce7858c server_cookie=d3b0d5cfae8a661c in_seq=0 out_seq=0 2024-09-11T00:59:45.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.280+0000 7f83dcff9640 10 client.15666 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:45.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.280+0000 7f83dcff9640 1 -- 172.21.15.78:0/2637914150 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab0601adb0 con 0x55ab060d7f10 2024-09-11T00:59:45.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.280+0000 7f83dcff9640 10 client.15666 handle_command_reply: tid=0 2024-09-11T00:59:45.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.280+0000 7f83fda5e740 1 client.15666 shutdown 2024-09-11T00:59:45.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.280+0000 7f83fda5e740 1 -- 172.21.15.78:0/2637914150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 msgr2=0x55ab060b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:45.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.280+0000 7f83fda5e740 1 --2- 172.21.15.78:0/2637914150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 0x55ab060b7260 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009580 tx=0x7f83ec009a40 comp rx=0 tx=0).stop 2024-09-11T00:59:45.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.281+0000 7f83fda5e740 1 -- 172.21.15.78:0/2637914150 shutdown_connections 2024-09-11T00:59:45.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.281+0000 7f83fda5e740 1 --2- 172.21.15.78:0/2637914150 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab060d7f10 0x55ab060da390 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.281+0000 7f83fda5e740 1 --2- 172.21.15.78:0/2637914150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 0x55ab060b7260 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.281+0000 7f83fda5e740 1 --2- 172.21.15.78:0/2637914150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060af280 0x55ab060b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.281+0000 7f83fda5e740 1 --2- 172.21.15.78:0/2637914150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab0609f960 0x55ab060b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.281+0000 7f83dcff9640 0 client.15666 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:45.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.281+0000 7f83fda5e740 1 -- 172.21.15.78:0/2637914150 >> 172.21.15.78:0/2637914150 conn(0x55ab060a1e80 msgr2=0x55ab0609ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:45.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.281+0000 7f83fda5e740 1 -- 172.21.15.78:0/2637914150 shutdown_connections 2024-09-11T00:59:45.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.281+0000 7f83fda5e740 1 -- 172.21.15.78:0/2637914150 wait complete. 2024-09-11T00:59:45.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.281+0000 7f83fda5e740 20 client.15666 trim_cache size 0 max 16384 2024-09-11T00:59:45.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.313+0000 7f83fda5e740 1 Processor -- start 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.313+0000 7f83fda5e740 1 -- start start 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.313+0000 7f83fda5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 0x55ab060b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.313+0000 7f83fda5e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060af280 0x55ab060b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.313+0000 7f83fda5e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060afc00 0x55ab060ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.313+0000 7f83fda5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83f40742c0 con 0x55ab0609f960 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.313+0000 7f83fda5e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f83f41a5650 con 0x55ab060af280 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.313+0000 7f83fda5e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4074990 con 0x55ab060afc00 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.313+0000 7f83f9425640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 0x55ab060b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.313+0000 7f83f9425640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 0x55ab060b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59048/0 (socket says 172.21.15.78:59048) 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.313+0000 7f83f9425640 1 -- 172.21.15.78:0/4079161041 learned_addr learned my addr 172.21.15.78:0/4079161041 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.314+0000 7f83f8c24640 1 --2- 172.21.15.78:0/4079161041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060af280 0x55ab060b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.314+0000 7f83f9c26640 1 --2- 172.21.15.78:0/4079161041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060afc00 0x55ab060ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.314+0000 7f83f9425640 1 -- 172.21.15.78:0/4079161041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060afc00 msgr2=0x55ab060ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:45.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.314+0000 7f83f9425640 1 --2- 172.21.15.78:0/4079161041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060afc00 0x55ab060ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.314+0000 7f83f9425640 1 -- 172.21.15.78:0/4079161041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060af280 msgr2=0x55ab060b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:45.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.314+0000 7f83f9425640 1 --2- 172.21.15.78:0/4079161041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060af280 0x55ab060b61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.314+0000 7f83f9425640 1 -- 172.21.15.78:0/4079161041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab060b7840 con 0x55ab0609f960 2024-09-11T00:59:45.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.314+0000 7f83f9c26640 1 --2- 172.21.15.78:0/4079161041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060afc00 0x55ab060ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:59:45.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.314+0000 7f83f9425640 1 --2- 172.21.15.78:0/4079161041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 0x55ab060b44e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009ab0 tx=0x7f83f0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5c6db188db52a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:45.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.314+0000 7f83deffd640 1 -- 172.21.15.78:0/4079161041 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0016040 con 0x55ab0609f960 2024-09-11T00:59:45.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.314+0000 7f83deffd640 1 -- 172.21.15.78:0/4079161041 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83f001fb00 con 0x55ab0609f960 2024-09-11T00:59:45.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.314+0000 7f83deffd640 1 -- 172.21.15.78:0/4079161041 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0023040 con 0x55ab0609f960 2024-09-11T00:59:45.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.315+0000 7f83fda5e740 1 -- 172.21.15.78:0/4079161041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 msgr2=0x55ab060b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:45.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.315+0000 7f83fda5e740 1 --2- 172.21.15.78:0/4079161041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 0x55ab060b44e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009ab0 tx=0x7f83f0015000 comp rx=0 tx=0).stop 2024-09-11T00:59:45.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.315+0000 7f83fda5e740 1 -- 172.21.15.78:0/4079161041 shutdown_connections 2024-09-11T00:59:45.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.315+0000 7f83fda5e740 1 --2- 172.21.15.78:0/4079161041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060afc00 0x55ab060ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.315+0000 7f83fda5e740 1 --2- 172.21.15.78:0/4079161041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab060af280 0x55ab060b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.315+0000 7f83fda5e740 1 --2- 172.21.15.78:0/4079161041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab0609f960 0x55ab060b44e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.315+0000 7f83fda5e740 1 -- 172.21.15.78:0/4079161041 >> 172.21.15.78:0/4079161041 conn(0x55ab060a1e80 msgr2=0x55ab060d9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:45.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.315+0000 7f83fda5e740 1 -- 172.21.15.78:0/4079161041 shutdown_connections 2024-09-11T00:59:45.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.315+0000 7f83fda5e740 1 -- 172.21.15.78:0/4079161041 wait complete. 2024-09-11T00:59:45.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.316+0000 7f83fda5e740 1 -- start start 2024-09-11T00:59:45.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.316+0000 7f83fda5e740 1 Processor -- start 2024-09-11T00:59:45.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.316+0000 7f83fda5e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab0609f960 0x55ab060d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.316+0000 7f83fda5e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060af280 0x55ab060d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.316+0000 7f83fda5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 0x55ab060dec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.316+0000 7f83fda5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83f41a4f80 con 0x55ab060afc00 2024-09-11T00:59:45.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.316+0000 7f83fda5e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4075060 con 0x55ab0609f960 2024-09-11T00:59:45.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.316+0000 7f83fda5e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83f40722a0 con 0x55ab060af280 2024-09-11T00:59:45.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83fda5e740 10 client.0 _ll_register_callbacks cb 0x7f83ec00d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:45.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83f9c26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 0x55ab060dec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83f9c26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 0x55ab060dec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59056/0 (socket says 172.21.15.78:59056) 2024-09-11T00:59:45.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83f9c26640 1 -- 172.21.15.78:0/3442348171 learned_addr learned my addr 172.21.15.78:0/3442348171 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:45.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83f9425640 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab0609f960 0x55ab060d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83f8c24640 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060af280 0x55ab060d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83f9c26640 1 -- 172.21.15.78:0/3442348171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060af280 msgr2=0x55ab060d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83f9c26640 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060af280 0x55ab060d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83f9c26640 1 -- 172.21.15.78:0/3442348171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab0609f960 msgr2=0x55ab060d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83f9c26640 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab0609f960 0x55ab060d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83f9c26640 1 -- 172.21.15.78:0/3442348171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab060df480 con 0x55ab060afc00 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83f9c26640 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 0x55ab060dec50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f83ec004e10 tx=0x7f83ec00c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc63688b537b2ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.317+0000 7f83dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.318+0000 7f83dcff9640 1 -- 172.21.15.78:0/3442348171 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec02f020 con 0x55ab060afc00 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.318+0000 7f83dcff9640 1 -- 172.21.15.78:0/3442348171 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f83ec0050c0 con 0x55ab060afc00 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.318+0000 7f83fda5e740 1 -- 172.21.15.78:0/3442348171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f001fb00 con 0x55ab060afc00 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.318+0000 7f83dcff9640 1 -- 172.21.15.78:0/3442348171 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f83ec03a050 con 0x55ab060afc00 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.318+0000 7f83fda5e740 10 client.15672 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:45.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.318+0000 7f83fda5e740 1 -- 172.21.15.78:0/3442348171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ab060e0730 con 0x55ab060afc00 2024-09-11T00:59:45.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.319+0000 7f83dcff9640 1 -- 172.21.15.78:0/3442348171 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83f001fb00 con 0x55ab060afc00 2024-09-11T00:59:45.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.319+0000 7f83dcff9640 1 -- 172.21.15.78:0/3442348171 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f83ec016070 con 0x55ab060afc00 2024-09-11T00:59:45.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.319+0000 7f83fda5e740 10 client.15672 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:45.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.319+0000 7f83fda5e740 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T00:59:45.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.319+0000 7f83fda5e740 20 client.15672 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:45.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.319+0000 7f83fda5e740 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab061461e0 0x55ab06148600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.319+0000 7f83fda5e740 4 client.15672 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.319+0000 7f83fda5e740 1 -- 172.21.15.78:0/3442348171 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ab06016860 con 0x55ab061461e0 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.319+0000 7f83f9425640 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab061461e0 0x55ab06148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.320+0000 7f83dcff9640 10 client.15672 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.320+0000 7f83f9425640 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab061461e0 0x55ab06148600 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95b3f1f15ad4e969 server_cookie=e5ec4570ddd861ff in_seq=0 out_seq=0 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.321+0000 7f83dcff9640 1 -- 172.21.15.78:0/3442348171 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ab06016860 con 0x55ab061461e0 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.321+0000 7f83dcff9640 10 client.15672 handle_command_reply: tid=0 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.321+0000 7f83fda5e740 1 client.15672 shutdown 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.322+0000 7f83fda5e740 1 -- 172.21.15.78:0/3442348171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 msgr2=0x55ab060dec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.322+0000 7f83fda5e740 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 0x55ab060dec50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f83ec004e10 tx=0x7f83ec00c600 comp rx=0 tx=0).stop 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.322+0000 7f83fda5e740 1 -- 172.21.15.78:0/3442348171 shutdown_connections 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.322+0000 7f83fda5e740 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab061461e0 0x55ab06148600 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.322+0000 7f83fda5e740 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab060afc00 0x55ab060dec50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.322+0000 7f83fda5e740 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab060af280 0x55ab060d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.322+0000 7f83fda5e740 1 --2- 172.21.15.78:0/3442348171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab0609f960 0x55ab060d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.322+0000 7f83fda5e740 1 -- 172.21.15.78:0/3442348171 >> 172.21.15.78:0/3442348171 conn(0x55ab060a1e80 msgr2=0x55ab060ad9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:45.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.322+0000 7f83dcff9640 0 client.15672 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:45.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.322+0000 7f83fda5e740 1 -- 172.21.15.78:0/3442348171 shutdown_connections 2024-09-11T00:59:45.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.322+0000 7f83fda5e740 1 -- 172.21.15.78:0/3442348171 wait complete. 2024-09-11T00:59:45.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.322+0000 7f83fda5e740 20 client.15672 trim_cache size 0 max 16384 2024-09-11T00:59:45.326 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:59:45.326 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T00:59:45.326 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "e30a4af9-93f2-424c-b1fd-c56f7384a2df", 2024-09-11T00:59:45.327 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T00:59:45.327 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:59:45.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.323+0000 7f83fb689640 1 -- 172.21.15.78:0/1241861269 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f83e00b54d0 msgr2=0x7f83e00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:45.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.323+0000 7f83fb689640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f83e00b54d0 0x7f83e00b7990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f83f0005fd0 tx=0x7f83f0009680 comp rx=0 tx=0).stop 2024-09-11T00:59:45.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.323+0000 7f83fb689640 1 -- 172.21.15.78:0/1241861269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83f41076a0 msgr2=0x7f83f41986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:45.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.323+0000 7f83fb689640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83f41076a0 0x7f83f41986a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f83e4005940 tx=0x7f83e40042f0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.323+0000 7f83fb689640 1 -- 172.21.15.78:0/1241861269 shutdown_connections 2024-09-11T00:59:45.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.323+0000 7f83fb689640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f83e00b54d0 0x7f83e00b7990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.323+0000 7f83fb689640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f4108020 0x7f83f419da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.323+0000 7f83fb689640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83f41076a0 0x7f83f41986a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.323+0000 7f83fb689640 1 --2- 172.21.15.78:0/1241861269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83f4101c90 0x7f83f4198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:45.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.323+0000 7f83fb689640 1 -- 172.21.15.78:0/1241861269 >> 172.21.15.78:0/1241861269 conn(0x7f83f40fd700 msgr2=0x7f83f40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:45.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.323+0000 7f83fb689640 1 -- 172.21.15.78:0/1241861269 shutdown_connections 2024-09-11T00:59:45.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:45.323+0000 7f83fb689640 1 -- 172.21.15.78:0/1241861269 wait complete. 2024-09-11T00:59:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:45 smithi110 ceph-mon[34712]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:59:45.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:45 smithi112 ceph-mon[35384]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:59:46.144 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:46 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T00:59:46.144 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:46 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T00:59:46.144 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:46 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T00:59:46.144 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:46 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T00:59:46.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T00:59:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:59:46.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:46 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T00:59:46.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:46 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T00:59:46.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:46 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T00:59:46.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:46 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T00:59:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:46 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T00:59:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:46 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T00:59:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:46 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T00:59:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:46 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T00:59:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.976+0000 7f2635f4b640 1 -- 172.21.15.78:0/2703977455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2630104640 msgr2=0x7f263010cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.976+0000 7f2635f4b640 1 --2- 172.21.15.78:0/2703977455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2630104640 0x7f263010cb30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2624009900 tx=0x7f262402a410 comp rx=0 tx=0).stop 2024-09-11T00:59:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.976+0000 7f2635f4b640 1 -- 172.21.15.78:0/2703977455 shutdown_connections 2024-09-11T00:59:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.976+0000 7f2635f4b640 1 --2- 172.21.15.78:0/2703977455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f263010d180 0x7f263010f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.976+0000 7f2635f4b640 1 --2- 172.21.15.78:0/2703977455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2630104640 0x7f263010cb30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.976+0000 7f2635f4b640 1 --2- 172.21.15.78:0/2703977455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2630103d20 0x7f2630104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.976+0000 7f2635f4b640 1 -- 172.21.15.78:0/2703977455 >> 172.21.15.78:0/2703977455 conn(0x7f26300fd7c0 msgr2=0x7f26300ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.976+0000 7f2635f4b640 1 -- 172.21.15.78:0/2703977455 shutdown_connections 2024-09-11T00:59:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.976+0000 7f2635f4b640 1 -- 172.21.15.78:0/2703977455 wait complete. 2024-09-11T00:59:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.977+0000 7f2635f4b640 1 Processor -- start 2024-09-11T00:59:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.977+0000 7f2635f4b640 1 -- start start 2024-09-11T00:59:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.977+0000 7f2635f4b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2630103d20 0x7f26301992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.977+0000 7f2635f4b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2630104640 0x7f2630199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.977+0000 7f2635f4b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f263010d180 0x7f263019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.977+0000 7f2635f4b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2630112df0 con 0x7f2630104640 2024-09-11T00:59:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.977+0000 7f2635f4b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2630112c70 con 0x7f263010d180 2024-09-11T00:59:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.977+0000 7f2635f4b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2630112f70 con 0x7f2630103d20 2024-09-11T00:59:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.978+0000 7f262effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2630104640 0x7f2630199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.978+0000 7f262f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2630103d20 0x7f26301992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.978+0000 7f262effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2630104640 0x7f2630199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59076/0 (socket says 172.21.15.78:59076) 2024-09-11T00:59:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.978+0000 7f262effd640 1 -- 172.21.15.78:0/853551185 learned_addr learned my addr 172.21.15.78:0/853551185 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.978+0000 7f262ffff640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f263010d180 0x7f263019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.978+0000 7f262ffff640 1 -- 172.21.15.78:0/853551185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2630103d20 msgr2=0x7f26301992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.978+0000 7f262ffff640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2630103d20 0x7f26301992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.978+0000 7f262ffff640 1 -- 172.21.15.78:0/853551185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2630104640 msgr2=0x7f2630199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:46.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.978+0000 7f262ffff640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2630104640 0x7f2630199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.978+0000 7f262ffff640 1 -- 172.21.15.78:0/853551185 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263019f370 con 0x7f263010d180 2024-09-11T00:59:46.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.978+0000 7f262ffff640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f263010d180 0x7f263019eba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2628007db0 tx=0x7f2628010420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.979+0000 7f262cff9640 1 -- 172.21.15.78:0/853551185 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628010d40 con 0x7f263010d180 2024-09-11T00:59:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.979+0000 7f2635f4b640 1 -- 172.21.15.78:0/853551185 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263019f660 con 0x7f263010d180 2024-09-11T00:59:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.979+0000 7f2635f4b640 1 -- 172.21.15.78:0/853551185 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26301018b0 con 0x7f263010d180 2024-09-11T00:59:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.979+0000 7f262cff9640 1 -- 172.21.15.78:0/853551185 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f262800c640 con 0x7f263010d180 2024-09-11T00:59:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.979+0000 7f262cff9640 1 -- 172.21.15.78:0/853551185 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628017650 con 0x7f263010d180 2024-09-11T00:59:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.982+0000 7f2638320740 1 Processor -- start 2024-09-11T00:59:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.982+0000 7f262cff9640 1 -- 172.21.15.78:0/853551185 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2628016070 con 0x7f263010d180 2024-09-11T00:59:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.982+0000 7f2638320740 1 -- start start 2024-09-11T00:59:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.983+0000 7f2638320740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 0x56111d821860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.983+0000 7f2638320740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d82d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.983+0000 7f2638320740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d82dd30 0x56111d830120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.983+0000 7f2638320740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56111d797e50 con 0x56111d821480 2024-09-11T00:59:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.983+0000 7f2638320740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56111d787280 con 0x56111d82dd30 2024-09-11T00:59:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.983+0000 7f2638320740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56111d78b090 con 0x56111d821da0 2024-09-11T00:59:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.983+0000 7f262effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 0x56111d821860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.983+0000 7f262effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 0x56111d821860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59080/0 (socket says 172.21.15.78:59080) 2024-09-11T00:59:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.983+0000 7f262effd640 1 -- 172.21.15.78:0/2145987191 learned_addr learned my addr 172.21.15.78:0/2145987191 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.983+0000 7f262f7fe640 1 --2- 172.21.15.78:0/2145987191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d82d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.983+0000 7f262effd640 1 --2- 172.21.15.78:0/2145987191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d82dd30 0x56111d830120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.984+0000 7f262f7fe640 1 -- 172.21.15.78:0/2145987191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d82dd30 msgr2=0x56111d830120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.984+0000 7f262f7fe640 1 --2- 172.21.15.78:0/2145987191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d82dd30 0x56111d830120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.984+0000 7f262f7fe640 1 -- 172.21.15.78:0/2145987191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 msgr2=0x56111d821860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.984+0000 7f262f7fe640 1 --2- 172.21.15.78:0/2145987191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 0x56111d821860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.984+0000 7f262f7fe640 1 -- 172.21.15.78:0/2145987191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56111d4e5650 con 0x56111d821da0 2024-09-11T00:59:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.984+0000 7f262cff9640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26180b54d0 0x7f26180b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.984+0000 7f262f7fe640 1 --2- 172.21.15.78:0/2145987191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d82d7f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f262000bbf0 tx=0x7f26200107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f80080d27214c4df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.984+0000 7f262cff9640 1 -- 172.21.15.78:0/853551185 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f26280e9780 con 0x7f263010d180 2024-09-11T00:59:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.984+0000 7f262effd640 1 --2- 172.21.15.78:0/2145987191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 0x56111d821860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:59:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.984+0000 7f2615ffb640 1 -- 172.21.15.78:0/2145987191 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620004030 con 0x56111d821da0 2024-09-11T00:59:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.985+0000 7f2615ffb640 1 -- 172.21.15.78:0/2145987191 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26200035f0 con 0x56111d821da0 2024-09-11T00:59:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.985+0000 7f262effd640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26180b54d0 0x7f26180b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.985+0000 7f2615ffb640 1 -- 172.21.15.78:0/2145987191 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620011680 con 0x56111d821da0 2024-09-11T00:59:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.985+0000 7f2638320740 1 -- 172.21.15.78:0/2145987191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 msgr2=0x56111d82d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.985+0000 7f262effd640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26180b54d0 0x7f26180b7990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f26240096c0 tx=0x7f2624046040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.985+0000 7f2638320740 1 --2- 172.21.15.78:0/2145987191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d82d7f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f262000bbf0 tx=0x7f26200107a0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.986+0000 7f2638320740 1 -- 172.21.15.78:0/2145987191 shutdown_connections 2024-09-11T00:59:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.986+0000 7f2638320740 1 --2- 172.21.15.78:0/2145987191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d82dd30 0x56111d830120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.986+0000 7f2638320740 1 --2- 172.21.15.78:0/2145987191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d82d7f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.986+0000 7f2638320740 1 --2- 172.21.15.78:0/2145987191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 0x56111d821860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.986+0000 7f2638320740 1 -- 172.21.15.78:0/2145987191 >> 172.21.15.78:0/2145987191 conn(0x56111d81f2b0 msgr2=0x56111d820450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.986+0000 7f2638320740 1 -- 172.21.15.78:0/2145987191 shutdown_connections 2024-09-11T00:59:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.986+0000 7f2638320740 1 -- 172.21.15.78:0/2145987191 wait complete. 2024-09-11T00:59:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.986+0000 7f2638320740 1 -- start start 2024-09-11T00:59:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.987+0000 7f2638320740 1 Processor -- start 2024-09-11T00:59:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.987+0000 7f2638320740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 0x56111d82d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.987+0000 7f2638320740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821da0 0x56111d82a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.987+0000 7f2638320740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d82dd30 0x56111d82adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.987+0000 7f2638320740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56111d5e8700 con 0x56111d821da0 2024-09-11T00:59:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.987+0000 7f2638320740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56111d722110 con 0x56111d821480 2024-09-11T00:59:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.987+0000 7f2638320740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56111d830a00 con 0x56111d82dd30 2024-09-11T00:59:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.987+0000 7f2638320740 10 client.0 _ll_register_callbacks cb 0x56111d73b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.987+0000 7f262f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 0x56111d82d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.987+0000 7f262ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d82dd30 0x56111d82adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.987+0000 7f262f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 0x56111d82d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:50792/0 (socket says 172.21.15.78:50792) 2024-09-11T00:59:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.987+0000 7f262f7fe640 1 -- 172.21.15.78:0/2847383117 learned_addr learned my addr 172.21.15.78:0/2847383117 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.988+0000 7f262f7fe640 1 -- 172.21.15.78:0/2847383117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d82dd30 msgr2=0x56111d82adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.988+0000 7f262f7fe640 1 --2- 172.21.15.78:0/2847383117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d82dd30 0x56111d82adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.988+0000 7f262f7fe640 1 -- 172.21.15.78:0/2847383117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821da0 msgr2=0x56111d82a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.988+0000 7f262f7fe640 1 --2- 172.21.15.78:0/2847383117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821da0 0x56111d82a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.988+0000 7f262f7fe640 1 -- 172.21.15.78:0/2847383117 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56111d74fc40 con 0x56111d821480 2024-09-11T00:59:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.988+0000 7f262f7fe640 1 --2- 172.21.15.78:0/2847383117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 0x56111d82d460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2620003ee0 tx=0x7f262000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8979fcd5a9a6aec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.988+0000 7f260effd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T00:59:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.988+0000 7f260effd640 1 -- 172.21.15.78:0/2847383117 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620003fe0 con 0x56111d821480 2024-09-11T00:59:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.988+0000 7f2638320740 1 -- 172.21.15.78:0/2847383117 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26200035f0 con 0x56111d821480 2024-09-11T00:59:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.988+0000 7f260effd640 1 -- 172.21.15.78:0/2847383117 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f2620004180 con 0x56111d821480 2024-09-11T00:59:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.989+0000 7f260effd640 1 -- 172.21.15.78:0/2847383117 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2620032050 con 0x56111d821480 2024-09-11T00:59:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.989+0000 7f2638320740 10 client.25115 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.989+0000 7f2638320740 1 -- 172.21.15.78:0/2847383117 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56111d83c260 con 0x56111d821480 2024-09-11T00:59:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.989+0000 7f260effd640 1 -- 172.21.15.78:0/2847383117 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26200035f0 con 0x56111d821480 2024-09-11T00:59:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.990+0000 7f260effd640 1 -- 172.21.15.78:0/2847383117 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2620005040 con 0x56111d821480 2024-09-11T00:59:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.990+0000 7f2638320740 10 client.25115 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.990+0000 7f2638320740 10 client.25115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T00:59:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.990+0000 7f2638320740 20 client.25115 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.990+0000 7f2638320740 1 --2- 172.21.15.78:0/2847383117 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56111d83dab0 0x56111d83ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.990+0000 7f2638320740 4 client.25115 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:59:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.990+0000 7f2638320740 1 -- 172.21.15.78:0/2847383117 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56111d799da0 con 0x56111d83dab0 2024-09-11T00:59:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.990+0000 7f262effd640 1 --2- 172.21.15.78:0/2847383117 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56111d83dab0 0x56111d83ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.991+0000 7f260effd640 10 client.25115 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.991+0000 7f262effd640 1 --2- 172.21.15.78:0/2847383117 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56111d83dab0 0x56111d83ff70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7ed1945de7a9807 server_cookie=39074b64b7b137fe in_seq=0 out_seq=0 2024-09-11T00:59:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.991+0000 7f260effd640 1 -- 172.21.15.78:0/2847383117 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56111d799da0 con 0x56111d83dab0 2024-09-11T00:59:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.991+0000 7f260effd640 10 client.25115 handle_command_reply: tid=0 2024-09-11T00:59:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.991+0000 7f2638320740 1 client.25115 shutdown 2024-09-11T00:59:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.992+0000 7f2638320740 1 -- 172.21.15.78:0/2847383117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 msgr2=0x56111d82d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.992+0000 7f2638320740 1 --2- 172.21.15.78:0/2847383117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 0x56111d82d460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2620003ee0 tx=0x7f262000bfd0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.992+0000 7f2638320740 1 -- 172.21.15.78:0/2847383117 shutdown_connections 2024-09-11T00:59:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.992+0000 7f2638320740 1 --2- 172.21.15.78:0/2847383117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d82dd30 0x56111d82adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.992+0000 7f2638320740 1 --2- 172.21.15.78:0/2847383117 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56111d83dab0 0x56111d83ff70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.992+0000 7f2638320740 1 --2- 172.21.15.78:0/2847383117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821da0 0x56111d82a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.992+0000 7f2638320740 1 --2- 172.21.15.78:0/2847383117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 0x56111d82d460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.992+0000 7f260effd640 0 client.25115 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:47.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.992+0000 7f2638320740 1 -- 172.21.15.78:0/2847383117 >> 172.21.15.78:0/2847383117 conn(0x56111d81f2b0 msgr2=0x56111d824e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:47.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.992+0000 7f2638320740 1 -- 172.21.15.78:0/2847383117 shutdown_connections 2024-09-11T00:59:47.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.992+0000 7f2638320740 1 -- 172.21.15.78:0/2847383117 wait complete. 2024-09-11T00:59:47.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:46.992+0000 7f2638320740 20 client.25115 trim_cache size 0 max 16384 2024-09-11T00:59:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f2638320740 1 Processor -- start 2024-09-11T00:59:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f2638320740 1 -- start start 2024-09-11T00:59:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f2638320740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 0x56111d8206a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f2638320740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d827b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f2638320740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d82dd30 0x56111d828060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f2638320740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f261c00e620 con 0x56111d82dd30 2024-09-11T00:59:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f2638320740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f261c010160 con 0x56111d821480 2024-09-11T00:59:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f2638320740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f261c00ecf0 con 0x56111d821da0 2024-09-11T00:59:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f262ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d82dd30 0x56111d828060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f262ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d82dd30 0x56111d828060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59096/0 (socket says 172.21.15.78:59096) 2024-09-11T00:59:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f262ffff640 1 -- 172.21.15.78:0/47829516 learned_addr learned my addr 172.21.15.78:0/47829516 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f262f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 0x56111d8206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f262effd640 1 --2- 172.21.15.78:0/47829516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d827b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f262f7fe640 1 -- 172.21.15.78:0/47829516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 msgr2=0x56111d827b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f262f7fe640 1 --2- 172.21.15.78:0/47829516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d827b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f262f7fe640 1 -- 172.21.15.78:0/47829516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d82dd30 msgr2=0x56111d828060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f262f7fe640 1 --2- 172.21.15.78:0/47829516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d82dd30 0x56111d828060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.010+0000 7f262f7fe640 1 -- 172.21.15.78:0/47829516 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56111d81fd30 con 0x56111d821480 2024-09-11T00:59:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.011+0000 7f262f7fe640 1 --2- 172.21.15.78:0/47829516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 0x56111d8206a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f262000b620 tx=0x7f26200058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fce5f87ac8b4c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.011+0000 7f26157fa640 1 -- 172.21.15.78:0/47829516 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262003a070 con 0x56111d821480 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.011+0000 7f26157fa640 1 -- 172.21.15.78:0/47829516 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2620029070 con 0x56111d821480 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.011+0000 7f26157fa640 1 -- 172.21.15.78:0/47829516 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26200118f0 con 0x56111d821480 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.012+0000 7f2638320740 1 -- 172.21.15.78:0/47829516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 msgr2=0x56111d8206a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.012+0000 7f2638320740 1 --2- 172.21.15.78:0/47829516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 0x56111d8206a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f262000b620 tx=0x7f26200058d0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.012+0000 7f2638320740 1 -- 172.21.15.78:0/47829516 shutdown_connections 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.012+0000 7f2638320740 1 --2- 172.21.15.78:0/47829516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d82dd30 0x56111d828060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.012+0000 7f2638320740 1 --2- 172.21.15.78:0/47829516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d827b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.012+0000 7f2638320740 1 --2- 172.21.15.78:0/47829516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d821480 0x56111d8206a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.012+0000 7f2638320740 1 -- 172.21.15.78:0/47829516 >> 172.21.15.78:0/47829516 conn(0x56111d81f2b0 msgr2=0x56111d81ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.012+0000 7f2638320740 1 -- 172.21.15.78:0/47829516 shutdown_connections 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.012+0000 7f2638320740 1 -- 172.21.15.78:0/47829516 wait complete. 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.012+0000 7f2638320740 1 -- start start 2024-09-11T00:59:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f2638320740 1 Processor -- start 2024-09-11T00:59:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f2638320740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 0x56111d835fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f2638320740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d836540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f2638320740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d82dd30 0x56111d827260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f2638320740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f261c00fa90 con 0x56111d821480 2024-09-11T00:59:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f2638320740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f261c00f3c0 con 0x56111d82dd30 2024-09-11T00:59:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f2638320740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f261c00c600 con 0x56111d821da0 2024-09-11T00:59:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f2638320740 10 client.0 _ll_register_callbacks cb 0x7f2624009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:59:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f262f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 0x56111d835fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f262f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 0x56111d835fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59110/0 (socket says 172.21.15.78:59110) 2024-09-11T00:59:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f262f7fe640 1 -- 172.21.15.78:0/53135860 learned_addr learned my addr 172.21.15.78:0/53135860 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T00:59:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f262ffff640 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d82dd30 0x56111d827260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.013+0000 7f262effd640 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d836540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f262ffff640 1 -- 172.21.15.78:0/53135860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 msgr2=0x56111d836540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f262ffff640 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d836540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f262ffff640 1 -- 172.21.15.78:0/53135860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 msgr2=0x56111d835fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f262ffff640 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 0x56111d835fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f262ffff640 1 -- 172.21.15.78:0/53135860 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56111d82b800 con 0x56111d82dd30 2024-09-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f262effd640 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d836540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f262f7fe640 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 0x56111d835fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f262ffff640 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d82dd30 0x56111d827260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26280e6ff0 tx=0x7f26280af6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a017089a114ad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f260effd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f260effd640 1 -- 172.21.15.78:0/53135860 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26280b0370 con 0x56111d82dd30 2024-09-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f260effd640 1 -- 172.21.15.78:0/53135860 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f26280b4210 con 0x56111d82dd30 2024-09-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f2638320740 1 -- 172.21.15.78:0/53135860 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2620029070 con 0x56111d82dd30 2024-09-11T00:59:47.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.014+0000 7f260effd640 1 -- 172.21.15.78:0/53135860 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f26280970b0 con 0x56111d82dd30 2024-09-11T00:59:47.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.015+0000 7f2638320740 10 client.25121 fetch_fsmap learned FSMap version 29 2024-09-11T00:59:47.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.015+0000 7f2638320740 1 -- 172.21.15.78:0/53135860 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56111d8aa390 con 0x56111d82dd30 2024-09-11T00:59:47.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.015+0000 7f260effd640 1 -- 172.21.15.78:0/53135860 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26280afc70 con 0x56111d82dd30 2024-09-11T00:59:47.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.015+0000 7f260effd640 1 -- 172.21.15.78:0/53135860 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f26280ab6c0 con 0x56111d82dd30 2024-09-11T00:59:47.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.015+0000 7f2638320740 10 client.25121 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:59:47.024 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T00:59:47.025 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T00:59:47.025 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T00:59:47.025 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T00:59:47.025 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T00:59:47.025 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "e30a4af9-93f2-424c-b1fd-c56f7384a2df", 2024-09-11T00:59:47.025 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T00:59:47.025 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T00:59:47.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T00:59:47.025 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T00:59:47.025 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T00:59:47.025 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T00:59:47.025 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T00:59:47.026 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T00:59:47.027 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T00:59:47.027 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T00:59:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.015+0000 7f2638320740 10 client.25121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T00:59:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.015+0000 7f2638320740 20 client.25121 populate_metadata read hostname 'smithi078' 2024-09-11T00:59:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.015+0000 7f2638320740 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56111d8aaae0 0x56111d8acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:59:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.015+0000 7f2638320740 4 client.25121 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T00:59:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.015+0000 7f2638320740 1 -- 172.21.15.78:0/53135860 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f2620029070 con 0x56111d8aaae0 2024-09-11T00:59:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.015+0000 7f262f7fe640 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56111d8aaae0 0x56111d8acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:59:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f260effd640 10 client.25121 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f262f7fe640 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56111d8aaae0 0x56111d8acf00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f48256595fe73a3 server_cookie=ed8bf72b49f3f7a in_seq=0 out_seq=0 2024-09-11T00:59:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f260effd640 1 -- 172.21.15.78:0/53135860 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x7f2620029070 con 0x56111d8aaae0 2024-09-11T00:59:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f260effd640 10 client.25121 handle_command_reply: tid=0 2024-09-11T00:59:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f2638320740 1 client.25121 shutdown 2024-09-11T00:59:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f2638320740 1 -- 172.21.15.78:0/53135860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d82dd30 msgr2=0x56111d827260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f2638320740 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d82dd30 0x56111d827260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26280e6ff0 tx=0x7f26280af6f0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f2638320740 1 -- 172.21.15.78:0/53135860 shutdown_connections 2024-09-11T00:59:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f2638320740 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56111d82dd30 0x56111d827260 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f2638320740 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111d821da0 0x56111d836540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f2638320740 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56111d8aaae0 0x56111d8acf00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f2638320740 1 --2- 172.21.15.78:0/53135860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56111d821480 0x56111d835fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f2638320740 1 -- 172.21.15.78:0/53135860 >> 172.21.15.78:0/53135860 conn(0x56111d81f2b0 msgr2=0x56111d8207b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f260effd640 0 client.25121 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T00:59:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f2638320740 1 -- 172.21.15.78:0/53135860 shutdown_connections 2024-09-11T00:59:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f2638320740 1 -- 172.21.15.78:0/53135860 wait complete. 2024-09-11T00:59:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.016+0000 7f2638320740 20 client.25121 trim_cache size 0 max 16384 2024-09-11T00:59:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.017+0000 7f2635f4b640 1 -- 172.21.15.78:0/853551185 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26180b54d0 msgr2=0x7f26180b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.017+0000 7f2635f4b640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26180b54d0 0x7f26180b7990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f26240096c0 tx=0x7f2624046040 comp rx=0 tx=0).stop 2024-09-11T00:59:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.017+0000 7f2635f4b640 1 -- 172.21.15.78:0/853551185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f263010d180 msgr2=0x7f263019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:59:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.017+0000 7f2635f4b640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f263010d180 0x7f263019eba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2628007db0 tx=0x7f2628010420 comp rx=0 tx=0).stop 2024-09-11T00:59:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.017+0000 7f2635f4b640 1 -- 172.21.15.78:0/853551185 shutdown_connections 2024-09-11T00:59:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.017+0000 7f2635f4b640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26180b54d0 0x7f26180b7990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.017+0000 7f2635f4b640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f263010d180 0x7f263019eba0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.017+0000 7f2635f4b640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2630104640 0x7f2630199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.017+0000 7f2635f4b640 1 --2- 172.21.15.78:0/853551185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2630103d20 0x7f26301992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:59:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.017+0000 7f2635f4b640 1 -- 172.21.15.78:0/853551185 >> 172.21.15.78:0/853551185 conn(0x7f26300fd7c0 msgr2=0x7f263010f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:59:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.017+0000 7f2635f4b640 1 -- 172.21.15.78:0/853551185 shutdown_connections 2024-09-11T00:59:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T00:59:47.017+0000 7f2635f4b640 1 -- 172.21.15.78:0/853551185 wait complete. 2024-09-11T00:59:47.404 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:47 smithi078 ceph-mon[31315]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:59:47.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:47 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T00:59:47.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:47 smithi112 ceph-mon[35384]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:59:47.505 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:47 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T00:59:47.506 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:47 smithi110 ceph-mon[34712]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:59:47.506 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:47 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T00:59:47.780 INFO:tasks.cephfs.filesystem:scrub status for tag:e30a4af9-93f2-424c-b1fd-c56f7384a2df - {'path': '/', 'tag': 'e30a4af9-93f2-424c-b1fd-c56f7384a2df', 'options': 'recursive,force'} 2024-09-11T00:59:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:48 smithi110 ceph-mon[34712]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:59:48.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:48 smithi112 ceph-mon[35384]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:59:48.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:48 smithi078 ceph-mon[31315]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T00:59:51.267 INFO:tasks.workunit.client.0.smithi078.stderr:Note: switching to '8f219c476cb4f26fc5719c661c0323719a45cb30'. 2024-09-11T00:59:51.267 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-09-11T00:59:51.267 INFO:tasks.workunit.client.0.smithi078.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T00:59:51.267 INFO:tasks.workunit.client.0.smithi078.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T00:59:51.267 INFO:tasks.workunit.client.0.smithi078.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T00:59:51.267 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-09-11T00:59:51.268 INFO:tasks.workunit.client.0.smithi078.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T00:59:51.268 INFO:tasks.workunit.client.0.smithi078.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T00:59:51.268 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-09-11T00:59:51.268 INFO:tasks.workunit.client.0.smithi078.stderr: git switch -c 2024-09-11T00:59:51.268 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-09-11T00:59:51.268 INFO:tasks.workunit.client.0.smithi078.stderr:Or undo this operation with: 2024-09-11T00:59:51.268 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-09-11T00:59:51.268 INFO:tasks.workunit.client.0.smithi078.stderr: git switch - 2024-09-11T00:59:51.268 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-09-11T00:59:51.268 INFO:tasks.workunit.client.0.smithi078.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T00:59:51.268 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-09-11T00:59:51.268 INFO:tasks.workunit.client.0.smithi078.stderr:HEAD is now at 8f219c476cb Merge branch 'mgr-vol-mod-disable' of https://github.com/rishabh-d-dave/ceph into wip-hemanth-testing-2024-09-10-0723 2024-09-11T00:59:51.272 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T00:59:51.300 INFO:tasks.workunit.client.0.smithi078.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T00:59:51.302 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T00:59:51.303 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T00:59:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:51 smithi110 ceph-mon[34712]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-09-11T00:59:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:51 smithi112 ceph-mon[35384]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-09-11T00:59:51.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:51 smithi078 ceph-mon[31315]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-09-11T00:59:52.032 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T00:59:52.100 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T00:59:52.142 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T00:59:52.143 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T00:59:52.144 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T00:59:52.182 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T00:59:52.185 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T00:59:52.186 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T00:59:52.242 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-11T00:59:52.243 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-11T00:59:52.243 DEBUG:teuthology.orchestra.run.smithi078:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f219c476cb4f26fc5719c661c0323719a45cb30 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-11T00:59:52.303 INFO:tasks.workunit.client.0.smithi078.stderr:+ dbench 1 2024-09-11T00:59:52.312 INFO:tasks.workunit.client.0.smithi078.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-11T00:59:52.313 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-09-11T00:59:52.313 INFO:tasks.workunit.client.0.smithi078.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-11T00:59:52.316 INFO:tasks.workunit.client.0.smithi078.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-11T00:59:52.416 INFO:tasks.workunit.client.0.smithi078.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-11T00:59:52.416 INFO:tasks.workunit.client.0.smithi078.stdout:releasing clients 2024-09-11T00:59:53.416 INFO:tasks.workunit.client.0.smithi078.stdout: 1 3627 38.60 MB/sec warmup 1 sec latency 8.867 ms 2024-09-11T00:59:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:53 smithi110 ceph-mon[34712]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-11T00:59:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:53 smithi112 ceph-mon[35384]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-11T00:59:53.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:53 smithi078 ceph-mon[31315]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-11T00:59:54.416 INFO:tasks.workunit.client.0.smithi078.stdout: 1 6860 27.24 MB/sec warmup 2 sec latency 10.295 ms 2024-09-11T00:59:55.416 INFO:tasks.workunit.client.0.smithi078.stdout: 1 9847 23.40 MB/sec warmup 3 sec latency 10.412 ms 2024-09-11T00:59:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:55 smithi110 ceph-mon[34712]: pgmap v354: 97 pgs: 97 active+clean; 1.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-11T00:59:55.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:55 smithi112 ceph-mon[35384]: pgmap v354: 97 pgs: 97 active+clean; 1.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-11T00:59:55.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:55 smithi078 ceph-mon[31315]: pgmap v354: 97 pgs: 97 active+clean; 1.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-11T00:59:56.416 INFO:tasks.workunit.client.0.smithi078.stdout: 1 13054 21.33 MB/sec warmup 4 sec latency 12.843 ms 2024-09-11T00:59:57.416 INFO:tasks.workunit.client.0.smithi078.stdout: 1 15714 20.51 MB/sec warmup 5 sec latency 11.324 ms 2024-09-11T00:59:57.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:57 smithi110 ceph-mon[34712]: pgmap v355: 97 pgs: 97 active+clean; 4.2 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 284 KiB/s wr, 48 op/s 2024-09-11T00:59:57.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:57 smithi112 ceph-mon[35384]: pgmap v355: 97 pgs: 97 active+clean; 4.2 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 284 KiB/s wr, 48 op/s 2024-09-11T00:59:57.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:57 smithi078 ceph-mon[31315]: pgmap v355: 97 pgs: 97 active+clean; 4.2 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 284 KiB/s wr, 48 op/s 2024-09-11T00:59:58.416 INFO:tasks.workunit.client.0.smithi078.stdout: 1 18851 19.76 MB/sec warmup 6 sec latency 10.753 ms 2024-09-11T00:59:59.416 INFO:tasks.workunit.client.0.smithi078.stdout: 1 22114 19.45 MB/sec warmup 7 sec latency 12.317 ms 2024-09-11T00:59:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 00:59:59 smithi110 ceph-mon[34712]: pgmap v356: 97 pgs: 97 active+clean; 4.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 321 KiB/s wr, 48 op/s 2024-09-11T00:59:59.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 00:59:59 smithi112 ceph-mon[35384]: pgmap v356: 97 pgs: 97 active+clean; 4.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 321 KiB/s wr, 48 op/s 2024-09-11T00:59:59.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 00:59:59 smithi078 ceph-mon[31315]: pgmap v356: 97 pgs: 97 active+clean; 4.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 321 KiB/s wr, 48 op/s 2024-09-11T01:00:00.416 INFO:tasks.workunit.client.0.smithi078.stdout: 1 25325 19.36 MB/sec warmup 8 sec latency 17.538 ms 2024-09-11T01:00:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:00 smithi110 ceph-mon[34712]: overall HEALTH_OK 2024-09-11T01:00:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:00 smithi112 ceph-mon[35384]: overall HEALTH_OK 2024-09-11T01:00:00.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:00 smithi078 ceph-mon[31315]: overall HEALTH_OK 2024-09-11T01:00:01.416 INFO:tasks.workunit.client.0.smithi078.stdout: 1 27890 18.60 MB/sec warmup 9 sec latency 19.429 ms 2024-09-11T01:00:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:01 smithi110 ceph-mon[34712]: pgmap v357: 97 pgs: 97 active+clean; 8.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 720 KiB/s wr, 95 op/s 2024-09-11T01:00:01.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:01 smithi112 ceph-mon[35384]: pgmap v357: 97 pgs: 97 active+clean; 8.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 720 KiB/s wr, 95 op/s 2024-09-11T01:00:01.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:01 smithi078 ceph-mon[31315]: pgmap v357: 97 pgs: 97 active+clean; 8.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 720 KiB/s wr, 95 op/s 2024-09-11T01:00:02.416 INFO:tasks.workunit.client.0.smithi078.stdout: 1 30999 18.20 MB/sec warmup 10 sec latency 10.439 ms 2024-09-11T01:00:03.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 34342 18.19 MB/sec warmup 11 sec latency 10.065 ms 2024-09-11T01:00:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:03 smithi110 ceph-mon[34712]: pgmap v358: 97 pgs: 97 active+clean; 14 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-09-11T01:00:03.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:03 smithi112 ceph-mon[35384]: pgmap v358: 97 pgs: 97 active+clean; 14 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-09-11T01:00:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:03 smithi078 ceph-mon[31315]: pgmap v358: 97 pgs: 97 active+clean; 14 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-09-11T01:00:04.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 36746 17.86 MB/sec warmup 12 sec latency 13.426 ms 2024-09-11T01:00:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:00:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:04 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:00:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:04 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:00:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:04 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:05.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 39958 17.85 MB/sec warmup 13 sec latency 10.484 ms 2024-09-11T01:00:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:05 smithi110 ceph-mon[34712]: pgmap v359: 97 pgs: 97 active+clean; 16 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-09-11T01:00:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:05 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:00:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:05 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:00:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:05 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:05 smithi112 ceph-mon[35384]: pgmap v359: 97 pgs: 97 active+clean; 16 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-09-11T01:00:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:05 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:00:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:05 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:00:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:05 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:05.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:05 smithi078 ceph-mon[31315]: pgmap v359: 97 pgs: 97 active+clean; 16 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-09-11T01:00:05.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:05 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:00:05.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:05 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:00:05.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:05 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:00:06.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 43155 17.89 MB/sec warmup 14 sec latency 12.538 ms 2024-09-11T01:00:07.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 46527 17.97 MB/sec warmup 15 sec latency 15.303 ms 2024-09-11T01:00:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:07 smithi110 ceph-mon[34712]: pgmap v360: 97 pgs: 97 active+clean; 19 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.7 MiB/s wr, 235 op/s 2024-09-11T01:00:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:07 smithi112 ceph-mon[35384]: pgmap v360: 97 pgs: 97 active+clean; 19 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.7 MiB/s wr, 235 op/s 2024-09-11T01:00:07.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:07 smithi078 ceph-mon[31315]: pgmap v360: 97 pgs: 97 active+clean; 19 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.7 MiB/s wr, 235 op/s 2024-09-11T01:00:08.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 49828 17.96 MB/sec warmup 16 sec latency 22.048 ms 2024-09-11T01:00:09.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 53052 17.83 MB/sec warmup 17 sec latency 10.923 ms 2024-09-11T01:00:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:09 smithi110 ceph-mon[34712]: pgmap v361: 97 pgs: 97 active+clean; 19 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-11T01:00:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:09 smithi112 ceph-mon[35384]: pgmap v361: 97 pgs: 97 active+clean; 19 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-11T01:00:09.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:09 smithi078 ceph-mon[31315]: pgmap v361: 97 pgs: 97 active+clean; 19 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-11T01:00:10.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 56157 17.80 MB/sec warmup 18 sec latency 15.990 ms 2024-09-11T01:00:11.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 59333 17.62 MB/sec warmup 19 sec latency 11.011 ms 2024-09-11T01:00:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:11 smithi110 ceph-mon[34712]: pgmap v362: 97 pgs: 97 active+clean; 27 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 302 op/s 2024-09-11T01:00:11.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:11 smithi112 ceph-mon[35384]: pgmap v362: 97 pgs: 97 active+clean; 27 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 302 op/s 2024-09-11T01:00:11.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:11 smithi078 ceph-mon[31315]: pgmap v362: 97 pgs: 97 active+clean; 27 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 302 op/s 2024-09-11T01:00:12.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 62155 17.57 MB/sec warmup 20 sec latency 11.941 ms 2024-09-11T01:00:13.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 65254 17.63 MB/sec warmup 21 sec latency 17.528 ms 2024-09-11T01:00:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:13 smithi110 ceph-mon[34712]: pgmap v363: 97 pgs: 97 active+clean; 29 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 266 op/s 2024-09-11T01:00:13.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:13 smithi112 ceph-mon[35384]: pgmap v363: 97 pgs: 97 active+clean; 29 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 266 op/s 2024-09-11T01:00:13.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:13 smithi078 ceph-mon[31315]: pgmap v363: 97 pgs: 97 active+clean; 29 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 266 op/s 2024-09-11T01:00:14.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 68165 17.48 MB/sec warmup 22 sec latency 29.464 ms 2024-09-11T01:00:15.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 71407 17.54 MB/sec warmup 23 sec latency 7.551 ms 2024-09-11T01:00:15.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:15 smithi110 ceph-mon[34712]: pgmap v364: 97 pgs: 97 active+clean; 29 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-09-11T01:00:15.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:15 smithi112 ceph-mon[35384]: pgmap v364: 97 pgs: 97 active+clean; 29 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-09-11T01:00:15.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:15 smithi078 ceph-mon[31315]: pgmap v364: 97 pgs: 97 active+clean; 29 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-09-11T01:00:16.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 74836 17.59 MB/sec warmup 24 sec latency 10.639 ms 2024-09-11T01:00:17.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 77915 17.48 MB/sec warmup 25 sec latency 21.026 ms 2024-09-11T01:00:17.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:17 smithi110 ceph-mon[34712]: pgmap v365: 97 pgs: 97 active+clean; 37 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 269 op/s 2024-09-11T01:00:17.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:17 smithi112 ceph-mon[35384]: pgmap v365: 97 pgs: 97 active+clean; 37 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 269 op/s 2024-09-11T01:00:17.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:17 smithi078 ceph-mon[31315]: pgmap v365: 97 pgs: 97 active+clean; 37 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 269 op/s 2024-09-11T01:00:17.781 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:00:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:00:18.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 80682 17.30 MB/sec warmup 26 sec latency 30.980 ms 2024-09-11T01:00:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:18 smithi110 ceph-mon[34712]: pgmap v366: 97 pgs: 97 active+clean; 37 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 229 op/s 2024-09-11T01:00:18.459 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:18 smithi078 ceph-mon[31315]: pgmap v366: 97 pgs: 97 active+clean; 37 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 229 op/s 2024-09-11T01:00:18.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:18 smithi112 ceph-mon[35384]: pgmap v366: 97 pgs: 97 active+clean; 37 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 229 op/s 2024-09-11T01:00:19.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.150+0000 7f69733e4640 1 -- 172.21.15.78:0/4250051401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f696c103ca0 msgr2=0x7f696c104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.150+0000 7f69733e4640 1 --2- 172.21.15.78:0/4250051401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f696c103ca0 0x7f696c104080 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6968009900 tx=0x7f696802a410 comp rx=0 tx=0).stop 2024-09-11T01:00:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.150+0000 7f69733e4640 1 -- 172.21.15.78:0/4250051401 shutdown_connections 2024-09-11T01:00:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.150+0000 7f69733e4640 1 --2- 172.21.15.78:0/4250051401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f696c10d100 0x7f696c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.150+0000 7f69733e4640 1 --2- 172.21.15.78:0/4250051401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f696c1045c0 0x7f696c10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.150+0000 7f69733e4640 1 --2- 172.21.15.78:0/4250051401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f696c103ca0 0x7f696c104080 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.150+0000 7f69733e4640 1 -- 172.21.15.78:0/4250051401 >> 172.21.15.78:0/4250051401 conn(0x7f696c0fd700 msgr2=0x7f696c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.151+0000 7f69733e4640 1 -- 172.21.15.78:0/4250051401 shutdown_connections 2024-09-11T01:00:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.151+0000 7f69733e4640 1 -- 172.21.15.78:0/4250051401 wait complete. 2024-09-11T01:00:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.151+0000 7f69733e4640 1 Processor -- start 2024-09-11T01:00:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.151+0000 7f69733e4640 1 -- start start 2024-09-11T01:00:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.151+0000 7f69733e4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f696c103ca0 0x7f696c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.151+0000 7f69733e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f696c1045c0 0x7f696c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.151+0000 7f69733e4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f696c10d100 0x7f696c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.151+0000 7f69733e4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f696c112d70 con 0x7f696c10d100 2024-09-11T01:00:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.151+0000 7f69733e4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f696c112bf0 con 0x7f696c103ca0 2024-09-11T01:00:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.151+0000 7f69733e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f696c112ef0 con 0x7f696c1045c0 2024-09-11T01:00:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.151+0000 7f6971981640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f696c10d100 0x7f696c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.151+0000 7f697097f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f696c1045c0 0x7f696c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f6971981640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f696c10d100 0x7f696c19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52658/0 (socket says 172.21.15.78:52658) 2024-09-11T01:00:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f6971981640 1 -- 172.21.15.78:0/2476509906 learned_addr learned my addr 172.21.15.78:0/2476509906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f6971180640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f696c103ca0 0x7f696c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f6971981640 1 -- 172.21.15.78:0/2476509906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f696c1045c0 msgr2=0x7f696c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f6971981640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f696c1045c0 0x7f696c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f6971981640 1 -- 172.21.15.78:0/2476509906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f696c103ca0 msgr2=0x7f696c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f6971981640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f696c103ca0 0x7f696c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f6971981640 1 -- 172.21.15.78:0/2476509906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c19f330 con 0x7f696c10d100 2024-09-11T01:00:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f697097f640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f696c1045c0 0x7f696c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:00:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f6971981640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f696c10d100 0x7f696c19eb60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6964009fd0 tx=0x7f696400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f69627fc640 1 -- 172.21.15.78:0/2476509906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964017070 con 0x7f696c10d100 2024-09-11T01:00:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f69627fc640 1 -- 172.21.15.78:0/2476509906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6964003e20 con 0x7f696c10d100 2024-09-11T01:00:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f69733e4640 1 -- 172.21.15.78:0/2476509906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696c19f620 con 0x7f696c10d100 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f69627fc640 1 -- 172.21.15.78:0/2476509906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964010460 con 0x7f696c10d100 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.152+0000 7f69733e4640 1 -- 172.21.15.78:0/2476509906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696c1a5ce0 con 0x7f696c10d100 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.155+0000 7f69757b9740 1 Processor -- start 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.156+0000 7f69627fc640 1 -- 172.21.15.78:0/2476509906 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6964003970 con 0x7f696c10d100 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.156+0000 7f69757b9740 1 -- start start 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.156+0000 7f69757b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 0x55b664113860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.156+0000 7f69757b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b66411d7e0 0x55b664113da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.156+0000 7f69757b9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6641283e0 0x55b66412a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.156+0000 7f69757b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b664089e50 con 0x55b66411d7e0 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.156+0000 7f69757b9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b664079280 con 0x55b6641283e0 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.156+0000 7f69757b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b66407d090 con 0x55b664113480 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.156+0000 7f697097f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 0x55b664113860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.156+0000 7f697097f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 0x55b664113860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42726/0 (socket says 172.21.15.78:42726) 2024-09-11T01:00:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.156+0000 7f697097f640 1 -- 172.21.15.78:0/579700100 learned_addr learned my addr 172.21.15.78:0/579700100 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:19.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.156+0000 7f697097f640 1 --2- 172.21.15.78:0/579700100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6641283e0 0x55b66412a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.157+0000 7f697097f640 1 -- 172.21.15.78:0/579700100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6641283e0 msgr2=0x55b66412a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:19.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.157+0000 7f697097f640 1 --2- 172.21.15.78:0/579700100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6641283e0 0x55b66412a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.157+0000 7f697097f640 1 -- 172.21.15.78:0/579700100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b66411d7e0 msgr2=0x55b664113da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:00:19.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.157+0000 7f697097f640 1 --2- 172.21.15.78:0/579700100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b66411d7e0 0x55b664113da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.157+0000 7f697097f640 1 -- 172.21.15.78:0/579700100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b663dd7650 con 0x55b664113480 2024-09-11T01:00:19.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.157+0000 7f69627fc640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69580b5450 0x7f69580b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.157+0000 7f697097f640 1 --2- 172.21.15.78:0/579700100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 0x55b664113860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f695c013990 tx=0x7f695c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96557b90b2f350aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:19.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.157+0000 7f6971180640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69580b5450 0x7f69580b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.157+0000 7f69577fe640 1 -- 172.21.15.78:0/579700100 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c004030 con 0x55b664113480 2024-09-11T01:00:19.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.157+0000 7f69577fe640 1 -- 172.21.15.78:0/579700100 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f695c00f8c0 con 0x55b664113480 2024-09-11T01:00:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.157+0000 7f69577fe640 1 -- 172.21.15.78:0/579700100 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c0039e0 con 0x55b664113480 2024-09-11T01:00:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.157+0000 7f69627fc640 1 -- 172.21.15.78:0/2476509906 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f69640e5cc0 con 0x7f696c10d100 2024-09-11T01:00:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.158+0000 7f69757b9740 1 -- 172.21.15.78:0/579700100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 msgr2=0x55b664113860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.158+0000 7f6971180640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69580b5450 0x7f69580b7910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6968031440 tx=0x7f6968005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.158+0000 7f69757b9740 1 --2- 172.21.15.78:0/579700100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 0x55b664113860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f695c013990 tx=0x7f695c005850 comp rx=0 tx=0).stop 2024-09-11T01:00:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.158+0000 7f69757b9740 1 -- 172.21.15.78:0/579700100 shutdown_connections 2024-09-11T01:00:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.158+0000 7f69757b9740 1 --2- 172.21.15.78:0/579700100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6641283e0 0x55b66412a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.158+0000 7f69757b9740 1 --2- 172.21.15.78:0/579700100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b66411d7e0 0x55b664113da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.158+0000 7f69757b9740 1 --2- 172.21.15.78:0/579700100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 0x55b664113860 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.158+0000 7f69757b9740 1 -- 172.21.15.78:0/579700100 >> 172.21.15.78:0/579700100 conn(0x55b6641112b0 msgr2=0x55b664112450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.159+0000 7f69757b9740 1 -- 172.21.15.78:0/579700100 shutdown_connections 2024-09-11T01:00:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.159+0000 7f69757b9740 1 -- 172.21.15.78:0/579700100 wait complete. 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.159+0000 7f69757b9740 1 -- start start 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.159+0000 7f69757b9740 1 Processor -- start 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f69757b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b664113480 0x55b66411d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f69757b9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b66411d7e0 0x55b66411a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f69757b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6641283e0 0x55b66411ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f69757b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b663eda700 con 0x55b664113480 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f69757b9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b664014110 con 0x55b66411d7e0 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f69757b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b66412b1d0 con 0x55b6641283e0 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f69757b9740 10 client.0 _ll_register_callbacks cb 0x55b66402d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6971180640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b664113480 0x55b66411d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6971981640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6641283e0 0x55b66411ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f697097f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b66411d7e0 0x55b66411a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6971180640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b664113480 0x55b66411d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52670/0 (socket says 172.21.15.78:52670) 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6971180640 1 -- 172.21.15.78:0/3171588246 learned_addr learned my addr 172.21.15.78:0/3171588246 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6971981640 1 -- 172.21.15.78:0/3171588246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b66411d7e0 msgr2=0x55b66411a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6971981640 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b66411d7e0 0x55b66411a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6971981640 1 -- 172.21.15.78:0/3171588246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b664113480 msgr2=0x55b66411d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6971981640 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b664113480 0x55b66411d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6971981640 1 -- 172.21.15.78:0/3171588246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b664041c40 con 0x55b6641283e0 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f697097f640 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b66411d7e0 0x55b66411a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6971180640 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b664113480 0x55b66411d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6971981640 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6641283e0 0x55b66411ac10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f69640e3420 tx=0x7f69640e3950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58754e73190ee793 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6954ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6954ff9640 1 -- 172.21.15.78:0/3171588246 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964020030 con 0x55b6641283e0 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f6954ff9640 1 -- 172.21.15.78:0/3171588246 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f69640e3b70 con 0x55b6641283e0 2024-09-11T01:00:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.160+0000 7f69757b9740 1 -- 172.21.15.78:0/3171588246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695c00f8c0 con 0x55b6641283e0 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.161+0000 7f6954ff9640 1 -- 172.21.15.78:0/3171588246 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69640abd10 con 0x55b6641283e0 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.161+0000 7f69757b9740 10 client.25123 fetch_fsmap learned FSMap version 29 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.161+0000 7f69757b9740 1 -- 172.21.15.78:0/3171588246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b664136c00 con 0x55b6641283e0 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.161+0000 7f6954ff9640 1 -- 172.21.15.78:0/3171588246 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f695c00f8c0 con 0x55b6641283e0 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.161+0000 7f6954ff9640 1 -- 172.21.15.78:0/3171588246 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f69640ab460 con 0x55b6641283e0 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.161+0000 7f69757b9740 10 client.25123 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.161+0000 7f69757b9740 10 client.25123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.161+0000 7f69757b9740 20 client.25123 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.161+0000 7f69757b9740 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b664138110 0x55b66413a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.161+0000 7f69757b9740 4 client.25123 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.161+0000 7f69757b9740 1 -- 172.21.15.78:0/3171588246 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b66408bda0 con 0x55b664138110 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.161+0000 7f6971180640 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b664138110 0x55b66413a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.162+0000 7f6954ff9640 10 client.25123 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:19.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.162+0000 7f6971180640 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b664138110 0x55b66413a550 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=881ebc44133b8ed3 server_cookie=7cc869fba02c5f16 in_seq=0 out_seq=0 2024-09-11T01:00:19.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.162+0000 7f6954ff9640 1 -- 172.21.15.78:0/3171588246 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b66408bda0 con 0x55b664138110 2024-09-11T01:00:19.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.162+0000 7f6954ff9640 10 client.25123 handle_command_reply: tid=0 2024-09-11T01:00:19.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f69757b9740 1 client.25123 shutdown 2024-09-11T01:00:19.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f69757b9740 1 -- 172.21.15.78:0/3171588246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6641283e0 msgr2=0x55b66411ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:19.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f69757b9740 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6641283e0 0x55b66411ac10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f69640e3420 tx=0x7f69640e3950 comp rx=0 tx=0).stop 2024-09-11T01:00:19.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f69757b9740 1 -- 172.21.15.78:0/3171588246 shutdown_connections 2024-09-11T01:00:19.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f69757b9740 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6641283e0 0x55b66411ac10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f69757b9740 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b66411d7e0 0x55b66411a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f69757b9740 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b664138110 0x55b66413a550 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f69757b9740 1 --2- 172.21.15.78:0/3171588246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b664113480 0x55b66411d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f69757b9740 1 -- 172.21.15.78:0/3171588246 >> 172.21.15.78:0/3171588246 conn(0x55b6641112b0 msgr2=0x55b6641165c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:19.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f6954ff9640 0 client.25123 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:19.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f69757b9740 1 -- 172.21.15.78:0/3171588246 shutdown_connections 2024-09-11T01:00:19.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f69757b9740 1 -- 172.21.15.78:0/3171588246 wait complete. 2024-09-11T01:00:19.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.163+0000 7f69757b9740 20 client.25123 trim_cache size 0 max 16384 2024-09-11T01:00:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.186+0000 7f69757b9740 1 Processor -- start 2024-09-11T01:00:19.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.186+0000 7f69757b9740 1 -- start start 2024-09-11T01:00:19.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f69757b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 0x55b664110bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f69757b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b66411d7e0 0x55b664118e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f69757b9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6641283e0 0x55b664119350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f69757b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1ac340 con 0x55b66411d7e0 2024-09-11T01:00:19.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f69757b9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1ade80 con 0x55b6641283e0 2024-09-11T01:00:19.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f69757b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1aca10 con 0x55b664113480 2024-09-11T01:00:19.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f697097f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b66411d7e0 0x55b664118e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f6971180640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 0x55b664110bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f697097f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b66411d7e0 0x55b664118e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52672/0 (socket says 172.21.15.78:52672) 2024-09-11T01:00:19.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f697097f640 1 -- 172.21.15.78:0/1001562787 learned_addr learned my addr 172.21.15.78:0/1001562787 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:19.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f6971981640 1 --2- 172.21.15.78:0/1001562787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6641283e0 0x55b664119350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f6971180640 1 -- 172.21.15.78:0/1001562787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6641283e0 msgr2=0x55b664119350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f6971180640 1 --2- 172.21.15.78:0/1001562787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6641283e0 0x55b664119350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f6971180640 1 -- 172.21.15.78:0/1001562787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b66411d7e0 msgr2=0x55b664118e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f6971180640 1 --2- 172.21.15.78:0/1001562787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b66411d7e0 0x55b664118e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f6971180640 1 -- 172.21.15.78:0/1001562787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b664114080 con 0x55b664113480 2024-09-11T01:00:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f697097f640 1 --2- 172.21.15.78:0/1001562787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b66411d7e0 0x55b664118e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:00:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f6971180640 1 --2- 172.21.15.78:0/1001562787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 0x55b664110bc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55b66413adc0 tx=0x7f696803d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddd2fe1e970ac5fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:19.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f6956ffd640 1 -- 172.21.15.78:0/1001562787 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968057070 con 0x55b664113480 2024-09-11T01:00:19.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f6956ffd640 1 -- 172.21.15.78:0/1001562787 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6968031720 con 0x55b664113480 2024-09-11T01:00:19.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.187+0000 7f6956ffd640 1 -- 172.21.15.78:0/1001562787 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968035040 con 0x55b664113480 2024-09-11T01:00:19.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.188+0000 7f69757b9740 1 -- 172.21.15.78:0/1001562787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 msgr2=0x55b664110bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:19.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.188+0000 7f69757b9740 1 --2- 172.21.15.78:0/1001562787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 0x55b664110bc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55b66413adc0 tx=0x7f696803d000 comp rx=0 tx=0).stop 2024-09-11T01:00:19.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.188+0000 7f69757b9740 1 -- 172.21.15.78:0/1001562787 shutdown_connections 2024-09-11T01:00:19.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.188+0000 7f69757b9740 1 --2- 172.21.15.78:0/1001562787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6641283e0 0x55b664119350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.188+0000 7f69757b9740 1 --2- 172.21.15.78:0/1001562787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b66411d7e0 0x55b664118e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.188+0000 7f69757b9740 1 --2- 172.21.15.78:0/1001562787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b664113480 0x55b664110bc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.188+0000 7f69757b9740 1 -- 172.21.15.78:0/1001562787 >> 172.21.15.78:0/1001562787 conn(0x55b6641112b0 msgr2=0x55b664128d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.188+0000 7f69757b9740 1 -- 172.21.15.78:0/1001562787 shutdown_connections 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.188+0000 7f69757b9740 1 -- 172.21.15.78:0/1001562787 wait complete. 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.189+0000 7f69757b9740 1 -- start start 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.189+0000 7f69757b9740 1 Processor -- start 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.189+0000 7f69757b9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b664113480 0x55b664130ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.189+0000 7f69757b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b66411d7e0 0x55b66413fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.189+0000 7f69757b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6641283e0 0x55b66413ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.189+0000 7f69757b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1ad7b0 con 0x55b6641283e0 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.189+0000 7f69757b9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1ad0e0 con 0x55b664113480 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.189+0000 7f69757b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1aa320 con 0x55b66411d7e0 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.189+0000 7f69757b9740 10 client.0 _ll_register_callbacks cb 0x7f696400bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.189+0000 7f6971981640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6641283e0 0x55b66413ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f697097f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b66411d7e0 0x55b66413fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6971981640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6641283e0 0x55b66413ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52686/0 (socket says 172.21.15.78:52686) 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6971981640 1 -- 172.21.15.78:0/1795421970 learned_addr learned my addr 172.21.15.78:0/1795421970 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6971180640 1 --2- 172.21.15.78:0/1795421970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b664113480 0x55b664130ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6971981640 1 -- 172.21.15.78:0/1795421970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b66411d7e0 msgr2=0x55b66413fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6971981640 1 --2- 172.21.15.78:0/1795421970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b66411d7e0 0x55b66413fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6971981640 1 -- 172.21.15.78:0/1795421970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b664113480 msgr2=0x55b664130ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6971981640 1 --2- 172.21.15.78:0/1795421970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b664113480 0x55b664130ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6971981640 1 -- 172.21.15.78:0/1795421970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b664140810 con 0x55b6641283e0 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6971981640 1 --2- 172.21.15.78:0/1795421970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6641283e0 0x55b66413ffe0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f696400bf50 tx=0x7f69640ac100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b340e3ad05461c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6954ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6954ff9640 1 -- 172.21.15.78:0/1795421970 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964020070 con 0x55b6641283e0 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6954ff9640 1 -- 172.21.15.78:0/1795421970 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f69640cdb30 con 0x55b6641283e0 2024-09-11T01:00:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f69757b9740 1 -- 172.21.15.78:0/1795421970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6968031720 con 0x55b6641283e0 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f6954ff9640 1 -- 172.21.15.78:0/1795421970 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69640abb50 con 0x55b6641283e0 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f69757b9740 10 client.15702 fetch_fsmap learned FSMap version 29 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.190+0000 7f69757b9740 1 -- 172.21.15.78:0/1795421970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6641a50c0 con 0x55b6641283e0 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.191+0000 7f6954ff9640 1 -- 172.21.15.78:0/1795421970 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6968031720 con 0x55b6641283e0 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.191+0000 7f6954ff9640 1 -- 172.21.15.78:0/1795421970 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f69640b1dc0 con 0x55b6641283e0 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.191+0000 7f69757b9740 10 client.15702 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.191+0000 7f69757b9740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.191+0000 7f69757b9740 20 client.15702 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.191+0000 7f69757b9740 1 --2- 172.21.15.78:0/1795421970 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b6641a5810 0x55b6641a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.191+0000 7f69757b9740 4 client.15702 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.191+0000 7f69757b9740 1 -- 172.21.15.78:0/1795421970 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b664087850 con 0x55b6641a5810 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.191+0000 7f6971180640 1 --2- 172.21.15.78:0/1795421970 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b6641a5810 0x55b6641a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.191+0000 7f6954ff9640 10 client.15702 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.191+0000 7f6971180640 1 --2- 172.21.15.78:0/1795421970 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b6641a5810 0x55b6641a7c30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eab3cbd5d96e52a server_cookie=c7bc30a4094d58c6 in_seq=0 out_seq=0 2024-09-11T01:00:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.192+0000 7f6954ff9640 1 -- 172.21.15.78:0/1795421970 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x55b664087850 con 0x55b6641a5810 2024-09-11T01:00:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.192+0000 7f6954ff9640 10 client.15702 handle_command_reply: tid=0 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.192+0000 7f69757b9740 1 client.15702 shutdown 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.192+0000 7f69757b9740 1 -- 172.21.15.78:0/1795421970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6641283e0 msgr2=0x55b66413ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.192+0000 7f69757b9740 1 --2- 172.21.15.78:0/1795421970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6641283e0 0x55b66413ffe0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f696400bf50 tx=0x7f69640ac100 comp rx=0 tx=0).stop 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.192+0000 7f69757b9740 1 -- 172.21.15.78:0/1795421970 shutdown_connections 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.192+0000 7f69757b9740 1 --2- 172.21.15.78:0/1795421970 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b6641a5810 0x55b6641a7c30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.192+0000 7f69757b9740 1 --2- 172.21.15.78:0/1795421970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6641283e0 0x55b66413ffe0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.192+0000 7f6954ff9640 0 client.15702 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.192+0000 7f69757b9740 1 --2- 172.21.15.78:0/1795421970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b66411d7e0 0x55b66413fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.192+0000 7f69757b9740 1 --2- 172.21.15.78:0/1795421970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b664113480 0x55b664130ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.192+0000 7f69757b9740 1 -- 172.21.15.78:0/1795421970 >> 172.21.15.78:0/1795421970 conn(0x55b6641112b0 msgr2=0x55b664128d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69757b9740 1 -- 172.21.15.78:0/1795421970 shutdown_connections 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69757b9740 1 -- 172.21.15.78:0/1795421970 wait complete. 2024-09-11T01:00:19.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69757b9740 20 client.15702 trim_cache size 0 max 16384 2024-09-11T01:00:19.199 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:00:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:00:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:00:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:00:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:00:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:00:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:00:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:00:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:00:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:00:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:00:19.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:00:19.200 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:00:19.200 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:00:19.200 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:00:19.200 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:00:19.200 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:00:19.200 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:00:19.200 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:00:19.200 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:00:19.200 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:00:19.200 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:00:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69733e4640 1 -- 172.21.15.78:0/2476509906 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69580b5450 msgr2=0x7f69580b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69733e4640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69580b5450 0x7f69580b7910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6968031440 tx=0x7f6968005b50 comp rx=0 tx=0).stop 2024-09-11T01:00:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69733e4640 1 -- 172.21.15.78:0/2476509906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f696c10d100 msgr2=0x7f696c19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69733e4640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f696c10d100 0x7f696c19eb60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6964009fd0 tx=0x7f696400b260 comp rx=0 tx=0).stop 2024-09-11T01:00:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69733e4640 1 -- 172.21.15.78:0/2476509906 shutdown_connections 2024-09-11T01:00:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69733e4640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69580b5450 0x7f69580b7910 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69733e4640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f696c10d100 0x7f696c19eb60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69733e4640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f696c1045c0 0x7f696c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69733e4640 1 --2- 172.21.15.78:0/2476509906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f696c103ca0 0x7f696c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69733e4640 1 -- 172.21.15.78:0/2476509906 >> 172.21.15.78:0/2476509906 conn(0x7f696c0fd700 msgr2=0x7f696c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69733e4640 1 -- 172.21.15.78:0/2476509906 shutdown_connections 2024-09-11T01:00:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:19.193+0000 7f69733e4640 1 -- 172.21.15.78:0/2476509906 wait complete. 2024-09-11T01:00:19.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 83330 17.26 MB/sec warmup 27 sec latency 20.296 ms 2024-09-11T01:00:20.030 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:00:20.030 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:00:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:00:20.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 86236 17.21 MB/sec warmup 28 sec latency 17.898 ms 2024-09-11T01:00:20.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.796+0000 7f8e73550640 1 -- 172.21.15.78:0/3669555161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6c108020 msgr2=0x7f8e6c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:20.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.796+0000 7f8e73550640 1 --2- 172.21.15.78:0/3669555161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6c108020 0x7f8e6c10e820 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009900 tx=0x7f8e6802a410 comp rx=0 tx=0).stop 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.796+0000 7f8e73550640 1 -- 172.21.15.78:0/3669555161 shutdown_connections 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.796+0000 7f8e73550640 1 --2- 172.21.15.78:0/3669555161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6c108020 0x7f8e6c10e820 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.796+0000 7f8e73550640 1 --2- 172.21.15.78:0/3669555161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e6c1076a0 0x7f8e6c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.796+0000 7f8e73550640 1 --2- 172.21.15.78:0/3669555161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6c101c90 0x7f8e6c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.796+0000 7f8e73550640 1 -- 172.21.15.78:0/3669555161 >> 172.21.15.78:0/3669555161 conn(0x7f8e6c0fd700 msgr2=0x7f8e6c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.796+0000 7f8e73550640 1 -- 172.21.15.78:0/3669555161 shutdown_connections 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.796+0000 7f8e73550640 1 -- 172.21.15.78:0/3669555161 wait complete. 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.796+0000 7f8e73550640 1 Processor -- start 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.796+0000 7f8e73550640 1 -- start start 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e73550640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e6c101c90 0x7f8e6c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e73550640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6c1076a0 0x7f8e6c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e73550640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6c108020 0x7f8e6c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e73550640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c111ce0 con 0x7f8e6c1076a0 2024-09-11T01:00:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e73550640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c111b60 con 0x7f8e6c101c90 2024-09-11T01:00:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e73550640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c111e60 con 0x7f8e6c108020 2024-09-11T01:00:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e70aeb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6c1076a0 0x7f8e6c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e712ec640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e6c101c90 0x7f8e6c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e70aeb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6c1076a0 0x7f8e6c19a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52704/0 (socket says 172.21.15.78:52704) 2024-09-11T01:00:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e70aeb640 1 -- 172.21.15.78:0/127208646 learned_addr learned my addr 172.21.15.78:0/127208646 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e71aed640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6c108020 0x7f8e6c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e70aeb640 1 -- 172.21.15.78:0/127208646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6c108020 msgr2=0x7f8e6c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e70aeb640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6c108020 0x7f8e6c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e70aeb640 1 -- 172.21.15.78:0/127208646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e6c101c90 msgr2=0x7f8e6c19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e70aeb640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e6c101c90 0x7f8e6c19a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:20.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e70aeb640 1 -- 172.21.15.78:0/127208646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c1a5410 con 0x7f8e6c1076a0 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e70aeb640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6c1076a0 0x7f8e6c19a570 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00e9f0 tx=0x7f8e5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e627fc640 1 -- 172.21.15.78:0/127208646 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c00cd80 con 0x7f8e6c1076a0 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e627fc640 1 -- 172.21.15.78:0/127208646 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e5c0108e0 con 0x7f8e6c1076a0 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e627fc640 1 -- 172.21.15.78:0/127208646 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c018980 con 0x7f8e6c1076a0 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.797+0000 7f8e73550640 1 -- 172.21.15.78:0/127208646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6c1a5700 con 0x7f8e6c1076a0 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.798+0000 7f8e73550640 1 -- 172.21.15.78:0/127208646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6c1a5c40 con 0x7f8e6c1076a0 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.799+0000 7f8e73550640 1 -- 172.21.15.78:0/127208646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e6c102070 con 0x7f8e6c1076a0 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.800+0000 7f8e627fc640 1 -- 172.21.15.78:0/127208646 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8e5c010430 con 0x7f8e6c1076a0 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.801+0000 7f8e627fc640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8e580b5450 0x7f8e580b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.801+0000 7f8e712ec640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8e580b5450 0x7f8e580b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.801+0000 7f8e627fc640 1 -- 172.21.15.78:0/127208646 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f8e5c0e67a0 con 0x7f8e6c1076a0 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.802+0000 7f8e712ec640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8e580b5450 0x7f8e580b7910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005fd0 tx=0x7f8e64009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:20.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:20.804+0000 7f8e627fc640 1 -- 172.21.15.78:0/127208646 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8e5c0acce0 con 0x7f8e6c1076a0 2024-09-11T01:00:21.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.030+0000 7f8e73550640 1 -- 172.21.15.78:0/127208646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e6c06aec0 con 0x7f8e6c1076a0 2024-09-11T01:00:21.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.031+0000 7f8e627fc640 1 -- 172.21.15.78:0/127208646 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11979 (secure 0 0 0) 0x7f8e6c06aec0 con 0x7f8e6c1076a0 2024-09-11T01:00:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:00:21.033 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"btime":"2024-09-11T00:58:50:118172+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T00:58:49.633286+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T01:00:21.033 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-09-11T01:00:21.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.033+0000 7f8e73550640 1 -- 172.21.15.78:0/127208646 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8e580b5450 msgr2=0x7f8e580b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:21.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.033+0000 7f8e73550640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8e580b5450 0x7f8e580b7910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005fd0 tx=0x7f8e64009500 comp rx=0 tx=0).stop 2024-09-11T01:00:21.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.033+0000 7f8e73550640 1 -- 172.21.15.78:0/127208646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6c1076a0 msgr2=0x7f8e6c19a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:21.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.033+0000 7f8e73550640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6c1076a0 0x7f8e6c19a570 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00e9f0 tx=0x7f8e5c00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:00:21.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.033+0000 7f8e73550640 1 -- 172.21.15.78:0/127208646 shutdown_connections 2024-09-11T01:00:21.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.033+0000 7f8e73550640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8e580b5450 0x7f8e580b7910 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:21.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.033+0000 7f8e73550640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6c108020 0x7f8e6c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:21.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.033+0000 7f8e73550640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6c1076a0 0x7f8e6c19a570 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:21.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.033+0000 7f8e73550640 1 --2- 172.21.15.78:0/127208646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e6c101c90 0x7f8e6c19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:21.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.033+0000 7f8e73550640 1 -- 172.21.15.78:0/127208646 >> 172.21.15.78:0/127208646 conn(0x7f8e6c0fd700 msgr2=0x7f8e6c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:21.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.033+0000 7f8e73550640 1 -- 172.21.15.78:0/127208646 shutdown_connections 2024-09-11T01:00:21.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:21.033+0000 7f8e73550640 1 -- 172.21.15.78:0/127208646 wait complete. 2024-09-11T01:00:21.417 INFO:tasks.workunit.client.0.smithi078.stdout: 1 89266 17.21 MB/sec warmup 29 sec latency 19.576 ms 2024-09-11T01:00:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:21 smithi110 ceph-mon[34712]: pgmap v367: 97 pgs: 97 active+clean; 41 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-09-11T01:00:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:21 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/127208646' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:00:21.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:21 smithi112 ceph-mon[35384]: pgmap v367: 97 pgs: 97 active+clean; 41 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-09-11T01:00:21.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:21 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/127208646' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:00:21.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:21 smithi078 ceph-mon[31315]: pgmap v367: 97 pgs: 97 active+clean; 41 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-09-11T01:00:21.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:21 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/127208646' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:00:21.766 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:00:22.028 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:00:22.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92357 17.23 MB/sec warmup 30 sec latency 99.954 ms 2024-09-11T01:00:22.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.539+0000 7efd5ec84640 1 -- 172.21.15.78:0/1310126111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd58101cb0 msgr2=0x7efd58102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:22.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.539+0000 7efd5ec84640 1 --2- 172.21.15.78:0/1310126111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd58101cb0 0x7efd58102090 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efd48009900 tx=0x7efd4802a410 comp rx=0 tx=0).stop 2024-09-11T01:00:22.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.539+0000 7efd5ec84640 1 -- 172.21.15.78:0/1310126111 shutdown_connections 2024-09-11T01:00:22.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.539+0000 7efd5ec84640 1 --2- 172.21.15.78:0/1310126111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd58108040 0x7efd5810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.539+0000 7efd5ec84640 1 --2- 172.21.15.78:0/1310126111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd581076c0 0x7efd58107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.539+0000 7efd5ec84640 1 --2- 172.21.15.78:0/1310126111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd58101cb0 0x7efd58102090 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.539+0000 7efd5ec84640 1 -- 172.21.15.78:0/1310126111 >> 172.21.15.78:0/1310126111 conn(0x7efd580fd740 msgr2=0x7efd580ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:22.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.539+0000 7efd5ec84640 1 -- 172.21.15.78:0/1310126111 shutdown_connections 2024-09-11T01:00:22.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.539+0000 7efd5ec84640 1 -- 172.21.15.78:0/1310126111 wait complete. 2024-09-11T01:00:22.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.540+0000 7efd5ec84640 1 Processor -- start 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.540+0000 7efd5ec84640 1 -- start start 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.540+0000 7efd5ec84640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd58101cb0 0x7efd58199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.540+0000 7efd5ec84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd581076c0 0x7efd5819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.540+0000 7efd5ec84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd58108040 0x7efd581a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.540+0000 7efd5ec84640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd58111cc0 con 0x7efd58108040 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.540+0000 7efd5ec84640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd58111b40 con 0x7efd58101cb0 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.540+0000 7efd5ec84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd58111e40 con 0x7efd581076c0 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.540+0000 7efd5d221640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd58108040 0x7efd581a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.540+0000 7efd5d221640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd58108040 0x7efd581a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52716/0 (socket says 172.21.15.78:52716) 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.540+0000 7efd5d221640 1 -- 172.21.15.78:0/3044773114 learned_addr learned my addr 172.21.15.78:0/3044773114 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd5ca20640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd58101cb0 0x7efd58199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd57fff640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd581076c0 0x7efd5819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd5d221640 1 -- 172.21.15.78:0/3044773114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd581076c0 msgr2=0x7efd5819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:22.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd5d221640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd581076c0 0x7efd5819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd5d221640 1 -- 172.21.15.78:0/3044773114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd58101cb0 msgr2=0x7efd58199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd5d221640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd58101cb0 0x7efd58199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd5d221640 1 -- 172.21.15.78:0/3044773114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd581a55b0 con 0x7efd58108040 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd5d221640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd58108040 0x7efd581a5070 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efd5000eaa0 tx=0x7efd5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd55ffb640 1 -- 172.21.15.78:0/3044773114 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5000cdb0 con 0x7efd58108040 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd55ffb640 1 -- 172.21.15.78:0/3044773114 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd50005020 con 0x7efd58108040 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd5ec84640 1 -- 172.21.15.78:0/3044773114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd581a5780 con 0x7efd58108040 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd55ffb640 1 -- 172.21.15.78:0/3044773114 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd500116d0 con 0x7efd58108040 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.541+0000 7efd5ec84640 1 -- 172.21.15.78:0/3044773114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd581a5c60 con 0x7efd58108040 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.543+0000 7efd55ffb640 1 -- 172.21.15.78:0/3044773114 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7efd5000f040 con 0x7efd58108040 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd61059740 1 Processor -- start 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd61059740 1 -- start start 2024-09-11T01:00:22.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd61059740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd61059740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd61059740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1c5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd61059740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a5cd12cea0 con 0x55a5cd1b6df0 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd61059740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a5cd11c2d0 con 0x55a5cd1b64d0 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd61059740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a5cd1200e0 con 0x55a5cd1c2d80 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd5ca20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd5ca20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52730/0 (socket says 172.21.15.78:52730) 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd5ca20640 1 -- 172.21.15.78:0/2428777856 learned_addr learned my addr 172.21.15.78:0/2428777856 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd57fff640 1 --2- 172.21.15.78:0/2428777856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd57fff640 1 --2- 172.21.15.78:0/2428777856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd55ffb640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efd440b54d0 0x7efd440b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd5ca20640 1 -- 172.21.15.78:0/2428777856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 msgr2=0x55a5cd1c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd5ca20640 1 --2- 172.21.15.78:0/2428777856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd5ca20640 1 -- 172.21.15.78:0/2428777856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b64d0 msgr2=0x55a5cd1b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd5ca20640 1 --2- 172.21.15.78:0/2428777856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd5d221640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efd440b54d0 0x7efd440b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd5ca20640 1 -- 172.21.15.78:0/2428777856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5cce7a650 con 0x55a5cd1b6df0 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.544+0000 7efd55ffb640 1 -- 172.21.15.78:0/3044773114 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efd500e5d90 con 0x7efd58108040 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.545+0000 7efd5ca20640 1 --2- 172.21.15.78:0/2428777856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1c2840 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efd4802a930 tx=0x7efd48032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163488fb4d4ab0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.545+0000 7efd4effd640 1 -- 172.21.15.78:0/2428777856 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4803f070 con 0x55a5cd1b6df0 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.545+0000 7efd4effd640 1 -- 172.21.15.78:0/2428777856 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd48032e50 con 0x55a5cd1b6df0 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.545+0000 7efd4effd640 1 -- 172.21.15.78:0/2428777856 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48035070 con 0x55a5cd1b6df0 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.545+0000 7efd61059740 1 -- 172.21.15.78:0/2428777856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 msgr2=0x55a5cd1c2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.545+0000 7efd5d221640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efd440b54d0 0x7efd440b7990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efd5006b7d0 tx=0x7efd500cf040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.545+0000 7efd61059740 1 --2- 172.21.15.78:0/2428777856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1c2840 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efd4802a930 tx=0x7efd48032ce0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 -- 172.21.15.78:0/2428777856 shutdown_connections 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 --2- 172.21.15.78:0/2428777856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 --2- 172.21.15.78:0/2428777856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1c2840 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 --2- 172.21.15.78:0/2428777856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 -- 172.21.15.78:0/2428777856 >> 172.21.15.78:0/2428777856 conn(0x55a5cd1b4300 msgr2=0x55a5cd1b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 -- 172.21.15.78:0/2428777856 shutdown_connections 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 -- 172.21.15.78:0/2428777856 wait complete. 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 -- start start 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 Processor -- start 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1bfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a5ccf7d700 con 0x55a5cd1b64d0 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a5cd0b7160 con 0x55a5cd1b6df0 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a5cd1c59a0 con 0x55a5cd1c2d80 2024-09-11T01:00:22.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd61059740 10 client.0 _ll_register_callbacks cb 0x55a5cd0d09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1c26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52740/0 (socket says 172.21.15.78:52740) 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.546+0000 7efd57fff640 1 -- 172.21.15.78:0/191044842 learned_addr learned my addr 172.21.15.78:0/191044842 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd5ca20640 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1bfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd57fff640 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd57fff640 1 -- 172.21.15.78:0/191044842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 msgr2=0x55a5cd1c01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd57fff640 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1c01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd57fff640 1 -- 172.21.15.78:0/191044842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b6df0 msgr2=0x55a5cd1bfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd57fff640 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1bfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd57fff640 1 -- 172.21.15.78:0/191044842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5cd0e4c90 con 0x55a5cd1b64d0 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd5ca20640 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1bfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd57fff640 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1c26e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efd40013ed0 tx=0x7efd40015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387e62ae81088f30 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:22.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd37fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd37fff640 1 -- 172.21.15.78:0/191044842 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40013110 con 0x55a5cd1b64d0 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd37fff640 1 -- 172.21.15.78:0/191044842 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efd40005020 con 0x55a5cd1b64d0 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd61059740 1 -- 172.21.15.78:0/191044842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd48032e50 con 0x55a5cd1b64d0 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd37fff640 1 -- 172.21.15.78:0/191044842 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd40026050 con 0x55a5cd1b64d0 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd61059740 10 client.15720 fetch_fsmap learned FSMap version 30 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd61059740 1 -- 172.21.15.78:0/191044842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a5cd1d13b0 con 0x55a5cd1b64d0 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.547+0000 7efd37fff640 1 -- 172.21.15.78:0/191044842 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd48032e50 con 0x55a5cd1b64d0 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.548+0000 7efd37fff640 1 -- 172.21.15.78:0/191044842 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7efd40004ca0 con 0x55a5cd1b64d0 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.548+0000 7efd61059740 10 client.15720 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.548+0000 7efd61059740 10 client.15720 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.548+0000 7efd61059740 20 client.15720 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.548+0000 7efd61059740 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a5cd1d28c0 0x55a5cd1d4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.548+0000 7efd61059740 4 client.15720 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:00:22.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.548+0000 7efd61059740 1 -- 172.21.15.78:0/191044842 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5cd12edf0 con 0x55a5cd1d28c0 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.548+0000 7efd5ca20640 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a5cd1d28c0 0x55a5cd1d4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.548+0000 7efd5ca20640 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a5cd1d28c0 0x55a5cd1d4cb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=519711a8e3b5f024 server_cookie=10973dcb30571059 in_seq=0 out_seq=0 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.548+0000 7efd37fff640 10 client.15720 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd37fff640 1 -- 172.21.15.78:0/191044842 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5cd12edf0 con 0x55a5cd1d28c0 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd37fff640 10 client.15720 handle_command_reply: tid=0 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd61059740 1 client.15720 shutdown 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd61059740 1 -- 172.21.15.78:0/191044842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b64d0 msgr2=0x55a5cd1c26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd61059740 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1c26e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efd40013ed0 tx=0x7efd40015040 comp rx=0 tx=0).stop 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd61059740 1 -- 172.21.15.78:0/191044842 shutdown_connections 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd61059740 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a5cd1d28c0 0x55a5cd1d4cb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd61059740 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1c01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd61059740 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1bfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd37fff640 0 client.15720 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:00:22.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd61059740 1 --2- 172.21.15.78:0/191044842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1c26e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd61059740 1 -- 172.21.15.78:0/191044842 >> 172.21.15.78:0/191044842 conn(0x55a5cd1b4300 msgr2=0x55a5cd1b9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:22.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd61059740 1 -- 172.21.15.78:0/191044842 shutdown_connections 2024-09-11T01:00:22.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd61059740 1 -- 172.21.15.78:0/191044842 wait complete. 2024-09-11T01:00:22.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.549+0000 7efd61059740 20 client.15720 trim_cache size 0 max 16384 2024-09-11T01:00:22.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.563+0000 7efd61059740 1 Processor -- start 2024-09-11T01:00:22.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.563+0000 7efd61059740 1 -- start start 2024-09-11T01:00:22.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.563+0000 7efd61059740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.563+0000 7efd61059740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1bf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.563+0000 7efd61059740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1badd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.563+0000 7efd61059740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd581ae490 con 0x55a5cd1b6df0 2024-09-11T01:00:22.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.563+0000 7efd61059740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd581b0000 con 0x55a5cd1b64d0 2024-09-11T01:00:22.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.563+0000 7efd61059740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd581aeb60 con 0x55a5cd1c2d80 2024-09-11T01:00:22.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.563+0000 7efd5ca20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1bf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.563+0000 7efd5ca20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1bf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52756/0 (socket says 172.21.15.78:52756) 2024-09-11T01:00:22.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.563+0000 7efd57fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.563+0000 7efd5ca20640 1 -- 172.21.15.78:0/3945047710 learned_addr learned my addr 172.21.15.78:0/3945047710 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:22.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.564+0000 7efd57fff640 1 --2- 172.21.15.78:0/3945047710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1badd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.564+0000 7efd5ca20640 1 -- 172.21.15.78:0/3945047710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 msgr2=0x55a5cd1badd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:22.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.564+0000 7efd5ca20640 1 --2- 172.21.15.78:0/3945047710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1badd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.564+0000 7efd5ca20640 1 -- 172.21.15.78:0/3945047710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b64d0 msgr2=0x55a5cd1cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:22.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.564+0000 7efd5ca20640 1 --2- 172.21.15.78:0/3945047710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.564+0000 7efd5ca20640 1 -- 172.21.15.78:0/3945047710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5cd1c0a80 con 0x55a5cd1b6df0 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.564+0000 7efd57fff640 1 --2- 172.21.15.78:0/3945047710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.564+0000 7efd5ca20640 1 --2- 172.21.15.78:0/3945047710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1bf020 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efd48004650 tx=0x7efd4802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f0d5fa371f6c2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.564+0000 7efd4e7fc640 1 -- 172.21.15.78:0/3945047710 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4803f040 con 0x55a5cd1b6df0 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.564+0000 7efd4e7fc640 1 -- 172.21.15.78:0/3945047710 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd480316a0 con 0x55a5cd1b6df0 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.564+0000 7efd4e7fc640 1 -- 172.21.15.78:0/3945047710 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48035040 con 0x55a5cd1b6df0 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.565+0000 7efd61059740 1 -- 172.21.15.78:0/3945047710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 msgr2=0x55a5cd1bf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.565+0000 7efd61059740 1 --2- 172.21.15.78:0/3945047710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1bf020 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efd48004650 tx=0x7efd4802a410 comp rx=0 tx=0).stop 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.565+0000 7efd61059740 1 -- 172.21.15.78:0/3945047710 shutdown_connections 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.565+0000 7efd61059740 1 --2- 172.21.15.78:0/3945047710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1badd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.565+0000 7efd61059740 1 --2- 172.21.15.78:0/3945047710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1bf020 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.565+0000 7efd61059740 1 --2- 172.21.15.78:0/3945047710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.565+0000 7efd61059740 1 -- 172.21.15.78:0/3945047710 >> 172.21.15.78:0/3945047710 conn(0x55a5cd1b4300 msgr2=0x55a5cd1b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.565+0000 7efd61059740 1 -- 172.21.15.78:0/3945047710 shutdown_connections 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.565+0000 7efd61059740 1 -- 172.21.15.78:0/3945047710 wait complete. 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.565+0000 7efd61059740 1 -- start start 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.565+0000 7efd61059740 1 Processor -- start 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd61059740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd61059740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd61059740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd61059740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd581af900 con 0x55a5cd1b6df0 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd61059740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd581af230 con 0x55a5cd1c2d80 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd61059740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd581ac470 con 0x55a5cd1b64d0 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd61059740 10 client.0 _ll_register_callbacks cb 0x7efd400134f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd5ca20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd57fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd57fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1bd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42826/0 (socket says 172.21.15.78:42826) 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd57fff640 1 -- 172.21.15.78:0/4223900027 learned_addr learned my addr 172.21.15.78:0/4223900027 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd5ca20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52762/0 (socket says 172.21.15.78:52762) 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd57fff640 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd57fff640 1 -- 172.21.15.78:0/4223900027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1c2d80 msgr2=0x55a5cd1cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd57fff640 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd57fff640 1 -- 172.21.15.78:0/4223900027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 msgr2=0x55a5cd1cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd57fff640 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd57fff640 1 -- 172.21.15.78:0/4223900027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5cd1da400 con 0x55a5cd1b64d0 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd5ca20640 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd57fff640 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1bd760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd40010f90 tx=0x7efd4000e430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5374d9d0eb783a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd37fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd37fff640 1 -- 172.21.15.78:0/4223900027 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4000fd90 con 0x55a5cd1b64d0 2024-09-11T01:00:22.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd61059740 1 -- 172.21.15.78:0/4223900027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd480316a0 con 0x55a5cd1b64d0 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.566+0000 7efd37fff640 1 -- 172.21.15.78:0/4223900027 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efd40004030 con 0x55a5cd1b64d0 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd37fff640 1 -- 172.21.15.78:0/4223900027 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd4003c050 con 0x55a5cd1b64d0 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd61059740 10 client.25153 fetch_fsmap learned FSMap version 30 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd61059740 1 -- 172.21.15.78:0/4223900027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a5cd23fa70 con 0x55a5cd1b64d0 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd37fff640 1 -- 172.21.15.78:0/4223900027 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd480316a0 con 0x55a5cd1b64d0 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd37fff640 1 -- 172.21.15.78:0/4223900027 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7efd40003b30 con 0x55a5cd1b64d0 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd61059740 10 client.25153 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd61059740 10 client.25153 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd61059740 20 client.25153 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd61059740 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a5cd240130 0x55a5cd242550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd61059740 4 client.25153 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd61059740 1 -- 172.21.15.78:0/4223900027 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5cd12a8a0 con 0x55a5cd240130 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd5ca20640 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a5cd240130 0x55a5cd242550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd37fff640 10 client.25153 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:00:22.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.567+0000 7efd5ca20640 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a5cd240130 0x55a5cd242550 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=27d885da5115eb2f server_cookie=36b9d8786b7ef7c4 in_seq=0 out_seq=0 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd37fff640 1 -- 172.21.15.78:0/4223900027 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5cd12a8a0 con 0x55a5cd240130 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd37fff640 10 client.25153 handle_command_reply: tid=0 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd61059740 1 client.25153 shutdown 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd61059740 1 -- 172.21.15.78:0/4223900027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1b64d0 msgr2=0x55a5cd1bd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd61059740 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1bd760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd40010f90 tx=0x7efd4000e430 comp rx=0 tx=0).stop 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd61059740 1 -- 172.21.15.78:0/4223900027 shutdown_connections 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd61059740 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a5cd240130 0x55a5cd242550 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd61059740 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5cd1c2d80 0x55a5cd1cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd37fff640 0 client.25153 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd61059740 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5cd1b6df0 0x55a5cd1cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd61059740 1 --2- 172.21.15.78:0/4223900027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5cd1b64d0 0x55a5cd1bd760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd61059740 1 -- 172.21.15.78:0/4223900027 >> 172.21.15.78:0/4223900027 conn(0x55a5cd1b4300 msgr2=0x55a5cd1b5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:22.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd61059740 1 -- 172.21.15.78:0/4223900027 shutdown_connections 2024-09-11T01:00:22.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd61059740 1 -- 172.21.15.78:0/4223900027 wait complete. 2024-09-11T01:00:22.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.568+0000 7efd61059740 20 client.25153 trim_cache size 0 max 16384 2024-09-11T01:00:22.579 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:00:22.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.569+0000 7efd5ec84640 1 -- 172.21.15.78:0/3044773114 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efd440b54d0 msgr2=0x7efd440b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:22.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.569+0000 7efd5ec84640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efd440b54d0 0x7efd440b7990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efd5006b7d0 tx=0x7efd500cf040 comp rx=0 tx=0).stop 2024-09-11T01:00:22.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.569+0000 7efd5ec84640 1 -- 172.21.15.78:0/3044773114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd58108040 msgr2=0x7efd581a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:22.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.569+0000 7efd5ec84640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd58108040 0x7efd581a5070 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efd5000eaa0 tx=0x7efd5000ef60 comp rx=0 tx=0).stop 2024-09-11T01:00:22.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.569+0000 7efd5ec84640 1 -- 172.21.15.78:0/3044773114 shutdown_connections 2024-09-11T01:00:22.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.569+0000 7efd5ec84640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efd440b54d0 0x7efd440b7990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.569+0000 7efd5ec84640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd58108040 0x7efd581a5070 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.569+0000 7efd5ec84640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd581076c0 0x7efd5819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.569+0000 7efd5ec84640 1 --2- 172.21.15.78:0/3044773114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd58101cb0 0x7efd58199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:22.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.569+0000 7efd5ec84640 1 -- 172.21.15.78:0/3044773114 >> 172.21.15.78:0/3044773114 conn(0x7efd580fd740 msgr2=0x7efd580ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:22.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.569+0000 7efd5ec84640 1 -- 172.21.15.78:0/3044773114 shutdown_connections 2024-09-11T01:00:22.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:22.569+0000 7efd5ec84640 1 -- 172.21.15.78:0/3044773114 wait complete. 2024-09-11T01:00:23.230 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:23 smithi078 ceph-mon[31315]: pgmap v368: 97 pgs: 97 active+clean; 43 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2024-09-11T01:00:23.230 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:23 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:00:23.257 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:00:23.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92357 16.67 MB/sec warmup 31 sec latency 1100.016 ms 2024-09-11T01:00:23.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:23 smithi110 ceph-mon[34712]: pgmap v368: 97 pgs: 97 active+clean; 43 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2024-09-11T01:00:23.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:23 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:00:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:23 smithi112 ceph-mon[35384]: pgmap v368: 97 pgs: 97 active+clean; 43 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2024-09-11T01:00:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:23 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:00:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:00:24.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.040+0000 7f4e73d77640 1 -- 172.21.15.78:0/2859042675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6c1087b0 msgr2=0x7f4e6c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:24.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.040+0000 7f4e73d77640 1 --2- 172.21.15.78:0/2859042675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6c1087b0 0x7f4e6c108bf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009900 tx=0x7f4e6002a410 comp rx=0 tx=0).stop 2024-09-11T01:00:24.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 -- 172.21.15.78:0/2859042675 shutdown_connections 2024-09-11T01:00:24.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 --2- 172.21.15.78:0/2859042675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c109130 0x7f4e6c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 --2- 172.21.15.78:0/2859042675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6c1087b0 0x7f4e6c108bf0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 --2- 172.21.15.78:0/2859042675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e6c102da0 0x7f4e6c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 -- 172.21.15.78:0/2859042675 >> 172.21.15.78:0/2859042675 conn(0x7f4e6c0fe810 msgr2=0x7f4e6c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 -- 172.21.15.78:0/2859042675 shutdown_connections 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 -- 172.21.15.78:0/2859042675 wait complete. 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 Processor -- start 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 -- start start 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6c102da0 0x7f4e6c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c1087b0 0x7f4e6c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e6c109130 0x7f4e6c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c112df0 con 0x7f4e6c102da0 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c112c70 con 0x7f4e6c109130 2024-09-11T01:00:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.041+0000 7f4e73d77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c112f70 con 0x7f4e6c1087b0 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e71b13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6c102da0 0x7f4e6c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e71b13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6c102da0 0x7f4e6c199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44714/0 (socket says 172.21.15.78:44714) 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e71b13640 1 -- 172.21.15.78:0/2269184657 learned_addr learned my addr 172.21.15.78:0/2269184657 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e71312640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c1087b0 0x7f4e6c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e72314640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e6c109130 0x7f4e6c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e71b13640 1 -- 172.21.15.78:0/2269184657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c1087b0 msgr2=0x7f4e6c199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e71b13640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c1087b0 0x7f4e6c199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e71b13640 1 -- 172.21.15.78:0/2269184657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e6c109130 msgr2=0x7f4e6c19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e71b13640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e6c109130 0x7f4e6c19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e71b13640 1 -- 172.21.15.78:0/2269184657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6c19f3d0 con 0x7f4e6c102da0 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e71b13640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6c102da0 0x7f4e6c199320 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00ba50 tx=0x7f4e5c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e6affd640 1 -- 172.21.15.78:0/2269184657 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c019070 con 0x7f4e6c102da0 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e6affd640 1 -- 172.21.15.78:0/2269184657 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e5c00cbe0 con 0x7f4e6c102da0 2024-09-11T01:00:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e73d77640 1 -- 172.21.15.78:0/2269184657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6c19f6c0 con 0x7f4e6c102da0 2024-09-11T01:00:24.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e6affd640 1 -- 172.21.15.78:0/2269184657 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0075d0 con 0x7f4e6c102da0 2024-09-11T01:00:24.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.042+0000 7f4e73d77640 1 -- 172.21.15.78:0/2269184657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6c1a5e00 con 0x7f4e6c102da0 2024-09-11T01:00:24.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e7614c740 1 Processor -- start 2024-09-11T01:00:24.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e7614c740 1 -- start start 2024-09-11T01:00:24.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e6affd640 1 -- 172.21.15.78:0/2269184657 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4e5c007830 con 0x7f4e6c102da0 2024-09-11T01:00:24.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e7614c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe80f0 0x55fed3fe84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e7614c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3fe8a10 0x55fed3ff4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e7614c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 0x55fed3ff6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e7614c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fed3f5eea0 con 0x55fed3ff49a0 2024-09-11T01:00:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e7614c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fed3f4e2d0 con 0x55fed3fe80f0 2024-09-11T01:00:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e7614c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fed3f520e0 con 0x55fed3fe8a10 2024-09-11T01:00:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e71312640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 0x55fed3ff6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e71312640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 0x55fed3ff6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44718/0 (socket says 172.21.15.78:44718) 2024-09-11T01:00:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e71312640 1 -- 172.21.15.78:0/2635834785 learned_addr learned my addr 172.21.15.78:0/2635834785 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e72314640 1 --2- 172.21.15.78:0/2635834785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe80f0 0x55fed3fe84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.045+0000 7f4e71b13640 1 --2- 172.21.15.78:0/2635834785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3fe8a10 0x55fed3ff4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.046+0000 7f4e71312640 1 -- 172.21.15.78:0/2635834785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3fe8a10 msgr2=0x55fed3ff4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.046+0000 7f4e71312640 1 --2- 172.21.15.78:0/2635834785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3fe8a10 0x55fed3ff4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.046+0000 7f4e71312640 1 -- 172.21.15.78:0/2635834785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe80f0 msgr2=0x55fed3fe84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.046+0000 7f4e71312640 1 --2- 172.21.15.78:0/2635834785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe80f0 0x55fed3fe84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.046+0000 7f4e71312640 1 -- 172.21.15.78:0/2635834785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed3cac650 con 0x55fed3ff49a0 2024-09-11T01:00:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.046+0000 7f4e71312640 1 --2- 172.21.15.78:0/2635834785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 0x55fed3ff6d90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009900 tx=0x7f4e60031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2667d79e405cad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.046+0000 7f4e6affd640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4e540b5450 0x7f4e540b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.046+0000 7f4e72314640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4e540b5450 0x7f4e540b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.046+0000 7f4e53fff640 1 -- 172.21.15.78:0/2635834785 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60040070 con 0x55fed3ff49a0 2024-09-11T01:00:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.046+0000 7f4e6affd640 1 -- 172.21.15.78:0/2269184657 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4e5c0e5720 con 0x7f4e6c102da0 2024-09-11T01:00:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.046+0000 7f4e53fff640 1 -- 172.21.15.78:0/2635834785 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e600316a0 con 0x55fed3ff49a0 2024-09-11T01:00:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.046+0000 7f4e53fff640 1 -- 172.21.15.78:0/2635834785 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60035040 con 0x55fed3ff49a0 2024-09-11T01:00:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.047+0000 7f4e7614c740 1 -- 172.21.15.78:0/2635834785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 msgr2=0x55fed3ff6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.047+0000 7f4e72314640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4e540b5450 0x7f4e540b7910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400cbd0 tx=0x7f4e6400c380 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.047+0000 7f4e7614c740 1 --2- 172.21.15.78:0/2635834785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 0x55fed3ff6d90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009900 tx=0x7f4e60031de0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.047+0000 7f4e7614c740 1 -- 172.21.15.78:0/2635834785 shutdown_connections 2024-09-11T01:00:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.047+0000 7f4e7614c740 1 --2- 172.21.15.78:0/2635834785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 0x55fed3ff6d90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.047+0000 7f4e7614c740 1 --2- 172.21.15.78:0/2635834785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3fe8a10 0x55fed3ff4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.047+0000 7f4e7614c740 1 --2- 172.21.15.78:0/2635834785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe80f0 0x55fed3fe84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.047+0000 7f4e7614c740 1 -- 172.21.15.78:0/2635834785 >> 172.21.15.78:0/2635834785 conn(0x55fed3fe5f20 msgr2=0x55fed3fe70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.047+0000 7f4e7614c740 1 -- 172.21.15.78:0/2635834785 shutdown_connections 2024-09-11T01:00:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.047+0000 7f4e7614c740 1 -- 172.21.15.78:0/2635834785 wait complete. 2024-09-11T01:00:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e7614c740 1 -- start start 2024-09-11T01:00:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e7614c740 1 Processor -- start 2024-09-11T01:00:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e7614c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 0x55fed3ff4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e7614c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe8a10 0x55fed3ff16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e7614c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3ff49a0 0x55fed3ff1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e7614c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fed3daf700 con 0x55fed3fe80f0 2024-09-11T01:00:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e7614c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fed3ee9160 con 0x55fed3fe8a10 2024-09-11T01:00:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e7614c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fed3ff75c0 con 0x55fed3ff49a0 2024-09-11T01:00:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e7614c740 10 client.0 _ll_register_callbacks cb 0x55fed3f029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e71b13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 0x55fed3ff4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e71b13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 0x55fed3ff4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44734/0 (socket says 172.21.15.78:44734) 2024-09-11T01:00:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e71b13640 1 -- 172.21.15.78:0/991258044 learned_addr learned my addr 172.21.15.78:0/991258044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e71312640 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe8a10 0x55fed3ff16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e72314640 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3ff49a0 0x55fed3ff1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e71b13640 1 -- 172.21.15.78:0/991258044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3ff49a0 msgr2=0x55fed3ff1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:24.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.048+0000 7f4e71b13640 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3ff49a0 0x55fed3ff1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e71b13640 1 -- 172.21.15.78:0/991258044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe8a10 msgr2=0x55fed3ff16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:24.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e71b13640 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe8a10 0x55fed3ff16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e71b13640 1 -- 172.21.15.78:0/991258044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed3f16c90 con 0x55fed3fe80f0 2024-09-11T01:00:24.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e71b13640 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 0x55fed3ff4280 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0ab9c0 tx=0x7f4e5c0abbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b94613686a4b3d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:24.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e517fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:00:24.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e517fa640 1 -- 172.21.15.78:0/991258044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0ab7d0 con 0x55fed3fe80f0 2024-09-11T01:00:24.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e517fa640 1 -- 172.21.15.78:0/991258044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4e5c0b4d10 con 0x55fed3fe80f0 2024-09-11T01:00:24.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e7614c740 1 -- 172.21.15.78:0/991258044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e600316a0 con 0x55fed3fe80f0 2024-09-11T01:00:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e517fa640 1 -- 172.21.15.78:0/991258044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e5c0abcc0 con 0x55fed3fe80f0 2024-09-11T01:00:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e7614c740 10 client.15738 fetch_fsmap learned FSMap version 30 2024-09-11T01:00:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e7614c740 1 -- 172.21.15.78:0/991258044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fed4002f40 con 0x55fed3fe80f0 2024-09-11T01:00:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e517fa640 1 -- 172.21.15.78:0/991258044 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e600316a0 con 0x55fed3fe80f0 2024-09-11T01:00:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e517fa640 1 -- 172.21.15.78:0/991258044 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f4e5c0e5bd0 con 0x55fed3fe80f0 2024-09-11T01:00:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e7614c740 10 client.15738 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T01:00:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e7614c740 10 client.15738 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:00:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e7614c740 20 client.15738 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e7614c740 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55fed4004350 0x55fed40067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e7614c740 4 client.15738 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:00:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.049+0000 7f4e7614c740 1 -- 172.21.15.78:0/991258044 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fed3f60df0 con 0x55fed4004350 2024-09-11T01:00:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.050+0000 7f4e71312640 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55fed4004350 0x55fed40067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.050+0000 7f4e71312640 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55fed4004350 0x55fed40067c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=49d80951e538fa0c server_cookie=75d5e821859fff4c in_seq=0 out_seq=0 2024-09-11T01:00:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.050+0000 7f4e517fa640 10 client.15738 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:00:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e517fa640 1 -- 172.21.15.78:0/991258044 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f4e6004ff90 con 0x55fed4004350 2024-09-11T01:00:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e517fa640 10 client.15738 handle_command_reply: tid=0 2024-09-11T01:00:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e7614c740 1 client.15738 shutdown 2024-09-11T01:00:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e7614c740 1 -- 172.21.15.78:0/991258044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 msgr2=0x55fed3ff4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e7614c740 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 0x55fed3ff4280 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0ab9c0 tx=0x7f4e5c0abbc0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e7614c740 1 -- 172.21.15.78:0/991258044 shutdown_connections 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e7614c740 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55fed4004350 0x55fed40067c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e7614c740 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3ff49a0 0x55fed3ff1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e517fa640 0 client.15738 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e7614c740 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe8a10 0x55fed3ff16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e7614c740 1 --2- 172.21.15.78:0/991258044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 0x55fed3ff4280 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e7614c740 1 -- 172.21.15.78:0/991258044 >> 172.21.15.78:0/991258044 conn(0x55fed3fe5f20 msgr2=0x55fed3feba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e7614c740 1 -- 172.21.15.78:0/991258044 shutdown_connections 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e7614c740 1 -- 172.21.15.78:0/991258044 wait complete. 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.051+0000 7f4e7614c740 20 client.15738 trim_cache size 0 max 16384 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.064+0000 7f4e7614c740 1 Processor -- start 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.064+0000 7f4e7614c740 1 -- start start 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.064+0000 7f4e7614c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe80f0 0x55fed3fe6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.064+0000 7f4e7614c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3fe8a10 0x55fed3ff5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.064+0000 7f4e7614c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 0x55fed3ff62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.064+0000 7f4e7614c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1ae4d0 con 0x55fed3ff49a0 2024-09-11T01:00:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.064+0000 7f4e7614c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1b0040 con 0x55fed3fe80f0 2024-09-11T01:00:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.064+0000 7f4e7614c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1aeba0 con 0x55fed3fe8a10 2024-09-11T01:00:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e72314640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 0x55fed3ff62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e72314640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 0x55fed3ff62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44744/0 (socket says 172.21.15.78:44744) 2024-09-11T01:00:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e72314640 1 -- 172.21.15.78:0/2965585993 learned_addr learned my addr 172.21.15.78:0/2965585993 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e71b13640 1 --2- 172.21.15.78:0/2965585993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe80f0 0x55fed3fe6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e71312640 1 --2- 172.21.15.78:0/2965585993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3fe8a10 0x55fed3ff5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e72314640 1 -- 172.21.15.78:0/2965585993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3fe8a10 msgr2=0x55fed3ff5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e72314640 1 --2- 172.21.15.78:0/2965585993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3fe8a10 0x55fed3ff5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e72314640 1 -- 172.21.15.78:0/2965585993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe80f0 msgr2=0x55fed3fe6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e72314640 1 --2- 172.21.15.78:0/2965585993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe80f0 0x55fed3fe6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e72314640 1 -- 172.21.15.78:0/2965585993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed3ff2070 con 0x55fed3ff49a0 2024-09-11T01:00:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e71b13640 1 --2- 172.21.15.78:0/2965585993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe80f0 0x55fed3fe6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e71312640 1 --2- 172.21.15.78:0/2965585993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3fe8a10 0x55fed3ff5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e72314640 1 --2- 172.21.15.78:0/2965585993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 0x55fed3ff62d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4e6401c860 tx=0x7f4e64014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58f0ba96e54870b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e537fe640 1 -- 172.21.15.78:0/2965585993 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6402a070 con 0x55fed3ff49a0 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e537fe640 1 -- 172.21.15.78:0/2965585993 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e640253f0 con 0x55fed3ff49a0 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.065+0000 7f4e537fe640 1 -- 172.21.15.78:0/2965585993 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6400e610 con 0x55fed3ff49a0 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.066+0000 7f4e7614c740 1 -- 172.21.15.78:0/2965585993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 msgr2=0x55fed3ff62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.066+0000 7f4e7614c740 1 --2- 172.21.15.78:0/2965585993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 0x55fed3ff62d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4e6401c860 tx=0x7f4e64014000 comp rx=0 tx=0).stop 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.066+0000 7f4e7614c740 1 -- 172.21.15.78:0/2965585993 shutdown_connections 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.066+0000 7f4e7614c740 1 --2- 172.21.15.78:0/2965585993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3ff49a0 0x55fed3ff62d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.066+0000 7f4e7614c740 1 --2- 172.21.15.78:0/2965585993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3fe8a10 0x55fed3ff5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.066+0000 7f4e7614c740 1 --2- 172.21.15.78:0/2965585993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe80f0 0x55fed3fe6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.066+0000 7f4e7614c740 1 -- 172.21.15.78:0/2965585993 >> 172.21.15.78:0/2965585993 conn(0x55fed3fe5f20 msgr2=0x55fed3feff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.066+0000 7f4e7614c740 1 -- 172.21.15.78:0/2965585993 shutdown_connections 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.066+0000 7f4e7614c740 1 -- 172.21.15.78:0/2965585993 wait complete. 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.066+0000 7f4e7614c740 1 -- start start 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.066+0000 7f4e7614c740 1 Processor -- start 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e7614c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 0x55fed3ff7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e7614c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe8a10 0x55fed3ff8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e7614c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3ff49a0 0x55fed4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e7614c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1af940 con 0x55fed3fe80f0 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e7614c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1af270 con 0x55fed3fe8a10 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e7614c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1ac4d0 con 0x55fed3ff49a0 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e7614c740 10 client.0 _ll_register_callbacks cb 0x7f4e5c0a7c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e71b13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 0x55fed3ff7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e71b13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 0x55fed3ff7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44750/0 (socket says 172.21.15.78:44750) 2024-09-11T01:00:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e71b13640 1 -- 172.21.15.78:0/3159005550 learned_addr learned my addr 172.21.15.78:0/3159005550 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e71312640 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe8a10 0x55fed3ff8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e72314640 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3ff49a0 0x55fed4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e71b13640 1 -- 172.21.15.78:0/3159005550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3ff49a0 msgr2=0x55fed4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e71b13640 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3ff49a0 0x55fed4071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e71b13640 1 -- 172.21.15.78:0/3159005550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe8a10 msgr2=0x55fed3ff8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e71b13640 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe8a10 0x55fed3ff8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e71b13640 1 -- 172.21.15.78:0/3159005550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed3ff8a30 con 0x55fed3fe80f0 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e71b13640 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 0x55fed3ff7c40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0abe80 tx=0x7f4e5c0ab6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19ab69ba22db287 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e517fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.067+0000 7f4e517fa640 1 -- 172.21.15.78:0/3159005550 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0ab430 con 0x55fed3fe80f0 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e517fa640 1 -- 172.21.15.78:0/3159005550 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4e5c0b4f10 con 0x55fed3fe80f0 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e7614c740 1 -- 172.21.15.78:0/3159005550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e640253f0 con 0x55fed3fe80f0 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e517fa640 1 -- 172.21.15.78:0/3159005550 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e5c0ab7d0 con 0x55fed3fe80f0 2024-09-11T01:00:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e7614c740 10 client.15744 fetch_fsmap learned FSMap version 30 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e7614c740 1 -- 172.21.15.78:0/3159005550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fed4072820 con 0x55fed3fe80f0 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e517fa640 1 -- 172.21.15.78:0/3159005550 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e640253f0 con 0x55fed3fe80f0 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e517fa640 1 -- 172.21.15.78:0/3159005550 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f4e5c0b01c0 con 0x55fed3fe80f0 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e7614c740 10 client.15744 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e7614c740 10 client.15744 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e7614c740 20 client.15744 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e7614c740 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55fed4072fa0 0x55fed40753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e7614c740 4 client.15744 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e7614c740 1 -- 172.21.15.78:0/3159005550 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fed3f5c8a0 con 0x55fed4072fa0 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.068+0000 7f4e71312640 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55fed4072fa0 0x55fed40753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e517fa640 10 client.15744 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e71312640 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55fed4072fa0 0x55fed40753c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e0dc07fb17b37857 server_cookie=250f7b8fa2b4cf4b in_seq=0 out_seq=0 2024-09-11T01:00:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e517fa640 1 -- 172.21.15.78:0/3159005550 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fed3f5c8a0 con 0x55fed4072fa0 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e517fa640 10 client.15744 handle_command_reply: tid=0 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e7614c740 1 client.15744 shutdown 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e7614c740 1 -- 172.21.15.78:0/3159005550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 msgr2=0x55fed3ff7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e7614c740 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 0x55fed3ff7c40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0abe80 tx=0x7f4e5c0ab6d0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e7614c740 1 -- 172.21.15.78:0/3159005550 shutdown_connections 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e7614c740 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55fed4072fa0 0x55fed40753c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e7614c740 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed3ff49a0 0x55fed4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e7614c740 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed3fe8a10 0x55fed3ff8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e7614c740 1 --2- 172.21.15.78:0/3159005550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed3fe80f0 0x55fed3ff7c40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e7614c740 1 -- 172.21.15.78:0/3159005550 >> 172.21.15.78:0/3159005550 conn(0x55fed3fe5f20 msgr2=0x55fed3fe4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e517fa640 0 client.15744 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e7614c740 1 -- 172.21.15.78:0/3159005550 shutdown_connections 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e7614c740 1 -- 172.21.15.78:0/3159005550 wait complete. 2024-09-11T01:00:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.069+0000 7f4e7614c740 20 client.15744 trim_cache size 0 max 16384 2024-09-11T01:00:24.090 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:00:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.070+0000 7f4e73d77640 1 -- 172.21.15.78:0/2269184657 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4e540b5450 msgr2=0x7f4e540b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.070+0000 7f4e73d77640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4e540b5450 0x7f4e540b7910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400cbd0 tx=0x7f4e6400c380 comp rx=0 tx=0).stop 2024-09-11T01:00:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.070+0000 7f4e73d77640 1 -- 172.21.15.78:0/2269184657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6c102da0 msgr2=0x7f4e6c199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.070+0000 7f4e73d77640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6c102da0 0x7f4e6c199320 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00ba50 tx=0x7f4e5c00bf10 comp rx=0 tx=0).stop 2024-09-11T01:00:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.070+0000 7f4e73d77640 1 -- 172.21.15.78:0/2269184657 shutdown_connections 2024-09-11T01:00:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.070+0000 7f4e73d77640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4e540b5450 0x7f4e540b7910 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.070+0000 7f4e73d77640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e6c109130 0x7f4e6c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.070+0000 7f4e73d77640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c1087b0 0x7f4e6c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.070+0000 7f4e73d77640 1 --2- 172.21.15.78:0/2269184657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6c102da0 0x7f4e6c199320 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.070+0000 7f4e73d77640 1 -- 172.21.15.78:0/2269184657 >> 172.21.15.78:0/2269184657 conn(0x7f4e6c0fe810 msgr2=0x7f4e6c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.070+0000 7f4e73d77640 1 -- 172.21.15.78:0/2269184657 shutdown_connections 2024-09-11T01:00:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:24.070+0000 7f4e73d77640 1 -- 172.21.15.78:0/2269184657 wait complete. 2024-09-11T01:00:24.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92357 16.15 MB/sec warmup 32 sec latency 2100.092 ms 2024-09-11T01:00:24.783 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:00:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:00:25.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92357 15.66 MB/sec warmup 33 sec latency 3100.163 ms 2024-09-11T01:00:25.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:25 smithi078 ceph-mon[31315]: pgmap v369: 97 pgs: 97 active+clean; 48 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 237 op/s 2024-09-11T01:00:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:25 smithi110 ceph-mon[34712]: pgmap v369: 97 pgs: 97 active+clean; 48 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 237 op/s 2024-09-11T01:00:25.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:25 smithi112 ceph-mon[35384]: pgmap v369: 97 pgs: 97 active+clean; 48 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 237 op/s 2024-09-11T01:00:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.513+0000 7fb0fd746640 1 -- 172.21.15.78:0/2156692833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103cc0 msgr2=0x7fb0f81040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.513+0000 7fb0fd746640 1 --2- 172.21.15.78:0/2156692833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103cc0 0x7fb0f81040a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009ca0 tx=0x7fb0e802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.513+0000 7fb0fd746640 1 -- 172.21.15.78:0/2156692833 shutdown_connections 2024-09-11T01:00:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.513+0000 7fb0fd746640 1 --2- 172.21.15.78:0/2156692833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f810d120 0x7fb0f810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.513+0000 7fb0fd746640 1 --2- 172.21.15.78:0/2156692833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0f81045e0 0x7fb0f810cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.513+0000 7fb0fd746640 1 --2- 172.21.15.78:0/2156692833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103cc0 0x7fb0f81040a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.513+0000 7fb0fd746640 1 -- 172.21.15.78:0/2156692833 >> 172.21.15.78:0/2156692833 conn(0x7fb0f80fd700 msgr2=0x7fb0f80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.514+0000 7fb0fd746640 1 -- 172.21.15.78:0/2156692833 shutdown_connections 2024-09-11T01:00:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.514+0000 7fb0fd746640 1 -- 172.21.15.78:0/2156692833 wait complete. 2024-09-11T01:00:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.514+0000 7fb0fd746640 1 Processor -- start 2024-09-11T01:00:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.514+0000 7fb0fd746640 1 -- start start 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.514+0000 7fb0fd746640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103cc0 0x7fb0f8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.514+0000 7fb0fd746640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f81045e0 0x7fb0f8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.514+0000 7fb0fd746640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0f810d120 0x7fb0f819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.514+0000 7fb0fd746640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f8112dd0 con 0x7fb0f8103cc0 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.514+0000 7fb0fd746640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f8112c50 con 0x7fb0f810d120 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.514+0000 7fb0fd746640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f8112f50 con 0x7fb0f81045e0 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0f6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103cc0 0x7fb0f8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0f6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103cc0 0x7fb0f8199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44772/0 (socket says 172.21.15.78:44772) 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0f6ffd640 1 -- 172.21.15.78:0/3576519637 learned_addr learned my addr 172.21.15.78:0/3576519637 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0f67fc640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f81045e0 0x7fb0f8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0f77fe640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0f810d120 0x7fb0f819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0f6ffd640 1 -- 172.21.15.78:0/3576519637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f81045e0 msgr2=0x7fb0f8199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0f6ffd640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f81045e0 0x7fb0f8199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0f6ffd640 1 -- 172.21.15.78:0/3576519637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0f810d120 msgr2=0x7fb0f819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0f6ffd640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0f810d120 0x7fb0f819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0f6ffd640 1 -- 172.21.15.78:0/3576519637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f819f2f0 con 0x7fb0f8103cc0 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0f6ffd640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103cc0 0x7fb0f8199240 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb0e802ade0 tx=0x7fb0e8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0effff640 1 -- 172.21.15.78:0/3576519637 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8003d60 con 0x7fb0f8103cc0 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0effff640 1 -- 172.21.15.78:0/3576519637 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e80044f0 con 0x7fb0f8103cc0 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0fd746640 1 -- 172.21.15.78:0/3576519637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f819f580 con 0x7fb0f8103cc0 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0effff640 1 -- 172.21.15.78:0/3576519637 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e802fd10 con 0x7fb0f8103cc0 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.515+0000 7fb0fd746640 1 -- 172.21.15.78:0/3576519637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f81a5d20 con 0x7fb0f8103cc0 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0ffb1b740 1 Processor -- start 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0ffb1b740 1 -- start start 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0effff640 1 -- 172.21.15.78:0/3576519637 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb0e8004040 con 0x7fb0f8103cc0 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0ffb1b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff50f0 0x55d986ff54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0ffb1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff5a10 0x55d987001460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0ffb1b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d987003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0ffb1b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d986f6bea0 con 0x55d9870019a0 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0ffb1b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d986f5b2d0 con 0x55d986ff50f0 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0ffb1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d986f5f0e0 con 0x55d986ff5a10 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0f67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d987003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0f67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d987003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44782/0 (socket says 172.21.15.78:44782) 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0f67fc640 1 -- 172.21.15.78:0/70018594 learned_addr learned my addr 172.21.15.78:0/70018594 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0f77fe640 1 --2- 172.21.15.78:0/70018594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff50f0 0x55d986ff54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.518+0000 7fb0f6ffd640 1 --2- 172.21.15.78:0/70018594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff5a10 0x55d987001460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0f67fc640 1 -- 172.21.15.78:0/70018594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff5a10 msgr2=0x55d987001460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0f67fc640 1 --2- 172.21.15.78:0/70018594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff5a10 0x55d987001460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0f67fc640 1 -- 172.21.15.78:0/70018594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff50f0 msgr2=0x55d986ff54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0f67fc640 1 --2- 172.21.15.78:0/70018594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff50f0 0x55d986ff54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0f67fc640 1 -- 172.21.15.78:0/70018594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d986cb9650 con 0x55d9870019a0 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0f77fe640 1 --2- 172.21.15.78:0/70018594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff50f0 0x55d986ff54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0effff640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0e40b5450 0x7fb0e40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0f67fc640 1 --2- 172.21.15.78:0/70018594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d987003d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000bae0 tx=0x7fb0e000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe2a4a2f1c6972 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0f77fe640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0e40b5450 0x7fb0e40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0ed7fa640 1 -- 172.21.15.78:0/70018594 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0017420 con 0x55d9870019a0 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0effff640 1 -- 172.21.15.78:0/3576519637 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb0e8107570 con 0x7fb0f8103cc0 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0ed7fa640 1 -- 172.21.15.78:0/70018594 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e0017a70 con 0x55d9870019a0 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.519+0000 7fb0ed7fa640 1 -- 172.21.15.78:0/70018594 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000f990 con 0x55d9870019a0 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.520+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/70018594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 msgr2=0x55d987003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.520+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/70018594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d987003d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000bae0 tx=0x7fb0e000d340 comp rx=0 tx=0).stop 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.520+0000 7fb0f77fe640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0e40b5450 0x7fb0e40b7910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000a930 tx=0x7fb0f0008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.520+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/70018594 shutdown_connections 2024-09-11T01:00:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.520+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/70018594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d987003d90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.520+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/70018594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff5a10 0x55d987001460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.520+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/70018594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff50f0 0x55d986ff54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.520+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/70018594 >> 172.21.15.78:0/70018594 conn(0x55d986ff2f20 msgr2=0x55d986ff40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.520+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/70018594 shutdown_connections 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.520+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/70018594 wait complete. 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.520+0000 7fb0ffb1b740 1 -- start start 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.520+0000 7fb0ffb1b740 1 Processor -- start 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0ffb1b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff50f0 0x55d9870011f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0ffb1b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d986ff5a10 0x55d986ffe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0ffb1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9870019a0 0x55d986ffeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0ffb1b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d986dbc700 con 0x55d986ff5a10 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0ffb1b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d986ef6160 con 0x55d986ff50f0 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0ffb1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d987004620 con 0x55d9870019a0 2024-09-11T01:00:25.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0ffb1b740 10 client.0 _ll_register_callbacks cb 0x55d986f0f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0f67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d986ff5a10 0x55d986ffe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0f67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d986ff5a10 0x55d986ffe640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44788/0 (socket says 172.21.15.78:44788) 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0f6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff50f0 0x55d9870011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0f67fc640 1 -- 172.21.15.78:0/2076380902 learned_addr learned my addr 172.21.15.78:0/2076380902 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0f77fe640 1 --2- 172.21.15.78:0/2076380902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9870019a0 0x55d986ffeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0f67fc640 1 -- 172.21.15.78:0/2076380902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9870019a0 msgr2=0x55d986ffeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0f67fc640 1 --2- 172.21.15.78:0/2076380902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9870019a0 0x55d986ffeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0f67fc640 1 -- 172.21.15.78:0/2076380902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff50f0 msgr2=0x55d9870011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0f67fc640 1 --2- 172.21.15.78:0/2076380902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff50f0 0x55d9870011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0f67fc640 1 -- 172.21.15.78:0/2076380902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d986f23c90 con 0x55d986ff5a10 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0f67fc640 1 --2- 172.21.15.78:0/2076380902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d986ff5a10 0x55d986ffe640 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0004770 tx=0x7fb0e0017ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742a76c8cf10f081 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.521+0000 7fb0de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:00:25.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0de7fc640 1 -- 172.21.15.78:0/2076380902 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0026070 con 0x55d986ff5a10 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0de7fc640 1 -- 172.21.15.78:0/2076380902 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb0e0017830 con 0x55d986ff5a10 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2076380902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e0017a70 con 0x55d986ff5a10 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0de7fc640 1 -- 172.21.15.78:0/2076380902 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb0e0032050 con 0x55d986ff5a10 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0ffb1b740 10 client.15756 fetch_fsmap learned FSMap version 30 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2076380902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d98700feb0 con 0x55d986ff5a10 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0de7fc640 1 -- 172.21.15.78:0/2076380902 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e0017a70 con 0x55d986ff5a10 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0de7fc640 1 -- 172.21.15.78:0/2076380902 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb0e0022810 con 0x55d986ff5a10 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0ffb1b740 10 client.15756 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0ffb1b740 10 client.15756 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0ffb1b740 20 client.15756 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/2076380902 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55d987011260 0x55d987013650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0ffb1b740 4 client.15756 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:00:25.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.522+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2076380902 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d986f6ddf0 con 0x55d987011260 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.523+0000 7fb0f6ffd640 1 --2- 172.21.15.78:0/2076380902 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55d987011260 0x55d987013650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.523+0000 7fb0de7fc640 10 client.15756 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.523+0000 7fb0f6ffd640 1 --2- 172.21.15.78:0/2076380902 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55d987011260 0x55d987013650 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5d3b011424d55cb server_cookie=19da6e923fb94a05 in_seq=0 out_seq=0 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0de7fc640 1 -- 172.21.15.78:0/2076380902 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d986f6ddf0 con 0x55d987011260 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0de7fc640 10 client.15756 handle_command_reply: tid=0 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0ffb1b740 1 client.15756 shutdown 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2076380902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d986ff5a10 msgr2=0x55d986ffe640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/2076380902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d986ff5a10 0x55d986ffe640 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0004770 tx=0x7fb0e0017ca0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2076380902 shutdown_connections 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/2076380902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d9870019a0 0x55d986ffeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/2076380902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d986ff5a10 0x55d986ffe640 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/2076380902 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55d987011260 0x55d987013650 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/2076380902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff50f0 0x55d9870011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0de7fc640 0 client.15756 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2076380902 >> 172.21.15.78:0/2076380902 conn(0x55d986ff2f20 msgr2=0x55d986ff8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2076380902 shutdown_connections 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2076380902 wait complete. 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.524+0000 7fb0ffb1b740 20 client.15756 trim_cache size 0 max 16384 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.537+0000 7fb0ffb1b740 1 Processor -- start 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.537+0000 7fb0ffb1b740 1 -- start start 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.537+0000 7fb0ffb1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff50f0 0x55d987002d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.537+0000 7fb0ffb1b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff5a10 0x55d9870032d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.537+0000 7fb0ffb1b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d986ffb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.537+0000 7fb0ffb1b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f81ae580 con 0x55d9870019a0 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.537+0000 7fb0ffb1b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f81b00f0 con 0x55d986ff5a10 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.537+0000 7fb0ffb1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f81aec50 con 0x55d986ff50f0 2024-09-11T01:00:25.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.537+0000 7fb0f77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d986ffb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.537+0000 7fb0f77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d986ffb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44802/0 (socket says 172.21.15.78:44802) 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.537+0000 7fb0f77fe640 1 -- 172.21.15.78:0/2504151600 learned_addr learned my addr 172.21.15.78:0/2504151600 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.537+0000 7fb0f6ffd640 1 --2- 172.21.15.78:0/2504151600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff50f0 0x55d987002d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.538+0000 7fb0f77fe640 1 -- 172.21.15.78:0/2504151600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff50f0 msgr2=0x55d987002d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.538+0000 7fb0f77fe640 1 --2- 172.21.15.78:0/2504151600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff50f0 0x55d987002d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.538+0000 7fb0f77fe640 1 -- 172.21.15.78:0/2504151600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff5a10 msgr2=0x55d9870032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.538+0000 7fb0f67fc640 1 --2- 172.21.15.78:0/2504151600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff5a10 0x55d9870032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.538+0000 7fb0f77fe640 1 --2- 172.21.15.78:0/2504151600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff5a10 0x55d9870032d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.538+0000 7fb0f77fe640 1 -- 172.21.15.78:0/2504151600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d986ff39a0 con 0x55d9870019a0 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.538+0000 7fb0f77fe640 1 --2- 172.21.15.78:0/2504151600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d986ffb850 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000eba0 tx=0x7fb0f0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb4e94e13b22189 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.538+0000 7fb0ecff9640 1 -- 172.21.15.78:0/2504151600 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0024070 con 0x55d9870019a0 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.538+0000 7fb0ecff9640 1 -- 172.21.15.78:0/2504151600 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0f001b040 con 0x55d9870019a0 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.538+0000 7fb0ecff9640 1 -- 172.21.15.78:0/2504151600 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f001a040 con 0x55d9870019a0 2024-09-11T01:00:25.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2504151600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 msgr2=0x55d986ffb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/2504151600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d986ffb850 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000eba0 tx=0x7fb0f0011000 comp rx=0 tx=0).stop 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2504151600 shutdown_connections 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/2504151600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d986ffb850 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/2504151600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff5a10 0x55d9870032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/2504151600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff50f0 0x55d987002d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2504151600 >> 172.21.15.78:0/2504151600 conn(0x55d986ff2f20 msgr2=0x55d987000490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2504151600 shutdown_connections 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/2504151600 wait complete. 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 -- start start 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 Processor -- start 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff50f0 0x55d987009d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff5a10 0x55d98700a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d987004ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f81af9f0 con 0x55d9870019a0 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f81af320 con 0x55d986ff5a10 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f81ac560 con 0x55d986ff50f0 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.539+0000 7fb0ffb1b740 10 client.0 _ll_register_callbacks cb 0x7fb0e000b930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0f77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d987004ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0f77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d987004ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44812/0 (socket says 172.21.15.78:44812) 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0f77fe640 1 -- 172.21.15.78:0/593547317 learned_addr learned my addr 172.21.15.78:0/593547317 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0f6ffd640 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff50f0 0x55d987009d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0f67fc640 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff5a10 0x55d98700a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0f77fe640 1 -- 172.21.15.78:0/593547317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff50f0 msgr2=0x55d987009d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0f77fe640 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff50f0 0x55d987009d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0f77fe640 1 -- 172.21.15.78:0/593547317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff5a10 msgr2=0x55d98700a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0f77fe640 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff5a10 0x55d98700a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0f77fe640 1 -- 172.21.15.78:0/593547317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9870052f0 con 0x55d9870019a0 2024-09-11T01:00:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0f77fe640 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d987004ac0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0014c30 tx=0x7fb0f0013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a40bfbdb8d810b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0de7fc640 1 -- 172.21.15.78:0/593547317 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f001a070 con 0x55d9870019a0 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0de7fc640 1 -- 172.21.15.78:0/593547317 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb0f0029450 con 0x55d9870019a0 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/593547317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f001b040 con 0x55d9870019a0 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0de7fc640 1 -- 172.21.15.78:0/593547317 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb0f0024040 con 0x55d9870019a0 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0ffb1b740 10 client.15762 fetch_fsmap learned FSMap version 30 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/593547317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d98707ebc0 con 0x55d9870019a0 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.540+0000 7fb0de7fc640 1 -- 172.21.15.78:0/593547317 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0f001b040 con 0x55d9870019a0 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.541+0000 7fb0de7fc640 1 -- 172.21.15.78:0/593547317 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb0f0030030 con 0x55d9870019a0 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.541+0000 7fb0ffb1b740 10 client.15762 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.541+0000 7fb0ffb1b740 10 client.15762 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.541+0000 7fb0ffb1b740 20 client.15762 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.541+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55d98707f300 0x55d987081720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.541+0000 7fb0ffb1b740 4 client.15762 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.541+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/593547317 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d986f698a0 con 0x55d98707f300 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.541+0000 7fb0f6ffd640 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55d98707f300 0x55d987081720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0de7fc640 10 client.15762 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0f6ffd640 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55d98707f300 0x55d987081720 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87d702424a520c4 server_cookie=9e05112a45432a1 in_seq=0 out_seq=0 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0de7fc640 1 -- 172.21.15.78:0/593547317 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d986f698a0 con 0x55d98707f300 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0de7fc640 10 client.15762 handle_command_reply: tid=0 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0ffb1b740 1 client.15762 shutdown 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/593547317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 msgr2=0x55d987004ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d987004ac0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0014c30 tx=0x7fb0f0013f90 comp rx=0 tx=0).stop 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/593547317 shutdown_connections 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d9870019a0 0x55d987004ac0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55d98707f300 0x55d987081720 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d986ff5a10 0x55d98700a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0ffb1b740 1 --2- 172.21.15.78:0/593547317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d986ff50f0 0x55d987009d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/593547317 >> 172.21.15.78:0/593547317 conn(0x55d986ff2f20 msgr2=0x55d986ff1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:25.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0de7fc640 0 client.15762 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:00:25.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/593547317 shutdown_connections 2024-09-11T01:00:25.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0ffb1b740 1 -- 172.21.15.78:0/593547317 wait complete. 2024-09-11T01:00:25.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.542+0000 7fb0ffb1b740 20 client.15762 trim_cache size 0 max 16384 2024-09-11T01:00:25.560 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:00:25.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.543+0000 7fb0fd746640 1 -- 172.21.15.78:0/3576519637 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0e40b5450 msgr2=0x7fb0e40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:25.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.543+0000 7fb0fd746640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0e40b5450 0x7fb0e40b7910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000a930 tx=0x7fb0f0008040 comp rx=0 tx=0).stop 2024-09-11T01:00:25.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.543+0000 7fb0fd746640 1 -- 172.21.15.78:0/3576519637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103cc0 msgr2=0x7fb0f8199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:25.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.543+0000 7fb0fd746640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103cc0 0x7fb0f8199240 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb0e802ade0 tx=0x7fb0e8003970 comp rx=0 tx=0).stop 2024-09-11T01:00:25.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.543+0000 7fb0fd746640 1 -- 172.21.15.78:0/3576519637 shutdown_connections 2024-09-11T01:00:25.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.543+0000 7fb0fd746640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0e40b5450 0x7fb0e40b7910 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.543+0000 7fb0fd746640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0f810d120 0x7fb0f819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.543+0000 7fb0fd746640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f81045e0 0x7fb0f8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.543+0000 7fb0fd746640 1 --2- 172.21.15.78:0/3576519637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103cc0 0x7fb0f8199240 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:25.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.543+0000 7fb0fd746640 1 -- 172.21.15.78:0/3576519637 >> 172.21.15.78:0/3576519637 conn(0x7fb0f80fd700 msgr2=0x7fb0f810f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:25.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.543+0000 7fb0fd746640 1 -- 172.21.15.78:0/3576519637 shutdown_connections 2024-09-11T01:00:25.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:25.543+0000 7fb0fd746640 1 -- 172.21.15.78:0/3576519637 wait complete. 2024-09-11T01:00:26.225 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:00:26.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92357 15.20 MB/sec warmup 34 sec latency 4100.186 ms 2024-09-11T01:00:26.468 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:00:26.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.937+0000 7f71cf1c0640 1 -- 172.21.15.78:0/824732851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c8102dc0 msgr2=0x7f71c81031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:26.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.937+0000 7f71cf1c0640 1 --2- 172.21.15.78:0/824732851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c8102dc0 0x7f71c81031a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009ca0 tx=0x7f71c402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.937+0000 7f71cf1c0640 1 -- 172.21.15.78:0/824732851 shutdown_connections 2024-09-11T01:00:26.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.937+0000 7f71cf1c0640 1 --2- 172.21.15.78:0/824732851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c8109150 0x7f71c810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.937+0000 7f71cf1c0640 1 --2- 172.21.15.78:0/824732851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c81087d0 0x7f71c8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.937+0000 7f71cf1c0640 1 --2- 172.21.15.78:0/824732851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c8102dc0 0x7f71c81031a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.937+0000 7f71cf1c0640 1 -- 172.21.15.78:0/824732851 >> 172.21.15.78:0/824732851 conn(0x7f71c80fe830 msgr2=0x7f71c8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:26.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.937+0000 7f71cf1c0640 1 -- 172.21.15.78:0/824732851 shutdown_connections 2024-09-11T01:00:26.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.937+0000 7f71cf1c0640 1 -- 172.21.15.78:0/824732851 wait complete. 2024-09-11T01:00:26.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71cf1c0640 1 Processor -- start 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71cf1c0640 1 -- start start 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71cf1c0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c8102dc0 0x7f71c80754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71cf1c0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c81087d0 0x7f71c80759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71cf1c0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c8109150 0x7f71c8075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71cf1c0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8112e10 con 0x7f71c81087d0 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71cf1c0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8112c90 con 0x7f71c8109150 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71cf1c0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8112f90 con 0x7f71c8102dc0 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c81087d0 0x7f71c80759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c81087d0 0x7f71c80759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44834/0 (socket says 172.21.15.78:44834) 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71bffff640 1 -- 172.21.15.78:0/104432366 learned_addr learned my addr 172.21.15.78:0/104432366 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71cd75d640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c8109150 0x7f71c8075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71ccf5c640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c8102dc0 0x7f71c80754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71bffff640 1 -- 172.21.15.78:0/104432366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c8102dc0 msgr2=0x7f71c80754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:26.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71bffff640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c8102dc0 0x7f71c80754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71bffff640 1 -- 172.21.15.78:0/104432366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c8109150 msgr2=0x7f71c8075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:26.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71bffff640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c8109150 0x7f71c8075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71bffff640 1 -- 172.21.15.78:0/104432366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c81a76a0 con 0x7f71c81087d0 2024-09-11T01:00:26.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71cd75d640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c8109150 0x7f71c8075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:00:26.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.938+0000 7f71ccf5c640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c8102dc0 0x7f71c80754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:00:26.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.939+0000 7f71bffff640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c81087d0 0x7f71c80759e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f71b800b4a0 tx=0x7f71b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:26.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.939+0000 7f71bdffb640 1 -- 172.21.15.78:0/104432366 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8014070 con 0x7f71c81087d0 2024-09-11T01:00:26.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.939+0000 7f71cf1c0640 1 -- 172.21.15.78:0/104432366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c81a7990 con 0x7f71c81087d0 2024-09-11T01:00:26.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.939+0000 7f71bdffb640 1 -- 172.21.15.78:0/104432366 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b8010ac0 con 0x7f71c81087d0 2024-09-11T01:00:26.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.939+0000 7f71bdffb640 1 -- 172.21.15.78:0/104432366 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8017c30 con 0x7f71c81087d0 2024-09-11T01:00:26.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.939+0000 7f71cf1c0640 1 -- 172.21.15.78:0/104432366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c81a7f00 con 0x7f71c81087d0 2024-09-11T01:00:26.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.941+0000 7f71bdffb640 1 -- 172.21.15.78:0/104432366 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f71b8010610 con 0x7f71c81087d0 2024-09-11T01:00:26.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.941+0000 7f71d1595740 1 Processor -- start 2024-09-11T01:00:26.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.941+0000 7f71d1595740 1 -- start start 2024-09-11T01:00:26.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71d1595740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d374d0 0x55b3d3d378b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71d1595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d37df0 0x55b3d3d43840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71d1595740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d43d80 0x55b3d3d46170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71d1595740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b3d3cadea0 con 0x55b3d3d43d80 2024-09-11T01:00:26.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71d1595740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b3d3c9d2d0 con 0x55b3d3d374d0 2024-09-11T01:00:26.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71d1595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b3d3ca10e0 con 0x55b3d3d37df0 2024-09-11T01:00:26.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71cd75d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d374d0 0x55b3d3d378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d43d80 0x55b3d3d46170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71ccf5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d37df0 0x55b3d3d43840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71cd75d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d374d0 0x55b3d3d378b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:54074/0 (socket says 172.21.15.78:54074) 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71cd75d640 1 -- 172.21.15.78:0/2825381435 learned_addr learned my addr 172.21.15.78:0/2825381435 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71bdffb640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71b00b5450 0x7f71b00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71cd75d640 1 -- 172.21.15.78:0/2825381435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d37df0 msgr2=0x55b3d3d43840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71cd75d640 1 --2- 172.21.15.78:0/2825381435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d37df0 0x55b3d3d43840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71cd75d640 1 -- 172.21.15.78:0/2825381435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d43d80 msgr2=0x55b3d3d46170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71cd75d640 1 --2- 172.21.15.78:0/2825381435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d43d80 0x55b3d3d46170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71cd75d640 1 -- 172.21.15.78:0/2825381435 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3d39fb650 con 0x55b3d3d374d0 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71ccf5c640 1 --2- 172.21.15.78:0/2825381435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d37df0 0x55b3d3d43840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71cd75d640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71b00b5450 0x7f71b00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.942+0000 7f71bdffb640 1 -- 172.21.15.78:0/104432366 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f71b80e55e0 con 0x7f71c81087d0 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.943+0000 7f71cd75d640 1 --2- 172.21.15.78:0/2825381435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d374d0 0x55b3d3d378b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f71c000cda0 tx=0x7f71c000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf99ef25a7e264b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.943+0000 7f71cd75d640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71b00b5450 0x7f71b00b7910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71c001f000 tx=0x7f71c0020000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:26.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.943+0000 7f71aeffd640 1 -- 172.21.15.78:0/2825381435 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0022070 con 0x55b3d3d374d0 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.943+0000 7f71aeffd640 1 -- 172.21.15.78:0/2825381435 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c0011d60 con 0x55b3d3d374d0 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.943+0000 7f71aeffd640 1 -- 172.21.15.78:0/2825381435 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c001cd70 con 0x55b3d3d374d0 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.944+0000 7f71d1595740 1 -- 172.21.15.78:0/2825381435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d374d0 msgr2=0x55b3d3d378b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.944+0000 7f71d1595740 1 --2- 172.21.15.78:0/2825381435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d374d0 0x55b3d3d378b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f71c000cda0 tx=0x7f71c000e720 comp rx=0 tx=0).stop 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.944+0000 7f71d1595740 1 -- 172.21.15.78:0/2825381435 shutdown_connections 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.944+0000 7f71d1595740 1 --2- 172.21.15.78:0/2825381435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d43d80 0x55b3d3d46170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.944+0000 7f71d1595740 1 --2- 172.21.15.78:0/2825381435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d37df0 0x55b3d3d43840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.944+0000 7f71d1595740 1 --2- 172.21.15.78:0/2825381435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d374d0 0x55b3d3d378b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.944+0000 7f71d1595740 1 -- 172.21.15.78:0/2825381435 >> 172.21.15.78:0/2825381435 conn(0x55b3d3d35300 msgr2=0x55b3d3d364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.944+0000 7f71d1595740 1 -- 172.21.15.78:0/2825381435 shutdown_connections 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.944+0000 7f71d1595740 1 -- 172.21.15.78:0/2825381435 wait complete. 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.944+0000 7f71d1595740 1 -- start start 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.944+0000 7f71d1595740 1 Processor -- start 2024-09-11T01:00:26.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.944+0000 7f71d1595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d374d0 0x55b3d3d43670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71d1595740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d37df0 0x55b3d3d40ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71d1595740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d43d80 0x55b3d3d410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71d1595740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b3d3afe700 con 0x55b3d3d43d80 2024-09-11T01:00:26.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71d1595740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b3d3c38160 con 0x55b3d3d37df0 2024-09-11T01:00:26.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71d1595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b3d3d46a00 con 0x55b3d3d374d0 2024-09-11T01:00:26.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71d1595740 10 client.0 _ll_register_callbacks cb 0x55b3d3c519e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:26.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71cd75d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d43d80 0x55b3d3d410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71ccf5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d374d0 0x55b3d3d43670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71ccf5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d374d0 0x55b3d3d43670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53998/0 (socket says 172.21.15.78:53998) 2024-09-11T01:00:26.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71ccf5c640 1 -- 172.21.15.78:0/4211187756 learned_addr learned my addr 172.21.15.78:0/4211187756 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:26.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71cd75d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d43d80 0x55b3d3d410f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44850/0 (socket says 172.21.15.78:44850) 2024-09-11T01:00:26.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71bffff640 1 --2- 172.21.15.78:0/4211187756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d37df0 0x55b3d3d40ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71cd75d640 1 -- 172.21.15.78:0/4211187756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d374d0 msgr2=0x55b3d3d43670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71cd75d640 1 --2- 172.21.15.78:0/4211187756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d374d0 0x55b3d3d43670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71cd75d640 1 -- 172.21.15.78:0/4211187756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d37df0 msgr2=0x55b3d3d40ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71cd75d640 1 --2- 172.21.15.78:0/4211187756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d37df0 0x55b3d3d40ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71cd75d640 1 -- 172.21.15.78:0/4211187756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3d3c65c90 con 0x55b3d3d43d80 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71cd75d640 1 --2- 172.21.15.78:0/4211187756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d43d80 0x55b3d3d410f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f71c000cda0 tx=0x7f71c000c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dc8bcf6f4f11ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.945+0000 7f71a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71a7fff640 1 -- 172.21.15.78:0/4211187756 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0035060 con 0x55b3d3d43d80 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71a7fff640 1 -- 172.21.15.78:0/4211187756 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f71c001cd00 con 0x55b3d3d43d80 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71d1595740 1 -- 172.21.15.78:0/4211187756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c0011d60 con 0x55b3d3d43d80 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71a7fff640 1 -- 172.21.15.78:0/4211187756 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f71c0022040 con 0x55b3d3d43d80 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71d1595740 10 client.15774 fetch_fsmap learned FSMap version 30 2024-09-11T01:00:26.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71d1595740 1 -- 172.21.15.78:0/4211187756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3d3d52360 con 0x55b3d3d43d80 2024-09-11T01:00:26.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71a7fff640 1 -- 172.21.15.78:0/4211187756 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c0011d60 con 0x55b3d3d43d80 2024-09-11T01:00:26.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71a7fff640 1 -- 172.21.15.78:0/4211187756 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f71c0038020 con 0x55b3d3d43d80 2024-09-11T01:00:26.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71d1595740 10 client.15774 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T01:00:26.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71d1595740 10 client.15774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:00:26.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71d1595740 20 client.15774 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:26.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71d1595740 1 --2- 172.21.15.78:0/4211187756 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b3d3d537c0 0x55b3d3d55bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71d1595740 4 client.15774 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:00:26.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71d1595740 1 -- 172.21.15.78:0/4211187756 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3d3cafdf0 con 0x55b3d3d537c0 2024-09-11T01:00:26.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.946+0000 7f71ccf5c640 1 --2- 172.21.15.78:0/4211187756 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b3d3d537c0 0x55b3d3d55bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.947+0000 7f71a7fff640 10 client.15774 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:00:26.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.947+0000 7f71ccf5c640 1 --2- 172.21.15.78:0/4211187756 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b3d3d537c0 0x55b3d3d55bb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d72a889af64bc58 server_cookie=afa0c26b07f7b545 in_seq=0 out_seq=0 2024-09-11T01:00:26.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71a7fff640 1 -- 172.21.15.78:0/4211187756 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b3d3cafdf0 con 0x55b3d3d537c0 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71a7fff640 10 client.15774 handle_command_reply: tid=0 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71d1595740 1 client.15774 shutdown 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71d1595740 1 -- 172.21.15.78:0/4211187756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d43d80 msgr2=0x55b3d3d410f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71d1595740 1 --2- 172.21.15.78:0/4211187756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d43d80 0x55b3d3d410f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f71c000cda0 tx=0x7f71c000c710 comp rx=0 tx=0).stop 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71d1595740 1 -- 172.21.15.78:0/4211187756 shutdown_connections 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71d1595740 1 --2- 172.21.15.78:0/4211187756 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b3d3d537c0 0x55b3d3d55bb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71d1595740 1 --2- 172.21.15.78:0/4211187756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d43d80 0x55b3d3d410f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71d1595740 1 --2- 172.21.15.78:0/4211187756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d37df0 0x55b3d3d40ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71d1595740 1 --2- 172.21.15.78:0/4211187756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d374d0 0x55b3d3d43670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71a7fff640 0 client.15774 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71d1595740 1 -- 172.21.15.78:0/4211187756 >> 172.21.15.78:0/4211187756 conn(0x55b3d3d35300 msgr2=0x55b3d3d3ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71d1595740 1 -- 172.21.15.78:0/4211187756 shutdown_connections 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71d1595740 1 -- 172.21.15.78:0/4211187756 wait complete. 2024-09-11T01:00:26.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.948+0000 7f71d1595740 20 client.15774 trim_cache size 0 max 16384 2024-09-11T01:00:26.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71d1595740 1 Processor -- start 2024-09-11T01:00:26.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71d1595740 1 -- start start 2024-09-11T01:00:26.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71d1595740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d374d0 0x55b3d3d4fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71d1595740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d37df0 0x55b3d3d30320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71d1595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d43d80 0x55b3d3d42800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71d1595740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81b07b0 con 0x55b3d3d374d0 2024-09-11T01:00:26.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71d1595740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81b2320 con 0x55b3d3d37df0 2024-09-11T01:00:26.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71d1595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81b0e80 con 0x55b3d3d43d80 2024-09-11T01:00:26.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71ccf5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d374d0 0x55b3d3d4fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71ccf5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d374d0 0x55b3d3d4fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44860/0 (socket says 172.21.15.78:44860) 2024-09-11T01:00:26.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71ccf5c640 1 -- 172.21.15.78:0/2421572441 learned_addr learned my addr 172.21.15.78:0/2421572441 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:26.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71ccf5c640 1 -- 172.21.15.78:0/2421572441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d43d80 msgr2=0x55b3d3d42800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:26.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71ccf5c640 1 --2- 172.21.15.78:0/2421572441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d43d80 0x55b3d3d42800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71ccf5c640 1 -- 172.21.15.78:0/2421572441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d37df0 msgr2=0x55b3d3d30320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:26.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71bffff640 1 --2- 172.21.15.78:0/2421572441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d37df0 0x55b3d3d30320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71ccf5c640 1 --2- 172.21.15.78:0/2421572441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d37df0 0x55b3d3d30320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.961+0000 7f71ccf5c640 1 -- 172.21.15.78:0/2421572441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3d3d40870 con 0x55b3d3d374d0 2024-09-11T01:00:26.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71ccf5c640 1 --2- 172.21.15.78:0/2421572441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d374d0 0x55b3d3d4fc60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x55b3d3d56310 tx=0x7f71c4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd8fb541f56ac3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:26.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71ae7fc640 1 -- 172.21.15.78:0/2421572441 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4052070 con 0x55b3d3d374d0 2024-09-11T01:00:26.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71ae7fc640 1 -- 172.21.15.78:0/2421572441 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4036040 con 0x55b3d3d374d0 2024-09-11T01:00:26.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71ae7fc640 1 -- 172.21.15.78:0/2421572441 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4004290 con 0x55b3d3d374d0 2024-09-11T01:00:26.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71d1595740 1 -- 172.21.15.78:0/2421572441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d374d0 msgr2=0x55b3d3d4fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:26.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71d1595740 1 --2- 172.21.15.78:0/2421572441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d374d0 0x55b3d3d4fc60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x55b3d3d56310 tx=0x7f71c4009360 comp rx=0 tx=0).stop 2024-09-11T01:00:26.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71d1595740 1 -- 172.21.15.78:0/2421572441 shutdown_connections 2024-09-11T01:00:26.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71d1595740 1 --2- 172.21.15.78:0/2421572441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d43d80 0x55b3d3d42800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71d1595740 1 --2- 172.21.15.78:0/2421572441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d37df0 0x55b3d3d30320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71d1595740 1 --2- 172.21.15.78:0/2421572441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d374d0 0x55b3d3d4fc60 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71d1595740 1 -- 172.21.15.78:0/2421572441 >> 172.21.15.78:0/2421572441 conn(0x55b3d3d35300 msgr2=0x55b3d3d420f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:26.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71d1595740 1 -- 172.21.15.78:0/2421572441 shutdown_connections 2024-09-11T01:00:26.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.962+0000 7f71d1595740 1 -- 172.21.15.78:0/2421572441 wait complete. 2024-09-11T01:00:26.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.963+0000 7f71d1595740 1 -- start start 2024-09-11T01:00:26.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.963+0000 7f71d1595740 1 Processor -- start 2024-09-11T01:00:26.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.963+0000 7f71d1595740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d374d0 0x55b3d3d48040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.963+0000 7f71d1595740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d37df0 0x55b3d3d48580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.963+0000 7f71d1595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d43d80 0x55b3d3dbf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.963+0000 7f71d1595740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81b1c20 con 0x55b3d3d37df0 2024-09-11T01:00:26.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.963+0000 7f71d1595740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81b1550 con 0x55b3d3d374d0 2024-09-11T01:00:26.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.963+0000 7f71d1595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81ae790 con 0x55b3d3d43d80 2024-09-11T01:00:26.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.963+0000 7f71bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d37df0 0x55b3d3d48580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.963+0000 7f71d1595740 10 client.0 _ll_register_callbacks cb 0x7f71c0020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:26.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.963+0000 7f71bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d37df0 0x55b3d3d48580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44864/0 (socket says 172.21.15.78:44864) 2024-09-11T01:00:26.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.963+0000 7f71bffff640 1 -- 172.21.15.78:0/405358045 learned_addr learned my addr 172.21.15.78:0/405358045 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:26.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71ccf5c640 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d374d0 0x55b3d3d48040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71cd75d640 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d43d80 0x55b3d3dbf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71bffff640 1 -- 172.21.15.78:0/405358045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d43d80 msgr2=0x55b3d3dbf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:26.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71bffff640 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d43d80 0x55b3d3dbf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71bffff640 1 -- 172.21.15.78:0/405358045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d374d0 msgr2=0x55b3d3d48040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:26.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71bffff640 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d374d0 0x55b3d3d48040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71bffff640 1 -- 172.21.15.78:0/405358045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3d3dbfab0 con 0x55b3d3d37df0 2024-09-11T01:00:26.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71bffff640 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d37df0 0x55b3d3d48580 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f71b80e2e00 tx=0x7f71b8010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d313b8301af8c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:26.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:00:26.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71a7fff640 1 -- 172.21.15.78:0/405358045 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b801e040 con 0x55b3d3d37df0 2024-09-11T01:00:26.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71d1595740 1 -- 172.21.15.78:0/405358045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c4036040 con 0x55b3d3d37df0 2024-09-11T01:00:26.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71a7fff640 1 -- 172.21.15.78:0/405358045 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f71b80a7040 con 0x55b3d3d37df0 2024-09-11T01:00:26.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71a7fff640 1 -- 172.21.15.78:0/405358045 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f71b80092b0 con 0x55b3d3d37df0 2024-09-11T01:00:26.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71d1595740 10 client.15780 fetch_fsmap learned FSMap version 30 2024-09-11T01:00:26.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71d1595740 1 -- 172.21.15.78:0/405358045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3d3dc0d30 con 0x55b3d3d37df0 2024-09-11T01:00:26.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.964+0000 7f71a7fff640 1 -- 172.21.15.78:0/405358045 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4036040 con 0x55b3d3d37df0 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71a7fff640 1 -- 172.21.15.78:0/405358045 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f71b80a7d40 con 0x55b3d3d37df0 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71d1595740 10 client.15780 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71d1595740 10 client.15780 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71d1595740 20 client.15780 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71d1595740 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b3d3dc14a0 0x55b3d3dc38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71d1595740 4 client.15780 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71d1595740 1 -- 172.21.15.78:0/405358045 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3d3cab8a0 con 0x55b3d3dc14a0 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71ccf5c640 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b3d3dc14a0 0x55b3d3dc38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71a7fff640 10 client.15780 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71ccf5c640 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b3d3dc14a0 0x55b3d3dc38c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3b39e1abbd6fedc server_cookie=43f4fae88136cd03 in_seq=0 out_seq=0 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71a7fff640 1 -- 172.21.15.78:0/405358045 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3d3cab8a0 con 0x55b3d3dc14a0 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71a7fff640 10 client.15780 handle_command_reply: tid=0 2024-09-11T01:00:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.965+0000 7f71d1595740 1 client.15780 shutdown 2024-09-11T01:00:26.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71d1595740 1 -- 172.21.15.78:0/405358045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d37df0 msgr2=0x55b3d3d48580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:26.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71d1595740 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d37df0 0x55b3d3d48580 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f71b80e2e00 tx=0x7f71b8010ef0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71d1595740 1 -- 172.21.15.78:0/405358045 shutdown_connections 2024-09-11T01:00:26.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71d1595740 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b3d3d43d80 0x55b3d3dbf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71d1595740 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b3d3dc14a0 0x55b3d3dc38c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71d1595740 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d3d37df0 0x55b3d3d48580 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71d1595740 1 --2- 172.21.15.78:0/405358045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b3d3d374d0 0x55b3d3d48040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71d1595740 1 -- 172.21.15.78:0/405358045 >> 172.21.15.78:0/405358045 conn(0x55b3d3d35300 msgr2=0x55b3d3d42910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:26.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71a7fff640 0 client.15780 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:00:26.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71d1595740 1 -- 172.21.15.78:0/405358045 shutdown_connections 2024-09-11T01:00:26.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71d1595740 1 -- 172.21.15.78:0/405358045 wait complete. 2024-09-11T01:00:26.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71d1595740 20 client.15780 trim_cache size 0 max 16384 2024-09-11T01:00:26.975 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:00:26.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71cf1c0640 1 -- 172.21.15.78:0/104432366 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71b00b5450 msgr2=0x7f71b00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:26.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71cf1c0640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71b00b5450 0x7f71b00b7910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71c001f000 tx=0x7f71c0020000 comp rx=0 tx=0).stop 2024-09-11T01:00:26.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71cf1c0640 1 -- 172.21.15.78:0/104432366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c81087d0 msgr2=0x7f71c80759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:26.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71cf1c0640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c81087d0 0x7f71c80759e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f71b800b4a0 tx=0x7f71b800b960 comp rx=0 tx=0).stop 2024-09-11T01:00:26.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71cf1c0640 1 -- 172.21.15.78:0/104432366 shutdown_connections 2024-09-11T01:00:26.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71cf1c0640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71b00b5450 0x7f71b00b7910 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71cf1c0640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c8109150 0x7f71c8075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71cf1c0640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c81087d0 0x7f71c80759e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71cf1c0640 1 --2- 172.21.15.78:0/104432366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c8102dc0 0x7f71c80754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:26.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.966+0000 7f71cf1c0640 1 -- 172.21.15.78:0/104432366 >> 172.21.15.78:0/104432366 conn(0x7f71c80fe830 msgr2=0x7f71c8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:26.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.967+0000 7f71cf1c0640 1 -- 172.21.15.78:0/104432366 shutdown_connections 2024-09-11T01:00:26.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:26.967+0000 7f71cf1c0640 1 -- 172.21.15.78:0/104432366 wait complete. 2024-09-11T01:00:27.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92508 14.80 MB/sec warmup 35 sec latency 4868.286 ms 2024-09-11T01:00:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:27 smithi110 ceph-mon[34712]: pgmap v370: 97 pgs: 97 active+clean; 62 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-09-11T01:00:27.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:27 smithi112 ceph-mon[35384]: pgmap v370: 97 pgs: 97 active+clean; 62 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-09-11T01:00:27.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:27 smithi078 ceph-mon[31315]: pgmap v370: 97 pgs: 97 active+clean; 62 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-09-11T01:00:27.701 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:00:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:00:28.353 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:28 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:00:28.530 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:28 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:00:28.530 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:28 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:00:28.530 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92508 14.39 MB/sec warmup 36 sec latency 1113.966 ms 2024-09-11T01:00:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.452+0000 7f1f97be4640 1 -- 172.21.15.78:0/2092594099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f9010b350 msgr2=0x7f1f90111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.452+0000 7f1f97be4640 1 --2- 172.21.15.78:0/2092594099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f9010b350 0x7f1f90111b50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009ca0 tx=0x7f1f8c02a8d0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.452+0000 7f1f97be4640 1 -- 172.21.15.78:0/2092594099 shutdown_connections 2024-09-11T01:00:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.452+0000 7f1f97be4640 1 --2- 172.21.15.78:0/2092594099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f9010b350 0x7f1f90111b50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.452+0000 7f1f97be4640 1 --2- 172.21.15.78:0/2092594099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f9010a9d0 0x7f1f9010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.452+0000 7f1f97be4640 1 --2- 172.21.15.78:0/2092594099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f90104fc0 0x7f1f901053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.452+0000 7f1f97be4640 1 -- 172.21.15.78:0/2092594099 >> 172.21.15.78:0/2092594099 conn(0x7f1f901009f0 msgr2=0x7f1f90102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.452+0000 7f1f97be4640 1 -- 172.21.15.78:0/2092594099 shutdown_connections 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.453+0000 7f1f97be4640 1 -- 172.21.15.78:0/2092594099 wait complete. 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.454+0000 7f1f97be4640 1 Processor -- start 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.454+0000 7f1f97be4640 1 -- start start 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f97be4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f90104fc0 0x7f1f9019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f97be4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f9010a9d0 0x7f1f9019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f97be4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f9010b350 0x7f1f901a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f97be4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90114fe0 con 0x7f1f9010a9d0 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f97be4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90114e60 con 0x7f1f90104fc0 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f97be4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90115160 con 0x7f1f9010b350 2024-09-11T01:00:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f9517f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f9010a9d0 0x7f1f9019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f9517f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f9010a9d0 0x7f1f9019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44874/0 (socket says 172.21.15.78:44874) 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f9517f640 1 -- 172.21.15.78:0/282403321 learned_addr learned my addr 172.21.15.78:0/282403321 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f95980640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f90104fc0 0x7f1f9019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f96181640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f9010b350 0x7f1f901a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f9517f640 1 -- 172.21.15.78:0/282403321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f9010b350 msgr2=0x7f1f901a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f9517f640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f9010b350 0x7f1f901a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f9517f640 1 -- 172.21.15.78:0/282403321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f90104fc0 msgr2=0x7f1f9019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f9517f640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f90104fc0 0x7f1f9019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f9517f640 1 -- 172.21.15.78:0/282403321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f901a1360 con 0x7f1f9010a9d0 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.455+0000 7f1f9517f640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f9010a9d0 0x7f1f9019b7f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000d950 tx=0x7f1f8000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.456+0000 7f1f86ffd640 1 -- 172.21.15.78:0/282403321 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80017070 con 0x7f1f9010a9d0 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.456+0000 7f1f86ffd640 1 -- 172.21.15.78:0/282403321 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f8000de80 con 0x7f1f9010a9d0 2024-09-11T01:00:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.456+0000 7f1f86ffd640 1 -- 172.21.15.78:0/282403321 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8000f620 con 0x7f1f9010a9d0 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.456+0000 7f1f97be4640 1 -- 172.21.15.78:0/282403321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f901a1650 con 0x7f1f9010a9d0 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.456+0000 7f1f97be4640 1 -- 172.21.15.78:0/282403321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f901a7d90 con 0x7f1f9010a9d0 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.459+0000 7f1f99fb9740 1 Processor -- start 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.459+0000 7f1f99fb9740 1 -- start start 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.459+0000 7f1f86ffd640 1 -- 172.21.15.78:0/282403321 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1f80010460 con 0x7f1f9010a9d0 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.459+0000 7f1f99fb9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 0x56052df808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.459+0000 7f1f99fb9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df80df0 0x56052df8c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.459+0000 7f1f99fb9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df8cd80 0x56052df8f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.459+0000 7f1f99fb9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56052def6ea0 con 0x56052df804d0 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.459+0000 7f1f99fb9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56052dee62d0 con 0x56052df8cd80 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.459+0000 7f1f99fb9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56052deea0e0 con 0x56052df80df0 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f96181640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 0x56052df808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f96181640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 0x56052df808b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44890/0 (socket says 172.21.15.78:44890) 2024-09-11T01:00:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f9517f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df8cd80 0x56052df8f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f96181640 1 -- 172.21.15.78:0/1171339250 learned_addr learned my addr 172.21.15.78:0/1171339250 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f95980640 1 --2- 172.21.15.78:0/1171339250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df80df0 0x56052df8c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f96181640 1 -- 172.21.15.78:0/1171339250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df80df0 msgr2=0x56052df8c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f96181640 1 --2- 172.21.15.78:0/1171339250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df80df0 0x56052df8c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f96181640 1 -- 172.21.15.78:0/1171339250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df8cd80 msgr2=0x56052df8f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f96181640 1 --2- 172.21.15.78:0/1171339250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df8cd80 0x56052df8f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f96181640 1 -- 172.21.15.78:0/1171339250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052dc44650 con 0x56052df804d0 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f9517f640 1 --2- 172.21.15.78:0/1171339250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df8cd80 0x56052df8f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f86ffd640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1f7c0b5500 0x7f1f7c0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f96181640 1 --2- 172.21.15.78:0/1171339250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 0x56052df808b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009830 tx=0x7f1f8c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a472382cfbc5f9e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f95980640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1f7c0b5500 0x7f1f7c0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f86ffd640 1 -- 172.21.15.78:0/282403321 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1f800e5b60 con 0x7f1f9010a9d0 2024-09-11T01:00:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f7bfff640 1 -- 172.21.15.78:0/1171339250 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c031660 con 0x56052df804d0 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f7bfff640 1 -- 172.21.15.78:0/1171339250 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f8c036040 con 0x56052df804d0 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.460+0000 7f1f7bfff640 1 -- 172.21.15.78:0/1171339250 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c004240 con 0x56052df804d0 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.461+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1171339250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 msgr2=0x56052df808b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.461+0000 7f1f95980640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1f7c0b5500 0x7f1f7c0b79c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x56052df7f1e0 tx=0x7f1f88009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.461+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/1171339250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 0x56052df808b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009830 tx=0x7f1f8c009860 comp rx=0 tx=0).stop 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.461+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1171339250 shutdown_connections 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.461+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/1171339250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df8cd80 0x56052df8f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.461+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/1171339250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df80df0 0x56052df8c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.461+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/1171339250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 0x56052df808b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.461+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1171339250 >> 172.21.15.78:0/1171339250 conn(0x56052df7e300 msgr2=0x56052df7f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.461+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1171339250 shutdown_connections 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.461+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1171339250 wait complete. 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f99fb9740 1 -- start start 2024-09-11T01:00:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f99fb9740 1 Processor -- start 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f99fb9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 0x56052df8c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f99fb9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 0x56052df89a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f99fb9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df8cd80 0x56052df89f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f99fb9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56052dd47700 con 0x56052df804d0 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f99fb9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56052de81160 con 0x56052df80df0 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f99fb9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56052df8f9a0 con 0x56052df8cd80 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f99fb9740 10 client.0 _ll_register_callbacks cb 0x56052de9a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f95980640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 0x56052df8c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f95980640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 0x56052df8c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44896/0 (socket says 172.21.15.78:44896) 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f9517f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 0x56052df89a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f95980640 1 -- 172.21.15.78:0/277288693 learned_addr learned my addr 172.21.15.78:0/277288693 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.462+0000 7f1f96181640 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df8cd80 0x56052df89f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f9517f640 1 -- 172.21.15.78:0/277288693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df8cd80 msgr2=0x56052df89f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f9517f640 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df8cd80 0x56052df89f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f9517f640 1 -- 172.21.15.78:0/277288693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 msgr2=0x56052df8c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f9517f640 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 0x56052df8c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f9517f640 1 -- 172.21.15.78:0/277288693 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052deaec90 con 0x56052df80df0 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f95980640 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 0x56052df8c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f9517f640 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 0x56052df89a20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f800e3320 tx=0x7f1f80010c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ef9da3d4f090b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f797fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f797fa640 1 -- 172.21.15.78:0/277288693 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80020070 con 0x56052df80df0 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f797fa640 1 -- 172.21.15.78:0/277288693 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1f80010980 con 0x56052df80df0 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f99fb9740 1 -- 172.21.15.78:0/277288693 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f8c036040 con 0x56052df80df0 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f797fa640 1 -- 172.21.15.78:0/277288693 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1f800a7ef0 con 0x56052df80df0 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f99fb9740 10 client.25205 fetch_fsmap learned FSMap version 31 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f99fb9740 1 -- 172.21.15.78:0/277288693 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56052df9b290 con 0x56052df80df0 2024-09-11T01:00:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.463+0000 7f1f797fa640 1 -- 172.21.15.78:0/277288693 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f8c036040 con 0x56052df80df0 2024-09-11T01:00:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.464+0000 7f1f797fa640 1 -- 172.21.15.78:0/277288693 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1f800a79a0 con 0x56052df80df0 2024-09-11T01:00:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.464+0000 7f1f99fb9740 10 client.25205 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:00:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.464+0000 7f1f99fb9740 10 client.25205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:00:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.464+0000 7f1f99fb9740 20 client.25205 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.464+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56052df9c7a0 0x56052df9ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.464+0000 7f1f99fb9740 4 client.25205 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:00:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.464+0000 7f1f99fb9740 1 -- 172.21.15.78:0/277288693 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56052def8df0 con 0x56052df9c7a0 2024-09-11T01:00:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.464+0000 7f1f95980640 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56052df9c7a0 0x56052df9ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.464+0000 7f1f797fa640 10 client.25205 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.464+0000 7f1f95980640 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56052df9c7a0 0x56052df9ec00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9daf0d7f288492a5 server_cookie=3ef18c844cbb2faa in_seq=0 out_seq=0 2024-09-11T01:00:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.465+0000 7f1f797fa640 1 -- 172.21.15.78:0/277288693 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56052def8df0 con 0x56052df9c7a0 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.465+0000 7f1f797fa640 10 client.25205 handle_command_reply: tid=0 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.465+0000 7f1f99fb9740 1 client.25205 shutdown 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.465+0000 7f1f99fb9740 1 -- 172.21.15.78:0/277288693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 msgr2=0x56052df89a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.465+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 0x56052df89a20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f800e3320 tx=0x7f1f80010c40 comp rx=0 tx=0).stop 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.466+0000 7f1f99fb9740 1 -- 172.21.15.78:0/277288693 shutdown_connections 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.466+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df8cd80 0x56052df89f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.466+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 0x56052df89a20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.466+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56052df9c7a0 0x56052df9ec00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.466+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/277288693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df804d0 0x56052df8c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.466+0000 7f1f797fa640 0 client.25205 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.466+0000 7f1f99fb9740 1 -- 172.21.15.78:0/277288693 >> 172.21.15.78:0/277288693 conn(0x56052df7e300 msgr2=0x56052df83e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.466+0000 7f1f99fb9740 1 -- 172.21.15.78:0/277288693 shutdown_connections 2024-09-11T01:00:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.466+0000 7f1f99fb9740 1 -- 172.21.15.78:0/277288693 wait complete. 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.466+0000 7f1f99fb9740 20 client.25205 trim_cache size 0 max 16384 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f99fb9740 1 Processor -- start 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f99fb9740 1 -- start start 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f99fb9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df804d0 0x56052df98c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f99fb9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df80df0 0x56052df7f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f99fb9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df8cd80 0x56052df7f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f99fb9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f901b0800 con 0x56052df80df0 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f99fb9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f901b2400 con 0x56052df804d0 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f99fb9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f901b0ed0 con 0x56052df8cd80 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f9517f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df80df0 0x56052df7f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f9517f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df80df0 0x56052df7f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44912/0 (socket says 172.21.15.78:44912) 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f95980640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df804d0 0x56052df98c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f9517f640 1 -- 172.21.15.78:0/3086531106 learned_addr learned my addr 172.21.15.78:0/3086531106 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f96181640 1 --2- 172.21.15.78:0/3086531106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df8cd80 0x56052df7f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f9517f640 1 -- 172.21.15.78:0/3086531106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df8cd80 msgr2=0x56052df7f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f9517f640 1 --2- 172.21.15.78:0/3086531106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df8cd80 0x56052df7f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f9517f640 1 -- 172.21.15.78:0/3086531106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df804d0 msgr2=0x56052df98c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f9517f640 1 --2- 172.21.15.78:0/3086531106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df804d0 0x56052df98c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f9517f640 1 -- 172.21.15.78:0/3086531106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052df7ed80 con 0x56052df80df0 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f96181640 1 --2- 172.21.15.78:0/3086531106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df8cd80 0x56052df7f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.479+0000 7f1f95980640 1 --2- 172.21.15.78:0/3086531106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df804d0 0x56052df98c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.480+0000 7f1f9517f640 1 --2- 172.21.15.78:0/3086531106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df80df0 0x56052df7f3c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f800abfe0 tx=0x7f1f80009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b0f719559b89d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.480+0000 7f1f7b7fe640 1 -- 172.21.15.78:0/3086531106 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80097070 con 0x56052df80df0 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.480+0000 7f1f7b7fe640 1 -- 172.21.15.78:0/3086531106 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f800abb60 con 0x56052df80df0 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.480+0000 7f1f7b7fe640 1 -- 172.21.15.78:0/3086531106 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80020070 con 0x56052df80df0 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.480+0000 7f1f99fb9740 1 -- 172.21.15.78:0/3086531106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df80df0 msgr2=0x56052df7f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.480+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/3086531106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df80df0 0x56052df7f3c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f800abfe0 tx=0x7f1f80009f50 comp rx=0 tx=0).stop 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.480+0000 7f1f99fb9740 1 -- 172.21.15.78:0/3086531106 shutdown_connections 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.480+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/3086531106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df8cd80 0x56052df7f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.480+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/3086531106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df80df0 0x56052df7f3c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.480+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/3086531106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df804d0 0x56052df98c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.480+0000 7f1f99fb9740 1 -- 172.21.15.78:0/3086531106 >> 172.21.15.78:0/3086531106 conn(0x56052df7e300 msgr2=0x56052df84c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.481+0000 7f1f99fb9740 1 -- 172.21.15.78:0/3086531106 shutdown_connections 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.481+0000 7f1f99fb9740 1 -- 172.21.15.78:0/3086531106 wait complete. 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.481+0000 7f1f99fb9740 1 -- start start 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.481+0000 7f1f99fb9740 1 Processor -- start 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f99fb9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df804d0 0x56052df94c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f99fb9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 0x56052df951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f99fb9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df8cd80 0x56052df91c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f99fb9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f901b1c70 con 0x56052df8cd80 2024-09-11T01:00:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f99fb9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f901b15a0 con 0x56052df80df0 2024-09-11T01:00:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f99fb9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f901ae7e0 con 0x56052df804d0 2024-09-11T01:00:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f99fb9740 10 client.0 _ll_register_callbacks cb 0x7f1f80009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f96181640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df8cd80 0x56052df91c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f9517f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 0x56052df951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f9517f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 0x56052df951d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:54168/0 (socket says 172.21.15.78:54168) 2024-09-11T01:00:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f9517f640 1 -- 172.21.15.78:0/1079032607 learned_addr learned my addr 172.21.15.78:0/1079032607 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f95980640 1 --2- 172.21.15.78:0/1079032607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df804d0 0x56052df94c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f9517f640 1 -- 172.21.15.78:0/1079032607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df804d0 msgr2=0x56052df94c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f9517f640 1 --2- 172.21.15.78:0/1079032607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df804d0 0x56052df94c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f9517f640 1 -- 172.21.15.78:0/1079032607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df8cd80 msgr2=0x56052df91c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f9517f640 1 --2- 172.21.15.78:0/1079032607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df8cd80 0x56052df91c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f9517f640 1 -- 172.21.15.78:0/1079032607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052df921a0 con 0x56052df80df0 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f9517f640 1 --2- 172.21.15.78:0/1079032607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 0x56052df951d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1f800e96b0 tx=0x7f1f800b0cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f808ff1e4d6acb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f797fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f797fa640 1 -- 172.21.15.78:0/1079032607 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80020070 con 0x56052df80df0 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f797fa640 1 -- 172.21.15.78:0/1079032607 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1f80010980 con 0x56052df80df0 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.482+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1079032607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f800abb60 con 0x56052df80df0 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f797fa640 1 -- 172.21.15.78:0/1079032607 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1f80097040 con 0x56052df80df0 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f99fb9740 10 client.25211 fetch_fsmap learned FSMap version 31 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1079032607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56052e009cb0 con 0x56052df80df0 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f797fa640 1 -- 172.21.15.78:0/1079032607 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f800abb60 con 0x56052df80df0 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f797fa640 1 -- 172.21.15.78:0/1079032607 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1f800aad20 con 0x56052df80df0 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f99fb9740 10 client.25211 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f99fb9740 10 client.25211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f99fb9740 20 client.25211 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/1079032607 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56052e00a210 0x56052e00c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f99fb9740 4 client.25211 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1079032607 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56052def48a0 con 0x56052e00a210 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f95980640 1 --2- 172.21.15.78:0/1079032607 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56052e00a210 0x56052e00c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f797fa640 10 client.25211 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.483+0000 7f1f95980640 1 --2- 172.21.15.78:0/1079032607 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56052e00a210 0x56052e00c630 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c1c7e4f150b17d server_cookie=5827cae158536ab5 in_seq=0 out_seq=0 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f797fa640 1 -- 172.21.15.78:0/1079032607 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56052def48a0 con 0x56052e00a210 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f797fa640 10 client.25211 handle_command_reply: tid=0 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f99fb9740 1 client.25211 shutdown 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1079032607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 msgr2=0x56052df951d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/1079032607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 0x56052df951d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1f800e96b0 tx=0x7f1f800b0cb0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1079032607 shutdown_connections 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/1079032607 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56052e00a210 0x56052e00c630 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/1079032607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56052df8cd80 0x56052df91c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/1079032607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052df80df0 0x56052df951d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f99fb9740 1 --2- 172.21.15.78:0/1079032607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56052df804d0 0x56052df94c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f797fa640 0 client.25211 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:28.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1079032607 >> 172.21.15.78:0/1079032607 conn(0x56052df7e300 msgr2=0x56052df7f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:28.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1079032607 shutdown_connections 2024-09-11T01:00:28.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f99fb9740 1 -- 172.21.15.78:0/1079032607 wait complete. 2024-09-11T01:00:28.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.485+0000 7f1f99fb9740 20 client.25211 trim_cache size 0 max 16384 2024-09-11T01:00:28.551 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:00:28.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.486+0000 7f1f97be4640 1 -- 172.21.15.78:0/282403321 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1f7c0b5500 msgr2=0x7f1f7c0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:28.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.486+0000 7f1f97be4640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1f7c0b5500 0x7f1f7c0b79c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x56052df7f1e0 tx=0x7f1f88009110 comp rx=0 tx=0).stop 2024-09-11T01:00:28.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.486+0000 7f1f97be4640 1 -- 172.21.15.78:0/282403321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f9010a9d0 msgr2=0x7f1f9019b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:28.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.486+0000 7f1f97be4640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f9010a9d0 0x7f1f9019b7f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000d950 tx=0x7f1f8000dd50 comp rx=0 tx=0).stop 2024-09-11T01:00:28.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.486+0000 7f1f97be4640 1 -- 172.21.15.78:0/282403321 shutdown_connections 2024-09-11T01:00:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.486+0000 7f1f97be4640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1f7c0b5500 0x7f1f7c0b79c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.486+0000 7f1f97be4640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f9010b350 0x7f1f901a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.486+0000 7f1f97be4640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f9010a9d0 0x7f1f9019b7f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.486+0000 7f1f97be4640 1 --2- 172.21.15.78:0/282403321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f90104fc0 0x7f1f9019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.486+0000 7f1f97be4640 1 -- 172.21.15.78:0/282403321 >> 172.21.15.78:0/282403321 conn(0x7f1f901009f0 msgr2=0x7f1f90100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.486+0000 7f1f97be4640 1 -- 172.21.15.78:0/282403321 shutdown_connections 2024-09-11T01:00:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:28.486+0000 7f1f97be4640 1 -- 172.21.15.78:0/282403321 wait complete. 2024-09-11T01:00:29.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92508 14.00 MB/sec warmup 37 sec latency 2114.051 ms 2024-09-11T01:00:29.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:29 smithi110 ceph-mon[34712]: pgmap v371: 97 pgs: 97 active+clean; 64 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 2.5 MiB/s wr, 149 op/s 2024-09-11T01:00:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:29 smithi112 ceph-mon[35384]: pgmap v371: 97 pgs: 97 active+clean; 64 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 2.5 MiB/s wr, 149 op/s 2024-09-11T01:00:29.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:29 smithi078 ceph-mon[31315]: pgmap v371: 97 pgs: 97 active+clean; 64 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 2.5 MiB/s wr, 149 op/s 2024-09-11T01:00:30.248 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:00:30.249 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:00:30.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92508 13.63 MB/sec warmup 38 sec latency 3114.132 ms 2024-09-11T01:00:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:30 smithi110 ceph-mon[34712]: pgmap v372: 97 pgs: 97 active+clean; 70 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.2 MiB/s wr, 162 op/s 2024-09-11T01:00:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:30 smithi112 ceph-mon[35384]: pgmap v372: 97 pgs: 97 active+clean; 70 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.2 MiB/s wr, 162 op/s 2024-09-11T01:00:30.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:30 smithi078 ceph-mon[31315]: pgmap v372: 97 pgs: 97 active+clean; 70 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.2 MiB/s wr, 162 op/s 2024-09-11T01:00:30.504 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:00:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.996+0000 7fb5219a0640 1 -- 172.21.15.78:0/934328252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c105410 msgr2=0x7fb51c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.996+0000 7fb5219a0640 1 --2- 172.21.15.78:0/934328252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c105410 0x7fb51c105890 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb510009900 tx=0x7fb51002a410 comp rx=0 tx=0).stop 2024-09-11T01:00:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.996+0000 7fb5219a0640 1 -- 172.21.15.78:0/934328252 shutdown_connections 2024-09-11T01:00:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.996+0000 7fb5219a0640 1 --2- 172.21.15.78:0/934328252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb51c105dd0 0x7fb51c10f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.996+0000 7fb5219a0640 1 --2- 172.21.15.78:0/934328252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c105410 0x7fb51c105890 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.996+0000 7fb5219a0640 1 --2- 172.21.15.78:0/934328252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c101230 0x7fb51c101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.996+0000 7fb5219a0640 1 -- 172.21.15.78:0/934328252 >> 172.21.15.78:0/934328252 conn(0x7fb51c0fd2f0 msgr2=0x7fb51c0ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.996+0000 7fb5219a0640 1 -- 172.21.15.78:0/934328252 shutdown_connections 2024-09-11T01:00:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.996+0000 7fb5219a0640 1 -- 172.21.15.78:0/934328252 wait complete. 2024-09-11T01:00:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.996+0000 7fb5219a0640 1 Processor -- start 2024-09-11T01:00:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.996+0000 7fb5219a0640 1 -- start start 2024-09-11T01:00:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb5219a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c101230 0x7fb51c198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb5219a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c105410 0x7fb51c199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb5219a0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb51c105dd0 0x7fb51c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb5219a0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c112bc0 con 0x7fb51c105410 2024-09-11T01:00:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb5219a0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c112a40 con 0x7fb51c105dd0 2024-09-11T01:00:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb5219a0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c112d40 con 0x7fb51c101230 2024-09-11T01:00:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb51bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c105410 0x7fb51c199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb51bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c105410 0x7fb51c199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44944/0 (socket says 172.21.15.78:44944) 2024-09-11T01:00:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb51bfff640 1 -- 172.21.15.78:0/665521704 learned_addr learned my addr 172.21.15.78:0/665521704 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb52099e640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c101230 0x7fb51c198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb52119f640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb51c105dd0 0x7fb51c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb51bfff640 1 -- 172.21.15.78:0/665521704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c101230 msgr2=0x7fb51c198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb51bfff640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c101230 0x7fb51c198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb51bfff640 1 -- 172.21.15.78:0/665521704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb51c105dd0 msgr2=0x7fb51c19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb51bfff640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb51c105dd0 0x7fb51c19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb51bfff640 1 -- 172.21.15.78:0/665521704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51c19f0a0 con 0x7fb51c105410 2024-09-11T01:00:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb52119f640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb51c105dd0 0x7fb51c19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:00:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.997+0000 7fb51bfff640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c105410 0x7fb51c199530 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb510005940 tx=0x7fb5100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.998+0000 7fb519ffb640 1 -- 172.21.15.78:0/665521704 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51003f030 con 0x7fb51c105410 2024-09-11T01:00:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.998+0000 7fb5219a0640 1 -- 172.21.15.78:0/665521704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51c19f330 con 0x7fb51c105410 2024-09-11T01:00:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.998+0000 7fb519ffb640 1 -- 172.21.15.78:0/665521704 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb510003cb0 con 0x7fb51c105410 2024-09-11T01:00:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.998+0000 7fb519ffb640 1 -- 172.21.15.78:0/665521704 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510035070 con 0x7fb51c105410 2024-09-11T01:00:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:30.998+0000 7fb5219a0640 1 -- 172.21.15.78:0/665521704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51c1a5a50 con 0x7fb51c105410 2024-09-11T01:00:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.000+0000 7fb524fd7740 1 Processor -- start 2024-09-11T01:00:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.000+0000 7fb524fd7740 1 -- start start 2024-09-11T01:00:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.000+0000 7fb519ffb640 1 -- 172.21.15.78:0/665521704 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb51003bbf0 con 0x7fb51c105410 2024-09-11T01:00:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.000+0000 7fb524fd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d020f0 0x55b575d024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.000+0000 7fb524fd7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 0x55b575d0e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.000+0000 7fb524fd7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 0x55b575d10d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.000+0000 7fb524fd7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b575c78e60 con 0x55b575d02a10 2024-09-11T01:00:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.000+0000 7fb524fd7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b575c68290 con 0x55b575d0e9a0 2024-09-11T01:00:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.000+0000 7fb524fd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b575c6c0a0 con 0x55b575d020f0 2024-09-11T01:00:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb52099e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 0x55b575d0e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb52099e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 0x55b575d0e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44960/0 (socket says 172.21.15.78:44960) 2024-09-11T01:00:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb52099e640 1 -- 172.21.15.78:0/3213676160 learned_addr learned my addr 172.21.15.78:0/3213676160 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb52119f640 1 --2- 172.21.15.78:0/3213676160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d020f0 0x55b575d024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb51bfff640 1 --2- 172.21.15.78:0/3213676160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 0x55b575d10d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb52119f640 1 -- 172.21.15.78:0/3213676160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 msgr2=0x55b575d10d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb52119f640 1 --2- 172.21.15.78:0/3213676160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 0x55b575d10d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb52119f640 1 -- 172.21.15.78:0/3213676160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 msgr2=0x55b575d0e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb52119f640 1 --2- 172.21.15.78:0/3213676160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 0x55b575d0e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb52119f640 1 -- 172.21.15.78:0/3213676160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5759c6930 con 0x55b575d020f0 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb52119f640 1 --2- 172.21.15.78:0/3213676160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d020f0 0x55b575d024d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb514009e90 tx=0x7fb514010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f41f20b27ee386 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb519ffb640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5040b5500 0x7fb5040b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb502ffd640 1 -- 172.21.15.78:0/3213676160 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514016070 con 0x55b575d020f0 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb52099e640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5040b5500 0x7fb5040b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb502ffd640 1 -- 172.21.15.78:0/3213676160 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb51401bb10 con 0x55b575d020f0 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb519ffb640 1 -- 172.21.15.78:0/665521704 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb510106b30 con 0x7fb51c105410 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.001+0000 7fb502ffd640 1 -- 172.21.15.78:0/3213676160 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514012b30 con 0x55b575d020f0 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.002+0000 7fb524fd7740 1 -- 172.21.15.78:0/3213676160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d020f0 msgr2=0x55b575d024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.002+0000 7fb52099e640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5040b5500 0x7fb5040b79c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55b575d05270 tx=0x7fb50c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.002+0000 7fb524fd7740 1 --2- 172.21.15.78:0/3213676160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d020f0 0x55b575d024d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb514009e90 tx=0x7fb514010740 comp rx=0 tx=0).stop 2024-09-11T01:00:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.002+0000 7fb524fd7740 1 -- 172.21.15.78:0/3213676160 shutdown_connections 2024-09-11T01:00:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.002+0000 7fb524fd7740 1 --2- 172.21.15.78:0/3213676160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 0x55b575d10d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.002+0000 7fb524fd7740 1 --2- 172.21.15.78:0/3213676160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 0x55b575d0e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.002+0000 7fb524fd7740 1 --2- 172.21.15.78:0/3213676160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d020f0 0x55b575d024d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.002+0000 7fb524fd7740 1 -- 172.21.15.78:0/3213676160 >> 172.21.15.78:0/3213676160 conn(0x55b575cfff20 msgr2=0x55b575d010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.002+0000 7fb524fd7740 1 -- 172.21.15.78:0/3213676160 shutdown_connections 2024-09-11T01:00:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.002+0000 7fb524fd7740 1 -- 172.21.15.78:0/3213676160 wait complete. 2024-09-11T01:00:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb524fd7740 1 -- start start 2024-09-11T01:00:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb524fd7740 1 Processor -- start 2024-09-11T01:00:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb524fd7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 0x55b575d0b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb524fd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d02a10 0x55b575d0be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb524fd7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 0x55b575d0c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb524fd7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b575ac9890 con 0x55b575d020f0 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb524fd7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b575c02d40 con 0x55b575d0e9a0 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb524fd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b575d11650 con 0x55b575d02a10 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb524fd7740 10 client.0 _ll_register_callbacks cb 0x55b575c66c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb52099e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 0x55b575d0b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb52099e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 0x55b575d0b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44974/0 (socket says 172.21.15.78:44974) 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb52099e640 1 -- 172.21.15.78:0/2807494433 learned_addr learned my addr 172.21.15.78:0/2807494433 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb52119f640 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 0x55b575d0c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.003+0000 7fb51bfff640 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d02a10 0x55b575d0be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb52099e640 1 -- 172.21.15.78:0/2807494433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d02a10 msgr2=0x55b575d0be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb52099e640 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d02a10 0x55b575d0be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb52099e640 1 -- 172.21.15.78:0/2807494433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 msgr2=0x55b575d0c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb52099e640 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 0x55b575d0c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb52099e640 1 -- 172.21.15.78:0/2807494433 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b575d1aa10 con 0x55b575d020f0 2024-09-11T01:00:31.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb52099e640 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 0x55b575d0b8e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb50c01f5f0 tx=0x7fb50c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f906b9086f6453 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:31.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb4fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:00:31.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb4fbfff640 1 -- 172.21.15.78:0/2807494433 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c02c060 con 0x55b575d020f0 2024-09-11T01:00:31.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb4fbfff640 1 -- 172.21.15.78:0/2807494433 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb50c017030 con 0x55b575d020f0 2024-09-11T01:00:31.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb524fd7740 1 -- 172.21.15.78:0/2807494433 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb51401bb10 con 0x55b575d020f0 2024-09-11T01:00:31.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb4fbfff640 1 -- 172.21.15.78:0/2807494433 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb50c010700 con 0x55b575d020f0 2024-09-11T01:00:31.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb524fd7740 10 client.15810 fetch_fsmap learned FSMap version 31 2024-09-11T01:00:31.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb524fd7740 1 -- 172.21.15.78:0/2807494433 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b575d1d420 con 0x55b575d020f0 2024-09-11T01:00:31.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb4fbfff640 1 -- 172.21.15.78:0/2807494433 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb51401bb10 con 0x55b575d020f0 2024-09-11T01:00:31.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.004+0000 7fb4fbfff640 1 -- 172.21.15.78:0/2807494433 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fb50c02f020 con 0x55b575d020f0 2024-09-11T01:00:31.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.005+0000 7fb524fd7740 10 client.15810 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:00:31.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.005+0000 7fb524fd7740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:00:31.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.005+0000 7fb524fd7740 20 client.15810 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:31.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.005+0000 7fb524fd7740 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b575d1e9f0 0x55b575d20e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.005+0000 7fb524fd7740 4 client.15810 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:00:31.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.005+0000 7fb524fd7740 1 -- 172.21.15.78:0/2807494433 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b575c7adb0 con 0x55b575d1e9f0 2024-09-11T01:00:31.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.005+0000 7fb51bfff640 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b575d1e9f0 0x55b575d20e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.005+0000 7fb51bfff640 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b575d1e9f0 0x55b575d20e70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12ad04f46f8ce0fc server_cookie=9148aced2f81a413 in_seq=0 out_seq=0 2024-09-11T01:00:31.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.005+0000 7fb4fbfff640 10 client.15810 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:31.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb4fbfff640 1 -- 172.21.15.78:0/2807494433 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b575c7adb0 con 0x55b575d1e9f0 2024-09-11T01:00:31.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb4fbfff640 10 client.15810 handle_command_reply: tid=0 2024-09-11T01:00:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb524fd7740 1 client.15810 shutdown 2024-09-11T01:00:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb524fd7740 1 -- 172.21.15.78:0/2807494433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 msgr2=0x55b575d0b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb524fd7740 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 0x55b575d0b8e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb50c01f5f0 tx=0x7fb50c004730 comp rx=0 tx=0).stop 2024-09-11T01:00:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb524fd7740 1 -- 172.21.15.78:0/2807494433 shutdown_connections 2024-09-11T01:00:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb524fd7740 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 0x55b575d0c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb524fd7740 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d02a10 0x55b575d0be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb524fd7740 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b575d1e9f0 0x55b575d20e70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb524fd7740 1 --2- 172.21.15.78:0/2807494433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 0x55b575d0b8e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb4fbfff640 0 client.15810 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb524fd7740 1 -- 172.21.15.78:0/2807494433 >> 172.21.15.78:0/2807494433 conn(0x55b575cfff20 msgr2=0x55b575d05ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb524fd7740 1 -- 172.21.15.78:0/2807494433 shutdown_connections 2024-09-11T01:00:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb524fd7740 1 -- 172.21.15.78:0/2807494433 wait complete. 2024-09-11T01:00:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.006+0000 7fb524fd7740 20 client.15810 trim_cache size 0 max 16384 2024-09-11T01:00:31.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.023+0000 7fb524fd7740 1 Processor -- start 2024-09-11T01:00:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.023+0000 7fb524fd7740 1 -- start start 2024-09-11T01:00:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.023+0000 7fb524fd7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 0x55b575d0bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb524fd7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d02a10 0x55b575d0adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb524fd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d0e9a0 0x55b575d16d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb524fd7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c1ae270 con 0x55b575d020f0 2024-09-11T01:00:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb524fd7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c1afe70 con 0x55b575d02a10 2024-09-11T01:00:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb524fd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c1ae940 con 0x55b575d0e9a0 2024-09-11T01:00:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb52099e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 0x55b575d0bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb52099e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 0x55b575d0bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44986/0 (socket says 172.21.15.78:44986) 2024-09-11T01:00:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb52099e640 1 -- 172.21.15.78:0/683951799 learned_addr learned my addr 172.21.15.78:0/683951799 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb51bfff640 1 --2- 172.21.15.78:0/683951799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d02a10 0x55b575d0adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb52119f640 1 --2- 172.21.15.78:0/683951799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d0e9a0 0x55b575d16d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb52099e640 1 -- 172.21.15.78:0/683951799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d0e9a0 msgr2=0x55b575d16d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb52099e640 1 --2- 172.21.15.78:0/683951799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d0e9a0 0x55b575d16d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb52099e640 1 -- 172.21.15.78:0/683951799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d02a10 msgr2=0x55b575d0adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb52099e640 1 --2- 172.21.15.78:0/683951799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d02a10 0x55b575d0adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb52099e640 1 -- 172.21.15.78:0/683951799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b575d009a0 con 0x55b575d020f0 2024-09-11T01:00:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb52099e640 1 --2- 172.21.15.78:0/683951799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 0x55b575d0bf60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb50c031ca0 tx=0x7fb50c027400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36906dd4adf479d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb5027fc640 1 -- 172.21.15.78:0/683951799 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c0317c0 con 0x55b575d020f0 2024-09-11T01:00:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb5027fc640 1 -- 172.21.15.78:0/683951799 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb50c010d50 con 0x55b575d020f0 2024-09-11T01:00:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.024+0000 7fb5027fc640 1 -- 172.21.15.78:0/683951799 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c027d10 con 0x55b575d020f0 2024-09-11T01:00:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.025+0000 7fb524fd7740 1 -- 172.21.15.78:0/683951799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 msgr2=0x55b575d0bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.025+0000 7fb524fd7740 1 --2- 172.21.15.78:0/683951799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 0x55b575d0bf60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb50c031ca0 tx=0x7fb50c027400 comp rx=0 tx=0).stop 2024-09-11T01:00:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.025+0000 7fb524fd7740 1 -- 172.21.15.78:0/683951799 shutdown_connections 2024-09-11T01:00:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.025+0000 7fb524fd7740 1 --2- 172.21.15.78:0/683951799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d0e9a0 0x55b575d16d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.025+0000 7fb524fd7740 1 --2- 172.21.15.78:0/683951799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d02a10 0x55b575d0adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.025+0000 7fb524fd7740 1 --2- 172.21.15.78:0/683951799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d020f0 0x55b575d0bf60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.025+0000 7fb524fd7740 1 -- 172.21.15.78:0/683951799 >> 172.21.15.78:0/683951799 conn(0x55b575cfff20 msgr2=0x55b575d0b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.025+0000 7fb524fd7740 1 -- 172.21.15.78:0/683951799 shutdown_connections 2024-09-11T01:00:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.025+0000 7fb524fd7740 1 -- 172.21.15.78:0/683951799 wait complete. 2024-09-11T01:00:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.025+0000 7fb524fd7740 1 -- start start 2024-09-11T01:00:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.025+0000 7fb524fd7740 1 Processor -- start 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb524fd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d020f0 0x55b575d87ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb524fd7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 0x55b575d88430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb524fd7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 0x55b575d83900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb524fd7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c1af6e0 con 0x55b575d02a10 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb524fd7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c1af010 con 0x55b575d0e9a0 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb524fd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c1ac250 con 0x55b575d020f0 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb524fd7740 10 client.0 _ll_register_callbacks cb 0x7fb50c005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb51bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 0x55b575d88430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb51bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 0x55b575d88430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44990/0 (socket says 172.21.15.78:44990) 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb51bfff640 1 -- 172.21.15.78:0/2442996864 learned_addr learned my addr 172.21.15.78:0/2442996864 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb52099e640 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d020f0 0x55b575d87ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb52119f640 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 0x55b575d83900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb51bfff640 1 -- 172.21.15.78:0/2442996864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d020f0 msgr2=0x55b575d87ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb51bfff640 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d020f0 0x55b575d87ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb51bfff640 1 -- 172.21.15.78:0/2442996864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 msgr2=0x55b575d83900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb51bfff640 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 0x55b575d83900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb51bfff640 1 -- 172.21.15.78:0/2442996864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b575d01600 con 0x55b575d02a10 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb51bfff640 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 0x55b575d88430 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb5101043a0 tx=0x7fb51002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafb684af30df7ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.026+0000 7fb4fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb4fbfff640 1 -- 172.21.15.78:0/2442996864 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510049040 con 0x55b575d02a10 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb4fbfff640 1 -- 172.21.15.78:0/2442996864 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb5100c9290 con 0x55b575d02a10 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb524fd7740 1 -- 172.21.15.78:0/2442996864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb50c010d50 con 0x55b575d02a10 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb4fbfff640 1 -- 172.21.15.78:0/2442996864 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb5100cc790 con 0x55b575d02a10 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb524fd7740 10 client.15816 fetch_fsmap learned FSMap version 31 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb524fd7740 1 -- 172.21.15.78:0/2442996864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b575d90830 con 0x55b575d02a10 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb4fbfff640 1 -- 172.21.15.78:0/2442996864 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb50c010d50 con 0x55b575d02a10 2024-09-11T01:00:31.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb4fbfff640 1 -- 172.21.15.78:0/2442996864 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fb5100b8170 con 0x55b575d02a10 2024-09-11T01:00:31.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb524fd7740 10 client.15816 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:00:31.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb524fd7740 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:00:31.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb524fd7740 20 client.15816 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:31.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb524fd7740 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b575d91020 0x55b575d93440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:31.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb524fd7740 4 client.15816 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:00:31.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb524fd7740 1 -- 172.21.15.78:0/2442996864 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b575cfb640 con 0x55b575d91020 2024-09-11T01:00:31.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.027+0000 7fb52099e640 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b575d91020 0x55b575d93440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:31.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.028+0000 7fb52099e640 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b575d91020 0x55b575d93440 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5548f380fbecb8f server_cookie=f87ea9a1256e7e37 in_seq=0 out_seq=0 2024-09-11T01:00:31.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.028+0000 7fb4fbfff640 10 client.15816 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:31.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb4fbfff640 1 -- 172.21.15.78:0/2442996864 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b575cfb640 con 0x55b575d91020 2024-09-11T01:00:31.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb4fbfff640 10 client.15816 handle_command_reply: tid=0 2024-09-11T01:00:31.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb524fd7740 1 client.15816 shutdown 2024-09-11T01:00:31.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb524fd7740 1 -- 172.21.15.78:0/2442996864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 msgr2=0x55b575d88430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:31.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb524fd7740 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 0x55b575d88430 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb5101043a0 tx=0x7fb51002a410 comp rx=0 tx=0).stop 2024-09-11T01:00:31.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb524fd7740 1 -- 172.21.15.78:0/2442996864 shutdown_connections 2024-09-11T01:00:31.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb524fd7740 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b575d0e9a0 0x55b575d83900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb524fd7740 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b575d91020 0x55b575d93440 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb524fd7740 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b575d02a10 0x55b575d88430 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb524fd7740 1 --2- 172.21.15.78:0/2442996864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b575d020f0 0x55b575d87ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb524fd7740 1 -- 172.21.15.78:0/2442996864 >> 172.21.15.78:0/2442996864 conn(0x55b575cfff20 msgr2=0x55b575cfef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:31.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb4fbfff640 0 client.15816 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:31.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb524fd7740 1 -- 172.21.15.78:0/2442996864 shutdown_connections 2024-09-11T01:00:31.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.029+0000 7fb524fd7740 1 -- 172.21.15.78:0/2442996864 wait complete. 2024-09-11T01:00:31.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb524fd7740 20 client.15816 trim_cache size 0 max 16384 2024-09-11T01:00:31.036 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:00:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:00:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "6ba15803-6da9-4d97-adb8-4d6230a3c7a3", 2024-09-11T01:00:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:00:31.037 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:00:31.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb5219a0640 1 -- 172.21.15.78:0/665521704 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5040b5500 msgr2=0x7fb5040b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:31.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb5219a0640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5040b5500 0x7fb5040b79c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55b575d05270 tx=0x7fb50c00b410 comp rx=0 tx=0).stop 2024-09-11T01:00:31.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb5219a0640 1 -- 172.21.15.78:0/665521704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c105410 msgr2=0x7fb51c199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:31.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb5219a0640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c105410 0x7fb51c199530 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb510005940 tx=0x7fb5100042f0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb5219a0640 1 -- 172.21.15.78:0/665521704 shutdown_connections 2024-09-11T01:00:31.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb5219a0640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5040b5500 0x7fb5040b79c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb5219a0640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb51c105dd0 0x7fb51c19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb5219a0640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c105410 0x7fb51c199530 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb5219a0640 1 --2- 172.21.15.78:0/665521704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c101230 0x7fb51c198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:31.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb5219a0640 1 -- 172.21.15.78:0/665521704 >> 172.21.15.78:0/665521704 conn(0x7fb51c0fd2f0 msgr2=0x7fb51c106cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:31.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb5219a0640 1 -- 172.21.15.78:0/665521704 shutdown_connections 2024-09-11T01:00:31.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:31.030+0000 7fb5219a0640 1 -- 172.21.15.78:0/665521704 wait complete. 2024-09-11T01:00:31.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:31 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:00:31.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:31 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:31.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:31 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:00:31.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92512 13.28 MB/sec warmup 39 sec latency 3859.342 ms 2024-09-11T01:00:31.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:31 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:00:31.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:31 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:31.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:31 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:00:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:31 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:00:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:31 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:31 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:00:31.755 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:00:32.001 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:00:32.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92595 12.95 MB/sec warmup 40 sec latency 1019.723 ms 2024-09-11T01:00:32.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:00:32.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:00:32.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:00:32.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:00:32.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:00:32.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:32 smithi110 ceph-mon[34712]: pgmap v373: 97 pgs: 97 active+clean; 76 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 3.4 MiB/s wr, 136 op/s 2024-09-11T01:00:32.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:00:32.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:00:32.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:00:32.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:00:32.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:00:32.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:32 smithi078 ceph-mon[31315]: pgmap v373: 97 pgs: 97 active+clean; 76 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 3.4 MiB/s wr, 136 op/s 2024-09-11T01:00:32.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:00:32.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:00:32.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:00:32.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:00:32.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:00:32.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:00:32.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:32 smithi112 ceph-mon[35384]: pgmap v373: 97 pgs: 97 active+clean; 76 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 3.4 MiB/s wr, 136 op/s 2024-09-11T01:00:32.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.519+0000 7f777bc7d640 1 -- 172.21.15.78:0/2944370144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7774104590 msgr2=0x7f777410cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.519+0000 7f777bc7d640 1 --2- 172.21.15.78:0/2944370144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7774104590 0x7f777410cb70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7768009900 tx=0x7f776802a410 comp rx=0 tx=0).stop 2024-09-11T01:00:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.519+0000 7f777bc7d640 1 -- 172.21.15.78:0/2944370144 shutdown_connections 2024-09-11T01:00:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.519+0000 7f777bc7d640 1 --2- 172.21.15.78:0/2944370144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f777410d140 0x7f777410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.519+0000 7f777bc7d640 1 --2- 172.21.15.78:0/2944370144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7774104590 0x7f777410cb70 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.519+0000 7f777bc7d640 1 --2- 172.21.15.78:0/2944370144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7774103b90 0x7f7774103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.519+0000 7f777bc7d640 1 -- 172.21.15.78:0/2944370144 >> 172.21.15.78:0/2944370144 conn(0x7f77740fd700 msgr2=0x7f77740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.519+0000 7f777bc7d640 1 -- 172.21.15.78:0/2944370144 shutdown_connections 2024-09-11T01:00:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.519+0000 7f777bc7d640 1 -- 172.21.15.78:0/2944370144 wait complete. 2024-09-11T01:00:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.519+0000 7f777bc7d640 1 Processor -- start 2024-09-11T01:00:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f777bc7d640 1 -- start start 2024-09-11T01:00:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f777bc7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7774103b90 0x7f7774199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f777bc7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7774104590 0x7f77741997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f777bc7d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f777410d140 0x7f777419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f777bc7d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7774112d80 con 0x7f7774103b90 2024-09-11T01:00:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f777bc7d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7774112c00 con 0x7f777410d140 2024-09-11T01:00:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f777bc7d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7774112f00 con 0x7f7774104590 2024-09-11T01:00:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f7779a19640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7774103b90 0x7f7774199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f7779a19640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7774103b90 0x7f7774199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45008/0 (socket says 172.21.15.78:45008) 2024-09-11T01:00:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f7779a19640 1 -- 172.21.15.78:0/2114147298 learned_addr learned my addr 172.21.15.78:0/2114147298 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f7779218640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7774104590 0x7f77741997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f777a21a640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f777410d140 0x7f777419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f7779a19640 1 -- 172.21.15.78:0/2114147298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7774104590 msgr2=0x7f77741997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f7779a19640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7774104590 0x7f77741997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f7779a19640 1 -- 172.21.15.78:0/2114147298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f777410d140 msgr2=0x7f777419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f7779a19640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f777410d140 0x7f777419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f7779a19640 1 -- 172.21.15.78:0/2114147298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777419f340 con 0x7f7774103b90 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f777a21a640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f777410d140 0x7f777419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.520+0000 7f7779a19640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7774103b90 0x7f7774199290 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f776400eb90 tx=0x7f776400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.521+0000 7f7772ffd640 1 -- 172.21.15.78:0/2114147298 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764004d20 con 0x7f7774103b90 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.521+0000 7f7772ffd640 1 -- 172.21.15.78:0/2114147298 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f776400ee90 con 0x7f7774103b90 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.521+0000 7f777bc7d640 1 -- 172.21.15.78:0/2114147298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777419f630 con 0x7f7774103b90 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.521+0000 7f7772ffd640 1 -- 172.21.15.78:0/2114147298 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764010670 con 0x7f7774103b90 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.521+0000 7f777bc7d640 1 -- 172.21.15.78:0/2114147298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77741a5cf0 con 0x7f7774103b90 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.523+0000 7f7772ffd640 1 -- 172.21.15.78:0/2114147298 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7764005020 con 0x7f7774103b90 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.523+0000 7f777e052740 1 Processor -- start 2024-09-11T01:00:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.523+0000 7f777e052740 1 -- start start 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777e052740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060480 0x55b559060860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777e052740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060da0 0x55b559070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777e052740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559071140 0x55b559073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777e052740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b558fd6e50 con 0x55b559060da0 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777e052740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b558fc6280 con 0x55b559071140 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777e052740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b558fca090 con 0x55b559060480 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f7779a19640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060da0 0x55b559070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777a21a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060480 0x55b559060860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f7779a19640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060da0 0x55b559070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45016/0 (socket says 172.21.15.78:45016) 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f7779a19640 1 -- 172.21.15.78:0/294116169 learned_addr learned my addr 172.21.15.78:0/294116169 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f7779218640 1 --2- 172.21.15.78:0/294116169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559071140 0x55b559073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777a21a640 1 -- 172.21.15.78:0/294116169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559071140 msgr2=0x55b559073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777a21a640 1 --2- 172.21.15.78:0/294116169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559071140 0x55b559073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777a21a640 1 -- 172.21.15.78:0/294116169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060da0 msgr2=0x55b559070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777a21a640 1 --2- 172.21.15.78:0/294116169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060da0 0x55b559070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777a21a640 1 -- 172.21.15.78:0/294116169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b558d24650 con 0x55b559060480 2024-09-11T01:00:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f7772ffd640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f775c0b5450 0x7f775c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f777a21a640 1 --2- 172.21.15.78:0/294116169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060480 0x55b559060860 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f776c009bf0 tx=0x7f776c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff2aa87e96918996 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f7779a19640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f775c0b5450 0x7f775c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f775bfff640 1 -- 172.21.15.78:0/294116169 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c004030 con 0x55b559060480 2024-09-11T01:00:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f775bfff640 1 -- 172.21.15.78:0/294116169 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f776c0035f0 con 0x55b559060480 2024-09-11T01:00:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.524+0000 7f775bfff640 1 -- 172.21.15.78:0/294116169 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c012690 con 0x55b559060480 2024-09-11T01:00:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.525+0000 7f7772ffd640 1 -- 172.21.15.78:0/2114147298 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f77640e66c0 con 0x7f7774103b90 2024-09-11T01:00:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.525+0000 7f777e052740 1 -- 172.21.15.78:0/294116169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060480 msgr2=0x55b559060860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.525+0000 7f7779a19640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f775c0b5450 0x7f775c0b7910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f77640acae0 tx=0x7f77640ec530 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.525+0000 7f777e052740 1 --2- 172.21.15.78:0/294116169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060480 0x55b559060860 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f776c009bf0 tx=0x7f776c010740 comp rx=0 tx=0).stop 2024-09-11T01:00:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.525+0000 7f777e052740 1 -- 172.21.15.78:0/294116169 shutdown_connections 2024-09-11T01:00:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.525+0000 7f777e052740 1 --2- 172.21.15.78:0/294116169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559071140 0x55b559073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.525+0000 7f777e052740 1 --2- 172.21.15.78:0/294116169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060da0 0x55b559070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.525+0000 7f777e052740 1 --2- 172.21.15.78:0/294116169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060480 0x55b559060860 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.525+0000 7f777e052740 1 -- 172.21.15.78:0/294116169 >> 172.21.15.78:0/294116169 conn(0x55b55905e2b0 msgr2=0x55b55905f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.525+0000 7f777e052740 1 -- 172.21.15.78:0/294116169 shutdown_connections 2024-09-11T01:00:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.525+0000 7f777e052740 1 -- 172.21.15.78:0/294116169 wait complete. 2024-09-11T01:00:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777e052740 1 -- start start 2024-09-11T01:00:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777e052740 1 Processor -- start 2024-09-11T01:00:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777e052740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 0x55b559070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777e052740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559060da0 0x55b55906de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777e052740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559071140 0x55b55906e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777e052740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b558e27700 con 0x55b559060480 2024-09-11T01:00:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777e052740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b558f61110 con 0x55b559060da0 2024-09-11T01:00:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777e052740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b559073df0 con 0x55b559071140 2024-09-11T01:00:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777e052740 10 client.0 _ll_register_callbacks cb 0x55b558f7a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777a21a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 0x55b559070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777a21a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 0x55b559070a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45018/0 (socket says 172.21.15.78:45018) 2024-09-11T01:00:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777a21a640 1 -- 172.21.15.78:0/1092442786 learned_addr learned my addr 172.21.15.78:0/1092442786 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f7779218640 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559060da0 0x55b55906de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.526+0000 7f777a21a640 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559071140 0x55b55906e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777a21a640 1 -- 172.21.15.78:0/1092442786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559071140 msgr2=0x55b55906e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777a21a640 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559071140 0x55b55906e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777a21a640 1 -- 172.21.15.78:0/1092442786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559060da0 msgr2=0x55b55906de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777a21a640 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559060da0 0x55b55906de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777a21a640 1 -- 172.21.15.78:0/1092442786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b558f8ec40 con 0x55b559060480 2024-09-11T01:00:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777a21a640 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 0x55b559070a20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f776c009460 tx=0x7f776c0093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ce7b81f6e49831 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f77597fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:00:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f77597fa640 1 -- 172.21.15.78:0/1092442786 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c005070 con 0x55b559060480 2024-09-11T01:00:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f77597fa640 1 -- 172.21.15.78:0/1092442786 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f776c005210 con 0x55b559060480 2024-09-11T01:00:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777e052740 1 -- 172.21.15.78:0/1092442786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f776c0035f0 con 0x55b559060480 2024-09-11T01:00:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f77597fa640 1 -- 172.21.15.78:0/1092442786 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f776c032050 con 0x55b559060480 2024-09-11T01:00:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777e052740 10 client.15828 fetch_fsmap learned FSMap version 31 2024-09-11T01:00:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777e052740 1 -- 172.21.15.78:0/1092442786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b55907f7a0 con 0x55b559060480 2024-09-11T01:00:32.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f77597fa640 1 -- 172.21.15.78:0/1092442786 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f776c0035f0 con 0x55b559060480 2024-09-11T01:00:32.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f77597fa640 1 -- 172.21.15.78:0/1092442786 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f776c02a020 con 0x55b559060480 2024-09-11T01:00:32.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777e052740 10 client.15828 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:00:32.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777e052740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:00:32.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777e052740 20 client.15828 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:32.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777e052740 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b559080c00 0x55b559082ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777e052740 4 client.15828 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:00:32.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.527+0000 7f777e052740 1 -- 172.21.15.78:0/1092442786 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b558fd8da0 con 0x55b559080c00 2024-09-11T01:00:32.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.528+0000 7f7779218640 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b559080c00 0x55b559082ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.528+0000 7f7779218640 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b559080c00 0x55b559082ff0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a9a2a2554221ec0 server_cookie=1f7e9672e194b833 in_seq=0 out_seq=0 2024-09-11T01:00:32.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.528+0000 7f77597fa640 10 client.15828 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:32.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f77597fa640 1 -- 172.21.15.78:0/1092442786 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b558fd8da0 con 0x55b559080c00 2024-09-11T01:00:32.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f77597fa640 10 client.15828 handle_command_reply: tid=0 2024-09-11T01:00:32.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f777e052740 1 client.15828 shutdown 2024-09-11T01:00:32.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f777e052740 1 -- 172.21.15.78:0/1092442786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 msgr2=0x55b559070a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:32.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f777e052740 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 0x55b559070a20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f776c009460 tx=0x7f776c0093b0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f777e052740 1 -- 172.21.15.78:0/1092442786 shutdown_connections 2024-09-11T01:00:32.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f777e052740 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559071140 0x55b55906e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f777e052740 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559060da0 0x55b55906de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f777e052740 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b559080c00 0x55b559082ff0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f777e052740 1 --2- 172.21.15.78:0/1092442786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 0x55b559070a20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f77597fa640 0 client.15828 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:32.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f777e052740 1 -- 172.21.15.78:0/1092442786 >> 172.21.15.78:0/1092442786 conn(0x55b55905e2b0 msgr2=0x55b559063e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:32.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f777e052740 1 -- 172.21.15.78:0/1092442786 shutdown_connections 2024-09-11T01:00:32.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f777e052740 1 -- 172.21.15.78:0/1092442786 wait complete. 2024-09-11T01:00:32.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.529+0000 7f777e052740 20 client.15828 trim_cache size 0 max 16384 2024-09-11T01:00:32.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.546+0000 7f777e052740 1 Processor -- start 2024-09-11T01:00:32.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.546+0000 7f777e052740 1 -- start start 2024-09-11T01:00:32.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.546+0000 7f777e052740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559060480 0x55b55907d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.546+0000 7f777e052740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060da0 0x55b55906fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.546+0000 7f777e052740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559071140 0x55b559088730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.546+0000 7f777e052740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77741ac390 con 0x55b559071140 2024-09-11T01:00:32.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.546+0000 7f777e052740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77741adf90 con 0x55b559060480 2024-09-11T01:00:32.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.546+0000 7f777e052740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77741aca60 con 0x55b559060da0 2024-09-11T01:00:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.547+0000 7f777a21a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559071140 0x55b559088730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.547+0000 7f777a21a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559071140 0x55b559088730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45020/0 (socket says 172.21.15.78:45020) 2024-09-11T01:00:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.547+0000 7f777a21a640 1 -- 172.21.15.78:0/856207998 learned_addr learned my addr 172.21.15.78:0/856207998 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.547+0000 7f777a21a640 1 -- 172.21.15.78:0/856207998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060da0 msgr2=0x55b55906fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.547+0000 7f777a21a640 1 --2- 172.21.15.78:0/856207998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060da0 0x55b55906fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.547+0000 7f777a21a640 1 -- 172.21.15.78:0/856207998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559060480 msgr2=0x55b55907d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.547+0000 7f777a21a640 1 --2- 172.21.15.78:0/856207998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559060480 0x55b55907d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.547+0000 7f777a21a640 1 -- 172.21.15.78:0/856207998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b55906dc20 con 0x55b559071140 2024-09-11T01:00:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.547+0000 7f777a21a640 1 --2- 172.21.15.78:0/856207998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559071140 0x55b559088730 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f776c012ef0 tx=0x7f776c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4660aa7a4f2f040d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.547+0000 7f775b7fe640 1 -- 172.21.15.78:0/856207998 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c0035f0 con 0x55b559071140 2024-09-11T01:00:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.547+0000 7f775b7fe640 1 -- 172.21.15.78:0/856207998 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f776c021020 con 0x55b559071140 2024-09-11T01:00:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.547+0000 7f775b7fe640 1 -- 172.21.15.78:0/856207998 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c003da0 con 0x55b559071140 2024-09-11T01:00:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.548+0000 7f777e052740 1 -- 172.21.15.78:0/856207998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559071140 msgr2=0x55b559088730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.548+0000 7f777e052740 1 --2- 172.21.15.78:0/856207998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559071140 0x55b559088730 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f776c012ef0 tx=0x7f776c010740 comp rx=0 tx=0).stop 2024-09-11T01:00:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.548+0000 7f777e052740 1 -- 172.21.15.78:0/856207998 shutdown_connections 2024-09-11T01:00:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.548+0000 7f777e052740 1 --2- 172.21.15.78:0/856207998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559071140 0x55b559088730 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.548+0000 7f777e052740 1 --2- 172.21.15.78:0/856207998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060da0 0x55b55906fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.548+0000 7f777e052740 1 --2- 172.21.15.78:0/856207998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559060480 0x55b55907d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.548+0000 7f777e052740 1 -- 172.21.15.78:0/856207998 >> 172.21.15.78:0/856207998 conn(0x55b55905e2b0 msgr2=0x55b55906f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:32.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.548+0000 7f777e052740 1 -- 172.21.15.78:0/856207998 shutdown_connections 2024-09-11T01:00:32.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.548+0000 7f777e052740 1 -- 172.21.15.78:0/856207998 wait complete. 2024-09-11T01:00:32.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.548+0000 7f777e052740 1 -- start start 2024-09-11T01:00:32.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.548+0000 7f777e052740 1 Processor -- start 2024-09-11T01:00:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777e052740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 0x55b5590792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777e052740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060da0 0x55b559079830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777e052740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559071140 0x55b5590760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777e052740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77741ad800 con 0x55b559060480 2024-09-11T01:00:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777e052740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77741ad130 con 0x55b559071140 2024-09-11T01:00:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777e052740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77741aa370 con 0x55b559060da0 2024-09-11T01:00:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777e052740 10 client.0 _ll_register_callbacks cb 0x7f77640ec400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777a21a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 0x55b5590792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777a21a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 0x55b5590792f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45024/0 (socket says 172.21.15.78:45024) 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777a21a640 1 -- 172.21.15.78:0/1848284497 learned_addr learned my addr 172.21.15.78:0/1848284497 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777a21a640 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559071140 0x55b5590760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f7779218640 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060da0 0x55b559079830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777a21a640 1 -- 172.21.15.78:0/1848284497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060da0 msgr2=0x55b559079830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777a21a640 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060da0 0x55b559079830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777a21a640 1 -- 172.21.15.78:0/1848284497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559071140 msgr2=0x55b5590760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777a21a640 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559071140 0x55b5590760e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777a21a640 1 -- 172.21.15.78:0/1848284497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b559076820 con 0x55b559060480 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f7779218640 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060da0 0x55b559079830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777a21a640 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 0x55b5590792f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f776c009bf0 tx=0x7f776c0098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcad17fbdd5255b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f77597fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:00:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f77597fa640 1 -- 172.21.15.78:0/1848284497 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c0050c0 con 0x55b559060480 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f77597fa640 1 -- 172.21.15.78:0/1848284497 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f776c005260 con 0x55b559060480 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.549+0000 7f777e052740 1 -- 172.21.15.78:0/1848284497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f776c021020 con 0x55b559060480 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.550+0000 7f77597fa640 1 -- 172.21.15.78:0/1848284497 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f776c005460 con 0x55b559060480 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.550+0000 7f777e052740 10 client.15834 fetch_fsmap learned FSMap version 31 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.550+0000 7f777e052740 1 -- 172.21.15.78:0/1848284497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b5590ee390 con 0x55b559060480 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.550+0000 7f77597fa640 1 -- 172.21.15.78:0/1848284497 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f776c021020 con 0x55b559060480 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.550+0000 7f77597fa640 1 -- 172.21.15.78:0/1848284497 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f776c02a040 con 0x55b559060480 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.550+0000 7f777e052740 10 client.15834 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.550+0000 7f777e052740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.550+0000 7f777e052740 20 client.15834 populate_metadata read hostname 'smithi078' 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.550+0000 7f777e052740 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b5590ee940 0x55b5590f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.550+0000 7f777e052740 4 client.15834 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:00:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.550+0000 7f777e052740 1 -- 172.21.15.78:0/1848284497 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b558fd4850 con 0x55b5590ee940 2024-09-11T01:00:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.550+0000 7f7779218640 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b5590ee940 0x55b5590f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:00:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f7779218640 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b5590ee940 0x55b5590f0d30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ecccfc8f588aed1 server_cookie=73bfc43d96dbf140 in_seq=0 out_seq=0 2024-09-11T01:00:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f77597fa640 10 client.15834 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f77597fa640 1 -- 172.21.15.78:0/1848284497 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x55b558fd4850 con 0x55b5590ee940 2024-09-11T01:00:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f77597fa640 10 client.15834 handle_command_reply: tid=0 2024-09-11T01:00:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f777e052740 1 client.15834 shutdown 2024-09-11T01:00:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f777e052740 1 -- 172.21.15.78:0/1848284497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 msgr2=0x55b5590792f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f777e052740 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 0x55b5590792f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f776c009bf0 tx=0x7f776c0098d0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f777e052740 1 -- 172.21.15.78:0/1848284497 shutdown_connections 2024-09-11T01:00:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f777e052740 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b559071140 0x55b5590760e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f777e052740 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b559060da0 0x55b559079830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f777e052740 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b5590ee940 0x55b5590f0d30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f777e052740 1 --2- 172.21.15.78:0/1848284497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b559060480 0x55b5590792f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f77597fa640 0 client.15834 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:00:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f777e052740 1 -- 172.21.15.78:0/1848284497 >> 172.21.15.78:0/1848284497 conn(0x55b55905e2b0 msgr2=0x55b55906fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f777e052740 1 -- 172.21.15.78:0/1848284497 shutdown_connections 2024-09-11T01:00:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f777e052740 1 -- 172.21.15.78:0/1848284497 wait complete. 2024-09-11T01:00:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.551+0000 7f777e052740 20 client.15834 trim_cache size 0 max 16384 2024-09-11T01:00:32.579 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:00:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:00:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:00:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:00:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:00:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "6ba15803-6da9-4d97-adb8-4d6230a3c7a3", 2024-09-11T01:00:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:00:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:00:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:00:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:00:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:00:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:00:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:00:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:00:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:00:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:00:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:00:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:00:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:00:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:00:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:00:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:00:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:00:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:00:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:00:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:00:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:00:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:00:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:00:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:00:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:00:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:00:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:00:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:00:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:00:32.582 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:00:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.552+0000 7f777bc7d640 1 -- 172.21.15.78:0/2114147298 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f775c0b5450 msgr2=0x7f775c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.552+0000 7f777bc7d640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f775c0b5450 0x7f775c0b7910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f77640acae0 tx=0x7f77640ec530 comp rx=0 tx=0).stop 2024-09-11T01:00:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.552+0000 7f777bc7d640 1 -- 172.21.15.78:0/2114147298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7774103b90 msgr2=0x7f7774199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:00:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.552+0000 7f777bc7d640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7774103b90 0x7f7774199290 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f776400eb90 tx=0x7f776400c6a0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.552+0000 7f777bc7d640 1 -- 172.21.15.78:0/2114147298 shutdown_connections 2024-09-11T01:00:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.552+0000 7f777bc7d640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f775c0b5450 0x7f775c0b7910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.552+0000 7f777bc7d640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f777410d140 0x7f777419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.552+0000 7f777bc7d640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7774104590 0x7f77741997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.552+0000 7f777bc7d640 1 --2- 172.21.15.78:0/2114147298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7774103b90 0x7f7774199290 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:00:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.552+0000 7f777bc7d640 1 -- 172.21.15.78:0/2114147298 >> 172.21.15.78:0/2114147298 conn(0x7f77740fd700 msgr2=0x7f77740fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:00:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.552+0000 7f777bc7d640 1 -- 172.21.15.78:0/2114147298 shutdown_connections 2024-09-11T01:00:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:00:32.552+0000 7f777bc7d640 1 -- 172.21.15.78:0/2114147298 wait complete. 2024-09-11T01:00:33.359 INFO:tasks.cephfs.filesystem:scrub status for tag:6ba15803-6da9-4d97-adb8-4d6230a3c7a3 - {'path': '/', 'tag': '6ba15803-6da9-4d97-adb8-4d6230a3c7a3', 'options': 'recursive,force'} 2024-09-11T01:00:33.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92595 12.63 MB/sec warmup 41 sec latency 1129.050 ms 2024-09-11T01:00:34.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92595 12.33 MB/sec warmup 42 sec latency 2129.147 ms 2024-09-11T01:00:35.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92595 12.04 MB/sec warmup 43 sec latency 3129.217 ms 2024-09-11T01:00:35.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:35 smithi110 ceph-mon[34712]: pgmap v374: 97 pgs: 97 active+clean; 78 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 3.6 MiB/s wr, 135 op/s 2024-09-11T01:00:35.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:35 smithi112 ceph-mon[35384]: pgmap v374: 97 pgs: 97 active+clean; 78 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 3.6 MiB/s wr, 135 op/s 2024-09-11T01:00:35.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:35 smithi078 ceph-mon[31315]: pgmap v374: 97 pgs: 97 active+clean; 78 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 3.6 MiB/s wr, 135 op/s 2024-09-11T01:00:36.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92595 11.77 MB/sec warmup 44 sec latency 4129.303 ms 2024-09-11T01:00:37.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 92737 11.52 MB/sec warmup 45 sec latency 4894.665 ms 2024-09-11T01:00:37.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:37 smithi110 ceph-mon[34712]: pgmap v375: 97 pgs: 97 active+clean; 77 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 3.9 MiB/s wr, 103 op/s 2024-09-11T01:00:37.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:37 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:00:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:37 smithi112 ceph-mon[35384]: pgmap v375: 97 pgs: 97 active+clean; 77 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 3.9 MiB/s wr, 103 op/s 2024-09-11T01:00:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:37 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:00:37.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:37 smithi078 ceph-mon[31315]: pgmap v375: 97 pgs: 97 active+clean; 77 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 3.9 MiB/s wr, 103 op/s 2024-09-11T01:00:37.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:37 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:00:38.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 95443 11.55 MB/sec warmup 46 sec latency 12.220 ms 2024-09-11T01:00:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:38 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:00:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:38 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:00:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:38 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:00:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:38 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:00:38.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:38 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:00:38.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:38 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:00:38.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:38 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:00:38.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:38 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:00:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:38 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:00:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:38 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:00:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:38 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:00:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:38 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:00:39.397 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:39 smithi078 ceph-mon[31315]: pgmap v376: 97 pgs: 97 active+clean; 77 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.7 MiB/s wr, 93 op/s 2024-09-11T01:00:39.418 INFO:tasks.workunit.client.0.smithi078.stdout: 1 96690 11.45 MB/sec warmup 47 sec latency 106.254 ms 2024-09-11T01:00:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:39 smithi110 ceph-mon[34712]: pgmap v376: 97 pgs: 97 active+clean; 77 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.7 MiB/s wr, 93 op/s 2024-09-11T01:00:39.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:39 smithi112 ceph-mon[35384]: pgmap v376: 97 pgs: 97 active+clean; 77 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.7 MiB/s wr, 93 op/s 2024-09-11T01:00:40.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 99590 11.59 MB/sec warmup 48 sec latency 11.496 ms 2024-09-11T01:00:41.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 101803 11.50 MB/sec warmup 49 sec latency 9.548 ms 2024-09-11T01:00:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:41 smithi110 ceph-mon[34712]: pgmap v377: 97 pgs: 97 active+clean; 76 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 3.2 MiB/s wr, 126 op/s 2024-09-11T01:00:41.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:41 smithi112 ceph-mon[35384]: pgmap v377: 97 pgs: 97 active+clean; 76 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 3.2 MiB/s wr, 126 op/s 2024-09-11T01:00:41.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:41 smithi078 ceph-mon[31315]: pgmap v377: 97 pgs: 97 active+clean; 76 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 3.2 MiB/s wr, 126 op/s 2024-09-11T01:00:42.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 103631 11.52 MB/sec warmup 50 sec latency 16.077 ms 2024-09-11T01:00:43.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 106598 11.64 MB/sec warmup 51 sec latency 9.336 ms 2024-09-11T01:00:43.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:43 smithi110 ceph-mon[34712]: pgmap v378: 97 pgs: 97 active+clean; 69 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 3.1 MiB/s wr, 181 op/s 2024-09-11T01:00:43.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:43 smithi112 ceph-mon[35384]: pgmap v378: 97 pgs: 97 active+clean; 69 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 3.1 MiB/s wr, 181 op/s 2024-09-11T01:00:43.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:43 smithi078 ceph-mon[31315]: pgmap v378: 97 pgs: 97 active+clean; 69 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 3.1 MiB/s wr, 181 op/s 2024-09-11T01:00:44.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 107658 11.48 MB/sec warmup 52 sec latency 68.846 ms 2024-09-11T01:00:45.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 110338 11.58 MB/sec warmup 53 sec latency 42.538 ms 2024-09-11T01:00:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:45 smithi110 ceph-mon[34712]: pgmap v379: 97 pgs: 97 active+clean; 71 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-09-11T01:00:45.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:45 smithi112 ceph-mon[35384]: pgmap v379: 97 pgs: 97 active+clean; 71 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-09-11T01:00:45.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:45 smithi078 ceph-mon[31315]: pgmap v379: 97 pgs: 97 active+clean; 71 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-09-11T01:00:46.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 112595 11.52 MB/sec warmup 54 sec latency 9.246 ms 2024-09-11T01:00:47.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 114457 11.52 MB/sec warmup 55 sec latency 11.268 ms 2024-09-11T01:00:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:47 smithi110 ceph-mon[34712]: pgmap v380: 97 pgs: 97 active+clean; 72 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 283 op/s 2024-09-11T01:00:47.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:47 smithi112 ceph-mon[35384]: pgmap v380: 97 pgs: 97 active+clean; 72 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 283 op/s 2024-09-11T01:00:47.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:47 smithi078 ceph-mon[31315]: pgmap v380: 97 pgs: 97 active+clean; 72 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 283 op/s 2024-09-11T01:00:48.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 117320 11.64 MB/sec warmup 56 sec latency 9.452 ms 2024-09-11T01:00:49.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 118649 11.53 MB/sec warmup 57 sec latency 124.504 ms 2024-09-11T01:00:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:49 smithi110 ceph-mon[34712]: pgmap v381: 97 pgs: 97 active+clean; 73 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 287 op/s 2024-09-11T01:00:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:49 smithi112 ceph-mon[35384]: pgmap v381: 97 pgs: 97 active+clean; 73 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 287 op/s 2024-09-11T01:00:49.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:49 smithi078 ceph-mon[31315]: pgmap v381: 97 pgs: 97 active+clean; 73 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 287 op/s 2024-09-11T01:00:50.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 121060 11.57 MB/sec warmup 58 sec latency 9.549 ms 2024-09-11T01:00:51.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 123827 11.60 MB/sec warmup 59 sec latency 19.930 ms 2024-09-11T01:00:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:51 smithi110 ceph-mon[34712]: pgmap v382: 97 pgs: 97 active+clean; 73 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 325 op/s 2024-09-11T01:00:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:51 smithi112 ceph-mon[35384]: pgmap v382: 97 pgs: 97 active+clean; 73 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 325 op/s 2024-09-11T01:00:51.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:51 smithi078 ceph-mon[31315]: pgmap v382: 97 pgs: 97 active+clean; 73 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 325 op/s 2024-09-11T01:00:52.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 125692 11.59 MB/sec warmup 60 sec latency 12.924 ms 2024-09-11T01:00:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:52 smithi112 ceph-mon[35384]: pgmap v383: 97 pgs: 97 active+clean; 78 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-09-11T01:00:52.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:52 smithi078 ceph-mon[31315]: pgmap v383: 97 pgs: 97 active+clean; 78 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-09-11T01:00:52.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:52 smithi110 ceph-mon[34712]: pgmap v383: 97 pgs: 97 active+clean; 78 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-09-11T01:00:53.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 128150 11.63 MB/sec warmup 61 sec latency 10.794 ms 2024-09-11T01:00:54.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 130628 11.63 MB/sec warmup 62 sec latency 55.008 ms 2024-09-11T01:00:55.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 132344 11.59 MB/sec warmup 63 sec latency 18.666 ms 2024-09-11T01:00:55.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:55 smithi110 ceph-mon[34712]: pgmap v384: 97 pgs: 97 active+clean; 79 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.3 MiB/s wr, 385 op/s 2024-09-11T01:00:55.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:55 smithi112 ceph-mon[35384]: pgmap v384: 97 pgs: 97 active+clean; 79 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.3 MiB/s wr, 385 op/s 2024-09-11T01:00:55.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:55 smithi078 ceph-mon[31315]: pgmap v384: 97 pgs: 97 active+clean; 79 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.3 MiB/s wr, 385 op/s 2024-09-11T01:00:56.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 135038 11.68 MB/sec warmup 64 sec latency 10.328 ms 2024-09-11T01:00:57.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 137269 11.61 MB/sec warmup 65 sec latency 9.741 ms 2024-09-11T01:00:57.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:57 smithi110 ceph-mon[34712]: pgmap v385: 97 pgs: 97 active+clean; 80 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 353 op/s 2024-09-11T01:00:57.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:57 smithi112 ceph-mon[35384]: pgmap v385: 97 pgs: 97 active+clean; 80 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 353 op/s 2024-09-11T01:00:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:57 smithi078 ceph-mon[31315]: pgmap v385: 97 pgs: 97 active+clean; 80 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 353 op/s 2024-09-11T01:00:58.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 139100 11.62 MB/sec warmup 66 sec latency 12.315 ms 2024-09-11T01:00:59.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 142051 11.71 MB/sec warmup 67 sec latency 10.834 ms 2024-09-11T01:00:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:00:59 smithi110 ceph-mon[34712]: pgmap v386: 97 pgs: 97 active+clean; 82 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-09-11T01:00:59.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:00:59 smithi112 ceph-mon[35384]: pgmap v386: 97 pgs: 97 active+clean; 82 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-09-11T01:00:59.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:00:59 smithi078 ceph-mon[31315]: pgmap v386: 97 pgs: 97 active+clean; 82 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-09-11T01:01:00.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 144246 11.66 MB/sec warmup 68 sec latency 8.865 ms 2024-09-11T01:01:01.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 145758 11.66 MB/sec warmup 69 sec latency 20.111 ms 2024-09-11T01:01:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:01 smithi110 ceph-mon[34712]: pgmap v387: 97 pgs: 97 active+clean; 83 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 361 op/s 2024-09-11T01:01:01.466 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:01 smithi078 ceph-mon[31315]: pgmap v387: 97 pgs: 97 active+clean; 83 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 361 op/s 2024-09-11T01:01:01.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:01 smithi112 ceph-mon[35384]: pgmap v387: 97 pgs: 97 active+clean; 83 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 361 op/s 2024-09-11T01:01:02.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 148013 11.61 MB/sec warmup 70 sec latency 12.791 ms 2024-09-11T01:01:03.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:01:03.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 149781 11.61 MB/sec warmup 71 sec latency 11.240 ms 2024-09-11T01:01:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:03 smithi110 ceph-mon[34712]: pgmap v388: 97 pgs: 97 active+clean; 86 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 374 op/s 2024-09-11T01:01:03.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:03 smithi112 ceph-mon[35384]: pgmap v388: 97 pgs: 97 active+clean; 86 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 374 op/s 2024-09-11T01:01:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:03 smithi078 ceph-mon[31315]: pgmap v388: 97 pgs: 97 active+clean; 86 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 374 op/s 2024-09-11T01:01:03.630 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:04.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.144+0000 7f80ff599640 1 -- 172.21.15.78:0/2367259154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f810d100 msgr2=0x7f80f810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.144+0000 7f80ff599640 1 --2- 172.21.15.78:0/2367259154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f810d100 0x7f80f810f560 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009ca0 tx=0x7f80f002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.144+0000 7f80ff599640 1 -- 172.21.15.78:0/2367259154 shutdown_connections 2024-09-11T01:01:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.144+0000 7f80ff599640 1 --2- 172.21.15.78:0/2367259154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f810d100 0x7f80f810f560 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.144+0000 7f80ff599640 1 --2- 172.21.15.78:0/2367259154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80f81045c0 0x7f80f810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.144+0000 7f80ff599640 1 --2- 172.21.15.78:0/2367259154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f8103ca0 0x7f80f8104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.144+0000 7f80ff599640 1 -- 172.21.15.78:0/2367259154 >> 172.21.15.78:0/2367259154 conn(0x7f80f80fd700 msgr2=0x7f80f80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.144+0000 7f80ff599640 1 -- 172.21.15.78:0/2367259154 shutdown_connections 2024-09-11T01:01:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.144+0000 7f80ff599640 1 -- 172.21.15.78:0/2367259154 wait complete. 2024-09-11T01:01:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80ff599640 1 Processor -- start 2024-09-11T01:01:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80ff599640 1 -- start start 2024-09-11T01:01:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80ff599640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f8103ca0 0x7f80f8101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80ff599640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f81045c0 0x7f80f8101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80ff599640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80f810d100 0x7f80f81023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80ff599640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8112d70 con 0x7f80f81045c0 2024-09-11T01:01:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80ff599640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8112bf0 con 0x7f80f810d100 2024-09-11T01:01:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80ff599640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8112ef0 con 0x7f80f8103ca0 2024-09-11T01:01:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80fcb34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f81045c0 0x7f80f8101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80fd335640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f8103ca0 0x7f80f8101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80fcb34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f81045c0 0x7f80f8101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60582/0 (socket says 172.21.15.78:60582) 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80fcb34640 1 -- 172.21.15.78:0/1096281422 learned_addr learned my addr 172.21.15.78:0/1096281422 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80fdb36640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80f810d100 0x7f80f81023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80fcb34640 1 -- 172.21.15.78:0/1096281422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f8103ca0 msgr2=0x7f80f8101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80fcb34640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f8103ca0 0x7f80f8101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80fcb34640 1 -- 172.21.15.78:0/1096281422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80f810d100 msgr2=0x7f80f81023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80fcb34640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80f810d100 0x7f80f81023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80fcb34640 1 -- 172.21.15.78:0/1096281422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f81a7760 con 0x7f80f81045c0 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.145+0000 7f80fcb34640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f81045c0 0x7f80f8101e90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f80e800e9c0 tx=0x7f80e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.146+0000 7f80ee7fc640 1 -- 172.21.15.78:0/1096281422 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e800ccc0 con 0x7f80f81045c0 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.146+0000 7f80ff599640 1 -- 172.21.15.78:0/1096281422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f81a7a50 con 0x7f80f81045c0 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.146+0000 7f80ee7fc640 1 -- 172.21.15.78:0/1096281422 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e80035f0 con 0x7f80f81045c0 2024-09-11T01:01:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.146+0000 7f80ee7fc640 1 -- 172.21.15.78:0/1096281422 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e800ce60 con 0x7f80f81045c0 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.146+0000 7f80ff599640 1 -- 172.21.15.78:0/1096281422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f81a7f90 con 0x7f80f81045c0 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f810196e740 1 Processor -- start 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f810196e740 1 -- start start 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f80ee7fc640 1 -- 172.21.15.78:0/1096281422 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f80e8004000 con 0x7f80f81045c0 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f810196e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed5962480 0x55fed5962860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f810196e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962da0 0x55fed596e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f810196e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed596ed30 0x55fed5971120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f810196e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fed58d8e50 con 0x55fed596ed30 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f810196e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fed58c8280 con 0x55fed5962480 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f810196e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fed58cc090 con 0x55fed5962da0 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f80fcb34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed596ed30 0x55fed5971120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f80fd335640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962da0 0x55fed596e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f80fd335640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962da0 0x55fed596e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35290/0 (socket says 172.21.15.78:35290) 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f80fd335640 1 -- 172.21.15.78:0/2707204456 learned_addr learned my addr 172.21.15.78:0/2707204456 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.149+0000 7f80fdb36640 1 --2- 172.21.15.78:0/2707204456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed5962480 0x55fed5962860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.150+0000 7f80fd335640 1 -- 172.21.15.78:0/2707204456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed5962480 msgr2=0x55fed5962860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.150+0000 7f80fd335640 1 --2- 172.21.15.78:0/2707204456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed5962480 0x55fed5962860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.150+0000 7f80fd335640 1 -- 172.21.15.78:0/2707204456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed596ed30 msgr2=0x55fed5971120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.150+0000 7f80fd335640 1 --2- 172.21.15.78:0/2707204456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed596ed30 0x55fed5971120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.150+0000 7f80fd335640 1 -- 172.21.15.78:0/2707204456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed5626650 con 0x55fed5962da0 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.150+0000 7f80fd335640 1 --2- 172.21.15.78:0/2707204456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962da0 0x55fed596e7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009cf0 tx=0x7f80f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51b3772bf3dd511f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.150+0000 7f80ee7fc640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80e40b5450 0x7f80e40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.150+0000 7f80e37fe640 1 -- 172.21.15.78:0/2707204456 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4004230 con 0x55fed5962da0 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.150+0000 7f80e37fe640 1 -- 172.21.15.78:0/2707204456 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80f40043d0 con 0x55fed5962da0 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.150+0000 7f80e37fe640 1 -- 172.21.15.78:0/2707204456 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4019840 con 0x55fed5962da0 2024-09-11T01:01:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.150+0000 7f80fdb36640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80e40b5450 0x7f80e40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.150+0000 7f80ee7fc640 1 -- 172.21.15.78:0/1096281422 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f80e80e61a0 con 0x7f80f81045c0 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.151+0000 7f810196e740 1 -- 172.21.15.78:0/2707204456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962da0 msgr2=0x55fed596e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.151+0000 7f80fdb36640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80e40b5450 0x7f80e40b7910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f80f0006010 tx=0x7f80f00099a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.151+0000 7f810196e740 1 --2- 172.21.15.78:0/2707204456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962da0 0x55fed596e7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009cf0 tx=0x7f80f4011810 comp rx=0 tx=0).stop 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.151+0000 7f810196e740 1 -- 172.21.15.78:0/2707204456 shutdown_connections 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.151+0000 7f810196e740 1 --2- 172.21.15.78:0/2707204456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed596ed30 0x55fed5971120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.151+0000 7f810196e740 1 --2- 172.21.15.78:0/2707204456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962da0 0x55fed596e7f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.151+0000 7f810196e740 1 --2- 172.21.15.78:0/2707204456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed5962480 0x55fed5962860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.151+0000 7f810196e740 1 -- 172.21.15.78:0/2707204456 >> 172.21.15.78:0/2707204456 conn(0x55fed59602b0 msgr2=0x55fed5961450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.151+0000 7f810196e740 1 -- 172.21.15.78:0/2707204456 shutdown_connections 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.151+0000 7f810196e740 1 -- 172.21.15.78:0/2707204456 wait complete. 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.151+0000 7f810196e740 1 -- start start 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f810196e740 1 Processor -- start 2024-09-11T01:01:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f810196e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962480 0x55fed596e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f810196e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed5962da0 0x55fed596ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f810196e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed596ed30 0x55fed596bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f810196e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fed5729700 con 0x55fed5962480 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f810196e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fed5863110 con 0x55fed5962da0 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f810196e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fed59719b0 con 0x55fed596ed30 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f810196e740 10 client.0 _ll_register_callbacks cb 0x55fed587c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f80fd335640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962480 0x55fed596e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f80fd335640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962480 0x55fed596e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60594/0 (socket says 172.21.15.78:60594) 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f80fd335640 1 -- 172.21.15.78:0/1530079980 learned_addr learned my addr 172.21.15.78:0/1530079980 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f80fcb34640 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed5962da0 0x55fed596ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f80fdb36640 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed596ed30 0x55fed596bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f80fd335640 1 -- 172.21.15.78:0/1530079980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed596ed30 msgr2=0x55fed596bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f80fd335640 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed596ed30 0x55fed596bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f80fd335640 1 -- 172.21.15.78:0/1530079980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed5962da0 msgr2=0x55fed596ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f80fd335640 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed5962da0 0x55fed596ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f80fd335640 1 -- 172.21.15.78:0/1530079980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed5890c40 con 0x55fed5962480 2024-09-11T01:01:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.152+0000 7f80fcb34640 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed5962da0 0x55fed596ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f80fd335640 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962480 0x55fed596e5d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f80f4019460 tx=0x7f80f4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6dda3242690535a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f80e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:01:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f80e0ff9640 1 -- 172.21.15.78:0/1530079980 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4027070 con 0x55fed5962480 2024-09-11T01:01:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f810196e740 1 -- 172.21.15.78:0/1530079980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f40043d0 con 0x55fed5962480 2024-09-11T01:01:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f80e0ff9640 1 -- 172.21.15.78:0/1530079980 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f80f4005240 con 0x55fed5962480 2024-09-11T01:01:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f80e0ff9640 1 -- 172.21.15.78:0/1530079980 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f80f4033050 con 0x55fed5962480 2024-09-11T01:01:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f810196e740 10 client.15846 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f810196e740 1 -- 172.21.15.78:0/1530079980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fed597d2a0 con 0x55fed5962480 2024-09-11T01:01:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f80e0ff9640 1 -- 172.21.15.78:0/1530079980 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80f40043d0 con 0x55fed5962480 2024-09-11T01:01:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f80e0ff9640 1 -- 172.21.15.78:0/1530079980 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f80f4004b70 con 0x55fed5962480 2024-09-11T01:01:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f810196e740 10 client.15846 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f810196e740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:01:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f810196e740 20 client.15846 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f810196e740 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fed597e650 0x55fed5980a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f810196e740 4 client.15846 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:01:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.153+0000 7f810196e740 1 -- 172.21.15.78:0/1530079980 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fed58dada0 con 0x55fed597e650 2024-09-11T01:01:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.154+0000 7f80fcb34640 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fed597e650 0x55fed5980a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.154+0000 7f80fcb34640 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fed597e650 0x55fed5980a40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d9a6173aead20b server_cookie=169e45e88786878f in_seq=0 out_seq=0 2024-09-11T01:01:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.154+0000 7f80e0ff9640 10 client.15846 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f80e0ff9640 1 -- 172.21.15.78:0/1530079980 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fed58dada0 con 0x55fed597e650 2024-09-11T01:01:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f80e0ff9640 10 client.15846 handle_command_reply: tid=0 2024-09-11T01:01:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f810196e740 1 client.15846 shutdown 2024-09-11T01:01:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f810196e740 1 -- 172.21.15.78:0/1530079980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962480 msgr2=0x55fed596e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f810196e740 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962480 0x55fed596e5d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f80f4019460 tx=0x7f80f4003820 comp rx=0 tx=0).stop 2024-09-11T01:01:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f810196e740 1 -- 172.21.15.78:0/1530079980 shutdown_connections 2024-09-11T01:01:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f810196e740 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed596ed30 0x55fed596bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f810196e740 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed5962da0 0x55fed596ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f810196e740 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fed597e650 0x55fed5980a40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f810196e740 1 --2- 172.21.15.78:0/1530079980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962480 0x55fed596e5d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f80e0ff9640 0 client.15846 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f810196e740 1 -- 172.21.15.78:0/1530079980 >> 172.21.15.78:0/1530079980 conn(0x55fed59602b0 msgr2=0x55fed5965e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f810196e740 1 -- 172.21.15.78:0/1530079980 shutdown_connections 2024-09-11T01:01:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f810196e740 1 -- 172.21.15.78:0/1530079980 wait complete. 2024-09-11T01:01:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.155+0000 7f810196e740 20 client.15846 trim_cache size 0 max 16384 2024-09-11T01:01:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f810196e740 1 Processor -- start 2024-09-11T01:01:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f810196e740 1 -- start start 2024-09-11T01:01:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f810196e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962480 0x55fed595b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f810196e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 0x55fed596a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f810196e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed596ed30 0x55fed596a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f810196e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81b0800 con 0x55fed5962da0 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f810196e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81b2400 con 0x55fed596ed30 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f810196e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81b0ed0 con 0x55fed5962480 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f80fcb34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 0x55fed596a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f80fcb34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 0x55fed596a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60610/0 (socket says 172.21.15.78:60610) 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f80fcb34640 1 -- 172.21.15.78:0/1012800749 learned_addr learned my addr 172.21.15.78:0/1012800749 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f80fd335640 1 --2- 172.21.15.78:0/1012800749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962480 0x55fed595b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f80fdb36640 1 --2- 172.21.15.78:0/1012800749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed596ed30 0x55fed596a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f80fcb34640 1 -- 172.21.15.78:0/1012800749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962480 msgr2=0x55fed595b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f80fcb34640 1 --2- 172.21.15.78:0/1012800749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962480 0x55fed595b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f80fcb34640 1 -- 172.21.15.78:0/1012800749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed596ed30 msgr2=0x55fed596a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f80fcb34640 1 --2- 172.21.15.78:0/1012800749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed596ed30 0x55fed596a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.174+0000 7f80fcb34640 1 -- 172.21.15.78:0/1012800749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed596c350 con 0x55fed5962da0 2024-09-11T01:01:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.175+0000 7f80fd335640 1 --2- 172.21.15.78:0/1012800749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962480 0x55fed595b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.175+0000 7f80fcb34640 1 --2- 172.21.15.78:0/1012800749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 0x55fed596a2a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f80e80ac620 tx=0x7f80e8003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02b97885b27baeb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.175+0000 7f80e2ffd640 1 -- 172.21.15.78:0/1012800749 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80a8530 con 0x55fed5962da0 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.175+0000 7f80e2ffd640 1 -- 172.21.15.78:0/1012800749 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e80ac1a0 con 0x55fed5962da0 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.175+0000 7f80e2ffd640 1 -- 172.21.15.78:0/1012800749 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80abe00 con 0x55fed5962da0 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 -- 172.21.15.78:0/1012800749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 msgr2=0x55fed596a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 --2- 172.21.15.78:0/1012800749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 0x55fed596a2a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f80e80ac620 tx=0x7f80e8003e20 comp rx=0 tx=0).stop 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 -- 172.21.15.78:0/1012800749 shutdown_connections 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 --2- 172.21.15.78:0/1012800749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed596ed30 0x55fed596a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 --2- 172.21.15.78:0/1012800749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 0x55fed596a2a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 --2- 172.21.15.78:0/1012800749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962480 0x55fed595b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 -- 172.21.15.78:0/1012800749 >> 172.21.15.78:0/1012800749 conn(0x55fed59602b0 msgr2=0x55fed5961300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 -- 172.21.15.78:0/1012800749 shutdown_connections 2024-09-11T01:01:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 -- 172.21.15.78:0/1012800749 wait complete. 2024-09-11T01:01:04.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 -- start start 2024-09-11T01:01:04.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 Processor -- start 2024-09-11T01:01:04.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962480 0x55fed59775e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.176+0000 7f810196e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 0x55fed5986100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f810196e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed596ed30 0x55fed5986890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f810196e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81b1c70 con 0x55fed5962da0 2024-09-11T01:01:04.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f810196e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81b15a0 con 0x55fed596ed30 2024-09-11T01:01:04.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f810196e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81ae7e0 con 0x55fed5962480 2024-09-11T01:01:04.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f810196e740 10 client.0 _ll_register_callbacks cb 0x7f80e80e9bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:04.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80fcb34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 0x55fed5986100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80fcb34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 0x55fed5986100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60614/0 (socket says 172.21.15.78:60614) 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80fcb34640 1 -- 172.21.15.78:0/2789153213 learned_addr learned my addr 172.21.15.78:0/2789153213 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80fd335640 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962480 0x55fed59775e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80fdb36640 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed596ed30 0x55fed5986890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80fcb34640 1 -- 172.21.15.78:0/2789153213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962480 msgr2=0x55fed59775e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80fcb34640 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962480 0x55fed59775e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80fcb34640 1 -- 172.21.15.78:0/2789153213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed596ed30 msgr2=0x55fed5986890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80fcb34640 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed596ed30 0x55fed5986890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80fcb34640 1 -- 172.21.15.78:0/2789153213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed5973050 con 0x55fed5962da0 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80fcb34640 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 0x55fed5986100 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f80e80a9900 tx=0x7f80e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47e4bcbad4dc13e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80e0ff9640 1 -- 172.21.15.78:0/2789153213 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80abfe0 con 0x55fed5962da0 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80e0ff9640 1 -- 172.21.15.78:0/2789153213 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f80e8004320 con 0x55fed5962da0 2024-09-11T01:01:04.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f810196e740 1 -- 172.21.15.78:0/2789153213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e80ac1a0 con 0x55fed5962da0 2024-09-11T01:01:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f80e0ff9640 1 -- 172.21.15.78:0/2789153213 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f80e80ac460 con 0x55fed5962da0 2024-09-11T01:01:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f810196e740 10 client.15852 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.177+0000 7f810196e740 1 -- 172.21.15.78:0/2789153213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fed59eb780 con 0x55fed5962da0 2024-09-11T01:01:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.178+0000 7f80e0ff9640 1 -- 172.21.15.78:0/2789153213 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e80ac1a0 con 0x55fed5962da0 2024-09-11T01:01:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.178+0000 7f80e0ff9640 1 -- 172.21.15.78:0/2789153213 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f80e80a7fe0 con 0x55fed5962da0 2024-09-11T01:01:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.178+0000 7f810196e740 10 client.15852 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.178+0000 7f810196e740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:01:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.178+0000 7f810196e740 20 client.15852 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.178+0000 7f810196e740 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fed59ebe80 0x55fed59ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.178+0000 7f810196e740 4 client.15852 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:01:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.178+0000 7f810196e740 1 -- 172.21.15.78:0/2789153213 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fed58d6850 con 0x55fed59ebe80 2024-09-11T01:01:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.178+0000 7f80fd335640 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fed59ebe80 0x55fed59ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:04.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.178+0000 7f80e0ff9640 10 client.15852 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:04.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.178+0000 7f80fd335640 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fed59ebe80 0x55fed59ee2a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b2e59faf373cddd server_cookie=edc7f7bb8f9654bd in_seq=0 out_seq=0 2024-09-11T01:01:04.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f80e0ff9640 1 -- 172.21.15.78:0/2789153213 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x55fed58d6850 con 0x55fed59ebe80 2024-09-11T01:01:04.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f80e0ff9640 10 client.15852 handle_command_reply: tid=0 2024-09-11T01:01:04.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f810196e740 1 client.15852 shutdown 2024-09-11T01:01:04.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f810196e740 1 -- 172.21.15.78:0/2789153213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 msgr2=0x55fed5986100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:04.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f810196e740 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 0x55fed5986100 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f80e80a9900 tx=0x7f80e8000bc0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f810196e740 1 -- 172.21.15.78:0/2789153213 shutdown_connections 2024-09-11T01:01:04.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f810196e740 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fed596ed30 0x55fed5986890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f810196e740 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fed59ebe80 0x55fed59ee2a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f810196e740 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fed5962da0 0x55fed5986100 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f810196e740 1 --2- 172.21.15.78:0/2789153213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed5962480 0x55fed59775e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f80e0ff9640 0 client.15852 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:04.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f810196e740 1 -- 172.21.15.78:0/2789153213 >> 172.21.15.78:0/2789153213 conn(0x55fed59602b0 msgr2=0x55fed596d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:04.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f810196e740 1 -- 172.21.15.78:0/2789153213 shutdown_connections 2024-09-11T01:01:04.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f810196e740 1 -- 172.21.15.78:0/2789153213 wait complete. 2024-09-11T01:01:04.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.179+0000 7f810196e740 20 client.15852 trim_cache size 0 max 16384 2024-09-11T01:01:04.199 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:01:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:01:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:01:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:01:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:01:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:01:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:01:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:01:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:01:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:01:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:01:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:01:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:01:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:01:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:01:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:01:04.201 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:01:04.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.180+0000 7f80ff599640 1 -- 172.21.15.78:0/1096281422 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80e40b5450 msgr2=0x7f80e40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:04.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.180+0000 7f80ff599640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80e40b5450 0x7f80e40b7910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f80f0006010 tx=0x7f80f00099a0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.180+0000 7f80ff599640 1 -- 172.21.15.78:0/1096281422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f81045c0 msgr2=0x7f80f8101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:04.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.180+0000 7f80ff599640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f81045c0 0x7f80f8101e90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f80e800e9c0 tx=0x7f80e800ee80 comp rx=0 tx=0).stop 2024-09-11T01:01:04.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.180+0000 7f80ff599640 1 -- 172.21.15.78:0/1096281422 shutdown_connections 2024-09-11T01:01:04.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.180+0000 7f80ff599640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80e40b5450 0x7f80e40b7910 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.180+0000 7f80ff599640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80f810d100 0x7f80f81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.180+0000 7f80ff599640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f81045c0 0x7f80f8101e90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.180+0000 7f80ff599640 1 --2- 172.21.15.78:0/1096281422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f8103ca0 0x7f80f8101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:04.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.180+0000 7f80ff599640 1 -- 172.21.15.78:0/1096281422 >> 172.21.15.78:0/1096281422 conn(0x7f80f80fd700 msgr2=0x7f80f80fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:04.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.180+0000 7f80ff599640 1 -- 172.21.15.78:0/1096281422 shutdown_connections 2024-09-11T01:01:04.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:04.180+0000 7f80ff599640 1 -- 172.21.15.78:0/1096281422 wait complete. 2024-09-11T01:01:04.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 152545 11.69 MB/sec warmup 72 sec latency 13.085 ms 2024-09-11T01:01:05.105 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:01:05.105 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:01:05.392 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:05.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 153985 11.62 MB/sec warmup 73 sec latency 11.271 ms 2024-09-11T01:01:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:05 smithi110 ceph-mon[34712]: pgmap v389: 97 pgs: 97 active+clean; 89 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 325 op/s 2024-09-11T01:01:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:05 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:01:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:05 smithi112 ceph-mon[35384]: pgmap v389: 97 pgs: 97 active+clean; 89 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 325 op/s 2024-09-11T01:01:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:05 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:01:05.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:05 smithi078 ceph-mon[31315]: pgmap v389: 97 pgs: 97 active+clean; 89 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 325 op/s 2024-09-11T01:01:05.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:05 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:01:05.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.914+0000 7fc850871640 1 -- 172.21.15.78:0/3754726417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc84c108740 msgr2=0x7fc84c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:05.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.914+0000 7fc850871640 1 --2- 172.21.15.78:0/3754726417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc84c108740 0x7fc84c108b80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc834009900 tx=0x7fc83402a410 comp rx=0 tx=0).stop 2024-09-11T01:01:05.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.914+0000 7fc850871640 1 -- 172.21.15.78:0/3754726417 shutdown_connections 2024-09-11T01:01:05.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.914+0000 7fc850871640 1 --2- 172.21.15.78:0/3754726417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc84c1090c0 0x7fc84c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:05.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.914+0000 7fc850871640 1 --2- 172.21.15.78:0/3754726417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc84c108740 0x7fc84c108b80 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:05.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.914+0000 7fc850871640 1 --2- 172.21.15.78:0/3754726417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc84c101260 0x7fc84c101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:05.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.914+0000 7fc850871640 1 -- 172.21.15.78:0/3754726417 >> 172.21.15.78:0/3754726417 conn(0x7fc84c078060 msgr2=0x7fc84c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:05.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.914+0000 7fc850871640 1 -- 172.21.15.78:0/3754726417 shutdown_connections 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.914+0000 7fc850871640 1 -- 172.21.15.78:0/3754726417 wait complete. 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc850871640 1 Processor -- start 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc850871640 1 -- start start 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc850871640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc84c101260 0x7fc84c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc850871640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc84c108740 0x7fc84c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc850871640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc84c1090c0 0x7fc84c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc850871640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c112da0 con 0x7fc84c1090c0 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc850871640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c112c20 con 0x7fc84c108740 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc850871640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c112f20 con 0x7fc84c101260 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc84ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc84c1090c0 0x7fc84c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc84ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc84c1090c0 0x7fc84c19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60624/0 (socket says 172.21.15.78:60624) 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc84ad9d640 1 -- 172.21.15.78:0/1490808562 learned_addr learned my addr 172.21.15.78:0/1490808562 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc849d9b640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc84c108740 0x7fc84c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.915+0000 7fc84a59c640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc84c101260 0x7fc84c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.916+0000 7fc84ad9d640 1 -- 172.21.15.78:0/1490808562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc84c101260 msgr2=0x7fc84c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.916+0000 7fc84ad9d640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc84c101260 0x7fc84c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.916+0000 7fc84ad9d640 1 -- 172.21.15.78:0/1490808562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc84c108740 msgr2=0x7fc84c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.916+0000 7fc84ad9d640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc84c108740 0x7fc84c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.916+0000 7fc84ad9d640 1 -- 172.21.15.78:0/1490808562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84c19f320 con 0x7fc84c1090c0 2024-09-11T01:01:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.916+0000 7fc84ad9d640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc84c1090c0 0x7fc84c19eb50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc84400eb10 tx=0x7fc84400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.916+0000 7fc8437fe640 1 -- 172.21.15.78:0/1490808562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844004b20 con 0x7fc84c1090c0 2024-09-11T01:01:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.916+0000 7fc850871640 1 -- 172.21.15.78:0/1490808562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84c19f610 con 0x7fc84c1090c0 2024-09-11T01:01:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.916+0000 7fc8437fe640 1 -- 172.21.15.78:0/1490808562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc844004cc0 con 0x7fc84c1090c0 2024-09-11T01:01:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.916+0000 7fc8437fe640 1 -- 172.21.15.78:0/1490808562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844010670 con 0x7fc84c1090c0 2024-09-11T01:01:05.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.916+0000 7fc850871640 1 -- 172.21.15.78:0/1490808562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84c1a5cd0 con 0x7fc84c1090c0 2024-09-11T01:01:05.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.917+0000 7fc850871640 1 -- 172.21.15.78:0/1490808562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc84c101640 con 0x7fc84c1090c0 2024-09-11T01:01:05.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.919+0000 7fc8437fe640 1 -- 172.21.15.78:0/1490808562 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc844005000 con 0x7fc84c1090c0 2024-09-11T01:01:05.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.920+0000 7fc8437fe640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc8380b5450 0x7fc8380b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:05.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.920+0000 7fc84a59c640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc8380b5450 0x7fc8380b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:05.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.920+0000 7fc8437fe640 1 -- 172.21.15.78:0/1490808562 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc8440e6730 con 0x7fc84c1090c0 2024-09-11T01:01:05.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.921+0000 7fc84a59c640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc8380b5450 0x7fc8380b7910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009bd0 tx=0x7fc83c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:05.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:05.923+0000 7fc8437fe640 1 -- 172.21.15.78:0/1490808562 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc844056220 con 0x7fc84c1090c0 2024-09-11T01:01:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.179+0000 7fc850871640 1 -- 172.21.15.78:0/1490808562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc84c19a3a0 con 0x7fc84c1090c0 2024-09-11T01:01:06.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.180+0000 7fc8437fe640 1 -- 172.21.15.78:0/1490808562 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11986 (secure 0 0 0) 0x7fc8440ad320 con 0x7fc84c1090c0 2024-09-11T01:01:06.182 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:01:06.182 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-09-11T01:00:27:135338+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:00:27.099230+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T01:01:06.182 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-09-11T01:01:06.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.182+0000 7fc850871640 1 -- 172.21.15.78:0/1490808562 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc8380b5450 msgr2=0x7fc8380b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:06.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.182+0000 7fc850871640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc8380b5450 0x7fc8380b7910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009bd0 tx=0x7fc83c005f90 comp rx=0 tx=0).stop 2024-09-11T01:01:06.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.182+0000 7fc850871640 1 -- 172.21.15.78:0/1490808562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc84c1090c0 msgr2=0x7fc84c19eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:06.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.182+0000 7fc850871640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc84c1090c0 0x7fc84c19eb50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc84400eb10 tx=0x7fc84400c2d0 comp rx=0 tx=0).stop 2024-09-11T01:01:06.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.182+0000 7fc850871640 1 -- 172.21.15.78:0/1490808562 shutdown_connections 2024-09-11T01:01:06.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.182+0000 7fc850871640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc8380b5450 0x7fc8380b7910 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:06.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.182+0000 7fc850871640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc84c1090c0 0x7fc84c19eb50 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:06.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.182+0000 7fc850871640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc84c108740 0x7fc84c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:06.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.182+0000 7fc850871640 1 --2- 172.21.15.78:0/1490808562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc84c101260 0x7fc84c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:06.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.182+0000 7fc850871640 1 -- 172.21.15.78:0/1490808562 >> 172.21.15.78:0/1490808562 conn(0x7fc84c078060 msgr2=0x7fc84c105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:06.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.182+0000 7fc850871640 1 -- 172.21.15.78:0/1490808562 shutdown_connections 2024-09-11T01:01:06.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:06.182+0000 7fc850871640 1 -- 172.21.15.78:0/1490808562 wait complete. 2024-09-11T01:01:06.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 156394 11.65 MB/sec warmup 74 sec latency 21.995 ms 2024-09-11T01:01:06.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:06 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:01:06.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:06 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:01:06.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:06 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:01:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:06 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:01:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:06 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:01:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:06 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:01:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:06 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:01:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:06 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:01:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:06 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:01:06.914 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:01:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:07.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 158548 11.59 MB/sec warmup 75 sec latency 10.899 ms 2024-09-11T01:01:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:07 smithi110 ceph-mon[34712]: pgmap v390: 97 pgs: 97 active+clean; 90 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 322 op/s 2024-09-11T01:01:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:07 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1490808562' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:01:07.457 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:07 smithi078 ceph-mon[31315]: pgmap v390: 97 pgs: 97 active+clean; 90 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 322 op/s 2024-09-11T01:01:07.457 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:07 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1490808562' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:01:07.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:07 smithi112 ceph-mon[35384]: pgmap v390: 97 pgs: 97 active+clean; 90 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 322 op/s 2024-09-11T01:01:07.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:07 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1490808562' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:01:07.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.705+0000 7fe388f2c640 1 -- 172.21.15.78:0/4001984503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe384101c90 msgr2=0x7fe384102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:07.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.705+0000 7fe388f2c640 1 --2- 172.21.15.78:0/4001984503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe384101c90 0x7fe384102070 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe374009900 tx=0x7fe37402a410 comp rx=0 tx=0).stop 2024-09-11T01:01:07.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.705+0000 7fe388f2c640 1 -- 172.21.15.78:0/4001984503 shutdown_connections 2024-09-11T01:01:07.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.705+0000 7fe388f2c640 1 --2- 172.21.15.78:0/4001984503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe384108020 0x7fe38410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.705+0000 7fe388f2c640 1 --2- 172.21.15.78:0/4001984503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3841076a0 0x7fe384107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.705+0000 7fe388f2c640 1 --2- 172.21.15.78:0/4001984503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe384101c90 0x7fe384102070 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.705+0000 7fe388f2c640 1 -- 172.21.15.78:0/4001984503 >> 172.21.15.78:0/4001984503 conn(0x7fe3840fd700 msgr2=0x7fe3840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:07.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.705+0000 7fe388f2c640 1 -- 172.21.15.78:0/4001984503 shutdown_connections 2024-09-11T01:01:07.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.705+0000 7fe388f2c640 1 -- 172.21.15.78:0/4001984503 wait complete. 2024-09-11T01:01:07.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.706+0000 7fe388f2c640 1 Processor -- start 2024-09-11T01:01:07.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.706+0000 7fe388f2c640 1 -- start start 2024-09-11T01:01:07.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.706+0000 7fe388f2c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe384101c90 0x7fe384103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.706+0000 7fe388f2c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3841076a0 0x7fe384103a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.706+0000 7fe388f2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe384108020 0x7fe384103fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.706+0000 7fe388f2c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe384111ce0 con 0x7fe384101c90 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.706+0000 7fe388f2c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe384111b60 con 0x7fe3841076a0 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.706+0000 7fe388f2c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe384111e60 con 0x7fe384108020 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.706+0000 7fe38259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe384101c90 0x7fe384103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe379d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3841076a0 0x7fe384103a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe38259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe384101c90 0x7fe384103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60648/0 (socket says 172.21.15.78:60648) 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe38259c640 1 -- 172.21.15.78:0/2865358998 learned_addr learned my addr 172.21.15.78:0/2865358998 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe382d9d640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe384108020 0x7fe384103fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe382d9d640 1 -- 172.21.15.78:0/2865358998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3841076a0 msgr2=0x7fe384103a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe382d9d640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3841076a0 0x7fe384103a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe382d9d640 1 -- 172.21.15.78:0/2865358998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe384101c90 msgr2=0x7fe384103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe382d9d640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe384101c90 0x7fe384103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe382d9d640 1 -- 172.21.15.78:0/2865358998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3841a76d0 con 0x7fe384108020 2024-09-11T01:01:07.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe38259c640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe384101c90 0x7fe384103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe382d9d640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe384108020 0x7fe384103fd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009fd0 tx=0x7fe37c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe37bfff640 1 -- 172.21.15.78:0/2865358998 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00ce20 con 0x7fe384108020 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe37bfff640 1 -- 172.21.15.78:0/2865358998 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe37c005020 con 0x7fe384108020 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe37bfff640 1 -- 172.21.15.78:0/2865358998 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c010690 con 0x7fe384108020 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe388f2c640 1 -- 172.21.15.78:0/2865358998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3841a79c0 con 0x7fe384108020 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.707+0000 7fe388f2c640 1 -- 172.21.15.78:0/2865358998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3841a7f50 con 0x7fe384108020 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.710+0000 7fe37bfff640 1 -- 172.21.15.78:0/2865358998 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe37c0054d0 con 0x7fe384108020 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.710+0000 7fe38b301740 1 Processor -- start 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38b301740 1 -- start start 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38b301740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0c4d0 0x558fbfb0c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38b301740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0cdf0 0x558fbfb18840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38b301740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb18d80 0x558fbfb1b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38b301740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fbfa82ea0 con 0x558fbfb0cdf0 2024-09-11T01:01:07.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38b301740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558fbfa722d0 con 0x558fbfb0c4d0 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38b301740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558fbfa760e0 con 0x558fbfb18d80 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0cdf0 0x558fbfb18840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0cdf0 0x558fbfb18840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60652/0 (socket says 172.21.15.78:60652) 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38259c640 1 -- 172.21.15.78:0/3940310288 learned_addr learned my addr 172.21.15.78:0/3940310288 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe379d9b640 1 --2- 172.21.15.78:0/3940310288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb18d80 0x558fbfb1b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe379d9b640 1 --2- 172.21.15.78:0/3940310288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0c4d0 0x558fbfb0c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38259c640 1 -- 172.21.15.78:0/3940310288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb18d80 msgr2=0x558fbfb1b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38259c640 1 --2- 172.21.15.78:0/3940310288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb18d80 0x558fbfb1b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38259c640 1 -- 172.21.15.78:0/3940310288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0c4d0 msgr2=0x558fbfb0c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38259c640 1 --2- 172.21.15.78:0/3940310288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0c4d0 0x558fbfb0c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38259c640 1 -- 172.21.15.78:0/3940310288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fbf7d0650 con 0x558fbfb0cdf0 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe37bfff640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe3700b54d0 0x7fe3700b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe38259c640 1 --2- 172.21.15.78:0/3940310288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0cdf0 0x558fbfb18840 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe374000c00 tx=0x7fe374032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efae4e2dfa67eb49 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.711+0000 7fe379d9b640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe3700b54d0 0x7fe3700b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.712+0000 7fe378d99640 1 -- 172.21.15.78:0/3940310288 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374035040 con 0x558fbfb0cdf0 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.712+0000 7fe37bfff640 1 -- 172.21.15.78:0/2865358998 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe37c0e65b0 con 0x7fe384108020 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.712+0000 7fe378d99640 1 -- 172.21.15.78:0/3940310288 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3740317a0 con 0x558fbfb0cdf0 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.712+0000 7fe378d99640 1 -- 172.21.15.78:0/3940310288 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374005650 con 0x558fbfb0cdf0 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.713+0000 7fe38b301740 1 -- 172.21.15.78:0/3940310288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0cdf0 msgr2=0x558fbfb18840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.713+0000 7fe379d9b640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe3700b54d0 0x7fe3700b7990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009fd0 tx=0x7fe36c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.713+0000 7fe38b301740 1 --2- 172.21.15.78:0/3940310288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0cdf0 0x558fbfb18840 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe374000c00 tx=0x7fe374032ed0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.713+0000 7fe38b301740 1 -- 172.21.15.78:0/3940310288 shutdown_connections 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.713+0000 7fe38b301740 1 --2- 172.21.15.78:0/3940310288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb18d80 0x558fbfb1b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.713+0000 7fe38b301740 1 --2- 172.21.15.78:0/3940310288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0cdf0 0x558fbfb18840 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.713+0000 7fe38b301740 1 --2- 172.21.15.78:0/3940310288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0c4d0 0x558fbfb0c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.713+0000 7fe38b301740 1 -- 172.21.15.78:0/3940310288 >> 172.21.15.78:0/3940310288 conn(0x558fbfb0a300 msgr2=0x558fbfb0b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.713+0000 7fe38b301740 1 -- 172.21.15.78:0/3940310288 shutdown_connections 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.713+0000 7fe38b301740 1 -- 172.21.15.78:0/3940310288 wait complete. 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.713+0000 7fe38b301740 1 -- start start 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.713+0000 7fe38b301740 1 Processor -- start 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38b301740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0c4d0 0x558fbfb18690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38b301740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0cdf0 0x558fbfb15ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38b301740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb18d80 0x558fbfb16110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38b301740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fbf8d3700 con 0x558fbfb0c4d0 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38b301740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558fbfa0d160 con 0x558fbfb18d80 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38b301740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558fbfb1b9a0 con 0x558fbfb0cdf0 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38b301740 10 client.0 _ll_register_callbacks cb 0x558fbfa269e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0c4d0 0x558fbfb18690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0c4d0 0x558fbfb18690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60656/0 (socket says 172.21.15.78:60656) 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe379d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0cdf0 0x558fbfb15ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38259c640 1 -- 172.21.15.78:0/3651615355 learned_addr learned my addr 172.21.15.78:0/3651615355 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38259c640 1 -- 172.21.15.78:0/3651615355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0cdf0 msgr2=0x558fbfb15ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38259c640 1 --2- 172.21.15.78:0/3651615355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0cdf0 0x558fbfb15ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38259c640 1 -- 172.21.15.78:0/3651615355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb18d80 msgr2=0x558fbfb16110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe382d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb18d80 0x558fbfb16110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38259c640 1 --2- 172.21.15.78:0/3651615355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb18d80 0x558fbfb16110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe38259c640 1 -- 172.21.15.78:0/3651615355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fbfa3ac90 con 0x558fbfb0c4d0 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.714+0000 7fe382d9d640 1 --2- 172.21.15.78:0/3651615355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb18d80 0x558fbfb16110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe38259c640 1 --2- 172.21.15.78:0/3651615355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0c4d0 0x558fbfb18690 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe374004250 tx=0x7fe3740053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d378e3abfd763d20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe369ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe369ffb640 1 -- 172.21.15.78:0/3651615355 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374035040 con 0x558fbfb0c4d0 2024-09-11T01:01:07.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe369ffb640 1 -- 172.21.15.78:0/3651615355 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe3740055f0 con 0x558fbfb0c4d0 2024-09-11T01:01:07.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe38b301740 1 -- 172.21.15.78:0/3651615355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3740317a0 con 0x558fbfb0c4d0 2024-09-11T01:01:07.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe369ffb640 1 -- 172.21.15.78:0/3651615355 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe374059050 con 0x558fbfb0c4d0 2024-09-11T01:01:07.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe38b301740 10 client.15870 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:07.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe38b301740 1 -- 172.21.15.78:0/3651615355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558fbfb27330 con 0x558fbfb0c4d0 2024-09-11T01:01:07.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe369ffb640 1 -- 172.21.15.78:0/3651615355 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3740317a0 con 0x558fbfb0c4d0 2024-09-11T01:01:07.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe369ffb640 1 -- 172.21.15.78:0/3651615355 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fe374046570 con 0x558fbfb0c4d0 2024-09-11T01:01:07.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe38b301740 10 client.15870 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:07.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe38b301740 10 client.15870 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:01:07.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.715+0000 7fe38b301740 20 client.15870 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:07.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.716+0000 7fe38b301740 1 --2- 172.21.15.78:0/3651615355 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558fbfb28680 0x558fbfb2aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.716+0000 7fe38b301740 4 client.15870 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.716+0000 7fe38b301740 1 -- 172.21.15.78:0/3651615355 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fbfa84df0 con 0x558fbfb28680 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.716+0000 7fe379d9b640 1 --2- 172.21.15.78:0/3651615355 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558fbfb28680 0x558fbfb2aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.716+0000 7fe369ffb640 10 client.15870 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.716+0000 7fe379d9b640 1 --2- 172.21.15.78:0/3651615355 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558fbfb28680 0x558fbfb2aa70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ae444913d1251bba server_cookie=56a01ffefaaa1c2a in_seq=0 out_seq=0 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.717+0000 7fe369ffb640 1 -- 172.21.15.78:0/3651615355 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558fbfa84df0 con 0x558fbfb28680 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.717+0000 7fe369ffb640 10 client.15870 handle_command_reply: tid=0 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.717+0000 7fe38b301740 1 client.15870 shutdown 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.717+0000 7fe38b301740 1 -- 172.21.15.78:0/3651615355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0c4d0 msgr2=0x558fbfb18690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.717+0000 7fe38b301740 1 --2- 172.21.15.78:0/3651615355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0c4d0 0x558fbfb18690 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe374004250 tx=0x7fe3740053d0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.717+0000 7fe38b301740 1 -- 172.21.15.78:0/3651615355 shutdown_connections 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.717+0000 7fe38b301740 1 --2- 172.21.15.78:0/3651615355 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558fbfb28680 0x558fbfb2aa70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.717+0000 7fe38b301740 1 --2- 172.21.15.78:0/3651615355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb18d80 0x558fbfb16110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.717+0000 7fe38b301740 1 --2- 172.21.15.78:0/3651615355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0cdf0 0x558fbfb15ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.717+0000 7fe38b301740 1 --2- 172.21.15.78:0/3651615355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb0c4d0 0x558fbfb18690 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.717+0000 7fe38b301740 1 -- 172.21.15.78:0/3651615355 >> 172.21.15.78:0/3651615355 conn(0x558fbfb0a300 msgr2=0x558fbfb0fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:07.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.717+0000 7fe369ffb640 0 client.15870 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:01:07.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.718+0000 7fe38b301740 1 -- 172.21.15.78:0/3651615355 shutdown_connections 2024-09-11T01:01:07.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.718+0000 7fe38b301740 1 -- 172.21.15.78:0/3651615355 wait complete. 2024-09-11T01:01:07.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.718+0000 7fe38b301740 20 client.15870 trim_cache size 0 max 16384 2024-09-11T01:01:07.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.732+0000 7fe38b301740 1 Processor -- start 2024-09-11T01:01:07.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.732+0000 7fe38b301740 1 -- start start 2024-09-11T01:01:07.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.732+0000 7fe38b301740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0cdf0 0x558fbfb24c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.732+0000 7fe38b301740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb18d80 0x558fbfb12cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.732+0000 7fe38b301740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0bab0 0x558fbfb13230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.732+0000 7fe38b301740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe3841b0750 con 0x558fbfb18d80 2024-09-11T01:01:07.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.732+0000 7fe38b301740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe3841b2350 con 0x558fbfb0bab0 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.732+0000 7fe38b301740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe3841b0e20 con 0x558fbfb0cdf0 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.732+0000 7fe38259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0cdf0 0x558fbfb24c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.732+0000 7fe382d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0bab0 0x558fbfb13230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.732+0000 7fe379d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb18d80 0x558fbfb12cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.732+0000 7fe38259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0cdf0 0x558fbfb24c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35376/0 (socket says 172.21.15.78:35376) 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe38259c640 1 -- 172.21.15.78:0/2967824002 learned_addr learned my addr 172.21.15.78:0/2967824002 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe382d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0bab0 0x558fbfb13230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39488/0 (socket says 172.21.15.78:39488) 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe379d9b640 1 -- 172.21.15.78:0/2967824002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0cdf0 msgr2=0x558fbfb24c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe379d9b640 1 --2- 172.21.15.78:0/2967824002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0cdf0 0x558fbfb24c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe379d9b640 1 -- 172.21.15.78:0/2967824002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0bab0 msgr2=0x558fbfb13230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe379d9b640 1 --2- 172.21.15.78:0/2967824002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0bab0 0x558fbfb13230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe379d9b640 1 -- 172.21.15.78:0/2967824002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fbfb05bb0 con 0x558fbfb18d80 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe38259c640 1 --2- 172.21.15.78:0/2967824002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0cdf0 0x558fbfb24c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe382d9d640 1 --2- 172.21.15.78:0/2967824002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0bab0 0x558fbfb13230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:07.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe379d9b640 1 --2- 172.21.15.78:0/2967824002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb18d80 0x558fbfb12cf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x558fbfb2b040 tx=0x7fe36c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f1258365e8855e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:07.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe36bfff640 1 -- 172.21.15.78:0/2967824002 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c011c00 con 0x558fbfb18d80 2024-09-11T01:01:07.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe36bfff640 1 -- 172.21.15.78:0/2967824002 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe36c011da0 con 0x558fbfb18d80 2024-09-11T01:01:07.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.733+0000 7fe36bfff640 1 -- 172.21.15.78:0/2967824002 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c004d40 con 0x558fbfb18d80 2024-09-11T01:01:07.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.734+0000 7fe38b301740 1 -- 172.21.15.78:0/2967824002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb18d80 msgr2=0x558fbfb12cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:07.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.734+0000 7fe38b301740 1 --2- 172.21.15.78:0/2967824002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb18d80 0x558fbfb12cf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x558fbfb2b040 tx=0x7fe36c0046c0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.734+0000 7fe38b301740 1 -- 172.21.15.78:0/2967824002 shutdown_connections 2024-09-11T01:01:07.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.734+0000 7fe38b301740 1 --2- 172.21.15.78:0/2967824002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0bab0 0x558fbfb13230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.734+0000 7fe38b301740 1 --2- 172.21.15.78:0/2967824002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb18d80 0x558fbfb12cf0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.734+0000 7fe38b301740 1 --2- 172.21.15.78:0/2967824002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0cdf0 0x558fbfb24c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.734+0000 7fe38b301740 1 -- 172.21.15.78:0/2967824002 >> 172.21.15.78:0/2967824002 conn(0x558fbfb0a300 msgr2=0x558fbfb0b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.734+0000 7fe38b301740 1 -- 172.21.15.78:0/2967824002 shutdown_connections 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.734+0000 7fe38b301740 1 -- 172.21.15.78:0/2967824002 wait complete. 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.734+0000 7fe38b301740 1 -- start start 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe38b301740 1 Processor -- start 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe38b301740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0bab0 0x558fbfb1d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe38b301740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0cdf0 0x558fbfb1d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe38b301740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb18d80 0x558fbfb942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe38b301740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe3841b1bc0 con 0x558fbfb18d80 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe38b301740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe3841b14f0 con 0x558fbfb0cdf0 2024-09-11T01:01:07.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe38b301740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe3841ae730 con 0x558fbfb0bab0 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe38b301740 10 client.0 _ll_register_callbacks cb 0x7fe374040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe379d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0cdf0 0x558fbfb1d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe379d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0cdf0 0x558fbfb1d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39490/0 (socket says 172.21.15.78:39490) 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe379d9b640 1 -- 172.21.15.78:0/2013704851 learned_addr learned my addr 172.21.15.78:0/2013704851 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe38259c640 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0bab0 0x558fbfb1d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe382d9d640 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb18d80 0x558fbfb942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe379d9b640 1 -- 172.21.15.78:0/2013704851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0bab0 msgr2=0x558fbfb1d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe379d9b640 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0bab0 0x558fbfb1d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe379d9b640 1 -- 172.21.15.78:0/2013704851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb18d80 msgr2=0x558fbfb942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe379d9b640 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb18d80 0x558fbfb942d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe379d9b640 1 -- 172.21.15.78:0/2013704851 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fbfb94890 con 0x558fbfb0cdf0 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.735+0000 7fe38259c640 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0bab0 0x558fbfb1d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.736+0000 7fe379d9b640 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0cdf0 0x558fbfb1d860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe36c011fd0 tx=0x7fe36c009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad2b9084d98e9444 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:07.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.736+0000 7fe369ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.736+0000 7fe369ffb640 1 -- 172.21.15.78:0/2013704851 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c011c00 con 0x558fbfb0cdf0 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.736+0000 7fe369ffb640 1 -- 172.21.15.78:0/2013704851 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe36c035440 con 0x558fbfb0cdf0 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.736+0000 7fe38b301740 1 -- 172.21.15.78:0/2013704851 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36c011da0 con 0x558fbfb0cdf0 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.736+0000 7fe369ffb640 1 -- 172.21.15.78:0/2013704851 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe36c035640 con 0x558fbfb0cdf0 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.736+0000 7fe38b301740 10 client.25280 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.736+0000 7fe38b301740 1 -- 172.21.15.78:0/2013704851 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558fbfb95b90 con 0x558fbfb0cdf0 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.736+0000 7fe369ffb640 1 -- 172.21.15.78:0/2013704851 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe36c011da0 con 0x558fbfb0cdf0 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe369ffb640 1 -- 172.21.15.78:0/2013704851 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fe36c03ea00 con 0x558fbfb0cdf0 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe38b301740 10 client.25280 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe38b301740 10 client.25280 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe38b301740 20 client.25280 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe38b301740 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558fbfb961f0 0x558fbfb98610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:07.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe38b301740 4 client.25280 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe38b301740 1 -- 172.21.15.78:0/2013704851 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558fbfa808a0 con 0x558fbfb961f0 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe38259c640 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558fbfb961f0 0x558fbfb98610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe369ffb640 10 client.25280 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe38259c640 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558fbfb961f0 0x558fbfb98610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da47e51d6309c36b server_cookie=c56732bbdecdf7ac in_seq=0 out_seq=0 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe369ffb640 1 -- 172.21.15.78:0/2013704851 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558fbfa808a0 con 0x558fbfb961f0 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe369ffb640 10 client.25280 handle_command_reply: tid=0 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.737+0000 7fe38b301740 1 client.25280 shutdown 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe38b301740 1 -- 172.21.15.78:0/2013704851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0cdf0 msgr2=0x558fbfb1d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe38b301740 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0cdf0 0x558fbfb1d860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe36c011fd0 tx=0x7fe36c009900 comp rx=0 tx=0).stop 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe38b301740 1 -- 172.21.15.78:0/2013704851 shutdown_connections 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe38b301740 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558fbfb961f0 0x558fbfb98610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe38b301740 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbfb18d80 0x558fbfb942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe38b301740 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558fbfb0cdf0 0x558fbfb1d860 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe38b301740 1 --2- 172.21.15.78:0/2013704851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558fbfb0bab0 0x558fbfb1d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe369ffb640 0 client.25280 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:01:07.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe38b301740 1 -- 172.21.15.78:0/2013704851 >> 172.21.15.78:0/2013704851 conn(0x558fbfb0a300 msgr2=0x558fbfb14360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:07.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe38b301740 1 -- 172.21.15.78:0/2013704851 shutdown_connections 2024-09-11T01:01:07.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe38b301740 1 -- 172.21.15.78:0/2013704851 wait complete. 2024-09-11T01:01:07.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe38b301740 20 client.25280 trim_cache size 0 max 16384 2024-09-11T01:01:07.745 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:01:07.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe388f2c640 1 -- 172.21.15.78:0/2865358998 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe3700b54d0 msgr2=0x7fe3700b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:07.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe388f2c640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe3700b54d0 0x7fe3700b7990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009fd0 tx=0x7fe36c005f90 comp rx=0 tx=0).stop 2024-09-11T01:01:07.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe388f2c640 1 -- 172.21.15.78:0/2865358998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe384108020 msgr2=0x7fe384103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:07.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.738+0000 7fe388f2c640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe384108020 0x7fe384103fd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009fd0 tx=0x7fe37c00ef10 comp rx=0 tx=0).stop 2024-09-11T01:01:07.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.739+0000 7fe388f2c640 1 -- 172.21.15.78:0/2865358998 shutdown_connections 2024-09-11T01:01:07.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.739+0000 7fe388f2c640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe3700b54d0 0x7fe3700b7990 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.739+0000 7fe388f2c640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe384108020 0x7fe384103fd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.739+0000 7fe388f2c640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3841076a0 0x7fe384103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.739+0000 7fe388f2c640 1 --2- 172.21.15.78:0/2865358998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe384101c90 0x7fe384103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:07.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.739+0000 7fe388f2c640 1 -- 172.21.15.78:0/2865358998 >> 172.21.15.78:0/2865358998 conn(0x7fe3840fd700 msgr2=0x7fe38406ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:07.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.739+0000 7fe388f2c640 1 -- 172.21.15.78:0/2865358998 shutdown_connections 2024-09-11T01:01:07.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:07.739+0000 7fe388f2c640 1 -- 172.21.15.78:0/2865358998 wait complete. 2024-09-11T01:01:08.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 160318 11.60 MB/sec warmup 76 sec latency 23.714 ms 2024-09-11T01:01:08.585 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:01:08.853 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:09 smithi078 ceph-mon[31315]: pgmap v391: 97 pgs: 97 active+clean; 92 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 314 op/s 2024-09-11T01:01:09.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.359+0000 7f5e10876640 1 -- 172.21.15.78:0/2642873790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e0c102150 msgr2=0x7f5e0c10eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:09.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.359+0000 7f5e10876640 1 --2- 172.21.15.78:0/2642873790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e0c102150 0x7f5e0c10eea0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009900 tx=0x7f5e0002a480 comp rx=0 tx=0).stop 2024-09-11T01:01:09.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.359+0000 7f5e10876640 1 -- 172.21.15.78:0/2642873790 shutdown_connections 2024-09-11T01:01:09.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.359+0000 7f5e10876640 1 --2- 172.21.15.78:0/2642873790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e0c10f3e0 0x7f5e0c1117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.359+0000 7f5e10876640 1 --2- 172.21.15.78:0/2642873790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e0c102150 0x7f5e0c10eea0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.359+0000 7f5e10876640 1 --2- 172.21.15.78:0/2642873790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e0c101830 0x7f5e0c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.359+0000 7f5e10876640 1 -- 172.21.15.78:0/2642873790 >> 172.21.15.78:0/2642873790 conn(0x7f5e0c0fd700 msgr2=0x7f5e0c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:09.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.359+0000 7f5e10876640 1 -- 172.21.15.78:0/2642873790 shutdown_connections 2024-09-11T01:01:09.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.359+0000 7f5e10876640 1 -- 172.21.15.78:0/2642873790 wait complete. 2024-09-11T01:01:09.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e10876640 1 Processor -- start 2024-09-11T01:01:09.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e10876640 1 -- start start 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e10876640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e0c101830 0x7f5e0c19b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e10876640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e0c102150 0x7f5e0c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e10876640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e0c10f3e0 0x7f5e0c1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e10876640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c114f40 con 0x7f5e0c10f3e0 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e10876640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c114dc0 con 0x7f5e0c101830 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e10876640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1150c0 con 0x7f5e0c102150 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e0ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e0c10f3e0 0x7f5e0c1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e0ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e0c10f3e0 0x7f5e0c1a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60702/0 (socket says 172.21.15.78:60702) 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e0ad9d640 1 -- 172.21.15.78:0/648754902 learned_addr learned my addr 172.21.15.78:0/648754902 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e0a59c640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e0c101830 0x7f5e0c19b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.360+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e0c102150 0x7f5e0c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.361+0000 7f5e0ad9d640 1 -- 172.21.15.78:0/648754902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e0c102150 msgr2=0x7f5e0c19ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:09.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.361+0000 7f5e0ad9d640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e0c102150 0x7f5e0c19ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.361+0000 7f5e0ad9d640 1 -- 172.21.15.78:0/648754902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e0c101830 msgr2=0x7f5e0c19b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.361+0000 7f5e0ad9d640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e0c101830 0x7f5e0c19b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.361+0000 7f5e0ad9d640 1 -- 172.21.15.78:0/648754902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c1a15a0 con 0x7f5e0c10f3e0 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.361+0000 7f5e0ad9d640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e0c10f3e0 0x7f5e0c1a0dd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400eb10 tx=0x7f5e0400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.361+0000 7f5dfb7fe640 1 -- 172.21.15.78:0/648754902 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04004b20 con 0x7f5e0c10f3e0 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.361+0000 7f5e10876640 1 -- 172.21.15.78:0/648754902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c1a1890 con 0x7f5e0c10f3e0 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.361+0000 7f5e10876640 1 -- 172.21.15.78:0/648754902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c1a7f50 con 0x7f5e0c10f3e0 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.361+0000 7f5dfb7fe640 1 -- 172.21.15.78:0/648754902 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e04004cc0 con 0x7f5e0c10f3e0 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.361+0000 7f5dfb7fe640 1 -- 172.21.15.78:0/648754902 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04010690 con 0x7f5e0c10f3e0 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5e12c4b740 1 Processor -- start 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5dfb7fe640 1 -- 172.21.15.78:0/648754902 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5e04010830 con 0x7f5e0c10f3e0 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5e12c4b740 1 -- start start 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5e12c4b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf34d0 0x56095abf38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5e12c4b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 0x56095abff840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5e12c4b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abffd80 0x56095ac02170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5e12c4b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56095ab69ea0 con 0x56095abf3df0 2024-09-11T01:01:09.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5e12c4b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56095ab592d0 con 0x56095abffd80 2024-09-11T01:01:09.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5e12c4b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56095ab5d0e0 con 0x56095abf34d0 2024-09-11T01:01:09.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5e0a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 0x56095abff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5e0a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 0x56095abff840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60710/0 (socket says 172.21.15.78:60710) 2024-09-11T01:01:09.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5e0a59c640 1 -- 172.21.15.78:0/386202199 learned_addr learned my addr 172.21.15.78:0/386202199 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:09.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.364+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/386202199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abffd80 0x56095ac02170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/386202199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf34d0 0x56095abf38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5e0a59c640 1 -- 172.21.15.78:0/386202199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf34d0 msgr2=0x56095abf38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:09.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5e0a59c640 1 --2- 172.21.15.78:0/386202199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf34d0 0x56095abf38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5e0a59c640 1 -- 172.21.15.78:0/386202199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abffd80 msgr2=0x56095ac02170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:09.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5e0a59c640 1 --2- 172.21.15.78:0/386202199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abffd80 0x56095ac02170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5e0a59c640 1 -- 172.21.15.78:0/386202199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56095a8b7650 con 0x56095abf3df0 2024-09-11T01:01:09.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5e0a59c640 1 --2- 172.21.15.78:0/386202199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 0x56095abff840 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009d20 tx=0x7f5dfc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ed05ae766ea64e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:09.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5dfb7fe640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5df40b54d0 0x7f5df40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5df8ff9640 1 -- 172.21.15.78:0/386202199 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc01e070 con 0x56095abf3df0 2024-09-11T01:01:09.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5df8ff9640 1 -- 172.21.15.78:0/386202199 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dfc009e10 con 0x56095abf3df0 2024-09-11T01:01:09.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5df8ff9640 1 -- 172.21.15.78:0/386202199 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0187a0 con 0x56095abf3df0 2024-09-11T01:01:09.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5df40b54d0 0x7f5df40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.365+0000 7f5dfb7fe640 1 -- 172.21.15.78:0/648754902 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5e040e63d0 con 0x7f5e0c10f3e0 2024-09-11T01:01:09.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.366+0000 7f5e12c4b740 1 -- 172.21.15.78:0/386202199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 msgr2=0x56095abff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:09.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.366+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/386202199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 0x56095abff840 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009d20 tx=0x7f5dfc011810 comp rx=0 tx=0).stop 2024-09-11T01:01:09.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.366+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5df40b54d0 0x7f5df40b7990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5e00031440 tx=0x7f5e00005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.366+0000 7f5e12c4b740 1 -- 172.21.15.78:0/386202199 shutdown_connections 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.366+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/386202199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abffd80 0x56095ac02170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.366+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/386202199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 0x56095abff840 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.366+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/386202199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf34d0 0x56095abf38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.366+0000 7f5e12c4b740 1 -- 172.21.15.78:0/386202199 >> 172.21.15.78:0/386202199 conn(0x56095abf1300 msgr2=0x56095abf24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.366+0000 7f5e12c4b740 1 -- 172.21.15.78:0/386202199 shutdown_connections 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.366+0000 7f5e12c4b740 1 -- 172.21.15.78:0/386202199 wait complete. 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.366+0000 7f5e12c4b740 1 -- start start 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e12c4b740 1 Processor -- start 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e12c4b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095abff550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e12c4b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 0x56095abfc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e12c4b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abffd80 0x56095abfcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e12c4b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56095a9ba700 con 0x56095abffd80 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e12c4b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56095aaf4160 con 0x56095abf34d0 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e12c4b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56095ac02a00 con 0x56095abf3df0 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e12c4b740 10 client.0 _ll_register_callbacks cb 0x56095ab0d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e09d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 0x56095abfc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e0a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095abff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e09d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 0x56095abfc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35420/0 (socket says 172.21.15.78:35420) 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e09d9b640 1 -- 172.21.15.78:0/3165806235 learned_addr learned my addr 172.21.15.78:0/3165806235 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e0ad9d640 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abffd80 0x56095abfcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e0a59c640 1 -- 172.21.15.78:0/3165806235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 msgr2=0x56095abfc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e0a59c640 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 0x56095abfc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e0a59c640 1 -- 172.21.15.78:0/3165806235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abffd80 msgr2=0x56095abfcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e0a59c640 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abffd80 0x56095abfcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e0a59c640 1 -- 172.21.15.78:0/3165806235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56095ab21c90 con 0x56095abf34d0 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.367+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 0x56095abfc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5e0a59c640 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095abff550 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc01c040 tx=0x7f5dfc018c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d29048c0b30acaf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5df1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5df1ffb640 1 -- 172.21.15.78:0/3165806235 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc01e040 con 0x56095abf34d0 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5e12c4b740 1 -- 172.21.15.78:0/3165806235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dfc009e10 con 0x56095abf34d0 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5df1ffb640 1 -- 172.21.15.78:0/3165806235 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5dfc018df0 con 0x56095abf34d0 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5df1ffb640 1 -- 172.21.15.78:0/3165806235 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5dfc005c50 con 0x56095abf34d0 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5e12c4b740 10 client.25292 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5e12c4b740 1 -- 172.21.15.78:0/3165806235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56095ac0e2a0 con 0x56095abf34d0 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5df1ffb640 1 -- 172.21.15.78:0/3165806235 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dfc009e10 con 0x56095abf34d0 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5df1ffb640 1 -- 172.21.15.78:0/3165806235 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f5dfc021070 con 0x56095abf34d0 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5e12c4b740 10 client.25292 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5e12c4b740 10 client.25292 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:01:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5e12c4b740 20 client.25292 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:09.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56095ac0f650 0x56095ac11a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5e12c4b740 4 client.25292 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:01:09.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.368+0000 7f5e12c4b740 1 -- 172.21.15.78:0/3165806235 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56095ab6bdf0 con 0x56095ac0f650 2024-09-11T01:01:09.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.369+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56095ac0f650 0x56095ac11a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.369+0000 7f5df1ffb640 10 client.25292 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:01:09.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.369+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56095ac0f650 0x56095ac11a40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5bc02e7ddd594f22 server_cookie=75875cabbd8e8e85 in_seq=0 out_seq=0 2024-09-11T01:01:09.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5df1ffb640 1 -- 172.21.15.78:0/3165806235 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56095ab6bdf0 con 0x56095ac0f650 2024-09-11T01:01:09.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5df1ffb640 10 client.25292 handle_command_reply: tid=0 2024-09-11T01:01:09.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5e12c4b740 1 client.25292 shutdown 2024-09-11T01:01:09.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5e12c4b740 1 -- 172.21.15.78:0/3165806235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 msgr2=0x56095abff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:09.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095abff550 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc01c040 tx=0x7f5dfc018c00 comp rx=0 tx=0).stop 2024-09-11T01:01:09.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5e12c4b740 1 -- 172.21.15.78:0/3165806235 shutdown_connections 2024-09-11T01:01:09.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56095ac0f650 0x56095ac11a40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abffd80 0x56095abfcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 0x56095abfc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/3165806235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095abff550 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5df1ffb640 0 client.25292 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:01:09.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5e12c4b740 1 -- 172.21.15.78:0/3165806235 >> 172.21.15.78:0/3165806235 conn(0x56095abf1300 msgr2=0x56095abf6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:09.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5e12c4b740 1 -- 172.21.15.78:0/3165806235 shutdown_connections 2024-09-11T01:01:09.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5e12c4b740 1 -- 172.21.15.78:0/3165806235 wait complete. 2024-09-11T01:01:09.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.370+0000 7f5e12c4b740 20 client.25292 trim_cache size 0 max 16384 2024-09-11T01:01:09.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e12c4b740 1 Processor -- start 2024-09-11T01:01:09.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e12c4b740 1 -- start start 2024-09-11T01:01:09.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e12c4b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095abf0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e12c4b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 0x56095abf2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e12c4b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abffd80 0x56095abf2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e12c4b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec00e620 con 0x56095abf3df0 2024-09-11T01:01:09.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e12c4b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec010220 con 0x56095abf34d0 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e12c4b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec00ecf0 con 0x56095abffd80 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e09d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 0x56095abf2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e09d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 0x56095abf2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60736/0 (socket says 172.21.15.78:60736) 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e09d9b640 1 -- 172.21.15.78:0/2013060000 learned_addr learned my addr 172.21.15.78:0/2013060000 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e0a59c640 1 --2- 172.21.15.78:0/2013060000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095abf0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e0ad9d640 1 --2- 172.21.15.78:0/2013060000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abffd80 0x56095abf2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e09d9b640 1 -- 172.21.15.78:0/2013060000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abffd80 msgr2=0x56095abf2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/2013060000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abffd80 0x56095abf2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e09d9b640 1 -- 172.21.15.78:0/2013060000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 msgr2=0x56095abf0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/2013060000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095abf0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.392+0000 7f5e09d9b640 1 -- 172.21.15.78:0/2013060000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56095abf1d80 con 0x56095abf3df0 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.393+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/2013060000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 0x56095abf2350 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5e00000c00 tx=0x7f5e00009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d447fede355429 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.393+0000 7f5df3fff640 1 -- 172.21.15.78:0/2013060000 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00035040 con 0x56095abf3df0 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.393+0000 7f5df3fff640 1 -- 172.21.15.78:0/2013060000 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e00003aa0 con 0x56095abf3df0 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.393+0000 7f5df3fff640 1 -- 172.21.15.78:0/2013060000 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00053b20 con 0x56095abf3df0 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2013060000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 msgr2=0x56095abf2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/2013060000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 0x56095abf2350 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5e00000c00 tx=0x7f5e00009260 comp rx=0 tx=0).stop 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2013060000 shutdown_connections 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/2013060000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abffd80 0x56095abf2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/2013060000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abf3df0 0x56095abf2350 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/2013060000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095abf0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2013060000 >> 172.21.15.78:0/2013060000 conn(0x56095abf1300 msgr2=0x56095abf7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2013060000 shutdown_connections 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2013060000 wait complete. 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 -- start start 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 Processor -- start 2024-09-11T01:01:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095ac03f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 0x56095ac044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abffd80 0x56095ac7b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec00fa90 con 0x56095abffd80 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec00f3c0 con 0x56095abf34d0 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec00c600 con 0x56095abf3df0 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.394+0000 7f5e12c4b740 10 client.0 _ll_register_callbacks cb 0x7f5dfc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5e09d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 0x56095ac044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5e0ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abffd80 0x56095ac7b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5e09d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 0x56095ac044d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35440/0 (socket says 172.21.15.78:35440) 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5e09d9b640 1 -- 172.21.15.78:0/2095084181 learned_addr learned my addr 172.21.15.78:0/2095084181 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5e0a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095ac03f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5e09d9b640 1 -- 172.21.15.78:0/2095084181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 msgr2=0x56095ac03f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:09.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/2095084181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095ac03f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5e09d9b640 1 -- 172.21.15.78:0/2095084181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abffd80 msgr2=0x56095ac7b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/2095084181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abffd80 0x56095ac7b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5e09d9b640 1 -- 172.21.15.78:0/2095084181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56095ac7b9e0 con 0x56095abf3df0 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5e09d9b640 1 --2- 172.21.15.78:0/2095084181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 0x56095ac044d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5e00041040 tx=0x7f5e00031a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=889c96339e221d0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5df1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5df1ffb640 1 -- 172.21.15.78:0/2095084181 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00035040 con 0x56095abf3df0 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2095084181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e00003aa0 con 0x56095abf3df0 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5df1ffb640 1 -- 172.21.15.78:0/2095084181 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5e00050040 con 0x56095abf3df0 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.395+0000 7f5df1ffb640 1 -- 172.21.15.78:0/2095084181 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5e00003900 con 0x56095abf3df0 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5e12c4b740 10 client.25285 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2095084181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56095ac7caf0 con 0x56095abf3df0 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5df1ffb640 1 -- 172.21.15.78:0/2095084181 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e00003aa0 con 0x56095abf3df0 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5df1ffb640 1 -- 172.21.15.78:0/2095084181 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f5e00004830 con 0x56095abf3df0 2024-09-11T01:01:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5e12c4b740 10 client.25285 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:09.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5e12c4b740 10 client.25285 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:01:09.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5e12c4b740 20 client.25285 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:09.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/2095084181 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56095ac7d290 0x56095ac7f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:09.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5e12c4b740 4 client.25285 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:01:09.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2095084181 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56095ab678a0 con 0x56095ac7d290 2024-09-11T01:01:09.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5e0a59c640 1 --2- 172.21.15.78:0/2095084181 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56095ac7d290 0x56095ac7f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:09.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5df1ffb640 10 client.25285 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:01:09.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.396+0000 7f5e0a59c640 1 --2- 172.21.15.78:0/2095084181 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56095ac7d290 0x56095ac7f6b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3fab49880e24df91 server_cookie=1b8e79a8df7103c5 in_seq=0 out_seq=0 2024-09-11T01:01:09.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5df1ffb640 1 -- 172.21.15.78:0/2095084181 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56095ab678a0 con 0x56095ac7d290 2024-09-11T01:01:09.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5df1ffb640 10 client.25285 handle_command_reply: tid=0 2024-09-11T01:01:09.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5e12c4b740 1 client.25285 shutdown 2024-09-11T01:01:09.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2095084181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 msgr2=0x56095ac044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:09.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/2095084181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 0x56095ac044d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5e00041040 tx=0x7f5e00031a10 comp rx=0 tx=0).stop 2024-09-11T01:01:09.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2095084181 shutdown_connections 2024-09-11T01:01:09.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/2095084181 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56095ac7d290 0x56095ac7f6b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/2095084181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56095abffd80 0x56095ac7b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5df1ffb640 0 client.25285 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:01:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/2095084181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56095abf3df0 0x56095ac044d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5e12c4b740 1 --2- 172.21.15.78:0/2095084181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56095abf34d0 0x56095ac03f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2095084181 >> 172.21.15.78:0/2095084181 conn(0x56095abf1300 msgr2=0x56095abf2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2095084181 shutdown_connections 2024-09-11T01:01:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5e12c4b740 1 -- 172.21.15.78:0/2095084181 wait complete. 2024-09-11T01:01:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.397+0000 7f5e12c4b740 20 client.25285 trim_cache size 0 max 16384 2024-09-11T01:01:09.405 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:01:09.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.398+0000 7f5e10876640 1 -- 172.21.15.78:0/648754902 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5df40b54d0 msgr2=0x7f5df40b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:09.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.398+0000 7f5e10876640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5df40b54d0 0x7f5df40b7990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5e00031440 tx=0x7f5e00005b50 comp rx=0 tx=0).stop 2024-09-11T01:01:09.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.398+0000 7f5e10876640 1 -- 172.21.15.78:0/648754902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e0c10f3e0 msgr2=0x7f5e0c1a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:09.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.398+0000 7f5e10876640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e0c10f3e0 0x7f5e0c1a0dd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400eb10 tx=0x7f5e0400c2d0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.398+0000 7f5e10876640 1 -- 172.21.15.78:0/648754902 shutdown_connections 2024-09-11T01:01:09.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.398+0000 7f5e10876640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5df40b54d0 0x7f5df40b7990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.398+0000 7f5e10876640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e0c10f3e0 0x7f5e0c1a0dd0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.398+0000 7f5e10876640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e0c102150 0x7f5e0c19ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.398+0000 7f5e10876640 1 --2- 172.21.15.78:0/648754902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e0c101830 0x7f5e0c19b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:09.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.398+0000 7f5e10876640 1 -- 172.21.15.78:0/648754902 >> 172.21.15.78:0/648754902 conn(0x7f5e0c0fd700 msgr2=0x7f5e0c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:09.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.398+0000 7f5e10876640 1 -- 172.21.15.78:0/648754902 shutdown_connections 2024-09-11T01:01:09.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:09.398+0000 7f5e10876640 1 -- 172.21.15.78:0/648754902 wait complete. 2024-09-11T01:01:09.419 INFO:tasks.workunit.client.0.smithi078.stdout: 1 163016 11.64 MB/sec warmup 77 sec latency 17.243 ms 2024-09-11T01:01:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:09 smithi110 ceph-mon[34712]: pgmap v391: 97 pgs: 97 active+clean; 92 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 314 op/s 2024-09-11T01:01:09.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:09 smithi112 ceph-mon[35384]: pgmap v391: 97 pgs: 97 active+clean; 92 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 314 op/s 2024-09-11T01:01:10.227 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:01:10.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 164715 11.61 MB/sec warmup 78 sec latency 14.353 ms 2024-09-11T01:01:10.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:10 smithi110 ceph-mon[34712]: pgmap v392: 97 pgs: 97 active+clean; 95 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-09-11T01:01:10.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:10 smithi112 ceph-mon[35384]: pgmap v392: 97 pgs: 97 active+clean; 95 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-09-11T01:01:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:10 smithi078 ceph-mon[31315]: pgmap v392: 97 pgs: 97 active+clean; 95 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-09-11T01:01:10.508 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:11.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.040+0000 7f0261414640 1 -- 172.21.15.78:0/4220371366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f025c103cc0 msgr2=0x7f025c1040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:11.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.040+0000 7f0261414640 1 --2- 172.21.15.78:0/4220371366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f025c103cc0 0x7f025c1040a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f024c009900 tx=0x7f024c02a410 comp rx=0 tx=0).stop 2024-09-11T01:01:11.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.040+0000 7f0261414640 1 -- 172.21.15.78:0/4220371366 shutdown_connections 2024-09-11T01:01:11.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.040+0000 7f0261414640 1 --2- 172.21.15.78:0/4220371366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c10d120 0x7f025c10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.040+0000 7f0261414640 1 --2- 172.21.15.78:0/4220371366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f025c1045e0 0x7f025c10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.040+0000 7f0261414640 1 --2- 172.21.15.78:0/4220371366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f025c103cc0 0x7f025c1040a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.040+0000 7f0261414640 1 -- 172.21.15.78:0/4220371366 >> 172.21.15.78:0/4220371366 conn(0x7f025c0fd740 msgr2=0x7f025c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:11.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.040+0000 7f0261414640 1 -- 172.21.15.78:0/4220371366 shutdown_connections 2024-09-11T01:01:11.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.040+0000 7f0261414640 1 -- 172.21.15.78:0/4220371366 wait complete. 2024-09-11T01:01:11.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.040+0000 7f0261414640 1 Processor -- start 2024-09-11T01:01:11.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.041+0000 7f0261414640 1 -- start start 2024-09-11T01:01:11.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.041+0000 7f0261414640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c103cc0 0x7f025c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.041+0000 7f0261414640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f025c1045e0 0x7f025c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.041+0000 7f0261414640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f025c10d120 0x7f025c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.041+0000 7f0261414640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f025c112d90 con 0x7f025c10d120 2024-09-11T01:01:11.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.041+0000 7f0261414640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f025c112c10 con 0x7f025c103cc0 2024-09-11T01:01:11.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.041+0000 7f0261414640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f025c112f10 con 0x7f025c1045e0 2024-09-11T01:01:11.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.041+0000 7f025b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f025c10d120 0x7f025c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.041+0000 7f025b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f025c10d120 0x7f025c19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60766/0 (socket says 172.21.15.78:60766) 2024-09-11T01:01:11.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.041+0000 7f025b7fe640 1 -- 172.21.15.78:0/2132380663 learned_addr learned my addr 172.21.15.78:0/2132380663 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.041+0000 7f025affd640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c103cc0 0x7f025c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.041+0000 7f02527fc640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f025c1045e0 0x7f025c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.042+0000 7f025b7fe640 1 -- 172.21.15.78:0/2132380663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f025c1045e0 msgr2=0x7f025c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.042+0000 7f025b7fe640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f025c1045e0 0x7f025c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.042+0000 7f025b7fe640 1 -- 172.21.15.78:0/2132380663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c103cc0 msgr2=0x7f025c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.042+0000 7f025b7fe640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c103cc0 0x7f025c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.042+0000 7f025b7fe640 1 -- 172.21.15.78:0/2132380663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c19f330 con 0x7f025c10d120 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.042+0000 7f025affd640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c103cc0 0x7f025c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.042+0000 7f025b7fe640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f025c10d120 0x7f025c19eb60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f025400eaa0 tx=0x7f025400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.042+0000 7f0258ff9640 1 -- 172.21.15.78:0/2132380663 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025400cdb0 con 0x7f025c10d120 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.042+0000 7f0258ff9640 1 -- 172.21.15.78:0/2132380663 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0254005020 con 0x7f025c10d120 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.042+0000 7f0258ff9640 1 -- 172.21.15.78:0/2132380663 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02540116d0 con 0x7f025c10d120 2024-09-11T01:01:11.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.042+0000 7f0261414640 1 -- 172.21.15.78:0/2132380663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c19f620 con 0x7f025c10d120 2024-09-11T01:01:11.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.042+0000 7f0261414640 1 -- 172.21.15.78:0/2132380663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c1a5ce0 con 0x7f025c10d120 2024-09-11T01:01:11.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.045+0000 7f0258ff9640 1 -- 172.21.15.78:0/2132380663 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f025400f040 con 0x7f025c10d120 2024-09-11T01:01:11.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.045+0000 7f02637e9740 1 Processor -- start 2024-09-11T01:01:11.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.045+0000 7f02637e9740 1 -- start start 2024-09-11T01:01:11.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.045+0000 7f02637e9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585300784d0 0x5585300788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.045+0000 7f02637e9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530078df0 0x558530084840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.045+0000 7f02637e9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530084d80 0x558530087170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.045+0000 7f02637e9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55852ffeeea0 con 0x558530084d80 2024-09-11T01:01:11.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.045+0000 7f02637e9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55852ffde2d0 con 0x558530078df0 2024-09-11T01:01:11.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.045+0000 7f02637e9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55852ffe20e0 con 0x5585300784d0 2024-09-11T01:01:11.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.045+0000 7f02527fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585300784d0 0x5585300788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f02527fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585300784d0 0x5585300788b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35474/0 (socket says 172.21.15.78:35474) 2024-09-11T01:01:11.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f02527fc640 1 -- 172.21.15.78:0/3539825207 learned_addr learned my addr 172.21.15.78:0/3539825207 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:11.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f025affd640 1 --2- 172.21.15.78:0/3539825207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530078df0 0x558530084840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f02527fc640 1 --2- 172.21.15.78:0/3539825207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530084d80 0x558530087170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f02527fc640 1 -- 172.21.15.78:0/3539825207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530078df0 msgr2=0x558530084840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:11.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f02527fc640 1 --2- 172.21.15.78:0/3539825207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530078df0 0x558530084840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f02527fc640 1 -- 172.21.15.78:0/3539825207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530084d80 msgr2=0x558530087170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f02527fc640 1 --2- 172.21.15.78:0/3539825207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530084d80 0x558530087170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f02527fc640 1 -- 172.21.15.78:0/3539825207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852fd3c650 con 0x5585300784d0 2024-09-11T01:01:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f02527fc640 1 --2- 172.21.15.78:0/3539825207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585300784d0 0x5585300788b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02440117c0 tx=0x7f0244005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cdf5c9ba4f7474 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f0258ff9640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02480b5450 0x7f02480b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f02517fa640 1 -- 172.21.15.78:0/3539825207 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244004050 con 0x5585300784d0 2024-09-11T01:01:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f02517fa640 1 -- 172.21.15.78:0/3539825207 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02440054d0 con 0x5585300784d0 2024-09-11T01:01:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f02517fa640 1 -- 172.21.15.78:0/3539825207 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244004b80 con 0x5585300784d0 2024-09-11T01:01:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f025affd640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02480b5450 0x7f02480b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.046+0000 7f0258ff9640 1 -- 172.21.15.78:0/2132380663 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f02540e5dd0 con 0x7f025c10d120 2024-09-11T01:01:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.047+0000 7f02637e9740 1 -- 172.21.15.78:0/3539825207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585300784d0 msgr2=0x5585300788b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.047+0000 7f02637e9740 1 --2- 172.21.15.78:0/3539825207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585300784d0 0x5585300788b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02440117c0 tx=0x7f0244005850 comp rx=0 tx=0).stop 2024-09-11T01:01:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.047+0000 7f025affd640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02480b5450 0x7f02480b7910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f024c031440 tx=0x7f024c005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.047+0000 7f02637e9740 1 -- 172.21.15.78:0/3539825207 shutdown_connections 2024-09-11T01:01:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.047+0000 7f02637e9740 1 --2- 172.21.15.78:0/3539825207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530084d80 0x558530087170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.047+0000 7f02637e9740 1 --2- 172.21.15.78:0/3539825207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530078df0 0x558530084840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.047+0000 7f02637e9740 1 --2- 172.21.15.78:0/3539825207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585300784d0 0x5585300788b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.047+0000 7f02637e9740 1 -- 172.21.15.78:0/3539825207 >> 172.21.15.78:0/3539825207 conn(0x558530076300 msgr2=0x5585300774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.047+0000 7f02637e9740 1 -- 172.21.15.78:0/3539825207 shutdown_connections 2024-09-11T01:01:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.047+0000 7f02637e9740 1 -- 172.21.15.78:0/3539825207 wait complete. 2024-09-11T01:01:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02637e9740 1 -- start start 2024-09-11T01:01:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02637e9740 1 Processor -- start 2024-09-11T01:01:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02637e9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585300784d0 0x5585300846f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02637e9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 0x558530081b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02637e9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558530084d80 0x5585300821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02637e9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55852fe3f700 con 0x558530078df0 2024-09-11T01:01:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02637e9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55852ff79160 con 0x5585300784d0 2024-09-11T01:01:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02637e9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55853008be10 con 0x558530084d80 2024-09-11T01:01:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02637e9740 10 client.0 _ll_register_callbacks cb 0x55852ff929e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02527fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 0x558530081b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f025affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585300784d0 0x5585300846f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02527fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 0x558530081b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60794/0 (socket says 172.21.15.78:60794) 2024-09-11T01:01:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02527fc640 1 -- 172.21.15.78:0/1680447237 learned_addr learned my addr 172.21.15.78:0/1680447237 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f025b7fe640 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558530084d80 0x5585300821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02527fc640 1 -- 172.21.15.78:0/1680447237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558530084d80 msgr2=0x5585300821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02527fc640 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558530084d80 0x5585300821c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.048+0000 7f02527fc640 1 -- 172.21.15.78:0/1680447237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585300784d0 msgr2=0x5585300846f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f02527fc640 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585300784d0 0x5585300846f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f02527fc640 1 -- 172.21.15.78:0/1680447237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55852ffa6c90 con 0x558530078df0 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f025b7fe640 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558530084d80 0x5585300821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f025affd640 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585300784d0 0x5585300846f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f02527fc640 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 0x558530081b40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0244004880 tx=0x7f0244009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7857e4c9c37d4414 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f02427fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f02427fc640 1 -- 172.21.15.78:0/1680447237 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244004d30 con 0x558530078df0 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f02637e9740 1 -- 172.21.15.78:0/1680447237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02440054d0 con 0x558530078df0 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f02427fc640 1 -- 172.21.15.78:0/1680447237 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f024400f7c0 con 0x558530078df0 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f02427fc640 1 -- 172.21.15.78:0/1680447237 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f024400f9c0 con 0x558530078df0 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f02637e9740 10 client.15906 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f02637e9740 1 -- 172.21.15.78:0/1680447237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5585300977c0 con 0x558530078df0 2024-09-11T01:01:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.049+0000 7f02427fc640 1 -- 172.21.15.78:0/1680447237 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02440054d0 con 0x558530078df0 2024-09-11T01:01:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.050+0000 7f02427fc640 1 -- 172.21.15.78:0/1680447237 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f0244031050 con 0x558530078df0 2024-09-11T01:01:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.050+0000 7f02637e9740 10 client.15906 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.050+0000 7f02637e9740 10 client.15906 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:01:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.050+0000 7f02637e9740 20 client.15906 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.050+0000 7f02637e9740 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x558530098b10 0x55853009af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.050+0000 7f02637e9740 4 client.15906 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:01:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.050+0000 7f02637e9740 1 -- 172.21.15.78:0/1680447237 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55852fff0df0 con 0x558530098b10 2024-09-11T01:01:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.050+0000 7f025affd640 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x558530098b10 0x55853009af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.050+0000 7f02427fc640 10 client.15906 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:01:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.050+0000 7f025affd640 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x558530098b10 0x55853009af00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f5b6cbef913e016 server_cookie=7d8c7d19aea2408b in_seq=0 out_seq=0 2024-09-11T01:01:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02427fc640 1 -- 172.21.15.78:0/1680447237 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55852fff0df0 con 0x558530098b10 2024-09-11T01:01:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02427fc640 10 client.15906 handle_command_reply: tid=0 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02637e9740 1 client.15906 shutdown 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02637e9740 1 -- 172.21.15.78:0/1680447237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 msgr2=0x558530081b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02637e9740 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 0x558530081b40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0244004880 tx=0x7f0244009d70 comp rx=0 tx=0).stop 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02637e9740 1 -- 172.21.15.78:0/1680447237 shutdown_connections 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02637e9740 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558530084d80 0x5585300821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02637e9740 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 0x558530081b40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02637e9740 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x558530098b10 0x55853009af00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02637e9740 1 --2- 172.21.15.78:0/1680447237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585300784d0 0x5585300846f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02427fc640 0 client.15906 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02637e9740 1 -- 172.21.15.78:0/1680447237 >> 172.21.15.78:0/1680447237 conn(0x558530076300 msgr2=0x55853007bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02637e9740 1 -- 172.21.15.78:0/1680447237 shutdown_connections 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02637e9740 1 -- 172.21.15.78:0/1680447237 wait complete. 2024-09-11T01:01:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.051+0000 7f02637e9740 20 client.15906 trim_cache size 0 max 16384 2024-09-11T01:01:11.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.065+0000 7f02637e9740 1 Processor -- start 2024-09-11T01:01:11.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.065+0000 7f02637e9740 1 -- start start 2024-09-11T01:01:11.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.065+0000 7f02637e9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585300784d0 0x5585300773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.065+0000 7f02637e9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 0x5585300803c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.065+0000 7f02637e9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530084d80 0x558530080900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f02637e9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1ac2a0 con 0x558530078df0 2024-09-11T01:01:11.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f02637e9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1adea0 con 0x558530084d80 2024-09-11T01:01:11.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f02637e9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1ac970 con 0x5585300784d0 2024-09-11T01:01:11.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f02527fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 0x5585300803c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f025b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530084d80 0x558530080900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f02527fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 0x5585300803c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60800/0 (socket says 172.21.15.78:60800) 2024-09-11T01:01:11.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f02527fc640 1 -- 172.21.15.78:0/3031186980 learned_addr learned my addr 172.21.15.78:0/3031186980 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f025affd640 1 --2- 172.21.15.78:0/3031186980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585300784d0 0x5585300773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f02527fc640 1 -- 172.21.15.78:0/3031186980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585300784d0 msgr2=0x5585300773c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f02527fc640 1 --2- 172.21.15.78:0/3031186980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585300784d0 0x5585300773c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f02527fc640 1 -- 172.21.15.78:0/3031186980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530084d80 msgr2=0x558530080900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f02527fc640 1 --2- 172.21.15.78:0/3031186980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530084d80 0x558530080900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f02527fc640 1 -- 172.21.15.78:0/3031186980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558530076d80 con 0x558530078df0 2024-09-11T01:01:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f025b7fe640 1 --2- 172.21.15.78:0/3031186980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530084d80 0x558530080900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f02527fc640 1 --2- 172.21.15.78:0/3031186980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 0x5585300803c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0244009e40 tx=0x7f02440047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a5909ddcde6584 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f0250ff9640 1 -- 172.21.15.78:0/3031186980 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024400f790 con 0x558530078df0 2024-09-11T01:01:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f0250ff9640 1 -- 172.21.15.78:0/3031186980 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0244037020 con 0x558530078df0 2024-09-11T01:01:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.066+0000 7f0250ff9640 1 -- 172.21.15.78:0/3031186980 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244029640 con 0x558530078df0 2024-09-11T01:01:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.067+0000 7f02637e9740 1 -- 172.21.15.78:0/3031186980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 msgr2=0x5585300803c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.067+0000 7f02637e9740 1 --2- 172.21.15.78:0/3031186980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 0x5585300803c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0244009e40 tx=0x7f02440047f0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.067+0000 7f02637e9740 1 -- 172.21.15.78:0/3031186980 shutdown_connections 2024-09-11T01:01:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.067+0000 7f02637e9740 1 --2- 172.21.15.78:0/3031186980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530084d80 0x558530080900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.067+0000 7f02637e9740 1 --2- 172.21.15.78:0/3031186980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558530078df0 0x5585300803c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.067+0000 7f02637e9740 1 --2- 172.21.15.78:0/3031186980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585300784d0 0x5585300773c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.067+0000 7f02637e9740 1 -- 172.21.15.78:0/3031186980 >> 172.21.15.78:0/3031186980 conn(0x558530076300 msgr2=0x558530071440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.067+0000 7f02637e9740 1 -- 172.21.15.78:0/3031186980 shutdown_connections 2024-09-11T01:01:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.067+0000 7f02637e9740 1 -- 172.21.15.78:0/3031186980 wait complete. 2024-09-11T01:01:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02637e9740 1 -- start start 2024-09-11T01:01:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02637e9740 1 Processor -- start 2024-09-11T01:01:11.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02637e9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585300784d0 0x5585300910b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02637e9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558530078df0 0x5585300915f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02637e9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530084d80 0x55853008c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02637e9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1ad710 con 0x5585300784d0 2024-09-11T01:01:11.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02637e9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1ad040 con 0x558530084d80 2024-09-11T01:01:11.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02637e9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1aa280 con 0x558530078df0 2024-09-11T01:01:11.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f025affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585300784d0 0x5585300910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f025affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585300784d0 0x5585300910b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60810/0 (socket says 172.21.15.78:60810) 2024-09-11T01:01:11.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f025affd640 1 -- 172.21.15.78:0/788239207 learned_addr learned my addr 172.21.15.78:0/788239207 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:11.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02527fc640 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558530078df0 0x5585300915f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02637e9740 10 client.0 _ll_register_callbacks cb 0x7f0244009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:11.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f025b7fe640 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530084d80 0x55853008c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02527fc640 1 -- 172.21.15.78:0/788239207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530084d80 msgr2=0x55853008c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:11.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02527fc640 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530084d80 0x55853008c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02527fc640 1 -- 172.21.15.78:0/788239207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585300784d0 msgr2=0x5585300910b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:11.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02527fc640 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585300784d0 0x5585300910b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f02527fc640 1 -- 172.21.15.78:0/788239207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558530095480 con 0x558530078df0 2024-09-11T01:01:11.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.068+0000 7f025affd640 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585300784d0 0x5585300910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:11.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.069+0000 7f02527fc640 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558530078df0 0x5585300915f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0244005ca0 tx=0x7f0244005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=995f05ba75688046 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:11.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.069+0000 7f02427fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:01:11.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.069+0000 7f02427fc640 1 -- 172.21.15.78:0/788239207 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244042060 con 0x558530078df0 2024-09-11T01:01:11.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.069+0000 7f02637e9740 1 -- 172.21.15.78:0/788239207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0244037020 con 0x558530078df0 2024-09-11T01:01:11.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.069+0000 7f02427fc640 1 -- 172.21.15.78:0/788239207 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0244004b30 con 0x558530078df0 2024-09-11T01:01:11.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.069+0000 7f02427fc640 1 -- 172.21.15.78:0/788239207 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0244009770 con 0x558530078df0 2024-09-11T01:01:11.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.069+0000 7f02637e9740 10 client.25303 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:11.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.069+0000 7f02637e9740 1 -- 172.21.15.78:0/788239207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558530105c00 con 0x558530078df0 2024-09-11T01:01:11.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.069+0000 7f02427fc640 1 -- 172.21.15.78:0/788239207 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0244037020 con 0x558530078df0 2024-09-11T01:01:11.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02427fc640 1 -- 172.21.15.78:0/788239207 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f0244045020 con 0x558530078df0 2024-09-11T01:01:11.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 10 client.25303 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:11.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 10 client.25303 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:01:11.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 20 client.25303 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:11.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5585301061e0 0x558530108600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:11.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 4 client.25303 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:01:11.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 1 -- 172.21.15.78:0/788239207 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55852ffec8a0 con 0x5585301061e0 2024-09-11T01:01:11.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f025affd640 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5585301061e0 0x558530108600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:11.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02427fc640 10 client.25303 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:01:11.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f025affd640 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5585301061e0 0x558530108600 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1183edcc92d2b63b server_cookie=761ece33686db6cb in_seq=0 out_seq=0 2024-09-11T01:01:11.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02427fc640 1 -- 172.21.15.78:0/788239207 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55852ffec8a0 con 0x5585301061e0 2024-09-11T01:01:11.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02427fc640 10 client.25303 handle_command_reply: tid=0 2024-09-11T01:01:11.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 1 client.25303 shutdown 2024-09-11T01:01:11.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 1 -- 172.21.15.78:0/788239207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558530078df0 msgr2=0x5585300915f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:11.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558530078df0 0x5585300915f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0244005ca0 tx=0x7f0244005f90 comp rx=0 tx=0).stop 2024-09-11T01:01:11.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 1 -- 172.21.15.78:0/788239207 shutdown_connections 2024-09-11T01:01:11.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5585301061e0 0x558530108600 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558530084d80 0x55853008c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02427fc640 0 client.25303 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:01:11.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558530078df0 0x5585300915f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 1 --2- 172.21.15.78:0/788239207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585300784d0 0x5585300910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.070+0000 7f02637e9740 1 -- 172.21.15.78:0/788239207 >> 172.21.15.78:0/788239207 conn(0x558530076300 msgr2=0x55853007cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:11.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f02637e9740 1 -- 172.21.15.78:0/788239207 shutdown_connections 2024-09-11T01:01:11.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f02637e9740 1 -- 172.21.15.78:0/788239207 wait complete. 2024-09-11T01:01:11.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f02637e9740 20 client.25303 trim_cache size 0 max 16384 2024-09-11T01:01:11.089 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:01:11.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f0261414640 1 -- 172.21.15.78:0/2132380663 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02480b5450 msgr2=0x7f02480b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:11.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f0261414640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02480b5450 0x7f02480b7910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f024c031440 tx=0x7f024c005b50 comp rx=0 tx=0).stop 2024-09-11T01:01:11.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f0261414640 1 -- 172.21.15.78:0/2132380663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f025c10d120 msgr2=0x7f025c19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:11.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f0261414640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f025c10d120 0x7f025c19eb60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f025400eaa0 tx=0x7f025400ef60 comp rx=0 tx=0).stop 2024-09-11T01:01:11.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f0261414640 1 -- 172.21.15.78:0/2132380663 shutdown_connections 2024-09-11T01:01:11.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f0261414640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02480b5450 0x7f02480b7910 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f0261414640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f025c10d120 0x7f025c19eb60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f0261414640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f025c1045e0 0x7f025c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f0261414640 1 --2- 172.21.15.78:0/2132380663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c103cc0 0x7f025c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:11.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.071+0000 7f0261414640 1 -- 172.21.15.78:0/2132380663 >> 172.21.15.78:0/2132380663 conn(0x7f025c0fd740 msgr2=0x7f025c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:11.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.072+0000 7f0261414640 1 -- 172.21.15.78:0/2132380663 shutdown_connections 2024-09-11T01:01:11.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:11.072+0000 7f0261414640 1 -- 172.21.15.78:0/2132380663 wait complete. 2024-09-11T01:01:11.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 167118 11.64 MB/sec warmup 79 sec latency 19.174 ms 2024-09-11T01:01:11.841 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:01:12.108 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:12.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 169309 11.60 MB/sec warmup 80 sec latency 25.946 ms 2024-09-11T01:01:12.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.588+0000 7f2681e7e640 1 -- 172.21.15.78:0/3672841864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f267c1076a0 msgr2=0x7f267c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:12.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.588+0000 7f2681e7e640 1 --2- 172.21.15.78:0/3672841864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f267c1076a0 0x7f267c107ae0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2670009900 tx=0x7f267002a410 comp rx=0 tx=0).stop 2024-09-11T01:01:12.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.588+0000 7f2681e7e640 1 -- 172.21.15.78:0/3672841864 shutdown_connections 2024-09-11T01:01:12.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.588+0000 7f2681e7e640 1 --2- 172.21.15.78:0/3672841864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f267c108020 0x7f267c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.588+0000 7f2681e7e640 1 --2- 172.21.15.78:0/3672841864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f267c1076a0 0x7f267c107ae0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.588+0000 7f2681e7e640 1 --2- 172.21.15.78:0/3672841864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f267c101c90 0x7f267c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.588+0000 7f2681e7e640 1 -- 172.21.15.78:0/3672841864 >> 172.21.15.78:0/3672841864 conn(0x7f267c0fd700 msgr2=0x7f267c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.588+0000 7f2681e7e640 1 -- 172.21.15.78:0/3672841864 shutdown_connections 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.588+0000 7f2681e7e640 1 -- 172.21.15.78:0/3672841864 wait complete. 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f2681e7e640 1 Processor -- start 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f2681e7e640 1 -- start start 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f2681e7e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f267c101c90 0x7f267c0fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f2681e7e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f267c1076a0 0x7f267c0fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f2681e7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f267c108020 0x7f267c0fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f2681e7e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f267c111ca0 con 0x7f267c1076a0 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f2681e7e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f267c111b20 con 0x7f267c101c90 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f2681e7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f267c111e20 con 0x7f267c108020 2024-09-11T01:01:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f267affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f267c1076a0 0x7f267c0fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f267affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f267c1076a0 0x7f267c0fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60832/0 (socket says 172.21.15.78:60832) 2024-09-11T01:01:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f267c101c90 0x7f267c0fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f267bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f267c108020 0x7f267c0fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.589+0000 7f267affd640 1 -- 172.21.15.78:0/1582138015 learned_addr learned my addr 172.21.15.78:0/1582138015 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.590+0000 7f267affd640 1 -- 172.21.15.78:0/1582138015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f267c108020 msgr2=0x7f267c0fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.590+0000 7f267affd640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f267c108020 0x7f267c0fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.590+0000 7f267affd640 1 -- 172.21.15.78:0/1582138015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f267c101c90 msgr2=0x7f267c0fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.590+0000 7f267affd640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f267c101c90 0x7f267c0fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.590+0000 7f267affd640 1 -- 172.21.15.78:0/1582138015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c1a76e0 con 0x7f267c1076a0 2024-09-11T01:01:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.590+0000 7f267bfff640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f267c108020 0x7f267c0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.590+0000 7f267affd640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f267c1076a0 0x7f267c0fe140 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2670005940 tx=0x7f26700040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.590+0000 7f2678ff9640 1 -- 172.21.15.78:0/1582138015 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267003f030 con 0x7f267c1076a0 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.590+0000 7f2681e7e640 1 -- 172.21.15.78:0/1582138015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c1a7970 con 0x7f267c1076a0 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.590+0000 7f2678ff9640 1 -- 172.21.15.78:0/1582138015 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26700044c0 con 0x7f267c1076a0 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.590+0000 7f2678ff9640 1 -- 172.21.15.78:0/1582138015 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670035070 con 0x7f267c1076a0 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.590+0000 7f2681e7e640 1 -- 172.21.15.78:0/1582138015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c1a7ea0 con 0x7f267c1076a0 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f2684253740 1 Processor -- start 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f2678ff9640 1 -- 172.21.15.78:0/1582138015 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f267003b9f0 con 0x7f267c1076a0 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f2684253740 1 -- start start 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f2684253740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 0x5557c07058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f2684253740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0705df0 0x5557c0711840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f2684253740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0711d80 0x5557c0714170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f2684253740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5557c067bea0 con 0x5557c0705df0 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f2684253740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5557c066b2d0 con 0x5557c0711d80 2024-09-11T01:01:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f2684253740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5557c066f0e0 con 0x5557c07054d0 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0705df0 0x5557c0711840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0705df0 0x5557c0711840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60840/0 (socket says 172.21.15.78:60840) 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f267b7fe640 1 -- 172.21.15.78:0/2062770142 learned_addr learned my addr 172.21.15.78:0/2062770142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f267bfff640 1 --2- 172.21.15.78:0/2062770142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 0x5557c07058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.593+0000 7f267affd640 1 --2- 172.21.15.78:0/2062770142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0711d80 0x5557c0714170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f267b7fe640 1 -- 172.21.15.78:0/2062770142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 msgr2=0x5557c07058b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f267b7fe640 1 --2- 172.21.15.78:0/2062770142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 0x5557c07058b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f267b7fe640 1 -- 172.21.15.78:0/2062770142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0711d80 msgr2=0x5557c0714170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f267b7fe640 1 --2- 172.21.15.78:0/2062770142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0711d80 0x5557c0714170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f267b7fe640 1 -- 172.21.15.78:0/2062770142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c03c9650 con 0x5557c0705df0 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f267bfff640 1 --2- 172.21.15.78:0/2062770142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 0x5557c07058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f267b7fe640 1 --2- 172.21.15.78:0/2062770142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0705df0 0x5557c0711840 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f266c009cf0 tx=0x7f266c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785876480e1d9eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f2661ffb640 1 -- 172.21.15.78:0/2062770142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c004230 con 0x5557c0705df0 2024-09-11T01:01:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f2678ff9640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26640b5450 0x7f26640b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f2661ffb640 1 -- 172.21.15.78:0/2062770142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c0043d0 con 0x5557c0705df0 2024-09-11T01:01:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f2661ffb640 1 -- 172.21.15.78:0/2062770142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c019820 con 0x5557c0705df0 2024-09-11T01:01:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f267bfff640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26640b5450 0x7f26640b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.594+0000 7f2678ff9640 1 -- 172.21.15.78:0/1582138015 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f2670106580 con 0x7f267c1076a0 2024-09-11T01:01:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.595+0000 7f2684253740 1 -- 172.21.15.78:0/2062770142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0705df0 msgr2=0x5557c0711840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.595+0000 7f267bfff640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26640b5450 0x7f26640b7910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2674009fd0 tx=0x7f2674009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.595+0000 7f2684253740 1 --2- 172.21.15.78:0/2062770142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0705df0 0x5557c0711840 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f266c009cf0 tx=0x7f266c011810 comp rx=0 tx=0).stop 2024-09-11T01:01:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.595+0000 7f2684253740 1 -- 172.21.15.78:0/2062770142 shutdown_connections 2024-09-11T01:01:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.595+0000 7f2684253740 1 --2- 172.21.15.78:0/2062770142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0711d80 0x5557c0714170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.595+0000 7f2684253740 1 --2- 172.21.15.78:0/2062770142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0705df0 0x5557c0711840 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.595+0000 7f2684253740 1 --2- 172.21.15.78:0/2062770142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 0x5557c07058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.595+0000 7f2684253740 1 -- 172.21.15.78:0/2062770142 >> 172.21.15.78:0/2062770142 conn(0x5557c0703300 msgr2=0x5557c07044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:12.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.595+0000 7f2684253740 1 -- 172.21.15.78:0/2062770142 shutdown_connections 2024-09-11T01:01:12.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.595+0000 7f2684253740 1 -- 172.21.15.78:0/2062770142 wait complete. 2024-09-11T01:01:12.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.595+0000 7f2684253740 1 -- start start 2024-09-11T01:01:12.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f2684253740 1 Processor -- start 2024-09-11T01:01:12.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f2684253740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 0x5557c07115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f2684253740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0705df0 0x5557c070ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f2684253740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 0x5557c070efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f2684253740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5557c04cc700 con 0x5557c0711d80 2024-09-11T01:01:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f2684253740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5557c0606160 con 0x5557c0705df0 2024-09-11T01:01:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f2684253740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5557c0714a00 con 0x5557c07054d0 2024-09-11T01:01:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f267bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 0x5557c070efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 0x5557c07115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f2684253740 10 client.0 _ll_register_callbacks cb 0x5557c061f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f267affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0705df0 0x5557c070ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f267bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 0x5557c070efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60848/0 (socket says 172.21.15.78:60848) 2024-09-11T01:01:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 0x5557c07115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35538/0 (socket says 172.21.15.78:35538) 2024-09-11T01:01:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f267bfff640 1 -- 172.21.15.78:0/3207027333 learned_addr learned my addr 172.21.15.78:0/3207027333 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f267bfff640 1 -- 172.21.15.78:0/3207027333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 msgr2=0x5557c07115f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f267bfff640 1 --2- 172.21.15.78:0/3207027333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 0x5557c07115f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f267bfff640 1 -- 172.21.15.78:0/3207027333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0705df0 msgr2=0x5557c070ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f267bfff640 1 --2- 172.21.15.78:0/3207027333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0705df0 0x5557c070ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f267bfff640 1 -- 172.21.15.78:0/3207027333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c0633c90 con 0x5557c0711d80 2024-09-11T01:01:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.596+0000 7f267b7fe640 1 --2- 172.21.15.78:0/3207027333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 0x5557c07115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.597+0000 7f267bfff640 1 --2- 172.21.15.78:0/3207027333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 0x5557c070efc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2674011280 tx=0x7f2674004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f9ac1e7540b700 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.597+0000 7f265affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:01:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.597+0000 7f265affd640 1 -- 172.21.15.78:0/3207027333 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674033060 con 0x5557c0711d80 2024-09-11T01:01:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.597+0000 7f265affd640 1 -- 172.21.15.78:0/3207027333 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f2674017030 con 0x5557c0711d80 2024-09-11T01:01:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.597+0000 7f2684253740 1 -- 172.21.15.78:0/3207027333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f266c0043d0 con 0x5557c0711d80 2024-09-11T01:01:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.597+0000 7f265affd640 1 -- 172.21.15.78:0/3207027333 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f26740113f0 con 0x5557c0711d80 2024-09-11T01:01:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.597+0000 7f2684253740 10 client.15924 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.597+0000 7f2684253740 1 -- 172.21.15.78:0/3207027333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5557c0720340 con 0x5557c0711d80 2024-09-11T01:01:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.597+0000 7f265affd640 1 -- 172.21.15.78:0/3207027333 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c0043d0 con 0x5557c0711d80 2024-09-11T01:01:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.598+0000 7f265affd640 1 -- 172.21.15.78:0/3207027333 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f2674036020 con 0x5557c0711d80 2024-09-11T01:01:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.598+0000 7f2684253740 10 client.15924 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.598+0000 7f2684253740 10 client.15924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:01:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.598+0000 7f2684253740 20 client.15924 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.598+0000 7f2684253740 1 --2- 172.21.15.78:0/3207027333 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5557c0721850 0x5557c0723c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.598+0000 7f2684253740 4 client.15924 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:01:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.598+0000 7f2684253740 1 -- 172.21.15.78:0/3207027333 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557c067ddf0 con 0x5557c0721850 2024-09-11T01:01:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.598+0000 7f267b7fe640 1 --2- 172.21.15.78:0/3207027333 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5557c0721850 0x5557c0723c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.598+0000 7f265affd640 10 client.15924 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:01:12.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.598+0000 7f267b7fe640 1 --2- 172.21.15.78:0/3207027333 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5557c0721850 0x5557c0723c40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa165a31baa1bcd server_cookie=24c8b72d57c1a309 in_seq=0 out_seq=0 2024-09-11T01:01:12.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.599+0000 7f265affd640 1 -- 172.21.15.78:0/3207027333 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5557c067ddf0 con 0x5557c0721850 2024-09-11T01:01:12.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.599+0000 7f265affd640 10 client.15924 handle_command_reply: tid=0 2024-09-11T01:01:12.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.599+0000 7f2684253740 1 client.15924 shutdown 2024-09-11T01:01:12.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.600+0000 7f2684253740 1 -- 172.21.15.78:0/3207027333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 msgr2=0x5557c070efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.600+0000 7f2684253740 1 --2- 172.21.15.78:0/3207027333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 0x5557c070efc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2674011280 tx=0x7f2674004ab0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.600+0000 7f2684253740 1 -- 172.21.15.78:0/3207027333 shutdown_connections 2024-09-11T01:01:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.600+0000 7f2684253740 1 --2- 172.21.15.78:0/3207027333 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5557c0721850 0x5557c0723c40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.600+0000 7f2684253740 1 --2- 172.21.15.78:0/3207027333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 0x5557c070efc0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.600+0000 7f2684253740 1 --2- 172.21.15.78:0/3207027333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0705df0 0x5557c070ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.600+0000 7f2684253740 1 --2- 172.21.15.78:0/3207027333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c07054d0 0x5557c07115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.600+0000 7f2684253740 1 -- 172.21.15.78:0/3207027333 >> 172.21.15.78:0/3207027333 conn(0x5557c0703300 msgr2=0x5557c0708ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.600+0000 7f265affd640 0 client.15924 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:01:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.600+0000 7f2684253740 1 -- 172.21.15.78:0/3207027333 shutdown_connections 2024-09-11T01:01:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.600+0000 7f2684253740 1 -- 172.21.15.78:0/3207027333 wait complete. 2024-09-11T01:01:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.600+0000 7f2684253740 20 client.15924 trim_cache size 0 max 16384 2024-09-11T01:01:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.614+0000 7f2684253740 1 Processor -- start 2024-09-11T01:01:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.614+0000 7f2684253740 1 -- start start 2024-09-11T01:01:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.614+0000 7f2684253740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c07054d0 0x5557c071dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.614+0000 7f2684253740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c0705df0 0x5557c070d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f2684253740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 0x5557c070d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f2684253740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1ac3b0 con 0x5557c0711d80 2024-09-11T01:01:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f2684253740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1adfb0 con 0x5557c07054d0 2024-09-11T01:01:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f2684253740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1aca80 con 0x5557c0705df0 2024-09-11T01:01:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c07054d0 0x5557c071dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c0705df0 0x5557c070d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 0x5557c070d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c0705df0 0x5557c070d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35540/0 (socket says 172.21.15.78:35540) 2024-09-11T01:01:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c07054d0 0x5557c071dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39666/0 (socket says 172.21.15.78:39666) 2024-09-11T01:01:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267affd640 1 -- 172.21.15.78:0/466695080 learned_addr learned my addr 172.21.15.78:0/466695080 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267b7fe640 1 -- 172.21.15.78:0/466695080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c0705df0 msgr2=0x5557c070d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267b7fe640 1 --2- 172.21.15.78:0/466695080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c0705df0 0x5557c070d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267b7fe640 1 -- 172.21.15.78:0/466695080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 msgr2=0x5557c070d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267b7fe640 1 --2- 172.21.15.78:0/466695080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 0x5557c070d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267b7fe640 1 -- 172.21.15.78:0/466695080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c0711960 con 0x5557c07054d0 2024-09-11T01:01:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267bfff640 1 --2- 172.21.15.78:0/466695080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 0x5557c070d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f267b7fe640 1 --2- 172.21.15.78:0/466695080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c07054d0 0x5557c071dc60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f266c01b040 tx=0x7f266c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11937ffa8f208197 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f26617fa640 1 -- 172.21.15.78:0/466695080 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c031070 con 0x5557c07054d0 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.615+0000 7f26617fa640 1 -- 172.21.15.78:0/466695080 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c003aa0 con 0x5557c07054d0 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.616+0000 7f26617fa640 1 -- 172.21.15.78:0/466695080 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c004600 con 0x5557c07054d0 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.616+0000 7f2684253740 1 -- 172.21.15.78:0/466695080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c07054d0 msgr2=0x5557c071dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.616+0000 7f2684253740 1 --2- 172.21.15.78:0/466695080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c07054d0 0x5557c071dc60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f266c01b040 tx=0x7f266c011810 comp rx=0 tx=0).stop 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.616+0000 7f2684253740 1 -- 172.21.15.78:0/466695080 shutdown_connections 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.616+0000 7f2684253740 1 --2- 172.21.15.78:0/466695080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c0711d80 0x5557c070d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.616+0000 7f2684253740 1 --2- 172.21.15.78:0/466695080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c0705df0 0x5557c070d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.616+0000 7f2684253740 1 --2- 172.21.15.78:0/466695080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c07054d0 0x5557c071dc60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.616+0000 7f2684253740 1 -- 172.21.15.78:0/466695080 >> 172.21.15.78:0/466695080 conn(0x5557c0703300 msgr2=0x5557c0704350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.616+0000 7f2684253740 1 -- 172.21.15.78:0/466695080 shutdown_connections 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.616+0000 7f2684253740 1 -- 172.21.15.78:0/466695080 wait complete. 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f2684253740 1 -- start start 2024-09-11T01:01:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f2684253740 1 Processor -- start 2024-09-11T01:01:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f2684253740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c07054d0 0x5557c071a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f2684253740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c0705df0 0x5557c0729300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f2684253740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0711d80 0x5557c07299d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f2684253740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1ad820 con 0x5557c07054d0 2024-09-11T01:01:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f2684253740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1ad150 con 0x5557c0711d80 2024-09-11T01:01:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f2684253740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1aa3f0 con 0x5557c0705df0 2024-09-11T01:01:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f2684253740 10 client.0 _ll_register_callbacks cb 0x7f2674003bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c07054d0 0x5557c071a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f267affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c0705df0 0x5557c0729300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f267bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0711d80 0x5557c07299d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c07054d0 0x5557c071a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60874/0 (socket says 172.21.15.78:60874) 2024-09-11T01:01:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.617+0000 7f267b7fe640 1 -- 172.21.15.78:0/1482936611 learned_addr learned my addr 172.21.15.78:0/1482936611 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f267bfff640 1 -- 172.21.15.78:0/1482936611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c0705df0 msgr2=0x5557c0729300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f267bfff640 1 --2- 172.21.15.78:0/1482936611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c0705df0 0x5557c0729300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f267bfff640 1 -- 172.21.15.78:0/1482936611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c07054d0 msgr2=0x5557c071a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f267bfff640 1 --2- 172.21.15.78:0/1482936611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c07054d0 0x5557c071a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f267bfff640 1 -- 172.21.15.78:0/1482936611 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c07191c0 con 0x5557c0711d80 2024-09-11T01:01:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f267b7fe640 1 --2- 172.21.15.78:0/1482936611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c07054d0 0x5557c071a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f267bfff640 1 --2- 172.21.15.78:0/1482936611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0711d80 0x5557c07299d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2674018040 tx=0x7f2674010d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e391489220256bc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f265affd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:01:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f265affd640 1 -- 172.21.15.78:0/1482936611 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26740048f0 con 0x5557c0711d80 2024-09-11T01:01:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f265affd640 1 -- 172.21.15.78:0/1482936611 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f26740250d0 con 0x5557c0711d80 2024-09-11T01:01:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f2684253740 1 -- 172.21.15.78:0/1482936611 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f266c003aa0 con 0x5557c0711d80 2024-09-11T01:01:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f265affd640 1 -- 172.21.15.78:0/1482936611 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f26740252d0 con 0x5557c0711d80 2024-09-11T01:01:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f2684253740 10 client.25334 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f2684253740 1 -- 172.21.15.78:0/1482936611 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5557c078e910 con 0x5557c0711d80 2024-09-11T01:01:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.618+0000 7f265affd640 1 -- 172.21.15.78:0/1482936611 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c003aa0 con 0x5557c0711d80 2024-09-11T01:01:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.619+0000 7f265affd640 1 -- 172.21.15.78:0/1482936611 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f267402f640 con 0x5557c0711d80 2024-09-11T01:01:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.619+0000 7f2684253740 10 client.25334 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.619+0000 7f2684253740 10 client.25334 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:01:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.619+0000 7f2684253740 20 client.25334 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.619+0000 7f2684253740 1 --2- 172.21.15.78:0/1482936611 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5557c078f080 0x5557c0791470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.619+0000 7f2684253740 4 client.25334 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:01:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.619+0000 7f2684253740 1 -- 172.21.15.78:0/1482936611 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557c06798a0 con 0x5557c078f080 2024-09-11T01:01:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.619+0000 7f267b7fe640 1 --2- 172.21.15.78:0/1482936611 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5557c078f080 0x5557c0791470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.619+0000 7f265affd640 10 client.25334 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.619+0000 7f267b7fe640 1 --2- 172.21.15.78:0/1482936611 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5557c078f080 0x5557c0791470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4df2ef08d88c0a28 server_cookie=96e37ffe08445fbd in_seq=0 out_seq=0 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.619+0000 7f265affd640 1 -- 172.21.15.78:0/1482936611 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557c06798a0 con 0x5557c078f080 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.619+0000 7f265affd640 10 client.25334 handle_command_reply: tid=0 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f2684253740 1 client.25334 shutdown 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f2684253740 1 -- 172.21.15.78:0/1482936611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0711d80 msgr2=0x5557c07299d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f2684253740 1 --2- 172.21.15.78:0/1482936611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0711d80 0x5557c07299d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2674018040 tx=0x7f2674010d00 comp rx=0 tx=0).stop 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f2684253740 1 -- 172.21.15.78:0/1482936611 shutdown_connections 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f2684253740 1 --2- 172.21.15.78:0/1482936611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557c0711d80 0x5557c07299d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f2684253740 1 --2- 172.21.15.78:0/1482936611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c0705df0 0x5557c0729300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f2684253740 1 --2- 172.21.15.78:0/1482936611 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5557c078f080 0x5557c0791470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f2684253740 1 --2- 172.21.15.78:0/1482936611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557c07054d0 0x5557c071a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f265affd640 0 client.25334 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:01:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f2684253740 1 -- 172.21.15.78:0/1482936611 >> 172.21.15.78:0/1482936611 conn(0x5557c0703300 msgr2=0x5557c0704350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f2684253740 1 -- 172.21.15.78:0/1482936611 shutdown_connections 2024-09-11T01:01:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f2684253740 1 -- 172.21.15.78:0/1482936611 wait complete. 2024-09-11T01:01:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.620+0000 7f2684253740 20 client.25334 trim_cache size 0 max 16384 2024-09-11T01:01:12.633 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:01:12.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.621+0000 7f2681e7e640 1 -- 172.21.15.78:0/1582138015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26640b5450 msgr2=0x7f26640b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:12.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.621+0000 7f2681e7e640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26640b5450 0x7f26640b7910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2674009fd0 tx=0x7f2674009500 comp rx=0 tx=0).stop 2024-09-11T01:01:12.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.621+0000 7f2681e7e640 1 -- 172.21.15.78:0/1582138015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f267c1076a0 msgr2=0x7f267c0fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:12.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.621+0000 7f2681e7e640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f267c1076a0 0x7f267c0fe140 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2670005940 tx=0x7f26700040c0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.621+0000 7f2681e7e640 1 -- 172.21.15.78:0/1582138015 shutdown_connections 2024-09-11T01:01:12.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.621+0000 7f2681e7e640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26640b5450 0x7f26640b7910 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.621+0000 7f2681e7e640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f267c108020 0x7f267c0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.621+0000 7f2681e7e640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f267c1076a0 0x7f267c0fe140 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.621+0000 7f2681e7e640 1 --2- 172.21.15.78:0/1582138015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f267c101c90 0x7f267c0fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:12.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.621+0000 7f2681e7e640 1 -- 172.21.15.78:0/1582138015 >> 172.21.15.78:0/1582138015 conn(0x7f267c0fd700 msgr2=0x7f267c1047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:12.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.621+0000 7f2681e7e640 1 -- 172.21.15.78:0/1582138015 shutdown_connections 2024-09-11T01:01:12.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:12.621+0000 7f2681e7e640 1 -- 172.21.15.78:0/1582138015 wait complete. 2024-09-11T01:01:13.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 170994 11.59 MB/sec warmup 81 sec latency 33.121 ms 2024-09-11T01:01:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:13 smithi110 ceph-mon[34712]: pgmap v393: 97 pgs: 97 active+clean; 102 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 349 op/s 2024-09-11T01:01:13.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:13 smithi112 ceph-mon[35384]: pgmap v393: 97 pgs: 97 active+clean; 102 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 349 op/s 2024-09-11T01:01:13.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:13 smithi078 ceph-mon[31315]: pgmap v393: 97 pgs: 97 active+clean; 102 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 349 op/s 2024-09-11T01:01:13.562 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:01:13.828 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.314+0000 7f5add194640 1 -- 172.21.15.78:0/2611203947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ad8108040 msgr2=0x7f5ad810e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.314+0000 7f5add194640 1 --2- 172.21.15.78:0/2611203947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ad8108040 0x7f5ad810e840 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009ca0 tx=0x7f5ad002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.314+0000 7f5add194640 1 -- 172.21.15.78:0/2611203947 shutdown_connections 2024-09-11T01:01:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.314+0000 7f5add194640 1 --2- 172.21.15.78:0/2611203947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ad8108040 0x7f5ad810e840 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.314+0000 7f5add194640 1 --2- 172.21.15.78:0/2611203947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ad81076c0 0x7f5ad8107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.314+0000 7f5add194640 1 --2- 172.21.15.78:0/2611203947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ad8101cb0 0x7f5ad8102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.314+0000 7f5add194640 1 -- 172.21.15.78:0/2611203947 >> 172.21.15.78:0/2611203947 conn(0x7f5ad80fd700 msgr2=0x7f5ad80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.314+0000 7f5add194640 1 -- 172.21.15.78:0/2611203947 shutdown_connections 2024-09-11T01:01:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.314+0000 7f5add194640 1 -- 172.21.15.78:0/2611203947 wait complete. 2024-09-11T01:01:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.315+0000 7f5add194640 1 Processor -- start 2024-09-11T01:01:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.315+0000 7f5add194640 1 -- start start 2024-09-11T01:01:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5add194640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ad8101cb0 0x7f5ad80fde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5add194640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ad81076c0 0x7f5ad80fe3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5add194640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ad8108040 0x7f5ad80fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5add194640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8111d10 con 0x7f5ad8101cb0 2024-09-11T01:01:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5add194640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8111b90 con 0x7f5ad8108040 2024-09-11T01:01:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5add194640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8111e90 con 0x7f5ad81076c0 2024-09-11T01:01:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5ad6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ad8101cb0 0x7f5ad80fde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5ad659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ad81076c0 0x7f5ad80fe3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5ad6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ad8101cb0 0x7f5ad80fde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49538/0 (socket says 172.21.15.78:49538) 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5ad6d9d640 1 -- 172.21.15.78:0/860368307 learned_addr learned my addr 172.21.15.78:0/860368307 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5ad759e640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ad8108040 0x7f5ad80fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5ad659c640 1 -- 172.21.15.78:0/860368307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ad8108040 msgr2=0x7f5ad80fe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5ad659c640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ad8108040 0x7f5ad80fe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5ad659c640 1 -- 172.21.15.78:0/860368307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ad8101cb0 msgr2=0x7f5ad80fde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5ad659c640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ad8101cb0 0x7f5ad80fde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5ad659c640 1 -- 172.21.15.78:0/860368307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad81a76e0 con 0x7f5ad81076c0 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.316+0000 7f5ad659c640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ad81076c0 0x7f5ad80fe3b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00e9f0 tx=0x7f5acc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.317+0000 7f5abffff640 1 -- 172.21.15.78:0/860368307 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc00ccc0 con 0x7f5ad81076c0 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.317+0000 7f5add194640 1 -- 172.21.15.78:0/860368307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad81a79d0 con 0x7f5ad81076c0 2024-09-11T01:01:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.317+0000 7f5abffff640 1 -- 172.21.15.78:0/860368307 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5acc0108e0 con 0x7f5ad81076c0 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.317+0000 7f5abffff640 1 -- 172.21.15.78:0/860368307 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc018880 con 0x7f5ad81076c0 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.317+0000 7f5add194640 1 -- 172.21.15.78:0/860368307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad81a7f10 con 0x7f5ad81076c0 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.319+0000 7f5abffff640 1 -- 172.21.15.78:0/860368307 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5acc010430 con 0x7f5ad81076c0 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.319+0000 7f5adf569740 1 Processor -- start 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.319+0000 7f5adf569740 1 -- start start 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5adf569740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5adf569740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66b0df0 0x5630c66bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5adf569740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66bcd80 0x5630c66bf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5adf569740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5630c6626ea0 con 0x5630c66b0df0 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5adf569740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5630c66162d0 con 0x5630c66bcd80 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5adf569740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630c661a0e0 con 0x5630c66b04d0 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5ad6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66b0df0 0x5630c66bc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5ad659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66bcd80 0x5630c66bf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5ad6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66b0df0 0x5630c66bc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49548/0 (socket says 172.21.15.78:49548) 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5ad6d9d640 1 -- 172.21.15.78:0/2503855524 learned_addr learned my addr 172.21.15.78:0/2503855524 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5ad759e640 1 --2- 172.21.15.78:0/2503855524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5ad659c640 1 -- 172.21.15.78:0/2503855524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 msgr2=0x5630c66b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5ad659c640 1 --2- 172.21.15.78:0/2503855524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5ad659c640 1 -- 172.21.15.78:0/2503855524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66b0df0 msgr2=0x5630c66bc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5ad659c640 1 --2- 172.21.15.78:0/2503855524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66b0df0 0x5630c66bc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5ad659c640 1 -- 172.21.15.78:0/2503855524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c6374650 con 0x5630c66bcd80 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.320+0000 7f5abffff640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5ac00b5450 0x7f5ac00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.321+0000 7f5ad659c640 1 --2- 172.21.15.78:0/2503855524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66bcd80 0x5630c66bf170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0e39b0 tx=0x7f5acc0ec430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43154339458ce4e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.321+0000 7f5ad759e640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5ac00b5450 0x7f5ac00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.321+0000 7f5abffff640 1 -- 172.21.15.78:0/860368307 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5acc0e61f0 con 0x7f5ad81076c0 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.321+0000 7f5abd7fa640 1 -- 172.21.15.78:0/2503855524 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0a8de0 con 0x5630c66bcd80 2024-09-11T01:01:14.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.321+0000 7f5abd7fa640 1 -- 172.21.15.78:0/2503855524 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5acc0ac2a0 con 0x5630c66bcd80 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.321+0000 7f5abd7fa640 1 -- 172.21.15.78:0/2503855524 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0abf00 con 0x5630c66bcd80 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.322+0000 7f5adf569740 1 -- 172.21.15.78:0/2503855524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66bcd80 msgr2=0x5630c66bf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.322+0000 7f5ad759e640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5ac00b5450 0x7f5ac00b7910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80fff30 tx=0x7f5ad0005d80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.322+0000 7f5adf569740 1 --2- 172.21.15.78:0/2503855524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66bcd80 0x5630c66bf170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0e39b0 tx=0x7f5acc0ec430 comp rx=0 tx=0).stop 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.322+0000 7f5adf569740 1 -- 172.21.15.78:0/2503855524 shutdown_connections 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.322+0000 7f5adf569740 1 --2- 172.21.15.78:0/2503855524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66bcd80 0x5630c66bf170 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.322+0000 7f5adf569740 1 --2- 172.21.15.78:0/2503855524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66b0df0 0x5630c66bc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.322+0000 7f5adf569740 1 --2- 172.21.15.78:0/2503855524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.322+0000 7f5adf569740 1 -- 172.21.15.78:0/2503855524 >> 172.21.15.78:0/2503855524 conn(0x5630c66ae300 msgr2=0x5630c66af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.322+0000 7f5adf569740 1 -- 172.21.15.78:0/2503855524 shutdown_connections 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.322+0000 7f5adf569740 1 -- 172.21.15.78:0/2503855524 wait complete. 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.322+0000 7f5adf569740 1 -- start start 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.322+0000 7f5adf569740 1 Processor -- start 2024-09-11T01:01:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5adf569740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b04d0 0x5630c66bc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5adf569740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b0df0 0x5630c66b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5adf569740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66ba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5adf569740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5630c6477700 con 0x5630c66bcd80 2024-09-11T01:01:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5adf569740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5630c65b1160 con 0x5630c66b04d0 2024-09-11T01:01:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5adf569740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630c66bfa00 con 0x5630c66b0df0 2024-09-11T01:01:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5adf569740 10 client.0 _ll_register_callbacks cb 0x5630c65ca9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5ad759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66ba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5ad6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b04d0 0x5630c66bc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5ad6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b04d0 0x5630c66bc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41126/0 (socket says 172.21.15.78:41126) 2024-09-11T01:01:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5ad659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b0df0 0x5630c66b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5ad759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66ba180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49562/0 (socket says 172.21.15.78:49562) 2024-09-11T01:01:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5ad6d9d640 1 -- 172.21.15.78:0/3597073313 learned_addr learned my addr 172.21.15.78:0/3597073313 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5ad6d9d640 1 -- 172.21.15.78:0/3597073313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b0df0 msgr2=0x5630c66b9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5ad6d9d640 1 --2- 172.21.15.78:0/3597073313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b0df0 0x5630c66b9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5ad6d9d640 1 -- 172.21.15.78:0/3597073313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 msgr2=0x5630c66ba180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5ad6d9d640 1 --2- 172.21.15.78:0/3597073313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66ba180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5ad6d9d640 1 -- 172.21.15.78:0/3597073313 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c65dec90 con 0x5630c66b04d0 2024-09-11T01:01:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5ad6d9d640 1 --2- 172.21.15.78:0/3597073313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b04d0 0x5630c66bc6d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8005f60 tx=0x7f5ac8012bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d87922708fbecde server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5abaff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:01:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.323+0000 7f5abaff5640 1 -- 172.21.15.78:0/3597073313 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac800f920 con 0x5630c66b04d0 2024-09-11T01:01:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5adf569740 1 -- 172.21.15.78:0/3597073313 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5acc0ac2a0 con 0x5630c66b04d0 2024-09-11T01:01:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5abaff5640 1 -- 172.21.15.78:0/3597073313 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5ac8012e00 con 0x5630c66b04d0 2024-09-11T01:01:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5abaff5640 1 -- 172.21.15.78:0/3597073313 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5ac800fac0 con 0x5630c66b04d0 2024-09-11T01:01:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5adf569740 10 client.25346 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5adf569740 1 -- 172.21.15.78:0/3597073313 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5630c66cb3f0 con 0x5630c66b04d0 2024-09-11T01:01:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5abaff5640 1 -- 172.21.15.78:0/3597073313 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5acc0ac2a0 con 0x5630c66b04d0 2024-09-11T01:01:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5abaff5640 1 -- 172.21.15.78:0/3597073313 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f5ac8010c40 con 0x5630c66b04d0 2024-09-11T01:01:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5adf569740 10 client.25346 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5adf569740 10 client.25346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:01:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5adf569740 20 client.25346 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5adf569740 1 --2- 172.21.15.78:0/3597073313 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5630c66cc9c0 0x5630c66cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5adf569740 4 client.25346 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:01:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5adf569740 1 -- 172.21.15.78:0/3597073313 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630c6628df0 con 0x5630c66cc9c0 2024-09-11T01:01:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.324+0000 7f5ad659c640 1 --2- 172.21.15.78:0/3597073313 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5630c66cc9c0 0x5630c66cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.325+0000 7f5ad659c640 1 --2- 172.21.15.78:0/3597073313 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5630c66cc9c0 0x5630c66cedd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9ef026e4d58af7a server_cookie=d798dd608b7bcd08 in_seq=0 out_seq=0 2024-09-11T01:01:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.325+0000 7f5abaff5640 10 client.25346 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.325+0000 7f5abaff5640 1 -- 172.21.15.78:0/3597073313 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5630c6628df0 con 0x5630c66cc9c0 2024-09-11T01:01:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.325+0000 7f5abaff5640 10 client.25346 handle_command_reply: tid=0 2024-09-11T01:01:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.325+0000 7f5adf569740 1 client.25346 shutdown 2024-09-11T01:01:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.326+0000 7f5adf569740 1 -- 172.21.15.78:0/3597073313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b04d0 msgr2=0x5630c66bc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.326+0000 7f5adf569740 1 --2- 172.21.15.78:0/3597073313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b04d0 0x5630c66bc6d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8005f60 tx=0x7f5ac8012bb0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.326+0000 7f5adf569740 1 -- 172.21.15.78:0/3597073313 shutdown_connections 2024-09-11T01:01:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.326+0000 7f5adf569740 1 --2- 172.21.15.78:0/3597073313 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5630c66cc9c0 0x5630c66cedd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.326+0000 7f5adf569740 1 --2- 172.21.15.78:0/3597073313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66ba180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.326+0000 7f5adf569740 1 --2- 172.21.15.78:0/3597073313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b0df0 0x5630c66b9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.326+0000 7f5adf569740 1 --2- 172.21.15.78:0/3597073313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b04d0 0x5630c66bc6d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.326+0000 7f5abaff5640 0 client.25346 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:14.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.326+0000 7f5adf569740 1 -- 172.21.15.78:0/3597073313 >> 172.21.15.78:0/3597073313 conn(0x5630c66ae300 msgr2=0x5630c66b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:14.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.326+0000 7f5adf569740 1 -- 172.21.15.78:0/3597073313 shutdown_connections 2024-09-11T01:01:14.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.326+0000 7f5adf569740 1 -- 172.21.15.78:0/3597073313 wait complete. 2024-09-11T01:01:14.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.326+0000 7f5adf569740 20 client.25346 trim_cache size 0 max 16384 2024-09-11T01:01:14.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.340+0000 7f5adf569740 1 Processor -- start 2024-09-11T01:01:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.340+0000 7f5adf569740 1 -- start start 2024-09-11T01:01:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5adf569740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5adf569740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b0df0 0x5630c66ad310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5adf569740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66be170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5adf569740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81ac480 con 0x5630c66bcd80 2024-09-11T01:01:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5adf569740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81ae080 con 0x5630c66b0df0 2024-09-11T01:01:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5adf569740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81acb50 con 0x5630c66b04d0 2024-09-11T01:01:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5ad759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66be170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5ad759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66be170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49570/0 (socket says 172.21.15.78:49570) 2024-09-11T01:01:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5ad759e640 1 -- 172.21.15.78:0/1205289664 learned_addr learned my addr 172.21.15.78:0/1205289664 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5ad659c640 1 --2- 172.21.15.78:0/1205289664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b0df0 0x5630c66ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5ad6d9d640 1 --2- 172.21.15.78:0/1205289664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5ad759e640 1 -- 172.21.15.78:0/1205289664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 msgr2=0x5630c66c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5ad759e640 1 --2- 172.21.15.78:0/1205289664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5ad759e640 1 -- 172.21.15.78:0/1205289664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b0df0 msgr2=0x5630c66ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5ad759e640 1 --2- 172.21.15.78:0/1205289664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b0df0 0x5630c66ad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5ad759e640 1 -- 172.21.15.78:0/1205289664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c66af9c0 con 0x5630c66bcd80 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5ad659c640 1 --2- 172.21.15.78:0/1205289664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b0df0 0x5630c66ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.341+0000 7f5ad759e640 1 --2- 172.21.15.78:0/1205289664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66be170 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009ca0 tx=0x7f5ad0044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e5523ac55fb321 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.342+0000 7f5abcff9640 1 -- 172.21.15.78:0/1205289664 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0057070 con 0x5630c66bcd80 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.342+0000 7f5abcff9640 1 -- 172.21.15.78:0/1205289664 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad0036050 con 0x5630c66bcd80 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.342+0000 7f5abcff9640 1 -- 172.21.15.78:0/1205289664 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0030a60 con 0x5630c66bcd80 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.342+0000 7f5adf569740 1 -- 172.21.15.78:0/1205289664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 msgr2=0x5630c66be170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.342+0000 7f5adf569740 1 --2- 172.21.15.78:0/1205289664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66be170 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009ca0 tx=0x7f5ad0044000 comp rx=0 tx=0).stop 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 -- 172.21.15.78:0/1205289664 shutdown_connections 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 --2- 172.21.15.78:0/1205289664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66be170 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 --2- 172.21.15.78:0/1205289664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b0df0 0x5630c66ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 --2- 172.21.15.78:0/1205289664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 -- 172.21.15.78:0/1205289664 >> 172.21.15.78:0/1205289664 conn(0x5630c66ae300 msgr2=0x5630c66acc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 -- 172.21.15.78:0/1205289664 shutdown_connections 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 -- 172.21.15.78:0/1205289664 wait complete. 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 -- start start 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 Processor -- start 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b0df0 0x5630c66c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81ad8f0 con 0x5630c66bcd80 2024-09-11T01:01:14.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81ad220 con 0x5630c66b0df0 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81aa480 con 0x5630c66b04d0 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5adf569740 10 client.0 _ll_register_callbacks cb 0x7f5ac8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5ad759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5ad759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49584/0 (socket says 172.21.15.78:49584) 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5ad759e640 1 -- 172.21.15.78:0/2468442308 learned_addr learned my addr 172.21.15.78:0/2468442308 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5ad6d9d640 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.343+0000 7f5ad659c640 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b0df0 0x5630c66c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5ad6d9d640 1 -- 172.21.15.78:0/2468442308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b0df0 msgr2=0x5630c66c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5ad6d9d640 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b0df0 0x5630c66c51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5ad6d9d640 1 -- 172.21.15.78:0/2468442308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 msgr2=0x5630c66b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5ad6d9d640 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5ad6d9d640 1 -- 172.21.15.78:0/2468442308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c66b6280 con 0x5630c66b04d0 2024-09-11T01:01:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5ad759e640 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5ad6d9d640 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66c4c90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5ac801c420 tx=0x7f5ac80052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c71c8d5b2a99ca08 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5abaff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5abaff5640 1 -- 172.21.15.78:0/2468442308 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8029030 con 0x5630c66b04d0 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5adf569740 1 -- 172.21.15.78:0/2468442308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ad0036050 con 0x5630c66b04d0 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5abaff5640 1 -- 172.21.15.78:0/2468442308 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5ac8012e00 con 0x5630c66b04d0 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5abaff5640 1 -- 172.21.15.78:0/2468442308 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5ac80122e0 con 0x5630c66b04d0 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5adf569740 10 client.25339 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.344+0000 7f5adf569740 1 -- 172.21.15.78:0/2468442308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5630c66b7520 con 0x5630c66b04d0 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.345+0000 7f5abaff5640 1 -- 172.21.15.78:0/2468442308 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad0036050 con 0x5630c66b04d0 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.345+0000 7f5abaff5640 1 -- 172.21.15.78:0/2468442308 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f5ac8036050 con 0x5630c66b04d0 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.345+0000 7f5adf569740 10 client.25339 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.345+0000 7f5adf569740 10 client.25339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:01:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.345+0000 7f5adf569740 20 client.25339 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.345+0000 7f5adf569740 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5630c673a8b0 0x5630c673cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.345+0000 7f5adf569740 4 client.25339 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:01:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.345+0000 7f5adf569740 1 -- 172.21.15.78:0/2468442308 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630c66248a0 con 0x5630c673a8b0 2024-09-11T01:01:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.345+0000 7f5ad659c640 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5630c673a8b0 0x5630c673cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.345+0000 7f5abaff5640 10 client.25339 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.345+0000 7f5ad659c640 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5630c673a8b0 0x5630c673cca0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745f006440ab4a2a server_cookie=abca62c1390c0c31 in_seq=0 out_seq=0 2024-09-11T01:01:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5abaff5640 1 -- 172.21.15.78:0/2468442308 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630c66248a0 con 0x5630c673a8b0 2024-09-11T01:01:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5abaff5640 10 client.25339 handle_command_reply: tid=0 2024-09-11T01:01:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5adf569740 1 client.25339 shutdown 2024-09-11T01:01:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5adf569740 1 -- 172.21.15.78:0/2468442308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 msgr2=0x5630c66c4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5adf569740 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66c4c90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5ac801c420 tx=0x7f5ac80052f0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5adf569740 1 -- 172.21.15.78:0/2468442308 shutdown_connections 2024-09-11T01:01:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5adf569740 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5630c673a8b0 0x5630c673cca0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5adf569740 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630c66bcd80 0x5630c66b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5adf569740 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630c66b0df0 0x5630c66c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5abaff5640 0 client.25339 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5adf569740 1 --2- 172.21.15.78:0/2468442308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630c66b04d0 0x5630c66c4c90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5adf569740 1 -- 172.21.15.78:0/2468442308 >> 172.21.15.78:0/2468442308 conn(0x5630c66ae300 msgr2=0x5630c66ad420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5adf569740 1 -- 172.21.15.78:0/2468442308 shutdown_connections 2024-09-11T01:01:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5adf569740 1 -- 172.21.15.78:0/2468442308 wait complete. 2024-09-11T01:01:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5adf569740 20 client.25339 trim_cache size 0 max 16384 2024-09-11T01:01:14.359 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:01:14.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5add194640 1 -- 172.21.15.78:0/860368307 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5ac00b5450 msgr2=0x7f5ac00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:14.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5add194640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5ac00b5450 0x7f5ac00b7910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80fff30 tx=0x7f5ad0005d80 comp rx=0 tx=0).stop 2024-09-11T01:01:14.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5add194640 1 -- 172.21.15.78:0/860368307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ad81076c0 msgr2=0x7f5ad80fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:14.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.346+0000 7f5add194640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ad81076c0 0x7f5ad80fe3b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00e9f0 tx=0x7f5acc00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.347+0000 7f5add194640 1 -- 172.21.15.78:0/860368307 shutdown_connections 2024-09-11T01:01:14.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.347+0000 7f5add194640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5ac00b5450 0x7f5ac00b7910 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.347+0000 7f5add194640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ad8108040 0x7f5ad80fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.347+0000 7f5add194640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ad81076c0 0x7f5ad80fe3b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.347+0000 7f5add194640 1 --2- 172.21.15.78:0/860368307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ad8101cb0 0x7f5ad80fde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:14.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.347+0000 7f5add194640 1 -- 172.21.15.78:0/860368307 >> 172.21.15.78:0/860368307 conn(0x7f5ad80fd700 msgr2=0x7f5ad8104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:14.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.347+0000 7f5add194640 1 -- 172.21.15.78:0/860368307 shutdown_connections 2024-09-11T01:01:14.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:14.347+0000 7f5add194640 1 -- 172.21.15.78:0/860368307 wait complete. 2024-09-11T01:01:14.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 173784 11.66 MB/sec warmup 82 sec latency 39.516 ms 2024-09-11T01:01:15.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:15 smithi078 ceph-mon[31315]: pgmap v394: 97 pgs: 97 active+clean; 103 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 308 op/s 2024-09-11T01:01:15.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 175224 11.60 MB/sec warmup 83 sec latency 38.625 ms 2024-09-11T01:01:15.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:15 smithi110 ceph-mon[34712]: pgmap v394: 97 pgs: 97 active+clean; 103 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 308 op/s 2024-09-11T01:01:15.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:15 smithi112 ceph-mon[35384]: pgmap v394: 97 pgs: 97 active+clean; 103 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 308 op/s 2024-09-11T01:01:16.270 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:01:16.270 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:01:16.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 177782 11.63 MB/sec warmup 84 sec latency 12.656 ms 2024-09-11T01:01:16.533 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:17.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.033+0000 7f088b14e640 1 -- 172.21.15.78:0/115528716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0884103b30 msgr2=0x7f0884103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:17.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.033+0000 7f088b14e640 1 --2- 172.21.15.78:0/115528716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0884103b30 0x7f0884103f10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f087c009900 tx=0x7f087c02a410 comp rx=0 tx=0).stop 2024-09-11T01:01:17.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.033+0000 7f088b14e640 1 -- 172.21.15.78:0/115528716 shutdown_connections 2024-09-11T01:01:17.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.033+0000 7f088b14e640 1 --2- 172.21.15.78:0/115528716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f088410d0e0 0x7f088410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.033+0000 7f088b14e640 1 --2- 172.21.15.78:0/115528716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0884104530 0x7f088410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.033+0000 7f088b14e640 1 --2- 172.21.15.78:0/115528716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0884103b30 0x7f0884103f10 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.033+0000 7f088b14e640 1 -- 172.21.15.78:0/115528716 >> 172.21.15.78:0/115528716 conn(0x7f08840fd700 msgr2=0x7f08840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.033+0000 7f088b14e640 1 -- 172.21.15.78:0/115528716 shutdown_connections 2024-09-11T01:01:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.033+0000 7f088b14e640 1 -- 172.21.15.78:0/115528716 wait complete. 2024-09-11T01:01:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.033+0000 7f088b14e640 1 Processor -- start 2024-09-11T01:01:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f088b14e640 1 -- start start 2024-09-11T01:01:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f088b14e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0884103b30 0x7f08841019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f088b14e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0884104530 0x7f0884101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f088b14e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f088410d0e0 0x7f0884102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f088b14e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0884112d60 con 0x7f088410d0e0 2024-09-11T01:01:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f088b14e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0884112be0 con 0x7f0884103b30 2024-09-11T01:01:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f088b14e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0884112ee0 con 0x7f0884104530 2024-09-11T01:01:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f08896eb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f088410d0e0 0x7f0884102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f08896eb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f088410d0e0 0x7f0884102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49606/0 (socket says 172.21.15.78:49606) 2024-09-11T01:01:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f08896eb640 1 -- 172.21.15.78:0/1880144679 learned_addr learned my addr 172.21.15.78:0/1880144679 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f0888eea640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0884103b30 0x7f08841019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f0883fff640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0884104530 0x7f0884101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f08896eb640 1 -- 172.21.15.78:0/1880144679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0884104530 msgr2=0x7f0884101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f08896eb640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0884104530 0x7f0884101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f08896eb640 1 -- 172.21.15.78:0/1880144679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0884103b30 msgr2=0x7f08841019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f08896eb640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0884103b30 0x7f08841019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f08896eb640 1 -- 172.21.15.78:0/1880144679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08841a7670 con 0x7f088410d0e0 2024-09-11T01:01:17.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f0883fff640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0884104530 0x7f0884101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:17.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f08896eb640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f088410d0e0 0x7f0884102430 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f087400eb10 tx=0x7f087400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:17.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.034+0000 7f0881ffb640 1 -- 172.21.15.78:0/1880144679 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874004b20 con 0x7f088410d0e0 2024-09-11T01:01:17.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.035+0000 7f088b14e640 1 -- 172.21.15.78:0/1880144679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08841a7960 con 0x7f088410d0e0 2024-09-11T01:01:17.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.035+0000 7f0881ffb640 1 -- 172.21.15.78:0/1880144679 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0874004cc0 con 0x7f088410d0e0 2024-09-11T01:01:17.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.035+0000 7f0881ffb640 1 -- 172.21.15.78:0/1880144679 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874010670 con 0x7f088410d0e0 2024-09-11T01:01:17.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.035+0000 7f088b14e640 1 -- 172.21.15.78:0/1880144679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08841a7ea0 con 0x7f088410d0e0 2024-09-11T01:01:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.037+0000 7f0881ffb640 1 -- 172.21.15.78:0/1880144679 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0874005020 con 0x7f088410d0e0 2024-09-11T01:01:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f088d523740 1 Processor -- start 2024-09-11T01:01:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f088d523740 1 -- start start 2024-09-11T01:01:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f088d523740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152796ee0 0x5571527972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f088d523740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152797800 0x5571527a3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f088d523740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571527a3790 0x5571527a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f088d523740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55715270de60 con 0x557152797800 2024-09-11T01:01:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f088d523740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5571526fd290 con 0x5571527a3790 2024-09-11T01:01:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f088d523740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5571527010a0 con 0x557152796ee0 2024-09-11T01:01:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0881ffb640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08700b5450 0x7f08700b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0888eea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152797800 0x5571527a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0883fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152796ee0 0x5571527972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f08896eb640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08700b5450 0x7f08700b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0883fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152796ee0 0x5571527972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54012/0 (socket says 172.21.15.78:54012) 2024-09-11T01:01:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0888eea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152797800 0x5571527a3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49616/0 (socket says 172.21.15.78:49616) 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0883fff640 1 -- 172.21.15.78:0/2242523682 learned_addr learned my addr 172.21.15.78:0/2242523682 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0883fff640 1 --2- 172.21.15.78:0/2242523682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571527a3790 0x5571527a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0881ffb640 1 -- 172.21.15.78:0/1880144679 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f08740e65c0 con 0x7f088410d0e0 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0883fff640 1 -- 172.21.15.78:0/2242523682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571527a3790 msgr2=0x5571527a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0883fff640 1 --2- 172.21.15.78:0/2242523682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571527a3790 0x5571527a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0883fff640 1 -- 172.21.15.78:0/2242523682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152797800 msgr2=0x5571527a3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0883fff640 1 --2- 172.21.15.78:0/2242523682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152797800 0x5571527a3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.038+0000 7f0883fff640 1 -- 172.21.15.78:0/2242523682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55715245b930 con 0x557152796ee0 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.039+0000 7f08896eb640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08700b5450 0x7f08700b7910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f08740ac9e0 tx=0x7f0874079ed0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.039+0000 7f0883fff640 1 --2- 172.21.15.78:0/2242523682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152796ee0 0x5571527972c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f086c011c00 tx=0x7f086c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=868e255822352900 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.039+0000 7f087affd640 1 -- 172.21.15.78:0/2242523682 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c00fc50 con 0x557152796ee0 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.039+0000 7f087affd640 1 -- 172.21.15.78:0/2242523682 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f086c00fdf0 con 0x557152796ee0 2024-09-11T01:01:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.039+0000 7f087affd640 1 -- 172.21.15.78:0/2242523682 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c00e610 con 0x557152796ee0 2024-09-11T01:01:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.040+0000 7f088d523740 1 -- 172.21.15.78:0/2242523682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152796ee0 msgr2=0x5571527972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.040+0000 7f088d523740 1 --2- 172.21.15.78:0/2242523682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152796ee0 0x5571527972c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f086c011c00 tx=0x7f086c009b80 comp rx=0 tx=0).stop 2024-09-11T01:01:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.040+0000 7f088d523740 1 -- 172.21.15.78:0/2242523682 shutdown_connections 2024-09-11T01:01:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.040+0000 7f088d523740 1 --2- 172.21.15.78:0/2242523682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571527a3790 0x5571527a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.040+0000 7f088d523740 1 --2- 172.21.15.78:0/2242523682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152797800 0x5571527a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.040+0000 7f088d523740 1 --2- 172.21.15.78:0/2242523682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152796ee0 0x5571527972c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.040+0000 7f088d523740 1 -- 172.21.15.78:0/2242523682 >> 172.21.15.78:0/2242523682 conn(0x557152794d10 msgr2=0x557152795eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.040+0000 7f088d523740 1 -- 172.21.15.78:0/2242523682 shutdown_connections 2024-09-11T01:01:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.040+0000 7f088d523740 1 -- 172.21.15.78:0/2242523682 wait complete. 2024-09-11T01:01:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.040+0000 7f088d523740 1 -- start start 2024-09-11T01:01:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.040+0000 7f088d523740 1 Processor -- start 2024-09-11T01:01:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f088d523740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 0x5571527a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f088d523740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557152797800 0x5571527a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f088d523740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571527a3790 0x5571527a1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f088d523740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55715255e890 con 0x557152796ee0 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f088d523740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557152697d40 con 0x557152797800 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f088d523740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5571527a6440 con 0x5571527a3790 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f088d523740 10 client.0 _ll_register_callbacks cb 0x5571526fbc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0883fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 0x5571527a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0883fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 0x5571527a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49624/0 (socket says 172.21.15.78:49624) 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0883fff640 1 -- 172.21.15.78:0/2052873454 learned_addr learned my addr 172.21.15.78:0/2052873454 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0888eea640 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557152797800 0x5571527a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0883fff640 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571527a3790 0x5571527a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0883fff640 1 -- 172.21.15.78:0/2052873454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571527a3790 msgr2=0x5571527a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0883fff640 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571527a3790 0x5571527a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0883fff640 1 -- 172.21.15.78:0/2052873454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557152797800 msgr2=0x5571527a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0883fff640 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557152797800 0x5571527a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0883fff640 1 -- 172.21.15.78:0/2052873454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571527af8d0 con 0x557152796ee0 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0883fff640 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 0x5571527a07f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f086c005bc0 tx=0x7f086c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea786746e3aaf22b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0863fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0863fff640 1 -- 172.21.15.78:0/2052873454 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c0044d0 con 0x557152796ee0 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f0863fff640 1 -- 172.21.15.78:0/2052873454 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f086c0035f0 con 0x557152796ee0 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.041+0000 7f088d523740 1 -- 172.21.15.78:0/2052873454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f086c00fdf0 con 0x557152796ee0 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.042+0000 7f0863fff640 1 -- 172.21.15.78:0/2052873454 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f086c0037f0 con 0x557152796ee0 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.042+0000 7f088d523740 10 client.15960 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.042+0000 7f088d523740 1 -- 172.21.15.78:0/2052873454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5571527b2260 con 0x557152796ee0 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.042+0000 7f0863fff640 1 -- 172.21.15.78:0/2052873454 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f086c00fdf0 con 0x557152796ee0 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.042+0000 7f0863fff640 1 -- 172.21.15.78:0/2052873454 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f086c035050 con 0x557152796ee0 2024-09-11T01:01:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.042+0000 7f088d523740 10 client.15960 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.042+0000 7f088d523740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:01:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.042+0000 7f088d523740 20 client.15960 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.042+0000 7f088d523740 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5571527b3610 0x5571527b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.042+0000 7f088d523740 4 client.15960 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:01:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.042+0000 7f088d523740 1 -- 172.21.15.78:0/2052873454 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55715270fdb0 con 0x5571527b3610 2024-09-11T01:01:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.042+0000 7f0888eea640 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5571527b3610 0x5571527b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.043+0000 7f0863fff640 10 client.15960 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.043+0000 7f0888eea640 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5571527b3610 0x5571527b5a00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd39c74ac3b1cbf8 server_cookie=158b62658da4b997 in_seq=0 out_seq=0 2024-09-11T01:01:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.044+0000 7f0863fff640 1 -- 172.21.15.78:0/2052873454 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55715270fdb0 con 0x5571527b3610 2024-09-11T01:01:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.044+0000 7f0863fff640 10 client.15960 handle_command_reply: tid=0 2024-09-11T01:01:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.044+0000 7f088d523740 1 client.15960 shutdown 2024-09-11T01:01:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.045+0000 7f088d523740 1 -- 172.21.15.78:0/2052873454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 msgr2=0x5571527a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.045+0000 7f088d523740 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 0x5571527a07f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f086c005bc0 tx=0x7f086c0096a0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.045+0000 7f088d523740 1 -- 172.21.15.78:0/2052873454 shutdown_connections 2024-09-11T01:01:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.045+0000 7f088d523740 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571527a3790 0x5571527a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.045+0000 7f088d523740 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557152797800 0x5571527a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.045+0000 7f088d523740 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5571527b3610 0x5571527b5a00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.045+0000 7f0863fff640 0 client.15960 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.045+0000 7f088d523740 1 --2- 172.21.15.78:0/2052873454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 0x5571527a07f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.045+0000 7f088d523740 1 -- 172.21.15.78:0/2052873454 >> 172.21.15.78:0/2052873454 conn(0x557152794d10 msgr2=0x557152793f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.045+0000 7f088d523740 1 -- 172.21.15.78:0/2052873454 shutdown_connections 2024-09-11T01:01:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.045+0000 7f088d523740 1 -- 172.21.15.78:0/2052873454 wait complete. 2024-09-11T01:01:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.045+0000 7f088d523740 20 client.15960 trim_cache size 0 max 16384 2024-09-11T01:01:17.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f088d523740 1 Processor -- start 2024-09-11T01:01:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f088d523740 1 -- start start 2024-09-11T01:01:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f088d523740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557152796ee0 0x55715279e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f088d523740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 0x5571527bb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f088d523740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571527a3790 0x5571527bb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f088d523740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08841b0720 con 0x5571527a3790 2024-09-11T01:01:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f088d523740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f08841b2320 con 0x557152796ee0 2024-09-11T01:01:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f088d523740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08841b0df0 con 0x557152797800 2024-09-11T01:01:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f0883fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571527a3790 0x5571527bb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f0883fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557152796ee0 0x55715279e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f0883fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557152796ee0 0x55715279e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41194/0 (socket says 172.21.15.78:41194) 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f0888eea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 0x5571527bb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f0883fff640 1 -- 172.21.15.78:0/695016287 learned_addr learned my addr 172.21.15.78:0/695016287 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f0888eea640 1 -- 172.21.15.78:0/695016287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557152796ee0 msgr2=0x55715279e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f0888eea640 1 --2- 172.21.15.78:0/695016287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557152796ee0 0x55715279e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f0888eea640 1 -- 172.21.15.78:0/695016287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571527a3790 msgr2=0x5571527bb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f0888eea640 1 --2- 172.21.15.78:0/695016287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571527a3790 0x5571527bb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.063+0000 7f0888eea640 1 -- 172.21.15.78:0/695016287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55715279f750 con 0x557152797800 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.064+0000 7f0883fff640 1 --2- 172.21.15.78:0/695016287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557152796ee0 0x55715279e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.064+0000 7f0888eea640 1 --2- 172.21.15.78:0/695016287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 0x5571527bb120 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f087c006010 tx=0x7f087c0092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c225c16ef0e8cd13 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.064+0000 7f087a7fc640 1 -- 172.21.15.78:0/695016287 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c035040 con 0x557152797800 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.064+0000 7f087a7fc640 1 -- 172.21.15.78:0/695016287 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f087c004380 con 0x557152797800 2024-09-11T01:01:17.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.064+0000 7f087a7fc640 1 -- 172.21.15.78:0/695016287 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c004d80 con 0x557152797800 2024-09-11T01:01:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.064+0000 7f088d523740 1 -- 172.21.15.78:0/695016287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 msgr2=0x5571527bb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 --2- 172.21.15.78:0/695016287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 0x5571527bb120 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f087c006010 tx=0x7f087c0092d0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 -- 172.21.15.78:0/695016287 shutdown_connections 2024-09-11T01:01:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 --2- 172.21.15.78:0/695016287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571527a3790 0x5571527bb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 --2- 172.21.15.78:0/695016287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 0x5571527bb120 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 --2- 172.21.15.78:0/695016287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557152796ee0 0x55715279e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 -- 172.21.15.78:0/695016287 >> 172.21.15.78:0/695016287 conn(0x557152794d10 msgr2=0x55715279dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 -- 172.21.15.78:0/695016287 shutdown_connections 2024-09-11T01:01:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 -- 172.21.15.78:0/695016287 wait complete. 2024-09-11T01:01:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 -- start start 2024-09-11T01:01:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 Processor -- start 2024-09-11T01:01:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 0x5571527abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 0x5571527a6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571527a3790 0x5571527a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08841b1b90 con 0x557152796ee0 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f08841b14c0 con 0x5571527a3790 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08841ae700 con 0x557152797800 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.065+0000 7f088d523740 10 client.0 _ll_register_callbacks cb 0x7f086c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0883fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 0x5571527abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0888eea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 0x5571527a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0883fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 0x5571527abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49644/0 (socket says 172.21.15.78:49644) 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0883fff640 1 -- 172.21.15.78:0/3680411944 learned_addr learned my addr 172.21.15.78:0/3680411944 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0888eea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 0x5571527a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54030/0 (socket says 172.21.15.78:54030) 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0883fff640 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571527a3790 0x5571527a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0883fff640 1 -- 172.21.15.78:0/3680411944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 msgr2=0x5571527a6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0883fff640 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 0x5571527a6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0883fff640 1 -- 172.21.15.78:0/3680411944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571527a3790 msgr2=0x5571527a6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0883fff640 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571527a3790 0x5571527a6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0883fff640 1 -- 172.21.15.78:0/3680411944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571527a7240 con 0x557152796ee0 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0888eea640 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 0x5571527a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0883fff640 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 0x5571527abcc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f086c005bc0 tx=0x7f086c00fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5740669e2d22256a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0863fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0863fff640 1 -- 172.21.15.78:0/3680411944 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c01f440 con 0x557152796ee0 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f0863fff640 1 -- 172.21.15.78:0/3680411944 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f086c01f5e0 con 0x557152796ee0 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.066+0000 7f088d523740 1 -- 172.21.15.78:0/3680411944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f087c004380 con 0x557152796ee0 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.067+0000 7f0863fff640 1 -- 172.21.15.78:0/3680411944 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f086c01f7e0 con 0x557152796ee0 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.067+0000 7f088d523740 10 client.15966 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.067+0000 7f088d523740 1 -- 172.21.15.78:0/3680411944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5571528211f0 con 0x557152796ee0 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.067+0000 7f0863fff640 1 -- 172.21.15.78:0/3680411944 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f087c004380 con 0x557152796ee0 2024-09-11T01:01:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.067+0000 7f0863fff640 1 -- 172.21.15.78:0/3680411944 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f086c035040 con 0x557152796ee0 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.067+0000 7f088d523740 10 client.15966 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.067+0000 7f088d523740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.067+0000 7f088d523740 20 client.15966 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.067+0000 7f088d523740 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5571527b3610 0x557152823810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.067+0000 7f088d523740 4 client.15966 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.067+0000 7f088d523740 1 -- 172.21.15.78:0/3680411944 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55715270b860 con 0x5571527b3610 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.067+0000 7f0888eea640 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5571527b3610 0x557152823810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.068+0000 7f0863fff640 10 client.15966 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.068+0000 7f0888eea640 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5571527b3610 0x557152823810 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9732522e74d66b7 server_cookie=107464a051238a6d in_seq=0 out_seq=0 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f0863fff640 1 -- 172.21.15.78:0/3680411944 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55715270b860 con 0x5571527b3610 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f0863fff640 10 client.15966 handle_command_reply: tid=0 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f088d523740 1 client.15966 shutdown 2024-09-11T01:01:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f088d523740 1 -- 172.21.15.78:0/3680411944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 msgr2=0x5571527abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f088d523740 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 0x5571527abcc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f086c005bc0 tx=0x7f086c00fcd0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f088d523740 1 -- 172.21.15.78:0/3680411944 shutdown_connections 2024-09-11T01:01:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f088d523740 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571527a3790 0x5571527a6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f088d523740 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557152797800 0x5571527a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f088d523740 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5571527b3610 0x557152823810 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f088d523740 1 --2- 172.21.15.78:0/3680411944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557152796ee0 0x5571527abcc0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f088d523740 1 -- 172.21.15.78:0/3680411944 >> 172.21.15.78:0/3680411944 conn(0x557152794d10 msgr2=0x5571527a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f0863fff640 0 client.15966 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f088d523740 1 -- 172.21.15.78:0/3680411944 shutdown_connections 2024-09-11T01:01:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.069+0000 7f088d523740 1 -- 172.21.15.78:0/3680411944 wait complete. 2024-09-11T01:01:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088d523740 20 client.15966 trim_cache size 0 max 16384 2024-09-11T01:01:17.077 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:01:17.077 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:01:17.077 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "cca472d2-e33e-45fa-840a-6bc7ba469dca", 2024-09-11T01:01:17.077 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:01:17.077 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:01:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088b14e640 1 -- 172.21.15.78:0/1880144679 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08700b5450 msgr2=0x7f08700b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088b14e640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08700b5450 0x7f08700b7910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f08740ac9e0 tx=0x7f0874079ed0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088b14e640 1 -- 172.21.15.78:0/1880144679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f088410d0e0 msgr2=0x7f0884102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088b14e640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f088410d0e0 0x7f0884102430 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f087400eb10 tx=0x7f087400c2d0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088b14e640 1 -- 172.21.15.78:0/1880144679 shutdown_connections 2024-09-11T01:01:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088b14e640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08700b5450 0x7f08700b7910 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088b14e640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f088410d0e0 0x7f0884102430 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088b14e640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0884104530 0x7f0884101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088b14e640 1 --2- 172.21.15.78:0/1880144679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0884103b30 0x7f08841019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088b14e640 1 -- 172.21.15.78:0/1880144679 >> 172.21.15.78:0/1880144679 conn(0x7f08840fd700 msgr2=0x7f08840fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088b14e640 1 -- 172.21.15.78:0/1880144679 shutdown_connections 2024-09-11T01:01:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:17.070+0000 7f088b14e640 1 -- 172.21.15.78:0/1880144679 wait complete. 2024-09-11T01:01:17.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:17 smithi078 ceph-mon[31315]: pgmap v395: 97 pgs: 97 active+clean; 108 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-09-11T01:01:17.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:17 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:01:17.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:17 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:17.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:17 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:01:17.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 180139 11.62 MB/sec warmup 85 sec latency 26.801 ms 2024-09-11T01:01:17.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:17 smithi110 ceph-mon[34712]: pgmap v395: 97 pgs: 97 active+clean; 108 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-09-11T01:01:17.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:17 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:01:17.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:17 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:17.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:17 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:01:17.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:17 smithi112 ceph-mon[35384]: pgmap v395: 97 pgs: 97 active+clean; 108 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-09-11T01:01:17.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:17 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:01:17.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:17 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:17.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:17 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:01:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:01:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:01:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:01:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:01:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:01:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:01:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:01:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.385 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:01:18.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 181972 11.61 MB/sec warmup 86 sec latency 10.859 ms 2024-09-11T01:01:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:01:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:01:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:01:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:01:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:01:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:01:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:01:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:01:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:01:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:01:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:01:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:01:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:01:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:01:18.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:01:18.645 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:19.324 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:19 smithi078 ceph-mon[31315]: pgmap v396: 97 pgs: 97 active+clean; 110 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 306 op/s 2024-09-11T01:01:19.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.154+0000 7fd569d6d640 1 -- 172.21.15.78:0/1539043657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd564108020 msgr2=0x7fd56410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:19.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.154+0000 7fd569d6d640 1 --2- 172.21.15.78:0/1539043657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd564108020 0x7fd56410e820 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009ca0 tx=0x7fd55c02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.154+0000 7fd569d6d640 1 -- 172.21.15.78:0/1539043657 shutdown_connections 2024-09-11T01:01:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.154+0000 7fd569d6d640 1 --2- 172.21.15.78:0/1539043657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd564108020 0x7fd56410e820 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.154+0000 7fd569d6d640 1 --2- 172.21.15.78:0/1539043657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5641076a0 0x7fd564107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.154+0000 7fd569d6d640 1 --2- 172.21.15.78:0/1539043657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd564101c90 0x7fd564102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.154+0000 7fd569d6d640 1 -- 172.21.15.78:0/1539043657 >> 172.21.15.78:0/1539043657 conn(0x7fd5640fd700 msgr2=0x7fd5640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.154+0000 7fd569d6d640 1 -- 172.21.15.78:0/1539043657 shutdown_connections 2024-09-11T01:01:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.154+0000 7fd569d6d640 1 -- 172.21.15.78:0/1539043657 wait complete. 2024-09-11T01:01:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd569d6d640 1 Processor -- start 2024-09-11T01:01:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd569d6d640 1 -- start start 2024-09-11T01:01:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd569d6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564101c90 0x7fd56419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd569d6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5641076a0 0x7fd56419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd569d6d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd564108020 0x7fd56419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd569d6d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd564111ca0 con 0x7fd5641076a0 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd569d6d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd564111b20 con 0x7fd564108020 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd569d6d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd564111e20 con 0x7fd564101c90 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5641076a0 0x7fd56419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd5637fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564101c90 0x7fd56419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd5637fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564101c90 0x7fd56419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54046/0 (socket says 172.21.15.78:54046) 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd5637fe640 1 -- 172.21.15.78:0/148047012 learned_addr learned my addr 172.21.15.78:0/148047012 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd563fff640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd564108020 0x7fd56419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd5637fe640 1 -- 172.21.15.78:0/148047012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd564108020 msgr2=0x7fd56419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd5637fe640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd564108020 0x7fd56419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd5637fe640 1 -- 172.21.15.78:0/148047012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5641076a0 msgr2=0x7fd56419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd5637fe640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5641076a0 0x7fd56419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd5637fe640 1 -- 172.21.15.78:0/148047012 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5641a5430 con 0x7fd564101c90 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.155+0000 7fd55bfff640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5641076a0 0x7fd56419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:19.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.156+0000 7fd5637fe640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564101c90 0x7fd56419a030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd55400e9f0 tx=0x7fd55400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.157+0000 7fd5617fa640 1 -- 172.21.15.78:0/148047012 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55400cd80 con 0x7fd564101c90 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.157+0000 7fd5617fa640 1 -- 172.21.15.78:0/148047012 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5540108e0 con 0x7fd564101c90 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.157+0000 7fd569d6d640 1 -- 172.21.15.78:0/148047012 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5641a5720 con 0x7fd564101c90 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.157+0000 7fd5617fa640 1 -- 172.21.15.78:0/148047012 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554018980 con 0x7fd564101c90 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.157+0000 7fd569d6d640 1 -- 172.21.15.78:0/148047012 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5641a5cb0 con 0x7fd564101c90 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd56c142740 1 Processor -- start 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd56c142740 1 -- start start 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd5617fa640 1 -- 172.21.15.78:0/148047012 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd554010430 con 0x7fd564101c90 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd56c142740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4e0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd56c142740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4e0da0 0x5593de4ec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd56c142740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4ecd30 0x5593de4ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd56c142740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5593de456e50 con 0x5593de4ecd30 2024-09-11T01:01:19.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd56c142740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5593de446280 con 0x5593de4e0480 2024-09-11T01:01:19.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd56c142740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5593de44a090 con 0x5593de4e0da0 2024-09-11T01:01:19.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd563fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd563fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4e0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41226/0 (socket says 172.21.15.78:41226) 2024-09-11T01:01:19.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4ecd30 0x5593de4ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd563fff640 1 -- 172.21.15.78:0/2505490019 learned_addr learned my addr 172.21.15.78:0/2505490019 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:19.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.160+0000 7fd5637fe640 1 --2- 172.21.15.78:0/2505490019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4e0da0 0x5593de4ec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.161+0000 7fd563fff640 1 -- 172.21.15.78:0/2505490019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4e0da0 msgr2=0x5593de4ec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:19.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.161+0000 7fd563fff640 1 --2- 172.21.15.78:0/2505490019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4e0da0 0x5593de4ec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.161+0000 7fd563fff640 1 -- 172.21.15.78:0/2505490019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4ecd30 msgr2=0x5593de4ef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:19.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.161+0000 7fd563fff640 1 --2- 172.21.15.78:0/2505490019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4ecd30 0x5593de4ef120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.161+0000 7fd563fff640 1 -- 172.21.15.78:0/2505490019 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593de1a4650 con 0x5593de4e0480 2024-09-11T01:01:19.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.161+0000 7fd563fff640 1 --2- 172.21.15.78:0/2505490019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4e0860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009410 tx=0x7fd55c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cb8b189703d5f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:19.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.161+0000 7fd5617fa640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd5500b54d0 0x7fd5500b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.161+0000 7fd559ffb640 1 -- 172.21.15.78:0/2505490019 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c031cf0 con 0x5593de4e0480 2024-09-11T01:01:19.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.161+0000 7fd559ffb640 1 -- 172.21.15.78:0/2505490019 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd55c036030 con 0x5593de4e0480 2024-09-11T01:01:19.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.161+0000 7fd559ffb640 1 -- 172.21.15.78:0/2505490019 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c004040 con 0x5593de4e0480 2024-09-11T01:01:19.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.161+0000 7fd5637fe640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd5500b54d0 0x7fd5500b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.161+0000 7fd5617fa640 1 -- 172.21.15.78:0/148047012 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd5540e61d0 con 0x7fd564101c90 2024-09-11T01:01:19.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.162+0000 7fd56c142740 1 -- 172.21.15.78:0/2505490019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 msgr2=0x5593de4e0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:19.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.162+0000 7fd5637fe640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd5500b54d0 0x7fd5500b7990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd5540e3a40 tx=0x7fd5540e9d20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:19.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.162+0000 7fd56c142740 1 --2- 172.21.15.78:0/2505490019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4e0860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009410 tx=0x7fd55c005ef0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.162+0000 7fd56c142740 1 -- 172.21.15.78:0/2505490019 shutdown_connections 2024-09-11T01:01:19.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.162+0000 7fd56c142740 1 --2- 172.21.15.78:0/2505490019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4ecd30 0x5593de4ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.162+0000 7fd56c142740 1 --2- 172.21.15.78:0/2505490019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4e0da0 0x5593de4ec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.162+0000 7fd56c142740 1 --2- 172.21.15.78:0/2505490019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4e0860 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.162+0000 7fd56c142740 1 -- 172.21.15.78:0/2505490019 >> 172.21.15.78:0/2505490019 conn(0x5593de4de2b0 msgr2=0x5593de4df450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.162+0000 7fd56c142740 1 -- 172.21.15.78:0/2505490019 shutdown_connections 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.162+0000 7fd56c142740 1 -- 172.21.15.78:0/2505490019 wait complete. 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd56c142740 1 -- start start 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd56c142740 1 Processor -- start 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd56c142740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd56c142740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4e0da0 0x5593de4e9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd56c142740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4ecd30 0x5593de4ea020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd56c142740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5593de2a7700 con 0x5593de4e0da0 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd56c142740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5593de3e1110 con 0x5593de4e0480 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd56c142740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5593de4ef7d0 con 0x5593de4ecd30 2024-09-11T01:01:19.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd56c142740 10 client.0 _ll_register_callbacks cb 0x5593de3fa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:19.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd563fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4e0da0 0x5593de4e9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd563fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4ec5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41230/0 (socket says 172.21.15.78:41230) 2024-09-11T01:01:19.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd563fff640 1 -- 172.21.15.78:0/2851111068 learned_addr learned my addr 172.21.15.78:0/2851111068 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:19.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.163+0000 7fd563fff640 1 --2- 172.21.15.78:0/2851111068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4ecd30 0x5593de4ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd563fff640 1 -- 172.21.15.78:0/2851111068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4ecd30 msgr2=0x5593de4ea020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:19.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd563fff640 1 --2- 172.21.15.78:0/2851111068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4ecd30 0x5593de4ea020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd563fff640 1 -- 172.21.15.78:0/2851111068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4e0da0 msgr2=0x5593de4e9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:19.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd563fff640 1 --2- 172.21.15.78:0/2851111068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4e0da0 0x5593de4e9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd563fff640 1 -- 172.21.15.78:0/2851111068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593de40ec40 con 0x5593de4e0480 2024-09-11T01:01:19.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd563fff640 1 --2- 172.21.15.78:0/2851111068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4ec5f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd55c02ade0 tx=0x7fd55c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48c7732a99975b09 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:19.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:01:19.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd542ffd640 1 -- 172.21.15.78:0/2851111068 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c031920 con 0x5593de4e0480 2024-09-11T01:01:19.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd56c142740 1 -- 172.21.15.78:0/2851111068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd55c036030 con 0x5593de4e0480 2024-09-11T01:01:19.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd542ffd640 1 -- 172.21.15.78:0/2851111068 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd55c004b40 con 0x5593de4e0480 2024-09-11T01:01:19.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd542ffd640 1 -- 172.21.15.78:0/2851111068 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd55c004d40 con 0x5593de4e0480 2024-09-11T01:01:19.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd56c142740 10 client.25382 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:19.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd56c142740 1 -- 172.21.15.78:0/2851111068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5593de4fb0b0 con 0x5593de4e0480 2024-09-11T01:01:19.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.164+0000 7fd542ffd640 1 -- 172.21.15.78:0/2851111068 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd55c036030 con 0x5593de4e0480 2024-09-11T01:01:19.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.165+0000 7fd542ffd640 1 -- 172.21.15.78:0/2851111068 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fd55c043070 con 0x5593de4e0480 2024-09-11T01:01:19.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.165+0000 7fd56c142740 10 client.25382 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:19.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.165+0000 7fd56c142740 10 client.25382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:01:19.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.165+0000 7fd56c142740 20 client.25382 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:19.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.165+0000 7fd56c142740 1 --2- 172.21.15.78:0/2851111068 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5593de4fc400 0x5593de4fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.165+0000 7fd56c142740 4 client.25382 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.165+0000 7fd56c142740 1 -- 172.21.15.78:0/2851111068 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593de458da0 con 0x5593de4fc400 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.165+0000 7fd55bfff640 1 --2- 172.21.15.78:0/2851111068 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5593de4fc400 0x5593de4fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.165+0000 7fd542ffd640 10 client.25382 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.165+0000 7fd55bfff640 1 --2- 172.21.15.78:0/2851111068 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5593de4fc400 0x5593de4fe7f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3741b7189af9b3 server_cookie=41cb608f9734a211 in_seq=0 out_seq=0 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd542ffd640 1 -- 172.21.15.78:0/2851111068 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5593de458da0 con 0x5593de4fc400 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd542ffd640 10 client.25382 handle_command_reply: tid=0 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd56c142740 1 client.25382 shutdown 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd56c142740 1 -- 172.21.15.78:0/2851111068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 msgr2=0x5593de4ec5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd56c142740 1 --2- 172.21.15.78:0/2851111068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4ec5f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd55c02ade0 tx=0x7fd55c003b90 comp rx=0 tx=0).stop 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd56c142740 1 -- 172.21.15.78:0/2851111068 shutdown_connections 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd56c142740 1 --2- 172.21.15.78:0/2851111068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4ecd30 0x5593de4ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd56c142740 1 --2- 172.21.15.78:0/2851111068 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5593de4fc400 0x5593de4fe7f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd56c142740 1 --2- 172.21.15.78:0/2851111068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4e0da0 0x5593de4e9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd56c142740 1 --2- 172.21.15.78:0/2851111068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4ec5f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd542ffd640 0 client.25382 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:19.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd56c142740 1 -- 172.21.15.78:0/2851111068 >> 172.21.15.78:0/2851111068 conn(0x5593de4de2b0 msgr2=0x5593de4e3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:19.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd56c142740 1 -- 172.21.15.78:0/2851111068 shutdown_connections 2024-09-11T01:01:19.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.166+0000 7fd56c142740 1 -- 172.21.15.78:0/2851111068 wait complete. 2024-09-11T01:01:19.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.167+0000 7fd56c142740 20 client.25382 trim_cache size 0 max 16384 2024-09-11T01:01:19.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.186+0000 7fd56c142740 1 Processor -- start 2024-09-11T01:01:19.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.186+0000 7fd56c142740 1 -- start start 2024-09-11T01:01:19.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.186+0000 7fd56c142740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4f8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.186+0000 7fd56c142740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4e0da0 0x5593de4eb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.186+0000 7fd56c142740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4ecd30 0x5593de4ee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.186+0000 7fd56c142740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641ac290 con 0x5593de4e0da0 2024-09-11T01:01:19.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.186+0000 7fd56c142740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641ade90 con 0x5593de4e0480 2024-09-11T01:01:19.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.186+0000 7fd56c142740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641ac960 con 0x5593de4ecd30 2024-09-11T01:01:19.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.186+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4e0da0 0x5593de4eb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4e0da0 0x5593de4eb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49678/0 (socket says 172.21.15.78:49678) 2024-09-11T01:01:19.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd563fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4f8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd55bfff640 1 -- 172.21.15.78:0/3273299127 learned_addr learned my addr 172.21.15.78:0/3273299127 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:19.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd563fff640 1 --2- 172.21.15.78:0/3273299127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4ecd30 0x5593de4ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd55bfff640 1 -- 172.21.15.78:0/3273299127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4ecd30 msgr2=0x5593de4ee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:19.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd55bfff640 1 --2- 172.21.15.78:0/3273299127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4ecd30 0x5593de4ee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd55bfff640 1 -- 172.21.15.78:0/3273299127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 msgr2=0x5593de4f8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:19.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd55bfff640 1 --2- 172.21.15.78:0/3273299127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4f8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd55bfff640 1 -- 172.21.15.78:0/3273299127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593de4e97f0 con 0x5593de4e0da0 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd563fff640 1 --2- 172.21.15.78:0/3273299127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4f8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd55bfff640 1 --2- 172.21.15.78:0/3273299127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4e0da0 0x5593de4eb780 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x5593de4ddc70 tx=0x7fd54c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78611b76fd21991f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd5597fa640 1 -- 172.21.15.78:0/3273299127 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c003c40 con 0x5593de4e0da0 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd5597fa640 1 -- 172.21.15.78:0/3273299127 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd54c003de0 con 0x5593de4e0da0 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.187+0000 7fd5597fa640 1 -- 172.21.15.78:0/3273299127 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c005020 con 0x5593de4e0da0 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.188+0000 7fd56c142740 1 -- 172.21.15.78:0/3273299127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4e0da0 msgr2=0x5593de4eb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.188+0000 7fd56c142740 1 --2- 172.21.15.78:0/3273299127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4e0da0 0x5593de4eb780 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x5593de4ddc70 tx=0x7fd54c005f90 comp rx=0 tx=0).stop 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.188+0000 7fd56c142740 1 -- 172.21.15.78:0/3273299127 shutdown_connections 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.188+0000 7fd56c142740 1 --2- 172.21.15.78:0/3273299127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4ecd30 0x5593de4ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.188+0000 7fd56c142740 1 --2- 172.21.15.78:0/3273299127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4e0da0 0x5593de4eb780 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.188+0000 7fd56c142740 1 --2- 172.21.15.78:0/3273299127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0480 0x5593de4f8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.188+0000 7fd56c142740 1 -- 172.21.15.78:0/3273299127 >> 172.21.15.78:0/3273299127 conn(0x5593de4de2b0 msgr2=0x5593de4eb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:19.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.188+0000 7fd56c142740 1 -- 172.21.15.78:0/3273299127 shutdown_connections 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.188+0000 7fd56c142740 1 -- 172.21.15.78:0/3273299127 wait complete. 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.188+0000 7fd56c142740 1 -- start start 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd56c142740 1 Processor -- start 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd56c142740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4e0480 0x5593de4f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd56c142740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0da0 0x5593de4efa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd56c142740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4ecd30 0x5593de4eff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd56c142740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641ad700 con 0x5593de4ecd30 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd56c142740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641ad030 con 0x5593de4e0da0 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd56c142740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641aa270 con 0x5593de4e0480 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd56c142740 10 client.0 _ll_register_callbacks cb 0x7fd54c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0da0 0x5593de4efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd563fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4ecd30 0x5593de4eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0da0 0x5593de4efa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41246/0 (socket says 172.21.15.78:41246) 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd55bfff640 1 -- 172.21.15.78:0/399211730 learned_addr learned my addr 172.21.15.78:0/399211730 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd563fff640 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4e0480 0x5593de4f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd55bfff640 1 -- 172.21.15.78:0/399211730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4e0480 msgr2=0x5593de4f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd55bfff640 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4e0480 0x5593de4f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd55bfff640 1 -- 172.21.15.78:0/399211730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4ecd30 msgr2=0x5593de4eff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd55bfff640 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4ecd30 0x5593de4eff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd55bfff640 1 -- 172.21.15.78:0/399211730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593de4f0620 con 0x5593de4e0da0 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.189+0000 7fd563fff640 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4ecd30 0x5593de4eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.190+0000 7fd55bfff640 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0da0 0x5593de4efa20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd54c004650 tx=0x7fd54c003a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46779e3b148d3cef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.190+0000 7fd542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.190+0000 7fd542ffd640 1 -- 172.21.15.78:0/399211730 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c038060 con 0x5593de4e0da0 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.190+0000 7fd542ffd640 1 -- 172.21.15.78:0/399211730 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd54c003bb0 con 0x5593de4e0da0 2024-09-11T01:01:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.190+0000 7fd56c142740 1 -- 172.21.15.78:0/399211730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd54c003de0 con 0x5593de4e0da0 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.190+0000 7fd542ffd640 1 -- 172.21.15.78:0/399211730 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd54c014840 con 0x5593de4e0da0 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.190+0000 7fd56c142740 10 client.25388 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.190+0000 7fd56c142740 1 -- 172.21.15.78:0/399211730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5593de56a0d0 con 0x5593de4e0da0 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.190+0000 7fd542ffd640 1 -- 172.21.15.78:0/399211730 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd54c003de0 con 0x5593de4e0da0 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.190+0000 7fd542ffd640 1 -- 172.21.15.78:0/399211730 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fd54c03b030 con 0x5593de4e0da0 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.190+0000 7fd56c142740 10 client.25388 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd56c142740 10 client.25388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd56c142740 20 client.25388 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd56c142740 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5593de56a880 0x5593de56cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd56c142740 4 client.25388 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd56c142740 1 -- 172.21.15.78:0/399211730 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5593de454850 con 0x5593de56a880 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd563fff640 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5593de56a880 0x5593de56cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd542ffd640 10 client.25388 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd563fff640 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5593de56a880 0x5593de56cca0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d819f3703f165e2d server_cookie=b4000f5a8fe2857c in_seq=0 out_seq=0 2024-09-11T01:01:19.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd542ffd640 1 -- 172.21.15.78:0/399211730 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x5593de454850 con 0x5593de56a880 2024-09-11T01:01:19.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd542ffd640 10 client.25388 handle_command_reply: tid=0 2024-09-11T01:01:19.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd56c142740 1 client.25388 shutdown 2024-09-11T01:01:19.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd56c142740 1 -- 172.21.15.78:0/399211730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0da0 msgr2=0x5593de4efa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:19.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.191+0000 7fd56c142740 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0da0 0x5593de4efa20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd54c004650 tx=0x7fd54c003a40 comp rx=0 tx=0).stop 2024-09-11T01:01:19.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd56c142740 1 -- 172.21.15.78:0/399211730 shutdown_connections 2024-09-11T01:01:19.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd56c142740 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5593de56a880 0x5593de56cca0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd56c142740 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5593de4ecd30 0x5593de4eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd56c142740 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593de4e0da0 0x5593de4efa20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd56c142740 1 --2- 172.21.15.78:0/399211730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593de4e0480 0x5593de4f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd542ffd640 0 client.25388 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:19.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd56c142740 1 -- 172.21.15.78:0/399211730 >> 172.21.15.78:0/399211730 conn(0x5593de4de2b0 msgr2=0x5593de4eb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:19.361 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:01:19.361 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:01:19.361 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:01:19.361 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:01:19.361 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:01:19.361 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "cca472d2-e33e-45fa-840a-6bc7ba469dca", 2024-09-11T01:01:19.361 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:01:19.361 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:01:19.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:01:19.361 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:01:19.362 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:01:19.363 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:01:19.364 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:01:19.364 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:01:19.364 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:01:19.364 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:01:19.364 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:01:19.364 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:01:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd56c142740 1 -- 172.21.15.78:0/399211730 shutdown_connections 2024-09-11T01:01:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd56c142740 1 -- 172.21.15.78:0/399211730 wait complete. 2024-09-11T01:01:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd56c142740 20 client.25388 trim_cache size 0 max 16384 2024-09-11T01:01:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd569d6d640 1 -- 172.21.15.78:0/148047012 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd5500b54d0 msgr2=0x7fd5500b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd569d6d640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd5500b54d0 0x7fd5500b7990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd5540e3a40 tx=0x7fd5540e9d20 comp rx=0 tx=0).stop 2024-09-11T01:01:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd569d6d640 1 -- 172.21.15.78:0/148047012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564101c90 msgr2=0x7fd56419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd569d6d640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564101c90 0x7fd56419a030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd55400e9f0 tx=0x7fd55400eeb0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd569d6d640 1 -- 172.21.15.78:0/148047012 shutdown_connections 2024-09-11T01:01:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd569d6d640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd5500b54d0 0x7fd5500b7990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd569d6d640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd564108020 0x7fd56419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd569d6d640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5641076a0 0x7fd56419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd569d6d640 1 --2- 172.21.15.78:0/148047012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564101c90 0x7fd56419a030 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd569d6d640 1 -- 172.21.15.78:0/148047012 >> 172.21.15.78:0/148047012 conn(0x7fd5640fd700 msgr2=0x7fd5640fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd569d6d640 1 -- 172.21.15.78:0/148047012 shutdown_connections 2024-09-11T01:01:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:19.192+0000 7fd569d6d640 1 -- 172.21.15.78:0/148047012 wait complete. 2024-09-11T01:01:19.420 INFO:tasks.workunit.client.0.smithi078.stdout: 1 184713 11.67 MB/sec warmup 87 sec latency 23.306 ms 2024-09-11T01:01:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:19 smithi110 ceph-mon[34712]: pgmap v396: 97 pgs: 97 active+clean; 110 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 306 op/s 2024-09-11T01:01:19.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:19 smithi112 ceph-mon[35384]: pgmap v396: 97 pgs: 97 active+clean; 110 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 306 op/s 2024-09-11T01:01:20.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 186705 11.60 MB/sec warmup 88 sec latency 39.665 ms 2024-09-11T01:01:20.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:20 smithi110 ceph-mon[34712]: pgmap v397: 97 pgs: 97 active+clean; 111 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 311 op/s 2024-09-11T01:01:20.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:20 smithi078 ceph-mon[31315]: pgmap v397: 97 pgs: 97 active+clean; 111 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 311 op/s 2024-09-11T01:01:20.469 INFO:tasks.cephfs.filesystem:scrub status for tag:cca472d2-e33e-45fa-840a-6bc7ba469dca - {'path': '/', 'tag': 'cca472d2-e33e-45fa-840a-6bc7ba469dca', 'options': 'recursive,force'} 2024-09-11T01:01:20.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:20 smithi112 ceph-mon[35384]: pgmap v397: 97 pgs: 97 active+clean; 111 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 311 op/s 2024-09-11T01:01:21.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 188566 11.62 MB/sec warmup 89 sec latency 15.101 ms 2024-09-11T01:01:22.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 191277 11.65 MB/sec warmup 90 sec latency 12.913 ms 2024-09-11T01:01:23.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 192784 11.60 MB/sec warmup 91 sec latency 14.526 ms 2024-09-11T01:01:23.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:23 smithi110 ceph-mon[34712]: pgmap v398: 97 pgs: 97 active+clean; 115 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-09-11T01:01:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:23 smithi112 ceph-mon[35384]: pgmap v398: 97 pgs: 97 active+clean; 115 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-09-11T01:01:23.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:23 smithi078 ceph-mon[31315]: pgmap v398: 97 pgs: 97 active+clean; 115 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-09-11T01:01:24.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 195460 11.66 MB/sec warmup 92 sec latency 11.705 ms 2024-09-11T01:01:25.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 197697 11.62 MB/sec warmup 93 sec latency 56.330 ms 2024-09-11T01:01:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:25 smithi110 ceph-mon[34712]: pgmap v399: 97 pgs: 97 active+clean; 115 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-11T01:01:25.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:25 smithi112 ceph-mon[35384]: pgmap v399: 97 pgs: 97 active+clean; 115 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-11T01:01:25.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:25 smithi078 ceph-mon[31315]: pgmap v399: 97 pgs: 97 active+clean; 115 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-11T01:01:26.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 199607 11.62 MB/sec warmup 94 sec latency 11.135 ms 2024-09-11T01:01:27.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 202455 11.69 MB/sec warmup 95 sec latency 10.164 ms 2024-09-11T01:01:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:27 smithi110 ceph-mon[34712]: pgmap v400: 97 pgs: 97 active+clean; 120 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 377 op/s 2024-09-11T01:01:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:27 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:01:27.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:27 smithi112 ceph-mon[35384]: pgmap v400: 97 pgs: 97 active+clean; 120 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 377 op/s 2024-09-11T01:01:27.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:27 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:01:27.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:27 smithi078 ceph-mon[31315]: pgmap v400: 97 pgs: 97 active+clean; 120 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 377 op/s 2024-09-11T01:01:27.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:27 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:01:28.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 204763 11.66 MB/sec warmup 96 sec latency 10.923 ms 2024-09-11T01:01:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:28 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:01:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:28 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:01:28.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:28 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:01:28.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:28 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:01:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:28 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:01:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:28 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:01:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:28 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:01:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:28 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:01:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:28 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:01:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:28 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:01:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:28 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:01:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:28 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:01:29.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 206632 11.66 MB/sec warmup 97 sec latency 12.115 ms 2024-09-11T01:01:29.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:29 smithi110 ceph-mon[34712]: pgmap v401: 97 pgs: 97 active+clean; 121 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-09-11T01:01:29.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:29 smithi112 ceph-mon[35384]: pgmap v401: 97 pgs: 97 active+clean; 121 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-09-11T01:01:29.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:29 smithi078 ceph-mon[31315]: pgmap v401: 97 pgs: 97 active+clean; 121 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-09-11T01:01:30.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 209369 11.71 MB/sec warmup 98 sec latency 36.279 ms 2024-09-11T01:01:31.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 211617 11.68 MB/sec warmup 99 sec latency 11.743 ms 2024-09-11T01:01:31.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:31 smithi110 ceph-mon[34712]: pgmap v402: 97 pgs: 97 active+clean; 124 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 360 op/s 2024-09-11T01:01:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:31 smithi112 ceph-mon[35384]: pgmap v402: 97 pgs: 97 active+clean; 124 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 360 op/s 2024-09-11T01:01:31.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:31 smithi078 ceph-mon[31315]: pgmap v402: 97 pgs: 97 active+clean; 124 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 360 op/s 2024-09-11T01:01:32.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 213420 11.68 MB/sec warmup 100 sec latency 9.788 ms 2024-09-11T01:01:33.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 216250 11.71 MB/sec warmup 101 sec latency 12.235 ms 2024-09-11T01:01:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:33 smithi110 ceph-mon[34712]: pgmap v403: 97 pgs: 97 active+clean; 126 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 420 op/s 2024-09-11T01:01:33.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:33 smithi112 ceph-mon[35384]: pgmap v403: 97 pgs: 97 active+clean; 126 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 420 op/s 2024-09-11T01:01:33.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:33 smithi078 ceph-mon[31315]: pgmap v403: 97 pgs: 97 active+clean; 126 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 420 op/s 2024-09-11T01:01:34.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 217915 11.69 MB/sec warmup 102 sec latency 10.914 ms 2024-09-11T01:01:35.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 220335 11.71 MB/sec warmup 103 sec latency 25.867 ms 2024-09-11T01:01:35.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:35 smithi110 ceph-mon[34712]: pgmap v404: 97 pgs: 97 active+clean; 127 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-09-11T01:01:35.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:35 smithi112 ceph-mon[35384]: pgmap v404: 97 pgs: 97 active+clean; 127 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-09-11T01:01:35.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:35 smithi078 ceph-mon[31315]: pgmap v404: 97 pgs: 97 active+clean; 127 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-09-11T01:01:36.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 222573 11.68 MB/sec warmup 104 sec latency 14.602 ms 2024-09-11T01:01:36.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:36 smithi110 ceph-mon[34712]: pgmap v405: 97 pgs: 97 active+clean; 130 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 361 op/s 2024-09-11T01:01:36.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:36 smithi112 ceph-mon[35384]: pgmap v405: 97 pgs: 97 active+clean; 130 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 361 op/s 2024-09-11T01:01:36.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:36 smithi078 ceph-mon[31315]: pgmap v405: 97 pgs: 97 active+clean; 130 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 361 op/s 2024-09-11T01:01:37.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 224457 11.68 MB/sec warmup 105 sec latency 10.935 ms 2024-09-11T01:01:38.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 227296 11.74 MB/sec warmup 106 sec latency 9.209 ms 2024-09-11T01:01:39.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 229433 11.70 MB/sec warmup 107 sec latency 10.429 ms 2024-09-11T01:01:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:39 smithi110 ceph-mon[34712]: pgmap v406: 97 pgs: 97 active+clean; 134 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 332 op/s 2024-09-11T01:01:39.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:39 smithi112 ceph-mon[35384]: pgmap v406: 97 pgs: 97 active+clean; 134 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 332 op/s 2024-09-11T01:01:39.488 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:39 smithi078 ceph-mon[31315]: pgmap v406: 97 pgs: 97 active+clean; 134 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 332 op/s 2024-09-11T01:01:40.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 231078 11.70 MB/sec warmup 108 sec latency 41.980 ms 2024-09-11T01:01:41.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 233627 11.71 MB/sec warmup 109 sec latency 14.057 ms 2024-09-11T01:01:41.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:41 smithi110 ceph-mon[34712]: pgmap v407: 97 pgs: 97 active+clean; 136 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 381 op/s 2024-09-11T01:01:41.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:41 smithi112 ceph-mon[35384]: pgmap v407: 97 pgs: 97 active+clean; 136 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 381 op/s 2024-09-11T01:01:41.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:41 smithi078 ceph-mon[31315]: pgmap v407: 97 pgs: 97 active+clean; 136 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 381 op/s 2024-09-11T01:01:42.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 235311 11.68 MB/sec warmup 110 sec latency 9.900 ms 2024-09-11T01:01:43.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 237961 11.73 MB/sec warmup 111 sec latency 9.884 ms 2024-09-11T01:01:43.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:43 smithi110 ceph-mon[34712]: pgmap v408: 97 pgs: 97 active+clean; 136 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 352 op/s 2024-09-11T01:01:43.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:43 smithi112 ceph-mon[35384]: pgmap v408: 97 pgs: 97 active+clean; 136 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 352 op/s 2024-09-11T01:01:43.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:43 smithi078 ceph-mon[31315]: pgmap v408: 97 pgs: 97 active+clean; 136 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 352 op/s 2024-09-11T01:01:44.421 INFO:tasks.workunit.client.0.smithi078.stdout: 1 240144 11.69 MB/sec warmup 112 sec latency 10.055 ms 2024-09-11T01:01:45.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 241791 11.69 MB/sec warmup 113 sec latency 25.026 ms 2024-09-11T01:01:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:45 smithi110 ceph-mon[34712]: pgmap v409: 97 pgs: 97 active+clean; 139 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 328 op/s 2024-09-11T01:01:45.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:45 smithi112 ceph-mon[35384]: pgmap v409: 97 pgs: 97 active+clean; 139 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 328 op/s 2024-09-11T01:01:45.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:45 smithi078 ceph-mon[31315]: pgmap v409: 97 pgs: 97 active+clean; 139 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 328 op/s 2024-09-11T01:01:46.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 244553 11.72 MB/sec warmup 114 sec latency 14.376 ms 2024-09-11T01:01:47.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 246049 11.67 MB/sec warmup 115 sec latency 10.317 ms 2024-09-11T01:01:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:47 smithi110 ceph-mon[34712]: pgmap v410: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 347 op/s 2024-09-11T01:01:47.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:47 smithi112 ceph-mon[35384]: pgmap v410: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 347 op/s 2024-09-11T01:01:47.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:47 smithi078 ceph-mon[31315]: pgmap v410: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 347 op/s 2024-09-11T01:01:48.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 248692 11.72 MB/sec warmup 116 sec latency 9.643 ms 2024-09-11T01:01:49.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 251086 11.71 MB/sec warmup 117 sec latency 11.959 ms 2024-09-11T01:01:49.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:49 smithi110 ceph-mon[34712]: pgmap v411: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 344 op/s 2024-09-11T01:01:49.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:49 smithi112 ceph-mon[35384]: pgmap v411: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 344 op/s 2024-09-11T01:01:49.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:49 smithi078 ceph-mon[31315]: pgmap v411: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 344 op/s 2024-09-11T01:01:50.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 252602 11.69 MB/sec warmup 118 sec latency 36.468 ms 2024-09-11T01:01:50.471 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:01:50.756 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:51.420 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:51 smithi078 ceph-mon[31315]: pgmap v412: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 351 op/s 2024-09-11T01:01:51.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 255366 11.73 MB/sec warmup 119 sec latency 20.133 ms 2024-09-11T01:01:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:51 smithi110 ceph-mon[34712]: pgmap v412: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 351 op/s 2024-09-11T01:01:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:51 smithi112 ceph-mon[35384]: pgmap v412: 97 pgs: 97 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 351 op/s 2024-09-11T01:01:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.622+0000 7f1e12a74640 1 -- 172.21.15.78:0/1406469449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c101c90 msgr2=0x7f1e0c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.622+0000 7f1e12a74640 1 --2- 172.21.15.78:0/1406469449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c101c90 0x7f1e0c102070 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009900 tx=0x7f1e0002a410 comp rx=0 tx=0).stop 2024-09-11T01:01:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.622+0000 7f1e12a74640 1 -- 172.21.15.78:0/1406469449 shutdown_connections 2024-09-11T01:01:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.622+0000 7f1e12a74640 1 --2- 172.21.15.78:0/1406469449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e0c108020 0x7f1e0c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.622+0000 7f1e12a74640 1 --2- 172.21.15.78:0/1406469449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e0c1076a0 0x7f1e0c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.622+0000 7f1e12a74640 1 --2- 172.21.15.78:0/1406469449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c101c90 0x7f1e0c102070 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.622+0000 7f1e12a74640 1 -- 172.21.15.78:0/1406469449 >> 172.21.15.78:0/1406469449 conn(0x7f1e0c0fd700 msgr2=0x7f1e0c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.622+0000 7f1e12a74640 1 -- 172.21.15.78:0/1406469449 shutdown_connections 2024-09-11T01:01:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.622+0000 7f1e12a74640 1 -- 172.21.15.78:0/1406469449 wait complete. 2024-09-11T01:01:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e12a74640 1 Processor -- start 2024-09-11T01:01:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e12a74640 1 -- start start 2024-09-11T01:01:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e12a74640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c101c90 0x7f1e0c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e12a74640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e0c1076a0 0x7f1e0c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e12a74640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e0c108020 0x7f1e0c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e12a74640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c111ce0 con 0x7f1e0c101c90 2024-09-11T01:01:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e12a74640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c111b60 con 0x7f1e0c1076a0 2024-09-11T01:01:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e12a74640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c111e60 con 0x7f1e0c108020 2024-09-11T01:01:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e10810640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c101c90 0x7f1e0c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e0c1076a0 0x7f1e0c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e0c1076a0 0x7f1e0c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:34086/0 (socket says 172.21.15.78:34086) 2024-09-11T01:01:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e10810640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c101c90 0x7f1e0c19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49652/0 (socket says 172.21.15.78:49652) 2024-09-11T01:01:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.623+0000 7f1e10810640 1 -- 172.21.15.78:0/3415876288 learned_addr learned my addr 172.21.15.78:0/3415876288 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.624+0000 7f1e11011640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e0c108020 0x7f1e0c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.624+0000 7f1e0bfff640 1 -- 172.21.15.78:0/3415876288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e0c108020 msgr2=0x7f1e0c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.624+0000 7f1e0bfff640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e0c108020 0x7f1e0c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.624+0000 7f1e0bfff640 1 -- 172.21.15.78:0/3415876288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c101c90 msgr2=0x7f1e0c19a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.624+0000 7f1e0bfff640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c101c90 0x7f1e0c19a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.624+0000 7f1e0bfff640 1 -- 172.21.15.78:0/3415876288 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c1a54b0 con 0x7f1e0c1076a0 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.624+0000 7f1e11011640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e0c108020 0x7f1e0c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.624+0000 7f1e0bfff640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e0c1076a0 0x7f1e0c19a610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9f0 tx=0x7f1df800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.624+0000 7f1e09ffb640 1 -- 172.21.15.78:0/3415876288 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df800ccc0 con 0x7f1e0c1076a0 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.624+0000 7f1e09ffb640 1 -- 172.21.15.78:0/3415876288 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df8005020 con 0x7f1e0c1076a0 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.624+0000 7f1e12a74640 1 -- 172.21.15.78:0/3415876288 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0c1a57a0 con 0x7f1e0c1076a0 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.624+0000 7f1e09ffb640 1 -- 172.21.15.78:0/3415876288 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8004a00 con 0x7f1e0c1076a0 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.625+0000 7f1e12a74640 1 -- 172.21.15.78:0/3415876288 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e0c1a5c90 con 0x7f1e0c1076a0 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.627+0000 7f1e09ffb640 1 -- 172.21.15.78:0/3415876288 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1df8004ba0 con 0x7f1e0c1076a0 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.627+0000 7f1e14e49740 1 Processor -- start 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.627+0000 7f1e14e49740 1 -- start start 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.627+0000 7f1e14e49740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e11480 0x556676e11860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.627+0000 7f1e14e49740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e21c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e14e49740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e22140 0x556676e24530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e14e49740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556676d87e50 con 0x556676e22140 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e14e49740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556676d77280 con 0x556676e11480 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e14e49740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556676d7b090 con 0x556676e11da0 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e11011640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e11480 0x556676e11860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e10810640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e21c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e10810640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e21c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:33124/0 (socket says 172.21.15.78:33124) 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e10810640 1 -- 172.21.15.78:0/2241473108 learned_addr learned my addr 172.21.15.78:0/2241473108 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e0bfff640 1 --2- 172.21.15.78:0/2241473108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e22140 0x556676e24530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e10810640 1 -- 172.21.15.78:0/2241473108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e11480 msgr2=0x556676e11860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e10810640 1 --2- 172.21.15.78:0/2241473108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e11480 0x556676e11860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e10810640 1 -- 172.21.15.78:0/2241473108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e22140 msgr2=0x556676e24530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:51.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e10810640 1 --2- 172.21.15.78:0/2241473108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e22140 0x556676e24530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e10810640 1 -- 172.21.15.78:0/2241473108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556676ad5650 con 0x556676e11da0 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e11011640 1 --2- 172.21.15.78:0/2241473108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e11480 0x556676e11860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e09ffb640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1dfc0b5450 0x7f1dfc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e11011640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1dfc0b5450 0x7f1dfc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e10810640 1 --2- 172.21.15.78:0/2241473108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e21c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009a20 tx=0x7f1e00031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87bc44c1139cf990 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1e09ffb640 1 -- 172.21.15.78:0/3415876288 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1df80e5f80 con 0x7f1e0c1076a0 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.628+0000 7f1df6ffd640 1 -- 172.21.15.78:0/2241473108 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00035040 con 0x556676e11da0 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.629+0000 7f1df6ffd640 1 -- 172.21.15.78:0/2241473108 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e00031760 con 0x556676e11da0 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.629+0000 7f1df6ffd640 1 -- 172.21.15.78:0/2241473108 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e000047a0 con 0x556676e11da0 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.629+0000 7f1e14e49740 1 -- 172.21.15.78:0/2241473108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 msgr2=0x556676e21c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.629+0000 7f1e14e49740 1 --2- 172.21.15.78:0/2241473108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e21c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009a20 tx=0x7f1e00031b10 comp rx=0 tx=0).stop 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.629+0000 7f1e11011640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1dfc0b5450 0x7f1dfc0b7910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009fd0 tx=0x7f1e04009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.629+0000 7f1e14e49740 1 -- 172.21.15.78:0/2241473108 shutdown_connections 2024-09-11T01:01:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.629+0000 7f1e14e49740 1 --2- 172.21.15.78:0/2241473108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e22140 0x556676e24530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.629+0000 7f1e14e49740 1 --2- 172.21.15.78:0/2241473108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e21c00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.629+0000 7f1e14e49740 1 --2- 172.21.15.78:0/2241473108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e11480 0x556676e11860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.629+0000 7f1e14e49740 1 -- 172.21.15.78:0/2241473108 >> 172.21.15.78:0/2241473108 conn(0x556676e0f2b0 msgr2=0x556676e10450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.629+0000 7f1e14e49740 1 -- 172.21.15.78:0/2241473108 shutdown_connections 2024-09-11T01:01:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.629+0000 7f1e14e49740 1 -- 172.21.15.78:0/2241473108 wait complete. 2024-09-11T01:01:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e14e49740 1 -- start start 2024-09-11T01:01:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e14e49740 1 Processor -- start 2024-09-11T01:01:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e14e49740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 0x556676e21a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e14e49740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e1ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e14e49740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e14e49740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556676bd8700 con 0x556676e11480 2024-09-11T01:01:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e14e49740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556676d12110 con 0x556676e22140 2024-09-11T01:01:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e14e49740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556676e24d90 con 0x556676e11da0 2024-09-11T01:01:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e14e49740 10 client.0 _ll_register_callbacks cb 0x556676d2b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e10810640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 0x556676e21a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e10810640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 0x556676e21a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49668/0 (socket says 172.21.15.78:49668) 2024-09-11T01:01:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e10810640 1 -- 172.21.15.78:0/1444408796 learned_addr learned my addr 172.21.15.78:0/1444408796 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e0bfff640 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e1ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.630+0000 7f1e11011640 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1e10810640 1 -- 172.21.15.78:0/1444408796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 msgr2=0x556676e1ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1e10810640 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e1ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1e10810640 1 -- 172.21.15.78:0/1444408796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 msgr2=0x556676e1f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1e10810640 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e1f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1e10810640 1 -- 172.21.15.78:0/1444408796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556676d3fc40 con 0x556676e11480 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1e0bfff640 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e1ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1e10810640 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 0x556676e21a00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1e00031440 tx=0x7f1e00005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f8825c848e7035 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1debfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1debfff640 1 -- 172.21.15.78:0/1444408796 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00035030 con 0x556676e11480 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1debfff640 1 -- 172.21.15.78:0/1444408796 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1e00031560 con 0x556676e11480 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1e14e49740 1 -- 172.21.15.78:0/1444408796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e00031760 con 0x556676e11480 2024-09-11T01:01:51.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1debfff640 1 -- 172.21.15.78:0/1444408796 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1e00031910 con 0x556676e11480 2024-09-11T01:01:51.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1e14e49740 10 client.15996 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:51.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1e14e49740 1 -- 172.21.15.78:0/1444408796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556676e30690 con 0x556676e11480 2024-09-11T01:01:51.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.631+0000 7f1debfff640 1 -- 172.21.15.78:0/1444408796 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e00031760 con 0x556676e11480 2024-09-11T01:01:51.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.632+0000 7f1debfff640 1 -- 172.21.15.78:0/1444408796 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1e000035f0 con 0x556676e11480 2024-09-11T01:01:51.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.632+0000 7f1e14e49740 10 client.15996 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:51.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.632+0000 7f1e14e49740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:01:51.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.632+0000 7f1e14e49740 20 client.15996 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:51.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.632+0000 7f1e14e49740 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x556676e319e0 0x556676e33dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.632+0000 7f1e14e49740 4 client.15996 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:01:51.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.632+0000 7f1e14e49740 1 -- 172.21.15.78:0/1444408796 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556676d89da0 con 0x556676e319e0 2024-09-11T01:01:51.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.632+0000 7f1e0bfff640 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x556676e319e0 0x556676e33dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.633+0000 7f1e0bfff640 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x556676e319e0 0x556676e33dd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aaf6ba47accb543 server_cookie=52f05bc6f872b832 in_seq=0 out_seq=0 2024-09-11T01:01:51.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.633+0000 7f1debfff640 10 client.15996 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:51.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.633+0000 7f1debfff640 1 -- 172.21.15.78:0/1444408796 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556676d89da0 con 0x556676e319e0 2024-09-11T01:01:51.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.633+0000 7f1debfff640 10 client.15996 handle_command_reply: tid=0 2024-09-11T01:01:51.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.633+0000 7f1e14e49740 1 client.15996 shutdown 2024-09-11T01:01:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.634+0000 7f1e14e49740 1 -- 172.21.15.78:0/1444408796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 msgr2=0x556676e21a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.634+0000 7f1e14e49740 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 0x556676e21a00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1e00031440 tx=0x7f1e00005e30 comp rx=0 tx=0).stop 2024-09-11T01:01:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.634+0000 7f1e14e49740 1 -- 172.21.15.78:0/1444408796 shutdown_connections 2024-09-11T01:01:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.634+0000 7f1e14e49740 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e1f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.634+0000 7f1e14e49740 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e1ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.634+0000 7f1e14e49740 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x556676e319e0 0x556676e33dd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.634+0000 7f1debfff640 0 client.15996 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:51.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.634+0000 7f1e14e49740 1 --2- 172.21.15.78:0/1444408796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 0x556676e21a00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.634+0000 7f1e14e49740 1 -- 172.21.15.78:0/1444408796 >> 172.21.15.78:0/1444408796 conn(0x556676e0f2b0 msgr2=0x556676e14e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:51.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.634+0000 7f1e14e49740 1 -- 172.21.15.78:0/1444408796 shutdown_connections 2024-09-11T01:01:51.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.634+0000 7f1e14e49740 1 -- 172.21.15.78:0/1444408796 wait complete. 2024-09-11T01:01:51.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.634+0000 7f1e14e49740 20 client.15996 trim_cache size 0 max 16384 2024-09-11T01:01:51.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.653+0000 7f1e14e49740 1 Processor -- start 2024-09-11T01:01:51.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.653+0000 7f1e14e49740 1 -- start start 2024-09-11T01:01:51.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e14e49740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 0x556676e0ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e14e49740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e2dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e14e49740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e14e49740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1ac300 con 0x556676e11480 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e14e49740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1adf00 con 0x556676e22140 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e14e49740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1ac9d0 con 0x556676e11da0 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e10810640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 0x556676e0ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e11011640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e10810640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 0x556676e0ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49674/0 (socket says 172.21.15.78:49674) 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e11011640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e103e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:34122/0 (socket says 172.21.15.78:34122) 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e10810640 1 -- 172.21.15.78:0/2944761613 learned_addr learned my addr 172.21.15.78:0/2944761613 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e0bfff640 1 --2- 172.21.15.78:0/2944761613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e2dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e10810640 1 -- 172.21.15.78:0/2944761613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 msgr2=0x556676e2dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e10810640 1 --2- 172.21.15.78:0/2944761613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e2dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e10810640 1 -- 172.21.15.78:0/2944761613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 msgr2=0x556676e103e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e10810640 1 --2- 172.21.15.78:0/2944761613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e103e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e10810640 1 -- 172.21.15.78:0/2944761613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556676e21d70 con 0x556676e11480 2024-09-11T01:01:51.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.654+0000 7f1e10810640 1 --2- 172.21.15.78:0/2944761613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 0x556676e0ebc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1e00005ec0 tx=0x7f1e0002ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04fb50bc0b23b83 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:51.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.655+0000 7f1df67fc640 1 -- 172.21.15.78:0/2944761613 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0005b070 con 0x556676e11480 2024-09-11T01:01:51.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.655+0000 7f1df67fc640 1 -- 172.21.15.78:0/2944761613 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e00005230 con 0x556676e11480 2024-09-11T01:01:51.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.655+0000 7f1df67fc640 1 -- 172.21.15.78:0/2944761613 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00004260 con 0x556676e11480 2024-09-11T01:01:51.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.655+0000 7f1e14e49740 1 -- 172.21.15.78:0/2944761613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 msgr2=0x556676e0ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:51.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.655+0000 7f1e14e49740 1 --2- 172.21.15.78:0/2944761613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 0x556676e0ebc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1e00005ec0 tx=0x7f1e0002ab10 comp rx=0 tx=0).stop 2024-09-11T01:01:51.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.655+0000 7f1e14e49740 1 -- 172.21.15.78:0/2944761613 shutdown_connections 2024-09-11T01:01:51.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.655+0000 7f1e14e49740 1 --2- 172.21.15.78:0/2944761613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.655+0000 7f1e14e49740 1 --2- 172.21.15.78:0/2944761613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11da0 0x556676e2dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.655+0000 7f1e14e49740 1 --2- 172.21.15.78:0/2944761613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11480 0x556676e0ebc0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.655+0000 7f1e14e49740 1 -- 172.21.15.78:0/2944761613 >> 172.21.15.78:0/2944761613 conn(0x556676e0f2b0 msgr2=0x556676e22520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:51.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.655+0000 7f1e14e49740 1 -- 172.21.15.78:0/2944761613 shutdown_connections 2024-09-11T01:01:51.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e14e49740 1 -- 172.21.15.78:0/2944761613 wait complete. 2024-09-11T01:01:51.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e14e49740 1 -- start start 2024-09-11T01:01:51.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e14e49740 1 Processor -- start 2024-09-11T01:01:51.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e14e49740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11480 0x556676e2a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e14e49740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11da0 0x556676e391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e14e49740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e39980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e14e49740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1ad770 con 0x556676e11da0 2024-09-11T01:01:51.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e14e49740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1ad0a0 con 0x556676e22140 2024-09-11T01:01:51.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e14e49740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1aa300 con 0x556676e11480 2024-09-11T01:01:51.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e14e49740 10 client.0 _ll_register_callbacks cb 0x7f1df80e9900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:51.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11da0 0x556676e391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e10810640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11480 0x556676e2a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11da0 0x556676e391f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49688/0 (socket says 172.21.15.78:49688) 2024-09-11T01:01:51.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e0bfff640 1 -- 172.21.15.78:0/620076445 learned_addr learned my addr 172.21.15.78:0/620076445 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:51.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.656+0000 7f1e11011640 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e39980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1e0bfff640 1 -- 172.21.15.78:0/620076445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11480 msgr2=0x556676e2a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:51.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1e0bfff640 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11480 0x556676e2a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1e0bfff640 1 -- 172.21.15.78:0/620076445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 msgr2=0x556676e39980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:51.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1e0bfff640 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e39980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1e0bfff640 1 -- 172.21.15.78:0/620076445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556676e9d600 con 0x556676e11da0 2024-09-11T01:01:51.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1e11011640 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e39980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:51.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1e0bfff640 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11da0 0x556676e391f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1df800c6a0 tx=0x7f1df80e9a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a38c36fe06b8d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:51.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1debfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:01:51.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1debfff640 1 -- 172.21.15.78:0/620076445 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df801c040 con 0x556676e11da0 2024-09-11T01:01:51.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1debfff640 1 -- 172.21.15.78:0/620076445 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1df80054f0 con 0x556676e11da0 2024-09-11T01:01:51.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1e14e49740 1 -- 172.21.15.78:0/620076445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e00005230 con 0x556676e11da0 2024-09-11T01:01:51.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1debfff640 1 -- 172.21.15.78:0/620076445 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1df80abb80 con 0x556676e11da0 2024-09-11T01:01:51.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1e14e49740 10 client.16002 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:51.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.657+0000 7f1e14e49740 1 -- 172.21.15.78:0/620076445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556676e9e850 con 0x556676e11da0 2024-09-11T01:01:51.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.658+0000 7f1debfff640 1 -- 172.21.15.78:0/620076445 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e00005230 con 0x556676e11da0 2024-09-11T01:01:51.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.658+0000 7f1debfff640 1 -- 172.21.15.78:0/620076445 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1df80d31b0 con 0x556676e11da0 2024-09-11T01:01:51.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.658+0000 7f1e14e49740 10 client.16002 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:51.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.658+0000 7f1e14e49740 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:01:51.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.658+0000 7f1e14e49740 20 client.16002 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:51.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.658+0000 7f1e14e49740 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x556676e319e0 0x556676ea10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:51.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.658+0000 7f1e14e49740 4 client.16002 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:01:51.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.658+0000 7f1e14e49740 1 -- 172.21.15.78:0/620076445 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556676d85850 con 0x556676e319e0 2024-09-11T01:01:51.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.658+0000 7f1e10810640 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x556676e319e0 0x556676ea10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:51.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1debfff640 10 client.16002 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:51.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e10810640 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x556676e319e0 0x556676ea10b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb743fdecc4cb2ff server_cookie=86c321d6cebae147 in_seq=0 out_seq=0 2024-09-11T01:01:51.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1debfff640 1 -- 172.21.15.78:0/620076445 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x556676d85850 con 0x556676e319e0 2024-09-11T01:01:51.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1debfff640 10 client.16002 handle_command_reply: tid=0 2024-09-11T01:01:51.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e14e49740 1 client.16002 shutdown 2024-09-11T01:01:51.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e14e49740 1 -- 172.21.15.78:0/620076445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11da0 msgr2=0x556676e391f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:51.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e14e49740 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11da0 0x556676e391f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1df800c6a0 tx=0x7f1df80e9a30 comp rx=0 tx=0).stop 2024-09-11T01:01:51.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e14e49740 1 -- 172.21.15.78:0/620076445 shutdown_connections 2024-09-11T01:01:51.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e14e49740 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556676e22140 0x556676e39980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e14e49740 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x556676e319e0 0x556676ea10b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e14e49740 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556676e11da0 0x556676e391f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1debfff640 0 client.16002 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:01:51.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e14e49740 1 --2- 172.21.15.78:0/620076445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556676e11480 0x556676e2a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e14e49740 1 -- 172.21.15.78:0/620076445 >> 172.21.15.78:0/620076445 conn(0x556676e0f2b0 msgr2=0x556676e23010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:51.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e14e49740 1 -- 172.21.15.78:0/620076445 shutdown_connections 2024-09-11T01:01:51.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e14e49740 1 -- 172.21.15.78:0/620076445 wait complete. 2024-09-11T01:01:51.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.659+0000 7f1e14e49740 20 client.16002 trim_cache size 0 max 16384 2024-09-11T01:01:51.675 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:01:51.675 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:01:51.675 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:01:51.675 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:01:51.675 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:01:51.676 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:01:51.676 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:01:51.676 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:01:51.676 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:01:51.676 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:01:51.676 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:01:51.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:01:51.676 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:01:51.676 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:01:51.676 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:01:51.676 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:01:51.676 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:01:51.677 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:01:51.678 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:01:51.678 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:01:51.678 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:01:51.678 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:01:51.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:01:51.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:01:51.678 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:01:51.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.660+0000 7f1e12a74640 1 -- 172.21.15.78:0/3415876288 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1dfc0b5450 msgr2=0x7f1dfc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:51.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.660+0000 7f1e12a74640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1dfc0b5450 0x7f1dfc0b7910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009fd0 tx=0x7f1e04009500 comp rx=0 tx=0).stop 2024-09-11T01:01:51.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.660+0000 7f1e12a74640 1 -- 172.21.15.78:0/3415876288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e0c1076a0 msgr2=0x7f1e0c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:51.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.660+0000 7f1e12a74640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e0c1076a0 0x7f1e0c19a610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9f0 tx=0x7f1df800eeb0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.660+0000 7f1e12a74640 1 -- 172.21.15.78:0/3415876288 shutdown_connections 2024-09-11T01:01:51.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.660+0000 7f1e12a74640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1dfc0b5450 0x7f1dfc0b7910 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.660+0000 7f1e12a74640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e0c108020 0x7f1e0c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.660+0000 7f1e12a74640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e0c1076a0 0x7f1e0c19a610 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.660+0000 7f1e12a74640 1 --2- 172.21.15.78:0/3415876288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c101c90 0x7f1e0c19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:51.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.660+0000 7f1e12a74640 1 -- 172.21.15.78:0/3415876288 >> 172.21.15.78:0/3415876288 conn(0x7f1e0c0fd700 msgr2=0x7f1e0c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:51.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.660+0000 7f1e12a74640 1 -- 172.21.15.78:0/3415876288 shutdown_connections 2024-09-11T01:01:51.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:51.660+0000 7f1e12a74640 1 -- 172.21.15.78:0/3415876288 wait complete. 2024-09-11T01:01:52.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:52 smithi112 ceph-mon[35384]: pgmap v413: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-09-11T01:01:52.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:52 smithi078 ceph-mon[31315]: pgmap v413: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-09-11T01:01:52.611 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:01:52.611 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:01:52.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:52 smithi110 ceph-mon[34712]: pgmap v413: 97 pgs: 97 active+clean; 146 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-09-11T01:01:52.884 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.386+0000 7fd7b9c53640 1 -- 172.21.15.78:0/521818485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b4103ca0 msgr2=0x7fd7b4104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.386+0000 7fd7b9c53640 1 --2- 172.21.15.78:0/521818485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b4103ca0 0x7fd7b4104080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009be0 tx=0x7fd7a402a6c0 comp rx=0 tx=0).stop 2024-09-11T01:01:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.386+0000 7fd7b9c53640 1 -- 172.21.15.78:0/521818485 shutdown_connections 2024-09-11T01:01:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.386+0000 7fd7b9c53640 1 --2- 172.21.15.78:0/521818485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7b410d100 0x7fd7b410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.386+0000 7fd7b9c53640 1 --2- 172.21.15.78:0/521818485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7b41045c0 0x7fd7b410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.386+0000 7fd7b9c53640 1 --2- 172.21.15.78:0/521818485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b4103ca0 0x7fd7b4104080 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.386+0000 7fd7b9c53640 1 -- 172.21.15.78:0/521818485 >> 172.21.15.78:0/521818485 conn(0x7fd7b40fd700 msgr2=0x7fd7b40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.386+0000 7fd7b9c53640 1 -- 172.21.15.78:0/521818485 shutdown_connections 2024-09-11T01:01:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.386+0000 7fd7b9c53640 1 -- 172.21.15.78:0/521818485 wait complete. 2024-09-11T01:01:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b9c53640 1 Processor -- start 2024-09-11T01:01:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b9c53640 1 -- start start 2024-09-11T01:01:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b9c53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b4103ca0 0x7fd7b4101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b9c53640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7b41045c0 0x7fd7b4101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b9c53640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7b410d100 0x7fd7b41023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b9c53640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b4112d70 con 0x7fd7b410d100 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b9c53640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b4112bf0 con 0x7fd7b41045c0 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b9c53640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b4112ef0 con 0x7fd7b4103ca0 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b4103ca0 0x7fd7b4101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b4103ca0 0x7fd7b4101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52492/0 (socket says 172.21.15.78:52492) 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b37fe640 1 -- 172.21.15.78:0/1519822505 learned_addr learned my addr 172.21.15.78:0/1519822505 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b3fff640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7b410d100 0x7fd7b41023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.387+0000 7fd7b2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7b41045c0 0x7fd7b4101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.388+0000 7fd7b37fe640 1 -- 172.21.15.78:0/1519822505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7b41045c0 msgr2=0x7fd7b4101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.388+0000 7fd7b37fe640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7b41045c0 0x7fd7b4101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.388+0000 7fd7b37fe640 1 -- 172.21.15.78:0/1519822505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7b410d100 msgr2=0x7fd7b41023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.388+0000 7fd7b37fe640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7b410d100 0x7fd7b41023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.388+0000 7fd7b37fe640 1 -- 172.21.15.78:0/1519822505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b41a76c0 con 0x7fd7b4103ca0 2024-09-11T01:01:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.388+0000 7fd7b37fe640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b4103ca0 0x7fd7b4101950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4004af0 tx=0x7fd7a4003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.388+0000 7fd7b0ff9640 1 -- 172.21.15.78:0/1519822505 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4040020 con 0x7fd7b4103ca0 2024-09-11T01:01:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.388+0000 7fd7b0ff9640 1 -- 172.21.15.78:0/1519822505 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a4030970 con 0x7fd7b4103ca0 2024-09-11T01:01:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.388+0000 7fd7b9c53640 1 -- 172.21.15.78:0/1519822505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b41a7950 con 0x7fd7b4103ca0 2024-09-11T01:01:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.388+0000 7fd7b0ff9640 1 -- 172.21.15.78:0/1519822505 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4036040 con 0x7fd7b4103ca0 2024-09-11T01:01:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.388+0000 7fd7b9c53640 1 -- 172.21.15.78:0/1519822505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b41a7e30 con 0x7fd7b4103ca0 2024-09-11T01:01:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.389+0000 7fd7b9c53640 1 -- 172.21.15.78:0/1519822505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b4104080 con 0x7fd7b4103ca0 2024-09-11T01:01:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.391+0000 7fd7b0ff9640 1 -- 172.21.15.78:0/1519822505 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd7a4004510 con 0x7fd7b4103ca0 2024-09-11T01:01:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.392+0000 7fd7b0ff9640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd79c0b5450 0x7fd79c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.392+0000 7fd7b2ffd640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd79c0b5450 0x7fd79c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.392+0000 7fd7b0ff9640 1 -- 172.21.15.78:0/1519822505 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd7a41071c0 con 0x7fd7b4103ca0 2024-09-11T01:01:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.392+0000 7fd7b2ffd640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd79c0b5450 0x7fd79c0b7910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80096c0 tx=0x7fd7a8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.394+0000 7fd7b0ff9640 1 -- 172.21.15.78:0/1519822505 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd7a404a020 con 0x7fd7b4103ca0 2024-09-11T01:01:53.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 259354 13.85 MB/sec execute 1 sec latency 45.037 ms 2024-09-11T01:01:53.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.635+0000 7fd7b9c53640 1 -- 172.21.15.78:0/1519822505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7b402e5c0 con 0x7fd7b4103ca0 2024-09-11T01:01:53.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.636+0000 7fd7b0ff9640 1 -- 172.21.15.78:0/1519822505 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11986 (secure 0 0 0) 0x7fd7a40cddb0 con 0x7fd7b4103ca0 2024-09-11T01:01:53.638 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:01:53.638 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-09-11T01:00:27:135338+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:00:27.099230+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T01:01:53.638 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-09-11T01:01:53.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.638+0000 7fd7b9c53640 1 -- 172.21.15.78:0/1519822505 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd79c0b5450 msgr2=0x7fd79c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:53.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.638+0000 7fd7b9c53640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd79c0b5450 0x7fd79c0b7910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80096c0 tx=0x7fd7a8009260 comp rx=0 tx=0).stop 2024-09-11T01:01:53.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.638+0000 7fd7b9c53640 1 -- 172.21.15.78:0/1519822505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b4103ca0 msgr2=0x7fd7b4101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:53.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.638+0000 7fd7b9c53640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b4103ca0 0x7fd7b4101950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4004af0 tx=0x7fd7a4003c20 comp rx=0 tx=0).stop 2024-09-11T01:01:53.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.638+0000 7fd7b9c53640 1 -- 172.21.15.78:0/1519822505 shutdown_connections 2024-09-11T01:01:53.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.638+0000 7fd7b9c53640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd79c0b5450 0x7fd79c0b7910 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:53.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.638+0000 7fd7b9c53640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7b410d100 0x7fd7b41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:53.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.638+0000 7fd7b9c53640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7b41045c0 0x7fd7b4101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:53.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.638+0000 7fd7b9c53640 1 --2- 172.21.15.78:0/1519822505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7b4103ca0 0x7fd7b4101950 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:53.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.638+0000 7fd7b9c53640 1 -- 172.21.15.78:0/1519822505 >> 172.21.15.78:0/1519822505 conn(0x7fd7b40fd700 msgr2=0x7fd7b40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:53.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.638+0000 7fd7b9c53640 1 -- 172.21.15.78:0/1519822505 shutdown_connections 2024-09-11T01:01:53.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:53.638+0000 7fd7b9c53640 1 -- 172.21.15.78:0/1519822505 wait complete. 2024-09-11T01:01:53.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:53 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1519822505' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:01:53.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:53 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1519822505' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:01:53.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:53 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1519822505' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:01:54.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 261728 12.43 MB/sec execute 2 sec latency 54.833 ms 2024-09-11T01:01:54.747 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:01:54.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:54 smithi112 ceph-mon[35384]: pgmap v414: 97 pgs: 97 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2024-09-11T01:01:54.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:54 smithi078 ceph-mon[31315]: pgmap v414: 97 pgs: 97 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2024-09-11T01:01:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:55.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:54 smithi110 ceph-mon[34712]: pgmap v414: 97 pgs: 97 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2024-09-11T01:01:55.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 263339 11.17 MB/sec execute 3 sec latency 22.170 ms 2024-09-11T01:01:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.573+0000 7f66ba2c6640 1 -- 172.21.15.78:0/2627408747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b4109150 msgr2=0x7f66b410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.573+0000 7f66ba2c6640 1 --2- 172.21.15.78:0/2627408747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b4109150 0x7f66b410f950 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009ca0 tx=0x7f66ac02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.573+0000 7f66ba2c6640 1 -- 172.21.15.78:0/2627408747 shutdown_connections 2024-09-11T01:01:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.573+0000 7f66ba2c6640 1 --2- 172.21.15.78:0/2627408747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b4109150 0x7f66b410f950 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.573+0000 7f66ba2c6640 1 --2- 172.21.15.78:0/2627408747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b41087d0 0x7f66b4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.573+0000 7f66ba2c6640 1 --2- 172.21.15.78:0/2627408747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b4102dc0 0x7f66b41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.573+0000 7f66ba2c6640 1 -- 172.21.15.78:0/2627408747 >> 172.21.15.78:0/2627408747 conn(0x7f66b40fe870 msgr2=0x7f66b4100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.573+0000 7f66ba2c6640 1 -- 172.21.15.78:0/2627408747 shutdown_connections 2024-09-11T01:01:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.573+0000 7f66ba2c6640 1 -- 172.21.15.78:0/2627408747 wait complete. 2024-09-11T01:01:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66ba2c6640 1 Processor -- start 2024-09-11T01:01:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66ba2c6640 1 -- start start 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66ba2c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b4102dc0 0x7f66b4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66ba2c6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b41087d0 0x7f66b4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66ba2c6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b4109150 0x7f66b419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66ba2c6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66b4112e10 con 0x7f66b4109150 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66ba2c6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66b4112c90 con 0x7f66b41087d0 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66ba2c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66b4112f90 con 0x7f66b4102dc0 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b4102dc0 0x7f66b4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66b8863640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b4109150 0x7f66b419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66b37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b41087d0 0x7f66b4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66b37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b41087d0 0x7f66b4199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:54972/0 (socket says 172.21.15.78:54972) 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b4102dc0 0x7f66b4199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52512/0 (socket says 172.21.15.78:52512) 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66b8863640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b4109150 0x7f66b419eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60934/0 (socket says 172.21.15.78:60934) 2024-09-11T01:01:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.574+0000 7f66b37fe640 1 -- 172.21.15.78:0/1702014486 learned_addr learned my addr 172.21.15.78:0/1702014486 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.575+0000 7f66b37fe640 1 -- 172.21.15.78:0/1702014486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b4102dc0 msgr2=0x7f66b4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.575+0000 7f66b37fe640 1 --2- 172.21.15.78:0/1702014486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b4102dc0 0x7f66b4199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.575+0000 7f66b37fe640 1 -- 172.21.15.78:0/1702014486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b4109150 msgr2=0x7f66b419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.575+0000 7f66b37fe640 1 --2- 172.21.15.78:0/1702014486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b4109150 0x7f66b419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.575+0000 7f66b37fe640 1 -- 172.21.15.78:0/1702014486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b419f2f0 con 0x7f66b41087d0 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.575+0000 7f66b37fe640 1 --2- 172.21.15.78:0/1702014486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b41087d0 0x7f66b4199780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66a800e9f0 tx=0x7f66a800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.575+0000 7f66b17fa640 1 -- 172.21.15.78:0/1702014486 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a800cd80 con 0x7f66b41087d0 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.575+0000 7f66b17fa640 1 -- 172.21.15.78:0/1702014486 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a80054d0 con 0x7f66b41087d0 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.575+0000 7f66ba2c6640 1 -- 172.21.15.78:0/1702014486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b419f5e0 con 0x7f66b41087d0 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.575+0000 7f66b17fa640 1 -- 172.21.15.78:0/1702014486 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8010690 con 0x7f66b41087d0 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.575+0000 7f66ba2c6640 1 -- 172.21.15.78:0/1702014486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b41a5ca0 con 0x7f66b41087d0 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.578+0000 7f66bc69b740 1 Processor -- start 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.578+0000 7f66bc69b740 1 -- start start 2024-09-11T01:01:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.578+0000 7f66b17fa640 1 -- 172.21.15.78:0/1702014486 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f66a8004e90 con 0x7f66b41087d0 2024-09-11T01:01:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.578+0000 7f66bc69b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 0x558719fc04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.578+0000 7f66bc69b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc0a10 0x558719fcc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.578+0000 7f66bc69b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fcc9a0 0x558719fced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.578+0000 7f66bc69b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558719f36ea0 con 0x558719fc0a10 2024-09-11T01:01:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.578+0000 7f66bc69b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558719f262d0 con 0x558719fc00f0 2024-09-11T01:01:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.578+0000 7f66bc69b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558719f2a0e0 con 0x558719fcc9a0 2024-09-11T01:01:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc0a10 0x558719fcc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b8863640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 0x558719fc04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc0a10 0x558719fcc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60948/0 (socket says 172.21.15.78:60948) 2024-09-11T01:01:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b3fff640 1 -- 172.21.15.78:0/3817408981 learned_addr learned my addr 172.21.15.78:0/3817408981 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b37fe640 1 --2- 172.21.15.78:0/3817408981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fcc9a0 0x558719fced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b3fff640 1 -- 172.21.15.78:0/3817408981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fcc9a0 msgr2=0x558719fced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b3fff640 1 --2- 172.21.15.78:0/3817408981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fcc9a0 0x558719fced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b3fff640 1 -- 172.21.15.78:0/3817408981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 msgr2=0x558719fc04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b3fff640 1 --2- 172.21.15.78:0/3817408981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 0x558719fc04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b3fff640 1 -- 172.21.15.78:0/3817408981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558719c84650 con 0x558719fc0a10 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b8863640 1 --2- 172.21.15.78:0/3817408981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 0x558719fc04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b3fff640 1 --2- 172.21.15.78:0/3817408981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc0a10 0x558719fcc460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009cf0 tx=0x7f66a4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6dc26d243845d3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b17fa640 1 --2- 172.21.15.78:0/1702014486 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f669c0b5450 0x7f669c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f669a7fc640 1 -- 172.21.15.78:0/3817408981 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a4004230 con 0x558719fc0a10 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b8863640 1 --2- 172.21.15.78:0/1702014486 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f669c0b5450 0x7f669c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f669a7fc640 1 -- 172.21.15.78:0/3817408981 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a40043d0 con 0x558719fc0a10 2024-09-11T01:01:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f669a7fc640 1 -- 172.21.15.78:0/3817408981 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a4019880 con 0x558719fc0a10 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.579+0000 7f66b17fa640 1 -- 172.21.15.78:0/1702014486 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f66a80e6380 con 0x7f66b41087d0 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.580+0000 7f66bc69b740 1 -- 172.21.15.78:0/3817408981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc0a10 msgr2=0x558719fcc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.580+0000 7f66b8863640 1 --2- 172.21.15.78:0/1702014486 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f669c0b5450 0x7f669c0b7910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f66ac02f500 tx=0x7f66ac005f50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.580+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3817408981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc0a10 0x558719fcc460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009cf0 tx=0x7f66a4011810 comp rx=0 tx=0).stop 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.580+0000 7f66bc69b740 1 -- 172.21.15.78:0/3817408981 shutdown_connections 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.580+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3817408981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fcc9a0 0x558719fced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.580+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3817408981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc0a10 0x558719fcc460 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.580+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3817408981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 0x558719fc04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.580+0000 7f66bc69b740 1 -- 172.21.15.78:0/3817408981 >> 172.21.15.78:0/3817408981 conn(0x558719fbdf20 msgr2=0x558719fbf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.580+0000 7f66bc69b740 1 -- 172.21.15.78:0/3817408981 shutdown_connections 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.580+0000 7f66bc69b740 1 -- 172.21.15.78:0/3817408981 wait complete. 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.581+0000 7f66bc69b740 1 -- start start 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.581+0000 7f66bc69b740 1 Processor -- start 2024-09-11T01:01:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.581+0000 7f66bc69b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 0x558719fcc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.581+0000 7f66bc69b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc0a10 0x558719fc96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.581+0000 7f66bc69b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fcc9a0 0x558719fc9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.581+0000 7f66bc69b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558719d87700 con 0x558719fc0a10 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.581+0000 7f66bc69b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558719ec1160 con 0x558719fc00f0 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.581+0000 7f66bc69b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558719fcf620 con 0x558719fcc9a0 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.581+0000 7f66bc69b740 10 client.0 _ll_register_callbacks cb 0x558719eda9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 0x558719fcc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 0x558719fcc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:54990/0 (socket says 172.21.15.78:54990) 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66b3fff640 1 -- 172.21.15.78:0/34997947 learned_addr learned my addr 172.21.15.78:0/34997947 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66b8863640 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fcc9a0 0x558719fc9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66b37fe640 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc0a10 0x558719fc96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66b3fff640 1 -- 172.21.15.78:0/34997947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fcc9a0 msgr2=0x558719fc9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66b3fff640 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fcc9a0 0x558719fc9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66b3fff640 1 -- 172.21.15.78:0/34997947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc0a10 msgr2=0x558719fc96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66b3fff640 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc0a10 0x558719fc96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66b3fff640 1 -- 172.21.15.78:0/34997947 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558719eeec90 con 0x558719fc00f0 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66b3fff640 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 0x558719fcc280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66a4019460 tx=0x7f66a4003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf2a85660490d469 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66937fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66937fe640 1 -- 172.21.15.78:0/34997947 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a4027070 con 0x558719fc00f0 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66937fe640 1 -- 172.21.15.78:0/34997947 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f66a4005240 con 0x558719fc00f0 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66bc69b740 1 -- 172.21.15.78:0/34997947 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a40043d0 con 0x558719fc00f0 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66937fe640 1 -- 172.21.15.78:0/34997947 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f66a4033050 con 0x558719fc00f0 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66bc69b740 10 client.25424 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.582+0000 7f66bc69b740 1 -- 172.21.15.78:0/34997947 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558719fdaf40 con 0x558719fc00f0 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.583+0000 7f66937fe640 1 -- 172.21.15.78:0/34997947 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a40043d0 con 0x558719fc00f0 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.583+0000 7f66937fe640 1 -- 172.21.15.78:0/34997947 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f66a4004b70 con 0x558719fc00f0 2024-09-11T01:01:55.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.583+0000 7f66bc69b740 10 client.25424 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.583+0000 7f66bc69b740 10 client.25424 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.583+0000 7f66bc69b740 20 client.25424 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.583+0000 7f66bc69b740 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558719fdc2f0 0x558719fde6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.583+0000 7f66bc69b740 4 client.25424 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.583+0000 7f66bc69b740 1 -- 172.21.15.78:0/34997947 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558719f38df0 con 0x558719fdc2f0 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.583+0000 7f66b37fe640 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558719fdc2f0 0x558719fde6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.583+0000 7f66937fe640 10 client.25424 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.583+0000 7f66b37fe640 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558719fdc2f0 0x558719fde6e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ea31773d025849c server_cookie=3757855848899126 in_seq=0 out_seq=0 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66937fe640 1 -- 172.21.15.78:0/34997947 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558719f38df0 con 0x558719fdc2f0 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66937fe640 10 client.25424 handle_command_reply: tid=0 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66bc69b740 1 client.25424 shutdown 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66bc69b740 1 -- 172.21.15.78:0/34997947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 msgr2=0x558719fcc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66bc69b740 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 0x558719fcc280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66a4019460 tx=0x7f66a4003820 comp rx=0 tx=0).stop 2024-09-11T01:01:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66bc69b740 1 -- 172.21.15.78:0/34997947 shutdown_connections 2024-09-11T01:01:55.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66bc69b740 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558719fdc2f0 0x558719fde6e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66bc69b740 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fcc9a0 0x558719fc9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66937fe640 0 client.25424 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:01:55.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66bc69b740 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc0a10 0x558719fc96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66bc69b740 1 --2- 172.21.15.78:0/34997947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc00f0 0x558719fcc280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66bc69b740 1 -- 172.21.15.78:0/34997947 >> 172.21.15.78:0/34997947 conn(0x558719fbdf20 msgr2=0x558719fc3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:55.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66bc69b740 1 -- 172.21.15.78:0/34997947 shutdown_connections 2024-09-11T01:01:55.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66bc69b740 1 -- 172.21.15.78:0/34997947 wait complete. 2024-09-11T01:01:55.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.584+0000 7f66bc69b740 20 client.25424 trim_cache size 0 max 16384 2024-09-11T01:01:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.599+0000 7f66bc69b740 1 Processor -- start 2024-09-11T01:01:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.599+0000 7f66bc69b740 1 -- start start 2024-09-11T01:01:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.599+0000 7f66bc69b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fc00f0 0x558719fd8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.599+0000 7f66bc69b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc0a10 0x558719fc38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.599+0000 7f66bc69b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fcc9a0 0x558719fc3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.599+0000 7f66bc69b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66b41ae4f0 con 0x558719fcc9a0 2024-09-11T01:01:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.599+0000 7f66bc69b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66b41b00f0 con 0x558719fc0a10 2024-09-11T01:01:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.599+0000 7f66bc69b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66b41aebc0 con 0x558719fc00f0 2024-09-11T01:01:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.599+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fc00f0 0x558719fd8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.599+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fc00f0 0x558719fd8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52534/0 (socket says 172.21.15.78:52534) 2024-09-11T01:01:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.599+0000 7f66b3fff640 1 -- 172.21.15.78:0/3190017744 learned_addr learned my addr 172.21.15.78:0/3190017744 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.599+0000 7f66b37fe640 1 --2- 172.21.15.78:0/3190017744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc0a10 0x558719fc38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.600+0000 7f66b8863640 1 --2- 172.21.15.78:0/3190017744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fcc9a0 0x558719fc3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.600+0000 7f66b37fe640 1 -- 172.21.15.78:0/3190017744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fc00f0 msgr2=0x558719fd8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.600+0000 7f66b37fe640 1 --2- 172.21.15.78:0/3190017744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fc00f0 0x558719fd8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.600+0000 7f66b37fe640 1 -- 172.21.15.78:0/3190017744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fcc9a0 msgr2=0x558719fc3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.600+0000 7f66b37fe640 1 --2- 172.21.15.78:0/3190017744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fcc9a0 0x558719fc3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.600+0000 7f66b37fe640 1 -- 172.21.15.78:0/3190017744 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558719fcc5f0 con 0x558719fc0a10 2024-09-11T01:01:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.600+0000 7f66b8863640 1 --2- 172.21.15.78:0/3190017744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fcc9a0 0x558719fc3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:01:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.600+0000 7f66b37fe640 1 --2- 172.21.15.78:0/3190017744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc0a10 0x558719fc38f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66a80e3b40 tx=0x7f66a8004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2d3d922f95c6154 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.600+0000 7f6699ffb640 1 -- 172.21.15.78:0/3190017744 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a80ac1c0 con 0x558719fc0a10 2024-09-11T01:01:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.600+0000 7f6699ffb640 1 -- 172.21.15.78:0/3190017744 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a80ac380 con 0x558719fc0a10 2024-09-11T01:01:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.600+0000 7f6699ffb640 1 -- 172.21.15.78:0/3190017744 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a80abfe0 con 0x558719fc0a10 2024-09-11T01:01:55.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.601+0000 7f66bc69b740 1 -- 172.21.15.78:0/3190017744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc0a10 msgr2=0x558719fc38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:55.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.601+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3190017744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc0a10 0x558719fc38f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66a80e3b40 tx=0x7f66a8004a90 comp rx=0 tx=0).stop 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.601+0000 7f66bc69b740 1 -- 172.21.15.78:0/3190017744 shutdown_connections 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.601+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3190017744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fcc9a0 0x558719fc3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.601+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3190017744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fc0a10 0x558719fc38f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.601+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3190017744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fc00f0 0x558719fd8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.601+0000 7f66bc69b740 1 -- 172.21.15.78:0/3190017744 >> 172.21.15.78:0/3190017744 conn(0x558719fbdf20 msgr2=0x558719fc31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.601+0000 7f66bc69b740 1 -- 172.21.15.78:0/3190017744 shutdown_connections 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.601+0000 7f66bc69b740 1 -- 172.21.15.78:0/3190017744 wait complete. 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.601+0000 7f66bc69b740 1 -- start start 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66bc69b740 1 Processor -- start 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66bc69b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc00f0 0x558719fd4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66bc69b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fc0a10 0x558719fd5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66bc69b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fcc9a0 0x558719fcf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66bc69b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66b41af960 con 0x558719fc00f0 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66bc69b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66b41af290 con 0x558719fcc9a0 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66bc69b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66b41ac4f0 con 0x558719fc0a10 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66bc69b740 10 client.0 _ll_register_callbacks cb 0x7f66a8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc00f0 0x558719fd4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc00f0 0x558719fd4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60982/0 (socket says 172.21.15.78:60982) 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66b3fff640 1 -- 172.21.15.78:0/3617475974 learned_addr learned my addr 172.21.15.78:0/3617475974 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66b37fe640 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fc0a10 0x558719fd5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.602+0000 7f66b8863640 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fcc9a0 0x558719fcf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66b37fe640 1 -- 172.21.15.78:0/3617475974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fcc9a0 msgr2=0x558719fcf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66b37fe640 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fcc9a0 0x558719fcf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66b37fe640 1 -- 172.21.15.78:0/3617475974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc00f0 msgr2=0x558719fd4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66b37fe640 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc00f0 0x558719fd4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66b37fe640 1 -- 172.21.15.78:0/3617475974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558719fd00a0 con 0x558719fc0a10 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66b37fe640 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fc0a10 0x558719fd5040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f66a80097d0 tx=0x7f66a80e3b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9afdec313d460556 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66937fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66937fe640 1 -- 172.21.15.78:0/3617475974 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a80abfe0 con 0x558719fc0a10 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66bc69b740 1 -- 172.21.15.78:0/3617475974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a80ac380 con 0x558719fc0a10 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66937fe640 1 -- 172.21.15.78:0/3617475974 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f66a80a8ae0 con 0x558719fc0a10 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66937fe640 1 -- 172.21.15.78:0/3617475974 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f66a80ac1c0 con 0x558719fc0a10 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66bc69b740 10 client.25417 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66bc69b740 1 -- 172.21.15.78:0/3617475974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55871a04dee0 con 0x558719fc0a10 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.603+0000 7f66937fe640 1 -- 172.21.15.78:0/3617475974 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a80ac380 con 0x558719fc0a10 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66937fe640 1 -- 172.21.15.78:0/3617475974 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f66a80b5970 con 0x558719fc0a10 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66bc69b740 10 client.25417 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66bc69b740 10 client.25417 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66bc69b740 20 client.25417 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55871a04e450 0x55871a050840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66bc69b740 4 client.25417 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66bc69b740 1 -- 172.21.15.78:0/3617475974 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558719f348a0 con 0x55871a04e450 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66b3fff640 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55871a04e450 0x55871a050840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66937fe640 10 client.25417 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66b3fff640 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55871a04e450 0x55871a050840 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=392bb529e45a2aa server_cookie=393b8bca03c214f in_seq=0 out_seq=0 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66937fe640 1 -- 172.21.15.78:0/3617475974 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558719f348a0 con 0x55871a04e450 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66937fe640 10 client.25417 handle_command_reply: tid=0 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.604+0000 7f66bc69b740 1 client.25417 shutdown 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66bc69b740 1 -- 172.21.15.78:0/3617475974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fc0a10 msgr2=0x558719fd5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fc0a10 0x558719fd5040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f66a80097d0 tx=0x7f66a80e3b60 comp rx=0 tx=0).stop 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66bc69b740 1 -- 172.21.15.78:0/3617475974 shutdown_connections 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55871a04e450 0x55871a050840 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558719fcc9a0 0x558719fcf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558719fc0a10 0x558719fd5040 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66937fe640 0 client.25417 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:01:55.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66bc69b740 1 --2- 172.21.15.78:0/3617475974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558719fc00f0 0x558719fd4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66bc69b740 1 -- 172.21.15.78:0/3617475974 >> 172.21.15.78:0/3617475974 conn(0x558719fbdf20 msgr2=0x558719fc3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:55.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66bc69b740 1 -- 172.21.15.78:0/3617475974 shutdown_connections 2024-09-11T01:01:55.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66bc69b740 1 -- 172.21.15.78:0/3617475974 wait complete. 2024-09-11T01:01:55.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66bc69b740 20 client.25417 trim_cache size 0 max 16384 2024-09-11T01:01:55.611 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:01:55.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66ba2c6640 1 -- 172.21.15.78:0/1702014486 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f669c0b5450 msgr2=0x7f669c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:55.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66ba2c6640 1 --2- 172.21.15.78:0/1702014486 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f669c0b5450 0x7f669c0b7910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f66ac02f500 tx=0x7f66ac005f50 comp rx=0 tx=0).stop 2024-09-11T01:01:55.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66ba2c6640 1 -- 172.21.15.78:0/1702014486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b41087d0 msgr2=0x7f66b4199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:55.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66ba2c6640 1 --2- 172.21.15.78:0/1702014486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b41087d0 0x7f66b4199780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66a800e9f0 tx=0x7f66a800edf0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66ba2c6640 1 -- 172.21.15.78:0/1702014486 shutdown_connections 2024-09-11T01:01:55.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66ba2c6640 1 --2- 172.21.15.78:0/1702014486 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f669c0b5450 0x7f669c0b7910 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66ba2c6640 1 --2- 172.21.15.78:0/1702014486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b4109150 0x7f66b419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66ba2c6640 1 --2- 172.21.15.78:0/1702014486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b41087d0 0x7f66b4199780 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66ba2c6640 1 --2- 172.21.15.78:0/1702014486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b4102dc0 0x7f66b4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:55.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.605+0000 7f66ba2c6640 1 -- 172.21.15.78:0/1702014486 >> 172.21.15.78:0/1702014486 conn(0x7f66b40fe870 msgr2=0x7f66b4100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:55.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.606+0000 7f66ba2c6640 1 -- 172.21.15.78:0/1702014486 shutdown_connections 2024-09-11T01:01:55.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:55.606+0000 7f66ba2c6640 1 -- 172.21.15.78:0/1702014486 wait complete. 2024-09-11T01:01:56.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 265998 12.59 MB/sec execute 4 sec latency 63.107 ms 2024-09-11T01:01:56.442 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:01:56.708 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.215+0000 7f71f807e640 1 -- 172.21.15.78:0/4175246237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71f0102180 msgr2=0x7f71f010eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.215+0000 7f71f807e640 1 --2- 172.21.15.78:0/4175246237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71f0102180 0x7f71f010eed0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009900 tx=0x7f71e402a480 comp rx=0 tx=0).stop 2024-09-11T01:01:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.215+0000 7f71f807e640 1 -- 172.21.15.78:0/4175246237 shutdown_connections 2024-09-11T01:01:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.215+0000 7f71f807e640 1 --2- 172.21.15.78:0/4175246237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71f010f410 0x7f71f0111800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.215+0000 7f71f807e640 1 --2- 172.21.15.78:0/4175246237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71f0102180 0x7f71f010eed0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.215+0000 7f71f807e640 1 --2- 172.21.15.78:0/4175246237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0101860 0x7f71f0101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.215+0000 7f71f807e640 1 -- 172.21.15.78:0/4175246237 >> 172.21.15.78:0/4175246237 conn(0x7f71f00fd770 msgr2=0x7f71f00ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.215+0000 7f71f807e640 1 -- 172.21.15.78:0/4175246237 shutdown_connections 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.215+0000 7f71f807e640 1 -- 172.21.15.78:0/4175246237 wait complete. 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.215+0000 7f71f807e640 1 Processor -- start 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.215+0000 7f71f807e640 1 -- start start 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f807e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0101860 0x7f71f019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f807e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71f0102180 0x7f71f019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f807e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71f010f410 0x7f71f01a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f807e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71f0114f50 con 0x7f71f010f410 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f807e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71f0114dd0 con 0x7f71f0101860 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f807e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01150d0 con 0x7f71f0102180 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f5e1a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0101860 0x7f71f019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f661b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71f010f410 0x7f71f01a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f5e1a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0101860 0x7f71f019b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55036/0 (socket says 172.21.15.78:55036) 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f661b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71f010f410 0x7f71f01a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32770/0 (socket says 172.21.15.78:32770) 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f5e1a640 1 -- 172.21.15.78:0/2377570391 learned_addr learned my addr 172.21.15.78:0/2377570391 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f5619640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71f0102180 0x7f71f019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f5e1a640 1 -- 172.21.15.78:0/2377570391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71f0102180 msgr2=0x7f71f019b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f5e1a640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71f0102180 0x7f71f019b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f5e1a640 1 -- 172.21.15.78:0/2377570391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71f010f410 msgr2=0x7f71f01a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f5e1a640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71f010f410 0x7f71f01a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f5e1a640 1 -- 172.21.15.78:0/2377570391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f01a1550 con 0x7f71f0101860 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.216+0000 7f71f5e1a640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0101860 0x7f71f019b4a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e9f0 tx=0x7f71e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.217+0000 7f71eeffd640 1 -- 172.21.15.78:0/2377570391 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e800cd80 con 0x7f71f0101860 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.217+0000 7f71eeffd640 1 -- 172.21.15.78:0/2377570391 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e80108e0 con 0x7f71f0101860 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.217+0000 7f71eeffd640 1 -- 172.21.15.78:0/2377570391 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8018980 con 0x7f71f0101860 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.217+0000 7f71f807e640 1 -- 172.21.15.78:0/2377570391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f01a1840 con 0x7f71f0101860 2024-09-11T01:01:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.217+0000 7f71f807e640 1 -- 172.21.15.78:0/2377570391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f01a80d0 con 0x7f71f0101860 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.219+0000 7f71eeffd640 1 -- 172.21.15.78:0/2377570391 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f71e8010430 con 0x7f71f0101860 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.219+0000 7f71fa453740 1 Processor -- start 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.219+0000 7f71fa453740 1 -- start start 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71fa453740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7618734d0 0x55c7618738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71fa453740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 0x55c76187f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71fa453740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c76187fd80 0x55c761882170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71fa453740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c7617e9ea0 con 0x55c761873df0 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71fa453740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c7617d92d0 con 0x55c76187fd80 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71fa453740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7617dd0e0 con 0x55c7618734d0 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71f5e1a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 0x55c76187f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71f661b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7618734d0 0x55c7618738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71f5e1a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 0x55c76187f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32778/0 (socket says 172.21.15.78:32778) 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71f5e1a640 1 -- 172.21.15.78:0/3230114815 learned_addr learned my addr 172.21.15.78:0/3230114815 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71f5619640 1 --2- 172.21.15.78:0/3230114815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c76187fd80 0x55c761882170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71f661b640 1 -- 172.21.15.78:0/3230114815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c76187fd80 msgr2=0x55c761882170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71f661b640 1 --2- 172.21.15.78:0/3230114815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c76187fd80 0x55c761882170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71f661b640 1 -- 172.21.15.78:0/3230114815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 msgr2=0x55c76187f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71f661b640 1 --2- 172.21.15.78:0/3230114815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 0x55c76187f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71f661b640 1 -- 172.21.15.78:0/3230114815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c761537650 con 0x55c7618734d0 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71f661b640 1 --2- 172.21.15.78:0/3230114815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7618734d0 0x55c7618738b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009f00 tx=0x7f71e0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed3524f0434742d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71eeffd640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71d80b54d0 0x7f71d80b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71f5e1a640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71d80b54d0 0x7f71d80b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.220+0000 7f71d7fff640 1 -- 172.21.15.78:0/3230114815 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0004d20 con 0x55c7618734d0 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.221+0000 7f71d7fff640 1 -- 172.21.15.78:0/3230114815 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e0004030 con 0x55c7618734d0 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.221+0000 7f71d7fff640 1 -- 172.21.15.78:0/3230114815 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0005260 con 0x55c7618734d0 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.221+0000 7f71eeffd640 1 -- 172.21.15.78:0/2377570391 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f71e80e62f0 con 0x7f71f0101860 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.221+0000 7f71fa453740 1 -- 172.21.15.78:0/3230114815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7618734d0 msgr2=0x55c7618738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.221+0000 7f71fa453740 1 --2- 172.21.15.78:0/3230114815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7618734d0 0x55c7618738b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009f00 tx=0x7f71e0011810 comp rx=0 tx=0).stop 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.221+0000 7f71f5e1a640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71d80b54d0 0x7f71d80b7990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f71e80ac4e0 tx=0x7f71e80e9e30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.221+0000 7f71fa453740 1 -- 172.21.15.78:0/3230114815 shutdown_connections 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.221+0000 7f71fa453740 1 --2- 172.21.15.78:0/3230114815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c76187fd80 0x55c761882170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.221+0000 7f71fa453740 1 --2- 172.21.15.78:0/3230114815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 0x55c76187f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.221+0000 7f71fa453740 1 --2- 172.21.15.78:0/3230114815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7618734d0 0x55c7618738b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.221+0000 7f71fa453740 1 -- 172.21.15.78:0/3230114815 >> 172.21.15.78:0/3230114815 conn(0x55c761871300 msgr2=0x55c7618724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71fa453740 1 -- 172.21.15.78:0/3230114815 shutdown_connections 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71fa453740 1 -- 172.21.15.78:0/3230114815 wait complete. 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71fa453740 1 -- start start 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71fa453740 1 Processor -- start 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71fa453740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 0x55c76187f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71fa453740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c761873df0 0x55c76187ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71fa453740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 0x55c76187d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71fa453740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c76163a700 con 0x55c7618734d0 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71fa453740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c761774160 con 0x55c761873df0 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71fa453740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c761882a00 con 0x55c76187fd80 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71fa453740 10 client.0 _ll_register_callbacks cb 0x55c76178d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71f661b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 0x55c76187f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71f661b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 0x55c76187f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32784/0 (socket says 172.21.15.78:32784) 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.222+0000 7f71f661b640 1 -- 172.21.15.78:0/484875811 learned_addr learned my addr 172.21.15.78:0/484875811 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71f5619640 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c761873df0 0x55c76187ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71f661b640 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 0x55c76187d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71f661b640 1 -- 172.21.15.78:0/484875811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 msgr2=0x55c76187d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71f661b640 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 0x55c76187d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71f661b640 1 -- 172.21.15.78:0/484875811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c761873df0 msgr2=0x55c76187ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71f661b640 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c761873df0 0x55c76187ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71f661b640 1 -- 172.21.15.78:0/484875811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7617a1c90 con 0x55c7618734d0 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71f661b640 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 0x55c76187f640 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f71e0004900 tx=0x7f71e0004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2aaf57e810d661d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71d57fa640 1 -- 172.21.15.78:0/484875811 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e002d060 con 0x55c7618734d0 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71d57fa640 1 -- 172.21.15.78:0/484875811 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f71e0003c40 con 0x55c7618734d0 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71fa453740 1 -- 172.21.15.78:0/484875811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e0004030 con 0x55c7618734d0 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71d57fa640 1 -- 172.21.15.78:0/484875811 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f71e0003e40 con 0x55c7618734d0 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71fa453740 10 client.16035 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71fa453740 1 -- 172.21.15.78:0/484875811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c76188e330 con 0x55c7618734d0 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.223+0000 7f71d57fa640 1 -- 172.21.15.78:0/484875811 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e0004030 con 0x55c7618734d0 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.224+0000 7f71d57fa640 1 -- 172.21.15.78:0/484875811 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f71e0030020 con 0x55c7618734d0 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.224+0000 7f71fa453740 10 client.16035 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.224+0000 7f71fa453740 10 client.16035 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.224+0000 7f71fa453740 20 client.16035 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.224+0000 7f71fa453740 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c76188f730 0x55c761891b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.224+0000 7f71fa453740 4 client.16035 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:01:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.224+0000 7f71fa453740 1 -- 172.21.15.78:0/484875811 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7617ebdf0 con 0x55c76188f730 2024-09-11T01:01:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.224+0000 7f71f5619640 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c76188f730 0x55c761891b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.224+0000 7f71d57fa640 10 client.16035 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:01:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.224+0000 7f71f5619640 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c76188f730 0x55c761891b20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4ce6441a8f6a94a3 server_cookie=327fe8d38d1e3ebc in_seq=0 out_seq=0 2024-09-11T01:01:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71d57fa640 1 -- 172.21.15.78:0/484875811 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c7617ebdf0 con 0x55c76188f730 2024-09-11T01:01:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71d57fa640 10 client.16035 handle_command_reply: tid=0 2024-09-11T01:01:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71fa453740 1 client.16035 shutdown 2024-09-11T01:01:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71fa453740 1 -- 172.21.15.78:0/484875811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 msgr2=0x55c76187f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71fa453740 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 0x55c76187f640 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f71e0004900 tx=0x7f71e0004720 comp rx=0 tx=0).stop 2024-09-11T01:01:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71fa453740 1 -- 172.21.15.78:0/484875811 shutdown_connections 2024-09-11T01:01:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71fa453740 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c76188f730 0x55c761891b20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71fa453740 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 0x55c76187d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71fa453740 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c761873df0 0x55c76187ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71d57fa640 0 client.16035 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:01:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71fa453740 1 --2- 172.21.15.78:0/484875811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 0x55c76187f640 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71fa453740 1 -- 172.21.15.78:0/484875811 >> 172.21.15.78:0/484875811 conn(0x55c761871300 msgr2=0x55c761876ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.225+0000 7f71fa453740 1 -- 172.21.15.78:0/484875811 shutdown_connections 2024-09-11T01:01:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.226+0000 7f71fa453740 1 -- 172.21.15.78:0/484875811 wait complete. 2024-09-11T01:01:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.226+0000 7f71fa453740 20 client.16035 trim_cache size 0 max 16384 2024-09-11T01:01:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71fa453740 1 Processor -- start 2024-09-11T01:01:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71fa453740 1 -- start start 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71fa453740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 0x55c76188bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71fa453740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c761873df0 0x55c761872a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71fa453740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 0x55c761881170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71fa453740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b0800 con 0x55c7618734d0 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71fa453740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b2400 con 0x55c761873df0 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71fa453740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b0ed0 con 0x55c76187fd80 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71f661b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 0x55c76188bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71f661b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 0x55c76188bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32800/0 (socket says 172.21.15.78:32800) 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71f661b640 1 -- 172.21.15.78:0/227624208 learned_addr learned my addr 172.21.15.78:0/227624208 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71f5619640 1 --2- 172.21.15.78:0/227624208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c761873df0 0x55c761872a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71f661b640 1 -- 172.21.15.78:0/227624208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 msgr2=0x55c761881170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71f661b640 1 --2- 172.21.15.78:0/227624208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 0x55c761881170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71f661b640 1 -- 172.21.15.78:0/227624208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c761873df0 msgr2=0x55c761872a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71f661b640 1 --2- 172.21.15.78:0/227624208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c761873df0 0x55c761872a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71f661b640 1 -- 172.21.15.78:0/227624208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c76187c840 con 0x55c7618734d0 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.240+0000 7f71f5619640 1 --2- 172.21.15.78:0/227624208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c761873df0 0x55c761872a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.241+0000 7f71f661b640 1 --2- 172.21.15.78:0/227624208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 0x55c76188bc60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009d40 tx=0x7f71e0032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554e0ab3180a0ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.241+0000 7f71d77fe640 1 -- 172.21.15.78:0/227624208 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e002d070 con 0x55c7618734d0 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.241+0000 7f71d77fe640 1 -- 172.21.15.78:0/227624208 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e0005bd0 con 0x55c7618734d0 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.241+0000 7f71d77fe640 1 -- 172.21.15.78:0/227624208 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0028480 con 0x55c7618734d0 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.242+0000 7f71fa453740 1 -- 172.21.15.78:0/227624208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 msgr2=0x55c76188bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.242+0000 7f71fa453740 1 --2- 172.21.15.78:0/227624208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 0x55c76188bc60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009d40 tx=0x7f71e0032440 comp rx=0 tx=0).stop 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.242+0000 7f71fa453740 1 -- 172.21.15.78:0/227624208 shutdown_connections 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.242+0000 7f71fa453740 1 --2- 172.21.15.78:0/227624208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 0x55c761881170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.242+0000 7f71fa453740 1 --2- 172.21.15.78:0/227624208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c761873df0 0x55c761872a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.242+0000 7f71fa453740 1 --2- 172.21.15.78:0/227624208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7618734d0 0x55c76188bc60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.242+0000 7f71fa453740 1 -- 172.21.15.78:0/227624208 >> 172.21.15.78:0/227624208 conn(0x55c761871300 msgr2=0x55c761872350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.242+0000 7f71fa453740 1 -- 172.21.15.78:0/227624208 shutdown_connections 2024-09-11T01:01:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.242+0000 7f71fa453740 1 -- 172.21.15.78:0/227624208 wait complete. 2024-09-11T01:01:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.242+0000 7f71fa453740 1 -- start start 2024-09-11T01:01:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.242+0000 7f71fa453740 1 Processor -- start 2024-09-11T01:01:57.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.243+0000 7f71fa453740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7618734d0 0x55c761887cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.243+0000 7f71fa453740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 0x55c761888210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.243+0000 7f71fa453740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 0x55c761882c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.243+0000 7f71fa453740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b1c70 con 0x55c761873df0 2024-09-11T01:01:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.243+0000 7f71fa453740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b15a0 con 0x55c7618734d0 2024-09-11T01:01:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.243+0000 7f71fa453740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01ae7e0 con 0x55c76187fd80 2024-09-11T01:01:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.243+0000 7f71fa453740 10 client.0 _ll_register_callbacks cb 0x7f71e80e9d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.243+0000 7f71f5619640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 0x55c761888210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.243+0000 7f71f5619640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 0x55c761888210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32812/0 (socket says 172.21.15.78:32812) 2024-09-11T01:01:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.243+0000 7f71f5619640 1 -- 172.21.15.78:0/3003811045 learned_addr learned my addr 172.21.15.78:0/3003811045 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.243+0000 7f71f661b640 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 0x55c761882c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.243+0000 7f71f661b640 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7618734d0 0x55c761887cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71f661b640 1 -- 172.21.15.78:0/3003811045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 msgr2=0x55c761882c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71f661b640 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 0x55c761882c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71f661b640 1 -- 172.21.15.78:0/3003811045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 msgr2=0x55c761888210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71f661b640 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 0x55c761888210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71f661b640 1 -- 172.21.15.78:0/3003811045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c761888900 con 0x55c7618734d0 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71f5619640 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 0x55c761888210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71f661b640 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7618734d0 0x55c761887cd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009270 tx=0x7f71e0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf6c2a60ef2fc36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71d57fa640 1 -- 172.21.15.78:0/3003811045 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e002d040 con 0x55c7618734d0 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71d57fa640 1 -- 172.21.15.78:0/3003811045 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f71e0032d50 con 0x55c7618734d0 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71fa453740 1 -- 172.21.15.78:0/3003811045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e0005bd0 con 0x55c7618734d0 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71d57fa640 1 -- 172.21.15.78:0/3003811045 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f71e00285c0 con 0x55c7618734d0 2024-09-11T01:01:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71fa453740 10 client.25448 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71fa453740 1 -- 172.21.15.78:0/3003811045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c7618fc970 con 0x55c7618734d0 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.244+0000 7f71d57fa640 1 -- 172.21.15.78:0/3003811045 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e0005bd0 con 0x55c7618734d0 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.245+0000 7f71d57fa640 1 -- 172.21.15.78:0/3003811045 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f71e0024070 con 0x55c7618734d0 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.245+0000 7f71fa453740 10 client.25448 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.245+0000 7f71fa453740 10 client.25448 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.245+0000 7f71fa453740 20 client.25448 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.245+0000 7f71fa453740 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c7618fcf80 0x55c7618ff370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.245+0000 7f71fa453740 4 client.25448 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.245+0000 7f71fa453740 1 -- 172.21.15.78:0/3003811045 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7617e78a0 con 0x55c7618fcf80 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.245+0000 7f71f5619640 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c7618fcf80 0x55c7618ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.245+0000 7f71f5619640 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c7618fcf80 0x55c7618ff370 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=550f5e4073ad9dfe server_cookie=b1f83471f49842b2 in_seq=0 out_seq=0 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.245+0000 7f71d57fa640 10 client.25448 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:01:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.246+0000 7f71d57fa640 1 -- 172.21.15.78:0/3003811045 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7617e78a0 con 0x55c7618fcf80 2024-09-11T01:01:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.246+0000 7f71d57fa640 10 client.25448 handle_command_reply: tid=0 2024-09-11T01:01:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.246+0000 7f71fa453740 1 client.25448 shutdown 2024-09-11T01:01:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.246+0000 7f71fa453740 1 -- 172.21.15.78:0/3003811045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7618734d0 msgr2=0x55c761887cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.246+0000 7f71fa453740 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7618734d0 0x55c761887cd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009270 tx=0x7f71e0005fd0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.246+0000 7f71fa453740 1 -- 172.21.15.78:0/3003811045 shutdown_connections 2024-09-11T01:01:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.246+0000 7f71fa453740 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c7618fcf80 0x55c7618ff370 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.246+0000 7f71fa453740 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c76187fd80 0x55c761882c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.246+0000 7f71fa453740 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c761873df0 0x55c761888210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.246+0000 7f71d57fa640 0 client.25448 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:01:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.246+0000 7f71fa453740 1 --2- 172.21.15.78:0/3003811045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7618734d0 0x55c761887cd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.246+0000 7f71fa453740 1 -- 172.21.15.78:0/3003811045 >> 172.21.15.78:0/3003811045 conn(0x55c761871300 msgr2=0x55c76187e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71fa453740 1 -- 172.21.15.78:0/3003811045 shutdown_connections 2024-09-11T01:01:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71fa453740 1 -- 172.21.15.78:0/3003811045 wait complete. 2024-09-11T01:01:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71fa453740 20 client.25448 trim_cache size 0 max 16384 2024-09-11T01:01:57.266 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:01:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71f807e640 1 -- 172.21.15.78:0/2377570391 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71d80b54d0 msgr2=0x7f71d80b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71f807e640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71d80b54d0 0x7f71d80b7990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f71e80ac4e0 tx=0x7f71e80e9e30 comp rx=0 tx=0).stop 2024-09-11T01:01:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71f807e640 1 -- 172.21.15.78:0/2377570391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0101860 msgr2=0x7f71f019b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71f807e640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0101860 0x7f71f019b4a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e9f0 tx=0x7f71e800eeb0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71f807e640 1 -- 172.21.15.78:0/2377570391 shutdown_connections 2024-09-11T01:01:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71f807e640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f71d80b54d0 0x7f71d80b7990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71f807e640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71f010f410 0x7f71f01a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71f807e640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71f0102180 0x7f71f019b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71f807e640 1 --2- 172.21.15.78:0/2377570391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0101860 0x7f71f019b4a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:57.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71f807e640 1 -- 172.21.15.78:0/2377570391 >> 172.21.15.78:0/2377570391 conn(0x7f71f00fd770 msgr2=0x7f71f00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:57.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71f807e640 1 -- 172.21.15.78:0/2377570391 shutdown_connections 2024-09-11T01:01:57.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:57.247+0000 7f71f807e640 1 -- 172.21.15.78:0/2377570391 wait complete. 2024-09-11T01:01:57.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 267579 11.44 MB/sec execute 5 sec latency 13.734 ms 2024-09-11T01:01:57.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:57 smithi110 ceph-mon[34712]: pgmap v415: 97 pgs: 97 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 340 op/s 2024-09-11T01:01:57.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:57 smithi112 ceph-mon[35384]: pgmap v415: 97 pgs: 97 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 340 op/s 2024-09-11T01:01:57.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:57 smithi078 ceph-mon[31315]: pgmap v415: 97 pgs: 97 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 340 op/s 2024-09-11T01:01:58.045 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:01:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:01:58.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 270017 11.87 MB/sec execute 6 sec latency 10.200 ms 2024-09-11T01:01:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.829+0000 7f1b15733640 1 -- 172.21.15.78:0/2998531787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b10108030 msgr2=0x7f1b1010e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.829+0000 7f1b15733640 1 --2- 172.21.15.78:0/2998531787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b10108030 0x7f1b1010e830 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009ca0 tx=0x7f1b0802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.829+0000 7f1b15733640 1 -- 172.21.15.78:0/2998531787 shutdown_connections 2024-09-11T01:01:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.829+0000 7f1b15733640 1 --2- 172.21.15.78:0/2998531787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b10108030 0x7f1b1010e830 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.829+0000 7f1b15733640 1 --2- 172.21.15.78:0/2998531787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b101076b0 0x7f1b10107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.829+0000 7f1b15733640 1 --2- 172.21.15.78:0/2998531787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b10101ca0 0x7f1b10102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.829+0000 7f1b15733640 1 -- 172.21.15.78:0/2998531787 >> 172.21.15.78:0/2998531787 conn(0x7f1b100fd730 msgr2=0x7f1b100ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.829+0000 7f1b15733640 1 -- 172.21.15.78:0/2998531787 shutdown_connections 2024-09-11T01:01:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.829+0000 7f1b15733640 1 -- 172.21.15.78:0/2998531787 wait complete. 2024-09-11T01:01:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.830+0000 7f1b15733640 1 Processor -- start 2024-09-11T01:01:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.830+0000 7f1b15733640 1 -- start start 2024-09-11T01:01:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.830+0000 7f1b15733640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b10101ca0 0x7f1b101992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.830+0000 7f1b15733640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b101076b0 0x7f1b10199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.830+0000 7f1b15733640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b10108030 0x7f1b1019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.830+0000 7f1b15733640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b10111cb0 con 0x7f1b10101ca0 2024-09-11T01:01:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.830+0000 7f1b15733640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b10111b30 con 0x7f1b10108030 2024-09-11T01:01:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.830+0000 7f1b15733640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b10111e30 con 0x7f1b101076b0 2024-09-11T01:01:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.830+0000 7f1b0effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b10101ca0 0x7f1b101992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.830+0000 7f1b0effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b10101ca0 0x7f1b101992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32830/0 (socket says 172.21.15.78:32830) 2024-09-11T01:01:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.830+0000 7f1b0effd640 1 -- 172.21.15.78:0/4162894037 learned_addr learned my addr 172.21.15.78:0/4162894037 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1b0e7fc640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b101076b0 0x7f1b10199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1b0f7fe640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b10108030 0x7f1b1019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1b0effd640 1 -- 172.21.15.78:0/4162894037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b101076b0 msgr2=0x7f1b10199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1b0effd640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b101076b0 0x7f1b10199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1b0effd640 1 -- 172.21.15.78:0/4162894037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b10108030 msgr2=0x7f1b1019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1b0effd640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b10108030 0x7f1b1019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1b0effd640 1 -- 172.21.15.78:0/4162894037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1019f370 con 0x7f1b10101ca0 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1b0effd640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b10101ca0 0x7f1b101992c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000e9f0 tx=0x7f1b0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1af7fff640 1 -- 172.21.15.78:0/4162894037 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000cd80 con 0x7f1b10101ca0 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1af7fff640 1 -- 172.21.15.78:0/4162894037 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b000108e0 con 0x7f1b10101ca0 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1b15733640 1 -- 172.21.15.78:0/4162894037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1019f660 con 0x7f1b10101ca0 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1af7fff640 1 -- 172.21.15.78:0/4162894037 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00018980 con 0x7f1b10101ca0 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.831+0000 7f1b15733640 1 -- 172.21.15.78:0/4162894037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b101a5ef0 con 0x7f1b10101ca0 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1af7fff640 1 -- 172.21.15.78:0/4162894037 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1b00010430 con 0x7f1b10101ca0 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b17b08740 1 Processor -- start 2024-09-11T01:01:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b17b08740 1 -- start start 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b17b08740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d56444d0 0x5654d56448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b17b08740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d5644df0 0x5654d5650840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b17b08740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d5653170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b17b08740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5654d55baea0 con 0x5654d5644df0 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b17b08740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654d55aa2d0 con 0x5654d5650d80 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b17b08740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5654d55ae0e0 con 0x5654d56444d0 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b0effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d5644df0 0x5654d5650840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b0effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d5644df0 0x5654d5650840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32846/0 (socket says 172.21.15.78:32846) 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b0effd640 1 -- 172.21.15.78:0/2233785296 learned_addr learned my addr 172.21.15.78:0/2233785296 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b0f7fe640 1 --2- 172.21.15.78:0/2233785296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d56444d0 0x5654d56448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.834+0000 7f1b0e7fc640 1 --2- 172.21.15.78:0/2233785296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d5653170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1af7fff640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1af80b54d0 0x7f1af80b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1b0effd640 1 -- 172.21.15.78:0/2233785296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d56444d0 msgr2=0x5654d56448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1b0effd640 1 --2- 172.21.15.78:0/2233785296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d56444d0 0x5654d56448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1b0effd640 1 -- 172.21.15.78:0/2233785296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 msgr2=0x5654d5653170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1b0effd640 1 --2- 172.21.15.78:0/2233785296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d5653170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1b0effd640 1 -- 172.21.15.78:0/2233785296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d5308650 con 0x5654d5644df0 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1b0e7fc640 1 --2- 172.21.15.78:0/2233785296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d5653170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1b0f7fe640 1 --2- 172.21.15.78:0/2233785296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d56444d0 0x5654d56448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1b0f7fe640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1af80b54d0 0x7f1af80b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1af7fff640 1 -- 172.21.15.78:0/4162894037 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1b000e6240 con 0x7f1b10101ca0 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1b0effd640 1 --2- 172.21.15.78:0/2233785296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d5644df0 0x5654d5650840 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1b000e3970 tx=0x7f1b00070fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8936495e713ee91b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1af57fa640 1 -- 172.21.15.78:0/2233785296 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b000ac5b0 con 0x5654d5644df0 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1af57fa640 1 -- 172.21.15.78:0/2233785296 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b000ac2f0 con 0x5654d5644df0 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1af57fa640 1 -- 172.21.15.78:0/2233785296 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b000abf50 con 0x5654d5644df0 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.835+0000 7f1b0f7fe640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1af80b54d0 0x7f1af80b7990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1b0802f500 tx=0x7f1b080095c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.836+0000 7f1b17b08740 1 -- 172.21.15.78:0/2233785296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d5644df0 msgr2=0x5654d5650840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.836+0000 7f1b17b08740 1 --2- 172.21.15.78:0/2233785296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d5644df0 0x5654d5650840 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1b000e3970 tx=0x7f1b00070fc0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.836+0000 7f1b17b08740 1 -- 172.21.15.78:0/2233785296 shutdown_connections 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.836+0000 7f1b17b08740 1 --2- 172.21.15.78:0/2233785296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d5653170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.836+0000 7f1b17b08740 1 --2- 172.21.15.78:0/2233785296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d5644df0 0x5654d5650840 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.836+0000 7f1b17b08740 1 --2- 172.21.15.78:0/2233785296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d56444d0 0x5654d56448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.836+0000 7f1b17b08740 1 -- 172.21.15.78:0/2233785296 >> 172.21.15.78:0/2233785296 conn(0x5654d5642300 msgr2=0x5654d56434a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.836+0000 7f1b17b08740 1 -- 172.21.15.78:0/2233785296 shutdown_connections 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.836+0000 7f1b17b08740 1 -- 172.21.15.78:0/2233785296 wait complete. 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.837+0000 7f1b17b08740 1 -- start start 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.837+0000 7f1b17b08740 1 Processor -- start 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.837+0000 7f1b17b08740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d5650680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.837+0000 7f1b17b08740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 0x5654d564dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.837+0000 7f1b17b08740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d564e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.837+0000 7f1b17b08740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5654d540b700 con 0x5654d56444d0 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.837+0000 7f1b17b08740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654d5545160 con 0x5654d5650d80 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.837+0000 7f1b17b08740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5654d56539a0 con 0x5654d5644df0 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.837+0000 7f1b17b08740 10 client.0 _ll_register_callbacks cb 0x5654d555e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.837+0000 7f1b0e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 0x5654d564dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b0effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d5650680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b0e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 0x5654d564dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52652/0 (socket says 172.21.15.78:52652) 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b0e7fc640 1 -- 172.21.15.78:0/4266940756 learned_addr learned my addr 172.21.15.78:0/4266940756 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b0f7fe640 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d564e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b0e7fc640 1 -- 172.21.15.78:0/4266940756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 msgr2=0x5654d564e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b0e7fc640 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d564e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b0e7fc640 1 -- 172.21.15.78:0/4266940756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 msgr2=0x5654d5650680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b0e7fc640 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d5650680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b0e7fc640 1 -- 172.21.15.78:0/4266940756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d5572c90 con 0x5654d5644df0 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b0effd640 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d5650680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b0e7fc640 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 0x5654d564dad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b040126e0 tx=0x7f1b04012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fd1b1442053be01 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1af2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1af2ff5640 1 -- 172.21.15.78:0/4266940756 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04012190 con 0x5654d5644df0 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b17b08740 1 -- 172.21.15.78:0/4266940756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b000ac2f0 con 0x5654d5644df0 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1af2ff5640 1 -- 172.21.15.78:0/4266940756 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1b0400f920 con 0x5654d5644df0 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1af2ff5640 1 -- 172.21.15.78:0/4266940756 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1b0400fb40 con 0x5654d5644df0 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b17b08740 10 client.25444 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.838+0000 7f1b17b08740 1 -- 172.21.15.78:0/4266940756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5654d56637e0 con 0x5654d5644df0 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.839+0000 7f1af2ff5640 1 -- 172.21.15.78:0/4266940756 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b000ac2f0 con 0x5654d5644df0 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.839+0000 7f1af2ff5640 1 -- 172.21.15.78:0/4266940756 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1b04026420 con 0x5654d5644df0 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.839+0000 7f1b17b08740 10 client.25444 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.839+0000 7f1b17b08740 10 client.25444 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:01:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.839+0000 7f1b17b08740 20 client.25444 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.839+0000 7f1b17b08740 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5654d5664ca0 0x5654d56670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.839+0000 7f1b17b08740 4 client.25444 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.839+0000 7f1b17b08740 1 -- 172.21.15.78:0/4266940756 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654d55bcdf0 con 0x5654d5664ca0 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.839+0000 7f1b0effd640 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5654d5664ca0 0x5654d56670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.839+0000 7f1b0effd640 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5654d5664ca0 0x5654d56670b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b3e01ac5ac98806 server_cookie=5051901974520cdb in_seq=0 out_seq=0 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.839+0000 7f1af2ff5640 10 client.25444 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1af2ff5640 1 -- 172.21.15.78:0/4266940756 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5654d55bcdf0 con 0x5654d5664ca0 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1af2ff5640 10 client.25444 handle_command_reply: tid=0 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1b17b08740 1 client.25444 shutdown 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1b17b08740 1 -- 172.21.15.78:0/4266940756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 msgr2=0x5654d564dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1b17b08740 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 0x5654d564dad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b040126e0 tx=0x7f1b04012ba0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1b17b08740 1 -- 172.21.15.78:0/4266940756 shutdown_connections 2024-09-11T01:01:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1b17b08740 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5654d5664ca0 0x5654d56670b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1b17b08740 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d564e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1b17b08740 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 0x5654d564dad0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1b17b08740 1 --2- 172.21.15.78:0/4266940756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d5650680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1af2ff5640 0 client.25444 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:01:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1b17b08740 1 -- 172.21.15.78:0/4266940756 >> 172.21.15.78:0/4266940756 conn(0x5654d5642300 msgr2=0x5654d5647ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.840+0000 7f1b17b08740 1 -- 172.21.15.78:0/4266940756 shutdown_connections 2024-09-11T01:01:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.841+0000 7f1b17b08740 1 -- 172.21.15.78:0/4266940756 wait complete. 2024-09-11T01:01:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.841+0000 7f1b17b08740 20 client.25444 trim_cache size 0 max 16384 2024-09-11T01:01:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.855+0000 7f1b17b08740 1 Processor -- start 2024-09-11T01:01:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.855+0000 7f1b17b08740 1 -- start start 2024-09-11T01:01:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b17b08740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d565cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b17b08740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 0x5654d5652170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b17b08740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d56526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b17b08740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101ac430 con 0x5654d56444d0 2024-09-11T01:01:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b17b08740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101ae030 con 0x5654d5650d80 2024-09-11T01:01:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b17b08740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101acb00 con 0x5654d5644df0 2024-09-11T01:01:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d565cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d565cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32864/0 (socket says 172.21.15.78:32864) 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0effd640 1 -- 172.21.15.78:0/934716758 learned_addr learned my addr 172.21.15.78:0/934716758 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0f7fe640 1 --2- 172.21.15.78:0/934716758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d56526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0e7fc640 1 --2- 172.21.15.78:0/934716758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 0x5654d5652170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0effd640 1 -- 172.21.15.78:0/934716758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 msgr2=0x5654d5652170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0effd640 1 --2- 172.21.15.78:0/934716758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 0x5654d5652170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0effd640 1 -- 172.21.15.78:0/934716758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 msgr2=0x5654d56526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0effd640 1 --2- 172.21.15.78:0/934716758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d56526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0effd640 1 -- 172.21.15.78:0/934716758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d5642d80 con 0x5654d56444d0 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0e7fc640 1 --2- 172.21.15.78:0/934716758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 0x5654d5652170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0f7fe640 1 --2- 172.21.15.78:0/934716758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d56526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1b0effd640 1 --2- 172.21.15.78:0/934716758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d565cc00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1b000ac4e0 tx=0x7f1b00010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ca02516194b709 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1af4ff9640 1 -- 172.21.15.78:0/934716758 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b000ac130 con 0x5654d56444d0 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.856+0000 7f1af4ff9640 1 -- 172.21.15.78:0/934716758 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b000e9a20 con 0x5654d56444d0 2024-09-11T01:01:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.857+0000 7f1af4ff9640 1 -- 172.21.15.78:0/934716758 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b000b1210 con 0x5654d56444d0 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.857+0000 7f1b17b08740 1 -- 172.21.15.78:0/934716758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 msgr2=0x5654d565cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.857+0000 7f1b17b08740 1 --2- 172.21.15.78:0/934716758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d565cc00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1b000ac4e0 tx=0x7f1b00010ca0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.857+0000 7f1b17b08740 1 -- 172.21.15.78:0/934716758 shutdown_connections 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.857+0000 7f1b17b08740 1 --2- 172.21.15.78:0/934716758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5650d80 0x5654d56526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.857+0000 7f1b17b08740 1 --2- 172.21.15.78:0/934716758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5644df0 0x5654d5652170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.857+0000 7f1b17b08740 1 --2- 172.21.15.78:0/934716758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d565cc00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.857+0000 7f1b17b08740 1 -- 172.21.15.78:0/934716758 >> 172.21.15.78:0/934716758 conn(0x5654d5642300 msgr2=0x5654d5648d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.857+0000 7f1b17b08740 1 -- 172.21.15.78:0/934716758 shutdown_connections 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.857+0000 7f1b17b08740 1 -- 172.21.15.78:0/934716758 wait complete. 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b17b08740 1 -- start start 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b17b08740 1 Processor -- start 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b17b08740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d5659120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b17b08740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5644df0 0x5654d5659680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b17b08740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5650d80 0x5654d5653e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b17b08740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101ad8a0 con 0x5654d56444d0 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b17b08740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101ad1d0 con 0x5654d5644df0 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b17b08740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101aa410 con 0x5654d5650d80 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b17b08740 10 client.0 _ll_register_callbacks cb 0x7f1b04009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b0effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d5659120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b0e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5644df0 0x5654d5659680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b0e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5644df0 0x5654d5659680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55124/0 (socket says 172.21.15.78:55124) 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b0effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d5659120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32876/0 (socket says 172.21.15.78:32876) 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b0e7fc640 1 -- 172.21.15.78:0/3245303900 learned_addr learned my addr 172.21.15.78:0/3245303900 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b0f7fe640 1 --2- 172.21.15.78:0/3245303900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5650d80 0x5654d5653e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b0e7fc640 1 -- 172.21.15.78:0/3245303900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5650d80 msgr2=0x5654d5653e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b0e7fc640 1 --2- 172.21.15.78:0/3245303900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5650d80 0x5654d5653e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b0e7fc640 1 -- 172.21.15.78:0/3245303900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 msgr2=0x5654d5659120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b0e7fc640 1 --2- 172.21.15.78:0/3245303900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d5659120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.858+0000 7f1b0e7fc640 1 -- 172.21.15.78:0/3245303900 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d563d990 con 0x5654d5644df0 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.859+0000 7f1b0e7fc640 1 --2- 172.21.15.78:0/3245303900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5644df0 0x5654d5659680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1b040126b0 tx=0x7f1b04009e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d667cce197dc0a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.859+0000 7f1af2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.859+0000 7f1af2ff5640 1 -- 172.21.15.78:0/3245303900 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04026420 con 0x5654d5644df0 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.859+0000 7f1af2ff5640 1 -- 172.21.15.78:0/3245303900 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1b04018040 con 0x5654d5644df0 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.859+0000 7f1b17b08740 1 -- 172.21.15.78:0/3245303900 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654d55b88a0 con 0x5654d5644df0 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.859+0000 7f1af2ff5640 1 -- 172.21.15.78:0/3245303900 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1b040265c0 con 0x5654d5644df0 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.859+0000 7f1b17b08740 10 client.25466 fetch_fsmap learned FSMap version 31 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.859+0000 7f1b17b08740 1 -- 172.21.15.78:0/3245303900 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5654d56538e0 con 0x5654d5644df0 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.859+0000 7f1af2ff5640 1 -- 172.21.15.78:0/3245303900 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b04026ab0 con 0x5654d5644df0 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.859+0000 7f1af2ff5640 1 -- 172.21.15.78:0/3245303900 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1b0401c630 con 0x5654d5644df0 2024-09-11T01:01:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.859+0000 7f1b17b08740 10 client.25466 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.859+0000 7f1b17b08740 10 client.25466 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.860+0000 7f1b17b08740 20 client.25466 populate_metadata read hostname 'smithi078' 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.860+0000 7f1b17b08740 1 --2- 172.21.15.78:0/3245303900 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5654d56d2d10 0x5654d56d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.860+0000 7f1b17b08740 4 client.25466 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.860+0000 7f1b17b08740 1 -- 172.21.15.78:0/3245303900 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654d55b88a0 con 0x5654d56d2d10 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.860+0000 7f1b0effd640 1 --2- 172.21.15.78:0/3245303900 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5654d56d2d10 0x5654d56d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.860+0000 7f1b0effd640 1 --2- 172.21.15.78:0/3245303900 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5654d56d2d10 0x5654d56d5130 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67fa1bef8a9954a1 server_cookie=724e8e72bd9bc3bd in_seq=0 out_seq=0 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.860+0000 7f1af2ff5640 10 client.25466 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.860+0000 7f1af2ff5640 1 -- 172.21.15.78:0/3245303900 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654d55b88a0 con 0x5654d56d2d10 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.860+0000 7f1af2ff5640 10 client.25466 handle_command_reply: tid=0 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.860+0000 7f1b17b08740 1 client.25466 shutdown 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.860+0000 7f1b17b08740 1 -- 172.21.15.78:0/3245303900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5644df0 msgr2=0x5654d5659680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.860+0000 7f1b17b08740 1 --2- 172.21.15.78:0/3245303900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5644df0 0x5654d5659680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1b040126b0 tx=0x7f1b04009e10 comp rx=0 tx=0).stop 2024-09-11T01:01:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b17b08740 1 -- 172.21.15.78:0/3245303900 shutdown_connections 2024-09-11T01:01:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b17b08740 1 --2- 172.21.15.78:0/3245303900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654d5650d80 0x5654d5653e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b17b08740 1 --2- 172.21.15.78:0/3245303900 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5654d56d2d10 0x5654d56d5130 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b17b08740 1 --2- 172.21.15.78:0/3245303900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654d5644df0 0x5654d5659680 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1af2ff5640 0 client.25466 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:01:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b17b08740 1 --2- 172.21.15.78:0/3245303900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654d56444d0 0x5654d5659120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b17b08740 1 -- 172.21.15.78:0/3245303900 >> 172.21.15.78:0/3245303900 conn(0x5654d5642300 msgr2=0x5654d564f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b17b08740 1 -- 172.21.15.78:0/3245303900 shutdown_connections 2024-09-11T01:01:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b17b08740 1 -- 172.21.15.78:0/3245303900 wait complete. 2024-09-11T01:01:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b17b08740 20 client.25466 trim_cache size 0 max 16384 2024-09-11T01:01:58.877 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:01:58.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b15733640 1 -- 172.21.15.78:0/4162894037 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1af80b54d0 msgr2=0x7f1af80b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:58.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b15733640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1af80b54d0 0x7f1af80b7990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1b0802f500 tx=0x7f1b080095c0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b15733640 1 -- 172.21.15.78:0/4162894037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b10101ca0 msgr2=0x7f1b101992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:01:58.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b15733640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b10101ca0 0x7f1b101992c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000e9f0 tx=0x7f1b0000eeb0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b15733640 1 -- 172.21.15.78:0/4162894037 shutdown_connections 2024-09-11T01:01:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b15733640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1af80b54d0 0x7f1af80b7990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b15733640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b10108030 0x7f1b1019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b15733640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b101076b0 0x7f1b10199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b15733640 1 --2- 172.21.15.78:0/4162894037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b10101ca0 0x7f1b101992c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:01:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.861+0000 7f1b15733640 1 -- 172.21.15.78:0/4162894037 >> 172.21.15.78:0/4162894037 conn(0x7f1b100fd730 msgr2=0x7f1b1006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:01:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.862+0000 7f1b15733640 1 -- 172.21.15.78:0/4162894037 shutdown_connections 2024-09-11T01:01:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:01:58.862+0000 7f1b15733640 1 -- 172.21.15.78:0/4162894037 wait complete. 2024-09-11T01:01:59.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 272496 11.87 MB/sec execute 7 sec latency 29.022 ms 2024-09-11T01:01:59.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:01:59 smithi110 ceph-mon[34712]: pgmap v416: 97 pgs: 97 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-09-11T01:01:59.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:01:59 smithi112 ceph-mon[35384]: pgmap v416: 97 pgs: 97 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-09-11T01:01:59.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:01:59 smithi078 ceph-mon[31315]: pgmap v416: 97 pgs: 97 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-09-11T01:02:00.074 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:02:00.340 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:00.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 274065 11.38 MB/sec execute 8 sec latency 52.882 ms 2024-09-11T01:02:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.899+0000 7f35bb9d1640 1 -- 172.21.15.78:0/1854065362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b4108020 msgr2=0x7f35b410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.899+0000 7f35bb9d1640 1 --2- 172.21.15.78:0/1854065362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b4108020 0x7f35b410e820 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009ca0 tx=0x7f35ac02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.899+0000 7f35bb9d1640 1 -- 172.21.15.78:0/1854065362 shutdown_connections 2024-09-11T01:02:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.899+0000 7f35bb9d1640 1 --2- 172.21.15.78:0/1854065362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b4108020 0x7f35b410e820 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.899+0000 7f35bb9d1640 1 --2- 172.21.15.78:0/1854065362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b41076a0 0x7f35b4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.899+0000 7f35bb9d1640 1 --2- 172.21.15.78:0/1854065362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b4101c90 0x7f35b4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.899+0000 7f35bb9d1640 1 -- 172.21.15.78:0/1854065362 >> 172.21.15.78:0/1854065362 conn(0x7f35b40fd700 msgr2=0x7f35b40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.899+0000 7f35bb9d1640 1 -- 172.21.15.78:0/1854065362 shutdown_connections 2024-09-11T01:02:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.899+0000 7f35bb9d1640 1 -- 172.21.15.78:0/1854065362 wait complete. 2024-09-11T01:02:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35bb9d1640 1 Processor -- start 2024-09-11T01:02:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35bb9d1640 1 -- start start 2024-09-11T01:02:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35bb9d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b4101c90 0x7f35b419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35bb9d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b41076a0 0x7f35b419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35bb9d1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b4108020 0x7f35b41a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35bb9d1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4111ca0 con 0x7f35b41076a0 2024-09-11T01:02:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35bb9d1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4111b20 con 0x7f35b4108020 2024-09-11T01:02:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35bb9d1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4111e20 con 0x7f35b4101c90 2024-09-11T01:02:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35b8f6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b41076a0 0x7f35b419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35b976d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b4101c90 0x7f35b419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35b976d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b4101c90 0x7f35b419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52696/0 (socket says 172.21.15.78:52696) 2024-09-11T01:02:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35b9f6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b4108020 0x7f35b41a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35b976d640 1 -- 172.21.15.78:0/2008807026 learned_addr learned my addr 172.21.15.78:0/2008807026 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.900+0000 7f35b8f6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b41076a0 0x7f35b419b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32886/0 (socket says 172.21.15.78:32886) 2024-09-11T01:02:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.901+0000 7f35b976d640 1 -- 172.21.15.78:0/2008807026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b4108020 msgr2=0x7f35b41a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.901+0000 7f35b976d640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b4108020 0x7f35b41a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.901+0000 7f35b976d640 1 -- 172.21.15.78:0/2008807026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b41076a0 msgr2=0x7f35b419b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.901+0000 7f35b976d640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b41076a0 0x7f35b419b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.901+0000 7f35b976d640 1 -- 172.21.15.78:0/2008807026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b41a14d0 con 0x7f35b4101c90 2024-09-11T01:02:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.901+0000 7f35b8f6c640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b41076a0 0x7f35b419b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.901+0000 7f35b976d640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b4101c90 0x7f35b419b420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35b000d950 tx=0x7f35b000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.901+0000 7f35aa7fc640 1 -- 172.21.15.78:0/2008807026 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0017070 con 0x7f35b4101c90 2024-09-11T01:02:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.901+0000 7f35bb9d1640 1 -- 172.21.15.78:0/2008807026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b41a17c0 con 0x7f35b4101c90 2024-09-11T01:02:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.901+0000 7f35aa7fc640 1 -- 172.21.15.78:0/2008807026 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35b0009c30 con 0x7f35b4101c90 2024-09-11T01:02:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.901+0000 7f35aa7fc640 1 -- 172.21.15.78:0/2008807026 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b000f620 con 0x7f35b4101c90 2024-09-11T01:02:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.901+0000 7f35bb9d1640 1 -- 172.21.15.78:0/2008807026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b41a8050 con 0x7f35b4101c90 2024-09-11T01:02:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35bdda6740 1 Processor -- start 2024-09-11T01:02:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35bdda6740 1 -- start start 2024-09-11T01:02:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35aa7fc640 1 -- 172.21.15.78:0/2008807026 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f35b0010460 con 0x7f35b4101c90 2024-09-11T01:02:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35bdda6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 0x55aacd02e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35bdda6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02ea10 0x55aacd03a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35bdda6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd03a9a0 0x55aacd03cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35bdda6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aaccfa4ea0 con 0x55aacd02ea10 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35bdda6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55aaccf942d0 con 0x55aacd02e0f0 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35bdda6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55aaccf980e0 con 0x55aacd03a9a0 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35b976d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02ea10 0x55aacd03a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35b9f6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 0x55aacd02e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35b9f6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 0x55aacd02e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55152/0 (socket says 172.21.15.78:55152) 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35b976d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02ea10 0x55aacd03a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32890/0 (socket says 172.21.15.78:32890) 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35b9f6e640 1 -- 172.21.15.78:0/3641115686 learned_addr learned my addr 172.21.15.78:0/3641115686 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.904+0000 7f35b8f6c640 1 --2- 172.21.15.78:0/3641115686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd03a9a0 0x55aacd03cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f35b9f6e640 1 -- 172.21.15.78:0/3641115686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd03a9a0 msgr2=0x55aacd03cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f35b9f6e640 1 --2- 172.21.15.78:0/3641115686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd03a9a0 0x55aacd03cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f35b9f6e640 1 -- 172.21.15.78:0/3641115686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02ea10 msgr2=0x55aacd03a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f35b9f6e640 1 --2- 172.21.15.78:0/3641115686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02ea10 0x55aacd03a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f35b9f6e640 1 -- 172.21.15.78:0/3641115686 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aacccf2650 con 0x55aacd02e0f0 2024-09-11T01:02:00.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f35b976d640 1 --2- 172.21.15.78:0/3641115686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02ea10 0x55aacd03a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:00.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f35aa7fc640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f35a00b5450 0x7f35a00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f35b9f6e640 1 --2- 172.21.15.78:0/3641115686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 0x55aacd02e4d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f35ac035000 tx=0x7f35ac009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=979c781e64b4bb60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:00.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f35b976d640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f35a00b5450 0x7f35a00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f359f7fe640 1 -- 172.21.15.78:0/3641115686 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac031cf0 con 0x55aacd02e0f0 2024-09-11T01:02:00.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f359f7fe640 1 -- 172.21.15.78:0/3641115686 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ac036030 con 0x55aacd02e0f0 2024-09-11T01:02:00.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f359f7fe640 1 -- 172.21.15.78:0/3641115686 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac0042a0 con 0x55aacd02e0f0 2024-09-11T01:02:00.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.905+0000 7f35aa7fc640 1 -- 172.21.15.78:0/2008807026 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f35b00e5b10 con 0x7f35b4101c90 2024-09-11T01:02:00.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.906+0000 7f35bdda6740 1 -- 172.21.15.78:0/3641115686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 msgr2=0x55aacd02e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:00.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.906+0000 7f35b976d640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f35a00b5450 0x7f35a00b7910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f35b00e3380 tx=0x7f35b00e9660 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:00.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.906+0000 7f35bdda6740 1 --2- 172.21.15.78:0/3641115686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 0x55aacd02e4d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f35ac035000 tx=0x7f35ac009800 comp rx=0 tx=0).stop 2024-09-11T01:02:00.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.906+0000 7f35bdda6740 1 -- 172.21.15.78:0/3641115686 shutdown_connections 2024-09-11T01:02:00.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.906+0000 7f35bdda6740 1 --2- 172.21.15.78:0/3641115686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd03a9a0 0x55aacd03cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.906+0000 7f35bdda6740 1 --2- 172.21.15.78:0/3641115686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02ea10 0x55aacd03a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.906+0000 7f35bdda6740 1 --2- 172.21.15.78:0/3641115686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 0x55aacd02e4d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.906+0000 7f35bdda6740 1 -- 172.21.15.78:0/3641115686 >> 172.21.15.78:0/3641115686 conn(0x55aacd02bf20 msgr2=0x55aacd02d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:00.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.906+0000 7f35bdda6740 1 -- 172.21.15.78:0/3641115686 shutdown_connections 2024-09-11T01:02:00.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.906+0000 7f35bdda6740 1 -- 172.21.15.78:0/3641115686 wait complete. 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.906+0000 7f35bdda6740 1 -- start start 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35bdda6740 1 Processor -- start 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35bdda6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02e0f0 0x55aacd03a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35bdda6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd037670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35bdda6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd03a9a0 0x55aacd037c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35bdda6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aaccdf5700 con 0x55aacd02e0f0 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35bdda6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55aaccf2f160 con 0x55aacd03a9a0 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35bdda6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55aacd03d440 con 0x55aacd02ea10 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35bdda6740 10 client.0 _ll_register_callbacks cb 0x55aaccf489e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35b9f6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02e0f0 0x55aacd03a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35b9f6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02e0f0 0x55aacd03a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32902/0 (socket says 172.21.15.78:32902) 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35b9f6e640 1 -- 172.21.15.78:0/2722785475 learned_addr learned my addr 172.21.15.78:0/2722785475 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:00.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35b8f6c640 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd037670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.907+0000 7f35b9f6e640 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd03a9a0 0x55aacd037c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f35b9f6e640 1 -- 172.21.15.78:0/2722785475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 msgr2=0x55aacd037670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f35b9f6e640 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd037670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f35b9f6e640 1 -- 172.21.15.78:0/2722785475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd03a9a0 msgr2=0x55aacd037c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f35b9f6e640 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd03a9a0 0x55aacd037c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f35b9f6e640 1 -- 172.21.15.78:0/2722785475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaccf5cc90 con 0x55aacd02e0f0 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f35b9f6e640 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02e0f0 0x55aacd03a220 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f35ac006010 tx=0x7f35ac003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da892749dac3aa0b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f359cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f359cff9640 1 -- 172.21.15.78:0/2722785475 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac058060 con 0x55aacd02e0f0 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f359cff9640 1 -- 172.21.15.78:0/2722785475 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f35ac02f660 con 0x55aacd02e0f0 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f35bdda6740 1 -- 172.21.15.78:0/2722785475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35ac036030 con 0x55aacd02e0f0 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f359cff9640 1 -- 172.21.15.78:0/2722785475 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f35ac02f860 con 0x55aacd02e0f0 2024-09-11T01:02:00.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f35bdda6740 10 client.16071 fetch_fsmap learned FSMap version 31 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f35bdda6740 1 -- 172.21.15.78:0/2722785475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55aacd048d10 con 0x55aacd02e0f0 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f359cff9640 1 -- 172.21.15.78:0/2722785475 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ac036030 con 0x55aacd02e0f0 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f359cff9640 1 -- 172.21.15.78:0/2722785475 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f35ac05b030 con 0x55aacd02e0f0 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f35bdda6740 10 client.16071 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f35bdda6740 10 client.16071 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.908+0000 7f35bdda6740 20 client.16071 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.909+0000 7f35bdda6740 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55aacd04a060 0x55aacd04c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.909+0000 7f35bdda6740 4 client.16071 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.909+0000 7f35bdda6740 1 -- 172.21.15.78:0/2722785475 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaccfa6df0 con 0x55aacd04a060 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.909+0000 7f35b8f6c640 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55aacd04a060 0x55aacd04c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.909+0000 7f359cff9640 10 client.16071 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.909+0000 7f35b8f6c640 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55aacd04a060 0x55aacd04c450 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef69491914dab3f0 server_cookie=3ee28498175876e in_seq=0 out_seq=0 2024-09-11T01:02:00.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f359cff9640 1 -- 172.21.15.78:0/2722785475 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aaccfa6df0 con 0x55aacd04a060 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f359cff9640 10 client.16071 handle_command_reply: tid=0 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f35bdda6740 1 client.16071 shutdown 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f35bdda6740 1 -- 172.21.15.78:0/2722785475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02e0f0 msgr2=0x55aacd03a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f35bdda6740 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02e0f0 0x55aacd03a220 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f35ac006010 tx=0x7f35ac003d50 comp rx=0 tx=0).stop 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f35bdda6740 1 -- 172.21.15.78:0/2722785475 shutdown_connections 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f35bdda6740 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd03a9a0 0x55aacd037c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f35bdda6740 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd037670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f35bdda6740 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55aacd04a060 0x55aacd04c450 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f35bdda6740 1 --2- 172.21.15.78:0/2722785475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02e0f0 0x55aacd03a220 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f359cff9640 0 client.16071 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f35bdda6740 1 -- 172.21.15.78:0/2722785475 >> 172.21.15.78:0/2722785475 conn(0x55aacd02bf20 msgr2=0x55aacd031ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f35bdda6740 1 -- 172.21.15.78:0/2722785475 shutdown_connections 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f35bdda6740 1 -- 172.21.15.78:0/2722785475 wait complete. 2024-09-11T01:02:00.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.910+0000 7f35bdda6740 20 client.16071 trim_cache size 0 max 16384 2024-09-11T01:02:00.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.924+0000 7f35bdda6740 1 Processor -- start 2024-09-11T01:02:00.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.924+0000 7f35bdda6740 1 -- start start 2024-09-11T01:02:00.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.924+0000 7f35bdda6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 0x55aacd0270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.924+0000 7f35bdda6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd0393b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.924+0000 7f35bdda6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd03a9a0 0x55aacd034880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.924+0000 7f35bdda6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41b0630 con 0x55aacd03a9a0 2024-09-11T01:02:00.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.924+0000 7f35bdda6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41b2230 con 0x55aacd02e0f0 2024-09-11T01:02:00.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.924+0000 7f35bdda6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41b0d00 con 0x55aacd02ea10 2024-09-11T01:02:00.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b9f6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 0x55aacd0270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b8f6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd0393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b8f6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd0393b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52726/0 (socket says 172.21.15.78:52726) 2024-09-11T01:02:00.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b9f6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 0x55aacd0270b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55178/0 (socket says 172.21.15.78:55178) 2024-09-11T01:02:00.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b8f6c640 1 -- 172.21.15.78:0/1704399890 learned_addr learned my addr 172.21.15.78:0/1704399890 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:00.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b9f6e640 1 --2- 172.21.15.78:0/1704399890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd03a9a0 0x55aacd034880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b8f6c640 1 -- 172.21.15.78:0/1704399890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 msgr2=0x55aacd0270b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b8f6c640 1 --2- 172.21.15.78:0/1704399890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 0x55aacd0270b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b8f6c640 1 -- 172.21.15.78:0/1704399890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd03a9a0 msgr2=0x55aacd034880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b8f6c640 1 --2- 172.21.15.78:0/1704399890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd03a9a0 0x55aacd034880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b8f6c640 1 -- 172.21.15.78:0/1704399890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aacd02d650 con 0x55aacd02ea10 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b9f6e640 1 --2- 172.21.15.78:0/1704399890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 0x55aacd0270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f35b8f6c640 1 --2- 172.21.15.78:0/1704399890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd0393b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f35b4103e50 tx=0x7f35a400b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a7f2f05fe11716 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.925+0000 7f359effd640 1 -- 172.21.15.78:0/1704399890 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4010ae0 con 0x55aacd02ea10 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.926+0000 7f359effd640 1 -- 172.21.15.78:0/1704399890 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35a40054d0 con 0x55aacd02ea10 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.926+0000 7f359effd640 1 -- 172.21.15.78:0/1704399890 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4004b10 con 0x55aacd02ea10 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.926+0000 7f35bdda6740 1 -- 172.21.15.78:0/1704399890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 msgr2=0x55aacd0393b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.926+0000 7f35bdda6740 1 --2- 172.21.15.78:0/1704399890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd0393b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f35b4103e50 tx=0x7f35a400b410 comp rx=0 tx=0).stop 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.926+0000 7f35bdda6740 1 -- 172.21.15.78:0/1704399890 shutdown_connections 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.926+0000 7f35bdda6740 1 --2- 172.21.15.78:0/1704399890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd03a9a0 0x55aacd034880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.926+0000 7f35bdda6740 1 --2- 172.21.15.78:0/1704399890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd0393b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.926+0000 7f35bdda6740 1 --2- 172.21.15.78:0/1704399890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd02e0f0 0x55aacd0270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.926+0000 7f35bdda6740 1 -- 172.21.15.78:0/1704399890 >> 172.21.15.78:0/1704399890 conn(0x55aacd02bf20 msgr2=0x55aacd038ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.926+0000 7f35bdda6740 1 -- 172.21.15.78:0/1704399890 shutdown_connections 2024-09-11T01:02:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.926+0000 7f35bdda6740 1 -- 172.21.15.78:0/1704399890 wait complete. 2024-09-11T01:02:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.927+0000 7f35bdda6740 1 -- start start 2024-09-11T01:02:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.927+0000 7f35bdda6740 1 Processor -- start 2024-09-11T01:02:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.927+0000 7f35bdda6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02e0f0 0x55aacd042b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.927+0000 7f35bdda6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd0430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.927+0000 7f35bdda6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd03a9a0 0x55aacd03f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.927+0000 7f35bdda6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41b1aa0 con 0x55aacd02e0f0 2024-09-11T01:02:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.927+0000 7f35bdda6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41b13d0 con 0x55aacd03a9a0 2024-09-11T01:02:00.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.927+0000 7f35bdda6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41ae610 con 0x55aacd02ea10 2024-09-11T01:02:00.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.927+0000 7f35bdda6740 10 client.0 _ll_register_callbacks cb 0x7f35b00e9530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:00.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.927+0000 7f35b9f6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02e0f0 0x55aacd042b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.927+0000 7f35b8f6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd0430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f35b8f6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd0430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52740/0 (socket says 172.21.15.78:52740) 2024-09-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f35b8f6c640 1 -- 172.21.15.78:0/3743905688 learned_addr learned my addr 172.21.15.78:0/3743905688 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f35b9f6e640 1 --2- 172.21.15.78:0/3743905688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd03a9a0 0x55aacd03f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f35b8f6c640 1 -- 172.21.15.78:0/3743905688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd03a9a0 msgr2=0x55aacd03f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f35b8f6c640 1 --2- 172.21.15.78:0/3743905688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd03a9a0 0x55aacd03f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f35b8f6c640 1 -- 172.21.15.78:0/3743905688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02e0f0 msgr2=0x55aacd042b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f35b8f6c640 1 --2- 172.21.15.78:0/3743905688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02e0f0 0x55aacd042b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f35b8f6c640 1 -- 172.21.15.78:0/3743905688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aacd03fae0 con 0x55aacd02ea10 2024-09-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f35b8f6c640 1 --2- 172.21.15.78:0/3743905688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd0430e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f35a4026040 tx=0x7f35a4005710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af3915a6dc977833 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f359cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f359cff9640 1 -- 172.21.15.78:0/3743905688 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4011790 con 0x55aacd02ea10 2024-09-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f359cff9640 1 -- 172.21.15.78:0/3743905688 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f35a4005230 con 0x55aacd02ea10 2024-09-11T01:02:00.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f35bdda6740 1 -- 172.21.15.78:0/3743905688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35a40054d0 con 0x55aacd02ea10 2024-09-11T01:02:00.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f359cff9640 1 -- 172.21.15.78:0/3743905688 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f35a40048f0 con 0x55aacd02ea10 2024-09-11T01:02:00.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.928+0000 7f35bdda6740 10 client.25468 fetch_fsmap learned FSMap version 31 2024-09-11T01:02:00.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.929+0000 7f35bdda6740 1 -- 172.21.15.78:0/3743905688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55aacd0b7b10 con 0x55aacd02ea10 2024-09-11T01:02:00.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.929+0000 7f359cff9640 1 -- 172.21.15.78:0/3743905688 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35a40054d0 con 0x55aacd02ea10 2024-09-11T01:02:00.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.929+0000 7f359cff9640 1 -- 172.21.15.78:0/3743905688 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f35a402d480 con 0x55aacd02ea10 2024-09-11T01:02:00.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.929+0000 7f35bdda6740 10 client.25468 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:02:00.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.929+0000 7f35bdda6740 10 client.25468 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:02:00.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.929+0000 7f35bdda6740 20 client.25468 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:00.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.929+0000 7f35bdda6740 1 --2- 172.21.15.78:0/3743905688 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55aacd0b81f0 0x55aacd0ba610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:00.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.929+0000 7f35bdda6740 4 client.25468 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:02:00.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.929+0000 7f35bdda6740 1 -- 172.21.15.78:0/3743905688 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aaccfa28a0 con 0x55aacd0b81f0 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.929+0000 7f35b9f6e640 1 --2- 172.21.15.78:0/3743905688 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55aacd0b81f0 0x55aacd0ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.929+0000 7f359cff9640 10 client.25468 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.929+0000 7f35b9f6e640 1 --2- 172.21.15.78:0/3743905688 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55aacd0b81f0 0x55aacd0ba610 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d956ad3d2db7fbb4 server_cookie=5dff67c65deff9e5 in_seq=0 out_seq=0 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f359cff9640 1 -- 172.21.15.78:0/3743905688 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aaccfa28a0 con 0x55aacd0b81f0 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f359cff9640 10 client.25468 handle_command_reply: tid=0 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f35bdda6740 1 client.25468 shutdown 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f35bdda6740 1 -- 172.21.15.78:0/3743905688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 msgr2=0x55aacd0430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f35bdda6740 1 --2- 172.21.15.78:0/3743905688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd0430e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f35a4026040 tx=0x7f35a4005710 comp rx=0 tx=0).stop 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f35bdda6740 1 -- 172.21.15.78:0/3743905688 shutdown_connections 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f35bdda6740 1 --2- 172.21.15.78:0/3743905688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55aacd03a9a0 0x55aacd03f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f35bdda6740 1 --2- 172.21.15.78:0/3743905688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aacd02ea10 0x55aacd0430e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f35bdda6740 1 --2- 172.21.15.78:0/3743905688 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55aacd0b81f0 0x55aacd0ba610 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f35bdda6740 1 --2- 172.21.15.78:0/3743905688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacd02e0f0 0x55aacd042b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f359cff9640 0 client.25468 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:02:00.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f35bdda6740 1 -- 172.21.15.78:0/3743905688 >> 172.21.15.78:0/3743905688 conn(0x55aacd02bf20 msgr2=0x55aacd02af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:00.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f35bdda6740 1 -- 172.21.15.78:0/3743905688 shutdown_connections 2024-09-11T01:02:00.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f35bdda6740 1 -- 172.21.15.78:0/3743905688 wait complete. 2024-09-11T01:02:00.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.930+0000 7f35bdda6740 20 client.25468 trim_cache size 0 max 16384 2024-09-11T01:02:00.938 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:02:00.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.931+0000 7f35bb9d1640 1 -- 172.21.15.78:0/2008807026 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f35a00b5450 msgr2=0x7f35a00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:00.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.931+0000 7f35bb9d1640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f35a00b5450 0x7f35a00b7910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f35b00e3380 tx=0x7f35b00e9660 comp rx=0 tx=0).stop 2024-09-11T01:02:00.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.931+0000 7f35bb9d1640 1 -- 172.21.15.78:0/2008807026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b4101c90 msgr2=0x7f35b419b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:00.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.931+0000 7f35bb9d1640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b4101c90 0x7f35b419b420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35b000d950 tx=0x7f35b000de10 comp rx=0 tx=0).stop 2024-09-11T01:02:00.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.931+0000 7f35bb9d1640 1 -- 172.21.15.78:0/2008807026 shutdown_connections 2024-09-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.931+0000 7f35bb9d1640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f35a00b5450 0x7f35a00b7910 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.931+0000 7f35bb9d1640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b4108020 0x7f35b41a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.931+0000 7f35bb9d1640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b41076a0 0x7f35b419b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.931+0000 7f35bb9d1640 1 --2- 172.21.15.78:0/2008807026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b4101c90 0x7f35b419b420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:00.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.931+0000 7f35bb9d1640 1 -- 172.21.15.78:0/2008807026 >> 172.21.15.78:0/2008807026 conn(0x7f35b40fd700 msgr2=0x7f35b406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:00.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.931+0000 7f35bb9d1640 1 -- 172.21.15.78:0/2008807026 shutdown_connections 2024-09-11T01:02:00.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:00.931+0000 7f35bb9d1640 1 -- 172.21.15.78:0/2008807026 wait complete. 2024-09-11T01:02:01.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 275869 11.26 MB/sec execute 9 sec latency 44.065 ms 2024-09-11T01:02:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:01 smithi110 ceph-mon[34712]: pgmap v417: 97 pgs: 97 active+clean; 155 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 370 op/s 2024-09-11T01:02:01.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:01 smithi112 ceph-mon[35384]: pgmap v417: 97 pgs: 97 active+clean; 155 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 370 op/s 2024-09-11T01:02:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:01 smithi078 ceph-mon[31315]: pgmap v417: 97 pgs: 97 active+clean; 155 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 370 op/s 2024-09-11T01:02:01.761 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:02:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:02.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 277483 10.99 MB/sec execute 10 sec latency 22.929 ms 2024-09-11T01:02:02.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.512+0000 7fe281f92640 1 -- 172.21.15.78:0/3291929484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe27c109150 msgr2=0x7fe27c10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:02.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.512+0000 7fe281f92640 1 --2- 172.21.15.78:0/3291929484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe27c109150 0x7fe27c10f950 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe274009ca0 tx=0x7fe27402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.512+0000 7fe281f92640 1 -- 172.21.15.78:0/3291929484 shutdown_connections 2024-09-11T01:02:02.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.512+0000 7fe281f92640 1 --2- 172.21.15.78:0/3291929484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe27c109150 0x7fe27c10f950 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.512+0000 7fe281f92640 1 --2- 172.21.15.78:0/3291929484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c1087d0 0x7fe27c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.512+0000 7fe281f92640 1 --2- 172.21.15.78:0/3291929484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c102dc0 0x7fe27c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.512+0000 7fe281f92640 1 -- 172.21.15.78:0/3291929484 >> 172.21.15.78:0/3291929484 conn(0x7fe27c0fe870 msgr2=0x7fe27c100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:02.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.512+0000 7fe281f92640 1 -- 172.21.15.78:0/3291929484 shutdown_connections 2024-09-11T01:02:02.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.512+0000 7fe281f92640 1 -- 172.21.15.78:0/3291929484 wait complete. 2024-09-11T01:02:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe281f92640 1 Processor -- start 2024-09-11T01:02:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe281f92640 1 -- start start 2024-09-11T01:02:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe281f92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c102dc0 0x7fe27c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe281f92640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c1087d0 0x7fe27c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe281f92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe27c109150 0x7fe27c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe281f92640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c112dd0 con 0x7fe27c109150 2024-09-11T01:02:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe281f92640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c112c50 con 0x7fe27c1087d0 2024-09-11T01:02:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe281f92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c112f50 con 0x7fe27c102dc0 2024-09-11T01:02:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe27c109150 0x7fe27c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe27c109150 0x7fe27c075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32938/0 (socket says 172.21.15.78:32938) 2024-09-11T01:02:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe27affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c1087d0 0x7fe27c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe27bfff640 1 -- 172.21.15.78:0/288479034 learned_addr learned my addr 172.21.15.78:0/288479034 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:02.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.513+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c102dc0 0x7fe27c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.514+0000 7fe27affd640 1 -- 172.21.15.78:0/288479034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c102dc0 msgr2=0x7fe27c0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:02.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.514+0000 7fe27affd640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c102dc0 0x7fe27c0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.514+0000 7fe27affd640 1 -- 172.21.15.78:0/288479034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe27c109150 msgr2=0x7fe27c075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:02.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.514+0000 7fe27affd640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe27c109150 0x7fe27c075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.514+0000 7fe27affd640 1 -- 172.21.15.78:0/288479034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c1a7700 con 0x7fe27c1087d0 2024-09-11T01:02:02.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.514+0000 7fe27bfff640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe27c109150 0x7fe27c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.514+0000 7fe27affd640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c1087d0 0x7fe27c0759e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe27000e9f0 tx=0x7fe27000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.514+0000 7fe278ff9640 1 -- 172.21.15.78:0/288479034 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27000ccc0 con 0x7fe27c1087d0 2024-09-11T01:02:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.514+0000 7fe278ff9640 1 -- 172.21.15.78:0/288479034 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe270005020 con 0x7fe27c1087d0 2024-09-11T01:02:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.514+0000 7fe281f92640 1 -- 172.21.15.78:0/288479034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27c1a79f0 con 0x7fe27c1087d0 2024-09-11T01:02:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.514+0000 7fe278ff9640 1 -- 172.21.15.78:0/288479034 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270004a00 con 0x7fe27c1087d0 2024-09-11T01:02:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.514+0000 7fe281f92640 1 -- 172.21.15.78:0/288479034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe27c1a7ee0 con 0x7fe27c1087d0 2024-09-11T01:02:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.516+0000 7fe278ff9640 1 -- 172.21.15.78:0/288479034 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe270004ba0 con 0x7fe27c1087d0 2024-09-11T01:02:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.516+0000 7fe284367740 1 Processor -- start 2024-09-11T01:02:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe284367740 1 -- start start 2024-09-11T01:02:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe284367740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335c0f0 0x563fc335c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe284367740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335ca10 0x563fc3368460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe284367740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc33689a0 0x563fc336ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe284367740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563fc32d2ea0 con 0x563fc335ca10 2024-09-11T01:02:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe284367740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563fc32c22d0 con 0x563fc335c0f0 2024-09-11T01:02:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe284367740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563fc32c60e0 con 0x563fc33689a0 2024-09-11T01:02:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335ca10 0x563fc3368460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335ca10 0x563fc3368460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32942/0 (socket says 172.21.15.78:32942) 2024-09-11T01:02:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27b7fe640 1 -- 172.21.15.78:0/3044708501 learned_addr learned my addr 172.21.15.78:0/3044708501 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27bfff640 1 --2- 172.21.15.78:0/3044708501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335c0f0 0x563fc335c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27affd640 1 --2- 172.21.15.78:0/3044708501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc33689a0 0x563fc336ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe278ff9640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe2640b5450 0x7fe2640b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27b7fe640 1 -- 172.21.15.78:0/3044708501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc33689a0 msgr2=0x563fc336ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27b7fe640 1 --2- 172.21.15.78:0/3044708501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc33689a0 0x563fc336ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27b7fe640 1 -- 172.21.15.78:0/3044708501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335c0f0 msgr2=0x563fc335c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27b7fe640 1 --2- 172.21.15.78:0/3044708501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335c0f0 0x563fc335c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27b7fe640 1 -- 172.21.15.78:0/3044708501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc3020650 con 0x563fc335ca10 2024-09-11T01:02:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27affd640 1 --2- 172.21.15.78:0/3044708501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc33689a0 0x563fc336ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27bfff640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe2640b5450 0x7fe2640b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe27b7fe640 1 --2- 172.21.15.78:0/3044708501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335ca10 0x563fc3368460 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009cf0 tx=0x7fe26c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2311ef7250f22f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.517+0000 7fe278ff9640 1 -- 172.21.15.78:0/288479034 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe2700e5ea0 con 0x7fe27c1087d0 2024-09-11T01:02:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.518+0000 7fe261ffb640 1 -- 172.21.15.78:0/3044708501 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c01e070 con 0x563fc335ca10 2024-09-11T01:02:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.518+0000 7fe261ffb640 1 -- 172.21.15.78:0/3044708501 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe26c019ae0 con 0x563fc335ca10 2024-09-11T01:02:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.518+0000 7fe261ffb640 1 -- 172.21.15.78:0/3044708501 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00e9e0 con 0x563fc335ca10 2024-09-11T01:02:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.518+0000 7fe284367740 1 -- 172.21.15.78:0/3044708501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335ca10 msgr2=0x563fc3368460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.518+0000 7fe27bfff640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe2640b5450 0x7fe2640b7910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe27c077560 tx=0x7fe2740095c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.518+0000 7fe284367740 1 --2- 172.21.15.78:0/3044708501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335ca10 0x563fc3368460 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009cf0 tx=0x7fe26c011810 comp rx=0 tx=0).stop 2024-09-11T01:02:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.518+0000 7fe284367740 1 -- 172.21.15.78:0/3044708501 shutdown_connections 2024-09-11T01:02:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.518+0000 7fe284367740 1 --2- 172.21.15.78:0/3044708501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc33689a0 0x563fc336ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.518+0000 7fe284367740 1 --2- 172.21.15.78:0/3044708501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335ca10 0x563fc3368460 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.518+0000 7fe284367740 1 --2- 172.21.15.78:0/3044708501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335c0f0 0x563fc335c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.518+0000 7fe284367740 1 -- 172.21.15.78:0/3044708501 >> 172.21.15.78:0/3044708501 conn(0x563fc3359f20 msgr2=0x563fc335b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.518+0000 7fe284367740 1 -- 172.21.15.78:0/3044708501 shutdown_connections 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.519+0000 7fe284367740 1 -- 172.21.15.78:0/3044708501 wait complete. 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.519+0000 7fe284367740 1 -- start start 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.519+0000 7fe284367740 1 Processor -- start 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.519+0000 7fe284367740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335c0f0 0x563fc33682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.519+0000 7fe284367740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335ca10 0x563fc3365720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.519+0000 7fe284367740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc33689a0 0x563fc3365da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.519+0000 7fe284367740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563fc3123700 con 0x563fc33689a0 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.519+0000 7fe284367740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563fc325d160 con 0x563fc335c0f0 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.519+0000 7fe284367740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563fc336b620 con 0x563fc335ca10 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.519+0000 7fe284367740 10 client.0 _ll_register_callbacks cb 0x563fc32769e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335c0f0 0x563fc33682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc33689a0 0x563fc3365da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe27affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335ca10 0x563fc3365720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335c0f0 0x563fc33682d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55210/0 (socket says 172.21.15.78:55210) 2024-09-11T01:02:02.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe27b7fe640 1 -- 172.21.15.78:0/664833313 learned_addr learned my addr 172.21.15.78:0/664833313 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:02.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe27b7fe640 1 -- 172.21.15.78:0/664833313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335ca10 msgr2=0x563fc3365720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:02.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe27b7fe640 1 --2- 172.21.15.78:0/664833313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335ca10 0x563fc3365720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe27b7fe640 1 -- 172.21.15.78:0/664833313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc33689a0 msgr2=0x563fc3365da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:02.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe27b7fe640 1 --2- 172.21.15.78:0/664833313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc33689a0 0x563fc3365da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe27b7fe640 1 -- 172.21.15.78:0/664833313 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc328ac90 con 0x563fc335c0f0 2024-09-11T01:02:02.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe27bfff640 1 --2- 172.21.15.78:0/664833313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc33689a0 0x563fc3365da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe27b7fe640 1 --2- 172.21.15.78:0/664833313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335c0f0 0x563fc33682d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0058c0 tx=0x7fe26c009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b5f14a75134632 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe25affd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:02:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe25affd640 1 -- 172.21.15.78:0/664833313 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c01e040 con 0x563fc335c0f0 2024-09-11T01:02:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe25affd640 1 -- 172.21.15.78:0/664833313 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe26c019840 con 0x563fc335c0f0 2024-09-11T01:02:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe284367740 1 -- 172.21.15.78:0/664833313 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe26c019ae0 con 0x563fc335c0f0 2024-09-11T01:02:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe25affd640 1 -- 172.21.15.78:0/664833313 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe26c032050 con 0x563fc335c0f0 2024-09-11T01:02:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe284367740 10 client.25496 fetch_fsmap learned FSMap version 31 2024-09-11T01:02:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.520+0000 7fe284367740 1 -- 172.21.15.78:0/664833313 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563fc3376fc0 con 0x563fc335c0f0 2024-09-11T01:02:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.521+0000 7fe25affd640 1 -- 172.21.15.78:0/664833313 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe26c019ae0 con 0x563fc335c0f0 2024-09-11T01:02:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.521+0000 7fe25affd640 1 -- 172.21.15.78:0/664833313 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fe26c00e760 con 0x563fc335c0f0 2024-09-11T01:02:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.521+0000 7fe284367740 10 client.25496 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:02:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.521+0000 7fe284367740 10 client.25496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.521+0000 7fe284367740 20 client.25496 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.521+0000 7fe284367740 1 --2- 172.21.15.78:0/664833313 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563fc33783c0 0x563fc337a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.521+0000 7fe284367740 4 client.25496 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:02:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.521+0000 7fe284367740 1 -- 172.21.15.78:0/664833313 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fc32d4df0 con 0x563fc33783c0 2024-09-11T01:02:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.521+0000 7fe27affd640 1 --2- 172.21.15.78:0/664833313 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563fc33783c0 0x563fc337a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.521+0000 7fe25affd640 10 client.25496 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.521+0000 7fe27affd640 1 --2- 172.21.15.78:0/664833313 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563fc33783c0 0x563fc337a7b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908e24cd05edea14 server_cookie=6ecfe41fe9d60b29 in_seq=0 out_seq=0 2024-09-11T01:02:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe25affd640 1 -- 172.21.15.78:0/664833313 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563fc32d4df0 con 0x563fc33783c0 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe25affd640 10 client.25496 handle_command_reply: tid=0 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe284367740 1 client.25496 shutdown 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe284367740 1 -- 172.21.15.78:0/664833313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335c0f0 msgr2=0x563fc33682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe284367740 1 --2- 172.21.15.78:0/664833313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335c0f0 0x563fc33682d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0058c0 tx=0x7fe26c009450 comp rx=0 tx=0).stop 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe284367740 1 -- 172.21.15.78:0/664833313 shutdown_connections 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe284367740 1 --2- 172.21.15.78:0/664833313 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563fc33783c0 0x563fc337a7b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe284367740 1 --2- 172.21.15.78:0/664833313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc33689a0 0x563fc3365da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe284367740 1 --2- 172.21.15.78:0/664833313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335ca10 0x563fc3365720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe25affd640 0 client.25496 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe284367740 1 --2- 172.21.15.78:0/664833313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335c0f0 0x563fc33682d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe284367740 1 -- 172.21.15.78:0/664833313 >> 172.21.15.78:0/664833313 conn(0x563fc3359f20 msgr2=0x563fc335fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe284367740 1 -- 172.21.15.78:0/664833313 shutdown_connections 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe284367740 1 -- 172.21.15.78:0/664833313 wait complete. 2024-09-11T01:02:02.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.522+0000 7fe284367740 20 client.25496 trim_cache size 0 max 16384 2024-09-11T01:02:02.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe284367740 1 Processor -- start 2024-09-11T01:02:02.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe284367740 1 -- start start 2024-09-11T01:02:02.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe284367740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335c0f0 0x563fc3374880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe284367740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335ca10 0x563fc3359760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe284367740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc33689a0 0x563fc3369d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe284367740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1b0830 con 0x563fc335c0f0 2024-09-11T01:02:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe284367740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1b2430 con 0x563fc33689a0 2024-09-11T01:02:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe284367740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1b0f00 con 0x563fc335ca10 2024-09-11T01:02:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335c0f0 0x563fc3374880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe27affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335ca10 0x563fc3359760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335c0f0 0x563fc3374880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32972/0 (socket says 172.21.15.78:32972) 2024-09-11T01:02:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe27b7fe640 1 -- 172.21.15.78:0/2839082912 learned_addr learned my addr 172.21.15.78:0/2839082912 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe27bfff640 1 --2- 172.21.15.78:0/2839082912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc33689a0 0x563fc3369d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe27b7fe640 1 -- 172.21.15.78:0/2839082912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335ca10 msgr2=0x563fc3359760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe27b7fe640 1 --2- 172.21.15.78:0/2839082912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335ca10 0x563fc3359760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe27b7fe640 1 -- 172.21.15.78:0/2839082912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc33689a0 msgr2=0x563fc3369d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe27b7fe640 1 --2- 172.21.15.78:0/2839082912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc33689a0 0x563fc3369d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.537+0000 7fe27b7fe640 1 -- 172.21.15.78:0/2839082912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc337ade0 con 0x563fc335c0f0 2024-09-11T01:02:02.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.538+0000 7fe27b7fe640 1 --2- 172.21.15.78:0/2839082912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335c0f0 0x563fc3374880 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe26c005bc0 tx=0x7fe26c009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f445902834ea598 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:02.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.538+0000 7fe2617fa640 1 -- 172.21.15.78:0/2839082912 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00eda0 con 0x563fc335c0f0 2024-09-11T01:02:02.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.538+0000 7fe2617fa640 1 -- 172.21.15.78:0/2839082912 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe26c015070 con 0x563fc335c0f0 2024-09-11T01:02:02.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.538+0000 7fe2617fa640 1 -- 172.21.15.78:0/2839082912 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c003850 con 0x563fc335c0f0 2024-09-11T01:02:02.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.538+0000 7fe284367740 1 -- 172.21.15.78:0/2839082912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335c0f0 msgr2=0x563fc3374880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:02.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.538+0000 7fe284367740 1 --2- 172.21.15.78:0/2839082912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335c0f0 0x563fc3374880 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe26c005bc0 tx=0x7fe26c009b70 comp rx=0 tx=0).stop 2024-09-11T01:02:02.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 -- 172.21.15.78:0/2839082912 shutdown_connections 2024-09-11T01:02:02.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 --2- 172.21.15.78:0/2839082912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc33689a0 0x563fc3369d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 --2- 172.21.15.78:0/2839082912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335ca10 0x563fc3359760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 --2- 172.21.15.78:0/2839082912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc335c0f0 0x563fc3374880 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 -- 172.21.15.78:0/2839082912 >> 172.21.15.78:0/2839082912 conn(0x563fc3359f20 msgr2=0x563fc335b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:02.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 -- 172.21.15.78:0/2839082912 shutdown_connections 2024-09-11T01:02:02.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 -- 172.21.15.78:0/2839082912 wait complete. 2024-09-11T01:02:02.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 -- start start 2024-09-11T01:02:02.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 Processor -- start 2024-09-11T01:02:02.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335c0f0 0x563fc3371160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335ca10 0x563fc337fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc33689a0 0x563fc3380330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.539+0000 7fe284367740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1b1ca0 con 0x563fc33689a0 2024-09-11T01:02:02.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe284367740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1b15d0 con 0x563fc335ca10 2024-09-11T01:02:02.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe284367740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1ae810 con 0x563fc335c0f0 2024-09-11T01:02:02.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe284367740 10 client.0 _ll_register_callbacks cb 0x7fe26c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:02.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335c0f0 0x563fc3371160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc33689a0 0x563fc3380330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335ca10 0x563fc337fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335ca10 0x563fc337fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55228/0 (socket says 172.21.15.78:55228) 2024-09-11T01:02:02.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335c0f0 0x563fc3371160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52804/0 (socket says 172.21.15.78:52804) 2024-09-11T01:02:02.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27affd640 1 -- 172.21.15.78:0/1599450681 learned_addr learned my addr 172.21.15.78:0/1599450681 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:02.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27bfff640 1 -- 172.21.15.78:0/1599450681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335c0f0 msgr2=0x563fc3371160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:02.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27bfff640 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335c0f0 0x563fc3371160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27bfff640 1 -- 172.21.15.78:0/1599450681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335ca10 msgr2=0x563fc337fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:02.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27bfff640 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335ca10 0x563fc337fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27bfff640 1 -- 172.21.15.78:0/1599450681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc336fdc0 con 0x563fc33689a0 2024-09-11T01:02:02.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27b7fe640 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335c0f0 0x563fc3371160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:02.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27affd640 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335ca10 0x563fc337fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:02.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe27bfff640 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc33689a0 0x563fc3380330 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe274005cc0 tx=0x7fe27402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f2044dfc2c8207 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:02.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe25affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:02.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe25affd640 1 -- 172.21.15.78:0/1599450681 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27405b060 con 0x563fc33689a0 2024-09-11T01:02:02.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe25affd640 1 -- 172.21.15.78:0/1599450681 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe274004790 con 0x563fc33689a0 2024-09-11T01:02:02.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe284367740 1 -- 172.21.15.78:0/1599450681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe26c015070 con 0x563fc33689a0 2024-09-11T01:02:02.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.540+0000 7fe25affd640 1 -- 172.21.15.78:0/1599450681 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe27402f660 con 0x563fc33689a0 2024-09-11T01:02:02.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.541+0000 7fe284367740 10 client.16095 fetch_fsmap learned FSMap version 31 2024-09-11T01:02:02.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.541+0000 7fe284367740 1 -- 172.21.15.78:0/1599450681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563fc33e5390 con 0x563fc33689a0 2024-09-11T01:02:02.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.541+0000 7fe25affd640 1 -- 172.21.15.78:0/1599450681 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe26c015070 con 0x563fc33689a0 2024-09-11T01:02:02.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.541+0000 7fe25affd640 1 -- 172.21.15.78:0/1599450681 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fe27405e030 con 0x563fc33689a0 2024-09-11T01:02:02.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.541+0000 7fe284367740 10 client.16095 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:02:02.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.541+0000 7fe284367740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:02.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.541+0000 7fe284367740 20 client.16095 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:02.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.541+0000 7fe284367740 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563fc33e5c00 0x563fc33e8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:02.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.541+0000 7fe284367740 4 client.16095 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:02:02.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.541+0000 7fe284367740 1 -- 172.21.15.78:0/1599450681 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fc32d08a0 con 0x563fc33e5c00 2024-09-11T01:02:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.541+0000 7fe27b7fe640 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563fc33e5c00 0x563fc33e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe25affd640 10 client.16095 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe27b7fe640 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563fc33e5c00 0x563fc33e8020 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=202025d93ca99ffa server_cookie=73e3fd312a790248 in_seq=0 out_seq=0 2024-09-11T01:02:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe25affd640 1 -- 172.21.15.78:0/1599450681 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fc32d08a0 con 0x563fc33e5c00 2024-09-11T01:02:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe25affd640 10 client.16095 handle_command_reply: tid=0 2024-09-11T01:02:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe284367740 1 client.16095 shutdown 2024-09-11T01:02:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe284367740 1 -- 172.21.15.78:0/1599450681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc33689a0 msgr2=0x563fc3380330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe284367740 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc33689a0 0x563fc3380330 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe274005cc0 tx=0x7fe27402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe284367740 1 -- 172.21.15.78:0/1599450681 shutdown_connections 2024-09-11T01:02:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe284367740 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563fc33e5c00 0x563fc33e8020 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe284367740 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc33689a0 0x563fc3380330 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe284367740 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563fc335ca10 0x563fc337fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe284367740 1 --2- 172.21.15.78:0/1599450681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fc335c0f0 0x563fc3371160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe284367740 1 -- 172.21.15.78:0/1599450681 >> 172.21.15.78:0/1599450681 conn(0x563fc3359f20 msgr2=0x563fc3366d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:02.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe25affd640 0 client.16095 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:02.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe284367740 1 -- 172.21.15.78:0/1599450681 shutdown_connections 2024-09-11T01:02:02.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe284367740 1 -- 172.21.15.78:0/1599450681 wait complete. 2024-09-11T01:02:02.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.542+0000 7fe284367740 20 client.16095 trim_cache size 0 max 16384 2024-09-11T01:02:02.567 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:02:02.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.543+0000 7fe281f92640 1 -- 172.21.15.78:0/288479034 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe2640b5450 msgr2=0x7fe2640b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:02.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.543+0000 7fe281f92640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe2640b5450 0x7fe2640b7910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe27c077560 tx=0x7fe2740095c0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.543+0000 7fe281f92640 1 -- 172.21.15.78:0/288479034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c1087d0 msgr2=0x7fe27c0759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:02.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.543+0000 7fe281f92640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c1087d0 0x7fe27c0759e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe27000e9f0 tx=0x7fe27000eeb0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.543+0000 7fe281f92640 1 -- 172.21.15.78:0/288479034 shutdown_connections 2024-09-11T01:02:02.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.543+0000 7fe281f92640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe2640b5450 0x7fe2640b7910 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.543+0000 7fe281f92640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe27c109150 0x7fe27c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.543+0000 7fe281f92640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c1087d0 0x7fe27c0759e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.543+0000 7fe281f92640 1 --2- 172.21.15.78:0/288479034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c102dc0 0x7fe27c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:02.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.543+0000 7fe281f92640 1 -- 172.21.15.78:0/288479034 >> 172.21.15.78:0/288479034 conn(0x7fe27c0fe870 msgr2=0x7fe27c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:02.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.543+0000 7fe281f92640 1 -- 172.21.15.78:0/288479034 shutdown_connections 2024-09-11T01:02:02.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:02.543+0000 7fe281f92640 1 -- 172.21.15.78:0/288479034 wait complete. 2024-09-11T01:02:03.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:03 smithi078 ceph-mon[31315]: pgmap v418: 97 pgs: 97 active+clean; 156 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 333 op/s 2024-09-11T01:02:03.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 280371 11.57 MB/sec execute 11 sec latency 9.891 ms 2024-09-11T01:02:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:03 smithi110 ceph-mon[34712]: pgmap v418: 97 pgs: 97 active+clean; 156 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 333 op/s 2024-09-11T01:02:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:03 smithi112 ceph-mon[35384]: pgmap v418: 97 pgs: 97 active+clean; 156 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 333 op/s 2024-09-11T01:02:04.364 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:02:04.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:02:04.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 281827 11.15 MB/sec execute 12 sec latency 17.618 ms 2024-09-11T01:02:04.635 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:05.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:05 smithi078 ceph-mon[31315]: pgmap v419: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 379 op/s 2024-09-11T01:02:05.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.239+0000 7f25dca95640 1 -- 172.21.15.78:0/1243942362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25d8101c90 msgr2=0x7f25d8102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:05.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.239+0000 7f25dca95640 1 --2- 172.21.15.78:0/1243942362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25d8101c90 0x7f25d8102070 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009ca0 tx=0x7f25d002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.239+0000 7f25dca95640 1 -- 172.21.15.78:0/1243942362 shutdown_connections 2024-09-11T01:02:05.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.239+0000 7f25dca95640 1 --2- 172.21.15.78:0/1243942362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8108020 0x7f25d810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.239+0000 7f25dca95640 1 --2- 172.21.15.78:0/1243942362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d81076a0 0x7f25d8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.239+0000 7f25dca95640 1 --2- 172.21.15.78:0/1243942362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25d8101c90 0x7f25d8102070 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.239+0000 7f25dca95640 1 -- 172.21.15.78:0/1243942362 >> 172.21.15.78:0/1243942362 conn(0x7f25d80fd700 msgr2=0x7f25d80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:05.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.239+0000 7f25dca95640 1 -- 172.21.15.78:0/1243942362 shutdown_connections 2024-09-11T01:02:05.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.239+0000 7f25dca95640 1 -- 172.21.15.78:0/1243942362 wait complete. 2024-09-11T01:02:05.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.240+0000 7f25dca95640 1 Processor -- start 2024-09-11T01:02:05.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.240+0000 7f25dca95640 1 -- start start 2024-09-11T01:02:05.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25dca95640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8101c90 0x7f25d819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25dca95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d81076a0 0x7f25d819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25dca95640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25d8108020 0x7f25d819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25dca95640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8111ca0 con 0x7f25d8108020 2024-09-11T01:02:05.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25dca95640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8111b20 con 0x7f25d8101c90 2024-09-11T01:02:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25dca95640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8111e20 con 0x7f25d81076a0 2024-09-11T01:02:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25d8108020 0x7f25d819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8101c90 0x7f25d819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d81076a0 0x7f25d819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8101c90 0x7f25d819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:37290/0 (socket says 172.21.15.78:37290) 2024-09-11T01:02:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25d8108020 0x7f25d819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55834/0 (socket says 172.21.15.78:55834) 2024-09-11T01:02:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d81076a0 0x7f25d819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42396/0 (socket says 172.21.15.78:42396) 2024-09-11T01:02:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d659c640 1 -- 172.21.15.78:0/1086852489 learned_addr learned my addr 172.21.15.78:0/1086852489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d6d9d640 1 -- 172.21.15.78:0/1086852489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d81076a0 msgr2=0x7f25d819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d6d9d640 1 --2- 172.21.15.78:0/1086852489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d81076a0 0x7f25d819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d6d9d640 1 -- 172.21.15.78:0/1086852489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8101c90 msgr2=0x7f25d819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d6d9d640 1 --2- 172.21.15.78:0/1086852489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8101c90 0x7f25d819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d6d9d640 1 -- 172.21.15.78:0/1086852489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d81a5490 con 0x7f25d8108020 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25d6d9d640 1 --2- 172.21.15.78:0/1086852489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25d8108020 0x7f25d819ab50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f25c800d680 tx=0x7f25c800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25cf7fe640 1 -- 172.21.15.78:0/1086852489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8017070 con 0x7f25d8108020 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25cf7fe640 1 -- 172.21.15.78:0/1086852489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c8009820 con 0x7f25d8108020 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25dca95640 1 -- 172.21.15.78:0/1086852489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d81a5780 con 0x7f25d8108020 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25cf7fe640 1 -- 172.21.15.78:0/1086852489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8004ce0 con 0x7f25d8108020 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.241+0000 7f25dca95640 1 -- 172.21.15.78:0/1086852489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d81a5c70 con 0x7f25d8108020 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.244+0000 7f25dee6a740 1 Processor -- start 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.244+0000 7f25cf7fe640 1 -- 172.21.15.78:0/1086852489 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f25c8005220 con 0x7f25d8108020 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.244+0000 7f25dee6a740 1 -- start start 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.244+0000 7f25dee6a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aa62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.244+0000 7f25dee6a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15aa6800 0x555d15ab2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.244+0000 7f25dee6a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15ab2790 0x555d15ab4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.244+0000 7f25dee6a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555d15a1ce60 con 0x555d15aa6800 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.244+0000 7f25dee6a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555d15a0c290 con 0x555d15ab2790 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.244+0000 7f25dee6a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555d15a100a0 con 0x555d15aa5ee0 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15aa6800 0x555d15ab2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aa62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aa62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42402/0 (socket says 172.21.15.78:42402) 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15aa6800 0x555d15ab2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55842/0 (socket says 172.21.15.78:55842) 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d5d9b640 1 -- 172.21.15.78:0/368804891 learned_addr learned my addr 172.21.15.78:0/368804891 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d5d9b640 1 --2- 172.21.15.78:0/368804891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15ab2790 0x555d15ab4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d5d9b640 1 -- 172.21.15.78:0/368804891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15ab2790 msgr2=0x555d15ab4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d5d9b640 1 --2- 172.21.15.78:0/368804891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15ab2790 0x555d15ab4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d5d9b640 1 -- 172.21.15.78:0/368804891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15aa6800 msgr2=0x555d15ab2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d5d9b640 1 --2- 172.21.15.78:0/368804891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15aa6800 0x555d15ab2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25cf7fe640 1 --2- 172.21.15.78:0/1086852489 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f25c40b5450 0x7f25c40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d5d9b640 1 -- 172.21.15.78:0/368804891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d1576a930 con 0x555d15aa5ee0 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d659c640 1 --2- 172.21.15.78:0/368804891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15aa6800 0x555d15ab2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d5d9b640 1 --2- 172.21.15.78:0/1086852489 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f25c40b5450 0x7f25c40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25cf7fe640 1 -- 172.21.15.78:0/1086852489 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f25c80e5730 con 0x7f25d8108020 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25d5d9b640 1 --2- 172.21.15.78:0/368804891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aa62c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25c0012840 tx=0x7f25c0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f24c0e57f556f81c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25ccff9640 1 -- 172.21.15.78:0/368804891 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0026070 con 0x555d15aa5ee0 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.245+0000 7f25ccff9640 1 -- 172.21.15.78:0/368804891 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c0020de0 con 0x555d15aa5ee0 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.246+0000 7f25ccff9640 1 -- 172.21.15.78:0/368804891 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c001f9d0 con 0x555d15aa5ee0 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.246+0000 7f25dee6a740 1 -- 172.21.15.78:0/368804891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 msgr2=0x555d15aa62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.246+0000 7f25d5d9b640 1 --2- 172.21.15.78:0/1086852489 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f25c40b5450 0x7f25c40b7910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25c0023000 tx=0x7f25c0024000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.246+0000 7f25dee6a740 1 --2- 172.21.15.78:0/368804891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aa62c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25c0012840 tx=0x7f25c0004730 comp rx=0 tx=0).stop 2024-09-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.246+0000 7f25dee6a740 1 -- 172.21.15.78:0/368804891 shutdown_connections 2024-09-11T01:02:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.246+0000 7f25dee6a740 1 --2- 172.21.15.78:0/368804891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15ab2790 0x555d15ab4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.246+0000 7f25dee6a740 1 --2- 172.21.15.78:0/368804891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15aa6800 0x555d15ab2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.246+0000 7f25dee6a740 1 --2- 172.21.15.78:0/368804891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aa62c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.246+0000 7f25dee6a740 1 -- 172.21.15.78:0/368804891 >> 172.21.15.78:0/368804891 conn(0x555d15aa3d10 msgr2=0x555d15aa4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.246+0000 7f25dee6a740 1 -- 172.21.15.78:0/368804891 shutdown_connections 2024-09-11T01:02:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.246+0000 7f25dee6a740 1 -- 172.21.15.78:0/368804891 wait complete. 2024-09-11T01:02:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25dee6a740 1 -- start start 2024-09-11T01:02:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25dee6a740 1 Processor -- start 2024-09-11T01:02:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25dee6a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aaf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25dee6a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15aa6800 0x555d15aafd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25dee6a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15ab2790 0x555d15ab0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25dee6a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555d1586d890 con 0x555d15ab2790 2024-09-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25dee6a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555d159a6d40 con 0x555d15aa6800 2024-09-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25dee6a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555d15ab5440 con 0x555d15aa5ee0 2024-09-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25d6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15ab2790 0x555d15ab0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25d659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aaf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25d659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aaf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42416/0 (socket says 172.21.15.78:42416) 2024-09-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25d5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15aa6800 0x555d15aafd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25d659c640 1 -- 172.21.15.78:0/4155608253 learned_addr learned my addr 172.21.15.78:0/4155608253 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.247+0000 7f25dee6a740 10 client.0 _ll_register_callbacks cb 0x555d15a0ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25d6d9d640 1 -- 172.21.15.78:0/4155608253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 msgr2=0x555d15aaf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25d6d9d640 1 --2- 172.21.15.78:0/4155608253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aaf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25d6d9d640 1 -- 172.21.15.78:0/4155608253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15aa6800 msgr2=0x555d15aafd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25d6d9d640 1 --2- 172.21.15.78:0/4155608253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15aa6800 0x555d15aafd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25d6d9d640 1 -- 172.21.15.78:0/4155608253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d15abe8f0 con 0x555d15ab2790 2024-09-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25d659c640 1 --2- 172.21.15.78:0/4155608253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aaf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25d6d9d640 1 --2- 172.21.15.78:0/4155608253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15ab2790 0x555d15ab0240 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f25c80e2ef0 tx=0x7f25c8009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7362a788d6414734 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25bdffb640 1 -- 172.21.15.78:0/4155608253 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8020070 con 0x555d15ab2790 2024-09-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25bdffb640 1 -- 172.21.15.78:0/4155608253 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f25c80e3170 con 0x555d15ab2790 2024-09-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25dee6a740 1 -- 172.21.15.78:0/4155608253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c0020de0 con 0x555d15ab2790 2024-09-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25bdffb640 1 -- 172.21.15.78:0/4155608253 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f25c80abcd0 con 0x555d15ab2790 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25dee6a740 10 client.16107 fetch_fsmap learned FSMap version 31 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25dee6a740 1 -- 172.21.15.78:0/4155608253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555d15ac1280 con 0x555d15ab2790 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.248+0000 7f25bdffb640 1 -- 172.21.15.78:0/4155608253 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c0020de0 con 0x555d15ab2790 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.249+0000 7f25bdffb640 1 -- 172.21.15.78:0/4155608253 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f25c80a7f40 con 0x555d15ab2790 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.249+0000 7f25dee6a740 10 client.16107 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.249+0000 7f25dee6a740 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.249+0000 7f25dee6a740 20 client.16107 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.249+0000 7f25dee6a740 1 --2- 172.21.15.78:0/4155608253 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555d15ac27e0 0x555d15ac4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.249+0000 7f25dee6a740 4 client.16107 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.249+0000 7f25dee6a740 1 -- 172.21.15.78:0/4155608253 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d15a1edb0 con 0x555d15ac27e0 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.249+0000 7f25d659c640 1 --2- 172.21.15.78:0/4155608253 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555d15ac27e0 0x555d15ac4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.249+0000 7f25bdffb640 10 client.16107 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.249+0000 7f25d659c640 1 --2- 172.21.15.78:0/4155608253 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555d15ac27e0 0x555d15ac4c60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d704e41da82b484 server_cookie=d1c204faeaa54934 in_seq=0 out_seq=0 2024-09-11T01:02:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25bdffb640 1 -- 172.21.15.78:0/4155608253 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555d15a1edb0 con 0x555d15ac27e0 2024-09-11T01:02:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25bdffb640 10 client.16107 handle_command_reply: tid=0 2024-09-11T01:02:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25dee6a740 1 client.16107 shutdown 2024-09-11T01:02:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25dee6a740 1 -- 172.21.15.78:0/4155608253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15ab2790 msgr2=0x555d15ab0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25dee6a740 1 --2- 172.21.15.78:0/4155608253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15ab2790 0x555d15ab0240 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f25c80e2ef0 tx=0x7f25c8009f20 comp rx=0 tx=0).stop 2024-09-11T01:02:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25dee6a740 1 -- 172.21.15.78:0/4155608253 shutdown_connections 2024-09-11T01:02:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25dee6a740 1 --2- 172.21.15.78:0/4155608253 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555d15ac27e0 0x555d15ac4c60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25dee6a740 1 --2- 172.21.15.78:0/4155608253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15ab2790 0x555d15ab0240 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25dee6a740 1 --2- 172.21.15.78:0/4155608253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15aa6800 0x555d15aafd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25bdffb640 0 client.16107 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25dee6a740 1 --2- 172.21.15.78:0/4155608253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aaf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25dee6a740 1 -- 172.21.15.78:0/4155608253 >> 172.21.15.78:0/4155608253 conn(0x555d15aa3d10 msgr2=0x555d15aa2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25dee6a740 1 -- 172.21.15.78:0/4155608253 shutdown_connections 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25dee6a740 1 -- 172.21.15.78:0/4155608253 wait complete. 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.250+0000 7f25dee6a740 20 client.16107 trim_cache size 0 max 16384 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.269+0000 7f25dee6a740 1 Processor -- start 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.269+0000 7f25dee6a740 1 -- start start 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25dee6a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aa9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25dee6a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15aa6800 0x555d15aafea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25dee6a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15ab2790 0x555d15aac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25dee6a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81ae440 con 0x555d15ab2790 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25dee6a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81b0040 con 0x555d15aa6800 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25dee6a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81aeb10 con 0x555d15aa5ee0 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25d6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15ab2790 0x555d15aac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25d5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15aa6800 0x555d15aafea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25d5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15aa6800 0x555d15aafea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:37308/0 (socket says 172.21.15.78:37308) 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25d5d9b640 1 -- 172.21.15.78:0/2404578380 learned_addr learned my addr 172.21.15.78:0/2404578380 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25d659c640 1 --2- 172.21.15.78:0/2404578380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aa9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25d5d9b640 1 -- 172.21.15.78:0/2404578380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 msgr2=0x555d15aa9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25d5d9b640 1 --2- 172.21.15.78:0/2404578380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aa9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25d5d9b640 1 -- 172.21.15.78:0/2404578380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15ab2790 msgr2=0x555d15aac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25d5d9b640 1 --2- 172.21.15.78:0/2404578380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15ab2790 0x555d15aac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25d5d9b640 1 -- 172.21.15.78:0/2404578380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d15aa4790 con 0x555d15aa6800 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25d6d9d640 1 --2- 172.21.15.78:0/2404578380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15ab2790 0x555d15aac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.270+0000 7f25d5d9b640 1 --2- 172.21.15.78:0/2404578380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15aa6800 0x555d15aafea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f25c0012840 tx=0x7f25c0020f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8424eeec98f268f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.271+0000 7f25bffff640 1 -- 172.21.15.78:0/2404578380 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0026040 con 0x555d15aa6800 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.271+0000 7f25bffff640 1 -- 172.21.15.78:0/2404578380 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c0016050 con 0x555d15aa6800 2024-09-11T01:02:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.271+0000 7f25bffff640 1 -- 172.21.15.78:0/2404578380 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0004310 con 0x555d15aa6800 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.271+0000 7f25dee6a740 1 -- 172.21.15.78:0/2404578380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15aa6800 msgr2=0x555d15aafea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.271+0000 7f25dee6a740 1 --2- 172.21.15.78:0/2404578380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15aa6800 0x555d15aafea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f25c0012840 tx=0x7f25c0020f90 comp rx=0 tx=0).stop 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.271+0000 7f25dee6a740 1 -- 172.21.15.78:0/2404578380 shutdown_connections 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.271+0000 7f25dee6a740 1 --2- 172.21.15.78:0/2404578380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15ab2790 0x555d15aac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.271+0000 7f25dee6a740 1 --2- 172.21.15.78:0/2404578380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15aa6800 0x555d15aafea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.271+0000 7f25dee6a740 1 --2- 172.21.15.78:0/2404578380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15aa9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.271+0000 7f25dee6a740 1 -- 172.21.15.78:0/2404578380 >> 172.21.15.78:0/2404578380 conn(0x555d15aa3d10 msgr2=0x555d15aaf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.271+0000 7f25dee6a740 1 -- 172.21.15.78:0/2404578380 shutdown_connections 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.271+0000 7f25dee6a740 1 -- 172.21.15.78:0/2404578380 wait complete. 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25dee6a740 1 -- start start 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25dee6a740 1 Processor -- start 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25dee6a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15ab70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25dee6a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15aa6800 0x555d15ab7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25dee6a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15ab2790 0x555d15b287e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25dee6a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81af8b0 con 0x555d15aa6800 2024-09-11T01:02:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25dee6a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81af1e0 con 0x555d15ab2790 2024-09-11T01:02:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25dee6a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81ac420 con 0x555d15aa5ee0 2024-09-11T01:02:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25dee6a740 10 client.0 _ll_register_callbacks cb 0x7f25c80a8320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25d659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15ab70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25d6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15ab2790 0x555d15b287e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25d659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15ab70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42444/0 (socket says 172.21.15.78:42444) 2024-09-11T01:02:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25d659c640 1 -- 172.21.15.78:0/4189013780 learned_addr learned my addr 172.21.15.78:0/4189013780 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.272+0000 7f25d5d9b640 1 --2- 172.21.15.78:0/4189013780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15aa6800 0x555d15ab7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.273+0000 7f25d659c640 1 -- 172.21.15.78:0/4189013780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15ab2790 msgr2=0x555d15b287e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.273+0000 7f25d659c640 1 --2- 172.21.15.78:0/4189013780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15ab2790 0x555d15b287e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.273+0000 7f25d659c640 1 -- 172.21.15.78:0/4189013780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15aa6800 msgr2=0x555d15ab7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.273+0000 7f25d659c640 1 --2- 172.21.15.78:0/4189013780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15aa6800 0x555d15ab7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.273+0000 7f25d659c640 1 -- 172.21.15.78:0/4189013780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d15b28ef0 con 0x555d15aa5ee0 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.273+0000 7f25d659c640 1 --2- 172.21.15.78:0/4189013780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15ab70b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f25d002ade0 tx=0x7f25d00318c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc032da09947b3a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.273+0000 7f25bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.273+0000 7f25bdffb640 1 -- 172.21.15.78:0/4189013780 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0003b80 con 0x555d15aa5ee0 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.273+0000 7f25bdffb640 1 -- 172.21.15.78:0/4189013780 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f25d0003d20 con 0x555d15aa5ee0 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.273+0000 7f25dee6a740 1 -- 172.21.15.78:0/4189013780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c0016050 con 0x555d15aa5ee0 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.274+0000 7f25bdffb640 1 -- 172.21.15.78:0/4189013780 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f25d005c050 con 0x555d15aa5ee0 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.274+0000 7f25dee6a740 10 client.25504 fetch_fsmap learned FSMap version 31 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.274+0000 7f25dee6a740 1 -- 172.21.15.78:0/4189013780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555d15b30340 con 0x555d15aa5ee0 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.274+0000 7f25bdffb640 1 -- 172.21.15.78:0/4189013780 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d0053070 con 0x555d15aa5ee0 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.275+0000 7f25bdffb640 1 -- 172.21.15.78:0/4189013780 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f25d002f7b0 con 0x555d15aa5ee0 2024-09-11T01:02:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.275+0000 7f25dee6a740 10 client.25504 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:02:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.275+0000 7f25dee6a740 10 client.25504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.275+0000 7f25dee6a740 20 client.25504 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.275+0000 7f25dee6a740 1 --2- 172.21.15.78:0/4189013780 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555d15b30b80 0x555d15b32fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.275+0000 7f25dee6a740 4 client.25504 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:02:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.275+0000 7f25dee6a740 1 -- 172.21.15.78:0/4189013780 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f25c0016050 con 0x555d15b30b80 2024-09-11T01:02:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.275+0000 7f25d5d9b640 1 --2- 172.21.15.78:0/4189013780 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555d15b30b80 0x555d15b32fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.275+0000 7f25d5d9b640 1 --2- 172.21.15.78:0/4189013780 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555d15b30b80 0x555d15b32fa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e2335bfdab97c7 server_cookie=dada15a05c1ed2df in_seq=0 out_seq=0 2024-09-11T01:02:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.275+0000 7f25bdffb640 10 client.25504 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:05.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.277+0000 7f25bdffb640 1 -- 172.21.15.78:0/4189013780 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f25c0016050 con 0x555d15b30b80 2024-09-11T01:02:05.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.277+0000 7f25bdffb640 10 client.25504 handle_command_reply: tid=0 2024-09-11T01:02:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.277+0000 7f25dee6a740 1 client.25504 shutdown 2024-09-11T01:02:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.277+0000 7f25dee6a740 1 -- 172.21.15.78:0/4189013780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 msgr2=0x555d15ab70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.277+0000 7f25dee6a740 1 --2- 172.21.15.78:0/4189013780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15ab70b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f25d002ade0 tx=0x7f25d00318c0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.277+0000 7f25dee6a740 1 -- 172.21.15.78:0/4189013780 shutdown_connections 2024-09-11T01:02:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dee6a740 1 --2- 172.21.15.78:0/4189013780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d15ab2790 0x555d15b287e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dee6a740 1 --2- 172.21.15.78:0/4189013780 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555d15b30b80 0x555d15b32fa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dee6a740 1 --2- 172.21.15.78:0/4189013780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d15aa6800 0x555d15ab7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dee6a740 1 --2- 172.21.15.78:0/4189013780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d15aa5ee0 0x555d15ab70b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25bdffb640 0 client.25504 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dee6a740 1 -- 172.21.15.78:0/4189013780 >> 172.21.15.78:0/4189013780 conn(0x555d15aa3d10 msgr2=0x555d15aaffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dee6a740 1 -- 172.21.15.78:0/4189013780 shutdown_connections 2024-09-11T01:02:05.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dee6a740 1 -- 172.21.15.78:0/4189013780 wait complete. 2024-09-11T01:02:05.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dee6a740 20 client.25504 trim_cache size 0 max 16384 2024-09-11T01:02:05.298 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:02:05.298 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:02:05.298 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "092ff724-3d6b-4fec-a529-7ae22738a268", 2024-09-11T01:02:05.298 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:02:05.298 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:02:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dca95640 1 -- 172.21.15.78:0/1086852489 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f25c40b5450 msgr2=0x7f25c40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dca95640 1 --2- 172.21.15.78:0/1086852489 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f25c40b5450 0x7f25c40b7910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25c0023000 tx=0x7f25c0024000 comp rx=0 tx=0).stop 2024-09-11T01:02:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dca95640 1 -- 172.21.15.78:0/1086852489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25d8108020 msgr2=0x7f25d819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dca95640 1 --2- 172.21.15.78:0/1086852489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25d8108020 0x7f25d819ab50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f25c800d680 tx=0x7f25c800db40 comp rx=0 tx=0).stop 2024-09-11T01:02:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dca95640 1 -- 172.21.15.78:0/1086852489 shutdown_connections 2024-09-11T01:02:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dca95640 1 --2- 172.21.15.78:0/1086852489 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f25c40b5450 0x7f25c40b7910 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dca95640 1 --2- 172.21.15.78:0/1086852489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25d8108020 0x7f25d819ab50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dca95640 1 --2- 172.21.15.78:0/1086852489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d81076a0 0x7f25d819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dca95640 1 --2- 172.21.15.78:0/1086852489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8101c90 0x7f25d819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.278+0000 7f25dca95640 1 -- 172.21.15.78:0/1086852489 >> 172.21.15.78:0/1086852489 conn(0x7f25d80fd700 msgr2=0x7f25d80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.279+0000 7f25dca95640 1 -- 172.21.15.78:0/1086852489 shutdown_connections 2024-09-11T01:02:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:05.279+0000 7f25dca95640 1 -- 172.21.15.78:0/1086852489 wait complete. 2024-09-11T01:02:05.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 284071 11.34 MB/sec execute 13 sec latency 87.586 ms 2024-09-11T01:02:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:05 smithi110 ceph-mon[34712]: pgmap v419: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 379 op/s 2024-09-11T01:02:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:05 smithi112 ceph-mon[35384]: pgmap v419: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 379 op/s 2024-09-11T01:02:06.106 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:02:06.379 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:06.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 284071 10.53 MB/sec execute 14 sec latency 1087.654 ms 2024-09-11T01:02:06.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:02:06.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:06.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:06.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:06.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:06.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:06.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:06.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:06 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:02:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:02:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:06.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:06.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:06.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:06.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:06.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:06 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:02:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:02:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:06.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:06.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:06.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:06.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:06 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:02:06.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.864+0000 7fc47ad10640 1 -- 172.21.15.78:0/2054692982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4741045d0 msgr2=0x7fc47410cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:06.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.864+0000 7fc47ad10640 1 --2- 172.21.15.78:0/2054692982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4741045d0 0x7fc47410cac0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc460009900 tx=0x7fc46002a410 comp rx=0 tx=0).stop 2024-09-11T01:02:06.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.864+0000 7fc47ad10640 1 -- 172.21.15.78:0/2054692982 shutdown_connections 2024-09-11T01:02:06.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.864+0000 7fc47ad10640 1 --2- 172.21.15.78:0/2054692982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc47410d110 0x7fc47410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.864+0000 7fc47ad10640 1 --2- 172.21.15.78:0/2054692982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4741045d0 0x7fc47410cac0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.864+0000 7fc47ad10640 1 --2- 172.21.15.78:0/2054692982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc474103cb0 0x7fc474104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.864+0000 7fc47ad10640 1 -- 172.21.15.78:0/2054692982 >> 172.21.15.78:0/2054692982 conn(0x7fc4740fd710 msgr2=0x7fc4740ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.864+0000 7fc47ad10640 1 -- 172.21.15.78:0/2054692982 shutdown_connections 2024-09-11T01:02:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.864+0000 7fc47ad10640 1 -- 172.21.15.78:0/2054692982 wait complete. 2024-09-11T01:02:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc47ad10640 1 Processor -- start 2024-09-11T01:02:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc47ad10640 1 -- start start 2024-09-11T01:02:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc47ad10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc474103cb0 0x7fc474199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc47ad10640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4741045d0 0x7fc474199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc47ad10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47410d110 0x7fc47419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc47ad10640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc474112d80 con 0x7fc474103cb0 2024-09-11T01:02:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc47ad10640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc474112c00 con 0x7fc4741045d0 2024-09-11T01:02:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc47ad10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc474112f00 con 0x7fc47410d110 2024-09-11T01:02:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc478aac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc474103cb0 0x7fc474199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc478aac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc474103cb0 0x7fc474199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55882/0 (socket says 172.21.15.78:55882) 2024-09-11T01:02:06.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc478aac640 1 -- 172.21.15.78:0/878680819 learned_addr learned my addr 172.21.15.78:0/878680819 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:06.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc4792ad640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47410d110 0x7fc47419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc473fff640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4741045d0 0x7fc474199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc478aac640 1 -- 172.21.15.78:0/878680819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47410d110 msgr2=0x7fc47419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:06.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc478aac640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47410d110 0x7fc47419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc478aac640 1 -- 172.21.15.78:0/878680819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4741045d0 msgr2=0x7fc474199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:06.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc478aac640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4741045d0 0x7fc474199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc478aac640 1 -- 172.21.15.78:0/878680819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47419f400 con 0x7fc474103cb0 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.865+0000 7fc473fff640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4741045d0 0x7fc474199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.866+0000 7fc478aac640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc474103cb0 0x7fc474199350 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc46800d950 tx=0x7fc46800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.866+0000 7fc471ffb640 1 -- 172.21.15.78:0/878680819 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc468017070 con 0x7fc474103cb0 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.866+0000 7fc471ffb640 1 -- 172.21.15.78:0/878680819 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc468009c30 con 0x7fc474103cb0 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.866+0000 7fc47ad10640 1 -- 172.21.15.78:0/878680819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47419f6f0 con 0x7fc474103cb0 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.866+0000 7fc471ffb640 1 -- 172.21.15.78:0/878680819 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46800f620 con 0x7fc474103cb0 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.866+0000 7fc47ad10640 1 -- 172.21.15.78:0/878680819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741a5db0 con 0x7fc474103cb0 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.868+0000 7fc471ffb640 1 -- 172.21.15.78:0/878680819 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc468010460 con 0x7fc474103cb0 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc47d0e5740 1 Processor -- start 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc47d0e5740 1 -- start start 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc47d0e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548237480 0x557548237860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc47d0e5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x557548247c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc47d0e5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548248140 0x55754824a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc47d0e5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5575481ade50 con 0x557548237da0 2024-09-11T01:02:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc47d0e5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55754819d280 con 0x557548248140 2024-09-11T01:02:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc47d0e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5575481a1090 con 0x557548237480 2024-09-11T01:02:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc478aac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x557548247c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc478aac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x557548247c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55898/0 (socket says 172.21.15.78:55898) 2024-09-11T01:02:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc478aac640 1 -- 172.21.15.78:0/2951169664 learned_addr learned my addr 172.21.15.78:0/2951169664 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc4792ad640 1 --2- 172.21.15.78:0/2951169664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548237480 0x557548237860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc471ffb640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc4640b5450 0x7fc4640b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc473fff640 1 --2- 172.21.15.78:0/2951169664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548248140 0x55754824a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc4792ad640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc4640b5450 0x7fc4640b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc471ffb640 1 -- 172.21.15.78:0/878680819 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc4680e5c10 con 0x7fc474103cb0 2024-09-11T01:02:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc4792ad640 1 -- 172.21.15.78:0/2951169664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548248140 msgr2=0x55754824a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc4792ad640 1 --2- 172.21.15.78:0/2951169664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548248140 0x55754824a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc4792ad640 1 -- 172.21.15.78:0/2951169664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 msgr2=0x557548247c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc4792ad640 1 --2- 172.21.15.78:0/2951169664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x557548247c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.869+0000 7fc4792ad640 1 -- 172.21.15.78:0/2951169664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557547efb650 con 0x557548237480 2024-09-11T01:02:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.870+0000 7fc478aac640 1 --2- 172.21.15.78:0/2951169664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x557548247c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.870+0000 7fc4792ad640 1 --2- 172.21.15.78:0/2951169664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548237480 0x557548237860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0149a0 tx=0x7fc46c014e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=109b68e8d39be10a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.870+0000 7fc4792ad640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc4640b5450 0x7fc4640b7910 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc46c01c000 tx=0x7fc46c01e000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.870+0000 7fc45effd640 1 -- 172.21.15.78:0/2951169664 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c023070 con 0x557548237480 2024-09-11T01:02:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.870+0000 7fc45effd640 1 -- 172.21.15.78:0/2951169664 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc46c014030 con 0x557548237480 2024-09-11T01:02:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.870+0000 7fc45effd640 1 -- 172.21.15.78:0/2951169664 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0138b0 con 0x557548237480 2024-09-11T01:02:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.871+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2951169664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548237480 msgr2=0x557548237860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.871+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2951169664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548237480 0x557548237860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0149a0 tx=0x7fc46c014e60 comp rx=0 tx=0).stop 2024-09-11T01:02:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.871+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2951169664 shutdown_connections 2024-09-11T01:02:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.871+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2951169664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548248140 0x55754824a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.871+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2951169664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x557548247c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.871+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2951169664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548237480 0x557548237860 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.871+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2951169664 >> 172.21.15.78:0/2951169664 conn(0x5575482352b0 msgr2=0x557548236450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:06.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.871+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2951169664 shutdown_connections 2024-09-11T01:02:06.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.871+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2951169664 wait complete. 2024-09-11T01:02:06.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.871+0000 7fc47d0e5740 1 -- start start 2024-09-11T01:02:06.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.871+0000 7fc47d0e5740 1 Processor -- start 2024-09-11T01:02:06.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc47d0e5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237480 0x557548247970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc47d0e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548237da0 0x557548244dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc47d0e5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548248140 0x557548245300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc47d0e5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557547ffe700 con 0x557548237480 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc47d0e5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557548138110 con 0x557548248140 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc47d0e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55754824adf0 con 0x557548237da0 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc47d0e5740 10 client.0 _ll_register_callbacks cb 0x557548151990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc478aac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237480 0x557548247970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc478aac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237480 0x557548247970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55910/0 (socket says 172.21.15.78:55910) 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc478aac640 1 -- 172.21.15.78:0/1089352767 learned_addr learned my addr 172.21.15.78:0/1089352767 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc473fff640 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548237da0 0x557548244dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc478aac640 1 -- 172.21.15.78:0/1089352767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548237da0 msgr2=0x557548244dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc478aac640 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548237da0 0x557548244dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc478aac640 1 -- 172.21.15.78:0/1089352767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548248140 msgr2=0x557548245300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc478aac640 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548248140 0x557548245300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc478aac640 1 -- 172.21.15.78:0/1089352767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557548165c40 con 0x557548237480 2024-09-11T01:02:06.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc473fff640 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548237da0 0x557548244dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc478aac640 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237480 0x557548247970 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc468000c00 tx=0x7fc4680b5120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b9b4c7ef345797 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc453fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc453fff640 1 -- 172.21.15.78:0/1089352767 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc468020030 con 0x557548237480 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc453fff640 1 -- 172.21.15.78:0/1089352767 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc4680ab560 con 0x557548237480 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.872+0000 7fc47d0e5740 1 -- 172.21.15.78:0/1089352767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc46c014030 con 0x557548237480 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.873+0000 7fc453fff640 1 -- 172.21.15.78:0/1089352767 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc4680abc10 con 0x557548237480 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.873+0000 7fc47d0e5740 10 client.16125 fetch_fsmap learned FSMap version 31 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.873+0000 7fc47d0e5740 1 -- 172.21.15.78:0/1089352767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5575482566c0 con 0x557548237480 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.873+0000 7fc453fff640 1 -- 172.21.15.78:0/1089352767 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc46c014030 con 0x557548237480 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.873+0000 7fc453fff640 1 -- 172.21.15.78:0/1089352767 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fc468010780 con 0x557548237480 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.873+0000 7fc47d0e5740 10 client.16125 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.873+0000 7fc47d0e5740 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.873+0000 7fc47d0e5740 20 client.16125 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.873+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557548257c70 0x55754825a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.873+0000 7fc47d0e5740 4 client.16125 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.873+0000 7fc47d0e5740 1 -- 172.21.15.78:0/1089352767 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575481afda0 con 0x557548257c70 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.873+0000 7fc473fff640 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557548257c70 0x55754825a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.874+0000 7fc453fff640 10 client.16125 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.874+0000 7fc473fff640 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557548257c70 0x55754825a100 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c0d3ba6a15ed04 server_cookie=577ea9479b3de583 in_seq=0 out_seq=0 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.874+0000 7fc453fff640 1 -- 172.21.15.78:0/1089352767 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5575481afda0 con 0x557548257c70 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.874+0000 7fc453fff640 10 client.16125 handle_command_reply: tid=0 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc47d0e5740 1 client.16125 shutdown 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc47d0e5740 1 -- 172.21.15.78:0/1089352767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237480 msgr2=0x557548247970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237480 0x557548247970 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc468000c00 tx=0x7fc4680b5120 comp rx=0 tx=0).stop 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc47d0e5740 1 -- 172.21.15.78:0/1089352767 shutdown_connections 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548248140 0x557548245300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548237da0 0x557548244dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557548257c70 0x55754825a100 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/1089352767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237480 0x557548247970 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc47d0e5740 1 -- 172.21.15.78:0/1089352767 >> 172.21.15.78:0/1089352767 conn(0x5575482352b0 msgr2=0x55754823ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc453fff640 0 client.16125 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc47d0e5740 1 -- 172.21.15.78:0/1089352767 shutdown_connections 2024-09-11T01:02:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc47d0e5740 1 -- 172.21.15.78:0/1089352767 wait complete. 2024-09-11T01:02:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.875+0000 7fc47d0e5740 20 client.16125 trim_cache size 0 max 16384 2024-09-11T01:02:06.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.892+0000 7fc47d0e5740 1 Processor -- start 2024-09-11T01:02:06.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.892+0000 7fc47d0e5740 1 -- start start 2024-09-11T01:02:06.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.892+0000 7fc47d0e5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548237480 0x55754825f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.892+0000 7fc47d0e5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x55754825ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.892+0000 7fc47d0e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548248140 0x55754823c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.892+0000 7fc47d0e5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741ac490 con 0x557548237da0 2024-09-11T01:02:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.892+0000 7fc47d0e5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741ae090 con 0x557548237480 2024-09-11T01:02:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.892+0000 7fc47d0e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741acb60 con 0x557548248140 2024-09-11T01:02:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc473fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x55754825ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc473fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x55754825ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55920/0 (socket says 172.21.15.78:55920) 2024-09-11T01:02:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc473fff640 1 -- 172.21.15.78:0/2012262492 learned_addr learned my addr 172.21.15.78:0/2012262492 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc478aac640 1 --2- 172.21.15.78:0/2012262492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548237480 0x55754825f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc4792ad640 1 --2- 172.21.15.78:0/2012262492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548248140 0x55754823c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc473fff640 1 -- 172.21.15.78:0/2012262492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548248140 msgr2=0x55754823c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc473fff640 1 --2- 172.21.15.78:0/2012262492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548248140 0x55754823c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc473fff640 1 -- 172.21.15.78:0/2012262492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548237480 msgr2=0x55754825f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc473fff640 1 --2- 172.21.15.78:0/2012262492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548237480 0x55754825f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc473fff640 1 -- 172.21.15.78:0/2012262492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557548236150 con 0x557548237da0 2024-09-11T01:02:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc478aac640 1 --2- 172.21.15.78:0/2012262492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548237480 0x55754825f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc473fff640 1 --2- 172.21.15.78:0/2012262492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x55754825ff10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc460031440 tx=0x7fc4600042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364e145fa6ffbec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc45e7fc640 1 -- 172.21.15.78:0/2012262492 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460049070 con 0x557548237da0 2024-09-11T01:02:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc45e7fc640 1 -- 172.21.15.78:0/2012262492 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4600043a0 con 0x557548237da0 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.893+0000 7fc45e7fc640 1 -- 172.21.15.78:0/2012262492 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460035040 con 0x557548237da0 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.894+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2012262492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 msgr2=0x55754825ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.894+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2012262492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x55754825ff10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc460031440 tx=0x7fc4600042b0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.894+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2012262492 shutdown_connections 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.894+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2012262492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548248140 0x55754823c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.894+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2012262492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x55754825ff10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.894+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2012262492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548237480 0x55754825f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.894+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2012262492 >> 172.21.15.78:0/2012262492 conn(0x5575482352b0 msgr2=0x557548249e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.894+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2012262492 shutdown_connections 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.894+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2012262492 wait complete. 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.894+0000 7fc47d0e5740 1 -- start start 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.894+0000 7fc47d0e5740 1 Processor -- start 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc47d0e5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548237480 0x557548250a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc47d0e5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x55754825f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc47d0e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548248140 0x55754825fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc47d0e5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741ad900 con 0x557548237da0 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc47d0e5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741ad230 con 0x557548237480 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc47d0e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741aa470 con 0x557548248140 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc47d0e5740 10 client.0 _ll_register_callbacks cb 0x7fc4680b5050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc473fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x55754825f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc473fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x55754825f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55926/0 (socket says 172.21.15.78:55926) 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc473fff640 1 -- 172.21.15.78:0/2785247490 learned_addr learned my addr 172.21.15.78:0/2785247490 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc478aac640 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548237480 0x557548250a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc4792ad640 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548248140 0x55754825fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc473fff640 1 -- 172.21.15.78:0/2785247490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548248140 msgr2=0x55754825fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc473fff640 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548248140 0x55754825fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc473fff640 1 -- 172.21.15.78:0/2785247490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548237480 msgr2=0x557548250a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc473fff640 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548237480 0x557548250a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc473fff640 1 -- 172.21.15.78:0/2785247490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575482c3bd0 con 0x557548237da0 2024-09-11T01:02:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc473fff640 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x55754825f7c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc460000c00 tx=0x7fc4600095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8482b16d31fe15 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.895+0000 7fc453fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc453fff640 1 -- 172.21.15.78:0/2785247490 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460035030 con 0x557548237da0 2024-09-11T01:02:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc453fff640 1 -- 172.21.15.78:0/2785247490 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc460003800 con 0x557548237da0 2024-09-11T01:02:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2785247490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4600043a0 con 0x557548237da0 2024-09-11T01:02:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc453fff640 1 -- 172.21.15.78:0/2785247490 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc460049070 con 0x557548237da0 2024-09-11T01:02:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc47d0e5740 10 client.16131 fetch_fsmap learned FSMap version 31 2024-09-11T01:02:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2785247490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5575482c4de0 con 0x557548237da0 2024-09-11T01:02:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc453fff640 1 -- 172.21.15.78:0/2785247490 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4600043a0 con 0x557548237da0 2024-09-11T01:02:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc453fff640 1 -- 172.21.15.78:0/2785247490 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fc460032660 con 0x557548237da0 2024-09-11T01:02:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc47d0e5740 10 client.16131 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T01:02:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc47d0e5740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc47d0e5740 20 client.16131 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5575482c5550 0x5575482c7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc47d0e5740 4 client.16131 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:02:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2785247490 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5575481ab850 con 0x5575482c5550 2024-09-11T01:02:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.896+0000 7fc478aac640 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5575482c5550 0x5575482c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc478aac640 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5575482c5550 0x5575482c7970 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=462a2c72dcf0bf2a server_cookie=150447ae43eb5b27 in_seq=0 out_seq=0 2024-09-11T01:02:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc453fff640 10 client.16131 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc453fff640 1 -- 172.21.15.78:0/2785247490 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1712 (crc 0 0 0) 0x5575481ab850 con 0x5575482c5550 2024-09-11T01:02:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc453fff640 10 client.16131 handle_command_reply: tid=0 2024-09-11T01:02:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc47d0e5740 1 client.16131 shutdown 2024-09-11T01:02:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2785247490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 msgr2=0x55754825f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x55754825f7c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc460000c00 tx=0x7fc4600095e0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2785247490 shutdown_connections 2024-09-11T01:02:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557548248140 0x55754825fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5575482c5550 0x5575482c7970 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557548237da0 0x55754825f7c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc47d0e5740 1 --2- 172.21.15.78:0/2785247490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557548237480 0x557548250a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc453fff640 0 client.16131 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.897+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2785247490 >> 172.21.15.78:0/2785247490 conn(0x5575482352b0 msgr2=0x5575482463f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2785247490 shutdown_connections 2024-09-11T01:02:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47d0e5740 1 -- 172.21.15.78:0/2785247490 wait complete. 2024-09-11T01:02:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47d0e5740 20 client.16131 trim_cache size 0 max 16384 2024-09-11T01:02:06.909 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:02:06.909 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (0 inodes in the stack)", 2024-09-11T01:02:06.909 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:02:06.909 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:02:06.909 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:06.909 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "092ff724-3d6b-4fec-a529-7ae22738a268", 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:02:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:06.911 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:06.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:06.912 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:02:06.912 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:02:06.912 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:06.912 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:06.912 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:06.912 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:06.912 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:06.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:06.913 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:02:06.913 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:02:06.913 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:06.913 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:06.913 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:06.913 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:06.913 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:06.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:02:06.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:02:06.913 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:02:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47ad10640 1 -- 172.21.15.78:0/878680819 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc4640b5450 msgr2=0x7fc4640b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47ad10640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc4640b5450 0x7fc4640b7910 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc46c01c000 tx=0x7fc46c01e000 comp rx=0 tx=0).stop 2024-09-11T01:02:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47ad10640 1 -- 172.21.15.78:0/878680819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc474103cb0 msgr2=0x7fc474199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47ad10640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc474103cb0 0x7fc474199350 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc46800d950 tx=0x7fc46800de10 comp rx=0 tx=0).stop 2024-09-11T01:02:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47ad10640 1 -- 172.21.15.78:0/878680819 shutdown_connections 2024-09-11T01:02:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47ad10640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc4640b5450 0x7fc4640b7910 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47ad10640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47410d110 0x7fc47419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47ad10640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4741045d0 0x7fc474199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47ad10640 1 --2- 172.21.15.78:0/878680819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc474103cb0 0x7fc474199350 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:06.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.898+0000 7fc47ad10640 1 -- 172.21.15.78:0/878680819 >> 172.21.15.78:0/878680819 conn(0x7fc4740fd710 msgr2=0x7fc4740fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:06.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.899+0000 7fc47ad10640 1 -- 172.21.15.78:0/878680819 shutdown_connections 2024-09-11T01:02:06.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:06.899+0000 7fc47ad10640 1 -- 172.21.15.78:0/878680819 wait complete. 2024-09-11T01:02:07.422 INFO:tasks.workunit.client.0.smithi078.stdout: 1 284249 9.85 MB/sec execute 15 sec latency 1850.079 ms 2024-09-11T01:02:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:07 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:07 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:07 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:07 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:07 smithi110 ceph-mon[34712]: pgmap v420: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 372 op/s 2024-09-11T01:02:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:02:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:02:07.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:02:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:07 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:07 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:07 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:07 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:07 smithi112 ceph-mon[35384]: pgmap v420: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 372 op/s 2024-09-11T01:02:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:02:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:02:07.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:02:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:07 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:07 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:07 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:07 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:07 smithi078 ceph-mon[31315]: pgmap v420: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 372 op/s 2024-09-11T01:02:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:02:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:02:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:02:07.762 INFO:tasks.cephfs.filesystem:scrub status for tag:092ff724-3d6b-4fec-a529-7ae22738a268 - {'path': '/', 'tag': '092ff724-3d6b-4fec-a529-7ae22738a268', 'options': 'recursive,force'} 2024-09-11T01:02:08.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 286958 10.05 MB/sec execute 16 sec latency 10.825 ms 2024-09-11T01:02:08.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:08 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:08.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:08 smithi110 ceph-mon[34712]: pgmap v421: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 331 op/s 2024-09-11T01:02:08.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:08 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:08 smithi112 ceph-mon[35384]: pgmap v421: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 331 op/s 2024-09-11T01:02:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:08 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:08 smithi078 ceph-mon[31315]: pgmap v421: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 331 op/s 2024-09-11T01:02:09.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 288571 9.94 MB/sec execute 17 sec latency 12.180 ms 2024-09-11T01:02:10.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 291238 10.33 MB/sec execute 18 sec latency 11.960 ms 2024-09-11T01:02:11.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 293343 10.19 MB/sec execute 19 sec latency 48.532 ms 2024-09-11T01:02:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:11 smithi110 ceph-mon[34712]: pgmap v422: 97 pgs: 97 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 403 op/s 2024-09-11T01:02:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:11 smithi112 ceph-mon[35384]: pgmap v422: 97 pgs: 97 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 403 op/s 2024-09-11T01:02:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:11 smithi078 ceph-mon[31315]: pgmap v422: 97 pgs: 97 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 403 op/s 2024-09-11T01:02:12.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 295236 10.28 MB/sec execute 20 sec latency 13.735 ms 2024-09-11T01:02:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:12 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:02:12.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:12 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:02:12.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:12 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:02:13.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 298114 10.61 MB/sec execute 21 sec latency 11.605 ms 2024-09-11T01:02:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:13 smithi110 ceph-mon[34712]: pgmap v423: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 323 op/s 2024-09-11T01:02:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:13 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:02:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:13 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:02:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:13 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:02:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:13 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:02:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:13 smithi112 ceph-mon[35384]: pgmap v423: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 323 op/s 2024-09-11T01:02:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:13 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:02:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:13 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:02:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:13 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:02:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:13 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:02:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:13 smithi078 ceph-mon[31315]: pgmap v423: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 323 op/s 2024-09-11T01:02:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:13 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:02:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:13 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:02:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:13 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:02:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:13 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:02:14.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 300321 10.51 MB/sec execute 22 sec latency 12.777 ms 2024-09-11T01:02:15.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 302090 10.58 MB/sec execute 23 sec latency 9.308 ms 2024-09-11T01:02:15.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:15 smithi110 ceph-mon[34712]: pgmap v424: 97 pgs: 97 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 373 op/s 2024-09-11T01:02:15.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:15 smithi112 ceph-mon[35384]: pgmap v424: 97 pgs: 97 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 373 op/s 2024-09-11T01:02:15.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:15 smithi078 ceph-mon[31315]: pgmap v424: 97 pgs: 97 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 373 op/s 2024-09-11T01:02:16.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 304512 10.66 MB/sec execute 24 sec latency 40.737 ms 2024-09-11T01:02:16.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:16 smithi110 ceph-mon[34712]: pgmap v425: 97 pgs: 97 active+clean; 170 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-09-11T01:02:16.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:16 smithi112 ceph-mon[35384]: pgmap v425: 97 pgs: 97 active+clean; 170 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-09-11T01:02:16.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:16 smithi078 ceph-mon[31315]: pgmap v425: 97 pgs: 97 active+clean; 170 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-09-11T01:02:17.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 306138 10.57 MB/sec execute 25 sec latency 13.340 ms 2024-09-11T01:02:18.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 308890 10.82 MB/sec execute 26 sec latency 11.149 ms 2024-09-11T01:02:19.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 311184 10.72 MB/sec execute 27 sec latency 12.017 ms 2024-09-11T01:02:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:19 smithi110 ceph-mon[34712]: pgmap v426: 97 pgs: 97 active+clean; 171 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 278 op/s 2024-09-11T01:02:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:19 smithi112 ceph-mon[35384]: pgmap v426: 97 pgs: 97 active+clean; 171 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 278 op/s 2024-09-11T01:02:19.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:19 smithi078 ceph-mon[31315]: pgmap v426: 97 pgs: 97 active+clean; 171 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 278 op/s 2024-09-11T01:02:20.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 312946 10.75 MB/sec execute 28 sec latency 11.786 ms 2024-09-11T01:02:21.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 315643 10.97 MB/sec execute 29 sec latency 37.496 ms 2024-09-11T01:02:21.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:21 smithi110 ceph-mon[34712]: pgmap v427: 97 pgs: 97 active+clean; 174 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 340 op/s 2024-09-11T01:02:21.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:21 smithi112 ceph-mon[35384]: pgmap v427: 97 pgs: 97 active+clean; 174 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 340 op/s 2024-09-11T01:02:21.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:21 smithi078 ceph-mon[31315]: pgmap v427: 97 pgs: 97 active+clean; 174 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 340 op/s 2024-09-11T01:02:22.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 317344 10.83 MB/sec execute 30 sec latency 10.587 ms 2024-09-11T01:02:23.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 319665 10.93 MB/sec execute 31 sec latency 11.662 ms 2024-09-11T01:02:23.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:23 smithi110 ceph-mon[34712]: pgmap v428: 97 pgs: 97 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 371 op/s 2024-09-11T01:02:23.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:23 smithi112 ceph-mon[35384]: pgmap v428: 97 pgs: 97 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 371 op/s 2024-09-11T01:02:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:23 smithi078 ceph-mon[31315]: pgmap v428: 97 pgs: 97 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 371 op/s 2024-09-11T01:02:24.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 322166 10.98 MB/sec execute 32 sec latency 14.606 ms 2024-09-11T01:02:25.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 323863 10.91 MB/sec execute 33 sec latency 10.842 ms 2024-09-11T01:02:25.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:25 smithi110 ceph-mon[34712]: pgmap v429: 97 pgs: 97 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 355 op/s 2024-09-11T01:02:25.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:25 smithi112 ceph-mon[35384]: pgmap v429: 97 pgs: 97 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 355 op/s 2024-09-11T01:02:25.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:25 smithi078 ceph-mon[31315]: pgmap v429: 97 pgs: 97 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 355 op/s 2024-09-11T01:02:26.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 326476 11.05 MB/sec execute 34 sec latency 28.328 ms 2024-09-11T01:02:27.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 328199 10.93 MB/sec execute 35 sec latency 9.200 ms 2024-09-11T01:02:27.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:27 smithi110 ceph-mon[34712]: pgmap v430: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-09-11T01:02:27.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:27 smithi112 ceph-mon[35384]: pgmap v430: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-09-11T01:02:27.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:27 smithi078 ceph-mon[31315]: pgmap v430: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-09-11T01:02:28.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 330324 11.01 MB/sec execute 36 sec latency 13.402 ms 2024-09-11T01:02:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:28 smithi110 ceph-mon[34712]: pgmap v431: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 340 op/s 2024-09-11T01:02:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:28 smithi112 ceph-mon[35384]: pgmap v431: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 340 op/s 2024-09-11T01:02:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:28 smithi078 ceph-mon[31315]: pgmap v431: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 340 op/s 2024-09-11T01:02:29.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 332895 11.05 MB/sec execute 37 sec latency 9.784 ms 2024-09-11T01:02:29.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:29 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:02:29.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:29 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:02:29.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:29 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:02:30.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 333437 10.89 MB/sec execute 38 sec latency 828.696 ms 2024-09-11T01:02:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:30 smithi110 ceph-mon[34712]: pgmap v432: 97 pgs: 97 active+clean; 187 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 375 op/s 2024-09-11T01:02:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:30 smithi112 ceph-mon[35384]: pgmap v432: 97 pgs: 97 active+clean; 187 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 375 op/s 2024-09-11T01:02:30.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:30 smithi078 ceph-mon[31315]: pgmap v432: 97 pgs: 97 active+clean; 187 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 375 op/s 2024-09-11T01:02:31.423 INFO:tasks.workunit.client.0.smithi078.stdout: 1 333437 10.61 MB/sec execute 39 sec latency 1828.741 ms 2024-09-11T01:02:32.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 333437 10.34 MB/sec execute 40 sec latency 2828.824 ms 2024-09-11T01:02:33.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 333437 10.09 MB/sec execute 41 sec latency 3828.931 ms 2024-09-11T01:02:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:33 smithi110 ceph-mon[34712]: pgmap v433: 97 pgs: 97 active+clean; 189 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 370 op/s 2024-09-11T01:02:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:33 smithi112 ceph-mon[35384]: pgmap v433: 97 pgs: 97 active+clean; 189 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 370 op/s 2024-09-11T01:02:33.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:33 smithi078 ceph-mon[31315]: pgmap v433: 97 pgs: 97 active+clean; 189 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 370 op/s 2024-09-11T01:02:34.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 333492 9.85 MB/sec execute 42 sec latency 4817.865 ms 2024-09-11T01:02:34.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:34 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:02:34.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:34 smithi110 ceph-mon[34712]: pgmap v434: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 289 op/s 2024-09-11T01:02:34.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:34 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:02:34.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:34 smithi112 ceph-mon[35384]: pgmap v434: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 289 op/s 2024-09-11T01:02:34.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:34 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:02:34.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:34 smithi078 ceph-mon[31315]: pgmap v434: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 289 op/s 2024-09-11T01:02:35.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 333502 9.63 MB/sec execute 43 sec latency 994.836 ms 2024-09-11T01:02:36.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 333502 9.41 MB/sec execute 44 sec latency 1994.894 ms 2024-09-11T01:02:37.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 333502 9.20 MB/sec execute 45 sec latency 2994.975 ms 2024-09-11T01:02:37.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:37 smithi110 ceph-mon[34712]: pgmap v435: 97 pgs: 97 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 283 op/s 2024-09-11T01:02:37.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:37 smithi112 ceph-mon[35384]: pgmap v435: 97 pgs: 97 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 283 op/s 2024-09-11T01:02:37.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:37 smithi078 ceph-mon[31315]: pgmap v435: 97 pgs: 97 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 283 op/s 2024-09-11T01:02:37.763 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:02:38.037 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:38.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 333502 9.00 MB/sec execute 46 sec latency 3995.055 ms 2024-09-11T01:02:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.540+0000 7fea3ac80640 1 -- 172.21.15.78:0/4056422017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea34109290 msgr2=0x7fea3410f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.540+0000 7fea3ac80640 1 --2- 172.21.15.78:0/4056422017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea34109290 0x7fea3410f990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009cd0 tx=0x7fea2c02a8e0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.540+0000 7fea3ac80640 1 -- 172.21.15.78:0/4056422017 shutdown_connections 2024-09-11T01:02:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.540+0000 7fea3ac80640 1 --2- 172.21.15.78:0/4056422017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea34109290 0x7fea3410f990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.540+0000 7fea3ac80640 1 --2- 172.21.15.78:0/4056422017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea34108910 0x7fea34108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.540+0000 7fea3ac80640 1 --2- 172.21.15.78:0/4056422017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea34101810 0x7fea34101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.540+0000 7fea3ac80640 1 -- 172.21.15.78:0/4056422017 >> 172.21.15.78:0/4056422017 conn(0x7fea34076f20 msgr2=0x7fea34077330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.540+0000 7fea3ac80640 1 -- 172.21.15.78:0/4056422017 shutdown_connections 2024-09-11T01:02:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.540+0000 7fea3ac80640 1 -- 172.21.15.78:0/4056422017 wait complete. 2024-09-11T01:02:38.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.541+0000 7fea3ac80640 1 Processor -- start 2024-09-11T01:02:38.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.541+0000 7fea3ac80640 1 -- start start 2024-09-11T01:02:38.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.541+0000 7fea3ac80640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea34101810 0x7fea34199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.541+0000 7fea3ac80640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea34108910 0x7fea34199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.541+0000 7fea3ac80640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea34109290 0x7fea3419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.541+0000 7fea3ac80640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea34112dd0 con 0x7fea34101810 2024-09-11T01:02:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.541+0000 7fea3ac80640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea34112c50 con 0x7fea34108910 2024-09-11T01:02:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.541+0000 7fea3ac80640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea34112f50 con 0x7fea34109290 2024-09-11T01:02:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.541+0000 7fea38a1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea34101810 0x7fea34199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.541+0000 7fea38a1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea34101810 0x7fea34199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42888/0 (socket says 172.21.15.78:42888) 2024-09-11T01:02:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.541+0000 7fea38a1c640 1 -- 172.21.15.78:0/1675534861 learned_addr learned my addr 172.21.15.78:0/1675534861 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.541+0000 7fea33fff640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea34108910 0x7fea34199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.542+0000 7fea3921d640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea34109290 0x7fea3419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.542+0000 7fea38a1c640 1 -- 172.21.15.78:0/1675534861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea34109290 msgr2=0x7fea3419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:38.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.542+0000 7fea38a1c640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea34109290 0x7fea3419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.542+0000 7fea38a1c640 1 -- 172.21.15.78:0/1675534861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea34108910 msgr2=0x7fea34199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:38.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.542+0000 7fea38a1c640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea34108910 0x7fea34199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.542+0000 7fea38a1c640 1 -- 172.21.15.78:0/1675534861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3419f3f0 con 0x7fea34101810 2024-09-11T01:02:38.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.542+0000 7fea38a1c640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea34101810 0x7fea34199340 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fea2800d950 tx=0x7fea2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:38.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.542+0000 7fea31ffb640 1 -- 172.21.15.78:0/1675534861 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28017070 con 0x7fea34101810 2024-09-11T01:02:38.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.542+0000 7fea3ac80640 1 -- 172.21.15.78:0/1675534861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3419f6e0 con 0x7fea34101810 2024-09-11T01:02:38.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.542+0000 7fea31ffb640 1 -- 172.21.15.78:0/1675534861 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea28009c30 con 0x7fea34101810 2024-09-11T01:02:38.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.542+0000 7fea31ffb640 1 -- 172.21.15.78:0/1675534861 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2800f620 con 0x7fea34101810 2024-09-11T01:02:38.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.542+0000 7fea3ac80640 1 -- 172.21.15.78:0/1675534861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea341a5f70 con 0x7fea34101810 2024-09-11T01:02:38.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.545+0000 7fea3d055740 1 Processor -- start 2024-09-11T01:02:38.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.545+0000 7fea3d055740 1 -- start start 2024-09-11T01:02:38.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.545+0000 7fea31ffb640 1 -- 172.21.15.78:0/1675534861 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fea28010460 con 0x7fea34101810 2024-09-11T01:02:38.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.545+0000 7fea3d055740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3480 0x55cede9a3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.545+0000 7fea3d055740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a3da0 0x55cede9af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.545+0000 7fea3d055740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 0x55cede9b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.545+0000 7fea3d055740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cede919e50 con 0x55cede9afd30 2024-09-11T01:02:38.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.545+0000 7fea3d055740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cede909280 con 0x55cede9a3da0 2024-09-11T01:02:38.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.545+0000 7fea3d055740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cede90d090 con 0x55cede9a3480 2024-09-11T01:02:38.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea33fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 0x55cede9b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea33fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 0x55cede9b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42896/0 (socket says 172.21.15.78:42896) 2024-09-11T01:02:38.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea33fff640 1 -- 172.21.15.78:0/1443721932 learned_addr learned my addr 172.21.15.78:0/1443721932 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:38.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea3921d640 1 --2- 172.21.15.78:0/1443721932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3480 0x55cede9a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea38a1c640 1 --2- 172.21.15.78:0/1443721932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a3da0 0x55cede9af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea33fff640 1 -- 172.21.15.78:0/1443721932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3480 msgr2=0x55cede9a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:38.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea33fff640 1 --2- 172.21.15.78:0/1443721932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3480 0x55cede9a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea33fff640 1 -- 172.21.15.78:0/1443721932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a3da0 msgr2=0x55cede9af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:38.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea33fff640 1 --2- 172.21.15.78:0/1443721932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a3da0 0x55cede9af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea33fff640 1 -- 172.21.15.78:0/1443721932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cede667650 con 0x55cede9afd30 2024-09-11T01:02:38.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea3921d640 1 --2- 172.21.15.78:0/1443721932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3480 0x55cede9a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:38.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea33fff640 1 --2- 172.21.15.78:0/1443721932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 0x55cede9b2120 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fea20009c20 tx=0x7fea20011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31f74553fa1d415 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:38.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea1effd640 1 -- 172.21.15.78:0/1443721932 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20004230 con 0x55cede9afd30 2024-09-11T01:02:38.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea1effd640 1 -- 172.21.15.78:0/1443721932 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea200043d0 con 0x55cede9afd30 2024-09-11T01:02:38.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea1effd640 1 -- 172.21.15.78:0/1443721932 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20004c80 con 0x55cede9afd30 2024-09-11T01:02:38.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.546+0000 7fea31ffb640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fea240b5450 0x7fea240b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3921d640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fea240b5450 0x7fea240b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea31ffb640 1 -- 172.21.15.78:0/1675534861 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fea280e5c10 con 0x7fea34101810 2024-09-11T01:02:38.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3d055740 1 -- 172.21.15.78:0/1443721932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 msgr2=0x55cede9b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:38.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3d055740 1 --2- 172.21.15.78:0/1443721932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 0x55cede9b2120 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fea20009c20 tx=0x7fea20011810 comp rx=0 tx=0).stop 2024-09-11T01:02:38.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3d055740 1 -- 172.21.15.78:0/1443721932 shutdown_connections 2024-09-11T01:02:38.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3d055740 1 --2- 172.21.15.78:0/1443721932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 0x55cede9b2120 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3d055740 1 --2- 172.21.15.78:0/1443721932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a3da0 0x55cede9af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3d055740 1 --2- 172.21.15.78:0/1443721932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3480 0x55cede9a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3d055740 1 -- 172.21.15.78:0/1443721932 >> 172.21.15.78:0/1443721932 conn(0x55cede9a12b0 msgr2=0x55cede9a2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:38.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3921d640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fea240b5450 0x7fea240b7910 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fea2c006010 tx=0x7fea2c0093f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:38.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3d055740 1 -- 172.21.15.78:0/1443721932 shutdown_connections 2024-09-11T01:02:38.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3d055740 1 -- 172.21.15.78:0/1443721932 wait complete. 2024-09-11T01:02:38.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3d055740 1 -- start start 2024-09-11T01:02:38.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.547+0000 7fea3d055740 1 Processor -- start 2024-09-11T01:02:38.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea3d055740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a3480 0x55cede9af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea3d055740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3da0 0x55cede9aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea3d055740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9afd30 0x55cede9acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea3d055740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cede76a700 con 0x55cede9a3480 2024-09-11T01:02:38.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea3d055740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cede8a4110 con 0x55cede9afd30 2024-09-11T01:02:38.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea3d055740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cede9b29b0 con 0x55cede9a3da0 2024-09-11T01:02:38.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea38a1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a3480 0x55cede9af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea3d055740 10 client.0 _ll_register_callbacks cb 0x55cede8bd990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:38.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea38a1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a3480 0x55cede9af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42902/0 (socket says 172.21.15.78:42902) 2024-09-11T01:02:38.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea38a1c640 1 -- 172.21.15.78:0/595362591 learned_addr learned my addr 172.21.15.78:0/595362591 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:38.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea3921d640 1 --2- 172.21.15.78:0/595362591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9afd30 0x55cede9acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea38a1c640 1 -- 172.21.15.78:0/595362591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3da0 msgr2=0x55cede9aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:38.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea38a1c640 1 --2- 172.21.15.78:0/595362591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3da0 0x55cede9aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea38a1c640 1 -- 172.21.15.78:0/595362591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9afd30 msgr2=0x55cede9acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:38.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea38a1c640 1 --2- 172.21.15.78:0/595362591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9afd30 0x55cede9acfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea38a1c640 1 -- 172.21.15.78:0/595362591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cede8d1c40 con 0x55cede9a3480 2024-09-11T01:02:38.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea38a1c640 1 --2- 172.21.15.78:0/595362591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a3480 0x55cede9af5c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fea280e3430 tx=0x7fea280a8010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e045eafa1b779273 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:38.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.548+0000 7fea13fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:38.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea13fff640 1 -- 172.21.15.78:0/595362591 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28020070 con 0x55cede9a3480 2024-09-11T01:02:38.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea13fff640 1 -- 172.21.15.78:0/595362591 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fea280ab560 con 0x55cede9a3480 2024-09-11T01:02:38.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea3d055740 1 -- 172.21.15.78:0/595362591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea200043d0 con 0x55cede9a3480 2024-09-11T01:02:38.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea13fff640 1 -- 172.21.15.78:0/595362591 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fea280ab870 con 0x55cede9a3480 2024-09-11T01:02:38.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea3d055740 10 client.16143 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:38.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea3d055740 1 -- 172.21.15.78:0/595362591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cede9be310 con 0x55cede9a3480 2024-09-11T01:02:38.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea13fff640 1 -- 172.21.15.78:0/595362591 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea200043d0 con 0x55cede9a3480 2024-09-11T01:02:38.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea13fff640 1 -- 172.21.15.78:0/595362591 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fea280971a0 con 0x55cede9a3480 2024-09-11T01:02:38.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea3d055740 10 client.16143 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:38.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea3d055740 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:38.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea3d055740 20 client.16143 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:38.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea3d055740 1 --2- 172.21.15.78:0/595362591 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cede9bf840 0x55cede9c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea3d055740 4 client.16143 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:02:38.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea3d055740 1 -- 172.21.15.78:0/595362591 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cede91bda0 con 0x55cede9bf840 2024-09-11T01:02:38.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.549+0000 7fea33fff640 1 --2- 172.21.15.78:0/595362591 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cede9bf840 0x55cede9c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.550+0000 7fea33fff640 1 --2- 172.21.15.78:0/595362591 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cede9bf840 0x55cede9c1d00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eb615f0adde22d3 server_cookie=9ec4b5950dca6c1e in_seq=0 out_seq=0 2024-09-11T01:02:38.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.550+0000 7fea13fff640 10 client.16143 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:38.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.551+0000 7fea13fff640 1 -- 172.21.15.78:0/595362591 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cede91bda0 con 0x55cede9bf840 2024-09-11T01:02:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.551+0000 7fea13fff640 10 client.16143 handle_command_reply: tid=0 2024-09-11T01:02:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.551+0000 7fea3d055740 1 client.16143 shutdown 2024-09-11T01:02:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.551+0000 7fea3d055740 1 -- 172.21.15.78:0/595362591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a3480 msgr2=0x55cede9af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.551+0000 7fea3d055740 1 --2- 172.21.15.78:0/595362591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a3480 0x55cede9af5c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fea280e3430 tx=0x7fea280a8010 comp rx=0 tx=0).stop 2024-09-11T01:02:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.551+0000 7fea3d055740 1 -- 172.21.15.78:0/595362591 shutdown_connections 2024-09-11T01:02:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.551+0000 7fea3d055740 1 --2- 172.21.15.78:0/595362591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9afd30 0x55cede9acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.551+0000 7fea3d055740 1 --2- 172.21.15.78:0/595362591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3da0 0x55cede9aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.551+0000 7fea3d055740 1 --2- 172.21.15.78:0/595362591 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cede9bf840 0x55cede9c1d00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.551+0000 7fea3d055740 1 --2- 172.21.15.78:0/595362591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a3480 0x55cede9af5c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.551+0000 7fea3d055740 1 -- 172.21.15.78:0/595362591 >> 172.21.15.78:0/595362591 conn(0x55cede9a12b0 msgr2=0x55cede9a6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.552+0000 7fea13fff640 0 client.16143 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.552+0000 7fea3d055740 1 -- 172.21.15.78:0/595362591 shutdown_connections 2024-09-11T01:02:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.552+0000 7fea3d055740 1 -- 172.21.15.78:0/595362591 wait complete. 2024-09-11T01:02:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.552+0000 7fea3d055740 20 client.16143 trim_cache size 0 max 16384 2024-09-11T01:02:38.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea3d055740 1 Processor -- start 2024-09-11T01:02:38.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea3d055740 1 -- start start 2024-09-11T01:02:38.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea3d055740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a3da0 0x55cede9bbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea3d055740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9afd30 0x55cede9a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea3d055740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a81e0 0x55cede9ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea3d055740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea341ae600 con 0x55cede9a81e0 2024-09-11T01:02:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea3d055740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea341b02c0 con 0x55cede9a3da0 2024-09-11T01:02:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea3d055740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea341aecd0 con 0x55cede9afd30 2024-09-11T01:02:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea3921d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a81e0 0x55cede9ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea3921d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a81e0 0x55cede9ab2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42906/0 (socket says 172.21.15.78:42906) 2024-09-11T01:02:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea3921d640 1 -- 172.21.15.78:0/228036003 learned_addr learned my addr 172.21.15.78:0/228036003 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea38a1c640 1 --2- 172.21.15.78:0/228036003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a3da0 0x55cede9bbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.569+0000 7fea3921d640 1 -- 172.21.15.78:0/228036003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9afd30 msgr2=0x55cede9a7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.570+0000 7fea33fff640 1 --2- 172.21.15.78:0/228036003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9afd30 0x55cede9a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.570+0000 7fea3921d640 1 --2- 172.21.15.78:0/228036003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9afd30 0x55cede9a7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.570+0000 7fea3921d640 1 -- 172.21.15.78:0/228036003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a3da0 msgr2=0x55cede9bbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.570+0000 7fea3921d640 1 --2- 172.21.15.78:0/228036003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a3da0 0x55cede9bbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.570+0000 7fea3921d640 1 -- 172.21.15.78:0/228036003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cede9a7790 con 0x55cede9a81e0 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.570+0000 7fea38a1c640 1 --2- 172.21.15.78:0/228036003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a3da0 0x55cede9bbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.570+0000 7fea3921d640 1 --2- 172.21.15.78:0/228036003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a81e0 0x55cede9ab2b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fea2c031770 tx=0x7fea2c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb902cb7fe92e95d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.570+0000 7fea1e7fc640 1 -- 172.21.15.78:0/228036003 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c003970 con 0x55cede9a81e0 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.570+0000 7fea1e7fc640 1 -- 172.21.15.78:0/228036003 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2c036040 con 0x55cede9a81e0 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.570+0000 7fea1e7fc640 1 -- 172.21.15.78:0/228036003 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c004800 con 0x55cede9a81e0 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 -- 172.21.15.78:0/228036003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a81e0 msgr2=0x55cede9ab2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 --2- 172.21.15.78:0/228036003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a81e0 0x55cede9ab2b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fea2c031770 tx=0x7fea2c03f000 comp rx=0 tx=0).stop 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 -- 172.21.15.78:0/228036003 shutdown_connections 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 --2- 172.21.15.78:0/228036003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9a81e0 0x55cede9ab2b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 --2- 172.21.15.78:0/228036003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9afd30 0x55cede9a7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 --2- 172.21.15.78:0/228036003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a3da0 0x55cede9bbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 -- 172.21.15.78:0/228036003 >> 172.21.15.78:0/228036003 conn(0x55cede9a12b0 msgr2=0x55cede9a2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 -- 172.21.15.78:0/228036003 shutdown_connections 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 -- 172.21.15.78:0/228036003 wait complete. 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 -- start start 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 Processor -- start 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3da0 0x55cede9b8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a81e0 0x55cede9b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 0x55cede9b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.571+0000 7fea3d055740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea341afb30 con 0x55cede9afd30 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3d055740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea341af3a0 con 0x55cede9a81e0 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3d055740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea341ac5e0 con 0x55cede9a3da0 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3d055740 10 client.0 _ll_register_callbacks cb 0x7fea2800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3921d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 0x55cede9b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3921d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 0x55cede9b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42910/0 (socket says 172.21.15.78:42910) 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3921d640 1 -- 172.21.15.78:0/1661667543 learned_addr learned my addr 172.21.15.78:0/1661667543 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea38a1c640 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3da0 0x55cede9b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea33fff640 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a81e0 0x55cede9b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3921d640 1 -- 172.21.15.78:0/1661667543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3da0 msgr2=0x55cede9b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3921d640 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3da0 0x55cede9b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3921d640 1 -- 172.21.15.78:0/1661667543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a81e0 msgr2=0x55cede9b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3921d640 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a81e0 0x55cede9b86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3921d640 1 -- 172.21.15.78:0/1661667543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cede9c2070 con 0x55cede9afd30 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3921d640 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 0x55cede9b4e90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fea2c02b060 tx=0x7fea2c030700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e98f26a5c560e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea13fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea13fff640 1 -- 172.21.15.78:0/1661667543 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c05f060 con 0x55cede9afd30 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea13fff640 1 -- 172.21.15.78:0/1661667543 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fea2c04d040 con 0x55cede9afd30 2024-09-11T01:02:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3d055740 1 -- 172.21.15.78:0/1661667543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea2c036040 con 0x55cede9afd30 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea13fff640 1 -- 172.21.15.78:0/1661667543 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fea2c030a80 con 0x55cede9afd30 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3d055740 10 client.16149 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.572+0000 7fea3d055740 1 -- 172.21.15.78:0/1661667543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cedea2d3d0 con 0x55cede9afd30 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.573+0000 7fea13fff640 1 -- 172.21.15.78:0/1661667543 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2c036040 con 0x55cede9afd30 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.573+0000 7fea13fff640 1 -- 172.21.15.78:0/1661667543 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fea2c062020 con 0x55cede9afd30 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.573+0000 7fea3d055740 10 client.16149 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.573+0000 7fea3d055740 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.573+0000 7fea3d055740 20 client.16149 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.573+0000 7fea3d055740 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cedea2db20 0x55cedea2ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.573+0000 7fea3d055740 4 client.16149 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.573+0000 7fea3d055740 1 -- 172.21.15.78:0/1661667543 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cede917850 con 0x55cedea2db20 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.573+0000 7fea38a1c640 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cedea2db20 0x55cedea2ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea38a1c640 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cedea2db20 0x55cedea2ff40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5d57899ac6f26c2 server_cookie=b893c21277085b5e in_seq=0 out_seq=0 2024-09-11T01:02:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea13fff640 10 client.16149 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea13fff640 1 -- 172.21.15.78:0/1661667543 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x55cede917850 con 0x55cedea2db20 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea13fff640 10 client.16149 handle_command_reply: tid=0 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea3d055740 1 client.16149 shutdown 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea3d055740 1 -- 172.21.15.78:0/1661667543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 msgr2=0x55cede9b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea3d055740 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 0x55cede9b4e90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fea2c02b060 tx=0x7fea2c030700 comp rx=0 tx=0).stop 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea3d055740 1 -- 172.21.15.78:0/1661667543 shutdown_connections 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea3d055740 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cedea2db20 0x55cedea2ff40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea3d055740 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cede9afd30 0x55cede9b4e90 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea13fff640 0 client.16149 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea3d055740 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cede9a81e0 0x55cede9b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea3d055740 1 --2- 172.21.15.78:0/1661667543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cede9a3da0 0x55cede9b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea3d055740 1 -- 172.21.15.78:0/1661667543 >> 172.21.15.78:0/1661667543 conn(0x55cede9a12b0 msgr2=0x55cede9aaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea3d055740 1 -- 172.21.15.78:0/1661667543 shutdown_connections 2024-09-11T01:02:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea3d055740 1 -- 172.21.15.78:0/1661667543 wait complete. 2024-09-11T01:02:38.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.574+0000 7fea3d055740 20 client.16149 trim_cache size 0 max 16384 2024-09-11T01:02:38.600 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:02:38.600 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:02:38.600 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:02:38.600 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:02:38.600 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:02:38.600 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:02:38.600 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:38.600 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:02:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:38.602 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:38.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:38.603 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:02:38.603 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:02:38.603 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:38.603 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:38.603 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:38.603 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:38.603 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:38.603 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:02:38.603 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:02:38.603 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:02:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.575+0000 7fea3ac80640 1 -- 172.21.15.78:0/1675534861 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fea240b5450 msgr2=0x7fea240b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.575+0000 7fea3ac80640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fea240b5450 0x7fea240b7910 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fea2c006010 tx=0x7fea2c0093f0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.575+0000 7fea3ac80640 1 -- 172.21.15.78:0/1675534861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea34101810 msgr2=0x7fea34199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.575+0000 7fea3ac80640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea34101810 0x7fea34199340 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fea2800d950 tx=0x7fea2800de10 comp rx=0 tx=0).stop 2024-09-11T01:02:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.575+0000 7fea3ac80640 1 -- 172.21.15.78:0/1675534861 shutdown_connections 2024-09-11T01:02:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.575+0000 7fea3ac80640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fea240b5450 0x7fea240b7910 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.575+0000 7fea3ac80640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea34109290 0x7fea3419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.575+0000 7fea3ac80640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea34108910 0x7fea34199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.575+0000 7fea3ac80640 1 --2- 172.21.15.78:0/1675534861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea34101810 0x7fea34199340 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.575+0000 7fea3ac80640 1 -- 172.21.15.78:0/1675534861 >> 172.21.15.78:0/1675534861 conn(0x7fea34076f20 msgr2=0x7fea34100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.575+0000 7fea3ac80640 1 -- 172.21.15.78:0/1675534861 shutdown_connections 2024-09-11T01:02:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:38.575+0000 7fea3ac80640 1 -- 172.21.15.78:0/1675534861 wait complete. 2024-09-11T01:02:39.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 333518 8.81 MB/sec execute 47 sec latency 4981.568 ms 2024-09-11T01:02:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:39 smithi110 ceph-mon[34712]: pgmap v436: 97 pgs: 97 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 205 op/s 2024-09-11T01:02:39.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:39 smithi078 ceph-mon[31315]: pgmap v436: 97 pgs: 97 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 205 op/s 2024-09-11T01:02:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:39 smithi112 ceph-mon[35384]: pgmap v436: 97 pgs: 97 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 205 op/s 2024-09-11T01:02:39.493 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:02:39.493 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:02:39.754 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:40.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.249+0000 7fbbedc44640 1 -- 172.21.15.78:0/990358945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe81076e0 msgr2=0x7fbbe8107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:40.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.249+0000 7fbbedc44640 1 --2- 172.21.15.78:0/990358945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe81076e0 0x7fbbe8107b20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc009900 tx=0x7fbbdc02a410 comp rx=0 tx=0).stop 2024-09-11T01:02:40.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.249+0000 7fbbedc44640 1 -- 172.21.15.78:0/990358945 shutdown_connections 2024-09-11T01:02:40.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.249+0000 7fbbedc44640 1 --2- 172.21.15.78:0/990358945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbbe8108060 0x7fbbe810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:40.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.249+0000 7fbbedc44640 1 --2- 172.21.15.78:0/990358945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe81076e0 0x7fbbe8107b20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:40.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.249+0000 7fbbedc44640 1 --2- 172.21.15.78:0/990358945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe8101cd0 0x7fbbe81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:40.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.249+0000 7fbbedc44640 1 -- 172.21.15.78:0/990358945 >> 172.21.15.78:0/990358945 conn(0x7fbbe80fd740 msgr2=0x7fbbe80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:40.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.249+0000 7fbbedc44640 1 -- 172.21.15.78:0/990358945 shutdown_connections 2024-09-11T01:02:40.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.249+0000 7fbbedc44640 1 -- 172.21.15.78:0/990358945 wait complete. 2024-09-11T01:02:40.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbedc44640 1 Processor -- start 2024-09-11T01:02:40.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbedc44640 1 -- start start 2024-09-11T01:02:40.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbedc44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe8101cd0 0x7fbbe80fdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:40.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbedc44640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbbe81076e0 0x7fbbe80fe180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:40.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbedc44640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8108060 0x7fbbe80fe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:40.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbedc44640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8111ce0 con 0x7fbbe8108060 2024-09-11T01:02:40.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbedc44640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8111b60 con 0x7fbbe81076e0 2024-09-11T01:02:40.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbedc44640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8111e60 con 0x7fbbe8101cd0 2024-09-11T01:02:40.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbe7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8108060 0x7fbbe80fe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:40.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbe77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe8101cd0 0x7fbbe80fdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:40.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbe7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8108060 0x7fbbe80fe6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42928/0 (socket says 172.21.15.78:42928) 2024-09-11T01:02:40.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbe7fff640 1 -- 172.21.15.78:0/363794412 learned_addr learned my addr 172.21.15.78:0/363794412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:40.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.250+0000 7fbbe6ffd640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbbe81076e0 0x7fbbe80fe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:40.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.251+0000 7fbbe7fff640 1 -- 172.21.15.78:0/363794412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe8101cd0 msgr2=0x7fbbe80fdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:40.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.251+0000 7fbbe7fff640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe8101cd0 0x7fbbe80fdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:40.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.251+0000 7fbbe7fff640 1 -- 172.21.15.78:0/363794412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbbe81076e0 msgr2=0x7fbbe80fe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:40.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.251+0000 7fbbe7fff640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbbe81076e0 0x7fbbe80fe180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:40.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.251+0000 7fbbe7fff640 1 -- 172.21.15.78:0/363794412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe81a76f0 con 0x7fbbe8108060 2024-09-11T01:02:40.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.251+0000 7fbbe77fe640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe8101cd0 0x7fbbe80fdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:40.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.251+0000 7fbbe7fff640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8108060 0x7fbbe80fe6c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000eb10 tx=0x7fbbe000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:40.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.251+0000 7fbbe4ff9640 1 -- 172.21.15.78:0/363794412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0004b20 con 0x7fbbe8108060 2024-09-11T01:02:40.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.251+0000 7fbbe4ff9640 1 -- 172.21.15.78:0/363794412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe0004cc0 con 0x7fbbe8108060 2024-09-11T01:02:40.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.251+0000 7fbbe4ff9640 1 -- 172.21.15.78:0/363794412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0010670 con 0x7fbbe8108060 2024-09-11T01:02:40.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.251+0000 7fbbedc44640 1 -- 172.21.15.78:0/363794412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe81a79e0 con 0x7fbbe8108060 2024-09-11T01:02:40.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.251+0000 7fbbedc44640 1 -- 172.21.15.78:0/363794412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe81a7ea0 con 0x7fbbe8108060 2024-09-11T01:02:40.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.252+0000 7fbbedc44640 1 -- 172.21.15.78:0/363794412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe81020b0 con 0x7fbbe8108060 2024-09-11T01:02:40.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.254+0000 7fbbe4ff9640 1 -- 172.21.15.78:0/363794412 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fbbe0005000 con 0x7fbbe8108060 2024-09-11T01:02:40.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.254+0000 7fbbe4ff9640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbbd00b5450 0x7fbbd00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:40.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.254+0000 7fbbe77fe640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbbd00b5450 0x7fbbd00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:40.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.255+0000 7fbbe4ff9640 1 -- 172.21.15.78:0/363794412 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbbe00e66d0 con 0x7fbbe8108060 2024-09-11T01:02:40.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.255+0000 7fbbe77fe640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbbd00b5450 0x7fbbd00b7910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009fc0 tx=0x7fbbd8005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:40.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.257+0000 7fbbe4ff9640 1 -- 172.21.15.78:0/363794412 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbbe0056220 con 0x7fbbe8108060 2024-09-11T01:02:40.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 335005 8.76 MB/sec execute 48 sec latency 45.360 ms 2024-09-11T01:02:40.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:40 smithi112 ceph-mon[35384]: pgmap v437: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 194 op/s 2024-09-11T01:02:40.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.491+0000 7fbbedc44640 1 -- 172.21.15.78:0/363794412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbe80fef40 con 0x7fbbe8108060 2024-09-11T01:02:40.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:40 smithi078 ceph-mon[31315]: pgmap v437: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 194 op/s 2024-09-11T01:02:40.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.491+0000 7fbbe4ff9640 1 -- 172.21.15.78:0/363794412 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11993 (secure 0 0 0) 0x7fbbe80fef40 con 0x7fbbe8108060 2024-09-11T01:02:40.495 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:02:40.495 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":33,"btime":"2024-09-11T01:02:33:154034+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:02:32.177690+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T01:02:40.495 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 33 2024-09-11T01:02:40.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.494+0000 7fbbedc44640 1 -- 172.21.15.78:0/363794412 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbbd00b5450 msgr2=0x7fbbd00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:40.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.494+0000 7fbbedc44640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbbd00b5450 0x7fbbd00b7910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009fc0 tx=0x7fbbd8005f90 comp rx=0 tx=0).stop 2024-09-11T01:02:40.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.494+0000 7fbbedc44640 1 -- 172.21.15.78:0/363794412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8108060 msgr2=0x7fbbe80fe6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:40.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.494+0000 7fbbedc44640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8108060 0x7fbbe80fe6c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000eb10 tx=0x7fbbe000c2d0 comp rx=0 tx=0).stop 2024-09-11T01:02:40.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.494+0000 7fbbedc44640 1 -- 172.21.15.78:0/363794412 shutdown_connections 2024-09-11T01:02:40.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.494+0000 7fbbedc44640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbbd00b5450 0x7fbbd00b7910 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:40.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.494+0000 7fbbedc44640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe8108060 0x7fbbe80fe6c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:40.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.494+0000 7fbbedc44640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbbe81076e0 0x7fbbe80fe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:40.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.494+0000 7fbbedc44640 1 --2- 172.21.15.78:0/363794412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbe8101cd0 0x7fbbe80fdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:40.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.494+0000 7fbbedc44640 1 -- 172.21.15.78:0/363794412 >> 172.21.15.78:0/363794412 conn(0x7fbbe80fd740 msgr2=0x7fbbe806b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:40.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.494+0000 7fbbedc44640 1 -- 172.21.15.78:0/363794412 shutdown_connections 2024-09-11T01:02:40.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:40.494+0000 7fbbedc44640 1 -- 172.21.15.78:0/363794412 wait complete. 2024-09-11T01:02:40.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:40 smithi110 ceph-mon[34712]: pgmap v437: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 194 op/s 2024-09-11T01:02:41.313 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:02:41.424 INFO:tasks.workunit.client.0.smithi078.stdout: 1 337061 8.84 MB/sec execute 49 sec latency 334.061 ms 2024-09-11T01:02:41.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:41 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/363794412' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:02:41.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:41 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/363794412' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:02:41.592 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:41.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:41 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/363794412' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:02:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.068+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3599839983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe18108030 msgr2=0x7efe1810e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.068+0000 7efe1cfbb640 1 --2- 172.21.15.78:0/3599839983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe18108030 0x7efe1810e830 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7efe10009ca0 tx=0x7efe1002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.068+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3599839983 shutdown_connections 2024-09-11T01:02:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.068+0000 7efe1cfbb640 1 --2- 172.21.15.78:0/3599839983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe18108030 0x7efe1810e830 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.068+0000 7efe1cfbb640 1 --2- 172.21.15.78:0/3599839983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe181076b0 0x7efe18107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.068+0000 7efe1cfbb640 1 --2- 172.21.15.78:0/3599839983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe18101ca0 0x7efe18102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.068+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3599839983 >> 172.21.15.78:0/3599839983 conn(0x7efe180fd6f0 msgr2=0x7efe180ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.068+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3599839983 shutdown_connections 2024-09-11T01:02:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.068+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3599839983 wait complete. 2024-09-11T01:02:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.069+0000 7efe1cfbb640 1 Processor -- start 2024-09-11T01:02:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.069+0000 7efe1cfbb640 1 -- start start 2024-09-11T01:02:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.069+0000 7efe1cfbb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe18101ca0 0x7efe18199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.069+0000 7efe1cfbb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe181076b0 0x7efe1819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.069+0000 7efe1cfbb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18108030 0x7efe181a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.069+0000 7efe1cfbb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe18111cb0 con 0x7efe181076b0 2024-09-11T01:02:42.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.069+0000 7efe1cfbb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe18111b30 con 0x7efe18101ca0 2024-09-11T01:02:42.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.069+0000 7efe1cfbb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe18111e30 con 0x7efe18108030 2024-09-11T01:02:42.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.069+0000 7efe0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe181076b0 0x7efe1819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.069+0000 7efe0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe181076b0 0x7efe1819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42944/0 (socket says 172.21.15.78:42944) 2024-09-11T01:02:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.069+0000 7efe0ffff640 1 -- 172.21.15.78:0/3493533759 learned_addr learned my addr 172.21.15.78:0/3493533759 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.069+0000 7efe1659c640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe18101ca0 0x7efe18199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe16d9d640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18108030 0x7efe181a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe0ffff640 1 -- 172.21.15.78:0/3493533759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18108030 msgr2=0x7efe181a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe0ffff640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18108030 0x7efe181a5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe0ffff640 1 -- 172.21.15.78:0/3493533759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe18101ca0 msgr2=0x7efe18199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe0ffff640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe18101ca0 0x7efe18199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe0ffff640 1 -- 172.21.15.78:0/3493533759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe181a5580 con 0x7efe181076b0 2024-09-11T01:02:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe16d9d640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18108030 0x7efe181a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe0ffff640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe181076b0 0x7efe1819a500 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efe0000d950 tx=0x7efe0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe0f7fe640 1 -- 172.21.15.78:0/3493533759 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00017070 con 0x7efe181076b0 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe0f7fe640 1 -- 172.21.15.78:0/3493533759 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe00009c30 con 0x7efe181076b0 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe0f7fe640 1 -- 172.21.15.78:0/3493533759 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0000f620 con 0x7efe181076b0 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3493533759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe181a5750 con 0x7efe181076b0 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.070+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3493533759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe181a5c30 con 0x7efe181076b0 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.072+0000 7efe0f7fe640 1 -- 172.21.15.78:0/3493533759 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7efe00010460 con 0x7efe181076b0 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe1f390740 1 Processor -- start 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe1f390740 1 -- start start 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe1f390740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f3864d0 0x55823f3868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe1f390740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 0x55823f392840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe1f390740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f392d80 0x55823f395170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe1f390740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55823f2fcea0 con 0x55823f386df0 2024-09-11T01:02:42.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe1f390740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55823f2ec2d0 con 0x55823f3864d0 2024-09-11T01:02:42.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe1f390740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55823f2f00e0 con 0x55823f392d80 2024-09-11T01:02:42.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe1659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 0x55823f392840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe1659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 0x55823f392840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42950/0 (socket says 172.21.15.78:42950) 2024-09-11T01:02:42.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe1659c640 1 -- 172.21.15.78:0/1507938441 learned_addr learned my addr 172.21.15.78:0/1507938441 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:42.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe16d9d640 1 --2- 172.21.15.78:0/1507938441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f3864d0 0x55823f3868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe0f7fe640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efe040b5450 0x7efe040b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe0ffff640 1 --2- 172.21.15.78:0/1507938441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f392d80 0x55823f395170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.073+0000 7efe1659c640 1 -- 172.21.15.78:0/1507938441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f392d80 msgr2=0x55823f395170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:42.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.074+0000 7efe1659c640 1 --2- 172.21.15.78:0/1507938441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f392d80 0x55823f395170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.074+0000 7efe1659c640 1 -- 172.21.15.78:0/1507938441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f3864d0 msgr2=0x55823f3868b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:42.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.074+0000 7efe1659c640 1 --2- 172.21.15.78:0/1507938441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f3864d0 0x55823f3868b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.074+0000 7efe1659c640 1 -- 172.21.15.78:0/1507938441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55823f04a650 con 0x55823f386df0 2024-09-11T01:02:42.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.074+0000 7efe16d9d640 1 --2- 172.21.15.78:0/1507938441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f3864d0 0x55823f3868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:42.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.074+0000 7efe16d9d640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efe040b5450 0x7efe040b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.074+0000 7efe0f7fe640 1 -- 172.21.15.78:0/3493533759 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efe000e5b80 con 0x7efe181076b0 2024-09-11T01:02:42.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.074+0000 7efe1659c640 1 --2- 172.21.15.78:0/1507938441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 0x55823f392840 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efe0800bd10 tx=0x7efe080107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31f58d9f18a5401 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:42.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.074+0000 7efe0cff9640 1 -- 172.21.15.78:0/1507938441 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08004030 con 0x55823f386df0 2024-09-11T01:02:42.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.074+0000 7efe0cff9640 1 -- 172.21.15.78:0/1507938441 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe080035f0 con 0x55823f386df0 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.074+0000 7efe0cff9640 1 -- 172.21.15.78:0/1507938441 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe080117b0 con 0x55823f386df0 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.075+0000 7efe1f390740 1 -- 172.21.15.78:0/1507938441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 msgr2=0x55823f392840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.075+0000 7efe16d9d640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efe040b5450 0x7efe040b7910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efe10005cc0 tx=0x7efe100095c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.075+0000 7efe1f390740 1 --2- 172.21.15.78:0/1507938441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 0x55823f392840 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efe0800bd10 tx=0x7efe080107a0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.075+0000 7efe1f390740 1 -- 172.21.15.78:0/1507938441 shutdown_connections 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.075+0000 7efe1f390740 1 --2- 172.21.15.78:0/1507938441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f392d80 0x55823f395170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.075+0000 7efe1f390740 1 --2- 172.21.15.78:0/1507938441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 0x55823f392840 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.075+0000 7efe1f390740 1 --2- 172.21.15.78:0/1507938441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f3864d0 0x55823f3868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.075+0000 7efe1f390740 1 -- 172.21.15.78:0/1507938441 >> 172.21.15.78:0/1507938441 conn(0x55823f384300 msgr2=0x55823f3854a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.075+0000 7efe1f390740 1 -- 172.21.15.78:0/1507938441 shutdown_connections 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.075+0000 7efe1f390740 1 -- 172.21.15.78:0/1507938441 wait complete. 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.075+0000 7efe1f390740 1 -- start start 2024-09-11T01:02:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.075+0000 7efe1f390740 1 Processor -- start 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe1f390740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f3864d0 0x55823f3925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe1f390740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f386df0 0x55823f38fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe1f390740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f392d80 0x55823f38ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe1f390740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55823f14d700 con 0x55823f392d80 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe1f390740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55823f287160 con 0x55823f386df0 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe1f390740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55823f395a00 con 0x55823f3864d0 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe1f390740 10 client.0 _ll_register_callbacks cb 0x55823f2a09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe16d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f392d80 0x55823f38ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe16d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f392d80 0x55823f38ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42952/0 (socket says 172.21.15.78:42952) 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe16d9d640 1 -- 172.21.15.78:0/435037041 learned_addr learned my addr 172.21.15.78:0/435037041 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe0ffff640 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f386df0 0x55823f38fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe1659c640 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f3864d0 0x55823f3925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe16d9d640 1 -- 172.21.15.78:0/435037041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f3864d0 msgr2=0x55823f3925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe16d9d640 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f3864d0 0x55823f3925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe16d9d640 1 -- 172.21.15.78:0/435037041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f386df0 msgr2=0x55823f38fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe16d9d640 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f386df0 0x55823f38fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe16d9d640 1 -- 172.21.15.78:0/435037041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823f2b4c90 con 0x55823f392d80 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe0ffff640 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f386df0 0x55823f38fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe16d9d640 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f392d80 0x55823f38ffc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efe100098d0 tx=0x7efe1002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bddd3dd918a3669 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efdfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efdfdffb640 1 -- 172.21.15.78:0/435037041 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10004740 con 0x55823f392d80 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efdfdffb640 1 -- 172.21.15.78:0/435037041 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efe1002f660 con 0x55823f392d80 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.076+0000 7efe1f390740 1 -- 172.21.15.78:0/435037041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe080035f0 con 0x55823f392d80 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efdfdffb640 1 -- 172.21.15.78:0/435037041 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efe1002f860 con 0x55823f392d80 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efe1f390740 10 client.16167 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efe1f390740 1 -- 172.21.15.78:0/435037041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55823f3a12b0 con 0x55823f392d80 2024-09-11T01:02:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efdfdffb640 1 -- 172.21.15.78:0/435037041 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe080035f0 con 0x55823f392d80 2024-09-11T01:02:42.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efdfdffb640 1 -- 172.21.15.78:0/435037041 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7efe10030a00 con 0x55823f392d80 2024-09-11T01:02:42.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efe1f390740 10 client.16167 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:42.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efe1f390740 10 client.16167 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:02:42.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efe1f390740 20 client.16167 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:42.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efe1f390740 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55823f3a2760 0x55823f3a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efe1f390740 4 client.16167 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:02:42.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efe1f390740 1 -- 172.21.15.78:0/435037041 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55823f2fedf0 con 0x55823f3a2760 2024-09-11T01:02:42.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efe1659c640 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55823f3a2760 0x55823f3a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efe1659c640 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55823f3a2760 0x55823f3a4b90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=381b353da41cdac6 server_cookie=7a2580c5ffbe50c5 in_seq=0 out_seq=0 2024-09-11T01:02:42.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.077+0000 7efdfdffb640 10 client.16167 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:02:42.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efdfdffb640 1 -- 172.21.15.78:0/435037041 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55823f2fedf0 con 0x55823f3a2760 2024-09-11T01:02:42.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efdfdffb640 10 client.16167 handle_command_reply: tid=0 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efe1f390740 1 client.16167 shutdown 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efe1f390740 1 -- 172.21.15.78:0/435037041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f392d80 msgr2=0x55823f38ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efe1f390740 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f392d80 0x55823f38ffc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efe100098d0 tx=0x7efe1002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efe1f390740 1 -- 172.21.15.78:0/435037041 shutdown_connections 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efe1f390740 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55823f3a2760 0x55823f3a4b90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efdfdffb640 0 client.16167 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efe1f390740 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f392d80 0x55823f38ffc0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efe1f390740 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f386df0 0x55823f38fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efe1f390740 1 --2- 172.21.15.78:0/435037041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f3864d0 0x55823f3925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efe1f390740 1 -- 172.21.15.78:0/435037041 >> 172.21.15.78:0/435037041 conn(0x55823f384300 msgr2=0x55823f389ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efe1f390740 1 -- 172.21.15.78:0/435037041 shutdown_connections 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.078+0000 7efe1f390740 1 -- 172.21.15.78:0/435037041 wait complete. 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.079+0000 7efe1f390740 20 client.16167 trim_cache size 0 max 16384 2024-09-11T01:02:42.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe1f390740 1 Processor -- start 2024-09-11T01:02:42.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe1f390740 1 -- start start 2024-09-11T01:02:42.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe1f390740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f3864d0 0x55823f39ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe1f390740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 0x55823f385a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe1f390740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f392d80 0x55823f39b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe1f390740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe181ae580 con 0x55823f386df0 2024-09-11T01:02:42.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe1f390740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe181b0240 con 0x55823f392d80 2024-09-11T01:02:42.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe1f390740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe181aec50 con 0x55823f3864d0 2024-09-11T01:02:42.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 0x55823f385a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe1659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f3864d0 0x55823f39ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 0x55823f385a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42962/0 (socket says 172.21.15.78:42962) 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe0ffff640 1 -- 172.21.15.78:0/3717575481 learned_addr learned my addr 172.21.15.78:0/3717575481 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe16d9d640 1 --2- 172.21.15.78:0/3717575481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f392d80 0x55823f39b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe0ffff640 1 -- 172.21.15.78:0/3717575481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f3864d0 msgr2=0x55823f39ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe0ffff640 1 --2- 172.21.15.78:0/3717575481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f3864d0 0x55823f39ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe0ffff640 1 -- 172.21.15.78:0/3717575481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f392d80 msgr2=0x55823f39b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe0ffff640 1 --2- 172.21.15.78:0/3717575481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f392d80 0x55823f39b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe0ffff640 1 -- 172.21.15.78:0/3717575481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55823f386c20 con 0x55823f386df0 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe1659c640 1 --2- 172.21.15.78:0/3717575481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f3864d0 0x55823f39ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efe0ffff640 1 --2- 172.21.15.78:0/3717575481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 0x55823f385a60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efe000abd70 tx=0x7efe000b5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4e6a81d92d185b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efdfffff640 1 -- 172.21.15.78:0/3717575481 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00020040 con 0x55823f386df0 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efdfffff640 1 -- 172.21.15.78:0/3717575481 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe000e37c0 con 0x55823f386df0 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.092+0000 7efdfffff640 1 -- 172.21.15.78:0/3717575481 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe000109e0 con 0x55823f386df0 2024-09-11T01:02:42.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 -- 172.21.15.78:0/3717575481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 msgr2=0x55823f385a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 --2- 172.21.15.78:0/3717575481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 0x55823f385a60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efe000abd70 tx=0x7efe000b5040 comp rx=0 tx=0).stop 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 -- 172.21.15.78:0/3717575481 shutdown_connections 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 --2- 172.21.15.78:0/3717575481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f392d80 0x55823f39b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 --2- 172.21.15.78:0/3717575481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f386df0 0x55823f385a60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 --2- 172.21.15.78:0/3717575481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f3864d0 0x55823f39ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 -- 172.21.15.78:0/3717575481 >> 172.21.15.78:0/3717575481 conn(0x55823f384300 msgr2=0x55823f385350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 -- 172.21.15.78:0/3717575481 shutdown_connections 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 -- 172.21.15.78:0/3717575481 wait complete. 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 -- start start 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 Processor -- start 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f3864d0 0x55823f3a9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f386df0 0x55823f3a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f392d80 0x55823f3aa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe181afab0 con 0x55823f3864d0 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe181af320 con 0x55823f392d80 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe181ac560 con 0x55823f386df0 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1f390740 10 client.0 _ll_register_callbacks cb 0x7efe10009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.093+0000 7efe1659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f3864d0 0x55823f3a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe1659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f3864d0 0x55823f3a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42970/0 (socket says 172.21.15.78:42970) 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe1659c640 1 -- 172.21.15.78:0/3341299481 learned_addr learned my addr 172.21.15.78:0/3341299481 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe16d9d640 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f392d80 0x55823f3aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe0ffff640 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f386df0 0x55823f3a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe1659c640 1 -- 172.21.15.78:0/3341299481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f386df0 msgr2=0x55823f3a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe1659c640 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f386df0 0x55823f3a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe1659c640 1 -- 172.21.15.78:0/3341299481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f392d80 msgr2=0x55823f3aa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe1659c640 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f392d80 0x55823f3aa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe1659c640 1 -- 172.21.15.78:0/3341299481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823f3aa800 con 0x55823f3864d0 2024-09-11T01:02:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe1659c640 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f3864d0 0x55823f3a9490 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efe0800bd10 tx=0x7efe0800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f521e06ab1a56d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efdfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efdfdffb640 1 -- 172.21.15.78:0/3341299481 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08032060 con 0x55823f3864d0 2024-09-11T01:02:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efdfdffb640 1 -- 172.21.15.78:0/3341299481 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efe080037f0 con 0x55823f3864d0 2024-09-11T01:02:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe1f390740 1 -- 172.21.15.78:0/3341299481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe000e37c0 con 0x55823f3864d0 2024-09-11T01:02:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efdfdffb640 1 -- 172.21.15.78:0/3341299481 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efe080039f0 con 0x55823f3864d0 2024-09-11T01:02:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe1f390740 10 client.16173 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efe1f390740 1 -- 172.21.15.78:0/3341299481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55823f413de0 con 0x55823f3864d0 2024-09-11T01:02:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.094+0000 7efdfdffb640 1 -- 172.21.15.78:0/3341299481 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe000e37c0 con 0x55823f3864d0 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efdfdffb640 1 -- 172.21.15.78:0/3341299481 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7efe08035020 con 0x55823f3864d0 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 10 client.16173 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 10 client.16173 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 20 client.16173 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55823f414280 0x55823f416670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 4 client.16173 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 1 -- 172.21.15.78:0/3341299481 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55823f2fa8a0 con 0x55823f414280 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe0ffff640 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55823f414280 0x55823f416670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe0ffff640 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55823f414280 0x55823f416670 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a6f6f81aa482b89 server_cookie=d9bd6ff561106d4d in_seq=0 out_seq=0 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efdfdffb640 10 client.16173 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efdfdffb640 1 -- 172.21.15.78:0/3341299481 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55823f2fa8a0 con 0x55823f414280 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efdfdffb640 10 client.16173 handle_command_reply: tid=0 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 1 client.16173 shutdown 2024-09-11T01:02:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 1 -- 172.21.15.78:0/3341299481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f3864d0 msgr2=0x55823f3a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f3864d0 0x55823f3a9490 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efe0800bd10 tx=0x7efe0800b780 comp rx=0 tx=0).stop 2024-09-11T01:02:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 1 -- 172.21.15.78:0/3341299481 shutdown_connections 2024-09-11T01:02:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55823f414280 0x55823f416670 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55823f392d80 0x55823f3aa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55823f386df0 0x55823f3a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efdfdffb640 0 client.16173 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:02:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 1 --2- 172.21.15.78:0/3341299481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823f3864d0 0x55823f3a9490 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 1 -- 172.21.15.78:0/3341299481 >> 172.21.15.78:0/3341299481 conn(0x55823f384300 msgr2=0x55823f382c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.095+0000 7efe1f390740 1 -- 172.21.15.78:0/3341299481 shutdown_connections 2024-09-11T01:02:42.123 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:02:42.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1f390740 1 -- 172.21.15.78:0/3341299481 wait complete. 2024-09-11T01:02:42.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1f390740 20 client.16173 trim_cache size 0 max 16384 2024-09-11T01:02:42.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3493533759 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efe040b5450 msgr2=0x7efe040b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:42.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1cfbb640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efe040b5450 0x7efe040b7910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efe10005cc0 tx=0x7efe100095c0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3493533759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe181076b0 msgr2=0x7efe1819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:42.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1cfbb640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe181076b0 0x7efe1819a500 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efe0000d950 tx=0x7efe0000de10 comp rx=0 tx=0).stop 2024-09-11T01:02:42.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3493533759 shutdown_connections 2024-09-11T01:02:42.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1cfbb640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efe040b5450 0x7efe040b7910 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1cfbb640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe18108030 0x7efe181a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1cfbb640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe181076b0 0x7efe1819a500 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1cfbb640 1 --2- 172.21.15.78:0/3493533759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe18101ca0 0x7efe18199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:42.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3493533759 >> 172.21.15.78:0/3493533759 conn(0x7efe180fd6f0 msgr2=0x7efe180fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:42.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3493533759 shutdown_connections 2024-09-11T01:02:42.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:42.096+0000 7efe1cfbb640 1 -- 172.21.15.78:0/3493533759 wait complete. 2024-09-11T01:02:42.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:42 smithi078 ceph-mon[31315]: pgmap v438: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 151 op/s 2024-09-11T01:02:42.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 337061 8.66 MB/sec execute 50 sec latency 1334.096 ms 2024-09-11T01:02:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:42 smithi112 ceph-mon[35384]: pgmap v438: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 151 op/s 2024-09-11T01:02:42.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:42 smithi110 ceph-mon[34712]: pgmap v438: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 151 op/s 2024-09-11T01:02:42.895 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:02:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:43.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 337061 8.49 MB/sec execute 51 sec latency 2334.136 ms 2024-09-11T01:02:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.702+0000 7f853badd640 1 -- 172.21.15.78:0/138983101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534103ca0 msgr2=0x7f8534104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.702+0000 7f853badd640 1 --2- 172.21.15.78:0/138983101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534103ca0 0x7f8534104080 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8530009900 tx=0x7f853002a410 comp rx=0 tx=0).stop 2024-09-11T01:02:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.702+0000 7f853badd640 1 -- 172.21.15.78:0/138983101 shutdown_connections 2024-09-11T01:02:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.702+0000 7f853badd640 1 --2- 172.21.15.78:0/138983101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f853410d100 0x7f853410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.702+0000 7f853badd640 1 --2- 172.21.15.78:0/138983101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85341045c0 0x7f853410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.702+0000 7f853badd640 1 --2- 172.21.15.78:0/138983101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534103ca0 0x7f8534104080 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.702+0000 7f853badd640 1 -- 172.21.15.78:0/138983101 >> 172.21.15.78:0/138983101 conn(0x7f85340fd700 msgr2=0x7f85340ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.702+0000 7f853badd640 1 -- 172.21.15.78:0/138983101 shutdown_connections 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.702+0000 7f853badd640 1 -- 172.21.15.78:0/138983101 wait complete. 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.702+0000 7f853badd640 1 Processor -- start 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f853badd640 1 -- start start 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f853badd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534103ca0 0x7f8534199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f853badd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85341045c0 0x7f8534199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f853badd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853410d100 0x7f853419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f853badd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8534112d70 con 0x7f8534103ca0 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f853badd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8534112bf0 con 0x7f85341045c0 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f853badd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8534112ef0 con 0x7f853410d100 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f8539879640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534103ca0 0x7f8534199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f8539879640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534103ca0 0x7f8534199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39904/0 (socket says 172.21.15.78:39904) 2024-09-11T01:02:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f8539879640 1 -- 172.21.15.78:0/70315560 learned_addr learned my addr 172.21.15.78:0/70315560 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f853a07a640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853410d100 0x7f853419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f8539879640 1 -- 172.21.15.78:0/70315560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853410d100 msgr2=0x7f853419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f8539879640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853410d100 0x7f853419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f8539879640 1 -- 172.21.15.78:0/70315560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85341045c0 msgr2=0x7f8534199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f8539879640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85341045c0 0x7f8534199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f8539879640 1 -- 172.21.15.78:0/70315560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853419f3b0 con 0x7f8534103ca0 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f853a07a640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853410d100 0x7f853419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.703+0000 7f8539879640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534103ca0 0x7f8534199300 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8530006010 tx=0x7f85300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.704+0000 7f852affd640 1 -- 172.21.15.78:0/70315560 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853003f030 con 0x7f8534103ca0 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.704+0000 7f852affd640 1 -- 172.21.15.78:0/70315560 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8530003800 con 0x7f8534103ca0 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.704+0000 7f853badd640 1 -- 172.21.15.78:0/70315560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853419f640 con 0x7f8534103ca0 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.704+0000 7f852affd640 1 -- 172.21.15.78:0/70315560 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530035070 con 0x7f8534103ca0 2024-09-11T01:02:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.704+0000 7f853badd640 1 -- 172.21.15.78:0/70315560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85341a5d60 con 0x7f8534103ca0 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.706+0000 7f853deb2740 1 Processor -- start 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.706+0000 7f853deb2740 1 -- start start 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.706+0000 7f853deb2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f0588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.706+0000 7f853deb2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f058df0 0x56236f068c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.706+0000 7f853deb2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f069190 0x56236f06b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.706+0000 7f852affd640 1 -- 172.21.15.78:0/70315560 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8530031b10 con 0x7f8534103ca0 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.706+0000 7f853deb2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56236efceea0 con 0x56236f069190 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.706+0000 7f853deb2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56236efbe2d0 con 0x56236f0584d0 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.706+0000 7f853deb2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56236efc20e0 con 0x56236f058df0 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f8539078640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f069190 0x56236f06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f8539078640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f069190 0x56236f06b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39918/0 (socket says 172.21.15.78:39918) 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f8539078640 1 -- 172.21.15.78:0/900040269 learned_addr learned my addr 172.21.15.78:0/900040269 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f853a07a640 1 --2- 172.21.15.78:0/900040269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f0588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f8539879640 1 --2- 172.21.15.78:0/900040269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f058df0 0x56236f068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f8539078640 1 -- 172.21.15.78:0/900040269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f058df0 msgr2=0x56236f068c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f8539078640 1 --2- 172.21.15.78:0/900040269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f058df0 0x56236f068c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f8539078640 1 -- 172.21.15.78:0/900040269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 msgr2=0x56236f0588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f8539078640 1 --2- 172.21.15.78:0/900040269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f0588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f8539078640 1 -- 172.21.15.78:0/900040269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236ed1c650 con 0x56236f069190 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f853a07a640 1 --2- 172.21.15.78:0/900040269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f0588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f8539078640 1 --2- 172.21.15.78:0/900040269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f069190 0x56236f06b580 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f852400a9d0 tx=0x7f852400ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf79ce53edfecf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f851ffff640 1 -- 172.21.15.78:0/900040269 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852400cde0 con 0x56236f069190 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f851ffff640 1 -- 172.21.15.78:0/900040269 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8524005020 con 0x56236f069190 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f851ffff640 1 -- 172.21.15.78:0/900040269 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524004a90 con 0x56236f069190 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f852affd640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f85200b54d0 0x7f85200b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.707+0000 7f853a07a640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f85200b54d0 0x7f85200b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.708+0000 7f852affd640 1 -- 172.21.15.78:0/70315560 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f8530106590 con 0x7f8534103ca0 2024-09-11T01:02:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.708+0000 7f853deb2740 1 -- 172.21.15.78:0/900040269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f069190 msgr2=0x56236f06b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.708+0000 7f853a07a640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f85200b54d0 0x7f85200b7990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f852c00af20 tx=0x7f852c00a900 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.708+0000 7f853deb2740 1 --2- 172.21.15.78:0/900040269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f069190 0x56236f06b580 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f852400a9d0 tx=0x7f852400ae90 comp rx=0 tx=0).stop 2024-09-11T01:02:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.708+0000 7f853deb2740 1 -- 172.21.15.78:0/900040269 shutdown_connections 2024-09-11T01:02:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.708+0000 7f853deb2740 1 --2- 172.21.15.78:0/900040269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f069190 0x56236f06b580 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.708+0000 7f853deb2740 1 --2- 172.21.15.78:0/900040269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f058df0 0x56236f068c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.708+0000 7f853deb2740 1 --2- 172.21.15.78:0/900040269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f0588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.708+0000 7f853deb2740 1 -- 172.21.15.78:0/900040269 >> 172.21.15.78:0/900040269 conn(0x56236f056300 msgr2=0x56236f0574a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.708+0000 7f853deb2740 1 -- 172.21.15.78:0/900040269 shutdown_connections 2024-09-11T01:02:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.708+0000 7f853deb2740 1 -- 172.21.15.78:0/900040269 wait complete. 2024-09-11T01:02:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.708+0000 7f853deb2740 1 -- start start 2024-09-11T01:02:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f853deb2740 1 Processor -- start 2024-09-11T01:02:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f853deb2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f068a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f853deb2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f065e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f853deb2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f069190 0x56236f0663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f853deb2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56236ee1f700 con 0x56236f058df0 2024-09-11T01:02:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f853deb2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56236ef59160 con 0x56236f0584d0 2024-09-11T01:02:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f853deb2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56236f06be40 con 0x56236f069190 2024-09-11T01:02:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f853deb2740 10 client.0 _ll_register_callbacks cb 0x56236ef729e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f8539078640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f065e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f8539078640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f065e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39928/0 (socket says 172.21.15.78:39928) 2024-09-11T01:02:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f8539078640 1 -- 172.21.15.78:0/1338698764 learned_addr learned my addr 172.21.15.78:0/1338698764 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f8539879640 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f8539078640 1 -- 172.21.15.78:0/1338698764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f069190 msgr2=0x56236f0663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f853a07a640 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f069190 0x56236f0663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f8539078640 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f069190 0x56236f0663d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f8539078640 1 -- 172.21.15.78:0/1338698764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 msgr2=0x56236f068a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f8539078640 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f068a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f8539078640 1 -- 172.21.15.78:0/1338698764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236ef86c90 con 0x56236f058df0 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f8539879640 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.709+0000 7f853a07a640 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f069190 0x56236f0663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f8539078640 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f065e50 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8524004700 tx=0x7f8524004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac51878b07f00d20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f851d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f851d7fa640 1 -- 172.21.15.78:0/1338698764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524028060 con 0x56236f058df0 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f851d7fa640 1 -- 172.21.15.78:0/1338698764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f85240053e0 con 0x56236f058df0 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f853deb2740 1 -- 172.21.15.78:0/1338698764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8524005020 con 0x56236f058df0 2024-09-11T01:02:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f851d7fa640 1 -- 172.21.15.78:0/1338698764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8524004810 con 0x56236f058df0 2024-09-11T01:02:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f853deb2740 10 client.16185 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f853deb2740 1 -- 172.21.15.78:0/1338698764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56236f077720 con 0x56236f058df0 2024-09-11T01:02:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f851d7fa640 1 -- 172.21.15.78:0/1338698764 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8524005020 con 0x56236f058df0 2024-09-11T01:02:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f851d7fa640 1 -- 172.21.15.78:0/1338698764 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f852402b030 con 0x56236f058df0 2024-09-11T01:02:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f853deb2740 10 client.16185 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f853deb2740 10 client.16185 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:02:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f853deb2740 20 client.16185 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f853deb2740 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56236f078e00 0x56236f07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f853deb2740 4 client.16185 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.710+0000 7f853deb2740 1 -- 172.21.15.78:0/1338698764 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56236efd0df0 con 0x56236f078e00 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.711+0000 7f8539879640 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56236f078e00 0x56236f07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.711+0000 7f851d7fa640 10 client.16185 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.711+0000 7f8539879640 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56236f078e00 0x56236f07b1f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3f9277b453d061f server_cookie=6577488ade9b87db in_seq=0 out_seq=0 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f851d7fa640 1 -- 172.21.15.78:0/1338698764 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56236efd0df0 con 0x56236f078e00 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f851d7fa640 10 client.16185 handle_command_reply: tid=0 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f853deb2740 1 client.16185 shutdown 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f853deb2740 1 -- 172.21.15.78:0/1338698764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 msgr2=0x56236f065e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f853deb2740 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f065e50 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8524004700 tx=0x7f8524004250 comp rx=0 tx=0).stop 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f853deb2740 1 -- 172.21.15.78:0/1338698764 shutdown_connections 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f853deb2740 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56236f078e00 0x56236f07b1f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f853deb2740 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f069190 0x56236f0663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f853deb2740 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f065e50 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f853deb2740 1 --2- 172.21.15.78:0/1338698764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f851d7fa640 0 client.16185 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f853deb2740 1 -- 172.21.15.78:0/1338698764 >> 172.21.15.78:0/1338698764 conn(0x56236f056300 msgr2=0x56236f05be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f853deb2740 1 -- 172.21.15.78:0/1338698764 shutdown_connections 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f853deb2740 1 -- 172.21.15.78:0/1338698764 wait complete. 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.712+0000 7f853deb2740 20 client.16185 trim_cache size 0 max 16384 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.725+0000 7f853deb2740 1 Processor -- start 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.725+0000 7f853deb2740 1 -- start start 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.725+0000 7f853deb2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f0584d0 0x56236f065340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.725+0000 7f853deb2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f06af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.725+0000 7f853deb2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f069190 0x56236f06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.725+0000 7f853deb2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85341ac380 con 0x56236f058df0 2024-09-11T01:02:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.725+0000 7f853deb2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85341ae040 con 0x56236f069190 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.725+0000 7f853deb2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85341aca50 con 0x56236f0584d0 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.725+0000 7f8539078640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f06af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f8539078640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f06af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39940/0 (socket says 172.21.15.78:39940) 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f8539078640 1 -- 172.21.15.78:0/249722589 learned_addr learned my addr 172.21.15.78:0/249722589 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f853a07a640 1 --2- 172.21.15.78:0/249722589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f069190 0x56236f06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f8539879640 1 --2- 172.21.15.78:0/249722589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f0584d0 0x56236f065340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f8539078640 1 -- 172.21.15.78:0/249722589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f0584d0 msgr2=0x56236f065340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f8539078640 1 --2- 172.21.15.78:0/249722589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f0584d0 0x56236f065340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f8539078640 1 -- 172.21.15.78:0/249722589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f069190 msgr2=0x56236f06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f8539078640 1 --2- 172.21.15.78:0/249722589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f069190 0x56236f06b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f8539078640 1 -- 172.21.15.78:0/249722589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236f064350 con 0x56236f058df0 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f8539078640 1 --2- 172.21.15.78:0/249722589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f06af00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f85240047e0 tx=0x7f852400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4436bdb7bae90a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f851f7fe640 1 -- 172.21.15.78:0/249722589 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852401d070 con 0x56236f058df0 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f851f7fe640 1 -- 172.21.15.78:0/249722589 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8524003cc0 con 0x56236f058df0 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.726+0000 7f851f7fe640 1 -- 172.21.15.78:0/249722589 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524028040 con 0x56236f058df0 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 -- 172.21.15.78:0/249722589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 msgr2=0x56236f06af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 --2- 172.21.15.78:0/249722589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f06af00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f85240047e0 tx=0x7f852400c830 comp rx=0 tx=0).stop 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 -- 172.21.15.78:0/249722589 shutdown_connections 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 --2- 172.21.15.78:0/249722589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f069190 0x56236f06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 --2- 172.21.15.78:0/249722589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f06af00 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 --2- 172.21.15.78:0/249722589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f0584d0 0x56236f065340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 -- 172.21.15.78:0/249722589 >> 172.21.15.78:0/249722589 conn(0x56236f056300 msgr2=0x56236f0646d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 -- 172.21.15.78:0/249722589 shutdown_connections 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 -- 172.21.15.78:0/249722589 wait complete. 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 -- start start 2024-09-11T01:02:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 Processor -- start 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f07ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f07a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f069190 0x56236f07a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85341ad8b0 con 0x56236f058df0 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85341ad120 con 0x56236f0584d0 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85341aa360 con 0x56236f069190 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f853deb2740 10 client.0 _ll_register_callbacks cb 0x7f852400a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.727+0000 7f8539078640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f07a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f8539078640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f07a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39946/0 (socket says 172.21.15.78:39946) 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f8539078640 1 -- 172.21.15.78:0/2166949128 learned_addr learned my addr 172.21.15.78:0/2166949128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f853a07a640 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f069190 0x56236f07a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f8539879640 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f07ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f8539078640 1 -- 172.21.15.78:0/2166949128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f069190 msgr2=0x56236f07a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f8539078640 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f069190 0x56236f07a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f8539078640 1 -- 172.21.15.78:0/2166949128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 msgr2=0x56236f07ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f8539078640 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f07ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f8539078640 1 -- 172.21.15.78:0/2166949128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236f0e4960 con 0x56236f058df0 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f8539078640 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f07a160 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8524005610 tx=0x7f8524005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339cd0942a10bf05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f851d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f851d7fa640 1 -- 172.21.15.78:0/2166949128 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852401d070 con 0x56236f058df0 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f851d7fa640 1 -- 172.21.15.78:0/2166949128 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f8524004230 con 0x56236f058df0 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f853deb2740 1 -- 172.21.15.78:0/2166949128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8524003cc0 con 0x56236f058df0 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f851d7fa640 1 -- 172.21.15.78:0/2166949128 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8524028040 con 0x56236f058df0 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f853deb2740 10 client.16191 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f853deb2740 1 -- 172.21.15.78:0/2166949128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56236f0e5c60 con 0x56236f058df0 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f851d7fa640 1 -- 172.21.15.78:0/2166949128 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8524003cc0 con 0x56236f058df0 2024-09-11T01:02:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.728+0000 7f851d7fa640 1 -- 172.21.15.78:0/2166949128 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f852400f9e0 con 0x56236f058df0 2024-09-11T01:02:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.729+0000 7f853deb2740 10 client.16191 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.729+0000 7f853deb2740 10 client.16191 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:02:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.729+0000 7f853deb2740 20 client.16191 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.729+0000 7f853deb2740 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56236f0e6310 0x56236f0e8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.729+0000 7f853deb2740 4 client.16191 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:02:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.729+0000 7f853deb2740 1 -- 172.21.15.78:0/2166949128 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56236efcc8a0 con 0x56236f0e6310 2024-09-11T01:02:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.729+0000 7f8539879640 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56236f0e6310 0x56236f0e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.729+0000 7f8539879640 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56236f0e6310 0x56236f0e8730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e1c35bfbba8e02e9 server_cookie=70514b7a1120fab8 in_seq=0 out_seq=0 2024-09-11T01:02:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.729+0000 7f851d7fa640 10 client.16191 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:02:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.729+0000 7f851d7fa640 1 -- 172.21.15.78:0/2166949128 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56236efcc8a0 con 0x56236f0e6310 2024-09-11T01:02:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.729+0000 7f851d7fa640 10 client.16191 handle_command_reply: tid=0 2024-09-11T01:02:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.729+0000 7f853deb2740 1 client.16191 shutdown 2024-09-11T01:02:43.822 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:02:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853deb2740 1 -- 172.21.15.78:0/2166949128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 msgr2=0x56236f07a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853deb2740 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f07a160 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8524005610 tx=0x7f8524005640 comp rx=0 tx=0).stop 2024-09-11T01:02:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853deb2740 1 -- 172.21.15.78:0/2166949128 shutdown_connections 2024-09-11T01:02:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853deb2740 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56236f0e6310 0x56236f0e8730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853deb2740 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56236f069190 0x56236f07a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853deb2740 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56236f058df0 0x56236f07a160 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f851d7fa640 0 client.16191 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853deb2740 1 --2- 172.21.15.78:0/2166949128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56236f0584d0 0x56236f07ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853deb2740 1 -- 172.21.15.78:0/2166949128 >> 172.21.15.78:0/2166949128 conn(0x56236f056300 msgr2=0x56236f064690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853deb2740 1 -- 172.21.15.78:0/2166949128 shutdown_connections 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853deb2740 1 -- 172.21.15.78:0/2166949128 wait complete. 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853deb2740 20 client.16191 trim_cache size 0 max 16384 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853badd640 1 -- 172.21.15.78:0/70315560 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f85200b54d0 msgr2=0x7f85200b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853badd640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f85200b54d0 0x7f85200b7990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f852c00af20 tx=0x7f852c00a900 comp rx=0 tx=0).stop 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853badd640 1 -- 172.21.15.78:0/70315560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534103ca0 msgr2=0x7f8534199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853badd640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534103ca0 0x7f8534199300 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8530006010 tx=0x7f85300042f0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853badd640 1 -- 172.21.15.78:0/70315560 shutdown_connections 2024-09-11T01:02:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853badd640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f85200b54d0 0x7f85200b7990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853badd640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853410d100 0x7f853419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853badd640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85341045c0 0x7f8534199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853badd640 1 --2- 172.21.15.78:0/70315560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8534103ca0 0x7f8534199300 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.730+0000 7f853badd640 1 -- 172.21.15.78:0/70315560 >> 172.21.15.78:0/70315560 conn(0x7f85340fd700 msgr2=0x7f85340fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.731+0000 7f853badd640 1 -- 172.21.15.78:0/70315560 shutdown_connections 2024-09-11T01:02:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:43.731+0000 7f853badd640 1 -- 172.21.15.78:0/70315560 wait complete. 2024-09-11T01:02:44.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 337080 8.33 MB/sec execute 52 sec latency 3322.842 ms 2024-09-11T01:02:44.896 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:02:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:45.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:45 smithi078 ceph-mon[31315]: pgmap v439: 97 pgs: 97 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-09-11T01:02:45.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 337131 8.17 MB/sec execute 53 sec latency 952.284 ms 2024-09-11T01:02:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:45 smithi110 ceph-mon[34712]: pgmap v439: 97 pgs: 97 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-09-11T01:02:45.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:45 smithi112 ceph-mon[35384]: pgmap v439: 97 pgs: 97 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-09-11T01:02:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.635+0000 7fa915901640 1 -- 172.21.15.78:0/500430260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101076a0 msgr2=0x7fa910107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.635+0000 7fa915901640 1 --2- 172.21.15.78:0/500430260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101076a0 0x7fa910107ae0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009ca0 tx=0x7fa8f802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.635+0000 7fa915901640 1 -- 172.21.15.78:0/500430260 shutdown_connections 2024-09-11T01:02:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.635+0000 7fa915901640 1 --2- 172.21.15.78:0/500430260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa910108020 0x7fa91010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.635+0000 7fa915901640 1 --2- 172.21.15.78:0/500430260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101076a0 0x7fa910107ae0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.635+0000 7fa915901640 1 --2- 172.21.15.78:0/500430260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa910101c90 0x7fa910102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.635+0000 7fa915901640 1 -- 172.21.15.78:0/500430260 >> 172.21.15.78:0/500430260 conn(0x7fa9100fd6c0 msgr2=0x7fa9100ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.635+0000 7fa915901640 1 -- 172.21.15.78:0/500430260 shutdown_connections 2024-09-11T01:02:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.635+0000 7fa915901640 1 -- 172.21.15.78:0/500430260 wait complete. 2024-09-11T01:02:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.636+0000 7fa915901640 1 Processor -- start 2024-09-11T01:02:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.636+0000 7fa915901640 1 -- start start 2024-09-11T01:02:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.636+0000 7fa915901640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa910101c90 0x7fa91019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.636+0000 7fa915901640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9101076a0 0x7fa91019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.636+0000 7fa915901640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa910108020 0x7fa91019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.636+0000 7fa915901640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa910111ce0 con 0x7fa910108020 2024-09-11T01:02:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.636+0000 7fa915901640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa910111b60 con 0x7fa910101c90 2024-09-11T01:02:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.636+0000 7fa915901640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa910111e60 con 0x7fa9101076a0 2024-09-11T01:02:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.636+0000 7fa90f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa910108020 0x7fa91019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.636+0000 7fa90f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa910108020 0x7fa91019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39970/0 (socket says 172.21.15.78:39970) 2024-09-11T01:02:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.636+0000 7fa90f7fe640 1 -- 172.21.15.78:0/1298983513 learned_addr learned my addr 172.21.15.78:0/1298983513 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa90effd640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa910101c90 0x7fa91019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa90e7fc640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9101076a0 0x7fa91019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa90f7fe640 1 -- 172.21.15.78:0/1298983513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9101076a0 msgr2=0x7fa91019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa90f7fe640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9101076a0 0x7fa91019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa90f7fe640 1 -- 172.21.15.78:0/1298983513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa910101c90 msgr2=0x7fa91019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa90f7fe640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa910101c90 0x7fa91019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa90f7fe640 1 -- 172.21.15.78:0/1298983513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9101a5450 con 0x7fa910108020 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa90f7fe640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa910108020 0x7fa91019ab10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c970 tx=0x7fa90000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa9148ff640 1 -- 172.21.15.78:0/1298983513 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900007bb0 con 0x7fa910108020 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa915901640 1 -- 172.21.15.78:0/1298983513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9101a5740 con 0x7fa910108020 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa9148ff640 1 -- 172.21.15.78:0/1298983513 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa900007d50 con 0x7fa910108020 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa9148ff640 1 -- 172.21.15.78:0/1298983513 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900018660 con 0x7fa910108020 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.637+0000 7fa915901640 1 -- 172.21.15.78:0/1298983513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9101a5cb0 con 0x7fa910108020 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.640+0000 7fa917cd6740 1 Processor -- start 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.640+0000 7fa9148ff640 1 -- 172.21.15.78:0/1298983513 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa90000f450 con 0x7fa910108020 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.640+0000 7fa917cd6740 1 -- start start 2024-09-11T01:02:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.640+0000 7fa917cd6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1e4d0 0x5610e7b1e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.640+0000 7fa917cd6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1edf0 0x5610e7b2a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa917cd6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b2ad80 0x5610e7b2d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa917cd6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5610e7a94ea0 con 0x5610e7b1edf0 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa917cd6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5610e7a842d0 con 0x5610e7b1e4d0 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa917cd6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5610e7a880e0 con 0x5610e7b2ad80 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1edf0 0x5610e7b2a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1edf0 0x5610e7b2a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39986/0 (socket says 172.21.15.78:39986) 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90effd640 1 -- 172.21.15.78:0/3262882833 learned_addr learned my addr 172.21.15.78:0/3262882833 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90e7fc640 1 --2- 172.21.15.78:0/3262882833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1e4d0 0x5610e7b1e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90e7fc640 1 --2- 172.21.15.78:0/3262882833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b2ad80 0x5610e7b2d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90effd640 1 -- 172.21.15.78:0/3262882833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b2ad80 msgr2=0x5610e7b2d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90effd640 1 --2- 172.21.15.78:0/3262882833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b2ad80 0x5610e7b2d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90effd640 1 -- 172.21.15.78:0/3262882833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1e4d0 msgr2=0x5610e7b1e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:45.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90effd640 1 --2- 172.21.15.78:0/3262882833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1e4d0 0x5610e7b1e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90effd640 1 -- 172.21.15.78:0/3262882833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e77e2650 con 0x5610e7b1edf0 2024-09-11T01:02:45.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90e7fc640 1 --2- 172.21.15.78:0/3262882833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b2ad80 0x5610e7b2d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:02:45.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90effd640 1 --2- 172.21.15.78:0/3262882833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1edf0 0x5610e7b2a840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa908009a70 tx=0x7fa908009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266d7c9c226a83cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:45.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa9148ff640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa8fc0b5450 0x7fa8fc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa905ffb640 1 -- 172.21.15.78:0/3262882833 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908017070 con 0x5610e7b1edf0 2024-09-11T01:02:45.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa905ffb640 1 -- 172.21.15.78:0/3262882833 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa908013470 con 0x5610e7b1edf0 2024-09-11T01:02:45.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa905ffb640 1 -- 172.21.15.78:0/3262882833 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908012630 con 0x5610e7b1edf0 2024-09-11T01:02:45.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa90e7fc640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa8fc0b5450 0x7fa8fc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.641+0000 7fa9148ff640 1 -- 172.21.15.78:0/1298983513 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fa9000e6080 con 0x7fa910108020 2024-09-11T01:02:45.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.642+0000 7fa917cd6740 1 -- 172.21.15.78:0/3262882833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1edf0 msgr2=0x5610e7b2a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:45.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.642+0000 7fa90e7fc640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa8fc0b5450 0x7fa8fc0b7910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa91006c0f0 tx=0x7fa8f802a8c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:45.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.642+0000 7fa917cd6740 1 --2- 172.21.15.78:0/3262882833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1edf0 0x5610e7b2a840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa908009a70 tx=0x7fa908009f30 comp rx=0 tx=0).stop 2024-09-11T01:02:45.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.642+0000 7fa917cd6740 1 -- 172.21.15.78:0/3262882833 shutdown_connections 2024-09-11T01:02:45.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.642+0000 7fa917cd6740 1 --2- 172.21.15.78:0/3262882833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b2ad80 0x5610e7b2d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.642+0000 7fa917cd6740 1 --2- 172.21.15.78:0/3262882833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1edf0 0x5610e7b2a840 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.642+0000 7fa917cd6740 1 --2- 172.21.15.78:0/3262882833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1e4d0 0x5610e7b1e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.642+0000 7fa917cd6740 1 -- 172.21.15.78:0/3262882833 >> 172.21.15.78:0/3262882833 conn(0x5610e7b1c300 msgr2=0x5610e7b1d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.642+0000 7fa917cd6740 1 -- 172.21.15.78:0/3262882833 shutdown_connections 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.642+0000 7fa917cd6740 1 -- 172.21.15.78:0/3262882833 wait complete. 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.642+0000 7fa917cd6740 1 -- start start 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa917cd6740 1 Processor -- start 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa917cd6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1e4d0 0x5610e7b2a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa917cd6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1edf0 0x5610e7b27ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa917cd6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 0x5610e7b28110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa917cd6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5610e78e5700 con 0x5610e7b2ad80 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa917cd6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5610e7a1f160 con 0x5610e7b1edf0 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa917cd6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5610e7b2da00 con 0x5610e7b1e4d0 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa917cd6740 10 client.0 _ll_register_callbacks cb 0x5610e7a389e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa90f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 0x5610e7b28110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa90f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 0x5610e7b28110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39990/0 (socket says 172.21.15.78:39990) 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa90f7fe640 1 -- 172.21.15.78:0/52335345 learned_addr learned my addr 172.21.15.78:0/52335345 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa90f7fe640 1 -- 172.21.15.78:0/52335345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1e4d0 msgr2=0x5610e7b2a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa90f7fe640 1 --2- 172.21.15.78:0/52335345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1e4d0 0x5610e7b2a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa90f7fe640 1 -- 172.21.15.78:0/52335345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1edf0 msgr2=0x5610e7b27ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa90f7fe640 1 --2- 172.21.15.78:0/52335345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1edf0 0x5610e7b27ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.643+0000 7fa90f7fe640 1 -- 172.21.15.78:0/52335345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e7a4cc90 con 0x5610e7b2ad80 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa90f7fe640 1 --2- 172.21.15.78:0/52335345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 0x5610e7b28110 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa9000ac500 tx=0x7fa9000ac340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1caec50bb9a519 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa8eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa8eeffd640 1 -- 172.21.15.78:0/52335345 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90000f770 con 0x5610e7b2ad80 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa8eeffd640 1 -- 172.21.15.78:0/52335345 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fa9000ab7d0 con 0x5610e7b2ad80 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa917cd6740 1 -- 172.21.15.78:0/52335345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa908013470 con 0x5610e7b2ad80 2024-09-11T01:02:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa8eeffd640 1 -- 172.21.15.78:0/52335345 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa9000ac080 con 0x5610e7b2ad80 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa917cd6740 10 client.16203 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa917cd6740 1 -- 172.21.15.78:0/52335345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5610e7b392d0 con 0x5610e7b2ad80 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa8eeffd640 1 -- 172.21.15.78:0/52335345 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa908013470 con 0x5610e7b2ad80 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa8eeffd640 1 -- 172.21.15.78:0/52335345 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fa9000d3170 con 0x5610e7b2ad80 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa917cd6740 10 client.16203 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa917cd6740 10 client.16203 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa917cd6740 20 client.16203 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa917cd6740 1 --2- 172.21.15.78:0/52335345 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5610e7b3a730 0x5610e7b3cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa917cd6740 4 client.16203 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa917cd6740 1 -- 172.21.15.78:0/52335345 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610e7a96df0 con 0x5610e7b3a730 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.644+0000 7fa90effd640 1 --2- 172.21.15.78:0/52335345 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5610e7b3a730 0x5610e7b3cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.645+0000 7fa90effd640 1 --2- 172.21.15.78:0/52335345 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5610e7b3a730 0x5610e7b3cba0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea3399e79aafb5c9 server_cookie=a8ab25185576bb23 in_seq=0 out_seq=0 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.645+0000 7fa8eeffd640 10 client.16203 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:02:45.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.646+0000 7fa8eeffd640 1 -- 172.21.15.78:0/52335345 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5610e7a96df0 con 0x5610e7b3a730 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.646+0000 7fa8eeffd640 10 client.16203 handle_command_reply: tid=0 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.646+0000 7fa917cd6740 1 client.16203 shutdown 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.646+0000 7fa917cd6740 1 -- 172.21.15.78:0/52335345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 msgr2=0x5610e7b28110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.646+0000 7fa917cd6740 1 --2- 172.21.15.78:0/52335345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 0x5610e7b28110 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa9000ac500 tx=0x7fa9000ac340 comp rx=0 tx=0).stop 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.647+0000 7fa917cd6740 1 -- 172.21.15.78:0/52335345 shutdown_connections 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.647+0000 7fa917cd6740 1 --2- 172.21.15.78:0/52335345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 0x5610e7b28110 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.647+0000 7fa917cd6740 1 --2- 172.21.15.78:0/52335345 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5610e7b3a730 0x5610e7b3cba0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.647+0000 7fa917cd6740 1 --2- 172.21.15.78:0/52335345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1edf0 0x5610e7b27ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.647+0000 7fa917cd6740 1 --2- 172.21.15.78:0/52335345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1e4d0 0x5610e7b2a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.647+0000 7fa8eeffd640 0 client.16203 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.647+0000 7fa917cd6740 1 -- 172.21.15.78:0/52335345 >> 172.21.15.78:0/52335345 conn(0x5610e7b1c300 msgr2=0x5610e7b21ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.647+0000 7fa917cd6740 1 -- 172.21.15.78:0/52335345 shutdown_connections 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.647+0000 7fa917cd6740 1 -- 172.21.15.78:0/52335345 wait complete. 2024-09-11T01:02:45.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.647+0000 7fa917cd6740 20 client.16203 trim_cache size 0 max 16384 2024-09-11T01:02:45.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa917cd6740 1 Processor -- start 2024-09-11T01:02:45.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa917cd6740 1 -- start start 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa917cd6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1e4d0 0x5610e7b36c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa917cd6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1edf0 0x5610e7b1b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa917cd6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 0x5610e7b2c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa917cd6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101ae4c0 con 0x5610e7b2ad80 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa917cd6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101b0180 con 0x5610e7b1edf0 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa917cd6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101aeb90 con 0x5610e7b1e4d0 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa90f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 0x5610e7b2c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa90f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 0x5610e7b2c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40006/0 (socket says 172.21.15.78:40006) 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa90f7fe640 1 -- 172.21.15.78:0/3966381711 learned_addr learned my addr 172.21.15.78:0/3966381711 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa90e7fc640 1 --2- 172.21.15.78:0/3966381711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1edf0 0x5610e7b1b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa90f7fe640 1 -- 172.21.15.78:0/3966381711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1e4d0 msgr2=0x5610e7b36c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa90f7fe640 1 --2- 172.21.15.78:0/3966381711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1e4d0 0x5610e7b36c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa90f7fe640 1 -- 172.21.15.78:0/3966381711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1edf0 msgr2=0x5610e7b1b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:45.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa90f7fe640 1 --2- 172.21.15.78:0/3966381711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1edf0 0x5610e7b1b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa90f7fe640 1 -- 172.21.15.78:0/3966381711 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e7b1ec20 con 0x5610e7b2ad80 2024-09-11T01:02:45.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.660+0000 7fa90e7fc640 1 --2- 172.21.15.78:0/3966381711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1edf0 0x5610e7b1b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:45.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.661+0000 7fa90f7fe640 1 --2- 172.21.15.78:0/3966381711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 0x5610e7b2c170 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa90001f000 tx=0x7fa90000a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12cceaea4b9ccde server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:45.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.661+0000 7fa9057fa640 1 -- 172.21.15.78:0/3966381711 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9000abec0 con 0x5610e7b2ad80 2024-09-11T01:02:45.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.661+0000 7fa9057fa640 1 -- 172.21.15.78:0/3966381711 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9000acc70 con 0x5610e7b2ad80 2024-09-11T01:02:45.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.661+0000 7fa9057fa640 1 -- 172.21.15.78:0/3966381711 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9000ab7d0 con 0x5610e7b2ad80 2024-09-11T01:02:45.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.661+0000 7fa917cd6740 1 -- 172.21.15.78:0/3966381711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 msgr2=0x5610e7b2c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:45.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.661+0000 7fa917cd6740 1 --2- 172.21.15.78:0/3966381711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 0x5610e7b2c170 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa90001f000 tx=0x7fa90000a5b0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.661+0000 7fa917cd6740 1 -- 172.21.15.78:0/3966381711 shutdown_connections 2024-09-11T01:02:45.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.661+0000 7fa917cd6740 1 --2- 172.21.15.78:0/3966381711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b2ad80 0x5610e7b2c170 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.661+0000 7fa917cd6740 1 --2- 172.21.15.78:0/3966381711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b1edf0 0x5610e7b1b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.661+0000 7fa917cd6740 1 --2- 172.21.15.78:0/3966381711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1e4d0 0x5610e7b36c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.661+0000 7fa917cd6740 1 -- 172.21.15.78:0/3966381711 >> 172.21.15.78:0/3966381711 conn(0x5610e7b1c300 msgr2=0x5610e7b1ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:45.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.661+0000 7fa917cd6740 1 -- 172.21.15.78:0/3966381711 shutdown_connections 2024-09-11T01:02:45.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa917cd6740 1 -- 172.21.15.78:0/3966381711 wait complete. 2024-09-11T01:02:45.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa917cd6740 1 -- start start 2024-09-11T01:02:45.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa917cd6740 1 Processor -- start 2024-09-11T01:02:45.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa917cd6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1e4d0 0x5610e7b32e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa917cd6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1edf0 0x5610e7b33400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa917cd6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b2ad80 0x5610e7b2dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa917cd6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101af9f0 con 0x5610e7b1e4d0 2024-09-11T01:02:45.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa917cd6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101af260 con 0x5610e7b2ad80 2024-09-11T01:02:45.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa917cd6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101ac4a0 con 0x5610e7b1edf0 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa917cd6740 10 client.0 _ll_register_callbacks cb 0x7fa90000af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa90effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1e4d0 0x5610e7b32e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa90effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1e4d0 0x5610e7b32e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40012/0 (socket says 172.21.15.78:40012) 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.662+0000 7fa90effd640 1 -- 172.21.15.78:0/2957049816 learned_addr learned my addr 172.21.15.78:0/2957049816 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa90f7fe640 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b2ad80 0x5610e7b2dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa90effd640 1 -- 172.21.15.78:0/2957049816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1edf0 msgr2=0x5610e7b33400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa90e7fc640 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1edf0 0x5610e7b33400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa90effd640 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1edf0 0x5610e7b33400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa90effd640 1 -- 172.21.15.78:0/2957049816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b2ad80 msgr2=0x5610e7b2dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa90effd640 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b2ad80 0x5610e7b2dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa90effd640 1 -- 172.21.15.78:0/2957049816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e7b2aa00 con 0x5610e7b1e4d0 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa90f7fe640 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b2ad80 0x5610e7b2dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:02:45.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa90effd640 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1e4d0 0x5610e7b32e80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa9080099c0 tx=0x7fa9080047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9acc294539a71c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa8eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa8eeffd640 1 -- 172.21.15.78:0/2957049816 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908028060 con 0x5610e7b1e4d0 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa8eeffd640 1 -- 172.21.15.78:0/2957049816 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fa908013470 con 0x5610e7b1e4d0 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa917cd6740 1 -- 172.21.15.78:0/2957049816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9000acc70 con 0x5610e7b1e4d0 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa8eeffd640 1 -- 172.21.15.78:0/2957049816 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa908017040 con 0x5610e7b1e4d0 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa917cd6740 10 client.16209 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa917cd6740 1 -- 172.21.15.78:0/2957049816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5610e7ba7f50 con 0x5610e7b1e4d0 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.663+0000 7fa8eeffd640 1 -- 172.21.15.78:0/2957049816 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9000acc70 con 0x5610e7b1e4d0 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.664+0000 7fa8eeffd640 1 -- 172.21.15.78:0/2957049816 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fa90802b020 con 0x5610e7b1e4d0 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.664+0000 7fa917cd6740 10 client.16209 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.664+0000 7fa917cd6740 10 client.16209 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.664+0000 7fa917cd6740 20 client.16209 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:45.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.664+0000 7fa917cd6740 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5610e7ba83a0 0x5610e7baa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.664+0000 7fa917cd6740 4 client.16209 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.664+0000 7fa917cd6740 1 -- 172.21.15.78:0/2957049816 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610e7a928a0 con 0x5610e7ba83a0 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.664+0000 7fa90e7fc640 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5610e7ba83a0 0x5610e7baa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.664+0000 7fa90e7fc640 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5610e7ba83a0 0x5610e7baa7c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a193f5fceba8036a server_cookie=fc77a8fd361466e2 in_seq=0 out_seq=0 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.664+0000 7fa8eeffd640 10 client.16209 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa8eeffd640 1 -- 172.21.15.78:0/2957049816 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610e7a928a0 con 0x5610e7ba83a0 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa8eeffd640 10 client.16209 handle_command_reply: tid=0 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa917cd6740 1 client.16209 shutdown 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa917cd6740 1 -- 172.21.15.78:0/2957049816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1e4d0 msgr2=0x5610e7b32e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa917cd6740 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1e4d0 0x5610e7b32e80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa9080099c0 tx=0x7fa9080047d0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa917cd6740 1 -- 172.21.15.78:0/2957049816 shutdown_connections 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa917cd6740 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5610e7ba83a0 0x5610e7baa7c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa917cd6740 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5610e7b2ad80 0x5610e7b2dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa917cd6740 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5610e7b1edf0 0x5610e7b33400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa917cd6740 1 --2- 172.21.15.78:0/2957049816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5610e7b1e4d0 0x5610e7b32e80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa8eeffd640 0 client.16209 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:02:45.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa917cd6740 1 -- 172.21.15.78:0/2957049816 >> 172.21.15.78:0/2957049816 conn(0x5610e7b1c300 msgr2=0x5610e7b1b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:45.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa917cd6740 1 -- 172.21.15.78:0/2957049816 shutdown_connections 2024-09-11T01:02:45.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa917cd6740 1 -- 172.21.15.78:0/2957049816 wait complete. 2024-09-11T01:02:45.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa917cd6740 20 client.16209 trim_cache size 0 max 16384 2024-09-11T01:02:45.673 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:02:45.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa915901640 1 -- 172.21.15.78:0/1298983513 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa8fc0b5450 msgr2=0x7fa8fc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:45.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa915901640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa8fc0b5450 0x7fa8fc0b7910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa91006c0f0 tx=0x7fa8f802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa915901640 1 -- 172.21.15.78:0/1298983513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa910108020 msgr2=0x7fa91019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:45.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa915901640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa910108020 0x7fa91019ab10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c970 tx=0x7fa90000cd70 comp rx=0 tx=0).stop 2024-09-11T01:02:45.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa915901640 1 -- 172.21.15.78:0/1298983513 shutdown_connections 2024-09-11T01:02:45.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa915901640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa8fc0b5450 0x7fa8fc0b7910 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa915901640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa910108020 0x7fa91019ab10 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa915901640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9101076a0 0x7fa91019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa915901640 1 --2- 172.21.15.78:0/1298983513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa910101c90 0x7fa91019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:45.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.665+0000 7fa915901640 1 -- 172.21.15.78:0/1298983513 >> 172.21.15.78:0/1298983513 conn(0x7fa9100fd6c0 msgr2=0x7fa9100ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:45.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.666+0000 7fa915901640 1 -- 172.21.15.78:0/1298983513 shutdown_connections 2024-09-11T01:02:45.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:45.666+0000 7fa915901640 1 -- 172.21.15.78:0/1298983513 wait complete. 2024-09-11T01:02:46.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 337131 8.02 MB/sec execute 54 sec latency 1952.357 ms 2024-09-11T01:02:46.599 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:02:46.868 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:47.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:47 smithi078 ceph-mon[31315]: pgmap v440: 97 pgs: 97 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.9 MiB/s wr, 121 op/s 2024-09-11T01:02:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.347+0000 7f34394e4640 1 -- 172.21.15.78:0/2653102176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34341076b0 msgr2=0x7f3434107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.347+0000 7f34394e4640 1 --2- 172.21.15.78:0/2653102176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34341076b0 0x7f3434107af0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f341c009900 tx=0x7f341c02a410 comp rx=0 tx=0).stop 2024-09-11T01:02:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.347+0000 7f34394e4640 1 -- 172.21.15.78:0/2653102176 shutdown_connections 2024-09-11T01:02:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.347+0000 7f34394e4640 1 --2- 172.21.15.78:0/2653102176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3434108030 0x7f343410e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.347+0000 7f34394e4640 1 --2- 172.21.15.78:0/2653102176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34341076b0 0x7f3434107af0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.347+0000 7f34394e4640 1 --2- 172.21.15.78:0/2653102176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3434101ca0 0x7f3434102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.347+0000 7f34394e4640 1 -- 172.21.15.78:0/2653102176 >> 172.21.15.78:0/2653102176 conn(0x7f34340fd730 msgr2=0x7f34340ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.347+0000 7f34394e4640 1 -- 172.21.15.78:0/2653102176 shutdown_connections 2024-09-11T01:02:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.347+0000 7f34394e4640 1 -- 172.21.15.78:0/2653102176 wait complete. 2024-09-11T01:02:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.347+0000 7f34394e4640 1 Processor -- start 2024-09-11T01:02:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.347+0000 7f34394e4640 1 -- start start 2024-09-11T01:02:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f34394e4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3434101ca0 0x7f3434198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f34394e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34341076b0 0x7f3434198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f34394e4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3434108030 0x7f343419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f34394e4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3434111d30 con 0x7f3434101ca0 2024-09-11T01:02:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f34394e4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3434111bb0 con 0x7f3434108030 2024-09-11T01:02:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f34394e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3434111eb0 con 0x7f34341076b0 2024-09-11T01:02:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f3432ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3434101ca0 0x7f3434198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f3432ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3434101ca0 0x7f3434198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40038/0 (socket says 172.21.15.78:40038) 2024-09-11T01:02:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f3432ffd640 1 -- 172.21.15.78:0/2054649080 learned_addr learned my addr 172.21.15.78:0/2054649080 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f342a7fc640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34341076b0 0x7f3434198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f34337fe640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3434108030 0x7f343419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f3432ffd640 1 -- 172.21.15.78:0/2054649080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34341076b0 msgr2=0x7f3434198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f3432ffd640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34341076b0 0x7f3434198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f3432ffd640 1 -- 172.21.15.78:0/2054649080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3434108030 msgr2=0x7f343419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f3432ffd640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3434108030 0x7f343419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f3432ffd640 1 -- 172.21.15.78:0/2054649080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343419e3b0 con 0x7f3434101ca0 2024-09-11T01:02:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f34337fe640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3434108030 0x7f343419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f3432ffd640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3434101ca0 0x7f3434198300 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f342400b4a0 tx=0x7f342400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f3430ff9640 1 -- 172.21.15.78:0/2054649080 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424014070 con 0x7f3434101ca0 2024-09-11T01:02:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f34394e4640 1 -- 172.21.15.78:0/2054649080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343419e6a0 con 0x7f3434101ca0 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f3430ff9640 1 -- 172.21.15.78:0/2054649080 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3424010ac0 con 0x7f3434101ca0 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f3430ff9640 1 -- 172.21.15.78:0/2054649080 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424017c30 con 0x7f3434101ca0 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.348+0000 7f34394e4640 1 -- 172.21.15.78:0/2054649080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34341a4f30 con 0x7f3434101ca0 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.351+0000 7f3430ff9640 1 -- 172.21.15.78:0/2054649080 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3424010610 con 0x7f3434101ca0 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.351+0000 7f343b8b9740 1 Processor -- start 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.351+0000 7f343b8b9740 1 -- start start 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.351+0000 7f343b8b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc30f0 0x56531cfc34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.351+0000 7f343b8b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc3a10 0x56531cfd1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.351+0000 7f343b8b9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfd3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.351+0000 7f343b8b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56531cf39ea0 con 0x56531cfc30f0 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.351+0000 7f343b8b9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56531cf292d0 con 0x56531cfd1bb0 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.351+0000 7f343b8b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56531cf2d0e0 con 0x56531cfc3a10 2024-09-11T01:02:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.351+0000 7f34337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc30f0 0x56531cfc34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.351+0000 7f34337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc30f0 0x56531cfc34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40054/0 (socket says 172.21.15.78:40054) 2024-09-11T01:02:47.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.351+0000 7f34337fe640 1 -- 172.21.15.78:0/3514995098 learned_addr learned my addr 172.21.15.78:0/3514995098 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:47.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f342a7fc640 1 --2- 172.21.15.78:0/3514995098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfd3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f3432ffd640 1 --2- 172.21.15.78:0/3514995098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc3a10 0x56531cfd1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f34337fe640 1 -- 172.21.15.78:0/3514995098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc3a10 msgr2=0x56531cfd1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:47.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f34337fe640 1 --2- 172.21.15.78:0/3514995098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc3a10 0x56531cfd1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f34337fe640 1 -- 172.21.15.78:0/3514995098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 msgr2=0x56531cfd3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:47.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f34337fe640 1 --2- 172.21.15.78:0/3514995098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfd3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f3430ff9640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f34200b5450 0x7f34200b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f34337fe640 1 -- 172.21.15.78:0/3514995098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56531cc87650 con 0x56531cfc30f0 2024-09-11T01:02:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f342a7fc640 1 --2- 172.21.15.78:0/3514995098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfd3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f3432ffd640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f34200b5450 0x7f34200b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f34337fe640 1 --2- 172.21.15.78:0/3514995098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc30f0 0x56531cfc34d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f342c00c450 tx=0x7f342c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8bb1bdda586c97 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f3430ff9640 1 -- 172.21.15.78:0/2054649080 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f34240e5890 con 0x7f3434101ca0 2024-09-11T01:02:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f34297fa640 1 -- 172.21.15.78:0/3514995098 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c007430 con 0x56531cfc30f0 2024-09-11T01:02:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f34297fa640 1 -- 172.21.15.78:0/3514995098 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f342c007a80 con 0x56531cfc30f0 2024-09-11T01:02:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.352+0000 7f34297fa640 1 -- 172.21.15.78:0/3514995098 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c00aa00 con 0x56531cfc30f0 2024-09-11T01:02:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.353+0000 7f343b8b9740 1 -- 172.21.15.78:0/3514995098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc30f0 msgr2=0x56531cfc34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.353+0000 7f343b8b9740 1 --2- 172.21.15.78:0/3514995098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc30f0 0x56531cfc34d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f342c00c450 tx=0x7f342c00c910 comp rx=0 tx=0).stop 2024-09-11T01:02:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.353+0000 7f3432ffd640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f34200b5450 0x7f34200b7910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f34240e3100 tx=0x7f3424010f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.353+0000 7f343b8b9740 1 -- 172.21.15.78:0/3514995098 shutdown_connections 2024-09-11T01:02:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.353+0000 7f343b8b9740 1 --2- 172.21.15.78:0/3514995098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfd3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.353+0000 7f343b8b9740 1 --2- 172.21.15.78:0/3514995098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc3a10 0x56531cfd1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.353+0000 7f343b8b9740 1 --2- 172.21.15.78:0/3514995098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc30f0 0x56531cfc34d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.353+0000 7f343b8b9740 1 -- 172.21.15.78:0/3514995098 >> 172.21.15.78:0/3514995098 conn(0x56531cfc0f20 msgr2=0x56531cfc20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.353+0000 7f343b8b9740 1 -- 172.21.15.78:0/3514995098 shutdown_connections 2024-09-11T01:02:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.353+0000 7f343b8b9740 1 -- 172.21.15.78:0/3514995098 wait complete. 2024-09-11T01:02:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.353+0000 7f343b8b9740 1 -- start start 2024-09-11T01:02:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.353+0000 7f343b8b9740 1 Processor -- start 2024-09-11T01:02:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f343b8b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfd1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f343b8b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f343b8b9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f343b8b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56531cd8a700 con 0x56531cfc3a10 2024-09-11T01:02:47.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f343b8b9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56531cec4160 con 0x56531cfd1bb0 2024-09-11T01:02:47.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f343b8b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56531cfd4860 con 0x56531cfc30f0 2024-09-11T01:02:47.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f343b8b9740 10 client.0 _ll_register_callbacks cb 0x56531cedd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:47.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f342a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f342a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfce900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40068/0 (socket says 172.21.15.78:40068) 2024-09-11T01:02:47.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f342a7fc640 1 -- 172.21.15.78:0/2585940885 learned_addr learned my addr 172.21.15.78:0/2585940885 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:47.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f34337fe640 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfd1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f34337fe640 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f342a7fc640 1 -- 172.21.15.78:0/2585940885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 msgr2=0x56531cfd1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:47.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f342a7fc640 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfd1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f342a7fc640 1 -- 172.21.15.78:0/2585940885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 msgr2=0x56531cfceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:47.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f342a7fc640 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f342a7fc640 1 -- 172.21.15.78:0/2585940885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56531cef1c90 con 0x56531cfc3a10 2024-09-11T01:02:47.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f34337fe640 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:02:47.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f342a7fc640 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfce900 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f341c009900 tx=0x7f341c0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500b5a9ff2a96af4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:47.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f341a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:47.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f341a7fc640 1 -- 172.21.15.78:0/2585940885 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c035040 con 0x56531cfc3a10 2024-09-11T01:02:47.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f343b8b9740 1 -- 172.21.15.78:0/2585940885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f342c007a80 con 0x56531cfc3a10 2024-09-11T01:02:47.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.354+0000 7f341a7fc640 1 -- 172.21.15.78:0/2585940885 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f341c004040 con 0x56531cfc3a10 2024-09-11T01:02:47.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.355+0000 7f341a7fc640 1 -- 172.21.15.78:0/2585940885 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f341c004240 con 0x56531cfc3a10 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.355+0000 7f343b8b9740 10 client.16221 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.355+0000 7f343b8b9740 1 -- 172.21.15.78:0/2585940885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56531cfe01a0 con 0x56531cfc3a10 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.355+0000 7f341a7fc640 1 -- 172.21.15.78:0/2585940885 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f342c007a80 con 0x56531cfc3a10 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.355+0000 7f341a7fc640 1 -- 172.21.15.78:0/2585940885 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f341c04d050 con 0x56531cfc3a10 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.355+0000 7f343b8b9740 10 client.16221 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.355+0000 7f343b8b9740 10 client.16221 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.355+0000 7f343b8b9740 20 client.16221 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.355+0000 7f343b8b9740 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56531cfe1670 0x56531cfe3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.355+0000 7f343b8b9740 4 client.16221 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.355+0000 7f343b8b9740 1 -- 172.21.15.78:0/2585940885 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56531cf3bdf0 con 0x56531cfe1670 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.355+0000 7f34337fe640 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56531cfe1670 0x56531cfe3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.356+0000 7f34337fe640 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56531cfe1670 0x56531cfe3a60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b41a72479825ba9 server_cookie=ee939ed900f8bc95 in_seq=0 out_seq=0 2024-09-11T01:02:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.356+0000 7f341a7fc640 10 client.16221 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.356+0000 7f341a7fc640 1 -- 172.21.15.78:0/2585940885 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56531cf3bdf0 con 0x56531cfe1670 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.356+0000 7f341a7fc640 10 client.16221 handle_command_reply: tid=0 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.356+0000 7f343b8b9740 1 client.16221 shutdown 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.357+0000 7f343b8b9740 1 -- 172.21.15.78:0/2585940885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 msgr2=0x56531cfce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.357+0000 7f343b8b9740 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfce900 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f341c009900 tx=0x7f341c0316d0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.357+0000 7f343b8b9740 1 -- 172.21.15.78:0/2585940885 shutdown_connections 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.357+0000 7f343b8b9740 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.357+0000 7f343b8b9740 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56531cfe1670 0x56531cfe3a60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.357+0000 7f343b8b9740 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfce900 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.357+0000 7f343b8b9740 1 --2- 172.21.15.78:0/2585940885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfd1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.357+0000 7f343b8b9740 1 -- 172.21.15.78:0/2585940885 >> 172.21.15.78:0/2585940885 conn(0x56531cfc0f20 msgr2=0x56531cfc6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.357+0000 7f341a7fc640 0 client.16221 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:02:47.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.357+0000 7f343b8b9740 1 -- 172.21.15.78:0/2585940885 shutdown_connections 2024-09-11T01:02:47.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.357+0000 7f343b8b9740 1 -- 172.21.15.78:0/2585940885 wait complete. 2024-09-11T01:02:47.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.357+0000 7f343b8b9740 20 client.16221 trim_cache size 0 max 16384 2024-09-11T01:02:47.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.371+0000 7f343b8b9740 1 Processor -- start 2024-09-11T01:02:47.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.371+0000 7f343b8b9740 1 -- start start 2024-09-11T01:02:47.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f343b8b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfdda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f343b8b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfd3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f343b8b9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfd3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f343b8b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34341ac550 con 0x56531cfc3a10 2024-09-11T01:02:47.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f343b8b9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f34341ae210 con 0x56531cfd1bb0 2024-09-11T01:02:47.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f343b8b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f34341acc20 con 0x56531cfc30f0 2024-09-11T01:02:47.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f342a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfd3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f342a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfd3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40078/0 (socket says 172.21.15.78:40078) 2024-09-11T01:02:47.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f342a7fc640 1 -- 172.21.15.78:0/689167021 learned_addr learned my addr 172.21.15.78:0/689167021 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:47.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f34337fe640 1 --2- 172.21.15.78:0/689167021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfd3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f34337fe640 1 --2- 172.21.15.78:0/689167021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfdda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f342a7fc640 1 -- 172.21.15.78:0/689167021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 msgr2=0x56531cfdda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:47.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f342a7fc640 1 --2- 172.21.15.78:0/689167021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfdda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f342a7fc640 1 -- 172.21.15.78:0/689167021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 msgr2=0x56531cfd3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:47.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f342a7fc640 1 --2- 172.21.15.78:0/689167021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfd3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f342a7fc640 1 -- 172.21.15.78:0/689167021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56531cfc0d50 con 0x56531cfc3a10 2024-09-11T01:02:47.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f342a7fc640 1 --2- 172.21.15.78:0/689167021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfd3960 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f341c000c00 tx=0x7f341c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5f87b960128e7f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:47.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.372+0000 7f3428ff9640 1 -- 172.21.15.78:0/689167021 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c035030 con 0x56531cfc3a10 2024-09-11T01:02:47.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.373+0000 7f3428ff9640 1 -- 172.21.15.78:0/689167021 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f341c045900 con 0x56531cfc3a10 2024-09-11T01:02:47.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.373+0000 7f3428ff9640 1 -- 172.21.15.78:0/689167021 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c0047a0 con 0x56531cfc3a10 2024-09-11T01:02:47.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.374+0000 7f343b8b9740 1 -- 172.21.15.78:0/689167021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 msgr2=0x56531cfd3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:47.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.374+0000 7f343b8b9740 1 --2- 172.21.15.78:0/689167021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfd3960 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f341c000c00 tx=0x7f341c009520 comp rx=0 tx=0).stop 2024-09-11T01:02:47.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.374+0000 7f343b8b9740 1 -- 172.21.15.78:0/689167021 shutdown_connections 2024-09-11T01:02:47.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.374+0000 7f343b8b9740 1 --2- 172.21.15.78:0/689167021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfd3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.374+0000 7f343b8b9740 1 --2- 172.21.15.78:0/689167021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfd3960 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.374+0000 7f343b8b9740 1 --2- 172.21.15.78:0/689167021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfdda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.374+0000 7f343b8b9740 1 -- 172.21.15.78:0/689167021 >> 172.21.15.78:0/689167021 conn(0x56531cfc0f20 msgr2=0x56531cfd2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:47.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.374+0000 7f343b8b9740 1 -- 172.21.15.78:0/689167021 shutdown_connections 2024-09-11T01:02:47.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.374+0000 7f343b8b9740 1 -- 172.21.15.78:0/689167021 wait complete. 2024-09-11T01:02:47.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.374+0000 7f343b8b9740 1 -- start start 2024-09-11T01:02:47.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.375+0000 7f343b8b9740 1 Processor -- start 2024-09-11T01:02:47.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.375+0000 7f343b8b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfe9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.375+0000 7f343b8b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfd9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.375+0000 7f343b8b9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfda050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.375+0000 7f343b8b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34341ada80 con 0x56531cfc3a10 2024-09-11T01:02:47.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.375+0000 7f343b8b9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f34341ad2f0 con 0x56531cfd1bb0 2024-09-11T01:02:47.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.375+0000 7f343b8b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f34341aa550 con 0x56531cfc30f0 2024-09-11T01:02:47.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.375+0000 7f343b8b9740 10 client.0 _ll_register_callbacks cb 0x7f341c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:47.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.376+0000 7f34337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfe9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.376+0000 7f342a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfd9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.376+0000 7f34337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfe9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:34530/0 (socket says 172.21.15.78:34530) 2024-09-11T01:02:47.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.376+0000 7f34337fe640 1 -- 172.21.15.78:0/891773300 learned_addr learned my addr 172.21.15.78:0/891773300 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:47.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.376+0000 7f34337fe640 1 --2- 172.21.15.78:0/891773300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfda050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.376+0000 7f342a7fc640 1 -- 172.21.15.78:0/891773300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 msgr2=0x56531cfe9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:47.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.376+0000 7f342a7fc640 1 --2- 172.21.15.78:0/891773300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfe9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.376+0000 7f342a7fc640 1 -- 172.21.15.78:0/891773300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 msgr2=0x56531cfda050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:47.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.376+0000 7f342a7fc640 1 --2- 172.21.15.78:0/891773300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfda050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.376+0000 7f342a7fc640 1 -- 172.21.15.78:0/891773300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56531d04f530 con 0x56531cfc3a10 2024-09-11T01:02:47.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.377+0000 7f342a7fc640 1 --2- 172.21.15.78:0/891773300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfd9910 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f341c056040 tx=0x7f341c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10e9b3967aab20d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:47.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.377+0000 7f341a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:47.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.377+0000 7f341a7fc640 1 -- 172.21.15.78:0/891773300 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c035040 con 0x56531cfc3a10 2024-09-11T01:02:47.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.377+0000 7f341a7fc640 1 -- 172.21.15.78:0/891773300 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f341c0049a0 con 0x56531cfc3a10 2024-09-11T01:02:47.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.377+0000 7f343b8b9740 1 -- 172.21.15.78:0/891773300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f341c045900 con 0x56531cfc3a10 2024-09-11T01:02:47.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.377+0000 7f341a7fc640 1 -- 172.21.15.78:0/891773300 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f341c078050 con 0x56531cfc3a10 2024-09-11T01:02:47.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.377+0000 7f343b8b9740 10 client.16227 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:47.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.377+0000 7f343b8b9740 1 -- 172.21.15.78:0/891773300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56531cfd4a20 con 0x56531cfc3a10 2024-09-11T01:02:47.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.378+0000 7f341a7fc640 1 -- 172.21.15.78:0/891773300 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f341c005850 con 0x56531cfc3a10 2024-09-11T01:02:47.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.378+0000 7f341a7fc640 1 -- 172.21.15.78:0/891773300 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f341c04d040 con 0x56531cfc3a10 2024-09-11T01:02:47.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.378+0000 7f343b8b9740 10 client.16227 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:47.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.378+0000 7f343b8b9740 10 client.16227 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:02:47.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.378+0000 7f343b8b9740 20 client.16227 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:47.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.378+0000 7f343b8b9740 1 --2- 172.21.15.78:0/891773300 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56531cfd52b0 0x56531cfd76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:47.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.378+0000 7f343b8b9740 4 client.16227 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:02:47.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.378+0000 7f343b8b9740 1 -- 172.21.15.78:0/891773300 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f341c045900 con 0x56531cfd52b0 2024-09-11T01:02:47.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.379+0000 7f34337fe640 1 --2- 172.21.15.78:0/891773300 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56531cfd52b0 0x56531cfd76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:47.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.379+0000 7f34337fe640 1 --2- 172.21.15.78:0/891773300 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56531cfd52b0 0x56531cfd76d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe0d79e355487cfe server_cookie=9f31b8f5f607c7f2 in_seq=0 out_seq=0 2024-09-11T01:02:47.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.379+0000 7f341a7fc640 10 client.16227 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:02:47.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.379+0000 7f341a7fc640 1 -- 172.21.15.78:0/891773300 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f341c045900 con 0x56531cfd52b0 2024-09-11T01:02:47.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.379+0000 7f341a7fc640 10 client.16227 handle_command_reply: tid=0 2024-09-11T01:02:47.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.379+0000 7f343b8b9740 1 client.16227 shutdown 2024-09-11T01:02:47.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.380+0000 7f343b8b9740 1 -- 172.21.15.78:0/891773300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 msgr2=0x56531cfd9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:47.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.380+0000 7f343b8b9740 1 --2- 172.21.15.78:0/891773300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfd9910 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f341c056040 tx=0x7f341c004610 comp rx=0 tx=0).stop 2024-09-11T01:02:47.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.380+0000 7f343b8b9740 1 -- 172.21.15.78:0/891773300 shutdown_connections 2024-09-11T01:02:47.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.380+0000 7f343b8b9740 1 --2- 172.21.15.78:0/891773300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56531cfd1bb0 0x56531cfda050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.380+0000 7f343b8b9740 1 --2- 172.21.15.78:0/891773300 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56531cfd52b0 0x56531cfd76d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.380+0000 7f341a7fc640 0 client.16227 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:02:47.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.380+0000 7f343b8b9740 1 --2- 172.21.15.78:0/891773300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56531cfc3a10 0x56531cfd9910 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.380+0000 7f343b8b9740 1 --2- 172.21.15.78:0/891773300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56531cfc30f0 0x56531cfe9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.380+0000 7f343b8b9740 1 -- 172.21.15.78:0/891773300 >> 172.21.15.78:0/891773300 conn(0x56531cfc0f20 msgr2=0x56531cfd2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:47.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.380+0000 7f343b8b9740 1 -- 172.21.15.78:0/891773300 shutdown_connections 2024-09-11T01:02:47.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.380+0000 7f343b8b9740 1 -- 172.21.15.78:0/891773300 wait complete. 2024-09-11T01:02:47.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.380+0000 7f343b8b9740 20 client.16227 trim_cache size 0 max 16384 2024-09-11T01:02:47.393 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:02:47.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.381+0000 7f34394e4640 1 -- 172.21.15.78:0/2054649080 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f34200b5450 msgr2=0x7f34200b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:47.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.381+0000 7f34394e4640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f34200b5450 0x7f34200b7910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f34240e3100 tx=0x7f3424010f90 comp rx=0 tx=0).stop 2024-09-11T01:02:47.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.381+0000 7f34394e4640 1 -- 172.21.15.78:0/2054649080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3434101ca0 msgr2=0x7f3434198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:47.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.381+0000 7f34394e4640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3434101ca0 0x7f3434198300 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f342400b4a0 tx=0x7f342400b960 comp rx=0 tx=0).stop 2024-09-11T01:02:47.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.381+0000 7f34394e4640 1 -- 172.21.15.78:0/2054649080 shutdown_connections 2024-09-11T01:02:47.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.381+0000 7f34394e4640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f34200b5450 0x7f34200b7910 secure :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f34240e3100 tx=0x7f3424010f90 comp rx=0 tx=0).stop 2024-09-11T01:02:47.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.381+0000 7f34394e4640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3434108030 0x7f343419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.381+0000 7f34394e4640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34341076b0 0x7f3434198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:47.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.381+0000 7f34394e4640 1 --2- 172.21.15.78:0/2054649080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3434101ca0 0x7f3434198300 secure :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f342400b4a0 tx=0x7f342400b960 comp rx=0 tx=0).stop 2024-09-11T01:02:47.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.381+0000 7f34394e4640 1 -- 172.21.15.78:0/2054649080 >> 172.21.15.78:0/2054649080 conn(0x7f34340fd730 msgr2=0x7f34340ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:47.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.381+0000 7f34394e4640 1 -- 172.21.15.78:0/2054649080 shutdown_connections 2024-09-11T01:02:47.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:47.381+0000 7f34394e4640 1 -- 172.21.15.78:0/2054649080 wait complete. 2024-09-11T01:02:47.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 337131 7.88 MB/sec execute 55 sec latency 2952.421 ms 2024-09-11T01:02:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:47 smithi110 ceph-mon[34712]: pgmap v440: 97 pgs: 97 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.9 MiB/s wr, 121 op/s 2024-09-11T01:02:47.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:47 smithi112 ceph-mon[35384]: pgmap v440: 97 pgs: 97 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.9 MiB/s wr, 121 op/s 2024-09-11T01:02:48.196 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:02:48.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 337131 7.74 MB/sec execute 56 sec latency 3952.447 ms 2024-09-11T01:02:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:48 smithi110 ceph-mon[34712]: pgmap v441: 97 pgs: 97 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-09-11T01:02:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:48.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:48 smithi112 ceph-mon[35384]: pgmap v441: 97 pgs: 97 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-09-11T01:02:48.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:48 smithi078 ceph-mon[31315]: pgmap v441: 97 pgs: 97 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-09-11T01:02:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.016+0000 7fb4e835e640 1 -- 172.21.15.78:0/3209861745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4e0107780 msgr2=0x7fb4e0107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.016+0000 7fb4e835e640 1 --2- 172.21.15.78:0/3209861745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4e0107780 0x7fb4e0107be0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009900 tx=0x7fb4d002a410 comp rx=0 tx=0).stop 2024-09-11T01:02:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.016+0000 7fb4e835e640 1 -- 172.21.15.78:0/3209861745 shutdown_connections 2024-09-11T01:02:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.016+0000 7fb4e835e640 1 --2- 172.21.15.78:0/3209861745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4e0108120 0x7fb4e010e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.016+0000 7fb4e835e640 1 --2- 172.21.15.78:0/3209861745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4e0107780 0x7fb4e0107be0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.016+0000 7fb4e835e640 1 --2- 172.21.15.78:0/3209861745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4e0102920 0x7fb4e0102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.016+0000 7fb4e835e640 1 -- 172.21.15.78:0/3209861745 >> 172.21.15.78:0/3209861745 conn(0x7fb4e0076f50 msgr2=0x7fb4e0077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.016+0000 7fb4e835e640 1 -- 172.21.15.78:0/3209861745 shutdown_connections 2024-09-11T01:02:49.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.016+0000 7fb4e835e640 1 -- 172.21.15.78:0/3209861745 wait complete. 2024-09-11T01:02:49.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.016+0000 7fb4e835e640 1 Processor -- start 2024-09-11T01:02:49.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.016+0000 7fb4e835e640 1 -- start start 2024-09-11T01:02:49.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e835e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4e0102920 0x7fb4e01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e835e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4e0107780 0x7fb4e01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e835e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4e0108120 0x7fb4e019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e835e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0111cc0 con 0x7fb4e0108120 2024-09-11T01:02:49.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e835e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0111b40 con 0x7fb4e0107780 2024-09-11T01:02:49.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e835e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0111e40 con 0x7fb4e0102920 2024-09-11T01:02:49.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e68fb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4e0108120 0x7fb4e019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e68fb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4e0108120 0x7fb4e019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40108/0 (socket says 172.21.15.78:40108) 2024-09-11T01:02:49.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e68fb640 1 -- 172.21.15.78:0/3646763863 learned_addr learned my addr 172.21.15.78:0/3646763863 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:49.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4e0107780 0x7fb4e01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e60fa640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4e0102920 0x7fb4e01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e68fb640 1 -- 172.21.15.78:0/3646763863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4e0102920 msgr2=0x7fb4e01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e68fb640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4e0102920 0x7fb4e01992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e68fb640 1 -- 172.21.15.78:0/3646763863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4e0107780 msgr2=0x7fb4e01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e68fb640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4e0107780 0x7fb4e01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e68fb640 1 -- 172.21.15.78:0/3646763863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e019f350 con 0x7fb4e0108120 2024-09-11T01:02:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4e0107780 0x7fb4e01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e68fb640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4e0108120 0x7fb4e019eb80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800b970 tx=0x7fb4d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4d77fe640 1 -- 172.21.15.78:0/3646763863 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8019070 con 0x7fb4e0108120 2024-09-11T01:02:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e835e640 1 -- 172.21.15.78:0/3646763863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e019f640 con 0x7fb4e0108120 2024-09-11T01:02:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4d77fe640 1 -- 172.21.15.78:0/3646763863 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d8014440 con 0x7fb4e0108120 2024-09-11T01:02:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4d77fe640 1 -- 172.21.15.78:0/3646763863 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80075d0 con 0x7fb4e0108120 2024-09-11T01:02:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.017+0000 7fb4e835e640 1 -- 172.21.15.78:0/3646763863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e01a5d00 con 0x7fb4e0108120 2024-09-11T01:02:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.020+0000 7fb4d77fe640 1 -- 172.21.15.78:0/3646763863 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb4d800cb40 con 0x7fb4e0108120 2024-09-11T01:02:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.020+0000 7fb4ea733740 1 Processor -- start 2024-09-11T01:02:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.020+0000 7fb4ea733740 1 -- start start 2024-09-11T01:02:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.020+0000 7fb4ea733740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be33d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.020+0000 7fb4ea733740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33da10 0x5572be349460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.020+0000 7fb4ea733740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 0x5572be34bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.020+0000 7fb4ea733740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572be2b3ea0 con 0x5572be33d0f0 2024-09-11T01:02:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.020+0000 7fb4ea733740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5572be2a32d0 con 0x5572be3499a0 2024-09-11T01:02:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.020+0000 7fb4ea733740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5572be2a70e0 con 0x5572be33da10 2024-09-11T01:02:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4e58f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be33d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4e58f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be33d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40122/0 (socket says 172.21.15.78:40122) 2024-09-11T01:02:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4e58f9640 1 -- 172.21.15.78:0/2471194317 learned_addr learned my addr 172.21.15.78:0/2471194317 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/2471194317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 0x5572be34bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4e60fa640 1 --2- 172.21.15.78:0/2471194317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33da10 0x5572be349460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4e58f9640 1 -- 172.21.15.78:0/2471194317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33da10 msgr2=0x5572be349460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:49.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/2471194317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33da10 0x5572be349460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4e58f9640 1 -- 172.21.15.78:0/2471194317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 msgr2=0x5572be34bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:49.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/2471194317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 0x5572be34bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4e58f9640 1 -- 172.21.15.78:0/2471194317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572be001650 con 0x5572be33d0f0 2024-09-11T01:02:49.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4d77fe640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb4cc0b5450 0x7fb4cc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/2471194317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be33d4d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005ab0 tx=0x7fb4d0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaeddc15c097fda server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:49.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4e60fa640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb4cc0b5450 0x7fb4cc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4d77fe640 1 -- 172.21.15.78:0/3646763863 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb4d80e57e0 con 0x7fb4e0108120 2024-09-11T01:02:49.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4d4de6640 1 -- 172.21.15.78:0/2471194317 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0043020 con 0x5572be33d0f0 2024-09-11T01:02:49.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4d4de6640 1 -- 172.21.15.78:0/2471194317 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d0031910 con 0x5572be33d0f0 2024-09-11T01:02:49.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.021+0000 7fb4d4de6640 1 -- 172.21.15.78:0/2471194317 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0035050 con 0x5572be33d0f0 2024-09-11T01:02:49.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.022+0000 7fb4ea733740 1 -- 172.21.15.78:0/2471194317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 msgr2=0x5572be33d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:49.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.022+0000 7fb4e60fa640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb4cc0b5450 0x7fb4cc0b7910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc005fd0 tx=0x7fb4dc005f20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:49.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.022+0000 7fb4ea733740 1 --2- 172.21.15.78:0/2471194317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be33d4d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005ab0 tx=0x7fb4d0009560 comp rx=0 tx=0).stop 2024-09-11T01:02:49.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.022+0000 7fb4ea733740 1 -- 172.21.15.78:0/2471194317 shutdown_connections 2024-09-11T01:02:49.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.022+0000 7fb4ea733740 1 --2- 172.21.15.78:0/2471194317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 0x5572be34bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.022+0000 7fb4ea733740 1 --2- 172.21.15.78:0/2471194317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33da10 0x5572be349460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.022+0000 7fb4ea733740 1 --2- 172.21.15.78:0/2471194317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be33d4d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.022+0000 7fb4ea733740 1 -- 172.21.15.78:0/2471194317 >> 172.21.15.78:0/2471194317 conn(0x5572be33af20 msgr2=0x5572be33c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:49.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.022+0000 7fb4ea733740 1 -- 172.21.15.78:0/2471194317 shutdown_connections 2024-09-11T01:02:49.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.022+0000 7fb4ea733740 1 -- 172.21.15.78:0/2471194317 wait complete. 2024-09-11T01:02:49.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.022+0000 7fb4ea733740 1 -- start start 2024-09-11T01:02:49.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4ea733740 1 Processor -- start 2024-09-11T01:02:49.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4ea733740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be349230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4ea733740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be33da10 0x5572be346680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4ea733740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be3499a0 0x5572be346c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4ea733740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572be104700 con 0x5572be33d0f0 2024-09-11T01:02:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4ea733740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5572be23e160 con 0x5572be33da10 2024-09-11T01:02:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4ea733740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5572be34c5c0 con 0x5572be3499a0 2024-09-11T01:02:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4ea733740 10 client.0 _ll_register_callbacks cb 0x5572be2579e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4e60fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be349230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4e60fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be349230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40136/0 (socket says 172.21.15.78:40136) 2024-09-11T01:02:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4e60fa640 1 -- 172.21.15.78:0/2252860663 learned_addr learned my addr 172.21.15.78:0/2252860663 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be33da10 0x5572be346680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4e68fb640 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be3499a0 0x5572be346c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4e60fa640 1 -- 172.21.15.78:0/2252860663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be3499a0 msgr2=0x5572be346c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4e60fa640 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be3499a0 0x5572be346c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4e60fa640 1 -- 172.21.15.78:0/2252860663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be33da10 msgr2=0x5572be346680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4e60fa640 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be33da10 0x5572be346680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4e60fa640 1 -- 172.21.15.78:0/2252860663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572be26bc90 con 0x5572be33d0f0 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.023+0000 7fb4e68fb640 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be3499a0 0x5572be346c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4e60fa640 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be349230 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc011b10 tx=0x7fb4dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6b094892b4299a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4c9ffb640 1 -- 172.21.15.78:0/2252860663 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00ee20 con 0x5572be33d0f0 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4c9ffb640 1 -- 172.21.15.78:0/2252860663 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb4dc01e420 con 0x5572be33d0f0 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4ea733740 1 -- 172.21.15.78:0/2252860663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4d0031910 con 0x5572be33d0f0 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4c9ffb640 1 -- 172.21.15.78:0/2252860663 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb4dc01e620 con 0x5572be33d0f0 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4ea733740 10 client.16239 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4ea733740 1 -- 172.21.15.78:0/2252860663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5572be357f20 con 0x5572be33d0f0 2024-09-11T01:02:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4c9ffb640 1 -- 172.21.15.78:0/2252860663 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d0031910 con 0x5572be33d0f0 2024-09-11T01:02:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4c9ffb640 1 -- 172.21.15.78:0/2252860663 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fb4dc02e050 con 0x5572be33d0f0 2024-09-11T01:02:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4ea733740 10 client.16239 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4ea733740 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4ea733740 20 client.16239 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4ea733740 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572be3593d0 0x5572be35b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4ea733740 4 client.16239 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:02:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.024+0000 7fb4ea733740 1 -- 172.21.15.78:0/2252860663 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572be2b5df0 con 0x5572be3593d0 2024-09-11T01:02:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.025+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572be3593d0 0x5572be35b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.025+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572be3593d0 0x5572be35b810 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb874ed4527b651 server_cookie=9d6b5533eda2aa0c in_seq=0 out_seq=0 2024-09-11T01:02:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.025+0000 7fb4c9ffb640 10 client.16239 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4c9ffb640 1 -- 172.21.15.78:0/2252860663 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5572be2b5df0 con 0x5572be3593d0 2024-09-11T01:02:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4c9ffb640 10 client.16239 handle_command_reply: tid=0 2024-09-11T01:02:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4ea733740 1 client.16239 shutdown 2024-09-11T01:02:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4ea733740 1 -- 172.21.15.78:0/2252860663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 msgr2=0x5572be349230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4ea733740 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be349230 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc011b10 tx=0x7fb4dc0046c0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4ea733740 1 -- 172.21.15.78:0/2252860663 shutdown_connections 2024-09-11T01:02:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4ea733740 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be3499a0 0x5572be346c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4ea733740 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be33da10 0x5572be346680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4ea733740 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572be3593d0 0x5572be35b810 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4ea733740 1 --2- 172.21.15.78:0/2252860663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be349230 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4c9ffb640 0 client.16239 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4ea733740 1 -- 172.21.15.78:0/2252860663 >> 172.21.15.78:0/2252860663 conn(0x5572be33af20 msgr2=0x5572be340a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4ea733740 1 -- 172.21.15.78:0/2252860663 shutdown_connections 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4ea733740 1 -- 172.21.15.78:0/2252860663 wait complete. 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.026+0000 7fb4ea733740 20 client.16239 trim_cache size 0 max 16384 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4ea733740 1 Processor -- start 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4ea733740 1 -- start start 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4ea733740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33d0f0 0x5572be34ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4ea733740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33da10 0x5572be34b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4ea733740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 0x5572be3438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4ea733740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01ae540 con 0x5572be33da10 2024-09-11T01:02:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4ea733740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01b0200 con 0x5572be3499a0 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4ea733740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01aec10 con 0x5572be33d0f0 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4e58f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33da10 0x5572be34b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4e58f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33da10 0x5572be34b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40150/0 (socket says 172.21.15.78:40150) 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4e58f9640 1 -- 172.21.15.78:0/3324641124 learned_addr learned my addr 172.21.15.78:0/3324641124 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4e60fa640 1 --2- 172.21.15.78:0/3324641124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33d0f0 0x5572be34ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4e58f9640 1 -- 172.21.15.78:0/3324641124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33d0f0 msgr2=0x5572be34ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/3324641124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33d0f0 0x5572be34ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4e58f9640 1 -- 172.21.15.78:0/3324641124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 msgr2=0x5572be3438b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/3324641124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 0x5572be3438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4e58f9640 1 -- 172.21.15.78:0/3324641124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572be33b9a0 con 0x5572be33da10 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.039+0000 7fb4e60fa640 1 --2- 172.21.15.78:0/3324641124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33d0f0 0x5572be34ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/3324641124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33da10 0x5572be34b2d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009900 tx=0x7fb4d002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffc04a6b53d3cec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4cbfff640 1 -- 172.21.15.78:0/3324641124 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0043040 con 0x5572be33da10 2024-09-11T01:02:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4cbfff640 1 -- 172.21.15.78:0/3324641124 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d0054070 con 0x5572be33da10 2024-09-11T01:02:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4cbfff640 1 -- 172.21.15.78:0/3324641124 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0035040 con 0x5572be33da10 2024-09-11T01:02:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4ea733740 1 -- 172.21.15.78:0/3324641124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33da10 msgr2=0x5572be34b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4ea733740 1 --2- 172.21.15.78:0/3324641124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33da10 0x5572be34b2d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009900 tx=0x7fb4d002a410 comp rx=0 tx=0).stop 2024-09-11T01:02:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4ea733740 1 -- 172.21.15.78:0/3324641124 shutdown_connections 2024-09-11T01:02:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4ea733740 1 --2- 172.21.15.78:0/3324641124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 0x5572be3438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4ea733740 1 --2- 172.21.15.78:0/3324641124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33da10 0x5572be34b2d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4ea733740 1 --2- 172.21.15.78:0/3324641124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33d0f0 0x5572be34ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4ea733740 1 -- 172.21.15.78:0/3324641124 >> 172.21.15.78:0/3324641124 conn(0x5572be33af20 msgr2=0x5572be341950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4ea733740 1 -- 172.21.15.78:0/3324641124 shutdown_connections 2024-09-11T01:02:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.040+0000 7fb4ea733740 1 -- 172.21.15.78:0/3324641124 wait complete. 2024-09-11T01:02:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4ea733740 1 -- start start 2024-09-11T01:02:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4ea733740 1 Processor -- start 2024-09-11T01:02:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4ea733740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be351880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4ea733740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33da10 0x5572be351dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4ea733740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 0x5572be3428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4ea733740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01afa70 con 0x5572be33d0f0 2024-09-11T01:02:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4ea733740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01af2e0 con 0x5572be3499a0 2024-09-11T01:02:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4ea733740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01ac540 con 0x5572be33da10 2024-09-11T01:02:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4ea733740 10 client.0 _ll_register_callbacks cb 0x7fb4dc00c900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4e60fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be351880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4e60fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be351880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40158/0 (socket says 172.21.15.78:40158) 2024-09-11T01:02:49.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4e60fa640 1 -- 172.21.15.78:0/1322020032 learned_addr learned my addr 172.21.15.78:0/1322020032 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:49.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4e68fb640 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 0x5572be3428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33da10 0x5572be351dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4e60fa640 1 -- 172.21.15.78:0/1322020032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33da10 msgr2=0x5572be351dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:49.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4e60fa640 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33da10 0x5572be351dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4e60fa640 1 -- 172.21.15.78:0/1322020032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 msgr2=0x5572be3428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:49.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4e60fa640 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 0x5572be3428a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.041+0000 7fb4e60fa640 1 -- 172.21.15.78:0/1322020032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572be352550 con 0x5572be33d0f0 2024-09-11T01:02:49.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4e60fa640 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be351880 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc011550 tx=0x7fb4dc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf25c042325a89e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:49.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:49.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4c9ffb640 1 -- 172.21.15.78:0/1322020032 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc02f580 con 0x5572be33d0f0 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4ea733740 1 -- 172.21.15.78:0/1322020032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4d0054070 con 0x5572be33d0f0 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4c9ffb640 1 -- 172.21.15.78:0/1322020032 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb4dc004020 con 0x5572be33d0f0 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4c9ffb640 1 -- 172.21.15.78:0/1322020032 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb4dc004220 con 0x5572be33d0f0 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4ea733740 10 client.16245 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4ea733740 1 -- 172.21.15.78:0/1322020032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5572be343e40 con 0x5572be33d0f0 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4c9ffb640 1 -- 172.21.15.78:0/1322020032 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d0054070 con 0x5572be33d0f0 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4c9ffb640 1 -- 172.21.15.78:0/1322020032 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fb4dc02e070 con 0x5572be33d0f0 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4ea733740 10 client.16245 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4ea733740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4ea733740 20 client.16245 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4ea733740 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572be3c7330 0x5572be3c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4ea733740 4 client.16245 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:02:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.042+0000 7fb4ea733740 1 -- 172.21.15.78:0/1322020032 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572be2b18a0 con 0x5572be3c7330 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572be3c7330 0x5572be3c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4e58f9640 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572be3c7330 0x5572be3c9720 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e86985c52e119db server_cookie=3edb69fb72a918f9 in_seq=0 out_seq=0 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4c9ffb640 10 client.16245 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4c9ffb640 1 -- 172.21.15.78:0/1322020032 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572be2b18a0 con 0x5572be3c7330 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4c9ffb640 10 client.16245 handle_command_reply: tid=0 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4ea733740 1 client.16245 shutdown 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4ea733740 1 -- 172.21.15.78:0/1322020032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 msgr2=0x5572be351880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4ea733740 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be351880 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc011550 tx=0x7fb4dc00ee50 comp rx=0 tx=0).stop 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4ea733740 1 -- 172.21.15.78:0/1322020032 shutdown_connections 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4ea733740 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572be3499a0 0x5572be3428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4ea733740 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572be33da10 0x5572be351dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4ea733740 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572be3c7330 0x5572be3c9720 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4ea733740 1 --2- 172.21.15.78:0/1322020032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572be33d0f0 0x5572be351880 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4c9ffb640 0 client.16245 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.043+0000 7fb4ea733740 1 -- 172.21.15.78:0/1322020032 >> 172.21.15.78:0/1322020032 conn(0x5572be33af20 msgr2=0x5572be347cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4ea733740 1 -- 172.21.15.78:0/1322020032 shutdown_connections 2024-09-11T01:02:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4ea733740 1 -- 172.21.15.78:0/1322020032 wait complete. 2024-09-11T01:02:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4ea733740 20 client.16245 trim_cache size 0 max 16384 2024-09-11T01:02:49.083 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:02:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4e835e640 1 -- 172.21.15.78:0/3646763863 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb4cc0b5450 msgr2=0x7fb4cc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4e835e640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb4cc0b5450 0x7fb4cc0b7910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc005fd0 tx=0x7fb4dc005f20 comp rx=0 tx=0).stop 2024-09-11T01:02:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4e835e640 1 -- 172.21.15.78:0/3646763863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4e0108120 msgr2=0x7fb4e019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4e835e640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4e0108120 0x7fb4e019eb80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800b970 tx=0x7fb4d800be30 comp rx=0 tx=0).stop 2024-09-11T01:02:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4e835e640 1 -- 172.21.15.78:0/3646763863 shutdown_connections 2024-09-11T01:02:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4e835e640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb4cc0b5450 0x7fb4cc0b7910 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4e835e640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4e0108120 0x7fb4e019eb80 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4e835e640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4e0107780 0x7fb4e01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4e835e640 1 --2- 172.21.15.78:0/3646763863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4e0102920 0x7fb4e01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4e835e640 1 -- 172.21.15.78:0/3646763863 >> 172.21.15.78:0/3646763863 conn(0x7fb4e0076f50 msgr2=0x7fb4e0106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4e835e640 1 -- 172.21.15.78:0/3646763863 shutdown_connections 2024-09-11T01:02:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:49.044+0000 7fb4e835e640 1 -- 172.21.15.78:0/3646763863 wait complete. 2024-09-11T01:02:49.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 337184 7.62 MB/sec execute 57 sec latency 4941.231 ms 2024-09-11T01:02:50.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 339347 7.61 MB/sec execute 58 sec latency 14.172 ms 2024-09-11T01:02:50.956 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:02:50.957 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:02:51.204 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:51.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:51 smithi078 ceph-mon[31315]: pgmap v442: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-11T01:02:51.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 340527 7.66 MB/sec execute 59 sec latency 540.018 ms 2024-09-11T01:02:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:51 smithi110 ceph-mon[34712]: pgmap v442: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-11T01:02:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:51 smithi112 ceph-mon[35384]: pgmap v442: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-11T01:02:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.904+0000 7f6bc5162640 1 -- 172.21.15.78:0/1191605267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0109150 msgr2=0x7f6bc010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.904+0000 7f6bc5162640 1 --2- 172.21.15.78:0/1191605267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0109150 0x7f6bc010f950 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009ca0 tx=0x7f6bb802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.904+0000 7f6bc5162640 1 -- 172.21.15.78:0/1191605267 shutdown_connections 2024-09-11T01:02:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.904+0000 7f6bc5162640 1 --2- 172.21.15.78:0/1191605267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0109150 0x7f6bc010f950 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.904+0000 7f6bc5162640 1 --2- 172.21.15.78:0/1191605267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bc01087d0 0x7f6bc0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.904+0000 7f6bc5162640 1 --2- 172.21.15.78:0/1191605267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc0102dc0 0x7f6bc01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.904+0000 7f6bc5162640 1 -- 172.21.15.78:0/1191605267 >> 172.21.15.78:0/1191605267 conn(0x7f6bc00fe830 msgr2=0x7f6bc0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.905+0000 7f6bc5162640 1 -- 172.21.15.78:0/1191605267 shutdown_connections 2024-09-11T01:02:51.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.905+0000 7f6bc5162640 1 -- 172.21.15.78:0/1191605267 wait complete. 2024-09-11T01:02:51.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.905+0000 7f6bc5162640 1 Processor -- start 2024-09-11T01:02:51.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.905+0000 7f6bc5162640 1 -- start start 2024-09-11T01:02:51.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bc5162640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0102dc0 0x7f6bc019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bc5162640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bc01087d0 0x7f6bc019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bc5162640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc0109150 0x7f6bc01a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bc5162640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0112e10 con 0x7f6bc0102dc0 2024-09-11T01:02:51.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bc5162640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0112c90 con 0x7f6bc01087d0 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bc5162640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0112f90 con 0x7f6bc0109150 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bbed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0102dc0 0x7f6bc019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bbed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0102dc0 0x7f6bc019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40166/0 (socket says 172.21.15.78:40166) 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bbed9d640 1 -- 172.21.15.78:0/3079529420 learned_addr learned my addr 172.21.15.78:0/3079529420 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bbe59c640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bc01087d0 0x7f6bc019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bbf59e640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc0109150 0x7f6bc01a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bbed9d640 1 -- 172.21.15.78:0/3079529420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc0109150 msgr2=0x7f6bc01a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc0109150 0x7f6bc01a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bbed9d640 1 -- 172.21.15.78:0/3079529420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bc01087d0 msgr2=0x7f6bc019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bc01087d0 0x7f6bc019b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bbed9d640 1 -- 172.21.15.78:0/3079529420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc01a14d0 con 0x7f6bc0102dc0 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0102dc0 0x7f6bc019b420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000d770 tx=0x7f6bb000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.906+0000 7f6ba7fff640 1 -- 172.21.15.78:0/3079529420 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0017070 con 0x7f6bc0102dc0 2024-09-11T01:02:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.907+0000 7f6bc5162640 1 -- 172.21.15.78:0/3079529420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc01a17c0 con 0x7f6bc0102dc0 2024-09-11T01:02:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.907+0000 7f6bc5162640 1 -- 172.21.15.78:0/3079529420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc01a7e80 con 0x7f6bc0102dc0 2024-09-11T01:02:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.907+0000 7f6ba7fff640 1 -- 172.21.15.78:0/3079529420 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb0009c30 con 0x7f6bc0102dc0 2024-09-11T01:02:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.907+0000 7f6ba7fff640 1 -- 172.21.15.78:0/3079529420 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0020060 con 0x7f6bc0102dc0 2024-09-11T01:02:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.909+0000 7f6ba7fff640 1 -- 172.21.15.78:0/3079529420 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6bb000f620 con 0x7f6bc0102dc0 2024-09-11T01:02:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.909+0000 7f6bc7537740 1 Processor -- start 2024-09-11T01:02:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.909+0000 7f6bc7537740 1 -- start start 2024-09-11T01:02:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.909+0000 7f6bc7537740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabbee0 0x55722cabc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.909+0000 7f6bc7537740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabc800 0x55722cac8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.909+0000 7f6bc7537740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cac8790 0x55722cacab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.909+0000 7f6bc7537740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55722ca32e60 con 0x55722cabc800 2024-09-11T01:02:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.909+0000 7f6bc7537740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55722ca22290 con 0x55722cabbee0 2024-09-11T01:02:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.909+0000 7f6bc7537740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55722ca260a0 con 0x55722cac8790 2024-09-11T01:02:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.909+0000 7f6bbed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabc800 0x55722cac8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.909+0000 7f6bbed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabc800 0x55722cac8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40176/0 (socket says 172.21.15.78:40176) 2024-09-11T01:02:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6bbed9d640 1 -- 172.21.15.78:0/3921617185 learned_addr learned my addr 172.21.15.78:0/3921617185 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6bbf59e640 1 --2- 172.21.15.78:0/3921617185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabbee0 0x55722cabc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6bbe59c640 1 --2- 172.21.15.78:0/3921617185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cac8790 0x55722cacab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6bbed9d640 1 -- 172.21.15.78:0/3921617185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cac8790 msgr2=0x55722cacab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/3921617185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cac8790 0x55722cacab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6bbed9d640 1 -- 172.21.15.78:0/3921617185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabbee0 msgr2=0x55722cabc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/3921617185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabbee0 0x55722cabc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6bbed9d640 1 -- 172.21.15.78:0/3921617185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55722c780930 con 0x55722cabc800 2024-09-11T01:02:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6ba7fff640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6ba80b5450 0x7f6ba80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6bbe59c640 1 --2- 172.21.15.78:0/3921617185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cac8790 0x55722cacab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6bbf59e640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6ba80b5450 0x7f6ba80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/3921617185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabc800 0x55722cac8250 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00e2fc0 tx=0x7f6bb00ed690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686891bbb0cec60a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6ba7fff640 1 -- 172.21.15.78:0/3079529420 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6bb00e58c0 con 0x7f6bc0102dc0 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6ba57fa640 1 -- 172.21.15.78:0/3921617185 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00a7d00 con 0x55722cabc800 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6ba57fa640 1 -- 172.21.15.78:0/3921617185 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb00ab970 con 0x55722cabc800 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.910+0000 7f6ba57fa640 1 -- 172.21.15.78:0/3921617185 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00ab5d0 con 0x55722cabc800 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.911+0000 7f6bc7537740 1 -- 172.21.15.78:0/3921617185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabc800 msgr2=0x55722cac8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.911+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3921617185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabc800 0x55722cac8250 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00e2fc0 tx=0x7f6bb00ed690 comp rx=0 tx=0).stop 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.911+0000 7f6bbf59e640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6ba80b5450 0x7f6ba80b7910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6bb802f500 tx=0x7f6bb802f370 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.911+0000 7f6bc7537740 1 -- 172.21.15.78:0/3921617185 shutdown_connections 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.911+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3921617185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cac8790 0x55722cacab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.911+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3921617185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabc800 0x55722cac8250 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.911+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3921617185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabbee0 0x55722cabc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.911+0000 7f6bc7537740 1 -- 172.21.15.78:0/3921617185 >> 172.21.15.78:0/3921617185 conn(0x55722cab9d10 msgr2=0x55722cabaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:51.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.911+0000 7f6bc7537740 1 -- 172.21.15.78:0/3921617185 shutdown_connections 2024-09-11T01:02:51.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.911+0000 7f6bc7537740 1 -- 172.21.15.78:0/3921617185 wait complete. 2024-09-11T01:02:51.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.911+0000 7f6bc7537740 1 -- start start 2024-09-11T01:02:51.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bc7537740 1 Processor -- start 2024-09-11T01:02:51.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bc7537740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 0x55722cac57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bc7537740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabc800 0x55722cac5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bc7537740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cac8790 0x55722cac6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bc7537740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55722c883890 con 0x55722cabbee0 2024-09-11T01:02:51.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bc7537740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55722c9bcd40 con 0x55722cabc800 2024-09-11T01:02:51.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bc7537740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55722cacb440 con 0x55722cac8790 2024-09-11T01:02:51.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bc7537740 10 client.0 _ll_register_callbacks cb 0x55722ca20c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 0x55722cac57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 0x55722cac57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40190/0 (socket says 172.21.15.78:40190) 2024-09-11T01:02:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbed9d640 1 -- 172.21.15.78:0/2713183237 learned_addr learned my addr 172.21.15.78:0/2713183237 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbe59c640 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabc800 0x55722cac5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbf59e640 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cac8790 0x55722cac6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbed9d640 1 -- 172.21.15.78:0/2713183237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cac8790 msgr2=0x55722cac6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cac8790 0x55722cac6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbed9d640 1 -- 172.21.15.78:0/2713183237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabc800 msgr2=0x55722cac5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabc800 0x55722cac5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbed9d640 1 -- 172.21.15.78:0/2713183237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55722cad4930 con 0x55722cabbee0 2024-09-11T01:02:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbe59c640 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabc800 0x55722cac5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:02:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbf59e640 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cac8790 0x55722cac6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:02:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 0x55722cac57c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00a8430 tx=0x7f6bb00eb390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb75da3f5efd4d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.912+0000 7f6ba2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6ba2ff5640 1 -- 172.21.15.78:0/2713183237 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00ab7b0 con 0x55722cabbee0 2024-09-11T01:02:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6ba2ff5640 1 -- 172.21.15.78:0/2713183237 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6bb00e3300 con 0x55722cabbee0 2024-09-11T01:02:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6bc7537740 1 -- 172.21.15.78:0/2713183237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bb00ab970 con 0x55722cabbee0 2024-09-11T01:02:51.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6ba2ff5640 1 -- 172.21.15.78:0/2713183237 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6bb00abc30 con 0x55722cabbee0 2024-09-11T01:02:51.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6bc7537740 10 client.16257 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:51.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6bc7537740 1 -- 172.21.15.78:0/2713183237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55722cad72c0 con 0x55722cabbee0 2024-09-11T01:02:51.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6ba2ff5640 1 -- 172.21.15.78:0/2713183237 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb00ab970 con 0x55722cabbee0 2024-09-11T01:02:51.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6ba2ff5640 1 -- 172.21.15.78:0/2713183237 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f6bb00a80d0 con 0x55722cabbee0 2024-09-11T01:02:51.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6bc7537740 10 client.16257 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:51.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6bc7537740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:51.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6bc7537740 20 client.16257 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:51.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6bc7537740 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55722cad8610 0x55722cadaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6bc7537740 4 client.16257 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:02:51.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6bc7537740 1 -- 172.21.15.78:0/2713183237 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55722ca34db0 con 0x55722cad8610 2024-09-11T01:02:51.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.913+0000 7f6bbe59c640 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55722cad8610 0x55722cadaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.914+0000 7f6ba2ff5640 10 client.16257 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:51.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.914+0000 7f6bbe59c640 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55722cad8610 0x55722cadaa00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc1206c8d6b24354 server_cookie=1630bf185be11eff in_seq=0 out_seq=0 2024-09-11T01:02:51.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.914+0000 7f6ba2ff5640 1 -- 172.21.15.78:0/2713183237 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55722ca34db0 con 0x55722cad8610 2024-09-11T01:02:51.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.914+0000 7f6ba2ff5640 10 client.16257 handle_command_reply: tid=0 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6bc7537740 1 client.16257 shutdown 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6bc7537740 1 -- 172.21.15.78:0/2713183237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 msgr2=0x55722cac57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6bc7537740 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 0x55722cac57c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00a8430 tx=0x7f6bb00eb390 comp rx=0 tx=0).stop 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6bc7537740 1 -- 172.21.15.78:0/2713183237 shutdown_connections 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6bc7537740 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cac8790 0x55722cac6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6bc7537740 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabc800 0x55722cac5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6bc7537740 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55722cad8610 0x55722cadaa00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6ba2ff5640 0 client.16257 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6bc7537740 1 --2- 172.21.15.78:0/2713183237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 0x55722cac57c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6bc7537740 1 -- 172.21.15.78:0/2713183237 >> 172.21.15.78:0/2713183237 conn(0x55722cab9d10 msgr2=0x55722cab8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6bc7537740 1 -- 172.21.15.78:0/2713183237 shutdown_connections 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6bc7537740 1 -- 172.21.15.78:0/2713183237 wait complete. 2024-09-11T01:02:51.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.915+0000 7f6bc7537740 20 client.16257 trim_cache size 0 max 16384 2024-09-11T01:02:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.932+0000 7f6bc7537740 1 Processor -- start 2024-09-11T01:02:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.932+0000 7f6bc7537740 1 -- start start 2024-09-11T01:02:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.932+0000 7f6bc7537740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cabbee0 0x55722cac5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.932+0000 7f6bc7537740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabc800 0x55722cac5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.932+0000 7f6bc7537740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cac8790 0x55722cac29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.932+0000 7f6bc7537740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac00e590 con 0x55722cac8790 2024-09-11T01:02:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.932+0000 7f6bc7537740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac010250 con 0x55722cabc800 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.932+0000 7f6bc7537740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac00ec60 con 0x55722cabbee0 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.932+0000 7f6bbf59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cac8790 0x55722cac29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.932+0000 7f6bbf59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cac8790 0x55722cac29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40202/0 (socket says 172.21.15.78:40202) 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.932+0000 7f6bbf59e640 1 -- 172.21.15.78:0/3507846614 learned_addr learned my addr 172.21.15.78:0/3507846614 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.933+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/3507846614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cabbee0 0x55722cac5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.933+0000 7f6bbe59c640 1 --2- 172.21.15.78:0/3507846614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabc800 0x55722cac5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.933+0000 7f6bbf59e640 1 -- 172.21.15.78:0/3507846614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cabbee0 msgr2=0x55722cac5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.933+0000 7f6bbf59e640 1 --2- 172.21.15.78:0/3507846614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cabbee0 0x55722cac5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.933+0000 7f6bbf59e640 1 -- 172.21.15.78:0/3507846614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabc800 msgr2=0x55722cac5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.933+0000 7f6bbf59e640 1 --2- 172.21.15.78:0/3507846614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabc800 0x55722cac5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.933+0000 7f6bbf59e640 1 -- 172.21.15.78:0/3507846614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55722cac7fa0 con 0x55722cac8790 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.933+0000 7f6bbe59c640 1 --2- 172.21.15.78:0/3507846614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabc800 0x55722cac5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:02:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.933+0000 7f6bbf59e640 1 --2- 172.21.15.78:0/3507846614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cac8790 0x55722cac29f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009a60 tx=0x7f6bb8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12309c15a1ec7306 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.933+0000 7f6ba4ff9640 1 -- 172.21.15.78:0/3507846614 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb802f660 con 0x55722cac8790 2024-09-11T01:02:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.933+0000 7f6ba4ff9640 1 -- 172.21.15.78:0/3507846614 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb8036050 con 0x55722cac8790 2024-09-11T01:02:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.933+0000 7f6ba4ff9640 1 -- 172.21.15.78:0/3507846614 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8030a40 con 0x55722cac8790 2024-09-11T01:02:51.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 -- 172.21.15.78:0/3507846614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cac8790 msgr2=0x55722cac29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:51.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3507846614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cac8790 0x55722cac29f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009a60 tx=0x7f6bb8041000 comp rx=0 tx=0).stop 2024-09-11T01:02:51.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 -- 172.21.15.78:0/3507846614 shutdown_connections 2024-09-11T01:02:51.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3507846614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cac8790 0x55722cac29f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3507846614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cabc800 0x55722cac5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3507846614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cabbee0 0x55722cac5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 -- 172.21.15.78:0/3507846614 >> 172.21.15.78:0/3507846614 conn(0x55722cab9d10 msgr2=0x55722cac5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:51.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 -- 172.21.15.78:0/3507846614 shutdown_connections 2024-09-11T01:02:51.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 -- 172.21.15.78:0/3507846614 wait complete. 2024-09-11T01:02:51.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 -- start start 2024-09-11T01:02:51.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 Processor -- start 2024-09-11T01:02:51.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 0x55722cadc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cabc800 0x55722cadc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cac8790 0x55722cb44180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac00fac0 con 0x55722cabbee0 2024-09-11T01:02:51.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac00f330 con 0x55722cac8790 2024-09-11T01:02:51.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.934+0000 7f6bc7537740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac00c570 con 0x55722cabc800 2024-09-11T01:02:51.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bc7537740 10 client.0 _ll_register_callbacks cb 0x7f6bb8009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bbed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 0x55722cadc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bbed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 0x55722cadc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40218/0 (socket says 172.21.15.78:40218) 2024-09-11T01:02:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bbed9d640 1 -- 172.21.15.78:0/3539367713 learned_addr learned my addr 172.21.15.78:0/3539367713 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bbe59c640 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cabc800 0x55722cadc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bbf59e640 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cac8790 0x55722cb44180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bbed9d640 1 -- 172.21.15.78:0/3539367713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cabc800 msgr2=0x55722cadc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cabc800 0x55722cadc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bbed9d640 1 -- 172.21.15.78:0/3539367713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cac8790 msgr2=0x55722cb44180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cac8790 0x55722cb44180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bbed9d640 1 -- 172.21.15.78:0/3539367713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55722cb44820 con 0x55722cabbee0 2024-09-11T01:02:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bbed9d640 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 0x55722cadc1c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00e3130 tx=0x7f6bb0005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1b42f17f245155 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6ba2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6ba2ff5640 1 -- 172.21.15.78:0/3539367713 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00ab7b0 con 0x55722cabbee0 2024-09-11T01:02:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6ba2ff5640 1 -- 172.21.15.78:0/3539367713 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6bb00eb400 con 0x55722cabbee0 2024-09-11T01:02:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bc7537740 1 -- 172.21.15.78:0/3539367713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bb8036050 con 0x55722cabbee0 2024-09-11T01:02:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6ba2ff5640 1 -- 172.21.15.78:0/3539367713 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6bb00c6030 con 0x55722cabbee0 2024-09-11T01:02:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bc7537740 10 client.16263 fetch_fsmap learned FSMap version 33 2024-09-11T01:02:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.935+0000 7f6bc7537740 1 -- 172.21.15.78:0/3539367713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55722cb44820 con 0x55722cabbee0 2024-09-11T01:02:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.936+0000 7f6ba2ff5640 1 -- 172.21.15.78:0/3539367713 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb8036050 con 0x55722cabbee0 2024-09-11T01:02:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.936+0000 7f6ba2ff5640 1 -- 172.21.15.78:0/3539367713 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f6bb00b4eb0 con 0x55722cabbee0 2024-09-11T01:02:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.936+0000 7f6bc7537740 10 client.16263 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T01:02:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.936+0000 7f6bc7537740 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.936+0000 7f6bc7537740 20 client.16263 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.936+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55722cad8610 0x55722cb47f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.936+0000 7f6bc7537740 4 client.16263 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.936+0000 7f6bc7537740 1 -- 172.21.15.78:0/3539367713 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55722ca30860 con 0x55722cad8610 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.936+0000 7f6bbe59c640 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55722cad8610 0x55722cb47f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.936+0000 7f6ba2ff5640 10 client.16263 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.936+0000 7f6bbe59c640 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55722cad8610 0x55722cb47f90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e477673b26f334 server_cookie=1b5a2f2b8e3542c8 in_seq=0 out_seq=0 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6ba2ff5640 1 -- 172.21.15.78:0/3539367713 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55722ca30860 con 0x55722cad8610 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6ba2ff5640 10 client.16263 handle_command_reply: tid=0 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6bc7537740 1 client.16263 shutdown 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6bc7537740 1 -- 172.21.15.78:0/3539367713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 msgr2=0x55722cadc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 0x55722cadc1c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00e3130 tx=0x7f6bb0005ab0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6bc7537740 1 -- 172.21.15.78:0/3539367713 shutdown_connections 2024-09-11T01:02:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55722cac8790 0x55722cb44180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55722cabc800 0x55722cadc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55722cad8610 0x55722cb47f90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6bc7537740 1 --2- 172.21.15.78:0/3539367713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55722cabbee0 0x55722cadc1c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6bc7537740 1 -- 172.21.15.78:0/3539367713 >> 172.21.15.78:0/3539367713 conn(0x55722cab9d10 msgr2=0x55722cad3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6ba2ff5640 0 client.16263 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6bc7537740 1 -- 172.21.15.78:0/3539367713 shutdown_connections 2024-09-11T01:02:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6bc7537740 1 -- 172.21.15.78:0/3539367713 wait complete. 2024-09-11T01:02:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.938+0000 7f6bc7537740 20 client.16263 trim_cache size 0 max 16384 2024-09-11T01:02:51.946 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:02:51.946 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:02:51.947 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "3aff237f-73e0-453f-ae2b-f1146929cd2f", 2024-09-11T01:02:51.947 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:02:51.947 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:02:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.939+0000 7f6bc5162640 1 -- 172.21.15.78:0/3079529420 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6ba80b5450 msgr2=0x7f6ba80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.939+0000 7f6bc5162640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6ba80b5450 0x7f6ba80b7910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6bb802f500 tx=0x7f6bb802f370 comp rx=0 tx=0).stop 2024-09-11T01:02:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.939+0000 7f6bc5162640 1 -- 172.21.15.78:0/3079529420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0102dc0 msgr2=0x7f6bc019b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.939+0000 7f6bc5162640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0102dc0 0x7f6bc019b420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000d770 tx=0x7f6bb000dc30 comp rx=0 tx=0).stop 2024-09-11T01:02:51.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.939+0000 7f6bc5162640 1 -- 172.21.15.78:0/3079529420 shutdown_connections 2024-09-11T01:02:51.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.939+0000 7f6bc5162640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6ba80b5450 0x7f6ba80b7910 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.939+0000 7f6bc5162640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc0109150 0x7f6bc01a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.939+0000 7f6bc5162640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bc01087d0 0x7f6bc019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.939+0000 7f6bc5162640 1 --2- 172.21.15.78:0/3079529420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0102dc0 0x7f6bc019b420 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:51.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.939+0000 7f6bc5162640 1 -- 172.21.15.78:0/3079529420 >> 172.21.15.78:0/3079529420 conn(0x7f6bc00fe830 msgr2=0x7f6bc0075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:51.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.939+0000 7f6bc5162640 1 -- 172.21.15.78:0/3079529420 shutdown_connections 2024-09-11T01:02:51.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:51.939+0000 7f6bc5162640 1 -- 172.21.15.78:0/3079529420 wait complete. 2024-09-11T01:02:52.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 340659 7.54 MB/sec execute 60 sec latency 1095.440 ms 2024-09-11T01:02:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:52 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:02:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:52 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:52 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:52 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:52 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:02:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:52 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:52 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:52 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:52 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:02:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:52 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:52 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:52 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:52.764 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:02:53.010 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:02:53.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 340659 7.41 MB/sec execute 61 sec latency 1338.707 ms 2024-09-11T01:02:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:53.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:02:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: pgmap v443: 97 pgs: 97 active+clean; 192 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 2.1 MiB/s wr, 126 op/s 2024-09-11T01:02:53.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:53 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:02:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:53.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:53.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:02:53.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: pgmap v443: 97 pgs: 97 active+clean; 192 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 2.1 MiB/s wr, 126 op/s 2024-09-11T01:02:53.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:53 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:02:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:02:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:02:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: pgmap v443: 97 pgs: 97 active+clean; 192 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 2.1 MiB/s wr, 126 op/s 2024-09-11T01:02:53.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:53 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:02:53.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.631+0000 7fdce7b67640 1 -- 172.21.15.78:0/259207565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce01087d0 msgr2=0x7fdce0108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:53.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.631+0000 7fdce7b67640 1 --2- 172.21.15.78:0/259207565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce01087d0 0x7fdce0108c10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009900 tx=0x7fdcd002a410 comp rx=0 tx=0).stop 2024-09-11T01:02:53.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.631+0000 7fdce7b67640 1 -- 172.21.15.78:0/259207565 shutdown_connections 2024-09-11T01:02:53.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.631+0000 7fdce7b67640 1 --2- 172.21.15.78:0/259207565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce0109150 0x7fdce010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.631+0000 7fdce7b67640 1 --2- 172.21.15.78:0/259207565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce01087d0 0x7fdce0108c10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.631+0000 7fdce7b67640 1 --2- 172.21.15.78:0/259207565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdce0102dc0 0x7fdce01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.631+0000 7fdce7b67640 1 -- 172.21.15.78:0/259207565 >> 172.21.15.78:0/259207565 conn(0x7fdce00fe830 msgr2=0x7fdce0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.631+0000 7fdce7b67640 1 -- 172.21.15.78:0/259207565 shutdown_connections 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.631+0000 7fdce7b67640 1 -- 172.21.15.78:0/259207565 wait complete. 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.632+0000 7fdce7b67640 1 Processor -- start 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.632+0000 7fdce7b67640 1 -- start start 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.632+0000 7fdce7b67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce0102dc0 0x7fdce0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.632+0000 7fdce7b67640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce01087d0 0x7fdce01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.632+0000 7fdce7b67640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdce0109150 0x7fdce019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.632+0000 7fdce7b67640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0112e10 con 0x7fdce01087d0 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.632+0000 7fdce7b67640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0112c90 con 0x7fdce0109150 2024-09-11T01:02:53.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.632+0000 7fdce7b67640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0112f90 con 0x7fdce0102dc0 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.632+0000 7fdce5102640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce01087d0 0x7fdce01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.632+0000 7fdce5102640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce01087d0 0x7fdce01997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42398/0 (socket says 172.21.15.78:42398) 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.632+0000 7fdce5102640 1 -- 172.21.15.78:0/4008851870 learned_addr learned my addr 172.21.15.78:0/4008851870 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.632+0000 7fdce5903640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce0102dc0 0x7fdce0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.633+0000 7fdce6104640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdce0109150 0x7fdce019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.633+0000 7fdce5102640 1 -- 172.21.15.78:0/4008851870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce0102dc0 msgr2=0x7fdce0199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.633+0000 7fdce5102640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce0102dc0 0x7fdce0199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.633+0000 7fdce5102640 1 -- 172.21.15.78:0/4008851870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdce0109150 msgr2=0x7fdce019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.633+0000 7fdce5102640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdce0109150 0x7fdce019eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.633+0000 7fdce5102640 1 -- 172.21.15.78:0/4008851870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce019f310 con 0x7fdce01087d0 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.633+0000 7fdce5102640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce01087d0 0x7fdce01997a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006010 tx=0x7fdcd00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.633+0000 7fdcd6ffd640 1 -- 172.21.15.78:0/4008851870 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd003f030 con 0x7fdce01087d0 2024-09-11T01:02:53.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.633+0000 7fdcd6ffd640 1 -- 172.21.15.78:0/4008851870 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd0003800 con 0x7fdce01087d0 2024-09-11T01:02:53.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.633+0000 7fdce7b67640 1 -- 172.21.15.78:0/4008851870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce019f5a0 con 0x7fdce01087d0 2024-09-11T01:02:53.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.633+0000 7fdcd6ffd640 1 -- 172.21.15.78:0/4008851870 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0035070 con 0x7fdce01087d0 2024-09-11T01:02:53.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.633+0000 7fdce7b67640 1 -- 172.21.15.78:0/4008851870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce01a5cc0 con 0x7fdce01087d0 2024-09-11T01:02:53.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.635+0000 7fdcd6ffd640 1 -- 172.21.15.78:0/4008851870 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fdcd0003cb0 con 0x7fdce01087d0 2024-09-11T01:02:53.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.635+0000 7fdce9f3c740 1 Processor -- start 2024-09-11T01:02:53.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce9f3c740 1 -- start start 2024-09-11T01:02:53.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce9f3c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56509346d0a0 0x56509346d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce9f3c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 0x565093479410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce9f3c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565093479950 0x56509347bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce9f3c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5650933e3e50 con 0x56509346d9c0 2024-09-11T01:02:53.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce9f3c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5650933d3280 con 0x565093479950 2024-09-11T01:02:53.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce9f3c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5650933d7090 con 0x56509346d0a0 2024-09-11T01:02:53.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce5903640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 0x565093479410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce5903640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 0x565093479410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42400/0 (socket says 172.21.15.78:42400) 2024-09-11T01:02:53.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce5903640 1 -- 172.21.15.78:0/4260628647 learned_addr learned my addr 172.21.15.78:0/4260628647 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:53.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce5102640 1 --2- 172.21.15.78:0/4260628647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565093479950 0x56509347bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce6104640 1 --2- 172.21.15.78:0/4260628647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56509346d0a0 0x56509346d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdcd6ffd640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdccc0b5450 0x7fdccc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce5903640 1 -- 172.21.15.78:0/4260628647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56509346d0a0 msgr2=0x56509346d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:53.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce5903640 1 --2- 172.21.15.78:0/4260628647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56509346d0a0 0x56509346d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce5903640 1 -- 172.21.15.78:0/4260628647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565093479950 msgr2=0x56509347bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce5903640 1 --2- 172.21.15.78:0/4260628647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565093479950 0x56509347bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.636+0000 7fdce5903640 1 -- 172.21.15.78:0/4260628647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565093131650 con 0x56509346d9c0 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.637+0000 7fdce6104640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdccc0b5450 0x7fdccc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.637+0000 7fdcd6ffd640 1 -- 172.21.15.78:0/4008851870 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdcd01068b0 con 0x7fdce01087d0 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.637+0000 7fdce5903640 1 --2- 172.21.15.78:0/4260628647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 0x565093479410 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00ad00 tx=0x7fdcdc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe166ac4eb3bec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.637+0000 7fdccbfff640 1 -- 172.21.15.78:0/4260628647 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc01d070 con 0x56509346d9c0 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.637+0000 7fdccbfff640 1 -- 172.21.15.78:0/4260628647 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc018af0 con 0x56509346d9c0 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.637+0000 7fdccbfff640 1 -- 172.21.15.78:0/4260628647 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc010b30 con 0x56509346d9c0 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.638+0000 7fdce9f3c740 1 -- 172.21.15.78:0/4260628647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 msgr2=0x565093479410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.638+0000 7fdce6104640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdccc0b5450 0x7fdccc0b7910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8007da0 tx=0x7fdcd8004b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.638+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/4260628647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 0x565093479410 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00ad00 tx=0x7fdcdc00f790 comp rx=0 tx=0).stop 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.638+0000 7fdce9f3c740 1 -- 172.21.15.78:0/4260628647 shutdown_connections 2024-09-11T01:02:53.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.638+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/4260628647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565093479950 0x56509347bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.638+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/4260628647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 0x565093479410 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.638+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/4260628647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56509346d0a0 0x56509346d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.638+0000 7fdce9f3c740 1 -- 172.21.15.78:0/4260628647 >> 172.21.15.78:0/4260628647 conn(0x56509346aed0 msgr2=0x56509346c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.638+0000 7fdce9f3c740 1 -- 172.21.15.78:0/4260628647 shutdown_connections 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.638+0000 7fdce9f3c740 1 -- 172.21.15.78:0/4260628647 wait complete. 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.638+0000 7fdce9f3c740 1 -- start start 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.638+0000 7fdce9f3c740 1 Processor -- start 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce9f3c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d0a0 0x565093479250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce9f3c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d9c0 0x5650934766a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce9f3c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565093479950 0x565093476cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce9f3c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x565093234700 con 0x56509346d0a0 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce9f3c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56509336e110 con 0x56509346d9c0 2024-09-11T01:02:53.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce9f3c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56509347c5d0 con 0x565093479950 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce9f3c740 10 client.0 _ll_register_callbacks cb 0x565093387990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce5903640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d0a0 0x565093479250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce5903640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d0a0 0x565093479250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42404/0 (socket says 172.21.15.78:42404) 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce5903640 1 -- 172.21.15.78:0/2701888654 learned_addr learned my addr 172.21.15.78:0/2701888654 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce6104640 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565093479950 0x565093476cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce5102640 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d9c0 0x5650934766a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce5903640 1 -- 172.21.15.78:0/2701888654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565093479950 msgr2=0x565093476cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce5903640 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565093479950 0x565093476cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce5903640 1 -- 172.21.15.78:0/2701888654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d9c0 msgr2=0x5650934766a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce5903640 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d9c0 0x5650934766a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce5903640 1 -- 172.21.15.78:0/2701888654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56509339bc40 con 0x56509346d0a0 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce5903640 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d0a0 0x565093479250 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc010470 tx=0x7fdcdc0186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499b4cdb3f399a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdcc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:53.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdcc97fa640 1 -- 172.21.15.78:0/2701888654 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc01d040 con 0x56509346d0a0 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdcc97fa640 1 -- 172.21.15.78:0/2701888654 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdcdc010870 con 0x56509346d0a0 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.639+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2701888654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcdc018af0 con 0x56509346d0a0 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.640+0000 7fdcc97fa640 1 -- 172.21.15.78:0/2701888654 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdcdc031050 con 0x56509346d0a0 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.640+0000 7fdce9f3c740 10 client.16275 fetch_fsmap learned FSMap version 34 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.640+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2701888654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x565093487f50 con 0x56509346d0a0 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.640+0000 7fdcc97fa640 1 -- 172.21.15.78:0/2701888654 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc018af0 con 0x56509346d0a0 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.640+0000 7fdcc97fa640 1 -- 172.21.15.78:0/2701888654 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8390+0+0 (secure 0 0 0) 0x7fdcdc020030 con 0x56509346d0a0 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.640+0000 7fdce9f3c740 10 client.16275 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.640+0000 7fdce9f3c740 10 client.16275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.640+0000 7fdce9f3c740 20 client.16275 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.640+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x565093489350 0x56509348b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.640+0000 7fdce9f3c740 4 client.16275 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:02:53.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.640+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2701888654 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650933e5da0 con 0x565093489350 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.640+0000 7fdce5102640 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x565093489350 0x56509348b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.641+0000 7fdcc97fa640 10 client.16275 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.641+0000 7fdce5102640 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x565093489350 0x56509348b740 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=685e7a2003cf608d server_cookie=94426f875a48f74a in_seq=0 out_seq=0 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.641+0000 7fdcc97fa640 1 -- 172.21.15.78:0/2701888654 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5650933e5da0 con 0x565093489350 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.641+0000 7fdcc97fa640 10 client.16275 handle_command_reply: tid=0 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdce9f3c740 1 client.16275 shutdown 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2701888654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d0a0 msgr2=0x565093479250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d0a0 0x565093479250 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc010470 tx=0x7fdcdc0186f0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2701888654 shutdown_connections 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565093479950 0x565093476cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d9c0 0x5650934766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x565093489350 0x56509348b740 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/2701888654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d0a0 0x565093479250 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdcc97fa640 0 client.16275 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2701888654 >> 172.21.15.78:0/2701888654 conn(0x56509346aed0 msgr2=0x565093470a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2701888654 shutdown_connections 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2701888654 wait complete. 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.642+0000 7fdce9f3c740 20 client.16275 trim_cache size 0 max 16384 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce9f3c740 1 Processor -- start 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce9f3c740 1 -- start start 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce9f3c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d0a0 0x565093485830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce9f3c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 0x565093474f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce9f3c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565093479950 0x565093475460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce9f3c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01ae5a0 con 0x56509346d9c0 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce9f3c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01b0230 con 0x56509346d0a0 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce9f3c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01aec70 con 0x565093479950 2024-09-11T01:02:53.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce5102640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 0x565093474f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce5102640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 0x565093474f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42412/0 (socket says 172.21.15.78:42412) 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce5102640 1 -- 172.21.15.78:0/1415603978 learned_addr learned my addr 172.21.15.78:0/1415603978 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce5102640 1 -- 172.21.15.78:0/1415603978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565093479950 msgr2=0x565093475460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce5102640 1 --2- 172.21.15.78:0/1415603978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565093479950 0x565093475460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce5102640 1 -- 172.21.15.78:0/1415603978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d0a0 msgr2=0x565093485830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.659+0000 7fdce5903640 1 --2- 172.21.15.78:0/1415603978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d0a0 0x565093485830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdce5102640 1 --2- 172.21.15.78:0/1415603978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d0a0 0x565093485830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdce5102640 1 -- 172.21.15.78:0/1415603978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565093466560 con 0x56509346d9c0 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdce5102640 1 --2- 172.21.15.78:0/1415603978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 0x565093474f20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00ccaa0 tx=0x7fdcd0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febf1fef709aa12c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdccb7fe640 1 -- 172.21.15.78:0/1415603978 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0049040 con 0x56509346d9c0 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdccb7fe640 1 -- 172.21.15.78:0/1415603978 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd00ccda0 con 0x56509346d9c0 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdccb7fe640 1 -- 172.21.15.78:0/1415603978 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd00c9010 con 0x56509346d9c0 2024-09-11T01:02:53.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdce9f3c740 1 -- 172.21.15.78:0/1415603978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 msgr2=0x565093474f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/1415603978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 0x565093474f20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00ccaa0 tx=0x7fdcd0003c20 comp rx=0 tx=0).stop 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdce9f3c740 1 -- 172.21.15.78:0/1415603978 shutdown_connections 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/1415603978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565093479950 0x565093475460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/1415603978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56509346d9c0 0x565093474f20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/1415603978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d0a0 0x565093485830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.660+0000 7fdce9f3c740 1 -- 172.21.15.78:0/1415603978 >> 172.21.15.78:0/1415603978 conn(0x56509346aed0 msgr2=0x56509346bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce9f3c740 1 -- 172.21.15.78:0/1415603978 shutdown_connections 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce9f3c740 1 -- 172.21.15.78:0/1415603978 wait complete. 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce9f3c740 1 -- start start 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce9f3c740 1 Processor -- start 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce9f3c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d0a0 0x565093481a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce9f3c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56509346d9c0 0x565093482000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce9f3c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565093479950 0x56509347c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce9f3c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01afad0 con 0x565093479950 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce9f3c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01af340 con 0x56509346d0a0 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce9f3c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01ac580 con 0x56509346d9c0 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce9f3c740 10 client.0 _ll_register_callbacks cb 0x7fdcdc01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce6104640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565093479950 0x56509347c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce6104640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565093479950 0x56509347c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42414/0 (socket says 172.21.15.78:42414) 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.661+0000 7fdce6104640 1 -- 172.21.15.78:0/2512546963 learned_addr learned my addr 172.21.15.78:0/2512546963 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdce5102640 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56509346d9c0 0x565093482000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdce5903640 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d0a0 0x565093481a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdce6104640 1 -- 172.21.15.78:0/2512546963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56509346d9c0 msgr2=0x565093482000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdce6104640 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56509346d9c0 0x565093482000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdce6104640 1 -- 172.21.15.78:0/2512546963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d0a0 msgr2=0x565093481a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdce6104640 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d0a0 0x565093481a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdce6104640 1 -- 172.21.15.78:0/2512546963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56509347d080 con 0x565093479950 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdce6104640 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565093479950 0x56509347c850 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8014b40 tx=0x7fdcd8014f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a01ee5c61898ec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdcc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdcc97fa640 1 -- 172.21.15.78:0/2512546963 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8024070 con 0x565093479950 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdcc97fa640 1 -- 172.21.15.78:0/2512546963 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdcd8020430 con 0x565093479950 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2512546963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd00ccda0 con 0x565093479950 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdcc97fa640 1 -- 172.21.15.78:0/2512546963 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdcd8030050 con 0x565093479950 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdce9f3c740 10 client.16281 fetch_fsmap learned FSMap version 34 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2512546963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5650934f6ae0 con 0x565093479950 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdcc97fa640 1 -- 172.21.15.78:0/2512546963 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd00ccda0 con 0x565093479950 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.662+0000 7fdcc97fa640 1 -- 172.21.15.78:0/2512546963 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8390+0+0 (secure 0 0 0) 0x7fdcd8028610 con 0x565093479950 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.663+0000 7fdce9f3c740 10 client.16281 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.663+0000 7fdce9f3c740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:02:53.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.663+0000 7fdce9f3c740 20 client.16281 populate_metadata read hostname 'smithi078' 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.663+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5650934f7350 0x5650934f9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.663+0000 7fdce9f3c740 4 client.16281 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.663+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2512546963 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650933e1850 con 0x5650934f7350 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.663+0000 7fdce5903640 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5650934f7350 0x5650934f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.663+0000 7fdce5903640 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5650934f7350 0x5650934f9770 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e1bc6636a9f5c3 server_cookie=9ba583a73039301e in_seq=0 out_seq=0 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.663+0000 7fdcc97fa640 10 client.16281 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.663+0000 7fdcc97fa640 1 -- 172.21.15.78:0/2512546963 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x5650933e1850 con 0x5650934f7350 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.663+0000 7fdcc97fa640 10 client.16281 handle_command_reply: tid=0 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.663+0000 7fdce9f3c740 1 client.16281 shutdown 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2512546963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565093479950 msgr2=0x56509347c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565093479950 0x56509347c850 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8014b40 tx=0x7fdcd8014f40 comp rx=0 tx=0).stop 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2512546963 shutdown_connections 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5650934f7350 0x5650934f9770 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565093479950 0x56509347c850 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56509346d9c0 0x565093482000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdcc97fa640 0 client.16281 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:02:53.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce9f3c740 1 --2- 172.21.15.78:0/2512546963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509346d0a0 0x565093481a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2512546963 >> 172.21.15.78:0/2512546963 conn(0x56509346aed0 msgr2=0x565093474ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:53.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2512546963 shutdown_connections 2024-09-11T01:02:53.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce9f3c740 1 -- 172.21.15.78:0/2512546963 wait complete. 2024-09-11T01:02:53.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce9f3c740 20 client.16281 trim_cache size 0 max 16384 2024-09-11T01:02:53.702 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "3aff237f-73e0-453f-ae2b-f1146929cd2f", 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:53.704 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:02:53.705 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:02:53.705 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:53.705 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:53.705 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:53.705 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:53.705 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:53.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:53.705 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:02:53.706 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:02:53.706 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:53.706 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:53.706 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:53.706 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:53.706 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:53.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:53.706 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:02:53.706 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:02:53.707 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:02:53.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:02:53.708 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:02:53.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce7b67640 1 -- 172.21.15.78:0/4008851870 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdccc0b5450 msgr2=0x7fdccc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:53.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce7b67640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdccc0b5450 0x7fdccc0b7910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8007da0 tx=0x7fdcd8004b50 comp rx=0 tx=0).stop 2024-09-11T01:02:53.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce7b67640 1 -- 172.21.15.78:0/4008851870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce01087d0 msgr2=0x7fdce01997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:02:53.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce7b67640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce01087d0 0x7fdce01997a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006010 tx=0x7fdcd00042f0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce7b67640 1 -- 172.21.15.78:0/4008851870 shutdown_connections 2024-09-11T01:02:53.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce7b67640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdccc0b5450 0x7fdccc0b7910 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce7b67640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdce0109150 0x7fdce019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce7b67640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce01087d0 0x7fdce01997a0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce7b67640 1 --2- 172.21.15.78:0/4008851870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce0102dc0 0x7fdce0199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:02:53.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.664+0000 7fdce7b67640 1 -- 172.21.15.78:0/4008851870 >> 172.21.15.78:0/4008851870 conn(0x7fdce00fe830 msgr2=0x7fdce0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:02:53.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.665+0000 7fdce7b67640 1 -- 172.21.15.78:0/4008851870 shutdown_connections 2024-09-11T01:02:53.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:02:53.665+0000 7fdce7b67640 1 -- 172.21.15.78:0/4008851870 wait complete. 2024-09-11T01:02:54.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 340670 7.30 MB/sec execute 62 sec latency 2328.911 ms 2024-09-11T01:02:54.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:54 smithi110 ceph-mon[34712]: pgmap v444: 97 pgs: 97 active+clean; 192 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.1 MiB/s wr, 140 op/s 2024-09-11T01:02:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:54 smithi112 ceph-mon[35384]: pgmap v444: 97 pgs: 97 active+clean; 192 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.1 MiB/s wr, 140 op/s 2024-09-11T01:02:54.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:54 smithi078 ceph-mon[31315]: pgmap v444: 97 pgs: 97 active+clean; 192 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.1 MiB/s wr, 140 op/s 2024-09-11T01:02:54.743 INFO:tasks.cephfs.filesystem:scrub status for tag:3aff237f-73e0-453f-ae2b-f1146929cd2f - {'path': '/', 'tag': '3aff237f-73e0-453f-ae2b-f1146929cd2f', 'options': 'recursive,force'} 2024-09-11T01:02:55.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 342908 7.31 MB/sec execute 63 sec latency 16.018 ms 2024-09-11T01:02:56.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 344137 7.36 MB/sec execute 64 sec latency 533.561 ms 2024-09-11T01:02:57.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 344137 7.25 MB/sec execute 65 sec latency 1533.605 ms 2024-09-11T01:02:57.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:57 smithi110 ceph-mon[34712]: pgmap v445: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 2.2 MiB/s wr, 147 op/s 2024-09-11T01:02:57.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:57 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:02:57.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:57 smithi112 ceph-mon[35384]: pgmap v445: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 2.2 MiB/s wr, 147 op/s 2024-09-11T01:02:57.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:57 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:02:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:57 smithi078 ceph-mon[31315]: pgmap v445: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 2.2 MiB/s wr, 147 op/s 2024-09-11T01:02:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:57 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:02:58.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 344137 7.14 MB/sec execute 66 sec latency 2533.680 ms 2024-09-11T01:02:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:58 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:02:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:58 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:02:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:58 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:02:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:58 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:02:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:58 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:02:58.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:02:58 smithi110 ceph-mon[34712]: pgmap v446: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 2.0 MiB/s wr, 127 op/s 2024-09-11T01:02:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:58 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:02:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:58 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:02:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:58 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:02:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:58 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:02:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:58 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:02:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:02:58 smithi112 ceph-mon[35384]: pgmap v446: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 2.0 MiB/s wr, 127 op/s 2024-09-11T01:02:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:58 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:02:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:58 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:02:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:58 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:02:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:58 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:02:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:58 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:02:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:02:58 smithi078 ceph-mon[31315]: pgmap v446: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 2.0 MiB/s wr, 127 op/s 2024-09-11T01:02:59.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 344148 7.03 MB/sec execute 67 sec latency 3522.258 ms 2024-09-11T01:03:00.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 344153 6.93 MB/sec execute 68 sec latency 995.882 ms 2024-09-11T01:03:01.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 344809 6.86 MB/sec execute 69 sec latency 1383.552 ms 2024-09-11T01:03:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:01 smithi110 ceph-mon[34712]: pgmap v447: 97 pgs: 97 active+clean; 196 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.5 MiB/s wr, 166 op/s 2024-09-11T01:03:01.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:01 smithi112 ceph-mon[35384]: pgmap v447: 97 pgs: 97 active+clean; 196 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.5 MiB/s wr, 166 op/s 2024-09-11T01:03:01.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:01 smithi078 ceph-mon[31315]: pgmap v447: 97 pgs: 97 active+clean; 196 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.5 MiB/s wr, 166 op/s 2024-09-11T01:03:02.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 347053 6.93 MB/sec execute 70 sec latency 14.868 ms 2024-09-11T01:03:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:02 smithi110 ceph-mon[34712]: pgmap v448: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.5 MiB/s wr, 160 op/s 2024-09-11T01:03:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:02 smithi112 ceph-mon[35384]: pgmap v448: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.5 MiB/s wr, 160 op/s 2024-09-11T01:03:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:02 smithi078 ceph-mon[31315]: pgmap v448: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.5 MiB/s wr, 160 op/s 2024-09-11T01:03:03.425 INFO:tasks.workunit.client.0.smithi078.stdout: 1 347617 6.90 MB/sec execute 71 sec latency 795.606 ms 2024-09-11T01:03:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:03 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:03:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:03 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:03:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:03 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:03:04.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 347681 6.81 MB/sec execute 72 sec latency 1781.992 ms 2024-09-11T01:03:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:04 smithi110 ceph-mon[34712]: pgmap v449: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.3 MiB/s wr, 162 op/s 2024-09-11T01:03:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:04 smithi112 ceph-mon[35384]: pgmap v449: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.3 MiB/s wr, 162 op/s 2024-09-11T01:03:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:04 smithi078 ceph-mon[31315]: pgmap v449: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.3 MiB/s wr, 162 op/s 2024-09-11T01:03:05.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 347698 6.72 MB/sec execute 73 sec latency 975.462 ms 2024-09-11T01:03:06.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 347698 6.63 MB/sec execute 74 sec latency 1975.554 ms 2024-09-11T01:03:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:06 smithi112 ceph-mon[35384]: pgmap v450: 97 pgs: 97 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.4 MiB/s wr, 185 op/s 2024-09-11T01:03:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:06 smithi078 ceph-mon[31315]: pgmap v450: 97 pgs: 97 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.4 MiB/s wr, 185 op/s 2024-09-11T01:03:06.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:06 smithi110 ceph-mon[34712]: pgmap v450: 97 pgs: 97 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.4 MiB/s wr, 185 op/s 2024-09-11T01:03:07.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 347698 6.54 MB/sec execute 75 sec latency 2975.648 ms 2024-09-11T01:03:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:03:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:03:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:03:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:03:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:03:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:03:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:03:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:03:07.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:03:07.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:03:07.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:03:07.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:03:08.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 347698 6.45 MB/sec execute 76 sec latency 3975.731 ms 2024-09-11T01:03:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:08 smithi078 ceph-mon[31315]: pgmap v451: 97 pgs: 97 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-09-11T01:03:08.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:08 smithi110 ceph-mon[34712]: pgmap v451: 97 pgs: 97 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-09-11T01:03:08.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:08 smithi112 ceph-mon[35384]: pgmap v451: 97 pgs: 97 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-09-11T01:03:09.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 347720 6.37 MB/sec execute 77 sec latency 4961.477 ms 2024-09-11T01:03:10.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 348204 6.31 MB/sec execute 78 sec latency 513.087 ms 2024-09-11T01:03:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:10 smithi112 ceph-mon[35384]: pgmap v452: 97 pgs: 97 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-09-11T01:03:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:10 smithi078 ceph-mon[31315]: pgmap v452: 97 pgs: 97 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-09-11T01:03:10.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:10 smithi110 ceph-mon[34712]: pgmap v452: 97 pgs: 97 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-09-11T01:03:11.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 348204 6.23 MB/sec execute 79 sec latency 1513.222 ms 2024-09-11T01:03:12.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 348204 6.15 MB/sec execute 80 sec latency 2513.296 ms 2024-09-11T01:03:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:12 smithi112 ceph-mon[35384]: pgmap v453: 97 pgs: 97 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-11T01:03:12.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:12 smithi078 ceph-mon[31315]: pgmap v453: 97 pgs: 97 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-11T01:03:12.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:12 smithi110 ceph-mon[34712]: pgmap v453: 97 pgs: 97 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-11T01:03:13.426 INFO:tasks.workunit.client.0.smithi078.stdout: 1 348204 6.07 MB/sec execute 81 sec latency 3513.349 ms 2024-09-11T01:03:14.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 348204 6.00 MB/sec execute 82 sec latency 4513.422 ms 2024-09-11T01:03:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:14 smithi112 ceph-mon[35384]: pgmap v454: 97 pgs: 97 active+clean; 196 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 978 KiB/s wr, 103 op/s 2024-09-11T01:03:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:14 smithi078 ceph-mon[31315]: pgmap v454: 97 pgs: 97 active+clean; 196 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 978 KiB/s wr, 103 op/s 2024-09-11T01:03:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:14 smithi110 ceph-mon[34712]: pgmap v454: 97 pgs: 97 active+clean; 196 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 978 KiB/s wr, 103 op/s 2024-09-11T01:03:15.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 348204 5.93 MB/sec execute 83 sec latency 5513.800 ms 2024-09-11T01:03:16.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 348204 5.86 MB/sec execute 84 sec latency 6513.926 ms 2024-09-11T01:03:16.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:16 smithi112 ceph-mon[35384]: pgmap v455: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 992 KiB/s wr, 94 op/s 2024-09-11T01:03:16.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:16 smithi078 ceph-mon[31315]: pgmap v455: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 992 KiB/s wr, 94 op/s 2024-09-11T01:03:16.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:16 smithi110 ceph-mon[34712]: pgmap v455: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 992 KiB/s wr, 94 op/s 2024-09-11T01:03:17.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 349314 5.85 MB/sec execute 85 sec latency 6934.838 ms 2024-09-11T01:03:18.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 351550 5.94 MB/sec execute 86 sec latency 16.297 ms 2024-09-11T01:03:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:18 smithi112 ceph-mon[35384]: pgmap v456: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 554 KiB/s wr, 37 op/s 2024-09-11T01:03:18.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:18 smithi078 ceph-mon[31315]: pgmap v456: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 554 KiB/s wr, 37 op/s 2024-09-11T01:03:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:18 smithi110 ceph-mon[34712]: pgmap v456: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 554 KiB/s wr, 37 op/s 2024-09-11T01:03:19.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 353588 5.96 MB/sec execute 87 sec latency 13.420 ms 2024-09-11T01:03:20.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 355368 6.02 MB/sec execute 88 sec latency 11.951 ms 2024-09-11T01:03:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:20 smithi112 ceph-mon[35384]: pgmap v457: 97 pgs: 97 active+clean; 201 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 779 KiB/s wr, 70 op/s 2024-09-11T01:03:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:20 smithi078 ceph-mon[31315]: pgmap v457: 97 pgs: 97 active+clean; 201 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 779 KiB/s wr, 70 op/s 2024-09-11T01:03:20.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:20 smithi110 ceph-mon[34712]: pgmap v457: 97 pgs: 97 active+clean; 201 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 779 KiB/s wr, 70 op/s 2024-09-11T01:03:21.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 357887 6.11 MB/sec execute 89 sec latency 15.372 ms 2024-09-11T01:03:22.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 359404 6.12 MB/sec execute 90 sec latency 14.675 ms 2024-09-11T01:03:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:22 smithi112 ceph-mon[35384]: pgmap v458: 97 pgs: 97 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 851 KiB/s wr, 77 op/s 2024-09-11T01:03:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:22 smithi078 ceph-mon[31315]: pgmap v458: 97 pgs: 97 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 851 KiB/s wr, 77 op/s 2024-09-11T01:03:22.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:22 smithi110 ceph-mon[34712]: pgmap v458: 97 pgs: 97 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 851 KiB/s wr, 77 op/s 2024-09-11T01:03:23.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 361871 6.23 MB/sec execute 91 sec latency 22.988 ms 2024-09-11T01:03:24.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 363353 6.23 MB/sec execute 92 sec latency 14.562 ms 2024-09-11T01:03:24.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:24 smithi112 ceph-mon[35384]: pgmap v459: 97 pgs: 97 active+clean; 207 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-09-11T01:03:24.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:24 smithi078 ceph-mon[31315]: pgmap v459: 97 pgs: 97 active+clean; 207 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-09-11T01:03:24.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:24 smithi110 ceph-mon[34712]: pgmap v459: 97 pgs: 97 active+clean; 207 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-09-11T01:03:24.744 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:03:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:03:25.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 365700 6.32 MB/sec execute 93 sec latency 14.168 ms 2024-09-11T01:03:25.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.487+0000 7fdd7a06b640 1 -- 172.21.15.78:0/3980804070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd741076a0 msgr2=0x7fdd74107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:25.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.487+0000 7fdd7a06b640 1 --2- 172.21.15.78:0/3980804070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd741076a0 0x7fdd74107ae0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009900 tx=0x7fdd6802a410 comp rx=0 tx=0).stop 2024-09-11T01:03:25.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.488+0000 7fdd7a06b640 1 -- 172.21.15.78:0/3980804070 shutdown_connections 2024-09-11T01:03:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.488+0000 7fdd7a06b640 1 --2- 172.21.15.78:0/3980804070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd74108020 0x7fdd7410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.488+0000 7fdd7a06b640 1 --2- 172.21.15.78:0/3980804070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd741076a0 0x7fdd74107ae0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.488+0000 7fdd7a06b640 1 --2- 172.21.15.78:0/3980804070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd74101c90 0x7fdd74102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.488+0000 7fdd7a06b640 1 -- 172.21.15.78:0/3980804070 >> 172.21.15.78:0/3980804070 conn(0x7fdd740fd700 msgr2=0x7fdd740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.488+0000 7fdd7a06b640 1 -- 172.21.15.78:0/3980804070 shutdown_connections 2024-09-11T01:03:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.488+0000 7fdd7a06b640 1 -- 172.21.15.78:0/3980804070 wait complete. 2024-09-11T01:03:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.488+0000 7fdd7a06b640 1 Processor -- start 2024-09-11T01:03:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd7a06b640 1 -- start start 2024-09-11T01:03:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd7a06b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd74101c90 0x7fdd741992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd7a06b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd741076a0 0x7fdd74199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd7a06b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd74108020 0x7fdd7419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd7a06b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd74111ca0 con 0x7fdd74108020 2024-09-11T01:03:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd7a06b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd74111b20 con 0x7fdd741076a0 2024-09-11T01:03:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd7a06b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdd74111e20 con 0x7fdd74101c90 2024-09-11T01:03:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd73fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd74108020 0x7fdd7419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd74101c90 0x7fdd741992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd73fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd74108020 0x7fdd7419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55856/0 (socket says 172.21.15.78:55856) 2024-09-11T01:03:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd72ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd741076a0 0x7fdd74199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd73fff640 1 -- 172.21.15.78:0/1756565284 learned_addr learned my addr 172.21.15.78:0/1756565284 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:25.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd73fff640 1 -- 172.21.15.78:0/1756565284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd74101c90 msgr2=0x7fdd741992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:25.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd73fff640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd74101c90 0x7fdd741992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.489+0000 7fdd73fff640 1 -- 172.21.15.78:0/1756565284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd741076a0 msgr2=0x7fdd74199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:25.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.490+0000 7fdd73fff640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd741076a0 0x7fdd74199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.490+0000 7fdd73fff640 1 -- 172.21.15.78:0/1756565284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7419f370 con 0x7fdd74108020 2024-09-11T01:03:25.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.490+0000 7fdd72ffd640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd741076a0 0x7fdd74199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:25.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.490+0000 7fdd73fff640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd74108020 0x7fdd7419eba0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00eb10 tx=0x7fdd6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.490+0000 7fdd70ff9640 1 -- 172.21.15.78:0/1756565284 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c004b20 con 0x7fdd74108020 2024-09-11T01:03:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.490+0000 7fdd7a06b640 1 -- 172.21.15.78:0/1756565284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd7419f660 con 0x7fdd74108020 2024-09-11T01:03:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.490+0000 7fdd70ff9640 1 -- 172.21.15.78:0/1756565284 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd6c004cc0 con 0x7fdd74108020 2024-09-11T01:03:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.490+0000 7fdd70ff9640 1 -- 172.21.15.78:0/1756565284 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c010690 con 0x7fdd74108020 2024-09-11T01:03:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.490+0000 7fdd7a06b640 1 -- 172.21.15.78:0/1756565284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd741a5ef0 con 0x7fdd74108020 2024-09-11T01:03:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd7c440740 1 Processor -- start 2024-09-11T01:03:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd70ff9640 1 -- 172.21.15.78:0/1756565284 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fdd6c005020 con 0x7fdd74108020 2024-09-11T01:03:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd7c440740 1 -- start start 2024-09-11T01:03:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd7c440740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b75480 0x563ce5b75860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd7c440740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd7c440740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b81d30 0x563ce5b84120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd7c440740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563ce5aebe50 con 0x563ce5b75da0 2024-09-11T01:03:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd7c440740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563ce5adb280 con 0x563ce5b75480 2024-09-11T01:03:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd7c440740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563ce5adf090 con 0x563ce5b81d30 2024-09-11T01:03:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd72ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b75480 0x563ce5b75860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd72ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b75480 0x563ce5b75860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:56046/0 (socket says 172.21.15.78:56046) 2024-09-11T01:03:25.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd72ffd640 1 -- 172.21.15.78:0/1126374119 learned_addr learned my addr 172.21.15.78:0/1126374119 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:25.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b817f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55872/0 (socket says 172.21.15.78:55872) 2024-09-11T01:03:25.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd72ffd640 1 --2- 172.21.15.78:0/1126374119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b81d30 0x563ce5b84120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd737fe640 1 -- 172.21.15.78:0/1126374119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b81d30 msgr2=0x563ce5b84120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:25.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd737fe640 1 --2- 172.21.15.78:0/1126374119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b81d30 0x563ce5b84120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd737fe640 1 -- 172.21.15.78:0/1126374119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b75480 msgr2=0x563ce5b75860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:25.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd737fe640 1 --2- 172.21.15.78:0/1126374119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b75480 0x563ce5b75860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd737fe640 1 -- 172.21.15.78:0/1126374119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce5839650 con 0x563ce5b75da0 2024-09-11T01:03:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.493+0000 7fdd70ff9640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdd5c0b5450 0x7fdd5c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.494+0000 7fdd72ffd640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdd5c0b5450 0x7fdd5c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.494+0000 7fdd737fe640 1 --2- 172.21.15.78:0/1126374119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b817f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009c20 tx=0x7fdd64011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de608743ad21d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.494+0000 7fdd70ff9640 1 -- 172.21.15.78:0/1756565284 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdd6c0e64f0 con 0x7fdd74108020 2024-09-11T01:03:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.494+0000 7fdd59ffb640 1 -- 172.21.15.78:0/1126374119 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64004050 con 0x563ce5b75da0 2024-09-11T01:03:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.494+0000 7fdd59ffb640 1 -- 172.21.15.78:0/1126374119 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd640054d0 con 0x563ce5b75da0 2024-09-11T01:03:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.494+0000 7fdd59ffb640 1 -- 172.21.15.78:0/1126374119 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64004d90 con 0x563ce5b75da0 2024-09-11T01:03:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.495+0000 7fdd7c440740 1 -- 172.21.15.78:0/1126374119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 msgr2=0x563ce5b817f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.495+0000 7fdd72ffd640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdd5c0b5450 0x7fdd5c0b7910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdd68005ec0 tx=0x7fdd68005780 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.495+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1126374119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b817f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009c20 tx=0x7fdd64011810 comp rx=0 tx=0).stop 2024-09-11T01:03:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.495+0000 7fdd7c440740 1 -- 172.21.15.78:0/1126374119 shutdown_connections 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.495+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1126374119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b81d30 0x563ce5b84120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.495+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1126374119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b817f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.495+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1126374119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b75480 0x563ce5b75860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.495+0000 7fdd7c440740 1 -- 172.21.15.78:0/1126374119 >> 172.21.15.78:0/1126374119 conn(0x563ce5b732b0 msgr2=0x563ce5b74450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.495+0000 7fdd7c440740 1 -- 172.21.15.78:0/1126374119 shutdown_connections 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.495+0000 7fdd7c440740 1 -- 172.21.15.78:0/1126374119 wait complete. 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.495+0000 7fdd7c440740 1 -- start start 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.495+0000 7fdd7c440740 1 Processor -- start 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd7c440740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75480 0x563ce5b815f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd7c440740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b75da0 0x563ce5b7ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd7c440740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b81d30 0x563ce5b7f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd7c440740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563ce593c700 con 0x563ce5b75480 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd7c440740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563ce5a76110 con 0x563ce5b75da0 2024-09-11T01:03:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd7c440740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563ce5b849b0 con 0x563ce5b81d30 2024-09-11T01:03:25.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd7c440740 10 client.0 _ll_register_callbacks cb 0x563ce5a8f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:25.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75480 0x563ce5b815f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75480 0x563ce5b815f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55880/0 (socket says 172.21.15.78:55880) 2024-09-11T01:03:25.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd737fe640 1 -- 172.21.15.78:0/739858749 learned_addr learned my addr 172.21.15.78:0/739858749 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:25.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd72ffd640 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b75da0 0x563ce5b7ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd73fff640 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b81d30 0x563ce5b7f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd737fe640 1 -- 172.21.15.78:0/739858749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b81d30 msgr2=0x563ce5b7f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:25.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd737fe640 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b81d30 0x563ce5b7f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd737fe640 1 -- 172.21.15.78:0/739858749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b75da0 msgr2=0x563ce5b7ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:25.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd737fe640 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b75da0 0x563ce5b7ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd737fe640 1 -- 172.21.15.78:0/739858749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce5aa3c40 con 0x563ce5b75480 2024-09-11T01:03:25.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd72ffd640 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b75da0 0x563ce5b7ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd737fe640 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75480 0x563ce5b815f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdd64000c00 tx=0x7fdd64004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72afe06426abc6ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd52ffd640 1 -- 172.21.15.78:0/739858749 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd640046c0 con 0x563ce5b75480 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd52ffd640 1 -- 172.21.15.78:0/739858749 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdd64005140 con 0x563ce5b75480 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.496+0000 7fdd7c440740 1 -- 172.21.15.78:0/739858749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd640054d0 con 0x563ce5b75480 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.497+0000 7fdd52ffd640 1 -- 172.21.15.78:0/739858749 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdd64032050 con 0x563ce5b75480 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.497+0000 7fdd7c440740 10 client.16293 fetch_fsmap learned FSMap version 36 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.497+0000 7fdd7c440740 1 -- 172.21.15.78:0/739858749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563ce5b902e0 con 0x563ce5b75480 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.497+0000 7fdd52ffd640 1 -- 172.21.15.78:0/739858749 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd640054d0 con 0x563ce5b75480 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.497+0000 7fdd52ffd640 1 -- 172.21.15.78:0/739858749 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fdd64018070 con 0x563ce5b75480 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.497+0000 7fdd7c440740 10 client.16293 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.497+0000 7fdd7c440740 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.497+0000 7fdd7c440740 20 client.16293 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:25.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.497+0000 7fdd7c440740 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ce5b916e0 0x563ce5b93ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.497+0000 7fdd7c440740 4 client.16293 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.497+0000 7fdd7c440740 1 -- 172.21.15.78:0/739858749 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ce5aedda0 con 0x563ce5b916e0 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.497+0000 7fdd72ffd640 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ce5b916e0 0x563ce5b93ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.498+0000 7fdd72ffd640 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ce5b916e0 0x563ce5b93ad0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d5da6336947ccf server_cookie=989bfb7feecb9b15 in_seq=0 out_seq=0 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.498+0000 7fdd52ffd640 10 client.16293 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.498+0000 7fdd52ffd640 1 -- 172.21.15.78:0/739858749 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563ce5aedda0 con 0x563ce5b916e0 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.498+0000 7fdd52ffd640 10 client.16293 handle_command_reply: tid=0 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.498+0000 7fdd7c440740 1 client.16293 shutdown 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.498+0000 7fdd7c440740 1 -- 172.21.15.78:0/739858749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75480 msgr2=0x563ce5b815f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.498+0000 7fdd7c440740 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75480 0x563ce5b815f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdd64000c00 tx=0x7fdd64004d40 comp rx=0 tx=0).stop 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.499+0000 7fdd7c440740 1 -- 172.21.15.78:0/739858749 shutdown_connections 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.499+0000 7fdd7c440740 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b81d30 0x563ce5b7f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.499+0000 7fdd7c440740 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b75da0 0x563ce5b7ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.499+0000 7fdd7c440740 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ce5b916e0 0x563ce5b93ad0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.499+0000 7fdd7c440740 1 --2- 172.21.15.78:0/739858749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75480 0x563ce5b815f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.499+0000 7fdd7c440740 1 -- 172.21.15.78:0/739858749 >> 172.21.15.78:0/739858749 conn(0x563ce5b732b0 msgr2=0x563ce5b78e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:25.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.499+0000 7fdd52ffd640 0 client.16293 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:25.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.499+0000 7fdd7c440740 1 -- 172.21.15.78:0/739858749 shutdown_connections 2024-09-11T01:03:25.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.499+0000 7fdd7c440740 1 -- 172.21.15.78:0/739858749 wait complete. 2024-09-11T01:03:25.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.499+0000 7fdd7c440740 20 client.16293 trim_cache size 0 max 16384 2024-09-11T01:03:25.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.516+0000 7fdd7c440740 1 Processor -- start 2024-09-11T01:03:25.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.516+0000 7fdd7c440740 1 -- start start 2024-09-11T01:03:25.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd7c440740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b75480 0x563ce5b8dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd7c440740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b83120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd7c440740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b81d30 0x563ce5b83660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd7c440740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741ae600 con 0x563ce5b75da0 2024-09-11T01:03:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd7c440740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741b0200 con 0x563ce5b81d30 2024-09-11T01:03:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd7c440740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741aecd0 con 0x563ce5b75480 2024-09-11T01:03:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b83120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b83120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55888/0 (socket says 172.21.15.78:55888) 2024-09-11T01:03:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd72ffd640 1 -- 172.21.15.78:0/1275798632 learned_addr learned my addr 172.21.15.78:0/1275798632 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd737fe640 1 --2- 172.21.15.78:0/1275798632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b75480 0x563ce5b8dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd73fff640 1 --2- 172.21.15.78:0/1275798632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b81d30 0x563ce5b83660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd72ffd640 1 -- 172.21.15.78:0/1275798632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b75480 msgr2=0x563ce5b8dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd72ffd640 1 --2- 172.21.15.78:0/1275798632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b75480 0x563ce5b8dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd72ffd640 1 -- 172.21.15.78:0/1275798632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b81d30 msgr2=0x563ce5b83660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd72ffd640 1 --2- 172.21.15.78:0/1275798632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b81d30 0x563ce5b83660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd72ffd640 1 -- 172.21.15.78:0/1275798632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce5b7e7f0 con 0x563ce5b75da0 2024-09-11T01:03:25.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd72ffd640 1 --2- 172.21.15.78:0/1275798632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b83120 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdd68031550 tx=0x7fdd68041070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e7c8df7ecbc8b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:25.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd597fa640 1 -- 172.21.15.78:0/1275798632 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68035040 con 0x563ce5b75da0 2024-09-11T01:03:25.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd597fa640 1 -- 172.21.15.78:0/1275798632 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd68003b60 con 0x563ce5b75da0 2024-09-11T01:03:25.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.517+0000 7fdd597fa640 1 -- 172.21.15.78:0/1275798632 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68057b80 con 0x563ce5b75da0 2024-09-11T01:03:25.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.518+0000 7fdd7c440740 1 -- 172.21.15.78:0/1275798632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 msgr2=0x563ce5b83120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:25.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.518+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1275798632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b83120 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdd68031550 tx=0x7fdd68041070 comp rx=0 tx=0).stop 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.518+0000 7fdd7c440740 1 -- 172.21.15.78:0/1275798632 shutdown_connections 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.518+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1275798632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b81d30 0x563ce5b83660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.518+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1275798632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b83120 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.518+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1275798632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b75480 0x563ce5b8dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.518+0000 7fdd7c440740 1 -- 172.21.15.78:0/1275798632 >> 172.21.15.78:0/1275798632 conn(0x563ce5b732b0 msgr2=0x563ce5b74300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.518+0000 7fdd7c440740 1 -- 172.21.15.78:0/1275798632 shutdown_connections 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.518+0000 7fdd7c440740 1 -- 172.21.15.78:0/1275798632 wait complete. 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.518+0000 7fdd7c440740 1 -- start start 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd7c440740 1 Processor -- start 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd7c440740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b75480 0x563ce5b7c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd7c440740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b89c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd7c440740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b81d30 0x563ce5b8a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd7c440740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741afb30 con 0x563ce5b75da0 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd7c440740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741af3a0 con 0x563ce5b81d30 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd7c440740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741ac5e0 con 0x563ce5b75480 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd7c440740 10 client.0 _ll_register_callbacks cb 0x7fdd6c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b89c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b75480 0x563ce5b7c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd73fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b81d30 0x563ce5b8a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b75480 0x563ce5b7c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:34910/0 (socket says 172.21.15.78:34910) 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd737fe640 1 -- 172.21.15.78:0/1787327904 learned_addr learned my addr 172.21.15.78:0/1787327904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd73fff640 1 -- 172.21.15.78:0/1787327904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b75480 msgr2=0x563ce5b7c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd73fff640 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b75480 0x563ce5b7c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd73fff640 1 -- 172.21.15.78:0/1787327904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 msgr2=0x563ce5b89c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.519+0000 7fdd73fff640 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b89c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd73fff640 1 -- 172.21.15.78:0/1787327904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce5b8a8a0 con 0x563ce5b81d30 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd737fe640 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b75480 0x563ce5b7c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd72ffd640 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b89c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd73fff640 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b81d30 0x563ce5b8a1b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0e3cb0 tx=0x7fdd6c00cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f658c76223b49727 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd52ffd640 1 -- 172.21.15.78:0/1787327904 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0a8150 con 0x563ce5b81d30 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd7c440740 1 -- 172.21.15.78:0/1787327904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd68003b60 con 0x563ce5b81d30 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd52ffd640 1 -- 172.21.15.78:0/1787327904 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdd6c0b5ce0 con 0x563ce5b81d30 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd52ffd640 1 -- 172.21.15.78:0/1787327904 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdd6c005550 con 0x563ce5b81d30 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd7c440740 10 client.25652 fetch_fsmap learned FSMap version 36 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd7c440740 1 -- 172.21.15.78:0/1787327904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563ce5bfe510 con 0x563ce5b81d30 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd52ffd640 1 -- 172.21.15.78:0/1787327904 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd68003b60 con 0x563ce5b81d30 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd52ffd640 1 -- 172.21.15.78:0/1787327904 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fdd6c0e60b0 con 0x563ce5b81d30 2024-09-11T01:03:25.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd7c440740 10 client.25652 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd7c440740 10 client.25652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd7c440740 20 client.25652 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.520+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ce5bfebf0 0x563ce5c01010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.521+0000 7fdd7c440740 4 client.25652 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.521+0000 7fdd7c440740 1 -- 172.21.15.78:0/1787327904 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ce5ae9850 con 0x563ce5bfebf0 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.521+0000 7fdd737fe640 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ce5bfebf0 0x563ce5c01010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.521+0000 7fdd52ffd640 10 client.25652 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.521+0000 7fdd737fe640 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ce5bfebf0 0x563ce5c01010 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc6e1b4eefd0b405 server_cookie=7785062d9e9ed47e in_seq=0 out_seq=0 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.521+0000 7fdd52ffd640 1 -- 172.21.15.78:0/1787327904 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x563ce5ae9850 con 0x563ce5bfebf0 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.521+0000 7fdd52ffd640 10 client.25652 handle_command_reply: tid=0 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.521+0000 7fdd7c440740 1 client.25652 shutdown 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7c440740 1 -- 172.21.15.78:0/1787327904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b81d30 msgr2=0x563ce5b8a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b81d30 0x563ce5b8a1b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0e3cb0 tx=0x7fdd6c00cee0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7c440740 1 -- 172.21.15.78:0/1787327904 shutdown_connections 2024-09-11T01:03:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ce5b81d30 0x563ce5b8a1b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ce5bfebf0 0x563ce5c01010 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ce5b75da0 0x563ce5b89c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7c440740 1 --2- 172.21.15.78:0/1787327904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ce5b75480 0x563ce5b7c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd52ffd640 0 client.25652 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7c440740 1 -- 172.21.15.78:0/1787327904 >> 172.21.15.78:0/1787327904 conn(0x563ce5b732b0 msgr2=0x563ce5b71bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7c440740 1 -- 172.21.15.78:0/1787327904 shutdown_connections 2024-09-11T01:03:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7c440740 1 -- 172.21.15.78:0/1787327904 wait complete. 2024-09-11T01:03:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7c440740 20 client.25652 trim_cache size 0 max 16384 2024-09-11T01:03:25.530 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:03:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:03:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:03:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:03:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:03:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:03:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:03:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:03:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:03:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:03:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:03:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:03:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:03:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:25.533 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:25.534 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:25.534 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:03:25.534 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:03:25.534 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:03:25.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7a06b640 1 -- 172.21.15.78:0/1756565284 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdd5c0b5450 msgr2=0x7fdd5c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:25.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7a06b640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdd5c0b5450 0x7fdd5c0b7910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdd68005ec0 tx=0x7fdd68005780 comp rx=0 tx=0).stop 2024-09-11T01:03:25.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7a06b640 1 -- 172.21.15.78:0/1756565284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd74108020 msgr2=0x7fdd7419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:25.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.522+0000 7fdd7a06b640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd74108020 0x7fdd7419eba0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00eb10 tx=0x7fdd6c00c2d0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.523+0000 7fdd7a06b640 1 -- 172.21.15.78:0/1756565284 shutdown_connections 2024-09-11T01:03:25.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.523+0000 7fdd7a06b640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdd5c0b5450 0x7fdd5c0b7910 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.523+0000 7fdd7a06b640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd74108020 0x7fdd7419eba0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.523+0000 7fdd7a06b640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd741076a0 0x7fdd74199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.523+0000 7fdd7a06b640 1 --2- 172.21.15.78:0/1756565284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd74101c90 0x7fdd741992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:25.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.523+0000 7fdd7a06b640 1 -- 172.21.15.78:0/1756565284 >> 172.21.15.78:0/1756565284 conn(0x7fdd740fd700 msgr2=0x7fdd741047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:25.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.523+0000 7fdd7a06b640 1 -- 172.21.15.78:0/1756565284 shutdown_connections 2024-09-11T01:03:25.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:25.523+0000 7fdd7a06b640 1 -- 172.21.15.78:0/1756565284 wait complete. 2024-09-11T01:03:26.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 367322 6.31 MB/sec execute 94 sec latency 16.174 ms 2024-09-11T01:03:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:26 smithi112 ceph-mon[35384]: pgmap v460: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-09-11T01:03:26.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:26 smithi078 ceph-mon[31315]: pgmap v460: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-09-11T01:03:26.601 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:03:26.601 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:03:26.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:26 smithi110 ceph-mon[34712]: pgmap v460: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-09-11T01:03:26.873 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:03:27.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.389+0000 7f10e087d640 1 -- 172.21.15.78:0/2258118985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc108020 msgr2=0x7f10dc10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:27.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.389+0000 7f10e087d640 1 --2- 172.21.15.78:0/2258118985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc108020 0x7f10dc10e820 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009ca0 tx=0x7f10d402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:03:27.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.389+0000 7f10e087d640 1 -- 172.21.15.78:0/2258118985 shutdown_connections 2024-09-11T01:03:27.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.389+0000 7f10e087d640 1 --2- 172.21.15.78:0/2258118985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc108020 0x7f10dc10e820 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:27.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.389+0000 7f10e087d640 1 --2- 172.21.15.78:0/2258118985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc1076a0 0x7f10dc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:27.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.389+0000 7f10e087d640 1 --2- 172.21.15.78:0/2258118985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc101c90 0x7f10dc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:27.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.389+0000 7f10e087d640 1 -- 172.21.15.78:0/2258118985 >> 172.21.15.78:0/2258118985 conn(0x7f10dc0fd700 msgr2=0x7f10dc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:27.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.389+0000 7f10e087d640 1 -- 172.21.15.78:0/2258118985 shutdown_connections 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.389+0000 7f10e087d640 1 -- 172.21.15.78:0/2258118985 wait complete. 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.390+0000 7f10e087d640 1 Processor -- start 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.390+0000 7f10e087d640 1 -- start start 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.390+0000 7f10e087d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc101c90 0x7f10dc19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.390+0000 7f10e087d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc1076a0 0x7f10dc19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.390+0000 7f10e087d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc108020 0x7f10dc1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.390+0000 7f10e087d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc111ca0 con 0x7f10dc1076a0 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.390+0000 7f10e087d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc111b20 con 0x7f10dc101c90 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.390+0000 7f10e087d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc111e20 con 0x7f10dc108020 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10d1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc1076a0 0x7f10dc19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10da59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc101c90 0x7f10dc19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10dad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc108020 0x7f10dc1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:27.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10d1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc1076a0 0x7f10dc19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55922/0 (socket says 172.21.15.78:55922) 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10d1d9b640 1 -- 172.21.15.78:0/2037597861 learned_addr learned my addr 172.21.15.78:0/2037597861 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10d1d9b640 1 -- 172.21.15.78:0/2037597861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc108020 msgr2=0x7f10dc1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10d1d9b640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc108020 0x7f10dc1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10d1d9b640 1 -- 172.21.15.78:0/2037597861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc101c90 msgr2=0x7f10dc19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10d1d9b640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc101c90 0x7f10dc19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10d1d9b640 1 -- 172.21.15.78:0/2037597861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc1a14e0 con 0x7f10dc1076a0 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10dad9d640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc108020 0x7f10dc1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10da59c640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc101c90 0x7f10dc19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10d1d9b640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc1076a0 0x7f10dc19b970 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f10c400e9f0 tx=0x7f10c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10d3fff640 1 -- 172.21.15.78:0/2037597861 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c400cd80 con 0x7f10dc1076a0 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10e087d640 1 -- 172.21.15.78:0/2037597861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc1a17d0 con 0x7f10dc1076a0 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10d3fff640 1 -- 172.21.15.78:0/2037597861 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10c40108e0 con 0x7f10dc1076a0 2024-09-11T01:03:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10d3fff640 1 -- 172.21.15.78:0/2037597861 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4018980 con 0x7f10dc1076a0 2024-09-11T01:03:27.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.391+0000 7f10e087d640 1 -- 172.21.15.78:0/2037597861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1a7e90 con 0x7f10dc1076a0 2024-09-11T01:03:27.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.393+0000 7f10e087d640 1 -- 172.21.15.78:0/2037597861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc102070 con 0x7f10dc1076a0 2024-09-11T01:03:27.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.394+0000 7f10d3fff640 1 -- 172.21.15.78:0/2037597861 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f10c4010430 con 0x7f10dc1076a0 2024-09-11T01:03:27.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.394+0000 7f10d3fff640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f10c80b5450 0x7f10c80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:27.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.395+0000 7f10da59c640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f10c80b5450 0x7f10c80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:27.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.395+0000 7f10d3fff640 1 -- 172.21.15.78:0/2037597861 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f10c40e6260 con 0x7f10dc1076a0 2024-09-11T01:03:27.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.395+0000 7f10da59c640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f10c80b5450 0x7f10c80b7910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009fc0 tx=0x7f10cc005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:27.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.399+0000 7f10d3fff640 1 -- 172.21.15.78:0/2037597861 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f10c40ac800 con 0x7f10dc1076a0 2024-09-11T01:03:27.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 369311 6.38 MB/sec execute 95 sec latency 27.846 ms 2024-09-11T01:03:27.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.638+0000 7f10e087d640 1 -- 172.21.15.78:0/2037597861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10dc1a81d0 con 0x7f10dc1076a0 2024-09-11T01:03:27.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.639+0000 7f10d3fff640 1 -- 172.21.15.78:0/2037597861 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11986 (secure 0 0 0) 0x7f10c40b1230 con 0x7f10dc1076a0 2024-09-11T01:03:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:03:27.642 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":36,"btime":"2024-09-11T01:03:02:162056+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:03:02.103905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:03:27.642 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 36 2024-09-11T01:03:27.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.641+0000 7f10e087d640 1 -- 172.21.15.78:0/2037597861 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f10c80b5450 msgr2=0x7f10c80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:27.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.642+0000 7f10e087d640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f10c80b5450 0x7f10c80b7910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009fc0 tx=0x7f10cc005f90 comp rx=0 tx=0).stop 2024-09-11T01:03:27.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.642+0000 7f10e087d640 1 -- 172.21.15.78:0/2037597861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc1076a0 msgr2=0x7f10dc19b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:27.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.642+0000 7f10e087d640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc1076a0 0x7f10dc19b970 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f10c400e9f0 tx=0x7f10c400eeb0 comp rx=0 tx=0).stop 2024-09-11T01:03:27.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.642+0000 7f10e087d640 1 -- 172.21.15.78:0/2037597861 shutdown_connections 2024-09-11T01:03:27.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.642+0000 7f10e087d640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f10c80b5450 0x7f10c80b7910 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:27.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.642+0000 7f10e087d640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc108020 0x7f10dc1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:27.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.642+0000 7f10e087d640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc1076a0 0x7f10dc19b970 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:27.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.642+0000 7f10e087d640 1 --2- 172.21.15.78:0/2037597861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc101c90 0x7f10dc19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:27.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.642+0000 7f10e087d640 1 -- 172.21.15.78:0/2037597861 >> 172.21.15.78:0/2037597861 conn(0x7f10dc0fd700 msgr2=0x7f10dc06aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:27.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.642+0000 7f10e087d640 1 -- 172.21.15.78:0/2037597861 shutdown_connections 2024-09-11T01:03:27.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:27.642+0000 7f10e087d640 1 -- 172.21.15.78:0/2037597861 wait complete. 2024-09-11T01:03:27.977 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:27 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2037597861' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:03:27.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:27 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2037597861' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:03:28.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:27 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2037597861' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:03:28.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 371376 6.40 MB/sec execute 96 sec latency 33.528 ms 2024-09-11T01:03:28.774 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:03:28.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:28 smithi112 ceph-mon[35384]: pgmap v461: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-09-11T01:03:28.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:28 smithi078 ceph-mon[31315]: pgmap v461: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-09-11T01:03:29.042 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:03:29.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:28 smithi110 ceph-mon[34712]: pgmap v461: 97 pgs: 97 active+clean; 208 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-09-11T01:03:29.427 INFO:tasks.workunit.client.0.smithi078.stdout: 1 373033 6.45 MB/sec execute 97 sec latency 22.079 ms 2024-09-11T01:03:29.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.520+0000 7f0c2df00640 1 -- 172.21.15.78:0/1638446805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c28102dc0 msgr2=0x7f0c281031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:29.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.520+0000 7f0c2df00640 1 --2- 172.21.15.78:0/1638446805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c28102dc0 0x7f0c281031a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009900 tx=0x7f0c1802a410 comp rx=0 tx=0).stop 2024-09-11T01:03:29.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.520+0000 7f0c2df00640 1 -- 172.21.15.78:0/1638446805 shutdown_connections 2024-09-11T01:03:29.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.520+0000 7f0c2df00640 1 --2- 172.21.15.78:0/1638446805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c28109150 0x7f0c2810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.520+0000 7f0c2df00640 1 --2- 172.21.15.78:0/1638446805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c281087d0 0x7f0c28108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.520+0000 7f0c2df00640 1 --2- 172.21.15.78:0/1638446805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c28102dc0 0x7f0c281031a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.520+0000 7f0c2df00640 1 -- 172.21.15.78:0/1638446805 >> 172.21.15.78:0/1638446805 conn(0x7f0c280fe870 msgr2=0x7f0c28100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:29.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.520+0000 7f0c2df00640 1 -- 172.21.15.78:0/1638446805 shutdown_connections 2024-09-11T01:03:29.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.520+0000 7f0c2df00640 1 -- 172.21.15.78:0/1638446805 wait complete. 2024-09-11T01:03:29.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c2df00640 1 Processor -- start 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c2df00640 1 -- start start 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c2df00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c28102dc0 0x7f0c2819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c2df00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c281087d0 0x7f0c2819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c2df00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c28109150 0x7f0c281a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c2df00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28112dd0 con 0x7f0c28102dc0 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c2df00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28112c50 con 0x7f0c28109150 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c2df00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28112f50 con 0x7f0c281087d0 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c28102dc0 0x7f0c2819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c28102dc0 0x7f0c2819b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55928/0 (socket says 172.21.15.78:55928) 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c277fe640 1 -- 172.21.15.78:0/3703618616 learned_addr learned my addr 172.21.15.78:0/3703618616 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c26ffd640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c281087d0 0x7f0c2819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.521+0000 7f0c27fff640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c28109150 0x7f0c281a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.522+0000 7f0c277fe640 1 -- 172.21.15.78:0/3703618616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c281087d0 msgr2=0x7f0c2819ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.522+0000 7f0c277fe640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c281087d0 0x7f0c2819ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.522+0000 7f0c277fe640 1 -- 172.21.15.78:0/3703618616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c28109150 msgr2=0x7f0c281a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.522+0000 7f0c277fe640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c28109150 0x7f0c281a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.522+0000 7f0c277fe640 1 -- 172.21.15.78:0/3703618616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c281a1570 con 0x7f0c28102dc0 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.522+0000 7f0c27fff640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c28109150 0x7f0c281a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.522+0000 7f0c277fe640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c28102dc0 0x7f0c2819b4c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005940 tx=0x7f0c18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.522+0000 7f0c24ff9640 1 -- 172.21.15.78:0/3703618616 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1803f030 con 0x7f0c28102dc0 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.522+0000 7f0c2df00640 1 -- 172.21.15.78:0/3703618616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c281a1800 con 0x7f0c28102dc0 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.522+0000 7f0c2df00640 1 -- 172.21.15.78:0/3703618616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c281a7f20 con 0x7f0c28102dc0 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.522+0000 7f0c24ff9640 1 -- 172.21.15.78:0/3703618616 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c18003aa0 con 0x7f0c28102dc0 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.522+0000 7f0c24ff9640 1 -- 172.21.15.78:0/3703618616 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18049060 con 0x7f0c28102dc0 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c302d5740 1 Processor -- start 2024-09-11T01:03:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c302d5740 1 -- start start 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c24ff9640 1 -- 172.21.15.78:0/3703618616 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0c1804c060 con 0x7f0c28102dc0 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c302d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 0x5580314418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c302d5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031441df0 0x55803144d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c302d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 0x558031450170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c302d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5580313b7ea0 con 0x5580314414d0 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c302d5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5580313a72d0 con 0x558031441df0 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c302d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5580313ab0e0 con 0x55803144dd80 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 0x5580314418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 0x5580314418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55940/0 (socket says 172.21.15.78:55940) 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c27fff640 1 -- 172.21.15.78:0/2054404478 learned_addr learned my addr 172.21.15.78:0/2054404478 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c277fe640 1 --2- 172.21.15.78:0/2054404478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031441df0 0x55803144d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.525+0000 7f0c26ffd640 1 --2- 172.21.15.78:0/2054404478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 0x558031450170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.526+0000 7f0c27fff640 1 -- 172.21.15.78:0/2054404478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 msgr2=0x558031450170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:29.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.526+0000 7f0c27fff640 1 --2- 172.21.15.78:0/2054404478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 0x558031450170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.526+0000 7f0c27fff640 1 -- 172.21.15.78:0/2054404478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031441df0 msgr2=0x55803144d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:29.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.526+0000 7f0c27fff640 1 --2- 172.21.15.78:0/2054404478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031441df0 0x55803144d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.526+0000 7f0c27fff640 1 -- 172.21.15.78:0/2054404478 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558031105650 con 0x5580314414d0 2024-09-11T01:03:29.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.526+0000 7f0c27fff640 1 --2- 172.21.15.78:0/2054404478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 0x5580314418b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009fd0 tx=0x7f0c20010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae487275738f737 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:29.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.526+0000 7f0c24ff9640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c100b54d0 0x7f0c100b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.526+0000 7f0c0dffb640 1 -- 172.21.15.78:0/2054404478 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20011700 con 0x5580314414d0 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.526+0000 7f0c277fe640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c100b54d0 0x7f0c100b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.526+0000 7f0c0dffb640 1 -- 172.21.15.78:0/2054404478 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c20011d50 con 0x5580314414d0 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.526+0000 7f0c0dffb640 1 -- 172.21.15.78:0/2054404478 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2000e3c0 con 0x5580314414d0 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.526+0000 7f0c24ff9640 1 -- 172.21.15.78:0/3703618616 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0c18106020 con 0x7f0c28102dc0 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.527+0000 7f0c302d5740 1 -- 172.21.15.78:0/2054404478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 msgr2=0x5580314418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.527+0000 7f0c302d5740 1 --2- 172.21.15.78:0/2054404478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 0x5580314418b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009fd0 tx=0x7f0c20010860 comp rx=0 tx=0).stop 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.527+0000 7f0c277fe640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c100b54d0 0x7f0c100b7990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0c18103890 tx=0x7f0c1810b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.527+0000 7f0c302d5740 1 -- 172.21.15.78:0/2054404478 shutdown_connections 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.527+0000 7f0c302d5740 1 --2- 172.21.15.78:0/2054404478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 0x558031450170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.527+0000 7f0c302d5740 1 --2- 172.21.15.78:0/2054404478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031441df0 0x55803144d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.527+0000 7f0c302d5740 1 --2- 172.21.15.78:0/2054404478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 0x5580314418b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.527+0000 7f0c302d5740 1 -- 172.21.15.78:0/2054404478 >> 172.21.15.78:0/2054404478 conn(0x55803143f300 msgr2=0x5580314404a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.527+0000 7f0c302d5740 1 -- 172.21.15.78:0/2054404478 shutdown_connections 2024-09-11T01:03:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.527+0000 7f0c302d5740 1 -- 172.21.15.78:0/2054404478 wait complete. 2024-09-11T01:03:29.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c302d5740 1 -- start start 2024-09-11T01:03:29.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c302d5740 1 Processor -- start 2024-09-11T01:03:29.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c302d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580314414d0 0x55803144d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c302d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558031441df0 0x55803144aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c302d5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55803144dd80 0x55803144b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c302d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558031208700 con 0x558031441df0 2024-09-11T01:03:29.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c302d5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558031342160 con 0x55803144dd80 2024-09-11T01:03:29.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c302d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558031450a00 con 0x5580314414d0 2024-09-11T01:03:29.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c302d5740 10 client.0 _ll_register_callbacks cb 0x55803135b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:29.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c26ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558031441df0 0x55803144aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c26ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558031441df0 0x55803144aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55948/0 (socket says 172.21.15.78:55948) 2024-09-11T01:03:29.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c26ffd640 1 -- 172.21.15.78:0/2162297874 learned_addr learned my addr 172.21.15.78:0/2162297874 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.528+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580314414d0 0x55803144d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c27fff640 1 --2- 172.21.15.78:0/2162297874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55803144dd80 0x55803144b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c26ffd640 1 -- 172.21.15.78:0/2162297874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580314414d0 msgr2=0x55803144d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c26ffd640 1 --2- 172.21.15.78:0/2162297874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580314414d0 0x55803144d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c26ffd640 1 -- 172.21.15.78:0/2162297874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55803144dd80 msgr2=0x55803144b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c26ffd640 1 --2- 172.21.15.78:0/2162297874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55803144dd80 0x55803144b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c26ffd640 1 -- 172.21.15.78:0/2162297874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55803136fc90 con 0x558031441df0 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c26ffd640 1 --2- 172.21.15.78:0/2162297874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558031441df0 0x55803144aab0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00bf10 tx=0x7f0c1c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b771afee349af7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c06ffd640 1 -- 172.21.15.78:0/2162297874 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c005220 con 0x558031441df0 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c06ffd640 1 -- 172.21.15.78:0/2162297874 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0c1c0053c0 con 0x558031441df0 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c302d5740 1 -- 172.21.15.78:0/2162297874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c20011d50 con 0x558031441df0 2024-09-11T01:03:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c06ffd640 1 -- 172.21.15.78:0/2162297874 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0c1c005600 con 0x558031441df0 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c302d5740 10 client.16317 fetch_fsmap learned FSMap version 36 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c302d5740 1 -- 172.21.15.78:0/2162297874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55803145c410 con 0x558031441df0 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.529+0000 7f0c06ffd640 1 -- 172.21.15.78:0/2162297874 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c20011d50 con 0x558031441df0 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.530+0000 7f0c06ffd640 1 -- 172.21.15.78:0/2162297874 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f0c1c010c10 con 0x558031441df0 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.530+0000 7f0c302d5740 10 client.16317 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.530+0000 7f0c302d5740 10 client.16317 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.530+0000 7f0c302d5740 20 client.16317 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.530+0000 7f0c302d5740 1 --2- 172.21.15.78:0/2162297874 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55803145da10 0x55803145fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.530+0000 7f0c302d5740 4 client.16317 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.530+0000 7f0c302d5740 1 -- 172.21.15.78:0/2162297874 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580313b9df0 con 0x55803145da10 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.530+0000 7f0c27fff640 1 --2- 172.21.15.78:0/2162297874 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55803145da10 0x55803145fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.530+0000 7f0c27fff640 1 --2- 172.21.15.78:0/2162297874 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55803145da10 0x55803145fe30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=165e5a323f6e42b2 server_cookie=3968402ff0fbdc0f in_seq=0 out_seq=0 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.530+0000 7f0c06ffd640 10 client.16317 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:03:29.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c06ffd640 1 -- 172.21.15.78:0/2162297874 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5580313b9df0 con 0x55803145da10 2024-09-11T01:03:29.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c06ffd640 10 client.16317 handle_command_reply: tid=0 2024-09-11T01:03:29.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c302d5740 1 client.16317 shutdown 2024-09-11T01:03:29.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c302d5740 1 -- 172.21.15.78:0/2162297874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558031441df0 msgr2=0x55803144aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:29.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c302d5740 1 --2- 172.21.15.78:0/2162297874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558031441df0 0x55803144aab0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00bf10 tx=0x7f0c1c012820 comp rx=0 tx=0).stop 2024-09-11T01:03:29.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c302d5740 1 -- 172.21.15.78:0/2162297874 shutdown_connections 2024-09-11T01:03:29.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c302d5740 1 --2- 172.21.15.78:0/2162297874 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55803145da10 0x55803145fe30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c302d5740 1 --2- 172.21.15.78:0/2162297874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55803144dd80 0x55803144b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c302d5740 1 --2- 172.21.15.78:0/2162297874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558031441df0 0x55803144aab0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c302d5740 1 --2- 172.21.15.78:0/2162297874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5580314414d0 0x55803144d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c302d5740 1 -- 172.21.15.78:0/2162297874 >> 172.21.15.78:0/2162297874 conn(0x55803143f300 msgr2=0x558031444e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:29.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c06ffd640 0 client.16317 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:03:29.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c302d5740 1 -- 172.21.15.78:0/2162297874 shutdown_connections 2024-09-11T01:03:29.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c302d5740 1 -- 172.21.15.78:0/2162297874 wait complete. 2024-09-11T01:03:29.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.531+0000 7f0c302d5740 20 client.16317 trim_cache size 0 max 16384 2024-09-11T01:03:29.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.548+0000 7f0c302d5740 1 Processor -- start 2024-09-11T01:03:29.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.548+0000 7f0c302d5740 1 -- start start 2024-09-11T01:03:29.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c302d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 0x558031459c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c302d5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031441df0 0x55803143ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c302d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 0x558031447cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c302d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1400e5d0 con 0x5580314414d0 2024-09-11T01:03:29.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c302d5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140101d0 con 0x558031441df0 2024-09-11T01:03:29.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c302d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1400eca0 con 0x55803144dd80 2024-09-11T01:03:29.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 0x558031459c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c26ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031441df0 0x55803143ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 0x558031459c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55962/0 (socket says 172.21.15.78:55962) 2024-09-11T01:03:29.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c27fff640 1 -- 172.21.15.78:0/974413875 learned_addr learned my addr 172.21.15.78:0/974413875 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:29.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c27fff640 1 --2- 172.21.15.78:0/974413875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 0x558031447cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c27fff640 1 -- 172.21.15.78:0/974413875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 msgr2=0x558031447cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:29.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c27fff640 1 --2- 172.21.15.78:0/974413875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 0x558031447cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c27fff640 1 -- 172.21.15.78:0/974413875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031441df0 msgr2=0x55803143ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:29.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c27fff640 1 --2- 172.21.15.78:0/974413875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031441df0 0x55803143ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c27fff640 1 -- 172.21.15.78:0/974413875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558031446510 con 0x5580314414d0 2024-09-11T01:03:29.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.549+0000 7f0c27fff640 1 --2- 172.21.15.78:0/974413875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 0x558031459c90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0c2001bbd0 tx=0x7f0c20010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdbdc4f194d2c65 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:29.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.550+0000 7f0c0d7fa640 1 -- 172.21.15.78:0/974413875 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20024040 con 0x5580314414d0 2024-09-11T01:03:29.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.550+0000 7f0c0d7fa640 1 -- 172.21.15.78:0/974413875 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c20011bb0 con 0x5580314414d0 2024-09-11T01:03:29.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.550+0000 7f0c0d7fa640 1 -- 172.21.15.78:0/974413875 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2002da10 con 0x5580314414d0 2024-09-11T01:03:29.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.550+0000 7f0c302d5740 1 -- 172.21.15.78:0/974413875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 msgr2=0x558031459c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:29.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.550+0000 7f0c302d5740 1 --2- 172.21.15.78:0/974413875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 0x558031459c90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0c2001bbd0 tx=0x7f0c20010fd0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.550+0000 7f0c302d5740 1 -- 172.21.15.78:0/974413875 shutdown_connections 2024-09-11T01:03:29.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.550+0000 7f0c302d5740 1 --2- 172.21.15.78:0/974413875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 0x558031447cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.550+0000 7f0c302d5740 1 --2- 172.21.15.78:0/974413875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031441df0 0x55803143ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.550+0000 7f0c302d5740 1 --2- 172.21.15.78:0/974413875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5580314414d0 0x558031459c90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.550+0000 7f0c302d5740 1 -- 172.21.15.78:0/974413875 >> 172.21.15.78:0/974413875 conn(0x55803143f300 msgr2=0x5580314403c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.550+0000 7f0c302d5740 1 -- 172.21.15.78:0/974413875 shutdown_connections 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.551+0000 7f0c302d5740 1 -- 172.21.15.78:0/974413875 wait complete. 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.551+0000 7f0c302d5740 1 -- start start 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.551+0000 7f0c302d5740 1 Processor -- start 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.551+0000 7f0c302d5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5580314414d0 0x558031451ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.551+0000 7f0c302d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558031441df0 0x558031452530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c302d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 0x5580314c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c302d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1400fb00 con 0x558031441df0 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c302d5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1400f370 con 0x5580314414d0 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c302d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1400c5b0 con 0x55803144dd80 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c302d5740 10 client.0 _ll_register_callbacks cb 0x7f0c1c00bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:29.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5580314414d0 0x558031451ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c26ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558031441df0 0x558031452530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5580314414d0 0x558031451ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:56142/0 (socket says 172.21.15.78:56142) 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c27fff640 1 -- 172.21.15.78:0/3610883333 learned_addr learned my addr 172.21.15.78:0/3610883333 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c27fff640 1 --2- 172.21.15.78:0/3610883333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 0x5580314c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c27fff640 1 -- 172.21.15.78:0/3610883333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 msgr2=0x5580314c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c27fff640 1 --2- 172.21.15.78:0/3610883333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 0x5580314c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c27fff640 1 -- 172.21.15.78:0/3610883333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558031441df0 msgr2=0x558031452530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c27fff640 1 --2- 172.21.15.78:0/3610883333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558031441df0 0x558031452530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c27fff640 1 -- 172.21.15.78:0/3610883333 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580314c9e40 con 0x5580314414d0 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c27fff640 1 --2- 172.21.15.78:0/3610883333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5580314414d0 0x558031451ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c20014000 tx=0x7f0c2001bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b8b0581da01ddf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c06ffd640 1 -- 172.21.15.78:0/3610883333 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20004bc0 con 0x5580314414d0 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c06ffd640 1 -- 172.21.15.78:0/3610883333 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0c20004d60 con 0x5580314414d0 2024-09-11T01:03:29.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.552+0000 7f0c302d5740 1 -- 172.21.15.78:0/3610883333 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c20011bb0 con 0x5580314414d0 2024-09-11T01:03:29.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.553+0000 7f0c06ffd640 1 -- 172.21.15.78:0/3610883333 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0c2001b470 con 0x5580314414d0 2024-09-11T01:03:29.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.553+0000 7f0c302d5740 10 client.25676 fetch_fsmap learned FSMap version 36 2024-09-11T01:03:29.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.553+0000 7f0c302d5740 1 -- 172.21.15.78:0/3610883333 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5580314cafe0 con 0x5580314414d0 2024-09-11T01:03:29.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.553+0000 7f0c06ffd640 1 -- 172.21.15.78:0/3610883333 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c20011bb0 con 0x5580314414d0 2024-09-11T01:03:29.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.553+0000 7f0c06ffd640 1 -- 172.21.15.78:0/3610883333 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f0c2002a040 con 0x5580314414d0 2024-09-11T01:03:29.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.553+0000 7f0c302d5740 10 client.25676 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T01:03:29.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.553+0000 7f0c302d5740 10 client.25676 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:03:29.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.553+0000 7f0c302d5740 20 client.25676 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:29.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.553+0000 7f0c302d5740 1 --2- 172.21.15.78:0/3610883333 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5580314cb4c0 0x5580314cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:29.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.553+0000 7f0c302d5740 4 client.25676 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:03:29.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.553+0000 7f0c302d5740 1 -- 172.21.15.78:0/3610883333 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580313b58a0 con 0x5580314cb4c0 2024-09-11T01:03:29.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.553+0000 7f0c26ffd640 1 --2- 172.21.15.78:0/3610883333 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5580314cb4c0 0x5580314cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:29.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c06ffd640 10 client.25676 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:03:29.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c26ffd640 1 --2- 172.21.15.78:0/3610883333 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5580314cb4c0 0x5580314cd8e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e6a22fd9a961e014 server_cookie=e2e6e6383e90ef11 in_seq=0 out_seq=0 2024-09-11T01:03:29.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c06ffd640 1 -- 172.21.15.78:0/3610883333 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580313b58a0 con 0x5580314cb4c0 2024-09-11T01:03:29.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c06ffd640 10 client.25676 handle_command_reply: tid=0 2024-09-11T01:03:29.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c302d5740 1 client.25676 shutdown 2024-09-11T01:03:29.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c302d5740 1 -- 172.21.15.78:0/3610883333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5580314414d0 msgr2=0x558031451ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:29.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c302d5740 1 --2- 172.21.15.78:0/3610883333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5580314414d0 0x558031451ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c20014000 tx=0x7f0c2001bc00 comp rx=0 tx=0).stop 2024-09-11T01:03:29.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c302d5740 1 -- 172.21.15.78:0/3610883333 shutdown_connections 2024-09-11T01:03:29.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c302d5740 1 --2- 172.21.15.78:0/3610883333 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5580314cb4c0 0x5580314cd8e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c302d5740 1 --2- 172.21.15.78:0/3610883333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55803144dd80 0x5580314c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c302d5740 1 --2- 172.21.15.78:0/3610883333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558031441df0 0x558031452530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c302d5740 1 --2- 172.21.15.78:0/3610883333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5580314414d0 0x558031451ff0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c302d5740 1 -- 172.21.15.78:0/3610883333 >> 172.21.15.78:0/3610883333 conn(0x55803143f300 msgr2=0x55803144c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:29.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c06ffd640 0 client.25676 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:03:29.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.554+0000 7f0c302d5740 1 -- 172.21.15.78:0/3610883333 shutdown_connections 2024-09-11T01:03:29.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.555+0000 7f0c302d5740 1 -- 172.21.15.78:0/3610883333 wait complete. 2024-09-11T01:03:29.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.555+0000 7f0c302d5740 20 client.25676 trim_cache size 0 max 16384 2024-09-11T01:03:29.562 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:03:29.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.555+0000 7f0c2df00640 1 -- 172.21.15.78:0/3703618616 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c100b54d0 msgr2=0x7f0c100b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:29.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.555+0000 7f0c2df00640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c100b54d0 0x7f0c100b7990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0c18103890 tx=0x7f0c1810b040 comp rx=0 tx=0).stop 2024-09-11T01:03:29.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.555+0000 7f0c2df00640 1 -- 172.21.15.78:0/3703618616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c28102dc0 msgr2=0x7f0c2819b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:29.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.555+0000 7f0c2df00640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c28102dc0 0x7f0c2819b4c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005940 tx=0x7f0c18004270 comp rx=0 tx=0).stop 2024-09-11T01:03:29.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.555+0000 7f0c2df00640 1 -- 172.21.15.78:0/3703618616 shutdown_connections 2024-09-11T01:03:29.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.555+0000 7f0c2df00640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c100b54d0 0x7f0c100b7990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.555+0000 7f0c2df00640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c28109150 0x7f0c281a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.555+0000 7f0c2df00640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c281087d0 0x7f0c2819ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.555+0000 7f0c2df00640 1 --2- 172.21.15.78:0/3703618616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c28102dc0 0x7f0c2819b4c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:29.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.555+0000 7f0c2df00640 1 -- 172.21.15.78:0/3703618616 >> 172.21.15.78:0/3703618616 conn(0x7f0c280fe870 msgr2=0x7f0c28104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:29.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.556+0000 7f0c2df00640 1 -- 172.21.15.78:0/3703618616 shutdown_connections 2024-09-11T01:03:29.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:29.556+0000 7f0c2df00640 1 -- 172.21.15.78:0/3703618616 wait complete. 2024-09-11T01:03:30.429 INFO:tasks.workunit.client.0.smithi078.stdout: 1 374821 6.46 MB/sec execute 98 sec latency 14.736 ms 2024-09-11T01:03:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:30 smithi112 ceph-mon[35384]: pgmap v462: 97 pgs: 97 active+clean; 213 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 269 op/s 2024-09-11T01:03:30.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:30 smithi078 ceph-mon[31315]: pgmap v462: 97 pgs: 97 active+clean; 213 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 269 op/s 2024-09-11T01:03:30.539 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:03:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:30 smithi110 ceph-mon[34712]: pgmap v462: 97 pgs: 97 active+clean; 213 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 269 op/s 2024-09-11T01:03:30.809 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:03:31.428 INFO:tasks.workunit.client.0.smithi078.stdout: 1 376454 6.51 MB/sec execute 99 sec latency 28.347 ms 2024-09-11T01:03:31.636 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:03:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.595+0000 7f5a168f4640 1 -- 172.21.15.78:0/2580892582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101045c0 msgr2=0x7f5a1010cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.595+0000 7f5a168f4640 1 --2- 172.21.15.78:0/2580892582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101045c0 0x7f5a1010cab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009900 tx=0x7f5a0402a410 comp rx=0 tx=0).stop 2024-09-11T01:03:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.595+0000 7f5a168f4640 1 -- 172.21.15.78:0/2580892582 shutdown_connections 2024-09-11T01:03:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.595+0000 7f5a168f4640 1 --2- 172.21.15.78:0/2580892582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a1010d100 0x7f5a1010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.595+0000 7f5a168f4640 1 --2- 172.21.15.78:0/2580892582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101045c0 0x7f5a1010cab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.595+0000 7f5a168f4640 1 --2- 172.21.15.78:0/2580892582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a10103ca0 0x7f5a10104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.595+0000 7f5a168f4640 1 -- 172.21.15.78:0/2580892582 >> 172.21.15.78:0/2580892582 conn(0x7f5a100fd700 msgr2=0x7f5a100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.595+0000 7f5a168f4640 1 -- 172.21.15.78:0/2580892582 shutdown_connections 2024-09-11T01:03:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.595+0000 7f5a168f4640 1 -- 172.21.15.78:0/2580892582 wait complete. 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.596+0000 7f5a168f4640 1 Processor -- start 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.596+0000 7f5a168f4640 1 -- start start 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a168f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a10103ca0 0x7f5a101116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a168f4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101045c0 0x7f5a10111c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a168f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1010d100 0x7f5a10112160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a168f4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10102fe0 con 0x7f5a10103ca0 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a168f4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10102e60 con 0x7f5a101045c0 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a168f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10103160 con 0x7f5a1010d100 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a10103ca0 0x7f5a101116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a10103ca0 0x7f5a101116e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55988/0 (socket says 172.21.15.78:55988) 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a0ffff640 1 -- 172.21.15.78:0/2945056309 learned_addr learned my addr 172.21.15.78:0/2945056309 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a14e91640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1010d100 0x7f5a10112160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a0f7fe640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101045c0 0x7f5a10111c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a0ffff640 1 -- 172.21.15.78:0/2945056309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1010d100 msgr2=0x7f5a10112160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a0ffff640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1010d100 0x7f5a10112160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a0ffff640 1 -- 172.21.15.78:0/2945056309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101045c0 msgr2=0x7f5a10111c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a0ffff640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101045c0 0x7f5a10111c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.597+0000 7f5a0ffff640 1 -- 172.21.15.78:0/2945056309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a101abaf0 con 0x7f5a10103ca0 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.598+0000 7f5a14e91640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1010d100 0x7f5a10112160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.598+0000 7f5a0ffff640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a10103ca0 0x7f5a101116e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000ea60 tx=0x7f5a0000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.598+0000 7f5a0d7fa640 1 -- 172.21.15.78:0/2945056309 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000cd20 con 0x7f5a10103ca0 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.598+0000 7f5a0d7fa640 1 -- 172.21.15.78:0/2945056309 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a000035f0 con 0x7f5a10103ca0 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.598+0000 7f5a168f4640 1 -- 172.21.15.78:0/2945056309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a101abde0 con 0x7f5a10103ca0 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.598+0000 7f5a0d7fa640 1 -- 172.21.15.78:0/2945056309 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00010690 con 0x7f5a10103ca0 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.598+0000 7f5a168f4640 1 -- 172.21.15.78:0/2945056309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a101ac2e0 con 0x7f5a10103ca0 2024-09-11T01:03:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a18cc9740 1 Processor -- start 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a0d7fa640 1 -- 172.21.15.78:0/2945056309 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5a00004000 con 0x7f5a10103ca0 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a18cc9740 1 -- start start 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a18cc9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 0x557323a264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a18cc9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a26a10 0x557323a32460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a18cc9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a329a0 0x557323a34d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a18cc9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55732399cea0 con 0x557323a260f0 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a18cc9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55732398c2d0 con 0x557323a26a10 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a18cc9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5573239900e0 con 0x557323a329a0 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a14e91640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 0x557323a264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a26a10 0x557323a32460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a329a0 0x557323a34d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a26a10 0x557323a32460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:56176/0 (socket says 172.21.15.78:56176) 2024-09-11T01:03:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a329a0 0x557323a34d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:34986/0 (socket says 172.21.15.78:34986) 2024-09-11T01:03:31.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a0ffff640 1 -- 172.21.15.78:0/2248540902 learned_addr learned my addr 172.21.15.78:0/2248540902 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:31.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a14e91640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 0x557323a264d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56004/0 (socket says 172.21.15.78:56004) 2024-09-11T01:03:31.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a0f7fe640 1 -- 172.21.15.78:0/2248540902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a26a10 msgr2=0x557323a32460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:31.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a0f7fe640 1 --2- 172.21.15.78:0/2248540902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a26a10 0x557323a32460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a0f7fe640 1 -- 172.21.15.78:0/2248540902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 msgr2=0x557323a264d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:31.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a0f7fe640 1 --2- 172.21.15.78:0/2248540902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 0x557323a264d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.602+0000 7f5a0f7fe640 1 -- 172.21.15.78:0/2248540902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573236ea650 con 0x557323a329a0 2024-09-11T01:03:31.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.603+0000 7f5a0f7fe640 1 --2- 172.21.15.78:0/2248540902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a329a0 0x557323a34d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a04006010 tx=0x7f5a04032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97e5dcfc88897dd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:31.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.603+0000 7f5a0d7fa640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f59f80b54d0 0x7f59f80b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.603+0000 7f5a14e91640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f59f80b54d0 0x7f59f80b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.603+0000 7f59f67fc640 1 -- 172.21.15.78:0/2248540902 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04040070 con 0x557323a329a0 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.603+0000 7f5a0d7fa640 1 -- 172.21.15.78:0/2945056309 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5a000e5f00 con 0x7f5a10103ca0 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.603+0000 7f59f67fc640 1 -- 172.21.15.78:0/2248540902 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a04004030 con 0x557323a329a0 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.603+0000 7f59f67fc640 1 -- 172.21.15.78:0/2248540902 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04035040 con 0x557323a329a0 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.604+0000 7f5a18cc9740 1 -- 172.21.15.78:0/2248540902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a329a0 msgr2=0x557323a34d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.604+0000 7f5a14e91640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f59f80b54d0 0x7f59f80b7990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5a08005fd0 tx=0x7f5a08009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.604+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/2248540902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a329a0 0x557323a34d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a04006010 tx=0x7f5a04032b20 comp rx=0 tx=0).stop 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.604+0000 7f5a18cc9740 1 -- 172.21.15.78:0/2248540902 shutdown_connections 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.604+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/2248540902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a329a0 0x557323a34d90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.604+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/2248540902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a26a10 0x557323a32460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.604+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/2248540902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 0x557323a264d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.604+0000 7f5a18cc9740 1 -- 172.21.15.78:0/2248540902 >> 172.21.15.78:0/2248540902 conn(0x557323a23f20 msgr2=0x557323a250c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.604+0000 7f5a18cc9740 1 -- 172.21.15.78:0/2248540902 shutdown_connections 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.604+0000 7f5a18cc9740 1 -- 172.21.15.78:0/2248540902 wait complete. 2024-09-11T01:03:31.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.604+0000 7f5a18cc9740 1 -- start start 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a18cc9740 1 Processor -- start 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a18cc9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 0x557323a32270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a18cc9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a26a10 0x557323a2f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a18cc9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a329a0 0x557323a2fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a18cc9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5573237ed700 con 0x557323a260f0 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a18cc9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557323927160 con 0x557323a329a0 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a18cc9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557323a355c0 con 0x557323a26a10 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a18cc9740 10 client.0 _ll_register_callbacks cb 0x5573239409e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 0x557323a32270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 0x557323a32270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56020/0 (socket says 172.21.15.78:56020) 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a0ffff640 1 -- 172.21.15.78:0/934549200 learned_addr learned my addr 172.21.15.78:0/934549200 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a14e91640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a329a0 0x557323a2fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a0f7fe640 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a26a10 0x557323a2f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a0ffff640 1 -- 172.21.15.78:0/934549200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a26a10 msgr2=0x557323a2f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a0ffff640 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a26a10 0x557323a2f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a0ffff640 1 -- 172.21.15.78:0/934549200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a329a0 msgr2=0x557323a2fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a0ffff640 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a329a0 0x557323a2fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a0ffff640 1 -- 172.21.15.78:0/934549200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557323954c90 con 0x557323a260f0 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.605+0000 7f5a0f7fe640 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a26a10 0x557323a2f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f5a0ffff640 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 0x557323a32270 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5a00000c00 tx=0x7f5a000098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f048a14a9f9bf13 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f59ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f59ef7fe640 1 -- 172.21.15.78:0/934549200 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a000a8290 con 0x557323a260f0 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f59ef7fe640 1 -- 172.21.15.78:0/934549200 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5a000b54f0 con 0x557323a260f0 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f5a18cc9740 1 -- 172.21.15.78:0/934549200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a04004030 con 0x557323a260f0 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f59ef7fe640 1 -- 172.21.15.78:0/934549200 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5a000abb60 con 0x557323a260f0 2024-09-11T01:03:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f5a18cc9740 10 client.16335 fetch_fsmap learned FSMap version 36 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f5a18cc9740 1 -- 172.21.15.78:0/934549200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557323a40f90 con 0x557323a260f0 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f59ef7fe640 1 -- 172.21.15.78:0/934549200 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a04004030 con 0x557323a260f0 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f59ef7fe640 1 -- 172.21.15.78:0/934549200 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f5a000d31b0 con 0x557323a260f0 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f5a18cc9740 10 client.16335 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f5a18cc9740 10 client.16335 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f5a18cc9740 20 client.16335 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557323a42490 0x557323a44950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f5a18cc9740 4 client.16335 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.606+0000 7f5a18cc9740 1 -- 172.21.15.78:0/934549200 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55732399edf0 con 0x557323a42490 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.607+0000 7f5a0f7fe640 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557323a42490 0x557323a44950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.607+0000 7f59ef7fe640 10 client.16335 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.607+0000 7f5a0f7fe640 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557323a42490 0x557323a44950 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=790912bba13762be server_cookie=4e7e47094edbbc5c in_seq=0 out_seq=0 2024-09-11T01:03:31.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f59ef7fe640 1 -- 172.21.15.78:0/934549200 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55732399edf0 con 0x557323a42490 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f59ef7fe640 10 client.16335 handle_command_reply: tid=0 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f5a18cc9740 1 client.16335 shutdown 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f5a18cc9740 1 -- 172.21.15.78:0/934549200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 msgr2=0x557323a32270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 0x557323a32270 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5a00000c00 tx=0x7f5a000098b0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f5a18cc9740 1 -- 172.21.15.78:0/934549200 shutdown_connections 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557323a42490 0x557323a44950 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f59ef7fe640 0 client.16335 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a329a0 0x557323a2fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a26a10 0x557323a2f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/934549200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a260f0 0x557323a32270 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f5a18cc9740 1 -- 172.21.15.78:0/934549200 >> 172.21.15.78:0/934549200 conn(0x557323a23f20 msgr2=0x557323a29a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f5a18cc9740 1 -- 172.21.15.78:0/934549200 shutdown_connections 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f5a18cc9740 1 -- 172.21.15.78:0/934549200 wait complete. 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.608+0000 7f5a18cc9740 20 client.16335 trim_cache size 0 max 16384 2024-09-11T01:03:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.622+0000 7f5a18cc9740 1 Processor -- start 2024-09-11T01:03:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.622+0000 7f5a18cc9740 1 -- start start 2024-09-11T01:03:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.622+0000 7f5a18cc9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a260f0 0x557323a3e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.622+0000 7f5a18cc9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a26a10 0x557323a2c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.622+0000 7f5a18cc9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a329a0 0x557323a2ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a18cc9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc00e5d0 con 0x557323a329a0 2024-09-11T01:03:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a18cc9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc0101d0 con 0x557323a260f0 2024-09-11T01:03:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a18cc9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc00eca0 con 0x557323a26a10 2024-09-11T01:03:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a26a10 0x557323a2c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a14e91640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a329a0 0x557323a2ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a26a10 0x557323a2c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35010/0 (socket says 172.21.15.78:35010) 2024-09-11T01:03:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a0ffff640 1 --2- 172.21.15.78:0/1694215165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a260f0 0x557323a3e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a0f7fe640 1 -- 172.21.15.78:0/1694215165 learned_addr learned my addr 172.21.15.78:0/1694215165 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a14e91640 1 -- 172.21.15.78:0/1694215165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a26a10 msgr2=0x557323a2c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a14e91640 1 --2- 172.21.15.78:0/1694215165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a26a10 0x557323a2c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a14e91640 1 -- 172.21.15.78:0/1694215165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a260f0 msgr2=0x557323a3e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a14e91640 1 --2- 172.21.15.78:0/1694215165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a260f0 0x557323a3e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a14e91640 1 -- 172.21.15.78:0/1694215165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557323a2f470 con 0x557323a329a0 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a0ffff640 1 --2- 172.21.15.78:0/1694215165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a260f0 0x557323a3e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f5a14e91640 1 --2- 172.21.15.78:0/1694215165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a329a0 0x557323a2ce10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800f790 tx=0x7f5a0801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5854043686a8985 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f59f5ffb640 1 -- 172.21.15.78:0/1694215165 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800fdd0 con 0x557323a329a0 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f59f5ffb640 1 -- 172.21.15.78:0/1694215165 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a08003e20 con 0x557323a329a0 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.623+0000 7f59f5ffb640 1 -- 172.21.15.78:0/1694215165 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08004390 con 0x557323a329a0 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.624+0000 7f5a18cc9740 1 -- 172.21.15.78:0/1694215165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a329a0 msgr2=0x557323a2ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.624+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/1694215165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a329a0 0x557323a2ce10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800f790 tx=0x7f5a0801a000 comp rx=0 tx=0).stop 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.624+0000 7f5a18cc9740 1 -- 172.21.15.78:0/1694215165 shutdown_connections 2024-09-11T01:03:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.624+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/1694215165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a329a0 0x557323a2ce10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.624+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/1694215165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a26a10 0x557323a2c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.624+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/1694215165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a260f0 0x557323a3e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.624+0000 7f5a18cc9740 1 -- 172.21.15.78:0/1694215165 >> 172.21.15.78:0/1694215165 conn(0x557323a23f20 msgr2=0x557323a2df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.624+0000 7f5a18cc9740 1 -- 172.21.15.78:0/1694215165 shutdown_connections 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.624+0000 7f5a18cc9740 1 -- 172.21.15.78:0/1694215165 wait complete. 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.624+0000 7f5a18cc9740 1 -- start start 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a18cc9740 1 Processor -- start 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a18cc9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a260f0 0x557323a3aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a18cc9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a26a10 0x557323a369d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a18cc9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a329a0 0x557323a36f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a18cc9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc00fb00 con 0x557323a26a10 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a18cc9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc00f370 con 0x557323a329a0 2024-09-11T01:03:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a18cc9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc00c5b0 con 0x557323a260f0 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a18cc9740 10 client.0 _ll_register_callbacks cb 0x7f5a0000e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a26a10 0x557323a369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a26a10 0x557323a369d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56038/0 (socket says 172.21.15.78:56038) 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a0f7fe640 1 -- 172.21.15.78:0/3761835422 learned_addr learned my addr 172.21.15.78:0/3761835422 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a0ffff640 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a260f0 0x557323a3aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a14e91640 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a329a0 0x557323a36f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a0f7fe640 1 -- 172.21.15.78:0/3761835422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a260f0 msgr2=0x557323a3aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a0f7fe640 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a260f0 0x557323a3aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a0f7fe640 1 -- 172.21.15.78:0/3761835422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a329a0 msgr2=0x557323a36f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a0f7fe640 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a329a0 0x557323a36f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.625+0000 7f5a0f7fe640 1 -- 172.21.15.78:0/3761835422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557323a37740 con 0x557323a26a10 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f5a0f7fe640 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a26a10 0x557323a369d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5a0402a930 tx=0x7f5a04004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce142279dfbcfa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f59ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:03:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f59ef7fe640 1 -- 172.21.15.78:0/3761835422 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04040040 con 0x557323a26a10 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f59ef7fe640 1 -- 172.21.15.78:0/3761835422 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5a04032e30 con 0x557323a26a10 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f5a18cc9740 1 -- 172.21.15.78:0/3761835422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a08003e20 con 0x557323a26a10 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f59ef7fe640 1 -- 172.21.15.78:0/3761835422 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5a0405d050 con 0x557323a26a10 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f5a18cc9740 10 client.16341 fetch_fsmap learned FSMap version 36 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f5a18cc9740 1 -- 172.21.15.78:0/3761835422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557323ab04a0 con 0x557323a26a10 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f59ef7fe640 1 -- 172.21.15.78:0/3761835422 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a08003e20 con 0x557323a26a10 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f59ef7fe640 1 -- 172.21.15.78:0/3761835422 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f5a04004850 con 0x557323a26a10 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f5a18cc9740 10 client.16341 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f5a18cc9740 10 client.16341 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f5a18cc9740 20 client.16341 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557323ab0940 0x557323ab2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f5a18cc9740 4 client.16341 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.626+0000 7f5a18cc9740 1 -- 172.21.15.78:0/3761835422 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55732399a8a0 con 0x557323ab0940 2024-09-11T01:03:31.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f5a0ffff640 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557323ab0940 0x557323ab2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:31.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f59ef7fe640 10 client.16341 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:03:31.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f5a0ffff640 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557323ab0940 0x557323ab2d30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9b92f1f12a4aef6f server_cookie=f54dca893724b601 in_seq=0 out_seq=0 2024-09-11T01:03:31.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f59ef7fe640 1 -- 172.21.15.78:0/3761835422 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55732399a8a0 con 0x557323ab0940 2024-09-11T01:03:31.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f59ef7fe640 10 client.16341 handle_command_reply: tid=0 2024-09-11T01:03:31.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f5a18cc9740 1 client.16341 shutdown 2024-09-11T01:03:31.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f5a18cc9740 1 -- 172.21.15.78:0/3761835422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a26a10 msgr2=0x557323a369d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:31.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a26a10 0x557323a369d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5a0402a930 tx=0x7f5a04004510 comp rx=0 tx=0).stop 2024-09-11T01:03:31.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f5a18cc9740 1 -- 172.21.15.78:0/3761835422 shutdown_connections 2024-09-11T01:03:31.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557323ab0940 0x557323ab2d30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557323a329a0 0x557323a36f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f59ef7fe640 0 client.16341 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:03:31.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.627+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557323a26a10 0x557323a369d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a18cc9740 1 --2- 172.21.15.78:0/3761835422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557323a260f0 0x557323a3aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a18cc9740 1 -- 172.21.15.78:0/3761835422 >> 172.21.15.78:0/3761835422 conn(0x557323a23f20 msgr2=0x557323a30cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a18cc9740 1 -- 172.21.15.78:0/3761835422 shutdown_connections 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a18cc9740 1 -- 172.21.15.78:0/3761835422 wait complete. 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a18cc9740 20 client.16341 trim_cache size 0 max 16384 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a168f4640 1 -- 172.21.15.78:0/2945056309 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f59f80b54d0 msgr2=0x7f59f80b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a168f4640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f59f80b54d0 0x7f59f80b7990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5a08005fd0 tx=0x7f5a08009500 comp rx=0 tx=0).stop 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a168f4640 1 -- 172.21.15.78:0/2945056309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a10103ca0 msgr2=0x7f5a101116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a168f4640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a10103ca0 0x7f5a101116e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000ea60 tx=0x7f5a0000ee60 comp rx=0 tx=0).stop 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a168f4640 1 -- 172.21.15.78:0/2945056309 shutdown_connections 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a168f4640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f59f80b54d0 0x7f59f80b7990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a168f4640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1010d100 0x7f5a10112160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a168f4640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101045c0 0x7f5a10111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a168f4640 1 --2- 172.21.15.78:0/2945056309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a10103ca0 0x7f5a101116e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:31.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a168f4640 1 -- 172.21.15.78:0/2945056309 >> 172.21.15.78:0/2945056309 conn(0x7f5a100fd700 msgr2=0x7f5a10106b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:31.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.628+0000 7f5a168f4640 1 -- 172.21.15.78:0/2945056309 shutdown_connections 2024-09-11T01:03:31.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:31.629+0000 7f5a168f4640 1 -- 172.21.15.78:0/2945056309 wait complete. 2024-09-11T01:03:32.428 INFO:tasks.workunit.client.0.smithi078.stdout: 1 378891 6.56 MB/sec execute 100 sec latency 16.777 ms 2024-09-11T01:03:32.558 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:03:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:03:33.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:33 smithi078 ceph-mon[31315]: pgmap v463: 97 pgs: 97 active+clean; 214 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2024-09-11T01:03:33.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:33 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:03:33.428 INFO:tasks.workunit.client.0.smithi078.stdout: 1 380297 6.58 MB/sec execute 101 sec latency 23.494 ms 2024-09-11T01:03:33.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.432+0000 7f9171ffb640 1 -- 172.21.15.78:0/2844788063 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91680318a0 con 0x7f91741023b0 2024-09-11T01:03:33.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.433+0000 7f9179889640 1 -- 172.21.15.78:0/2844788063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91741023b0 msgr2=0x7f9174106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:33.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.433+0000 7f9179889640 1 --2- 172.21.15.78:0/2844788063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91741023b0 0x7f9174106880 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9168009900 tx=0x7f916802a410 comp rx=0 tx=0).stop 2024-09-11T01:03:33.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.433+0000 7f9179889640 1 -- 172.21.15.78:0/2844788063 shutdown_connections 2024-09-11T01:03:33.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.433+0000 7f9179889640 1 --2- 172.21.15.78:0/2844788063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174106ed0 0x7f9174107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.433+0000 7f9179889640 1 --2- 172.21.15.78:0/2844788063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91741023b0 0x7f9174106880 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.433+0000 7f9179889640 1 --2- 172.21.15.78:0/2844788063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9174101a90 0x7f9174101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.433+0000 7f9179889640 1 -- 172.21.15.78:0/2844788063 >> 172.21.15.78:0/2844788063 conn(0x7f91740fd700 msgr2=0x7f91740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:33.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.433+0000 7f9179889640 1 -- 172.21.15.78:0/2844788063 shutdown_connections 2024-09-11T01:03:33.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.433+0000 7f9179889640 1 -- 172.21.15.78:0/2844788063 wait complete. 2024-09-11T01:03:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f9179889640 1 Processor -- start 2024-09-11T01:03:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f9179889640 1 -- start start 2024-09-11T01:03:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f9179889640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9174101a90 0x7f917419faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f9179889640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174106ed0 0x7f917419ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f9179889640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91741a5380 0x7f91741a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f9179889640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9174119390 con 0x7f9174101a90 2024-09-11T01:03:33.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f9179889640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9174119210 con 0x7f91741a5380 2024-09-11T01:03:33.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f9179889640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9174119510 con 0x7f9174106ed0 2024-09-11T01:03:33.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f9172ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9174101a90 0x7f917419faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f9172ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9174101a90 0x7f917419faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50398/0 (socket says 172.21.15.78:50398) 2024-09-11T01:03:33.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f9172ffd640 1 -- 172.21.15.78:0/3453241634 learned_addr learned my addr 172.21.15.78:0/3453241634 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:33.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f91727fc640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174106ed0 0x7f917419ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.434+0000 7f91737fe640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91741a5380 0x7f91741a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f91727fc640 1 -- 172.21.15.78:0/3453241634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91741a5380 msgr2=0x7f91741a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:33.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f91727fc640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91741a5380 0x7f91741a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f91727fc640 1 -- 172.21.15.78:0/3453241634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9174101a90 msgr2=0x7f917419faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f91727fc640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9174101a90 0x7f917419faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f91727fc640 1 -- 172.21.15.78:0/3453241634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91741a5f10 con 0x7f9174106ed0 2024-09-11T01:03:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f9172ffd640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9174101a90 0x7f917419faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f91737fe640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91741a5380 0x7f91741a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f91727fc640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174106ed0 0x7f917419ffe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91741027f0 tx=0x7f91680319a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f9178887640 1 -- 172.21.15.78:0/3453241634 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916803f070 con 0x7f9174106ed0 2024-09-11T01:03:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f9178887640 1 -- 172.21.15.78:0/3453241634 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9168003aa0 con 0x7f9174106ed0 2024-09-11T01:03:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f9179889640 1 -- 172.21.15.78:0/3453241634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91741a61a0 con 0x7f9174106ed0 2024-09-11T01:03:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f9178887640 1 -- 172.21.15.78:0/3453241634 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168035070 con 0x7f9174106ed0 2024-09-11T01:03:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.435+0000 7f9179889640 1 -- 172.21.15.78:0/3453241634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91741ac790 con 0x7f9174106ed0 2024-09-11T01:03:33.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:33 smithi110 ceph-mon[34712]: pgmap v463: 97 pgs: 97 active+clean; 214 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2024-09-11T01:03:33.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:33 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:03:33.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.438+0000 7f9178887640 1 -- 172.21.15.78:0/3453241634 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f916804b060 con 0x7f9174106ed0 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.438+0000 7f917bc5e740 1 Processor -- start 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.438+0000 7f917bc5e740 1 -- start start 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.438+0000 7f917bc5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bff4d0 0x5631d3bff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.438+0000 7f917bc5e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 0x5631d3c0b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.438+0000 7f917bc5e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3c0bd80 0x5631d3c0e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.438+0000 7f917bc5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5631d3b75ea0 con 0x5631d3bff4d0 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.438+0000 7f917bc5e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5631d3b652d0 con 0x5631d3c0bd80 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.438+0000 7f917bc5e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5631d3b690e0 con 0x5631d3bffdf0 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.438+0000 7f91737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bff4d0 0x5631d3bff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f9172ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 0x5631d3c0b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f9172ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 0x5631d3c0b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:36550/0 (socket says 172.21.15.78:36550) 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f91727fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3c0bd80 0x5631d3c0e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f9172ffd640 1 -- 172.21.15.78:0/373447216 learned_addr learned my addr 172.21.15.78:0/373447216 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f9172ffd640 1 -- 172.21.15.78:0/373447216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3c0bd80 msgr2=0x5631d3c0e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f9172ffd640 1 --2- 172.21.15.78:0/373447216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3c0bd80 0x5631d3c0e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f9172ffd640 1 -- 172.21.15.78:0/373447216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bff4d0 msgr2=0x5631d3bff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f9172ffd640 1 --2- 172.21.15.78:0/373447216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bff4d0 0x5631d3bff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f9172ffd640 1 -- 172.21.15.78:0/373447216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d38c3650 con 0x5631d3bffdf0 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f91737fe640 1 --2- 172.21.15.78:0/373447216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bff4d0 0x5631d3bff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f9178887640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f915c0b5450 0x7f915c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f91737fe640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f915c0b5450 0x7f915c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f9178887640 1 -- 172.21.15.78:0/3453241634 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f9168105d40 con 0x7f9174106ed0 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f9172ffd640 1 --2- 172.21.15.78:0/373447216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 0x5631d3c0b840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f916400f8d0 tx=0x7f916400fd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=180c53a5c25e7a4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.439+0000 7f9159ffb640 1 -- 172.21.15.78:0/373447216 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164011c20 con 0x5631d3bffdf0 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.440+0000 7f9159ffb640 1 -- 172.21.15.78:0/373447216 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9164011dc0 con 0x5631d3bffdf0 2024-09-11T01:03:33.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.440+0000 7f9159ffb640 1 -- 172.21.15.78:0/373447216 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164018700 con 0x5631d3bffdf0 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.440+0000 7f917bc5e740 1 -- 172.21.15.78:0/373447216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 msgr2=0x5631d3c0b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.440+0000 7f91737fe640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f915c0b5450 0x7f915c0b7910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f916c009fd0 tx=0x7f916c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.440+0000 7f917bc5e740 1 --2- 172.21.15.78:0/373447216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 0x5631d3c0b840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f916400f8d0 tx=0x7f916400fd90 comp rx=0 tx=0).stop 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.440+0000 7f917bc5e740 1 -- 172.21.15.78:0/373447216 shutdown_connections 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.440+0000 7f917bc5e740 1 --2- 172.21.15.78:0/373447216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3c0bd80 0x5631d3c0e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.440+0000 7f917bc5e740 1 --2- 172.21.15.78:0/373447216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 0x5631d3c0b840 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.440+0000 7f917bc5e740 1 --2- 172.21.15.78:0/373447216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bff4d0 0x5631d3bff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.440+0000 7f917bc5e740 1 -- 172.21.15.78:0/373447216 >> 172.21.15.78:0/373447216 conn(0x5631d3bfd300 msgr2=0x5631d3bfe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.440+0000 7f917bc5e740 1 -- 172.21.15.78:0/373447216 shutdown_connections 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.440+0000 7f917bc5e740 1 -- 172.21.15.78:0/373447216 wait complete. 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.441+0000 7f917bc5e740 1 -- start start 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.441+0000 7f917bc5e740 1 Processor -- start 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f917bc5e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bff4d0 0x5631d3c0b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f917bc5e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 0x5631d3c08a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f917bc5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 0x5631d3c09000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f917bc5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5631d39c6700 con 0x5631d3c0bd80 2024-09-11T01:03:33.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f917bc5e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5631d3b00160 con 0x5631d3bff4d0 2024-09-11T01:03:33.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f917bc5e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5631d3c0ea00 con 0x5631d3bffdf0 2024-09-11T01:03:33.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f917bc5e740 10 client.0 _ll_register_callbacks cb 0x5631d3b199e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:33.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f91737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 0x5631d3c09000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f9172ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bff4d0 0x5631d3c0b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f91737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 0x5631d3c09000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50418/0 (socket says 172.21.15.78:50418) 2024-09-11T01:03:33.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f91737fe640 1 -- 172.21.15.78:0/982060430 learned_addr learned my addr 172.21.15.78:0/982060430 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:33.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f9172ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bff4d0 0x5631d3c0b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:47318/0 (socket says 172.21.15.78:47318) 2024-09-11T01:03:33.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f91727fc640 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 0x5631d3c08a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f91737fe640 1 -- 172.21.15.78:0/982060430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 msgr2=0x5631d3c08a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:33.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f91737fe640 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 0x5631d3c08a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f91737fe640 1 -- 172.21.15.78:0/982060430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bff4d0 msgr2=0x5631d3c0b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:33.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f91737fe640 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bff4d0 0x5631d3c0b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f91737fe640 1 -- 172.21.15.78:0/982060430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d3b2dc90 con 0x5631d3c0bd80 2024-09-11T01:03:33.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f9172ffd640 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bff4d0 0x5631d3c0b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:33.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f91727fc640 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 0x5631d3c08a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:33.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f917bc5e740 1 -- 172.21.15.78:0/982060430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9164011dc0 con 0x5631d3c0bd80 2024-09-11T01:03:33.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f91737fe640 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 0x5631d3c09000 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f916c012e50 tx=0x7f916c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f156fcdd46a1344 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:33.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.442+0000 7f9152ffd640 10 client.16353 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:03:33.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.443+0000 7f9152ffd640 1 -- 172.21.15.78:0/982060430 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c004020 con 0x5631d3c0bd80 2024-09-11T01:03:33.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.443+0000 7f9152ffd640 1 -- 172.21.15.78:0/982060430 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f916c0041c0 con 0x5631d3c0bd80 2024-09-11T01:03:33.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.443+0000 7f9152ffd640 1 -- 172.21.15.78:0/982060430 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f916c0043c0 con 0x5631d3c0bd80 2024-09-11T01:03:33.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.443+0000 7f917bc5e740 10 client.16353 fetch_fsmap learned FSMap version 37 2024-09-11T01:03:33.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.443+0000 7f917bc5e740 1 -- 172.21.15.78:0/982060430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5631d3c1a290 con 0x5631d3c0bd80 2024-09-11T01:03:33.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.443+0000 7f9152ffd640 1 -- 172.21.15.78:0/982060430 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f916c027070 con 0x5631d3c0bd80 2024-09-11T01:03:33.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.443+0000 7f917bc5e740 10 client.16353 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T01:03:33.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.443+0000 7f917bc5e740 10 client.16353 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:03:33.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.443+0000 7f917bc5e740 20 client.16353 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:33.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.443+0000 7f917bc5e740 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5631d3c1b6e0 0x5631d3c1db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.443+0000 7f917bc5e740 4 client.16353 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.443+0000 7f917bc5e740 1 -- 172.21.15.78:0/982060430 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631d3b77df0 con 0x5631d3c1b6e0 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.444+0000 7f9172ffd640 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5631d3c1b6e0 0x5631d3c1db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.444+0000 7f9152ffd640 10 client.16353 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.444+0000 7f9172ffd640 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5631d3c1b6e0 0x5631d3c1db40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a081f0b6e3da13e server_cookie=96e54e4a755b275c in_seq=0 out_seq=0 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f9152ffd640 1 -- 172.21.15.78:0/982060430 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5631d3b77df0 con 0x5631d3c1b6e0 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f9152ffd640 10 client.16353 handle_command_reply: tid=0 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f917bc5e740 1 client.16353 shutdown 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f917bc5e740 1 -- 172.21.15.78:0/982060430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 msgr2=0x5631d3c09000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f917bc5e740 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 0x5631d3c09000 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f916c012e50 tx=0x7f916c004730 comp rx=0 tx=0).stop 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f917bc5e740 1 -- 172.21.15.78:0/982060430 shutdown_connections 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f917bc5e740 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 0x5631d3c09000 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f917bc5e740 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bffdf0 0x5631d3c08a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f917bc5e740 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5631d3c1b6e0 0x5631d3c1db40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f917bc5e740 1 --2- 172.21.15.78:0/982060430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bff4d0 0x5631d3c0b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f917bc5e740 1 -- 172.21.15.78:0/982060430 >> 172.21.15.78:0/982060430 conn(0x5631d3bfd300 msgr2=0x5631d3c02e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:33.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f9152ffd640 0 client.16353 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:03:33.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f917bc5e740 1 -- 172.21.15.78:0/982060430 shutdown_connections 2024-09-11T01:03:33.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f917bc5e740 1 -- 172.21.15.78:0/982060430 wait complete. 2024-09-11T01:03:33.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.445+0000 7f917bc5e740 20 client.16353 trim_cache size 0 max 16384 2024-09-11T01:03:33.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f917bc5e740 1 Processor -- start 2024-09-11T01:03:33.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f917bc5e740 1 -- start start 2024-09-11T01:03:33.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f917bc5e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bff4d0 0x5631d3c08e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f917bc5e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bffdf0 0x5631d3c05cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f917bc5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 0x5631d3c061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f917bc5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91741b5100 con 0x5631d3c0bd80 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f917bc5e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f91741b6d60 con 0x5631d3bffdf0 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f917bc5e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91741b57d0 con 0x5631d3bff4d0 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f9172ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bff4d0 0x5631d3c08e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f91737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 0x5631d3c061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f91727fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bffdf0 0x5631d3c05cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f91737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 0x5631d3c061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50434/0 (socket says 172.21.15.78:50434) 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f91737fe640 1 -- 172.21.15.78:0/4154723705 learned_addr learned my addr 172.21.15.78:0/4154723705 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f9172ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bff4d0 0x5631d3c08e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:36568/0 (socket says 172.21.15.78:36568) 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.465+0000 7f91727fc640 1 --2- 172.21.15.78:0/4154723705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bffdf0 0x5631d3c05cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:47320/0 (socket says 172.21.15.78:47320) 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.466+0000 7f91737fe640 1 -- 172.21.15.78:0/4154723705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bff4d0 msgr2=0x5631d3c08e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:33.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.466+0000 7f91737fe640 1 --2- 172.21.15.78:0/4154723705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bff4d0 0x5631d3c08e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.466+0000 7f91737fe640 1 -- 172.21.15.78:0/4154723705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bffdf0 msgr2=0x5631d3c05cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:33.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.466+0000 7f91737fe640 1 --2- 172.21.15.78:0/4154723705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bffdf0 0x5631d3c05cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.466+0000 7f91737fe640 1 -- 172.21.15.78:0/4154723705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d3bfdd80 con 0x5631d3c0bd80 2024-09-11T01:03:33.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.466+0000 7f91737fe640 1 --2- 172.21.15.78:0/4154723705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 0x5631d3c061f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f916c012df0 tx=0x7f916c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a276721792ab322 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:33.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.466+0000 7f91597fa640 1 -- 172.21.15.78:0/4154723705 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c011eb0 con 0x5631d3c0bd80 2024-09-11T01:03:33.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.466+0000 7f91597fa640 1 -- 172.21.15.78:0/4154723705 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9164011dc0 con 0x5631d3c0bd80 2024-09-11T01:03:33.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.466+0000 7f91597fa640 1 -- 172.21.15.78:0/4154723705 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c028650 con 0x5631d3c0bd80 2024-09-11T01:03:33.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.467+0000 7f917bc5e740 1 -- 172.21.15.78:0/4154723705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 msgr2=0x5631d3c061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:33.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.467+0000 7f917bc5e740 1 --2- 172.21.15.78:0/4154723705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 0x5631d3c061f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f916c012df0 tx=0x7f916c023000 comp rx=0 tx=0).stop 2024-09-11T01:03:33.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.467+0000 7f917bc5e740 1 -- 172.21.15.78:0/4154723705 shutdown_connections 2024-09-11T01:03:33.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.467+0000 7f917bc5e740 1 --2- 172.21.15.78:0/4154723705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3c0bd80 0x5631d3c061f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.467+0000 7f917bc5e740 1 --2- 172.21.15.78:0/4154723705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3bffdf0 0x5631d3c05cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.467+0000 7f917bc5e740 1 --2- 172.21.15.78:0/4154723705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bff4d0 0x5631d3c08e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.467+0000 7f917bc5e740 1 -- 172.21.15.78:0/4154723705 >> 172.21.15.78:0/4154723705 conn(0x5631d3bfd300 msgr2=0x5631d3c0a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.467+0000 7f917bc5e740 1 -- 172.21.15.78:0/4154723705 shutdown_connections 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.467+0000 7f917bc5e740 1 -- 172.21.15.78:0/4154723705 wait complete. 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.467+0000 7f917bc5e740 1 -- start start 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.467+0000 7f917bc5e740 1 Processor -- start 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f917bc5e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bff4d0 0x5631d3c13e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f917bc5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bffdf0 0x5631d3c143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f917bc5e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3c0bd80 0x5631d3c10c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f917bc5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91741b6630 con 0x5631d3bffdf0 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f917bc5e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f91741b5ea0 con 0x5631d3c0bd80 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f917bc5e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91741b30e0 con 0x5631d3bff4d0 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f917bc5e740 10 client.0 _ll_register_callbacks cb 0x7f916c009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f91727fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bffdf0 0x5631d3c143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f9172ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bff4d0 0x5631d3c13e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f91737fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3c0bd80 0x5631d3c10c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f91727fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bffdf0 0x5631d3c143b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50450/0 (socket says 172.21.15.78:50450) 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f9172ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bff4d0 0x5631d3c13e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:36578/0 (socket says 172.21.15.78:36578) 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f91727fc640 1 -- 172.21.15.78:0/936750931 learned_addr learned my addr 172.21.15.78:0/936750931 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f91737fe640 1 -- 172.21.15.78:0/936750931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bff4d0 msgr2=0x5631d3c13e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f91737fe640 1 --2- 172.21.15.78:0/936750931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bff4d0 0x5631d3c13e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f91737fe640 1 -- 172.21.15.78:0/936750931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bffdf0 msgr2=0x5631d3c143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f91737fe640 1 --2- 172.21.15.78:0/936750931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bffdf0 0x5631d3c143b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f91737fe640 1 -- 172.21.15.78:0/936750931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d3c11350 con 0x5631d3c0bd80 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.468+0000 7f91727fc640 1 --2- 172.21.15.78:0/936750931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bffdf0 0x5631d3c143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.469+0000 7f91737fe640 1 --2- 172.21.15.78:0/936750931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3c0bd80 0x5631d3c10c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f916c012f60 tx=0x7f916c01f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=160f930e58c145f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.469+0000 7f9152ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.469+0000 7f9152ffd640 1 -- 172.21.15.78:0/936750931 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c041060 con 0x5631d3c0bd80 2024-09-11T01:03:33.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.469+0000 7f917bc5e740 1 -- 172.21.15.78:0/936750931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9164011dc0 con 0x5631d3c0bd80 2024-09-11T01:03:33.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.469+0000 7f9152ffd640 1 -- 172.21.15.78:0/936750931 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f916c005240 con 0x5631d3c0bd80 2024-09-11T01:03:33.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.469+0000 7f9152ffd640 1 -- 172.21.15.78:0/936750931 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f916c011eb0 con 0x5631d3c0bd80 2024-09-11T01:03:33.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.469+0000 7f917bc5e740 10 client.25712 fetch_fsmap learned FSMap version 37 2024-09-11T01:03:33.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.469+0000 7f917bc5e740 1 -- 172.21.15.78:0/936750931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5631d3c89000 con 0x5631d3c0bd80 2024-09-11T01:03:33.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.469+0000 7f9152ffd640 1 -- 172.21.15.78:0/936750931 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9164011dc0 con 0x5631d3c0bd80 2024-09-11T01:03:33.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.469+0000 7f9152ffd640 1 -- 172.21.15.78:0/936750931 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7f916c044020 con 0x5631d3c0bd80 2024-09-11T01:03:33.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.470+0000 7f917bc5e740 10 client.25712 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T01:03:33.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.470+0000 7f917bc5e740 10 client.25712 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:03:33.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.470+0000 7f917bc5e740 20 client.25712 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:33.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.470+0000 7f917bc5e740 1 --2- 172.21.15.78:0/936750931 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5631d3c89660 0x5631d3c8ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.470+0000 7f917bc5e740 4 client.25712 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.470+0000 7f917bc5e740 1 -- 172.21.15.78:0/936750931 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631d3b738a0 con 0x5631d3c89660 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.470+0000 7f9172ffd640 1 --2- 172.21.15.78:0/936750931 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5631d3c89660 0x5631d3c8ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.470+0000 7f9172ffd640 1 --2- 172.21.15.78:0/936750931 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5631d3c89660 0x5631d3c8ba50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef02fffa9fdef536 server_cookie=e4c24a6c0c428401 in_seq=0 out_seq=0 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.470+0000 7f9152ffd640 10 client.25712 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.470+0000 7f9152ffd640 1 -- 172.21.15.78:0/936750931 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631d3b738a0 con 0x5631d3c89660 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.470+0000 7f9152ffd640 10 client.25712 handle_command_reply: tid=0 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.470+0000 7f917bc5e740 1 client.25712 shutdown 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.471+0000 7f917bc5e740 1 -- 172.21.15.78:0/936750931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3c0bd80 msgr2=0x5631d3c10c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.471+0000 7f917bc5e740 1 --2- 172.21.15.78:0/936750931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3c0bd80 0x5631d3c10c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f916c012f60 tx=0x7f916c01f850 comp rx=0 tx=0).stop 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.471+0000 7f917bc5e740 1 -- 172.21.15.78:0/936750931 shutdown_connections 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.471+0000 7f917bc5e740 1 --2- 172.21.15.78:0/936750931 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5631d3c89660 0x5631d3c8ba50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.471+0000 7f917bc5e740 1 --2- 172.21.15.78:0/936750931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631d3c0bd80 0x5631d3c10c60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.471+0000 7f9152ffd640 0 client.25712 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:03:33.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.471+0000 7f917bc5e740 1 --2- 172.21.15.78:0/936750931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631d3bffdf0 0x5631d3c143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.471+0000 7f917bc5e740 1 --2- 172.21.15.78:0/936750931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5631d3bff4d0 0x5631d3c13e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.471+0000 7f917bc5e740 1 -- 172.21.15.78:0/936750931 >> 172.21.15.78:0/936750931 conn(0x5631d3bfd300 msgr2=0x5631d3bfbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:33.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.471+0000 7f917bc5e740 1 -- 172.21.15.78:0/936750931 shutdown_connections 2024-09-11T01:03:33.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.471+0000 7f917bc5e740 1 -- 172.21.15.78:0/936750931 wait complete. 2024-09-11T01:03:33.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.471+0000 7f917bc5e740 20 client.25712 trim_cache size 0 max 16384 2024-09-11T01:03:33.479 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:03:33.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.472+0000 7f9179889640 1 -- 172.21.15.78:0/3453241634 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f915c0b5450 msgr2=0x7f915c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:33.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.472+0000 7f9179889640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f915c0b5450 0x7f915c0b7910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f916c009fd0 tx=0x7f916c005f90 comp rx=0 tx=0).stop 2024-09-11T01:03:33.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.472+0000 7f9179889640 1 -- 172.21.15.78:0/3453241634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174106ed0 msgr2=0x7f917419ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:33.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.472+0000 7f9179889640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174106ed0 0x7f917419ffe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91741027f0 tx=0x7f91680319a0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.472+0000 7f9179889640 1 -- 172.21.15.78:0/3453241634 shutdown_connections 2024-09-11T01:03:33.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.472+0000 7f9179889640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f915c0b5450 0x7f915c0b7910 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.472+0000 7f9179889640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91741a5380 0x7f91741a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.472+0000 7f9179889640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9174106ed0 0x7f917419ffe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.472+0000 7f9179889640 1 --2- 172.21.15.78:0/3453241634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9174101a90 0x7f917419faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:33.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.472+0000 7f9179889640 1 -- 172.21.15.78:0/3453241634 >> 172.21.15.78:0/3453241634 conn(0x7f91740fd700 msgr2=0x7f9174104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:33.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.472+0000 7f9179889640 1 -- 172.21.15.78:0/3453241634 shutdown_connections 2024-09-11T01:03:33.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:33.472+0000 7f9179889640 1 -- 172.21.15.78:0/3453241634 wait complete. 2024-09-11T01:03:33.481 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:33 smithi112 ceph-mon[35384]: pgmap v463: 97 pgs: 97 active+clean; 214 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2024-09-11T01:03:33.482 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:33 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:03:34.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:34 smithi078 ceph-mon[31315]: pgmap v464: 97 pgs: 97 active+clean; 219 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2024-09-11T01:03:34.428 INFO:tasks.workunit.client.0.smithi078.stdout: 1 381255 6.56 MB/sec execute 102 sec latency 549.349 ms 2024-09-11T01:03:34.437 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:03:34.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:34 smithi110 ceph-mon[34712]: pgmap v464: 97 pgs: 97 active+clean; 219 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2024-09-11T01:03:34.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:34 smithi112 ceph-mon[35384]: pgmap v464: 97 pgs: 97 active+clean; 219 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2024-09-11T01:03:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:03:35.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.260+0000 7faada1e4640 1 -- 172.21.15.78:0/482661897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faad41087d0 msgr2=0x7faad4108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:35.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.260+0000 7faada1e4640 1 --2- 172.21.15.78:0/482661897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faad41087d0 0x7faad4108c10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7faac8009900 tx=0x7faac802a410 comp rx=0 tx=0).stop 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.260+0000 7faada1e4640 1 -- 172.21.15.78:0/482661897 shutdown_connections 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.260+0000 7faada1e4640 1 --2- 172.21.15.78:0/482661897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4109150 0x7faad410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.260+0000 7faada1e4640 1 --2- 172.21.15.78:0/482661897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faad41087d0 0x7faad4108c10 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.260+0000 7faada1e4640 1 --2- 172.21.15.78:0/482661897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faad4102dc0 0x7faad41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.260+0000 7faada1e4640 1 -- 172.21.15.78:0/482661897 >> 172.21.15.78:0/482661897 conn(0x7faad40fe830 msgr2=0x7faad4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.260+0000 7faada1e4640 1 -- 172.21.15.78:0/482661897 shutdown_connections 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.260+0000 7faada1e4640 1 -- 172.21.15.78:0/482661897 wait complete. 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faada1e4640 1 Processor -- start 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faada1e4640 1 -- start start 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faada1e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4102dc0 0x7faad4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faada1e4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faad41087d0 0x7faad4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faada1e4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faad4109150 0x7faad419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faada1e4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faad4112e10 con 0x7faad4109150 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faada1e4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faad4112c90 con 0x7faad41087d0 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faada1e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faad4112f90 con 0x7faad4102dc0 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faad3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faad4109150 0x7faad419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faad3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faad4109150 0x7faad419ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50466/0 (socket says 172.21.15.78:50466) 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faad3fff640 1 -- 172.21.15.78:0/1728661601 learned_addr learned my addr 172.21.15.78:0/1728661601 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faad2ffd640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faad41087d0 0x7faad4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faad3fff640 1 -- 172.21.15.78:0/1728661601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4102dc0 msgr2=0x7faad4199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faad37fe640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4102dc0 0x7faad4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faad3fff640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4102dc0 0x7faad4199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faad3fff640 1 -- 172.21.15.78:0/1728661601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faad41087d0 msgr2=0x7faad4199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faad3fff640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faad41087d0 0x7faad4199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faad3fff640 1 -- 172.21.15.78:0/1728661601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad419f3b0 con 0x7faad4109150 2024-09-11T01:03:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.261+0000 7faad37fe640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4102dc0 0x7faad4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.262+0000 7faad3fff640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faad4109150 0x7faad419ebe0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7faacc00eaa0 tx=0x7faacc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.262+0000 7faad0ff9640 1 -- 172.21.15.78:0/1728661601 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc00cdb0 con 0x7faad4109150 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.262+0000 7faad0ff9640 1 -- 172.21.15.78:0/1728661601 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faacc005020 con 0x7faad4109150 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.262+0000 7faad0ff9640 1 -- 172.21.15.78:0/1728661601 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc0116d0 con 0x7faad4109150 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.262+0000 7faada1e4640 1 -- 172.21.15.78:0/1728661601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad419f6a0 con 0x7faad4109150 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.262+0000 7faada1e4640 1 -- 172.21.15.78:0/1728661601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad41a5d60 con 0x7faad4109150 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.264+0000 7faad0ff9640 1 -- 172.21.15.78:0/1728661601 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7faacc00f040 con 0x7faad4109150 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.264+0000 7faadc5b9740 1 Processor -- start 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faadc5b9740 1 -- start start 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faadc5b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x56178426c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faadc5b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178426ca10 0x56178427c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faadc5b9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178427cdb0 0x56178427f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faadc5b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5617841e2ea0 con 0x56178426c0f0 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faadc5b9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5617841d22d0 con 0x56178427cdb0 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faadc5b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5617841d60e0 con 0x56178426ca10 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x56178426c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x56178426c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50482/0 (socket says 172.21.15.78:50482) 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad2ffd640 1 -- 172.21.15.78:0/3547884174 learned_addr learned my addr 172.21.15.78:0/3547884174 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad37fe640 1 --2- 172.21.15.78:0/3547884174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178426ca10 0x56178427c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad2ffd640 1 --2- 172.21.15.78:0/3547884174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178427cdb0 0x56178427f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad2ffd640 1 -- 172.21.15.78:0/3547884174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178426ca10 msgr2=0x56178427c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad2ffd640 1 --2- 172.21.15.78:0/3547884174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178426ca10 0x56178427c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad2ffd640 1 -- 172.21.15.78:0/3547884174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178427cdb0 msgr2=0x56178427f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad2ffd640 1 --2- 172.21.15.78:0/3547884174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178427cdb0 0x56178427f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad2ffd640 1 -- 172.21.15.78:0/3547884174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561783f30650 con 0x56178426c0f0 2024-09-11T01:03:35.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad0ff9640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faabc0b5450 0x7faabc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad2ffd640 1 --2- 172.21.15.78:0/3547884174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x56178426c4d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7faac8033000 tx=0x7faac8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a415e198a0da30c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad37fe640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faabc0b5450 0x7faabc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.265+0000 7faad0ff9640 1 -- 172.21.15.78:0/1728661601 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7faacc0e5d00 con 0x7faad4109150 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faab9ffb640 1 -- 172.21.15.78:0/3547884174 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8035040 con 0x56178426c0f0 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faab9ffb640 1 -- 172.21.15.78:0/3547884174 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faac8045070 con 0x56178426c0f0 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faab9ffb640 1 -- 172.21.15.78:0/3547884174 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8004750 con 0x56178426c0f0 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faadc5b9740 1 -- 172.21.15.78:0/3547884174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 msgr2=0x56178426c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3547884174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x56178426c4d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7faac8033000 tx=0x7faac8005c00 comp rx=0 tx=0).stop 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faadc5b9740 1 -- 172.21.15.78:0/3547884174 shutdown_connections 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3547884174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178427cdb0 0x56178427f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3547884174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178426ca10 0x56178427c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3547884174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x56178426c4d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faadc5b9740 1 -- 172.21.15.78:0/3547884174 >> 172.21.15.78:0/3547884174 conn(0x561784269f20 msgr2=0x56178426b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:35.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faadc5b9740 1 -- 172.21.15.78:0/3547884174 shutdown_connections 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faadc5b9740 1 -- 172.21.15.78:0/3547884174 wait complete. 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.266+0000 7faad37fe640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faabc0b5450 0x7faabc0b7910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faac4005fd0 tx=0x7faac4009350 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.267+0000 7faadc5b9740 1 -- start start 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.267+0000 7faadc5b9740 1 Processor -- start 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.267+0000 7faadc5b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x56178427c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.267+0000 7faadc5b9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178426ca10 0x561784279ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.267+0000 7faadc5b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178427cdb0 0x56178427a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.267+0000 7faadc5b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561784033700 con 0x56178426c0f0 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.267+0000 7faadc5b9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56178416d160 con 0x56178426ca10 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.267+0000 7faadc5b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56178427fa00 con 0x56178427cdb0 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.267+0000 7faadc5b9740 10 client.0 _ll_register_callbacks cb 0x5617841869e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.267+0000 7faad37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x56178427c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.267+0000 7faad37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x56178427c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50486/0 (socket says 172.21.15.78:50486) 2024-09-11T01:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.267+0000 7faad37fe640 1 -- 172.21.15.78:0/2815276405 learned_addr learned my addr 172.21.15.78:0/2815276405 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faad2ffd640 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178426ca10 0x561784279ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faad3fff640 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178427cdb0 0x56178427a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faad37fe640 1 -- 172.21.15.78:0/2815276405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178427cdb0 msgr2=0x56178427a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faad37fe640 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178427cdb0 0x56178427a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faad37fe640 1 -- 172.21.15.78:0/2815276405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178426ca10 msgr2=0x561784279ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faad37fe640 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178426ca10 0x561784279ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faad37fe640 1 -- 172.21.15.78:0/2815276405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56178419ac90 con 0x56178426c0f0 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faad2ffd640 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178426ca10 0x561784279ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faad3fff640 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178427cdb0 0x56178427a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faad37fe640 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x56178427c750 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7faac40126d0 tx=0x7faac4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95535b90d22f3f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faab2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faab2ffd640 1 -- 172.21.15.78:0/2815276405 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4023070 con 0x56178426c0f0 2024-09-11T01:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faab2ffd640 1 -- 172.21.15.78:0/2815276405 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7faac4004220 con 0x56178426c0f0 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faadc5b9740 1 -- 172.21.15.78:0/2815276405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faac8045070 con 0x56178426c0f0 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faab2ffd640 1 -- 172.21.15.78:0/2815276405 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7faac402e050 con 0x56178426c0f0 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faadc5b9740 10 client.16371 fetch_fsmap learned FSMap version 37 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faadc5b9740 1 -- 172.21.15.78:0/2815276405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56178428b470 con 0x56178426c0f0 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faab2ffd640 1 -- 172.21.15.78:0/2815276405 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faac8045070 con 0x56178426c0f0 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.268+0000 7faab2ffd640 1 -- 172.21.15.78:0/2815276405 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7faac400f040 con 0x56178426c0f0 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.269+0000 7faadc5b9740 10 client.16371 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.269+0000 7faadc5b9740 10 client.16371 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.269+0000 7faadc5b9740 20 client.16371 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.269+0000 7faadc5b9740 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56178428c990 0x56178428ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.269+0000 7faadc5b9740 4 client.16371 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.269+0000 7faadc5b9740 1 -- 172.21.15.78:0/2815276405 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617841e4df0 con 0x56178428c990 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.269+0000 7faad2ffd640 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56178428c990 0x56178428ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.269+0000 7faad2ffd640 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56178428c990 0x56178428ee00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50d0badeabf7394f server_cookie=dc11a841a637db4e in_seq=0 out_seq=0 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.269+0000 7faab2ffd640 10 client.16371 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faab2ffd640 1 -- 172.21.15.78:0/2815276405 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617841e4df0 con 0x56178428c990 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faab2ffd640 10 client.16371 handle_command_reply: tid=0 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faadc5b9740 1 client.16371 shutdown 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faadc5b9740 1 -- 172.21.15.78:0/2815276405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 msgr2=0x56178427c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faadc5b9740 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x56178427c750 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7faac40126d0 tx=0x7faac4004730 comp rx=0 tx=0).stop 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faadc5b9740 1 -- 172.21.15.78:0/2815276405 shutdown_connections 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faadc5b9740 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178427cdb0 0x56178427a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faadc5b9740 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178426ca10 0x561784279ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faadc5b9740 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56178428c990 0x56178428ee00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faadc5b9740 1 --2- 172.21.15.78:0/2815276405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x56178427c750 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faab2ffd640 0 client.16371 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:03:35.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faadc5b9740 1 -- 172.21.15.78:0/2815276405 >> 172.21.15.78:0/2815276405 conn(0x561784269f20 msgr2=0x56178426fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faadc5b9740 1 -- 172.21.15.78:0/2815276405 shutdown_connections 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faadc5b9740 1 -- 172.21.15.78:0/2815276405 wait complete. 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.270+0000 7faadc5b9740 20 client.16371 trim_cache size 0 max 16384 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.283+0000 7faadc5b9740 1 Processor -- start 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.283+0000 7faadc5b9740 1 -- start start 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.283+0000 7faadc5b9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178426c0f0 0x561784288c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.283+0000 7faadc5b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426ca10 0x561784269760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.283+0000 7faadc5b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178427cdb0 0x56178427b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.283+0000 7faadc5b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faad41ac370 con 0x56178426ca10 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.283+0000 7faadc5b9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faad41adfd0 con 0x56178426c0f0 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faadc5b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faad41aca40 con 0x56178427cdb0 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faad2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426ca10 0x561784269760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faad2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426ca10 0x561784269760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50502/0 (socket says 172.21.15.78:50502) 2024-09-11T01:03:35.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faad2ffd640 1 -- 172.21.15.78:0/3247095416 learned_addr learned my addr 172.21.15.78:0/3247095416 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faad37fe640 1 --2- 172.21.15.78:0/3247095416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178426c0f0 0x561784288c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faad2ffd640 1 -- 172.21.15.78:0/3247095416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178427cdb0 msgr2=0x56178427b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faad2ffd640 1 --2- 172.21.15.78:0/3247095416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178427cdb0 0x56178427b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faad2ffd640 1 -- 172.21.15.78:0/3247095416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178426c0f0 msgr2=0x561784288c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faad2ffd640 1 --2- 172.21.15.78:0/3247095416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178426c0f0 0x561784288c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faad2ffd640 1 -- 172.21.15.78:0/3247095416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561784272250 con 0x56178426ca10 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faad2ffd640 1 --2- 172.21.15.78:0/3247095416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426ca10 0x561784269760 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7faac8005260 tx=0x7faac8005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632a292cbd435a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faab97fa640 1 -- 172.21.15.78:0/3247095416 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8035040 con 0x56178426ca10 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faab97fa640 1 -- 172.21.15.78:0/3247095416 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faac8045050 con 0x56178426ca10 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.284+0000 7faab97fa640 1 -- 172.21.15.78:0/3247095416 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8004ad0 con 0x56178426ca10 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.285+0000 7faadc5b9740 1 -- 172.21.15.78:0/3247095416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426ca10 msgr2=0x561784269760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.285+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3247095416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426ca10 0x561784269760 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7faac8005260 tx=0x7faac8005290 comp rx=0 tx=0).stop 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.285+0000 7faadc5b9740 1 -- 172.21.15.78:0/3247095416 shutdown_connections 2024-09-11T01:03:35.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.285+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3247095416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178427cdb0 0x56178427b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.285+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3247095416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426ca10 0x561784269760 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.285+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3247095416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178426c0f0 0x561784288c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.285+0000 7faadc5b9740 1 -- 172.21.15.78:0/3247095416 >> 172.21.15.78:0/3247095416 conn(0x561784269f20 msgr2=0x56178427b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.285+0000 7faadc5b9740 1 -- 172.21.15.78:0/3247095416 shutdown_connections 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.285+0000 7faadc5b9740 1 -- 172.21.15.78:0/3247095416 wait complete. 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.285+0000 7faadc5b9740 1 -- start start 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.285+0000 7faadc5b9740 1 Processor -- start 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faadc5b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x561784284cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faadc5b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178426ca10 0x561784285200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faadc5b9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178427cdb0 0x561784281c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faadc5b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faad41ad8a0 con 0x56178426c0f0 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faadc5b9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faad41ad110 con 0x56178427cdb0 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faadc5b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faad41aa370 con 0x56178426ca10 2024-09-11T01:03:35.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faadc5b9740 10 client.0 _ll_register_callbacks cb 0x7faac4016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faad37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x561784284cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faad37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x561784284cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50510/0 (socket says 172.21.15.78:50510) 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faad37fe640 1 -- 172.21.15.78:0/3732067783 learned_addr learned my addr 172.21.15.78:0/3732067783 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faad3fff640 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178427cdb0 0x561784281c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faad37fe640 1 -- 172.21.15.78:0/3732067783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178426ca10 msgr2=0x561784285200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faad2ffd640 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178426ca10 0x561784285200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faad37fe640 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178426ca10 0x561784285200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faad37fe640 1 -- 172.21.15.78:0/3732067783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178427cdb0 msgr2=0x561784281c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faad37fe640 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178427cdb0 0x561784281c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faad37fe640 1 -- 172.21.15.78:0/3732067783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561784285970 con 0x56178426c0f0 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faad37fe640 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x561784284cc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7faac4009320 tx=0x7faac4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e458654d3de65c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faab2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:03:35.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faab2ffd640 1 -- 172.21.15.78:0/3732067783 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4023040 con 0x56178426c0f0 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faab2ffd640 1 -- 172.21.15.78:0/3732067783 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7faac4003cf0 con 0x56178426c0f0 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.286+0000 7faadc5b9740 1 -- 172.21.15.78:0/3732067783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faac8045050 con 0x56178426c0f0 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.287+0000 7faab2ffd640 1 -- 172.21.15.78:0/3732067783 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7faac4043050 con 0x56178426c0f0 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.287+0000 7faadc5b9740 10 client.16377 fetch_fsmap learned FSMap version 37 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.287+0000 7faadc5b9740 1 -- 172.21.15.78:0/3732067783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5617842f9ee0 con 0x56178426c0f0 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.287+0000 7faab2ffd640 1 -- 172.21.15.78:0/3732067783 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faac8045050 con 0x56178426c0f0 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.287+0000 7faab2ffd640 1 -- 172.21.15.78:0/3732067783 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7faac4011b00 con 0x56178426c0f0 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.287+0000 7faadc5b9740 10 client.16377 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.287+0000 7faadc5b9740 10 client.16377 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.287+0000 7faadc5b9740 20 client.16377 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.287+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5617842fa750 0x5617842fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.287+0000 7faadc5b9740 4 client.16377 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.287+0000 7faadc5b9740 1 -- 172.21.15.78:0/3732067783 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617841e08a0 con 0x5617842fa750 2024-09-11T01:03:35.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.287+0000 7faad2ffd640 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5617842fa750 0x5617842fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faab2ffd640 10 client.16377 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faad2ffd640 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5617842fa750 0x5617842fcb70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7def67e6e9610732 server_cookie=b91160cb6390e859 in_seq=0 out_seq=0 2024-09-11T01:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faab2ffd640 1 -- 172.21.15.78:0/3732067783 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617841e08a0 con 0x5617842fa750 2024-09-11T01:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faab2ffd640 10 client.16377 handle_command_reply: tid=0 2024-09-11T01:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faadc5b9740 1 client.16377 shutdown 2024-09-11T01:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faadc5b9740 1 -- 172.21.15.78:0/3732067783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 msgr2=0x561784284cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x561784284cc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7faac4009320 tx=0x7faac4009f90 comp rx=0 tx=0).stop 2024-09-11T01:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faadc5b9740 1 -- 172.21.15.78:0/3732067783 shutdown_connections 2024-09-11T01:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56178427cdb0 0x561784281c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56178426ca10 0x561784285200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5617842fa750 0x5617842fcb70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faadc5b9740 1 --2- 172.21.15.78:0/3732067783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56178426c0f0 0x561784284cc0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faadc5b9740 1 -- 172.21.15.78:0/3732067783 >> 172.21.15.78:0/3732067783 conn(0x561784269f20 msgr2=0x561784271a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faab2ffd640 0 client.16377 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faadc5b9740 1 -- 172.21.15.78:0/3732067783 shutdown_connections 2024-09-11T01:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faadc5b9740 1 -- 172.21.15.78:0/3732067783 wait complete. 2024-09-11T01:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.288+0000 7faadc5b9740 20 client.16377 trim_cache size 0 max 16384 2024-09-11T01:03:35.309 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.289+0000 7faada1e4640 1 -- 172.21.15.78:0/1728661601 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faabc0b5450 msgr2=0x7faabc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.289+0000 7faada1e4640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faabc0b5450 0x7faabc0b7910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faac4005fd0 tx=0x7faac4009350 comp rx=0 tx=0).stop 2024-09-11T01:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.289+0000 7faada1e4640 1 -- 172.21.15.78:0/1728661601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faad4109150 msgr2=0x7faad419ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.289+0000 7faada1e4640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faad4109150 0x7faad419ebe0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7faacc00eaa0 tx=0x7faacc00ef60 comp rx=0 tx=0).stop 2024-09-11T01:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.289+0000 7faada1e4640 1 -- 172.21.15.78:0/1728661601 shutdown_connections 2024-09-11T01:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.289+0000 7faada1e4640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faabc0b5450 0x7faabc0b7910 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.289+0000 7faada1e4640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faad4109150 0x7faad419ebe0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.289+0000 7faada1e4640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faad41087d0 0x7faad4199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.289+0000 7faada1e4640 1 --2- 172.21.15.78:0/1728661601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4102dc0 0x7faad4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.289+0000 7faada1e4640 1 -- 172.21.15.78:0/1728661601 >> 172.21.15.78:0/1728661601 conn(0x7faad40fe830 msgr2=0x7faad4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.289+0000 7faada1e4640 1 -- 172.21.15.78:0/1728661601 shutdown_connections 2024-09-11T01:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:35.289+0000 7faada1e4640 1 -- 172.21.15.78:0/1728661601 wait complete. 2024-09-11T01:03:35.428 INFO:tasks.workunit.client.0.smithi078.stdout: 1 381255 6.50 MB/sec execute 103 sec latency 1549.431 ms 2024-09-11T01:03:36.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:36 smithi078 ceph-mon[31315]: pgmap v465: 97 pgs: 97 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 259 op/s 2024-09-11T01:03:36.411 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:03:36.428 INFO:tasks.workunit.client.0.smithi078.stdout: 1 381255 6.43 MB/sec execute 104 sec latency 2549.465 ms 2024-09-11T01:03:36.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:36 smithi112 ceph-mon[35384]: pgmap v465: 97 pgs: 97 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 259 op/s 2024-09-11T01:03:36.662 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:03:36.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:36 smithi110 ceph-mon[34712]: pgmap v465: 97 pgs: 97 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 259 op/s 2024-09-11T01:03:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.178+0000 7feac9158640 1 -- 172.21.15.78:0/3566649778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feac41045c0 msgr2=0x7feac410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.178+0000 7feac9158640 1 --2- 172.21.15.78:0/3566649778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feac41045c0 0x7feac410cab0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7feab8009900 tx=0x7feab802a410 comp rx=0 tx=0).stop 2024-09-11T01:03:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.178+0000 7feac9158640 1 -- 172.21.15.78:0/3566649778 shutdown_connections 2024-09-11T01:03:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.178+0000 7feac9158640 1 --2- 172.21.15.78:0/3566649778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac410d100 0x7feac410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.178+0000 7feac9158640 1 --2- 172.21.15.78:0/3566649778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feac41045c0 0x7feac410cab0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.178+0000 7feac9158640 1 --2- 172.21.15.78:0/3566649778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac4103ca0 0x7feac4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.178+0000 7feac9158640 1 -- 172.21.15.78:0/3566649778 >> 172.21.15.78:0/3566649778 conn(0x7feac40fd700 msgr2=0x7feac40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.178+0000 7feac9158640 1 -- 172.21.15.78:0/3566649778 shutdown_connections 2024-09-11T01:03:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.178+0000 7feac9158640 1 -- 172.21.15.78:0/3566649778 wait complete. 2024-09-11T01:03:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.178+0000 7feac9158640 1 Processor -- start 2024-09-11T01:03:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac9158640 1 -- start start 2024-09-11T01:03:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac9158640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac4103ca0 0x7feac4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac9158640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feac41045c0 0x7feac4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac9158640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac410d100 0x7feac419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac9158640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feac4112db0 con 0x7feac41045c0 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac9158640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feac4112c30 con 0x7feac410d100 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac9158640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feac4112f30 con 0x7feac4103ca0 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feac41045c0 0x7feac4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feac41045c0 0x7feac4199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50526/0 (socket says 172.21.15.78:50526) 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac259c640 1 -- 172.21.15.78:0/389439798 learned_addr learned my addr 172.21.15.78:0/389439798 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac2d9d640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac4103ca0 0x7feac4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac359e640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac410d100 0x7feac419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac259c640 1 -- 172.21.15.78:0/389439798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac4103ca0 msgr2=0x7feac4199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac259c640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac4103ca0 0x7feac4199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac259c640 1 -- 172.21.15.78:0/389439798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac410d100 msgr2=0x7feac419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac259c640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac410d100 0x7feac419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac259c640 1 -- 172.21.15.78:0/389439798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac419f3b0 con 0x7feac41045c0 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.179+0000 7feac259c640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feac41045c0 0x7feac4199840 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7feab8004270 tx=0x7feab80042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.180+0000 7feaabfff640 1 -- 172.21.15.78:0/389439798 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab803f030 con 0x7feac41045c0 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.180+0000 7feac9158640 1 -- 172.21.15.78:0/389439798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac419f640 con 0x7feac41045c0 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.180+0000 7feaabfff640 1 -- 172.21.15.78:0/389439798 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab80044c0 con 0x7feac41045c0 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.180+0000 7feaabfff640 1 -- 172.21.15.78:0/389439798 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8035070 con 0x7feac41045c0 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.180+0000 7feac9158640 1 -- 172.21.15.78:0/389439798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac41a5d60 con 0x7feac41045c0 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.182+0000 7feacb52d740 1 Processor -- start 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.182+0000 7feacb52d740 1 -- start start 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.182+0000 7feacb52d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.182+0000 7feacb52d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106d1a10 0x562d106e1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.182+0000 7feacb52d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106e1db0 0x562d106e41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feacb52d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562d10647ea0 con 0x562d106d10f0 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feacb52d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562d106372d0 con 0x562d106e1db0 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feacb52d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562d1063b0e0 con 0x562d106d1a10 2024-09-11T01:03:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feaabfff640 1 -- 172.21.15.78:0/389439798 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7feab803bbe0 con 0x7feac41045c0 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feac359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feac359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50536/0 (socket says 172.21.15.78:50536) 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feac359e640 1 -- 172.21.15.78:0/2842958548 learned_addr learned my addr 172.21.15.78:0/2842958548 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feac2d9d640 1 --2- 172.21.15.78:0/2842958548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106d1a10 0x562d106e1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feac359e640 1 -- 172.21.15.78:0/2842958548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106d1a10 msgr2=0x562d106e1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feac359e640 1 --2- 172.21.15.78:0/2842958548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106d1a10 0x562d106e1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feac259c640 1 --2- 172.21.15.78:0/2842958548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106e1db0 0x562d106e41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feac359e640 1 -- 172.21.15.78:0/2842958548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106e1db0 msgr2=0x562d106e41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feac359e640 1 --2- 172.21.15.78:0/2842958548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106e1db0 0x562d106e41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feac359e640 1 -- 172.21.15.78:0/2842958548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d10395650 con 0x562d106d10f0 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feac259c640 1 --2- 172.21.15.78:0/2842958548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106e1db0 0x562d106e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feac359e640 1 --2- 172.21.15.78:0/2842958548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106d14d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7feabc00abb0 tx=0x7feabc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e015b0d3dfc50fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.183+0000 7feaa97fa640 1 -- 172.21.15.78:0/2842958548 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc01d070 con 0x562d106d10f0 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feaa97fa640 1 -- 172.21.15.78:0/2842958548 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feabc018af0 con 0x562d106d10f0 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feaa97fa640 1 -- 172.21.15.78:0/2842958548 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc0109e0 con 0x562d106d10f0 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feaabfff640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feaac0b5450 0x7feaac0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feac2d9d640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feaac0b5450 0x7feaac0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feaabfff640 1 -- 172.21.15.78:0/389439798 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7feab81067c0 con 0x7feac41045c0 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feacb52d740 1 -- 172.21.15.78:0/2842958548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 msgr2=0x562d106d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feac2d9d640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feaac0b5450 0x7feaac0b7910 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x562d106d4270 tx=0x7feab400d040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feacb52d740 1 --2- 172.21.15.78:0/2842958548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106d14d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7feabc00abb0 tx=0x7feabc00f790 comp rx=0 tx=0).stop 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feacb52d740 1 -- 172.21.15.78:0/2842958548 shutdown_connections 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feacb52d740 1 --2- 172.21.15.78:0/2842958548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106e1db0 0x562d106e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feacb52d740 1 --2- 172.21.15.78:0/2842958548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106d1a10 0x562d106e1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feacb52d740 1 --2- 172.21.15.78:0/2842958548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106d14d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feacb52d740 1 -- 172.21.15.78:0/2842958548 >> 172.21.15.78:0/2842958548 conn(0x562d106cef20 msgr2=0x562d106d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:37.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feacb52d740 1 -- 172.21.15.78:0/2842958548 shutdown_connections 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.184+0000 7feacb52d740 1 -- 172.21.15.78:0/2842958548 wait complete. 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feacb52d740 1 -- start start 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feacb52d740 1 Processor -- start 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feacb52d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106e1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feacb52d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106d1a10 0x562d106deb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feacb52d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106e1db0 0x562d106df260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feacb52d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562d10498700 con 0x562d106d10f0 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feacb52d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562d105d2160 con 0x562d106e1db0 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feacb52d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562d106e4a60 con 0x562d106d1a10 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feacb52d740 10 client.0 _ll_register_callbacks cb 0x562d105eb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feac2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106e1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feac2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106e1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50544/0 (socket says 172.21.15.78:50544) 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feac2d9d640 1 -- 172.21.15.78:0/247029136 learned_addr learned my addr 172.21.15.78:0/247029136 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:37.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feac359e640 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106e1db0 0x562d106df260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.185+0000 7feac259c640 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106d1a10 0x562d106deb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feac2d9d640 1 -- 172.21.15.78:0/247029136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106d1a10 msgr2=0x562d106deb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feac2d9d640 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106d1a10 0x562d106deb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feac2d9d640 1 -- 172.21.15.78:0/247029136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106e1db0 msgr2=0x562d106df260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feac2d9d640 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106e1db0 0x562d106df260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feac2d9d640 1 -- 172.21.15.78:0/247029136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d105ffc90 con 0x562d106d10f0 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feac2d9d640 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106e1740 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7feab4009960 tx=0x7feab4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91aaf14294e125af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feaa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feaa6ff5640 1 -- 172.21.15.78:0/247029136 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4028060 con 0x562d106d10f0 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feaa6ff5640 1 -- 172.21.15.78:0/247029136 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7feab4016e00 con 0x562d106d10f0 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feacb52d740 1 -- 172.21.15.78:0/247029136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feabc018af0 con 0x562d106d10f0 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feaa6ff5640 1 -- 172.21.15.78:0/247029136 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7feab402d440 con 0x562d106d10f0 2024-09-11T01:03:37.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feacb52d740 10 client.16389 fetch_fsmap learned FSMap version 37 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.186+0000 7feacb52d740 1 -- 172.21.15.78:0/247029136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562d106f0430 con 0x562d106d10f0 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.187+0000 7feaa6ff5640 1 -- 172.21.15.78:0/247029136 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feabc018af0 con 0x562d106d10f0 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.187+0000 7feaa6ff5640 1 -- 172.21.15.78:0/247029136 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7feab402b020 con 0x562d106d10f0 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.187+0000 7feacb52d740 10 client.16389 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.187+0000 7feacb52d740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.187+0000 7feacb52d740 20 client.16389 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.187+0000 7feacb52d740 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562d106f1990 0x562d106f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.187+0000 7feacb52d740 4 client.16389 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.187+0000 7feacb52d740 1 -- 172.21.15.78:0/247029136 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d10649df0 con 0x562d106f1990 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.187+0000 7feac259c640 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562d106f1990 0x562d106f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.188+0000 7feac259c640 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562d106f1990 0x562d106f3dd0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56ae06949bad43ef server_cookie=b689c4e343961cb1 in_seq=0 out_seq=0 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.188+0000 7feaa6ff5640 10 client.16389 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feaa6ff5640 1 -- 172.21.15.78:0/247029136 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d10649df0 con 0x562d106f1990 2024-09-11T01:03:37.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feaa6ff5640 10 client.16389 handle_command_reply: tid=0 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feacb52d740 1 client.16389 shutdown 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feacb52d740 1 -- 172.21.15.78:0/247029136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 msgr2=0x562d106e1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feacb52d740 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106e1740 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7feab4009960 tx=0x7feab4004a40 comp rx=0 tx=0).stop 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feacb52d740 1 -- 172.21.15.78:0/247029136 shutdown_connections 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feacb52d740 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106e1db0 0x562d106df260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feacb52d740 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106d1a10 0x562d106deb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feacb52d740 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562d106f1990 0x562d106f3dd0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feacb52d740 1 --2- 172.21.15.78:0/247029136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106e1740 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feaa6ff5640 0 client.16389 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feacb52d740 1 -- 172.21.15.78:0/247029136 >> 172.21.15.78:0/247029136 conn(0x562d106cef20 msgr2=0x562d106d4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feacb52d740 1 -- 172.21.15.78:0/247029136 shutdown_connections 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feacb52d740 1 -- 172.21.15.78:0/247029136 wait complete. 2024-09-11T01:03:37.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.189+0000 7feacb52d740 20 client.16389 trim_cache size 0 max 16384 2024-09-11T01:03:37.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.203+0000 7feacb52d740 1 Processor -- start 2024-09-11T01:03:37.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.203+0000 7feacb52d740 1 -- start start 2024-09-11T01:03:37.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.203+0000 7feacb52d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.203+0000 7feacb52d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106d1a10 0x562d106e08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.203+0000 7feacb52d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106e1db0 0x562d106f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.203+0000 7feacb52d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feac41ac440 con 0x562d106d10f0 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.203+0000 7feacb52d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feac41ae0a0 con 0x562d106d1a10 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.203+0000 7feacb52d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feac41acb10 con 0x562d106e1db0 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feac2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feac2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106edcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50556/0 (socket says 172.21.15.78:50556) 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feac2d9d640 1 -- 172.21.15.78:0/259446243 learned_addr learned my addr 172.21.15.78:0/259446243 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feac259c640 1 --2- 172.21.15.78:0/259446243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106d1a10 0x562d106e08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feac359e640 1 --2- 172.21.15.78:0/259446243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106e1db0 0x562d106f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feac2d9d640 1 -- 172.21.15.78:0/259446243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106e1db0 msgr2=0x562d106f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feac2d9d640 1 --2- 172.21.15.78:0/259446243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106e1db0 0x562d106f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feac2d9d640 1 -- 172.21.15.78:0/259446243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106d1a10 msgr2=0x562d106e08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feac2d9d640 1 --2- 172.21.15.78:0/259446243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106d1a10 0x562d106e08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feac2d9d640 1 -- 172.21.15.78:0/259446243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d106de940 con 0x562d106d10f0 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feac2d9d640 1 --2- 172.21.15.78:0/259446243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106edcc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7feab4009500 tx=0x7feab4009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84985abe4a7bc879 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feaa8ff9640 1 -- 172.21.15.78:0/259446243 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4018420 con 0x562d106d10f0 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feaa8ff9640 1 -- 172.21.15.78:0/259446243 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab402b9d0 con 0x562d106d10f0 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.204+0000 7feaa8ff9640 1 -- 172.21.15.78:0/259446243 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab402dcf0 con 0x562d106d10f0 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.205+0000 7feacb52d740 1 -- 172.21.15.78:0/259446243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 msgr2=0x562d106edcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.205+0000 7feacb52d740 1 --2- 172.21.15.78:0/259446243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106edcc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7feab4009500 tx=0x7feab4009470 comp rx=0 tx=0).stop 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.205+0000 7feacb52d740 1 -- 172.21.15.78:0/259446243 shutdown_connections 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.205+0000 7feacb52d740 1 --2- 172.21.15.78:0/259446243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106e1db0 0x562d106f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.205+0000 7feacb52d740 1 --2- 172.21.15.78:0/259446243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106d1a10 0x562d106e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.205+0000 7feacb52d740 1 --2- 172.21.15.78:0/259446243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d10f0 0x562d106edcc0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.205+0000 7feacb52d740 1 -- 172.21.15.78:0/259446243 >> 172.21.15.78:0/259446243 conn(0x562d106cef20 msgr2=0x562d106e01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.205+0000 7feacb52d740 1 -- 172.21.15.78:0/259446243 shutdown_connections 2024-09-11T01:03:37.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.205+0000 7feacb52d740 1 -- 172.21.15.78:0/259446243 wait complete. 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feacb52d740 1 -- start start 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feacb52d740 1 Processor -- start 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feacb52d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106d10f0 0x562d106e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feacb52d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d1a10 0x562d106ea530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feacb52d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106e1db0 0x562d106e6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feacb52d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feac41ad970 con 0x562d106d1a10 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feacb52d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feac41ad1e0 con 0x562d106d10f0 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feacb52d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feac41aa420 con 0x562d106e1db0 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feacb52d740 10 client.0 _ll_register_callbacks cb 0x7feab40099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feac259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d1a10 0x562d106ea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feac2d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106d10f0 0x562d106e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feac259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d1a10 0x562d106ea530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50572/0 (socket says 172.21.15.78:50572) 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feac259c640 1 -- 172.21.15.78:0/1123338476 learned_addr learned my addr 172.21.15.78:0/1123338476 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:37.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feac359e640 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106e1db0 0x562d106e6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feac259c640 1 -- 172.21.15.78:0/1123338476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106e1db0 msgr2=0x562d106e6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feac259c640 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106e1db0 0x562d106e6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feac259c640 1 -- 172.21.15.78:0/1123338476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106d10f0 msgr2=0x562d106e9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feac259c640 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106d10f0 0x562d106e9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feac259c640 1 -- 172.21.15.78:0/1123338476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d106e7540 con 0x562d106d1a10 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.206+0000 7feac2d9d640 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106d10f0 0x562d106e9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.207+0000 7feac259c640 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d1a10 0x562d106ea530 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7feab8006010 tx=0x7feab802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea312048d5e11f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.207+0000 7feaa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.207+0000 7feaa6ff5640 1 -- 172.21.15.78:0/1123338476 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8049040 con 0x562d106d1a10 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.207+0000 7feaa6ff5640 1 -- 172.21.15.78:0/1123338476 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7feab80c8f30 con 0x562d106d1a10 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.207+0000 7feacb52d740 1 -- 172.21.15.78:0/1123338476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feab402b9d0 con 0x562d106d1a10 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.207+0000 7feaa6ff5640 1 -- 172.21.15.78:0/1123338476 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7feab80cc7d0 con 0x562d106d1a10 2024-09-11T01:03:37.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.207+0000 7feacb52d740 10 client.16395 fetch_fsmap learned FSMap version 37 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.207+0000 7feacb52d740 1 -- 172.21.15.78:0/1123338476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562d1075f450 con 0x562d106d1a10 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.207+0000 7feaa6ff5640 1 -- 172.21.15.78:0/1123338476 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab402b9d0 con 0x562d106d1a10 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.208+0000 7feaa6ff5640 1 -- 172.21.15.78:0/1123338476 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8386+0+0 (secure 0 0 0) 0x7feab80ccf20 con 0x562d106d1a10 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.208+0000 7feacb52d740 10 client.16395 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.208+0000 7feacb52d740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.208+0000 7feacb52d740 20 client.16395 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.208+0000 7feacb52d740 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562d1075fb90 0x562d10761fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.208+0000 7feacb52d740 4 client.16395 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.208+0000 7feacb52d740 1 -- 172.21.15.78:0/1123338476 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d106458a0 con 0x562d1075fb90 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.208+0000 7feac2d9d640 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562d1075fb90 0x562d10761fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.208+0000 7feac2d9d640 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562d1075fb90 0x562d10761fb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3be11bcc004c2db0 server_cookie=bd471114944b7e6a in_seq=0 out_seq=0 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.208+0000 7feaa6ff5640 10 client.16395 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.209+0000 7feaa6ff5640 1 -- 172.21.15.78:0/1123338476 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d106458a0 con 0x562d1075fb90 2024-09-11T01:03:37.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.209+0000 7feaa6ff5640 10 client.16395 handle_command_reply: tid=0 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.209+0000 7feacb52d740 1 client.16395 shutdown 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feacb52d740 1 -- 172.21.15.78:0/1123338476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d1a10 msgr2=0x562d106ea530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feacb52d740 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d1a10 0x562d106ea530 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7feab8006010 tx=0x7feab802a410 comp rx=0 tx=0).stop 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feacb52d740 1 -- 172.21.15.78:0/1123338476 shutdown_connections 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feacb52d740 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d106e1db0 0x562d106e6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feacb52d740 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562d1075fb90 0x562d10761fb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feacb52d740 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d106d1a10 0x562d106ea530 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feacb52d740 1 --2- 172.21.15.78:0/1123338476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d106d10f0 0x562d106e9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feacb52d740 1 -- 172.21.15.78:0/1123338476 >> 172.21.15.78:0/1123338476 conn(0x562d106cef20 msgr2=0x562d106e09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feaa6ff5640 0 client.16395 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feacb52d740 1 -- 172.21.15.78:0/1123338476 shutdown_connections 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feacb52d740 1 -- 172.21.15.78:0/1123338476 wait complete. 2024-09-11T01:03:37.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feacb52d740 20 client.16395 trim_cache size 0 max 16384 2024-09-11T01:03:37.222 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:03:37.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feac9158640 1 -- 172.21.15.78:0/389439798 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feaac0b5450 msgr2=0x7feaac0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:37.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feac9158640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feaac0b5450 0x7feaac0b7910 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x562d106d4270 tx=0x7feab400d040 comp rx=0 tx=0).stop 2024-09-11T01:03:37.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feac9158640 1 -- 172.21.15.78:0/389439798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feac41045c0 msgr2=0x7feac4199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:37.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.210+0000 7feac9158640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feac41045c0 0x7feac4199840 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7feab8004270 tx=0x7feab80042a0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.211+0000 7feac9158640 1 -- 172.21.15.78:0/389439798 shutdown_connections 2024-09-11T01:03:37.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.211+0000 7feac9158640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feaac0b5450 0x7feaac0b7910 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.211+0000 7feac9158640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac410d100 0x7feac419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.211+0000 7feac9158640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feac41045c0 0x7feac4199840 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.211+0000 7feac9158640 1 --2- 172.21.15.78:0/389439798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac4103ca0 0x7feac4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:37.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.211+0000 7feac9158640 1 -- 172.21.15.78:0/389439798 >> 172.21.15.78:0/389439798 conn(0x7feac40fd700 msgr2=0x7feac40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:37.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.211+0000 7feac9158640 1 -- 172.21.15.78:0/389439798 shutdown_connections 2024-09-11T01:03:37.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:37.211+0000 7feac9158640 1 -- 172.21.15.78:0/389439798 wait complete. 2024-09-11T01:03:37.428 INFO:tasks.workunit.client.0.smithi078.stdout: 1 381971 6.39 MB/sec execute 105 sec latency 3311.942 ms 2024-09-11T01:03:38.428 INFO:tasks.workunit.client.0.smithi078.stdout: 1 383697 6.44 MB/sec execute 106 sec latency 11.474 ms 2024-09-11T01:03:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:38 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:03:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:38 smithi078 ceph-mon[31315]: pgmap v466: 97 pgs: 97 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2024-09-11T01:03:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:38 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:03:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:38 smithi110 ceph-mon[34712]: pgmap v466: 97 pgs: 97 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2024-09-11T01:03:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:38 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:03:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:38 smithi112 ceph-mon[35384]: pgmap v466: 97 pgs: 97 active+clean; 221 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2024-09-11T01:03:38.995 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:03:38.996 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:03:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:03:39.428 INFO:tasks.workunit.client.0.smithi078.stdout: 1 384796 6.43 MB/sec execute 107 sec latency 269.213 ms 2024-09-11T01:03:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.759+0000 7f77399d2640 1 -- 172.21.15.78:0/2242321900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734108020 msgr2=0x7f773410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.759+0000 7f77399d2640 1 --2- 172.21.15.78:0/2242321900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734108020 0x7f773410e820 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f772c009ca0 tx=0x7f772c02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 -- 172.21.15.78:0/2242321900 shutdown_connections 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 --2- 172.21.15.78:0/2242321900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734108020 0x7f773410e820 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 --2- 172.21.15.78:0/2242321900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77341076a0 0x7f7734107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 --2- 172.21.15.78:0/2242321900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7734101c90 0x7f7734102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 -- 172.21.15.78:0/2242321900 >> 172.21.15.78:0/2242321900 conn(0x7f77340fd700 msgr2=0x7f77340ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 -- 172.21.15.78:0/2242321900 shutdown_connections 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 -- 172.21.15.78:0/2242321900 wait complete. 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 Processor -- start 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 -- start start 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7734101c90 0x7f7734110650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77341076a0 0x7f7734110b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7734108020 0x7f77341110d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7734112df0 con 0x7f77341076a0 2024-09-11T01:03:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7734112c70 con 0x7f7734108020 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.760+0000 7f77399d2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7734112f70 con 0x7f7734101c90 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f772bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77341076a0 0x7f7734110b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f772bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77341076a0 0x7f7734110b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50594/0 (socket says 172.21.15.78:50594) 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f772bfff640 1 -- 172.21.15.78:0/388366292 learned_addr learned my addr 172.21.15.78:0/388366292 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f77337fe640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7734108020 0x7f77341110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f7732ffd640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7734101c90 0x7f7734110650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f772bfff640 1 -- 172.21.15.78:0/388366292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7734101c90 msgr2=0x7f7734110650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f772bfff640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7734101c90 0x7f7734110650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f772bfff640 1 -- 172.21.15.78:0/388366292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7734108020 msgr2=0x7f77341110d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f772bfff640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7734108020 0x7f77341110d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f772bfff640 1 -- 172.21.15.78:0/388366292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77341a7640 con 0x7f77341076a0 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f772bfff640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77341076a0 0x7f7734110b90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f771c00e9f0 tx=0x7f771c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f77389d0640 1 -- 172.21.15.78:0/388366292 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c00cd80 con 0x7f77341076a0 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f77389d0640 1 -- 172.21.15.78:0/388366292 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f771c0108e0 con 0x7f77341076a0 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f77389d0640 1 -- 172.21.15.78:0/388366292 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c018980 con 0x7f77341076a0 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f77399d2640 1 -- 172.21.15.78:0/388366292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77341a7930 con 0x7f77341076a0 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.761+0000 7f77399d2640 1 -- 172.21.15.78:0/388366292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77341a7e20 con 0x7f77341076a0 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.764+0000 7f77389d0640 1 -- 172.21.15.78:0/388366292 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f771c010430 con 0x7f77341076a0 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.764+0000 7f773bda7740 1 Processor -- start 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.764+0000 7f773bda7740 1 -- start start 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.764+0000 7f773bda7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.764+0000 7f773bda7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b50250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.764+0000 7f773bda7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 0x55c5f7b52b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.764+0000 7f773bda7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c5f7abae60 con 0x55c5f7b43ee0 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.764+0000 7f773bda7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c5f7aaa290 con 0x55c5f7b50790 2024-09-11T01:03:39.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.764+0000 7f773bda7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c5f7aae0a0 con 0x55c5f7b44800 2024-09-11T01:03:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.764+0000 7f77337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f77337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b442c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50596/0 (socket says 172.21.15.78:50596) 2024-09-11T01:03:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f77337fe640 1 -- 172.21.15.78:0/1238217432 learned_addr learned my addr 172.21.15.78:0/1238217432 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f7732ffd640 1 --2- 172.21.15.78:0/1238217432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b50250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f772bfff640 1 --2- 172.21.15.78:0/1238217432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 0x55c5f7b52b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f77337fe640 1 -- 172.21.15.78:0/1238217432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 msgr2=0x55c5f7b50250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f77337fe640 1 --2- 172.21.15.78:0/1238217432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b50250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f77337fe640 1 -- 172.21.15.78:0/1238217432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 msgr2=0x55c5f7b52b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f77337fe640 1 --2- 172.21.15.78:0/1238217432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 0x55c5f7b52b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f77389d0640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f77200b5450 0x7f77200b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f77337fe640 1 -- 172.21.15.78:0/1238217432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5f7808930 con 0x55c5f7b43ee0 2024-09-11T01:03:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f7732ffd640 1 --2- 172.21.15.78:0/1238217432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b50250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f7732ffd640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f77200b5450 0x7f77200b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f77337fe640 1 --2- 172.21.15.78:0/1238217432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b442c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f772c009430 tx=0x7f772c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49c4acb2f7687c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f77389d0640 1 -- 172.21.15.78:0/388366292 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f771c0e62c0 con 0x7f77341076a0 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.765+0000 7f7729ffb640 1 -- 172.21.15.78:0/1238217432 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c031cf0 con 0x55c5f7b43ee0 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.766+0000 7f7729ffb640 1 -- 172.21.15.78:0/1238217432 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f772c036070 con 0x55c5f7b43ee0 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.766+0000 7f7729ffb640 1 -- 172.21.15.78:0/1238217432 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c0042a0 con 0x55c5f7b43ee0 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.766+0000 7f7732ffd640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f77200b5450 0x7f77200b7910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55c5f7b47060 tx=0x7f7724005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.766+0000 7f773bda7740 1 -- 172.21.15.78:0/1238217432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 msgr2=0x55c5f7b442c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.766+0000 7f773bda7740 1 --2- 172.21.15.78:0/1238217432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b442c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f772c009430 tx=0x7f772c005ef0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.767+0000 7f773bda7740 1 -- 172.21.15.78:0/1238217432 shutdown_connections 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.767+0000 7f773bda7740 1 --2- 172.21.15.78:0/1238217432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 0x55c5f7b52b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.767+0000 7f773bda7740 1 --2- 172.21.15.78:0/1238217432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b50250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.767+0000 7f773bda7740 1 --2- 172.21.15.78:0/1238217432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b442c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.767+0000 7f773bda7740 1 -- 172.21.15.78:0/1238217432 >> 172.21.15.78:0/1238217432 conn(0x55c5f7b41d10 msgr2=0x55c5f7b42eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.767+0000 7f773bda7740 1 -- 172.21.15.78:0/1238217432 shutdown_connections 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.767+0000 7f773bda7740 1 -- 172.21.15.78:0/1238217432 wait complete. 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.767+0000 7f773bda7740 1 -- start start 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.767+0000 7f773bda7740 1 Processor -- start 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f773bda7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b4d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f773bda7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b4dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f773bda7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 0x55c5f7b4e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f773bda7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c5f790b890 con 0x55c5f7b43ee0 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f773bda7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c5f7a44d40 con 0x55c5f7b50790 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f773bda7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c5f7b53260 con 0x55c5f7b44800 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f773bda7740 10 client.0 _ll_register_callbacks cb 0x55c5f7aa8c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f7732ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b4d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f7732ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b4d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50610/0 (socket says 172.21.15.78:50610) 2024-09-11T01:03:39.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f7732ffd640 1 -- 172.21.15.78:0/1647818680 learned_addr learned my addr 172.21.15.78:0/1647818680 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:39.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f77337fe640 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 0x55c5f7b4e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f772bfff640 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b4dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f7732ffd640 1 -- 172.21.15.78:0/1647818680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 msgr2=0x55c5f7b4dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:39.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f7732ffd640 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b4dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f7732ffd640 1 -- 172.21.15.78:0/1647818680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 msgr2=0x55c5f7b4e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f7732ffd640 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 0x55c5f7b4e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f7732ffd640 1 -- 172.21.15.78:0/1647818680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5f7b5c710 con 0x55c5f7b43ee0 2024-09-11T01:03:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f7732ffd640 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b4d820 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f772401e980 tx=0x7f7724004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647738990c6e5686 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f7712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:03:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f7712ffd640 1 -- 172.21.15.78:0/1647818680 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724009be0 con 0x55c5f7b43ee0 2024-09-11T01:03:39.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f7712ffd640 1 -- 172.21.15.78:0/1647818680 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7724018030 con 0x55c5f7b43ee0 2024-09-11T01:03:39.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.768+0000 7f773bda7740 1 -- 172.21.15.78:0/1647818680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f772c036070 con 0x55c5f7b43ee0 2024-09-11T01:03:39.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.769+0000 7f7712ffd640 1 -- 172.21.15.78:0/1647818680 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7724005020 con 0x55c5f7b43ee0 2024-09-11T01:03:39.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.769+0000 7f773bda7740 10 client.16407 fetch_fsmap learned FSMap version 38 2024-09-11T01:03:39.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.769+0000 7f773bda7740 1 -- 172.21.15.78:0/1647818680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c5f7b5f120 con 0x55c5f7b43ee0 2024-09-11T01:03:39.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.769+0000 7f7712ffd640 1 -- 172.21.15.78:0/1647818680 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f772c036070 con 0x55c5f7b43ee0 2024-09-11T01:03:39.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.769+0000 7f7712ffd640 1 -- 172.21.15.78:0/1647818680 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f7724031030 con 0x55c5f7b43ee0 2024-09-11T01:03:39.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.769+0000 7f773bda7740 10 client.16407 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:03:39.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.769+0000 7f773bda7740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:03:39.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.769+0000 7f773bda7740 20 client.16407 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:39.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.769+0000 7f773bda7740 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c5f7b606d0 0x55c5f7b62b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.769+0000 7f773bda7740 4 client.16407 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:03:39.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.769+0000 7f773bda7740 1 -- 172.21.15.78:0/1647818680 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5f7abcdb0 con 0x55c5f7b606d0 2024-09-11T01:03:39.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.769+0000 7f772bfff640 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c5f7b606d0 0x55c5f7b62b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.770+0000 7f7712ffd640 10 client.16407 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:39.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.770+0000 7f772bfff640 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c5f7b606d0 0x55c5f7b62b10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f370364aca3bde86 server_cookie=d3d6d7dfac952ee3 in_seq=0 out_seq=0 2024-09-11T01:03:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.770+0000 7f7712ffd640 1 -- 172.21.15.78:0/1647818680 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c5f7abcdb0 con 0x55c5f7b606d0 2024-09-11T01:03:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.770+0000 7f7712ffd640 10 client.16407 handle_command_reply: tid=0 2024-09-11T01:03:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.770+0000 7f773bda7740 1 client.16407 shutdown 2024-09-11T01:03:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.770+0000 7f773bda7740 1 -- 172.21.15.78:0/1647818680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 msgr2=0x55c5f7b4d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.770+0000 7f773bda7740 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b4d820 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f772401e980 tx=0x7f7724004730 comp rx=0 tx=0).stop 2024-09-11T01:03:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.771+0000 7f773bda7740 1 -- 172.21.15.78:0/1647818680 shutdown_connections 2024-09-11T01:03:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.771+0000 7f773bda7740 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 0x55c5f7b4e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.771+0000 7f773bda7740 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b4dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.771+0000 7f773bda7740 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c5f7b606d0 0x55c5f7b62b10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.771+0000 7f773bda7740 1 --2- 172.21.15.78:0/1647818680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b4d820 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.771+0000 7f7712ffd640 0 client.16407 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.771+0000 7f773bda7740 1 -- 172.21.15.78:0/1647818680 >> 172.21.15.78:0/1647818680 conn(0x55c5f7b41d10 msgr2=0x55c5f7b40f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.771+0000 7f773bda7740 1 -- 172.21.15.78:0/1647818680 shutdown_connections 2024-09-11T01:03:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.771+0000 7f773bda7740 1 -- 172.21.15.78:0/1647818680 wait complete. 2024-09-11T01:03:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.771+0000 7f773bda7740 20 client.16407 trim_cache size 0 max 16384 2024-09-11T01:03:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f773bda7740 1 Processor -- start 2024-09-11T01:03:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f773bda7740 1 -- start start 2024-09-11T01:03:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f773bda7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b4aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f773bda7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b4af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f773bda7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 0x55c5f7b683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f773bda7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77341b06a0 con 0x55c5f7b43ee0 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f773bda7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77341b2360 con 0x55c5f7b50790 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f773bda7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77341b0d70 con 0x55c5f7b44800 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f7732ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b4aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f7732ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b4aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50620/0 (socket says 172.21.15.78:50620) 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f7732ffd640 1 -- 172.21.15.78:0/614855764 learned_addr learned my addr 172.21.15.78:0/614855764 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f77337fe640 1 --2- 172.21.15.78:0/614855764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 0x55c5f7b683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f772bfff640 1 --2- 172.21.15.78:0/614855764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b4af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f7732ffd640 1 -- 172.21.15.78:0/614855764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 msgr2=0x55c5f7b4af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f7732ffd640 1 --2- 172.21.15.78:0/614855764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b4af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f7732ffd640 1 -- 172.21.15.78:0/614855764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 msgr2=0x55c5f7b683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f7732ffd640 1 --2- 172.21.15.78:0/614855764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 0x55c5f7b683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f7732ffd640 1 -- 172.21.15.78:0/614855764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5f7b41b40 con 0x55c5f7b43ee0 2024-09-11T01:03:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f772bfff640 1 --2- 172.21.15.78:0/614855764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b4af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:03:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.789+0000 7f7732ffd640 1 --2- 172.21.15.78:0/614855764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b4aa50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7724019040 tx=0x7f77240092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efd1a29998e8696 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.790+0000 7f77297fa640 1 -- 172.21.15.78:0/614855764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724029610 con 0x55c5f7b43ee0 2024-09-11T01:03:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.790+0000 7f77297fa640 1 -- 172.21.15.78:0/614855764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77240297b0 con 0x55c5f7b43ee0 2024-09-11T01:03:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.790+0000 7f77297fa640 1 -- 172.21.15.78:0/614855764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77240049e0 con 0x55c5f7b43ee0 2024-09-11T01:03:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.790+0000 7f773bda7740 1 -- 172.21.15.78:0/614855764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 msgr2=0x55c5f7b4aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.790+0000 7f773bda7740 1 --2- 172.21.15.78:0/614855764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b4aa50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7724019040 tx=0x7f77240092a0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.790+0000 7f773bda7740 1 -- 172.21.15.78:0/614855764 shutdown_connections 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.790+0000 7f773bda7740 1 --2- 172.21.15.78:0/614855764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b50790 0x55c5f7b683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.790+0000 7f773bda7740 1 --2- 172.21.15.78:0/614855764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b44800 0x55c5f7b4af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.790+0000 7f773bda7740 1 --2- 172.21.15.78:0/614855764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b4aa50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.790+0000 7f773bda7740 1 -- 172.21.15.78:0/614855764 >> 172.21.15.78:0/614855764 conn(0x55c5f7b41d10 msgr2=0x55c5f7b4f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f773bda7740 1 -- 172.21.15.78:0/614855764 shutdown_connections 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f773bda7740 1 -- 172.21.15.78:0/614855764 wait complete. 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f773bda7740 1 -- start start 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f773bda7740 1 Processor -- start 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f773bda7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b54fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f773bda7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b44800 0x55c5f7b55560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f773bda7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b50790 0x55c5f7bc6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f773bda7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77341b1bd0 con 0x55c5f7b43ee0 2024-09-11T01:03:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f773bda7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77341b1440 con 0x55c5f7b44800 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f773bda7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77341ae680 con 0x55c5f7b50790 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f773bda7740 10 client.0 _ll_register_callbacks cb 0x7f7724009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f7732ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b54fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f7732ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b54fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50626/0 (socket says 172.21.15.78:50626) 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f7732ffd640 1 -- 172.21.15.78:0/224712892 learned_addr learned my addr 172.21.15.78:0/224712892 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.791+0000 7f772bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b44800 0x55c5f7b55560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f77337fe640 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b50790 0x55c5f7bc6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f7732ffd640 1 -- 172.21.15.78:0/224712892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b50790 msgr2=0x55c5f7bc6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f7732ffd640 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b50790 0x55c5f7bc6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f7732ffd640 1 -- 172.21.15.78:0/224712892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b44800 msgr2=0x55c5f7b55560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f7732ffd640 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b44800 0x55c5f7b55560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f7732ffd640 1 -- 172.21.15.78:0/224712892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5f7b4fec0 con 0x55c5f7b43ee0 2024-09-11T01:03:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f772bfff640 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b44800 0x55c5f7b55560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f7732ffd640 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b54fe0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7724029510 tx=0x7f7724004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c02ab4d769064f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f7712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:03:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f7712ffd640 1 -- 172.21.15.78:0/224712892 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724042070 con 0x55c5f7b43ee0 2024-09-11T01:03:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f773bda7740 1 -- 172.21.15.78:0/224712892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77240297b0 con 0x55c5f7b43ee0 2024-09-11T01:03:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f7712ffd640 1 -- 172.21.15.78:0/224712892 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7724029d70 con 0x55c5f7b43ee0 2024-09-11T01:03:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f7712ffd640 1 -- 172.21.15.78:0/224712892 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7724004730 con 0x55c5f7b43ee0 2024-09-11T01:03:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f773bda7740 10 client.16413 fetch_fsmap learned FSMap version 38 2024-09-11T01:03:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.792+0000 7f773bda7740 1 -- 172.21.15.78:0/224712892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c5f7b42790 con 0x55c5f7b43ee0 2024-09-11T01:03:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.793+0000 7f7712ffd640 1 -- 172.21.15.78:0/224712892 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77240297b0 con 0x55c5f7b43ee0 2024-09-11T01:03:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.793+0000 7f7712ffd640 1 -- 172.21.15.78:0/224712892 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f772403a070 con 0x55c5f7b43ee0 2024-09-11T01:03:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.793+0000 7f773bda7740 10 client.16413 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:03:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.793+0000 7f773bda7740 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:03:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.793+0000 7f773bda7740 20 client.16413 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.793+0000 7f773bda7740 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c5f7bce700 0x55c5f7bd0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.793+0000 7f773bda7740 4 client.16413 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:03:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.793+0000 7f773bda7740 1 -- 172.21.15.78:0/224712892 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c5f7ab8860 con 0x55c5f7bce700 2024-09-11T01:03:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.793+0000 7f772bfff640 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c5f7bce700 0x55c5f7bd0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.793+0000 7f7712ffd640 10 client.16413 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.793+0000 7f772bfff640 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c5f7bce700 0x55c5f7bd0b20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=871b911ba93b5a94 server_cookie=9d06e4296ac19d1c in_seq=0 out_seq=0 2024-09-11T01:03:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f7712ffd640 1 -- 172.21.15.78:0/224712892 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c5f7ab8860 con 0x55c5f7bce700 2024-09-11T01:03:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f7712ffd640 10 client.16413 handle_command_reply: tid=0 2024-09-11T01:03:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f773bda7740 1 client.16413 shutdown 2024-09-11T01:03:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f773bda7740 1 -- 172.21.15.78:0/224712892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 msgr2=0x55c5f7b54fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f773bda7740 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b54fe0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7724029510 tx=0x7f7724004640 comp rx=0 tx=0).stop 2024-09-11T01:03:39.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f773bda7740 1 -- 172.21.15.78:0/224712892 shutdown_connections 2024-09-11T01:03:39.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f773bda7740 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5f7b50790 0x55c5f7bc6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f773bda7740 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5f7b44800 0x55c5f7b55560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f773bda7740 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c5f7bce700 0x55c5f7bd0b20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f773bda7740 1 --2- 172.21.15.78:0/224712892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5f7b43ee0 0x55c5f7b54fe0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f7712ffd640 0 client.16413 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:39.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f773bda7740 1 -- 172.21.15.78:0/224712892 >> 172.21.15.78:0/224712892 conn(0x55c5f7b41d10 msgr2=0x55c5f7b47090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:39.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f773bda7740 1 -- 172.21.15.78:0/224712892 shutdown_connections 2024-09-11T01:03:39.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f773bda7740 1 -- 172.21.15.78:0/224712892 wait complete. 2024-09-11T01:03:39.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.796+0000 7f773bda7740 20 client.16413 trim_cache size 0 max 16384 2024-09-11T01:03:39.804 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:03:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:03:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "8eb1d395-7f13-4fb9-9ea8-5a80169444b5", 2024-09-11T01:03:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:03:39.804 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:03:39.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.797+0000 7f77399d2640 1 -- 172.21.15.78:0/388366292 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f77200b5450 msgr2=0x7f77200b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.797+0000 7f77399d2640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f77200b5450 0x7f77200b7910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55c5f7b47060 tx=0x7f7724005f90 comp rx=0 tx=0).stop 2024-09-11T01:03:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.797+0000 7f77399d2640 1 -- 172.21.15.78:0/388366292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77341076a0 msgr2=0x7f7734110b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.797+0000 7f77399d2640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77341076a0 0x7f7734110b90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f771c00e9f0 tx=0x7f771c00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.797+0000 7f77399d2640 1 -- 172.21.15.78:0/388366292 shutdown_connections 2024-09-11T01:03:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.797+0000 7f77399d2640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f77200b5450 0x7f77200b7910 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.797+0000 7f77399d2640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7734108020 0x7f77341110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.797+0000 7f77399d2640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77341076a0 0x7f7734110b90 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.797+0000 7f77399d2640 1 --2- 172.21.15.78:0/388366292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7734101c90 0x7f7734110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.797+0000 7f77399d2640 1 -- 172.21.15.78:0/388366292 >> 172.21.15.78:0/388366292 conn(0x7f77340fd700 msgr2=0x7f77340ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.797+0000 7f77399d2640 1 -- 172.21.15.78:0/388366292 shutdown_connections 2024-09-11T01:03:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:39.797+0000 7f77399d2640 1 -- 172.21.15.78:0/388366292 wait complete. 2024-09-11T01:03:39.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:39 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:03:39.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:39 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:39.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:39 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:03:39.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:39 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:40.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:39 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:03:40.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:39 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:40.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:39 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:03:40.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:39 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:40.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:39 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:03:40.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:39 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:40.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:39 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:03:40.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:39 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:40.428 INFO:tasks.workunit.client.0.smithi078.stdout: 1 386478 6.46 MB/sec execute 108 sec latency 349.109 ms 2024-09-11T01:03:40.630 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:03:40.892 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:03:41.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:03:41.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:03:41.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:03:41.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:03:41.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:03:41.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:03:41.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:03:41.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:03:41.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:03:41.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:40 smithi110 ceph-mon[34712]: pgmap v467: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-09-11T01:03:41.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:03:41.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:03:41.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.226 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:40 smithi112 ceph-mon[35384]: pgmap v467: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-09-11T01:03:41.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:03:41.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:03:41.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:03:41.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:03:41.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:03:41.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:03:41.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:03:41.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:03:41.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:03:41.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:03:41.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:40 smithi078 ceph-mon[31315]: pgmap v467: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-09-11T01:03:41.428 INFO:tasks.workunit.client.0.smithi078.stdout: 1 387831 6.46 MB/sec execute 109 sec latency 59.985 ms 2024-09-11T01:03:41.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.460+0000 7fe0b6079640 1 -- 172.21.15.78:0/2187881743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b0103be0 msgr2=0x7fe0b0104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:41.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.460+0000 7fe0b6079640 1 --2- 172.21.15.78:0/2187881743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b0103be0 0x7fe0b0104020 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400d910 tx=0x7fe0a402c380 comp rx=0 tx=0).stop 2024-09-11T01:03:41.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.461+0000 7fe0b6079640 1 -- 172.21.15.78:0/2187881743 shutdown_connections 2024-09-11T01:03:41.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.461+0000 7fe0b6079640 1 --2- 172.21.15.78:0/2187881743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0b0104560 0x7fe0b010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.461+0000 7fe0b6079640 1 --2- 172.21.15.78:0/2187881743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b0103be0 0x7fe0b0104020 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.461+0000 7fe0b6079640 1 --2- 172.21.15.78:0/2187881743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0101660 0x7fe0b0101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.461+0000 7fe0b6079640 1 -- 172.21.15.78:0/2187881743 >> 172.21.15.78:0/2187881743 conn(0x7fe0b00fca20 msgr2=0x7fe0b00fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:41.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.461+0000 7fe0b6079640 1 -- 172.21.15.78:0/2187881743 shutdown_connections 2024-09-11T01:03:41.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.461+0000 7fe0b6079640 1 -- 172.21.15.78:0/2187881743 wait complete. 2024-09-11T01:03:41.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.462+0000 7fe0b6079640 1 Processor -- start 2024-09-11T01:03:41.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.462+0000 7fe0b6079640 1 -- start start 2024-09-11T01:03:41.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.462+0000 7fe0b6079640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0101660 0x7fe0b019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.462+0000 7fe0b6079640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b0103be0 0x7fe0b019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.462+0000 7fe0b6079640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0b0104560 0x7fe0b01a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.462+0000 7fe0b6079640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0110be0 con 0x7fe0b0103be0 2024-09-11T01:03:41.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.462+0000 7fe0b6079640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0110a60 con 0x7fe0b0104560 2024-09-11T01:03:41.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.462+0000 7fe0b6079640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0110d60 con 0x7fe0b0101660 2024-09-11T01:03:41.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b0103be0 0x7fe0b019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b0103be0 0x7fe0b019fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50642/0 (socket says 172.21.15.78:50642) 2024-09-11T01:03:41.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0affff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0b0104560 0x7fe0b01a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0aeffd640 1 -- 172.21.15.78:0/117915389 learned_addr learned my addr 172.21.15.78:0/117915389 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:41.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0101660 0x7fe0b019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0aeffd640 1 -- 172.21.15.78:0/117915389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0101660 msgr2=0x7fe0b019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:41.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0aeffd640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0101660 0x7fe0b019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0aeffd640 1 -- 172.21.15.78:0/117915389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0b0104560 msgr2=0x7fe0b01a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:41.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0aeffd640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0b0104560 0x7fe0b01a51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0aeffd640 1 -- 172.21.15.78:0/117915389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b01a5990 con 0x7fe0b0103be0 2024-09-11T01:03:41.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0101660 0x7fe0b019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:03:41.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0affff640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0b0104560 0x7fe0b01a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:41.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.463+0000 7fe0aeffd640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b0103be0 0x7fe0b019fe20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe0a402f040 tx=0x7fe0a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:41.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.464+0000 7fe0acff9640 1 -- 172.21.15.78:0/117915389 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4004030 con 0x7fe0b0103be0 2024-09-11T01:03:41.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.464+0000 7fe0b6079640 1 -- 172.21.15.78:0/117915389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b01a5c20 con 0x7fe0b0103be0 2024-09-11T01:03:41.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.464+0000 7fe0b6079640 1 -- 172.21.15.78:0/117915389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b01ac340 con 0x7fe0b0103be0 2024-09-11T01:03:41.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.464+0000 7fe0acff9640 1 -- 172.21.15.78:0/117915389 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a4005130 con 0x7fe0b0103be0 2024-09-11T01:03:41.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.464+0000 7fe0acff9640 1 -- 172.21.15.78:0/117915389 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4003ac0 con 0x7fe0b0103be0 2024-09-11T01:03:41.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.467+0000 7fe0b844e740 1 Processor -- start 2024-09-11T01:03:41.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.467+0000 7fe0b844e740 1 -- start start 2024-09-11T01:03:41.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.467+0000 7fe0acff9640 1 -- 172.21.15.78:0/117915389 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe0a4003c60 con 0x7fe0b0103be0 2024-09-11T01:03:41.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.467+0000 7fe0b844e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab90680860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.467+0000 7fe0b844e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab90680da0 0x55ab9068c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.467+0000 7fe0b844e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab9068cd30 0x55ab9068f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.467+0000 7fe0b844e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ab905f6e50 con 0x55ab9068cd30 2024-09-11T01:03:41.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.467+0000 7fe0b844e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ab905e6280 con 0x55ab90680480 2024-09-11T01:03:41.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.467+0000 7fe0b844e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ab905ea090 con 0x55ab90680da0 2024-09-11T01:03:41.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.468+0000 7fe0affff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab90680860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.468+0000 7fe0affff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab90680860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:47504/0 (socket says 172.21.15.78:47504) 2024-09-11T01:03:41.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.468+0000 7fe0affff640 1 -- 172.21.15.78:0/1065603553 learned_addr learned my addr 172.21.15.78:0/1065603553 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:41.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.468+0000 7fe0aeffd640 1 --2- 172.21.15.78:0/1065603553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab9068cd30 0x55ab9068f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.468+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/1065603553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab90680da0 0x55ab9068c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.468+0000 7fe0affff640 1 -- 172.21.15.78:0/1065603553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab90680da0 msgr2=0x55ab9068c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:41.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.468+0000 7fe0affff640 1 --2- 172.21.15.78:0/1065603553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab90680da0 0x55ab9068c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.468+0000 7fe0affff640 1 -- 172.21.15.78:0/1065603553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab9068cd30 msgr2=0x55ab9068f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:41.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.468+0000 7fe0affff640 1 --2- 172.21.15.78:0/1065603553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab9068cd30 0x55ab9068f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.468+0000 7fe0affff640 1 -- 172.21.15.78:0/1065603553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab90344650 con 0x55ab90680480 2024-09-11T01:03:41.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.468+0000 7fe0affff640 1 --2- 172.21.15.78:0/1065603553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab90680860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800ad50 tx=0x7fe0a800a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69eb0a69e446f588 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:41.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.469+0000 7fe0acff9640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe0980b54d0 0x7fe0980b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.469+0000 7fe095ffb640 1 -- 172.21.15.78:0/1065603553 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8004b20 con 0x55ab90680480 2024-09-11T01:03:41.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.469+0000 7fe095ffb640 1 -- 172.21.15.78:0/1065603553 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a8004cc0 con 0x55ab90680480 2024-09-11T01:03:41.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.469+0000 7fe095ffb640 1 -- 172.21.15.78:0/1065603553 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a80054f0 con 0x55ab90680480 2024-09-11T01:03:41.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.469+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe0980b54d0 0x7fe0980b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.469+0000 7fe0acff9640 1 -- 172.21.15.78:0/117915389 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe0a4105820 con 0x7fe0b0103be0 2024-09-11T01:03:41.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.469+0000 7fe0b844e740 1 -- 172.21.15.78:0/1065603553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 msgr2=0x55ab90680860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:41.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.469+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe0980b54d0 0x7fe0980b7990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55ab9067f190 tx=0x7fe0a0005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:41.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.469+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1065603553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab90680860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800ad50 tx=0x7fe0a800a560 comp rx=0 tx=0).stop 2024-09-11T01:03:41.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.470+0000 7fe0b844e740 1 -- 172.21.15.78:0/1065603553 shutdown_connections 2024-09-11T01:03:41.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.470+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1065603553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab9068cd30 0x55ab9068f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.470+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1065603553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab90680da0 0x55ab9068c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.470+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1065603553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab90680860 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.470+0000 7fe0b844e740 1 -- 172.21.15.78:0/1065603553 >> 172.21.15.78:0/1065603553 conn(0x55ab9067e2b0 msgr2=0x55ab9067f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:41.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.470+0000 7fe0b844e740 1 -- 172.21.15.78:0/1065603553 shutdown_connections 2024-09-11T01:03:41.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.470+0000 7fe0b844e740 1 -- 172.21.15.78:0/1065603553 wait complete. 2024-09-11T01:03:41.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.470+0000 7fe0b844e740 1 -- start start 2024-09-11T01:03:41.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.470+0000 7fe0b844e740 1 Processor -- start 2024-09-11T01:03:41.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0b844e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680480 0x55ab9068c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0b844e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680da0 0x55ab90689950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0b844e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab9068cd30 0x55ab90689e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0b844e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ab90447700 con 0x55ab90680480 2024-09-11T01:03:41.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0b844e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ab90581110 con 0x55ab90680da0 2024-09-11T01:03:41.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0b844e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ab9068f9b0 con 0x55ab9068cd30 2024-09-11T01:03:41.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0b844e740 10 client.0 _ll_register_callbacks cb 0x55ab9059a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:41.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680480 0x55ab9068c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0aeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680da0 0x55ab90689950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0aeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680da0 0x55ab90689950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:47518/0 (socket says 172.21.15.78:47518) 2024-09-11T01:03:41.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680480 0x55ab9068c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50662/0 (socket says 172.21.15.78:50662) 2024-09-11T01:03:41.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0aeffd640 1 -- 172.21.15.78:0/946361569 learned_addr learned my addr 172.21.15.78:0/946361569 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:41.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0affff640 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab9068cd30 0x55ab90689e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0aeffd640 1 -- 172.21.15.78:0/946361569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab9068cd30 msgr2=0x55ab90689e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:41.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0aeffd640 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab9068cd30 0x55ab90689e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0aeffd640 1 -- 172.21.15.78:0/946361569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680480 msgr2=0x55ab9068c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:41.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0aeffd640 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680480 0x55ab9068c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0aeffd640 1 -- 172.21.15.78:0/946361569 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab905aec40 con 0x55ab90680da0 2024-09-11T01:03:41.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680480 0x55ab9068c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:41.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0affff640 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab9068cd30 0x55ab90689e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:03:41.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0aeffd640 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680da0 0x55ab90689950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4103040 tx=0x7fe0a4009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de676c773669f5aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:41.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe08effd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:03:41.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe08effd640 1 -- 172.21.15.78:0/946361569 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a40c7bb0 con 0x55ab90680da0 2024-09-11T01:03:41.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe08effd640 1 -- 172.21.15.78:0/946361569 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe0a40c7f80 con 0x55ab90680da0 2024-09-11T01:03:41.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.471+0000 7fe0b844e740 1 -- 172.21.15.78:0/946361569 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a8004cc0 con 0x55ab90680da0 2024-09-11T01:03:41.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.472+0000 7fe08effd640 1 -- 172.21.15.78:0/946361569 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe0a40cb480 con 0x55ab90680da0 2024-09-11T01:03:41.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.472+0000 7fe0b844e740 10 client.25766 fetch_fsmap learned FSMap version 38 2024-09-11T01:03:41.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.472+0000 7fe0b844e740 1 -- 172.21.15.78:0/946361569 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ab9069b250 con 0x55ab90680da0 2024-09-11T01:03:41.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.472+0000 7fe08effd640 1 -- 172.21.15.78:0/946361569 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a8004cc0 con 0x55ab90680da0 2024-09-11T01:03:41.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.472+0000 7fe08effd640 1 -- 172.21.15.78:0/946361569 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fe0a403ba40 con 0x55ab90680da0 2024-09-11T01:03:41.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.472+0000 7fe0b844e740 10 client.25766 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:03:41.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.472+0000 7fe0b844e740 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:03:41.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.472+0000 7fe0b844e740 20 client.25766 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:41.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.472+0000 7fe0b844e740 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab9069c6e0 0x55ab9069eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.472+0000 7fe0b844e740 4 client.25766 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:03:41.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.472+0000 7fe0b844e740 1 -- 172.21.15.78:0/946361569 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab905f8da0 con 0x55ab9069c6e0 2024-09-11T01:03:41.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.472+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab9069c6e0 0x55ab9069eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe08effd640 10 client.25766 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:41.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab9069c6e0 0x55ab9069eb60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97bba120dfb2ddb9 server_cookie=2ef7252c0b76f631 in_seq=0 out_seq=0 2024-09-11T01:03:41.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe08effd640 1 -- 172.21.15.78:0/946361569 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab905f8da0 con 0x55ab9069c6e0 2024-09-11T01:03:41.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe08effd640 10 client.25766 handle_command_reply: tid=0 2024-09-11T01:03:41.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe0b844e740 1 client.25766 shutdown 2024-09-11T01:03:41.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe0b844e740 1 -- 172.21.15.78:0/946361569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680da0 msgr2=0x55ab90689950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:41.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe0b844e740 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680da0 0x55ab90689950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4103040 tx=0x7fe0a4009230 comp rx=0 tx=0).stop 2024-09-11T01:03:41.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe0b844e740 1 -- 172.21.15.78:0/946361569 shutdown_connections 2024-09-11T01:03:41.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe0b844e740 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab9068cd30 0x55ab90689e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe0b844e740 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680da0 0x55ab90689950 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe0b844e740 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab9069c6e0 0x55ab9069eb60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe0b844e740 1 --2- 172.21.15.78:0/946361569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680480 0x55ab9068c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe08effd640 0 client.25766 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:41.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.473+0000 7fe0b844e740 1 -- 172.21.15.78:0/946361569 >> 172.21.15.78:0/946361569 conn(0x55ab9067e2b0 msgr2=0x55ab90683e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:41.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.474+0000 7fe0b844e740 1 -- 172.21.15.78:0/946361569 shutdown_connections 2024-09-11T01:03:41.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.474+0000 7fe0b844e740 1 -- 172.21.15.78:0/946361569 wait complete. 2024-09-11T01:03:41.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.474+0000 7fe0b844e740 20 client.25766 trim_cache size 0 max 16384 2024-09-11T01:03:41.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0b844e740 1 Processor -- start 2024-09-11T01:03:41.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0b844e740 1 -- start start 2024-09-11T01:03:41.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0b844e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab9067f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0b844e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680da0 0x55ab9067f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0b844e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab9068cd30 0x55ab90686b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0b844e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c00e620 con 0x55ab90680da0 2024-09-11T01:03:41.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0b844e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c0102e0 con 0x55ab90680480 2024-09-11T01:03:41.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0b844e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c00ecf0 con 0x55ab9068cd30 2024-09-11T01:03:41.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0af7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab9067f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680da0 0x55ab9067f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0af7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab9067f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:47524/0 (socket says 172.21.15.78:47524) 2024-09-11T01:03:41.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0af7fe640 1 -- 172.21.15.78:0/1553941154 learned_addr learned my addr 172.21.15.78:0/1553941154 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:41.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.493+0000 7fe0affff640 1 --2- 172.21.15.78:0/1553941154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab9068cd30 0x55ab90686b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.494+0000 7fe0aeffd640 1 -- 172.21.15.78:0/1553941154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab9068cd30 msgr2=0x55ab90686b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:41.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.494+0000 7fe0aeffd640 1 --2- 172.21.15.78:0/1553941154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab9068cd30 0x55ab90686b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.494+0000 7fe0aeffd640 1 -- 172.21.15.78:0/1553941154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 msgr2=0x55ab9067f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:41.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.494+0000 7fe0aeffd640 1 --2- 172.21.15.78:0/1553941154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab9067f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.494+0000 7fe0aeffd640 1 -- 172.21.15.78:0/1553941154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab9068f490 con 0x55ab90680da0 2024-09-11T01:03:41.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.494+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/1553941154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab9067f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:41.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.494+0000 7fe0aeffd640 1 --2- 172.21.15.78:0/1553941154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680da0 0x55ab9067f8b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40092a0 tx=0x7fe0a40b3050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccc4ef4ff79f6da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:41.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.494+0000 7fe0957fa640 1 -- 172.21.15.78:0/1553941154 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a40cb660 con 0x55ab90680da0 2024-09-11T01:03:41.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.494+0000 7fe0957fa640 1 -- 172.21.15.78:0/1553941154 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a4030020 con 0x55ab90680da0 2024-09-11T01:03:41.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.494+0000 7fe0957fa640 1 -- 172.21.15.78:0/1553941154 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a40cbae0 con 0x55ab90680da0 2024-09-11T01:03:41.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.495+0000 7fe0b844e740 1 -- 172.21.15.78:0/1553941154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680da0 msgr2=0x55ab9067f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:41.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.495+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1553941154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680da0 0x55ab9067f8b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40092a0 tx=0x7fe0a40b3050 comp rx=0 tx=0).stop 2024-09-11T01:03:41.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.495+0000 7fe0b844e740 1 -- 172.21.15.78:0/1553941154 shutdown_connections 2024-09-11T01:03:41.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.495+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1553941154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab9068cd30 0x55ab90686b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.495+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1553941154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab90680da0 0x55ab9067f8b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.495+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1553941154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab9067f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.495+0000 7fe0b844e740 1 -- 172.21.15.78:0/1553941154 >> 172.21.15.78:0/1553941154 conn(0x55ab9067e2b0 msgr2=0x55ab906881d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:41.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.495+0000 7fe0b844e740 1 -- 172.21.15.78:0/1553941154 shutdown_connections 2024-09-11T01:03:41.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.495+0000 7fe0b844e740 1 -- 172.21.15.78:0/1553941154 wait complete. 2024-09-11T01:03:41.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.495+0000 7fe0b844e740 1 -- start start 2024-09-11T01:03:41.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.495+0000 7fe0b844e740 1 Processor -- start 2024-09-11T01:03:41.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0b844e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab90694ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0b844e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab90680da0 0x55ab90695220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0b844e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab9068cd30 0x55ab9068fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0b844e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c00fb50 con 0x55ab9068cd30 2024-09-11T01:03:41.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0b844e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c00f3c0 con 0x55ab90680480 2024-09-11T01:03:41.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0b844e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c00c600 con 0x55ab90680da0 2024-09-11T01:03:41.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0b844e740 10 client.0 _ll_register_callbacks cb 0x7fe0a410b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:03:41.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab9068cd30 0x55ab9068fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0af7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab90694ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab9068cd30 0x55ab9068fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50680/0 (socket says 172.21.15.78:50680) 2024-09-11T01:03:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0affff640 1 -- 172.21.15.78:0/1732288763 learned_addr learned my addr 172.21.15.78:0/1732288763 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:03:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0aeffd640 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab90680da0 0x55ab90695220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0affff640 1 -- 172.21.15.78:0/1732288763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab90680da0 msgr2=0x55ab90695220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0affff640 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab90680da0 0x55ab90695220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0affff640 1 -- 172.21.15.78:0/1732288763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 msgr2=0x55ab90694ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:03:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0affff640 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab90694ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0affff640 1 -- 172.21.15.78:0/1732288763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab906901d0 con 0x55ab9068cd30 2024-09-11T01:03:41.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0aeffd640 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab90680da0 0x55ab90695220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:41.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab90694ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:03:41.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe0affff640 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab9068cd30 0x55ab9068fc90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800ad20 tx=0x7fe0a8004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5e4c309242f13f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:03:41.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe08effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:03:41.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.496+0000 7fe08effd640 1 -- 172.21.15.78:0/1732288763 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a800d9e0 con 0x55ab9068cd30 2024-09-11T01:03:41.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe08effd640 1 -- 172.21.15.78:0/1732288763 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe0a800db80 con 0x55ab9068cd30 2024-09-11T01:03:41.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe0b844e740 1 -- 172.21.15.78:0/1732288763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a4030020 con 0x55ab9068cd30 2024-09-11T01:03:41.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe08effd640 1 -- 172.21.15.78:0/1732288763 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe0a800ddc0 con 0x55ab9068cd30 2024-09-11T01:03:41.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe0b844e740 10 client.16431 fetch_fsmap learned FSMap version 38 2024-09-11T01:03:41.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe0b844e740 1 -- 172.21.15.78:0/1732288763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ab90709e60 con 0x55ab9068cd30 2024-09-11T01:03:41.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe08effd640 1 -- 172.21.15.78:0/1732288763 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a4030020 con 0x55ab9068cd30 2024-09-11T01:03:41.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe08effd640 1 -- 172.21.15.78:0/1732288763 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fe0a8033020 con 0x55ab9068cd30 2024-09-11T01:03:41.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe0b844e740 10 client.16431 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:03:41.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe0b844e740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:03:41.549 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:03:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:03:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:03:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:03:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "8eb1d395-7f13-4fb9-9ea8-5a80169444b5", 2024-09-11T01:03:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:03:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:03:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:03:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:03:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:03:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:03:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:03:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:03:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:03:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:03:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:03:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:03:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:03:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:03:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:03:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:03:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:03:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:03:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:03:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:03:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:03:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:03:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:03:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:03:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:03:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:03:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:03:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:03:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:03:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:03:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:03:41.556 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:03:41.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe0b844e740 20 client.16431 populate_metadata read hostname 'smithi078' 2024-09-11T01:03:41.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab9070a2f0 0x55ab9070c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:03:41.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe0b844e740 4 client.16431 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:03:41.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe0b844e740 1 -- 172.21.15.78:0/1732288763 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab905f4850 con 0x55ab9070a2f0 2024-09-11T01:03:41.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.497+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab9070a2f0 0x55ab9070c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:03:41.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe08effd640 10 client.16431 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:41.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab9070a2f0 0x55ab9070c710 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ee8f67c857a4d07 server_cookie=b3207fc468c9413b in_seq=0 out_seq=0 2024-09-11T01:03:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe08effd640 1 -- 172.21.15.78:0/1732288763 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x55ab905f4850 con 0x55ab9070a2f0 2024-09-11T01:03:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe08effd640 10 client.16431 handle_command_reply: tid=0 2024-09-11T01:03:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe0b844e740 1 client.16431 shutdown 2024-09-11T01:03:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe0b844e740 1 -- 172.21.15.78:0/1732288763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab9068cd30 msgr2=0x55ab9068fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab9068cd30 0x55ab9068fc90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800ad20 tx=0x7fe0a8004c30 comp rx=0 tx=0).stop 2024-09-11T01:03:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe0b844e740 1 -- 172.21.15.78:0/1732288763 shutdown_connections 2024-09-11T01:03:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ab9070a2f0 0x55ab9070c710 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab9068cd30 0x55ab9068fc90 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab90680da0 0x55ab90695220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe0b844e740 1 --2- 172.21.15.78:0/1732288763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab90680480 0x55ab90694ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe0b844e740 1 -- 172.21.15.78:0/1732288763 >> 172.21.15.78:0/1732288763 conn(0x55ab9067e2b0 msgr2=0x55ab9067f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe08effd640 0 client.16431 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:03:41.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.498+0000 7fe0b844e740 1 -- 172.21.15.78:0/1732288763 shutdown_connections 2024-09-11T01:03:41.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b844e740 1 -- 172.21.15.78:0/1732288763 wait complete. 2024-09-11T01:03:41.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b844e740 20 client.16431 trim_cache size 0 max 16384 2024-09-11T01:03:41.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b6079640 1 -- 172.21.15.78:0/117915389 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe0980b54d0 msgr2=0x7fe0980b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:41.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b6079640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe0980b54d0 0x7fe0980b7990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55ab9067f190 tx=0x7fe0a0005f90 comp rx=0 tx=0).stop 2024-09-11T01:03:41.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b6079640 1 -- 172.21.15.78:0/117915389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b0103be0 msgr2=0x7fe0b019fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:03:41.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b6079640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b0103be0 0x7fe0b019fe20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe0a402f040 tx=0x7fe0a4004730 comp rx=0 tx=0).stop 2024-09-11T01:03:41.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b6079640 1 -- 172.21.15.78:0/117915389 shutdown_connections 2024-09-11T01:03:41.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b6079640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe0980b54d0 0x7fe0980b7990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b6079640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0b0104560 0x7fe0b01a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b6079640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b0103be0 0x7fe0b019fe20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b6079640 1 --2- 172.21.15.78:0/117915389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0101660 0x7fe0b019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:03:41.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b6079640 1 -- 172.21.15.78:0/117915389 >> 172.21.15.78:0/117915389 conn(0x7fe0b00fca20 msgr2=0x7fe0b00fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:03:41.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b6079640 1 -- 172.21.15.78:0/117915389 shutdown_connections 2024-09-11T01:03:41.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:03:41.499+0000 7fe0b6079640 1 -- 172.21.15.78:0/117915389 wait complete. 2024-09-11T01:03:42.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:42 smithi078 ceph-mon[31315]: pgmap v468: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-09-11T01:03:42.428 INFO:tasks.workunit.client.0.smithi078.stdout: 1 389100 6.45 MB/sec execute 110 sec latency 484.348 ms 2024-09-11T01:03:42.451 INFO:tasks.cephfs.filesystem:scrub status for tag:8eb1d395-7f13-4fb9-9ea8-5a80169444b5 - {'path': '/', 'tag': '8eb1d395-7f13-4fb9-9ea8-5a80169444b5', 'options': 'recursive,force'} 2024-09-11T01:03:42.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:42 smithi112 ceph-mon[35384]: pgmap v468: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-09-11T01:03:42.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:42 smithi110 ceph-mon[34712]: pgmap v468: 97 pgs: 97 active+clean; 224 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-09-11T01:03:43.429 INFO:tasks.workunit.client.0.smithi078.stdout: 1 390784 6.50 MB/sec execute 111 sec latency 11.024 ms 2024-09-11T01:03:44.429 INFO:tasks.workunit.client.0.smithi078.stdout: 1 391911 6.49 MB/sec execute 112 sec latency 288.677 ms 2024-09-11T01:03:44.855 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:44 smithi110 ceph-mon[34712]: pgmap v469: 97 pgs: 97 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-09-11T01:03:44.855 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:44 smithi112 ceph-mon[35384]: pgmap v469: 97 pgs: 97 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-09-11T01:03:44.855 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:44 smithi078 ceph-mon[31315]: pgmap v469: 97 pgs: 97 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-09-11T01:03:45.429 INFO:tasks.workunit.client.0.smithi078.stdout: 1 391911 6.43 MB/sec execute 113 sec latency 1289.013 ms 2024-09-11T01:03:46.429 INFO:tasks.workunit.client.0.smithi078.stdout: 1 391911 6.37 MB/sec execute 114 sec latency 2289.134 ms 2024-09-11T01:03:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:46 smithi112 ceph-mon[35384]: pgmap v470: 97 pgs: 97 active+clean; 229 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-09-11T01:03:46.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:46 smithi078 ceph-mon[31315]: pgmap v470: 97 pgs: 97 active+clean; 229 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-09-11T01:03:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:46 smithi110 ceph-mon[34712]: pgmap v470: 97 pgs: 97 active+clean; 229 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-09-11T01:03:47.429 INFO:tasks.workunit.client.0.smithi078.stdout: 1 392621 6.33 MB/sec execute 115 sec latency 3048.900 ms 2024-09-11T01:03:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:47 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:03:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:47 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:03:47.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:47 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:03:47.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:47 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:03:47.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:47 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:03:47.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:47 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:03:48.429 INFO:tasks.workunit.client.0.smithi078.stdout: 1 394340 6.38 MB/sec execute 116 sec latency 24.882 ms 2024-09-11T01:03:48.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:48 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:03:48.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:48 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:03:48.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:48 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:03:48.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:48 smithi078 ceph-mon[31315]: pgmap v471: 97 pgs: 97 active+clean; 230 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 986 KiB/s wr, 154 op/s 2024-09-11T01:03:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:48 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:03:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:48 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:03:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:48 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:03:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:48 smithi110 ceph-mon[34712]: pgmap v471: 97 pgs: 97 active+clean; 230 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 986 KiB/s wr, 154 op/s 2024-09-11T01:03:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:48 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:03:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:48 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:03:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:48 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:03:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:48 smithi112 ceph-mon[35384]: pgmap v471: 97 pgs: 97 active+clean; 230 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 986 KiB/s wr, 154 op/s 2024-09-11T01:03:49.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 395459 6.37 MB/sec execute 117 sec latency 313.169 ms 2024-09-11T01:03:50.429 INFO:tasks.workunit.client.0.smithi078.stdout: 1 395459 6.32 MB/sec execute 118 sec latency 1313.313 ms 2024-09-11T01:03:50.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:50 smithi112 ceph-mon[35384]: pgmap v472: 97 pgs: 97 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-09-11T01:03:50.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:50 smithi078 ceph-mon[31315]: pgmap v472: 97 pgs: 97 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-09-11T01:03:50.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:50 smithi110 ceph-mon[34712]: pgmap v472: 97 pgs: 97 active+clean; 232 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-09-11T01:03:51.429 INFO:tasks.workunit.client.0.smithi078.stdout: 1 395459 6.27 MB/sec execute 119 sec latency 2313.414 ms 2024-09-11T01:03:52.429 INFO:tasks.workunit.client.0.smithi078.stdout: 1 396169 6.23 MB/sec execute 120 sec latency 3072.458 ms 2024-09-11T01:03:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:52 smithi112 ceph-mon[35384]: pgmap v473: 97 pgs: 97 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-09-11T01:03:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:52 smithi078 ceph-mon[31315]: pgmap v473: 97 pgs: 97 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-09-11T01:03:52.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:52 smithi110 ceph-mon[34712]: pgmap v473: 97 pgs: 97 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-09-11T01:03:53.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 397853 6.28 MB/sec execute 121 sec latency 57.230 ms 2024-09-11T01:03:54.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 399004 6.27 MB/sec execute 122 sec latency 335.116 ms 2024-09-11T01:03:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:54 smithi112 ceph-mon[35384]: pgmap v474: 97 pgs: 97 active+clean; 235 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-09-11T01:03:54.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:54 smithi078 ceph-mon[31315]: pgmap v474: 97 pgs: 97 active+clean; 235 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-09-11T01:03:54.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:54 smithi110 ceph-mon[34712]: pgmap v474: 97 pgs: 97 active+clean; 235 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-09-11T01:03:55.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 401246 6.33 MB/sec execute 123 sec latency 20.034 ms 2024-09-11T01:03:56.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 402551 6.32 MB/sec execute 124 sec latency 44.590 ms 2024-09-11T01:03:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:56 smithi112 ceph-mon[35384]: pgmap v475: 97 pgs: 97 active+clean; 236 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-09-11T01:03:56.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:56 smithi078 ceph-mon[31315]: pgmap v475: 97 pgs: 97 active+clean; 236 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-09-11T01:03:56.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:56 smithi110 ceph-mon[34712]: pgmap v475: 97 pgs: 97 active+clean; 236 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-09-11T01:03:57.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 402853 6.27 MB/sec execute 125 sec latency 803.768 ms 2024-09-11T01:03:58.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 404792 6.33 MB/sec execute 126 sec latency 34.967 ms 2024-09-11T01:03:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:03:58 smithi112 ceph-mon[35384]: pgmap v476: 97 pgs: 97 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-11T01:03:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:03:58 smithi078 ceph-mon[31315]: pgmap v476: 97 pgs: 97 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-11T01:03:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:03:58 smithi110 ceph-mon[34712]: pgmap v476: 97 pgs: 97 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-11T01:03:59.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 406092 6.32 MB/sec execute 127 sec latency 204.435 ms 2024-09-11T01:04:00.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 406092 6.27 MB/sec execute 128 sec latency 1204.535 ms 2024-09-11T01:04:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:00 smithi112 ceph-mon[35384]: pgmap v477: 97 pgs: 97 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2024-09-11T01:04:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:00 smithi078 ceph-mon[31315]: pgmap v477: 97 pgs: 97 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2024-09-11T01:04:00.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:00 smithi110 ceph-mon[34712]: pgmap v477: 97 pgs: 97 active+clean; 239 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2024-09-11T01:04:01.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 406092 6.23 MB/sec execute 129 sec latency 2204.668 ms 2024-09-11T01:04:02.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 406851 6.19 MB/sec execute 130 sec latency 2599.472 ms 2024-09-11T01:04:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:02 smithi112 ceph-mon[35384]: pgmap v478: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 202 op/s 2024-09-11T01:04:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:02 smithi078 ceph-mon[31315]: pgmap v478: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 202 op/s 2024-09-11T01:04:02.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:02 smithi110 ceph-mon[34712]: pgmap v478: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 202 op/s 2024-09-11T01:04:03.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 408568 6.24 MB/sec execute 131 sec latency 9.831 ms 2024-09-11T01:04:04.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 409641 6.23 MB/sec execute 132 sec latency 375.176 ms 2024-09-11T01:04:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:04 smithi112 ceph-mon[35384]: pgmap v479: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 204 op/s 2024-09-11T01:04:04.490 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:04 smithi110 ceph-mon[34712]: pgmap v479: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 204 op/s 2024-09-11T01:04:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:04 smithi078 ceph-mon[31315]: pgmap v479: 97 pgs: 97 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 204 op/s 2024-09-11T01:04:05.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 409641 6.18 MB/sec execute 133 sec latency 1375.213 ms 2024-09-11T01:04:06.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 409641 6.14 MB/sec execute 134 sec latency 2375.296 ms 2024-09-11T01:04:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:06 smithi112 ceph-mon[35384]: pgmap v480: 97 pgs: 97 active+clean; 242 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 256 op/s 2024-09-11T01:04:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:06 smithi078 ceph-mon[31315]: pgmap v480: 97 pgs: 97 active+clean; 242 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 256 op/s 2024-09-11T01:04:06.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:06 smithi110 ceph-mon[34712]: pgmap v480: 97 pgs: 97 active+clean; 242 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 256 op/s 2024-09-11T01:04:07.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 410350 6.10 MB/sec execute 135 sec latency 3132.975 ms 2024-09-11T01:04:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:04:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:04:07.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:04:08.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 412029 6.15 MB/sec execute 136 sec latency 10.598 ms 2024-09-11T01:04:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:04:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:04:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:04:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:08 smithi078 ceph-mon[31315]: pgmap v481: 97 pgs: 97 active+clean; 242 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-09-11T01:04:08.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:04:08.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:04:08.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:04:08.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:08 smithi110 ceph-mon[34712]: pgmap v481: 97 pgs: 97 active+clean; 242 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-09-11T01:04:08.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:04:08.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:04:08.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:04:08.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:08 smithi112 ceph-mon[35384]: pgmap v481: 97 pgs: 97 active+clean; 242 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-09-11T01:04:09.430 INFO:tasks.workunit.client.0.smithi078.stdout: 1 413183 6.14 MB/sec execute 137 sec latency 345.721 ms 2024-09-11T01:04:10.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 413183 6.10 MB/sec execute 138 sec latency 1345.798 ms 2024-09-11T01:04:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:10 smithi112 ceph-mon[35384]: pgmap v482: 97 pgs: 97 active+clean; 243 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 246 op/s 2024-09-11T01:04:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:10 smithi078 ceph-mon[31315]: pgmap v482: 97 pgs: 97 active+clean; 243 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 246 op/s 2024-09-11T01:04:10.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:10 smithi110 ceph-mon[34712]: pgmap v482: 97 pgs: 97 active+clean; 243 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 246 op/s 2024-09-11T01:04:11.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 413183 6.05 MB/sec execute 139 sec latency 2345.897 ms 2024-09-11T01:04:12.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 413901 6.02 MB/sec execute 140 sec latency 3104.442 ms 2024-09-11T01:04:12.453 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:04:12.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:12 smithi112 ceph-mon[35384]: pgmap v483: 97 pgs: 97 active+clean; 245 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 198 op/s 2024-09-11T01:04:12.477 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:12 smithi078 ceph-mon[31315]: pgmap v483: 97 pgs: 97 active+clean; 245 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 198 op/s 2024-09-11T01:04:12.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:12 smithi110 ceph-mon[34712]: pgmap v483: 97 pgs: 97 active+clean; 245 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 198 op/s 2024-09-11T01:04:12.734 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:04:13.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.264+0000 7f699f054640 1 -- 172.21.15.78:0/2161227266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6998103cd0 msgr2=0x7f69981040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:13.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.264+0000 7f699f054640 1 --2- 172.21.15.78:0/2161227266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6998103cd0 0x7f69981040b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f698c009860 tx=0x7f698c02a340 comp rx=0 tx=0).stop 2024-09-11T01:04:13.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.264+0000 7f699f054640 1 -- 172.21.15.78:0/2161227266 shutdown_connections 2024-09-11T01:04:13.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.264+0000 7f699f054640 1 --2- 172.21.15.78:0/2161227266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f699810d130 0x7f699810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.264+0000 7f699f054640 1 --2- 172.21.15.78:0/2161227266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69981045f0 0x7f699810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.264+0000 7f699f054640 1 --2- 172.21.15.78:0/2161227266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6998103cd0 0x7f69981040b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.264+0000 7f699f054640 1 -- 172.21.15.78:0/2161227266 >> 172.21.15.78:0/2161227266 conn(0x7f69980fd770 msgr2=0x7f69980ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:13.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.264+0000 7f699f054640 1 -- 172.21.15.78:0/2161227266 shutdown_connections 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.264+0000 7f699f054640 1 -- 172.21.15.78:0/2161227266 wait complete. 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.264+0000 7f699f054640 1 Processor -- start 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.264+0000 7f699f054640 1 -- start start 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699f054640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6998103cd0 0x7f69981992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699f054640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69981045f0 0x7f69981997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699f054640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f699810d130 0x7f699819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699f054640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6998112da0 con 0x7f69981045f0 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699f054640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6998112c20 con 0x7f699810d130 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699f054640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6998112f20 con 0x7f6998103cd0 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f6997fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69981045f0 0x7f69981997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699cdf0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6998103cd0 0x7f69981992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699cdf0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6998103cd0 0x7f69981992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:59522/0 (socket says 172.21.15.78:59522) 2024-09-11T01:04:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699cdf0640 1 -- 172.21.15.78:0/801835029 learned_addr learned my addr 172.21.15.78:0/801835029 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f6997fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69981045f0 0x7f69981997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39274/0 (socket says 172.21.15.78:39274) 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699d5f1640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f699810d130 0x7f699819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699cdf0640 1 -- 172.21.15.78:0/801835029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f699810d130 msgr2=0x7f699819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699cdf0640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f699810d130 0x7f699819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699cdf0640 1 -- 172.21.15.78:0/801835029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69981045f0 msgr2=0x7f69981997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699cdf0640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69981045f0 0x7f69981997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699cdf0640 1 -- 172.21.15.78:0/801835029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699819f360 con 0x7f6998103cd0 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.265+0000 7f699cdf0640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6998103cd0 0x7f69981992b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f698c035040 tx=0x7f698c032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.266+0000 7f6995ffb640 1 -- 172.21.15.78:0/801835029 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c037040 con 0x7f6998103cd0 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.266+0000 7f6995ffb640 1 -- 172.21.15.78:0/801835029 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698c041070 con 0x7f6998103cd0 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.266+0000 7f699f054640 1 -- 172.21.15.78:0/801835029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699819f5f0 con 0x7f6998103cd0 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.266+0000 7f6995ffb640 1 -- 172.21.15.78:0/801835029 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c003850 con 0x7f6998103cd0 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.266+0000 7f699f054640 1 -- 172.21.15.78:0/801835029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69981a5d10 con 0x7f6998103cd0 2024-09-11T01:04:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.268+0000 7f6995ffb640 1 -- 172.21.15.78:0/801835029 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f698c0039f0 con 0x7f6998103cd0 2024-09-11T01:04:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.268+0000 7f69a1429740 1 Processor -- start 2024-09-11T01:04:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.268+0000 7f69a1429740 1 -- start start 2024-09-11T01:04:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f69a1429740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df0a0 0x5641b28df480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f69a1429740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28df9c0 0x5641b28eb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f69a1429740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28eb950 0x5641b28edd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f69a1429740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5641b2855e50 con 0x5641b28df0a0 2024-09-11T01:04:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f69a1429740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5641b2845280 con 0x5641b28df9c0 2024-09-11T01:04:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f69a1429740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5641b2849090 con 0x5641b28eb950 2024-09-11T01:04:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f699d5f1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df0a0 0x5641b28df480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f699d5f1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df0a0 0x5641b28df480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39276/0 (socket says 172.21.15.78:39276) 2024-09-11T01:04:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f699d5f1640 1 -- 172.21.15.78:0/1502030575 learned_addr learned my addr 172.21.15.78:0/1502030575 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f699cdf0640 1 --2- 172.21.15.78:0/1502030575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28df9c0 0x5641b28eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f6997fff640 1 --2- 172.21.15.78:0/1502030575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28eb950 0x5641b28edd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f699d5f1640 1 -- 172.21.15.78:0/1502030575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28eb950 msgr2=0x5641b28edd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f699d5f1640 1 --2- 172.21.15.78:0/1502030575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28eb950 0x5641b28edd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f699d5f1640 1 -- 172.21.15.78:0/1502030575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28df9c0 msgr2=0x5641b28eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f699d5f1640 1 --2- 172.21.15.78:0/1502030575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28df9c0 0x5641b28eb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f699d5f1640 1 -- 172.21.15.78:0/1502030575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b25a3650 con 0x5641b28df0a0 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f6995ffb640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69880b5450 0x7f69880b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.269+0000 7f699cdf0640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69880b5450 0x7f69880b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.270+0000 7f699d5f1640 1 --2- 172.21.15.78:0/1502030575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df0a0 0x5641b28df480 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f699000e580 tx=0x7f69900095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e695d500535250a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.270+0000 7f6995ffb640 1 -- 172.21.15.78:0/801835029 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f698c105c20 con 0x7f6998103cd0 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.270+0000 7f6982ffd640 1 -- 172.21.15.78:0/1502030575 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990003970 con 0x5641b28df0a0 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.270+0000 7f6982ffd640 1 -- 172.21.15.78:0/1502030575 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f699000d830 con 0x5641b28df0a0 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.270+0000 7f6982ffd640 1 -- 172.21.15.78:0/1502030575 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69900043a0 con 0x5641b28df0a0 2024-09-11T01:04:13.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.270+0000 7f69a1429740 1 -- 172.21.15.78:0/1502030575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df0a0 msgr2=0x5641b28df480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.270+0000 7f69a1429740 1 --2- 172.21.15.78:0/1502030575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df0a0 0x5641b28df480 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f699000e580 tx=0x7f69900095c0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.270+0000 7f699cdf0640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69880b5450 0x7f69880b7910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5641b28dddb0 tx=0x7f698c1096d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 -- 172.21.15.78:0/1502030575 shutdown_connections 2024-09-11T01:04:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 --2- 172.21.15.78:0/1502030575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28eb950 0x5641b28edd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 --2- 172.21.15.78:0/1502030575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28df9c0 0x5641b28eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 --2- 172.21.15.78:0/1502030575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df0a0 0x5641b28df480 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 -- 172.21.15.78:0/1502030575 >> 172.21.15.78:0/1502030575 conn(0x5641b28dced0 msgr2=0x5641b28de070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 -- 172.21.15.78:0/1502030575 shutdown_connections 2024-09-11T01:04:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 -- 172.21.15.78:0/1502030575 wait complete. 2024-09-11T01:04:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 -- start start 2024-09-11T01:04:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 Processor -- start 2024-09-11T01:04:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28df0a0 0x5641b28eb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28e8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28eb950 0x5641b28e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5641b26a6700 con 0x5641b28df9c0 2024-09-11T01:04:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5641b27e0110 con 0x5641b28eb950 2024-09-11T01:04:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.271+0000 7f69a1429740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5641b28ee5d0 con 0x5641b28df0a0 2024-09-11T01:04:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f69a1429740 10 client.0 _ll_register_callbacks cb 0x5641b27f9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f6997fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28e8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f6997fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28e8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39278/0 (socket says 172.21.15.78:39278) 2024-09-11T01:04:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f6997fff640 1 -- 172.21.15.78:0/2624206517 learned_addr learned my addr 172.21.15.78:0/2624206517 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f699d5f1640 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28df0a0 0x5641b28eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f699d5f1640 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28eb950 0x5641b28e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f6997fff640 1 -- 172.21.15.78:0/2624206517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28df0a0 msgr2=0x5641b28eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f6997fff640 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28df0a0 0x5641b28eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f6997fff640 1 -- 172.21.15.78:0/2624206517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28eb950 msgr2=0x5641b28e8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f6997fff640 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28eb950 0x5641b28e8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f6997fff640 1 -- 172.21.15.78:0/2624206517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b280dc40 con 0x5641b28df9c0 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f699d5f1640 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28df0a0 0x5641b28eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f6997fff640 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28e8670 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6984012700 tx=0x7f6984012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5997e46e9fce3796 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f6977fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f6977fff640 1 -- 172.21.15.78:0/2624206517 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984004050 con 0x5641b28df9c0 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f6977fff640 1 -- 172.21.15.78:0/2624206517 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f69840041f0 con 0x5641b28df9c0 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.272+0000 7f69a1429740 1 -- 172.21.15.78:0/2624206517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f699000d830 con 0x5641b28df9c0 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.273+0000 7f6977fff640 1 -- 172.21.15.78:0/2624206517 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f69840043f0 con 0x5641b28df9c0 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.273+0000 7f69a1429740 10 client.16443 fetch_fsmap learned FSMap version 38 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.273+0000 7f69a1429740 1 -- 172.21.15.78:0/2624206517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5641b28f9f40 con 0x5641b28df9c0 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.273+0000 7f6977fff640 1 -- 172.21.15.78:0/2624206517 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f699000d830 con 0x5641b28df9c0 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.273+0000 7f6977fff640 1 -- 172.21.15.78:0/2624206517 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f6984024020 con 0x5641b28df9c0 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.273+0000 7f69a1429740 10 client.16443 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:04:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.273+0000 7f69a1429740 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.273+0000 7f69a1429740 20 client.16443 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.273+0000 7f69a1429740 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5641b28fb460 0x5641b28fd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.273+0000 7f69a1429740 4 client.16443 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.273+0000 7f69a1429740 1 -- 172.21.15.78:0/2624206517 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641b2857da0 con 0x5641b28fb460 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.273+0000 7f699d5f1640 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5641b28fb460 0x5641b28fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f6977fff640 10 client.16443 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f699d5f1640 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5641b28fb460 0x5641b28fd850 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=452e03a50b2bca41 server_cookie=fc13f11e122a2306 in_seq=0 out_seq=0 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f6977fff640 1 -- 172.21.15.78:0/2624206517 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5641b2857da0 con 0x5641b28fb460 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f6977fff640 10 client.16443 handle_command_reply: tid=0 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f69a1429740 1 client.16443 shutdown 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f69a1429740 1 -- 172.21.15.78:0/2624206517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 msgr2=0x5641b28e8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f69a1429740 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28e8670 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6984012700 tx=0x7f6984012bc0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f69a1429740 1 -- 172.21.15.78:0/2624206517 shutdown_connections 2024-09-11T01:04:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f69a1429740 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28eb950 0x5641b28e8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f69a1429740 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5641b28fb460 0x5641b28fd850 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f69a1429740 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28e8670 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f69a1429740 1 --2- 172.21.15.78:0/2624206517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28df0a0 0x5641b28eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f69a1429740 1 -- 172.21.15.78:0/2624206517 >> 172.21.15.78:0/2624206517 conn(0x5641b28dced0 msgr2=0x5641b28e2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f6977fff640 0 client.16443 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.274+0000 7f69a1429740 1 -- 172.21.15.78:0/2624206517 shutdown_connections 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.275+0000 7f69a1429740 1 -- 172.21.15.78:0/2624206517 wait complete. 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.275+0000 7f69a1429740 20 client.16443 trim_cache size 0 max 16384 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f69a1429740 1 Processor -- start 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f69a1429740 1 -- start start 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f69a1429740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28df0a0 0x5641b28f7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f69a1429740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28ecd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f69a1429740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28eb950 0x5641b28ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f69a1429740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69981ae560 con 0x5641b28df9c0 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f69a1429740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69981b0220 con 0x5641b28df0a0 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f69a1429740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69981aec30 con 0x5641b28eb950 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f6997fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28ecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f6997fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28ecd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56396/0 (socket says 172.21.15.78:56396) 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f6997fff640 1 -- 172.21.15.78:0/849471967 learned_addr learned my addr 172.21.15.78:0/849471967 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f699d5f1640 1 --2- 172.21.15.78:0/849471967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28eb950 0x5641b28ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f699d5f1640 1 --2- 172.21.15.78:0/849471967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28df0a0 0x5641b28f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f6997fff640 1 -- 172.21.15.78:0/849471967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28eb950 msgr2=0x5641b28ed280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f6997fff640 1 --2- 172.21.15.78:0/849471967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28eb950 0x5641b28ed280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f6997fff640 1 -- 172.21.15.78:0/849471967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28df0a0 msgr2=0x5641b28f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f6997fff640 1 --2- 172.21.15.78:0/849471967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28df0a0 0x5641b28f7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f6997fff640 1 -- 172.21.15.78:0/849471967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b28dea80 con 0x5641b28df9c0 2024-09-11T01:04:13.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.292+0000 7f699d5f1640 1 --2- 172.21.15.78:0/849471967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28eb950 0x5641b28ed280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.293+0000 7f6997fff640 1 --2- 172.21.15.78:0/849471967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28ecd40 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6984001eb0 tx=0x7f6984009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2176b8816cbbfde9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.293+0000 7f69827fc640 1 -- 172.21.15.78:0/849471967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984003670 con 0x5641b28df9c0 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.293+0000 7f69827fc640 1 -- 172.21.15.78:0/849471967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6984003810 con 0x5641b28df9c0 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.293+0000 7f69827fc640 1 -- 172.21.15.78:0/849471967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698400f9f0 con 0x5641b28df9c0 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 -- 172.21.15.78:0/849471967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 msgr2=0x5641b28ecd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 --2- 172.21.15.78:0/849471967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28ecd40 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6984001eb0 tx=0x7f6984009530 comp rx=0 tx=0).stop 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 -- 172.21.15.78:0/849471967 shutdown_connections 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 --2- 172.21.15.78:0/849471967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28eb950 0x5641b28ed280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 --2- 172.21.15.78:0/849471967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28ecd40 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 --2- 172.21.15.78:0/849471967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28df0a0 0x5641b28f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 -- 172.21.15.78:0/849471967 >> 172.21.15.78:0/849471967 conn(0x5641b28dced0 msgr2=0x5641b28dbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 -- 172.21.15.78:0/849471967 shutdown_connections 2024-09-11T01:04:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 -- 172.21.15.78:0/849471967 wait complete. 2024-09-11T01:04:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 -- start start 2024-09-11T01:04:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 Processor -- start 2024-09-11T01:04:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28df0a0 0x5641b28f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.294+0000 7f69a1429740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f69a1429740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28eb950 0x5641b28f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f69a1429740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69981afa90 con 0x5641b28df9c0 2024-09-11T01:04:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f69a1429740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69981af300 con 0x5641b28eb950 2024-09-11T01:04:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f69a1429740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69981ac560 con 0x5641b28df0a0 2024-09-11T01:04:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f69a1429740 10 client.0 _ll_register_callbacks cb 0x7f6984009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6997fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6997fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28f28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56412/0 (socket says 172.21.15.78:56412) 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6997fff640 1 -- 172.21.15.78:0/3885813664 learned_addr learned my addr 172.21.15.78:0/3885813664 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f699d5f1640 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28df0a0 0x5641b28f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f699d5f1640 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28eb950 0x5641b28f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6997fff640 1 -- 172.21.15.78:0/3885813664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28df0a0 msgr2=0x5641b28f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6997fff640 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28df0a0 0x5641b28f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6997fff640 1 -- 172.21.15.78:0/3885813664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28eb950 msgr2=0x5641b28f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6997fff640 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28eb950 0x5641b28f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6997fff640 1 -- 172.21.15.78:0/3885813664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b28f33a0 con 0x5641b28df9c0 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6997fff640 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28f28a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6984003cc0 tx=0x7f69840040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103e9caf3c4729e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6977fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6977fff640 1 -- 172.21.15.78:0/3885813664 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984034020 con 0x5641b28df9c0 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6977fff640 1 -- 172.21.15.78:0/3885813664 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6984005450 con 0x5641b28df9c0 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f69a1429740 1 -- 172.21.15.78:0/3885813664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6984003810 con 0x5641b28df9c0 2024-09-11T01:04:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f6977fff640 1 -- 172.21.15.78:0/3885813664 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f698403f050 con 0x5641b28df9c0 2024-09-11T01:04:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f69a1429740 10 client.16449 fetch_fsmap learned FSMap version 38 2024-09-11T01:04:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.295+0000 7f69a1429740 1 -- 172.21.15.78:0/3885813664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5641b29690c0 con 0x5641b28df9c0 2024-09-11T01:04:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.296+0000 7f6977fff640 1 -- 172.21.15.78:0/3885813664 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6984003810 con 0x5641b28df9c0 2024-09-11T01:04:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.296+0000 7f6977fff640 1 -- 172.21.15.78:0/3885813664 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f6984024040 con 0x5641b28df9c0 2024-09-11T01:04:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.296+0000 7f69a1429740 10 client.16449 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:04:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.296+0000 7f69a1429740 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:04:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.296+0000 7f69a1429740 20 client.16449 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.296+0000 7f69a1429740 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5641b2969810 0x5641b296bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.296+0000 7f69a1429740 4 client.16449 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.296+0000 7f69a1429740 1 -- 172.21.15.78:0/3885813664 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5641b2853850 con 0x5641b2969810 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.296+0000 7f699d5f1640 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5641b2969810 0x5641b296bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.296+0000 7f699d5f1640 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5641b2969810 0x5641b296bc30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7926a82c4d3d26e1 server_cookie=3220454800c89587 in_seq=0 out_seq=0 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.296+0000 7f6977fff640 10 client.16449 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f6977fff640 1 -- 172.21.15.78:0/3885813664 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x5641b2853850 con 0x5641b2969810 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f6977fff640 10 client.16449 handle_command_reply: tid=0 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f69a1429740 1 client.16449 shutdown 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f69a1429740 1 -- 172.21.15.78:0/3885813664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 msgr2=0x5641b28f28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f69a1429740 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28f28a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6984003cc0 tx=0x7f69840040c0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f69a1429740 1 -- 172.21.15.78:0/3885813664 shutdown_connections 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f69a1429740 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5641b28eb950 0x5641b28f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f69a1429740 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5641b2969810 0x5641b296bc30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f69a1429740 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5641b28df9c0 0x5641b28f28a0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f69a1429740 1 --2- 172.21.15.78:0/3885813664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b28df0a0 0x5641b28f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f6977fff640 0 client.16449 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:13.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f69a1429740 1 -- 172.21.15.78:0/3885813664 >> 172.21.15.78:0/3885813664 conn(0x5641b28dced0 msgr2=0x5641b28e9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:13.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f69a1429740 1 -- 172.21.15.78:0/3885813664 shutdown_connections 2024-09-11T01:04:13.322 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:04:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:04:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:04:13.327 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:04:13.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f69a1429740 1 -- 172.21.15.78:0/3885813664 wait complete. 2024-09-11T01:04:13.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.297+0000 7f69a1429740 20 client.16449 trim_cache size 0 max 16384 2024-09-11T01:04:13.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.298+0000 7f699f054640 1 -- 172.21.15.78:0/801835029 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69880b5450 msgr2=0x7f69880b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:13.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.298+0000 7f699f054640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69880b5450 0x7f69880b7910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5641b28dddb0 tx=0x7f698c1096d0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.298+0000 7f699f054640 1 -- 172.21.15.78:0/801835029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6998103cd0 msgr2=0x7f69981992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:13.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.298+0000 7f699f054640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6998103cd0 0x7f69981992b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f698c035040 tx=0x7f698c032d50 comp rx=0 tx=0).stop 2024-09-11T01:04:13.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.298+0000 7f699f054640 1 -- 172.21.15.78:0/801835029 shutdown_connections 2024-09-11T01:04:13.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.298+0000 7f699f054640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69880b5450 0x7f69880b7910 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.298+0000 7f699f054640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f699810d130 0x7f699819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.298+0000 7f699f054640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69981045f0 0x7f69981997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.298+0000 7f699f054640 1 --2- 172.21.15.78:0/801835029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6998103cd0 0x7f69981992b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:13.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.298+0000 7f699f054640 1 -- 172.21.15.78:0/801835029 >> 172.21.15.78:0/801835029 conn(0x7f69980fd770 msgr2=0x7f69980fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:13.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.298+0000 7f699f054640 1 -- 172.21.15.78:0/801835029 shutdown_connections 2024-09-11T01:04:13.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:13.298+0000 7f699f054640 1 -- 172.21.15.78:0/801835029 wait complete. 2024-09-11T01:04:13.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 415610 6.07 MB/sec execute 141 sec latency 24.311 ms 2024-09-11T01:04:14.152 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:04:14.152 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:04:14.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 416744 6.06 MB/sec execute 142 sec latency 345.243 ms 2024-09-11T01:04:14.455 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:04:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:14 smithi112 ceph-mon[35384]: pgmap v484: 97 pgs: 97 active+clean; 246 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-09-11T01:04:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:14 smithi078 ceph-mon[31315]: pgmap v484: 97 pgs: 97 active+clean; 246 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-09-11T01:04:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:14 smithi110 ceph-mon[34712]: pgmap v484: 97 pgs: 97 active+clean; 246 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-09-11T01:04:15.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.047+0000 7fc459c19640 1 -- 172.21.15.78:0/247371101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4541076a0 msgr2=0x7fc454107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:15.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.047+0000 7fc459c19640 1 --2- 172.21.15.78:0/247371101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4541076a0 0x7fc454107ae0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc448009900 tx=0x7fc44802a410 comp rx=0 tx=0).stop 2024-09-11T01:04:15.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.047+0000 7fc459c19640 1 -- 172.21.15.78:0/247371101 shutdown_connections 2024-09-11T01:04:15.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.047+0000 7fc459c19640 1 --2- 172.21.15.78:0/247371101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc454108020 0x7fc45410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:15.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.047+0000 7fc459c19640 1 --2- 172.21.15.78:0/247371101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4541076a0 0x7fc454107ae0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:15.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.047+0000 7fc459c19640 1 --2- 172.21.15.78:0/247371101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc454101c90 0x7fc454102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:15.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.047+0000 7fc459c19640 1 -- 172.21.15.78:0/247371101 >> 172.21.15.78:0/247371101 conn(0x7fc4540fd700 msgr2=0x7fc4540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:15.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.047+0000 7fc459c19640 1 -- 172.21.15.78:0/247371101 shutdown_connections 2024-09-11T01:04:15.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.047+0000 7fc459c19640 1 -- 172.21.15.78:0/247371101 wait complete. 2024-09-11T01:04:15.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc459c19640 1 Processor -- start 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc459c19640 1 -- start start 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc459c19640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc454101c90 0x7fc4541992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc459c19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4541076a0 0x7fc454199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc459c19640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc454108020 0x7fc45419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc459c19640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc454111c70 con 0x7fc454101c90 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc459c19640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc454111af0 con 0x7fc454108020 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc459c19640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc454111df0 con 0x7fc4541076a0 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc4537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc454101c90 0x7fc4541992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc4537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc454101c90 0x7fc4541992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56428/0 (socket says 172.21.15.78:56428) 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc4537fe640 1 -- 172.21.15.78:0/2065094296 learned_addr learned my addr 172.21.15.78:0/2065094296 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc452ffd640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4541076a0 0x7fc454199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.048+0000 7fc453fff640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc454108020 0x7fc45419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:15.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.049+0000 7fc4537fe640 1 -- 172.21.15.78:0/2065094296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4541076a0 msgr2=0x7fc454199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.049+0000 7fc4537fe640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4541076a0 0x7fc454199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.049+0000 7fc4537fe640 1 -- 172.21.15.78:0/2065094296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc454108020 msgr2=0x7fc45419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.049+0000 7fc4537fe640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc454108020 0x7fc45419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.049+0000 7fc4537fe640 1 -- 172.21.15.78:0/2065094296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45419f380 con 0x7fc454101c90 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.049+0000 7fc4537fe640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc454101c90 0x7fc4541992d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc44400e9f0 tx=0x7fc44400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.049+0000 7fc450ff9640 1 -- 172.21.15.78:0/2065094296 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44400cd80 con 0x7fc454101c90 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.049+0000 7fc459c19640 1 -- 172.21.15.78:0/2065094296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45419f670 con 0x7fc454101c90 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.049+0000 7fc450ff9640 1 -- 172.21.15.78:0/2065094296 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4440108e0 con 0x7fc454101c90 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.049+0000 7fc450ff9640 1 -- 172.21.15.78:0/2065094296 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc444018980 con 0x7fc454101c90 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.049+0000 7fc459c19640 1 -- 172.21.15.78:0/2065094296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4541a5f00 con 0x7fc454101c90 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.050+0000 7fc459c19640 1 -- 172.21.15.78:0/2065094296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc454102f40 con 0x7fc454101c90 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.052+0000 7fc450ff9640 1 -- 172.21.15.78:0/2065094296 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc444010430 con 0x7fc454101c90 2024-09-11T01:04:15.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.053+0000 7fc450ff9640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc43c0b5450 0x7fc43c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:15.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.053+0000 7fc452ffd640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc43c0b5450 0x7fc43c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:15.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.053+0000 7fc450ff9640 1 -- 172.21.15.78:0/2065094296 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc4440e6700 con 0x7fc454101c90 2024-09-11T01:04:15.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.054+0000 7fc452ffd640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc43c0b5450 0x7fc43c0b7910 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc448031440 tx=0x7fc448005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:15.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.057+0000 7fc450ff9640 1 -- 172.21.15.78:0/2065094296 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc4440acca0 con 0x7fc454101c90 2024-09-11T01:04:15.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.290+0000 7fc459c19640 1 -- 172.21.15.78:0/2065094296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc454102070 con 0x7fc454101c90 2024-09-11T01:04:15.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.291+0000 7fc450ff9640 1 -- 172.21.15.78:0/2065094296 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11993 (secure 0 0 0) 0x7fc454102070 con 0x7fc454101c90 2024-09-11T01:04:15.293 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:04:15.294 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":38,"btime":"2024-09-11T01:03:37:217249+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:03:37.105186+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:04:15.294 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 38 2024-09-11T01:04:15.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.293+0000 7fc459c19640 1 -- 172.21.15.78:0/2065094296 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc43c0b5450 msgr2=0x7fc43c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:15.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.293+0000 7fc459c19640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc43c0b5450 0x7fc43c0b7910 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc448031440 tx=0x7fc448005b50 comp rx=0 tx=0).stop 2024-09-11T01:04:15.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.293+0000 7fc459c19640 1 -- 172.21.15.78:0/2065094296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc454101c90 msgr2=0x7fc4541992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:15.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.293+0000 7fc459c19640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc454101c90 0x7fc4541992d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc44400e9f0 tx=0x7fc44400eeb0 comp rx=0 tx=0).stop 2024-09-11T01:04:15.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.294+0000 7fc459c19640 1 -- 172.21.15.78:0/2065094296 shutdown_connections 2024-09-11T01:04:15.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.294+0000 7fc459c19640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc43c0b5450 0x7fc43c0b7910 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:15.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.294+0000 7fc459c19640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc454108020 0x7fc45419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:15.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.294+0000 7fc459c19640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4541076a0 0x7fc454199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:15.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.294+0000 7fc459c19640 1 --2- 172.21.15.78:0/2065094296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc454101c90 0x7fc4541992d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:15.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.294+0000 7fc459c19640 1 -- 172.21.15.78:0/2065094296 >> 172.21.15.78:0/2065094296 conn(0x7fc4540fd700 msgr2=0x7fc454100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:15.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.294+0000 7fc459c19640 1 -- 172.21.15.78:0/2065094296 shutdown_connections 2024-09-11T01:04:15.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:15.294+0000 7fc459c19640 1 -- 172.21.15.78:0/2065094296 wait complete. 2024-09-11T01:04:15.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 418942 6.11 MB/sec execute 143 sec latency 41.468 ms 2024-09-11T01:04:15.617 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:15 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2065094296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:04:15.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:15 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2065094296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:04:15.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:15 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2065094296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:04:15.990 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:04:16.252 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:04:16.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 420275 6.11 MB/sec execute 144 sec latency 14.738 ms 2024-09-11T01:04:16.674 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:16 smithi078 ceph-mon[31315]: pgmap v485: 97 pgs: 97 active+clean; 247 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-09-11T01:04:16.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:16 smithi110 ceph-mon[34712]: pgmap v485: 97 pgs: 97 active+clean; 247 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-09-11T01:04:16.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:16 smithi112 ceph-mon[35384]: pgmap v485: 97 pgs: 97 active+clean; 247 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-09-11T01:04:16.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.752+0000 7f598c754640 1 -- 172.21.15.78:0/167834528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5984103ce0 msgr2=0x7f59841040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:16.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.752+0000 7f598c754640 1 --2- 172.21.15.78:0/167834528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5984103ce0 0x7f59841040c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5980009900 tx=0x7f598002a410 comp rx=0 tx=0).stop 2024-09-11T01:04:16.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 -- 172.21.15.78:0/167834528 shutdown_connections 2024-09-11T01:04:16.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 --2- 172.21.15.78:0/167834528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f598410d140 0x7f598410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 --2- 172.21.15.78:0/167834528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5984104600 0x7f598410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 --2- 172.21.15.78:0/167834528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5984103ce0 0x7f59841040c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 -- 172.21.15.78:0/167834528 >> 172.21.15.78:0/167834528 conn(0x7f59840fd700 msgr2=0x7f59840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 -- 172.21.15.78:0/167834528 shutdown_connections 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 -- 172.21.15.78:0/167834528 wait complete. 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 Processor -- start 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 -- start start 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5984103ce0 0x7f59841992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5984104600 0x7f5984199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f598410d140 0x7f598419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5984112db0 con 0x7f598410d140 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5984112c30 con 0x7f5984104600 2024-09-11T01:04:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598c754640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5984112f30 con 0x7f5984103ce0 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598acf1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f598410d140 0x7f598419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.753+0000 7f598acf1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f598410d140 0x7f598419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56442/0 (socket says 172.21.15.78:56442) 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f598acf1640 1 -- 172.21.15.78:0/3971311457 learned_addr learned my addr 172.21.15.78:0/3971311457 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f5989cef640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5984104600 0x7f5984199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f598a4f0640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5984103ce0 0x7f59841992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f598acf1640 1 -- 172.21.15.78:0/3971311457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5984103ce0 msgr2=0x7f59841992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f598acf1640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5984103ce0 0x7f59841992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f598acf1640 1 -- 172.21.15.78:0/3971311457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5984104600 msgr2=0x7f5984199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f598acf1640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5984104600 0x7f5984199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f598acf1640 1 -- 172.21.15.78:0/3971311457 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598419f370 con 0x7f598410d140 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f598acf1640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f598410d140 0x7f598419eba0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f597c00eb10 tx=0x7f597c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f597b7fe640 1 -- 172.21.15.78:0/3971311457 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c004b20 con 0x7f598410d140 2024-09-11T01:04:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f597b7fe640 1 -- 172.21.15.78:0/3971311457 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f597c004cc0 con 0x7f598410d140 2024-09-11T01:04:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f598c754640 1 -- 172.21.15.78:0/3971311457 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f598419f660 con 0x7f598410d140 2024-09-11T01:04:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f597b7fe640 1 -- 172.21.15.78:0/3971311457 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c010670 con 0x7f598410d140 2024-09-11T01:04:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.754+0000 7f598c754640 1 -- 172.21.15.78:0/3971311457 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59841a5d20 con 0x7f598410d140 2024-09-11T01:04:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f597b7fe640 1 -- 172.21.15.78:0/3971311457 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f597c005020 con 0x7f598410d140 2024-09-11T01:04:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f598eb29740 1 Processor -- start 2024-09-11T01:04:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f598eb29740 1 -- start start 2024-09-11T01:04:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f598eb29740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb9434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f598eb29740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb943a10 0x5649cb953870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f598eb29740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb953db0 0x5649cb9561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f598eb29740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649cb8b9ea0 con 0x5649cb9430f0 2024-09-11T01:04:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f598eb29740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5649cb8a92d0 con 0x5649cb953db0 2024-09-11T01:04:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f598eb29740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5649cb8ad0e0 con 0x5649cb943a10 2024-09-11T01:04:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f5989cef640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb9434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f5989cef640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb9434d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56458/0 (socket says 172.21.15.78:56458) 2024-09-11T01:04:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f5989cef640 1 -- 172.21.15.78:0/115716239 learned_addr learned my addr 172.21.15.78:0/115716239 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f5989cef640 1 --2- 172.21.15.78:0/115716239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb953db0 0x5649cb9561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f598a4f0640 1 --2- 172.21.15.78:0/115716239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb943a10 0x5649cb953870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f597b7fe640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f59700b5450 0x7f59700b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.757+0000 7f5989cef640 1 -- 172.21.15.78:0/115716239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb943a10 msgr2=0x5649cb953870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.758+0000 7f5989cef640 1 --2- 172.21.15.78:0/115716239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb943a10 0x5649cb953870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.758+0000 7f5989cef640 1 -- 172.21.15.78:0/115716239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb953db0 msgr2=0x5649cb9561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.758+0000 7f5989cef640 1 --2- 172.21.15.78:0/115716239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb953db0 0x5649cb9561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.758+0000 7f5989cef640 1 -- 172.21.15.78:0/115716239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649cb607650 con 0x5649cb9430f0 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.758+0000 7f598a4f0640 1 --2- 172.21.15.78:0/115716239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb943a10 0x5649cb953870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.758+0000 7f598acf1640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f59700b5450 0x7f59700b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.758+0000 7f5989cef640 1 --2- 172.21.15.78:0/115716239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb9434d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f59740137f0 tx=0x7f5974009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4576f537f66f4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.758+0000 7f597b7fe640 1 -- 172.21.15.78:0/3971311457 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f597c0e6560 con 0x7f598410d140 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.758+0000 7f5978ff9640 1 -- 172.21.15.78:0/115716239 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974013d70 con 0x5649cb9430f0 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.758+0000 7f5978ff9640 1 -- 172.21.15.78:0/115716239 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5974005020 con 0x5649cb9430f0 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.758+0000 7f5978ff9640 1 -- 172.21.15.78:0/115716239 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974004a00 con 0x5649cb9430f0 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 -- 172.21.15.78:0/115716239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 msgr2=0x5649cb9434d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598acf1640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f59700b5450 0x7f59700b7910 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f597c0a9cc0 tx=0x7f597c079ed0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 --2- 172.21.15.78:0/115716239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb9434d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f59740137f0 tx=0x7f5974009770 comp rx=0 tx=0).stop 2024-09-11T01:04:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 -- 172.21.15.78:0/115716239 shutdown_connections 2024-09-11T01:04:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 --2- 172.21.15.78:0/115716239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb953db0 0x5649cb9561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 --2- 172.21.15.78:0/115716239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb943a10 0x5649cb953870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 --2- 172.21.15.78:0/115716239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb9434d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 -- 172.21.15.78:0/115716239 >> 172.21.15.78:0/115716239 conn(0x5649cb940f20 msgr2=0x5649cb9420c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 -- 172.21.15.78:0/115716239 shutdown_connections 2024-09-11T01:04:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 -- 172.21.15.78:0/115716239 wait complete. 2024-09-11T01:04:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 -- start start 2024-09-11T01:04:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 Processor -- start 2024-09-11T01:04:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb9430f0 0x5649cb9536e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb943a10 0x5649cb950b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb953db0 0x5649cb9511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649cb70a700 con 0x5649cb943a10 2024-09-11T01:04:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5649cb844160 con 0x5649cb9430f0 2024-09-11T01:04:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.759+0000 7f598eb29740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5649cb956a00 con 0x5649cb953db0 2024-09-11T01:04:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598eb29740 10 client.0 _ll_register_callbacks cb 0x5649cb85d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598a4f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb943a10 0x5649cb950b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598a4f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb943a10 0x5649cb950b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56474/0 (socket says 172.21.15.78:56474) 2024-09-11T01:04:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598a4f0640 1 -- 172.21.15.78:0/2737538838 learned_addr learned my addr 172.21.15.78:0/2737538838 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f5989cef640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb9430f0 0x5649cb9536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f5989cef640 1 --2- 172.21.15.78:0/2737538838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb953db0 0x5649cb9511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598a4f0640 1 -- 172.21.15.78:0/2737538838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb953db0 msgr2=0x5649cb9511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598a4f0640 1 --2- 172.21.15.78:0/2737538838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb953db0 0x5649cb9511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598a4f0640 1 -- 172.21.15.78:0/2737538838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb9430f0 msgr2=0x5649cb9536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598a4f0640 1 --2- 172.21.15.78:0/2737538838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb9430f0 0x5649cb9536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598a4f0640 1 -- 172.21.15.78:0/2737538838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649cb871c90 con 0x5649cb943a10 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598a4f0640 1 --2- 172.21.15.78:0/2737538838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb943a10 0x5649cb950b30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f598002a930 tx=0x7f5980032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774300d042cab059 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f596dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f596dffb640 1 -- 172.21.15.78:0/2737538838 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980035040 con 0x5649cb943a10 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f596dffb640 1 -- 172.21.15.78:0/2737538838 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f59800316a0 con 0x5649cb943a10 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598eb29740 1 -- 172.21.15.78:0/2737538838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5974005020 con 0x5649cb943a10 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f596dffb640 1 -- 172.21.15.78:0/2737538838 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f59800318a0 con 0x5649cb943a10 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598eb29740 10 client.16467 fetch_fsmap learned FSMap version 38 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.760+0000 7f598eb29740 1 -- 172.21.15.78:0/2737538838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5649cb962370 con 0x5649cb943a10 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.761+0000 7f596dffb640 1 -- 172.21.15.78:0/2737538838 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5974005020 con 0x5649cb943a10 2024-09-11T01:04:16.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.761+0000 7f596dffb640 1 -- 172.21.15.78:0/2737538838 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f598004d050 con 0x5649cb943a10 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.761+0000 7f598eb29740 10 client.16467 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.761+0000 7f598eb29740 10 client.16467 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.761+0000 7f598eb29740 20 client.16467 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.761+0000 7f598eb29740 1 --2- 172.21.15.78:0/2737538838 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5649cb963780 0x5649cb965b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.761+0000 7f598eb29740 4 client.16467 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.761+0000 7f598eb29740 1 -- 172.21.15.78:0/2737538838 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649cb8bbdf0 con 0x5649cb963780 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.761+0000 7f5989cef640 1 --2- 172.21.15.78:0/2737538838 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5649cb963780 0x5649cb965b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.761+0000 7f596dffb640 10 client.16467 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.761+0000 7f5989cef640 1 --2- 172.21.15.78:0/2737538838 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5649cb963780 0x5649cb965b90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a636a99b049a0f7d server_cookie=49d3c152b3246bac in_seq=0 out_seq=0 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f596dffb640 1 -- 172.21.15.78:0/2737538838 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649cb8bbdf0 con 0x5649cb963780 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f596dffb640 10 client.16467 handle_command_reply: tid=0 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f598eb29740 1 client.16467 shutdown 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f598eb29740 1 -- 172.21.15.78:0/2737538838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb943a10 msgr2=0x5649cb950b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:16.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f598eb29740 1 --2- 172.21.15.78:0/2737538838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb943a10 0x5649cb950b30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f598002a930 tx=0x7f5980032d50 comp rx=0 tx=0).stop 2024-09-11T01:04:16.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f598eb29740 1 -- 172.21.15.78:0/2737538838 shutdown_connections 2024-09-11T01:04:16.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f598eb29740 1 --2- 172.21.15.78:0/2737538838 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5649cb963780 0x5649cb965b90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f598eb29740 1 --2- 172.21.15.78:0/2737538838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb953db0 0x5649cb9511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f598eb29740 1 --2- 172.21.15.78:0/2737538838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb943a10 0x5649cb950b30 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f598eb29740 1 --2- 172.21.15.78:0/2737538838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb9430f0 0x5649cb9536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f598eb29740 1 -- 172.21.15.78:0/2737538838 >> 172.21.15.78:0/2737538838 conn(0x5649cb940f20 msgr2=0x5649cb946a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:16.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f596dffb640 0 client.16467 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:04:16.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f598eb29740 1 -- 172.21.15.78:0/2737538838 shutdown_connections 2024-09-11T01:04:16.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f598eb29740 1 -- 172.21.15.78:0/2737538838 wait complete. 2024-09-11T01:04:16.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.762+0000 7f598eb29740 20 client.16467 trim_cache size 0 max 16384 2024-09-11T01:04:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f598eb29740 1 Processor -- start 2024-09-11T01:04:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f598eb29740 1 -- start start 2024-09-11T01:04:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f598eb29740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb95fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f598eb29740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb943a10 0x5649cb940760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f598eb29740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb953db0 0x5649cb94f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f598eb29740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59841ac3b0 con 0x5649cb9430f0 2024-09-11T01:04:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f598eb29740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f59841ae070 con 0x5649cb943a10 2024-09-11T01:04:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f598eb29740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59841aca80 con 0x5649cb953db0 2024-09-11T01:04:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f5989cef640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb95fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f5989cef640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb95fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56482/0 (socket says 172.21.15.78:56482) 2024-09-11T01:04:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f5989cef640 1 -- 172.21.15.78:0/3095792906 learned_addr learned my addr 172.21.15.78:0/3095792906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f598a4f0640 1 --2- 172.21.15.78:0/3095792906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb943a10 0x5649cb940760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f5989cef640 1 -- 172.21.15.78:0/3095792906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb953db0 msgr2=0x5649cb94f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f5989cef640 1 --2- 172.21.15.78:0/3095792906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb953db0 0x5649cb94f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f5989cef640 1 -- 172.21.15.78:0/3095792906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb943a10 msgr2=0x5649cb940760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f5989cef640 1 --2- 172.21.15.78:0/3095792906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb943a10 0x5649cb940760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f5989cef640 1 -- 172.21.15.78:0/3095792906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649cb966210 con 0x5649cb9430f0 2024-09-11T01:04:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.776+0000 7f5989cef640 1 --2- 172.21.15.78:0/3095792906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb95fc30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5974004650 tx=0x7f5974005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2944c5fa9f31178a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.777+0000 7f596ffff640 1 -- 172.21.15.78:0/3095792906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59740052f0 con 0x5649cb9430f0 2024-09-11T01:04:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.777+0000 7f596ffff640 1 -- 172.21.15.78:0/3095792906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5974005490 con 0x5649cb9430f0 2024-09-11T01:04:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.777+0000 7f596ffff640 1 -- 172.21.15.78:0/3095792906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974004ae0 con 0x5649cb9430f0 2024-09-11T01:04:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.777+0000 7f598eb29740 1 -- 172.21.15.78:0/3095792906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 msgr2=0x5649cb95fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.777+0000 7f598eb29740 1 --2- 172.21.15.78:0/3095792906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb95fc30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5974004650 tx=0x7f5974005f90 comp rx=0 tx=0).stop 2024-09-11T01:04:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.777+0000 7f598eb29740 1 -- 172.21.15.78:0/3095792906 shutdown_connections 2024-09-11T01:04:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.777+0000 7f598eb29740 1 --2- 172.21.15.78:0/3095792906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb953db0 0x5649cb94f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.777+0000 7f598eb29740 1 --2- 172.21.15.78:0/3095792906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb943a10 0x5649cb940760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.777+0000 7f598eb29740 1 --2- 172.21.15.78:0/3095792906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb95fc30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.777+0000 7f598eb29740 1 -- 172.21.15.78:0/3095792906 >> 172.21.15.78:0/3095792906 conn(0x5649cb940f20 msgr2=0x5649cb954190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.777+0000 7f598eb29740 1 -- 172.21.15.78:0/3095792906 shutdown_connections 2024-09-11T01:04:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f598eb29740 1 -- 172.21.15.78:0/3095792906 wait complete. 2024-09-11T01:04:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f598eb29740 1 -- start start 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f598eb29740 1 Processor -- start 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f598eb29740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb95be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f598eb29740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb943a10 0x5649cb95c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f598eb29740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb953db0 0x5649cb958c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f598eb29740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59841ad8e0 con 0x5649cb9430f0 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f598eb29740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f59841ad150 con 0x5649cb953db0 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f598eb29740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59841aa390 con 0x5649cb943a10 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f598eb29740 10 client.0 _ll_register_callbacks cb 0x7f5974005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f5989cef640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb95be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f5989cef640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb95be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56488/0 (socket says 172.21.15.78:56488) 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f5989cef640 1 -- 172.21.15.78:0/3979605846 learned_addr learned my addr 172.21.15.78:0/3979605846 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.778+0000 7f598a4f0640 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb943a10 0x5649cb95c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f5989cef640 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb953db0 0x5649cb958c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f5989cef640 1 -- 172.21.15.78:0/3979605846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb943a10 msgr2=0x5649cb95c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f5989cef640 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb943a10 0x5649cb95c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f5989cef640 1 -- 172.21.15.78:0/3979605846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb953db0 msgr2=0x5649cb958c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f5989cef640 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb953db0 0x5649cb958c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f5989cef640 1 -- 172.21.15.78:0/3979605846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649cb9593f0 con 0x5649cb9430f0 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f5989cef640 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb95be70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5974009620 tx=0x7f5974009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4118868461c04a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f596dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f596dffb640 1 -- 172.21.15.78:0/3979605846 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59740119c0 con 0x5649cb9430f0 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f598eb29740 1 -- 172.21.15.78:0/3979605846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5974005490 con 0x5649cb9430f0 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f596dffb640 1 -- 172.21.15.78:0/3979605846 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5974027040 con 0x5649cb9430f0 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f596dffb640 1 -- 172.21.15.78:0/3979605846 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5974011b60 con 0x5649cb9430f0 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f598eb29740 10 client.16473 fetch_fsmap learned FSMap version 38 2024-09-11T01:04:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f598eb29740 1 -- 172.21.15.78:0/3979605846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5649cb9d1130 con 0x5649cb9430f0 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.779+0000 7f596dffb640 1 -- 172.21.15.78:0/3979605846 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5974005490 con 0x5649cb9430f0 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f596dffb640 1 -- 172.21.15.78:0/3979605846 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f597402d040 con 0x5649cb9430f0 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f598eb29740 10 client.16473 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f598eb29740 10 client.16473 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f598eb29740 20 client.16473 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f598eb29740 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5649cb9d1590 0x5649cb9d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f598eb29740 4 client.16473 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f598eb29740 1 -- 172.21.15.78:0/3979605846 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649cb8b78a0 con 0x5649cb9d1590 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f598a4f0640 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5649cb9d1590 0x5649cb9d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f598a4f0640 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5649cb9d1590 0x5649cb9d39b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee1b5dfb5504a393 server_cookie=f593e808ac817ba4 in_seq=0 out_seq=0 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f596dffb640 10 client.16473 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f596dffb640 1 -- 172.21.15.78:0/3979605846 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649cb8b78a0 con 0x5649cb9d1590 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f596dffb640 10 client.16473 handle_command_reply: tid=0 2024-09-11T01:04:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f598eb29740 1 client.16473 shutdown 2024-09-11T01:04:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f598eb29740 1 -- 172.21.15.78:0/3979605846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 msgr2=0x5649cb95be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.780+0000 7f598eb29740 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb95be70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5974009620 tx=0x7f5974009590 comp rx=0 tx=0).stop 2024-09-11T01:04:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598eb29740 1 -- 172.21.15.78:0/3979605846 shutdown_connections 2024-09-11T01:04:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598eb29740 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5649cb9d1590 0x5649cb9d39b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598eb29740 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649cb953db0 0x5649cb958c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598eb29740 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649cb943a10 0x5649cb95c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598eb29740 1 --2- 172.21.15.78:0/3979605846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649cb9430f0 0x5649cb95be70 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598eb29740 1 -- 172.21.15.78:0/3979605846 >> 172.21.15.78:0/3979605846 conn(0x5649cb940f20 msgr2=0x5649cb94f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f596dffb640 0 client.16473 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:04:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598eb29740 1 -- 172.21.15.78:0/3979605846 shutdown_connections 2024-09-11T01:04:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598eb29740 1 -- 172.21.15.78:0/3979605846 wait complete. 2024-09-11T01:04:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598eb29740 20 client.16473 trim_cache size 0 max 16384 2024-09-11T01:04:16.796 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:04:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598c754640 1 -- 172.21.15.78:0/3971311457 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f59700b5450 msgr2=0x7f59700b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598c754640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f59700b5450 0x7f59700b7910 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f597c0a9cc0 tx=0x7f597c079ed0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598c754640 1 -- 172.21.15.78:0/3971311457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f598410d140 msgr2=0x7f598419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598c754640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f598410d140 0x7f598419eba0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f597c00eb10 tx=0x7f597c00c2d0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598c754640 1 -- 172.21.15.78:0/3971311457 shutdown_connections 2024-09-11T01:04:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598c754640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f59700b5450 0x7f59700b7910 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598c754640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f598410d140 0x7f598419eba0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598c754640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5984104600 0x7f5984199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598c754640 1 --2- 172.21.15.78:0/3971311457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5984103ce0 0x7f59841992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.781+0000 7f598c754640 1 -- 172.21.15.78:0/3971311457 >> 172.21.15.78:0/3971311457 conn(0x7f59840fd700 msgr2=0x7f59840fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.782+0000 7f598c754640 1 -- 172.21.15.78:0/3971311457 shutdown_connections 2024-09-11T01:04:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:16.782+0000 7f598c754640 1 -- 172.21.15.78:0/3971311457 wait complete. 2024-09-11T01:04:17.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 420587 6.07 MB/sec execute 145 sec latency 757.976 ms 2024-09-11T01:04:18.143 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:04:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:04:18.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 422561 6.12 MB/sec execute 146 sec latency 11.989 ms 2024-09-11T01:04:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:18 smithi112 ceph-mon[35384]: pgmap v486: 97 pgs: 97 active+clean; 251 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-09-11T01:04:18.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:18 smithi078 ceph-mon[31315]: pgmap v486: 97 pgs: 97 active+clean; 251 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-09-11T01:04:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:18 smithi110 ceph-mon[34712]: pgmap v486: 97 pgs: 97 active+clean; 251 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-09-11T01:04:18.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.906+0000 7fc83d144640 1 -- 172.21.15.78:0/2919214298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8381055a0 msgr2=0x7fc838105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:18.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.906+0000 7fc83d144640 1 --2- 172.21.15.78:0/2919214298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8381055a0 0x7fc838105a00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009900 tx=0x7fc82c02a410 comp rx=0 tx=0).stop 2024-09-11T01:04:18.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.906+0000 7fc83d144640 1 -- 172.21.15.78:0/2919214298 shutdown_connections 2024-09-11T01:04:18.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.906+0000 7fc83d144640 1 --2- 172.21.15.78:0/2919214298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc838105f40 0x7fc83810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.906+0000 7fc83d144640 1 --2- 172.21.15.78:0/2919214298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8381055a0 0x7fc838105a00 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.906+0000 7fc83d144640 1 --2- 172.21.15.78:0/2919214298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838101250 0x7fc838101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.906+0000 7fc83d144640 1 -- 172.21.15.78:0/2919214298 >> 172.21.15.78:0/2919214298 conn(0x7fc838078140 msgr2=0x7fc8380ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:18.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.906+0000 7fc83d144640 1 -- 172.21.15.78:0/2919214298 shutdown_connections 2024-09-11T01:04:18.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.906+0000 7fc83d144640 1 -- 172.21.15.78:0/2919214298 wait complete. 2024-09-11T01:04:18.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.907+0000 7fc83d144640 1 Processor -- start 2024-09-11T01:04:18.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.907+0000 7fc83d144640 1 -- start start 2024-09-11T01:04:18.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83d144640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838101250 0x7fc838199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83d144640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8381055a0 0x7fc838199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83d144640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc838105f40 0x7fc83819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83d144640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc838112d80 con 0x7fc8381055a0 2024-09-11T01:04:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83d144640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc838112c00 con 0x7fc838105f40 2024-09-11T01:04:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83d144640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc838112f00 con 0x7fc838101250 2024-09-11T01:04:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8381055a0 0x7fc838199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8381055a0 0x7fc838199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56502/0 (socket says 172.21.15.78:56502) 2024-09-11T01:04:18.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83659c640 1 -- 172.21.15.78:0/1327495431 learned_addr learned my addr 172.21.15.78:0/1327495431 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:18.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc836d9d640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838101250 0x7fc838199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83759e640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc838105f40 0x7fc83819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83659c640 1 -- 172.21.15.78:0/1327495431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838101250 msgr2=0x7fc838199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:18.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83659c640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838101250 0x7fc838199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83659c640 1 -- 172.21.15.78:0/1327495431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc838105f40 msgr2=0x7fc83819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:18.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83659c640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc838105f40 0x7fc83819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83659c640 1 -- 172.21.15.78:0/1327495431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83819f3c0 con 0x7fc8381055a0 2024-09-11T01:04:18.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc836d9d640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838101250 0x7fc838199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:18.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83759e640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc838105f40 0x7fc83819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:18.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83659c640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8381055a0 0x7fc838199850 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc82c005940 tx=0x7fc82c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:18.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc81ffff640 1 -- 172.21.15.78:0/1327495431 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c03f030 con 0x7fc8381055a0 2024-09-11T01:04:18.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc81ffff640 1 -- 172.21.15.78:0/1327495431 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82c0044c0 con 0x7fc8381055a0 2024-09-11T01:04:18.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.908+0000 7fc83d144640 1 -- 172.21.15.78:0/1327495431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83819f650 con 0x7fc8381055a0 2024-09-11T01:04:18.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.909+0000 7fc81ffff640 1 -- 172.21.15.78:0/1327495431 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c035070 con 0x7fc8381055a0 2024-09-11T01:04:18.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.909+0000 7fc83d144640 1 -- 172.21.15.78:0/1327495431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8381a5d70 con 0x7fc8381055a0 2024-09-11T01:04:18.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.911+0000 7fc81ffff640 1 -- 172.21.15.78:0/1327495431 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc82c03b9f0 con 0x7fc8381055a0 2024-09-11T01:04:18.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.911+0000 7fc83f519740 1 Processor -- start 2024-09-11T01:04:18.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.911+0000 7fc83f519740 1 -- start start 2024-09-11T01:04:18.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.911+0000 7fc83f519740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc83f519740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c6a10 0x5568b78d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc83f519740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78d6db0 0x5568b78d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc83f519740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5568b783cea0 con 0x5568b78c60f0 2024-09-11T01:04:18.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc83f519740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5568b782c2d0 con 0x5568b78c6a10 2024-09-11T01:04:18.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc83f519740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5568b78300e0 con 0x5568b78d6db0 2024-09-11T01:04:18.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc836d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c6a10 0x5568b78d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc83759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc83659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78d6db0 0x5568b78d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc836d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c6a10 0x5568b78d6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:35686/0 (socket says 172.21.15.78:35686) 2024-09-11T01:04:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc836d9d640 1 -- 172.21.15.78:0/1785036477 learned_addr learned my addr 172.21.15.78:0/1785036477 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc81ffff640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc8200b5450 0x7fc8200b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc836d9d640 1 -- 172.21.15.78:0/1785036477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78d6db0 msgr2=0x5568b78d91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc836d9d640 1 --2- 172.21.15.78:0/1785036477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78d6db0 0x5568b78d91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc836d9d640 1 -- 172.21.15.78:0/1785036477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 msgr2=0x5568b78c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc836d9d640 1 --2- 172.21.15.78:0/1785036477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc836d9d640 1 -- 172.21.15.78:0/1785036477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b758a650 con 0x5568b78c6a10 2024-09-11T01:04:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc83759e640 1 --2- 172.21.15.78:0/1785036477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc83759e640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc8200b5450 0x7fc8200b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc81ffff640 1 -- 172.21.15.78:0/1327495431 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc82c1065e0 con 0x7fc8381055a0 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc836d9d640 1 --2- 172.21.15.78:0/1785036477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c6a10 0x5568b78d6870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8280115f0 tx=0x7fc828011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44153ac38b365372 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.912+0000 7fc81d7fa640 1 -- 172.21.15.78:0/1785036477 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82801e070 con 0x5568b78c6a10 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.913+0000 7fc81d7fa640 1 -- 172.21.15.78:0/1785036477 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82800faf0 con 0x5568b78c6a10 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.913+0000 7fc81d7fa640 1 -- 172.21.15.78:0/1785036477 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828019b90 con 0x5568b78c6a10 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.913+0000 7fc83f519740 1 -- 172.21.15.78:0/1785036477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c6a10 msgr2=0x5568b78d6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.913+0000 7fc83759e640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc8200b5450 0x7fc8200b7910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc830005c40 tx=0x7fc830004a40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.913+0000 7fc83f519740 1 --2- 172.21.15.78:0/1785036477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c6a10 0x5568b78d6870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8280115f0 tx=0x7fc828011ab0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.913+0000 7fc83f519740 1 -- 172.21.15.78:0/1785036477 shutdown_connections 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.913+0000 7fc83f519740 1 --2- 172.21.15.78:0/1785036477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78d6db0 0x5568b78d91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.913+0000 7fc83f519740 1 --2- 172.21.15.78:0/1785036477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c6a10 0x5568b78d6870 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.913+0000 7fc83f519740 1 --2- 172.21.15.78:0/1785036477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.913+0000 7fc83f519740 1 -- 172.21.15.78:0/1785036477 >> 172.21.15.78:0/1785036477 conn(0x5568b78c3f20 msgr2=0x5568b78c50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.913+0000 7fc83f519740 1 -- 172.21.15.78:0/1785036477 shutdown_connections 2024-09-11T01:04:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.913+0000 7fc83f519740 1 -- 172.21.15.78:0/1785036477 wait complete. 2024-09-11T01:04:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83f519740 1 -- start start 2024-09-11T01:04:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83f519740 1 Processor -- start 2024-09-11T01:04:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83f519740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c60f0 0x5568b78d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83f519740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c6a10 0x5568b78d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83f519740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78d6db0 0x5568b78d4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83f519740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5568b768d700 con 0x5568b78c6a10 2024-09-11T01:04:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83f519740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5568b77c7160 con 0x5568b78c60f0 2024-09-11T01:04:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83f519740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5568b78d9a60 con 0x5568b78d6db0 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83f519740 10 client.0 _ll_register_callbacks cb 0x5568b77e09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c6a10 0x5568b78d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc836d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c60f0 0x5568b78d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c6a10 0x5568b78d3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56520/0 (socket says 172.21.15.78:56520) 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83659c640 1 -- 172.21.15.78:0/2756626306 learned_addr learned my addr 172.21.15.78:0/2756626306 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83759e640 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78d6db0 0x5568b78d4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83659c640 1 -- 172.21.15.78:0/2756626306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78d6db0 msgr2=0x5568b78d4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83659c640 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78d6db0 0x5568b78d4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83659c640 1 -- 172.21.15.78:0/2756626306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c60f0 msgr2=0x5568b78d6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83659c640 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c60f0 0x5568b78d6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83659c640 1 -- 172.21.15.78:0/2756626306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b77f4c90 con 0x5568b78c6a10 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc83759e640 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78d6db0 0x5568b78d4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:04:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.914+0000 7fc836d9d640 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c60f0 0x5568b78d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc83659c640 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c6a10 0x5568b78d3ae0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc82c109fd0 tx=0x7fc82c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906fd191f892a9e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc81aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc81aff5640 1 -- 172.21.15.78:0/2756626306 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c049070 con 0x5568b78c6a10 2024-09-11T01:04:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc83f519740 1 -- 172.21.15.78:0/2756626306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc82800faf0 con 0x5568b78c6a10 2024-09-11T01:04:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc81aff5640 1 -- 172.21.15.78:0/2756626306 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc82c0ccd80 con 0x5568b78c6a10 2024-09-11T01:04:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc81aff5640 1 -- 172.21.15.78:0/2756626306 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc82c0cc470 con 0x5568b78c6a10 2024-09-11T01:04:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc83f519740 10 client.16485 fetch_fsmap learned FSMap version 38 2024-09-11T01:04:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc83f519740 1 -- 172.21.15.78:0/2756626306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5568b78e5380 con 0x5568b78c6a10 2024-09-11T01:04:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc81aff5640 1 -- 172.21.15.78:0/2756626306 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82800faf0 con 0x5568b78c6a10 2024-09-11T01:04:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc81aff5640 1 -- 172.21.15.78:0/2756626306 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fc82c0d1170 con 0x5568b78c6a10 2024-09-11T01:04:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc83f519740 10 client.16485 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:04:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc83f519740 10 client.16485 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc83f519740 20 client.16485 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc83f519740 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5568b78e6790 0x5568b78e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc83f519740 4 client.16485 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.915+0000 7fc83f519740 1 -- 172.21.15.78:0/2756626306 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568b783edf0 con 0x5568b78e6790 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.916+0000 7fc836d9d640 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5568b78e6790 0x5568b78e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.916+0000 7fc836d9d640 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5568b78e6790 0x5568b78e8be0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8e2b637fcc77a060 server_cookie=d89c1f95014dcebf in_seq=0 out_seq=0 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.916+0000 7fc81aff5640 10 client.16485 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc81aff5640 1 -- 172.21.15.78:0/2756626306 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5568b783edf0 con 0x5568b78e6790 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc81aff5640 10 client.16485 handle_command_reply: tid=0 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc83f519740 1 client.16485 shutdown 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc83f519740 1 -- 172.21.15.78:0/2756626306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c6a10 msgr2=0x5568b78d3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc83f519740 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c6a10 0x5568b78d3ae0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc82c109fd0 tx=0x7fc82c009300 comp rx=0 tx=0).stop 2024-09-11T01:04:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc83f519740 1 -- 172.21.15.78:0/2756626306 shutdown_connections 2024-09-11T01:04:18.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc83f519740 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5568b78e6790 0x5568b78e8be0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc83f519740 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78d6db0 0x5568b78d4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc83f519740 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c6a10 0x5568b78d3ae0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc81aff5640 0 client.16485 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:04:18.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc83f519740 1 --2- 172.21.15.78:0/2756626306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78c60f0 0x5568b78d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc83f519740 1 -- 172.21.15.78:0/2756626306 >> 172.21.15.78:0/2756626306 conn(0x5568b78c3f20 msgr2=0x5568b78c9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:18.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc83f519740 1 -- 172.21.15.78:0/2756626306 shutdown_connections 2024-09-11T01:04:18.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc83f519740 1 -- 172.21.15.78:0/2756626306 wait complete. 2024-09-11T01:04:18.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.917+0000 7fc83f519740 20 client.16485 trim_cache size 0 max 16384 2024-09-11T01:04:18.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.930+0000 7fc83f519740 1 Processor -- start 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc83f519740 1 -- start start 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc83f519740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78e2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc83f519740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78c6a10 0x5568b78d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc83f519740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78d6db0 0x5568b78d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc83f519740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381ac380 con 0x5568b78c60f0 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc83f519740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381ae040 con 0x5568b78d6db0 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc83f519740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381aca50 con 0x5568b78c6a10 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc836d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78e2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc836d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78e2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56524/0 (socket says 172.21.15.78:56524) 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc836d9d640 1 -- 172.21.15.78:0/2204926912 learned_addr learned my addr 172.21.15.78:0/2204926912 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc83759e640 1 --2- 172.21.15.78:0/2204926912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78d6db0 0x5568b78d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc83659c640 1 --2- 172.21.15.78:0/2204926912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78c6a10 0x5568b78d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc836d9d640 1 -- 172.21.15.78:0/2204926912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78c6a10 msgr2=0x5568b78d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:18.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc836d9d640 1 --2- 172.21.15.78:0/2204926912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78c6a10 0x5568b78d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc836d9d640 1 -- 172.21.15.78:0/2204926912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78d6db0 msgr2=0x5568b78d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:18.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc836d9d640 1 --2- 172.21.15.78:0/2204926912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78d6db0 0x5568b78d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc836d9d640 1 -- 172.21.15.78:0/2204926912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b78cc250 con 0x5568b78c60f0 2024-09-11T01:04:18.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc83759e640 1 --2- 172.21.15.78:0/2204926912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78d6db0 0x5568b78d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:04:18.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc836d9d640 1 --2- 172.21.15.78:0/2204926912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78e2c90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc828009250 tx=0x7fc8280197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b1678c41464513 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:18.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc81cff9640 1 -- 172.21.15.78:0/2204926912 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82801e040 con 0x5568b78c60f0 2024-09-11T01:04:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc81cff9640 1 -- 172.21.15.78:0/2204926912 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8280050b0 con 0x5568b78c60f0 2024-09-11T01:04:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.931+0000 7fc81cff9640 1 -- 172.21.15.78:0/2204926912 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828019aa0 con 0x5568b78c60f0 2024-09-11T01:04:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.932+0000 7fc83f519740 1 -- 172.21.15.78:0/2204926912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 msgr2=0x5568b78e2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.932+0000 7fc83f519740 1 --2- 172.21.15.78:0/2204926912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78e2c90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc828009250 tx=0x7fc8280197d0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.932+0000 7fc83f519740 1 -- 172.21.15.78:0/2204926912 shutdown_connections 2024-09-11T01:04:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.932+0000 7fc83f519740 1 --2- 172.21.15.78:0/2204926912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78d6db0 0x5568b78d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.932+0000 7fc83f519740 1 --2- 172.21.15.78:0/2204926912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78c6a10 0x5568b78d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.932+0000 7fc83f519740 1 --2- 172.21.15.78:0/2204926912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78e2c90 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.932+0000 7fc83f519740 1 -- 172.21.15.78:0/2204926912 >> 172.21.15.78:0/2204926912 conn(0x5568b78c3f20 msgr2=0x5568b78d8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:18.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.932+0000 7fc83f519740 1 -- 172.21.15.78:0/2204926912 shutdown_connections 2024-09-11T01:04:18.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.932+0000 7fc83f519740 1 -- 172.21.15.78:0/2204926912 wait complete. 2024-09-11T01:04:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.932+0000 7fc83f519740 1 -- start start 2024-09-11T01:04:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc83f519740 1 Processor -- start 2024-09-11T01:04:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc83f519740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78ded00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc83f519740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78c6a10 0x5568b78df240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc83f519740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78d6db0 0x5568b78d9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc83f519740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381ad8b0 con 0x5568b78c60f0 2024-09-11T01:04:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc83f519740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381ad120 con 0x5568b78d6db0 2024-09-11T01:04:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc83f519740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381aa380 con 0x5568b78c6a10 2024-09-11T01:04:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc83f519740 10 client.0 _ll_register_callbacks cb 0x7fc82c03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc836d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78ded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc836d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78ded00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56534/0 (socket says 172.21.15.78:56534) 2024-09-11T01:04:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc836d9d640 1 -- 172.21.15.78:0/3154701634 learned_addr learned my addr 172.21.15.78:0/3154701634 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc83659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78c6a10 0x5568b78df240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc83759e640 1 --2- 172.21.15.78:0/3154701634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78d6db0 0x5568b78d9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc836d9d640 1 -- 172.21.15.78:0/3154701634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78c6a10 msgr2=0x5568b78df240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc836d9d640 1 --2- 172.21.15.78:0/3154701634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78c6a10 0x5568b78df240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc836d9d640 1 -- 172.21.15.78:0/3154701634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78d6db0 msgr2=0x5568b78d9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc836d9d640 1 --2- 172.21.15.78:0/3154701634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78d6db0 0x5568b78d9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc836d9d640 1 -- 172.21.15.78:0/3154701634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b78da1f0 con 0x5568b78c60f0 2024-09-11T01:04:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc836d9d640 1 --2- 172.21.15.78:0/3154701634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78ded00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc82801c040 tx=0x7fc82800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed39c82d2450763 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.933+0000 7fc81aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc81aff5640 1 -- 172.21.15.78:0/3154701634 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828031070 con 0x5568b78c60f0 2024-09-11T01:04:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc81aff5640 1 -- 172.21.15.78:0/3154701634 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc828005260 con 0x5568b78c60f0 2024-09-11T01:04:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc83f519740 1 -- 172.21.15.78:0/3154701634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8280050b0 con 0x5568b78c60f0 2024-09-11T01:04:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc81aff5640 1 -- 172.21.15.78:0/3154701634 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc82801e050 con 0x5568b78c60f0 2024-09-11T01:04:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc83f519740 10 client.16491 fetch_fsmap learned FSMap version 38 2024-09-11T01:04:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc83f519740 1 -- 172.21.15.78:0/3154701634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5568b7953d10 con 0x5568b78c60f0 2024-09-11T01:04:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc81aff5640 1 -- 172.21.15.78:0/3154701634 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8280050b0 con 0x5568b78c60f0 2024-09-11T01:04:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc81aff5640 1 -- 172.21.15.78:0/3154701634 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fc828019b10 con 0x5568b78c60f0 2024-09-11T01:04:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc83f519740 10 client.16491 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:04:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc83f519740 10 client.16491 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:04:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc83f519740 20 client.16491 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc83f519740 1 --2- 172.21.15.78:0/3154701634 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5568b7954260 0x5568b7956680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc83f519740 4 client.16491 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:04:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc83f519740 1 -- 172.21.15.78:0/3154701634 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568b783a8a0 con 0x5568b7954260 2024-09-11T01:04:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.934+0000 7fc83659c640 1 --2- 172.21.15.78:0/3154701634 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5568b7954260 0x5568b7956680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83659c640 1 --2- 172.21.15.78:0/3154701634 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5568b7954260 0x5568b7956680 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f2f63dbb3cc8960e server_cookie=fc542ced83565a1d in_seq=0 out_seq=0 2024-09-11T01:04:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc81aff5640 10 client.16491 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:04:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc81aff5640 1 -- 172.21.15.78:0/3154701634 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568b783a8a0 con 0x5568b7954260 2024-09-11T01:04:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc81aff5640 10 client.16491 handle_command_reply: tid=0 2024-09-11T01:04:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83f519740 1 client.16491 shutdown 2024-09-11T01:04:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83f519740 1 -- 172.21.15.78:0/3154701634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 msgr2=0x5568b78ded00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83f519740 1 --2- 172.21.15.78:0/3154701634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78ded00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc82801c040 tx=0x7fc82800f3c0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83f519740 1 -- 172.21.15.78:0/3154701634 shutdown_connections 2024-09-11T01:04:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83f519740 1 --2- 172.21.15.78:0/3154701634 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5568b7954260 0x5568b7956680 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83f519740 1 --2- 172.21.15.78:0/3154701634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568b78d6db0 0x5568b78d9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83f519740 1 --2- 172.21.15.78:0/3154701634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568b78c6a10 0x5568b78df240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc81aff5640 0 client.16491 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:04:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83f519740 1 --2- 172.21.15.78:0/3154701634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568b78c60f0 0x5568b78ded00 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83f519740 1 -- 172.21.15.78:0/3154701634 >> 172.21.15.78:0/3154701634 conn(0x5568b78c3f20 msgr2=0x5568b78c4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83f519740 1 -- 172.21.15.78:0/3154701634 shutdown_connections 2024-09-11T01:04:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83f519740 1 -- 172.21.15.78:0/3154701634 wait complete. 2024-09-11T01:04:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.935+0000 7fc83f519740 20 client.16491 trim_cache size 0 max 16384 2024-09-11T01:04:18.989 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:04:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.936+0000 7fc83d144640 1 -- 172.21.15.78:0/1327495431 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc8200b5450 msgr2=0x7fc8200b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.936+0000 7fc83d144640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc8200b5450 0x7fc8200b7910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc830005c40 tx=0x7fc830004a40 comp rx=0 tx=0).stop 2024-09-11T01:04:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.936+0000 7fc83d144640 1 -- 172.21.15.78:0/1327495431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8381055a0 msgr2=0x7fc838199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.936+0000 7fc83d144640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8381055a0 0x7fc838199850 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc82c005940 tx=0x7fc82c0040c0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.936+0000 7fc83d144640 1 -- 172.21.15.78:0/1327495431 shutdown_connections 2024-09-11T01:04:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.936+0000 7fc83d144640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc8200b5450 0x7fc8200b7910 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.936+0000 7fc83d144640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc838105f40 0x7fc83819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.936+0000 7fc83d144640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8381055a0 0x7fc838199850 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.936+0000 7fc83d144640 1 --2- 172.21.15.78:0/1327495431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838101250 0x7fc838199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.936+0000 7fc83d144640 1 -- 172.21.15.78:0/1327495431 >> 172.21.15.78:0/1327495431 conn(0x7fc838078140 msgr2=0x7fc8380ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.936+0000 7fc83d144640 1 -- 172.21.15.78:0/1327495431 shutdown_connections 2024-09-11T01:04:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:18.936+0000 7fc83d144640 1 -- 172.21.15.78:0/1327495431 wait complete. 2024-09-11T01:04:19.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 423824 6.11 MB/sec execute 147 sec latency 159.628 ms 2024-09-11T01:04:19.752 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:04:19.997 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:04:20.405 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:20 smithi078 ceph-mon[31315]: pgmap v487: 97 pgs: 97 active+clean; 250 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-11T01:04:20.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 423824 6.07 MB/sec execute 148 sec latency 1159.663 ms 2024-09-11T01:04:20.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.486+0000 7f85665e9640 1 -- 172.21.15.78:0/2991789218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8560108060 msgr2=0x7f856010e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:20.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.486+0000 7f85665e9640 1 --2- 172.21.15.78:0/2991789218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8560108060 0x7f856010e860 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8558009ca0 tx=0x7f855802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.486+0000 7f85665e9640 1 -- 172.21.15.78:0/2991789218 shutdown_connections 2024-09-11T01:04:20.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.486+0000 7f85665e9640 1 --2- 172.21.15.78:0/2991789218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8560108060 0x7f856010e860 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.486+0000 7f85665e9640 1 --2- 172.21.15.78:0/2991789218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85601076e0 0x7f8560107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.486+0000 7f85665e9640 1 --2- 172.21.15.78:0/2991789218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8560101cd0 0x7f85601020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.486+0000 7f85665e9640 1 -- 172.21.15.78:0/2991789218 >> 172.21.15.78:0/2991789218 conn(0x7f85600fd700 msgr2=0x7f85600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:20.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.486+0000 7f85665e9640 1 -- 172.21.15.78:0/2991789218 shutdown_connections 2024-09-11T01:04:20.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.486+0000 7f85665e9640 1 -- 172.21.15.78:0/2991789218 wait complete. 2024-09-11T01:04:20.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.486+0000 7f85665e9640 1 Processor -- start 2024-09-11T01:04:20.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f85665e9640 1 -- start start 2024-09-11T01:04:20.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f85665e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8560101cd0 0x7f85601980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f85665e9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85601076e0 0x7f8560198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f85665e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8560108060 0x7f856019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f85665e9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8560111ce0 con 0x7f8560108060 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f85665e9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8560111b60 con 0x7f85601076e0 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f85665e9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8560111e60 con 0x7f8560101cd0 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f8564b86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8560108060 0x7f856019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f8564b86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8560108060 0x7f856019d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56540/0 (socket says 172.21.15.78:56540) 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f8564b86640 1 -- 172.21.15.78:0/711324326 learned_addr learned my addr 172.21.15.78:0/711324326 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f855ffff640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8560101cd0 0x7f85601980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f855f7fe640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85601076e0 0x7f8560198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f8564b86640 1 -- 172.21.15.78:0/711324326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8560101cd0 msgr2=0x7f85601980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f8564b86640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8560101cd0 0x7f85601980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f8564b86640 1 -- 172.21.15.78:0/711324326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85601076e0 msgr2=0x7f8560198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f8564b86640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85601076e0 0x7f8560198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.487+0000 7f8564b86640 1 -- 172.21.15.78:0/711324326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856019e1a0 con 0x7f8560108060 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.488+0000 7f8564b86640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8560108060 0x7f856019d9d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f85580098d0 tx=0x7f85580317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.488+0000 7f855d7fa640 1 -- 172.21.15.78:0/711324326 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558003b70 con 0x7f8560108060 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.488+0000 7f855d7fa640 1 -- 172.21.15.78:0/711324326 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8558003d10 con 0x7f8560108060 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.488+0000 7f85665e9640 1 -- 172.21.15.78:0/711324326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856019e430 con 0x7f8560108060 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.488+0000 7f855d7fa640 1 -- 172.21.15.78:0/711324326 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855802f9d0 con 0x7f8560108060 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.488+0000 7f85665e9640 1 -- 172.21.15.78:0/711324326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85601a4b50 con 0x7f8560108060 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.490+0000 7f85689be740 1 Processor -- start 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.490+0000 7f855d7fa640 1 -- 172.21.15.78:0/711324326 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f855802fb70 con 0x7f8560108060 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.490+0000 7f85689be740 1 -- start start 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f85689be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 0x5642cc3d68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f85689be740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3e4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f85689be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3e7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f85689be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5642cc34cea0 con 0x5642cc3e4f90 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f85689be740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5642cc33c2d0 con 0x5642cc3d6df0 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f85689be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5642cc3400e0 con 0x5642cc3d64d0 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f855f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f855f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3e7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56554/0 (socket says 172.21.15.78:56554) 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f855f7fe640 1 -- 172.21.15.78:0/1300462984 learned_addr learned my addr 172.21.15.78:0/1300462984 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f855ffff640 1 --2- 172.21.15.78:0/1300462984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3e4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f855f7fe640 1 --2- 172.21.15.78:0/1300462984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 0x5642cc3d68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f855d7fa640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f85480b5450 0x7f85480b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f855f7fe640 1 -- 172.21.15.78:0/1300462984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 msgr2=0x5642cc3d68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f855f7fe640 1 --2- 172.21.15.78:0/1300462984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 0x5642cc3d68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f855f7fe640 1 -- 172.21.15.78:0/1300462984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 msgr2=0x5642cc3e4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f855f7fe640 1 --2- 172.21.15.78:0/1300462984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3e4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f855f7fe640 1 -- 172.21.15.78:0/1300462984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cc09a650 con 0x5642cc3e4f90 2024-09-11T01:04:20.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f8564b86640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f85480b5450 0x7f85480b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.491+0000 7f855f7fe640 1 --2- 172.21.15.78:0/1300462984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3e7380 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8554014880 tx=0x7f8554009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8838220099a26cea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:20.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f855d7fa640 1 -- 172.21.15.78:0/711324326 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f8558107290 con 0x7f8560108060 2024-09-11T01:04:20.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f85467fc640 1 -- 172.21.15.78:0/1300462984 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85540140d0 con 0x5642cc3e4f90 2024-09-11T01:04:20.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f85467fc640 1 -- 172.21.15.78:0/1300462984 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8554014270 con 0x5642cc3e4f90 2024-09-11T01:04:20.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f85467fc640 1 -- 172.21.15.78:0/1300462984 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85540047a0 con 0x5642cc3e4f90 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f85689be740 1 -- 172.21.15.78:0/1300462984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 msgr2=0x5642cc3e7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f8564b86640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f85480b5450 0x7f85480b7910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f85580ca990 tx=0x7f855802a8c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f85689be740 1 --2- 172.21.15.78:0/1300462984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3e7380 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8554014880 tx=0x7f8554009630 comp rx=0 tx=0).stop 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f85689be740 1 -- 172.21.15.78:0/1300462984 shutdown_connections 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f85689be740 1 --2- 172.21.15.78:0/1300462984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3e7380 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f85689be740 1 --2- 172.21.15.78:0/1300462984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3e4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f85689be740 1 --2- 172.21.15.78:0/1300462984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 0x5642cc3d68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f85689be740 1 -- 172.21.15.78:0/1300462984 >> 172.21.15.78:0/1300462984 conn(0x5642cc3d4300 msgr2=0x5642cc3d54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f85689be740 1 -- 172.21.15.78:0/1300462984 shutdown_connections 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.492+0000 7f85689be740 1 -- 172.21.15.78:0/1300462984 wait complete. 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.493+0000 7f85689be740 1 -- start start 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.493+0000 7f85689be740 1 Processor -- start 2024-09-11T01:04:20.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.493+0000 7f85689be740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d64d0 0x5642cc3e4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.493+0000 7f85689be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3d6df0 0x5642cc3e1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.493+0000 7f85689be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3e4f90 0x5642cc3e22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.493+0000 7f85689be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5642cc19d700 con 0x5642cc3d6df0 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.493+0000 7f85689be740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5642cc2d7160 con 0x5642cc3d64d0 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.493+0000 7f85689be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5642cc3e7be0 con 0x5642cc3e4f90 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.493+0000 7f85689be740 10 client.0 _ll_register_callbacks cb 0x5642cc2f09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.493+0000 7f855ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3d6df0 0x5642cc3e1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.493+0000 7f855ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3d6df0 0x5642cc3e1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56566/0 (socket says 172.21.15.78:56566) 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.493+0000 7f855ffff640 1 -- 172.21.15.78:0/536887512 learned_addr learned my addr 172.21.15.78:0/536887512 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f855f7fe640 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d64d0 0x5642cc3e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f855f7fe640 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3e4f90 0x5642cc3e22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f855ffff640 1 -- 172.21.15.78:0/536887512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3e4f90 msgr2=0x5642cc3e22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f855ffff640 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3e4f90 0x5642cc3e22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f855ffff640 1 -- 172.21.15.78:0/536887512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d64d0 msgr2=0x5642cc3e4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:20.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f855ffff640 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d64d0 0x5642cc3e4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f855ffff640 1 -- 172.21.15.78:0/536887512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cc304c90 con 0x5642cc3d6df0 2024-09-11T01:04:20.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f855f7fe640 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3e4f90 0x5642cc3e22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:04:20.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f855ffff640 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3d6df0 0x5642cc3e1ce0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8550010a50 tx=0x7f8550010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737b0554554b86a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:20.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f853f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:20.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f853f7fe640 1 -- 172.21.15.78:0/536887512 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855001e060 con 0x5642cc3d6df0 2024-09-11T01:04:20.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f853f7fe640 1 -- 172.21.15.78:0/536887512 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f855000fc30 con 0x5642cc3d6df0 2024-09-11T01:04:20.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f85689be740 1 -- 172.21.15.78:0/536887512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8554014270 con 0x5642cc3d6df0 2024-09-11T01:04:20.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f853f7fe640 1 -- 172.21.15.78:0/536887512 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f855000fe30 con 0x5642cc3d6df0 2024-09-11T01:04:20.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f85689be740 10 client.16503 fetch_fsmap learned FSMap version 38 2024-09-11T01:04:20.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f85689be740 1 -- 172.21.15.78:0/536887512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5642cc3f3560 con 0x5642cc3d6df0 2024-09-11T01:04:20.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.494+0000 7f853f7fe640 1 -- 172.21.15.78:0/536887512 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8554014270 con 0x5642cc3d6df0 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.495+0000 7f853f7fe640 1 -- 172.21.15.78:0/536887512 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f8550021020 con 0x5642cc3d6df0 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.495+0000 7f85689be740 10 client.16503 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.495+0000 7f85689be740 10 client.16503 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.495+0000 7f85689be740 20 client.16503 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.495+0000 7f85689be740 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5642cc3f4a20 0x5642cc3f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.495+0000 7f85689be740 4 client.16503 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.495+0000 7f85689be740 1 -- 172.21.15.78:0/536887512 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642cc34edf0 con 0x5642cc3f4a20 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.495+0000 7f855f7fe640 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5642cc3f4a20 0x5642cc3f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.495+0000 7f853f7fe640 10 client.16503 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.495+0000 7f855f7fe640 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5642cc3f4a20 0x5642cc3f6e50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82f159de5ac2e607 server_cookie=66b1d6b83049eac5 in_seq=0 out_seq=0 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f853f7fe640 1 -- 172.21.15.78:0/536887512 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5642cc34edf0 con 0x5642cc3f4a20 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f853f7fe640 10 client.16503 handle_command_reply: tid=0 2024-09-11T01:04:20.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f85689be740 1 client.16503 shutdown 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f85689be740 1 -- 172.21.15.78:0/536887512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3d6df0 msgr2=0x5642cc3e1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f85689be740 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3d6df0 0x5642cc3e1ce0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8550010a50 tx=0x7f8550010f10 comp rx=0 tx=0).stop 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f85689be740 1 -- 172.21.15.78:0/536887512 shutdown_connections 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f85689be740 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3e4f90 0x5642cc3e22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f85689be740 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3d6df0 0x5642cc3e1ce0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f85689be740 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5642cc3f4a20 0x5642cc3f6e50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f85689be740 1 --2- 172.21.15.78:0/536887512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d64d0 0x5642cc3e4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f853f7fe640 0 client.16503 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f85689be740 1 -- 172.21.15.78:0/536887512 >> 172.21.15.78:0/536887512 conn(0x5642cc3d4300 msgr2=0x5642cc3d9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f85689be740 1 -- 172.21.15.78:0/536887512 shutdown_connections 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f85689be740 1 -- 172.21.15.78:0/536887512 wait complete. 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.496+0000 7f85689be740 20 client.16503 trim_cache size 0 max 16384 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.509+0000 7f85689be740 1 Processor -- start 2024-09-11T01:04:20.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.509+0000 7f85689be740 1 -- start start 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.509+0000 7f85689be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 0x5642cc3f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.509+0000 7f85689be740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3df7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.509+0000 7f85689be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.509+0000 7f85689be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85601ac2c0 con 0x5642cc3e4f90 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.509+0000 7f85689be740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85601adf80 con 0x5642cc3d6df0 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.509+0000 7f85689be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85601ac990 con 0x5642cc3d64d0 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f855f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f855f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3ed280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56572/0 (socket says 172.21.15.78:56572) 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f855f7fe640 1 -- 172.21.15.78:0/3718678144 learned_addr learned my addr 172.21.15.78:0/3718678144 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f855ffff640 1 --2- 172.21.15.78:0/3718678144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3df7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f855f7fe640 1 -- 172.21.15.78:0/3718678144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 msgr2=0x5642cc3f0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f855f7fe640 1 --2- 172.21.15.78:0/3718678144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 0x5642cc3f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f855f7fe640 1 -- 172.21.15.78:0/3718678144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 msgr2=0x5642cc3df7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:20.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f855f7fe640 1 --2- 172.21.15.78:0/3718678144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3df7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f855f7fe640 1 -- 172.21.15.78:0/3718678144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cc3d51a0 con 0x5642cc3e4f90 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f855ffff640 1 --2- 172.21.15.78:0/3718678144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3df7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f855f7fe640 1 --2- 172.21.15.78:0/3718678144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3ed280 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8554009c60 tx=0x7f8554014420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83c119dde647060 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f8545ffb640 1 -- 172.21.15.78:0/3718678144 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855402f020 con 0x5642cc3e4f90 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f8545ffb640 1 -- 172.21.15.78:0/3718678144 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8554009530 con 0x5642cc3e4f90 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.510+0000 7f8545ffb640 1 -- 172.21.15.78:0/3718678144 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855400fbc0 con 0x5642cc3e4f90 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.511+0000 7f85689be740 1 -- 172.21.15.78:0/3718678144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 msgr2=0x5642cc3ed280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.511+0000 7f85689be740 1 --2- 172.21.15.78:0/3718678144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3ed280 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8554009c60 tx=0x7f8554014420 comp rx=0 tx=0).stop 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.511+0000 7f85689be740 1 -- 172.21.15.78:0/3718678144 shutdown_connections 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.511+0000 7f85689be740 1 --2- 172.21.15.78:0/3718678144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3ed280 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.511+0000 7f85689be740 1 --2- 172.21.15.78:0/3718678144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3df7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.511+0000 7f85689be740 1 --2- 172.21.15.78:0/3718678144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 0x5642cc3f0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.511+0000 7f85689be740 1 -- 172.21.15.78:0/3718678144 >> 172.21.15.78:0/3718678144 conn(0x5642cc3d4300 msgr2=0x5642cc3e6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:20.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.511+0000 7f85689be740 1 -- 172.21.15.78:0/3718678144 shutdown_connections 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.511+0000 7f85689be740 1 -- 172.21.15.78:0/3718678144 wait complete. 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.511+0000 7f85689be740 1 -- start start 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.511+0000 7f85689be740 1 Processor -- start 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f85689be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 0x5642cc3fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f85689be740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3fb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f85689be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f85689be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85601ad7f0 con 0x5642cc3e4f90 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f85689be740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85601ad060 con 0x5642cc3d6df0 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f85689be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85601aa2c0 con 0x5642cc3d64d0 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f85689be740 10 client.0 _ll_register_callbacks cb 0x7f855000a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f855ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3fb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f855ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3fb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:35800/0 (socket says 172.21.15.78:35800) 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f855ffff640 1 -- 172.21.15.78:0/1198883557 learned_addr learned my addr 172.21.15.78:0/1198883557 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:20.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f855ffff640 1 -- 172.21.15.78:0/1198883557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 msgr2=0x5642cc3fd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:20.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f855f7fe640 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f855ffff640 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 0x5642cc3fd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f855ffff640 1 -- 172.21.15.78:0/1198883557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 msgr2=0x5642cc3fbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:20.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f855ffff640 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3fbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.512+0000 7f855ffff640 1 -- 172.21.15.78:0/1198883557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cc463950 con 0x5642cc3d6df0 2024-09-11T01:04:20.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.513+0000 7f855f7fe640 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:04:20.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.513+0000 7f855ffff640 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3fb710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8550001eb0 tx=0x7f855000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7677a600f88a410d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:20.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.513+0000 7f853f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:04:20.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.513+0000 7f853f7fe640 1 -- 172.21.15.78:0/1198883557 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855000a340 con 0x5642cc3d6df0 2024-09-11T01:04:20.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.513+0000 7f853f7fe640 1 -- 172.21.15.78:0/1198883557 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f855000fc30 con 0x5642cc3d6df0 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.513+0000 7f85689be740 1 -- 172.21.15.78:0/1198883557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8554009530 con 0x5642cc3d6df0 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.513+0000 7f853f7fe640 1 -- 172.21.15.78:0/1198883557 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f855001e020 con 0x5642cc3d6df0 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.513+0000 7f85689be740 10 client.25838 fetch_fsmap learned FSMap version 38 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.513+0000 7f85689be740 1 -- 172.21.15.78:0/1198883557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5642cc464bb0 con 0x5642cc3d6df0 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.513+0000 7f853f7fe640 1 -- 172.21.15.78:0/1198883557 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8554009530 con 0x5642cc3d6df0 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f853f7fe640 1 -- 172.21.15.78:0/1198883557 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f85500049d0 con 0x5642cc3d6df0 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f85689be740 10 client.25838 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f85689be740 10 client.25838 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f85689be740 20 client.25838 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f85689be740 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5642cc465380 0x5642cc4677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f85689be740 4 client.25838 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f85689be740 1 -- 172.21.15.78:0/1198883557 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642cc34a8a0 con 0x5642cc465380 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f855f7fe640 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5642cc465380 0x5642cc4677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:20.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f853f7fe640 10 client.25838 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f855f7fe640 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5642cc465380 0x5642cc4677a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21c3c3e16fd367f8 server_cookie=c5ee900acaae5e0d in_seq=0 out_seq=0 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f853f7fe640 1 -- 172.21.15.78:0/1198883557 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642cc34a8a0 con 0x5642cc465380 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f853f7fe640 10 client.25838 handle_command_reply: tid=0 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.514+0000 7f85689be740 1 client.25838 shutdown 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85689be740 1 -- 172.21.15.78:0/1198883557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 msgr2=0x5642cc3fb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85689be740 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3fb710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8550001eb0 tx=0x7f855000ed60 comp rx=0 tx=0).stop 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85689be740 1 -- 172.21.15.78:0/1198883557 shutdown_connections 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85689be740 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cc3e4f90 0x5642cc3fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85689be740 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5642cc465380 0x5642cc4677a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85689be740 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642cc3d6df0 0x5642cc3fb710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85689be740 1 --2- 172.21.15.78:0/1198883557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642cc3d64d0 0x5642cc3fd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f853f7fe640 0 client.25838 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85689be740 1 -- 172.21.15.78:0/1198883557 >> 172.21.15.78:0/1198883557 conn(0x5642cc3d4300 msgr2=0x5642cc3e6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:20.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85689be740 1 -- 172.21.15.78:0/1198883557 shutdown_connections 2024-09-11T01:04:20.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85689be740 1 -- 172.21.15.78:0/1198883557 wait complete. 2024-09-11T01:04:20.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85689be740 20 client.25838 trim_cache size 0 max 16384 2024-09-11T01:04:20.536 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:04:20.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85665e9640 1 -- 172.21.15.78:0/711324326 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f85480b5450 msgr2=0x7f85480b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:20.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85665e9640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f85480b5450 0x7f85480b7910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f85580ca990 tx=0x7f855802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85665e9640 1 -- 172.21.15.78:0/711324326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8560108060 msgr2=0x7f856019d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:20.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85665e9640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8560108060 0x7f856019d9d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f85580098d0 tx=0x7f85580317b0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85665e9640 1 -- 172.21.15.78:0/711324326 shutdown_connections 2024-09-11T01:04:20.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85665e9640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f85480b5450 0x7f85480b7910 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85665e9640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8560108060 0x7f856019d9d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85665e9640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85601076e0 0x7f8560198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85665e9640 1 --2- 172.21.15.78:0/711324326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8560101cd0 0x7f85601980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:20.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85665e9640 1 -- 172.21.15.78:0/711324326 >> 172.21.15.78:0/711324326 conn(0x7f85600fd700 msgr2=0x7f85600ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:20.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85665e9640 1 -- 172.21.15.78:0/711324326 shutdown_connections 2024-09-11T01:04:20.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:20.515+0000 7f85665e9640 1 -- 172.21.15.78:0/711324326 wait complete. 2024-09-11T01:04:20.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:20 smithi110 ceph-mon[34712]: pgmap v487: 97 pgs: 97 active+clean; 250 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-11T01:04:20.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:20 smithi112 ceph-mon[35384]: pgmap v487: 97 pgs: 97 active+clean; 250 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-11T01:04:21.255 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:04:21.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 423824 6.03 MB/sec execute 149 sec latency 2159.739 ms 2024-09-11T01:04:21.500 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:04:22.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.185+0000 7f3dec10d640 1 -- 172.21.15.78:0/1531419652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3de4103b30 msgr2=0x7f3de4103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:22.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.185+0000 7f3dec10d640 1 --2- 172.21.15.78:0/1531419652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3de4103b30 0x7f3de4103f10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de002a410 comp rx=0 tx=0).stop 2024-09-11T01:04:22.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.185+0000 7f3dec10d640 1 -- 172.21.15.78:0/1531419652 shutdown_connections 2024-09-11T01:04:22.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.185+0000 7f3dec10d640 1 --2- 172.21.15.78:0/1531419652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de410d0e0 0x7f3de410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.185+0000 7f3dec10d640 1 --2- 172.21.15.78:0/1531419652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de4104530 0x7f3de410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.185+0000 7f3dec10d640 1 --2- 172.21.15.78:0/1531419652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3de4103b30 0x7f3de4103f10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.185+0000 7f3dec10d640 1 -- 172.21.15.78:0/1531419652 >> 172.21.15.78:0/1531419652 conn(0x7f3de40fd700 msgr2=0x7f3de40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:22.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.185+0000 7f3dec10d640 1 -- 172.21.15.78:0/1531419652 shutdown_connections 2024-09-11T01:04:22.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.185+0000 7f3dec10d640 1 -- 172.21.15.78:0/1531419652 wait complete. 2024-09-11T01:04:22.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.185+0000 7f3dec10d640 1 Processor -- start 2024-09-11T01:04:22.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3dec10d640 1 -- start start 2024-09-11T01:04:22.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3dec10d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3de4103b30 0x7f3de4199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3dec10d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4104530 0x7f3de41997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3dec10d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de410d0e0 0x7f3de419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3dec10d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4112d60 con 0x7f3de4103b30 2024-09-11T01:04:22.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3dec10d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4112be0 con 0x7f3de410d0e0 2024-09-11T01:04:22.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3dec10d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4112ee0 con 0x7f3de4104530 2024-09-11T01:04:22.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3de9ea9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3de4103b30 0x7f3de4199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3de9ea9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3de4103b30 0x7f3de4199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56592/0 (socket says 172.21.15.78:56592) 2024-09-11T01:04:22.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3de9ea9640 1 -- 172.21.15.78:0/4267207193 learned_addr learned my addr 172.21.15.78:0/4267207193 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:22.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de410d0e0 0x7f3de419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3de96a8640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4104530 0x7f3de41997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3de9ea9640 1 -- 172.21.15.78:0/4267207193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4104530 msgr2=0x7f3de41997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:22.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3de9ea9640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4104530 0x7f3de41997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3de9ea9640 1 -- 172.21.15.78:0/4267207193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de410d0e0 msgr2=0x7f3de419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:22.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3de9ea9640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de410d0e0 0x7f3de419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3de9ea9640 1 -- 172.21.15.78:0/4267207193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de419f340 con 0x7f3de4103b30 2024-09-11T01:04:22.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de410d0e0 0x7f3de419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:22.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3de96a8640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4104530 0x7f3de41997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:04:22.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.186+0000 7f3de9ea9640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3de4103b30 0x7f3de4199290 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005940 tx=0x7f3de00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:22.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.187+0000 7f3ddaffd640 1 -- 172.21.15.78:0/4267207193 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de003f030 con 0x7f3de4103b30 2024-09-11T01:04:22.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.187+0000 7f3ddaffd640 1 -- 172.21.15.78:0/4267207193 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de00044c0 con 0x7f3de4103b30 2024-09-11T01:04:22.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.187+0000 7f3ddaffd640 1 -- 172.21.15.78:0/4267207193 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0035070 con 0x7f3de4103b30 2024-09-11T01:04:22.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.187+0000 7f3dec10d640 1 -- 172.21.15.78:0/4267207193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de419f5d0 con 0x7f3de4103b30 2024-09-11T01:04:22.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.187+0000 7f3dec10d640 1 -- 172.21.15.78:0/4267207193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de41a5cf0 con 0x7f3de4103b30 2024-09-11T01:04:22.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3dee4e2740 1 Processor -- start 2024-09-11T01:04:22.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3dee4e2740 1 -- start start 2024-09-11T01:04:22.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3ddaffd640 1 -- 172.21.15.78:0/4267207193 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3de003b9f0 con 0x7f3de4103b30 2024-09-11T01:04:22.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3dee4e2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bf4d0 0x5630110bf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3dee4e2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bfdf0 0x5630110cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3dee4e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 0x5630110ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3dee4e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563011035ea0 con 0x5630110cbd80 2024-09-11T01:04:22.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3dee4e2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5630110252d0 con 0x5630110bfdf0 2024-09-11T01:04:22.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3dee4e2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630110290e0 con 0x5630110bf4d0 2024-09-11T01:04:22.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3de96a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 0x5630110ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3de9ea9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bfdf0 0x5630110cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3dea6aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bf4d0 0x5630110bf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3de96a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 0x5630110ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56596/0 (socket says 172.21.15.78:56596) 2024-09-11T01:04:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3dea6aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bf4d0 0x5630110bf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:55016/0 (socket says 172.21.15.78:55016) 2024-09-11T01:04:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.190+0000 7f3de96a8640 1 -- 172.21.15.78:0/143306590 learned_addr learned my addr 172.21.15.78:0/143306590 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3dea6aa640 1 -- 172.21.15.78:0/143306590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bfdf0 msgr2=0x5630110cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/143306590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bfdf0 0x5630110cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3dea6aa640 1 -- 172.21.15.78:0/143306590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 msgr2=0x5630110ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/143306590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 0x5630110ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3dea6aa640 1 -- 172.21.15.78:0/143306590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563010d83650 con 0x5630110bf4d0 2024-09-11T01:04:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3de9ea9640 1 --2- 172.21.15.78:0/143306590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bfdf0 0x5630110cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3de96a8640 1 --2- 172.21.15.78:0/143306590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 0x5630110ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3ddaffd640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3dd00b5450 0x7f3dd00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/143306590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bf4d0 0x5630110bf8b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc005fd0 tx=0x7f3ddc011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b14e7cde6a98b6e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3de9ea9640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3dd00b5450 0x7f3dd00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3dcffff640 1 -- 172.21.15.78:0/143306590 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc004d20 con 0x5630110bf4d0 2024-09-11T01:04:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3dcffff640 1 -- 172.21.15.78:0/143306590 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc00f9a0 con 0x5630110bf4d0 2024-09-11T01:04:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3dcffff640 1 -- 172.21.15.78:0/143306590 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc004470 con 0x5630110bf4d0 2024-09-11T01:04:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.191+0000 7f3ddaffd640 1 -- 172.21.15.78:0/4267207193 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3de0106630 con 0x7f3de4103b30 2024-09-11T01:04:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.192+0000 7f3dee4e2740 1 -- 172.21.15.78:0/143306590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bf4d0 msgr2=0x5630110bf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.192+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/143306590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bf4d0 0x5630110bf8b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc005fd0 tx=0x7f3ddc011bf0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.192+0000 7f3de9ea9640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3dd00b5450 0x7f3dd00b7910 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3de0103ea0 tx=0x7f3de010c550 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.192+0000 7f3dee4e2740 1 -- 172.21.15.78:0/143306590 shutdown_connections 2024-09-11T01:04:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.192+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/143306590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 0x5630110ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.192+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/143306590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bfdf0 0x5630110cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.192+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/143306590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bf4d0 0x5630110bf8b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.192+0000 7f3dee4e2740 1 -- 172.21.15.78:0/143306590 >> 172.21.15.78:0/143306590 conn(0x5630110bd300 msgr2=0x5630110be4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.192+0000 7f3dee4e2740 1 -- 172.21.15.78:0/143306590 shutdown_connections 2024-09-11T01:04:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.192+0000 7f3dee4e2740 1 -- 172.21.15.78:0/143306590 wait complete. 2024-09-11T01:04:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.192+0000 7f3dee4e2740 1 -- start start 2024-09-11T01:04:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.192+0000 7f3dee4e2740 1 Processor -- start 2024-09-11T01:04:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dee4e2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dee4e2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bfdf0 0x5630110c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dee4e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 0x5630110c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dee4e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563010e86700 con 0x5630110cbd80 2024-09-11T01:04:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dee4e2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563010fc0160 con 0x5630110bf4d0 2024-09-11T01:04:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dee4e2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630110d2e10 con 0x5630110bfdf0 2024-09-11T01:04:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dee4e2740 10 client.0 _ll_register_callbacks cb 0x563010fd99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dea6aa640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dea6aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 0x5630110c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dea6aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 0x5630110c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56612/0 (socket says 172.21.15.78:56612) 2024-09-11T01:04:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dea6aa640 1 -- 172.21.15.78:0/1285077590 learned_addr learned my addr 172.21.15.78:0/1285077590 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3de96a8640 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bfdf0 0x5630110c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dea6aa640 1 -- 172.21.15.78:0/1285077590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bfdf0 msgr2=0x5630110c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:22.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bfdf0 0x5630110c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dea6aa640 1 -- 172.21.15.78:0/1285077590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 msgr2=0x5630110cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:22.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3dea6aa640 1 -- 172.21.15.78:0/1285077590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563010fedc90 con 0x5630110cbd80 2024-09-11T01:04:22.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.193+0000 7f3de96a8640 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bfdf0 0x5630110c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:22.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 0x5630110c9180 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009fd0 tx=0x7f3ddc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3652a500fe53f04f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:22.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:22.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dcd7fa640 1 -- 172.21.15.78:0/1285077590 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc01e040 con 0x5630110cbd80 2024-09-11T01:04:22.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dcd7fa640 1 -- 172.21.15.78:0/1285077590 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3ddc011250 con 0x5630110cbd80 2024-09-11T01:04:22.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dee4e2740 1 -- 172.21.15.78:0/1285077590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ddc00f9a0 con 0x5630110cbd80 2024-09-11T01:04:22.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dcd7fa640 1 -- 172.21.15.78:0/1285077590 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3ddc0041f0 con 0x5630110cbd80 2024-09-11T01:04:22.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dee4e2740 10 client.16518 fetch_fsmap learned FSMap version 38 2024-09-11T01:04:22.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dee4e2740 1 -- 172.21.15.78:0/1285077590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5630110de7a0 con 0x5630110cbd80 2024-09-11T01:04:22.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dcd7fa640 1 -- 172.21.15.78:0/1285077590 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc00f9a0 con 0x5630110cbd80 2024-09-11T01:04:22.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dcd7fa640 1 -- 172.21.15.78:0/1285077590 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f3ddc024070 con 0x5630110cbd80 2024-09-11T01:04:22.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dee4e2740 10 client.16518 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:04:22.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dee4e2740 10 client.16518 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:04:22.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dee4e2740 20 client.16518 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:22.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5630110dfb50 0x5630110e1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dee4e2740 4 client.16518 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:04:22.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.194+0000 7f3dee4e2740 1 -- 172.21.15.78:0/1285077590 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563011037df0 con 0x5630110dfb50 2024-09-11T01:04:22.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.195+0000 7f3de96a8640 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5630110dfb50 0x5630110e1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.195+0000 7f3dcd7fa640 10 client.16518 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:04:22.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.195+0000 7f3de96a8640 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5630110dfb50 0x5630110e1f40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d684c6edf444b7f server_cookie=dcf1933afc2f332d in_seq=0 out_seq=0 2024-09-11T01:04:22.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dcd7fa640 1 -- 172.21.15.78:0/1285077590 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563011037df0 con 0x5630110dfb50 2024-09-11T01:04:22.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dcd7fa640 10 client.16518 handle_command_reply: tid=0 2024-09-11T01:04:22.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dee4e2740 1 client.16518 shutdown 2024-09-11T01:04:22.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dee4e2740 1 -- 172.21.15.78:0/1285077590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 msgr2=0x5630110c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:22.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 0x5630110c9180 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009fd0 tx=0x7f3ddc005bd0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dee4e2740 1 -- 172.21.15.78:0/1285077590 shutdown_connections 2024-09-11T01:04:22.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5630110dfb50 0x5630110e1f40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110cbd80 0x5630110c9180 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110bfdf0 0x5630110c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dcd7fa640 0 client.16518 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:04:22.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/1285077590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dee4e2740 1 -- 172.21.15.78:0/1285077590 >> 172.21.15.78:0/1285077590 conn(0x5630110bd300 msgr2=0x5630110c2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:22.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dee4e2740 1 -- 172.21.15.78:0/1285077590 shutdown_connections 2024-09-11T01:04:22.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dee4e2740 1 -- 172.21.15.78:0/1285077590 wait complete. 2024-09-11T01:04:22.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.196+0000 7f3dee4e2740 20 client.16518 trim_cache size 0 max 16384 2024-09-11T01:04:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.210+0000 7f3dee4e2740 1 Processor -- start 2024-09-11T01:04:22.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.210+0000 7f3dee4e2740 1 -- start start 2024-09-11T01:04:22.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.210+0000 7f3dee4e2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.210+0000 7f3dee4e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 0x5630110cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.210+0000 7f3dee4e2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110cbd80 0x5630110c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.210+0000 7f3dee4e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ac2f0 con 0x5630110bfdf0 2024-09-11T01:04:22.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.210+0000 7f3dee4e2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41adfb0 con 0x5630110bf4d0 2024-09-11T01:04:22.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.210+0000 7f3dee4e2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ac9c0 con 0x5630110cbd80 2024-09-11T01:04:22.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.210+0000 7f3de96a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 0x5630110cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3de96a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 0x5630110cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56616/0 (socket says 172.21.15.78:56616) 2024-09-11T01:04:22.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3de96a8640 1 -- 172.21.15.78:0/618442732 learned_addr learned my addr 172.21.15.78:0/618442732 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:22.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/618442732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/618442732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110cbd80 0x5630110c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3de96a8640 1 -- 172.21.15.78:0/618442732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110cbd80 msgr2=0x5630110c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3de96a8640 1 --2- 172.21.15.78:0/618442732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110cbd80 0x5630110c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3de96a8640 1 -- 172.21.15.78:0/618442732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 msgr2=0x5630110cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3de96a8640 1 --2- 172.21.15.78:0/618442732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3de96a8640 1 -- 172.21.15.78:0/618442732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630110c7020 con 0x5630110bfdf0 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/618442732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3de96a8640 1 --2- 172.21.15.78:0/618442732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 0x5630110cd6b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40127d0 tx=0x7f3dd4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45adc3898f4b3051 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3dcf7fe640 1 -- 172.21.15.78:0/618442732 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4011a40 con 0x5630110bfdf0 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3dcf7fe640 1 -- 172.21.15.78:0/618442732 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dd402a070 con 0x5630110bfdf0 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.211+0000 7f3dcf7fe640 1 -- 172.21.15.78:0/618442732 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd401f400 con 0x5630110bfdf0 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.212+0000 7f3dee4e2740 1 -- 172.21.15.78:0/618442732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 msgr2=0x5630110cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.212+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/618442732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 0x5630110cd6b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40127d0 tx=0x7f3dd4005f90 comp rx=0 tx=0).stop 2024-09-11T01:04:22.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.212+0000 7f3dee4e2740 1 -- 172.21.15.78:0/618442732 shutdown_connections 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.212+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/618442732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110cbd80 0x5630110c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.212+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/618442732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 0x5630110cd6b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.212+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/618442732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.212+0000 7f3dee4e2740 1 -- 172.21.15.78:0/618442732 >> 172.21.15.78:0/618442732 conn(0x5630110bd300 msgr2=0x5630110c73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.212+0000 7f3dee4e2740 1 -- 172.21.15.78:0/618442732 shutdown_connections 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.212+0000 7f3dee4e2740 1 -- 172.21.15.78:0/618442732 wait complete. 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.212+0000 7f3dee4e2740 1 -- start start 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3dee4e2740 1 Processor -- start 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3dee4e2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110d80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3dee4e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 0x5630110d85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3dee4e2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110cbd80 0x5630110c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3dee4e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ad820 con 0x5630110bfdf0 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3dee4e2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ad090 con 0x5630110bf4d0 2024-09-11T01:04:22.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3dee4e2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41aa2d0 con 0x5630110cbd80 2024-09-11T01:04:22.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3dee4e2740 10 client.0 _ll_register_callbacks cb 0x7f3dd4009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:22.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3de96a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 0x5630110d85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3de96a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 0x5630110d85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56620/0 (socket says 172.21.15.78:56620) 2024-09-11T01:04:22.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3de96a8640 1 -- 172.21.15.78:0/2611304395 learned_addr learned my addr 172.21.15.78:0/2611304395 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:22.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3dea6aa640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110d80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/2611304395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110cbd80 0x5630110c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3de96a8640 1 -- 172.21.15.78:0/2611304395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110cbd80 msgr2=0x5630110c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:22.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3de96a8640 1 --2- 172.21.15.78:0/2611304395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110cbd80 0x5630110c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3de96a8640 1 -- 172.21.15.78:0/2611304395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 msgr2=0x5630110d80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:22.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3de96a8640 1 --2- 172.21.15.78:0/2611304395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110d80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.213+0000 7f3de96a8640 1 -- 172.21.15.78:0/2611304395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630110d8df0 con 0x5630110bfdf0 2024-09-11T01:04:22.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3de96a8640 1 --2- 172.21.15.78:0/2611304395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 0x5630110d85e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4025b70 tx=0x7f3dd40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365ed999c0c56f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:22.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3dcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:22.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3dcd7fa640 1 -- 172.21.15.78:0/2611304395 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4025670 con 0x5630110bfdf0 2024-09-11T01:04:22.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3dcd7fa640 1 -- 172.21.15.78:0/2611304395 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3dd4025810 con 0x5630110bfdf0 2024-09-11T01:04:22.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3dee4e2740 1 -- 172.21.15.78:0/2611304395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dd402a070 con 0x5630110bfdf0 2024-09-11T01:04:22.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3dcd7fa640 1 -- 172.21.15.78:0/2611304395 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3dd4011840 con 0x5630110bfdf0 2024-09-11T01:04:22.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3dee4e2740 10 client.16524 fetch_fsmap learned FSMap version 38 2024-09-11T01:04:22.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3dee4e2740 1 -- 172.21.15.78:0/2611304395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5630110c6350 con 0x5630110bfdf0 2024-09-11T01:04:22.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3dcd7fa640 1 -- 172.21.15.78:0/2611304395 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dd402a070 con 0x5630110bfdf0 2024-09-11T01:04:22.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3dcd7fa640 1 -- 172.21.15.78:0/2611304395 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f3dd4041050 con 0x5630110bfdf0 2024-09-11T01:04:22.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3dee4e2740 10 client.16524 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3dee4e2740 10 client.16524 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.214+0000 7f3dee4e2740 20 client.16524 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.215+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/2611304395 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56301114d930 0x56301114fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.215+0000 7f3dee4e2740 4 client.16524 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.215+0000 7f3dee4e2740 1 -- 172.21.15.78:0/2611304395 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630110338a0 con 0x56301114d930 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.215+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/2611304395 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56301114d930 0x56301114fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.215+0000 7f3dcd7fa640 10 client.16524 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.215+0000 7f3dea6aa640 1 --2- 172.21.15.78:0/2611304395 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56301114d930 0x56301114fd20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=befaddab13f5a72a server_cookie=c1670b66a2a21600 in_seq=0 out_seq=0 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.215+0000 7f3dcd7fa640 1 -- 172.21.15.78:0/2611304395 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630110338a0 con 0x56301114d930 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.215+0000 7f3dcd7fa640 10 client.16524 handle_command_reply: tid=0 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.215+0000 7f3dee4e2740 1 client.16524 shutdown 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dee4e2740 1 -- 172.21.15.78:0/2611304395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 msgr2=0x5630110d85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/2611304395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 0x5630110d85e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4025b70 tx=0x7f3dd40056d0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dee4e2740 1 -- 172.21.15.78:0/2611304395 shutdown_connections 2024-09-11T01:04:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/2611304395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630110cbd80 0x5630110c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/2611304395 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x56301114d930 0x56301114fd20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/2611304395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630110bfdf0 0x5630110d85e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dee4e2740 1 --2- 172.21.15.78:0/2611304395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630110bf4d0 0x5630110d80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dcd7fa640 0 client.16524 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:04:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dee4e2740 1 -- 172.21.15.78:0/2611304395 >> 172.21.15.78:0/2611304395 conn(0x5630110bd300 msgr2=0x5630110ca150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dee4e2740 1 -- 172.21.15.78:0/2611304395 shutdown_connections 2024-09-11T01:04:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dee4e2740 1 -- 172.21.15.78:0/2611304395 wait complete. 2024-09-11T01:04:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dee4e2740 20 client.16524 trim_cache size 0 max 16384 2024-09-11T01:04:22.239 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:04:22.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dec10d640 1 -- 172.21.15.78:0/4267207193 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3dd00b5450 msgr2=0x7f3dd00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:22.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dec10d640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3dd00b5450 0x7f3dd00b7910 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3de0103ea0 tx=0x7f3de010c550 comp rx=0 tx=0).stop 2024-09-11T01:04:22.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dec10d640 1 -- 172.21.15.78:0/4267207193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3de4103b30 msgr2=0x7f3de4199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:22.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dec10d640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3de4103b30 0x7f3de4199290 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005940 tx=0x7f3de00040c0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dec10d640 1 -- 172.21.15.78:0/4267207193 shutdown_connections 2024-09-11T01:04:22.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dec10d640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3dd00b5450 0x7f3dd00b7910 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.216+0000 7f3dec10d640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de410d0e0 0x7f3de419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.217+0000 7f3dec10d640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4104530 0x7f3de41997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.217+0000 7f3dec10d640 1 --2- 172.21.15.78:0/4267207193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3de4103b30 0x7f3de4199290 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:22.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.217+0000 7f3dec10d640 1 -- 172.21.15.78:0/4267207193 >> 172.21.15.78:0/4267207193 conn(0x7f3de40fd700 msgr2=0x7f3de40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:22.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.217+0000 7f3dec10d640 1 -- 172.21.15.78:0/4267207193 shutdown_connections 2024-09-11T01:04:22.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:22.217+0000 7f3dec10d640 1 -- 172.21.15.78:0/4267207193 wait complete. 2024-09-11T01:04:22.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 425153 6.04 MB/sec execute 150 sec latency 2556.890 ms 2024-09-11T01:04:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:22 smithi112 ceph-mon[35384]: pgmap v488: 97 pgs: 97 active+clean; 252 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 188 op/s 2024-09-11T01:04:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:22 smithi078 ceph-mon[31315]: pgmap v488: 97 pgs: 97 active+clean; 252 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 188 op/s 2024-09-11T01:04:22.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:22 smithi110 ceph-mon[34712]: pgmap v488: 97 pgs: 97 active+clean; 252 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 188 op/s 2024-09-11T01:04:23.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 426597 6.05 MB/sec execute 151 sec latency 37.585 ms 2024-09-11T01:04:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:23 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:04:23.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:23 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:04:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:23 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:04:23.797 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:04:24.067 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:04:24.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 427365 6.04 MB/sec execute 152 sec latency 422.335 ms 2024-09-11T01:04:24.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:24 smithi078 ceph-mon[31315]: pgmap v489: 97 pgs: 97 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-09-11T01:04:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.582+0000 7f6db37ba640 1 -- 172.21.15.78:0/1343243095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac10d140 msgr2=0x7f6dac10f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.582+0000 7f6db37ba640 1 --2- 172.21.15.78:0/1343243095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac10d140 0x7f6dac10f5a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009ca0 tx=0x7f6da402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.583+0000 7f6db37ba640 1 -- 172.21.15.78:0/1343243095 shutdown_connections 2024-09-11T01:04:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.583+0000 7f6db37ba640 1 --2- 172.21.15.78:0/1343243095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac10d140 0x7f6dac10f5a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.583+0000 7f6db37ba640 1 --2- 172.21.15.78:0/1343243095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac104600 0x7f6dac10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.583+0000 7f6db37ba640 1 --2- 172.21.15.78:0/1343243095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac103ce0 0x7f6dac1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.583+0000 7f6db37ba640 1 -- 172.21.15.78:0/1343243095 >> 172.21.15.78:0/1343243095 conn(0x7f6dac0fd700 msgr2=0x7f6dac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.583+0000 7f6db37ba640 1 -- 172.21.15.78:0/1343243095 shutdown_connections 2024-09-11T01:04:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.583+0000 7f6db37ba640 1 -- 172.21.15.78:0/1343243095 wait complete. 2024-09-11T01:04:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.583+0000 7f6db37ba640 1 Processor -- start 2024-09-11T01:04:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.583+0000 7f6db37ba640 1 -- start start 2024-09-11T01:04:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.583+0000 7f6db37ba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac103ce0 0x7f6dac19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.583+0000 7f6db37ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac104600 0x7f6dac19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db37ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac10d140 0x7f6dac1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db37ba640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac112e10 con 0x7f6dac10d140 2024-09-11T01:04:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db37ba640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac112c90 con 0x7f6dac103ce0 2024-09-11T01:04:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db37ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac112f90 con 0x7f6dac104600 2024-09-11T01:04:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db1d57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac10d140 0x7f6dac1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db1d57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac10d140 0x7f6dac1a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44942/0 (socket says 172.21.15.78:44942) 2024-09-11T01:04:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db1d57640 1 -- 172.21.15.78:0/3877715558 learned_addr learned my addr 172.21.15.78:0/3877715558 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db1556640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac103ce0 0x7f6dac19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db0d55640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac104600 0x7f6dac19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db1d57640 1 -- 172.21.15.78:0/3877715558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac104600 msgr2=0x7f6dac19b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db1d57640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac104600 0x7f6dac19b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db1d57640 1 -- 172.21.15.78:0/3877715558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac103ce0 msgr2=0x7f6dac19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db1d57640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac103ce0 0x7f6dac19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db1d57640 1 -- 172.21.15.78:0/3877715558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dac1a1520 con 0x7f6dac10d140 2024-09-11T01:04:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db0d55640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac104600 0x7f6dac19b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:04:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db1d57640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac10d140 0x7f6dac1a0d50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6da4005cc0 tx=0x7f6da402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6da27fc640 1 -- 172.21.15.78:0/3877715558 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4003cf0 con 0x7f6dac10d140 2024-09-11T01:04:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6da27fc640 1 -- 172.21.15.78:0/3877715558 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da40044f0 con 0x7f6dac10d140 2024-09-11T01:04:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db37ba640 1 -- 172.21.15.78:0/3877715558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dac1a17b0 con 0x7f6dac10d140 2024-09-11T01:04:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6da27fc640 1 -- 172.21.15.78:0/3877715558 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da402fc30 con 0x7f6dac10d140 2024-09-11T01:04:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.584+0000 7f6db37ba640 1 -- 172.21.15.78:0/3877715558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dac1a7f50 con 0x7f6dac10d140 2024-09-11T01:04:24.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.587+0000 7f6da27fc640 1 -- 172.21.15.78:0/3877715558 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6da4004040 con 0x7f6dac10d140 2024-09-11T01:04:24.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.587+0000 7f6db5b8f740 1 Processor -- start 2024-09-11T01:04:24.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.587+0000 7f6db5b8f740 1 -- start start 2024-09-11T01:04:24.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.587+0000 7f6db5b8f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48fea4d0 0x55af48fea8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.587+0000 7f6db5b8f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48feadf0 0x55af48ff6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.587+0000 7f6db5b8f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48ff6d80 0x55af48ff9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.587+0000 7f6db5b8f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af48f60ea0 con 0x55af48ff6d80 2024-09-11T01:04:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.587+0000 7f6db5b8f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55af48f502d0 con 0x55af48feadf0 2024-09-11T01:04:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.587+0000 7f6db5b8f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55af48f540e0 con 0x55af48fea4d0 2024-09-11T01:04:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db0d55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48ff6d80 0x55af48ff9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db0d55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48fea4d0 0x55af48fea8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db0d55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48fea4d0 0x55af48fea8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:33604/0 (socket says 172.21.15.78:33604) 2024-09-11T01:04:24.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db0d55640 1 -- 172.21.15.78:0/3220067351 learned_addr learned my addr 172.21.15.78:0/3220067351 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:24.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db1556640 1 --2- 172.21.15.78:0/3220067351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48feadf0 0x55af48ff6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6da27fc640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d980b5450 0x7f6d980b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db0d55640 1 -- 172.21.15.78:0/3220067351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48feadf0 msgr2=0x55af48ff6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:24.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db0d55640 1 --2- 172.21.15.78:0/3220067351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48feadf0 0x55af48ff6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db0d55640 1 -- 172.21.15.78:0/3220067351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48ff6d80 msgr2=0x55af48ff9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:24.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db0d55640 1 --2- 172.21.15.78:0/3220067351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48ff6d80 0x55af48ff9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db0d55640 1 -- 172.21.15.78:0/3220067351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af48cae650 con 0x55af48fea4d0 2024-09-11T01:04:24.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db1d57640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d980b5450 0x7f6d980b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db0d55640 1 --2- 172.21.15.78:0/3220067351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48fea4d0 0x55af48fea8b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00dfd0 tx=0x7f6d9c006da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=969eb38f3fc4bc06 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6da27fc640 1 -- 172.21.15.78:0/3877715558 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6da41074c0 con 0x7f6dac10d140 2024-09-11T01:04:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6d977fe640 1 -- 172.21.15.78:0/3220067351 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c013580 con 0x55af48fea4d0 2024-09-11T01:04:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6db1d57640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d980b5450 0x7f6d980b7910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6da40c7f80 tx=0x7f6da4099c40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6d977fe640 1 -- 172.21.15.78:0/3220067351 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c01d070 con 0x55af48fea4d0 2024-09-11T01:04:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.588+0000 7f6d977fe640 1 -- 172.21.15.78:0/3220067351 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c018810 con 0x55af48fea4d0 2024-09-11T01:04:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.589+0000 7f6db5b8f740 1 -- 172.21.15.78:0/3220067351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48fea4d0 msgr2=0x55af48fea8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:24.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.589+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/3220067351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48fea4d0 0x55af48fea8b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00dfd0 tx=0x7f6d9c006da0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.589+0000 7f6db5b8f740 1 -- 172.21.15.78:0/3220067351 shutdown_connections 2024-09-11T01:04:24.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.589+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/3220067351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48ff6d80 0x55af48ff9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.589+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/3220067351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48feadf0 0x55af48ff6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.589+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/3220067351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48fea4d0 0x55af48fea8b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.589+0000 7f6db5b8f740 1 -- 172.21.15.78:0/3220067351 >> 172.21.15.78:0/3220067351 conn(0x55af48fe8300 msgr2=0x55af48fe94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:24.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.589+0000 7f6db5b8f740 1 -- 172.21.15.78:0/3220067351 shutdown_connections 2024-09-11T01:04:24.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.589+0000 7f6db5b8f740 1 -- 172.21.15.78:0/3220067351 wait complete. 2024-09-11T01:04:24.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.589+0000 7f6db5b8f740 1 -- start start 2024-09-11T01:04:24.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db5b8f740 1 Processor -- start 2024-09-11T01:04:24.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db5b8f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48fea4d0 0x55af48ff65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db5b8f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48feadf0 0x55af48ff3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db5b8f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48ff6d80 0x55af48ff3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db5b8f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af48db1700 con 0x55af48fea4d0 2024-09-11T01:04:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db5b8f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55af48eeb160 con 0x55af48ff6d80 2024-09-11T01:04:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db5b8f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55af48ff99a0 con 0x55af48feadf0 2024-09-11T01:04:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db5b8f740 10 client.0 _ll_register_callbacks cb 0x55af48f049e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db0d55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48fea4d0 0x55af48ff65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db0d55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48fea4d0 0x55af48ff65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44950/0 (socket says 172.21.15.78:44950) 2024-09-11T01:04:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db0d55640 1 -- 172.21.15.78:0/1998026533 learned_addr learned my addr 172.21.15.78:0/1998026533 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db1556640 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48feadf0 0x55af48ff3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db0d55640 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48ff6d80 0x55af48ff3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db0d55640 1 -- 172.21.15.78:0/1998026533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48feadf0 msgr2=0x55af48ff3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db0d55640 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48feadf0 0x55af48ff3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db0d55640 1 -- 172.21.15.78:0/1998026533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48ff6d80 msgr2=0x55af48ff3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db0d55640 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48ff6d80 0x55af48ff3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.590+0000 7f6db0d55640 1 -- 172.21.15.78:0/1998026533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af48f18c90 con 0x55af48fea4d0 2024-09-11T01:04:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6db1556640 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48feadf0 0x55af48ff3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6db0d55640 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48fea4d0 0x55af48ff65f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009000 tx=0x7f6d9c018490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5440fac39f0d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6d94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6d94ff9640 1 -- 172.21.15.78:0/1998026533 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c018770 con 0x55af48fea4d0 2024-09-11T01:04:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6d94ff9640 1 -- 172.21.15.78:0/1998026533 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6d9c018910 con 0x55af48fea4d0 2024-09-11T01:04:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1998026533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d9c01d070 con 0x55af48fea4d0 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6d94ff9640 1 -- 172.21.15.78:0/1998026533 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6d9c033050 con 0x55af48fea4d0 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6db5b8f740 10 client.16536 fetch_fsmap learned FSMap version 39 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1998026533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55af49005280 con 0x55af48fea4d0 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6d94ff9640 1 -- 172.21.15.78:0/1998026533 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c01d070 con 0x55af48fea4d0 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6d94ff9640 1 -- 172.21.15.78:0/1998026533 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8382+0+0 (secure 0 0 0) 0x7f6d9c015050 con 0x55af48fea4d0 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6db5b8f740 10 client.16536 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6db5b8f740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.591+0000 7f6db5b8f740 20 client.16536 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.592+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55af49006630 0x55af49008a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.592+0000 7f6db5b8f740 4 client.16536 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.592+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1998026533 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af48f62df0 con 0x55af49006630 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.592+0000 7f6db1556640 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55af49006630 0x55af49008a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.592+0000 7f6db1556640 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55af49006630 0x55af49008a20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d0d05876f9a4be5 server_cookie=38f6e8a9d06efcdf in_seq=0 out_seq=0 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.592+0000 7f6d94ff9640 10 client.16536 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6d94ff9640 1 -- 172.21.15.78:0/1998026533 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af48f62df0 con 0x55af49006630 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6d94ff9640 10 client.16536 handle_command_reply: tid=0 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6db5b8f740 1 client.16536 shutdown 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1998026533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48fea4d0 msgr2=0x55af48ff65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48fea4d0 0x55af48ff65f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009000 tx=0x7f6d9c018490 comp rx=0 tx=0).stop 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1998026533 shutdown_connections 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48ff6d80 0x55af48ff3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48feadf0 0x55af48ff3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55af49006630 0x55af49008a20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1998026533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48fea4d0 0x55af48ff65f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6d94ff9640 0 client.16536 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1998026533 >> 172.21.15.78:0/1998026533 conn(0x55af48fe8300 msgr2=0x55af48fede70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:24.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1998026533 shutdown_connections 2024-09-11T01:04:24.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1998026533 wait complete. 2024-09-11T01:04:24.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.593+0000 7f6db5b8f740 20 client.16536 trim_cache size 0 max 16384 2024-09-11T01:04:24.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.606+0000 7f6db5b8f740 1 Processor -- start 2024-09-11T01:04:24.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.606+0000 7f6db5b8f740 1 -- start start 2024-09-11T01:04:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.606+0000 7f6db5b8f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48fea4d0 0x55af48fe9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.606+0000 7f6db5b8f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 0x55af48fe7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.606+0000 7f6db5b8f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48ff6d80 0x55af48ff0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db5b8f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1b06f0 con 0x55af48feadf0 2024-09-11T01:04:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db5b8f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1b2320 con 0x55af48fea4d0 2024-09-11T01:04:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db5b8f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1b0dc0 con 0x55af48ff6d80 2024-09-11T01:04:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db1556640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 0x55af48fe7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db1556640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 0x55af48fe7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44966/0 (socket says 172.21.15.78:44966) 2024-09-11T01:04:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db1556640 1 -- 172.21.15.78:0/1477593863 learned_addr learned my addr 172.21.15.78:0/1477593863 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db0d55640 1 --2- 172.21.15.78:0/1477593863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48fea4d0 0x55af48fe9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db0d55640 1 --2- 172.21.15.78:0/1477593863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48ff6d80 0x55af48ff0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db1556640 1 -- 172.21.15.78:0/1477593863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48ff6d80 msgr2=0x55af48ff0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:24.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db1556640 1 --2- 172.21.15.78:0/1477593863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48ff6d80 0x55af48ff0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db1556640 1 -- 172.21.15.78:0/1477593863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48fea4d0 msgr2=0x55af48fe9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:24.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db1556640 1 --2- 172.21.15.78:0/1477593863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48fea4d0 0x55af48fe9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db1556640 1 -- 172.21.15.78:0/1477593863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af48ff4330 con 0x55af48feadf0 2024-09-11T01:04:24.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6db1556640 1 --2- 172.21.15.78:0/1477593863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 0x55af48fe7c10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x55af48fe8eb0 tx=0x7f6da800a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3586ed89874a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:24.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6d96ffd640 1 -- 172.21.15.78:0/1477593863 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da800fad0 con 0x55af48feadf0 2024-09-11T01:04:24.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6d96ffd640 1 -- 172.21.15.78:0/1477593863 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da80054d0 con 0x55af48feadf0 2024-09-11T01:04:24.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.607+0000 7f6d96ffd640 1 -- 172.21.15.78:0/1477593863 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8004bc0 con 0x55af48feadf0 2024-09-11T01:04:24.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.608+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1477593863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 msgr2=0x55af48fe7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:24.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.608+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1477593863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 0x55af48fe7c10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x55af48fe8eb0 tx=0x7f6da800a2d0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.608+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1477593863 shutdown_connections 2024-09-11T01:04:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.608+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1477593863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48ff6d80 0x55af48ff0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.608+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1477593863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 0x55af48fe7c10 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.608+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1477593863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48fea4d0 0x55af48fe9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.608+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1477593863 >> 172.21.15.78:0/1477593863 conn(0x55af48fe8300 msgr2=0x55af48fe9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.608+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1477593863 shutdown_connections 2024-09-11T01:04:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.608+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1477593863 wait complete. 2024-09-11T01:04:24.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.608+0000 7f6db5b8f740 1 -- start start 2024-09-11T01:04:24.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.608+0000 7f6db5b8f740 1 Processor -- start 2024-09-11T01:04:24.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db5b8f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48fea4d0 0x55af48ffb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db5b8f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 0x55af48ffb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db5b8f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48ff6d80 0x55af49072400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db5b8f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1b1b90 con 0x55af48feadf0 2024-09-11T01:04:24.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db5b8f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1b1490 con 0x55af48ff6d80 2024-09-11T01:04:24.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db5b8f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1ae6d0 con 0x55af48fea4d0 2024-09-11T01:04:24.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db5b8f740 10 client.0 _ll_register_callbacks cb 0x7f6d9c00dc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:24.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db1556640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 0x55af48ffb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db1556640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 0x55af48ffb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44976/0 (socket says 172.21.15.78:44976) 2024-09-11T01:04:24.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db1556640 1 -- 172.21.15.78:0/1175201049 learned_addr learned my addr 172.21.15.78:0/1175201049 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:24.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db0d55640 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48fea4d0 0x55af48ffb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db0d55640 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48ff6d80 0x55af49072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db1556640 1 -- 172.21.15.78:0/1175201049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48fea4d0 msgr2=0x55af48ffb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:24.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db1556640 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48fea4d0 0x55af48ffb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db1556640 1 -- 172.21.15.78:0/1175201049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48ff6d80 msgr2=0x55af49072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:24.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db1556640 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48ff6d80 0x55af49072400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db1556640 1 -- 172.21.15.78:0/1175201049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af49072ac0 con 0x55af48feadf0 2024-09-11T01:04:24.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.609+0000 7f6db0d55640 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48ff6d80 0x55af49072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:04:24.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6db1556640 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 0x55af48ffb930 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6da80048a0 tx=0x7f6da8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c12b28f0ee30c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:24.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6d94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:24.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6d94ff9640 1 -- 172.21.15.78:0/1175201049 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da80052f0 con 0x55af48feadf0 2024-09-11T01:04:24.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6d94ff9640 1 -- 172.21.15.78:0/1175201049 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6da800f8d0 con 0x55af48feadf0 2024-09-11T01:04:24.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1175201049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da80054d0 con 0x55af48feadf0 2024-09-11T01:04:24.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6d94ff9640 1 -- 172.21.15.78:0/1175201049 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6da803e050 con 0x55af48feadf0 2024-09-11T01:04:24.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6db5b8f740 10 client.16542 fetch_fsmap learned FSMap version 39 2024-09-11T01:04:24.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1175201049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55af49073da0 con 0x55af48feadf0 2024-09-11T01:04:24.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6d94ff9640 1 -- 172.21.15.78:0/1175201049 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da80054d0 con 0x55af48feadf0 2024-09-11T01:04:24.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6d94ff9640 1 -- 172.21.15.78:0/1175201049 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8382+0+0 (secure 0 0 0) 0x7f6da8030500 con 0x55af48feadf0 2024-09-11T01:04:24.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6db5b8f740 10 client.16542 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T01:04:24.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6db5b8f740 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:04:24.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6db5b8f740 20 client.16542 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:24.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55af490745d0 0x55af490769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:24.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6db5b8f740 4 client.16542 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:04:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.610+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1175201049 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af48f5e8a0 con 0x55af490745d0 2024-09-11T01:04:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db0d55640 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55af490745d0 0x55af490769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db0d55640 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55af490745d0 0x55af490769f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ac9abbcf13f6a9 server_cookie=ae6095fdd4b2295 in_seq=0 out_seq=0 2024-09-11T01:04:24.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6d94ff9640 10 client.16542 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:24.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6d94ff9640 1 -- 172.21.15.78:0/1175201049 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af48f5e8a0 con 0x55af490745d0 2024-09-11T01:04:24.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6d94ff9640 10 client.16542 handle_command_reply: tid=0 2024-09-11T01:04:24.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db5b8f740 1 client.16542 shutdown 2024-09-11T01:04:24.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1175201049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 msgr2=0x55af48ffb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:24.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 0x55af48ffb930 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6da80048a0 tx=0x7f6da8004730 comp rx=0 tx=0).stop 2024-09-11T01:04:24.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1175201049 shutdown_connections 2024-09-11T01:04:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af48ff6d80 0x55af49072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55af490745d0 0x55af490769f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af48feadf0 0x55af48ffb930 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6d94ff9640 0 client.16542 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db5b8f740 1 --2- 172.21.15.78:0/1175201049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af48fea4d0 0x55af48ffb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1175201049 >> 172.21.15.78:0/1175201049 conn(0x55af48fe8300 msgr2=0x55af48ff5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:24.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1175201049 shutdown_connections 2024-09-11T01:04:24.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.611+0000 7f6db5b8f740 1 -- 172.21.15.78:0/1175201049 wait complete. 2024-09-11T01:04:24.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db5b8f740 20 client.16542 trim_cache size 0 max 16384 2024-09-11T01:04:24.643 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:04:24.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db37ba640 1 -- 172.21.15.78:0/3877715558 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d980b5450 msgr2=0x7f6d980b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db37ba640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d980b5450 0x7f6d980b7910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6da40c7f80 tx=0x7f6da4099c40 comp rx=0 tx=0).stop 2024-09-11T01:04:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db37ba640 1 -- 172.21.15.78:0/3877715558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac10d140 msgr2=0x7f6dac1a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db37ba640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac10d140 0x7f6dac1a0d50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6da4005cc0 tx=0x7f6da402f470 comp rx=0 tx=0).stop 2024-09-11T01:04:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db37ba640 1 -- 172.21.15.78:0/3877715558 shutdown_connections 2024-09-11T01:04:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db37ba640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d980b5450 0x7f6d980b7910 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db37ba640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac10d140 0x7f6dac1a0d50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db37ba640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac104600 0x7f6dac19b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db37ba640 1 --2- 172.21.15.78:0/3877715558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac103ce0 0x7f6dac19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db37ba640 1 -- 172.21.15.78:0/3877715558 >> 172.21.15.78:0/3877715558 conn(0x7f6dac0fd700 msgr2=0x7f6dac10f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db37ba640 1 -- 172.21.15.78:0/3877715558 shutdown_connections 2024-09-11T01:04:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:24.612+0000 7f6db37ba640 1 -- 172.21.15.78:0/3877715558 wait complete. 2024-09-11T01:04:24.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:24 smithi110 ceph-mon[34712]: pgmap v489: 97 pgs: 97 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-09-11T01:04:24.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:24 smithi112 ceph-mon[35384]: pgmap v489: 97 pgs: 97 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-09-11T01:04:25.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 427365 6.00 MB/sec execute 153 sec latency 1422.418 ms 2024-09-11T01:04:26.387 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:04:26.387 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:04:26.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 427365 5.96 MB/sec execute 154 sec latency 2422.481 ms 2024-09-11T01:04:26.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:26 smithi078 ceph-mon[31315]: pgmap v490: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-11T01:04:26.644 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:04:26.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:26 smithi110 ceph-mon[34712]: pgmap v490: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-11T01:04:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:26 smithi112 ceph-mon[35384]: pgmap v490: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-11T01:04:27.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.109+0000 7f89f8c77640 1 -- 172.21.15.78:0/3536215835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f4108020 msgr2=0x7f89f410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:27.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.109+0000 7f89f8c77640 1 --2- 172.21.15.78:0/3536215835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f4108020 0x7f89f410e820 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009ca0 tx=0x7f89ec02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.109+0000 7f89f8c77640 1 -- 172.21.15.78:0/3536215835 shutdown_connections 2024-09-11T01:04:27.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.109+0000 7f89f8c77640 1 --2- 172.21.15.78:0/3536215835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f4108020 0x7f89f410e820 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.109+0000 7f89f8c77640 1 --2- 172.21.15.78:0/3536215835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f41076a0 0x7f89f4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.109+0000 7f89f8c77640 1 --2- 172.21.15.78:0/3536215835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89f4101c90 0x7f89f4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.109+0000 7f89f8c77640 1 -- 172.21.15.78:0/3536215835 >> 172.21.15.78:0/3536215835 conn(0x7f89f40fd700 msgr2=0x7f89f40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:27.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.109+0000 7f89f8c77640 1 -- 172.21.15.78:0/3536215835 shutdown_connections 2024-09-11T01:04:27.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.109+0000 7f89f8c77640 1 -- 172.21.15.78:0/3536215835 wait complete. 2024-09-11T01:04:27.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f8c77640 1 Processor -- start 2024-09-11T01:04:27.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f8c77640 1 -- start start 2024-09-11T01:04:27.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f8c77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f4101c90 0x7f89f4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f8c77640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89f41076a0 0x7f89f4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f8c77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f4108020 0x7f89f419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f8c77640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89f4111ca0 con 0x7f89f4101c90 2024-09-11T01:04:27.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f8c77640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f89f4111b20 con 0x7f89f41076a0 2024-09-11T01:04:27.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f8c77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89f4111e20 con 0x7f89f4108020 2024-09-11T01:04:27.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f4101c90 0x7f89f4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f4101c90 0x7f89f4199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44998/0 (socket says 172.21.15.78:44998) 2024-09-11T01:04:27.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f259c640 1 -- 172.21.15.78:0/144130946 learned_addr learned my addr 172.21.15.78:0/144130946 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:27.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f2d9d640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f4108020 0x7f89f419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89e9d9b640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89f41076a0 0x7f89f4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f259c640 1 -- 172.21.15.78:0/144130946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f4108020 msgr2=0x7f89f419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:27.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f259c640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f4108020 0x7f89f419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f259c640 1 -- 172.21.15.78:0/144130946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89f41076a0 msgr2=0x7f89f4199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:27.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f259c640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89f41076a0 0x7f89f4199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f259c640 1 -- 172.21.15.78:0/144130946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f419f2f0 con 0x7f89f4101c90 2024-09-11T01:04:27.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.110+0000 7f89f2d9d640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f4108020 0x7f89f419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:04:27.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.111+0000 7f89f259c640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f4101c90 0x7f89f4199240 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f89e400d950 tx=0x7f89e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:27.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.111+0000 7f89ebfff640 1 -- 172.21.15.78:0/144130946 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4017070 con 0x7f89f4101c90 2024-09-11T01:04:27.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.111+0000 7f89f8c77640 1 -- 172.21.15.78:0/144130946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f419f5e0 con 0x7f89f4101c90 2024-09-11T01:04:27.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.111+0000 7f89ebfff640 1 -- 172.21.15.78:0/144130946 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e4009c30 con 0x7f89f4101c90 2024-09-11T01:04:27.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.111+0000 7f89ebfff640 1 -- 172.21.15.78:0/144130946 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e400f620 con 0x7f89f4101c90 2024-09-11T01:04:27.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.111+0000 7f89f8c77640 1 -- 172.21.15.78:0/144130946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f41a5ca0 con 0x7f89f4101c90 2024-09-11T01:04:27.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.114+0000 7f89ebfff640 1 -- 172.21.15.78:0/144130946 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f89e4010460 con 0x7f89f4101c90 2024-09-11T01:04:27.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.114+0000 7f89fb04c740 1 Processor -- start 2024-09-11T01:04:27.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.114+0000 7f89fb04c740 1 -- start start 2024-09-11T01:04:27.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.114+0000 7f89fb04c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79e7ee0 0x5629b79e82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.114+0000 7f89fb04c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b79f8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.114+0000 7f89fb04c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79f8ba0 0x5629b79faf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.114+0000 7f89fb04c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5629b795ee60 con 0x5629b79e8800 2024-09-11T01:04:27.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.114+0000 7f89fb04c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5629b794e290 con 0x5629b79e7ee0 2024-09-11T01:04:27.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.114+0000 7f89fb04c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5629b79520a0 con 0x5629b79f8ba0 2024-09-11T01:04:27.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.114+0000 7f89f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b79f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89f2d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79e7ee0 0x5629b79e82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b79f8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45006/0 (socket says 172.21.15.78:45006) 2024-09-11T01:04:27.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89f259c640 1 -- 172.21.15.78:0/1944573403 learned_addr learned my addr 172.21.15.78:0/1944573403 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:27.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89e9d9b640 1 --2- 172.21.15.78:0/1944573403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79f8ba0 0x5629b79faf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89f259c640 1 -- 172.21.15.78:0/1944573403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79f8ba0 msgr2=0x5629b79faf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:27.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89f259c640 1 --2- 172.21.15.78:0/1944573403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79f8ba0 0x5629b79faf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89f259c640 1 -- 172.21.15.78:0/1944573403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79e7ee0 msgr2=0x5629b79e82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:27.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89f259c640 1 --2- 172.21.15.78:0/1944573403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79e7ee0 0x5629b79e82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89f259c640 1 -- 172.21.15.78:0/1944573403 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b76ac930 con 0x5629b79e8800 2024-09-11T01:04:27.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89f259c640 1 --2- 172.21.15.78:0/1944573403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b79f8660 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f89e40e33c0 tx=0x7f89e40e9f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54e5ad861258130 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:27.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89e8d99640 1 -- 172.21.15.78:0/1944573403 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e40e7030 con 0x5629b79e8800 2024-09-11T01:04:27.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89ebfff640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f89d40b5450 0x7f89d40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89e8d99640 1 -- 172.21.15.78:0/1944573403 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e40ebcd0 con 0x5629b79e8800 2024-09-11T01:04:27.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89e8d99640 1 -- 172.21.15.78:0/1944573403 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e40f6d60 con 0x5629b79e8800 2024-09-11T01:04:27.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89f2d9d640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f89d40b5450 0x7f89d40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.115+0000 7f89ebfff640 1 -- 172.21.15.78:0/144130946 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f89e40e5cc0 con 0x7f89f4101c90 2024-09-11T01:04:27.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.116+0000 7f89fb04c740 1 -- 172.21.15.78:0/1944573403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 msgr2=0x5629b79f8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:27.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.116+0000 7f89f2d9d640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f89d40b5450 0x7f89d40b7910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f89ec004af0 tx=0x7f89ec005c30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:27.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.116+0000 7f89fb04c740 1 --2- 172.21.15.78:0/1944573403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b79f8660 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f89e40e33c0 tx=0x7f89e40e9f90 comp rx=0 tx=0).stop 2024-09-11T01:04:27.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.116+0000 7f89fb04c740 1 -- 172.21.15.78:0/1944573403 shutdown_connections 2024-09-11T01:04:27.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.116+0000 7f89fb04c740 1 --2- 172.21.15.78:0/1944573403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79f8ba0 0x5629b79faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.116+0000 7f89fb04c740 1 --2- 172.21.15.78:0/1944573403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b79f8660 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.116+0000 7f89fb04c740 1 --2- 172.21.15.78:0/1944573403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79e7ee0 0x5629b79e82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.116+0000 7f89fb04c740 1 -- 172.21.15.78:0/1944573403 >> 172.21.15.78:0/1944573403 conn(0x5629b79e5d10 msgr2=0x5629b79e6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:27.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.116+0000 7f89fb04c740 1 -- 172.21.15.78:0/1944573403 shutdown_connections 2024-09-11T01:04:27.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.116+0000 7f89fb04c740 1 -- 172.21.15.78:0/1944573403 wait complete. 2024-09-11T01:04:27.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.116+0000 7f89fb04c740 1 -- start start 2024-09-11T01:04:27.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89fb04c740 1 Processor -- start 2024-09-11T01:04:27.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89fb04c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e7ee0 0x5629b79f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89fb04c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b79f6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89fb04c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b79f6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89fb04c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5629b77af890 con 0x5629b79e8800 2024-09-11T01:04:27.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89fb04c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5629b78e8d40 con 0x5629b79f8ba0 2024-09-11T01:04:27.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89fb04c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5629b79fb850 con 0x5629b79e7ee0 2024-09-11T01:04:27.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89fb04c740 10 client.0 _ll_register_callbacks cb 0x5629b794cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:27.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89e9d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b79f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89e9d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b79f6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45008/0 (socket says 172.21.15.78:45008) 2024-09-11T01:04:27.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89e9d9b640 1 -- 172.21.15.78:0/3293850029 learned_addr learned my addr 172.21.15.78:0/3293850029 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:27.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89f259c640 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e7ee0 0x5629b79f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89f2d9d640 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b79f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89e9d9b640 1 -- 172.21.15.78:0/3293850029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e7ee0 msgr2=0x5629b79f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89e9d9b640 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e7ee0 0x5629b79f5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89e9d9b640 1 -- 172.21.15.78:0/3293850029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 msgr2=0x5629b79f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89e9d9b640 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b79f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89e9d9b640 1 -- 172.21.15.78:0/3293850029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b7a04c00 con 0x5629b79e8800 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89f2d9d640 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b79f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89e9d9b640 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b79f6050 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00bf80 tx=0x7f89dc0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d1b78ea78e4d02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.117+0000 7f89e1ffb640 1 -- 172.21.15.78:0/3293850029 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc01f060 con 0x5629b79e8800 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89e1ffb640 1 -- 172.21.15.78:0/3293850029 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f89dc010900 con 0x5629b79e8800 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89fb04c740 1 -- 172.21.15.78:0/3293850029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89e40ebcd0 con 0x5629b79e8800 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89e1ffb640 1 -- 172.21.15.78:0/3293850029 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f89dc010b00 con 0x5629b79e8800 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89fb04c740 10 client.16554 fetch_fsmap learned FSMap version 39 2024-09-11T01:04:27.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89fb04c740 1 -- 172.21.15.78:0/3293850029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5629b7a07610 con 0x5629b79e8800 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89e1ffb640 1 -- 172.21.15.78:0/3293850029 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e40ebcd0 con 0x5629b79e8800 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89e1ffb640 1 -- 172.21.15.78:0/3293850029 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8382+0+0 (secure 0 0 0) 0x7f89dc022020 con 0x5629b79e8800 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89fb04c740 10 client.16554 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89fb04c740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89fb04c740 20 client.16554 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89fb04c740 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629b7a08bc0 0x5629b7a0b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89fb04c740 4 client.16554 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89fb04c740 1 -- 172.21.15.78:0/3293850029 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629b7960db0 con 0x5629b7a08bc0 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.118+0000 7f89f259c640 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629b7a08bc0 0x5629b7a0b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.119+0000 7f89e1ffb640 10 client.16554 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.119+0000 7f89f259c640 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629b7a08bc0 0x5629b7a0b000 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3f051f2fd69cf01 server_cookie=3782736c4cc69f1a in_seq=0 out_seq=0 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.119+0000 7f89e1ffb640 1 -- 172.21.15.78:0/3293850029 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5629b7960db0 con 0x5629b7a08bc0 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89e1ffb640 10 client.16554 handle_command_reply: tid=0 2024-09-11T01:04:27.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89fb04c740 1 client.16554 shutdown 2024-09-11T01:04:27.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89fb04c740 1 -- 172.21.15.78:0/3293850029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 msgr2=0x5629b79f6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:27.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89fb04c740 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b79f6050 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00bf80 tx=0x7f89dc0128e0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89fb04c740 1 -- 172.21.15.78:0/3293850029 shutdown_connections 2024-09-11T01:04:27.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89fb04c740 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b79f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89fb04c740 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629b7a08bc0 0x5629b7a0b000 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89fb04c740 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b79f6050 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89e1ffb640 0 client.16554 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:27.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89fb04c740 1 --2- 172.21.15.78:0/3293850029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e7ee0 0x5629b79f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89fb04c740 1 -- 172.21.15.78:0/3293850029 >> 172.21.15.78:0/3293850029 conn(0x5629b79e5d10 msgr2=0x5629b79e4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:27.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89fb04c740 1 -- 172.21.15.78:0/3293850029 shutdown_connections 2024-09-11T01:04:27.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89fb04c740 1 -- 172.21.15.78:0/3293850029 wait complete. 2024-09-11T01:04:27.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.120+0000 7f89fb04c740 20 client.16554 trim_cache size 0 max 16384 2024-09-11T01:04:27.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.137+0000 7f89fb04c740 1 Processor -- start 2024-09-11T01:04:27.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.137+0000 7f89fb04c740 1 -- start start 2024-09-11T01:04:27.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.137+0000 7f89fb04c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e7ee0 0x5629b7a108d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.137+0000 7f89fb04c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b7a10e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.137+0000 7f89fb04c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b7a00b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.137+0000 7f89fb04c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41ac3d0 con 0x5629b79e8800 2024-09-11T01:04:27.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.137+0000 7f89fb04c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41ae000 con 0x5629b79f8ba0 2024-09-11T01:04:27.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.137+0000 7f89fb04c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41acaa0 con 0x5629b79e7ee0 2024-09-11T01:04:27.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.137+0000 7f89e9d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b7a10e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.137+0000 7f89e9d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b7a10e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45018/0 (socket says 172.21.15.78:45018) 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.137+0000 7f89e9d9b640 1 -- 172.21.15.78:0/1865328610 learned_addr learned my addr 172.21.15.78:0/1865328610 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.137+0000 7f89f259c640 1 --2- 172.21.15.78:0/1865328610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e7ee0 0x5629b7a108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.138+0000 7f89e9d9b640 1 -- 172.21.15.78:0/1865328610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e7ee0 msgr2=0x5629b7a108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.138+0000 7f89e9d9b640 1 --2- 172.21.15.78:0/1865328610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e7ee0 0x5629b7a108d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.138+0000 7f89e9d9b640 1 -- 172.21.15.78:0/1865328610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 msgr2=0x5629b7a00b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.138+0000 7f89f2d9d640 1 --2- 172.21.15.78:0/1865328610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b7a00b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.138+0000 7f89e9d9b640 1 --2- 172.21.15.78:0/1865328610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b7a00b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.138+0000 7f89e9d9b640 1 -- 172.21.15.78:0/1865328610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b79e1640 con 0x5629b79e8800 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.138+0000 7f89e9d9b640 1 --2- 172.21.15.78:0/1865328610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b7a10e10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00ff30 tx=0x7f89dc00bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db54a097b4bfc06 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.138+0000 7f89e3fff640 1 -- 172.21.15.78:0/1865328610 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc010ea0 con 0x5629b79e8800 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.138+0000 7f89e3fff640 1 -- 172.21.15.78:0/1865328610 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89dc00dcf0 con 0x5629b79e8800 2024-09-11T01:04:27.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.138+0000 7f89e3fff640 1 -- 172.21.15.78:0/1865328610 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc00f620 con 0x5629b79e8800 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.138+0000 7f89fb04c740 1 -- 172.21.15.78:0/1865328610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 msgr2=0x5629b7a10e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 --2- 172.21.15.78:0/1865328610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b7a10e10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00ff30 tx=0x7f89dc00bbd0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 -- 172.21.15.78:0/1865328610 shutdown_connections 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 --2- 172.21.15.78:0/1865328610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b7a00b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 --2- 172.21.15.78:0/1865328610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e8800 0x5629b7a10e10 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 --2- 172.21.15.78:0/1865328610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e7ee0 0x5629b7a108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 -- 172.21.15.78:0/1865328610 >> 172.21.15.78:0/1865328610 conn(0x5629b79e5d10 msgr2=0x5629b79f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 -- 172.21.15.78:0/1865328610 shutdown_connections 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 -- 172.21.15.78:0/1865328610 wait complete. 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 -- start start 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 Processor -- start 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e7ee0 0x5629b79fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e8800 0x5629b79fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b7a74ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41ad870 con 0x5629b79e7ee0 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41ad170 con 0x5629b79f8ba0 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41aa3d0 con 0x5629b79e8800 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89fb04c740 10 client.0 _ll_register_callbacks cb 0x7f89dc00b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e7ee0 0x5629b79fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e7ee0 0x5629b79fcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45024/0 (socket says 172.21.15.78:45024) 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.139+0000 7f89f259c640 1 -- 172.21.15.78:0/2248425986 learned_addr learned my addr 172.21.15.78:0/2248425986 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89f2d9d640 1 --2- 172.21.15.78:0/2248425986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b7a74ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89f259c640 1 -- 172.21.15.78:0/2248425986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e8800 msgr2=0x5629b79fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89f259c640 1 --2- 172.21.15.78:0/2248425986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e8800 0x5629b79fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89f259c640 1 -- 172.21.15.78:0/2248425986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 msgr2=0x5629b7a74ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89f259c640 1 --2- 172.21.15.78:0/2248425986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b7a74ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89f259c640 1 -- 172.21.15.78:0/2248425986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b7a75010 con 0x5629b79e7ee0 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89f259c640 1 --2- 172.21.15.78:0/2248425986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e7ee0 0x5629b79fcec0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f89e40ac140 tx=0x7f89e40f68c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a220625570815d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89e1ffb640 1 -- 172.21.15.78:0/2248425986 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e40e7060 con 0x5629b79e7ee0 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89e1ffb640 1 -- 172.21.15.78:0/2248425986 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f89e40e99f0 con 0x5629b79e7ee0 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89fb04c740 1 -- 172.21.15.78:0/2248425986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89dc00dcf0 con 0x5629b79e7ee0 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89e1ffb640 1 -- 172.21.15.78:0/2248425986 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f89e4020040 con 0x5629b79e7ee0 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89fb04c740 10 client.16560 fetch_fsmap learned FSMap version 39 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89fb04c740 1 -- 172.21.15.78:0/2248425986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5629b7a762d0 con 0x5629b79e7ee0 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89e1ffb640 1 -- 172.21.15.78:0/2248425986 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89dc00dcf0 con 0x5629b79e7ee0 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.140+0000 7f89e1ffb640 1 -- 172.21.15.78:0/2248425986 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8382+0+0 (secure 0 0 0) 0x7f89e40f5070 con 0x5629b79e7ee0 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.141+0000 7f89fb04c740 10 client.16560 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T01:04:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.141+0000 7f89fb04c740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:04:27.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.141+0000 7f89fb04c740 20 client.16560 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:27.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.141+0000 7f89fb04c740 1 --2- 172.21.15.78:0/2248425986 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629b7a08bc0 0x5629b7a78730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:27.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.141+0000 7f89fb04c740 4 client.16560 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:04:27.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.141+0000 7f89fb04c740 1 -- 172.21.15.78:0/2248425986 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5629b795c860 con 0x5629b7a08bc0 2024-09-11T01:04:27.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.141+0000 7f89e9d9b640 1 --2- 172.21.15.78:0/2248425986 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629b7a08bc0 0x5629b7a78730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:27.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.141+0000 7f89e9d9b640 1 --2- 172.21.15.78:0/2248425986 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629b7a08bc0 0x5629b7a78730 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4de5390850732c75 server_cookie=f05967d339fcc4aa in_seq=0 out_seq=0 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.141+0000 7f89e1ffb640 10 client.16560 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89e1ffb640 1 -- 172.21.15.78:0/2248425986 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5629b795c860 con 0x5629b7a08bc0 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89e1ffb640 10 client.16560 handle_command_reply: tid=0 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89fb04c740 1 client.16560 shutdown 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89fb04c740 1 -- 172.21.15.78:0/2248425986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e7ee0 msgr2=0x5629b79fcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89fb04c740 1 --2- 172.21.15.78:0/2248425986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e7ee0 0x5629b79fcec0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f89e40ac140 tx=0x7f89e40f68c0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89fb04c740 1 -- 172.21.15.78:0/2248425986 shutdown_connections 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89fb04c740 1 --2- 172.21.15.78:0/2248425986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629b79f8ba0 0x5629b7a74ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89fb04c740 1 --2- 172.21.15.78:0/2248425986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629b79e8800 0x5629b79fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89fb04c740 1 --2- 172.21.15.78:0/2248425986 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629b7a08bc0 0x5629b7a78730 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89e1ffb640 0 client.16560 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89fb04c740 1 --2- 172.21.15.78:0/2248425986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629b79e7ee0 0x5629b79fcec0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89fb04c740 1 -- 172.21.15.78:0/2248425986 >> 172.21.15.78:0/2248425986 conn(0x5629b79e5d10 msgr2=0x5629b79f7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:27.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89fb04c740 1 -- 172.21.15.78:0/2248425986 shutdown_connections 2024-09-11T01:04:27.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89fb04c740 1 -- 172.21.15.78:0/2248425986 wait complete. 2024-09-11T01:04:27.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89fb04c740 20 client.16560 trim_cache size 0 max 16384 2024-09-11T01:04:27.151 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:04:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:04:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e", 2024-09-11T01:04:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:04:27.151 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:04:27.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89f8c77640 1 -- 172.21.15.78:0/144130946 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f89d40b5450 msgr2=0x7f89d40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:27.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89f8c77640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f89d40b5450 0x7f89d40b7910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f89ec004af0 tx=0x7f89ec005c30 comp rx=0 tx=0).stop 2024-09-11T01:04:27.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.143+0000 7f89f8c77640 1 -- 172.21.15.78:0/144130946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f4101c90 msgr2=0x7f89f4199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:27.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.144+0000 7f89f8c77640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f4101c90 0x7f89f4199240 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f89e400d950 tx=0x7f89e400de10 comp rx=0 tx=0).stop 2024-09-11T01:04:27.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.144+0000 7f89f8c77640 1 -- 172.21.15.78:0/144130946 shutdown_connections 2024-09-11T01:04:27.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.144+0000 7f89f8c77640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f89d40b5450 0x7f89d40b7910 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.144+0000 7f89f8c77640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89f4108020 0x7f89f419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.144+0000 7f89f8c77640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89f41076a0 0x7f89f4199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.144+0000 7f89f8c77640 1 --2- 172.21.15.78:0/144130946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f4101c90 0x7f89f4199240 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:27.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.144+0000 7f89f8c77640 1 -- 172.21.15.78:0/144130946 >> 172.21.15.78:0/144130946 conn(0x7f89f40fd700 msgr2=0x7f89f406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:27.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.144+0000 7f89f8c77640 1 -- 172.21.15.78:0/144130946 shutdown_connections 2024-09-11T01:04:27.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:27.144+0000 7f89f8c77640 1 -- 172.21.15.78:0/144130946 wait complete. 2024-09-11T01:04:27.431 INFO:tasks.workunit.client.0.smithi078.stdout: 1 428076 5.93 MB/sec execute 155 sec latency 3194.732 ms 2024-09-11T01:04:27.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:27 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:04:27.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:27 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:27.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:27 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:04:27.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:27 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:27.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:27 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:04:27.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:27 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:27.900 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:04:28.162 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:04:28.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 429756 5.97 MB/sec execute 156 sec latency 9.810 ms 2024-09-11T01:04:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:04:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:04:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:28 smithi078 ceph-mon[31315]: pgmap v491: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-09-11T01:04:28.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:04:28.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:04:28.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:04:28.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:28 smithi110 ceph-mon[34712]: pgmap v491: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-09-11T01:04:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.703+0000 7fdce637d640 1 -- 172.21.15.78:0/2232949644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce0108020 msgr2=0x7fdce010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.703+0000 7fdce637d640 1 --2- 172.21.15.78:0/2232949644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce0108020 0x7fdce010e820 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009ca0 tx=0x7fdcd802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.703+0000 7fdce637d640 1 -- 172.21.15.78:0/2232949644 shutdown_connections 2024-09-11T01:04:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.703+0000 7fdce637d640 1 --2- 172.21.15.78:0/2232949644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce0108020 0x7fdce010e820 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.703+0000 7fdce637d640 1 --2- 172.21.15.78:0/2232949644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdce01076a0 0x7fdce0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.703+0000 7fdce637d640 1 --2- 172.21.15.78:0/2232949644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce0101c90 0x7fdce0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.703+0000 7fdce637d640 1 -- 172.21.15.78:0/2232949644 >> 172.21.15.78:0/2232949644 conn(0x7fdce00fd700 msgr2=0x7fdce00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.703+0000 7fdce637d640 1 -- 172.21.15.78:0/2232949644 shutdown_connections 2024-09-11T01:04:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.703+0000 7fdce637d640 1 -- 172.21.15.78:0/2232949644 wait complete. 2024-09-11T01:04:28.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.703+0000 7fdce637d640 1 Processor -- start 2024-09-11T01:04:28.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.703+0000 7fdce637d640 1 -- start start 2024-09-11T01:04:28.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdce637d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce0101c90 0x7fdce00fde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdce637d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdce01076a0 0x7fdce00fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdce637d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce0108020 0x7fdce00fe900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdce637d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0111ca0 con 0x7fdce0101c90 2024-09-11T01:04:28.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdce637d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0111b20 con 0x7fdce01076a0 2024-09-11T01:04:28.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdce637d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0111e20 con 0x7fdce0108020 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce0101c90 0x7fdce00fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce0101c90 0x7fdce00fde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45048/0 (socket says 172.21.15.78:45048) 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdcdffff640 1 -- 172.21.15.78:0/3626442969 learned_addr learned my addr 172.21.15.78:0/3626442969 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdcdf7fe640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdce01076a0 0x7fdce00fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdce491a640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce0108020 0x7fdce00fe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdcdffff640 1 -- 172.21.15.78:0/3626442969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce0108020 msgr2=0x7fdce00fe900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdcdffff640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce0108020 0x7fdce00fe900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdcdffff640 1 -- 172.21.15.78:0/3626442969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdce01076a0 msgr2=0x7fdce00fe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdcdffff640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdce01076a0 0x7fdce00fe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.704+0000 7fdcdffff640 1 -- 172.21.15.78:0/3626442969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce01a76a0 con 0x7fdce0101c90 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.705+0000 7fdcdffff640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce0101c90 0x7fdce00fde80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000e9f0 tx=0x7fdcd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.705+0000 7fdcdd7fa640 1 -- 172.21.15.78:0/3626442969 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd000cd80 con 0x7fdce0101c90 2024-09-11T01:04:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.705+0000 7fdcdd7fa640 1 -- 172.21.15.78:0/3626442969 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd00054d0 con 0x7fdce0101c90 2024-09-11T01:04:28.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.705+0000 7fdce637d640 1 -- 172.21.15.78:0/3626442969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce01a7990 con 0x7fdce0101c90 2024-09-11T01:04:28.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.705+0000 7fdcdd7fa640 1 -- 172.21.15.78:0/3626442969 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0010690 con 0x7fdce0101c90 2024-09-11T01:04:28.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.705+0000 7fdce637d640 1 -- 172.21.15.78:0/3626442969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce01a7e80 con 0x7fdce0101c90 2024-09-11T01:04:28.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.707+0000 7fdcdd7fa640 1 -- 172.21.15.78:0/3626442969 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fdcd0004e90 con 0x7fdce0101c90 2024-09-11T01:04:28.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdce8752740 1 Processor -- start 2024-09-11T01:04:28.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdce8752740 1 -- start start 2024-09-11T01:04:28.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdce8752740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af780a0 0x55f59af78480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdce8752740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af789c0 0x55f59af84410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdce8752740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af84950 0x55f59af86d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdce8752740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f59aeeee50 con 0x55f59af780a0 2024-09-11T01:04:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdce8752740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f59aede280 con 0x55f59af84950 2024-09-11T01:04:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdce8752740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f59aee2090 con 0x55f59af789c0 2024-09-11T01:04:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdcdd7fa640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdcc80b5450 0x7fdcc80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdce491a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af780a0 0x55f59af78480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af789c0 0x55f59af84410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdce491a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af780a0 0x55f59af78480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45054/0 (socket says 172.21.15.78:45054) 2024-09-11T01:04:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdce491a640 1 -- 172.21.15.78:0/2792051962 learned_addr learned my addr 172.21.15.78:0/2792051962 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdce491a640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdcc80b5450 0x7fdcc80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdcdf7fe640 1 --2- 172.21.15.78:0/2792051962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af84950 0x55f59af86d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdcdffff640 1 -- 172.21.15.78:0/2792051962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af84950 msgr2=0x55f59af86d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdcdffff640 1 --2- 172.21.15.78:0/2792051962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af84950 0x55f59af86d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdcdffff640 1 -- 172.21.15.78:0/2792051962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af780a0 msgr2=0x55f59af78480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdcdffff640 1 --2- 172.21.15.78:0/2792051962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af780a0 0x55f59af78480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.708+0000 7fdcdffff640 1 -- 172.21.15.78:0/2792051962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f59ac3c650 con 0x55f59af789c0 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.709+0000 7fdcdd7fa640 1 -- 172.21.15.78:0/3626442969 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdcd00e63e0 con 0x7fdce0101c90 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.709+0000 7fdce491a640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdcc80b5450 0x7fdcc80b7910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8005fe0 tx=0x7fdcd8005ef0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.709+0000 7fdcdffff640 1 --2- 172.21.15.78:0/2792051962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af789c0 0x55f59af84410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00e3b40 tx=0x7fdcd00ac4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58ee17a78aeeeebc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.709+0000 7fdcc67fc640 1 -- 172.21.15.78:0/2792051962 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0005680 con 0x55f59af789c0 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.709+0000 7fdcc67fc640 1 -- 172.21.15.78:0/2792051962 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd00acf70 con 0x55f59af789c0 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.709+0000 7fdcc67fc640 1 -- 172.21.15.78:0/2792051962 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd00b1350 con 0x55f59af789c0 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.710+0000 7fdce8752740 1 -- 172.21.15.78:0/2792051962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af789c0 msgr2=0x55f59af84410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.710+0000 7fdce8752740 1 --2- 172.21.15.78:0/2792051962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af789c0 0x55f59af84410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00e3b40 tx=0x7fdcd00ac4b0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.710+0000 7fdce8752740 1 -- 172.21.15.78:0/2792051962 shutdown_connections 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.710+0000 7fdce8752740 1 --2- 172.21.15.78:0/2792051962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af84950 0x55f59af86d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.710+0000 7fdce8752740 1 --2- 172.21.15.78:0/2792051962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af789c0 0x55f59af84410 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.710+0000 7fdce8752740 1 --2- 172.21.15.78:0/2792051962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af780a0 0x55f59af78480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.710+0000 7fdce8752740 1 -- 172.21.15.78:0/2792051962 >> 172.21.15.78:0/2792051962 conn(0x55f59af75ed0 msgr2=0x55f59af77070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.710+0000 7fdce8752740 1 -- 172.21.15.78:0/2792051962 shutdown_connections 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.710+0000 7fdce8752740 1 -- 172.21.15.78:0/2792051962 wait complete. 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.710+0000 7fdce8752740 1 -- start start 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdce8752740 1 Processor -- start 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdce8752740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 0x55f59af841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdce8752740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 0x55f59af81610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdce8752740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af84950 0x55f59af81b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdce8752740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f59ad3f700 con 0x55f59af789c0 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdce8752740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f59ae79110 con 0x55f59af84950 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdce8752740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f59af875d0 con 0x55f59af780a0 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdce8752740 10 client.0 _ll_register_callbacks cb 0x55f59ae92990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdcdf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 0x55f59af81610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 0x55f59af841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdce491a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af84950 0x55f59af81b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 0x55f59af841c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:33754/0 (socket says 172.21.15.78:33754) 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdcdffff640 1 -- 172.21.15.78:0/2288076626 learned_addr learned my addr 172.21.15.78:0/2288076626 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdcdf7fe640 1 -- 172.21.15.78:0/2288076626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 msgr2=0x55f59af841c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdcdf7fe640 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 0x55f59af841c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdcdf7fe640 1 -- 172.21.15.78:0/2288076626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af84950 msgr2=0x55f59af81b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdcdf7fe640 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af84950 0x55f59af81b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdcdf7fe640 1 -- 172.21.15.78:0/2288076626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f59aea6c40 con 0x55f59af789c0 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdcdffff640 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 0x55f59af841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.711+0000 7fdce491a640 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af84950 0x55f59af81b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdcbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdcdf7fe640 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 0x55f59af81610 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40126d0 tx=0x7fdcd4012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312214f82495ad62 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdcbf7fe640 1 -- 172.21.15.78:0/2288076626 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400f8c0 con 0x55f59af789c0 2024-09-11T01:04:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdcbf7fe640 1 -- 172.21.15.78:0/2288076626 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdcd400fa60 con 0x55f59af789c0 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdce8752740 1 -- 172.21.15.78:0/2288076626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd00acf70 con 0x55f59af789c0 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdcbf7fe640 1 -- 172.21.15.78:0/2288076626 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdcd400fc60 con 0x55f59af789c0 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdce8752740 10 client.16572 fetch_fsmap learned FSMap version 39 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdce8752740 1 -- 172.21.15.78:0/2288076626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f59af92ee0 con 0x55f59af789c0 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdcbf7fe640 1 -- 172.21.15.78:0/2288076626 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd00acf70 con 0x55f59af789c0 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdcbf7fe640 1 -- 172.21.15.78:0/2288076626 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8382+0+0 (secure 0 0 0) 0x7fdcd4024050 con 0x55f59af789c0 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdce8752740 10 client.16572 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdce8752740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdce8752740 20 client.16572 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.712+0000 7fdce8752740 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f59af94400 0x55f59af96850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.713+0000 7fdce8752740 4 client.16572 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.713+0000 7fdce8752740 1 -- 172.21.15.78:0/2288076626 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f59aef0da0 con 0x55f59af94400 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.713+0000 7fdcdffff640 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f59af94400 0x55f59af96850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.713+0000 7fdcbf7fe640 10 client.16572 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:28.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.713+0000 7fdcdffff640 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f59af94400 0x55f59af96850 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=659aabbf849a5452 server_cookie=191fd96a0f767210 in_seq=0 out_seq=0 2024-09-11T01:04:28.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.714+0000 7fdcbf7fe640 1 -- 172.21.15.78:0/2288076626 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f59aef0da0 con 0x55f59af94400 2024-09-11T01:04:28.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.714+0000 7fdcbf7fe640 10 client.16572 handle_command_reply: tid=0 2024-09-11T01:04:28.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.714+0000 7fdce8752740 1 client.16572 shutdown 2024-09-11T01:04:28.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.714+0000 7fdce8752740 1 -- 172.21.15.78:0/2288076626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 msgr2=0x55f59af81610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:28.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.714+0000 7fdce8752740 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 0x55f59af81610 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40126d0 tx=0x7fdcd4012b90 comp rx=0 tx=0).stop 2024-09-11T01:04:28.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.714+0000 7fdce8752740 1 -- 172.21.15.78:0/2288076626 shutdown_connections 2024-09-11T01:04:28.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.714+0000 7fdce8752740 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af84950 0x55f59af81b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.714+0000 7fdce8752740 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f59af94400 0x55f59af96850 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.714+0000 7fdce8752740 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 0x55f59af81610 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.714+0000 7fdce8752740 1 --2- 172.21.15.78:0/2288076626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 0x55f59af841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.714+0000 7fdce8752740 1 -- 172.21.15.78:0/2288076626 >> 172.21.15.78:0/2288076626 conn(0x55f59af75ed0 msgr2=0x55f59af7ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:28.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.714+0000 7fdcbf7fe640 0 client.16572 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:28.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.715+0000 7fdce8752740 1 -- 172.21.15.78:0/2288076626 shutdown_connections 2024-09-11T01:04:28.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.715+0000 7fdce8752740 1 -- 172.21.15.78:0/2288076626 wait complete. 2024-09-11T01:04:28.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.715+0000 7fdce8752740 20 client.16572 trim_cache size 0 max 16384 2024-09-11T01:04:28.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:04:28.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:04:28.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:04:28.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:04:28.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:04:28.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:04:28.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:04:28.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:04:28.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:04:28.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:04:28.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:04:28.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:28 smithi112 ceph-mon[35384]: pgmap v491: 97 pgs: 97 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-09-11T01:04:28.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdce8752740 1 Processor -- start 2024-09-11T01:04:28.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdce8752740 1 -- start start 2024-09-11T01:04:28.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdce8752740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af780a0 0x55f59af90800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdce8752740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 0x55f59af772c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdce8752740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af84950 0x55f59af85d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdce8752740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01b06f0 con 0x55f59af789c0 2024-09-11T01:04:28.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdce8752740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01b2320 con 0x55f59af780a0 2024-09-11T01:04:28.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdce8752740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01b0dc0 con 0x55f59af84950 2024-09-11T01:04:28.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdcdf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 0x55f59af772c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdcdf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 0x55f59af772c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45070/0 (socket says 172.21.15.78:45070) 2024-09-11T01:04:28.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdcdf7fe640 1 -- 172.21.15.78:0/2423200527 learned_addr learned my addr 172.21.15.78:0/2423200527 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:28.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdce491a640 1 --2- 172.21.15.78:0/2423200527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af84950 0x55f59af85d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.732+0000 7fdcdffff640 1 --2- 172.21.15.78:0/2423200527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af780a0 0x55f59af90800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.733+0000 7fdcdf7fe640 1 -- 172.21.15.78:0/2423200527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af84950 msgr2=0x55f59af85d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:28.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.733+0000 7fdcdf7fe640 1 --2- 172.21.15.78:0/2423200527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af84950 0x55f59af85d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.733+0000 7fdcdf7fe640 1 -- 172.21.15.78:0/2423200527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af780a0 msgr2=0x55f59af90800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.733+0000 7fdcdf7fe640 1 --2- 172.21.15.78:0/2423200527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af780a0 0x55f59af90800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.733+0000 7fdcdf7fe640 1 -- 172.21.15.78:0/2423200527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f59af7fb10 con 0x55f59af789c0 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.733+0000 7fdcdf7fe640 1 --2- 172.21.15.78:0/2423200527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 0x55f59af772c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009e40 tx=0x7fdcd400ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1482af06cf0bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.733+0000 7fdcc5ffb640 1 -- 172.21.15.78:0/2423200527 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd402a070 con 0x55f59af789c0 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.733+0000 7fdcc5ffb640 1 -- 172.21.15.78:0/2423200527 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd4005560 con 0x55f59af789c0 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.733+0000 7fdcc5ffb640 1 -- 172.21.15.78:0/2423200527 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40039e0 con 0x55f59af789c0 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.734+0000 7fdce8752740 1 -- 172.21.15.78:0/2423200527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 msgr2=0x55f59af772c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.734+0000 7fdce8752740 1 --2- 172.21.15.78:0/2423200527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 0x55f59af772c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009e40 tx=0x7fdcd400ff90 comp rx=0 tx=0).stop 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.734+0000 7fdce8752740 1 -- 172.21.15.78:0/2423200527 shutdown_connections 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.734+0000 7fdce8752740 1 --2- 172.21.15.78:0/2423200527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af84950 0x55f59af85d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.734+0000 7fdce8752740 1 --2- 172.21.15.78:0/2423200527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af789c0 0x55f59af772c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.734+0000 7fdce8752740 1 --2- 172.21.15.78:0/2423200527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af780a0 0x55f59af90800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.734+0000 7fdce8752740 1 -- 172.21.15.78:0/2423200527 >> 172.21.15.78:0/2423200527 conn(0x55f59af75ed0 msgr2=0x55f59af76bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:28.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.734+0000 7fdce8752740 1 -- 172.21.15.78:0/2423200527 shutdown_connections 2024-09-11T01:04:28.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.734+0000 7fdce8752740 1 -- 172.21.15.78:0/2423200527 wait complete. 2024-09-11T01:04:28.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdce8752740 1 -- start start 2024-09-11T01:04:28.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdce8752740 1 Processor -- start 2024-09-11T01:04:28.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdce8752740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 0x55f59af9bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdce8752740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af789c0 0x55f59af9c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdce8752740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af84950 0x55f59af9ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdce8752740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01b1b90 con 0x55f59af84950 2024-09-11T01:04:28.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdce8752740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01b1490 con 0x55f59af789c0 2024-09-11T01:04:28.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdce8752740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01ae6d0 con 0x55f59af780a0 2024-09-11T01:04:28.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdce8752740 10 client.0 _ll_register_callbacks cb 0x7fdcd4009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:04:28.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 0x55f59af9bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdcdf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af789c0 0x55f59af9c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 0x55f59af9bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:33768/0 (socket says 172.21.15.78:33768) 2024-09-11T01:04:28.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdce491a640 1 --2- 172.21.15.78:0/1412554748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af84950 0x55f59af9ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.735+0000 7fdcdffff640 1 -- 172.21.15.78:0/1412554748 learned_addr learned my addr 172.21.15.78:0/1412554748 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:04:28.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdcdffff640 1 -- 172.21.15.78:0/1412554748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af789c0 msgr2=0x55f59af9c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:28.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdcdffff640 1 --2- 172.21.15.78:0/1412554748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af789c0 0x55f59af9c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdcdffff640 1 -- 172.21.15.78:0/1412554748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af84950 msgr2=0x55f59af9ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdcdffff640 1 --2- 172.21.15.78:0/1412554748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af84950 0x55f59af9ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdcdffff640 1 -- 172.21.15.78:0/1412554748 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f59af8bf40 con 0x55f59af780a0 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdcdffff640 1 --2- 172.21.15.78:0/1412554748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 0x55f59af9bfe0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0005820 tx=0x7fdcd00a8710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772e864f422f43fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdcbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdcbf7fe640 1 -- 172.21.15.78:0/1412554748 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0005680 con 0x55f59af780a0 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdcbf7fe640 1 -- 172.21.15.78:0/1412554748 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdcd00a8ae0 con 0x55f59af780a0 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdce8752740 1 -- 172.21.15.78:0/1412554748 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd4005560 con 0x55f59af780a0 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdcbf7fe640 1 -- 172.21.15.78:0/1412554748 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdcd00ac1c0 con 0x55f59af780a0 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdce8752740 10 client.25876 fetch_fsmap learned FSMap version 39 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdce8752740 1 -- 172.21.15.78:0/1412554748 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f59b001650 con 0x55f59af780a0 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdcbf7fe640 1 -- 172.21.15.78:0/1412554748 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd4005560 con 0x55f59af780a0 2024-09-11T01:04:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdcbf7fe640 1 -- 172.21.15.78:0/1412554748 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8382+0+0 (secure 0 0 0) 0x7fdcd00e3dc0 con 0x55f59af780a0 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.736+0000 7fdce8752740 10 client.25876 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.737+0000 7fdce8752740 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.737+0000 7fdce8752740 20 client.25876 populate_metadata read hostname 'smithi078' 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.737+0000 7fdce8752740 1 --2- 172.21.15.78:0/1412554748 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f59b001e30 0x55f59b004220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.737+0000 7fdce8752740 4 client.25876 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.737+0000 7fdce8752740 1 -- 172.21.15.78:0/1412554748 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f59aeec850 con 0x55f59b001e30 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.737+0000 7fdcdf7fe640 1 --2- 172.21.15.78:0/1412554748 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f59b001e30 0x55f59b004220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.737+0000 7fdcdf7fe640 1 --2- 172.21.15.78:0/1412554748 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f59b001e30 0x55f59b004220 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c6609df8cb1d76 server_cookie=203049cb83b91ba4 in_seq=0 out_seq=0 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.737+0000 7fdcbf7fe640 10 client.25876 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.737+0000 7fdcbf7fe640 1 -- 172.21.15.78:0/1412554748 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x55f59aeec850 con 0x55f59b001e30 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.737+0000 7fdcbf7fe640 10 client.25876 handle_command_reply: tid=0 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.737+0000 7fdce8752740 1 client.25876 shutdown 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.738+0000 7fdce8752740 1 -- 172.21.15.78:0/1412554748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 msgr2=0x55f59af9bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.738+0000 7fdce8752740 1 --2- 172.21.15.78:0/1412554748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 0x55f59af9bfe0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0005820 tx=0x7fdcd00a8710 comp rx=0 tx=0).stop 2024-09-11T01:04:28.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.738+0000 7fdce8752740 1 -- 172.21.15.78:0/1412554748 shutdown_connections 2024-09-11T01:04:28.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.738+0000 7fdce8752740 1 --2- 172.21.15.78:0/1412554748 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f59b001e30 0x55f59b004220 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.738+0000 7fdcbf7fe640 0 client.25876 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:04:28.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.738+0000 7fdce8752740 1 --2- 172.21.15.78:0/1412554748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f59af84950 0x55f59af9ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.738+0000 7fdce8752740 1 --2- 172.21.15.78:0/1412554748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f59af789c0 0x55f59af9c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.738+0000 7fdce8752740 1 --2- 172.21.15.78:0/1412554748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f59af780a0 0x55f59af9bfe0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.738+0000 7fdce8752740 1 -- 172.21.15.78:0/1412554748 >> 172.21.15.78:0/1412554748 conn(0x55f59af75ed0 msgr2=0x55f59af83460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:28.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.738+0000 7fdce8752740 1 -- 172.21.15.78:0/1412554748 shutdown_connections 2024-09-11T01:04:28.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.738+0000 7fdce8752740 1 -- 172.21.15.78:0/1412554748 wait complete. 2024-09-11T01:04:28.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.738+0000 7fdce8752740 20 client.25876 trim_cache size 0 max 16384 2024-09-11T01:04:28.746 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e", 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:28.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:28.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:28.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:28.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:28.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:28.748 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:04:28.748 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:04:28.748 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:28.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:28.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:04:28.749 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:04:28.750 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:28.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:04:28.752 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:04:28.752 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:04:28.752 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:04:28.752 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:04:28.752 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:04:28.752 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:04:28.752 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:04:28.752 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:04:28.752 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:04:28.752 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:04:28.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.739+0000 7fdce637d640 1 -- 172.21.15.78:0/3626442969 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdcc80b5450 msgr2=0x7fdcc80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:28.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.739+0000 7fdce637d640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdcc80b5450 0x7fdcc80b7910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8005fe0 tx=0x7fdcd8005ef0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.739+0000 7fdce637d640 1 -- 172.21.15.78:0/3626442969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce0101c90 msgr2=0x7fdce00fde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:04:28.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.739+0000 7fdce637d640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce0101c90 0x7fdce00fde80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000e9f0 tx=0x7fdcd000edf0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.739+0000 7fdce637d640 1 -- 172.21.15.78:0/3626442969 shutdown_connections 2024-09-11T01:04:28.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.739+0000 7fdce637d640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdcc80b5450 0x7fdcc80b7910 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.739+0000 7fdce637d640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce0108020 0x7fdce00fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.739+0000 7fdce637d640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdce01076a0 0x7fdce00fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.739+0000 7fdce637d640 1 --2- 172.21.15.78:0/3626442969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdce0101c90 0x7fdce00fde80 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:04:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.739+0000 7fdce637d640 1 -- 172.21.15.78:0/3626442969 >> 172.21.15.78:0/3626442969 conn(0x7fdce00fd700 msgr2=0x7fdce0105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:04:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.739+0000 7fdce637d640 1 -- 172.21.15.78:0/3626442969 shutdown_connections 2024-09-11T01:04:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:04:28.739+0000 7fdce637d640 1 -- 172.21.15.78:0/3626442969 wait complete. 2024-09-11T01:04:29.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 430914 5.97 MB/sec execute 157 sec latency 213.081 ms 2024-09-11T01:04:29.641 INFO:tasks.cephfs.filesystem:scrub status for tag:ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e - {'path': '/', 'tag': 'ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e', 'options': 'recursive,force'} 2024-09-11T01:04:30.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 430914 5.93 MB/sec execute 158 sec latency 1213.109 ms 2024-09-11T01:04:30.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:30 smithi078 ceph-mon[31315]: pgmap v492: 97 pgs: 97 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-11T01:04:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:30 smithi110 ceph-mon[34712]: pgmap v492: 97 pgs: 97 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-11T01:04:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:30 smithi112 ceph-mon[35384]: pgmap v492: 97 pgs: 97 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-11T01:04:31.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 430914 5.89 MB/sec execute 159 sec latency 2213.482 ms 2024-09-11T01:04:32.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 431611 5.87 MB/sec execute 160 sec latency 2974.001 ms 2024-09-11T01:04:32.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:32 smithi078 ceph-mon[31315]: pgmap v493: 97 pgs: 97 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-11T01:04:32.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:32 smithi110 ceph-mon[34712]: pgmap v493: 97 pgs: 97 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-11T01:04:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:32 smithi112 ceph-mon[35384]: pgmap v493: 97 pgs: 97 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-11T01:04:33.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 433323 5.91 MB/sec execute 161 sec latency 29.060 ms 2024-09-11T01:04:34.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 434470 5.90 MB/sec execute 162 sec latency 192.078 ms 2024-09-11T01:04:34.491 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:34 smithi110 ceph-mon[34712]: pgmap v494: 97 pgs: 97 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-09-11T01:04:34.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:34 smithi078 ceph-mon[31315]: pgmap v494: 97 pgs: 97 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-09-11T01:04:34.525 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:34 smithi112 ceph-mon[35384]: pgmap v494: 97 pgs: 97 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-09-11T01:04:35.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 436701 5.95 MB/sec execute 163 sec latency 19.287 ms 2024-09-11T01:04:36.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 438005 5.95 MB/sec execute 164 sec latency 54.887 ms 2024-09-11T01:04:36.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:36 smithi078 ceph-mon[31315]: pgmap v495: 97 pgs: 97 active+clean; 261 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-09-11T01:04:36.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:36 smithi110 ceph-mon[34712]: pgmap v495: 97 pgs: 97 active+clean; 261 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-09-11T01:04:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:36 smithi112 ceph-mon[35384]: pgmap v495: 97 pgs: 97 active+clean; 261 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-09-11T01:04:37.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 438721 5.92 MB/sec execute 165 sec latency 811.506 ms 2024-09-11T01:04:38.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 440467 5.96 MB/sec execute 166 sec latency 10.829 ms 2024-09-11T01:04:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:38 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:04:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:38 smithi078 ceph-mon[31315]: pgmap v496: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-09-11T01:04:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:38 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:04:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:38 smithi110 ceph-mon[34712]: pgmap v496: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-09-11T01:04:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:38 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:04:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:38 smithi112 ceph-mon[35384]: pgmap v496: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-09-11T01:04:39.432 INFO:tasks.workunit.client.0.smithi078.stdout: 1 441547 5.95 MB/sec execute 167 sec latency 287.671 ms 2024-09-11T01:04:40.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 441547 5.92 MB/sec execute 168 sec latency 1287.767 ms 2024-09-11T01:04:40.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:40 smithi078 ceph-mon[31315]: pgmap v497: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-09-11T01:04:40.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:40 smithi110 ceph-mon[34712]: pgmap v497: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-09-11T01:04:40.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:40 smithi112 ceph-mon[35384]: pgmap v497: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-09-11T01:04:41.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 441547 5.88 MB/sec execute 169 sec latency 2287.799 ms 2024-09-11T01:04:42.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 443194 5.90 MB/sec execute 170 sec latency 2681.888 ms 2024-09-11T01:04:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:42 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:04:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:42 smithi112 ceph-mon[35384]: pgmap v498: 97 pgs: 97 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-09-11T01:04:42.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:42 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:04:42.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:42 smithi078 ceph-mon[31315]: pgmap v498: 97 pgs: 97 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-09-11T01:04:42.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:42 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:04:42.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:42 smithi110 ceph-mon[34712]: pgmap v498: 97 pgs: 97 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-09-11T01:04:43.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 444814 5.93 MB/sec execute 171 sec latency 11.345 ms 2024-09-11T01:04:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:43 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:04:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:43 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:04:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:43 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:04:43.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:43 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:04:43.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:43 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:04:43.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:43 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:04:43.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:43 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:04:43.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:43 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:04:43.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:43 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:04:43.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:43 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:04:43.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:43 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:04:43.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:43 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:04:44.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 445096 5.89 MB/sec execute 172 sec latency 841.233 ms 2024-09-11T01:04:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:44 smithi112 ceph-mon[35384]: pgmap v499: 97 pgs: 97 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-09-11T01:04:44.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:44 smithi078 ceph-mon[31315]: pgmap v499: 97 pgs: 97 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-09-11T01:04:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:44 smithi110 ceph-mon[34712]: pgmap v499: 97 pgs: 97 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-09-11T01:04:45.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 445096 5.86 MB/sec execute 173 sec latency 1841.340 ms 2024-09-11T01:04:46.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 445096 5.82 MB/sec execute 174 sec latency 2841.449 ms 2024-09-11T01:04:46.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:46 smithi078 ceph-mon[31315]: pgmap v500: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-11T01:04:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:46 smithi110 ceph-mon[34712]: pgmap v500: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-11T01:04:46.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:46 smithi112 ceph-mon[35384]: pgmap v500: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-11T01:04:47.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 445806 5.80 MB/sec execute 175 sec latency 3600.717 ms 2024-09-11T01:04:48.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 447522 5.84 MB/sec execute 176 sec latency 9.505 ms 2024-09-11T01:04:48.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:48 smithi078 ceph-mon[31315]: pgmap v501: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-09-11T01:04:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:48 smithi110 ceph-mon[34712]: pgmap v501: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-09-11T01:04:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:48 smithi112 ceph-mon[35384]: pgmap v501: 97 pgs: 97 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-09-11T01:04:49.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 448638 5.83 MB/sec execute 177 sec latency 449.237 ms 2024-09-11T01:04:50.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 448638 5.80 MB/sec execute 178 sec latency 1449.369 ms 2024-09-11T01:04:50.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:50 smithi078 ceph-mon[31315]: pgmap v502: 97 pgs: 97 active+clean; 271 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-09-11T01:04:50.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:50 smithi110 ceph-mon[34712]: pgmap v502: 97 pgs: 97 active+clean; 271 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-09-11T01:04:50.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:50 smithi112 ceph-mon[35384]: pgmap v502: 97 pgs: 97 active+clean; 271 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-09-11T01:04:51.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 448638 5.77 MB/sec execute 179 sec latency 2449.468 ms 2024-09-11T01:04:52.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 449347 5.75 MB/sec execute 180 sec latency 3208.524 ms 2024-09-11T01:04:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:52 smithi078 ceph-mon[31315]: pgmap v503: 97 pgs: 97 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-11T01:04:52.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:52 smithi110 ceph-mon[34712]: pgmap v503: 97 pgs: 97 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-11T01:04:52.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:52 smithi112 ceph-mon[35384]: pgmap v503: 97 pgs: 97 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-11T01:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout: 1 451088 5.78 MB/sec execute 181 sec latency 13.126 ms 2024-09-11T01:04:54.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 452199 5.78 MB/sec execute 182 sec latency 409.002 ms 2024-09-11T01:04:54.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:54 smithi078 ceph-mon[31315]: pgmap v504: 97 pgs: 97 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-09-11T01:04:54.693 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:54 smithi110 ceph-mon[34712]: pgmap v504: 97 pgs: 97 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-09-11T01:04:54.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:54 smithi112 ceph-mon[35384]: pgmap v504: 97 pgs: 97 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-09-11T01:04:55.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 454400 5.82 MB/sec execute 183 sec latency 22.189 ms 2024-09-11T01:04:56.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 455730 5.82 MB/sec execute 184 sec latency 140.217 ms 2024-09-11T01:04:56.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:56 smithi078 ceph-mon[31315]: pgmap v505: 97 pgs: 97 active+clean; 273 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-11T01:04:56.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:56 smithi110 ceph-mon[34712]: pgmap v505: 97 pgs: 97 active+clean; 273 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-11T01:04:56.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:56 smithi112 ceph-mon[35384]: pgmap v505: 97 pgs: 97 active+clean; 273 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-11T01:04:57.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 456446 5.80 MB/sec execute 185 sec latency 898.305 ms 2024-09-11T01:04:58.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 458120 5.83 MB/sec execute 186 sec latency 12.303 ms 2024-09-11T01:04:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:04:58 smithi078 ceph-mon[31315]: pgmap v506: 97 pgs: 97 active+clean; 273 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-09-11T01:04:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:04:58 smithi110 ceph-mon[34712]: pgmap v506: 97 pgs: 97 active+clean; 273 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-09-11T01:04:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:04:58 smithi112 ceph-mon[35384]: pgmap v506: 97 pgs: 97 active+clean; 273 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-09-11T01:04:59.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 459113 5.92 MB/sec execute 187 sec latency 50.627 ms 2024-09-11T01:04:59.642 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:04:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:00.402 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:00 smithi078 ceph-mon[31315]: pgmap v507: 97 pgs: 97 active+clean; 278 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-09-11T01:05:00.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 460030 5.91 MB/sec execute 188 sec latency 503.760 ms 2024-09-11T01:05:00.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.496+0000 7fe95b776640 1 -- 172.21.15.78:0/1814759182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe954101cd0 msgr2=0x7fe9541020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:00.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.496+0000 7fe95b776640 1 --2- 172.21.15.78:0/1814759182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe954101cd0 0x7fe9541020b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe950009900 tx=0x7fe95002a410 comp rx=0 tx=0).stop 2024-09-11T01:05:00.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.496+0000 7fe95b776640 1 -- 172.21.15.78:0/1814759182 shutdown_connections 2024-09-11T01:05:00.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.496+0000 7fe95b776640 1 --2- 172.21.15.78:0/1814759182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954108060 0x7fe95410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.496+0000 7fe95b776640 1 --2- 172.21.15.78:0/1814759182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9541076e0 0x7fe954107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.496+0000 7fe95b776640 1 --2- 172.21.15.78:0/1814759182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe954101cd0 0x7fe9541020b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.496+0000 7fe95b776640 1 -- 172.21.15.78:0/1814759182 >> 172.21.15.78:0/1814759182 conn(0x7fe9540fd700 msgr2=0x7fe9540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:00.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.496+0000 7fe95b776640 1 -- 172.21.15.78:0/1814759182 shutdown_connections 2024-09-11T01:05:00.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.496+0000 7fe95b776640 1 -- 172.21.15.78:0/1814759182 wait complete. 2024-09-11T01:05:00.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.497+0000 7fe95b776640 1 Processor -- start 2024-09-11T01:05:00.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.497+0000 7fe95b776640 1 -- start start 2024-09-11T01:05:00.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.497+0000 7fe95b776640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954101cd0 0x7fe954198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.497+0000 7fe95b776640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9541076e0 0x7fe9541986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.497+0000 7fe95b776640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe954108060 0x7fe95419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.497+0000 7fe95b776640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe954111d20 con 0x7fe954108060 2024-09-11T01:05:00.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.497+0000 7fe95b776640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe954111ba0 con 0x7fe9541076e0 2024-09-11T01:05:00.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.497+0000 7fe95b776640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe954111ea0 con 0x7fe954101cd0 2024-09-11T01:05:00.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe959d13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe954108060 0x7fe95419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe959d13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe954108060 0x7fe95419da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51492/0 (socket says 172.21.15.78:51492) 2024-09-11T01:05:00.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe959d13640 1 -- 172.21.15.78:0/2479675702 learned_addr learned my addr 172.21.15.78:0/2479675702 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:00.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe959512640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954101cd0 0x7fe954198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe958d11640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9541076e0 0x7fe9541986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe959d13640 1 -- 172.21.15.78:0/2479675702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954101cd0 msgr2=0x7fe954198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:00.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe959d13640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954101cd0 0x7fe954198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe959d13640 1 -- 172.21.15.78:0/2479675702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9541076e0 msgr2=0x7fe9541986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe959d13640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9541076e0 0x7fe9541986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe959d13640 1 -- 172.21.15.78:0/2479675702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95419e240 con 0x7fe954108060 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe959d13640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe954108060 0x7fe95419da70 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00eb10 tx=0x7fe94c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe94a7fc640 1 -- 172.21.15.78:0/2479675702 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c004b20 con 0x7fe954108060 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe94a7fc640 1 -- 172.21.15.78:0/2479675702 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94c004cc0 con 0x7fe954108060 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe95b776640 1 -- 172.21.15.78:0/2479675702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95419e530 con 0x7fe954108060 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe94a7fc640 1 -- 172.21.15.78:0/2479675702 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c010670 con 0x7fe954108060 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.498+0000 7fe95b776640 1 -- 172.21.15.78:0/2479675702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9541a4dc0 con 0x7fe954108060 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.501+0000 7fe95db4b740 1 Processor -- start 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.501+0000 7fe94a7fc640 1 -- 172.21.15.78:0/2479675702 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe94c005020 con 0x7fe954108060 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.501+0000 7fe95db4b740 1 -- start start 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.501+0000 7fe95db4b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 0x55e6c5083480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.501+0000 7fe95db4b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50839c0 0x55e6c5091620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.501+0000 7fe95db4b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c5091b60 0x55e6c5093f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.501+0000 7fe95db4b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e6c4ff9e50 con 0x55e6c5091b60 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.501+0000 7fe95db4b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e6c4fe9280 con 0x55e6c50839c0 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.501+0000 7fe95db4b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e6c4fed090 con 0x55e6c50830a0 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.501+0000 7fe958d11640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c5091b60 0x55e6c5093f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe958d11640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 0x55e6c5083480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe959512640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50839c0 0x55e6c5091620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe959512640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50839c0 0x55e6c5091620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43766/0 (socket says 172.21.15.78:43766) 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe958d11640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 0x55e6c5083480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:55834/0 (socket says 172.21.15.78:55834) 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe959512640 1 -- 172.21.15.78:0/2026443037 learned_addr learned my addr 172.21.15.78:0/2026443037 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe959512640 1 -- 172.21.15.78:0/2026443037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 msgr2=0x55e6c5083480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe959512640 1 --2- 172.21.15.78:0/2026443037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 0x55e6c5083480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe959512640 1 -- 172.21.15.78:0/2026443037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c5091b60 msgr2=0x55e6c5093f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe959512640 1 --2- 172.21.15.78:0/2026443037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c5091b60 0x55e6c5093f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe959512640 1 -- 172.21.15.78:0/2026443037 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c4d47650 con 0x55e6c50839c0 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe94a7fc640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe9400b54d0 0x7fe9400b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe959512640 1 --2- 172.21.15.78:0/2026443037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50839c0 0x55e6c5091620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe950009900 tx=0x7fe950032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d701b360054b66c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe958d11640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe9400b54d0 0x7fe9400b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe93f7fe640 1 -- 172.21.15.78:0/2026443037 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95003f070 con 0x55e6c50839c0 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe94a7fc640 1 -- 172.21.15.78:0/2479675702 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe94c0e6540 con 0x7fe954108060 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe93f7fe640 1 -- 172.21.15.78:0/2026443037 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe950004030 con 0x55e6c50839c0 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.502+0000 7fe93f7fe640 1 -- 172.21.15.78:0/2026443037 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950035070 con 0x55e6c50839c0 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.503+0000 7fe95db4b740 1 -- 172.21.15.78:0/2026443037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50839c0 msgr2=0x55e6c5091620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.503+0000 7fe958d11640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe9400b54d0 0x7fe9400b7990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55e6c5083050 tx=0x7fe944004730 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.503+0000 7fe95db4b740 1 --2- 172.21.15.78:0/2026443037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50839c0 0x55e6c5091620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe950009900 tx=0x7fe950032b90 comp rx=0 tx=0).stop 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.503+0000 7fe95db4b740 1 -- 172.21.15.78:0/2026443037 shutdown_connections 2024-09-11T01:05:00.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.503+0000 7fe95db4b740 1 --2- 172.21.15.78:0/2026443037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c5091b60 0x55e6c5093f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.503+0000 7fe95db4b740 1 --2- 172.21.15.78:0/2026443037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50839c0 0x55e6c5091620 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.503+0000 7fe95db4b740 1 --2- 172.21.15.78:0/2026443037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 0x55e6c5083480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.503+0000 7fe95db4b740 1 -- 172.21.15.78:0/2026443037 >> 172.21.15.78:0/2026443037 conn(0x55e6c5080ed0 msgr2=0x55e6c5082070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.503+0000 7fe95db4b740 1 -- 172.21.15.78:0/2026443037 shutdown_connections 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.503+0000 7fe95db4b740 1 -- 172.21.15.78:0/2026443037 wait complete. 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.503+0000 7fe95db4b740 1 -- start start 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.503+0000 7fe95db4b740 1 Processor -- start 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe95db4b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50830a0 0x55e6c5091400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe95db4b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50839c0 0x55e6c508e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe95db4b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c5091b60 0x55e6c508ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe95db4b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e6c4e4a700 con 0x55e6c50830a0 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe95db4b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e6c4f84110 con 0x55e6c5091b60 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe95db4b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e6c50947b0 con 0x55e6c50839c0 2024-09-11T01:05:00.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe95db4b740 10 client.0 _ll_register_callbacks cb 0x55e6c4f9d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:00.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe959512640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50830a0 0x55e6c5091400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe959512640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50830a0 0x55e6c5091400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51522/0 (socket says 172.21.15.78:51522) 2024-09-11T01:05:00.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe959512640 1 -- 172.21.15.78:0/3469153913 learned_addr learned my addr 172.21.15.78:0/3469153913 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:00.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe959d13640 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c5091b60 0x55e6c508ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe958d11640 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50839c0 0x55e6c508e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe959512640 1 -- 172.21.15.78:0/3469153913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50839c0 msgr2=0x55e6c508e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:00.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe959512640 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50839c0 0x55e6c508e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe959512640 1 -- 172.21.15.78:0/3469153913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c5091b60 msgr2=0x55e6c508ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:00.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe959512640 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c5091b60 0x55e6c508ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe959512640 1 -- 172.21.15.78:0/3469153913 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c4fb1c40 con 0x55e6c50830a0 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.504+0000 7fe958d11640 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50839c0 0x55e6c508e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe959512640 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50830a0 0x55e6c5091400 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe9500096c0 tx=0x7fe95002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8a0bb899e76c61 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe93cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe93cff9640 1 -- 172.21.15.78:0/3469153913 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950035070 con 0x55e6c50830a0 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe93cff9640 1 -- 172.21.15.78:0/3469153913 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe9500314a0 con 0x55e6c50830a0 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe95db4b740 1 -- 172.21.15.78:0/3469153913 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe950004030 con 0x55e6c50830a0 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe93cff9640 1 -- 172.21.15.78:0/3469153913 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe950055050 con 0x55e6c50830a0 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe95db4b740 10 client.16590 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe95db4b740 1 -- 172.21.15.78:0/3469153913 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e6c50a00a0 con 0x55e6c50830a0 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe93cff9640 1 -- 172.21.15.78:0/3469153913 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe950004030 con 0x55e6c50830a0 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe93cff9640 1 -- 172.21.15.78:0/3469153913 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fe9500035f0 con 0x55e6c50830a0 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe95db4b740 10 client.16590 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:00.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe95db4b740 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe95db4b740 20 client.16590 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe95db4b740 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e6c50a13f0 0x55e6c50a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe95db4b740 4 client.16590 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.505+0000 7fe95db4b740 1 -- 172.21.15.78:0/3469153913 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6c4ffbda0 con 0x55e6c50a13f0 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.506+0000 7fe958d11640 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e6c50a13f0 0x55e6c50a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.506+0000 7fe958d11640 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e6c50a13f0 0x55e6c50a37e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be450dd0560a8021 server_cookie=a091512ac68ce7ad in_seq=0 out_seq=0 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.506+0000 7fe93cff9640 10 client.16590 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.506+0000 7fe93cff9640 1 -- 172.21.15.78:0/3469153913 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e6c4ffbda0 con 0x55e6c50a13f0 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.506+0000 7fe93cff9640 10 client.16590 handle_command_reply: tid=0 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe95db4b740 1 client.16590 shutdown 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe95db4b740 1 -- 172.21.15.78:0/3469153913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50830a0 msgr2=0x55e6c5091400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe95db4b740 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50830a0 0x55e6c5091400 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe9500096c0 tx=0x7fe95002a410 comp rx=0 tx=0).stop 2024-09-11T01:05:00.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe95db4b740 1 -- 172.21.15.78:0/3469153913 shutdown_connections 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe95db4b740 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c5091b60 0x55e6c508ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe95db4b740 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50839c0 0x55e6c508e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe95db4b740 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e6c50a13f0 0x55e6c50a37e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe95db4b740 1 --2- 172.21.15.78:0/3469153913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50830a0 0x55e6c5091400 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe95db4b740 1 -- 172.21.15.78:0/3469153913 >> 172.21.15.78:0/3469153913 conn(0x55e6c5080ed0 msgr2=0x55e6c5086a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe93cff9640 0 client.16590 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe95db4b740 1 -- 172.21.15.78:0/3469153913 shutdown_connections 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe95db4b740 1 -- 172.21.15.78:0/3469153913 wait complete. 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.507+0000 7fe95db4b740 20 client.16590 trim_cache size 0 max 16384 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.526+0000 7fe95db4b740 1 Processor -- start 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.526+0000 7fe95db4b740 1 -- start start 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe95db4b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50830a0 0x55e6c509d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe95db4b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50839c0 0x55e6c508dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe95db4b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c5091b60 0x55e6c5092950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe95db4b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541ac390 con 0x55e6c50839c0 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe95db4b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541adf90 con 0x55e6c50830a0 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe95db4b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541aca60 con 0x55e6c5091b60 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe958d11640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50839c0 0x55e6c508dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe958d11640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50839c0 0x55e6c508dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51538/0 (socket says 172.21.15.78:51538) 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe958d11640 1 -- 172.21.15.78:0/4004770806 learned_addr learned my addr 172.21.15.78:0/4004770806 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe959d13640 1 --2- 172.21.15.78:0/4004770806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c5091b60 0x55e6c5092950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe959512640 1 --2- 172.21.15.78:0/4004770806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50830a0 0x55e6c509d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe958d11640 1 -- 172.21.15.78:0/4004770806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c5091b60 msgr2=0x55e6c5092950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe958d11640 1 --2- 172.21.15.78:0/4004770806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c5091b60 0x55e6c5092950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe958d11640 1 -- 172.21.15.78:0/4004770806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50830a0 msgr2=0x55e6c509d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe958d11640 1 --2- 172.21.15.78:0/4004770806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50830a0 0x55e6c509d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe958d11640 1 -- 172.21.15.78:0/4004770806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c50837f0 con 0x55e6c50839c0 2024-09-11T01:05:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe959512640 1 --2- 172.21.15.78:0/4004770806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50830a0 0x55e6c509d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe959d13640 1 --2- 172.21.15.78:0/4004770806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c5091b60 0x55e6c5092950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe958d11640 1 --2- 172.21.15.78:0/4004770806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50839c0 0x55e6c508dd40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe944005fd0 tx=0x7fe944016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baedad3f285e5fca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.527+0000 7fe93effd640 1 -- 172.21.15.78:0/4004770806 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944010660 con 0x55e6c50839c0 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.528+0000 7fe93effd640 1 -- 172.21.15.78:0/4004770806 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94402c030 con 0x55e6c50839c0 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.528+0000 7fe93effd640 1 -- 172.21.15.78:0/4004770806 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944013ac0 con 0x55e6c50839c0 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.528+0000 7fe95db4b740 1 -- 172.21.15.78:0/4004770806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50839c0 msgr2=0x55e6c508dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.528+0000 7fe95db4b740 1 --2- 172.21.15.78:0/4004770806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50839c0 0x55e6c508dd40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe944005fd0 tx=0x7fe944016040 comp rx=0 tx=0).stop 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.528+0000 7fe95db4b740 1 -- 172.21.15.78:0/4004770806 shutdown_connections 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.528+0000 7fe95db4b740 1 --2- 172.21.15.78:0/4004770806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c5091b60 0x55e6c5092950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.528+0000 7fe95db4b740 1 --2- 172.21.15.78:0/4004770806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c50839c0 0x55e6c508dd40 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.528+0000 7fe95db4b740 1 --2- 172.21.15.78:0/4004770806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50830a0 0x55e6c509d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.528+0000 7fe95db4b740 1 -- 172.21.15.78:0/4004770806 >> 172.21.15.78:0/4004770806 conn(0x55e6c5080ed0 msgr2=0x55e6c50a3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.528+0000 7fe95db4b740 1 -- 172.21.15.78:0/4004770806 shutdown_connections 2024-09-11T01:05:00.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.528+0000 7fe95db4b740 1 -- 172.21.15.78:0/4004770806 wait complete. 2024-09-11T01:05:00.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe95db4b740 1 -- start start 2024-09-11T01:05:00.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe95db4b740 1 Processor -- start 2024-09-11T01:05:00.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe95db4b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 0x55e6c509a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.632 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:05:00.632 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:05:00.632 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:00.633 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:00.634 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:00.634 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:00.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:00.634 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:05:00.634 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:05:00.634 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:00.634 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:00.634 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:00.634 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:00.634 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:00.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:00.635 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:00.636 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:00.637 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:00.637 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:00.637 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:00.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:00.637 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:05:00.637 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:05:00.637 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:00.637 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:00.637 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:00.637 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:00.638 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:00.638 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:05:00.638 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:05:00.638 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:05:00.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe95db4b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50839c0 0x55e6c50a8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe95db4b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c5091b60 0x55e6c50a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe95db4b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541ad800 con 0x55e6c5091b60 2024-09-11T01:05:00.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe95db4b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541ad130 con 0x55e6c50839c0 2024-09-11T01:05:00.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe95db4b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541aa390 con 0x55e6c50830a0 2024-09-11T01:05:00.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe95db4b740 10 client.0 _ll_register_callbacks cb 0x7fe94400ef20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:00.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe959d13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c5091b60 0x55e6c50a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe959512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 0x55e6c509a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe959d13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c5091b60 0x55e6c50a9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51542/0 (socket says 172.21.15.78:51542) 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe959512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 0x55e6c509a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:55862/0 (socket says 172.21.15.78:55862) 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe959d13640 1 -- 172.21.15.78:0/1350422954 learned_addr learned my addr 172.21.15.78:0/1350422954 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.529+0000 7fe958d11640 1 --2- 172.21.15.78:0/1350422954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50839c0 0x55e6c50a8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe959512640 1 -- 172.21.15.78:0/1350422954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50839c0 msgr2=0x55e6c50a8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe959512640 1 --2- 172.21.15.78:0/1350422954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50839c0 0x55e6c50a8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe959512640 1 -- 172.21.15.78:0/1350422954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c5091b60 msgr2=0x55e6c50a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe959512640 1 --2- 172.21.15.78:0/1350422954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c5091b60 0x55e6c50a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe959512640 1 -- 172.21.15.78:0/1350422954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c50a9950 con 0x55e6c50830a0 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe959512640 1 --2- 172.21.15.78:0/1350422954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 0x55e6c509a290 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe9500039c0 tx=0x7fe9500040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eedcf2ea1c351f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe93cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe93cff9640 1 -- 172.21.15.78:0/1350422954 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950035030 con 0x55e6c50830a0 2024-09-11T01:05:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe95db4b740 1 -- 172.21.15.78:0/1350422954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe94402c030 con 0x55e6c50830a0 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe93cff9640 1 -- 172.21.15.78:0/1350422954 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe9500314a0 con 0x55e6c50830a0 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe93cff9640 1 -- 172.21.15.78:0/1350422954 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe95003f040 con 0x55e6c50830a0 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe95db4b740 10 client.25891 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe95db4b740 1 -- 172.21.15.78:0/1350422954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e6c510e1a0 con 0x55e6c50830a0 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe93cff9640 1 -- 172.21.15.78:0/1350422954 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94402c030 con 0x55e6c50830a0 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe93cff9640 1 -- 172.21.15.78:0/1350422954 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fe9500635d0 con 0x55e6c50830a0 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe95db4b740 10 client.25891 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe95db4b740 10 client.25891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.530+0000 7fe95db4b740 20 client.25891 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe95db4b740 1 --2- 172.21.15.78:0/1350422954 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e6c510e940 0x55e6c5110d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe95db4b740 4 client.25891 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe95db4b740 1 -- 172.21.15.78:0/1350422954 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e6c4ff7850 con 0x55e6c510e940 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe958d11640 1 --2- 172.21.15.78:0/1350422954 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e6c510e940 0x55e6c5110d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe93cff9640 10 client.25891 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe958d11640 1 --2- 172.21.15.78:0/1350422954 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e6c510e940 0x55e6c5110d60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c30f4af735e50dd server_cookie=ef49f3dc832e1ad3 in_seq=0 out_seq=0 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe93cff9640 1 -- 172.21.15.78:0/1350422954 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x55e6c4ff7850 con 0x55e6c510e940 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe93cff9640 10 client.25891 handle_command_reply: tid=0 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe95db4b740 1 client.25891 shutdown 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe95db4b740 1 -- 172.21.15.78:0/1350422954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 msgr2=0x55e6c509a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe95db4b740 1 --2- 172.21.15.78:0/1350422954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 0x55e6c509a290 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe9500039c0 tx=0x7fe9500040f0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe95db4b740 1 -- 172.21.15.78:0/1350422954 shutdown_connections 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe95db4b740 1 --2- 172.21.15.78:0/1350422954 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e6c510e940 0x55e6c5110d60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe95db4b740 1 --2- 172.21.15.78:0/1350422954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e6c5091b60 0x55e6c50a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe93cff9640 0 client.25891 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe95db4b740 1 --2- 172.21.15.78:0/1350422954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6c50839c0 0x55e6c50a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe95db4b740 1 --2- 172.21.15.78:0/1350422954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6c50830a0 0x55e6c509a290 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.531+0000 7fe95db4b740 1 -- 172.21.15.78:0/1350422954 >> 172.21.15.78:0/1350422954 conn(0x55e6c5080ed0 msgr2=0x55e6c5092950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95db4b740 1 -- 172.21.15.78:0/1350422954 shutdown_connections 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95db4b740 1 -- 172.21.15.78:0/1350422954 wait complete. 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95db4b740 20 client.25891 trim_cache size 0 max 16384 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95b776640 1 -- 172.21.15.78:0/2479675702 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe9400b54d0 msgr2=0x7fe9400b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95b776640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe9400b54d0 0x7fe9400b7990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55e6c5083050 tx=0x7fe944004730 comp rx=0 tx=0).stop 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95b776640 1 -- 172.21.15.78:0/2479675702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe954108060 msgr2=0x7fe95419da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95b776640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe954108060 0x7fe95419da70 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00eb10 tx=0x7fe94c00c2d0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95b776640 1 -- 172.21.15.78:0/2479675702 shutdown_connections 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95b776640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe9400b54d0 0x7fe9400b7990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95b776640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe954108060 0x7fe95419da70 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95b776640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9541076e0 0x7fe9541986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95b776640 1 --2- 172.21.15.78:0/2479675702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954101cd0 0x7fe954198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95b776640 1 -- 172.21.15.78:0/2479675702 >> 172.21.15.78:0/2479675702 conn(0x7fe9540fd700 msgr2=0x7fe9540ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95b776640 1 -- 172.21.15.78:0/2479675702 shutdown_connections 2024-09-11T01:05:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:00.532+0000 7fe95b776640 1 -- 172.21.15.78:0/2479675702 wait complete. 2024-09-11T01:05:00.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:00 smithi110 ceph-mon[34712]: pgmap v507: 97 pgs: 97 active+clean; 278 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-09-11T01:05:00.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:00 smithi112 ceph-mon[35384]: pgmap v507: 97 pgs: 97 active+clean; 278 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-09-11T01:05:01.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 460030 5.88 MB/sec execute 189 sec latency 1503.812 ms 2024-09-11T01:05:01.645 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:05:01.645 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:05:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:02.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:02 smithi078 ceph-mon[31315]: pgmap v508: 97 pgs: 97 active+clean; 279 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 193 op/s 2024-09-11T01:05:02.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 460784 5.86 MB/sec execute 190 sec latency 1897.316 ms 2024-09-11T01:05:02.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.455+0000 7fdc2f356640 1 -- 172.21.15.78:0/2356551312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc28102da0 msgr2=0x7fdc28103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:02.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.455+0000 7fdc2f356640 1 --2- 172.21.15.78:0/2356551312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc28102da0 0x7fdc28103180 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009900 tx=0x7fdc2402a410 comp rx=0 tx=0).stop 2024-09-11T01:05:02.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 -- 172.21.15.78:0/2356551312 shutdown_connections 2024-09-11T01:05:02.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 --2- 172.21.15.78:0/2356551312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc28109130 0x7fdc2810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:02.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 --2- 172.21.15.78:0/2356551312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc281087b0 0x7fdc28108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:02.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 --2- 172.21.15.78:0/2356551312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc28102da0 0x7fdc28103180 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:02.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 -- 172.21.15.78:0/2356551312 >> 172.21.15.78:0/2356551312 conn(0x7fdc280fe810 msgr2=0x7fdc28100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 -- 172.21.15.78:0/2356551312 shutdown_connections 2024-09-11T01:05:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 -- 172.21.15.78:0/2356551312 wait complete. 2024-09-11T01:05:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 Processor -- start 2024-09-11T01:05:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 -- start start 2024-09-11T01:05:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc28102da0 0x7fdc28199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc281087b0 0x7fdc28199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc28109130 0x7fdc2819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc28112df0 con 0x7fdc28109130 2024-09-11T01:05:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdc28112c70 con 0x7fdc28102da0 2024-09-11T01:05:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.456+0000 7fdc2f356640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc28112f70 con 0x7fdc281087b0 2024-09-11T01:05:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2d8f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc28109130 0x7fdc2819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2d8f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc28109130 0x7fdc2819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51552/0 (socket says 172.21.15.78:51552) 2024-09-11T01:05:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2d8f3640 1 -- 172.21.15.78:0/3949772920 learned_addr learned my addr 172.21.15.78:0/3949772920 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2c8f1640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc281087b0 0x7fdc28199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2d0f2640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc28102da0 0x7fdc28199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2d8f3640 1 -- 172.21.15.78:0/3949772920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc281087b0 msgr2=0x7fdc28199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2d8f3640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc281087b0 0x7fdc28199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2d8f3640 1 -- 172.21.15.78:0/3949772920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc28102da0 msgr2=0x7fdc28199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2d8f3640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc28102da0 0x7fdc28199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2d8f3640 1 -- 172.21.15.78:0/3949772920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2819f2f0 con 0x7fdc28109130 2024-09-11T01:05:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2d8f3640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc28109130 0x7fdc2819eb20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000eb10 tx=0x7fdc2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc1e7fc640 1 -- 172.21.15.78:0/3949772920 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20004b20 con 0x7fdc28109130 2024-09-11T01:05:02.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc1e7fc640 1 -- 172.21.15.78:0/3949772920 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc20004cc0 con 0x7fdc28109130 2024-09-11T01:05:02.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2f356640 1 -- 172.21.15.78:0/3949772920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2819f5e0 con 0x7fdc28109130 2024-09-11T01:05:02.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc1e7fc640 1 -- 172.21.15.78:0/3949772920 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20010670 con 0x7fdc28109130 2024-09-11T01:05:02.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.457+0000 7fdc2f356640 1 -- 172.21.15.78:0/3949772920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc281a5ca0 con 0x7fdc28109130 2024-09-11T01:05:02.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.458+0000 7fdc2f356640 1 -- 172.21.15.78:0/3949772920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc28103180 con 0x7fdc28109130 2024-09-11T01:05:02.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.460+0000 7fdc1e7fc640 1 -- 172.21.15.78:0/3949772920 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fdc20005020 con 0x7fdc28109130 2024-09-11T01:05:02.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.461+0000 7fdc1e7fc640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdc140b5450 0x7fdc140b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:02.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.461+0000 7fdc2d0f2640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdc140b5450 0x7fdc140b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:02.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.461+0000 7fdc1e7fc640 1 -- 172.21.15.78:0/3949772920 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdc200e6560 con 0x7fdc28109130 2024-09-11T01:05:02.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.461+0000 7fdc2d0f2640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdc140b5450 0x7fdc140b7910 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc24031440 tx=0x7fdc24005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:02.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.467+0000 7fdc1e7fc640 1 -- 172.21.15.78:0/3949772920 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdc200ad150 con 0x7fdc28109130 2024-09-11T01:05:02.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:02 smithi110 ceph-mon[34712]: pgmap v508: 97 pgs: 97 active+clean; 279 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 193 op/s 2024-09-11T01:05:02.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.690+0000 7fdc2f356640 1 -- 172.21.15.78:0/3949772920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc2802e5c0 con 0x7fdc28109130 2024-09-11T01:05:02.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.691+0000 7fdc1e7fc640 1 -- 172.21.15.78:0/3949772920 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11986 (secure 0 0 0) 0x7fdc200aca50 con 0x7fdc28109130 2024-09-11T01:05:02.694 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:05:02.694 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":40,"btime":"2024-09-11T01:04:37:224228+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:04:37.180149+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:05:02.694 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 40 2024-09-11T01:05:02.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.693+0000 7fdc2f356640 1 -- 172.21.15.78:0/3949772920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdc140b5450 msgr2=0x7fdc140b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:02.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.693+0000 7fdc2f356640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdc140b5450 0x7fdc140b7910 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc24031440 tx=0x7fdc24005b50 comp rx=0 tx=0).stop 2024-09-11T01:05:02.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.693+0000 7fdc2f356640 1 -- 172.21.15.78:0/3949772920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc28109130 msgr2=0x7fdc2819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:02.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.693+0000 7fdc2f356640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc28109130 0x7fdc2819eb20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000eb10 tx=0x7fdc2000c2d0 comp rx=0 tx=0).stop 2024-09-11T01:05:02.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.693+0000 7fdc2f356640 1 -- 172.21.15.78:0/3949772920 shutdown_connections 2024-09-11T01:05:02.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.693+0000 7fdc2f356640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdc140b5450 0x7fdc140b7910 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:02.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.693+0000 7fdc2f356640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc28109130 0x7fdc2819eb20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:02.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.693+0000 7fdc2f356640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc281087b0 0x7fdc28199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:02.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.694+0000 7fdc2f356640 1 --2- 172.21.15.78:0/3949772920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc28102da0 0x7fdc28199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:02.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.694+0000 7fdc2f356640 1 -- 172.21.15.78:0/3949772920 >> 172.21.15.78:0/3949772920 conn(0x7fdc280fe810 msgr2=0x7fdc28100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:02.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.694+0000 7fdc2f356640 1 -- 172.21.15.78:0/3949772920 shutdown_connections 2024-09-11T01:05:02.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:02.694+0000 7fdc2f356640 1 -- 172.21.15.78:0/3949772920 wait complete. 2024-09-11T01:05:02.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:02 smithi112 ceph-mon[35384]: pgmap v508: 97 pgs: 97 active+clean; 279 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 193 op/s 2024-09-11T01:05:03.399 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:05:03.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 462343 5.89 MB/sec execute 191 sec latency 81.633 ms 2024-09-11T01:05:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:03 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3949772920' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:05:03.663 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:03.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:03 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3949772920' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:05:03.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:03 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3949772920' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:05:04.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.287+0000 7fc11fa34640 1 -- 172.21.15.78:0/1208951589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc118104600 msgr2=0x7fc11810caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:04.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.287+0000 7fc11fa34640 1 --2- 172.21.15.78:0/1208951589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc118104600 0x7fc11810caf0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc108009900 tx=0x7fc10802a410 comp rx=0 tx=0).stop 2024-09-11T01:05:04.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.287+0000 7fc11fa34640 1 -- 172.21.15.78:0/1208951589 shutdown_connections 2024-09-11T01:05:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.287+0000 7fc11fa34640 1 --2- 172.21.15.78:0/1208951589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc11810d140 0x7fc11810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.287+0000 7fc11fa34640 1 --2- 172.21.15.78:0/1208951589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc118104600 0x7fc11810caf0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.287+0000 7fc11fa34640 1 --2- 172.21.15.78:0/1208951589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc118103ce0 0x7fc1181040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.287+0000 7fc11fa34640 1 -- 172.21.15.78:0/1208951589 >> 172.21.15.78:0/1208951589 conn(0x7fc1180fd700 msgr2=0x7fc1180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.287+0000 7fc11fa34640 1 -- 172.21.15.78:0/1208951589 shutdown_connections 2024-09-11T01:05:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.287+0000 7fc11fa34640 1 -- 172.21.15.78:0/1208951589 wait complete. 2024-09-11T01:05:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.288+0000 7fc11fa34640 1 Processor -- start 2024-09-11T01:05:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.288+0000 7fc11fa34640 1 -- start start 2024-09-11T01:05:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.288+0000 7fc11fa34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc118103ce0 0x7fc1181992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.288+0000 7fc11fa34640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc118104600 0x7fc118199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.288+0000 7fc11fa34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc11810d140 0x7fc11819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.288+0000 7fc11fa34640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc118112db0 con 0x7fc118103ce0 2024-09-11T01:05:04.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.288+0000 7fc11fa34640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc118112c30 con 0x7fc118104600 2024-09-11T01:05:04.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.288+0000 7fc11fa34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc118112f30 con 0x7fc11810d140 2024-09-11T01:05:04.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.288+0000 7fc11d7d0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc118103ce0 0x7fc1181992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.288+0000 7fc11d7d0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc118103ce0 0x7fc1181992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38274/0 (socket says 172.21.15.78:38274) 2024-09-11T01:05:04.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.288+0000 7fc11d7d0640 1 -- 172.21.15.78:0/839473815 learned_addr learned my addr 172.21.15.78:0/839473815 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:04.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.288+0000 7fc11cfcf640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc118104600 0x7fc118199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.289+0000 7fc11d7d0640 1 -- 172.21.15.78:0/839473815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc11810d140 msgr2=0x7fc11819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:04.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.289+0000 7fc11d7d0640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc11810d140 0x7fc11819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.289+0000 7fc11d7d0640 1 -- 172.21.15.78:0/839473815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc118104600 msgr2=0x7fc118199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.289+0000 7fc11d7d0640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc118104600 0x7fc118199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.289+0000 7fc11d7d0640 1 -- 172.21.15.78:0/839473815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11819f390 con 0x7fc118103ce0 2024-09-11T01:05:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.289+0000 7fc11d7d0640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc118103ce0 0x7fc1181992e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc11400d950 tx=0x7fc11400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.289+0000 7fc10e7fc640 1 -- 172.21.15.78:0/839473815 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114017070 con 0x7fc118103ce0 2024-09-11T01:05:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.289+0000 7fc10e7fc640 1 -- 172.21.15.78:0/839473815 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc114009c30 con 0x7fc118103ce0 2024-09-11T01:05:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.289+0000 7fc10e7fc640 1 -- 172.21.15.78:0/839473815 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11400f620 con 0x7fc118103ce0 2024-09-11T01:05:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.289+0000 7fc11fa34640 1 -- 172.21.15.78:0/839473815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11819f680 con 0x7fc118103ce0 2024-09-11T01:05:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.289+0000 7fc11fa34640 1 -- 172.21.15.78:0/839473815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1181a5f10 con 0x7fc118103ce0 2024-09-11T01:05:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc121e09740 1 Processor -- start 2024-09-11T01:05:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc10e7fc640 1 -- 172.21.15.78:0/839473815 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc114010460 con 0x7fc118103ce0 2024-09-11T01:05:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc121e09740 1 -- start start 2024-09-11T01:05:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc121e09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723480f0 0x559e723484d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc121e09740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e72354460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc121e09740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 0x559e72356d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc121e09740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559e722beea0 con 0x559e723549a0 2024-09-11T01:05:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc121e09740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559e722ae2d0 con 0x559e72348a10 2024-09-11T01:05:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc121e09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559e722b20e0 con 0x559e723480f0 2024-09-11T01:05:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc11cfcf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 0x559e72356d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc11dfd1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723480f0 0x559e723484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc11cfcf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 0x559e72356d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38276/0 (socket says 172.21.15.78:38276) 2024-09-11T01:05:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc11dfd1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723480f0 0x559e723484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:33900/0 (socket says 172.21.15.78:33900) 2024-09-11T01:05:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc11cfcf640 1 -- 172.21.15.78:0/1601154804 learned_addr learned my addr 172.21.15.78:0/1601154804 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc11d7d0640 1 --2- 172.21.15.78:0/1601154804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e72354460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc11cfcf640 1 -- 172.21.15.78:0/1601154804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723480f0 msgr2=0x559e723484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc11cfcf640 1 --2- 172.21.15.78:0/1601154804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723480f0 0x559e723484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc11cfcf640 1 -- 172.21.15.78:0/1601154804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 msgr2=0x559e72354460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc11cfcf640 1 --2- 172.21.15.78:0/1601154804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e72354460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc11cfcf640 1 -- 172.21.15.78:0/1601154804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e7200c650 con 0x559e723549a0 2024-09-11T01:05:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc11d7d0640 1 --2- 172.21.15.78:0/1601154804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e72354460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:05:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.292+0000 7fc10e7fc640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc1040b54d0 0x7fc1040b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.293+0000 7fc11cfcf640 1 --2- 172.21.15.78:0/1601154804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 0x559e72356d90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc11806c330 tx=0x7fc108032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ab234b6a8936d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.293+0000 7fc11dfd1640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc1040b54d0 0x7fc1040b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.293+0000 7fc10e7fc640 1 -- 172.21.15.78:0/839473815 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc1140e5b90 con 0x7fc118103ce0 2024-09-11T01:05:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.293+0000 7fc1037fe640 1 -- 172.21.15.78:0/1601154804 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108040070 con 0x559e723549a0 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.293+0000 7fc1037fe640 1 -- 172.21.15.78:0/1601154804 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc108004030 con 0x559e723549a0 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.293+0000 7fc1037fe640 1 -- 172.21.15.78:0/1601154804 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108035040 con 0x559e723549a0 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc121e09740 1 -- 172.21.15.78:0/1601154804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 msgr2=0x559e72356d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc121e09740 1 --2- 172.21.15.78:0/1601154804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 0x559e72356d90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc11806c330 tx=0x7fc108032bd0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc11dfd1640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc1040b54d0 0x7fc1040b7990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc110009270 tx=0x7fc1100095e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc121e09740 1 -- 172.21.15.78:0/1601154804 shutdown_connections 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc121e09740 1 --2- 172.21.15.78:0/1601154804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 0x559e72356d90 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc121e09740 1 --2- 172.21.15.78:0/1601154804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e72354460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc121e09740 1 --2- 172.21.15.78:0/1601154804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723480f0 0x559e723484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc121e09740 1 -- 172.21.15.78:0/1601154804 >> 172.21.15.78:0/1601154804 conn(0x559e72345f20 msgr2=0x559e723470c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc121e09740 1 -- 172.21.15.78:0/1601154804 shutdown_connections 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc121e09740 1 -- 172.21.15.78:0/1601154804 wait complete. 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc121e09740 1 -- start start 2024-09-11T01:05:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc121e09740 1 Processor -- start 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.294+0000 7fc121e09740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 0x559e72354260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc121e09740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e723516b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc121e09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723549a0 0x559e72351c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc121e09740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559e7210f700 con 0x559e723480f0 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc121e09740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559e72249160 con 0x559e72348a10 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc121e09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559e723575c0 con 0x559e723549a0 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc121e09740 10 client.0 _ll_register_callbacks cb 0x559e722629e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc11d7d0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 0x559e72354260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc11d7d0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 0x559e72354260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38286/0 (socket says 172.21.15.78:38286) 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc11d7d0640 1 -- 172.21.15.78:0/1728595404 learned_addr learned my addr 172.21.15.78:0/1728595404 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc11dfd1640 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723549a0 0x559e72351c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc11cfcf640 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e723516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc11d7d0640 1 -- 172.21.15.78:0/1728595404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723549a0 msgr2=0x559e72351c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc11d7d0640 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723549a0 0x559e72351c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc11d7d0640 1 -- 172.21.15.78:0/1728595404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 msgr2=0x559e723516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc11d7d0640 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e723516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc11d7d0640 1 -- 172.21.15.78:0/1728595404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e72276c90 con 0x559e723480f0 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc11d7d0640 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 0x559e72354260 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc114000c00 tx=0x7fc1140a8570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dba92e8911e1fb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc100ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc100ff9640 1 -- 172.21.15.78:0/1728595404 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114020030 con 0x559e723480f0 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc100ff9640 1 -- 172.21.15.78:0/1728595404 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc1140ab4f0 con 0x559e723480f0 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc121e09740 1 -- 172.21.15.78:0/1728595404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc108004030 con 0x559e723480f0 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc100ff9640 1 -- 172.21.15.78:0/1728595404 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc1140ab9d0 con 0x559e723480f0 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc121e09740 10 client.16614 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.295+0000 7fc121e09740 1 -- 172.21.15.78:0/1728595404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559e72362ec0 con 0x559e723480f0 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.296+0000 7fc100ff9640 1 -- 172.21.15.78:0/1728595404 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc108004030 con 0x559e723480f0 2024-09-11T01:05:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.296+0000 7fc100ff9640 1 -- 172.21.15.78:0/1728595404 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fc1140ec070 con 0x559e723480f0 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.296+0000 7fc121e09740 10 client.16614 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.296+0000 7fc121e09740 10 client.16614 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.296+0000 7fc121e09740 20 client.16614 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.296+0000 7fc121e09740 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x559e72364380 0x559e72366800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.296+0000 7fc121e09740 4 client.16614 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.296+0000 7fc121e09740 1 -- 172.21.15.78:0/1728595404 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e722c0df0 con 0x559e72364380 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.296+0000 7fc11cfcf640 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x559e72364380 0x559e72366800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.296+0000 7fc100ff9640 10 client.16614 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.296+0000 7fc11cfcf640 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x559e72364380 0x559e72366800 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=61aecbccaed23b0e server_cookie=379aaad5b821a032 in_seq=0 out_seq=0 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.297+0000 7fc100ff9640 1 -- 172.21.15.78:0/1728595404 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e722c0df0 con 0x559e72364380 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.297+0000 7fc100ff9640 10 client.16614 handle_command_reply: tid=0 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.297+0000 7fc121e09740 1 client.16614 shutdown 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.297+0000 7fc121e09740 1 -- 172.21.15.78:0/1728595404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 msgr2=0x559e72354260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.297+0000 7fc121e09740 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 0x559e72354260 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc114000c00 tx=0x7fc1140a8570 comp rx=0 tx=0).stop 2024-09-11T01:05:04.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.297+0000 7fc121e09740 1 -- 172.21.15.78:0/1728595404 shutdown_connections 2024-09-11T01:05:04.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.297+0000 7fc121e09740 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x559e72364380 0x559e72366800 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.297+0000 7fc121e09740 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723549a0 0x559e72351c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.297+0000 7fc121e09740 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e723516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.297+0000 7fc121e09740 1 --2- 172.21.15.78:0/1728595404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 0x559e72354260 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.297+0000 7fc121e09740 1 -- 172.21.15.78:0/1728595404 >> 172.21.15.78:0/1728595404 conn(0x559e72345f20 msgr2=0x559e7234bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:04.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.297+0000 7fc100ff9640 0 client.16614 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:05:04.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.298+0000 7fc121e09740 1 -- 172.21.15.78:0/1728595404 shutdown_connections 2024-09-11T01:05:04.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.298+0000 7fc121e09740 1 -- 172.21.15.78:0/1728595404 wait complete. 2024-09-11T01:05:04.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.298+0000 7fc121e09740 20 client.16614 trim_cache size 0 max 16384 2024-09-11T01:05:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc121e09740 1 Processor -- start 2024-09-11T01:05:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc121e09740 1 -- start start 2024-09-11T01:05:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc121e09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723480f0 0x559e723607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc121e09740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e72340eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc121e09740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 0x559e72346f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc121e09740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181ae550 con 0x559e723549a0 2024-09-11T01:05:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc121e09740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181b0150 con 0x559e72348a10 2024-09-11T01:05:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc121e09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181aec20 con 0x559e723480f0 2024-09-11T01:05:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc11dfd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 0x559e72346f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc11dfd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 0x559e72346f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38302/0 (socket says 172.21.15.78:38302) 2024-09-11T01:05:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc11dfd1640 1 -- 172.21.15.78:0/3973260922 learned_addr learned my addr 172.21.15.78:0/3973260922 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc11cfcf640 1 --2- 172.21.15.78:0/3973260922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e72340eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc11d7d0640 1 --2- 172.21.15.78:0/3973260922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723480f0 0x559e723607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc11dfd1640 1 -- 172.21.15.78:0/3973260922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723480f0 msgr2=0x559e723607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc11dfd1640 1 --2- 172.21.15.78:0/3973260922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723480f0 0x559e723607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc11dfd1640 1 -- 172.21.15.78:0/3973260922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 msgr2=0x559e72340eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc11dfd1640 1 --2- 172.21.15.78:0/3973260922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e72340eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc11dfd1640 1 -- 172.21.15.78:0/3973260922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e7234fbb0 con 0x559e723549a0 2024-09-11T01:05:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.311+0000 7fc11dfd1640 1 --2- 172.21.15.78:0/3973260922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 0x559e72346f70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc110012790 tx=0x7fc110014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6398f94c11cc676 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.312+0000 7fc102ffd640 1 -- 172.21.15.78:0/3973260922 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11001f780 con 0x559e723549a0 2024-09-11T01:05:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.312+0000 7fc102ffd640 1 -- 172.21.15.78:0/3973260922 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc11001fdd0 con 0x559e723549a0 2024-09-11T01:05:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.312+0000 7fc102ffd640 1 -- 172.21.15.78:0/3973260922 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100283d0 con 0x559e723549a0 2024-09-11T01:05:04.316 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:04 smithi078 ceph-mon[31315]: pgmap v509: 97 pgs: 97 active+clean; 281 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-11T01:05:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.312+0000 7fc121e09740 1 -- 172.21.15.78:0/3973260922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 msgr2=0x559e72346f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.312+0000 7fc121e09740 1 --2- 172.21.15.78:0/3973260922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 0x559e72346f70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc110012790 tx=0x7fc110014000 comp rx=0 tx=0).stop 2024-09-11T01:05:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.312+0000 7fc121e09740 1 -- 172.21.15.78:0/3973260922 shutdown_connections 2024-09-11T01:05:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.312+0000 7fc121e09740 1 --2- 172.21.15.78:0/3973260922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723549a0 0x559e72346f70 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.312+0000 7fc121e09740 1 --2- 172.21.15.78:0/3973260922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e72340eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.312+0000 7fc121e09740 1 --2- 172.21.15.78:0/3973260922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723480f0 0x559e723607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.312+0000 7fc121e09740 1 -- 172.21.15.78:0/3973260922 >> 172.21.15.78:0/3973260922 conn(0x559e72345f20 msgr2=0x559e7234ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.313+0000 7fc121e09740 1 -- 172.21.15.78:0/3973260922 shutdown_connections 2024-09-11T01:05:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.313+0000 7fc121e09740 1 -- 172.21.15.78:0/3973260922 wait complete. 2024-09-11T01:05:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.313+0000 7fc121e09740 1 -- start start 2024-09-11T01:05:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.313+0000 7fc121e09740 1 Processor -- start 2024-09-11T01:05:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.313+0000 7fc121e09740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 0x559e7235ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.313+0000 7fc121e09740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e7235cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.313+0000 7fc121e09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723549a0 0x559e7234d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc121e09740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181af9c0 con 0x559e723480f0 2024-09-11T01:05:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc121e09740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181af2f0 con 0x559e72348a10 2024-09-11T01:05:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc121e09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181ac530 con 0x559e723549a0 2024-09-11T01:05:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc11d7d0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 0x559e7235ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc11cfcf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e7235cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc11d7d0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 0x559e7235ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38310/0 (socket says 172.21.15.78:38310) 2024-09-11T01:05:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc121e09740 10 client.0 _ll_register_callbacks cb 0x7fc1140e37e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc11dfd1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723549a0 0x559e7234d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc11d7d0640 1 -- 172.21.15.78:0/4007051637 learned_addr learned my addr 172.21.15.78:0/4007051637 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc11d7d0640 1 -- 172.21.15.78:0/4007051637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723549a0 msgr2=0x559e7234d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc11d7d0640 1 --2- 172.21.15.78:0/4007051637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723549a0 0x559e7234d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc11d7d0640 1 -- 172.21.15.78:0/4007051637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 msgr2=0x559e7235cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc11d7d0640 1 --2- 172.21.15.78:0/4007051637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e7235cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc11d7d0640 1 -- 172.21.15.78:0/4007051637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e7234dfc0 con 0x559e723480f0 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc11dfd1640 1 --2- 172.21.15.78:0/4007051637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723549a0 0x559e7234d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc11d7d0640 1 --2- 172.21.15.78:0/4007051637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 0x559e7235ca30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc1140e3950 tx=0x7fc114005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7058ac68d9dd4cd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc100ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc100ff9640 1 -- 172.21.15.78:0/4007051637 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114020070 con 0x559e723480f0 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc100ff9640 1 -- 172.21.15.78:0/4007051637 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc1140a79d0 con 0x559e723480f0 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.314+0000 7fc121e09740 1 -- 172.21.15.78:0/4007051637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc11001fdd0 con 0x559e723480f0 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.315+0000 7fc100ff9640 1 -- 172.21.15.78:0/4007051637 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc1140ab7f0 con 0x559e723480f0 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.315+0000 7fc121e09740 10 client.16620 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.315+0000 7fc121e09740 1 -- 172.21.15.78:0/4007051637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559e7234f280 con 0x559e723480f0 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.315+0000 7fc100ff9640 1 -- 172.21.15.78:0/4007051637 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc11001fdd0 con 0x559e723480f0 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.315+0000 7fc100ff9640 1 -- 172.21.15.78:0/4007051637 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fc1140ec070 con 0x559e723480f0 2024-09-11T01:05:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.315+0000 7fc121e09740 10 client.16620 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.315+0000 7fc121e09740 10 client.16620 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:05:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.315+0000 7fc121e09740 20 client.16620 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.315+0000 7fc121e09740 1 --2- 172.21.15.78:0/4007051637 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x559e723d22d0 0x559e723d46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.315+0000 7fc121e09740 4 client.16620 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:05:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.315+0000 7fc121e09740 1 -- 172.21.15.78:0/4007051637 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e722bc8a0 con 0x559e723d22d0 2024-09-11T01:05:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.315+0000 7fc11cfcf640 1 --2- 172.21.15.78:0/4007051637 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x559e723d22d0 0x559e723d46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc100ff9640 10 client.16620 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:05:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc11cfcf640 1 --2- 172.21.15.78:0/4007051637 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x559e723d22d0 0x559e723d46c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea025efd5b4eeff3 server_cookie=f6980e8f976b730f in_seq=0 out_seq=0 2024-09-11T01:05:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc100ff9640 1 -- 172.21.15.78:0/4007051637 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e722bc8a0 con 0x559e723d22d0 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc100ff9640 10 client.16620 handle_command_reply: tid=0 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc121e09740 1 client.16620 shutdown 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc121e09740 1 -- 172.21.15.78:0/4007051637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 msgr2=0x559e7235ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc121e09740 1 --2- 172.21.15.78:0/4007051637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 0x559e7235ca30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc1140e3950 tx=0x7fc114005880 comp rx=0 tx=0).stop 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc121e09740 1 -- 172.21.15.78:0/4007051637 shutdown_connections 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc121e09740 1 --2- 172.21.15.78:0/4007051637 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x559e723d22d0 0x559e723d46c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc121e09740 1 --2- 172.21.15.78:0/4007051637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e723549a0 0x559e7234d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc121e09740 1 --2- 172.21.15.78:0/4007051637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e72348a10 0x559e7235cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc121e09740 1 --2- 172.21.15.78:0/4007051637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e723480f0 0x559e7235ca30 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc121e09740 1 -- 172.21.15.78:0/4007051637 >> 172.21.15.78:0/4007051637 conn(0x559e72345f20 msgr2=0x559e7234fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc100ff9640 0 client.16620 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc121e09740 1 -- 172.21.15.78:0/4007051637 shutdown_connections 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc121e09740 1 -- 172.21.15.78:0/4007051637 wait complete. 2024-09-11T01:05:04.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.316+0000 7fc121e09740 20 client.16620 trim_cache size 0 max 16384 2024-09-11T01:05:04.325 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:05:04.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.317+0000 7fc11fa34640 1 -- 172.21.15.78:0/839473815 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc1040b54d0 msgr2=0x7fc1040b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:04.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.317+0000 7fc11fa34640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc1040b54d0 0x7fc1040b7990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc110009270 tx=0x7fc1100095e0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.317+0000 7fc11fa34640 1 -- 172.21.15.78:0/839473815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc118103ce0 msgr2=0x7fc1181992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:04.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.317+0000 7fc11fa34640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc118103ce0 0x7fc1181992e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc11400d950 tx=0x7fc11400de10 comp rx=0 tx=0).stop 2024-09-11T01:05:04.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.317+0000 7fc11fa34640 1 -- 172.21.15.78:0/839473815 shutdown_connections 2024-09-11T01:05:04.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.317+0000 7fc11fa34640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc1040b54d0 0x7fc1040b7990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.317+0000 7fc11fa34640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc11810d140 0x7fc11819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.317+0000 7fc11fa34640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc118104600 0x7fc118199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.317+0000 7fc11fa34640 1 --2- 172.21.15.78:0/839473815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc118103ce0 0x7fc1181992e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:04.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.317+0000 7fc11fa34640 1 -- 172.21.15.78:0/839473815 >> 172.21.15.78:0/839473815 conn(0x7fc1180fd700 msgr2=0x7fc1180fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:04.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.317+0000 7fc11fa34640 1 -- 172.21.15.78:0/839473815 shutdown_connections 2024-09-11T01:05:04.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:04.317+0000 7fc11fa34640 1 -- 172.21.15.78:0/839473815 wait complete. 2024-09-11T01:05:04.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 463595 5.89 MB/sec execute 192 sec latency 273.519 ms 2024-09-11T01:05:04.591 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:04 smithi110 ceph-mon[34712]: pgmap v509: 97 pgs: 97 active+clean; 281 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-11T01:05:04.625 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:04 smithi112 ceph-mon[35384]: pgmap v509: 97 pgs: 97 active+clean; 281 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-11T01:05:05.056 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:05:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:05.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 463595 5.86 MB/sec execute 193 sec latency 1273.635 ms 2024-09-11T01:05:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.849+0000 7fd3f9646640 1 -- 172.21.15.78:0/2647791646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3f41087d0 msgr2=0x7fd3f4108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.849+0000 7fd3f9646640 1 --2- 172.21.15.78:0/2647791646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3f41087d0 0x7fd3f4108c10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009900 tx=0x7fd3dc02a410 comp rx=0 tx=0).stop 2024-09-11T01:05:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.849+0000 7fd3f9646640 1 -- 172.21.15.78:0/2647791646 shutdown_connections 2024-09-11T01:05:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.849+0000 7fd3f9646640 1 --2- 172.21.15.78:0/2647791646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f4109150 0x7fd3f410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.849+0000 7fd3f9646640 1 --2- 172.21.15.78:0/2647791646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3f41087d0 0x7fd3f4108c10 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.849+0000 7fd3f9646640 1 --2- 172.21.15.78:0/2647791646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3f4102dc0 0x7fd3f41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.849+0000 7fd3f9646640 1 -- 172.21.15.78:0/2647791646 >> 172.21.15.78:0/2647791646 conn(0x7fd3f40fe830 msgr2=0x7fd3f4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.849+0000 7fd3f9646640 1 -- 172.21.15.78:0/2647791646 shutdown_connections 2024-09-11T01:05:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.849+0000 7fd3f9646640 1 -- 172.21.15.78:0/2647791646 wait complete. 2024-09-11T01:05:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.849+0000 7fd3f9646640 1 Processor -- start 2024-09-11T01:05:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f9646640 1 -- start start 2024-09-11T01:05:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f9646640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3f4102dc0 0x7fd3f41992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f9646640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3f41087d0 0x7fd3f4199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f9646640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f4109150 0x7fd3f419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f9646640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4112e10 con 0x7fd3f4102dc0 2024-09-11T01:05:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f9646640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4112c90 con 0x7fd3f41087d0 2024-09-11T01:05:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f9646640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4112f90 con 0x7fd3f4109150 2024-09-11T01:05:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3f4102dc0 0x7fd3f41992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3f4102dc0 0x7fd3f41992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38330/0 (socket says 172.21.15.78:38330) 2024-09-11T01:05:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f2ffd640 1 -- 172.21.15.78:0/3155703217 learned_addr learned my addr 172.21.15.78:0/3155703217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f37fe640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f4109150 0x7fd3f419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f27fc640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3f41087d0 0x7fd3f4199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f2ffd640 1 -- 172.21.15.78:0/3155703217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f4109150 msgr2=0x7fd3f419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f4109150 0x7fd3f419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f2ffd640 1 -- 172.21.15.78:0/3155703217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3f41087d0 msgr2=0x7fd3f4199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3f41087d0 0x7fd3f4199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f2ffd640 1 -- 172.21.15.78:0/3155703217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f419f390 con 0x7fd3f4102dc0 2024-09-11T01:05:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f37fe640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f4109150 0x7fd3f419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:05:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3f4102dc0 0x7fd3f41992e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400d950 tx=0x7fd3e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.850+0000 7fd3ebfff640 1 -- 172.21.15.78:0/3155703217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4017070 con 0x7fd3f4102dc0 2024-09-11T01:05:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.851+0000 7fd3f9646640 1 -- 172.21.15.78:0/3155703217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f419f680 con 0x7fd3f4102dc0 2024-09-11T01:05:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.851+0000 7fd3ebfff640 1 -- 172.21.15.78:0/3155703217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e4009c30 con 0x7fd3f4102dc0 2024-09-11T01:05:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.851+0000 7fd3ebfff640 1 -- 172.21.15.78:0/3155703217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e400f620 con 0x7fd3f4102dc0 2024-09-11T01:05:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.851+0000 7fd3f9646640 1 -- 172.21.15.78:0/3155703217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f41a5f10 con 0x7fd3f4102dc0 2024-09-11T01:05:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.853+0000 7fd3fba1b740 1 Processor -- start 2024-09-11T01:05:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.853+0000 7fd3fba1b740 1 -- start start 2024-09-11T01:05:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.853+0000 7fd3fba1b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb6ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.853+0000 7fd3fba1b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb70b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.853+0000 7fd3fba1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb70e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.853+0000 7fd3ebfff640 1 -- 172.21.15.78:0/3155703217 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd3e4010460 con 0x7fd3f4102dc0 2024-09-11T01:05:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.853+0000 7fd3fba1b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5592bb675ea0 con 0x5592bb6ffdf0 2024-09-11T01:05:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.853+0000 7fd3fba1b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5592bb6652d0 con 0x5592bb6ff4d0 2024-09-11T01:05:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.853+0000 7fd3fba1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592bb6690e0 con 0x5592bb70bd80 2024-09-11T01:05:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb70b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb70b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38340/0 (socket says 172.21.15.78:38340) 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f2ffd640 1 -- 172.21.15.78:0/1671847191 learned_addr learned my addr 172.21.15.78:0/1671847191 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f37fe640 1 --2- 172.21.15.78:0/1671847191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb6ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f27fc640 1 --2- 172.21.15.78:0/1671847191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb70e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f2ffd640 1 -- 172.21.15.78:0/1671847191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 msgr2=0x5592bb70e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/1671847191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb70e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f2ffd640 1 -- 172.21.15.78:0/1671847191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 msgr2=0x5592bb6ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/1671847191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb6ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f2ffd640 1 -- 172.21.15.78:0/1671847191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592bb3c3650 con 0x5592bb6ffdf0 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f27fc640 1 --2- 172.21.15.78:0/1671847191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb70e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f37fe640 1 --2- 172.21.15.78:0/1671847191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb6ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/1671847191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb70b840 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40e33a0 tx=0x7fd3e40ee7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f001781120e1303c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3ebfff640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd3e00b5450 0x7fd3e00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3e97fa640 1 -- 172.21.15.78:0/1671847191 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e40ec860 con 0x5592bb6ffdf0 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3e97fa640 1 -- 172.21.15.78:0/1671847191 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e40f3070 con 0x5592bb6ffdf0 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3f37fe640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd3e00b5450 0x7fd3e00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.854+0000 7fd3e97fa640 1 -- 172.21.15.78:0/1671847191 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e40abab0 con 0x5592bb6ffdf0 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.855+0000 7fd3ebfff640 1 -- 172.21.15.78:0/3155703217 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd3e40e5c70 con 0x7fd3f4102dc0 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.855+0000 7fd3f37fe640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd3e00b5450 0x7fd3e00b7910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00a4c0 tx=0x7fd3ec0096f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.855+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1671847191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 msgr2=0x5592bb70b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.855+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/1671847191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb70b840 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40e33a0 tx=0x7fd3e40ee7a0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.855+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1671847191 shutdown_connections 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.855+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/1671847191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb70e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.855+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/1671847191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb70b840 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.855+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/1671847191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb6ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.855+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1671847191 >> 172.21.15.78:0/1671847191 conn(0x5592bb6fd300 msgr2=0x5592bb6fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.855+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1671847191 shutdown_connections 2024-09-11T01:05:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.855+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1671847191 wait complete. 2024-09-11T01:05:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3fba1b740 1 -- start start 2024-09-11T01:05:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3fba1b740 1 Processor -- start 2024-09-11T01:05:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3fba1b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb70b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3fba1b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb7089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3fba1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb708f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3fba1b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5592bb4c6700 con 0x5592bb6ffdf0 2024-09-11T01:05:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3fba1b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5592bb600160 con 0x5592bb6ff4d0 2024-09-11T01:05:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3fba1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592bb70ea50 con 0x5592bb70bd80 2024-09-11T01:05:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3fba1b740 10 client.0 _ll_register_callbacks cb 0x5592bb6199e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb7089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb7089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38342/0 (socket says 172.21.15.78:38342) 2024-09-11T01:05:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3f27fc640 1 -- 172.21.15.78:0/1419080682 learned_addr learned my addr 172.21.15.78:0/1419080682 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb70b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3f37fe640 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb708f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3f27fc640 1 -- 172.21.15.78:0/1419080682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 msgr2=0x5592bb708f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3f27fc640 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb708f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3f27fc640 1 -- 172.21.15.78:0/1419080682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 msgr2=0x5592bb70b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3f27fc640 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb70b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3f27fc640 1 -- 172.21.15.78:0/1419080682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592bb62dc90 con 0x5592bb6ffdf0 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3f37fe640 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb708f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.856+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb70b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3f27fc640 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb7089e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc005ab0 tx=0x7fd3dc031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6096d6681b52dd62 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3da7fc640 1 -- 172.21.15.78:0/1419080682 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc03f070 con 0x5592bb6ffdf0 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3da7fc640 1 -- 172.21.15.78:0/1419080682 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd3dc004450 con 0x5592bb6ffdf0 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1419080682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3e40f3070 con 0x5592bb6ffdf0 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3da7fc640 1 -- 172.21.15.78:0/1419080682 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd3dc035070 con 0x5592bb6ffdf0 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3fba1b740 10 client.16632 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1419080682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5592bb71a270 con 0x5592bb6ffdf0 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3da7fc640 1 -- 172.21.15.78:0/1419080682 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e40f3070 con 0x5592bb6ffdf0 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3da7fc640 1 -- 172.21.15.78:0/1419080682 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fd3dc04b020 con 0x5592bb6ffdf0 2024-09-11T01:05:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3fba1b740 10 client.16632 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3fba1b740 10 client.16632 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3fba1b740 20 client.16632 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5592bb71b6d0 0x5592bb71dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3fba1b740 4 client.16632 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.857+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1419080682 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592bb677df0 con 0x5592bb71b6d0 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.858+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5592bb71b6d0 0x5592bb71dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.858+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5592bb71b6d0 0x5592bb71dac0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eb5ce950144324af server_cookie=31eb94c67b3b4b62 in_seq=0 out_seq=0 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.858+0000 7fd3da7fc640 10 client.16632 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3da7fc640 1 -- 172.21.15.78:0/1419080682 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5592bb677df0 con 0x5592bb71b6d0 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3da7fc640 10 client.16632 handle_command_reply: tid=0 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3fba1b740 1 client.16632 shutdown 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1419080682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 msgr2=0x5592bb7089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb7089e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc005ab0 tx=0x7fd3dc031750 comp rx=0 tx=0).stop 2024-09-11T01:05:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1419080682 shutdown_connections 2024-09-11T01:05:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5592bb71b6d0 0x5592bb71dac0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb708f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb7089e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3da7fc640 0 client.16632 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:05:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/1419080682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb70b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1419080682 >> 172.21.15.78:0/1419080682 conn(0x5592bb6fd300 msgr2=0x5592bb702ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1419080682 shutdown_connections 2024-09-11T01:05:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3fba1b740 1 -- 172.21.15.78:0/1419080682 wait complete. 2024-09-11T01:05:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.859+0000 7fd3fba1b740 20 client.16632 trim_cache size 0 max 16384 2024-09-11T01:05:05.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3fba1b740 1 Processor -- start 2024-09-11T01:05:05.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3fba1b740 1 -- start start 2024-09-11T01:05:05.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3fba1b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb6fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3fba1b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb6fe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3fba1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb705bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3fba1b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41ae630 con 0x5592bb6ffdf0 2024-09-11T01:05:05.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3fba1b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41b0230 con 0x5592bb6ff4d0 2024-09-11T01:05:05.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3fba1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41aed00 con 0x5592bb70bd80 2024-09-11T01:05:05.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb6fe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb6fe210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38354/0 (socket says 172.21.15.78:38354) 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3f27fc640 1 -- 172.21.15.78:0/831729581 learned_addr learned my addr 172.21.15.78:0/831729581 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/831729581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb6fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3f37fe640 1 --2- 172.21.15.78:0/831729581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb705bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3f27fc640 1 -- 172.21.15.78:0/831729581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 msgr2=0x5592bb705bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3f27fc640 1 --2- 172.21.15.78:0/831729581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb705bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3f27fc640 1 -- 172.21.15.78:0/831729581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 msgr2=0x5592bb6fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3f27fc640 1 --2- 172.21.15.78:0/831729581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb6fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.873+0000 7fd3f27fc640 1 -- 172.21.15.78:0/831729581 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592bb6fdd80 con 0x5592bb6ffdf0 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.874+0000 7fd3f27fc640 1 --2- 172.21.15.78:0/831729581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb6fe210 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009900 tx=0x7fd3dc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea135f3e135a0429 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.874+0000 7fd3e8ff9640 1 -- 172.21.15.78:0/831729581 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc03f040 con 0x5592bb6ffdf0 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.874+0000 7fd3e8ff9640 1 -- 172.21.15.78:0/831729581 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3dc004250 con 0x5592bb6ffdf0 2024-09-11T01:05:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.874+0000 7fd3e8ff9640 1 -- 172.21.15.78:0/831729581 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc003830 con 0x5592bb6ffdf0 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 -- 172.21.15.78:0/831729581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 msgr2=0x5592bb6fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/831729581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb6fe210 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009900 tx=0x7fd3dc02a410 comp rx=0 tx=0).stop 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 -- 172.21.15.78:0/831729581 shutdown_connections 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/831729581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb705bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/831729581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ffdf0 0x5592bb6fe210 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/831729581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ff4d0 0x5592bb6fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 -- 172.21.15.78:0/831729581 >> 172.21.15.78:0/831729581 conn(0x5592bb6fd300 msgr2=0x5592bb707260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 -- 172.21.15.78:0/831729581 shutdown_connections 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 -- 172.21.15.78:0/831729581 wait complete. 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 -- start start 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 Processor -- start 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ff4d0 0x5592bb7141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ffdf0 0x5592bb714760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb7108f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41afaa0 con 0x5592bb6ff4d0 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41af3d0 con 0x5592bb6ffdf0 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41ac610 con 0x5592bb70bd80 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.875+0000 7fd3fba1b740 10 client.0 _ll_register_callbacks cb 0x7fd3dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ff4d0 0x5592bb7141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ff4d0 0x5592bb7141e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38356/0 (socket says 172.21.15.78:38356) 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3f2ffd640 1 -- 172.21.15.78:0/590643195 learned_addr learned my addr 172.21.15.78:0/590643195 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3f27fc640 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ffdf0 0x5592bb714760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3f37fe640 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb7108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3f2ffd640 1 -- 172.21.15.78:0/590643195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 msgr2=0x5592bb7108f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb7108f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3f2ffd640 1 -- 172.21.15.78:0/590643195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ffdf0 msgr2=0x5592bb714760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ffdf0 0x5592bb714760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3f2ffd640 1 -- 172.21.15.78:0/590643195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592bb711120 con 0x5592bb6ff4d0 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3f2ffd640 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ff4d0 0x5592bb7141e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40abe60 tx=0x7fd3e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3077be1d4a2cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3da7fc640 1 -- 172.21.15.78:0/590643195 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e40de7a0 con 0x5592bb6ff4d0 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3da7fc640 1 -- 172.21.15.78:0/590643195 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd3e40e9580 con 0x5592bb6ff4d0 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3fba1b740 1 -- 172.21.15.78:0/590643195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3dc004250 con 0x5592bb6ff4d0 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3da7fc640 1 -- 172.21.15.78:0/590643195 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd3e40db730 con 0x5592bb6ff4d0 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3fba1b740 10 client.16638 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3fba1b740 1 -- 172.21.15.78:0/590643195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5592bb789330 con 0x5592bb6ff4d0 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.876+0000 7fd3da7fc640 1 -- 172.21.15.78:0/590643195 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3dc004250 con 0x5592bb6ff4d0 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.877+0000 7fd3da7fc640 1 -- 172.21.15.78:0/590643195 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fd3e40a83d0 con 0x5592bb6ff4d0 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.877+0000 7fd3fba1b740 10 client.16638 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.877+0000 7fd3fba1b740 10 client.16638 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:05:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.877+0000 7fd3fba1b740 20 client.16638 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.877+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5592bb7895c0 0x5592bb78b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.877+0000 7fd3fba1b740 4 client.16638 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:05:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.877+0000 7fd3fba1b740 1 -- 172.21.15.78:0/590643195 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592bb6738a0 con 0x5592bb7895c0 2024-09-11T01:05:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.877+0000 7fd3f27fc640 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5592bb7895c0 0x5592bb78b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.877+0000 7fd3f27fc640 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5592bb7895c0 0x5592bb78b9b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=37d8a88e02e21fa1 server_cookie=71403054954af256 in_seq=0 out_seq=0 2024-09-11T01:05:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.877+0000 7fd3da7fc640 10 client.16638 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:05:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3da7fc640 1 -- 172.21.15.78:0/590643195 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592bb6738a0 con 0x5592bb7895c0 2024-09-11T01:05:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3da7fc640 10 client.16638 handle_command_reply: tid=0 2024-09-11T01:05:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3fba1b740 1 client.16638 shutdown 2024-09-11T01:05:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3fba1b740 1 -- 172.21.15.78:0/590643195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ff4d0 msgr2=0x5592bb7141e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ff4d0 0x5592bb7141e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40abe60 tx=0x7fd3e4005880 comp rx=0 tx=0).stop 2024-09-11T01:05:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3fba1b740 1 -- 172.21.15.78:0/590643195 shutdown_connections 2024-09-11T01:05:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5592bb7895c0 0x5592bb78b9b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592bb70bd80 0x5592bb7108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592bb6ffdf0 0x5592bb714760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3da7fc640 0 client.16638 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:05:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3fba1b740 1 --2- 172.21.15.78:0/590643195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592bb6ff4d0 0x5592bb7141e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3fba1b740 1 -- 172.21.15.78:0/590643195 >> 172.21.15.78:0/590643195 conn(0x5592bb6fd300 msgr2=0x5592bb70a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3fba1b740 1 -- 172.21.15.78:0/590643195 shutdown_connections 2024-09-11T01:05:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3fba1b740 1 -- 172.21.15.78:0/590643195 wait complete. 2024-09-11T01:05:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3fba1b740 20 client.16638 trim_cache size 0 max 16384 2024-09-11T01:05:05.889 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:05:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3f9646640 1 -- 172.21.15.78:0/3155703217 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd3e00b5450 msgr2=0x7fd3e00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3f9646640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd3e00b5450 0x7fd3e00b7910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00a4c0 tx=0x7fd3ec0096f0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3f9646640 1 -- 172.21.15.78:0/3155703217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3f4102dc0 msgr2=0x7fd3f41992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.878+0000 7fd3f9646640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3f4102dc0 0x7fd3f41992e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400d950 tx=0x7fd3e400de10 comp rx=0 tx=0).stop 2024-09-11T01:05:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.879+0000 7fd3f9646640 1 -- 172.21.15.78:0/3155703217 shutdown_connections 2024-09-11T01:05:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.879+0000 7fd3f9646640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd3e00b5450 0x7fd3e00b7910 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.879+0000 7fd3f9646640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f4109150 0x7fd3f419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.879+0000 7fd3f9646640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3f41087d0 0x7fd3f4199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.879+0000 7fd3f9646640 1 --2- 172.21.15.78:0/3155703217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3f4102dc0 0x7fd3f41992e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.879+0000 7fd3f9646640 1 -- 172.21.15.78:0/3155703217 >> 172.21.15.78:0/3155703217 conn(0x7fd3f40fe830 msgr2=0x7fd3f4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.879+0000 7fd3f9646640 1 -- 172.21.15.78:0/3155703217 shutdown_connections 2024-09-11T01:05:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:05.879+0000 7fd3f9646640 1 -- 172.21.15.78:0/3155703217 wait complete. 2024-09-11T01:05:06.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 463595 5.83 MB/sec execute 194 sec latency 2273.725 ms 2024-09-11T01:05:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:06 smithi078 ceph-mon[31315]: pgmap v510: 97 pgs: 97 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-11T01:05:06.607 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:05:06.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:06 smithi110 ceph-mon[34712]: pgmap v510: 97 pgs: 97 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-11T01:05:06.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:06 smithi112 ceph-mon[35384]: pgmap v510: 97 pgs: 97 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-11T01:05:06.853 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:07.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.395+0000 7f25229b7640 1 -- 172.21.15.78:0/3431849006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251c108020 msgr2=0x7f251c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:07.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.395+0000 7f25229b7640 1 --2- 172.21.15.78:0/3431849006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251c108020 0x7f251c10e820 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2514009ca0 tx=0x7f251402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.395+0000 7f25229b7640 1 -- 172.21.15.78:0/3431849006 shutdown_connections 2024-09-11T01:05:07.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.395+0000 7f25229b7640 1 --2- 172.21.15.78:0/3431849006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251c108020 0x7f251c10e820 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.395+0000 7f25229b7640 1 --2- 172.21.15.78:0/3431849006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f251c1076a0 0x7f251c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.395+0000 7f25229b7640 1 --2- 172.21.15.78:0/3431849006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f251c101c90 0x7f251c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.395+0000 7f25229b7640 1 -- 172.21.15.78:0/3431849006 >> 172.21.15.78:0/3431849006 conn(0x7f251c0fd700 msgr2=0x7f251c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:07.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.395+0000 7f25229b7640 1 -- 172.21.15.78:0/3431849006 shutdown_connections 2024-09-11T01:05:07.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.395+0000 7f25229b7640 1 -- 172.21.15.78:0/3431849006 wait complete. 2024-09-11T01:05:07.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.396+0000 7f25229b7640 1 Processor -- start 2024-09-11T01:05:07.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.396+0000 7f25229b7640 1 -- start start 2024-09-11T01:05:07.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.396+0000 7f25229b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f251c101c90 0x7f251c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.396+0000 7f25229b7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f251c1076a0 0x7f251c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.396+0000 7f25229b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251c108020 0x7f251c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.396+0000 7f25229b7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f251c111ce0 con 0x7f251c108020 2024-09-11T01:05:07.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.396+0000 7f25229b7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f251c111b60 con 0x7f251c1076a0 2024-09-11T01:05:07.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.396+0000 7f25229b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f251c111e60 con 0x7f251c101c90 2024-09-11T01:05:07.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.396+0000 7f2520f54640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251c108020 0x7f251c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.396+0000 7f251b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f251c1076a0 0x7f251c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.396+0000 7f2520f54640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251c108020 0x7f251c19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38376/0 (socket says 172.21.15.78:38376) 2024-09-11T01:05:07.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.396+0000 7f2520f54640 1 -- 172.21.15.78:0/745780915 learned_addr learned my addr 172.21.15.78:0/745780915 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:07.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.397+0000 7f251bfff640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f251c101c90 0x7f251c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.397+0000 7f251b7fe640 1 -- 172.21.15.78:0/745780915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f251c101c90 msgr2=0x7f251c19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:07.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.397+0000 7f251b7fe640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f251c101c90 0x7f251c19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.397+0000 7f251b7fe640 1 -- 172.21.15.78:0/745780915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251c108020 msgr2=0x7f251c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:07.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.397+0000 7f251b7fe640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251c108020 0x7f251c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.397+0000 7f251b7fe640 1 -- 172.21.15.78:0/745780915 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c1a5410 con 0x7f251c1076a0 2024-09-11T01:05:07.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.397+0000 7f251b7fe640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f251c1076a0 0x7f251c19a550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f251000e9f0 tx=0x7f251000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:07.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.397+0000 7f25197fa640 1 -- 172.21.15.78:0/745780915 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251000ccc0 con 0x7f251c1076a0 2024-09-11T01:05:07.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.397+0000 7f25229b7640 1 -- 172.21.15.78:0/745780915 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251c1a5700 con 0x7f251c1076a0 2024-09-11T01:05:07.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.397+0000 7f25197fa640 1 -- 172.21.15.78:0/745780915 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25100108e0 con 0x7f251c1076a0 2024-09-11T01:05:07.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.397+0000 7f25197fa640 1 -- 172.21.15.78:0/745780915 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510018880 con 0x7f251c1076a0 2024-09-11T01:05:07.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.397+0000 7f25229b7640 1 -- 172.21.15.78:0/745780915 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251c1a5c90 con 0x7f251c1076a0 2024-09-11T01:05:07.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f2524d8c740 1 Processor -- start 2024-09-11T01:05:07.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f2524d8c740 1 -- start start 2024-09-11T01:05:07.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f25197fa640 1 -- 172.21.15.78:0/745780915 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2510010430 con 0x7f251c1076a0 2024-09-11T01:05:07.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f2524d8c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 0x560e6d8c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f2524d8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c7df0 0x560e6d8d7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f2524d8c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8d8190 0x560e6d8da580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f2524d8c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560e6d83dea0 con 0x560e6d8c74d0 2024-09-11T01:05:07.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f2524d8c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560e6d82d2d0 con 0x560e6d8d8190 2024-09-11T01:05:07.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f2524d8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560e6d8310e0 con 0x560e6d8c7df0 2024-09-11T01:05:07.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f2520f54640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 0x560e6d8c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f251bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c7df0 0x560e6d8d7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f2520f54640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 0x560e6d8c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38378/0 (socket says 172.21.15.78:38378) 2024-09-11T01:05:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f2520f54640 1 -- 172.21.15.78:0/1171500876 learned_addr learned my addr 172.21.15.78:0/1171500876 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.400+0000 7f251b7fe640 1 --2- 172.21.15.78:0/1171500876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8d8190 0x560e6d8da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.401+0000 7f2520f54640 1 -- 172.21.15.78:0/1171500876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c7df0 msgr2=0x560e6d8d7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.401+0000 7f2520f54640 1 --2- 172.21.15.78:0/1171500876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c7df0 0x560e6d8d7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.401+0000 7f2520f54640 1 -- 172.21.15.78:0/1171500876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8d8190 msgr2=0x560e6d8da580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.401+0000 7f2520f54640 1 --2- 172.21.15.78:0/1171500876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8d8190 0x560e6d8da580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.401+0000 7f2520f54640 1 -- 172.21.15.78:0/1171500876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e6d58b650 con 0x560e6d8c74d0 2024-09-11T01:05:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.401+0000 7f2520f54640 1 --2- 172.21.15.78:0/1171500876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 0x560e6d8c78b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2514009430 tx=0x7f2514005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd76f7e5e576a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.401+0000 7f25027fc640 1 -- 172.21.15.78:0/1171500876 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514031cf0 con 0x560e6d8c74d0 2024-09-11T01:05:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.401+0000 7f25027fc640 1 -- 172.21.15.78:0/1171500876 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2514036030 con 0x560e6d8c74d0 2024-09-11T01:05:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.401+0000 7f25027fc640 1 -- 172.21.15.78:0/1171500876 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25140042a0 con 0x560e6d8c74d0 2024-09-11T01:05:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.401+0000 7f25197fa640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f25040b5450 0x7f25040b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.401+0000 7f251bfff640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f25040b5450 0x7f25040b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.401+0000 7f25197fa640 1 -- 172.21.15.78:0/745780915 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f25100e6290 con 0x7f251c1076a0 2024-09-11T01:05:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.402+0000 7f2524d8c740 1 -- 172.21.15.78:0/1171500876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 msgr2=0x560e6d8c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.402+0000 7f251bfff640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f25040b5450 0x7f25040b7910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x560e6d8c61e0 tx=0x7f250c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.402+0000 7f2524d8c740 1 --2- 172.21.15.78:0/1171500876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 0x560e6d8c78b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2514009430 tx=0x7f2514005ef0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.402+0000 7f2524d8c740 1 -- 172.21.15.78:0/1171500876 shutdown_connections 2024-09-11T01:05:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.402+0000 7f2524d8c740 1 --2- 172.21.15.78:0/1171500876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8d8190 0x560e6d8da580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.402+0000 7f2524d8c740 1 --2- 172.21.15.78:0/1171500876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c7df0 0x560e6d8d7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.402+0000 7f2524d8c740 1 --2- 172.21.15.78:0/1171500876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 0x560e6d8c78b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.402+0000 7f2524d8c740 1 -- 172.21.15.78:0/1171500876 >> 172.21.15.78:0/1171500876 conn(0x560e6d8c5300 msgr2=0x560e6d8c64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:07.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.402+0000 7f2524d8c740 1 -- 172.21.15.78:0/1171500876 shutdown_connections 2024-09-11T01:05:07.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.402+0000 7f2524d8c740 1 -- 172.21.15.78:0/1171500876 wait complete. 2024-09-11T01:05:07.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.402+0000 7f2524d8c740 1 -- start start 2024-09-11T01:05:07.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f2524d8c740 1 Processor -- start 2024-09-11T01:05:07.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f2524d8c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 0x560e6d8d7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f2524d8c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8d4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f2524d8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8d8190 0x560e6d8d55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f251bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 0x560e6d8d7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f2524d8c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560e6d68e700 con 0x560e6d8c74d0 2024-09-11T01:05:07.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f2524d8c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560e6d7c8160 con 0x560e6d8c7df0 2024-09-11T01:05:07.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f251bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 0x560e6d8d7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38382/0 (socket says 172.21.15.78:38382) 2024-09-11T01:05:07.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f2524d8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560e6d8dac30 con 0x560e6d8d8190 2024-09-11T01:05:07.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f251bfff640 1 -- 172.21.15.78:0/656465240 learned_addr learned my addr 172.21.15.78:0/656465240 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:07.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f251b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8d4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f2524d8c740 10 client.0 _ll_register_callbacks cb 0x560e6d7e19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:07.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f2520f54640 1 --2- 172.21.15.78:0/656465240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8d8190 0x560e6d8d55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f251bfff640 1 -- 172.21.15.78:0/656465240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8d8190 msgr2=0x560e6d8d55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:07.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f251bfff640 1 --2- 172.21.15.78:0/656465240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8d8190 0x560e6d8d55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f251bfff640 1 -- 172.21.15.78:0/656465240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 msgr2=0x560e6d8d4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:07.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f251bfff640 1 --2- 172.21.15.78:0/656465240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8d4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.403+0000 7f251bfff640 1 -- 172.21.15.78:0/656465240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e6d7f5c90 con 0x560e6d8c74d0 2024-09-11T01:05:07.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f251bfff640 1 --2- 172.21.15.78:0/656465240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 0x560e6d8d7ad0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f250c0045d0 tx=0x7f250c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e714eae080408590 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:07.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f24fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:07.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f24fb7fe640 1 -- 172.21.15.78:0/656465240 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c009e60 con 0x560e6d8c74d0 2024-09-11T01:05:07.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f24fb7fe640 1 -- 172.21.15.78:0/656465240 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f250c01f640 con 0x560e6d8c74d0 2024-09-11T01:05:07.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f2524d8c740 1 -- 172.21.15.78:0/656465240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2514036030 con 0x560e6d8c74d0 2024-09-11T01:05:07.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f24fb7fe640 1 -- 172.21.15.78:0/656465240 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f250c01f840 con 0x560e6d8c74d0 2024-09-11T01:05:07.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f2524d8c740 10 client.16650 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:07.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f2524d8c740 1 -- 172.21.15.78:0/656465240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560e6d8e6610 con 0x560e6d8c74d0 2024-09-11T01:05:07.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f24fb7fe640 1 -- 172.21.15.78:0/656465240 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2514036030 con 0x560e6d8c74d0 2024-09-11T01:05:07.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f24fb7fe640 1 -- 172.21.15.78:0/656465240 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f250c027070 con 0x560e6d8c74d0 2024-09-11T01:05:07.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f2524d8c740 10 client.16650 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:07.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f2524d8c740 10 client.16650 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:05:07.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f2524d8c740 20 client.16650 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:07.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f2524d8c740 1 --2- 172.21.15.78:0/656465240 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x560e6d8e7d50 0x560e6d8ea140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f2524d8c740 4 client.16650 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:07.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.404+0000 7f2524d8c740 1 -- 172.21.15.78:0/656465240 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e6d83fdf0 con 0x560e6d8e7d50 2024-09-11T01:05:07.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.405+0000 7f251b7fe640 1 --2- 172.21.15.78:0/656465240 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x560e6d8e7d50 0x560e6d8ea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.405+0000 7f24fb7fe640 10 client.16650 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:05:07.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.405+0000 7f251b7fe640 1 --2- 172.21.15.78:0/656465240 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x560e6d8e7d50 0x560e6d8ea140 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ca1fcbe4b29bf0 server_cookie=b0a5f984feabf1d5 in_seq=0 out_seq=0 2024-09-11T01:05:07.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f24fb7fe640 1 -- 172.21.15.78:0/656465240 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e6d83fdf0 con 0x560e6d8e7d50 2024-09-11T01:05:07.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f24fb7fe640 10 client.16650 handle_command_reply: tid=0 2024-09-11T01:05:07.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f2524d8c740 1 client.16650 shutdown 2024-09-11T01:05:07.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f2524d8c740 1 -- 172.21.15.78:0/656465240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 msgr2=0x560e6d8d7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:07.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f2524d8c740 1 --2- 172.21.15.78:0/656465240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 0x560e6d8d7ad0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f250c0045d0 tx=0x7f250c004730 comp rx=0 tx=0).stop 2024-09-11T01:05:07.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f2524d8c740 1 -- 172.21.15.78:0/656465240 shutdown_connections 2024-09-11T01:05:07.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f2524d8c740 1 --2- 172.21.15.78:0/656465240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8d8190 0x560e6d8d55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f2524d8c740 1 --2- 172.21.15.78:0/656465240 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x560e6d8e7d50 0x560e6d8ea140 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f2524d8c740 1 --2- 172.21.15.78:0/656465240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8d4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f2524d8c740 1 --2- 172.21.15.78:0/656465240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8c74d0 0x560e6d8d7ad0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f2524d8c740 1 -- 172.21.15.78:0/656465240 >> 172.21.15.78:0/656465240 conn(0x560e6d8c5300 msgr2=0x560e6d8cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:07.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f24fb7fe640 0 client.16650 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:05:07.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f2524d8c740 1 -- 172.21.15.78:0/656465240 shutdown_connections 2024-09-11T01:05:07.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.406+0000 7f2524d8c740 1 -- 172.21.15.78:0/656465240 wait complete. 2024-09-11T01:05:07.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.407+0000 7f2524d8c740 20 client.16650 trim_cache size 0 max 16384 2024-09-11T01:05:07.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.420+0000 7f2524d8c740 1 Processor -- start 2024-09-11T01:05:07.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.420+0000 7f2524d8c740 1 -- start start 2024-09-11T01:05:07.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2524d8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c74d0 0x560e6d8e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2524d8c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8ea140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2524d8c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8d8190 0x560e6d8d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2524d8c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1ac2c0 con 0x560e6d8d8190 2024-09-11T01:05:07.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2524d8c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1adec0 con 0x560e6d8c7df0 2024-09-11T01:05:07.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2524d8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1ac990 con 0x560e6d8c74d0 2024-09-11T01:05:07.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2520f54640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8d8190 0x560e6d8d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2520f54640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8d8190 0x560e6d8d37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38386/0 (socket says 172.21.15.78:38386) 2024-09-11T01:05:07.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2520f54640 1 -- 172.21.15.78:0/863316789 learned_addr learned my addr 172.21.15.78:0/863316789 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:07.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f251b7fe640 1 --2- 172.21.15.78:0/863316789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8ea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f251bfff640 1 --2- 172.21.15.78:0/863316789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c74d0 0x560e6d8e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2520f54640 1 -- 172.21.15.78:0/863316789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c74d0 msgr2=0x560e6d8e3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:07.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2520f54640 1 --2- 172.21.15.78:0/863316789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c74d0 0x560e6d8e3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2520f54640 1 -- 172.21.15.78:0/863316789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 msgr2=0x560e6d8ea140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:07.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2520f54640 1 --2- 172.21.15.78:0/863316789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8ea140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2520f54640 1 -- 172.21.15.78:0/863316789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e6d8c6a10 con 0x560e6d8d8190 2024-09-11T01:05:07.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f251b7fe640 1 --2- 172.21.15.78:0/863316789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8ea140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:07.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2520f54640 1 --2- 172.21.15.78:0/863316789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8d8190 0x560e6d8d37f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2514005f60 tx=0x7f2514003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2338fb5b5718e0bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:07.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.421+0000 7f2501ffb640 1 -- 172.21.15.78:0/863316789 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514053070 con 0x560e6d8d8190 2024-09-11T01:05:07.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.422+0000 7f2501ffb640 1 -- 172.21.15.78:0/863316789 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2514036040 con 0x560e6d8d8190 2024-09-11T01:05:07.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.422+0000 7f2501ffb640 1 -- 172.21.15.78:0/863316789 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251404a040 con 0x560e6d8d8190 2024-09-11T01:05:07.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.422+0000 7f2524d8c740 1 -- 172.21.15.78:0/863316789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8d8190 msgr2=0x560e6d8d37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:07.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.422+0000 7f2524d8c740 1 --2- 172.21.15.78:0/863316789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8d8190 0x560e6d8d37f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2514005f60 tx=0x7f2514003cc0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.422+0000 7f2524d8c740 1 -- 172.21.15.78:0/863316789 shutdown_connections 2024-09-11T01:05:07.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.422+0000 7f2524d8c740 1 --2- 172.21.15.78:0/863316789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8d8190 0x560e6d8d37f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.422+0000 7f2524d8c740 1 --2- 172.21.15.78:0/863316789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8ea140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.422+0000 7f2524d8c740 1 --2- 172.21.15.78:0/863316789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c74d0 0x560e6d8e3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.422+0000 7f2524d8c740 1 -- 172.21.15.78:0/863316789 >> 172.21.15.78:0/863316789 conn(0x560e6d8c5300 msgr2=0x560e6d8d9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:07.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.422+0000 7f2524d8c740 1 -- 172.21.15.78:0/863316789 shutdown_connections 2024-09-11T01:05:07.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.422+0000 7f2524d8c740 1 -- 172.21.15.78:0/863316789 wait complete. 2024-09-11T01:05:07.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.423+0000 7f2524d8c740 1 -- start start 2024-09-11T01:05:07.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.423+0000 7f2524d8c740 1 Processor -- start 2024-09-11T01:05:07.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.423+0000 7f2524d8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c74d0 0x560e6d8e00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.423+0000 7f2524d8c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8e05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.423+0000 7f2524d8c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8d8190 0x560e6d8dae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.423+0000 7f2524d8c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1ad730 con 0x560e6d8d8190 2024-09-11T01:05:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.423+0000 7f2524d8c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1ad060 con 0x560e6d8c7df0 2024-09-11T01:05:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.423+0000 7f2524d8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1aa2a0 con 0x560e6d8c74d0 2024-09-11T01:05:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.423+0000 7f2524d8c740 10 client.0 _ll_register_callbacks cb 0x7f250c009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.423+0000 7f251bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c74d0 0x560e6d8e00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f251b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8e05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f251bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c74d0 0x560e6d8e00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:34044/0 (socket says 172.21.15.78:34044) 2024-09-11T01:05:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f251bfff640 1 -- 172.21.15.78:0/1521016752 learned_addr learned my addr 172.21.15.78:0/1521016752 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f2520f54640 1 --2- 172.21.15.78:0/1521016752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8d8190 0x560e6d8dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f251bfff640 1 -- 172.21.15.78:0/1521016752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 msgr2=0x560e6d8e05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:07.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f251bfff640 1 --2- 172.21.15.78:0/1521016752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8e05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f251bfff640 1 -- 172.21.15.78:0/1521016752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8d8190 msgr2=0x560e6d8dae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:07.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f251bfff640 1 --2- 172.21.15.78:0/1521016752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8d8190 0x560e6d8dae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f251bfff640 1 -- 172.21.15.78:0/1521016752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e6d8db570 con 0x560e6d8c74d0 2024-09-11T01:05:07.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f251bfff640 1 --2- 172.21.15.78:0/1521016752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c74d0 0x560e6d8e00b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f250c029c80 tx=0x7f250c029b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ae4c90cf904740e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:07.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f24fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:05:07.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f24fb7fe640 1 -- 172.21.15.78:0/1521016752 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c036070 con 0x560e6d8c74d0 2024-09-11T01:05:07.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f2524d8c740 1 -- 172.21.15.78:0/1521016752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2514036040 con 0x560e6d8c74d0 2024-09-11T01:05:07.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f24fb7fe640 1 -- 172.21.15.78:0/1521016752 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f250c01fbf0 con 0x560e6d8c74d0 2024-09-11T01:05:07.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f24fb7fe640 1 -- 172.21.15.78:0/1521016752 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f250c009e60 con 0x560e6d8c74d0 2024-09-11T01:05:07.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f2524d8c740 10 client.25933 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:07.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.424+0000 7f2524d8c740 1 -- 172.21.15.78:0/1521016752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560e6d955410 con 0x560e6d8c74d0 2024-09-11T01:05:07.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.425+0000 7f24fb7fe640 1 -- 172.21.15.78:0/1521016752 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2514036040 con 0x560e6d8c74d0 2024-09-11T01:05:07.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.425+0000 7f24fb7fe640 1 -- 172.21.15.78:0/1521016752 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f250c043020 con 0x560e6d8c74d0 2024-09-11T01:05:07.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.425+0000 7f2524d8c740 10 client.25933 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:07.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.425+0000 7f2524d8c740 10 client.25933 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:05:07.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.425+0000 7f2524d8c740 20 client.25933 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:07.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.425+0000 7f2524d8c740 1 --2- 172.21.15.78:0/1521016752 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x560e6d955b70 0x560e6d957f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:07.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.425+0000 7f2524d8c740 4 client.25933 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:05:07.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.425+0000 7f2524d8c740 1 -- 172.21.15.78:0/1521016752 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e6d83b8a0 con 0x560e6d955b70 2024-09-11T01:05:07.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.425+0000 7f251b7fe640 1 --2- 172.21.15.78:0/1521016752 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x560e6d955b70 0x560e6d957f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:07.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.425+0000 7f24fb7fe640 10 client.25933 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:05:07.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.425+0000 7f251b7fe640 1 --2- 172.21.15.78:0/1521016752 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x560e6d955b70 0x560e6d957f90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7860db625c723b2 server_cookie=c6c188177c98b9a8 in_seq=0 out_seq=0 2024-09-11T01:05:07.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f24fb7fe640 1 -- 172.21.15.78:0/1521016752 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e6d83b8a0 con 0x560e6d955b70 2024-09-11T01:05:07.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f24fb7fe640 10 client.25933 handle_command_reply: tid=0 2024-09-11T01:05:07.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f2524d8c740 1 client.25933 shutdown 2024-09-11T01:05:07.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f2524d8c740 1 -- 172.21.15.78:0/1521016752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c74d0 msgr2=0x560e6d8e00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:07.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f2524d8c740 1 --2- 172.21.15.78:0/1521016752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c74d0 0x560e6d8e00b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f250c029c80 tx=0x7f250c029b50 comp rx=0 tx=0).stop 2024-09-11T01:05:07.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f2524d8c740 1 -- 172.21.15.78:0/1521016752 shutdown_connections 2024-09-11T01:05:07.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f2524d8c740 1 --2- 172.21.15.78:0/1521016752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e6d8d8190 0x560e6d8dae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f2524d8c740 1 --2- 172.21.15.78:0/1521016752 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x560e6d955b70 0x560e6d957f90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f2524d8c740 1 --2- 172.21.15.78:0/1521016752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e6d8c7df0 0x560e6d8e05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f2524d8c740 1 --2- 172.21.15.78:0/1521016752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e6d8c74d0 0x560e6d8e00b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f2524d8c740 1 -- 172.21.15.78:0/1521016752 >> 172.21.15.78:0/1521016752 conn(0x560e6d8c5300 msgr2=0x560e6d8d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:07.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f24fb7fe640 0 client.25933 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:05:07.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f2524d8c740 1 -- 172.21.15.78:0/1521016752 shutdown_connections 2024-09-11T01:05:07.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f2524d8c740 1 -- 172.21.15.78:0/1521016752 wait complete. 2024-09-11T01:05:07.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.426+0000 7f2524d8c740 20 client.25933 trim_cache size 0 max 16384 2024-09-11T01:05:07.451 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:05:07.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.427+0000 7f25229b7640 1 -- 172.21.15.78:0/745780915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f25040b5450 msgr2=0x7f25040b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:07.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.427+0000 7f25229b7640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f25040b5450 0x7f25040b7910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x560e6d8c61e0 tx=0x7f250c005f90 comp rx=0 tx=0).stop 2024-09-11T01:05:07.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.427+0000 7f25229b7640 1 -- 172.21.15.78:0/745780915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f251c1076a0 msgr2=0x7f251c19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:07.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.427+0000 7f25229b7640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f251c1076a0 0x7f251c19a550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f251000e9f0 tx=0x7f251000eeb0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.427+0000 7f25229b7640 1 -- 172.21.15.78:0/745780915 shutdown_connections 2024-09-11T01:05:07.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.427+0000 7f25229b7640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f25040b5450 0x7f25040b7910 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.427+0000 7f25229b7640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251c108020 0x7f251c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.427+0000 7f25229b7640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f251c1076a0 0x7f251c19a550 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.427+0000 7f25229b7640 1 --2- 172.21.15.78:0/745780915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f251c101c90 0x7f251c19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:07.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.427+0000 7f25229b7640 1 -- 172.21.15.78:0/745780915 >> 172.21.15.78:0/745780915 conn(0x7f251c0fd700 msgr2=0x7f251c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:07.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.427+0000 7f25229b7640 1 -- 172.21.15.78:0/745780915 shutdown_connections 2024-09-11T01:05:07.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:07.427+0000 7f25229b7640 1 -- 172.21.15.78:0/745780915 wait complete. 2024-09-11T01:05:07.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 464271 5.80 MB/sec execute 195 sec latency 3067.825 ms 2024-09-11T01:05:07.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:07 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:05:07.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:07 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:05:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:07 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:05:08.127 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:05:08.407 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:08.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 465887 5.84 MB/sec execute 196 sec latency 12.910 ms 2024-09-11T01:05:08.837 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:08 smithi078 ceph-mon[31315]: pgmap v511: 97 pgs: 97 active+clean; 285 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-09-11T01:05:08.837 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T01:05:08.837 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T01:05:08.837 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T01:05:08.838 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T01:05:08.838 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:05:08.838 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:05:08.838 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:08 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:05:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.923+0000 7f26adc68640 1 -- 172.21.15.78:0/3943422668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26a8108020 msgr2=0x7f26a810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.923+0000 7f26adc68640 1 --2- 172.21.15.78:0/3943422668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26a8108020 0x7f26a810e820 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009900 tx=0x7f26a002a410 comp rx=0 tx=0).stop 2024-09-11T01:05:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.923+0000 7f26adc68640 1 -- 172.21.15.78:0/3943422668 shutdown_connections 2024-09-11T01:05:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.923+0000 7f26adc68640 1 --2- 172.21.15.78:0/3943422668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26a8108020 0x7f26a810e820 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.923+0000 7f26adc68640 1 --2- 172.21.15.78:0/3943422668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26a81076a0 0x7f26a8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.923+0000 7f26adc68640 1 --2- 172.21.15.78:0/3943422668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26a8101c90 0x7f26a8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.924+0000 7f26adc68640 1 -- 172.21.15.78:0/3943422668 >> 172.21.15.78:0/3943422668 conn(0x7f26a80fd700 msgr2=0x7f26a80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.924+0000 7f26adc68640 1 -- 172.21.15.78:0/3943422668 shutdown_connections 2024-09-11T01:05:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.924+0000 7f26adc68640 1 -- 172.21.15.78:0/3943422668 wait complete. 2024-09-11T01:05:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.924+0000 7f26adc68640 1 Processor -- start 2024-09-11T01:05:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.924+0000 7f26adc68640 1 -- start start 2024-09-11T01:05:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26adc68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26a8101c90 0x7f26a8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26adc68640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26a81076a0 0x7f26a81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26adc68640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26a8108020 0x7f26a8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26adc68640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f26a8111ca0 con 0x7f26a8108020 2024-09-11T01:05:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26adc68640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26a8111b20 con 0x7f26a81076a0 2024-09-11T01:05:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26adc68640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26a8111e20 con 0x7f26a8101c90 2024-09-11T01:05:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26a8101c90 0x7f26a8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26a8108020 0x7f26a8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26a81076a0 0x7f26a81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26a8101c90 0x7f26a8197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:34072/0 (socket says 172.21.15.78:34072) 2024-09-11T01:05:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a77fe640 1 -- 172.21.15.78:0/2535284950 learned_addr learned my addr 172.21.15.78:0/2535284950 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a77fe640 1 -- 172.21.15.78:0/2535284950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26a81076a0 msgr2=0x7f26a81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a77fe640 1 --2- 172.21.15.78:0/2535284950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26a81076a0 0x7f26a81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a77fe640 1 -- 172.21.15.78:0/2535284950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26a8108020 msgr2=0x7f26a8198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a77fe640 1 --2- 172.21.15.78:0/2535284950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26a8108020 0x7f26a8198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a77fe640 1 -- 172.21.15.78:0/2535284950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a81a7660 con 0x7f26a8101c90 2024-09-11T01:05:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a77fe640 1 --2- 172.21.15.78:0/2535284950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26a8101c90 0x7f26a8197e80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f269800ebc0 tx=0x7f269800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a4ff9640 1 -- 172.21.15.78:0/2535284950 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698010630 con 0x7f26a8101c90 2024-09-11T01:05:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a4ff9640 1 -- 172.21.15.78:0/2535284950 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2698010c80 con 0x7f26a8101c90 2024-09-11T01:05:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26a4ff9640 1 -- 172.21.15.78:0/2535284950 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698018b60 con 0x7f26a8101c90 2024-09-11T01:05:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.925+0000 7f26adc68640 1 -- 172.21.15.78:0/2535284950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a81a7950 con 0x7f26a8101c90 2024-09-11T01:05:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.926+0000 7f26adc68640 1 -- 172.21.15.78:0/2535284950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a81a7ee0 con 0x7f26a8101c90 2024-09-11T01:05:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.928+0000 7f26b003d740 1 Processor -- start 2024-09-11T01:05:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.928+0000 7f26b003d740 1 -- start start 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.928+0000 7f26b003d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f612d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.928+0000 7f26a4ff9640 1 -- 172.21.15.78:0/2535284950 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f26980107d0 con 0x7f26a8101c90 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.928+0000 7f26b003d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612da10 0x5606f6139460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.928+0000 7f26b003d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f61399a0 0x5606f613bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.928+0000 7f26b003d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5606f60a3ea0 con 0x5606f612d0f0 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.928+0000 7f26b003d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5606f60932d0 con 0x5606f612da10 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.928+0000 7f26b003d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5606f60970e0 con 0x5606f61399a0 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.928+0000 7f26a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f612d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612da10 0x5606f6139460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f26a6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f61399a0 0x5606f613bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f26a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f612d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38418/0 (socket says 172.21.15.78:38418) 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f26a7fff640 1 -- 172.21.15.78:0/2177492482 learned_addr learned my addr 172.21.15.78:0/2177492482 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f26a6ffd640 1 -- 172.21.15.78:0/2177492482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612da10 msgr2=0x5606f6139460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f26a6ffd640 1 --2- 172.21.15.78:0/2177492482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612da10 0x5606f6139460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f26a6ffd640 1 -- 172.21.15.78:0/2177492482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 msgr2=0x5606f612d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f26a6ffd640 1 --2- 172.21.15.78:0/2177492482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f612d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f26a6ffd640 1 -- 172.21.15.78:0/2177492482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f5df1650 con 0x5606f61399a0 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f26a77fe640 1 --2- 172.21.15.78:0/2177492482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612da10 0x5606f6139460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f26a6ffd640 1 --2- 172.21.15.78:0/2177492482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f61399a0 0x5606f613bd90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f269c009bd0 tx=0x7f269c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6e3b83ca541cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f268dffb640 1 -- 172.21.15.78:0/2177492482 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c004230 con 0x5606f61399a0 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f26a4ff9640 1 --2- 172.21.15.78:0/2535284950 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26900b54d0 0x7f26900b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f268dffb640 1 -- 172.21.15.78:0/2177492482 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f269c009e40 con 0x5606f61399a0 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.929+0000 7f268dffb640 1 -- 172.21.15.78:0/2177492482 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c004ca0 con 0x5606f61399a0 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.930+0000 7f26a7fff640 1 --2- 172.21.15.78:0/2535284950 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26900b54d0 0x7f26900b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.930+0000 7f26a4ff9640 1 -- 172.21.15.78:0/2535284950 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f26980e64d0 con 0x7f26a8101c90 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.930+0000 7f26a7fff640 1 --2- 172.21.15.78:0/2535284950 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26900b54d0 0x7f26900b7990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f26a00096c0 tx=0x7f26a00058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.930+0000 7f26b003d740 1 -- 172.21.15.78:0/2177492482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f61399a0 msgr2=0x5606f613bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.930+0000 7f26b003d740 1 --2- 172.21.15.78:0/2177492482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f61399a0 0x5606f613bd90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f269c009bd0 tx=0x7f269c011810 comp rx=0 tx=0).stop 2024-09-11T01:05:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.930+0000 7f26b003d740 1 -- 172.21.15.78:0/2177492482 shutdown_connections 2024-09-11T01:05:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.930+0000 7f26b003d740 1 --2- 172.21.15.78:0/2177492482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f61399a0 0x5606f613bd90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.930+0000 7f26b003d740 1 --2- 172.21.15.78:0/2177492482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612da10 0x5606f6139460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.930+0000 7f26b003d740 1 --2- 172.21.15.78:0/2177492482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f612d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.930+0000 7f26b003d740 1 -- 172.21.15.78:0/2177492482 >> 172.21.15.78:0/2177492482 conn(0x5606f612af20 msgr2=0x5606f612c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.930+0000 7f26b003d740 1 -- 172.21.15.78:0/2177492482 shutdown_connections 2024-09-11T01:05:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.930+0000 7f26b003d740 1 -- 172.21.15.78:0/2177492482 wait complete. 2024-09-11T01:05:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26b003d740 1 -- start start 2024-09-11T01:05:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26b003d740 1 Processor -- start 2024-09-11T01:05:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26b003d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f6139290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26b003d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612da10 0x5606f61366e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26b003d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f61399a0 0x5606f6136d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26b003d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5606f5ef4700 con 0x5606f612d0f0 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26b003d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5606f602e160 con 0x5606f612da10 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26b003d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5606f613c620 con 0x5606f61399a0 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26b003d740 10 client.0 _ll_register_callbacks cb 0x5606f60479e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f6139290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f6139290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38430/0 (socket says 172.21.15.78:38430) 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26a77fe640 1 -- 172.21.15.78:0/1895840169 learned_addr learned my addr 172.21.15.78:0/1895840169 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26a6ffd640 1 --2- 172.21.15.78:0/1895840169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612da10 0x5606f61366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26a77fe640 1 -- 172.21.15.78:0/1895840169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f61399a0 msgr2=0x5606f6136d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.931+0000 7f26a77fe640 1 --2- 172.21.15.78:0/1895840169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f61399a0 0x5606f6136d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26a77fe640 1 -- 172.21.15.78:0/1895840169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612da10 msgr2=0x5606f61366e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26a77fe640 1 --2- 172.21.15.78:0/1895840169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612da10 0x5606f61366e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26a77fe640 1 -- 172.21.15.78:0/1895840169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f605bc90 con 0x5606f612d0f0 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26a77fe640 1 --2- 172.21.15.78:0/1895840169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f6139290 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f269800cba0 tx=0x7f2698009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139fee768341e789 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f2686ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f2686ffd640 1 -- 172.21.15.78:0/1895840169 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980a8050 con 0x5606f612d0f0 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f2686ffd640 1 -- 172.21.15.78:0/1895840169 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f26980e6020 con 0x5606f612d0f0 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26b003d740 1 -- 172.21.15.78:0/1895840169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5606f60a5df0 con 0x5606f612d0f0 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f2686ffd640 1 -- 172.21.15.78:0/1895840169 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f26980a90c0 con 0x5606f612d0f0 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26b003d740 10 client.16668 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26b003d740 1 -- 172.21.15.78:0/1895840169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5606f6147fb0 con 0x5606f612d0f0 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f2686ffd640 1 -- 172.21.15.78:0/1895840169 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5606f60a5df0 con 0x5606f612d0f0 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f2686ffd640 1 -- 172.21.15.78:0/1895840169 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f26980b10f0 con 0x5606f612d0f0 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26b003d740 10 client.16668 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26b003d740 10 client.16668 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:05:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26b003d740 20 client.16668 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26b003d740 1 --2- 172.21.15.78:0/1895840169 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5606f61494c0 0x5606f614b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26b003d740 4 client.16668 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.932+0000 7f26b003d740 1 -- 172.21.15.78:0/1895840169 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606f60a18a0 con 0x5606f61494c0 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.933+0000 7f26a6ffd640 1 --2- 172.21.15.78:0/1895840169 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5606f61494c0 0x5606f614b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.933+0000 7f26a6ffd640 1 --2- 172.21.15.78:0/1895840169 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5606f61494c0 0x5606f614b910 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72f494c5e2d61165 server_cookie=3800372c4c4c751b in_seq=0 out_seq=0 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.933+0000 7f2686ffd640 10 client.16668 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f2686ffd640 1 -- 172.21.15.78:0/1895840169 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606f60a18a0 con 0x5606f61494c0 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f2686ffd640 10 client.16668 handle_command_reply: tid=0 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f26b003d740 1 client.16668 shutdown 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f26b003d740 1 -- 172.21.15.78:0/1895840169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 msgr2=0x5606f6139290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f26b003d740 1 --2- 172.21.15.78:0/1895840169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f6139290 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f269800cba0 tx=0x7f2698009200 comp rx=0 tx=0).stop 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f26b003d740 1 -- 172.21.15.78:0/1895840169 shutdown_connections 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f26b003d740 1 --2- 172.21.15.78:0/1895840169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f61399a0 0x5606f6136d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f26b003d740 1 --2- 172.21.15.78:0/1895840169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612da10 0x5606f61366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f26b003d740 1 --2- 172.21.15.78:0/1895840169 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5606f61494c0 0x5606f614b910 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f26b003d740 1 --2- 172.21.15.78:0/1895840169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f6139290 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f2686ffd640 0 client.16668 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:05:08.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f26b003d740 1 -- 172.21.15.78:0/1895840169 >> 172.21.15.78:0/1895840169 conn(0x5606f612af20 msgr2=0x5606f6130ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:08.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f26b003d740 1 -- 172.21.15.78:0/1895840169 shutdown_connections 2024-09-11T01:05:08.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f26b003d740 1 -- 172.21.15.78:0/1895840169 wait complete. 2024-09-11T01:05:08.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.934+0000 7f26b003d740 20 client.16668 trim_cache size 0 max 16384 2024-09-11T01:05:08.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:08 smithi110 ceph-mon[34712]: pgmap v511: 97 pgs: 97 active+clean; 285 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-09-11T01:05:08.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T01:05:08.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T01:05:08.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T01:05:08.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T01:05:08.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:05:08.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:05:08.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:08 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:05:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26b003d740 1 Processor -- start 2024-09-11T01:05:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26b003d740 1 -- start start 2024-09-11T01:05:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26b003d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f61387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26b003d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f612da10 0x5606f612a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26b003d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f61399a0 0x5606f613ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26b003d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81b07b0 con 0x5606f612d0f0 2024-09-11T01:05:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26b003d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81b23b0 con 0x5606f61399a0 2024-09-11T01:05:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26b003d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81b0e80 con 0x5606f612da10 2024-09-11T01:05:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f61387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f61387e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38442/0 (socket says 172.21.15.78:38442) 2024-09-11T01:05:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26a77fe640 1 -- 172.21.15.78:0/4090230625 learned_addr learned my addr 172.21.15.78:0/4090230625 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26a6ffd640 1 --2- 172.21.15.78:0/4090230625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f612da10 0x5606f612a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26a7fff640 1 --2- 172.21.15.78:0/4090230625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f61399a0 0x5606f613ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26a77fe640 1 -- 172.21.15.78:0/4090230625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f612da10 msgr2=0x5606f612a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26a77fe640 1 --2- 172.21.15.78:0/4090230625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f612da10 0x5606f612a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26a77fe640 1 -- 172.21.15.78:0/4090230625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f61399a0 msgr2=0x5606f613ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26a77fe640 1 --2- 172.21.15.78:0/4090230625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f61399a0 0x5606f613ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.948+0000 7f26a77fe640 1 -- 172.21.15.78:0/4090230625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f6136490 con 0x5606f612d0f0 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.949+0000 7f26a77fe640 1 --2- 172.21.15.78:0/4090230625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f61387e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f26980aca00 tx=0x7f26980099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e686b0b98f83171 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.949+0000 7f268d7fa640 1 -- 172.21.15.78:0/4090230625 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980a8040 con 0x5606f612d0f0 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.949+0000 7f268d7fa640 1 -- 172.21.15.78:0/4090230625 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26980d3020 con 0x5606f612d0f0 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.949+0000 7f268d7fa640 1 -- 172.21.15.78:0/4090230625 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980ac1e0 con 0x5606f612d0f0 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.949+0000 7f26b003d740 1 -- 172.21.15.78:0/4090230625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 msgr2=0x5606f61387e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.949+0000 7f26b003d740 1 --2- 172.21.15.78:0/4090230625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f61387e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f26980aca00 tx=0x7f26980099b0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.950+0000 7f26b003d740 1 -- 172.21.15.78:0/4090230625 shutdown_connections 2024-09-11T01:05:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.950+0000 7f26b003d740 1 --2- 172.21.15.78:0/4090230625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f61399a0 0x5606f613ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.950+0000 7f26b003d740 1 --2- 172.21.15.78:0/4090230625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f612da10 0x5606f612a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.950+0000 7f26b003d740 1 --2- 172.21.15.78:0/4090230625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f612d0f0 0x5606f61387e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.950+0000 7f26b003d740 1 -- 172.21.15.78:0/4090230625 >> 172.21.15.78:0/4090230625 conn(0x5606f612af20 msgr2=0x5606f612c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.950+0000 7f26b003d740 1 -- 172.21.15.78:0/4090230625 shutdown_connections 2024-09-11T01:05:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.950+0000 7f26b003d740 1 -- 172.21.15.78:0/4090230625 wait complete. 2024-09-11T01:05:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.950+0000 7f26b003d740 1 -- start start 2024-09-11T01:05:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.950+0000 7f26b003d740 1 Processor -- start 2024-09-11T01:05:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26b003d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612d0f0 0x5606f6141d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26b003d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f612da10 0x5606f613c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26b003d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f61399a0 0x5606f613cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26b003d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81b1c20 con 0x5606f61399a0 2024-09-11T01:05:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26b003d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81b1550 con 0x5606f612d0f0 2024-09-11T01:05:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26b003d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81ae790 con 0x5606f612da10 2024-09-11T01:05:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26b003d740 10 client.0 _ll_register_callbacks cb 0x7f26980a8f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26a6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f612da10 0x5606f613c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612d0f0 0x5606f6141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f61399a0 0x5606f613cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26a6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f612da10 0x5606f613c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:34104/0 (socket says 172.21.15.78:34104) 2024-09-11T01:05:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612d0f0 0x5606f6141d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43442/0 (socket says 172.21.15.78:43442) 2024-09-11T01:05:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26a6ffd640 1 -- 172.21.15.78:0/2074211244 learned_addr learned my addr 172.21.15.78:0/2074211244 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26a6ffd640 1 -- 172.21.15.78:0/2074211244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612d0f0 msgr2=0x5606f6141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26a6ffd640 1 --2- 172.21.15.78:0/2074211244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612d0f0 0x5606f6141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26a6ffd640 1 -- 172.21.15.78:0/2074211244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f61399a0 msgr2=0x5606f613cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26a6ffd640 1 --2- 172.21.15.78:0/2074211244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f61399a0 0x5606f613cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26a6ffd640 1 -- 172.21.15.78:0/2074211244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f613d320 con 0x5606f612da10 2024-09-11T01:05:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26a6ffd640 1 --2- 172.21.15.78:0/2074211244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f612da10 0x5606f613c870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f269c011d10 tx=0x7f269c0048e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0bbfaae27a138f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f2686ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:05:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f2686ffd640 1 -- 172.21.15.78:0/2074211244 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c01e040 con 0x5606f612da10 2024-09-11T01:05:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f2686ffd640 1 -- 172.21.15.78:0/2074211244 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f269c004440 con 0x5606f612da10 2024-09-11T01:05:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.951+0000 7f26b003d740 1 -- 172.21.15.78:0/2074211244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26980d3020 con 0x5606f612da10 2024-09-11T01:05:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.952+0000 7f2686ffd640 1 -- 172.21.15.78:0/2074211244 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f269c038050 con 0x5606f612da10 2024-09-11T01:05:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.952+0000 7f26b003d740 10 client.25948 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.952+0000 7f26b003d740 1 -- 172.21.15.78:0/2074211244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5606f61b7160 con 0x5606f612da10 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.952+0000 7f2686ffd640 1 -- 172.21.15.78:0/2074211244 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26980d3020 con 0x5606f612da10 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.952+0000 7f2686ffd640 1 -- 172.21.15.78:0/2074211244 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f269c018070 con 0x5606f612da10 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.952+0000 7f26b003d740 10 client.25948 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.952+0000 7f26b003d740 10 client.25948 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.952+0000 7f26b003d740 20 client.25948 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.952+0000 7f26b003d740 1 --2- 172.21.15.78:0/2074211244 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5606f61b7790 0x5606f61b9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.952+0000 7f26b003d740 4 client.25948 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.952+0000 7f26b003d740 1 -- 172.21.15.78:0/2074211244 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606f609ecd0 con 0x5606f61b7790 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.952+0000 7f26a77fe640 1 --2- 172.21.15.78:0/2074211244 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5606f61b7790 0x5606f61b9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f2686ffd640 10 client.25948 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26a77fe640 1 --2- 172.21.15.78:0/2074211244 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5606f61b7790 0x5606f61b9bb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2675b9bd5c141261 server_cookie=9185ce1989288aa in_seq=0 out_seq=0 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f2686ffd640 1 -- 172.21.15.78:0/2074211244 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606f609ecd0 con 0x5606f61b7790 2024-09-11T01:05:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f2686ffd640 10 client.25948 handle_command_reply: tid=0 2024-09-11T01:05:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26b003d740 1 client.25948 shutdown 2024-09-11T01:05:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26b003d740 1 -- 172.21.15.78:0/2074211244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f612da10 msgr2=0x5606f613c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26b003d740 1 --2- 172.21.15.78:0/2074211244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f612da10 0x5606f613c870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f269c011d10 tx=0x7f269c0048e0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26b003d740 1 -- 172.21.15.78:0/2074211244 shutdown_connections 2024-09-11T01:05:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26b003d740 1 --2- 172.21.15.78:0/2074211244 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5606f61b7790 0x5606f61b9bb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26b003d740 1 --2- 172.21.15.78:0/2074211244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606f61399a0 0x5606f613cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26b003d740 1 --2- 172.21.15.78:0/2074211244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5606f612da10 0x5606f613c870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26b003d740 1 --2- 172.21.15.78:0/2074211244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606f612d0f0 0x5606f6141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f2686ffd640 0 client.25948 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:05:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26b003d740 1 -- 172.21.15.78:0/2074211244 >> 172.21.15.78:0/2074211244 conn(0x5606f612af20 msgr2=0x5606f6130150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26b003d740 1 -- 172.21.15.78:0/2074211244 shutdown_connections 2024-09-11T01:05:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26b003d740 1 -- 172.21.15.78:0/2074211244 wait complete. 2024-09-11T01:05:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.953+0000 7f26b003d740 20 client.25948 trim_cache size 0 max 16384 2024-09-11T01:05:08.964 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:05:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.954+0000 7f26adc68640 1 -- 172.21.15.78:0/2535284950 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26900b54d0 msgr2=0x7f26900b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.954+0000 7f26adc68640 1 --2- 172.21.15.78:0/2535284950 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26900b54d0 0x7f26900b7990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f26a00096c0 tx=0x7f26a00058d0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.954+0000 7f26adc68640 1 -- 172.21.15.78:0/2535284950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26a8101c90 msgr2=0x7f26a8197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:08.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.954+0000 7f26adc68640 1 --2- 172.21.15.78:0/2535284950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26a8101c90 0x7f26a8197e80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f269800ebc0 tx=0x7f269800c6a0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.954+0000 7f26adc68640 1 -- 172.21.15.78:0/2535284950 shutdown_connections 2024-09-11T01:05:08.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.954+0000 7f26adc68640 1 --2- 172.21.15.78:0/2535284950 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f26900b54d0 0x7f26900b7990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.954+0000 7f26adc68640 1 --2- 172.21.15.78:0/2535284950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26a8108020 0x7f26a8198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.954+0000 7f26adc68640 1 --2- 172.21.15.78:0/2535284950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26a81076a0 0x7f26a81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.954+0000 7f26adc68640 1 --2- 172.21.15.78:0/2535284950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26a8101c90 0x7f26a8197e80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:08.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.954+0000 7f26adc68640 1 -- 172.21.15.78:0/2535284950 >> 172.21.15.78:0/2535284950 conn(0x7f26a80fd700 msgr2=0x7f26a80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:08.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.954+0000 7f26adc68640 1 -- 172.21.15.78:0/2535284950 shutdown_connections 2024-09-11T01:05:08.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:08.954+0000 7f26adc68640 1 -- 172.21.15.78:0/2535284950 wait complete. 2024-09-11T01:05:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:08 smithi112 ceph-mon[35384]: pgmap v511: 97 pgs: 97 active+clean; 285 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-09-11T01:05:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T01:05:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T01:05:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T01:05:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T01:05:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:05:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:05:08.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:08 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:05:09.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 467138 5.83 MB/sec execute 197 sec latency 278.256 ms 2024-09-11T01:05:09.770 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:05:09.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:09 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-11T01:05:09.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:09 smithi110 ceph-mon[34712]: Unable to set osd_memory_target on smithi078 to 184555929: error parsing value: Value '184555929' is below minimum 939524096 2024-09-11T01:05:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:09 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-11T01:05:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:09 smithi112 ceph-mon[35384]: Unable to set osd_memory_target on smithi078 to 184555929: error parsing value: Value '184555929' is below minimum 939524096 2024-09-11T01:05:09.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:09 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-11T01:05:09.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:09 smithi078 ceph-mon[31315]: Unable to set osd_memory_target on smithi078 to 184555929: error parsing value: Value '184555929' is below minimum 939524096 2024-09-11T01:05:10.015 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:10.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 467138 5.80 MB/sec execute 198 sec latency 1278.333 ms 2024-09-11T01:05:10.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.523+0000 7f562b6d8640 1 -- 172.21.15.78:0/3573728955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562410d100 msgr2=0x7f562410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:10.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.523+0000 7f562b6d8640 1 --2- 172.21.15.78:0/3573728955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562410d100 0x7f562410f560 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f561c009ca0 tx=0x7f561c02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.523+0000 7f562b6d8640 1 -- 172.21.15.78:0/3573728955 shutdown_connections 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.523+0000 7f562b6d8640 1 --2- 172.21.15.78:0/3573728955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562410d100 0x7f562410f560 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.523+0000 7f562b6d8640 1 --2- 172.21.15.78:0/3573728955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56241045c0 0x7f562410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.523+0000 7f562b6d8640 1 --2- 172.21.15.78:0/3573728955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624103ca0 0x7f5624104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.523+0000 7f562b6d8640 1 -- 172.21.15.78:0/3573728955 >> 172.21.15.78:0/3573728955 conn(0x7f56240fd6e0 msgr2=0x7f56240ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.524+0000 7f562b6d8640 1 -- 172.21.15.78:0/3573728955 shutdown_connections 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.524+0000 7f562b6d8640 1 -- 172.21.15.78:0/3573728955 wait complete. 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.524+0000 7f562b6d8640 1 Processor -- start 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.524+0000 7f562b6d8640 1 -- start start 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.524+0000 7f562b6d8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624103ca0 0x7f5624199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.524+0000 7f562b6d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56241045c0 0x7f56241997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.524+0000 7f562b6d8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562410d100 0x7f562419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.524+0000 7f562b6d8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5624112d70 con 0x7f562410d100 2024-09-11T01:05:10.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.524+0000 7f562b6d8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5624112bf0 con 0x7f5624103ca0 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.524+0000 7f562b6d8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5624112ef0 con 0x7f56241045c0 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.524+0000 7f5629c75640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562410d100 0x7f562419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f5629c75640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562410d100 0x7f562419eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38448/0 (socket says 172.21.15.78:38448) 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f5629c75640 1 -- 172.21.15.78:0/3083038829 learned_addr learned my addr 172.21.15.78:0/3083038829 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f5628c73640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56241045c0 0x7f56241997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f5629474640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624103ca0 0x7f5624199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f5629c75640 1 -- 172.21.15.78:0/3083038829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56241045c0 msgr2=0x7f56241997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f5629c75640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56241045c0 0x7f56241997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f5629c75640 1 -- 172.21.15.78:0/3083038829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624103ca0 msgr2=0x7f5624199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f5629c75640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624103ca0 0x7f5624199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f5629c75640 1 -- 172.21.15.78:0/3083038829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562419f310 con 0x7f562410d100 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f5629474640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624103ca0 0x7f5624199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:05:10.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f5629c75640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562410d100 0x7f562419eb40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f561c005cc0 tx=0x7f561c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f561a7fc640 1 -- 172.21.15.78:0/3083038829 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c003bf0 con 0x7f562410d100 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f561a7fc640 1 -- 172.21.15.78:0/3083038829 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f561c003d90 con 0x7f562410d100 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f562b6d8640 1 -- 172.21.15.78:0/3083038829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562419f5a0 con 0x7f562410d100 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f561a7fc640 1 -- 172.21.15.78:0/3083038829 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c02fbc0 con 0x7f562410d100 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.525+0000 7f562b6d8640 1 -- 172.21.15.78:0/3083038829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56241a5cc0 con 0x7f562410d100 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.528+0000 7f561a7fc640 1 -- 172.21.15.78:0/3083038829 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f561c004610 con 0x7f562410d100 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.528+0000 7f562daad740 1 Processor -- start 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.528+0000 7f562daad740 1 -- start start 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.528+0000 7f562daad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee04d0 0x55ce4cee08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.528+0000 7f562daad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee0df0 0x55ce4ceec840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.528+0000 7f562daad740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4ceecd80 0x55ce4ceef170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.528+0000 7f562daad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce4ce56ea0 con 0x55ce4cee0df0 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.528+0000 7f562daad740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ce4ce462d0 con 0x55ce4ceecd80 2024-09-11T01:05:10.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.528+0000 7f562daad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ce4ce4a0e0 con 0x55ce4cee04d0 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.528+0000 7f5629474640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee0df0 0x55ce4ceec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.528+0000 7f5629474640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee0df0 0x55ce4ceec840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38458/0 (socket says 172.21.15.78:38458) 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.528+0000 7f5629474640 1 -- 172.21.15.78:0/1812697589 learned_addr learned my addr 172.21.15.78:0/1812697589 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f561a7fc640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f56100b5450 0x7f56100b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f5628c73640 1 --2- 172.21.15.78:0/1812697589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee04d0 0x55ce4cee08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f5629c75640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f56100b5450 0x7f56100b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f5628c73640 1 --2- 172.21.15.78:0/1812697589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4ceecd80 0x55ce4ceef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f5629474640 1 -- 172.21.15.78:0/1812697589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee04d0 msgr2=0x55ce4cee08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f5629474640 1 --2- 172.21.15.78:0/1812697589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee04d0 0x55ce4cee08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f5629474640 1 -- 172.21.15.78:0/1812697589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4ceecd80 msgr2=0x55ce4ceef170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f5629474640 1 --2- 172.21.15.78:0/1812697589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4ceecd80 0x55ce4ceef170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f5629474640 1 -- 172.21.15.78:0/1812697589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce4cba4650 con 0x55ce4cee0df0 2024-09-11T01:05:10.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f5628c73640 1 --2- 172.21.15.78:0/1812697589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4ceecd80 0x55ce4ceef170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:10.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f5629474640 1 --2- 172.21.15.78:0/1812697589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee0df0 0x55ce4ceec840 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f562000bd40 tx=0x7f562000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666269bd74903a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:10.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f561a7fc640 1 -- 172.21.15.78:0/3083038829 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f561c1074f0 con 0x7f562410d100 2024-09-11T01:05:10.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f560f7fe640 1 -- 172.21.15.78:0/1812697589 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620015070 con 0x55ce4cee0df0 2024-09-11T01:05:10.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f560f7fe640 1 -- 172.21.15.78:0/1812697589 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f562000be30 con 0x55ce4cee0df0 2024-09-11T01:05:10.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.529+0000 7f560f7fe640 1 -- 172.21.15.78:0/1812697589 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620011900 con 0x55ce4cee0df0 2024-09-11T01:05:10.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.530+0000 7f562daad740 1 -- 172.21.15.78:0/1812697589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee0df0 msgr2=0x55ce4ceec840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:10.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.530+0000 7f562daad740 1 --2- 172.21.15.78:0/1812697589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee0df0 0x55ce4ceec840 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f562000bd40 tx=0x7f562000f730 comp rx=0 tx=0).stop 2024-09-11T01:05:10.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.530+0000 7f5629c75640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f56100b5450 0x7f56100b7910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f561c0d1170 tx=0x7f561c0925a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:10.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.530+0000 7f562daad740 1 -- 172.21.15.78:0/1812697589 shutdown_connections 2024-09-11T01:05:10.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.530+0000 7f562daad740 1 --2- 172.21.15.78:0/1812697589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4ceecd80 0x55ce4ceef170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.530+0000 7f562daad740 1 --2- 172.21.15.78:0/1812697589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee0df0 0x55ce4ceec840 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.530+0000 7f562daad740 1 --2- 172.21.15.78:0/1812697589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee04d0 0x55ce4cee08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.530+0000 7f562daad740 1 -- 172.21.15.78:0/1812697589 >> 172.21.15.78:0/1812697589 conn(0x55ce4cede300 msgr2=0x55ce4cedf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:10.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.530+0000 7f562daad740 1 -- 172.21.15.78:0/1812697589 shutdown_connections 2024-09-11T01:05:10.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.530+0000 7f562daad740 1 -- 172.21.15.78:0/1812697589 wait complete. 2024-09-11T01:05:10.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.530+0000 7f562daad740 1 -- start start 2024-09-11T01:05:10.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.530+0000 7f562daad740 1 Processor -- start 2024-09-11T01:05:10.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f562daad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee04d0 0x55ce4ceec670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f562daad740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4cee0df0 0x55ce4cee9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f562daad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4ceecd80 0x55ce4ceea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f562daad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce4cca7700 con 0x55ce4ceecd80 2024-09-11T01:05:10.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f562daad740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ce4cde1160 con 0x55ce4cee0df0 2024-09-11T01:05:10.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f562daad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ce4ceefa00 con 0x55ce4cee04d0 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f562daad740 10 client.0 _ll_register_callbacks cb 0x55ce4cdfa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f5628c73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4ceecd80 0x55ce4ceea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f5628c73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4ceecd80 0x55ce4ceea0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38464/0 (socket says 172.21.15.78:38464) 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f5628c73640 1 -- 172.21.15.78:0/477711395 learned_addr learned my addr 172.21.15.78:0/477711395 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f5629474640 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4cee0df0 0x55ce4cee9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f5628c73640 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee04d0 0x55ce4ceec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f5628c73640 1 -- 172.21.15.78:0/477711395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee04d0 msgr2=0x55ce4ceec670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f5628c73640 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee04d0 0x55ce4ceec670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f5628c73640 1 -- 172.21.15.78:0/477711395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4cee0df0 msgr2=0x55ce4cee9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f5628c73640 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4cee0df0 0x55ce4cee9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f5628c73640 1 -- 172.21.15.78:0/477711395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce4ce0ec90 con 0x55ce4ceecd80 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f5628c73640 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4ceecd80 0x55ce4ceea0f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5614012b00 tx=0x7f56140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edaaa77b072cb1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:10.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f560cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.531+0000 7f560cff9640 1 -- 172.21.15.78:0/477711395 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614010aa0 con 0x55ce4ceecd80 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f560cff9640 1 -- 172.21.15.78:0/477711395 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5614010c40 con 0x55ce4ceecd80 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f562daad740 1 -- 172.21.15.78:0/477711395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f562000be30 con 0x55ce4ceecd80 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f560cff9640 1 -- 172.21.15.78:0/477711395 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5614010e40 con 0x55ce4ceecd80 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f562daad740 10 client.16686 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f562daad740 1 -- 172.21.15.78:0/477711395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ce4cefb300 con 0x55ce4ceecd80 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f560cff9640 1 -- 172.21.15.78:0/477711395 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f562000be30 con 0x55ce4ceecd80 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f560cff9640 1 -- 172.21.15.78:0/477711395 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f5614029050 con 0x55ce4ceecd80 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f562daad740 10 client.16686 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f562daad740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f562daad740 20 client.16686 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f562daad740 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce4cefc890 0x55ce4cefecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f562daad740 4 client.16686 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:10.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f562daad740 1 -- 172.21.15.78:0/477711395 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce4ce58df0 con 0x55ce4cefc890 2024-09-11T01:05:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.532+0000 7f5629474640 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce4cefc890 0x55ce4cefecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.533+0000 7f5629474640 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce4cefc890 0x55ce4cefecd0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bdb20bdaf190f26 server_cookie=90e86853d0938477 in_seq=0 out_seq=0 2024-09-11T01:05:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.533+0000 7f560cff9640 10 client.16686 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.533+0000 7f560cff9640 1 -- 172.21.15.78:0/477711395 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ce4ce58df0 con 0x55ce4cefc890 2024-09-11T01:05:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.533+0000 7f560cff9640 10 client.16686 handle_command_reply: tid=0 2024-09-11T01:05:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f562daad740 1 client.16686 shutdown 2024-09-11T01:05:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f562daad740 1 -- 172.21.15.78:0/477711395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4ceecd80 msgr2=0x55ce4ceea0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f562daad740 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4ceecd80 0x55ce4ceea0f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5614012b00 tx=0x7f56140046c0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f562daad740 1 -- 172.21.15.78:0/477711395 shutdown_connections 2024-09-11T01:05:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f562daad740 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce4cefc890 0x55ce4cefecd0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f562daad740 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4ceecd80 0x55ce4ceea0f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f562daad740 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4cee0df0 0x55ce4cee9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f562daad740 1 --2- 172.21.15.78:0/477711395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee04d0 0x55ce4ceec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f560cff9640 0 client.16686 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:10.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f562daad740 1 -- 172.21.15.78:0/477711395 >> 172.21.15.78:0/477711395 conn(0x55ce4cede300 msgr2=0x55ce4cee3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:10.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f562daad740 1 -- 172.21.15.78:0/477711395 shutdown_connections 2024-09-11T01:05:10.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f562daad740 1 -- 172.21.15.78:0/477711395 wait complete. 2024-09-11T01:05:10.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.534+0000 7f562daad740 20 client.16686 trim_cache size 0 max 16384 2024-09-11T01:05:10.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f562daad740 1 Processor -- start 2024-09-11T01:05:10.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f562daad740 1 -- start start 2024-09-11T01:05:10.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f562daad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 0x55ce4cef8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f562daad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee0df0 0x55ce4cee4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f562daad740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4ceecd80 0x55ce4cee5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f562daad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f56241ae570 con 0x55ce4cee04d0 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f562daad740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f56241b0170 con 0x55ce4ceecd80 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f562daad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56241aec40 con 0x55ce4cee0df0 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f5628c73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 0x55ce4cef8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f5628c73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 0x55ce4cef8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38470/0 (socket says 172.21.15.78:38470) 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f5628c73640 1 -- 172.21.15.78:0/2054581956 learned_addr learned my addr 172.21.15.78:0/2054581956 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f5629474640 1 --2- 172.21.15.78:0/2054581956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee0df0 0x55ce4cee4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f5628c73640 1 -- 172.21.15.78:0/2054581956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee0df0 msgr2=0x55ce4cee4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f5628c73640 1 --2- 172.21.15.78:0/2054581956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee0df0 0x55ce4cee4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f5628c73640 1 -- 172.21.15.78:0/2054581956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4ceecd80 msgr2=0x55ce4cee5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f5628c73640 1 --2- 172.21.15.78:0/2054581956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4ceecd80 0x55ce4cee5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.547+0000 7f5628c73640 1 -- 172.21.15.78:0/2054581956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce4cedefe0 con 0x55ce4cee04d0 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f5628c73640 1 --2- 172.21.15.78:0/2054581956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 0x55ce4cef8c90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f561400b6a0 tx=0x7f561400b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff501ee1cd10378 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f560effd640 1 -- 172.21.15.78:0/2054581956 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561402f070 con 0x55ce4cee04d0 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f560effd640 1 -- 172.21.15.78:0/2054581956 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5614005850 con 0x55ce4cee04d0 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f560effd640 1 -- 172.21.15.78:0/2054581956 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614005100 con 0x55ce4cee04d0 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f562daad740 1 -- 172.21.15.78:0/2054581956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 msgr2=0x55ce4cef8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f562daad740 1 --2- 172.21.15.78:0/2054581956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 0x55ce4cef8c90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f561400b6a0 tx=0x7f561400b500 comp rx=0 tx=0).stop 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f562daad740 1 -- 172.21.15.78:0/2054581956 shutdown_connections 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f562daad740 1 --2- 172.21.15.78:0/2054581956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4ceecd80 0x55ce4cee5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f562daad740 1 --2- 172.21.15.78:0/2054581956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4cee0df0 0x55ce4cee4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f562daad740 1 --2- 172.21.15.78:0/2054581956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 0x55ce4cef8c90 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f562daad740 1 -- 172.21.15.78:0/2054581956 >> 172.21.15.78:0/2054581956 conn(0x55ce4cede300 msgr2=0x55ce4cefe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f562daad740 1 -- 172.21.15.78:0/2054581956 shutdown_connections 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.548+0000 7f562daad740 1 -- 172.21.15.78:0/2054581956 wait complete. 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f562daad740 1 -- start start 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f562daad740 1 Processor -- start 2024-09-11T01:05:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f562daad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 0x55ce4cef5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f562daad740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4cee0df0 0x55ce4cf04180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f562daad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4ceecd80 0x55ce4cf046c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f562daad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f56241af9e0 con 0x55ce4cee04d0 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f562daad740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f56241af310 con 0x55ce4cee0df0 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f562daad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56241ac550 con 0x55ce4ceecd80 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f562daad740 10 client.0 _ll_register_callbacks cb 0x7f561400bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f5628c73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 0x55ce4cef5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f5628c73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 0x55ce4cef5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38478/0 (socket says 172.21.15.78:38478) 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f5628c73640 1 -- 172.21.15.78:0/2756141418 learned_addr learned my addr 172.21.15.78:0/2756141418 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f5628c73640 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4ceecd80 0x55ce4cf046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f5629474640 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4cee0df0 0x55ce4cf04180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f5628c73640 1 -- 172.21.15.78:0/2756141418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4ceecd80 msgr2=0x55ce4cf046c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f5628c73640 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4ceecd80 0x55ce4cf046c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f5628c73640 1 -- 172.21.15.78:0/2756141418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4cee0df0 msgr2=0x55ce4cf04180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f5628c73640 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4cee0df0 0x55ce4cf04180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f5628c73640 1 -- 172.21.15.78:0/2756141418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce4cf04ed0 con 0x55ce4cee04d0 2024-09-11T01:05:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.549+0000 7f5629474640 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4cee0df0 0x55ce4cf04180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f5628c73640 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 0x55ce4cef5480 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5614006010 tx=0x7f5614010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b60ccd772e2e1c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f560cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f560cff9640 1 -- 172.21.15.78:0/2756141418 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561402f050 con 0x55ce4cee04d0 2024-09-11T01:05:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f560cff9640 1 -- 172.21.15.78:0/2756141418 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5614004ea0 con 0x55ce4cee04d0 2024-09-11T01:05:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f562daad740 1 -- 172.21.15.78:0/2756141418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5614005850 con 0x55ce4cee04d0 2024-09-11T01:05:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f560cff9640 1 -- 172.21.15.78:0/2756141418 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5614045050 con 0x55ce4cee04d0 2024-09-11T01:05:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f562daad740 10 client.16692 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f562daad740 1 -- 172.21.15.78:0/2756141418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ce4cf69800 con 0x55ce4cee04d0 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f560cff9640 1 -- 172.21.15.78:0/2756141418 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5614005850 con 0x55ce4cee04d0 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f560cff9640 1 -- 172.21.15.78:0/2756141418 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7f5614029040 con 0x55ce4cee04d0 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f562daad740 10 client.16692 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f562daad740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f562daad740 20 client.16692 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f562daad740 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce4cf69ec0 0x55ce4cf6c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f562daad740 4 client.16692 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.550+0000 7f562daad740 1 -- 172.21.15.78:0/2756141418 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce4ce548a0 con 0x55ce4cf69ec0 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.551+0000 7f5629474640 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce4cf69ec0 0x55ce4cf6c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.551+0000 7f5629474640 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce4cf69ec0 0x55ce4cf6c2e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f890e9d804b980 server_cookie=4f25fa5e501a129a in_seq=0 out_seq=0 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.551+0000 7f560cff9640 10 client.16692 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.551+0000 7f560cff9640 1 -- 172.21.15.78:0/2756141418 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce4ce548a0 con 0x55ce4cf69ec0 2024-09-11T01:05:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.551+0000 7f560cff9640 10 client.16692 handle_command_reply: tid=0 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.551+0000 7f562daad740 1 client.16692 shutdown 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.551+0000 7f562daad740 1 -- 172.21.15.78:0/2756141418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 msgr2=0x55ce4cef5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.551+0000 7f562daad740 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 0x55ce4cef5480 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5614006010 tx=0x7f5614010bb0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562daad740 1 -- 172.21.15.78:0/2756141418 shutdown_connections 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562daad740 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce4ceecd80 0x55ce4cf046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562daad740 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce4cee0df0 0x55ce4cf04180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562daad740 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce4cf69ec0 0x55ce4cf6c2e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562daad740 1 --2- 172.21.15.78:0/2756141418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce4cee04d0 0x55ce4cef5480 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f560cff9640 0 client.16692 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562daad740 1 -- 172.21.15.78:0/2756141418 >> 172.21.15.78:0/2756141418 conn(0x55ce4cede300 msgr2=0x55ce4cee4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562daad740 1 -- 172.21.15.78:0/2756141418 shutdown_connections 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562daad740 1 -- 172.21.15.78:0/2756141418 wait complete. 2024-09-11T01:05:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562daad740 20 client.16692 trim_cache size 0 max 16384 2024-09-11T01:05:10.571 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:05:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562b6d8640 1 -- 172.21.15.78:0/3083038829 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f56100b5450 msgr2=0x7f56100b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562b6d8640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f56100b5450 0x7f56100b7910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f561c0d1170 tx=0x7f561c0925a0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562b6d8640 1 -- 172.21.15.78:0/3083038829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562410d100 msgr2=0x7f562419eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562b6d8640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562410d100 0x7f562419eb40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f561c005cc0 tx=0x7f561c02f380 comp rx=0 tx=0).stop 2024-09-11T01:05:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562b6d8640 1 -- 172.21.15.78:0/3083038829 shutdown_connections 2024-09-11T01:05:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562b6d8640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f56100b5450 0x7f56100b7910 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562b6d8640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562410d100 0x7f562419eb40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562b6d8640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56241045c0 0x7f56241997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562b6d8640 1 --2- 172.21.15.78:0/3083038829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624103ca0 0x7f5624199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.552+0000 7f562b6d8640 1 -- 172.21.15.78:0/3083038829 >> 172.21.15.78:0/3083038829 conn(0x7f56240fd6e0 msgr2=0x7f56240fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.553+0000 7f562b6d8640 1 -- 172.21.15.78:0/3083038829 shutdown_connections 2024-09-11T01:05:10.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:10.553+0000 7f562b6d8640 1 -- 172.21.15.78:0/3083038829 wait complete. 2024-09-11T01:05:10.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:10 smithi078 ceph-mon[31315]: pgmap v512: 97 pgs: 97 active+clean; 286 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 240 op/s 2024-09-11T01:05:10.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:10 smithi110 ceph-mon[34712]: pgmap v512: 97 pgs: 97 active+clean; 286 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 240 op/s 2024-09-11T01:05:10.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:10 smithi112 ceph-mon[35384]: pgmap v512: 97 pgs: 97 active+clean; 286 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 240 op/s 2024-09-11T01:05:11.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 467138 5.78 MB/sec execute 199 sec latency 2278.424 ms 2024-09-11T01:05:12.265 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:05:12.268 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:05:12.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 467837 5.76 MB/sec execute 200 sec latency 3038.475 ms 2024-09-11T01:05:12.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:12 smithi078 ceph-mon[31315]: pgmap v513: 97 pgs: 97 active+clean; 286 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-09-11T01:05:12.558 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:12.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:12 smithi110 ceph-mon[34712]: pgmap v513: 97 pgs: 97 active+clean; 286 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-09-11T01:05:12.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:12 smithi112 ceph-mon[35384]: pgmap v513: 97 pgs: 97 active+clean; 286 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-09-11T01:05:13.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.091+0000 7fef914ee640 1 -- 172.21.15.78:0/3607831803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef8c108020 msgr2=0x7fef8c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.091+0000 7fef914ee640 1 --2- 172.21.15.78:0/3607831803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef8c108020 0x7fef8c10e820 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fef84009ca0 tx=0x7fef8402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.091+0000 7fef914ee640 1 -- 172.21.15.78:0/3607831803 shutdown_connections 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.091+0000 7fef914ee640 1 --2- 172.21.15.78:0/3607831803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef8c108020 0x7fef8c10e820 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.091+0000 7fef914ee640 1 --2- 172.21.15.78:0/3607831803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef8c1076a0 0x7fef8c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.091+0000 7fef914ee640 1 --2- 172.21.15.78:0/3607831803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef8c101c90 0x7fef8c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.091+0000 7fef914ee640 1 -- 172.21.15.78:0/3607831803 >> 172.21.15.78:0/3607831803 conn(0x7fef8c0fd700 msgr2=0x7fef8c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.091+0000 7fef914ee640 1 -- 172.21.15.78:0/3607831803 shutdown_connections 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.091+0000 7fef914ee640 1 -- 172.21.15.78:0/3607831803 wait complete. 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.092+0000 7fef914ee640 1 Processor -- start 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.092+0000 7fef914ee640 1 -- start start 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.092+0000 7fef914ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef8c101c90 0x7fef8c197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.092+0000 7fef914ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef8c1076a0 0x7fef8c197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.092+0000 7fef914ee640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef8c108020 0x7fef8c19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.092+0000 7fef914ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c110c20 con 0x7fef8c1076a0 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.092+0000 7fef914ee640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c110aa0 con 0x7fef8c108020 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.092+0000 7fef914ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c110da0 con 0x7fef8c101c90 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef8a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef8c1076a0 0x7fef8c197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef8affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef8c101c90 0x7fef8c197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef8a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef8c1076a0 0x7fef8c197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38500/0 (socket says 172.21.15.78:38500) 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef8a7fc640 1 -- 172.21.15.78:0/1295007832 learned_addr learned my addr 172.21.15.78:0/1295007832 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef8b7fe640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef8c108020 0x7fef8c19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef8affd640 1 -- 172.21.15.78:0/1295007832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef8c108020 msgr2=0x7fef8c19c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef8affd640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef8c108020 0x7fef8c19c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef8affd640 1 -- 172.21.15.78:0/1295007832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef8c1076a0 msgr2=0x7fef8c197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef8affd640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef8c1076a0 0x7fef8c197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef8affd640 1 -- 172.21.15.78:0/1295007832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c19d100 con 0x7fef8c101c90 2024-09-11T01:05:13.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef8affd640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef8c101c90 0x7fef8c197050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00e9f0 tx=0x7fef7c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:13.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef73fff640 1 -- 172.21.15.78:0/1295007832 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00ccc0 con 0x7fef8c101c90 2024-09-11T01:05:13.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef73fff640 1 -- 172.21.15.78:0/1295007832 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c005020 con 0x7fef8c101c90 2024-09-11T01:05:13.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef73fff640 1 -- 172.21.15.78:0/1295007832 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c004a00 con 0x7fef8c101c90 2024-09-11T01:05:13.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef914ee640 1 -- 172.21.15.78:0/1295007832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8c19d3f0 con 0x7fef8c101c90 2024-09-11T01:05:13.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.093+0000 7fef914ee640 1 -- 172.21.15.78:0/1295007832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8c1a3ab0 con 0x7fef8c101c90 2024-09-11T01:05:13.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.096+0000 7fef73fff640 1 -- 172.21.15.78:0/1295007832 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fef7c004ba0 con 0x7fef8c101c90 2024-09-11T01:05:13.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef938c3740 1 Processor -- start 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef938c3740 1 -- start start 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef73fff640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fef740b7660 0x7fef740b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef938c3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 0x561b8feb82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef938c3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fec4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef938c3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8fec4790 0x561b8fec6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef938c3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561b8fe2ee60 con 0x561b8fec4790 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef938c3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561b8fe1e290 con 0x561b8feb7ee0 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef938c3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561b8fe220a0 con 0x561b8feb8800 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef8affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 0x561b8feb82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef8a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fec4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef8a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fec4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:34166/0 (socket says 172.21.15.78:34166) 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef8affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 0x561b8feb82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43536/0 (socket says 172.21.15.78:43536) 2024-09-11T01:05:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef8a7fc640 1 -- 172.21.15.78:0/1725879843 learned_addr learned my addr 172.21.15.78:0/1725879843 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:13.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.097+0000 7fef73fff640 1 -- 172.21.15.78:0/1295007832 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fef7c0e5f20 con 0x7fef8c101c90 2024-09-11T01:05:13.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef8b7fe640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fef740b7660 0x7fef740b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef8b7fe640 1 --2- 172.21.15.78:0/1725879843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8fec4790 0x561b8fec6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef8a7fc640 1 -- 172.21.15.78:0/1725879843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 msgr2=0x561b8feb82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:13.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef8a7fc640 1 --2- 172.21.15.78:0/1725879843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 0x561b8feb82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef8a7fc640 1 -- 172.21.15.78:0/1725879843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8fec4790 msgr2=0x561b8fec6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:13.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef8a7fc640 1 --2- 172.21.15.78:0/1725879843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8fec4790 0x561b8fec6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef8a7fc640 1 -- 172.21.15.78:0/1725879843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b8fb7c930 con 0x561b8feb8800 2024-09-11T01:05:13.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef8b7fe640 1 --2- 172.21.15.78:0/1725879843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8fec4790 0x561b8fec6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:13.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef8a7fc640 1 --2- 172.21.15.78:0/1725879843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fec4250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fef80009d90 tx=0x7fef800107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6f401729f49c1a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:13.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef717fa640 1 -- 172.21.15.78:0/1725879843 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80004230 con 0x561b8feb8800 2024-09-11T01:05:13.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef8b7fe640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fef740b7660 0x7fef740b9b20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fef84006010 tx=0x7fef84009360 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:13.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef717fa640 1 -- 172.21.15.78:0/1725879843 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef800043d0 con 0x561b8feb8800 2024-09-11T01:05:13.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.098+0000 7fef717fa640 1 -- 172.21.15.78:0/1725879843 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800047a0 con 0x561b8feb8800 2024-09-11T01:05:13.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.099+0000 7fef938c3740 1 -- 172.21.15.78:0/1725879843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 msgr2=0x561b8fec4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:13.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.099+0000 7fef938c3740 1 --2- 172.21.15.78:0/1725879843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fec4250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fef80009d90 tx=0x7fef800107a0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.099+0000 7fef938c3740 1 -- 172.21.15.78:0/1725879843 shutdown_connections 2024-09-11T01:05:13.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.099+0000 7fef938c3740 1 --2- 172.21.15.78:0/1725879843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8fec4790 0x561b8fec6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.099+0000 7fef938c3740 1 --2- 172.21.15.78:0/1725879843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fec4250 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.099+0000 7fef938c3740 1 --2- 172.21.15.78:0/1725879843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 0x561b8feb82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.099+0000 7fef938c3740 1 -- 172.21.15.78:0/1725879843 >> 172.21.15.78:0/1725879843 conn(0x561b8feb5d10 msgr2=0x561b8feb6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.099+0000 7fef938c3740 1 -- 172.21.15.78:0/1725879843 shutdown_connections 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.099+0000 7fef938c3740 1 -- 172.21.15.78:0/1725879843 wait complete. 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef938c3740 1 -- start start 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef938c3740 1 Processor -- start 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef938c3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 0x561b8fec1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef938c3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fec1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef938c3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8fec4790 0x561b8fec2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef8affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 0x561b8fec1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef8affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 0x561b8fec1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38528/0 (socket says 172.21.15.78:38528) 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef8affd640 1 -- 172.21.15.78:0/2757100874 learned_addr learned my addr 172.21.15.78:0/2757100874 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef8a7fc640 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fec1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef938c3740 1 -- 172.21.15.78:0/2757100874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561b8fc7f890 con 0x561b8feb7ee0 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef938c3740 1 -- 172.21.15.78:0/2757100874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561b8fdb8d40 con 0x561b8fec4790 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef938c3740 1 -- 172.21.15.78:0/2757100874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561b8fec7440 con 0x561b8feb8800 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.100+0000 7fef8b7fe640 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8fec4790 0x561b8fec2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef938c3740 10 client.0 _ll_register_callbacks cb 0x561b8fe1cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef8affd640 1 -- 172.21.15.78:0/2757100874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 msgr2=0x561b8fec1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef8affd640 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fec1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef8affd640 1 -- 172.21.15.78:0/2757100874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8fec4790 msgr2=0x561b8fec2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef8affd640 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8fec4790 0x561b8fec2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef8affd640 1 -- 172.21.15.78:0/2757100874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b8fed07c0 con 0x561b8feb7ee0 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef8b7fe640 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8fec4790 0x561b8fec2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef8affd640 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 0x561b8fec1700 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fef7c004770 tx=0x7fef7c0abe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e775494fee45f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef6eff5640 1 -- 172.21.15.78:0/2757100874 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c01c040 con 0x561b8feb7ee0 2024-09-11T01:05:13.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef6eff5640 1 -- 172.21.15.78:0/2757100874 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fef7c0054f0 con 0x561b8feb7ee0 2024-09-11T01:05:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef938c3740 1 -- 172.21.15.78:0/2757100874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef800043d0 con 0x561b8feb7ee0 2024-09-11T01:05:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef6eff5640 1 -- 172.21.15.78:0/2757100874 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fef7c0abf20 con 0x561b8feb7ee0 2024-09-11T01:05:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef938c3740 10 client.16701 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.101+0000 7fef938c3740 1 -- 172.21.15.78:0/2757100874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561b8fed3120 con 0x561b8feb7ee0 2024-09-11T01:05:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.102+0000 7fef6eff5640 1 -- 172.21.15.78:0/2757100874 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef800043d0 con 0x561b8feb7ee0 2024-09-11T01:05:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.102+0000 7fef6eff5640 1 -- 172.21.15.78:0/2757100874 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fef7c0a8680 con 0x561b8feb7ee0 2024-09-11T01:05:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.102+0000 7fef938c3740 10 client.16701 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.102+0000 7fef938c3740 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.102+0000 7fef938c3740 20 client.16701 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.102+0000 7fef938c3740 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b8fed4ac0 0x561b8fed6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.102+0000 7fef938c3740 4 client.16701 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:13.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.102+0000 7fef938c3740 1 -- 172.21.15.78:0/2757100874 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b8fe30db0 con 0x561b8fed4ac0 2024-09-11T01:05:13.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.102+0000 7fef8a7fc640 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b8fed4ac0 0x561b8fed6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.103+0000 7fef8a7fc640 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b8fed4ac0 0x561b8fed6f20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5343144064c23826 server_cookie=fe78b8695547a58 in_seq=0 out_seq=0 2024-09-11T01:05:13.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.103+0000 7fef6eff5640 10 client.16701 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:13.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.103+0000 7fef6eff5640 1 -- 172.21.15.78:0/2757100874 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561b8fe30db0 con 0x561b8fed4ac0 2024-09-11T01:05:13.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.103+0000 7fef6eff5640 10 client.16701 handle_command_reply: tid=0 2024-09-11T01:05:13.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef938c3740 1 client.16701 shutdown 2024-09-11T01:05:13.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef938c3740 1 -- 172.21.15.78:0/2757100874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 msgr2=0x561b8fec1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:13.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef938c3740 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 0x561b8fec1700 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fef7c004770 tx=0x7fef7c0abe20 comp rx=0 tx=0).stop 2024-09-11T01:05:13.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef938c3740 1 -- 172.21.15.78:0/2757100874 shutdown_connections 2024-09-11T01:05:13.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef938c3740 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8fec4790 0x561b8fec2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef938c3740 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fec1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef938c3740 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b8fed4ac0 0x561b8fed6f20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef938c3740 1 --2- 172.21.15.78:0/2757100874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 0x561b8fec1700 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef938c3740 1 -- 172.21.15.78:0/2757100874 >> 172.21.15.78:0/2757100874 conn(0x561b8feb5d10 msgr2=0x561b8feb4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:13.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef6eff5640 0 client.16701 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:13.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef938c3740 1 -- 172.21.15.78:0/2757100874 shutdown_connections 2024-09-11T01:05:13.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef938c3740 1 -- 172.21.15.78:0/2757100874 wait complete. 2024-09-11T01:05:13.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.104+0000 7fef938c3740 20 client.16701 trim_cache size 0 max 16384 2024-09-11T01:05:13.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.123+0000 7fef938c3740 1 Processor -- start 2024-09-11T01:05:13.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.123+0000 7fef938c3740 1 -- start start 2024-09-11T01:05:13.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.123+0000 7fef938c3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 0x561b8febb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef938c3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fedc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef938c3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8fec4790 0x561b8fedca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef938c3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c1ac3b0 con 0x561b8fec4790 2024-09-11T01:05:13.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef938c3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c1adfb0 con 0x561b8feb7ee0 2024-09-11T01:05:13.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef938c3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c1aca80 con 0x561b8feb8800 2024-09-11T01:05:13.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef8affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 0x561b8febb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef8affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 0x561b8febb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43558/0 (socket says 172.21.15.78:43558) 2024-09-11T01:05:13.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef8affd640 1 -- 172.21.15.78:0/2005192366 learned_addr learned my addr 172.21.15.78:0/2005192366 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:13.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef8a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fedc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef8b7fe640 1 --2- 172.21.15.78:0/2005192366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8fec4790 0x561b8fedca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef8affd640 1 -- 172.21.15.78:0/2005192366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 msgr2=0x561b8fedc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:13.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef8affd640 1 --2- 172.21.15.78:0/2005192366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fedc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef8affd640 1 -- 172.21.15.78:0/2005192366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8fec4790 msgr2=0x561b8fedca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:13.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef8affd640 1 --2- 172.21.15.78:0/2005192366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8fec4790 0x561b8fedca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef8affd640 1 -- 172.21.15.78:0/2005192366 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b8feb6790 con 0x561b8feb7ee0 2024-09-11T01:05:13.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef8a7fc640 1 --2- 172.21.15.78:0/2005192366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fedc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:13.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.124+0000 7fef8affd640 1 --2- 172.21.15.78:0/2005192366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 0x561b8febb7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0a89e0 tx=0x7fef7c0096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49470374054cad4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:13.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.125+0000 7fef70ff9640 1 -- 172.21.15.78:0/2005192366 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c01c040 con 0x561b8feb7ee0 2024-09-11T01:05:13.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.125+0000 7fef70ff9640 1 -- 172.21.15.78:0/2005192366 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c0ac410 con 0x561b8feb7ee0 2024-09-11T01:05:13.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.125+0000 7fef70ff9640 1 -- 172.21.15.78:0/2005192366 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0b5510 con 0x561b8feb7ee0 2024-09-11T01:05:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.125+0000 7fef938c3740 1 -- 172.21.15.78:0/2005192366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 msgr2=0x561b8febb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.125+0000 7fef938c3740 1 --2- 172.21.15.78:0/2005192366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 0x561b8febb7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0a89e0 tx=0x7fef7c0096e0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.125+0000 7fef938c3740 1 -- 172.21.15.78:0/2005192366 shutdown_connections 2024-09-11T01:05:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.125+0000 7fef938c3740 1 --2- 172.21.15.78:0/2005192366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8fec4790 0x561b8fedca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.125+0000 7fef938c3740 1 --2- 172.21.15.78:0/2005192366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8feb8800 0x561b8fedc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.125+0000 7fef938c3740 1 --2- 172.21.15.78:0/2005192366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb7ee0 0x561b8febb7a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.125+0000 7fef938c3740 1 -- 172.21.15.78:0/2005192366 >> 172.21.15.78:0/2005192366 conn(0x561b8feb5d10 msgr2=0x561b8febb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.125+0000 7fef938c3740 1 -- 172.21.15.78:0/2005192366 shutdown_connections 2024-09-11T01:05:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.125+0000 7fef938c3740 1 -- 172.21.15.78:0/2005192366 wait complete. 2024-09-11T01:05:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef938c3740 1 -- start start 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef938c3740 1 Processor -- start 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef938c3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 0x561b8fed87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef938c3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb8800 0x561b8fed8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef938c3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8fec4790 0x561b8ff407d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef938c3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c1ad820 con 0x561b8feb7ee0 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef938c3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c1ad150 con 0x561b8feb8800 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef938c3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c1aa3b0 con 0x561b8fec4790 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef938c3740 10 client.0 _ll_register_callbacks cb 0x7fef7c0a8af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef8a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb8800 0x561b8fed8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef8affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 0x561b8fed87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef8a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb8800 0x561b8fed8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43562/0 (socket says 172.21.15.78:43562) 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef8a7fc640 1 -- 172.21.15.78:0/3141322872 learned_addr learned my addr 172.21.15.78:0/3141322872 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.126+0000 7fef8b7fe640 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8fec4790 0x561b8ff407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef8a7fc640 1 -- 172.21.15.78:0/3141322872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8fec4790 msgr2=0x561b8ff407d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef8a7fc640 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8fec4790 0x561b8ff407d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef8a7fc640 1 -- 172.21.15.78:0/3141322872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 msgr2=0x561b8fed87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef8a7fc640 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 0x561b8fed87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef8a7fc640 1 -- 172.21.15.78:0/3141322872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b8fed95c0 con 0x561b8feb8800 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef8affd640 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 0x561b8fed87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef8a7fc640 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb8800 0x561b8fed8d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fef80009d60 tx=0x7fef80005260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c00e4070cf1dd66 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef6eff5640 1 -- 172.21.15.78:0/3141322872 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80005650 con 0x561b8feb8800 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef6eff5640 1 -- 172.21.15.78:0/3141322872 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fef8001ca50 con 0x561b8feb8800 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef938c3740 1 -- 172.21.15.78:0/3141322872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef7c0ac410 con 0x561b8feb8800 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef6eff5640 1 -- 172.21.15.78:0/3141322872 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fef800050b0 con 0x561b8feb8800 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef938c3740 10 client.26024 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef938c3740 1 -- 172.21.15.78:0/3141322872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561b8ff41e50 con 0x561b8feb8800 2024-09-11T01:05:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef6eff5640 1 -- 172.21.15.78:0/3141322872 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c0ac410 con 0x561b8feb8800 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.127+0000 7fef6eff5640 1 -- 172.21.15.78:0/3141322872 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fef80025440 con 0x561b8feb8800 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.128+0000 7fef938c3740 10 client.26024 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.128+0000 7fef938c3740 10 client.26024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.128+0000 7fef938c3740 20 client.26024 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.128+0000 7fef938c3740 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b8ff42340 0x561b8ff44760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.128+0000 7fef938c3740 4 client.26024 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.128+0000 7fef938c3740 1 -- 172.21.15.78:0/3141322872 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561b8fe2c860 con 0x561b8ff42340 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.128+0000 7fef8affd640 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b8ff42340 0x561b8ff44760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.128+0000 7fef8affd640 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b8ff42340 0x561b8ff44760 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853ca5810c3b2e80 server_cookie=368974969a65723e in_seq=0 out_seq=0 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.128+0000 7fef6eff5640 10 client.26024 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.129+0000 7fef6eff5640 1 -- 172.21.15.78:0/3141322872 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561b8fe2c860 con 0x561b8ff42340 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.129+0000 7fef6eff5640 10 client.26024 handle_command_reply: tid=0 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.129+0000 7fef938c3740 1 client.26024 shutdown 2024-09-11T01:05:13.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.130+0000 7fef938c3740 1 -- 172.21.15.78:0/3141322872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb8800 msgr2=0x561b8fed8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.130+0000 7fef938c3740 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb8800 0x561b8fed8d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fef80009d60 tx=0x7fef80005260 comp rx=0 tx=0).stop 2024-09-11T01:05:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.130+0000 7fef938c3740 1 -- 172.21.15.78:0/3141322872 shutdown_connections 2024-09-11T01:05:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.130+0000 7fef938c3740 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b8fec4790 0x561b8ff407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.130+0000 7fef938c3740 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b8feb8800 0x561b8fed8d30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.130+0000 7fef938c3740 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b8ff42340 0x561b8ff44760 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.130+0000 7fef6eff5640 0 client.26024 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.130+0000 7fef938c3740 1 --2- 172.21.15.78:0/3141322872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b8feb7ee0 0x561b8fed87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.138 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:05:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:05:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "6b1f53cf-607b-417b-b81b-ce9af7347e46", 2024-09-11T01:05:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:05:13.139 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:05:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.130+0000 7fef938c3740 1 -- 172.21.15.78:0/3141322872 >> 172.21.15.78:0/3141322872 conn(0x561b8feb5d10 msgr2=0x561b8fecfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.130+0000 7fef938c3740 1 -- 172.21.15.78:0/3141322872 shutdown_connections 2024-09-11T01:05:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.130+0000 7fef938c3740 1 -- 172.21.15.78:0/3141322872 wait complete. 2024-09-11T01:05:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.130+0000 7fef938c3740 20 client.26024 trim_cache size 0 max 16384 2024-09-11T01:05:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.131+0000 7fef914ee640 1 -- 172.21.15.78:0/1295007832 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fef740b7660 msgr2=0x7fef740b9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:13.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.131+0000 7fef914ee640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fef740b7660 0x7fef740b9b20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fef84006010 tx=0x7fef84009360 comp rx=0 tx=0).stop 2024-09-11T01:05:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.131+0000 7fef914ee640 1 -- 172.21.15.78:0/1295007832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef8c101c90 msgr2=0x7fef8c197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.131+0000 7fef914ee640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef8c101c90 0x7fef8c197050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00e9f0 tx=0x7fef7c00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.131+0000 7fef914ee640 1 -- 172.21.15.78:0/1295007832 shutdown_connections 2024-09-11T01:05:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.131+0000 7fef914ee640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fef740b7660 0x7fef740b9b20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.131+0000 7fef914ee640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef8c108020 0x7fef8c19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.131+0000 7fef914ee640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef8c1076a0 0x7fef8c197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.131+0000 7fef914ee640 1 --2- 172.21.15.78:0/1295007832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef8c101c90 0x7fef8c197050 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.131+0000 7fef914ee640 1 -- 172.21.15.78:0/1295007832 >> 172.21.15.78:0/1295007832 conn(0x7fef8c0fd700 msgr2=0x7fef8c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.131+0000 7fef914ee640 1 -- 172.21.15.78:0/1295007832 shutdown_connections 2024-09-11T01:05:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:13.131+0000 7fef914ee640 1 -- 172.21.15.78:0/1295007832 wait complete. 2024-09-11T01:05:13.434 INFO:tasks.workunit.client.0.smithi078.stdout: 1 469401 5.79 MB/sec execute 201 sec latency 51.613 ms 2024-09-11T01:05:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:13 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:05:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:13 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:13 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:13 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:13 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:13 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:13 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:13.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:13 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:05:13.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:13 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:13.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:13 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:13.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:13 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:13.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:13 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:13.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:13 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:13.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:13 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:13 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:05:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:13 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:13 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:13 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:13 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:13 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:13 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:13.976 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:05:14.237 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:14.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 470685 5.78 MB/sec execute 202 sec latency 344.410 ms 2024-09-11T01:05:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:14.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:14.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:14.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:14.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:14.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:14.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:14 smithi078 ceph-mon[31315]: pgmap v514: 97 pgs: 97 active+clean; 287 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-09-11T01:05:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:14.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:14.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:14.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:14.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:14.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:14.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:14.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:14.693 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:14.693 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.693 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.693 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.693 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:14 smithi110 ceph-mon[34712]: pgmap v514: 97 pgs: 97 active+clean; 287 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-09-11T01:05:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:14.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:14.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:14.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:14.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:14.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:14.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:14.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:14.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:14 smithi112 ceph-mon[35384]: pgmap v514: 97 pgs: 97 active+clean; 287 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-09-11T01:05:14.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.741+0000 7fce40727640 1 -- 172.21.15.78:0/289915688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce381076a0 msgr2=0x7fce38107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:14.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.741+0000 7fce40727640 1 --2- 172.21.15.78:0/289915688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce381076a0 0x7fce38107ae0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009900 tx=0x7fce2c02a410 comp rx=0 tx=0).stop 2024-09-11T01:05:14.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.741+0000 7fce40727640 1 -- 172.21.15.78:0/289915688 shutdown_connections 2024-09-11T01:05:14.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.741+0000 7fce40727640 1 --2- 172.21.15.78:0/289915688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce38108020 0x7fce3810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.741+0000 7fce40727640 1 --2- 172.21.15.78:0/289915688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce381076a0 0x7fce38107ae0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.741+0000 7fce40727640 1 --2- 172.21.15.78:0/289915688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce38101c90 0x7fce38102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.741+0000 7fce40727640 1 -- 172.21.15.78:0/289915688 >> 172.21.15.78:0/289915688 conn(0x7fce380fd700 msgr2=0x7fce380ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:14.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.741+0000 7fce40727640 1 -- 172.21.15.78:0/289915688 shutdown_connections 2024-09-11T01:05:14.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.741+0000 7fce40727640 1 -- 172.21.15.78:0/289915688 wait complete. 2024-09-11T01:05:14.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce40727640 1 Processor -- start 2024-09-11T01:05:14.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce40727640 1 -- start start 2024-09-11T01:05:14.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce40727640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce38101c90 0x7fce38199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce40727640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce381076a0 0x7fce381997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce40727640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce38108020 0x7fce3819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce40727640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce38112dc0 con 0x7fce38101c90 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce40727640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fce38112c40 con 0x7fce381076a0 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce40727640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce38112f40 con 0x7fce38108020 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3e4c3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce38101c90 0x7fce38199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3e4c3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce38101c90 0x7fce38199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57808/0 (socket says 172.21.15.78:57808) 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3e4c3640 1 -- 172.21.15.78:0/3379750057 learned_addr learned my addr 172.21.15.78:0/3379750057 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3dcc2640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce381076a0 0x7fce381997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3ecc4640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce38108020 0x7fce3819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3e4c3640 1 -- 172.21.15.78:0/3379750057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce38108020 msgr2=0x7fce3819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce38108020 0x7fce3819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3e4c3640 1 -- 172.21.15.78:0/3379750057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce381076a0 msgr2=0x7fce381997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce381076a0 0x7fce381997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3e4c3640 1 -- 172.21.15.78:0/3379750057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3819f320 con 0x7fce38101c90 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3dcc2640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce381076a0 0x7fce381997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3ecc4640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce38108020 0x7fce3819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.742+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce38101c90 0x7fce38199270 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fce3400c890 tx=0x7fce3400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.743+0000 7fce2b7fe640 1 -- 172.21.15.78:0/3379750057 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34004220 con 0x7fce38101c90 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.743+0000 7fce2b7fe640 1 -- 172.21.15.78:0/3379750057 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce340043c0 con 0x7fce38101c90 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.743+0000 7fce40727640 1 -- 172.21.15.78:0/3379750057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3819f610 con 0x7fce38101c90 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.743+0000 7fce2b7fe640 1 -- 172.21.15.78:0/3379750057 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34004a00 con 0x7fce38101c90 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.743+0000 7fce40727640 1 -- 172.21.15.78:0/3379750057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce381a5cd0 con 0x7fce38101c90 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.745+0000 7fce2b7fe640 1 -- 172.21.15.78:0/3379750057 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fce340079b0 con 0x7fce38101c90 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.745+0000 7fce42afc740 1 Processor -- start 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.745+0000 7fce42afc740 1 -- start start 2024-09-11T01:05:14.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce42afc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a4f480 0x558c41a4f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce42afc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a5b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce42afc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 0x558c41a5e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce42afc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c419c5e50 con 0x558c41a5bd30 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce42afc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558c419b5280 con 0x558c41a4f480 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce42afc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558c419b9090 con 0x558c41a4fda0 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce3dcc2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 0x558c41a5e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce3dcc2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 0x558c41a5e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57820/0 (socket says 172.21.15.78:57820) 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce3dcc2640 1 -- 172.21.15.78:0/3309032325 learned_addr learned my addr 172.21.15.78:0/3309032325 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/3309032325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a5b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce3ecc4640 1 --2- 172.21.15.78:0/3309032325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a4f480 0x558c41a4f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce3dcc2640 1 -- 172.21.15.78:0/3309032325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 msgr2=0x558c41a5b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce3dcc2640 1 --2- 172.21.15.78:0/3309032325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a5b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce3dcc2640 1 -- 172.21.15.78:0/3309032325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a4f480 msgr2=0x558c41a4f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:14.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce3dcc2640 1 --2- 172.21.15.78:0/3309032325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a4f480 0x558c41a4f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce3dcc2640 1 -- 172.21.15.78:0/3309032325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c41713650 con 0x558c41a5bd30 2024-09-11T01:05:14.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce2b7fe640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fce240b5450 0x7fce240b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce3ecc4640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fce240b5450 0x7fce240b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce3dcc2640 1 --2- 172.21.15.78:0/3309032325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 0x558c41a5e120 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fce2c006010 tx=0x7fce2c032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758f7c906839b41a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:14.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.746+0000 7fce2b7fe640 1 -- 172.21.15.78:0/3379750057 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fce340e5dc0 con 0x7fce38101c90 2024-09-11T01:05:14.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.747+0000 7fce28ff9640 1 -- 172.21.15.78:0/3309032325 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c03f070 con 0x558c41a5bd30 2024-09-11T01:05:14.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.747+0000 7fce28ff9640 1 -- 172.21.15.78:0/3309032325 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce2c032e20 con 0x558c41a5bd30 2024-09-11T01:05:14.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.747+0000 7fce28ff9640 1 -- 172.21.15.78:0/3309032325 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c035070 con 0x558c41a5bd30 2024-09-11T01:05:14.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.747+0000 7fce42afc740 1 -- 172.21.15.78:0/3309032325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 msgr2=0x558c41a5e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:14.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.747+0000 7fce42afc740 1 --2- 172.21.15.78:0/3309032325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 0x558c41a5e120 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fce2c006010 tx=0x7fce2c032c30 comp rx=0 tx=0).stop 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.747+0000 7fce3ecc4640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fce240b5450 0x7fce240b7910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fce30005fd0 tx=0x7fce30009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.747+0000 7fce42afc740 1 -- 172.21.15.78:0/3309032325 shutdown_connections 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.747+0000 7fce42afc740 1 --2- 172.21.15.78:0/3309032325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 0x558c41a5e120 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.747+0000 7fce42afc740 1 --2- 172.21.15.78:0/3309032325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a5b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.747+0000 7fce42afc740 1 --2- 172.21.15.78:0/3309032325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a4f480 0x558c41a4f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.747+0000 7fce42afc740 1 -- 172.21.15.78:0/3309032325 >> 172.21.15.78:0/3309032325 conn(0x558c41a4d2b0 msgr2=0x558c41a4e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce42afc740 1 -- 172.21.15.78:0/3309032325 shutdown_connections 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce42afc740 1 -- 172.21.15.78:0/3309032325 wait complete. 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce42afc740 1 -- start start 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce42afc740 1 Processor -- start 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce42afc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a4f480 0x558c41a5b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce42afc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a58a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce42afc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 0x558c41a59090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce42afc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c41816700 con 0x558c41a5bd30 2024-09-11T01:05:14.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce42afc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558c41950110 con 0x558c41a4f480 2024-09-11T01:05:14.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce42afc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558c41a5e950 con 0x558c41a4fda0 2024-09-11T01:05:14.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce42afc740 10 client.0 _ll_register_callbacks cb 0x558c41969990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:14.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce3ecc4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 0x558c41a59090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce3ecc4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 0x558c41a59090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57822/0 (socket says 172.21.15.78:57822) 2024-09-11T01:05:14.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce3ecc4640 1 -- 172.21.15.78:0/2234590592 learned_addr learned my addr 172.21.15.78:0/2234590592 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:14.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.748+0000 7fce3dcc2640 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a58a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a4f480 0x558c41a5b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce3ecc4640 1 -- 172.21.15.78:0/2234590592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 msgr2=0x558c41a58a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:14.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce3ecc4640 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a58a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce3ecc4640 1 -- 172.21.15.78:0/2234590592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a4f480 msgr2=0x558c41a5b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:14.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce3ecc4640 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a4f480 0x558c41a5b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce3ecc4640 1 -- 172.21.15.78:0/2234590592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c4197dc40 con 0x558c41a5bd30 2024-09-11T01:05:14.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a4f480 0x558c41a5b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:05:14.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce3dcc2640 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a58a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:14.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce3ecc4640 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 0x558c41a59090 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fce30012010 tx=0x7fce30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2f1a7ad42e0cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:14.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:14.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce21ffb640 1 -- 172.21.15.78:0/2234590592 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3000fcb0 con 0x558c41a5bd30 2024-09-11T01:05:14.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce21ffb640 1 -- 172.21.15.78:0/2234590592 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fce3001f420 con 0x558c41a5bd30 2024-09-11T01:05:14.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce42afc740 1 -- 172.21.15.78:0/2234590592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce2c032e20 con 0x558c41a5bd30 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce21ffb640 1 -- 172.21.15.78:0/2234590592 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fce30012e50 con 0x558c41a5bd30 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce42afc740 10 client.16719 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce42afc740 1 -- 172.21.15.78:0/2234590592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558c41a6a2d0 con 0x558c41a5bd30 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.749+0000 7fce21ffb640 1 -- 172.21.15.78:0/2234590592 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce2c032e20 con 0x558c41a5bd30 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.750+0000 7fce21ffb640 1 -- 172.21.15.78:0/2234590592 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fce30030620 con 0x558c41a5bd30 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.750+0000 7fce42afc740 10 client.16719 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.750+0000 7fce42afc740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.750+0000 7fce42afc740 20 client.16719 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.750+0000 7fce42afc740 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558c41a6b880 0x558c41a6dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.750+0000 7fce42afc740 4 client.16719 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.750+0000 7fce42afc740 1 -- 172.21.15.78:0/2234590592 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c419c7da0 con 0x558c41a6b880 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.750+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558c41a6b880 0x558c41a6dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.750+0000 7fce21ffb640 10 client.16719 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.750+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558c41a6b880 0x558c41a6dd20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db86f99186bd0a3a server_cookie=e65b780064dc6e6c in_seq=0 out_seq=0 2024-09-11T01:05:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce21ffb640 1 -- 172.21.15.78:0/2234590592 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c419c7da0 con 0x558c41a6b880 2024-09-11T01:05:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce21ffb640 10 client.16719 handle_command_reply: tid=0 2024-09-11T01:05:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce42afc740 1 client.16719 shutdown 2024-09-11T01:05:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce42afc740 1 -- 172.21.15.78:0/2234590592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 msgr2=0x558c41a59090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce42afc740 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 0x558c41a59090 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fce30012010 tx=0x7fce30004a40 comp rx=0 tx=0).stop 2024-09-11T01:05:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce42afc740 1 -- 172.21.15.78:0/2234590592 shutdown_connections 2024-09-11T01:05:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce42afc740 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558c41a6b880 0x558c41a6dd20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce42afc740 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a5bd30 0x558c41a59090 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce42afc740 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a58a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce42afc740 1 --2- 172.21.15.78:0/2234590592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a4f480 0x558c41a5b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce42afc740 1 -- 172.21.15.78:0/2234590592 >> 172.21.15.78:0/2234590592 conn(0x558c41a4d2b0 msgr2=0x558c41a52e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:14.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce21ffb640 0 client.16719 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:14.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce42afc740 1 -- 172.21.15.78:0/2234590592 shutdown_connections 2024-09-11T01:05:14.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce42afc740 1 -- 172.21.15.78:0/2234590592 wait complete. 2024-09-11T01:05:14.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.751+0000 7fce42afc740 20 client.16719 trim_cache size 0 max 16384 2024-09-11T01:05:14.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.768+0000 7fce42afc740 1 Processor -- start 2024-09-11T01:05:14.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.768+0000 7fce42afc740 1 -- start start 2024-09-11T01:05:14.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce42afc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 0x558c41a67b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce42afc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a5d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce42afc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a5bd30 0x558c41a5d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce42afc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce381ae5f0 con 0x558c41a4f480 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce42afc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fce381b01f0 con 0x558c41a5bd30 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce42afc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce381aecc0 con 0x558c41a4fda0 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce3e4c3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 0x558c41a67b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce3e4c3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 0x558c41a67b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57838/0 (socket says 172.21.15.78:57838) 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce3e4c3640 1 -- 172.21.15.78:0/1425232345 learned_addr learned my addr 172.21.15.78:0/1425232345 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce3ecc4640 1 --2- 172.21.15.78:0/1425232345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a5bd30 0x558c41a5d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce3e4c3640 1 -- 172.21.15.78:0/1425232345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 msgr2=0x558c41a5d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/1425232345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a5d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce3e4c3640 1 -- 172.21.15.78:0/1425232345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a5bd30 msgr2=0x558c41a5d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/1425232345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a5bd30 0x558c41a5d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce3e4c3640 1 -- 172.21.15.78:0/1425232345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c41a5b940 con 0x558c41a4f480 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce3ecc4640 1 --2- 172.21.15.78:0/1425232345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a5bd30 0x558c41a5d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:05:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/1425232345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 0x558c41a67b80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fce34005840 tx=0x7fce340e8340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1956eb24cdb210a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:14.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce23fff640 1 -- 172.21.15.78:0/1425232345 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3401d040 con 0x558c41a4f480 2024-09-11T01:05:14.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce23fff640 1 -- 172.21.15.78:0/1425232345 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce34005290 con 0x558c41a4f480 2024-09-11T01:05:14.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.769+0000 7fce23fff640 1 -- 172.21.15.78:0/1425232345 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce340ac2b0 con 0x558c41a4f480 2024-09-11T01:05:14.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.770+0000 7fce42afc740 1 -- 172.21.15.78:0/1425232345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 msgr2=0x558c41a67b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.770+0000 7fce42afc740 1 --2- 172.21.15.78:0/1425232345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 0x558c41a67b80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fce34005840 tx=0x7fce340e8340 comp rx=0 tx=0).stop 2024-09-11T01:05:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.770+0000 7fce42afc740 1 -- 172.21.15.78:0/1425232345 shutdown_connections 2024-09-11T01:05:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.770+0000 7fce42afc740 1 --2- 172.21.15.78:0/1425232345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a5bd30 0x558c41a5d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.770+0000 7fce42afc740 1 --2- 172.21.15.78:0/1425232345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a5d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.770+0000 7fce42afc740 1 --2- 172.21.15.78:0/1425232345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 0x558c41a67b80 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.770+0000 7fce42afc740 1 -- 172.21.15.78:0/1425232345 >> 172.21.15.78:0/1425232345 conn(0x558c41a4d2b0 msgr2=0x558c41a4e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.770+0000 7fce42afc740 1 -- 172.21.15.78:0/1425232345 shutdown_connections 2024-09-11T01:05:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.770+0000 7fce42afc740 1 -- 172.21.15.78:0/1425232345 wait complete. 2024-09-11T01:05:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce42afc740 1 -- start start 2024-09-11T01:05:14.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce42afc740 1 Processor -- start 2024-09-11T01:05:14.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce42afc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 0x558c41a63bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce42afc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a64130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce42afc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a5bd30 0x558c41a5eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce42afc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce381afa60 con 0x558c41a4f480 2024-09-11T01:05:14.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce42afc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fce381af390 con 0x558c41a5bd30 2024-09-11T01:05:14.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce42afc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce381ac5d0 con 0x558c41a4fda0 2024-09-11T01:05:14.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce42afc740 10 client.0 _ll_register_callbacks cb 0x7fce30005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:14.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce3e4c3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 0x558c41a63bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce3e4c3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 0x558c41a63bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57854/0 (socket says 172.21.15.78:57854) 2024-09-11T01:05:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce3e4c3640 1 -- 172.21.15.78:0/3726522168 learned_addr learned my addr 172.21.15.78:0/3726522168 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce3dcc2640 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a64130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce3ecc4640 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a5bd30 0x558c41a5eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce3e4c3640 1 -- 172.21.15.78:0/3726522168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 msgr2=0x558c41a64130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a64130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce3e4c3640 1 -- 172.21.15.78:0/3726522168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a5bd30 msgr2=0x558c41a5eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a5bd30 0x558c41a5eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce3e4c3640 1 -- 172.21.15.78:0/3726522168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c41a5f0e0 con 0x558c41a4f480 2024-09-11T01:05:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce3ecc4640 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a5bd30 0x558c41a5eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.771+0000 7fce3dcc2640 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a64130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:05:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.772+0000 7fce3e4c3640 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 0x558c41a63bf0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fce340e3b80 tx=0x7fce340076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c6ad0f5802087 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.772+0000 7fce21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.772+0000 7fce21ffb640 1 -- 172.21.15.78:0/3726522168 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34097020 con 0x558c41a4f480 2024-09-11T01:05:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.772+0000 7fce21ffb640 1 -- 172.21.15.78:0/3726522168 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fce340a8520 con 0x558c41a4f480 2024-09-11T01:05:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.772+0000 7fce42afc740 1 -- 172.21.15.78:0/3726522168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558c419c3850 con 0x558c41a4f480 2024-09-11T01:05:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.772+0000 7fce21ffb640 1 -- 172.21.15.78:0/3726522168 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fce3401d040 con 0x558c41a4f480 2024-09-11T01:05:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.772+0000 7fce42afc740 10 client.16725 fetch_fsmap learned FSMap version 40 2024-09-11T01:05:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.772+0000 7fce42afc740 1 -- 172.21.15.78:0/3726522168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558c41ad8f00 con 0x558c41a4f480 2024-09-11T01:05:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.772+0000 7fce21ffb640 1 -- 172.21.15.78:0/3726522168 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558c419c3850 con 0x558c41a4f480 2024-09-11T01:05:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.772+0000 7fce21ffb640 1 -- 172.21.15.78:0/3726522168 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8378+0+0 (secure 0 0 0) 0x7fce340ab510 con 0x558c41a4f480 2024-09-11T01:05:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.773+0000 7fce42afc740 10 client.16725 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T01:05:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.773+0000 7fce42afc740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.773+0000 7fce42afc740 20 client.16725 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.773+0000 7fce42afc740 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558c41ad9600 0x558c41adba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.773+0000 7fce42afc740 4 client.16725 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:05:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.773+0000 7fce42afc740 1 -- 172.21.15.78:0/3726522168 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c419c0c80 con 0x558c41ad9600 2024-09-11T01:05:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.773+0000 7fce3dcc2640 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558c41ad9600 0x558c41adba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.773+0000 7fce21ffb640 10 client.16725 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.773+0000 7fce3dcc2640 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558c41ad9600 0x558c41adba20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e15e2fbb57aa365c server_cookie=822a1681a9b116fe in_seq=0 out_seq=0 2024-09-11T01:05:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.773+0000 7fce21ffb640 1 -- 172.21.15.78:0/3726522168 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x558c419c0c80 con 0x558c41ad9600 2024-09-11T01:05:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.773+0000 7fce21ffb640 10 client.16725 handle_command_reply: tid=0 2024-09-11T01:05:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.773+0000 7fce42afc740 1 client.16725 shutdown 2024-09-11T01:05:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce42afc740 1 -- 172.21.15.78:0/3726522168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 msgr2=0x558c41a63bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce42afc740 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 0x558c41a63bf0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fce340e3b80 tx=0x7fce340076c0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce42afc740 1 -- 172.21.15.78:0/3726522168 shutdown_connections 2024-09-11T01:05:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce42afc740 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c41a5bd30 0x558c41a5eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce42afc740 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558c41a4fda0 0x558c41a64130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce42afc740 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558c41ad9600 0x558c41adba20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce42afc740 1 --2- 172.21.15.78:0/3726522168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c41a4f480 0x558c41a63bf0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce42afc740 1 -- 172.21.15.78:0/3726522168 >> 172.21.15.78:0/3726522168 conn(0x558c41a4d2b0 msgr2=0x558c41a4bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce21ffb640 0 client.16725 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce42afc740 1 -- 172.21.15.78:0/3726522168 shutdown_connections 2024-09-11T01:05:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce42afc740 1 -- 172.21.15.78:0/3726522168 wait complete. 2024-09-11T01:05:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce42afc740 20 client.16725 trim_cache size 0 max 16384 2024-09-11T01:05:14.784 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:05:14.784 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:05:14.784 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:05:14.784 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:05:14.784 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:14.784 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "6b1f53cf-607b-417b-b81b-ce9af7347e46", 2024-09-11T01:05:14.784 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:05:14.784 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:05:14.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:14.784 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:05:14.785 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:05:14.785 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:05:14.785 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:14.785 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:14.785 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:14.785 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:14.785 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:14.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:14.785 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:05:14.785 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:05:14.785 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:14.785 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:05:14.786 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:14.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:14.788 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:05:14.788 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:05:14.788 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:14.788 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:14.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:14.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:14.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:14.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:14.789 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:05:14.789 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:05:14.789 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:14.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:14.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:14.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:14.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:14.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:14.790 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:05:14.790 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:05:14.790 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:14.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:14.791 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:14.791 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:14.791 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:14.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:14.791 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:05:14.792 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:05:14.792 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:14.792 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:14.792 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:14.792 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:14.793 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:14.793 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:05:14.793 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:05:14.793 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:05:14.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce40727640 1 -- 172.21.15.78:0/3379750057 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fce240b5450 msgr2=0x7fce240b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:14.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce40727640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fce240b5450 0x7fce240b7910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fce30005fd0 tx=0x7fce30009500 comp rx=0 tx=0).stop 2024-09-11T01:05:14.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce40727640 1 -- 172.21.15.78:0/3379750057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce38101c90 msgr2=0x7fce38199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:14.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce40727640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce38101c90 0x7fce38199270 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fce3400c890 tx=0x7fce3400cd50 comp rx=0 tx=0).stop 2024-09-11T01:05:14.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.774+0000 7fce40727640 1 -- 172.21.15.78:0/3379750057 shutdown_connections 2024-09-11T01:05:14.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.775+0000 7fce40727640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fce240b5450 0x7fce240b7910 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.775+0000 7fce40727640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce38108020 0x7fce3819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.775+0000 7fce40727640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce381076a0 0x7fce381997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.775+0000 7fce40727640 1 --2- 172.21.15.78:0/3379750057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce38101c90 0x7fce38199270 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:14.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.775+0000 7fce40727640 1 -- 172.21.15.78:0/3379750057 >> 172.21.15.78:0/3379750057 conn(0x7fce380fd700 msgr2=0x7fce380ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:14.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.775+0000 7fce40727640 1 -- 172.21.15.78:0/3379750057 shutdown_connections 2024-09-11T01:05:14.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:14.775+0000 7fce40727640 1 -- 172.21.15.78:0/3379750057 wait complete. 2024-09-11T01:05:15.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 470685 5.76 MB/sec execute 203 sec latency 1344.450 ms 2024-09-11T01:05:15.594 INFO:tasks.cephfs.filesystem:scrub status for tag:6b1f53cf-607b-417b-b81b-ce9af7347e46 - {'path': '/', 'tag': '6b1f53cf-607b-417b-b81b-ce9af7347e46', 'options': 'recursive,force'} 2024-09-11T01:05:16.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 470685 5.73 MB/sec execute 204 sec latency 2344.548 ms 2024-09-11T01:05:16.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:16 smithi078 ceph-mon[31315]: pgmap v515: 97 pgs: 97 active+clean; 292 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.2 MiB/s wr, 200 op/s 2024-09-11T01:05:16.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:16 smithi110 ceph-mon[34712]: pgmap v515: 97 pgs: 97 active+clean; 292 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.2 MiB/s wr, 200 op/s 2024-09-11T01:05:16.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:16 smithi112 ceph-mon[35384]: pgmap v515: 97 pgs: 97 active+clean; 292 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.2 MiB/s wr, 200 op/s 2024-09-11T01:05:17.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 471386 5.71 MB/sec execute 205 sec latency 3104.260 ms 2024-09-11T01:05:18.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 473216 5.74 MB/sec execute 206 sec latency 9.353 ms 2024-09-11T01:05:18.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:18 smithi078 ceph-mon[31315]: pgmap v516: 97 pgs: 97 active+clean; 294 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 1.9 MiB/s wr, 139 op/s 2024-09-11T01:05:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:18 smithi110 ceph-mon[34712]: pgmap v516: 97 pgs: 97 active+clean; 294 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 1.9 MiB/s wr, 139 op/s 2024-09-11T01:05:18.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:18 smithi112 ceph-mon[35384]: pgmap v516: 97 pgs: 97 active+clean; 294 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 1.9 MiB/s wr, 139 op/s 2024-09-11T01:05:19.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 474231 5.74 MB/sec execute 207 sec latency 515.050 ms 2024-09-11T01:05:20.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 474231 5.71 MB/sec execute 208 sec latency 1515.137 ms 2024-09-11T01:05:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:20 smithi078 ceph-mon[31315]: pgmap v517: 97 pgs: 97 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-09-11T01:05:20.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:20 smithi110 ceph-mon[34712]: pgmap v517: 97 pgs: 97 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-09-11T01:05:20.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:20 smithi112 ceph-mon[35384]: pgmap v517: 97 pgs: 97 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-09-11T01:05:21.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 474231 5.68 MB/sec execute 209 sec latency 2515.236 ms 2024-09-11T01:05:22.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 475875 5.70 MB/sec execute 210 sec latency 2906.047 ms 2024-09-11T01:05:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:22 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:05:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:22 smithi112 ceph-mon[35384]: pgmap v518: 97 pgs: 97 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.7 MiB/s wr, 138 op/s 2024-09-11T01:05:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:22 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:05:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:22 smithi078 ceph-mon[31315]: pgmap v518: 97 pgs: 97 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.7 MiB/s wr, 138 op/s 2024-09-11T01:05:22.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:22 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:05:22.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:22 smithi110 ceph-mon[34712]: pgmap v518: 97 pgs: 97 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.7 MiB/s wr, 138 op/s 2024-09-11T01:05:23.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 477523 5.72 MB/sec execute 211 sec latency 12.620 ms 2024-09-11T01:05:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:23 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:05:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:23 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:05:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:23 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:05:23.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:23 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:05:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:23 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:05:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:23 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:05:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:23 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:05:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:23 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:05:23.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:23 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:05:23.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:23 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:05:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:23 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:05:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:23 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:05:24.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 477780 5.69 MB/sec execute 212 sec latency 863.734 ms 2024-09-11T01:05:24.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:24 smithi112 ceph-mon[35384]: pgmap v519: 97 pgs: 97 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-09-11T01:05:24.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:24 smithi078 ceph-mon[31315]: pgmap v519: 97 pgs: 97 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-09-11T01:05:24.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:24 smithi110 ceph-mon[34712]: pgmap v519: 97 pgs: 97 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-09-11T01:05:25.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 477780 5.67 MB/sec execute 213 sec latency 1863.820 ms 2024-09-11T01:05:26.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 477780 5.64 MB/sec execute 214 sec latency 2863.929 ms 2024-09-11T01:05:26.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:26 smithi078 ceph-mon[31315]: pgmap v520: 97 pgs: 97 active+clean; 302 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 2.2 MiB/s wr, 164 op/s 2024-09-11T01:05:26.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:26 smithi110 ceph-mon[34712]: pgmap v520: 97 pgs: 97 active+clean; 302 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 2.2 MiB/s wr, 164 op/s 2024-09-11T01:05:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:26 smithi112 ceph-mon[35384]: pgmap v520: 97 pgs: 97 active+clean; 302 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 2.2 MiB/s wr, 164 op/s 2024-09-11T01:05:27.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 478535 5.62 MB/sec execute 215 sec latency 3624.967 ms 2024-09-11T01:05:28.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 480379 5.65 MB/sec execute 216 sec latency 11.377 ms 2024-09-11T01:05:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:28 smithi078 ceph-mon[31315]: pgmap v521: 97 pgs: 97 active+clean; 304 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-09-11T01:05:28.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:28 smithi110 ceph-mon[34712]: pgmap v521: 97 pgs: 97 active+clean; 304 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-09-11T01:05:28.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:28 smithi112 ceph-mon[35384]: pgmap v521: 97 pgs: 97 active+clean; 304 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-09-11T01:05:29.435 INFO:tasks.workunit.client.0.smithi078.stdout: 1 481360 5.65 MB/sec execute 217 sec latency 516.917 ms 2024-09-11T01:05:30.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 481360 5.62 MB/sec execute 218 sec latency 1516.963 ms 2024-09-11T01:05:30.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:30 smithi078 ceph-mon[31315]: pgmap v522: 97 pgs: 97 active+clean; 315 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.7 MiB/s wr, 165 op/s 2024-09-11T01:05:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:30 smithi110 ceph-mon[34712]: pgmap v522: 97 pgs: 97 active+clean; 315 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.7 MiB/s wr, 165 op/s 2024-09-11T01:05:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:30 smithi112 ceph-mon[35384]: pgmap v522: 97 pgs: 97 active+clean; 315 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.7 MiB/s wr, 165 op/s 2024-09-11T01:05:31.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 483491 5.66 MB/sec execute 219 sec latency 1610.217 ms 2024-09-11T01:05:32.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 484911 5.66 MB/sec execute 220 sec latency 106.153 ms 2024-09-11T01:05:33.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:32 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:05:33.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:32 smithi110 ceph-mon[34712]: pgmap v523: 97 pgs: 97 active+clean; 316 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-09-11T01:05:33.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:32 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:05:33.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:32 smithi112 ceph-mon[35384]: pgmap v523: 97 pgs: 97 active+clean; 316 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-09-11T01:05:33.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:32 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:05:33.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:32 smithi078 ceph-mon[31315]: pgmap v523: 97 pgs: 97 active+clean; 316 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-09-11T01:05:33.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 484911 5.63 MB/sec execute 221 sec latency 1106.275 ms 2024-09-11T01:05:34.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 484923 5.61 MB/sec execute 222 sec latency 2086.115 ms 2024-09-11T01:05:34.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:34 smithi078 ceph-mon[31315]: pgmap v524: 97 pgs: 97 active+clean; 317 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 2.8 MiB/s wr, 151 op/s 2024-09-11T01:05:34.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:34 smithi110 ceph-mon[34712]: pgmap v524: 97 pgs: 97 active+clean; 317 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 2.8 MiB/s wr, 151 op/s 2024-09-11T01:05:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:34 smithi112 ceph-mon[35384]: pgmap v524: 97 pgs: 97 active+clean; 317 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 2.8 MiB/s wr, 151 op/s 2024-09-11T01:05:35.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 487158 5.64 MB/sec execute 223 sec latency 14.756 ms 2024-09-11T01:05:36.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 488453 5.64 MB/sec execute 224 sec latency 261.031 ms 2024-09-11T01:05:36.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:36 smithi078 ceph-mon[31315]: pgmap v525: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.1 MiB/s wr, 171 op/s 2024-09-11T01:05:36.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:36 smithi110 ceph-mon[34712]: pgmap v525: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.1 MiB/s wr, 171 op/s 2024-09-11T01:05:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:36 smithi112 ceph-mon[35384]: pgmap v525: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.1 MiB/s wr, 171 op/s 2024-09-11T01:05:37.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 489169 5.63 MB/sec execute 225 sec latency 1017.185 ms 2024-09-11T01:05:38.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 491030 5.66 MB/sec execute 226 sec latency 10.269 ms 2024-09-11T01:05:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:38 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:05:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:38 smithi078 ceph-mon[31315]: pgmap v526: 97 pgs: 97 active+clean; 320 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 2.6 MiB/s wr, 155 op/s 2024-09-11T01:05:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:38 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:05:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:38 smithi110 ceph-mon[34712]: pgmap v526: 97 pgs: 97 active+clean; 320 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 2.6 MiB/s wr, 155 op/s 2024-09-11T01:05:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:38 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:05:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:38 smithi112 ceph-mon[35384]: pgmap v526: 97 pgs: 97 active+clean; 320 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 2.6 MiB/s wr, 155 op/s 2024-09-11T01:05:39.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 492000 5.65 MB/sec execute 227 sec latency 523.841 ms 2024-09-11T01:05:40.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 492000 5.63 MB/sec execute 228 sec latency 1523.921 ms 2024-09-11T01:05:40.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:40 smithi078 ceph-mon[31315]: pgmap v527: 97 pgs: 97 active+clean; 324 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-09-11T01:05:40.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:40 smithi110 ceph-mon[34712]: pgmap v527: 97 pgs: 97 active+clean; 324 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-09-11T01:05:40.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:40 smithi112 ceph-mon[35384]: pgmap v527: 97 pgs: 97 active+clean; 324 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-09-11T01:05:41.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 492000 5.60 MB/sec execute 229 sec latency 2523.955 ms 2024-09-11T01:05:42.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 493569 5.62 MB/sec execute 230 sec latency 2914.263 ms 2024-09-11T01:05:42.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:42 smithi078 ceph-mon[31315]: pgmap v528: 97 pgs: 97 active+clean; 325 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-09-11T01:05:42.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:42 smithi110 ceph-mon[34712]: pgmap v528: 97 pgs: 97 active+clean; 325 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-09-11T01:05:42.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:42 smithi112 ceph-mon[35384]: pgmap v528: 97 pgs: 97 active+clean; 325 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-09-11T01:05:43.436 INFO:tasks.workunit.client.0.smithi078.stdout: 1 495104 5.62 MB/sec execute 231 sec latency 11.181 ms 2024-09-11T01:05:44.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 495544 5.61 MB/sec execute 232 sec latency 788.118 ms 2024-09-11T01:05:44.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:44 smithi078 ceph-mon[31315]: pgmap v529: 97 pgs: 97 active+clean; 327 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-09-11T01:05:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:44 smithi110 ceph-mon[34712]: pgmap v529: 97 pgs: 97 active+clean; 327 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-09-11T01:05:44.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:44 smithi112 ceph-mon[35384]: pgmap v529: 97 pgs: 97 active+clean; 327 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-09-11T01:05:45.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 495544 5.59 MB/sec execute 233 sec latency 1788.228 ms 2024-09-11T01:05:45.597 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:05:45.873 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:46 smithi078 ceph-mon[31315]: pgmap v530: 97 pgs: 97 active+clean; 327 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-09-11T01:05:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.382+0000 7fee1d99e640 1 -- 172.21.15.78:0/1924906180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee18103a00 msgr2=0x7fee18103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.382+0000 7fee1d99e640 1 --2- 172.21.15.78:0/1924906180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee18103a00 0x7fee18103de0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fee08009900 tx=0x7fee0802a410 comp rx=0 tx=0).stop 2024-09-11T01:05:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.382+0000 7fee1d99e640 1 -- 172.21.15.78:0/1924906180 shutdown_connections 2024-09-11T01:05:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.382+0000 7fee1d99e640 1 --2- 172.21.15.78:0/1924906180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1810cfb0 0x7fee1810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.382+0000 7fee1d99e640 1 --2- 172.21.15.78:0/1924906180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee18104400 0x7fee1810c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.382+0000 7fee1d99e640 1 --2- 172.21.15.78:0/1924906180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee18103a00 0x7fee18103de0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.382+0000 7fee1d99e640 1 -- 172.21.15.78:0/1924906180 >> 172.21.15.78:0/1924906180 conn(0x7fee180fd5b0 msgr2=0x7fee180ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:46.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.382+0000 7fee1d99e640 1 -- 172.21.15.78:0/1924906180 shutdown_connections 2024-09-11T01:05:46.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.382+0000 7fee1d99e640 1 -- 172.21.15.78:0/1924906180 wait complete. 2024-09-11T01:05:46.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.382+0000 7fee1d99e640 1 Processor -- start 2024-09-11T01:05:46.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.382+0000 7fee1d99e640 1 -- start start 2024-09-11T01:05:46.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d99e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee18103a00 0x7fee18199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d99e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee18104400 0x7fee18199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d99e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee1810cfb0 0x7fee1819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d99e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee18112cb0 con 0x7fee1810cfb0 2024-09-11T01:05:46.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d99e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee18112b30 con 0x7fee18103a00 2024-09-11T01:05:46.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d99e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee18112e30 con 0x7fee18104400 2024-09-11T01:05:46.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d19d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee1810cfb0 0x7fee1819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d19d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee1810cfb0 0x7fee1819eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59664/0 (socket says 172.21.15.78:59664) 2024-09-11T01:05:46.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d19d640 1 -- 172.21.15.78:0/3818349210 learned_addr learned my addr 172.21.15.78:0/3818349210 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:46.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1c99c640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee18103a00 0x7fee18199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee17fff640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee18104400 0x7fee18199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d19d640 1 -- 172.21.15.78:0/3818349210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee18104400 msgr2=0x7fee18199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:46.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d19d640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee18104400 0x7fee18199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d19d640 1 -- 172.21.15.78:0/3818349210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee18103a00 msgr2=0x7fee18199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:46.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d19d640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee18103a00 0x7fee18199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d19d640 1 -- 172.21.15.78:0/3818349210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1819f2d0 con 0x7fee1810cfb0 2024-09-11T01:05:46.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee17fff640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee18104400 0x7fee18199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:46.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d19d640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee1810cfb0 0x7fee1819eb00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fee1000b9e0 tx=0x7fee1000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:46.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee15ffb640 1 -- 172.21.15.78:0/3818349210 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10019070 con 0x7fee1810cfb0 2024-09-11T01:05:46.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee15ffb640 1 -- 172.21.15.78:0/3818349210 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee10014440 con 0x7fee1810cfb0 2024-09-11T01:05:46.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee1d99e640 1 -- 172.21.15.78:0/3818349210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1819f5c0 con 0x7fee1810cfb0 2024-09-11T01:05:46.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.383+0000 7fee15ffb640 1 -- 172.21.15.78:0/3818349210 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee100125a0 con 0x7fee1810cfb0 2024-09-11T01:05:46.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.384+0000 7fee1d99e640 1 -- 172.21.15.78:0/3818349210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee181a5c80 con 0x7fee1810cfb0 2024-09-11T01:05:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.386+0000 7fee15ffb640 1 -- 172.21.15.78:0/3818349210 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fee100145f0 con 0x7fee1810cfb0 2024-09-11T01:05:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.386+0000 7fee20fd5740 1 Processor -- start 2024-09-11T01:05:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.386+0000 7fee20fd5740 1 -- start start 2024-09-11T01:05:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.386+0000 7fee20fd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550a98a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.386+0000 7fee20fd5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550a98f70 0x557550aa49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.386+0000 7fee20fd5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550aa4f00 0x557550aa72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.386+0000 7fee20fd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557550a0ee50 con 0x557550a98650 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.386+0000 7fee20fd5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5575509fe280 con 0x557550a98f70 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.386+0000 7fee20fd5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557550a02090 con 0x557550aa4f00 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.386+0000 7fee17fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550a98a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.386+0000 7fee17fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550a98a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59672/0 (socket says 172.21.15.78:59672) 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.386+0000 7fee17fff640 1 -- 172.21.15.78:0/978248322 learned_addr learned my addr 172.21.15.78:0/978248322 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fee1c99c640 1 --2- 172.21.15.78:0/978248322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550a98f70 0x557550aa49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fee17fff640 1 --2- 172.21.15.78:0/978248322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550aa4f00 0x557550aa72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fee17fff640 1 -- 172.21.15.78:0/978248322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550aa4f00 msgr2=0x557550aa72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fee17fff640 1 --2- 172.21.15.78:0/978248322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550aa4f00 0x557550aa72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fee17fff640 1 -- 172.21.15.78:0/978248322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550a98f70 msgr2=0x557550aa49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fee17fff640 1 --2- 172.21.15.78:0/978248322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550a98f70 0x557550aa49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fee17fff640 1 -- 172.21.15.78:0/978248322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55755075c650 con 0x557550a98650 2024-09-11T01:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fee15ffb640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee000b5500 0x7fee000b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fee1c99c640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee000b5500 0x7fee000b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fee17fff640 1 --2- 172.21.15.78:0/978248322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550a98a30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00af40 tx=0x7fee0c00a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af160bc5e4687a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fee15ffb640 1 -- 172.21.15.78:0/3818349210 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fee100e5670 con 0x7fee1810cfb0 2024-09-11T01:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fedfeffd640 1 -- 172.21.15.78:0/978248322 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c01b070 con 0x557550a98650 2024-09-11T01:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fedfeffd640 1 -- 172.21.15.78:0/978248322 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee0c0168a0 con 0x557550a98650 2024-09-11T01:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.387+0000 7fedfeffd640 1 -- 172.21.15.78:0/978248322 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c00f040 con 0x557550a98650 2024-09-11T01:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.388+0000 7fee20fd5740 1 -- 172.21.15.78:0/978248322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 msgr2=0x557550a98a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.388+0000 7fee1c99c640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee000b5500 0x7fee000b79c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fee08031440 tx=0x7fee08005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.388+0000 7fee20fd5740 1 --2- 172.21.15.78:0/978248322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550a98a30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00af40 tx=0x7fee0c00a410 comp rx=0 tx=0).stop 2024-09-11T01:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.388+0000 7fee20fd5740 1 -- 172.21.15.78:0/978248322 shutdown_connections 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.388+0000 7fee20fd5740 1 --2- 172.21.15.78:0/978248322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550aa4f00 0x557550aa72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.388+0000 7fee20fd5740 1 --2- 172.21.15.78:0/978248322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550a98f70 0x557550aa49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.388+0000 7fee20fd5740 1 --2- 172.21.15.78:0/978248322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550a98a30 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.388+0000 7fee20fd5740 1 -- 172.21.15.78:0/978248322 >> 172.21.15.78:0/978248322 conn(0x557550a96480 msgr2=0x557550a97620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.388+0000 7fee20fd5740 1 -- 172.21.15.78:0/978248322 shutdown_connections 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.388+0000 7fee20fd5740 1 -- 172.21.15.78:0/978248322 wait complete. 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.388+0000 7fee20fd5740 1 -- start start 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.388+0000 7fee20fd5740 1 Processor -- start 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee20fd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550aa4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee20fd5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550aa1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee20fd5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 0x557550aa22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee20fd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55755085f700 con 0x557550a98650 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee20fd5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557550999110 con 0x557550aa4f00 2024-09-11T01:05:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee20fd5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557550aa7b20 con 0x557550a98f70 2024-09-11T01:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee20fd5740 10 client.0 _ll_register_callbacks cb 0x5575509b2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee1c99c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550aa4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee1c99c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550aa4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59676/0 (socket says 172.21.15.78:59676) 2024-09-11T01:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee1c99c640 1 -- 172.21.15.78:0/1521903244 learned_addr learned my addr 172.21.15.78:0/1521903244 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee17fff640 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550aa1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee1c99c640 1 -- 172.21.15.78:0/1521903244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 msgr2=0x557550aa1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee1c99c640 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550aa1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee1c99c640 1 -- 172.21.15.78:0/1521903244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 msgr2=0x557550aa22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee1d19d640 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 0x557550aa22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee1c99c640 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 0x557550aa22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee1c99c640 1 -- 172.21.15.78:0/1521903244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575509c6c40 con 0x557550a98650 2024-09-11T01:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fee1c99c640 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550aa4820 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fee08033000 tx=0x7fee08009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be3aee2324ee3a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.389+0000 7fedf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fedf7fff640 1 -- 172.21.15.78:0/1521903244 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0804c070 con 0x557550a98650 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fee20fd5740 1 -- 172.21.15.78:0/1521903244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee0c0168a0 con 0x557550a98650 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fedf7fff640 1 -- 172.21.15.78:0/1521903244 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fee080035f0 con 0x557550a98650 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fedf7fff640 1 -- 172.21.15.78:0/1521903244 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fee08058050 con 0x557550a98650 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fee20fd5740 10 client.16737 fetch_fsmap learned FSMap version 42 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fee20fd5740 1 -- 172.21.15.78:0/1521903244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557550ab34d0 con 0x557550a98650 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fedf7fff640 1 -- 172.21.15.78:0/1521903244 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee0c0168a0 con 0x557550a98650 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fedf7fff640 1 -- 172.21.15.78:0/1521903244 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7fee0803c9c0 con 0x557550a98650 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fee20fd5740 10 client.16737 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fee20fd5740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fee20fd5740 20 client.16737 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fee20fd5740 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557550ab49f0 0x557550ab6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fee20fd5740 4 client.16737 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.390+0000 7fee20fd5740 1 -- 172.21.15.78:0/1521903244 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557550a10da0 con 0x557550ab49f0 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.391+0000 7fee17fff640 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557550ab49f0 0x557550ab6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.391+0000 7fee17fff640 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557550ab49f0 0x557550ab6e70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab4d7d00ab60208c server_cookie=383bcb915a75098a in_seq=0 out_seq=0 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.391+0000 7fedf7fff640 10 client.16737 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fedf7fff640 1 -- 172.21.15.78:0/1521903244 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557550a10da0 con 0x557550ab49f0 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fedf7fff640 10 client.16737 handle_command_reply: tid=0 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fee20fd5740 1 client.16737 shutdown 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fee20fd5740 1 -- 172.21.15.78:0/1521903244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 msgr2=0x557550aa4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fee20fd5740 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550aa4820 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fee08033000 tx=0x7fee08009260 comp rx=0 tx=0).stop 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fee20fd5740 1 -- 172.21.15.78:0/1521903244 shutdown_connections 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fee20fd5740 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 0x557550aa22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fee20fd5740 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550aa1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fee20fd5740 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557550ab49f0 0x557550ab6e70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fee20fd5740 1 --2- 172.21.15.78:0/1521903244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550aa4820 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fee20fd5740 1 -- 172.21.15.78:0/1521903244 >> 172.21.15.78:0/1521903244 conn(0x557550a96480 msgr2=0x557550a9c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fedf7fff640 0 client.16737 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fee20fd5740 1 -- 172.21.15.78:0/1521903244 shutdown_connections 2024-09-11T01:05:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fee20fd5740 1 -- 172.21.15.78:0/1521903244 wait complete. 2024-09-11T01:05:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.392+0000 7fee20fd5740 20 client.16737 trim_cache size 0 max 16384 2024-09-11T01:05:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee20fd5740 1 Processor -- start 2024-09-11T01:05:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee20fd5740 1 -- start start 2024-09-11T01:05:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee20fd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550ab0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee20fd5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550a9ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee20fd5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 0x557550a9d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee20fd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee181ae430 con 0x557550a98650 2024-09-11T01:05:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee20fd5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee181b0090 con 0x557550aa4f00 2024-09-11T01:05:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee20fd5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee181aeb00 con 0x557550a98f70 2024-09-11T01:05:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee1c99c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550ab0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee1c99c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550ab0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59690/0 (socket says 172.21.15.78:59690) 2024-09-11T01:05:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee1c99c640 1 -- 172.21.15.78:0/679027444 learned_addr learned my addr 172.21.15.78:0/679027444 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee17fff640 1 --2- 172.21.15.78:0/679027444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550a9ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee1c99c640 1 -- 172.21.15.78:0/679027444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 msgr2=0x557550a9ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee1c99c640 1 --2- 172.21.15.78:0/679027444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550a9ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee1c99c640 1 -- 172.21.15.78:0/679027444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 msgr2=0x557550a9d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee1c99c640 1 --2- 172.21.15.78:0/679027444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 0x557550a9d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee1c99c640 1 -- 172.21.15.78:0/679027444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557550aa2610 con 0x557550a98650 2024-09-11T01:05:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee17fff640 1 --2- 172.21.15.78:0/679027444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550a9ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.409+0000 7fee1c99c640 1 --2- 172.21.15.78:0/679027444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550ab0d50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fee0803cd90 tx=0x7fee0802abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bc17431449be1a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.410+0000 7fedfe7fc640 1 -- 172.21.15.78:0/679027444 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0804c070 con 0x557550a98650 2024-09-11T01:05:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.410+0000 7fedfe7fc640 1 -- 172.21.15.78:0/679027444 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee08061070 con 0x557550a98650 2024-09-11T01:05:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.410+0000 7fedfe7fc640 1 -- 172.21.15.78:0/679027444 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08035040 con 0x557550a98650 2024-09-11T01:05:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.410+0000 7fee20fd5740 1 -- 172.21.15.78:0/679027444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 msgr2=0x557550ab0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.410+0000 7fee20fd5740 1 --2- 172.21.15.78:0/679027444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550ab0d50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fee0803cd90 tx=0x7fee0802abc0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.410+0000 7fee20fd5740 1 -- 172.21.15.78:0/679027444 shutdown_connections 2024-09-11T01:05:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.410+0000 7fee20fd5740 1 --2- 172.21.15.78:0/679027444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 0x557550a9d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.410+0000 7fee20fd5740 1 --2- 172.21.15.78:0/679027444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550a9ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.410+0000 7fee20fd5740 1 --2- 172.21.15.78:0/679027444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550ab0d50 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.410+0000 7fee20fd5740 1 -- 172.21.15.78:0/679027444 >> 172.21.15.78:0/679027444 conn(0x557550a96480 msgr2=0x557550a91520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.410+0000 7fee20fd5740 1 -- 172.21.15.78:0/679027444 shutdown_connections 2024-09-11T01:05:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.410+0000 7fee20fd5740 1 -- 172.21.15.78:0/679027444 wait complete. 2024-09-11T01:05:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee20fd5740 1 -- start start 2024-09-11T01:05:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee20fd5740 1 Processor -- start 2024-09-11T01:05:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee20fd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550aacdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee20fd5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550aad300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee20fd5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 0x557550aa7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee20fd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee181af900 con 0x557550a98650 2024-09-11T01:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee20fd5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee181af230 con 0x557550aa4f00 2024-09-11T01:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee20fd5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee181ac410 con 0x557550a98f70 2024-09-11T01:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee20fd5740 10 client.0 _ll_register_callbacks cb 0x7fee08009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee1c99c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550aacdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee1c99c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550aacdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59704/0 (socket says 172.21.15.78:59704) 2024-09-11T01:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee1c99c640 1 -- 172.21.15.78:0/1443154765 learned_addr learned my addr 172.21.15.78:0/1443154765 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee17fff640 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550aad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee1c99c640 1 -- 172.21.15.78:0/1443154765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 msgr2=0x557550aad300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee1c99c640 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550aad300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee1c99c640 1 -- 172.21.15.78:0/1443154765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 msgr2=0x557550aa7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.411+0000 7fee1d19d640 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 0x557550aa7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fee1c99c640 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 0x557550aa7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fee1c99c640 1 -- 172.21.15.78:0/1443154765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557550aadb00 con 0x557550a98650 2024-09-11T01:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fee17fff640 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550aad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fee1c99c640 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550aacdc0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fee08003890 tx=0x7fee080045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25644cd52250e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fedf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fedf7fff640 1 -- 172.21.15.78:0/1443154765 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0804c040 con 0x557550a98650 2024-09-11T01:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fedf7fff640 1 -- 172.21.15.78:0/1443154765 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fee0803c580 con 0x557550a98650 2024-09-11T01:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fee20fd5740 1 -- 172.21.15.78:0/1443154765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee08061070 con 0x557550a98650 2024-09-11T01:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fedf7fff640 1 -- 172.21.15.78:0/1443154765 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fee08075050 con 0x557550a98650 2024-09-11T01:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fee20fd5740 10 client.16743 fetch_fsmap learned FSMap version 42 2024-09-11T01:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fee20fd5740 1 -- 172.21.15.78:0/1443154765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557550b21ec0 con 0x557550a98650 2024-09-11T01:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fedf7fff640 1 -- 172.21.15.78:0/1443154765 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee08061070 con 0x557550a98650 2024-09-11T01:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fedf7fff640 1 -- 172.21.15.78:0/1443154765 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7fee08067a60 con 0x557550a98650 2024-09-11T01:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fee20fd5740 10 client.16743 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T01:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fee20fd5740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.412+0000 7fee20fd5740 20 client.16743 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee20fd5740 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557550b225b0 0x557550b249d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee20fd5740 4 client.16743 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee20fd5740 1 -- 172.21.15.78:0/1443154765 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557550a0c850 con 0x557550b225b0 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee17fff640 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557550b225b0 0x557550b249d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fedf7fff640 10 client.16743 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee17fff640 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557550b225b0 0x557550b249d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4556ca9974f8fc9 server_cookie=77ad0745a2beaf3d in_seq=0 out_seq=0 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fedf7fff640 1 -- 172.21.15.78:0/1443154765 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x557550a0c850 con 0x557550b225b0 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fedf7fff640 10 client.16743 handle_command_reply: tid=0 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee20fd5740 1 client.16743 shutdown 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee20fd5740 1 -- 172.21.15.78:0/1443154765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 msgr2=0x557550aacdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee20fd5740 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550aacdc0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fee08003890 tx=0x7fee080045c0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee20fd5740 1 -- 172.21.15.78:0/1443154765 shutdown_connections 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee20fd5740 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557550aa4f00 0x557550aa7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee20fd5740 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557550a98f70 0x557550aad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee20fd5740 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557550b225b0 0x557550b249d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee20fd5740 1 --2- 172.21.15.78:0/1443154765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557550a98650 0x557550aacdc0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fedf7fff640 0 client.16743 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:46.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.413+0000 7fee20fd5740 1 -- 172.21.15.78:0/1443154765 >> 172.21.15.78:0/1443154765 conn(0x557550a96480 msgr2=0x557550a9ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:46.439 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:05:46.439 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:05:46.439 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:05:46.439 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:05:46.439 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:05:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:05:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:05:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:05:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:46.441 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:46.441 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:46.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:46.441 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:05:46.441 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:05:46.441 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:46.441 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:46.441 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:46.441 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:46.441 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:46.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:05:46.443 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:46.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:05:46.445 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:05:46.445 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:05:46.445 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:05:46.445 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:05:46.445 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:05:46.445 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:05:46.445 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:05:46.445 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:05:46.445 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:05:46.445 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:05:46.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee20fd5740 1 -- 172.21.15.78:0/1443154765 shutdown_connections 2024-09-11T01:05:46.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee20fd5740 1 -- 172.21.15.78:0/1443154765 wait complete. 2024-09-11T01:05:46.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee20fd5740 20 client.16743 trim_cache size 0 max 16384 2024-09-11T01:05:46.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee1d99e640 1 -- 172.21.15.78:0/3818349210 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee000b5500 msgr2=0x7fee000b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:46.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee1d99e640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee000b5500 0x7fee000b79c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fee08031440 tx=0x7fee08005e80 comp rx=0 tx=0).stop 2024-09-11T01:05:46.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee1d99e640 1 -- 172.21.15.78:0/3818349210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee1810cfb0 msgr2=0x7fee1819eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:46.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee1d99e640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee1810cfb0 0x7fee1819eb00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fee1000b9e0 tx=0x7fee1000bea0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee1d99e640 1 -- 172.21.15.78:0/3818349210 shutdown_connections 2024-09-11T01:05:46.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee1d99e640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee000b5500 0x7fee000b79c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee1d99e640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee1810cfb0 0x7fee1819eb00 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee1d99e640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee18104400 0x7fee18199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee1d99e640 1 --2- 172.21.15.78:0/3818349210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee18103a00 0x7fee18199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:46.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee1d99e640 1 -- 172.21.15.78:0/3818349210 >> 172.21.15.78:0/3818349210 conn(0x7fee180fd5b0 msgr2=0x7fee180fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:46.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee1d99e640 1 -- 172.21.15.78:0/3818349210 shutdown_connections 2024-09-11T01:05:46.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:46.414+0000 7fee1d99e640 1 -- 172.21.15.78:0/3818349210 wait complete. 2024-09-11T01:05:46.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 495544 5.57 MB/sec execute 234 sec latency 2788.271 ms 2024-09-11T01:05:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:46 smithi110 ceph-mon[34712]: pgmap v530: 97 pgs: 97 active+clean; 327 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-09-11T01:05:46.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:46 smithi112 ceph-mon[35384]: pgmap v530: 97 pgs: 97 active+clean; 327 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-09-11T01:05:47.328 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:05:47.329 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:05:47.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 496254 5.55 MB/sec execute 235 sec latency 3543.346 ms 2024-09-11T01:05:47.598 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 -- 172.21.15.78:0/3523427592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79b4101c90 msgr2=0x7f79b4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 --2- 172.21.15.78:0/3523427592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79b4101c90 0x7f79b4102070 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009900 tx=0x7f79a802a410 comp rx=0 tx=0).stop 2024-09-11T01:05:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 -- 172.21.15.78:0/3523427592 shutdown_connections 2024-09-11T01:05:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 --2- 172.21.15.78:0/3523427592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b4108020 0x7f79b410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 --2- 172.21.15.78:0/3523427592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b41076a0 0x7f79b4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 --2- 172.21.15.78:0/3523427592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79b4101c90 0x7f79b4102070 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 -- 172.21.15.78:0/3523427592 >> 172.21.15.78:0/3523427592 conn(0x7f79b40fd700 msgr2=0x7f79b40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 -- 172.21.15.78:0/3523427592 shutdown_connections 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 -- 172.21.15.78:0/3523427592 wait complete. 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 Processor -- start 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 -- start start 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101c90 0x7f79b419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b41076a0 0x7f79b419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79b4108020 0x7f79b41a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4111d30 con 0x7f79b4108020 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4111bb0 con 0x7f79b41076a0 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.116+0000 7f79bb00a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4111eb0 con 0x7f79b4101c90 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b8da6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101c90 0x7f79b419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b95a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79b4108020 0x7f79b41a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b41076a0 0x7f79b419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b8da6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101c90 0x7f79b419a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35922/0 (socket says 172.21.15.78:35922) 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b8da6640 1 -- 172.21.15.78:0/752474296 learned_addr learned my addr 172.21.15.78:0/752474296 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b8da6640 1 -- 172.21.15.78:0/752474296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b41076a0 msgr2=0x7f79b419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b8da6640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b41076a0 0x7f79b419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b8da6640 1 -- 172.21.15.78:0/752474296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79b4108020 msgr2=0x7f79b41a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b8da6640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79b4108020 0x7f79b41a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b8da6640 1 -- 172.21.15.78:0/752474296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b41a5750 con 0x7f79b4101c90 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b3fff640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b41076a0 0x7f79b419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b8da6640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101c90 0x7f79b419a190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005ab0 tx=0x7f79a8003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b1ffb640 1 -- 172.21.15.78:0/752474296 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a803f070 con 0x7f79b4101c90 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79bb00a640 1 -- 172.21.15.78:0/752474296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b41a5920 con 0x7f79b4101c90 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b1ffb640 1 -- 172.21.15.78:0/752474296 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79a8031d60 con 0x7f79b4101c90 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.117+0000 7f79b1ffb640 1 -- 172.21.15.78:0/752474296 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8035030 con 0x7f79b4101c90 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.118+0000 7f79bb00a640 1 -- 172.21.15.78:0/752474296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b41a5d80 con 0x7f79b4101c90 2024-09-11T01:05:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.118+0000 7f79bb00a640 1 -- 172.21.15.78:0/752474296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b4102f40 con 0x7f79b4101c90 2024-09-11T01:05:48.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.120+0000 7f79b1ffb640 1 -- 172.21.15.78:0/752474296 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f79a803b4f0 con 0x7f79b4101c90 2024-09-11T01:05:48.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.121+0000 7f79b1ffb640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f79a40b5450 0x7f79a40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:48.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.121+0000 7f79b3fff640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f79a40b5450 0x7f79a40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:48.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.121+0000 7f79b1ffb640 1 -- 172.21.15.78:0/752474296 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f79a8107090 con 0x7f79b4101c90 2024-09-11T01:05:48.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.121+0000 7f79b3fff640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f79a40b5450 0x7f79a40b7910 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f79b406b710 tx=0x7f79a0005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:48.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.124+0000 7f79b1ffb640 1 -- 172.21.15.78:0/752474296 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f79a80cd5d0 con 0x7f79b4101c90 2024-09-11T01:05:48.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.360+0000 7f79bb00a640 1 -- 172.21.15.78:0/752474296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79b4109e30 con 0x7f79b4101c90 2024-09-11T01:05:48.365 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:48 smithi078 ceph-mon[31315]: pgmap v531: 97 pgs: 97 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-09-11T01:05:48.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.362+0000 7f79b1ffb640 1 -- 172.21.15.78:0/752474296 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11989 (secure 0 0 0) 0x7f79a80d2000 con 0x7f79b4101c90 2024-09-11T01:05:48.365 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:05:48.366 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":42,"btime":"2024-09-11T01:05:37:232299+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:05:36.814780+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:05:48.366 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 42 2024-09-11T01:05:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.364+0000 7f79bb00a640 1 -- 172.21.15.78:0/752474296 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f79a40b5450 msgr2=0x7f79a40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.364+0000 7f79bb00a640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f79a40b5450 0x7f79a40b7910 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f79b406b710 tx=0x7f79a0005f90 comp rx=0 tx=0).stop 2024-09-11T01:05:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.364+0000 7f79bb00a640 1 -- 172.21.15.78:0/752474296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101c90 msgr2=0x7f79b419a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.364+0000 7f79bb00a640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101c90 0x7f79b419a190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005ab0 tx=0x7f79a8003c80 comp rx=0 tx=0).stop 2024-09-11T01:05:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.364+0000 7f79bb00a640 1 -- 172.21.15.78:0/752474296 shutdown_connections 2024-09-11T01:05:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.364+0000 7f79bb00a640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f79a40b5450 0x7f79a40b7910 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.364+0000 7f79bb00a640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79b4108020 0x7f79b41a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:48.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.364+0000 7f79bb00a640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b41076a0 0x7f79b419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:48.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.364+0000 7f79bb00a640 1 --2- 172.21.15.78:0/752474296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79b4101c90 0x7f79b419a190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:48.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.364+0000 7f79bb00a640 1 -- 172.21.15.78:0/752474296 >> 172.21.15.78:0/752474296 conn(0x7f79b40fd700 msgr2=0x7f79b4105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:48.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.364+0000 7f79bb00a640 1 -- 172.21.15.78:0/752474296 shutdown_connections 2024-09-11T01:05:48.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:48.364+0000 7f79bb00a640 1 -- 172.21.15.78:0/752474296 wait complete. 2024-09-11T01:05:48.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 497882 5.58 MB/sec execute 236 sec latency 31.402 ms 2024-09-11T01:05:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:48 smithi110 ceph-mon[34712]: pgmap v531: 97 pgs: 97 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-09-11T01:05:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:48 smithi112 ceph-mon[35384]: pgmap v531: 97 pgs: 97 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-09-11T01:05:49.097 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:05:49.346 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:49.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 499092 5.58 MB/sec execute 237 sec latency 363.795 ms 2024-09-11T01:05:49.467 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:49 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/752474296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:05:49.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:49 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/752474296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:05:49.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:49 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/752474296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:05:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.850+0000 7f89ce757640 1 -- 172.21.15.78:0/3486427036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c810d120 msgr2=0x7f89c810f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.850+0000 7f89ce757640 1 --2- 172.21.15.78:0/3486427036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c810d120 0x7f89c810f590 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009ca0 tx=0x7f89c002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.850+0000 7f89ce757640 1 -- 172.21.15.78:0/3486427036 shutdown_connections 2024-09-11T01:05:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.850+0000 7f89ce757640 1 --2- 172.21.15.78:0/3486427036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c810d120 0x7f89c810f590 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.850+0000 7f89ce757640 1 --2- 172.21.15.78:0/3486427036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c8104570 0x7f89c810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.850+0000 7f89ce757640 1 --2- 172.21.15.78:0/3486427036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89c8103b70 0x7f89c8103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.850+0000 7f89ce757640 1 -- 172.21.15.78:0/3486427036 >> 172.21.15.78:0/3486427036 conn(0x7f89c80fd700 msgr2=0x7f89c80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.850+0000 7f89ce757640 1 -- 172.21.15.78:0/3486427036 shutdown_connections 2024-09-11T01:05:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.850+0000 7f89ce757640 1 -- 172.21.15.78:0/3486427036 wait complete. 2024-09-11T01:05:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.851+0000 7f89ce757640 1 Processor -- start 2024-09-11T01:05:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.851+0000 7f89ce757640 1 -- start start 2024-09-11T01:05:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.851+0000 7f89ce757640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c8103b70 0x7f89c81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.851+0000 7f89ce757640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c8104570 0x7f89c81997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.851+0000 7f89ce757640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89c810d120 0x7f89c819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.851+0000 7f89ce757640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89c8112da0 con 0x7f89c8103b70 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.851+0000 7f89ce757640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f89c8112c20 con 0x7f89c810d120 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.851+0000 7f89ce757640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89c8112f20 con 0x7f89c8104570 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.851+0000 7f89c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c8103b70 0x7f89c81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.851+0000 7f89c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c8103b70 0x7f89c81992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59736/0 (socket says 172.21.15.78:59736) 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.851+0000 7f89c7fff640 1 -- 172.21.15.78:0/3820353744 learned_addr learned my addr 172.21.15.78:0/3820353744 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.851+0000 7f89bffff640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c8104570 0x7f89c81997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.852+0000 7f89cccf4640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89c810d120 0x7f89c819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.852+0000 7f89c7fff640 1 -- 172.21.15.78:0/3820353744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c8104570 msgr2=0x7f89c81997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.852+0000 7f89c7fff640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c8104570 0x7f89c81997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.852+0000 7f89c7fff640 1 -- 172.21.15.78:0/3820353744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89c810d120 msgr2=0x7f89c819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.852+0000 7f89c7fff640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89c810d120 0x7f89c819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.852+0000 7f89c7fff640 1 -- 172.21.15.78:0/3820353744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c819f360 con 0x7f89c8103b70 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.852+0000 7f89c7fff640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c8103b70 0x7f89c81992b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f89b800d950 tx=0x7f89b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.852+0000 7f89c5ffb640 1 -- 172.21.15.78:0/3820353744 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8017030 con 0x7f89c8103b70 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.852+0000 7f89c5ffb640 1 -- 172.21.15.78:0/3820353744 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b8004020 con 0x7f89c8103b70 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.852+0000 7f89ce757640 1 -- 172.21.15.78:0/3820353744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c819f650 con 0x7f89c8103b70 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.852+0000 7f89c5ffb640 1 -- 172.21.15.78:0/3820353744 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b80047a0 con 0x7f89c8103b70 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.852+0000 7f89ce757640 1 -- 172.21.15.78:0/3820353744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c81a5ee0 con 0x7f89c8103b70 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.854+0000 7f89d0b2c740 1 Processor -- start 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.854+0000 7f89d0b2c740 1 -- start start 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89d0b2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f2448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89c5ffb640 1 -- 172.21.15.78:0/3820353744 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f89b8009a30 con 0x7f89c8103b70 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89d0b2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 0x56457f250840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89d0b2c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f253170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89d0b2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56457f1baea0 con 0x56457f244df0 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89d0b2c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56457f1aa2d0 con 0x56457f250d80 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89d0b2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56457f1ae0e0 con 0x56457f2444d0 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 0x56457f250840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 0x56457f250840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59742/0 (socket says 172.21.15.78:59742) 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89c7fff640 1 -- 172.21.15.78:0/1812726531 learned_addr learned my addr 172.21.15.78:0/1812726531 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89bffff640 1 --2- 172.21.15.78:0/1812726531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f253170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89cccf4640 1 --2- 172.21.15.78:0/1812726531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f2448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89c7fff640 1 -- 172.21.15.78:0/1812726531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 msgr2=0x56457f2448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89c7fff640 1 --2- 172.21.15.78:0/1812726531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f2448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89c7fff640 1 -- 172.21.15.78:0/1812726531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 msgr2=0x56457f253170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89c7fff640 1 --2- 172.21.15.78:0/1812726531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f253170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89c7fff640 1 -- 172.21.15.78:0/1812726531 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56457ef08650 con 0x56457f244df0 2024-09-11T01:05:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89bffff640 1 --2- 172.21.15.78:0/1812726531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f253170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.855+0000 7f89c7fff640 1 --2- 172.21.15.78:0/1812726531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 0x56457f250840 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f89b80e3060 tx=0x7f89b80ebea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f296da0953893d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.856+0000 7f89be7fc640 1 -- 172.21.15.78:0/1812726531 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b80f4070 con 0x56457f244df0 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.856+0000 7f89be7fc640 1 -- 172.21.15.78:0/1812726531 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b80edda0 con 0x56457f244df0 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.856+0000 7f89be7fc640 1 -- 172.21.15.78:0/1812726531 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b80ef7d0 con 0x56457f244df0 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.856+0000 7f89c5ffb640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f89b40b54d0 0x7f89b40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.856+0000 7f89cccf4640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f89b40b54d0 0x7f89b40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.856+0000 7f89c5ffb640 1 -- 172.21.15.78:0/3820353744 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f89b80e5930 con 0x7f89c8103b70 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.856+0000 7f89d0b2c740 1 -- 172.21.15.78:0/1812726531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 msgr2=0x56457f250840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.856+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/1812726531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 0x56457f250840 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f89b80e3060 tx=0x7f89b80ebea0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 -- 172.21.15.78:0/1812726531 shutdown_connections 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/1812726531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f253170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/1812726531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 0x56457f250840 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/1812726531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f2448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 -- 172.21.15.78:0/1812726531 >> 172.21.15.78:0/1812726531 conn(0x56457f242300 msgr2=0x56457f2434a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 -- 172.21.15.78:0/1812726531 shutdown_connections 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89cccf4640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f89b40b54d0 0x7f89b40b7990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f89c0005e30 tx=0x7f89c0005c30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 -- 172.21.15.78:0/1812726531 wait complete. 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 -- start start 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 Processor -- start 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f2506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f244df0 0x56457f24daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f250d80 0x56457f24e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.857+0000 7f89d0b2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56457f00b700 con 0x56457f250d80 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89d0b2c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56457f145160 con 0x56457f244df0 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89d0b2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56457f253a00 con 0x56457f2444d0 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89d0b2c740 10 client.0 _ll_register_callbacks cb 0x56457f15e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89cccf4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f250d80 0x56457f24e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89cccf4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f250d80 0x56457f24e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59754/0 (socket says 172.21.15.78:59754) 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89cccf4640 1 -- 172.21.15.78:0/2140030263 learned_addr learned my addr 172.21.15.78:0/2140030263 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89bffff640 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f244df0 0x56457f24daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89c7fff640 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f2506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89cccf4640 1 -- 172.21.15.78:0/2140030263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 msgr2=0x56457f2506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89cccf4640 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f2506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89cccf4640 1 -- 172.21.15.78:0/2140030263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f244df0 msgr2=0x56457f24daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89cccf4640 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f244df0 0x56457f24daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89cccf4640 1 -- 172.21.15.78:0/2140030263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56457f172c90 con 0x56457f250d80 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89c7fff640 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f2506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89cccf4640 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f250d80 0x56457f24e120 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f89c0004af0 tx=0x7f89c002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600b2f687c7f96d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89a77fe640 1 -- 172.21.15.78:0/2140030263 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0003c40 con 0x56457f250d80 2024-09-11T01:05:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89a77fe640 1 -- 172.21.15.78:0/2140030263 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f89c0003de0 con 0x56457f250d80 2024-09-11T01:05:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89d0b2c740 1 -- 172.21.15.78:0/2140030263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b80edda0 con 0x56457f250d80 2024-09-11T01:05:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89a77fe640 1 -- 172.21.15.78:0/2140030263 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f89c0004040 con 0x56457f250d80 2024-09-11T01:05:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89d0b2c740 10 client.16761 fetch_fsmap learned FSMap version 42 2024-09-11T01:05:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.858+0000 7f89d0b2c740 1 -- 172.21.15.78:0/2140030263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56457f25f3c0 con 0x56457f250d80 2024-09-11T01:05:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.859+0000 7f89a77fe640 1 -- 172.21.15.78:0/2140030263 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b80edda0 con 0x56457f250d80 2024-09-11T01:05:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.859+0000 7f89a77fe640 1 -- 172.21.15.78:0/2140030263 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f89c0051070 con 0x56457f250d80 2024-09-11T01:05:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.859+0000 7f89d0b2c740 10 client.16761 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T01:05:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.859+0000 7f89d0b2c740 10 client.16761 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:05:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.859+0000 7f89d0b2c740 20 client.16761 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.859+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56457f2608c0 0x56457f262d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.859+0000 7f89d0b2c740 4 client.16761 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.859+0000 7f89d0b2c740 1 -- 172.21.15.78:0/2140030263 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56457f1bcdf0 con 0x56457f2608c0 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.859+0000 7f89c7fff640 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56457f2608c0 0x56457f262d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.859+0000 7f89a77fe640 10 client.16761 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.859+0000 7f89c7fff640 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56457f2608c0 0x56457f262d00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f0db2370427e9937 server_cookie=4d897a25433f93d3 in_seq=0 out_seq=0 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89a77fe640 1 -- 172.21.15.78:0/2140030263 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56457f1bcdf0 con 0x56457f2608c0 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89a77fe640 10 client.16761 handle_command_reply: tid=0 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89d0b2c740 1 client.16761 shutdown 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89d0b2c740 1 -- 172.21.15.78:0/2140030263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f250d80 msgr2=0x56457f24e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f250d80 0x56457f24e120 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f89c0004af0 tx=0x7f89c002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89d0b2c740 1 -- 172.21.15.78:0/2140030263 shutdown_connections 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56457f2608c0 0x56457f262d00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f250d80 0x56457f24e120 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f244df0 0x56457f24daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89a77fe640 0 client.16761 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:05:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/2140030263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f2506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89d0b2c740 1 -- 172.21.15.78:0/2140030263 >> 172.21.15.78:0/2140030263 conn(0x56457f242300 msgr2=0x56457f247ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89d0b2c740 1 -- 172.21.15.78:0/2140030263 shutdown_connections 2024-09-11T01:05:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89d0b2c740 1 -- 172.21.15.78:0/2140030263 wait complete. 2024-09-11T01:05:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.860+0000 7f89d0b2c740 20 client.16761 trim_cache size 0 max 16384 2024-09-11T01:05:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.873+0000 7f89d0b2c740 1 Processor -- start 2024-09-11T01:05:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.873+0000 7f89d0b2c740 1 -- start start 2024-09-11T01:05:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.873+0000 7f89d0b2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f25cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.873+0000 7f89d0b2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 0x56457f248d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.873+0000 7f89d0b2c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f249270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.873+0000 7f89d0b2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89c81ae5d0 con 0x56457f244df0 2024-09-11T01:05:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.873+0000 7f89d0b2c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f89c81b0230 con 0x56457f250d80 2024-09-11T01:05:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.873+0000 7f89d0b2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89c81aeca0 con 0x56457f2444d0 2024-09-11T01:05:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 0x56457f248d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 0x56457f248d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59770/0 (socket says 172.21.15.78:59770) 2024-09-11T01:05:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89bffff640 1 -- 172.21.15.78:0/1866755061 learned_addr learned my addr 172.21.15.78:0/1866755061 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89c7fff640 1 --2- 172.21.15.78:0/1866755061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f25cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89bffff640 1 -- 172.21.15.78:0/1866755061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 msgr2=0x56457f25cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89cccf4640 1 --2- 172.21.15.78:0/1866755061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f249270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89bffff640 1 --2- 172.21.15.78:0/1866755061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f25cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89bffff640 1 -- 172.21.15.78:0/1866755061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 msgr2=0x56457f249270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89bffff640 1 --2- 172.21.15.78:0/1866755061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f249270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89bffff640 1 -- 172.21.15.78:0/1866755061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56457f242fe0 con 0x56457f244df0 2024-09-11T01:05:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89bffff640 1 --2- 172.21.15.78:0/1866755061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 0x56457f248d30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f89b000b9f0 tx=0x7f89b000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3b4583b1b82903 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89bdffb640 1 -- 172.21.15.78:0/1866755061 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0003cf0 con 0x56457f244df0 2024-09-11T01:05:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89bdffb640 1 -- 172.21.15.78:0/1866755061 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b0020070 con 0x56457f244df0 2024-09-11T01:05:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.874+0000 7f89bdffb640 1 -- 172.21.15.78:0/1866755061 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b001b6f0 con 0x56457f244df0 2024-09-11T01:05:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 -- 172.21.15.78:0/1866755061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 msgr2=0x56457f248d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/1866755061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 0x56457f248d30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f89b000b9f0 tx=0x7f89b000beb0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 -- 172.21.15.78:0/1866755061 shutdown_connections 2024-09-11T01:05:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/1866755061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f249270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/1866755061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f244df0 0x56457f248d30 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/1866755061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f2444d0 0x56457f25cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 -- 172.21.15.78:0/1866755061 >> 172.21.15.78:0/1866755061 conn(0x56457f242300 msgr2=0x56457f2433a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 -- 172.21.15.78:0/1866755061 shutdown_connections 2024-09-11T01:05:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 -- 172.21.15.78:0/1866755061 wait complete. 2024-09-11T01:05:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 -- start start 2024-09-11T01:05:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 Processor -- start 2024-09-11T01:05:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f2444d0 0x56457f258f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f244df0 0x56457f259510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f253cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.875+0000 7f89d0b2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89c81afaa0 con 0x56457f2444d0 2024-09-11T01:05:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89d0b2c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f89c81af3d0 con 0x56457f250d80 2024-09-11T01:05:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89d0b2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89c81ac5b0 con 0x56457f244df0 2024-09-11T01:05:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89d0b2c740 10 client.0 _ll_register_callbacks cb 0x7f89c0004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f2444d0 0x56457f258f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f2444d0 0x56457f258f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59778/0 (socket says 172.21.15.78:59778) 2024-09-11T01:05:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89c7fff640 1 -- 172.21.15.78:0/3849325736 learned_addr learned my addr 172.21.15.78:0/3849325736 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89bffff640 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f244df0 0x56457f259510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89cccf4640 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f253cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89c7fff640 1 -- 172.21.15.78:0/3849325736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f244df0 msgr2=0x56457f259510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89c7fff640 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f244df0 0x56457f259510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89c7fff640 1 -- 172.21.15.78:0/3849325736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 msgr2=0x56457f253cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89c7fff640 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f253cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89c7fff640 1 -- 172.21.15.78:0/3849325736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56457f254520 con 0x56457f2444d0 2024-09-11T01:05:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89c7fff640 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f2444d0 0x56457f258f70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f89b80e36f0 tx=0x7f89b80ed790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e705e1aacc1d97 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89a77fe640 1 -- 172.21.15.78:0/3849325736 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b80f4040 con 0x56457f2444d0 2024-09-11T01:05:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89a77fe640 1 -- 172.21.15.78:0/3849325736 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f89b80eb320 con 0x56457f2444d0 2024-09-11T01:05:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89d0b2c740 1 -- 172.21.15.78:0/3849325736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b0020070 con 0x56457f2444d0 2024-09-11T01:05:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89a77fe640 1 -- 172.21.15.78:0/3849325736 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f89b80d0050 con 0x56457f2444d0 2024-09-11T01:05:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89d0b2c740 10 client.16767 fetch_fsmap learned FSMap version 42 2024-09-11T01:05:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.876+0000 7f89d0b2c740 1 -- 172.21.15.78:0/3849325736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56457f2d6b40 con 0x56457f2444d0 2024-09-11T01:05:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89a77fe640 1 -- 172.21.15.78:0/3849325736 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b0020070 con 0x56457f2444d0 2024-09-11T01:05:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89a77fe640 1 -- 172.21.15.78:0/3849325736 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f89b80e3370 con 0x56457f2444d0 2024-09-11T01:05:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89d0b2c740 10 client.16767 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T01:05:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89d0b2c740 10 client.16767 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:05:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89d0b2c740 20 client.16767 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56457f2d6e80 0x56457f2d9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89d0b2c740 4 client.16767 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:05:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89d0b2c740 1 -- 172.21.15.78:0/3849325736 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56457f1b88a0 con 0x56457f2d6e80 2024-09-11T01:05:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89bffff640 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56457f2d6e80 0x56457f2d9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89a77fe640 10 client.16767 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:05:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89bffff640 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56457f2d6e80 0x56457f2d9270 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=28e4b635db5ab935 server_cookie=ce80f7c7b017e61 in_seq=0 out_seq=0 2024-09-11T01:05:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89a77fe640 1 -- 172.21.15.78:0/3849325736 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56457f1b88a0 con 0x56457f2d6e80 2024-09-11T01:05:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89a77fe640 10 client.16767 handle_command_reply: tid=0 2024-09-11T01:05:49.906 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:05:49.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.877+0000 7f89d0b2c740 1 client.16767 shutdown 2024-09-11T01:05:49.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89d0b2c740 1 -- 172.21.15.78:0/3849325736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f2444d0 msgr2=0x56457f258f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:49.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f2444d0 0x56457f258f70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f89b80e36f0 tx=0x7f89b80ed790 comp rx=0 tx=0).stop 2024-09-11T01:05:49.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89d0b2c740 1 -- 172.21.15.78:0/3849325736 shutdown_connections 2024-09-11T01:05:49.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56457f2d6e80 0x56457f2d9270 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56457f250d80 0x56457f253cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56457f244df0 0x56457f259510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89a77fe640 0 client.16767 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89d0b2c740 1 --2- 172.21.15.78:0/3849325736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56457f2444d0 0x56457f258f70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89d0b2c740 1 -- 172.21.15.78:0/3849325736 >> 172.21.15.78:0/3849325736 conn(0x56457f242300 msgr2=0x56457f248d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89d0b2c740 1 -- 172.21.15.78:0/3849325736 shutdown_connections 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89d0b2c740 1 -- 172.21.15.78:0/3849325736 wait complete. 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89d0b2c740 20 client.16767 trim_cache size 0 max 16384 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89ce757640 1 -- 172.21.15.78:0/3820353744 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f89b40b54d0 msgr2=0x7f89b40b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89ce757640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f89b40b54d0 0x7f89b40b7990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f89c0005e30 tx=0x7f89c0005c30 comp rx=0 tx=0).stop 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89ce757640 1 -- 172.21.15.78:0/3820353744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c8103b70 msgr2=0x7f89c81992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89ce757640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c8103b70 0x7f89c81992b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f89b800d950 tx=0x7f89b800de10 comp rx=0 tx=0).stop 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89ce757640 1 -- 172.21.15.78:0/3820353744 shutdown_connections 2024-09-11T01:05:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89ce757640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f89b40b54d0 0x7f89b40b7990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89ce757640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89c810d120 0x7f89c819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89ce757640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c8104570 0x7f89c81997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89ce757640 1 --2- 172.21.15.78:0/3820353744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c8103b70 0x7f89c81992b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:49.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.878+0000 7f89ce757640 1 -- 172.21.15.78:0/3820353744 >> 172.21.15.78:0/3820353744 conn(0x7f89c80fd700 msgr2=0x7f89c80fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:49.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.879+0000 7f89ce757640 1 -- 172.21.15.78:0/3820353744 shutdown_connections 2024-09-11T01:05:49.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:49.879+0000 7f89ce757640 1 -- 172.21.15.78:0/3820353744 wait complete. 2024-09-11T01:05:50.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 499092 5.55 MB/sec execute 238 sec latency 1363.859 ms 2024-09-11T01:05:50.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:50 smithi078 ceph-mon[31315]: pgmap v532: 97 pgs: 97 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-09-11T01:05:50.656 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:05:50.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:50 smithi110 ceph-mon[34712]: pgmap v532: 97 pgs: 97 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-09-11T01:05:50.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:50 smithi112 ceph-mon[35384]: pgmap v532: 97 pgs: 97 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-09-11T01:05:50.902 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:51.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 499092 5.53 MB/sec execute 239 sec latency 2363.954 ms 2024-09-11T01:05:51.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.522+0000 7f53f3267640 1 -- 172.21.15.78:0/3428426526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53ec102da0 msgr2=0x7f53ec103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:51.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.522+0000 7f53f3267640 1 --2- 172.21.15.78:0/3428426526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53ec102da0 0x7f53ec103180 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009ca0 tx=0x7f53dc02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.522+0000 7f53f3267640 1 -- 172.21.15.78:0/3428426526 shutdown_connections 2024-09-11T01:05:51.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.522+0000 7f53f3267640 1 --2- 172.21.15.78:0/3428426526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53ec109130 0x7f53ec10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.522+0000 7f53f3267640 1 --2- 172.21.15.78:0/3428426526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53ec1087b0 0x7f53ec108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.522+0000 7f53f3267640 1 --2- 172.21.15.78:0/3428426526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53ec102da0 0x7f53ec103180 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.522+0000 7f53f3267640 1 -- 172.21.15.78:0/3428426526 >> 172.21.15.78:0/3428426526 conn(0x7f53ec0fe810 msgr2=0x7f53ec100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:51.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.522+0000 7f53f3267640 1 -- 172.21.15.78:0/3428426526 shutdown_connections 2024-09-11T01:05:51.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.522+0000 7f53f3267640 1 -- 172.21.15.78:0/3428426526 wait complete. 2024-09-11T01:05:51.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.522+0000 7f53f3267640 1 Processor -- start 2024-09-11T01:05:51.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.522+0000 7f53f3267640 1 -- start start 2024-09-11T01:05:51.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f3267640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53ec102da0 0x7f53ec19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f3267640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53ec1087b0 0x7f53ec19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f3267640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53ec109130 0x7f53ec1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f3267640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec112df0 con 0x7f53ec109130 2024-09-11T01:05:51.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f3267640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec112c70 con 0x7f53ec102da0 2024-09-11T01:05:51.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f3267640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec112f70 con 0x7f53ec1087b0 2024-09-11T01:05:51.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f1804640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53ec109130 0x7f53ec1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f1804640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53ec109130 0x7f53ec1a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59794/0 (socket says 172.21.15.78:59794) 2024-09-11T01:05:51.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f1804640 1 -- 172.21.15.78:0/3196679561 learned_addr learned my addr 172.21.15.78:0/3196679561 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:51.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f1003640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53ec102da0 0x7f53ec19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f0802640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53ec1087b0 0x7f53ec19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f1804640 1 -- 172.21.15.78:0/3196679561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53ec1087b0 msgr2=0x7f53ec19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:51.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f1804640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53ec1087b0 0x7f53ec19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f1804640 1 -- 172.21.15.78:0/3196679561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53ec102da0 msgr2=0x7f53ec19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f1804640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53ec102da0 0x7f53ec19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f1804640 1 -- 172.21.15.78:0/3196679561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec1a1510 con 0x7f53ec109130 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f1804640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53ec109130 0x7f53ec1a0d40 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f53e400e970 tx=0x7f53e400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53ea7fc640 1 -- 172.21.15.78:0/3196679561 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e400ccc0 con 0x7f53ec109130 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53ea7fc640 1 -- 172.21.15.78:0/3196679561 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e400ce60 con 0x7f53ec109130 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f3267640 1 -- 172.21.15.78:0/3196679561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ec1a1800 con 0x7f53ec109130 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53ea7fc640 1 -- 172.21.15.78:0/3196679561 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4010430 con 0x7f53ec109130 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.523+0000 7f53f3267640 1 -- 172.21.15.78:0/3196679561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ec1a7ec0 con 0x7f53ec109130 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.526+0000 7f53ea7fc640 1 -- 172.21.15.78:0/3196679561 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f53e4004f90 con 0x7f53ec109130 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.526+0000 7f53f563c740 1 Processor -- start 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.526+0000 7f53f563c740 1 -- start start 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.526+0000 7f53f563c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc0472c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.526+0000 7f53f563c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0472cdf0 0x55dc04738840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.526+0000 7f53f563c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc04738d80 0x55dc0473b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.526+0000 7f53f563c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dc046a2ea0 con 0x55dc0472c4d0 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.526+0000 7f53f563c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dc046922d0 con 0x55dc04738d80 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.526+0000 7f53f563c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc046960e0 con 0x55dc0472cdf0 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f0802640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc0472c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f0802640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc0472c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59804/0 (socket says 172.21.15.78:59804) 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f0802640 1 -- 172.21.15.78:0/675483634 learned_addr learned my addr 172.21.15.78:0/675483634 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f1003640 1 --2- 172.21.15.78:0/675483634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0472cdf0 0x55dc04738840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f0802640 1 --2- 172.21.15.78:0/675483634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc04738d80 0x55dc0473b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f0802640 1 -- 172.21.15.78:0/675483634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0472cdf0 msgr2=0x55dc04738840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f0802640 1 --2- 172.21.15.78:0/675483634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0472cdf0 0x55dc04738840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f0802640 1 -- 172.21.15.78:0/675483634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc04738d80 msgr2=0x55dc0473b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:51.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f0802640 1 --2- 172.21.15.78:0/675483634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc04738d80 0x55dc0473b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f0802640 1 -- 172.21.15.78:0/675483634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc043f0650 con 0x55dc0472c4d0 2024-09-11T01:05:51.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f1003640 1 --2- 172.21.15.78:0/675483634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0472cdf0 0x55dc04738840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:51.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53ea7fc640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f53d40b54d0 0x7f53d40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f0802640 1 --2- 172.21.15.78:0/675483634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc0472c8b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f53e000bfa0 tx=0x7f53e000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299338225a488f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:51.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53f1003640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f53d40b54d0 0x7f53d40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53ea7fc640 1 -- 172.21.15.78:0/3196679561 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f53e40e62b0 con 0x7f53ec109130 2024-09-11T01:05:51.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53d37fe640 1 -- 172.21.15.78:0/675483634 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0004030 con 0x55dc0472c4d0 2024-09-11T01:05:51.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53d37fe640 1 -- 172.21.15.78:0/675483634 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e00041d0 con 0x55dc0472c4d0 2024-09-11T01:05:51.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.527+0000 7f53d37fe640 1 -- 172.21.15.78:0/675483634 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0010460 con 0x55dc0472c4d0 2024-09-11T01:05:51.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.528+0000 7f53f563c740 1 -- 172.21.15.78:0/675483634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 msgr2=0x55dc0472c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:51.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.528+0000 7f53f563c740 1 --2- 172.21.15.78:0/675483634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc0472c8b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f53e000bfa0 tx=0x7f53e000b840 comp rx=0 tx=0).stop 2024-09-11T01:05:51.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.528+0000 7f53f1003640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f53d40b54d0 0x7f53d40b7990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55dc0472b1e0 tx=0x7f53dc009360 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:51.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.528+0000 7f53f563c740 1 -- 172.21.15.78:0/675483634 shutdown_connections 2024-09-11T01:05:51.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.528+0000 7f53f563c740 1 --2- 172.21.15.78:0/675483634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc04738d80 0x55dc0473b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.528+0000 7f53f563c740 1 --2- 172.21.15.78:0/675483634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0472cdf0 0x55dc04738840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.528+0000 7f53f563c740 1 --2- 172.21.15.78:0/675483634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc0472c8b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.528+0000 7f53f563c740 1 -- 172.21.15.78:0/675483634 >> 172.21.15.78:0/675483634 conn(0x55dc0472a300 msgr2=0x55dc0472b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:51.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.528+0000 7f53f563c740 1 -- 172.21.15.78:0/675483634 shutdown_connections 2024-09-11T01:05:51.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.528+0000 7f53f563c740 1 -- 172.21.15.78:0/675483634 wait complete. 2024-09-11T01:05:51.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f563c740 1 -- start start 2024-09-11T01:05:51.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f563c740 1 Processor -- start 2024-09-11T01:05:51.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f563c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc04738660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f563c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04735ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f563c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 0x55dc047360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f563c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dc044f3700 con 0x55dc0472c4d0 2024-09-11T01:05:51.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f563c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dc0462d160 con 0x55dc0472cdf0 2024-09-11T01:05:51.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f563c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc0473ba00 con 0x55dc04738d80 2024-09-11T01:05:51.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f563c740 10 client.0 _ll_register_callbacks cb 0x55dc046469e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:51.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f1003640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc04738660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f1003640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc04738660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59816/0 (socket says 172.21.15.78:59816) 2024-09-11T01:05:51.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f1003640 1 -- 172.21.15.78:0/3602057515 learned_addr learned my addr 172.21.15.78:0/3602057515 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:51.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f1804640 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 0x55dc047360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.529+0000 7f53f0802640 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04735ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53f1003640 1 -- 172.21.15.78:0/3602057515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 msgr2=0x55dc047360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53f1003640 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 0x55dc047360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53f1003640 1 -- 172.21.15.78:0/3602057515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 msgr2=0x55dc04735ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53f1003640 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04735ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53f1003640 1 -- 172.21.15.78:0/3602057515 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc0465ac90 con 0x55dc0472c4d0 2024-09-11T01:05:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53f1804640 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 0x55dc047360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53f1003640 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc04738660 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f53dc035000 tx=0x7f53dc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52cdcbd33ef3c40 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53d0ff9640 1 -- 172.21.15.78:0/3602057515 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc003c40 con 0x55dc0472c4d0 2024-09-11T01:05:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53d0ff9640 1 -- 172.21.15.78:0/3602057515 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f53dc003de0 con 0x55dc0472c4d0 2024-09-11T01:05:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53f563c740 1 -- 172.21.15.78:0/3602057515 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e00041d0 con 0x55dc0472c4d0 2024-09-11T01:05:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53d0ff9640 1 -- 172.21.15.78:0/3602057515 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f53dc004040 con 0x55dc0472c4d0 2024-09-11T01:05:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53f563c740 10 client.16779 fetch_fsmap learned FSMap version 42 2024-09-11T01:05:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53f563c740 1 -- 172.21.15.78:0/3602057515 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55dc04747350 con 0x55dc0472c4d0 2024-09-11T01:05:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.530+0000 7f53d0ff9640 1 -- 172.21.15.78:0/3602057515 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e00041d0 con 0x55dc0472c4d0 2024-09-11T01:05:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.531+0000 7f53d0ff9640 1 -- 172.21.15.78:0/3602057515 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f53dc051070 con 0x55dc0472c4d0 2024-09-11T01:05:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.531+0000 7f53f563c740 10 client.16779 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T01:05:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.531+0000 7f53f563c740 10 client.16779 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:05:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.531+0000 7f53f563c740 20 client.16779 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.531+0000 7f53f563c740 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55dc04748800 0x55dc0474ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.531+0000 7f53f563c740 4 client.16779 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.531+0000 7f53f563c740 1 -- 172.21.15.78:0/3602057515 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc046a4df0 con 0x55dc04748800 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.531+0000 7f53f0802640 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55dc04748800 0x55dc0474ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.531+0000 7f53f0802640 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55dc04748800 0x55dc0474ac40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=94c184f586fc7398 server_cookie=63a714b14de43ca0 in_seq=0 out_seq=0 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.531+0000 7f53d0ff9640 10 client.16779 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53d0ff9640 1 -- 172.21.15.78:0/3602057515 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc046a4df0 con 0x55dc04748800 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53d0ff9640 10 client.16779 handle_command_reply: tid=0 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53f563c740 1 client.16779 shutdown 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53f563c740 1 -- 172.21.15.78:0/3602057515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 msgr2=0x55dc04738660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53f563c740 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc04738660 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f53dc035000 tx=0x7f53dc02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53f563c740 1 -- 172.21.15.78:0/3602057515 shutdown_connections 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53f563c740 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55dc04748800 0x55dc0474ac40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53f563c740 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 0x55dc047360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53f563c740 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04735ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53f563c740 1 --2- 172.21.15.78:0/3602057515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc04738660 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53f563c740 1 -- 172.21.15.78:0/3602057515 >> 172.21.15.78:0/3602057515 conn(0x55dc0472a300 msgr2=0x55dc0472fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:51.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53d0ff9640 0 client.16779 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:05:51.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53f563c740 1 -- 172.21.15.78:0/3602057515 shutdown_connections 2024-09-11T01:05:51.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.532+0000 7f53f563c740 1 -- 172.21.15.78:0/3602057515 wait complete. 2024-09-11T01:05:51.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.533+0000 7f53f563c740 20 client.16779 trim_cache size 0 max 16384 2024-09-11T01:05:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f563c740 1 Processor -- start 2024-09-11T01:05:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f563c740 1 -- start start 2024-09-11T01:05:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f563c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc0472bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f563c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04744c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f563c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 0x55dc04725320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f563c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1b06d0 con 0x55dc0472c4d0 2024-09-11T01:05:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f563c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1b2330 con 0x55dc0472cdf0 2024-09-11T01:05:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f563c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1b0da0 con 0x55dc04738d80 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f1003640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc0472bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f1804640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 0x55dc04725320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f1804640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 0x55dc04725320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:36030/0 (socket says 172.21.15.78:36030) 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f1804640 1 -- 172.21.15.78:0/256306192 learned_addr learned my addr 172.21.15.78:0/256306192 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f0802640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04744c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f0802640 1 -- 172.21.15.78:0/256306192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 msgr2=0x55dc04725320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f0802640 1 --2- 172.21.15.78:0/256306192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 0x55dc04725320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f0802640 1 -- 172.21.15.78:0/256306192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 msgr2=0x55dc0472bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f0802640 1 --2- 172.21.15.78:0/256306192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc0472bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.547+0000 7f53f0802640 1 -- 172.21.15.78:0/256306192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc04725bb0 con 0x55dc0472cdf0 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.548+0000 7f53f1003640 1 --2- 172.21.15.78:0/256306192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc0472bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.548+0000 7f53f1804640 1 --2- 172.21.15.78:0/256306192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 0x55dc04725320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.548+0000 7f53f0802640 1 --2- 172.21.15.78:0/256306192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04744c60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55dc0472ae40 tx=0x7f53e00123f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cd438f12e7ccc01 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.548+0000 7f53d2ffd640 1 -- 172.21.15.78:0/256306192 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e000bca0 con 0x55dc0472cdf0 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.548+0000 7f53d2ffd640 1 -- 172.21.15.78:0/256306192 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e0005040 con 0x55dc0472cdf0 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.548+0000 7f53d2ffd640 1 -- 172.21.15.78:0/256306192 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0003f10 con 0x55dc0472cdf0 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.549+0000 7f53f563c740 1 -- 172.21.15.78:0/256306192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 msgr2=0x55dc04744c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.549+0000 7f53f563c740 1 --2- 172.21.15.78:0/256306192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04744c60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55dc0472ae40 tx=0x7f53e00123f0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.549+0000 7f53f563c740 1 -- 172.21.15.78:0/256306192 shutdown_connections 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.549+0000 7f53f563c740 1 --2- 172.21.15.78:0/256306192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc04738d80 0x55dc04725320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.549+0000 7f53f563c740 1 --2- 172.21.15.78:0/256306192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04744c60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.549+0000 7f53f563c740 1 --2- 172.21.15.78:0/256306192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0472c4d0 0x55dc0472bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.549+0000 7f53f563c740 1 -- 172.21.15.78:0/256306192 >> 172.21.15.78:0/256306192 conn(0x55dc0472a300 msgr2=0x55dc0472b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.549+0000 7f53f563c740 1 -- 172.21.15.78:0/256306192 shutdown_connections 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.549+0000 7f53f563c740 1 -- 172.21.15.78:0/256306192 wait complete. 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f563c740 1 -- start start 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f563c740 1 Processor -- start 2024-09-11T01:05:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f563c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0472c4d0 0x55dc04741660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f563c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04750320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f563c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc04738d80 0x55dc04750ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f563c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1b1ba0 con 0x55dc04738d80 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f563c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1b14d0 con 0x55dc0472cdf0 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f563c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1ae6b0 con 0x55dc0472c4d0 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f563c740 10 client.0 _ll_register_callbacks cb 0x7f53dc0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f1003640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0472c4d0 0x55dc04741660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f1003640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0472c4d0 0x55dc04741660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:36044/0 (socket says 172.21.15.78:36044) 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f0802640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04750320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f1003640 1 -- 172.21.15.78:0/2069901214 learned_addr learned my addr 172.21.15.78:0/2069901214 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.550+0000 7f53f1804640 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc04738d80 0x55dc04750ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53f0802640 1 -- 172.21.15.78:0/2069901214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0472c4d0 msgr2=0x55dc04741660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53f0802640 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0472c4d0 0x55dc04741660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53f0802640 1 -- 172.21.15.78:0/2069901214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc04738d80 msgr2=0x55dc04750ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53f0802640 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc04738d80 0x55dc04750ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53f0802640 1 -- 172.21.15.78:0/2069901214 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc047402c0 con 0x55dc0472cdf0 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53f1804640 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc04738d80 0x55dc04750ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53f0802640 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04750320 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f53e000bee0 tx=0x7f53e00120d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=274bacd315509999 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53d0ff9640 1 -- 172.21.15.78:0/2069901214 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00037f0 con 0x55dc0472cdf0 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53f563c740 1 -- 172.21.15.78:0/2069901214 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e0005040 con 0x55dc0472cdf0 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53d0ff9640 1 -- 172.21.15.78:0/2069901214 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f53e00051f0 con 0x55dc0472cdf0 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53d0ff9640 1 -- 172.21.15.78:0/2069901214 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f53e00053f0 con 0x55dc0472cdf0 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53f563c740 10 client.26078 fetch_fsmap learned FSMap version 42 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.551+0000 7f53f563c740 1 -- 172.21.15.78:0/2069901214 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55dc047b58f0 con 0x55dc0472cdf0 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.552+0000 7f53d0ff9640 1 -- 172.21.15.78:0/2069901214 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e0005040 con 0x55dc0472cdf0 2024-09-11T01:05:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.552+0000 7f53d0ff9640 1 -- 172.21.15.78:0/2069901214 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f53e0030040 con 0x55dc0472cdf0 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.552+0000 7f53f563c740 10 client.26078 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.552+0000 7f53f563c740 10 client.26078 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.552+0000 7f53f563c740 20 client.26078 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.552+0000 7f53f563c740 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55dc047b5e70 0x55dc047b8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.552+0000 7f53f563c740 4 client.26078 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.552+0000 7f53f563c740 1 -- 172.21.15.78:0/2069901214 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc046a08a0 con 0x55dc047b5e70 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.552+0000 7f53f1003640 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55dc047b5e70 0x55dc047b8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.552+0000 7f53d0ff9640 10 client.26078 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.552+0000 7f53f1003640 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55dc047b5e70 0x55dc047b8290 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6bb9797c0cc94896 server_cookie=e417cb739c00a5fd in_seq=0 out_seq=0 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53d0ff9640 1 -- 172.21.15.78:0/2069901214 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc046a08a0 con 0x55dc047b5e70 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53d0ff9640 10 client.26078 handle_command_reply: tid=0 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f563c740 1 client.26078 shutdown 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f563c740 1 -- 172.21.15.78:0/2069901214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 msgr2=0x55dc04750320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f563c740 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04750320 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f53e000bee0 tx=0x7f53e00120d0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f563c740 1 -- 172.21.15.78:0/2069901214 shutdown_connections 2024-09-11T01:05:51.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f563c740 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55dc047b5e70 0x55dc047b8290 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f563c740 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc04738d80 0x55dc04750ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f563c740 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc0472cdf0 0x55dc04750320 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53d0ff9640 0 client.26078 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:05:51.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f563c740 1 --2- 172.21.15.78:0/2069901214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0472c4d0 0x55dc04741660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f563c740 1 -- 172.21.15.78:0/2069901214 >> 172.21.15.78:0/2069901214 conn(0x55dc0472a300 msgr2=0x55dc047370e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:51.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f563c740 1 -- 172.21.15.78:0/2069901214 shutdown_connections 2024-09-11T01:05:51.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f563c740 1 -- 172.21.15.78:0/2069901214 wait complete. 2024-09-11T01:05:51.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f563c740 20 client.26078 trim_cache size 0 max 16384 2024-09-11T01:05:51.565 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:05:51.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f3267640 1 -- 172.21.15.78:0/3196679561 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f53d40b54d0 msgr2=0x7f53d40b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:51.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f3267640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f53d40b54d0 0x7f53d40b7990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55dc0472b1e0 tx=0x7f53dc009360 comp rx=0 tx=0).stop 2024-09-11T01:05:51.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f3267640 1 -- 172.21.15.78:0/3196679561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53ec109130 msgr2=0x7f53ec1a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:51.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.553+0000 7f53f3267640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53ec109130 0x7f53ec1a0d40 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f53e400e970 tx=0x7f53e400ed70 comp rx=0 tx=0).stop 2024-09-11T01:05:51.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.554+0000 7f53f3267640 1 -- 172.21.15.78:0/3196679561 shutdown_connections 2024-09-11T01:05:51.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.554+0000 7f53f3267640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f53d40b54d0 0x7f53d40b7990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.554+0000 7f53f3267640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53ec109130 0x7f53ec1a0d40 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.554+0000 7f53f3267640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53ec1087b0 0x7f53ec19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.554+0000 7f53f3267640 1 --2- 172.21.15.78:0/3196679561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53ec102da0 0x7f53ec19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:51.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.554+0000 7f53f3267640 1 -- 172.21.15.78:0/3196679561 >> 172.21.15.78:0/3196679561 conn(0x7f53ec0fe810 msgr2=0x7f53ec0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:51.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.554+0000 7f53f3267640 1 -- 172.21.15.78:0/3196679561 shutdown_connections 2024-09-11T01:05:51.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:51.554+0000 7f53f3267640 1 -- 172.21.15.78:0/3196679561 wait complete. 2024-09-11T01:05:52.275 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:05:52.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 499799 5.51 MB/sec execute 240 sec latency 3117.529 ms 2024-09-11T01:05:52.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:52 smithi078 ceph-mon[31315]: pgmap v533: 97 pgs: 97 active+clean; 330 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-09-11T01:05:52.545 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:52.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:52 smithi110 ceph-mon[34712]: pgmap v533: 97 pgs: 97 active+clean; 330 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-09-11T01:05:52.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:52 smithi112 ceph-mon[35384]: pgmap v533: 97 pgs: 97 active+clean; 330 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-09-11T01:05:53.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.054+0000 7fe6a42a1640 1 -- 172.21.15.78:0/4076624351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c1076a0 msgr2=0x7fe69c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.054+0000 7fe6a42a1640 1 --2- 172.21.15.78:0/4076624351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c1076a0 0x7fe69c107ae0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009900 tx=0x7fe68c02a410 comp rx=0 tx=0).stop 2024-09-11T01:05:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.054+0000 7fe6a42a1640 1 -- 172.21.15.78:0/4076624351 shutdown_connections 2024-09-11T01:05:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.054+0000 7fe6a42a1640 1 --2- 172.21.15.78:0/4076624351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c108020 0x7fe69c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.054+0000 7fe6a42a1640 1 --2- 172.21.15.78:0/4076624351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c1076a0 0x7fe69c107ae0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.054+0000 7fe6a42a1640 1 --2- 172.21.15.78:0/4076624351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe69c101c90 0x7fe69c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.054+0000 7fe6a42a1640 1 -- 172.21.15.78:0/4076624351 >> 172.21.15.78:0/4076624351 conn(0x7fe69c0fd700 msgr2=0x7fe69c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.054+0000 7fe6a42a1640 1 -- 172.21.15.78:0/4076624351 shutdown_connections 2024-09-11T01:05:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.054+0000 7fe6a42a1640 1 -- 172.21.15.78:0/4076624351 wait complete. 2024-09-11T01:05:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.055+0000 7fe6a42a1640 1 Processor -- start 2024-09-11T01:05:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.055+0000 7fe6a42a1640 1 -- start start 2024-09-11T01:05:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.055+0000 7fe6a42a1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c101c90 0x7fe69c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.055+0000 7fe6a42a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c1076a0 0x7fe69c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.055+0000 7fe6a42a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe69c108020 0x7fe69c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.055+0000 7fe6a42a1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c111ce0 con 0x7fe69c1076a0 2024-09-11T01:05:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.055+0000 7fe6a42a1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c111b60 con 0x7fe69c101c90 2024-09-11T01:05:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.055+0000 7fe6a42a1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c111e60 con 0x7fe69c108020 2024-09-11T01:05:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.055+0000 7fe6a183c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c1076a0 0x7fe69c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.055+0000 7fe6a183c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c1076a0 0x7fe69c1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59844/0 (socket says 172.21.15.78:59844) 2024-09-11T01:05:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.055+0000 7fe6a183c640 1 -- 172.21.15.78:0/1919452426 learned_addr learned my addr 172.21.15.78:0/1919452426 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.055+0000 7fe6a203d640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c101c90 0x7fe69c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6a283e640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe69c108020 0x7fe69c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6a183c640 1 -- 172.21.15.78:0/1919452426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe69c108020 msgr2=0x7fe69c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6a183c640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe69c108020 0x7fe69c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6a183c640 1 -- 172.21.15.78:0/1919452426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c101c90 msgr2=0x7fe69c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6a183c640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c101c90 0x7fe69c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6a183c640 1 -- 172.21.15.78:0/1919452426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c19f320 con 0x7fe69c1076a0 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6a283e640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe69c108020 0x7fe69c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6a203d640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c101c90 0x7fe69c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6a183c640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c1076a0 0x7fe69c1997b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe68c005940 tx=0x7fe68c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6937fe640 1 -- 172.21.15.78:0/1919452426 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c03f030 con 0x7fe69c1076a0 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6a42a1640 1 -- 172.21.15.78:0/1919452426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69c19f5b0 con 0x7fe69c1076a0 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6a42a1640 1 -- 172.21.15.78:0/1919452426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69c1a5cd0 con 0x7fe69c1076a0 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.056+0000 7fe6937fe640 1 -- 172.21.15.78:0/1919452426 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe68c0044c0 con 0x7fe69c1076a0 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.057+0000 7fe6937fe640 1 -- 172.21.15.78:0/1919452426 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c049060 con 0x7fe69c1076a0 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.059+0000 7fe6937fe640 1 -- 172.21.15.78:0/1919452426 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe68c04c060 con 0x7fe69c1076a0 2024-09-11T01:05:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.059+0000 7fe6a6676740 1 Processor -- start 2024-09-11T01:05:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a6676740 1 -- start start 2024-09-11T01:05:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a6676740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f044d0 0x55bf34f048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a6676740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f04df0 0x55bf34f14c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a6676740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f15190 0x55bf34f17580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a6676740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf34e7aea0 con 0x55bf34f044d0 2024-09-11T01:05:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a6676740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55bf34e6a2d0 con 0x55bf34f15190 2024-09-11T01:05:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a6676740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bf34e6e0e0 con 0x55bf34f04df0 2024-09-11T01:05:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a283e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f044d0 0x55bf34f048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a203d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f04df0 0x55bf34f14c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a283e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f044d0 0x55bf34f048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59858/0 (socket says 172.21.15.78:59858) 2024-09-11T01:05:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a203d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f04df0 0x55bf34f14c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:36068/0 (socket says 172.21.15.78:36068) 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a283e640 1 -- 172.21.15.78:0/2663426235 learned_addr learned my addr 172.21.15.78:0/2663426235 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a183c640 1 --2- 172.21.15.78:0/2663426235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f15190 0x55bf34f17580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a203d640 1 -- 172.21.15.78:0/2663426235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f15190 msgr2=0x55bf34f17580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a203d640 1 --2- 172.21.15.78:0/2663426235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f15190 0x55bf34f17580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a203d640 1 -- 172.21.15.78:0/2663426235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f044d0 msgr2=0x55bf34f048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6937fe640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe6880b5450 0x7fe6880b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a203d640 1 --2- 172.21.15.78:0/2663426235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f044d0 0x55bf34f048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a203d640 1 -- 172.21.15.78:0/2663426235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf34bc8650 con 0x55bf34f04df0 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.060+0000 7fe6a283e640 1 --2- 172.21.15.78:0/2663426235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f044d0 0x55bf34f048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.061+0000 7fe6a283e640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe6880b5450 0x7fe6880b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.061+0000 7fe6937fe640 1 -- 172.21.15.78:0/1919452426 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe68c105da0 con 0x7fe69c1076a0 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.061+0000 7fe6a203d640 1 --2- 172.21.15.78:0/2663426235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f04df0 0x55bf34f14c50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe698011440 tx=0x7fe698011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14964304d7bb4537 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.061+0000 7fe690de6640 1 -- 172.21.15.78:0/2663426235 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698004030 con 0x55bf34f04df0 2024-09-11T01:05:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.061+0000 7fe690de6640 1 -- 172.21.15.78:0/2663426235 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6980054d0 con 0x55bf34f04df0 2024-09-11T01:05:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.061+0000 7fe690de6640 1 -- 172.21.15.78:0/2663426235 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698004ce0 con 0x55bf34f04df0 2024-09-11T01:05:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.062+0000 7fe6a6676740 1 -- 172.21.15.78:0/2663426235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f04df0 msgr2=0x55bf34f14c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.062+0000 7fe6a283e640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe6880b5450 0x7fe6880b7910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe694009fd0 tx=0x7fe694004a40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.062+0000 7fe6a6676740 1 --2- 172.21.15.78:0/2663426235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f04df0 0x55bf34f14c50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe698011440 tx=0x7fe698011900 comp rx=0 tx=0).stop 2024-09-11T01:05:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.062+0000 7fe6a6676740 1 -- 172.21.15.78:0/2663426235 shutdown_connections 2024-09-11T01:05:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.062+0000 7fe6a6676740 1 --2- 172.21.15.78:0/2663426235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f15190 0x55bf34f17580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.062+0000 7fe6a6676740 1 --2- 172.21.15.78:0/2663426235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f04df0 0x55bf34f14c50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.062+0000 7fe6a6676740 1 --2- 172.21.15.78:0/2663426235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f044d0 0x55bf34f048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.062+0000 7fe6a6676740 1 -- 172.21.15.78:0/2663426235 >> 172.21.15.78:0/2663426235 conn(0x55bf34f02300 msgr2=0x55bf34f034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.062+0000 7fe6a6676740 1 -- 172.21.15.78:0/2663426235 shutdown_connections 2024-09-11T01:05:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.062+0000 7fe6a6676740 1 -- 172.21.15.78:0/2663426235 wait complete. 2024-09-11T01:05:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.062+0000 7fe6a6676740 1 -- start start 2024-09-11T01:05:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.062+0000 7fe6a6676740 1 Processor -- start 2024-09-11T01:05:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a6676740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f14a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a6676740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f04df0 0x55bf34f11ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a6676740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 0x55bf34f124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a6676740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf34ccb700 con 0x55bf34f15190 2024-09-11T01:05:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a6676740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55bf34e05160 con 0x55bf34f04df0 2024-09-11T01:05:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a6676740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bf34f17e40 con 0x55bf34f044d0 2024-09-11T01:05:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a6676740 10 client.0 _ll_register_callbacks cb 0x55bf34e1e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a283e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 0x55bf34f124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a203d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f14a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a283e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 0x55bf34f124f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59864/0 (socket says 172.21.15.78:59864) 2024-09-11T01:05:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a183c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f04df0 0x55bf34f11ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a283e640 1 -- 172.21.15.78:0/4141784371 learned_addr learned my addr 172.21.15.78:0/4141784371 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a283e640 1 -- 172.21.15.78:0/4141784371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 msgr2=0x55bf34f14a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a283e640 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f14a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a283e640 1 -- 172.21.15.78:0/4141784371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f04df0 msgr2=0x55bf34f11ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a283e640 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f04df0 0x55bf34f11ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a283e640 1 -- 172.21.15.78:0/4141784371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf34e32c90 con 0x55bf34f15190 2024-09-11T01:05:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a203d640 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f14a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a183c640 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f04df0 0x55bf34f11ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a283e640 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 0x55bf34f124f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe694011560 tx=0x7fe694004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8719c8cd3f374861 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe685ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe685ffb640 1 -- 172.21.15.78:0/4141784371 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69402d060 con 0x55bf34f15190 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe6a6676740 1 -- 172.21.15.78:0/4141784371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6980054d0 con 0x55bf34f15190 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.063+0000 7fe685ffb640 1 -- 172.21.15.78:0/4141784371 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe694028600 con 0x55bf34f15190 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.064+0000 7fe685ffb640 1 -- 172.21.15.78:0/4141784371 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe694028800 con 0x55bf34f15190 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.064+0000 7fe6a6676740 10 client.16797 fetch_fsmap learned FSMap version 42 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.064+0000 7fe6a6676740 1 -- 172.21.15.78:0/4141784371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bf34f23850 con 0x55bf34f15190 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.064+0000 7fe685ffb640 1 -- 172.21.15.78:0/4141784371 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6980054d0 con 0x55bf34f15190 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.064+0000 7fe685ffb640 1 -- 172.21.15.78:0/4141784371 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7fe694030020 con 0x55bf34f15190 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.064+0000 7fe6a6676740 10 client.16797 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.064+0000 7fe6a6676740 10 client.16797 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.064+0000 7fe6a6676740 20 client.16797 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.064+0000 7fe6a6676740 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55bf34f24e00 0x55bf34f272a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.064+0000 7fe6a6676740 4 client.16797 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.064+0000 7fe6a6676740 1 -- 172.21.15.78:0/4141784371 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf34e7cdf0 con 0x55bf34f24e00 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.064+0000 7fe6a203d640 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55bf34f24e00 0x55bf34f272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.065+0000 7fe685ffb640 10 client.16797 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.065+0000 7fe6a203d640 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55bf34f24e00 0x55bf34f272a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d056fc65bc504c08 server_cookie=940d615024ac7488 in_seq=0 out_seq=0 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.065+0000 7fe685ffb640 1 -- 172.21.15.78:0/4141784371 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bf34e7cdf0 con 0x55bf34f24e00 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.065+0000 7fe685ffb640 10 client.16797 handle_command_reply: tid=0 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.065+0000 7fe6a6676740 1 client.16797 shutdown 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.065+0000 7fe6a6676740 1 -- 172.21.15.78:0/4141784371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 msgr2=0x55bf34f124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.065+0000 7fe6a6676740 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 0x55bf34f124f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe694011560 tx=0x7fe694004ab0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.066+0000 7fe6a6676740 1 -- 172.21.15.78:0/4141784371 shutdown_connections 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.066+0000 7fe6a6676740 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 0x55bf34f124f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.066+0000 7fe6a6676740 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55bf34f24e00 0x55bf34f272a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.066+0000 7fe6a6676740 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f04df0 0x55bf34f11ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.066+0000 7fe6a6676740 1 --2- 172.21.15.78:0/4141784371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f14a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.066+0000 7fe6a6676740 1 -- 172.21.15.78:0/4141784371 >> 172.21.15.78:0/4141784371 conn(0x55bf34f02300 msgr2=0x55bf34f07ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.066+0000 7fe685ffb640 0 client.16797 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.066+0000 7fe6a6676740 1 -- 172.21.15.78:0/4141784371 shutdown_connections 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.066+0000 7fe6a6676740 1 -- 172.21.15.78:0/4141784371 wait complete. 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.066+0000 7fe6a6676740 20 client.16797 trim_cache size 0 max 16384 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a6676740 1 Processor -- start 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a6676740 1 -- start start 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a6676740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f21100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a6676740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f04df0 0x55bf34f2cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a6676740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 0x55bf34f2d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a6676740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe68000e620 con 0x55bf34f15190 2024-09-11T01:05:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a6676740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe680010280 con 0x55bf34f04df0 2024-09-11T01:05:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a6676740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe68000ecf0 con 0x55bf34f044d0 2024-09-11T01:05:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a283e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 0x55bf34f2d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a283e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 0x55bf34f2d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59868/0 (socket says 172.21.15.78:59868) 2024-09-11T01:05:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a283e640 1 -- 172.21.15.78:0/1766457298 learned_addr learned my addr 172.21.15.78:0/1766457298 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a183c640 1 --2- 172.21.15.78:0/1766457298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f04df0 0x55bf34f2cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a203d640 1 --2- 172.21.15.78:0/1766457298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f21100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a283e640 1 -- 172.21.15.78:0/1766457298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 msgr2=0x55bf34f21100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a283e640 1 --2- 172.21.15.78:0/1766457298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f21100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a283e640 1 -- 172.21.15.78:0/1766457298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f04df0 msgr2=0x55bf34f2cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a283e640 1 --2- 172.21.15.78:0/1766457298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f04df0 0x55bf34f2cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a283e640 1 -- 172.21.15.78:0/1766457298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf34f02130 con 0x55bf34f15190 2024-09-11T01:05:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.080+0000 7fe6a283e640 1 --2- 172.21.15.78:0/1766457298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 0x55bf34f2d0b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe694006010 tx=0x7fe69401b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcde24f3452217cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.081+0000 7fe687fff640 1 -- 172.21.15.78:0/1766457298 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694028400 con 0x55bf34f15190 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.081+0000 7fe687fff640 1 -- 172.21.15.78:0/1766457298 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6940285a0 con 0x55bf34f15190 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.081+0000 7fe687fff640 1 -- 172.21.15.78:0/1766457298 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69400f690 con 0x55bf34f15190 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.081+0000 7fe6a6676740 1 -- 172.21.15.78:0/1766457298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 msgr2=0x55bf34f2d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.081+0000 7fe6a6676740 1 --2- 172.21.15.78:0/1766457298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 0x55bf34f2d0b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe694006010 tx=0x7fe69401b430 comp rx=0 tx=0).stop 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.081+0000 7fe6a6676740 1 -- 172.21.15.78:0/1766457298 shutdown_connections 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.081+0000 7fe6a6676740 1 --2- 172.21.15.78:0/1766457298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f15190 0x55bf34f2d0b0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.081+0000 7fe6a6676740 1 --2- 172.21.15.78:0/1766457298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f04df0 0x55bf34f2cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.081+0000 7fe6a6676740 1 --2- 172.21.15.78:0/1766457298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f21100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.081+0000 7fe6a6676740 1 -- 172.21.15.78:0/1766457298 >> 172.21.15.78:0/1766457298 conn(0x55bf34f02300 msgr2=0x55bf34f02fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.081+0000 7fe6a6676740 1 -- 172.21.15.78:0/1766457298 shutdown_connections 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.081+0000 7fe6a6676740 1 -- 172.21.15.78:0/1766457298 wait complete. 2024-09-11T01:05:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a6676740 1 -- start start 2024-09-11T01:05:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a6676740 1 Processor -- start 2024-09-11T01:05:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a6676740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f1d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a6676740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f04df0 0x55bf34f1d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a6676740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f15190 0x55bf34f1a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a6676740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe68000faf0 con 0x55bf34f04df0 2024-09-11T01:05:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a6676740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe68000f420 con 0x55bf34f15190 2024-09-11T01:05:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a6676740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe68000c600 con 0x55bf34f044d0 2024-09-11T01:05:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a6676740 10 client.0 _ll_register_callbacks cb 0x7fe6940112e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a283e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f15190 0x55bf34f1a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a203d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f1d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a183c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f04df0 0x55bf34f1d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a283e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f15190 0x55bf34f1a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:44448/0 (socket says 172.21.15.78:44448) 2024-09-11T01:05:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a203d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f1d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:36092/0 (socket says 172.21.15.78:36092) 2024-09-11T01:05:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.082+0000 7fe6a283e640 1 -- 172.21.15.78:0/618147120 learned_addr learned my addr 172.21.15.78:0/618147120 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe6a183c640 1 -- 172.21.15.78:0/618147120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 msgr2=0x55bf34f1d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe6a183c640 1 --2- 172.21.15.78:0/618147120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f1d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe6a183c640 1 -- 172.21.15.78:0/618147120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f15190 msgr2=0x55bf34f1a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe6a183c640 1 --2- 172.21.15.78:0/618147120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f15190 0x55bf34f1a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe6a183c640 1 -- 172.21.15.78:0/618147120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf34f1a670 con 0x55bf34f04df0 2024-09-11T01:05:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe6a183c640 1 --2- 172.21.15.78:0/618147120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f04df0 0x55bf34f1d6a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe68c103500 tx=0x7fe68c103ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7801c06f985cb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe685ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe685ffb640 1 -- 172.21.15.78:0/618147120 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c035040 con 0x55bf34f04df0 2024-09-11T01:05:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe6a6676740 1 -- 172.21.15.78:0/618147120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6940285a0 con 0x55bf34f04df0 2024-09-11T01:05:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe685ffb640 1 -- 172.21.15.78:0/618147120 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe68c0cc740 con 0x55bf34f04df0 2024-09-11T01:05:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe685ffb640 1 -- 172.21.15.78:0/618147120 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe68c0cc0b0 con 0x55bf34f04df0 2024-09-11T01:05:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe6a6676740 10 client.16803 fetch_fsmap learned FSMap version 42 2024-09-11T01:05:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe6a6676740 1 -- 172.21.15.78:0/618147120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bf34f92520 con 0x55bf34f04df0 2024-09-11T01:05:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.083+0000 7fe685ffb640 1 -- 172.21.15.78:0/618147120 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6940285a0 con 0x55bf34f04df0 2024-09-11T01:05:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe685ffb640 1 -- 172.21.15.78:0/618147120 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7fe68c0d5330 con 0x55bf34f04df0 2024-09-11T01:05:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe6a6676740 10 client.16803 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T01:05:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe6a6676740 10 client.16803 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:05:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe6a6676740 20 client.16803 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe6a6676740 1 --2- 172.21.15.78:0/618147120 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55bf34f92cb0 0x55bf34f950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe6a6676740 4 client.16803 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe6a6676740 1 -- 172.21.15.78:0/618147120 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf34e788a0 con 0x55bf34f92cb0 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe6a203d640 1 --2- 172.21.15.78:0/618147120 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55bf34f92cb0 0x55bf34f950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe685ffb640 10 client.16803 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe6a203d640 1 --2- 172.21.15.78:0/618147120 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55bf34f92cb0 0x55bf34f950d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=acb961034a3d523a server_cookie=4f4a95d2963b5e47 in_seq=0 out_seq=0 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe685ffb640 1 -- 172.21.15.78:0/618147120 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf34e788a0 con 0x55bf34f92cb0 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe685ffb640 10 client.16803 handle_command_reply: tid=0 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.084+0000 7fe6a6676740 1 client.16803 shutdown 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a6676740 1 -- 172.21.15.78:0/618147120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f04df0 msgr2=0x55bf34f1d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a6676740 1 --2- 172.21.15.78:0/618147120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f04df0 0x55bf34f1d6a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe68c103500 tx=0x7fe68c103ac0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a6676740 1 -- 172.21.15.78:0/618147120 shutdown_connections 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a6676740 1 --2- 172.21.15.78:0/618147120 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55bf34f92cb0 0x55bf34f950d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a6676740 1 --2- 172.21.15.78:0/618147120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bf34f15190 0x55bf34f1a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a6676740 1 --2- 172.21.15.78:0/618147120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf34f04df0 0x55bf34f1d6a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a6676740 1 --2- 172.21.15.78:0/618147120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf34f044d0 0x55bf34f1d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a6676740 1 -- 172.21.15.78:0/618147120 >> 172.21.15.78:0/618147120 conn(0x55bf34f02300 msgr2=0x55bf34f134f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:53.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe685ffb640 0 client.16803 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:05:53.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a6676740 1 -- 172.21.15.78:0/618147120 shutdown_connections 2024-09-11T01:05:53.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a6676740 1 -- 172.21.15.78:0/618147120 wait complete. 2024-09-11T01:05:53.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a6676740 20 client.16803 trim_cache size 0 max 16384 2024-09-11T01:05:53.106 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:05:53.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a42a1640 1 -- 172.21.15.78:0/1919452426 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe6880b5450 msgr2=0x7fe6880b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:53.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a42a1640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe6880b5450 0x7fe6880b7910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe694009fd0 tx=0x7fe694004a40 comp rx=0 tx=0).stop 2024-09-11T01:05:53.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a42a1640 1 -- 172.21.15.78:0/1919452426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c1076a0 msgr2=0x7fe69c1997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:53.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a42a1640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c1076a0 0x7fe69c1997b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe68c005940 tx=0x7fe68c0040c0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a42a1640 1 -- 172.21.15.78:0/1919452426 shutdown_connections 2024-09-11T01:05:53.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a42a1640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe6880b5450 0x7fe6880b7910 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a42a1640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe69c108020 0x7fe69c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a42a1640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c1076a0 0x7fe69c1997b0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a42a1640 1 --2- 172.21.15.78:0/1919452426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c101c90 0x7fe69c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:53.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a42a1640 1 -- 172.21.15.78:0/1919452426 >> 172.21.15.78:0/1919452426 conn(0x7fe69c0fd700 msgr2=0x7fe69c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:53.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a42a1640 1 -- 172.21.15.78:0/1919452426 shutdown_connections 2024-09-11T01:05:53.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:53.085+0000 7fe6a42a1640 1 -- 172.21.15.78:0/1919452426 wait complete. 2024-09-11T01:05:53.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 501451 5.54 MB/sec execute 241 sec latency 30.436 ms 2024-09-11T01:05:53.768 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:05:54.030 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:54.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 502645 5.54 MB/sec execute 242 sec latency 322.515 ms 2024-09-11T01:05:54.485 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:54 smithi078 ceph-mon[31315]: pgmap v534: 97 pgs: 97 active+clean; 331 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-09-11T01:05:54.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:54 smithi110 ceph-mon[34712]: pgmap v534: 97 pgs: 97 active+clean; 331 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-09-11T01:05:54.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.702+0000 7f508f971640 1 -- 172.21.15.78:0/1739566193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50881090c0 msgr2=0x7f508810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:54.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.702+0000 7f508f971640 1 --2- 172.21.15.78:0/1739566193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50881090c0 0x7f508810f950 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f5080009be0 tx=0x7f508002a6c0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.702+0000 7f508f971640 1 -- 172.21.15.78:0/1739566193 shutdown_connections 2024-09-11T01:05:54.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.702+0000 7f508f971640 1 --2- 172.21.15.78:0/1739566193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50881090c0 0x7f508810f950 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.702+0000 7f508f971640 1 --2- 172.21.15.78:0/1739566193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5088108740 0x7f5088108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.702+0000 7f508f971640 1 --2- 172.21.15.78:0/1739566193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5088101260 0x7f5088101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.702+0000 7f508f971640 1 -- 172.21.15.78:0/1739566193 >> 172.21.15.78:0/1739566193 conn(0x7f5088078060 msgr2=0x7f50880ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:54.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.702+0000 7f508f971640 1 -- 172.21.15.78:0/1739566193 shutdown_connections 2024-09-11T01:05:54.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.702+0000 7f508f971640 1 -- 172.21.15.78:0/1739566193 wait complete. 2024-09-11T01:05:54.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508f971640 1 Processor -- start 2024-09-11T01:05:54.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508f971640 1 -- start start 2024-09-11T01:05:54.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508f971640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5088101260 0x7f5088102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508f971640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5088108740 0x7f5088102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508f971640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50881090c0 0x7f5088102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508f971640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5088112e10 con 0x7f5088108740 2024-09-11T01:05:54.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508f971640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5088112c90 con 0x7f50881090c0 2024-09-11T01:05:54.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508f971640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5088112f90 con 0x7f5088101260 2024-09-11T01:05:54.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508cf0c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5088108740 0x7f5088102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508d70d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5088101260 0x7f5088102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508cf0c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5088108740 0x7f5088102770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48832/0 (socket says 172.21.15.78:48832) 2024-09-11T01:05:54.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508cf0c640 1 -- 172.21.15.78:0/3744376023 learned_addr learned my addr 172.21.15.78:0/3744376023 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:54.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.703+0000 7f508df0e640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50881090c0 0x7f5088102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.704+0000 7f508cf0c640 1 -- 172.21.15.78:0/3744376023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5088101260 msgr2=0x7f5088102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.704+0000 7f508cf0c640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5088101260 0x7f5088102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.704+0000 7f508cf0c640 1 -- 172.21.15.78:0/3744376023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50881090c0 msgr2=0x7f5088102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.704+0000 7f508cf0c640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50881090c0 0x7f5088102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.704+0000 7f508cf0c640 1 -- 172.21.15.78:0/3744376023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50881a75e0 con 0x7f5088108740 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.704+0000 7f508df0e640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50881090c0 0x7f5088102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.704+0000 7f508cf0c640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5088108740 0x7f5088102770 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f507800e9f0 tx=0x7f507800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.704+0000 7f507e7fc640 1 -- 172.21.15.78:0/3744376023 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507800ccc0 con 0x7f5088108740 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.704+0000 7f507e7fc640 1 -- 172.21.15.78:0/3744376023 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50780108e0 con 0x7f5088108740 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.704+0000 7f507e7fc640 1 -- 172.21.15.78:0/3744376023 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078018880 con 0x7f5088108740 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.704+0000 7f508f971640 1 -- 172.21.15.78:0/3744376023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50881a78d0 con 0x7f5088108740 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.704+0000 7f508f971640 1 -- 172.21.15.78:0/3744376023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50881a7d90 con 0x7f5088108740 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f5091d46740 1 Processor -- start 2024-09-11T01:05:54.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f5091d46740 1 -- start start 2024-09-11T01:05:54.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f507e7fc640 1 -- 172.21.15.78:0/3744376023 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5078010430 con 0x7f5088108740 2024-09-11T01:05:54.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f5091d46740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 0x5628806458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f5091d46740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562880645df0 0x562880651840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f5091d46740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x562880654170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f5091d46740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5628805bbea0 con 0x5628806454d0 2024-09-11T01:05:54.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f5091d46740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5628805ab2d0 con 0x562880645df0 2024-09-11T01:05:54.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f5091d46740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5628805af0e0 con 0x562880651d80 2024-09-11T01:05:54.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f508df0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 0x5628806458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f508d70d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562880645df0 0x562880651840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f508df0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 0x5628806458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48842/0 (socket says 172.21.15.78:48842) 2024-09-11T01:05:54.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f508df0e640 1 -- 172.21.15.78:0/2005241489 learned_addr learned my addr 172.21.15.78:0/2005241489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:54.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.707+0000 7f508cf0c640 1 --2- 172.21.15.78:0/2005241489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x562880654170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.708+0000 7f508d70d640 1 -- 172.21.15.78:0/2005241489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 msgr2=0x562880654170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:54.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.708+0000 7f508d70d640 1 --2- 172.21.15.78:0/2005241489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x562880654170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.708+0000 7f508d70d640 1 -- 172.21.15.78:0/2005241489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 msgr2=0x5628806458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:54.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.708+0000 7f508d70d640 1 --2- 172.21.15.78:0/2005241489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 0x5628806458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.708+0000 7f508d70d640 1 -- 172.21.15.78:0/2005241489 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562880309650 con 0x562880645df0 2024-09-11T01:05:54.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.708+0000 7f508d70d640 1 --2- 172.21.15.78:0/2005241489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562880645df0 0x562880651840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5084009c30 tx=0x7f5084011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14c749cbfc5f0fef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:54.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.708+0000 7f507e7fc640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50740b5450 0x7f50740b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.708+0000 7f50737fe640 1 -- 172.21.15.78:0/2005241489 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508401e070 con 0x562880645df0 2024-09-11T01:05:54.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.708+0000 7f508df0e640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50740b5450 0x7f50740b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.708+0000 7f50737fe640 1 -- 172.21.15.78:0/2005241489 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50840035f0 con 0x562880645df0 2024-09-11T01:05:54.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.708+0000 7f50737fe640 1 -- 172.21.15.78:0/2005241489 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084019690 con 0x562880645df0 2024-09-11T01:05:54.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.708+0000 7f507e7fc640 1 -- 172.21.15.78:0/3744376023 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f50780e6350 con 0x7f5088108740 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.709+0000 7f5091d46740 1 -- 172.21.15.78:0/2005241489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562880645df0 msgr2=0x562880651840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.709+0000 7f5091d46740 1 --2- 172.21.15.78:0/2005241489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562880645df0 0x562880651840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5084009c30 tx=0x7f5084011810 comp rx=0 tx=0).stop 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.709+0000 7f508df0e640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50740b5450 0x7f50740b7910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5080006010 tx=0x7f5080004a40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.709+0000 7f5091d46740 1 -- 172.21.15.78:0/2005241489 shutdown_connections 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.709+0000 7f5091d46740 1 --2- 172.21.15.78:0/2005241489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x562880654170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.709+0000 7f5091d46740 1 --2- 172.21.15.78:0/2005241489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562880645df0 0x562880651840 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.709+0000 7f5091d46740 1 --2- 172.21.15.78:0/2005241489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 0x5628806458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.709+0000 7f5091d46740 1 -- 172.21.15.78:0/2005241489 >> 172.21.15.78:0/2005241489 conn(0x562880643300 msgr2=0x5628806444a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.709+0000 7f5091d46740 1 -- 172.21.15.78:0/2005241489 shutdown_connections 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.709+0000 7f5091d46740 1 -- 172.21.15.78:0/2005241489 wait complete. 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.709+0000 7f5091d46740 1 -- start start 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.709+0000 7f5091d46740 1 Processor -- start 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f5091d46740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 0x562880651600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f5091d46740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562880645df0 0x56288064ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f5091d46740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x56288064eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f5091d46740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56288040c700 con 0x5628806454d0 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f5091d46740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562880546160 con 0x562880645df0 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f5091d46740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562880654a00 con 0x562880651d80 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f5091d46740 10 client.0 _ll_register_callbacks cb 0x56288055f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f508d70d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 0x562880651600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f508cf0c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562880645df0 0x56288064ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f508d70d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 0x562880651600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48844/0 (socket says 172.21.15.78:48844) 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f508d70d640 1 -- 172.21.15.78:0/960539705 learned_addr learned my addr 172.21.15.78:0/960539705 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f508df0e640 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x56288064eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f508d70d640 1 -- 172.21.15.78:0/960539705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 msgr2=0x56288064eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f508d70d640 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x56288064eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f508d70d640 1 -- 172.21.15.78:0/960539705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562880645df0 msgr2=0x56288064ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f508d70d640 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562880645df0 0x56288064ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f508d70d640 1 -- 172.21.15.78:0/960539705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562880573c90 con 0x5628806454d0 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f508cf0c640 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562880645df0 0x56288064ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.710+0000 7f508df0e640 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x56288064eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f508d70d640 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 0x562880651600 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f508401c040 tx=0x7f5084003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de181c36896dce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5070ff9640 1 -- 172.21.15.78:0/960539705 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084028020 con 0x5628806454d0 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5070ff9640 1 -- 172.21.15.78:0/960539705 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5084004250 con 0x5628806454d0 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5091d46740 1 -- 172.21.15.78:0/960539705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50840035f0 con 0x5628806454d0 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5070ff9640 1 -- 172.21.15.78:0/960539705 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5084033050 con 0x5628806454d0 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5091d46740 10 client.16815 fetch_fsmap learned FSMap version 42 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5091d46740 1 -- 172.21.15.78:0/960539705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562880660330 con 0x5628806454d0 2024-09-11T01:05:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5070ff9640 1 -- 172.21.15.78:0/960539705 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50840035f0 con 0x5628806454d0 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5070ff9640 1 -- 172.21.15.78:0/960539705 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f5084021030 con 0x5628806454d0 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5091d46740 10 client.16815 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5091d46740 10 client.16815 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5091d46740 20 client.16815 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5091d46740 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x562880661730 0x562880663b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5091d46740 4 client.16815 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.711+0000 7f5091d46740 1 -- 172.21.15.78:0/960539705 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628805bddf0 con 0x562880661730 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.712+0000 7f508cf0c640 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x562880661730 0x562880663b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.712+0000 7f5070ff9640 10 client.16815 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.712+0000 7f508cf0c640 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x562880661730 0x562880663b20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d0ba1fdb94364c server_cookie=807be6540aafdf9e in_seq=0 out_seq=0 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5070ff9640 1 -- 172.21.15.78:0/960539705 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5628805bddf0 con 0x562880661730 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5070ff9640 10 client.16815 handle_command_reply: tid=0 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5091d46740 1 client.16815 shutdown 2024-09-11T01:05:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5091d46740 1 -- 172.21.15.78:0/960539705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 msgr2=0x562880651600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:54.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5091d46740 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 0x562880651600 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f508401c040 tx=0x7f5084003c60 comp rx=0 tx=0).stop 2024-09-11T01:05:54.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5091d46740 1 -- 172.21.15.78:0/960539705 shutdown_connections 2024-09-11T01:05:54.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5091d46740 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x56288064eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5091d46740 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562880645df0 0x56288064ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5091d46740 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x562880661730 0x562880663b20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5091d46740 1 --2- 172.21.15.78:0/960539705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628806454d0 0x562880651600 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5070ff9640 0 client.16815 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:05:54.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5091d46740 1 -- 172.21.15.78:0/960539705 >> 172.21.15.78:0/960539705 conn(0x562880643300 msgr2=0x562880648e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:54.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5091d46740 1 -- 172.21.15.78:0/960539705 shutdown_connections 2024-09-11T01:05:54.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5091d46740 1 -- 172.21.15.78:0/960539705 wait complete. 2024-09-11T01:05:54.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.713+0000 7f5091d46740 20 client.16815 trim_cache size 0 max 16384 2024-09-11T01:05:54.731 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:54 smithi112 ceph-mon[35384]: pgmap v534: 97 pgs: 97 active+clean; 331 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.727+0000 7f5091d46740 1 Processor -- start 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.727+0000 7f5091d46740 1 -- start start 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.727+0000 7f5091d46740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5628806454d0 0x562880642c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.727+0000 7f5091d46740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880645df0 0x56288064d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.727+0000 7f5091d46740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x56288064df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.727+0000 7f5091d46740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50881b0740 con 0x562880645df0 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.727+0000 7f5091d46740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50881b23a0 con 0x5628806454d0 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.727+0000 7f5091d46740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50881b0e10 con 0x562880651d80 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f508cf0c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880645df0 0x56288064d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f508cf0c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880645df0 0x56288064d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48846/0 (socket says 172.21.15.78:48846) 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f508cf0c640 1 -- 172.21.15.78:0/2992977601 learned_addr learned my addr 172.21.15.78:0/2992977601 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f508df0e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x56288064df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f508d70d640 1 --2- 172.21.15.78:0/2992977601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5628806454d0 0x562880642c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f508cf0c640 1 -- 172.21.15.78:0/2992977601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 msgr2=0x56288064df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f508cf0c640 1 --2- 172.21.15.78:0/2992977601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x56288064df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f508cf0c640 1 -- 172.21.15.78:0/2992977601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5628806454d0 msgr2=0x562880642c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f508cf0c640 1 --2- 172.21.15.78:0/2992977601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5628806454d0 0x562880642c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f508cf0c640 1 -- 172.21.15.78:0/2992977601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56288064f360 con 0x562880645df0 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f508cf0c640 1 --2- 172.21.15.78:0/2992977601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880645df0 0x56288064d9e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f50780e3bc0 tx=0x7f50780e8860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403eca8eb3cec130 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f5072ffd640 1 -- 172.21.15.78:0/2992977601 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50780a86e0 con 0x562880645df0 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f5072ffd640 1 -- 172.21.15.78:0/2992977601 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50780ac350 con 0x562880645df0 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.728+0000 7f5072ffd640 1 -- 172.21.15.78:0/2992977601 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50780abfb0 con 0x562880645df0 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.729+0000 7f5091d46740 1 -- 172.21.15.78:0/2992977601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880645df0 msgr2=0x56288064d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.729+0000 7f5091d46740 1 --2- 172.21.15.78:0/2992977601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880645df0 0x56288064d9e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f50780e3bc0 tx=0x7f50780e8860 comp rx=0 tx=0).stop 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.729+0000 7f5091d46740 1 -- 172.21.15.78:0/2992977601 shutdown_connections 2024-09-11T01:05:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.729+0000 7f5091d46740 1 --2- 172.21.15.78:0/2992977601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880651d80 0x56288064df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.729+0000 7f5091d46740 1 --2- 172.21.15.78:0/2992977601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880645df0 0x56288064d9e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.729+0000 7f5091d46740 1 --2- 172.21.15.78:0/2992977601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5628806454d0 0x562880642c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.729+0000 7f5091d46740 1 -- 172.21.15.78:0/2992977601 >> 172.21.15.78:0/2992977601 conn(0x562880643300 msgr2=0x56288064d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:54.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.729+0000 7f5091d46740 1 -- 172.21.15.78:0/2992977601 shutdown_connections 2024-09-11T01:05:54.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.729+0000 7f5091d46740 1 -- 172.21.15.78:0/2992977601 wait complete. 2024-09-11T01:05:54.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f5091d46740 1 -- start start 2024-09-11T01:05:54.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f5091d46740 1 Processor -- start 2024-09-11T01:05:54.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f5091d46740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5628806454d0 0x56288065a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f5091d46740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880645df0 0x56288064b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f5091d46740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880651d80 0x56288064be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f5091d46740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50881b1c10 con 0x562880651d80 2024-09-11T01:05:54.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f5091d46740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50881b1540 con 0x5628806454d0 2024-09-11T01:05:54.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f5091d46740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50881ae720 con 0x562880645df0 2024-09-11T01:05:54.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f5091d46740 10 client.0 _ll_register_callbacks cb 0x7f508401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:54.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f508df0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880651d80 0x56288064be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f508d70d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5628806454d0 0x56288065a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f508d70d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5628806454d0 0x56288065a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59980/0 (socket says 172.21.15.78:59980) 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.730+0000 7f508d70d640 1 -- 172.21.15.78:0/1033540997 learned_addr learned my addr 172.21.15.78:0/1033540997 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f508cf0c640 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880645df0 0x56288064b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f508df0e640 1 -- 172.21.15.78:0/1033540997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880645df0 msgr2=0x56288064b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f508df0e640 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880645df0 0x56288064b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f508df0e640 1 -- 172.21.15.78:0/1033540997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5628806454d0 msgr2=0x56288065a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f508df0e640 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5628806454d0 0x56288065a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f508df0e640 1 -- 172.21.15.78:0/1033540997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56288064c630 con 0x562880651d80 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f508d70d640 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5628806454d0 0x56288065a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f508df0e640 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880651d80 0x56288064be30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5080034040 tx=0x7f5080009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569495af0f90c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f5070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f5070ff9640 1 -- 172.21.15.78:0/1033540997 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080036040 con 0x562880651d80 2024-09-11T01:05:54.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f5070ff9640 1 -- 172.21.15.78:0/1033540997 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5080031d60 con 0x562880651d80 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f5091d46740 1 -- 172.21.15.78:0/1033540997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50780ac350 con 0x562880651d80 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f5070ff9640 1 -- 172.21.15.78:0/1033540997 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f508003e440 con 0x562880651d80 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f5091d46740 10 client.16821 fetch_fsmap learned FSMap version 42 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.731+0000 7f5091d46740 1 -- 172.21.15.78:0/1033540997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5628806ce4c0 con 0x562880651d80 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.732+0000 7f5070ff9640 1 -- 172.21.15.78:0/1033540997 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50780ac350 con 0x562880651d80 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.732+0000 7f5070ff9640 1 -- 172.21.15.78:0/1033540997 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f508004c660 con 0x562880651d80 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.732+0000 7f5091d46740 10 client.16821 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.732+0000 7f5091d46740 10 client.16821 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.732+0000 7f5091d46740 20 client.16821 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.732+0000 7f5091d46740 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5628806cec90 0x5628806d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.732+0000 7f5091d46740 4 client.16821 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.732+0000 7f5091d46740 1 -- 172.21.15.78:0/1033540997 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628805b98a0 con 0x5628806cec90 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.732+0000 7f508d70d640 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5628806cec90 0x5628806d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:54.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.732+0000 7f5070ff9640 10 client.16821 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.732+0000 7f508d70d640 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5628806cec90 0x5628806d10b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d751d4b19d81bd29 server_cookie=816ad58d142b320d in_seq=0 out_seq=0 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5070ff9640 1 -- 172.21.15.78:0/1033540997 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628805b98a0 con 0x5628806cec90 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5070ff9640 10 client.16821 handle_command_reply: tid=0 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5091d46740 1 client.16821 shutdown 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5091d46740 1 -- 172.21.15.78:0/1033540997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880651d80 msgr2=0x56288064be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5091d46740 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880651d80 0x56288064be30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5080034040 tx=0x7f5080009500 comp rx=0 tx=0).stop 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5091d46740 1 -- 172.21.15.78:0/1033540997 shutdown_connections 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5091d46740 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5628806cec90 0x5628806d10b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5091d46740 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562880651d80 0x56288064be30 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5091d46740 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562880645df0 0x56288064b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5091d46740 1 --2- 172.21.15.78:0/1033540997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5628806454d0 0x56288065a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5091d46740 1 -- 172.21.15.78:0/1033540997 >> 172.21.15.78:0/1033540997 conn(0x562880643300 msgr2=0x56288064daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:54.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5070ff9640 0 client.16821 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:05:54.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5091d46740 1 -- 172.21.15.78:0/1033540997 shutdown_connections 2024-09-11T01:05:54.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5091d46740 1 -- 172.21.15.78:0/1033540997 wait complete. 2024-09-11T01:05:54.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.733+0000 7f5091d46740 20 client.16821 trim_cache size 0 max 16384 2024-09-11T01:05:54.742 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:05:54.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.734+0000 7f508f971640 1 -- 172.21.15.78:0/3744376023 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50740b5450 msgr2=0x7f50740b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:54.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.734+0000 7f508f971640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50740b5450 0x7f50740b7910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5080006010 tx=0x7f5080004a40 comp rx=0 tx=0).stop 2024-09-11T01:05:54.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.734+0000 7f508f971640 1 -- 172.21.15.78:0/3744376023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5088108740 msgr2=0x7f5088102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:54.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.734+0000 7f508f971640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5088108740 0x7f5088102770 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f507800e9f0 tx=0x7f507800edf0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.734+0000 7f508f971640 1 -- 172.21.15.78:0/3744376023 shutdown_connections 2024-09-11T01:05:54.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.734+0000 7f508f971640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50740b5450 0x7f50740b7910 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.734+0000 7f508f971640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50881090c0 0x7f5088102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.734+0000 7f508f971640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5088108740 0x7f5088102770 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.734+0000 7f508f971640 1 --2- 172.21.15.78:0/3744376023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5088101260 0x7f5088102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:54.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.734+0000 7f508f971640 1 -- 172.21.15.78:0/3744376023 >> 172.21.15.78:0/3744376023 conn(0x7f5088078060 msgr2=0x7f5088105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:54.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.734+0000 7f508f971640 1 -- 172.21.15.78:0/3744376023 shutdown_connections 2024-09-11T01:05:54.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:54.734+0000 7f508f971640 1 -- 172.21.15.78:0/3744376023 wait complete. 2024-09-11T01:05:55.420 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:05:55.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 504864 5.57 MB/sec execute 243 sec latency 17.725 ms 2024-09-11T01:05:55.683 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:56.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.180+0000 7f120f511640 1 -- 172.21.15.78:0/1841945623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12081087f0 msgr2=0x7f1208108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:56.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.180+0000 7f120f511640 1 --2- 172.21.15.78:0/1841945623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12081087f0 0x7f1208108c30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009900 tx=0x7f11f802a410 comp rx=0 tx=0).stop 2024-09-11T01:05:56.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.180+0000 7f120f511640 1 -- 172.21.15.78:0/1841945623 shutdown_connections 2024-09-11T01:05:56.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.180+0000 7f120f511640 1 --2- 172.21.15.78:0/1841945623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1208109170 0x7f120810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.180+0000 7f120f511640 1 --2- 172.21.15.78:0/1841945623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12081087f0 0x7f1208108c30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.180+0000 7f120f511640 1 --2- 172.21.15.78:0/1841945623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1208102de0 0x7f12081031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.180+0000 7f120f511640 1 -- 172.21.15.78:0/1841945623 >> 172.21.15.78:0/1841945623 conn(0x7f12080fe830 msgr2=0x7f1208100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:56.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.180+0000 7f120f511640 1 -- 172.21.15.78:0/1841945623 shutdown_connections 2024-09-11T01:05:56.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.180+0000 7f120f511640 1 -- 172.21.15.78:0/1841945623 wait complete. 2024-09-11T01:05:56.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.181+0000 7f120f511640 1 Processor -- start 2024-09-11T01:05:56.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.181+0000 7f120f511640 1 -- start start 2024-09-11T01:05:56.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.181+0000 7f120f511640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1208102de0 0x7f12081992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.181+0000 7f120f511640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12081087f0 0x7f12081997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.181+0000 7f120f511640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1208109170 0x7f120819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.181+0000 7f120f511640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1208112e30 con 0x7f12081087f0 2024-09-11T01:05:56.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.181+0000 7f120f511640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1208112cb0 con 0x7f1208102de0 2024-09-11T01:05:56.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.181+0000 7f120f511640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1208112fb0 con 0x7f1208109170 2024-09-11T01:05:56.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.181+0000 7f120caac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12081087f0 0x7f12081997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.181+0000 7f120d2ad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1208102de0 0x7f12081992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120d2ad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1208102de0 0x7f12081992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59994/0 (socket says 172.21.15.78:59994) 2024-09-11T01:05:56.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120d2ad640 1 -- 172.21.15.78:0/760577924 learned_addr learned my addr 172.21.15.78:0/760577924 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:56.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120caac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12081087f0 0x7f12081997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48868/0 (socket says 172.21.15.78:48868) 2024-09-11T01:05:56.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120daae640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1208109170 0x7f120819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120d2ad640 1 -- 172.21.15.78:0/760577924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1208109170 msgr2=0x7f120819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:56.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120d2ad640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1208109170 0x7f120819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120d2ad640 1 -- 172.21.15.78:0/760577924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12081087f0 msgr2=0x7f12081997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:56.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120d2ad640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12081087f0 0x7f12081997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120d2ad640 1 -- 172.21.15.78:0/760577924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120819f350 con 0x7f1208102de0 2024-09-11T01:05:56.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120caac640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12081087f0 0x7f12081997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:56.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120d2ad640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1208102de0 0x7f12081992a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f120400d950 tx=0x7f120400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:56.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f11fe7fc640 1 -- 172.21.15.78:0/760577924 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204017070 con 0x7f1208102de0 2024-09-11T01:05:56.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120f511640 1 -- 172.21.15.78:0/760577924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120819f640 con 0x7f1208102de0 2024-09-11T01:05:56.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f11fe7fc640 1 -- 172.21.15.78:0/760577924 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1204009c30 con 0x7f1208102de0 2024-09-11T01:05:56.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f11fe7fc640 1 -- 172.21.15.78:0/760577924 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120400f620 con 0x7f1208102de0 2024-09-11T01:05:56.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.182+0000 7f120f511640 1 -- 172.21.15.78:0/760577924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12081a5d00 con 0x7f1208102de0 2024-09-11T01:05:56.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f11fe7fc640 1 -- 172.21.15.78:0/760577924 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1204010460 con 0x7f1208102de0 2024-09-11T01:05:56.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f12118e6740 1 Processor -- start 2024-09-11T01:05:56.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f12118e6740 1 -- start start 2024-09-11T01:05:56.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f12118e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 0x55d81b8638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f12118e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b863df0 0x55d81b86f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f12118e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b86fd80 0x55d81b872170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f12118e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d81b7d9ea0 con 0x55d81b863df0 2024-09-11T01:05:56.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f12118e6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d81b7c92d0 con 0x55d81b86fd80 2024-09-11T01:05:56.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f12118e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d81b7cd0e0 con 0x55d81b8634d0 2024-09-11T01:05:56.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f120caac640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b86fd80 0x55d81b872170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f120d2ad640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b863df0 0x55d81b86f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f120daae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 0x55d81b8638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f120caac640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b86fd80 0x55d81b872170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:60002/0 (socket says 172.21.15.78:60002) 2024-09-11T01:05:56.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f120caac640 1 -- 172.21.15.78:0/1712876659 learned_addr learned my addr 172.21.15.78:0/1712876659 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:56.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.185+0000 7f120daae640 1 --2- 172.21.15.78:0/1712876659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 0x55d81b8638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:47710/0 (socket says 172.21.15.78:47710) 2024-09-11T01:05:56.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.186+0000 7f120caac640 1 -- 172.21.15.78:0/1712876659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 msgr2=0x55d81b8638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.186+0000 7f120caac640 1 --2- 172.21.15.78:0/1712876659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 0x55d81b8638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.186+0000 7f120caac640 1 -- 172.21.15.78:0/1712876659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b863df0 msgr2=0x55d81b86f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.186+0000 7f120caac640 1 --2- 172.21.15.78:0/1712876659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b863df0 0x55d81b86f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.186+0000 7f120caac640 1 -- 172.21.15.78:0/1712876659 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81b527650 con 0x55d81b86fd80 2024-09-11T01:05:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.186+0000 7f120daae640 1 --2- 172.21.15.78:0/1712876659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 0x55d81b8638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.186+0000 7f120caac640 1 --2- 172.21.15.78:0/1712876659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b86fd80 0x55d81b872170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009900 tx=0x7f11f8005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde908fc44f8f08b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:56.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.186+0000 7f11fe7fc640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f11f40b5450 0x7f11f40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.186+0000 7f120daae640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f11f40b5450 0x7f11f40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.187+0000 7f11f37fe640 1 -- 172.21.15.78:0/1712876659 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8040070 con 0x55d81b86fd80 2024-09-11T01:05:56.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.187+0000 7f11fe7fc640 1 -- 172.21.15.78:0/760577924 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f12040e5b70 con 0x7f1208102de0 2024-09-11T01:05:56.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.187+0000 7f11f37fe640 1 -- 172.21.15.78:0/1712876659 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f80318e0 con 0x55d81b86fd80 2024-09-11T01:05:56.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.187+0000 7f11f37fe640 1 -- 172.21.15.78:0/1712876659 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8035040 con 0x55d81b86fd80 2024-09-11T01:05:56.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.187+0000 7f12118e6740 1 -- 172.21.15.78:0/1712876659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b86fd80 msgr2=0x55d81b872170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:56.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.187+0000 7f120daae640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f11f40b5450 0x7f11f40b7910 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1200009fd0 tx=0x7f1200009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:56.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.187+0000 7f12118e6740 1 --2- 172.21.15.78:0/1712876659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b86fd80 0x55d81b872170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009900 tx=0x7f11f8005e40 comp rx=0 tx=0).stop 2024-09-11T01:05:56.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.188+0000 7f12118e6740 1 -- 172.21.15.78:0/1712876659 shutdown_connections 2024-09-11T01:05:56.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.188+0000 7f12118e6740 1 --2- 172.21.15.78:0/1712876659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b86fd80 0x55d81b872170 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.188+0000 7f12118e6740 1 --2- 172.21.15.78:0/1712876659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b863df0 0x55d81b86f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.188+0000 7f12118e6740 1 --2- 172.21.15.78:0/1712876659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 0x55d81b8638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.188+0000 7f12118e6740 1 -- 172.21.15.78:0/1712876659 >> 172.21.15.78:0/1712876659 conn(0x55d81b861300 msgr2=0x55d81b8624a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:56.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.188+0000 7f12118e6740 1 -- 172.21.15.78:0/1712876659 shutdown_connections 2024-09-11T01:05:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.188+0000 7f12118e6740 1 -- 172.21.15.78:0/1712876659 wait complete. 2024-09-11T01:05:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.188+0000 7f12118e6740 1 -- start start 2024-09-11T01:05:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.188+0000 7f12118e6740 1 Processor -- start 2024-09-11T01:05:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f12118e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b8634d0 0x55d81b86f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f12118e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b863df0 0x55d81b86cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f12118e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b86fd80 0x55d81b86d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f12118e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d81b62a700 con 0x55d81b8634d0 2024-09-11T01:05:56.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f12118e6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d81b764160 con 0x55d81b863df0 2024-09-11T01:05:56.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f12118e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d81b876db0 con 0x55d81b86fd80 2024-09-11T01:05:56.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f12118e6740 10 client.0 _ll_register_callbacks cb 0x55d81b77d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:56.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120d2ad640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b8634d0 0x55d81b86f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120caac640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b863df0 0x55d81b86cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120daae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b86fd80 0x55d81b86d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120d2ad640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b8634d0 0x55d81b86f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48890/0 (socket says 172.21.15.78:48890) 2024-09-11T01:05:56.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120d2ad640 1 -- 172.21.15.78:0/1399531552 learned_addr learned my addr 172.21.15.78:0/1399531552 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:56.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120d2ad640 1 -- 172.21.15.78:0/1399531552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b86fd80 msgr2=0x55d81b86d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:56.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120d2ad640 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b86fd80 0x55d81b86d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120d2ad640 1 -- 172.21.15.78:0/1399531552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b863df0 msgr2=0x55d81b86cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:56.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120d2ad640 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b863df0 0x55d81b86cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120d2ad640 1 -- 172.21.15.78:0/1399531552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81b791c90 con 0x55d81b8634d0 2024-09-11T01:05:56.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120caac640 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b863df0 0x55d81b86cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:56.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120daae640 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b86fd80 0x55d81b86d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:56.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f120d2ad640 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b8634d0 0x55d81b86f670 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f12040e3330 tx=0x7f12040e9150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0eab1613cab7e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:56.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f11f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f11f0ff9640 1 -- 172.21.15.78:0/1399531552 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204020070 con 0x55d81b8634d0 2024-09-11T01:05:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f11f0ff9640 1 -- 172.21.15.78:0/1399531552 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f12040ab510 con 0x55d81b8634d0 2024-09-11T01:05:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f12118e6740 1 -- 172.21.15.78:0/1399531552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f80318e0 con 0x55d81b8634d0 2024-09-11T01:05:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.189+0000 7f11f0ff9640 1 -- 172.21.15.78:0/1399531552 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f12040b3e60 con 0x55d81b8634d0 2024-09-11T01:05:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f12118e6740 10 client.16833 fetch_fsmap learned FSMap version 42 2024-09-11T01:05:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f12118e6740 1 -- 172.21.15.78:0/1399531552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d81b8826e0 con 0x55d81b8634d0 2024-09-11T01:05:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f11f0ff9640 1 -- 172.21.15.78:0/1399531552 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f80318e0 con 0x55d81b8634d0 2024-09-11T01:05:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f11f0ff9640 1 -- 172.21.15.78:0/1399531552 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f12040a7a00 con 0x55d81b8634d0 2024-09-11T01:05:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f12118e6740 10 client.16833 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T01:05:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f12118e6740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f12118e6740 20 client.16833 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:56.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f12118e6740 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d81b883b40 0x55d81b885fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f12118e6740 4 client.16833 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:56.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f12118e6740 1 -- 172.21.15.78:0/1399531552 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d81b7dbdf0 con 0x55d81b883b40 2024-09-11T01:05:56.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f120caac640 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d81b883b40 0x55d81b885fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f120caac640 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d81b883b40 0x55d81b885fc0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed5e39d3d4e79d8 server_cookie=31d0e2e5ca7dfee2 in_seq=0 out_seq=0 2024-09-11T01:05:56.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.190+0000 7f11f0ff9640 10 client.16833 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.191+0000 7f11f0ff9640 1 -- 172.21.15.78:0/1399531552 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d81b7dbdf0 con 0x55d81b883b40 2024-09-11T01:05:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.191+0000 7f11f0ff9640 10 client.16833 handle_command_reply: tid=0 2024-09-11T01:05:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.191+0000 7f12118e6740 1 client.16833 shutdown 2024-09-11T01:05:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.191+0000 7f12118e6740 1 -- 172.21.15.78:0/1399531552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b8634d0 msgr2=0x55d81b86f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.191+0000 7f12118e6740 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b8634d0 0x55d81b86f670 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f12040e3330 tx=0x7f12040e9150 comp rx=0 tx=0).stop 2024-09-11T01:05:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.191+0000 7f12118e6740 1 -- 172.21.15.78:0/1399531552 shutdown_connections 2024-09-11T01:05:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.191+0000 7f12118e6740 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b86fd80 0x55d81b86d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.191+0000 7f12118e6740 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b863df0 0x55d81b86cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.191+0000 7f12118e6740 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d81b883b40 0x55d81b885fc0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.191+0000 7f12118e6740 1 --2- 172.21.15.78:0/1399531552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b8634d0 0x55d81b86f670 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.191+0000 7f12118e6740 1 -- 172.21.15.78:0/1399531552 >> 172.21.15.78:0/1399531552 conn(0x55d81b861300 msgr2=0x55d81b866ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.191+0000 7f11f0ff9640 0 client.16833 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:56.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.192+0000 7f12118e6740 1 -- 172.21.15.78:0/1399531552 shutdown_connections 2024-09-11T01:05:56.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.192+0000 7f12118e6740 1 -- 172.21.15.78:0/1399531552 wait complete. 2024-09-11T01:05:56.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.192+0000 7f12118e6740 20 client.16833 trim_cache size 0 max 16384 2024-09-11T01:05:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.207+0000 7f12118e6740 1 Processor -- start 2024-09-11T01:05:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.207+0000 7f12118e6740 1 -- start start 2024-09-11T01:05:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.207+0000 7f12118e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 0x55d81b87ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.207+0000 7f12118e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b863df0 0x55d81b871170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.207+0000 7f12118e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 0x55d81b8716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.207+0000 7f12118e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ac3a0 con 0x55d81b86fd80 2024-09-11T01:05:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.207+0000 7f12118e6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ae000 con 0x55d81b863df0 2024-09-11T01:05:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f12118e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12081aca70 con 0x55d81b8634d0 2024-09-11T01:05:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f120daae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 0x55d81b8716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f120daae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 0x55d81b8716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48898/0 (socket says 172.21.15.78:48898) 2024-09-11T01:05:56.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f120daae640 1 -- 172.21.15.78:0/4207814587 learned_addr learned my addr 172.21.15.78:0/4207814587 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:56.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f120d2ad640 1 --2- 172.21.15.78:0/4207814587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 0x55d81b87ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f120caac640 1 --2- 172.21.15.78:0/4207814587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b863df0 0x55d81b871170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f120daae640 1 -- 172.21.15.78:0/4207814587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 msgr2=0x55d81b87ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:56.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f120daae640 1 --2- 172.21.15.78:0/4207814587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 0x55d81b87ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f120daae640 1 -- 172.21.15.78:0/4207814587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b863df0 msgr2=0x55d81b871170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:56.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f120daae640 1 --2- 172.21.15.78:0/4207814587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b863df0 0x55d81b871170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f120daae640 1 -- 172.21.15.78:0/4207814587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81b86f2a0 con 0x55d81b86fd80 2024-09-11T01:05:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f120d2ad640 1 --2- 172.21.15.78:0/4207814587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 0x55d81b87ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f120daae640 1 --2- 172.21.15.78:0/4207814587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 0x55d81b8716b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f120000fa10 tx=0x7f1200019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14867cfc40d5ad32 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.208+0000 7f11f2ffd640 1 -- 172.21.15.78:0/4207814587 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200010930 con 0x55d81b86fd80 2024-09-11T01:05:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.209+0000 7f11f2ffd640 1 -- 172.21.15.78:0/4207814587 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f120001e420 con 0x55d81b86fd80 2024-09-11T01:05:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.209+0000 7f11f2ffd640 1 -- 172.21.15.78:0/4207814587 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120001c650 con 0x55d81b86fd80 2024-09-11T01:05:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.209+0000 7f12118e6740 1 -- 172.21.15.78:0/4207814587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 msgr2=0x55d81b8716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.209+0000 7f12118e6740 1 --2- 172.21.15.78:0/4207814587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 0x55d81b8716b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f120000fa10 tx=0x7f1200019000 comp rx=0 tx=0).stop 2024-09-11T01:05:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 -- 172.21.15.78:0/4207814587 shutdown_connections 2024-09-11T01:05:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 --2- 172.21.15.78:0/4207814587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 0x55d81b8716b0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 --2- 172.21.15.78:0/4207814587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b863df0 0x55d81b871170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 --2- 172.21.15.78:0/4207814587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b8634d0 0x55d81b87ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 -- 172.21.15.78:0/4207814587 >> 172.21.15.78:0/4207814587 conn(0x55d81b861300 msgr2=0x55d81b861fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 -- 172.21.15.78:0/4207814587 shutdown_connections 2024-09-11T01:05:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 -- 172.21.15.78:0/4207814587 wait complete. 2024-09-11T01:05:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 -- start start 2024-09-11T01:05:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 Processor -- start 2024-09-11T01:05:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b8634d0 0x55d81b88bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b863df0 0x55d81b87c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 0x55d81b87c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ad870 con 0x55d81b86fd80 2024-09-11T01:05:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ad1a0 con 0x55d81b8634d0 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12081aa380 con 0x55d81b863df0 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.210+0000 7f12118e6740 10 client.0 _ll_register_callbacks cb 0x7f120400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120daae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 0x55d81b87c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120d2ad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b8634d0 0x55d81b88bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120d2ad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b8634d0 0x55d81b88bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:60034/0 (socket says 172.21.15.78:60034) 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120d2ad640 1 -- 172.21.15.78:0/3130727114 learned_addr learned my addr 172.21.15.78:0/3130727114 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120daae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 0x55d81b87c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48906/0 (socket says 172.21.15.78:48906) 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120caac640 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b863df0 0x55d81b87c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120d2ad640 1 -- 172.21.15.78:0/3130727114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b863df0 msgr2=0x55d81b87c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120d2ad640 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b863df0 0x55d81b87c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120d2ad640 1 -- 172.21.15.78:0/3130727114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 msgr2=0x55d81b87c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120d2ad640 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 0x55d81b87c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120d2ad640 1 -- 172.21.15.78:0/3130727114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81b87cc10 con 0x55d81b8634d0 2024-09-11T01:05:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120daae640 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 0x55d81b87c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:05:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f120d2ad640 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b8634d0 0x55d81b88bfc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f12040a9320 tx=0x7f12040ab540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67a9cebddbe26dda server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f11f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:05:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f11f0ff9640 1 -- 172.21.15.78:0/3130727114 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204020070 con 0x55d81b8634d0 2024-09-11T01:05:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f11f0ff9640 1 -- 172.21.15.78:0/3130727114 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f12040aad80 con 0x55d81b8634d0 2024-09-11T01:05:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f12118e6740 1 -- 172.21.15.78:0/3130727114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120001e420 con 0x55d81b8634d0 2024-09-11T01:05:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f11f0ff9640 1 -- 172.21.15.78:0/3130727114 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f12040aba00 con 0x55d81b8634d0 2024-09-11T01:05:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f12118e6740 10 client.26132 fetch_fsmap learned FSMap version 42 2024-09-11T01:05:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.211+0000 7f12118e6740 1 -- 172.21.15.78:0/3130727114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d81b8f5280 con 0x55d81b8634d0 2024-09-11T01:05:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.212+0000 7f11f0ff9640 1 -- 172.21.15.78:0/3130727114 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f120001e420 con 0x55d81b8634d0 2024-09-11T01:05:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.212+0000 7f11f0ff9640 1 -- 172.21.15.78:0/3130727114 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f12040b1c70 con 0x55d81b8634d0 2024-09-11T01:05:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.212+0000 7f12118e6740 10 client.26132 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T01:05:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.212+0000 7f12118e6740 10 client.26132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.212+0000 7f12118e6740 20 client.26132 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.212+0000 7f12118e6740 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d81b8f5a80 0x55d81b8f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.212+0000 7f12118e6740 4 client.26132 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:05:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.212+0000 7f12118e6740 1 -- 172.21.15.78:0/3130727114 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d81b7d78a0 con 0x55d81b8f5a80 2024-09-11T01:05:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.212+0000 7f120caac640 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d81b8f5a80 0x55d81b8f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.212+0000 7f120caac640 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d81b8f5a80 0x55d81b8f7ea0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b0b5a46c07be1ec server_cookie=8b48afe175059619 in_seq=0 out_seq=0 2024-09-11T01:05:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.212+0000 7f11f0ff9640 10 client.26132 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f11f0ff9640 1 -- 172.21.15.78:0/3130727114 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d81b7d78a0 con 0x55d81b8f5a80 2024-09-11T01:05:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f11f0ff9640 10 client.26132 handle_command_reply: tid=0 2024-09-11T01:05:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f12118e6740 1 client.26132 shutdown 2024-09-11T01:05:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f12118e6740 1 -- 172.21.15.78:0/3130727114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b8634d0 msgr2=0x55d81b88bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f12118e6740 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b8634d0 0x55d81b88bfc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f12040a9320 tx=0x7f12040ab540 comp rx=0 tx=0).stop 2024-09-11T01:05:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f12118e6740 1 -- 172.21.15.78:0/3130727114 shutdown_connections 2024-09-11T01:05:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f12118e6740 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d81b8f5a80 0x55d81b8f7ea0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f12118e6740 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d81b86fd80 0x55d81b87c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f12118e6740 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d81b863df0 0x55d81b87c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f12118e6740 1 --2- 172.21.15.78:0/3130727114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d81b8634d0 0x55d81b88bfc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f11f0ff9640 0 client.26132 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f12118e6740 1 -- 172.21.15.78:0/3130727114 >> 172.21.15.78:0/3130727114 conn(0x55d81b861300 msgr2=0x55d81b86e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f12118e6740 1 -- 172.21.15.78:0/3130727114 shutdown_connections 2024-09-11T01:05:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f12118e6740 1 -- 172.21.15.78:0/3130727114 wait complete. 2024-09-11T01:05:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.213+0000 7f12118e6740 20 client.26132 trim_cache size 0 max 16384 2024-09-11T01:05:56.239 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:05:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.214+0000 7f120f511640 1 -- 172.21.15.78:0/760577924 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f11f40b5450 msgr2=0x7f11f40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.214+0000 7f120f511640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f11f40b5450 0x7f11f40b7910 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1200009fd0 tx=0x7f1200009500 comp rx=0 tx=0).stop 2024-09-11T01:05:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.214+0000 7f120f511640 1 -- 172.21.15.78:0/760577924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1208102de0 msgr2=0x7f12081992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.214+0000 7f120f511640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1208102de0 0x7f12081992a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f120400d950 tx=0x7f120400de10 comp rx=0 tx=0).stop 2024-09-11T01:05:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.214+0000 7f120f511640 1 -- 172.21.15.78:0/760577924 shutdown_connections 2024-09-11T01:05:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.214+0000 7f120f511640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f11f40b5450 0x7f11f40b7910 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.214+0000 7f120f511640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1208109170 0x7f120819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.214+0000 7f120f511640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12081087f0 0x7f12081997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.214+0000 7f120f511640 1 --2- 172.21.15.78:0/760577924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1208102de0 0x7f12081992a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.214+0000 7f120f511640 1 -- 172.21.15.78:0/760577924 >> 172.21.15.78:0/760577924 conn(0x7f12080fe830 msgr2=0x7f1208100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.214+0000 7f120f511640 1 -- 172.21.15.78:0/760577924 shutdown_connections 2024-09-11T01:05:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:56.214+0000 7f120f511640 1 -- 172.21.15.78:0/760577924 wait complete. 2024-09-11T01:05:56.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 506183 5.57 MB/sec execute 244 sec latency 159.786 ms 2024-09-11T01:05:56.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:56 smithi078 ceph-mon[31315]: pgmap v535: 97 pgs: 97 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-09-11T01:05:56.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:56 smithi110 ceph-mon[34712]: pgmap v535: 97 pgs: 97 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-09-11T01:05:56.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:56 smithi112 ceph-mon[35384]: pgmap v535: 97 pgs: 97 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-09-11T01:05:57.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 506898 5.56 MB/sec execute 245 sec latency 913.541 ms 2024-09-11T01:05:58.304 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:05:58.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:05:58.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 508699 5.58 MB/sec execute 246 sec latency 10.154 ms 2024-09-11T01:05:58.575 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:05:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:58 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:05:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:58 smithi110 ceph-mon[34712]: pgmap v536: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.5 MiB/s wr, 162 op/s 2024-09-11T01:05:58.698 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:58 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:05:58.698 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:58 smithi078 ceph-mon[31315]: pgmap v536: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.5 MiB/s wr, 162 op/s 2024-09-11T01:05:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:58 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:05:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:58 smithi112 ceph-mon[35384]: pgmap v536: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.5 MiB/s wr, 162 op/s 2024-09-11T01:05:59.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.074+0000 7fbca9c38640 1 -- 172.21.15.78:0/1602084112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbca41076e0 msgr2=0x7fbca4107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:59.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.074+0000 7fbca9c38640 1 --2- 172.21.15.78:0/1602084112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbca41076e0 0x7fbca4107b20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009900 tx=0x7fbc8c02a3f0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.074+0000 7fbca9c38640 1 -- 172.21.15.78:0/1602084112 shutdown_connections 2024-09-11T01:05:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.074+0000 7fbca9c38640 1 --2- 172.21.15.78:0/1602084112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca4108060 0x7fbca410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.074+0000 7fbca9c38640 1 --2- 172.21.15.78:0/1602084112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbca41076e0 0x7fbca4107b20 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.074+0000 7fbca9c38640 1 --2- 172.21.15.78:0/1602084112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbca4101cd0 0x7fbca41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.074+0000 7fbca9c38640 1 -- 172.21.15.78:0/1602084112 >> 172.21.15.78:0/1602084112 conn(0x7fbca40fd700 msgr2=0x7fbca40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.074+0000 7fbca9c38640 1 -- 172.21.15.78:0/1602084112 shutdown_connections 2024-09-11T01:05:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.074+0000 7fbca9c38640 1 -- 172.21.15.78:0/1602084112 wait complete. 2024-09-11T01:05:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.075+0000 7fbca9c38640 1 Processor -- start 2024-09-11T01:05:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.075+0000 7fbca9c38640 1 -- start start 2024-09-11T01:05:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.075+0000 7fbca9c38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbca4101cd0 0x7fbca4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.075+0000 7fbca9c38640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca41076e0 0x7fbca4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.075+0000 7fbca9c38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbca4108060 0x7fbca4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.075+0000 7fbca9c38640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4111d20 con 0x7fbca4101cd0 2024-09-11T01:05:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.075+0000 7fbca9c38640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4111ba0 con 0x7fbca41076e0 2024-09-11T01:05:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.075+0000 7fbca9c38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4111ea0 con 0x7fbca4108060 2024-09-11T01:05:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.075+0000 7fbca37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbca4101cd0 0x7fbca4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.075+0000 7fbca37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbca4101cd0 0x7fbca4197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48920/0 (socket says 172.21.15.78:48920) 2024-09-11T01:05:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca37fe640 1 -- 172.21.15.78:0/1832398599 learned_addr learned my addr 172.21.15.78:0/1832398599 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbc9affd640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca41076e0 0x7fbca4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca3fff640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbca4108060 0x7fbca4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca37fe640 1 -- 172.21.15.78:0/1832398599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbca4108060 msgr2=0x7fbca4198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca37fe640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbca4108060 0x7fbca4198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca37fe640 1 -- 172.21.15.78:0/1832398599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca41076e0 msgr2=0x7fbca4198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca37fe640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca41076e0 0x7fbca4198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca37fe640 1 -- 172.21.15.78:0/1832398599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca41a75e0 con 0x7fbca4101cd0 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca3fff640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbca4108060 0x7fbca4198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca37fe640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbca4101cd0 0x7fbca4197e00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400c920 tx=0x7fbc9400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca17fa640 1 -- 172.21.15.78:0/1832398599 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94007c70 con 0x7fbca4101cd0 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca17fa640 1 -- 172.21.15.78:0/1832398599 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc94007e10 con 0x7fbca4101cd0 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca9c38640 1 -- 172.21.15.78:0/1832398599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca41a78d0 con 0x7fbca4101cd0 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca17fa640 1 -- 172.21.15.78:0/1832398599 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94018820 con 0x7fbca4101cd0 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.076+0000 7fbca9c38640 1 -- 172.21.15.78:0/1832398599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca41a7e30 con 0x7fbca4101cd0 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbca17fa640 1 -- 172.21.15.78:0/1832398599 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fbc9400f450 con 0x7fbca4101cd0 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbcac00d740 1 Processor -- start 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbcac00d740 1 -- start start 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbcac00d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969d62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbcac00d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 0x55df969e2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbcac00d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969e2790 0x55df969e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbcac00d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55df9694ce60 con 0x55df969d5ee0 2024-09-11T01:05:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbcac00d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55df9693c290 con 0x55df969d6800 2024-09-11T01:05:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbcac00d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55df969400a0 con 0x55df969e2790 2024-09-11T01:05:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbca3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969d62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbca3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969d62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48932/0 (socket says 172.21.15.78:48932) 2024-09-11T01:05:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbca3fff640 1 -- 172.21.15.78:0/1045396160 learned_addr learned my addr 172.21.15.78:0/1045396160 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbca37fe640 1 --2- 172.21.15.78:0/1045396160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 0x55df969e2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbc9affd640 1 --2- 172.21.15.78:0/1045396160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969e2790 0x55df969e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbca3fff640 1 -- 172.21.15.78:0/1045396160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969e2790 msgr2=0x55df969e4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbca3fff640 1 --2- 172.21.15.78:0/1045396160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969e2790 0x55df969e4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbca3fff640 1 -- 172.21.15.78:0/1045396160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 msgr2=0x55df969e2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.079+0000 7fbca3fff640 1 --2- 172.21.15.78:0/1045396160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 0x55df969e2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.080+0000 7fbca3fff640 1 -- 172.21.15.78:0/1045396160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df9669a930 con 0x55df969d5ee0 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.080+0000 7fbca17fa640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbc900b54d0 0x7fbc900b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.080+0000 7fbca3fff640 1 --2- 172.21.15.78:0/1045396160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969d62c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009f40 tx=0x7fbc9c00e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9390cb81c028ff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.080+0000 7fbca37fe640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbc900b54d0 0x7fbc900b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.080+0000 7fbca17fa640 1 -- 172.21.15.78:0/1832398599 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbc940e6020 con 0x7fbca4101cd0 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.080+0000 7fbc99ffb640 1 -- 172.21.15.78:0/1045396160 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c016070 con 0x55df969d5ee0 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.080+0000 7fbc99ffb640 1 -- 172.21.15.78:0/1045396160 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc9c004d20 con 0x55df969d5ee0 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.080+0000 7fbc99ffb640 1 -- 172.21.15.78:0/1045396160 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c0117a0 con 0x55df969d5ee0 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbcac00d740 1 -- 172.21.15.78:0/1045396160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 msgr2=0x55df969d62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbcac00d740 1 --2- 172.21.15.78:0/1045396160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969d62c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009f40 tx=0x7fbc9c00e780 comp rx=0 tx=0).stop 2024-09-11T01:05:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbca37fe640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbc900b54d0 0x7fbc900b7990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbc940e3890 tx=0x7fbc94000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbcac00d740 1 -- 172.21.15.78:0/1045396160 shutdown_connections 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbcac00d740 1 --2- 172.21.15.78:0/1045396160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969e2790 0x55df969e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbcac00d740 1 --2- 172.21.15.78:0/1045396160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 0x55df969e2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbcac00d740 1 --2- 172.21.15.78:0/1045396160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969d62c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbcac00d740 1 -- 172.21.15.78:0/1045396160 >> 172.21.15.78:0/1045396160 conn(0x55df969d3d10 msgr2=0x55df969d4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbcac00d740 1 -- 172.21.15.78:0/1045396160 shutdown_connections 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbcac00d740 1 -- 172.21.15.78:0/1045396160 wait complete. 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbcac00d740 1 -- start start 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbcac00d740 1 Processor -- start 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.081+0000 7fbcac00d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbcac00d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969d6800 0x55df969dfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbcac00d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969e2790 0x55df969e0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbcac00d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55df9679d890 con 0x55df969d5ee0 2024-09-11T01:05:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbcac00d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55df968d6d40 con 0x55df969e2790 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbcac00d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55df969e5440 con 0x55df969d6800 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbcac00d740 10 client.0 _ll_register_callbacks cb 0x55df9693ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbca3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbca3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969df6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48936/0 (socket says 172.21.15.78:48936) 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbca3fff640 1 -- 172.21.15.78:0/3693190835 learned_addr learned my addr 172.21.15.78:0/3693190835 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbc9affd640 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969d6800 0x55df969dfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbca3fff640 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969e2790 0x55df969e0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbca3fff640 1 -- 172.21.15.78:0/3693190835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969d6800 msgr2=0x55df969dfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbca3fff640 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969d6800 0x55df969dfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbca3fff640 1 -- 172.21.15.78:0/3693190835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969e2790 msgr2=0x55df969e0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbca3fff640 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969e2790 0x55df969e0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbca3fff640 1 -- 172.21.15.78:0/3693190835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df969ee7a0 con 0x55df969d5ee0 2024-09-11T01:05:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbc9affd640 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969d6800 0x55df969dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbca3fff640 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969df6c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00a010 tx=0x7fbc9c0094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71a6058ccefdbad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbc82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbc82ffd640 1 -- 172.21.15.78:0/3693190835 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c016040 con 0x55df969d5ee0 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbcac00d740 1 -- 172.21.15.78:0/3693190835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc9c004d20 con 0x55df969d5ee0 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbc82ffd640 1 -- 172.21.15.78:0/3693190835 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbc9c003970 con 0x55df969d5ee0 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbc82ffd640 1 -- 172.21.15.78:0/3693190835 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbc9c003b70 con 0x55df969d5ee0 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbcac00d740 10 client.16851 fetch_fsmap learned FSMap version 43 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.082+0000 7fbcac00d740 1 -- 172.21.15.78:0/3693190835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55df969f1130 con 0x55df969d5ee0 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.083+0000 7fbc82ffd640 1 -- 172.21.15.78:0/3693190835 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc9c004d20 con 0x55df969d5ee0 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.083+0000 7fbc82ffd640 1 -- 172.21.15.78:0/3693190835 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fbc9c030020 con 0x55df969d5ee0 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.083+0000 7fbcac00d740 10 client.16851 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.083+0000 7fbcac00d740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.083+0000 7fbcac00d740 20 client.16851 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:59.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.083+0000 7fbcac00d740 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55df969f24e0 0x55df969f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.083+0000 7fbcac00d740 4 client.16851 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:05:59.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.083+0000 7fbcac00d740 1 -- 172.21.15.78:0/3693190835 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df9694edb0 con 0x55df969f24e0 2024-09-11T01:05:59.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.083+0000 7fbc9affd640 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55df969f24e0 0x55df969f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.083+0000 7fbc82ffd640 10 client.16851 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:59.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.083+0000 7fbc9affd640 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55df969f24e0 0x55df969f48d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b08b1f96397d4a8 server_cookie=2cbc57094efbb66f in_seq=0 out_seq=0 2024-09-11T01:05:59.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbc82ffd640 1 -- 172.21.15.78:0/3693190835 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df9694edb0 con 0x55df969f24e0 2024-09-11T01:05:59.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbc82ffd640 10 client.16851 handle_command_reply: tid=0 2024-09-11T01:05:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbcac00d740 1 client.16851 shutdown 2024-09-11T01:05:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbcac00d740 1 -- 172.21.15.78:0/3693190835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 msgr2=0x55df969df6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbcac00d740 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969df6c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00a010 tx=0x7fbc9c0094d0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbcac00d740 1 -- 172.21.15.78:0/3693190835 shutdown_connections 2024-09-11T01:05:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbcac00d740 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969e2790 0x55df969e0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbcac00d740 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969d6800 0x55df969dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbcac00d740 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55df969f24e0 0x55df969f48d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbcac00d740 1 --2- 172.21.15.78:0/3693190835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969df6c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbc82ffd640 0 client.16851 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbcac00d740 1 -- 172.21.15.78:0/3693190835 >> 172.21.15.78:0/3693190835 conn(0x55df969d3d10 msgr2=0x55df969d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbcac00d740 1 -- 172.21.15.78:0/3693190835 shutdown_connections 2024-09-11T01:05:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.084+0000 7fbcac00d740 1 -- 172.21.15.78:0/3693190835 wait complete. 2024-09-11T01:05:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.085+0000 7fbcac00d740 20 client.16851 trim_cache size 0 max 16384 2024-09-11T01:05:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.102+0000 7fbcac00d740 1 Processor -- start 2024-09-11T01:05:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.102+0000 7fbcac00d740 1 -- start start 2024-09-11T01:05:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.102+0000 7fbcac00d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.102+0000 7fbcac00d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 0x55df969dc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbcac00d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969e2790 0x55df969dce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbcac00d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbca41b0720 con 0x55df969d5ee0 2024-09-11T01:05:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbcac00d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbca41b22f0 con 0x55df969d6800 2024-09-11T01:05:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbcac00d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbca41b0df0 con 0x55df969e2790 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbca3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbca3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969de5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48942/0 (socket says 172.21.15.78:48942) 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbca3fff640 1 -- 172.21.15.78:0/3261343790 learned_addr learned my addr 172.21.15.78:0/3261343790 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbc9affd640 1 --2- 172.21.15.78:0/3261343790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 0x55df969dc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbca3fff640 1 -- 172.21.15.78:0/3261343790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969e2790 msgr2=0x55df969dce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbca3fff640 1 --2- 172.21.15.78:0/3261343790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969e2790 0x55df969dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbca3fff640 1 -- 172.21.15.78:0/3261343790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 msgr2=0x55df969dc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbca3fff640 1 --2- 172.21.15.78:0/3261343790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 0x55df969dc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbca3fff640 1 -- 172.21.15.78:0/3261343790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df969d4790 con 0x55df969d5ee0 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbca3fff640 1 --2- 172.21.15.78:0/3261343790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969de5b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c002230 tx=0x7fbc9c0053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436312d02d452bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbc997fa640 1 -- 172.21.15.78:0/3261343790 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c016030 con 0x55df969d5ee0 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbc997fa640 1 -- 172.21.15.78:0/3261343790 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc9c01e070 con 0x55df969d5ee0 2024-09-11T01:05:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.103+0000 7fbc997fa640 1 -- 172.21.15.78:0/3261343790 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c0116d0 con 0x55df969d5ee0 2024-09-11T01:05:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.104+0000 7fbcac00d740 1 -- 172.21.15.78:0/3261343790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 msgr2=0x55df969de5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.104+0000 7fbcac00d740 1 --2- 172.21.15.78:0/3261343790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969de5b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c002230 tx=0x7fbc9c0053d0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.104+0000 7fbcac00d740 1 -- 172.21.15.78:0/3261343790 shutdown_connections 2024-09-11T01:05:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.104+0000 7fbcac00d740 1 --2- 172.21.15.78:0/3261343790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969e2790 0x55df969dce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.104+0000 7fbcac00d740 1 --2- 172.21.15.78:0/3261343790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 0x55df969dc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.104+0000 7fbcac00d740 1 --2- 172.21.15.78:0/3261343790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969d5ee0 0x55df969de5b0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.104+0000 7fbcac00d740 1 -- 172.21.15.78:0/3261343790 >> 172.21.15.78:0/3261343790 conn(0x55df969d3d10 msgr2=0x55df969e1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.104+0000 7fbcac00d740 1 -- 172.21.15.78:0/3261343790 shutdown_connections 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.104+0000 7fbcac00d740 1 -- 172.21.15.78:0/3261343790 wait complete. 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.104+0000 7fbcac00d740 1 -- start start 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.104+0000 7fbcac00d740 1 Processor -- start 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbcac00d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969d5ee0 0x55df969e5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbcac00d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 0x55df969e5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbcac00d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969e2790 0x55df969e61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbcac00d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbca41b1bf0 con 0x55df969e2790 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbcac00d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbca41b1520 con 0x55df969d6800 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbcac00d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbca41ae700 con 0x55df969d5ee0 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbcac00d740 10 client.0 _ll_register_callbacks cb 0x7fbc9c009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbca3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969e2790 0x55df969e61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbca3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969e2790 0x55df969e61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48946/0 (socket says 172.21.15.78:48946) 2024-09-11T01:05:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbca3fff640 1 -- 172.21.15.78:0/1698428576 learned_addr learned my addr 172.21.15.78:0/1698428576 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:05:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbca3fff640 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969d5ee0 0x55df969e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbc9affd640 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 0x55df969e5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbca3fff640 1 -- 172.21.15.78:0/1698428576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969d5ee0 msgr2=0x55df969e5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbca3fff640 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969d5ee0 0x55df969e5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbca3fff640 1 -- 172.21.15.78:0/1698428576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 msgr2=0x55df969e5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:05:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbca3fff640 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 0x55df969e5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbca3fff640 1 -- 172.21.15.78:0/1698428576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df96a5f200 con 0x55df969e2790 2024-09-11T01:05:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbca3fff640 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969e2790 0x55df969e61c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c000f60 tx=0x7fbc9c02b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1acffd4e309e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:05:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbc82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:05:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.105+0000 7fbc82ffd640 1 -- 172.21.15.78:0/1698428576 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c016030 con 0x55df969e2790 2024-09-11T01:05:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbc82ffd640 1 -- 172.21.15.78:0/1698428576 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbc9c003970 con 0x55df969e2790 2024-09-11T01:05:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbcac00d740 1 -- 172.21.15.78:0/1698428576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc9c01e070 con 0x55df969e2790 2024-09-11T01:05:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbc82ffd640 1 -- 172.21.15.78:0/1698428576 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbc9c032b30 con 0x55df969e2790 2024-09-11T01:05:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbcac00d740 10 client.16857 fetch_fsmap learned FSMap version 43 2024-09-11T01:05:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbcac00d740 1 -- 172.21.15.78:0/1698428576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55df96a60540 con 0x55df969e2790 2024-09-11T01:05:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbc82ffd640 1 -- 172.21.15.78:0/1698428576 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc9c01e070 con 0x55df969e2790 2024-09-11T01:05:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbc82ffd640 1 -- 172.21.15.78:0/1698428576 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fbc9c030040 con 0x55df969e2790 2024-09-11T01:05:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbcac00d740 10 client.16857 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T01:05:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbcac00d740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:05:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbcac00d740 20 client.16857 populate_metadata read hostname 'smithi078' 2024-09-11T01:05:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbcac00d740 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55df96a60b70 0x55df96a62f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:05:59.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbcac00d740 4 client.16857 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:05:59.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbcac00d740 1 -- 172.21.15.78:0/1698428576 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55df9694a860 con 0x55df96a60b70 2024-09-11T01:05:59.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.106+0000 7fbc9affd640 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55df96a60b70 0x55df96a62f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:05:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.107+0000 7fbc9affd640 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55df96a60b70 0x55df96a62f90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5df84b0e382f84ae server_cookie=e4b4a3478bc885e6 in_seq=0 out_seq=0 2024-09-11T01:05:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.107+0000 7fbc82ffd640 10 client.16857 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbc82ffd640 1 -- 172.21.15.78:0/1698428576 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55df9694a860 con 0x55df96a60b70 2024-09-11T01:05:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbc82ffd640 10 client.16857 handle_command_reply: tid=0 2024-09-11T01:05:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbcac00d740 1 client.16857 shutdown 2024-09-11T01:05:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbcac00d740 1 -- 172.21.15.78:0/1698428576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969e2790 msgr2=0x55df969e61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbcac00d740 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969e2790 0x55df969e61c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c000f60 tx=0x7fbc9c02b040 comp rx=0 tx=0).stop 2024-09-11T01:05:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbcac00d740 1 -- 172.21.15.78:0/1698428576 shutdown_connections 2024-09-11T01:05:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbcac00d740 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55df96a60b70 0x55df96a62f90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbcac00d740 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df969e2790 0x55df969e61c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbcac00d740 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55df969d6800 0x55df969e5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbcac00d740 1 --2- 172.21.15.78:0/1698428576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df969d5ee0 0x55df969e5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbc82ffd640 0 client.16857 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:05:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbcac00d740 1 -- 172.21.15.78:0/1698428576 >> 172.21.15.78:0/1698428576 conn(0x55df969d3d10 msgr2=0x55df969e0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbcac00d740 1 -- 172.21.15.78:0/1698428576 shutdown_connections 2024-09-11T01:05:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbcac00d740 1 -- 172.21.15.78:0/1698428576 wait complete. 2024-09-11T01:05:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.108+0000 7fbcac00d740 20 client.16857 trim_cache size 0 max 16384 2024-09-11T01:05:59.122 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:05:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:05:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "c3c800f7-782d-4ca3-b38b-b456abe7aba2", 2024-09-11T01:05:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:05:59.122 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:05:59.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.109+0000 7fbca9c38640 1 -- 172.21.15.78:0/1832398599 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbc900b54d0 msgr2=0x7fbc900b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.109+0000 7fbca9c38640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbc900b54d0 0x7fbc900b7990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbc940e3890 tx=0x7fbc94000bc0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.109+0000 7fbca9c38640 1 -- 172.21.15.78:0/1832398599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbca4101cd0 msgr2=0x7fbca4197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:05:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.109+0000 7fbca9c38640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbca4101cd0 0x7fbca4197e00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400c920 tx=0x7fbc9400cde0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.109+0000 7fbca9c38640 1 -- 172.21.15.78:0/1832398599 shutdown_connections 2024-09-11T01:05:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.109+0000 7fbca9c38640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbc900b54d0 0x7fbc900b7990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.109+0000 7fbca9c38640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbca4108060 0x7fbca4198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.109+0000 7fbca9c38640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbca41076e0 0x7fbca4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.109+0000 7fbca9c38640 1 --2- 172.21.15.78:0/1832398599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbca4101cd0 0x7fbca4197e00 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:05:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.109+0000 7fbca9c38640 1 -- 172.21.15.78:0/1832398599 >> 172.21.15.78:0/1832398599 conn(0x7fbca40fd700 msgr2=0x7fbca40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:05:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.109+0000 7fbca9c38640 1 -- 172.21.15.78:0/1832398599 shutdown_connections 2024-09-11T01:05:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:05:59.109+0000 7fbca9c38640 1 -- 172.21.15.78:0/1832398599 wait complete. 2024-09-11T01:05:59.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 510488 5.59 MB/sec execute 247 sec latency 122.218 ms 2024-09-11T01:05:59.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:59 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:05:59.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:59 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:59.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:59 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:59.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:59 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:59.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:59 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:59.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:59 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:59.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:59 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:59.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:59 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:05:59.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:05:59 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:59.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:59 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:05:59.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:59 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:59.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:59 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:59.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:59 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:59.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:59 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:59.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:59 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:59.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:59 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:59.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:59 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:05:59.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:05:59 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:59.732 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:59 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:05:59.733 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:59 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:59.733 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:59 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:59.733 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:59 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:59.733 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:59 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:59.733 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:59 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:59.733 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:59 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:05:59.733 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:59 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:05:59.733 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:05:59 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 512163 5.62 MB/sec execute 248 sec latency 39.948 ms 2024-09-11T01:06:00.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:00.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:00.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:00.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:00.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:00.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:00.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:00.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:00.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:00 smithi078 ceph-mon[31315]: pgmap v537: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-09-11T01:06:00.521 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:06:00.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:00.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:00.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:00.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:00.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:00.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:00.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:00.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:00.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:00 smithi110 ceph-mon[34712]: pgmap v537: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-09-11T01:06:00.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:00.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:00.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:00.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:00 smithi112 ceph-mon[35384]: pgmap v537: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-09-11T01:06:00.797 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:06:01.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.339+0000 7f02d3b8b640 1 -- 172.21.15.78:0/861809758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02cc1076a0 msgr2=0x7f02cc107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:01.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.339+0000 7f02d3b8b640 1 --2- 172.21.15.78:0/861809758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02cc1076a0 0x7f02cc107ae0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009900 tx=0x7f02bc02a410 comp rx=0 tx=0).stop 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.340+0000 7f02d3b8b640 1 -- 172.21.15.78:0/861809758 shutdown_connections 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.340+0000 7f02d3b8b640 1 --2- 172.21.15.78:0/861809758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02cc108020 0x7f02cc10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.340+0000 7f02d3b8b640 1 --2- 172.21.15.78:0/861809758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02cc1076a0 0x7f02cc107ae0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.340+0000 7f02d3b8b640 1 --2- 172.21.15.78:0/861809758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc101c90 0x7f02cc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.340+0000 7f02d3b8b640 1 -- 172.21.15.78:0/861809758 >> 172.21.15.78:0/861809758 conn(0x7f02cc0fd700 msgr2=0x7f02cc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.340+0000 7f02d3b8b640 1 -- 172.21.15.78:0/861809758 shutdown_connections 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.340+0000 7f02d3b8b640 1 -- 172.21.15.78:0/861809758 wait complete. 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.340+0000 7f02d3b8b640 1 Processor -- start 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.340+0000 7f02d3b8b640 1 -- start start 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.340+0000 7f02d3b8b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc101c90 0x7f02cc1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d3b8b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02cc1076a0 0x7f02cc1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d3b8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02cc108020 0x7f02cc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d3b8b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc111ce0 con 0x7f02cc1076a0 2024-09-11T01:06:01.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d3b8b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc111b60 con 0x7f02cc101c90 2024-09-11T01:06:01.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d3b8b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc111e60 con 0x7f02cc108020 2024-09-11T01:06:01.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d1927640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc101c90 0x7f02cc1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d1126640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02cc1076a0 0x7f02cc1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d1927640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc101c90 0x7f02cc1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:60104/0 (socket says 172.21.15.78:60104) 2024-09-11T01:06:01.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d1927640 1 -- 172.21.15.78:0/2809682559 learned_addr learned my addr 172.21.15.78:0/2809682559 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:01.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d2128640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02cc108020 0x7f02cc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d2128640 1 -- 172.21.15.78:0/2809682559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc101c90 msgr2=0x7f02cc1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:01.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d2128640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc101c90 0x7f02cc1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d2128640 1 -- 172.21.15.78:0/2809682559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02cc1076a0 msgr2=0x7f02cc1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:01.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d2128640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02cc1076a0 0x7f02cc1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d2128640 1 -- 172.21.15.78:0/2809682559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02cc19f350 con 0x7f02cc108020 2024-09-11T01:06:01.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d1126640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02cc1076a0 0x7f02cc1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:01.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02d2128640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02cc108020 0x7f02cc19eb80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f02c400da10 tx=0x7f02c400ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:01.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02c2ffd640 1 -- 172.21.15.78:0/2809682559 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c400bd70 con 0x7f02cc108020 2024-09-11T01:06:01.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.341+0000 7f02c2ffd640 1 -- 172.21.15.78:0/2809682559 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c4003e20 con 0x7f02cc108020 2024-09-11T01:06:01.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.342+0000 7f02d3b8b640 1 -- 172.21.15.78:0/2809682559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02cc19f640 con 0x7f02cc108020 2024-09-11T01:06:01.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.342+0000 7f02c2ffd640 1 -- 172.21.15.78:0/2809682559 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c40106c0 con 0x7f02cc108020 2024-09-11T01:06:01.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.342+0000 7f02d3b8b640 1 -- 172.21.15.78:0/2809682559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02cc1a5d00 con 0x7f02cc108020 2024-09-11T01:06:01.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.344+0000 7f02d5f60740 1 Processor -- start 2024-09-11T01:06:01.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.345+0000 7f02c2ffd640 1 -- 172.21.15.78:0/2809682559 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f02c4024060 con 0x7f02cc108020 2024-09-11T01:06:01.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.345+0000 7f02d5f60740 1 -- start start 2024-09-11T01:06:01.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.345+0000 7f02d5f60740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee715d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.345+0000 7f02d5f60740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee715dda0 0x563ee716dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.345+0000 7f02d5f60740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee716e140 0x563ee7170530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.345+0000 7f02d5f60740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563ee70d3e50 con 0x563ee716e140 2024-09-11T01:06:01.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.345+0000 7f02d5f60740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563ee70c3280 con 0x563ee715dda0 2024-09-11T01:06:01.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.345+0000 7f02d5f60740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563ee70c7090 con 0x563ee715d480 2024-09-11T01:06:01.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.345+0000 7f02d1126640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee715d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.345+0000 7f02d1927640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee715dda0 0x563ee716dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.345+0000 7f02d1126640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee715d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:47804/0 (socket says 172.21.15.78:47804) 2024-09-11T01:06:01.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.345+0000 7f02d1126640 1 -- 172.21.15.78:0/2777333232 learned_addr learned my addr 172.21.15.78:0/2777333232 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:01.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02d1126640 1 --2- 172.21.15.78:0/2777333232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee716e140 0x563ee7170530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02d1927640 1 -- 172.21.15.78:0/2777333232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 msgr2=0x563ee715d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:01.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02d1927640 1 --2- 172.21.15.78:0/2777333232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee715d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02d1927640 1 -- 172.21.15.78:0/2777333232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee716e140 msgr2=0x563ee7170530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02d1927640 1 --2- 172.21.15.78:0/2777333232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee716e140 0x563ee7170530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02d1927640 1 -- 172.21.15.78:0/2777333232 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee6e21650 con 0x563ee715dda0 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02c2ffd640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02b80b5450 0x7f02b80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02d1927640 1 --2- 172.21.15.78:0/2777333232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee715dda0 0x563ee716dc00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009c80 tx=0x7f02c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a810daaf34e3a55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02d1126640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02b80b5450 0x7f02b80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02b7fff640 1 -- 172.21.15.78:0/2777333232 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8004250 con 0x563ee715dda0 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02c2ffd640 1 -- 172.21.15.78:0/2809682559 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f02c40e6040 con 0x7f02cc108020 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02b7fff640 1 -- 172.21.15.78:0/2777333232 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c8009e30 con 0x563ee715dda0 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.346+0000 7f02b7fff640 1 -- 172.21.15.78:0/2777333232 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8004d40 con 0x563ee715dda0 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.347+0000 7f02d1126640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02b80b5450 0x7f02b80b7910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f02bc005ec0 tx=0x7f02bc005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.347+0000 7f02d5f60740 1 -- 172.21.15.78:0/2777333232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee715dda0 msgr2=0x563ee716dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.347+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2777333232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee715dda0 0x563ee716dc00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009c80 tx=0x7f02c8011810 comp rx=0 tx=0).stop 2024-09-11T01:06:01.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.347+0000 7f02d5f60740 1 -- 172.21.15.78:0/2777333232 shutdown_connections 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.347+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2777333232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee716e140 0x563ee7170530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.347+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2777333232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee715dda0 0x563ee716dc00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.347+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2777333232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee715d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.347+0000 7f02d5f60740 1 -- 172.21.15.78:0/2777333232 >> 172.21.15.78:0/2777333232 conn(0x563ee715b2b0 msgr2=0x563ee715c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.348+0000 7f02d5f60740 1 -- 172.21.15.78:0/2777333232 shutdown_connections 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.348+0000 7f02d5f60740 1 -- 172.21.15.78:0/2777333232 wait complete. 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.348+0000 7f02d5f60740 1 -- start start 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.348+0000 7f02d5f60740 1 Processor -- start 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d5f60740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee716d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d5f60740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee715dda0 0x563ee716ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d5f60740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee716e140 0x563ee716b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d5f60740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563ee6f24700 con 0x563ee716e140 2024-09-11T01:06:01.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d5f60740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563ee705e110 con 0x563ee715dda0 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d5f60740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563ee7170df0 con 0x563ee715d480 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d2128640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee716e140 0x563ee716b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d5f60740 10 client.0 _ll_register_callbacks cb 0x563ee7077990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d1126640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee715dda0 0x563ee716ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d2128640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee716e140 0x563ee716b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48980/0 (socket says 172.21.15.78:48980) 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d2128640 1 -- 172.21.15.78:0/1113787704 learned_addr learned my addr 172.21.15.78:0/1113787704 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d1927640 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee716d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d2128640 1 -- 172.21.15.78:0/1113787704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 msgr2=0x563ee716d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d2128640 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee716d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d2128640 1 -- 172.21.15.78:0/1113787704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee715dda0 msgr2=0x563ee716ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d2128640 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee715dda0 0x563ee716ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d2128640 1 -- 172.21.15.78:0/1113787704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee708bc40 con 0x563ee716e140 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d1126640 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee715dda0 0x563ee716ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02d2128640 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee716e140 0x563ee716b3b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f02c40ac4c0 tx=0x7f02c40ac100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1259a8e7af599de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.349+0000 7f02b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02b57fa640 1 -- 172.21.15.78:0/1113787704 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c40b1010 con 0x563ee716e140 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02b57fa640 1 -- 172.21.15.78:0/1113787704 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f02c4004270 con 0x563ee716e140 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02d5f60740 1 -- 172.21.15.78:0/1113787704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02c8009e30 con 0x563ee716e140 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02b57fa640 1 -- 172.21.15.78:0/1113787704 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f02c40b5ac0 con 0x563ee716e140 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02d5f60740 10 client.16869 fetch_fsmap learned FSMap version 43 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02d5f60740 1 -- 172.21.15.78:0/1113787704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563ee717c770 con 0x563ee716e140 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02b57fa640 1 -- 172.21.15.78:0/1113787704 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c8009e30 con 0x563ee716e140 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02b57fa640 1 -- 172.21.15.78:0/1113787704 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f02c4004bb0 con 0x563ee716e140 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02d5f60740 10 client.16869 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02d5f60740 10 client.16869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02d5f60740 20 client.16869 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02d5f60740 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ee717e060 0x563ee7180500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02d5f60740 4 client.16869 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:06:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.350+0000 7f02d5f60740 1 -- 172.21.15.78:0/1113787704 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ee70d5da0 con 0x563ee717e060 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.351+0000 7f02d1927640 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ee717e060 0x563ee7180500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.351+0000 7f02d1927640 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ee717e060 0x563ee7180500 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15c38f395d067b7 server_cookie=c3b0d1d18c7e3447 in_seq=0 out_seq=0 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.351+0000 7f02b57fa640 10 client.16869 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.352+0000 7f02b57fa640 1 -- 172.21.15.78:0/1113787704 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563ee70d5da0 con 0x563ee717e060 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.352+0000 7f02b57fa640 10 client.16869 handle_command_reply: tid=0 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.352+0000 7f02d5f60740 1 client.16869 shutdown 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.352+0000 7f02d5f60740 1 -- 172.21.15.78:0/1113787704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee716e140 msgr2=0x563ee716b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.352+0000 7f02d5f60740 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee716e140 0x563ee716b3b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f02c40ac4c0 tx=0x7f02c40ac100 comp rx=0 tx=0).stop 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.352+0000 7f02d5f60740 1 -- 172.21.15.78:0/1113787704 shutdown_connections 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.352+0000 7f02d5f60740 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ee717e060 0x563ee7180500 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.352+0000 7f02d5f60740 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee716e140 0x563ee716b3b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.352+0000 7f02d5f60740 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee715dda0 0x563ee716ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.352+0000 7f02d5f60740 1 --2- 172.21.15.78:0/1113787704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee716d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.352+0000 7f02d5f60740 1 -- 172.21.15.78:0/1113787704 >> 172.21.15.78:0/1113787704 conn(0x563ee715b2b0 msgr2=0x563ee7160de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:01.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.352+0000 7f02b57fa640 0 client.16869 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:01.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.353+0000 7f02d5f60740 1 -- 172.21.15.78:0/1113787704 shutdown_connections 2024-09-11T01:06:01.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.353+0000 7f02d5f60740 1 -- 172.21.15.78:0/1113787704 wait complete. 2024-09-11T01:06:01.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.353+0000 7f02d5f60740 20 client.16869 trim_cache size 0 max 16384 2024-09-11T01:06:01.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.371+0000 7f02d5f60740 1 Processor -- start 2024-09-11T01:06:01.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.371+0000 7f02d5f60740 1 -- start start 2024-09-11T01:06:01.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.371+0000 7f02d5f60740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee716ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.371+0000 7f02d5f60740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715dda0 0x563ee715c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.371+0000 7f02d5f60740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee716e140 0x563ee715c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.371+0000 7f02d5f60740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc1ac350 con 0x563ee715dda0 2024-09-11T01:06:01.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.371+0000 7f02d5f60740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc1adf20 con 0x563ee716e140 2024-09-11T01:06:01.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.371+0000 7f02d5f60740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc1aca20 con 0x563ee715d480 2024-09-11T01:06:01.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02d1126640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715dda0 0x563ee715c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02d1126640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715dda0 0x563ee715c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48990/0 (socket says 172.21.15.78:48990) 2024-09-11T01:06:01.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02d1126640 1 -- 172.21.15.78:0/2853614329 learned_addr learned my addr 172.21.15.78:0/2853614329 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:01.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02d2128640 1 --2- 172.21.15.78:0/2853614329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee716e140 0x563ee715c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02d1927640 1 --2- 172.21.15.78:0/2853614329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee716ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02d1126640 1 -- 172.21.15.78:0/2853614329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 msgr2=0x563ee716ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:01.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02d1126640 1 --2- 172.21.15.78:0/2853614329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee716ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02d1126640 1 -- 172.21.15.78:0/2853614329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee716e140 msgr2=0x563ee715c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:01.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02d1126640 1 --2- 172.21.15.78:0/2853614329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee716e140 0x563ee715c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02d1126640 1 -- 172.21.15.78:0/2853614329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee71705a0 con 0x563ee715dda0 2024-09-11T01:06:01.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02d1126640 1 --2- 172.21.15.78:0/2853614329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715dda0 0x563ee715c3e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009900 tx=0x7f02bc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6c6d71e759abdf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:01.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02b77fe640 1 -- 172.21.15.78:0/2853614329 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc035040 con 0x563ee715dda0 2024-09-11T01:06:01.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02b77fe640 1 -- 172.21.15.78:0/2853614329 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02bc003e20 con 0x563ee715dda0 2024-09-11T01:06:01.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.372+0000 7f02b77fe640 1 -- 172.21.15.78:0/2853614329 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc03de50 con 0x563ee715dda0 2024-09-11T01:06:01.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.373+0000 7f02d5f60740 1 -- 172.21.15.78:0/2853614329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715dda0 msgr2=0x563ee715c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:01.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.373+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2853614329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715dda0 0x563ee715c3e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009900 tx=0x7f02bc009260 comp rx=0 tx=0).stop 2024-09-11T01:06:01.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.373+0000 7f02d5f60740 1 -- 172.21.15.78:0/2853614329 shutdown_connections 2024-09-11T01:06:01.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.373+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2853614329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee716e140 0x563ee715c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.373+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2853614329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715dda0 0x563ee715c3e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.373+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2853614329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715d480 0x563ee716ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.373+0000 7f02d5f60740 1 -- 172.21.15.78:0/2853614329 >> 172.21.15.78:0/2853614329 conn(0x563ee715b2b0 msgr2=0x563ee71696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:01.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.373+0000 7f02d5f60740 1 -- 172.21.15.78:0/2853614329 shutdown_connections 2024-09-11T01:06:01.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.373+0000 7f02d5f60740 1 -- 172.21.15.78:0/2853614329 wait complete. 2024-09-11T01:06:01.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.373+0000 7f02d5f60740 1 -- start start 2024-09-11T01:06:01.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.373+0000 7f02d5f60740 1 Processor -- start 2024-09-11T01:06:01.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d5f60740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715d480 0x563ee7185d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d5f60740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715dda0 0x563ee7186240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d5f60740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee716e140 0x563ee7186780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d5f60740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc1ad820 con 0x563ee715d480 2024-09-11T01:06:01.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d5f60740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc1ad150 con 0x563ee716e140 2024-09-11T01:06:01.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d5f60740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc1aa330 con 0x563ee715dda0 2024-09-11T01:06:01.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d1927640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715d480 0x563ee7185d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d1927640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715d480 0x563ee7185d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48998/0 (socket says 172.21.15.78:48998) 2024-09-11T01:06:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d1927640 1 -- 172.21.15.78:0/2614760172 learned_addr learned my addr 172.21.15.78:0/2614760172 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d1126640 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715dda0 0x563ee7186240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d2128640 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee716e140 0x563ee7186780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d5f60740 10 client.0 _ll_register_callbacks cb 0x7f02c4004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d1927640 1 -- 172.21.15.78:0/2614760172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715dda0 msgr2=0x563ee7186240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d1927640 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715dda0 0x563ee7186240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d1927640 1 -- 172.21.15.78:0/2614760172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee716e140 msgr2=0x563ee7186780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d1927640 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee716e140 0x563ee7186780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d1927640 1 -- 172.21.15.78:0/2614760172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee71ea060 con 0x563ee715d480 2024-09-11T01:06:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.374+0000 7f02d2128640 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee716e140 0x563ee7186780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02d1927640 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715d480 0x563ee7185d00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009bd0 tx=0x7f02c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd35b49b2cf9f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02b57fa640 1 -- 172.21.15.78:0/2614760172 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c80037f0 con 0x563ee715d480 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02b57fa640 1 -- 172.21.15.78:0/2614760172 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f02c8003990 con 0x563ee715d480 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02d5f60740 1 -- 172.21.15.78:0/2614760172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02bc003e20 con 0x563ee715d480 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02b57fa640 1 -- 172.21.15.78:0/2614760172 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f02c803d050 con 0x563ee715d480 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02d5f60740 10 client.16875 fetch_fsmap learned FSMap version 43 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02d5f60740 1 -- 172.21.15.78:0/2614760172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563ee71eb380 con 0x563ee715d480 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02b57fa640 1 -- 172.21.15.78:0/2614760172 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02bc003e20 con 0x563ee715d480 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02b57fa640 1 -- 172.21.15.78:0/2614760172 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f02c80198e0 con 0x563ee715d480 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02d5f60740 10 client.16875 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02d5f60740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:06:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02d5f60740 20 client.16875 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:01.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.375+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ee71eba40 0x563ee71ede60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:01.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02d5f60740 4 client.16875 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:06:01.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02d5f60740 1 -- 172.21.15.78:0/2614760172 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ee70d1850 con 0x563ee71eba40 2024-09-11T01:06:01.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02d1126640 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ee71eba40 0x563ee71ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:01.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02b57fa640 10 client.16875 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:01.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02d1126640 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ee71eba40 0x563ee71ede60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d57842833ecfeff8 server_cookie=1afb511be727236c in_seq=0 out_seq=0 2024-09-11T01:06:01.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02b57fa640 1 -- 172.21.15.78:0/2614760172 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x563ee70d1850 con 0x563ee71eba40 2024-09-11T01:06:01.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02b57fa640 10 client.16875 handle_command_reply: tid=0 2024-09-11T01:06:01.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02d5f60740 1 client.16875 shutdown 2024-09-11T01:06:01.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02d5f60740 1 -- 172.21.15.78:0/2614760172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715d480 msgr2=0x563ee7185d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:01.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715d480 0x563ee7185d00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009bd0 tx=0x7f02c8004270 comp rx=0 tx=0).stop 2024-09-11T01:06:01.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02d5f60740 1 -- 172.21.15.78:0/2614760172 shutdown_connections 2024-09-11T01:06:01.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ee716e140 0x563ee7186780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ee715dda0 0x563ee7186240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.376+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563ee71eba40 0x563ee71ede60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d5f60740 1 --2- 172.21.15.78:0/2614760172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563ee715d480 0x563ee7185d00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02b57fa640 0 client.16875 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:01.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d5f60740 1 -- 172.21.15.78:0/2614760172 >> 172.21.15.78:0/2614760172 conn(0x563ee715b2b0 msgr2=0x563ee715c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:01.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d5f60740 1 -- 172.21.15.78:0/2614760172 shutdown_connections 2024-09-11T01:06:01.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d5f60740 1 -- 172.21.15.78:0/2614760172 wait complete. 2024-09-11T01:06:01.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d5f60740 20 client.16875 trim_cache size 0 max 16384 2024-09-11T01:06:01.385 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:06:01.386 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:06:01.386 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:06:01.386 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:06:01.386 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:01.386 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "c3c800f7-782d-4ca3-b38b-b456abe7aba2", 2024-09-11T01:06:01.386 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:06:01.386 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:06:01.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:01.386 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:06:01.386 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:06:01.386 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:06:01.386 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:01.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:01.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:01.389 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:01.389 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:01.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:01.389 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:06:01.389 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:06:01.389 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:01.389 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:01.389 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:01.389 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:01.389 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:01.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:01.389 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:01.390 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:01.391 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:06:01.392 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:06:01.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d3b8b640 1 -- 172.21.15.78:0/2809682559 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02b80b5450 msgr2=0x7f02b80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:01.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d3b8b640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02b80b5450 0x7f02b80b7910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f02bc005ec0 tx=0x7f02bc005b50 comp rx=0 tx=0).stop 2024-09-11T01:06:01.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d3b8b640 1 -- 172.21.15.78:0/2809682559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02cc108020 msgr2=0x7f02cc19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:01.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d3b8b640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02cc108020 0x7f02cc19eb80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f02c400da10 tx=0x7f02c400ded0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d3b8b640 1 -- 172.21.15.78:0/2809682559 shutdown_connections 2024-09-11T01:06:01.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d3b8b640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02b80b5450 0x7f02b80b7910 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d3b8b640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02cc108020 0x7f02cc19eb80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d3b8b640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02cc1076a0 0x7f02cc1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d3b8b640 1 --2- 172.21.15.78:0/2809682559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc101c90 0x7f02cc1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:01.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.377+0000 7f02d3b8b640 1 -- 172.21.15.78:0/2809682559 >> 172.21.15.78:0/2809682559 conn(0x7f02cc0fd700 msgr2=0x7f02cc06aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:01.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.378+0000 7f02d3b8b640 1 -- 172.21.15.78:0/2809682559 shutdown_connections 2024-09-11T01:06:01.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:01.378+0000 7f02d3b8b640 1 -- 172.21.15.78:0/2809682559 wait complete. 2024-09-11T01:06:01.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 513919 5.62 MB/sec execute 249 sec latency 208.905 ms 2024-09-11T01:06:02.216 INFO:tasks.cephfs.filesystem:scrub status for tag:c3c800f7-782d-4ca3-b38b-b456abe7aba2 - {'path': '/', 'tag': 'c3c800f7-782d-4ca3-b38b-b456abe7aba2', 'options': 'recursive,force'} 2024-09-11T01:06:02.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 515596 5.65 MB/sec execute 250 sec latency 23.762 ms 2024-09-11T01:06:03.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 516816 5.65 MB/sec execute 251 sec latency 364.227 ms 2024-09-11T01:06:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:03 smithi110 ceph-mon[34712]: pgmap v538: 97 pgs: 97 active+clean; 341 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-09-11T01:06:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:03 smithi110 ceph-mon[34712]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T01:06:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:03 smithi112 ceph-mon[35384]: pgmap v538: 97 pgs: 97 active+clean; 341 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-09-11T01:06:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:03 smithi112 ceph-mon[35384]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T01:06:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:03 smithi078 ceph-mon[31315]: pgmap v538: 97 pgs: 97 active+clean; 341 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-09-11T01:06:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:03 smithi078 ceph-mon[31315]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T01:06:04.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 516828 5.62 MB/sec execute 252 sec latency 1344.751 ms 2024-09-11T01:06:05.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 519051 5.66 MB/sec execute 253 sec latency 15.294 ms 2024-09-11T01:06:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:05 smithi110 ceph-mon[34712]: pgmap v540: 97 pgs: 97 active+clean; 344 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 269 op/s 2024-09-11T01:06:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:05 smithi112 ceph-mon[35384]: pgmap v540: 97 pgs: 97 active+clean; 344 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 269 op/s 2024-09-11T01:06:05.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:05 smithi078 ceph-mon[31315]: pgmap v540: 97 pgs: 97 active+clean; 344 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 269 op/s 2024-09-11T01:06:06.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 520365 5.65 MB/sec execute 254 sec latency 236.017 ms 2024-09-11T01:06:07.437 INFO:tasks.workunit.client.0.smithi078.stdout: 1 521075 5.64 MB/sec execute 255 sec latency 992.535 ms 2024-09-11T01:06:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:07 smithi110 ceph-mon[34712]: pgmap v541: 97 pgs: 97 active+clean; 345 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 242 op/s 2024-09-11T01:06:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:07 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:06:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:07 smithi112 ceph-mon[35384]: pgmap v541: 97 pgs: 97 active+clean; 345 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 242 op/s 2024-09-11T01:06:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:07 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:06:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:07 smithi078 ceph-mon[31315]: pgmap v541: 97 pgs: 97 active+clean; 345 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 242 op/s 2024-09-11T01:06:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:07 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:06:08.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 522684 5.66 MB/sec execute 256 sec latency 86.659 ms 2024-09-11T01:06:08.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:08 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:06:08.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:08 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:06:08.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:08 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:06:08.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:08 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:06:08.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:08 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:06:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:08 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:06:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:08 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:06:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:08 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:06:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:08 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:06:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:08 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:06:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:08 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:06:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:08 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:06:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:08 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:06:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:08 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:06:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:08 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:06:09.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 523906 5.66 MB/sec execute 257 sec latency 334.198 ms 2024-09-11T01:06:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: pgmap v542: 97 pgs: 97 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 255 op/s 2024-09-11T01:06:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T01:06:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:06:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:06:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:06:09.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:06:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: pgmap v542: 97 pgs: 97 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 255 op/s 2024-09-11T01:06:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T01:06:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:06:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:06:09.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:06:09.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:06:09.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: pgmap v542: 97 pgs: 97 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 255 op/s 2024-09-11T01:06:09.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T01:06:09.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:06:09.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T01:06:09.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:06:09.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:06:09.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:06:10.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 523906 5.64 MB/sec execute 258 sec latency 1334.289 ms 2024-09-11T01:06:10.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:10 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi110 to 176.0M 2024-09-11T01:06:10.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:10 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-11T01:06:10.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:10 smithi110 ceph-mon[34712]: Unable to set osd_memory_target on smithi110 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-11T01:06:10.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:10 smithi110 ceph-mon[34712]: Unable to set osd_memory_target on smithi112 to 184567398: error parsing value: Value '184567398' is below minimum 939524096 2024-09-11T01:06:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:10 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi110 to 176.0M 2024-09-11T01:06:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:10 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-11T01:06:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:10 smithi112 ceph-mon[35384]: Unable to set osd_memory_target on smithi110 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-11T01:06:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:10 smithi112 ceph-mon[35384]: Unable to set osd_memory_target on smithi112 to 184567398: error parsing value: Value '184567398' is below minimum 939524096 2024-09-11T01:06:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:10 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi110 to 176.0M 2024-09-11T01:06:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:10 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-11T01:06:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:10 smithi078 ceph-mon[31315]: Unable to set osd_memory_target on smithi110 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-11T01:06:10.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:10 smithi078 ceph-mon[31315]: Unable to set osd_memory_target on smithi112 to 184567398: error parsing value: Value '184567398' is below minimum 939524096 2024-09-11T01:06:11.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 523906 5.62 MB/sec execute 259 sec latency 2334.376 ms 2024-09-11T01:06:11.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:11 smithi110 ceph-mon[34712]: pgmap v544: 97 pgs: 97 active+clean; 348 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 2.5 MiB/s wr, 235 op/s 2024-09-11T01:06:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:11 smithi112 ceph-mon[35384]: pgmap v544: 97 pgs: 97 active+clean; 348 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 2.5 MiB/s wr, 235 op/s 2024-09-11T01:06:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:11 smithi078 ceph-mon[31315]: pgmap v544: 97 pgs: 97 active+clean; 348 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 2.5 MiB/s wr, 235 op/s 2024-09-11T01:06:12.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 525424 5.63 MB/sec execute 260 sec latency 2727.758 ms 2024-09-11T01:06:13.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 527026 5.64 MB/sec execute 261 sec latency 10.707 ms 2024-09-11T01:06:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:13 smithi110 ceph-mon[34712]: pgmap v545: 97 pgs: 97 active+clean; 351 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2024-09-11T01:06:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:13 smithi112 ceph-mon[35384]: pgmap v545: 97 pgs: 97 active+clean; 351 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2024-09-11T01:06:13.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:13 smithi078 ceph-mon[31315]: pgmap v545: 97 pgs: 97 active+clean; 351 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2024-09-11T01:06:14.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 527456 5.63 MB/sec execute 262 sec latency 790.022 ms 2024-09-11T01:06:15.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 527456 5.61 MB/sec execute 263 sec latency 1790.112 ms 2024-09-11T01:06:15.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:15 smithi112 ceph-mon[35384]: pgmap v546: 97 pgs: 97 active+clean; 352 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2024-09-11T01:06:15.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:15 smithi078 ceph-mon[31315]: pgmap v546: 97 pgs: 97 active+clean; 352 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2024-09-11T01:06:15.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:15 smithi110 ceph-mon[34712]: pgmap v546: 97 pgs: 97 active+clean; 352 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2024-09-11T01:06:16.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 527456 5.58 MB/sec execute 264 sec latency 2790.219 ms 2024-09-11T01:06:16.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:16 smithi078 ceph-mon[31315]: pgmap v547: 97 pgs: 97 active+clean; 355 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-09-11T01:06:16.693 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:16 smithi110 ceph-mon[34712]: pgmap v547: 97 pgs: 97 active+clean; 355 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-09-11T01:06:16.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:16 smithi112 ceph-mon[35384]: pgmap v547: 97 pgs: 97 active+clean; 355 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-09-11T01:06:17.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 528161 5.57 MB/sec execute 265 sec latency 3546.326 ms 2024-09-11T01:06:18.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 529902 5.60 MB/sec execute 266 sec latency 11.868 ms 2024-09-11T01:06:18.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:18 smithi078 ceph-mon[31315]: pgmap v548: 97 pgs: 97 active+clean; 355 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-09-11T01:06:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:18 smithi110 ceph-mon[34712]: pgmap v548: 97 pgs: 97 active+clean; 355 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-09-11T01:06:18.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:18 smithi112 ceph-mon[35384]: pgmap v548: 97 pgs: 97 active+clean; 355 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-09-11T01:06:19.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 530999 5.59 MB/sec execute 267 sec latency 458.539 ms 2024-09-11T01:06:20.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 530999 5.57 MB/sec execute 268 sec latency 1458.614 ms 2024-09-11T01:06:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:20 smithi078 ceph-mon[31315]: pgmap v549: 97 pgs: 97 active+clean; 355 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-09-11T01:06:20.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:20 smithi110 ceph-mon[34712]: pgmap v549: 97 pgs: 97 active+clean; 355 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-09-11T01:06:20.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:20 smithi112 ceph-mon[35384]: pgmap v549: 97 pgs: 97 active+clean; 355 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-09-11T01:06:21.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 530999 5.55 MB/sec execute 269 sec latency 2458.677 ms 2024-09-11T01:06:22.438 INFO:tasks.workunit.client.0.smithi078.stdout: 1 531742 5.54 MB/sec execute 270 sec latency 3213.312 ms 2024-09-11T01:06:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:22 smithi078 ceph-mon[31315]: pgmap v550: 97 pgs: 97 active+clean; 358 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-09-11T01:06:22.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:22 smithi110 ceph-mon[34712]: pgmap v550: 97 pgs: 97 active+clean; 358 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-09-11T01:06:22.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:22 smithi112 ceph-mon[35384]: pgmap v550: 97 pgs: 97 active+clean; 358 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-09-11T01:06:23.439 INFO:tasks.workunit.client.0.smithi078.stdout: 1 533480 5.56 MB/sec execute 271 sec latency 12.524 ms 2024-09-11T01:06:24.439 INFO:tasks.workunit.client.0.smithi078.stdout: 1 534591 5.56 MB/sec execute 272 sec latency 424.062 ms 2024-09-11T01:06:24.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:24 smithi078 ceph-mon[31315]: pgmap v551: 97 pgs: 97 active+clean; 359 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-09-11T01:06:24.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:24 smithi110 ceph-mon[34712]: pgmap v551: 97 pgs: 97 active+clean; 359 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-09-11T01:06:24.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:24 smithi112 ceph-mon[35384]: pgmap v551: 97 pgs: 97 active+clean; 359 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-09-11T01:06:25.439 INFO:tasks.workunit.client.0.smithi078.stdout: 1 536813 5.59 MB/sec execute 273 sec latency 12.129 ms 2024-09-11T01:06:26.439 INFO:tasks.workunit.client.0.smithi078.stdout: 1 538122 5.59 MB/sec execute 274 sec latency 234.081 ms 2024-09-11T01:06:26.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:26 smithi078 ceph-mon[31315]: pgmap v552: 97 pgs: 97 active+clean; 363 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-09-11T01:06:26.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:26 smithi110 ceph-mon[34712]: pgmap v552: 97 pgs: 97 active+clean; 363 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-09-11T01:06:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:26 smithi112 ceph-mon[35384]: pgmap v552: 97 pgs: 97 active+clean; 363 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-09-11T01:06:27.439 INFO:tasks.workunit.client.0.smithi078.stdout: 1 538820 5.58 MB/sec execute 275 sec latency 988.553 ms 2024-09-11T01:06:28.439 INFO:tasks.workunit.client.0.smithi078.stdout: 1 540452 5.60 MB/sec execute 276 sec latency 13.849 ms 2024-09-11T01:06:28.694 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:28 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:06:28.694 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:28 smithi110 ceph-mon[34712]: pgmap v553: 97 pgs: 97 active+clean; 363 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-09-11T01:06:28.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:28 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:06:28.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:28 smithi112 ceph-mon[35384]: pgmap v553: 97 pgs: 97 active+clean; 363 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-09-11T01:06:28.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:28 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:06:28.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:28 smithi078 ceph-mon[31315]: pgmap v553: 97 pgs: 97 active+clean; 363 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-09-11T01:06:29.439 INFO:tasks.workunit.client.0.smithi078.stdout: 1 541671 5.60 MB/sec execute 277 sec latency 340.367 ms 2024-09-11T01:06:30.439 INFO:tasks.workunit.client.0.smithi078.stdout: 1 541671 5.58 MB/sec execute 278 sec latency 1340.452 ms 2024-09-11T01:06:30.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:30 smithi078 ceph-mon[31315]: pgmap v554: 97 pgs: 97 active+clean; 367 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-09-11T01:06:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:30 smithi110 ceph-mon[34712]: pgmap v554: 97 pgs: 97 active+clean; 367 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-09-11T01:06:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:30 smithi112 ceph-mon[35384]: pgmap v554: 97 pgs: 97 active+clean; 367 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-09-11T01:06:31.439 INFO:tasks.workunit.client.0.smithi078.stdout: 1 541671 5.56 MB/sec execute 279 sec latency 2340.545 ms 2024-09-11T01:06:32.221 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:06:32.439 INFO:tasks.workunit.client.0.smithi078.stdout: 1 543167 5.57 MB/sec execute 280 sec latency 2733.694 ms 2024-09-11T01:06:32.486 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:06:32.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:32 smithi078 ceph-mon[31315]: pgmap v555: 97 pgs: 97 active+clean; 368 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-09-11T01:06:32.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:32 smithi110 ceph-mon[34712]: pgmap v555: 97 pgs: 97 active+clean; 368 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-09-11T01:06:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:32 smithi112 ceph-mon[35384]: pgmap v555: 97 pgs: 97 active+clean; 368 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-09-11T01:06:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:32.999+0000 7f6385aad640 1 -- 172.21.15.78:0/537490814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6380102dc0 msgr2=0x7f63801031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:32.999+0000 7f6385aad640 1 --2- 172.21.15.78:0/537490814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6380102dc0 0x7f63801031a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6370009900 tx=0x7f637002a410 comp rx=0 tx=0).stop 2024-09-11T01:06:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:32.999+0000 7f6385aad640 1 -- 172.21.15.78:0/537490814 shutdown_connections 2024-09-11T01:06:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:32.999+0000 7f6385aad640 1 --2- 172.21.15.78:0/537490814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6380109150 0x7f638010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:32.999+0000 7f6385aad640 1 --2- 172.21.15.78:0/537490814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63801087d0 0x7f6380108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:32.999+0000 7f6385aad640 1 --2- 172.21.15.78:0/537490814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6380102dc0 0x7f63801031a0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:32.999+0000 7f6385aad640 1 -- 172.21.15.78:0/537490814 >> 172.21.15.78:0/537490814 conn(0x7f63800fe830 msgr2=0x7f6380100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:32.999+0000 7f6385aad640 1 -- 172.21.15.78:0/537490814 shutdown_connections 2024-09-11T01:06:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:32.999+0000 7f6385aad640 1 -- 172.21.15.78:0/537490814 wait complete. 2024-09-11T01:06:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6385aad640 1 Processor -- start 2024-09-11T01:06:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6385aad640 1 -- start start 2024-09-11T01:06:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6385aad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6380102dc0 0x7f63801992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6385aad640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63801087d0 0x7f6380199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6385aad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6380109150 0x7f638019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6385aad640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6380112e10 con 0x7f63801087d0 2024-09-11T01:06:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6385aad640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6380112c90 con 0x7f6380109150 2024-09-11T01:06:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6385aad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6380112f90 con 0x7f6380102dc0 2024-09-11T01:06:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6377fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63801087d0 0x7f6380199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6377fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63801087d0 0x7f6380199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43734/0 (socket says 172.21.15.78:43734) 2024-09-11T01:06:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f637ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6380109150 0x7f638019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6377fff640 1 -- 172.21.15.78:0/252736340 learned_addr learned my addr 172.21.15.78:0/252736340 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f637f7fe640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6380102dc0 0x7f63801992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6377fff640 1 -- 172.21.15.78:0/252736340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6380102dc0 msgr2=0x7f63801992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6377fff640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6380102dc0 0x7f63801992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6377fff640 1 -- 172.21.15.78:0/252736340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6380109150 msgr2=0x7f638019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6377fff640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6380109150 0x7f638019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f6377fff640 1 -- 172.21.15.78:0/252736340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638019f370 con 0x7f63801087d0 2024-09-11T01:06:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.000+0000 7f637ffff640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6380109150 0x7f638019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.001+0000 7f6377fff640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63801087d0 0x7f6380199800 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f636800e9f0 tx=0x7f636800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.001+0000 7f637d7fa640 1 -- 172.21.15.78:0/252736340 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636800cd80 con 0x7f63801087d0 2024-09-11T01:06:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.001+0000 7f637d7fa640 1 -- 172.21.15.78:0/252736340 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63680054d0 con 0x7f63801087d0 2024-09-11T01:06:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.001+0000 7f6385aad640 1 -- 172.21.15.78:0/252736340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638019f660 con 0x7f63801087d0 2024-09-11T01:06:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.001+0000 7f637d7fa640 1 -- 172.21.15.78:0/252736340 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368010690 con 0x7f63801087d0 2024-09-11T01:06:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.001+0000 7f6385aad640 1 -- 172.21.15.78:0/252736340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63801a5d20 con 0x7f63801087d0 2024-09-11T01:06:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.003+0000 7f637d7fa640 1 -- 172.21.15.78:0/252736340 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6368004e90 con 0x7f63801087d0 2024-09-11T01:06:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.003+0000 7f6387e82740 1 Processor -- start 2024-09-11T01:06:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.003+0000 7f6387e82740 1 -- start start 2024-09-11T01:06:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.003+0000 7f6387e82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f70a0 0x5601641f7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.003+0000 7f6387e82740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601641f79c0 0x560164203410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.003+0000 7f6387e82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560164203950 0x560164205d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.003+0000 7f6387e82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56016416de50 con 0x560164203950 2024-09-11T01:06:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.003+0000 7f6387e82740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56016415d280 con 0x5601641f79c0 2024-09-11T01:06:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.003+0000 7f6387e82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560164161090 con 0x5601641f70a0 2024-09-11T01:06:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f6377fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560164203950 0x560164205d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f637ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f70a0 0x5601641f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f6377fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560164203950 0x560164205d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43742/0 (socket says 172.21.15.78:43742) 2024-09-11T01:06:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f6377fff640 1 -- 172.21.15.78:0/900708180 learned_addr learned my addr 172.21.15.78:0/900708180 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f637f7fe640 1 --2- 172.21.15.78:0/900708180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601641f79c0 0x560164203410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f637ffff640 1 -- 172.21.15.78:0/900708180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601641f79c0 msgr2=0x560164203410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f637ffff640 1 --2- 172.21.15.78:0/900708180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601641f79c0 0x560164203410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f637ffff640 1 -- 172.21.15.78:0/900708180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560164203950 msgr2=0x560164205d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f637ffff640 1 --2- 172.21.15.78:0/900708180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560164203950 0x560164205d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f637ffff640 1 -- 172.21.15.78:0/900708180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560163ebb650 con 0x5601641f70a0 2024-09-11T01:06:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f637ffff640 1 --2- 172.21.15.78:0/900708180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f70a0 0x5601641f7480 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6378005fd0 tx=0x7f6378011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=832b68782103ecc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f637d7fa640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f636c0b54d0 0x7f636c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f6375ffb640 1 -- 172.21.15.78:0/900708180 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378004d20 con 0x5601641f70a0 2024-09-11T01:06:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f6375ffb640 1 -- 172.21.15.78:0/900708180 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f637800f9a0 con 0x5601641f70a0 2024-09-11T01:06:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.004+0000 7f6375ffb640 1 -- 172.21.15.78:0/900708180 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63780043d0 con 0x5601641f70a0 2024-09-11T01:06:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.005+0000 7f637f7fe640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f636c0b54d0 0x7f636c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.005+0000 7f637d7fa640 1 -- 172.21.15.78:0/252736340 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f63680e6380 con 0x7f63801087d0 2024-09-11T01:06:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.005+0000 7f6387e82740 1 -- 172.21.15.78:0/900708180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f70a0 msgr2=0x5601641f7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.005+0000 7f637f7fe640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f636c0b54d0 0x7f636c0b7990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6370031440 tx=0x7f6370005b90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.005+0000 7f6387e82740 1 --2- 172.21.15.78:0/900708180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f70a0 0x5601641f7480 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6378005fd0 tx=0x7f6378011bf0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.005+0000 7f6387e82740 1 -- 172.21.15.78:0/900708180 shutdown_connections 2024-09-11T01:06:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.005+0000 7f6387e82740 1 --2- 172.21.15.78:0/900708180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560164203950 0x560164205d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.005+0000 7f6387e82740 1 --2- 172.21.15.78:0/900708180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5601641f79c0 0x560164203410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.005+0000 7f6387e82740 1 --2- 172.21.15.78:0/900708180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f70a0 0x5601641f7480 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.005+0000 7f6387e82740 1 -- 172.21.15.78:0/900708180 >> 172.21.15.78:0/900708180 conn(0x5601641f4ed0 msgr2=0x5601641f6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.005+0000 7f6387e82740 1 -- 172.21.15.78:0/900708180 shutdown_connections 2024-09-11T01:06:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.005+0000 7f6387e82740 1 -- 172.21.15.78:0/900708180 wait complete. 2024-09-11T01:06:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f6387e82740 1 -- start start 2024-09-11T01:06:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f6387e82740 1 Processor -- start 2024-09-11T01:06:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f6387e82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f70a0 0x5601642031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f6387e82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f79c0 0x560164200630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f6387e82740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 0x560164200bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f6387e82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560163fbe700 con 0x5601641f79c0 2024-09-11T01:06:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f6387e82740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5601640f8110 con 0x560164203950 2024-09-11T01:06:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f6387e82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5601642065d0 con 0x5601641f70a0 2024-09-11T01:06:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f6387e82740 10 client.0 _ll_register_callbacks cb 0x560164111990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f6377fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f79c0 0x560164200630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f637f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f70a0 0x5601642031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f6377fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f79c0 0x560164200630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43758/0 (socket says 172.21.15.78:43758) 2024-09-11T01:06:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f6377fff640 1 -- 172.21.15.78:0/4239406472 learned_addr learned my addr 172.21.15.78:0/4239406472 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.006+0000 7f637ffff640 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 0x560164200bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6377fff640 1 -- 172.21.15.78:0/4239406472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f70a0 msgr2=0x5601642031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6377fff640 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f70a0 0x5601642031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6377fff640 1 -- 172.21.15.78:0/4239406472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 msgr2=0x560164200bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6377fff640 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 0x560164200bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6377fff640 1 -- 172.21.15.78:0/4239406472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560164125c40 con 0x5601641f79c0 2024-09-11T01:06:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f637f7fe640 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f70a0 0x5601642031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6377fff640 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f79c0 0x560164200630 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f63680a9ab0 tx=0x7f63680e9ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656dc1be7be02cda server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f635effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f635effd640 1 -- 172.21.15.78:0/4239406472 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63680a8e90 con 0x5601641f79c0 2024-09-11T01:06:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f635effd640 1 -- 172.21.15.78:0/4239406472 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f63680b5940 con 0x5601641f79c0 2024-09-11T01:06:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6387e82740 1 -- 172.21.15.78:0/4239406472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f637800f9a0 con 0x5601641f79c0 2024-09-11T01:06:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f635effd640 1 -- 172.21.15.78:0/4239406472 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f63680abfb0 con 0x5601641f79c0 2024-09-11T01:06:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6387e82740 10 client.16887 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6387e82740 1 -- 172.21.15.78:0/4239406472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560164211f00 con 0x5601641f79c0 2024-09-11T01:06:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f635effd640 1 -- 172.21.15.78:0/4239406472 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f637800f9a0 con 0x5601641f79c0 2024-09-11T01:06:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f635effd640 1 -- 172.21.15.78:0/4239406472 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f63680a8190 con 0x5601641f79c0 2024-09-11T01:06:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6387e82740 10 client.16887 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6387e82740 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:06:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6387e82740 20 client.16887 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6387e82740 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5601642133b0 0x5601642157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6387e82740 4 client.16887 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:06:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.007+0000 7f6387e82740 1 -- 172.21.15.78:0/4239406472 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56016416fda0 con 0x5601642133b0 2024-09-11T01:06:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.008+0000 7f637f7fe640 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5601642133b0 0x5601642157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.008+0000 7f637f7fe640 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5601642133b0 0x5601642157a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=824f356c333e066d server_cookie=f9499bb3b9f7efe0 in_seq=0 out_seq=0 2024-09-11T01:06:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.008+0000 7f635effd640 10 client.16887 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f635effd640 1 -- 172.21.15.78:0/4239406472 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56016416fda0 con 0x5601642133b0 2024-09-11T01:06:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f635effd640 10 client.16887 handle_command_reply: tid=0 2024-09-11T01:06:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f6387e82740 1 client.16887 shutdown 2024-09-11T01:06:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f6387e82740 1 -- 172.21.15.78:0/4239406472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f79c0 msgr2=0x560164200630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f6387e82740 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f79c0 0x560164200630 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f63680a9ab0 tx=0x7f63680e9ac0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f6387e82740 1 -- 172.21.15.78:0/4239406472 shutdown_connections 2024-09-11T01:06:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f6387e82740 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 0x560164200bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f6387e82740 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5601642133b0 0x5601642157a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f6387e82740 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f79c0 0x560164200630 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f6387e82740 1 --2- 172.21.15.78:0/4239406472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f70a0 0x5601642031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f6387e82740 1 -- 172.21.15.78:0/4239406472 >> 172.21.15.78:0/4239406472 conn(0x5601641f4ed0 msgr2=0x5601641faa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f635effd640 0 client.16887 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:33.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f6387e82740 1 -- 172.21.15.78:0/4239406472 shutdown_connections 2024-09-11T01:06:33.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f6387e82740 1 -- 172.21.15.78:0/4239406472 wait complete. 2024-09-11T01:06:33.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.009+0000 7f6387e82740 20 client.16887 trim_cache size 0 max 16384 2024-09-11T01:06:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f6387e82740 1 Processor -- start 2024-09-11T01:06:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f6387e82740 1 -- start start 2024-09-11T01:06:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f6387e82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 0x5601641f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f6387e82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f79c0 0x560164204d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f6387e82740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 0x560164205280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f6387e82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63801ae620 con 0x5601641f70a0 2024-09-11T01:06:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f6387e82740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f63801b01f0 con 0x560164203950 2024-09-11T01:06:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f6387e82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63801aecf0 con 0x5601641f79c0 2024-09-11T01:06:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f637f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 0x5601641f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f637f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 0x5601641f5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43772/0 (socket says 172.21.15.78:43772) 2024-09-11T01:06:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f637f7fe640 1 -- 172.21.15.78:0/314448597 learned_addr learned my addr 172.21.15.78:0/314448597 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f6377fff640 1 --2- 172.21.15.78:0/314448597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f79c0 0x560164204d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f637ffff640 1 --2- 172.21.15.78:0/314448597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 0x560164205280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f637f7fe640 1 -- 172.21.15.78:0/314448597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f79c0 msgr2=0x560164204d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f637f7fe640 1 --2- 172.21.15.78:0/314448597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f79c0 0x560164204d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f637f7fe640 1 -- 172.21.15.78:0/314448597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 msgr2=0x560164205280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f637f7fe640 1 --2- 172.21.15.78:0/314448597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 0x560164205280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f637f7fe640 1 -- 172.21.15.78:0/314448597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601641f0570 con 0x5601641f70a0 2024-09-11T01:06:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.027+0000 7f637f7fe640 1 --2- 172.21.15.78:0/314448597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 0x5601641f5f90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6370009900 tx=0x7f637003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0880036df6f9b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.028+0000 7f63757fa640 1 -- 172.21.15.78:0/314448597 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370035040 con 0x5601641f70a0 2024-09-11T01:06:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.028+0000 7f63757fa640 1 -- 172.21.15.78:0/314448597 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6370056030 con 0x5601641f70a0 2024-09-11T01:06:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.028+0000 7f63757fa640 1 -- 172.21.15.78:0/314448597 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370004ad0 con 0x5601641f70a0 2024-09-11T01:06:33.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.028+0000 7f6387e82740 1 -- 172.21.15.78:0/314448597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 msgr2=0x5601641f5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:33.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.028+0000 7f6387e82740 1 --2- 172.21.15.78:0/314448597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 0x5601641f5f90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6370009900 tx=0x7f637003f000 comp rx=0 tx=0).stop 2024-09-11T01:06:33.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.028+0000 7f6387e82740 1 -- 172.21.15.78:0/314448597 shutdown_connections 2024-09-11T01:06:33.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.028+0000 7f6387e82740 1 --2- 172.21.15.78:0/314448597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 0x560164205280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.028+0000 7f6387e82740 1 --2- 172.21.15.78:0/314448597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f79c0 0x560164204d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.028+0000 7f6387e82740 1 --2- 172.21.15.78:0/314448597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 0x5601641f5f90 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.028+0000 7f6387e82740 1 -- 172.21.15.78:0/314448597 >> 172.21.15.78:0/314448597 conn(0x5601641f4ed0 msgr2=0x5601641fb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:33.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.028+0000 7f6387e82740 1 -- 172.21.15.78:0/314448597 shutdown_connections 2024-09-11T01:06:33.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.028+0000 7f6387e82740 1 -- 172.21.15.78:0/314448597 wait complete. 2024-09-11T01:06:33.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f6387e82740 1 -- start start 2024-09-11T01:06:33.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f6387e82740 1 Processor -- start 2024-09-11T01:06:33.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f6387e82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 0x56016427d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f6387e82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f79c0 0x56016427d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f6387e82740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 0x56016427dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f6387e82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63801afaf0 con 0x5601641f70a0 2024-09-11T01:06:33.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f6387e82740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f63801af420 con 0x560164203950 2024-09-11T01:06:33.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f6387e82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63801ac620 con 0x5601641f79c0 2024-09-11T01:06:33.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f6387e82740 10 client.0 _ll_register_callbacks cb 0x7f636800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:33.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f637f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 0x56016427d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f637f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 0x56016427d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43774/0 (socket says 172.21.15.78:43774) 2024-09-11T01:06:33.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f637f7fe640 1 -- 172.21.15.78:0/4018986984 learned_addr learned my addr 172.21.15.78:0/4018986984 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:33.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.029+0000 7f637ffff640 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 0x56016427dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f6377fff640 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f79c0 0x56016427d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f637f7fe640 1 -- 172.21.15.78:0/4018986984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f79c0 msgr2=0x56016427d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f637f7fe640 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f79c0 0x56016427d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f637f7fe640 1 -- 172.21.15.78:0/4018986984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 msgr2=0x56016427dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f637f7fe640 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 0x56016427dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f637f7fe640 1 -- 172.21.15.78:0/4018986984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56016427a310 con 0x5601641f70a0 2024-09-11T01:06:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f637f7fe640 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 0x56016427d290 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6370031730 tx=0x7f63700035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398431b945a90a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f635effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f635effd640 1 -- 172.21.15.78:0/4018986984 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370003a70 con 0x5601641f70a0 2024-09-11T01:06:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f635effd640 1 -- 172.21.15.78:0/4018986984 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f6370003c10 con 0x5601641f70a0 2024-09-11T01:06:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f6387e82740 1 -- 172.21.15.78:0/4018986984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6370056030 con 0x5601641f70a0 2024-09-11T01:06:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f635effd640 1 -- 172.21.15.78:0/4018986984 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6370003e10 con 0x5601641f70a0 2024-09-11T01:06:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f6387e82740 10 client.16893 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f6387e82740 1 -- 172.21.15.78:0/4018986984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56016420ac00 con 0x5601641f70a0 2024-09-11T01:06:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.030+0000 7f635effd640 1 -- 172.21.15.78:0/4018986984 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6370056030 con 0x5601641f70a0 2024-09-11T01:06:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f635effd640 1 -- 172.21.15.78:0/4018986984 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f63680051b0 con 0x5601641f70a0 2024-09-11T01:06:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f6387e82740 10 client.16893 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f6387e82740 10 client.16893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:06:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f6387e82740 20 client.16893 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:33.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f6387e82740 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56016420b420 0x56016420b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:33.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f6387e82740 4 client.16893 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:06:33.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f6387e82740 1 -- 172.21.15.78:0/4018986984 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56016416b850 con 0x56016420b420 2024-09-11T01:06:33.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f6377fff640 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56016420b420 0x56016420b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:33.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f635effd640 10 client.16893 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:33.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f6377fff640 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56016420b420 0x56016420b830 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f765967aff624075 server_cookie=21fd0ee751c729ef in_seq=0 out_seq=0 2024-09-11T01:06:33.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f635effd640 1 -- 172.21.15.78:0/4018986984 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x56016416b850 con 0x56016420b420 2024-09-11T01:06:33.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f635effd640 10 client.16893 handle_command_reply: tid=0 2024-09-11T01:06:33.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.031+0000 7f6387e82740 1 client.16893 shutdown 2024-09-11T01:06:33.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6387e82740 1 -- 172.21.15.78:0/4018986984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 msgr2=0x56016427d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:33.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6387e82740 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 0x56016427d290 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6370031730 tx=0x7f63700035f0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6387e82740 1 -- 172.21.15.78:0/4018986984 shutdown_connections 2024-09-11T01:06:33.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6387e82740 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560164203950 0x56016427dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6387e82740 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601641f79c0 0x56016427d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6387e82740 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56016420b420 0x56016420b830 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6387e82740 1 --2- 172.21.15.78:0/4018986984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601641f70a0 0x56016427d290 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6387e82740 1 -- 172.21.15.78:0/4018986984 >> 172.21.15.78:0/4018986984 conn(0x5601641f4ed0 msgr2=0x560164201c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:33.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f635effd640 0 client.16893 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:33.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6387e82740 1 -- 172.21.15.78:0/4018986984 shutdown_connections 2024-09-11T01:06:33.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6387e82740 1 -- 172.21.15.78:0/4018986984 wait complete. 2024-09-11T01:06:33.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6387e82740 20 client.16893 trim_cache size 0 max 16384 2024-09-11T01:06:33.047 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:06:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:06:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:06:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:06:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:06:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:06:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:06:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:06:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:06:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:06:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:06:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:06:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:06:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:06:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:06:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:06:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:06:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:06:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:06:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:06:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:06:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:06:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:06:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:06:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:06:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:06:33.056 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:06:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6385aad640 1 -- 172.21.15.78:0/252736340 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f636c0b54d0 msgr2=0x7f636c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6385aad640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f636c0b54d0 0x7f636c0b7990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6370031440 tx=0x7f6370005b90 comp rx=0 tx=0).stop 2024-09-11T01:06:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6385aad640 1 -- 172.21.15.78:0/252736340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63801087d0 msgr2=0x7f6380199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6385aad640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63801087d0 0x7f6380199800 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f636800e9f0 tx=0x7f636800edf0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6385aad640 1 -- 172.21.15.78:0/252736340 shutdown_connections 2024-09-11T01:06:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6385aad640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f636c0b54d0 0x7f636c0b7990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6385aad640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6380109150 0x7f638019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6385aad640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63801087d0 0x7f6380199800 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6385aad640 1 --2- 172.21.15.78:0/252736340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6380102dc0 0x7f63801992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6385aad640 1 -- 172.21.15.78:0/252736340 >> 172.21.15.78:0/252736340 conn(0x7f63800fe830 msgr2=0x7f6380100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6385aad640 1 -- 172.21.15.78:0/252736340 shutdown_connections 2024-09-11T01:06:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:33.032+0000 7f6385aad640 1 -- 172.21.15.78:0/252736340 wait complete. 2024-09-11T01:06:33.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 544699 5.58 MB/sec execute 281 sec latency 22.477 ms 2024-09-11T01:06:33.964 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:06:33.965 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:06:34.219 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:06:34.439 INFO:tasks.workunit.client.0.smithi078.stdout: 1 545212 5.57 MB/sec execute 282 sec latency 762.757 ms 2024-09-11T01:06:34.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:34 smithi078 ceph-mon[31315]: pgmap v556: 97 pgs: 97 active+clean; 370 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-09-11T01:06:34.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:34 smithi110 ceph-mon[34712]: pgmap v556: 97 pgs: 97 active+clean; 370 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-09-11T01:06:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:34 smithi112 ceph-mon[35384]: pgmap v556: 97 pgs: 97 active+clean; 370 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-09-11T01:06:34.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.728+0000 7faf5c2ea640 1 -- 172.21.15.78:0/218287470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf54103b50 msgr2=0x7faf54103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:34.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.728+0000 7faf5c2ea640 1 --2- 172.21.15.78:0/218287470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf54103b50 0x7faf54103f30 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7faf50009900 tx=0x7faf5002a410 comp rx=0 tx=0).stop 2024-09-11T01:06:34.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.728+0000 7faf5c2ea640 1 -- 172.21.15.78:0/218287470 shutdown_connections 2024-09-11T01:06:34.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.728+0000 7faf5c2ea640 1 --2- 172.21.15.78:0/218287470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf5410d100 0x7faf5410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:34.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.728+0000 7faf5c2ea640 1 --2- 172.21.15.78:0/218287470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf54104550 0x7faf5410cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:34.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.728+0000 7faf5c2ea640 1 --2- 172.21.15.78:0/218287470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf54103b50 0x7faf54103f30 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:34.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.728+0000 7faf5c2ea640 1 -- 172.21.15.78:0/218287470 >> 172.21.15.78:0/218287470 conn(0x7faf540fd700 msgr2=0x7faf540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:34.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.728+0000 7faf5c2ea640 1 -- 172.21.15.78:0/218287470 shutdown_connections 2024-09-11T01:06:34.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.728+0000 7faf5c2ea640 1 -- 172.21.15.78:0/218287470 wait complete. 2024-09-11T01:06:34.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.729+0000 7faf5c2ea640 1 Processor -- start 2024-09-11T01:06:34.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.729+0000 7faf5c2ea640 1 -- start start 2024-09-11T01:06:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.729+0000 7faf5c2ea640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf54103b50 0x7faf541991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.729+0000 7faf5c2ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf54104550 0x7faf54199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.729+0000 7faf5c2ea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf5410d100 0x7faf5419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.729+0000 7faf5c2ea640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf54112d80 con 0x7faf54104550 2024-09-11T01:06:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.729+0000 7faf5c2ea640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faf54112c00 con 0x7faf54103b50 2024-09-11T01:06:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.729+0000 7faf5c2ea640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf54112f00 con 0x7faf5410d100 2024-09-11T01:06:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.729+0000 7faf59885640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf54104550 0x7faf54199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.729+0000 7faf59885640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf54104550 0x7faf54199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46668/0 (socket says 172.21.15.78:46668) 2024-09-11T01:06:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.729+0000 7faf59885640 1 -- 172.21.15.78:0/3412948534 learned_addr learned my addr 172.21.15.78:0/3412948534 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.729+0000 7faf5a086640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf54103b50 0x7faf541991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf5a887640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf5410d100 0x7faf5419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf59885640 1 -- 172.21.15.78:0/3412948534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf5410d100 msgr2=0x7faf5419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf59885640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf5410d100 0x7faf5419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf59885640 1 -- 172.21.15.78:0/3412948534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf54103b50 msgr2=0x7faf541991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf59885640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf54103b50 0x7faf541991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf59885640 1 -- 172.21.15.78:0/3412948534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5419f2a0 con 0x7faf54104550 2024-09-11T01:06:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf5a887640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf5410d100 0x7faf5419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:06:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf59885640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf54104550 0x7faf54199730 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faf4400c920 tx=0x7faf4400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf4b7fe640 1 -- 172.21.15.78:0/3412948534 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44007c70 con 0x7faf54104550 2024-09-11T01:06:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf4b7fe640 1 -- 172.21.15.78:0/3412948534 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf44007e10 con 0x7faf54104550 2024-09-11T01:06:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf5c2ea640 1 -- 172.21.15.78:0/3412948534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5419f590 con 0x7faf54104550 2024-09-11T01:06:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf4b7fe640 1 -- 172.21.15.78:0/3412948534 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44018820 con 0x7faf54104550 2024-09-11T01:06:34.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.730+0000 7faf5c2ea640 1 -- 172.21.15.78:0/3412948534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf541a5c50 con 0x7faf54104550 2024-09-11T01:06:34.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.731+0000 7faf5c2ea640 1 -- 172.21.15.78:0/3412948534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf54103f30 con 0x7faf54104550 2024-09-11T01:06:34.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.733+0000 7faf4b7fe640 1 -- 172.21.15.78:0/3412948534 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7faf4400f450 con 0x7faf54104550 2024-09-11T01:06:34.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.734+0000 7faf4b7fe640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faf400b5450 0x7faf400b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:34.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.734+0000 7faf5a086640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faf400b5450 0x7faf400b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:34.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.734+0000 7faf4b7fe640 1 -- 172.21.15.78:0/3412948534 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7faf440e6380 con 0x7faf54104550 2024-09-11T01:06:34.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.734+0000 7faf5a086640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faf400b5450 0x7faf400b7910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faf50031440 tx=0x7faf50005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:34.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.737+0000 7faf4b7fe640 1 -- 172.21.15.78:0/3412948534 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7faf44014030 con 0x7faf54104550 2024-09-11T01:06:34.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.948+0000 7faf5c2ea640 1 -- 172.21.15.78:0/3412948534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf5402e5c0 con 0x7faf54104550 2024-09-11T01:06:34.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.949+0000 7faf4b7fe640 1 -- 172.21.15.78:0/3412948534 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11983 (secure 0 0 0) 0x7faf440acfd0 con 0x7faf54104550 2024-09-11T01:06:34.951 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:06:34.951 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":45,"btime":"2024-09-11T01:06:27:244567+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:06:27.108442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T01:06:34.951 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 45 2024-09-11T01:06:34.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.951+0000 7faf5c2ea640 1 -- 172.21.15.78:0/3412948534 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faf400b5450 msgr2=0x7faf400b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:34.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.951+0000 7faf5c2ea640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faf400b5450 0x7faf400b7910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faf50031440 tx=0x7faf50005b50 comp rx=0 tx=0).stop 2024-09-11T01:06:34.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.951+0000 7faf5c2ea640 1 -- 172.21.15.78:0/3412948534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf54104550 msgr2=0x7faf54199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:34.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.951+0000 7faf5c2ea640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf54104550 0x7faf54199730 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faf4400c920 tx=0x7faf4400cde0 comp rx=0 tx=0).stop 2024-09-11T01:06:34.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.951+0000 7faf5c2ea640 1 -- 172.21.15.78:0/3412948534 shutdown_connections 2024-09-11T01:06:34.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.951+0000 7faf5c2ea640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faf400b5450 0x7faf400b7910 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:34.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.951+0000 7faf5c2ea640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf5410d100 0x7faf5419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:34.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.951+0000 7faf5c2ea640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf54104550 0x7faf54199730 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:34.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.951+0000 7faf5c2ea640 1 --2- 172.21.15.78:0/3412948534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf54103b50 0x7faf541991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:34.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.951+0000 7faf5c2ea640 1 -- 172.21.15.78:0/3412948534 >> 172.21.15.78:0/3412948534 conn(0x7faf540fd700 msgr2=0x7faf540fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:34.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.951+0000 7faf5c2ea640 1 -- 172.21.15.78:0/3412948534 shutdown_connections 2024-09-11T01:06:34.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:34.951+0000 7faf5c2ea640 1 -- 172.21.15.78:0/3412948534 wait complete. 2024-09-11T01:06:35.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 545212 5.55 MB/sec execute 283 sec latency 1762.878 ms 2024-09-11T01:06:35.592 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:35 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3412948534' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:06:35.625 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:35 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3412948534' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:06:35.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:35 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3412948534' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:06:35.665 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:06:35.914 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:06:36.301 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:36 smithi078 ceph-mon[31315]: pgmap v557: 97 pgs: 97 active+clean; 370 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 2.7 MiB/s wr, 200 op/s 2024-09-11T01:06:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.378+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2246774371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a41087b0 msgr2=0x7fa3a4108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.378+0000 7fa3ac6d7640 1 --2- 172.21.15.78:0/2246774371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a41087b0 0x7fa3a4108bf0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa394009900 tx=0x7fa39402a410 comp rx=0 tx=0).stop 2024-09-11T01:06:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.378+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2246774371 shutdown_connections 2024-09-11T01:06:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.378+0000 7fa3ac6d7640 1 --2- 172.21.15.78:0/2246774371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3a4109130 0x7fa3a410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.378+0000 7fa3ac6d7640 1 --2- 172.21.15.78:0/2246774371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a41087b0 0x7fa3a4108bf0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.378+0000 7fa3ac6d7640 1 --2- 172.21.15.78:0/2246774371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3a4102da0 0x7fa3a4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.378+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2246774371 >> 172.21.15.78:0/2246774371 conn(0x7fa3a40fe810 msgr2=0x7fa3a4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.378+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2246774371 shutdown_connections 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.378+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2246774371 wait complete. 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.379+0000 7fa3ac6d7640 1 Processor -- start 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.379+0000 7fa3ac6d7640 1 -- start start 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.379+0000 7fa3ac6d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a4102da0 0x7fa3a4199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.379+0000 7fa3ac6d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3a41087b0 0x7fa3a4199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.379+0000 7fa3ac6d7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3a4109130 0x7fa3a419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.379+0000 7fa3ac6d7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4112d80 con 0x7fa3a4102da0 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.379+0000 7fa3ac6d7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4112c00 con 0x7fa3a4109130 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.379+0000 7fa3ac6d7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4112f00 con 0x7fa3a41087b0 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.379+0000 7fa3aa473640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a4102da0 0x7fa3a4199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.379+0000 7fa3aa473640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a4102da0 0x7fa3a4199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46676/0 (socket says 172.21.15.78:46676) 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.379+0000 7fa3aa473640 1 -- 172.21.15.78:0/2646091035 learned_addr learned my addr 172.21.15.78:0/2646091035 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.380+0000 7fa3aa473640 1 -- 172.21.15.78:0/2646091035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3a41087b0 msgr2=0x7fa3a4199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.380+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2646091035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3a41087b0 0x7fa3a4199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.380+0000 7fa3aa473640 1 -- 172.21.15.78:0/2646091035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3a4109130 msgr2=0x7fa3a419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.380+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2646091035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3a4109130 0x7fa3a419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.380+0000 7fa3aa473640 1 -- 172.21.15.78:0/2646091035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a419f2c0 con 0x7fa3a4102da0 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.380+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2646091035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a4102da0 0x7fa3a4199210 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000c920 tx=0x7fa3a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.380+0000 7fa39b7fe640 1 -- 172.21.15.78:0/2646091035 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0007c70 con 0x7fa3a4102da0 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.380+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2646091035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a419f5b0 con 0x7fa3a4102da0 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.380+0000 7fa39b7fe640 1 -- 172.21.15.78:0/2646091035 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a0007e10 con 0x7fa3a4102da0 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.380+0000 7fa39b7fe640 1 -- 172.21.15.78:0/2646091035 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0018820 con 0x7fa3a4102da0 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.380+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2646091035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a41a5c70 con 0x7fa3a4102da0 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.382+0000 7fa3aeaac740 1 Processor -- start 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aeaac740 1 -- start start 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa39b7fe640 1 -- 172.21.15.78:0/2646091035 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa3a000f450 con 0x7fa3a4102da0 2024-09-11T01:06:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aeaac740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee796588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aeaac740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79668c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aeaac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aeaac740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee795ceea0 con 0x55ee79658df0 2024-09-11T01:06:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aeaac740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ee795be2d0 con 0x55ee796584d0 2024-09-11T01:06:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aeaac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ee795c20e0 con 0x55ee79669190 2024-09-11T01:06:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aa473640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79668c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aa473640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79668c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46688/0 (socket says 172.21.15.78:46688) 2024-09-11T01:06:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aa473640 1 -- 172.21.15.78:0/3482709899 learned_addr learned my addr 172.21.15.78:0/3482709899 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aac74640 1 --2- 172.21.15.78:0/3482709899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee796588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3a9c72640 1 --2- 172.21.15.78:0/3482709899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aa473640 1 -- 172.21.15.78:0/3482709899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 msgr2=0x55ee7966b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aa473640 1 --2- 172.21.15.78:0/3482709899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aa473640 1 -- 172.21.15.78:0/3482709899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 msgr2=0x55ee796588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aa473640 1 --2- 172.21.15.78:0/3482709899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee796588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3aa473640 1 -- 172.21.15.78:0/3482709899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee7931c650 con 0x55ee79658df0 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.383+0000 7fa3a9c72640 1 --2- 172.21.15.78:0/3482709899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.384+0000 7fa3aa473640 1 --2- 172.21.15.78:0/3482709899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79668c50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00e38a0 tx=0x7fa3a00ee5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ed3422b64173f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.384+0000 7fa39b7fe640 1 --2- 172.21.15.78:0/2646091035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa3900b5450 0x7fa3900b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.384+0000 7fa398ff9640 1 -- 172.21.15.78:0/3482709899 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00eee10 con 0x55ee79658df0 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.384+0000 7fa398ff9640 1 -- 172.21.15.78:0/3482709899 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a00ac1a0 con 0x55ee79658df0 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.384+0000 7fa398ff9640 1 -- 172.21.15.78:0/3482709899 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00abe00 con 0x55ee79658df0 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.384+0000 7fa3aac74640 1 --2- 172.21.15.78:0/2646091035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa3900b5450 0x7fa3900b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.384+0000 7fa39b7fe640 1 -- 172.21.15.78:0/2646091035 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fa3a00e61d0 con 0x7fa3a4102da0 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 -- 172.21.15.78:0/3482709899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 msgr2=0x55ee79668c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aac74640 1 --2- 172.21.15.78:0/2646091035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa3900b5450 0x7fa3900b7910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa39c007750 tx=0x7fa39c0076e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/3482709899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79668c50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00e38a0 tx=0x7fa3a00ee5e0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 -- 172.21.15.78:0/3482709899 shutdown_connections 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/3482709899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/3482709899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79668c50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/3482709899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee796588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 -- 172.21.15.78:0/3482709899 >> 172.21.15.78:0/3482709899 conn(0x55ee79656300 msgr2=0x55ee796574a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 -- 172.21.15.78:0/3482709899 shutdown_connections 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 -- 172.21.15.78:0/3482709899 wait complete. 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 -- start start 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 Processor -- start 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee79668a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79665e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee79666440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee7941f700 con 0x55ee79658df0 2024-09-11T01:06:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ee79559160 con 0x55ee796584d0 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ee7966be40 con 0x55ee79669190 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3aeaac740 10 client.0 _ll_register_callbacks cb 0x55ee795729e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.385+0000 7fa3a9c72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79665e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3a9c72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79665e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46690/0 (socket says 172.21.15.78:46690) 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3a9c72640 1 -- 172.21.15.78:0/2220644305 learned_addr learned my addr 172.21.15.78:0/2220644305 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee79668a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3aac74640 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee79666440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3a9c72640 1 -- 172.21.15.78:0/2220644305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 msgr2=0x55ee79666440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3a9c72640 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee79666440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3a9c72640 1 -- 172.21.15.78:0/2220644305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 msgr2=0x55ee79668a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3a9c72640 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee79668a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3a9c72640 1 -- 172.21.15.78:0/2220644305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee79586c90 con 0x55ee79658df0 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee79668a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:06:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3a9c72640 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79665e80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa394031750 tx=0x7fa394031780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e86aec31e6f565 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa38dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa38dffb640 1 -- 172.21.15.78:0/2220644305 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394040030 con 0x55ee79658df0 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa38dffb640 1 -- 172.21.15.78:0/2220644305 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fa394004250 con 0x55ee79658df0 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2220644305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a00ac1a0 con 0x55ee79658df0 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa38dffb640 1 -- 172.21.15.78:0/2220644305 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa394035040 con 0x55ee79658df0 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3aeaac740 10 client.16911 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2220644305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ee79677760 con 0x55ee79658df0 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.386+0000 7fa38dffb640 1 -- 172.21.15.78:0/2220644305 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a00ac1a0 con 0x55ee79658df0 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.387+0000 7fa38dffb640 1 -- 172.21.15.78:0/2220644305 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7fa39403a040 con 0x55ee79658df0 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.387+0000 7fa3aeaac740 10 client.16911 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.387+0000 7fa3aeaac740 10 client.16911 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.387+0000 7fa3aeaac740 20 client.16911 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.387+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55ee79678c30 0x55ee7967b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.387+0000 7fa3aeaac740 4 client.16911 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.387+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2220644305 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee795d0df0 con 0x55ee79678c30 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.387+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55ee79678c30 0x55ee7967b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.387+0000 7fa38dffb640 10 client.16911 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.387+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55ee79678c30 0x55ee7967b020 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8744e0f9d90b0aea server_cookie=22b7840a719defcc in_seq=0 out_seq=0 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa38dffb640 1 -- 172.21.15.78:0/2220644305 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ee795d0df0 con 0x55ee79678c30 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa38dffb640 10 client.16911 handle_command_reply: tid=0 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa3aeaac740 1 client.16911 shutdown 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2220644305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 msgr2=0x55ee79665e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79665e80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa394031750 tx=0x7fa394031780 comp rx=0 tx=0).stop 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2220644305 shutdown_connections 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55ee79678c30 0x55ee7967b020 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee79666440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79665e80 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2220644305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee79668a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2220644305 >> 172.21.15.78:0/2220644305 conn(0x55ee79656300 msgr2=0x55ee7965bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa38dffb640 0 client.16911 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2220644305 shutdown_connections 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2220644305 wait complete. 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.388+0000 7fa3aeaac740 20 client.16911 trim_cache size 0 max 16384 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aeaac740 1 Processor -- start 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aeaac740 1 -- start start 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aeaac740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee796584d0 0x55ee796750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aeaac740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee79658df0 0x55ee7966af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aeaac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aeaac740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41ac3d0 con 0x55ee796584d0 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aeaac740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41adfa0 con 0x55ee79658df0 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aeaac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41acaa0 con 0x55ee79669190 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aa473640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee796584d0 0x55ee796750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aa473640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee796584d0 0x55ee796750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46704/0 (socket says 172.21.15.78:46704) 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aa473640 1 -- 172.21.15.78:0/2258325612 learned_addr learned my addr 172.21.15.78:0/2258325612 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3a9c72640 1 --2- 172.21.15.78:0/2258325612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee79658df0 0x55ee7966af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aa473640 1 -- 172.21.15.78:0/2258325612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 msgr2=0x55ee7966b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2258325612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aa473640 1 -- 172.21.15.78:0/2258325612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee79658df0 msgr2=0x55ee7966af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2258325612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee79658df0 0x55ee7966af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aa473640 1 -- 172.21.15.78:0/2258325612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee79667150 con 0x55ee796584d0 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3a9c72640 1 --2- 172.21.15.78:0/2258325612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee79658df0 0x55ee7966af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.401+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2258325612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee796584d0 0x55ee796750a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000c8f0 tx=0x7fa3a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6572f2f3296a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa38ffff640 1 -- 172.21.15.78:0/2258325612 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0077b00 con 0x55ee796584d0 2024-09-11T01:06:36.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa38ffff640 1 -- 172.21.15.78:0/2258325612 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a00ee150 con 0x55ee796584d0 2024-09-11T01:06:36.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa38ffff640 1 -- 172.21.15.78:0/2258325612 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0077d50 con 0x55ee796584d0 2024-09-11T01:06:36.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2258325612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee796584d0 msgr2=0x55ee796750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:36.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2258325612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee796584d0 0x55ee796750a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000c8f0 tx=0x7fa3a0000bc0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2258325612 shutdown_connections 2024-09-11T01:06:36.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2258325612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2258325612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee79658df0 0x55ee7966af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2258325612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee796584d0 0x55ee796750a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2258325612 >> 172.21.15.78:0/2258325612 conn(0x55ee79656300 msgr2=0x55ee796513d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:36.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2258325612 shutdown_connections 2024-09-11T01:06:36.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2258325612 wait complete. 2024-09-11T01:06:36.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.402+0000 7fa3aeaac740 1 -- start start 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3aeaac740 1 Processor -- start 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3aeaac740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee796712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3aeaac740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79671820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3aeaac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3aeaac740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41ad8a0 con 0x55ee79658df0 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3aeaac740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41ad1d0 con 0x55ee796584d0 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3aeaac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41aa3d0 con 0x55ee79669190 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3aeaac740 10 client.0 _ll_register_callbacks cb 0x7fa394009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3a9c72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79671820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3a9c72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79671820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46710/0 (socket says 172.21.15.78:46710) 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3a9c72640 1 -- 172.21.15.78:0/2882944762 learned_addr learned my addr 172.21.15.78:0/2882944762 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee796712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3aac74640 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3a9c72640 1 -- 172.21.15.78:0/2882944762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 msgr2=0x55ee7966e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3a9c72640 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3a9c72640 1 -- 172.21.15.78:0/2882944762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 msgr2=0x55ee796712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3a9c72640 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee796712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3a9c72640 1 -- 172.21.15.78:0/2882944762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee7966e710 con 0x55ee79658df0 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3aac74640 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa3a9c72640 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79671820 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa394006010 tx=0x7fa3940048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86c8722d889ca13 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa38dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.403+0000 7fa38dffb640 1 -- 172.21.15.78:0/2882944762 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394040070 con 0x55ee79658df0 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa38dffb640 1 -- 172.21.15.78:0/2882944762 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fa39404a030 con 0x55ee79658df0 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2882944762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee795cc8a0 con 0x55ee79658df0 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa38dffb640 1 -- 172.21.15.78:0/2882944762 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa394004250 con 0x55ee79658df0 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa3aeaac740 10 client.16917 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2882944762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ee796e63b0 con 0x55ee79658df0 2024-09-11T01:06:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa38dffb640 1 -- 172.21.15.78:0/2882944762 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ee795cc8a0 con 0x55ee79658df0 2024-09-11T01:06:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa38dffb640 1 -- 172.21.15.78:0/2882944762 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7fa3940038e0 con 0x55ee79658df0 2024-09-11T01:06:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa3aeaac740 10 client.16917 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa3aeaac740 10 client.16917 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:06:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa3aeaac740 20 client.16917 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:36.423 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:06:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55ee796e6b60 0x55ee796e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa3aeaac740 4 client.16917 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:06:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2882944762 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee795c9cd0 con 0x55ee796e6b60 2024-09-11T01:06:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55ee796e6b60 0x55ee796e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa38dffb640 10 client.16917 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:06:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.404+0000 7fa3aa473640 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55ee796e6b60 0x55ee796e8f80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=39b0ada995f0adf8 server_cookie=585ec5990a83ca78 in_seq=0 out_seq=0 2024-09-11T01:06:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa38dffb640 1 -- 172.21.15.78:0/2882944762 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee795c9cd0 con 0x55ee796e6b60 2024-09-11T01:06:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa38dffb640 10 client.16917 handle_command_reply: tid=0 2024-09-11T01:06:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3aeaac740 1 client.16917 shutdown 2024-09-11T01:06:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2882944762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 msgr2=0x55ee79671820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79671820 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa394006010 tx=0x7fa3940048f0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2882944762 shutdown_connections 2024-09-11T01:06:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55ee796e6b60 0x55ee796e8f80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee79669190 0x55ee7966e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa38dffb640 0 client.16917 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee79658df0 0x55ee79671820 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3aeaac740 1 --2- 172.21.15.78:0/2882944762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee796584d0 0x55ee796712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2882944762 >> 172.21.15.78:0/2882944762 conn(0x55ee79656300 msgr2=0x55ee7966af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2882944762 shutdown_connections 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3aeaac740 1 -- 172.21.15.78:0/2882944762 wait complete. 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3aeaac740 20 client.16917 trim_cache size 0 max 16384 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2646091035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa3900b5450 msgr2=0x7fa3900b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3ac6d7640 1 --2- 172.21.15.78:0/2646091035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa3900b5450 0x7fa3900b7910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa39c007750 tx=0x7fa39c0076e0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2646091035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a4102da0 msgr2=0x7fa3a4199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3ac6d7640 1 --2- 172.21.15.78:0/2646091035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a4102da0 0x7fa3a4199210 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000c920 tx=0x7fa3a000cde0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2646091035 shutdown_connections 2024-09-11T01:06:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3ac6d7640 1 --2- 172.21.15.78:0/2646091035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa3900b5450 0x7fa3900b7910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3ac6d7640 1 --2- 172.21.15.78:0/2646091035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3a4109130 0x7fa3a419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3ac6d7640 1 --2- 172.21.15.78:0/2646091035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3a41087b0 0x7fa3a4199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3ac6d7640 1 --2- 172.21.15.78:0/2646091035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a4102da0 0x7fa3a4199210 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.405+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2646091035 >> 172.21.15.78:0/2646091035 conn(0x7fa3a40fe810 msgr2=0x7fa3a4100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.406+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2646091035 shutdown_connections 2024-09-11T01:06:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:36.406+0000 7fa3ac6d7640 1 -- 172.21.15.78:0/2646091035 wait complete. 2024-09-11T01:06:36.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 545212 5.53 MB/sec execute 284 sec latency 2762.929 ms 2024-09-11T01:06:36.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:36 smithi110 ceph-mon[34712]: pgmap v557: 97 pgs: 97 active+clean; 370 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 2.7 MiB/s wr, 200 op/s 2024-09-11T01:06:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:36 smithi112 ceph-mon[35384]: pgmap v557: 97 pgs: 97 active+clean; 370 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 2.7 MiB/s wr, 200 op/s 2024-09-11T01:06:37.219 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:06:37.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 545922 5.52 MB/sec execute 285 sec latency 3517.907 ms 2024-09-11T01:06:37.489 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:06:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.110+0000 7f9416637640 1 -- 172.21.15.78:0/2960760625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9410102dc0 msgr2=0x7f94101031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.110+0000 7f9416637640 1 --2- 172.21.15.78:0/2960760625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9410102dc0 0x7f94101031a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9400009900 tx=0x7f940002a410 comp rx=0 tx=0).stop 2024-09-11T01:06:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.110+0000 7f9416637640 1 -- 172.21.15.78:0/2960760625 shutdown_connections 2024-09-11T01:06:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.110+0000 7f9416637640 1 --2- 172.21.15.78:0/2960760625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410109150 0x7f941010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.110+0000 7f9416637640 1 --2- 172.21.15.78:0/2960760625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94101087d0 0x7f9410108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.110+0000 7f9416637640 1 --2- 172.21.15.78:0/2960760625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9410102dc0 0x7f94101031a0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.110+0000 7f9416637640 1 -- 172.21.15.78:0/2960760625 >> 172.21.15.78:0/2960760625 conn(0x7f94100fe830 msgr2=0x7f9410100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.110+0000 7f9416637640 1 -- 172.21.15.78:0/2960760625 shutdown_connections 2024-09-11T01:06:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.110+0000 7f9416637640 1 -- 172.21.15.78:0/2960760625 wait complete. 2024-09-11T01:06:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f9416637640 1 Processor -- start 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f9416637640 1 -- start start 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f9416637640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9410102dc0 0x7f9410199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f9416637640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94101087d0 0x7f94101997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f9416637640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410109150 0x7f941019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f9416637640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9410112e10 con 0x7f9410102dc0 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f9416637640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9410112c90 con 0x7f94101087d0 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f9416637640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9410112f90 con 0x7f9410109150 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f940ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9410102dc0 0x7f9410199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f940ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9410102dc0 0x7f9410199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46724/0 (socket says 172.21.15.78:46724) 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f940ffff640 1 -- 172.21.15.78:0/943689636 learned_addr learned my addr 172.21.15.78:0/943689636 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f940f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94101087d0 0x7f94101997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.111+0000 7f9414bd4640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410109150 0x7f941019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.112+0000 7f940ffff640 1 -- 172.21.15.78:0/943689636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410109150 msgr2=0x7f941019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.112+0000 7f940ffff640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410109150 0x7f941019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.112+0000 7f940ffff640 1 -- 172.21.15.78:0/943689636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94101087d0 msgr2=0x7f94101997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.112+0000 7f940ffff640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94101087d0 0x7f94101997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.112+0000 7f940ffff640 1 -- 172.21.15.78:0/943689636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941019f330 con 0x7f9410102dc0 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.112+0000 7f940ffff640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9410102dc0 0x7f9410199280 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9400005940 tx=0x7f94000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.112+0000 7f940d7fa640 1 -- 172.21.15.78:0/943689636 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940003f030 con 0x7f9410102dc0 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.112+0000 7f940d7fa640 1 -- 172.21.15.78:0/943689636 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9400003cb0 con 0x7f9410102dc0 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.112+0000 7f9416637640 1 -- 172.21.15.78:0/943689636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941019f5c0 con 0x7f9410102dc0 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.112+0000 7f940d7fa640 1 -- 172.21.15.78:0/943689636 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400035070 con 0x7f9410102dc0 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.112+0000 7f9416637640 1 -- 172.21.15.78:0/943689636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94101a5ce0 con 0x7f9410102dc0 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f9418a0c740 1 Processor -- start 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f9418a0c740 1 -- start start 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f9418a0c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef34d0 0x55d373ef38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f9418a0c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef3df0 0x55d373f03c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f9418a0c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373f04190 0x55d373f06580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f9418a0c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d373e69ea0 con 0x55d373ef34d0 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f9418a0c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d373e592d0 con 0x55d373f04190 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f9418a0c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d373e5d0e0 con 0x55d373ef3df0 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f940d7fa640 1 -- 172.21.15.78:0/943689636 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f940003bbf0 con 0x7f9410102dc0 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f9414bd4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef34d0 0x55d373ef38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f940f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373f04190 0x55d373f06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f9414bd4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef34d0 0x55d373ef38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46726/0 (socket says 172.21.15.78:46726) 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f9414bd4640 1 -- 172.21.15.78:0/3465105072 learned_addr learned my addr 172.21.15.78:0/3465105072 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f940ffff640 1 --2- 172.21.15.78:0/3465105072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef3df0 0x55d373f03c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f940f7fe640 1 -- 172.21.15.78:0/3465105072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef3df0 msgr2=0x55d373f03c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f940f7fe640 1 --2- 172.21.15.78:0/3465105072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef3df0 0x55d373f03c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f940f7fe640 1 -- 172.21.15.78:0/3465105072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef34d0 msgr2=0x55d373ef38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f940f7fe640 1 --2- 172.21.15.78:0/3465105072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef34d0 0x55d373ef38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f940f7fe640 1 -- 172.21.15.78:0/3465105072 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d373bb7650 con 0x55d373f04190 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.115+0000 7f9414bd4640 1 --2- 172.21.15.78:0/3465105072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef34d0 0x55d373ef38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.116+0000 7f940f7fe640 1 --2- 172.21.15.78:0/3465105072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373f04190 0x55d373f06580 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f940400bbc0 tx=0x7f94040107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c082dee99de941f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.116+0000 7f93f67fc640 1 -- 172.21.15.78:0/3465105072 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404004030 con 0x55d373f04190 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.116+0000 7f93f67fc640 1 -- 172.21.15.78:0/3465105072 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94040035f0 con 0x55d373f04190 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.116+0000 7f93f67fc640 1 -- 172.21.15.78:0/3465105072 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94040116a0 con 0x55d373f04190 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.116+0000 7f940d7fa640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93f80b5450 0x7f93f80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.116+0000 7f9414bd4640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93f80b5450 0x7f93f80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.116+0000 7f940d7fa640 1 -- 172.21.15.78:0/943689636 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f9400106b10 con 0x7f9410102dc0 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 -- 172.21.15.78:0/3465105072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373f04190 msgr2=0x55d373f06580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9414bd4640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93f80b5450 0x7f93f80b7910 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9408010680 tx=0x7f9408007520 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 --2- 172.21.15.78:0/3465105072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373f04190 0x55d373f06580 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f940400bbc0 tx=0x7f94040107a0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 -- 172.21.15.78:0/3465105072 shutdown_connections 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 --2- 172.21.15.78:0/3465105072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373f04190 0x55d373f06580 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 --2- 172.21.15.78:0/3465105072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef3df0 0x55d373f03c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 --2- 172.21.15.78:0/3465105072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef34d0 0x55d373ef38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 -- 172.21.15.78:0/3465105072 >> 172.21.15.78:0/3465105072 conn(0x55d373ef1300 msgr2=0x55d373ef24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 -- 172.21.15.78:0/3465105072 shutdown_connections 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 -- 172.21.15.78:0/3465105072 wait complete. 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 -- start start 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 Processor -- start 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 0x55d373f03a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef3df0 0x55d373f00ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373f04190 0x55d373f01480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d373cba700 con 0x55d373ef3df0 2024-09-11T01:06:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d373df4160 con 0x55d373f04190 2024-09-11T01:06:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d373f06e40 con 0x55d373ef34d0 2024-09-11T01:06:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.117+0000 7f9418a0c740 10 client.0 _ll_register_callbacks cb 0x55d373e0d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f940ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 0x55d373f03a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f940f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef3df0 0x55d373f00ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f940ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 0x55d373f03a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:40404/0 (socket says 172.21.15.78:40404) 2024-09-11T01:06:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f940f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef3df0 0x55d373f00ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46732/0 (socket says 172.21.15.78:46732) 2024-09-11T01:06:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f9414bd4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373f04190 0x55d373f01480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f940ffff640 1 -- 172.21.15.78:0/2268913693 learned_addr learned my addr 172.21.15.78:0/2268913693 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:38.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f940f7fe640 1 -- 172.21.15.78:0/2268913693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 msgr2=0x55d373f03a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f940f7fe640 1 --2- 172.21.15.78:0/2268913693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 0x55d373f03a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f940f7fe640 1 -- 172.21.15.78:0/2268913693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373f04190 msgr2=0x55d373f01480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f940f7fe640 1 --2- 172.21.15.78:0/2268913693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373f04190 0x55d373f01480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f940f7fe640 1 -- 172.21.15.78:0/2268913693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d373e21c90 con 0x55d373ef3df0 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f940ffff640 1 --2- 172.21.15.78:0/2268913693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 0x55d373f03a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f940f7fe640 1 --2- 172.21.15.78:0/2268913693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef3df0 0x55d373f00ea0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9404003f10 tx=0x7f940400bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d513211e617d995e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f93ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f93ef7fe640 1 -- 172.21.15.78:0/2268913693 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940402a060 con 0x55d373ef3df0 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f9418a0c740 1 -- 172.21.15.78:0/2268913693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94040035f0 con 0x55d373ef3df0 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f93ef7fe640 1 -- 172.21.15.78:0/2268913693 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f94040039b0 con 0x55d373ef3df0 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f93ef7fe640 1 -- 172.21.15.78:0/2268913693 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9404003f70 con 0x55d373ef3df0 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f9418a0c740 10 client.16929 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f9418a0c740 1 -- 172.21.15.78:0/2268913693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d373f127a0 con 0x55d373ef3df0 2024-09-11T01:06:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.118+0000 7f93ef7fe640 1 -- 172.21.15.78:0/2268913693 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94040035f0 con 0x55d373ef3df0 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.119+0000 7f93ef7fe640 1 -- 172.21.15.78:0/2268913693 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f9404004750 con 0x55d373ef3df0 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.119+0000 7f9418a0c740 10 client.16929 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.119+0000 7f9418a0c740 10 client.16929 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.119+0000 7f9418a0c740 20 client.16929 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.119+0000 7f9418a0c740 1 --2- 172.21.15.78:0/2268913693 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d373f13ba0 0x55d373f15f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.119+0000 7f9418a0c740 4 client.16929 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.119+0000 7f9418a0c740 1 -- 172.21.15.78:0/2268913693 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d373e6bdf0 con 0x55d373f13ba0 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.119+0000 7f940ffff640 1 --2- 172.21.15.78:0/2268913693 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d373f13ba0 0x55d373f15f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.119+0000 7f93ef7fe640 10 client.16929 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.119+0000 7f940ffff640 1 --2- 172.21.15.78:0/2268913693 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d373f13ba0 0x55d373f15f90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8c4ff9cf7c2a939 server_cookie=94b8ec301e5339f8 in_seq=0 out_seq=0 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.120+0000 7f93ef7fe640 1 -- 172.21.15.78:0/2268913693 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d373e6bdf0 con 0x55d373f13ba0 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.120+0000 7f93ef7fe640 10 client.16929 handle_command_reply: tid=0 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.120+0000 7f9418a0c740 1 client.16929 shutdown 2024-09-11T01:06:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.120+0000 7f9418a0c740 1 -- 172.21.15.78:0/2268913693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef3df0 msgr2=0x55d373f00ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.120+0000 7f9418a0c740 1 --2- 172.21.15.78:0/2268913693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef3df0 0x55d373f00ea0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9404003f10 tx=0x7f940400bfa0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.120+0000 7f9418a0c740 1 -- 172.21.15.78:0/2268913693 shutdown_connections 2024-09-11T01:06:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.120+0000 7f9418a0c740 1 --2- 172.21.15.78:0/2268913693 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d373f13ba0 0x55d373f15f90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.120+0000 7f9418a0c740 1 --2- 172.21.15.78:0/2268913693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373f04190 0x55d373f01480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.120+0000 7f9418a0c740 1 --2- 172.21.15.78:0/2268913693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373ef3df0 0x55d373f00ea0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.120+0000 7f93ef7fe640 0 client.16929 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:06:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.120+0000 7f9418a0c740 1 --2- 172.21.15.78:0/2268913693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 0x55d373f03a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.120+0000 7f9418a0c740 1 -- 172.21.15.78:0/2268913693 >> 172.21.15.78:0/2268913693 conn(0x55d373ef1300 msgr2=0x55d373ef6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.121+0000 7f9418a0c740 1 -- 172.21.15.78:0/2268913693 shutdown_connections 2024-09-11T01:06:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.121+0000 7f9418a0c740 1 -- 172.21.15.78:0/2268913693 wait complete. 2024-09-11T01:06:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.121+0000 7f9418a0c740 20 client.16929 trim_cache size 0 max 16384 2024-09-11T01:06:38.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f9418a0c740 1 Processor -- start 2024-09-11T01:06:38.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f9418a0c740 1 -- start start 2024-09-11T01:06:38.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f9418a0c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 0x55d373ef6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f9418a0c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373ef3df0 0x55d373ef6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f9418a0c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373f04190 0x55d373ef7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f9418a0c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94101ac390 con 0x55d373f04190 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f9418a0c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f94101adf60 con 0x55d373ef3df0 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f9418a0c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94101aca60 con 0x55d373ef34d0 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f940ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 0x55d373ef6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f9414bd4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373f04190 0x55d373ef7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f940ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 0x55d373ef6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:40418/0 (socket says 172.21.15.78:40418) 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f940f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373ef3df0 0x55d373ef6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.134+0000 7f940ffff640 1 -- 172.21.15.78:0/1498745755 learned_addr learned my addr 172.21.15.78:0/1498745755 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.135+0000 7f940f7fe640 1 -- 172.21.15.78:0/1498745755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 msgr2=0x55d373ef6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.135+0000 7f940f7fe640 1 --2- 172.21.15.78:0/1498745755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 0x55d373ef6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.135+0000 7f940f7fe640 1 -- 172.21.15.78:0/1498745755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373f04190 msgr2=0x55d373ef7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.135+0000 7f940f7fe640 1 --2- 172.21.15.78:0/1498745755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373f04190 0x55d373ef7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.135+0000 7f940f7fe640 1 -- 172.21.15.78:0/1498745755 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d373ef75d0 con 0x55d373ef3df0 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.135+0000 7f940ffff640 1 --2- 172.21.15.78:0/1498745755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 0x55d373ef6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.135+0000 7f940f7fe640 1 --2- 172.21.15.78:0/1498745755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373ef3df0 0x55d373ef6b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9404003ec0 tx=0x7f94040107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dfcfed42189cdef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.135+0000 7f93f5ffb640 1 -- 172.21.15.78:0/1498745755 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940402a070 con 0x55d373ef3df0 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.135+0000 7f93f5ffb640 1 -- 172.21.15.78:0/1498745755 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9404004d10 con 0x55d373ef3df0 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.135+0000 7f93f5ffb640 1 -- 172.21.15.78:0/1498745755 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404004f30 con 0x55d373ef3df0 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.136+0000 7f9418a0c740 1 -- 172.21.15.78:0/1498745755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373ef3df0 msgr2=0x55d373ef6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.136+0000 7f9418a0c740 1 --2- 172.21.15.78:0/1498745755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373ef3df0 0x55d373ef6b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9404003ec0 tx=0x7f94040107a0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.136+0000 7f9418a0c740 1 -- 172.21.15.78:0/1498745755 shutdown_connections 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.136+0000 7f9418a0c740 1 --2- 172.21.15.78:0/1498745755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373f04190 0x55d373ef7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.136+0000 7f9418a0c740 1 --2- 172.21.15.78:0/1498745755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373ef3df0 0x55d373ef6b50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.136+0000 7f9418a0c740 1 --2- 172.21.15.78:0/1498745755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef34d0 0x55d373ef6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.136+0000 7f9418a0c740 1 -- 172.21.15.78:0/1498745755 >> 172.21.15.78:0/1498745755 conn(0x55d373ef1300 msgr2=0x55d373eff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.136+0000 7f9418a0c740 1 -- 172.21.15.78:0/1498745755 shutdown_connections 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.136+0000 7f9418a0c740 1 -- 172.21.15.78:0/1498745755 wait complete. 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.136+0000 7f9418a0c740 1 -- start start 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.136+0000 7f9418a0c740 1 Processor -- start 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f9418a0c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373ef34d0 0x55d373f17860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f9418a0c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef3df0 0x55d373f17de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f9418a0c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373f04190 0x55d373f796e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f9418a0c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94101ad860 con 0x55d373f04190 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f9418a0c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f94101ad190 con 0x55d373ef34d0 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f9418a0c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94101aa3b0 con 0x55d373ef3df0 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f9418a0c740 10 client.0 _ll_register_callbacks cb 0x7f9408007960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f9414bd4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373f04190 0x55d373f796e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f940ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373ef34d0 0x55d373f17860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f9414bd4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373f04190 0x55d373f796e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46752/0 (socket says 172.21.15.78:46752) 2024-09-11T01:06:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f9414bd4640 1 -- 172.21.15.78:0/1682671094 learned_addr learned my addr 172.21.15.78:0/1682671094 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f940f7fe640 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef3df0 0x55d373f17de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f940ffff640 1 -- 172.21.15.78:0/1682671094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef3df0 msgr2=0x55d373f17de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f940ffff640 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef3df0 0x55d373f17de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f940ffff640 1 -- 172.21.15.78:0/1682671094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373f04190 msgr2=0x55d373f796e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f940ffff640 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373f04190 0x55d373f796e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f940ffff640 1 -- 172.21.15.78:0/1682671094 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d373f79df0 con 0x55d373ef34d0 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f9414bd4640 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373f04190 0x55d373f796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f940ffff640 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373ef34d0 0x55d373f17860 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9400031440 tx=0x7f940003b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc83d72895394d7a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.137+0000 7f93ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f93ef7fe640 1 -- 172.21.15.78:0/1682671094 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400049040 con 0x55d373ef34d0 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f9418a0c740 1 -- 172.21.15.78:0/1682671094 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9404004d10 con 0x55d373ef34d0 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f93ef7fe640 1 -- 172.21.15.78:0/1682671094 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f94001044c0 con 0x55d373ef34d0 2024-09-11T01:06:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f93ef7fe640 1 -- 172.21.15.78:0/1682671094 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f94000cc790 con 0x55d373ef34d0 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f9418a0c740 10 client.26219 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f9418a0c740 1 -- 172.21.15.78:0/1682671094 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d373f81190 con 0x55d373ef34d0 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f93ef7fe640 1 -- 172.21.15.78:0/1682671094 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9404004d10 con 0x55d373ef34d0 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f93ef7fe640 1 -- 172.21.15.78:0/1682671094 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f94000b8190 con 0x55d373ef34d0 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f9418a0c740 10 client.26219 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f9418a0c740 10 client.26219 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f9418a0c740 20 client.26219 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f9418a0c740 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d373f81910 0x55d373f83d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f9418a0c740 4 client.26219 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f9418a0c740 1 -- 172.21.15.78:0/1682671094 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d373e678a0 con 0x55d373f81910 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.138+0000 7f940f7fe640 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d373f81910 0x55d373f83d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f93ef7fe640 10 client.26219 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f940f7fe640 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d373f81910 0x55d373f83d30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9063f934e56e466 server_cookie=a8a36bffdf83f22c in_seq=0 out_seq=0 2024-09-11T01:06:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f93ef7fe640 1 -- 172.21.15.78:0/1682671094 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d373e678a0 con 0x55d373f81910 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f93ef7fe640 10 client.26219 handle_command_reply: tid=0 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f9418a0c740 1 client.26219 shutdown 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f9418a0c740 1 -- 172.21.15.78:0/1682671094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373ef34d0 msgr2=0x55d373f17860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f9418a0c740 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373ef34d0 0x55d373f17860 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9400031440 tx=0x7f940003b6b0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f9418a0c740 1 -- 172.21.15.78:0/1682671094 shutdown_connections 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f9418a0c740 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d373f81910 0x55d373f83d30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f9418a0c740 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d373f04190 0x55d373f796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f93ef7fe640 0 client.26219 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f9418a0c740 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d373ef3df0 0x55d373f17de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f9418a0c740 1 --2- 172.21.15.78:0/1682671094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d373ef34d0 0x55d373f17860 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f9418a0c740 1 -- 172.21.15.78:0/1682671094 >> 172.21.15.78:0/1682671094 conn(0x55d373ef1300 msgr2=0x55d373ef6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f9418a0c740 1 -- 172.21.15.78:0/1682671094 shutdown_connections 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f9418a0c740 1 -- 172.21.15.78:0/1682671094 wait complete. 2024-09-11T01:06:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.139+0000 7f9418a0c740 20 client.26219 trim_cache size 0 max 16384 2024-09-11T01:06:38.152 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:06:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.140+0000 7f9416637640 1 -- 172.21.15.78:0/943689636 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93f80b5450 msgr2=0x7f93f80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.140+0000 7f9416637640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93f80b5450 0x7f93f80b7910 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9408010680 tx=0x7f9408007520 comp rx=0 tx=0).stop 2024-09-11T01:06:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.140+0000 7f9416637640 1 -- 172.21.15.78:0/943689636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9410102dc0 msgr2=0x7f9410199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.140+0000 7f9416637640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9410102dc0 0x7f9410199280 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9400005940 tx=0x7f94000042f0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.140+0000 7f9416637640 1 -- 172.21.15.78:0/943689636 shutdown_connections 2024-09-11T01:06:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.140+0000 7f9416637640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93f80b5450 0x7f93f80b7910 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.140+0000 7f9416637640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410109150 0x7f941019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.140+0000 7f9416637640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94101087d0 0x7f94101997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.140+0000 7f9416637640 1 --2- 172.21.15.78:0/943689636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9410102dc0 0x7f9410199280 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.140+0000 7f9416637640 1 -- 172.21.15.78:0/943689636 >> 172.21.15.78:0/943689636 conn(0x7f94100fe830 msgr2=0x7f94101000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.140+0000 7f9416637640 1 -- 172.21.15.78:0/943689636 shutdown_connections 2024-09-11T01:06:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:38.141+0000 7f9416637640 1 -- 172.21.15.78:0/943689636 wait complete. 2024-09-11T01:06:38.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 547615 5.54 MB/sec execute 286 sec latency 32.759 ms 2024-09-11T01:06:38.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:38 smithi078 ceph-mon[31315]: pgmap v558: 97 pgs: 97 active+clean; 370 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 2.2 MiB/s wr, 165 op/s 2024-09-11T01:06:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:38 smithi110 ceph-mon[34712]: pgmap v558: 97 pgs: 97 active+clean; 370 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 2.2 MiB/s wr, 165 op/s 2024-09-11T01:06:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:38 smithi112 ceph-mon[35384]: pgmap v558: 97 pgs: 97 active+clean; 370 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 2.2 MiB/s wr, 165 op/s 2024-09-11T01:06:38.830 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:06:39.091 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:06:39.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 548762 5.54 MB/sec execute 287 sec latency 398.151 ms 2024-09-11T01:06:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.578+0000 7fa477361640 1 -- 172.21.15.78:0/319416487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470102dc0 msgr2=0x7fa4701031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.578+0000 7fa477361640 1 --2- 172.21.15.78:0/319416487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470102dc0 0x7fa4701031a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009900 tx=0x7fa46c02a410 comp rx=0 tx=0).stop 2024-09-11T01:06:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.578+0000 7fa477361640 1 -- 172.21.15.78:0/319416487 shutdown_connections 2024-09-11T01:06:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.578+0000 7fa477361640 1 --2- 172.21.15.78:0/319416487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa470109150 0x7fa47010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.578+0000 7fa477361640 1 --2- 172.21.15.78:0/319416487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4701087d0 0x7fa470108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.578+0000 7fa477361640 1 --2- 172.21.15.78:0/319416487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470102dc0 0x7fa4701031a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.578+0000 7fa477361640 1 -- 172.21.15.78:0/319416487 >> 172.21.15.78:0/319416487 conn(0x7fa4700fe830 msgr2=0x7fa470100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.578+0000 7fa477361640 1 -- 172.21.15.78:0/319416487 shutdown_connections 2024-09-11T01:06:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.578+0000 7fa477361640 1 -- 172.21.15.78:0/319416487 wait complete. 2024-09-11T01:06:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa477361640 1 Processor -- start 2024-09-11T01:06:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa477361640 1 -- start start 2024-09-11T01:06:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa477361640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470102dc0 0x7fa470199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa477361640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4701087d0 0x7fa470199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa477361640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa470109150 0x7fa47019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa477361640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa470112dd0 con 0x7fa470102dc0 2024-09-11T01:06:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa477361640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa470112c50 con 0x7fa4701087d0 2024-09-11T01:06:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa477361640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa470112f50 con 0x7fa470109150 2024-09-11T01:06:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa4750fd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470102dc0 0x7fa470199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa4750fd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470102dc0 0x7fa470199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46768/0 (socket says 172.21.15.78:46768) 2024-09-11T01:06:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa4750fd640 1 -- 172.21.15.78:0/86777044 learned_addr learned my addr 172.21.15.78:0/86777044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa4748fc640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4701087d0 0x7fa470199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa4758fe640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa470109150 0x7fa47019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa4750fd640 1 -- 172.21.15.78:0/86777044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa470109150 msgr2=0x7fa47019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa4750fd640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa470109150 0x7fa47019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa4750fd640 1 -- 172.21.15.78:0/86777044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4701087d0 msgr2=0x7fa470199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa4750fd640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4701087d0 0x7fa470199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa4750fd640 1 -- 172.21.15.78:0/86777044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47019f2f0 con 0x7fa470102dc0 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.579+0000 7fa4758fe640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa470109150 0x7fa47019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.580+0000 7fa4750fd640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470102dc0 0x7fa470199240 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa46c005940 tx=0x7fa46c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.580+0000 7fa4667fc640 1 -- 172.21.15.78:0/86777044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c03f030 con 0x7fa470102dc0 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.580+0000 7fa4667fc640 1 -- 172.21.15.78:0/86777044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c003cb0 con 0x7fa470102dc0 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.580+0000 7fa477361640 1 -- 172.21.15.78:0/86777044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47019f580 con 0x7fa470102dc0 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.580+0000 7fa4667fc640 1 -- 172.21.15.78:0/86777044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c035070 con 0x7fa470102dc0 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.580+0000 7fa477361640 1 -- 172.21.15.78:0/86777044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4701a5ca0 con 0x7fa470102dc0 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.582+0000 7fa479736740 1 Processor -- start 2024-09-11T01:06:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.582+0000 7fa479736740 1 -- start start 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.582+0000 7fa4667fc640 1 -- 172.21.15.78:0/86777044 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa46c003800 con 0x7fa470102dc0 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa479736740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 0x564d2830d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa479736740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 0x564d28319460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa479736740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d283199a0 0x564d2831bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa479736740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d28283ea0 con 0x564d2830d0f0 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa479736740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564d282732d0 con 0x564d283199a0 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa479736740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564d282770e0 con 0x564d2830da10 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4758fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 0x564d2830d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4758fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 0x564d2830d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46776/0 (socket says 172.21.15.78:46776) 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4758fe640 1 -- 172.21.15.78:0/778366385 learned_addr learned my addr 172.21.15.78:0/778366385 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4750fd640 1 --2- 172.21.15.78:0/778366385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 0x564d28319460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4748fc640 1 --2- 172.21.15.78:0/778366385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d283199a0 0x564d2831bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4758fe640 1 -- 172.21.15.78:0/778366385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 msgr2=0x564d28319460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4758fe640 1 --2- 172.21.15.78:0/778366385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 0x564d28319460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4758fe640 1 -- 172.21.15.78:0/778366385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d283199a0 msgr2=0x564d2831bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4758fe640 1 --2- 172.21.15.78:0/778366385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d283199a0 0x564d2831bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4758fe640 1 -- 172.21.15.78:0/778366385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d27fd1650 con 0x564d2830d0f0 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4748fc640 1 --2- 172.21.15.78:0/778366385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d283199a0 0x564d2831bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4758fe640 1 --2- 172.21.15.78:0/778366385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 0x564d2830d4d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa468009ea0 tx=0x7fa468009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ba4d7b2581a8c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa4667fc640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa45c0b54d0 0x7fa45c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.583+0000 7fa45b7fe640 1 -- 172.21.15.78:0/778366385 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468004d20 con 0x564d2830d0f0 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa45b7fe640 1 -- 172.21.15.78:0/778366385 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4680054d0 con 0x564d2830d0f0 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa45b7fe640 1 -- 172.21.15.78:0/778366385 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4680117a0 con 0x564d2830d0f0 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa4750fd640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa45c0b54d0 0x7fa45c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa4667fc640 1 -- 172.21.15.78:0/86777044 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fa46c106a90 con 0x7fa470102dc0 2024-09-11T01:06:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa479736740 1 -- 172.21.15.78:0/778366385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 msgr2=0x564d2830d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:39.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa4750fd640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa45c0b54d0 0x7fa45c0b7990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa46c1042d0 tx=0x7fa46c03b5d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:39.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa479736740 1 --2- 172.21.15.78:0/778366385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 0x564d2830d4d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa468009ea0 tx=0x7fa468009f90 comp rx=0 tx=0).stop 2024-09-11T01:06:39.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa479736740 1 -- 172.21.15.78:0/778366385 shutdown_connections 2024-09-11T01:06:39.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa479736740 1 --2- 172.21.15.78:0/778366385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d283199a0 0x564d2831bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa479736740 1 --2- 172.21.15.78:0/778366385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 0x564d28319460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa479736740 1 --2- 172.21.15.78:0/778366385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 0x564d2830d4d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa479736740 1 -- 172.21.15.78:0/778366385 >> 172.21.15.78:0/778366385 conn(0x564d2830af20 msgr2=0x564d2830c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:39.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa479736740 1 -- 172.21.15.78:0/778366385 shutdown_connections 2024-09-11T01:06:39.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.584+0000 7fa479736740 1 -- 172.21.15.78:0/778366385 wait complete. 2024-09-11T01:06:39.622 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:06:39.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.585+0000 7fa479736740 1 -- start start 2024-09-11T01:06:39.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.585+0000 7fa479736740 1 Processor -- start 2024-09-11T01:06:39.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.585+0000 7fa479736740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d2830d0f0 0x564d283192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.585+0000 7fa479736740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830da10 0x564d28316710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.585+0000 7fa479736740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d283199a0 0x564d28316d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.585+0000 7fa479736740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d280d4700 con 0x564d2830da10 2024-09-11T01:06:39.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.585+0000 7fa479736740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564d2820e160 con 0x564d2830d0f0 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.585+0000 7fa479736740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564d2831c620 con 0x564d283199a0 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.585+0000 7fa479736740 10 client.0 _ll_register_callbacks cb 0x564d282279e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.585+0000 7fa4748fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830da10 0x564d28316710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa4748fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830da10 0x564d28316710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46786/0 (socket says 172.21.15.78:46786) 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa4748fc640 1 -- 172.21.15.78:0/1315328662 learned_addr learned my addr 172.21.15.78:0/1315328662 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa4758fe640 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d2830d0f0 0x564d283192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa4758fe640 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d283199a0 0x564d28316d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa4748fc640 1 -- 172.21.15.78:0/1315328662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d283199a0 msgr2=0x564d28316d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa4748fc640 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d283199a0 0x564d28316d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa4748fc640 1 -- 172.21.15.78:0/1315328662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d2830d0f0 msgr2=0x564d283192c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa4748fc640 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d2830d0f0 0x564d283192c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa4748fc640 1 -- 172.21.15.78:0/1315328662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d2823bc90 con 0x564d2830da10 2024-09-11T01:06:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa4758fe640 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d2830d0f0 0x564d283192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa4748fc640 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830da10 0x564d28316710 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa460012850 tx=0x7fa460012d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd320685c9d8c10 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa458ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa458ff9640 1 -- 172.21.15.78:0/1315328662 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46001f060 con 0x564d2830da10 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa458ff9640 1 -- 172.21.15.78:0/1315328662 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fa460010dc0 con 0x564d2830da10 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa479736740 1 -- 172.21.15.78:0/1315328662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4680054d0 con 0x564d2830da10 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa458ff9640 1 -- 172.21.15.78:0/1315328662 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa460005020 con 0x564d2830da10 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa479736740 10 client.16947 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa479736740 1 -- 172.21.15.78:0/1315328662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564d28327fe0 con 0x564d2830da10 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.586+0000 7fa458ff9640 1 -- 172.21.15.78:0/1315328662 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4680054d0 con 0x564d2830da10 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.587+0000 7fa458ff9640 1 -- 172.21.15.78:0/1315328662 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7fa46000e3b0 con 0x564d2830da10 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.587+0000 7fa479736740 10 client.16947 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.587+0000 7fa479736740 10 client.16947 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.587+0000 7fa479736740 20 client.16947 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.587+0000 7fa479736740 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564d283294a0 0x564d2832b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.587+0000 7fa479736740 4 client.16947 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.587+0000 7fa479736740 1 -- 172.21.15.78:0/1315328662 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d28285df0 con 0x564d283294a0 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.587+0000 7fa4758fe640 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564d283294a0 0x564d2832b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.587+0000 7fa4758fe640 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564d283294a0 0x564d2832b910 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e3450a6e0661c17 server_cookie=99b4ff308ce9feba in_seq=0 out_seq=0 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.587+0000 7fa458ff9640 10 client.16947 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa458ff9640 1 -- 172.21.15.78:0/1315328662 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564d28285df0 con 0x564d283294a0 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa458ff9640 10 client.16947 handle_command_reply: tid=0 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa479736740 1 client.16947 shutdown 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa479736740 1 -- 172.21.15.78:0/1315328662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830da10 msgr2=0x564d28316710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa479736740 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830da10 0x564d28316710 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa460012850 tx=0x7fa460012d10 comp rx=0 tx=0).stop 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa479736740 1 -- 172.21.15.78:0/1315328662 shutdown_connections 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa479736740 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d283199a0 0x564d28316d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa479736740 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830da10 0x564d28316710 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa479736740 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564d283294a0 0x564d2832b910 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa479736740 1 --2- 172.21.15.78:0/1315328662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d2830d0f0 0x564d283192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa479736740 1 -- 172.21.15.78:0/1315328662 >> 172.21.15.78:0/1315328662 conn(0x564d2830af20 msgr2=0x564d28310a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa458ff9640 0 client.16947 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa479736740 1 -- 172.21.15.78:0/1315328662 shutdown_connections 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa479736740 1 -- 172.21.15.78:0/1315328662 wait complete. 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.588+0000 7fa479736740 20 client.16947 trim_cache size 0 max 16384 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.601+0000 7fa479736740 1 Processor -- start 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.601+0000 7fa479736740 1 -- start start 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.601+0000 7fa479736740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 0x564d28325850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.601+0000 7fa479736740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 0x564d28314f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.601+0000 7fa479736740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d283199a0 0x564d283154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.601+0000 7fa479736740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701ae590 con 0x564d2830d0f0 2024-09-11T01:06:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.601+0000 7fa479736740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701b0160 con 0x564d283199a0 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.601+0000 7fa479736740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701aec60 con 0x564d2830da10 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.601+0000 7fa4758fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 0x564d28325850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.601+0000 7fa4758fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 0x564d28325850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46802/0 (socket says 172.21.15.78:46802) 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.601+0000 7fa4758fe640 1 -- 172.21.15.78:0/3498803584 learned_addr learned my addr 172.21.15.78:0/3498803584 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.602+0000 7fa4748fc640 1 --2- 172.21.15.78:0/3498803584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 0x564d28314f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.602+0000 7fa4758fe640 1 -- 172.21.15.78:0/3498803584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 msgr2=0x564d28314f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.602+0000 7fa4758fe640 1 --2- 172.21.15.78:0/3498803584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 0x564d28314f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.602+0000 7fa4758fe640 1 -- 172.21.15.78:0/3498803584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d283199a0 msgr2=0x564d283154d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.602+0000 7fa4758fe640 1 --2- 172.21.15.78:0/3498803584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d283199a0 0x564d283154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.602+0000 7fa4758fe640 1 -- 172.21.15.78:0/3498803584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d2830b9a0 con 0x564d2830d0f0 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.602+0000 7fa4758fe640 1 --2- 172.21.15.78:0/3498803584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 0x564d28325850 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa468010010 tx=0x7fa468009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a48fc741270c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.602+0000 7fa45affd640 1 -- 172.21.15.78:0/3498803584 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468004bc0 con 0x564d2830d0f0 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.602+0000 7fa45affd640 1 -- 172.21.15.78:0/3498803584 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa468004d60 con 0x564d2830d0f0 2024-09-11T01:06:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.602+0000 7fa45affd640 1 -- 172.21.15.78:0/3498803584 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468011980 con 0x564d2830d0f0 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 -- 172.21.15.78:0/3498803584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 msgr2=0x564d28325850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 --2- 172.21.15.78:0/3498803584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 0x564d28325850 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa468010010 tx=0x7fa468009790 comp rx=0 tx=0).stop 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 -- 172.21.15.78:0/3498803584 shutdown_connections 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 --2- 172.21.15.78:0/3498803584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d283199a0 0x564d283154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 --2- 172.21.15.78:0/3498803584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 0x564d28314f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 --2- 172.21.15.78:0/3498803584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d2830d0f0 0x564d28325850 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 -- 172.21.15.78:0/3498803584 >> 172.21.15.78:0/3498803584 conn(0x564d2830af20 msgr2=0x564d283119b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 -- 172.21.15.78:0/3498803584 shutdown_connections 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 -- 172.21.15.78:0/3498803584 wait complete. 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 -- start start 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 Processor -- start 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d2830d0f0 0x564d28321aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 0x564d28321fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d283199a0 0x564d2831c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701afa60 con 0x564d283199a0 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701af390 con 0x564d2830d0f0 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701ac590 con 0x564d2830da10 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.603+0000 7fa479736740 10 client.0 _ll_register_callbacks cb 0x7fa46000a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa4758fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d283199a0 0x564d2831c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa4758fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d283199a0 0x564d2831c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46814/0 (socket says 172.21.15.78:46814) 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa4758fe640 1 -- 172.21.15.78:0/3300426546 learned_addr learned my addr 172.21.15.78:0/3300426546 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa4748fc640 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 0x564d28321fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa4758fe640 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d2830d0f0 0x564d28321aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa4758fe640 1 -- 172.21.15.78:0/3300426546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 msgr2=0x564d28321fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa4758fe640 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 0x564d28321fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa4758fe640 1 -- 172.21.15.78:0/3300426546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d2830d0f0 msgr2=0x564d28321aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa4758fe640 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d2830d0f0 0x564d28321aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa4758fe640 1 -- 172.21.15.78:0/3300426546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d2831cfb0 con 0x564d283199a0 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa4758fe640 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d283199a0 0x564d2831c870 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa468009c30 tx=0x7fa46800fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c1453ada4d482d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa458ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa458ff9640 1 -- 172.21.15.78:0/3300426546 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468011e60 con 0x564d283199a0 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa458ff9640 1 -- 172.21.15.78:0/3300426546 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fa468011700 con 0x564d283199a0 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa479736740 1 -- 172.21.15.78:0/3300426546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa468004d60 con 0x564d283199a0 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa458ff9640 1 -- 172.21.15.78:0/3300426546 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa468042050 con 0x564d283199a0 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa479736740 10 client.16953 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.604+0000 7fa479736740 1 -- 172.21.15.78:0/3300426546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564d28396eb0 con 0x564d283199a0 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa458ff9640 1 -- 172.21.15.78:0/3300426546 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa468004d60 con 0x564d283199a0 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa458ff9640 1 -- 172.21.15.78:0/3300426546 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7fa46803a020 con 0x564d283199a0 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa479736740 10 client.16953 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:39.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa479736740 10 client.16953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa479736740 20 client.16953 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa479736740 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564d28397510 0x564d28399930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa479736740 4 client.16953 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa479736740 1 -- 172.21.15.78:0/3300426546 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d282818a0 con 0x564d28397510 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa4748fc640 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564d28397510 0x564d28399930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa4748fc640 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564d28397510 0x564d28399930 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d7ce82530e70db6 server_cookie=883392cc43092a38 in_seq=0 out_seq=0 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa458ff9640 10 client.16953 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa458ff9640 1 -- 172.21.15.78:0/3300426546 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d282818a0 con 0x564d28397510 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa458ff9640 10 client.16953 handle_command_reply: tid=0 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.605+0000 7fa479736740 1 client.16953 shutdown 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa479736740 1 -- 172.21.15.78:0/3300426546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d283199a0 msgr2=0x564d2831c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa479736740 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d283199a0 0x564d2831c870 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa468009c30 tx=0x7fa46800fc70 comp rx=0 tx=0).stop 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa479736740 1 -- 172.21.15.78:0/3300426546 shutdown_connections 2024-09-11T01:06:39.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa479736740 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d283199a0 0x564d2831c870 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa479736740 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d2830da10 0x564d28321fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa479736740 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x564d28397510 0x564d28399930 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa479736740 1 --2- 172.21.15.78:0/3300426546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d2830d0f0 0x564d28321aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa479736740 1 -- 172.21.15.78:0/3300426546 >> 172.21.15.78:0/3300426546 conn(0x564d2830af20 msgr2=0x564d28314ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:39.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa458ff9640 0 client.16953 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:06:39.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa479736740 1 -- 172.21.15.78:0/3300426546 shutdown_connections 2024-09-11T01:06:39.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa479736740 1 -- 172.21.15.78:0/3300426546 wait complete. 2024-09-11T01:06:39.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa479736740 20 client.16953 trim_cache size 0 max 16384 2024-09-11T01:06:39.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa477361640 1 -- 172.21.15.78:0/86777044 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa45c0b54d0 msgr2=0x7fa45c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:39.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa477361640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa45c0b54d0 0x7fa45c0b7990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa46c1042d0 tx=0x7fa46c03b5d0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa477361640 1 -- 172.21.15.78:0/86777044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470102dc0 msgr2=0x7fa470199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:39.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa477361640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470102dc0 0x7fa470199240 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa46c005940 tx=0x7fa46c0042f0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa477361640 1 -- 172.21.15.78:0/86777044 shutdown_connections 2024-09-11T01:06:39.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa477361640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa45c0b54d0 0x7fa45c0b7990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa477361640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa470109150 0x7fa47019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa477361640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4701087d0 0x7fa470199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa477361640 1 --2- 172.21.15.78:0/86777044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470102dc0 0x7fa470199240 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:39.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.606+0000 7fa477361640 1 -- 172.21.15.78:0/86777044 >> 172.21.15.78:0/86777044 conn(0x7fa4700fe830 msgr2=0x7fa470100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:39.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.607+0000 7fa477361640 1 -- 172.21.15.78:0/86777044 shutdown_connections 2024-09-11T01:06:39.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:39.607+0000 7fa477361640 1 -- 172.21.15.78:0/86777044 wait complete. 2024-09-11T01:06:40.427 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:06:40.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 548762 5.52 MB/sec execute 288 sec latency 1398.188 ms 2024-09-11T01:06:40.675 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:06:40.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:40 smithi110 ceph-mon[34712]: pgmap v559: 97 pgs: 97 active+clean; 372 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-09-11T01:06:40.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:40 smithi112 ceph-mon[35384]: pgmap v559: 97 pgs: 97 active+clean; 372 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-09-11T01:06:40.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:40 smithi078 ceph-mon[31315]: pgmap v559: 97 pgs: 97 active+clean; 372 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-09-11T01:06:41.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.096+0000 7fc98aee9640 1 -- 172.21.15.78:0/231180940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984102dc0 msgr2=0x7fc9841031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.096+0000 7fc98aee9640 1 --2- 172.21.15.78:0/231180940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984102dc0 0x7fc9841031a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc980009900 tx=0x7fc98002a410 comp rx=0 tx=0).stop 2024-09-11T01:06:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.096+0000 7fc98aee9640 1 -- 172.21.15.78:0/231180940 shutdown_connections 2024-09-11T01:06:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.096+0000 7fc98aee9640 1 --2- 172.21.15.78:0/231180940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984109150 0x7fc98410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.096+0000 7fc98aee9640 1 --2- 172.21.15.78:0/231180940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9841087d0 0x7fc984108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.096+0000 7fc98aee9640 1 --2- 172.21.15.78:0/231180940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984102dc0 0x7fc9841031a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.096+0000 7fc98aee9640 1 -- 172.21.15.78:0/231180940 >> 172.21.15.78:0/231180940 conn(0x7fc9840fe830 msgr2=0x7fc984100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.096+0000 7fc98aee9640 1 -- 172.21.15.78:0/231180940 shutdown_connections 2024-09-11T01:06:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.096+0000 7fc98aee9640 1 -- 172.21.15.78:0/231180940 wait complete. 2024-09-11T01:06:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.097+0000 7fc98aee9640 1 Processor -- start 2024-09-11T01:06:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.097+0000 7fc98aee9640 1 -- start start 2024-09-11T01:06:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.097+0000 7fc98aee9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102dc0 0x7fc9840754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.097+0000 7fc98aee9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9841087d0 0x7fc9840759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.097+0000 7fc98aee9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984109150 0x7fc984075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.097+0000 7fc98aee9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc984112e50 con 0x7fc984109150 2024-09-11T01:06:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.097+0000 7fc98aee9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc984112cd0 con 0x7fc9841087d0 2024-09-11T01:06:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.097+0000 7fc98aee9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc984112fd0 con 0x7fc984102dc0 2024-09-11T01:06:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.097+0000 7fc989486640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984109150 0x7fc984075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.097+0000 7fc989486640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984109150 0x7fc984075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46844/0 (socket says 172.21.15.78:46844) 2024-09-11T01:06:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.097+0000 7fc989486640 1 -- 172.21.15.78:0/1784948596 learned_addr learned my addr 172.21.15.78:0/1784948596 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc988c85640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102dc0 0x7fc9840754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc97bfff640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9841087d0 0x7fc9840759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc989486640 1 -- 172.21.15.78:0/1784948596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102dc0 msgr2=0x7fc9840754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc989486640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102dc0 0x7fc9840754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc989486640 1 -- 172.21.15.78:0/1784948596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9841087d0 msgr2=0x7fc9840759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc989486640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9841087d0 0x7fc9840759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc989486640 1 -- 172.21.15.78:0/1784948596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9841a7860 con 0x7fc984109150 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc97bfff640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9841087d0 0x7fc9840759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc989486640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984109150 0x7fc984075f20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00b970 tx=0x7fc97c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc979ffb640 1 -- 172.21.15.78:0/1784948596 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c019070 con 0x7fc984109150 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc979ffb640 1 -- 172.21.15.78:0/1784948596 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc97c014440 con 0x7fc984109150 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc98aee9640 1 -- 172.21.15.78:0/1784948596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9841a7b50 con 0x7fc984109150 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc979ffb640 1 -- 172.21.15.78:0/1784948596 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0075d0 con 0x7fc984109150 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.098+0000 7fc98aee9640 1 -- 172.21.15.78:0/1784948596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9841a8040 con 0x7fc984109150 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc979ffb640 1 -- 172.21.15.78:0/1784948596 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc97c00cb40 con 0x7fc984109150 2024-09-11T01:06:41.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc98d2be740 1 Processor -- start 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc98d2be740 1 -- start start 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc98d2be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe5d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc98d2be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe69840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc98d2be740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe6c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc98d2be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637dbdd3ea0 con 0x5637dbe5ddf0 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc98d2be740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5637dbdc32d0 con 0x5637dbe69d80 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc98d2be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5637dbdc70e0 con 0x5637dbe5d4d0 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc988c85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe69840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc988c85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe69840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46858/0 (socket says 172.21.15.78:46858) 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc988c85640 1 -- 172.21.15.78:0/585859019 learned_addr learned my addr 172.21.15.78:0/585859019 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc979ffb640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc96c0b5450 0x7fc96c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc97bfff640 1 --2- 172.21.15.78:0/585859019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe6c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc989486640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc96c0b5450 0x7fc96c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.101+0000 7fc97bfff640 1 --2- 172.21.15.78:0/585859019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe5d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.102+0000 7fc988c85640 1 -- 172.21.15.78:0/585859019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 msgr2=0x5637dbe5d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:41.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.102+0000 7fc988c85640 1 --2- 172.21.15.78:0/585859019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe5d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.102+0000 7fc988c85640 1 -- 172.21.15.78:0/585859019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 msgr2=0x5637dbe6c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:41.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.102+0000 7fc988c85640 1 --2- 172.21.15.78:0/585859019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe6c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.102+0000 7fc988c85640 1 -- 172.21.15.78:0/585859019 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637dbb21650 con 0x5637dbe5ddf0 2024-09-11T01:06:41.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.102+0000 7fc97bfff640 1 --2- 172.21.15.78:0/585859019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe5d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:06:41.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.102+0000 7fc979ffb640 1 -- 172.21.15.78:0/1784948596 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fc97c0e5740 con 0x7fc984109150 2024-09-11T01:06:41.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.102+0000 7fc988c85640 1 --2- 172.21.15.78:0/585859019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe69840 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc980009900 tx=0x7fc980031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bfe47969654c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:41.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.102+0000 7fc989486640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc96c0b5450 0x7fc96c0b7910 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0abc40 tx=0x7fc97c0dc070 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:41.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.102+0000 7fc96affd640 1 -- 172.21.15.78:0/585859019 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980040070 con 0x5637dbe5ddf0 2024-09-11T01:06:41.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.102+0000 7fc96affd640 1 -- 172.21.15.78:0/585859019 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9800316a0 con 0x5637dbe5ddf0 2024-09-11T01:06:41.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.102+0000 7fc96affd640 1 -- 172.21.15.78:0/585859019 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980035040 con 0x5637dbe5ddf0 2024-09-11T01:06:41.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.103+0000 7fc98d2be740 1 -- 172.21.15.78:0/585859019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 msgr2=0x5637dbe69840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:41.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.103+0000 7fc98d2be740 1 --2- 172.21.15.78:0/585859019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe69840 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc980009900 tx=0x7fc980031de0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.103+0000 7fc98d2be740 1 -- 172.21.15.78:0/585859019 shutdown_connections 2024-09-11T01:06:41.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.103+0000 7fc98d2be740 1 --2- 172.21.15.78:0/585859019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe6c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.103+0000 7fc98d2be740 1 --2- 172.21.15.78:0/585859019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe69840 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.103+0000 7fc98d2be740 1 --2- 172.21.15.78:0/585859019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe5d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.103+0000 7fc98d2be740 1 -- 172.21.15.78:0/585859019 >> 172.21.15.78:0/585859019 conn(0x5637dbe5b300 msgr2=0x5637dbe5c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:41.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.103+0000 7fc98d2be740 1 -- 172.21.15.78:0/585859019 shutdown_connections 2024-09-11T01:06:41.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.103+0000 7fc98d2be740 1 -- 172.21.15.78:0/585859019 wait complete. 2024-09-11T01:06:41.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.103+0000 7fc98d2be740 1 -- start start 2024-09-11T01:06:41.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.103+0000 7fc98d2be740 1 Processor -- start 2024-09-11T01:06:41.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc98d2be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5d4d0 0x5637dbe69650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc98d2be740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe5ddf0 0x5637dbe66aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc98d2be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe69d80 0x5637dbe67080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc98d2be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637dbc24700 con 0x5637dbe5d4d0 2024-09-11T01:06:41.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc98d2be740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5637dbd5e160 con 0x5637dbe5ddf0 2024-09-11T01:06:41.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc98d2be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5637dbe6c9a0 con 0x5637dbe69d80 2024-09-11T01:06:41.135 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:06:41.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc98d2be740 10 client.0 _ll_register_callbacks cb 0x5637dbd779e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:41.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc97bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5d4d0 0x5637dbe69650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc97bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5d4d0 0x5637dbe69650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46862/0 (socket says 172.21.15.78:46862) 2024-09-11T01:06:41.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc97bfff640 1 -- 172.21.15.78:0/2221216400 learned_addr learned my addr 172.21.15.78:0/2221216400 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:41.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc988c85640 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe5ddf0 0x5637dbe66aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc97bfff640 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe69d80 0x5637dbe67080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc97bfff640 1 -- 172.21.15.78:0/2221216400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe69d80 msgr2=0x5637dbe67080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:41.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc97bfff640 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe69d80 0x5637dbe67080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc97bfff640 1 -- 172.21.15.78:0/2221216400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe5ddf0 msgr2=0x5637dbe66aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc97bfff640 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe5ddf0 0x5637dbe66aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc97bfff640 1 -- 172.21.15.78:0/2221216400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637dbd8bc90 con 0x5637dbe5d4d0 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc97bfff640 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5d4d0 0x5637dbe69650 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc974016760 tx=0x7fc9740047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55a1c0908677cd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc963fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc963fff640 1 -- 172.21.15.78:0/2221216400 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97401d060 con 0x5637dbe5d4d0 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc963fff640 1 -- 172.21.15.78:0/2221216400 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fc9740117e0 con 0x5637dbe5d4d0 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.104+0000 7fc98d2be740 1 -- 172.21.15.78:0/2221216400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9800316a0 con 0x5637dbe5d4d0 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.105+0000 7fc963fff640 1 -- 172.21.15.78:0/2221216400 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc9740119e0 con 0x5637dbe5d4d0 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.105+0000 7fc98d2be740 10 client.16965 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.105+0000 7fc98d2be740 1 -- 172.21.15.78:0/2221216400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5637dbe782b0 con 0x5637dbe5d4d0 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.105+0000 7fc963fff640 1 -- 172.21.15.78:0/2221216400 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9800316a0 con 0x5637dbe5d4d0 2024-09-11T01:06:41.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.105+0000 7fc963fff640 1 -- 172.21.15.78:0/2221216400 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7fc974018bd0 con 0x5637dbe5d4d0 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.105+0000 7fc98d2be740 10 client.16965 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.105+0000 7fc98d2be740 10 client.16965 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.105+0000 7fc98d2be740 20 client.16965 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.105+0000 7fc98d2be740 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5637dbe79770 0x5637dbe7bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.105+0000 7fc98d2be740 4 client.16965 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.105+0000 7fc98d2be740 1 -- 172.21.15.78:0/2221216400 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637dbdd5df0 con 0x5637dbe79770 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.105+0000 7fc988c85640 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5637dbe79770 0x5637dbe7bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.106+0000 7fc988c85640 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5637dbe79770 0x5637dbe7bb60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cba61ad7ddc3abd2 server_cookie=f72173228d99bf1a in_seq=0 out_seq=0 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.106+0000 7fc963fff640 10 client.16965 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.106+0000 7fc963fff640 1 -- 172.21.15.78:0/2221216400 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5637dbdd5df0 con 0x5637dbe79770 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.106+0000 7fc963fff640 10 client.16965 handle_command_reply: tid=0 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.106+0000 7fc98d2be740 1 client.16965 shutdown 2024-09-11T01:06:41.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.107+0000 7fc98d2be740 1 -- 172.21.15.78:0/2221216400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5d4d0 msgr2=0x5637dbe69650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:41.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.107+0000 7fc98d2be740 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5d4d0 0x5637dbe69650 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc974016760 tx=0x7fc9740047d0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.107+0000 7fc98d2be740 1 -- 172.21.15.78:0/2221216400 shutdown_connections 2024-09-11T01:06:41.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.107+0000 7fc98d2be740 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe69d80 0x5637dbe67080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.107+0000 7fc98d2be740 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe5ddf0 0x5637dbe66aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.107+0000 7fc98d2be740 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5637dbe79770 0x5637dbe7bb60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.107+0000 7fc98d2be740 1 --2- 172.21.15.78:0/2221216400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5d4d0 0x5637dbe69650 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.107+0000 7fc98d2be740 1 -- 172.21.15.78:0/2221216400 >> 172.21.15.78:0/2221216400 conn(0x5637dbe5b300 msgr2=0x5637dbe60ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:41.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.107+0000 7fc963fff640 0 client.16965 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:06:41.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.107+0000 7fc98d2be740 1 -- 172.21.15.78:0/2221216400 shutdown_connections 2024-09-11T01:06:41.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.107+0000 7fc98d2be740 1 -- 172.21.15.78:0/2221216400 wait complete. 2024-09-11T01:06:41.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.107+0000 7fc98d2be740 20 client.16965 trim_cache size 0 max 16384 2024-09-11T01:06:41.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc98d2be740 1 Processor -- start 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc98d2be740 1 -- start start 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc98d2be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe65f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc98d2be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe61cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc98d2be740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc98d2be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841b0940 con 0x5637dbe5ddf0 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc98d2be740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841b2510 con 0x5637dbe69d80 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc98d2be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841b1010 con 0x5637dbe5d4d0 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc988c85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe61cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc988c85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe61cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46878/0 (socket says 172.21.15.78:46878) 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc988c85640 1 -- 172.21.15.78:0/2165559655 learned_addr learned my addr 172.21.15.78:0/2165559655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc97bfff640 1 --2- 172.21.15.78:0/2165559655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc97bfff640 1 --2- 172.21.15.78:0/2165559655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe65f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc988c85640 1 -- 172.21.15.78:0/2165559655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 msgr2=0x5637dbe65f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc988c85640 1 --2- 172.21.15.78:0/2165559655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe65f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc988c85640 1 -- 172.21.15.78:0/2165559655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 msgr2=0x5637dbe621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc988c85640 1 --2- 172.21.15.78:0/2165559655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc988c85640 1 -- 172.21.15.78:0/2165559655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637dbe5bfe0 con 0x5637dbe5ddf0 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc97bfff640 1 --2- 172.21.15.78:0/2165559655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc988c85640 1 --2- 172.21.15.78:0/2165559655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe61cb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x5637dbe5c1e0 tx=0x7fc980031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484ee91bc819c537 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.120+0000 7fc96a7fc640 1 -- 172.21.15.78:0/2165559655 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980040050 con 0x5637dbe5ddf0 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.121+0000 7fc96a7fc640 1 -- 172.21.15.78:0/2165559655 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9800035f0 con 0x5637dbe5ddf0 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.121+0000 7fc96a7fc640 1 -- 172.21.15.78:0/2165559655 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980035040 con 0x5637dbe5ddf0 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.121+0000 7fc98d2be740 1 -- 172.21.15.78:0/2165559655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 msgr2=0x5637dbe61cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.121+0000 7fc98d2be740 1 --2- 172.21.15.78:0/2165559655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe61cb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x5637dbe5c1e0 tx=0x7fc980031400 comp rx=0 tx=0).stop 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.121+0000 7fc98d2be740 1 -- 172.21.15.78:0/2165559655 shutdown_connections 2024-09-11T01:06:41.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.121+0000 7fc98d2be740 1 --2- 172.21.15.78:0/2165559655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.121+0000 7fc98d2be740 1 --2- 172.21.15.78:0/2165559655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe61cb0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.121+0000 7fc98d2be740 1 --2- 172.21.15.78:0/2165559655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe65f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.121+0000 7fc98d2be740 1 -- 172.21.15.78:0/2165559655 >> 172.21.15.78:0/2165559655 conn(0x5637dbe5b300 msgr2=0x5637dbe7b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.121+0000 7fc98d2be740 1 -- 172.21.15.78:0/2165559655 shutdown_connections 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.121+0000 7fc98d2be740 1 -- 172.21.15.78:0/2165559655 wait complete. 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.121+0000 7fc98d2be740 1 -- start start 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc98d2be740 1 Processor -- start 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc98d2be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe64750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc98d2be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe71c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc98d2be740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe72420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc98d2be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841b1e10 con 0x5637dbe5ddf0 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc98d2be740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841b1740 con 0x5637dbe69d80 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc98d2be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841ae940 con 0x5637dbe5d4d0 2024-09-11T01:06:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc98d2be740 10 client.0 _ll_register_callbacks cb 0x7fc97400a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:41.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc988c85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe71c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc988c85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe71c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46886/0 (socket says 172.21.15.78:46886) 2024-09-11T01:06:41.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc988c85640 1 -- 172.21.15.78:0/1153588401 learned_addr learned my addr 172.21.15.78:0/1153588401 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:41.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc97bfff640 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe72420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc97bfff640 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe64750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc988c85640 1 -- 172.21.15.78:0/1153588401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 msgr2=0x5637dbe64750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:41.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc988c85640 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe64750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc988c85640 1 -- 172.21.15.78:0/1153588401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 msgr2=0x5637dbe72420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:41.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc988c85640 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe72420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc988c85640 1 -- 172.21.15.78:0/1153588401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637dbe6cca0 con 0x5637dbe5ddf0 2024-09-11T01:06:41.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.122+0000 7fc97bfff640 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe72420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:06:41.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc988c85640 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe71c90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc980009900 tx=0x7fc980000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f73c35fd47b3da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:41.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc963fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:41.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc963fff640 1 -- 172.21.15.78:0/1153588401 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980035040 con 0x5637dbe5ddf0 2024-09-11T01:06:41.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc963fff640 1 -- 172.21.15.78:0/1153588401 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fc98004b070 con 0x5637dbe5ddf0 2024-09-11T01:06:41.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc98d2be740 1 -- 172.21.15.78:0/1153588401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9800035f0 con 0x5637dbe5ddf0 2024-09-11T01:06:41.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc963fff640 1 -- 172.21.15.78:0/1153588401 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc9800315c0 con 0x5637dbe5ddf0 2024-09-11T01:06:41.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc98d2be740 10 client.16971 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:41.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc98d2be740 1 -- 172.21.15.78:0/1153588401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5637dbee6920 con 0x5637dbe5ddf0 2024-09-11T01:06:41.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc963fff640 1 -- 172.21.15.78:0/1153588401 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9800035f0 con 0x5637dbe5ddf0 2024-09-11T01:06:41.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc963fff640 1 -- 172.21.15.78:0/1153588401 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7fc980055030 con 0x5637dbe5ddf0 2024-09-11T01:06:41.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc98d2be740 10 client.16971 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:41.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc98d2be740 10 client.16971 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:06:41.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc98d2be740 20 client.16971 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc98d2be740 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5637dbee6b90 0x5637dbee8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc98d2be740 4 client.16971 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.123+0000 7fc98d2be740 1 -- 172.21.15.78:0/1153588401 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637dbdd18a0 con 0x5637dbee6b90 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc97bfff640 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5637dbee6b90 0x5637dbee8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc97bfff640 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5637dbee6b90 0x5637dbee8f80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74ead8fbf8ed6d04 server_cookie=810d32cd8ed227d2 in_seq=0 out_seq=0 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc963fff640 10 client.16971 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc963fff640 1 -- 172.21.15.78:0/1153588401 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637dbdd18a0 con 0x5637dbee6b90 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc963fff640 10 client.16971 handle_command_reply: tid=0 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc98d2be740 1 client.16971 shutdown 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc98d2be740 1 -- 172.21.15.78:0/1153588401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 msgr2=0x5637dbe71c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc98d2be740 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe71c90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc980009900 tx=0x7fc980000bc0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc98d2be740 1 -- 172.21.15.78:0/1153588401 shutdown_connections 2024-09-11T01:06:41.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc98d2be740 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637dbe69d80 0x5637dbe72420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc98d2be740 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5637dbee6b90 0x5637dbee8f80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc963fff640 0 client.16971 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc98d2be740 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637dbe5ddf0 0x5637dbe71c90 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc98d2be740 1 --2- 172.21.15.78:0/1153588401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637dbe5d4d0 0x5637dbe64750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc98d2be740 1 -- 172.21.15.78:0/1153588401 >> 172.21.15.78:0/1153588401 conn(0x5637dbe5b300 msgr2=0x5637dbe61cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc98d2be740 1 -- 172.21.15.78:0/1153588401 shutdown_connections 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.124+0000 7fc98d2be740 1 -- 172.21.15.78:0/1153588401 wait complete. 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98d2be740 20 client.16971 trim_cache size 0 max 16384 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98aee9640 1 -- 172.21.15.78:0/1784948596 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc96c0b5450 msgr2=0x7fc96c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98aee9640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc96c0b5450 0x7fc96c0b7910 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0abc40 tx=0x7fc97c0dc070 comp rx=0 tx=0).stop 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98aee9640 1 -- 172.21.15.78:0/1784948596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984109150 msgr2=0x7fc984075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98aee9640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984109150 0x7fc984075f20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00b970 tx=0x7fc97c00be30 comp rx=0 tx=0).stop 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98aee9640 1 -- 172.21.15.78:0/1784948596 shutdown_connections 2024-09-11T01:06:41.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98aee9640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc96c0b5450 0x7fc96c0b7910 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98aee9640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984109150 0x7fc984075f20 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98aee9640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9841087d0 0x7fc9840759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98aee9640 1 --2- 172.21.15.78:0/1784948596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102dc0 0x7fc9840754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:41.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98aee9640 1 -- 172.21.15.78:0/1784948596 >> 172.21.15.78:0/1784948596 conn(0x7fc9840fe830 msgr2=0x7fc984100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:41.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98aee9640 1 -- 172.21.15.78:0/1784948596 shutdown_connections 2024-09-11T01:06:41.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:41.125+0000 7fc98aee9640 1 -- 172.21.15.78:0/1784948596 wait complete. 2024-09-11T01:06:41.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 548762 5.50 MB/sec execute 289 sec latency 2398.213 ms 2024-09-11T01:06:41.955 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:06:42.265 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:06:42.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 549460 5.49 MB/sec execute 290 sec latency 3165.021 ms 2024-09-11T01:06:42.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:42 smithi078 ceph-mon[31315]: pgmap v560: 97 pgs: 97 active+clean; 373 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-11T01:06:42.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:42 smithi110 ceph-mon[34712]: pgmap v560: 97 pgs: 97 active+clean; 373 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-11T01:06:42.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:42 smithi112 ceph-mon[35384]: pgmap v560: 97 pgs: 97 active+clean; 373 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-11T01:06:42.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.796+0000 7fdb134fe640 1 -- 172.21.15.78:0/951467333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb0c101c90 msgr2=0x7fdb0c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:42.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.796+0000 7fdb134fe640 1 --2- 172.21.15.78:0/951467333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb0c101c90 0x7fdb0c102070 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009900 tx=0x7fdafc02a410 comp rx=0 tx=0).stop 2024-09-11T01:06:42.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.796+0000 7fdb134fe640 1 -- 172.21.15.78:0/951467333 shutdown_connections 2024-09-11T01:06:42.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.796+0000 7fdb134fe640 1 --2- 172.21.15.78:0/951467333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb0c108020 0x7fdb0c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.796+0000 7fdb134fe640 1 --2- 172.21.15.78:0/951467333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb0c1076a0 0x7fdb0c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.796+0000 7fdb134fe640 1 --2- 172.21.15.78:0/951467333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb0c101c90 0x7fdb0c102070 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.796+0000 7fdb134fe640 1 -- 172.21.15.78:0/951467333 >> 172.21.15.78:0/951467333 conn(0x7fdb0c0fd700 msgr2=0x7fdb0c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:42.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.796+0000 7fdb134fe640 1 -- 172.21.15.78:0/951467333 shutdown_connections 2024-09-11T01:06:42.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.796+0000 7fdb134fe640 1 -- 172.21.15.78:0/951467333 wait complete. 2024-09-11T01:06:42.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.796+0000 7fdb134fe640 1 Processor -- start 2024-09-11T01:06:42.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb134fe640 1 -- start start 2024-09-11T01:06:42.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb134fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb0c101c90 0x7fdb0c0fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb134fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb0c1076a0 0x7fdb0c0fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb134fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb0c108020 0x7fdb0c0fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb134fe640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c111ca0 con 0x7fdb0c101c90 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb134fe640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c111b20 con 0x7fdb0c108020 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb134fe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c111e20 con 0x7fdb0c1076a0 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb1129a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb0c101c90 0x7fdb0c0fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb1129a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb0c101c90 0x7fdb0c0fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46900/0 (socket says 172.21.15.78:46900) 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb1129a640 1 -- 172.21.15.78:0/4147837534 learned_addr learned my addr 172.21.15.78:0/4147837534 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb0c108020 0x7fdb0c0fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb10a99640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb0c1076a0 0x7fdb0c0fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb1129a640 1 -- 172.21.15.78:0/4147837534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb0c1076a0 msgr2=0x7fdb0c0fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb1129a640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb0c1076a0 0x7fdb0c0fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb1129a640 1 -- 172.21.15.78:0/4147837534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb0c108020 msgr2=0x7fdb0c0fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb1129a640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb0c108020 0x7fdb0c0fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb1129a640 1 -- 172.21.15.78:0/4147837534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0c1a7650 con 0x7fdb0c101c90 2024-09-11T01:06:42.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.797+0000 7fdb1129a640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb0c101c90 0x7fdb0c0fdc00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fdafc005940 tx=0x7fdafc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:42.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.798+0000 7fdb027fc640 1 -- 172.21.15.78:0/4147837534 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc03f030 con 0x7fdb0c101c90 2024-09-11T01:06:42.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.798+0000 7fdb134fe640 1 -- 172.21.15.78:0/4147837534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0c1a78e0 con 0x7fdb0c101c90 2024-09-11T01:06:42.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.798+0000 7fdb027fc640 1 -- 172.21.15.78:0/4147837534 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdafc003cb0 con 0x7fdb0c101c90 2024-09-11T01:06:42.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.798+0000 7fdb027fc640 1 -- 172.21.15.78:0/4147837534 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc035070 con 0x7fdb0c101c90 2024-09-11T01:06:42.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.798+0000 7fdb134fe640 1 -- 172.21.15.78:0/4147837534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0c1a7e10 con 0x7fdb0c101c90 2024-09-11T01:06:42.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.800+0000 7fdb027fc640 1 -- 172.21.15.78:0/4147837534 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fdafc003800 con 0x7fdb0c101c90 2024-09-11T01:06:42.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.800+0000 7fdb158d3740 1 Processor -- start 2024-09-11T01:06:42.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.800+0000 7fdb158d3740 1 -- start start 2024-09-11T01:06:42.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb158d3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b44d0 0x55939f2b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb158d3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b4df0 0x55939f2c0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb158d3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb158d3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55939f22aea0 con 0x55939f2c0d80 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb158d3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55939f21a2d0 con 0x55939f2b4df0 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb158d3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55939f21e0e0 con 0x55939f2b44d0 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb1129a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b4df0 0x55939f2c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb10a99640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb11a9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b44d0 0x55939f2b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb10a99640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2c3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46906/0 (socket says 172.21.15.78:46906) 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb10a99640 1 -- 172.21.15.78:0/3679381358 learned_addr learned my addr 172.21.15.78:0/3679381358 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb11a9b640 1 -- 172.21.15.78:0/3679381358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b4df0 msgr2=0x55939f2c0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/3679381358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b4df0 0x55939f2c0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb11a9b640 1 -- 172.21.15.78:0/3679381358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 msgr2=0x55939f2c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/3679381358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2c3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb11a9b640 1 -- 172.21.15.78:0/3679381358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55939ef78650 con 0x55939f2b44d0 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb10a99640 1 --2- 172.21.15.78:0/3679381358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb027fc640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdaf80b5450 0x7fdaf80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/3679381358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b44d0 0x55939f2b48b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdb04005fd0 tx=0x7fdb04010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36ffce354c77b7ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.801+0000 7fdb1129a640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdaf80b5450 0x7fdaf80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.802+0000 7fdaf77fe640 1 -- 172.21.15.78:0/3679381358 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04016070 con 0x55939f2b44d0 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.802+0000 7fdb027fc640 1 -- 172.21.15.78:0/4147837534 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fdafc106a40 con 0x7fdb0c101c90 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.802+0000 7fdaf77fe640 1 -- 172.21.15.78:0/3679381358 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb04004d20 con 0x55939f2b44d0 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.802+0000 7fdaf77fe640 1 -- 172.21.15.78:0/3679381358 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb040126b0 con 0x55939f2b44d0 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.802+0000 7fdb158d3740 1 -- 172.21.15.78:0/3679381358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b44d0 msgr2=0x55939f2b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.802+0000 7fdb1129a640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdaf80b5450 0x7fdaf80b7910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdafc104280 tx=0x7fdafc10a590 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:42.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.802+0000 7fdb158d3740 1 --2- 172.21.15.78:0/3679381358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b44d0 0x55939f2b48b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdb04005fd0 tx=0x7fdb04010740 comp rx=0 tx=0).stop 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.802+0000 7fdb158d3740 1 -- 172.21.15.78:0/3679381358 shutdown_connections 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.802+0000 7fdb158d3740 1 --2- 172.21.15.78:0/3679381358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.802+0000 7fdb158d3740 1 --2- 172.21.15.78:0/3679381358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b4df0 0x55939f2c0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.802+0000 7fdb158d3740 1 --2- 172.21.15.78:0/3679381358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b44d0 0x55939f2b48b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.803+0000 7fdb158d3740 1 -- 172.21.15.78:0/3679381358 >> 172.21.15.78:0/3679381358 conn(0x55939f2b2300 msgr2=0x55939f2b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.803+0000 7fdb158d3740 1 -- 172.21.15.78:0/3679381358 shutdown_connections 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.803+0000 7fdb158d3740 1 -- 172.21.15.78:0/3679381358 wait complete. 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.803+0000 7fdb158d3740 1 -- start start 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.803+0000 7fdb158d3740 1 Processor -- start 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb158d3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b44d0 0x55939f2c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb158d3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2bda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb158d3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb158d3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55939f07b700 con 0x55939f2c0d80 2024-09-11T01:06:42.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb158d3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55939f1b5160 con 0x55939f2b44d0 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb158d3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55939f2c3a00 con 0x55939f2b4df0 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb158d3740 10 client.0 _ll_register_callbacks cb 0x55939f1ce9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb11a9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b44d0 0x55939f2c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb10a99640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2bda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb11a9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb10a99640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2bda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:40558/0 (socket says 172.21.15.78:40558) 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb10a99640 1 -- 172.21.15.78:0/404103905 learned_addr learned my addr 172.21.15.78:0/404103905 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb11a9b640 1 -- 172.21.15.78:0/404103905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 msgr2=0x55939f2bda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/404103905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2bda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb11a9b640 1 -- 172.21.15.78:0/404103905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b44d0 msgr2=0x55939f2c05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/404103905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b44d0 0x55939f2c05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb11a9b640 1 -- 172.21.15.78:0/404103905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55939f1e2c90 con 0x55939f2c0d80 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/404103905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2bdfc0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fdb04005fd0 tx=0x7fdb04005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6996eceec11510a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:42.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdaf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdaf4ff9640 1 -- 172.21.15.78:0/404103905 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04016030 con 0x55939f2c0d80 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdaf4ff9640 1 -- 172.21.15.78:0/404103905 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fdb04012b10 con 0x55939f2c0d80 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.804+0000 7fdb158d3740 1 -- 172.21.15.78:0/404103905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb04004d20 con 0x55939f2c0d80 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdaf4ff9640 1 -- 172.21.15.78:0/404103905 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdb04023040 con 0x55939f2c0d80 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdb158d3740 10 client.16983 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdb158d3740 1 -- 172.21.15.78:0/404103905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55939f2cf2e0 con 0x55939f2c0d80 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdaf4ff9640 1 -- 172.21.15.78:0/404103905 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb04004d20 con 0x55939f2c0d80 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdaf4ff9640 1 -- 172.21.15.78:0/404103905 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7fdb0400f040 con 0x55939f2c0d80 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdb158d3740 10 client.16983 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdb158d3740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdb158d3740 20 client.16983 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdb158d3740 1 --2- 172.21.15.78:0/404103905 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55939f2d06f0 0x55939f2d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdb158d3740 4 client.16983 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdb158d3740 1 -- 172.21.15.78:0/404103905 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55939f22cdf0 con 0x55939f2d06f0 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdb10a99640 1 --2- 172.21.15.78:0/404103905 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55939f2d06f0 0x55939f2d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdaf4ff9640 10 client.16983 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.805+0000 7fdb10a99640 1 --2- 172.21.15.78:0/404103905 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55939f2d06f0 0x55939f2d2ae0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d61ce4b5ad66aaba server_cookie=570c1384577bd1b0 in_seq=0 out_seq=0 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdaf4ff9640 1 -- 172.21.15.78:0/404103905 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55939f22cdf0 con 0x55939f2d06f0 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdaf4ff9640 10 client.16983 handle_command_reply: tid=0 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdb158d3740 1 client.16983 shutdown 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdb158d3740 1 -- 172.21.15.78:0/404103905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 msgr2=0x55939f2bdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdb158d3740 1 --2- 172.21.15.78:0/404103905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2bdfc0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fdb04005fd0 tx=0x7fdb04005b00 comp rx=0 tx=0).stop 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdb158d3740 1 -- 172.21.15.78:0/404103905 shutdown_connections 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdb158d3740 1 --2- 172.21.15.78:0/404103905 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55939f2d06f0 0x55939f2d2ae0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdb158d3740 1 --2- 172.21.15.78:0/404103905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2bdfc0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdb158d3740 1 --2- 172.21.15.78:0/404103905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2bda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdaf4ff9640 0 client.16983 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdb158d3740 1 --2- 172.21.15.78:0/404103905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b44d0 0x55939f2c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdb158d3740 1 -- 172.21.15.78:0/404103905 >> 172.21.15.78:0/404103905 conn(0x55939f2b2300 msgr2=0x55939f2b7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.806+0000 7fdb158d3740 1 -- 172.21.15.78:0/404103905 shutdown_connections 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.807+0000 7fdb158d3740 1 -- 172.21.15.78:0/404103905 wait complete. 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.807+0000 7fdb158d3740 20 client.16983 trim_cache size 0 max 16384 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb158d3740 1 Processor -- start 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb158d3740 1 -- start start 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb158d3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b44d0 0x55939f2b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb158d3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2bf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb158d3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2bac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb158d3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c1b0740 con 0x55939f2c0d80 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb158d3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c1b2310 con 0x55939f2b44d0 2024-09-11T01:06:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb158d3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c1b0e10 con 0x55939f2b4df0 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb11a9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2bac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb11a9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b44d0 0x55939f2b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb11a9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b44d0 0x55939f2b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:53722/0 (socket says 172.21.15.78:53722) 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb11a9b640 1 -- 172.21.15.78:0/1630630703 learned_addr learned my addr 172.21.15.78:0/1630630703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb10a99640 1 --2- 172.21.15.78:0/1630630703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2bf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb11a9b640 1 -- 172.21.15.78:0/1630630703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 msgr2=0x55939f2bf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/1630630703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2bf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb11a9b640 1 -- 172.21.15.78:0/1630630703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 msgr2=0x55939f2bac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/1630630703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2bac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.820+0000 7fdb11a9b640 1 -- 172.21.15.78:0/1630630703 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55939f2b4be0 con 0x55939f2b44d0 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.821+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/1630630703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b44d0 0x55939f2b1c10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb040098b0 tx=0x7fdb04005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb4979679c60d1e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.821+0000 7fdaf6ffd640 1 -- 172.21.15.78:0/1630630703 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04016070 con 0x55939f2b44d0 2024-09-11T01:06:42.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.821+0000 7fdaf6ffd640 1 -- 172.21.15.78:0/1630630703 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb0402e020 con 0x55939f2b44d0 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.821+0000 7fdaf6ffd640 1 -- 172.21.15.78:0/1630630703 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0402c2d0 con 0x55939f2b44d0 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.821+0000 7fdb158d3740 1 -- 172.21.15.78:0/1630630703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b44d0 msgr2=0x55939f2b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.821+0000 7fdb158d3740 1 --2- 172.21.15.78:0/1630630703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b44d0 0x55939f2b1c10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb040098b0 tx=0x7fdb04005c40 comp rx=0 tx=0).stop 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.821+0000 7fdb158d3740 1 -- 172.21.15.78:0/1630630703 shutdown_connections 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.821+0000 7fdb158d3740 1 --2- 172.21.15.78:0/1630630703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2c0d80 0x55939f2bac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.821+0000 7fdb158d3740 1 --2- 172.21.15.78:0/1630630703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2bf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.821+0000 7fdb158d3740 1 --2- 172.21.15.78:0/1630630703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2b44d0 0x55939f2b1c10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.821+0000 7fdb158d3740 1 -- 172.21.15.78:0/1630630703 >> 172.21.15.78:0/1630630703 conn(0x55939f2b2300 msgr2=0x55939f2bf070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb158d3740 1 -- 172.21.15.78:0/1630630703 shutdown_connections 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb158d3740 1 -- 172.21.15.78:0/1630630703 wait complete. 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb158d3740 1 -- start start 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb158d3740 1 Processor -- start 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb158d3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2b44d0 0x55939f2c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb158d3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb158d3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2c0d80 0x55939f336470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb158d3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c1b1c10 con 0x55939f2b44d0 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb158d3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c1b1540 con 0x55939f2c0d80 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb158d3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c1ae740 con 0x55939f2b4df0 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb158d3740 10 client.0 _ll_register_callbacks cb 0x7fdb0800a6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb11a9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2b44d0 0x55939f2c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb11a9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2b44d0 0x55939f2c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46930/0 (socket says 172.21.15.78:46930) 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb11a9b640 1 -- 172.21.15.78:0/3326669122 learned_addr learned my addr 172.21.15.78:0/3326669122 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb10a99640 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.822+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2c0d80 0x55939f336470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb11a9b640 1 -- 172.21.15.78:0/3326669122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 msgr2=0x55939f2c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb11a9b640 1 -- 172.21.15.78:0/3326669122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2c0d80 msgr2=0x55939f336470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2c0d80 0x55939f336470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb11a9b640 1 -- 172.21.15.78:0/3326669122 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55939f336b80 con 0x55939f2b44d0 2024-09-11T01:06:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb11a9b640 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2b44d0 0x55939f2c5590 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fdb04005c10 tx=0x7fdb04000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae05b4c1395b8f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdaf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdaf4ff9640 1 -- 172.21.15.78:0/3326669122 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04016070 con 0x55939f2b44d0 2024-09-11T01:06:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdaf4ff9640 1 -- 172.21.15.78:0/3326669122 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fdb0402cbf0 con 0x55939f2b44d0 2024-09-11T01:06:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb158d3740 1 -- 172.21.15.78:0/3326669122 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb0402e020 con 0x55939f2b44d0 2024-09-11T01:06:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdaf4ff9640 1 -- 172.21.15.78:0/3326669122 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdb04005a40 con 0x55939f2b44d0 2024-09-11T01:06:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb158d3740 10 client.16989 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb158d3740 1 -- 172.21.15.78:0/3326669122 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55939f33df20 con 0x55939f2b44d0 2024-09-11T01:06:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdaf4ff9640 1 -- 172.21.15.78:0/3326669122 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb0402e020 con 0x55939f2b44d0 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdaf4ff9640 1 -- 172.21.15.78:0/3326669122 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7fdb04004f90 con 0x55939f2b44d0 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb158d3740 10 client.16989 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb158d3740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb158d3740 20 client.16989 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb158d3740 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55939f33e480 0x55939f340870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb158d3740 4 client.16989 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.823+0000 7fdb158d3740 1 -- 172.21.15.78:0/3326669122 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55939f2288a0 con 0x55939f33e480 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.824+0000 7fdb10a99640 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55939f33e480 0x55939f340870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.824+0000 7fdb10a99640 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55939f33e480 0x55939f340870 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e25b2cd61662ea9 server_cookie=3b35c99623b5090a in_seq=0 out_seq=0 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.824+0000 7fdaf4ff9640 10 client.16989 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.824+0000 7fdaf4ff9640 1 -- 172.21.15.78:0/3326669122 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55939f2288a0 con 0x55939f33e480 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.824+0000 7fdaf4ff9640 10 client.16989 handle_command_reply: tid=0 2024-09-11T01:06:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb158d3740 1 client.16989 shutdown 2024-09-11T01:06:42.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb158d3740 1 -- 172.21.15.78:0/3326669122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2b44d0 msgr2=0x55939f2c5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:42.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb158d3740 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2b44d0 0x55939f2c5590 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fdb04005c10 tx=0x7fdb04000f40 comp rx=0 tx=0).stop 2024-09-11T01:06:42.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb158d3740 1 -- 172.21.15.78:0/3326669122 shutdown_connections 2024-09-11T01:06:42.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb158d3740 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55939f2c0d80 0x55939f336470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb158d3740 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55939f2b4df0 0x55939f2c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb158d3740 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55939f33e480 0x55939f340870 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.904 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:06:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb158d3740 1 --2- 172.21.15.78:0/3326669122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939f2b44d0 0x55939f2c5590 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdaf4ff9640 0 client.16989 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb158d3740 1 -- 172.21.15.78:0/3326669122 >> 172.21.15.78:0/3326669122 conn(0x55939f2b2300 msgr2=0x55939f2bf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb158d3740 1 -- 172.21.15.78:0/3326669122 shutdown_connections 2024-09-11T01:06:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb158d3740 1 -- 172.21.15.78:0/3326669122 wait complete. 2024-09-11T01:06:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb158d3740 20 client.16989 trim_cache size 0 max 16384 2024-09-11T01:06:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb134fe640 1 -- 172.21.15.78:0/4147837534 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdaf80b5450 msgr2=0x7fdaf80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb134fe640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdaf80b5450 0x7fdaf80b7910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdafc104280 tx=0x7fdafc10a590 comp rx=0 tx=0).stop 2024-09-11T01:06:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb134fe640 1 -- 172.21.15.78:0/4147837534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb0c101c90 msgr2=0x7fdb0c0fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb134fe640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb0c101c90 0x7fdb0c0fdc00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fdafc005940 tx=0x7fdafc0042f0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.825+0000 7fdb134fe640 1 -- 172.21.15.78:0/4147837534 shutdown_connections 2024-09-11T01:06:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.826+0000 7fdb134fe640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdaf80b5450 0x7fdaf80b7910 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.826+0000 7fdb134fe640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb0c108020 0x7fdb0c0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.826+0000 7fdb134fe640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb0c1076a0 0x7fdb0c0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.826+0000 7fdb134fe640 1 --2- 172.21.15.78:0/4147837534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb0c101c90 0x7fdb0c0fdc00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.826+0000 7fdb134fe640 1 -- 172.21.15.78:0/4147837534 >> 172.21.15.78:0/4147837534 conn(0x7fdb0c0fd700 msgr2=0x7fdb0c06aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.826+0000 7fdb134fe640 1 -- 172.21.15.78:0/4147837534 shutdown_connections 2024-09-11T01:06:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:42.826+0000 7fdb134fe640 1 -- 172.21.15.78:0/4147837534 wait complete. 2024-09-11T01:06:43.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 551030 5.51 MB/sec execute 291 sec latency 29.643 ms 2024-09-11T01:06:44.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 552308 5.51 MB/sec execute 292 sec latency 257.763 ms 2024-09-11T01:06:44.597 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:06:44.598 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:06:44.623 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:44 smithi078 ceph-mon[31315]: pgmap v561: 97 pgs: 97 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-09-11T01:06:44.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:44 smithi110 ceph-mon[34712]: pgmap v561: 97 pgs: 97 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-09-11T01:06:44.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:44 smithi112 ceph-mon[35384]: pgmap v561: 97 pgs: 97 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-09-11T01:06:44.871 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:06:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.391+0000 7f8d98f65640 1 -- 172.21.15.78:0/1473257153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d941076c0 msgr2=0x7f8d94107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.391+0000 7f8d98f65640 1 --2- 172.21.15.78:0/1473257153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d941076c0 0x7f8d94107b00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009900 tx=0x7f8d7c02a410 comp rx=0 tx=0).stop 2024-09-11T01:06:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.391+0000 7f8d98f65640 1 -- 172.21.15.78:0/1473257153 shutdown_connections 2024-09-11T01:06:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.391+0000 7f8d98f65640 1 --2- 172.21.15.78:0/1473257153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d94108040 0x7f8d9410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.391+0000 7f8d98f65640 1 --2- 172.21.15.78:0/1473257153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d941076c0 0x7f8d94107b00 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.391+0000 7f8d98f65640 1 --2- 172.21.15.78:0/1473257153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94101cb0 0x7f8d94102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.391+0000 7f8d98f65640 1 -- 172.21.15.78:0/1473257153 >> 172.21.15.78:0/1473257153 conn(0x7f8d940fd700 msgr2=0x7f8d940ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:45.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.391+0000 7f8d98f65640 1 -- 172.21.15.78:0/1473257153 shutdown_connections 2024-09-11T01:06:45.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.391+0000 7f8d98f65640 1 -- 172.21.15.78:0/1473257153 wait complete. 2024-09-11T01:06:45.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d98f65640 1 Processor -- start 2024-09-11T01:06:45.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d98f65640 1 -- start start 2024-09-11T01:06:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d98f65640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94101cb0 0x7f8d9419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d98f65640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d941076c0 0x7f8d9419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d98f65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d94108040 0x7f8d941a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d98f65640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d94111d00 con 0x7f8d94108040 2024-09-11T01:06:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d98f65640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d94111b80 con 0x7f8d941076c0 2024-09-11T01:06:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d98f65640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d94111e80 con 0x7f8d94101cb0 2024-09-11T01:06:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d92d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d94108040 0x7f8d941a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d92d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d94108040 0x7f8d941a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41820/0 (socket says 172.21.15.78:41820) 2024-09-11T01:06:45.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d92d9d640 1 -- 172.21.15.78:0/456660564 learned_addr learned my addr 172.21.15.78:0/456660564 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:45.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d941076c0 0x7f8d9419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d9259c640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94101cb0 0x7f8d9419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d92d9d640 1 -- 172.21.15.78:0/456660564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94101cb0 msgr2=0x7f8d9419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:45.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.392+0000 7f8d92d9d640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94101cb0 0x7f8d9419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.393+0000 7f8d92d9d640 1 -- 172.21.15.78:0/456660564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d941076c0 msgr2=0x7f8d9419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:45.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.393+0000 7f8d92d9d640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d941076c0 0x7f8d9419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.393+0000 7f8d92d9d640 1 -- 172.21.15.78:0/456660564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d941a1550 con 0x7f8d94108040 2024-09-11T01:06:45.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.393+0000 7f8d92d9d640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d94108040 0x7f8d941a0d80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00efd0 tx=0x7f8d8c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:45.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.393+0000 7f8d8b7fe640 1 -- 172.21.15.78:0/456660564 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c003b70 con 0x7f8d94108040 2024-09-11T01:06:45.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.393+0000 7f8d8b7fe640 1 -- 172.21.15.78:0/456660564 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d8c003d10 con 0x7f8d94108040 2024-09-11T01:06:45.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.393+0000 7f8d98f65640 1 -- 172.21.15.78:0/456660564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d941a1840 con 0x7f8d94108040 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.393+0000 7f8d8b7fe640 1 -- 172.21.15.78:0/456660564 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c0108a0 con 0x7f8d94108040 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.393+0000 7f8d98f65640 1 -- 172.21.15.78:0/456660564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d941a7f00 con 0x7f8d94108040 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d8b7fe640 1 -- 172.21.15.78:0/456660564 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8d8c01f020 con 0x7f8d94108040 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d9b33a740 1 Processor -- start 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d9b33a740 1 -- start start 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d9b33a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26dee0 0x5598bb26e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d9b33a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26e800 0x5598bb27a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d9b33a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb27a790 0x5598bb27cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d9b33a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5598bb1e4e60 con 0x5598bb26e800 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d9b33a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5598bb1d4290 con 0x5598bb27a790 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d9b33a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5598bb1d80a0 con 0x5598bb26dee0 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d9259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26e800 0x5598bb27a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26dee0 0x5598bb26e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26dee0 0x5598bb26e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54122/0 (socket says 172.21.15.78:54122) 2024-09-11T01:06:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d8bfff640 1 -- 172.21.15.78:0/1021931702 learned_addr learned my addr 172.21.15.78:0/1021931702 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.396+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/1021931702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb27a790 0x5598bb27cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d8b7fe640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d800b54d0 0x7f8d800b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d9259c640 1 -- 172.21.15.78:0/1021931702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26dee0 msgr2=0x5598bb26e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d9259c640 1 --2- 172.21.15.78:0/1021931702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26dee0 0x5598bb26e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d9259c640 1 -- 172.21.15.78:0/1021931702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb27a790 msgr2=0x5598bb27cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d9259c640 1 --2- 172.21.15.78:0/1021931702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb27a790 0x5598bb27cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d9259c640 1 -- 172.21.15.78:0/1021931702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598baf32930 con 0x5598bb26e800 2024-09-11T01:06:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/1021931702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26dee0 0x5598bb26e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d92d9d640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d800b54d0 0x7f8d800b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d8b7fe640 1 -- 172.21.15.78:0/456660564 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f8d8c0e60b0 con 0x7f8d94108040 2024-09-11T01:06:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d9259c640 1 --2- 172.21.15.78:0/1021931702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26e800 0x5598bb27a250 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009ba0 tx=0x7f8d840117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b76a51e66cc1d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d88ff9640 1 -- 172.21.15.78:0/1021931702 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84012450 con 0x5598bb26e800 2024-09-11T01:06:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d88ff9640 1 -- 172.21.15.78:0/1021931702 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d84012aa0 con 0x5598bb26e800 2024-09-11T01:06:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.397+0000 7f8d88ff9640 1 -- 172.21.15.78:0/1021931702 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8400faa0 con 0x5598bb26e800 2024-09-11T01:06:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.398+0000 7f8d9b33a740 1 -- 172.21.15.78:0/1021931702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26e800 msgr2=0x5598bb27a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.398+0000 7f8d92d9d640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d800b54d0 0x7f8d800b7990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0e38f0 tx=0x7f8d8c0cf040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.398+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/1021931702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26e800 0x5598bb27a250 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009ba0 tx=0x7f8d840117b0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.398+0000 7f8d9b33a740 1 -- 172.21.15.78:0/1021931702 shutdown_connections 2024-09-11T01:06:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.398+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/1021931702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb27a790 0x5598bb27cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.398+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/1021931702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26e800 0x5598bb27a250 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.398+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/1021931702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26dee0 0x5598bb26e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.398+0000 7f8d9b33a740 1 -- 172.21.15.78:0/1021931702 >> 172.21.15.78:0/1021931702 conn(0x5598bb26bd10 msgr2=0x5598bb26ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.398+0000 7f8d9b33a740 1 -- 172.21.15.78:0/1021931702 shutdown_connections 2024-09-11T01:06:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.398+0000 7f8d9b33a740 1 -- 172.21.15.78:0/1021931702 wait complete. 2024-09-11T01:06:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.398+0000 7f8d9b33a740 1 -- start start 2024-09-11T01:06:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d9b33a740 1 Processor -- start 2024-09-11T01:06:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d9b33a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 0x5598bb2777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d9b33a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26e800 0x5598bb277cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d9b33a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb27a790 0x5598bb278230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d9b33a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5598bb035890 con 0x5598bb26dee0 2024-09-11T01:06:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d9b33a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5598bb16ed40 con 0x5598bb27a790 2024-09-11T01:06:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d9b33a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5598bb27d440 con 0x5598bb26e800 2024-09-11T01:06:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d9b33a740 10 client.0 _ll_register_callbacks cb 0x5598bb1d2c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d8bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 0x5598bb2777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d8bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 0x5598bb2777b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41830/0 (socket says 172.21.15.78:41830) 2024-09-11T01:06:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d8bfff640 1 -- 172.21.15.78:0/2470853417 learned_addr learned my addr 172.21.15.78:0/2470853417 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d9259c640 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26e800 0x5598bb277cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb27a790 0x5598bb278230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d8bfff640 1 -- 172.21.15.78:0/2470853417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26e800 msgr2=0x5598bb277cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26e800 0x5598bb277cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d8bfff640 1 -- 172.21.15.78:0/2470853417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb27a790 msgr2=0x5598bb278230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb27a790 0x5598bb278230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d8bfff640 1 -- 172.21.15.78:0/2470853417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598bb2868a0 con 0x5598bb26dee0 2024-09-11T01:06:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 0x5598bb2777b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c000c00 tx=0x7f8d7c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c8d52a3f22c111 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.399+0000 7f8d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d79ffb640 1 -- 172.21.15.78:0/2470853417 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c035030 con 0x5598bb26dee0 2024-09-11T01:06:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d79ffb640 1 -- 172.21.15.78:0/2470853417 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f8d7c004320 con 0x5598bb26dee0 2024-09-11T01:06:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2470853417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d84012aa0 con 0x5598bb26dee0 2024-09-11T01:06:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d79ffb640 1 -- 172.21.15.78:0/2470853417 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8d7c054050 con 0x5598bb26dee0 2024-09-11T01:06:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d9b33a740 10 client.17001 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2470853417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5598bb289250 con 0x5598bb26dee0 2024-09-11T01:06:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d79ffb640 1 -- 172.21.15.78:0/2470853417 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d84012aa0 con 0x5598bb26dee0 2024-09-11T01:06:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d79ffb640 1 -- 172.21.15.78:0/2470853417 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f8d7c04b070 con 0x5598bb26dee0 2024-09-11T01:06:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d9b33a740 10 client.17001 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d9b33a740 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:06:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d9b33a740 20 client.17001 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5598bb28a700 0x5598bb28caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d9b33a740 4 client.17001 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:06:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.400+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2470853417 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598bb1e6db0 con 0x5598bb28a700 2024-09-11T01:06:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.401+0000 7f8d9259c640 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5598bb28a700 0x5598bb28caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.401+0000 7f8d79ffb640 10 client.17001 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.401+0000 7f8d9259c640 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5598bb28a700 0x5598bb28caf0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=931dac7935d86a12 server_cookie=ec13797134c9b1f4 in_seq=0 out_seq=0 2024-09-11T01:06:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.402+0000 7f8d79ffb640 1 -- 172.21.15.78:0/2470853417 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5598bb1e6db0 con 0x5598bb28a700 2024-09-11T01:06:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.402+0000 7f8d79ffb640 10 client.17001 handle_command_reply: tid=0 2024-09-11T01:06:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.402+0000 7f8d9b33a740 1 client.17001 shutdown 2024-09-11T01:06:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.402+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2470853417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 msgr2=0x5598bb2777b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.402+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 0x5598bb2777b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c000c00 tx=0x7f8d7c009260 comp rx=0 tx=0).stop 2024-09-11T01:06:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.402+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2470853417 shutdown_connections 2024-09-11T01:06:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.402+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb27a790 0x5598bb278230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.402+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26e800 0x5598bb277cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.402+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5598bb28a700 0x5598bb28caf0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.402+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/2470853417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 0x5598bb2777b0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.402+0000 7f8d79ffb640 0 client.17001 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.402+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2470853417 >> 172.21.15.78:0/2470853417 conn(0x5598bb26bd10 msgr2=0x5598bb26af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.403+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2470853417 shutdown_connections 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.403+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2470853417 wait complete. 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.403+0000 7f8d9b33a740 20 client.17001 trim_cache size 0 max 16384 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d9b33a740 1 Processor -- start 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d9b33a740 1 -- start start 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d9b33a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 0x5598bb277040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d9b33a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb26e800 0x5598bb277e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d9b33a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb27a790 0x5598bb2749c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d9b33a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941b0790 con 0x5598bb26dee0 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d9b33a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941b2360 con 0x5598bb26e800 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d9b33a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941b0e60 con 0x5598bb27a790 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d8bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 0x5598bb277040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d8bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 0x5598bb277040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41832/0 (socket says 172.21.15.78:41832) 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d8bfff640 1 -- 172.21.15.78:0/2109189350 learned_addr learned my addr 172.21.15.78:0/2109189350 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d9259c640 1 --2- 172.21.15.78:0/2109189350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb26e800 0x5598bb277e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.421+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/2109189350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb27a790 0x5598bb2749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.422+0000 7f8d8bfff640 1 -- 172.21.15.78:0/2109189350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb27a790 msgr2=0x5598bb2749c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.422+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/2109189350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb27a790 0x5598bb2749c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.422+0000 7f8d8bfff640 1 -- 172.21.15.78:0/2109189350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb26e800 msgr2=0x5598bb277e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.422+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/2109189350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb26e800 0x5598bb277e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.422+0000 7f8d8bfff640 1 -- 172.21.15.78:0/2109189350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598bb26cc00 con 0x5598bb26dee0 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.422+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/2109189350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 0x5598bb277040 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0059d0 tx=0x7f8d7c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9cd7ac07b3bab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.422+0000 7f8d7bfff640 1 -- 172.21.15.78:0/2109189350 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c035040 con 0x5598bb26dee0 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.422+0000 7f8d7bfff640 1 -- 172.21.15.78:0/2109189350 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d7c005070 con 0x5598bb26dee0 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.422+0000 7f8d7bfff640 1 -- 172.21.15.78:0/2109189350 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c0049b0 con 0x5598bb26dee0 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2109189350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 msgr2=0x5598bb277040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/2109189350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 0x5598bb277040 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0059d0 tx=0x7f8d7c02a410 comp rx=0 tx=0).stop 2024-09-11T01:06:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2109189350 shutdown_connections 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/2109189350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb27a790 0x5598bb2749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/2109189350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb26e800 0x5598bb277e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/2109189350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb26dee0 0x5598bb277040 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2109189350 >> 172.21.15.78:0/2109189350 conn(0x5598bb26bd10 msgr2=0x5598bb277720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2109189350 shutdown_connections 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 -- 172.21.15.78:0/2109189350 wait complete. 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 -- start start 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 Processor -- start 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26dee0 0x5598bb27e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb26e800 0x5598bb27eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb27a790 0x5598bb2f6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941b1c60 con 0x5598bb27a790 2024-09-11T01:06:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941b1590 con 0x5598bb26e800 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.423+0000 7f8d9b33a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941ae790 con 0x5598bb26dee0 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d9b33a740 10 client.0 _ll_register_callbacks cb 0x7f8d7c032da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d8bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb27a790 0x5598bb2f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d9259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb26e800 0x5598bb27eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d9259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb26e800 0x5598bb27eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:32816/0 (socket says 172.21.15.78:32816) 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d8bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb27a790 0x5598bb2f6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41834/0 (socket says 172.21.15.78:41834) 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d9259c640 1 -- 172.21.15.78:0/140497331 learned_addr learned my addr 172.21.15.78:0/140497331 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26dee0 0x5598bb27e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d9259c640 1 -- 172.21.15.78:0/140497331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26dee0 msgr2=0x5598bb27e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d9259c640 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26dee0 0x5598bb27e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d9259c640 1 -- 172.21.15.78:0/140497331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb27a790 msgr2=0x5598bb2f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d9259c640 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb27a790 0x5598bb2f6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d9259c640 1 -- 172.21.15.78:0/140497331 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598bb27f7b0 con 0x5598bb26e800 2024-09-11T01:06:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb27a790 0x5598bb2f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d9259c640 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb26e800 0x5598bb27eec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009ba0 tx=0x7f8d84009860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d005883c7653bcdb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d79ffb640 1 -- 172.21.15.78:0/140497331 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8400f980 con 0x5598bb26e800 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d9b33a740 1 -- 172.21.15.78:0/140497331 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d7c005070 con 0x5598bb26e800 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d79ffb640 1 -- 172.21.15.78:0/140497331 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f8d84020040 con 0x5598bb26e800 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d79ffb640 1 -- 172.21.15.78:0/140497331 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8d8400fb20 con 0x5598bb26e800 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.424+0000 7f8d9b33a740 10 client.26276 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.425+0000 7f8d9b33a740 1 -- 172.21.15.78:0/140497331 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5598bb2f7ce0 con 0x5598bb26e800 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.425+0000 7f8d79ffb640 1 -- 172.21.15.78:0/140497331 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d7c005070 con 0x5598bb26e800 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.425+0000 7f8d79ffb640 1 -- 172.21.15.78:0/140497331 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f8d840035f0 con 0x5598bb26e800 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.425+0000 7f8d9b33a740 10 client.26276 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.425+0000 7f8d9b33a740 10 client.26276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:06:45.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.425+0000 7f8d9b33a740 20 client.26276 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.425+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5598bb2f82a0 0x5598bb2fa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.425+0000 7f8d9b33a740 4 client.26276 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:06:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.425+0000 7f8d9b33a740 1 -- 172.21.15.78:0/140497331 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5598bb1e2860 con 0x5598bb2f82a0 2024-09-11T01:06:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.425+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5598bb2f82a0 0x5598bb2fa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.425+0000 7f8d79ffb640 10 client.26276 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.425+0000 7f8d8bfff640 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5598bb2f82a0 0x5598bb2fa6c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87d25adae74eab2b server_cookie=52bee1c798bb0b2 in_seq=0 out_seq=0 2024-09-11T01:06:45.452 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:06:45.452 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:06:45.452 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "371f8253-b1f4-400e-94c8-988b8dc35dba", 2024-09-11T01:06:45.452 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:06:45.453 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:06:45.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d79ffb640 1 -- 172.21.15.78:0/140497331 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5598bb1e2860 con 0x5598bb2f82a0 2024-09-11T01:06:45.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d79ffb640 10 client.26276 handle_command_reply: tid=0 2024-09-11T01:06:45.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d9b33a740 1 client.26276 shutdown 2024-09-11T01:06:45.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d9b33a740 1 -- 172.21.15.78:0/140497331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb26e800 msgr2=0x5598bb27eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:45.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb26e800 0x5598bb27eec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009ba0 tx=0x7f8d84009860 comp rx=0 tx=0).stop 2024-09-11T01:06:45.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d9b33a740 1 -- 172.21.15.78:0/140497331 shutdown_connections 2024-09-11T01:06:45.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5598bb2f82a0 0x5598bb2fa6c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598bb27a790 0x5598bb2f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598bb26e800 0x5598bb27eec0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d79ffb640 0 client.26276 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:45.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d9b33a740 1 --2- 172.21.15.78:0/140497331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598bb26dee0 0x5598bb27e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d9b33a740 1 -- 172.21.15.78:0/140497331 >> 172.21.15.78:0/140497331 conn(0x5598bb26bd10 msgr2=0x5598bb277f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:45.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d9b33a740 1 -- 172.21.15.78:0/140497331 shutdown_connections 2024-09-11T01:06:45.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d9b33a740 1 -- 172.21.15.78:0/140497331 wait complete. 2024-09-11T01:06:45.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.427+0000 7f8d9b33a740 20 client.26276 trim_cache size 0 max 16384 2024-09-11T01:06:45.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.428+0000 7f8d98f65640 1 -- 172.21.15.78:0/456660564 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d800b54d0 msgr2=0x7f8d800b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:45.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.428+0000 7f8d98f65640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d800b54d0 0x7f8d800b7990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0e38f0 tx=0x7f8d8c0cf040 comp rx=0 tx=0).stop 2024-09-11T01:06:45.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.428+0000 7f8d98f65640 1 -- 172.21.15.78:0/456660564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d94108040 msgr2=0x7f8d941a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:45.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.428+0000 7f8d98f65640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d94108040 0x7f8d941a0d80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00efd0 tx=0x7f8d8c00c6a0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.428+0000 7f8d98f65640 1 -- 172.21.15.78:0/456660564 shutdown_connections 2024-09-11T01:06:45.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.428+0000 7f8d98f65640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d800b54d0 0x7f8d800b7990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.428+0000 7f8d98f65640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d94108040 0x7f8d941a0d80 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.428+0000 7f8d98f65640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d941076c0 0x7f8d9419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.428+0000 7f8d98f65640 1 --2- 172.21.15.78:0/456660564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94101cb0 0x7f8d9419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:45.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.428+0000 7f8d98f65640 1 -- 172.21.15.78:0/456660564 >> 172.21.15.78:0/456660564 conn(0x7f8d940fd700 msgr2=0x7f8d9406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:45.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.428+0000 7f8d98f65640 1 -- 172.21.15.78:0/456660564 shutdown_connections 2024-09-11T01:06:45.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:45.428+0000 7f8d98f65640 1 -- 172.21.15.78:0/456660564 wait complete. 2024-09-11T01:06:45.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 554445 5.54 MB/sec execute 293 sec latency 17.568 ms 2024-09-11T01:06:45.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:45 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:45.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:45 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:45.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:45 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:45.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:45 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:45.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:45 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:45.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:45 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:45.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:45 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:45.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:45 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:45.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:45 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:45.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:45 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:45.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:45 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:45.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:45 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:45.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:45 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:45.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:45 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:45.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:45 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:45.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:45 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:45.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:45 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:45.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:45 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.167 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:06:46.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 555852 5.54 MB/sec execute 294 sec latency 82.101 ms 2024-09-11T01:06:46.469 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:06:46.868 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:06:46.868 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.869 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:46.869 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:46.869 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:46.869 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:46.869 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.869 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.869 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.869 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:46.869 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:46.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:46.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:46.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:06:46.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:46 smithi078 ceph-mon[31315]: pgmap v562: 97 pgs: 97 active+clean; 377 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.9 MiB/s wr, 158 op/s 2024-09-11T01:06:46.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:46.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:06:46.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:46 smithi110 ceph-mon[34712]: pgmap v562: 97 pgs: 97 active+clean; 377 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.9 MiB/s wr, 158 op/s 2024-09-11T01:06:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.948+0000 7f300fa07640 1 -- 172.21.15.78:0/1393919972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3008104600 msgr2=0x7f300810caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.948+0000 7f300fa07640 1 --2- 172.21.15.78:0/1393919972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3008104600 0x7f300810caf0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3004009900 tx=0x7f300402a410 comp rx=0 tx=0).stop 2024-09-11T01:06:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.948+0000 7f300fa07640 1 -- 172.21.15.78:0/1393919972 shutdown_connections 2024-09-11T01:06:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.948+0000 7f300fa07640 1 --2- 172.21.15.78:0/1393919972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f300810d140 0x7f300810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.948+0000 7f300fa07640 1 --2- 172.21.15.78:0/1393919972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3008104600 0x7f300810caf0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.948+0000 7f300fa07640 1 --2- 172.21.15.78:0/1393919972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3008103ce0 0x7f30081040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.948+0000 7f300fa07640 1 -- 172.21.15.78:0/1393919972 >> 172.21.15.78:0/1393919972 conn(0x7f30080fd720 msgr2=0x7f30080ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.948+0000 7f300fa07640 1 -- 172.21.15.78:0/1393919972 shutdown_connections 2024-09-11T01:06:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.948+0000 7f300fa07640 1 -- 172.21.15.78:0/1393919972 wait complete. 2024-09-11T01:06:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300fa07640 1 Processor -- start 2024-09-11T01:06:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300fa07640 1 -- start start 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300fa07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3008103ce0 0x7f30081993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300fa07640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3008104600 0x7f30081998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300fa07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300810d140 0x7f300819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300fa07640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3008112db0 con 0x7f300810d140 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300fa07640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3008112c30 con 0x7f3008104600 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300fa07640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3008112f30 con 0x7f3008103ce0 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300dfa4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300810d140 0x7f300819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300dfa4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300810d140 0x7f300819ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41848/0 (socket says 172.21.15.78:41848) 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300dfa4640 1 -- 172.21.15.78:0/2271579346 learned_addr learned my addr 172.21.15.78:0/2271579346 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300cfa2640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3008104600 0x7f30081998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300d7a3640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3008103ce0 0x7f30081993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300dfa4640 1 -- 172.21.15.78:0/2271579346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3008103ce0 msgr2=0x7f30081993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300dfa4640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3008103ce0 0x7f30081993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300dfa4640 1 -- 172.21.15.78:0/2271579346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3008104600 msgr2=0x7f30081998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300dfa4640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3008104600 0x7f30081998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.949+0000 7f300dfa4640 1 -- 172.21.15.78:0/2271579346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300819f450 con 0x7f300810d140 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.950+0000 7f300dfa4640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300810d140 0x7f300819ec80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f300000eb10 tx=0x7f300000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.950+0000 7f2ffe7fc640 1 -- 172.21.15.78:0/2271579346 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000004b20 con 0x7f300810d140 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.950+0000 7f2ffe7fc640 1 -- 172.21.15.78:0/2271579346 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3000004cc0 con 0x7f300810d140 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.950+0000 7f300fa07640 1 -- 172.21.15.78:0/2271579346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300819f740 con 0x7f300810d140 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.950+0000 7f2ffe7fc640 1 -- 172.21.15.78:0/2271579346 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000010670 con 0x7f300810d140 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.950+0000 7f300fa07640 1 -- 172.21.15.78:0/2271579346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30081a5e00 con 0x7f300810d140 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.953+0000 7f3011ddc740 1 Processor -- start 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.953+0000 7f2ffe7fc640 1 -- 172.21.15.78:0/2271579346 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3000005020 con 0x7f300810d140 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.953+0000 7f3011ddc740 1 -- start start 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.953+0000 7f3011ddc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e109480 0x56423e109860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.953+0000 7f3011ddc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 0x56423e1157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.953+0000 7f3011ddc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e115d30 0x56423e118120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.953+0000 7f3011ddc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56423e07fe50 con 0x56423e109da0 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.953+0000 7f3011ddc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56423e06f280 con 0x56423e109480 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.953+0000 7f3011ddc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56423e073090 con 0x56423e115d30 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.953+0000 7f300d7a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 0x56423e1157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f300d7a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 0x56423e1157f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41860/0 (socket says 172.21.15.78:41860) 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f300d7a3640 1 -- 172.21.15.78:0/3474564360 learned_addr learned my addr 172.21.15.78:0/3474564360 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f300cfa2640 1 --2- 172.21.15.78:0/3474564360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e109480 0x56423e109860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f300cfa2640 1 --2- 172.21.15.78:0/3474564360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e115d30 0x56423e118120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f2ffe7fc640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2ff40b5450 0x7f2ff40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f300d7a3640 1 -- 172.21.15.78:0/3474564360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e115d30 msgr2=0x56423e118120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f300d7a3640 1 --2- 172.21.15.78:0/3474564360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e115d30 0x56423e118120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f300d7a3640 1 -- 172.21.15.78:0/3474564360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e109480 msgr2=0x56423e109860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f300d7a3640 1 --2- 172.21.15.78:0/3474564360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e109480 0x56423e109860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f300d7a3640 1 -- 172.21.15.78:0/3474564360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423ddcd650 con 0x56423e109da0 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f300cfa2640 1 --2- 172.21.15.78:0/3474564360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e115d30 0x56423e118120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f300dfa4640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2ff40b5450 0x7f2ff40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f2ffe7fc640 1 -- 172.21.15.78:0/2271579346 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f30000e64d0 con 0x7f300810d140 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f300d7a3640 1 --2- 172.21.15.78:0/3474564360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 0x56423e1157f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009d20 tx=0x7f2ff8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01a9f62e7442b63 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f2ff37fe640 1 -- 172.21.15.78:0/3474564360 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff801e070 con 0x56423e109da0 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f2ff37fe640 1 -- 172.21.15.78:0/3474564360 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff8019ae0 con 0x56423e109da0 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.954+0000 7f2ff37fe640 1 -- 172.21.15.78:0/3474564360 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff800ea10 con 0x56423e109da0 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.955+0000 7f3011ddc740 1 -- 172.21.15.78:0/3474564360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 msgr2=0x56423e1157f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.955+0000 7f3011ddc740 1 --2- 172.21.15.78:0/3474564360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 0x56423e1157f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009d20 tx=0x7f2ff8011810 comp rx=0 tx=0).stop 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.955+0000 7f300dfa4640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2ff40b5450 0x7f2ff40b7910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f30000e3d10 tx=0x7f30000a11f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.955+0000 7f3011ddc740 1 -- 172.21.15.78:0/3474564360 shutdown_connections 2024-09-11T01:06:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.955+0000 7f3011ddc740 1 --2- 172.21.15.78:0/3474564360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e115d30 0x56423e118120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.955+0000 7f3011ddc740 1 --2- 172.21.15.78:0/3474564360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 0x56423e1157f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.955+0000 7f3011ddc740 1 --2- 172.21.15.78:0/3474564360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e109480 0x56423e109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.955+0000 7f3011ddc740 1 -- 172.21.15.78:0/3474564360 >> 172.21.15.78:0/3474564360 conn(0x56423e1072b0 msgr2=0x56423e108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.955+0000 7f3011ddc740 1 -- 172.21.15.78:0/3474564360 shutdown_connections 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.955+0000 7f3011ddc740 1 -- 172.21.15.78:0/3474564360 wait complete. 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.955+0000 7f3011ddc740 1 -- start start 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f3011ddc740 1 Processor -- start 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f3011ddc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109480 0x56423e1155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f3011ddc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109da0 0x56423e112a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f3011ddc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e115d30 0x56423e113020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f3011ddc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56423ded0700 con 0x56423e109480 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f3011ddc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56423e00a110 con 0x56423e115d30 2024-09-11T01:06:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f3011ddc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56423e1189b0 con 0x56423e109da0 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f3011ddc740 10 client.0 _ll_register_callbacks cb 0x56423e023990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f300cfa2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109480 0x56423e1155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f300cfa2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109480 0x56423e1155f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41868/0 (socket says 172.21.15.78:41868) 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f300cfa2640 1 -- 172.21.15.78:0/918139868 learned_addr learned my addr 172.21.15.78:0/918139868 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f300d7a3640 1 --2- 172.21.15.78:0/918139868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109da0 0x56423e112a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f300cfa2640 1 -- 172.21.15.78:0/918139868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109da0 msgr2=0x56423e112a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f300cfa2640 1 --2- 172.21.15.78:0/918139868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109da0 0x56423e112a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f300cfa2640 1 -- 172.21.15.78:0/918139868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e115d30 msgr2=0x56423e113020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f300cfa2640 1 --2- 172.21.15.78:0/918139868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e115d30 0x56423e113020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f300cfa2640 1 -- 172.21.15.78:0/918139868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56423e037c40 con 0x56423e109480 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f300cfa2640 1 --2- 172.21.15.78:0/918139868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109480 0x56423e1155f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f300806c330 tx=0x7f3004032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270c2687813b2505 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.956+0000 7f2ff0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f2ff0ff9640 1 -- 172.21.15.78:0/918139868 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004035040 con 0x56423e109480 2024-09-11T01:06:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f2ff0ff9640 1 -- 172.21.15.78:0/918139868 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f3004004240 con 0x56423e109480 2024-09-11T01:06:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f3011ddc740 1 -- 172.21.15.78:0/918139868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff8019ae0 con 0x56423e109480 2024-09-11T01:06:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f2ff0ff9640 1 -- 172.21.15.78:0/918139868 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3004004440 con 0x56423e109480 2024-09-11T01:06:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f3011ddc740 10 client.17019 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f3011ddc740 1 -- 172.21.15.78:0/918139868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56423e1242e0 con 0x56423e109480 2024-09-11T01:06:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f2ff0ff9640 1 -- 172.21.15.78:0/918139868 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff8019ae0 con 0x56423e109480 2024-09-11T01:06:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f2ff0ff9640 1 -- 172.21.15.78:0/918139868 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f300403d5e0 con 0x56423e109480 2024-09-11T01:06:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f3011ddc740 10 client.17019 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f3011ddc740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:06:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f3011ddc740 20 client.17019 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f3011ddc740 1 --2- 172.21.15.78:0/918139868 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56423e125790 0x56423e127bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f3011ddc740 4 client.17019 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.957+0000 7f3011ddc740 1 -- 172.21.15.78:0/918139868 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56423e081da0 con 0x56423e125790 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.958+0000 7f300d7a3640 1 --2- 172.21.15.78:0/918139868 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56423e125790 0x56423e127bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.958+0000 7f2ff0ff9640 10 client.17019 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.958+0000 7f300d7a3640 1 --2- 172.21.15.78:0/918139868 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56423e125790 0x56423e127bd0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=666a0ca7df231036 server_cookie=c0dbb0eeab0e86c1 in_seq=0 out_seq=0 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f2ff0ff9640 1 -- 172.21.15.78:0/918139868 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56423e081da0 con 0x56423e125790 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f2ff0ff9640 10 client.17019 handle_command_reply: tid=0 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f3011ddc740 1 client.17019 shutdown 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f3011ddc740 1 -- 172.21.15.78:0/918139868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109480 msgr2=0x56423e1155f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f3011ddc740 1 --2- 172.21.15.78:0/918139868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109480 0x56423e1155f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f300806c330 tx=0x7f3004032f40 comp rx=0 tx=0).stop 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f3011ddc740 1 -- 172.21.15.78:0/918139868 shutdown_connections 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f3011ddc740 1 --2- 172.21.15.78:0/918139868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e115d30 0x56423e113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f3011ddc740 1 --2- 172.21.15.78:0/918139868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109da0 0x56423e112a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f3011ddc740 1 --2- 172.21.15.78:0/918139868 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56423e125790 0x56423e127bd0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f3011ddc740 1 --2- 172.21.15.78:0/918139868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109480 0x56423e1155f0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f2ff0ff9640 0 client.17019 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f3011ddc740 1 -- 172.21.15.78:0/918139868 >> 172.21.15.78:0/918139868 conn(0x56423e1072b0 msgr2=0x56423e10ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f3011ddc740 1 -- 172.21.15.78:0/918139868 shutdown_connections 2024-09-11T01:06:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f3011ddc740 1 -- 172.21.15.78:0/918139868 wait complete. 2024-09-11T01:06:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.959+0000 7f3011ddc740 20 client.17019 trim_cache size 0 max 16384 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:46.979 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.980 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.980 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:46.980 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.980 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.980 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:06:46.980 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.980 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:06:46.980 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:46 smithi112 ceph-mon[35384]: pgmap v562: 97 pgs: 97 active+clean; 377 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.9 MiB/s wr, 158 op/s 2024-09-11T01:06:46.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.976+0000 7f3011ddc740 1 Processor -- start 2024-09-11T01:06:46.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.976+0000 7f3011ddc740 1 -- start start 2024-09-11T01:06:46.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.976+0000 7f3011ddc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109480 0x56423e121c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.976+0000 7f3011ddc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e109da0 0x56423e117120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.976+0000 7f3011ddc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e115d30 0x56423e117660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.976+0000 7f3011ddc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30081ae6d0 con 0x56423e115d30 2024-09-11T01:06:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f3011ddc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f30081b02a0 con 0x56423e109da0 2024-09-11T01:06:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f3011ddc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30081aeda0 con 0x56423e109480 2024-09-11T01:06:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f300cfa2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e115d30 0x56423e117660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f300cfa2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e115d30 0x56423e117660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41880/0 (socket says 172.21.15.78:41880) 2024-09-11T01:06:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f300cfa2640 1 -- 172.21.15.78:0/2200397559 learned_addr learned my addr 172.21.15.78:0/2200397559 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f300cfa2640 1 --2- 172.21.15.78:0/2200397559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109480 0x56423e121c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f300d7a3640 1 --2- 172.21.15.78:0/2200397559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e109da0 0x56423e117120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f300cfa2640 1 -- 172.21.15.78:0/2200397559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109480 msgr2=0x56423e121c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f300cfa2640 1 --2- 172.21.15.78:0/2200397559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109480 0x56423e121c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f300cfa2640 1 -- 172.21.15.78:0/2200397559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e109da0 msgr2=0x56423e117120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f300cfa2640 1 --2- 172.21.15.78:0/2200397559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e109da0 0x56423e117120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f300cfa2640 1 -- 172.21.15.78:0/2200397559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423e1127f0 con 0x56423e115d30 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f300cfa2640 1 --2- 172.21.15.78:0/2200397559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e115d30 0x56423e117660 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3004032c80 tx=0x7f3004031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f21e489f7bf6189 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f2ff2ffd640 1 -- 172.21.15.78:0/2200397559 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004035020 con 0x56423e115d30 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f2ff2ffd640 1 -- 172.21.15.78:0/2200397559 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f300404e020 con 0x56423e115d30 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.977+0000 7f2ff2ffd640 1 -- 172.21.15.78:0/2200397559 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004032290 con 0x56423e115d30 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.978+0000 7f3011ddc740 1 -- 172.21.15.78:0/2200397559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e115d30 msgr2=0x56423e117660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.978+0000 7f3011ddc740 1 --2- 172.21.15.78:0/2200397559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e115d30 0x56423e117660 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3004032c80 tx=0x7f3004031400 comp rx=0 tx=0).stop 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.978+0000 7f3011ddc740 1 -- 172.21.15.78:0/2200397559 shutdown_connections 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.978+0000 7f3011ddc740 1 --2- 172.21.15.78:0/2200397559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e115d30 0x56423e117660 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.978+0000 7f3011ddc740 1 --2- 172.21.15.78:0/2200397559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e109da0 0x56423e117120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.978+0000 7f3011ddc740 1 --2- 172.21.15.78:0/2200397559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109480 0x56423e121c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.978+0000 7f3011ddc740 1 -- 172.21.15.78:0/2200397559 >> 172.21.15.78:0/2200397559 conn(0x56423e1072b0 msgr2=0x56423e10dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.978+0000 7f3011ddc740 1 -- 172.21.15.78:0/2200397559 shutdown_connections 2024-09-11T01:06:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.978+0000 7f3011ddc740 1 -- 172.21.15.78:0/2200397559 wait complete. 2024-09-11T01:06:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.978+0000 7f3011ddc740 1 -- start start 2024-09-11T01:06:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.978+0000 7f3011ddc740 1 Processor -- start 2024-09-11T01:06:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f3011ddc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109480 0x56423e11dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f3011ddc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 0x56423e11e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f3011ddc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e115d30 0x56423e118c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f3011ddc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30081afba0 con 0x56423e109da0 2024-09-11T01:06:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f3011ddc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f30081af4d0 con 0x56423e115d30 2024-09-11T01:06:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f3011ddc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30081ac6d0 con 0x56423e109480 2024-09-11T01:06:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f3011ddc740 10 client.0 _ll_register_callbacks cb 0x7f3004009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:06:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f300d7a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 0x56423e11e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f300d7a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 0x56423e11e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41888/0 (socket says 172.21.15.78:41888) 2024-09-11T01:06:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f300d7a3640 1 -- 172.21.15.78:0/458919600 learned_addr learned my addr 172.21.15.78:0/458919600 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f300cfa2640 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e115d30 0x56423e118c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f300cfa2640 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109480 0x56423e11dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f300d7a3640 1 -- 172.21.15.78:0/458919600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109480 msgr2=0x56423e11dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f300d7a3640 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109480 0x56423e11dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f300d7a3640 1 -- 172.21.15.78:0/458919600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e115d30 msgr2=0x56423e118c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f300d7a3640 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e115d30 0x56423e118c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f300d7a3640 1 -- 172.21.15.78:0/458919600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56423e119140 con 0x56423e109da0 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f300cfa2640 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109480 0x56423e11dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f300d7a3640 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 0x56423e11e190 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80058c0 tx=0x7f2ff8009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ba4a3239d51b5e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.979+0000 7f2ff0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f2ff0ff9640 1 -- 172.21.15.78:0/458919600 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff801e040 con 0x56423e109da0 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f3011ddc740 1 -- 172.21.15.78:0/458919600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f300404e020 con 0x56423e109da0 2024-09-11T01:06:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f2ff0ff9640 1 -- 172.21.15.78:0/458919600 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f2ff8005040 con 0x56423e109da0 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f2ff0ff9640 1 -- 172.21.15.78:0/458919600 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f2ff8005240 con 0x56423e109da0 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f3011ddc740 10 client.17025 fetch_fsmap learned FSMap version 45 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f3011ddc740 1 -- 172.21.15.78:0/458919600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56423e192eb0 con 0x56423e109da0 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f2ff0ff9640 1 -- 172.21.15.78:0/458919600 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f300404e020 con 0x56423e109da0 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f2ff0ff9640 1 -- 172.21.15.78:0/458919600 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f2ff80035f0 con 0x56423e109da0 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f3011ddc740 10 client.17025 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f3011ddc740 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f3011ddc740 20 client.17025 populate_metadata read hostname 'smithi078' 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f3011ddc740 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56423e1935c0 0x56423e1959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f3011ddc740 4 client.17025 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f3011ddc740 1 -- 172.21.15.78:0/458919600 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56423e07d850 con 0x56423e1935c0 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.980+0000 7f300cfa2640 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56423e1935c0 0x56423e1959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:06:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f2ff0ff9640 10 client.17025 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f300cfa2640 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56423e1935c0 0x56423e1959e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8acdab5f5e5b8f1 server_cookie=74e69ba0cc4e1fb2 in_seq=0 out_seq=0 2024-09-11T01:06:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f2ff0ff9640 1 -- 172.21.15.78:0/458919600 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x56423e07d850 con 0x56423e1935c0 2024-09-11T01:06:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f2ff0ff9640 10 client.17025 handle_command_reply: tid=0 2024-09-11T01:06:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f3011ddc740 1 client.17025 shutdown 2024-09-11T01:06:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f3011ddc740 1 -- 172.21.15.78:0/458919600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 msgr2=0x56423e11e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f3011ddc740 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 0x56423e11e190 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80058c0 tx=0x7f2ff8009ed0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f3011ddc740 1 -- 172.21.15.78:0/458919600 shutdown_connections 2024-09-11T01:06:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f3011ddc740 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423e115d30 0x56423e118c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f3011ddc740 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56423e1935c0 0x56423e1959e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f3011ddc740 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56423e109da0 0x56423e11e190 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f3011ddc740 1 --2- 172.21.15.78:0/458919600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56423e109480 0x56423e11dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f3011ddc740 1 -- 172.21.15.78:0/458919600 >> 172.21.15.78:0/458919600 conn(0x56423e1072b0 msgr2=0x56423e114070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:46.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f2ff0ff9640 0 client.17025 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:06:46.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f3011ddc740 1 -- 172.21.15.78:0/458919600 shutdown_connections 2024-09-11T01:06:46.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f3011ddc740 1 -- 172.21.15.78:0/458919600 wait complete. 2024-09-11T01:06:46.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.981+0000 7f3011ddc740 20 client.17025 trim_cache size 0 max 16384 2024-09-11T01:06:46.992 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:06:46.992 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:06:46.992 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:06:46.992 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "371f8253-b1f4-400e-94c8-988b8dc35dba", 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:46.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:46.995 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:46.996 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:46.997 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:46.998 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:06:46.999 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:06:47.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.982+0000 7f300fa07640 1 -- 172.21.15.78:0/2271579346 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2ff40b5450 msgr2=0x7f2ff40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:47.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.982+0000 7f300fa07640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2ff40b5450 0x7f2ff40b7910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f30000e3d10 tx=0x7f30000a11f0 comp rx=0 tx=0).stop 2024-09-11T01:06:47.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.982+0000 7f300fa07640 1 -- 172.21.15.78:0/2271579346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300810d140 msgr2=0x7f300819ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:06:47.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.982+0000 7f300fa07640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300810d140 0x7f300819ec80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f300000eb10 tx=0x7f300000c2d0 comp rx=0 tx=0).stop 2024-09-11T01:06:47.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.982+0000 7f300fa07640 1 -- 172.21.15.78:0/2271579346 shutdown_connections 2024-09-11T01:06:47.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.982+0000 7f300fa07640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2ff40b5450 0x7f2ff40b7910 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:47.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.982+0000 7f300fa07640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f300810d140 0x7f300819ec80 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:47.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.982+0000 7f300fa07640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3008104600 0x7f30081998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:47.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.982+0000 7f300fa07640 1 --2- 172.21.15.78:0/2271579346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3008103ce0 0x7f30081993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:06:47.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.982+0000 7f300fa07640 1 -- 172.21.15.78:0/2271579346 >> 172.21.15.78:0/2271579346 conn(0x7f30080fd720 msgr2=0x7f30080fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:06:47.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.982+0000 7f300fa07640 1 -- 172.21.15.78:0/2271579346 shutdown_connections 2024-09-11T01:06:47.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:06:46.982+0000 7f300fa07640 1 -- 172.21.15.78:0/2271579346 wait complete. 2024-09-11T01:06:47.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 556556 5.52 MB/sec execute 295 sec latency 836.097 ms 2024-09-11T01:06:47.778 INFO:tasks.cephfs.filesystem:scrub status for tag:371f8253-b1f4-400e-94c8-988b8dc35dba - {'path': '/', 'tag': '371f8253-b1f4-400e-94c8-988b8dc35dba', 'options': 'recursive,force'} 2024-09-11T01:06:48.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 558263 5.55 MB/sec execute 296 sec latency 12.456 ms 2024-09-11T01:06:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:48 smithi110 ceph-mon[34712]: pgmap v563: 97 pgs: 97 active+clean; 380 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-09-11T01:06:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:48 smithi112 ceph-mon[35384]: pgmap v563: 97 pgs: 97 active+clean; 380 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-09-11T01:06:48.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:48 smithi078 ceph-mon[31315]: pgmap v563: 97 pgs: 97 active+clean; 380 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-09-11T01:06:49.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 559397 5.54 MB/sec execute 297 sec latency 445.921 ms 2024-09-11T01:06:50.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 559397 5.53 MB/sec execute 298 sec latency 1446.020 ms 2024-09-11T01:06:50.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:50 smithi110 ceph-mon[34712]: pgmap v564: 97 pgs: 97 active+clean; 382 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-09-11T01:06:50.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:50 smithi112 ceph-mon[35384]: pgmap v564: 97 pgs: 97 active+clean; 382 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-09-11T01:06:50.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:50 smithi078 ceph-mon[31315]: pgmap v564: 97 pgs: 97 active+clean; 382 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-09-11T01:06:51.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 559397 5.51 MB/sec execute 299 sec latency 2446.076 ms 2024-09-11T01:06:52.440 INFO:tasks.workunit.client.0.smithi078.stdout: 1 560910 5.52 MB/sec execute 300 sec latency 2834.528 ms 2024-09-11T01:06:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:52 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:06:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:52 smithi112 ceph-mon[35384]: pgmap v565: 97 pgs: 97 active+clean; 384 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-09-11T01:06:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:52 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:06:52.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:52 smithi078 ceph-mon[31315]: pgmap v565: 97 pgs: 97 active+clean; 384 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-09-11T01:06:52.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:52 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:06:52.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:52 smithi110 ceph-mon[34712]: pgmap v565: 97 pgs: 97 active+clean; 384 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-09-11T01:06:53.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 562474 5.52 MB/sec execute 301 sec latency 11.909 ms 2024-09-11T01:06:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:53 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:06:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:53 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:06:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:53 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:06:53.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:53 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:06:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:53 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:06:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:53 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:06:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:53 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:06:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:53 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:06:53.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:53 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:06:53.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:53 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:06:53.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:53 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:06:53.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:53 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:06:54.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 562946 5.52 MB/sec execute 302 sec latency 774.027 ms 2024-09-11T01:06:54.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:54 smithi110 ceph-mon[34712]: pgmap v566: 97 pgs: 97 active+clean; 384 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-09-11T01:06:54.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:54 smithi112 ceph-mon[35384]: pgmap v566: 97 pgs: 97 active+clean; 384 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-09-11T01:06:54.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:54 smithi078 ceph-mon[31315]: pgmap v566: 97 pgs: 97 active+clean; 384 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-09-11T01:06:55.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 562946 5.50 MB/sec execute 303 sec latency 1774.112 ms 2024-09-11T01:06:56.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 562946 5.48 MB/sec execute 304 sec latency 2774.207 ms 2024-09-11T01:06:56.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:56 smithi110 ceph-mon[34712]: pgmap v567: 97 pgs: 97 active+clean; 387 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-09-11T01:06:56.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:56 smithi112 ceph-mon[35384]: pgmap v567: 97 pgs: 97 active+clean; 387 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-09-11T01:06:56.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:56 smithi078 ceph-mon[31315]: pgmap v567: 97 pgs: 97 active+clean; 387 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-09-11T01:06:57.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 563650 5.47 MB/sec execute 305 sec latency 3531.541 ms 2024-09-11T01:06:58.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 565267 5.49 MB/sec execute 306 sec latency 19.363 ms 2024-09-11T01:06:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:06:58 smithi110 ceph-mon[34712]: pgmap v568: 97 pgs: 97 active+clean; 388 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-09-11T01:06:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:06:58 smithi112 ceph-mon[35384]: pgmap v568: 97 pgs: 97 active+clean; 388 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-09-11T01:06:58.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:06:58 smithi078 ceph-mon[31315]: pgmap v568: 97 pgs: 97 active+clean; 388 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-09-11T01:06:59.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 566486 5.49 MB/sec execute 307 sec latency 344.855 ms 2024-09-11T01:07:00.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 566486 5.47 MB/sec execute 308 sec latency 1344.958 ms 2024-09-11T01:07:00.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:00 smithi110 ceph-mon[34712]: pgmap v569: 97 pgs: 97 active+clean; 390 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-09-11T01:07:00.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:00 smithi112 ceph-mon[35384]: pgmap v569: 97 pgs: 97 active+clean; 390 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-09-11T01:07:00.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:00 smithi078 ceph-mon[31315]: pgmap v569: 97 pgs: 97 active+clean; 390 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-09-11T01:07:01.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 566486 5.45 MB/sec execute 309 sec latency 2345.063 ms 2024-09-11T01:07:02.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 567065 5.44 MB/sec execute 310 sec latency 3098.703 ms 2024-09-11T01:07:02.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:02 smithi110 ceph-mon[34712]: pgmap v570: 97 pgs: 97 active+clean; 391 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-09-11T01:07:02.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:02 smithi112 ceph-mon[35384]: pgmap v570: 97 pgs: 97 active+clean; 391 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-09-11T01:07:02.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:02 smithi078 ceph-mon[31315]: pgmap v570: 97 pgs: 97 active+clean; 391 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-09-11T01:07:03.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 568629 5.46 MB/sec execute 311 sec latency 19.806 ms 2024-09-11T01:07:04.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 570039 5.46 MB/sec execute 312 sec latency 82.361 ms 2024-09-11T01:07:04.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:04 smithi110 ceph-mon[34712]: pgmap v571: 97 pgs: 97 active+clean; 393 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-09-11T01:07:04.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:04 smithi112 ceph-mon[35384]: pgmap v571: 97 pgs: 97 active+clean; 393 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-09-11T01:07:04.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:04 smithi078 ceph-mon[31315]: pgmap v571: 97 pgs: 97 active+clean; 393 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-09-11T01:07:05.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 572255 5.49 MB/sec execute 313 sec latency 14.534 ms 2024-09-11T01:07:06.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 573579 5.49 MB/sec execute 314 sec latency 208.139 ms 2024-09-11T01:07:06.526 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:06 smithi112 ceph-mon[35384]: pgmap v572: 97 pgs: 97 active+clean; 396 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.3 MiB/s wr, 206 op/s 2024-09-11T01:07:06.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:06 smithi110 ceph-mon[34712]: pgmap v572: 97 pgs: 97 active+clean; 396 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.3 MiB/s wr, 206 op/s 2024-09-11T01:07:06.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:06 smithi078 ceph-mon[31315]: pgmap v572: 97 pgs: 97 active+clean; 396 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.3 MiB/s wr, 206 op/s 2024-09-11T01:07:07.441 INFO:tasks.workunit.client.0.smithi078.stdout: 1 574283 5.48 MB/sec execute 315 sec latency 958.983 ms 2024-09-11T01:07:08.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 576008 5.50 MB/sec execute 316 sec latency 9.865 ms 2024-09-11T01:07:08.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:08 smithi110 ceph-mon[34712]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T01:07:08.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:08 smithi110 ceph-mon[34712]: pgmap v574: 97 pgs: 97 active+clean; 397 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.2 MiB/s wr, 202 op/s 2024-09-11T01:07:08.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:08 smithi112 ceph-mon[35384]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T01:07:08.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:08 smithi112 ceph-mon[35384]: pgmap v574: 97 pgs: 97 active+clean; 397 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.2 MiB/s wr, 202 op/s 2024-09-11T01:07:08.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:08 smithi078 ceph-mon[31315]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T01:07:08.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:08 smithi078 ceph-mon[31315]: pgmap v574: 97 pgs: 97 active+clean; 397 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.2 MiB/s wr, 202 op/s 2024-09-11T01:07:09.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 577126 5.50 MB/sec execute 317 sec latency 395.806 ms 2024-09-11T01:07:09.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:09 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:07:09.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:09 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:07:09.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:09 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:07:10.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 577126 5.48 MB/sec execute 318 sec latency 1395.888 ms 2024-09-11T01:07:10.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:07:10.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:07:10.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:07:10.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:10 smithi110 ceph-mon[34712]: pgmap v575: 97 pgs: 97 active+clean; 398 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-09-11T01:07:10.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:07:10.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:07:10.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:07:10.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:10 smithi112 ceph-mon[35384]: pgmap v575: 97 pgs: 97 active+clean; 398 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-09-11T01:07:10.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:07:10.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:07:10.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:07:10.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:10 smithi078 ceph-mon[31315]: pgmap v575: 97 pgs: 97 active+clean; 398 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-09-11T01:07:11.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 577126 5.46 MB/sec execute 319 sec latency 2395.987 ms 2024-09-11T01:07:12.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 578771 5.47 MB/sec execute 320 sec latency 2781.474 ms 2024-09-11T01:07:12.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:12 smithi110 ceph-mon[34712]: pgmap v576: 97 pgs: 97 active+clean; 399 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-09-11T01:07:12.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:12 smithi112 ceph-mon[35384]: pgmap v576: 97 pgs: 97 active+clean; 399 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-09-11T01:07:12.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:12 smithi078 ceph-mon[31315]: pgmap v576: 97 pgs: 97 active+clean; 399 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-09-11T01:07:13.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 580133 5.48 MB/sec execute 321 sec latency 21.485 ms 2024-09-11T01:07:14.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 580669 5.47 MB/sec execute 322 sec latency 706.953 ms 2024-09-11T01:07:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:14 smithi110 ceph-mon[34712]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T01:07:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:14 smithi110 ceph-mon[34712]: pgmap v578: 97 pgs: 97 active+clean; 401 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-09-11T01:07:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:14 smithi112 ceph-mon[35384]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T01:07:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:14 smithi112 ceph-mon[35384]: pgmap v578: 97 pgs: 97 active+clean; 401 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-09-11T01:07:14.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:14 smithi078 ceph-mon[31315]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T01:07:14.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:14 smithi078 ceph-mon[31315]: pgmap v578: 97 pgs: 97 active+clean; 401 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-09-11T01:07:15.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 580669 5.45 MB/sec execute 323 sec latency 1707.015 ms 2024-09-11T01:07:16.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 580669 5.44 MB/sec execute 324 sec latency 2707.111 ms 2024-09-11T01:07:16.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:16 smithi110 ceph-mon[34712]: pgmap v579: 97 pgs: 97 active+clean; 404 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 198 op/s 2024-09-11T01:07:16.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:16 smithi112 ceph-mon[35384]: pgmap v579: 97 pgs: 97 active+clean; 404 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 198 op/s 2024-09-11T01:07:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:16 smithi078 ceph-mon[31315]: pgmap v579: 97 pgs: 97 active+clean; 404 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 198 op/s 2024-09-11T01:07:17.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 581379 5.42 MB/sec execute 325 sec latency 3462.697 ms 2024-09-11T01:07:17.780 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:07:18.077 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:07:18.442 INFO:tasks.workunit.client.0.smithi078.stdout: 1 582956 5.45 MB/sec execute 326 sec latency 29.760 ms 2024-09-11T01:07:18.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:18 smithi078 ceph-mon[31315]: pgmap v580: 97 pgs: 97 active+clean; 405 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 170 op/s 2024-09-11T01:07:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.623+0000 7f1e937ad640 1 -- 172.21.15.78:0/824548710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e8c10d0e0 msgr2=0x7f1e8c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.623+0000 7f1e937ad640 1 --2- 172.21.15.78:0/824548710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e8c10d0e0 0x7f1e8c10f550 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009ca0 tx=0x7f1e8402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.623+0000 7f1e937ad640 1 -- 172.21.15.78:0/824548710 shutdown_connections 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.623+0000 7f1e937ad640 1 --2- 172.21.15.78:0/824548710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e8c10d0e0 0x7f1e8c10f550 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.623+0000 7f1e937ad640 1 --2- 172.21.15.78:0/824548710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e8c104530 0x7f1e8c10cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.623+0000 7f1e937ad640 1 --2- 172.21.15.78:0/824548710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8c103b30 0x7f1e8c103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.623+0000 7f1e937ad640 1 -- 172.21.15.78:0/824548710 >> 172.21.15.78:0/824548710 conn(0x7f1e8c0fd700 msgr2=0x7f1e8c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.623+0000 7f1e937ad640 1 -- 172.21.15.78:0/824548710 shutdown_connections 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.623+0000 7f1e937ad640 1 -- 172.21.15.78:0/824548710 wait complete. 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e937ad640 1 Processor -- start 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e937ad640 1 -- start start 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e937ad640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e8c103b30 0x7f1e8c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e937ad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8c104530 0x7f1e8c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e937ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e8c10d0e0 0x7f1e8c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e937ad640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c112d60 con 0x7f1e8c103b30 2024-09-11T01:07:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e937ad640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c112be0 con 0x7f1e8c104530 2024-09-11T01:07:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e937ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c112ee0 con 0x7f1e8c10d0e0 2024-09-11T01:07:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e91549640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e8c103b30 0x7f1e8c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e90d48640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8c104530 0x7f1e8c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e91549640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e8c103b30 0x7f1e8c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53584/0 (socket says 172.21.15.78:53584) 2024-09-11T01:07:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e91549640 1 -- 172.21.15.78:0/3259902488 learned_addr learned my addr 172.21.15.78:0/3259902488 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e91d4a640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e8c10d0e0 0x7f1e8c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e90d48640 1 -- 172.21.15.78:0/3259902488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e8c10d0e0 msgr2=0x7f1e8c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e90d48640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e8c10d0e0 0x7f1e8c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e90d48640 1 -- 172.21.15.78:0/3259902488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e8c103b30 msgr2=0x7f1e8c199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e90d48640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e8c103b30 0x7f1e8c199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.624+0000 7f1e90d48640 1 -- 172.21.15.78:0/3259902488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c19f2c0 con 0x7f1e8c104530 2024-09-11T01:07:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.625+0000 7f1e90d48640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8c104530 0x7f1e8c199750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00e9f0 tx=0x7f1e7c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:18.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.625+0000 7f1e827fc640 1 -- 172.21.15.78:0/3259902488 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c00ccc0 con 0x7f1e8c104530 2024-09-11T01:07:18.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.625+0000 7f1e937ad640 1 -- 172.21.15.78:0/3259902488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c19f5b0 con 0x7f1e8c104530 2024-09-11T01:07:18.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.625+0000 7f1e827fc640 1 -- 172.21.15.78:0/3259902488 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e7c005020 con 0x7f1e8c104530 2024-09-11T01:07:18.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.625+0000 7f1e827fc640 1 -- 172.21.15.78:0/3259902488 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c004a00 con 0x7f1e8c104530 2024-09-11T01:07:18.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.625+0000 7f1e937ad640 1 -- 172.21.15.78:0/3259902488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c101890 con 0x7f1e8c104530 2024-09-11T01:07:18.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.627+0000 7f1e95b82740 1 Processor -- start 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.627+0000 7f1e827fc640 1 -- 172.21.15.78:0/3259902488 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1e7c004ba0 con 0x7f1e8c104530 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.627+0000 7f1e95b82740 1 -- start start 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e95b82740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e0a0 0x5629e1b6e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e95b82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 0x5629e1b7a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e95b82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b7a950 0x5629e1b7cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e95b82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5629e1ae4e50 con 0x5629e1b6e9c0 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e95b82740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5629e1ad4280 con 0x5629e1b6e0a0 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e95b82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5629e1ad8090 con 0x5629e1b7a950 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e91549640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 0x5629e1b7a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e91d4a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e0a0 0x5629e1b6e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e91549640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 0x5629e1b7a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53592/0 (socket says 172.21.15.78:53592) 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e91549640 1 -- 172.21.15.78:0/634949787 learned_addr learned my addr 172.21.15.78:0/634949787 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e90d48640 1 --2- 172.21.15.78:0/634949787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b7a950 0x5629e1b7cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e91549640 1 -- 172.21.15.78:0/634949787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b7a950 msgr2=0x5629e1b7cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e91549640 1 --2- 172.21.15.78:0/634949787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b7a950 0x5629e1b7cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e91549640 1 -- 172.21.15.78:0/634949787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e0a0 msgr2=0x5629e1b6e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e91549640 1 --2- 172.21.15.78:0/634949787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e0a0 0x5629e1b6e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e91549640 1 -- 172.21.15.78:0/634949787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629e1832650 con 0x5629e1b6e9c0 2024-09-11T01:07:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e91d4a640 1 --2- 172.21.15.78:0/634949787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e0a0 0x5629e1b6e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e827fc640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1e780b5450 0x7f1e780b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.628+0000 7f1e91549640 1 --2- 172.21.15.78:0/634949787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 0x5629e1b7a410 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009dc0 tx=0x7f1e88011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8159475a18a16 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:18.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e91d4a640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1e780b5450 0x7f1e780b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e777fe640 1 -- 172.21.15.78:0/634949787 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88005220 con 0x5629e1b6e9c0 2024-09-11T01:07:18.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e777fe640 1 -- 172.21.15.78:0/634949787 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e88004c50 con 0x5629e1b6e9c0 2024-09-11T01:07:18.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e777fe640 1 -- 172.21.15.78:0/634949787 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88019790 con 0x5629e1b6e9c0 2024-09-11T01:07:18.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e827fc640 1 -- 172.21.15.78:0/3259902488 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f1e7c0e5f70 con 0x7f1e8c104530 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e95b82740 1 -- 172.21.15.78:0/634949787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 msgr2=0x5629e1b7a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e91d4a640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1e780b5450 0x7f1e780b7910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1e8402f500 tx=0x7f1e84005f50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e95b82740 1 --2- 172.21.15.78:0/634949787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 0x5629e1b7a410 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009dc0 tx=0x7f1e88011810 comp rx=0 tx=0).stop 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e95b82740 1 -- 172.21.15.78:0/634949787 shutdown_connections 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e95b82740 1 --2- 172.21.15.78:0/634949787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b7a950 0x5629e1b7cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e95b82740 1 --2- 172.21.15.78:0/634949787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 0x5629e1b7a410 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e95b82740 1 --2- 172.21.15.78:0/634949787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e0a0 0x5629e1b6e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e95b82740 1 -- 172.21.15.78:0/634949787 >> 172.21.15.78:0/634949787 conn(0x5629e1b6bed0 msgr2=0x5629e1b6d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.629+0000 7f1e95b82740 1 -- 172.21.15.78:0/634949787 shutdown_connections 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.630+0000 7f1e95b82740 1 -- 172.21.15.78:0/634949787 wait complete. 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.630+0000 7f1e95b82740 1 -- start start 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.630+0000 7f1e95b82740 1 Processor -- start 2024-09-11T01:07:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.630+0000 7f1e95b82740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e0a0 0x5629e1b7a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.630+0000 7f1e95b82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e9c0 0x5629e1b77680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.630+0000 7f1e95b82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b7a950 0x5629e1b77cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.630+0000 7f1e95b82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5629e1935700 con 0x5629e1b7a950 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.630+0000 7f1e95b82740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5629e1a6f110 con 0x5629e1b6e0a0 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.630+0000 7f1e95b82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5629e1b7d5d0 con 0x5629e1b6e9c0 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e95b82740 10 client.0 _ll_register_callbacks cb 0x5629e1a88990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e91d4a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b7a950 0x5629e1b77cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e91549640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e0a0 0x5629e1b7a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e90d48640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e9c0 0x5629e1b77680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e91549640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e0a0 0x5629e1b7a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:40278/0 (socket says 172.21.15.78:40278) 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e91d4a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b7a950 0x5629e1b77cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53606/0 (socket says 172.21.15.78:53606) 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e91549640 1 -- 172.21.15.78:0/251227149 learned_addr learned my addr 172.21.15.78:0/251227149 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e90d48640 1 -- 172.21.15.78:0/251227149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e0a0 msgr2=0x5629e1b7a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e90d48640 1 --2- 172.21.15.78:0/251227149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e0a0 0x5629e1b7a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e90d48640 1 -- 172.21.15.78:0/251227149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b7a950 msgr2=0x5629e1b77cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e90d48640 1 --2- 172.21.15.78:0/251227149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b7a950 0x5629e1b77cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e90d48640 1 -- 172.21.15.78:0/251227149 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629e1a9cc40 con 0x5629e1b6e9c0 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e90d48640 1 --2- 172.21.15.78:0/251227149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e9c0 0x5629e1b77680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0e36e0 tx=0x7f1e7c009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42b1a01bac47a575 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e74ff9640 1 -- 172.21.15.78:0/251227149 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c01c040 con 0x5629e1b6e9c0 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e95b82740 1 -- 172.21.15.78:0/251227149 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e88004c50 con 0x5629e1b6e9c0 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e74ff9640 1 -- 172.21.15.78:0/251227149 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f1e7c0054f0 con 0x5629e1b6e9c0 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e74ff9640 1 -- 172.21.15.78:0/251227149 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1e7c0ac1e0 con 0x5629e1b6e9c0 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e95b82740 10 client.26239 fetch_fsmap learned FSMap version 45 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.631+0000 7f1e95b82740 1 -- 172.21.15.78:0/251227149 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5629e1b88f40 con 0x5629e1b6e9c0 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.632+0000 7f1e74ff9640 1 -- 172.21.15.78:0/251227149 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e88004c50 con 0x5629e1b6e9c0 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.632+0000 7f1e74ff9640 1 -- 172.21.15.78:0/251227149 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f1e7c0e3960 con 0x5629e1b6e9c0 2024-09-11T01:07:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.632+0000 7f1e95b82740 10 client.26239 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.632+0000 7f1e95b82740 10 client.26239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.632+0000 7f1e95b82740 20 client.26239 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.632+0000 7f1e95b82740 1 --2- 172.21.15.78:0/251227149 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629e1b8a350 0x5629e1b8c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.632+0000 7f1e95b82740 4 client.26239 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.632+0000 7f1e95b82740 1 -- 172.21.15.78:0/251227149 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629e1ae6da0 con 0x5629e1b8a350 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.632+0000 7f1e91549640 1 --2- 172.21.15.78:0/251227149 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629e1b8a350 0x5629e1b8c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.632+0000 7f1e91549640 1 --2- 172.21.15.78:0/251227149 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629e1b8a350 0x5629e1b8c790 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e2d3e565ffbafb4 server_cookie=f240fbd9c7ad25a5 in_seq=0 out_seq=0 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.632+0000 7f1e74ff9640 10 client.26239 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e74ff9640 1 -- 172.21.15.78:0/251227149 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5629e1ae6da0 con 0x5629e1b8a350 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e74ff9640 10 client.26239 handle_command_reply: tid=0 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e95b82740 1 client.26239 shutdown 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e95b82740 1 -- 172.21.15.78:0/251227149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e9c0 msgr2=0x5629e1b77680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e95b82740 1 --2- 172.21.15.78:0/251227149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e9c0 0x5629e1b77680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0e36e0 tx=0x7f1e7c009f60 comp rx=0 tx=0).stop 2024-09-11T01:07:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e95b82740 1 -- 172.21.15.78:0/251227149 shutdown_connections 2024-09-11T01:07:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e95b82740 1 --2- 172.21.15.78:0/251227149 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629e1b8a350 0x5629e1b8c790 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e95b82740 1 --2- 172.21.15.78:0/251227149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b7a950 0x5629e1b77cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e95b82740 1 --2- 172.21.15.78:0/251227149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e9c0 0x5629e1b77680 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e95b82740 1 --2- 172.21.15.78:0/251227149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e0a0 0x5629e1b7a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e74ff9640 0 client.26239 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e95b82740 1 -- 172.21.15.78:0/251227149 >> 172.21.15.78:0/251227149 conn(0x5629e1b6bed0 msgr2=0x5629e1b71a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e95b82740 1 -- 172.21.15.78:0/251227149 shutdown_connections 2024-09-11T01:07:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.633+0000 7f1e95b82740 1 -- 172.21.15.78:0/251227149 wait complete. 2024-09-11T01:07:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.634+0000 7f1e95b82740 20 client.26239 trim_cache size 0 max 16384 2024-09-11T01:07:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.652+0000 7f1e95b82740 1 Processor -- start 2024-09-11T01:07:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.652+0000 7f1e95b82740 1 -- start start 2024-09-11T01:07:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.652+0000 7f1e95b82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e0a0 0x5629e1b86800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.652+0000 7f1e95b82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 0x5629e1b793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.652+0000 7f1e95b82740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b7a950 0x5629e1b7bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.652+0000 7f1e95b82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1b4b50 con 0x5629e1b6e9c0 2024-09-11T01:07:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.652+0000 7f1e95b82740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1b6720 con 0x5629e1b7a950 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.652+0000 7f1e95b82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1b5220 con 0x5629e1b6e0a0 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e91549640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e0a0 0x5629e1b86800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e90d48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 0x5629e1b793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e91549640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e0a0 0x5629e1b86800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:34472/0 (socket says 172.21.15.78:34472) 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e91549640 1 -- 172.21.15.78:0/2284571974 learned_addr learned my addr 172.21.15.78:0/2284571974 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e90d48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 0x5629e1b793c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53622/0 (socket says 172.21.15.78:53622) 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e91d4a640 1 --2- 172.21.15.78:0/2284571974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b7a950 0x5629e1b7bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e90d48640 1 -- 172.21.15.78:0/2284571974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e0a0 msgr2=0x5629e1b86800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e90d48640 1 --2- 172.21.15.78:0/2284571974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e0a0 0x5629e1b86800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e90d48640 1 -- 172.21.15.78:0/2284571974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b7a950 msgr2=0x5629e1b7bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e90d48640 1 --2- 172.21.15.78:0/2284571974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b7a950 0x5629e1b7bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e90d48640 1 -- 172.21.15.78:0/2284571974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629e1b6e7f0 con 0x5629e1b6e9c0 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e91549640 1 --2- 172.21.15.78:0/2284571974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e0a0 0x5629e1b86800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e90d48640 1 --2- 172.21.15.78:0/2284571974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 0x5629e1b793c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0ac3a0 tx=0x7f1e7c0096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb881550d84169f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:18.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e76ffd640 1 -- 172.21.15.78:0/2284571974 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c01c070 con 0x5629e1b6e9c0 2024-09-11T01:07:18.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e76ffd640 1 -- 172.21.15.78:0/2284571974 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e7c0abf20 con 0x5629e1b6e9c0 2024-09-11T01:07:18.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.653+0000 7f1e76ffd640 1 -- 172.21.15.78:0/2284571974 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0abb80 con 0x5629e1b6e9c0 2024-09-11T01:07:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.654+0000 7f1e95b82740 1 -- 172.21.15.78:0/2284571974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 msgr2=0x5629e1b793c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.654+0000 7f1e95b82740 1 --2- 172.21.15.78:0/2284571974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 0x5629e1b793c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0ac3a0 tx=0x7f1e7c0096e0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.654+0000 7f1e95b82740 1 -- 172.21.15.78:0/2284571974 shutdown_connections 2024-09-11T01:07:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.654+0000 7f1e95b82740 1 --2- 172.21.15.78:0/2284571974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b7a950 0x5629e1b7bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.654+0000 7f1e95b82740 1 --2- 172.21.15.78:0/2284571974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e9c0 0x5629e1b793c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.654+0000 7f1e95b82740 1 --2- 172.21.15.78:0/2284571974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b6e0a0 0x5629e1b86800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.654+0000 7f1e95b82740 1 -- 172.21.15.78:0/2284571974 >> 172.21.15.78:0/2284571974 conn(0x5629e1b6bed0 msgr2=0x5629e1b78cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.655+0000 7f1e95b82740 1 -- 172.21.15.78:0/2284571974 shutdown_connections 2024-09-11T01:07:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.655+0000 7f1e95b82740 1 -- 172.21.15.78:0/2284571974 wait complete. 2024-09-11T01:07:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.655+0000 7f1e95b82740 1 -- start start 2024-09-11T01:07:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.655+0000 7f1e95b82740 1 Processor -- start 2024-09-11T01:07:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.655+0000 7f1e95b82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e0a0 0x5629e1b82d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.655+0000 7f1e95b82740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e9c0 0x5629e1b832a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.655+0000 7f1e95b82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b7a950 0x5629e1b7da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.655+0000 7f1e95b82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1b6020 con 0x5629e1b6e0a0 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.655+0000 7f1e95b82740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1b5950 con 0x5629e1b6e9c0 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.655+0000 7f1e95b82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1b2b50 con 0x5629e1b7a950 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.655+0000 7f1e95b82740 10 client.0 _ll_register_callbacks cb 0x7f1e7c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.655+0000 7f1e91549640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e0a0 0x5629e1b82d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e91549640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e0a0 0x5629e1b82d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53638/0 (socket says 172.21.15.78:53638) 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e91549640 1 -- 172.21.15.78:0/339159064 learned_addr learned my addr 172.21.15.78:0/339159064 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e90d48640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e9c0 0x5629e1b832a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e91d4a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b7a950 0x5629e1b7da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e91549640 1 -- 172.21.15.78:0/339159064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b7a950 msgr2=0x5629e1b7da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e91549640 1 --2- 172.21.15.78:0/339159064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b7a950 0x5629e1b7da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e91549640 1 -- 172.21.15.78:0/339159064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e9c0 msgr2=0x5629e1b832a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e91549640 1 --2- 172.21.15.78:0/339159064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e9c0 0x5629e1b832a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e91549640 1 -- 172.21.15.78:0/339159064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629e1b7e2a0 con 0x5629e1b6e0a0 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e91549640 1 --2- 172.21.15.78:0/339159064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e0a0 0x5629e1b82d40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1e88011d10 tx=0x7f1e88005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d446336f48a3292 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e74ff9640 1 -- 172.21.15.78:0/339159064 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e880099d0 con 0x5629e1b6e0a0 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e74ff9640 1 -- 172.21.15.78:0/339159064 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f1e88005150 con 0x5629e1b6e0a0 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e95b82740 1 -- 172.21.15.78:0/339159064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e7c0abf20 con 0x5629e1b6e0a0 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e74ff9640 1 -- 172.21.15.78:0/339159064 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1e8803a050 con 0x5629e1b6e0a0 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e95b82740 10 client.17043 fetch_fsmap learned FSMap version 45 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.656+0000 7f1e95b82740 1 -- 172.21.15.78:0/339159064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5629e1bf7fe0 con 0x5629e1b6e0a0 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.657+0000 7f1e74ff9640 1 -- 172.21.15.78:0/339159064 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e7c0abf20 con 0x5629e1b6e0a0 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.657+0000 7f1e74ff9640 1 -- 172.21.15.78:0/339159064 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7f1e880364d0 con 0x5629e1b6e0a0 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.657+0000 7f1e95b82740 10 client.17043 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T01:07:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.657+0000 7f1e95b82740 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.657+0000 7f1e95b82740 20 client.17043 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.657+0000 7f1e95b82740 1 --2- 172.21.15.78:0/339159064 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629e1bf8530 0x5629e1bfa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.657+0000 7f1e95b82740 4 client.17043 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.657+0000 7f1e95b82740 1 -- 172.21.15.78:0/339159064 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629e1ae2850 con 0x5629e1bf8530 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.657+0000 7f1e90d48640 1 --2- 172.21.15.78:0/339159064 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629e1bf8530 0x5629e1bfa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.657+0000 7f1e74ff9640 10 client.17043 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.657+0000 7f1e90d48640 1 --2- 172.21.15.78:0/339159064 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629e1bf8530 0x5629e1bfa950 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5988dbf9ea21e6b4 server_cookie=d62087600ab0ade1 in_seq=0 out_seq=0 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e74ff9640 1 -- 172.21.15.78:0/339159064 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x5629e1ae2850 con 0x5629e1bf8530 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e74ff9640 10 client.17043 handle_command_reply: tid=0 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e95b82740 1 client.17043 shutdown 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e95b82740 1 -- 172.21.15.78:0/339159064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e0a0 msgr2=0x5629e1b82d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e95b82740 1 --2- 172.21.15.78:0/339159064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e0a0 0x5629e1b82d40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1e88011d10 tx=0x7f1e88005a90 comp rx=0 tx=0).stop 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e95b82740 1 -- 172.21.15.78:0/339159064 shutdown_connections 2024-09-11T01:07:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e95b82740 1 --2- 172.21.15.78:0/339159064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629e1b7a950 0x5629e1b7da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e95b82740 1 --2- 172.21.15.78:0/339159064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629e1b6e9c0 0x5629e1b832a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e95b82740 1 --2- 172.21.15.78:0/339159064 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5629e1bf8530 0x5629e1bfa950 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e95b82740 1 --2- 172.21.15.78:0/339159064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629e1b6e0a0 0x5629e1b82d40 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e74ff9640 0 client.17043 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:18.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e95b82740 1 -- 172.21.15.78:0/339159064 >> 172.21.15.78:0/339159064 conn(0x5629e1b6bed0 msgr2=0x5629e1b794d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:18.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e95b82740 1 -- 172.21.15.78:0/339159064 shutdown_connections 2024-09-11T01:07:18.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e95b82740 1 -- 172.21.15.78:0/339159064 wait complete. 2024-09-11T01:07:18.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e95b82740 20 client.17043 trim_cache size 0 max 16384 2024-09-11T01:07:18.665 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:07:18.665 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:18.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:07:18.667 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:18.668 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:18.669 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:18.670 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:18.671 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:18.672 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:18.672 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:18.672 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:18.672 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:07:18.672 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:07:18.672 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:07:18.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e937ad640 1 -- 172.21.15.78:0/3259902488 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1e780b5450 msgr2=0x7f1e780b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:18.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e937ad640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1e780b5450 0x7f1e780b7910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1e8402f500 tx=0x7f1e84005f50 comp rx=0 tx=0).stop 2024-09-11T01:07:18.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e937ad640 1 -- 172.21.15.78:0/3259902488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8c104530 msgr2=0x7f1e8c199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:18.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.658+0000 7f1e937ad640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8c104530 0x7f1e8c199750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00e9f0 tx=0x7f1e7c00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.659+0000 7f1e937ad640 1 -- 172.21.15.78:0/3259902488 shutdown_connections 2024-09-11T01:07:18.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.659+0000 7f1e937ad640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1e780b5450 0x7f1e780b7910 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.659+0000 7f1e937ad640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e8c10d0e0 0x7f1e8c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.659+0000 7f1e937ad640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8c104530 0x7f1e8c199750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.659+0000 7f1e937ad640 1 --2- 172.21.15.78:0/3259902488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e8c103b30 0x7f1e8c199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:18.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.659+0000 7f1e937ad640 1 -- 172.21.15.78:0/3259902488 >> 172.21.15.78:0/3259902488 conn(0x7f1e8c0fd700 msgr2=0x7f1e8c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:18.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.659+0000 7f1e937ad640 1 -- 172.21.15.78:0/3259902488 shutdown_connections 2024-09-11T01:07:18.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:18.659+0000 7f1e937ad640 1 -- 172.21.15.78:0/3259902488 wait complete. 2024-09-11T01:07:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:18 smithi110 ceph-mon[34712]: pgmap v580: 97 pgs: 97 active+clean; 405 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 170 op/s 2024-09-11T01:07:18.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:18 smithi112 ceph-mon[35384]: pgmap v580: 97 pgs: 97 active+clean; 405 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 170 op/s 2024-09-11T01:07:19.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 584222 5.44 MB/sec execute 327 sec latency 267.854 ms 2024-09-11T01:07:19.866 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:07:19.866 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:07:20.116 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:07:20.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 584222 5.43 MB/sec execute 328 sec latency 1268.154 ms 2024-09-11T01:07:20.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:20 smithi078 ceph-mon[31315]: pgmap v581: 97 pgs: 97 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-11T01:07:20.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.570+0000 7fa6fa693640 1 -- 172.21.15.78:0/3971135104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f4108040 msgr2=0x7fa6f410e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.570+0000 7fa6fa693640 1 --2- 172.21.15.78:0/3971135104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f4108040 0x7fa6f410e840 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009ca0 tx=0x7fa6ec02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:07:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.570+0000 7fa6fa693640 1 -- 172.21.15.78:0/3971135104 shutdown_connections 2024-09-11T01:07:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.570+0000 7fa6fa693640 1 --2- 172.21.15.78:0/3971135104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f4108040 0x7fa6f410e840 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.570+0000 7fa6fa693640 1 --2- 172.21.15.78:0/3971135104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f41076c0 0x7fa6f4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.570+0000 7fa6fa693640 1 --2- 172.21.15.78:0/3971135104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6f4101cb0 0x7fa6f4102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.570+0000 7fa6fa693640 1 -- 172.21.15.78:0/3971135104 >> 172.21.15.78:0/3971135104 conn(0x7fa6f40fd700 msgr2=0x7fa6f40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.570+0000 7fa6fa693640 1 -- 172.21.15.78:0/3971135104 shutdown_connections 2024-09-11T01:07:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.570+0000 7fa6fa693640 1 -- 172.21.15.78:0/3971135104 wait complete. 2024-09-11T01:07:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.570+0000 7fa6fa693640 1 Processor -- start 2024-09-11T01:07:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.570+0000 7fa6fa693640 1 -- start start 2024-09-11T01:07:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6fa693640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f4101cb0 0x7fa6f419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6fa693640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f41076c0 0x7fa6f419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6fa693640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6f4108040 0x7fa6f41a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6fa693640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4111d00 con 0x7fa6f4101cb0 2024-09-11T01:07:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6fa693640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4111b80 con 0x7fa6f4108040 2024-09-11T01:07:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6fa693640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4111e80 con 0x7fa6f41076c0 2024-09-11T01:07:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f4101cb0 0x7fa6f419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f4101cb0 0x7fa6f419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53652/0 (socket says 172.21.15.78:53652) 2024-09-11T01:07:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f3fff640 1 -- 172.21.15.78:0/1392088301 learned_addr learned my addr 172.21.15.78:0/1392088301 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f37fe640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f41076c0 0x7fa6f419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f8c30640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6f4108040 0x7fa6f41a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f3fff640 1 -- 172.21.15.78:0/1392088301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f41076c0 msgr2=0x7fa6f419b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f3fff640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f41076c0 0x7fa6f419b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f3fff640 1 -- 172.21.15.78:0/1392088301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6f4108040 msgr2=0x7fa6f41a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f3fff640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6f4108040 0x7fa6f41a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f3fff640 1 -- 172.21.15.78:0/1392088301 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f41a1450 con 0x7fa6f4101cb0 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f37fe640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f41076c0 0x7fa6f419b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f3fff640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f4101cb0 0x7fa6f419b3a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400c890 tx=0x7fa6e400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f17fa640 1 -- 172.21.15.78:0/1392088301 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4004220 con 0x7fa6f4101cb0 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f17fa640 1 -- 172.21.15.78:0/1392088301 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6e40043c0 con 0x7fa6f4101cb0 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6fa693640 1 -- 172.21.15.78:0/1392088301 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f41a1740 con 0x7fa6f4101cb0 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.571+0000 7fa6f17fa640 1 -- 172.21.15.78:0/1392088301 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4004a00 con 0x7fa6f4101cb0 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.572+0000 7fa6fa693640 1 -- 172.21.15.78:0/1392088301 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f41a7e00 con 0x7fa6f4101cb0 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.572+0000 7fa6fa693640 1 -- 172.21.15.78:0/1392088301 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f4102090 con 0x7fa6f4101cb0 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.574+0000 7fa6f17fa640 1 -- 172.21.15.78:0/1392088301 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa6e40079b0 con 0x7fa6f4101cb0 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.574+0000 7fa6f17fa640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa6e00b5450 0x7fa6e00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.575+0000 7fa6f37fe640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa6e00b5450 0x7fa6e00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:20.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.575+0000 7fa6f17fa640 1 -- 172.21.15.78:0/1392088301 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fa6e40e6180 con 0x7fa6f4101cb0 2024-09-11T01:07:20.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.575+0000 7fa6f37fe640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa6e00b5450 0x7fa6e00b7910 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa6f41070e0 tx=0x7fa6dc00a5c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:20.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.577+0000 7fa6f17fa640 1 -- 172.21.15.78:0/1392088301 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa6e4014030 con 0x7fa6f4101cb0 2024-09-11T01:07:20.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:20 smithi110 ceph-mon[34712]: pgmap v581: 97 pgs: 97 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-11T01:07:20.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:20 smithi112 ceph-mon[35384]: pgmap v581: 97 pgs: 97 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-11T01:07:20.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.790+0000 7fa6fa693640 1 -- 172.21.15.78:0/1392088301 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6f419c4d0 con 0x7fa6f4101cb0 2024-09-11T01:07:20.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.790+0000 7fa6f17fa640 1 -- 172.21.15.78:0/1392088301 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11983 (secure 0 0 0) 0x7fa6e40acd40 con 0x7fa6f4101cb0 2024-09-11T01:07:20.793 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:07:20.793 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":45,"btime":"2024-09-11T01:06:27:244567+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:06:27.108442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T01:07:20.793 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 45 2024-09-11T01:07:20.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.792+0000 7fa6fa693640 1 -- 172.21.15.78:0/1392088301 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa6e00b5450 msgr2=0x7fa6e00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.792+0000 7fa6fa693640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa6e00b5450 0x7fa6e00b7910 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa6f41070e0 tx=0x7fa6dc00a5c0 comp rx=0 tx=0).stop 2024-09-11T01:07:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.792+0000 7fa6fa693640 1 -- 172.21.15.78:0/1392088301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f4101cb0 msgr2=0x7fa6f419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.792+0000 7fa6fa693640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f4101cb0 0x7fa6f419b3a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400c890 tx=0x7fa6e400cd50 comp rx=0 tx=0).stop 2024-09-11T01:07:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.793+0000 7fa6fa693640 1 -- 172.21.15.78:0/1392088301 shutdown_connections 2024-09-11T01:07:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.793+0000 7fa6fa693640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa6e00b5450 0x7fa6e00b7910 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.793+0000 7fa6fa693640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6f4108040 0x7fa6f41a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.793+0000 7fa6fa693640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f41076c0 0x7fa6f419b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.793+0000 7fa6fa693640 1 --2- 172.21.15.78:0/1392088301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f4101cb0 0x7fa6f419b3a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.793+0000 7fa6fa693640 1 -- 172.21.15.78:0/1392088301 >> 172.21.15.78:0/1392088301 conn(0x7fa6f40fd700 msgr2=0x7fa6f406b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.793+0000 7fa6fa693640 1 -- 172.21.15.78:0/1392088301 shutdown_connections 2024-09-11T01:07:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:20.793+0000 7fa6fa693640 1 -- 172.21.15.78:0/1392088301 wait complete. 2024-09-11T01:07:21.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 584222 5.41 MB/sec execute 329 sec latency 2268.222 ms 2024-09-11T01:07:21.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:21 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1392088301' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:07:21.508 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:07:21.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:21 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1392088301' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:07:21.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:21 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1392088301' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:07:21.760 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:07:22.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.270+0000 7f61312c7640 1 -- 172.21.15.78:0/3238492491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f612c1076a0 msgr2=0x7f612c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.270+0000 7f61312c7640 1 --2- 172.21.15.78:0/3238492491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f612c1076a0 0x7f612c107ae0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6114009900 tx=0x7f611402a410 comp rx=0 tx=0).stop 2024-09-11T01:07:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.271+0000 7f61312c7640 1 -- 172.21.15.78:0/3238492491 shutdown_connections 2024-09-11T01:07:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.271+0000 7f61312c7640 1 --2- 172.21.15.78:0/3238492491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c108020 0x7f612c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.271+0000 7f61312c7640 1 --2- 172.21.15.78:0/3238492491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f612c1076a0 0x7f612c107ae0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.271+0000 7f61312c7640 1 --2- 172.21.15.78:0/3238492491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f612c101c90 0x7f612c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.271+0000 7f61312c7640 1 -- 172.21.15.78:0/3238492491 >> 172.21.15.78:0/3238492491 conn(0x7f612c0fd700 msgr2=0x7f612c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.271+0000 7f61312c7640 1 -- 172.21.15.78:0/3238492491 shutdown_connections 2024-09-11T01:07:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.271+0000 7f61312c7640 1 -- 172.21.15.78:0/3238492491 wait complete. 2024-09-11T01:07:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.271+0000 7f61312c7640 1 Processor -- start 2024-09-11T01:07:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61312c7640 1 -- start start 2024-09-11T01:07:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61312c7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f612c101c90 0x7f612c19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61312c7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c1076a0 0x7f612c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61312c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f612c108020 0x7f612c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61312c7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f612c111c70 con 0x7f612c108020 2024-09-11T01:07:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61312c7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f612c111af0 con 0x7f612c101c90 2024-09-11T01:07:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61312c7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f612c111df0 con 0x7f612c1076a0 2024-09-11T01:07:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f612b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f612c108020 0x7f612c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f612b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f612c108020 0x7f612c19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53664/0 (socket says 172.21.15.78:53664) 2024-09-11T01:07:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f612b7fe640 1 -- 172.21.15.78:0/1094736139 learned_addr learned my addr 172.21.15.78:0/1094736139 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c1076a0 0x7f612c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f612affd640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f612c101c90 0x7f612c19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61227fc640 1 -- 172.21.15.78:0/1094736139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f612c101c90 msgr2=0x7f612c19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61227fc640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f612c101c90 0x7f612c19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61227fc640 1 -- 172.21.15.78:0/1094736139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f612c108020 msgr2=0x7f612c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61227fc640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f612c108020 0x7f612c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f61227fc640 1 -- 172.21.15.78:0/1094736139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c1a5480 con 0x7f612c1076a0 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.272+0000 7f612b7fe640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f612c108020 0x7f612c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.273+0000 7f612affd640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f612c101c90 0x7f612c19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.273+0000 7f61227fc640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c1076a0 0x7f612c19a5e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f611402a930 tx=0x7f6114031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.273+0000 7f6128ff9640 1 -- 172.21.15.78:0/1094736139 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611403f030 con 0x7f612c1076a0 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.273+0000 7f61312c7640 1 -- 172.21.15.78:0/1094736139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612c1a5710 con 0x7f612c1076a0 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.273+0000 7f6128ff9640 1 -- 172.21.15.78:0/1094736139 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6114004490 con 0x7f612c1076a0 2024-09-11T01:07:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.273+0000 7f6128ff9640 1 -- 172.21.15.78:0/1094736139 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114035070 con 0x7f612c1076a0 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.273+0000 7f61312c7640 1 -- 172.21.15.78:0/1094736139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612c1a5c40 con 0x7f612c1076a0 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.275+0000 7f613369c740 1 Processor -- start 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.275+0000 7f6128ff9640 1 -- 172.21.15.78:0/1094736139 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f611403ba30 con 0x7f612c1076a0 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.275+0000 7f613369c740 1 -- start start 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f613369c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f613369c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c54840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f613369c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c57170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f613369c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563d02bbeea0 con 0x563d02c484d0 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f613369c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563d02bae2d0 con 0x563d02c54d80 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f613369c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563d02bb20e0 con 0x563d02c48df0 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f612b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f612b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53672/0 (socket says 172.21.15.78:53672) 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f612b7fe640 1 -- 172.21.15.78:0/272940624 learned_addr learned my addr 172.21.15.78:0/272940624 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f61227fc640 1 --2- 172.21.15.78:0/272940624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c57170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f612affd640 1 --2- 172.21.15.78:0/272940624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c54840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f612b7fe640 1 -- 172.21.15.78:0/272940624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 msgr2=0x563d02c54840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f612b7fe640 1 --2- 172.21.15.78:0/272940624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c54840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f612b7fe640 1 -- 172.21.15.78:0/272940624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 msgr2=0x563d02c57170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f612b7fe640 1 --2- 172.21.15.78:0/272940624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c57170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.276+0000 7f612b7fe640 1 -- 172.21.15.78:0/272940624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d0290c650 con 0x563d02c484d0 2024-09-11T01:07:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.277+0000 7f61227fc640 1 --2- 172.21.15.78:0/272940624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c57170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.277+0000 7f612affd640 1 --2- 172.21.15.78:0/272940624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c54840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.277+0000 7f612b7fe640 1 --2- 172.21.15.78:0/272940624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c488b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6124011570 tx=0x7f6124011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8c6268b4f4059e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.277+0000 7f6128ff9640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f61180b5450 0x7f61180b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.277+0000 7f612affd640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f61180b5450 0x7f61180b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.277+0000 7f61217fa640 1 -- 172.21.15.78:0/272940624 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124005020 con 0x563d02c484d0 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.277+0000 7f61217fa640 1 -- 172.21.15.78:0/272940624 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61240047a0 con 0x563d02c484d0 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.277+0000 7f61217fa640 1 -- 172.21.15.78:0/272940624 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612400f8a0 con 0x563d02c484d0 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.277+0000 7f6128ff9640 1 -- 172.21.15.78:0/1094736139 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f6114106600 con 0x7f612c1076a0 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.278+0000 7f613369c740 1 -- 172.21.15.78:0/272940624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 msgr2=0x563d02c488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.278+0000 7f613369c740 1 --2- 172.21.15.78:0/272940624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c488b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6124011570 tx=0x7f6124011a30 comp rx=0 tx=0).stop 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.278+0000 7f612affd640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f61180b5450 0x7f61180b7910 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x563d02c471e0 tx=0x7f611c009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.278+0000 7f613369c740 1 -- 172.21.15.78:0/272940624 shutdown_connections 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.278+0000 7f613369c740 1 --2- 172.21.15.78:0/272940624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c57170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.278+0000 7f613369c740 1 --2- 172.21.15.78:0/272940624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c54840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.278+0000 7f613369c740 1 --2- 172.21.15.78:0/272940624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c488b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.278+0000 7f613369c740 1 -- 172.21.15.78:0/272940624 >> 172.21.15.78:0/272940624 conn(0x563d02c46300 msgr2=0x563d02c474a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.278+0000 7f613369c740 1 -- 172.21.15.78:0/272940624 shutdown_connections 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.278+0000 7f613369c740 1 -- 172.21.15.78:0/272940624 wait complete. 2024-09-11T01:07:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.278+0000 7f613369c740 1 -- start start 2024-09-11T01:07:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.278+0000 7f613369c740 1 Processor -- start 2024-09-11T01:07:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f613369c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c54650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f613369c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c51b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f613369c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c52120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f613369c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563d02a0f700 con 0x563d02c484d0 2024-09-11T01:07:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f613369c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563d02b49160 con 0x563d02c54d80 2024-09-11T01:07:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f613369c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563d02c5bd80 con 0x563d02c48df0 2024-09-11T01:07:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f613369c740 10 client.0 _ll_register_callbacks cb 0x563d02b629e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f612affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c54650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f612affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c54650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53682/0 (socket says 172.21.15.78:53682) 2024-09-11T01:07:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f612affd640 1 -- 172.21.15.78:0/3444566970 learned_addr learned my addr 172.21.15.78:0/3444566970 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f612b7fe640 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c52120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f61227fc640 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c51b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f612affd640 1 -- 172.21.15.78:0/3444566970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 msgr2=0x563d02c51b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f612affd640 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c51b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f612affd640 1 -- 172.21.15.78:0/3444566970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 msgr2=0x563d02c52120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f612affd640 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c52120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f612affd640 1 -- 172.21.15.78:0/3444566970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d02b76c90 con 0x563d02c484d0 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f612affd640 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c54650 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f611c003e20 tx=0x7f611c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee162dec2c11ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.279+0000 7f61127fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f61127fc640 1 -- 172.21.15.78:0/3444566970 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c004450 con 0x563d02c484d0 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f613369c740 1 -- 172.21.15.78:0/3444566970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61240047a0 con 0x563d02c484d0 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f61127fc640 1 -- 172.21.15.78:0/3444566970 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f611c0045f0 con 0x563d02c484d0 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f61127fc640 1 -- 172.21.15.78:0/3444566970 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f611c0047f0 con 0x563d02c484d0 2024-09-11T01:07:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f613369c740 10 client.17061 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f613369c740 1 -- 172.21.15.78:0/3444566970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563d02c67790 con 0x563d02c484d0 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f61127fc640 1 -- 172.21.15.78:0/3444566970 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61240047a0 con 0x563d02c484d0 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f61127fc640 1 -- 172.21.15.78:0/3444566970 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f611c01f590 con 0x563d02c484d0 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f613369c740 10 client.17061 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f613369c740 10 client.17061 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f613369c740 20 client.17061 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f613369c740 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x563d02c68ba0 0x563d02c6af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f613369c740 4 client.17061 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f613369c740 1 -- 172.21.15.78:0/3444566970 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d02bc0df0 con 0x563d02c68ba0 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.280+0000 7f61227fc640 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x563d02c68ba0 0x563d02c6af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f61127fc640 10 client.17061 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f61227fc640 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x563d02c68ba0 0x563d02c6af90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=88faa21459656dd4 server_cookie=79ac8767f52b211d in_seq=0 out_seq=0 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f61127fc640 1 -- 172.21.15.78:0/3444566970 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563d02bc0df0 con 0x563d02c68ba0 2024-09-11T01:07:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f61127fc640 10 client.17061 handle_command_reply: tid=0 2024-09-11T01:07:22.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f613369c740 1 client.17061 shutdown 2024-09-11T01:07:22.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f613369c740 1 -- 172.21.15.78:0/3444566970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 msgr2=0x563d02c54650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:22.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f613369c740 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c54650 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f611c003e20 tx=0x7f611c004a40 comp rx=0 tx=0).stop 2024-09-11T01:07:22.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f613369c740 1 -- 172.21.15.78:0/3444566970 shutdown_connections 2024-09-11T01:07:22.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f613369c740 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x563d02c68ba0 0x563d02c6af90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f61127fc640 0 client.17061 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:07:22.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f613369c740 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c52120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f613369c740 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c51b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f613369c740 1 --2- 172.21.15.78:0/3444566970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c54650 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.281+0000 7f613369c740 1 -- 172.21.15.78:0/3444566970 >> 172.21.15.78:0/3444566970 conn(0x563d02c46300 msgr2=0x563d02c4be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:22.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.282+0000 7f613369c740 1 -- 172.21.15.78:0/3444566970 shutdown_connections 2024-09-11T01:07:22.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.282+0000 7f613369c740 1 -- 172.21.15.78:0/3444566970 wait complete. 2024-09-11T01:07:22.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.282+0000 7f613369c740 20 client.17061 trim_cache size 0 max 16384 2024-09-11T01:07:22.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.298+0000 7f613369c740 1 Processor -- start 2024-09-11T01:07:22.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f613369c740 1 -- start start 2024-09-11T01:07:22.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f613369c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c65000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f613369c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c56170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f613369c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f613369c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1ac340 con 0x563d02c484d0 2024-09-11T01:07:22.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f613369c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1adee0 con 0x563d02c54d80 2024-09-11T01:07:22.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f613369c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1aca10 con 0x563d02c48df0 2024-09-11T01:07:22.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f612affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c65000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f612affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c65000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53692/0 (socket says 172.21.15.78:53692) 2024-09-11T01:07:22.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f612b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f612affd640 1 -- 172.21.15.78:0/2921509770 learned_addr learned my addr 172.21.15.78:0/2921509770 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:22.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f61227fc640 1 --2- 172.21.15.78:0/2921509770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c56170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f612affd640 1 -- 172.21.15.78:0/2921509770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 msgr2=0x563d02c56170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:22.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.299+0000 7f612affd640 1 --2- 172.21.15.78:0/2921509770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c56170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.300+0000 7f612affd640 1 -- 172.21.15.78:0/2921509770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 msgr2=0x563d02c566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:22.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.300+0000 7f612affd640 1 --2- 172.21.15.78:0/2921509770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.300+0000 7f612affd640 1 -- 172.21.15.78:0/2921509770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d02c46fe0 con 0x563d02c484d0 2024-09-11T01:07:22.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.300+0000 7f612b7fe640 1 --2- 172.21.15.78:0/2921509770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:22.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.300+0000 7f612affd640 1 --2- 172.21.15.78:0/2921509770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c65000 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f611c005b00 tx=0x7f611c0039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47924d4b8dd94a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:22.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.300+0000 7f6120ff9640 1 -- 172.21.15.78:0/2921509770 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c036040 con 0x563d02c484d0 2024-09-11T01:07:22.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.300+0000 7f6120ff9640 1 -- 172.21.15.78:0/2921509770 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611c0099d0 con 0x563d02c484d0 2024-09-11T01:07:22.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.300+0000 7f6120ff9640 1 -- 172.21.15.78:0/2921509770 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c004250 con 0x563d02c484d0 2024-09-11T01:07:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.301+0000 7f613369c740 1 -- 172.21.15.78:0/2921509770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 msgr2=0x563d02c65000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.301+0000 7f613369c740 1 --2- 172.21.15.78:0/2921509770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c65000 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f611c005b00 tx=0x7f611c0039b0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.301+0000 7f613369c740 1 -- 172.21.15.78:0/2921509770 shutdown_connections 2024-09-11T01:07:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.301+0000 7f613369c740 1 --2- 172.21.15.78:0/2921509770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c54d80 0x563d02c566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.301+0000 7f613369c740 1 --2- 172.21.15.78:0/2921509770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c48df0 0x563d02c56170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.301+0000 7f613369c740 1 --2- 172.21.15.78:0/2921509770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c484d0 0x563d02c65000 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.301+0000 7f613369c740 1 -- 172.21.15.78:0/2921509770 >> 172.21.15.78:0/2921509770 conn(0x563d02c46300 msgr2=0x563d02c53880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.301+0000 7f613369c740 1 -- 172.21.15.78:0/2921509770 shutdown_connections 2024-09-11T01:07:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.301+0000 7f613369c740 1 -- 172.21.15.78:0/2921509770 wait complete. 2024-09-11T01:07:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.301+0000 7f613369c740 1 -- start start 2024-09-11T01:07:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.301+0000 7f613369c740 1 Processor -- start 2024-09-11T01:07:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f613369c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c484d0 0x563d02c61650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f613369c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c48df0 0x563d02c5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f613369c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c54d80 0x563d02c5f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f613369c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1ad7e0 con 0x563d02c48df0 2024-09-11T01:07:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f613369c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1ad110 con 0x563d02c484d0 2024-09-11T01:07:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f613369c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1aa340 con 0x563d02c54d80 2024-09-11T01:07:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f613369c740 10 client.0 _ll_register_callbacks cb 0x7f611c009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f61227fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c48df0 0x563d02c5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f612b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c54d80 0x563d02c5f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f612b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c54d80 0x563d02c5f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:34572/0 (socket says 172.21.15.78:34572) 2024-09-11T01:07:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f612b7fe640 1 -- 172.21.15.78:0/73800704 learned_addr learned my addr 172.21.15.78:0/73800704 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f61227fc640 1 -- 172.21.15.78:0/73800704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c54d80 msgr2=0x563d02c5f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f61227fc640 1 --2- 172.21.15.78:0/73800704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c54d80 0x563d02c5f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f61227fc640 1 -- 172.21.15.78:0/73800704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c484d0 msgr2=0x563d02c61650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:07:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f612affd640 1 --2- 172.21.15.78:0/73800704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c484d0 0x563d02c61650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f61227fc640 1 --2- 172.21.15.78:0/73800704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c484d0 0x563d02c61650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f61227fc640 1 -- 172.21.15.78:0/73800704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d02c5fc60 con 0x563d02c48df0 2024-09-11T01:07:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f61227fc640 1 --2- 172.21.15.78:0/73800704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c48df0 0x563d02c5f0b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6114103d30 tx=0x7f6114108a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871b525d090dbc03 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.302+0000 7f61127fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:07:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f61127fc640 1 -- 172.21.15.78:0/73800704 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114049040 con 0x563d02c48df0 2024-09-11T01:07:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f61127fc640 1 -- 172.21.15.78:0/73800704 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f6114103fb0 con 0x563d02c48df0 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f613369c740 1 -- 172.21.15.78:0/73800704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f611c0099d0 con 0x563d02c48df0 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f61127fc640 1 -- 172.21.15.78:0/73800704 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f61140cc1d0 con 0x563d02c48df0 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f613369c740 10 client.17067 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f613369c740 1 -- 172.21.15.78:0/73800704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563d02cd6970 con 0x563d02c48df0 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f61127fc640 1 -- 172.21.15.78:0/73800704 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611c0099d0 con 0x563d02c48df0 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f61127fc640 1 -- 172.21.15.78:0/73800704 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f61140d10d0 con 0x563d02c48df0 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f613369c740 10 client.17067 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f613369c740 10 client.17067 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f613369c740 20 client.17067 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f613369c740 1 --2- 172.21.15.78:0/73800704 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x563d02cd7120 0x563d02cd9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f613369c740 4 client.17067 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.303+0000 7f613369c740 1 -- 172.21.15.78:0/73800704 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d02bbc8a0 con 0x563d02cd7120 2024-09-11T01:07:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f612affd640 1 --2- 172.21.15.78:0/73800704 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x563d02cd7120 0x563d02cd9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f61127fc640 10 client.17067 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:07:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f612affd640 1 --2- 172.21.15.78:0/73800704 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x563d02cd7120 0x563d02cd9510 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3f6e867ae4c494eb server_cookie=674bd5cbaa54a186 in_seq=0 out_seq=0 2024-09-11T01:07:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f61127fc640 1 -- 172.21.15.78:0/73800704 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d02bbc8a0 con 0x563d02cd7120 2024-09-11T01:07:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f61127fc640 10 client.17067 handle_command_reply: tid=0 2024-09-11T01:07:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f613369c740 1 client.17067 shutdown 2024-09-11T01:07:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f613369c740 1 -- 172.21.15.78:0/73800704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c48df0 msgr2=0x563d02c5f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f613369c740 1 --2- 172.21.15.78:0/73800704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c48df0 0x563d02c5f0b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6114103d30 tx=0x7f6114108a50 comp rx=0 tx=0).stop 2024-09-11T01:07:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f613369c740 1 -- 172.21.15.78:0/73800704 shutdown_connections 2024-09-11T01:07:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f613369c740 1 --2- 172.21.15.78:0/73800704 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x563d02cd7120 0x563d02cd9510 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f613369c740 1 --2- 172.21.15.78:0/73800704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d02c54d80 0x563d02c5f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f613369c740 1 --2- 172.21.15.78:0/73800704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d02c48df0 0x563d02c5f0b0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f613369c740 1 --2- 172.21.15.78:0/73800704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d02c484d0 0x563d02c61650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f613369c740 1 -- 172.21.15.78:0/73800704 >> 172.21.15.78:0/73800704 conn(0x563d02c46300 msgr2=0x563d02c52c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:22.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f61127fc640 0 client.17067 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:07:22.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f613369c740 1 -- 172.21.15.78:0/73800704 shutdown_connections 2024-09-11T01:07:22.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f613369c740 1 -- 172.21.15.78:0/73800704 wait complete. 2024-09-11T01:07:22.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.304+0000 7f613369c740 20 client.17067 trim_cache size 0 max 16384 2024-09-11T01:07:22.313 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:07:22.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.305+0000 7f61312c7640 1 -- 172.21.15.78:0/1094736139 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f61180b5450 msgr2=0x7f61180b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:22.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.305+0000 7f61312c7640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f61180b5450 0x7f61180b7910 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x563d02c471e0 tx=0x7f611c009500 comp rx=0 tx=0).stop 2024-09-11T01:07:22.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.305+0000 7f61312c7640 1 -- 172.21.15.78:0/1094736139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c1076a0 msgr2=0x7f612c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:22.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.305+0000 7f61312c7640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c1076a0 0x7f612c19a5e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f611402a930 tx=0x7f6114031a20 comp rx=0 tx=0).stop 2024-09-11T01:07:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.305+0000 7f61312c7640 1 -- 172.21.15.78:0/1094736139 shutdown_connections 2024-09-11T01:07:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.305+0000 7f61312c7640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f61180b5450 0x7f61180b7910 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.305+0000 7f61312c7640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f612c108020 0x7f612c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.305+0000 7f61312c7640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c1076a0 0x7f612c19a5e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.305+0000 7f61312c7640 1 --2- 172.21.15.78:0/1094736139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f612c101c90 0x7f612c19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.305+0000 7f61312c7640 1 -- 172.21.15.78:0/1094736139 >> 172.21.15.78:0/1094736139 conn(0x7f612c0fd700 msgr2=0x7f612c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.305+0000 7f61312c7640 1 -- 172.21.15.78:0/1094736139 shutdown_connections 2024-09-11T01:07:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:22.305+0000 7f61312c7640 1 -- 172.21.15.78:0/1094736139 wait complete. 2024-09-11T01:07:22.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 584932 5.40 MB/sec execute 330 sec latency 3020.810 ms 2024-09-11T01:07:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:22 smithi078 ceph-mon[31315]: pgmap v582: 97 pgs: 97 active+clean; 409 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-09-11T01:07:22.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:22 smithi110 ceph-mon[34712]: pgmap v582: 97 pgs: 97 active+clean; 409 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-09-11T01:07:22.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:22 smithi112 ceph-mon[35384]: pgmap v582: 97 pgs: 97 active+clean; 409 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-09-11T01:07:23.029 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:07:23.303 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:07:23.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 586549 5.42 MB/sec execute 331 sec latency 15.831 ms 2024-09-11T01:07:23.571 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:23 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:07:23.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:23 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:07:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:23 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:07:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.924+0000 7ff5124a4640 1 -- 172.21.15.78:0/797308703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c101c90 msgr2=0x7ff50c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.924+0000 7ff5124a4640 1 --2- 172.21.15.78:0/797308703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c101c90 0x7ff50c102070 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009900 tx=0x7ff4fc02a410 comp rx=0 tx=0).stop 2024-09-11T01:07:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.924+0000 7ff5124a4640 1 -- 172.21.15.78:0/797308703 shutdown_connections 2024-09-11T01:07:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.924+0000 7ff5124a4640 1 --2- 172.21.15.78:0/797308703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff50c108020 0x7ff50c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.924+0000 7ff5124a4640 1 --2- 172.21.15.78:0/797308703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c1076a0 0x7ff50c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.924+0000 7ff5124a4640 1 --2- 172.21.15.78:0/797308703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c101c90 0x7ff50c102070 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.924+0000 7ff5124a4640 1 -- 172.21.15.78:0/797308703 >> 172.21.15.78:0/797308703 conn(0x7ff50c0fd700 msgr2=0x7ff50c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.924+0000 7ff5124a4640 1 -- 172.21.15.78:0/797308703 shutdown_connections 2024-09-11T01:07:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.924+0000 7ff5124a4640 1 -- 172.21.15.78:0/797308703 wait complete. 2024-09-11T01:07:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.924+0000 7ff5124a4640 1 Processor -- start 2024-09-11T01:07:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.924+0000 7ff5124a4640 1 -- start start 2024-09-11T01:07:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff5124a4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff50c101c90 0x7ff50c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff5124a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c1076a0 0x7ff50c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff5124a4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c108020 0x7ff50c19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff5124a4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c111d30 con 0x7ff50c108020 2024-09-11T01:07:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff5124a4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c111bb0 con 0x7ff50c101c90 2024-09-11T01:07:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff5124a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c111eb0 con 0x7ff50c1076a0 2024-09-11T01:07:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff510a41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c108020 0x7ff50c19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff50c101c90 0x7ff50c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff510a41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c108020 0x7ff50c19dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54988/0 (socket says 172.21.15.78:54988) 2024-09-11T01:07:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff510a41640 1 -- 172.21.15.78:0/2330318745 learned_addr learned my addr 172.21.15.78:0/2330318745 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff503fff640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c1076a0 0x7ff50c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff510a41640 1 -- 172.21.15.78:0/2330318745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c1076a0 msgr2=0x7ff50c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff510a41640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c1076a0 0x7ff50c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff510a41640 1 -- 172.21.15.78:0/2330318745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff50c101c90 msgr2=0x7ff50c1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff510a41640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff50c101c90 0x7ff50c1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff510a41640 1 -- 172.21.15.78:0/2330318745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c19e290 con 0x7ff50c108020 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.925+0000 7ff510a41640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c108020 0x7ff50c19dac0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff50400eb10 tx=0x7ff50400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.926+0000 7ff509ffb640 1 -- 172.21.15.78:0/2330318745 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504004b20 con 0x7ff50c108020 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.926+0000 7ff509ffb640 1 -- 172.21.15.78:0/2330318745 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff504004cc0 con 0x7ff50c108020 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.926+0000 7ff5124a4640 1 -- 172.21.15.78:0/2330318745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c19e580 con 0x7ff50c108020 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.926+0000 7ff509ffb640 1 -- 172.21.15.78:0/2330318745 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504010670 con 0x7ff50c108020 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.926+0000 7ff5124a4640 1 -- 172.21.15.78:0/2330318745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c06aab0 con 0x7ff50c108020 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.928+0000 7ff509ffb640 1 -- 172.21.15.78:0/2330318745 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff504005020 con 0x7ff50c108020 2024-09-11T01:07:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff514879740 1 Processor -- start 2024-09-11T01:07:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff514879740 1 -- start start 2024-09-11T01:07:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff514879740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae7900f0 0x55bfae7904d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff514879740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 0x55bfae79c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff514879740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 0x55bfae79ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff514879740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bfae706ea0 con 0x55bfae790a10 2024-09-11T01:07:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff514879740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55bfae6f62d0 con 0x55bfae7900f0 2024-09-11T01:07:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff514879740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bfae6fa0e0 con 0x55bfae79c9a0 2024-09-11T01:07:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 0x55bfae79c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff503fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae7900f0 0x55bfae7904d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 0x55bfae79c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54990/0 (socket says 172.21.15.78:54990) 2024-09-11T01:07:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff50bfff640 1 -- 172.21.15.78:0/549964110 learned_addr learned my addr 172.21.15.78:0/549964110 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff503fff640 1 --2- 172.21.15.78:0/549964110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 0x55bfae79ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff509ffb640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff4f80b5450 0x7ff4f80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff503fff640 1 -- 172.21.15.78:0/549964110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 msgr2=0x55bfae79ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff503fff640 1 --2- 172.21.15.78:0/549964110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 0x55bfae79ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff503fff640 1 -- 172.21.15.78:0/549964110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 msgr2=0x55bfae79c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff503fff640 1 --2- 172.21.15.78:0/549964110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 0x55bfae79c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.929+0000 7ff503fff640 1 -- 172.21.15.78:0/549964110 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfae454650 con 0x55bfae7900f0 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.930+0000 7ff510a41640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff4f80b5450 0x7ff4f80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.930+0000 7ff503fff640 1 --2- 172.21.15.78:0/549964110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae7900f0 0x55bfae7904d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009f40 tx=0x7ff4f4009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=194118b49c761ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.930+0000 7ff509ffb640 1 -- 172.21.15.78:0/2330318745 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7ff5040e6590 con 0x7ff50c108020 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.930+0000 7ff5027fc640 1 -- 172.21.15.78:0/549964110 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4004050 con 0x55bfae7900f0 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.930+0000 7ff5027fc640 1 -- 172.21.15.78:0/549964110 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f40054d0 con 0x55bfae7900f0 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.930+0000 7ff5027fc640 1 -- 172.21.15.78:0/549964110 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4004c60 con 0x55bfae7900f0 2024-09-11T01:07:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 -- 172.21.15.78:0/549964110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae7900f0 msgr2=0x55bfae7904d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:23.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff510a41640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff4f80b5450 0x7ff4f80b7910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff5040e3dd0 tx=0x7ff504078c30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:23.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 --2- 172.21.15.78:0/549964110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae7900f0 0x55bfae7904d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009f40 tx=0x7ff4f4009940 comp rx=0 tx=0).stop 2024-09-11T01:07:23.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 -- 172.21.15.78:0/549964110 shutdown_connections 2024-09-11T01:07:23.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 --2- 172.21.15.78:0/549964110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 0x55bfae79ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 --2- 172.21.15.78:0/549964110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 0x55bfae79c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 --2- 172.21.15.78:0/549964110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae7900f0 0x55bfae7904d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 -- 172.21.15.78:0/549964110 >> 172.21.15.78:0/549964110 conn(0x55bfae78df20 msgr2=0x55bfae78f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:23.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 -- 172.21.15.78:0/549964110 shutdown_connections 2024-09-11T01:07:23.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 -- 172.21.15.78:0/549964110 wait complete. 2024-09-11T01:07:23.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 -- start start 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 Processor -- start 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 0x55bfae79c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae790a10 0x55bfae799690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 0x55bfae799c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bfae557700 con 0x55bfae7900f0 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55bfae691160 con 0x55bfae790a10 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.931+0000 7ff514879740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bfae79f620 con 0x55bfae79c9a0 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff514879740 10 client.0 _ll_register_callbacks cb 0x55bfae6aa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff503fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 0x55bfae79c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff503fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 0x55bfae79c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55006/0 (socket says 172.21.15.78:55006) 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae790a10 0x55bfae799690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff503fff640 1 -- 172.21.15.78:0/4243860363 learned_addr learned my addr 172.21.15.78:0/4243860363 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff503fff640 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 0x55bfae799c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff503fff640 1 -- 172.21.15.78:0/4243860363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 msgr2=0x55bfae799c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff503fff640 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 0x55bfae799c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff503fff640 1 -- 172.21.15.78:0/4243860363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae790a10 msgr2=0x55bfae799690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff503fff640 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae790a10 0x55bfae799690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff503fff640 1 -- 172.21.15.78:0/4243860363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfae6bec90 con 0x55bfae7900f0 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff50bfff640 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae790a10 0x55bfae799690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff503fff640 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 0x55bfae79c240 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40058c0 tx=0x7ff4f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dae74665a00866a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff4eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff4eb7fe640 1 -- 172.21.15.78:0/4243860363 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4004e70 con 0x55bfae7900f0 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff4eb7fe640 1 -- 172.21.15.78:0/4243860363 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7ff4f401e460 con 0x55bfae7900f0 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff514879740 1 -- 172.21.15.78:0/4243860363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f40054d0 con 0x55bfae7900f0 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff4eb7fe640 1 -- 172.21.15.78:0/4243860363 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff4f401e660 con 0x55bfae7900f0 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff514879740 10 client.17079 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.932+0000 7ff514879740 1 -- 172.21.15.78:0/4243860363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bfae7aaf10 con 0x55bfae7900f0 2024-09-11T01:07:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.933+0000 7ff4eb7fe640 1 -- 172.21.15.78:0/4243860363 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f40054d0 con 0x55bfae7900f0 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.933+0000 7ff4eb7fe640 1 -- 172.21.15.78:0/4243860363 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7ff4f40367c0 con 0x55bfae7900f0 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.933+0000 7ff514879740 10 client.17079 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.933+0000 7ff514879740 10 client.17079 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.933+0000 7ff514879740 20 client.17079 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.933+0000 7ff514879740 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55bfae7ac260 0x55bfae7ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.933+0000 7ff514879740 4 client.17079 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.933+0000 7ff514879740 1 -- 172.21.15.78:0/4243860363 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfae708df0 con 0x55bfae7ac260 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.933+0000 7ff50bfff640 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55bfae7ac260 0x55bfae7ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.933+0000 7ff4eb7fe640 10 client.17079 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.933+0000 7ff50bfff640 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55bfae7ac260 0x55bfae7ae650 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e35b6ae280228852 server_cookie=f561cdbe7a91c267 in_seq=0 out_seq=0 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.934+0000 7ff4eb7fe640 1 -- 172.21.15.78:0/4243860363 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bfae708df0 con 0x55bfae7ac260 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.934+0000 7ff4eb7fe640 10 client.17079 handle_command_reply: tid=0 2024-09-11T01:07:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.934+0000 7ff514879740 1 client.17079 shutdown 2024-09-11T01:07:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.934+0000 7ff514879740 1 -- 172.21.15.78:0/4243860363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 msgr2=0x55bfae79c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.934+0000 7ff514879740 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 0x55bfae79c240 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40058c0 tx=0x7ff4f4005f90 comp rx=0 tx=0).stop 2024-09-11T01:07:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.934+0000 7ff514879740 1 -- 172.21.15.78:0/4243860363 shutdown_connections 2024-09-11T01:07:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.934+0000 7ff514879740 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55bfae7ac260 0x55bfae7ae650 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.934+0000 7ff514879740 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 0x55bfae799c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.934+0000 7ff514879740 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae790a10 0x55bfae799690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.934+0000 7ff514879740 1 --2- 172.21.15.78:0/4243860363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 0x55bfae79c240 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.934+0000 7ff514879740 1 -- 172.21.15.78:0/4243860363 >> 172.21.15.78:0/4243860363 conn(0x55bfae78df20 msgr2=0x55bfae793ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.935+0000 7ff4eb7fe640 0 client.17079 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:07:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.935+0000 7ff514879740 1 -- 172.21.15.78:0/4243860363 shutdown_connections 2024-09-11T01:07:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.935+0000 7ff514879740 1 -- 172.21.15.78:0/4243860363 wait complete. 2024-09-11T01:07:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.935+0000 7ff514879740 20 client.17079 trim_cache size 0 max 16384 2024-09-11T01:07:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.949+0000 7ff514879740 1 Processor -- start 2024-09-11T01:07:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.949+0000 7ff514879740 1 -- start start 2024-09-11T01:07:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.949+0000 7ff514879740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae7900f0 0x55bfae7a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.949+0000 7ff514879740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 0x55bfae7890b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.949+0000 7ff514879740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 0x55bfae79b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.949+0000 7ff514879740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1ae680 con 0x55bfae790a10 2024-09-11T01:07:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.949+0000 7ff514879740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1b0220 con 0x55bfae7900f0 2024-09-11T01:07:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.949+0000 7ff514879740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1aed50 con 0x55bfae79c9a0 2024-09-11T01:07:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.949+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 0x55bfae7890b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 0x55bfae7890b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55008/0 (socket says 172.21.15.78:55008) 2024-09-11T01:07:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff50bfff640 1 -- 172.21.15.78:0/3963525781 learned_addr learned my addr 172.21.15.78:0/3963525781 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff503fff640 1 --2- 172.21.15.78:0/3963525781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae7900f0 0x55bfae7a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff503fff640 1 --2- 172.21.15.78:0/3963525781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 0x55bfae79b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff50bfff640 1 -- 172.21.15.78:0/3963525781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 msgr2=0x55bfae79b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff50bfff640 1 --2- 172.21.15.78:0/3963525781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 0x55bfae79b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff50bfff640 1 -- 172.21.15.78:0/3963525781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae7900f0 msgr2=0x55bfae7a8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff50bfff640 1 --2- 172.21.15.78:0/3963525781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae7900f0 0x55bfae7a8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff50bfff640 1 -- 172.21.15.78:0/3963525781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfae799fc0 con 0x55bfae790a10 2024-09-11T01:07:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff503fff640 1 --2- 172.21.15.78:0/3963525781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae7900f0 0x55bfae7a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff50bfff640 1 --2- 172.21.15.78:0/3963525781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 0x55bfae7890b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc005e00 tx=0x7ff4fc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b756794c3f13fedd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:23.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff501ffb640 1 -- 172.21.15.78:0/3963525781 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc04a020 con 0x55bfae790a10 2024-09-11T01:07:23.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff501ffb640 1 -- 172.21.15.78:0/3963525781 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4fc004770 con 0x55bfae790a10 2024-09-11T01:07:23.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.950+0000 7ff501ffb640 1 -- 172.21.15.78:0/3963525781 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc035040 con 0x55bfae790a10 2024-09-11T01:07:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.951+0000 7ff514879740 1 -- 172.21.15.78:0/3963525781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 msgr2=0x55bfae7890b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.951+0000 7ff514879740 1 --2- 172.21.15.78:0/3963525781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 0x55bfae7890b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc005e00 tx=0x7ff4fc005a20 comp rx=0 tx=0).stop 2024-09-11T01:07:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.951+0000 7ff514879740 1 -- 172.21.15.78:0/3963525781 shutdown_connections 2024-09-11T01:07:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.951+0000 7ff514879740 1 --2- 172.21.15.78:0/3963525781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae79c9a0 0x55bfae79b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.951+0000 7ff514879740 1 --2- 172.21.15.78:0/3963525781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae790a10 0x55bfae7890b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.951+0000 7ff514879740 1 --2- 172.21.15.78:0/3963525781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae7900f0 0x55bfae7a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.951+0000 7ff514879740 1 -- 172.21.15.78:0/3963525781 >> 172.21.15.78:0/3963525781 conn(0x55bfae78df20 msgr2=0x55bfae79acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.951+0000 7ff514879740 1 -- 172.21.15.78:0/3963525781 shutdown_connections 2024-09-11T01:07:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.951+0000 7ff514879740 1 -- 172.21.15.78:0/3963525781 wait complete. 2024-09-11T01:07:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff514879740 1 -- start start 2024-09-11T01:07:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff514879740 1 Processor -- start 2024-09-11T01:07:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff514879740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 0x55bfae7a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff514879740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae790a10 0x55bfae7a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff514879740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae79c9a0 0x55bfae7958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff514879740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1afb20 con 0x55bfae7900f0 2024-09-11T01:07:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff514879740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1af450 con 0x55bfae79c9a0 2024-09-11T01:07:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff514879740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1ac680 con 0x55bfae790a10 2024-09-11T01:07:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff514879740 10 client.0 _ll_register_callbacks cb 0x7ff4f4011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff503fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 0x55bfae7a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae790a10 0x55bfae7a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff503fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 0x55bfae7a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55020/0 (socket says 172.21.15.78:55020) 2024-09-11T01:07:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff503fff640 1 -- 172.21.15.78:0/1007528809 learned_addr learned my addr 172.21.15.78:0/1007528809 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.952+0000 7ff503fff640 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae79c9a0 0x55bfae7958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff50bfff640 1 -- 172.21.15.78:0/1007528809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae79c9a0 msgr2=0x55bfae7958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff50bfff640 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae79c9a0 0x55bfae7958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff50bfff640 1 -- 172.21.15.78:0/1007528809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 msgr2=0x55bfae7a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff50bfff640 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 0x55bfae7a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff50bfff640 1 -- 172.21.15.78:0/1007528809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfae7a5560 con 0x55bfae790a10 2024-09-11T01:07:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff503fff640 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 0x55bfae7a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff50bfff640 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae790a10 0x55bfae7a4df0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc005ab0 tx=0x7ff4fc005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5710f468d85383 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff4eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:07:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff4eb7fe640 1 -- 172.21.15.78:0/1007528809 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc04a030 con 0x55bfae790a10 2024-09-11T01:07:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff4eb7fe640 1 -- 172.21.15.78:0/1007528809 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7ff4fc005040 con 0x55bfae790a10 2024-09-11T01:07:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff514879740 1 -- 172.21.15.78:0/1007528809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc004770 con 0x55bfae790a10 2024-09-11T01:07:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff4eb7fe640 1 -- 172.21.15.78:0/1007528809 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff4fc035050 con 0x55bfae790a10 2024-09-11T01:07:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff514879740 10 client.26284 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.953+0000 7ff514879740 1 -- 172.21.15.78:0/1007528809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bfae796e40 con 0x55bfae790a10 2024-09-11T01:07:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.954+0000 7ff4eb7fe640 1 -- 172.21.15.78:0/1007528809 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4fc004770 con 0x55bfae790a10 2024-09-11T01:07:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.954+0000 7ff4eb7fe640 1 -- 172.21.15.78:0/1007528809 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7ff4fc003ca0 con 0x55bfae790a10 2024-09-11T01:07:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.954+0000 7ff514879740 10 client.26284 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.954+0000 7ff514879740 10 client.26284 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:07:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.954+0000 7ff514879740 20 client.26284 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.954+0000 7ff514879740 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55bfae819f80 0x55bfae81c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.954+0000 7ff514879740 4 client.26284 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.954+0000 7ff514879740 1 -- 172.21.15.78:0/1007528809 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfae7048a0 con 0x55bfae819f80 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.954+0000 7ff503fff640 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55bfae819f80 0x55bfae81c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.955+0000 7ff4eb7fe640 10 client.26284 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.955+0000 7ff503fff640 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55bfae819f80 0x55bfae81c370 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fbc50140969a742e server_cookie=6ac92a7bcee636a in_seq=0 out_seq=0 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff4eb7fe640 1 -- 172.21.15.78:0/1007528809 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfae7048a0 con 0x55bfae819f80 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff4eb7fe640 10 client.26284 handle_command_reply: tid=0 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff514879740 1 client.26284 shutdown 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff514879740 1 -- 172.21.15.78:0/1007528809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae790a10 msgr2=0x55bfae7a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff514879740 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae790a10 0x55bfae7a4df0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc005ab0 tx=0x7ff4fc005fd0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff514879740 1 -- 172.21.15.78:0/1007528809 shutdown_connections 2024-09-11T01:07:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff514879740 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55bfae819f80 0x55bfae81c370 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff514879740 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bfae79c9a0 0x55bfae7958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff514879740 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfae790a10 0x55bfae7a4df0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff4eb7fe640 0 client.26284 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:07:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff514879740 1 --2- 172.21.15.78:0/1007528809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfae7900f0 0x55bfae7a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff514879740 1 -- 172.21.15.78:0/1007528809 >> 172.21.15.78:0/1007528809 conn(0x55bfae78df20 msgr2=0x55bfae79b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff514879740 1 -- 172.21.15.78:0/1007528809 shutdown_connections 2024-09-11T01:07:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff514879740 1 -- 172.21.15.78:0/1007528809 wait complete. 2024-09-11T01:07:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff514879740 20 client.26284 trim_cache size 0 max 16384 2024-09-11T01:07:23.963 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:07:23.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff5124a4640 1 -- 172.21.15.78:0/2330318745 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff4f80b5450 msgr2=0x7ff4f80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:23.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff5124a4640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff4f80b5450 0x7ff4f80b7910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff5040e3dd0 tx=0x7ff504078c30 comp rx=0 tx=0).stop 2024-09-11T01:07:23.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff5124a4640 1 -- 172.21.15.78:0/2330318745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c108020 msgr2=0x7ff50c19dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:23.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.956+0000 7ff5124a4640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c108020 0x7ff50c19dac0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff50400eb10 tx=0x7ff50400c2d0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.957+0000 7ff5124a4640 1 -- 172.21.15.78:0/2330318745 shutdown_connections 2024-09-11T01:07:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.957+0000 7ff5124a4640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff4f80b5450 0x7ff4f80b7910 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.957+0000 7ff5124a4640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c108020 0x7ff50c19dac0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.957+0000 7ff5124a4640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c1076a0 0x7ff50c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.957+0000 7ff5124a4640 1 --2- 172.21.15.78:0/2330318745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff50c101c90 0x7ff50c1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.957+0000 7ff5124a4640 1 -- 172.21.15.78:0/2330318745 >> 172.21.15.78:0/2330318745 conn(0x7ff50c0fd700 msgr2=0x7ff50c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.957+0000 7ff5124a4640 1 -- 172.21.15.78:0/2330318745 shutdown_connections 2024-09-11T01:07:23.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:23.957+0000 7ff5124a4640 1 -- 172.21.15.78:0/2330318745 wait complete. 2024-09-11T01:07:24.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 587766 5.42 MB/sec execute 332 sec latency 265.587 ms 2024-09-11T01:07:24.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:24 smithi078 ceph-mon[31315]: pgmap v583: 97 pgs: 97 active+clean; 407 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-09-11T01:07:24.667 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:07:24.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:24 smithi110 ceph-mon[34712]: pgmap v583: 97 pgs: 97 active+clean; 407 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-09-11T01:07:24.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:24 smithi112 ceph-mon[35384]: pgmap v583: 97 pgs: 97 active+clean; 407 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-09-11T01:07:24.928 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:07:25.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 589866 5.44 MB/sec execute 333 sec latency 42.579 ms 2024-09-11T01:07:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.560+0000 7f15a8b55640 1 -- 172.21.15.78:0/65834588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15a41076a0 msgr2=0x7f15a4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.560+0000 7f15a8b55640 1 --2- 172.21.15.78:0/65834588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15a41076a0 0x7f15a4107ae0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f158c009900 tx=0x7f158c02a410 comp rx=0 tx=0).stop 2024-09-11T01:07:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.560+0000 7f15a8b55640 1 -- 172.21.15.78:0/65834588 shutdown_connections 2024-09-11T01:07:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.560+0000 7f15a8b55640 1 --2- 172.21.15.78:0/65834588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a4108020 0x7f15a410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.560+0000 7f15a8b55640 1 --2- 172.21.15.78:0/65834588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15a41076a0 0x7f15a4107ae0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.560+0000 7f15a8b55640 1 --2- 172.21.15.78:0/65834588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a4101c90 0x7f15a4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.560+0000 7f15a8b55640 1 -- 172.21.15.78:0/65834588 >> 172.21.15.78:0/65834588 conn(0x7f15a40fd700 msgr2=0x7f15a40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.560+0000 7f15a8b55640 1 -- 172.21.15.78:0/65834588 shutdown_connections 2024-09-11T01:07:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.560+0000 7f15a8b55640 1 -- 172.21.15.78:0/65834588 wait complete. 2024-09-11T01:07:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.560+0000 7f15a8b55640 1 Processor -- start 2024-09-11T01:07:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f15a8b55640 1 -- start start 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f15a8b55640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a4101c90 0x7f15a40fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f15a8b55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15a41076a0 0x7f15a40fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f15a8b55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a4108020 0x7f15a40fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f15a8b55640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4111cf0 con 0x7f15a41076a0 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f15a8b55640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4111b70 con 0x7f15a4101c90 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f15a8b55640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4111e70 con 0x7f15a4108020 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f1599d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15a41076a0 0x7f15a40fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f15a259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a4101c90 0x7f15a40fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f1599d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15a41076a0 0x7f15a40fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55036/0 (socket says 172.21.15.78:55036) 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f1599d9b640 1 -- 172.21.15.78:0/690871144 learned_addr learned my addr 172.21.15.78:0/690871144 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f15a2d9d640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a4108020 0x7f15a40fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f1599d9b640 1 -- 172.21.15.78:0/690871144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a4108020 msgr2=0x7f15a40fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f1599d9b640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a4108020 0x7f15a40fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f1599d9b640 1 -- 172.21.15.78:0/690871144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a4101c90 msgr2=0x7f15a40fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f1599d9b640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a4101c90 0x7f15a40fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f1599d9b640 1 -- 172.21.15.78:0/690871144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a41a76c0 con 0x7f15a41076a0 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.561+0000 7f15a259c640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a4101c90 0x7f15a40fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.562+0000 7f1599d9b640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15a41076a0 0x7f15a40fe140 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f158c005ab0 tx=0x7f158c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.562+0000 7f159bfff640 1 -- 172.21.15.78:0/690871144 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c03f070 con 0x7f15a41076a0 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.562+0000 7f15a8b55640 1 -- 172.21.15.78:0/690871144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a41a7950 con 0x7f15a41076a0 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.562+0000 7f159bfff640 1 -- 172.21.15.78:0/690871144 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f158c031b60 con 0x7f15a41076a0 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.562+0000 7f159bfff640 1 -- 172.21.15.78:0/690871144 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c035030 con 0x7f15a41076a0 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.562+0000 7f15a8b55640 1 -- 172.21.15.78:0/690871144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a41a7e30 con 0x7f15a41076a0 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f15aaf2a740 1 Processor -- start 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f15aaf2a740 1 -- start start 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f159bfff640 1 -- 172.21.15.78:0/690871144 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f158c03b460 con 0x7f15a41076a0 2024-09-11T01:07:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f15aaf2a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da010f0 0x55a18da014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f15aaf2a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 0x55a18da0d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f15aaf2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da0fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f15aaf2a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a18d977ea0 con 0x55a18da01a10 2024-09-11T01:07:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f15aaf2a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a18d9672d0 con 0x55a18da010f0 2024-09-11T01:07:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f15aaf2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a18d96b0e0 con 0x55a18da0d9a0 2024-09-11T01:07:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f15a259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 0x55a18da0d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f15a259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 0x55a18da0d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55046/0 (socket says 172.21.15.78:55046) 2024-09-11T01:07:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f15a259c640 1 -- 172.21.15.78:0/2259064564 learned_addr learned my addr 172.21.15.78:0/2259064564 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f1599d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da0fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.565+0000 7f15a2d9d640 1 --2- 172.21.15.78:0/2259064564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da010f0 0x55a18da014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f15a259c640 1 -- 172.21.15.78:0/2259064564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 msgr2=0x55a18da0fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f15a259c640 1 --2- 172.21.15.78:0/2259064564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da0fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f15a259c640 1 -- 172.21.15.78:0/2259064564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da010f0 msgr2=0x55a18da014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f15a259c640 1 --2- 172.21.15.78:0/2259064564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da010f0 0x55a18da014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f15a259c640 1 -- 172.21.15.78:0/2259064564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18d6c5650 con 0x55a18da01a10 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f15a2d9d640 1 --2- 172.21.15.78:0/2259064564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da010f0 0x55a18da014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f15a259c640 1 --2- 172.21.15.78:0/2259064564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 0x55a18da0d460 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f15940115f0 tx=0x7f1594011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fff90dc27012820 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f1598d99640 1 -- 172.21.15.78:0/2259064564 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159401e070 con 0x55a18da01a10 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f1598d99640 1 -- 172.21.15.78:0/2259064564 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159400faf0 con 0x55a18da01a10 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f1598d99640 1 -- 172.21.15.78:0/2259064564 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594019ac0 con 0x55a18da01a10 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f159bfff640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15900b5450 0x7f15900b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f15a2d9d640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15900b5450 0x7f15900b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.566+0000 7f159bfff640 1 -- 172.21.15.78:0/690871144 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f158c1068f0 con 0x7f15a41076a0 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.567+0000 7f15a2d9d640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15900b5450 0x7f15900b7910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f159c009fd0 tx=0x7f159c009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.567+0000 7f15aaf2a740 1 -- 172.21.15.78:0/2259064564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 msgr2=0x55a18da0d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.567+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/2259064564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 0x55a18da0d460 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f15940115f0 tx=0x7f1594011ab0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.567+0000 7f15aaf2a740 1 -- 172.21.15.78:0/2259064564 shutdown_connections 2024-09-11T01:07:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.567+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/2259064564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da0fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.567+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/2259064564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 0x55a18da0d460 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.567+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/2259064564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da010f0 0x55a18da014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.567+0000 7f15aaf2a740 1 -- 172.21.15.78:0/2259064564 >> 172.21.15.78:0/2259064564 conn(0x55a18d9fef20 msgr2=0x55a18da000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.567+0000 7f15aaf2a740 1 -- 172.21.15.78:0/2259064564 shutdown_connections 2024-09-11T01:07:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.567+0000 7f15aaf2a740 1 -- 172.21.15.78:0/2259064564 wait complete. 2024-09-11T01:07:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f15aaf2a740 1 -- start start 2024-09-11T01:07:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f15aaf2a740 1 Processor -- start 2024-09-11T01:07:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f15aaf2a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da010f0 0x55a18da0d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f15aaf2a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 0x55a18da0a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f15aaf2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da0adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f15aaf2a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a18d7c8700 con 0x55a18da01a10 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f15aaf2a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a18d902160 con 0x55a18da010f0 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f15aaf2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a18da10620 con 0x55a18da0d9a0 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f15aaf2a740 10 client.0 _ll_register_callbacks cb 0x55a18d91b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f1599d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 0x55a18da0a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f1599d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 0x55a18da0a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55056/0 (socket says 172.21.15.78:55056) 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f1599d9b640 1 -- 172.21.15.78:0/1017448323 learned_addr learned my addr 172.21.15.78:0/1017448323 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f15a2d9d640 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da0adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f15a259c640 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da010f0 0x55a18da0d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f1599d9b640 1 -- 172.21.15.78:0/1017448323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 msgr2=0x55a18da0adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f1599d9b640 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da0adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f1599d9b640 1 -- 172.21.15.78:0/1017448323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da010f0 msgr2=0x55a18da0d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f1599d9b640 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da010f0 0x55a18da0d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f1599d9b640 1 -- 172.21.15.78:0/1017448323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18d92fc90 con 0x55a18da01a10 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f15a259c640 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da010f0 0x55a18da0d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.568+0000 7f1599d9b640 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 0x55a18da0a730 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f158c031440 tx=0x7f158c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1653056af1667 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f1589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f1589ffb640 1 -- 172.21.15.78:0/1017448323 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c049070 con 0x55a18da01a10 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1017448323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159400faf0 con 0x55a18da01a10 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f1589ffb640 1 -- 172.21.15.78:0/1017448323 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f158c0cd0c0 con 0x55a18da01a10 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f1589ffb640 1 -- 172.21.15.78:0/1017448323 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f158c0ccc30 con 0x55a18da01a10 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f15aaf2a740 10 client.17097 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1017448323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55a18da1bfd0 con 0x55a18da01a10 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f1589ffb640 1 -- 172.21.15.78:0/1017448323 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159400faf0 con 0x55a18da01a10 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f1589ffb640 1 -- 172.21.15.78:0/1017448323 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f158c0d5eb0 con 0x55a18da01a10 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f15aaf2a740 10 client.17097 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f15aaf2a740 10 client.17097 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f15aaf2a740 20 client.17097 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a18da1d430 0x55a18da1f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f15aaf2a740 4 client.17097 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1017448323 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a18d979df0 con 0x55a18da1d430 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.569+0000 7f15a259c640 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a18da1d430 0x55a18da1f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.570+0000 7f15a259c640 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a18da1d430 0x55a18da1f8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3735d42b81225301 server_cookie=c3997ba194a4ba4f in_seq=0 out_seq=0 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.570+0000 7f1589ffb640 10 client.17097 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.570+0000 7f1589ffb640 1 -- 172.21.15.78:0/1017448323 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a18d979df0 con 0x55a18da1d430 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.570+0000 7f1589ffb640 10 client.17097 handle_command_reply: tid=0 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.570+0000 7f15aaf2a740 1 client.17097 shutdown 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.571+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1017448323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 msgr2=0x55a18da0a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.571+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 0x55a18da0a730 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f158c031440 tx=0x7f158c009260 comp rx=0 tx=0).stop 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.571+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1017448323 shutdown_connections 2024-09-11T01:07:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.571+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da0adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.571+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da01a10 0x55a18da0a730 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.571+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a18da1d430 0x55a18da1f8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.571+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/1017448323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da010f0 0x55a18da0d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.571+0000 7f1589ffb640 0 client.17097 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:07:25.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.571+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1017448323 >> 172.21.15.78:0/1017448323 conn(0x55a18d9fef20 msgr2=0x55a18da04ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:25.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.571+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1017448323 shutdown_connections 2024-09-11T01:07:25.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.571+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1017448323 wait complete. 2024-09-11T01:07:25.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.571+0000 7f15aaf2a740 20 client.17097 trim_cache size 0 max 16384 2024-09-11T01:07:25.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.585+0000 7f15aaf2a740 1 Processor -- start 2024-09-11T01:07:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.585+0000 7f15aaf2a740 1 -- start start 2024-09-11T01:07:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.585+0000 7f15aaf2a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 0x55a18d9fe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.585+0000 7f15aaf2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da01a10 0x55a18da08fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.585+0000 7f15aaf2a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da0d9a0 0x55a18da094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.585+0000 7f15aaf2a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b0810 con 0x55a18da010f0 2024-09-11T01:07:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.585+0000 7f15aaf2a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b23b0 con 0x55a18da0d9a0 2024-09-11T01:07:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.585+0000 7f15aaf2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b0ee0 con 0x55a18da01a10 2024-09-11T01:07:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.585+0000 7f15a259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 0x55a18d9fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.585+0000 7f15a259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 0x55a18d9fe760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55060/0 (socket says 172.21.15.78:55060) 2024-09-11T01:07:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.585+0000 7f15a259c640 1 -- 172.21.15.78:0/298896707 learned_addr learned my addr 172.21.15.78:0/298896707 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.585+0000 7f1599d9b640 1 --2- 172.21.15.78:0/298896707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da01a10 0x55a18da08fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.586+0000 7f15a2d9d640 1 --2- 172.21.15.78:0/298896707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da0d9a0 0x55a18da094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.586+0000 7f15a259c640 1 -- 172.21.15.78:0/298896707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da01a10 msgr2=0x55a18da08fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.586+0000 7f15a259c640 1 --2- 172.21.15.78:0/298896707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da01a10 0x55a18da08fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.586+0000 7f15a259c640 1 -- 172.21.15.78:0/298896707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da0d9a0 msgr2=0x55a18da094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.586+0000 7f15a259c640 1 --2- 172.21.15.78:0/298896707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da0d9a0 0x55a18da094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.586+0000 7f15a259c640 1 -- 172.21.15.78:0/298896707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18d9ffc00 con 0x55a18da010f0 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.586+0000 7f15a259c640 1 --2- 172.21.15.78:0/298896707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 0x55a18d9fe760 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1594019670 tx=0x7f15940196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdd630ed7b0b42a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.586+0000 7f158bfff640 1 -- 172.21.15.78:0/298896707 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159401e040 con 0x55a18da010f0 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.586+0000 7f158bfff640 1 -- 172.21.15.78:0/298896707 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159400faf0 con 0x55a18da010f0 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.586+0000 7f158bfff640 1 -- 172.21.15.78:0/298896707 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594019710 con 0x55a18da010f0 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.587+0000 7f15aaf2a740 1 -- 172.21.15.78:0/298896707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 msgr2=0x55a18d9fe760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.587+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/298896707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 0x55a18d9fe760 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1594019670 tx=0x7f15940196a0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.587+0000 7f15aaf2a740 1 -- 172.21.15.78:0/298896707 shutdown_connections 2024-09-11T01:07:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.587+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/298896707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da0d9a0 0x55a18da094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.587+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/298896707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da01a10 0x55a18da08fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.587+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/298896707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 0x55a18d9fe760 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.587+0000 7f15aaf2a740 1 -- 172.21.15.78:0/298896707 >> 172.21.15.78:0/298896707 conn(0x55a18d9fef20 msgr2=0x55a18d9fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.587+0000 7f15aaf2a740 1 -- 172.21.15.78:0/298896707 shutdown_connections 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.587+0000 7f15aaf2a740 1 -- 172.21.15.78:0/298896707 wait complete. 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.587+0000 7f15aaf2a740 1 -- start start 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.587+0000 7f15aaf2a740 1 Processor -- start 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f15aaf2a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 0x55a18da158f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f15aaf2a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da01a10 0x55a18da15e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f15aaf2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da108a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f15aaf2a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b1cb0 con 0x55a18da010f0 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f15aaf2a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b15e0 con 0x55a18da01a10 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f15aaf2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41ae810 con 0x55a18da0d9a0 2024-09-11T01:07:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f15aaf2a740 10 client.0 _ll_register_callbacks cb 0x7f158c10c240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f1599d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da01a10 0x55a18da15e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f15a2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f15a259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 0x55a18da158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f1599d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da01a10 0x55a18da15e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:57074/0 (socket says 172.21.15.78:57074) 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f15a2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da108a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:55326/0 (socket says 172.21.15.78:55326) 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f1599d9b640 1 -- 172.21.15.78:0/1804238717 learned_addr learned my addr 172.21.15.78:0/1804238717 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f1599d9b640 1 -- 172.21.15.78:0/1804238717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 msgr2=0x55a18da108a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f1599d9b640 1 --2- 172.21.15.78:0/1804238717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da108a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f1599d9b640 1 -- 172.21.15.78:0/1804238717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 msgr2=0x55a18da158f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f1599d9b640 1 --2- 172.21.15.78:0/1804238717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 0x55a18da158f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f1599d9b640 1 -- 172.21.15.78:0/1804238717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18da10e30 con 0x55a18da01a10 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f15a259c640 1 --2- 172.21.15.78:0/1804238717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 0x55a18da158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f1599d9b640 1 --2- 172.21.15.78:0/1804238717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da01a10 0x55a18da15e30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f158c104680 tx=0x7f158c104150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b75dc12bfb0bae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:25.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.588+0000 7f1589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f1589ffb640 1 -- 172.21.15.78:0/1804238717 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c049070 con 0x55a18da01a10 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1804238717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159400faf0 con 0x55a18da01a10 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f1589ffb640 1 -- 172.21.15.78:0/1804238717 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f158c0316b0 con 0x55a18da01a10 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f1589ffb640 1 -- 172.21.15.78:0/1804238717 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f158c0b42e0 con 0x55a18da01a10 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f15aaf2a740 10 client.26360 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1804238717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55a18da8abf0 con 0x55a18da01a10 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f1589ffb640 1 -- 172.21.15.78:0/1804238717 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159400faf0 con 0x55a18da01a10 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f1589ffb640 1 -- 172.21.15.78:0/1804238717 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f158c0c87b0 con 0x55a18da01a10 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f15aaf2a740 10 client.26360 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f15aaf2a740 10 client.26360 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f15aaf2a740 20 client.26360 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/1804238717 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a18da8b030 0x55a18da8d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f15aaf2a740 4 client.26360 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1804238717 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a18d9758a0 con 0x55a18da8b030 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.589+0000 7f15a259c640 1 --2- 172.21.15.78:0/1804238717 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a18da8b030 0x55a18da8d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15a259c640 1 --2- 172.21.15.78:0/1804238717 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a18da8b030 0x55a18da8d450 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4dd538569b77e2f0 server_cookie=13fef87040d5b82f in_seq=0 out_seq=0 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f1589ffb640 10 client.26360 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f1589ffb640 1 -- 172.21.15.78:0/1804238717 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a18d9758a0 con 0x55a18da8b030 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f1589ffb640 10 client.26360 handle_command_reply: tid=0 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15aaf2a740 1 client.26360 shutdown 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1804238717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da01a10 msgr2=0x55a18da15e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/1804238717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da01a10 0x55a18da15e30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f158c104680 tx=0x7f158c104150 comp rx=0 tx=0).stop 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1804238717 shutdown_connections 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/1804238717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a18da0d9a0 0x55a18da108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/1804238717 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a18da8b030 0x55a18da8d450 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/1804238717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a18da01a10 0x55a18da15e30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15aaf2a740 1 --2- 172.21.15.78:0/1804238717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a18da010f0 0x55a18da158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f1589ffb640 0 client.26360 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:07:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1804238717 >> 172.21.15.78:0/1804238717 conn(0x55a18d9fef20 msgr2=0x55a18da08b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1804238717 shutdown_connections 2024-09-11T01:07:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15aaf2a740 1 -- 172.21.15.78:0/1804238717 wait complete. 2024-09-11T01:07:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.590+0000 7f15aaf2a740 20 client.26360 trim_cache size 0 max 16384 2024-09-11T01:07:25.605 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:07:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.591+0000 7f15a8b55640 1 -- 172.21.15.78:0/690871144 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15900b5450 msgr2=0x7f15900b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.591+0000 7f15a8b55640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15900b5450 0x7f15900b7910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f159c009fd0 tx=0x7f159c009500 comp rx=0 tx=0).stop 2024-09-11T01:07:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.591+0000 7f15a8b55640 1 -- 172.21.15.78:0/690871144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15a41076a0 msgr2=0x7f15a40fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.591+0000 7f15a8b55640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15a41076a0 0x7f15a40fe140 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f158c005ab0 tx=0x7f158c003c10 comp rx=0 tx=0).stop 2024-09-11T01:07:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.591+0000 7f15a8b55640 1 -- 172.21.15.78:0/690871144 shutdown_connections 2024-09-11T01:07:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.591+0000 7f15a8b55640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15900b5450 0x7f15900b7910 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.591+0000 7f15a8b55640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a4108020 0x7f15a40fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.591+0000 7f15a8b55640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15a41076a0 0x7f15a40fe140 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.591+0000 7f15a8b55640 1 --2- 172.21.15.78:0/690871144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a4101c90 0x7f15a40fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:25.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.591+0000 7f15a8b55640 1 -- 172.21.15.78:0/690871144 >> 172.21.15.78:0/690871144 conn(0x7f15a40fd700 msgr2=0x7f15a4104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:25.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.591+0000 7f15a8b55640 1 -- 172.21.15.78:0/690871144 shutdown_connections 2024-09-11T01:07:25.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:25.591+0000 7f15a8b55640 1 -- 172.21.15.78:0/690871144 wait complete. 2024-09-11T01:07:26.270 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:07:26.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 591312 5.45 MB/sec execute 334 sec latency 59.181 ms 2024-09-11T01:07:26.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:26 smithi078 ceph-mon[31315]: pgmap v584: 97 pgs: 97 active+clean; 412 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-09-11T01:07:26.527 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:07:26.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:26 smithi110 ceph-mon[34712]: pgmap v584: 97 pgs: 97 active+clean; 412 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-09-11T01:07:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:26 smithi112 ceph-mon[35384]: pgmap v584: 97 pgs: 97 active+clean; 412 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-09-11T01:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.024+0000 7fd532cde640 1 -- 172.21.15.78:0/1744301416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd52c1076a0 msgr2=0x7fd52c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.024+0000 7fd532cde640 1 --2- 172.21.15.78:0/1744301416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd52c1076a0 0x7fd52c107ae0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd518009900 tx=0x7fd51802a410 comp rx=0 tx=0).stop 2024-09-11T01:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.024+0000 7fd532cde640 1 -- 172.21.15.78:0/1744301416 shutdown_connections 2024-09-11T01:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.024+0000 7fd532cde640 1 --2- 172.21.15.78:0/1744301416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd52c108020 0x7fd52c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.024+0000 7fd532cde640 1 --2- 172.21.15.78:0/1744301416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd52c1076a0 0x7fd52c107ae0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.024+0000 7fd532cde640 1 --2- 172.21.15.78:0/1744301416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd52c101c90 0x7fd52c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.024+0000 7fd532cde640 1 -- 172.21.15.78:0/1744301416 >> 172.21.15.78:0/1744301416 conn(0x7fd52c0fd700 msgr2=0x7fd52c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.024+0000 7fd532cde640 1 -- 172.21.15.78:0/1744301416 shutdown_connections 2024-09-11T01:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.024+0000 7fd532cde640 1 -- 172.21.15.78:0/1744301416 wait complete. 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd532cde640 1 Processor -- start 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd532cde640 1 -- start start 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd532cde640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd52c101c90 0x7fd52c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd532cde640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd52c1076a0 0x7fd52c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd532cde640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd52c108020 0x7fd52c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd532cde640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c111cf0 con 0x7fd52c101c90 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd532cde640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c111b70 con 0x7fd52c1076a0 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd532cde640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c111e70 con 0x7fd52c108020 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd530a7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd52c101c90 0x7fd52c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd530a7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd52c101c90 0x7fd52c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55078/0 (socket says 172.21.15.78:55078) 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd530a7a640 1 -- 172.21.15.78:0/376060297 learned_addr learned my addr 172.21.15.78:0/376060297 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd53127b640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd52c108020 0x7fd52c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd52bfff640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd52c1076a0 0x7fd52c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd530a7a640 1 -- 172.21.15.78:0/376060297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd52c108020 msgr2=0x7fd52c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd530a7a640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd52c108020 0x7fd52c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd530a7a640 1 -- 172.21.15.78:0/376060297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd52c1076a0 msgr2=0x7fd52c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd530a7a640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd52c1076a0 0x7fd52c199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.025+0000 7fd530a7a640 1 -- 172.21.15.78:0/376060297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c19f3b0 con 0x7fd52c101c90 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.026+0000 7fd530a7a640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd52c101c90 0x7fd52c199300 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd52000d950 tx=0x7fd52000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.026+0000 7fd529ffb640 1 -- 172.21.15.78:0/376060297 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520017070 con 0x7fd52c101c90 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.026+0000 7fd529ffb640 1 -- 172.21.15.78:0/376060297 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd520009c30 con 0x7fd52c101c90 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.026+0000 7fd532cde640 1 -- 172.21.15.78:0/376060297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52c19f6a0 con 0x7fd52c101c90 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.026+0000 7fd529ffb640 1 -- 172.21.15.78:0/376060297 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52000f620 con 0x7fd52c101c90 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.026+0000 7fd532cde640 1 -- 172.21.15.78:0/376060297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52c1a5d60 con 0x7fd52c101c90 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.028+0000 7fd529ffb640 1 -- 172.21.15.78:0/376060297 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd520010460 con 0x7fd52c101c90 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.028+0000 7fd5350b3740 1 Processor -- start 2024-09-11T01:07:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.028+0000 7fd5350b3740 1 -- start start 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd5350b3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acc80f0 0x55e42acc84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd5350b3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc8a10 0x55e42acd4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd5350b3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acd49a0 0x55e42acd6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd5350b3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e42ac3eea0 con 0x55e42acc8a10 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd5350b3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e42ac2e2d0 con 0x55e42acc80f0 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd5350b3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e42ac320e0 con 0x55e42acd49a0 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd53127b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acc80f0 0x55e42acc84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd530a7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc8a10 0x55e42acd4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd53127b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acc80f0 0x55e42acc84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:57098/0 (socket says 172.21.15.78:57098) 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd53127b640 1 -- 172.21.15.78:0/55294005 learned_addr learned my addr 172.21.15.78:0/55294005 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd52bfff640 1 --2- 172.21.15.78:0/55294005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acd49a0 0x55e42acd6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd53127b640 1 -- 172.21.15.78:0/55294005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acd49a0 msgr2=0x55e42acd6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd53127b640 1 --2- 172.21.15.78:0/55294005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acd49a0 0x55e42acd6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd53127b640 1 -- 172.21.15.78:0/55294005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc8a10 msgr2=0x55e42acd4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd53127b640 1 --2- 172.21.15.78:0/55294005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc8a10 0x55e42acd4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd53127b640 1 -- 172.21.15.78:0/55294005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e42a98c650 con 0x55e42acc80f0 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd529ffb640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd51c0b5450 0x7fd51c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd53127b640 1 --2- 172.21.15.78:0/55294005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acc80f0 0x55e42acc84d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd524010370 tx=0x7fd524010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ae55b98e42d8b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd530a7a640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd51c0b5450 0x7fd51c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd529ffb640 1 -- 172.21.15.78:0/376060297 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fd5200e5bf0 con 0x7fd52c101c90 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.029+0000 7fd516ffd640 1 -- 172.21.15.78:0/55294005 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524016070 con 0x55e42acc80f0 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.030+0000 7fd516ffd640 1 -- 172.21.15.78:0/55294005 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd524004d20 con 0x55e42acc80f0 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.030+0000 7fd516ffd640 1 -- 172.21.15.78:0/55294005 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524005350 con 0x55e42acc80f0 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.030+0000 7fd5350b3740 1 -- 172.21.15.78:0/55294005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acc80f0 msgr2=0x55e42acc84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.030+0000 7fd530a7a640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd51c0b5450 0x7fd51c0b7910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd5200e3430 tx=0x7fd5200e9740 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.030+0000 7fd5350b3740 1 --2- 172.21.15.78:0/55294005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acc80f0 0x55e42acc84d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd524010370 tx=0x7fd524010830 comp rx=0 tx=0).stop 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.030+0000 7fd5350b3740 1 -- 172.21.15.78:0/55294005 shutdown_connections 2024-09-11T01:07:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.030+0000 7fd5350b3740 1 --2- 172.21.15.78:0/55294005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acd49a0 0x55e42acd6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.030+0000 7fd5350b3740 1 --2- 172.21.15.78:0/55294005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc8a10 0x55e42acd4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.030+0000 7fd5350b3740 1 --2- 172.21.15.78:0/55294005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acc80f0 0x55e42acc84d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.030+0000 7fd5350b3740 1 -- 172.21.15.78:0/55294005 >> 172.21.15.78:0/55294005 conn(0x55e42acc5f20 msgr2=0x55e42acc70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.030+0000 7fd5350b3740 1 -- 172.21.15.78:0/55294005 shutdown_connections 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.030+0000 7fd5350b3740 1 -- 172.21.15.78:0/55294005 wait complete. 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd5350b3740 1 -- start start 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd5350b3740 1 Processor -- start 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd5350b3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 0x55e42acd4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd5350b3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acc8a10 0x55e42acd1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd5350b3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acd49a0 0x55e42acd1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd5350b3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e42aa8f700 con 0x55e42acc80f0 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd5350b3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e42abc9160 con 0x55e42acd49a0 2024-09-11T01:07:27.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd5350b3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e42acd7620 con 0x55e42acc8a10 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd5350b3740 10 client.0 _ll_register_callbacks cb 0x55e42abe29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd53127b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 0x55e42acd4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd53127b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 0x55e42acd4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55106/0 (socket says 172.21.15.78:55106) 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd53127b640 1 -- 172.21.15.78:0/1423847975 learned_addr learned my addr 172.21.15.78:0/1423847975 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.031+0000 7fd52bfff640 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acc8a10 0x55e42acd1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd53127b640 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acd49a0 0x55e42acd1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd53127b640 1 -- 172.21.15.78:0/1423847975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acc8a10 msgr2=0x55e42acd1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd53127b640 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acc8a10 0x55e42acd1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd53127b640 1 -- 172.21.15.78:0/1423847975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acd49a0 msgr2=0x55e42acd1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd53127b640 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acd49a0 0x55e42acd1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd53127b640 1 -- 172.21.15.78:0/1423847975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e42abf6c90 con 0x55e42acc80f0 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd53127b640 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 0x55e42acd4230 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd524005cc0 tx=0x7fd524009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8250dd3f8c99c412 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd50bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd50bfff640 1 -- 172.21.15.78:0/1423847975 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524016070 con 0x55e42acc80f0 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd50bfff640 1 -- 172.21.15.78:0/1423847975 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fd524012cd0 con 0x55e42acc80f0 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd5350b3740 1 -- 172.21.15.78:0/1423847975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd524004d20 con 0x55e42acc80f0 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd50bfff640 1 -- 172.21.15.78:0/1423847975 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd524003970 con 0x55e42acc80f0 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd5350b3740 10 client.17115 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd5350b3740 1 -- 172.21.15.78:0/1423847975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e42ace2f20 con 0x55e42acc80f0 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd50bfff640 1 -- 172.21.15.78:0/1423847975 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd524004d20 con 0x55e42acc80f0 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.032+0000 7fd50bfff640 1 -- 172.21.15.78:0/1423847975 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fd524028070 con 0x55e42acc80f0 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.033+0000 7fd5350b3740 10 client.17115 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.033+0000 7fd5350b3740 10 client.17115 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.033+0000 7fd5350b3740 20 client.17115 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.033+0000 7fd5350b3740 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55e42ace42d0 0x55e42ace66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.033+0000 7fd5350b3740 4 client.17115 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.033+0000 7fd5350b3740 1 -- 172.21.15.78:0/1423847975 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e42ac40df0 con 0x55e42ace42d0 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.033+0000 7fd52bfff640 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55e42ace42d0 0x55e42ace66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.033+0000 7fd52bfff640 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55e42ace42d0 0x55e42ace66c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=593074ec13d7f24d server_cookie=573e9466be08fadd in_seq=0 out_seq=0 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.033+0000 7fd50bfff640 10 client.17115 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd50bfff640 1 -- 172.21.15.78:0/1423847975 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e42ac40df0 con 0x55e42ace42d0 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd50bfff640 10 client.17115 handle_command_reply: tid=0 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd5350b3740 1 client.17115 shutdown 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd5350b3740 1 -- 172.21.15.78:0/1423847975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 msgr2=0x55e42acd4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd5350b3740 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 0x55e42acd4230 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd524005cc0 tx=0x7fd524009940 comp rx=0 tx=0).stop 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd5350b3740 1 -- 172.21.15.78:0/1423847975 shutdown_connections 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd5350b3740 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acd49a0 0x55e42acd1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd5350b3740 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acc8a10 0x55e42acd1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd5350b3740 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55e42ace42d0 0x55e42ace66c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd5350b3740 1 --2- 172.21.15.78:0/1423847975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 0x55e42acd4230 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd50bfff640 0 client.17115 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:07:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd5350b3740 1 -- 172.21.15.78:0/1423847975 >> 172.21.15.78:0/1423847975 conn(0x55e42acc5f20 msgr2=0x55e42accbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd5350b3740 1 -- 172.21.15.78:0/1423847975 shutdown_connections 2024-09-11T01:07:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd5350b3740 1 -- 172.21.15.78:0/1423847975 wait complete. 2024-09-11T01:07:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.034+0000 7fd5350b3740 20 client.17115 trim_cache size 0 max 16384 2024-09-11T01:07:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.047+0000 7fd5350b3740 1 Processor -- start 2024-09-11T01:07:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.047+0000 7fd5350b3740 1 -- start start 2024-09-11T01:07:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.047+0000 7fd5350b3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 0x55e42acc5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.047+0000 7fd5350b3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acc8a10 0x55e42acd33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.047+0000 7fd5350b3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acd49a0 0x55e42acd5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.047+0000 7fd5350b3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c1ae690 con 0x55e42acc80f0 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.047+0000 7fd5350b3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c1b0230 con 0x55e42acc8a10 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.047+0000 7fd5350b3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c1aed60 con 0x55e42acd49a0 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.047+0000 7fd53127b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 0x55e42acc5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.047+0000 7fd53127b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 0x55e42acc5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55118/0 (socket says 172.21.15.78:55118) 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.047+0000 7fd53127b640 1 -- 172.21.15.78:0/800987106 learned_addr learned my addr 172.21.15.78:0/800987106 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.048+0000 7fd52bfff640 1 --2- 172.21.15.78:0/800987106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acc8a10 0x55e42acd33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.048+0000 7fd53127b640 1 -- 172.21.15.78:0/800987106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acd49a0 msgr2=0x55e42acd5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.048+0000 7fd53127b640 1 --2- 172.21.15.78:0/800987106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acd49a0 0x55e42acd5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.048+0000 7fd53127b640 1 -- 172.21.15.78:0/800987106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acc8a10 msgr2=0x55e42acd33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.048+0000 7fd53127b640 1 --2- 172.21.15.78:0/800987106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acc8a10 0x55e42acd33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.048+0000 7fd53127b640 1 -- 172.21.15.78:0/800987106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e42accfb80 con 0x55e42acc80f0 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.048+0000 7fd53127b640 1 --2- 172.21.15.78:0/800987106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 0x55e42acc5760 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd524011010 tx=0x7fd524009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5381c2d60a37301d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.048+0000 7fd5167fc640 1 -- 172.21.15.78:0/800987106 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524016040 con 0x55e42acc80f0 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.048+0000 7fd5167fc640 1 -- 172.21.15.78:0/800987106 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd52401f030 con 0x55e42acc80f0 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.048+0000 7fd5167fc640 1 -- 172.21.15.78:0/800987106 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524005770 con 0x55e42acc80f0 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 -- 172.21.15.78:0/800987106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 msgr2=0x55e42acc5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 --2- 172.21.15.78:0/800987106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 0x55e42acc5760 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd524011010 tx=0x7fd524009710 comp rx=0 tx=0).stop 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 -- 172.21.15.78:0/800987106 shutdown_connections 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 --2- 172.21.15.78:0/800987106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acd49a0 0x55e42acd5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 --2- 172.21.15.78:0/800987106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acc8a10 0x55e42acd33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 --2- 172.21.15.78:0/800987106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc80f0 0x55e42acc5760 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 -- 172.21.15.78:0/800987106 >> 172.21.15.78:0/800987106 conn(0x55e42acc5f20 msgr2=0x55e42acd2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 -- 172.21.15.78:0/800987106 shutdown_connections 2024-09-11T01:07:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 -- 172.21.15.78:0/800987106 wait complete. 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 -- start start 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 Processor -- start 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acc80f0 0x55e42ad4e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc8a10 0x55e42ad4cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.049+0000 7fd5350b3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acd49a0 0x55e42ad4d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd5350b3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c1afb30 con 0x55e42acc8a10 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd5350b3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c1af460 con 0x55e42acd49a0 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd5350b3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c1ac690 con 0x55e42acc80f0 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd5350b3740 10 client.0 _ll_register_callbacks cb 0x7fd5200e9610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd52bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc8a10 0x55e42ad4cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd52bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc8a10 0x55e42ad4cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55134/0 (socket says 172.21.15.78:55134) 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd52bfff640 1 -- 172.21.15.78:0/1604479545 learned_addr learned my addr 172.21.15.78:0/1604479545 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd53127b640 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acc80f0 0x55e42ad4e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd53127b640 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acd49a0 0x55e42ad4d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd52bfff640 1 -- 172.21.15.78:0/1604479545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acc80f0 msgr2=0x55e42ad4e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd52bfff640 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acc80f0 0x55e42ad4e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd52bfff640 1 -- 172.21.15.78:0/1604479545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acd49a0 msgr2=0x55e42ad4d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd52bfff640 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acd49a0 0x55e42ad4d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd52bfff640 1 -- 172.21.15.78:0/1604479545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e42ad4da30 con 0x55e42acc8a10 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd53127b640 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acc80f0 0x55e42ad4e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd52bfff640 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc8a10 0x55e42ad4cf00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd518000c00 tx=0x7fd518004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba2885a68c27a21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd50bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd50bfff640 1 -- 172.21.15.78:0/1604479545 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518049070 con 0x55e42acc8a10 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd50bfff640 1 -- 172.21.15.78:0/1604479545 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fd518005450 con 0x55e42acc8a10 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.050+0000 7fd5350b3740 1 -- 172.21.15.78:0/1604479545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd52401f030 con 0x55e42acc8a10 2024-09-11T01:07:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd50bfff640 1 -- 172.21.15.78:0/1604479545 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd518055050 con 0x55e42acc8a10 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd5350b3740 10 client.17121 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd5350b3740 1 -- 172.21.15.78:0/1604479545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e42acdc780 con 0x55e42acc8a10 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd50bfff640 1 -- 172.21.15.78:0/1604479545 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd52401f030 con 0x55e42acc8a10 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd50bfff640 1 -- 172.21.15.78:0/1604479545 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fd518003680 con 0x55e42acc8a10 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd5350b3740 10 client.17121 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd5350b3740 10 client.17121 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd5350b3740 20 client.17121 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd5350b3740 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55e42acdcf00 0x55e42acdd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd5350b3740 4 client.17121 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd5350b3740 1 -- 172.21.15.78:0/1604479545 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e42ac3c8a0 con 0x55e42acdcf00 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd53127b640 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55e42acdcf00 0x55e42acdd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd50bfff640 10 client.17121 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.051+0000 7fd53127b640 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55e42acdcf00 0x55e42acdd310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7525aaae23723c9 server_cookie=84728e2ee7e13857 in_seq=0 out_seq=0 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd50bfff640 1 -- 172.21.15.78:0/1604479545 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e42ac3c8a0 con 0x55e42acdcf00 2024-09-11T01:07:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd50bfff640 10 client.17121 handle_command_reply: tid=0 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd5350b3740 1 client.17121 shutdown 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd5350b3740 1 -- 172.21.15.78:0/1604479545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc8a10 msgr2=0x55e42ad4cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd5350b3740 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc8a10 0x55e42ad4cf00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd518000c00 tx=0x7fd518004480 comp rx=0 tx=0).stop 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd5350b3740 1 -- 172.21.15.78:0/1604479545 shutdown_connections 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd5350b3740 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e42acd49a0 0x55e42ad4d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd5350b3740 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55e42acdcf00 0x55e42acdd310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd5350b3740 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42acc8a10 0x55e42ad4cf00 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd5350b3740 1 --2- 172.21.15.78:0/1604479545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e42acc80f0 0x55e42ad4e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd5350b3740 1 -- 172.21.15.78:0/1604479545 >> 172.21.15.78:0/1604479545 conn(0x55e42acc5f20 msgr2=0x55e42acd34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd50bfff640 0 client.17121 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd5350b3740 1 -- 172.21.15.78:0/1604479545 shutdown_connections 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd5350b3740 1 -- 172.21.15.78:0/1604479545 wait complete. 2024-09-11T01:07:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.052+0000 7fd5350b3740 20 client.17121 trim_cache size 0 max 16384 2024-09-11T01:07:27.065 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:07:27.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.053+0000 7fd532cde640 1 -- 172.21.15.78:0/376060297 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd51c0b5450 msgr2=0x7fd51c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.053+0000 7fd532cde640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd51c0b5450 0x7fd51c0b7910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd5200e3430 tx=0x7fd5200e9740 comp rx=0 tx=0).stop 2024-09-11T01:07:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.053+0000 7fd532cde640 1 -- 172.21.15.78:0/376060297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd52c101c90 msgr2=0x7fd52c199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.053+0000 7fd532cde640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd52c101c90 0x7fd52c199300 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd52000d950 tx=0x7fd52000de10 comp rx=0 tx=0).stop 2024-09-11T01:07:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.053+0000 7fd532cde640 1 -- 172.21.15.78:0/376060297 shutdown_connections 2024-09-11T01:07:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.053+0000 7fd532cde640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd51c0b5450 0x7fd51c0b7910 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.053+0000 7fd532cde640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd52c108020 0x7fd52c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.053+0000 7fd532cde640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd52c1076a0 0x7fd52c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.053+0000 7fd532cde640 1 --2- 172.21.15.78:0/376060297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd52c101c90 0x7fd52c199300 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.053+0000 7fd532cde640 1 -- 172.21.15.78:0/376060297 >> 172.21.15.78:0/376060297 conn(0x7fd52c0fd700 msgr2=0x7fd52c06adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.053+0000 7fd532cde640 1 -- 172.21.15.78:0/376060297 shutdown_connections 2024-09-11T01:07:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:27.053+0000 7fd532cde640 1 -- 172.21.15.78:0/376060297 wait complete. 2024-09-11T01:07:27.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 592029 5.43 MB/sec execute 335 sec latency 809.708 ms 2024-09-11T01:07:27.740 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:07:28.009 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:07:28.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:28 smithi078 ceph-mon[31315]: pgmap v585: 97 pgs: 97 active+clean; 412 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.8 MiB/s wr, 155 op/s 2024-09-11T01:07:28.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 593612 5.45 MB/sec execute 336 sec latency 67.664 ms 2024-09-11T01:07:28.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.500+0000 7ff29a35e640 1 -- 172.21.15.78:0/105223955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff294101c90 msgr2=0x7ff294102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:28.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.500+0000 7ff29a35e640 1 --2- 172.21.15.78:0/105223955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff294101c90 0x7ff294102070 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff284009900 tx=0x7ff28402a410 comp rx=0 tx=0).stop 2024-09-11T01:07:28.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.500+0000 7ff29a35e640 1 -- 172.21.15.78:0/105223955 shutdown_connections 2024-09-11T01:07:28.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.500+0000 7ff29a35e640 1 --2- 172.21.15.78:0/105223955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff294108020 0x7ff29410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.500+0000 7ff29a35e640 1 --2- 172.21.15.78:0/105223955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2941076a0 0x7ff294107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.500+0000 7ff29a35e640 1 --2- 172.21.15.78:0/105223955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff294101c90 0x7ff294102070 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.500+0000 7ff29a35e640 1 -- 172.21.15.78:0/105223955 >> 172.21.15.78:0/105223955 conn(0x7ff2940fd700 msgr2=0x7ff2940ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.500+0000 7ff29a35e640 1 -- 172.21.15.78:0/105223955 shutdown_connections 2024-09-11T01:07:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.500+0000 7ff29a35e640 1 -- 172.21.15.78:0/105223955 wait complete. 2024-09-11T01:07:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff29a35e640 1 Processor -- start 2024-09-11T01:07:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff29a35e640 1 -- start start 2024-09-11T01:07:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff29a35e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff294101c90 0x7ff2941a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff29a35e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2941076a0 0x7ff2941a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff29a35e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff294108020 0x7ff29419f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff29a35e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff294111ce0 con 0x7ff2941076a0 2024-09-11T01:07:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff29a35e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff294111b60 con 0x7ff294101c90 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff29a35e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff294111e60 con 0x7ff294108020 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff28bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2941076a0 0x7ff2941a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff28bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2941076a0 0x7ff2941a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55158/0 (socket says 172.21.15.78:55158) 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff28bfff640 1 -- 172.21.15.78:0/4003677160 learned_addr learned my addr 172.21.15.78:0/4003677160 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff293fff640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff294101c90 0x7ff2941a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff2988fb640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff294108020 0x7ff29419f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff28bfff640 1 -- 172.21.15.78:0/4003677160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff294108020 msgr2=0x7ff29419f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff28bfff640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff294108020 0x7ff29419f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff28bfff640 1 -- 172.21.15.78:0/4003677160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff294101c90 msgr2=0x7ff2941a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff28bfff640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff294101c90 0x7ff2941a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff28bfff640 1 -- 172.21.15.78:0/4003677160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2941a0220 con 0x7ff2941076a0 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.501+0000 7ff2988fb640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff294108020 0x7ff29419f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:07:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.502+0000 7ff28bfff640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2941076a0 0x7ff2941a5990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00e9f0 tx=0x7ff27c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.502+0000 7ff291ffb640 1 -- 172.21.15.78:0/4003677160 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00cd80 con 0x7ff2941076a0 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.502+0000 7ff291ffb640 1 -- 172.21.15.78:0/4003677160 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff27c0054d0 con 0x7ff2941076a0 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.502+0000 7ff29a35e640 1 -- 172.21.15.78:0/4003677160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2941a04b0 con 0x7ff2941076a0 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.502+0000 7ff291ffb640 1 -- 172.21.15.78:0/4003677160 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c010690 con 0x7ff2941076a0 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.502+0000 7ff29a35e640 1 -- 172.21.15.78:0/4003677160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2941ac460 con 0x7ff2941076a0 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff291ffb640 1 -- 172.21.15.78:0/4003677160 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff27c004e90 con 0x7ff2941076a0 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff29c733740 1 Processor -- start 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff29c733740 1 -- start start 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff29c733740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1374d0 0x5597cf1378b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff29c733740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf137df0 0x5597cf143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff29c733740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf143d80 0x5597cf146170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff29c733740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5597cf0adea0 con 0x5597cf137df0 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff29c733740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5597cf09d2d0 con 0x5597cf1374d0 2024-09-11T01:07:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff29c733740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5597cf0a10e0 con 0x5597cf143d80 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff293fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf137df0 0x5597cf143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff2988fb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1374d0 0x5597cf1378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff293fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf137df0 0x5597cf143840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55166/0 (socket says 172.21.15.78:55166) 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff293fff640 1 -- 172.21.15.78:0/80747424 learned_addr learned my addr 172.21.15.78:0/80747424 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.505+0000 7ff28bfff640 1 --2- 172.21.15.78:0/80747424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf143d80 0x5597cf146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.506+0000 7ff293fff640 1 -- 172.21.15.78:0/80747424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf143d80 msgr2=0x5597cf146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.506+0000 7ff293fff640 1 --2- 172.21.15.78:0/80747424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf143d80 0x5597cf146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.506+0000 7ff293fff640 1 -- 172.21.15.78:0/80747424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1374d0 msgr2=0x5597cf1378b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.506+0000 7ff293fff640 1 --2- 172.21.15.78:0/80747424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1374d0 0x5597cf1378b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.506+0000 7ff293fff640 1 -- 172.21.15.78:0/80747424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597cedfb650 con 0x5597cf137df0 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.506+0000 7ff291ffb640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff2800b54d0 0x7ff2800b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.506+0000 7ff293fff640 1 --2- 172.21.15.78:0/80747424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf137df0 0x5597cf143840 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff284009900 tx=0x7ff284032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7f193952a93825 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.506+0000 7ff2988fb640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff2800b54d0 0x7ff2800b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.506+0000 7ff28a7fc640 1 -- 172.21.15.78:0/80747424 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284040070 con 0x5597cf137df0 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.506+0000 7ff291ffb640 1 -- 172.21.15.78:0/4003677160 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7ff27c0e64c0 con 0x7ff2941076a0 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.506+0000 7ff28a7fc640 1 -- 172.21.15.78:0/80747424 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff284004030 con 0x5597cf137df0 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.506+0000 7ff28a7fc640 1 -- 172.21.15.78:0/80747424 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284035040 con 0x5597cf137df0 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.507+0000 7ff29c733740 1 -- 172.21.15.78:0/80747424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf137df0 msgr2=0x5597cf143840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.507+0000 7ff2988fb640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff2800b54d0 0x7ff2800b7990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff28c005fd0 tx=0x7ff28c009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.507+0000 7ff29c733740 1 --2- 172.21.15.78:0/80747424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf137df0 0x5597cf143840 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff284009900 tx=0x7ff284032b20 comp rx=0 tx=0).stop 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.507+0000 7ff29c733740 1 -- 172.21.15.78:0/80747424 shutdown_connections 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.507+0000 7ff29c733740 1 --2- 172.21.15.78:0/80747424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf143d80 0x5597cf146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.507+0000 7ff29c733740 1 --2- 172.21.15.78:0/80747424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf137df0 0x5597cf143840 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.507+0000 7ff29c733740 1 --2- 172.21.15.78:0/80747424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1374d0 0x5597cf1378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.507+0000 7ff29c733740 1 -- 172.21.15.78:0/80747424 >> 172.21.15.78:0/80747424 conn(0x5597cf135300 msgr2=0x5597cf1364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.507+0000 7ff29c733740 1 -- 172.21.15.78:0/80747424 shutdown_connections 2024-09-11T01:07:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.507+0000 7ff29c733740 1 -- 172.21.15.78:0/80747424 wait complete. 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.507+0000 7ff29c733740 1 -- start start 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.508+0000 7ff29c733740 1 Processor -- start 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.508+0000 7ff29c733740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf1374d0 0x5597cf143830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.508+0000 7ff29c733740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf143d80 0x5597cf140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.508+0000 7ff29c733740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 0x5597cf14fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.508+0000 7ff29c733740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5597ceefe700 con 0x5597cf143d80 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.508+0000 7ff29c733740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5597cf038160 con 0x5597cf1413b0 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.508+0000 7ff29c733740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5597cf1469f0 con 0x5597cf1374d0 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.508+0000 7ff29c733740 10 client.0 _ll_register_callbacks cb 0x5597cf0519e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.508+0000 7ff28bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf143d80 0x5597cf140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.508+0000 7ff28bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf143d80 0x5597cf140cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55178/0 (socket says 172.21.15.78:55178) 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.508+0000 7ff28bfff640 1 -- 172.21.15.78:0/2033870206 learned_addr learned my addr 172.21.15.78:0/2033870206 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.508+0000 7ff2988fb640 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 0x5597cf14fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff2988fb640 1 -- 172.21.15.78:0/2033870206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf1374d0 msgr2=0x5597cf143830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff293fff640 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf1374d0 0x5597cf143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff2988fb640 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf1374d0 0x5597cf143830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff2988fb640 1 -- 172.21.15.78:0/2033870206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf143d80 msgr2=0x5597cf140cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff2988fb640 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf143d80 0x5597cf140cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff2988fb640 1 -- 172.21.15.78:0/2033870206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597cf065c90 con 0x5597cf1413b0 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff28bfff640 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf143d80 0x5597cf140cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff2988fb640 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 0x5597cf14fd50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0126d0 tx=0x7ff28c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=884ffd9f55ea0453 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff2737fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff2737fe640 1 -- 172.21.15.78:0/2033870206 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c01d980 con 0x5597cf1413b0 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff2737fe640 1 -- 172.21.15.78:0/2033870206 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7ff28c01db20 con 0x5597cf1413b0 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff29c733740 1 -- 172.21.15.78:0/2033870206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff284004030 con 0x5597cf1413b0 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff2737fe640 1 -- 172.21.15.78:0/2033870206 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff28c01dd20 con 0x5597cf1413b0 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff29c733740 10 client.26387 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff29c733740 1 -- 172.21.15.78:0/2033870206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5597cf152970 con 0x5597cf1413b0 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.509+0000 7ff2737fe640 1 -- 172.21.15.78:0/2033870206 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff28c022070 con 0x5597cf1413b0 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.510+0000 7ff2737fe640 1 -- 172.21.15.78:0/2033870206 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7ff28c030560 con 0x5597cf1413b0 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.510+0000 7ff29c733740 10 client.26387 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.510+0000 7ff29c733740 10 client.26387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.510+0000 7ff29c733740 20 client.26387 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.510+0000 7ff29c733740 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5597cf153dd0 0x5597cf1561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.510+0000 7ff29c733740 4 client.26387 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.510+0000 7ff29c733740 1 -- 172.21.15.78:0/2033870206 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff284004030 con 0x5597cf153dd0 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.510+0000 7ff293fff640 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5597cf153dd0 0x5597cf1561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.510+0000 7ff293fff640 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5597cf153dd0 0x5597cf1561c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d78fb542594eda53 server_cookie=678949cf2e75f548 in_seq=0 out_seq=0 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.510+0000 7ff2737fe640 10 client.26387 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.511+0000 7ff2737fe640 1 -- 172.21.15.78:0/2033870206 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff284004030 con 0x5597cf153dd0 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.511+0000 7ff2737fe640 10 client.26387 handle_command_reply: tid=0 2024-09-11T01:07:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.511+0000 7ff29c733740 1 client.26387 shutdown 2024-09-11T01:07:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.512+0000 7ff29c733740 1 -- 172.21.15.78:0/2033870206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 msgr2=0x5597cf14fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.512+0000 7ff29c733740 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 0x5597cf14fd50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0126d0 tx=0x7ff28c004a40 comp rx=0 tx=0).stop 2024-09-11T01:07:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.512+0000 7ff29c733740 1 -- 172.21.15.78:0/2033870206 shutdown_connections 2024-09-11T01:07:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.512+0000 7ff29c733740 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 0x5597cf14fd50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.512+0000 7ff29c733740 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5597cf153dd0 0x5597cf1561c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.512+0000 7ff29c733740 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf143d80 0x5597cf140cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.512+0000 7ff29c733740 1 --2- 172.21.15.78:0/2033870206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf1374d0 0x5597cf143830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.512+0000 7ff29c733740 1 -- 172.21.15.78:0/2033870206 >> 172.21.15.78:0/2033870206 conn(0x5597cf135300 msgr2=0x5597cf13aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.512+0000 7ff2737fe640 0 client.26387 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.512+0000 7ff29c733740 1 -- 172.21.15.78:0/2033870206 shutdown_connections 2024-09-11T01:07:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.512+0000 7ff29c733740 1 -- 172.21.15.78:0/2033870206 wait complete. 2024-09-11T01:07:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.512+0000 7ff29c733740 20 client.26387 trim_cache size 0 max 16384 2024-09-11T01:07:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff29c733740 1 Processor -- start 2024-09-11T01:07:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff29c733740 1 -- start start 2024-09-11T01:07:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff29c733740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1374d0 0x5597cf155de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff29c733740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 0x5597cf14fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff29c733740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf143d80 0x5597cf150190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff29c733740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941b4c00 con 0x5597cf1374d0 2024-09-11T01:07:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff29c733740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941b67a0 con 0x5597cf1413b0 2024-09-11T01:07:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff29c733740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941b52d0 con 0x5597cf143d80 2024-09-11T01:07:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff293fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1374d0 0x5597cf155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff28bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 0x5597cf14fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff293fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1374d0 0x5597cf155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55188/0 (socket says 172.21.15.78:55188) 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff28bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 0x5597cf14fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:57192/0 (socket says 172.21.15.78:57192) 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff2988fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf143d80 0x5597cf150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff28bfff640 1 -- 172.21.15.78:0/1661887501 learned_addr learned my addr 172.21.15.78:0/1661887501 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff293fff640 1 -- 172.21.15.78:0/1661887501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf143d80 msgr2=0x5597cf150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff293fff640 1 --2- 172.21.15.78:0/1661887501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf143d80 0x5597cf150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff293fff640 1 -- 172.21.15.78:0/1661887501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 msgr2=0x5597cf14fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff293fff640 1 --2- 172.21.15.78:0/1661887501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 0x5597cf14fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff293fff640 1 -- 172.21.15.78:0/1661887501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597cf137c20 con 0x5597cf1374d0 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff28bfff640 1 --2- 172.21.15.78:0/1661887501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 0x5597cf14fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.526+0000 7ff293fff640 1 --2- 172.21.15.78:0/1661887501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1374d0 0x5597cf155de0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x5597cf136c50 tx=0x7ff284005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49847daf8c9e753d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.527+0000 7ff289ffb640 1 -- 172.21.15.78:0/1661887501 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284040040 con 0x5597cf1374d0 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.527+0000 7ff289ffb640 1 -- 172.21.15.78:0/1661887501 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2840316a0 con 0x5597cf1374d0 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.527+0000 7ff289ffb640 1 -- 172.21.15.78:0/1661887501 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284035040 con 0x5597cf1374d0 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.527+0000 7ff29c733740 1 -- 172.21.15.78:0/1661887501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1374d0 msgr2=0x5597cf155de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.527+0000 7ff29c733740 1 --2- 172.21.15.78:0/1661887501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1374d0 0x5597cf155de0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x5597cf136c50 tx=0x7ff284005150 comp rx=0 tx=0).stop 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.527+0000 7ff29c733740 1 -- 172.21.15.78:0/1661887501 shutdown_connections 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.527+0000 7ff29c733740 1 --2- 172.21.15.78:0/1661887501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf143d80 0x5597cf150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.527+0000 7ff29c733740 1 --2- 172.21.15.78:0/1661887501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf1413b0 0x5597cf14fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.527+0000 7ff29c733740 1 --2- 172.21.15.78:0/1661887501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1374d0 0x5597cf155de0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.527+0000 7ff29c733740 1 -- 172.21.15.78:0/1661887501 >> 172.21.15.78:0/1661887501 conn(0x5597cf135300 msgr2=0x5597cf13f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.527+0000 7ff29c733740 1 -- 172.21.15.78:0/1661887501 shutdown_connections 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.528+0000 7ff29c733740 1 -- 172.21.15.78:0/1661887501 wait complete. 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.528+0000 7ff29c733740 1 -- start start 2024-09-11T01:07:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.528+0000 7ff29c733740 1 Processor -- start 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.528+0000 7ff29c733740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf1374d0 0x5597cf14bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.528+0000 7ff29c733740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1413b0 0x5597cf14c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.528+0000 7ff29c733740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf143d80 0x5597cf146c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.528+0000 7ff29c733740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941b60a0 con 0x5597cf1413b0 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.528+0000 7ff29c733740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941b59d0 con 0x5597cf143d80 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.528+0000 7ff29c733740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941b2c00 con 0x5597cf1374d0 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.528+0000 7ff29c733740 10 client.0 _ll_register_callbacks cb 0x7ff28c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff293fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf1374d0 0x5597cf14bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff28bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1413b0 0x5597cf14c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff293fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf1374d0 0x5597cf14bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:55432/0 (socket says 172.21.15.78:55432) 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff293fff640 1 -- 172.21.15.78:0/2338503141 learned_addr learned my addr 172.21.15.78:0/2338503141 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff2988fb640 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf143d80 0x5597cf146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff293fff640 1 -- 172.21.15.78:0/2338503141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf143d80 msgr2=0x5597cf146c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff293fff640 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf143d80 0x5597cf146c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff293fff640 1 -- 172.21.15.78:0/2338503141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1413b0 msgr2=0x5597cf14c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff293fff640 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1413b0 0x5597cf14c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff293fff640 1 -- 172.21.15.78:0/2338503141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597cf147200 con 0x5597cf1374d0 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff28bfff640 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1413b0 0x5597cf14c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff2988fb640 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf143d80 0x5597cf146c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff293fff640 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf1374d0 0x5597cf14bcc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff284009850 tx=0x7ff2840051c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eecd450211a9bfaa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff2737fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff2737fe640 1 -- 172.21.15.78:0/2338503141 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28405b070 con 0x5597cf1374d0 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff2737fe640 1 -- 172.21.15.78:0/2338503141 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7ff284005470 con 0x5597cf1374d0 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.529+0000 7ff29c733740 1 -- 172.21.15.78:0/2338503141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2840316a0 con 0x5597cf1374d0 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff2737fe640 1 -- 172.21.15.78:0/2338503141 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff284067050 con 0x5597cf1374d0 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff29c733740 10 client.26329 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff29c733740 1 -- 172.21.15.78:0/2338503141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5597cf1c1530 con 0x5597cf1374d0 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff2737fe640 1 -- 172.21.15.78:0/2338503141 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2840316a0 con 0x5597cf1374d0 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff2737fe640 1 -- 172.21.15.78:0/2338503141 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7ff284003680 con 0x5597cf1374d0 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff29c733740 10 client.26329 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff29c733740 10 client.26329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff29c733740 20 client.26329 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff29c733740 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5597cf1c1800 0x5597cf1c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff29c733740 4 client.26329 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff29c733740 1 -- 172.21.15.78:0/2338503141 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597cf0afdf0 con 0x5597cf1c1800 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff28bfff640 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5597cf1c1800 0x5597cf1c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff2737fe640 10 client.26329 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.530+0000 7ff28bfff640 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5597cf1c1800 0x5597cf1c3bf0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4450da011ef1648 server_cookie=e3f65e037756b37b in_seq=0 out_seq=0 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff2737fe640 1 -- 172.21.15.78:0/2338503141 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597cf0afdf0 con 0x5597cf1c1800 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff2737fe640 10 client.26329 handle_command_reply: tid=0 2024-09-11T01:07:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff29c733740 1 client.26329 shutdown 2024-09-11T01:07:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff29c733740 1 -- 172.21.15.78:0/2338503141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf1374d0 msgr2=0x5597cf14bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff29c733740 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf1374d0 0x5597cf14bcc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff284009850 tx=0x7ff2840051c0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff29c733740 1 -- 172.21.15.78:0/2338503141 shutdown_connections 2024-09-11T01:07:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff29c733740 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597cf143d80 0x5597cf146c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff29c733740 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5597cf1c1800 0x5597cf1c3bf0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff29c733740 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597cf1413b0 0x5597cf14c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff2737fe640 0 client.26329 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff29c733740 1 --2- 172.21.15.78:0/2338503141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597cf1374d0 0x5597cf14bcc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff29c733740 1 -- 172.21.15.78:0/2338503141 >> 172.21.15.78:0/2338503141 conn(0x5597cf135300 msgr2=0x5597cf1422b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff29c733740 1 -- 172.21.15.78:0/2338503141 shutdown_connections 2024-09-11T01:07:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff29c733740 1 -- 172.21.15.78:0/2338503141 wait complete. 2024-09-11T01:07:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.531+0000 7ff29c733740 20 client.26329 trim_cache size 0 max 16384 2024-09-11T01:07:28.545 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:07:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.532+0000 7ff29a35e640 1 -- 172.21.15.78:0/4003677160 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff2800b54d0 msgr2=0x7ff2800b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.532+0000 7ff29a35e640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff2800b54d0 0x7ff2800b7990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff28c005fd0 tx=0x7ff28c009500 comp rx=0 tx=0).stop 2024-09-11T01:07:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.532+0000 7ff29a35e640 1 -- 172.21.15.78:0/4003677160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2941076a0 msgr2=0x7ff2941a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.532+0000 7ff29a35e640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2941076a0 0x7ff2941a5990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00e9f0 tx=0x7ff27c00edf0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.532+0000 7ff29a35e640 1 -- 172.21.15.78:0/4003677160 shutdown_connections 2024-09-11T01:07:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.532+0000 7ff29a35e640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff2800b54d0 0x7ff2800b7990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.532+0000 7ff29a35e640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff294108020 0x7ff29419f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.532+0000 7ff29a35e640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2941076a0 0x7ff2941a5990 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.532+0000 7ff29a35e640 1 --2- 172.21.15.78:0/4003677160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff294101c90 0x7ff2941a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.532+0000 7ff29a35e640 1 -- 172.21.15.78:0/4003677160 >> 172.21.15.78:0/4003677160 conn(0x7ff2940fd700 msgr2=0x7ff2940ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.532+0000 7ff29a35e640 1 -- 172.21.15.78:0/4003677160 shutdown_connections 2024-09-11T01:07:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:28.532+0000 7ff29a35e640 1 -- 172.21.15.78:0/4003677160 wait complete. 2024-09-11T01:07:28.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:28 smithi110 ceph-mon[34712]: pgmap v585: 97 pgs: 97 active+clean; 412 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.8 MiB/s wr, 155 op/s 2024-09-11T01:07:28.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:28 smithi112 ceph-mon[35384]: pgmap v585: 97 pgs: 97 active+clean; 412 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.8 MiB/s wr, 155 op/s 2024-09-11T01:07:29.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 594856 5.45 MB/sec execute 337 sec latency 278.344 ms 2024-09-11T01:07:30.240 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:07:30.240 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:07:30.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 594856 5.44 MB/sec execute 338 sec latency 1278.382 ms 2024-09-11T01:07:30.491 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:07:30.611 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:30 smithi078 ceph-mon[31315]: pgmap v586: 97 pgs: 97 active+clean; 415 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-09-11T01:07:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:30 smithi110 ceph-mon[34712]: pgmap v586: 97 pgs: 97 active+clean; 415 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-09-11T01:07:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:30 smithi112 ceph-mon[35384]: pgmap v586: 97 pgs: 97 active+clean; 415 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-09-11T01:07:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.972+0000 7fb2bfa96640 1 -- 172.21.15.78:0/1629220377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b8075e60 msgr2=0x7fb2b81135e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.972+0000 7fb2bfa96640 1 --2- 172.21.15.78:0/1629220377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b8075e60 0x7fb2b81135e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009900 tx=0x7fb2ac02a410 comp rx=0 tx=0).stop 2024-09-11T01:07:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.973+0000 7fb2bfa96640 1 -- 172.21.15.78:0/1629220377 shutdown_connections 2024-09-11T01:07:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.973+0000 7fb2bfa96640 1 --2- 172.21.15.78:0/1629220377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2b8113b20 0x7fb2b8115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.973+0000 7fb2bfa96640 1 --2- 172.21.15.78:0/1629220377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b8075e60 0x7fb2b81135e0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.973+0000 7fb2bfa96640 1 --2- 172.21.15.78:0/1629220377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b8075540 0x7fb2b8075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.973+0000 7fb2bfa96640 1 -- 172.21.15.78:0/1629220377 >> 172.21.15.78:0/1629220377 conn(0x7fb2b80fe830 msgr2=0x7fb2b8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.973+0000 7fb2bfa96640 1 -- 172.21.15.78:0/1629220377 shutdown_connections 2024-09-11T01:07:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.973+0000 7fb2bfa96640 1 -- 172.21.15.78:0/1629220377 wait complete. 2024-09-11T01:07:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.973+0000 7fb2bfa96640 1 Processor -- start 2024-09-11T01:07:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.973+0000 7fb2bfa96640 1 -- start start 2024-09-11T01:07:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bfa96640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b8075540 0x7fb2b819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bfa96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b8075e60 0x7fb2b819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bfa96640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2b8113b20 0x7fb2b81a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bfa96640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8119410 con 0x7fb2b8075540 2024-09-11T01:07:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bfa96640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8119290 con 0x7fb2b8113b20 2024-09-11T01:07:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bfa96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8119590 con 0x7fb2b8075e60 2024-09-11T01:07:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bd832640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b8075540 0x7fb2b819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bd832640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b8075540 0x7fb2b819f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55220/0 (socket says 172.21.15.78:55220) 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bd832640 1 -- 172.21.15.78:0/3485328458 learned_addr learned my addr 172.21.15.78:0/3485328458 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bd031640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b8075e60 0x7fb2b819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2be033640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2b8113b20 0x7fb2b81a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bd832640 1 -- 172.21.15.78:0/3485328458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b8075e60 msgr2=0x7fb2b819fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bd832640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b8075e60 0x7fb2b819fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bd832640 1 -- 172.21.15.78:0/3485328458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2b8113b20 msgr2=0x7fb2b81a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bd832640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2b8113b20 0x7fb2b81a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bd832640 1 -- 172.21.15.78:0/3485328458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b81a59f0 con 0x7fb2b8075540 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2bd832640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b8075540 0x7fb2b819f940 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800b4a0 tx=0x7fb2a800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.974+0000 7fb2b6ffd640 1 -- 172.21.15.78:0/3485328458 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8014070 con 0x7fb2b8075540 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.975+0000 7fb2b6ffd640 1 -- 172.21.15.78:0/3485328458 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a8010ac0 con 0x7fb2b8075540 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.975+0000 7fb2bfa96640 1 -- 172.21.15.78:0/3485328458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b81a5ce0 con 0x7fb2b8075540 2024-09-11T01:07:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.975+0000 7fb2b6ffd640 1 -- 172.21.15.78:0/3485328458 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8017c30 con 0x7fb2b8075540 2024-09-11T01:07:30.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.975+0000 7fb2bfa96640 1 -- 172.21.15.78:0/3485328458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b81ac3a0 con 0x7fb2b8075540 2024-09-11T01:07:30.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.977+0000 7fb2b6ffd640 1 -- 172.21.15.78:0/3485328458 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb2a8010610 con 0x7fb2b8075540 2024-09-11T01:07:30.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.977+0000 7fb2c1e6b740 1 Processor -- start 2024-09-11T01:07:30.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.977+0000 7fb2c1e6b740 1 -- start start 2024-09-11T01:07:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.977+0000 7fb2c1e6b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715d902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.977+0000 7fb2c1e6b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d90800 0x55a715d9c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2c1e6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d9c790 0x55a715d9eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2c1e6b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a715d06e60 con 0x55a715d90800 2024-09-11T01:07:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2c1e6b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a715cf6290 con 0x55a715d8fee0 2024-09-11T01:07:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2c1e6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a715cfa0a0 con 0x55a715d9c790 2024-09-11T01:07:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2bd832640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d90800 0x55a715d9c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2be033640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715d902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2bd832640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d90800 0x55a715d9c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55222/0 (socket says 172.21.15.78:55222) 2024-09-11T01:07:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2bd832640 1 -- 172.21.15.78:0/3982485194 learned_addr learned my addr 172.21.15.78:0/3982485194 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2bd031640 1 --2- 172.21.15.78:0/3982485194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d9c790 0x55a715d9eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:30.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2bd832640 1 -- 172.21.15.78:0/3982485194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d9c790 msgr2=0x55a715d9eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:30.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2bd832640 1 --2- 172.21.15.78:0/3982485194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d9c790 0x55a715d9eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2bd832640 1 -- 172.21.15.78:0/3982485194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 msgr2=0x55a715d902c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:30.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2bd832640 1 --2- 172.21.15.78:0/3982485194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715d902c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2bd832640 1 -- 172.21.15.78:0/3982485194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a715a54930 con 0x55a715d90800 2024-09-11T01:07:30.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2be033640 1 --2- 172.21.15.78:0/3982485194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715d902c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:30.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2b6ffd640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb2a00b5450 0x7fb2a00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:30.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2bd031640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb2a00b5450 0x7fb2a00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:30.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2bd832640 1 --2- 172.21.15.78:0/3982485194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d90800 0x55a715d9c250 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80e2ce0 tx=0x7fb2a80ed690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e87c1024b027331 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:30.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.978+0000 7fb2b6ffd640 1 -- 172.21.15.78:0/3485328458 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fb2a80e5610 con 0x7fb2b8075540 2024-09-11T01:07:30.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.979+0000 7fb29ffff640 1 -- 172.21.15.78:0/3982485194 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a801e070 con 0x55a715d90800 2024-09-11T01:07:30.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.979+0000 7fb29ffff640 1 -- 172.21.15.78:0/3982485194 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a80a7230 con 0x55a715d90800 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.979+0000 7fb29ffff640 1 -- 172.21.15.78:0/3982485194 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80a7970 con 0x55a715d90800 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.979+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3982485194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d90800 msgr2=0x55a715d9c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.979+0000 7fb2bd031640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb2a00b5450 0x7fb2a00b7910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac031440 tx=0x7fb2ac005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.979+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/3982485194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d90800 0x55a715d9c250 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80e2ce0 tx=0x7fb2a80ed690 comp rx=0 tx=0).stop 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.979+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3982485194 shutdown_connections 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.979+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/3982485194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d9c790 0x55a715d9eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.979+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/3982485194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d90800 0x55a715d9c250 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.979+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/3982485194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715d902c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.979+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3982485194 >> 172.21.15.78:0/3982485194 conn(0x55a715d8dd10 msgr2=0x55a715d8eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.980+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3982485194 shutdown_connections 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.980+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3982485194 wait complete. 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.980+0000 7fb2c1e6b740 1 -- start start 2024-09-11T01:07:30.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.980+0000 7fb2c1e6b740 1 Processor -- start 2024-09-11T01:07:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.980+0000 7fb2c1e6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d8fee0 0x55a715d99710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.980+0000 7fb2c1e6b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d90800 0x55a715d99c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.980+0000 7fb2c1e6b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715d9a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.980+0000 7fb2c1e6b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a715b57890 con 0x55a715d9c790 2024-09-11T01:07:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.980+0000 7fb2c1e6b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a715c90d40 con 0x55a715d90800 2024-09-11T01:07:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.980+0000 7fb2c1e6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a715d9f440 con 0x55a715d8fee0 2024-09-11T01:07:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.980+0000 7fb2c1e6b740 10 client.0 _ll_register_callbacks cb 0x55a715cf4c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2be033640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715d9a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2be033640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715d9a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55224/0 (socket says 172.21.15.78:55224) 2024-09-11T01:07:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2be033640 1 -- 172.21.15.78:0/4255898525 learned_addr learned my addr 172.21.15.78:0/4255898525 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2bd031640 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d90800 0x55a715d99c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2bd832640 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d8fee0 0x55a715d99710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2be033640 1 -- 172.21.15.78:0/4255898525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d8fee0 msgr2=0x55a715d99710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2be033640 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d8fee0 0x55a715d99710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2be033640 1 -- 172.21.15.78:0/4255898525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d90800 msgr2=0x55a715d99c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2be033640 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d90800 0x55a715d99c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2be033640 1 -- 172.21.15.78:0/4255898525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a715da87f0 con 0x55a715d9c790 2024-09-11T01:07:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2be033640 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715d9a190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00129a0 tx=0x7fb2b0012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9112e5ec9b4cf7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb29d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:07:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb29d7fa640 1 -- 172.21.15.78:0/4255898525 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0010e80 con 0x55a715d9c790 2024-09-11T01:07:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb29d7fa640 1 -- 172.21.15.78:0/4255898525 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fb2b001b390 con 0x55a715d9c790 2024-09-11T01:07:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/4255898525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2a80a7230 con 0x55a715d9c790 2024-09-11T01:07:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb29d7fa640 1 -- 172.21.15.78:0/4255898525 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fb2b0004b20 con 0x55a715d9c790 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2c1e6b740 10 client.17151 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.981+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/4255898525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55a715dab1e0 con 0x55a715d9c790 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.982+0000 7fb29d7fa640 1 -- 172.21.15.78:0/4255898525 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a80a7230 con 0x55a715d9c790 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.982+0000 7fb29d7fa640 1 -- 172.21.15.78:0/4255898525 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fb2b000f040 con 0x55a715d9c790 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.982+0000 7fb2c1e6b740 10 client.17151 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.982+0000 7fb2c1e6b740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.982+0000 7fb2c1e6b740 20 client.17151 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.982+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a715dac6e0 0x55a715daeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.982+0000 7fb2c1e6b740 4 client.17151 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.982+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/4255898525 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a715d08db0 con 0x55a715dac6e0 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.982+0000 7fb2bd832640 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a715dac6e0 0x55a715daeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.982+0000 7fb29d7fa640 10 client.17151 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:30.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.982+0000 7fb2bd832640 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a715dac6e0 0x55a715daeb00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=952cad17fb7fb57f server_cookie=bd809ca3975802fb in_seq=0 out_seq=0 2024-09-11T01:07:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.983+0000 7fb29d7fa640 1 -- 172.21.15.78:0/4255898525 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a715d08db0 con 0x55a715dac6e0 2024-09-11T01:07:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.983+0000 7fb29d7fa640 10 client.17151 handle_command_reply: tid=0 2024-09-11T01:07:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.983+0000 7fb2c1e6b740 1 client.17151 shutdown 2024-09-11T01:07:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.983+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/4255898525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 msgr2=0x55a715d9a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.983+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715d9a190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00129a0 tx=0x7fb2b0012e60 comp rx=0 tx=0).stop 2024-09-11T01:07:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.983+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/4255898525 shutdown_connections 2024-09-11T01:07:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.983+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a715dac6e0 0x55a715daeb00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.983+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715d9a190 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.983+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d90800 0x55a715d99c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.983+0000 7fb29d7fa640 0 client.17151 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.983+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/4255898525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d8fee0 0x55a715d99710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.983+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/4255898525 >> 172.21.15.78:0/4255898525 conn(0x55a715d8dd10 msgr2=0x55a715d8cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.984+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/4255898525 shutdown_connections 2024-09-11T01:07:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.984+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/4255898525 wait complete. 2024-09-11T01:07:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:30.984+0000 7fb2c1e6b740 20 client.17151 trim_cache size 0 max 16384 2024-09-11T01:07:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.000+0000 7fb2c1e6b740 1 Processor -- start 2024-09-11T01:07:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2c1e6b740 1 -- start start 2024-09-11T01:07:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2c1e6b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715d9b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2c1e6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d90800 0x55a715d96920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2c1e6b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715d96e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2c1e6b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b4d50 con 0x55a715d9c790 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2c1e6b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b68f0 con 0x55a715d8fee0 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2c1e6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b5420 con 0x55a715d90800 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2be033640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715d96e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2be033640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715d96e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55236/0 (socket says 172.21.15.78:55236) 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2be033640 1 -- 172.21.15.78:0/2554582963 learned_addr learned my addr 172.21.15.78:0/2554582963 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2bd832640 1 --2- 172.21.15.78:0/2554582963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715d9b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2bd031640 1 --2- 172.21.15.78:0/2554582963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d90800 0x55a715d96920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2be033640 1 -- 172.21.15.78:0/2554582963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d90800 msgr2=0x55a715d96920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2be033640 1 --2- 172.21.15.78:0/2554582963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d90800 0x55a715d96920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2be033640 1 -- 172.21.15.78:0/2554582963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 msgr2=0x55a715d9b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2be033640 1 --2- 172.21.15.78:0/2554582963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715d9b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2be033640 1 -- 172.21.15.78:0/2554582963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a715d98670 con 0x55a715d9c790 2024-09-11T01:07:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.001+0000 7fb2be033640 1 --2- 172.21.15.78:0/2554582963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715d96e60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00051c0 tx=0x7fb2b0012510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c4369c3aaaef5b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb29f7fe640 1 -- 172.21.15.78:0/2554582963 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0004cc0 con 0x55a715d9c790 2024-09-11T01:07:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb29f7fe640 1 -- 172.21.15.78:0/2554582963 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2b0004e60 con 0x55a715d9c790 2024-09-11T01:07:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb29f7fe640 1 -- 172.21.15.78:0/2554582963 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b002f770 con 0x55a715d9c790 2024-09-11T01:07:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/2554582963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 msgr2=0x55a715d96e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/2554582963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715d96e60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00051c0 tx=0x7fb2b0012510 comp rx=0 tx=0).stop 2024-09-11T01:07:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/2554582963 shutdown_connections 2024-09-11T01:07:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/2554582963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715d96e60 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/2554582963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d90800 0x55a715d96920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/2554582963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715d9b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/2554582963 >> 172.21.15.78:0/2554582963 conn(0x55a715d8dd10 msgr2=0x55a715d9ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/2554582963 shutdown_connections 2024-09-11T01:07:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/2554582963 wait complete. 2024-09-11T01:07:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.002+0000 7fb2c1e6b740 1 -- start start 2024-09-11T01:07:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2c1e6b740 1 Processor -- start 2024-09-11T01:07:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2c1e6b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715da4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2c1e6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d90800 0x55a715da13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2c1e6b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715da18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2c1e6b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b61f0 con 0x55a715d9c790 2024-09-11T01:07:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2c1e6b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b5b20 con 0x55a715d8fee0 2024-09-11T01:07:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2c1e6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b2d50 con 0x55a715d90800 2024-09-11T01:07:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2c1e6b740 10 client.0 _ll_register_callbacks cb 0x7fb2b0009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2be033640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715da18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2be033640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715da18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55246/0 (socket says 172.21.15.78:55246) 2024-09-11T01:07:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2be033640 1 -- 172.21.15.78:0/3176047528 learned_addr learned my addr 172.21.15.78:0/3176047528 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2bd832640 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715da4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2bd031640 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d90800 0x55a715da13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2be033640 1 -- 172.21.15.78:0/3176047528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d90800 msgr2=0x55a715da13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2be033640 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d90800 0x55a715da13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2be033640 1 -- 172.21.15.78:0/3176047528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 msgr2=0x55a715da4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2be033640 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715da4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2be033640 1 -- 172.21.15.78:0/3176047528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a715da2120 con 0x55a715d9c790 2024-09-11T01:07:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.003+0000 7fb2bd832640 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715da4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:07:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb2be033640 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715da18f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0010920 tx=0x7fb2b0012960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b803923d04e04693 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb29d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:07:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb29d7fa640 1 -- 172.21.15.78:0/3176047528 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0037060 con 0x55a715d9c790 2024-09-11T01:07:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb29d7fa640 1 -- 172.21.15.78:0/3176047528 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fb2b0005740 con 0x55a715d9c790 2024-09-11T01:07:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3176047528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b0004e60 con 0x55a715d9c790 2024-09-11T01:07:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb29d7fa640 1 -- 172.21.15.78:0/3176047528 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fb2b0005940 con 0x55a715d9c790 2024-09-11T01:07:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb2c1e6b740 10 client.17157 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3176047528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55a715e1a680 con 0x55a715d9c790 2024-09-11T01:07:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb29d7fa640 1 -- 172.21.15.78:0/3176047528 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2b0004e60 con 0x55a715d9c790 2024-09-11T01:07:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb29d7fa640 1 -- 172.21.15.78:0/3176047528 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fb2b0025510 con 0x55a715d9c790 2024-09-11T01:07:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb2c1e6b740 10 client.17157 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb2c1e6b740 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:07:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb2c1e6b740 20 client.17157 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a715e1ad40 0x55a715e1d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb2c1e6b740 4 client.17157 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:07:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.004+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3176047528 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a715d04860 con 0x55a715e1ad40 2024-09-11T01:07:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.005+0000 7fb2bd832640 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a715e1ad40 0x55a715e1d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.005+0000 7fb29d7fa640 10 client.17157 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.005+0000 7fb2bd832640 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a715e1ad40 0x55a715e1d160 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768d47721067becd server_cookie=568e0daa360f922b in_seq=0 out_seq=0 2024-09-11T01:07:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb29d7fa640 1 -- 172.21.15.78:0/3176047528 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a715d04860 con 0x55a715e1ad40 2024-09-11T01:07:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb29d7fa640 10 client.17157 handle_command_reply: tid=0 2024-09-11T01:07:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb2c1e6b740 1 client.17157 shutdown 2024-09-11T01:07:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3176047528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 msgr2=0x55a715da18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715da18f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0010920 tx=0x7fb2b0012960 comp rx=0 tx=0).stop 2024-09-11T01:07:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3176047528 shutdown_connections 2024-09-11T01:07:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a715e1ad40 0x55a715e1d160 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a715d9c790 0x55a715da18f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a715d90800 0x55a715da13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb2c1e6b740 1 --2- 172.21.15.78:0/3176047528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a715d8fee0 0x55a715da4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb29d7fa640 0 client.17157 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3176047528 >> 172.21.15.78:0/3176047528 conn(0x55a715d8dd10 msgr2=0x55a715d9b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3176047528 shutdown_connections 2024-09-11T01:07:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb2c1e6b740 1 -- 172.21.15.78:0/3176047528 wait complete. 2024-09-11T01:07:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.007+0000 7fb2c1e6b740 20 client.17157 trim_cache size 0 max 16384 2024-09-11T01:07:31.018 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:07:31.018 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:07:31.018 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "c382b80f-4a8b-44bf-875d-48c566a029c2", 2024-09-11T01:07:31.018 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:07:31.018 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:07:31.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.008+0000 7fb2bfa96640 1 -- 172.21.15.78:0/3485328458 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb2a00b5450 msgr2=0x7fb2a00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:31.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.008+0000 7fb2bfa96640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb2a00b5450 0x7fb2a00b7910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac031440 tx=0x7fb2ac005a20 comp rx=0 tx=0).stop 2024-09-11T01:07:31.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.008+0000 7fb2bfa96640 1 -- 172.21.15.78:0/3485328458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b8075540 msgr2=0x7fb2b819f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:31.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.008+0000 7fb2bfa96640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b8075540 0x7fb2b819f940 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800b4a0 tx=0x7fb2a800b960 comp rx=0 tx=0).stop 2024-09-11T01:07:31.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.008+0000 7fb2bfa96640 1 -- 172.21.15.78:0/3485328458 shutdown_connections 2024-09-11T01:07:31.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.008+0000 7fb2bfa96640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb2a00b5450 0x7fb2a00b7910 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.008+0000 7fb2bfa96640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2b8113b20 0x7fb2b81a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.008+0000 7fb2bfa96640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2b8075e60 0x7fb2b819fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.008+0000 7fb2bfa96640 1 --2- 172.21.15.78:0/3485328458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b8075540 0x7fb2b819f940 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:31.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.008+0000 7fb2bfa96640 1 -- 172.21.15.78:0/3485328458 >> 172.21.15.78:0/3485328458 conn(0x7fb2b80fe830 msgr2=0x7fb2b80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:31.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.008+0000 7fb2bfa96640 1 -- 172.21.15.78:0/3485328458 shutdown_connections 2024-09-11T01:07:31.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:31.008+0000 7fb2bfa96640 1 -- 172.21.15.78:0/3485328458 wait complete. 2024-09-11T01:07:31.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 595772 5.43 MB/sec execute 339 sec latency 1875.875 ms 2024-09-11T01:07:31.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:31 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:07:31.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:31 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:31 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:07:31.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:31 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:07:31.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:31 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:31 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:07:31.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:31 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:31 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:07:31.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:31 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:31 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:31 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:07:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:31 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:31 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:07:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:31 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:07:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:31 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:31 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:07:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:31 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:31 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:07:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:31 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:31 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:31 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:07:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:31 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:31 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:07:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:31 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:07:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:31 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:31 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:07:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:31 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:31 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:07:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:31 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:31 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:31.786 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:07:32.050 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:07:32.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 597512 5.45 MB/sec execute 340 sec latency 27.854 ms 2024-09-11T01:07:32.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:07:32.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:07:32.472 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.472 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.472 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:32 smithi078 ceph-mon[31315]: pgmap v587: 97 pgs: 97 active+clean; 418 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 175 op/s 2024-09-11T01:07:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.555+0000 7f5b76d24640 1 -- 172.21.15.78:0/2003377915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b70103b30 msgr2=0x7f5b70103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.555+0000 7f5b76d24640 1 --2- 172.21.15.78:0/2003377915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b70103b30 0x7f5b70103f10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009900 tx=0x7f5b6c02a410 comp rx=0 tx=0).stop 2024-09-11T01:07:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.555+0000 7f5b76d24640 1 -- 172.21.15.78:0/2003377915 shutdown_connections 2024-09-11T01:07:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.555+0000 7f5b76d24640 1 --2- 172.21.15.78:0/2003377915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b7010d0e0 0x7f5b7010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.555+0000 7f5b76d24640 1 --2- 172.21.15.78:0/2003377915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b70104530 0x7f5b7010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.555+0000 7f5b76d24640 1 --2- 172.21.15.78:0/2003377915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b70103b30 0x7f5b70103f10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.555+0000 7f5b76d24640 1 -- 172.21.15.78:0/2003377915 >> 172.21.15.78:0/2003377915 conn(0x7f5b700fd700 msgr2=0x7f5b700ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.555+0000 7f5b76d24640 1 -- 172.21.15.78:0/2003377915 shutdown_connections 2024-09-11T01:07:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.555+0000 7f5b76d24640 1 -- 172.21.15.78:0/2003377915 wait complete. 2024-09-11T01:07:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b76d24640 1 Processor -- start 2024-09-11T01:07:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b76d24640 1 -- start start 2024-09-11T01:07:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b76d24640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b70103b30 0x7f5b701992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b76d24640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b70104530 0x7f5b701997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b76d24640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b7010d0e0 0x7f5b7019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b76d24640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70112d60 con 0x7f5b70103b30 2024-09-11T01:07:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b76d24640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70112be0 con 0x7f5b70104530 2024-09-11T01:07:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b76d24640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70112ee0 con 0x7f5b7010d0e0 2024-09-11T01:07:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b74ac0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b70103b30 0x7f5b701992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b74ac0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b70103b30 0x7f5b701992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55266/0 (socket says 172.21.15.78:55266) 2024-09-11T01:07:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b74ac0640 1 -- 172.21.15.78:0/2403291260 learned_addr learned my addr 172.21.15.78:0/2403291260 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b67fff640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b70104530 0x7f5b701997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.556+0000 7f5b752c1640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b7010d0e0 0x7f5b7019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.557+0000 7f5b74ac0640 1 -- 172.21.15.78:0/2403291260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b7010d0e0 msgr2=0x7f5b7019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.557+0000 7f5b74ac0640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b7010d0e0 0x7f5b7019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.557+0000 7f5b74ac0640 1 -- 172.21.15.78:0/2403291260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b70104530 msgr2=0x7f5b701997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.557+0000 7f5b74ac0640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b70104530 0x7f5b701997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.557+0000 7f5b74ac0640 1 -- 172.21.15.78:0/2403291260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7019f360 con 0x7f5b70103b30 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.557+0000 7f5b67fff640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b70104530 0x7f5b701997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.557+0000 7f5b74ac0640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b70103b30 0x7f5b701992b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c005940 tx=0x7f5b6c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.557+0000 7f5b65ffb640 1 -- 172.21.15.78:0/2403291260 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c03f030 con 0x7f5b70103b30 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.557+0000 7f5b76d24640 1 -- 172.21.15.78:0/2403291260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7019f5f0 con 0x7f5b70103b30 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.557+0000 7f5b65ffb640 1 -- 172.21.15.78:0/2403291260 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6c0044c0 con 0x7f5b70103b30 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.557+0000 7f5b65ffb640 1 -- 172.21.15.78:0/2403291260 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c035070 con 0x7f5b70103b30 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.557+0000 7f5b76d24640 1 -- 172.21.15.78:0/2403291260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b701a5d10 con 0x7f5b70103b30 2024-09-11T01:07:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.560+0000 7f5b790f9740 1 Processor -- start 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.560+0000 7f5b65ffb640 1 -- 172.21.15.78:0/2403291260 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5b6c03ba20 con 0x7f5b70103b30 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.560+0000 7f5b790f9740 1 -- start start 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.560+0000 7f5b790f9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 0x5600f596b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.560+0000 7f5b790f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b9c0 0x5600f597b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.560+0000 7f5b790f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f597bd60 0x5600f597e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.560+0000 7f5b790f9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5600f58e1e50 con 0x5600f596b0a0 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.560+0000 7f5b790f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5600f58d1280 con 0x5600f596b9c0 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.560+0000 7f5b790f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5600f58d5090 con 0x5600f597bd60 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.560+0000 7f5b752c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 0x5600f596b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b74ac0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b9c0 0x5600f597b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b752c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 0x5600f596b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55280/0 (socket says 172.21.15.78:55280) 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b752c1640 1 -- 172.21.15.78:0/4035575820 learned_addr learned my addr 172.21.15.78:0/4035575820 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b67fff640 1 --2- 172.21.15.78:0/4035575820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f597bd60 0x5600f597e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b752c1640 1 -- 172.21.15.78:0/4035575820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f597bd60 msgr2=0x5600f597e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b752c1640 1 --2- 172.21.15.78:0/4035575820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f597bd60 0x5600f597e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b752c1640 1 -- 172.21.15.78:0/4035575820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b9c0 msgr2=0x5600f597b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b752c1640 1 --2- 172.21.15.78:0/4035575820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b9c0 0x5600f597b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b752c1640 1 -- 172.21.15.78:0/4035575820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600f562f650 con 0x5600f596b0a0 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b65ffb640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5b580b5450 0x7f5b580b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b752c1640 1 --2- 172.21.15.78:0/4035575820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 0x5600f596b480 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5b68011440 tx=0x7f5b68011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0c1c386df1c250 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b74ac0640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5b580b5450 0x7f5b580b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b56ffd640 1 -- 172.21.15.78:0/4035575820 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68004d20 con 0x5600f596b0a0 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b56ffd640 1 -- 172.21.15.78:0/4035575820 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b68004050 con 0x5600f596b0a0 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b56ffd640 1 -- 172.21.15.78:0/4035575820 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6800f7a0 con 0x5600f596b0a0 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.561+0000 7f5b65ffb640 1 -- 172.21.15.78:0/2403291260 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f5b6c106570 con 0x7f5b70103b30 2024-09-11T01:07:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.562+0000 7f5b790f9740 1 -- 172.21.15.78:0/4035575820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 msgr2=0x5600f596b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.562+0000 7f5b74ac0640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5b580b5450 0x7f5b580b7910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c103db0 tx=0x7f5b6c10b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.562+0000 7f5b790f9740 1 --2- 172.21.15.78:0/4035575820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 0x5600f596b480 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5b68011440 tx=0x7f5b68011900 comp rx=0 tx=0).stop 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.562+0000 7f5b790f9740 1 -- 172.21.15.78:0/4035575820 shutdown_connections 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.562+0000 7f5b790f9740 1 --2- 172.21.15.78:0/4035575820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f597bd60 0x5600f597e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.562+0000 7f5b790f9740 1 --2- 172.21.15.78:0/4035575820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b9c0 0x5600f597b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.562+0000 7f5b790f9740 1 --2- 172.21.15.78:0/4035575820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 0x5600f596b480 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.562+0000 7f5b790f9740 1 -- 172.21.15.78:0/4035575820 >> 172.21.15.78:0/4035575820 conn(0x5600f5968ed0 msgr2=0x5600f596a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.562+0000 7f5b790f9740 1 -- 172.21.15.78:0/4035575820 shutdown_connections 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.562+0000 7f5b790f9740 1 -- 172.21.15.78:0/4035575820 wait complete. 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b790f9740 1 -- start start 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b790f9740 1 Processor -- start 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b790f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b0a0 0x5600f597b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b790f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f596b9c0 0x5600f5978b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b790f9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f597bd60 0x5600f5979160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b790f9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5600f5732700 con 0x5600f597bd60 2024-09-11T01:07:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b790f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5600f586c110 con 0x5600f596b0a0 2024-09-11T01:07:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b790f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5600f597ea10 con 0x5600f596b9c0 2024-09-11T01:07:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b790f9740 10 client.0 _ll_register_callbacks cb 0x5600f5885990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b752c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f597bd60 0x5600f5979160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b752c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f597bd60 0x5600f5979160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55294/0 (socket says 172.21.15.78:55294) 2024-09-11T01:07:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b752c1640 1 -- 172.21.15.78:0/4289042229 learned_addr learned my addr 172.21.15.78:0/4289042229 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b67fff640 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f596b9c0 0x5600f5978b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.563+0000 7f5b752c1640 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b0a0 0x5600f597b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.564+0000 7f5b752c1640 1 -- 172.21.15.78:0/4289042229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f596b9c0 msgr2=0x5600f5978b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.564+0000 7f5b752c1640 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f596b9c0 0x5600f5978b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.564+0000 7f5b752c1640 1 -- 172.21.15.78:0/4289042229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b0a0 msgr2=0x5600f597b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.564+0000 7f5b752c1640 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b0a0 0x5600f597b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.564+0000 7f5b752c1640 1 -- 172.21.15.78:0/4289042229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600f5899c40 con 0x5600f597bd60 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.564+0000 7f5b752c1640 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f597bd60 0x5600f5979160 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009270 tx=0x7f5b68009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79da43c0e9c51e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.564+0000 7f5b4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.564+0000 7f5b4ffff640 1 -- 172.21.15.78:0/4289042229 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6801d070 con 0x5600f597bd60 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.564+0000 7f5b4ffff640 1 -- 172.21.15.78:0/4289042229 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f5b6800f540 con 0x5600f597bd60 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.564+0000 7f5b790f9740 1 -- 172.21.15.78:0/4289042229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b68004050 con 0x5600f597bd60 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.565+0000 7f5b4ffff640 1 -- 172.21.15.78:0/4289042229 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5b6800f740 con 0x5600f597bd60 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.565+0000 7f5b790f9740 10 client.17169 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.565+0000 7f5b790f9740 1 -- 172.21.15.78:0/4289042229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5600f598a3a0 con 0x5600f597bd60 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.565+0000 7f5b4ffff640 1 -- 172.21.15.78:0/4289042229 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b68004050 con 0x5600f597bd60 2024-09-11T01:07:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.565+0000 7f5b4ffff640 1 -- 172.21.15.78:0/4289042229 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f5b68030030 con 0x5600f597bd60 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.565+0000 7f5b790f9740 10 client.17169 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.565+0000 7f5b790f9740 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.565+0000 7f5b790f9740 20 client.17169 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.565+0000 7f5b790f9740 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5600f598b750 0x5600f598db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.565+0000 7f5b790f9740 4 client.17169 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.565+0000 7f5b790f9740 1 -- 172.21.15.78:0/4289042229 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600f58e3da0 con 0x5600f598b750 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.565+0000 7f5b67fff640 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5600f598b750 0x5600f598db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.566+0000 7f5b67fff640 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5600f598b750 0x5600f598db40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c2d497991b272e3 server_cookie=c74fbb6ce9f85171 in_seq=0 out_seq=0 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.566+0000 7f5b4ffff640 10 client.17169 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b4ffff640 1 -- 172.21.15.78:0/4289042229 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5600f58e3da0 con 0x5600f598b750 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b4ffff640 10 client.17169 handle_command_reply: tid=0 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b790f9740 1 client.17169 shutdown 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b790f9740 1 -- 172.21.15.78:0/4289042229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f597bd60 msgr2=0x5600f5979160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b790f9740 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f597bd60 0x5600f5979160 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009270 tx=0x7f5b68009500 comp rx=0 tx=0).stop 2024-09-11T01:07:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b790f9740 1 -- 172.21.15.78:0/4289042229 shutdown_connections 2024-09-11T01:07:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b790f9740 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5600f598b750 0x5600f598db40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b790f9740 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f597bd60 0x5600f5979160 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b790f9740 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f596b9c0 0x5600f5978b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b790f9740 1 --2- 172.21.15.78:0/4289042229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b0a0 0x5600f597b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b4ffff640 0 client.17169 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b790f9740 1 -- 172.21.15.78:0/4289042229 >> 172.21.15.78:0/4289042229 conn(0x5600f5968ed0 msgr2=0x5600f596ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b790f9740 1 -- 172.21.15.78:0/4289042229 shutdown_connections 2024-09-11T01:07:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b790f9740 1 -- 172.21.15.78:0/4289042229 wait complete. 2024-09-11T01:07:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.567+0000 7f5b790f9740 20 client.17169 trim_cache size 0 max 16384 2024-09-11T01:07:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.586+0000 7f5b790f9740 1 Processor -- start 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.586+0000 7f5b790f9740 1 -- start start 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.586+0000 7f5b790f9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 0x5600f5987c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.586+0000 7f5b790f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b9c0 0x5600f5978010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.586+0000 7f5b790f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f597bd60 0x5600f5993320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.586+0000 7f5b790f9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ac400 con 0x5600f596b0a0 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.586+0000 7f5b790f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701adfa0 con 0x5600f596b9c0 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.586+0000 7f5b790f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701acad0 con 0x5600f597bd60 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b752c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 0x5600f5987c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b9c0 0x5600f5978010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b752c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 0x5600f5987c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55310/0 (socket says 172.21.15.78:55310) 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b752c1640 1 -- 172.21.15.78:0/562561969 learned_addr learned my addr 172.21.15.78:0/562561969 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b752c1640 1 --2- 172.21.15.78:0/562561969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f597bd60 0x5600f5993320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b67fff640 1 -- 172.21.15.78:0/562561969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f597bd60 msgr2=0x5600f5993320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b67fff640 1 --2- 172.21.15.78:0/562561969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f597bd60 0x5600f5993320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b67fff640 1 -- 172.21.15.78:0/562561969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 msgr2=0x5600f5987c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b67fff640 1 --2- 172.21.15.78:0/562561969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 0x5600f5987c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b67fff640 1 -- 172.21.15.78:0/562561969 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600f596b7f0 con 0x5600f596b9c0 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b67fff640 1 --2- 172.21.15.78:0/562561969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b9c0 0x5600f5978010 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x5600f598e260 tx=0x7f5b60005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4610fcb3723a34db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b567fc640 1 -- 172.21.15.78:0/562561969 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60005220 con 0x5600f596b9c0 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b567fc640 1 -- 172.21.15.78:0/562561969 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b60004c50 con 0x5600f596b9c0 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.587+0000 7f5b567fc640 1 -- 172.21.15.78:0/562561969 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60010a20 con 0x5600f596b9c0 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.588+0000 7f5b790f9740 1 -- 172.21.15.78:0/562561969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b9c0 msgr2=0x5600f5978010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.588+0000 7f5b790f9740 1 --2- 172.21.15.78:0/562561969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b9c0 0x5600f5978010 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x5600f598e260 tx=0x7f5b60005f90 comp rx=0 tx=0).stop 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.588+0000 7f5b790f9740 1 -- 172.21.15.78:0/562561969 shutdown_connections 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.588+0000 7f5b790f9740 1 --2- 172.21.15.78:0/562561969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f597bd60 0x5600f5993320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.588+0000 7f5b790f9740 1 --2- 172.21.15.78:0/562561969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f596b9c0 0x5600f5978010 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.588+0000 7f5b790f9740 1 --2- 172.21.15.78:0/562561969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b0a0 0x5600f5987c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.588+0000 7f5b790f9740 1 -- 172.21.15.78:0/562561969 >> 172.21.15.78:0/562561969 conn(0x5600f5968ed0 msgr2=0x5600f597a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.588+0000 7f5b790f9740 1 -- 172.21.15.78:0/562561969 shutdown_connections 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.588+0000 7f5b790f9740 1 -- 172.21.15.78:0/562561969 wait complete. 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b790f9740 1 -- start start 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b790f9740 1 Processor -- start 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b790f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f596b0a0 0x5600f5983ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b790f9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b9c0 0x5600f59841e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b790f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f597bd60 0x5600f5980c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b790f9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ad8a0 con 0x5600f596b9c0 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b790f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ad1d0 con 0x5600f597bd60 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b790f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701aa440 con 0x5600f596b0a0 2024-09-11T01:07:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b790f9740 10 client.0 _ll_register_callbacks cb 0x7f5b6c10c060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b9c0 0x5600f59841e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b752c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f596b0a0 0x5600f5983ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b9c0 0x5600f59841e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55312/0 (socket says 172.21.15.78:55312) 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b67fff640 1 -- 172.21.15.78:0/1212282759 learned_addr learned my addr 172.21.15.78:0/1212282759 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.589+0000 7f5b752c1640 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f597bd60 0x5600f5980c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b67fff640 1 -- 172.21.15.78:0/1212282759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f596b0a0 msgr2=0x5600f5983ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b67fff640 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f596b0a0 0x5600f5983ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b67fff640 1 -- 172.21.15.78:0/1212282759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f597bd60 msgr2=0x5600f5980c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b67fff640 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f597bd60 0x5600f5980c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b67fff640 1 -- 172.21.15.78:0/1212282759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600f59811b0 con 0x5600f596b9c0 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b752c1640 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f596b0a0 0x5600f5983ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b67fff640 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b9c0 0x5600f59841e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5b600120f0 tx=0x7f5b60012120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162d360c0bba1bde server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:07:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b4ffff640 1 -- 172.21.15.78:0/1212282759 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b600048d0 con 0x5600f596b9c0 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b4ffff640 1 -- 172.21.15.78:0/1212282759 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f5b60004f40 con 0x5600f596b9c0 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b790f9740 1 -- 172.21.15.78:0/1212282759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b60004c50 con 0x5600f596b9c0 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b4ffff640 1 -- 172.21.15.78:0/1212282759 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5b60004a70 con 0x5600f596b9c0 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b790f9740 10 client.17175 fetch_fsmap learned FSMap version 46 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b790f9740 1 -- 172.21.15.78:0/1212282759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5600f59f8d20 con 0x5600f596b9c0 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.590+0000 7f5b4ffff640 1 -- 172.21.15.78:0/1212282759 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b60004c50 con 0x5600f596b9c0 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.591+0000 7f5b4ffff640 1 -- 172.21.15.78:0/1212282759 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f5b60023070 con 0x5600f596b9c0 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.591+0000 7f5b790f9740 10 client.17175 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.591+0000 7f5b790f9740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.591+0000 7f5b790f9740 20 client.17175 populate_metadata read hostname 'smithi078' 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.591+0000 7f5b790f9740 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5600f59f9470 0x5600f59fb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:07:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.591+0000 7f5b790f9740 4 client.17175 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:07:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.591+0000 7f5b790f9740 1 -- 172.21.15.78:0/1212282759 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5600f58df850 con 0x5600f59f9470 2024-09-11T01:07:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.591+0000 7f5b752c1640 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5600f59f9470 0x5600f59fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:07:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.591+0000 7f5b752c1640 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5600f59f9470 0x5600f59fb890 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965721b70b72c9a7 server_cookie=999e4956e74b2144 in_seq=0 out_seq=0 2024-09-11T01:07:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.591+0000 7f5b4ffff640 10 client.17175 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:32.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b4ffff640 1 -- 172.21.15.78:0/1212282759 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x5600f58df850 con 0x5600f59f9470 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b4ffff640 10 client.17175 handle_command_reply: tid=0 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b790f9740 1 client.17175 shutdown 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b790f9740 1 -- 172.21.15.78:0/1212282759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b9c0 msgr2=0x5600f59841e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b790f9740 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b9c0 0x5600f59841e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5b600120f0 tx=0x7f5b60012120 comp rx=0 tx=0).stop 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b790f9740 1 -- 172.21.15.78:0/1212282759 shutdown_connections 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b790f9740 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600f597bd60 0x5600f5980c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b790f9740 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5600f59f9470 0x5600f59fb890 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b790f9740 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600f596b9c0 0x5600f59841e0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b790f9740 1 --2- 172.21.15.78:0/1212282759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600f596b0a0 0x5600f5983ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b790f9740 1 -- 172.21.15.78:0/1212282759 >> 172.21.15.78:0/1212282759 conn(0x5600f5968ed0 msgr2=0x5600f5969bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b4ffff640 0 client.17175 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b790f9740 1 -- 172.21.15.78:0/1212282759 shutdown_connections 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.592+0000 7f5b790f9740 1 -- 172.21.15.78:0/1212282759 wait complete. 2024-09-11T01:07:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b790f9740 20 client.17175 trim_cache size 0 max 16384 2024-09-11T01:07:32.602 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:07:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:07:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:07:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:07:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "c382b80f-4a8b-44bf-875d-48c566a029c2", 2024-09-11T01:07:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:07:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:07:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:07:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:07:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:07:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:07:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:07:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:07:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:07:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:07:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:07:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:07:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:07:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:07:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:07:32.610 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:07:32.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b76d24640 1 -- 172.21.15.78:0/2403291260 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5b580b5450 msgr2=0x7f5b580b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:32.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b76d24640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5b580b5450 0x7f5b580b7910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c103db0 tx=0x7f5b6c10b040 comp rx=0 tx=0).stop 2024-09-11T01:07:32.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b76d24640 1 -- 172.21.15.78:0/2403291260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b70103b30 msgr2=0x7f5b701992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:07:32.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b76d24640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b70103b30 0x7f5b701992b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c005940 tx=0x7f5b6c0040c0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b76d24640 1 -- 172.21.15.78:0/2403291260 shutdown_connections 2024-09-11T01:07:32.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b76d24640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5b580b5450 0x7f5b580b7910 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b76d24640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b7010d0e0 0x7f5b7019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b76d24640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b70104530 0x7f5b701997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b76d24640 1 --2- 172.21.15.78:0/2403291260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b70103b30 0x7f5b701992b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:07:32.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b76d24640 1 -- 172.21.15.78:0/2403291260 >> 172.21.15.78:0/2403291260 conn(0x7f5b700fd700 msgr2=0x7f5b700fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:07:32.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b76d24640 1 -- 172.21.15.78:0/2403291260 shutdown_connections 2024-09-11T01:07:32.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:07:32.593+0000 7f5b76d24640 1 -- 172.21.15.78:0/2403291260 wait complete. 2024-09-11T01:07:32.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:07:32.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:07:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:32 smithi110 ceph-mon[34712]: pgmap v587: 97 pgs: 97 active+clean; 418 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 175 op/s 2024-09-11T01:07:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:07:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:07:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:07:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:07:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:07:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:07:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:07:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:07:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:07:32.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:32 smithi112 ceph-mon[35384]: pgmap v587: 97 pgs: 97 active+clean; 418 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 175 op/s 2024-09-11T01:07:33.336 INFO:tasks.cephfs.filesystem:scrub status for tag:c382b80f-4a8b-44bf-875d-48c566a029c2 - {'path': '/', 'tag': 'c382b80f-4a8b-44bf-875d-48c566a029c2', 'options': 'recursive,force'} 2024-09-11T01:07:33.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 598403 5.45 MB/sec execute 341 sec latency 581.225 ms 2024-09-11T01:07:34.443 INFO:tasks.workunit.client.0.smithi078.stdout: 1 598403 5.43 MB/sec execute 342 sec latency 1581.297 ms 2024-09-11T01:07:34.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:34 smithi110 ceph-mon[34712]: pgmap v588: 97 pgs: 97 active+clean; 419 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-09-11T01:07:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:34 smithi112 ceph-mon[35384]: pgmap v588: 97 pgs: 97 active+clean; 419 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-09-11T01:07:34.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:34 smithi078 ceph-mon[31315]: pgmap v588: 97 pgs: 97 active+clean; 419 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-09-11T01:07:35.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 598403 5.41 MB/sec execute 343 sec latency 2581.390 ms 2024-09-11T01:07:36.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 598403 5.40 MB/sec execute 344 sec latency 3581.477 ms 2024-09-11T01:07:36.526 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:36 smithi112 ceph-mon[35384]: pgmap v589: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-09-11T01:07:36.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:36 smithi110 ceph-mon[34712]: pgmap v589: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-09-11T01:07:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:36 smithi078 ceph-mon[31315]: pgmap v589: 97 pgs: 97 active+clean; 421 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-09-11T01:07:37.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 600050 5.41 MB/sec execute 345 sec latency 3967.390 ms 2024-09-11T01:07:37.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:37 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:07:37.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:37 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:07:37.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:37 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:07:37.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:37 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:07:37.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:37 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:07:37.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:37 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:07:38.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 601524 5.41 MB/sec execute 346 sec latency 10.204 ms 2024-09-11T01:07:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:38 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:07:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:38 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:07:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:38 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:07:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:38 smithi110 ceph-mon[34712]: pgmap v590: 97 pgs: 97 active+clean; 422 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-09-11T01:07:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:38 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:07:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:38 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:07:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:38 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:07:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:38 smithi112 ceph-mon[35384]: pgmap v590: 97 pgs: 97 active+clean; 422 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-09-11T01:07:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:38 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:07:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:38 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:07:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:38 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:07:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:38 smithi078 ceph-mon[31315]: pgmap v590: 97 pgs: 97 active+clean; 422 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-09-11T01:07:39.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 601947 5.41 MB/sec execute 347 sec latency 750.598 ms 2024-09-11T01:07:40.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 601947 5.39 MB/sec execute 348 sec latency 1750.698 ms 2024-09-11T01:07:40.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:40 smithi110 ceph-mon[34712]: pgmap v591: 97 pgs: 97 active+clean; 422 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-09-11T01:07:40.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:40 smithi112 ceph-mon[35384]: pgmap v591: 97 pgs: 97 active+clean; 422 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-09-11T01:07:40.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:40 smithi078 ceph-mon[31315]: pgmap v591: 97 pgs: 97 active+clean; 422 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-09-11T01:07:41.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 601947 5.38 MB/sec execute 349 sec latency 2750.814 ms 2024-09-11T01:07:42.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 602656 5.37 MB/sec execute 350 sec latency 3500.463 ms 2024-09-11T01:07:43.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:42 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:07:43.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:42 smithi110 ceph-mon[34712]: pgmap v592: 97 pgs: 97 active+clean; 424 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.8 MiB/s wr, 150 op/s 2024-09-11T01:07:43.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:42 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:07:43.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:42 smithi112 ceph-mon[35384]: pgmap v592: 97 pgs: 97 active+clean; 424 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.8 MiB/s wr, 150 op/s 2024-09-11T01:07:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:42 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:07:43.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:42 smithi078 ceph-mon[31315]: pgmap v592: 97 pgs: 97 active+clean; 424 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.8 MiB/s wr, 150 op/s 2024-09-11T01:07:43.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 604360 5.38 MB/sec execute 351 sec latency 10.905 ms 2024-09-11T01:07:44.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 605495 5.38 MB/sec execute 352 sec latency 372.375 ms 2024-09-11T01:07:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:44 smithi110 ceph-mon[34712]: pgmap v593: 97 pgs: 97 active+clean; 425 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.8 MiB/s wr, 152 op/s 2024-09-11T01:07:44.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:44 smithi112 ceph-mon[35384]: pgmap v593: 97 pgs: 97 active+clean; 425 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.8 MiB/s wr, 152 op/s 2024-09-11T01:07:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:44 smithi078 ceph-mon[31315]: pgmap v593: 97 pgs: 97 active+clean; 425 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.8 MiB/s wr, 152 op/s 2024-09-11T01:07:45.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 605495 5.37 MB/sec execute 353 sec latency 1372.405 ms 2024-09-11T01:07:46.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 605495 5.35 MB/sec execute 354 sec latency 2372.480 ms 2024-09-11T01:07:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:46 smithi110 ceph-mon[34712]: pgmap v594: 97 pgs: 97 active+clean; 428 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-11T01:07:46.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:46 smithi112 ceph-mon[35384]: pgmap v594: 97 pgs: 97 active+clean; 428 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-11T01:07:46.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:46 smithi078 ceph-mon[31315]: pgmap v594: 97 pgs: 97 active+clean; 428 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-11T01:07:47.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 606200 5.34 MB/sec execute 355 sec latency 3122.959 ms 2024-09-11T01:07:48.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 607898 5.36 MB/sec execute 356 sec latency 10.466 ms 2024-09-11T01:07:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:48 smithi110 ceph-mon[34712]: pgmap v595: 97 pgs: 97 active+clean; 428 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-09-11T01:07:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:48 smithi112 ceph-mon[35384]: pgmap v595: 97 pgs: 97 active+clean; 428 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-09-11T01:07:48.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:48 smithi078 ceph-mon[31315]: pgmap v595: 97 pgs: 97 active+clean; 428 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-09-11T01:07:49.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 609048 5.36 MB/sec execute 357 sec latency 376.923 ms 2024-09-11T01:07:50.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 611262 5.39 MB/sec execute 358 sec latency 17.112 ms 2024-09-11T01:07:50.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:50 smithi110 ceph-mon[34712]: pgmap v596: 97 pgs: 97 active+clean; 431 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.9 MiB/s wr, 158 op/s 2024-09-11T01:07:50.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:50 smithi112 ceph-mon[35384]: pgmap v596: 97 pgs: 97 active+clean; 431 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.9 MiB/s wr, 158 op/s 2024-09-11T01:07:50.764 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:50 smithi078 ceph-mon[31315]: pgmap v596: 97 pgs: 97 active+clean; 431 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.9 MiB/s wr, 158 op/s 2024-09-11T01:07:51.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 612586 5.39 MB/sec execute 359 sec latency 218.802 ms 2024-09-11T01:07:52.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 613295 5.38 MB/sec execute 360 sec latency 967.156 ms 2024-09-11T01:07:52.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:52 smithi110 ceph-mon[34712]: pgmap v597: 97 pgs: 97 active+clean; 431 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.8 MiB/s wr, 140 op/s 2024-09-11T01:07:52.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:52 smithi112 ceph-mon[35384]: pgmap v597: 97 pgs: 97 active+clean; 431 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.8 MiB/s wr, 140 op/s 2024-09-11T01:07:52.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:52 smithi078 ceph-mon[31315]: pgmap v597: 97 pgs: 97 active+clean; 431 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.8 MiB/s wr, 140 op/s 2024-09-11T01:07:53.444 INFO:tasks.workunit.client.0.smithi078.stdout: 1 615056 5.39 MB/sec execute 361 sec latency 12.013 ms 2024-09-11T01:07:54.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 616129 5.39 MB/sec execute 362 sec latency 457.135 ms 2024-09-11T01:07:54.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:54 smithi110 ceph-mon[34712]: pgmap v598: 97 pgs: 97 active+clean; 433 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-09-11T01:07:54.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:54 smithi112 ceph-mon[35384]: pgmap v598: 97 pgs: 97 active+clean; 433 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-09-11T01:07:54.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:54 smithi078 ceph-mon[31315]: pgmap v598: 97 pgs: 97 active+clean; 433 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-09-11T01:07:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 616129 5.38 MB/sec execute 363 sec latency 1457.175 ms 2024-09-11T01:07:56.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 616129 5.36 MB/sec execute 364 sec latency 2457.278 ms 2024-09-11T01:07:56.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:56 smithi078 ceph-mon[31315]: pgmap v599: 97 pgs: 97 active+clean; 437 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-09-11T01:07:56.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:56 smithi110 ceph-mon[34712]: pgmap v599: 97 pgs: 97 active+clean; 437 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-09-11T01:07:56.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:56 smithi112 ceph-mon[35384]: pgmap v599: 97 pgs: 97 active+clean; 437 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-09-11T01:07:57.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 617644 5.37 MB/sec execute 365 sec latency 2841.615 ms 2024-09-11T01:07:58.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 619267 5.38 MB/sec execute 366 sec latency 11.451 ms 2024-09-11T01:07:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:07:58 smithi110 ceph-mon[34712]: pgmap v600: 97 pgs: 97 active+clean; 441 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-09-11T01:07:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:07:58 smithi112 ceph-mon[35384]: pgmap v600: 97 pgs: 97 active+clean; 441 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-09-11T01:07:58.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:07:58 smithi078 ceph-mon[31315]: pgmap v600: 97 pgs: 97 active+clean; 441 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-09-11T01:07:59.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 619675 5.37 MB/sec execute 367 sec latency 796.020 ms 2024-09-11T01:08:00.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 619675 5.36 MB/sec execute 368 sec latency 1796.118 ms 2024-09-11T01:08:00.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:00 smithi110 ceph-mon[34712]: pgmap v601: 97 pgs: 97 active+clean; 442 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-09-11T01:08:00.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:00 smithi112 ceph-mon[35384]: pgmap v601: 97 pgs: 97 active+clean; 442 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-09-11T01:08:00.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:00 smithi078 ceph-mon[31315]: pgmap v601: 97 pgs: 97 active+clean; 442 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-09-11T01:08:01.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 619675 5.34 MB/sec execute 369 sec latency 2796.216 ms 2024-09-11T01:08:02.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 620254 5.33 MB/sec execute 370 sec latency 3547.011 ms 2024-09-11T01:08:02.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:02 smithi110 ceph-mon[34712]: pgmap v602: 97 pgs: 97 active+clean; 444 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.8 MiB/s wr, 152 op/s 2024-09-11T01:08:02.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:02 smithi112 ceph-mon[35384]: pgmap v602: 97 pgs: 97 active+clean; 444 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.8 MiB/s wr, 152 op/s 2024-09-11T01:08:02.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:02 smithi078 ceph-mon[31315]: pgmap v602: 97 pgs: 97 active+clean; 444 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.8 MiB/s wr, 152 op/s 2024-09-11T01:08:03.338 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:08:03.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 621776 5.35 MB/sec execute 371 sec latency 23.883 ms 2024-09-11T01:08:03.599 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:04.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.219+0000 7f2defd9f640 1 -- 172.21.15.78:0/3057942626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de8109170 msgr2=0x7f2de810f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:04.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.219+0000 7f2defd9f640 1 --2- 172.21.15.78:0/3057942626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de8109170 0x7f2de810f970 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009ca0 tx=0x7f2de002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.219+0000 7f2defd9f640 1 -- 172.21.15.78:0/3057942626 shutdown_connections 2024-09-11T01:08:04.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.219+0000 7f2defd9f640 1 --2- 172.21.15.78:0/3057942626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de8109170 0x7f2de810f970 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.219+0000 7f2defd9f640 1 --2- 172.21.15.78:0/3057942626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2de81087f0 0x7f2de8108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.219+0000 7f2defd9f640 1 --2- 172.21.15.78:0/3057942626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2de8102de0 0x7f2de81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.219+0000 7f2defd9f640 1 -- 172.21.15.78:0/3057942626 >> 172.21.15.78:0/3057942626 conn(0x7f2de80fe830 msgr2=0x7f2de8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.219+0000 7f2defd9f640 1 -- 172.21.15.78:0/3057942626 shutdown_connections 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.219+0000 7f2defd9f640 1 -- 172.21.15.78:0/3057942626 wait complete. 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2defd9f640 1 Processor -- start 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2defd9f640 1 -- start start 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2defd9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2de8102de0 0x7f2de8199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2defd9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de81087f0 0x7f2de8199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2defd9f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2de8109170 0x7f2de819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2defd9f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2de8112df0 con 0x7f2de81087f0 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2defd9f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2de8112c70 con 0x7f2de8109170 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2defd9f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2de8112f70 con 0x7f2de8102de0 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2ded33a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de81087f0 0x7f2de8199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2ded33a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de81087f0 0x7f2de8199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54048/0 (socket says 172.21.15.78:54048) 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2ded33a640 1 -- 172.21.15.78:0/3519777372 learned_addr learned my addr 172.21.15.78:0/3519777372 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2dee33c640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2de8109170 0x7f2de819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2de8102de0 0x7f2de8199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2ded33a640 1 -- 172.21.15.78:0/3519777372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2de8102de0 msgr2=0x7f2de8199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2ded33a640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2de8102de0 0x7f2de8199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2ded33a640 1 -- 172.21.15.78:0/3519777372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2de8109170 msgr2=0x7f2de819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2ded33a640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2de8109170 0x7f2de819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2ded33a640 1 -- 172.21.15.78:0/3519777372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de819f2d0 con 0x7f2de81087f0 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.220+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2de8102de0 0x7f2de8199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.221+0000 7f2ded33a640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de81087f0 0x7f2de8199760 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800e9f0 tx=0x7f2dd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.221+0000 7f2ddeffd640 1 -- 172.21.15.78:0/3519777372 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd800cd80 con 0x7f2de81087f0 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.221+0000 7f2defd9f640 1 -- 172.21.15.78:0/3519777372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de819f5c0 con 0x7f2de81087f0 2024-09-11T01:08:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.221+0000 7f2ddeffd640 1 -- 172.21.15.78:0/3519777372 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dd80108e0 con 0x7f2de81087f0 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.221+0000 7f2ddeffd640 1 -- 172.21.15.78:0/3519777372 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8018980 con 0x7f2de81087f0 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.221+0000 7f2defd9f640 1 -- 172.21.15.78:0/3519777372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de81a5e50 con 0x7f2de81087f0 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.223+0000 7f2ddeffd640 1 -- 172.21.15.78:0/3519777372 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2dd8010430 con 0x7f2de81087f0 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.223+0000 7f2df2174740 1 Processor -- start 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2df2174740 1 -- start start 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2df2174740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bcc480 0x564037bcc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2df2174740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 0x564037bd87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2df2174740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bd8d30 0x564037bdb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2df2174740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564037b42e50 con 0x564037bd8d30 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2df2174740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564037b32280 con 0x564037bccda0 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2df2174740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564037b36090 con 0x564037bcc480 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2dedb3b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 0x564037bd87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2dedb3b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 0x564037bd87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55894/0 (socket says 172.21.15.78:55894) 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2dedb3b640 1 -- 172.21.15.78:0/3697632432 learned_addr learned my addr 172.21.15.78:0/3697632432 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2ded33a640 1 --2- 172.21.15.78:0/3697632432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bd8d30 0x564037bdb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2dee33c640 1 --2- 172.21.15.78:0/3697632432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bcc480 0x564037bcc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2dedb3b640 1 -- 172.21.15.78:0/3697632432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bcc480 msgr2=0x564037bcc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/3697632432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bcc480 0x564037bcc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2dedb3b640 1 -- 172.21.15.78:0/3697632432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bd8d30 msgr2=0x564037bdb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/3697632432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bd8d30 0x564037bdb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2dedb3b640 1 -- 172.21.15.78:0/3697632432 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564037890650 con 0x564037bccda0 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2ddeffd640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2dd40b54d0 0x7f2dd40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2ded33a640 1 --2- 172.21.15.78:0/3697632432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bd8d30 0x564037bdb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2dee33c640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2dd40b54d0 0x7f2dd40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2ddeffd640 1 -- 172.21.15.78:0/3519777372 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f2dd80e6200 con 0x7f2de81087f0 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.224+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/3697632432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 0x564037bd87f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2de40115f0 tx=0x7f2de4011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6abdeda20bbb7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.225+0000 7f2dd3fff640 1 -- 172.21.15.78:0/3697632432 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de401e070 con 0x564037bccda0 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.225+0000 7f2dd3fff640 1 -- 172.21.15.78:0/3697632432 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de400faf0 con 0x564037bccda0 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.225+0000 7f2dd3fff640 1 -- 172.21.15.78:0/3697632432 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4019bc0 con 0x564037bccda0 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.226+0000 7f2df2174740 1 -- 172.21.15.78:0/3697632432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 msgr2=0x564037bd87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.226+0000 7f2dee33c640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2dd40b54d0 0x7f2dd40b7990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006010 tx=0x7f2de00099a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.226+0000 7f2df2174740 1 --2- 172.21.15.78:0/3697632432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 0x564037bd87f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2de40115f0 tx=0x7f2de4011ab0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.226+0000 7f2df2174740 1 -- 172.21.15.78:0/3697632432 shutdown_connections 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.226+0000 7f2df2174740 1 --2- 172.21.15.78:0/3697632432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bd8d30 0x564037bdb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.226+0000 7f2df2174740 1 --2- 172.21.15.78:0/3697632432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 0x564037bd87f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.226+0000 7f2df2174740 1 --2- 172.21.15.78:0/3697632432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bcc480 0x564037bcc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.226+0000 7f2df2174740 1 -- 172.21.15.78:0/3697632432 >> 172.21.15.78:0/3697632432 conn(0x564037bca2b0 msgr2=0x564037bcb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.226+0000 7f2df2174740 1 -- 172.21.15.78:0/3697632432 shutdown_connections 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.226+0000 7f2df2174740 1 -- 172.21.15.78:0/3697632432 wait complete. 2024-09-11T01:08:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.226+0000 7f2df2174740 1 -- start start 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.226+0000 7f2df2174740 1 Processor -- start 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2df2174740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 0x564037bd85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2df2174740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 0x564037bd5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2df2174740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 0x564037bd5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2df2174740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564037993700 con 0x564037bccda0 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2df2174740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564037acd110 con 0x564037bcc480 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2df2174740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564037bdba50 con 0x564037bd8d30 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2df2174740 10 client.0 _ll_register_callbacks cb 0x564037ae6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2dedb3b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 0x564037bd85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2ded33a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 0x564037bd5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2dedb3b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 0x564037bd85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55908/0 (socket says 172.21.15.78:55908) 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2dee33c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 0x564037bd5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2dedb3b640 1 -- 172.21.15.78:0/1875501252 learned_addr learned my addr 172.21.15.78:0/1875501252 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2ded33a640 1 -- 172.21.15.78:0/1875501252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 msgr2=0x564037bd5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2ded33a640 1 --2- 172.21.15.78:0/1875501252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 0x564037bd5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2ded33a640 1 -- 172.21.15.78:0/1875501252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 msgr2=0x564037bd85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2ded33a640 1 --2- 172.21.15.78:0/1875501252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 0x564037bd85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2ded33a640 1 -- 172.21.15.78:0/1875501252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564037afac40 con 0x564037bccda0 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/1875501252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 0x564037bd85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2ded33a640 1 --2- 172.21.15.78:0/1875501252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 0x564037bd5a20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80e3990 tx=0x7f2dd80e9b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902375b84f7fd03a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.227+0000 7f2dd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2dd17fa640 1 -- 172.21.15.78:0/1875501252 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80ac0c0 con 0x564037bccda0 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2dd17fa640 1 -- 172.21.15.78:0/1875501252 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f2dd80b59c0 con 0x564037bccda0 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2df2174740 1 -- 172.21.15.78:0/1875501252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de400faf0 con 0x564037bccda0 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2dd17fa640 1 -- 172.21.15.78:0/1875501252 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2dd80abee0 con 0x564037bccda0 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2df2174740 10 client.17187 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2df2174740 1 -- 172.21.15.78:0/1875501252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564037be73d0 con 0x564037bccda0 2024-09-11T01:08:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2dd17fa640 1 -- 172.21.15.78:0/1875501252 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de400faf0 con 0x564037bccda0 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2dd17fa640 1 -- 172.21.15.78:0/1875501252 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f2dd8018b20 con 0x564037bccda0 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2df2174740 10 client.17187 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2df2174740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2df2174740 20 client.17187 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2df2174740 1 --2- 172.21.15.78:0/1875501252 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564037be8890 0x564037beacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2df2174740 4 client.17187 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2df2174740 1 -- 172.21.15.78:0/1875501252 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564037b44da0 con 0x564037be8890 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.228+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/1875501252 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564037be8890 0x564037beacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.229+0000 7f2dd17fa640 10 client.17187 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.229+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/1875501252 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564037be8890 0x564037beacc0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6fcf644772799e7 server_cookie=2439a277f6cd0725 in_seq=0 out_seq=0 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.229+0000 7f2dd17fa640 1 -- 172.21.15.78:0/1875501252 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564037b44da0 con 0x564037be8890 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.229+0000 7f2dd17fa640 10 client.17187 handle_command_reply: tid=0 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.229+0000 7f2df2174740 1 client.17187 shutdown 2024-09-11T01:08:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.230+0000 7f2df2174740 1 -- 172.21.15.78:0/1875501252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 msgr2=0x564037bd5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.230+0000 7f2df2174740 1 --2- 172.21.15.78:0/1875501252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 0x564037bd5a20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80e3990 tx=0x7f2dd80e9b30 comp rx=0 tx=0).stop 2024-09-11T01:08:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.230+0000 7f2df2174740 1 -- 172.21.15.78:0/1875501252 shutdown_connections 2024-09-11T01:08:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.230+0000 7f2df2174740 1 --2- 172.21.15.78:0/1875501252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 0x564037bd5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.230+0000 7f2df2174740 1 --2- 172.21.15.78:0/1875501252 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564037be8890 0x564037beacc0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.230+0000 7f2df2174740 1 --2- 172.21.15.78:0/1875501252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 0x564037bd5a20 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.230+0000 7f2df2174740 1 --2- 172.21.15.78:0/1875501252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 0x564037bd85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.230+0000 7f2dd17fa640 0 client.17187 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.230+0000 7f2df2174740 1 -- 172.21.15.78:0/1875501252 >> 172.21.15.78:0/1875501252 conn(0x564037bca2b0 msgr2=0x564037bcfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.230+0000 7f2df2174740 1 -- 172.21.15.78:0/1875501252 shutdown_connections 2024-09-11T01:08:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.230+0000 7f2df2174740 1 -- 172.21.15.78:0/1875501252 wait complete. 2024-09-11T01:08:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.230+0000 7f2df2174740 20 client.17187 trim_cache size 0 max 16384 2024-09-11T01:08:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.251+0000 7f2df2174740 1 Processor -- start 2024-09-11T01:08:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.251+0000 7f2df2174740 1 -- start start 2024-09-11T01:08:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2df2174740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bcc480 0x564037be4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2df2174740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 0x564037bea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2df2174740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 0x564037bcb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2df2174740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81ae5b0 con 0x564037bcc480 2024-09-11T01:08:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2df2174740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81b0120 con 0x564037bccda0 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2df2174740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81aec80 con 0x564037bd8d30 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2dedb3b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bcc480 0x564037be4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2ded33a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 0x564037bea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2dedb3b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bcc480 0x564037be4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54082/0 (socket says 172.21.15.78:54082) 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2dedb3b640 1 -- 172.21.15.78:0/4282512554 learned_addr learned my addr 172.21.15.78:0/4282512554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2dee33c640 1 --2- 172.21.15.78:0/4282512554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 0x564037bcb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2ded33a640 1 -- 172.21.15.78:0/4282512554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 msgr2=0x564037bcb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2ded33a640 1 --2- 172.21.15.78:0/4282512554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 0x564037bcb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2ded33a640 1 -- 172.21.15.78:0/4282512554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bcc480 msgr2=0x564037be4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2ded33a640 1 --2- 172.21.15.78:0/4282512554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bcc480 0x564037be4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2ded33a640 1 -- 172.21.15.78:0/4282512554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564037bd07a0 con 0x564037bccda0 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/4282512554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bcc480 0x564037be4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.252+0000 7f2ded33a640 1 --2- 172.21.15.78:0/4282512554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 0x564037bea8a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c6a0 tx=0x7f2dd80e9a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d582a5319255664 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.253+0000 7f2dd37fe640 1 -- 172.21.15.78:0/4282512554 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80ac0c0 con 0x564037bccda0 2024-09-11T01:08:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.253+0000 7f2dd37fe640 1 -- 172.21.15.78:0/4282512554 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dd80ac280 con 0x564037bccda0 2024-09-11T01:08:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.253+0000 7f2dd37fe640 1 -- 172.21.15.78:0/4282512554 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80b5c50 con 0x564037bccda0 2024-09-11T01:08:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.253+0000 7f2df2174740 1 -- 172.21.15.78:0/4282512554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 msgr2=0x564037bea8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.253+0000 7f2df2174740 1 --2- 172.21.15.78:0/4282512554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 0x564037bea8a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c6a0 tx=0x7f2dd80e9a40 comp rx=0 tx=0).stop 2024-09-11T01:08:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.254+0000 7f2df2174740 1 -- 172.21.15.78:0/4282512554 shutdown_connections 2024-09-11T01:08:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.254+0000 7f2df2174740 1 --2- 172.21.15.78:0/4282512554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 0x564037bcb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.254+0000 7f2df2174740 1 --2- 172.21.15.78:0/4282512554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bccda0 0x564037bea8a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.254+0000 7f2df2174740 1 --2- 172.21.15.78:0/4282512554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bcc480 0x564037be4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.254+0000 7f2df2174740 1 -- 172.21.15.78:0/4282512554 >> 172.21.15.78:0/4282512554 conn(0x564037bca2b0 msgr2=0x564037bd42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.254+0000 7f2df2174740 1 -- 172.21.15.78:0/4282512554 shutdown_connections 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.254+0000 7f2df2174740 1 -- 172.21.15.78:0/4282512554 wait complete. 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.254+0000 7f2df2174740 1 -- start start 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.254+0000 7f2df2174740 1 Processor -- start 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2df2174740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 0x564037be0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2df2174740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 0x564037be12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2df2174740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 0x564037bdbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2df2174740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81afa20 con 0x564037bccda0 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2df2174740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81af350 con 0x564037bcc480 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2df2174740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81ac5b0 con 0x564037bd8d30 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2df2174740 10 client.0 _ll_register_callbacks cb 0x7f2dd8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2ded33a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 0x564037be12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2ded33a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 0x564037be12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54098/0 (socket says 172.21.15.78:54098) 2024-09-11T01:08:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2ded33a640 1 -- 172.21.15.78:0/267580883 learned_addr learned my addr 172.21.15.78:0/267580883 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:04.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 0x564037be0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2dee33c640 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 0x564037bdbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2dedb3b640 1 -- 172.21.15.78:0/267580883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 msgr2=0x564037bdbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:04.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 0x564037bdbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2dedb3b640 1 -- 172.21.15.78:0/267580883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 msgr2=0x564037be12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:04.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 0x564037be12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2dedb3b640 1 -- 172.21.15.78:0/267580883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564037bdc2c0 con 0x564037bcc480 2024-09-11T01:08:04.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2ded33a640 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 0x564037be12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:04.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2dedb3b640 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 0x564037be0d20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2de40058c0 tx=0x7f2de4009420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28b587c728f4c194 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:04.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.255+0000 7f2dd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:08:04.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.257+0000 7f2dd17fa640 1 -- 172.21.15.78:0/267580883 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de401e040 con 0x564037bcc480 2024-09-11T01:08:04.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.257+0000 7f2dd17fa640 1 -- 172.21.15.78:0/267580883 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f2de400f640 con 0x564037bcc480 2024-09-11T01:08:04.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.257+0000 7f2df2174740 1 -- 172.21.15.78:0/267580883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dd80ac280 con 0x564037bcc480 2024-09-11T01:08:04.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.260+0000 7f2dd17fa640 1 -- 172.21.15.78:0/267580883 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2de400f860 con 0x564037bcc480 2024-09-11T01:08:04.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.260+0000 7f2df2174740 10 client.26444 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:04.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.260+0000 7f2df2174740 1 -- 172.21.15.78:0/267580883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564037c5a470 con 0x564037bcc480 2024-09-11T01:08:04.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.260+0000 7f2dd17fa640 1 -- 172.21.15.78:0/267580883 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de40055c0 con 0x564037bcc480 2024-09-11T01:08:04.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.260+0000 7f2dd17fa640 1 -- 172.21.15.78:0/267580883 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f2de40199d0 con 0x564037bcc480 2024-09-11T01:08:04.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.260+0000 7f2df2174740 10 client.26444 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:04.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.260+0000 7f2df2174740 10 client.26444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:08:04.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.260+0000 7f2df2174740 20 client.26444 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:04.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.260+0000 7f2df2174740 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564037c5a900 0x564037c5ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:04.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.260+0000 7f2df2174740 4 client.26444 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:08:04.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.260+0000 7f2df2174740 1 -- 172.21.15.78:0/267580883 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f2dd80ac280 con 0x564037c5a900 2024-09-11T01:08:04.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.260+0000 7f2ded33a640 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564037c5a900 0x564037c5ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:04.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2dd17fa640 10 client.26444 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:04.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2ded33a640 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564037c5a900 0x564037c5ccf0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=201faef3ffb012c8 server_cookie=4de239bac6f075e5 in_seq=0 out_seq=0 2024-09-11T01:08:04.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2dd17fa640 1 -- 172.21.15.78:0/267580883 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x7f2dd80ac280 con 0x564037c5a900 2024-09-11T01:08:04.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2dd17fa640 10 client.26444 handle_command_reply: tid=0 2024-09-11T01:08:04.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2df2174740 1 client.26444 shutdown 2024-09-11T01:08:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2df2174740 1 -- 172.21.15.78:0/267580883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 msgr2=0x564037be0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2df2174740 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 0x564037be0d20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2de40058c0 tx=0x7f2de4009420 comp rx=0 tx=0).stop 2024-09-11T01:08:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2df2174740 1 -- 172.21.15.78:0/267580883 shutdown_connections 2024-09-11T01:08:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2df2174740 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564037bd8d30 0x564037bdbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2df2174740 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564037c5a900 0x564037c5ccf0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2df2174740 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564037bccda0 0x564037be12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2df2174740 1 --2- 172.21.15.78:0/267580883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564037bcc480 0x564037be0d20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2df2174740 1 -- 172.21.15.78:0/267580883 >> 172.21.15.78:0/267580883 conn(0x564037bca2b0 msgr2=0x564037bcb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.261+0000 7f2dd17fa640 0 client.26444 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.262+0000 7f2df2174740 1 -- 172.21.15.78:0/267580883 shutdown_connections 2024-09-11T01:08:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.262+0000 7f2df2174740 1 -- 172.21.15.78:0/267580883 wait complete. 2024-09-11T01:08:04.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.262+0000 7f2df2174740 20 client.26444 trim_cache size 0 max 16384 2024-09-11T01:08:04.267 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:08:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:08:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:08:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:08:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:08:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:08:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:08:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:08:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:08:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:08:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:08:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:08:04.274 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:08:04.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.262+0000 7f2defd9f640 1 -- 172.21.15.78:0/3519777372 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2dd40b54d0 msgr2=0x7f2dd40b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:04.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.262+0000 7f2defd9f640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2dd40b54d0 0x7f2dd40b7990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006010 tx=0x7f2de00099a0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.262+0000 7f2defd9f640 1 -- 172.21.15.78:0/3519777372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de81087f0 msgr2=0x7f2de8199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:04.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.262+0000 7f2defd9f640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de81087f0 0x7f2de8199760 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800e9f0 tx=0x7f2dd800eeb0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.263+0000 7f2defd9f640 1 -- 172.21.15.78:0/3519777372 shutdown_connections 2024-09-11T01:08:04.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.263+0000 7f2defd9f640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2dd40b54d0 0x7f2dd40b7990 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.263+0000 7f2defd9f640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2de8109170 0x7f2de819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.263+0000 7f2defd9f640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de81087f0 0x7f2de8199760 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.263+0000 7f2defd9f640 1 --2- 172.21.15.78:0/3519777372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2de8102de0 0x7f2de8199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:04.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.263+0000 7f2defd9f640 1 -- 172.21.15.78:0/3519777372 >> 172.21.15.78:0/3519777372 conn(0x7f2de80fe830 msgr2=0x7f2de80ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:04.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.263+0000 7f2defd9f640 1 -- 172.21.15.78:0/3519777372 shutdown_connections 2024-09-11T01:08:04.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:04.263+0000 7f2defd9f640 1 -- 172.21.15.78:0/3519777372 wait complete. 2024-09-11T01:08:04.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 623176 5.35 MB/sec execute 372 sec latency 47.862 ms 2024-09-11T01:08:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:04 smithi078 ceph-mon[31315]: pgmap v603: 97 pgs: 97 active+clean; 446 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-09-11T01:08:04.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:04 smithi110 ceph-mon[34712]: pgmap v603: 97 pgs: 97 active+clean; 446 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-09-11T01:08:04.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:04 smithi112 ceph-mon[35384]: pgmap v603: 97 pgs: 97 active+clean; 446 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-09-11T01:08:05.092 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:08:05.093 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:08:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:05.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 623219 5.34 MB/sec execute 373 sec latency 967.684 ms 2024-09-11T01:08:05.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.870+0000 7f4f90998640 1 -- 172.21.15.78:0/1719418416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8c108910 msgr2=0x7f4f8c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:05.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.870+0000 7f4f90998640 1 --2- 172.21.15.78:0/1719418416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8c108910 0x7f4f8c108d50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009930 tx=0x7f4f7402a420 comp rx=0 tx=0).stop 2024-09-11T01:08:05.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.871+0000 7f4f90998640 1 -- 172.21.15.78:0/1719418416 shutdown_connections 2024-09-11T01:08:05.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.871+0000 7f4f90998640 1 --2- 172.21.15.78:0/1719418416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f8c109290 0x7f4f8c10f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:05.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.871+0000 7f4f90998640 1 --2- 172.21.15.78:0/1719418416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8c108910 0x7f4f8c108d50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:05.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.871+0000 7f4f90998640 1 --2- 172.21.15.78:0/1719418416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8c101810 0x7f4f8c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:05.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.871+0000 7f4f90998640 1 -- 172.21.15.78:0/1719418416 >> 172.21.15.78:0/1719418416 conn(0x7f4f8c076f20 msgr2=0x7f4f8c077330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:05.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.871+0000 7f4f90998640 1 -- 172.21.15.78:0/1719418416 shutdown_connections 2024-09-11T01:08:05.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.871+0000 7f4f90998640 1 -- 172.21.15.78:0/1719418416 wait complete. 2024-09-11T01:08:05.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.871+0000 7f4f90998640 1 Processor -- start 2024-09-11T01:08:05.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.871+0000 7f4f90998640 1 -- start start 2024-09-11T01:08:05.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f90998640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8c101810 0x7f4f8c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:05.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f90998640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f8c108910 0x7f4f8c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:05.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f90998640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8c109290 0x7f4f8c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:05.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f90998640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c112df0 con 0x7f4f8c101810 2024-09-11T01:08:05.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f90998640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c112c70 con 0x7f4f8c108910 2024-09-11T01:08:05.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f90998640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c112f70 con 0x7f4f8c109290 2024-09-11T01:08:05.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f8a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8c101810 0x7f4f8c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:05.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f8a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8c101810 0x7f4f8c199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54116/0 (socket says 172.21.15.78:54116) 2024-09-11T01:08:05.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f8a59c640 1 -- 172.21.15.78:0/3503145331 learned_addr learned my addr 172.21.15.78:0/3503145331 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:05.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f89d9b640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f8c108910 0x7f4f8c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:05.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f8a59c640 1 -- 172.21.15.78:0/3503145331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8c109290 msgr2=0x7f4f8c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:05.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f8a59c640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8c109290 0x7f4f8c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f8a59c640 1 -- 172.21.15.78:0/3503145331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f8c108910 msgr2=0x7f4f8c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f8a59c640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f8c108910 0x7f4f8c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f8a59c640 1 -- 172.21.15.78:0/3503145331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c19f4b0 con 0x7f4f8c101810 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f89d9b640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f8c108910 0x7f4f8c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f8a59c640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8c101810 0x7f4f8c199400 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00ba50 tx=0x7f4f7c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f837fe640 1 -- 172.21.15.78:0/3503145331 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c019070 con 0x7f4f8c101810 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f90998640 1 -- 172.21.15.78:0/3503145331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8c19f7a0 con 0x7f4f8c101810 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f837fe640 1 -- 172.21.15.78:0/3503145331 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f7c00cbe0 con 0x7f4f8c101810 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f837fe640 1 -- 172.21.15.78:0/3503145331 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c0075d0 con 0x7f4f8c101810 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.872+0000 7f4f90998640 1 -- 172.21.15.78:0/3503145331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8c1a6030 con 0x7f4f8c101810 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.874+0000 7f4f90998640 1 -- 172.21.15.78:0/3503145331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f8c06ae30 con 0x7f4f8c101810 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.876+0000 7f4f837fe640 1 -- 172.21.15.78:0/3503145331 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4f7c007830 con 0x7f4f8c101810 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.877+0000 7f4f837fe640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4f780b5450 0x7f4f780b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.877+0000 7f4f89d9b640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4f780b5450 0x7f4f780b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.877+0000 7f4f837fe640 1 -- 172.21.15.78:0/3503145331 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f4f7c0e5ce0 con 0x7f4f8c101810 2024-09-11T01:08:05.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.877+0000 7f4f89d9b640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4f780b5450 0x7f4f780b7910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f74000c00 tx=0x7f4f74031d10 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:05.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:05.879+0000 7f4f837fe640 1 -- 172.21.15.78:0/3503145331 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4f7c0ac8a0 con 0x7f4f8c101810 2024-09-11T01:08:06.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.088+0000 7f4f90998640 1 -- 172.21.15.78:0/3503145331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f8c101bf0 con 0x7f4f8c101810 2024-09-11T01:08:06.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.088+0000 7f4f837fe640 1 -- 172.21.15.78:0/3503145331 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11980 (secure 0 0 0) 0x7f4f8c101bf0 con 0x7f4f8c101810 2024-09-11T01:08:06.092 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:08:06.092 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":47,"btime":"2024-09-11T01:07:41:867729+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:07:41.817504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T01:08:06.092 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 47 2024-09-11T01:08:06.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.091+0000 7f4f90998640 1 -- 172.21.15.78:0/3503145331 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4f780b5450 msgr2=0x7f4f780b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:06.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.091+0000 7f4f90998640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4f780b5450 0x7f4f780b7910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f74000c00 tx=0x7f4f74031d10 comp rx=0 tx=0).stop 2024-09-11T01:08:06.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.091+0000 7f4f90998640 1 -- 172.21.15.78:0/3503145331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8c101810 msgr2=0x7f4f8c199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:06.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.091+0000 7f4f90998640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8c101810 0x7f4f8c199400 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00ba50 tx=0x7f4f7c00bf10 comp rx=0 tx=0).stop 2024-09-11T01:08:06.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.091+0000 7f4f90998640 1 -- 172.21.15.78:0/3503145331 shutdown_connections 2024-09-11T01:08:06.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.091+0000 7f4f90998640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4f780b5450 0x7f4f780b7910 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:06.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.091+0000 7f4f90998640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8c109290 0x7f4f8c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:06.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.091+0000 7f4f90998640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f8c108910 0x7f4f8c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:06.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.091+0000 7f4f90998640 1 --2- 172.21.15.78:0/3503145331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8c101810 0x7f4f8c199400 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:06.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.091+0000 7f4f90998640 1 -- 172.21.15.78:0/3503145331 >> 172.21.15.78:0/3503145331 conn(0x7f4f8c076f20 msgr2=0x7f4f8c1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:06.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.091+0000 7f4f90998640 1 -- 172.21.15.78:0/3503145331 shutdown_connections 2024-09-11T01:08:06.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:06.091+0000 7f4f90998640 1 -- 172.21.15.78:0/3503145331 wait complete. 2024-09-11T01:08:06.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:06 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3503145331' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:08:06.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 623219 5.32 MB/sec execute 374 sec latency 1967.765 ms 2024-09-11T01:08:06.472 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:06 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3503145331' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:08:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:06 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3503145331' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:08:06.846 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:08:07.096 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:07.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:07 smithi110 ceph-mon[34712]: pgmap v604: 97 pgs: 97 active+clean; 446 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 204 op/s 2024-09-11T01:08:07.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 623895 5.31 MB/sec execute 375 sec latency 2717.394 ms 2024-09-11T01:08:07.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:07 smithi112 ceph-mon[35384]: pgmap v604: 97 pgs: 97 active+clean; 446 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 204 op/s 2024-09-11T01:08:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:07 smithi078 ceph-mon[31315]: pgmap v604: 97 pgs: 97 active+clean; 446 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 204 op/s 2024-09-11T01:08:07.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.605+0000 7f180f180640 1 -- 172.21.15.78:0/2734864384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18081080a0 msgr2=0x7f180810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:07.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.605+0000 7f180f180640 1 --2- 172.21.15.78:0/2734864384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18081080a0 0x7f180810e8a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1800009900 tx=0x7f180002a410 comp rx=0 tx=0).stop 2024-09-11T01:08:07.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.605+0000 7f180f180640 1 -- 172.21.15.78:0/2734864384 shutdown_connections 2024-09-11T01:08:07.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.605+0000 7f180f180640 1 --2- 172.21.15.78:0/2734864384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18081080a0 0x7f180810e8a0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.605+0000 7f180f180640 1 --2- 172.21.15.78:0/2734864384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1808107720 0x7f1808107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.605+0000 7f180f180640 1 --2- 172.21.15.78:0/2734864384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1808101d10 0x7f18081020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.605+0000 7f180f180640 1 -- 172.21.15.78:0/2734864384 >> 172.21.15.78:0/2734864384 conn(0x7f18080fd7c0 msgr2=0x7f18080ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:07.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.605+0000 7f180f180640 1 -- 172.21.15.78:0/2734864384 shutdown_connections 2024-09-11T01:08:07.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.605+0000 7f180f180640 1 -- 172.21.15.78:0/2734864384 wait complete. 2024-09-11T01:08:07.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.605+0000 7f180f180640 1 Processor -- start 2024-09-11T01:08:07.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.605+0000 7f180f180640 1 -- start start 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f180f180640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1808101d10 0x7f18081973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f180f180640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1808107720 0x7f18081978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f180f180640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18081080a0 0x7f180819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f180f180640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1808111d60 con 0x7f1808101d10 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f180f180640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1808111be0 con 0x7f1808107720 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f180f180640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1808111ee0 con 0x7f18081080a0 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f180cf1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1808101d10 0x7f18081973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f180cf1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1808101d10 0x7f18081973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54134/0 (socket says 172.21.15.78:54134) 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f17fffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1808107720 0x7f18081978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f180cf1c640 1 -- 172.21.15.78:0/3790792057 learned_addr learned my addr 172.21.15.78:0/3790792057 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f180d71d640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18081080a0 0x7f180819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f17fffff640 1 -- 172.21.15.78:0/3790792057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18081080a0 msgr2=0x7f180819c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f17fffff640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18081080a0 0x7f180819c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f17fffff640 1 -- 172.21.15.78:0/3790792057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1808101d10 msgr2=0x7f18081973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:07.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f17fffff640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1808101d10 0x7f18081973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f17fffff640 1 -- 172.21.15.78:0/3790792057 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180819ce60 con 0x7f1808107720 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f180cf1c640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1808101d10 0x7f18081973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f17fffff640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1808107720 0x7f18081978e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f17f800e9f0 tx=0x7f17f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f17fdffb640 1 -- 172.21.15.78:0/3790792057 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f800ccc0 con 0x7f1808107720 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.606+0000 7f17fdffb640 1 -- 172.21.15.78:0/3790792057 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f80108e0 con 0x7f1808107720 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.607+0000 7f180f180640 1 -- 172.21.15.78:0/3790792057 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180819d150 con 0x7f1808107720 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.607+0000 7f17fdffb640 1 -- 172.21.15.78:0/3790792057 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8018880 con 0x7f1808107720 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.607+0000 7f180f180640 1 -- 172.21.15.78:0/3790792057 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180819d6e0 con 0x7f1808107720 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.609+0000 7f1811555740 1 Processor -- start 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.609+0000 7f1811555740 1 -- start start 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.609+0000 7f1811555740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a84d0 0x55b13c2a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.609+0000 7f1811555740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a8df0 0x55b13c2b4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.609+0000 7f1811555740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.609+0000 7f1811555740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b13c21eea0 con 0x55b13c2a8df0 2024-09-11T01:08:07.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.609+0000 7f1811555740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b13c20e2d0 con 0x55b13c2a84d0 2024-09-11T01:08:07.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.609+0000 7f1811555740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b13c2120e0 con 0x55b13c2b4d80 2024-09-11T01:08:07.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.609+0000 7f17fdffb640 1 -- 172.21.15.78:0/3790792057 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f17f8010430 con 0x7f1808107720 2024-09-11T01:08:07.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.610+0000 7f180d71d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a84d0 0x55b13c2a88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.610+0000 7f180d71d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a84d0 0x55b13c2a88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55974/0 (socket says 172.21.15.78:55974) 2024-09-11T01:08:07.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.610+0000 7f180d71d640 1 -- 172.21.15.78:0/3155866389 learned_addr learned my addr 172.21.15.78:0/3155866389 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:07.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.610+0000 7f17fffff640 1 --2- 172.21.15.78:0/3155866389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.610+0000 7f180cf1c640 1 --2- 172.21.15.78:0/3155866389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a8df0 0x55b13c2b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.610+0000 7f180d71d640 1 -- 172.21.15.78:0/3155866389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 msgr2=0x55b13c2b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.610+0000 7f180d71d640 1 --2- 172.21.15.78:0/3155866389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.610+0000 7f180d71d640 1 -- 172.21.15.78:0/3155866389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a8df0 msgr2=0x55b13c2b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.610+0000 7f180d71d640 1 --2- 172.21.15.78:0/3155866389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a8df0 0x55b13c2b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.610+0000 7f180d71d640 1 -- 172.21.15.78:0/3155866389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13bf6c650 con 0x55b13c2a84d0 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.610+0000 7f180d71d640 1 --2- 172.21.15.78:0/3155866389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a84d0 0x55b13c2a88b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f180002a930 tx=0x7f1800032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e19c3328a5583cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f17fdffb640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f17f00b54d0 0x7f17f00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f17eeffd640 1 -- 172.21.15.78:0/3155866389 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800040070 con 0x55b13c2a84d0 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f17eeffd640 1 -- 172.21.15.78:0/3155866389 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1800004030 con 0x55b13c2a84d0 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f17eeffd640 1 -- 172.21.15.78:0/3155866389 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800035040 con 0x55b13c2a84d0 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f180cf1c640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f17f00b54d0 0x7f17f00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f17fdffb640 1 -- 172.21.15.78:0/3790792057 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f17f80e6270 con 0x7f1808107720 2024-09-11T01:08:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f1811555740 1 -- 172.21.15.78:0/3155866389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a84d0 msgr2=0x55b13c2a88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f180cf1c640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f17f00b54d0 0x7f17f00b7990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55b13c2a71e0 tx=0x7f1804009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f1811555740 1 --2- 172.21.15.78:0/3155866389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a84d0 0x55b13c2a88b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f180002a930 tx=0x7f1800032b20 comp rx=0 tx=0).stop 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f1811555740 1 -- 172.21.15.78:0/3155866389 shutdown_connections 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f1811555740 1 --2- 172.21.15.78:0/3155866389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f1811555740 1 --2- 172.21.15.78:0/3155866389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a8df0 0x55b13c2b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f1811555740 1 --2- 172.21.15.78:0/3155866389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a84d0 0x55b13c2a88b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f1811555740 1 -- 172.21.15.78:0/3155866389 >> 172.21.15.78:0/3155866389 conn(0x55b13c2a6300 msgr2=0x55b13c2a74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f1811555740 1 -- 172.21.15.78:0/3155866389 shutdown_connections 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.611+0000 7f1811555740 1 -- 172.21.15.78:0/3155866389 wait complete. 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.612+0000 7f1811555740 1 -- start start 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.612+0000 7f1811555740 1 Processor -- start 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.612+0000 7f1811555740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a84d0 0x55b13c2b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.612+0000 7f1811555740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2a8df0 0x55b13c2b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.612+0000 7f1811555740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2b4d80 0x55b13c2b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.612+0000 7f1811555740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b13c06f700 con 0x55b13c2b4d80 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.612+0000 7f1811555740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b13c1a9160 con 0x55b13c2a84d0 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.612+0000 7f1811555740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b13c2b79a0 con 0x55b13c2a8df0 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.612+0000 7f180d71d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2b4d80 0x55b13c2b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.612+0000 7f1811555740 10 client.0 _ll_register_callbacks cb 0x55b13c1c29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.612+0000 7f180d71d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2b4d80 0x55b13c2b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54162/0 (socket says 172.21.15.78:54162) 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f180d71d640 1 -- 172.21.15.78:0/2954043741 learned_addr learned my addr 172.21.15.78:0/2954043741 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f17fffff640 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2a8df0 0x55b13c2b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f180cf1c640 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a84d0 0x55b13c2b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f180d71d640 1 -- 172.21.15.78:0/2954043741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2a8df0 msgr2=0x55b13c2b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f180d71d640 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2a8df0 0x55b13c2b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f180d71d640 1 -- 172.21.15.78:0/2954043741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a84d0 msgr2=0x55b13c2b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f180d71d640 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a84d0 0x55b13c2b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f180d71d640 1 -- 172.21.15.78:0/2954043741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13c1d6c90 con 0x55b13c2b4d80 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f180d71d640 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2b4d80 0x55b13c2b21c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f1800005940 tx=0x7f18000095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac160b13060caa33 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f17e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f17e7fff640 1 -- 172.21.15.78:0/2954043741 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180004c020 con 0x55b13c2b4d80 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f17e7fff640 1 -- 172.21.15.78:0/2954043741 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f18000316a0 con 0x55b13c2b4d80 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f1811555740 1 -- 172.21.15.78:0/2954043741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1800004030 con 0x55b13c2b4d80 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f17e7fff640 1 -- 172.21.15.78:0/2954043741 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1800057050 con 0x55b13c2b4d80 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f1811555740 10 client.17211 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f1811555740 1 -- 172.21.15.78:0/2954043741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b13c2c3380 con 0x55b13c2b4d80 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.613+0000 7f17e7fff640 1 -- 172.21.15.78:0/2954043741 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1800004030 con 0x55b13c2b4d80 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.614+0000 7f17e7fff640 1 -- 172.21.15.78:0/2954043741 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f1800003b30 con 0x55b13c2b4d80 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.614+0000 7f1811555740 10 client.17211 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.614+0000 7f1811555740 10 client.17211 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:08:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.614+0000 7f1811555740 20 client.17211 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.614+0000 7f1811555740 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b13c2c46d0 0x55b13c2c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.614+0000 7f1811555740 4 client.17211 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.614+0000 7f1811555740 1 -- 172.21.15.78:0/2954043741 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b13c220df0 con 0x55b13c2c46d0 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.614+0000 7f180cf1c640 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b13c2c46d0 0x55b13c2c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.614+0000 7f180cf1c640 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b13c2c46d0 0x55b13c2c6ac0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d1e0ab8da2be672c server_cookie=673dbdde70036ff6 in_seq=0 out_seq=0 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.614+0000 7f17e7fff640 10 client.17211 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f17e7fff640 1 -- 172.21.15.78:0/2954043741 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b13c220df0 con 0x55b13c2c46d0 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f17e7fff640 10 client.17211 handle_command_reply: tid=0 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f1811555740 1 client.17211 shutdown 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f1811555740 1 -- 172.21.15.78:0/2954043741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2b4d80 msgr2=0x55b13c2b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f1811555740 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2b4d80 0x55b13c2b21c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f1800005940 tx=0x7f18000095a0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f1811555740 1 -- 172.21.15.78:0/2954043741 shutdown_connections 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f1811555740 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b13c2c46d0 0x55b13c2c6ac0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f1811555740 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2b4d80 0x55b13c2b21c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f1811555740 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2a8df0 0x55b13c2b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f17e7fff640 0 client.17211 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f1811555740 1 --2- 172.21.15.78:0/2954043741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a84d0 0x55b13c2b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f1811555740 1 -- 172.21.15.78:0/2954043741 >> 172.21.15.78:0/2954043741 conn(0x55b13c2a6300 msgr2=0x55b13c2abea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f1811555740 1 -- 172.21.15.78:0/2954043741 shutdown_connections 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f1811555740 1 -- 172.21.15.78:0/2954043741 wait complete. 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.615+0000 7f1811555740 20 client.17211 trim_cache size 0 max 16384 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f1811555740 1 Processor -- start 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f1811555740 1 -- start start 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f1811555740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 0x55b13c2c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f1811555740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a8df0 0x55b13c2b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f1811555740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f1811555740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18081bd400 con 0x55b13c2a84d0 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f1811555740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f18081bef70 con 0x55b13c2a8df0 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f1811555740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f18081bdad0 con 0x55b13c2b4d80 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f180cf1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 0x55b13c2c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f180d71d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f180cf1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 0x55b13c2c0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54170/0 (socket says 172.21.15.78:54170) 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f180cf1c640 1 -- 172.21.15.78:0/2497296608 learned_addr learned my addr 172.21.15.78:0/2497296608 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.630+0000 7f17fffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a8df0 0x55b13c2b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.631+0000 7f17fffff640 1 -- 172.21.15.78:0/2497296608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 msgr2=0x55b13c2b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.631+0000 7f17fffff640 1 --2- 172.21.15.78:0/2497296608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.631+0000 7f17fffff640 1 -- 172.21.15.78:0/2497296608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 msgr2=0x55b13c2c0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.631+0000 7f17fffff640 1 --2- 172.21.15.78:0/2497296608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 0x55b13c2c0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.631+0000 7f17fffff640 1 -- 172.21.15.78:0/2497296608 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13c2a6fe0 con 0x55b13c2a8df0 2024-09-11T01:08:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.631+0000 7f180cf1c640 1 --2- 172.21.15.78:0/2497296608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 0x55b13c2c0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.631+0000 7f17fffff640 1 --2- 172.21.15.78:0/2497296608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a8df0 0x55b13c2b3990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f17f80e39e0 tx=0x7f17f80e9dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a69c71fe3799e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.631+0000 7f17ee7fc640 1 -- 172.21.15.78:0/2497296608 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f80ac110 con 0x55b13c2a8df0 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.631+0000 7f17ee7fc640 1 -- 172.21.15.78:0/2497296608 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f80ac2d0 con 0x55b13c2a8df0 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.631+0000 7f17ee7fc640 1 -- 172.21.15.78:0/2497296608 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f80abf30 con 0x55b13c2a8df0 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.632+0000 7f1811555740 1 -- 172.21.15.78:0/2497296608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a8df0 msgr2=0x55b13c2b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.632+0000 7f1811555740 1 --2- 172.21.15.78:0/2497296608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a8df0 0x55b13c2b3990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f17f80e39e0 tx=0x7f17f80e9dc0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.632+0000 7f1811555740 1 -- 172.21.15.78:0/2497296608 shutdown_connections 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.632+0000 7f1811555740 1 --2- 172.21.15.78:0/2497296608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.632+0000 7f1811555740 1 --2- 172.21.15.78:0/2497296608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a8df0 0x55b13c2b3990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.632+0000 7f1811555740 1 --2- 172.21.15.78:0/2497296608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 0x55b13c2c0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.632+0000 7f1811555740 1 -- 172.21.15.78:0/2497296608 >> 172.21.15.78:0/2497296608 conn(0x55b13c2a6300 msgr2=0x55b13c2b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.632+0000 7f1811555740 1 -- 172.21.15.78:0/2497296608 shutdown_connections 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.632+0000 7f1811555740 1 -- 172.21.15.78:0/2497296608 wait complete. 2024-09-11T01:08:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.632+0000 7f1811555740 1 -- start start 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.632+0000 7f1811555740 1 Processor -- start 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f1811555740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 0x55b13c2bd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f1811555740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a8df0 0x55b13c2bd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f1811555740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f1811555740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18081be870 con 0x55b13c2a84d0 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f1811555740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f18081be1a0 con 0x55b13c2a8df0 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f1811555740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f18081bb400 con 0x55b13c2b4d80 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f1811555740 10 client.0 _ll_register_callbacks cb 0x7f17f80e9c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f180cf1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 0x55b13c2bd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f180cf1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 0x55b13c2bd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54176/0 (socket says 172.21.15.78:54176) 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f180cf1c640 1 -- 172.21.15.78:0/1273872162 learned_addr learned my addr 172.21.15.78:0/1273872162 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f17fffff640 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a8df0 0x55b13c2bd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f180d71d640 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f180cf1c640 1 -- 172.21.15.78:0/1273872162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 msgr2=0x55b13c2b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f180cf1c640 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f180cf1c640 1 -- 172.21.15.78:0/1273872162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a8df0 msgr2=0x55b13c2bd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f180cf1c640 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a8df0 0x55b13c2bd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f180cf1c640 1 -- 172.21.15.78:0/1273872162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13c2b8650 con 0x55b13c2a84d0 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.633+0000 7f180d71d640 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f180cf1c640 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 0x55b13c2bd0d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f180400fce0 tx=0x7f1804004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eedfe1e99d76e78 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f17e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f17e7fff640 1 -- 172.21.15.78:0/1273872162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180401f8a0 con 0x55b13c2a84d0 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f1811555740 1 -- 172.21.15.78:0/1273872162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f80ac2d0 con 0x55b13c2a84d0 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f17e7fff640 1 -- 172.21.15.78:0/1273872162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f180401fa40 con 0x55b13c2a84d0 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f17e7fff640 1 -- 172.21.15.78:0/1273872162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f180401fc40 con 0x55b13c2a84d0 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f1811555740 10 client.17217 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f1811555740 1 -- 172.21.15.78:0/1273872162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b13c332050 con 0x55b13c2a84d0 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f17e7fff640 1 -- 172.21.15.78:0/1273872162 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f80ac2d0 con 0x55b13c2a84d0 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f17e7fff640 1 -- 172.21.15.78:0/1273872162 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f1804038020 con 0x55b13c2a84d0 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f1811555740 10 client.17217 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f1811555740 10 client.17217 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f1811555740 20 client.17217 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f1811555740 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b13c332800 0x55b13c334c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f1811555740 4 client.17217 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f1811555740 1 -- 172.21.15.78:0/1273872162 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b13c21c8a0 con 0x55b13c332800 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.634+0000 7f17fffff640 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b13c332800 0x55b13c334c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f17e7fff640 10 client.17217 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f17fffff640 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b13c332800 0x55b13c334c20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=864514c6a99bea26 server_cookie=28c293d0d9e318e0 in_seq=0 out_seq=0 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f17e7fff640 1 -- 172.21.15.78:0/1273872162 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b13c21c8a0 con 0x55b13c332800 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f17e7fff640 10 client.17217 handle_command_reply: tid=0 2024-09-11T01:08:07.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f1811555740 1 client.17217 shutdown 2024-09-11T01:08:07.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f1811555740 1 -- 172.21.15.78:0/1273872162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 msgr2=0x55b13c2bd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:07.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f1811555740 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 0x55b13c2bd0d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f180400fce0 tx=0x7f1804004a40 comp rx=0 tx=0).stop 2024-09-11T01:08:07.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f1811555740 1 -- 172.21.15.78:0/1273872162 shutdown_connections 2024-09-11T01:08:07.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f1811555740 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b13c332800 0x55b13c334c20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f1811555740 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13c2b4d80 0x55b13c2b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f1811555740 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b13c2a8df0 0x55b13c2bd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f17e7fff640 0 client.17217 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:08:07.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f1811555740 1 --2- 172.21.15.78:0/1273872162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b13c2a84d0 0x55b13c2bd0d0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f1811555740 1 -- 172.21.15.78:0/1273872162 >> 172.21.15.78:0/1273872162 conn(0x55b13c2a6300 msgr2=0x55b13c2a5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:07.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f1811555740 1 -- 172.21.15.78:0/1273872162 shutdown_connections 2024-09-11T01:08:07.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.635+0000 7f1811555740 1 -- 172.21.15.78:0/1273872162 wait complete. 2024-09-11T01:08:07.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f1811555740 20 client.17217 trim_cache size 0 max 16384 2024-09-11T01:08:07.653 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:08:07.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f180f180640 1 -- 172.21.15.78:0/3790792057 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f17f00b54d0 msgr2=0x7f17f00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:07.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f180f180640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f17f00b54d0 0x7f17f00b7990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55b13c2a71e0 tx=0x7f1804009500 comp rx=0 tx=0).stop 2024-09-11T01:08:07.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f180f180640 1 -- 172.21.15.78:0/3790792057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1808107720 msgr2=0x7f18081978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:07.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f180f180640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1808107720 0x7f18081978e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f17f800e9f0 tx=0x7f17f800eeb0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f180f180640 1 -- 172.21.15.78:0/3790792057 shutdown_connections 2024-09-11T01:08:07.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f180f180640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f17f00b54d0 0x7f17f00b7990 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f180f180640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18081080a0 0x7f180819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f180f180640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1808107720 0x7f18081978e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f180f180640 1 --2- 172.21.15.78:0/3790792057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1808101d10 0x7f18081973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:07.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f180f180640 1 -- 172.21.15.78:0/3790792057 >> 172.21.15.78:0/3790792057 conn(0x7f18080fd7c0 msgr2=0x7f180806a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:07.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f180f180640 1 -- 172.21.15.78:0/3790792057 shutdown_connections 2024-09-11T01:08:07.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:07.636+0000 7f180f180640 1 -- 172.21.15.78:0/3790792057 wait complete. 2024-09-11T01:08:08.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:08 smithi078 ceph-mon[31315]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T01:08:08.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:08 smithi110 ceph-mon[34712]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T01:08:08.441 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:08:08.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 625609 5.33 MB/sec execute 376 sec latency 10.404 ms 2024-09-11T01:08:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:08 smithi112 ceph-mon[35384]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T01:08:08.714 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:09.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.241+0000 7fd47d415640 1 -- 172.21.15.78:0/996334319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781076c0 msgr2=0x7fd478107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:09.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.241+0000 7fd47d415640 1 --2- 172.21.15.78:0/996334319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781076c0 0x7fd478107b00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd460009900 tx=0x7fd46002a410 comp rx=0 tx=0).stop 2024-09-11T01:08:09.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.241+0000 7fd47d415640 1 -- 172.21.15.78:0/996334319 shutdown_connections 2024-09-11T01:08:09.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.241+0000 7fd47d415640 1 --2- 172.21.15.78:0/996334319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd478108040 0x7fd47810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.241+0000 7fd47d415640 1 --2- 172.21.15.78:0/996334319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781076c0 0x7fd478107b00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.241+0000 7fd47d415640 1 --2- 172.21.15.78:0/996334319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd478101cb0 0x7fd478102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.241+0000 7fd47d415640 1 -- 172.21.15.78:0/996334319 >> 172.21.15.78:0/996334319 conn(0x7fd4780fd700 msgr2=0x7fd4780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:09.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.241+0000 7fd47d415640 1 -- 172.21.15.78:0/996334319 shutdown_connections 2024-09-11T01:08:09.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.241+0000 7fd47d415640 1 -- 172.21.15.78:0/996334319 wait complete. 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.241+0000 7fd47d415640 1 Processor -- start 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.241+0000 7fd47d415640 1 -- start start 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd47d415640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd478101cb0 0x7fd478199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd47d415640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781076c0 0x7fd4781997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd47d415640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd478108040 0x7fd47819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd47d415640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd478111d00 con 0x7fd4781076c0 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd47d415640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd478111b80 con 0x7fd478101cb0 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd47d415640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd478111e80 con 0x7fd478108040 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd46e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781076c0 0x7fd4781997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd46e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781076c0 0x7fd4781997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54200/0 (socket says 172.21.15.78:54200) 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd476ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd478101cb0 0x7fd478199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd46e7fc640 1 -- 172.21.15.78:0/4172977953 learned_addr learned my addr 172.21.15.78:0/4172977953 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:09.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd4777fe640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd478108040 0x7fd47819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd46e7fc640 1 -- 172.21.15.78:0/4172977953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd478108040 msgr2=0x7fd47819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd46e7fc640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd478108040 0x7fd47819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd46e7fc640 1 -- 172.21.15.78:0/4172977953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd478101cb0 msgr2=0x7fd478199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd46e7fc640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd478101cb0 0x7fd478199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd46e7fc640 1 -- 172.21.15.78:0/4172977953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47819f330 con 0x7fd4781076c0 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd46e7fc640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781076c0 0x7fd4781997c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd460005940 tx=0x7fd4600040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd474ff9640 1 -- 172.21.15.78:0/4172977953 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46003f030 con 0x7fd4781076c0 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd47d415640 1 -- 172.21.15.78:0/4172977953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47819f5c0 con 0x7fd4781076c0 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.242+0000 7fd474ff9640 1 -- 172.21.15.78:0/4172977953 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4600044c0 con 0x7fd4781076c0 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.243+0000 7fd474ff9640 1 -- 172.21.15.78:0/4172977953 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460035070 con 0x7fd4781076c0 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.243+0000 7fd47d415640 1 -- 172.21.15.78:0/4172977953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4781a5ce0 con 0x7fd4781076c0 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.245+0000 7fd474ff9640 1 -- 172.21.15.78:0/4172977953 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd46003ba20 con 0x7fd4781076c0 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.245+0000 7fd47f7ea740 1 Processor -- start 2024-09-11T01:08:09.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.245+0000 7fd47f7ea740 1 -- start start 2024-09-11T01:08:09.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd47f7ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c874628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd47f7ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 0x561c8746e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd47f7ea740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c8746ed80 0x561c87471170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd47f7ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561c873d8ea0 con 0x561c87462df0 2024-09-11T01:08:09.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd47f7ea740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561c873c82d0 con 0x561c8746ed80 2024-09-11T01:08:09.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd47f7ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561c873cc0e0 con 0x561c874624d0 2024-09-11T01:08:09.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd4777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c874628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd476ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 0x561c8746e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd46e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c8746ed80 0x561c87471170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd4777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c874628b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:50552/0 (socket says 172.21.15.78:50552) 2024-09-11T01:08:09.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd4777fe640 1 -- 172.21.15.78:0/1785015278 learned_addr learned my addr 172.21.15.78:0/1785015278 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd4777fe640 1 -- 172.21.15.78:0/1785015278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c8746ed80 msgr2=0x561c87471170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd4777fe640 1 --2- 172.21.15.78:0/1785015278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c8746ed80 0x561c87471170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd4777fe640 1 -- 172.21.15.78:0/1785015278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 msgr2=0x561c8746e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd4777fe640 1 --2- 172.21.15.78:0/1785015278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 0x561c8746e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd4777fe640 1 -- 172.21.15.78:0/1785015278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c87126650 con 0x561c874624d0 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd476ffd640 1 --2- 172.21.15.78:0/1785015278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 0x561c8746e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd474ff9640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4640b5450 0x7fd4640b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd476ffd640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4640b5450 0x7fd4640b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd4777fe640 1 --2- 172.21.15.78:0/1785015278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c874628b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd470009fd0 tx=0x7fd4700118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e9d7d5490aa31a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.246+0000 7fd474ff9640 1 -- 172.21.15.78:0/4172977953 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7fd460106590 con 0x7fd4781076c0 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.247+0000 7fd46d7fa640 1 -- 172.21.15.78:0/1785015278 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470004d20 con 0x561c874624d0 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.247+0000 7fd46d7fa640 1 -- 172.21.15.78:0/1785015278 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd470004050 con 0x561c874624d0 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.247+0000 7fd46d7fa640 1 -- 172.21.15.78:0/1785015278 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470005280 con 0x561c874624d0 2024-09-11T01:08:09.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.247+0000 7fd47f7ea740 1 -- 172.21.15.78:0/1785015278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 msgr2=0x561c874628b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:09.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.247+0000 7fd476ffd640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4640b5450 0x7fd4640b7910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x561c874611e0 tx=0x7fd468005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:09.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.247+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/1785015278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c874628b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd470009fd0 tx=0x7fd4700118d0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.247+0000 7fd47f7ea740 1 -- 172.21.15.78:0/1785015278 shutdown_connections 2024-09-11T01:08:09.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.247+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/1785015278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c8746ed80 0x561c87471170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.247+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/1785015278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 0x561c8746e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.247+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/1785015278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c874628b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd47f7ea740 1 -- 172.21.15.78:0/1785015278 >> 172.21.15.78:0/1785015278 conn(0x561c87460300 msgr2=0x561c874614a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:09.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd47f7ea740 1 -- 172.21.15.78:0/1785015278 shutdown_connections 2024-09-11T01:08:09.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd47f7ea740 1 -- 172.21.15.78:0/1785015278 wait complete. 2024-09-11T01:08:09.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd47f7ea740 1 -- start start 2024-09-11T01:08:09.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd47f7ea740 1 Processor -- start 2024-09-11T01:08:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd47f7ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c8746e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd47f7ea740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 0x561c8746bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd47f7ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c8746ed80 0x561c8746c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd47f7ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561c87229700 con 0x561c8746ed80 2024-09-11T01:08:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd47f7ea740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561c87363160 con 0x561c87462df0 2024-09-11T01:08:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd47f7ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561c87471a00 con 0x561c874624d0 2024-09-11T01:08:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd47f7ea740 10 client.0 _ll_register_callbacks cb 0x561c8737c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd46e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 0x561c8746bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd4777fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c8746ed80 0x561c8746c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd476ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c8746e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.248+0000 7fd46e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 0x561c8746bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:56056/0 (socket says 172.21.15.78:56056) 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.249+0000 7fd46e7fc640 1 -- 172.21.15.78:0/4127822149 learned_addr learned my addr 172.21.15.78:0/4127822149 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.249+0000 7fd46e7fc640 1 -- 172.21.15.78:0/4127822149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 msgr2=0x561c8746e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.249+0000 7fd46e7fc640 1 --2- 172.21.15.78:0/4127822149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c8746e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.249+0000 7fd46e7fc640 1 -- 172.21.15.78:0/4127822149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c8746ed80 msgr2=0x561c8746c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.249+0000 7fd46e7fc640 1 --2- 172.21.15.78:0/4127822149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c8746ed80 0x561c8746c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.249+0000 7fd46e7fc640 1 -- 172.21.15.78:0/4127822149 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c87390c90 con 0x561c87462df0 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.249+0000 7fd46e7fc640 1 --2- 172.21.15.78:0/4127822149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 0x561c8746bae0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd460103d60 tx=0x7fd4600c8b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7348e3201f4cc5bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.249+0000 7fd45e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.249+0000 7fd45e7fc640 1 -- 172.21.15.78:0/4127822149 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460049070 con 0x561c87462df0 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.249+0000 7fd45e7fc640 1 -- 172.21.15.78:0/4127822149 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7fd460103f80 con 0x561c87462df0 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.249+0000 7fd47f7ea740 1 -- 172.21.15.78:0/4127822149 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd470004050 con 0x561c87462df0 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.249+0000 7fd45e7fc640 1 -- 172.21.15.78:0/4127822149 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd4600c8980 con 0x561c87462df0 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.250+0000 7fd47f7ea740 10 client.26477 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.250+0000 7fd47f7ea740 1 -- 172.21.15.78:0/4127822149 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561c8747d3c0 con 0x561c87462df0 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.250+0000 7fd45e7fc640 1 -- 172.21.15.78:0/4127822149 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd470004050 con 0x561c87462df0 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.250+0000 7fd45e7fc640 1 -- 172.21.15.78:0/4127822149 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7fd4600d1190 con 0x561c87462df0 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.250+0000 7fd47f7ea740 10 client.26477 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.250+0000 7fd47f7ea740 10 client.26477 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.250+0000 7fd47f7ea740 20 client.26477 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.250+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/4127822149 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561c8747e870 0x561c87480c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.250+0000 7fd47f7ea740 4 client.26477 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.250+0000 7fd47f7ea740 1 -- 172.21.15.78:0/4127822149 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c873dadf0 con 0x561c8747e870 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.250+0000 7fd476ffd640 1 --2- 172.21.15.78:0/4127822149 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561c8747e870 0x561c87480c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.251+0000 7fd45e7fc640 10 client.26477 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.251+0000 7fd476ffd640 1 --2- 172.21.15.78:0/4127822149 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561c8747e870 0x561c87480c60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=71956e2e4c9561b2 server_cookie=f62ed6d3a17c224b in_seq=0 out_seq=0 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.251+0000 7fd45e7fc640 1 -- 172.21.15.78:0/4127822149 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561c873dadf0 con 0x561c8747e870 2024-09-11T01:08:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.251+0000 7fd45e7fc640 10 client.26477 handle_command_reply: tid=0 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.251+0000 7fd47f7ea740 1 client.26477 shutdown 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.251+0000 7fd47f7ea740 1 -- 172.21.15.78:0/4127822149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 msgr2=0x561c8746bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.251+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/4127822149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 0x561c8746bae0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd460103d60 tx=0x7fd4600c8b40 comp rx=0 tx=0).stop 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.251+0000 7fd47f7ea740 1 -- 172.21.15.78:0/4127822149 shutdown_connections 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.251+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/4127822149 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561c8747e870 0x561c87480c60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.252+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/4127822149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c8746ed80 0x561c8746c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.252+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/4127822149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 0x561c8746bae0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.252+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/4127822149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c8746e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.252+0000 7fd47f7ea740 1 -- 172.21.15.78:0/4127822149 >> 172.21.15.78:0/4127822149 conn(0x561c87460300 msgr2=0x561c87465ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.252+0000 7fd45e7fc640 0 client.26477 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.252+0000 7fd47f7ea740 1 -- 172.21.15.78:0/4127822149 shutdown_connections 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.252+0000 7fd47f7ea740 1 -- 172.21.15.78:0/4127822149 wait complete. 2024-09-11T01:08:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.252+0000 7fd47f7ea740 20 client.26477 trim_cache size 0 max 16384 2024-09-11T01:08:09.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.266+0000 7fd47f7ea740 1 Processor -- start 2024-09-11T01:08:09.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.266+0000 7fd47f7ea740 1 -- start start 2024-09-11T01:08:09.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.266+0000 7fd47f7ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c874624d0 0x561c8747ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.266+0000 7fd47f7ea740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 0x561c87480880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.266+0000 7fd47f7ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c8746ed80 0x561c87470170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd47f7ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4781ac360 con 0x561c874624d0 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd47f7ea740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd4781aded0 con 0x561c87462df0 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd47f7ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4781aca30 con 0x561c8746ed80 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd476ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c874624d0 0x561c8747ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd46e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 0x561c87480880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd476ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c874624d0 0x561c8747ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54232/0 (socket says 172.21.15.78:54232) 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd476ffd640 1 -- 172.21.15.78:0/1223426158 learned_addr learned my addr 172.21.15.78:0/1223426158 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd4777fe640 1 --2- 172.21.15.78:0/1223426158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c8746ed80 0x561c87470170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd476ffd640 1 -- 172.21.15.78:0/1223426158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c8746ed80 msgr2=0x561c87470170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd476ffd640 1 --2- 172.21.15.78:0/1223426158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c8746ed80 0x561c87470170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd476ffd640 1 -- 172.21.15.78:0/1223426158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 msgr2=0x561c87480880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd476ffd640 1 --2- 172.21.15.78:0/1223426158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 0x561c87480880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd476ffd640 1 -- 172.21.15.78:0/1223426158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c87481240 con 0x561c874624d0 2024-09-11T01:08:09.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd46e7fc640 1 --2- 172.21.15.78:0/1223426158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 0x561c87480880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:09.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd476ffd640 1 --2- 172.21.15.78:0/1223426158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c874624d0 0x561c8747ac90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd4680045d0 tx=0x7fd46801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469895857c9fd81a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:09.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd46cff9640 1 -- 172.21.15.78:0/1223426158 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468022de0 con 0x561c874624d0 2024-09-11T01:08:09.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd46cff9640 1 -- 172.21.15.78:0/1223426158 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd468022580 con 0x561c874624d0 2024-09-11T01:08:09.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.267+0000 7fd46cff9640 1 -- 172.21.15.78:0/1223426158 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468005280 con 0x561c874624d0 2024-09-11T01:08:09.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.268+0000 7fd47f7ea740 1 -- 172.21.15.78:0/1223426158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c874624d0 msgr2=0x561c8747ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:09.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.268+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/1223426158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c874624d0 0x561c8747ac90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd4680045d0 tx=0x7fd46801b000 comp rx=0 tx=0).stop 2024-09-11T01:08:09.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.268+0000 7fd47f7ea740 1 -- 172.21.15.78:0/1223426158 shutdown_connections 2024-09-11T01:08:09.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.268+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/1223426158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c8746ed80 0x561c87470170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.268+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/1223426158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c87462df0 0x561c87480880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.268+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/1223426158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c874624d0 0x561c8747ac90 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.268+0000 7fd47f7ea740 1 -- 172.21.15.78:0/1223426158 >> 172.21.15.78:0/1223426158 conn(0x561c87460300 msgr2=0x561c8746a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.268+0000 7fd47f7ea740 1 -- 172.21.15.78:0/1223426158 shutdown_connections 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.268+0000 7fd47f7ea740 1 -- 172.21.15.78:0/1223426158 wait complete. 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd47f7ea740 1 -- start start 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd47f7ea740 1 Processor -- start 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd47f7ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c87477360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd47f7ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 0x561c87471cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd47f7ea740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c8746ed80 0x561c874721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd47f7ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4781ad7d0 con 0x561c87462df0 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd47f7ea740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd4781ad100 con 0x561c8746ed80 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd47f7ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4781aa360 con 0x561c874624d0 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd47f7ea740 10 client.0 _ll_register_callbacks cb 0x7fd460109dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd46e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 0x561c87471cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd46e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 0x561c87471cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54234/0 (socket says 172.21.15.78:54234) 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd46e7fc640 1 -- 172.21.15.78:0/3262875959 learned_addr learned my addr 172.21.15.78:0/3262875959 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd476ffd640 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c87477360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd4777fe640 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c8746ed80 0x561c874721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd46e7fc640 1 -- 172.21.15.78:0/3262875959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 msgr2=0x561c87477360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd46e7fc640 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c87477360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd46e7fc640 1 -- 172.21.15.78:0/3262875959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c8746ed80 msgr2=0x561c874721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd46e7fc640 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c8746ed80 0x561c874721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd46e7fc640 1 -- 172.21.15.78:0/3262875959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c87460d80 con 0x561c87462df0 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.269+0000 7fd476ffd640 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c87477360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd46e7fc640 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 0x561c87471cb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd4600cca70 tx=0x7fd4600c8f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd64712dc4f9350 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd45e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd45e7fc640 1 -- 172.21.15.78:0/3262875959 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460049070 con 0x561c87462df0 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd45e7fc640 1 -- 172.21.15.78:0/3262875959 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7fd4600d5d30 con 0x561c87462df0 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd47f7ea740 1 -- 172.21.15.78:0/3262875959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd468022580 con 0x561c87462df0 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd45e7fc640 1 -- 172.21.15.78:0/3262875959 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd4600cc250 con 0x561c87462df0 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd47f7ea740 10 client.17235 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd47f7ea740 1 -- 172.21.15.78:0/3262875959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561c874ec420 con 0x561c87462df0 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd45e7fc640 1 -- 172.21.15.78:0/3262875959 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd468022580 con 0x561c87462df0 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd45e7fc640 1 -- 172.21.15.78:0/3262875959 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7fd4600c8430 con 0x561c87462df0 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd47f7ea740 10 client.17235 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd47f7ea740 10 client.17235 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd47f7ea740 20 client.17235 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561c874eca40 0x561c874eee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd47f7ea740 4 client.17235 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.270+0000 7fd47f7ea740 1 -- 172.21.15.78:0/3262875959 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c873d68a0 con 0x561c874eca40 2024-09-11T01:08:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.271+0000 7fd476ffd640 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561c874eca40 0x561c874eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.271+0000 7fd45e7fc640 10 client.17235 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.271+0000 7fd476ffd640 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561c874eca40 0x561c874eee60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=502077b7c6aa2fd server_cookie=d5314e75db6a7b48 in_seq=0 out_seq=0 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.271+0000 7fd45e7fc640 1 -- 172.21.15.78:0/3262875959 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c873d68a0 con 0x561c874eca40 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.271+0000 7fd45e7fc640 10 client.17235 handle_command_reply: tid=0 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.271+0000 7fd47f7ea740 1 client.17235 shutdown 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47f7ea740 1 -- 172.21.15.78:0/3262875959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 msgr2=0x561c87471cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 0x561c87471cb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd4600cca70 tx=0x7fd4600c8f80 comp rx=0 tx=0).stop 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47f7ea740 1 -- 172.21.15.78:0/3262875959 shutdown_connections 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561c874eca40 0x561c874eee60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c8746ed80 0x561c874721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c87462df0 0x561c87471cb0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47f7ea740 1 --2- 172.21.15.78:0/3262875959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c874624d0 0x561c87477360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47f7ea740 1 -- 172.21.15.78:0/3262875959 >> 172.21.15.78:0/3262875959 conn(0x561c87460300 msgr2=0x561c8746d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd45e7fc640 0 client.17235 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47f7ea740 1 -- 172.21.15.78:0/3262875959 shutdown_connections 2024-09-11T01:08:09.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47f7ea740 1 -- 172.21.15.78:0/3262875959 wait complete. 2024-09-11T01:08:09.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47f7ea740 20 client.17235 trim_cache size 0 max 16384 2024-09-11T01:08:09.280 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:08:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47d415640 1 -- 172.21.15.78:0/4172977953 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4640b5450 msgr2=0x7fd4640b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47d415640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4640b5450 0x7fd4640b7910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x561c874611e0 tx=0x7fd468005f90 comp rx=0 tx=0).stop 2024-09-11T01:08:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47d415640 1 -- 172.21.15.78:0/4172977953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781076c0 msgr2=0x7fd4781997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.272+0000 7fd47d415640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781076c0 0x7fd4781997c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd460005940 tx=0x7fd4600040c0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.273+0000 7fd47d415640 1 -- 172.21.15.78:0/4172977953 shutdown_connections 2024-09-11T01:08:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.273+0000 7fd47d415640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4640b5450 0x7fd4640b7910 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.273+0000 7fd47d415640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd478108040 0x7fd47819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.273+0000 7fd47d415640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4781076c0 0x7fd4781997c0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.273+0000 7fd47d415640 1 --2- 172.21.15.78:0/4172977953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd478101cb0 0x7fd478199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.273+0000 7fd47d415640 1 -- 172.21.15.78:0/4172977953 >> 172.21.15.78:0/4172977953 conn(0x7fd4780fd700 msgr2=0x7fd47806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.273+0000 7fd47d415640 1 -- 172.21.15.78:0/4172977953 shutdown_connections 2024-09-11T01:08:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:09.273+0000 7fd47d415640 1 -- 172.21.15.78:0/4172977953 wait complete. 2024-09-11T01:08:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:09 smithi110 ceph-mon[34712]: pgmap v606: 97 pgs: 97 active+clean; 446 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-09-11T01:08:09.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 627412 5.34 MB/sec execute 377 sec latency 148.011 ms 2024-09-11T01:08:09.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:09 smithi112 ceph-mon[35384]: pgmap v606: 97 pgs: 97 active+clean; 446 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-09-11T01:08:09.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:09 smithi078 ceph-mon[31315]: pgmap v606: 97 pgs: 97 active+clean; 446 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-09-11T01:08:10.110 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:08:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:10.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:10 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:08:10.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 629024 5.35 MB/sec execute 378 sec latency 16.948 ms 2024-09-11T01:08:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:10 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:08:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:10 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:08:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.954+0000 7f2f1d61e640 1 -- 172.21.15.78:0/597768536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f181076a0 msgr2=0x7f2f18107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.954+0000 7f2f1d61e640 1 --2- 172.21.15.78:0/597768536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f181076a0 0x7f2f18107ae0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009ca0 tx=0x7f2f0c02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.954+0000 7f2f1d61e640 1 -- 172.21.15.78:0/597768536 shutdown_connections 2024-09-11T01:08:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.954+0000 7f2f1d61e640 1 --2- 172.21.15.78:0/597768536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f18108020 0x7f2f1810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.954+0000 7f2f1d61e640 1 --2- 172.21.15.78:0/597768536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f181076a0 0x7f2f18107ae0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.954+0000 7f2f1d61e640 1 --2- 172.21.15.78:0/597768536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f18101c90 0x7f2f18102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.954+0000 7f2f1d61e640 1 -- 172.21.15.78:0/597768536 >> 172.21.15.78:0/597768536 conn(0x7f2f180fd700 msgr2=0x7f2f180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.954+0000 7f2f1d61e640 1 -- 172.21.15.78:0/597768536 shutdown_connections 2024-09-11T01:08:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.954+0000 7f2f1d61e640 1 -- 172.21.15.78:0/597768536 wait complete. 2024-09-11T01:08:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.954+0000 7f2f1d61e640 1 Processor -- start 2024-09-11T01:08:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f1d61e640 1 -- start start 2024-09-11T01:08:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f1d61e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f18101c90 0x7f2f180fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f1d61e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f181076a0 0x7f2f180fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f1d61e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18108020 0x7f2f180fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f1d61e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f18111c70 con 0x7f2f18108020 2024-09-11T01:08:10.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f1d61e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f18111af0 con 0x7f2f181076a0 2024-09-11T01:08:10.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f1d61e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f18111df0 con 0x7f2f18101c90 2024-09-11T01:08:10.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18108020 0x7f2f180fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18108020 0x7f2f180fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54246/0 (socket says 172.21.15.78:54246) 2024-09-11T01:08:10.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f167fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f181076a0 0x7f2f180fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f16ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f18101c90 0x7f2f180fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f177fe640 1 -- 172.21.15.78:0/4097943166 learned_addr learned my addr 172.21.15.78:0/4097943166 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f177fe640 1 -- 172.21.15.78:0/4097943166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f18101c90 msgr2=0x7f2f180fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f177fe640 1 --2- 172.21.15.78:0/4097943166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f18101c90 0x7f2f180fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f177fe640 1 -- 172.21.15.78:0/4097943166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f181076a0 msgr2=0x7f2f180fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f177fe640 1 --2- 172.21.15.78:0/4097943166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f181076a0 0x7f2f180fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.955+0000 7f2f177fe640 1 -- 172.21.15.78:0/4097943166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f181a76a0 con 0x7f2f18108020 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.956+0000 7f2f177fe640 1 --2- 172.21.15.78:0/4097943166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18108020 0x7f2f180fe680 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800e970 tx=0x7f2f0800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.956+0000 7f2efffff640 1 -- 172.21.15.78:0/4097943166 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800ccc0 con 0x7f2f18108020 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.956+0000 7f2efffff640 1 -- 172.21.15.78:0/4097943166 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f0800ce60 con 0x7f2f18108020 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.956+0000 7f2f1d61e640 1 -- 172.21.15.78:0/4097943166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f181a7990 con 0x7f2f18108020 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.956+0000 7f2efffff640 1 -- 172.21.15.78:0/4097943166 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08010430 con 0x7f2f18108020 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.956+0000 7f2f1d61e640 1 -- 172.21.15.78:0/4097943166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f181a7ed0 con 0x7f2f18108020 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.958+0000 7f2efffff640 1 -- 172.21.15.78:0/4097943166 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2f08004ff0 con 0x7f2f18108020 2024-09-11T01:08:10.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f1f9f3740 1 Processor -- start 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f1f9f3740 1 -- start start 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f1f9f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871e4d0 0x56103871e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f1f9f3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871edf0 0x56103872a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f1f9f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103872ad80 0x56103872d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f1f9f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561038694ea0 con 0x56103872ad80 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f1f9f3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5610386842d0 con 0x56103871edf0 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f1f9f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5610386880e0 con 0x56103871e4d0 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f167fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871e4d0 0x56103871e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f167fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871e4d0 0x56103871e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:50592/0 (socket says 172.21.15.78:50592) 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f16ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871edf0 0x56103872a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f167fc640 1 -- 172.21.15.78:0/729008972 learned_addr learned my addr 172.21.15.78:0/729008972 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2f167fc640 1 --2- 172.21.15.78:0/729008972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103872ad80 0x56103872d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.959+0000 7f2efffff640 1 --2- 172.21.15.78:0/4097943166 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2f000b54d0 0x7f2f000b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2f16ffd640 1 -- 172.21.15.78:0/729008972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871e4d0 msgr2=0x56103871e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:10.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/729008972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871e4d0 0x56103871e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2f16ffd640 1 -- 172.21.15.78:0/729008972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103872ad80 msgr2=0x56103872d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:10.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/729008972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103872ad80 0x56103872d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2f16ffd640 1 -- 172.21.15.78:0/729008972 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610383e2650 con 0x56103871edf0 2024-09-11T01:08:10.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2f177fe640 1 --2- 172.21.15.78:0/4097943166 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2f000b54d0 0x7f2f000b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2f167fc640 1 --2- 172.21.15.78:0/729008972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103872ad80 0x56103872d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:10.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2efffff640 1 -- 172.21.15.78:0/4097943166 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f2f080e6390 con 0x7f2f18108020 2024-09-11T01:08:10.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/729008972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871edf0 0x56103872a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009c20 tx=0x7f2f10011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c8026105085d128 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:10.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2efd7fa640 1 -- 172.21.15.78:0/729008972 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1001e070 con 0x56103871edf0 2024-09-11T01:08:10.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2f177fe640 1 --2- 172.21.15.78:0/4097943166 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2f000b54d0 0x7f2f000b7990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2f080a8da0 tx=0x7f2f080789b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:10.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2efd7fa640 1 -- 172.21.15.78:0/729008972 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f10009dd0 con 0x56103871edf0 2024-09-11T01:08:10.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.960+0000 7f2efd7fa640 1 -- 172.21.15.78:0/729008972 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10018770 con 0x56103871edf0 2024-09-11T01:08:10.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.961+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/729008972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871edf0 msgr2=0x56103872a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:10.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.961+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/729008972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871edf0 0x56103872a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009c20 tx=0x7f2f10011810 comp rx=0 tx=0).stop 2024-09-11T01:08:10.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.961+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/729008972 shutdown_connections 2024-09-11T01:08:10.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.961+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/729008972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103872ad80 0x56103872d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.961+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/729008972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871edf0 0x56103872a840 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.961+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/729008972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871e4d0 0x56103871e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.961+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/729008972 >> 172.21.15.78:0/729008972 conn(0x56103871c300 msgr2=0x56103871d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:10.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.961+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/729008972 shutdown_connections 2024-09-11T01:08:10.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.961+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/729008972 wait complete. 2024-09-11T01:08:10.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.961+0000 7f2f1f9f3740 1 -- start start 2024-09-11T01:08:10.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.961+0000 7f2f1f9f3740 1 Processor -- start 2024-09-11T01:08:10.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.961+0000 7f2f1f9f3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871e4d0 0x56103872a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f1f9f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871edf0 0x561038727a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f1f9f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103872ad80 0x561038728070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f1f9f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5610384e5700 con 0x56103872ad80 2024-09-11T01:08:10.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f1f9f3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56103861f160 con 0x56103871e4d0 2024-09-11T01:08:10.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f1f9f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56103872da00 con 0x56103871edf0 2024-09-11T01:08:10.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f1f9f3740 10 client.0 _ll_register_callbacks cb 0x5610386389e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:10.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f167fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103872ad80 0x561038728070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f167fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103872ad80 0x561038728070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54266/0 (socket says 172.21.15.78:54266) 2024-09-11T01:08:10.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f167fc640 1 -- 172.21.15.78:0/844183012 learned_addr learned my addr 172.21.15.78:0/844183012 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:10.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f167fc640 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871e4d0 0x56103872a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871edf0 0x561038727a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f167fc640 1 -- 172.21.15.78:0/844183012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871edf0 msgr2=0x561038727a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:10.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f167fc640 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871edf0 0x561038727a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f167fc640 1 -- 172.21.15.78:0/844183012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103872ad80 msgr2=0x561038728070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:10.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f167fc640 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103872ad80 0x561038728070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f167fc640 1 -- 172.21.15.78:0/844183012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56103864cc90 con 0x56103871e4d0 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f167fc640 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871e4d0 0x56103872a640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c005e30 tx=0x7f2f0c009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33f857dc1abfb233 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2efaff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2efaff5640 1 -- 172.21.15.78:0/844183012 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c031d00 con 0x56103871e4d0 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2efaff5640 1 -- 172.21.15.78:0/844183012 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f2f0c031450 con 0x56103871e4d0 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.962+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/844183012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f10009dd0 con 0x56103871e4d0 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2efaff5640 1 -- 172.21.15.78:0/844183012 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2f0c031650 con 0x56103871e4d0 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2f1f9f3740 10 client.26495 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/844183012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561038739330 con 0x56103871e4d0 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2efaff5640 1 -- 172.21.15.78:0/844183012 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f10009dd0 con 0x56103871e4d0 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2efaff5640 1 -- 172.21.15.78:0/844183012 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f2f0c054070 con 0x56103871e4d0 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2f1f9f3740 10 client.26495 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2f1f9f3740 10 client.26495 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2f1f9f3740 20 client.26495 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:10.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x56103873a740 0x56103873cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2f1f9f3740 4 client.26495 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:10.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/844183012 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561038696df0 con 0x56103873a740 2024-09-11T01:08:10.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x56103873a740 0x56103873cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2efaff5640 10 client.26495 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:08:10.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.963+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x56103873a740 0x56103873cb60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb2bc64ce1247c9f server_cookie=49ef6fa2e558008e in_seq=0 out_seq=0 2024-09-11T01:08:10.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2efaff5640 1 -- 172.21.15.78:0/844183012 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561038696df0 con 0x56103873a740 2024-09-11T01:08:10.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2efaff5640 10 client.26495 handle_command_reply: tid=0 2024-09-11T01:08:10.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2f1f9f3740 1 client.26495 shutdown 2024-09-11T01:08:10.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/844183012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871e4d0 msgr2=0x56103872a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:10.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871e4d0 0x56103872a640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c005e30 tx=0x7f2f0c009360 comp rx=0 tx=0).stop 2024-09-11T01:08:10.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/844183012 shutdown_connections 2024-09-11T01:08:10.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103872ad80 0x561038728070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871edf0 0x561038727a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x56103873a740 0x56103873cb60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/844183012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871e4d0 0x56103872a640 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2efaff5640 0 client.26495 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:08:10.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/844183012 >> 172.21.15.78:0/844183012 conn(0x56103871c300 msgr2=0x561038721e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:10.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/844183012 shutdown_connections 2024-09-11T01:08:10.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/844183012 wait complete. 2024-09-11T01:08:10.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.964+0000 7f2f1f9f3740 20 client.26495 trim_cache size 0 max 16384 2024-09-11T01:08:10.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.978+0000 7f2f1f9f3740 1 Processor -- start 2024-09-11T01:08:10.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.978+0000 7f2f1f9f3740 1 -- start start 2024-09-11T01:08:10.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.978+0000 7f2f1f9f3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871e4d0 0x561038736c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.978+0000 7f2f1f9f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 0x5610387297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.978+0000 7f2f1f9f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103872ad80 0x561038724ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:10.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.978+0000 7f2f1f9f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181b08c0 con 0x56103871edf0 2024-09-11T01:08:10.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.978+0000 7f2f1f9f3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181b2430 con 0x56103871e4d0 2024-09-11T01:08:10.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.978+0000 7f2f1f9f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181b0f90 con 0x56103872ad80 2024-09-11T01:08:10.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.978+0000 7f2f16ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 0x5610387297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2f167fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871e4d0 0x561038736c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2f16ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 0x5610387297d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54276/0 (socket says 172.21.15.78:54276) 2024-09-11T01:08:10.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2f16ffd640 1 -- 172.21.15.78:0/2663312817 learned_addr learned my addr 172.21.15.78:0/2663312817 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:10.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2f167fc640 1 --2- 172.21.15.78:0/2663312817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103872ad80 0x561038724ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:10.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2f16ffd640 1 -- 172.21.15.78:0/2663312817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103872ad80 msgr2=0x561038724ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:10.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/2663312817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103872ad80 0x561038724ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2f16ffd640 1 -- 172.21.15.78:0/2663312817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871e4d0 msgr2=0x561038736c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:10.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/2663312817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871e4d0 0x561038736c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:10.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2f16ffd640 1 -- 172.21.15.78:0/2663312817 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56103871d9c0 con 0x56103871edf0 2024-09-11T01:08:10.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/2663312817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 0x5610387297d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009c20 tx=0x7f2f10019f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60825cd69b3244be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:10.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2efcff9640 1 -- 172.21.15.78:0/2663312817 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10031070 con 0x56103871edf0 2024-09-11T01:08:10.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2efcff9640 1 -- 172.21.15.78:0/2663312817 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f10009430 con 0x56103871edf0 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.979+0000 7f2efcff9640 1 -- 172.21.15.78:0/2663312817 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1001e040 con 0x56103871edf0 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.980+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/2663312817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 msgr2=0x5610387297d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.980+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/2663312817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 0x5610387297d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009c20 tx=0x7f2f10019f20 comp rx=0 tx=0).stop 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.980+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/2663312817 shutdown_connections 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.980+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/2663312817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103872ad80 0x561038724ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.980+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/2663312817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 0x5610387297d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.980+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/2663312817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103871e4d0 0x561038736c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.980+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/2663312817 >> 172.21.15.78:0/2663312817 conn(0x56103871c300 msgr2=0x5610387290c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.980+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/2663312817 shutdown_connections 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.980+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/2663312817 wait complete. 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f1f9f3740 1 -- start start 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f1f9f3740 1 Processor -- start 2024-09-11T01:08:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f1f9f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871e4d0 0x56103872f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f1f9f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 0x56103872fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f1f9f3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103872ad80 0x5610387a0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f1f9f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181b1d30 con 0x56103871edf0 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f1f9f3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181b1660 con 0x56103872ad80 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f1f9f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181ae8c0 con 0x56103871e4d0 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f1f9f3740 10 client.0 _ll_register_callbacks cb 0x7f2f0c0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f16ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 0x56103872fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f16ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 0x56103872fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54290/0 (socket says 172.21.15.78:54290) 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f16ffd640 1 -- 172.21.15.78:0/227120910 learned_addr learned my addr 172.21.15.78:0/227120910 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f167fc640 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871e4d0 0x56103872f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f167fc640 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103872ad80 0x5610387a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f16ffd640 1 -- 172.21.15.78:0/227120910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871e4d0 msgr2=0x56103872f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871e4d0 0x56103872f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f16ffd640 1 -- 172.21.15.78:0/227120910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103872ad80 msgr2=0x5610387a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.981+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103872ad80 0x5610387a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2f16ffd640 1 -- 172.21.15.78:0/227120910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610387a0e90 con 0x56103871edf0 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 0x56103872fda0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2f100093a0 tx=0x7f2f10019c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672102e5b407f8de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2efaff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2efaff5640 1 -- 172.21.15.78:0/227120910 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1001e070 con 0x56103871edf0 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2efaff5640 1 -- 172.21.15.78:0/227120910 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f2f10005440 con 0x56103871edf0 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/227120910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f10009430 con 0x56103871edf0 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2efaff5640 1 -- 172.21.15.78:0/227120910 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2f10031040 con 0x56103871edf0 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2f1f9f3740 10 client.17253 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/227120910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5610387a82f0 con 0x56103871edf0 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2efaff5640 1 -- 172.21.15.78:0/227120910 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f10009430 con 0x56103871edf0 2024-09-11T01:08:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2efaff5640 1 -- 172.21.15.78:0/227120910 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f2f100266d0 con 0x56103871edf0 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2f1f9f3740 10 client.17253 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2f1f9f3740 10 client.17253 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2f1f9f3740 20 client.17253 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5610387a87d0 0x5610387aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2f1f9f3740 4 client.17253 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.982+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/227120910 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610386928a0 con 0x5610387a87d0 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f167fc640 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5610387a87d0 0x5610387aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2efaff5640 10 client.17253 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f167fc640 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5610387a87d0 0x5610387aabf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=827a957671ece77d server_cookie=31c92ac055079383 in_seq=0 out_seq=0 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2efaff5640 1 -- 172.21.15.78:0/227120910 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610386928a0 con 0x5610387a87d0 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2efaff5640 10 client.17253 handle_command_reply: tid=0 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f1f9f3740 1 client.17253 shutdown 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/227120910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 msgr2=0x56103872fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 0x56103872fda0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2f100093a0 tx=0x7f2f10019c30 comp rx=0 tx=0).stop 2024-09-11T01:08:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/227120910 shutdown_connections 2024-09-11T01:08:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5610387a87d0 0x5610387aabf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56103872ad80 0x5610387a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56103871edf0 0x56103872fda0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f1f9f3740 1 --2- 172.21.15.78:0/227120910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103871e4d0 0x56103872f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/227120910 >> 172.21.15.78:0/227120910 conn(0x56103871c300 msgr2=0x56103871b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2efaff5640 0 client.17253 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:08:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/227120910 shutdown_connections 2024-09-11T01:08:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.983+0000 7f2f1f9f3740 1 -- 172.21.15.78:0/227120910 wait complete. 2024-09-11T01:08:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.984+0000 7f2f1f9f3740 20 client.17253 trim_cache size 0 max 16384 2024-09-11T01:08:11.005 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:08:11.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.984+0000 7f2f1d61e640 1 -- 172.21.15.78:0/4097943166 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2f000b54d0 msgr2=0x7f2f000b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:11.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.984+0000 7f2f1d61e640 1 --2- 172.21.15.78:0/4097943166 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2f000b54d0 0x7f2f000b7990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2f080a8da0 tx=0x7f2f080789b0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.984+0000 7f2f1d61e640 1 -- 172.21.15.78:0/4097943166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18108020 msgr2=0x7f2f180fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.984+0000 7f2f1d61e640 1 --2- 172.21.15.78:0/4097943166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18108020 0x7f2f180fe680 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800e970 tx=0x7f2f0800ed70 comp rx=0 tx=0).stop 2024-09-11T01:08:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.984+0000 7f2f1d61e640 1 -- 172.21.15.78:0/4097943166 shutdown_connections 2024-09-11T01:08:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.984+0000 7f2f1d61e640 1 --2- 172.21.15.78:0/4097943166 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2f000b54d0 0x7f2f000b7990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.984+0000 7f2f1d61e640 1 --2- 172.21.15.78:0/4097943166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18108020 0x7f2f180fe680 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.984+0000 7f2f1d61e640 1 --2- 172.21.15.78:0/4097943166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f181076a0 0x7f2f180fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.984+0000 7f2f1d61e640 1 --2- 172.21.15.78:0/4097943166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f18101c90 0x7f2f180fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.984+0000 7f2f1d61e640 1 -- 172.21.15.78:0/4097943166 >> 172.21.15.78:0/4097943166 conn(0x7f2f180fd700 msgr2=0x7f2f18104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.984+0000 7f2f1d61e640 1 -- 172.21.15.78:0/4097943166 shutdown_connections 2024-09-11T01:08:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:10.985+0000 7f2f1d61e640 1 -- 172.21.15.78:0/4097943166 wait complete. 2024-09-11T01:08:11.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:11 smithi110 ceph-mon[34712]: pgmap v607: 97 pgs: 97 active+clean; 447 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-11T01:08:11.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:11 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:08:11.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:11 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:08:11.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:11 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:08:11.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 630309 5.35 MB/sec execute 379 sec latency 212.137 ms 2024-09-11T01:08:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:11 smithi112 ceph-mon[35384]: pgmap v607: 97 pgs: 97 active+clean; 447 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-11T01:08:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:11 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:08:11.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:11 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:08:11.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:11 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:08:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:11 smithi078 ceph-mon[31315]: pgmap v607: 97 pgs: 97 active+clean; 447 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-11T01:08:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:11 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:08:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:11 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:08:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:11 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:08:11.661 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:08:11.908 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:12.322 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:12 smithi078 ceph-mon[31315]: pgmap v608: 97 pgs: 97 active+clean; 449 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.9 MiB/s wr, 188 op/s 2024-09-11T01:08:12.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.401+0000 7f9c75af9640 1 -- 172.21.15.78:0/2179973636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c70109150 msgr2=0x7f9c7010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:12.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.401+0000 7f9c75af9640 1 --2- 172.21.15.78:0/2179973636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c70109150 0x7f9c7010f950 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009ca0 tx=0x7f9c6802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 -- 172.21.15.78:0/2179973636 shutdown_connections 2024-09-11T01:08:12.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 --2- 172.21.15.78:0/2179973636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c70109150 0x7f9c7010f950 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 --2- 172.21.15.78:0/2179973636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c701087d0 0x7f9c70108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 --2- 172.21.15.78:0/2179973636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c70102dc0 0x7f9c701031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 -- 172.21.15.78:0/2179973636 >> 172.21.15.78:0/2179973636 conn(0x7f9c700fe830 msgr2=0x7f9c70100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:12.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 -- 172.21.15.78:0/2179973636 shutdown_connections 2024-09-11T01:08:12.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 -- 172.21.15.78:0/2179973636 wait complete. 2024-09-11T01:08:12.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 Processor -- start 2024-09-11T01:08:12.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 -- start start 2024-09-11T01:08:12.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c70102dc0 0x7f9c70199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c701087d0 0x7f9c70199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c70109150 0x7f9c7019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c70112dd0 con 0x7f9c70102dc0 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9c70112c50 con 0x7f9c70109150 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c75af9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c70112f50 con 0x7f9c701087d0 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.402+0000 7f9c6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c70102dc0 0x7f9c70199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c70102dc0 0x7f9c70199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54304/0 (socket says 172.21.15.78:54304) 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/2732411497 learned_addr learned my addr 172.21.15.78:0/2732411497 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c67fff640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c701087d0 0x7f9c70199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c6ffff640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c70109150 0x7f9c7019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/2732411497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c701087d0 msgr2=0x7f9c70199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c701087d0 0x7f9c70199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/2732411497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c70109150 msgr2=0x7f9c7019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c70109150 0x7f9c7019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/2732411497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7019f2e0 con 0x7f9c70102dc0 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c70102dc0 0x7f9c70199230 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000e9f0 tx=0x7f9c6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c6d7fa640 1 -- 172.21.15.78:0/2732411497 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6000cd80 con 0x7f9c70102dc0 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c6d7fa640 1 -- 172.21.15.78:0/2732411497 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c600108e0 con 0x7f9c70102dc0 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c75af9640 1 -- 172.21.15.78:0/2732411497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7019f5d0 con 0x7f9c70102dc0 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c6d7fa640 1 -- 172.21.15.78:0/2732411497 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60018980 con 0x7f9c70102dc0 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.403+0000 7f9c75af9640 1 -- 172.21.15.78:0/2732411497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c701a5c90 con 0x7f9c70102dc0 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c6d7fa640 1 -- 172.21.15.78:0/2732411497 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9c60010430 con 0x7f9c70102dc0 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c77ece740 1 Processor -- start 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c77ece740 1 -- start start 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c77ece740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd00f0 0x55b36ddd04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c77ece740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 0x55b36dddc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c77ece740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 0x55b36ddded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c77ece740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b36dd46ea0 con 0x55b36ddd0a10 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c77ece740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b36dd362d0 con 0x55b36ddd00f0 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c77ece740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b36dd3a0e0 con 0x55b36dddc9a0 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 0x55b36dddc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 0x55b36dddc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54306/0 (socket says 172.21.15.78:54306) 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/307077786 learned_addr learned my addr 172.21.15.78:0/307077786 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c6ffff640 1 --2- 172.21.15.78:0/307077786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd00f0 0x55b36ddd04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c67fff640 1 --2- 172.21.15.78:0/307077786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 0x55b36ddded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/307077786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 msgr2=0x55b36ddded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.406+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/307077786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 0x55b36ddded90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.407+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/307077786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd00f0 msgr2=0x55b36ddd04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.407+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/307077786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd00f0 0x55b36ddd04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.407+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/307077786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b36da94650 con 0x55b36ddd0a10 2024-09-11T01:08:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.407+0000 7f9c6ffff640 1 --2- 172.21.15.78:0/307077786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd00f0 0x55b36ddd04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.407+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/307077786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 0x55b36dddc460 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9c600e3990 tx=0x7f9c600ee9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696fc2572ac2668b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.407+0000 7f9c6d7fa640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9c5c0b5450 0x7f9c5c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.407+0000 7f9c6ffff640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9c5c0b5450 0x7f9c5c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.407+0000 7f9c65ffb640 1 -- 172.21.15.78:0/307077786 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c600fa070 con 0x55b36ddd0a10 2024-09-11T01:08:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.407+0000 7f9c65ffb640 1 -- 172.21.15.78:0/307077786 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c600954a0 con 0x55b36ddd0a10 2024-09-11T01:08:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.407+0000 7f9c65ffb640 1 -- 172.21.15.78:0/307077786 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c600abf70 con 0x55b36ddd0a10 2024-09-11T01:08:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.407+0000 7f9c6d7fa640 1 -- 172.21.15.78:0/2732411497 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f9c600e62b0 con 0x7f9c70102dc0 2024-09-11T01:08:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.408+0000 7f9c77ece740 1 -- 172.21.15.78:0/307077786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 msgr2=0x55b36dddc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.408+0000 7f9c6ffff640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9c5c0b5450 0x7f9c5c0b7910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9c6802f500 tx=0x7f9c68005f50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.408+0000 7f9c77ece740 1 --2- 172.21.15.78:0/307077786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 0x55b36dddc460 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9c600e3990 tx=0x7f9c600ee9b0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.408+0000 7f9c77ece740 1 -- 172.21.15.78:0/307077786 shutdown_connections 2024-09-11T01:08:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.408+0000 7f9c77ece740 1 --2- 172.21.15.78:0/307077786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 0x55b36ddded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.408+0000 7f9c77ece740 1 --2- 172.21.15.78:0/307077786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 0x55b36dddc460 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.408+0000 7f9c77ece740 1 --2- 172.21.15.78:0/307077786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd00f0 0x55b36ddd04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.408+0000 7f9c77ece740 1 -- 172.21.15.78:0/307077786 >> 172.21.15.78:0/307077786 conn(0x55b36ddcdf20 msgr2=0x55b36ddcf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.408+0000 7f9c77ece740 1 -- 172.21.15.78:0/307077786 shutdown_connections 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.408+0000 7f9c77ece740 1 -- 172.21.15.78:0/307077786 wait complete. 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.408+0000 7f9c77ece740 1 -- start start 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.408+0000 7f9c77ece740 1 Processor -- start 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c77ece740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 0x55b36dddc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c77ece740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36ddd0a10 0x55b36ddd9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c77ece740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36dddc9a0 0x55b36ddd9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c77ece740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b36db97700 con 0x55b36ddd00f0 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c77ece740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b36dcd1160 con 0x55b36dddc9a0 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c77ece740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b36dddf620 con 0x55b36ddd0a10 2024-09-11T01:08:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c77ece740 10 client.0 _ll_register_callbacks cb 0x55b36dcea9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 0x55b36dddc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 0x55b36dddc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54314/0 (socket says 172.21.15.78:54314) 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/3974977104 learned_addr learned my addr 172.21.15.78:0/3974977104 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c67fff640 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36ddd0a10 0x55b36ddd9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c6ffff640 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36dddc9a0 0x55b36ddd9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/3974977104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36ddd0a10 msgr2=0x55b36ddd9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36ddd0a10 0x55b36ddd9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/3974977104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36dddc9a0 msgr2=0x55b36ddd9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36dddc9a0 0x55b36ddd9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/3974977104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b36dcfec90 con 0x55b36ddd00f0 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c6ffff640 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36dddc9a0 0x55b36ddd9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 0x55b36dddc240 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9c600a9a70 tx=0x7f9c60017f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ff24de53a14fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c4effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c4effd640 1 -- 172.21.15.78:0/3974977104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60016840 con 0x55b36ddd00f0 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c4effd640 1 -- 172.21.15.78:0/3974977104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f9c600e60b0 con 0x55b36ddd00f0 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.409+0000 7f9c77ece740 1 -- 172.21.15.78:0/3974977104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c600954a0 con 0x55b36ddd00f0 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.410+0000 7f9c4effd640 1 -- 172.21.15.78:0/3974977104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9c60016a20 con 0x55b36ddd00f0 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.410+0000 7f9c77ece740 10 client.17265 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.410+0000 7f9c77ece740 1 -- 172.21.15.78:0/3974977104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b36ddeaf40 con 0x55b36ddd00f0 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.410+0000 7f9c4effd640 1 -- 172.21.15.78:0/3974977104 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c600954a0 con 0x55b36ddd00f0 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.410+0000 7f9c4effd640 1 -- 172.21.15.78:0/3974977104 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f9c600a8a70 con 0x55b36ddd00f0 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.410+0000 7f9c77ece740 10 client.17265 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.410+0000 7f9c77ece740 10 client.17265 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.410+0000 7f9c77ece740 20 client.17265 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.410+0000 7f9c77ece740 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b36ddec290 0x55b36ddee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.410+0000 7f9c77ece740 4 client.17265 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.410+0000 7f9c77ece740 1 -- 172.21.15.78:0/3974977104 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b36dd48df0 con 0x55b36ddec290 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.410+0000 7f9c67fff640 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b36ddec290 0x55b36ddee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.411+0000 7f9c4effd640 10 client.17265 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.411+0000 7f9c67fff640 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b36ddec290 0x55b36ddee680 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9919e0707f56da9b server_cookie=c4bcd51bffa64863 in_seq=0 out_seq=0 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.411+0000 7f9c4effd640 1 -- 172.21.15.78:0/3974977104 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b36dd48df0 con 0x55b36ddec290 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.411+0000 7f9c4effd640 10 client.17265 handle_command_reply: tid=0 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.411+0000 7f9c77ece740 1 client.17265 shutdown 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.412+0000 7f9c77ece740 1 -- 172.21.15.78:0/3974977104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 msgr2=0x55b36dddc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.412+0000 7f9c77ece740 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 0x55b36dddc240 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9c600a9a70 tx=0x7f9c60017f70 comp rx=0 tx=0).stop 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.412+0000 7f9c77ece740 1 -- 172.21.15.78:0/3974977104 shutdown_connections 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.412+0000 7f9c77ece740 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36dddc9a0 0x55b36ddd9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.412+0000 7f9c77ece740 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36ddd0a10 0x55b36ddd9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.412+0000 7f9c77ece740 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b36ddec290 0x55b36ddee680 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.412+0000 7f9c77ece740 1 --2- 172.21.15.78:0/3974977104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 0x55b36dddc240 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.412+0000 7f9c4effd640 0 client.17265 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.412+0000 7f9c77ece740 1 -- 172.21.15.78:0/3974977104 >> 172.21.15.78:0/3974977104 conn(0x55b36ddcdf20 msgr2=0x55b36ddd3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.412+0000 7f9c77ece740 1 -- 172.21.15.78:0/3974977104 shutdown_connections 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.412+0000 7f9c77ece740 1 -- 172.21.15.78:0/3974977104 wait complete. 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.412+0000 7f9c77ece740 20 client.17265 trim_cache size 0 max 16384 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c77ece740 1 Processor -- start 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c77ece740 1 -- start start 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c77ece740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 0x55b36ddcef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c77ece740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd0a10 0x55b36ddcd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c77ece740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 0x55b36ddf1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c77ece740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c701ae630 con 0x55b36ddd00f0 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c77ece740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9c701b01a0 con 0x55b36ddd0a10 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c77ece740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c701aed00 con 0x55b36dddc9a0 2024-09-11T01:08:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 0x55b36ddcef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 0x55b36ddcef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54324/0 (socket says 172.21.15.78:54324) 2024-09-11T01:08:12.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/207367136 learned_addr learned my addr 172.21.15.78:0/207367136 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:12.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/207367136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 msgr2=0x55b36ddf1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:12.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/207367136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 0x55b36ddf1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/207367136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd0a10 msgr2=0x55b36ddcd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:12.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/207367136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd0a10 0x55b36ddcd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c6f7fe640 1 -- 172.21.15.78:0/207367136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b36ddd0840 con 0x55b36ddd00f0 2024-09-11T01:08:12.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/207367136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 0x55b36ddcef70 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9c60016dc0 tx=0x7f9c600ee9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c52b8f5a4b1a1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.425+0000 7f9c657fa640 1 -- 172.21.15.78:0/207367136 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c600e5070 con 0x55b36ddd00f0 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.426+0000 7f9c657fa640 1 -- 172.21.15.78:0/207367136 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c600e9bd0 con 0x55b36ddd00f0 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.426+0000 7f9c657fa640 1 -- 172.21.15.78:0/207367136 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60016c00 con 0x55b36ddd00f0 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.426+0000 7f9c77ece740 1 -- 172.21.15.78:0/207367136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 msgr2=0x55b36ddcef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.426+0000 7f9c77ece740 1 --2- 172.21.15.78:0/207367136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 0x55b36ddcef70 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9c60016dc0 tx=0x7f9c600ee9b0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.426+0000 7f9c77ece740 1 -- 172.21.15.78:0/207367136 shutdown_connections 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.426+0000 7f9c77ece740 1 --2- 172.21.15.78:0/207367136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 0x55b36ddf1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.426+0000 7f9c77ece740 1 --2- 172.21.15.78:0/207367136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd0a10 0x55b36ddcd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.426+0000 7f9c77ece740 1 --2- 172.21.15.78:0/207367136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd00f0 0x55b36ddcef70 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.426+0000 7f9c77ece740 1 -- 172.21.15.78:0/207367136 >> 172.21.15.78:0/207367136 conn(0x55b36ddcdf20 msgr2=0x55b36ddd7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.426+0000 7f9c77ece740 1 -- 172.21.15.78:0/207367136 shutdown_connections 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.426+0000 7f9c77ece740 1 -- 172.21.15.78:0/207367136 wait complete. 2024-09-11T01:08:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.426+0000 7f9c77ece740 1 -- start start 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c77ece740 1 Processor -- start 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c77ece740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd00f0 0x55b36de520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c77ece740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 0x55b36de52690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c77ece740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 0x55b36de561f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c77ece740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c701afaa0 con 0x55b36ddd0a10 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c77ece740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9c701af3d0 con 0x55b36ddd00f0 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c77ece740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c701ac630 con 0x55b36dddc9a0 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c77ece740 10 client.0 _ll_register_callbacks cb 0x7f9c600f8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c67fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 0x55b36de52690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c67fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 0x55b36de52690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54330/0 (socket says 172.21.15.78:54330) 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c67fff640 1 -- 172.21.15.78:0/3934006788 learned_addr learned my addr 172.21.15.78:0/3934006788 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd00f0 0x55b36de520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c67fff640 1 -- 172.21.15.78:0/3934006788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 msgr2=0x55b36de561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:12.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c67fff640 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 0x55b36de561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c67fff640 1 -- 172.21.15.78:0/3934006788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd00f0 msgr2=0x55b36de520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:12.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c67fff640 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd00f0 0x55b36de520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c67fff640 1 -- 172.21.15.78:0/3934006788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b36de56930 con 0x55b36ddd0a10 2024-09-11T01:08:12.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd00f0 0x55b36de520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:12.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c67fff640 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 0x55b36de52690 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9c58000c00 tx=0x7f9c580044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8441e1e1f1e211d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:12.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.427+0000 7f9c4effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:12.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.428+0000 7f9c4effd640 1 -- 172.21.15.78:0/3934006788 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c58025630 con 0x55b36ddd0a10 2024-09-11T01:08:12.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.428+0000 7f9c4effd640 1 -- 172.21.15.78:0/3934006788 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f9c580257d0 con 0x55b36ddd0a10 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.428+0000 7f9c77ece740 1 -- 172.21.15.78:0/3934006788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c600e9bd0 con 0x55b36ddd0a10 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.428+0000 7f9c4effd640 1 -- 172.21.15.78:0/3934006788 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9c58003e30 con 0x55b36ddd0a10 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.428+0000 7f9c77ece740 10 client.17271 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.428+0000 7f9c77ece740 1 -- 172.21.15.78:0/3934006788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b36de5de10 con 0x55b36ddd0a10 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.429+0000 7f9c4effd640 1 -- 172.21.15.78:0/3934006788 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c600e9bd0 con 0x55b36ddd0a10 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.429+0000 7f9c4effd640 1 -- 172.21.15.78:0/3934006788 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f9c58010c60 con 0x55b36ddd0a10 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.429+0000 7f9c77ece740 10 client.17271 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.429+0000 7f9c77ece740 10 client.17271 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.429+0000 7f9c77ece740 20 client.17271 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.429+0000 7f9c77ece740 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b36de5e5c0 0x55b36de609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.429+0000 7f9c77ece740 4 client.17271 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.429+0000 7f9c77ece740 1 -- 172.21.15.78:0/3934006788 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b36dd448a0 con 0x55b36de5e5c0 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.429+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b36de5e5c0 0x55b36de609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c6f7fe640 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b36de5e5c0 0x55b36de609e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1f721b767b46fe0 server_cookie=7281c473810ba196 in_seq=0 out_seq=0 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c4effd640 10 client.17271 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c4effd640 1 -- 172.21.15.78:0/3934006788 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b36dd448a0 con 0x55b36de5e5c0 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c4effd640 10 client.17271 handle_command_reply: tid=0 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c77ece740 1 client.17271 shutdown 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c77ece740 1 -- 172.21.15.78:0/3934006788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 msgr2=0x55b36de52690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c77ece740 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 0x55b36de52690 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9c58000c00 tx=0x7f9c580044d0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c77ece740 1 -- 172.21.15.78:0/3934006788 shutdown_connections 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c77ece740 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b36dddc9a0 0x55b36de561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c77ece740 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b36de5e5c0 0x55b36de609e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c77ece740 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36ddd0a10 0x55b36de52690 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c77ece740 1 --2- 172.21.15.78:0/3934006788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b36ddd00f0 0x55b36de520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c77ece740 1 -- 172.21.15.78:0/3934006788 >> 172.21.15.78:0/3934006788 conn(0x55b36ddcdf20 msgr2=0x55b36dddacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c4effd640 0 client.17271 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:08:12.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c77ece740 1 -- 172.21.15.78:0/3934006788 shutdown_connections 2024-09-11T01:08:12.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c77ece740 1 -- 172.21.15.78:0/3934006788 wait complete. 2024-09-11T01:08:12.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.430+0000 7f9c77ece740 20 client.17271 trim_cache size 0 max 16384 2024-09-11T01:08:12.449 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:08:12.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.431+0000 7f9c75af9640 1 -- 172.21.15.78:0/2732411497 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9c5c0b5450 msgr2=0x7f9c5c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:12.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.431+0000 7f9c75af9640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9c5c0b5450 0x7f9c5c0b7910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9c6802f500 tx=0x7f9c68005f50 comp rx=0 tx=0).stop 2024-09-11T01:08:12.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.431+0000 7f9c75af9640 1 -- 172.21.15.78:0/2732411497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c70102dc0 msgr2=0x7f9c70199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:12.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.431+0000 7f9c75af9640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c70102dc0 0x7f9c70199230 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000e9f0 tx=0x7f9c6000eeb0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.431+0000 7f9c75af9640 1 -- 172.21.15.78:0/2732411497 shutdown_connections 2024-09-11T01:08:12.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.431+0000 7f9c75af9640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9c5c0b5450 0x7f9c5c0b7910 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.431+0000 7f9c75af9640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c70109150 0x7f9c7019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.431+0000 7f9c75af9640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c701087d0 0x7f9c70199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.431+0000 7f9c75af9640 1 --2- 172.21.15.78:0/2732411497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c70102dc0 0x7f9c70199230 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:12.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.431+0000 7f9c75af9640 1 -- 172.21.15.78:0/2732411497 >> 172.21.15.78:0/2732411497 conn(0x7f9c700fe830 msgr2=0x7f9c700ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:12.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.431+0000 7f9c75af9640 1 -- 172.21.15.78:0/2732411497 shutdown_connections 2024-09-11T01:08:12.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:12.431+0000 7f9c75af9640 1 -- 172.21.15.78:0/2732411497 wait complete. 2024-09-11T01:08:12.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 630463 5.34 MB/sec execute 380 sec latency 960.328 ms 2024-09-11T01:08:12.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:12 smithi110 ceph-mon[34712]: pgmap v608: 97 pgs: 97 active+clean; 449 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.9 MiB/s wr, 188 op/s 2024-09-11T01:08:12.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:12 smithi112 ceph-mon[35384]: pgmap v608: 97 pgs: 97 active+clean; 449 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.9 MiB/s wr, 188 op/s 2024-09-11T01:08:13.181 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:08:13.431 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:13.445 INFO:tasks.workunit.client.0.smithi078.stdout: 1 630463 5.33 MB/sec execute 381 sec latency 1159.493 ms 2024-09-11T01:08:14.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.970+0000 7fb197a10640 1 -- 172.21.15.78:0/197674269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb190102920 msgr2=0x7fb190102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:14.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.970+0000 7fb197a10640 1 --2- 172.21.15.78:0/197674269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb190102920 0x7fb190102d00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb180009ca0 tx=0x7fb18002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.970+0000 7fb197a10640 1 -- 172.21.15.78:0/197674269 shutdown_connections 2024-09-11T01:08:14.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.970+0000 7fb197a10640 1 --2- 172.21.15.78:0/197674269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb190108120 0x7fb19010e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 --2- 172.21.15.78:0/197674269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb190107780 0x7fb190107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 --2- 172.21.15.78:0/197674269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb190102920 0x7fb190102d00 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 -- 172.21.15.78:0/197674269 >> 172.21.15.78:0/197674269 conn(0x7fb190076f50 msgr2=0x7fb190077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:14.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 -- 172.21.15.78:0/197674269 shutdown_connections 2024-09-11T01:08:14.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 -- 172.21.15.78:0/197674269 wait complete. 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 Processor -- start 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 -- start start 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb190102920 0x7fb19019b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb190107780 0x7fb19019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb190108120 0x7fb1901a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb190111d40 con 0x7fb190107780 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb190111bc0 con 0x7fb190108120 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.971+0000 7fb197a10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb190111ec0 con 0x7fb190102920 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb194fab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb190107780 0x7fb19019bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb194fab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb190107780 0x7fb19019bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41278/0 (socket says 172.21.15.78:41278) 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb194fab640 1 -- 172.21.15.78:0/823200998 learned_addr learned my addr 172.21.15.78:0/823200998 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb1957ac640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb190102920 0x7fb19019b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb195fad640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb190108120 0x7fb1901a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb194fab640 1 -- 172.21.15.78:0/823200998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb190102920 msgr2=0x7fb19019b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb194fab640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb190102920 0x7fb19019b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb194fab640 1 -- 172.21.15.78:0/823200998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb190108120 msgr2=0x7fb1901a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb194fab640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb190108120 0x7fb1901a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb194fab640 1 -- 172.21.15.78:0/823200998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1901a16c0 con 0x7fb190107780 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb1957ac640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb190102920 0x7fb19019b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb195fad640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb190108120 0x7fb1901a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb194fab640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb190107780 0x7fb19019bb50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb18400b4a0 tx=0x7fb18400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb18e7fc640 1 -- 172.21.15.78:0/823200998 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184014070 con 0x7fb190107780 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb197a10640 1 -- 172.21.15.78:0/823200998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1901a19b0 con 0x7fb190107780 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb18e7fc640 1 -- 172.21.15.78:0/823200998 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb184010ac0 con 0x7fb190107780 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb18e7fc640 1 -- 172.21.15.78:0/823200998 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184017c30 con 0x7fb190107780 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.972+0000 7fb197a10640 1 -- 172.21.15.78:0/823200998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1901a80f0 con 0x7fb190107780 2024-09-11T01:08:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.975+0000 7fb18e7fc640 1 -- 172.21.15.78:0/823200998 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb184010610 con 0x7fb190107780 2024-09-11T01:08:14.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.975+0000 7fb199de5740 1 Processor -- start 2024-09-11T01:08:14.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.975+0000 7fb199de5740 1 -- start start 2024-09-11T01:08:14.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.975+0000 7fb199de5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x55757180a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.975+0000 7fb199de5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757180aa10 0x557571816460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.975+0000 7fb199de5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 0x557571818d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.975+0000 7fb199de5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557571780ea0 con 0x5575718169a0 2024-09-11T01:08:14.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.975+0000 7fb199de5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5575717702d0 con 0x55757180a0f0 2024-09-11T01:08:14.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.975+0000 7fb199de5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5575717740e0 con 0x55757180aa10 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.975+0000 7fb194fab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 0x557571818d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.975+0000 7fb194fab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 0x557571818d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41294/0 (socket says 172.21.15.78:41294) 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.975+0000 7fb194fab640 1 -- 172.21.15.78:0/2736804289 learned_addr learned my addr 172.21.15.78:0/2736804289 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb195fad640 1 --2- 172.21.15.78:0/2736804289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x55757180a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb1957ac640 1 --2- 172.21.15.78:0/2736804289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757180aa10 0x557571816460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb194fab640 1 -- 172.21.15.78:0/2736804289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757180aa10 msgr2=0x557571816460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb194fab640 1 --2- 172.21.15.78:0/2736804289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757180aa10 0x557571816460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb194fab640 1 -- 172.21.15.78:0/2736804289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 msgr2=0x55757180a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb18e7fc640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb1780b5450 0x7fb1780b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb194fab640 1 --2- 172.21.15.78:0/2736804289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x55757180a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb194fab640 1 -- 172.21.15.78:0/2736804289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575714ce650 con 0x5575718169a0 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb195fad640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb1780b5450 0x7fb1780b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb194fab640 1 --2- 172.21.15.78:0/2736804289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 0x557571818d90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb1840e2ce0 tx=0x7fb1840a81e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2546e8848f7bad56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb18e7fc640 1 -- 172.21.15.78:0/823200998 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7fb1840e5670 con 0x7fb190107780 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb1777fe640 1 -- 172.21.15.78:0/2736804289 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18401e020 con 0x5575718169a0 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb1777fe640 1 -- 172.21.15.78:0/2736804289 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1840abd30 con 0x5575718169a0 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.976+0000 7fb1777fe640 1 -- 172.21.15.78:0/2736804289 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1840ab420 con 0x5575718169a0 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.977+0000 7fb199de5740 1 -- 172.21.15.78:0/2736804289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 msgr2=0x557571818d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.977+0000 7fb195fad640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb1780b5450 0x7fb1780b7910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb18800aa40 tx=0x7fb1880047d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.977+0000 7fb199de5740 1 --2- 172.21.15.78:0/2736804289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 0x557571818d90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb1840e2ce0 tx=0x7fb1840a81e0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.977+0000 7fb199de5740 1 -- 172.21.15.78:0/2736804289 shutdown_connections 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.977+0000 7fb199de5740 1 --2- 172.21.15.78:0/2736804289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 0x557571818d90 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.977+0000 7fb199de5740 1 --2- 172.21.15.78:0/2736804289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757180aa10 0x557571816460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.977+0000 7fb199de5740 1 --2- 172.21.15.78:0/2736804289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x55757180a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.977+0000 7fb199de5740 1 -- 172.21.15.78:0/2736804289 >> 172.21.15.78:0/2736804289 conn(0x557571807f20 msgr2=0x5575718090c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.977+0000 7fb199de5740 1 -- 172.21.15.78:0/2736804289 shutdown_connections 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.977+0000 7fb199de5740 1 -- 172.21.15.78:0/2736804289 wait complete. 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.977+0000 7fb199de5740 1 -- start start 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.977+0000 7fb199de5740 1 Processor -- start 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb199de5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757180a0f0 0x557571816250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb199de5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180aa10 0x5575718136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb199de5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 0x557571813c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb199de5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5575715d1700 con 0x5575718169a0 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb199de5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55757170b160 con 0x55757180aa10 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb199de5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557571819620 con 0x55757180a0f0 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb199de5740 10 client.0 _ll_register_callbacks cb 0x5575717249e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb195fad640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 0x557571813c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb195fad640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 0x557571813c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41296/0 (socket says 172.21.15.78:41296) 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb195fad640 1 -- 172.21.15.78:0/2999599626 learned_addr learned my addr 172.21.15.78:0/2999599626 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb1957ac640 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757180a0f0 0x557571816250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb194fab640 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180aa10 0x5575718136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb195fad640 1 -- 172.21.15.78:0/2999599626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757180a0f0 msgr2=0x557571816250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb195fad640 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757180a0f0 0x557571816250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb195fad640 1 -- 172.21.15.78:0/2999599626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180aa10 msgr2=0x5575718136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb195fad640 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180aa10 0x5575718136a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb195fad640 1 -- 172.21.15.78:0/2999599626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557571738c90 con 0x5575718169a0 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb195fad640 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 0x557571813c60 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb188014670 tx=0x7fb188008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2caca1d940edd53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.978+0000 7fb174ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb174ff9640 1 -- 172.21.15.78:0/2999599626 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188024020 con 0x5575718169a0 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb174ff9640 1 -- 172.21.15.78:0/2999599626 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7fb188013940 con 0x5575718169a0 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb199de5740 1 -- 172.21.15.78:0/2999599626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557571782df0 con 0x5575718169a0 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb174ff9640 1 -- 172.21.15.78:0/2999599626 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb18802f050 con 0x5575718169a0 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb199de5740 10 client.17283 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:14.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb199de5740 1 -- 172.21.15.78:0/2999599626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557571824fa0 con 0x5575718169a0 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb174ff9640 1 -- 172.21.15.78:0/2999599626 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557571782df0 con 0x5575718169a0 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb174ff9640 1 -- 172.21.15.78:0/2999599626 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7fb188020c80 con 0x5575718169a0 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb199de5740 10 client.17283 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb199de5740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb199de5740 20 client.17283 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb199de5740 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557571826500 0x557571828930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb199de5740 4 client.17283 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.979+0000 7fb199de5740 1 -- 172.21.15.78:0/2999599626 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55757177e8a0 con 0x557571826500 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.980+0000 7fb1957ac640 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557571826500 0x557571828930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.980+0000 7fb174ff9640 10 client.17283 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.980+0000 7fb1957ac640 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557571826500 0x557571828930 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd90b4dad8642e89 server_cookie=f42684a94908e212 in_seq=0 out_seq=0 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb174ff9640 1 -- 172.21.15.78:0/2999599626 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55757177e8a0 con 0x557571826500 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb174ff9640 10 client.17283 handle_command_reply: tid=0 2024-09-11T01:08:14.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb199de5740 1 client.17283 shutdown 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb199de5740 1 -- 172.21.15.78:0/2999599626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 msgr2=0x557571813c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb199de5740 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 0x557571813c60 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb188014670 tx=0x7fb188008040 comp rx=0 tx=0).stop 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb199de5740 1 -- 172.21.15.78:0/2999599626 shutdown_connections 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb199de5740 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557571826500 0x557571828930 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb199de5740 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575718169a0 0x557571813c60 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb199de5740 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180aa10 0x5575718136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb174ff9640 0 client.17283 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb199de5740 1 --2- 172.21.15.78:0/2999599626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757180a0f0 0x557571816250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb199de5740 1 -- 172.21.15.78:0/2999599626 >> 172.21.15.78:0/2999599626 conn(0x557571807f20 msgr2=0x55757180da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb199de5740 1 -- 172.21.15.78:0/2999599626 shutdown_connections 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb199de5740 1 -- 172.21.15.78:0/2999599626 wait complete. 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.981+0000 7fb199de5740 20 client.17283 trim_cache size 0 max 16384 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb199de5740 1 Processor -- start 2024-09-11T01:08:14.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb199de5740 1 -- start start 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb199de5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x557571822880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb199de5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 0x5575718153e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb199de5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575718169a0 0x557571817d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb199de5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901b09d0 con 0x55757180aa10 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb199de5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901b2540 con 0x55757180a0f0 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb199de5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901b10a0 con 0x5575718169a0 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb194fab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 0x5575718153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb194fab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 0x5575718153e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41304/0 (socket says 172.21.15.78:41304) 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb194fab640 1 -- 172.21.15.78:0/1273490447 learned_addr learned my addr 172.21.15.78:0/1273490447 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb194fab640 1 -- 172.21.15.78:0/1273490447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575718169a0 msgr2=0x557571817d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb194fab640 1 --2- 172.21.15.78:0/1273490447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575718169a0 0x557571817d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb194fab640 1 -- 172.21.15.78:0/1273490447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 msgr2=0x557571822880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:14.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb1957ac640 1 --2- 172.21.15.78:0/1273490447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x557571822880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb194fab640 1 --2- 172.21.15.78:0/1273490447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x557571822880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb194fab640 1 -- 172.21.15.78:0/1273490447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757180a840 con 0x55757180aa10 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.994+0000 7fb1957ac640 1 --2- 172.21.15.78:0/1273490447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x557571822880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb194fab640 1 --2- 172.21.15.78:0/1273490447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 0x5575718153e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb18400b470 tx=0x7fb1840e2d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b1d92c918df913 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb176ffd640 1 -- 172.21.15.78:0/1273490447 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18401e070 con 0x55757180aa10 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb176ffd640 1 -- 172.21.15.78:0/1273490447 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb184017360 con 0x55757180aa10 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb176ffd640 1 -- 172.21.15.78:0/1273490447 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1840092b0 con 0x55757180aa10 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb199de5740 1 -- 172.21.15.78:0/1273490447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 msgr2=0x5575718153e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb199de5740 1 --2- 172.21.15.78:0/1273490447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 0x5575718153e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb18400b470 tx=0x7fb1840e2d40 comp rx=0 tx=0).stop 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb199de5740 1 -- 172.21.15.78:0/1273490447 shutdown_connections 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb199de5740 1 --2- 172.21.15.78:0/1273490447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575718169a0 0x557571817d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb199de5740 1 --2- 172.21.15.78:0/1273490447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 0x5575718153e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb199de5740 1 --2- 172.21.15.78:0/1273490447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x557571822880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb199de5740 1 -- 172.21.15.78:0/1273490447 >> 172.21.15.78:0/1273490447 conn(0x557571807f20 msgr2=0x557571814cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb199de5740 1 -- 172.21.15.78:0/1273490447 shutdown_connections 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.995+0000 7fb199de5740 1 -- 172.21.15.78:0/1273490447 wait complete. 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb199de5740 1 -- start start 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb199de5740 1 Processor -- start 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb199de5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x55757181ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb199de5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 0x55757181f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb199de5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575718169a0 0x5575718198a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb199de5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901b1e40 con 0x55757180aa10 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb199de5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901b1770 con 0x55757180a0f0 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb199de5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901ae9d0 con 0x5575718169a0 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb199de5740 10 client.0 _ll_register_callbacks cb 0x7fb188010cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb194fab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 0x55757181f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb194fab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 0x55757181f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41310/0 (socket says 172.21.15.78:41310) 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb194fab640 1 -- 172.21.15.78:0/3784137983 learned_addr learned my addr 172.21.15.78:0/3784137983 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb195fad640 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575718169a0 0x5575718198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb194fab640 1 -- 172.21.15.78:0/3784137983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575718169a0 msgr2=0x5575718198a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb194fab640 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575718169a0 0x5575718198a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb194fab640 1 -- 172.21.15.78:0/3784137983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 msgr2=0x55757181ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb1957ac640 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x55757181ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb194fab640 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x55757181ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.996+0000 7fb194fab640 1 -- 172.21.15.78:0/3784137983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557571819fe0 con 0x55757180aa10 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.997+0000 7fb195fad640 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575718169a0 0x5575718198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.997+0000 7fb194fab640 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 0x55757181f010 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb1840e2e00 tx=0x7fb184017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e113c7902eecd10 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.997+0000 7fb174ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.997+0000 7fb174ff9640 1 -- 172.21.15.78:0/3784137983 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18401e070 con 0x55757180aa10 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.997+0000 7fb174ff9640 1 -- 172.21.15.78:0/3784137983 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7fb184010c70 con 0x55757180aa10 2024-09-11T01:08:14.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.997+0000 7fb199de5740 1 -- 172.21.15.78:0/3784137983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55757177bcd0 con 0x55757180aa10 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.997+0000 7fb174ff9640 1 -- 172.21.15.78:0/3784137983 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb1840ab240 con 0x55757180aa10 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.997+0000 7fb199de5740 10 client.17289 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.997+0000 7fb199de5740 1 -- 172.21.15.78:0/3784137983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557571893db0 con 0x55757180aa10 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.997+0000 7fb174ff9640 1 -- 172.21.15.78:0/3784137983 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55757177bcd0 con 0x55757180aa10 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.998+0000 7fb174ff9640 1 -- 172.21.15.78:0/3784137983 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7fb1840a7d40 con 0x55757180aa10 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.998+0000 7fb199de5740 10 client.17289 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.998+0000 7fb199de5740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.998+0000 7fb199de5740 20 client.17289 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.998+0000 7fb199de5740 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557571894030 0x557571896420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.998+0000 7fb199de5740 4 client.17289 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.998+0000 7fb199de5740 1 -- 172.21.15.78:0/3784137983 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575716f1aa0 con 0x557571894030 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.998+0000 7fb1957ac640 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557571894030 0x557571896420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:14.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb1957ac640 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557571894030 0x557571896420 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5440b3c176daa0e server_cookie=760489e66fe6289 in_seq=0 out_seq=0 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb174ff9640 10 client.17289 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb174ff9640 1 -- 172.21.15.78:0/3784137983 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575716f1aa0 con 0x557571894030 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb174ff9640 10 client.17289 handle_command_reply: tid=0 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb199de5740 1 client.17289 shutdown 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb199de5740 1 -- 172.21.15.78:0/3784137983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 msgr2=0x55757181f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb199de5740 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 0x55757181f010 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb1840e2e00 tx=0x7fb184017650 comp rx=0 tx=0).stop 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb199de5740 1 -- 172.21.15.78:0/3784137983 shutdown_connections 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb199de5740 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575718169a0 0x5575718198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb199de5740 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x557571894030 0x557571896420 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb199de5740 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55757180aa10 0x55757181f010 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb199de5740 1 --2- 172.21.15.78:0/3784137983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55757180a0f0 0x55757181ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb174ff9640 0 client.17289 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb199de5740 1 -- 172.21.15.78:0/3784137983 >> 172.21.15.78:0/3784137983 conn(0x557571807f20 msgr2=0x5575718154f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:14.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb199de5740 1 -- 172.21.15.78:0/3784137983 shutdown_connections 2024-09-11T01:08:14.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb199de5740 1 -- 172.21.15.78:0/3784137983 wait complete. 2024-09-11T01:08:14.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:13.999+0000 7fb199de5740 20 client.17289 trim_cache size 0 max 16384 2024-09-11T01:08:14.018 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:08:14.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:14.000+0000 7fb197a10640 1 -- 172.21.15.78:0/823200998 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb1780b5450 msgr2=0x7fb1780b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:14.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:14.000+0000 7fb197a10640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb1780b5450 0x7fb1780b7910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb18800aa40 tx=0x7fb1880047d0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:14.000+0000 7fb197a10640 1 -- 172.21.15.78:0/823200998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb190107780 msgr2=0x7fb19019bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:14.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:14.000+0000 7fb197a10640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb190107780 0x7fb19019bb50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb18400b4a0 tx=0x7fb18400b960 comp rx=0 tx=0).stop 2024-09-11T01:08:14.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:14.000+0000 7fb197a10640 1 -- 172.21.15.78:0/823200998 shutdown_connections 2024-09-11T01:08:14.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:14.000+0000 7fb197a10640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb1780b5450 0x7fb1780b7910 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:14.000+0000 7fb197a10640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb190108120 0x7fb1901a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:14.000+0000 7fb197a10640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb190107780 0x7fb19019bb50 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:14.000+0000 7fb197a10640 1 --2- 172.21.15.78:0/823200998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb190102920 0x7fb19019b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:14.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:14.000+0000 7fb197a10640 1 -- 172.21.15.78:0/823200998 >> 172.21.15.78:0/823200998 conn(0x7fb190076f50 msgr2=0x7fb19010cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:14.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:14.000+0000 7fb197a10640 1 -- 172.21.15.78:0/823200998 shutdown_connections 2024-09-11T01:08:14.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:14.000+0000 7fb197a10640 1 -- 172.21.15.78:0/823200998 wait complete. 2024-09-11T01:08:14.446 INFO:tasks.workunit.client.0.smithi078.stdout: 1 630463 5.31 MB/sec execute 382 sec latency 2159.569 ms 2024-09-11T01:08:14.591 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:14 smithi078 ceph-mon[31315]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T01:08:14.591 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:14 smithi078 ceph-mon[31315]: pgmap v610: 97 pgs: 97 active+clean; 454 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 168 op/s 2024-09-11T01:08:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:14 smithi110 ceph-mon[34712]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T01:08:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:14 smithi110 ceph-mon[34712]: pgmap v610: 97 pgs: 97 active+clean; 454 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 168 op/s 2024-09-11T01:08:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:14 smithi112 ceph-mon[35384]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T01:08:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:14 smithi112 ceph-mon[35384]: pgmap v610: 97 pgs: 97 active+clean; 454 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 168 op/s 2024-09-11T01:08:15.446 INFO:tasks.workunit.client.0.smithi078.stdout: 1 630463 5.30 MB/sec execute 383 sec latency 3159.647 ms 2024-09-11T01:08:15.751 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:08:15.752 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:08:15.999 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:16.437 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:16 smithi078 ceph-mon[31315]: pgmap v611: 97 pgs: 97 active+clean; 455 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 163 op/s 2024-09-11T01:08:16.446 INFO:tasks.workunit.client.0.smithi078.stdout: 1 630463 5.28 MB/sec execute 384 sec latency 4159.690 ms 2024-09-11T01:08:16.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.522+0000 7f939eeb9640 1 -- 172.21.15.78:0/960611973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9398104550 msgr2=0x7f939810cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.522+0000 7f939eeb9640 1 --2- 172.21.15.78:0/960611973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9398104550 0x7f939810cb30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9384009900 tx=0x7f938402a410 comp rx=0 tx=0).stop 2024-09-11T01:08:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.522+0000 7f939eeb9640 1 -- 172.21.15.78:0/960611973 shutdown_connections 2024-09-11T01:08:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.522+0000 7f939eeb9640 1 --2- 172.21.15.78:0/960611973 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939810d100 0x7f939810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.522+0000 7f939eeb9640 1 --2- 172.21.15.78:0/960611973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9398104550 0x7f939810cb30 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.522+0000 7f939eeb9640 1 --2- 172.21.15.78:0/960611973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398103b50 0x7f9398103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.522+0000 7f939eeb9640 1 -- 172.21.15.78:0/960611973 >> 172.21.15.78:0/960611973 conn(0x7f93980fd700 msgr2=0x7f93980ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.522+0000 7f939eeb9640 1 -- 172.21.15.78:0/960611973 shutdown_connections 2024-09-11T01:08:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.522+0000 7f939eeb9640 1 -- 172.21.15.78:0/960611973 wait complete. 2024-09-11T01:08:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939eeb9640 1 Processor -- start 2024-09-11T01:08:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939eeb9640 1 -- start start 2024-09-11T01:08:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939eeb9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9398103b50 0x7f93981019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939eeb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398104550 0x7f9398101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939eeb9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939810d100 0x7f9398102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939eeb9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9398112d60 con 0x7f9398103b50 2024-09-11T01:08:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939eeb9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9398112be0 con 0x7f939810d100 2024-09-11T01:08:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939eeb9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9398112ee0 con 0x7f9398104550 2024-09-11T01:08:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939cc55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9398103b50 0x7f93981019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939cc55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9398103b50 0x7f93981019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41318/0 (socket says 172.21.15.78:41318) 2024-09-11T01:08:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939cc55640 1 -- 172.21.15.78:0/3976841038 learned_addr learned my addr 172.21.15.78:0/3976841038 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f9397fff640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398104550 0x7f9398101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939d456640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939810d100 0x7f9398102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939cc55640 1 -- 172.21.15.78:0/3976841038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398104550 msgr2=0x7f9398101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939cc55640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398104550 0x7f9398101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939cc55640 1 -- 172.21.15.78:0/3976841038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939810d100 msgr2=0x7f9398102450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939cc55640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939810d100 0x7f9398102450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939cc55640 1 -- 172.21.15.78:0/3976841038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93981a7690 con 0x7f9398103b50 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.523+0000 7f939d456640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939810d100 0x7f9398102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.524+0000 7f939cc55640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9398103b50 0x7f93981019d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f938c00d950 tx=0x7f938c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.524+0000 7f9395ffb640 1 -- 172.21.15.78:0/3976841038 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c017070 con 0x7f9398103b50 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.524+0000 7f939eeb9640 1 -- 172.21.15.78:0/3976841038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93981a7980 con 0x7f9398103b50 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.524+0000 7f9395ffb640 1 -- 172.21.15.78:0/3976841038 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c009c30 con 0x7f9398103b50 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.524+0000 7f9395ffb640 1 -- 172.21.15.78:0/3976841038 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c020060 con 0x7f9398103b50 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.524+0000 7f939eeb9640 1 -- 172.21.15.78:0/3976841038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93981a7ee0 con 0x7f9398103b50 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.526+0000 7f9395ffb640 1 -- 172.21.15.78:0/3976841038 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f938c023060 con 0x7f9398103b50 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f93a128e740 1 Processor -- start 2024-09-11T01:08:16.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f93a128e740 1 -- start start 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f93a128e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065aee0 0x56268065b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f93a128e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065b800 0x562680667250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f93a128e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562680667790 0x562680669b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f93a128e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5626805d1e60 con 0x56268065b800 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f93a128e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5626805c1290 con 0x56268065aee0 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f93a128e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5626805c50a0 con 0x562680667790 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f939cc55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065b800 0x562680667250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f939cc55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065b800 0x562680667250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41330/0 (socket says 172.21.15.78:41330) 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f939cc55640 1 -- 172.21.15.78:0/2274805118 learned_addr learned my addr 172.21.15.78:0/2274805118 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f9395ffb640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93880b5450 0x7f93880b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.527+0000 7f9397fff640 1 --2- 172.21.15.78:0/2274805118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562680667790 0x562680669b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f939d456640 1 --2- 172.21.15.78:0/2274805118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065aee0 0x56268065b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f939d456640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93880b5450 0x7f93880b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f9395ffb640 1 -- 172.21.15.78:0/3976841038 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f938c0e5550 con 0x7f9398103b50 2024-09-11T01:08:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f939cc55640 1 -- 172.21.15.78:0/2274805118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562680667790 msgr2=0x562680669b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f939cc55640 1 --2- 172.21.15.78:0/2274805118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562680667790 0x562680669b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f939cc55640 1 -- 172.21.15.78:0/2274805118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065aee0 msgr2=0x56268065b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f939cc55640 1 --2- 172.21.15.78:0/2274805118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065aee0 0x56268065b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f939cc55640 1 -- 172.21.15.78:0/2274805118 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268031f930 con 0x56268065b800 2024-09-11T01:08:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f9397fff640 1 --2- 172.21.15.78:0/2274805118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562680667790 0x562680669b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f939d456640 1 --2- 172.21.15.78:0/2274805118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065aee0 0x56268065b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f939cc55640 1 --2- 172.21.15.78:0/2274805118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065b800 0x562680667250 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f938c0e2c20 tx=0x7f938c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb6c48c076c3b77 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f9382ffd640 1 -- 172.21.15.78:0/2274805118 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0ab360 con 0x56268065b800 2024-09-11T01:08:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f939d456640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93880b5450 0x7f93880b7910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9390005c40 tx=0x7f9390009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f9382ffd640 1 -- 172.21.15.78:0/2274805118 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c009de0 con 0x56268065b800 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.528+0000 7f9382ffd640 1 -- 172.21.15.78:0/2274805118 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0ab180 con 0x56268065b800 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.529+0000 7f93a128e740 1 -- 172.21.15.78:0/2274805118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065b800 msgr2=0x562680667250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.529+0000 7f93a128e740 1 --2- 172.21.15.78:0/2274805118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065b800 0x562680667250 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f938c0e2c20 tx=0x7f938c000bc0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.529+0000 7f93a128e740 1 -- 172.21.15.78:0/2274805118 shutdown_connections 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.529+0000 7f93a128e740 1 --2- 172.21.15.78:0/2274805118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562680667790 0x562680669b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.529+0000 7f93a128e740 1 --2- 172.21.15.78:0/2274805118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065b800 0x562680667250 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.529+0000 7f93a128e740 1 --2- 172.21.15.78:0/2274805118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065aee0 0x56268065b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.529+0000 7f93a128e740 1 -- 172.21.15.78:0/2274805118 >> 172.21.15.78:0/2274805118 conn(0x562680658d10 msgr2=0x562680659eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.529+0000 7f93a128e740 1 -- 172.21.15.78:0/2274805118 shutdown_connections 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.529+0000 7f93a128e740 1 -- 172.21.15.78:0/2274805118 wait complete. 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.529+0000 7f93a128e740 1 -- start start 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.529+0000 7f93a128e740 1 Processor -- start 2024-09-11T01:08:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f93a128e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065aee0 0x562680664740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f93a128e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065b800 0x562680664c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f93a128e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 0x5626806651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f93a128e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562680422890 con 0x562680667790 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f93a128e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56268055bd40 con 0x56268065aee0 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f93a128e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56268066a380 con 0x56268065b800 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f93a128e740 10 client.0 _ll_register_callbacks cb 0x5626805bfc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f939d456640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 0x5626806651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f939d456640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 0x5626806651c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41346/0 (socket says 172.21.15.78:41346) 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f939d456640 1 -- 172.21.15.78:0/1253418874 learned_addr learned my addr 172.21.15.78:0/1253418874 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f939cc55640 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065aee0 0x562680664740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f9397fff640 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065b800 0x562680664c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f939d456640 1 -- 172.21.15.78:0/1253418874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065b800 msgr2=0x562680664c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f939d456640 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065b800 0x562680664c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f939d456640 1 -- 172.21.15.78:0/1253418874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065aee0 msgr2=0x562680664740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f939d456640 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065aee0 0x562680664740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.530+0000 7f939d456640 1 -- 172.21.15.78:0/1253418874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562680673760 con 0x562680667790 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f939d456640 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 0x5626806651c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f939000edf0 tx=0x7f9390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7423e307fe76 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f9377fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f9377fff640 1 -- 172.21.15.78:0/1253418874 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390018070 con 0x562680667790 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f9377fff640 1 -- 172.21.15.78:0/1253418874 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f9390004d20 con 0x562680667790 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f93a128e740 1 -- 172.21.15.78:0/1253418874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5626805d3db0 con 0x562680667790 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f9377fff640 1 -- 172.21.15.78:0/1253418874 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9390005020 con 0x562680667790 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f93a128e740 10 client.17301 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f93a128e740 1 -- 172.21.15.78:0/1253418874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562680676150 con 0x562680667790 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f9377fff640 1 -- 172.21.15.78:0/1253418874 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5626805d3db0 con 0x562680667790 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f9377fff640 1 -- 172.21.15.78:0/1253418874 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f939001e960 con 0x562680667790 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f93a128e740 10 client.17301 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:16.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f93a128e740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f93a128e740 20 client.17301 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.531+0000 7f93a128e740 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562680677660 0x562680679a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.532+0000 7f93a128e740 4 client.17301 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.532+0000 7f93a128e740 1 -- 172.21.15.78:0/1253418874 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626805cf860 con 0x562680677660 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.532+0000 7f939cc55640 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562680677660 0x562680679a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.532+0000 7f9377fff640 10 client.17301 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.532+0000 7f939cc55640 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562680677660 0x562680679a80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bdb3c3cbd350c0c server_cookie=c0a42ed5b370204e in_seq=0 out_seq=0 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.533+0000 7f9377fff640 1 -- 172.21.15.78:0/1253418874 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5626805cf860 con 0x562680677660 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.533+0000 7f9377fff640 10 client.17301 handle_command_reply: tid=0 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.533+0000 7f93a128e740 1 client.17301 shutdown 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.533+0000 7f93a128e740 1 -- 172.21.15.78:0/1253418874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 msgr2=0x5626806651c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.533+0000 7f93a128e740 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 0x5626806651c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f939000edf0 tx=0x7f9390004a40 comp rx=0 tx=0).stop 2024-09-11T01:08:16.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.533+0000 7f93a128e740 1 -- 172.21.15.78:0/1253418874 shutdown_connections 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.533+0000 7f93a128e740 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562680677660 0x562680679a80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.533+0000 7f93a128e740 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 0x5626806651c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.533+0000 7f93a128e740 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065b800 0x562680664c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.533+0000 7f93a128e740 1 --2- 172.21.15.78:0/1253418874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065aee0 0x562680664740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.533+0000 7f9377fff640 0 client.17301 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.533+0000 7f93a128e740 1 -- 172.21.15.78:0/1253418874 >> 172.21.15.78:0/1253418874 conn(0x562680658d10 msgr2=0x562680657f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.534+0000 7f93a128e740 1 -- 172.21.15.78:0/1253418874 shutdown_connections 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.534+0000 7f93a128e740 1 -- 172.21.15.78:0/1253418874 wait complete. 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.534+0000 7f93a128e740 20 client.17301 trim_cache size 0 max 16384 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.551+0000 7f93a128e740 1 Processor -- start 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.551+0000 7f93a128e740 1 -- start start 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.551+0000 7f93a128e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065aee0 0x5626806663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.551+0000 7f93a128e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065b800 0x562680664dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.551+0000 7f93a128e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562680667790 0x562680661950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.551+0000 7f93a128e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93981b0830 con 0x56268065aee0 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.551+0000 7f93a128e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93981b23a0 con 0x562680667790 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.551+0000 7f93a128e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93981b0f00 con 0x56268065b800 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f939cc55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065aee0 0x5626806663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f939cc55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065aee0 0x5626806663e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41360/0 (socket says 172.21.15.78:41360) 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f939cc55640 1 -- 172.21.15.78:0/812823506 learned_addr learned my addr 172.21.15.78:0/812823506 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f939d456640 1 --2- 172.21.15.78:0/812823506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562680667790 0x562680661950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f9397fff640 1 --2- 172.21.15.78:0/812823506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065b800 0x562680664dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f939cc55640 1 -- 172.21.15.78:0/812823506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065b800 msgr2=0x562680664dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f939cc55640 1 --2- 172.21.15.78:0/812823506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065b800 0x562680664dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f939cc55640 1 -- 172.21.15.78:0/812823506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562680667790 msgr2=0x562680661950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f939cc55640 1 --2- 172.21.15.78:0/812823506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562680667790 0x562680661950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f939cc55640 1 -- 172.21.15.78:0/812823506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562680659790 con 0x56268065aee0 2024-09-11T01:08:16.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f9397fff640 1 --2- 172.21.15.78:0/812823506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065b800 0x562680664dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f939d456640 1 --2- 172.21.15.78:0/812823506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562680667790 0x562680661950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f939cc55640 1 --2- 172.21.15.78:0/812823506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065aee0 0x5626806663e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f938c0ab710 tx=0x7f938c0e2d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e394d396f171c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f93827fc640 1 -- 172.21.15.78:0/812823506 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0ac110 con 0x56268065aee0 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f93827fc640 1 -- 172.21.15.78:0/812823506 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c0b4fa0 con 0x56268065aee0 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.552+0000 7f93827fc640 1 -- 172.21.15.78:0/812823506 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0e50e0 con 0x56268065aee0 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.553+0000 7f93a128e740 1 -- 172.21.15.78:0/812823506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065aee0 msgr2=0x5626806663e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.553+0000 7f93a128e740 1 --2- 172.21.15.78:0/812823506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065aee0 0x5626806663e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f938c0ab710 tx=0x7f938c0e2d00 comp rx=0 tx=0).stop 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.553+0000 7f93a128e740 1 -- 172.21.15.78:0/812823506 shutdown_connections 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.553+0000 7f93a128e740 1 --2- 172.21.15.78:0/812823506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562680667790 0x562680661950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.553+0000 7f93a128e740 1 --2- 172.21.15.78:0/812823506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065b800 0x562680664dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.553+0000 7f93a128e740 1 --2- 172.21.15.78:0/812823506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268065aee0 0x5626806663e0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.553+0000 7f93a128e740 1 -- 172.21.15.78:0/812823506 >> 172.21.15.78:0/812823506 conn(0x562680658d10 msgr2=0x5626806646b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.553+0000 7f93a128e740 1 -- 172.21.15.78:0/812823506 shutdown_connections 2024-09-11T01:08:16.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.553+0000 7f93a128e740 1 -- 172.21.15.78:0/812823506 wait complete. 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.553+0000 7f93a128e740 1 -- start start 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.553+0000 7f93a128e740 1 Processor -- start 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f93a128e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065aee0 0x56268066fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f93a128e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065b800 0x56268066c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f93a128e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 0x56268066c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f93a128e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93981b1ca0 con 0x562680667790 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f93a128e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93981b15d0 con 0x56268065b800 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f93a128e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93981ae830 con 0x56268065aee0 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f93a128e740 10 client.0 _ll_register_callbacks cb 0x7f9390009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f939d456640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 0x56268066c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f939d456640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 0x56268066c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41370/0 (socket says 172.21.15.78:41370) 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f939d456640 1 -- 172.21.15.78:0/722740844 learned_addr learned my addr 172.21.15.78:0/722740844 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f939cc55640 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065aee0 0x56268066fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f9397fff640 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065b800 0x56268066c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f939d456640 1 -- 172.21.15.78:0/722740844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065aee0 msgr2=0x56268066fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f939d456640 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065aee0 0x56268066fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f939d456640 1 -- 172.21.15.78:0/722740844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065b800 msgr2=0x56268066c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f939d456640 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065b800 0x56268066c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f939d456640 1 -- 172.21.15.78:0/722740844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562680673e30 con 0x562680667790 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.554+0000 7f939d456640 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 0x56268066c7d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9390013010 tx=0x7f9390004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78acf40562df1e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f9377fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f9377fff640 1 -- 172.21.15.78:0/722740844 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390018040 con 0x562680667790 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f9377fff640 1 -- 172.21.15.78:0/722740844 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f9390005620 con 0x562680667790 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f93a128e740 1 -- 172.21.15.78:0/722740844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938c0b4fa0 con 0x562680667790 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f9377fff640 1 -- 172.21.15.78:0/722740844 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f939000e9a0 con 0x562680667790 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f93a128e740 10 client.17307 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:16.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f93a128e740 1 -- 172.21.15.78:0/722740844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5626806602f0 con 0x562680667790 2024-09-11T01:08:16.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f9377fff640 1 -- 172.21.15.78:0/722740844 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c0b4fa0 con 0x562680667790 2024-09-11T01:08:16.578 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:08:16.578 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:08:16.578 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "dc23cd05-9f92-4cee-b281-6ddbecb84d9f", 2024-09-11T01:08:16.578 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:08:16.578 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f9377fff640 1 -- 172.21.15.78:0/722740844 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f939001fca0 con 0x562680667790 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f93a128e740 10 client.17307 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f93a128e740 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f93a128e740 20 client.17307 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f93a128e740 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5626806e58f0 0x5626806e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f93a128e740 4 client.17307 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.555+0000 7f93a128e740 1 -- 172.21.15.78:0/722740844 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5626805ccc90 con 0x5626806e58f0 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.556+0000 7f939cc55640 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5626806e58f0 0x5626806e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.556+0000 7f9377fff640 10 client.17307 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.556+0000 7f939cc55640 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5626806e58f0 0x5626806e7d10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16afc50980a32a2b server_cookie=195288618da5ac4f in_seq=0 out_seq=0 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f9377fff640 1 -- 172.21.15.78:0/722740844 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5626805ccc90 con 0x5626806e58f0 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f9377fff640 10 client.17307 handle_command_reply: tid=0 2024-09-11T01:08:16.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f93a128e740 1 client.17307 shutdown 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f93a128e740 1 -- 172.21.15.78:0/722740844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 msgr2=0x56268066c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f93a128e740 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 0x56268066c7d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9390013010 tx=0x7f9390004d60 comp rx=0 tx=0).stop 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f93a128e740 1 -- 172.21.15.78:0/722740844 shutdown_connections 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f93a128e740 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5626806e58f0 0x5626806e7d10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f93a128e740 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562680667790 0x56268066c7d0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f93a128e740 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268065b800 0x56268066c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f93a128e740 1 --2- 172.21.15.78:0/722740844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268065aee0 0x56268066fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f93a128e740 1 -- 172.21.15.78:0/722740844 >> 172.21.15.78:0/722740844 conn(0x562680658d10 msgr2=0x562680664ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f9377fff640 0 client.17307 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f93a128e740 1 -- 172.21.15.78:0/722740844 shutdown_connections 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f93a128e740 1 -- 172.21.15.78:0/722740844 wait complete. 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.558+0000 7f93a128e740 20 client.17307 trim_cache size 0 max 16384 2024-09-11T01:08:16.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.559+0000 7f939eeb9640 1 -- 172.21.15.78:0/3976841038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93880b5450 msgr2=0x7f93880b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:16.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.559+0000 7f939eeb9640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93880b5450 0x7f93880b7910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9390005c40 tx=0x7f9390009500 comp rx=0 tx=0).stop 2024-09-11T01:08:16.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.559+0000 7f939eeb9640 1 -- 172.21.15.78:0/3976841038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9398103b50 msgr2=0x7f93981019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:16.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.559+0000 7f939eeb9640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9398103b50 0x7f93981019d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f938c00d950 tx=0x7f938c00de10 comp rx=0 tx=0).stop 2024-09-11T01:08:16.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.559+0000 7f939eeb9640 1 -- 172.21.15.78:0/3976841038 shutdown_connections 2024-09-11T01:08:16.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.559+0000 7f939eeb9640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93880b5450 0x7f93880b7910 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.559+0000 7f939eeb9640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939810d100 0x7f9398102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.559+0000 7f939eeb9640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398104550 0x7f9398101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.559+0000 7f939eeb9640 1 --2- 172.21.15.78:0/3976841038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9398103b50 0x7f93981019d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:16.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.559+0000 7f939eeb9640 1 -- 172.21.15.78:0/3976841038 >> 172.21.15.78:0/3976841038 conn(0x7f93980fd700 msgr2=0x7f93980fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:16.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.559+0000 7f939eeb9640 1 -- 172.21.15.78:0/3976841038 shutdown_connections 2024-09-11T01:08:16.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:16.559+0000 7f939eeb9640 1 -- 172.21.15.78:0/3976841038 wait complete. 2024-09-11T01:08:16.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:16 smithi110 ceph-mon[34712]: pgmap v611: 97 pgs: 97 active+clean; 455 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 163 op/s 2024-09-11T01:08:16.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:16 smithi112 ceph-mon[35384]: pgmap v611: 97 pgs: 97 active+clean; 455 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 163 op/s 2024-09-11T01:08:17.255 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:08:17.446 INFO:tasks.workunit.client.0.smithi078.stdout: 1 630463 5.27 MB/sec execute 385 sec latency 5159.719 ms 2024-09-11T01:08:17.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:08:17.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:17.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:17.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:17.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:17.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:17.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:17.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:17.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:17 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.515 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:08:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:17.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:17 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:08:17.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:17.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:17.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:17.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:17.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:17.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:17.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:17.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:17.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:17 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:18.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.102+0000 7fb9d1571640 1 -- 172.21.15.78:0/1171642667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9cc1087d0 msgr2=0x7fb9cc108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:18.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.102+0000 7fb9d1571640 1 --2- 172.21.15.78:0/1171642667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9cc1087d0 0x7fb9cc108c10 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009900 tx=0x7fb9c002a410 comp rx=0 tx=0).stop 2024-09-11T01:08:18.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.102+0000 7fb9d1571640 1 -- 172.21.15.78:0/1171642667 shutdown_connections 2024-09-11T01:08:18.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.103+0000 7fb9d1571640 1 --2- 172.21.15.78:0/1171642667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc109150 0x7fb9cc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.103+0000 7fb9d1571640 1 --2- 172.21.15.78:0/1171642667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9cc1087d0 0x7fb9cc108c10 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.103+0000 7fb9d1571640 1 --2- 172.21.15.78:0/1171642667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9cc102dc0 0x7fb9cc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.103+0000 7fb9d1571640 1 -- 172.21.15.78:0/1171642667 >> 172.21.15.78:0/1171642667 conn(0x7fb9cc0fe830 msgr2=0x7fb9cc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:18.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.103+0000 7fb9d1571640 1 -- 172.21.15.78:0/1171642667 shutdown_connections 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.103+0000 7fb9d1571640 1 -- 172.21.15.78:0/1171642667 wait complete. 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.103+0000 7fb9d1571640 1 Processor -- start 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.103+0000 7fb9d1571640 1 -- start start 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.103+0000 7fb9d1571640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9cc102dc0 0x7fb9cc0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9d1571640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc1087d0 0x7fb9cc0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9d1571640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9cc109150 0x7fb9cc075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9d1571640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc112dd0 con 0x7fb9cc109150 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9d1571640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc112c50 con 0x7fb9cc1087d0 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9d1571640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc112f50 con 0x7fb9cc102dc0 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9cb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9cc109150 0x7fb9cc075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9cb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9cc109150 0x7fb9cc075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41390/0 (socket says 172.21.15.78:41390) 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9cb7fe640 1 -- 172.21.15.78:0/1351125983 learned_addr learned my addr 172.21.15.78:0/1351125983 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:18.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9caffd640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9cc102dc0 0x7fb9cc0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc1087d0 0x7fb9cc0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9cb7fe640 1 -- 172.21.15.78:0/1351125983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9cc102dc0 msgr2=0x7fb9cc0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9cb7fe640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9cc102dc0 0x7fb9cc0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9cb7fe640 1 -- 172.21.15.78:0/1351125983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc1087d0 msgr2=0x7fb9cc0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9cb7fe640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc1087d0 0x7fb9cc0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9cb7fe640 1 -- 172.21.15.78:0/1351125983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc1a76e0 con 0x7fb9cc109150 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc1087d0 0x7fb9cc0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9cb7fe640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9cc109150 0x7fb9cc075f20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400eaa0 tx=0x7fb9c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9b3fff640 1 -- 172.21.15.78:0/1351125983 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400cdb0 con 0x7fb9cc109150 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9b3fff640 1 -- 172.21.15.78:0/1351125983 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c4005020 con 0x7fb9cc109150 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9b3fff640 1 -- 172.21.15.78:0/1351125983 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c40116d0 con 0x7fb9cc109150 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9d1571640 1 -- 172.21.15.78:0/1351125983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9cc1a79d0 con 0x7fb9cc109150 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.104+0000 7fb9d1571640 1 -- 172.21.15.78:0/1351125983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9cc1a7ec0 con 0x7fb9cc109150 2024-09-11T01:08:18.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9b3fff640 1 -- 172.21.15.78:0/1351125983 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb9c400f040 con 0x7fb9cc109150 2024-09-11T01:08:18.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9d3946740 1 Processor -- start 2024-09-11T01:08:18.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9d3946740 1 -- start start 2024-09-11T01:08:18.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9d3946740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcabd480 0x5615bcabd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9d3946740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcac97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9d3946740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 0x5615bcacc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9d3946740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615bca33e50 con 0x5615bcac9d30 2024-09-11T01:08:18.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9d3946740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5615bca23280 con 0x5615bcabd480 2024-09-11T01:08:18.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9d3946740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5615bca27090 con 0x5615bcabdda0 2024-09-11T01:08:18.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9ca7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 0x5615bcacc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9ca7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 0x5615bcacc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41396/0 (socket says 172.21.15.78:41396) 2024-09-11T01:08:18.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9ca7fc640 1 -- 172.21.15.78:0/2956089170 learned_addr learned my addr 172.21.15.78:0/2956089170 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:18.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2956089170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcabd480 0x5615bcabd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9caffd640 1 --2- 172.21.15.78:0/2956089170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcac97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9b3fff640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9b40b5450 0x7fb9b40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9ca7fc640 1 -- 172.21.15.78:0/2956089170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 msgr2=0x5615bcac97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:18.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2956089170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcac97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9ca7fc640 1 -- 172.21.15.78:0/2956089170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcabd480 msgr2=0x5615bcabd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:18.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.107+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2956089170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcabd480 0x5615bcabd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.108+0000 7fb9ca7fc640 1 -- 172.21.15.78:0/2956089170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615bc781650 con 0x5615bcac9d30 2024-09-11T01:08:18.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.108+0000 7fb9cb7fe640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9b40b5450 0x7fb9b40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.108+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2956089170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 0x5615bcacc120 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0032e00 tx=0x7fb9c0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6353c152122485d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.108+0000 7fb9b3fff640 1 -- 172.21.15.78:0/1351125983 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7fb9c40e5e10 con 0x7fb9cc109150 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.108+0000 7fb9b17fa640 1 -- 172.21.15.78:0/2956089170 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0035040 con 0x5615bcac9d30 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.108+0000 7fb9b17fa640 1 -- 172.21.15.78:0/2956089170 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c0045070 con 0x5615bcac9d30 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.108+0000 7fb9b17fa640 1 -- 172.21.15.78:0/2956089170 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0003a90 con 0x5615bcac9d30 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 -- 172.21.15.78:0/2956089170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 msgr2=0x5615bcacc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9cb7fe640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9b40b5450 0x7fb9b40b7910 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40e3650 tx=0x7fb9c40cf040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2956089170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 0x5615bcacc120 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0032e00 tx=0x7fb9c0031400 comp rx=0 tx=0).stop 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 -- 172.21.15.78:0/2956089170 shutdown_connections 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2956089170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 0x5615bcacc120 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2956089170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcac97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2956089170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcabd480 0x5615bcabd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 -- 172.21.15.78:0/2956089170 >> 172.21.15.78:0/2956089170 conn(0x5615bcabb2b0 msgr2=0x5615bcabc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 -- 172.21.15.78:0/2956089170 shutdown_connections 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 -- 172.21.15.78:0/2956089170 wait complete. 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 -- start start 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 Processor -- start 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabd480 0x5615bcac9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcac6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcac9d30 0x5615bcac7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615bc884700 con 0x5615bcabd480 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5615bc9be110 con 0x5615bcac9d30 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5615bcacc950 con 0x5615bcabdda0 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9d3946740 10 client.0 _ll_register_callbacks cb 0x5615bc9d7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.109+0000 7fb9ca7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabd480 0x5615bcac9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9ca7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabd480 0x5615bcac9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41398/0 (socket says 172.21.15.78:41398) 2024-09-11T01:08:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9ca7fc640 1 -- 172.21.15.78:0/909330062 learned_addr learned my addr 172.21.15.78:0/909330062 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9caffd640 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcac6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcac9d30 0x5615bcac7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9ca7fc640 1 -- 172.21.15.78:0/909330062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 msgr2=0x5615bcac6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcac6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9ca7fc640 1 -- 172.21.15.78:0/909330062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcac9d30 msgr2=0x5615bcac7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcac9d30 0x5615bcac7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9ca7fc640 1 -- 172.21.15.78:0/909330062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615bc9ebc40 con 0x5615bcabd480 2024-09-11T01:08:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9caffd640 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcac6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabd480 0x5615bcac9660 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0005ec0 tx=0x7fb9c0003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ea4bd43a88010f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9aeff5640 1 -- 172.21.15.78:0/909330062 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0053060 con 0x5615bcabd480 2024-09-11T01:08:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9aeff5640 1 -- 172.21.15.78:0/909330062 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7fb9c0031d10 con 0x5615bcabd480 2024-09-11T01:08:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9d3946740 1 -- 172.21.15.78:0/909330062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c0045070 con 0x5615bcabd480 2024-09-11T01:08:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9aeff5640 1 -- 172.21.15.78:0/909330062 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb9c0035040 con 0x5615bcabd480 2024-09-11T01:08:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9d3946740 10 client.17319 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9d3946740 1 -- 172.21.15.78:0/909330062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5615bcad82f0 con 0x5615bcabd480 2024-09-11T01:08:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.110+0000 7fb9aeff5640 1 -- 172.21.15.78:0/909330062 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c0045070 con 0x5615bcabd480 2024-09-11T01:08:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.111+0000 7fb9aeff5640 1 -- 172.21.15.78:0/909330062 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7fb9c003a040 con 0x5615bcabd480 2024-09-11T01:08:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.111+0000 7fb9d3946740 10 client.17319 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.111+0000 7fb9d3946740 10 client.17319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:08:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.111+0000 7fb9d3946740 20 client.17319 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.111+0000 7fb9d3946740 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5615bcad9640 0x5615bcadba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.111+0000 7fb9d3946740 4 client.17319 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.111+0000 7fb9d3946740 1 -- 172.21.15.78:0/909330062 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615bca35da0 con 0x5615bcad9640 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.111+0000 7fb9caffd640 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5615bcad9640 0x5615bcadba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.111+0000 7fb9aeff5640 10 client.17319 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.111+0000 7fb9caffd640 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5615bcad9640 0x5615bcadba30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6294cde9a463af5c server_cookie=fbc7af84f68d9fd8 in_seq=0 out_seq=0 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.112+0000 7fb9aeff5640 1 -- 172.21.15.78:0/909330062 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615bca35da0 con 0x5615bcad9640 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.112+0000 7fb9aeff5640 10 client.17319 handle_command_reply: tid=0 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.112+0000 7fb9d3946740 1 client.17319 shutdown 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.112+0000 7fb9d3946740 1 -- 172.21.15.78:0/909330062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabd480 msgr2=0x5615bcac9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.112+0000 7fb9d3946740 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabd480 0x5615bcac9660 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0005ec0 tx=0x7fb9c0003730 comp rx=0 tx=0).stop 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.112+0000 7fb9d3946740 1 -- 172.21.15.78:0/909330062 shutdown_connections 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.112+0000 7fb9d3946740 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcac9d30 0x5615bcac7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.112+0000 7fb9d3946740 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcac6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.112+0000 7fb9d3946740 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5615bcad9640 0x5615bcadba30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.112+0000 7fb9d3946740 1 --2- 172.21.15.78:0/909330062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabd480 0x5615bcac9660 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.112+0000 7fb9aeff5640 0 client.17319 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.112+0000 7fb9d3946740 1 -- 172.21.15.78:0/909330062 >> 172.21.15.78:0/909330062 conn(0x5615bcabb2b0 msgr2=0x5615bcac0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.113+0000 7fb9d3946740 1 -- 172.21.15.78:0/909330062 shutdown_connections 2024-09-11T01:08:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.113+0000 7fb9d3946740 1 -- 172.21.15.78:0/909330062 wait complete. 2024-09-11T01:08:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.113+0000 7fb9d3946740 20 client.17319 trim_cache size 0 max 16384 2024-09-11T01:08:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.131+0000 7fb9d3946740 1 Processor -- start 2024-09-11T01:08:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.131+0000 7fb9d3946740 1 -- start start 2024-09-11T01:08:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9d3946740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcabd480 0x5615bcac1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9d3946740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcab62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9d3946740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 0x5615bcac87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9d3946740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1b0820 con 0x5615bcac9d30 2024-09-11T01:08:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9d3946740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1b2390 con 0x5615bcabd480 2024-09-11T01:08:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9d3946740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1b0ef0 con 0x5615bcabdda0 2024-09-11T01:08:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9ca7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 0x5615bcac87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9ca7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 0x5615bcac87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41412/0 (socket says 172.21.15.78:41412) 2024-09-11T01:08:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9ca7fc640 1 -- 172.21.15.78:0/2078611483 learned_addr learned my addr 172.21.15.78:0/2078611483 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9caffd640 1 --2- 172.21.15.78:0/2078611483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcab62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2078611483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcabd480 0x5615bcac1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9ca7fc640 1 -- 172.21.15.78:0/2078611483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 msgr2=0x5615bcab62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2078611483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcab62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9ca7fc640 1 -- 172.21.15.78:0/2078611483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcabd480 msgr2=0x5615bcac1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2078611483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcabd480 0x5615bcac1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9ca7fc640 1 -- 172.21.15.78:0/2078611483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615bcac6860 con 0x5615bcac9d30 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2078611483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 0x5615bcac87f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb9c004d400 tx=0x7fb9c0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d850018ef7f00eb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9b0ff9640 1 -- 172.21.15.78:0/2078611483 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0053020 con 0x5615bcac9d30 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9b0ff9640 1 -- 172.21.15.78:0/2078611483 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c0045070 con 0x5615bcac9d30 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.132+0000 7fb9b0ff9640 1 -- 172.21.15.78:0/2078611483 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c004dd00 con 0x5615bcac9d30 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.133+0000 7fb9d3946740 1 -- 172.21.15.78:0/2078611483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 msgr2=0x5615bcac87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.133+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2078611483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 0x5615bcac87f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb9c004d400 tx=0x7fb9c0005fd0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.133+0000 7fb9d3946740 1 -- 172.21.15.78:0/2078611483 shutdown_connections 2024-09-11T01:08:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.133+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2078611483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcac9d30 0x5615bcac87f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.133+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2078611483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabdda0 0x5615bcab62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.133+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2078611483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcabd480 0x5615bcac1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.133+0000 7fb9d3946740 1 -- 172.21.15.78:0/2078611483 >> 172.21.15.78:0/2078611483 conn(0x5615bcabb2b0 msgr2=0x5615bcac80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.133+0000 7fb9d3946740 1 -- 172.21.15.78:0/2078611483 shutdown_connections 2024-09-11T01:08:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.133+0000 7fb9d3946740 1 -- 172.21.15.78:0/2078611483 wait complete. 2024-09-11T01:08:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9d3946740 1 -- start start 2024-09-11T01:08:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9d3946740 1 Processor -- start 2024-09-11T01:08:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9d3946740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabd480 0x5615bcadabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9d3946740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabdda0 0x5615bcadb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9d3946740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcac9d30 0x5615bcadb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9d3946740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1b1c90 con 0x5615bcabdda0 2024-09-11T01:08:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9d3946740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1b15c0 con 0x5615bcac9d30 2024-09-11T01:08:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9d3946740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1ae820 con 0x5615bcabd480 2024-09-11T01:08:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9d3946740 10 client.0 _ll_register_callbacks cb 0x7fb9bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9caffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabdda0 0x5615bcadb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9caffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabdda0 0x5615bcadb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41422/0 (socket says 172.21.15.78:41422) 2024-09-11T01:08:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9caffd640 1 -- 172.21.15.78:0/2236117979 learned_addr learned my addr 172.21.15.78:0/2236117979 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabd480 0x5615bcadabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcac9d30 0x5615bcadb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9caffd640 1 -- 172.21.15.78:0/2236117979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabd480 msgr2=0x5615bcadabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9caffd640 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabd480 0x5615bcadabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9caffd640 1 -- 172.21.15.78:0/2236117979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcac9d30 msgr2=0x5615bcadb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9caffd640 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcac9d30 0x5615bcadb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9caffd640 1 -- 172.21.15.78:0/2236117979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615bcb453f0 con 0x5615bcabdda0 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.134+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcac9d30 0x5615bcadb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9caffd640 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabdda0 0x5615bcadb100 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x5615bca271e0 tx=0x7fb9bc004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8821df8eaf5df6d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9aeff5640 1 -- 172.21.15.78:0/2236117979 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc0047a0 con 0x5615bcabdda0 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9aeff5640 1 -- 172.21.15.78:0/2236117979 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7fb9bc004940 con 0x5615bcabdda0 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9d3946740 1 -- 172.21.15.78:0/2236117979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c0045070 con 0x5615bcabdda0 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9aeff5640 1 -- 172.21.15.78:0/2236117979 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb9bc004b40 con 0x5615bcabdda0 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9d3946740 10 client.17325 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9d3946740 1 -- 172.21.15.78:0/2236117979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5615bcb46790 con 0x5615bcabdda0 2024-09-11T01:08:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9aeff5640 1 -- 172.21.15.78:0/2236117979 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c0045070 con 0x5615bcabdda0 2024-09-11T01:08:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9aeff5640 1 -- 172.21.15.78:0/2236117979 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7fb9bc0266f0 con 0x5615bcabdda0 2024-09-11T01:08:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9d3946740 10 client.17325 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9d3946740 10 client.17325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:08:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9d3946740 20 client.17325 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5615bcb46f20 0x5615bcb49340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9d3946740 4 client.17325 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:08:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.135+0000 7fb9d3946740 1 -- 172.21.15.78:0/2236117979 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5615bca31850 con 0x5615bcb46f20 2024-09-11T01:08:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.136+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5615bcb46f20 0x5615bcb49340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.136+0000 7fb9ca7fc640 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5615bcb46f20 0x5615bcb49340 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e11f2e19879e18ed server_cookie=75698bb6f326f2 in_seq=0 out_seq=0 2024-09-11T01:08:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.136+0000 7fb9aeff5640 10 client.17325 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.136+0000 7fb9aeff5640 1 -- 172.21.15.78:0/2236117979 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3984 (crc 0 0 0) 0x5615bca31850 con 0x5615bcb46f20 2024-09-11T01:08:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.136+0000 7fb9aeff5640 10 client.17325 handle_command_reply: tid=0 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.136+0000 7fb9d3946740 1 client.17325 shutdown 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d3946740 1 -- 172.21.15.78:0/2236117979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabdda0 msgr2=0x5615bcadb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabdda0 0x5615bcadb100 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x5615bca271e0 tx=0x7fb9bc004410 comp rx=0 tx=0).stop 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d3946740 1 -- 172.21.15.78:0/2236117979 shutdown_connections 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615bcac9d30 0x5615bcadb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5615bcb46f20 0x5615bcb49340 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615bcabdda0 0x5615bcadb100 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d3946740 1 --2- 172.21.15.78:0/2236117979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615bcabd480 0x5615bcadabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9aeff5640 0 client.17325 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d3946740 1 -- 172.21.15.78:0/2236117979 >> 172.21.15.78:0/2236117979 conn(0x5615bcabb2b0 msgr2=0x5615bcac8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d3946740 1 -- 172.21.15.78:0/2236117979 shutdown_connections 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d3946740 1 -- 172.21.15.78:0/2236117979 wait complete. 2024-09-11T01:08:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d3946740 20 client.17325 trim_cache size 0 max 16384 2024-09-11T01:08:18.146 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:08:18.146 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (1 inodes in the stack)", 2024-09-11T01:08:18.146 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:08:18.146 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:08:18.146 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.146 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "dc23cd05-9f92-4cee-b281-6ddbecb84d9f", 2024-09-11T01:08:18.146 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:08:18.146 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:08:18.147 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:08:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:08:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.150 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.152 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:08:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:08:18.155 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:08:18.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d1571640 1 -- 172.21.15.78:0/1351125983 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9b40b5450 msgr2=0x7fb9b40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:18.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d1571640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9b40b5450 0x7fb9b40b7910 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40e3650 tx=0x7fb9c40cf040 comp rx=0 tx=0).stop 2024-09-11T01:08:18.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d1571640 1 -- 172.21.15.78:0/1351125983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9cc109150 msgr2=0x7fb9cc075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:18.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.137+0000 7fb9d1571640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9cc109150 0x7fb9cc075f20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400eaa0 tx=0x7fb9c400ef60 comp rx=0 tx=0).stop 2024-09-11T01:08:18.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.138+0000 7fb9d1571640 1 -- 172.21.15.78:0/1351125983 shutdown_connections 2024-09-11T01:08:18.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.138+0000 7fb9d1571640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9b40b5450 0x7fb9b40b7910 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.138+0000 7fb9d1571640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9cc109150 0x7fb9cc075f20 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.138+0000 7fb9d1571640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc1087d0 0x7fb9cc0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.138+0000 7fb9d1571640 1 --2- 172.21.15.78:0/1351125983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9cc102dc0 0x7fb9cc0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.138+0000 7fb9d1571640 1 -- 172.21.15.78:0/1351125983 >> 172.21.15.78:0/1351125983 conn(0x7fb9cc0fe830 msgr2=0x7fb9cc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.138+0000 7fb9d1571640 1 -- 172.21.15.78:0/1351125983 shutdown_connections 2024-09-11T01:08:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:18.138+0000 7fb9d1571640 1 -- 172.21.15.78:0/1351125983 wait complete. 2024-09-11T01:08:18.446 INFO:tasks.workunit.client.0.smithi078.stdout: 1 630463 5.26 MB/sec execute 386 sec latency 6159.758 ms 2024-09-11T01:08:18.493 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:18.493 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:18.493 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:18.493 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:18.493 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:18 smithi078 ceph-mon[31315]: pgmap v612: 97 pgs: 97 active+clean; 456 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-09-11T01:08:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:18.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:18.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:18 smithi110 ceph-mon[34712]: pgmap v612: 97 pgs: 97 active+clean; 456 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-09-11T01:08:18.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:18.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:18.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:18.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:18.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:18 smithi112 ceph-mon[35384]: pgmap v612: 97 pgs: 97 active+clean; 456 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-09-11T01:08:18.918 INFO:tasks.cephfs.filesystem:scrub status for tag:dc23cd05-9f92-4cee-b281-6ddbecb84d9f - {'path': '/', 'tag': 'dc23cd05-9f92-4cee-b281-6ddbecb84d9f', 'options': 'recursive,force'} 2024-09-11T01:08:19.446 INFO:tasks.workunit.client.0.smithi078.stdout: 1 630463 5.24 MB/sec execute 387 sec latency 7159.866 ms 2024-09-11T01:08:20.446 INFO:tasks.workunit.client.0.smithi078.stdout: 1 630463 5.23 MB/sec execute 388 sec latency 8159.978 ms 2024-09-11T01:08:20.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:20 smithi110 ceph-mon[34712]: pgmap v613: 97 pgs: 97 active+clean; 455 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-09-11T01:08:20.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:20 smithi112 ceph-mon[35384]: pgmap v613: 97 pgs: 97 active+clean; 455 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-09-11T01:08:20.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:20 smithi078 ceph-mon[31315]: pgmap v613: 97 pgs: 97 active+clean; 455 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-09-11T01:08:21.446 INFO:tasks.workunit.client.0.smithi078.stdout: 1 630463 5.22 MB/sec execute 389 sec latency 9160.065 ms 2024-09-11T01:08:22.446 INFO:tasks.workunit.client.0.smithi078.stdout: 1 631175 5.21 MB/sec execute 390 sec latency 9844.501 ms 2024-09-11T01:08:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:22 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:22 smithi112 ceph-mon[35384]: pgmap v614: 97 pgs: 97 active+clean; 455 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-09-11T01:08:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:22 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:22 smithi078 ceph-mon[31315]: pgmap v614: 97 pgs: 97 active+clean; 455 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-09-11T01:08:22.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:22 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:22.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:22 smithi110 ceph-mon[34712]: pgmap v614: 97 pgs: 97 active+clean; 455 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-09-11T01:08:23.446 INFO:tasks.workunit.client.0.smithi078.stdout: 1 632762 5.22 MB/sec execute 391 sec latency 11.145 ms 2024-09-11T01:08:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:23 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:23 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:23 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:23 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:08:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:23 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:23 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:23 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:23 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:23.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:23 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:23.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:23 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:23.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:23 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:23 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:08:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:23 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:23 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:23 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:23 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:23 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:23 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:23 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:23 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:08:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:23 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:23 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:23 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:23 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:08:24.446 INFO:tasks.workunit.client.0.smithi078.stdout: 1 633858 5.22 MB/sec execute 392 sec latency 397.121 ms 2024-09-11T01:08:24.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:24 smithi078 ceph-mon[31315]: pgmap v615: 97 pgs: 97 active+clean; 456 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 887 KiB/s wr, 67 op/s 2024-09-11T01:08:24.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:24 smithi110 ceph-mon[34712]: pgmap v615: 97 pgs: 97 active+clean; 456 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 887 KiB/s wr, 67 op/s 2024-09-11T01:08:24.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:24 smithi112 ceph-mon[35384]: pgmap v615: 97 pgs: 97 active+clean; 456 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 887 KiB/s wr, 67 op/s 2024-09-11T01:08:25.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 633858 5.21 MB/sec execute 393 sec latency 1397.245 ms 2024-09-11T01:08:26.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 633858 5.20 MB/sec execute 394 sec latency 2397.307 ms 2024-09-11T01:08:26.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:26 smithi110 ceph-mon[34712]: pgmap v616: 97 pgs: 97 active+clean; 460 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.3 MiB/s wr, 88 op/s 2024-09-11T01:08:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:26 smithi112 ceph-mon[35384]: pgmap v616: 97 pgs: 97 active+clean; 460 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.3 MiB/s wr, 88 op/s 2024-09-11T01:08:26.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:26 smithi078 ceph-mon[31315]: pgmap v616: 97 pgs: 97 active+clean; 460 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.3 MiB/s wr, 88 op/s 2024-09-11T01:08:27.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 634556 5.19 MB/sec execute 395 sec latency 3149.446 ms 2024-09-11T01:08:28.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 636150 5.21 MB/sec execute 396 sec latency 12.951 ms 2024-09-11T01:08:28.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:28 smithi110 ceph-mon[34712]: pgmap v617: 97 pgs: 97 active+clean; 461 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-09-11T01:08:28.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:28 smithi112 ceph-mon[35384]: pgmap v617: 97 pgs: 97 active+clean; 461 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-09-11T01:08:28.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:28 smithi078 ceph-mon[31315]: pgmap v617: 97 pgs: 97 active+clean; 461 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-09-11T01:08:29.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 637402 5.21 MB/sec execute 397 sec latency 292.921 ms 2024-09-11T01:08:30.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 637402 5.19 MB/sec execute 398 sec latency 1292.990 ms 2024-09-11T01:08:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:30 smithi110 ceph-mon[34712]: pgmap v618: 97 pgs: 97 active+clean; 461 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-09-11T01:08:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:30 smithi112 ceph-mon[35384]: pgmap v618: 97 pgs: 97 active+clean; 461 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-09-11T01:08:30.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:30 smithi078 ceph-mon[31315]: pgmap v618: 97 pgs: 97 active+clean; 461 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-09-11T01:08:31.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 637402 5.18 MB/sec execute 399 sec latency 2293.094 ms 2024-09-11T01:08:32.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 638107 5.17 MB/sec execute 400 sec latency 3043.418 ms 2024-09-11T01:08:32.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:32 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:08:32.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:32 smithi112 ceph-mon[35384]: pgmap v619: 97 pgs: 97 active+clean; 462 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-09-11T01:08:32.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:32 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:08:32.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:32 smithi078 ceph-mon[31315]: pgmap v619: 97 pgs: 97 active+clean; 462 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-09-11T01:08:32.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:32 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:08:32.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:32 smithi110 ceph-mon[34712]: pgmap v619: 97 pgs: 97 active+clean; 462 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-09-11T01:08:33.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 639783 5.19 MB/sec execute 401 sec latency 11.392 ms 2024-09-11T01:08:33.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:08:33.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:08:33.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:08:33.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:08:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:08:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:08:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:08:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:08:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:08:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:08:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:08:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:08:34.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 640960 5.19 MB/sec execute 402 sec latency 324.192 ms 2024-09-11T01:08:34.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:34 smithi110 ceph-mon[34712]: pgmap v620: 97 pgs: 97 active+clean; 465 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-09-11T01:08:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:34 smithi112 ceph-mon[35384]: pgmap v620: 97 pgs: 97 active+clean; 465 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-09-11T01:08:34.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:34 smithi078 ceph-mon[31315]: pgmap v620: 97 pgs: 97 active+clean; 465 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-09-11T01:08:35.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 643121 5.21 MB/sec execute 403 sec latency 19.936 ms 2024-09-11T01:08:36.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 644492 5.21 MB/sec execute 404 sec latency 168.707 ms 2024-09-11T01:08:36.528 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:36 smithi112 ceph-mon[35384]: pgmap v621: 97 pgs: 97 active+clean; 467 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1.9 MiB/s wr, 158 op/s 2024-09-11T01:08:36.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:36 smithi110 ceph-mon[34712]: pgmap v621: 97 pgs: 97 active+clean; 467 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1.9 MiB/s wr, 158 op/s 2024-09-11T01:08:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:36 smithi078 ceph-mon[31315]: pgmap v621: 97 pgs: 97 active+clean; 467 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1.9 MiB/s wr, 158 op/s 2024-09-11T01:08:37.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 645202 5.20 MB/sec execute 405 sec latency 919.445 ms 2024-09-11T01:08:38.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 646943 5.22 MB/sec execute 406 sec latency 10.572 ms 2024-09-11T01:08:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:38 smithi110 ceph-mon[34712]: pgmap v622: 97 pgs: 97 active+clean; 470 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.8 MiB/s wr, 159 op/s 2024-09-11T01:08:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:38 smithi112 ceph-mon[35384]: pgmap v622: 97 pgs: 97 active+clean; 470 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.8 MiB/s wr, 159 op/s 2024-09-11T01:08:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:38 smithi078 ceph-mon[31315]: pgmap v622: 97 pgs: 97 active+clean; 470 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.8 MiB/s wr, 159 op/s 2024-09-11T01:08:39.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 648042 5.22 MB/sec execute 407 sec latency 437.696 ms 2024-09-11T01:08:40.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 649264 5.23 MB/sec execute 408 sec latency 958.902 ms 2024-09-11T01:08:40.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:40 smithi110 ceph-mon[34712]: pgmap v623: 97 pgs: 97 active+clean; 472 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-09-11T01:08:40.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:40 smithi112 ceph-mon[35384]: pgmap v623: 97 pgs: 97 active+clean; 472 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-09-11T01:08:40.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:40 smithi078 ceph-mon[31315]: pgmap v623: 97 pgs: 97 active+clean; 472 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-09-11T01:08:41.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 650803 5.23 MB/sec execute 409 sec latency 11.021 ms 2024-09-11T01:08:42.447 INFO:tasks.workunit.client.0.smithi078.stdout: 1 652300 5.23 MB/sec execute 410 sec latency 342.670 ms 2024-09-11T01:08:42.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:42 smithi110 ceph-mon[34712]: pgmap v624: 97 pgs: 97 active+clean; 476 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 180 op/s 2024-09-11T01:08:42.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:42 smithi112 ceph-mon[35384]: pgmap v624: 97 pgs: 97 active+clean; 476 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 180 op/s 2024-09-11T01:08:42.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:42 smithi078 ceph-mon[31315]: pgmap v624: 97 pgs: 97 active+clean; 476 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 180 op/s 2024-09-11T01:08:43.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 653992 5.25 MB/sec execute 411 sec latency 11.083 ms 2024-09-11T01:08:44.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 655132 5.25 MB/sec execute 412 sec latency 399.704 ms 2024-09-11T01:08:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:44 smithi110 ceph-mon[34712]: pgmap v625: 97 pgs: 97 active+clean; 481 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-09-11T01:08:44.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:44 smithi112 ceph-mon[35384]: pgmap v625: 97 pgs: 97 active+clean; 481 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-09-11T01:08:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:44 smithi078 ceph-mon[31315]: pgmap v625: 97 pgs: 97 active+clean; 481 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-09-11T01:08:45.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 655132 5.24 MB/sec execute 413 sec latency 1399.781 ms 2024-09-11T01:08:46.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 655132 5.22 MB/sec execute 414 sec latency 2399.863 ms 2024-09-11T01:08:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:46 smithi110 ceph-mon[34712]: pgmap v626: 97 pgs: 97 active+clean; 480 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 207 op/s 2024-09-11T01:08:46.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:46 smithi112 ceph-mon[35384]: pgmap v626: 97 pgs: 97 active+clean; 480 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 207 op/s 2024-09-11T01:08:46.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:46 smithi078 ceph-mon[31315]: pgmap v626: 97 pgs: 97 active+clean; 480 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 207 op/s 2024-09-11T01:08:47.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 655829 5.22 MB/sec execute 415 sec latency 3149.387 ms 2024-09-11T01:08:48.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 657537 5.23 MB/sec execute 416 sec latency 11.141 ms 2024-09-11T01:08:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:48 smithi110 ceph-mon[34712]: pgmap v627: 97 pgs: 97 active+clean; 483 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-09-11T01:08:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:48 smithi112 ceph-mon[35384]: pgmap v627: 97 pgs: 97 active+clean; 483 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-09-11T01:08:48.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:48 smithi078 ceph-mon[31315]: pgmap v627: 97 pgs: 97 active+clean; 483 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-09-11T01:08:48.919 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:08:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:49.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 658675 5.23 MB/sec execute 417 sec latency 388.026 ms 2024-09-11T01:08:49.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.655+0000 7f4cf70e7640 1 -- 172.21.15.78:0/3910507340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf0102dc0 msgr2=0x7f4cf01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:49.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.655+0000 7f4cf70e7640 1 --2- 172.21.15.78:0/3910507340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf0102dc0 0x7f4cf01031a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4cec009900 tx=0x7f4cec02a410 comp rx=0 tx=0).stop 2024-09-11T01:08:49.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.655+0000 7f4cf70e7640 1 -- 172.21.15.78:0/3910507340 shutdown_connections 2024-09-11T01:08:49.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.655+0000 7f4cf70e7640 1 --2- 172.21.15.78:0/3910507340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cf0109150 0x7f4cf010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.655+0000 7f4cf70e7640 1 --2- 172.21.15.78:0/3910507340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf01087d0 0x7f4cf0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.655+0000 7f4cf70e7640 1 --2- 172.21.15.78:0/3910507340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf0102dc0 0x7f4cf01031a0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.655+0000 7f4cf70e7640 1 -- 172.21.15.78:0/3910507340 >> 172.21.15.78:0/3910507340 conn(0x7f4cf00fe830 msgr2=0x7f4cf0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.655+0000 7f4cf70e7640 1 -- 172.21.15.78:0/3910507340 shutdown_connections 2024-09-11T01:08:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.655+0000 7f4cf70e7640 1 -- 172.21.15.78:0/3910507340 wait complete. 2024-09-11T01:08:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4cf70e7640 1 Processor -- start 2024-09-11T01:08:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4cf70e7640 1 -- start start 2024-09-11T01:08:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4cf70e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cf0102dc0 0x7f4cf01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4cf70e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf01087d0 0x7f4cf01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4cf70e7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf0109150 0x7f4cf019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4cf70e7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0112e10 con 0x7f4cf01087d0 2024-09-11T01:08:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4cf70e7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0112c90 con 0x7f4cf0109150 2024-09-11T01:08:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4cf70e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0112f90 con 0x7f4cf0102dc0 2024-09-11T01:08:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf01087d0 0x7f4cf01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf01087d0 0x7f4cf01997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47896/0 (socket says 172.21.15.78:47896) 2024-09-11T01:08:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4ce7fff640 1 -- 172.21.15.78:0/687036674 learned_addr learned my addr 172.21.15.78:0/687036674 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4cf5684640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf0109150 0x7f4cf019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4cf4e83640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cf0102dc0 0x7f4cf01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4ce7fff640 1 -- 172.21.15.78:0/687036674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cf0102dc0 msgr2=0x7f4cf01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4ce7fff640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cf0102dc0 0x7f4cf01992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4ce7fff640 1 -- 172.21.15.78:0/687036674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf0109150 msgr2=0x7f4cf019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4ce7fff640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf0109150 0x7f4cf019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.656+0000 7f4ce7fff640 1 -- 172.21.15.78:0/687036674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf019f350 con 0x7f4cf01087d0 2024-09-11T01:08:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.657+0000 7f4ce7fff640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf01087d0 0x7f4cf01997e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000e9f0 tx=0x7f4ce000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.657+0000 7f4ce5ffb640 1 -- 172.21.15.78:0/687036674 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce000cd80 con 0x7f4cf01087d0 2024-09-11T01:08:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.657+0000 7f4ce5ffb640 1 -- 172.21.15.78:0/687036674 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce00108e0 con 0x7f4cf01087d0 2024-09-11T01:08:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.657+0000 7f4cf70e7640 1 -- 172.21.15.78:0/687036674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf019f640 con 0x7f4cf01087d0 2024-09-11T01:08:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.657+0000 7f4ce5ffb640 1 -- 172.21.15.78:0/687036674 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0018980 con 0x7f4cf01087d0 2024-09-11T01:08:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.657+0000 7f4cf70e7640 1 -- 172.21.15.78:0/687036674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf01a5d00 con 0x7f4cf01087d0 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4cf94bc740 1 Processor -- start 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4cf94bc740 1 -- start start 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4ce5ffb640 1 -- 172.21.15.78:0/687036674 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4ce0010430 con 0x7f4cf01087d0 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4cf94bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590578490a0 0x559057849480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4cf94bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578499c0 0x559057855410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4cf94bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 0x559057857d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4cf94bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5590577bfe50 con 0x559057855950 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4cf94bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5590577af280 con 0x5590578490a0 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4cf94bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5590577b3090 con 0x5590578499c0 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 0x559057857d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 0x559057857d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47912/0 (socket says 172.21.15.78:47912) 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4ce7fff640 1 -- 172.21.15.78:0/4232506675 learned_addr learned my addr 172.21.15.78:0/4232506675 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.660+0000 7f4cf5684640 1 --2- 172.21.15.78:0/4232506675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590578490a0 0x559057849480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4cf4e83640 1 --2- 172.21.15.78:0/4232506675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578499c0 0x559057855410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4ce7fff640 1 -- 172.21.15.78:0/4232506675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578499c0 msgr2=0x559057855410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4ce7fff640 1 --2- 172.21.15.78:0/4232506675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578499c0 0x559057855410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4ce7fff640 1 -- 172.21.15.78:0/4232506675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590578490a0 msgr2=0x559057849480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4ce7fff640 1 --2- 172.21.15.78:0/4232506675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590578490a0 0x559057849480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4ce7fff640 1 -- 172.21.15.78:0/4232506675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55905750d650 con 0x559057855950 2024-09-11T01:08:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4cf5684640 1 --2- 172.21.15.78:0/4232506675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590578490a0 0x559057849480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4ce7fff640 1 --2- 172.21.15.78:0/4232506675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 0x559057857d40 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00e3a40 tx=0x7f4ce00ee9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e819f2e85f8eb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4cd6ffd640 1 -- 172.21.15.78:0/4232506675 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00fa070 con 0x559057855950 2024-09-11T01:08:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4cd6ffd640 1 -- 172.21.15.78:0/4232506675 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce00e84e0 con 0x559057855950 2024-09-11T01:08:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4cd6ffd640 1 -- 172.21.15.78:0/4232506675 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00f29f0 con 0x559057855950 2024-09-11T01:08:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4ce5ffb640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4cd80b5450 0x7f4cd80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.661+0000 7f4cf5684640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4cd80b5450 0x7f4cd80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.662+0000 7f4ce5ffb640 1 -- 172.21.15.78:0/687036674 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f4ce00e6340 con 0x7f4cf01087d0 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.662+0000 7f4cf94bc740 1 -- 172.21.15.78:0/4232506675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 msgr2=0x559057857d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.662+0000 7f4cf5684640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4cd80b5450 0x7f4cd80b7910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8005fd0 tx=0x7f4ce8009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.662+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/4232506675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 0x559057857d40 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00e3a40 tx=0x7f4ce00ee9d0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.662+0000 7f4cf94bc740 1 -- 172.21.15.78:0/4232506675 shutdown_connections 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.662+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/4232506675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 0x559057857d40 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.662+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/4232506675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578499c0 0x559057855410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.662+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/4232506675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590578490a0 0x559057849480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.662+0000 7f4cf94bc740 1 -- 172.21.15.78:0/4232506675 >> 172.21.15.78:0/4232506675 conn(0x559057846ed0 msgr2=0x559057848070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.662+0000 7f4cf94bc740 1 -- 172.21.15.78:0/4232506675 shutdown_connections 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.662+0000 7f4cf94bc740 1 -- 172.21.15.78:0/4232506675 wait complete. 2024-09-11T01:08:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.662+0000 7f4cf94bc740 1 -- start start 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4cf94bc740 1 Processor -- start 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4cf94bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578490a0 0x559057855280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4cf94bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578499c0 0x5590578526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4cf94bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559057855950 0x559057852d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4cf94bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559057610700 con 0x5590578499c0 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4cf94bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55905774a110 con 0x559057855950 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4cf94bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5590578585d0 con 0x5590578490a0 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4cf94bc740 10 client.0 _ll_register_callbacks cb 0x559057763990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578499c0 0x5590578526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578499c0 0x5590578526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47914/0 (socket says 172.21.15.78:47914) 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4ce7fff640 1 -- 172.21.15.78:0/3316456156 learned_addr learned my addr 172.21.15.78:0/3316456156 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4cf5684640 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559057855950 0x559057852d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4cf4e83640 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578490a0 0x559057855280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4ce7fff640 1 -- 172.21.15.78:0/3316456156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578490a0 msgr2=0x559057855280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4ce7fff640 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578490a0 0x559057855280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4ce7fff640 1 -- 172.21.15.78:0/3316456156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559057855950 msgr2=0x559057852d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4ce7fff640 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559057855950 0x559057852d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.663+0000 7f4ce7fff640 1 -- 172.21.15.78:0/3316456156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559057777c40 con 0x5590578499c0 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4ce7fff640 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578499c0 0x5590578526d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00a8dd0 tx=0x7f4ce0009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1db72c1352ab26a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4ccffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4ccffff640 1 -- 172.21.15.78:0/3316456156 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00a8e50 con 0x5590578499c0 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4ccffff640 1 -- 172.21.15.78:0/3316456156 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f4ce00f2300 con 0x5590578499c0 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4cf94bc740 1 -- 172.21.15.78:0/3316456156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce00e84e0 con 0x5590578499c0 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4ccffff640 1 -- 172.21.15.78:0/3316456156 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4ce00abf70 con 0x5590578499c0 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4cf94bc740 10 client.17337 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4cf94bc740 1 -- 172.21.15.78:0/3316456156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559057863f40 con 0x5590578499c0 2024-09-11T01:08:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4ccffff640 1 -- 172.21.15.78:0/3316456156 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce00e84e0 con 0x5590578499c0 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4ccffff640 1 -- 172.21.15.78:0/3316456156 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f4ce00a8a70 con 0x5590578499c0 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4cf94bc740 10 client.17337 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4cf94bc740 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4cf94bc740 20 client.17337 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559057865290 0x559057867680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4cf94bc740 4 client.17337 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.664+0000 7f4cf94bc740 1 -- 172.21.15.78:0/3316456156 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590577c1da0 con 0x559057865290 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.665+0000 7f4cf4e83640 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559057865290 0x559057867680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.665+0000 7f4ccffff640 10 client.17337 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.665+0000 7f4cf4e83640 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559057865290 0x559057867680 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40bfb526d11e59a4 server_cookie=39dd3f4ef15a787d in_seq=0 out_seq=0 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4ccffff640 1 -- 172.21.15.78:0/3316456156 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5590577c1da0 con 0x559057865290 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4ccffff640 10 client.17337 handle_command_reply: tid=0 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4cf94bc740 1 client.17337 shutdown 2024-09-11T01:08:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4cf94bc740 1 -- 172.21.15.78:0/3316456156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578499c0 msgr2=0x5590578526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:49.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578499c0 0x5590578526d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00a8dd0 tx=0x7f4ce0009770 comp rx=0 tx=0).stop 2024-09-11T01:08:49.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4cf94bc740 1 -- 172.21.15.78:0/3316456156 shutdown_connections 2024-09-11T01:08:49.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559057855950 0x559057852d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559057865290 0x559057867680 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578499c0 0x5590578526d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/3316456156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578490a0 0x559057855280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4ccffff640 0 client.17337 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:49.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4cf94bc740 1 -- 172.21.15.78:0/3316456156 >> 172.21.15.78:0/3316456156 conn(0x559057846ed0 msgr2=0x55905784ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:49.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4cf94bc740 1 -- 172.21.15.78:0/3316456156 shutdown_connections 2024-09-11T01:08:49.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4cf94bc740 1 -- 172.21.15.78:0/3316456156 wait complete. 2024-09-11T01:08:49.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.666+0000 7f4cf94bc740 20 client.17337 trim_cache size 0 max 16384 2024-09-11T01:08:49.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.683+0000 7f4cf94bc740 1 Processor -- start 2024-09-11T01:08:49.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.683+0000 7f4cf94bc740 1 -- start start 2024-09-11T01:08:49.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.683+0000 7f4cf94bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578490a0 0x559057861870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.786 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:08:49.786 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:08:49.786 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:08:49.786 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:08:49.786 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:08:49.787 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:08:49.787 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:49.787 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:08:49.787 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:08:49.787 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.788 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.788 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.788 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:49.788 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:08:49.788 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:08:49.788 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.788 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.788 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:08:49.789 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:08:49.790 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:49.792 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:08:49.792 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:08:49.792 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.792 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.792 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.792 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.792 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:49.792 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:08:49.792 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:08:49.792 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.793 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.793 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.793 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.793 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:49.793 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:08:49.793 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:08:49.793 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.793 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:08:49.794 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:08:49.795 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:08:49.795 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:08:49.795 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:08:49.795 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:08:49.795 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:08:49.795 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:08:49.795 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:08:49.795 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:08:49.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.683+0000 7f4cf94bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590578499c0 0x559057852480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.683+0000 7f4cf94bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 0x559057845e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.683+0000 7f4cf94bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01ae690 con 0x559057855950 2024-09-11T01:08:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.683+0000 7f4cf94bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01b0200 con 0x5590578499c0 2024-09-11T01:08:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.683+0000 7f4cf94bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01aed60 con 0x5590578490a0 2024-09-11T01:08:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.683+0000 7f4cf5684640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 0x559057845e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.683+0000 7f4cf5684640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 0x559057845e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47916/0 (socket says 172.21.15.78:47916) 2024-09-11T01:08:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.683+0000 7f4cf5684640 1 -- 172.21.15.78:0/2743754161 learned_addr learned my addr 172.21.15.78:0/2743754161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.683+0000 7f4cf4e83640 1 --2- 172.21.15.78:0/2743754161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578490a0 0x559057861870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf5684640 1 -- 172.21.15.78:0/2743754161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578490a0 msgr2=0x559057861870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf5684640 1 --2- 172.21.15.78:0/2743754161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578490a0 0x559057861870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf5684640 1 -- 172.21.15.78:0/2743754161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590578499c0 msgr2=0x559057852480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf5684640 1 --2- 172.21.15.78:0/2743754161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590578499c0 0x559057852480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf5684640 1 -- 172.21.15.78:0/2743754161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590578530c0 con 0x559057855950 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf5684640 1 --2- 172.21.15.78:0/2743754161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 0x559057845e10 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8003d60 tx=0x7f4ce8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7d417708b8982a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cd67fc640 1 -- 172.21.15.78:0/2743754161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8004540 con 0x559057855950 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cd67fc640 1 -- 172.21.15.78:0/2743754161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce80046e0 con 0x559057855950 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cd67fc640 1 -- 172.21.15.78:0/2743754161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce801f430 con 0x559057855950 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf94bc740 1 -- 172.21.15.78:0/2743754161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 msgr2=0x559057845e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/2743754161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 0x559057845e10 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8003d60 tx=0x7f4ce8017000 comp rx=0 tx=0).stop 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf94bc740 1 -- 172.21.15.78:0/2743754161 shutdown_connections 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/2743754161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559057855950 0x559057845e10 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/2743754161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590578499c0 0x559057852480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/2743754161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578490a0 0x559057861870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.684+0000 7f4cf94bc740 1 -- 172.21.15.78:0/2743754161 >> 172.21.15.78:0/2743754161 conn(0x559057846ed0 msgr2=0x559057845700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf94bc740 1 -- 172.21.15.78:0/2743754161 shutdown_connections 2024-09-11T01:08:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf94bc740 1 -- 172.21.15.78:0/2743754161 wait complete. 2024-09-11T01:08:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf94bc740 1 -- start start 2024-09-11T01:08:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf94bc740 1 Processor -- start 2024-09-11T01:08:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf94bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578490a0 0x55905786aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf94bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578499c0 0x55905786afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf94bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559057855950 0x5590578677c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf94bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01afb00 con 0x5590578490a0 2024-09-11T01:08:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf94bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01af430 con 0x559057855950 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf94bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01ac690 con 0x5590578499c0 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf94bc740 10 client.0 _ll_register_callbacks cb 0x7f4ce00f8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf4e83640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578490a0 0x55905786aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf4e83640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578490a0 0x55905786aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47924/0 (socket says 172.21.15.78:47924) 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4cf4e83640 1 -- 172.21.15.78:0/1148279966 learned_addr learned my addr 172.21.15.78:0/1148279966 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.685+0000 7f4ce7fff640 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578499c0 0x55905786afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4cf4e83640 1 -- 172.21.15.78:0/1148279966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578499c0 msgr2=0x55905786afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4cf4e83640 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578499c0 0x55905786afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4cf4e83640 1 -- 172.21.15.78:0/1148279966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559057855950 msgr2=0x5590578677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4cf5684640 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559057855950 0x5590578677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4cf4e83640 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559057855950 0x5590578677c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4cf4e83640 1 -- 172.21.15.78:0/1148279966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559057867ff0 con 0x5590578490a0 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4cf5684640 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559057855950 0x5590578677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:08:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4cf4e83640 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578490a0 0x55905786aa40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4cec02a450 tx=0x7f4cec005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967551630691957e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4ccffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4ccffff640 1 -- 172.21.15.78:0/1148279966 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec035040 con 0x5590578490a0 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4ccffff640 1 -- 172.21.15.78:0/1148279966 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f4cec005040 con 0x5590578490a0 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4cf94bc740 1 -- 172.21.15.78:0/1148279966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce80046e0 con 0x5590578490a0 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4ccffff640 1 -- 172.21.15.78:0/1148279966 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4cec053050 con 0x5590578490a0 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4cf94bc740 10 client.17343 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4cf94bc740 1 -- 172.21.15.78:0/1148279966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5590578d2a90 con 0x5590578490a0 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4ccffff640 1 -- 172.21.15.78:0/1148279966 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce80046e0 con 0x5590578490a0 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.686+0000 7f4ccffff640 1 -- 172.21.15.78:0/1148279966 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f4cec004c50 con 0x5590578490a0 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4cf94bc740 10 client.17343 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4cf94bc740 10 client.17343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4cf94bc740 20 client.17343 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5590578d3290 0x5590578d56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4cf94bc740 4 client.17343 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4cf94bc740 1 -- 172.21.15.78:0/1148279966 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5590577bd850 con 0x5590578d3290 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4ce7fff640 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5590578d3290 0x5590578d56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4ccffff640 10 client.17343 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4ce7fff640 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5590578d3290 0x5590578d56b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffbb7e095db523b3 server_cookie=60fd62bd32c6ed5d in_seq=0 out_seq=0 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4ccffff640 1 -- 172.21.15.78:0/1148279966 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x5590577bd850 con 0x5590578d3290 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4ccffff640 10 client.17343 handle_command_reply: tid=0 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4cf94bc740 1 client.17343 shutdown 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4cf94bc740 1 -- 172.21.15.78:0/1148279966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578490a0 msgr2=0x55905786aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578490a0 0x55905786aa40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4cec02a450 tx=0x7f4cec005d50 comp rx=0 tx=0).stop 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.687+0000 7f4cf94bc740 1 -- 172.21.15.78:0/1148279966 shutdown_connections 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559057855950 0x5590578677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590578499c0 0x55905786afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5590578d3290 0x5590578d56b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf94bc740 1 --2- 172.21.15.78:0/1148279966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5590578490a0 0x55905786aa40 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf94bc740 1 -- 172.21.15.78:0/1148279966 >> 172.21.15.78:0/1148279966 conn(0x559057846ed0 msgr2=0x559057845f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4ccffff640 0 client.17343 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf94bc740 1 -- 172.21.15.78:0/1148279966 shutdown_connections 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf94bc740 1 -- 172.21.15.78:0/1148279966 wait complete. 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf94bc740 20 client.17343 trim_cache size 0 max 16384 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf70e7640 1 -- 172.21.15.78:0/687036674 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4cd80b5450 msgr2=0x7f4cd80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf70e7640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4cd80b5450 0x7f4cd80b7910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8005fd0 tx=0x7f4ce8009500 comp rx=0 tx=0).stop 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf70e7640 1 -- 172.21.15.78:0/687036674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf01087d0 msgr2=0x7f4cf01997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf70e7640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf01087d0 0x7f4cf01997e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000e9f0 tx=0x7f4ce000eeb0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf70e7640 1 -- 172.21.15.78:0/687036674 shutdown_connections 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf70e7640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4cd80b5450 0x7f4cd80b7910 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf70e7640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf0109150 0x7f4cf019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf70e7640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf01087d0 0x7f4cf01997e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf70e7640 1 --2- 172.21.15.78:0/687036674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cf0102dc0 0x7f4cf01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.688+0000 7f4cf70e7640 1 -- 172.21.15.78:0/687036674 >> 172.21.15.78:0/687036674 conn(0x7f4cf00fe830 msgr2=0x7f4cf0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:49.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.689+0000 7f4cf70e7640 1 -- 172.21.15.78:0/687036674 shutdown_connections 2024-09-11T01:08:49.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:49.689+0000 7f4cf70e7640 1 -- 172.21.15.78:0/687036674 wait complete. 2024-09-11T01:08:50.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 658675 5.22 MB/sec execute 418 sec latency 1388.098 ms 2024-09-11T01:08:50.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:50 smithi110 ceph-mon[34712]: pgmap v628: 97 pgs: 97 active+clean; 483 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-09-11T01:08:50.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:50 smithi112 ceph-mon[35384]: pgmap v628: 97 pgs: 97 active+clean; 483 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-09-11T01:08:50.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:50 smithi078 ceph-mon[31315]: pgmap v628: 97 pgs: 97 active+clean; 483 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-09-11T01:08:50.806 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:08:50.807 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:08:51.059 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:51.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 658675 5.21 MB/sec execute 419 sec latency 2388.197 ms 2024-09-11T01:08:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.562+0000 7f11d70e8640 1 -- 172.21.15.78:0/748710806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0109170 msgr2=0x7f11d010f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.562+0000 7f11d70e8640 1 --2- 172.21.15.78:0/748710806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0109170 0x7f11d010f970 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009ca0 tx=0x7f11c802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:08:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.562+0000 7f11d70e8640 1 -- 172.21.15.78:0/748710806 shutdown_connections 2024-09-11T01:08:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.562+0000 7f11d70e8640 1 --2- 172.21.15.78:0/748710806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0109170 0x7f11d010f970 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.562+0000 7f11d70e8640 1 --2- 172.21.15.78:0/748710806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d01087f0 0x7f11d0108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.562+0000 7f11d70e8640 1 --2- 172.21.15.78:0/748710806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d0102de0 0x7f11d01031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.562+0000 7f11d70e8640 1 -- 172.21.15.78:0/748710806 >> 172.21.15.78:0/748710806 conn(0x7f11d00fe830 msgr2=0x7f11d0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.562+0000 7f11d70e8640 1 -- 172.21.15.78:0/748710806 shutdown_connections 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.562+0000 7f11d70e8640 1 -- 172.21.15.78:0/748710806 wait complete. 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d70e8640 1 Processor -- start 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d70e8640 1 -- start start 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d70e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0102de0 0x7f11d019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d70e8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d01087f0 0x7f11d019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d70e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d0109170 0x7f11d01a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d70e8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0112e30 con 0x7f11d0102de0 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d70e8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0112cb0 con 0x7f11d01087f0 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d70e8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0112fb0 con 0x7f11d0109170 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d4e84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0102de0 0x7f11d019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d4e84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0102de0 0x7f11d019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47942/0 (socket says 172.21.15.78:47942) 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d4e84640 1 -- 172.21.15.78:0/2578035511 learned_addr learned my addr 172.21.15.78:0/2578035511 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11cffff640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d01087f0 0x7f11d019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d4e84640 1 -- 172.21.15.78:0/2578035511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d0109170 msgr2=0x7f11d01a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d4e84640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d0109170 0x7f11d01a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d4e84640 1 -- 172.21.15.78:0/2578035511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d01087f0 msgr2=0x7f11d019b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d4e84640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d01087f0 0x7f11d019b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d4e84640 1 -- 172.21.15.78:0/2578035511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d01a14e0 con 0x7f11d0102de0 2024-09-11T01:08:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11cffff640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d01087f0 0x7f11d019b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.563+0000 7f11d4e84640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0102de0 0x7f11d019b430 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f11c400c970 tx=0x7f11c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.564+0000 7f11cdffb640 1 -- 172.21.15.78:0/2578035511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4007bb0 con 0x7f11d0102de0 2024-09-11T01:08:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.564+0000 7f11cdffb640 1 -- 172.21.15.78:0/2578035511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c4007d50 con 0x7f11d0102de0 2024-09-11T01:08:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.564+0000 7f11d70e8640 1 -- 172.21.15.78:0/2578035511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d01a17d0 con 0x7f11d0102de0 2024-09-11T01:08:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.564+0000 7f11cdffb640 1 -- 172.21.15.78:0/2578035511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c400f450 con 0x7f11d0102de0 2024-09-11T01:08:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.564+0000 7f11d70e8640 1 -- 172.21.15.78:0/2578035511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d01a7e90 con 0x7f11d0102de0 2024-09-11T01:08:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.565+0000 7f11d70e8640 1 -- 172.21.15.78:0/2578035511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d01031c0 con 0x7f11d0102de0 2024-09-11T01:08:51.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.566+0000 7f11cdffb640 1 -- 172.21.15.78:0/2578035511 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f11c4004f20 con 0x7f11d0102de0 2024-09-11T01:08:51.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.567+0000 7f11cdffb640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f11c00b5450 0x7f11c00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:51.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.567+0000 7f11cffff640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f11c00b5450 0x7f11c00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:51.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.567+0000 7f11cdffb640 1 -- 172.21.15.78:0/2578035511 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f11c40e5e00 con 0x7f11d0102de0 2024-09-11T01:08:51.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.567+0000 7f11cffff640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f11c00b5450 0x7f11c00b7910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0058c0 tx=0x7f11bc005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:51.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.571+0000 7f11cdffb640 1 -- 172.21.15.78:0/2578035511 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f11c4014030 con 0x7f11d0102de0 2024-09-11T01:08:51.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.799+0000 7f11d70e8640 1 -- 172.21.15.78:0/2578035511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11d002e5c0 con 0x7f11d0102de0 2024-09-11T01:08:51.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.799+0000 7f11cdffb640 1 -- 172.21.15.78:0/2578035511 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11980 (secure 0 0 0) 0x7f11c40ac960 con 0x7f11d0102de0 2024-09-11T01:08:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:08:51.802 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":47,"btime":"2024-09-11T01:07:41:867729+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:07:41.817504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T01:08:51.802 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 47 2024-09-11T01:08:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.801+0000 7f11d70e8640 1 -- 172.21.15.78:0/2578035511 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f11c00b5450 msgr2=0x7f11c00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.801+0000 7f11d70e8640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f11c00b5450 0x7f11c00b7910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0058c0 tx=0x7f11bc005850 comp rx=0 tx=0).stop 2024-09-11T01:08:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.801+0000 7f11d70e8640 1 -- 172.21.15.78:0/2578035511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0102de0 msgr2=0x7f11d019b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.801+0000 7f11d70e8640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0102de0 0x7f11d019b430 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f11c400c970 tx=0x7f11c400ce30 comp rx=0 tx=0).stop 2024-09-11T01:08:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.802+0000 7f11d70e8640 1 -- 172.21.15.78:0/2578035511 shutdown_connections 2024-09-11T01:08:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.802+0000 7f11d70e8640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f11c00b5450 0x7f11c00b7910 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.802+0000 7f11d70e8640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d0109170 0x7f11d01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.802+0000 7f11d70e8640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d01087f0 0x7f11d019b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.802+0000 7f11d70e8640 1 --2- 172.21.15.78:0/2578035511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0102de0 0x7f11d019b430 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.802+0000 7f11d70e8640 1 -- 172.21.15.78:0/2578035511 >> 172.21.15.78:0/2578035511 conn(0x7f11d00fe830 msgr2=0x7f11d0104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.802+0000 7f11d70e8640 1 -- 172.21.15.78:0/2578035511 shutdown_connections 2024-09-11T01:08:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:51.802+0000 7f11d70e8640 1 -- 172.21.15.78:0/2578035511 wait complete. 2024-09-11T01:08:52.165 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:51 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2578035511' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:08:52.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:51 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2578035511' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:08:52.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:51 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2578035511' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:08:52.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 659386 5.20 MB/sec execute 420 sec latency 3135.249 ms 2024-09-11T01:08:52.521 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:08:52.788 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:53.109 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:52 smithi078 ceph-mon[31315]: pgmap v629: 97 pgs: 97 active+clean; 484 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-09-11T01:08:53.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:52 smithi110 ceph-mon[34712]: pgmap v629: 97 pgs: 97 active+clean; 484 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-09-11T01:08:53.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:52 smithi112 ceph-mon[35384]: pgmap v629: 97 pgs: 97 active+clean; 484 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-09-11T01:08:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.331+0000 7f360b7a4640 1 -- 172.21.15.78:0/4088660403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36041076a0 msgr2=0x7f3604107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.331+0000 7f360b7a4640 1 --2- 172.21.15.78:0/4088660403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36041076a0 0x7f3604107ae0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009900 tx=0x7f35f402a410 comp rx=0 tx=0).stop 2024-09-11T01:08:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.331+0000 7f360b7a4640 1 -- 172.21.15.78:0/4088660403 shutdown_connections 2024-09-11T01:08:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.331+0000 7f360b7a4640 1 --2- 172.21.15.78:0/4088660403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3604108020 0x7f360410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.331+0000 7f360b7a4640 1 --2- 172.21.15.78:0/4088660403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36041076a0 0x7f3604107ae0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.331+0000 7f360b7a4640 1 --2- 172.21.15.78:0/4088660403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3604101c90 0x7f3604102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.331+0000 7f360b7a4640 1 -- 172.21.15.78:0/4088660403 >> 172.21.15.78:0/4088660403 conn(0x7f36040fd700 msgr2=0x7f36040ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.331+0000 7f360b7a4640 1 -- 172.21.15.78:0/4088660403 shutdown_connections 2024-09-11T01:08:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.331+0000 7f360b7a4640 1 -- 172.21.15.78:0/4088660403 wait complete. 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f360b7a4640 1 Processor -- start 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f360b7a4640 1 -- start start 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f360b7a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3604101c90 0x7f360419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f360b7a4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36041076a0 0x7f360419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f360b7a4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3604108020 0x7f36041a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f360b7a4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3604111ca0 con 0x7f3604108020 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f360b7a4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3604111b20 con 0x7f36041076a0 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f360b7a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3604111e20 con 0x7f3604101c90 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f3608d3f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36041076a0 0x7f360419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f3609d41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3604108020 0x7f36041a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f3608d3f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36041076a0 0x7f360419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55584/0 (socket says 172.21.15.78:55584) 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f3608d3f640 1 -- 172.21.15.78:0/774594322 learned_addr learned my addr 172.21.15.78:0/774594322 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.332+0000 7f3609540640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3604101c90 0x7f360419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.333+0000 7f3608d3f640 1 -- 172.21.15.78:0/774594322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3604101c90 msgr2=0x7f360419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.333+0000 7f3608d3f640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3604101c90 0x7f360419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.333+0000 7f3608d3f640 1 -- 172.21.15.78:0/774594322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3604108020 msgr2=0x7f36041a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.333+0000 7f3608d3f640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3604108020 0x7f36041a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.333+0000 7f3608d3f640 1 -- 172.21.15.78:0/774594322 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36041a5630 con 0x7f36041076a0 2024-09-11T01:08:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.333+0000 7f3609d41640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3604108020 0x7f36041a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.333+0000 7f3608d3f640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36041076a0 0x7f360419a5b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f35f4005940 tx=0x7f35f40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.333+0000 7f35fa7fc640 1 -- 172.21.15.78:0/774594322 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f403f030 con 0x7f36041076a0 2024-09-11T01:08:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.333+0000 7f360b7a4640 1 -- 172.21.15.78:0/774594322 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3604197e60 con 0x7f36041076a0 2024-09-11T01:08:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.333+0000 7f35fa7fc640 1 -- 172.21.15.78:0/774594322 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f40044c0 con 0x7f36041076a0 2024-09-11T01:08:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.333+0000 7f35fa7fc640 1 -- 172.21.15.78:0/774594322 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4035070 con 0x7f36041076a0 2024-09-11T01:08:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.333+0000 7f360b7a4640 1 -- 172.21.15.78:0/774594322 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3604198390 con 0x7f36041076a0 2024-09-11T01:08:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.336+0000 7f35fa7fc640 1 -- 172.21.15.78:0/774594322 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f35f403ba20 con 0x7f36041076a0 2024-09-11T01:08:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f35fa7fc640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f35f00b5450 0x7f35f00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f3609540640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f35f00b5450 0x7f35f00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f360db79740 1 Processor -- start 2024-09-11T01:08:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f35fa7fc640 1 -- 172.21.15.78:0/774594322 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f35f4106570 con 0x7f36041076a0 2024-09-11T01:08:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f360db79740 1 -- start start 2024-09-11T01:08:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f3609540640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f35f00b5450 0x7f35f00b7910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3600009bd0 tx=0x7f3600005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f360db79740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 0x5608d9cc98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f360db79740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f360db79740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f360db79740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5608d9c3fea0 con 0x5608d9cd5d80 2024-09-11T01:08:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f360db79740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5608d9c2f2d0 con 0x5608d9cc9df0 2024-09-11T01:08:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f360db79740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5608d9c330e0 con 0x5608d9cc94d0 2024-09-11T01:08:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.337+0000 7f3609d41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f3609540640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 0x5608d9cc98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f3608d3f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f3609d41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50278/0 (socket says 172.21.15.78:50278) 2024-09-11T01:08:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f3609540640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 0x5608d9cc98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35576/0 (socket says 172.21.15.78:35576) 2024-09-11T01:08:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f3609d41640 1 -- 172.21.15.78:0/2699070937 learned_addr learned my addr 172.21.15.78:0/2699070937 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f3608d3f640 1 -- 172.21.15.78:0/2699070937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 msgr2=0x5608d9cc98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f3608d3f640 1 --2- 172.21.15.78:0/2699070937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 0x5608d9cc98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f3608d3f640 1 -- 172.21.15.78:0/2699070937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 msgr2=0x5608d9cd8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f3608d3f640 1 --2- 172.21.15.78:0/2699070937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f3608d3f640 1 -- 172.21.15.78:0/2699070937 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d998d650 con 0x5608d9cc9df0 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f3608d3f640 1 --2- 172.21.15.78:0/2699070937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd5840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35f4103d00 tx=0x7f35f40cc250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=481b891c6f20884d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f35ef7fe640 1 -- 172.21.15.78:0/2699070937 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4049040 con 0x5608d9cc9df0 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f35ef7fe640 1 -- 172.21.15.78:0/2699070937 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f40d1510 con 0x5608d9cc9df0 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.338+0000 7f35ef7fe640 1 -- 172.21.15.78:0/2699070937 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4103f80 con 0x5608d9cc9df0 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.339+0000 7f360db79740 1 -- 172.21.15.78:0/2699070937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 msgr2=0x5608d9cd5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.339+0000 7f360db79740 1 --2- 172.21.15.78:0/2699070937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd5840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35f4103d00 tx=0x7f35f40cc250 comp rx=0 tx=0).stop 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.339+0000 7f360db79740 1 -- 172.21.15.78:0/2699070937 shutdown_connections 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.339+0000 7f360db79740 1 --2- 172.21.15.78:0/2699070937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.339+0000 7f360db79740 1 --2- 172.21.15.78:0/2699070937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd5840 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.339+0000 7f360db79740 1 --2- 172.21.15.78:0/2699070937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 0x5608d9cc98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.339+0000 7f360db79740 1 -- 172.21.15.78:0/2699070937 >> 172.21.15.78:0/2699070937 conn(0x5608d9cc7300 msgr2=0x5608d9cc84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.339+0000 7f360db79740 1 -- 172.21.15.78:0/2699070937 shutdown_connections 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.339+0000 7f360db79740 1 -- 172.21.15.78:0/2699070937 wait complete. 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f360db79740 1 -- start start 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f360db79740 1 Processor -- start 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f360db79740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 0x5608d9cd5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f360db79740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f360db79740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f360db79740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5608d9a90700 con 0x5608d9cd5d80 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f360db79740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5608d9bca160 con 0x5608d9cc9df0 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f360db79740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5608d9cd87c0 con 0x5608d9cc94d0 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f360db79740 10 client.0 _ll_register_callbacks cb 0x5608d9be39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f3609d41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f3608d3f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f3609d41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50282/0 (socket says 172.21.15.78:50282) 2024-09-11T01:08:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.340+0000 7f3609d41640 1 -- 172.21.15.78:0/3007895999 learned_addr learned my addr 172.21.15.78:0/3007895999 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f3609d41640 1 -- 172.21.15.78:0/3007895999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 msgr2=0x5608d9cd5710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:08:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f3609d41640 1 --2- 172.21.15.78:0/3007895999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 0x5608d9cd5710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f3609d41640 1 -- 172.21.15.78:0/3007895999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 msgr2=0x5608d9cd2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f3609d41640 1 --2- 172.21.15.78:0/3007895999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f3609d41640 1 -- 172.21.15.78:0/3007895999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608d9bf7c90 con 0x5608d9cd5d80 2024-09-11T01:08:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f3608d3f640 1 --2- 172.21.15.78:0/3007895999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f3609d41640 1 --2- 172.21.15.78:0/3007895999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd3230 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00b340 tx=0x7f35fc00b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4948543935a6bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f35ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f35ecff9640 1 -- 172.21.15.78:0/3007895999 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc01f070 con 0x5608d9cd5d80 2024-09-11T01:08:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f35ecff9640 1 -- 172.21.15.78:0/3007895999 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f35fc004d20 con 0x5608d9cd5d80 2024-09-11T01:08:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f360db79740 1 -- 172.21.15.78:0/3007895999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f40d1510 con 0x5608d9cd5d80 2024-09-11T01:08:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f35ecff9640 1 -- 172.21.15.78:0/3007895999 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f35fc02a050 con 0x5608d9cd5d80 2024-09-11T01:08:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f360db79740 10 client.17361 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f360db79740 1 -- 172.21.15.78:0/3007895999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5608d9ce4130 con 0x5608d9cd5d80 2024-09-11T01:08:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.341+0000 7f35ecff9640 1 -- 172.21.15.78:0/3007895999 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f40d1510 con 0x5608d9cd5d80 2024-09-11T01:08:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.342+0000 7f35ecff9640 1 -- 172.21.15.78:0/3007895999 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f35fc005330 con 0x5608d9cd5d80 2024-09-11T01:08:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.342+0000 7f360db79740 10 client.17361 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.342+0000 7f360db79740 10 client.17361 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:08:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.342+0000 7f360db79740 20 client.17361 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.342+0000 7f360db79740 1 --2- 172.21.15.78:0/3007895999 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5608d9ce55e0 0x5608d9ce7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.342+0000 7f360db79740 4 client.17361 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.342+0000 7f360db79740 1 -- 172.21.15.78:0/3007895999 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608d9c41df0 con 0x5608d9ce55e0 2024-09-11T01:08:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.342+0000 7f3609540640 1 --2- 172.21.15.78:0/3007895999 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5608d9ce55e0 0x5608d9ce7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.342+0000 7f35ecff9640 10 client.17361 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:08:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.342+0000 7f3609540640 1 --2- 172.21.15.78:0/3007895999 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5608d9ce55e0 0x5608d9ce7a40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=811518329b7d260c server_cookie=a1cefaa7b01ade15 in_seq=0 out_seq=0 2024-09-11T01:08:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f35ecff9640 1 -- 172.21.15.78:0/3007895999 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5608d9c41df0 con 0x5608d9ce55e0 2024-09-11T01:08:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f35ecff9640 10 client.17361 handle_command_reply: tid=0 2024-09-11T01:08:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f360db79740 1 client.17361 shutdown 2024-09-11T01:08:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f360db79740 1 -- 172.21.15.78:0/3007895999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 msgr2=0x5608d9cd3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f360db79740 1 --2- 172.21.15.78:0/3007895999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd3230 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00b340 tx=0x7f35fc00b800 comp rx=0 tx=0).stop 2024-09-11T01:08:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f360db79740 1 -- 172.21.15.78:0/3007895999 shutdown_connections 2024-09-11T01:08:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f360db79740 1 --2- 172.21.15.78:0/3007895999 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5608d9ce55e0 0x5608d9ce7a40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f35ecff9640 0 client.17361 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:08:53.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f360db79740 1 --2- 172.21.15.78:0/3007895999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd3230 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f360db79740 1 --2- 172.21.15.78:0/3007895999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f360db79740 1 --2- 172.21.15.78:0/3007895999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 0x5608d9cd5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f360db79740 1 -- 172.21.15.78:0/3007895999 >> 172.21.15.78:0/3007895999 conn(0x5608d9cc7300 msgr2=0x5608d9cccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:53.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f360db79740 1 -- 172.21.15.78:0/3007895999 shutdown_connections 2024-09-11T01:08:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f360db79740 1 -- 172.21.15.78:0/3007895999 wait complete. 2024-09-11T01:08:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.343+0000 7f360db79740 20 client.17361 trim_cache size 0 max 16384 2024-09-11T01:08:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.357+0000 7f360db79740 1 Processor -- start 2024-09-11T01:08:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.357+0000 7f360db79740 1 -- start start 2024-09-11T01:08:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.357+0000 7f360db79740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 0x5608d9ce19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.357+0000 7f360db79740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f360db79740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f360db79740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36041ac3d0 con 0x5608d9cd5d80 2024-09-11T01:08:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f360db79740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36041adf40 con 0x5608d9cc9df0 2024-09-11T01:08:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f360db79740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f36041acaa0 con 0x5608d9cc94d0 2024-09-11T01:08:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f3609d41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f3609540640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 0x5608d9ce19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f3609d41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50294/0 (socket says 172.21.15.78:50294) 2024-09-11T01:08:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f3609d41640 1 -- 172.21.15.78:0/2452185382 learned_addr learned my addr 172.21.15.78:0/2452185382 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f3608d3f640 1 --2- 172.21.15.78:0/2452185382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f3609d41640 1 -- 172.21.15.78:0/2452185382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 msgr2=0x5608d9ce19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f3609d41640 1 --2- 172.21.15.78:0/2452185382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 0x5608d9ce19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f3609d41640 1 -- 172.21.15.78:0/2452185382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 msgr2=0x5608d9cd7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f3609d41640 1 --2- 172.21.15.78:0/2452185382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f3609d41640 1 -- 172.21.15.78:0/2452185382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d9cc89e0 con 0x5608d9cd5d80 2024-09-11T01:08:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f3609d41640 1 --2- 172.21.15.78:0/2452185382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd76b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0097a0 tx=0x7f35fc00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05ce44a83aedf21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f35eeffd640 1 -- 172.21.15.78:0/2452185382 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc01f030 con 0x5608d9cd5d80 2024-09-11T01:08:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f35eeffd640 1 -- 172.21.15.78:0/2452185382 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35fc005970 con 0x5608d9cd5d80 2024-09-11T01:08:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.358+0000 7f35eeffd640 1 -- 172.21.15.78:0/2452185382 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc01aa30 con 0x5608d9cd5d80 2024-09-11T01:08:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.359+0000 7f360db79740 1 -- 172.21.15.78:0/2452185382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 msgr2=0x5608d9cd76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.359+0000 7f360db79740 1 --2- 172.21.15.78:0/2452185382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd76b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0097a0 tx=0x7f35fc00bf50 comp rx=0 tx=0).stop 2024-09-11T01:08:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.359+0000 7f360db79740 1 -- 172.21.15.78:0/2452185382 shutdown_connections 2024-09-11T01:08:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.359+0000 7f360db79740 1 --2- 172.21.15.78:0/2452185382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cd5d80 0x5608d9cd76b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.359+0000 7f360db79740 1 --2- 172.21.15.78:0/2452185382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cd7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.359+0000 7f360db79740 1 --2- 172.21.15.78:0/2452185382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cc94d0 0x5608d9ce19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.359+0000 7f360db79740 1 -- 172.21.15.78:0/2452185382 >> 172.21.15.78:0/2452185382 conn(0x5608d9cc7300 msgr2=0x5608d9cd2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.359+0000 7f360db79740 1 -- 172.21.15.78:0/2452185382 shutdown_connections 2024-09-11T01:08:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.359+0000 7f360db79740 1 -- 172.21.15.78:0/2452185382 wait complete. 2024-09-11T01:08:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f360db79740 1 -- start start 2024-09-11T01:08:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f360db79740 1 Processor -- start 2024-09-11T01:08:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f360db79740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cc94d0 0x5608d9cd05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f360db79740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cdda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f360db79740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cd5d80 0x5608d9cddf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f360db79740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36041ad840 con 0x5608d9cc94d0 2024-09-11T01:08:53.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f360db79740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36041ad170 con 0x5608d9cc9df0 2024-09-11T01:08:53.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f360db79740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f36041aa3d0 con 0x5608d9cd5d80 2024-09-11T01:08:53.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f360db79740 10 client.0 _ll_register_callbacks cb 0x7f35fc010760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:53.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f3609540640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cc94d0 0x5608d9cd05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f3608d3f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cdda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f3609540640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cc94d0 0x5608d9cd05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50296/0 (socket says 172.21.15.78:50296) 2024-09-11T01:08:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f3609540640 1 -- 172.21.15.78:0/197584416 learned_addr learned my addr 172.21.15.78:0/197584416 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f3608d3f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cdda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55610/0 (socket says 172.21.15.78:55610) 2024-09-11T01:08:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.360+0000 7f3609d41640 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cd5d80 0x5608d9cddf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f3608d3f640 1 -- 172.21.15.78:0/197584416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cd5d80 msgr2=0x5608d9cddf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f3608d3f640 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cd5d80 0x5608d9cddf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f3608d3f640 1 -- 172.21.15.78:0/197584416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cc94d0 msgr2=0x5608d9cd05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f3608d3f640 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cc94d0 0x5608d9cd05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f3608d3f640 1 -- 172.21.15.78:0/197584416 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608d9cde780 con 0x5608d9cc9df0 2024-09-11T01:08:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f3609540640 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cc94d0 0x5608d9cd05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f3608d3f640 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cdda10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x5608d9cca4a0 tx=0x7f35f40ccc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9281dc794d5e4972 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f35ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:08:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f35ecff9640 1 -- 172.21.15.78:0/197584416 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4049040 con 0x5608d9cc9df0 2024-09-11T01:08:53.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f360db79740 1 -- 172.21.15.78:0/197584416 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35fc005970 con 0x5608d9cc9df0 2024-09-11T01:08:53.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f35ecff9640 1 -- 172.21.15.78:0/197584416 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f35f4103f80 con 0x5608d9cc9df0 2024-09-11T01:08:53.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f35ecff9640 1 -- 172.21.15.78:0/197584416 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f35f40d5fc0 con 0x5608d9cc9df0 2024-09-11T01:08:53.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f360db79740 10 client.26588 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f360db79740 1 -- 172.21.15.78:0/197584416 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5608d9d52600 con 0x5608d9cc9df0 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.361+0000 7f35ecff9640 1 -- 172.21.15.78:0/197584416 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35fc005970 con 0x5608d9cc9df0 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f35ecff9640 1 -- 172.21.15.78:0/197584416 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f35f40d5b30 con 0x5608d9cc9df0 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f360db79740 10 client.26588 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f360db79740 10 client.26588 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f360db79740 20 client.26588 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f360db79740 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5608d9ce55e0 0x5608d9d54b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f360db79740 4 client.26588 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f360db79740 1 -- 172.21.15.78:0/197584416 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608d9c3d8a0 con 0x5608d9ce55e0 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f3609540640 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5608d9ce55e0 0x5608d9d54b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f35ecff9640 10 client.26588 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f3609540640 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5608d9ce55e0 0x5608d9d54b90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a0d4acfd5217318 server_cookie=73b9a1c871741c68 in_seq=0 out_seq=0 2024-09-11T01:08:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f35ecff9640 1 -- 172.21.15.78:0/197584416 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608d9c3d8a0 con 0x5608d9ce55e0 2024-09-11T01:08:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f35ecff9640 10 client.26588 handle_command_reply: tid=0 2024-09-11T01:08:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f360db79740 1 client.26588 shutdown 2024-09-11T01:08:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f360db79740 1 -- 172.21.15.78:0/197584416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 msgr2=0x5608d9cdda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.362+0000 7f360db79740 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cdda10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x5608d9cca4a0 tx=0x7f35f40ccc60 comp rx=0 tx=0).stop 2024-09-11T01:08:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360db79740 1 -- 172.21.15.78:0/197584416 shutdown_connections 2024-09-11T01:08:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360db79740 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5608d9ce55e0 0x5608d9d54b90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360db79740 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608d9cd5d80 0x5608d9cddf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360db79740 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608d9cc9df0 0x5608d9cdda10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f35ecff9640 0 client.26588 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:08:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360db79740 1 --2- 172.21.15.78:0/197584416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608d9cc94d0 0x5608d9cd05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360db79740 1 -- 172.21.15.78:0/197584416 >> 172.21.15.78:0/197584416 conn(0x5608d9cc7300 msgr2=0x5608d9cc5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:53.416 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:08:53.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360db79740 1 -- 172.21.15.78:0/197584416 shutdown_connections 2024-09-11T01:08:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360db79740 1 -- 172.21.15.78:0/197584416 wait complete. 2024-09-11T01:08:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360db79740 20 client.26588 trim_cache size 0 max 16384 2024-09-11T01:08:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360b7a4640 1 -- 172.21.15.78:0/774594322 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f35f00b5450 msgr2=0x7f35f00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360b7a4640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f35f00b5450 0x7f35f00b7910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3600009bd0 tx=0x7f3600005f90 comp rx=0 tx=0).stop 2024-09-11T01:08:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360b7a4640 1 -- 172.21.15.78:0/774594322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36041076a0 msgr2=0x7f360419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360b7a4640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36041076a0 0x7f360419a5b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f35f4005940 tx=0x7f35f40040c0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360b7a4640 1 -- 172.21.15.78:0/774594322 shutdown_connections 2024-09-11T01:08:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360b7a4640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f35f00b5450 0x7f35f00b7910 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360b7a4640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3604108020 0x7f36041a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360b7a4640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36041076a0 0x7f360419a5b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360b7a4640 1 --2- 172.21.15.78:0/774594322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3604101c90 0x7f360419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.363+0000 7f360b7a4640 1 -- 172.21.15.78:0/774594322 >> 172.21.15.78:0/774594322 conn(0x7f36040fd700 msgr2=0x7f36040ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.364+0000 7f360b7a4640 1 -- 172.21.15.78:0/774594322 shutdown_connections 2024-09-11T01:08:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:53.364+0000 7f360b7a4640 1 -- 172.21.15.78:0/774594322 wait complete. 2024-09-11T01:08:53.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 660998 5.22 MB/sec execute 421 sec latency 13.122 ms 2024-09-11T01:08:54.190 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:08:54.440 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:54.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 662234 5.22 MB/sec execute 422 sec latency 301.733 ms 2024-09-11T01:08:54.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:54 smithi078 ceph-mon[31315]: pgmap v630: 97 pgs: 97 active+clean; 486 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-09-11T01:08:54.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:54 smithi110 ceph-mon[34712]: pgmap v630: 97 pgs: 97 active+clean; 486 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-09-11T01:08:54.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:54 smithi112 ceph-mon[35384]: pgmap v630: 97 pgs: 97 active+clean; 486 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-09-11T01:08:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.226+0000 7f551df38640 1 -- 172.21.15.78:0/4035454472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55181076d0 msgr2=0x7f5518107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.226+0000 7f551df38640 1 --2- 172.21.15.78:0/4035454472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55181076d0 0x7f5518107b10 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5500009900 tx=0x7f550002a410 comp rx=0 tx=0).stop 2024-09-11T01:08:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.226+0000 7f551df38640 1 -- 172.21.15.78:0/4035454472 shutdown_connections 2024-09-11T01:08:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.226+0000 7f551df38640 1 --2- 172.21.15.78:0/4035454472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5518108050 0x7f551810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.226+0000 7f551df38640 1 --2- 172.21.15.78:0/4035454472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55181076d0 0x7f5518107b10 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.226+0000 7f551df38640 1 --2- 172.21.15.78:0/4035454472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5518101cc0 0x7f55181020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.226+0000 7f551df38640 1 -- 172.21.15.78:0/4035454472 >> 172.21.15.78:0/4035454472 conn(0x7f55180fd770 msgr2=0x7f55180ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.226+0000 7f551df38640 1 -- 172.21.15.78:0/4035454472 shutdown_connections 2024-09-11T01:08:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.226+0000 7f551df38640 1 -- 172.21.15.78:0/4035454472 wait complete. 2024-09-11T01:08:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f551df38640 1 Processor -- start 2024-09-11T01:08:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f551df38640 1 -- start start 2024-09-11T01:08:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f551df38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5518101cc0 0x7f5518197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f551df38640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55181076d0 0x7f5518198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f551df38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5518108050 0x7f55181989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f551df38640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5518111d10 con 0x7f5518101cc0 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f551df38640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5518111b90 con 0x7f55181076d0 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f551df38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5518111e90 con 0x7f5518108050 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f55177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5518101cc0 0x7f5518197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f55177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5518101cc0 0x7f5518197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50308/0 (socket says 172.21.15.78:50308) 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f55177fe640 1 -- 172.21.15.78:0/2544484364 learned_addr learned my addr 172.21.15.78:0/2544484364 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f5517fff640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5518108050 0x7f55181989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.227+0000 7f550ffff640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55181076d0 0x7f5518198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.228+0000 7f55177fe640 1 -- 172.21.15.78:0/2544484364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5518108050 msgr2=0x7f55181989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.228+0000 7f55177fe640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5518108050 0x7f55181989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.228+0000 7f55177fe640 1 -- 172.21.15.78:0/2544484364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55181076d0 msgr2=0x7f5518198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.228+0000 7f55177fe640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55181076d0 0x7f5518198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.228+0000 7f55177fe640 1 -- 172.21.15.78:0/2544484364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55181a7700 con 0x7f5518101cc0 2024-09-11T01:08:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.228+0000 7f55177fe640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5518101cc0 0x7f5518197f20 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f550800e9f0 tx=0x7f550800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:55.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.228+0000 7f55157fa640 1 -- 172.21.15.78:0/2544484364 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550800cd80 con 0x7f5518101cc0 2024-09-11T01:08:55.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.228+0000 7f55157fa640 1 -- 172.21.15.78:0/2544484364 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55080054d0 con 0x7f5518101cc0 2024-09-11T01:08:55.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.228+0000 7f551df38640 1 -- 172.21.15.78:0/2544484364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55181a79f0 con 0x7f5518101cc0 2024-09-11T01:08:55.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.228+0000 7f551df38640 1 -- 172.21.15.78:0/2544484364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55181a7f00 con 0x7f5518101cc0 2024-09-11T01:08:55.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.228+0000 7f55157fa640 1 -- 172.21.15.78:0/2544484364 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508010690 con 0x7f5518101cc0 2024-09-11T01:08:55.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.231+0000 7f552030d740 1 Processor -- start 2024-09-11T01:08:55.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.231+0000 7f552030d740 1 -- start start 2024-09-11T01:08:55.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.231+0000 7f552030d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 0x555902c1b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.231+0000 7f552030d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1bdf0 0x555902c27840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.231+0000 7f552030d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 0x555902c2a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.231+0000 7f552030d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555902b91ea0 con 0x555902c1b4d0 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.231+0000 7f552030d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555902b812d0 con 0x555902c1bdf0 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.231+0000 7f552030d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555902b850e0 con 0x555902c27d80 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f55157fa640 1 -- 172.21.15.78:0/2544484364 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5508004e90 con 0x7f5518101cc0 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f5517fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 0x555902c1b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f5517fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 0x555902c1b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50318/0 (socket says 172.21.15.78:50318) 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f5517fff640 1 -- 172.21.15.78:0/3772966130 learned_addr learned my addr 172.21.15.78:0/3772966130 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f55177fe640 1 --2- 172.21.15.78:0/3772966130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1bdf0 0x555902c27840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f550ffff640 1 --2- 172.21.15.78:0/3772966130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 0x555902c2a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f5517fff640 1 -- 172.21.15.78:0/3772966130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 msgr2=0x555902c2a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f5517fff640 1 --2- 172.21.15.78:0/3772966130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 0x555902c2a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f5517fff640 1 -- 172.21.15.78:0/3772966130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1bdf0 msgr2=0x555902c27840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f5517fff640 1 --2- 172.21.15.78:0/3772966130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1bdf0 0x555902c27840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f5517fff640 1 -- 172.21.15.78:0/3772966130 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559028df650 con 0x555902c1b4d0 2024-09-11T01:08:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f55177fe640 1 --2- 172.21.15.78:0/3772966130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1bdf0 0x555902c27840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:08:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f5517fff640 1 --2- 172.21.15.78:0/3772966130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 0x555902c1b8b0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5510009f50 tx=0x7f55100118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a827c5cef6291a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.232+0000 7f550dffb640 1 -- 172.21.15.78:0/3772966130 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510004d20 con 0x555902c1b4d0 2024-09-11T01:08:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.233+0000 7f550dffb640 1 -- 172.21.15.78:0/3772966130 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5510004050 con 0x555902c1b4d0 2024-09-11T01:08:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.233+0000 7f550dffb640 1 -- 172.21.15.78:0/3772966130 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510005280 con 0x555902c1b4d0 2024-09-11T01:08:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.233+0000 7f55157fa640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55040b5550 0x7f55040b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.233+0000 7f55177fe640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55040b5550 0x7f55040b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.233+0000 7f55157fa640 1 -- 172.21.15.78:0/2544484364 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f55080e6360 con 0x7f5518101cc0 2024-09-11T01:08:55.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.233+0000 7f55177fe640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55040b5550 0x7f55040b7a10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x555902c1a1e0 tx=0x7f55080af040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.233+0000 7f552030d740 1 -- 172.21.15.78:0/3772966130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 msgr2=0x555902c1b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.233+0000 7f552030d740 1 --2- 172.21.15.78:0/3772966130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 0x555902c1b8b0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5510009f50 tx=0x7f55100118d0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 -- 172.21.15.78:0/3772966130 shutdown_connections 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 --2- 172.21.15.78:0/3772966130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 0x555902c2a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 --2- 172.21.15.78:0/3772966130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1bdf0 0x555902c27840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 --2- 172.21.15.78:0/3772966130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 0x555902c1b8b0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 -- 172.21.15.78:0/3772966130 >> 172.21.15.78:0/3772966130 conn(0x555902c19300 msgr2=0x555902c1a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 -- 172.21.15.78:0/3772966130 shutdown_connections 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 -- 172.21.15.78:0/3772966130 wait complete. 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 -- start start 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 Processor -- start 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1b4d0 0x555902c27690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 0x555902c24ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 0x555902c25110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.234+0000 7f552030d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5559029e2700 con 0x555902c1bdf0 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f552030d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555902b1c160 con 0x555902c1b4d0 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f552030d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555902c2aa00 con 0x555902c27d80 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f552030d740 10 client.0 _ll_register_callbacks cb 0x555902b359e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f550ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 0x555902c24ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f5517fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1b4d0 0x555902c27690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f5517fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1b4d0 0x555902c27690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55650/0 (socket says 172.21.15.78:55650) 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f5517fff640 1 -- 172.21.15.78:0/1152289205 learned_addr learned my addr 172.21.15.78:0/1152289205 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f5517fff640 1 --2- 172.21.15.78:0/1152289205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 0x555902c25110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f550ffff640 1 -- 172.21.15.78:0/1152289205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 msgr2=0x555902c25110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f550ffff640 1 --2- 172.21.15.78:0/1152289205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 0x555902c25110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f550ffff640 1 -- 172.21.15.78:0/1152289205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1b4d0 msgr2=0x555902c27690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f550ffff640 1 --2- 172.21.15.78:0/1152289205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1b4d0 0x555902c27690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f550ffff640 1 -- 172.21.15.78:0/1152289205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555902b49c90 con 0x555902c1bdf0 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f550ffff640 1 --2- 172.21.15.78:0/1152289205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 0x555902c24ae0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5500009230 tx=0x7f5500009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac80fc2fde5a414 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f54f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f54f6ffd640 1 -- 172.21.15.78:0/1152289205 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500035040 con 0x555902c1bdf0 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f54f6ffd640 1 -- 172.21.15.78:0/1152289205 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f5500031720 con 0x555902c1bdf0 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.235+0000 7f552030d740 1 -- 172.21.15.78:0/1152289205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5510004050 con 0x555902c1bdf0 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.236+0000 7f54f6ffd640 1 -- 172.21.15.78:0/1152289205 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5500004040 con 0x555902c1bdf0 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.236+0000 7f552030d740 10 client.17379 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.236+0000 7f552030d740 1 -- 172.21.15.78:0/1152289205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555902c36300 con 0x555902c1bdf0 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.236+0000 7f54f6ffd640 1 -- 172.21.15.78:0/1152289205 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5510004050 con 0x555902c1bdf0 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.236+0000 7f54f6ffd640 1 -- 172.21.15.78:0/1152289205 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f5500004750 con 0x555902c1bdf0 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.236+0000 7f552030d740 10 client.17379 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.236+0000 7f552030d740 10 client.17379 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:08:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.236+0000 7f552030d740 20 client.17379 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:55.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.236+0000 7f552030d740 1 --2- 172.21.15.78:0/1152289205 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x555902c377c0 0x555902c39bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.236+0000 7f552030d740 4 client.17379 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:55.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.236+0000 7f552030d740 1 -- 172.21.15.78:0/1152289205 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555902b93df0 con 0x555902c377c0 2024-09-11T01:08:55.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.236+0000 7f5517fff640 1 --2- 172.21.15.78:0/1152289205 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x555902c377c0 0x555902c39bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.237+0000 7f5517fff640 1 --2- 172.21.15.78:0/1152289205 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x555902c377c0 0x555902c39bb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=60dd35cbb27a5aed server_cookie=328a1180c2dec96e in_seq=0 out_seq=0 2024-09-11T01:08:55.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.237+0000 7f54f6ffd640 10 client.17379 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:08:55.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.237+0000 7f54f6ffd640 1 -- 172.21.15.78:0/1152289205 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555902b93df0 con 0x555902c377c0 2024-09-11T01:08:55.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.237+0000 7f54f6ffd640 10 client.17379 handle_command_reply: tid=0 2024-09-11T01:08:55.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.237+0000 7f552030d740 1 client.17379 shutdown 2024-09-11T01:08:55.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.238+0000 7f552030d740 1 -- 172.21.15.78:0/1152289205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 msgr2=0x555902c24ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:55.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.238+0000 7f552030d740 1 --2- 172.21.15.78:0/1152289205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 0x555902c24ae0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5500009230 tx=0x7f5500009660 comp rx=0 tx=0).stop 2024-09-11T01:08:55.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.238+0000 7f552030d740 1 -- 172.21.15.78:0/1152289205 shutdown_connections 2024-09-11T01:08:55.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.238+0000 7f552030d740 1 --2- 172.21.15.78:0/1152289205 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x555902c377c0 0x555902c39bb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.238+0000 7f552030d740 1 --2- 172.21.15.78:0/1152289205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 0x555902c25110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.238+0000 7f54f6ffd640 0 client.17379 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:08:55.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.238+0000 7f552030d740 1 --2- 172.21.15.78:0/1152289205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 0x555902c24ae0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.238+0000 7f552030d740 1 --2- 172.21.15.78:0/1152289205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1b4d0 0x555902c27690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.238+0000 7f552030d740 1 -- 172.21.15.78:0/1152289205 >> 172.21.15.78:0/1152289205 conn(0x555902c19300 msgr2=0x555902c1ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:55.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.238+0000 7f552030d740 1 -- 172.21.15.78:0/1152289205 shutdown_connections 2024-09-11T01:08:55.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.238+0000 7f552030d740 1 -- 172.21.15.78:0/1152289205 wait complete. 2024-09-11T01:08:55.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.238+0000 7f552030d740 20 client.17379 trim_cache size 0 max 16384 2024-09-11T01:08:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.252+0000 7f552030d740 1 Processor -- start 2024-09-11T01:08:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.252+0000 7f552030d740 1 -- start start 2024-09-11T01:08:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.252+0000 7f552030d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c1b4d0 0x555902c33c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.252+0000 7f552030d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 0x555902c1a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.252+0000 7f552030d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c27d80 0x555902c1a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.252+0000 7f552030d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55181b07e0 con 0x555902c1bdf0 2024-09-11T01:08:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.252+0000 7f552030d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55181b2350 con 0x555902c27d80 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.252+0000 7f552030d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55181b0eb0 con 0x555902c1b4d0 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f550ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 0x555902c1a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f5517fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c1b4d0 0x555902c33c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f550ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 0x555902c1a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50332/0 (socket says 172.21.15.78:50332) 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f550ffff640 1 -- 172.21.15.78:0/966003593 learned_addr learned my addr 172.21.15.78:0/966003593 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f5517fff640 1 --2- 172.21.15.78:0/966003593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c27d80 0x555902c1a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f550ffff640 1 -- 172.21.15.78:0/966003593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c1b4d0 msgr2=0x555902c33c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f550ffff640 1 --2- 172.21.15.78:0/966003593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c1b4d0 0x555902c33c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f550ffff640 1 -- 172.21.15.78:0/966003593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c27d80 msgr2=0x555902c1a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f550ffff640 1 --2- 172.21.15.78:0/966003593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c27d80 0x555902c1a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f550ffff640 1 -- 172.21.15.78:0/966003593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555902c14990 con 0x555902c1bdf0 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f5517fff640 1 --2- 172.21.15.78:0/966003593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c1b4d0 0x555902c33c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f550ffff640 1 --2- 172.21.15.78:0/966003593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 0x555902c1a430 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f550002a930 tx=0x7f550002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def394e0979ba139 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f550d7fa640 1 -- 172.21.15.78:0/966003593 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500035070 con 0x555902c1bdf0 2024-09-11T01:08:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f550d7fa640 1 -- 172.21.15.78:0/966003593 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5500004470 con 0x555902c1bdf0 2024-09-11T01:08:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.253+0000 7f550d7fa640 1 -- 172.21.15.78:0/966003593 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500053070 con 0x555902c1bdf0 2024-09-11T01:08:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.254+0000 7f552030d740 1 -- 172.21.15.78:0/966003593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 msgr2=0x555902c1a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.254+0000 7f552030d740 1 --2- 172.21.15.78:0/966003593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 0x555902c1a430 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f550002a930 tx=0x7f550002a410 comp rx=0 tx=0).stop 2024-09-11T01:08:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.254+0000 7f552030d740 1 -- 172.21.15.78:0/966003593 shutdown_connections 2024-09-11T01:08:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.254+0000 7f552030d740 1 --2- 172.21.15.78:0/966003593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c27d80 0x555902c1a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.254+0000 7f552030d740 1 --2- 172.21.15.78:0/966003593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1bdf0 0x555902c1a430 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.254+0000 7f552030d740 1 --2- 172.21.15.78:0/966003593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c1b4d0 0x555902c33c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.254+0000 7f552030d740 1 -- 172.21.15.78:0/966003593 >> 172.21.15.78:0/966003593 conn(0x555902c19300 msgr2=0x555902c1fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.254+0000 7f552030d740 1 -- 172.21.15.78:0/966003593 shutdown_connections 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.254+0000 7f552030d740 1 -- 172.21.15.78:0/966003593 wait complete. 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f552030d740 1 -- start start 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f552030d740 1 Processor -- start 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f552030d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 0x555902c2feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f552030d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1bdf0 0x555902c30430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f552030d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 0x555902c2ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f552030d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55181b1c50 con 0x555902c1b4d0 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f552030d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55181b1580 con 0x555902c1bdf0 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f552030d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55181ae7e0 con 0x555902c27d80 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f552030d740 10 client.0 _ll_register_callbacks cb 0x7f5500009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f5517fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 0x555902c2feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f550ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1bdf0 0x555902c30430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f5517fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 0x555902c2feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50346/0 (socket says 172.21.15.78:50346) 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.255+0000 7f5517fff640 1 -- 172.21.15.78:0/1185089359 learned_addr learned my addr 172.21.15.78:0/1185089359 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f5517fff640 1 --2- 172.21.15.78:0/1185089359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 0x555902c2ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f550ffff640 1 -- 172.21.15.78:0/1185089359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 msgr2=0x555902c2ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f550ffff640 1 --2- 172.21.15.78:0/1185089359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 0x555902c2ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f550ffff640 1 -- 172.21.15.78:0/1185089359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 msgr2=0x555902c2feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f550ffff640 1 --2- 172.21.15.78:0/1185089359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 0x555902c2feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f550ffff640 1 -- 172.21.15.78:0/1185089359 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555902c2b4b0 con 0x555902c1bdf0 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f550ffff640 1 --2- 172.21.15.78:0/1185089359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1bdf0 0x555902c30430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5500004ad0 tx=0x7f55000039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b14d68bb5cfafbc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f54f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f54f6ffd640 1 -- 172.21.15.78:0/1185089359 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500035070 con 0x555902c1bdf0 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f552030d740 1 -- 172.21.15.78:0/1185089359 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5500004470 con 0x555902c1bdf0 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f54f6ffd640 1 -- 172.21.15.78:0/1185089359 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f5500005150 con 0x555902c1bdf0 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f54f6ffd640 1 -- 172.21.15.78:0/1185089359 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5500067050 con 0x555902c1bdf0 2024-09-11T01:08:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f552030d740 10 client.26603 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.256+0000 7f552030d740 1 -- 172.21.15.78:0/1185089359 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555902ca5100 con 0x555902c1bdf0 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.257+0000 7f54f6ffd640 1 -- 172.21.15.78:0/1185089359 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5500004470 con 0x555902c1bdf0 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.257+0000 7f54f6ffd640 1 -- 172.21.15.78:0/1185089359 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f55000053e0 con 0x555902c1bdf0 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.257+0000 7f552030d740 10 client.26603 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.257+0000 7f552030d740 10 client.26603 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.257+0000 7f552030d740 20 client.26603 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.257+0000 7f552030d740 1 --2- 172.21.15.78:0/1185089359 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x555902c377c0 0x555902ca7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.257+0000 7f552030d740 4 client.26603 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.257+0000 7f552030d740 1 -- 172.21.15.78:0/1185089359 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555902b8f8a0 con 0x555902c377c0 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.257+0000 7f5517fff640 1 --2- 172.21.15.78:0/1185089359 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x555902c377c0 0x555902ca7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.257+0000 7f5517fff640 1 --2- 172.21.15.78:0/1185089359 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x555902c377c0 0x555902ca7770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5e8d555fa78c3556 server_cookie=84ea41f3cb78fec4 in_seq=0 out_seq=0 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.257+0000 7f54f6ffd640 10 client.26603 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f54f6ffd640 1 -- 172.21.15.78:0/1185089359 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555902b8f8a0 con 0x555902c377c0 2024-09-11T01:08:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f54f6ffd640 10 client.26603 handle_command_reply: tid=0 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f552030d740 1 client.26603 shutdown 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f552030d740 1 -- 172.21.15.78:0/1185089359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1bdf0 msgr2=0x555902c30430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f552030d740 1 --2- 172.21.15.78:0/1185089359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1bdf0 0x555902c30430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5500004ad0 tx=0x7f55000039e0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f552030d740 1 -- 172.21.15.78:0/1185089359 shutdown_connections 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f552030d740 1 --2- 172.21.15.78:0/1185089359 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x555902c377c0 0x555902ca7770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f552030d740 1 --2- 172.21.15.78:0/1185089359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555902c27d80 0x555902c2ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f552030d740 1 --2- 172.21.15.78:0/1185089359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555902c1bdf0 0x555902c30430 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f552030d740 1 --2- 172.21.15.78:0/1185089359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555902c1b4d0 0x555902c2feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f552030d740 1 -- 172.21.15.78:0/1185089359 >> 172.21.15.78:0/1185089359 conn(0x555902c19300 msgr2=0x555902c1a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f54f6ffd640 0 client.26603 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f552030d740 1 -- 172.21.15.78:0/1185089359 shutdown_connections 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f552030d740 1 -- 172.21.15.78:0/1185089359 wait complete. 2024-09-11T01:08:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.258+0000 7f552030d740 20 client.26603 trim_cache size 0 max 16384 2024-09-11T01:08:55.265 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:08:55.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.259+0000 7f551df38640 1 -- 172.21.15.78:0/2544484364 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55040b5550 msgr2=0x7f55040b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:55.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.259+0000 7f551df38640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55040b5550 0x7f55040b7a10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x555902c1a1e0 tx=0x7f55080af040 comp rx=0 tx=0).stop 2024-09-11T01:08:55.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.259+0000 7f551df38640 1 -- 172.21.15.78:0/2544484364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5518101cc0 msgr2=0x7f5518197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:55.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.259+0000 7f551df38640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5518101cc0 0x7f5518197f20 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f550800e9f0 tx=0x7f550800edf0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.259+0000 7f551df38640 1 -- 172.21.15.78:0/2544484364 shutdown_connections 2024-09-11T01:08:55.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.259+0000 7f551df38640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55040b5550 0x7f55040b7a10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.259+0000 7f551df38640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5518108050 0x7f55181989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.259+0000 7f551df38640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55181076d0 0x7f5518198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.259+0000 7f551df38640 1 --2- 172.21.15.78:0/2544484364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5518101cc0 0x7f5518197f20 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:55.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.259+0000 7f551df38640 1 -- 172.21.15.78:0/2544484364 >> 172.21.15.78:0/2544484364 conn(0x7f55180fd770 msgr2=0x7f55180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:55.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.259+0000 7f551df38640 1 -- 172.21.15.78:0/2544484364 shutdown_connections 2024-09-11T01:08:55.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:55.259+0000 7f551df38640 1 -- 172.21.15.78:0/2544484364 wait complete. 2024-09-11T01:08:55.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 664314 5.24 MB/sec execute 423 sec latency 64.857 ms 2024-09-11T01:08:55.988 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:08:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:56.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 665722 5.24 MB/sec execute 424 sec latency 14.671 ms 2024-09-11T01:08:56.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:56 smithi078 ceph-mon[31315]: pgmap v631: 97 pgs: 97 active+clean; 492 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-09-11T01:08:56.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:56 smithi110 ceph-mon[34712]: pgmap v631: 97 pgs: 97 active+clean; 492 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-09-11T01:08:56.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:56 smithi112 ceph-mon[35384]: pgmap v631: 97 pgs: 97 active+clean; 492 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-09-11T01:08:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.756+0000 7fcde9c5d640 1 -- 172.21.15.78:0/2771209387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde41087d0 msgr2=0x7fcde4108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.756+0000 7fcde9c5d640 1 --2- 172.21.15.78:0/2771209387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde41087d0 0x7fcde4108c10 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009900 tx=0x7fcdcc02a410 comp rx=0 tx=0).stop 2024-09-11T01:08:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.756+0000 7fcde9c5d640 1 -- 172.21.15.78:0/2771209387 shutdown_connections 2024-09-11T01:08:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.756+0000 7fcde9c5d640 1 --2- 172.21.15.78:0/2771209387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde4109150 0x7fcde410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.756+0000 7fcde9c5d640 1 --2- 172.21.15.78:0/2771209387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde41087d0 0x7fcde4108c10 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.756+0000 7fcde9c5d640 1 --2- 172.21.15.78:0/2771209387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcde4102dc0 0x7fcde41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.756+0000 7fcde9c5d640 1 -- 172.21.15.78:0/2771209387 >> 172.21.15.78:0/2771209387 conn(0x7fcde40fe830 msgr2=0x7fcde4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.756+0000 7fcde9c5d640 1 -- 172.21.15.78:0/2771209387 shutdown_connections 2024-09-11T01:08:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.756+0000 7fcde9c5d640 1 -- 172.21.15.78:0/2771209387 wait complete. 2024-09-11T01:08:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.756+0000 7fcde9c5d640 1 Processor -- start 2024-09-11T01:08:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.756+0000 7fcde9c5d640 1 -- start start 2024-09-11T01:08:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde9c5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcde4102dc0 0x7fcde40754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde9c5d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde41087d0 0x7fcde40759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde9c5d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde4109150 0x7fcde4075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde9c5d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcde4112e50 con 0x7fcde4109150 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde9c5d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcde4112cd0 con 0x7fcde41087d0 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde9c5d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcde4112fd0 con 0x7fcde4102dc0 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde4109150 0x7fcde4075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde4109150 0x7fcde4075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50362/0 (socket says 172.21.15.78:50362) 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde3fff640 1 -- 172.21.15.78:0/478512798 learned_addr learned my addr 172.21.15.78:0/478512798 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde37fe640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcde4102dc0 0x7fcde40754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcddbfff640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde41087d0 0x7fcde40759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde3fff640 1 -- 172.21.15.78:0/478512798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcde4102dc0 msgr2=0x7fcde40754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde3fff640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcde4102dc0 0x7fcde40754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde3fff640 1 -- 172.21.15.78:0/478512798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde41087d0 msgr2=0x7fcde40759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde3fff640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde41087d0 0x7fcde40759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde3fff640 1 -- 172.21.15.78:0/478512798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde41a7910 con 0x7fcde4109150 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcddbfff640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde41087d0 0x7fcde40759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:08:56.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde3fff640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde4109150 0x7fcde4075f20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00b970 tx=0x7fcddc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde17fa640 1 -- 172.21.15.78:0/478512798 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc019070 con 0x7fcde4109150 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde17fa640 1 -- 172.21.15.78:0/478512798 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcddc014440 con 0x7fcde4109150 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde9c5d640 1 -- 172.21.15.78:0/478512798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde41a7c00 con 0x7fcde4109150 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde17fa640 1 -- 172.21.15.78:0/478512798 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0075d0 con 0x7fcde4109150 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.757+0000 7fcde9c5d640 1 -- 172.21.15.78:0/478512798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde41a8110 con 0x7fcde4109150 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcde17fa640 1 -- 172.21.15.78:0/478512798 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fcddc0145f0 con 0x7fcde4109150 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcdec032740 1 Processor -- start 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcdec032740 1 -- start start 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcdec032740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5d4d0 0x55b445e5d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcdec032740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5ddf0 0x55b445e69840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcdec032740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445e6c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcdec032740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b445dd3ea0 con 0x55b445e5d4d0 2024-09-11T01:08:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcdec032740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b445dc32d0 con 0x55b445e69d80 2024-09-11T01:08:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcdec032740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b445dc70e0 con 0x55b445e5ddf0 2024-09-11T01:08:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5d4d0 0x55b445e5d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcde37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5ddf0 0x55b445e69840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5d4d0 0x55b445e5d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50370/0 (socket says 172.21.15.78:50370) 2024-09-11T01:08:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.760+0000 7fcddbfff640 1 -- 172.21.15.78:0/3012106344 learned_addr learned my addr 172.21.15.78:0/3012106344 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcddbfff640 1 --2- 172.21.15.78:0/3012106344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445e6c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcde37fe640 1 -- 172.21.15.78:0/3012106344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 msgr2=0x55b445e6c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcde37fe640 1 --2- 172.21.15.78:0/3012106344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445e6c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcde37fe640 1 -- 172.21.15.78:0/3012106344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5d4d0 msgr2=0x55b445e5d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcde37fe640 1 --2- 172.21.15.78:0/3012106344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5d4d0 0x55b445e5d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcde37fe640 1 -- 172.21.15.78:0/3012106344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b445b21650 con 0x55b445e5ddf0 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcde17fa640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdd00b54d0 0x7fcdd00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcddbfff640 1 --2- 172.21.15.78:0/3012106344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5d4d0 0x55b445e5d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcddbfff640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdd00b54d0 0x7fcdd00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcde37fe640 1 --2- 172.21.15.78:0/3012106344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5ddf0 0x55b445e69840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40099a0 tx=0x7fcdd4009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29d62407ac56eee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcde17fa640 1 -- 172.21.15.78:0/478512798 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7fcddc0e56c0 con 0x7fcde4109150 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcdd9ffb640 1 -- 172.21.15.78:0/3012106344 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4017070 con 0x55b445e5ddf0 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcdd9ffb640 1 -- 172.21.15.78:0/3012106344 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd4013470 con 0x55b445e5ddf0 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.761+0000 7fcdd9ffb640 1 -- 172.21.15.78:0/3012106344 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4012630 con 0x55b445e5ddf0 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.762+0000 7fcdec032740 1 -- 172.21.15.78:0/3012106344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5ddf0 msgr2=0x55b445e69840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.762+0000 7fcddbfff640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdd00b54d0 0x7fcdd00b7990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc005ec0 tx=0x7fcdcc03f040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.762+0000 7fcdec032740 1 --2- 172.21.15.78:0/3012106344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5ddf0 0x55b445e69840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40099a0 tx=0x7fcdd4009e60 comp rx=0 tx=0).stop 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.762+0000 7fcdec032740 1 -- 172.21.15.78:0/3012106344 shutdown_connections 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.762+0000 7fcdec032740 1 --2- 172.21.15.78:0/3012106344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445e6c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.762+0000 7fcdec032740 1 --2- 172.21.15.78:0/3012106344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5ddf0 0x55b445e69840 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.762+0000 7fcdec032740 1 --2- 172.21.15.78:0/3012106344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5d4d0 0x55b445e5d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.762+0000 7fcdec032740 1 -- 172.21.15.78:0/3012106344 >> 172.21.15.78:0/3012106344 conn(0x55b445e5b300 msgr2=0x55b445e5c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.762+0000 7fcdec032740 1 -- 172.21.15.78:0/3012106344 shutdown_connections 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.762+0000 7fcdec032740 1 -- 172.21.15.78:0/3012106344 wait complete. 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.762+0000 7fcdec032740 1 -- start start 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcdec032740 1 Processor -- start 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcdec032740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e69650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcdec032740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e66aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcdec032740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445e67080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcdec032740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b445c24700 con 0x55b445e5ddf0 2024-09-11T01:08:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcdec032740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b445d5e160 con 0x55b445e69d80 2024-09-11T01:08:56.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcdec032740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b445e6ca00 con 0x55b445e5d4d0 2024-09-11T01:08:56.824 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:08:56.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcdec032740 10 client.0 _ll_register_callbacks cb 0x55b445d779e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e66aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e66aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50376/0 (socket says 172.21.15.78:50376) 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcddbfff640 1 -- 172.21.15.78:0/546719994 learned_addr learned my addr 172.21.15.78:0/546719994 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcde37fe640 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e69650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcde3fff640 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445e67080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcde37fe640 1 -- 172.21.15.78:0/546719994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 msgr2=0x55b445e67080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcde37fe640 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445e67080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcde37fe640 1 -- 172.21.15.78:0/546719994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 msgr2=0x55b445e66aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcde37fe640 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e66aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcde37fe640 1 -- 172.21.15.78:0/546719994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b445d8bc90 con 0x55b445e5d4d0 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcde37fe640 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e69650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4000c00 tx=0x7fcdd400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5957790f163d0d93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.763+0000 7fcdc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:08:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdc2ffd640 1 -- 172.21.15.78:0/546719994 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4017050 con 0x55b445e5d4d0 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdec032740 1 -- 172.21.15.78:0/546719994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd4013470 con 0x55b445e5d4d0 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdc2ffd640 1 -- 172.21.15.78:0/546719994 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7fcdd401c450 con 0x55b445e5d4d0 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdc2ffd640 1 -- 172.21.15.78:0/546719994 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fcdd401c690 con 0x55b445e5d4d0 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdec032740 10 client.26533 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdec032740 1 -- 172.21.15.78:0/546719994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b445e78310 con 0x55b445e5d4d0 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdc2ffd640 1 -- 172.21.15.78:0/546719994 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd4013470 con 0x55b445e5d4d0 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdc2ffd640 1 -- 172.21.15.78:0/546719994 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7fcdd4026610 con 0x55b445e5d4d0 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdec032740 10 client.26533 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdec032740 10 client.26533 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdec032740 20 client.26533 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdec032740 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b445e79660 0x55b445e7ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdec032740 4 client.26533 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.764+0000 7fcdec032740 1 -- 172.21.15.78:0/546719994 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b445dd5df0 con 0x55b445e79660 2024-09-11T01:08:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.765+0000 7fcddbfff640 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b445e79660 0x55b445e7ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.765+0000 7fcdc2ffd640 10 client.26533 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.765+0000 7fcddbfff640 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b445e79660 0x55b445e7ba50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c6ebc1f624b3a8 server_cookie=623010e97a8e0cf5 in_seq=0 out_seq=0 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.765+0000 7fcdc2ffd640 1 -- 172.21.15.78:0/546719994 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b445dd5df0 con 0x55b445e79660 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.765+0000 7fcdc2ffd640 10 client.26533 handle_command_reply: tid=0 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdec032740 1 client.26533 shutdown 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdec032740 1 -- 172.21.15.78:0/546719994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 msgr2=0x55b445e69650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdec032740 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e69650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4000c00 tx=0x7fcdd400eeb0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdec032740 1 -- 172.21.15.78:0/546719994 shutdown_connections 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdec032740 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b445e79660 0x55b445e7ba50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdec032740 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445e67080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdc2ffd640 0 client.26533 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdec032740 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e66aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdec032740 1 --2- 172.21.15.78:0/546719994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e69650 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdec032740 1 -- 172.21.15.78:0/546719994 >> 172.21.15.78:0/546719994 conn(0x55b445e5b300 msgr2=0x55b445e60e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdec032740 1 -- 172.21.15.78:0/546719994 shutdown_connections 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdec032740 1 -- 172.21.15.78:0/546719994 wait complete. 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.766+0000 7fcdec032740 20 client.26533 trim_cache size 0 max 16384 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcdec032740 1 Processor -- start 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcdec032740 1 -- start start 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcdec032740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e5ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcdec032740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e5bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcdec032740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445e5c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcdec032740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41b0990 con 0x55b445e5ddf0 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcdec032740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41b2500 con 0x55b445e69d80 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcdec032740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41b1060 con 0x55b445e5d4d0 2024-09-11T01:08:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e5bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e5bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50388/0 (socket says 172.21.15.78:50388) 2024-09-11T01:08:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcddbfff640 1 -- 172.21.15.78:0/1467170922 learned_addr learned my addr 172.21.15.78:0/1467170922 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcde37fe640 1 --2- 172.21.15.78:0/1467170922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e5ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcddbfff640 1 -- 172.21.15.78:0/1467170922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 msgr2=0x55b445e5ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcddbfff640 1 --2- 172.21.15.78:0/1467170922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e5ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcddbfff640 1 -- 172.21.15.78:0/1467170922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 msgr2=0x55b445e5c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcde3fff640 1 --2- 172.21.15.78:0/1467170922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445e5c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcddbfff640 1 --2- 172.21.15.78:0/1467170922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445e5c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.779+0000 7fcddbfff640 1 -- 172.21.15.78:0/1467170922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b445e7bdc0 con 0x55b445e5ddf0 2024-09-11T01:08:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcddbfff640 1 --2- 172.21.15.78:0/1467170922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e5bfe0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009900 tx=0x7fcdcc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9903e3173881a57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcdd97fa640 1 -- 172.21.15.78:0/1467170922 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc043070 con 0x55b445e5ddf0 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcdd97fa640 1 -- 172.21.15.78:0/1467170922 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdcc057070 con 0x55b445e5ddf0 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcdd97fa640 1 -- 172.21.15.78:0/1467170922 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc03d6b0 con 0x55b445e5ddf0 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcdec032740 1 -- 172.21.15.78:0/1467170922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 msgr2=0x55b445e5bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcdec032740 1 --2- 172.21.15.78:0/1467170922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e5bfe0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009900 tx=0x7fcdcc009260 comp rx=0 tx=0).stop 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcdec032740 1 -- 172.21.15.78:0/1467170922 shutdown_connections 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcdec032740 1 --2- 172.21.15.78:0/1467170922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445e5c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcdec032740 1 --2- 172.21.15.78:0/1467170922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e5bfe0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcdec032740 1 --2- 172.21.15.78:0/1467170922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e5ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcdec032740 1 -- 172.21.15.78:0/1467170922 >> 172.21.15.78:0/1467170922 conn(0x55b445e5b300 msgr2=0x55b445e61cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcdec032740 1 -- 172.21.15.78:0/1467170922 shutdown_connections 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.780+0000 7fcdec032740 1 -- 172.21.15.78:0/1467170922 wait complete. 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcdec032740 1 -- start start 2024-09-11T01:08:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcdec032740 1 Processor -- start 2024-09-11T01:08:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcdec032740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e6e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcdec032740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e6ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcdec032740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445edf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcdec032740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41b1e00 con 0x55b445e5ddf0 2024-09-11T01:08:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcdec032740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41b1730 con 0x55b445e69d80 2024-09-11T01:08:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcdec032740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcde41ae990 con 0x55b445e5d4d0 2024-09-11T01:08:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcdec032740 10 client.0 _ll_register_callbacks cb 0x7fcdcc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e6ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcde37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e6e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e6ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50394/0 (socket says 172.21.15.78:50394) 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcddbfff640 1 -- 172.21.15.78:0/1846558350 learned_addr learned my addr 172.21.15.78:0/1846558350 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.781+0000 7fcde3fff640 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445edf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcde37fe640 1 -- 172.21.15.78:0/1846558350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 msgr2=0x55b445edf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcde37fe640 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445edf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcde37fe640 1 -- 172.21.15.78:0/1846558350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 msgr2=0x55b445e6ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcde37fe640 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e6ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcde37fe640 1 -- 172.21.15.78:0/1846558350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b445e5dc20 con 0x55b445e5d4d0 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcddbfff640 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e6ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcde37fe640 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e6e6e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40269e0 tx=0x7fcdd4009110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68b3bfb23f10feb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcdc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcdc2ffd640 1 -- 172.21.15.78:0/1846558350 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4033020 con 0x55b445e5d4d0 2024-09-11T01:08:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcdec032740 1 -- 172.21.15.78:0/1846558350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdcc057070 con 0x55b445e5d4d0 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcdc2ffd640 1 -- 172.21.15.78:0/1846558350 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7fcdd4026620 con 0x55b445e5d4d0 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcdc2ffd640 1 -- 172.21.15.78:0/1846558350 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fcdd403e050 con 0x55b445e5d4d0 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcdec032740 10 client.26539 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcdec032740 1 -- 172.21.15.78:0/1846558350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b445ee6c80 con 0x55b445e5d4d0 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcdc2ffd640 1 -- 172.21.15.78:0/1846558350 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdcc057070 con 0x55b445e5d4d0 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.782+0000 7fcdc2ffd640 1 -- 172.21.15.78:0/1846558350 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7fcdd401c950 con 0x55b445e5d4d0 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcdec032740 10 client.26539 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcdec032740 10 client.26539 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcdec032740 20 client.26539 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcdec032740 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b445ee7300 0x55b445ee9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcdec032740 4 client.26539 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcdec032740 1 -- 172.21.15.78:0/1846558350 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b445dd18a0 con 0x55b445ee7300 2024-09-11T01:08:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcddbfff640 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b445ee7300 0x55b445ee9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:56.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcdc2ffd640 10 client.26539 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:08:56.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcddbfff640 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b445ee7300 0x55b445ee9720 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=458ca65413c65e38 server_cookie=2901da0bcd1d972f in_seq=0 out_seq=0 2024-09-11T01:08:56.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcdc2ffd640 1 -- 172.21.15.78:0/1846558350 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b445dd18a0 con 0x55b445ee7300 2024-09-11T01:08:56.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcdc2ffd640 10 client.26539 handle_command_reply: tid=0 2024-09-11T01:08:56.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcdec032740 1 client.26539 shutdown 2024-09-11T01:08:56.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcdec032740 1 -- 172.21.15.78:0/1846558350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 msgr2=0x55b445e6e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:56.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.783+0000 7fcdec032740 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e6e6e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40269e0 tx=0x7fcdd4009110 comp rx=0 tx=0).stop 2024-09-11T01:08:56.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcdec032740 1 -- 172.21.15.78:0/1846558350 shutdown_connections 2024-09-11T01:08:56.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcdec032740 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b445ee7300 0x55b445ee9720 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcdec032740 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b445e69d80 0x55b445edf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcdc2ffd640 0 client.26539 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:08:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcdec032740 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b445e5ddf0 0x55b445e6ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcdec032740 1 --2- 172.21.15.78:0/1846558350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b445e5d4d0 0x55b445e6e6e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcdec032740 1 -- 172.21.15.78:0/1846558350 >> 172.21.15.78:0/1846558350 conn(0x55b445e5b300 msgr2=0x55b445e5bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcdec032740 1 -- 172.21.15.78:0/1846558350 shutdown_connections 2024-09-11T01:08:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcdec032740 1 -- 172.21.15.78:0/1846558350 wait complete. 2024-09-11T01:08:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcdec032740 20 client.26539 trim_cache size 0 max 16384 2024-09-11T01:08:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcde9c5d640 1 -- 172.21.15.78:0/478512798 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdd00b54d0 msgr2=0x7fcdd00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcde9c5d640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdd00b54d0 0x7fcdd00b7990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc005ec0 tx=0x7fcdcc03f040 comp rx=0 tx=0).stop 2024-09-11T01:08:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcde9c5d640 1 -- 172.21.15.78:0/478512798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde4109150 msgr2=0x7fcde4075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcde9c5d640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde4109150 0x7fcde4075f20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00b970 tx=0x7fcddc00bd70 comp rx=0 tx=0).stop 2024-09-11T01:08:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcde9c5d640 1 -- 172.21.15.78:0/478512798 shutdown_connections 2024-09-11T01:08:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcde9c5d640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdd00b54d0 0x7fcdd00b7990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcde9c5d640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde4109150 0x7fcde4075f20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcde9c5d640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde41087d0 0x7fcde40759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcde9c5d640 1 --2- 172.21.15.78:0/478512798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcde4102dc0 0x7fcde40754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcde9c5d640 1 -- 172.21.15.78:0/478512798 >> 172.21.15.78:0/478512798 conn(0x7fcde40fe830 msgr2=0x7fcde4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcde9c5d640 1 -- 172.21.15.78:0/478512798 shutdown_connections 2024-09-11T01:08:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:56.784+0000 7fcde9c5d640 1 -- 172.21.15.78:0/478512798 wait complete. 2024-09-11T01:08:57.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 666480 5.23 MB/sec execute 425 sec latency 713.502 ms 2024-09-11T01:08:57.607 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:08:57.876 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.399+0000 7f28df719640 1 -- 172.21.15.78:0/2565969632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d8101c90 msgr2=0x7f28d8102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.399+0000 7f28df719640 1 --2- 172.21.15.78:0/2565969632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d8101c90 0x7f28d8102070 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009ca0 tx=0x7f28c802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.400+0000 7f28df719640 1 -- 172.21.15.78:0/2565969632 shutdown_connections 2024-09-11T01:08:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.400+0000 7f28df719640 1 --2- 172.21.15.78:0/2565969632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108020 0x7f28d810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.400+0000 7f28df719640 1 --2- 172.21.15.78:0/2565969632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d81076a0 0x7f28d8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.400+0000 7f28df719640 1 --2- 172.21.15.78:0/2565969632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d8101c90 0x7f28d8102070 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.400+0000 7f28df719640 1 -- 172.21.15.78:0/2565969632 >> 172.21.15.78:0/2565969632 conn(0x7f28d80fd700 msgr2=0x7f28d80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.400+0000 7f28df719640 1 -- 172.21.15.78:0/2565969632 shutdown_connections 2024-09-11T01:08:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.400+0000 7f28df719640 1 -- 172.21.15.78:0/2565969632 wait complete. 2024-09-11T01:08:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.400+0000 7f28df719640 1 Processor -- start 2024-09-11T01:08:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.400+0000 7f28df719640 1 -- start start 2024-09-11T01:08:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.400+0000 7f28df719640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d8101c90 0x7f28d81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.400+0000 7f28df719640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d81076a0 0x7f28d8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.400+0000 7f28df719640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108020 0x7f28d819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28df719640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8112e00 con 0x7f28d8101c90 2024-09-11T01:08:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28df719640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8112c80 con 0x7f28d8108020 2024-09-11T01:08:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28df719640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8112f80 con 0x7f28d81076a0 2024-09-11T01:08:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28dd4b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d8101c90 0x7f28d81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28dd4b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d8101c90 0x7f28d81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50410/0 (socket says 172.21.15.78:50410) 2024-09-11T01:08:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28dd4b5640 1 -- 172.21.15.78:0/468950946 learned_addr learned my addr 172.21.15.78:0/468950946 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28dccb4640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d81076a0 0x7f28d8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108020 0x7f28d819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28dd4b5640 1 -- 172.21.15.78:0/468950946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d81076a0 msgr2=0x7f28d8199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28dd4b5640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d81076a0 0x7f28d8199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28dd4b5640 1 -- 172.21.15.78:0/468950946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108020 msgr2=0x7f28d819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28dd4b5640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108020 0x7f28d819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28dd4b5640 1 -- 172.21.15.78:0/468950946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d819f3a0 con 0x7f28d8101c90 2024-09-11T01:08:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28dccb4640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d81076a0 0x7f28d8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108020 0x7f28d819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28dd4b5640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d8101c90 0x7f28d81992f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f28c8033040 tx=0x7f28c80316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28d67fc640 1 -- 172.21.15.78:0/468950946 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8003b70 con 0x7f28d8101c90 2024-09-11T01:08:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28d67fc640 1 -- 172.21.15.78:0/468950946 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c8003d10 con 0x7f28d8101c90 2024-09-11T01:08:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28df719640 1 -- 172.21.15.78:0/468950946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d819f630 con 0x7f28d8101c90 2024-09-11T01:08:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.401+0000 7f28d67fc640 1 -- 172.21.15.78:0/468950946 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c802f9e0 con 0x7f28d8101c90 2024-09-11T01:08:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.402+0000 7f28df719640 1 -- 172.21.15.78:0/468950946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d81a5d50 con 0x7f28d8101c90 2024-09-11T01:08:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.404+0000 7f28d67fc640 1 -- 172.21.15.78:0/468950946 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f28c802fb80 con 0x7f28d8101c90 2024-09-11T01:08:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.404+0000 7f28e1aee740 1 Processor -- start 2024-09-11T01:08:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.404+0000 7f28e1aee740 1 -- start start 2024-09-11T01:08:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28e1aee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28e1aee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e34840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28e1aee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 0x556bc2e37170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28e1aee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556bc2d9eea0 con 0x556bc2e28df0 2024-09-11T01:08:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28e1aee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556bc2d8e2d0 con 0x556bc2e284d0 2024-09-11T01:08:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28e1aee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556bc2d920e0 con 0x556bc2e34d80 2024-09-11T01:08:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28dd4b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e34840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28ddcb6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28ddcb6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e288b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55746/0 (socket says 172.21.15.78:55746) 2024-09-11T01:08:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28dd4b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e34840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50412/0 (socket says 172.21.15.78:50412) 2024-09-11T01:08:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28ddcb6640 1 -- 172.21.15.78:0/1692439155 learned_addr learned my addr 172.21.15.78:0/1692439155 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28dccb4640 1 --2- 172.21.15.78:0/1692439155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 0x556bc2e37170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28ddcb6640 1 -- 172.21.15.78:0/1692439155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 msgr2=0x556bc2e37170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/1692439155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 0x556bc2e37170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28ddcb6640 1 -- 172.21.15.78:0/1692439155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 msgr2=0x556bc2e34840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/1692439155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e34840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28ddcb6640 1 -- 172.21.15.78:0/1692439155 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc2aec650 con 0x556bc2e284d0 2024-09-11T01:08:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28d67fc640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f28c00b54d0 0x7f28c00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28dd4b5640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f28c00b54d0 0x7f28c00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/1692439155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e288b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f28d0011540 tx=0x7f28d0011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c19354a258ffc84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.405+0000 7f28d67fc640 1 -- 172.21.15.78:0/468950946 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f28c8107340 con 0x7f28d8101c90 2024-09-11T01:08:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.406+0000 7f28bf7fe640 1 -- 172.21.15.78:0/1692439155 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d000f5c0 con 0x556bc2e284d0 2024-09-11T01:08:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.406+0000 7f28bf7fe640 1 -- 172.21.15.78:0/1692439155 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d000fc10 con 0x556bc2e284d0 2024-09-11T01:08:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.406+0000 7f28bf7fe640 1 -- 172.21.15.78:0/1692439155 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0019bf0 con 0x556bc2e284d0 2024-09-11T01:08:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.406+0000 7f28e1aee740 1 -- 172.21.15.78:0/1692439155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 msgr2=0x556bc2e288b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.406+0000 7f28dd4b5640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f28c00b54d0 0x7f28c00b7990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f28c8104b80 tx=0x7f28c810c040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.406+0000 7f28e1aee740 1 --2- 172.21.15.78:0/1692439155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e288b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f28d0011540 tx=0x7f28d0011a00 comp rx=0 tx=0).stop 2024-09-11T01:08:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.406+0000 7f28e1aee740 1 -- 172.21.15.78:0/1692439155 shutdown_connections 2024-09-11T01:08:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.406+0000 7f28e1aee740 1 --2- 172.21.15.78:0/1692439155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 0x556bc2e37170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.406+0000 7f28e1aee740 1 --2- 172.21.15.78:0/1692439155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e34840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.406+0000 7f28e1aee740 1 --2- 172.21.15.78:0/1692439155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e288b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.406+0000 7f28e1aee740 1 -- 172.21.15.78:0/1692439155 >> 172.21.15.78:0/1692439155 conn(0x556bc2e26300 msgr2=0x556bc2e274a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.407+0000 7f28e1aee740 1 -- 172.21.15.78:0/1692439155 shutdown_connections 2024-09-11T01:08:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.407+0000 7f28e1aee740 1 -- 172.21.15.78:0/1692439155 wait complete. 2024-09-11T01:08:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.407+0000 7f28e1aee740 1 -- start start 2024-09-11T01:08:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.407+0000 7f28e1aee740 1 Processor -- start 2024-09-11T01:08:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.407+0000 7f28e1aee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.407+0000 7f28e1aee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e31b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.407+0000 7f28e1aee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 0x556bc2e32180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.407+0000 7f28e1aee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556bc2bef700 con 0x556bc2e28df0 2024-09-11T01:08:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.407+0000 7f28e1aee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556bc2d29160 con 0x556bc2e284d0 2024-09-11T01:08:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.407+0000 7f28e1aee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556bc2e37a00 con 0x556bc2e34d80 2024-09-11T01:08:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.407+0000 7f28e1aee740 10 client.0 _ll_register_callbacks cb 0x556bc2d429e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28dccb4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e31b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28dccb4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e31b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50416/0 (socket says 172.21.15.78:50416) 2024-09-11T01:08:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28dccb4640 1 -- 172.21.15.78:0/1498097987 learned_addr learned my addr 172.21.15.78:0/1498097987 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 0x556bc2e32180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28dccb4640 1 -- 172.21.15.78:0/1498097987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 msgr2=0x556bc2e32180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28dccb4640 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 0x556bc2e32180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28dccb4640 1 -- 172.21.15.78:0/1498097987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 msgr2=0x556bc2e346d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28dccb4640 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e346d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28dccb4640 1 -- 172.21.15.78:0/1498097987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc2d56c90 con 0x556bc2e28df0 2024-09-11T01:08:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28dccb4640 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e31b20 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f28cc012d20 tx=0x7f28cc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4ad2f24b95e1fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28bcff9640 1 -- 172.21.15.78:0/1498097987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc012110 con 0x556bc2e28df0 2024-09-11T01:08:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28bcff9640 1 -- 172.21.15.78:0/1498097987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f28cc004050 con 0x556bc2e28df0 2024-09-11T01:08:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28e1aee740 1 -- 172.21.15.78:0/1498097987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d000fc10 con 0x556bc2e28df0 2024-09-11T01:08:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28bcff9640 1 -- 172.21.15.78:0/1498097987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f28cc0122b0 con 0x556bc2e28df0 2024-09-11T01:08:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28e1aee740 10 client.17415 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28e1aee740 1 -- 172.21.15.78:0/1498097987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x556bc2e433f0 con 0x556bc2e28df0 2024-09-11T01:08:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.408+0000 7f28bcff9640 1 -- 172.21.15.78:0/1498097987 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d000fc10 con 0x556bc2e28df0 2024-09-11T01:08:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.409+0000 7f28bcff9640 1 -- 172.21.15.78:0/1498097987 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f28cc010430 con 0x556bc2e28df0 2024-09-11T01:08:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.409+0000 7f28e1aee740 10 client.17415 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.409+0000 7f28e1aee740 10 client.17415 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:08:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.409+0000 7f28e1aee740 20 client.17415 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.409+0000 7f28e1aee740 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556bc2e448b0 0x556bc2e46cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.409+0000 7f28e1aee740 4 client.17415 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.409+0000 7f28e1aee740 1 -- 172.21.15.78:0/1498097987 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bc2da0df0 con 0x556bc2e448b0 2024-09-11T01:08:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.409+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556bc2e448b0 0x556bc2e46cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.409+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556bc2e448b0 0x556bc2e46cc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f6dce168635229b server_cookie=86de7382516b3f76 in_seq=0 out_seq=0 2024-09-11T01:08:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.409+0000 7f28bcff9640 10 client.17415 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:08:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28bcff9640 1 -- 172.21.15.78:0/1498097987 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556bc2da0df0 con 0x556bc2e448b0 2024-09-11T01:08:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28bcff9640 10 client.17415 handle_command_reply: tid=0 2024-09-11T01:08:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28e1aee740 1 client.17415 shutdown 2024-09-11T01:08:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28e1aee740 1 -- 172.21.15.78:0/1498097987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 msgr2=0x556bc2e31b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28e1aee740 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e31b20 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f28cc012d20 tx=0x7f28cc00f790 comp rx=0 tx=0).stop 2024-09-11T01:08:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28e1aee740 1 -- 172.21.15.78:0/1498097987 shutdown_connections 2024-09-11T01:08:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28e1aee740 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 0x556bc2e32180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28e1aee740 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556bc2e448b0 0x556bc2e46cc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28e1aee740 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e31b20 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28bcff9640 0 client.17415 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:08:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28e1aee740 1 --2- 172.21.15.78:0/1498097987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e346d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28e1aee740 1 -- 172.21.15.78:0/1498097987 >> 172.21.15.78:0/1498097987 conn(0x556bc2e26300 msgr2=0x556bc2e2bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28e1aee740 1 -- 172.21.15.78:0/1498097987 shutdown_connections 2024-09-11T01:08:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28e1aee740 1 -- 172.21.15.78:0/1498097987 wait complete. 2024-09-11T01:08:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.410+0000 7f28e1aee740 20 client.17415 trim_cache size 0 max 16384 2024-09-11T01:08:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.424+0000 7f28e1aee740 1 Processor -- start 2024-09-11T01:08:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.424+0000 7f28e1aee740 1 -- start start 2024-09-11T01:08:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.424+0000 7f28e1aee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e284d0 0x556bc2e40c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.424+0000 7f28e1aee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e28df0 0x556bc2e31010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.424+0000 7f28e1aee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e34d80 0x556bc2e273c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.424+0000 7f28e1aee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81ae650 con 0x556bc2e34d80 2024-09-11T01:08:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.424+0000 7f28e1aee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b01c0 con 0x556bc2e28df0 2024-09-11T01:08:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.424+0000 7f28e1aee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81aed20 con 0x556bc2e284d0 2024-09-11T01:08:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28ddcb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e284d0 0x556bc2e40c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28ddcb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e284d0 0x556bc2e40c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35770/0 (socket says 172.21.15.78:35770) 2024-09-11T01:08:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28dccb4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e28df0 0x556bc2e31010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28ddcb6640 1 -- 172.21.15.78:0/2809810741 learned_addr learned my addr 172.21.15.78:0/2809810741 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/2809810741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e34d80 0x556bc2e273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28ddcb6640 1 -- 172.21.15.78:0/2809810741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e28df0 msgr2=0x556bc2e31010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/2809810741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e28df0 0x556bc2e31010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28ddcb6640 1 -- 172.21.15.78:0/2809810741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e34d80 msgr2=0x556bc2e273c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/2809810741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e34d80 0x556bc2e273c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28ddcb6640 1 -- 172.21.15.78:0/2809810741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc2e324f0 con 0x556bc2e284d0 2024-09-11T01:08:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/2809810741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e284d0 0x556bc2e40c30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f28d001b040 tx=0x7f28d0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c57ff26190187e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28beffd640 1 -- 172.21.15.78:0/2809810741 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d00050e0 con 0x556bc2e284d0 2024-09-11T01:08:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28beffd640 1 -- 172.21.15.78:0/2809810741 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d00198c0 con 0x556bc2e284d0 2024-09-11T01:08:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.425+0000 7f28beffd640 1 -- 172.21.15.78:0/2809810741 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d000f6b0 con 0x556bc2e284d0 2024-09-11T01:08:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.426+0000 7f28e1aee740 1 -- 172.21.15.78:0/2809810741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e284d0 msgr2=0x556bc2e40c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.426+0000 7f28e1aee740 1 --2- 172.21.15.78:0/2809810741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e284d0 0x556bc2e40c30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f28d001b040 tx=0x7f28d0005f90 comp rx=0 tx=0).stop 2024-09-11T01:08:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.426+0000 7f28e1aee740 1 -- 172.21.15.78:0/2809810741 shutdown_connections 2024-09-11T01:08:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.426+0000 7f28e1aee740 1 --2- 172.21.15.78:0/2809810741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e34d80 0x556bc2e273c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.426+0000 7f28e1aee740 1 --2- 172.21.15.78:0/2809810741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e28df0 0x556bc2e31010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.426+0000 7f28e1aee740 1 --2- 172.21.15.78:0/2809810741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e284d0 0x556bc2e40c30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.426+0000 7f28e1aee740 1 -- 172.21.15.78:0/2809810741 >> 172.21.15.78:0/2809810741 conn(0x556bc2e26300 msgr2=0x556bc2e303a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.426+0000 7f28e1aee740 1 -- 172.21.15.78:0/2809810741 shutdown_connections 2024-09-11T01:08:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.426+0000 7f28e1aee740 1 -- 172.21.15.78:0/2809810741 wait complete. 2024-09-11T01:08:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.426+0000 7f28e1aee740 1 -- start start 2024-09-11T01:08:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28e1aee740 1 Processor -- start 2024-09-11T01:08:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28e1aee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e3ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28e1aee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e3d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28e1aee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 0x556bc2e37c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28e1aee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81afac0 con 0x556bc2e28df0 2024-09-11T01:08:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28e1aee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81af3f0 con 0x556bc2e284d0 2024-09-11T01:08:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28e1aee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81ac650 con 0x556bc2e34d80 2024-09-11T01:08:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28e1aee740 10 client.0 _ll_register_callbacks cb 0x7f28cc009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28dccb4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e3d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28dccb4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e3d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50434/0 (socket says 172.21.15.78:50434) 2024-09-11T01:08:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28ddcb6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e3ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28dccb4640 1 -- 172.21.15.78:0/3756291249 learned_addr learned my addr 172.21.15.78:0/3756291249 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/3756291249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 0x556bc2e37c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28dccb4640 1 -- 172.21.15.78:0/3756291249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 msgr2=0x556bc2e37c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28dccb4640 1 --2- 172.21.15.78:0/3756291249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 0x556bc2e37c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28dccb4640 1 -- 172.21.15.78:0/3756291249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 msgr2=0x556bc2e3ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28dccb4640 1 --2- 172.21.15.78:0/3756291249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e3ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.427+0000 7f28dccb4640 1 -- 172.21.15.78:0/3756291249 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc2e38220 con 0x556bc2e28df0 2024-09-11T01:08:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28dccb4640 1 --2- 172.21.15.78:0/3756291249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e3d3a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0059f0 tx=0x7f28cc010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666d8870287c71af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28bcff9640 1 -- 172.21.15.78:0/3756291249 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc029070 con 0x556bc2e28df0 2024-09-11T01:08:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28bcff9640 1 -- 172.21.15.78:0/3756291249 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f28cc021020 con 0x556bc2e28df0 2024-09-11T01:08:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28e1aee740 1 -- 172.21.15.78:0/3756291249 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d00198c0 con 0x556bc2e28df0 2024-09-11T01:08:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28bcff9640 1 -- 172.21.15.78:0/3756291249 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f28cc012880 con 0x556bc2e28df0 2024-09-11T01:08:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28e1aee740 10 client.17421 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28e1aee740 1 -- 172.21.15.78:0/3756291249 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x556bc2eb20a0 con 0x556bc2e28df0 2024-09-11T01:08:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28bcff9640 1 -- 172.21.15.78:0/3756291249 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d00198c0 con 0x556bc2e28df0 2024-09-11T01:08:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28bcff9640 1 -- 172.21.15.78:0/3756291249 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f28cc003c00 con 0x556bc2e28df0 2024-09-11T01:08:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28e1aee740 10 client.17421 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28e1aee740 10 client.17421 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:08:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28e1aee740 20 client.17421 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28e1aee740 1 --2- 172.21.15.78:0/3756291249 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556bc2eb27f0 0x556bc2eb4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28e1aee740 4 client.17421 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:08:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.428+0000 7f28e1aee740 1 -- 172.21.15.78:0/3756291249 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556bc2d9c8a0 con 0x556bc2eb27f0 2024-09-11T01:08:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/3756291249 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556bc2eb27f0 0x556bc2eb4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28ddcb6640 1 --2- 172.21.15.78:0/3756291249 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556bc2eb27f0 0x556bc2eb4c10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6766989f63f7bf4 server_cookie=52847cf4a63e976a in_seq=0 out_seq=0 2024-09-11T01:08:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28bcff9640 10 client.17421 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:08:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28bcff9640 1 -- 172.21.15.78:0/3756291249 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556bc2d9c8a0 con 0x556bc2eb27f0 2024-09-11T01:08:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28bcff9640 10 client.17421 handle_command_reply: tid=0 2024-09-11T01:08:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28e1aee740 1 client.17421 shutdown 2024-09-11T01:08:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28e1aee740 1 -- 172.21.15.78:0/3756291249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 msgr2=0x556bc2e3d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28e1aee740 1 --2- 172.21.15.78:0/3756291249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e3d3a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0059f0 tx=0x7f28cc010cf0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28e1aee740 1 -- 172.21.15.78:0/3756291249 shutdown_connections 2024-09-11T01:08:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28e1aee740 1 --2- 172.21.15.78:0/3756291249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556bc2e34d80 0x556bc2e37c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28e1aee740 1 --2- 172.21.15.78:0/3756291249 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556bc2eb27f0 0x556bc2eb4c10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28e1aee740 1 --2- 172.21.15.78:0/3756291249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bc2e28df0 0x556bc2e3d3a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28e1aee740 1 --2- 172.21.15.78:0/3756291249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556bc2e284d0 0x556bc2e3ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28bcff9640 0 client.17421 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:08:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28e1aee740 1 -- 172.21.15.78:0/3756291249 >> 172.21.15.78:0/3756291249 conn(0x556bc2e26300 msgr2=0x556bc2e30200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28e1aee740 1 -- 172.21.15.78:0/3756291249 shutdown_connections 2024-09-11T01:08:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.429+0000 7f28e1aee740 1 -- 172.21.15.78:0/3756291249 wait complete. 2024-09-11T01:08:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28e1aee740 20 client.17421 trim_cache size 0 max 16384 2024-09-11T01:08:58.450 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:08:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28df719640 1 -- 172.21.15.78:0/468950946 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f28c00b54d0 msgr2=0x7f28c00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28df719640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f28c00b54d0 0x7f28c00b7990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f28c8104b80 tx=0x7f28c810c040 comp rx=0 tx=0).stop 2024-09-11T01:08:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28df719640 1 -- 172.21.15.78:0/468950946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d8101c90 msgr2=0x7f28d81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28df719640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d8101c90 0x7f28d81992f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f28c8033040 tx=0x7f28c80316f0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28df719640 1 -- 172.21.15.78:0/468950946 shutdown_connections 2024-09-11T01:08:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28df719640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f28c00b54d0 0x7f28c00b7990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28df719640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108020 0x7f28d819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28df719640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d81076a0 0x7f28d8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28df719640 1 --2- 172.21.15.78:0/468950946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d8101c90 0x7f28d81992f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28df719640 1 -- 172.21.15.78:0/468950946 >> 172.21.15.78:0/468950946 conn(0x7f28d80fd700 msgr2=0x7f28d80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28df719640 1 -- 172.21.15.78:0/468950946 shutdown_connections 2024-09-11T01:08:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:58.430+0000 7f28df719640 1 -- 172.21.15.78:0/468950946 wait complete. 2024-09-11T01:08:58.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 668061 5.24 MB/sec execute 426 sec latency 40.344 ms 2024-09-11T01:08:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:08:58 smithi110 ceph-mon[34712]: pgmap v632: 97 pgs: 97 active+clean; 495 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-09-11T01:08:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:08:58 smithi112 ceph-mon[35384]: pgmap v632: 97 pgs: 97 active+clean; 495 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-09-11T01:08:58.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:08:58 smithi078 ceph-mon[31315]: pgmap v632: 97 pgs: 97 active+clean; 495 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-09-11T01:08:59.151 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:08:59.399 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:08:59.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 669314 5.24 MB/sec execute 427 sec latency 288.361 ms 2024-09-11T01:08:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.872+0000 7f6c50742640 1 -- 172.21.15.78:0/697905325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c48102dc0 msgr2=0x7f6c481031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.872+0000 7f6c50742640 1 --2- 172.21.15.78:0/697905325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c48102dc0 0x7f6c481031a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009900 tx=0x7f6c4402a410 comp rx=0 tx=0).stop 2024-09-11T01:08:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.872+0000 7f6c50742640 1 -- 172.21.15.78:0/697905325 shutdown_connections 2024-09-11T01:08:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.872+0000 7f6c50742640 1 --2- 172.21.15.78:0/697905325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48109150 0x7f6c4810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.872+0000 7f6c50742640 1 --2- 172.21.15.78:0/697905325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c481087d0 0x7f6c48108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.872+0000 7f6c50742640 1 --2- 172.21.15.78:0/697905325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c48102dc0 0x7f6c481031a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.872+0000 7f6c50742640 1 -- 172.21.15.78:0/697905325 >> 172.21.15.78:0/697905325 conn(0x7f6c480fe830 msgr2=0x7f6c48100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.872+0000 7f6c50742640 1 -- 172.21.15.78:0/697905325 shutdown_connections 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.872+0000 7f6c50742640 1 -- 172.21.15.78:0/697905325 wait complete. 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c50742640 1 Processor -- start 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c50742640 1 -- start start 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c50742640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c48102dc0 0x7f6c480754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c50742640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c481087d0 0x7f6c480759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c50742640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c48109150 0x7f6c48075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c50742640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c48112e50 con 0x7f6c48109150 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c50742640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c48112cd0 con 0x7f6c481087d0 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c50742640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c48112fd0 con 0x7f6c48102dc0 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c4ecdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c48109150 0x7f6c48075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c4ecdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c48109150 0x7f6c48075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50450/0 (socket says 172.21.15.78:50450) 2024-09-11T01:08:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c4ecdf640 1 -- 172.21.15.78:0/2307647225 learned_addr learned my addr 172.21.15.78:0/2307647225 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c48102dc0 0x7f6c480754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c481087d0 0x7f6c480759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c4ecdf640 1 -- 172.21.15.78:0/2307647225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c48102dc0 msgr2=0x7f6c480754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c4ecdf640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c48102dc0 0x7f6c480754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c4ecdf640 1 -- 172.21.15.78:0/2307647225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c481087d0 msgr2=0x7f6c480759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c4ecdf640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c481087d0 0x7f6c480759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c4ecdf640 1 -- 172.21.15.78:0/2307647225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c481a7860 con 0x7f6c48109150 2024-09-11T01:08:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c481087d0 0x7f6c480759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:08:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.873+0000 7f6c4ecdf640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c48109150 0x7f6c48075f20 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000b970 tx=0x7f6c4000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.874+0000 7f6c3f7fe640 1 -- 172.21.15.78:0/2307647225 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40019070 con 0x7f6c48109150 2024-09-11T01:08:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.874+0000 7f6c3f7fe640 1 -- 172.21.15.78:0/2307647225 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c40014440 con 0x7f6c48109150 2024-09-11T01:08:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.874+0000 7f6c50742640 1 -- 172.21.15.78:0/2307647225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c481a7b50 con 0x7f6c48109150 2024-09-11T01:08:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.874+0000 7f6c3f7fe640 1 -- 172.21.15.78:0/2307647225 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400075d0 con 0x7f6c48109150 2024-09-11T01:08:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.874+0000 7f6c50742640 1 -- 172.21.15.78:0/2307647225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c481a8040 con 0x7f6c48109150 2024-09-11T01:08:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.877+0000 7f6c52b17740 1 Processor -- start 2024-09-11T01:08:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.877+0000 7f6c3f7fe640 1 -- 172.21.15.78:0/2307647225 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6c4000cb40 con 0x7f6c48109150 2024-09-11T01:08:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.877+0000 7f6c52b17740 1 -- start start 2024-09-11T01:08:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.877+0000 7f6c52b17740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed865364d0 0x55ed865368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.877+0000 7f6c52b17740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed86536df0 0x55ed86542840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.877+0000 7f6c52b17740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 0x55ed86545170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.877+0000 7f6c52b17740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ed864acea0 con 0x55ed86542d80 2024-09-11T01:08:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.877+0000 7f6c52b17740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ed8649c2d0 con 0x55ed865364d0 2024-09-11T01:08:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.877+0000 7f6c52b17740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ed864a00e0 con 0x55ed86536df0 2024-09-11T01:08:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.877+0000 7f6c4dcdd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 0x55ed86545170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c4e4de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed86536df0 0x55ed86542840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c4dcdd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 0x55ed86545170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50452/0 (socket says 172.21.15.78:50452) 2024-09-11T01:08:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c4dcdd640 1 -- 172.21.15.78:0/829381142 learned_addr learned my addr 172.21.15.78:0/829381142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/829381142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed865364d0 0x55ed865368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c3f7fe640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6c340b5450 0x7f6c340b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c4e4de640 1 -- 172.21.15.78:0/829381142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed865364d0 msgr2=0x55ed865368b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/829381142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed865364d0 0x55ed865368b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c4e4de640 1 -- 172.21.15.78:0/829381142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 msgr2=0x55ed86545170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/829381142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 0x55ed86545170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c4e4de640 1 -- 172.21.15.78:0/829381142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed861fa650 con 0x55ed86536df0 2024-09-11T01:08:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/829381142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 0x55ed86545170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:08:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c4ecdf640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6c340b5450 0x7f6c340b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c3f7fe640 1 -- 172.21.15.78:0/2307647225 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f6c400e57f0 con 0x7f6c48109150 2024-09-11T01:08:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/829381142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed86536df0 0x55ed86542840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009230 tx=0x7f6c44031cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21a199126a0b1aff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c3cff9640 1 -- 172.21.15.78:0/829381142 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4403f070 con 0x55ed86536df0 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c3cff9640 1 -- 172.21.15.78:0/829381142 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c440316a0 con 0x55ed86536df0 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.878+0000 7f6c3cff9640 1 -- 172.21.15.78:0/829381142 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44035070 con 0x55ed86536df0 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.879+0000 7f6c52b17740 1 -- 172.21.15.78:0/829381142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed86536df0 msgr2=0x55ed86542840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.879+0000 7f6c4ecdf640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6c340b5450 0x7f6c340b7910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6c400e3030 tx=0x7f6c4000e040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.879+0000 7f6c52b17740 1 --2- 172.21.15.78:0/829381142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed86536df0 0x55ed86542840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009230 tx=0x7f6c44031cd0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.879+0000 7f6c52b17740 1 -- 172.21.15.78:0/829381142 shutdown_connections 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.879+0000 7f6c52b17740 1 --2- 172.21.15.78:0/829381142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 0x55ed86545170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.879+0000 7f6c52b17740 1 --2- 172.21.15.78:0/829381142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed86536df0 0x55ed86542840 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.879+0000 7f6c52b17740 1 --2- 172.21.15.78:0/829381142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed865364d0 0x55ed865368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.879+0000 7f6c52b17740 1 -- 172.21.15.78:0/829381142 >> 172.21.15.78:0/829381142 conn(0x55ed86534300 msgr2=0x55ed865354a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.879+0000 7f6c52b17740 1 -- 172.21.15.78:0/829381142 shutdown_connections 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.879+0000 7f6c52b17740 1 -- 172.21.15.78:0/829381142 wait complete. 2024-09-11T01:08:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.879+0000 7f6c52b17740 1 -- start start 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c52b17740 1 Processor -- start 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c52b17740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed865364d0 0x55ed865425d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c52b17740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed86536df0 0x55ed8653fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c52b17740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 0x55ed8653ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c52b17740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ed862fd700 con 0x55ed86542d80 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c52b17740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ed86437160 con 0x55ed86536df0 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c52b17740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ed865459a0 con 0x55ed865364d0 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c52b17740 10 client.0 _ll_register_callbacks cb 0x55ed864509e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c4dcdd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 0x55ed8653ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c4dcdd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 0x55ed8653ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50460/0 (socket says 172.21.15.78:50460) 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c4dcdd640 1 -- 172.21.15.78:0/1051757330 learned_addr learned my addr 172.21.15.78:0/1051757330 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed86536df0 0x55ed8653fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed865364d0 0x55ed865425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c4dcdd640 1 -- 172.21.15.78:0/1051757330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed865364d0 msgr2=0x55ed865425d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:59.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed865364d0 0x55ed865425d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c4dcdd640 1 -- 172.21.15.78:0/1051757330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed86536df0 msgr2=0x55ed8653fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:59.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed86536df0 0x55ed8653fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c4dcdd640 1 -- 172.21.15.78:0/1051757330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed86464c90 con 0x55ed86542d80 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 0x55ed8653ff80 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6c38018f20 tx=0x7f6c380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27c521935f6d47 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.880+0000 7f6c31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c31ffb640 1 -- 172.21.15.78:0/1051757330 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800f040 con 0x55ed86542d80 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c31ffb640 1 -- 172.21.15.78:0/1051757330 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f6c38015560 con 0x55ed86542d80 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c52b17740 1 -- 172.21.15.78:0/1051757330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c440316a0 con 0x55ed86542d80 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c31ffb640 1 -- 172.21.15.78:0/1051757330 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6c38028050 con 0x55ed86542d80 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c52b17740 10 client.17433 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c52b17740 1 -- 172.21.15.78:0/1051757330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ed86551290 con 0x55ed86542d80 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c31ffb640 1 -- 172.21.15.78:0/1051757330 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c440316a0 con 0x55ed86542d80 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c31ffb640 1 -- 172.21.15.78:0/1051757330 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f6c38015040 con 0x55ed86542d80 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c52b17740 10 client.17433 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c52b17740 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:08:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c52b17740 20 client.17433 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c52b17740 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ed865527f0 0x55ed86554be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c52b17740 4 client.17433 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c52b17740 1 -- 172.21.15.78:0/1051757330 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed864aedf0 con 0x55ed865527f0 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.881+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ed865527f0 0x55ed86554be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.882+0000 7f6c31ffb640 10 client.17433 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.882+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ed865527f0 0x55ed86554be0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1468eb6296c7c88f server_cookie=254f159ec0ed04e2 in_seq=0 out_seq=0 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c31ffb640 1 -- 172.21.15.78:0/1051757330 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed864aedf0 con 0x55ed865527f0 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c31ffb640 10 client.17433 handle_command_reply: tid=0 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c52b17740 1 client.17433 shutdown 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c52b17740 1 -- 172.21.15.78:0/1051757330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 msgr2=0x55ed8653ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c52b17740 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 0x55ed8653ff80 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6c38018f20 tx=0x7f6c380046c0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c52b17740 1 -- 172.21.15.78:0/1051757330 shutdown_connections 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c52b17740 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ed865527f0 0x55ed86554be0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c52b17740 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86542d80 0x55ed8653ff80 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c52b17740 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed86536df0 0x55ed8653fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c52b17740 1 --2- 172.21.15.78:0/1051757330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed865364d0 0x55ed865425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c52b17740 1 -- 172.21.15.78:0/1051757330 >> 172.21.15.78:0/1051757330 conn(0x55ed86534300 msgr2=0x55ed86539e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:59.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c31ffb640 0 client.17433 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:59.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c52b17740 1 -- 172.21.15.78:0/1051757330 shutdown_connections 2024-09-11T01:08:59.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c52b17740 1 -- 172.21.15.78:0/1051757330 wait complete. 2024-09-11T01:08:59.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.883+0000 7f6c52b17740 20 client.17433 trim_cache size 0 max 16384 2024-09-11T01:08:59.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.896+0000 7f6c52b17740 1 Processor -- start 2024-09-11T01:08:59.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.896+0000 7f6c52b17740 1 -- start start 2024-09-11T01:08:59.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.896+0000 7f6c52b17740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed865364d0 0x55ed86535a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.896+0000 7f6c52b17740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86536df0 0x55ed8653cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c52b17740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed8654ebd0 0x55ed8653d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c52b17740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481b08a0 con 0x55ed86536df0 2024-09-11T01:08:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c52b17740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481b2410 con 0x55ed8654ebd0 2024-09-11T01:08:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c52b17740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481b0f70 con 0x55ed865364d0 2024-09-11T01:08:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c4e4de640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86536df0 0x55ed8653cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c4e4de640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86536df0 0x55ed8653cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50472/0 (socket says 172.21.15.78:50472) 2024-09-11T01:08:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c4e4de640 1 -- 172.21.15.78:0/3638754800 learned_addr learned my addr 172.21.15.78:0/3638754800 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/3638754800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed865364d0 0x55ed86535a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/3638754800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed8654ebd0 0x55ed8653d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c4e4de640 1 -- 172.21.15.78:0/3638754800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed865364d0 msgr2=0x55ed86535a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/3638754800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed865364d0 0x55ed86535a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c4e4de640 1 -- 172.21.15.78:0/3638754800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed8654ebd0 msgr2=0x55ed8653d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/3638754800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed8654ebd0 0x55ed8653d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c4e4de640 1 -- 172.21.15.78:0/3638754800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed86536c20 con 0x55ed86536df0 2024-09-11T01:08:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/3638754800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86536df0 0x55ed8653cc50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009900 tx=0x7f6c44005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fca0aaaf3ddeedf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c33fff640 1 -- 172.21.15.78:0/3638754800 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44050020 con 0x55ed86536df0 2024-09-11T01:08:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c33fff640 1 -- 172.21.15.78:0/3638754800 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c44005480 con 0x55ed86536df0 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.897+0000 7f6c33fff640 1 -- 172.21.15.78:0/3638754800 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4403f040 con 0x55ed86536df0 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.898+0000 7f6c52b17740 1 -- 172.21.15.78:0/3638754800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86536df0 msgr2=0x55ed8653cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.898+0000 7f6c52b17740 1 --2- 172.21.15.78:0/3638754800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86536df0 0x55ed8653cc50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009900 tx=0x7f6c44005210 comp rx=0 tx=0).stop 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.898+0000 7f6c52b17740 1 -- 172.21.15.78:0/3638754800 shutdown_connections 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.898+0000 7f6c52b17740 1 --2- 172.21.15.78:0/3638754800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed8654ebd0 0x55ed8653d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.898+0000 7f6c52b17740 1 --2- 172.21.15.78:0/3638754800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed86536df0 0x55ed8653cc50 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.898+0000 7f6c52b17740 1 --2- 172.21.15.78:0/3638754800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed865364d0 0x55ed86535a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.898+0000 7f6c52b17740 1 -- 172.21.15.78:0/3638754800 >> 172.21.15.78:0/3638754800 conn(0x55ed86534300 msgr2=0x55ed86535350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.898+0000 7f6c52b17740 1 -- 172.21.15.78:0/3638754800 shutdown_connections 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.898+0000 7f6c52b17740 1 -- 172.21.15.78:0/3638754800 wait complete. 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.898+0000 7f6c52b17740 1 -- start start 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c52b17740 1 Processor -- start 2024-09-11T01:08:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c52b17740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed865364d0 0x55ed8654ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c52b17740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed86536df0 0x55ed8654b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c52b17740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed8654ebd0 0x55ed86547c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c52b17740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481b1d10 con 0x55ed865364d0 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c52b17740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481b1640 con 0x55ed8654ebd0 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c52b17740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481ae8a0 con 0x55ed86536df0 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c52b17740 10 client.0 _ll_register_callbacks cb 0x7f6c38006c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c4dcdd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed865364d0 0x55ed8654ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c4dcdd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed865364d0 0x55ed8654ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50478/0 (socket says 172.21.15.78:50478) 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c4dcdd640 1 -- 172.21.15.78:0/1618099802 learned_addr learned my addr 172.21.15.78:0/1618099802 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed8654ebd0 0x55ed86547c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed86536df0 0x55ed8654b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c4dcdd640 1 -- 172.21.15.78:0/1618099802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed86536df0 msgr2=0x55ed8654b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed86536df0 0x55ed8654b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c4dcdd640 1 -- 172.21.15.78:0/1618099802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed8654ebd0 msgr2=0x55ed86547c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed8654ebd0 0x55ed86547c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c4dcdd640 1 -- 172.21.15.78:0/1618099802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8654b8b0 con 0x55ed865364d0 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c4dcdd640 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed865364d0 0x55ed8654ac30 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6c38016c90 tx=0x7f6c380047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917ad8e06a7ff32a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.899+0000 7f6c31ffb640 1 -- 172.21.15.78:0/1618099802 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800c040 con 0x55ed865364d0 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c31ffb640 1 -- 172.21.15.78:0/1618099802 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f6c380237f0 con 0x55ed865364d0 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c52b17740 1 -- 172.21.15.78:0/1618099802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c44005480 con 0x55ed865364d0 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c31ffb640 1 -- 172.21.15.78:0/1618099802 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6c38023a10 con 0x55ed865364d0 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c52b17740 10 client.17439 fetch_fsmap learned FSMap version 47 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c52b17740 1 -- 172.21.15.78:0/1618099802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ed865c8420 con 0x55ed865364d0 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c31ffb640 1 -- 172.21.15.78:0/1618099802 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c44005480 con 0x55ed865364d0 2024-09-11T01:08:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c31ffb640 1 -- 172.21.15.78:0/1618099802 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f6c38015030 con 0x55ed865364d0 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c52b17740 10 client.17439 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c52b17740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c52b17740 20 client.17439 populate_metadata read hostname 'smithi078' 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c52b17740 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ed865c8bd0 0x55ed865caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c52b17740 4 client.17439 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c52b17740 1 -- 172.21.15.78:0/1618099802 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed864aa8a0 con 0x55ed865c8bd0 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.900+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ed865c8bd0 0x55ed865caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c31ffb640 10 client.17439 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c4e4de640 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ed865c8bd0 0x55ed865caff0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df644f3a678d4c0b server_cookie=a3c5d7d894a6e343 in_seq=0 out_seq=0 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c31ffb640 1 -- 172.21.15.78:0/1618099802 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed864aa8a0 con 0x55ed865c8bd0 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c31ffb640 10 client.17439 handle_command_reply: tid=0 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c52b17740 1 client.17439 shutdown 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c52b17740 1 -- 172.21.15.78:0/1618099802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed865364d0 msgr2=0x55ed8654ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c52b17740 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed865364d0 0x55ed8654ac30 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6c38016c90 tx=0x7f6c380047d0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c52b17740 1 -- 172.21.15.78:0/1618099802 shutdown_connections 2024-09-11T01:08:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c52b17740 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed8654ebd0 0x55ed86547c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c52b17740 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed86536df0 0x55ed8654b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c52b17740 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ed865c8bd0 0x55ed865caff0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c52b17740 1 --2- 172.21.15.78:0/1618099802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed865364d0 0x55ed8654ac30 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c31ffb640 0 client.17439 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:08:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c52b17740 1 -- 172.21.15.78:0/1618099802 >> 172.21.15.78:0/1618099802 conn(0x55ed86534300 msgr2=0x55ed86541050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c52b17740 1 -- 172.21.15.78:0/1618099802 shutdown_connections 2024-09-11T01:08:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c52b17740 1 -- 172.21.15.78:0/1618099802 wait complete. 2024-09-11T01:08:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.901+0000 7f6c52b17740 20 client.17439 trim_cache size 0 max 16384 2024-09-11T01:08:59.919 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:08:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.902+0000 7f6c50742640 1 -- 172.21.15.78:0/2307647225 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6c340b5450 msgr2=0x7f6c340b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.902+0000 7f6c50742640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6c340b5450 0x7f6c340b7910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6c400e3030 tx=0x7f6c4000e040 comp rx=0 tx=0).stop 2024-09-11T01:08:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.902+0000 7f6c50742640 1 -- 172.21.15.78:0/2307647225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c48109150 msgr2=0x7f6c48075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:08:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.902+0000 7f6c50742640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c48109150 0x7f6c48075f20 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000b970 tx=0x7f6c4000be30 comp rx=0 tx=0).stop 2024-09-11T01:08:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.902+0000 7f6c50742640 1 -- 172.21.15.78:0/2307647225 shutdown_connections 2024-09-11T01:08:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.902+0000 7f6c50742640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6c340b5450 0x7f6c340b7910 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.902+0000 7f6c50742640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c48109150 0x7f6c48075f20 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.902+0000 7f6c50742640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c481087d0 0x7f6c480759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.902+0000 7f6c50742640 1 --2- 172.21.15.78:0/2307647225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c48102dc0 0x7f6c480754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:08:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.902+0000 7f6c50742640 1 -- 172.21.15.78:0/2307647225 >> 172.21.15.78:0/2307647225 conn(0x7f6c480fe830 msgr2=0x7f6c48100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:08:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.902+0000 7f6c50742640 1 -- 172.21.15.78:0/2307647225 shutdown_connections 2024-09-11T01:08:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:08:59.902+0000 7f6c50742640 1 -- 172.21.15.78:0/2307647225 wait complete. 2024-09-11T01:09:00.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 669314 5.23 MB/sec execute 428 sec latency 1288.433 ms 2024-09-11T01:09:00.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:00 smithi110 ceph-mon[34712]: pgmap v633: 97 pgs: 97 active+clean; 495 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-09-11T01:09:00.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:00 smithi112 ceph-mon[35384]: pgmap v633: 97 pgs: 97 active+clean; 495 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-09-11T01:09:00.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:00 smithi078 ceph-mon[31315]: pgmap v633: 97 pgs: 97 active+clean; 495 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-09-11T01:09:01.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 669314 5.22 MB/sec execute 429 sec latency 2288.551 ms 2024-09-11T01:09:02.134 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:09:02.135 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:09:02.415 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:09:02.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 670040 5.21 MB/sec execute 430 sec latency 2673.929 ms 2024-09-11T01:09:02.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.928+0000 7fb97b356640 1 -- 172.21.15.78:0/1293559165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb974109150 msgr2=0x7fb97410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:02.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.928+0000 7fb97b356640 1 --2- 172.21.15.78:0/1293559165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb974109150 0x7fb97410f950 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009ca0 tx=0x7fb96c02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.928+0000 7fb97b356640 1 -- 172.21.15.78:0/1293559165 shutdown_connections 2024-09-11T01:09:02.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.928+0000 7fb97b356640 1 --2- 172.21.15.78:0/1293559165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb974109150 0x7fb97410f950 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.928+0000 7fb97b356640 1 --2- 172.21.15.78:0/1293559165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9741087d0 0x7fb974108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.928+0000 7fb97b356640 1 --2- 172.21.15.78:0/1293559165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb974102dc0 0x7fb9741031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.928+0000 7fb97b356640 1 -- 172.21.15.78:0/1293559165 >> 172.21.15.78:0/1293559165 conn(0x7fb9740fe870 msgr2=0x7fb974100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:02.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.929+0000 7fb97b356640 1 -- 172.21.15.78:0/1293559165 shutdown_connections 2024-09-11T01:09:02.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.929+0000 7fb97b356640 1 -- 172.21.15.78:0/1293559165 wait complete. 2024-09-11T01:09:02.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.929+0000 7fb97b356640 1 Processor -- start 2024-09-11T01:09:02.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.929+0000 7fb97b356640 1 -- start start 2024-09-11T01:09:02.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.929+0000 7fb97b356640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb974102dc0 0x7fb9741992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.929+0000 7fb97b356640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9741087d0 0x7fb974199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.929+0000 7fb97b356640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb974109150 0x7fb97419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.929+0000 7fb97b356640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb974112dd0 con 0x7fb974102dc0 2024-09-11T01:09:02.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.929+0000 7fb97b356640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb974112c50 con 0x7fb9741087d0 2024-09-11T01:09:02.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.929+0000 7fb97b356640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb974112f50 con 0x7fb974109150 2024-09-11T01:09:02.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb9790f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb974102dc0 0x7fb9741992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb9790f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb974102dc0 0x7fb9741992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50494/0 (socket says 172.21.15.78:50494) 2024-09-11T01:09:02.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb9790f2640 1 -- 172.21.15.78:0/245953915 learned_addr learned my addr 172.21.15.78:0/245953915 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:02.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb9788f1640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9741087d0 0x7fb974199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb9798f3640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb974109150 0x7fb97419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb9790f2640 1 -- 172.21.15.78:0/245953915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb974109150 msgr2=0x7fb97419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:02.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb9790f2640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb974109150 0x7fb97419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb9790f2640 1 -- 172.21.15.78:0/245953915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9741087d0 msgr2=0x7fb974199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:02.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb9790f2640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9741087d0 0x7fb974199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb9790f2640 1 -- 172.21.15.78:0/245953915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97419f390 con 0x7fb974102dc0 2024-09-11T01:09:02.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb9788f1640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9741087d0 0x7fb974199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:02.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb9790f2640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb974102dc0 0x7fb9741992e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb97000ea60 tx=0x7fb97000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:02.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb96a7fc640 1 -- 172.21.15.78:0/245953915 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97000cde0 con 0x7fb974102dc0 2024-09-11T01:09:02.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb96a7fc640 1 -- 172.21.15.78:0/245953915 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9700035f0 con 0x7fb974102dc0 2024-09-11T01:09:02.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb97b356640 1 -- 172.21.15.78:0/245953915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97419f680 con 0x7fb974102dc0 2024-09-11T01:09:02.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb96a7fc640 1 -- 172.21.15.78:0/245953915 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970010690 con 0x7fb974102dc0 2024-09-11T01:09:02.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.930+0000 7fb97b356640 1 -- 172.21.15.78:0/245953915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9741a5d40 con 0x7fb974102dc0 2024-09-11T01:09:02.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.933+0000 7fb96a7fc640 1 -- 172.21.15.78:0/245953915 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb970004000 con 0x7fb974102dc0 2024-09-11T01:09:02.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.933+0000 7fb97d72b740 1 Processor -- start 2024-09-11T01:09:02.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.933+0000 7fb97d72b740 1 -- start start 2024-09-11T01:09:02.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.933+0000 7fb97d72b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee3ee0 0x55d48fee42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.933+0000 7fb97d72b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee4800 0x55d48fef0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.933+0000 7fb97d72b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fef0790 0x55d48fef2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.933+0000 7fb97d72b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d48fe5ae60 con 0x55d48fee4800 2024-09-11T01:09:02.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.933+0000 7fb97d72b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d48fe4a290 con 0x55d48fef0790 2024-09-11T01:09:02.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.933+0000 7fb97d72b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d48fe4e0a0 con 0x55d48fee3ee0 2024-09-11T01:09:02.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb9798f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee3ee0 0x55d48fee42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb9798f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee3ee0 0x55d48fee42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35862/0 (socket says 172.21.15.78:35862) 2024-09-11T01:09:02.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb9798f3640 1 -- 172.21.15.78:0/3300808364 learned_addr learned my addr 172.21.15.78:0/3300808364 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:02.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb9790f2640 1 --2- 172.21.15.78:0/3300808364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee4800 0x55d48fef0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb9788f1640 1 --2- 172.21.15.78:0/3300808364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fef0790 0x55d48fef2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb9798f3640 1 -- 172.21.15.78:0/3300808364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fef0790 msgr2=0x55d48fef2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:02.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb9798f3640 1 --2- 172.21.15.78:0/3300808364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fef0790 0x55d48fef2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb9798f3640 1 -- 172.21.15.78:0/3300808364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee4800 msgr2=0x55d48fef0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:02.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb9798f3640 1 --2- 172.21.15.78:0/3300808364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee4800 0x55d48fef0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb9798f3640 1 -- 172.21.15.78:0/3300808364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d48fba8930 con 0x55d48fee3ee0 2024-09-11T01:09:02.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb96a7fc640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9600b5450 0x7fb9600b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb9790f2640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9600b5450 0x7fb9600b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.934+0000 7fb9798f3640 1 --2- 172.21.15.78:0/3300808364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee3ee0 0x55d48fee42c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005ef0 tx=0x7fb96c005f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a42f21bd13e62d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:02.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.935+0000 7fb95f7fe640 1 -- 172.21.15.78:0/3300808364 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c031cf0 con 0x55d48fee3ee0 2024-09-11T01:09:02.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.935+0000 7fb96a7fc640 1 -- 172.21.15.78:0/245953915 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fb9700e62a0 con 0x7fb974102dc0 2024-09-11T01:09:02.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.935+0000 7fb95f7fe640 1 -- 172.21.15.78:0/3300808364 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb96c036070 con 0x55d48fee3ee0 2024-09-11T01:09:02.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.935+0000 7fb95f7fe640 1 -- 172.21.15.78:0/3300808364 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0042a0 con 0x55d48fee3ee0 2024-09-11T01:09:02.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.935+0000 7fb97d72b740 1 -- 172.21.15.78:0/3300808364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee3ee0 msgr2=0x55d48fee42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:02.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.935+0000 7fb97d72b740 1 --2- 172.21.15.78:0/3300808364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee3ee0 0x55d48fee42c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005ef0 tx=0x7fb96c005f20 comp rx=0 tx=0).stop 2024-09-11T01:09:02.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.935+0000 7fb9790f2640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9600b5450 0x7fb9600b7910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb9700e3ac0 tx=0x7fb9700e8040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:02.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.936+0000 7fb97d72b740 1 -- 172.21.15.78:0/3300808364 shutdown_connections 2024-09-11T01:09:02.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.936+0000 7fb97d72b740 1 --2- 172.21.15.78:0/3300808364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fef0790 0x55d48fef2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.936+0000 7fb97d72b740 1 --2- 172.21.15.78:0/3300808364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee4800 0x55d48fef0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.936+0000 7fb97d72b740 1 --2- 172.21.15.78:0/3300808364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee3ee0 0x55d48fee42c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.936+0000 7fb97d72b740 1 -- 172.21.15.78:0/3300808364 >> 172.21.15.78:0/3300808364 conn(0x55d48fee1d10 msgr2=0x55d48fee2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:02.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.936+0000 7fb97d72b740 1 -- 172.21.15.78:0/3300808364 shutdown_connections 2024-09-11T01:09:02.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.936+0000 7fb97d72b740 1 -- 172.21.15.78:0/3300808364 wait complete. 2024-09-11T01:09:02.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.936+0000 7fb97d72b740 1 -- start start 2024-09-11T01:09:02.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.936+0000 7fb97d72b740 1 Processor -- start 2024-09-11T01:09:02.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb97d72b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 0x55d48feed840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb97d72b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee4800 0x55d48feedd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb97d72b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fef0790 0x55d48feee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb97d72b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d48fcab890 con 0x55d48fee3ee0 2024-09-11T01:09:02.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb97d72b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d48fde4d40 con 0x55d48fee4800 2024-09-11T01:09:02.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb97d72b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d48fef7610 con 0x55d48fef0790 2024-09-11T01:09:02.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb9798f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 0x55d48feed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb97d72b740 10 client.0 _ll_register_callbacks cb 0x55d48fe48c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:02.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb9798f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 0x55d48feed840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50516/0 (socket says 172.21.15.78:50516) 2024-09-11T01:09:02.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb9798f3640 1 -- 172.21.15.78:0/2351587504 learned_addr learned my addr 172.21.15.78:0/2351587504 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:02.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb9798f3640 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fef0790 0x55d48feee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb9788f1640 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee4800 0x55d48feedd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb9798f3640 1 -- 172.21.15.78:0/2351587504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fef0790 msgr2=0x55d48feee2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:02.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb9798f3640 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fef0790 0x55d48feee2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb9798f3640 1 -- 172.21.15.78:0/2351587504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee4800 msgr2=0x55d48feedd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:02.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb9798f3640 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee4800 0x55d48feedd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb9798f3640 1 -- 172.21.15.78:0/2351587504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d48ff00af0 con 0x55d48fee3ee0 2024-09-11T01:09:02.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb9788f1640 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee4800 0x55d48feedd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:09:02.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb9798f3640 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 0x55d48feed840 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009620 tx=0x7fb96c003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f42dd4960914786 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:02.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.937+0000 7fb95cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:09:02.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb95cff9640 1 -- 172.21.15.78:0/2351587504 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c055060 con 0x55d48fee3ee0 2024-09-11T01:09:02.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb95cff9640 1 -- 172.21.15.78:0/2351587504 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fb96c02f660 con 0x55d48fee3ee0 2024-09-11T01:09:02.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb97d72b740 1 -- 172.21.15.78:0/2351587504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb96c036070 con 0x55d48fee3ee0 2024-09-11T01:09:02.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb95cff9640 1 -- 172.21.15.78:0/2351587504 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb96c02f860 con 0x55d48fee3ee0 2024-09-11T01:09:02.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb97d72b740 10 client.17451 fetch_fsmap learned FSMap version 47 2024-09-11T01:09:02.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb97d72b740 1 -- 172.21.15.78:0/2351587504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d48ff03480 con 0x55d48fee3ee0 2024-09-11T01:09:02.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb95cff9640 1 -- 172.21.15.78:0/2351587504 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb96c036070 con 0x55d48fee3ee0 2024-09-11T01:09:02.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb95cff9640 1 -- 172.21.15.78:0/2351587504 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7fb96c043020 con 0x55d48fee3ee0 2024-09-11T01:09:02.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb97d72b740 10 client.17451 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:09:02.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb97d72b740 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:09:02.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb97d72b740 20 client.17451 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:02.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d48ff047d0 0x55d48ff06bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb97d72b740 4 client.17451 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:09:02.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.938+0000 7fb97d72b740 1 -- 172.21.15.78:0/2351587504 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d48fe5cdb0 con 0x55d48ff047d0 2024-09-11T01:09:02.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.939+0000 7fb9788f1640 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d48ff047d0 0x55d48ff06bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.939+0000 7fb95cff9640 10 client.17451 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:02.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.939+0000 7fb9788f1640 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d48ff047d0 0x55d48ff06bc0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bec5a318479c51a server_cookie=7a1b03cd1ec30766 in_seq=0 out_seq=0 2024-09-11T01:09:02.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.939+0000 7fb95cff9640 1 -- 172.21.15.78:0/2351587504 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d48fe5cdb0 con 0x55d48ff047d0 2024-09-11T01:09:02.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb95cff9640 10 client.17451 handle_command_reply: tid=0 2024-09-11T01:09:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb97d72b740 1 client.17451 shutdown 2024-09-11T01:09:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb97d72b740 1 -- 172.21.15.78:0/2351587504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 msgr2=0x55d48feed840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 0x55d48feed840 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009620 tx=0x7fb96c003e10 comp rx=0 tx=0).stop 2024-09-11T01:09:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb97d72b740 1 -- 172.21.15.78:0/2351587504 shutdown_connections 2024-09-11T01:09:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fef0790 0x55d48feee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee4800 0x55d48feedd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d48ff047d0 0x55d48ff06bc0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2351587504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 0x55d48feed840 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb97d72b740 1 -- 172.21.15.78:0/2351587504 >> 172.21.15.78:0/2351587504 conn(0x55d48fee1d10 msgr2=0x55d48fee0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:02.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb95cff9640 0 client.17451 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:02.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb97d72b740 1 -- 172.21.15.78:0/2351587504 shutdown_connections 2024-09-11T01:09:02.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb97d72b740 1 -- 172.21.15.78:0/2351587504 wait complete. 2024-09-11T01:09:02.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.940+0000 7fb97d72b740 20 client.17451 trim_cache size 0 max 16384 2024-09-11T01:09:02.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.958+0000 7fb97d72b740 1 Processor -- start 2024-09-11T01:09:02.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb97d72b740 1 -- start start 2024-09-11T01:09:02.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb97d72b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee3ee0 0x55d48fefff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb97d72b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee4800 0x55d48ff0c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb97d72b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fef0790 0x55d48ff0c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb97d72b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741ac4f0 con 0x55d48fef0790 2024-09-11T01:09:02.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb97d72b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741ae060 con 0x55d48fee3ee0 2024-09-11T01:09:02.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb97d72b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741acbc0 con 0x55d48fee4800 2024-09-11T01:09:02.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb9798f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fef0790 0x55d48ff0c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb9788f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee4800 0x55d48ff0c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb9798f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee3ee0 0x55d48fefff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb9788f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee4800 0x55d48ff0c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35884/0 (socket says 172.21.15.78:35884) 2024-09-11T01:09:02.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb9798f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee3ee0 0x55d48fefff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:55864/0 (socket says 172.21.15.78:55864) 2024-09-11T01:09:02.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb9788f1640 1 -- 172.21.15.78:0/2305711893 learned_addr learned my addr 172.21.15.78:0/2305711893 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:02.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb9788f1640 1 -- 172.21.15.78:0/2305711893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee3ee0 msgr2=0x55d48fefff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:02.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb9788f1640 1 --2- 172.21.15.78:0/2305711893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee3ee0 0x55d48fefff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb9788f1640 1 -- 172.21.15.78:0/2305711893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fef0790 msgr2=0x55d48ff0c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb9788f1640 1 --2- 172.21.15.78:0/2305711893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fef0790 0x55d48ff0c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb9788f1640 1 -- 172.21.15.78:0/2305711893 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d48fee2790 con 0x55d48fee4800 2024-09-11T01:09:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.959+0000 7fb9798f3640 1 --2- 172.21.15.78:0/2305711893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee3ee0 0x55d48fefff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.960+0000 7fb9788f1640 1 --2- 172.21.15.78:0/2305711893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee4800 0x55d48ff0c1f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb964011a30 tx=0x7fb964005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=915c03389a41387a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.960+0000 7fb95effd640 1 -- 172.21.15.78:0/2305711893 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964020040 con 0x55d48fee4800 2024-09-11T01:09:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.960+0000 7fb95effd640 1 -- 172.21.15.78:0/2305711893 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb96401e440 con 0x55d48fee4800 2024-09-11T01:09:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.960+0000 7fb95effd640 1 -- 172.21.15.78:0/2305711893 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964024630 con 0x55d48fee4800 2024-09-11T01:09:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.961+0000 7fb97d72b740 1 -- 172.21.15.78:0/2305711893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee4800 msgr2=0x55d48ff0c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.961+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2305711893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee4800 0x55d48ff0c1f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb964011a30 tx=0x7fb964005f90 comp rx=0 tx=0).stop 2024-09-11T01:09:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.961+0000 7fb97d72b740 1 -- 172.21.15.78:0/2305711893 shutdown_connections 2024-09-11T01:09:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.961+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2305711893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fef0790 0x55d48ff0c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.961+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2305711893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fee4800 0x55d48ff0c1f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.961+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2305711893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee3ee0 0x55d48fefff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.961+0000 7fb97d72b740 1 -- 172.21.15.78:0/2305711893 >> 172.21.15.78:0/2305711893 conn(0x55d48fee1d10 msgr2=0x55d48feff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.961+0000 7fb97d72b740 1 -- 172.21.15.78:0/2305711893 shutdown_connections 2024-09-11T01:09:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.961+0000 7fb97d72b740 1 -- 172.21.15.78:0/2305711893 wait complete. 2024-09-11T01:09:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.961+0000 7fb97d72b740 1 -- start start 2024-09-11T01:09:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.961+0000 7fb97d72b740 1 Processor -- start 2024-09-11T01:09:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb97d72b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 0x55d48fefcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb97d72b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee4800 0x55d48fefd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb97d72b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fef0790 0x55d48fef9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb97d72b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741ad960 con 0x55d48fee3ee0 2024-09-11T01:09:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb97d72b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741ad290 con 0x55d48fee4800 2024-09-11T01:09:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb97d72b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741aa4f0 con 0x55d48fef0790 2024-09-11T01:09:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb97d72b740 10 client.0 _ll_register_callbacks cb 0x7fb96c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb9798f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 0x55d48fefcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb9798f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 0x55d48fefcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50530/0 (socket says 172.21.15.78:50530) 2024-09-11T01:09:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb9798f3640 1 -- 172.21.15.78:0/2717754478 learned_addr learned my addr 172.21.15.78:0/2717754478 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb9788f1640 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee4800 0x55d48fefd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb9798f3640 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fef0790 0x55d48fef9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb9788f1640 1 -- 172.21.15.78:0/2717754478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fef0790 msgr2=0x55d48fef9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb9788f1640 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fef0790 0x55d48fef9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb9788f1640 1 -- 172.21.15.78:0/2717754478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 msgr2=0x55d48fefcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb9788f1640 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 0x55d48fefcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb9788f1640 1 -- 172.21.15.78:0/2717754478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d48feefee0 con 0x55d48fee4800 2024-09-11T01:09:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb9798f3640 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 0x55d48fefcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb9788f1640 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee4800 0x55d48fefd2c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb964011a30 tx=0x7fb964009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4c23da485b4bbd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb95cff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:09:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.962+0000 7fb95cff9640 1 -- 172.21.15.78:0/2717754478 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964020040 con 0x55d48fee4800 2024-09-11T01:09:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb95cff9640 1 -- 172.21.15.78:0/2717754478 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fb96401e5f0 con 0x55d48fee4800 2024-09-11T01:09:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb97d72b740 1 -- 172.21.15.78:0/2717754478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb96401e440 con 0x55d48fee4800 2024-09-11T01:09:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb95cff9640 1 -- 172.21.15.78:0/2717754478 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb96401e7f0 con 0x55d48fee4800 2024-09-11T01:09:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb97d72b740 10 client.26663 fetch_fsmap learned FSMap version 47 2024-09-11T01:09:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb97d72b740 1 -- 172.21.15.78:0/2717754478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d48ff71fc0 con 0x55d48fee4800 2024-09-11T01:09:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb95cff9640 1 -- 172.21.15.78:0/2717754478 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb96401e440 con 0x55d48fee4800 2024-09-11T01:09:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb95cff9640 1 -- 172.21.15.78:0/2717754478 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7fb96402f700 con 0x55d48fee4800 2024-09-11T01:09:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb97d72b740 10 client.26663 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:09:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb97d72b740 10 client.26663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:09:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb97d72b740 20 client.26663 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d48ff726a0 0x55d48ff74ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb97d72b740 4 client.26663 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:09:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.963+0000 7fb97d72b740 1 -- 172.21.15.78:0/2717754478 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d48fe58860 con 0x55d48ff726a0 2024-09-11T01:09:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.964+0000 7fb9798f3640 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d48ff726a0 0x55d48ff74ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.964+0000 7fb95cff9640 10 client.26663 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.964+0000 7fb9798f3640 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d48ff726a0 0x55d48ff74ac0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a039d6a3e990a90 server_cookie=7d6975abb7b4d09b in_seq=0 out_seq=0 2024-09-11T01:09:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.965+0000 7fb95cff9640 1 -- 172.21.15.78:0/2717754478 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d48fe58860 con 0x55d48ff726a0 2024-09-11T01:09:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.965+0000 7fb95cff9640 10 client.26663 handle_command_reply: tid=0 2024-09-11T01:09:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.965+0000 7fb97d72b740 1 client.26663 shutdown 2024-09-11T01:09:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.965+0000 7fb97d72b740 1 -- 172.21.15.78:0/2717754478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee4800 msgr2=0x55d48fefd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.965+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee4800 0x55d48fefd2c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb964011a30 tx=0x7fb964009330 comp rx=0 tx=0).stop 2024-09-11T01:09:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.966+0000 7fb97d72b740 1 -- 172.21.15.78:0/2717754478 shutdown_connections 2024-09-11T01:09:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.966+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d48fef0790 0x55d48fef9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.966+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d48fee4800 0x55d48fefd2c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.966+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d48ff726a0 0x55d48ff74ac0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.966+0000 7fb97d72b740 1 --2- 172.21.15.78:0/2717754478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d48fee3ee0 0x55d48fefcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.966+0000 7fb97d72b740 1 -- 172.21.15.78:0/2717754478 >> 172.21.15.78:0/2717754478 conn(0x55d48fee1d10 msgr2=0x55d48fee0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.966+0000 7fb95cff9640 0 client.26663 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.966+0000 7fb97d72b740 1 -- 172.21.15.78:0/2717754478 shutdown_connections 2024-09-11T01:09:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.966+0000 7fb97d72b740 1 -- 172.21.15.78:0/2717754478 wait complete. 2024-09-11T01:09:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.966+0000 7fb97d72b740 20 client.26663 trim_cache size 0 max 16384 2024-09-11T01:09:02.977 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:09:02.977 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:09:02.978 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "114b76b0-8477-4d97-b164-754a49f52148", 2024-09-11T01:09:02.978 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:09:02.978 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:09:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.966+0000 7fb97b356640 1 -- 172.21.15.78:0/245953915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9600b5450 msgr2=0x7fb9600b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.966+0000 7fb97b356640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9600b5450 0x7fb9600b7910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb9700e3ac0 tx=0x7fb9700e8040 comp rx=0 tx=0).stop 2024-09-11T01:09:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.967+0000 7fb97b356640 1 -- 172.21.15.78:0/245953915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb974102dc0 msgr2=0x7fb9741992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.967+0000 7fb97b356640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb974102dc0 0x7fb9741992e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb97000ea60 tx=0x7fb97000ef20 comp rx=0 tx=0).stop 2024-09-11T01:09:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.967+0000 7fb97b356640 1 -- 172.21.15.78:0/245953915 shutdown_connections 2024-09-11T01:09:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.967+0000 7fb97b356640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9600b5450 0x7fb9600b7910 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.967+0000 7fb97b356640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb974109150 0x7fb97419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.967+0000 7fb97b356640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9741087d0 0x7fb974199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.967+0000 7fb97b356640 1 --2- 172.21.15.78:0/245953915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb974102dc0 0x7fb9741992e0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.967+0000 7fb97b356640 1 -- 172.21.15.78:0/245953915 >> 172.21.15.78:0/245953915 conn(0x7fb9740fe870 msgr2=0x7fb974100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.967+0000 7fb97b356640 1 -- 172.21.15.78:0/245953915 shutdown_connections 2024-09-11T01:09:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:02.967+0000 7fb97b356640 1 -- 172.21.15.78:0/245953915 wait complete. 2024-09-11T01:09:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:03 smithi110 ceph-mon[34712]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T01:09:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:03 smithi110 ceph-mon[34712]: pgmap v635: 97 pgs: 97 active+clean; 496 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-09-11T01:09:03.448 INFO:tasks.workunit.client.0.smithi078.stdout: 1 671559 5.23 MB/sec execute 431 sec latency 34.088 ms 2024-09-11T01:09:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:03 smithi112 ceph-mon[35384]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T01:09:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:03 smithi112 ceph-mon[35384]: pgmap v635: 97 pgs: 97 active+clean; 496 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-09-11T01:09:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:03 smithi078 ceph-mon[31315]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T01:09:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:03 smithi078 ceph-mon[31315]: pgmap v635: 97 pgs: 97 active+clean; 496 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-09-11T01:09:03.729 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:09:04.003 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:09:04.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:09:04.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:04.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:04.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:04.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:04.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:04.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:04.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:09:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 672856 5.23 MB/sec execute 432 sec latency 213.566 ms 2024-09-11T01:09:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:09:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:04.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:04.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:04.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:04.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.505+0000 7f63229bc640 1 -- 172.21.15.78:0/652963257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631c1076c0 msgr2=0x7f631c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.505+0000 7f63229bc640 1 --2- 172.21.15.78:0/652963257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631c1076c0 0x7f631c107b00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6304009900 tx=0x7f630402a410 comp rx=0 tx=0).stop 2024-09-11T01:09:04.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.505+0000 7f63229bc640 1 -- 172.21.15.78:0/652963257 shutdown_connections 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.505+0000 7f63229bc640 1 --2- 172.21.15.78:0/652963257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f631c108040 0x7f631c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.505+0000 7f63229bc640 1 --2- 172.21.15.78:0/652963257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631c1076c0 0x7f631c107b00 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.505+0000 7f63229bc640 1 --2- 172.21.15.78:0/652963257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f631c101cb0 0x7f631c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.505+0000 7f63229bc640 1 -- 172.21.15.78:0/652963257 >> 172.21.15.78:0/652963257 conn(0x7f631c0fd700 msgr2=0x7f631c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.505+0000 7f63229bc640 1 -- 172.21.15.78:0/652963257 shutdown_connections 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.505+0000 7f63229bc640 1 -- 172.21.15.78:0/652963257 wait complete. 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f63229bc640 1 Processor -- start 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f63229bc640 1 -- start start 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f63229bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f631c101cb0 0x7f631c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f63229bc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f631c1076c0 0x7f631c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f63229bc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631c108040 0x7f631c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f63229bc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f631c111cc0 con 0x7f631c108040 2024-09-11T01:09:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f63229bc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f631c111b40 con 0x7f631c1076c0 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f63229bc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f631c111e40 con 0x7f631c101cb0 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f6320f59640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631c108040 0x7f631c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f6320f59640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631c108040 0x7f631c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39120/0 (socket says 172.21.15.78:39120) 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f6320f59640 1 -- 172.21.15.78:0/3073903729 learned_addr learned my addr 172.21.15.78:0/3073903729 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f631b7fe640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f631c1076c0 0x7f631c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.506+0000 7f631bfff640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f631c101cb0 0x7f631c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.507+0000 7f6320f59640 1 -- 172.21.15.78:0/3073903729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f631c101cb0 msgr2=0x7f631c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.507+0000 7f6320f59640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f631c101cb0 0x7f631c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.507+0000 7f6320f59640 1 -- 172.21.15.78:0/3073903729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f631c1076c0 msgr2=0x7f631c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.507+0000 7f6320f59640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f631c1076c0 0x7f631c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.507+0000 7f6320f59640 1 -- 172.21.15.78:0/3073903729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c1a7680 con 0x7f631c108040 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.507+0000 7f6320f59640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631c108040 0x7f631c198920 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f631400eb10 tx=0x7f631400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.507+0000 7f63197fa640 1 -- 172.21.15.78:0/3073903729 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314004b20 con 0x7f631c108040 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.507+0000 7f63229bc640 1 -- 172.21.15.78:0/3073903729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631c1a7970 con 0x7f631c108040 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.507+0000 7f63197fa640 1 -- 172.21.15.78:0/3073903729 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6314004cc0 con 0x7f631c108040 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.507+0000 7f63197fa640 1 -- 172.21.15.78:0/3073903729 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314010670 con 0x7f631c108040 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.507+0000 7f63229bc640 1 -- 172.21.15.78:0/3073903729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631c1a7e30 con 0x7f631c108040 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.510+0000 7f6324d91740 1 Processor -- start 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.510+0000 7f63197fa640 1 -- 172.21.15.78:0/3073903729 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6314005020 con 0x7f631c108040 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.510+0000 7f6324d91740 1 -- start start 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f6324d91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 0x5591c644e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f6324d91740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c645a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f6324d91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c645ad30 0x5591c645d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f6324d91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5591c63c4e50 con 0x5591c644e480 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f6324d91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591c63b4280 con 0x5591c645ad30 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f6324d91740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591c63b8090 con 0x5591c644eda0 2024-09-11T01:09:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 0x5591c644e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 0x5591c644e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39134/0 (socket says 172.21.15.78:39134) 2024-09-11T01:09:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631b7fe640 1 -- 172.21.15.78:0/535168604 learned_addr learned my addr 172.21.15.78:0/535168604 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631bfff640 1 --2- 172.21.15.78:0/535168604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c645a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631b7fe640 1 --2- 172.21.15.78:0/535168604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c645ad30 0x5591c645d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631b7fe640 1 -- 172.21.15.78:0/535168604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 msgr2=0x5591c645a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631b7fe640 1 --2- 172.21.15.78:0/535168604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c645a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631b7fe640 1 -- 172.21.15.78:0/535168604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c645ad30 msgr2=0x5591c645d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631b7fe640 1 --2- 172.21.15.78:0/535168604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c645ad30 0x5591c645d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631b7fe640 1 -- 172.21.15.78:0/535168604 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c6112650 con 0x5591c644e480 2024-09-11T01:09:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631bfff640 1 --2- 172.21.15.78:0/535168604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c645a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f63197fa640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63080b5450 0x7f63080b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631b7fe640 1 --2- 172.21.15.78:0/535168604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 0x5591c644e860 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f630402a930 tx=0x7f63040095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9423eb4f364365 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f631bfff640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63080b5450 0x7f63080b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.511+0000 7f63197fa640 1 -- 172.21.15.78:0/3073903729 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f63140e65b0 con 0x7f631c108040 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f63127fc640 1 -- 172.21.15.78:0/535168604 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304035040 con 0x5591c644e480 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f63127fc640 1 -- 172.21.15.78:0/535168604 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6304004030 con 0x5591c644e480 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f63127fc640 1 -- 172.21.15.78:0/535168604 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304004750 con 0x5591c644e480 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f6324d91740 1 -- 172.21.15.78:0/535168604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 msgr2=0x5591c644e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f6324d91740 1 --2- 172.21.15.78:0/535168604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 0x5591c644e860 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f630402a930 tx=0x7f63040095d0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f631bfff640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63080b5450 0x7f63080b7910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x5591c644d190 tx=0x7f630c00a250 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f6324d91740 1 -- 172.21.15.78:0/535168604 shutdown_connections 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f6324d91740 1 --2- 172.21.15.78:0/535168604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c645ad30 0x5591c645d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f6324d91740 1 --2- 172.21.15.78:0/535168604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c645a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f6324d91740 1 --2- 172.21.15.78:0/535168604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 0x5591c644e860 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f6324d91740 1 -- 172.21.15.78:0/535168604 >> 172.21.15.78:0/535168604 conn(0x5591c644c2b0 msgr2=0x5591c644d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:04.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f6324d91740 1 -- 172.21.15.78:0/535168604 shutdown_connections 2024-09-11T01:09:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.512+0000 7f6324d91740 1 -- 172.21.15.78:0/535168604 wait complete. 2024-09-11T01:09:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.513+0000 7f6324d91740 1 -- start start 2024-09-11T01:09:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.513+0000 7f6324d91740 1 Processor -- start 2024-09-11T01:09:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.513+0000 7f6324d91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c644e480 0x5591c645a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.513+0000 7f6324d91740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c6457a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.513+0000 7f6324d91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 0x5591c64580c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.513+0000 7f6324d91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5591c6215700 con 0x5591c645ad30 2024-09-11T01:09:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.513+0000 7f6324d91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591c634f110 con 0x5591c644e480 2024-09-11T01:09:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.513+0000 7f6324d91740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591c645d950 con 0x5591c644eda0 2024-09-11T01:09:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.513+0000 7f6324d91740 10 client.0 _ll_register_callbacks cb 0x5591c6368990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:04.615 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "114b76b0-8477-4d97-b164-754a49f52148", 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.616 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:09:04.617 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.618 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.620 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:09:04.620 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:09:04.620 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.620 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.620 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.620 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.620 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.620 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:09:04.620 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:09:04.620 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.620 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:09:04.621 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.622 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.623 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:04.624 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:09:04.624 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:09:04.624 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:04.624 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:04.624 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:04.624 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:04.624 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:04.624 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:09:04.624 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:09:04.624 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:09:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.513+0000 7f6320f59640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 0x5591c64580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.513+0000 7f6320f59640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 0x5591c64580c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39138/0 (socket says 172.21.15.78:39138) 2024-09-11T01:09:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.513+0000 7f6320f59640 1 -- 172.21.15.78:0/3551161104 learned_addr learned my addr 172.21.15.78:0/3551161104 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f631b7fe640 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c6457a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f631bfff640 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c644e480 0x5591c645a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f6320f59640 1 -- 172.21.15.78:0/3551161104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 msgr2=0x5591c6457a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f6320f59640 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c6457a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f6320f59640 1 -- 172.21.15.78:0/3551161104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c644e480 msgr2=0x5591c645a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f6320f59640 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c644e480 0x5591c645a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f6320f59640 1 -- 172.21.15.78:0/3551161104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c637cc40 con 0x5591c645ad30 2024-09-11T01:09:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f6320f59640 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 0x5591c64580c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f63140ac9e0 tx=0x7f63140ac950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4d8e533240a1d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f62fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f62fb7fe640 1 -- 172.21.15.78:0/3551161104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63140b5d50 con 0x5591c645ad30 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f62fb7fe640 1 -- 172.21.15.78:0/3551161104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f63140b1070 con 0x5591c645ad30 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f6324d91740 1 -- 172.21.15.78:0/3551161104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6304004030 con 0x5591c645ad30 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f62fb7fe640 1 -- 172.21.15.78:0/3551161104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f63140c7050 con 0x5591c645ad30 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f6324d91740 10 client.17469 fetch_fsmap learned FSMap version 47 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f6324d91740 1 -- 172.21.15.78:0/3551161104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5591c6469310 con 0x5591c645ad30 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f62fb7fe640 1 -- 172.21.15.78:0/3551161104 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6304004030 con 0x5591c645ad30 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.514+0000 7f62fb7fe640 1 -- 172.21.15.78:0/3551161104 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f63140abcb0 con 0x5591c645ad30 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.515+0000 7f6324d91740 10 client.17469 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.515+0000 7f6324d91740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.515+0000 7f6324d91740 20 client.17469 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.515+0000 7f6324d91740 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5591c646a820 0x5591c646cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.515+0000 7f6324d91740 4 client.17469 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.515+0000 7f6324d91740 1 -- 172.21.15.78:0/3551161104 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591c63c6da0 con 0x5591c646a820 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.515+0000 7f631bfff640 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5591c646a820 0x5591c646cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.515+0000 7f62fb7fe640 10 client.17469 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.515+0000 7f631bfff640 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5591c646a820 0x5591c646cc10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0e49fe89fedfbf0 server_cookie=57f68d4c17941ccb in_seq=0 out_seq=0 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f62fb7fe640 1 -- 172.21.15.78:0/3551161104 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591c63c6da0 con 0x5591c646a820 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f62fb7fe640 10 client.17469 handle_command_reply: tid=0 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f6324d91740 1 client.17469 shutdown 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f6324d91740 1 -- 172.21.15.78:0/3551161104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 msgr2=0x5591c64580c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f6324d91740 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 0x5591c64580c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f63140ac9e0 tx=0x7f63140ac950 comp rx=0 tx=0).stop 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f6324d91740 1 -- 172.21.15.78:0/3551161104 shutdown_connections 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f6324d91740 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5591c646a820 0x5591c646cc10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f6324d91740 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 0x5591c64580c0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f62fb7fe640 0 client.17469 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f6324d91740 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c6457a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f6324d91740 1 --2- 172.21.15.78:0/3551161104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c644e480 0x5591c645a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f6324d91740 1 -- 172.21.15.78:0/3551161104 >> 172.21.15.78:0/3551161104 conn(0x5591c644c2b0 msgr2=0x5591c6451e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f6324d91740 1 -- 172.21.15.78:0/3551161104 shutdown_connections 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f6324d91740 1 -- 172.21.15.78:0/3551161104 wait complete. 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.516+0000 7f6324d91740 20 client.17469 trim_cache size 0 max 16384 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.533+0000 7f6324d91740 1 Processor -- start 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.533+0000 7f6324d91740 1 -- start start 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6324d91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c644e480 0x5591c64472d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6324d91740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c644bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6324d91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 0x5591c6456310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6324d91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1b07f0 con 0x5591c645ad30 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6324d91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1b2360 con 0x5591c644e480 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6324d91740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1b0ec0 con 0x5591c644eda0 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6320f59640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 0x5591c6456310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6320f59640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 0x5591c6456310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39154/0 (socket says 172.21.15.78:39154) 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6320f59640 1 -- 172.21.15.78:0/235251086 learned_addr learned my addr 172.21.15.78:0/235251086 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f631b7fe640 1 --2- 172.21.15.78:0/235251086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c644bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f631bfff640 1 --2- 172.21.15.78:0/235251086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c644e480 0x5591c64472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6320f59640 1 -- 172.21.15.78:0/235251086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 msgr2=0x5591c644bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6320f59640 1 --2- 172.21.15.78:0/235251086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c644bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6320f59640 1 -- 172.21.15.78:0/235251086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c644e480 msgr2=0x5591c64472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6320f59640 1 --2- 172.21.15.78:0/235251086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c644e480 0x5591c64472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6320f59640 1 -- 172.21.15.78:0/235251086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c644cf90 con 0x5591c645ad30 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f631bfff640 1 --2- 172.21.15.78:0/235251086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c644e480 0x5591c64472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6320f59640 1 --2- 172.21.15.78:0/235251086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 0x5591c6456310 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f63140ac020 tx=0x7f6314004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996a7441d245c9fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6311ffb640 1 -- 172.21.15.78:0/235251086 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63140cb070 con 0x5591c645ad30 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6311ffb640 1 -- 172.21.15.78:0/235251086 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6314005550 con 0x5591c645ad30 2024-09-11T01:09:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.534+0000 7f6311ffb640 1 -- 172.21.15.78:0/235251086 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63140ad150 con 0x5591c645ad30 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.535+0000 7f6324d91740 1 -- 172.21.15.78:0/235251086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 msgr2=0x5591c6456310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.535+0000 7f6324d91740 1 --2- 172.21.15.78:0/235251086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 0x5591c6456310 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f63140ac020 tx=0x7f6314004f30 comp rx=0 tx=0).stop 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.535+0000 7f6324d91740 1 -- 172.21.15.78:0/235251086 shutdown_connections 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.535+0000 7f6324d91740 1 --2- 172.21.15.78:0/235251086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c645ad30 0x5591c6456310 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.535+0000 7f6324d91740 1 --2- 172.21.15.78:0/235251086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c644bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.535+0000 7f6324d91740 1 --2- 172.21.15.78:0/235251086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c644e480 0x5591c64472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.535+0000 7f6324d91740 1 -- 172.21.15.78:0/235251086 >> 172.21.15.78:0/235251086 conn(0x5591c644c2b0 msgr2=0x5591c6452ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.535+0000 7f6324d91740 1 -- 172.21.15.78:0/235251086 shutdown_connections 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.535+0000 7f6324d91740 1 -- 172.21.15.78:0/235251086 wait complete. 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.535+0000 7f6324d91740 1 -- start start 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f6324d91740 1 Processor -- start 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f6324d91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 0x5591c6462bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f6324d91740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c6463130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f6324d91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c645ad30 0x5591c645dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f6324d91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1b1c60 con 0x5591c644e480 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f6324d91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1b1590 con 0x5591c645ad30 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f6324d91740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1ae7f0 con 0x5591c644eda0 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f6324d91740 10 client.0 _ll_register_callbacks cb 0x7f63140a8960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f631bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 0x5591c6462bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f631bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 0x5591c6462bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39156/0 (socket says 172.21.15.78:39156) 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f631bfff640 1 -- 172.21.15.78:0/215026559 learned_addr learned my addr 172.21.15.78:0/215026559 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f631b7fe640 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c6463130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f6320f59640 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c645ad30 0x5591c645dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f631bfff640 1 -- 172.21.15.78:0/215026559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 msgr2=0x5591c6463130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f631bfff640 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c6463130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f631bfff640 1 -- 172.21.15.78:0/215026559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c645ad30 msgr2=0x5591c645dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f631bfff640 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c645ad30 0x5591c645dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f631bfff640 1 -- 172.21.15.78:0/215026559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c6463870 con 0x5591c644e480 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f631bfff640 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 0x5591c6462bf0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f630c01b710 tx=0x7f630c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f20875ea4ae26a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.536+0000 7f62fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f62fb7fe640 1 -- 172.21.15.78:0/215026559 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c033060 con 0x5591c644e480 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f62fb7fe640 1 -- 172.21.15.78:0/215026559 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f630c028430 con 0x5591c644e480 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f6324d91740 1 -- 172.21.15.78:0/215026559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6314005550 con 0x5591c644e480 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f62fb7fe640 1 -- 172.21.15.78:0/215026559 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f630c02a040 con 0x5591c644e480 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f6324d91740 10 client.17475 fetch_fsmap learned FSMap version 47 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f6324d91740 1 -- 172.21.15.78:0/215026559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5591c64e04a0 con 0x5591c644e480 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f62fb7fe640 1 -- 172.21.15.78:0/215026559 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6314005550 con 0x5591c644e480 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f62fb7fe640 1 -- 172.21.15.78:0/215026559 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f630c02e680 con 0x5591c644e480 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f6324d91740 10 client.17475 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f6324d91740 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:09:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f6324d91740 20 client.17475 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f6324d91740 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5591c646a820 0x5591c64e2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f6324d91740 4 client.17475 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f6324d91740 1 -- 172.21.15.78:0/215026559 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5591c63c2850 con 0x5591c646a820 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.537+0000 7f631b7fe640 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5591c646a820 0x5591c64e2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f62fb7fe640 10 client.17475 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f631b7fe640 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5591c646a820 0x5591c64e2ca0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e54feab9ae3a8379 server_cookie=5511de383be283b1 in_seq=0 out_seq=0 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f62fb7fe640 1 -- 172.21.15.78:0/215026559 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x5591c63c2850 con 0x5591c646a820 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f62fb7fe640 10 client.17475 handle_command_reply: tid=0 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f6324d91740 1 client.17475 shutdown 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f6324d91740 1 -- 172.21.15.78:0/215026559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 msgr2=0x5591c6462bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f6324d91740 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 0x5591c6462bf0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f630c01b710 tx=0x7f630c0046c0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f6324d91740 1 -- 172.21.15.78:0/215026559 shutdown_connections 2024-09-11T01:09:04.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f6324d91740 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591c645ad30 0x5591c645dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f6324d91740 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591c644eda0 0x5591c6463130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f6324d91740 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5591c646a820 0x5591c64e2ca0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f62fb7fe640 0 client.17475 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f6324d91740 1 --2- 172.21.15.78:0/215026559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591c644e480 0x5591c6462bf0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f6324d91740 1 -- 172.21.15.78:0/215026559 >> 172.21.15.78:0/215026559 conn(0x5591c644c2b0 msgr2=0x5591c64590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f6324d91740 1 -- 172.21.15.78:0/215026559 shutdown_connections 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f6324d91740 1 -- 172.21.15.78:0/215026559 wait complete. 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.538+0000 7f6324d91740 20 client.17475 trim_cache size 0 max 16384 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.539+0000 7f63229bc640 1 -- 172.21.15.78:0/3073903729 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63080b5450 msgr2=0x7f63080b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.539+0000 7f63229bc640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63080b5450 0x7f63080b7910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x5591c644d190 tx=0x7f630c00a250 comp rx=0 tx=0).stop 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.539+0000 7f63229bc640 1 -- 172.21.15.78:0/3073903729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631c108040 msgr2=0x7f631c198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.539+0000 7f63229bc640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631c108040 0x7f631c198920 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f631400eb10 tx=0x7f631400c2d0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.539+0000 7f63229bc640 1 -- 172.21.15.78:0/3073903729 shutdown_connections 2024-09-11T01:09:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.539+0000 7f63229bc640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63080b5450 0x7f63080b7910 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.539+0000 7f63229bc640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f631c108040 0x7f631c198920 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.539+0000 7f63229bc640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f631c1076c0 0x7f631c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.539+0000 7f63229bc640 1 --2- 172.21.15.78:0/3073903729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f631c101cb0 0x7f631c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:04.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.539+0000 7f63229bc640 1 -- 172.21.15.78:0/3073903729 >> 172.21.15.78:0/3073903729 conn(0x7f631c0fd700 msgr2=0x7f631c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:04.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.539+0000 7f63229bc640 1 -- 172.21.15.78:0/3073903729 shutdown_connections 2024-09-11T01:09:04.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:04.539+0000 7f63229bc640 1 -- 172.21.15.78:0/3073903729 wait complete. 2024-09-11T01:09:05.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:05 smithi110 ceph-mon[34712]: pgmap v636: 97 pgs: 97 active+clean; 499 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-09-11T01:09:05.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 672856 5.22 MB/sec execute 433 sec latency 1213.603 ms 2024-09-11T01:09:05.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:05 smithi112 ceph-mon[35384]: pgmap v636: 97 pgs: 97 active+clean; 499 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-09-11T01:09:05.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:05 smithi078 ceph-mon[31315]: pgmap v636: 97 pgs: 97 active+clean; 499 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 203 op/s 2024-09-11T01:09:05.591 INFO:tasks.cephfs.filesystem:scrub status for tag:114b76b0-8477-4d97-b164-754a49f52148 - {'path': '/', 'tag': '114b76b0-8477-4d97-b164-754a49f52148', 'options': 'recursive,force'} 2024-09-11T01:09:06.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 672856 5.20 MB/sec execute 434 sec latency 2213.701 ms 2024-09-11T01:09:06.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:06 smithi110 ceph-mon[34712]: pgmap v637: 97 pgs: 97 active+clean; 499 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-09-11T01:09:06.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:06 smithi112 ceph-mon[35384]: pgmap v637: 97 pgs: 97 active+clean; 499 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-09-11T01:09:06.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:06 smithi078 ceph-mon[31315]: pgmap v637: 97 pgs: 97 active+clean; 499 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-09-11T01:09:07.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 673588 5.20 MB/sec execute 435 sec latency 2966.542 ms 2024-09-11T01:09:08.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 675206 5.21 MB/sec execute 436 sec latency 13.592 ms 2024-09-11T01:09:08.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:08 smithi110 ceph-mon[34712]: pgmap v638: 97 pgs: 97 active+clean; 501 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-11T01:09:08.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:08 smithi112 ceph-mon[35384]: pgmap v638: 97 pgs: 97 active+clean; 501 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-11T01:09:08.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:08 smithi078 ceph-mon[31315]: pgmap v638: 97 pgs: 97 active+clean; 501 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-11T01:09:09.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 676458 5.21 MB/sec execute 437 sec latency 284.504 ms 2024-09-11T01:09:09.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:09 smithi110 ceph-mon[34712]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T01:09:09.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:09 smithi112 ceph-mon[35384]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T01:09:09.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:09 smithi078 ceph-mon[31315]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T01:09:10.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 676458 5.20 MB/sec execute 438 sec latency 1284.535 ms 2024-09-11T01:09:10.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:10 smithi110 ceph-mon[34712]: pgmap v640: 97 pgs: 97 active+clean; 502 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-09-11T01:09:10.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:10 smithi112 ceph-mon[35384]: pgmap v640: 97 pgs: 97 active+clean; 502 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-09-11T01:09:10.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:10 smithi078 ceph-mon[31315]: pgmap v640: 97 pgs: 97 active+clean; 502 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-09-11T01:09:11.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 678135 5.21 MB/sec execute 439 sec latency 1551.475 ms 2024-09-11T01:09:11.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:11 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:09:11.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:11 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:09:11.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:11 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:09:12.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 679348 5.21 MB/sec execute 440 sec latency 16.093 ms 2024-09-11T01:09:13.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:09:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:09:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:09:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:09:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:09:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: pgmap v641: 97 pgs: 97 active+clean; 503 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-09-11T01:09:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:09:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:09:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:12 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:09:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:09:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:09:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:09:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:09:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: pgmap v641: 97 pgs: 97 active+clean; 503 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-09-11T01:09:13.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:09:13.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:09:13.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:12 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 680001 5.21 MB/sec execute 441 sec latency 651.119 ms 2024-09-11T01:09:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:09:13.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:09:13.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:09:13.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:09:13.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:09:13.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:13.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: pgmap v641: 97 pgs: 97 active+clean; 503 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-09-11T01:09:13.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:09:13.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:09:13.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:12 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:09:14.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 680001 5.20 MB/sec execute 442 sec latency 1651.162 ms 2024-09-11T01:09:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:14 smithi110 ceph-mon[34712]: pgmap v642: 97 pgs: 97 active+clean; 504 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-11T01:09:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:14 smithi112 ceph-mon[35384]: pgmap v642: 97 pgs: 97 active+clean; 504 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-11T01:09:14.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:14 smithi078 ceph-mon[31315]: pgmap v642: 97 pgs: 97 active+clean; 504 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-11T01:09:15.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 680001 5.18 MB/sec execute 443 sec latency 2651.253 ms 2024-09-11T01:09:16.449 INFO:tasks.workunit.client.0.smithi078.stdout: 1 680001 5.17 MB/sec execute 444 sec latency 3651.382 ms 2024-09-11T01:09:16.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:16 smithi110 ceph-mon[34712]: pgmap v643: 97 pgs: 97 active+clean; 508 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 167 op/s 2024-09-11T01:09:16.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:16 smithi112 ceph-mon[35384]: pgmap v643: 97 pgs: 97 active+clean; 508 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 167 op/s 2024-09-11T01:09:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:16 smithi078 ceph-mon[31315]: pgmap v643: 97 pgs: 97 active+clean; 508 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 167 op/s 2024-09-11T01:09:17.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 680717 5.17 MB/sec execute 445 sec latency 4404.395 ms 2024-09-11T01:09:18.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 682449 5.18 MB/sec execute 446 sec latency 11.966 ms 2024-09-11T01:09:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:18 smithi110 ceph-mon[34712]: pgmap v644: 97 pgs: 97 active+clean; 508 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.9 MiB/s wr, 143 op/s 2024-09-11T01:09:18.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:18 smithi112 ceph-mon[35384]: pgmap v644: 97 pgs: 97 active+clean; 508 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.9 MiB/s wr, 143 op/s 2024-09-11T01:09:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:18 smithi078 ceph-mon[31315]: pgmap v644: 97 pgs: 97 active+clean; 508 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.9 MiB/s wr, 143 op/s 2024-09-11T01:09:19.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 683549 5.18 MB/sec execute 447 sec latency 418.300 ms 2024-09-11T01:09:20.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 683549 5.17 MB/sec execute 448 sec latency 1418.390 ms 2024-09-11T01:09:20.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:20 smithi110 ceph-mon[34712]: pgmap v645: 97 pgs: 97 active+clean; 509 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.0 MiB/s wr, 134 op/s 2024-09-11T01:09:20.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:20 smithi112 ceph-mon[35384]: pgmap v645: 97 pgs: 97 active+clean; 509 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.0 MiB/s wr, 134 op/s 2024-09-11T01:09:20.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:20 smithi078 ceph-mon[31315]: pgmap v645: 97 pgs: 97 active+clean; 509 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.0 MiB/s wr, 134 op/s 2024-09-11T01:09:21.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 683549 5.16 MB/sec execute 449 sec latency 2418.493 ms 2024-09-11T01:09:22.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 684259 5.15 MB/sec execute 450 sec latency 3164.253 ms 2024-09-11T01:09:22.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:22 smithi110 ceph-mon[34712]: pgmap v646: 97 pgs: 97 active+clean; 512 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-11T01:09:22.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:22 smithi112 ceph-mon[35384]: pgmap v646: 97 pgs: 97 active+clean; 512 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-11T01:09:22.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:22 smithi078 ceph-mon[31315]: pgmap v646: 97 pgs: 97 active+clean; 512 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-11T01:09:23.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 685952 5.17 MB/sec execute 451 sec latency 9.635 ms 2024-09-11T01:09:24.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 687104 5.17 MB/sec execute 452 sec latency 314.904 ms 2024-09-11T01:09:24.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:24 smithi110 ceph-mon[34712]: pgmap v647: 97 pgs: 97 active+clean; 513 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-09-11T01:09:24.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:24 smithi112 ceph-mon[35384]: pgmap v647: 97 pgs: 97 active+clean; 513 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-09-11T01:09:24.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:24 smithi078 ceph-mon[31315]: pgmap v647: 97 pgs: 97 active+clean; 513 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-09-11T01:09:25.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 689272 5.18 MB/sec execute 453 sec latency 12.355 ms 2024-09-11T01:09:26.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 690640 5.19 MB/sec execute 454 sec latency 95.506 ms 2024-09-11T01:09:26.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:26 smithi110 ceph-mon[34712]: pgmap v648: 97 pgs: 97 active+clean; 517 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.1 MiB/s wr, 161 op/s 2024-09-11T01:09:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:26 smithi112 ceph-mon[35384]: pgmap v648: 97 pgs: 97 active+clean; 517 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.1 MiB/s wr, 161 op/s 2024-09-11T01:09:26.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:26 smithi078 ceph-mon[31315]: pgmap v648: 97 pgs: 97 active+clean; 517 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.1 MiB/s wr, 161 op/s 2024-09-11T01:09:27.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 691346 5.18 MB/sec execute 455 sec latency 842.779 ms 2024-09-11T01:09:28.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 692963 5.19 MB/sec execute 456 sec latency 13.169 ms 2024-09-11T01:09:28.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:28 smithi110 ceph-mon[34712]: pgmap v649: 97 pgs: 97 active+clean; 521 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-11T01:09:28.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:28 smithi112 ceph-mon[35384]: pgmap v649: 97 pgs: 97 active+clean; 521 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-11T01:09:28.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:28 smithi078 ceph-mon[31315]: pgmap v649: 97 pgs: 97 active+clean; 521 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-11T01:09:29.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 694183 5.19 MB/sec execute 457 sec latency 307.671 ms 2024-09-11T01:09:30.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 694183 5.18 MB/sec execute 458 sec latency 1307.761 ms 2024-09-11T01:09:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:30 smithi110 ceph-mon[34712]: pgmap v650: 97 pgs: 97 active+clean; 524 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 185 op/s 2024-09-11T01:09:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:30 smithi112 ceph-mon[35384]: pgmap v650: 97 pgs: 97 active+clean; 524 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 185 op/s 2024-09-11T01:09:30.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:30 smithi078 ceph-mon[31315]: pgmap v650: 97 pgs: 97 active+clean; 524 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 185 op/s 2024-09-11T01:09:31.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 694183 5.17 MB/sec execute 459 sec latency 2307.872 ms 2024-09-11T01:09:32.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 695665 5.18 MB/sec execute 460 sec latency 2691.872 ms 2024-09-11T01:09:32.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:32 smithi110 ceph-mon[34712]: pgmap v651: 97 pgs: 97 active+clean; 524 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-09-11T01:09:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:32 smithi112 ceph-mon[35384]: pgmap v651: 97 pgs: 97 active+clean; 524 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-09-11T01:09:32.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:32 smithi078 ceph-mon[31315]: pgmap v651: 97 pgs: 97 active+clean; 524 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-09-11T01:09:33.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 697210 5.18 MB/sec execute 461 sec latency 11.353 ms 2024-09-11T01:09:34.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 699565 5.20 MB/sec execute 462 sec latency 16.830 ms 2024-09-11T01:09:34.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:34 smithi110 ceph-mon[34712]: pgmap v652: 97 pgs: 97 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-09-11T01:09:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:34 smithi112 ceph-mon[35384]: pgmap v652: 97 pgs: 97 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-09-11T01:09:34.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:34 smithi078 ceph-mon[31315]: pgmap v652: 97 pgs: 97 active+clean; 527 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-09-11T01:09:35.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 700985 5.21 MB/sec execute 463 sec latency 14.570 ms 2024-09-11T01:09:35.592 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:09:35.880 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:09:36.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.382+0000 7f056e259640 1 -- 172.21.15.78:0/2288666386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568102dd0 msgr2=0x7f05681031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:36.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.382+0000 7f056e259640 1 --2- 172.21.15.78:0/2288666386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568102dd0 0x7f05681031b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0558009900 tx=0x7f055802a410 comp rx=0 tx=0).stop 2024-09-11T01:09:36.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.382+0000 7f056e259640 1 -- 172.21.15.78:0/2288666386 shutdown_connections 2024-09-11T01:09:36.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.382+0000 7f056e259640 1 --2- 172.21.15.78:0/2288666386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0568109160 0x7f056810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.382+0000 7f056e259640 1 --2- 172.21.15.78:0/2288666386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05681087e0 0x7f0568108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.382+0000 7f056e259640 1 --2- 172.21.15.78:0/2288666386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568102dd0 0x7f05681031b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.382+0000 7f056e259640 1 -- 172.21.15.78:0/2288666386 >> 172.21.15.78:0/2288666386 conn(0x7f05680fe800 msgr2=0x7f0568100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:36.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.382+0000 7f056e259640 1 -- 172.21.15.78:0/2288666386 shutdown_connections 2024-09-11T01:09:36.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.382+0000 7f056e259640 1 -- 172.21.15.78:0/2288666386 wait complete. 2024-09-11T01:09:36.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.383+0000 7f056e259640 1 Processor -- start 2024-09-11T01:09:36.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.383+0000 7f056e259640 1 -- start start 2024-09-11T01:09:36.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.383+0000 7f056e259640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0568102dd0 0x7f0568199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.383+0000 7f056e259640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05681087e0 0x7f05681997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.383+0000 7f056e259640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568109160 0x7f056819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.383+0000 7f056e259640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0568112de0 con 0x7f0568109160 2024-09-11T01:09:36.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.383+0000 7f056e259640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0568112c60 con 0x7f05681087e0 2024-09-11T01:09:36.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.383+0000 7f056e259640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0568112f60 con 0x7f0568102dd0 2024-09-11T01:09:36.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0567fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568109160 0x7f056819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0567fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568109160 0x7f056819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60096/0 (socket says 172.21.15.78:60096) 2024-09-11T01:09:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0567fff640 1 -- 172.21.15.78:0/1505890770 learned_addr learned my addr 172.21.15.78:0/1505890770 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0566ffd640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05681087e0 0x7f05681997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f05677fe640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0568102dd0 0x7f0568199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0567fff640 1 -- 172.21.15.78:0/1505890770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0568102dd0 msgr2=0x7f0568199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0567fff640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0568102dd0 0x7f0568199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0567fff640 1 -- 172.21.15.78:0/1505890770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05681087e0 msgr2=0x7f05681997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0567fff640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05681087e0 0x7f05681997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0567fff640 1 -- 172.21.15.78:0/1505890770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056819f340 con 0x7f0568109160 2024-09-11T01:09:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0566ffd640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05681087e0 0x7f05681997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0567fff640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568109160 0x7f056819eb70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f056000eb10 tx=0x7f056000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0564ff9640 1 -- 172.21.15.78:0/1505890770 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560004b20 con 0x7f0568109160 2024-09-11T01:09:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f056e259640 1 -- 172.21.15.78:0/1505890770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056819f630 con 0x7f0568109160 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0564ff9640 1 -- 172.21.15.78:0/1505890770 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0560004cc0 con 0x7f0568109160 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.384+0000 7f0564ff9640 1 -- 172.21.15.78:0/1505890770 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560010670 con 0x7f0568109160 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.385+0000 7f056e259640 1 -- 172.21.15.78:0/1505890770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0568075380 con 0x7f0568109160 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.387+0000 7f0564ff9640 1 -- 172.21.15.78:0/1505890770 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0560010830 con 0x7f0568109160 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f057062e740 1 Processor -- start 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f057062e740 1 -- start start 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f057062e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cef0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f057062e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cef0da0 0x564a2cefc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f057062e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cefcd30 0x564a2ceff120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f057062e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564a2ce66e50 con 0x564a2cef0480 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f057062e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564a2ce56280 con 0x564a2cefcd30 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f057062e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564a2ce5a090 con 0x564a2cef0da0 2024-09-11T01:09:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f0566ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cef0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f05677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cef0da0 0x564a2cefc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f0566ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cef0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60106/0 (socket says 172.21.15.78:60106) 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f0566ffd640 1 -- 172.21.15.78:0/3679460631 learned_addr learned my addr 172.21.15.78:0/3679460631 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.388+0000 7f05677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cef0da0 0x564a2cefc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52920/0 (socket says 172.21.15.78:52920) 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.389+0000 7f0564ff9640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f05500b5450 0x7f05500b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.389+0000 7f05677fe640 1 -- 172.21.15.78:0/3679460631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cefcd30 msgr2=0x564a2ceff120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.389+0000 7f0566ffd640 1 --2- 172.21.15.78:0/3679460631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cefcd30 0x564a2ceff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.389+0000 7f0567fff640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f05500b5450 0x7f05500b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.389+0000 7f05677fe640 1 --2- 172.21.15.78:0/3679460631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cefcd30 0x564a2ceff120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.389+0000 7f05677fe640 1 -- 172.21.15.78:0/3679460631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 msgr2=0x564a2cef0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.389+0000 7f05677fe640 1 --2- 172.21.15.78:0/3679460631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cef0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.389+0000 7f05677fe640 1 -- 172.21.15.78:0/3679460631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a2cbb4650 con 0x564a2cef0da0 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.389+0000 7f0566ffd640 1 --2- 172.21.15.78:0/3679460631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cefcd30 0x564a2ceff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:09:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.389+0000 7f0564ff9640 1 -- 172.21.15.78:0/1505890770 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f05600e6600 con 0x7f0568109160 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.389+0000 7f05677fe640 1 --2- 172.21.15.78:0/3679460631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cef0da0 0x564a2cefc7f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0558009230 tx=0x7f0558032bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe32fb38114e5eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.390+0000 7f0567fff640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f05500b5450 0x7f05500b7910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f05600e3e40 tx=0x7f0560096040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.390+0000 7f054dffb640 1 -- 172.21.15.78:0/3679460631 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558040070 con 0x564a2cef0da0 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.390+0000 7f054dffb640 1 -- 172.21.15.78:0/3679460631 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0558004030 con 0x564a2cef0da0 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.390+0000 7f054dffb640 1 -- 172.21.15.78:0/3679460631 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558035040 con 0x564a2cef0da0 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.391+0000 7f057062e740 1 -- 172.21.15.78:0/3679460631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cef0da0 msgr2=0x564a2cefc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.391+0000 7f057062e740 1 --2- 172.21.15.78:0/3679460631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cef0da0 0x564a2cefc7f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0558009230 tx=0x7f0558032bd0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.391+0000 7f057062e740 1 -- 172.21.15.78:0/3679460631 shutdown_connections 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.391+0000 7f057062e740 1 --2- 172.21.15.78:0/3679460631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cefcd30 0x564a2ceff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.391+0000 7f057062e740 1 --2- 172.21.15.78:0/3679460631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cef0da0 0x564a2cefc7f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.391+0000 7f057062e740 1 --2- 172.21.15.78:0/3679460631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cef0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.391+0000 7f057062e740 1 -- 172.21.15.78:0/3679460631 >> 172.21.15.78:0/3679460631 conn(0x564a2ceee2b0 msgr2=0x564a2ceef450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.391+0000 7f057062e740 1 -- 172.21.15.78:0/3679460631 shutdown_connections 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.391+0000 7f057062e740 1 -- 172.21.15.78:0/3679460631 wait complete. 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.391+0000 7f057062e740 1 -- start start 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f057062e740 1 Processor -- start 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f057062e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cefc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f057062e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 0x564a2cef9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f057062e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cefcd30 0x564a2cefa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f057062e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564a2ccb7700 con 0x564a2cef0480 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f057062e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564a2cdf1110 con 0x564a2cef0da0 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f057062e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564a2ceff950 con 0x564a2cefcd30 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f057062e740 10 client.0 _ll_register_callbacks cb 0x564a2ce0a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f0566ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cefc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f05677fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 0x564a2cef9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f05677fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 0x564a2cef9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:32910/0 (socket says 172.21.15.78:32910) 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f0566ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cefc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60110/0 (socket says 172.21.15.78:60110) 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f05677fe640 1 -- 172.21.15.78:0/3367842720 learned_addr learned my addr 172.21.15.78:0/3367842720 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f0566ffd640 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cefcd30 0x564a2cefa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f0566ffd640 1 -- 172.21.15.78:0/3367842720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cefcd30 msgr2=0x564a2cefa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f0566ffd640 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cefcd30 0x564a2cefa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f0566ffd640 1 -- 172.21.15.78:0/3367842720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 msgr2=0x564a2cef9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.392+0000 7f0566ffd640 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 0x564a2cef9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.393+0000 7f0566ffd640 1 -- 172.21.15.78:0/3367842720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a2ce1ec40 con 0x564a2cef0480 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.393+0000 7f05677fe640 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 0x564a2cef9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.393+0000 7f0566ffd640 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cefc610 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f055c013e30 tx=0x7f055c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9332f713a4e847 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.393+0000 7f0546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.393+0000 7f0546ffd640 1 -- 172.21.15.78:0/3367842720 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0130d0 con 0x564a2cef0480 2024-09-11T01:09:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.393+0000 7f057062e740 1 -- 172.21.15.78:0/3367842720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0558004030 con 0x564a2cef0480 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.393+0000 7f0546ffd640 1 -- 172.21.15.78:0/3367842720 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f055c011440 con 0x564a2cef0480 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.393+0000 7f0546ffd640 1 -- 172.21.15.78:0/3367842720 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f055c013270 con 0x564a2cef0480 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.393+0000 7f057062e740 10 client.17487 fetch_fsmap learned FSMap version 47 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.393+0000 7f057062e740 1 -- 172.21.15.78:0/3367842720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564a2cf0b230 con 0x564a2cef0480 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.393+0000 7f0546ffd640 1 -- 172.21.15.78:0/3367842720 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0558004030 con 0x564a2cef0480 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.394+0000 7f0546ffd640 1 -- 172.21.15.78:0/3367842720 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f055c011a30 con 0x564a2cef0480 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.394+0000 7f057062e740 10 client.17487 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.394+0000 7f057062e740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.394+0000 7f057062e740 20 client.17487 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.394+0000 7f057062e740 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564a2cf0c6e0 0x564a2cf0ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.394+0000 7f057062e740 4 client.17487 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.394+0000 7f057062e740 1 -- 172.21.15.78:0/3367842720 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a2ce68da0 con 0x564a2cf0c6e0 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.394+0000 7f05677fe640 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564a2cf0c6e0 0x564a2cf0ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.394+0000 7f0546ffd640 10 client.17487 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.394+0000 7f05677fe640 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564a2cf0c6e0 0x564a2cf0ead0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e3d840c1883a78 server_cookie=a98da6a97f3da49b in_seq=0 out_seq=0 2024-09-11T01:09:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.395+0000 7f0546ffd640 1 -- 172.21.15.78:0/3367842720 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564a2ce68da0 con 0x564a2cf0c6e0 2024-09-11T01:09:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.395+0000 7f0546ffd640 10 client.17487 handle_command_reply: tid=0 2024-09-11T01:09:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.395+0000 7f057062e740 1 client.17487 shutdown 2024-09-11T01:09:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.395+0000 7f057062e740 1 -- 172.21.15.78:0/3367842720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 msgr2=0x564a2cefc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.395+0000 7f057062e740 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cefc610 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f055c013e30 tx=0x7f055c00b390 comp rx=0 tx=0).stop 2024-09-11T01:09:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.395+0000 7f057062e740 1 -- 172.21.15.78:0/3367842720 shutdown_connections 2024-09-11T01:09:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.395+0000 7f057062e740 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cefcd30 0x564a2cefa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.395+0000 7f057062e740 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 0x564a2cef9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.395+0000 7f057062e740 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564a2cf0c6e0 0x564a2cf0ead0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.395+0000 7f0546ffd640 0 client.17487 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.395+0000 7f057062e740 1 --2- 172.21.15.78:0/3367842720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cefc610 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.395+0000 7f057062e740 1 -- 172.21.15.78:0/3367842720 >> 172.21.15.78:0/3367842720 conn(0x564a2ceee2b0 msgr2=0x564a2cef3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.396+0000 7f057062e740 1 -- 172.21.15.78:0/3367842720 shutdown_connections 2024-09-11T01:09:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.396+0000 7f057062e740 1 -- 172.21.15.78:0/3367842720 wait complete. 2024-09-11T01:09:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.396+0000 7f057062e740 20 client.17487 trim_cache size 0 max 16384 2024-09-11T01:09:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.419+0000 7f057062e740 1 Processor -- start 2024-09-11T01:09:36.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.419+0000 7f057062e740 1 -- start start 2024-09-11T01:09:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.419+0000 7f057062e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cee92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.419+0000 7f057062e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 0x564a2ceedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.419+0000 7f057062e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cefcd30 0x564a2cefb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.419+0000 7f057062e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05681b0790 con 0x564a2cef0480 2024-09-11T01:09:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.419+0000 7f057062e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f05681b2300 con 0x564a2cef0da0 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.419+0000 7f057062e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05681b0e60 con 0x564a2cefcd30 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.419+0000 7f05677fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 0x564a2ceedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.419+0000 7f05677fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 0x564a2ceedbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:32922/0 (socket says 172.21.15.78:32922) 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.419+0000 7f05677fe640 1 -- 172.21.15.78:0/1667483108 learned_addr learned my addr 172.21.15.78:0/1667483108 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.419+0000 7f0566ffd640 1 --2- 172.21.15.78:0/1667483108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cee92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.420+0000 7f0566ffd640 1 --2- 172.21.15.78:0/1667483108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cefcd30 0x564a2cefb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.420+0000 7f0566ffd640 1 -- 172.21.15.78:0/1667483108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cefcd30 msgr2=0x564a2cefb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.420+0000 7f0566ffd640 1 --2- 172.21.15.78:0/1667483108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cefcd30 0x564a2cefb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.420+0000 7f0566ffd640 1 -- 172.21.15.78:0/1667483108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 msgr2=0x564a2ceedbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.420+0000 7f0566ffd640 1 --2- 172.21.15.78:0/1667483108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 0x564a2ceedbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.420+0000 7f0566ffd640 1 -- 172.21.15.78:0/1667483108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a2ceef670 con 0x564a2cef0480 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.420+0000 7f0566ffd640 1 --2- 172.21.15.78:0/1667483108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cee92d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f055c013430 tx=0x7f055c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45421dfeda7734b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:36.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.420+0000 7f054d7fa640 1 -- 172.21.15.78:0/1667483108 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0130d0 con 0x564a2cef0480 2024-09-11T01:09:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.420+0000 7f054d7fa640 1 -- 172.21.15.78:0/1667483108 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f055c017030 con 0x564a2cef0480 2024-09-11T01:09:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.420+0000 7f054d7fa640 1 -- 172.21.15.78:0/1667483108 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c011440 con 0x564a2cef0480 2024-09-11T01:09:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.421+0000 7f057062e740 1 -- 172.21.15.78:0/1667483108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 msgr2=0x564a2cee92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.421+0000 7f057062e740 1 --2- 172.21.15.78:0/1667483108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cee92d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f055c013430 tx=0x7f055c00b5b0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.421+0000 7f057062e740 1 -- 172.21.15.78:0/1667483108 shutdown_connections 2024-09-11T01:09:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.421+0000 7f057062e740 1 --2- 172.21.15.78:0/1667483108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cefcd30 0x564a2cefb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.421+0000 7f057062e740 1 --2- 172.21.15.78:0/1667483108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cef0da0 0x564a2ceedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.421+0000 7f057062e740 1 --2- 172.21.15.78:0/1667483108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cee92d0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.421+0000 7f057062e740 1 -- 172.21.15.78:0/1667483108 >> 172.21.15.78:0/1667483108 conn(0x564a2ceee2b0 msgr2=0x564a2cefb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.421+0000 7f057062e740 1 -- 172.21.15.78:0/1667483108 shutdown_connections 2024-09-11T01:09:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.421+0000 7f057062e740 1 -- 172.21.15.78:0/1667483108 wait complete. 2024-09-11T01:09:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.421+0000 7f057062e740 1 -- start start 2024-09-11T01:09:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f057062e740 1 Processor -- start 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f057062e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cf04bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f057062e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cef0da0 0x564a2cf05130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f057062e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cefcd30 0x564a2ceffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f057062e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05681b1c00 con 0x564a2cef0480 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f057062e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f05681b1530 con 0x564a2cefcd30 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f057062e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05681ae790 con 0x564a2cef0da0 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f057062e740 10 client.0 _ll_register_callbacks cb 0x7f055c00bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f0566ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cf04bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f0566ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cf04bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60120/0 (socket says 172.21.15.78:60120) 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f0566ffd640 1 -- 172.21.15.78:0/1066528168 learned_addr learned my addr 172.21.15.78:0/1066528168 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f0566ffd640 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cefcd30 0x564a2ceffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.422+0000 7f05677fe640 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cef0da0 0x564a2cf05130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f0566ffd640 1 -- 172.21.15.78:0/1066528168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cef0da0 msgr2=0x564a2cf05130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f0566ffd640 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cef0da0 0x564a2cf05130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f0566ffd640 1 -- 172.21.15.78:0/1066528168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cefcd30 msgr2=0x564a2ceffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f0566ffd640 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cefcd30 0x564a2ceffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f0566ffd640 1 -- 172.21.15.78:0/1066528168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a2cefa400 con 0x564a2cef0480 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f0566ffd640 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cf04bf0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f055c013e00 tx=0x7f055c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b000e30d5478a83 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f0546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f0546ffd640 1 -- 172.21.15.78:0/1066528168 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c005450 con 0x564a2cef0480 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f0546ffd640 1 -- 172.21.15.78:0/1066528168 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f055c011440 con 0x564a2cef0480 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f057062e740 1 -- 172.21.15.78:0/1066528168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f055c017030 con 0x564a2cef0480 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f0546ffd640 1 -- 172.21.15.78:0/1066528168 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f055c011640 con 0x564a2cef0480 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f057062e740 10 client.17493 fetch_fsmap learned FSMap version 47 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f057062e740 1 -- 172.21.15.78:0/1066528168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564a2cf79b10 con 0x564a2cef0480 2024-09-11T01:09:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.423+0000 7f0546ffd640 1 -- 172.21.15.78:0/1066528168 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f055c017030 con 0x564a2cef0480 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.424+0000 7f0546ffd640 1 -- 172.21.15.78:0/1066528168 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f055c004b90 con 0x564a2cef0480 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.424+0000 7f057062e740 10 client.17493 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.424+0000 7f057062e740 10 client.17493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.424+0000 7f057062e740 20 client.17493 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.424+0000 7f057062e740 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564a2cf0c6e0 0x564a2cf7c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.424+0000 7f057062e740 4 client.17493 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.424+0000 7f057062e740 1 -- 172.21.15.78:0/1066528168 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a2ce64850 con 0x564a2cf0c6e0 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.424+0000 7f05677fe640 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564a2cf0c6e0 0x564a2cf7c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.424+0000 7f05677fe640 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564a2cf0c6e0 0x564a2cf7c1c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=385e0829695c559d server_cookie=f39f466eed18f51e in_seq=0 out_seq=0 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.424+0000 7f0546ffd640 10 client.17493 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f0546ffd640 1 -- 172.21.15.78:0/1066528168 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x564a2ce64850 con 0x564a2cf0c6e0 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f0546ffd640 10 client.17493 handle_command_reply: tid=0 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f057062e740 1 client.17493 shutdown 2024-09-11T01:09:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f057062e740 1 -- 172.21.15.78:0/1066528168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 msgr2=0x564a2cf04bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f057062e740 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cf04bf0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f055c013e00 tx=0x7f055c001e50 comp rx=0 tx=0).stop 2024-09-11T01:09:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f057062e740 1 -- 172.21.15.78:0/1066528168 shutdown_connections 2024-09-11T01:09:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f057062e740 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564a2cefcd30 0x564a2ceffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f057062e740 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a2cef0da0 0x564a2cf05130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f057062e740 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x564a2cf0c6e0 0x564a2cf7c1c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f057062e740 1 --2- 172.21.15.78:0/1066528168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2cef0480 0x564a2cf04bf0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f057062e740 1 -- 172.21.15.78:0/1066528168 >> 172.21.15.78:0/1066528168 conn(0x564a2ceee2b0 msgr2=0x564a2cefb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f0546ffd640 0 client.17493 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f057062e740 1 -- 172.21.15.78:0/1066528168 shutdown_connections 2024-09-11T01:09:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f057062e740 1 -- 172.21.15.78:0/1066528168 wait complete. 2024-09-11T01:09:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.425+0000 7f057062e740 20 client.17493 trim_cache size 0 max 16384 2024-09-11T01:09:36.433 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:09:36.433 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:09:36.433 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:09:36.433 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:09:36.433 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:09:36.433 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:09:36.433 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.435 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.436 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.438 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:09:36.439 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:36.441 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:09:36.441 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:09:36.441 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:36.441 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:36.441 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:36.441 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:36.441 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:36.441 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:09:36.441 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:09:36.441 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:09:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.426+0000 7f056e259640 1 -- 172.21.15.78:0/1505890770 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f05500b5450 msgr2=0x7f05500b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.426+0000 7f056e259640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f05500b5450 0x7f05500b7910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f05600e3e40 tx=0x7f0560096040 comp rx=0 tx=0).stop 2024-09-11T01:09:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.426+0000 7f056e259640 1 -- 172.21.15.78:0/1505890770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568109160 msgr2=0x7f056819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.426+0000 7f056e259640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568109160 0x7f056819eb70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f056000eb10 tx=0x7f056000c2d0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.426+0000 7f056e259640 1 -- 172.21.15.78:0/1505890770 shutdown_connections 2024-09-11T01:09:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.426+0000 7f056e259640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f05500b5450 0x7f05500b7910 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.426+0000 7f056e259640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568109160 0x7f056819eb70 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.426+0000 7f056e259640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05681087e0 0x7f05681997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.426+0000 7f056e259640 1 --2- 172.21.15.78:0/1505890770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0568102dd0 0x7f0568199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.426+0000 7f056e259640 1 -- 172.21.15.78:0/1505890770 >> 172.21.15.78:0/1505890770 conn(0x7f05680fe800 msgr2=0x7f0568100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.426+0000 7f056e259640 1 -- 172.21.15.78:0/1505890770 shutdown_connections 2024-09-11T01:09:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:36.426+0000 7f056e259640 1 -- 172.21.15.78:0/1505890770 wait complete. 2024-09-11T01:09:36.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 702958 5.22 MB/sec execute 464 sec latency 31.623 ms 2024-09-11T01:09:36.525 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:36 smithi112 ceph-mon[35384]: pgmap v653: 97 pgs: 97 active+clean; 532 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 210 op/s 2024-09-11T01:09:36.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:36 smithi110 ceph-mon[34712]: pgmap v653: 97 pgs: 97 active+clean; 532 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 210 op/s 2024-09-11T01:09:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:36 smithi078 ceph-mon[31315]: pgmap v653: 97 pgs: 97 active+clean; 532 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 210 op/s 2024-09-11T01:09:37.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 704043 5.22 MB/sec execute 465 sec latency 77.069 ms 2024-09-11T01:09:37.623 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:09:37.623 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:09:37.906 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:09:38.332 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:38 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:09:38.332 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:38 smithi078 ceph-mon[31315]: pgmap v654: 97 pgs: 97 active+clean; 536 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-09-11T01:09:38.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.414+0000 7fbfb0e03640 1 -- 172.21.15.78:0/1732384388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfac109140 msgr2=0x7fbfac10f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:38.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.414+0000 7fbfb0e03640 1 --2- 172.21.15.78:0/1732384388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfac109140 0x7fbfac10f940 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009ca0 tx=0x7fbfa402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:09:38.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.414+0000 7fbfb0e03640 1 -- 172.21.15.78:0/1732384388 shutdown_connections 2024-09-11T01:09:38.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.414+0000 7fbfb0e03640 1 --2- 172.21.15.78:0/1732384388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfac109140 0x7fbfac10f940 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.414+0000 7fbfb0e03640 1 --2- 172.21.15.78:0/1732384388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfac1087c0 0x7fbfac108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.414+0000 7fbfb0e03640 1 --2- 172.21.15.78:0/1732384388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfac102db0 0x7fbfac103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.414+0000 7fbfb0e03640 1 -- 172.21.15.78:0/1732384388 >> 172.21.15.78:0/1732384388 conn(0x7fbfac0fe860 msgr2=0x7fbfac100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.415+0000 7fbfb0e03640 1 -- 172.21.15.78:0/1732384388 shutdown_connections 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.415+0000 7fbfb0e03640 1 -- 172.21.15.78:0/1732384388 wait complete. 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfb0e03640 1 Processor -- start 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfb0e03640 1 -- start start 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfb0e03640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfac102db0 0x7fbfac0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfb0e03640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfac1087c0 0x7fbfac0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfb0e03640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfac109140 0x7fbfac075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfb0e03640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac112dc0 con 0x7fbfac109140 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfb0e03640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac112c40 con 0x7fbfac1087c0 2024-09-11T01:09:38.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfb0e03640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac112f40 con 0x7fbfac102db0 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfaa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfac102db0 0x7fbfac0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfaa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfac102db0 0x7fbfac0754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52962/0 (socket says 172.21.15.78:52962) 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfaa59c640 1 -- 172.21.15.78:0/2371535419 learned_addr learned my addr 172.21.15.78:0/2371535419 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfaad9d640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfac109140 0x7fbfac075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfa9d9b640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfac1087c0 0x7fbfac0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfaa59c640 1 -- 172.21.15.78:0/2371535419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfac1087c0 msgr2=0x7fbfac0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfaa59c640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfac1087c0 0x7fbfac0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfaa59c640 1 -- 172.21.15.78:0/2371535419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfac109140 msgr2=0x7fbfac075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfaa59c640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfac109140 0x7fbfac075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfaa59c640 1 -- 172.21.15.78:0/2371535419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac1a7670 con 0x7fbfac102db0 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.416+0000 7fbfaad9d640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfac109140 0x7fbfac075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.417+0000 7fbfaa59c640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfac102db0 0x7fbfac0754a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00e9f0 tx=0x7fbf9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.417+0000 7fbf9b7fe640 1 -- 172.21.15.78:0/2371535419 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00cd80 con 0x7fbfac102db0 2024-09-11T01:09:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.417+0000 7fbfb0e03640 1 -- 172.21.15.78:0/2371535419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfac1a7960 con 0x7fbfac102db0 2024-09-11T01:09:38.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.417+0000 7fbf9b7fe640 1 -- 172.21.15.78:0/2371535419 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf9c0108e0 con 0x7fbfac102db0 2024-09-11T01:09:38.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.417+0000 7fbf9b7fe640 1 -- 172.21.15.78:0/2371535419 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c018980 con 0x7fbfac102db0 2024-09-11T01:09:38.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.417+0000 7fbfb0e03640 1 -- 172.21.15.78:0/2371535419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfac1a7ea0 con 0x7fbfac102db0 2024-09-11T01:09:38.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.418+0000 7fbfb0e03640 1 -- 172.21.15.78:0/2371535419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfac103190 con 0x7fbfac102db0 2024-09-11T01:09:38.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.419+0000 7fbf9b7fe640 1 -- 172.21.15.78:0/2371535419 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fbf9c010430 con 0x7fbfac102db0 2024-09-11T01:09:38.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.420+0000 7fbf9b7fe640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbf940b5450 0x7fbf940b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:38.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.420+0000 7fbfa9d9b640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbf940b5450 0x7fbf940b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:38.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.420+0000 7fbf9b7fe640 1 -- 172.21.15.78:0/2371535419 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7fbf9c0e6730 con 0x7fbfac102db0 2024-09-11T01:09:38.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.421+0000 7fbfa9d9b640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbf940b5450 0x7fbf940b7910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbfac076f90 tx=0x7fbfa0005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:38.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.423+0000 7fbf9b7fe640 1 -- 172.21.15.78:0/2371535419 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbf9c0acca0 con 0x7fbfac102db0 2024-09-11T01:09:38.450 INFO:tasks.workunit.client.0.smithi078.stdout: 1 706060 5.23 MB/sec execute 466 sec latency 34.411 ms 2024-09-11T01:09:38.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.666+0000 7fbfb0e03640 1 -- 172.21.15.78:0/2371535419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfac0767a0 con 0x7fbfac102db0 2024-09-11T01:09:38.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.667+0000 7fbf9b7fe640 1 -- 172.21.15.78:0/2371535419 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11986 (secure 0 0 0) 0x7fbfac0767a0 con 0x7fbfac102db0 2024-09-11T01:09:38.669 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:09:38.669 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":48,"btime":"2024-09-11T01:09:37:269802+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:09:37.112141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:09:38.669 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 48 2024-09-11T01:09:38.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.669+0000 7fbfb0e03640 1 -- 172.21.15.78:0/2371535419 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbf940b5450 msgr2=0x7fbf940b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:38.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.669+0000 7fbfb0e03640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbf940b5450 0x7fbf940b7910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbfac076f90 tx=0x7fbfa0005f90 comp rx=0 tx=0).stop 2024-09-11T01:09:38.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.669+0000 7fbfb0e03640 1 -- 172.21.15.78:0/2371535419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfac102db0 msgr2=0x7fbfac0754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:38.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.669+0000 7fbfb0e03640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfac102db0 0x7fbfac0754a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00e9f0 tx=0x7fbf9c00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:09:38.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.669+0000 7fbfb0e03640 1 -- 172.21.15.78:0/2371535419 shutdown_connections 2024-09-11T01:09:38.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.669+0000 7fbfb0e03640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbf940b5450 0x7fbf940b7910 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:38.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.669+0000 7fbfb0e03640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfac109140 0x7fbfac075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:38.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.669+0000 7fbfb0e03640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfac1087c0 0x7fbfac0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:38.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.669+0000 7fbfb0e03640 1 --2- 172.21.15.78:0/2371535419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfac102db0 0x7fbfac0754a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:38.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.669+0000 7fbfb0e03640 1 -- 172.21.15.78:0/2371535419 >> 172.21.15.78:0/2371535419 conn(0x7fbfac0fe860 msgr2=0x7fbfac100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:38.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.669+0000 7fbfb0e03640 1 -- 172.21.15.78:0/2371535419 shutdown_connections 2024-09-11T01:09:38.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:38.669+0000 7fbfb0e03640 1 -- 172.21.15.78:0/2371535419 wait complete. 2024-09-11T01:09:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:38 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:09:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:38 smithi110 ceph-mon[34712]: pgmap v654: 97 pgs: 97 active+clean; 536 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-09-11T01:09:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:38 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:09:38.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:38 smithi112 ceph-mon[35384]: pgmap v654: 97 pgs: 97 active+clean; 536 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-09-11T01:09:39.363 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:39 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2371535419' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:09:39.393 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:09:39.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 707541 5.24 MB/sec execute 467 sec latency 10.880 ms 2024-09-11T01:09:39.659 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:09:39.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:39 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2371535419' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:09:39.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:39 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2371535419' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:09:40.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.169+0000 7f83e2673640 1 -- 172.21.15.78:0/76448055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc1055a0 msgr2=0x7f83dc105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:40.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.169+0000 7f83e2673640 1 --2- 172.21.15.78:0/76448055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc1055a0 0x7f83dc105a00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009900 tx=0x7f83d002a410 comp rx=0 tx=0).stop 2024-09-11T01:09:40.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.170+0000 7f83e2673640 1 -- 172.21.15.78:0/76448055 shutdown_connections 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.170+0000 7f83e2673640 1 --2- 172.21.15.78:0/76448055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc105f40 0x7f83dc10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.170+0000 7f83e2673640 1 --2- 172.21.15.78:0/76448055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc1055a0 0x7f83dc105a00 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.170+0000 7f83e2673640 1 --2- 172.21.15.78:0/76448055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83dc101250 0x7f83dc101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.170+0000 7f83e2673640 1 -- 172.21.15.78:0/76448055 >> 172.21.15.78:0/76448055 conn(0x7f83dc078140 msgr2=0x7f83dc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.170+0000 7f83e2673640 1 -- 172.21.15.78:0/76448055 shutdown_connections 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.170+0000 7f83e2673640 1 -- 172.21.15.78:0/76448055 wait complete. 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.170+0000 7f83e2673640 1 Processor -- start 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.170+0000 7f83e2673640 1 -- start start 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83e2673640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc101250 0x7f83dc1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83e2673640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc1055a0 0x7f83dc1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83e2673640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83dc105f40 0x7f83dc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83e2673640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc112db0 con 0x7f83dc1055a0 2024-09-11T01:09:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83e2673640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc112c30 con 0x7f83dc105f40 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83e2673640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc112f30 con 0x7f83dc101250 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc101250 0x7f83dc1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc101250 0x7f83dc1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52980/0 (socket says 172.21.15.78:52980) 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83dbfff640 1 -- 172.21.15.78:0/873144172 learned_addr learned my addr 172.21.15.78:0/873144172 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83e0c10640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83dc105f40 0x7f83dc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83db7fe640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc1055a0 0x7f83dc1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83dbfff640 1 -- 172.21.15.78:0/873144172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83dc105f40 msgr2=0x7f83dc19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83dbfff640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83dc105f40 0x7f83dc19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83dbfff640 1 -- 172.21.15.78:0/873144172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc1055a0 msgr2=0x7f83dc1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83dbfff640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc1055a0 0x7f83dc1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83dbfff640 1 -- 172.21.15.78:0/873144172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc19f350 con 0x7f83dc101250 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83dbfff640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc101250 0x7f83dc1992a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e9f0 tx=0x7f83cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:40.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.171+0000 7f83d97fa640 1 -- 172.21.15.78:0/873144172 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc00cd80 con 0x7f83dc101250 2024-09-11T01:09:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.172+0000 7f83d97fa640 1 -- 172.21.15.78:0/873144172 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83cc0108e0 con 0x7f83dc101250 2024-09-11T01:09:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.172+0000 7f83e2673640 1 -- 172.21.15.78:0/873144172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83dc19f640 con 0x7f83dc101250 2024-09-11T01:09:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.172+0000 7f83d97fa640 1 -- 172.21.15.78:0/873144172 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc018980 con 0x7f83dc101250 2024-09-11T01:09:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.172+0000 7f83e2673640 1 -- 172.21.15.78:0/873144172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83dc102100 con 0x7f83dc101250 2024-09-11T01:09:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83e4a48740 1 Processor -- start 2024-09-11T01:09:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83e4a48740 1 -- start start 2024-09-11T01:09:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83d97fa640 1 -- 172.21.15.78:0/873144172 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f83cc010430 con 0x7f83dc101250 2024-09-11T01:09:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83e4a48740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd250f0 0x55a68bd254d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83e4a48740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd25a10 0x55a68bd31460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83e4a48740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 0x55a68bd33d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83e4a48740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a68bc9bea0 con 0x55a68bd319a0 2024-09-11T01:09:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83e4a48740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a68bc8b2d0 con 0x55a68bd250f0 2024-09-11T01:09:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83e4a48740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a68bc8f0e0 con 0x55a68bd25a10 2024-09-11T01:09:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd25a10 0x55a68bd31460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83e0c10640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd250f0 0x55a68bd254d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 0x55a68bd33d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.174+0000 7f83db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 0x55a68bd33d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60156/0 (socket says 172.21.15.78:60156) 2024-09-11T01:09:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.175+0000 7f83db7fe640 1 -- 172.21.15.78:0/2259572725 learned_addr learned my addr 172.21.15.78:0/2259572725 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.175+0000 7f83db7fe640 1 -- 172.21.15.78:0/2259572725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd25a10 msgr2=0x55a68bd31460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.175+0000 7f83db7fe640 1 --2- 172.21.15.78:0/2259572725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd25a10 0x55a68bd31460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.175+0000 7f83db7fe640 1 -- 172.21.15.78:0/2259572725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd250f0 msgr2=0x55a68bd254d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.175+0000 7f83db7fe640 1 --2- 172.21.15.78:0/2259572725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd250f0 0x55a68bd254d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.175+0000 7f83db7fe640 1 -- 172.21.15.78:0/2259572725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a68b9e9650 con 0x55a68bd319a0 2024-09-11T01:09:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.175+0000 7f83db7fe640 1 --2- 172.21.15.78:0/2259572725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 0x55a68bd33d90 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009900 tx=0x7f83d0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855b85d7e1f44b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.175+0000 7f83c27fc640 1 -- 172.21.15.78:0/2259572725 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0035050 con 0x55a68bd319a0 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.175+0000 7f83c27fc640 1 -- 172.21.15.78:0/2259572725 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83d0032da0 con 0x55a68bd319a0 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.175+0000 7f83c27fc640 1 -- 172.21.15.78:0/2259572725 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d00052f0 con 0x55a68bd319a0 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.175+0000 7f83d97fa640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f83c40b5450 0x7f83c40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.175+0000 7f83e0c10640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f83c40b5450 0x7f83c40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.176+0000 7f83d97fa640 1 -- 172.21.15.78:0/873144172 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f83cc0e62f0 con 0x7f83dc101250 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.176+0000 7f83e4a48740 1 -- 172.21.15.78:0/2259572725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 msgr2=0x55a68bd33d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.176+0000 7f83e0c10640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f83c40b5450 0x7f83c40b7910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f83d4005fd0 tx=0x7f83d4009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.176+0000 7f83e4a48740 1 --2- 172.21.15.78:0/2259572725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 0x55a68bd33d90 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009900 tx=0x7f83d0005e40 comp rx=0 tx=0).stop 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.176+0000 7f83e4a48740 1 -- 172.21.15.78:0/2259572725 shutdown_connections 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.176+0000 7f83e4a48740 1 --2- 172.21.15.78:0/2259572725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 0x55a68bd33d90 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.176+0000 7f83e4a48740 1 --2- 172.21.15.78:0/2259572725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd25a10 0x55a68bd31460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.176+0000 7f83e4a48740 1 --2- 172.21.15.78:0/2259572725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd250f0 0x55a68bd254d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.176+0000 7f83e4a48740 1 -- 172.21.15.78:0/2259572725 >> 172.21.15.78:0/2259572725 conn(0x55a68bd22f20 msgr2=0x55a68bd240c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.176+0000 7f83e4a48740 1 -- 172.21.15.78:0/2259572725 shutdown_connections 2024-09-11T01:09:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.176+0000 7f83e4a48740 1 -- 172.21.15.78:0/2259572725 wait complete. 2024-09-11T01:09:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.177+0000 7f83e4a48740 1 -- start start 2024-09-11T01:09:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.177+0000 7f83e4a48740 1 Processor -- start 2024-09-11T01:09:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.177+0000 7f83e4a48740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd250f0 0x55a68bd312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.177+0000 7f83e4a48740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd25a10 0x55a68bd2e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.177+0000 7f83e4a48740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd319a0 0x55a68bd2ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.177+0000 7f83e4a48740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a68baec700 con 0x55a68bd250f0 2024-09-11T01:09:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.177+0000 7f83e4a48740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a68bc26160 con 0x55a68bd25a10 2024-09-11T01:09:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.177+0000 7f83e4a48740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a68bd345c0 con 0x55a68bd319a0 2024-09-11T01:09:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.177+0000 7f83e4a48740 10 client.0 _ll_register_callbacks cb 0x55a68bc3f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.177+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd250f0 0x55a68bd312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd250f0 0x55a68bd312c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60168/0 (socket says 172.21.15.78:60168) 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83dbfff640 1 -- 172.21.15.78:0/2975389211 learned_addr learned my addr 172.21.15.78:0/2975389211 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83db7fe640 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd25a10 0x55a68bd2e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83e0c10640 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd319a0 0x55a68bd2ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83dbfff640 1 -- 172.21.15.78:0/2975389211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd319a0 msgr2=0x55a68bd2ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83dbfff640 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd319a0 0x55a68bd2ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83dbfff640 1 -- 172.21.15.78:0/2975389211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd25a10 msgr2=0x55a68bd2e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83dbfff640 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd25a10 0x55a68bd2e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83dbfff640 1 -- 172.21.15.78:0/2975389211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a68bc53c90 con 0x55a68bd250f0 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83dbfff640 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd250f0 0x55a68bd312c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00c6a0 tx=0x7f83cc010750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9cfd4058fa8e68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:09:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83bb7fe640 1 -- 172.21.15.78:0/2975389211 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0e5070 con 0x55a68bd250f0 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83bb7fe640 1 -- 172.21.15.78:0/2975389211 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f83cc0abc60 con 0x55a68bd250f0 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83e4a48740 1 -- 172.21.15.78:0/2975389211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d0032da0 con 0x55a68bd250f0 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83bb7fe640 1 -- 172.21.15.78:0/2975389211 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f83cc0ac5d0 con 0x55a68bd250f0 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83e4a48740 10 client.17511 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83e4a48740 1 -- 172.21.15.78:0/2975389211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a68bd3ff20 con 0x55a68bd250f0 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.178+0000 7f83bb7fe640 1 -- 172.21.15.78:0/2975389211 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83d0032da0 con 0x55a68bd250f0 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.179+0000 7f83bb7fe640 1 -- 172.21.15.78:0/2975389211 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f83cc0a8150 con 0x55a68bd250f0 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.179+0000 7f83e4a48740 10 client.17511 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.179+0000 7f83e4a48740 10 client.17511 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.179+0000 7f83e4a48740 20 client.17511 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.179+0000 7f83e4a48740 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a68bd41270 0x55a68bd436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.179+0000 7f83e4a48740 4 client.17511 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.179+0000 7f83e4a48740 1 -- 172.21.15.78:0/2975389211 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a68bc9ddf0 con 0x55a68bd41270 2024-09-11T01:09:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.179+0000 7f83db7fe640 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a68bd41270 0x55a68bd436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.179+0000 7f83bb7fe640 10 client.17511 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.179+0000 7f83db7fe640 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a68bd41270 0x55a68bd436c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71f2a05d1833af02 server_cookie=186ed7024810c8f0 in_seq=0 out_seq=0 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83bb7fe640 1 -- 172.21.15.78:0/2975389211 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a68bc9ddf0 con 0x55a68bd41270 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83bb7fe640 10 client.17511 handle_command_reply: tid=0 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83e4a48740 1 client.17511 shutdown 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83e4a48740 1 -- 172.21.15.78:0/2975389211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd250f0 msgr2=0x55a68bd312c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83e4a48740 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd250f0 0x55a68bd312c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00c6a0 tx=0x7f83cc010750 comp rx=0 tx=0).stop 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83e4a48740 1 -- 172.21.15.78:0/2975389211 shutdown_connections 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83e4a48740 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a68bd41270 0x55a68bd436c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83e4a48740 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd319a0 0x55a68bd2ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83e4a48740 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd25a10 0x55a68bd2e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83e4a48740 1 --2- 172.21.15.78:0/2975389211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd250f0 0x55a68bd312c0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83e4a48740 1 -- 172.21.15.78:0/2975389211 >> 172.21.15.78:0/2975389211 conn(0x55a68bd22f20 msgr2=0x55a68bd28a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83bb7fe640 0 client.17511 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:09:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83e4a48740 1 -- 172.21.15.78:0/2975389211 shutdown_connections 2024-09-11T01:09:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83e4a48740 1 -- 172.21.15.78:0/2975389211 wait complete. 2024-09-11T01:09:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.180+0000 7f83e4a48740 20 client.17511 trim_cache size 0 max 16384 2024-09-11T01:09:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.194+0000 7f83e4a48740 1 Processor -- start 2024-09-11T01:09:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.194+0000 7f83e4a48740 1 -- start start 2024-09-11T01:09:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.194+0000 7f83e4a48740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd250f0 0x55a68bd3d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e4a48740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd25a10 0x55a68bd43280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e4a48740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 0x55a68bd30450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e4a48740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1ac3d0 con 0x55a68bd319a0 2024-09-11T01:09:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e4a48740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1adfd0 con 0x55a68bd25a10 2024-09-11T01:09:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e4a48740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1acaa0 con 0x55a68bd250f0 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e0c10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 0x55a68bd30450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e0c10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 0x55a68bd30450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60174/0 (socket says 172.21.15.78:60174) 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83db7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd25a10 0x55a68bd43280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e0c10640 1 -- 172.21.15.78:0/1883831671 learned_addr learned my addr 172.21.15.78:0/1883831671 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83dbfff640 1 --2- 172.21.15.78:0/1883831671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd250f0 0x55a68bd3d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e0c10640 1 -- 172.21.15.78:0/1883831671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd250f0 msgr2=0x55a68bd3d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e0c10640 1 --2- 172.21.15.78:0/1883831671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd250f0 0x55a68bd3d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e0c10640 1 -- 172.21.15.78:0/1883831671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd25a10 msgr2=0x55a68bd43280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e0c10640 1 --2- 172.21.15.78:0/1883831671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd25a10 0x55a68bd43280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e0c10640 1 -- 172.21.15.78:0/1883831671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a68bd29410 con 0x55a68bd319a0 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83db7fe640 1 --2- 172.21.15.78:0/1883831671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd25a10 0x55a68bd43280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83e0c10640 1 --2- 172.21.15.78:0/1883831671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 0x55a68bd30450 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f83d4003d20 tx=0x7f83d401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef293ad73b1d03b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83c1ffb640 1 -- 172.21.15.78:0/1883831671 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d400f4e0 con 0x55a68bd319a0 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83c1ffb640 1 -- 172.21.15.78:0/1883831671 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83d400fb30 con 0x55a68bd319a0 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.195+0000 7f83c1ffb640 1 -- 172.21.15.78:0/1883831671 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4029ba0 con 0x55a68bd319a0 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.196+0000 7f83e4a48740 1 -- 172.21.15.78:0/1883831671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 msgr2=0x55a68bd30450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.196+0000 7f83e4a48740 1 --2- 172.21.15.78:0/1883831671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 0x55a68bd30450 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f83d4003d20 tx=0x7f83d401a000 comp rx=0 tx=0).stop 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.196+0000 7f83e4a48740 1 -- 172.21.15.78:0/1883831671 shutdown_connections 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.196+0000 7f83e4a48740 1 --2- 172.21.15.78:0/1883831671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd319a0 0x55a68bd30450 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.196+0000 7f83e4a48740 1 --2- 172.21.15.78:0/1883831671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd25a10 0x55a68bd43280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.196+0000 7f83e4a48740 1 --2- 172.21.15.78:0/1883831671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd250f0 0x55a68bd3d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.196+0000 7f83e4a48740 1 -- 172.21.15.78:0/1883831671 >> 172.21.15.78:0/1883831671 conn(0x55a68bd22f20 msgr2=0x55a68bd2fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.196+0000 7f83e4a48740 1 -- 172.21.15.78:0/1883831671 shutdown_connections 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.196+0000 7f83e4a48740 1 -- 172.21.15.78:0/1883831671 wait complete. 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83e4a48740 1 -- start start 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83e4a48740 1 Processor -- start 2024-09-11T01:09:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83e4a48740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd250f0 0x55a68bd39d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83e4a48740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd25a10 0x55a68bd36220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83e4a48740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd319a0 0x55a68bd36760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83e4a48740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1ad840 con 0x55a68bd25a10 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83e4a48740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1ad170 con 0x55a68bd319a0 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83e4a48740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1aa3d0 con 0x55a68bd250f0 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83e4a48740 10 client.0 _ll_register_callbacks cb 0x7f83cc0e9f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83e0c10640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd319a0 0x55a68bd36760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd250f0 0x55a68bd39d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd250f0 0x55a68bd39d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53020/0 (socket says 172.21.15.78:53020) 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd25a10 0x55a68bd36220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83dbfff640 1 -- 172.21.15.78:0/3136071023 learned_addr learned my addr 172.21.15.78:0/3136071023 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.197+0000 7f83e0c10640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd319a0 0x55a68bd36760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:32984/0 (socket says 172.21.15.78:32984) 2024-09-11T01:09:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.198+0000 7f83dbfff640 1 -- 172.21.15.78:0/3136071023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd319a0 msgr2=0x55a68bd36760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.198+0000 7f83dbfff640 1 --2- 172.21.15.78:0/3136071023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd319a0 0x55a68bd36760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.198+0000 7f83dbfff640 1 -- 172.21.15.78:0/3136071023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd25a10 msgr2=0x55a68bd36220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.198+0000 7f83dbfff640 1 --2- 172.21.15.78:0/3136071023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd25a10 0x55a68bd36220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.198+0000 7f83dbfff640 1 -- 172.21.15.78:0/3136071023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a68bd36cd0 con 0x55a68bd250f0 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.198+0000 7f83dbfff640 1 --2- 172.21.15.78:0/3136071023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd250f0 0x55a68bd39d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f83cc000c00 tx=0x7f83cc0ab840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5855025188c3c143 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.198+0000 7f83bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.198+0000 7f83bb7fe640 1 -- 172.21.15.78:0/3136071023 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0e5070 con 0x55a68bd250f0 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.198+0000 7f83bb7fe640 1 -- 172.21.15.78:0/3136071023 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f83cc018b20 con 0x55a68bd250f0 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.198+0000 7f83e4a48740 1 -- 172.21.15.78:0/3136071023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d400fb30 con 0x55a68bd250f0 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.198+0000 7f83bb7fe640 1 -- 172.21.15.78:0/3136071023 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f83cc0e3eb0 con 0x55a68bd250f0 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.198+0000 7f83e4a48740 10 client.26647 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.199+0000 7f83e4a48740 1 -- 172.21.15.78:0/3136071023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a68bdaef40 con 0x55a68bd250f0 2024-09-11T01:09:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.199+0000 7f83bb7fe640 1 -- 172.21.15.78:0/3136071023 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83d400fb30 con 0x55a68bd250f0 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.199+0000 7f83bb7fe640 1 -- 172.21.15.78:0/3136071023 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f83cc0b23c0 con 0x55a68bd250f0 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.199+0000 7f83e4a48740 10 client.26647 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.199+0000 7f83e4a48740 10 client.26647 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.199+0000 7f83e4a48740 20 client.26647 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.199+0000 7f83e4a48740 1 --2- 172.21.15.78:0/3136071023 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a68bdaf6b0 0x55a68bdb1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.199+0000 7f83e4a48740 4 client.26647 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.199+0000 7f83e4a48740 1 -- 172.21.15.78:0/3136071023 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a68bd43a50 con 0x55a68bdaf6b0 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.199+0000 7f83db7fe640 1 --2- 172.21.15.78:0/3136071023 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a68bdaf6b0 0x55a68bdb1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83db7fe640 1 --2- 172.21.15.78:0/3136071023 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a68bdaf6b0 0x55a68bdb1ad0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=faff81e6cd58955f server_cookie=1405764295188ce9 in_seq=0 out_seq=0 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83bb7fe640 10 client.26647 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83bb7fe640 1 -- 172.21.15.78:0/3136071023 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a68bd43a50 con 0x55a68bdaf6b0 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83bb7fe640 10 client.26647 handle_command_reply: tid=0 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83e4a48740 1 client.26647 shutdown 2024-09-11T01:09:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83e4a48740 1 -- 172.21.15.78:0/3136071023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd250f0 msgr2=0x55a68bd39d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83e4a48740 1 --2- 172.21.15.78:0/3136071023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd250f0 0x55a68bd39d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f83cc000c00 tx=0x7f83cc0ab840 comp rx=0 tx=0).stop 2024-09-11T01:09:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83e4a48740 1 -- 172.21.15.78:0/3136071023 shutdown_connections 2024-09-11T01:09:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83e4a48740 1 --2- 172.21.15.78:0/3136071023 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55a68bdaf6b0 0x55a68bdb1ad0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83e4a48740 1 --2- 172.21.15.78:0/3136071023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a68bd319a0 0x55a68bd36760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83e4a48740 1 --2- 172.21.15.78:0/3136071023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a68bd25a10 0x55a68bd36220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83bb7fe640 0 client.26647 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:09:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83e4a48740 1 --2- 172.21.15.78:0/3136071023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a68bd250f0 0x55a68bd39d40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83e4a48740 1 -- 172.21.15.78:0/3136071023 >> 172.21.15.78:0/3136071023 conn(0x55a68bd22f20 msgr2=0x55a68bd21f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83e4a48740 1 -- 172.21.15.78:0/3136071023 shutdown_connections 2024-09-11T01:09:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83e4a48740 1 -- 172.21.15.78:0/3136071023 wait complete. 2024-09-11T01:09:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.200+0000 7f83e4a48740 20 client.26647 trim_cache size 0 max 16384 2024-09-11T01:09:40.215 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:09:40.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.201+0000 7f83e2673640 1 -- 172.21.15.78:0/873144172 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f83c40b5450 msgr2=0x7f83c40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:40.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.201+0000 7f83e2673640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f83c40b5450 0x7f83c40b7910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f83d4005fd0 tx=0x7f83d4009500 comp rx=0 tx=0).stop 2024-09-11T01:09:40.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.201+0000 7f83e2673640 1 -- 172.21.15.78:0/873144172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc101250 msgr2=0x7f83dc1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:40.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.201+0000 7f83e2673640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc101250 0x7f83dc1992a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e9f0 tx=0x7f83cc00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.201+0000 7f83e2673640 1 -- 172.21.15.78:0/873144172 shutdown_connections 2024-09-11T01:09:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.201+0000 7f83e2673640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f83c40b5450 0x7f83c40b7910 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.201+0000 7f83e2673640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83dc105f40 0x7f83dc19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.201+0000 7f83e2673640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc1055a0 0x7f83dc1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.201+0000 7f83e2673640 1 --2- 172.21.15.78:0/873144172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83dc101250 0x7f83dc1992a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.201+0000 7f83e2673640 1 -- 172.21.15.78:0/873144172 >> 172.21.15.78:0/873144172 conn(0x7f83dc078140 msgr2=0x7f83dc108fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.201+0000 7f83e2673640 1 -- 172.21.15.78:0/873144172 shutdown_connections 2024-09-11T01:09:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:40.201+0000 7f83e2673640 1 -- 172.21.15.78:0/873144172 wait complete. 2024-09-11T01:09:40.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 709587 5.25 MB/sec execute 468 sec latency 31.248 ms 2024-09-11T01:09:40.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:40 smithi078 ceph-mon[31315]: pgmap v655: 97 pgs: 97 active+clean; 541 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 225 op/s 2024-09-11T01:09:40.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:40 smithi110 ceph-mon[34712]: pgmap v655: 97 pgs: 97 active+clean; 541 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 225 op/s 2024-09-11T01:09:40.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:40 smithi112 ceph-mon[35384]: pgmap v655: 97 pgs: 97 active+clean; 541 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 225 op/s 2024-09-11T01:09:40.896 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:09:41.166 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:09:41.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 711013 5.25 MB/sec execute 469 sec latency 28.962 ms 2024-09-11T01:09:41.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.803+0000 7f490cd5e640 1 -- 172.21.15.78:0/1332096291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4908108020 msgr2=0x7f490810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.803+0000 7f490cd5e640 1 --2- 172.21.15.78:0/1332096291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4908108020 0x7f490810e820 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f4900009900 tx=0x7f490002a410 comp rx=0 tx=0).stop 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.803+0000 7f490cd5e640 1 -- 172.21.15.78:0/1332096291 shutdown_connections 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.803+0000 7f490cd5e640 1 --2- 172.21.15.78:0/1332096291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4908108020 0x7f490810e820 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.803+0000 7f490cd5e640 1 --2- 172.21.15.78:0/1332096291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49081076a0 0x7f4908107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.803+0000 7f490cd5e640 1 --2- 172.21.15.78:0/1332096291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4908101c90 0x7f4908102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.803+0000 7f490cd5e640 1 -- 172.21.15.78:0/1332096291 >> 172.21.15.78:0/1332096291 conn(0x7f49080fd700 msgr2=0x7f49080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.803+0000 7f490cd5e640 1 -- 172.21.15.78:0/1332096291 shutdown_connections 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.803+0000 7f490cd5e640 1 -- 172.21.15.78:0/1332096291 wait complete. 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f490cd5e640 1 Processor -- start 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f490cd5e640 1 -- start start 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f490cd5e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4908101c90 0x7f49080fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f490cd5e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49081076a0 0x7f49080fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f490cd5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4908108020 0x7f49080fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f490cd5e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4908111ca0 con 0x7f4908101c90 2024-09-11T01:09:41.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f490cd5e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4908111b20 con 0x7f49081076a0 2024-09-11T01:09:41.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f490cd5e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4908111e20 con 0x7f4908108020 2024-09-11T01:09:41.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f490659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4908101c90 0x7f49080fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f490659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4908101c90 0x7f49080fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60210/0 (socket says 172.21.15.78:60210) 2024-09-11T01:09:41.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f490659c640 1 -- 172.21.15.78:0/3435566930 learned_addr learned my addr 172.21.15.78:0/3435566930 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:41.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f4905d9b640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49081076a0 0x7f49080fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.804+0000 7f4906d9d640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4908108020 0x7f49080fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.805+0000 7f490659c640 1 -- 172.21.15.78:0/3435566930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4908108020 msgr2=0x7f49080fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:41.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.805+0000 7f490659c640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4908108020 0x7f49080fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.805+0000 7f490659c640 1 -- 172.21.15.78:0/3435566930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49081076a0 msgr2=0x7f49080fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:41.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.805+0000 7f490659c640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49081076a0 0x7f49080fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.805+0000 7f490659c640 1 -- 172.21.15.78:0/3435566930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49081a7670 con 0x7f4908101c90 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.805+0000 7f4905d9b640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49081076a0 0x7f49080fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.805+0000 7f490659c640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4908101c90 0x7f49080fdc00 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f48f800ebc0 tx=0x7f48f800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.805+0000 7f48ff7fe640 1 -- 172.21.15.78:0/3435566930 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8010630 con 0x7f4908101c90 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.805+0000 7f490cd5e640 1 -- 172.21.15.78:0/3435566930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49081a7960 con 0x7f4908101c90 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.805+0000 7f48ff7fe640 1 -- 172.21.15.78:0/3435566930 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48f8010c80 con 0x7f4908101c90 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.805+0000 7f48ff7fe640 1 -- 172.21.15.78:0/3435566930 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8018b60 con 0x7f4908101c90 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.805+0000 7f490cd5e640 1 -- 172.21.15.78:0/3435566930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49081a7ef0 con 0x7f4908101c90 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.808+0000 7f490f133740 1 Processor -- start 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.808+0000 7f490f133740 1 -- start start 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.808+0000 7f490f133740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 0x5600459838b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.808+0000 7f490f133740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560045983df0 0x56004598f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.808+0000 7f490f133740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56004598fd80 0x560045992170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.808+0000 7f490f133740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5600458f9ea0 con 0x5600459834d0 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.808+0000 7f490f133740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5600458e92d0 con 0x560045983df0 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.808+0000 7f490f133740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5600458ed0e0 con 0x56004598fd80 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.809+0000 7f4906d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 0x5600459838b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.809+0000 7f48ff7fe640 1 -- 172.21.15.78:0/3435566930 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f48f80107d0 con 0x7f4908101c90 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.809+0000 7f4906d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 0x5600459838b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60216/0 (socket says 172.21.15.78:60216) 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.809+0000 7f4906d9d640 1 -- 172.21.15.78:0/1304089976 learned_addr learned my addr 172.21.15.78:0/1304089976 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.809+0000 7f4905d9b640 1 --2- 172.21.15.78:0/1304089976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56004598fd80 0x560045992170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.809+0000 7f4906d9d640 1 -- 172.21.15.78:0/1304089976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56004598fd80 msgr2=0x560045992170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.809+0000 7f4906d9d640 1 --2- 172.21.15.78:0/1304089976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56004598fd80 0x560045992170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.809+0000 7f4906d9d640 1 -- 172.21.15.78:0/1304089976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560045983df0 msgr2=0x56004598f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.809+0000 7f4906d9d640 1 --2- 172.21.15.78:0/1304089976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560045983df0 0x56004598f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.809+0000 7f4906d9d640 1 -- 172.21.15.78:0/1304089976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560045647650 con 0x5600459834d0 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.809+0000 7f4906d9d640 1 --2- 172.21.15.78:0/1304089976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 0x5600459838b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f490002a930 tx=0x7f4900032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85696c3482da7911 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.809+0000 7f48fcff9640 1 -- 172.21.15.78:0/1304089976 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490003f070 con 0x5600459834d0 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.810+0000 7f48fcff9640 1 -- 172.21.15.78:0/1304089976 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4900004030 con 0x5600459834d0 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.810+0000 7f48fcff9640 1 -- 172.21.15.78:0/1304089976 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900035070 con 0x5600459834d0 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.810+0000 7f48ff7fe640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f48f40b5450 0x7f48f40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.810+0000 7f490659c640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f48f40b5450 0x7f48f40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.810+0000 7f48ff7fe640 1 -- 172.21.15.78:0/3435566930 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f48f80e6730 con 0x7f4908101c90 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.810+0000 7f490f133740 1 -- 172.21.15.78:0/1304089976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 msgr2=0x5600459838b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.810+0000 7f490f133740 1 --2- 172.21.15.78:0/1304089976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 0x5600459838b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f490002a930 tx=0x7f4900032b20 comp rx=0 tx=0).stop 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.810+0000 7f490659c640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f48f40b5450 0x7f48f40b7910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x5600459821e0 tx=0x7f48f8000f40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 -- 172.21.15.78:0/1304089976 shutdown_connections 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 --2- 172.21.15.78:0/1304089976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56004598fd80 0x560045992170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 --2- 172.21.15.78:0/1304089976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560045983df0 0x56004598f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 --2- 172.21.15.78:0/1304089976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 0x5600459838b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 -- 172.21.15.78:0/1304089976 >> 172.21.15.78:0/1304089976 conn(0x560045981300 msgr2=0x5600459824a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 -- 172.21.15.78:0/1304089976 shutdown_connections 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 -- 172.21.15.78:0/1304089976 wait complete. 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 -- start start 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 Processor -- start 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 0x56004598f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 0x56004598caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56004598fd80 0x56004598d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56004574a700 con 0x5600459834d0 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560045884160 con 0x56004598fd80 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5600459929a0 con 0x560045983df0 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.811+0000 7f490f133740 10 client.0 _ll_register_callbacks cb 0x56004589d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4906d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 0x56004598f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4905d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 0x56004598caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4905d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 0x56004598caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53054/0 (socket says 172.21.15.78:53054) 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4906d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 0x56004598f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60230/0 (socket says 172.21.15.78:60230) 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4905d9b640 1 -- 172.21.15.78:0/1570213906 learned_addr learned my addr 172.21.15.78:0/1570213906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4906d9d640 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56004598fd80 0x56004598d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4905d9b640 1 -- 172.21.15.78:0/1570213906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56004598fd80 msgr2=0x56004598d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4905d9b640 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56004598fd80 0x56004598d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4905d9b640 1 -- 172.21.15.78:0/1570213906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 msgr2=0x56004598f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4905d9b640 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 0x56004598f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4905d9b640 1 -- 172.21.15.78:0/1570213906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600458b1c90 con 0x560045983df0 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4906d9d640 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56004598fd80 0x56004598d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f4905d9b640 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 0x56004598caa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f48f0013710 tx=0x7f48f0013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8471d95d8b2600ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f48edffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:09:41.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f48edffb640 1 -- 172.21.15.78:0/1570213906 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0020060 con 0x560045983df0 2024-09-11T01:09:41.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f48edffb640 1 -- 172.21.15.78:0/1570213906 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f48f0011e30 con 0x560045983df0 2024-09-11T01:09:41.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f490f133740 1 -- 172.21.15.78:0/1570213906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4900004030 con 0x560045983df0 2024-09-11T01:09:41.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f48edffb640 1 -- 172.21.15.78:0/1570213906 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f48f0025470 con 0x560045983df0 2024-09-11T01:09:41.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f490f133740 10 client.26659 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:41.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.812+0000 7f490f133740 1 -- 172.21.15.78:0/1570213906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56004599e340 con 0x560045983df0 2024-09-11T01:09:41.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.813+0000 7f48edffb640 1 -- 172.21.15.78:0/1570213906 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4900004030 con 0x560045983df0 2024-09-11T01:09:41.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.813+0000 7f48edffb640 1 -- 172.21.15.78:0/1570213906 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f48f001a030 con 0x560045983df0 2024-09-11T01:09:41.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.813+0000 7f490f133740 10 client.26659 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:41.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.813+0000 7f490f133740 10 client.26659 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:09:41.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.813+0000 7f490f133740 20 client.26659 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:41.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.813+0000 7f490f133740 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56004599f800 0x5600459a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.813+0000 7f490f133740 4 client.26659 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.813+0000 7f490f133740 1 -- 172.21.15.78:0/1570213906 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600458fbdf0 con 0x56004599f800 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.813+0000 7f4906d9d640 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56004599f800 0x5600459a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f48edffb640 10 client.26659 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.813+0000 7f4906d9d640 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56004599f800 0x5600459a1c50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=def962fc11c147b2 server_cookie=7ae34a3bd4b1efa4 in_seq=0 out_seq=0 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f48edffb640 1 -- 172.21.15.78:0/1570213906 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5600458fbdf0 con 0x56004599f800 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f48edffb640 10 client.26659 handle_command_reply: tid=0 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f490f133740 1 client.26659 shutdown 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f490f133740 1 -- 172.21.15.78:0/1570213906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 msgr2=0x56004598caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f490f133740 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 0x56004598caa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f48f0013710 tx=0x7f48f0013bd0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f490f133740 1 -- 172.21.15.78:0/1570213906 shutdown_connections 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f490f133740 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56004599f800 0x5600459a1c50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f490f133740 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56004598fd80 0x56004598d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f490f133740 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 0x56004598caa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f490f133740 1 --2- 172.21.15.78:0/1570213906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600459834d0 0x56004598f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f48edffb640 0 client.26659 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:09:41.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f490f133740 1 -- 172.21.15.78:0/1570213906 >> 172.21.15.78:0/1570213906 conn(0x560045981300 msgr2=0x560045986e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:41.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f490f133740 1 -- 172.21.15.78:0/1570213906 shutdown_connections 2024-09-11T01:09:41.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.814+0000 7f490f133740 1 -- 172.21.15.78:0/1570213906 wait complete. 2024-09-11T01:09:41.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.815+0000 7f490f133740 20 client.26659 trim_cache size 0 max 16384 2024-09-11T01:09:41.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f490f133740 1 Processor -- start 2024-09-11T01:09:41.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f490f133740 1 -- start start 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f490f133740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600459834d0 0x56004599bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f490f133740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560045983df0 0x5600459823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f490f133740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56004598fd80 0x560045982900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f490f133740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f49081b0830 con 0x56004598fd80 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f490f133740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49081b2430 con 0x560045983df0 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f490f133740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49081b0f00 con 0x5600459834d0 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f4905d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560045983df0 0x5600459823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f4905d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560045983df0 0x5600459823c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:33032/0 (socket says 172.21.15.78:33032) 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f4905d9b640 1 -- 172.21.15.78:0/644570433 learned_addr learned my addr 172.21.15.78:0/644570433 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f4906d9d640 1 --2- 172.21.15.78:0/644570433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600459834d0 0x56004599bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f4906d9d640 1 --2- 172.21.15.78:0/644570433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56004598fd80 0x560045982900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f4905d9b640 1 -- 172.21.15.78:0/644570433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600459834d0 msgr2=0x56004599bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:41.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f4905d9b640 1 --2- 172.21.15.78:0/644570433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600459834d0 0x56004599bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f4905d9b640 1 -- 172.21.15.78:0/644570433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56004598fd80 msgr2=0x560045982900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f4905d9b640 1 --2- 172.21.15.78:0/644570433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56004598fd80 0x560045982900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.829+0000 7f4905d9b640 1 -- 172.21.15.78:0/644570433 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560045983c20 con 0x560045983df0 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.830+0000 7f4905d9b640 1 --2- 172.21.15.78:0/644570433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560045983df0 0x5600459823c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f48f000f660 tx=0x7f48f0005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d546d0aee34e863 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.830+0000 7f48effff640 1 -- 172.21.15.78:0/644570433 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f000f850 con 0x560045983df0 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.830+0000 7f48effff640 1 -- 172.21.15.78:0/644570433 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48f000f9f0 con 0x560045983df0 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.830+0000 7f48effff640 1 -- 172.21.15.78:0/644570433 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f00048c0 con 0x560045983df0 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.830+0000 7f490f133740 1 -- 172.21.15.78:0/644570433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560045983df0 msgr2=0x5600459823c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.830+0000 7f490f133740 1 --2- 172.21.15.78:0/644570433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560045983df0 0x5600459823c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f48f000f660 tx=0x7f48f0005af0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 -- 172.21.15.78:0/644570433 shutdown_connections 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 --2- 172.21.15.78:0/644570433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56004598fd80 0x560045982900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 --2- 172.21.15.78:0/644570433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560045983df0 0x5600459823c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 --2- 172.21.15.78:0/644570433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5600459834d0 0x56004599bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 -- 172.21.15.78:0/644570433 >> 172.21.15.78:0/644570433 conn(0x560045981300 msgr2=0x560045987d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 -- 172.21.15.78:0/644570433 shutdown_connections 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 -- 172.21.15.78:0/644570433 wait complete. 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 -- start start 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 Processor -- start 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600459834d0 0x560045997e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 0x560045998390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56004598fd80 0x560045992c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f49081b1ca0 con 0x56004598fd80 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49081b15d0 con 0x5600459834d0 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49081ae830 con 0x560045983df0 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f490f133740 10 client.0 _ll_register_callbacks cb 0x7f48f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f4906d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56004598fd80 0x560045992c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f4905d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 0x560045998390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f4906d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56004598fd80 0x560045992c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60256/0 (socket says 172.21.15.78:60256) 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f4906d9d640 1 -- 172.21.15.78:0/662872382 learned_addr learned my addr 172.21.15.78:0/662872382 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.831+0000 7f4906d9d640 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600459834d0 0x560045997e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f4905d9b640 1 -- 172.21.15.78:0/662872382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600459834d0 msgr2=0x560045997e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f4905d9b640 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600459834d0 0x560045997e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f4905d9b640 1 -- 172.21.15.78:0/662872382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56004598fd80 msgr2=0x560045992c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f4905d9b640 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56004598fd80 0x560045992c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f4905d9b640 1 -- 172.21.15.78:0/662872382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560045993390 con 0x560045983df0 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f4906d9d640 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56004598fd80 0x560045992c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f4905d9b640 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 0x560045998390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48f0005b80 tx=0x7f48f000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa8729fc1591e9f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f48edffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f48edffb640 1 -- 172.21.15.78:0/662872382 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0020040 con 0x560045983df0 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f48edffb640 1 -- 172.21.15.78:0/662872382 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f48f0005040 con 0x560045983df0 2024-09-11T01:09:41.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f490f133740 1 -- 172.21.15.78:0/662872382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48f000f9f0 con 0x560045983df0 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f48edffb640 1 -- 172.21.15.78:0/662872382 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f48f003b050 con 0x560045983df0 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f490f133740 10 client.26665 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.832+0000 7f490f133740 1 -- 172.21.15.78:0/662872382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560045a0d1c0 con 0x560045983df0 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f48edffb640 1 -- 172.21.15.78:0/662872382 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48f000f9f0 con 0x560045983df0 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f48edffb640 1 -- 172.21.15.78:0/662872382 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f48f002d030 con 0x560045983df0 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f490f133740 10 client.26665 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f490f133740 10 client.26665 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f490f133740 20 client.26665 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f490f133740 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x560045a0d840 0x560045a0fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f490f133740 4 client.26665 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f490f133740 1 -- 172.21.15.78:0/662872382 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600458f78a0 con 0x560045a0d840 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f4906d9d640 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x560045a0d840 0x560045a0fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f4906d9d640 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x560045a0d840 0x560045a0fc60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=298ba2187650127f server_cookie=f2ed5bd7ae9f9e1c in_seq=0 out_seq=0 2024-09-11T01:09:41.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f48edffb640 10 client.26665 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f48edffb640 1 -- 172.21.15.78:0/662872382 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600458f78a0 con 0x560045a0d840 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f48edffb640 10 client.26665 handle_command_reply: tid=0 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.833+0000 7f490f133740 1 client.26665 shutdown 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490f133740 1 -- 172.21.15.78:0/662872382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 msgr2=0x560045998390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490f133740 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 0x560045998390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48f0005b80 tx=0x7f48f000f3c0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490f133740 1 -- 172.21.15.78:0/662872382 shutdown_connections 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490f133740 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x560045a0d840 0x560045a0fc60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490f133740 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56004598fd80 0x560045992c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490f133740 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560045983df0 0x560045998390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490f133740 1 --2- 172.21.15.78:0/662872382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600459834d0 0x560045997e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f48edffb640 0 client.26665 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490f133740 1 -- 172.21.15.78:0/662872382 >> 172.21.15.78:0/662872382 conn(0x560045981300 msgr2=0x5600459823a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:41.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490f133740 1 -- 172.21.15.78:0/662872382 shutdown_connections 2024-09-11T01:09:41.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490f133740 1 -- 172.21.15.78:0/662872382 wait complete. 2024-09-11T01:09:41.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490f133740 20 client.26665 trim_cache size 0 max 16384 2024-09-11T01:09:41.846 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:09:41.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490cd5e640 1 -- 172.21.15.78:0/3435566930 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f48f40b5450 msgr2=0x7f48f40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:41.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490cd5e640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f48f40b5450 0x7f48f40b7910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x5600459821e0 tx=0x7f48f8000f40 comp rx=0 tx=0).stop 2024-09-11T01:09:41.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490cd5e640 1 -- 172.21.15.78:0/3435566930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4908101c90 msgr2=0x7f49080fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:41.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490cd5e640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4908101c90 0x7f49080fdc00 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f48f800ebc0 tx=0x7f48f800c6a0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490cd5e640 1 -- 172.21.15.78:0/3435566930 shutdown_connections 2024-09-11T01:09:41.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490cd5e640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f48f40b5450 0x7f48f40b7910 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490cd5e640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4908108020 0x7f49080fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490cd5e640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49081076a0 0x7f49080fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490cd5e640 1 --2- 172.21.15.78:0/3435566930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4908101c90 0x7f49080fdc00 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:41.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490cd5e640 1 -- 172.21.15.78:0/3435566930 >> 172.21.15.78:0/3435566930 conn(0x7f49080fd700 msgr2=0x7f490806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:41.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490cd5e640 1 -- 172.21.15.78:0/3435566930 shutdown_connections 2024-09-11T01:09:41.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:41.834+0000 7f490cd5e640 1 -- 172.21.15.78:0/3435566930 wait complete. 2024-09-11T01:09:42.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 713116 5.27 MB/sec execute 470 sec latency 65.431 ms 2024-09-11T01:09:42.686 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:42 smithi078 ceph-mon[31315]: pgmap v656: 97 pgs: 97 active+clean; 542 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 206 op/s 2024-09-11T01:09:42.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:42 smithi110 ceph-mon[34712]: pgmap v656: 97 pgs: 97 active+clean; 542 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 206 op/s 2024-09-11T01:09:42.716 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:09:42.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:42 smithi112 ceph-mon[35384]: pgmap v656: 97 pgs: 97 active+clean; 542 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 206 op/s 2024-09-11T01:09:42.979 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:09:43.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 714545 5.27 MB/sec execute 471 sec latency 12.502 ms 2024-09-11T01:09:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.496+0000 7fb89eec0640 1 -- 172.21.15.78:0/3998662174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8981076c0 msgr2=0x7fb898107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.496+0000 7fb89eec0640 1 --2- 172.21.15.78:0/3998662174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8981076c0 0x7fb898107b00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb884009900 tx=0x7fb88402a410 comp rx=0 tx=0).stop 2024-09-11T01:09:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.496+0000 7fb89eec0640 1 -- 172.21.15.78:0/3998662174 shutdown_connections 2024-09-11T01:09:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.496+0000 7fb89eec0640 1 --2- 172.21.15.78:0/3998662174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb898108040 0x7fb89810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.496+0000 7fb89eec0640 1 --2- 172.21.15.78:0/3998662174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8981076c0 0x7fb898107b00 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.496+0000 7fb89eec0640 1 --2- 172.21.15.78:0/3998662174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb898101cb0 0x7fb898102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.496+0000 7fb89eec0640 1 -- 172.21.15.78:0/3998662174 >> 172.21.15.78:0/3998662174 conn(0x7fb8980fd700 msgr2=0x7fb8980ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.496+0000 7fb89eec0640 1 -- 172.21.15.78:0/3998662174 shutdown_connections 2024-09-11T01:09:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.496+0000 7fb89eec0640 1 -- 172.21.15.78:0/3998662174 wait complete. 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.496+0000 7fb89eec0640 1 Processor -- start 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.496+0000 7fb89eec0640 1 -- start start 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb89eec0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb898101cb0 0x7fb89810f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb89eec0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8981076c0 0x7fb89810f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb89eec0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb898108040 0x7fb89810fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb89eec0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb898112de0 con 0x7fb8981076c0 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb89eec0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb898112c60 con 0x7fb898101cb0 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb89eec0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb898112f60 con 0x7fb898108040 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb897fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8981076c0 0x7fb89810f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb897fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8981076c0 0x7fb89810f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32846/0 (socket says 172.21.15.78:32846) 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb89cc5c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb898101cb0 0x7fb89810f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb897fff640 1 -- 172.21.15.78:0/145054039 learned_addr learned my addr 172.21.15.78:0/145054039 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb89d45d640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb898108040 0x7fb89810fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb897fff640 1 -- 172.21.15.78:0/145054039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb898108040 msgr2=0x7fb89810fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb897fff640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb898108040 0x7fb89810fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb897fff640 1 -- 172.21.15.78:0/145054039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb898101cb0 msgr2=0x7fb89810f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb897fff640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb898101cb0 0x7fb89810f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb897fff640 1 -- 172.21.15.78:0/145054039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8981a7670 con 0x7fb8981076c0 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb89cc5c640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb898101cb0 0x7fb89810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb897fff640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8981076c0 0x7fb89810f890 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb884005940 tx=0x7fb8840040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb895ffb640 1 -- 172.21.15.78:0/145054039 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88403f030 con 0x7fb8981076c0 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb895ffb640 1 -- 172.21.15.78:0/145054039 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8840044c0 con 0x7fb8981076c0 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb895ffb640 1 -- 172.21.15.78:0/145054039 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884035070 con 0x7fb8981076c0 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.497+0000 7fb89eec0640 1 -- 172.21.15.78:0/145054039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8981a7900 con 0x7fb8981076c0 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.498+0000 7fb89eec0640 1 -- 172.21.15.78:0/145054039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8981a7e30 con 0x7fb8981076c0 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.500+0000 7fb895ffb640 1 -- 172.21.15.78:0/145054039 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb88403ba20 con 0x7fb8981076c0 2024-09-11T01:09:43.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb8a1295740 1 Processor -- start 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb8a1295740 1 -- start start 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb8a1295740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb8a1295740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 0x55c70fd72840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb8a1295740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd75170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb8a1295740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c70fcdcea0 con 0x55c70fd66df0 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb8a1295740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c70fccc2d0 con 0x55c70fd664d0 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb8a1295740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c70fcd00e0 con 0x55c70fd72d80 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb895ffb640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb8880b54d0 0x7fb8880b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb89cc5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 0x55c70fd72840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb89cc5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 0x55c70fd72840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32858/0 (socket says 172.21.15.78:32858) 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb89d45d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb89cc5c640 1 -- 172.21.15.78:0/729860154 learned_addr learned my addr 172.21.15.78:0/729860154 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:43.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb89d45d640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb8880b54d0 0x7fb8880b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb897fff640 1 --2- 172.21.15.78:0/729860154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd75170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb89cc5c640 1 -- 172.21.15.78:0/729860154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 msgr2=0x55c70fd75170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb89cc5c640 1 --2- 172.21.15.78:0/729860154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd75170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb89cc5c640 1 -- 172.21.15.78:0/729860154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 msgr2=0x55c70fd668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb895ffb640 1 -- 172.21.15.78:0/145054039 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7fb884106630 con 0x7fb8981076c0 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb89cc5c640 1 --2- 172.21.15.78:0/729860154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb89cc5c640 1 -- 172.21.15.78:0/729860154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70fa2a650 con 0x55c70fd66df0 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb897fff640 1 --2- 172.21.15.78:0/729860154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd75170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.501+0000 7fb89d45d640 1 --2- 172.21.15.78:0/729860154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.502+0000 7fb89d45d640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb8880b54d0 0x7fb8880b7990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb890005c40 tx=0x7fb890009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.502+0000 7fb89cc5c640 1 --2- 172.21.15.78:0/729860154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 0x55c70fd72840 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0115f0 tx=0x7fb88c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d971c580130f218 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.502+0000 7fb882ffd640 1 -- 172.21.15.78:0/729860154 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c01e070 con 0x55c70fd66df0 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.502+0000 7fb882ffd640 1 -- 172.21.15.78:0/729860154 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb88c00faf0 con 0x55c70fd66df0 2024-09-11T01:09:43.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.502+0000 7fb882ffd640 1 -- 172.21.15.78:0/729860154 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c019ac0 con 0x55c70fd66df0 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 -- 172.21.15.78:0/729860154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 msgr2=0x55c70fd72840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 --2- 172.21.15.78:0/729860154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 0x55c70fd72840 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0115f0 tx=0x7fb88c011ab0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 -- 172.21.15.78:0/729860154 shutdown_connections 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 --2- 172.21.15.78:0/729860154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd75170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 --2- 172.21.15.78:0/729860154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 0x55c70fd72840 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 --2- 172.21.15.78:0/729860154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 -- 172.21.15.78:0/729860154 >> 172.21.15.78:0/729860154 conn(0x55c70fd64300 msgr2=0x55c70fd654a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 -- 172.21.15.78:0/729860154 shutdown_connections 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 -- 172.21.15.78:0/729860154 wait complete. 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 -- start start 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 Processor -- start 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd72610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 0x55c70fd6fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd70000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c70fb2d700 con 0x55c70fd66df0 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c70fc67160 con 0x55c70fd664d0 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.503+0000 7fb8a1295740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c70fd75a00 con 0x55c70fd72d80 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb8a1295740 10 client.0 _ll_register_callbacks cb 0x55c70fc809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb897fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 0x55c70fd6fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb89cc5c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd72610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb897fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 0x55c70fd6fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32868/0 (socket says 172.21.15.78:32868) 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb897fff640 1 -- 172.21.15.78:0/2667200600 learned_addr learned my addr 172.21.15.78:0/2667200600 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb89cc5c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd72610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59710/0 (socket says 172.21.15.78:59710) 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb89d45d640 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd70000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb897fff640 1 -- 172.21.15.78:0/2667200600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 msgr2=0x55c70fd70000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb897fff640 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd70000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb897fff640 1 -- 172.21.15.78:0/2667200600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 msgr2=0x55c70fd72610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb897fff640 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd72610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb897fff640 1 -- 172.21.15.78:0/2667200600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70fc94c90 con 0x55c70fd66df0 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb89cc5c640 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd72610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb897fff640 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 0x55c70fd6fa60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb8840cca20 tx=0x7fb88404ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f453424bbb01dab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb877fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb877fff640 1 -- 172.21.15.78:0/2667200600 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8840c90e0 con 0x55c70fd66df0 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb877fff640 1 -- 172.21.15.78:0/2667200600 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7fb8840c8d00 con 0x55c70fd66df0 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb8a1295740 1 -- 172.21.15.78:0/2667200600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb88c00faf0 con 0x55c70fd66df0 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb877fff640 1 -- 172.21.15.78:0/2667200600 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb8840cc5a0 con 0x55c70fd66df0 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb8a1295740 10 client.17547 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.504+0000 7fb8a1295740 1 -- 172.21.15.78:0/2667200600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c70fd81270 con 0x55c70fd66df0 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.505+0000 7fb877fff640 1 -- 172.21.15.78:0/2667200600 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb88c00faf0 con 0x55c70fd66df0 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.505+0000 7fb877fff640 1 -- 172.21.15.78:0/2667200600 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fb8840d1180 con 0x55c70fd66df0 2024-09-11T01:09:43.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.505+0000 7fb8a1295740 10 client.17547 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.505+0000 7fb8a1295740 10 client.17547 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.505+0000 7fb8a1295740 20 client.17547 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.505+0000 7fb8a1295740 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55c70fd82680 0x55c70fd84b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.505+0000 7fb8a1295740 4 client.17547 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.505+0000 7fb8a1295740 1 -- 172.21.15.78:0/2667200600 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c70fcdedf0 con 0x55c70fd82680 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.505+0000 7fb89cc5c640 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55c70fd82680 0x55c70fd84b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.505+0000 7fb877fff640 10 client.17547 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.505+0000 7fb89cc5c640 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55c70fd82680 0x55c70fd84b40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4512448a5085c9b server_cookie=aefdf6228a5f8e29 in_seq=0 out_seq=0 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.506+0000 7fb877fff640 1 -- 172.21.15.78:0/2667200600 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c70fcdedf0 con 0x55c70fd82680 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.506+0000 7fb877fff640 10 client.17547 handle_command_reply: tid=0 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.506+0000 7fb8a1295740 1 client.17547 shutdown 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.506+0000 7fb8a1295740 1 -- 172.21.15.78:0/2667200600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 msgr2=0x55c70fd6fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:43.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.506+0000 7fb8a1295740 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 0x55c70fd6fa60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb8840cca20 tx=0x7fb88404ace0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.506+0000 7fb8a1295740 1 -- 172.21.15.78:0/2667200600 shutdown_connections 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.506+0000 7fb8a1295740 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd70000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.506+0000 7fb8a1295740 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd66df0 0x55c70fd6fa60 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.506+0000 7fb8a1295740 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55c70fd82680 0x55c70fd84b40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.506+0000 7fb8a1295740 1 --2- 172.21.15.78:0/2667200600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd72610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.506+0000 7fb8a1295740 1 -- 172.21.15.78:0/2667200600 >> 172.21.15.78:0/2667200600 conn(0x55c70fd64300 msgr2=0x55c70fd69ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.506+0000 7fb877fff640 0 client.17547 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.507+0000 7fb8a1295740 1 -- 172.21.15.78:0/2667200600 shutdown_connections 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.507+0000 7fb8a1295740 1 -- 172.21.15.78:0/2667200600 wait complete. 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.507+0000 7fb8a1295740 20 client.17547 trim_cache size 0 max 16384 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb8a1295740 1 Processor -- start 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb8a1295740 1 -- start start 2024-09-11T01:09:43.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb8a1295740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd63c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb8a1295740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd66df0 0x55c70fd6cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb8a1295740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd72d80 0x55c70fd6d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb8a1295740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b0770 con 0x55c70fd72d80 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb8a1295740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b2370 con 0x55c70fd664d0 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb8a1295740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b0e40 con 0x55c70fd66df0 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb89d45d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd72d80 0x55c70fd6d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb897fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd66df0 0x55c70fd6cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb89d45d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd72d80 0x55c70fd6d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32872/0 (socket says 172.21.15.78:32872) 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb89d45d640 1 -- 172.21.15.78:0/832592528 learned_addr learned my addr 172.21.15.78:0/832592528 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.520+0000 7fb89cc5c640 1 --2- 172.21.15.78:0/832592528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd63c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.521+0000 7fb89d45d640 1 -- 172.21.15.78:0/832592528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd66df0 msgr2=0x55c70fd6cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.521+0000 7fb89d45d640 1 --2- 172.21.15.78:0/832592528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd66df0 0x55c70fd6cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.521+0000 7fb89d45d640 1 -- 172.21.15.78:0/832592528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 msgr2=0x55c70fd63c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.521+0000 7fb89d45d640 1 --2- 172.21.15.78:0/832592528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd63c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.521+0000 7fb89d45d640 1 -- 172.21.15.78:0/832592528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70fd64d80 con 0x55c70fd72d80 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.521+0000 7fb89d45d640 1 --2- 172.21.15.78:0/832592528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd72d80 0x55c70fd6d1b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb890013000 tx=0x7fb890009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d20dff4de3da71 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.521+0000 7fb8827fc640 1 -- 172.21.15.78:0/832592528 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890004b20 con 0x55c70fd72d80 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.521+0000 7fb8827fc640 1 -- 172.21.15.78:0/832592528 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb890004cc0 con 0x55c70fd72d80 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.521+0000 7fb8827fc640 1 -- 172.21.15.78:0/832592528 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890005560 con 0x55c70fd72d80 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.522+0000 7fb8a1295740 1 -- 172.21.15.78:0/832592528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd72d80 msgr2=0x55c70fd6d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.522+0000 7fb8a1295740 1 --2- 172.21.15.78:0/832592528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd72d80 0x55c70fd6d1b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb890013000 tx=0x7fb890009a00 comp rx=0 tx=0).stop 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.522+0000 7fb8a1295740 1 -- 172.21.15.78:0/832592528 shutdown_connections 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.522+0000 7fb8a1295740 1 --2- 172.21.15.78:0/832592528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd72d80 0x55c70fd6d1b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.522+0000 7fb8a1295740 1 --2- 172.21.15.78:0/832592528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd66df0 0x55c70fd6cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.522+0000 7fb8a1295740 1 --2- 172.21.15.78:0/832592528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd664d0 0x55c70fd63c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.522+0000 7fb8a1295740 1 -- 172.21.15.78:0/832592528 >> 172.21.15.78:0/832592528 conn(0x55c70fd64300 msgr2=0x55c70fd6acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:43.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.522+0000 7fb8a1295740 1 -- 172.21.15.78:0/832592528 shutdown_connections 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.522+0000 7fb8a1295740 1 -- 172.21.15.78:0/832592528 wait complete. 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb8a1295740 1 -- start start 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb8a1295740 1 Processor -- start 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb8a1295740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd664d0 0x55c70fd76c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb8a1295740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd66df0 0x55c70fd771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb8a1295740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd776e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb8a1295740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b1be0 con 0x55c70fd664d0 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb8a1295740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b1510 con 0x55c70fd66df0 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb8a1295740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981ae770 con 0x55c70fd72d80 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb8a1295740 10 client.0 _ll_register_callbacks cb 0x7fb88402bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb89cc5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd664d0 0x55c70fd76c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb897fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd66df0 0x55c70fd771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb89cc5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd664d0 0x55c70fd76c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32876/0 (socket says 172.21.15.78:32876) 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb89cc5c640 1 -- 172.21.15.78:0/4005930127 learned_addr learned my addr 172.21.15.78:0/4005930127 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.523+0000 7fb89d45d640 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd776e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb89cc5c640 1 -- 172.21.15.78:0/4005930127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 msgr2=0x55c70fd776e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb89cc5c640 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd776e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb89cc5c640 1 -- 172.21.15.78:0/4005930127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd66df0 msgr2=0x55c70fd771a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb89cc5c640 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd66df0 0x55c70fd771a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb89cc5c640 1 -- 172.21.15.78:0/4005930127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70fd84eb0 con 0x55c70fd664d0 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb897fff640 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd66df0 0x55c70fd771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb89cc5c640 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd664d0 0x55c70fd76c60 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb88c019670 tx=0x7fb88c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438e09f9362a98de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb877fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb877fff640 1 -- 172.21.15.78:0/4005930127 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c032060 con 0x55c70fd664d0 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb8a1295740 1 -- 172.21.15.78:0/4005930127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb890004cc0 con 0x55c70fd664d0 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb877fff640 1 -- 172.21.15.78:0/4005930127 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7fb88c00f6c0 con 0x55c70fd664d0 2024-09-11T01:09:43.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb877fff640 1 -- 172.21.15.78:0/4005930127 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb88c01e040 con 0x55c70fd664d0 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb8a1295740 10 client.17553 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb8a1295740 1 -- 172.21.15.78:0/4005930127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c70fdef770 con 0x55c70fd664d0 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb877fff640 1 -- 172.21.15.78:0/4005930127 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb890004cc0 con 0x55c70fd664d0 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.524+0000 7fb877fff640 1 -- 172.21.15.78:0/4005930127 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fb88c019910 con 0x55c70fd664d0 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 10 client.17553 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 10 client.17553 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 20 client.17553 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55c70fdefd60 0x55c70fdf2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 4 client.17553 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 1 -- 172.21.15.78:0/4005930127 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c70fcda8a0 con 0x55c70fdefd60 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb897fff640 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55c70fdefd60 0x55c70fdf2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb877fff640 10 client.17553 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:09:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb897fff640 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55c70fdefd60 0x55c70fdf2180 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f54bf2a345219b83 server_cookie=206698a35dc42ca6 in_seq=0 out_seq=0 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb877fff640 1 -- 172.21.15.78:0/4005930127 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c70fcda8a0 con 0x55c70fdefd60 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb877fff640 10 client.17553 handle_command_reply: tid=0 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 1 client.17553 shutdown 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 1 -- 172.21.15.78:0/4005930127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd664d0 msgr2=0x55c70fd76c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd664d0 0x55c70fd76c60 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb88c019670 tx=0x7fb88c0096c0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 1 -- 172.21.15.78:0/4005930127 shutdown_connections 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c70fd72d80 0x55c70fd776e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55c70fdefd60 0x55c70fdf2180 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c70fd66df0 0x55c70fd771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 1 --2- 172.21.15.78:0/4005930127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c70fd664d0 0x55c70fd76c60 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb877fff640 0 client.17553 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.525+0000 7fb8a1295740 1 -- 172.21.15.78:0/4005930127 >> 172.21.15.78:0/4005930127 conn(0x55c70fd64300 msgr2=0x55c70fd71090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb8a1295740 1 -- 172.21.15.78:0/4005930127 shutdown_connections 2024-09-11T01:09:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb8a1295740 1 -- 172.21.15.78:0/4005930127 wait complete. 2024-09-11T01:09:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb8a1295740 20 client.17553 trim_cache size 0 max 16384 2024-09-11T01:09:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb89eec0640 1 -- 172.21.15.78:0/145054039 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb8880b54d0 msgr2=0x7fb8880b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb89eec0640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb8880b54d0 0x7fb8880b7990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb890005c40 tx=0x7fb890009500 comp rx=0 tx=0).stop 2024-09-11T01:09:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb89eec0640 1 -- 172.21.15.78:0/145054039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8981076c0 msgr2=0x7fb89810f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb89eec0640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8981076c0 0x7fb89810f890 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb884005940 tx=0x7fb8840040c0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.826 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:09:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb89eec0640 1 -- 172.21.15.78:0/145054039 shutdown_connections 2024-09-11T01:09:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb89eec0640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb8880b54d0 0x7fb8880b7990 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb89eec0640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb898108040 0x7fb89810fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb89eec0640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8981076c0 0x7fb89810f890 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb89eec0640 1 --2- 172.21.15.78:0/145054039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb898101cb0 0x7fb89810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb89eec0640 1 -- 172.21.15.78:0/145054039 >> 172.21.15.78:0/145054039 conn(0x7fb8980fd700 msgr2=0x7fb8980ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb89eec0640 1 -- 172.21.15.78:0/145054039 shutdown_connections 2024-09-11T01:09:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:43.526+0000 7fb89eec0640 1 -- 172.21.15.78:0/145054039 wait complete. 2024-09-11T01:09:44.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 716694 5.29 MB/sec execute 472 sec latency 17.104 ms 2024-09-11T01:09:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:44 smithi110 ceph-mon[34712]: pgmap v657: 97 pgs: 97 active+clean; 545 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2024-09-11T01:09:44.716 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:44 smithi078 ceph-mon[31315]: pgmap v657: 97 pgs: 97 active+clean; 545 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2024-09-11T01:09:44.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:44 smithi112 ceph-mon[35384]: pgmap v657: 97 pgs: 97 active+clean; 545 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2024-09-11T01:09:44.746 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:09:45.015 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:09:45.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 718140 5.29 MB/sec execute 473 sec latency 37.614 ms 2024-09-11T01:09:45.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.540+0000 7f4b42d80640 1 -- 172.21.15.78:0/2492817120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c1045c0 msgr2=0x7f4b3c10cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:45.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.540+0000 7f4b42d80640 1 --2- 172.21.15.78:0/2492817120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c1045c0 0x7f4b3c10cab0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009900 tx=0x7f4b2802a410 comp rx=0 tx=0).stop 2024-09-11T01:09:45.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.540+0000 7f4b42d80640 1 -- 172.21.15.78:0/2492817120 shutdown_connections 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.540+0000 7f4b42d80640 1 --2- 172.21.15.78:0/2492817120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b3c10d100 0x7f4b3c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.540+0000 7f4b42d80640 1 --2- 172.21.15.78:0/2492817120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c1045c0 0x7f4b3c10cab0 secure :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009900 tx=0x7f4b2802a410 comp rx=0 tx=0).stop 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.540+0000 7f4b42d80640 1 --2- 172.21.15.78:0/2492817120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c103ca0 0x7f4b3c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.540+0000 7f4b42d80640 1 -- 172.21.15.78:0/2492817120 >> 172.21.15.78:0/2492817120 conn(0x7f4b3c0fd700 msgr2=0x7f4b3c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.540+0000 7f4b42d80640 1 -- 172.21.15.78:0/2492817120 shutdown_connections 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.540+0000 7f4b42d80640 1 -- 172.21.15.78:0/2492817120 wait complete. 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b42d80640 1 Processor -- start 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b42d80640 1 -- start start 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b42d80640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c103ca0 0x7f4b3c101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b42d80640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b3c10d100 0x7f4b3c101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b42d80640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c1023d0 0x7f4b3c1a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b42d80640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c112d70 con 0x7f4b3c1023d0 2024-09-11T01:09:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b42d80640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c112bf0 con 0x7f4b3c103ca0 2024-09-11T01:09:45.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b42d80640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c112ef0 con 0x7f4b3c10d100 2024-09-11T01:09:45.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b4131d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c1023d0 0x7f4b3c1a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b4131d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c1023d0 0x7f4b3c1a7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32880/0 (socket says 172.21.15.78:32880) 2024-09-11T01:09:45.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b3bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b3c10d100 0x7f4b3c101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b4131d640 1 -- 172.21.15.78:0/3381946559 learned_addr learned my addr 172.21.15.78:0/3381946559 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:45.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.541+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c103ca0 0x7f4b3c101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.542+0000 7f4b4131d640 1 -- 172.21.15.78:0/3381946559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b3c10d100 msgr2=0x7f4b3c101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:45.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.542+0000 7f4b4131d640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b3c10d100 0x7f4b3c101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.542+0000 7f4b4131d640 1 -- 172.21.15.78:0/3381946559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c103ca0 msgr2=0x7f4b3c101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:45.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.542+0000 7f4b4131d640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c103ca0 0x7f4b3c101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.542+0000 7f4b4131d640 1 -- 172.21.15.78:0/3381946559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c1a7ca0 con 0x7f4b3c1023d0 2024-09-11T01:09:45.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.542+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c103ca0 0x7f4b3c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:45.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.542+0000 7f4b4131d640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c1023d0 0x7f4b3c1a7630 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400eb10 tx=0x7f4b3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:45.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.542+0000 7f4b39ffb640 1 -- 172.21.15.78:0/3381946559 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34004b20 con 0x7f4b3c1023d0 2024-09-11T01:09:45.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.542+0000 7f4b42d80640 1 -- 172.21.15.78:0/3381946559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c1a7f90 con 0x7f4b3c1023d0 2024-09-11T01:09:45.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.542+0000 7f4b39ffb640 1 -- 172.21.15.78:0/3381946559 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b34004cc0 con 0x7f4b3c1023d0 2024-09-11T01:09:45.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.542+0000 7f4b39ffb640 1 -- 172.21.15.78:0/3381946559 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34010690 con 0x7f4b3c1023d0 2024-09-11T01:09:45.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.542+0000 7f4b42d80640 1 -- 172.21.15.78:0/3381946559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c1a8440 con 0x7f4b3c1023d0 2024-09-11T01:09:45.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.544+0000 7f4b39ffb640 1 -- 172.21.15.78:0/3381946559 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4b34005020 con 0x7f4b3c1023d0 2024-09-11T01:09:45.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b45155740 1 Processor -- start 2024-09-11T01:09:45.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b45155740 1 -- start start 2024-09-11T01:09:45.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b45155740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a04d0 0x5649d48a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b45155740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 0x5649d48ac840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b45155740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48acd80 0x5649d48af170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b45155740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649d4816ea0 con 0x5649d48a0df0 2024-09-11T01:09:45.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b45155740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5649d48062d0 con 0x5649d48a04d0 2024-09-11T01:09:45.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b45155740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5649d480a0e0 con 0x5649d48acd80 2024-09-11T01:09:45.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b40b1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 0x5649d48ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b40b1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 0x5649d48ac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32882/0 (socket says 172.21.15.78:32882) 2024-09-11T01:09:45.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b40b1c640 1 -- 172.21.15.78:0/718542094 learned_addr learned my addr 172.21.15.78:0/718542094 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:45.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/718542094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48acd80 0x5649d48af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/718542094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a04d0 0x5649d48a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b40b1c640 1 -- 172.21.15.78:0/718542094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48acd80 msgr2=0x5649d48af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:45.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/718542094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48acd80 0x5649d48af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b39ffb640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4b2c0b5480 0x7f4b2c0b7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b40b1c640 1 -- 172.21.15.78:0/718542094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a04d0 msgr2=0x5649d48a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:45.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/718542094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a04d0 0x5649d48a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.545+0000 7f4b40b1c640 1 -- 172.21.15.78:0/718542094 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d4564650 con 0x5649d48a0df0 2024-09-11T01:09:45.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.546+0000 7f4b4131d640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4b2c0b5480 0x7f4b2c0b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.546+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/718542094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 0x5649d48ac840 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f4b300115f0 tx=0x7f4b30011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85fbfe86ae4d0bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:45.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.546+0000 7f4b39ffb640 1 -- 172.21.15.78:0/3381946559 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f4b340e64d0 con 0x7f4b3c1023d0 2024-09-11T01:09:45.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.546+0000 7f4b26ffd640 1 -- 172.21.15.78:0/718542094 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3001e070 con 0x5649d48a0df0 2024-09-11T01:09:45.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.546+0000 7f4b26ffd640 1 -- 172.21.15.78:0/718542094 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b3000faf0 con 0x5649d48a0df0 2024-09-11T01:09:45.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.546+0000 7f4b26ffd640 1 -- 172.21.15.78:0/718542094 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30019b50 con 0x5649d48a0df0 2024-09-11T01:09:45.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.547+0000 7f4b45155740 1 -- 172.21.15.78:0/718542094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 msgr2=0x5649d48ac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:45.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.547+0000 7f4b45155740 1 --2- 172.21.15.78:0/718542094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 0x5649d48ac840 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f4b300115f0 tx=0x7f4b30011ab0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.547+0000 7f4b4131d640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4b2c0b5480 0x7f4b2c0b7940 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4b340e3d10 tx=0x7f4b340af040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:45.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.547+0000 7f4b45155740 1 -- 172.21.15.78:0/718542094 shutdown_connections 2024-09-11T01:09:45.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.547+0000 7f4b45155740 1 --2- 172.21.15.78:0/718542094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48acd80 0x5649d48af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.547+0000 7f4b45155740 1 --2- 172.21.15.78:0/718542094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 0x5649d48ac840 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.547+0000 7f4b45155740 1 --2- 172.21.15.78:0/718542094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a04d0 0x5649d48a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.547+0000 7f4b45155740 1 -- 172.21.15.78:0/718542094 >> 172.21.15.78:0/718542094 conn(0x5649d489e300 msgr2=0x5649d489f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:45.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.547+0000 7f4b45155740 1 -- 172.21.15.78:0/718542094 shutdown_connections 2024-09-11T01:09:45.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.547+0000 7f4b45155740 1 -- 172.21.15.78:0/718542094 wait complete. 2024-09-11T01:09:45.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.547+0000 7f4b45155740 1 -- start start 2024-09-11T01:09:45.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.547+0000 7f4b45155740 1 Processor -- start 2024-09-11T01:09:45.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b45155740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a04d0 0x5649d48ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b45155740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48a0df0 0x5649d48a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b45155740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48acd80 0x5649d48aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b45155740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649d4667700 con 0x5649d48a04d0 2024-09-11T01:09:45.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b45155740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5649d47a1160 con 0x5649d48acd80 2024-09-11T01:09:45.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b45155740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5649d48afa00 con 0x5649d48a0df0 2024-09-11T01:09:45.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b45155740 10 client.0 _ll_register_callbacks cb 0x5649d47ba9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b3bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a04d0 0x5649d48ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b3bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a04d0 0x5649d48ac640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32894/0 (socket says 172.21.15.78:32894) 2024-09-11T01:09:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b3bfff640 1 -- 172.21.15.78:0/3489005230 learned_addr learned my addr 172.21.15.78:0/3489005230 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48a0df0 0x5649d48a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48acd80 0x5649d48aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b40b1c640 1 -- 172.21.15.78:0/3489005230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48acd80 msgr2=0x5649d48aa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48acd80 0x5649d48aa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b40b1c640 1 -- 172.21.15.78:0/3489005230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a04d0 msgr2=0x5649d48ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a04d0 0x5649d48ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b40b1c640 1 -- 172.21.15.78:0/3489005230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d47cec90 con 0x5649d48a0df0 2024-09-11T01:09:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.548+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a04d0 0x5649d48ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.549+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48a0df0 0x5649d48a9a90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b300115c0 tx=0x7f4b30009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52e43e03c1b754c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.549+0000 7f4b1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:09:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.549+0000 7f4b1bfff640 1 -- 172.21.15.78:0/3489005230 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30027060 con 0x5649d48a0df0 2024-09-11T01:09:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.549+0000 7f4b1bfff640 1 -- 172.21.15.78:0/3489005230 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f4b3000f700 con 0x5649d48a0df0 2024-09-11T01:09:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.549+0000 7f4b45155740 1 -- 172.21.15.78:0/3489005230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b3000faf0 con 0x5649d48a0df0 2024-09-11T01:09:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.549+0000 7f4b1bfff640 1 -- 172.21.15.78:0/3489005230 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4b3001e040 con 0x5649d48a0df0 2024-09-11T01:09:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.549+0000 7f4b45155740 10 client.26692 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.549+0000 7f4b45155740 1 -- 172.21.15.78:0/3489005230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5649d48bb330 con 0x5649d48a0df0 2024-09-11T01:09:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.550+0000 7f4b1bfff640 1 -- 172.21.15.78:0/3489005230 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b3000faf0 con 0x5649d48a0df0 2024-09-11T01:09:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.550+0000 7f4b1bfff640 1 -- 172.21.15.78:0/3489005230 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f4b30021070 con 0x5649d48a0df0 2024-09-11T01:09:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.550+0000 7f4b45155740 10 client.26692 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.550+0000 7f4b45155740 10 client.26692 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:09:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.550+0000 7f4b45155740 20 client.26692 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.550+0000 7f4b45155740 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5649d48bc730 0x5649d48beb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.550+0000 7f4b45155740 4 client.26692 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:09:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.550+0000 7f4b45155740 1 -- 172.21.15.78:0/3489005230 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649d4818df0 con 0x5649d48bc730 2024-09-11T01:09:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.550+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5649d48bc730 0x5649d48beb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.550+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5649d48bc730 0x5649d48beb20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d59d9177610270d server_cookie=b8d2d17ece27567c in_seq=0 out_seq=0 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.550+0000 7f4b1bfff640 10 client.26692 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b1bfff640 1 -- 172.21.15.78:0/3489005230 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649d4818df0 con 0x5649d48bc730 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b1bfff640 10 client.26692 handle_command_reply: tid=0 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b45155740 1 client.26692 shutdown 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b45155740 1 -- 172.21.15.78:0/3489005230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48a0df0 msgr2=0x5649d48a9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b45155740 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48a0df0 0x5649d48a9a90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b300115c0 tx=0x7f4b30009e10 comp rx=0 tx=0).stop 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b45155740 1 -- 172.21.15.78:0/3489005230 shutdown_connections 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b45155740 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48acd80 0x5649d48aa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b45155740 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48a0df0 0x5649d48a9a90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b45155740 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5649d48bc730 0x5649d48beb20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b45155740 1 --2- 172.21.15.78:0/3489005230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a04d0 0x5649d48ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b1bfff640 0 client.26692 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:09:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b45155740 1 -- 172.21.15.78:0/3489005230 >> 172.21.15.78:0/3489005230 conn(0x5649d489e300 msgr2=0x5649d48a3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b45155740 1 -- 172.21.15.78:0/3489005230 shutdown_connections 2024-09-11T01:09:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.551+0000 7f4b45155740 1 -- 172.21.15.78:0/3489005230 wait complete. 2024-09-11T01:09:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.552+0000 7f4b45155740 20 client.26692 trim_cache size 0 max 16384 2024-09-11T01:09:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.565+0000 7f4b45155740 1 Processor -- start 2024-09-11T01:09:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.565+0000 7f4b45155740 1 -- start start 2024-09-11T01:09:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.565+0000 7f4b45155740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48a04d0 0x5649d48a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.565+0000 7f4b45155740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a0df0 0x5649d48a8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.565+0000 7f4b45155740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48acd80 0x5649d48a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.565+0000 7f4b45155740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1b0e90 con 0x5649d48acd80 2024-09-11T01:09:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.565+0000 7f4b45155740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1b2a90 con 0x5649d48a0df0 2024-09-11T01:09:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.565+0000 7f4b45155740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1b1560 con 0x5649d48a04d0 2024-09-11T01:09:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b3bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48acd80 0x5649d48a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b40b1c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a0df0 0x5649d48a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b40b1c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a0df0 0x5649d48a8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59778/0 (socket says 172.21.15.78:59778) 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b40b1c640 1 -- 172.21.15.78:0/2955018474 learned_addr learned my addr 172.21.15.78:0/2955018474 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/2955018474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48a04d0 0x5649d48a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b3bfff640 1 -- 172.21.15.78:0/2955018474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48a04d0 msgr2=0x5649d48a8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/2955018474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48a04d0 0x5649d48a8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b3bfff640 1 -- 172.21.15.78:0/2955018474 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a0df0 msgr2=0x5649d48a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/2955018474 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a0df0 0x5649d48a8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b3bfff640 1 -- 172.21.15.78:0/2955018474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d4899bb0 con 0x5649d48acd80 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/2955018474 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a0df0 0x5649d48a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/2955018474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48acd80 0x5649d48a8850 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f4b28005940 tx=0x7f4b280058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5190b9c7a2e8246e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b267fc640 1 -- 172.21.15.78:0/2955018474 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28035040 con 0x5649d48acd80 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b267fc640 1 -- 172.21.15.78:0/2955018474 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b280040f0 con 0x5649d48acd80 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.566+0000 7f4b267fc640 1 -- 172.21.15.78:0/2955018474 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28003c00 con 0x5649d48acd80 2024-09-11T01:09:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.567+0000 7f4b45155740 1 -- 172.21.15.78:0/2955018474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48acd80 msgr2=0x5649d48a8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.567+0000 7f4b45155740 1 --2- 172.21.15.78:0/2955018474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48acd80 0x5649d48a8850 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f4b28005940 tx=0x7f4b280058d0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.567+0000 7f4b45155740 1 -- 172.21.15.78:0/2955018474 shutdown_connections 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.567+0000 7f4b45155740 1 --2- 172.21.15.78:0/2955018474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48acd80 0x5649d48a8850 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.567+0000 7f4b45155740 1 --2- 172.21.15.78:0/2955018474 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a0df0 0x5649d48a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.567+0000 7f4b45155740 1 --2- 172.21.15.78:0/2955018474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48a04d0 0x5649d48a8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.567+0000 7f4b45155740 1 -- 172.21.15.78:0/2955018474 >> 172.21.15.78:0/2955018474 conn(0x5649d489e300 msgr2=0x5649d489efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.567+0000 7f4b45155740 1 -- 172.21.15.78:0/2955018474 shutdown_connections 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.567+0000 7f4b45155740 1 -- 172.21.15.78:0/2955018474 wait complete. 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.567+0000 7f4b45155740 1 -- start start 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.567+0000 7f4b45155740 1 Processor -- start 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b45155740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a04d0 0x5649d48b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b45155740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 0x5649d48c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b45155740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48acd80 0x5649d48c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b45155740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1b2300 con 0x5649d48a0df0 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b45155740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1b1c30 con 0x5649d48a04d0 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b45155740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1aee90 con 0x5649d48acd80 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b45155740 10 client.0 _ll_register_callbacks cb 0x7f4b3001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b3bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a04d0 0x5649d48b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b3bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a04d0 0x5649d48b5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59782/0 (socket says 172.21.15.78:59782) 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b3bfff640 1 -- 172.21.15.78:0/1776799783 learned_addr learned my addr 172.21.15.78:0/1776799783 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 0x5649d48c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48acd80 0x5649d48c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b3bfff640 1 -- 172.21.15.78:0/1776799783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48acd80 msgr2=0x5649d48c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48acd80 0x5649d48c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b3bfff640 1 -- 172.21.15.78:0/1776799783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 msgr2=0x5649d48c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 0x5649d48c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b3bfff640 1 -- 172.21.15.78:0/1776799783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d48b1080 con 0x5649d48a04d0 2024-09-11T01:09:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.568+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 0x5649d48c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.569+0000 7f4b3bfff640 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a04d0 0x5649d48b5610 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b28031a10 tx=0x7f4b280037e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ada951e957dee952 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.569+0000 7f4b1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.569+0000 7f4b1bfff640 1 -- 172.21.15.78:0/1776799783 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28035040 con 0x5649d48a04d0 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.569+0000 7f4b45155740 1 -- 172.21.15.78:0/1776799783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b280040f0 con 0x5649d48a04d0 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.569+0000 7f4b1bfff640 1 -- 172.21.15.78:0/1776799783 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f4b280042a0 con 0x5649d48a04d0 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.569+0000 7f4b1bfff640 1 -- 172.21.15.78:0/1776799783 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4b280044a0 con 0x5649d48a04d0 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.569+0000 7f4b45155740 10 client.26771 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.569+0000 7f4b45155740 1 -- 172.21.15.78:0/1776799783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5649d4929620 con 0x5649d48a04d0 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.569+0000 7f4b1bfff640 1 -- 172.21.15.78:0/1776799783 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b280040f0 con 0x5649d48a04d0 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.570+0000 7f4b1bfff640 1 -- 172.21.15.78:0/1776799783 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f4b280321d0 con 0x5649d48a04d0 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.570+0000 7f4b45155740 10 client.26771 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.570+0000 7f4b45155740 10 client.26771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.570+0000 7f4b45155740 20 client.26771 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.570+0000 7f4b45155740 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5649d4929d30 0x5649d492c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.570+0000 7f4b45155740 4 client.26771 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.570+0000 7f4b45155740 1 -- 172.21.15.78:0/1776799783 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649d48148a0 con 0x5649d4929d30 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.570+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5649d4929d30 0x5649d492c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.570+0000 7f4b1bfff640 10 client.26771 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.570+0000 7f4b40b1c640 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5649d4929d30 0x5649d492c150 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56a66557b1f6b2eb server_cookie=ffc3f022b2892b9b in_seq=0 out_seq=0 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b1bfff640 1 -- 172.21.15.78:0/1776799783 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649d48148a0 con 0x5649d4929d30 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b1bfff640 10 client.26771 handle_command_reply: tid=0 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b45155740 1 client.26771 shutdown 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b45155740 1 -- 172.21.15.78:0/1776799783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a04d0 msgr2=0x5649d48b5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b45155740 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a04d0 0x5649d48b5610 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b28031a10 tx=0x7f4b280037e0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b45155740 1 -- 172.21.15.78:0/1776799783 shutdown_connections 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b45155740 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649d48acd80 0x5649d48c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b45155740 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5649d4929d30 0x5649d492c150 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b45155740 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d48a0df0 0x5649d48c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b45155740 1 --2- 172.21.15.78:0/1776799783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649d48a04d0 0x5649d48b5610 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b45155740 1 -- 172.21.15.78:0/1776799783 >> 172.21.15.78:0/1776799783 conn(0x5649d489e300 msgr2=0x5649d48ab0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b1bfff640 0 client.26771 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:09:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b45155740 1 -- 172.21.15.78:0/1776799783 shutdown_connections 2024-09-11T01:09:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b45155740 1 -- 172.21.15.78:0/1776799783 wait complete. 2024-09-11T01:09:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.571+0000 7f4b45155740 20 client.26771 trim_cache size 0 max 16384 2024-09-11T01:09:45.594 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:09:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.572+0000 7f4b42d80640 1 -- 172.21.15.78:0/3381946559 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4b2c0b5480 msgr2=0x7f4b2c0b7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.572+0000 7f4b42d80640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4b2c0b5480 0x7f4b2c0b7940 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4b340e3d10 tx=0x7f4b340af040 comp rx=0 tx=0).stop 2024-09-11T01:09:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.572+0000 7f4b42d80640 1 -- 172.21.15.78:0/3381946559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c1023d0 msgr2=0x7f4b3c1a7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.572+0000 7f4b42d80640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c1023d0 0x7f4b3c1a7630 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400eb10 tx=0x7f4b3400c2d0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.572+0000 7f4b42d80640 1 -- 172.21.15.78:0/3381946559 shutdown_connections 2024-09-11T01:09:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.572+0000 7f4b42d80640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4b2c0b5480 0x7f4b2c0b7940 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.572+0000 7f4b42d80640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c1023d0 0x7f4b3c1a7630 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.572+0000 7f4b42d80640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b3c10d100 0x7f4b3c101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.572+0000 7f4b42d80640 1 --2- 172.21.15.78:0/3381946559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c103ca0 0x7f4b3c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.572+0000 7f4b42d80640 1 -- 172.21.15.78:0/3381946559 >> 172.21.15.78:0/3381946559 conn(0x7f4b3c0fd700 msgr2=0x7f4b3c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:45.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.572+0000 7f4b42d80640 1 -- 172.21.15.78:0/3381946559 shutdown_connections 2024-09-11T01:09:45.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:45.572+0000 7f4b42d80640 1 -- 172.21.15.78:0/3381946559 wait complete. 2024-09-11T01:09:46.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 720245 5.30 MB/sec execute 474 sec latency 22.632 ms 2024-09-11T01:09:46.463 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:46 smithi078 ceph-mon[31315]: pgmap v658: 97 pgs: 97 active+clean; 552 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2024-09-11T01:09:46.465 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:09:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:46 smithi110 ceph-mon[34712]: pgmap v658: 97 pgs: 97 active+clean; 552 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2024-09-11T01:09:46.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:46 smithi112 ceph-mon[35384]: pgmap v658: 97 pgs: 97 active+clean; 552 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2024-09-11T01:09:46.762 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:09:47.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.278+0000 7f62fe955640 1 -- 172.21.15.78:0/3826715208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62f8101850 msgr2=0x7f62f8101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:47.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.278+0000 7f62fe955640 1 --2- 172.21.15.78:0/3826715208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62f8101850 0x7f62f8101c30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009900 tx=0x7f62e802a480 comp rx=0 tx=0).stop 2024-09-11T01:09:47.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.278+0000 7f62fe955640 1 -- 172.21.15.78:0/3826715208 shutdown_connections 2024-09-11T01:09:47.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.278+0000 7f62fe955640 1 --2- 172.21.15.78:0/3826715208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62f810f400 0x7f62f81117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.278+0000 7f62fe955640 1 --2- 172.21.15.78:0/3826715208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62f8102170 0x7f62f810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.278+0000 7f62fe955640 1 --2- 172.21.15.78:0/3826715208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62f8101850 0x7f62f8101c30 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.278+0000 7f62fe955640 1 -- 172.21.15.78:0/3826715208 >> 172.21.15.78:0/3826715208 conn(0x7f62f80fd700 msgr2=0x7f62f80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.278+0000 7f62fe955640 1 -- 172.21.15.78:0/3826715208 shutdown_connections 2024-09-11T01:09:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.278+0000 7f62fe955640 1 -- 172.21.15.78:0/3826715208 wait complete. 2024-09-11T01:09:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62fe955640 1 Processor -- start 2024-09-11T01:09:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62fe955640 1 -- start start 2024-09-11T01:09:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62fe955640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62f8101850 0x7f62f819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62fe955640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62f8102170 0x7f62f819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62fe955640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62f810f400 0x7f62f81a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62fe955640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8114fb0 con 0x7f62f810f400 2024-09-11T01:09:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62fe955640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8114e30 con 0x7f62f8101850 2024-09-11T01:09:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62fe955640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8115130 con 0x7f62f8102170 2024-09-11T01:09:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62fcef2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62f810f400 0x7f62f81a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62f7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62f8101850 0x7f62f819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62f7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62f8101850 0x7f62f819b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59802/0 (socket says 172.21.15.78:59802) 2024-09-11T01:09:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62fcef2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62f810f400 0x7f62f81a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32920/0 (socket says 172.21.15.78:32920) 2024-09-11T01:09:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.279+0000 7f62f7fff640 1 -- 172.21.15.78:0/811351651 learned_addr learned my addr 172.21.15.78:0/811351651 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.280+0000 7f62fcef2640 1 -- 172.21.15.78:0/811351651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62f8102170 msgr2=0x7f62f819ba30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T01:09:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.280+0000 7f62fcef2640 1 --2- 172.21.15.78:0/811351651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62f8102170 0x7f62f819ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.280+0000 7f62fcef2640 1 -- 172.21.15.78:0/811351651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62f8101850 msgr2=0x7f62f819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.280+0000 7f62fcef2640 1 --2- 172.21.15.78:0/811351651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62f8101850 0x7f62f819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.280+0000 7f62fcef2640 1 -- 172.21.15.78:0/811351651 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f81a15a0 con 0x7f62f810f400 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.280+0000 7f62fcef2640 1 --2- 172.21.15.78:0/811351651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62f810f400 0x7f62f81a0dd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009fd0 tx=0x7f62f000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.280+0000 7f62f57fa640 1 -- 172.21.15.78:0/811351651 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f000ce20 con 0x7f62f810f400 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.280+0000 7f62fe955640 1 -- 172.21.15.78:0/811351651 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f81a1890 con 0x7f62f810f400 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.280+0000 7f62f57fa640 1 -- 172.21.15.78:0/811351651 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62f0005020 con 0x7f62f810f400 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.280+0000 7f62fe955640 1 -- 172.21.15.78:0/811351651 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f81a7f50 con 0x7f62f810f400 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.280+0000 7f62f57fa640 1 -- 172.21.15.78:0/811351651 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0010690 con 0x7f62f810f400 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f62f57fa640 1 -- 172.21.15.78:0/811351651 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f62f00054d0 con 0x7f62f810f400 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f6300d2a740 1 Processor -- start 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f6300d2a740 1 -- start start 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f6300d2a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 0x55a0c0b114d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f6300d2a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 0x55a0c0b1d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f6300d2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b1fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f6300d2a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a0c0a87ea0 con 0x55a0c0b11a10 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f6300d2a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a0c0a772d0 con 0x55a0c0b110f0 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f6300d2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a0c0a7b0e0 con 0x55a0c0b1d9a0 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f62f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 0x55a0c0b1d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f62f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 0x55a0c0b1d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32926/0 (socket says 172.21.15.78:32926) 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 0x55a0c0b114d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.283+0000 7f62f7fff640 1 -- 172.21.15.78:0/912928115 learned_addr learned my addr 172.21.15.78:0/912928115 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62f77fe640 1 --2- 172.21.15.78:0/912928115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b1fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62f77fe640 1 -- 172.21.15.78:0/912928115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b1d9a0 msgr2=0x55a0c0b1fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62f77fe640 1 --2- 172.21.15.78:0/912928115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b1fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62f77fe640 1 -- 172.21.15.78:0/912928115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 msgr2=0x55a0c0b1d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62f77fe640 1 --2- 172.21.15.78:0/912928115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 0x55a0c0b1d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62f77fe640 1 -- 172.21.15.78:0/912928115 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c07d5650 con 0x55a0c0b110f0 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62f7fff640 1 --2- 172.21.15.78:0/912928115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 0x55a0c0b1d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62f77fe640 1 --2- 172.21.15.78:0/912928115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 0x55a0c0b114d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009f60 tx=0x7f62ec009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1077609758efc4d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62f57fa640 1 --2- 172.21.15.78:0/811351651 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62e00b5450 0x7f62e00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62de7fc640 1 -- 172.21.15.78:0/912928115 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0130d0 con 0x55a0c0b110f0 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62f7fff640 1 --2- 172.21.15.78:0/811351651 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62e00b5450 0x7f62e00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62f57fa640 1 -- 172.21.15.78:0/811351651 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f62f00e66f0 con 0x7f62f810f400 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62de7fc640 1 -- 172.21.15.78:0/912928115 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62ec013270 con 0x55a0c0b110f0 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.284+0000 7f62de7fc640 1 -- 172.21.15.78:0/912928115 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec005610 con 0x55a0c0b110f0 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.285+0000 7f62f7fff640 1 --2- 172.21.15.78:0/811351651 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62e00b5450 0x7f62e00b7910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55a0c0b0fe00 tx=0x7f62e8005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.285+0000 7f6300d2a740 1 -- 172.21.15.78:0/912928115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 msgr2=0x55a0c0b114d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.285+0000 7f6300d2a740 1 --2- 172.21.15.78:0/912928115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 0x55a0c0b114d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009f60 tx=0x7f62ec009620 comp rx=0 tx=0).stop 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.285+0000 7f6300d2a740 1 -- 172.21.15.78:0/912928115 shutdown_connections 2024-09-11T01:09:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.285+0000 7f6300d2a740 1 --2- 172.21.15.78:0/912928115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b1fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.285+0000 7f6300d2a740 1 --2- 172.21.15.78:0/912928115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 0x55a0c0b1d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.285+0000 7f6300d2a740 1 --2- 172.21.15.78:0/912928115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 0x55a0c0b114d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.285+0000 7f6300d2a740 1 -- 172.21.15.78:0/912928115 >> 172.21.15.78:0/912928115 conn(0x55a0c0b0ef20 msgr2=0x55a0c0b100c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.285+0000 7f6300d2a740 1 -- 172.21.15.78:0/912928115 shutdown_connections 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.285+0000 7f6300d2a740 1 -- 172.21.15.78:0/912928115 wait complete. 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.285+0000 7f6300d2a740 1 -- start start 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f6300d2a740 1 Processor -- start 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f6300d2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b110f0 0x55a0c0b1d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f6300d2a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 0x55a0c0b1a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f6300d2a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b1ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f6300d2a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a0c08d8700 con 0x55a0c0b11a10 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f6300d2a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a0c0a12160 con 0x55a0c0b1d9a0 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f6300d2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a0c0b205c0 con 0x55a0c0b110f0 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f6300d2a740 10 client.0 _ll_register_callbacks cb 0x55a0c0a2b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 0x55a0c0b1a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 0x55a0c0b1a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32928/0 (socket says 172.21.15.78:32928) 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f62f77fe640 1 -- 172.21.15.78:0/2499286144 learned_addr learned my addr 172.21.15.78:0/2499286144 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f62f7fff640 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b110f0 0x55a0c0b1d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f62fcef2640 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b1ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f62f77fe640 1 -- 172.21.15.78:0/2499286144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b110f0 msgr2=0x55a0c0b1d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f62f77fe640 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b110f0 0x55a0c0b1d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f62f77fe640 1 -- 172.21.15.78:0/2499286144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b1d9a0 msgr2=0x55a0c0b1ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f62f77fe640 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b1ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.286+0000 7f62f77fe640 1 -- 172.21.15.78:0/2499286144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c0a3fc90 con 0x55a0c0b11a10 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.287+0000 7f62f77fe640 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 0x55a0c0b1a6f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0096f0 tx=0x7f62ec00ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cafe59b6d58a92e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.287+0000 7f62d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:09:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.287+0000 7f62d77fe640 1 -- 172.21.15.78:0/2499286144 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0130d0 con 0x55a0c0b11a10 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.287+0000 7f62d77fe640 1 -- 172.21.15.78:0/2499286144 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f62ec0046c0 con 0x55a0c0b11a10 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.287+0000 7f6300d2a740 1 -- 172.21.15.78:0/2499286144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62ec013270 con 0x55a0c0b11a10 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.287+0000 7f62d77fe640 1 -- 172.21.15.78:0/2499286144 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f62ec013420 con 0x55a0c0b11a10 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.287+0000 7f6300d2a740 10 client.17583 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.287+0000 7f6300d2a740 1 -- 172.21.15.78:0/2499286144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a0c0b2bf40 con 0x55a0c0b11a10 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.287+0000 7f62d77fe640 1 -- 172.21.15.78:0/2499286144 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62ec013270 con 0x55a0c0b11a10 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.287+0000 7f62d77fe640 1 -- 172.21.15.78:0/2499286144 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f62ec0115d0 con 0x55a0c0b11a10 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.288+0000 7f6300d2a740 10 client.17583 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.288+0000 7f6300d2a740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.288+0000 7f6300d2a740 20 client.17583 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.288+0000 7f6300d2a740 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a0c0b2d3e0 0x55a0c0b2f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.288+0000 7f6300d2a740 4 client.17583 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:09:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.288+0000 7f6300d2a740 1 -- 172.21.15.78:0/2499286144 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0c0a89df0 con 0x55a0c0b2d3e0 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.288+0000 7f62f7fff640 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a0c0b2d3e0 0x55a0c0b2f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.288+0000 7f62d77fe640 10 client.17583 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.288+0000 7f62f7fff640 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a0c0b2d3e0 0x55a0c0b2f7d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=760fb5fb2aac0e8c server_cookie=4256cc5397a71548 in_seq=0 out_seq=0 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.289+0000 7f62d77fe640 1 -- 172.21.15.78:0/2499286144 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a0c0a89df0 con 0x55a0c0b2d3e0 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.289+0000 7f62d77fe640 10 client.17583 handle_command_reply: tid=0 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.289+0000 7f6300d2a740 1 client.17583 shutdown 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.289+0000 7f6300d2a740 1 -- 172.21.15.78:0/2499286144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 msgr2=0x55a0c0b1a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.289+0000 7f6300d2a740 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 0x55a0c0b1a6f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0096f0 tx=0x7f62ec00ea00 comp rx=0 tx=0).stop 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.289+0000 7f6300d2a740 1 -- 172.21.15.78:0/2499286144 shutdown_connections 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.289+0000 7f6300d2a740 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b1ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.289+0000 7f6300d2a740 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a0c0b2d3e0 0x55a0c0b2f7d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.289+0000 7f6300d2a740 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b11a10 0x55a0c0b1a6f0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.289+0000 7f6300d2a740 1 --2- 172.21.15.78:0/2499286144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b110f0 0x55a0c0b1d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.289+0000 7f6300d2a740 1 -- 172.21.15.78:0/2499286144 >> 172.21.15.78:0/2499286144 conn(0x55a0c0b0ef20 msgr2=0x55a0c0b14ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:47.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.289+0000 7f62d77fe640 0 client.17583 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:47.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.290+0000 7f6300d2a740 1 -- 172.21.15.78:0/2499286144 shutdown_connections 2024-09-11T01:09:47.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.290+0000 7f6300d2a740 1 -- 172.21.15.78:0/2499286144 wait complete. 2024-09-11T01:09:47.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.290+0000 7f6300d2a740 20 client.17583 trim_cache size 0 max 16384 2024-09-11T01:09:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f6300d2a740 1 Processor -- start 2024-09-11T01:09:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f6300d2a740 1 -- start start 2024-09-11T01:09:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f6300d2a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b110f0 0x55a0c0b29820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f6300d2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b19680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f6300d2a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b10800 0x55a0c0b1ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f6300d2a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81b0800 con 0x55a0c0b110f0 2024-09-11T01:09:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f6300d2a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81b2400 con 0x55a0c0b10800 2024-09-11T01:09:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f6300d2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81b0ed0 con 0x55a0c0b1d9a0 2024-09-11T01:09:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f62f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b110f0 0x55a0c0b29820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f62f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b110f0 0x55a0c0b29820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32930/0 (socket says 172.21.15.78:32930) 2024-09-11T01:09:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f62f7fff640 1 -- 172.21.15.78:0/3958008304 learned_addr learned my addr 172.21.15.78:0/3958008304 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f62fcef2640 1 --2- 172.21.15.78:0/3958008304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b10800 0x55a0c0b1ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.304+0000 7f62f77fe640 1 --2- 172.21.15.78:0/3958008304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b19680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.305+0000 7f62f7fff640 1 -- 172.21.15.78:0/3958008304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b1d9a0 msgr2=0x55a0c0b19680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.305+0000 7f62f7fff640 1 --2- 172.21.15.78:0/3958008304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b19680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.305+0000 7f62f7fff640 1 -- 172.21.15.78:0/3958008304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b10800 msgr2=0x55a0c0b1ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.305+0000 7f62f7fff640 1 --2- 172.21.15.78:0/3958008304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b10800 0x55a0c0b1ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.305+0000 7f62f7fff640 1 -- 172.21.15.78:0/3958008304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c0b0fc00 con 0x55a0c0b110f0 2024-09-11T01:09:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.305+0000 7f62f7fff640 1 --2- 172.21.15.78:0/3958008304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b110f0 0x55a0c0b29820 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f62e802aa70 tx=0x7f62e8040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1584f995444ffea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.305+0000 7f62ddffb640 1 -- 172.21.15.78:0/3958008304 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8035040 con 0x55a0c0b110f0 2024-09-11T01:09:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.305+0000 7f62ddffb640 1 -- 172.21.15.78:0/3958008304 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e8003cc0 con 0x55a0c0b110f0 2024-09-11T01:09:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.305+0000 7f62ddffb640 1 -- 172.21.15.78:0/3958008304 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8054680 con 0x55a0c0b110f0 2024-09-11T01:09:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.306+0000 7f6300d2a740 1 -- 172.21.15.78:0/3958008304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b110f0 msgr2=0x55a0c0b29820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.306+0000 7f6300d2a740 1 --2- 172.21.15.78:0/3958008304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b110f0 0x55a0c0b29820 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f62e802aa70 tx=0x7f62e8040000 comp rx=0 tx=0).stop 2024-09-11T01:09:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.306+0000 7f6300d2a740 1 -- 172.21.15.78:0/3958008304 shutdown_connections 2024-09-11T01:09:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.306+0000 7f6300d2a740 1 --2- 172.21.15.78:0/3958008304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b10800 0x55a0c0b1ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.306+0000 7f6300d2a740 1 --2- 172.21.15.78:0/3958008304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b19680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.306+0000 7f6300d2a740 1 --2- 172.21.15.78:0/3958008304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b110f0 0x55a0c0b29820 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.306+0000 7f6300d2a740 1 -- 172.21.15.78:0/3958008304 >> 172.21.15.78:0/3958008304 conn(0x55a0c0b0ef20 msgr2=0x55a0c0b18f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.306+0000 7f6300d2a740 1 -- 172.21.15.78:0/3958008304 shutdown_connections 2024-09-11T01:09:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.307+0000 7f6300d2a740 1 -- 172.21.15.78:0/3958008304 wait complete. 2024-09-11T01:09:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.307+0000 7f6300d2a740 1 -- start start 2024-09-11T01:09:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.307+0000 7f6300d2a740 1 Processor -- start 2024-09-11T01:09:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f6300d2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b10800 0x55a0c0b261f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f6300d2a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 0x55a0c0b20840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f6300d2a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b20fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62fcef2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b20fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 0x55a0c0b20840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f6300d2a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81b1c70 con 0x55a0c0b1d9a0 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 0x55a0c0b20840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59836/0 (socket says 172.21.15.78:59836) 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f6300d2a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81b15a0 con 0x55a0c0b110f0 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62f77fe640 1 -- 172.21.15.78:0/1967235972 learned_addr learned my addr 172.21.15.78:0/1967235972 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f6300d2a740 1 -- 172.21.15.78:0/1967235972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81ae800 con 0x55a0c0b10800 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62fcef2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b20fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32944/0 (socket says 172.21.15.78:32944) 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62f7fff640 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b10800 0x55a0c0b261f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f6300d2a740 10 client.0 _ll_register_callbacks cb 0x7f62ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62f77fe640 1 -- 172.21.15.78:0/1967235972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b10800 msgr2=0x55a0c0b261f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62f77fe640 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b10800 0x55a0c0b261f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62f77fe640 1 -- 172.21.15.78:0/1967235972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b1d9a0 msgr2=0x55a0c0b20fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62f77fe640 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b20fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62f77fe640 1 -- 172.21.15.78:0/1967235972 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c0ba2830 con 0x55a0c0b110f0 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62fcef2640 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b20fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62f77fe640 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 0x55a0c0b20840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f62ec006010 tx=0x7f62ec0093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fffc6af829b8f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.308+0000 7f62d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f62d77fe640 1 -- 172.21.15.78:0/1967235972 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec009a90 con 0x55a0c0b110f0 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f62d77fe640 1 -- 172.21.15.78:0/1967235972 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f62ec011440 con 0x55a0c0b110f0 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f6300d2a740 1 -- 172.21.15.78:0/1967235972 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e8003cc0 con 0x55a0c0b110f0 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f62d77fe640 1 -- 172.21.15.78:0/1967235972 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f62ec005a70 con 0x55a0c0b110f0 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f6300d2a740 10 client.26789 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f6300d2a740 1 -- 172.21.15.78:0/1967235972 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a0c0ba3a30 con 0x55a0c0b110f0 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f62d77fe640 1 -- 172.21.15.78:0/1967235972 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62ec011660 con 0x55a0c0b110f0 2024-09-11T01:09:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f62d77fe640 1 -- 172.21.15.78:0/1967235972 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f62ec038070 con 0x55a0c0b110f0 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f6300d2a740 10 client.26789 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f6300d2a740 10 client.26789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f6300d2a740 20 client.26789 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f6300d2a740 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a0c0ba4060 0x55a0c0ba6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f6300d2a740 4 client.26789 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f6300d2a740 1 -- 172.21.15.78:0/1967235972 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f62e8003cc0 con 0x55a0c0ba4060 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.309+0000 7f62f7fff640 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a0c0ba4060 0x55a0c0ba6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.310+0000 7f62d77fe640 10 client.26789 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.310+0000 7f62f7fff640 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a0c0ba4060 0x55a0c0ba6480 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dcbd9f98d79677d server_cookie=601b716452f69dbb in_seq=0 out_seq=0 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.310+0000 7f62d77fe640 1 -- 172.21.15.78:0/1967235972 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f62e8003cc0 con 0x55a0c0ba4060 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.310+0000 7f62d77fe640 10 client.26789 handle_command_reply: tid=0 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.310+0000 7f6300d2a740 1 client.26789 shutdown 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f6300d2a740 1 -- 172.21.15.78:0/1967235972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 msgr2=0x55a0c0b20840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:47.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f6300d2a740 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 0x55a0c0b20840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f62ec006010 tx=0x7f62ec0093e0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f6300d2a740 1 -- 172.21.15.78:0/1967235972 shutdown_connections 2024-09-11T01:09:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f6300d2a740 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a0c0ba4060 0x55a0c0ba6480 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f6300d2a740 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c0b1d9a0 0x55a0c0b20fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f6300d2a740 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0c0b110f0 0x55a0c0b20840 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f62d77fe640 0 client.26789 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f6300d2a740 1 --2- 172.21.15.78:0/1967235972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0c0b10800 0x55a0c0b261f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f6300d2a740 1 -- 172.21.15.78:0/1967235972 >> 172.21.15.78:0/1967235972 conn(0x55a0c0b0ef20 msgr2=0x55a0c0b19790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f6300d2a740 1 -- 172.21.15.78:0/1967235972 shutdown_connections 2024-09-11T01:09:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f6300d2a740 1 -- 172.21.15.78:0/1967235972 wait complete. 2024-09-11T01:09:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f6300d2a740 20 client.26789 trim_cache size 0 max 16384 2024-09-11T01:09:47.318 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:09:47.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f62fe955640 1 -- 172.21.15.78:0/811351651 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62e00b5450 msgr2=0x7f62e00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:47.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f62fe955640 1 --2- 172.21.15.78:0/811351651 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62e00b5450 0x7f62e00b7910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55a0c0b0fe00 tx=0x7f62e8005a20 comp rx=0 tx=0).stop 2024-09-11T01:09:47.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f62fe955640 1 -- 172.21.15.78:0/811351651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62f810f400 msgr2=0x7f62f81a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:47.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f62fe955640 1 --2- 172.21.15.78:0/811351651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62f810f400 0x7f62f81a0dd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009fd0 tx=0x7f62f000ef10 comp rx=0 tx=0).stop 2024-09-11T01:09:47.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f62fe955640 1 -- 172.21.15.78:0/811351651 shutdown_connections 2024-09-11T01:09:47.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.311+0000 7f62fe955640 1 --2- 172.21.15.78:0/811351651 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62e00b5450 0x7f62e00b7910 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.312+0000 7f62fe955640 1 --2- 172.21.15.78:0/811351651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62f810f400 0x7f62f81a0dd0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.312+0000 7f62fe955640 1 --2- 172.21.15.78:0/811351651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62f8102170 0x7f62f819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.312+0000 7f62fe955640 1 --2- 172.21.15.78:0/811351651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62f8101850 0x7f62f819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:47.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.312+0000 7f62fe955640 1 -- 172.21.15.78:0/811351651 >> 172.21.15.78:0/811351651 conn(0x7f62f80fd700 msgr2=0x7f62f8110740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:47.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.312+0000 7f62fe955640 1 -- 172.21.15.78:0/811351651 shutdown_connections 2024-09-11T01:09:47.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:47.312+0000 7f62fe955640 1 -- 172.21.15.78:0/811351651 wait complete. 2024-09-11T01:09:47.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 721637 5.31 MB/sec execute 475 sec latency 34.949 ms 2024-09-11T01:09:48.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 723787 5.32 MB/sec execute 476 sec latency 20.809 ms 2024-09-11T01:09:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:48 smithi110 ceph-mon[34712]: pgmap v659: 97 pgs: 97 active+clean; 556 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-09-11T01:09:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:48 smithi112 ceph-mon[35384]: pgmap v659: 97 pgs: 97 active+clean; 556 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-09-11T01:09:48.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:48 smithi078 ceph-mon[31315]: pgmap v659: 97 pgs: 97 active+clean; 556 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-09-11T01:09:49.021 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:09:49.021 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:09:49.289 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:09:49.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 725384 5.33 MB/sec execute 477 sec latency 10.467 ms 2024-09-11T01:09:50.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.162+0000 7f0a97744640 1 -- 172.21.15.78:0/3674763236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a90102dc0 msgr2=0x7f0a901031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:50.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.162+0000 7f0a97744640 1 --2- 172.21.15.78:0/3674763236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a90102dc0 0x7f0a901031a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009900 tx=0x7f0a8c02a410 comp rx=0 tx=0).stop 2024-09-11T01:09:50.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.162+0000 7f0a97744640 1 -- 172.21.15.78:0/3674763236 shutdown_connections 2024-09-11T01:09:50.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.162+0000 7f0a97744640 1 --2- 172.21.15.78:0/3674763236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a90109150 0x7f0a9010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.162+0000 7f0a97744640 1 --2- 172.21.15.78:0/3674763236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a901087d0 0x7f0a90108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.162+0000 7f0a97744640 1 --2- 172.21.15.78:0/3674763236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a90102dc0 0x7f0a901031a0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.162+0000 7f0a97744640 1 -- 172.21.15.78:0/3674763236 >> 172.21.15.78:0/3674763236 conn(0x7f0a900fe870 msgr2=0x7f0a90100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:50.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.163+0000 7f0a97744640 1 -- 172.21.15.78:0/3674763236 shutdown_connections 2024-09-11T01:09:50.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.163+0000 7f0a97744640 1 -- 172.21.15.78:0/3674763236 wait complete. 2024-09-11T01:09:50.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.163+0000 7f0a97744640 1 Processor -- start 2024-09-11T01:09:50.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.163+0000 7f0a97744640 1 -- start start 2024-09-11T01:09:50.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.163+0000 7f0a97744640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a90102dc0 0x7f0a90199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.163+0000 7f0a97744640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a901087d0 0x7f0a90199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.163+0000 7f0a97744640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a90109150 0x7f0a9019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.163+0000 7f0a97744640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90112e10 con 0x7f0a901087d0 2024-09-11T01:09:50.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.163+0000 7f0a97744640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90112c90 con 0x7f0a90102dc0 2024-09-11T01:09:50.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.163+0000 7f0a97744640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90112f90 con 0x7f0a90109150 2024-09-11T01:09:50.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a94cdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a901087d0 0x7f0a90199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a954e0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a90102dc0 0x7f0a90199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a954e0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a90102dc0 0x7f0a90199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59852/0 (socket says 172.21.15.78:59852) 2024-09-11T01:09:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a94cdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a901087d0 0x7f0a90199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32960/0 (socket says 172.21.15.78:32960) 2024-09-11T01:09:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a95ce1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a90109150 0x7f0a9019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a954e0640 1 -- 172.21.15.78:0/3279387115 learned_addr learned my addr 172.21.15.78:0/3279387115 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a954e0640 1 -- 172.21.15.78:0/3279387115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a90109150 msgr2=0x7f0a9019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a954e0640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a90109150 0x7f0a9019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a954e0640 1 -- 172.21.15.78:0/3279387115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a901087d0 msgr2=0x7f0a90199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a954e0640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a901087d0 0x7f0a90199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a954e0640 1 -- 172.21.15.78:0/3279387115 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9019f4b0 con 0x7f0a90102dc0 2024-09-11T01:09:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a90109150 0x7f0a9019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a94cdf640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a901087d0 0x7f0a90199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a954e0640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a90102dc0 0x7f0a90199400 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005940 tx=0x7f0a8c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:50.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a867fc640 1 -- 172.21.15.78:0/3279387115 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c03f030 con 0x7f0a90102dc0 2024-09-11T01:09:50.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a97744640 1 -- 172.21.15.78:0/3279387115 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9019f740 con 0x7f0a90102dc0 2024-09-11T01:09:50.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a867fc640 1 -- 172.21.15.78:0/3279387115 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a8c0044c0 con 0x7f0a90102dc0 2024-09-11T01:09:50.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a867fc640 1 -- 172.21.15.78:0/3279387115 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c035070 con 0x7f0a90102dc0 2024-09-11T01:09:50.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.164+0000 7f0a97744640 1 -- 172.21.15.78:0/3279387115 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a90075380 con 0x7f0a90102dc0 2024-09-11T01:09:50.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.167+0000 7f0a99b19740 1 Processor -- start 2024-09-11T01:09:50.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.167+0000 7f0a99b19740 1 -- start start 2024-09-11T01:09:50.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.167+0000 7f0a99b19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 0x5572c59d72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.167+0000 7f0a99b19740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d7800 0x5572c59e3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.167+0000 7f0a99b19740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59e3790 0x5572c59e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.167+0000 7f0a99b19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572c594de60 con 0x5572c59d6ee0 2024-09-11T01:09:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.167+0000 7f0a99b19740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5572c593d290 con 0x5572c59e3790 2024-09-11T01:09:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.167+0000 7f0a99b19740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5572c59410a0 con 0x5572c59d7800 2024-09-11T01:09:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a867fc640 1 -- 172.21.15.78:0/3279387115 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0a8c03b9f0 con 0x7f0a90102dc0 2024-09-11T01:09:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a95ce1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 0x5572c59d72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a95ce1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 0x5572c59d72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32964/0 (socket says 172.21.15.78:32964) 2024-09-11T01:09:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a95ce1640 1 -- 172.21.15.78:0/1417981300 learned_addr learned my addr 172.21.15.78:0/1417981300 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a94cdf640 1 --2- 172.21.15.78:0/1417981300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59e3790 0x5572c59e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a954e0640 1 --2- 172.21.15.78:0/1417981300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d7800 0x5572c59e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a95ce1640 1 -- 172.21.15.78:0/1417981300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d7800 msgr2=0x5572c59e3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/1417981300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d7800 0x5572c59e3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a95ce1640 1 -- 172.21.15.78:0/1417981300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59e3790 msgr2=0x5572c59e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/1417981300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59e3790 0x5572c59e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a95ce1640 1 -- 172.21.15.78:0/1417981300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c569b930 con 0x5572c59d6ee0 2024-09-11T01:09:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/1417981300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 0x5572c59d72c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0a88010ab0 tx=0x7f0a88009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc470e196905e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a7b7fe640 1 -- 172.21.15.78:0/1417981300 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8801e020 con 0x5572c59d6ee0 2024-09-11T01:09:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a7b7fe640 1 -- 172.21.15.78:0/1417981300 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a88004d20 con 0x5572c59d6ee0 2024-09-11T01:09:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.168+0000 7f0a7b7fe640 1 -- 172.21.15.78:0/1417981300 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88004290 con 0x5572c59d6ee0 2024-09-11T01:09:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a867fc640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a7c0b5450 0x7f0a7c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a954e0640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a7c0b5450 0x7f0a7c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a867fc640 1 -- 172.21.15.78:0/3279387115 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f0a8c106610 con 0x7f0a90102dc0 2024-09-11T01:09:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a99b19740 1 -- 172.21.15.78:0/1417981300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 msgr2=0x5572c59d72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a99b19740 1 --2- 172.21.15.78:0/1417981300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 0x5572c59d72c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0a88010ab0 tx=0x7f0a88009500 comp rx=0 tx=0).stop 2024-09-11T01:09:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a954e0640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a7c0b5450 0x7f0a7c0b7910 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5572c59da060 tx=0x7f0a8c02a410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a99b19740 1 -- 172.21.15.78:0/1417981300 shutdown_connections 2024-09-11T01:09:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a99b19740 1 --2- 172.21.15.78:0/1417981300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59e3790 0x5572c59e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a99b19740 1 --2- 172.21.15.78:0/1417981300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d7800 0x5572c59e3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a99b19740 1 --2- 172.21.15.78:0/1417981300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 0x5572c59d72c0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a99b19740 1 -- 172.21.15.78:0/1417981300 >> 172.21.15.78:0/1417981300 conn(0x5572c59d4d10 msgr2=0x5572c59d5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a99b19740 1 -- 172.21.15.78:0/1417981300 shutdown_connections 2024-09-11T01:09:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.169+0000 7f0a99b19740 1 -- 172.21.15.78:0/1417981300 wait complete. 2024-09-11T01:09:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a99b19740 1 -- start start 2024-09-11T01:09:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a99b19740 1 Processor -- start 2024-09-11T01:09:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a99b19740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d6ee0 0x5572c59e0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a99b19740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 0x5572c59e0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a99b19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 0x5572c59e11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a99b19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572c579e890 con 0x5572c59e3790 2024-09-11T01:09:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a99b19740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5572c58d7d40 con 0x5572c59d7800 2024-09-11T01:09:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a99b19740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5572c59e6440 con 0x5572c59d6ee0 2024-09-11T01:09:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a99b19740 10 client.0 _ll_register_callbacks cb 0x5572c593bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a95ce1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 0x5572c59e11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a94cdf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 0x5572c59e0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a95ce1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 0x5572c59e11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32970/0 (socket says 172.21.15.78:32970) 2024-09-11T01:09:50.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a95ce1640 1 -- 172.21.15.78:0/3166588258 learned_addr learned my addr 172.21.15.78:0/3166588258 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:50.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.170+0000 7f0a94cdf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 0x5572c59e0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59868/0 (socket says 172.21.15.78:59868) 2024-09-11T01:09:50.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d6ee0 0x5572c59e0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a94cdf640 1 -- 172.21.15.78:0/3166588258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d6ee0 msgr2=0x5572c59e0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:50.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a94cdf640 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d6ee0 0x5572c59e0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a94cdf640 1 -- 172.21.15.78:0/3166588258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 msgr2=0x5572c59e11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:50.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a94cdf640 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 0x5572c59e11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a94cdf640 1 -- 172.21.15.78:0/3166588258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c59ef820 con 0x5572c59d7800 2024-09-11T01:09:50.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d6ee0 0x5572c59e0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:09:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a94cdf640 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 0x5572c59e0c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0a90107dc0 tx=0x7f0a8000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc19939475ec748f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:09:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a78ff9640 1 -- 172.21.15.78:0/3166588258 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80012110 con 0x5572c59d7800 2024-09-11T01:09:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a78ff9640 1 -- 172.21.15.78:0/3166588258 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f0a80004050 con 0x5572c59d7800 2024-09-11T01:09:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a99b19740 1 -- 172.21.15.78:0/3166588258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a88004d20 con 0x5572c59d7800 2024-09-11T01:09:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.171+0000 7f0a78ff9640 1 -- 172.21.15.78:0/3166588258 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0a800122b0 con 0x5572c59d7800 2024-09-11T01:09:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.172+0000 7f0a99b19740 10 client.26801 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.172+0000 7f0a99b19740 1 -- 172.21.15.78:0/3166588258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5572c59f21d0 con 0x5572c59d7800 2024-09-11T01:09:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.172+0000 7f0a78ff9640 1 -- 172.21.15.78:0/3166588258 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a88004d20 con 0x5572c59d7800 2024-09-11T01:09:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.172+0000 7f0a78ff9640 1 -- 172.21.15.78:0/3166588258 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f0a80010430 con 0x5572c59d7800 2024-09-11T01:09:50.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.172+0000 7f0a99b19740 10 client.26801 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:50.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.172+0000 7f0a99b19740 10 client.26801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:09:50.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.172+0000 7f0a99b19740 20 client.26801 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:50.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.172+0000 7f0a99b19740 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572c59f3830 0x5572c59f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.172+0000 7f0a99b19740 4 client.26801 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:09:50.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.172+0000 7f0a99b19740 1 -- 172.21.15.78:0/3166588258 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572c594fdb0 con 0x5572c59f3830 2024-09-11T01:09:50.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.172+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572c59f3830 0x5572c59f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.173+0000 7f0a78ff9640 10 client.26801 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.173+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572c59f3830 0x5572c59f5c50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31bc1ae9a228cc8 server_cookie=f057fbec7f5ca088 in_seq=0 out_seq=0 2024-09-11T01:09:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.173+0000 7f0a78ff9640 1 -- 172.21.15.78:0/3166588258 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5572c594fdb0 con 0x5572c59f3830 2024-09-11T01:09:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.173+0000 7f0a78ff9640 10 client.26801 handle_command_reply: tid=0 2024-09-11T01:09:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.173+0000 7f0a99b19740 1 client.26801 shutdown 2024-09-11T01:09:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.173+0000 7f0a99b19740 1 -- 172.21.15.78:0/3166588258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 msgr2=0x5572c59e0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.173+0000 7f0a99b19740 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 0x5572c59e0c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0a90107dc0 tx=0x7f0a8000f790 comp rx=0 tx=0).stop 2024-09-11T01:09:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.174+0000 7f0a99b19740 1 -- 172.21.15.78:0/3166588258 shutdown_connections 2024-09-11T01:09:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.174+0000 7f0a99b19740 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572c59f3830 0x5572c59f5c50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.174+0000 7f0a99b19740 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 0x5572c59e11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.174+0000 7f0a78ff9640 0 client.26801 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.174+0000 7f0a99b19740 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 0x5572c59e0c90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.174+0000 7f0a99b19740 1 --2- 172.21.15.78:0/3166588258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d6ee0 0x5572c59e0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.174+0000 7f0a99b19740 1 -- 172.21.15.78:0/3166588258 >> 172.21.15.78:0/3166588258 conn(0x5572c59d4d10 msgr2=0x5572c59d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:50.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.174+0000 7f0a99b19740 1 -- 172.21.15.78:0/3166588258 shutdown_connections 2024-09-11T01:09:50.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.174+0000 7f0a99b19740 1 -- 172.21.15.78:0/3166588258 wait complete. 2024-09-11T01:09:50.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.174+0000 7f0a99b19740 20 client.26801 trim_cache size 0 max 16384 2024-09-11T01:09:50.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.192+0000 7f0a99b19740 1 Processor -- start 2024-09-11T01:09:50.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.192+0000 7f0a99b19740 1 -- start start 2024-09-11T01:09:50.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.192+0000 7f0a99b19740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d6ee0 0x5572c59de1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.192+0000 7f0a99b19740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 0x5572c59fb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.192+0000 7f0a99b19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 0x5572c59fba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.192+0000 7f0a99b19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a901ac330 con 0x5572c59e3790 2024-09-11T01:09:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.192+0000 7f0a99b19740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a901adf30 con 0x5572c59d7800 2024-09-11T01:09:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.192+0000 7f0a99b19740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a901aca00 con 0x5572c59d6ee0 2024-09-11T01:09:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.192+0000 7f0a95ce1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 0x5572c59fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a94cdf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 0x5572c59fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a95ce1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 0x5572c59fba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32978/0 (socket says 172.21.15.78:32978) 2024-09-11T01:09:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a95ce1640 1 -- 172.21.15.78:0/1312153681 learned_addr learned my addr 172.21.15.78:0/1312153681 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/1312153681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d6ee0 0x5572c59de1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a95ce1640 1 -- 172.21.15.78:0/1312153681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d6ee0 msgr2=0x5572c59de1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/1312153681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d6ee0 0x5572c59de1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a95ce1640 1 -- 172.21.15.78:0/1312153681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 msgr2=0x5572c59fb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/1312153681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 0x5572c59fb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a95ce1640 1 -- 172.21.15.78:0/1312153681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c59d5790 con 0x5572c59e3790 2024-09-11T01:09:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a94cdf640 1 --2- 172.21.15.78:0/1312153681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 0x5572c59fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/1312153681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 0x5572c59fba60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0a88003ef0 tx=0x7f0a8802a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b614839122fc93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a7affd640 1 -- 172.21.15.78:0/1312153681 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8801e040 con 0x5572c59e3790 2024-09-11T01:09:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a7affd640 1 -- 172.21.15.78:0/1312153681 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a8800bc60 con 0x5572c59e3790 2024-09-11T01:09:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.193+0000 7f0a7affd640 1 -- 172.21.15.78:0/1312153681 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88028770 con 0x5572c59e3790 2024-09-11T01:09:50.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.194+0000 7f0a99b19740 1 -- 172.21.15.78:0/1312153681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 msgr2=0x5572c59fba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:50.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.194+0000 7f0a99b19740 1 --2- 172.21.15.78:0/1312153681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 0x5572c59fba60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0a88003ef0 tx=0x7f0a8802a040 comp rx=0 tx=0).stop 2024-09-11T01:09:50.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.194+0000 7f0a99b19740 1 -- 172.21.15.78:0/1312153681 shutdown_connections 2024-09-11T01:09:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.194+0000 7f0a99b19740 1 --2- 172.21.15.78:0/1312153681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59e3790 0x5572c59fba60 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.194+0000 7f0a99b19740 1 --2- 172.21.15.78:0/1312153681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59d7800 0x5572c59fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.194+0000 7f0a99b19740 1 --2- 172.21.15.78:0/1312153681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d6ee0 0x5572c59de1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.194+0000 7f0a99b19740 1 -- 172.21.15.78:0/1312153681 >> 172.21.15.78:0/1312153681 conn(0x5572c59d4d10 msgr2=0x5572c59da8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.194+0000 7f0a99b19740 1 -- 172.21.15.78:0/1312153681 shutdown_connections 2024-09-11T01:09:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.194+0000 7f0a99b19740 1 -- 172.21.15.78:0/1312153681 wait complete. 2024-09-11T01:09:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.194+0000 7f0a99b19740 1 -- start start 2024-09-11T01:09:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a99b19740 1 Processor -- start 2024-09-11T01:09:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a99b19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 0x5572c59ebf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a99b19740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d7800 0x5572c59e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a99b19740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59e3790 0x5572c59e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a99b19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a901ad7a0 con 0x5572c59d6ee0 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a99b19740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a901ad0d0 con 0x5572c59e3790 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a99b19740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a901aa350 con 0x5572c59d7800 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a99b19740 10 client.0 _ll_register_callbacks cb 0x7f0a80009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a95ce1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 0x5572c59ebf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a95ce1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 0x5572c59ebf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32980/0 (socket says 172.21.15.78:32980) 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a95ce1640 1 -- 172.21.15.78:0/264950603 learned_addr learned my addr 172.21.15.78:0/264950603 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59e3790 0x5572c59e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a94cdf640 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d7800 0x5572c59e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a95ce1640 1 -- 172.21.15.78:0/264950603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d7800 msgr2=0x5572c59e83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d7800 0x5572c59e83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a95ce1640 1 -- 172.21.15.78:0/264950603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59e3790 msgr2=0x5572c59e89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:50.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59e3790 0x5572c59e89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.195+0000 7f0a95ce1640 1 -- 172.21.15.78:0/264950603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c5a60730 con 0x5572c59d6ee0 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a95ce1640 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 0x5572c59ebf10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800bc30 tx=0x7f0a88003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3167e4b875aa93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a78ff9640 1 -- 172.21.15.78:0/264950603 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8801e070 con 0x5572c59d6ee0 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a78ff9640 1 -- 172.21.15.78:0/264950603 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f0a88039040 con 0x5572c59d6ee0 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a99b19740 1 -- 172.21.15.78:0/264950603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a8800bc60 con 0x5572c59d6ee0 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a78ff9640 1 -- 172.21.15.78:0/264950603 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0a88047050 con 0x5572c59d6ee0 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a99b19740 10 client.17607 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a99b19740 1 -- 172.21.15.78:0/264950603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5572c5a61860 con 0x5572c59d6ee0 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a78ff9640 1 -- 172.21.15.78:0/264950603 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a88004500 con 0x5572c59d6ee0 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a78ff9640 1 -- 172.21.15.78:0/264950603 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f0a88023040 con 0x5572c59d6ee0 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a99b19740 10 client.17607 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a99b19740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a99b19740 20 client.17607 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a99b19740 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572c5a61de0 0x5572c5a64200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:50.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a99b19740 4 client.17607 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:09:50.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.196+0000 7f0a99b19740 1 -- 172.21.15.78:0/264950603 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f0a8800bc60 con 0x5572c5a61de0 2024-09-11T01:09:50.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.197+0000 7f0a94cdf640 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572c5a61de0 0x5572c5a64200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:50.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.197+0000 7f0a78ff9640 10 client.17607 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:50.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.197+0000 7f0a94cdf640 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572c5a61de0 0x5572c5a64200 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89ad1166dbf1158a server_cookie=6774076dc758e083 in_seq=0 out_seq=0 2024-09-11T01:09:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.199+0000 7f0a78ff9640 1 -- 172.21.15.78:0/264950603 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f0a8800bc60 con 0x5572c5a61de0 2024-09-11T01:09:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.199+0000 7f0a78ff9640 10 client.17607 handle_command_reply: tid=0 2024-09-11T01:09:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.199+0000 7f0a99b19740 1 client.17607 shutdown 2024-09-11T01:09:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.199+0000 7f0a99b19740 1 -- 172.21.15.78:0/264950603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 msgr2=0x5572c59ebf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.199+0000 7f0a99b19740 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 0x5572c59ebf10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800bc30 tx=0x7f0a88003f50 comp rx=0 tx=0).stop 2024-09-11T01:09:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.199+0000 7f0a99b19740 1 -- 172.21.15.78:0/264950603 shutdown_connections 2024-09-11T01:09:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.199+0000 7f0a99b19740 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5572c59e3790 0x5572c59e89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.199+0000 7f0a99b19740 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572c59d7800 0x5572c59e83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.199+0000 7f0a99b19740 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5572c5a61de0 0x5572c5a64200 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.199+0000 7f0a99b19740 1 --2- 172.21.15.78:0/264950603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572c59d6ee0 0x5572c59ebf10 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a78ff9640 0 client.17607 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:50.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a99b19740 1 -- 172.21.15.78:0/264950603 >> 172.21.15.78:0/264950603 conn(0x5572c59d4d10 msgr2=0x5572c59d3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:50.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a99b19740 1 -- 172.21.15.78:0/264950603 shutdown_connections 2024-09-11T01:09:50.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a99b19740 1 -- 172.21.15.78:0/264950603 wait complete. 2024-09-11T01:09:50.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a99b19740 20 client.17607 trim_cache size 0 max 16384 2024-09-11T01:09:50.206 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:09:50.206 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:09:50.206 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "4de34141-1c94-4d41-a1fc-9ca074174c31", 2024-09-11T01:09:50.206 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:09:50.206 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:09:50.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a97744640 1 -- 172.21.15.78:0/3279387115 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a7c0b5450 msgr2=0x7f0a7c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:50.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a97744640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a7c0b5450 0x7f0a7c0b7910 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5572c59da060 tx=0x7f0a8c02a410 comp rx=0 tx=0).stop 2024-09-11T01:09:50.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a97744640 1 -- 172.21.15.78:0/3279387115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a90102dc0 msgr2=0x7f0a90199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:50.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a97744640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a90102dc0 0x7f0a90199400 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005940 tx=0x7f0a8c0040c0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a97744640 1 -- 172.21.15.78:0/3279387115 shutdown_connections 2024-09-11T01:09:50.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a97744640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a7c0b5450 0x7f0a7c0b7910 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a97744640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a90109150 0x7f0a9019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a97744640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a901087d0 0x7f0a90199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a97744640 1 --2- 172.21.15.78:0/3279387115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a90102dc0 0x7f0a90199400 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:50.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.200+0000 7f0a97744640 1 -- 172.21.15.78:0/3279387115 >> 172.21.15.78:0/3279387115 conn(0x7f0a900fe870 msgr2=0x7f0a900ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:50.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.201+0000 7f0a97744640 1 -- 172.21.15.78:0/3279387115 shutdown_connections 2024-09-11T01:09:50.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:50.201+0000 7f0a97744640 1 -- 172.21.15.78:0/3279387115 wait complete. 2024-09-11T01:09:50.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 727414 5.34 MB/sec execute 478 sec latency 43.880 ms 2024-09-11T01:09:50.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:50 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:50.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:50 smithi078 ceph-mon[31315]: pgmap v660: 97 pgs: 97 active+clean; 560 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-09-11T01:09:50.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:50 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:50.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:50 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:50.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:50 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:50.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:50 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:50.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:50 smithi110 ceph-mon[34712]: pgmap v660: 97 pgs: 97 active+clean; 560 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-09-11T01:09:50.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:50 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:50.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:50 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:50.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:50 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:50.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:50 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:50.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:50 smithi112 ceph-mon[35384]: pgmap v660: 97 pgs: 97 active+clean; 560 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-09-11T01:09:50.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:50 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:50.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:50 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:50.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:50 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:50.963 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:09:51.221 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:09:51.451 INFO:tasks.workunit.client.0.smithi078.stdout: 1 728883 5.34 MB/sec execute 479 sec latency 13.669 ms 2024-09-11T01:09:51.529 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:09:51.529 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.529 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:51.529 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:51.529 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:51.529 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:51.529 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.529 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.529 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.529 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:51.529 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:51.529 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.530 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.530 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:51.530 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:51.530 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.530 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:09:51.530 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.530 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.530 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:51 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:09:51.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:51.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:09:51.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:51 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:09:51.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:51.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:51.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:51.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:51 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:09:51.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.829+0000 7fe8615b2640 1 -- 172.21.15.78:0/544300656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85c101c90 msgr2=0x7fe85c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:51.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.829+0000 7fe8615b2640 1 --2- 172.21.15.78:0/544300656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85c101c90 0x7fe85c102070 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009900 tx=0x7fe84c02a410 comp rx=0 tx=0).stop 2024-09-11T01:09:51.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.830+0000 7fe8615b2640 1 -- 172.21.15.78:0/544300656 shutdown_connections 2024-09-11T01:09:51.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.830+0000 7fe8615b2640 1 --2- 172.21.15.78:0/544300656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe85c108020 0x7fe85c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:51.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.830+0000 7fe8615b2640 1 --2- 172.21.15.78:0/544300656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85c1076a0 0x7fe85c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:51.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.830+0000 7fe8615b2640 1 --2- 172.21.15.78:0/544300656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85c101c90 0x7fe85c102070 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:51.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.830+0000 7fe8615b2640 1 -- 172.21.15.78:0/544300656 >> 172.21.15.78:0/544300656 conn(0x7fe85c0fd700 msgr2=0x7fe85c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:51.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.830+0000 7fe8615b2640 1 -- 172.21.15.78:0/544300656 shutdown_connections 2024-09-11T01:09:51.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.830+0000 7fe8615b2640 1 -- 172.21.15.78:0/544300656 wait complete. 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.830+0000 7fe8615b2640 1 Processor -- start 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe8615b2640 1 -- start start 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe8615b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe85c101c90 0x7fe85c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe8615b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85c1076a0 0x7fe85c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe8615b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85c108020 0x7fe85c1a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe8615b2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c111ce0 con 0x7fe85c108020 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe8615b2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c111b60 con 0x7fe85c101c90 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe8615b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c111e60 con 0x7fe85c1076a0 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85c108020 0x7fe85c1a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe85c101c90 0x7fe85c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85c1076a0 0x7fe85c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe85c101c90 0x7fe85c19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59900/0 (socket says 172.21.15.78:59900) 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85c108020 0x7fe85c1a5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33010/0 (socket says 172.21.15.78:33010) 2024-09-11T01:09:51.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85c1076a0 0x7fe85c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35336/0 (socket says 172.21.15.78:35336) 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85affd640 1 -- 172.21.15.78:0/1556880035 learned_addr learned my addr 172.21.15.78:0/1556880035 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85a7fc640 1 -- 172.21.15.78:0/1556880035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe85c101c90 msgr2=0x7fe85c19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85a7fc640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe85c101c90 0x7fe85c19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85a7fc640 1 -- 172.21.15.78:0/1556880035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85c108020 msgr2=0x7fe85c1a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85a7fc640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85c108020 0x7fe85c1a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85a7fc640 1 -- 172.21.15.78:0/1556880035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85c1a5650 con 0x7fe85c1076a0 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.831+0000 7fe85affd640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe85c101c90 0x7fe85c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.832+0000 7fe85a7fc640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85c1076a0 0x7fe85c19a5d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe85000e9f0 tx=0x7fe85000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.832+0000 7fe843fff640 1 -- 172.21.15.78:0/1556880035 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85000ccc0 con 0x7fe85c1076a0 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.832+0000 7fe8615b2640 1 -- 172.21.15.78:0/1556880035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe85c1a5820 con 0x7fe85c1076a0 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.832+0000 7fe843fff640 1 -- 172.21.15.78:0/1556880035 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe850005020 con 0x7fe85c1076a0 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.832+0000 7fe843fff640 1 -- 172.21.15.78:0/1556880035 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850004a00 con 0x7fe85c1076a0 2024-09-11T01:09:51.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.832+0000 7fe8615b2640 1 -- 172.21.15.78:0/1556880035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe85c1a5d00 con 0x7fe85c1076a0 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.834+0000 7fe843fff640 1 -- 172.21.15.78:0/1556880035 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe850004ba0 con 0x7fe85c1076a0 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe863987740 1 Processor -- start 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe863987740 1 -- start start 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe863987740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 0x55824bdcf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe863987740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcfda0 0x55824bddb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe863987740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bddbd30 0x55824bdde120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe863987740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55824bd45e50 con 0x55824bdcf480 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe863987740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55824bd35280 con 0x55824bdcfda0 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe863987740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55824bd39090 con 0x55824bddbd30 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 0x55824bdcf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 0x55824bdcf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33022/0 (socket says 172.21.15.78:33022) 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe85b7fe640 1 -- 172.21.15.78:0/4202919429 learned_addr learned my addr 172.21.15.78:0/4202919429 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe85affd640 1 --2- 172.21.15.78:0/4202919429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcfda0 0x55824bddb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe85a7fc640 1 --2- 172.21.15.78:0/4202919429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bddbd30 0x55824bdde120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.835+0000 7fe843fff640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe8440b54d0 0x7fe8440b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.836+0000 7fe85affd640 1 -- 172.21.15.78:0/4202919429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bddbd30 msgr2=0x55824bdde120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.836+0000 7fe85affd640 1 --2- 172.21.15.78:0/4202919429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bddbd30 0x55824bdde120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.836+0000 7fe85affd640 1 -- 172.21.15.78:0/4202919429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 msgr2=0x55824bdcf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.836+0000 7fe85affd640 1 --2- 172.21.15.78:0/4202919429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 0x55824bdcf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.836+0000 7fe85affd640 1 -- 172.21.15.78:0/4202919429 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55824ba93650 con 0x55824bdcfda0 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.836+0000 7fe85a7fc640 1 --2- 172.21.15.78:0/4202919429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bddbd30 0x55824bdde120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.836+0000 7fe85b7fe640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe8440b54d0 0x7fe8440b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.836+0000 7fe843fff640 1 -- 172.21.15.78:0/1556880035 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7fe8500e5e00 con 0x7fe85c1076a0 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.836+0000 7fe85affd640 1 --2- 172.21.15.78:0/4202919429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcfda0 0x55824bddb7f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe84c004770 tx=0x7fe84c032ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c72dc612ff9a574a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.836+0000 7fe8417fa640 1 -- 172.21.15.78:0/4202919429 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c035040 con 0x55824bdcfda0 2024-09-11T01:09:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.836+0000 7fe8417fa640 1 -- 172.21.15.78:0/4202919429 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe84c0317a0 con 0x55824bdcfda0 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.836+0000 7fe8417fa640 1 -- 172.21.15.78:0/4202919429 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0047a0 con 0x55824bdcfda0 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.837+0000 7fe863987740 1 -- 172.21.15.78:0/4202919429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcfda0 msgr2=0x55824bddb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.837+0000 7fe85b7fe640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe8440b54d0 0x7fe8440b7990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe854009fd0 tx=0x7fe854004a40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.837+0000 7fe863987740 1 --2- 172.21.15.78:0/4202919429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcfda0 0x55824bddb7f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe84c004770 tx=0x7fe84c032ed0 comp rx=0 tx=0).stop 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.837+0000 7fe863987740 1 -- 172.21.15.78:0/4202919429 shutdown_connections 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.837+0000 7fe863987740 1 --2- 172.21.15.78:0/4202919429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bddbd30 0x55824bdde120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.837+0000 7fe863987740 1 --2- 172.21.15.78:0/4202919429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcfda0 0x55824bddb7f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.837+0000 7fe863987740 1 --2- 172.21.15.78:0/4202919429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 0x55824bdcf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.837+0000 7fe863987740 1 -- 172.21.15.78:0/4202919429 >> 172.21.15.78:0/4202919429 conn(0x55824bdcd2b0 msgr2=0x55824bdce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.837+0000 7fe863987740 1 -- 172.21.15.78:0/4202919429 shutdown_connections 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.837+0000 7fe863987740 1 -- 172.21.15.78:0/4202919429 wait complete. 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.837+0000 7fe863987740 1 -- start start 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.837+0000 7fe863987740 1 Processor -- start 2024-09-11T01:09:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe863987740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bdcf480 0x55824bddb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe863987740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcfda0 0x55824bdd89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe863987740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bddbd30 0x55824bdd8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe863987740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55824bb96700 con 0x55824bddbd30 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe863987740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55824bcd0110 con 0x55824bdcfda0 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe863987740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55824bde2d60 con 0x55824bdcf480 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe863987740 10 client.0 _ll_register_callbacks cb 0x55824bce9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bddbd30 0x55824bdd8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe85affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bdcf480 0x55824bddb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe85a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcfda0 0x55824bdd89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe85affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bdcf480 0x55824bddb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35350/0 (socket says 172.21.15.78:35350) 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe85affd640 1 -- 172.21.15.78:0/3715116439 learned_addr learned my addr 172.21.15.78:0/3715116439 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe85a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcfda0 0x55824bdd89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59924/0 (socket says 172.21.15.78:59924) 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe85affd640 1 -- 172.21.15.78:0/3715116439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcfda0 msgr2=0x55824bdd89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:51.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe85affd640 1 --2- 172.21.15.78:0/3715116439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcfda0 0x55824bdd89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe85affd640 1 -- 172.21.15.78:0/3715116439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bddbd30 msgr2=0x55824bdd8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe85affd640 1 --2- 172.21.15.78:0/3715116439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bddbd30 0x55824bdd8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe85affd640 1 -- 172.21.15.78:0/3715116439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55824bcfdc40 con 0x55824bdcf480 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe85affd640 1 --2- 172.21.15.78:0/3715116439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bdcf480 0x55824bddb560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0057c0 tx=0x7fe84c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=469fe0a9ba2efc89 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe83eff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe83eff5640 1 -- 172.21.15.78:0/3715116439 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c04e070 con 0x55824bdcf480 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe83eff5640 1 -- 172.21.15.78:0/3715116439 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7fe84c004250 con 0x55824bdcf480 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.838+0000 7fe863987740 1 -- 172.21.15.78:0/3715116439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe84c0317a0 con 0x55824bdcf480 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe83eff5640 1 -- 172.21.15.78:0/3715116439 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe84c05a050 con 0x55824bdcf480 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe863987740 10 client.26740 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe863987740 1 -- 172.21.15.78:0/3715116439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55824bdee5d0 con 0x55824bdcf480 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe83eff5640 1 -- 172.21.15.78:0/3715116439 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe84c0317a0 con 0x55824bdcf480 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe83eff5640 1 -- 172.21.15.78:0/3715116439 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fe84c0464e0 con 0x55824bdcf480 2024-09-11T01:09:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe863987740 10 client.26740 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe863987740 10 client.26740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe863987740 20 client.26740 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe863987740 1 --2- 172.21.15.78:0/3715116439 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55824bdef920 0x55824bdf1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe863987740 4 client.26740 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe863987740 1 -- 172.21.15.78:0/3715116439 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55824bd47da0 con 0x55824bdef920 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe85a7fc640 1 --2- 172.21.15.78:0/3715116439 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55824bdef920 0x55824bdf1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe83eff5640 10 client.26740 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.839+0000 7fe85a7fc640 1 --2- 172.21.15.78:0/3715116439 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55824bdef920 0x55824bdf1d10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916295aef91a3242 server_cookie=c59507145fb6a2f0 in_seq=0 out_seq=0 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.840+0000 7fe83eff5640 1 -- 172.21.15.78:0/3715116439 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55824bd47da0 con 0x55824bdef920 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.840+0000 7fe83eff5640 10 client.26740 handle_command_reply: tid=0 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.840+0000 7fe863987740 1 client.26740 shutdown 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.840+0000 7fe863987740 1 -- 172.21.15.78:0/3715116439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bdcf480 msgr2=0x55824bddb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.840+0000 7fe863987740 1 --2- 172.21.15.78:0/3715116439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bdcf480 0x55824bddb560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0057c0 tx=0x7fe84c009260 comp rx=0 tx=0).stop 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.841+0000 7fe863987740 1 -- 172.21.15.78:0/3715116439 shutdown_connections 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.841+0000 7fe863987740 1 --2- 172.21.15.78:0/3715116439 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55824bdef920 0x55824bdf1d10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.841+0000 7fe863987740 1 --2- 172.21.15.78:0/3715116439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bddbd30 0x55824bdd8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.841+0000 7fe863987740 1 --2- 172.21.15.78:0/3715116439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcfda0 0x55824bdd89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.841+0000 7fe83eff5640 0 client.26740 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.841+0000 7fe863987740 1 --2- 172.21.15.78:0/3715116439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bdcf480 0x55824bddb560 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.841+0000 7fe863987740 1 -- 172.21.15.78:0/3715116439 >> 172.21.15.78:0/3715116439 conn(0x55824bdcd2b0 msgr2=0x55824bdd2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.841+0000 7fe863987740 1 -- 172.21.15.78:0/3715116439 shutdown_connections 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.841+0000 7fe863987740 1 -- 172.21.15.78:0/3715116439 wait complete. 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.841+0000 7fe863987740 20 client.26740 trim_cache size 0 max 16384 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.859+0000 7fe863987740 1 Processor -- start 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.859+0000 7fe863987740 1 -- start start 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.859+0000 7fe863987740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 0x55824bdcea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.859+0000 7fe863987740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bdcfda0 0x55824bdd5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.859+0000 7fe863987740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bddbd30 0x55824bdd6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.859+0000 7fe863987740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1ac370 con 0x55824bdcf480 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.859+0000 7fe863987740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1adf70 con 0x55824bddbd30 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.859+0000 7fe863987740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1aca40 con 0x55824bdcfda0 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe85affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 0x55824bdcea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe85affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 0x55824bdcea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33042/0 (socket says 172.21.15.78:33042) 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe85affd640 1 -- 172.21.15.78:0/701381666 learned_addr learned my addr 172.21.15.78:0/701381666 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe85a7fc640 1 --2- 172.21.15.78:0/701381666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bdcfda0 0x55824bdd5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe85affd640 1 -- 172.21.15.78:0/701381666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bdcfda0 msgr2=0x55824bdd5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe85affd640 1 --2- 172.21.15.78:0/701381666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bdcfda0 0x55824bdd5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe85affd640 1 -- 172.21.15.78:0/701381666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bddbd30 msgr2=0x55824bdd6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe85b7fe640 1 --2- 172.21.15.78:0/701381666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bddbd30 0x55824bdd6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe85affd640 1 --2- 172.21.15.78:0/701381666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bddbd30 0x55824bdd6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe85affd640 1 -- 172.21.15.78:0/701381666 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55824bdcdd30 con 0x55824bdcf480 2024-09-11T01:09:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe85affd640 1 --2- 172.21.15.78:0/701381666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 0x55824bdcea10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe84c031a70 tx=0x7fe84c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b55251c38c3946 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe840ff9640 1 -- 172.21.15.78:0/701381666 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c063070 con 0x55824bdcf480 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe840ff9640 1 -- 172.21.15.78:0/701381666 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe84c004450 con 0x55824bdcf480 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.860+0000 7fe840ff9640 1 -- 172.21.15.78:0/701381666 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c04e070 con 0x55824bdcf480 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.861+0000 7fe863987740 1 -- 172.21.15.78:0/701381666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 msgr2=0x55824bdcea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.861+0000 7fe863987740 1 --2- 172.21.15.78:0/701381666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 0x55824bdcea10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe84c031a70 tx=0x7fe84c005520 comp rx=0 tx=0).stop 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.861+0000 7fe863987740 1 -- 172.21.15.78:0/701381666 shutdown_connections 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.861+0000 7fe863987740 1 --2- 172.21.15.78:0/701381666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bddbd30 0x55824bdd6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.861+0000 7fe863987740 1 --2- 172.21.15.78:0/701381666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bdcfda0 0x55824bdd5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.861+0000 7fe863987740 1 --2- 172.21.15.78:0/701381666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcf480 0x55824bdcea10 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.861+0000 7fe863987740 1 -- 172.21.15.78:0/701381666 >> 172.21.15.78:0/701381666 conn(0x55824bdcd2b0 msgr2=0x55824bdce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.861+0000 7fe863987740 1 -- 172.21.15.78:0/701381666 shutdown_connections 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.861+0000 7fe863987740 1 -- 172.21.15.78:0/701381666 wait complete. 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.861+0000 7fe863987740 1 -- start start 2024-09-11T01:09:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.861+0000 7fe863987740 1 Processor -- start 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe863987740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcf480 0x55824bde47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe863987740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcfda0 0x55824bde4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe863987740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bddbd30 0x55824be5b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe863987740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1ad7e0 con 0x55824bdcfda0 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe863987740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1ad110 con 0x55824bdcf480 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe863987740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1aa390 con 0x55824bddbd30 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe863987740 10 client.0 _ll_register_callbacks cb 0x7fe84c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe85a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcfda0 0x55824bde4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe85affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcf480 0x55824bde47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe85affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcf480 0x55824bde47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59946/0 (socket says 172.21.15.78:59946) 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe85affd640 1 -- 172.21.15.78:0/3023572456 learned_addr learned my addr 172.21.15.78:0/3023572456 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe85b7fe640 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bddbd30 0x55824be5b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe85b7fe640 1 -- 172.21.15.78:0/3023572456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcf480 msgr2=0x55824bde47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe85b7fe640 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcf480 0x55824bde47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe85b7fe640 1 -- 172.21.15.78:0/3023572456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcfda0 msgr2=0x55824bde4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe85b7fe640 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcfda0 0x55824bde4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe85b7fe640 1 -- 172.21.15.78:0/3023572456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55824be5bc00 con 0x55824bddbd30 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe85affd640 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcf480 0x55824bde47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe85b7fe640 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bddbd30 0x55824be5b510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8540044d0 tx=0x7fe854004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e4ae8105683369 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe83eff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.862+0000 7fe83eff5640 1 -- 172.21.15.78:0/3023572456 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85401e8b0 con 0x55824bddbd30 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe863987740 1 -- 172.21.15.78:0/3023572456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe84c004450 con 0x55824bddbd30 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe83eff5640 1 -- 172.21.15.78:0/3023572456 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7fe85401ea50 con 0x55824bddbd30 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe83eff5640 1 -- 172.21.15.78:0/3023572456 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe85401ec50 con 0x55824bddbd30 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe863987740 10 client.26746 fetch_fsmap learned FSMap version 48 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe863987740 1 -- 172.21.15.78:0/3023572456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55824be5cfa0 con 0x55824bddbd30 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe83eff5640 1 -- 172.21.15.78:0/3023572456 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe84c004450 con 0x55824bddbd30 2024-09-11T01:09:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe83eff5640 1 -- 172.21.15.78:0/3023572456 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fe85402c420 con 0x55824bddbd30 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe863987740 10 client.26746 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe863987740 10 client.26746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe863987740 20 client.26746 populate_metadata read hostname 'smithi078' 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe863987740 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55824be5d7c0 0x55824be5fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe863987740 4 client.26746 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe863987740 1 -- 172.21.15.78:0/3023572456 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55824bd43850 con 0x55824be5d7c0 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.863+0000 7fe85affd640 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55824be5d7c0 0x55824be5fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe85affd640 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55824be5d7c0 0x55824be5fbe0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20da923758010de5 server_cookie=a4421b5f4f203c23 in_seq=0 out_seq=0 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe83eff5640 10 client.26746 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe83eff5640 1 -- 172.21.15.78:0/3023572456 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x55824bd43850 con 0x55824be5d7c0 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe83eff5640 10 client.26746 handle_command_reply: tid=0 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe863987740 1 client.26746 shutdown 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe863987740 1 -- 172.21.15.78:0/3023572456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bddbd30 msgr2=0x55824be5b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe863987740 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bddbd30 0x55824be5b510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8540044d0 tx=0x7fe854004ab0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe863987740 1 -- 172.21.15.78:0/3023572456 shutdown_connections 2024-09-11T01:09:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe863987740 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55824bddbd30 0x55824be5b510 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe863987740 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55824be5d7c0 0x55824be5fbe0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe863987740 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55824bdcfda0 0x55824bde4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe83eff5640 0 client.26746 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:09:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe863987740 1 --2- 172.21.15.78:0/3023572456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55824bdcf480 0x55824bde47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe863987740 1 -- 172.21.15.78:0/3023572456 >> 172.21.15.78:0/3023572456 conn(0x55824bdcd2b0 msgr2=0x55824bdcbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe863987740 1 -- 172.21.15.78:0/3023572456 shutdown_connections 2024-09-11T01:09:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe863987740 1 -- 172.21.15.78:0/3023572456 wait complete. 2024-09-11T01:09:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.864+0000 7fe863987740 20 client.26746 trim_cache size 0 max 16384 2024-09-11T01:09:52.008 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:09:52.009 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:09:52.009 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:09:52.009 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:09:52.009 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.009 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "4de34141-1c94-4d41-a1fc-9ca074174c31", 2024-09-11T01:09:52.009 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:09:52.009 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:09:52.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.009 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:09:52.009 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:09:52.009 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:09:52.009 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:09:52.010 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.011 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.012 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.015 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.016 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:09:52.017 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:09:52.018 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:09:52.018 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:09:52.018 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:09:52.018 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:09:52.018 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:09:52.018 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:09:52.018 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:09:52.018 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:09:52.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.865+0000 7fe8615b2640 1 -- 172.21.15.78:0/1556880035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe8440b54d0 msgr2=0x7fe8440b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:52.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.865+0000 7fe8615b2640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe8440b54d0 0x7fe8440b7990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe854009fd0 tx=0x7fe854004a40 comp rx=0 tx=0).stop 2024-09-11T01:09:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.865+0000 7fe8615b2640 1 -- 172.21.15.78:0/1556880035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85c1076a0 msgr2=0x7fe85c19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:09:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.865+0000 7fe8615b2640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85c1076a0 0x7fe85c19a5d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe85000e9f0 tx=0x7fe85000eeb0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.865+0000 7fe8615b2640 1 -- 172.21.15.78:0/1556880035 shutdown_connections 2024-09-11T01:09:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.865+0000 7fe8615b2640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe8440b54d0 0x7fe8440b7990 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.865+0000 7fe8615b2640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85c108020 0x7fe85c1a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.865+0000 7fe8615b2640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85c1076a0 0x7fe85c19a5d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.865+0000 7fe8615b2640 1 --2- 172.21.15.78:0/1556880035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe85c101c90 0x7fe85c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:09:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.865+0000 7fe8615b2640 1 -- 172.21.15.78:0/1556880035 >> 172.21.15.78:0/1556880035 conn(0x7fe85c0fd700 msgr2=0x7fe85c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:09:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.865+0000 7fe8615b2640 1 -- 172.21.15.78:0/1556880035 shutdown_connections 2024-09-11T01:09:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:09:51.865+0000 7fe8615b2640 1 -- 172.21.15.78:0/1556880035 wait complete. 2024-09-11T01:09:52.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 730846 5.36 MB/sec execute 480 sec latency 44.033 ms 2024-09-11T01:09:52.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:52 smithi110 ceph-mon[34712]: pgmap v661: 97 pgs: 97 active+clean; 564 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 238 op/s 2024-09-11T01:09:52.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:52 smithi112 ceph-mon[35384]: pgmap v661: 97 pgs: 97 active+clean; 564 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 238 op/s 2024-09-11T01:09:52.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:52 smithi078 ceph-mon[31315]: pgmap v661: 97 pgs: 97 active+clean; 564 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 238 op/s 2024-09-11T01:09:52.800 INFO:tasks.cephfs.filesystem:scrub status for tag:4de34141-1c94-4d41-a1fc-9ca074174c31 - {'path': '/', 'tag': '4de34141-1c94-4d41-a1fc-9ca074174c31', 'options': 'recursive,force'} 2024-09-11T01:09:53.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 732369 5.36 MB/sec execute 481 sec latency 10.478 ms 2024-09-11T01:09:54.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 734505 5.38 MB/sec execute 482 sec latency 14.242 ms 2024-09-11T01:09:54.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:54 smithi110 ceph-mon[34712]: pgmap v662: 97 pgs: 97 active+clean; 571 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 294 op/s 2024-09-11T01:09:54.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:54 smithi112 ceph-mon[35384]: pgmap v662: 97 pgs: 97 active+clean; 571 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 294 op/s 2024-09-11T01:09:54.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:54 smithi078 ceph-mon[31315]: pgmap v662: 97 pgs: 97 active+clean; 571 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 294 op/s 2024-09-11T01:09:55.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 736110 5.38 MB/sec execute 483 sec latency 9.899 ms 2024-09-11T01:09:56.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 738396 5.40 MB/sec execute 484 sec latency 16.003 ms 2024-09-11T01:09:56.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:56 smithi110 ceph-mon[34712]: pgmap v663: 97 pgs: 97 active+clean; 574 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 281 op/s 2024-09-11T01:09:56.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:56 smithi112 ceph-mon[35384]: pgmap v663: 97 pgs: 97 active+clean; 574 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 281 op/s 2024-09-11T01:09:56.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:56 smithi078 ceph-mon[31315]: pgmap v663: 97 pgs: 97 active+clean; 574 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 281 op/s 2024-09-11T01:09:57.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 739586 5.40 MB/sec execute 485 sec latency 39.123 ms 2024-09-11T01:09:57.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:57 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:09:57.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:57 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:09:57.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:57 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:09:57.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:57 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:09:57.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:57 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:09:57.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:57 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:09:58.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 741579 5.41 MB/sec execute 486 sec latency 25.148 ms 2024-09-11T01:09:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:58 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:09:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:58 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:09:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:58 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:09:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:09:58 smithi110 ceph-mon[34712]: pgmap v664: 97 pgs: 97 active+clean; 575 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2024-09-11T01:09:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:58 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:09:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:58 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:09:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:58 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:09:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:09:58 smithi112 ceph-mon[35384]: pgmap v664: 97 pgs: 97 active+clean; 575 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2024-09-11T01:09:58.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:58 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:09:58.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:58 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:09:58.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:58 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:09:58.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:09:58 smithi078 ceph-mon[31315]: pgmap v664: 97 pgs: 97 active+clean; 575 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2024-09-11T01:09:59.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 743072 5.41 MB/sec execute 487 sec latency 10.388 ms 2024-09-11T01:10:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:00 smithi110 ceph-mon[34712]: overall HEALTH_OK 2024-09-11T01:10:00.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 745302 5.43 MB/sec execute 488 sec latency 15.878 ms 2024-09-11T01:10:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:00 smithi112 ceph-mon[35384]: overall HEALTH_OK 2024-09-11T01:10:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:00 smithi078 ceph-mon[31315]: overall HEALTH_OK 2024-09-11T01:10:01.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:01 smithi110 ceph-mon[34712]: pgmap v665: 97 pgs: 97 active+clean; 582 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-09-11T01:10:01.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 746685 5.43 MB/sec execute 489 sec latency 19.061 ms 2024-09-11T01:10:01.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:01 smithi112 ceph-mon[35384]: pgmap v665: 97 pgs: 97 active+clean; 582 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-09-11T01:10:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:01 smithi078 ceph-mon[31315]: pgmap v665: 97 pgs: 97 active+clean; 582 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-09-11T01:10:02.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 748131 5.43 MB/sec execute 490 sec latency 39.303 ms 2024-09-11T01:10:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:03 smithi110 ceph-mon[34712]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T01:10:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:03 smithi110 ceph-mon[34712]: pgmap v667: 97 pgs: 97 active+clean; 586 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2024-09-11T01:10:03.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 749681 5.45 MB/sec execute 491 sec latency 17.068 ms 2024-09-11T01:10:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:03 smithi112 ceph-mon[35384]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T01:10:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:03 smithi112 ceph-mon[35384]: pgmap v667: 97 pgs: 97 active+clean; 586 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2024-09-11T01:10:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:03 smithi078 ceph-mon[31315]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T01:10:03.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:03 smithi078 ceph-mon[31315]: pgmap v667: 97 pgs: 97 active+clean; 586 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2024-09-11T01:10:04.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 751616 5.45 MB/sec execute 492 sec latency 15.618 ms 2024-09-11T01:10:04.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:04 smithi110 ceph-mon[34712]: pgmap v668: 97 pgs: 97 active+clean; 580 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 256 op/s 2024-09-11T01:10:04.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:04 smithi112 ceph-mon[35384]: pgmap v668: 97 pgs: 97 active+clean; 580 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 256 op/s 2024-09-11T01:10:04.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:04 smithi078 ceph-mon[31315]: pgmap v668: 97 pgs: 97 active+clean; 580 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 256 op/s 2024-09-11T01:10:05.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 753213 5.46 MB/sec execute 493 sec latency 10.994 ms 2024-09-11T01:10:06.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 755123 5.47 MB/sec execute 494 sec latency 17.684 ms 2024-09-11T01:10:06.525 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:06 smithi112 ceph-mon[35384]: pgmap v669: 97 pgs: 97 active+clean; 577 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 324 op/s 2024-09-11T01:10:06.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:06 smithi110 ceph-mon[34712]: pgmap v669: 97 pgs: 97 active+clean; 577 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 324 op/s 2024-09-11T01:10:06.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:06 smithi078 ceph-mon[31315]: pgmap v669: 97 pgs: 97 active+clean; 577 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 324 op/s 2024-09-11T01:10:07.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 756681 5.48 MB/sec execute 495 sec latency 26.600 ms 2024-09-11T01:10:08.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 758021 5.48 MB/sec execute 496 sec latency 21.134 ms 2024-09-11T01:10:08.693 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:08 smithi110 ceph-mon[34712]: pgmap v670: 97 pgs: 97 active+clean; 583 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2024-09-11T01:10:08.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:08 smithi112 ceph-mon[35384]: pgmap v670: 97 pgs: 97 active+clean; 583 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2024-09-11T01:10:08.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:08 smithi078 ceph-mon[31315]: pgmap v670: 97 pgs: 97 active+clean; 583 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2024-09-11T01:10:09.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 760163 5.49 MB/sec execute 497 sec latency 17.509 ms 2024-09-11T01:10:09.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:09 smithi110 ceph-mon[34712]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T01:10:09.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:09 smithi112 ceph-mon[35384]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T01:10:09.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:09 smithi078 ceph-mon[31315]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T01:10:10.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 761692 5.50 MB/sec execute 498 sec latency 15.978 ms 2024-09-11T01:10:10.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:10 smithi110 ceph-mon[34712]: pgmap v672: 97 pgs: 97 active+clean; 584 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 310 op/s 2024-09-11T01:10:10.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:10 smithi112 ceph-mon[35384]: pgmap v672: 97 pgs: 97 active+clean; 584 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 310 op/s 2024-09-11T01:10:10.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:10 smithi078 ceph-mon[31315]: pgmap v672: 97 pgs: 97 active+clean; 584 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 310 op/s 2024-09-11T01:10:11.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 763708 5.51 MB/sec execute 499 sec latency 25.065 ms 2024-09-11T01:10:12.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 765077 5.51 MB/sec execute 500 sec latency 48.558 ms 2024-09-11T01:10:12.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:12 smithi110 ceph-mon[34712]: pgmap v673: 97 pgs: 97 active+clean; 589 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-09-11T01:10:12.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:12 smithi112 ceph-mon[35384]: pgmap v673: 97 pgs: 97 active+clean; 589 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-09-11T01:10:12.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:12 smithi078 ceph-mon[31315]: pgmap v673: 97 pgs: 97 active+clean; 589 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-09-11T01:10:13.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 767071 5.53 MB/sec execute 501 sec latency 20.330 ms 2024-09-11T01:10:13.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:10:13.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:13 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:10:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:13 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:10:13.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:13.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:13 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:14.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 768410 5.53 MB/sec execute 502 sec latency 15.980 ms 2024-09-11T01:10:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:14 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:10:14.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:14 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:10:14.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:14 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:14.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:14 smithi110 ceph-mon[34712]: pgmap v674: 97 pgs: 97 active+clean; 595 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 323 op/s 2024-09-11T01:10:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:14 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:10:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:14 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:10:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:14 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:14 smithi112 ceph-mon[35384]: pgmap v674: 97 pgs: 97 active+clean; 595 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 323 op/s 2024-09-11T01:10:14.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:14 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:10:14.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:14 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:10:14.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:14 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:10:14.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:14 smithi078 ceph-mon[31315]: pgmap v674: 97 pgs: 97 active+clean; 595 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 323 op/s 2024-09-11T01:10:15.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 770616 5.54 MB/sec execute 503 sec latency 15.656 ms 2024-09-11T01:10:16.452 INFO:tasks.workunit.client.0.smithi078.stdout: 1 771934 5.54 MB/sec execute 504 sec latency 14.082 ms 2024-09-11T01:10:16.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:16 smithi110 ceph-mon[34712]: pgmap v675: 97 pgs: 97 active+clean; 599 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-09-11T01:10:16.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:16 smithi112 ceph-mon[35384]: pgmap v675: 97 pgs: 97 active+clean; 599 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-09-11T01:10:16.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:16 smithi078 ceph-mon[31315]: pgmap v675: 97 pgs: 97 active+clean; 599 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-09-11T01:10:17.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 774028 5.56 MB/sec execute 505 sec latency 30.948 ms 2024-09-11T01:10:18.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 775236 5.55 MB/sec execute 506 sec latency 18.061 ms 2024-09-11T01:10:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:18 smithi110 ceph-mon[34712]: pgmap v676: 97 pgs: 97 active+clean; 605 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 278 op/s 2024-09-11T01:10:18.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:18 smithi112 ceph-mon[35384]: pgmap v676: 97 pgs: 97 active+clean; 605 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 278 op/s 2024-09-11T01:10:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:18 smithi078 ceph-mon[31315]: pgmap v676: 97 pgs: 97 active+clean; 605 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 278 op/s 2024-09-11T01:10:19.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 777387 5.57 MB/sec execute 507 sec latency 16.514 ms 2024-09-11T01:10:20.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 778745 5.57 MB/sec execute 508 sec latency 11.128 ms 2024-09-11T01:10:20.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:20 smithi110 ceph-mon[34712]: pgmap v677: 97 pgs: 97 active+clean; 607 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 265 op/s 2024-09-11T01:10:20.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:20 smithi112 ceph-mon[35384]: pgmap v677: 97 pgs: 97 active+clean; 607 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 265 op/s 2024-09-11T01:10:20.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:20 smithi078 ceph-mon[31315]: pgmap v677: 97 pgs: 97 active+clean; 607 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 265 op/s 2024-09-11T01:10:21.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 780960 5.58 MB/sec execute 509 sec latency 17.606 ms 2024-09-11T01:10:22.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 782232 5.58 MB/sec execute 510 sec latency 43.414 ms 2024-09-11T01:10:22.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:22 smithi110 ceph-mon[34712]: pgmap v678: 97 pgs: 97 active+clean; 615 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 287 op/s 2024-09-11T01:10:22.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:22 smithi112 ceph-mon[35384]: pgmap v678: 97 pgs: 97 active+clean; 615 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 287 op/s 2024-09-11T01:10:22.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:22 smithi078 ceph-mon[31315]: pgmap v678: 97 pgs: 97 active+clean; 615 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 287 op/s 2024-09-11T01:10:22.801 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:10:23.073 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:10:23.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 784136 5.59 MB/sec execute 511 sec latency 36.162 ms 2024-09-11T01:10:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.688+0000 7f6122e63640 1 -- 172.21.15.78:0/4057473927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f611c109170 msgr2=0x7f611c10f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.688+0000 7f6122e63640 1 --2- 172.21.15.78:0/4057473927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f611c109170 0x7f611c10f970 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6114009ca0 tx=0x7f611402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.688+0000 7f6122e63640 1 -- 172.21.15.78:0/4057473927 shutdown_connections 2024-09-11T01:10:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.688+0000 7f6122e63640 1 --2- 172.21.15.78:0/4057473927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f611c109170 0x7f611c10f970 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.688+0000 7f6122e63640 1 --2- 172.21.15.78:0/4057473927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f611c1087f0 0x7f611c108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.688+0000 7f6122e63640 1 --2- 172.21.15.78:0/4057473927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c102de0 0x7f611c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.689+0000 7f6122e63640 1 -- 172.21.15.78:0/4057473927 >> 172.21.15.78:0/4057473927 conn(0x7f611c0fe870 msgr2=0x7f611c100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.689+0000 7f6122e63640 1 -- 172.21.15.78:0/4057473927 shutdown_connections 2024-09-11T01:10:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.689+0000 7f6122e63640 1 -- 172.21.15.78:0/4057473927 wait complete. 2024-09-11T01:10:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.689+0000 7f6122e63640 1 Processor -- start 2024-09-11T01:10:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.689+0000 7f6122e63640 1 -- start start 2024-09-11T01:10:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f6122e63640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f611c102de0 0x7f611c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f6122e63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c1087f0 0x7f611c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f6122e63640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f611c109170 0x7f611c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f6122e63640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f611c112e30 con 0x7f611c102de0 2024-09-11T01:10:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f6122e63640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f611c112cb0 con 0x7f611c109170 2024-09-11T01:10:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f6122e63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f611c112fb0 con 0x7f611c1087f0 2024-09-11T01:10:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f6120bff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f611c102de0 0x7f611c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f611bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c1087f0 0x7f611c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f6120bff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f611c102de0 0x7f611c199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46534/0 (socket says 172.21.15.78:46534) 2024-09-11T01:10:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f6120bff640 1 -- 172.21.15.78:0/1903444564 learned_addr learned my addr 172.21.15.78:0/1903444564 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f6121400640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f611c109170 0x7f611c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f611bfff640 1 -- 172.21.15.78:0/1903444564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f611c109170 msgr2=0x7f611c19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f611bfff640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f611c109170 0x7f611c19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f611bfff640 1 -- 172.21.15.78:0/1903444564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f611c102de0 msgr2=0x7f611c199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f611bfff640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f611c102de0 0x7f611c199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f611bfff640 1 -- 172.21.15.78:0/1903444564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611c19f2d0 con 0x7f611c1087f0 2024-09-11T01:10:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f6120bff640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f611c102de0 0x7f611c199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f6121400640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f611c109170 0x7f611c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:10:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.690+0000 7f611bfff640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c1087f0 0x7f611c199760 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f610800e9f0 tx=0x7f610800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.691+0000 7f6119ffb640 1 -- 172.21.15.78:0/1903444564 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610800ccc0 con 0x7f611c1087f0 2024-09-11T01:10:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.691+0000 7f6122e63640 1 -- 172.21.15.78:0/1903444564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611c19f5c0 con 0x7f611c1087f0 2024-09-11T01:10:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.691+0000 7f6119ffb640 1 -- 172.21.15.78:0/1903444564 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61080108e0 con 0x7f611c1087f0 2024-09-11T01:10:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.691+0000 7f6119ffb640 1 -- 172.21.15.78:0/1903444564 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108018880 con 0x7f611c1087f0 2024-09-11T01:10:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.691+0000 7f6122e63640 1 -- 172.21.15.78:0/1903444564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f611c1a5e50 con 0x7f611c1087f0 2024-09-11T01:10:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.693+0000 7f6125238740 1 Processor -- start 2024-09-11T01:10:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.693+0000 7f6119ffb640 1 -- 172.21.15.78:0/1903444564 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6108010430 con 0x7f611c1087f0 2024-09-11T01:10:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.693+0000 7f6125238740 1 -- start start 2024-09-11T01:10:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6125238740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 0x55c96e504860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6125238740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e514c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6125238740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e517530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6125238740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c96e47ae50 con 0x55c96e504480 2024-09-11T01:10:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6125238740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c96e46a280 con 0x55c96e504da0 2024-09-11T01:10:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6125238740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c96e46e090 con 0x55c96e515140 2024-09-11T01:10:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6121400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 0x55c96e504860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6121400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 0x55c96e504860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46546/0 (socket says 172.21.15.78:46546) 2024-09-11T01:10:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6120bff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e514c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6121400640 1 -- 172.21.15.78:0/4173395423 learned_addr learned my addr 172.21.15.78:0/4173395423 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f611bfff640 1 --2- 172.21.15.78:0/4173395423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e517530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6120bff640 1 -- 172.21.15.78:0/4173395423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 msgr2=0x55c96e517530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6120bff640 1 --2- 172.21.15.78:0/4173395423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e517530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6120bff640 1 -- 172.21.15.78:0/4173395423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 msgr2=0x55c96e504860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6120bff640 1 --2- 172.21.15.78:0/4173395423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 0x55c96e504860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6120bff640 1 -- 172.21.15.78:0/4173395423 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96e1c8650 con 0x55c96e504da0 2024-09-11T01:10:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.694+0000 7f6121400640 1 --2- 172.21.15.78:0/4173395423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 0x55c96e504860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.695+0000 7f6119ffb640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f610c0b54d0 0x7f610c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.695+0000 7f6120bff640 1 --2- 172.21.15.78:0/4173395423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e514c00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f61100114f0 tx=0x7f61100119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd6c8939eb9bbbf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.695+0000 7f6121400640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f610c0b54d0 0x7f610c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.695+0000 7f6106ffd640 1 -- 172.21.15.78:0/4173395423 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611001e070 con 0x55c96e504da0 2024-09-11T01:10:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.695+0000 7f6119ffb640 1 -- 172.21.15.78:0/1903444564 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f61080e6360 con 0x7f611c1087f0 2024-09-11T01:10:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.695+0000 7f6106ffd640 1 -- 172.21.15.78:0/4173395423 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61100035f0 con 0x55c96e504da0 2024-09-11T01:10:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.695+0000 7f6106ffd640 1 -- 172.21.15.78:0/4173395423 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611000f820 con 0x55c96e504da0 2024-09-11T01:10:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 -- 172.21.15.78:0/4173395423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 msgr2=0x55c96e514c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6121400640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f610c0b54d0 0x7f610c0b7990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6114009a60 tx=0x7f61140098b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 --2- 172.21.15.78:0/4173395423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e514c00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f61100114f0 tx=0x7f61100119b0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 -- 172.21.15.78:0/4173395423 shutdown_connections 2024-09-11T01:10:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 --2- 172.21.15.78:0/4173395423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e517530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 --2- 172.21.15.78:0/4173395423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e514c00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 --2- 172.21.15.78:0/4173395423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 0x55c96e504860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 -- 172.21.15.78:0/4173395423 >> 172.21.15.78:0/4173395423 conn(0x55c96e5022b0 msgr2=0x55c96e503450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 -- 172.21.15.78:0/4173395423 shutdown_connections 2024-09-11T01:10:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 -- 172.21.15.78:0/4173395423 wait complete. 2024-09-11T01:10:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 -- start start 2024-09-11T01:10:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 Processor -- start 2024-09-11T01:10:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 0x55c96e5149b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.696+0000 7f6125238740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e511e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6125238740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e512380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6125238740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c96e2cb700 con 0x55c96e504480 2024-09-11T01:10:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6125238740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c96e405110 con 0x55c96e504da0 2024-09-11T01:10:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6125238740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c96e517df0 con 0x55c96e515140 2024-09-11T01:10:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6125238740 10 client.0 _ll_register_callbacks cb 0x55c96e41e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6120bff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 0x55c96e5149b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6121400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e512380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6120bff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 0x55c96e5149b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46554/0 (socket says 172.21.15.78:46554) 2024-09-11T01:10:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f611bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e511e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6121400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e512380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:40764/0 (socket says 172.21.15.78:40764) 2024-09-11T01:10:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f611bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e511e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36724/0 (socket says 172.21.15.78:36724) 2024-09-11T01:10:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6120bff640 1 -- 172.21.15.78:0/19826524 learned_addr learned my addr 172.21.15.78:0/19826524 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6120bff640 1 -- 172.21.15.78:0/19826524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 msgr2=0x55c96e512380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6120bff640 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e512380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6120bff640 1 -- 172.21.15.78:0/19826524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 msgr2=0x55c96e511e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6120bff640 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e511e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6120bff640 1 -- 172.21.15.78:0/19826524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96e432c40 con 0x55c96e504480 2024-09-11T01:10:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f611bfff640 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e511e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6121400640 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e512380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f6120bff640 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 0x55c96e5149b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6110003f00 tx=0x7f61100039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f128657c619fd4fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f60fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:10:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f60fbfff640 1 -- 172.21.15.78:0/19826524 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611001e070 con 0x55c96e504480 2024-09-11T01:10:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.697+0000 7f60fbfff640 1 -- 172.21.15.78:0/19826524 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f6110005030 con 0x55c96e504480 2024-09-11T01:10:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f6125238740 1 -- 172.21.15.78:0/19826524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61100035f0 con 0x55c96e504480 2024-09-11T01:10:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f60fbfff640 1 -- 172.21.15.78:0/19826524 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6110005c50 con 0x55c96e504480 2024-09-11T01:10:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f6125238740 10 client.17637 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f6125238740 1 -- 172.21.15.78:0/19826524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c96e5236a0 con 0x55c96e504480 2024-09-11T01:10:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f60fbfff640 1 -- 172.21.15.78:0/19826524 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61100035f0 con 0x55c96e504480 2024-09-11T01:10:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f60fbfff640 1 -- 172.21.15.78:0/19826524 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f611000f5c0 con 0x55c96e504480 2024-09-11T01:10:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f6125238740 10 client.17637 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f6125238740 10 client.17637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:10:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f6125238740 20 client.17637 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f6125238740 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c96e5249f0 0x55c96e526de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f6125238740 4 client.17637 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:10:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f6125238740 1 -- 172.21.15.78:0/19826524 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c96e47cda0 con 0x55c96e5249f0 2024-09-11T01:10:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.698+0000 7f611bfff640 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c96e5249f0 0x55c96e526de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.699+0000 7f60fbfff640 10 client.17637 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.699+0000 7f611bfff640 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c96e5249f0 0x55c96e526de0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab2b47aaa0586c8 server_cookie=165203c7eba4c85c in_seq=0 out_seq=0 2024-09-11T01:10:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.699+0000 7f60fbfff640 1 -- 172.21.15.78:0/19826524 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c96e47cda0 con 0x55c96e5249f0 2024-09-11T01:10:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.699+0000 7f60fbfff640 10 client.17637 handle_command_reply: tid=0 2024-09-11T01:10:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.699+0000 7f6125238740 1 client.17637 shutdown 2024-09-11T01:10:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.700+0000 7f6125238740 1 -- 172.21.15.78:0/19826524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 msgr2=0x55c96e5149b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.700+0000 7f6125238740 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 0x55c96e5149b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6110003f00 tx=0x7f61100039d0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.700+0000 7f6125238740 1 -- 172.21.15.78:0/19826524 shutdown_connections 2024-09-11T01:10:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.700+0000 7f6125238740 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e512380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.700+0000 7f6125238740 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e511e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.700+0000 7f6125238740 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c96e5249f0 0x55c96e526de0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.700+0000 7f6125238740 1 --2- 172.21.15.78:0/19826524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504480 0x55c96e5149b0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.700+0000 7f6125238740 1 -- 172.21.15.78:0/19826524 >> 172.21.15.78:0/19826524 conn(0x55c96e5022b0 msgr2=0x55c96e507de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.700+0000 7f60fbfff640 0 client.17637 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.700+0000 7f6125238740 1 -- 172.21.15.78:0/19826524 shutdown_connections 2024-09-11T01:10:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.700+0000 7f6125238740 1 -- 172.21.15.78:0/19826524 wait complete. 2024-09-11T01:10:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.700+0000 7f6125238740 20 client.17637 trim_cache size 0 max 16384 2024-09-11T01:10:23.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f6125238740 1 Processor -- start 2024-09-11T01:10:23.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f6125238740 1 -- start start 2024-09-11T01:10:23.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f6125238740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504480 0x55c96e521020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f6125238740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504da0 0x55c96e5033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f6125238740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e503920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f6125238740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f611c1ac480 con 0x55c96e504da0 2024-09-11T01:10:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f6125238740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f611c1ae080 con 0x55c96e504480 2024-09-11T01:10:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f6125238740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f611c1acb50 con 0x55c96e515140 2024-09-11T01:10:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f611bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504da0 0x55c96e5033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f6120bff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504480 0x55c96e521020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f6120bff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504480 0x55c96e521020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36738/0 (socket says 172.21.15.78:36738) 2024-09-11T01:10:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f6120bff640 1 -- 172.21.15.78:0/3219022888 learned_addr learned my addr 172.21.15.78:0/3219022888 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.720+0000 7f6121400640 1 --2- 172.21.15.78:0/3219022888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e503920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.721+0000 7f6120bff640 1 -- 172.21.15.78:0/3219022888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 msgr2=0x55c96e503920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.721+0000 7f6120bff640 1 --2- 172.21.15.78:0/3219022888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e503920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.721+0000 7f6120bff640 1 -- 172.21.15.78:0/3219022888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504da0 msgr2=0x55c96e5033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.721+0000 7f6120bff640 1 --2- 172.21.15.78:0/3219022888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504da0 0x55c96e5033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.721+0000 7f6120bff640 1 -- 172.21.15.78:0/3219022888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96e5126f0 con 0x55c96e504480 2024-09-11T01:10:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.721+0000 7f611bfff640 1 --2- 172.21.15.78:0/3219022888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504da0 0x55c96e5033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.721+0000 7f6120bff640 1 --2- 172.21.15.78:0/3219022888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504480 0x55c96e521020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f611002e040 tx=0x7f6110009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bd9c1d87b2536e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.721+0000 7f61067fc640 1 -- 172.21.15.78:0/3219022888 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611001e070 con 0x55c96e504480 2024-09-11T01:10:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.721+0000 7f61067fc640 1 -- 172.21.15.78:0/3219022888 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611001f040 con 0x55c96e504480 2024-09-11T01:10:23.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.721+0000 7f61067fc640 1 -- 172.21.15.78:0/3219022888 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110004600 con 0x55c96e504480 2024-09-11T01:10:23.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.722+0000 7f6125238740 1 -- 172.21.15.78:0/3219022888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504480 msgr2=0x55c96e521020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:23.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.722+0000 7f6125238740 1 --2- 172.21.15.78:0/3219022888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504480 0x55c96e521020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f611002e040 tx=0x7f6110009d90 comp rx=0 tx=0).stop 2024-09-11T01:10:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.722+0000 7f6125238740 1 -- 172.21.15.78:0/3219022888 shutdown_connections 2024-09-11T01:10:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.722+0000 7f6125238740 1 --2- 172.21.15.78:0/3219022888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e515140 0x55c96e503920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.722+0000 7f6125238740 1 --2- 172.21.15.78:0/3219022888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e504da0 0x55c96e5033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.722+0000 7f6125238740 1 --2- 172.21.15.78:0/3219022888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504480 0x55c96e521020 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.722+0000 7f6125238740 1 -- 172.21.15.78:0/3219022888 >> 172.21.15.78:0/3219022888 conn(0x55c96e5022b0 msgr2=0x55c96e516eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.722+0000 7f6125238740 1 -- 172.21.15.78:0/3219022888 shutdown_connections 2024-09-11T01:10:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.722+0000 7f6125238740 1 -- 172.21.15.78:0/3219022888 wait complete. 2024-09-11T01:10:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.722+0000 7f6125238740 1 -- start start 2024-09-11T01:10:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.722+0000 7f6125238740 1 Processor -- start 2024-09-11T01:10:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6125238740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e504480 0x55c96e51d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6125238740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e51d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6125238740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e515140 0x55c96e52bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6125238740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f611c1ad8f0 con 0x55c96e515140 2024-09-11T01:10:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6125238740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f611c1ad220 con 0x55c96e504da0 2024-09-11T01:10:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6125238740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f611c1aa480 con 0x55c96e504480 2024-09-11T01:10:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6125238740 10 client.0 _ll_register_callbacks cb 0x7f611001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f611bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e51d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6120bff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e504480 0x55c96e51d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f611bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e51d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36752/0 (socket says 172.21.15.78:36752) 2024-09-11T01:10:23.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f611bfff640 1 -- 172.21.15.78:0/801360356 learned_addr learned my addr 172.21.15.78:0/801360356 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:23.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6121400640 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e515140 0x55c96e52bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6120bff640 1 -- 172.21.15.78:0/801360356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 msgr2=0x55c96e51d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:23.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6120bff640 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e51d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6120bff640 1 -- 172.21.15.78:0/801360356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e515140 msgr2=0x55c96e52bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:23.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6120bff640 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e515140 0x55c96e52bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f6120bff640 1 -- 172.21.15.78:0/801360356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96e52c590 con 0x55c96e504480 2024-09-11T01:10:23.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.723+0000 7f611bfff640 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e51d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:23.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f6120bff640 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e504480 0x55c96e51d310 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f611002e040 tx=0x7f611000fae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b66f114a11f6a5b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:23.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f60fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:10:23.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f60fbfff640 1 -- 172.21.15.78:0/801360356 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611001e070 con 0x55c96e504480 2024-09-11T01:10:23.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f6125238740 1 -- 172.21.15.78:0/801360356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f611001f040 con 0x55c96e504480 2024-09-11T01:10:23.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f60fbfff640 1 -- 172.21.15.78:0/801360356 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f611000fe10 con 0x55c96e504480 2024-09-11T01:10:23.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f60fbfff640 1 -- 172.21.15.78:0/801360356 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f61100035f0 con 0x55c96e504480 2024-09-11T01:10:23.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f6125238740 10 client.26764 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:23.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f6125238740 1 -- 172.21.15.78:0/801360356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c96e50a7b0 con 0x55c96e504480 2024-09-11T01:10:23.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f60fbfff640 1 -- 172.21.15.78:0/801360356 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611001f040 con 0x55c96e504480 2024-09-11T01:10:23.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f60fbfff640 1 -- 172.21.15.78:0/801360356 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f6110044030 con 0x55c96e504480 2024-09-11T01:10:23.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f6125238740 10 client.26764 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:23.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f6125238740 10 client.26764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:10:23.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.724+0000 7f6125238740 20 client.26764 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:23.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.725+0000 7f6125238740 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c96e50afd0 0x55c96e50d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:23.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.725+0000 7f6125238740 4 client.26764 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:10:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.725+0000 7f6125238740 1 -- 172.21.15.78:0/801360356 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c96e478850 con 0x55c96e50afd0 2024-09-11T01:10:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.725+0000 7f611bfff640 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c96e50afd0 0x55c96e50d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.725+0000 7f60fbfff640 10 client.26764 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.725+0000 7f611bfff640 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c96e50afd0 0x55c96e50d3f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69f8ff548ca3c0df server_cookie=3b40f52ede3eacaa in_seq=0 out_seq=0 2024-09-11T01:10:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.725+0000 7f60fbfff640 1 -- 172.21.15.78:0/801360356 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x55c96e478850 con 0x55c96e50afd0 2024-09-11T01:10:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.725+0000 7f60fbfff640 10 client.26764 handle_command_reply: tid=0 2024-09-11T01:10:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.725+0000 7f6125238740 1 client.26764 shutdown 2024-09-11T01:10:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.726+0000 7f6125238740 1 -- 172.21.15.78:0/801360356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e504480 msgr2=0x55c96e51d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.726+0000 7f6125238740 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e504480 0x55c96e51d310 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f611002e040 tx=0x7f611000fae0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.726+0000 7f6125238740 1 -- 172.21.15.78:0/801360356 shutdown_connections 2024-09-11T01:10:23.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.726+0000 7f6125238740 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c96e50afd0 0x55c96e50d3f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.726+0000 7f6125238740 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c96e515140 0x55c96e52bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.726+0000 7f6125238740 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c96e504da0 0x55c96e51d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.726+0000 7f60fbfff640 0 client.26764 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:23.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.726+0000 7f6125238740 1 --2- 172.21.15.78:0/801360356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c96e504480 0x55c96e51d310 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.726+0000 7f6125238740 1 -- 172.21.15.78:0/801360356 >> 172.21.15.78:0/801360356 conn(0x55c96e5022b0 msgr2=0x55c96e5031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:23.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.726+0000 7f6125238740 1 -- 172.21.15.78:0/801360356 shutdown_connections 2024-09-11T01:10:23.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.726+0000 7f6125238740 1 -- 172.21.15.78:0/801360356 wait complete. 2024-09-11T01:10:23.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.726+0000 7f6125238740 20 client.26764 trim_cache size 0 max 16384 2024-09-11T01:10:23.738 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:10:23.738 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:10:23.738 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:10:23.738 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:10:23.738 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:10:23.738 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:10:23.739 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.739 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.739 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.739 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.739 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.739 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:10:23.739 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:10:23.739 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.739 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.739 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.740 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.740 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.740 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:10:23.740 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:10:23.740 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.740 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.740 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.740 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.740 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.741 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:10:23.741 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:10:23.741 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.741 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.741 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.741 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.741 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.741 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:10:23.741 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:10:23.741 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.742 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:10:23.742 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:10:23.742 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.743 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:10:23.743 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:10:23.743 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.743 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.743 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.743 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.743 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.743 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:10:23.743 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:10:23.743 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:10:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:10:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.745 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:10:23.745 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:10:23.745 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.745 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.745 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.745 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.745 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.745 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:10:23.745 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.747 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:10:23.747 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:10:23.747 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.748 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:10:23.748 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:10:23.748 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:23.748 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:10:23.748 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:10:23.749 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:23.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:23.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:23.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:23.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:23.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:10:23.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:10:23.749 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:10:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.727+0000 7f6122e63640 1 -- 172.21.15.78:0/1903444564 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f610c0b54d0 msgr2=0x7f610c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.727+0000 7f6122e63640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f610c0b54d0 0x7f610c0b7990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6114009a60 tx=0x7f61140098b0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.727+0000 7f6122e63640 1 -- 172.21.15.78:0/1903444564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c1087f0 msgr2=0x7f611c199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.727+0000 7f6122e63640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c1087f0 0x7f611c199760 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f610800e9f0 tx=0x7f610800eeb0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.727+0000 7f6122e63640 1 -- 172.21.15.78:0/1903444564 shutdown_connections 2024-09-11T01:10:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.727+0000 7f6122e63640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f610c0b54d0 0x7f610c0b7990 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.727+0000 7f6122e63640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f611c109170 0x7f611c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.727+0000 7f6122e63640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c1087f0 0x7f611c199760 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.727+0000 7f6122e63640 1 --2- 172.21.15.78:0/1903444564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f611c102de0 0x7f611c199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:23.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.727+0000 7f6122e63640 1 -- 172.21.15.78:0/1903444564 >> 172.21.15.78:0/1903444564 conn(0x7f611c0fe870 msgr2=0x7f611c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:23.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.727+0000 7f6122e63640 1 -- 172.21.15.78:0/1903444564 shutdown_connections 2024-09-11T01:10:23.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:23.727+0000 7f6122e63640 1 -- 172.21.15.78:0/1903444564 wait complete. 2024-09-11T01:10:24.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 785555 5.60 MB/sec execute 512 sec latency 21.039 ms 2024-09-11T01:10:24.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:24 smithi078 ceph-mon[31315]: pgmap v679: 97 pgs: 97 active+clean; 617 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 300 op/s 2024-09-11T01:10:24.589 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:10:24.589 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:10:24.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:24 smithi110 ceph-mon[34712]: pgmap v679: 97 pgs: 97 active+clean; 617 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 300 op/s 2024-09-11T01:10:24.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:24 smithi112 ceph-mon[35384]: pgmap v679: 97 pgs: 97 active+clean; 617 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 300 op/s 2024-09-11T01:10:24.870 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:10:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.353+0000 7fb0acd3d640 1 -- 172.21.15.78:0/2958640966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a81087d0 msgr2=0x7fb0a8108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.353+0000 7fb0acd3d640 1 --2- 172.21.15.78:0/2958640966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a81087d0 0x7fb0a8108c10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009900 tx=0x7fb09c02a410 comp rx=0 tx=0).stop 2024-09-11T01:10:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.353+0000 7fb0acd3d640 1 -- 172.21.15.78:0/2958640966 shutdown_connections 2024-09-11T01:10:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.353+0000 7fb0acd3d640 1 --2- 172.21.15.78:0/2958640966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a8109150 0x7fb0a810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.353+0000 7fb0acd3d640 1 --2- 172.21.15.78:0/2958640966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a81087d0 0x7fb0a8108c10 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.353+0000 7fb0acd3d640 1 --2- 172.21.15.78:0/2958640966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a8102dc0 0x7fb0a81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.353+0000 7fb0acd3d640 1 -- 172.21.15.78:0/2958640966 >> 172.21.15.78:0/2958640966 conn(0x7fb0a80fe830 msgr2=0x7fb0a8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.353+0000 7fb0acd3d640 1 -- 172.21.15.78:0/2958640966 shutdown_connections 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.353+0000 7fb0acd3d640 1 -- 172.21.15.78:0/2958640966 wait complete. 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.354+0000 7fb0acd3d640 1 Processor -- start 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.354+0000 7fb0acd3d640 1 -- start start 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.354+0000 7fb0acd3d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a8102dc0 0x7fb0a8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.354+0000 7fb0acd3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a81087d0 0x7fb0a8199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.354+0000 7fb0acd3d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a8109150 0x7fb0a819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.354+0000 7fb0acd3d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8112dd0 con 0x7fb0a81087d0 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.354+0000 7fb0acd3d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8112c50 con 0x7fb0a8102dc0 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.354+0000 7fb0acd3d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8112f50 con 0x7fb0a8109150 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a81087d0 0x7fb0a8199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a8102dc0 0x7fb0a8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a81087d0 0x7fb0a8199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46604/0 (socket says 172.21.15.78:46604) 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a5d9b640 1 -- 172.21.15.78:0/706547406 learned_addr learned my addr 172.21.15.78:0/706547406 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a6d9d640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a8109150 0x7fb0a819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a5d9b640 1 -- 172.21.15.78:0/706547406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a8109150 msgr2=0x7fb0a819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a5d9b640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a8109150 0x7fb0a819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a5d9b640 1 -- 172.21.15.78:0/706547406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a8102dc0 msgr2=0x7fb0a8199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a5d9b640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a8102dc0 0x7fb0a8199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a5d9b640 1 -- 172.21.15.78:0/706547406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a819f3d0 con 0x7fb0a81087d0 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a6d9d640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a8109150 0x7fb0a819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a659c640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a8102dc0 0x7fb0a8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0a5d9b640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a81087d0 0x7fb0a8199860 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb09c005940 tx=0x7fb09c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0977fe640 1 -- 172.21.15.78:0/706547406 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c03f030 con 0x7fb0a81087d0 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0acd3d640 1 -- 172.21.15.78:0/706547406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a819f660 con 0x7fb0a81087d0 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0977fe640 1 -- 172.21.15.78:0/706547406 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb09c0044c0 con 0x7fb0a81087d0 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0977fe640 1 -- 172.21.15.78:0/706547406 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c035070 con 0x7fb0a81087d0 2024-09-11T01:10:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.355+0000 7fb0acd3d640 1 -- 172.21.15.78:0/706547406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a81a5d80 con 0x7fb0a81087d0 2024-09-11T01:10:25.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.356+0000 7fb0acd3d640 1 -- 172.21.15.78:0/706547406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a81031a0 con 0x7fb0a81087d0 2024-09-11T01:10:25.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.358+0000 7fb0977fe640 1 -- 172.21.15.78:0/706547406 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb09c03b9d0 con 0x7fb0a81087d0 2024-09-11T01:10:25.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.359+0000 7fb0977fe640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0900b54d0 0x7fb0900b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:25.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.359+0000 7fb0a659c640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0900b54d0 0x7fb0900b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:25.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.359+0000 7fb0977fe640 1 -- 172.21.15.78:0/706547406 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7fb09c106870 con 0x7fb0a81087d0 2024-09-11T01:10:25.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.359+0000 7fb0a659c640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0900b54d0 0x7fb0900b7990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb098009fc0 tx=0x7fb098005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:25.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.362+0000 7fb0977fe640 1 -- 172.21.15.78:0/706547406 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb09c0ccd80 con 0x7fb0a81087d0 2024-09-11T01:10:25.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 787459 5.61 MB/sec execute 513 sec latency 35.550 ms 2024-09-11T01:10:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.598+0000 7fb0acd3d640 1 -- 172.21.15.78:0/706547406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0a810af30 con 0x7fb0a81087d0 2024-09-11T01:10:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.599+0000 7fb0977fe640 1 -- 172.21.15.78:0/706547406 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11986 (secure 0 0 0) 0x7fb09c0d17b0 con 0x7fb0a81087d0 2024-09-11T01:10:25.601 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:10:25.601 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":48,"btime":"2024-09-11T01:09:37:269802+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:09:37.112141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:10:25.601 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 48 2024-09-11T01:10:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.602+0000 7fb0acd3d640 1 -- 172.21.15.78:0/706547406 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0900b54d0 msgr2=0x7fb0900b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.602+0000 7fb0acd3d640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0900b54d0 0x7fb0900b7990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb098009fc0 tx=0x7fb098005f90 comp rx=0 tx=0).stop 2024-09-11T01:10:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.602+0000 7fb0acd3d640 1 -- 172.21.15.78:0/706547406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a81087d0 msgr2=0x7fb0a8199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.602+0000 7fb0acd3d640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a81087d0 0x7fb0a8199860 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb09c005940 tx=0x7fb09c0040c0 comp rx=0 tx=0).stop 2024-09-11T01:10:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.602+0000 7fb0acd3d640 1 -- 172.21.15.78:0/706547406 shutdown_connections 2024-09-11T01:10:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.602+0000 7fb0acd3d640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0900b54d0 0x7fb0900b7990 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.602+0000 7fb0acd3d640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a8109150 0x7fb0a819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.602+0000 7fb0acd3d640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a81087d0 0x7fb0a8199860 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.602+0000 7fb0acd3d640 1 --2- 172.21.15.78:0/706547406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a8102dc0 0x7fb0a8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.602+0000 7fb0acd3d640 1 -- 172.21.15.78:0/706547406 >> 172.21.15.78:0/706547406 conn(0x7fb0a80fe830 msgr2=0x7fb0a8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.602+0000 7fb0acd3d640 1 -- 172.21.15.78:0/706547406 shutdown_connections 2024-09-11T01:10:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:25.602+0000 7fb0acd3d640 1 -- 172.21.15.78:0/706547406 wait complete. 2024-09-11T01:10:25.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:25 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/706547406' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:10:25.963 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:25 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/706547406' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:10:25.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:25 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/706547406' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:10:26.292 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:10:26.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 788952 5.61 MB/sec execute 514 sec latency 26.892 ms 2024-09-11T01:10:26.577 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:10:26.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:26 smithi110 ceph-mon[34712]: pgmap v680: 97 pgs: 97 active+clean; 624 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 303 op/s 2024-09-11T01:10:26.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:26 smithi112 ceph-mon[35384]: pgmap v680: 97 pgs: 97 active+clean; 624 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 303 op/s 2024-09-11T01:10:26.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:26 smithi078 ceph-mon[31315]: pgmap v680: 97 pgs: 97 active+clean; 624 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 303 op/s 2024-09-11T01:10:27.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.394+0000 7f402f7ba640 1 -- 172.21.15.78:0/3013614477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4028101830 msgr2=0x7f4028101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.395+0000 7f402f7ba640 1 --2- 172.21.15.78:0/3013614477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4028101830 0x7f4028101c10 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4024009900 tx=0x7f402402a480 comp rx=0 tx=0).stop 2024-09-11T01:10:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.395+0000 7f402f7ba640 1 -- 172.21.15.78:0/3013614477 shutdown_connections 2024-09-11T01:10:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.395+0000 7f402f7ba640 1 --2- 172.21.15.78:0/3013614477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f402810f3e0 0x7f40281117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.395+0000 7f402f7ba640 1 --2- 172.21.15.78:0/3013614477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4028102150 0x7f402810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.395+0000 7f402f7ba640 1 --2- 172.21.15.78:0/3013614477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4028101830 0x7f4028101c10 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.395+0000 7f402f7ba640 1 -- 172.21.15.78:0/3013614477 >> 172.21.15.78:0/3013614477 conn(0x7f40280fd700 msgr2=0x7f40280ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.395+0000 7f402f7ba640 1 -- 172.21.15.78:0/3013614477 shutdown_connections 2024-09-11T01:10:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.395+0000 7f402f7ba640 1 -- 172.21.15.78:0/3013614477 wait complete. 2024-09-11T01:10:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.395+0000 7f402f7ba640 1 Processor -- start 2024-09-11T01:10:27.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.395+0000 7f402f7ba640 1 -- start start 2024-09-11T01:10:27.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402f7ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4028101830 0x7f402819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402f7ba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4028102150 0x7f402819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402f7ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f402810f3e0 0x7f40281a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402f7ba640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4028114f30 con 0x7f402810f3e0 2024-09-11T01:10:27.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402f7ba640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4028114db0 con 0x7f4028102150 2024-09-11T01:10:27.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402f7ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40281150b0 con 0x7f4028101830 2024-09-11T01:10:27.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402dd57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f402810f3e0 0x7f40281a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402dd57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f402810f3e0 0x7f40281a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46620/0 (socket says 172.21.15.78:46620) 2024-09-11T01:10:27.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402dd57640 1 -- 172.21.15.78:0/2763225293 learned_addr learned my addr 172.21.15.78:0/2763225293 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:27.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402cd55640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4028102150 0x7f402819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402d556640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4028101830 0x7f402819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402dd57640 1 -- 172.21.15.78:0/2763225293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4028101830 msgr2=0x7f402819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:27.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402dd57640 1 --2- 172.21.15.78:0/2763225293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4028101830 0x7f402819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402dd57640 1 -- 172.21.15.78:0/2763225293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4028102150 msgr2=0x7f402819ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:27.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402dd57640 1 --2- 172.21.15.78:0/2763225293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4028102150 0x7f402819ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402dd57640 1 -- 172.21.15.78:0/2763225293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40281a1590 con 0x7f402810f3e0 2024-09-11T01:10:27.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402dd57640 1 --2- 172.21.15.78:0/2763225293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f402810f3e0 0x7f40281a0dc0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f402000eb10 tx=0x7f402000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:27.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f401e7fc640 1 -- 172.21.15.78:0/2763225293 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020004b20 con 0x7f402810f3e0 2024-09-11T01:10:27.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f402f7ba640 1 -- 172.21.15.78:0/2763225293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40281a1880 con 0x7f402810f3e0 2024-09-11T01:10:27.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f401e7fc640 1 -- 172.21.15.78:0/2763225293 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4020004cc0 con 0x7f402810f3e0 2024-09-11T01:10:27.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.396+0000 7f401e7fc640 1 -- 172.21.15.78:0/2763225293 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020010670 con 0x7f402810f3e0 2024-09-11T01:10:27.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.397+0000 7f402f7ba640 1 -- 172.21.15.78:0/2763225293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40281a7f40 con 0x7f402810f3e0 2024-09-11T01:10:27.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.399+0000 7f401e7fc640 1 -- 172.21.15.78:0/2763225293 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4020005020 con 0x7f402810f3e0 2024-09-11T01:10:27.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.399+0000 7f4031b8f740 1 Processor -- start 2024-09-11T01:10:27.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.399+0000 7f4031b8f740 1 -- start start 2024-09-11T01:10:27.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.399+0000 7f4031b8f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11e4d0 0x55f0ce11e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.399+0000 7f4031b8f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11edf0 0x55f0ce12a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.399+0000 7f4031b8f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 0x55f0ce12d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.399+0000 7f4031b8f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f0ce094ea0 con 0x55f0ce12ad80 2024-09-11T01:10:27.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.399+0000 7f4031b8f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f0ce0842d0 con 0x55f0ce11edf0 2024-09-11T01:10:27.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.399+0000 7f4031b8f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f0ce0880e0 con 0x55f0ce11e4d0 2024-09-11T01:10:27.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f402cd55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11e4d0 0x55f0ce11e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f402cd55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 0x55f0ce12d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f402cd55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 0x55f0ce12d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46628/0 (socket says 172.21.15.78:46628) 2024-09-11T01:10:27.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f402d556640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11edf0 0x55f0ce12a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f402cd55640 1 -- 172.21.15.78:0/3482960956 learned_addr learned my addr 172.21.15.78:0/3482960956 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:27.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f402d556640 1 -- 172.21.15.78:0/3482960956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11e4d0 msgr2=0x55f0ce11e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:27.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f402d556640 1 --2- 172.21.15.78:0/3482960956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11e4d0 0x55f0ce11e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f402d556640 1 -- 172.21.15.78:0/3482960956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 msgr2=0x55f0ce12d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:27.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f402d556640 1 --2- 172.21.15.78:0/3482960956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 0x55f0ce12d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f402d556640 1 -- 172.21.15.78:0/3482960956 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0cdde2650 con 0x55f0ce11edf0 2024-09-11T01:10:27.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f401e7fc640 1 --2- 172.21.15.78:0/2763225293 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f40140b5450 0x7f40140b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f402d556640 1 --2- 172.21.15.78:0/3482960956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11edf0 0x55f0ce12a840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4024009230 tx=0x7f4024032bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=322868d62d54869b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:27.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f402cd55640 1 --2- 172.21.15.78:0/2763225293 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f40140b5450 0x7f40140b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f401e7fc640 1 -- 172.21.15.78:0/2763225293 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f40200e6530 con 0x7f402810f3e0 2024-09-11T01:10:27.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f40137fe640 1 -- 172.21.15.78:0/3482960956 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024040070 con 0x55f0ce11edf0 2024-09-11T01:10:27.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f40137fe640 1 -- 172.21.15.78:0/3482960956 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4024004030 con 0x55f0ce11edf0 2024-09-11T01:10:27.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.400+0000 7f40137fe640 1 -- 172.21.15.78:0/3482960956 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024035040 con 0x55f0ce11edf0 2024-09-11T01:10:27.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.401+0000 7f4031b8f740 1 -- 172.21.15.78:0/3482960956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11edf0 msgr2=0x55f0ce12a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:27.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.401+0000 7f402cd55640 1 --2- 172.21.15.78:0/2763225293 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f40140b5450 0x7f40140b7910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f401800bfd0 tx=0x7f4018004730 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:27.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.401+0000 7f4031b8f740 1 --2- 172.21.15.78:0/3482960956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11edf0 0x55f0ce12a840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4024009230 tx=0x7f4024032bd0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.401+0000 7f4031b8f740 1 -- 172.21.15.78:0/3482960956 shutdown_connections 2024-09-11T01:10:27.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.401+0000 7f4031b8f740 1 --2- 172.21.15.78:0/3482960956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 0x55f0ce12d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.401+0000 7f4031b8f740 1 --2- 172.21.15.78:0/3482960956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11edf0 0x55f0ce12a840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.401+0000 7f4031b8f740 1 --2- 172.21.15.78:0/3482960956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11e4d0 0x55f0ce11e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.401+0000 7f4031b8f740 1 -- 172.21.15.78:0/3482960956 >> 172.21.15.78:0/3482960956 conn(0x55f0ce11c300 msgr2=0x55f0ce11d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:27.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.401+0000 7f4031b8f740 1 -- 172.21.15.78:0/3482960956 shutdown_connections 2024-09-11T01:10:27.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.401+0000 7f4031b8f740 1 -- 172.21.15.78:0/3482960956 wait complete. 2024-09-11T01:10:27.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f4031b8f740 1 -- start start 2024-09-11T01:10:27.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f4031b8f740 1 Processor -- start 2024-09-11T01:10:27.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f4031b8f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11e4d0 0x55f0ce12a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f4031b8f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11edf0 0x55f0ce127ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f4031b8f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce12ad80 0x55f0ce128110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f4031b8f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f0cdee5700 con 0x55f0ce11e4d0 2024-09-11T01:10:27.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f4031b8f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f0ce01f160 con 0x55f0ce11edf0 2024-09-11T01:10:27.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f4031b8f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f0ce12d9a0 con 0x55f0ce12ad80 2024-09-11T01:10:27.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f4031b8f740 10 client.0 _ll_register_callbacks cb 0x55f0ce0389e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:27.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f402d556640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11e4d0 0x55f0ce12a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f402d556640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11e4d0 0x55f0ce12a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46630/0 (socket says 172.21.15.78:46630) 2024-09-11T01:10:27.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f402d556640 1 -- 172.21.15.78:0/609535501 learned_addr learned my addr 172.21.15.78:0/609535501 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:27.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f402cd55640 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11edf0 0x55f0ce127ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.402+0000 7f402dd57640 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce12ad80 0x55f0ce128110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f402cd55640 1 -- 172.21.15.78:0/609535501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce12ad80 msgr2=0x55f0ce128110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:27.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f402cd55640 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce12ad80 0x55f0ce128110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f402cd55640 1 -- 172.21.15.78:0/609535501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11e4d0 msgr2=0x55f0ce12a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:27.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f402cd55640 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11e4d0 0x55f0ce12a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f402cd55640 1 -- 172.21.15.78:0/609535501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0ce04cc90 con 0x55f0ce11edf0 2024-09-11T01:10:27.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f402d556640 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11e4d0 0x55f0ce12a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:27.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f402cd55640 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11edf0 0x55f0ce127ae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4018005b80 tx=0x7f40180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4171c58e23282a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:27.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f4010ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:10:27.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f4010ff9640 1 -- 172.21.15.78:0/609535501 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401802e060 con 0x55f0ce11edf0 2024-09-11T01:10:27.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f4031b8f740 1 -- 172.21.15.78:0/609535501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4024004030 con 0x55f0ce11edf0 2024-09-11T01:10:27.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f4010ff9640 1 -- 172.21.15.78:0/609535501 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f401801b030 con 0x55f0ce11edf0 2024-09-11T01:10:27.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f4010ff9640 1 -- 172.21.15.78:0/609535501 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4018004260 con 0x55f0ce11edf0 2024-09-11T01:10:27.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f4031b8f740 10 client.26855 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:27.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.403+0000 7f4031b8f740 1 -- 172.21.15.78:0/609535501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f0ce1393b0 con 0x55f0ce11edf0 2024-09-11T01:10:27.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.404+0000 7f4010ff9640 1 -- 172.21.15.78:0/609535501 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4024004030 con 0x55f0ce11edf0 2024-09-11T01:10:27.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.404+0000 7f4010ff9640 1 -- 172.21.15.78:0/609535501 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f4018003890 con 0x55f0ce11edf0 2024-09-11T01:10:27.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.404+0000 7f4031b8f740 10 client.26855 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:27.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.404+0000 7f4031b8f740 10 client.26855 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:10:27.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.404+0000 7f4031b8f740 20 client.26855 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:27.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.404+0000 7f4031b8f740 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55f0ce13a7c0 0x55f0ce13cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.404+0000 7f4031b8f740 4 client.26855 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:10:27.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.404+0000 7f4031b8f740 1 -- 172.21.15.78:0/609535501 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0ce096df0 con 0x55f0ce13a7c0 2024-09-11T01:10:27.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.404+0000 7f402d556640 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55f0ce13a7c0 0x55f0ce13cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.404+0000 7f4010ff9640 10 client.26855 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:10:27.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.404+0000 7f402d556640 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55f0ce13a7c0 0x55f0ce13cc10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=54277af60756ca07 server_cookie=9d1c1650c98c531d in_seq=0 out_seq=0 2024-09-11T01:10:27.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4010ff9640 1 -- 172.21.15.78:0/609535501 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f0ce096df0 con 0x55f0ce13a7c0 2024-09-11T01:10:27.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4010ff9640 10 client.26855 handle_command_reply: tid=0 2024-09-11T01:10:27.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4031b8f740 1 client.26855 shutdown 2024-09-11T01:10:27.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4031b8f740 1 -- 172.21.15.78:0/609535501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11edf0 msgr2=0x55f0ce127ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:27.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4031b8f740 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11edf0 0x55f0ce127ae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4018005b80 tx=0x7f40180046c0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4031b8f740 1 -- 172.21.15.78:0/609535501 shutdown_connections 2024-09-11T01:10:27.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4031b8f740 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55f0ce13a7c0 0x55f0ce13cc10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4031b8f740 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce12ad80 0x55f0ce128110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4031b8f740 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11edf0 0x55f0ce127ae0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4031b8f740 1 --2- 172.21.15.78:0/609535501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11e4d0 0x55f0ce12a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4010ff9640 0 client.26855 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:10:27.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4031b8f740 1 -- 172.21.15.78:0/609535501 >> 172.21.15.78:0/609535501 conn(0x55f0ce11c300 msgr2=0x55f0ce121ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:27.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4031b8f740 1 -- 172.21.15.78:0/609535501 shutdown_connections 2024-09-11T01:10:27.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4031b8f740 1 -- 172.21.15.78:0/609535501 wait complete. 2024-09-11T01:10:27.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.405+0000 7f4031b8f740 20 client.26855 trim_cache size 0 max 16384 2024-09-11T01:10:27.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.418+0000 7f4031b8f740 1 Processor -- start 2024-09-11T01:10:27.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f4031b8f740 1 -- start start 2024-09-11T01:10:27.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f4031b8f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11e4d0 0x55f0ce136bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f4031b8f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11edf0 0x55f0ce11d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f4031b8f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce12ad80 0x55f0ce126fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f4031b8f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40281b0820 con 0x55f0ce11edf0 2024-09-11T01:10:27.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f4031b8f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f40281b2420 con 0x55f0ce12ad80 2024-09-11T01:10:27.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f4031b8f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40281b0ef0 con 0x55f0ce11e4d0 2024-09-11T01:10:27.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f402cd55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11edf0 0x55f0ce11d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f402dd57640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce12ad80 0x55f0ce126fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f402d556640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11e4d0 0x55f0ce136bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f402cd55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11edf0 0x55f0ce11d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46636/0 (socket says 172.21.15.78:46636) 2024-09-11T01:10:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f402cd55640 1 -- 172.21.15.78:0/667346559 learned_addr learned my addr 172.21.15.78:0/667346559 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f402cd55640 1 -- 172.21.15.78:0/667346559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11e4d0 msgr2=0x55f0ce136bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f402cd55640 1 --2- 172.21.15.78:0/667346559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11e4d0 0x55f0ce136bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f402cd55640 1 -- 172.21.15.78:0/667346559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce12ad80 msgr2=0x55f0ce126fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f402cd55640 1 --2- 172.21.15.78:0/667346559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce12ad80 0x55f0ce126fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f402cd55640 1 -- 172.21.15.78:0/667346559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0ce126a40 con 0x55f0ce11edf0 2024-09-11T01:10:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f402dd57640 1 --2- 172.21.15.78:0/667346559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce12ad80 0x55f0ce126fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.419+0000 7f402cd55640 1 --2- 172.21.15.78:0/667346559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11edf0 0x55f0ce11d6f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4018003cb0 tx=0x7f4018005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16602fbc303b9a98 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.420+0000 7f4012ffd640 1 -- 172.21.15.78:0/667346559 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018004400 con 0x55f0ce11edf0 2024-09-11T01:10:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.420+0000 7f4012ffd640 1 -- 172.21.15.78:0/667346559 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40180207d0 con 0x55f0ce11edf0 2024-09-11T01:10:27.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.420+0000 7f4012ffd640 1 -- 172.21.15.78:0/667346559 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40180336d0 con 0x55f0ce11edf0 2024-09-11T01:10:27.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.420+0000 7f4031b8f740 1 -- 172.21.15.78:0/667346559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11edf0 msgr2=0x55f0ce11d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:27.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.420+0000 7f4031b8f740 1 --2- 172.21.15.78:0/667346559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11edf0 0x55f0ce11d6f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4018003cb0 tx=0x7f4018005c50 comp rx=0 tx=0).stop 2024-09-11T01:10:27.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.420+0000 7f4031b8f740 1 -- 172.21.15.78:0/667346559 shutdown_connections 2024-09-11T01:10:27.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.420+0000 7f4031b8f740 1 --2- 172.21.15.78:0/667346559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce12ad80 0x55f0ce126fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.420+0000 7f4031b8f740 1 --2- 172.21.15.78:0/667346559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce11edf0 0x55f0ce11d6f0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.420+0000 7f4031b8f740 1 --2- 172.21.15.78:0/667346559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11e4d0 0x55f0ce136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.420+0000 7f4031b8f740 1 -- 172.21.15.78:0/667346559 >> 172.21.15.78:0/667346559 conn(0x55f0ce11c300 msgr2=0x55f0ce11cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:27.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.421+0000 7f4031b8f740 1 -- 172.21.15.78:0/667346559 shutdown_connections 2024-09-11T01:10:27.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.421+0000 7f4031b8f740 1 -- 172.21.15.78:0/667346559 wait complete. 2024-09-11T01:10:27.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.421+0000 7f4031b8f740 1 -- start start 2024-09-11T01:10:27.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.421+0000 7f4031b8f740 1 Processor -- start 2024-09-11T01:10:27.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.421+0000 7f4031b8f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11e4d0 0x55f0ce124cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.421+0000 7f4031b8f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11edf0 0x55f0ce125210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.421+0000 7f4031b8f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 0x55f0ce125750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.421+0000 7f4031b8f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40281b1c90 con 0x55f0ce12ad80 2024-09-11T01:10:27.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.421+0000 7f4031b8f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f40281b15c0 con 0x55f0ce11e4d0 2024-09-11T01:10:27.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.421+0000 7f4031b8f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40281ae820 con 0x55f0ce11edf0 2024-09-11T01:10:27.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f4031b8f740 10 client.0 _ll_register_callbacks cb 0x7f4018005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:27.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f402dd57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 0x55f0ce125750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f402d556640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11e4d0 0x55f0ce124cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f402dd57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 0x55f0ce125750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46642/0 (socket says 172.21.15.78:46642) 2024-09-11T01:10:27.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f402dd57640 1 -- 172.21.15.78:0/1993925376 learned_addr learned my addr 172.21.15.78:0/1993925376 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:27.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f402cd55640 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11edf0 0x55f0ce125210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f402dd57640 1 -- 172.21.15.78:0/1993925376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11edf0 msgr2=0x55f0ce125210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:27.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f402dd57640 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11edf0 0x55f0ce125210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f402dd57640 1 -- 172.21.15.78:0/1993925376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11e4d0 msgr2=0x55f0ce124cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:27.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f402dd57640 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11e4d0 0x55f0ce124cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f402dd57640 1 -- 172.21.15.78:0/1993925376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0ce12f230 con 0x55f0ce12ad80 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f402d556640 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11e4d0 0x55f0ce124cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f402dd57640 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 0x55f0ce125750 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f40200a8f60 tx=0x7f40200055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910fe2140f49f61f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f4010ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f4010ff9640 1 -- 172.21.15.78:0/1993925376 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020096020 con 0x55f0ce12ad80 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f4010ff9640 1 -- 172.21.15.78:0/1993925376 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f40200b5c90 con 0x55f0ce12ad80 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f4031b8f740 1 -- 172.21.15.78:0/1993925376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40180207d0 con 0x55f0ce12ad80 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f4010ff9640 1 -- 172.21.15.78:0/1993925376 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f40200ac100 con 0x55f0ce12ad80 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f4031b8f740 10 client.17667 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.422+0000 7f4031b8f740 1 -- 172.21.15.78:0/1993925376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f0ce1a7b30 con 0x55f0ce12ad80 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.423+0000 7f4010ff9640 1 -- 172.21.15.78:0/1993925376 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40180207d0 con 0x55f0ce12ad80 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.423+0000 7f4010ff9640 1 -- 172.21.15.78:0/1993925376 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f40200cfce0 con 0x55f0ce12ad80 2024-09-11T01:10:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.423+0000 7f4031b8f740 10 client.17667 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.423+0000 7f4031b8f740 10 client.17667 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:10:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.423+0000 7f4031b8f740 20 client.17667 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.423+0000 7f4031b8f740 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55f0ce1a82b0 0x55f0ce1aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.423+0000 7f4031b8f740 4 client.17667 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:10:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.423+0000 7f4031b8f740 1 -- 172.21.15.78:0/1993925376 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0ce0928a0 con 0x55f0ce1a82b0 2024-09-11T01:10:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.423+0000 7f402d556640 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55f0ce1a82b0 0x55f0ce1aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.423+0000 7f4010ff9640 10 client.17667 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:10:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.423+0000 7f402d556640 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55f0ce1a82b0 0x55f0ce1aa6d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c87b1f8674bc4e83 server_cookie=28edb6af79334a83 in_seq=0 out_seq=0 2024-09-11T01:10:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4010ff9640 1 -- 172.21.15.78:0/1993925376 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0ce0928a0 con 0x55f0ce1a82b0 2024-09-11T01:10:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4010ff9640 10 client.17667 handle_command_reply: tid=0 2024-09-11T01:10:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4031b8f740 1 client.17667 shutdown 2024-09-11T01:10:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4031b8f740 1 -- 172.21.15.78:0/1993925376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 msgr2=0x55f0ce125750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4031b8f740 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 0x55f0ce125750 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f40200a8f60 tx=0x7f40200055c0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4031b8f740 1 -- 172.21.15.78:0/1993925376 shutdown_connections 2024-09-11T01:10:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4031b8f740 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55f0ce1a82b0 0x55f0ce1aa6d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4031b8f740 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0ce12ad80 0x55f0ce125750 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4010ff9640 0 client.17667 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:10:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4031b8f740 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0ce11edf0 0x55f0ce125210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4031b8f740 1 --2- 172.21.15.78:0/1993925376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f0ce11e4d0 0x55f0ce124cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4031b8f740 1 -- 172.21.15.78:0/1993925376 >> 172.21.15.78:0/1993925376 conn(0x55f0ce11c300 msgr2=0x55f0ce129110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4031b8f740 1 -- 172.21.15.78:0/1993925376 shutdown_connections 2024-09-11T01:10:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4031b8f740 1 -- 172.21.15.78:0/1993925376 wait complete. 2024-09-11T01:10:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f4031b8f740 20 client.17667 trim_cache size 0 max 16384 2024-09-11T01:10:27.441 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:10:27.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f402f7ba640 1 -- 172.21.15.78:0/2763225293 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f40140b5450 msgr2=0x7f40140b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:27.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f402f7ba640 1 --2- 172.21.15.78:0/2763225293 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f40140b5450 0x7f40140b7910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f401800bfd0 tx=0x7f4018004730 comp rx=0 tx=0).stop 2024-09-11T01:10:27.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f402f7ba640 1 -- 172.21.15.78:0/2763225293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f402810f3e0 msgr2=0x7f40281a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:27.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.424+0000 7f402f7ba640 1 --2- 172.21.15.78:0/2763225293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f402810f3e0 0x7f40281a0dc0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f402000eb10 tx=0x7f402000c2d0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.425+0000 7f402f7ba640 1 -- 172.21.15.78:0/2763225293 shutdown_connections 2024-09-11T01:10:27.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.425+0000 7f402f7ba640 1 --2- 172.21.15.78:0/2763225293 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f40140b5450 0x7f40140b7910 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.425+0000 7f402f7ba640 1 --2- 172.21.15.78:0/2763225293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f402810f3e0 0x7f40281a0dc0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.425+0000 7f402f7ba640 1 --2- 172.21.15.78:0/2763225293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4028102150 0x7f402819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.425+0000 7f402f7ba640 1 --2- 172.21.15.78:0/2763225293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4028101830 0x7f402819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:27.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.425+0000 7f402f7ba640 1 -- 172.21.15.78:0/2763225293 >> 172.21.15.78:0/2763225293 conn(0x7f40280fd700 msgr2=0x7f40281115e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:27.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.425+0000 7f402f7ba640 1 -- 172.21.15.78:0/2763225293 shutdown_connections 2024-09-11T01:10:27.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:27.425+0000 7f402f7ba640 1 -- 172.21.15.78:0/2763225293 wait complete. 2024-09-11T01:10:27.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 790939 5.62 MB/sec execute 515 sec latency 43.231 ms 2024-09-11T01:10:28.223 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:10:28.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 792375 5.63 MB/sec execute 516 sec latency 30.782 ms 2024-09-11T01:10:28.490 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:10:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:28 smithi078 ceph-mon[31315]: pgmap v681: 97 pgs: 97 active+clean; 627 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 288 op/s 2024-09-11T01:10:28.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:28 smithi110 ceph-mon[34712]: pgmap v681: 97 pgs: 97 active+clean; 627 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 288 op/s 2024-09-11T01:10:28.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:28 smithi112 ceph-mon[35384]: pgmap v681: 97 pgs: 97 active+clean; 627 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 288 op/s 2024-09-11T01:10:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.992+0000 7f0a55256640 1 -- 172.21.15.78:0/2056166906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a50108020 msgr2=0x7f0a5010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.992+0000 7f0a55256640 1 --2- 172.21.15.78:0/2056166906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a50108020 0x7f0a5010e820 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009ca0 tx=0x7f0a4802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.992+0000 7f0a55256640 1 -- 172.21.15.78:0/2056166906 shutdown_connections 2024-09-11T01:10:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.992+0000 7f0a55256640 1 --2- 172.21.15.78:0/2056166906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a50108020 0x7f0a5010e820 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.992+0000 7f0a55256640 1 --2- 172.21.15.78:0/2056166906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a501076a0 0x7f0a50107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.992+0000 7f0a55256640 1 --2- 172.21.15.78:0/2056166906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a50101c90 0x7f0a50102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.992+0000 7f0a55256640 1 -- 172.21.15.78:0/2056166906 >> 172.21.15.78:0/2056166906 conn(0x7f0a500fd700 msgr2=0x7f0a500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.992+0000 7f0a55256640 1 -- 172.21.15.78:0/2056166906 shutdown_connections 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.992+0000 7f0a55256640 1 -- 172.21.15.78:0/2056166906 wait complete. 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a55256640 1 Processor -- start 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a55256640 1 -- start start 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a55256640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a50101c90 0x7f0a500fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a55256640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a501076a0 0x7f0a500fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a55256640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a50108020 0x7f0a500fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a55256640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50111ce0 con 0x7f0a501076a0 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a55256640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50111b60 con 0x7f0a50108020 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a55256640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50111e60 con 0x7f0a50101c90 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a4e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a501076a0 0x7f0a500fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a4ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a50101c90 0x7f0a500fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a4e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a501076a0 0x7f0a500fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46658/0 (socket says 172.21.15.78:46658) 2024-09-11T01:10:29.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a4e59c640 1 -- 172.21.15.78:0/2540980093 learned_addr learned my addr 172.21.15.78:0/2540980093 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.993+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a50108020 0x7f0a500fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.994+0000 7f0a4ed9d640 1 -- 172.21.15.78:0/2540980093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a50108020 msgr2=0x7f0a500fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.994+0000 7f0a4ed9d640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a50108020 0x7f0a500fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.994+0000 7f0a4ed9d640 1 -- 172.21.15.78:0/2540980093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a501076a0 msgr2=0x7f0a500fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.994+0000 7f0a4ed9d640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a501076a0 0x7f0a500fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.994+0000 7f0a4ed9d640 1 -- 172.21.15.78:0/2540980093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a501a76e0 con 0x7f0a50101c90 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.994+0000 7f0a4ed9d640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a50101c90 0x7f0a500fdc00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000e9f0 tx=0x7f0a4000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.994+0000 7f0a37fff640 1 -- 172.21.15.78:0/2540980093 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4000ccc0 con 0x7f0a50101c90 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.994+0000 7f0a37fff640 1 -- 172.21.15.78:0/2540980093 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40005020 con 0x7f0a50101c90 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.994+0000 7f0a55256640 1 -- 172.21.15.78:0/2540980093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a501a79d0 con 0x7f0a50101c90 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.994+0000 7f0a37fff640 1 -- 172.21.15.78:0/2540980093 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40004a00 con 0x7f0a50101c90 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.994+0000 7f0a55256640 1 -- 172.21.15.78:0/2540980093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a501a7ec0 con 0x7f0a50101c90 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a5762b740 1 Processor -- start 2024-09-11T01:10:29.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a37fff640 1 -- 172.21.15.78:0/2540980093 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0a40004ba0 con 0x7f0a50101c90 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a5762b740 1 -- start start 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a5762b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f39b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a5762b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f39ba10 0x563b4f3a7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a5762b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f3a79a0 0x563b4f3a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a5762b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563b4f311ea0 con 0x563b4f39b0f0 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a5762b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563b4f3012d0 con 0x563b4f3a79a0 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a5762b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563b4f3050e0 con 0x563b4f39ba10 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a4f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f39b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a4f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f39b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46662/0 (socket says 172.21.15.78:46662) 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a4f59e640 1 -- 172.21.15.78:0/4067127976 learned_addr learned my addr 172.21.15.78:0/4067127976 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a4ed9d640 1 --2- 172.21.15.78:0/4067127976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f39ba10 0x563b4f3a7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a4e59c640 1 --2- 172.21.15.78:0/4067127976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f3a79a0 0x563b4f3a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a4f59e640 1 -- 172.21.15.78:0/4067127976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f39ba10 msgr2=0x563b4f3a7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/4067127976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f39ba10 0x563b4f3a7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a4f59e640 1 -- 172.21.15.78:0/4067127976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f3a79a0 msgr2=0x563b4f3a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/4067127976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f3a79a0 0x563b4f3a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a4f59e640 1 -- 172.21.15.78:0/4067127976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4f05f650 con 0x563b4f39b0f0 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.997+0000 7f0a4e59c640 1 --2- 172.21.15.78:0/4067127976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f3a79a0 0x563b4f3a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.998+0000 7f0a37fff640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a380b5450 0x7f0a380b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.998+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/4067127976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f39b4d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0a48035000 tx=0x7f0a48005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798a1aaacd92fb58 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.998+0000 7f0a37fff640 1 -- 172.21.15.78:0/2540980093 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f0a400e5fb0 con 0x7f0a50101c90 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.998+0000 7f0a357fa640 1 -- 172.21.15.78:0/4067127976 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48031cf0 con 0x563b4f39b0f0 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.998+0000 7f0a357fa640 1 -- 172.21.15.78:0/4067127976 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a48036030 con 0x563b4f39b0f0 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.998+0000 7f0a357fa640 1 -- 172.21.15.78:0/4067127976 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a480042a0 con 0x563b4f39b0f0 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.998+0000 7f0a4ed9d640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a380b5450 0x7f0a380b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.999+0000 7f0a5762b740 1 -- 172.21.15.78:0/4067127976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 msgr2=0x563b4f39b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.999+0000 7f0a4ed9d640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a380b5450 0x7f0a380b7910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x563b4f399e00 tx=0x7f0a400e9a60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.999+0000 7f0a5762b740 1 --2- 172.21.15.78:0/4067127976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f39b4d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0a48035000 tx=0x7f0a48005fa0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.999+0000 7f0a5762b740 1 -- 172.21.15.78:0/4067127976 shutdown_connections 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.999+0000 7f0a5762b740 1 --2- 172.21.15.78:0/4067127976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f3a79a0 0x563b4f3a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.999+0000 7f0a5762b740 1 --2- 172.21.15.78:0/4067127976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f39ba10 0x563b4f3a7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.999+0000 7f0a5762b740 1 --2- 172.21.15.78:0/4067127976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f39b4d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.999+0000 7f0a5762b740 1 -- 172.21.15.78:0/4067127976 >> 172.21.15.78:0/4067127976 conn(0x563b4f398f20 msgr2=0x563b4f39a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.999+0000 7f0a5762b740 1 -- 172.21.15.78:0/4067127976 shutdown_connections 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.999+0000 7f0a5762b740 1 -- 172.21.15.78:0/4067127976 wait complete. 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.999+0000 7f0a5762b740 1 -- start start 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:28.999+0000 7f0a5762b740 1 Processor -- start 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a5762b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f3a7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a5762b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 0x563b4f3a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a5762b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f3a79a0 0x563b4f3a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a5762b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563b4f162700 con 0x563b4f39b0f0 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a5762b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563b4f29c160 con 0x563b4f39ba10 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a5762b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563b4f3aa440 con 0x563b4f3a79a0 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a5762b740 10 client.0 _ll_register_callbacks cb 0x563b4f2b59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a4f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f3a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a4f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f3a7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46670/0 (socket says 172.21.15.78:46670) 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a4f59e640 1 -- 172.21.15.78:0/960134360 learned_addr learned my addr 172.21.15.78:0/960134360 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a4e59c640 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 0x563b4f3a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f3a79a0 0x563b4f3a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a4f59e640 1 -- 172.21.15.78:0/960134360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f3a79a0 msgr2=0x563b4f3a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f3a79a0 0x563b4f3a4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a4f59e640 1 -- 172.21.15.78:0/960134360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 msgr2=0x563b4f3a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 0x563b4f3a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a4f59e640 1 -- 172.21.15.78:0/960134360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4f2c9c90 con 0x563b4f39b0f0 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f3a7280 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0a4802f470 tx=0x7f0a48003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84b08ddeb3d9886 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.000+0000 7f0a32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a32ff5640 1 -- 172.21.15.78:0/960134360 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48059060 con 0x563b4f39b0f0 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a5762b740 1 -- 172.21.15.78:0/960134360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a48036030 con 0x563b4f39b0f0 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a32ff5640 1 -- 172.21.15.78:0/960134360 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f0a4802f660 con 0x563b4f39b0f0 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a32ff5640 1 -- 172.21.15.78:0/960134360 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0a4802f860 con 0x563b4f39b0f0 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a5762b740 10 client.17679 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a5762b740 1 -- 172.21.15.78:0/960134360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563b4f3b5d60 con 0x563b4f39b0f0 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a32ff5640 1 -- 172.21.15.78:0/960134360 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a48036030 con 0x563b4f39b0f0 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a32ff5640 1 -- 172.21.15.78:0/960134360 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f0a48043070 con 0x563b4f39b0f0 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a5762b740 10 client.17679 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a5762b740 10 client.17679 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:10:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a5762b740 20 client.17679 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a5762b740 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x563b4f3b70b0 0x563b4f3b94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a5762b740 4 client.17679 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.001+0000 7f0a5762b740 1 -- 172.21.15.78:0/960134360 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b4f313df0 con 0x563b4f3b70b0 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.002+0000 7f0a4e59c640 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x563b4f3b70b0 0x563b4f3b94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.002+0000 7f0a32ff5640 10 client.17679 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.002+0000 7f0a4e59c640 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x563b4f3b70b0 0x563b4f3b94a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5a08fb449f2ed74 server_cookie=e774f699420352c7 in_seq=0 out_seq=0 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.002+0000 7f0a32ff5640 1 -- 172.21.15.78:0/960134360 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b4f313df0 con 0x563b4f3b70b0 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.002+0000 7f0a32ff5640 10 client.17679 handle_command_reply: tid=0 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a5762b740 1 client.17679 shutdown 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a5762b740 1 -- 172.21.15.78:0/960134360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 msgr2=0x563b4f3a7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a5762b740 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f3a7280 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0a4802f470 tx=0x7f0a48003cd0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a5762b740 1 -- 172.21.15.78:0/960134360 shutdown_connections 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a5762b740 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x563b4f3b70b0 0x563b4f3b94a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a32ff5640 0 client.17679 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:10:29.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a5762b740 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f3a79a0 0x563b4f3a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a5762b740 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 0x563b4f3a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a5762b740 1 --2- 172.21.15.78:0/960134360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f3a7280 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.084 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a5762b740 1 -- 172.21.15.78:0/960134360 >> 172.21.15.78:0/960134360 conn(0x563b4f398f20 msgr2=0x563b4f39ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a5762b740 1 -- 172.21.15.78:0/960134360 shutdown_connections 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a5762b740 1 -- 172.21.15.78:0/960134360 wait complete. 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.003+0000 7f0a5762b740 20 client.17679 trim_cache size 0 max 16384 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.017+0000 7f0a5762b740 1 Processor -- start 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.017+0000 7f0a5762b740 1 -- start start 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a5762b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f39b0f0 0x563b4f3b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a5762b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 0x563b4f3a6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a5762b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f3a79a0 0x563b4f3a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a5762b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501b07d0 con 0x563b4f3a79a0 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a5762b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501b23d0 con 0x563b4f39ba10 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a5762b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501b0ea0 con 0x563b4f39b0f0 2024-09-11T01:10:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a4f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f3a79a0 0x563b4f3a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a4f59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f39b0f0 0x563b4f3b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a4e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 0x563b4f3a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a4f59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f39b0f0 0x563b4f3b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:40886/0 (socket says 172.21.15.78:40886) 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a4f59e640 1 -- 172.21.15.78:0/3989734635 learned_addr learned my addr 172.21.15.78:0/3989734635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a4f59e640 1 -- 172.21.15.78:0/3989734635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 msgr2=0x563b4f3a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/3989734635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 0x563b4f3a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a4f59e640 1 -- 172.21.15.78:0/3989734635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f3a79a0 msgr2=0x563b4f3a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/3989734635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f3a79a0 0x563b4f3a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a4f59e640 1 -- 172.21.15.78:0/3989734635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4f3b9810 con 0x563b4f39b0f0 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/3989734635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f39b0f0 0x563b4f3b3670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0a48031e80 tx=0x7f0a48000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ab58b15beae120 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a34ff9640 1 -- 172.21.15.78:0/3989734635 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48059070 con 0x563b4f39b0f0 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a34ff9640 1 -- 172.21.15.78:0/3989734635 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a48036040 con 0x563b4f39b0f0 2024-09-11T01:10:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.018+0000 7f0a34ff9640 1 -- 172.21.15.78:0/3989734635 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48003d40 con 0x563b4f39b0f0 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.019+0000 7f0a5762b740 1 -- 172.21.15.78:0/3989734635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f39b0f0 msgr2=0x563b4f3b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.019+0000 7f0a5762b740 1 --2- 172.21.15.78:0/3989734635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f39b0f0 0x563b4f3b3670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0a48031e80 tx=0x7f0a48000f40 comp rx=0 tx=0).stop 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.019+0000 7f0a5762b740 1 -- 172.21.15.78:0/3989734635 shutdown_connections 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.019+0000 7f0a5762b740 1 --2- 172.21.15.78:0/3989734635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f3a79a0 0x563b4f3a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.019+0000 7f0a5762b740 1 --2- 172.21.15.78:0/3989734635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 0x563b4f3a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.019+0000 7f0a5762b740 1 --2- 172.21.15.78:0/3989734635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f39b0f0 0x563b4f3b3670 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.019+0000 7f0a5762b740 1 -- 172.21.15.78:0/3989734635 >> 172.21.15.78:0/3989734635 conn(0x563b4f398f20 msgr2=0x563b4f3a5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.019+0000 7f0a5762b740 1 -- 172.21.15.78:0/3989734635 shutdown_connections 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.019+0000 7f0a5762b740 1 -- 172.21.15.78:0/3989734635 wait complete. 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.019+0000 7f0a5762b740 1 -- start start 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a5762b740 1 Processor -- start 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a5762b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f3af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a5762b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 0x563b4f3afc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a5762b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f3a79a0 0x563b4f3aa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a5762b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501b1c40 con 0x563b4f39b0f0 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a5762b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501b1570 con 0x563b4f39ba10 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a5762b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501ae7d0 con 0x563b4f3a79a0 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a5762b740 10 client.0 _ll_register_callbacks cb 0x7f0a400e9930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a4f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f3af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a4e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 0x563b4f3afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a4f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f3af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46686/0 (socket says 172.21.15.78:46686) 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a4f59e640 1 -- 172.21.15.78:0/134477657 learned_addr learned my addr 172.21.15.78:0/134477657 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/134477657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f3a79a0 0x563b4f3aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a4f59e640 1 -- 172.21.15.78:0/134477657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f3a79a0 msgr2=0x563b4f3aa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/134477657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f3a79a0 0x563b4f3aa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a4f59e640 1 -- 172.21.15.78:0/134477657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 msgr2=0x563b4f3afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:29.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/134477657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 0x563b4f3afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.020+0000 7f0a4f59e640 1 -- 172.21.15.78:0/134477657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4f3aabd0 con 0x563b4f39b0f0 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a4f59e640 1 --2- 172.21.15.78:0/134477657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f3af6e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0a48061040 tx=0x7f0a48003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619a93c2504ca870 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a32ff5640 1 -- 172.21.15.78:0/134477657 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4806d060 con 0x563b4f39b0f0 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a32ff5640 1 -- 172.21.15.78:0/134477657 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f0a4802f660 con 0x563b4f39b0f0 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a5762b740 1 -- 172.21.15.78:0/134477657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a48036040 con 0x563b4f39b0f0 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a32ff5640 1 -- 172.21.15.78:0/134477657 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0a48059070 con 0x563b4f39b0f0 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a5762b740 10 client.17685 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a5762b740 1 -- 172.21.15.78:0/134477657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563b4f424460 con 0x563b4f39b0f0 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a32ff5640 1 -- 172.21.15.78:0/134477657 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a48036040 con 0x563b4f39b0f0 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a32ff5640 1 -- 172.21.15.78:0/134477657 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f0a48043040 con 0x563b4f39b0f0 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a5762b740 10 client.17685 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:29.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a5762b740 10 client.17685 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a5762b740 20 client.17685 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a5762b740 1 --2- 172.21.15.78:0/134477657 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x563b4f3b70b0 0x563b4f426ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a5762b740 4 client.17685 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.021+0000 7f0a5762b740 1 -- 172.21.15.78:0/134477657 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b4f30f8a0 con 0x563b4f3b70b0 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a4e59c640 1 --2- 172.21.15.78:0/134477657 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x563b4f3b70b0 0x563b4f426ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a4e59c640 1 --2- 172.21.15.78:0/134477657 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x563b4f3b70b0 0x563b4f426ae0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c28fd129559dc8e9 server_cookie=7208e51583cdf5ee in_seq=0 out_seq=0 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a32ff5640 10 client.17685 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a32ff5640 1 -- 172.21.15.78:0/134477657 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b4f30f8a0 con 0x563b4f3b70b0 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a32ff5640 10 client.17685 handle_command_reply: tid=0 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a5762b740 1 client.17685 shutdown 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a5762b740 1 -- 172.21.15.78:0/134477657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 msgr2=0x563b4f3af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a5762b740 1 --2- 172.21.15.78:0/134477657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f3af6e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0a48061040 tx=0x7f0a48003b10 comp rx=0 tx=0).stop 2024-09-11T01:10:29.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a5762b740 1 -- 172.21.15.78:0/134477657 shutdown_connections 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a5762b740 1 --2- 172.21.15.78:0/134477657 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x563b4f3b70b0 0x563b4f426ae0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a5762b740 1 --2- 172.21.15.78:0/134477657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b4f3a79a0 0x563b4f3aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a5762b740 1 --2- 172.21.15.78:0/134477657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b4f39ba10 0x563b4f3afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a5762b740 1 --2- 172.21.15.78:0/134477657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b4f39b0f0 0x563b4f3af6e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a32ff5640 0 client.17685 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a5762b740 1 -- 172.21.15.78:0/134477657 >> 172.21.15.78:0/134477657 conn(0x563b4f398f20 msgr2=0x563b4f3a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a5762b740 1 -- 172.21.15.78:0/134477657 shutdown_connections 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.022+0000 7f0a5762b740 1 -- 172.21.15.78:0/134477657 wait complete. 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a5762b740 20 client.17685 trim_cache size 0 max 16384 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a55256640 1 -- 172.21.15.78:0/2540980093 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a380b5450 msgr2=0x7f0a380b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a55256640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a380b5450 0x7f0a380b7910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x563b4f399e00 tx=0x7f0a400e9a60 comp rx=0 tx=0).stop 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a55256640 1 -- 172.21.15.78:0/2540980093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a50101c90 msgr2=0x7f0a500fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a55256640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a50101c90 0x7f0a500fdc00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000e9f0 tx=0x7f0a4000eeb0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a55256640 1 -- 172.21.15.78:0/2540980093 shutdown_connections 2024-09-11T01:10:29.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a55256640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0a380b5450 0x7f0a380b7910 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a55256640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a50108020 0x7f0a500fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a55256640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a501076a0 0x7f0a500fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a55256640 1 --2- 172.21.15.78:0/2540980093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a50101c90 0x7f0a500fdc00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:29.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a55256640 1 -- 172.21.15.78:0/2540980093 >> 172.21.15.78:0/2540980093 conn(0x7f0a500fd700 msgr2=0x7f0a5006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:29.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a55256640 1 -- 172.21.15.78:0/2540980093 shutdown_connections 2024-09-11T01:10:29.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:29.023+0000 7f0a55256640 1 -- 172.21.15.78:0/2540980093 wait complete. 2024-09-11T01:10:29.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 794226 5.63 MB/sec execute 517 sec latency 45.185 ms 2024-09-11T01:10:30.043 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:10:30.313 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:10:30.435 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:30 smithi078 ceph-mon[31315]: pgmap v682: 97 pgs: 97 active+clean; 630 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 289 op/s 2024-09-11T01:10:30.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 795926 5.64 MB/sec execute 518 sec latency 12.124 ms 2024-09-11T01:10:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:30 smithi110 ceph-mon[34712]: pgmap v682: 97 pgs: 97 active+clean; 630 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 289 op/s 2024-09-11T01:10:30.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:30 smithi112 ceph-mon[35384]: pgmap v682: 97 pgs: 97 active+clean; 630 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 289 op/s 2024-09-11T01:10:30.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.794+0000 7f7042513640 1 -- 172.21.15.78:0/2629848900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f703c101c90 msgr2=0x7f703c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.794+0000 7f7042513640 1 --2- 172.21.15.78:0/2629848900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f703c101c90 0x7f703c102070 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f702c009900 tx=0x7f702c02a410 comp rx=0 tx=0).stop 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.794+0000 7f7042513640 1 -- 172.21.15.78:0/2629848900 shutdown_connections 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.794+0000 7f7042513640 1 --2- 172.21.15.78:0/2629848900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c108020 0x7f703c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.794+0000 7f7042513640 1 --2- 172.21.15.78:0/2629848900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c1076a0 0x7f703c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.794+0000 7f7042513640 1 --2- 172.21.15.78:0/2629848900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f703c101c90 0x7f703c102070 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.794+0000 7f7042513640 1 -- 172.21.15.78:0/2629848900 >> 172.21.15.78:0/2629848900 conn(0x7f703c0fd700 msgr2=0x7f703c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.794+0000 7f7042513640 1 -- 172.21.15.78:0/2629848900 shutdown_connections 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.794+0000 7f7042513640 1 -- 172.21.15.78:0/2629848900 wait complete. 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f7042513640 1 Processor -- start 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f7042513640 1 -- start start 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f7042513640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f703c101c90 0x7f703c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f7042513640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c1076a0 0x7f703c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f7042513640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c108020 0x7f703c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f7042513640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f703c111ca0 con 0x7f703c101c90 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f7042513640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f703c111b20 con 0x7f703c1076a0 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f7042513640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f703c111e20 con 0x7f703c108020 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f703bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f703c101c90 0x7f703c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f703bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f703c101c90 0x7f703c19a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46692/0 (socket says 172.21.15.78:46692) 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f703bfff640 1 -- 172.21.15.78:0/829642585 learned_addr learned my addr 172.21.15.78:0/829642585 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f7040ab0640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c108020 0x7f703c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f703b7fe640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c1076a0 0x7f703c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f703bfff640 1 -- 172.21.15.78:0/829642585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c108020 msgr2=0x7f703c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f703bfff640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c108020 0x7f703c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.795+0000 7f703bfff640 1 -- 172.21.15.78:0/829642585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c1076a0 msgr2=0x7f703c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.796+0000 7f703bfff640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c1076a0 0x7f703c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.796+0000 7f703bfff640 1 -- 172.21.15.78:0/829642585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c1a54a0 con 0x7f703c101c90 2024-09-11T01:10:30.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.796+0000 7f703bfff640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f703c101c90 0x7f703c19a0c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f702c005940 tx=0x7f702c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.796+0000 7f70397fa640 1 -- 172.21.15.78:0/829642585 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c03f030 con 0x7f703c101c90 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.796+0000 7f70397fa640 1 -- 172.21.15.78:0/829642585 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f702c0044c0 con 0x7f703c101c90 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.796+0000 7f7042513640 1 -- 172.21.15.78:0/829642585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c1a5730 con 0x7f703c101c90 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.796+0000 7f70397fa640 1 -- 172.21.15.78:0/829642585 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c035070 con 0x7f703c101c90 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.796+0000 7f7042513640 1 -- 172.21.15.78:0/829642585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c1a5c10 con 0x7f703c101c90 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.799+0000 7f70448e8740 1 Processor -- start 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.799+0000 7f70397fa640 1 -- 172.21.15.78:0/829642585 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f702c03ba20 con 0x7f703c101c90 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.799+0000 7f70448e8740 1 -- start start 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f70448e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e8518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f70448e8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e851df0 0x562f5e85d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f70448e8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e860170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f70448e8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562f5e7c7ea0 con 0x562f5e851df0 2024-09-11T01:10:30.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f70448e8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562f5e7b72d0 con 0x562f5e85dd80 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f70448e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f5e7bb0e0 con 0x562f5e8514d0 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f703bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e851df0 0x562f5e85d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f7040ab0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e8518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f703b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e860170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f703bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e851df0 0x562f5e85d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46694/0 (socket says 172.21.15.78:46694) 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f703bfff640 1 -- 172.21.15.78:0/3442460382 learned_addr learned my addr 172.21.15.78:0/3442460382 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f7040ab0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e8518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:40926/0 (socket says 172.21.15.78:40926) 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f7040ab0640 1 -- 172.21.15.78:0/3442460382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 msgr2=0x562f5e860170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f7040ab0640 1 --2- 172.21.15.78:0/3442460382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e860170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f7040ab0640 1 -- 172.21.15.78:0/3442460382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e851df0 msgr2=0x562f5e85d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f7040ab0640 1 --2- 172.21.15.78:0/3442460382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e851df0 0x562f5e85d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f7040ab0640 1 -- 172.21.15.78:0/3442460382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5e515650 con 0x562f5e8514d0 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f703b7fe640 1 --2- 172.21.15.78:0/3442460382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e860170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f70397fa640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f70240b5450 0x7f70240b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f7040ab0640 1 --2- 172.21.15.78:0/3442460382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e8518b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7034005fd0 tx=0x7f7034011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6a2ca9f78c0e8b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.800+0000 7f703bfff640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f70240b5450 0x7f70240b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.801+0000 7f70397fa640 1 -- 172.21.15.78:0/829642585 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f702c106570 con 0x7f703c101c90 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.801+0000 7f70227fc640 1 -- 172.21.15.78:0/3442460382 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034004d20 con 0x562f5e8514d0 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.801+0000 7f70227fc640 1 -- 172.21.15.78:0/3442460382 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7034004030 con 0x562f5e8514d0 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.801+0000 7f70227fc640 1 -- 172.21.15.78:0/3442460382 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034005280 con 0x562f5e8514d0 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.801+0000 7f70448e8740 1 -- 172.21.15.78:0/3442460382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 msgr2=0x562f5e8518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.801+0000 7f70448e8740 1 --2- 172.21.15.78:0/3442460382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e8518b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7034005fd0 tx=0x7f7034011810 comp rx=0 tx=0).stop 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.801+0000 7f703bfff640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f70240b5450 0x7f70240b7910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f702c006010 tx=0x7f702c0b4040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.801+0000 7f70448e8740 1 -- 172.21.15.78:0/3442460382 shutdown_connections 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.801+0000 7f70448e8740 1 --2- 172.21.15.78:0/3442460382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e860170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.801+0000 7f70448e8740 1 --2- 172.21.15.78:0/3442460382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e851df0 0x562f5e85d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.801+0000 7f70448e8740 1 --2- 172.21.15.78:0/3442460382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e8518b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.801+0000 7f70448e8740 1 -- 172.21.15.78:0/3442460382 >> 172.21.15.78:0/3442460382 conn(0x562f5e84f300 msgr2=0x562f5e8504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.802+0000 7f70448e8740 1 -- 172.21.15.78:0/3442460382 shutdown_connections 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.802+0000 7f70448e8740 1 -- 172.21.15.78:0/3442460382 wait complete. 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.802+0000 7f70448e8740 1 -- start start 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.802+0000 7f70448e8740 1 Processor -- start 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.802+0000 7f70448e8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e8514d0 0x562f5e85d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.802+0000 7f70448e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e851df0 0x562f5e85ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.802+0000 7f70448e8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e85b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.802+0000 7f70448e8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562f5e618700 con 0x562f5e8514d0 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.802+0000 7f70448e8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562f5e752160 con 0x562f5e85dd80 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.802+0000 7f70448e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f5e860a00 con 0x562f5e851df0 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.802+0000 7f70448e8740 10 client.0 _ll_register_callbacks cb 0x562f5e76b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:30.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.802+0000 7f7040ab0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e8514d0 0x562f5e85d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f7040ab0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e8514d0 0x562f5e85d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46706/0 (socket says 172.21.15.78:46706) 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f703b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e851df0 0x562f5e85ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f7040ab0640 1 -- 172.21.15.78:0/2185028752 learned_addr learned my addr 172.21.15.78:0/2185028752 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f7040ab0640 1 --2- 172.21.15.78:0/2185028752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e85b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f703b7fe640 1 -- 172.21.15.78:0/2185028752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 msgr2=0x562f5e85b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f703b7fe640 1 --2- 172.21.15.78:0/2185028752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e85b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f703b7fe640 1 -- 172.21.15.78:0/2185028752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e8514d0 msgr2=0x562f5e85d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f703b7fe640 1 --2- 172.21.15.78:0/2185028752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e8514d0 0x562f5e85d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f703b7fe640 1 -- 172.21.15.78:0/2185028752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f5e77fc90 con 0x562f5e851df0 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f703b7fe640 1 --2- 172.21.15.78:0/2185028752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e851df0 0x562f5e85ab10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70300126d0 tx=0x7f7030012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ca414901df5fd6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f701b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f701b7fe640 1 -- 172.21.15.78:0/2185028752 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030004030 con 0x562f5e851df0 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f701b7fe640 1 -- 172.21.15.78:0/2185028752 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f70300041d0 con 0x562f5e851df0 2024-09-11T01:10:30.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f70448e8740 1 -- 172.21.15.78:0/2185028752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7034004030 con 0x562f5e851df0 2024-09-11T01:10:30.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.803+0000 7f701b7fe640 1 -- 172.21.15.78:0/2185028752 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f70300043d0 con 0x562f5e851df0 2024-09-11T01:10:30.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f70448e8740 10 client.26818 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:30.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f70448e8740 1 -- 172.21.15.78:0/2185028752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562f5e8707f0 con 0x562f5e851df0 2024-09-11T01:10:30.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f701b7fe640 1 -- 172.21.15.78:0/2185028752 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7034004030 con 0x562f5e851df0 2024-09-11T01:10:30.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f701b7fe640 1 -- 172.21.15.78:0/2185028752 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f7030003b20 con 0x562f5e851df0 2024-09-11T01:10:30.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f70448e8740 10 client.26818 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:30.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f70448e8740 10 client.26818 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:10:30.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f70448e8740 20 client.26818 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:30.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f70448e8740 1 --2- 172.21.15.78:0/2185028752 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562f5e871c00 0x562f5e873ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f70448e8740 4 client.26818 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:10:30.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f70448e8740 1 -- 172.21.15.78:0/2185028752 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f5e7c9df0 con 0x562f5e871c00 2024-09-11T01:10:30.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f7040ab0640 1 --2- 172.21.15.78:0/2185028752 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562f5e871c00 0x562f5e873ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f701b7fe640 10 client.26818 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:10:30.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.804+0000 7f7040ab0640 1 --2- 172.21.15.78:0/2185028752 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562f5e871c00 0x562f5e873ff0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0dc32222d519c9d server_cookie=9b3d03e918a608c7 in_seq=0 out_seq=0 2024-09-11T01:10:30.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.805+0000 7f701b7fe640 1 -- 172.21.15.78:0/2185028752 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562f5e7c9df0 con 0x562f5e871c00 2024-09-11T01:10:30.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.805+0000 7f701b7fe640 10 client.26818 handle_command_reply: tid=0 2024-09-11T01:10:30.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.805+0000 7f70448e8740 1 client.26818 shutdown 2024-09-11T01:10:30.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.805+0000 7f70448e8740 1 -- 172.21.15.78:0/2185028752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e851df0 msgr2=0x562f5e85ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:30.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.805+0000 7f70448e8740 1 --2- 172.21.15.78:0/2185028752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e851df0 0x562f5e85ab10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70300126d0 tx=0x7f7030012b90 comp rx=0 tx=0).stop 2024-09-11T01:10:30.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.805+0000 7f70448e8740 1 -- 172.21.15.78:0/2185028752 shutdown_connections 2024-09-11T01:10:30.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.805+0000 7f70448e8740 1 --2- 172.21.15.78:0/2185028752 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562f5e871c00 0x562f5e873ff0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.805+0000 7f70448e8740 1 --2- 172.21.15.78:0/2185028752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e85b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.805+0000 7f70448e8740 1 --2- 172.21.15.78:0/2185028752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e851df0 0x562f5e85ab10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.805+0000 7f70448e8740 1 --2- 172.21.15.78:0/2185028752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e8514d0 0x562f5e85d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.805+0000 7f701b7fe640 0 client.26818 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:10:30.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.805+0000 7f70448e8740 1 -- 172.21.15.78:0/2185028752 >> 172.21.15.78:0/2185028752 conn(0x562f5e84f300 msgr2=0x562f5e854ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:30.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.806+0000 7f70448e8740 1 -- 172.21.15.78:0/2185028752 shutdown_connections 2024-09-11T01:10:30.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.806+0000 7f70448e8740 1 -- 172.21.15.78:0/2185028752 wait complete. 2024-09-11T01:10:30.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.806+0000 7f70448e8740 20 client.26818 trim_cache size 0 max 16384 2024-09-11T01:10:30.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.820+0000 7f70448e8740 1 Processor -- start 2024-09-11T01:10:30.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.820+0000 7f70448e8740 1 -- start start 2024-09-11T01:10:30.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.820+0000 7f70448e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e850ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.820+0000 7f70448e8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e851df0 0x562f5e869c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.820+0000 7f70448e8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e85f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.820+0000 7f70448e8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1ac440 con 0x562f5e851df0 2024-09-11T01:10:30.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.820+0000 7f70448e8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1ae040 con 0x562f5e85dd80 2024-09-11T01:10:30.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.820+0000 7f70448e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1acb10 con 0x562f5e8514d0 2024-09-11T01:10:30.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.821+0000 7f7040ab0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e85f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.821+0000 7f7040ab0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e850ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.821+0000 7f7040ab0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e850ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:40946/0 (socket says 172.21.15.78:40946) 2024-09-11T01:10:30.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.821+0000 7f7040ab0640 1 -- 172.21.15.78:0/3397097610 learned_addr learned my addr 172.21.15.78:0/3397097610 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:30.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.821+0000 7f703b7fe640 1 --2- 172.21.15.78:0/3397097610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e851df0 0x562f5e869c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.821+0000 7f7040ab0640 1 -- 172.21.15.78:0/3397097610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 msgr2=0x562f5e850ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:30.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.821+0000 7f7040ab0640 1 --2- 172.21.15.78:0/3397097610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e850ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.821+0000 7f7040ab0640 1 -- 172.21.15.78:0/3397097610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e851df0 msgr2=0x562f5e869c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:30.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.821+0000 7f7040ab0640 1 --2- 172.21.15.78:0/3397097610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e851df0 0x562f5e869c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.821+0000 7f7040ab0640 1 -- 172.21.15.78:0/3397097610 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5e85b500 con 0x562f5e85dd80 2024-09-11T01:10:30.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.822+0000 7f7040ab0640 1 --2- 172.21.15.78:0/3397097610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e85f170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7034009ed0 tx=0x7f7034009a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40140795c5acfa0a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:30.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.822+0000 7f7021ffb640 1 -- 172.21.15.78:0/3397097610 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703401e040 con 0x562f5e85dd80 2024-09-11T01:10:30.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.822+0000 7f7021ffb640 1 -- 172.21.15.78:0/3397097610 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7034004570 con 0x562f5e85dd80 2024-09-11T01:10:30.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.822+0000 7f7021ffb640 1 -- 172.21.15.78:0/3397097610 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703400fb20 con 0x562f5e85dd80 2024-09-11T01:10:30.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.823+0000 7f70448e8740 1 -- 172.21.15.78:0/3397097610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 msgr2=0x562f5e85f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:30.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.823+0000 7f70448e8740 1 --2- 172.21.15.78:0/3397097610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e85f170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7034009ed0 tx=0x7f7034009a90 comp rx=0 tx=0).stop 2024-09-11T01:10:30.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.823+0000 7f70448e8740 1 -- 172.21.15.78:0/3397097610 shutdown_connections 2024-09-11T01:10:30.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.823+0000 7f70448e8740 1 --2- 172.21.15.78:0/3397097610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e85dd80 0x562f5e85f170 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.823+0000 7f70448e8740 1 --2- 172.21.15.78:0/3397097610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e851df0 0x562f5e869c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.823+0000 7f70448e8740 1 --2- 172.21.15.78:0/3397097610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e850ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.823+0000 7f70448e8740 1 -- 172.21.15.78:0/3397097610 >> 172.21.15.78:0/3397097610 conn(0x562f5e84f300 msgr2=0x562f5e8503c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.823+0000 7f70448e8740 1 -- 172.21.15.78:0/3397097610 shutdown_connections 2024-09-11T01:10:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.823+0000 7f70448e8740 1 -- 172.21.15.78:0/3397097610 wait complete. 2024-09-11T01:10:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.823+0000 7f70448e8740 1 -- start start 2024-09-11T01:10:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f70448e8740 1 Processor -- start 2024-09-11T01:10:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f70448e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e8dbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f70448e8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e851df0 0x562f5e8dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f70448e8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e85dd80 0x562f5e8dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f70448e8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1ad8b0 con 0x562f5e85dd80 2024-09-11T01:10:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f70448e8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1ad1e0 con 0x562f5e851df0 2024-09-11T01:10:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f70448e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1aa440 con 0x562f5e8514d0 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f70448e8740 10 client.0 _ll_register_callbacks cb 0x7f7030009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f703b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e851df0 0x562f5e8dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f703b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e851df0 0x562f5e8dc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36944/0 (socket says 172.21.15.78:36944) 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f703b7fe640 1 -- 172.21.15.78:0/3384378633 learned_addr learned my addr 172.21.15.78:0/3384378633 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f7040ab0640 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e85dd80 0x562f5e8dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.824+0000 7f7040ab0640 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e8dbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f703b7fe640 1 -- 172.21.15.78:0/3384378633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 msgr2=0x562f5e8dbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f703b7fe640 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e8dbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f703b7fe640 1 -- 172.21.15.78:0/3384378633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e85dd80 msgr2=0x562f5e8dc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f703b7fe640 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e85dd80 0x562f5e8dc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f703b7fe640 1 -- 172.21.15.78:0/3384378633 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f5e8d8bd0 con 0x562f5e851df0 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f7040ab0640 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e85dd80 0x562f5e8dc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f703b7fe640 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e851df0 0x562f5e8dc040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7030012420 tx=0x7f703000f800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95f61acbc8dc8856 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f701b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f701b7fe640 1 -- 172.21.15.78:0/3384378633 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030026c20 con 0x562f5e851df0 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f701b7fe640 1 -- 172.21.15.78:0/3384378633 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f7030021020 con 0x562f5e851df0 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f70448e8740 1 -- 172.21.15.78:0/3384378633 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562f5e7c58a0 con 0x562f5e851df0 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f701b7fe640 1 -- 172.21.15.78:0/3384378633 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7030036050 con 0x562f5e851df0 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f70448e8740 10 client.26897 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.825+0000 7f70448e8740 1 -- 172.21.15.78:0/3384378633 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562f5e8650d0 con 0x562f5e851df0 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.826+0000 7f701b7fe640 1 -- 172.21.15.78:0/3384378633 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f703000fdd0 con 0x562f5e851df0 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.826+0000 7f701b7fe640 1 -- 172.21.15.78:0/3384378633 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f7030005520 con 0x562f5e851df0 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.826+0000 7f70448e8740 10 client.26897 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.826+0000 7f70448e8740 10 client.26897 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.827+0000 7f70448e8740 20 client.26897 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.827+0000 7f70448e8740 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562f5e865820 0x562f5e865c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.827+0000 7f70448e8740 4 client.26897 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:10:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.827+0000 7f70448e8740 1 -- 172.21.15.78:0/3384378633 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f5e7c58a0 con 0x562f5e865820 2024-09-11T01:10:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.827+0000 7f7040ab0640 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562f5e865820 0x562f5e865c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.827+0000 7f701b7fe640 10 client.26897 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:10:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.827+0000 7f7040ab0640 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562f5e865820 0x562f5e865c30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af7e70abee3756ec server_cookie=b91315aec39f9c9e in_seq=0 out_seq=0 2024-09-11T01:10:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f701b7fe640 1 -- 172.21.15.78:0/3384378633 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f5e7c58a0 con 0x562f5e865820 2024-09-11T01:10:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f701b7fe640 10 client.26897 handle_command_reply: tid=0 2024-09-11T01:10:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f70448e8740 1 client.26897 shutdown 2024-09-11T01:10:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f70448e8740 1 -- 172.21.15.78:0/3384378633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e851df0 msgr2=0x562f5e8dc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f70448e8740 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e851df0 0x562f5e8dc040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7030012420 tx=0x7f703000f800 comp rx=0 tx=0).stop 2024-09-11T01:10:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f70448e8740 1 -- 172.21.15.78:0/3384378633 shutdown_connections 2024-09-11T01:10:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f70448e8740 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f5e85dd80 0x562f5e8dc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f70448e8740 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562f5e865820 0x562f5e865c30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f70448e8740 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f5e851df0 0x562f5e8dc040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f70448e8740 1 --2- 172.21.15.78:0/3384378633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f5e8514d0 0x562f5e8dbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f701b7fe640 0 client.26897 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:10:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f70448e8740 1 -- 172.21.15.78:0/3384378633 >> 172.21.15.78:0/3384378633 conn(0x562f5e84f300 msgr2=0x562f5e85c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f70448e8740 1 -- 172.21.15.78:0/3384378633 shutdown_connections 2024-09-11T01:10:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.828+0000 7f70448e8740 1 -- 172.21.15.78:0/3384378633 wait complete. 2024-09-11T01:10:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.829+0000 7f70448e8740 20 client.26897 trim_cache size 0 max 16384 2024-09-11T01:10:30.841 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:10:30.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.829+0000 7f7042513640 1 -- 172.21.15.78:0/829642585 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f70240b5450 msgr2=0x7f70240b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:30.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.829+0000 7f7042513640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f70240b5450 0x7f70240b7910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f702c006010 tx=0x7f702c0b4040 comp rx=0 tx=0).stop 2024-09-11T01:10:30.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.829+0000 7f7042513640 1 -- 172.21.15.78:0/829642585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f703c101c90 msgr2=0x7f703c19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:30.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.829+0000 7f7042513640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f703c101c90 0x7f703c19a0c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f702c005940 tx=0x7f702c0040c0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.829+0000 7f7042513640 1 -- 172.21.15.78:0/829642585 shutdown_connections 2024-09-11T01:10:30.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.829+0000 7f7042513640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f70240b5450 0x7f70240b7910 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.829+0000 7f7042513640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c108020 0x7f703c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.829+0000 7f7042513640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c1076a0 0x7f703c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.829+0000 7f7042513640 1 --2- 172.21.15.78:0/829642585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f703c101c90 0x7f703c19a0c0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.829+0000 7f7042513640 1 -- 172.21.15.78:0/829642585 >> 172.21.15.78:0/829642585 conn(0x7f703c0fd700 msgr2=0x7f703c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.830+0000 7f7042513640 1 -- 172.21.15.78:0/829642585 shutdown_connections 2024-09-11T01:10:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:30.830+0000 7f7042513640 1 -- 172.21.15.78:0/829642585 wait complete. 2024-09-11T01:10:31.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 797897 5.65 MB/sec execute 519 sec latency 19.172 ms 2024-09-11T01:10:31.545 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:10:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:10:32.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.306+0000 7f2250e3b640 1 -- 172.21.15.78:0/4165605918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f224c101c90 msgr2=0x7f224c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:32.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.306+0000 7f2250e3b640 1 --2- 172.21.15.78:0/4165605918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f224c101c90 0x7f224c102070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f223c009860 tx=0x7f223c02a340 comp rx=0 tx=0).stop 2024-09-11T01:10:32.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.306+0000 7f2250e3b640 1 -- 172.21.15.78:0/4165605918 shutdown_connections 2024-09-11T01:10:32.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.306+0000 7f2250e3b640 1 --2- 172.21.15.78:0/4165605918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f224c108020 0x7f224c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.306+0000 7f2250e3b640 1 --2- 172.21.15.78:0/4165605918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f224c1076a0 0x7f224c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.306+0000 7f2250e3b640 1 --2- 172.21.15.78:0/4165605918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f224c101c90 0x7f224c102070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.306+0000 7f2250e3b640 1 -- 172.21.15.78:0/4165605918 >> 172.21.15.78:0/4165605918 conn(0x7f224c0fd700 msgr2=0x7f224c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:32.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.306+0000 7f2250e3b640 1 -- 172.21.15.78:0/4165605918 shutdown_connections 2024-09-11T01:10:32.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.306+0000 7f2250e3b640 1 -- 172.21.15.78:0/4165605918 wait complete. 2024-09-11T01:10:32.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.307+0000 7f2250e3b640 1 Processor -- start 2024-09-11T01:10:32.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.307+0000 7f2250e3b640 1 -- start start 2024-09-11T01:10:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.307+0000 7f2250e3b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f224c101c90 0x7f224c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.307+0000 7f2250e3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f224c1076a0 0x7f224c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.307+0000 7f2250e3b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f224c108020 0x7f224c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.307+0000 7f2250e3b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f224c111ce0 con 0x7f224c101c90 2024-09-11T01:10:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.307+0000 7f2250e3b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f224c111b60 con 0x7f224c108020 2024-09-11T01:10:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.307+0000 7f2250e3b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f224c111e60 con 0x7f224c1076a0 2024-09-11T01:10:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f224a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f224c101c90 0x7f224c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f224a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f224c101c90 0x7f224c19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46742/0 (socket says 172.21.15.78:46742) 2024-09-11T01:10:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f224a59c640 1 -- 172.21.15.78:0/1638711675 learned_addr learned my addr 172.21.15.78:0/1638711675 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f2243fff640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f224c1076a0 0x7f224c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f224ad9d640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f224c108020 0x7f224c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f224a59c640 1 -- 172.21.15.78:0/1638711675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f224c1076a0 msgr2=0x7f224c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f224a59c640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f224c1076a0 0x7f224c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f224a59c640 1 -- 172.21.15.78:0/1638711675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f224c108020 msgr2=0x7f224c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f224a59c640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f224c108020 0x7f224c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f224a59c640 1 -- 172.21.15.78:0/1638711675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c1a54d0 con 0x7f224c101c90 2024-09-11T01:10:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f224a59c640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f224c101c90 0x7f224c19a0d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f223c035040 tx=0x7f223c032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f22437fe640 1 -- 172.21.15.78:0/1638711675 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c037040 con 0x7f224c101c90 2024-09-11T01:10:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f22437fe640 1 -- 172.21.15.78:0/1638711675 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f223c041070 con 0x7f224c101c90 2024-09-11T01:10:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f2250e3b640 1 -- 172.21.15.78:0/1638711675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c1a5760 con 0x7f224c101c90 2024-09-11T01:10:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f22437fe640 1 -- 172.21.15.78:0/1638711675 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c003850 con 0x7f224c101c90 2024-09-11T01:10:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.308+0000 7f2250e3b640 1 -- 172.21.15.78:0/1638711675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c1a5be0 con 0x7f224c101c90 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f22437fe640 1 -- 172.21.15.78:0/1638711675 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f223c0039f0 con 0x7f224c101c90 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f2253210740 1 Processor -- start 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f2253210740 1 -- start start 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f2253210740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf496e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f2253210740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 0x55edf497ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f2253210740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4981580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f2253210740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55edf48e4ea0 con 0x55edf497f190 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f2253210740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55edf48d42d0 con 0x55edf496e4d0 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f2253210740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55edf48d80e0 con 0x55edf496edf0 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f224ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf496e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f224ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf496e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36964/0 (socket says 172.21.15.78:36964) 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f224ad9d640 1 -- 172.21.15.78:0/2918689576 learned_addr learned my addr 172.21.15.78:0/2918689576 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f2243fff640 1 --2- 172.21.15.78:0/2918689576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4981580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.311+0000 7f224a59c640 1 --2- 172.21.15.78:0/2918689576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 0x55edf497ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.312+0000 7f224ad9d640 1 -- 172.21.15.78:0/2918689576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 msgr2=0x55edf497ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.312+0000 7f224ad9d640 1 --2- 172.21.15.78:0/2918689576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 0x55edf497ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.312+0000 7f224ad9d640 1 -- 172.21.15.78:0/2918689576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 msgr2=0x55edf4981580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.312+0000 7f22437fe640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f22380b5450 0x7f22380b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.312+0000 7f224ad9d640 1 --2- 172.21.15.78:0/2918689576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4981580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.312+0000 7f224ad9d640 1 -- 172.21.15.78:0/2918689576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf4632650 con 0x55edf496e4d0 2024-09-11T01:10:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.312+0000 7f22437fe640 1 -- 172.21.15.78:0/1638711675 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f223c105bf0 con 0x7f224c101c90 2024-09-11T01:10:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.312+0000 7f2243fff640 1 --2- 172.21.15.78:0/2918689576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4981580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.312+0000 7f224a59c640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f22380b5450 0x7f22380b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.313+0000 7f224ad9d640 1 --2- 172.21.15.78:0/2918689576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf496e8b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2244017850 tx=0x7f2244017d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d67ec41c5f02b23f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.313+0000 7f2240ff9640 1 -- 172.21.15.78:0/2918689576 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244025070 con 0x55edf496e4d0 2024-09-11T01:10:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.313+0000 7f2240ff9640 1 -- 172.21.15.78:0/2918689576 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f224400ebc0 con 0x55edf496e4d0 2024-09-11T01:10:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.313+0000 7f2240ff9640 1 -- 172.21.15.78:0/2918689576 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224400d640 con 0x55edf496e4d0 2024-09-11T01:10:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.314+0000 7f2253210740 1 -- 172.21.15.78:0/2918689576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 msgr2=0x55edf496e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.314+0000 7f224a59c640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f22380b5450 0x7f22380b7910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f223c103430 tx=0x7f223c0d5040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.314+0000 7f2253210740 1 --2- 172.21.15.78:0/2918689576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf496e8b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2244017850 tx=0x7f2244017d10 comp rx=0 tx=0).stop 2024-09-11T01:10:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.314+0000 7f2253210740 1 -- 172.21.15.78:0/2918689576 shutdown_connections 2024-09-11T01:10:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.314+0000 7f2253210740 1 --2- 172.21.15.78:0/2918689576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4981580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.314+0000 7f2253210740 1 --2- 172.21.15.78:0/2918689576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 0x55edf497ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.314+0000 7f2253210740 1 --2- 172.21.15.78:0/2918689576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf496e8b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.314+0000 7f2253210740 1 -- 172.21.15.78:0/2918689576 >> 172.21.15.78:0/2918689576 conn(0x55edf496c300 msgr2=0x55edf496d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.314+0000 7f2253210740 1 -- 172.21.15.78:0/2918689576 shutdown_connections 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.314+0000 7f2253210740 1 -- 172.21.15.78:0/2918689576 wait complete. 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.314+0000 7f2253210740 1 -- start start 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.314+0000 7f2253210740 1 Processor -- start 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f2253210740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf497ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f2253210740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 0x55edf49780e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f2253210740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4978870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f2253210740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55edf4735700 con 0x55edf497f190 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f2253210740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55edf486f160 con 0x55edf496e4d0 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f2253210740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55edf4974a40 con 0x55edf496edf0 2024-09-11T01:10:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f2253210740 10 client.0 _ll_register_callbacks cb 0x55edf48889e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f224ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf497ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f224ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf497ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36980/0 (socket says 172.21.15.78:36980) 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f224ad9d640 1 -- 172.21.15.78:0/1599745872 learned_addr learned my addr 172.21.15.78:0/1599745872 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f2243fff640 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 0x55edf49780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f224ad9d640 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4978870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f224ad9d640 1 -- 172.21.15.78:0/1599745872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 msgr2=0x55edf49780e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f224ad9d640 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 0x55edf49780e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f224ad9d640 1 -- 172.21.15.78:0/1599745872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 msgr2=0x55edf4978870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f224ad9d640 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4978870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.315+0000 7f224ad9d640 1 -- 172.21.15.78:0/1599745872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf489cc90 con 0x55edf496e4d0 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f224ad9d640 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf497ec50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2244023040 tx=0x7f2244020cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27b0aae66085c47e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2231ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2231ffb640 1 -- 172.21.15.78:0/1599745872 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244032020 con 0x55edf496e4d0 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2231ffb640 1 -- 172.21.15.78:0/1599745872 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f2244003970 con 0x55edf496e4d0 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2253210740 1 -- 172.21.15.78:0/1599745872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f224400ebc0 con 0x55edf496e4d0 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2231ffb640 1 -- 172.21.15.78:0/1599745872 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2244025040 con 0x55edf496e4d0 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2253210740 10 client.26909 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2253210740 1 -- 172.21.15.78:0/1599745872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55edf4989410 con 0x55edf496e4d0 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2231ffb640 1 -- 172.21.15.78:0/1599745872 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f224400ebc0 con 0x55edf496e4d0 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2231ffb640 1 -- 172.21.15.78:0/1599745872 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f224402b050 con 0x55edf496e4d0 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2253210740 10 client.26909 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2253210740 10 client.26909 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2253210740 20 client.26909 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2253210740 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55edf498a7c0 0x55edf498cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2253210740 4 client.26909 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:10:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.316+0000 7f2253210740 1 -- 172.21.15.78:0/1599745872 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edf48e6df0 con 0x55edf498a7c0 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.317+0000 7f2243fff640 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55edf498a7c0 0x55edf498cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.317+0000 7f2243fff640 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55edf498a7c0 0x55edf498cbb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c812df077db1090 server_cookie=50a73ff3e613b1dc in_seq=0 out_seq=0 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.317+0000 7f2231ffb640 10 client.26909 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2231ffb640 1 -- 172.21.15.78:0/1599745872 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55edf48e6df0 con 0x55edf498a7c0 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2231ffb640 10 client.26909 handle_command_reply: tid=0 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2253210740 1 client.26909 shutdown 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2253210740 1 -- 172.21.15.78:0/1599745872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 msgr2=0x55edf497ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2253210740 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf497ec50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2244023040 tx=0x7f2244020cf0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2253210740 1 -- 172.21.15.78:0/1599745872 shutdown_connections 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2253210740 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55edf498a7c0 0x55edf498cbb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2253210740 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4978870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2253210740 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 0x55edf49780e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2253210740 1 --2- 172.21.15.78:0/1599745872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf497ec50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2231ffb640 0 client.26909 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:10:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2253210740 1 -- 172.21.15.78:0/1599745872 >> 172.21.15.78:0/1599745872 conn(0x55edf496c300 msgr2=0x55edf4971e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2253210740 1 -- 172.21.15.78:0/1599745872 shutdown_connections 2024-09-11T01:10:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2253210740 1 -- 172.21.15.78:0/1599745872 wait complete. 2024-09-11T01:10:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.318+0000 7f2253210740 20 client.26909 trim_cache size 0 max 16384 2024-09-11T01:10:32.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f2253210740 1 Processor -- start 2024-09-11T01:10:32.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f2253210740 1 -- start start 2024-09-11T01:10:32.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f2253210740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf49921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f2253210740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 0x55edf4992720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f2253210740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4971610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f2253210740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1ac3c0 con 0x55edf497f190 2024-09-11T01:10:32.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f2253210740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1adfc0 con 0x55edf496e4d0 2024-09-11T01:10:32.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f2253210740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1aca90 con 0x55edf496edf0 2024-09-11T01:10:32.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f224ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf49921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f224ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4971610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f224ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf49921e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36992/0 (socket says 172.21.15.78:36992) 2024-09-11T01:10:32.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f224ad9d640 1 -- 172.21.15.78:0/1666272128 learned_addr learned my addr 172.21.15.78:0/1666272128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:32.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.332+0000 7f2243fff640 1 --2- 172.21.15.78:0/1666272128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 0x55edf4992720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.333+0000 7f224ad9d640 1 -- 172.21.15.78:0/1666272128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 msgr2=0x55edf4992720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:32.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.333+0000 7f224ad9d640 1 --2- 172.21.15.78:0/1666272128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 0x55edf4992720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.333+0000 7f224ad9d640 1 -- 172.21.15.78:0/1666272128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 msgr2=0x55edf4971610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:32.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.333+0000 7f224ad9d640 1 --2- 172.21.15.78:0/1666272128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4971610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.333+0000 7f224ad9d640 1 -- 172.21.15.78:0/1666272128 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf496ec20 con 0x55edf496e4d0 2024-09-11T01:10:32.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.333+0000 7f224ad9d640 1 --2- 172.21.15.78:0/1666272128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf49921e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f224402ef50 tx=0x7f2244005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d87ddf64fdc0f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.333+0000 7f2233fff640 1 -- 172.21.15.78:0/1666272128 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244032070 con 0x55edf496e4d0 2024-09-11T01:10:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.333+0000 7f2233fff640 1 -- 172.21.15.78:0/1666272128 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2244003e20 con 0x55edf496e4d0 2024-09-11T01:10:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.333+0000 7f2233fff640 1 -- 172.21.15.78:0/1666272128 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224400d4d0 con 0x55edf496e4d0 2024-09-11T01:10:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.334+0000 7f2253210740 1 -- 172.21.15.78:0/1666272128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 msgr2=0x55edf49921e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.334+0000 7f2253210740 1 --2- 172.21.15.78:0/1666272128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf49921e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f224402ef50 tx=0x7f2244005db0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.334+0000 7f2253210740 1 -- 172.21.15.78:0/1666272128 shutdown_connections 2024-09-11T01:10:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.334+0000 7f2253210740 1 --2- 172.21.15.78:0/1666272128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf4971610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.334+0000 7f2253210740 1 --2- 172.21.15.78:0/1666272128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496edf0 0x55edf4992720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.334+0000 7f2253210740 1 --2- 172.21.15.78:0/1666272128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496e4d0 0x55edf49921e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.334+0000 7f2253210740 1 -- 172.21.15.78:0/1666272128 >> 172.21.15.78:0/1666272128 conn(0x55edf496c300 msgr2=0x55edf496b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.334+0000 7f2253210740 1 -- 172.21.15.78:0/1666272128 shutdown_connections 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.334+0000 7f2253210740 1 -- 172.21.15.78:0/1666272128 wait complete. 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.334+0000 7f2253210740 1 -- start start 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.334+0000 7f2253210740 1 Processor -- start 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.334+0000 7f2253210740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496e4d0 0x55edf49856a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2253210740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496edf0 0x55edf4985c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2253210740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf49f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2253210740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1ad830 con 0x55edf497f190 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2253210740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1ad160 con 0x55edf496edf0 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2253210740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1aa3a0 con 0x55edf496e4d0 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2253210740 10 client.0 _ll_register_callbacks cb 0x7f2244023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f224ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf49f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f224ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf49f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46780/0 (socket says 172.21.15.78:46780) 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2243fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496edf0 0x55edf4985c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f224ad9d640 1 -- 172.21.15.78:0/3049314822 learned_addr learned my addr 172.21.15.78:0/3049314822 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f224ad9d640 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496e4d0 0x55edf49856a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2243fff640 1 -- 172.21.15.78:0/3049314822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496e4d0 msgr2=0x55edf49856a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2243fff640 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496e4d0 0x55edf49856a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2243fff640 1 -- 172.21.15.78:0/3049314822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 msgr2=0x55edf49f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2243fff640 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf49f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2243fff640 1 -- 172.21.15.78:0/3049314822 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf4978be0 con 0x55edf496edf0 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f224ad9d640 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf49f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2243fff640 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496edf0 0x55edf4985c20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f224c06b6d0 tx=0x7f2234010c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff7a2b65dd8995ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2231ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2231ffb640 1 -- 172.21.15.78:0/3049314822 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234004230 con 0x55edf496edf0 2024-09-11T01:10:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2231ffb640 1 -- 172.21.15.78:0/3049314822 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f22340043d0 con 0x55edf496edf0 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.335+0000 7f2253210740 1 -- 172.21.15.78:0/3049314822 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2244003e20 con 0x55edf496edf0 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.336+0000 7f2231ffb640 1 -- 172.21.15.78:0/3049314822 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f223401f8d0 con 0x55edf496edf0 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.336+0000 7f2253210740 10 client.26915 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.336+0000 7f2253210740 1 -- 172.21.15.78:0/3049314822 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55edf49f7e10 con 0x55edf496edf0 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.336+0000 7f2231ffb640 1 -- 172.21.15.78:0/3049314822 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2244003e20 con 0x55edf496edf0 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.336+0000 7f2231ffb640 1 -- 172.21.15.78:0/3049314822 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f2234023070 con 0x55edf496edf0 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.336+0000 7f2253210740 10 client.26915 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.336+0000 7f2253210740 10 client.26915 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.336+0000 7f2253210740 20 client.26915 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.336+0000 7f2253210740 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55edf49f8610 0x55edf49faa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.336+0000 7f2253210740 4 client.26915 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.336+0000 7f2253210740 1 -- 172.21.15.78:0/3049314822 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55edf48e28a0 con 0x55edf49f8610 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.336+0000 7f224ad9d640 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55edf49f8610 0x55edf49faa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.337+0000 7f2231ffb640 10 client.26915 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:10:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.337+0000 7f224ad9d640 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55edf49f8610 0x55edf49faa30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bed49e9bf6a27e07 server_cookie=c4b582883e46ad93 in_seq=0 out_seq=0 2024-09-11T01:10:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.337+0000 7f2231ffb640 1 -- 172.21.15.78:0/3049314822 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55edf48e28a0 con 0x55edf49f8610 2024-09-11T01:10:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.337+0000 7f2231ffb640 10 client.26915 handle_command_reply: tid=0 2024-09-11T01:10:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.337+0000 7f2253210740 1 client.26915 shutdown 2024-09-11T01:10:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.337+0000 7f2253210740 1 -- 172.21.15.78:0/3049314822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496edf0 msgr2=0x55edf4985c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.337+0000 7f2253210740 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496edf0 0x55edf4985c20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f224c06b6d0 tx=0x7f2234010c30 comp rx=0 tx=0).stop 2024-09-11T01:10:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2253210740 1 -- 172.21.15.78:0/3049314822 shutdown_connections 2024-09-11T01:10:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2253210740 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55edf49f8610 0x55edf49faa30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2253210740 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edf497f190 0x55edf49f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2231ffb640 0 client.26915 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:10:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2253210740 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55edf496edf0 0x55edf4985c20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2253210740 1 --2- 172.21.15.78:0/3049314822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edf496e4d0 0x55edf49856a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2253210740 1 -- 172.21.15.78:0/3049314822 >> 172.21.15.78:0/3049314822 conn(0x55edf496c300 msgr2=0x55edf496ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:32.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2253210740 1 -- 172.21.15.78:0/3049314822 shutdown_connections 2024-09-11T01:10:32.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2253210740 1 -- 172.21.15.78:0/3049314822 wait complete. 2024-09-11T01:10:32.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2253210740 20 client.26915 trim_cache size 0 max 16384 2024-09-11T01:10:32.345 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:10:32.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2250e3b640 1 -- 172.21.15.78:0/1638711675 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f22380b5450 msgr2=0x7f22380b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:32.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2250e3b640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f22380b5450 0x7f22380b7910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f223c103430 tx=0x7f223c0d5040 comp rx=0 tx=0).stop 2024-09-11T01:10:32.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2250e3b640 1 -- 172.21.15.78:0/1638711675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f224c101c90 msgr2=0x7f224c19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:32.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2250e3b640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f224c101c90 0x7f224c19a0d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f223c035040 tx=0x7f223c032d50 comp rx=0 tx=0).stop 2024-09-11T01:10:32.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2250e3b640 1 -- 172.21.15.78:0/1638711675 shutdown_connections 2024-09-11T01:10:32.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2250e3b640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f22380b5450 0x7f22380b7910 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2250e3b640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f224c108020 0x7f224c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2250e3b640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f224c1076a0 0x7f224c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2250e3b640 1 --2- 172.21.15.78:0/1638711675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f224c101c90 0x7f224c19a0d0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:32.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.338+0000 7f2250e3b640 1 -- 172.21.15.78:0/1638711675 >> 172.21.15.78:0/1638711675 conn(0x7f224c0fd700 msgr2=0x7f224c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:32.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.339+0000 7f2250e3b640 1 -- 172.21.15.78:0/1638711675 shutdown_connections 2024-09-11T01:10:32.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:32.339+0000 7f2250e3b640 1 -- 172.21.15.78:0/1638711675 wait complete. 2024-09-11T01:10:32.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 799417 5.66 MB/sec execute 520 sec latency 44.293 ms 2024-09-11T01:10:32.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:32 smithi078 ceph-mon[31315]: pgmap v683: 97 pgs: 97 active+clean; 634 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 288 op/s 2024-09-11T01:10:32.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:32 smithi110 ceph-mon[34712]: pgmap v683: 97 pgs: 97 active+clean; 634 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 288 op/s 2024-09-11T01:10:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:32 smithi112 ceph-mon[35384]: pgmap v683: 97 pgs: 97 active+clean; 634 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 288 op/s 2024-09-11T01:10:33.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 801292 5.66 MB/sec execute 521 sec latency 26.217 ms 2024-09-11T01:10:33.719 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:10:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:10:34.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:34 smithi078 ceph-mon[31315]: pgmap v684: 97 pgs: 97 active+clean; 636 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-09-11T01:10:34.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 802854 5.67 MB/sec execute 522 sec latency 17.729 ms 2024-09-11T01:10:34.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:34 smithi110 ceph-mon[34712]: pgmap v684: 97 pgs: 97 active+clean; 636 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-09-11T01:10:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:34 smithi112 ceph-mon[35384]: pgmap v684: 97 pgs: 97 active+clean; 636 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-09-11T01:10:34.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.806+0000 7f0d126ec640 1 -- 172.21.15.78:0/603190581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c108020 msgr2=0x7f0d0c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:34.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.806+0000 7f0d126ec640 1 --2- 172.21.15.78:0/603190581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c108020 0x7f0d0c10e820 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009ca0 tx=0x7f0d0402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.806+0000 7f0d126ec640 1 -- 172.21.15.78:0/603190581 shutdown_connections 2024-09-11T01:10:34.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.806+0000 7f0d126ec640 1 --2- 172.21.15.78:0/603190581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c108020 0x7f0d0c10e820 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.806+0000 7f0d126ec640 1 --2- 172.21.15.78:0/603190581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c1076a0 0x7f0d0c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.806+0000 7f0d126ec640 1 --2- 172.21.15.78:0/603190581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d0c101c90 0x7f0d0c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.806+0000 7f0d126ec640 1 -- 172.21.15.78:0/603190581 >> 172.21.15.78:0/603190581 conn(0x7f0d0c0fd6c0 msgr2=0x7f0d0c0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:34.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.806+0000 7f0d126ec640 1 -- 172.21.15.78:0/603190581 shutdown_connections 2024-09-11T01:10:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.806+0000 7f0d126ec640 1 -- 172.21.15.78:0/603190581 wait complete. 2024-09-11T01:10:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.807+0000 7f0d126ec640 1 Processor -- start 2024-09-11T01:10:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.807+0000 7f0d126ec640 1 -- start start 2024-09-11T01:10:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.807+0000 7f0d126ec640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d0c101c90 0x7f0d0c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.807+0000 7f0d126ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c1076a0 0x7f0d0c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.807+0000 7f0d126ec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c108020 0x7f0d0c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d126ec640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c111cf0 con 0x7f0d0c108020 2024-09-11T01:10:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d126ec640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c111b70 con 0x7f0d0c101c90 2024-09-11T01:10:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d126ec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c111e70 con 0x7f0d0c1076a0 2024-09-11T01:10:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d10c89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c108020 0x7f0d0c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d0c101c90 0x7f0d0c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d0c101c90 0x7f0d0c199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39918/0 (socket says 172.21.15.78:39918) 2024-09-11T01:10:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d10c89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c108020 0x7f0d0c19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50936/0 (socket says 172.21.15.78:50936) 2024-09-11T01:10:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d0bfff640 1 -- 172.21.15.78:0/4225033321 learned_addr learned my addr 172.21.15.78:0/4225033321 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d03fff640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c1076a0 0x7f0d0c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d0bfff640 1 -- 172.21.15.78:0/4225033321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c1076a0 msgr2=0x7f0d0c199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c1076a0 0x7f0d0c199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d0bfff640 1 -- 172.21.15.78:0/4225033321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c108020 msgr2=0x7f0d0c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c108020 0x7f0d0c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d0bfff640 1 -- 172.21.15.78:0/4225033321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c19f3e0 con 0x7f0d0c101c90 2024-09-11T01:10:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d0c101c90 0x7f0d0c199330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00e9f0 tx=0x7f0cfc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d09ffb640 1 -- 172.21.15.78:0/4225033321 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc00cd80 con 0x7f0d0c101c90 2024-09-11T01:10:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d09ffb640 1 -- 172.21.15.78:0/4225033321 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cfc0108e0 con 0x7f0d0c101c90 2024-09-11T01:10:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d09ffb640 1 -- 172.21.15.78:0/4225033321 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc018980 con 0x7f0d0c101c90 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.808+0000 7f0d126ec640 1 -- 172.21.15.78:0/4225033321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0c19f6d0 con 0x7f0d0c101c90 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.809+0000 7f0d126ec640 1 -- 172.21.15.78:0/4225033321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0c1a5f60 con 0x7f0d0c101c90 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.811+0000 7f0d09ffb640 1 -- 172.21.15.78:0/4225033321 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0cfc010430 con 0x7f0d0c101c90 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.811+0000 7f0d14ac1740 1 Processor -- start 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.811+0000 7f0d14ac1740 1 -- start start 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.811+0000 7f0d14ac1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.811+0000 7f0d14ac1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3bdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d14ac1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3be190 0x56260b3c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d14ac1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56260b323ea0 con 0x56260b3be190 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d14ac1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56260b3132d0 con 0x56260b3b18e0 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d14ac1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56260b3170e0 con 0x56260b3b2200 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3be190 0x56260b3c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d10c89640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d10c89640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39922/0 (socket says 172.21.15.78:39922) 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3bdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d10c89640 1 -- 172.21.15.78:0/2925026838 learned_addr learned my addr 172.21.15.78:0/2925026838 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d03fff640 1 --2- 172.21.15.78:0/2925026838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3be190 0x56260b3c0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50946/0 (socket says 172.21.15.78:50946) 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d03fff640 1 -- 172.21.15.78:0/2925026838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 msgr2=0x56260b3bdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d03fff640 1 --2- 172.21.15.78:0/2925026838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3bdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d03fff640 1 -- 172.21.15.78:0/2925026838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 msgr2=0x56260b3b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d03fff640 1 --2- 172.21.15.78:0/2925026838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d03fff640 1 -- 172.21.15.78:0/2925026838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260b071650 con 0x56260b3be190 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d03fff640 1 --2- 172.21.15.78:0/2925026838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3be190 0x56260b3c0580 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009bf0 tx=0x7f0cf4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea67013e778ffa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d09ffb640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0cf80b54d0 0x7f0cf80b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d027fc640 1 -- 172.21.15.78:0/2925026838 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4004050 con 0x56260b3be190 2024-09-11T01:10:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d027fc640 1 -- 172.21.15.78:0/2925026838 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf40054d0 con 0x56260b3be190 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.812+0000 7f0d027fc640 1 -- 172.21.15.78:0/2925026838 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4004ca0 con 0x56260b3be190 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.813+0000 7f0d09ffb640 1 -- 172.21.15.78:0/4225033321 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f0cfc0e6260 con 0x7f0d0c101c90 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.813+0000 7f0d10c89640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0cf80b54d0 0x7f0cf80b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.813+0000 7f0d14ac1740 1 -- 172.21.15.78:0/2925026838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3be190 msgr2=0x56260b3c0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.813+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/2925026838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3be190 0x56260b3c0580 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009bf0 tx=0x7f0cf4011810 comp rx=0 tx=0).stop 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.814+0000 7f0d10c89640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0cf80b54d0 0x7f0cf80b7990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009a60 tx=0x7f0d04005d80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.814+0000 7f0d14ac1740 1 -- 172.21.15.78:0/2925026838 shutdown_connections 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.814+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/2925026838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3be190 0x56260b3c0580 secure :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009bf0 tx=0x7f0cf4011810 comp rx=0 tx=0).stop 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.814+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/2925026838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3bdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.814+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/2925026838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.814+0000 7f0d14ac1740 1 -- 172.21.15.78:0/2925026838 >> 172.21.15.78:0/2925026838 conn(0x56260b3af710 msgr2=0x56260b3b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.814+0000 7f0d14ac1740 1 -- 172.21.15.78:0/2925026838 shutdown_connections 2024-09-11T01:10:34.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.814+0000 7f0d14ac1740 1 -- 172.21.15.78:0/2925026838 wait complete. 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.814+0000 7f0d14ac1740 1 -- start start 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d14ac1740 1 Processor -- start 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d14ac1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3bd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d14ac1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3bb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d14ac1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 0x56260b3bba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d14ac1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56260b174700 con 0x56260b3bb5f0 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d14ac1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56260b2ae160 con 0x56260b3b18e0 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d14ac1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56260b3c0e10 con 0x56260b3b2200 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d14ac1740 10 client.0 _ll_register_callbacks cb 0x56260b2c79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3bd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d10c89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 0x56260b3bba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d10c89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 0x56260b3bba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50962/0 (socket says 172.21.15.78:50962) 2024-09-11T01:10:34.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3bb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3bd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39932/0 (socket says 172.21.15.78:39932) 2024-09-11T01:10:34.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.815+0000 7f0d10c89640 1 -- 172.21.15.78:0/1766256615 learned_addr learned my addr 172.21.15.78:0/1766256615 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:34.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0d0bfff640 1 -- 172.21.15.78:0/1766256615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 msgr2=0x56260b3bb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:34.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/1766256615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3bb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0d0bfff640 1 -- 172.21.15.78:0/1766256615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 msgr2=0x56260b3bba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:34.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/1766256615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 0x56260b3bba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0d0bfff640 1 -- 172.21.15.78:0/1766256615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260b2dbc90 con 0x56260b3b18e0 2024-09-11T01:10:34.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0d10c89640 1 --2- 172.21.15.78:0/1766256615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 0x56260b3bba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:34.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/1766256615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3bd8b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0e3990 tx=0x7f0cfc010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f11538958c2ea5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:34.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0ceb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:10:34.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/1766256615 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0abee0 con 0x56260b3b18e0 2024-09-11T01:10:34.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0d14ac1740 1 -- 172.21.15.78:0/1766256615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf40054d0 con 0x56260b3b18e0 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/1766256615 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f0cfc0abbd0 con 0x56260b3b18e0 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/1766256615 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc0e3e20 con 0x56260b3b18e0 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0d14ac1740 10 client.26927 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0d14ac1740 1 -- 172.21.15.78:0/1766256615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56260b3d1160 con 0x56260b3b18e0 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.816+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/1766256615 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf40054d0 con 0x56260b3b18e0 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.817+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/1766256615 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f0cfc0a80c0 con 0x56260b3b18e0 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.817+0000 7f0d14ac1740 10 client.26927 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.817+0000 7f0d14ac1740 10 client.26927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.817+0000 7f0d14ac1740 20 client.26927 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.817+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/1766256615 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56260b3d2550 0x56260b3d49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.817+0000 7f0d14ac1740 4 client.26927 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.817+0000 7f0d14ac1740 1 -- 172.21.15.78:0/1766256615 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56260b325df0 con 0x56260b3d2550 2024-09-11T01:10:34.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.817+0000 7f0d03fff640 1 --2- 172.21.15.78:0/1766256615 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56260b3d2550 0x56260b3d49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.817+0000 7f0d03fff640 1 --2- 172.21.15.78:0/1766256615 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56260b3d2550 0x56260b3d49b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf09f0168ef8123 server_cookie=9403586dff3a3d2a in_seq=0 out_seq=0 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.817+0000 7f0ceb7fe640 10 client.26927 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.818+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/1766256615 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56260b325df0 con 0x56260b3d2550 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.818+0000 7f0ceb7fe640 10 client.26927 handle_command_reply: tid=0 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.818+0000 7f0d14ac1740 1 client.26927 shutdown 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.818+0000 7f0d14ac1740 1 -- 172.21.15.78:0/1766256615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 msgr2=0x56260b3bd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.818+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/1766256615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3bd8b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0e3990 tx=0x7f0cfc010cc0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.818+0000 7f0d14ac1740 1 -- 172.21.15.78:0/1766256615 shutdown_connections 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.818+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/1766256615 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56260b3d2550 0x56260b3d49b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.819+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/1766256615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 0x56260b3bba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.819+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/1766256615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3bb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.819+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/1766256615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3bd8b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.819+0000 7f0ceb7fe640 0 client.26927 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:34.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.819+0000 7f0d14ac1740 1 -- 172.21.15.78:0/1766256615 >> 172.21.15.78:0/1766256615 conn(0x56260b3af710 msgr2=0x56260b3aa420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.819+0000 7f0d14ac1740 1 -- 172.21.15.78:0/1766256615 shutdown_connections 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.819+0000 7f0d14ac1740 1 -- 172.21.15.78:0/1766256615 wait complete. 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.819+0000 7f0d14ac1740 20 client.26927 trim_cache size 0 max 16384 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.832+0000 7f0d14ac1740 1 Processor -- start 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.832+0000 7f0d14ac1740 1 -- start start 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.832+0000 7f0d14ac1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3bcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.832+0000 7f0d14ac1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3da280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.832+0000 7f0d14ac1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 0x56260b3da7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d14ac1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1ac4d0 con 0x56260b3bb5f0 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d14ac1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1ae0d0 con 0x56260b3b18e0 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d14ac1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1acba0 con 0x56260b3b2200 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d10c89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 0x56260b3da7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3bcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3da280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3bcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39936/0 (socket says 172.21.15.78:39936) 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d0bfff640 1 -- 172.21.15.78:0/263178044 learned_addr learned my addr 172.21.15.78:0/263178044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d10c89640 1 -- 172.21.15.78:0/263178044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 msgr2=0x56260b3da280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d10c89640 1 --2- 172.21.15.78:0/263178044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3da280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d10c89640 1 -- 172.21.15.78:0/263178044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 msgr2=0x56260b3bcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d10c89640 1 --2- 172.21.15.78:0/263178044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3bcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d10c89640 1 -- 172.21.15.78:0/263178044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260b3b0190 con 0x56260b3bb5f0 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d03fff640 1 --2- 172.21.15.78:0/263178044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3da280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/263178044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3bcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d10c89640 1 --2- 172.21.15.78:0/263178044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 0x56260b3da7c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009ca0 tx=0x7f0d04041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c73be8e940db8d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d01ffb640 1 -- 172.21.15.78:0/263178044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04003970 con 0x56260b3bb5f0 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d01ffb640 1 -- 172.21.15.78:0/263178044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d04036050 con 0x56260b3bb5f0 2024-09-11T01:10:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.833+0000 7f0d01ffb640 1 -- 172.21.15.78:0/263178044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04004740 con 0x56260b3bb5f0 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.834+0000 7f0d14ac1740 1 -- 172.21.15.78:0/263178044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 msgr2=0x56260b3da7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.834+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/263178044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 0x56260b3da7c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009ca0 tx=0x7f0d04041000 comp rx=0 tx=0).stop 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.834+0000 7f0d14ac1740 1 -- 172.21.15.78:0/263178044 shutdown_connections 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.834+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/263178044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3bb5f0 0x56260b3da7c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.834+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/263178044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3da280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.834+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/263178044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3b18e0 0x56260b3bcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.834+0000 7f0d14ac1740 1 -- 172.21.15.78:0/263178044 >> 172.21.15.78:0/263178044 conn(0x56260b3af710 msgr2=0x56260b3bc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.834+0000 7f0d14ac1740 1 -- 172.21.15.78:0/263178044 shutdown_connections 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.834+0000 7f0d14ac1740 1 -- 172.21.15.78:0/263178044 wait complete. 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d14ac1740 1 -- start start 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d14ac1740 1 Processor -- start 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d14ac1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3b18e0 0x56260b3cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d14ac1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d14ac1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3bb5f0 0x56260b43e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d14ac1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1ad940 con 0x56260b3b18e0 2024-09-11T01:10:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d14ac1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1ad270 con 0x56260b3bb5f0 2024-09-11T01:10:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d14ac1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1aa4d0 con 0x56260b3b2200 2024-09-11T01:10:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d14ac1740 10 client.0 _ll_register_callbacks cb 0x7f0cfc009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3b18e0 0x56260b3cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d10c89640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3bb5f0 0x56260b43e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3b18e0 0x56260b3cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50988/0 (socket says 172.21.15.78:50988) 2024-09-11T01:10:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d10c89640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3bb5f0 0x56260b43e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39952/0 (socket says 172.21.15.78:39952) 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d0bfff640 1 -- 172.21.15.78:0/4194576783 learned_addr learned my addr 172.21.15.78:0/4194576783 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d03fff640 1 --2- 172.21.15.78:0/4194576783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d0bfff640 1 -- 172.21.15.78:0/4194576783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 msgr2=0x56260b3cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/4194576783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.835+0000 7f0d0bfff640 1 -- 172.21.15.78:0/4194576783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3bb5f0 msgr2=0x56260b43e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/4194576783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3bb5f0 0x56260b43e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0d0bfff640 1 -- 172.21.15.78:0/4194576783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260b43e9c0 con 0x56260b3b18e0 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/4194576783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3b18e0 0x56260b3cadc0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0097b0 tx=0x7f0cfc0abf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ade3c21bd038783 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0ceb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/4194576783 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0ac280 con 0x56260b3b18e0 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/4194576783 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f0cfc018b20 con 0x56260b3b18e0 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0d14ac1740 1 -- 172.21.15.78:0/4194576783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d04036050 con 0x56260b3b18e0 2024-09-11T01:10:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/4194576783 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc0ab440 con 0x56260b3b18e0 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0d14ac1740 10 client.17739 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0d14ac1740 1 -- 172.21.15.78:0/4194576783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56260b43fbd0 con 0x56260b3b18e0 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/4194576783 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d04036050 con 0x56260b3b18e0 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/4194576783 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f0cfc0cfa10 con 0x56260b3b18e0 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0d14ac1740 10 client.17739 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0d14ac1740 10 client.17739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0d14ac1740 20 client.17739 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/4194576783 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56260b440240 0x56260b442660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0d14ac1740 4 client.17739 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.836+0000 7f0d14ac1740 1 -- 172.21.15.78:0/4194576783 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56260b3218a0 con 0x56260b440240 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d03fff640 1 --2- 172.21.15.78:0/4194576783 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56260b440240 0x56260b442660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0ceb7fe640 10 client.17739 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d03fff640 1 --2- 172.21.15.78:0/4194576783 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56260b440240 0x56260b442660 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2272a1569db28d23 server_cookie=680faa3518a2054e in_seq=0 out_seq=0 2024-09-11T01:10:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/4194576783 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56260b3218a0 con 0x56260b440240 2024-09-11T01:10:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0ceb7fe640 10 client.17739 handle_command_reply: tid=0 2024-09-11T01:10:34.998 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:10:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d14ac1740 1 client.17739 shutdown 2024-09-11T01:10:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d14ac1740 1 -- 172.21.15.78:0/4194576783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3b18e0 msgr2=0x56260b3cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/4194576783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3b18e0 0x56260b3cadc0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0097b0 tx=0x7f0cfc0abf50 comp rx=0 tx=0).stop 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d14ac1740 1 -- 172.21.15.78:0/4194576783 shutdown_connections 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/4194576783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56260b3bb5f0 0x56260b43e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/4194576783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56260b3b2200 0x56260b3cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/4194576783 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56260b440240 0x56260b442660 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d14ac1740 1 --2- 172.21.15.78:0/4194576783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56260b3b18e0 0x56260b3cadc0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0ceb7fe640 0 client.17739 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d14ac1740 1 -- 172.21.15.78:0/4194576783 >> 172.21.15.78:0/4194576783 conn(0x56260b3af710 msgr2=0x56260b3bcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d14ac1740 1 -- 172.21.15.78:0/4194576783 shutdown_connections 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d14ac1740 1 -- 172.21.15.78:0/4194576783 wait complete. 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.837+0000 7f0d14ac1740 20 client.17739 trim_cache size 0 max 16384 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.838+0000 7f0d126ec640 1 -- 172.21.15.78:0/4225033321 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0cf80b54d0 msgr2=0x7f0cf80b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.838+0000 7f0d126ec640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0cf80b54d0 0x7f0cf80b7990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009a60 tx=0x7f0d04005d80 comp rx=0 tx=0).stop 2024-09-11T01:10:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.838+0000 7f0d126ec640 1 -- 172.21.15.78:0/4225033321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d0c101c90 msgr2=0x7f0d0c199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:35.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.838+0000 7f0d126ec640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d0c101c90 0x7f0d0c199330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00e9f0 tx=0x7f0cfc00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:10:35.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.838+0000 7f0d126ec640 1 -- 172.21.15.78:0/4225033321 shutdown_connections 2024-09-11T01:10:35.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.838+0000 7f0d126ec640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0cf80b54d0 0x7f0cf80b7990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:35.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.838+0000 7f0d126ec640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c108020 0x7f0d0c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:35.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.838+0000 7f0d126ec640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c1076a0 0x7f0d0c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:35.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.838+0000 7f0d126ec640 1 --2- 172.21.15.78:0/4225033321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d0c101c90 0x7f0d0c199330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:35.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.838+0000 7f0d126ec640 1 -- 172.21.15.78:0/4225033321 >> 172.21.15.78:0/4225033321 conn(0x7f0d0c0fd6c0 msgr2=0x7f0d0c106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:35.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.838+0000 7f0d126ec640 1 -- 172.21.15.78:0/4225033321 shutdown_connections 2024-09-11T01:10:35.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:34.838+0000 7f0d126ec640 1 -- 172.21.15.78:0/4225033321 wait complete. 2024-09-11T01:10:35.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 804709 5.67 MB/sec execute 523 sec latency 29.820 ms 2024-09-11T01:10:36.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 806345 5.69 MB/sec execute 524 sec latency 13.396 ms 2024-09-11T01:10:36.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:36 smithi110 ceph-mon[34712]: pgmap v685: 97 pgs: 97 active+clean; 644 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-09-11T01:10:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:36 smithi112 ceph-mon[35384]: pgmap v685: 97 pgs: 97 active+clean; 644 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-09-11T01:10:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:36 smithi078 ceph-mon[31315]: pgmap v685: 97 pgs: 97 active+clean; 644 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-09-11T01:10:36.971 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:10:36.972 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:10:37.235 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:10:37.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 808096 5.69 MB/sec execute 525 sec latency 17.654 ms 2024-09-11T01:10:37.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.738+0000 7f822a424640 1 -- 172.21.15.78:0/2986662846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8224102dc0 msgr2=0x7f82241031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.738+0000 7f822a424640 1 --2- 172.21.15.78:0/2986662846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8224102dc0 0x7f82241031a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8214009900 tx=0x7f821402a410 comp rx=0 tx=0).stop 2024-09-11T01:10:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.738+0000 7f822a424640 1 -- 172.21.15.78:0/2986662846 shutdown_connections 2024-09-11T01:10:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.738+0000 7f822a424640 1 --2- 172.21.15.78:0/2986662846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8224109150 0x7f822410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.738+0000 7f822a424640 1 --2- 172.21.15.78:0/2986662846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82241087d0 0x7f8224108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.738+0000 7f822a424640 1 --2- 172.21.15.78:0/2986662846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8224102dc0 0x7f82241031a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.738+0000 7f822a424640 1 -- 172.21.15.78:0/2986662846 >> 172.21.15.78:0/2986662846 conn(0x7f82240fe830 msgr2=0x7f8224100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.738+0000 7f822a424640 1 -- 172.21.15.78:0/2986662846 shutdown_connections 2024-09-11T01:10:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.738+0000 7f822a424640 1 -- 172.21.15.78:0/2986662846 wait complete. 2024-09-11T01:10:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.739+0000 7f822a424640 1 Processor -- start 2024-09-11T01:10:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.739+0000 7f822a424640 1 -- start start 2024-09-11T01:10:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f822a424640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8224102dc0 0x7f82241992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f822a424640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82241087d0 0x7f8224199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f822a424640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8224109150 0x7f822419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f822a424640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8224112e10 con 0x7f8224109150 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f822a424640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8224112c90 con 0x7f8224102dc0 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f822a424640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8224112f90 con 0x7f82241087d0 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f82289c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8224109150 0x7f822419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f82289c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8224109150 0x7f822419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51000/0 (socket says 172.21.15.78:51000) 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f82289c1640 1 -- 172.21.15.78:0/1659895216 learned_addr learned my addr 172.21.15.78:0/1659895216 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f8223fff640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8224102dc0 0x7f82241992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f82237fe640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82241087d0 0x7f8224199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f82289c1640 1 -- 172.21.15.78:0/1659895216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82241087d0 msgr2=0x7f8224199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f82289c1640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82241087d0 0x7f8224199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f82289c1640 1 -- 172.21.15.78:0/1659895216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8224102dc0 msgr2=0x7f82241992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f82289c1640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8224102dc0 0x7f82241992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f82289c1640 1 -- 172.21.15.78:0/1659895216 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822419f370 con 0x7f8224109150 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.740+0000 7f82289c1640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8224109150 0x7f822419eba0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f821c00efd0 tx=0x7f821c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.741+0000 7f82217fa640 1 -- 172.21.15.78:0/1659895216 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c003b70 con 0x7f8224109150 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.741+0000 7f82217fa640 1 -- 172.21.15.78:0/1659895216 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821c003d10 con 0x7f8224109150 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.741+0000 7f82217fa640 1 -- 172.21.15.78:0/1659895216 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c0108a0 con 0x7f8224109150 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.741+0000 7f822a424640 1 -- 172.21.15.78:0/1659895216 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822419f660 con 0x7f8224109150 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.741+0000 7f822a424640 1 -- 172.21.15.78:0/1659895216 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8224075380 con 0x7f8224109150 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.743+0000 7f82217fa640 1 -- 172.21.15.78:0/1659895216 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f821c01f020 con 0x7f8224109150 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f822c7f9740 1 Processor -- start 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f822c7f9740 1 -- start start 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f822c7f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb10212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f822c7f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb102d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f822c7f9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb102fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f822c7f9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cdb0f97e60 con 0x55cdb102d790 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f822c7f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cdb0f87290 con 0x55cdb1020ee0 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f822c7f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cdb0f8b0a0 con 0x55cdb1021800 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f82237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb102fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f8223fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb102d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f82237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb102fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51008/0 (socket says 172.21.15.78:51008) 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f82237fe640 1 -- 172.21.15.78:0/783318661 learned_addr learned my addr 172.21.15.78:0/783318661 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f82237fe640 1 --2- 172.21.15.78:0/783318661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb10212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f8223fff640 1 -- 172.21.15.78:0/783318661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 msgr2=0x55cdb10212c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f8223fff640 1 --2- 172.21.15.78:0/783318661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb10212c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f8223fff640 1 -- 172.21.15.78:0/783318661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 msgr2=0x55cdb102fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.744+0000 7f8223fff640 1 --2- 172.21.15.78:0/783318661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb102fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.745+0000 7f82217fa640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f820c0b9860 0x7f820c0bbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.745+0000 7f8223fff640 1 -- 172.21.15.78:0/783318661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdb0ce5930 con 0x55cdb1021800 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.745+0000 7f82237fe640 1 --2- 172.21.15.78:0/783318661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb102fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.745+0000 7f82237fe640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f820c0b9860 0x7f820c0bbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.745+0000 7f82217fa640 1 -- 172.21.15.78:0/1659895216 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f821c0e6100 con 0x7f8224109150 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.745+0000 7f8223fff640 1 --2- 172.21.15.78:0/783318661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb102d250 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8214009230 tx=0x7f8214032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e185b8789e11bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.745+0000 7f820a7fc640 1 -- 172.21.15.78:0/783318661 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214035040 con 0x55cdb1021800 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.745+0000 7f820a7fc640 1 -- 172.21.15.78:0/783318661 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8214004030 con 0x55cdb1021800 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.745+0000 7f820a7fc640 1 -- 172.21.15.78:0/783318661 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82140052e0 con 0x55cdb1021800 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.746+0000 7f822c7f9740 1 -- 172.21.15.78:0/783318661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 msgr2=0x55cdb102d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.746+0000 7f822c7f9740 1 --2- 172.21.15.78:0/783318661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb102d250 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8214009230 tx=0x7f8214032b20 comp rx=0 tx=0).stop 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.746+0000 7f82237fe640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f820c0b9860 0x7f820c0bbd20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55cdb1020e90 tx=0x7f8218009370 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.746+0000 7f822c7f9740 1 -- 172.21.15.78:0/783318661 shutdown_connections 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.746+0000 7f822c7f9740 1 --2- 172.21.15.78:0/783318661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb102fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.746+0000 7f822c7f9740 1 --2- 172.21.15.78:0/783318661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb102d250 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.746+0000 7f822c7f9740 1 --2- 172.21.15.78:0/783318661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb10212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.746+0000 7f822c7f9740 1 -- 172.21.15.78:0/783318661 >> 172.21.15.78:0/783318661 conn(0x55cdb101ed10 msgr2=0x55cdb101feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.746+0000 7f822c7f9740 1 -- 172.21.15.78:0/783318661 shutdown_connections 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.746+0000 7f822c7f9740 1 -- 172.21.15.78:0/783318661 wait complete. 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.746+0000 7f822c7f9740 1 -- start start 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.746+0000 7f822c7f9740 1 Processor -- start 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f822c7f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1020ee0 0x55cdb102a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f822c7f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1021800 0x55cdb102ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f822c7f9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb102b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f822c7f9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cdb0de8890 con 0x55cdb102d790 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f822c7f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cdb0f21d40 con 0x55cdb1021800 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f822c7f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cdb10303e0 con 0x55cdb1020ee0 2024-09-11T01:10:37.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f822c7f9740 10 client.0 _ll_register_callbacks cb 0x55cdb0f85c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f8223fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1020ee0 0x55cdb102a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f8223fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1020ee0 0x55cdb102a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:37048/0 (socket says 172.21.15.78:37048) 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f8223fff640 1 -- 172.21.15.78:0/3586799755 learned_addr learned my addr 172.21.15.78:0/3586799755 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f82289c1640 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb102b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f82237fe640 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1021800 0x55cdb102ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f8223fff640 1 -- 172.21.15.78:0/3586799755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1021800 msgr2=0x55cdb102ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f8223fff640 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1021800 0x55cdb102ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f8223fff640 1 -- 172.21.15.78:0/3586799755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 msgr2=0x55cdb102b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f8223fff640 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb102b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f8223fff640 1 -- 172.21.15.78:0/3586799755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdb1039790 con 0x55cdb1020ee0 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f82289c1640 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb102b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f82237fe640 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1021800 0x55cdb102ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f8223fff640 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1020ee0 0x55cdb102a6f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f821402a450 tx=0x7f8214032f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b19c2f17434166ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:37.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.747+0000 7f82037fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f82037fe640 1 -- 172.21.15.78:0/3586799755 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821404c070 con 0x55cdb1020ee0 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f82037fe640 1 -- 172.21.15.78:0/3586799755 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f82140314a0 con 0x55cdb1020ee0 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f822c7f9740 1 -- 172.21.15.78:0/3586799755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8214004030 con 0x55cdb1020ee0 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f82037fe640 1 -- 172.21.15.78:0/3586799755 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8214035040 con 0x55cdb1020ee0 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f822c7f9740 10 client.26872 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f822c7f9740 1 -- 172.21.15.78:0/3586799755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55cdb103c140 con 0x55cdb1020ee0 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f82037fe640 1 -- 172.21.15.78:0/3586799755 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8214004030 con 0x55cdb1020ee0 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f82037fe640 1 -- 172.21.15.78:0/3586799755 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f8214004960 con 0x55cdb1020ee0 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f822c7f9740 10 client.26872 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f822c7f9740 10 client.26872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f822c7f9740 20 client.26872 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f822c7f9740 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cdb103d490 0x55cdb103f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f822c7f9740 4 client.26872 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.748+0000 7f822c7f9740 1 -- 172.21.15.78:0/3586799755 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdb0f99db0 con 0x55cdb103d490 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.749+0000 7f82237fe640 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cdb103d490 0x55cdb103f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.749+0000 7f82037fe640 10 client.26872 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.749+0000 7f82237fe640 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cdb103d490 0x55cdb103f880 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c3b99ca08ee272f server_cookie=47f2259e35400a11 in_seq=0 out_seq=0 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f82037fe640 1 -- 172.21.15.78:0/3586799755 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cdb0f99db0 con 0x55cdb103d490 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f82037fe640 10 client.26872 handle_command_reply: tid=0 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f822c7f9740 1 client.26872 shutdown 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f822c7f9740 1 -- 172.21.15.78:0/3586799755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1020ee0 msgr2=0x55cdb102a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f822c7f9740 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1020ee0 0x55cdb102a6f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f821402a450 tx=0x7f8214032f40 comp rx=0 tx=0).stop 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f822c7f9740 1 -- 172.21.15.78:0/3586799755 shutdown_connections 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f822c7f9740 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cdb103d490 0x55cdb103f880 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f82037fe640 0 client.26872 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f822c7f9740 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb102b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f822c7f9740 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1021800 0x55cdb102ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f822c7f9740 1 --2- 172.21.15.78:0/3586799755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1020ee0 0x55cdb102a6f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f822c7f9740 1 -- 172.21.15.78:0/3586799755 >> 172.21.15.78:0/3586799755 conn(0x55cdb101ed10 msgr2=0x55cdb101df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:37.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f822c7f9740 1 -- 172.21.15.78:0/3586799755 shutdown_connections 2024-09-11T01:10:37.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f822c7f9740 1 -- 172.21.15.78:0/3586799755 wait complete. 2024-09-11T01:10:37.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.750+0000 7f822c7f9740 20 client.26872 trim_cache size 0 max 16384 2024-09-11T01:10:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.768+0000 7f822c7f9740 1 Processor -- start 2024-09-11T01:10:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.768+0000 7f822c7f9740 1 -- start start 2024-09-11T01:10:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f822c7f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb1028180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f822c7f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb102c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f822c7f9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb1045010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f822c7f9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f82241ac340 con 0x55cdb102d790 2024-09-11T01:10:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f822c7f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f82241adf40 con 0x55cdb1020ee0 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f822c7f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82241aca10 con 0x55cdb1021800 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f82237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb102c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f82237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb102c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:37056/0 (socket says 172.21.15.78:37056) 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f82237fe640 1 -- 172.21.15.78:0/3132271064 learned_addr learned my addr 172.21.15.78:0/3132271064 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f8223fff640 1 --2- 172.21.15.78:0/3132271064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb1028180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f82289c1640 1 --2- 172.21.15.78:0/3132271064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb1045010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f82237fe640 1 -- 172.21.15.78:0/3132271064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 msgr2=0x55cdb1028180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f82237fe640 1 --2- 172.21.15.78:0/3132271064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb1028180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f82237fe640 1 -- 172.21.15.78:0/3132271064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 msgr2=0x55cdb1045010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f82237fe640 1 --2- 172.21.15.78:0/3132271064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb1045010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f82237fe640 1 -- 172.21.15.78:0/3132271064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdb102ff70 con 0x55cdb1021800 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.769+0000 7f82237fe640 1 --2- 172.21.15.78:0/3132271064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb102c430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55cdb103ffb0 tx=0x7f82180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b00fe1830521510 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.770+0000 7f8209ffb640 1 -- 172.21.15.78:0/3132271064 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180035f0 con 0x55cdb1021800 2024-09-11T01:10:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.770+0000 7f8209ffb640 1 -- 172.21.15.78:0/3132271064 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8218003c40 con 0x55cdb1021800 2024-09-11T01:10:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.770+0000 7f8209ffb640 1 -- 172.21.15.78:0/3132271064 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821802bbe0 con 0x55cdb1021800 2024-09-11T01:10:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.771+0000 7f822c7f9740 1 -- 172.21.15.78:0/3132271064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 msgr2=0x55cdb102c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.771+0000 7f822c7f9740 1 --2- 172.21.15.78:0/3132271064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb102c430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55cdb103ffb0 tx=0x7f82180046c0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.771+0000 7f822c7f9740 1 -- 172.21.15.78:0/3132271064 shutdown_connections 2024-09-11T01:10:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.771+0000 7f822c7f9740 1 --2- 172.21.15.78:0/3132271064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb1045010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.771+0000 7f822c7f9740 1 --2- 172.21.15.78:0/3132271064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb102c430 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.771+0000 7f822c7f9740 1 --2- 172.21.15.78:0/3132271064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb1028180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.771+0000 7f822c7f9740 1 -- 172.21.15.78:0/3132271064 >> 172.21.15.78:0/3132271064 conn(0x55cdb101ed10 msgr2=0x55cdb102bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.771+0000 7f822c7f9740 1 -- 172.21.15.78:0/3132271064 shutdown_connections 2024-09-11T01:10:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.771+0000 7f822c7f9740 1 -- 172.21.15.78:0/3132271064 wait complete. 2024-09-11T01:10:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.771+0000 7f822c7f9740 1 -- start start 2024-09-11T01:10:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.772+0000 7f822c7f9740 1 Processor -- start 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.772+0000 7f822c7f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb1031a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.772+0000 7f822c7f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb1031f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.772+0000 7f8223fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb1031a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.772+0000 7f8223fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb1031a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:40014/0 (socket says 172.21.15.78:40014) 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.772+0000 7f8223fff640 1 -- 172.21.15.78:0/534902907 learned_addr learned my addr 172.21.15.78:0/534902907 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.772+0000 7f822c7f9740 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb10a9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.772+0000 7f822c7f9740 1 -- 172.21.15.78:0/534902907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f82241ad7b0 con 0x55cdb102d790 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.772+0000 7f822c7f9740 1 -- 172.21.15.78:0/534902907 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f82241ad0e0 con 0x55cdb1020ee0 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.772+0000 7f822c7f9740 1 -- 172.21.15.78:0/534902907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82241aa340 con 0x55cdb1021800 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.772+0000 7f82237fe640 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb1031f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.772+0000 7f822c7f9740 10 client.0 _ll_register_callbacks cb 0x7f8218009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.773+0000 7f82289c1640 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb10a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.773+0000 7f8223fff640 1 -- 172.21.15.78:0/534902907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 msgr2=0x55cdb1031f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.773+0000 7f8223fff640 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb1031f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.773+0000 7f8223fff640 1 -- 172.21.15.78:0/534902907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 msgr2=0x55cdb10a9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.773+0000 7f8223fff640 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb10a9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.773+0000 7f8223fff640 1 -- 172.21.15.78:0/534902907 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdb10a9750 con 0x55cdb1020ee0 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.773+0000 7f82289c1640 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb10a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.773+0000 7f8223fff640 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb1031a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8214005ec0 tx=0x7f8214004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5547406d1c91c494 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.773+0000 7f82037fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.774+0000 7f82037fe640 1 -- 172.21.15.78:0/534902907 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821404c030 con 0x55cdb1020ee0 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.774+0000 7f82037fe640 1 -- 172.21.15.78:0/534902907 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f8214031c70 con 0x55cdb1020ee0 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.774+0000 7f822c7f9740 1 -- 172.21.15.78:0/534902907 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8218003c40 con 0x55cdb1020ee0 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.774+0000 7f82037fe640 1 -- 172.21.15.78:0/534902907 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8214058030 con 0x55cdb1020ee0 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.774+0000 7f822c7f9740 10 client.26951 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.774+0000 7f822c7f9740 1 -- 172.21.15.78:0/534902907 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55cdb10aa6f0 con 0x55cdb1020ee0 2024-09-11T01:10:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.774+0000 7f82037fe640 1 -- 172.21.15.78:0/534902907 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8218003c40 con 0x55cdb1020ee0 2024-09-11T01:10:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.774+0000 7f82037fe640 1 -- 172.21.15.78:0/534902907 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f8214005390 con 0x55cdb1020ee0 2024-09-11T01:10:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.775+0000 7f822c7f9740 10 client.26951 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.775+0000 7f822c7f9740 10 client.26951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:10:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.775+0000 7f822c7f9740 20 client.26951 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.775+0000 7f822c7f9740 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cdb10aaf10 0x55cdb10ad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.775+0000 7f822c7f9740 4 client.26951 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:10:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.775+0000 7f822c7f9740 1 -- 172.21.15.78:0/534902907 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cdb0f95860 con 0x55cdb10aaf10 2024-09-11T01:10:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.775+0000 7f82237fe640 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cdb10aaf10 0x55cdb10ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.775+0000 7f82037fe640 10 client.26951 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.775+0000 7f82237fe640 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cdb10aaf10 0x55cdb10ad330 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4295048780c077 server_cookie=49463e2f20215858 in_seq=0 out_seq=0 2024-09-11T01:10:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f82037fe640 1 -- 172.21.15.78:0/534902907 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cdb0f95860 con 0x55cdb10aaf10 2024-09-11T01:10:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f82037fe640 10 client.26951 handle_command_reply: tid=0 2024-09-11T01:10:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f822c7f9740 1 client.26951 shutdown 2024-09-11T01:10:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f822c7f9740 1 -- 172.21.15.78:0/534902907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 msgr2=0x55cdb1031a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f822c7f9740 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb1031a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8214005ec0 tx=0x7f8214004600 comp rx=0 tx=0).stop 2024-09-11T01:10:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f822c7f9740 1 -- 172.21.15.78:0/534902907 shutdown_connections 2024-09-11T01:10:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f822c7f9740 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cdb10aaf10 0x55cdb10ad330 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f822c7f9740 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdb102d790 0x55cdb10a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f822c7f9740 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdb1021800 0x55cdb1031f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f822c7f9740 1 --2- 172.21.15.78:0/534902907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cdb1020ee0 0x55cdb1031a50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f822c7f9740 1 -- 172.21.15.78:0/534902907 >> 172.21.15.78:0/534902907 conn(0x55cdb101ed10 msgr2=0x55cdb102c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f82037fe640 0 client.26951 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.778+0000 7f822c7f9740 1 -- 172.21.15.78:0/534902907 shutdown_connections 2024-09-11T01:10:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.779+0000 7f822c7f9740 1 -- 172.21.15.78:0/534902907 wait complete. 2024-09-11T01:10:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.779+0000 7f822c7f9740 20 client.26951 trim_cache size 0 max 16384 2024-09-11T01:10:37.783 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:10:37.783 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:10:37.783 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "2f8adbde-7a68-4c03-95dd-9f3a43878c82", 2024-09-11T01:10:37.783 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:10:37.783 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:10:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.779+0000 7f822a424640 1 -- 172.21.15.78:0/1659895216 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f820c0b9860 msgr2=0x7f820c0bbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.779+0000 7f822a424640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f820c0b9860 0x7f820c0bbd20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55cdb1020e90 tx=0x7f8218009370 comp rx=0 tx=0).stop 2024-09-11T01:10:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.779+0000 7f822a424640 1 -- 172.21.15.78:0/1659895216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8224109150 msgr2=0x7f822419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.779+0000 7f822a424640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8224109150 0x7f822419eba0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f821c00efd0 tx=0x7f821c00c6a0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.780+0000 7f822a424640 1 -- 172.21.15.78:0/1659895216 shutdown_connections 2024-09-11T01:10:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.780+0000 7f822a424640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f820c0b9860 0x7f820c0bbd20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.780+0000 7f822a424640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8224109150 0x7f822419eba0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.780+0000 7f822a424640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82241087d0 0x7f8224199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.780+0000 7f822a424640 1 --2- 172.21.15.78:0/1659895216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8224102dc0 0x7f82241992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.780+0000 7f822a424640 1 -- 172.21.15.78:0/1659895216 >> 172.21.15.78:0/1659895216 conn(0x7f82240fe830 msgr2=0x7f8224100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.780+0000 7f822a424640 1 -- 172.21.15.78:0/1659895216 shutdown_connections 2024-09-11T01:10:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:37.780+0000 7f822a424640 1 -- 172.21.15.78:0/1659895216 wait complete. 2024-09-11T01:10:38.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:37 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:10:38.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:37 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:38.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:37 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:10:38.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:37 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:38.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:37 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:10:38.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:37 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:38.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:37 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:10:38.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:37 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:38.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:37 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:10:38.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:37 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:38.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:37 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:10:38.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:37 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:37 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:10:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:37 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:37 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:10:38.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:37 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:38.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:37 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:10:38.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:37 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:38.453 INFO:tasks.workunit.client.0.smithi078.stdout: 1 809770 5.70 MB/sec execute 526 sec latency 36.710 ms 2024-09-11T01:10:38.459 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:10:38.738 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:10:39.161 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:10:39.161 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.161 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:10:39.161 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:10:39.161 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:10:39.161 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:10:39.161 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.161 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.161 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:10:39.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:10:39.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:10:39.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:10:39.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:10:39.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:38 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:10:39.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:10:39.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.226 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:10:39.226 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.226 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:38 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.243+0000 7f0ae71cb640 1 -- 172.21.15.78:0/376699393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae0104570 msgr2=0x7f0ae010cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.243+0000 7f0ae71cb640 1 --2- 172.21.15.78:0/376699393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae0104570 0x7f0ae010cb50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009900 tx=0x7f0acc02a410 comp rx=0 tx=0).stop 2024-09-11T01:10:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.243+0000 7f0ae71cb640 1 -- 172.21.15.78:0/376699393 shutdown_connections 2024-09-11T01:10:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.243+0000 7f0ae71cb640 1 --2- 172.21.15.78:0/376699393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ae010d120 0x7f0ae010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.243+0000 7f0ae71cb640 1 --2- 172.21.15.78:0/376699393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae0104570 0x7f0ae010cb50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.243+0000 7f0ae71cb640 1 --2- 172.21.15.78:0/376699393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ae0103b70 0x7f0ae0103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.243+0000 7f0ae71cb640 1 -- 172.21.15.78:0/376699393 >> 172.21.15.78:0/376699393 conn(0x7f0ae00fd700 msgr2=0x7f0ae00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.243+0000 7f0ae71cb640 1 -- 172.21.15.78:0/376699393 shutdown_connections 2024-09-11T01:10:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.243+0000 7f0ae71cb640 1 -- 172.21.15.78:0/376699393 wait complete. 2024-09-11T01:10:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae71cb640 1 Processor -- start 2024-09-11T01:10:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae71cb640 1 -- start start 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae71cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ae0103b70 0x7f0ae019b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae71cb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae0104570 0x7f0ae019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae71cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ae010d120 0x7f0ae01a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae71cb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0112d60 con 0x7f0ae010d120 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae71cb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0112be0 con 0x7f0ae0104570 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae71cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0112ee0 con 0x7f0ae0103b70 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae5768640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ae010d120 0x7f0ae01a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae5768640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ae010d120 0x7f0ae01a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51050/0 (socket says 172.21.15.78:51050) 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae5768640 1 -- 172.21.15.78:0/3093260872 learned_addr learned my addr 172.21.15.78:0/3093260872 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ae0103b70 0x7f0ae019b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0adffff640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae0104570 0x7f0ae019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae5768640 1 -- 172.21.15.78:0/3093260872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ae0103b70 msgr2=0x7f0ae019b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae5768640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ae0103b70 0x7f0ae019b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae5768640 1 -- 172.21.15.78:0/3093260872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae0104570 msgr2=0x7f0ae019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae5768640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae0104570 0x7f0ae019ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0ae5768640 1 -- 172.21.15.78:0/3093260872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae01a15c0 con 0x7f0ae010d120 2024-09-11T01:10:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.244+0000 7f0adffff640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae0104570 0x7f0ae019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.245+0000 7f0ae5768640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ae010d120 0x7f0ae01a0df0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800eb10 tx=0x7f0ad800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.245+0000 7f0addffb640 1 -- 172.21.15.78:0/3093260872 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8004b20 con 0x7f0ae010d120 2024-09-11T01:10:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.245+0000 7f0ae71cb640 1 -- 172.21.15.78:0/3093260872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae01a18b0 con 0x7f0ae010d120 2024-09-11T01:10:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.245+0000 7f0addffb640 1 -- 172.21.15.78:0/3093260872 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad8004cc0 con 0x7f0ae010d120 2024-09-11T01:10:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.245+0000 7f0addffb640 1 -- 172.21.15.78:0/3093260872 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8010670 con 0x7f0ae010d120 2024-09-11T01:10:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.245+0000 7f0ae71cb640 1 -- 172.21.15.78:0/3093260872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae01a7f70 con 0x7f0ae010d120 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.248+0000 7f0ae95a0740 1 Processor -- start 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.248+0000 7f0addffb640 1 -- 172.21.15.78:0/3093260872 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0ad8005020 con 0x7f0ae010d120 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.248+0000 7f0ae95a0740 1 -- start start 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.248+0000 7f0ae95a0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23480 0x558f27b23860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.248+0000 7f0ae95a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23da0 0x558f27b2f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.248+0000 7f0ae95a0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b2fd30 0x558f27b32120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.248+0000 7f0ae95a0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558f27a99e50 con 0x558f27b23480 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.248+0000 7f0ae95a0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558f27a89280 con 0x558f27b2fd30 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.248+0000 7f0ae95a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558f27a8d090 con 0x558f27b23da0 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.248+0000 7f0ae4f67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23da0 0x558f27b2f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.248+0000 7f0ae4f67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23da0 0x558f27b2f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:37086/0 (socket says 172.21.15.78:37086) 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.248+0000 7f0ae4f67640 1 -- 172.21.15.78:0/522332212 learned_addr learned my addr 172.21.15.78:0/522332212 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.249+0000 7f0ae4f67640 1 -- 172.21.15.78:0/522332212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b2fd30 msgr2=0x558f27b32120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:10:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.249+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/522332212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b2fd30 0x558f27b32120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.249+0000 7f0ae4f67640 1 -- 172.21.15.78:0/522332212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23480 msgr2=0x558f27b23860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.249+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/522332212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23480 0x558f27b23860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.249+0000 7f0ae4f67640 1 -- 172.21.15.78:0/522332212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f277e7650 con 0x558f27b23da0 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.249+0000 7f0addffb640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0ad00b5450 0x7f0ad00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.249+0000 7f0addffb640 1 -- 172.21.15.78:0/3093260872 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f0ad80e64d0 con 0x7f0ae010d120 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.249+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/522332212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23da0 0x558f27b2f7f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009cf0 tx=0x7f0ad4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6143945a32749c07 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.249+0000 7f0adffff640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0ad00b5450 0x7f0ad00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.249+0000 7f0acaffd640 1 -- 172.21.15.78:0/522332212 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4004230 con 0x558f27b23da0 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.249+0000 7f0acaffd640 1 -- 172.21.15.78:0/522332212 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad40043d0 con 0x558f27b23da0 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.249+0000 7f0acaffd640 1 -- 172.21.15.78:0/522332212 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40197c0 con 0x558f27b23da0 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.250+0000 7f0ae95a0740 1 -- 172.21.15.78:0/522332212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23da0 msgr2=0x558f27b2f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.250+0000 7f0adffff640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0ad00b5450 0x7f0ad00b7910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0acc031440 tx=0x7f0acc005bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.250+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/522332212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23da0 0x558f27b2f7f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009cf0 tx=0x7f0ad4011810 comp rx=0 tx=0).stop 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.250+0000 7f0ae95a0740 1 -- 172.21.15.78:0/522332212 shutdown_connections 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.250+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/522332212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b2fd30 0x558f27b32120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.250+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/522332212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23da0 0x558f27b2f7f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.250+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/522332212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23480 0x558f27b23860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.250+0000 7f0ae95a0740 1 -- 172.21.15.78:0/522332212 >> 172.21.15.78:0/522332212 conn(0x558f27b212b0 msgr2=0x558f27b22450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.250+0000 7f0ae95a0740 1 -- 172.21.15.78:0/522332212 shutdown_connections 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.250+0000 7f0ae95a0740 1 -- 172.21.15.78:0/522332212 wait complete. 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.250+0000 7f0ae95a0740 1 -- start start 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.250+0000 7f0ae95a0740 1 Processor -- start 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae95a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23480 0x558f27b2f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae95a0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 0x558f27b2c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae95a0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b2fd30 0x558f27b2cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae95a0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558f278ea700 con 0x558f27b23da0 2024-09-11T01:10:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae95a0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558f27a24110 con 0x558f27b2fd30 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae95a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558f27b329b0 con 0x558f27b23480 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae95a0740 10 client.0 _ll_register_callbacks cb 0x558f27a3d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0adffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 0x558f27b2c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0adffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 0x558f27b2c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51058/0 (socket says 172.21.15.78:51058) 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0adffff640 1 -- 172.21.15.78:0/15085146 learned_addr learned my addr 172.21.15.78:0/15085146 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae5768640 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b2fd30 0x558f27b2cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23480 0x558f27b2f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae5768640 1 -- 172.21.15.78:0/15085146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23480 msgr2=0x558f27b2f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae5768640 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23480 0x558f27b2f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae5768640 1 -- 172.21.15.78:0/15085146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 msgr2=0x558f27b2c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae5768640 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 0x558f27b2c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae5768640 1 -- 172.21.15.78:0/15085146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f27a51c40 con 0x558f27b2fd30 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.251+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23480 0x558f27b2f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0ae5768640 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b2fd30 0x558f27b2cef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80e3c60 tx=0x7f0ad80057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6556c073af906c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0abffff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0abffff640 1 -- 172.21.15.78:0/15085146 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80a8100 con 0x558f27b2fd30 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0ae95a0740 1 -- 172.21.15.78:0/15085146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad40043d0 con 0x558f27b2fd30 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0abffff640 1 -- 172.21.15.78:0/15085146 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f0ad80b5a90 con 0x558f27b2fd30 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0abffff640 1 -- 172.21.15.78:0/15085146 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0ad8005540 con 0x558f27b2fd30 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0ae95a0740 10 client.26960 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0ae95a0740 1 -- 172.21.15.78:0/15085146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558f27b3e2d0 con 0x558f27b2fd30 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0abffff640 1 -- 172.21.15.78:0/15085146 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad40043d0 con 0x558f27b2fd30 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0abffff640 1 -- 172.21.15.78:0/15085146 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f0ad80cfce0 con 0x558f27b2fd30 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0ae95a0740 10 client.26960 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0ae95a0740 10 client.26960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0ae95a0740 20 client.26960 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558f27b3f820 0x558f27b41c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0ae95a0740 4 client.26960 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0ae95a0740 1 -- 172.21.15.78:0/15085146 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f27a9bda0 con 0x558f27b3f820 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.252+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558f27b3f820 0x558f27b41c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.253+0000 7f0abffff640 10 client.26960 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.253+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558f27b3f820 0x558f27b41c80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad0a666f576f696b server_cookie=3e8b62db67e1a967 in_seq=0 out_seq=0 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0abffff640 1 -- 172.21.15.78:0/15085146 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f27a9bda0 con 0x558f27b3f820 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0abffff640 10 client.26960 handle_command_reply: tid=0 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0ae95a0740 1 client.26960 shutdown 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0ae95a0740 1 -- 172.21.15.78:0/15085146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b2fd30 msgr2=0x558f27b2cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b2fd30 0x558f27b2cef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80e3c60 tx=0x7f0ad80057e0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0ae95a0740 1 -- 172.21.15.78:0/15085146 shutdown_connections 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b2fd30 0x558f27b2cef0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558f27b3f820 0x558f27b41c80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 0x558f27b2c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/15085146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23480 0x558f27b2f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0abffff640 0 client.26960 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:39.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0ae95a0740 1 -- 172.21.15.78:0/15085146 >> 172.21.15.78:0/15085146 conn(0x558f27b212b0 msgr2=0x558f27b26e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:39.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0ae95a0740 1 -- 172.21.15.78:0/15085146 shutdown_connections 2024-09-11T01:10:39.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0ae95a0740 1 -- 172.21.15.78:0/15085146 wait complete. 2024-09-11T01:10:39.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.254+0000 7f0ae95a0740 20 client.26960 trim_cache size 0 max 16384 2024-09-11T01:10:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0ae95a0740 1 Processor -- start 2024-09-11T01:10:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0ae95a0740 1 -- start start 2024-09-11T01:10:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0ae95a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23480 0x558f27b2e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0ae95a0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23da0 0x558f27b29bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0ae95a0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b2fd30 0x558f27b2a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0ae95a0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01b0890 con 0x558f27b2fd30 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0ae95a0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01b2490 con 0x558f27b23da0 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0ae95a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01b0f60 con 0x558f27b23480 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0ae5768640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b2fd30 0x558f27b2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0adffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23da0 0x558f27b29bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0ae5768640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b2fd30 0x558f27b2a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51060/0 (socket says 172.21.15.78:51060) 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0adffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23da0 0x558f27b29bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:40058/0 (socket says 172.21.15.78:40058) 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/1821246083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23480 0x558f27b2e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.274+0000 7f0ae5768640 1 -- 172.21.15.78:0/1821246083 learned_addr learned my addr 172.21.15.78:0/1821246083 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.275+0000 7f0ae5768640 1 -- 172.21.15.78:0/1821246083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23480 msgr2=0x558f27b2e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.275+0000 7f0ae5768640 1 --2- 172.21.15.78:0/1821246083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23480 0x558f27b2e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.275+0000 7f0ae5768640 1 -- 172.21.15.78:0/1821246083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23da0 msgr2=0x558f27b29bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.275+0000 7f0ae5768640 1 --2- 172.21.15.78:0/1821246083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23da0 0x558f27b29bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.275+0000 7f0ae5768640 1 -- 172.21.15.78:0/1821246083 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f27b21d30 con 0x558f27b2fd30 2024-09-11T01:10:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.275+0000 7f0adffff640 1 --2- 172.21.15.78:0/1821246083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23da0 0x558f27b29bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.275+0000 7f0ae5768640 1 --2- 172.21.15.78:0/1821246083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b2fd30 0x558f27b2a100 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80a8f60 tx=0x7f0ad800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9f3f8ca38f9fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.275+0000 7f0aca7fc640 1 -- 172.21.15.78:0/1821246083 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80cc070 con 0x558f27b2fd30 2024-09-11T01:10:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.275+0000 7f0aca7fc640 1 -- 172.21.15.78:0/1821246083 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad80ac4a0 con 0x558f27b2fd30 2024-09-11T01:10:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.275+0000 7f0aca7fc640 1 -- 172.21.15.78:0/1821246083 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80b5cf0 con 0x558f27b2fd30 2024-09-11T01:10:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.276+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1821246083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b2fd30 msgr2=0x558f27b2a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.276+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/1821246083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b2fd30 0x558f27b2a100 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80a8f60 tx=0x7f0ad800cf70 comp rx=0 tx=0).stop 2024-09-11T01:10:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.276+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1821246083 shutdown_connections 2024-09-11T01:10:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.276+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/1821246083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b2fd30 0x558f27b2a100 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.276+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/1821246083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23da0 0x558f27b29bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.276+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/1821246083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b23480 0x558f27b2e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.276+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1821246083 >> 172.21.15.78:0/1821246083 conn(0x558f27b212b0 msgr2=0x558f27b2dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.276+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1821246083 shutdown_connections 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.276+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1821246083 wait complete. 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.276+0000 7f0ae95a0740 1 -- start start 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.276+0000 7f0ae95a0740 1 Processor -- start 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0ae95a0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23480 0x558f27b37e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0ae95a0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 0x558f27b383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0ae95a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b2fd30 0x558f27b32c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0ae95a0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01b1d00 con 0x558f27b23da0 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0ae95a0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01b1630 con 0x558f27b23480 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0ae95a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01ae890 con 0x558f27b2fd30 2024-09-11T01:10:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0ae95a0740 10 client.0 _ll_register_callbacks cb 0x7f0ad80a8830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0adffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 0x558f27b383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0adffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 0x558f27b383d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51072/0 (socket says 172.21.15.78:51072) 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0adffff640 1 -- 172.21.15.78:0/1480351837 learned_addr learned my addr 172.21.15.78:0/1480351837 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23480 0x558f27b37e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0ae5768640 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b2fd30 0x558f27b32c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0adffff640 1 -- 172.21.15.78:0/1480351837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b2fd30 msgr2=0x558f27b32c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0adffff640 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b2fd30 0x558f27b32c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0adffff640 1 -- 172.21.15.78:0/1480351837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23480 msgr2=0x558f27b37e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0adffff640 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23480 0x558f27b37e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0adffff640 1 -- 172.21.15.78:0/1480351837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f27b33350 con 0x558f27b23da0 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23480 0x558f27b37e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0adffff640 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 0x558f27b383d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0acc032fd0 tx=0x7f0acc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c1d26a02b4d2b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:10:39.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.277+0000 7f0abffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:10:39.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.278+0000 7f0abffff640 1 -- 172.21.15.78:0/1480351837 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc035040 con 0x558f27b23da0 2024-09-11T01:10:39.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.278+0000 7f0abffff640 1 -- 172.21.15.78:0/1480351837 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8449+0+0 (secure 0 0 0) 0x7f0acc005040 con 0x558f27b23da0 2024-09-11T01:10:39.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.278+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1480351837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad80ac4a0 con 0x558f27b23da0 2024-09-11T01:10:39.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.278+0000 7f0abffff640 1 -- 172.21.15.78:0/1480351837 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0acc005240 con 0x558f27b23da0 2024-09-11T01:10:39.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.278+0000 7f0ae95a0740 10 client.17769 fetch_fsmap learned FSMap version 48 2024-09-11T01:10:39.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.278+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1480351837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558f27bad120 con 0x558f27b23da0 2024-09-11T01:10:39.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.278+0000 7f0abffff640 1 -- 172.21.15.78:0/1480351837 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad80ac4a0 con 0x558f27b23da0 2024-09-11T01:10:39.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.278+0000 7f0abffff640 1 -- 172.21.15.78:0/1480351837 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f0acc052020 con 0x558f27b23da0 2024-09-11T01:10:39.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.278+0000 7f0ae95a0740 10 client.17769 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:10:39.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.278+0000 7f0ae95a0740 10 client.17769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:10:39.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.278+0000 7f0ae95a0740 20 client.17769 populate_metadata read hostname 'smithi078' 2024-09-11T01:10:39.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.278+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558f27bad8e0 0x558f27bafd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:10:39.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.279+0000 7f0ae95a0740 4 client.17769 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:10:39.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.279+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1480351837 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f27a97850 con 0x558f27bad8e0 2024-09-11T01:10:39.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.279+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558f27bad8e0 0x558f27bafd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:10:39.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.279+0000 7f0ae4f67640 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558f27bad8e0 0x558f27bafd00 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6caefd96f690e2f server_cookie=f574bcc3305a68ed in_seq=0 out_seq=0 2024-09-11T01:10:39.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.279+0000 7f0abffff640 10 client.17769 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:39.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0abffff640 1 -- 172.21.15.78:0/1480351837 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x558f27a97850 con 0x558f27bad8e0 2024-09-11T01:10:39.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0abffff640 10 client.17769 handle_command_reply: tid=0 2024-09-11T01:10:39.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0ae95a0740 1 client.17769 shutdown 2024-09-11T01:10:39.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1480351837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 msgr2=0x558f27b383d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:39.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 0x558f27b383d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0acc032fd0 tx=0x7f0acc02a410 comp rx=0 tx=0).stop 2024-09-11T01:10:39.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1480351837 shutdown_connections 2024-09-11T01:10:39.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558f27b2fd30 0x558f27b32c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558f27bad8e0 0x558f27bafd00 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f27b23da0 0x558f27b383d0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0ae95a0740 1 --2- 172.21.15.78:0/1480351837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f27b23480 0x558f27b37e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0abffff640 0 client.17769 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:10:39.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1480351837 >> 172.21.15.78:0/1480351837 conn(0x558f27b212b0 msgr2=0x558f27b2e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:39.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1480351837 shutdown_connections 2024-09-11T01:10:39.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0ae95a0740 1 -- 172.21.15.78:0/1480351837 wait complete. 2024-09-11T01:10:39.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.280+0000 7f0ae95a0740 20 client.17769 trim_cache size 0 max 16384 2024-09-11T01:10:39.288 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:10:39.288 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:10:39.288 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:10:39.288 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:10:39.288 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.288 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "2f8adbde-7a68-4c03-95dd-9f3a43878c82", 2024-09-11T01:10:39.288 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:10:39.288 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:10:39.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.288 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:10:39.288 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:10:39.288 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.290 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:10:39.291 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.293 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.295 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.296 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:10:39.297 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:10:39.298 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:10:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.281+0000 7f0ae71cb640 1 -- 172.21.15.78:0/3093260872 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0ad00b5450 msgr2=0x7f0ad00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.281+0000 7f0ae71cb640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0ad00b5450 0x7f0ad00b7910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0acc031440 tx=0x7f0acc005bc0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.281+0000 7f0ae71cb640 1 -- 172.21.15.78:0/3093260872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ae010d120 msgr2=0x7f0ae01a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:10:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.281+0000 7f0ae71cb640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ae010d120 0x7f0ae01a0df0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800eb10 tx=0x7f0ad800c2d0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.281+0000 7f0ae71cb640 1 -- 172.21.15.78:0/3093260872 shutdown_connections 2024-09-11T01:10:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.281+0000 7f0ae71cb640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0ad00b5450 0x7f0ad00b7910 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.281+0000 7f0ae71cb640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ae010d120 0x7f0ae01a0df0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.281+0000 7f0ae71cb640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae0104570 0x7f0ae019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.281+0000 7f0ae71cb640 1 --2- 172.21.15.78:0/3093260872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ae0103b70 0x7f0ae019b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:10:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.281+0000 7f0ae71cb640 1 -- 172.21.15.78:0/3093260872 >> 172.21.15.78:0/3093260872 conn(0x7f0ae00fd700 msgr2=0x7f0ae00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:10:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.281+0000 7f0ae71cb640 1 -- 172.21.15.78:0/3093260872 shutdown_connections 2024-09-11T01:10:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:10:39.281+0000 7f0ae71cb640 1 -- 172.21.15.78:0/3093260872 wait complete. 2024-09-11T01:10:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:10:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:10:39.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:38 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:10:39.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 811188 5.70 MB/sec execute 527 sec latency 24.405 ms 2024-09-11T01:10:40.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:39 smithi112 ceph-mon[35384]: pgmap v686: 97 pgs: 97 active+clean; 649 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-09-11T01:10:40.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:39 smithi078 ceph-mon[31315]: pgmap v686: 97 pgs: 97 active+clean; 649 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-09-11T01:10:40.416 INFO:tasks.cephfs.filesystem:scrub status for tag:2f8adbde-7a68-4c03-95dd-9f3a43878c82 - {'path': '/', 'tag': '2f8adbde-7a68-4c03-95dd-9f3a43878c82', 'options': 'recursive,force'} 2024-09-11T01:10:40.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:39 smithi110 ceph-mon[34712]: pgmap v686: 97 pgs: 97 active+clean; 649 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-09-11T01:10:40.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 813261 5.72 MB/sec execute 528 sec latency 43.135 ms 2024-09-11T01:10:41.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 814553 5.71 MB/sec execute 529 sec latency 15.954 ms 2024-09-11T01:10:42.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:41 smithi112 ceph-mon[35384]: pgmap v687: 97 pgs: 97 active+clean; 654 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-09-11T01:10:42.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:41 smithi078 ceph-mon[31315]: pgmap v687: 97 pgs: 97 active+clean; 654 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-09-11T01:10:42.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:41 smithi110 ceph-mon[34712]: pgmap v687: 97 pgs: 97 active+clean; 654 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-09-11T01:10:42.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 816806 5.73 MB/sec execute 530 sec latency 18.276 ms 2024-09-11T01:10:43.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 818151 5.73 MB/sec execute 531 sec latency 35.431 ms 2024-09-11T01:10:44.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:43 smithi112 ceph-mon[35384]: pgmap v688: 97 pgs: 97 active+clean; 658 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 284 op/s 2024-09-11T01:10:44.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:43 smithi078 ceph-mon[31315]: pgmap v688: 97 pgs: 97 active+clean; 658 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 284 op/s 2024-09-11T01:10:44.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:43 smithi110 ceph-mon[34712]: pgmap v688: 97 pgs: 97 active+clean; 658 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 284 op/s 2024-09-11T01:10:44.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 820355 5.74 MB/sec execute 532 sec latency 20.417 ms 2024-09-11T01:10:45.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 821694 5.74 MB/sec execute 533 sec latency 14.772 ms 2024-09-11T01:10:46.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:45 smithi112 ceph-mon[35384]: pgmap v689: 97 pgs: 97 active+clean; 660 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 293 op/s 2024-09-11T01:10:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:45 smithi078 ceph-mon[31315]: pgmap v689: 97 pgs: 97 active+clean; 660 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 293 op/s 2024-09-11T01:10:46.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:45 smithi110 ceph-mon[34712]: pgmap v689: 97 pgs: 97 active+clean; 660 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 293 op/s 2024-09-11T01:10:46.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 823897 5.76 MB/sec execute 534 sec latency 18.086 ms 2024-09-11T01:10:47.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 825203 5.76 MB/sec execute 535 sec latency 16.845 ms 2024-09-11T01:10:48.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:47 smithi112 ceph-mon[35384]: pgmap v690: 97 pgs: 97 active+clean; 667 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 319 op/s 2024-09-11T01:10:48.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:47 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:10:48.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:47 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:10:48.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:47 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:10:48.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:47 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:10:48.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:47 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:10:48.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:47 smithi078 ceph-mon[31315]: pgmap v690: 97 pgs: 97 active+clean; 667 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 319 op/s 2024-09-11T01:10:48.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:47 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:10:48.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:47 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:10:48.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:47 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:10:48.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:47 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:10:48.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:47 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:10:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:47 smithi110 ceph-mon[34712]: pgmap v690: 97 pgs: 97 active+clean; 667 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 319 op/s 2024-09-11T01:10:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:47 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:10:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:47 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:10:48.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:47 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:10:48.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:47 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:10:48.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:47 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:10:48.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 827386 5.77 MB/sec execute 536 sec latency 31.494 ms 2024-09-11T01:10:49.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 828715 5.77 MB/sec execute 537 sec latency 16.239 ms 2024-09-11T01:10:50.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:49 smithi112 ceph-mon[35384]: pgmap v691: 97 pgs: 97 active+clean; 669 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 259 op/s 2024-09-11T01:10:50.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:49 smithi078 ceph-mon[31315]: pgmap v691: 97 pgs: 97 active+clean; 669 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 259 op/s 2024-09-11T01:10:50.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:49 smithi110 ceph-mon[34712]: pgmap v691: 97 pgs: 97 active+clean; 669 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 259 op/s 2024-09-11T01:10:50.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 830989 5.79 MB/sec execute 538 sec latency 14.570 ms 2024-09-11T01:10:51.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 832424 5.79 MB/sec execute 539 sec latency 14.599 ms 2024-09-11T01:10:52.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:51 smithi112 ceph-mon[35384]: pgmap v692: 97 pgs: 97 active+clean; 677 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 292 op/s 2024-09-11T01:10:52.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:51 smithi078 ceph-mon[31315]: pgmap v692: 97 pgs: 97 active+clean; 677 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 292 op/s 2024-09-11T01:10:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:51 smithi110 ceph-mon[34712]: pgmap v692: 97 pgs: 97 active+clean; 677 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 292 op/s 2024-09-11T01:10:52.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 834538 5.80 MB/sec execute 540 sec latency 58.804 ms 2024-09-11T01:10:53.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 835814 5.80 MB/sec execute 541 sec latency 40.691 ms 2024-09-11T01:10:54.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:53 smithi112 ceph-mon[35384]: pgmap v693: 97 pgs: 97 active+clean; 680 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 286 op/s 2024-09-11T01:10:54.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:53 smithi078 ceph-mon[31315]: pgmap v693: 97 pgs: 97 active+clean; 680 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 286 op/s 2024-09-11T01:10:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:53 smithi110 ceph-mon[34712]: pgmap v693: 97 pgs: 97 active+clean; 680 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 286 op/s 2024-09-11T01:10:54.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 838020 5.81 MB/sec execute 542 sec latency 22.083 ms 2024-09-11T01:10:55.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 839317 5.81 MB/sec execute 543 sec latency 14.580 ms 2024-09-11T01:10:56.226 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:55 smithi112 ceph-mon[35384]: pgmap v694: 97 pgs: 97 active+clean; 682 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 288 op/s 2024-09-11T01:10:56.694 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:55 smithi110 ceph-mon[34712]: pgmap v694: 97 pgs: 97 active+clean; 682 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 288 op/s 2024-09-11T01:10:56.694 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:55 smithi078 ceph-mon[31315]: pgmap v694: 97 pgs: 97 active+clean; 682 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 288 op/s 2024-09-11T01:10:56.694 INFO:tasks.workunit.client.0.smithi078.stdout: 1 841432 5.83 MB/sec execute 544 sec latency 39.740 ms 2024-09-11T01:10:57.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 842612 5.82 MB/sec execute 545 sec latency 12.024 ms 2024-09-11T01:10:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:57 smithi112 ceph-mon[35384]: pgmap v695: 97 pgs: 97 active+clean; 688 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 311 op/s 2024-09-11T01:10:58.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:57 smithi078 ceph-mon[31315]: pgmap v695: 97 pgs: 97 active+clean; 688 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 311 op/s 2024-09-11T01:10:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:57 smithi110 ceph-mon[34712]: pgmap v695: 97 pgs: 97 active+clean; 688 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 311 op/s 2024-09-11T01:10:58.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 844647 5.83 MB/sec execute 546 sec latency 23.202 ms 2024-09-11T01:10:59.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 846089 5.84 MB/sec execute 547 sec latency 17.387 ms 2024-09-11T01:11:00.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:10:59 smithi078 ceph-mon[31315]: pgmap v696: 97 pgs: 97 active+clean; 689 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 247 op/s 2024-09-11T01:11:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:10:59 smithi110 ceph-mon[34712]: pgmap v696: 97 pgs: 97 active+clean; 689 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 247 op/s 2024-09-11T01:11:00.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 848382 5.85 MB/sec execute 548 sec latency 14.928 ms 2024-09-11T01:11:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:10:59 smithi112 ceph-mon[35384]: pgmap v696: 97 pgs: 97 active+clean; 689 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 247 op/s 2024-09-11T01:11:01.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 849580 5.85 MB/sec execute 549 sec latency 19.980 ms 2024-09-11T01:11:02.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:01 smithi078 ceph-mon[31315]: pgmap v697: 97 pgs: 97 active+clean; 694 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-09-11T01:11:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:01 smithi110 ceph-mon[34712]: pgmap v697: 97 pgs: 97 active+clean; 694 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-09-11T01:11:02.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 851139 5.85 MB/sec execute 550 sec latency 22.354 ms 2024-09-11T01:11:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:01 smithi112 ceph-mon[35384]: pgmap v697: 97 pgs: 97 active+clean; 694 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-09-11T01:11:03.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 852635 5.86 MB/sec execute 551 sec latency 39.214 ms 2024-09-11T01:11:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:03 smithi078 ceph-mon[31315]: pgmap v698: 97 pgs: 97 active+clean; 699 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 278 op/s 2024-09-11T01:11:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:03 smithi078 ceph-mon[31315]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T01:11:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:03 smithi110 ceph-mon[34712]: pgmap v698: 97 pgs: 97 active+clean; 699 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 278 op/s 2024-09-11T01:11:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:03 smithi110 ceph-mon[34712]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T01:11:04.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 854447 5.86 MB/sec execute 552 sec latency 36.945 ms 2024-09-11T01:11:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:03 smithi112 ceph-mon[35384]: pgmap v698: 97 pgs: 97 active+clean; 699 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 278 op/s 2024-09-11T01:11:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:03 smithi112 ceph-mon[35384]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T01:11:05.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 856125 5.87 MB/sec execute 553 sec latency 12.273 ms 2024-09-11T01:11:06.242 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:05 smithi110 ceph-mon[34712]: pgmap v700: 97 pgs: 97 active+clean; 707 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 319 op/s 2024-09-11T01:11:06.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:05 smithi078 ceph-mon[31315]: pgmap v700: 97 pgs: 97 active+clean; 707 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 319 op/s 2024-09-11T01:11:06.274 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:05 smithi112 ceph-mon[35384]: pgmap v700: 97 pgs: 97 active+clean; 707 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 319 op/s 2024-09-11T01:11:06.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 858132 5.88 MB/sec execute 554 sec latency 16.247 ms 2024-09-11T01:11:07.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 859913 5.89 MB/sec execute 555 sec latency 12.682 ms 2024-09-11T01:11:08.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:07 smithi078 ceph-mon[31315]: pgmap v701: 97 pgs: 97 active+clean; 711 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 329 op/s 2024-09-11T01:11:08.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:07 smithi110 ceph-mon[34712]: pgmap v701: 97 pgs: 97 active+clean; 711 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 329 op/s 2024-09-11T01:11:08.454 INFO:tasks.workunit.client.0.smithi078.stdout: 1 861964 5.90 MB/sec execute 556 sec latency 41.755 ms 2024-09-11T01:11:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:07 smithi112 ceph-mon[35384]: pgmap v701: 97 pgs: 97 active+clean; 711 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 329 op/s 2024-09-11T01:11:09.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 863406 5.90 MB/sec execute 557 sec latency 19.099 ms 2024-09-11T01:11:09.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:09 smithi110 ceph-mon[34712]: pgmap v702: 97 pgs: 97 active+clean; 711 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 326 op/s 2024-09-11T01:11:09.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:09 smithi110 ceph-mon[34712]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T01:11:09.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:09 smithi112 ceph-mon[35384]: pgmap v702: 97 pgs: 97 active+clean; 711 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 326 op/s 2024-09-11T01:11:09.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:09 smithi112 ceph-mon[35384]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T01:11:09.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:09 smithi078 ceph-mon[31315]: pgmap v702: 97 pgs: 97 active+clean; 711 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 326 op/s 2024-09-11T01:11:09.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:09 smithi078 ceph-mon[31315]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T01:11:10.418 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:11:10.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 865321 5.90 MB/sec execute 558 sec latency 15.353 ms 2024-09-11T01:11:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:11:11.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.393+0000 7f8d492c8640 1 -- 172.21.15.78:0/3228003062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44103ca0 msgr2=0x7f8d44104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:11.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.393+0000 7f8d492c8640 1 --2- 172.21.15.78:0/3228003062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44103ca0 0x7f8d44104080 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009900 tx=0x7f8d3402a410 comp rx=0 tx=0).stop 2024-09-11T01:11:11.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.393+0000 7f8d492c8640 1 -- 172.21.15.78:0/3228003062 shutdown_connections 2024-09-11T01:11:11.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.393+0000 7f8d492c8640 1 --2- 172.21.15.78:0/3228003062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d4410d100 0x7f8d4410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.393+0000 7f8d492c8640 1 --2- 172.21.15.78:0/3228003062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d441045c0 0x7f8d4410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.393+0000 7f8d492c8640 1 --2- 172.21.15.78:0/3228003062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44103ca0 0x7f8d44104080 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.393+0000 7f8d492c8640 1 -- 172.21.15.78:0/3228003062 >> 172.21.15.78:0/3228003062 conn(0x7f8d440fd700 msgr2=0x7f8d440ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.393+0000 7f8d492c8640 1 -- 172.21.15.78:0/3228003062 shutdown_connections 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.393+0000 7f8d492c8640 1 -- 172.21.15.78:0/3228003062 wait complete. 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.393+0000 7f8d492c8640 1 Processor -- start 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.393+0000 7f8d492c8640 1 -- start start 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d492c8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d44103ca0 0x7f8d44199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d492c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d441045c0 0x7f8d44199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d492c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d4410d100 0x7f8d4419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d492c8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d44112d70 con 0x7f8d441045c0 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d492c8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d44112bf0 con 0x7f8d44103ca0 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d492c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d44112ef0 con 0x7f8d4410d100 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d427fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d441045c0 0x7f8d44199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d42ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d44103ca0 0x7f8d44199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d427fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d441045c0 0x7f8d44199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46458/0 (socket says 172.21.15.78:46458) 2024-09-11T01:11:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d427fc640 1 -- 172.21.15.78:0/3000904979 learned_addr learned my addr 172.21.15.78:0/3000904979 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d437fe640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d4410d100 0x7f8d4419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d427fc640 1 -- 172.21.15.78:0/3000904979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d4410d100 msgr2=0x7f8d4419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d427fc640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d4410d100 0x7f8d4419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d427fc640 1 -- 172.21.15.78:0/3000904979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d44103ca0 msgr2=0x7f8d44199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d427fc640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d44103ca0 0x7f8d44199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d427fc640 1 -- 172.21.15.78:0/3000904979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4419f3f0 con 0x7f8d441045c0 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d427fc640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d441045c0 0x7f8d44199880 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e9c0 tx=0x7f8d3800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d2bfff640 1 -- 172.21.15.78:0/3000904979 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800ccc0 con 0x7f8d441045c0 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d492c8640 1 -- 172.21.15.78:0/3000904979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4419f6e0 con 0x7f8d441045c0 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.394+0000 7f8d2bfff640 1 -- 172.21.15.78:0/3000904979 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d380035f0 con 0x7f8d441045c0 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.395+0000 7f8d2bfff640 1 -- 172.21.15.78:0/3000904979 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800ce60 con 0x7f8d441045c0 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.395+0000 7f8d492c8640 1 -- 172.21.15.78:0/3000904979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d441a5da0 con 0x7f8d441045c0 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d2bfff640 1 -- 172.21.15.78:0/3000904979 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8d38004000 con 0x7f8d441045c0 2024-09-11T01:11:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d4b69d740 1 Processor -- start 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d4b69d740 1 -- start start 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d4b69d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff90a0 0x562f32ff9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d4b69d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f33005410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d4b69d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f33005950 0x562f33007d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d4b69d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562f32f6fe50 con 0x562f32ff99c0 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d4b69d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562f32f5f280 con 0x562f33005950 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d4b69d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f32f63090 con 0x562f32ff90a0 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f33005410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff90a0 0x562f32ff9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f33005410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46460/0 (socket says 172.21.15.78:46460) 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.397+0000 7f8d42ffd640 1 -- 172.21.15.78:0/1221936961 learned_addr learned my addr 172.21.15.78:0/1221936961 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d427fc640 1 --2- 172.21.15.78:0/1221936961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f33005950 0x562f33007d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d42ffd640 1 -- 172.21.15.78:0/1221936961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff90a0 msgr2=0x562f32ff9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/1221936961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff90a0 0x562f32ff9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d42ffd640 1 -- 172.21.15.78:0/1221936961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f33005950 msgr2=0x562f33007d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/1221936961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f33005950 0x562f33007d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d42ffd640 1 -- 172.21.15.78:0/1221936961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f32cbd650 con 0x562f32ff99c0 2024-09-11T01:11:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d437fe640 1 --2- 172.21.15.78:0/1221936961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff90a0 0x562f32ff9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d2bfff640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d2c0b5450 0x7f8d2c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/1221936961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f33005410 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009900 tx=0x7f8d34032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0f12ea470b92e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d437fe640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d2c0b5450 0x7f8d2c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d2bfff640 1 -- 172.21.15.78:0/3000904979 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f8d380e6270 con 0x7f8d441045c0 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d297fa640 1 -- 172.21.15.78:0/1221936961 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34040070 con 0x562f32ff99c0 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d297fa640 1 -- 172.21.15.78:0/1221936961 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d34004030 con 0x562f32ff99c0 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.398+0000 7f8d297fa640 1 -- 172.21.15.78:0/1221936961 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34035040 con 0x562f32ff99c0 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.399+0000 7f8d4b69d740 1 -- 172.21.15.78:0/1221936961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 msgr2=0x562f33005410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.399+0000 7f8d437fe640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d2c0b5450 0x7f8d2c0b7910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c005fd0 tx=0x7f8d3c0096a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.399+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/1221936961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f33005410 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009900 tx=0x7f8d34032bd0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.399+0000 7f8d4b69d740 1 -- 172.21.15.78:0/1221936961 shutdown_connections 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.399+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/1221936961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f33005950 0x562f33007d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.399+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/1221936961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f33005410 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.399+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/1221936961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff90a0 0x562f32ff9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.399+0000 7f8d4b69d740 1 -- 172.21.15.78:0/1221936961 >> 172.21.15.78:0/1221936961 conn(0x562f32ff6ed0 msgr2=0x562f32ff8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.399+0000 7f8d4b69d740 1 -- 172.21.15.78:0/1221936961 shutdown_connections 2024-09-11T01:11:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.399+0000 7f8d4b69d740 1 -- 172.21.15.78:0/1221936961 wait complete. 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.399+0000 7f8d4b69d740 1 -- start start 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.399+0000 7f8d4b69d740 1 Processor -- start 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d4b69d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f32ff90a0 0x562f33005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d4b69d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff99c0 0x562f330026e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d4b69d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f33005950 0x562f33002d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d4b69d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562f32dc0700 con 0x562f33005950 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d4b69d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562f32efa110 con 0x562f32ff90a0 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d4b69d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f33008570 con 0x562f32ff99c0 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d4b69d740 10 client.0 _ll_register_callbacks cb 0x562f32f13990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f33005950 0x562f33002d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f33005950 0x562f33002d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46468/0 (socket says 172.21.15.78:46468) 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d42ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f32ff90a0 0x562f33005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d437fe640 1 -- 172.21.15.78:0/3403316676 learned_addr learned my addr 172.21.15.78:0/3403316676 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d427fc640 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff99c0 0x562f330026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d437fe640 1 -- 172.21.15.78:0/3403316676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff99c0 msgr2=0x562f330026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d437fe640 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff99c0 0x562f330026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d437fe640 1 -- 172.21.15.78:0/3403316676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f32ff90a0 msgr2=0x562f33005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d437fe640 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f32ff90a0 0x562f33005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d437fe640 1 -- 172.21.15.78:0/3403316676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f32f27c40 con 0x562f33005950 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f32ff90a0 0x562f33005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d437fe640 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f33005950 0x562f33002d60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009fd0 tx=0x7f8d3c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8317a27efd387df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d26ff5640 1 -- 172.21.15.78:0/3403316676 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c02c060 con 0x562f33005950 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d26ff5640 1 -- 172.21.15.78:0/3403316676 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f8d3c017030 con 0x562f33005950 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3403316676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d34004030 con 0x562f33005950 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.400+0000 7f8d26ff5640 1 -- 172.21.15.78:0/3403316676 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8d3c01f8b0 con 0x562f33005950 2024-09-11T01:11:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d4b69d740 10 client.17781 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3403316676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562f33013f50 con 0x562f33005950 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d26ff5640 1 -- 172.21.15.78:0/3403316676 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d34004030 con 0x562f33005950 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d26ff5640 1 -- 172.21.15.78:0/3403316676 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f8d3c027c10 con 0x562f33005950 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d4b69d740 10 client.17781 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d4b69d740 10 client.17781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d4b69d740 20 client.17781 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562f330154e0 0x562f33017940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d4b69d740 4 client.17781 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3403316676 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f32f71da0 con 0x562f330154e0 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562f330154e0 0x562f33017940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d26ff5640 10 client.17781 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.401+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562f330154e0 0x562f33017940 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc90f02decd06a08 server_cookie=bf1d853b59b908a in_seq=0 out_seq=0 2024-09-11T01:11:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d26ff5640 1 -- 172.21.15.78:0/3403316676 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562f32f71da0 con 0x562f330154e0 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d26ff5640 10 client.17781 handle_command_reply: tid=0 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d4b69d740 1 client.17781 shutdown 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3403316676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f33005950 msgr2=0x562f33002d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f33005950 0x562f33002d60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009fd0 tx=0x7f8d3c004ab0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3403316676 shutdown_connections 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562f330154e0 0x562f33017940 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f33005950 0x562f33002d60 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d26ff5640 0 client.17781 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff99c0 0x562f330026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/3403316676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f32ff90a0 0x562f33005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3403316676 >> 172.21.15.78:0/3403316676 conn(0x562f32ff6ed0 msgr2=0x562f32ffca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3403316676 shutdown_connections 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3403316676 wait complete. 2024-09-11T01:11:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.402+0000 7f8d4b69d740 20 client.17781 trim_cache size 0 max 16384 2024-09-11T01:11:11.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d4b69d740 1 Processor -- start 2024-09-11T01:11:11.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d4b69d740 1 -- start start 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d4b69d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f32ff90a0 0x562f32ff6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d4b69d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f33000f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d4b69d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f33005950 0x562f330014a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d4b69d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441ae690 con 0x562f32ff99c0 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d4b69d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441b0290 con 0x562f32ff90a0 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d4b69d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441aed60 con 0x562f33005950 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d427fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f33000f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d427fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f33000f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46478/0 (socket says 172.21.15.78:46478) 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d42ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f32ff90a0 0x562f32ff6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d427fc640 1 -- 172.21.15.78:0/882668076 learned_addr learned my addr 172.21.15.78:0/882668076 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.421+0000 7f8d437fe640 1 --2- 172.21.15.78:0/882668076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f33005950 0x562f330014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.422+0000 7f8d42ffd640 1 -- 172.21.15.78:0/882668076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f33005950 msgr2=0x562f330014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.422+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/882668076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f33005950 0x562f330014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.422+0000 7f8d42ffd640 1 -- 172.21.15.78:0/882668076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 msgr2=0x562f33000f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.422+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/882668076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f33000f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.422+0000 7f8d42ffd640 1 -- 172.21.15.78:0/882668076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f32ff8590 con 0x562f32ff90a0 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.422+0000 7f8d427fc640 1 --2- 172.21.15.78:0/882668076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f33000f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.422+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/882668076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f32ff90a0 0x562f32ff6710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8d34031a90 tx=0x7f8d34005150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63449a2b4cba3f7f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.422+0000 7f8d28ff9640 1 -- 172.21.15.78:0/882668076 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34040040 con 0x562f32ff90a0 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.422+0000 7f8d28ff9640 1 -- 172.21.15.78:0/882668076 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d34005460 con 0x562f32ff90a0 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.422+0000 7f8d28ff9640 1 -- 172.21.15.78:0/882668076 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34035040 con 0x562f32ff90a0 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 -- 172.21.15.78:0/882668076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f32ff90a0 msgr2=0x562f32ff6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/882668076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f32ff90a0 0x562f32ff6710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8d34031a90 tx=0x7f8d34005150 comp rx=0 tx=0).stop 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 -- 172.21.15.78:0/882668076 shutdown_connections 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/882668076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f33005950 0x562f330014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/882668076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f33000f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/882668076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f32ff90a0 0x562f32ff6710 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 -- 172.21.15.78:0/882668076 >> 172.21.15.78:0/882668076 conn(0x562f32ff6ed0 msgr2=0x562f32ffd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 -- 172.21.15.78:0/882668076 shutdown_connections 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 -- 172.21.15.78:0/882668076 wait complete. 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 -- start start 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 Processor -- start 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff90a0 0x562f3300d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f3300dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f33005950 0x562f330087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441afb00 con 0x562f32ff99c0 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441af430 con 0x562f33005950 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.423+0000 7f8d4b69d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441ac690 con 0x562f32ff90a0 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d4b69d740 10 client.0 _ll_register_callbacks cb 0x7f8d3c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d427fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f3300dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d427fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f3300dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46480/0 (socket says 172.21.15.78:46480) 2024-09-11T01:11:11.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d427fc640 1 -- 172.21.15.78:0/3630179612 learned_addr learned my addr 172.21.15.78:0/3630179612 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d437fe640 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f33005950 0x562f330087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff90a0 0x562f3300d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d427fc640 1 -- 172.21.15.78:0/3630179612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff90a0 msgr2=0x562f3300d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d427fc640 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff90a0 0x562f3300d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d427fc640 1 -- 172.21.15.78:0/3630179612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f33005950 msgr2=0x562f330087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d427fc640 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f33005950 0x562f330087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d427fc640 1 -- 172.21.15.78:0/3630179612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f33008d00 con 0x562f32ff99c0 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d437fe640 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f33005950 0x562f330087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d427fc640 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f3300dd50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8d380e4000 tx=0x7f8d38003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c46405fdf418378 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d26ff5640 1 -- 172.21.15.78:0/3630179612 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d380ac240 con 0x562f32ff99c0 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d26ff5640 1 -- 172.21.15.78:0/3630179612 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f8d38004320 con 0x562f32ff99c0 2024-09-11T01:11:11.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3630179612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d34005460 con 0x562f32ff99c0 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d26ff5640 1 -- 172.21.15.78:0/3630179612 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8d380ace30 con 0x562f32ff99c0 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d4b69d740 10 client.17787 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.424+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3630179612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562f33087090 con 0x562f32ff99c0 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.425+0000 7f8d26ff5640 1 -- 172.21.15.78:0/3630179612 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d34005460 con 0x562f32ff99c0 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.425+0000 7f8d26ff5640 1 -- 172.21.15.78:0/3630179612 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f8d380b5830 con 0x562f32ff99c0 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.425+0000 7f8d4b69d740 10 client.17787 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.425+0000 7f8d4b69d740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.425+0000 7f8d4b69d740 20 client.17787 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.425+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562f330878c0 0x562f33089ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.425+0000 7f8d4b69d740 4 client.17787 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.425+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3630179612 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562f32f6d850 con 0x562f330878c0 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.425+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562f330878c0 0x562f33089ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.425+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562f330878c0 0x562f33089ce0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25b08528392a9f85 server_cookie=8a6b5552efb27c85 in_seq=0 out_seq=0 2024-09-11T01:11:11.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.425+0000 7f8d26ff5640 10 client.17787 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d26ff5640 1 -- 172.21.15.78:0/3630179612 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x562f32f6d850 con 0x562f330878c0 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d26ff5640 10 client.17787 handle_command_reply: tid=0 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d4b69d740 1 client.17787 shutdown 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3630179612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 msgr2=0x562f3300dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f3300dd50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8d380e4000 tx=0x7f8d38003cd0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3630179612 shutdown_connections 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f33005950 0x562f330087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562f330878c0 0x562f33089ce0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562f32ff99c0 0x562f3300dd50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d26ff5640 0 client.17787 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d4b69d740 1 --2- 172.21.15.78:0/3630179612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f32ff90a0 0x562f3300d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3630179612 >> 172.21.15.78:0/3630179612 conn(0x562f32ff6ed0 msgr2=0x562f33000f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3630179612 shutdown_connections 2024-09-11T01:11:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d4b69d740 1 -- 172.21.15.78:0/3630179612 wait complete. 2024-09-11T01:11:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.426+0000 7f8d4b69d740 20 client.17787 trim_cache size 0 max 16384 2024-09-11T01:11:11.441 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:11:11.441 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:11:11.441 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:11:11.441 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:11:11.441 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:11:11.441 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:11:11.441 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.441 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.442 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.443 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.444 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.446 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.447 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.448 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:11.450 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:11.451 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:11.451 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:11:11.451 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:11:11.451 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:11:11.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.427+0000 7f8d492c8640 1 -- 172.21.15.78:0/3000904979 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d2c0b5450 msgr2=0x7f8d2c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:11.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.427+0000 7f8d492c8640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d2c0b5450 0x7f8d2c0b7910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c005fd0 tx=0x7f8d3c0096a0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.427+0000 7f8d492c8640 1 -- 172.21.15.78:0/3000904979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d441045c0 msgr2=0x7f8d44199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.427+0000 7f8d492c8640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d441045c0 0x7f8d44199880 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e9c0 tx=0x7f8d3800ee80 comp rx=0 tx=0).stop 2024-09-11T01:11:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.427+0000 7f8d492c8640 1 -- 172.21.15.78:0/3000904979 shutdown_connections 2024-09-11T01:11:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.427+0000 7f8d492c8640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8d2c0b5450 0x7f8d2c0b7910 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.427+0000 7f8d492c8640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d4410d100 0x7f8d4419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.427+0000 7f8d492c8640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d441045c0 0x7f8d44199880 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.427+0000 7f8d492c8640 1 --2- 172.21.15.78:0/3000904979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d44103ca0 0x7f8d44199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.427+0000 7f8d492c8640 1 -- 172.21.15.78:0/3000904979 >> 172.21.15.78:0/3000904979 conn(0x7f8d440fd700 msgr2=0x7f8d440fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.427+0000 7f8d492c8640 1 -- 172.21.15.78:0/3000904979 shutdown_connections 2024-09-11T01:11:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:11.427+0000 7f8d492c8640 1 -- 172.21.15.78:0/3000904979 wait complete. 2024-09-11T01:11:11.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 866832 5.91 MB/sec execute 559 sec latency 33.897 ms 2024-09-11T01:11:11.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:11 smithi110 ceph-mon[34712]: pgmap v704: 97 pgs: 97 active+clean; 717 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 377 op/s 2024-09-11T01:11:11.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:11 smithi112 ceph-mon[35384]: pgmap v704: 97 pgs: 97 active+clean; 717 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 377 op/s 2024-09-11T01:11:11.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:11 smithi078 ceph-mon[31315]: pgmap v704: 97 pgs: 97 active+clean; 717 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 377 op/s 2024-09-11T01:11:12.365 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:11:12.365 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:11:12.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 868728 5.91 MB/sec execute 560 sec latency 18.573 ms 2024-09-11T01:11:12.667 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:11:13.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.197+0000 7fb281493640 1 -- 172.21.15.78:0/3601057308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c10d100 msgr2=0x7fb27c10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:13.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.197+0000 7fb281493640 1 --2- 172.21.15.78:0/3601057308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c10d100 0x7fb27c10f560 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb274009ca0 tx=0x7fb27402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:11:13.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.197+0000 7fb281493640 1 -- 172.21.15.78:0/3601057308 shutdown_connections 2024-09-11T01:11:13.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.197+0000 7fb281493640 1 --2- 172.21.15.78:0/3601057308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c10d100 0x7fb27c10f560 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.197+0000 7fb281493640 1 --2- 172.21.15.78:0/3601057308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb27c1045c0 0x7fb27c10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.197+0000 7fb281493640 1 --2- 172.21.15.78:0/3601057308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb27c103ca0 0x7fb27c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.197+0000 7fb281493640 1 -- 172.21.15.78:0/3601057308 >> 172.21.15.78:0/3601057308 conn(0x7fb27c0fd700 msgr2=0x7fb27c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.197+0000 7fb281493640 1 -- 172.21.15.78:0/3601057308 shutdown_connections 2024-09-11T01:11:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.197+0000 7fb281493640 1 -- 172.21.15.78:0/3601057308 wait complete. 2024-09-11T01:11:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.197+0000 7fb281493640 1 Processor -- start 2024-09-11T01:11:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb281493640 1 -- start start 2024-09-11T01:11:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb281493640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c103ca0 0x7fb27c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb281493640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb27c1045c0 0x7fb27c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb281493640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb27c10d100 0x7fb27c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb281493640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c112d70 con 0x7fb27c103ca0 2024-09-11T01:11:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb281493640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c112bf0 con 0x7fb27c10d100 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb281493640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c112ef0 con 0x7fb27c1045c0 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb27affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c103ca0 0x7fb27c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb27affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c103ca0 0x7fb27c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46494/0 (socket says 172.21.15.78:46494) 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb27affd640 1 -- 172.21.15.78:0/354080873 learned_addr learned my addr 172.21.15.78:0/354080873 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb27a7fc640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb27c1045c0 0x7fb27c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb27b7fe640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb27c10d100 0x7fb27c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb27a7fc640 1 -- 172.21.15.78:0/354080873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb27c10d100 msgr2=0x7fb27c19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb27a7fc640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb27c10d100 0x7fb27c19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb27a7fc640 1 -- 172.21.15.78:0/354080873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c103ca0 msgr2=0x7fb27c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb27a7fc640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c103ca0 0x7fb27c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb27a7fc640 1 -- 172.21.15.78:0/354080873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27c19f310 con 0x7fb27c1045c0 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.198+0000 7fb27affd640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c103ca0 0x7fb27c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.199+0000 7fb27a7fc640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb27c1045c0 0x7fb27c1997a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb27000e770 tx=0x7fb27000ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.199+0000 7fb263fff640 1 -- 172.21.15.78:0/354080873 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27000ccc0 con 0x7fb27c1045c0 2024-09-11T01:11:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.199+0000 7fb263fff640 1 -- 172.21.15.78:0/354080873 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb27000ee80 con 0x7fb27c1045c0 2024-09-11T01:11:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.199+0000 7fb281493640 1 -- 172.21.15.78:0/354080873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27c19f600 con 0x7fb27c1045c0 2024-09-11T01:11:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.199+0000 7fb263fff640 1 -- 172.21.15.78:0/354080873 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270003e40 con 0x7fb27c1045c0 2024-09-11T01:11:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.199+0000 7fb281493640 1 -- 172.21.15.78:0/354080873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27c1a5e90 con 0x7fb27c1045c0 2024-09-11T01:11:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.200+0000 7fb281493640 1 -- 172.21.15.78:0/354080873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27c104080 con 0x7fb27c1045c0 2024-09-11T01:11:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.201+0000 7fb263fff640 1 -- 172.21.15.78:0/354080873 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb270004000 con 0x7fb27c1045c0 2024-09-11T01:11:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.202+0000 7fb263fff640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb2640b54d0 0x7fb2640b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.202+0000 7fb27affd640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb2640b54d0 0x7fb2640b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.202+0000 7fb263fff640 1 -- 172.21.15.78:0/354080873 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7fb2700e6240 con 0x7fb27c1045c0 2024-09-11T01:11:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.203+0000 7fb27affd640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb2640b54d0 0x7fb2640b7990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb26c011500 tx=0x7fb26c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.206+0000 7fb263fff640 1 -- 172.21.15.78:0/354080873 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb2700ace30 con 0x7fb27c1045c0 2024-09-11T01:11:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.433+0000 7fb281493640 1 -- 172.21.15.78:0/354080873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb27c1a60c0 con 0x7fb27c1045c0 2024-09-11T01:11:13.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.434+0000 7fb263fff640 1 -- 172.21.15.78:0/354080873 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11986 (secure 0 0 0) 0x7fb2700ac730 con 0x7fb27c1045c0 2024-09-11T01:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":48,"btime":"2024-09-11T01:09:37:269802+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:09:37.112141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:11:13.437 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 48 2024-09-11T01:11:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.436+0000 7fb281493640 1 -- 172.21.15.78:0/354080873 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb2640b54d0 msgr2=0x7fb2640b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.436+0000 7fb281493640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb2640b54d0 0x7fb2640b7990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb26c011500 tx=0x7fb26c005f90 comp rx=0 tx=0).stop 2024-09-11T01:11:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.436+0000 7fb281493640 1 -- 172.21.15.78:0/354080873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb27c1045c0 msgr2=0x7fb27c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.436+0000 7fb281493640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb27c1045c0 0x7fb27c1997a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb27000e770 tx=0x7fb27000ec30 comp rx=0 tx=0).stop 2024-09-11T01:11:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.436+0000 7fb281493640 1 -- 172.21.15.78:0/354080873 shutdown_connections 2024-09-11T01:11:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.436+0000 7fb281493640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb2640b54d0 0x7fb2640b7990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.436+0000 7fb281493640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb27c10d100 0x7fb27c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.436+0000 7fb281493640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb27c1045c0 0x7fb27c1997a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.436+0000 7fb281493640 1 --2- 172.21.15.78:0/354080873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c103ca0 0x7fb27c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.436+0000 7fb281493640 1 -- 172.21.15.78:0/354080873 >> 172.21.15.78:0/354080873 conn(0x7fb27c0fd700 msgr2=0x7fb27c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.436+0000 7fb281493640 1 -- 172.21.15.78:0/354080873 shutdown_connections 2024-09-11T01:11:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:13.436+0000 7fb281493640 1 -- 172.21.15.78:0/354080873 wait complete. 2024-09-11T01:11:13.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 870300 5.93 MB/sec execute 561 sec latency 11.957 ms 2024-09-11T01:11:13.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:13 smithi110 ceph-mon[34712]: pgmap v705: 97 pgs: 97 active+clean; 718 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 350 op/s 2024-09-11T01:11:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:13 smithi112 ceph-mon[35384]: pgmap v705: 97 pgs: 97 active+clean; 718 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 350 op/s 2024-09-11T01:11:13.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:13 smithi078 ceph-mon[31315]: pgmap v705: 97 pgs: 97 active+clean; 718 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 350 op/s 2024-09-11T01:11:14.276 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:11:14.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 871662 5.92 MB/sec execute 562 sec latency 49.751 ms 2024-09-11T01:11:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:14 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/354080873' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:11:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:14 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:11:14.569 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:11:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:14 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/354080873' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:11:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:14 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:11:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:14 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/354080873' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:11:14.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:14 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:11:15.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.075+0000 7f0307491640 1 -- 172.21.15.78:0/3845749873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03001055a0 msgr2=0x7f0300105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:15.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.075+0000 7f0307491640 1 --2- 172.21.15.78:0/3845749873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03001055a0 0x7f0300105a00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009900 tx=0x7f02f002a410 comp rx=0 tx=0).stop 2024-09-11T01:11:15.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.075+0000 7f0307491640 1 -- 172.21.15.78:0/3845749873 shutdown_connections 2024-09-11T01:11:15.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.075+0000 7f0307491640 1 --2- 172.21.15.78:0/3845749873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300105f40 0x7f030010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.075+0000 7f0307491640 1 --2- 172.21.15.78:0/3845749873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03001055a0 0x7f0300105a00 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.075+0000 7f0307491640 1 --2- 172.21.15.78:0/3845749873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0300101250 0x7f0300101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.075+0000 7f0307491640 1 -- 172.21.15.78:0/3845749873 >> 172.21.15.78:0/3845749873 conn(0x7f0300078140 msgr2=0x7f03000ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:15.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.075+0000 7f0307491640 1 -- 172.21.15.78:0/3845749873 shutdown_connections 2024-09-11T01:11:15.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.075+0000 7f0307491640 1 -- 172.21.15.78:0/3845749873 wait complete. 2024-09-11T01:11:15.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.076+0000 7f0307491640 1 Processor -- start 2024-09-11T01:11:15.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.076+0000 7f0307491640 1 -- start start 2024-09-11T01:11:15.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.076+0000 7f0307491640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300101250 0x7f03001992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.076+0000 7f0307491640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03001055a0 0x7f0300199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.076+0000 7f0307491640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0300105f40 0x7f030019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.076+0000 7f0307491640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0300112d80 con 0x7f0300105f40 2024-09-11T01:11:15.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.076+0000 7f0307491640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0300112c00 con 0x7f03001055a0 2024-09-11T01:11:15.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.076+0000 7f0307491640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0300112f00 con 0x7f0300101250 2024-09-11T01:11:15.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.076+0000 7f030522d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300101250 0x7f03001992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.076+0000 7f030522d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300101250 0x7f03001992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:37992/0 (socket says 172.21.15.78:37992) 2024-09-11T01:11:15.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.076+0000 7f030522d640 1 -- 172.21.15.78:0/1904903830 learned_addr learned my addr 172.21.15.78:0/1904903830 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:15.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f0305a2e640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0300105f40 0x7f030019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f0304a2c640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03001055a0 0x7f0300199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f030522d640 1 -- 172.21.15.78:0/1904903830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03001055a0 msgr2=0x7f0300199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f030522d640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03001055a0 0x7f0300199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f030522d640 1 -- 172.21.15.78:0/1904903830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0300105f40 msgr2=0x7f030019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f030522d640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0300105f40 0x7f030019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f030522d640 1 -- 172.21.15.78:0/1904903830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030019f3a0 con 0x7f0300101250 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f0304a2c640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03001055a0 0x7f0300199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f030522d640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300101250 0x7f03001992f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00e9f0 tx=0x7f02fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f02f67fc640 1 -- 172.21.15.78:0/1904903830 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc00cd80 con 0x7f0300101250 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f02f67fc640 1 -- 172.21.15.78:0/1904903830 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02fc0108e0 con 0x7f0300101250 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f0307491640 1 -- 172.21.15.78:0/1904903830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030019f690 con 0x7f0300101250 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f02f67fc640 1 -- 172.21.15.78:0/1904903830 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc018980 con 0x7f0300101250 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.077+0000 7f0307491640 1 -- 172.21.15.78:0/1904903830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03001a5d50 con 0x7f0300101250 2024-09-11T01:11:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.080+0000 7f0309866740 1 Processor -- start 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.080+0000 7f02f67fc640 1 -- 172.21.15.78:0/1904903830 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f02fc018b20 con 0x7f0300101250 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.080+0000 7f0309866740 1 -- start start 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0309866740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded6218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0309866740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded62d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0309866740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded630170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0309866740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ded597ea0 con 0x564ded62dd80 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0309866740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564ded5872d0 con 0x564ded6214d0 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0309866740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564ded58b0e0 con 0x564ded621df0 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0305a2e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded6218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0305a2e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded6218b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36014/0 (socket says 172.21.15.78:36014) 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0304a2c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded630170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0305a2e640 1 -- 172.21.15.78:0/2691283442 learned_addr learned my addr 172.21.15.78:0/2691283442 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f030522d640 1 --2- 172.21.15.78:0/2691283442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded62d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0305a2e640 1 -- 172.21.15.78:0/2691283442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 msgr2=0x564ded62d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0305a2e640 1 --2- 172.21.15.78:0/2691283442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded62d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0305a2e640 1 -- 172.21.15.78:0/2691283442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 msgr2=0x564ded630170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0305a2e640 1 --2- 172.21.15.78:0/2691283442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded630170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0305a2e640 1 -- 172.21.15.78:0/2691283442 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ded2e5650 con 0x564ded6214d0 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.081+0000 7f0304a2c640 1 --2- 172.21.15.78:0/2691283442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded630170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.082+0000 7f0305a2e640 1 --2- 172.21.15.78:0/2691283442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded6218b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009fc0 tx=0x7f02f80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eb9c14acf9d9ea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.082+0000 7f02f67fc640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02ec0b5450 0x7f02ec0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.082+0000 7f02eb7fe640 1 -- 172.21.15.78:0/2691283442 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8004d20 con 0x564ded6214d0 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.082+0000 7f02eb7fe640 1 -- 172.21.15.78:0/2691283442 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f8004050 con 0x564ded6214d0 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.082+0000 7f02eb7fe640 1 -- 172.21.15.78:0/2691283442 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8005280 con 0x564ded6214d0 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.082+0000 7f030522d640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02ec0b5450 0x7f02ec0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.082+0000 7f02f67fc640 1 -- 172.21.15.78:0/1904903830 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f02fc0e63b0 con 0x7f0300101250 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.083+0000 7f0309866740 1 -- 172.21.15.78:0/2691283442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 msgr2=0x564ded6218b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.083+0000 7f0309866740 1 --2- 172.21.15.78:0/2691283442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded6218b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009fc0 tx=0x7f02f80118d0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.083+0000 7f030522d640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02ec0b5450 0x7f02ec0b7910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0e3bf0 tx=0x7f02fc0eb040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.083+0000 7f0309866740 1 -- 172.21.15.78:0/2691283442 shutdown_connections 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.083+0000 7f0309866740 1 --2- 172.21.15.78:0/2691283442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded630170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.083+0000 7f0309866740 1 --2- 172.21.15.78:0/2691283442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded62d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.083+0000 7f0309866740 1 --2- 172.21.15.78:0/2691283442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded6218b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.083+0000 7f0309866740 1 -- 172.21.15.78:0/2691283442 >> 172.21.15.78:0/2691283442 conn(0x564ded61f300 msgr2=0x564ded6204a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.083+0000 7f0309866740 1 -- 172.21.15.78:0/2691283442 shutdown_connections 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.083+0000 7f0309866740 1 -- 172.21.15.78:0/2691283442 wait complete. 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.084+0000 7f0309866740 1 -- start start 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.084+0000 7f0309866740 1 Processor -- start 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.084+0000 7f0309866740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded62d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.084+0000 7f0309866740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded621df0 0x564ded62aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.084+0000 7f0309866740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded62dd80 0x564ded62b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.084+0000 7f0309866740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ded3e8700 con 0x564ded621df0 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.084+0000 7f0309866740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564ded522160 con 0x564ded6214d0 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.084+0000 7f0309866740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564ded630a00 con 0x564ded62dd80 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.085+0000 7f0309866740 10 client.0 _ll_register_callbacks cb 0x564ded53b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.085+0000 7f0304a2c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded621df0 0x564ded62aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.085+0000 7f0305a2e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded62d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.085+0000 7f0305a2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded62dd80 0x564ded62b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.085+0000 7f0304a2c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded621df0 0x564ded62aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57062/0 (socket says 172.21.15.78:57062) 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.085+0000 7f0304a2c640 1 -- 172.21.15.78:0/4056923558 learned_addr learned my addr 172.21.15.78:0/4056923558 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.085+0000 7f0305a2e640 1 -- 172.21.15.78:0/4056923558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 msgr2=0x564ded62d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.085+0000 7f0305a2e640 1 --2- 172.21.15.78:0/4056923558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded62d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.085+0000 7f0305a2e640 1 -- 172.21.15.78:0/4056923558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded621df0 msgr2=0x564ded62aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.085+0000 7f0305a2e640 1 --2- 172.21.15.78:0/4056923558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded621df0 0x564ded62aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.085+0000 7f0305a2e640 1 -- 172.21.15.78:0/4056923558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ded54fc90 con 0x564ded62dd80 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.085+0000 7f0304a2c640 1 --2- 172.21.15.78:0/4056923558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded621df0 0x564ded62aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.086+0000 7f0305a2e640 1 --2- 172.21.15.78:0/4056923558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded62dd80 0x564ded62b040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f02f8011dd0 tx=0x7f02f8000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7292784191e20533 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.086+0000 7f02e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.086+0000 7f02e8ff9640 1 -- 172.21.15.78:0/4056923558 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8029d90 con 0x564ded62dd80 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.086+0000 7f02e8ff9640 1 -- 172.21.15.78:0/4056923558 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f02f8004200 con 0x564ded62dd80 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.086+0000 7f0309866740 1 -- 172.21.15.78:0/4056923558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f8004050 con 0x564ded62dd80 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.086+0000 7f02e8ff9640 1 -- 172.21.15.78:0/4056923558 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f02f8004400 con 0x564ded62dd80 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.087+0000 7f0309866740 10 client.26932 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.087+0000 7f0309866740 1 -- 172.21.15.78:0/4056923558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564ded63c2f0 con 0x564ded62dd80 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.087+0000 7f02e8ff9640 1 -- 172.21.15.78:0/4056923558 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f8004050 con 0x564ded62dd80 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.087+0000 7f02e8ff9640 1 -- 172.21.15.78:0/4056923558 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f02f802b4d0 con 0x564ded62dd80 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.088+0000 7f0309866740 10 client.26932 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.088+0000 7f0309866740 10 client.26932 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:11:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.088+0000 7f0309866740 20 client.26932 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.088+0000 7f0309866740 1 --2- 172.21.15.78:0/4056923558 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x564ded63d6a0 0x564ded63fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.088+0000 7f0309866740 4 client.26932 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.088+0000 7f0309866740 1 -- 172.21.15.78:0/4056923558 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ded599df0 con 0x564ded63d6a0 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.088+0000 7f0304a2c640 1 --2- 172.21.15.78:0/4056923558 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x564ded63d6a0 0x564ded63fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.088+0000 7f0304a2c640 1 --2- 172.21.15.78:0/4056923558 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x564ded63d6a0 0x564ded63fa90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5fe6ec21e39710cc server_cookie=cf56f5ba6ee4dc94 in_seq=0 out_seq=0 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.088+0000 7f02e8ff9640 10 client.26932 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.089+0000 7f02e8ff9640 1 -- 172.21.15.78:0/4056923558 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ded599df0 con 0x564ded63d6a0 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.089+0000 7f02e8ff9640 10 client.26932 handle_command_reply: tid=0 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.089+0000 7f0309866740 1 client.26932 shutdown 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.089+0000 7f0309866740 1 -- 172.21.15.78:0/4056923558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded62dd80 msgr2=0x564ded62b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.089+0000 7f0309866740 1 --2- 172.21.15.78:0/4056923558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded62dd80 0x564ded62b040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f02f8011dd0 tx=0x7f02f8000f40 comp rx=0 tx=0).stop 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.089+0000 7f0309866740 1 -- 172.21.15.78:0/4056923558 shutdown_connections 2024-09-11T01:11:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.089+0000 7f0309866740 1 --2- 172.21.15.78:0/4056923558 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x564ded63d6a0 0x564ded63fa90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.089+0000 7f0309866740 1 --2- 172.21.15.78:0/4056923558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded62dd80 0x564ded62b040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.089+0000 7f02e8ff9640 0 client.26932 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.089+0000 7f0309866740 1 --2- 172.21.15.78:0/4056923558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded621df0 0x564ded62aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.089+0000 7f0309866740 1 --2- 172.21.15.78:0/4056923558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded62d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.089+0000 7f0309866740 1 -- 172.21.15.78:0/4056923558 >> 172.21.15.78:0/4056923558 conn(0x564ded61f300 msgr2=0x564ded624e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.090+0000 7f0309866740 1 -- 172.21.15.78:0/4056923558 shutdown_connections 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.090+0000 7f0309866740 1 -- 172.21.15.78:0/4056923558 wait complete. 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.090+0000 7f0309866740 20 client.26932 trim_cache size 0 max 16384 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.104+0000 7f0309866740 1 Processor -- start 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.104+0000 7f0309866740 1 -- start start 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0309866740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded639c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0309866740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded62c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0309866740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded627c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0309866740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03001ac530 con 0x564ded62dd80 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0309866740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03001ae130 con 0x564ded6214d0 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0309866740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03001acc00 con 0x564ded621df0 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0305a2e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded627c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0304a2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded62c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0305a2e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded627c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57070/0 (socket says 172.21.15.78:57070) 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0305a2e640 1 -- 172.21.15.78:0/2998960296 learned_addr learned my addr 172.21.15.78:0/2998960296 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0305a2e640 1 --2- 172.21.15.78:0/2998960296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded639c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0304a2c640 1 -- 172.21.15.78:0/2998960296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 msgr2=0x564ded639c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0304a2c640 1 --2- 172.21.15.78:0/2998960296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded639c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0304a2c640 1 -- 172.21.15.78:0/2998960296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 msgr2=0x564ded627c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0304a2c640 1 --2- 172.21.15.78:0/2998960296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded627c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0304a2c640 1 -- 172.21.15.78:0/2998960296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ded61f080 con 0x564ded621df0 2024-09-11T01:11:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.105+0000 7f0304a2c640 1 --2- 172.21.15.78:0/2998960296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded62c7c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x564ded6400e0 tx=0x7f02f0005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9244fe9b163d4b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.106+0000 7f02eaffd640 1 -- 172.21.15.78:0/2998960296 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0035040 con 0x564ded621df0 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.106+0000 7f02eaffd640 1 -- 172.21.15.78:0/2998960296 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f0005040 con 0x564ded621df0 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.106+0000 7f02eaffd640 1 -- 172.21.15.78:0/2998960296 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0004840 con 0x564ded621df0 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.106+0000 7f0309866740 1 -- 172.21.15.78:0/2998960296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 msgr2=0x564ded62c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.106+0000 7f0309866740 1 --2- 172.21.15.78:0/2998960296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded62c7c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x564ded6400e0 tx=0x7f02f0005a20 comp rx=0 tx=0).stop 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.106+0000 7f0309866740 1 -- 172.21.15.78:0/2998960296 shutdown_connections 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.106+0000 7f0309866740 1 --2- 172.21.15.78:0/2998960296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded627c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.106+0000 7f0309866740 1 --2- 172.21.15.78:0/2998960296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded62c7c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.106+0000 7f0309866740 1 --2- 172.21.15.78:0/2998960296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded639c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.106+0000 7f0309866740 1 -- 172.21.15.78:0/2998960296 >> 172.21.15.78:0/2998960296 conn(0x564ded61f300 msgr2=0x564ded62c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.106+0000 7f0309866740 1 -- 172.21.15.78:0/2998960296 shutdown_connections 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.106+0000 7f0309866740 1 -- 172.21.15.78:0/2998960296 wait complete. 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0309866740 1 -- start start 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0309866740 1 Processor -- start 2024-09-11T01:11:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0309866740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded632500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0309866740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded632a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0309866740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded6a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0309866740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03001ad9a0 con 0x564ded62dd80 2024-09-11T01:11:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0309866740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03001ad2d0 con 0x564ded6214d0 2024-09-11T01:11:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0309866740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03001aa530 con 0x564ded621df0 2024-09-11T01:11:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0309866740 10 client.0 _ll_register_callbacks cb 0x7f02fc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0305a2e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded632500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0305a2e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded632500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36036/0 (socket says 172.21.15.78:36036) 2024-09-11T01:11:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0304a2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded632a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.107+0000 7f0305a2e640 1 -- 172.21.15.78:0/805547724 learned_addr learned my addr 172.21.15.78:0/805547724 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f0305a2e640 1 --2- 172.21.15.78:0/805547724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded6a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f0305a2e640 1 -- 172.21.15.78:0/805547724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 msgr2=0x564ded632a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f0305a2e640 1 --2- 172.21.15.78:0/805547724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded632a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f0305a2e640 1 -- 172.21.15.78:0/805547724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 msgr2=0x564ded6a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f0305a2e640 1 --2- 172.21.15.78:0/805547724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded6a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f0305a2e640 1 -- 172.21.15.78:0/805547724 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ded6a3a20 con 0x564ded6214d0 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f0305a2e640 1 --2- 172.21.15.78:0/805547724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded632500 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02f8002230 tx=0x7f02f802e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4675128b4655cc0c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f02e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f02e8ff9640 1 -- 172.21.15.78:0/805547724 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8003e80 con 0x564ded6214d0 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f0309866740 1 -- 172.21.15.78:0/805547724 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f0005040 con 0x564ded6214d0 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f02e8ff9640 1 -- 172.21.15.78:0/805547724 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f02f8004020 con 0x564ded6214d0 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f02e8ff9640 1 -- 172.21.15.78:0/805547724 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f02f8004220 con 0x564ded6214d0 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f0309866740 10 client.27005 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.108+0000 7f0309866740 1 -- 172.21.15.78:0/805547724 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564ded6aadc0 con 0x564ded6214d0 2024-09-11T01:11:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.109+0000 7f02e8ff9640 1 -- 172.21.15.78:0/805547724 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f0005040 con 0x564ded6214d0 2024-09-11T01:11:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.109+0000 7f02e8ff9640 1 -- 172.21.15.78:0/805547724 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f02f802b4b0 con 0x564ded6214d0 2024-09-11T01:11:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.109+0000 7f0309866740 10 client.27005 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.109+0000 7f0309866740 10 client.27005 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:11:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.109+0000 7f0309866740 20 client.27005 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.109+0000 7f0309866740 1 --2- 172.21.15.78:0/805547724 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x564ded6ab4b0 0x564ded6ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.109+0000 7f0309866740 4 client.27005 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:11:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.109+0000 7f0309866740 1 -- 172.21.15.78:0/805547724 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ded5958a0 con 0x564ded6ab4b0 2024-09-11T01:11:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.109+0000 7f0304a2c640 1 --2- 172.21.15.78:0/805547724 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x564ded6ab4b0 0x564ded6ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.109+0000 7f02e8ff9640 10 client.27005 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:11:15.133 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:11:15.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.109+0000 7f0304a2c640 1 --2- 172.21.15.78:0/805547724 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x564ded6ab4b0 0x564ded6ad8d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f37cc9a93d47d364 server_cookie=866237ef9b060f1e in_seq=0 out_seq=0 2024-09-11T01:11:15.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f02e8ff9640 1 -- 172.21.15.78:0/805547724 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ded5958a0 con 0x564ded6ab4b0 2024-09-11T01:11:15.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f02e8ff9640 10 client.27005 handle_command_reply: tid=0 2024-09-11T01:11:15.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f0309866740 1 client.27005 shutdown 2024-09-11T01:11:15.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f0309866740 1 -- 172.21.15.78:0/805547724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 msgr2=0x564ded632500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:15.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f0309866740 1 --2- 172.21.15.78:0/805547724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded632500 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02f8002230 tx=0x7f02f802e040 comp rx=0 tx=0).stop 2024-09-11T01:11:15.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f0309866740 1 -- 172.21.15.78:0/805547724 shutdown_connections 2024-09-11T01:11:15.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f0309866740 1 --2- 172.21.15.78:0/805547724 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x564ded6ab4b0 0x564ded6ad8d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f0309866740 1 --2- 172.21.15.78:0/805547724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ded62dd80 0x564ded6a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f0309866740 1 --2- 172.21.15.78:0/805547724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ded621df0 0x564ded632a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f02e8ff9640 0 client.27005 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f0309866740 1 --2- 172.21.15.78:0/805547724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ded6214d0 0x564ded632500 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f0309866740 1 -- 172.21.15.78:0/805547724 >> 172.21.15.78:0/805547724 conn(0x564ded61f300 msgr2=0x564ded62c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f0309866740 1 -- 172.21.15.78:0/805547724 shutdown_connections 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f0309866740 1 -- 172.21.15.78:0/805547724 wait complete. 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.110+0000 7f0309866740 20 client.27005 trim_cache size 0 max 16384 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.111+0000 7f0307491640 1 -- 172.21.15.78:0/1904903830 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02ec0b5450 msgr2=0x7f02ec0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.111+0000 7f0307491640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02ec0b5450 0x7f02ec0b7910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0e3bf0 tx=0x7f02fc0eb040 comp rx=0 tx=0).stop 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.111+0000 7f0307491640 1 -- 172.21.15.78:0/1904903830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300101250 msgr2=0x7f03001992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.111+0000 7f0307491640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300101250 0x7f03001992f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00e9f0 tx=0x7f02fc00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.111+0000 7f0307491640 1 -- 172.21.15.78:0/1904903830 shutdown_connections 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.111+0000 7f0307491640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f02ec0b5450 0x7f02ec0b7910 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.111+0000 7f0307491640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0300105f40 0x7f030019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.111+0000 7f0307491640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03001055a0 0x7f0300199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.111+0000 7f0307491640 1 --2- 172.21.15.78:0/1904903830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300101250 0x7f03001992f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:15.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.111+0000 7f0307491640 1 -- 172.21.15.78:0/1904903830 >> 172.21.15.78:0/1904903830 conn(0x7f0300078140 msgr2=0x7f03000ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:15.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.111+0000 7f0307491640 1 -- 172.21.15.78:0/1904903830 shutdown_connections 2024-09-11T01:11:15.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:15.111+0000 7f0307491640 1 -- 172.21.15.78:0/1904903830 wait complete. 2024-09-11T01:11:15.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 873696 5.94 MB/sec execute 563 sec latency 38.401 ms 2024-09-11T01:11:15.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:15 smithi110 ceph-mon[34712]: pgmap v706: 97 pgs: 97 active+clean; 724 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 322 op/s 2024-09-11T01:11:15.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:11:15.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:11:15.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:11:15.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:15 smithi112 ceph-mon[35384]: pgmap v706: 97 pgs: 97 active+clean; 724 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 322 op/s 2024-09-11T01:11:15.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:11:15.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:11:15.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:11:15.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:15 smithi078 ceph-mon[31315]: pgmap v706: 97 pgs: 97 active+clean; 724 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 322 op/s 2024-09-11T01:11:15.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:11:15.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:11:15.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:11:16.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 875079 5.94 MB/sec execute 564 sec latency 56.717 ms 2024-09-11T01:11:16.888 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:11:17.161 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:11:17.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 877291 5.95 MB/sec execute 565 sec latency 15.854 ms 2024-09-11T01:11:17.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:17 smithi078 ceph-mon[31315]: pgmap v707: 97 pgs: 97 active+clean; 730 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 294 op/s 2024-09-11T01:11:17.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.650+0000 7feb1cd81640 1 -- 172.21.15.78:0/3977930879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb180754e0 msgr2=0x7feb180758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:17.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.650+0000 7feb1cd81640 1 --2- 172.21.15.78:0/3977930879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb180754e0 0x7feb180758c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7feb08009900 tx=0x7feb0802a410 comp rx=0 tx=0).stop 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.650+0000 7feb1cd81640 1 -- 172.21.15.78:0/3977930879 shutdown_connections 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.650+0000 7feb1cd81640 1 --2- 172.21.15.78:0/3977930879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb1810f3d0 0x7feb181117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.650+0000 7feb1cd81640 1 --2- 172.21.15.78:0/3977930879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb18075e00 0x7feb1810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.650+0000 7feb1cd81640 1 --2- 172.21.15.78:0/3977930879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb180754e0 0x7feb180758c0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.650+0000 7feb1cd81640 1 -- 172.21.15.78:0/3977930879 >> 172.21.15.78:0/3977930879 conn(0x7feb180fe830 msgr2=0x7feb18100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.650+0000 7feb1cd81640 1 -- 172.21.15.78:0/3977930879 shutdown_connections 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.650+0000 7feb1cd81640 1 -- 172.21.15.78:0/3977930879 wait complete. 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.651+0000 7feb1cd81640 1 Processor -- start 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.651+0000 7feb1cd81640 1 -- start start 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1cd81640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb180754e0 0x7feb1819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1cd81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb18075e00 0x7feb1819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1cd81640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb1810f3d0 0x7feb181a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1cd81640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb18114f90 con 0x7feb180754e0 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1cd81640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feb18114e10 con 0x7feb1810f3d0 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1cd81640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb18115110 con 0x7feb18075e00 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb180754e0 0x7feb1819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb180754e0 0x7feb1819b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57100/0 (socket says 172.21.15.78:57100) 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1659c640 1 -- 172.21.15.78:0/1911954190 learned_addr learned my addr 172.21.15.78:0/1911954190 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb16d9d640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb1810f3d0 0x7feb181a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb15d9b640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb18075e00 0x7feb1819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1659c640 1 -- 172.21.15.78:0/1911954190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb18075e00 msgr2=0x7feb1819b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1659c640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb18075e00 0x7feb1819b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1659c640 1 -- 172.21.15.78:0/1911954190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb1810f3d0 msgr2=0x7feb181a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1659c640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb1810f3d0 0x7feb181a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1659c640 1 -- 172.21.15.78:0/1911954190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb181a1550 con 0x7feb180754e0 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1659c640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb180754e0 0x7feb1819b4a0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7feb08005940 tx=0x7feb080040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:17.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb077fe640 1 -- 172.21.15.78:0/1911954190 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0803f030 con 0x7feb180754e0 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb077fe640 1 -- 172.21.15.78:0/1911954190 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb080044c0 con 0x7feb180754e0 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.652+0000 7feb1cd81640 1 -- 172.21.15.78:0/1911954190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb181a17e0 con 0x7feb180754e0 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.653+0000 7feb077fe640 1 -- 172.21.15.78:0/1911954190 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08035070 con 0x7feb180754e0 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.653+0000 7feb1cd81640 1 -- 172.21.15.78:0/1911954190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb181a7f00 con 0x7feb180754e0 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.655+0000 7feb1f156740 1 Processor -- start 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.655+0000 7feb1f156740 1 -- start start 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.655+0000 7feb077fe640 1 -- 172.21.15.78:0/1911954190 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7feb0803ba20 con 0x7feb180754e0 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.655+0000 7feb1f156740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c644d0 0x55c433c648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.655+0000 7feb1f156740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c64df0 0x55c433c70840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.655+0000 7feb1f156740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c73170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.655+0000 7feb1f156740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c433bdaea0 con 0x55c433c70d80 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.655+0000 7feb1f156740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c433bca2d0 con 0x55c433c644d0 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.655+0000 7feb1f156740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c433bce0e0 con 0x55c433c64df0 2024-09-11T01:11:17.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb16d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c644d0 0x55c433c648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb15d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c73170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb16d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c644d0 0x55c433c648b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36068/0 (socket says 172.21.15.78:36068) 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb16d9d640 1 -- 172.21.15.78:0/511825010 learned_addr learned my addr 172.21.15.78:0/511825010 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb1659c640 1 --2- 172.21.15.78:0/511825010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c64df0 0x55c433c70840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb15d9b640 1 -- 172.21.15.78:0/511825010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c64df0 msgr2=0x55c433c70840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb15d9b640 1 --2- 172.21.15.78:0/511825010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c64df0 0x55c433c70840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb15d9b640 1 -- 172.21.15.78:0/511825010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c644d0 msgr2=0x55c433c648b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb15d9b640 1 --2- 172.21.15.78:0/511825010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c644d0 0x55c433c648b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb15d9b640 1 -- 172.21.15.78:0/511825010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c433928650 con 0x55c433c70d80 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb16d9d640 1 --2- 172.21.15.78:0/511825010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c644d0 0x55c433c648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb15d9b640 1 --2- 172.21.15.78:0/511825010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c73170 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009bf0 tx=0x7feb0c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5f41ffe850dacb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb04ff9640 1 -- 172.21.15.78:0/511825010 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c004050 con 0x55c433c70d80 2024-09-11T01:11:17.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb04ff9640 1 -- 172.21.15.78:0/511825010 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0c0054d0 con 0x55c433c70d80 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb04ff9640 1 -- 172.21.15.78:0/511825010 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c004d00 con 0x55c433c70d80 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.656+0000 7feb077fe640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feb000b54d0 0x7feb000b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.657+0000 7feb16d9d640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feb000b54d0 0x7feb000b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.657+0000 7feb077fe640 1 -- 172.21.15.78:0/1911954190 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7feb081064e0 con 0x7feb180754e0 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.657+0000 7feb16d9d640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feb000b54d0 0x7feb000b7990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feb10009fd0 tx=0x7feb10009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.657+0000 7feb1f156740 1 -- 172.21.15.78:0/511825010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 msgr2=0x55c433c73170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.657+0000 7feb1f156740 1 --2- 172.21.15.78:0/511825010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c73170 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009bf0 tx=0x7feb0c011810 comp rx=0 tx=0).stop 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.657+0000 7feb1f156740 1 -- 172.21.15.78:0/511825010 shutdown_connections 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.657+0000 7feb1f156740 1 --2- 172.21.15.78:0/511825010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c73170 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.657+0000 7feb1f156740 1 --2- 172.21.15.78:0/511825010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c64df0 0x55c433c70840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.657+0000 7feb1f156740 1 --2- 172.21.15.78:0/511825010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c644d0 0x55c433c648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.657+0000 7feb1f156740 1 -- 172.21.15.78:0/511825010 >> 172.21.15.78:0/511825010 conn(0x55c433c62300 msgr2=0x55c433c634a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:17.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.657+0000 7feb1f156740 1 -- 172.21.15.78:0/511825010 shutdown_connections 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.657+0000 7feb1f156740 1 -- 172.21.15.78:0/511825010 wait complete. 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb1f156740 1 -- start start 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb1f156740 1 Processor -- start 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb1f156740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c644d0 0x55c433c705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb1f156740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c6d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb1f156740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c6df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb1f156740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c433a2b700 con 0x55c433c70d80 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb1f156740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c433b65160 con 0x55c433c64df0 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb1f156740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c433c73a00 con 0x55c433c644d0 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb1f156740 10 client.0 _ll_register_callbacks cb 0x55c433b7e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb16d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c6df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb16d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c6df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57104/0 (socket says 172.21.15.78:57104) 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb15d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c6d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb16d9d640 1 -- 172.21.15.78:0/3791719358 learned_addr learned my addr 172.21.15.78:0/3791719358 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:17.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb1659c640 1 --2- 172.21.15.78:0/3791719358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c644d0 0x55c433c705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb16d9d640 1 -- 172.21.15.78:0/3791719358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c644d0 msgr2=0x55c433c705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:17.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb16d9d640 1 --2- 172.21.15.78:0/3791719358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c644d0 0x55c433c705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb16d9d640 1 -- 172.21.15.78:0/3791719358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 msgr2=0x55c433c6d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:17.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb16d9d640 1 --2- 172.21.15.78:0/3791719358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c6d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.658+0000 7feb16d9d640 1 -- 172.21.15.78:0/3791719358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c433b92c90 con 0x55c433c70d80 2024-09-11T01:11:17.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feb16d9d640 1 --2- 172.21.15.78:0/3791719358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c6df30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7feb1000fc10 tx=0x7feb10004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ee0b52929ad371 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:17.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feafdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:11:17.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feafdffb640 1 -- 172.21.15.78:0/3791719358 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1000fdc0 con 0x55c433c70d80 2024-09-11T01:11:17.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feafdffb640 1 -- 172.21.15.78:0/3791719358 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7feb10017030 con 0x55c433c70d80 2024-09-11T01:11:17.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feb1f156740 1 -- 172.21.15.78:0/3791719358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0c0054d0 con 0x55c433c70d80 2024-09-11T01:11:17.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feafdffb640 1 -- 172.21.15.78:0/3791719358 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7feb10004b20 con 0x55c433c70d80 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feb1f156740 10 client.17823 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feb1f156740 1 -- 172.21.15.78:0/3791719358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c433c7f300 con 0x55c433c70d80 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feafdffb640 1 -- 172.21.15.78:0/3791719358 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0c0054d0 con 0x55c433c70d80 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feafdffb640 1 -- 172.21.15.78:0/3791719358 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7feb100052e0 con 0x55c433c70d80 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feb1f156740 10 client.17823 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feb1f156740 10 client.17823 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feb1f156740 20 client.17823 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feb1f156740 1 --2- 172.21.15.78:0/3791719358 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c433c808b0 0x55c433c82d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feb1f156740 4 client.17823 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.659+0000 7feb1f156740 1 -- 172.21.15.78:0/3791719358 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c433bdcdf0 con 0x55c433c808b0 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.660+0000 7feb1659c640 1 --2- 172.21.15.78:0/3791719358 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c433c808b0 0x55c433c82d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.660+0000 7feb1659c640 1 --2- 172.21.15.78:0/3791719358 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c433c808b0 0x55c433c82d30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c198657871fd13c1 server_cookie=fdc89fd805cd92e4 in_seq=0 out_seq=0 2024-09-11T01:11:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.660+0000 7feafdffb640 10 client.17823 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feafdffb640 1 -- 172.21.15.78:0/3791719358 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c433bdcdf0 con 0x55c433c808b0 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feafdffb640 10 client.17823 handle_command_reply: tid=0 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feb1f156740 1 client.17823 shutdown 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feb1f156740 1 -- 172.21.15.78:0/3791719358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 msgr2=0x55c433c6df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feb1f156740 1 --2- 172.21.15.78:0/3791719358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c6df30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7feb1000fc10 tx=0x7feb10004ab0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feb1f156740 1 -- 172.21.15.78:0/3791719358 shutdown_connections 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feb1f156740 1 --2- 172.21.15.78:0/3791719358 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c433c808b0 0x55c433c82d30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feb1f156740 1 --2- 172.21.15.78:0/3791719358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c6df30 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feb1f156740 1 --2- 172.21.15.78:0/3791719358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c6d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feb1f156740 1 --2- 172.21.15.78:0/3791719358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c644d0 0x55c433c705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feb1f156740 1 -- 172.21.15.78:0/3791719358 >> 172.21.15.78:0/3791719358 conn(0x55c433c62300 msgr2=0x55c433c67e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feafdffb640 0 client.17823 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feb1f156740 1 -- 172.21.15.78:0/3791719358 shutdown_connections 2024-09-11T01:11:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feb1f156740 1 -- 172.21.15.78:0/3791719358 wait complete. 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.661+0000 7feb1f156740 20 client.17823 trim_cache size 0 max 16384 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.676+0000 7feb1f156740 1 Processor -- start 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.676+0000 7feb1f156740 1 -- start start 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.676+0000 7feb1f156740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c644d0 0x55c433c6ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.676+0000 7feb1f156740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c6b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.676+0000 7feb1f156740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c70d80 0x55c433c88600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.676+0000 7feb1f156740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb181b07b0 con 0x55c433c644d0 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.676+0000 7feb1f156740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feb181b23b0 con 0x55c433c64df0 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.676+0000 7feb1f156740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb181b0e80 con 0x55c433c70d80 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.677+0000 7feb1659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c644d0 0x55c433c6ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.677+0000 7feb1659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c644d0 0x55c433c6ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57112/0 (socket says 172.21.15.78:57112) 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.677+0000 7feb1659c640 1 -- 172.21.15.78:0/1448549011 learned_addr learned my addr 172.21.15.78:0/1448549011 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.677+0000 7feb15d9b640 1 --2- 172.21.15.78:0/1448549011 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c6b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.677+0000 7feb16d9d640 1 --2- 172.21.15.78:0/1448549011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c70d80 0x55c433c88600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.677+0000 7feb1659c640 1 -- 172.21.15.78:0/1448549011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c70d80 msgr2=0x55c433c88600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.677+0000 7feb1659c640 1 --2- 172.21.15.78:0/1448549011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c70d80 0x55c433c88600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.677+0000 7feb1659c640 1 -- 172.21.15.78:0/1448549011 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 msgr2=0x55c433c6b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.677+0000 7feb1659c640 1 --2- 172.21.15.78:0/1448549011 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c6b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.677+0000 7feb1659c640 1 -- 172.21.15.78:0/1448549011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c433c70910 con 0x55c433c644d0 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.677+0000 7feb16d9d640 1 --2- 172.21.15.78:0/1448549011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c70d80 0x55c433c88600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.677+0000 7feb1659c640 1 --2- 172.21.15.78:0/1448549011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c644d0 0x55c433c6ac00 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7feb08006010 tx=0x7feb0803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccd6fe4b3a548dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.678+0000 7feafffff640 1 -- 172.21.15.78:0/1448549011 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08049040 con 0x55c433c644d0 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.678+0000 7feafffff640 1 -- 172.21.15.78:0/1448549011 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb080cc560 con 0x55c433c644d0 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.678+0000 7feafffff640 1 -- 172.21.15.78:0/1448549011 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08103ef0 con 0x55c433c644d0 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.679+0000 7feb1f156740 1 -- 172.21.15.78:0/1448549011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c644d0 msgr2=0x55c433c6ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.679+0000 7feb1f156740 1 --2- 172.21.15.78:0/1448549011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c644d0 0x55c433c6ac00 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7feb08006010 tx=0x7feb0803b460 comp rx=0 tx=0).stop 2024-09-11T01:11:17.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.679+0000 7feb1f156740 1 -- 172.21.15.78:0/1448549011 shutdown_connections 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.679+0000 7feb1f156740 1 --2- 172.21.15.78:0/1448549011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c70d80 0x55c433c88600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.679+0000 7feb1f156740 1 --2- 172.21.15.78:0/1448549011 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c6b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.679+0000 7feb1f156740 1 --2- 172.21.15.78:0/1448549011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c644d0 0x55c433c6ac00 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.679+0000 7feb1f156740 1 -- 172.21.15.78:0/1448549011 >> 172.21.15.78:0/1448549011 conn(0x55c433c62300 msgr2=0x55c433c63350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.679+0000 7feb1f156740 1 -- 172.21.15.78:0/1448549011 shutdown_connections 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.679+0000 7feb1f156740 1 -- 172.21.15.78:0/1448549011 wait complete. 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.679+0000 7feb1f156740 1 -- start start 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.679+0000 7feb1f156740 1 Processor -- start 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb1f156740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c644d0 0x55c433c74ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb1f156740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c75220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb1f156740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c75760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb1f156740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb181b1c20 con 0x55c433c70d80 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb1f156740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feb181b1550 con 0x55c433c64df0 2024-09-11T01:11:17.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb1f156740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb181ae7b0 con 0x55c433c644d0 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb1f156740 10 client.0 _ll_register_callbacks cb 0x7feb10011e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb16d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c75760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb16d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c75760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57116/0 (socket says 172.21.15.78:57116) 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb16d9d640 1 -- 172.21.15.78:0/1418930687 learned_addr learned my addr 172.21.15.78:0/1418930687 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb15d9b640 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c75220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb1659c640 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c644d0 0x55c433c74ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb15d9b640 1 -- 172.21.15.78:0/1418930687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c644d0 msgr2=0x55c433c74ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb15d9b640 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c644d0 0x55c433c74ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb15d9b640 1 -- 172.21.15.78:0/1418930687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 msgr2=0x55c433c75760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb15d9b640 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c75760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.680+0000 7feb15d9b640 1 -- 172.21.15.78:0/1418930687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c433cec900 con 0x55c433c64df0 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.681+0000 7feb1659c640 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c644d0 0x55c433c74ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:11:17.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.681+0000 7feb16d9d640 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c75760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.681+0000 7feb15d9b640 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c75220 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feb0c000c00 tx=0x7feb0c0096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5295e18491f27b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.681+0000 7feafdffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.681+0000 7feafdffb640 1 -- 172.21.15.78:0/1418930687 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0035f0 con 0x55c433c64df0 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.681+0000 7feb1f156740 1 -- 172.21.15.78:0/1418930687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb080cc560 con 0x55c433c64df0 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.681+0000 7feafdffb640 1 -- 172.21.15.78:0/1418930687 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7feb0c003790 con 0x55c433c64df0 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.681+0000 7feafdffb640 1 -- 172.21.15.78:0/1418930687 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7feb0c003990 con 0x55c433c64df0 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.681+0000 7feb1f156740 10 client.27023 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.681+0000 7feb1f156740 1 -- 172.21.15.78:0/1418930687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c433cedc00 con 0x55c433c64df0 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.682+0000 7feafdffb640 1 -- 172.21.15.78:0/1418930687 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb080cc560 con 0x55c433c64df0 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.682+0000 7feafdffb640 1 -- 172.21.15.78:0/1418930687 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7feb0c02db90 con 0x55c433c64df0 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.682+0000 7feb1f156740 10 client.27023 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.682+0000 7feb1f156740 10 client.27023 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.682+0000 7feb1f156740 20 client.27023 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.682+0000 7feb1f156740 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c433cee190 0x55c433cf05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.682+0000 7feb1f156740 4 client.27023 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.682+0000 7feb1f156740 1 -- 172.21.15.78:0/1418930687 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c433bd88a0 con 0x55c433cee190 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.682+0000 7feb1659c640 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c433cee190 0x55c433cf05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.682+0000 7feafdffb640 10 client.27023 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.682+0000 7feb1659c640 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c433cee190 0x55c433cf05b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=debafe07920efb45 server_cookie=4637ac6d4954ed21 in_seq=0 out_seq=0 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feafdffb640 1 -- 172.21.15.78:0/1418930687 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c433bd88a0 con 0x55c433cee190 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feafdffb640 10 client.27023 handle_command_reply: tid=0 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1f156740 1 client.27023 shutdown 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1f156740 1 -- 172.21.15.78:0/1418930687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 msgr2=0x55c433c75220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1f156740 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c75220 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feb0c000c00 tx=0x7feb0c0096e0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1f156740 1 -- 172.21.15.78:0/1418930687 shutdown_connections 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1f156740 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c433cee190 0x55c433cf05b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1f156740 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c433c70d80 0x55c433c75760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1f156740 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c433c64df0 0x55c433c75220 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feafdffb640 0 client.27023 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:11:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1f156740 1 --2- 172.21.15.78:0/1418930687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c433c644d0 0x55c433c74ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1f156740 1 -- 172.21.15.78:0/1418930687 >> 172.21.15.78:0/1418930687 conn(0x55c433c62300 msgr2=0x55c433c6f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1f156740 1 -- 172.21.15.78:0/1418930687 shutdown_connections 2024-09-11T01:11:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1f156740 1 -- 172.21.15.78:0/1418930687 wait complete. 2024-09-11T01:11:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1f156740 20 client.27023 trim_cache size 0 max 16384 2024-09-11T01:11:17.700 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:11:17.701 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:17 smithi110 ceph-mon[34712]: pgmap v707: 97 pgs: 97 active+clean; 730 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 294 op/s 2024-09-11T01:11:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1cd81640 1 -- 172.21.15.78:0/1911954190 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feb000b54d0 msgr2=0x7feb000b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1cd81640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feb000b54d0 0x7feb000b7990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feb10009fd0 tx=0x7feb10009500 comp rx=0 tx=0).stop 2024-09-11T01:11:17.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1cd81640 1 -- 172.21.15.78:0/1911954190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb180754e0 msgr2=0x7feb1819b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:17.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.683+0000 7feb1cd81640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb180754e0 0x7feb1819b4a0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7feb08005940 tx=0x7feb080040c0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.684+0000 7feb1cd81640 1 -- 172.21.15.78:0/1911954190 shutdown_connections 2024-09-11T01:11:17.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.684+0000 7feb1cd81640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feb000b54d0 0x7feb000b7990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.684+0000 7feb1cd81640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb1810f3d0 0x7feb181a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.684+0000 7feb1cd81640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb18075e00 0x7feb1819b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.684+0000 7feb1cd81640 1 --2- 172.21.15.78:0/1911954190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb180754e0 0x7feb1819b4a0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:17.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.684+0000 7feb1cd81640 1 -- 172.21.15.78:0/1911954190 >> 172.21.15.78:0/1911954190 conn(0x7feb180fe830 msgr2=0x7feb18111590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:17.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.684+0000 7feb1cd81640 1 -- 172.21.15.78:0/1911954190 shutdown_connections 2024-09-11T01:11:17.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:17.684+0000 7feb1cd81640 1 -- 172.21.15.78:0/1911954190 wait complete. 2024-09-11T01:11:17.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:17 smithi112 ceph-mon[35384]: pgmap v707: 97 pgs: 97 active+clean; 730 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 294 op/s 2024-09-11T01:11:18.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 879131 5.95 MB/sec execute 566 sec latency 16.388 ms 2024-09-11T01:11:18.616 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:11:18.956 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:11:19.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.415+0000 7f06eabdf640 1 -- 172.21.15.78:0/1024220844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e41045c0 msgr2=0x7f06e410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:19.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.415+0000 7f06eabdf640 1 --2- 172.21.15.78:0/1024220844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e41045c0 0x7f06e410cab0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009900 tx=0x7f06d002a410 comp rx=0 tx=0).stop 2024-09-11T01:11:19.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.415+0000 7f06eabdf640 1 -- 172.21.15.78:0/1024220844 shutdown_connections 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.415+0000 7f06eabdf640 1 --2- 172.21.15.78:0/1024220844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e410d100 0x7f06e410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.415+0000 7f06eabdf640 1 --2- 172.21.15.78:0/1024220844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e41045c0 0x7f06e410cab0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.415+0000 7f06eabdf640 1 --2- 172.21.15.78:0/1024220844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06e4103ca0 0x7f06e4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.415+0000 7f06eabdf640 1 -- 172.21.15.78:0/1024220844 >> 172.21.15.78:0/1024220844 conn(0x7f06e40fd700 msgr2=0x7f06e40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.415+0000 7f06eabdf640 1 -- 172.21.15.78:0/1024220844 shutdown_connections 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.415+0000 7f06eabdf640 1 -- 172.21.15.78:0/1024220844 wait complete. 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06eabdf640 1 Processor -- start 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06eabdf640 1 -- start start 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06eabdf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06e4103ca0 0x7f06e41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06eabdf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e41045c0 0x7f06e41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06eabdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e410d100 0x7f06e419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06eabdf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4112d70 con 0x7f06e410d100 2024-09-11T01:11:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06eabdf640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4112bf0 con 0x7f06e4103ca0 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06eabdf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4112ef0 con 0x7f06e41045c0 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06e917c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e410d100 0x7f06e419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e41045c0 0x7f06e41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06e897b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06e4103ca0 0x7f06e41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06e917c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e410d100 0x7f06e419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57136/0 (socket says 172.21.15.78:57136) 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e41045c0 0x7f06e41997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:38086/0 (socket says 172.21.15.78:38086) 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.416+0000 7f06e917c640 1 -- 172.21.15.78:0/1356922443 learned_addr learned my addr 172.21.15.78:0/1356922443 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.417+0000 7f06e917c640 1 -- 172.21.15.78:0/1356922443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e41045c0 msgr2=0x7f06e41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.417+0000 7f06e917c640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e41045c0 0x7f06e41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.417+0000 7f06e917c640 1 -- 172.21.15.78:0/1356922443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06e4103ca0 msgr2=0x7f06e41992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.417+0000 7f06e917c640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06e4103ca0 0x7f06e41992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.417+0000 7f06e917c640 1 -- 172.21.15.78:0/1356922443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e419f350 con 0x7f06e410d100 2024-09-11T01:11:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.417+0000 7f06e897b640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06e4103ca0 0x7f06e41992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.417+0000 7f06e917c640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e410d100 0x7f06e419eb80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00eb10 tx=0x7f06dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.417+0000 7f06e1ffb640 1 -- 172.21.15.78:0/1356922443 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc004b20 con 0x7f06e410d100 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.417+0000 7f06e1ffb640 1 -- 172.21.15.78:0/1356922443 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06dc004cc0 con 0x7f06e410d100 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.417+0000 7f06eabdf640 1 -- 172.21.15.78:0/1356922443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e419f640 con 0x7f06e410d100 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.417+0000 7f06e1ffb640 1 -- 172.21.15.78:0/1356922443 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc010670 con 0x7f06e410d100 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.417+0000 7f06eabdf640 1 -- 172.21.15.78:0/1356922443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e41a5ed0 con 0x7f06e410d100 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.420+0000 7f06e1ffb640 1 -- 172.21.15.78:0/1356922443 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f06dc005020 con 0x7f06e410d100 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.420+0000 7f06ecfb4740 1 Processor -- start 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.420+0000 7f06ecfb4740 1 -- start start 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.421+0000 7f06ecfb4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 0x561becebf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.421+0000 7f06ecfb4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 0x561bececfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.421+0000 7f06ecfb4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561beced0190 0x561beced2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.421+0000 7f06ecfb4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561bece35ea0 con 0x561becebfdf0 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.421+0000 7f06ecfb4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561bece252d0 con 0x561beced0190 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.421+0000 7f06ecfb4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561bece290e0 con 0x561becebf4d0 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.421+0000 7f06e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 0x561becebf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.421+0000 7f06e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 0x561becebf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:38088/0 (socket says 172.21.15.78:38088) 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.421+0000 7f06e3fff640 1 -- 172.21.15.78:0/1139366453 learned_addr learned my addr 172.21.15.78:0/1139366453 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.421+0000 7f06e897b640 1 --2- 172.21.15.78:0/1139366453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 0x561bececfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.421+0000 7f06e3fff640 1 --2- 172.21.15.78:0/1139366453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561beced0190 0x561beced2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.422+0000 7f06e897b640 1 -- 172.21.15.78:0/1139366453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 msgr2=0x561becebf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.422+0000 7f06e1ffb640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f06d40b54d0 0x7f06d40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.422+0000 7f06e897b640 1 --2- 172.21.15.78:0/1139366453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 0x561becebf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.422+0000 7f06e897b640 1 -- 172.21.15.78:0/1139366453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561beced0190 msgr2=0x561beced2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.422+0000 7f06e897b640 1 --2- 172.21.15.78:0/1139366453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561beced0190 0x561beced2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.422+0000 7f06e897b640 1 -- 172.21.15.78:0/1139366453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561becb83650 con 0x561becebfdf0 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.422+0000 7f06e3fff640 1 --2- 172.21.15.78:0/1139366453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 0x561becebf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.422+0000 7f06e917c640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f06d40b54d0 0x7f06d40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.422+0000 7f06e897b640 1 --2- 172.21.15.78:0/1139366453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 0x561bececfc50 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f06d8011440 tx=0x7f06d8011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ffd142a408d31f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.422+0000 7f06e1ffb640 1 -- 172.21.15.78:0/1356922443 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f06dc0e6480 con 0x7f06e410d100 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.423+0000 7f06ceffd640 1 -- 172.21.15.78:0/1139366453 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8004030 con 0x561becebfdf0 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.423+0000 7f06ceffd640 1 -- 172.21.15.78:0/1139366453 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d80054d0 con 0x561becebfdf0 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.423+0000 7f06ceffd640 1 -- 172.21.15.78:0/1139366453 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8004c20 con 0x561becebfdf0 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.424+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1139366453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 msgr2=0x561bececfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.424+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1139366453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 0x561bececfc50 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f06d8011440 tx=0x7f06d8011900 comp rx=0 tx=0).stop 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.424+0000 7f06e917c640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f06d40b54d0 0x7f06d40b7990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0a8f10 tx=0x7f06dc0a8040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.424+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1139366453 shutdown_connections 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.424+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1139366453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561beced0190 0x561beced2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.424+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1139366453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 0x561bececfc50 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.424+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1139366453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 0x561becebf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.424+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1139366453 >> 172.21.15.78:0/1139366453 conn(0x561becebd300 msgr2=0x561becebe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.424+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1139366453 shutdown_connections 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.424+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1139366453 wait complete. 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.424+0000 7f06ecfb4740 1 -- start start 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.425+0000 7f06ecfb4740 1 Processor -- start 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.425+0000 7f06ecfb4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebf4d0 0x561bececfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.425+0000 7f06ecfb4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebfdf0 0x561bececcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.425+0000 7f06ecfb4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561beced0190 0x561bececd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.425+0000 7f06ecfb4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561becc86700 con 0x561beced0190 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.425+0000 7f06ecfb4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561becdc0160 con 0x561becebf4d0 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.425+0000 7f06ecfb4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561beced2e40 con 0x561becebfdf0 2024-09-11T01:11:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.425+0000 7f06ecfb4740 10 client.0 _ll_register_callbacks cb 0x561becdd99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.425+0000 7f06e3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebf4d0 0x561bececfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.425+0000 7f06e3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebf4d0 0x561bececfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36128/0 (socket says 172.21.15.78:36128) 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.425+0000 7f06e3fff640 1 -- 172.21.15.78:0/1182106473 learned_addr learned my addr 172.21.15.78:0/1182106473 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.425+0000 7f06e3fff640 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561beced0190 0x561bececd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.426+0000 7f06e897b640 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebfdf0 0x561bececcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.426+0000 7f06e3fff640 1 -- 172.21.15.78:0/1182106473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebfdf0 msgr2=0x561bececcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.426+0000 7f06e3fff640 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebfdf0 0x561bececcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.426+0000 7f06e3fff640 1 -- 172.21.15.78:0/1182106473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561beced0190 msgr2=0x561bececd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.426+0000 7f06e3fff640 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561beced0190 0x561bececd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.426+0000 7f06e3fff640 1 -- 172.21.15.78:0/1182106473 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561becdedc90 con 0x561becebf4d0 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.426+0000 7f06e3fff640 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebf4d0 0x561bececfb20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06d0006010 tx=0x7f06d0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6efea6d1e4fb68cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.426+0000 7f06c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:11:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.428+0000 7f06c3fff640 1 -- 172.21.15.78:0/1182106473 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0035040 con 0x561becebf4d0 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.428+0000 7f06c3fff640 1 -- 172.21.15.78:0/1182106473 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f06d0004260 con 0x561becebf4d0 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.428+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1182106473 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d80054d0 con 0x561becebf4d0 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.429+0000 7f06c3fff640 1 -- 172.21.15.78:0/1182106473 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f06d0004460 con 0x561becebf4d0 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.429+0000 7f06ecfb4740 10 client.27035 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.429+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1182106473 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561becec5850 con 0x561becebf4d0 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.429+0000 7f06c3fff640 1 -- 172.21.15.78:0/1182106473 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d80054d0 con 0x561becebf4d0 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.429+0000 7f06c3fff640 1 -- 172.21.15.78:0/1182106473 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f06d003d570 con 0x561becebf4d0 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.429+0000 7f06ecfb4740 10 client.27035 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.429+0000 7f06ecfb4740 10 client.27035 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.429+0000 7f06ecfb4740 20 client.27035 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.429+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x561becec6c60 0x561becec90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.429+0000 7f06ecfb4740 4 client.27035 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.429+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1182106473 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bece37df0 con 0x561becec6c60 2024-09-11T01:11:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.430+0000 7f06e897b640 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x561becec6c60 0x561becec90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.430+0000 7f06c3fff640 10 client.27035 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.430+0000 7f06e897b640 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x561becec6c60 0x561becec90a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7cfe87ea612211be server_cookie=65578751958d8241 in_seq=0 out_seq=0 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06c3fff640 1 -- 172.21.15.78:0/1182106473 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561bece37df0 con 0x561becec6c60 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06c3fff640 10 client.27035 handle_command_reply: tid=0 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06ecfb4740 1 client.27035 shutdown 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1182106473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebf4d0 msgr2=0x561bececfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebf4d0 0x561bececfb20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06d0006010 tx=0x7f06d0009260 comp rx=0 tx=0).stop 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1182106473 shutdown_connections 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561beced0190 0x561bececd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebfdf0 0x561bececcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x561becec6c60 0x561becec90a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1182106473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebf4d0 0x561bececfb20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06c3fff640 0 client.27035 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:11:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1182106473 >> 172.21.15.78:0/1182106473 conn(0x561becebd300 msgr2=0x561becec2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1182106473 shutdown_connections 2024-09-11T01:11:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1182106473 wait complete. 2024-09-11T01:11:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.431+0000 7f06ecfb4740 20 client.27035 trim_cache size 0 max 16384 2024-09-11T01:11:19.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06ecfb4740 1 Processor -- start 2024-09-11T01:11:19.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06ecfb4740 1 -- start start 2024-09-11T01:11:19.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06ecfb4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebf4d0 0x561becedc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06ecfb4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 0x561becebe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06ecfb4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561beced0190 0x561bececa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06ecfb4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06e41ac4e0 con 0x561becebfdf0 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06ecfb4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f06e41ae0e0 con 0x561becebf4d0 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06ecfb4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06e41acbb0 con 0x561beced0190 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06e897b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 0x561becebe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06e897b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 0x561becebe6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57160/0 (socket says 172.21.15.78:57160) 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06e897b640 1 -- 172.21.15.78:0/3911694812 learned_addr learned my addr 172.21.15.78:0/3911694812 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06e3fff640 1 --2- 172.21.15.78:0/3911694812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebf4d0 0x561becedc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06e3fff640 1 --2- 172.21.15.78:0/3911694812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561beced0190 0x561bececa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06e897b640 1 -- 172.21.15.78:0/3911694812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561beced0190 msgr2=0x561bececa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06e897b640 1 --2- 172.21.15.78:0/3911694812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561beced0190 0x561bececa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06e897b640 1 -- 172.21.15.78:0/3911694812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebf4d0 msgr2=0x561becedc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:19.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06e897b640 1 --2- 172.21.15.78:0/3911694812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebf4d0 0x561becedc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06e897b640 1 -- 172.21.15.78:0/3911694812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561becebdd80 con 0x561becebfdf0 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.445+0000 7f06e3fff640 1 --2- 172.21.15.78:0/3911694812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561beced0190 0x561bececa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.446+0000 7f06e897b640 1 --2- 172.21.15.78:0/3911694812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 0x561becebe6f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009230 tx=0x7f06d80045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8662704d3b79bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.446+0000 7f06ce7fc640 1 -- 172.21.15.78:0/3911694812 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8005040 con 0x561becebfdf0 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.446+0000 7f06ce7fc640 1 -- 172.21.15.78:0/3911694812 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d8024e30 con 0x561becebfdf0 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.446+0000 7f06ce7fc640 1 -- 172.21.15.78:0/3911694812 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8004a60 con 0x561becebfdf0 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.446+0000 7f06ecfb4740 1 -- 172.21.15.78:0/3911694812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 msgr2=0x561becebe6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.446+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/3911694812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 0x561becebe6f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009230 tx=0x7f06d80045a0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 -- 172.21.15.78:0/3911694812 shutdown_connections 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/3911694812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561beced0190 0x561bececa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/3911694812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561becebfdf0 0x561becebe6f0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/3911694812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebf4d0 0x561becedc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 -- 172.21.15.78:0/3911694812 >> 172.21.15.78:0/3911694812 conn(0x561becebd300 msgr2=0x561becebdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:19.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 -- 172.21.15.78:0/3911694812 shutdown_connections 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 -- 172.21.15.78:0/3911694812 wait complete. 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 -- start start 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 Processor -- start 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 0x561becee1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebfdf0 0x561becee2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561beced0190 0x561becf47a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06e41ad950 con 0x561beced0190 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f06e41ad280 con 0x561becebfdf0 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06e41aa4e0 con 0x561becebf4d0 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06ecfb4740 10 client.0 _ll_register_callbacks cb 0x7f06d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.447+0000 7f06e3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561beced0190 0x561becf47a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06e897b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebfdf0 0x561becee2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 0x561becee1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06e897b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebfdf0 0x561becee2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36148/0 (socket says 172.21.15.78:36148) 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06e897b640 1 -- 172.21.15.78:0/1282576215 learned_addr learned my addr 172.21.15.78:0/1282576215 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06e3fff640 1 -- 172.21.15.78:0/1282576215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebfdf0 msgr2=0x561becee2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06e3fff640 1 --2- 172.21.15.78:0/1282576215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebfdf0 0x561becee2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06e3fff640 1 -- 172.21.15.78:0/1282576215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561beced0190 msgr2=0x561becf47a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06e3fff640 1 --2- 172.21.15.78:0/1282576215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561beced0190 0x561becf47a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06e3fff640 1 -- 172.21.15.78:0/1282576215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561becf47fc0 con 0x561becebf4d0 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06e897b640 1 --2- 172.21.15.78:0/1282576215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebfdf0 0x561becee2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06e3fff640 1 --2- 172.21.15.78:0/1282576215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 0x561becee1c10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06d00315e0 tx=0x7f06d0032d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d036bd5832cc1cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06c3fff640 1 -- 172.21.15.78:0/1282576215 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0060060 con 0x561becebf4d0 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1282576215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d8024e30 con 0x561becebf4d0 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06c3fff640 1 -- 172.21.15.78:0/1282576215 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f06d0005030 con 0x561becebf4d0 2024-09-11T01:11:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06c3fff640 1 -- 172.21.15.78:0/1282576215 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f06d0035070 con 0x561becebf4d0 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06ecfb4740 10 client.26974 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.448+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1282576215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561becf49170 con 0x561becebf4d0 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.449+0000 7f06c3fff640 1 -- 172.21.15.78:0/1282576215 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d8024e30 con 0x561becebf4d0 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.449+0000 7f06c3fff640 1 -- 172.21.15.78:0/1282576215 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f06d0004bd0 con 0x561becebf4d0 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.449+0000 7f06ecfb4740 10 client.26974 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.449+0000 7f06ecfb4740 10 client.26974 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.449+0000 7f06ecfb4740 20 client.26974 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.449+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1282576215 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x561becf498f0 0x561becf4bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.449+0000 7f06ecfb4740 4 client.26974 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.449+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1282576215 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bece338a0 con 0x561becf498f0 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.449+0000 7f06e897b640 1 --2- 172.21.15.78:0/1282576215 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x561becf498f0 0x561becf4bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.449+0000 7f06e897b640 1 --2- 172.21.15.78:0/1282576215 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x561becf498f0 0x561becf4bce0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a173d42ef126160 server_cookie=ffbd37a8c0316a50 in_seq=0 out_seq=0 2024-09-11T01:11:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.449+0000 7f06c3fff640 10 client.26974 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06c3fff640 1 -- 172.21.15.78:0/1282576215 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bece338a0 con 0x561becf498f0 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06c3fff640 10 client.26974 handle_command_reply: tid=0 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06ecfb4740 1 client.26974 shutdown 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1282576215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 msgr2=0x561becee1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1282576215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 0x561becee1c10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06d00315e0 tx=0x7f06d0032d70 comp rx=0 tx=0).stop 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1282576215 shutdown_connections 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1282576215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561beced0190 0x561becf47a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1282576215 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x561becf498f0 0x561becf4bce0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1282576215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561becebfdf0 0x561becee2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06ecfb4740 1 --2- 172.21.15.78:0/1282576215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561becebf4d0 0x561becee1c10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1282576215 >> 172.21.15.78:0/1282576215 conn(0x561becebd300 msgr2=0x561becece5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06c3fff640 0 client.26974 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1282576215 shutdown_connections 2024-09-11T01:11:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06ecfb4740 1 -- 172.21.15.78:0/1282576215 wait complete. 2024-09-11T01:11:19.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.450+0000 7f06ecfb4740 20 client.26974 trim_cache size 0 max 16384 2024-09-11T01:11:19.463 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:11:19.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.451+0000 7f06eabdf640 1 -- 172.21.15.78:0/1356922443 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f06d40b54d0 msgr2=0x7f06d40b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:19.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.451+0000 7f06eabdf640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f06d40b54d0 0x7f06d40b7990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0a8f10 tx=0x7f06dc0a8040 comp rx=0 tx=0).stop 2024-09-11T01:11:19.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.451+0000 7f06eabdf640 1 -- 172.21.15.78:0/1356922443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e410d100 msgr2=0x7f06e419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:19.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.451+0000 7f06eabdf640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e410d100 0x7f06e419eb80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00eb10 tx=0x7f06dc00c2d0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.451+0000 7f06eabdf640 1 -- 172.21.15.78:0/1356922443 shutdown_connections 2024-09-11T01:11:19.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.451+0000 7f06eabdf640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f06d40b54d0 0x7f06d40b7990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.451+0000 7f06eabdf640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e410d100 0x7f06e419eb80 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.451+0000 7f06eabdf640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06e41045c0 0x7f06e41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.451+0000 7f06eabdf640 1 --2- 172.21.15.78:0/1356922443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06e4103ca0 0x7f06e41992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:19.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.451+0000 7f06eabdf640 1 -- 172.21.15.78:0/1356922443 >> 172.21.15.78:0/1356922443 conn(0x7f06e40fd700 msgr2=0x7f06e40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:19.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.451+0000 7f06eabdf640 1 -- 172.21.15.78:0/1356922443 shutdown_connections 2024-09-11T01:11:19.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:19.451+0000 7f06eabdf640 1 -- 172.21.15.78:0/1356922443 wait complete. 2024-09-11T01:11:19.465 INFO:tasks.workunit.client.0.smithi078.stdout: 1 880799 5.96 MB/sec execute 567 sec latency 41.613 ms 2024-09-11T01:11:19.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:19 smithi110 ceph-mon[34712]: pgmap v708: 97 pgs: 97 active+clean; 731 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 301 op/s 2024-09-11T01:11:19.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:19 smithi112 ceph-mon[35384]: pgmap v708: 97 pgs: 97 active+clean; 731 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 301 op/s 2024-09-11T01:11:19.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:19 smithi078 ceph-mon[31315]: pgmap v708: 97 pgs: 97 active+clean; 731 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 301 op/s 2024-09-11T01:11:20.167 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:11:20.430 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:11:20.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 882180 5.96 MB/sec execute 568 sec latency 16.050 ms 2024-09-11T01:11:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.932+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2481386849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ef0103ca0 msgr2=0x7f0ef0104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.932+0000 7f0ef71c5640 1 --2- 172.21.15.78:0/2481386849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ef0103ca0 0x7f0ef0104080 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009900 tx=0x7f0ee402a410 comp rx=0 tx=0).stop 2024-09-11T01:11:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.932+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2481386849 shutdown_connections 2024-09-11T01:11:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.932+0000 7f0ef71c5640 1 --2- 172.21.15.78:0/2481386849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef010d100 0x7f0ef010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.932+0000 7f0ef71c5640 1 --2- 172.21.15.78:0/2481386849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ef01045c0 0x7f0ef010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.932+0000 7f0ef71c5640 1 --2- 172.21.15.78:0/2481386849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ef0103ca0 0x7f0ef0104080 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.932+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2481386849 >> 172.21.15.78:0/2481386849 conn(0x7f0ef00fd700 msgr2=0x7f0ef00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.932+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2481386849 shutdown_connections 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.932+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2481386849 wait complete. 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.932+0000 7f0ef71c5640 1 Processor -- start 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef71c5640 1 -- start start 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef71c5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ef0103ca0 0x7f0ef0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef71c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef01045c0 0x7f0ef01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef71c5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ef010d100 0x7f0ef019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef71c5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0112d70 con 0x7f0ef0103ca0 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef71c5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0112bf0 con 0x7f0ef010d100 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef71c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0112ef0 con 0x7f0ef01045c0 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef4f61640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ef0103ca0 0x7f0ef0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef4f61640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ef0103ca0 0x7f0ef0199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57188/0 (socket says 172.21.15.78:57188) 2024-09-11T01:11:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef4f61640 1 -- 172.21.15.78:0/2186036617 learned_addr learned my addr 172.21.15.78:0/2186036617 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0eeffff640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef01045c0 0x7f0ef01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef5762640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ef010d100 0x7f0ef019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef4f61640 1 -- 172.21.15.78:0/2186036617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef01045c0 msgr2=0x7f0ef01997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef4f61640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef01045c0 0x7f0ef01997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef4f61640 1 -- 172.21.15.78:0/2186036617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ef010d100 msgr2=0x7f0ef019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef4f61640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ef010d100 0x7f0ef019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef4f61640 1 -- 172.21.15.78:0/2186036617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef019f310 con 0x7f0ef0103ca0 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0ef4f61640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ef0103ca0 0x7f0ef0199260 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4005940 tx=0x7f0ee40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.933+0000 7f0eedffb640 1 -- 172.21.15.78:0/2186036617 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee403f030 con 0x7f0ef0103ca0 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.934+0000 7f0eedffb640 1 -- 172.21.15.78:0/2186036617 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee40044c0 con 0x7f0ef0103ca0 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.934+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2186036617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef019f5a0 con 0x7f0ef0103ca0 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.934+0000 7f0eedffb640 1 -- 172.21.15.78:0/2186036617 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4035070 con 0x7f0ef0103ca0 2024-09-11T01:11:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.934+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2186036617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef01a5cc0 con 0x7f0ef0103ca0 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.936+0000 7f0eedffb640 1 -- 172.21.15.78:0/2186036617 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0ee403ba20 con 0x7f0ef0103ca0 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.936+0000 7f0ef959a740 1 Processor -- start 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.936+0000 7f0ef959a740 1 -- start start 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef959a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 0x55ea0e07b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef959a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07ba10 0x55ea0e08b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef959a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 0x55ea0e08e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef959a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ea0dff1ea0 con 0x55ea0e08bdb0 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef959a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ea0dfe12d0 con 0x55ea0e07b0f0 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef959a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ea0dfe50e0 con 0x55ea0e07ba10 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef5762640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 0x55ea0e07b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef4f61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07ba10 0x55ea0e08b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef5762640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 0x55ea0e07b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36168/0 (socket says 172.21.15.78:36168) 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef5762640 1 -- 172.21.15.78:0/1258686588 learned_addr learned my addr 172.21.15.78:0/1258686588 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0eeffff640 1 --2- 172.21.15.78:0/1258686588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 0x55ea0e08e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef5762640 1 -- 172.21.15.78:0/1258686588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07ba10 msgr2=0x55ea0e08b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef5762640 1 --2- 172.21.15.78:0/1258686588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07ba10 0x55ea0e08b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef5762640 1 -- 172.21.15.78:0/1258686588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 msgr2=0x55ea0e08e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef5762640 1 --2- 172.21.15.78:0/1258686588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 0x55ea0e08e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef5762640 1 -- 172.21.15.78:0/1258686588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0dd3f650 con 0x55ea0e07b0f0 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef5762640 1 --2- 172.21.15.78:0/1258686588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 0x55ea0e07b4d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009fd0 tx=0x7f0ee80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5e66f0bff72053e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0eedffb640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0ee00b5450 0x7f0ee00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0edaffd640 1 -- 172.21.15.78:0/1258686588 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8004d20 con 0x55ea0e07b0f0 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.937+0000 7f0ef4f61640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0ee00b5450 0x7f0ee00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0edaffd640 1 -- 172.21.15.78:0/1258686588 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee8004050 con 0x55ea0e07b0f0 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0edaffd640 1 -- 172.21.15.78:0/1258686588 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8005280 con 0x55ea0e07b0f0 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0eedffb640 1 -- 172.21.15.78:0/2186036617 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f0ee4106570 con 0x7f0ef0103ca0 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0ef959a740 1 -- 172.21.15.78:0/1258686588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 msgr2=0x55ea0e07b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0ef959a740 1 --2- 172.21.15.78:0/1258686588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 0x55ea0e07b4d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009fd0 tx=0x7f0ee80118d0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0ef4f61640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0ee00b5450 0x7f0ee00b7910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4103db0 tx=0x7f0ee410b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0ef959a740 1 -- 172.21.15.78:0/1258686588 shutdown_connections 2024-09-11T01:11:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0ef959a740 1 --2- 172.21.15.78:0/1258686588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 0x55ea0e08e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0ef959a740 1 --2- 172.21.15.78:0/1258686588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07ba10 0x55ea0e08b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0ef959a740 1 --2- 172.21.15.78:0/1258686588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 0x55ea0e07b4d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0ef959a740 1 -- 172.21.15.78:0/1258686588 >> 172.21.15.78:0/1258686588 conn(0x55ea0e078f20 msgr2=0x55ea0e07a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0ef959a740 1 -- 172.21.15.78:0/1258686588 shutdown_connections 2024-09-11T01:11:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.938+0000 7f0ef959a740 1 -- 172.21.15.78:0/1258686588 wait complete. 2024-09-11T01:11:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef959a740 1 -- start start 2024-09-11T01:11:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef959a740 1 Processor -- start 2024-09-11T01:11:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef959a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 0x55ea0e08b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef959a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07ba10 0x55ea0e088a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef959a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 0x55ea0e088ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef959a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ea0de42700 con 0x55ea0e08bdb0 2024-09-11T01:11:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef959a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ea0df7c160 con 0x55ea0e07b0f0 2024-09-11T01:11:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef959a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ea0e08ea60 con 0x55ea0e07ba10 2024-09-11T01:11:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef959a740 10 client.0 _ll_register_callbacks cb 0x55ea0df959e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef5762640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 0x55ea0e08b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef5762640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 0x55ea0e08b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36178/0 (socket says 172.21.15.78:36178) 2024-09-11T01:11:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef5762640 1 -- 172.21.15.78:0/3560973527 learned_addr learned my addr 172.21.15.78:0/3560973527 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0eeffff640 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07ba10 0x55ea0e088a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.939+0000 7f0ef5762640 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 0x55ea0e088ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0eeffff640 1 -- 172.21.15.78:0/3560973527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 msgr2=0x55ea0e08b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0eeffff640 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 0x55ea0e08b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0eeffff640 1 -- 172.21.15.78:0/3560973527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 msgr2=0x55ea0e088ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0eeffff640 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 0x55ea0e088ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0eeffff640 1 -- 172.21.15.78:0/3560973527 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0dfa9c90 con 0x55ea0e07ba10 2024-09-11T01:11:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0eeffff640 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07ba10 0x55ea0e088a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ef006b950 tx=0x7f0edc012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d47f1f0479c6de5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0ecffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:11:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0ecffff640 1 -- 172.21.15.78:0/3560973527 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc00f920 con 0x55ea0e07ba10 2024-09-11T01:11:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0ef959a740 1 -- 172.21.15.78:0/3560973527 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee8004050 con 0x55ea0e07ba10 2024-09-11T01:11:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0ecffff640 1 -- 172.21.15.78:0/3560973527 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f0edc00fac0 con 0x55ea0e07ba10 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0ecffff640 1 -- 172.21.15.78:0/3560973527 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0edc00fcc0 con 0x55ea0e07ba10 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0ef959a740 10 client.26986 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.940+0000 7f0ef959a740 1 -- 172.21.15.78:0/3560973527 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55ea0e09a3f0 con 0x55ea0e07ba10 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.941+0000 7f0ecffff640 1 -- 172.21.15.78:0/3560973527 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee8004050 con 0x55ea0e07ba10 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.941+0000 7f0ecffff640 1 -- 172.21.15.78:0/3560973527 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f0edc01c420 con 0x55ea0e07ba10 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.941+0000 7f0ef959a740 10 client.26986 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.941+0000 7f0ef959a740 10 client.26986 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.941+0000 7f0ef959a740 20 client.26986 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.941+0000 7f0ef959a740 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ea0e09b8c0 0x55ea0e09dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.941+0000 7f0ef959a740 4 client.26986 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.941+0000 7f0ef959a740 1 -- 172.21.15.78:0/3560973527 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea0dff3df0 con 0x55ea0e09b8c0 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.941+0000 7f0ef5762640 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ea0e09b8c0 0x55ea0e09dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.941+0000 7f0ecffff640 10 client.26986 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.941+0000 7f0ef5762640 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ea0e09b8c0 0x55ea0e09dcb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73e5c47613f4eb3c server_cookie=b8ffcc8c5f9fd111 in_seq=0 out_seq=0 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ecffff640 1 -- 172.21.15.78:0/3560973527 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ea0dff3df0 con 0x55ea0e09b8c0 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ecffff640 10 client.26986 handle_command_reply: tid=0 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ef959a740 1 client.26986 shutdown 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ef959a740 1 -- 172.21.15.78:0/3560973527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07ba10 msgr2=0x55ea0e088a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ef959a740 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07ba10 0x55ea0e088a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ef006b950 tx=0x7f0edc012b60 comp rx=0 tx=0).stop 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ef959a740 1 -- 172.21.15.78:0/3560973527 shutdown_connections 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ef959a740 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ea0e09b8c0 0x55ea0e09dcb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ef959a740 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 0x55ea0e088ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ef959a740 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07ba10 0x55ea0e088a70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ecffff640 0 client.26986 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ef959a740 1 --2- 172.21.15.78:0/3560973527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07b0f0 0x55ea0e08b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ef959a740 1 -- 172.21.15.78:0/3560973527 >> 172.21.15.78:0/3560973527 conn(0x55ea0e078f20 msgr2=0x55ea0e07ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ef959a740 1 -- 172.21.15.78:0/3560973527 shutdown_connections 2024-09-11T01:11:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ef959a740 1 -- 172.21.15.78:0/3560973527 wait complete. 2024-09-11T01:11:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.942+0000 7f0ef959a740 20 client.26986 trim_cache size 0 max 16384 2024-09-11T01:11:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0ef959a740 1 Processor -- start 2024-09-11T01:11:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0ef959a740 1 -- start start 2024-09-11T01:11:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0ef959a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07b0f0 0x55ea0e097cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0ef959a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07ba10 0x55ea0e078760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0ef959a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 0x55ea0e0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0ef959a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01ac380 con 0x55ea0e08bdb0 2024-09-11T01:11:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0ef959a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01adf80 con 0x55ea0e07ba10 2024-09-11T01:11:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0ef959a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01aca50 con 0x55ea0e07b0f0 2024-09-11T01:11:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0ef5762640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 0x55ea0e0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0eeffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07ba10 0x55ea0e078760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0ef5762640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07b0f0 0x55ea0e097cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0eeffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07ba10 0x55ea0e078760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36184/0 (socket says 172.21.15.78:36184) 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0eeffff640 1 -- 172.21.15.78:0/326366573 learned_addr learned my addr 172.21.15.78:0/326366573 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0eeffff640 1 -- 172.21.15.78:0/326366573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07b0f0 msgr2=0x55ea0e097cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0eeffff640 1 --2- 172.21.15.78:0/326366573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07b0f0 0x55ea0e097cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0eeffff640 1 -- 172.21.15.78:0/326366573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 msgr2=0x55ea0e0a3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0eeffff640 1 --2- 172.21.15.78:0/326366573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 0x55ea0e0a3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0eeffff640 1 -- 172.21.15.78:0/326366573 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0e0747e0 con 0x55ea0e07ba10 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.957+0000 7f0eeffff640 1 --2- 172.21.15.78:0/326366573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07ba10 0x55ea0e078760 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0edc005bc0 tx=0x7f0edc0057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66ed69a3bdb0e595 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.958+0000 7f0eda7fc640 1 -- 172.21.15.78:0/326366573 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc029070 con 0x55ea0e07ba10 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.958+0000 7f0eda7fc640 1 -- 172.21.15.78:0/326366573 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0edc005560 con 0x55ea0e07ba10 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.958+0000 7f0eda7fc640 1 -- 172.21.15.78:0/326366573 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0035f0 con 0x55ea0e07ba10 2024-09-11T01:11:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.958+0000 7f0ef959a740 1 -- 172.21.15.78:0/326366573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07ba10 msgr2=0x55ea0e078760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.958+0000 7f0ef959a740 1 --2- 172.21.15.78:0/326366573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07ba10 0x55ea0e078760 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0edc005bc0 tx=0x7f0edc0057d0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.958+0000 7f0ef959a740 1 -- 172.21.15.78:0/326366573 shutdown_connections 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.958+0000 7f0ef959a740 1 --2- 172.21.15.78:0/326366573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e08bdb0 0x55ea0e0a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.958+0000 7f0ef959a740 1 --2- 172.21.15.78:0/326366573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e07ba10 0x55ea0e078760 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.958+0000 7f0ef959a740 1 --2- 172.21.15.78:0/326366573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07b0f0 0x55ea0e097cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.958+0000 7f0ef959a740 1 -- 172.21.15.78:0/326366573 >> 172.21.15.78:0/326366573 conn(0x55ea0e078f20 msgr2=0x55ea0e08db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.958+0000 7f0ef959a740 1 -- 172.21.15.78:0/326366573 shutdown_connections 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.958+0000 7f0ef959a740 1 -- 172.21.15.78:0/326366573 wait complete. 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0ef959a740 1 -- start start 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0ef959a740 1 Processor -- start 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0ef959a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07b0f0 0x55ea0e094120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0ef959a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e07ba10 0x55ea0e0946a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0ef959a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e08bdb0 0x55ea0e090e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0ef959a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01ad7f0 con 0x55ea0e07ba10 2024-09-11T01:11:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0ef959a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01ad120 con 0x55ea0e08bdb0 2024-09-11T01:11:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0ef959a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01aa3a0 con 0x55ea0e07b0f0 2024-09-11T01:11:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0ef959a740 10 client.0 _ll_register_callbacks cb 0x7f0edc009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0eeffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e07ba10 0x55ea0e0946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0ef5762640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07b0f0 0x55ea0e094120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0eeffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e07ba10 0x55ea0e0946a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57212/0 (socket says 172.21.15.78:57212) 2024-09-11T01:11:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0eeffff640 1 -- 172.21.15.78:0/2697489815 learned_addr learned my addr 172.21.15.78:0/2697489815 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.959+0000 7f0ef5762640 1 --2- 172.21.15.78:0/2697489815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e08bdb0 0x55ea0e090e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0eeffff640 1 -- 172.21.15.78:0/2697489815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07b0f0 msgr2=0x55ea0e094120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0eeffff640 1 --2- 172.21.15.78:0/2697489815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07b0f0 0x55ea0e094120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0eeffff640 1 -- 172.21.15.78:0/2697489815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e08bdb0 msgr2=0x55ea0e090e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0eeffff640 1 --2- 172.21.15.78:0/2697489815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e08bdb0 0x55ea0e090e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0eeffff640 1 -- 172.21.15.78:0/2697489815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0e0916b0 con 0x55ea0e07ba10 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0eeffff640 1 --2- 172.21.15.78:0/2697489815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e07ba10 0x55ea0e0946a0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f0edc01ca50 tx=0x7f0edc01c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf57d9cc8442e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0ecffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0ecffff640 1 -- 172.21.15.78:0/2697489815 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc029040 con 0x55ea0e07ba10 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0ecffff640 1 -- 172.21.15.78:0/2697489815 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f0edc018030 con 0x55ea0e07ba10 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0ef959a740 1 -- 172.21.15.78:0/2697489815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0edc005560 con 0x55ea0e07ba10 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0ecffff640 1 -- 172.21.15.78:0/2697489815 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0edc01cad0 con 0x55ea0e07ba10 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0ef959a740 10 client.17865 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0ef959a740 1 -- 172.21.15.78:0/2697489815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55ea0e109410 con 0x55ea0e07ba10 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.960+0000 7f0ecffff640 1 -- 172.21.15.78:0/2697489815 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0edc005560 con 0x55ea0e07ba10 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ecffff640 1 -- 172.21.15.78:0/2697489815 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f0edc02e480 con 0x55ea0e07ba10 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ef959a740 10 client.17865 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ef959a740 10 client.17865 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ef959a740 20 client.17865 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ef959a740 1 --2- 172.21.15.78:0/2697489815 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ea0e1099f0 0x55ea0e10be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ef959a740 4 client.17865 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ef959a740 1 -- 172.21.15.78:0/2697489815 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea0dfef8a0 con 0x55ea0e1099f0 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ef5762640 1 --2- 172.21.15.78:0/2697489815 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ea0e1099f0 0x55ea0e10be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ef5762640 1 --2- 172.21.15.78:0/2697489815 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ea0e1099f0 0x55ea0e10be10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6f5cb700a2add6b server_cookie=39596cb2d679c2de in_seq=0 out_seq=0 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ecffff640 10 client.17865 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ecffff640 1 -- 172.21.15.78:0/2697489815 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea0dfef8a0 con 0x55ea0e1099f0 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ecffff640 10 client.17865 handle_command_reply: tid=0 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.961+0000 7f0ef959a740 1 client.17865 shutdown 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.962+0000 7f0ef959a740 1 -- 172.21.15.78:0/2697489815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e07ba10 msgr2=0x55ea0e0946a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.962+0000 7f0ef959a740 1 --2- 172.21.15.78:0/2697489815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e07ba10 0x55ea0e0946a0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f0edc01ca50 tx=0x7f0edc01c630 comp rx=0 tx=0).stop 2024-09-11T01:11:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.962+0000 7f0ef959a740 1 -- 172.21.15.78:0/2697489815 shutdown_connections 2024-09-11T01:11:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.962+0000 7f0ef959a740 1 --2- 172.21.15.78:0/2697489815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea0e08bdb0 0x55ea0e090e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.962+0000 7f0ef959a740 1 --2- 172.21.15.78:0/2697489815 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ea0e1099f0 0x55ea0e10be10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.962+0000 7f0ef959a740 1 --2- 172.21.15.78:0/2697489815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea0e07ba10 0x55ea0e0946a0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.962+0000 7f0ef959a740 1 --2- 172.21.15.78:0/2697489815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea0e07b0f0 0x55ea0e094120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.962+0000 7f0ef959a740 1 -- 172.21.15.78:0/2697489815 >> 172.21.15.78:0/2697489815 conn(0x55ea0e078f20 msgr2=0x55ea0e07e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.962+0000 7f0ecffff640 0 client.17865 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:11:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.962+0000 7f0ef959a740 1 -- 172.21.15.78:0/2697489815 shutdown_connections 2024-09-11T01:11:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.962+0000 7f0ef959a740 1 -- 172.21.15.78:0/2697489815 wait complete. 2024-09-11T01:11:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.962+0000 7f0ef959a740 20 client.17865 trim_cache size 0 max 16384 2024-09-11T01:11:20.972 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:11:20.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.963+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2186036617 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0ee00b5450 msgr2=0x7f0ee00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:20.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.963+0000 7f0ef71c5640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0ee00b5450 0x7f0ee00b7910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4103db0 tx=0x7f0ee410b040 comp rx=0 tx=0).stop 2024-09-11T01:11:20.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.963+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2186036617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ef0103ca0 msgr2=0x7f0ef0199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.963+0000 7f0ef71c5640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ef0103ca0 0x7f0ef0199260 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4005940 tx=0x7f0ee40040c0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.963+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2186036617 shutdown_connections 2024-09-11T01:11:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.963+0000 7f0ef71c5640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0ee00b5450 0x7f0ee00b7910 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.963+0000 7f0ef71c5640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ef010d100 0x7f0ef019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.963+0000 7f0ef71c5640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef01045c0 0x7f0ef01997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.963+0000 7f0ef71c5640 1 --2- 172.21.15.78:0/2186036617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ef0103ca0 0x7f0ef0199260 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.963+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2186036617 >> 172.21.15.78:0/2186036617 conn(0x7f0ef00fd700 msgr2=0x7f0ef00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.963+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2186036617 shutdown_connections 2024-09-11T01:11:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:20.963+0000 7f0ef71c5640 1 -- 172.21.15.78:0/2186036617 wait complete. 2024-09-11T01:11:21.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 884258 5.98 MB/sec execute 569 sec latency 37.540 ms 2024-09-11T01:11:21.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:21 smithi110 ceph-mon[34712]: pgmap v709: 97 pgs: 97 active+clean; 737 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 303 op/s 2024-09-11T01:11:21.698 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:21 smithi078 ceph-mon[31315]: pgmap v709: 97 pgs: 97 active+clean; 737 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 303 op/s 2024-09-11T01:11:21.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:21 smithi112 ceph-mon[35384]: pgmap v709: 97 pgs: 97 active+clean; 737 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 303 op/s 2024-09-11T01:11:21.726 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:11:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:11:22.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 885513 5.97 MB/sec execute 570 sec latency 39.984 ms 2024-09-11T01:11:22.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.628+0000 7f677b2a2640 1 -- 172.21.15.78:0/3853067160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6774108040 msgr2=0x7f677410e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:22.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.628+0000 7f677b2a2640 1 --2- 172.21.15.78:0/3853067160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6774108040 0x7f677410e840 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f676c009ca0 tx=0x7f676c02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.628+0000 7f677b2a2640 1 -- 172.21.15.78:0/3853067160 shutdown_connections 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.628+0000 7f677b2a2640 1 --2- 172.21.15.78:0/3853067160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6774108040 0x7f677410e840 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.628+0000 7f677b2a2640 1 --2- 172.21.15.78:0/3853067160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67741076c0 0x7f6774107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.628+0000 7f677b2a2640 1 --2- 172.21.15.78:0/3853067160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6774101cb0 0x7f6774102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.628+0000 7f677b2a2640 1 -- 172.21.15.78:0/3853067160 >> 172.21.15.78:0/3853067160 conn(0x7f67740fd700 msgr2=0x7f67740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.628+0000 7f677b2a2640 1 -- 172.21.15.78:0/3853067160 shutdown_connections 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.628+0000 7f677b2a2640 1 -- 172.21.15.78:0/3853067160 wait complete. 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677b2a2640 1 Processor -- start 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677b2a2640 1 -- start start 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677b2a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6774101cb0 0x7f67741992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677b2a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67741076c0 0x7f6774199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677b2a2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6774108040 0x7f677419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677b2a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6774112dd0 con 0x7f6774101cb0 2024-09-11T01:11:22.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677b2a2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6774112c50 con 0x7f6774108040 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677b2a2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6774112f50 con 0x7f67741076c0 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677903e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6774101cb0 0x7f67741992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677903e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6774101cb0 0x7f67741992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57232/0 (socket says 172.21.15.78:57232) 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677903e640 1 -- 172.21.15.78:0/1677649638 learned_addr learned my addr 172.21.15.78:0/1677649638 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677883d640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67741076c0 0x7f6774199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.629+0000 7f677983f640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6774108040 0x7f677419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.630+0000 7f677903e640 1 -- 172.21.15.78:0/1677649638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67741076c0 msgr2=0x7f6774199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.630+0000 7f677903e640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67741076c0 0x7f6774199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.630+0000 7f677903e640 1 -- 172.21.15.78:0/1677649638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6774108040 msgr2=0x7f677419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.630+0000 7f677903e640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6774108040 0x7f677419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.630+0000 7f677903e640 1 -- 172.21.15.78:0/1677649638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677419f390 con 0x7f6774101cb0 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.630+0000 7f677983f640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6774108040 0x7f677419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.630+0000 7f677903e640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6774101cb0 0x7f67741992e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f677000e9f0 tx=0x7f677000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:22.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.630+0000 7f676a7fc640 1 -- 172.21.15.78:0/1677649638 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677000cd80 con 0x7f6774101cb0 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.630+0000 7f677b2a2640 1 -- 172.21.15.78:0/1677649638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677419f680 con 0x7f6774101cb0 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.630+0000 7f676a7fc640 1 -- 172.21.15.78:0/1677649638 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67700108e0 con 0x7f6774101cb0 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.630+0000 7f676a7fc640 1 -- 172.21.15.78:0/1677649638 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770018980 con 0x7f6774101cb0 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.630+0000 7f677b2a2640 1 -- 172.21.15.78:0/1677649638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67741a5f10 con 0x7f6774101cb0 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.632+0000 7f676a7fc640 1 -- 172.21.15.78:0/1677649638 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6770010430 con 0x7f6774101cb0 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677d677740 1 Processor -- start 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677d677740 1 -- start start 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677d677740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3def8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677d677740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3dfb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677d677740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3dfe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677d677740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b5d3d65ea0 con 0x55b5d3def4d0 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677d677740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b5d3d552d0 con 0x55b5d3defdf0 2024-09-11T01:11:22.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677d677740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b5d3d590e0 con 0x55b5d3dfbd80 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677903e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3dfb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677983f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3def8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677883d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3dfe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677983f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3def8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57236/0 (socket says 172.21.15.78:57236) 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677983f640 1 -- 172.21.15.78:0/1621415931 learned_addr learned my addr 172.21.15.78:0/1621415931 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f676a7fc640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f67600b5450 0x7f67600b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677883d640 1 -- 172.21.15.78:0/1621415931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 msgr2=0x55b5d3dfb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.633+0000 7f677883d640 1 --2- 172.21.15.78:0/1621415931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3dfb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.634+0000 7f677883d640 1 -- 172.21.15.78:0/1621415931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 msgr2=0x55b5d3def8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.634+0000 7f677883d640 1 --2- 172.21.15.78:0/1621415931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3def8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.634+0000 7f677883d640 1 -- 172.21.15.78:0/1621415931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d3ab3650 con 0x55b5d3dfbd80 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.634+0000 7f677983f640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f67600b5450 0x7f67600b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.634+0000 7f676a7fc640 1 -- 172.21.15.78:0/1677649638 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f67700e6290 con 0x7f6774101cb0 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.634+0000 7f677883d640 1 --2- 172.21.15.78:0/1621415931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3dfe170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6764009bf0 tx=0x7f6764011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c49382724e6d096 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.634+0000 7f675f7fe640 1 -- 172.21.15.78:0/1621415931 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764004050 con 0x55b5d3dfbd80 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.634+0000 7f675f7fe640 1 -- 172.21.15.78:0/1621415931 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67640054d0 con 0x55b5d3dfbd80 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.634+0000 7f675f7fe640 1 -- 172.21.15.78:0/1621415931 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764004cc0 con 0x55b5d3dfbd80 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.635+0000 7f677d677740 1 -- 172.21.15.78:0/1621415931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 msgr2=0x55b5d3dfe170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.635+0000 7f677983f640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f67600b5450 0x7f67600b7910 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f676c035000 tx=0x7f676c02a8c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.635+0000 7f677d677740 1 --2- 172.21.15.78:0/1621415931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3dfe170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6764009bf0 tx=0x7f6764011810 comp rx=0 tx=0).stop 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.635+0000 7f677d677740 1 -- 172.21.15.78:0/1621415931 shutdown_connections 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.635+0000 7f677d677740 1 --2- 172.21.15.78:0/1621415931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3dfe170 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.635+0000 7f677d677740 1 --2- 172.21.15.78:0/1621415931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3dfb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.635+0000 7f677d677740 1 --2- 172.21.15.78:0/1621415931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3def8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.635+0000 7f677d677740 1 -- 172.21.15.78:0/1621415931 >> 172.21.15.78:0/1621415931 conn(0x55b5d3ded300 msgr2=0x55b5d3dee4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.635+0000 7f677d677740 1 -- 172.21.15.78:0/1621415931 shutdown_connections 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.635+0000 7f677d677740 1 -- 172.21.15.78:0/1621415931 wait complete. 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.635+0000 7f677d677740 1 -- start start 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.635+0000 7f677d677740 1 Processor -- start 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677d677740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3def4d0 0x55b5d3dfb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677d677740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3defdf0 0x55b5d3df8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677d677740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3df9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677d677740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b5d3bb6700 con 0x55b5d3defdf0 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677d677740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b5d3cf0160 con 0x55b5d3def4d0 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677d677740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b5d3dfea00 con 0x55b5d3dfbd80 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677d677740 10 client.0 _ll_register_callbacks cb 0x55b5d3d099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677883d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3defdf0 0x55b5d3df8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677903e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3def4d0 0x55b5d3dfb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677903e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3def4d0 0x55b5d3dfb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36228/0 (socket says 172.21.15.78:36228) 2024-09-11T01:11:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677903e640 1 -- 172.21.15.78:0/809473038 learned_addr learned my addr 172.21.15.78:0/809473038 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677983f640 1 --2- 172.21.15.78:0/809473038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3df9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677883d640 1 -- 172.21.15.78:0/809473038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 msgr2=0x55b5d3df9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677883d640 1 --2- 172.21.15.78:0/809473038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3df9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677883d640 1 -- 172.21.15.78:0/809473038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3def4d0 msgr2=0x55b5d3dfb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677883d640 1 --2- 172.21.15.78:0/809473038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3def4d0 0x55b5d3dfb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677883d640 1 -- 172.21.15.78:0/809473038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d3d1dc90 con 0x55b5d3defdf0 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f677883d640 1 --2- 172.21.15.78:0/809473038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3defdf0 0x55b5d3df8a70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6764000c00 tx=0x7f6764009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa27774ae1769868 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f675cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f675cff9640 1 -- 172.21.15.78:0/809473038 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764029070 con 0x55b5d3defdf0 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.636+0000 7f675cff9640 1 -- 172.21.15.78:0/809473038 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f6764003a00 con 0x55b5d3defdf0 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f677d677740 1 -- 172.21.15.78:0/809473038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67640054d0 con 0x55b5d3defdf0 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f675cff9640 1 -- 172.21.15.78:0/809473038 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6764035050 con 0x55b5d3defdf0 2024-09-11T01:11:22.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f677d677740 10 client.17877 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f677d677740 1 -- 172.21.15.78:0/809473038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b5d3e0a2f0 con 0x55b5d3defdf0 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f675cff9640 1 -- 172.21.15.78:0/809473038 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67640054d0 con 0x55b5d3defdf0 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f675cff9640 1 -- 172.21.15.78:0/809473038 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f6764004e70 con 0x55b5d3defdf0 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f677d677740 10 client.17877 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f677d677740 10 client.17877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f677d677740 20 client.17877 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f677d677740 1 --2- 172.21.15.78:0/809473038 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b5d3e0b640 0x55b5d3e0da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f677d677740 4 client.17877 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f677d677740 1 -- 172.21.15.78:0/809473038 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5d3d67df0 con 0x55b5d3e0b640 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.637+0000 7f677903e640 1 --2- 172.21.15.78:0/809473038 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b5d3e0b640 0x55b5d3e0da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.638+0000 7f677903e640 1 --2- 172.21.15.78:0/809473038 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b5d3e0b640 0x55b5d3e0da30 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b2799ea5d3743db server_cookie=cbbe14aca989a0d7 in_seq=0 out_seq=0 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.638+0000 7f675cff9640 10 client.17877 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.638+0000 7f675cff9640 1 -- 172.21.15.78:0/809473038 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b5d3d67df0 con 0x55b5d3e0b640 2024-09-11T01:11:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.638+0000 7f675cff9640 10 client.17877 handle_command_reply: tid=0 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.638+0000 7f677d677740 1 client.17877 shutdown 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.639+0000 7f677d677740 1 -- 172.21.15.78:0/809473038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3defdf0 msgr2=0x55b5d3df8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.639+0000 7f677d677740 1 --2- 172.21.15.78:0/809473038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3defdf0 0x55b5d3df8a70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6764000c00 tx=0x7f6764009fd0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.639+0000 7f677d677740 1 -- 172.21.15.78:0/809473038 shutdown_connections 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.639+0000 7f677d677740 1 --2- 172.21.15.78:0/809473038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3df9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.639+0000 7f677d677740 1 --2- 172.21.15.78:0/809473038 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b5d3e0b640 0x55b5d3e0da30 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.639+0000 7f677d677740 1 --2- 172.21.15.78:0/809473038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3defdf0 0x55b5d3df8a70 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.639+0000 7f677d677740 1 --2- 172.21.15.78:0/809473038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3def4d0 0x55b5d3dfb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.639+0000 7f675cff9640 0 client.17877 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.639+0000 7f677d677740 1 -- 172.21.15.78:0/809473038 >> 172.21.15.78:0/809473038 conn(0x55b5d3ded300 msgr2=0x55b5d3df2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.639+0000 7f677d677740 1 -- 172.21.15.78:0/809473038 shutdown_connections 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.639+0000 7f677d677740 1 -- 172.21.15.78:0/809473038 wait complete. 2024-09-11T01:11:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.639+0000 7f677d677740 20 client.17877 trim_cache size 0 max 16384 2024-09-11T01:11:22.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677d677740 1 Processor -- start 2024-09-11T01:11:22.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677d677740 1 -- start start 2024-09-11T01:11:22.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677d677740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3e07c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677d677740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3dfa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677d677740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3df5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677d677740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67741ae660 con 0x55b5d3def4d0 2024-09-11T01:11:22.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677d677740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b0260 con 0x55b5d3defdf0 2024-09-11T01:11:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677d677740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67741aed30 con 0x55b5d3dfbd80 2024-09-11T01:11:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677903e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3e07c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677903e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3e07c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57258/0 (socket says 172.21.15.78:57258) 2024-09-11T01:11:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677903e640 1 -- 172.21.15.78:0/4049583952 learned_addr learned my addr 172.21.15.78:0/4049583952 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677883d640 1 --2- 172.21.15.78:0/4049583952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3dfa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.656+0000 7f677983f640 1 --2- 172.21.15.78:0/4049583952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3df5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.657+0000 7f677883d640 1 -- 172.21.15.78:0/4049583952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 msgr2=0x55b5d3df5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.657+0000 7f677883d640 1 --2- 172.21.15.78:0/4049583952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3df5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.657+0000 7f677883d640 1 -- 172.21.15.78:0/4049583952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 msgr2=0x55b5d3e07c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.657+0000 7f677883d640 1 --2- 172.21.15.78:0/4049583952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3e07c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.657+0000 7f677883d640 1 -- 172.21.15.78:0/4049583952 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d3defc20 con 0x55b5d3defdf0 2024-09-11T01:11:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.657+0000 7f677903e640 1 --2- 172.21.15.78:0/4049583952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3e07c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:22.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.657+0000 7f677983f640 1 --2- 172.21.15.78:0/4049583952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3df5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:22.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.657+0000 7f677883d640 1 --2- 172.21.15.78:0/4049583952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3dfa7b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6764005270 tx=0x7f67640096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f8a3a7f59e4895c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:22.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.657+0000 7f675effd640 1 -- 172.21.15.78:0/4049583952 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764032070 con 0x55b5d3defdf0 2024-09-11T01:11:22.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.657+0000 7f675effd640 1 -- 172.21.15.78:0/4049583952 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6764009820 con 0x55b5d3defdf0 2024-09-11T01:11:22.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.657+0000 7f675effd640 1 -- 172.21.15.78:0/4049583952 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764029070 con 0x55b5d3defdf0 2024-09-11T01:11:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.658+0000 7f677d677740 1 -- 172.21.15.78:0/4049583952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 msgr2=0x55b5d3dfa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.658+0000 7f677d677740 1 --2- 172.21.15.78:0/4049583952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3dfa7b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6764005270 tx=0x7f67640096c0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.658+0000 7f677d677740 1 -- 172.21.15.78:0/4049583952 shutdown_connections 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.658+0000 7f677d677740 1 --2- 172.21.15.78:0/4049583952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3df5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.658+0000 7f677d677740 1 --2- 172.21.15.78:0/4049583952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3dfa7b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.658+0000 7f677d677740 1 --2- 172.21.15.78:0/4049583952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3e07c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.658+0000 7f677d677740 1 -- 172.21.15.78:0/4049583952 >> 172.21.15.78:0/4049583952 conn(0x55b5d3ded300 msgr2=0x55b5d3dfa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.658+0000 7f677d677740 1 -- 172.21.15.78:0/4049583952 shutdown_connections 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.658+0000 7f677d677740 1 -- 172.21.15.78:0/4049583952 wait complete. 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.658+0000 7f677d677740 1 -- start start 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.658+0000 7f677d677740 1 Processor -- start 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677d677740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3e00790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677d677740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3e00d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677d677740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3e759f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677d677740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67741afad0 con 0x55b5d3def4d0 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677d677740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f67741af400 con 0x55b5d3defdf0 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677d677740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67741ac660 con 0x55b5d3dfbd80 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677d677740 10 client.0 _ll_register_callbacks cb 0x7f6764005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677903e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3e00790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677883d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3e00d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677903e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3e00790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57270/0 (socket says 172.21.15.78:57270) 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677903e640 1 -- 172.21.15.78:0/1684431810 learned_addr learned my addr 172.21.15.78:0/1684431810 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677983f640 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3e759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677903e640 1 -- 172.21.15.78:0/1684431810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 msgr2=0x55b5d3e759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677903e640 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3e759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677903e640 1 -- 172.21.15.78:0/1684431810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 msgr2=0x55b5d3e00d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677903e640 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3e00d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677903e640 1 -- 172.21.15.78:0/1684431810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d3e76100 con 0x55b5d3def4d0 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677883d640 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3e00d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f677903e640 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3e00790 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f67700a9a70 tx=0x7f6770000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ea2fb02c1a793f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.659+0000 7f675cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f675cff9640 1 -- 172.21.15.78:0/1684431810 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67700e5040 con 0x55b5d3def4d0 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f675cff9640 1 -- 172.21.15.78:0/1684431810 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f67700a8a70 con 0x55b5d3def4d0 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f677d677740 1 -- 172.21.15.78:0/1684431810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5d3d638a0 con 0x55b5d3def4d0 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f675cff9640 1 -- 172.21.15.78:0/1684431810 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f67700abf70 con 0x55b5d3def4d0 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f677d677740 10 client.17883 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f677d677740 1 -- 172.21.15.78:0/1684431810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b5d3e7d5f0 con 0x55b5d3def4d0 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f675cff9640 1 -- 172.21.15.78:0/1684431810 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b5d3d638a0 con 0x55b5d3def4d0 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f675cff9640 1 -- 172.21.15.78:0/1684431810 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f6770018b20 con 0x55b5d3def4d0 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f677d677740 10 client.17883 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f677d677740 10 client.17883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f677d677740 20 client.17883 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f677d677740 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b5d3e7dde0 0x55b5d3e80200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f677d677740 4 client.17883 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f677d677740 1 -- 172.21.15.78:0/1684431810 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5d3d60cd0 con 0x55b5d3e7dde0 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.660+0000 7f677883d640 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b5d3e7dde0 0x55b5d3e80200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f675cff9640 10 client.17883 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677883d640 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b5d3e7dde0 0x55b5d3e80200 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb6a12bdf45e5f7 server_cookie=9edc38d91cc229de in_seq=0 out_seq=0 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f675cff9640 1 -- 172.21.15.78:0/1684431810 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5d3d60cd0 con 0x55b5d3e7dde0 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f675cff9640 10 client.17883 handle_command_reply: tid=0 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677d677740 1 client.17883 shutdown 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677d677740 1 -- 172.21.15.78:0/1684431810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 msgr2=0x55b5d3e00790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677d677740 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3e00790 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f67700a9a70 tx=0x7f6770000bc0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677d677740 1 -- 172.21.15.78:0/1684431810 shutdown_connections 2024-09-11T01:11:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677d677740 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5d3dfbd80 0x55b5d3e759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677d677740 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5d3defdf0 0x55b5d3e00d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677d677740 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b5d3e7dde0 0x55b5d3e80200 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677d677740 1 --2- 172.21.15.78:0/1684431810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5d3def4d0 0x55b5d3e00790 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f675cff9640 0 client.17883 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677d677740 1 -- 172.21.15.78:0/1684431810 >> 172.21.15.78:0/1684431810 conn(0x55b5d3ded300 msgr2=0x55b5d3dfa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677d677740 1 -- 172.21.15.78:0/1684431810 shutdown_connections 2024-09-11T01:11:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677d677740 1 -- 172.21.15.78:0/1684431810 wait complete. 2024-09-11T01:11:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.661+0000 7f677d677740 20 client.17883 trim_cache size 0 max 16384 2024-09-11T01:11:22.670 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:11:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.662+0000 7f677b2a2640 1 -- 172.21.15.78:0/1677649638 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f67600b5450 msgr2=0x7f67600b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.662+0000 7f677b2a2640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f67600b5450 0x7f67600b7910 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f676c035000 tx=0x7f676c02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.662+0000 7f677b2a2640 1 -- 172.21.15.78:0/1677649638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6774101cb0 msgr2=0x7f67741992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.662+0000 7f677b2a2640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6774101cb0 0x7f67741992e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f677000e9f0 tx=0x7f677000eeb0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.662+0000 7f677b2a2640 1 -- 172.21.15.78:0/1677649638 shutdown_connections 2024-09-11T01:11:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.662+0000 7f677b2a2640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f67600b5450 0x7f67600b7910 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.662+0000 7f677b2a2640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6774108040 0x7f677419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.662+0000 7f677b2a2640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67741076c0 0x7f6774199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.662+0000 7f677b2a2640 1 --2- 172.21.15.78:0/1677649638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6774101cb0 0x7f67741992e0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:22.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.662+0000 7f677b2a2640 1 -- 172.21.15.78:0/1677649638 >> 172.21.15.78:0/1677649638 conn(0x7f67740fd700 msgr2=0x7f67740fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:22.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.662+0000 7f677b2a2640 1 -- 172.21.15.78:0/1677649638 shutdown_connections 2024-09-11T01:11:22.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:22.662+0000 7f677b2a2640 1 -- 172.21.15.78:0/1677649638 wait complete. 2024-09-11T01:11:23.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 887658 5.99 MB/sec execute 571 sec latency 18.625 ms 2024-09-11T01:11:23.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:23 smithi110 ceph-mon[34712]: pgmap v710: 97 pgs: 97 active+clean; 741 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 280 op/s 2024-09-11T01:11:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:23 smithi112 ceph-mon[35384]: pgmap v710: 97 pgs: 97 active+clean; 741 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 280 op/s 2024-09-11T01:11:23.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:23 smithi078 ceph-mon[31315]: pgmap v710: 97 pgs: 97 active+clean; 741 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 280 op/s 2024-09-11T01:11:24.399 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:11:24.399 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:11:24.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 888750 5.98 MB/sec execute 572 sec latency 25.170 ms 2024-09-11T01:11:24.662 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:11:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.188+0000 7f167f5b5640 1 -- 172.21.15.78:0/2370789909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1678103b50 msgr2=0x7f1678103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.188+0000 7f167f5b5640 1 --2- 172.21.15.78:0/2370789909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1678103b50 0x7f1678103f30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f1674009900 tx=0x7f167402a410 comp rx=0 tx=0).stop 2024-09-11T01:11:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.188+0000 7f167f5b5640 1 -- 172.21.15.78:0/2370789909 shutdown_connections 2024-09-11T01:11:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.188+0000 7f167f5b5640 1 --2- 172.21.15.78:0/2370789909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f167810d100 0x7f167810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.188+0000 7f167f5b5640 1 --2- 172.21.15.78:0/2370789909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1678104550 0x7f167810cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.188+0000 7f167f5b5640 1 --2- 172.21.15.78:0/2370789909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1678103b50 0x7f1678103f30 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.188+0000 7f167f5b5640 1 -- 172.21.15.78:0/2370789909 >> 172.21.15.78:0/2370789909 conn(0x7f16780fd700 msgr2=0x7f16780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.188+0000 7f167f5b5640 1 -- 172.21.15.78:0/2370789909 shutdown_connections 2024-09-11T01:11:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.188+0000 7f167f5b5640 1 -- 172.21.15.78:0/2370789909 wait complete. 2024-09-11T01:11:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167f5b5640 1 Processor -- start 2024-09-11T01:11:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167f5b5640 1 -- start start 2024-09-11T01:11:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167f5b5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1678103b50 0x7f1678199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167f5b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1678104550 0x7f1678199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167f5b5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f167810d100 0x7f167819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167f5b5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1678112d60 con 0x7f1678104550 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167f5b5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1678112be0 con 0x7f167810d100 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167f5b5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1678112ee0 con 0x7f1678103b50 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167cb50640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1678104550 0x7f1678199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167cb50640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1678104550 0x7f1678199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40520/0 (socket says 172.21.15.78:40520) 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167cb50640 1 -- 172.21.15.78:0/3249242773 learned_addr learned my addr 172.21.15.78:0/3249242773 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167db52640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f167810d100 0x7f167819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167d351640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1678103b50 0x7f1678199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167cb50640 1 -- 172.21.15.78:0/3249242773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1678103b50 msgr2=0x7f1678199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167cb50640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1678103b50 0x7f1678199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167cb50640 1 -- 172.21.15.78:0/3249242773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f167810d100 msgr2=0x7f167819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167cb50640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f167810d100 0x7f167819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.189+0000 7f167cb50640 1 -- 172.21.15.78:0/3249242773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167819f3c0 con 0x7f1678104550 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.190+0000 7f167cb50640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1678104550 0x7f1678199850 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f166800e9f0 tx=0x7f166800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.190+0000 7f166e7fc640 1 -- 172.21.15.78:0/3249242773 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166800cd80 con 0x7f1678104550 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.190+0000 7f166e7fc640 1 -- 172.21.15.78:0/3249242773 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16680054d0 con 0x7f1678104550 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.190+0000 7f167f5b5640 1 -- 172.21.15.78:0/3249242773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167819f6b0 con 0x7f1678104550 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.190+0000 7f166e7fc640 1 -- 172.21.15.78:0/3249242773 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668010690 con 0x7f1678104550 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.190+0000 7f167f5b5640 1 -- 172.21.15.78:0/3249242773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16781a5d70 con 0x7f1678104550 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.192+0000 7f166e7fc640 1 -- 172.21.15.78:0/3249242773 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1668004e90 con 0x7f1678104550 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.192+0000 7f168198a740 1 Processor -- start 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f168198a740 1 -- start start 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f168198a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 0x55f12abe32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f168198a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe3800 0x55f12abef250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f168198a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abef790 0x55f12abf1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f168198a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f12ab59e60 con 0x55f12abe2ee0 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f168198a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f12ab49290 con 0x55f12abe3800 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f168198a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f12ab4d0a0 con 0x55f12abef790 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167db52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 0x55f12abe32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167db52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 0x55f12abe32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40536/0 (socket says 172.21.15.78:40536) 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167db52640 1 -- 172.21.15.78:0/2851472428 learned_addr learned my addr 172.21.15.78:0/2851472428 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167d351640 1 --2- 172.21.15.78:0/2851472428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe3800 0x55f12abef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167cb50640 1 --2- 172.21.15.78:0/2851472428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abef790 0x55f12abf1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f166e7fc640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16640b5450 0x7f16640b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167d351640 1 -- 172.21.15.78:0/2851472428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abef790 msgr2=0x55f12abf1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167d351640 1 --2- 172.21.15.78:0/2851472428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abef790 0x55f12abf1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167d351640 1 -- 172.21.15.78:0/2851472428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 msgr2=0x55f12abe32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167d351640 1 --2- 172.21.15.78:0/2851472428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 0x55f12abe32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167d351640 1 -- 172.21.15.78:0/2851472428 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f12a8a7930 con 0x55f12abe3800 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167db52640 1 --2- 172.21.15.78:0/2851472428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 0x55f12abe32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167db52640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16640b5450 0x7f16640b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f166e7fc640 1 -- 172.21.15.78:0/3249242773 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f16680e6410 con 0x7f1678104550 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.193+0000 7f167d351640 1 --2- 172.21.15.78:0/2851472428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe3800 0x55f12abef250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1674009230 tx=0x7f1674032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=888a114f1ec6c56f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.194+0000 7f16637fe640 1 -- 172.21.15.78:0/2851472428 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1674040070 con 0x55f12abe3800 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.194+0000 7f16637fe640 1 -- 172.21.15.78:0/2851472428 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1674004030 con 0x55f12abe3800 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.194+0000 7f167db52640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16640b5450 0x7f16640b7910 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f16700107d0 tx=0x7f1670004a40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.194+0000 7f16637fe640 1 -- 172.21.15.78:0/2851472428 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1674035040 con 0x55f12abe3800 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.195+0000 7f168198a740 1 -- 172.21.15.78:0/2851472428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe3800 msgr2=0x55f12abef250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.195+0000 7f168198a740 1 --2- 172.21.15.78:0/2851472428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe3800 0x55f12abef250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1674009230 tx=0x7f1674032b20 comp rx=0 tx=0).stop 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.195+0000 7f168198a740 1 -- 172.21.15.78:0/2851472428 shutdown_connections 2024-09-11T01:11:25.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.195+0000 7f168198a740 1 --2- 172.21.15.78:0/2851472428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abef790 0x55f12abf1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.195+0000 7f168198a740 1 --2- 172.21.15.78:0/2851472428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe3800 0x55f12abef250 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.195+0000 7f168198a740 1 --2- 172.21.15.78:0/2851472428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 0x55f12abe32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.195+0000 7f168198a740 1 -- 172.21.15.78:0/2851472428 >> 172.21.15.78:0/2851472428 conn(0x55f12abe0d10 msgr2=0x55f12abe1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.195+0000 7f168198a740 1 -- 172.21.15.78:0/2851472428 shutdown_connections 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.195+0000 7f168198a740 1 -- 172.21.15.78:0/2851472428 wait complete. 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.195+0000 7f168198a740 1 -- start start 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.195+0000 7f168198a740 1 Processor -- start 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f168198a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 0x55f12abec750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f168198a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 0x55f12abecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f168198a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abef790 0x55f12abed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f168198a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f12a9aa890 con 0x55f12abe2ee0 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f168198a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f12aae3d40 con 0x55f12abef790 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f168198a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f12abf23e0 con 0x55f12abe3800 2024-09-11T01:11:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f168198a740 10 client.0 _ll_register_callbacks cb 0x55f12ab47c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167db52640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abef790 0x55f12abed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167cb50640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 0x55f12abecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167cb50640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 0x55f12abecc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:33296/0 (socket says 172.21.15.78:33296) 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167db52640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abef790 0x55f12abed1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:53156/0 (socket says 172.21.15.78:53156) 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167cb50640 1 -- 172.21.15.78:0/2433400630 learned_addr learned my addr 172.21.15.78:0/2433400630 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167d351640 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 0x55f12abec750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167db52640 1 -- 172.21.15.78:0/2433400630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 msgr2=0x55f12abecc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167db52640 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 0x55f12abecc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167db52640 1 -- 172.21.15.78:0/2433400630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 msgr2=0x55f12abec750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167db52640 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 0x55f12abec750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167db52640 1 -- 172.21.15.78:0/2433400630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f12abfb7c0 con 0x55f12abef790 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167cb50640 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 0x55f12abecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.196+0000 7f167d351640 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 0x55f12abec750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f167db52640 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abef790 0x55f12abed1d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f167000f790 tx=0x7f1670004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aa1e8b92ce49f81 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f1660ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f1660ff9640 1 -- 172.21.15.78:0/2433400630 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670004d20 con 0x55f12abef790 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f1660ff9640 1 -- 172.21.15.78:0/2433400630 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f1670005020 con 0x55f12abef790 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f168198a740 1 -- 172.21.15.78:0/2433400630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1674004030 con 0x55f12abef790 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f1660ff9640 1 -- 172.21.15.78:0/2433400630 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1670005220 con 0x55f12abef790 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f168198a740 10 client.27089 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f168198a740 1 -- 172.21.15.78:0/2433400630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f12abfe170 con 0x55f12abef790 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f1660ff9640 1 -- 172.21.15.78:0/2433400630 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1674004030 con 0x55f12abef790 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f1660ff9640 1 -- 172.21.15.78:0/2433400630 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f1670026070 con 0x55f12abef790 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f168198a740 10 client.27089 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f168198a740 10 client.27089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f168198a740 20 client.27089 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f168198a740 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f12abff810 0x55f12ac01c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f168198a740 4 client.27089 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:11:25.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.197+0000 7f168198a740 1 -- 172.21.15.78:0/2433400630 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f12ab5bdb0 con 0x55f12abff810 2024-09-11T01:11:25.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.198+0000 7f167d351640 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f12abff810 0x55f12ac01c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.198+0000 7f167d351640 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f12abff810 0x55f12ac01c50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a724d17027d5a04 server_cookie=9ce4cf55703f1bb1 in_seq=0 out_seq=0 2024-09-11T01:11:25.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.198+0000 7f1660ff9640 10 client.27089 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:25.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f1660ff9640 1 -- 172.21.15.78:0/2433400630 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f12ab5bdb0 con 0x55f12abff810 2024-09-11T01:11:25.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f1660ff9640 10 client.27089 handle_command_reply: tid=0 2024-09-11T01:11:25.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f168198a740 1 client.27089 shutdown 2024-09-11T01:11:25.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f168198a740 1 -- 172.21.15.78:0/2433400630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abef790 msgr2=0x55f12abed1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:25.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f168198a740 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abef790 0x55f12abed1d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f167000f790 tx=0x7f1670004ab0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f168198a740 1 -- 172.21.15.78:0/2433400630 shutdown_connections 2024-09-11T01:11:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f168198a740 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abef790 0x55f12abed1d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f168198a740 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 0x55f12abecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f168198a740 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f12abff810 0x55f12ac01c50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f168198a740 1 --2- 172.21.15.78:0/2433400630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abe2ee0 0x55f12abec750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f168198a740 1 -- 172.21.15.78:0/2433400630 >> 172.21.15.78:0/2433400630 conn(0x55f12abe0d10 msgr2=0x55f12abdff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f1660ff9640 0 client.27089 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f168198a740 1 -- 172.21.15.78:0/2433400630 shutdown_connections 2024-09-11T01:11:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.199+0000 7f168198a740 1 -- 172.21.15.78:0/2433400630 wait complete. 2024-09-11T01:11:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.200+0000 7f168198a740 20 client.27089 trim_cache size 0 max 16384 2024-09-11T01:11:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.218+0000 7f168198a740 1 Processor -- start 2024-09-11T01:11:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.218+0000 7f168198a740 1 -- start start 2024-09-11T01:11:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f168198a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe2ee0 0x55f12abe67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f168198a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe3800 0x55f12abe9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f168198a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 0x55f12abe9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f168198a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16781ae650 con 0x55f12abef790 2024-09-11T01:11:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f168198a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f16781b0250 con 0x55f12abe3800 2024-09-11T01:11:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f168198a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16781aed20 con 0x55f12abe2ee0 2024-09-11T01:11:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f167db52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 0x55f12abe9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f167d351640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe2ee0 0x55f12abe67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f167db52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 0x55f12abe9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40546/0 (socket says 172.21.15.78:40546) 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f167db52640 1 -- 172.21.15.78:0/162809599 learned_addr learned my addr 172.21.15.78:0/162809599 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f167cb50640 1 --2- 172.21.15.78:0/162809599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe3800 0x55f12abe9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f167db52640 1 -- 172.21.15.78:0/162809599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe2ee0 msgr2=0x55f12abe67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f167db52640 1 --2- 172.21.15.78:0/162809599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe2ee0 0x55f12abe67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f167db52640 1 -- 172.21.15.78:0/162809599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe3800 msgr2=0x55f12abe9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f167db52640 1 --2- 172.21.15.78:0/162809599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe3800 0x55f12abe9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f167db52640 1 -- 172.21.15.78:0/162809599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f12abeb6b0 con 0x55f12abef790 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f167cb50640 1 --2- 172.21.15.78:0/162809599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe3800 0x55f12abe9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.219+0000 7f167d351640 1 --2- 172.21.15.78:0/162809599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe2ee0 0x55f12abe67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.220+0000 7f167db52640 1 --2- 172.21.15.78:0/162809599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 0x55f12abe9ea0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1670000f80 tx=0x7f1670009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45de8a7d13650d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.220+0000 7f1662ffd640 1 -- 172.21.15.78:0/162809599 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167003b020 con 0x55f12abef790 2024-09-11T01:11:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.220+0000 7f1662ffd640 1 -- 172.21.15.78:0/162809599 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f167001cc60 con 0x55f12abef790 2024-09-11T01:11:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.220+0000 7f1662ffd640 1 -- 172.21.15.78:0/162809599 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670005620 con 0x55f12abef790 2024-09-11T01:11:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.221+0000 7f168198a740 1 -- 172.21.15.78:0/162809599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 msgr2=0x55f12abe9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.221+0000 7f168198a740 1 --2- 172.21.15.78:0/162809599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 0x55f12abe9ea0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1670000f80 tx=0x7f1670009fd0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.221+0000 7f168198a740 1 -- 172.21.15.78:0/162809599 shutdown_connections 2024-09-11T01:11:25.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.221+0000 7f168198a740 1 --2- 172.21.15.78:0/162809599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 0x55f12abe9ea0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.221+0000 7f168198a740 1 --2- 172.21.15.78:0/162809599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe3800 0x55f12abe9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.221+0000 7f168198a740 1 --2- 172.21.15.78:0/162809599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe2ee0 0x55f12abe67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.221+0000 7f168198a740 1 -- 172.21.15.78:0/162809599 >> 172.21.15.78:0/162809599 conn(0x55f12abe0d10 msgr2=0x55f12abe6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.221+0000 7f168198a740 1 -- 172.21.15.78:0/162809599 shutdown_connections 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.221+0000 7f168198a740 1 -- 172.21.15.78:0/162809599 wait complete. 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f168198a740 1 -- start start 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f168198a740 1 Processor -- start 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f168198a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe2ee0 0x55f12abf3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f168198a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 0x55f12abf4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f168198a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 0x55f12ac65730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f167cb50640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 0x55f12abf4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f167db52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 0x55f12ac65730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f168198a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16781afac0 con 0x55f12abef790 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f168198a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f16781af3f0 con 0x55f12abe2ee0 2024-09-11T01:11:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f168198a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16781ac650 con 0x55f12abe3800 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f167cb50640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 0x55f12abf4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:33308/0 (socket says 172.21.15.78:33308) 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f167db52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 0x55f12ac65730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40560/0 (socket says 172.21.15.78:40560) 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f167cb50640 1 -- 172.21.15.78:0/3759483300 learned_addr learned my addr 172.21.15.78:0/3759483300 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f167d351640 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe2ee0 0x55f12abf3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.222+0000 7f168198a740 10 client.0 _ll_register_callbacks cb 0x7f1670009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f167cb50640 1 -- 172.21.15.78:0/3759483300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe2ee0 msgr2=0x55f12abf3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f167cb50640 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe2ee0 0x55f12abf3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f167cb50640 1 -- 172.21.15.78:0/3759483300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 msgr2=0x55f12ac65730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f167cb50640 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 0x55f12ac65730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f167cb50640 1 -- 172.21.15.78:0/3759483300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f12ac65e40 con 0x55f12abe3800 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f167db52640 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 0x55f12ac65730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f167cb50640 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 0x55f12abf4470 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16680e66b0 tx=0x7f16680ac700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a587198bfb22bd73 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f1660ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:11:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f1660ff9640 1 -- 172.21.15.78:0/3759483300 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16680e3fd0 con 0x55f12abe3800 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f1660ff9640 1 -- 172.21.15.78:0/3759483300 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7f16680b5b70 con 0x55f12abe3800 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f168198a740 1 -- 172.21.15.78:0/3759483300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f167001cc60 con 0x55f12abe3800 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f1660ff9640 1 -- 172.21.15.78:0/3759483300 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1668005680 con 0x55f12abe3800 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f168198a740 10 client.27028 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f168198a740 1 -- 172.21.15.78:0/3759483300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f12ac6d260 con 0x55f12abe3800 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.223+0000 7f1660ff9640 1 -- 172.21.15.78:0/3759483300 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f167001cc60 con 0x55f12abe3800 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.224+0000 7f1660ff9640 1 -- 172.21.15.78:0/3759483300 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f16680cfbc0 con 0x55f12abe3800 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.224+0000 7f168198a740 10 client.27028 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.224+0000 7f168198a740 10 client.27028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.224+0000 7f168198a740 20 client.27028 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.224+0000 7f168198a740 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f12ac6da80 0x55f12ac6fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.224+0000 7f168198a740 4 client.27028 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:11:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.224+0000 7f168198a740 1 -- 172.21.15.78:0/3759483300 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f12ab57860 con 0x55f12ac6da80 2024-09-11T01:11:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.224+0000 7f167d351640 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f12ac6da80 0x55f12ac6fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.224+0000 7f167d351640 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f12ac6da80 0x55f12ac6fea0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1178a24b47ebdfe6 server_cookie=86f4cf1e48c5e42 in_seq=0 out_seq=0 2024-09-11T01:11:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.224+0000 7f1660ff9640 10 client.27028 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.225+0000 7f1660ff9640 1 -- 172.21.15.78:0/3759483300 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f12ab57860 con 0x55f12ac6da80 2024-09-11T01:11:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.225+0000 7f1660ff9640 10 client.27028 handle_command_reply: tid=0 2024-09-11T01:11:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.225+0000 7f168198a740 1 client.27028 shutdown 2024-09-11T01:11:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f168198a740 1 -- 172.21.15.78:0/3759483300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 msgr2=0x55f12abf4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f168198a740 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 0x55f12abf4470 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16680e66b0 tx=0x7f16680ac700 comp rx=0 tx=0).stop 2024-09-11T01:11:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f168198a740 1 -- 172.21.15.78:0/3759483300 shutdown_connections 2024-09-11T01:11:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f168198a740 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f12ac6da80 0x55f12ac6fea0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f168198a740 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f12abef790 0x55f12ac65730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f168198a740 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f12abe3800 0x55f12abf4470 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f1660ff9640 0 client.27028 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f168198a740 1 --2- 172.21.15.78:0/3759483300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12abe2ee0 0x55f12abf3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f168198a740 1 -- 172.21.15.78:0/3759483300 >> 172.21.15.78:0/3759483300 conn(0x55f12abe0d10 msgr2=0x55f12abe68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f168198a740 1 -- 172.21.15.78:0/3759483300 shutdown_connections 2024-09-11T01:11:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f168198a740 1 -- 172.21.15.78:0/3759483300 wait complete. 2024-09-11T01:11:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f168198a740 20 client.27028 trim_cache size 0 max 16384 2024-09-11T01:11:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:11:25.233 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:11:25.233 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "75a1f644-bccb-4806-b819-99e9253c572d", 2024-09-11T01:11:25.233 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:11:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:11:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f167f5b5640 1 -- 172.21.15.78:0/3249242773 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16640b5450 msgr2=0x7f16640b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f167f5b5640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16640b5450 0x7f16640b7910 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f16700107d0 tx=0x7f1670004a40 comp rx=0 tx=0).stop 2024-09-11T01:11:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f167f5b5640 1 -- 172.21.15.78:0/3249242773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1678104550 msgr2=0x7f1678199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.226+0000 7f167f5b5640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1678104550 0x7f1678199850 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f166800e9f0 tx=0x7f166800edf0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.227+0000 7f167f5b5640 1 -- 172.21.15.78:0/3249242773 shutdown_connections 2024-09-11T01:11:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.227+0000 7f167f5b5640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16640b5450 0x7f16640b7910 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.227+0000 7f167f5b5640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f167810d100 0x7f167819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.227+0000 7f167f5b5640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1678104550 0x7f1678199850 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.227+0000 7f167f5b5640 1 --2- 172.21.15.78:0/3249242773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1678103b50 0x7f1678199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.227+0000 7f167f5b5640 1 -- 172.21.15.78:0/3249242773 >> 172.21.15.78:0/3249242773 conn(0x7f16780fd700 msgr2=0x7f16780fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.227+0000 7f167f5b5640 1 -- 172.21.15.78:0/3249242773 shutdown_connections 2024-09-11T01:11:25.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:25.227+0000 7f167f5b5640 1 -- 172.21.15.78:0/3249242773 wait complete. 2024-09-11T01:11:25.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 890563 5.99 MB/sec execute 573 sec latency 48.877 ms 2024-09-11T01:11:25.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:25 smithi110 ceph-mon[34712]: pgmap v711: 97 pgs: 97 active+clean; 742 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 301 op/s 2024-09-11T01:11:25.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:25 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:11:25.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:25 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:25.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:25 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:11:25.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:25 smithi112 ceph-mon[35384]: pgmap v711: 97 pgs: 97 active+clean; 742 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 301 op/s 2024-09-11T01:11:25.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:25 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:11:25.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:25 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:25.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:25 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:11:25.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:25 smithi078 ceph-mon[31315]: pgmap v711: 97 pgs: 97 active+clean; 742 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 301 op/s 2024-09-11T01:11:25.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:25 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:11:25.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:25 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:25.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:25 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:11:25.999 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:11:26.269 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:11:26.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 891901 5.99 MB/sec execute 574 sec latency 17.768 ms 2024-09-11T01:11:26.683 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:11:26.684 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.684 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:11:26.684 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:11:26.684 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:11:26.684 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:11:26.684 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.684 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.686 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:26 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:11:26.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:11:26.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:11:26.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:11:26.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:11:26.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:26 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:11:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:11:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:11:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:11:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:11:26.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:26 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:11:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.768+0000 7fc66b3c6640 1 -- 172.21.15.78:0/2876297141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc66410d0e0 msgr2=0x7fc66410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.768+0000 7fc66b3c6640 1 --2- 172.21.15.78:0/2876297141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc66410d0e0 0x7fc66410f550 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009900 tx=0x7fc65c02a410 comp rx=0 tx=0).stop 2024-09-11T01:11:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.768+0000 7fc66b3c6640 1 -- 172.21.15.78:0/2876297141 shutdown_connections 2024-09-11T01:11:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.768+0000 7fc66b3c6640 1 --2- 172.21.15.78:0/2876297141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc66410d0e0 0x7fc66410f550 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.768+0000 7fc66b3c6640 1 --2- 172.21.15.78:0/2876297141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc664104530 0x7fc66410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.768+0000 7fc66b3c6640 1 --2- 172.21.15.78:0/2876297141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc664103b30 0x7fc664103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.768+0000 7fc66b3c6640 1 -- 172.21.15.78:0/2876297141 >> 172.21.15.78:0/2876297141 conn(0x7fc6640fd700 msgr2=0x7fc6640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.768+0000 7fc66b3c6640 1 -- 172.21.15.78:0/2876297141 shutdown_connections 2024-09-11T01:11:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.768+0000 7fc66b3c6640 1 -- 172.21.15.78:0/2876297141 wait complete. 2024-09-11T01:11:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc66b3c6640 1 Processor -- start 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc66b3c6640 1 -- start start 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc66b3c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc664103b30 0x7fc664199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc66b3c6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc664104530 0x7fc664199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc66b3c6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc66410d0e0 0x7fc66419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc66b3c6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc664112db0 con 0x7fc664104530 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc66b3c6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc664112c30 con 0x7fc66410d0e0 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc66b3c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc664112f30 con 0x7fc664103b30 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc668961640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc664104530 0x7fc664199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc668961640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc664104530 0x7fc664199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40580/0 (socket says 172.21.15.78:40580) 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc668961640 1 -- 172.21.15.78:0/4137451430 learned_addr learned my addr 172.21.15.78:0/4137451430 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc669963640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc66410d0e0 0x7fc66419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.769+0000 7fc669162640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc664103b30 0x7fc664199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc668961640 1 -- 172.21.15.78:0/4137451430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc664103b30 msgr2=0x7fc664199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc668961640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc664103b30 0x7fc664199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc668961640 1 -- 172.21.15.78:0/4137451430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc66410d0e0 msgr2=0x7fc66419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc668961640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc66410d0e0 0x7fc66419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc668961640 1 -- 172.21.15.78:0/4137451430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66419f2d0 con 0x7fc664104530 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc669162640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc664103b30 0x7fc664199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc669963640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc66410d0e0 0x7fc66419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc668961640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc664104530 0x7fc664199760 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc65800ebc0 tx=0x7fc65800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc6627fc640 1 -- 172.21.15.78:0/4137451430 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658010630 con 0x7fc664104530 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc6627fc640 1 -- 172.21.15.78:0/4137451430 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc658010c80 con 0x7fc664104530 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc66b3c6640 1 -- 172.21.15.78:0/4137451430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66419f5c0 con 0x7fc664104530 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc6627fc640 1 -- 172.21.15.78:0/4137451430 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658018b60 con 0x7fc664104530 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.770+0000 7fc66b3c6640 1 -- 172.21.15.78:0/4137451430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6641a5c80 con 0x7fc664104530 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.773+0000 7fc66d79b740 1 Processor -- start 2024-09-11T01:11:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.773+0000 7fc66d79b740 1 -- start start 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.773+0000 7fc6627fc640 1 -- 172.21.15.78:0/4137451430 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc6580107d0 con 0x7fc664104530 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.773+0000 7fc66d79b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8c480 0x55f217c8c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.773+0000 7fc66d79b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8cda0 0x55f217c987f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.773+0000 7fc66d79b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c98d30 0x55f217c9b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.773+0000 7fc66d79b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f217c02e50 con 0x55f217c8c480 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.773+0000 7fc66d79b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f217bf2280 con 0x55f217c98d30 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.773+0000 7fc66d79b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f217bf6090 con 0x55f217c8cda0 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.773+0000 7fc669963640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8c480 0x55f217c8c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc669963640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8c480 0x55f217c8c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40590/0 (socket says 172.21.15.78:40590) 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc669963640 1 -- 172.21.15.78:0/1875162085 learned_addr learned my addr 172.21.15.78:0/1875162085 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc669162640 1 --2- 172.21.15.78:0/1875162085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8cda0 0x55f217c987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc668961640 1 --2- 172.21.15.78:0/1875162085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c98d30 0x55f217c9b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc669963640 1 -- 172.21.15.78:0/1875162085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8cda0 msgr2=0x55f217c987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc669963640 1 --2- 172.21.15.78:0/1875162085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8cda0 0x55f217c987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc669963640 1 -- 172.21.15.78:0/1875162085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c98d30 msgr2=0x55f217c9b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc669963640 1 --2- 172.21.15.78:0/1875162085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c98d30 0x55f217c9b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc669963640 1 -- 172.21.15.78:0/1875162085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f217950650 con 0x55f217c8c480 2024-09-11T01:11:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc668961640 1 --2- 172.21.15.78:0/1875162085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c98d30 0x55f217c9b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:11:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc669963640 1 --2- 172.21.15.78:0/1875162085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8c480 0x55f217c8c860 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc65c02aa00 tx=0x7fc65c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1232881e9e4348a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc64b7fe640 1 -- 172.21.15.78:0/1875162085 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c040070 con 0x55f217c8c480 2024-09-11T01:11:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc6627fc640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc64c0b5450 0x7fc64c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc64b7fe640 1 -- 172.21.15.78:0/1875162085 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc65c0047a0 con 0x55f217c8c480 2024-09-11T01:11:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc64b7fe640 1 -- 172.21.15.78:0/1875162085 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c035040 con 0x55f217c8c480 2024-09-11T01:11:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.774+0000 7fc669162640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc64c0b5450 0x7fc64c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.775+0000 7fc6627fc640 1 -- 172.21.15.78:0/4137451430 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7fc6580e6680 con 0x7fc664104530 2024-09-11T01:11:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.775+0000 7fc66d79b740 1 -- 172.21.15.78:0/1875162085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8c480 msgr2=0x55f217c8c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.775+0000 7fc669162640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc64c0b5450 0x7fc64c0b7910 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55f217c8b190 tx=0x7fc654005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.775+0000 7fc66d79b740 1 --2- 172.21.15.78:0/1875162085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8c480 0x55f217c8c860 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc65c02aa00 tx=0x7fc65c004430 comp rx=0 tx=0).stop 2024-09-11T01:11:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.775+0000 7fc66d79b740 1 -- 172.21.15.78:0/1875162085 shutdown_connections 2024-09-11T01:11:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.775+0000 7fc66d79b740 1 --2- 172.21.15.78:0/1875162085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c98d30 0x55f217c9b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.775+0000 7fc66d79b740 1 --2- 172.21.15.78:0/1875162085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8cda0 0x55f217c987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.775+0000 7fc66d79b740 1 --2- 172.21.15.78:0/1875162085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8c480 0x55f217c8c860 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.775+0000 7fc66d79b740 1 -- 172.21.15.78:0/1875162085 >> 172.21.15.78:0/1875162085 conn(0x55f217c8a2b0 msgr2=0x55f217c8b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.775+0000 7fc66d79b740 1 -- 172.21.15.78:0/1875162085 shutdown_connections 2024-09-11T01:11:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.775+0000 7fc66d79b740 1 -- 172.21.15.78:0/1875162085 wait complete. 2024-09-11T01:11:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc66d79b740 1 -- start start 2024-09-11T01:11:26.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc66d79b740 1 Processor -- start 2024-09-11T01:11:26.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc66d79b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 0x55f217c98580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc66d79b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8cda0 0x55f217c959d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc66d79b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c98d30 0x55f217c95f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc66d79b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f217a53700 con 0x55f217c8cda0 2024-09-11T01:11:26.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc66d79b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f217b8d110 con 0x55f217c8c480 2024-09-11T01:11:26.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc66d79b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f217cac960 con 0x55f217c98d30 2024-09-11T01:11:26.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc66d79b740 10 client.0 _ll_register_callbacks cb 0x55f217ba6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:26.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc668961640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8cda0 0x55f217c959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc669162640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 0x55f217c98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc668961640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8cda0 0x55f217c959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40604/0 (socket says 172.21.15.78:40604) 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc668961640 1 -- 172.21.15.78:0/1869821215 learned_addr learned my addr 172.21.15.78:0/1869821215 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc669162640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 0x55f217c98580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:53210/0 (socket says 172.21.15.78:53210) 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.776+0000 7fc669963640 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c98d30 0x55f217c95f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc669162640 1 -- 172.21.15.78:0/1869821215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c98d30 msgr2=0x55f217c95f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc669162640 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c98d30 0x55f217c95f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc669162640 1 -- 172.21.15.78:0/1869821215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8cda0 msgr2=0x55f217c959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc669162640 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8cda0 0x55f217c959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc669162640 1 -- 172.21.15.78:0/1869821215 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f217bbac40 con 0x55f217c8c480 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc669963640 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c98d30 0x55f217c95f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc669162640 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 0x55f217c98580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc654012cb0 tx=0x7fc654004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=854a9c3bb586735 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc648ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc648ff9640 1 -- 172.21.15.78:0/1869821215 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654004020 con 0x55f217c8c480 2024-09-11T01:11:26.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc648ff9640 1 -- 172.21.15.78:0/1869821215 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7fc6540041c0 con 0x55f217c8c480 2024-09-11T01:11:26.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc66d79b740 1 -- 172.21.15.78:0/1869821215 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc65c0047a0 con 0x55f217c8c480 2024-09-11T01:11:26.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc648ff9640 1 -- 172.21.15.78:0/1869821215 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc6540043c0 con 0x55f217c8c480 2024-09-11T01:11:26.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc66d79b740 10 client.27104 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:26.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.777+0000 7fc66d79b740 1 -- 172.21.15.78:0/1869821215 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f217cb8220 con 0x55f217c8c480 2024-09-11T01:11:26.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.778+0000 7fc648ff9640 1 -- 172.21.15.78:0/1869821215 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc65c0047a0 con 0x55f217c8c480 2024-09-11T01:11:26.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.778+0000 7fc648ff9640 1 -- 172.21.15.78:0/1869821215 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fc654003740 con 0x55f217c8c480 2024-09-11T01:11:26.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.778+0000 7fc66d79b740 10 client.27104 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:26.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.778+0000 7fc66d79b740 10 client.27104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:11:26.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.778+0000 7fc66d79b740 20 client.27104 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:26.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.778+0000 7fc66d79b740 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f217cb9850 0x55f217cbbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.778+0000 7fc66d79b740 4 client.27104 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:11:26.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.778+0000 7fc66d79b740 1 -- 172.21.15.78:0/1869821215 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f217c04da0 con 0x55f217cb9850 2024-09-11T01:11:26.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.778+0000 7fc668961640 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f217cb9850 0x55f217cbbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.778+0000 7fc648ff9640 10 client.27104 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:26.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.778+0000 7fc668961640 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f217cb9850 0x55f217cbbc90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b0713f2ef417a9e server_cookie=a5c43bc35550c697 in_seq=0 out_seq=0 2024-09-11T01:11:26.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc648ff9640 1 -- 172.21.15.78:0/1869821215 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f217c04da0 con 0x55f217cb9850 2024-09-11T01:11:26.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc648ff9640 10 client.27104 handle_command_reply: tid=0 2024-09-11T01:11:26.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc66d79b740 1 client.27104 shutdown 2024-09-11T01:11:26.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc66d79b740 1 -- 172.21.15.78:0/1869821215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 msgr2=0x55f217c98580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:26.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc66d79b740 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 0x55f217c98580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc654012cb0 tx=0x7fc654004730 comp rx=0 tx=0).stop 2024-09-11T01:11:26.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc66d79b740 1 -- 172.21.15.78:0/1869821215 shutdown_connections 2024-09-11T01:11:26.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc66d79b740 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c98d30 0x55f217c95f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc66d79b740 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f217cb9850 0x55f217cbbc90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc66d79b740 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c8cda0 0x55f217c959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc66d79b740 1 --2- 172.21.15.78:0/1869821215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 0x55f217c98580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc648ff9640 0 client.27104 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:26.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc66d79b740 1 -- 172.21.15.78:0/1869821215 >> 172.21.15.78:0/1869821215 conn(0x55f217c8a2b0 msgr2=0x55f217c8fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:26.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc66d79b740 1 -- 172.21.15.78:0/1869821215 shutdown_connections 2024-09-11T01:11:26.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc66d79b740 1 -- 172.21.15.78:0/1869821215 wait complete. 2024-09-11T01:11:26.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.779+0000 7fc66d79b740 20 client.27104 trim_cache size 0 max 16384 2024-09-11T01:11:26.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc66d79b740 1 Processor -- start 2024-09-11T01:11:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc66d79b740 1 -- start start 2024-09-11T01:11:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc66d79b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 0x55f217c8ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc66d79b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8cda0 0x55f217cb5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc66d79b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c98d30 0x55f217c92c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc66d79b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641a5f40 con 0x55f217c98d30 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc66d79b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641a7a20 con 0x55f217c8c480 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc66d79b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641a64f0 con 0x55f217c8cda0 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc669162640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 0x55f217c8ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc669162640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 0x55f217c8ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:53212/0 (socket says 172.21.15.78:53212) 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc669162640 1 -- 172.21.15.78:0/2257285085 learned_addr learned my addr 172.21.15.78:0/2257285085 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc668961640 1 --2- 172.21.15.78:0/2257285085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8cda0 0x55f217cb5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc669963640 1 --2- 172.21.15.78:0/2257285085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c98d30 0x55f217c92c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc669162640 1 -- 172.21.15.78:0/2257285085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8cda0 msgr2=0x55f217cb5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc669162640 1 --2- 172.21.15.78:0/2257285085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8cda0 0x55f217cb5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc669162640 1 -- 172.21.15.78:0/2257285085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c98d30 msgr2=0x55f217c92c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc669162640 1 --2- 172.21.15.78:0/2257285085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c98d30 0x55f217c92c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.800+0000 7fc669162640 1 -- 172.21.15.78:0/2257285085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f217c8ad30 con 0x55f217c8c480 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.801+0000 7fc669162640 1 --2- 172.21.15.78:0/2257285085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 0x55f217c8ba80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc654019040 tx=0x7fc654004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47208075b1229205 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.801+0000 7fc64affd640 1 -- 172.21.15.78:0/2257285085 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65403f070 con 0x55f217c8c480 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.801+0000 7fc64affd640 1 -- 172.21.15.78:0/2257285085 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc65401f7c0 con 0x55f217c8c480 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.801+0000 7fc64affd640 1 -- 172.21.15.78:0/2257285085 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654036030 con 0x55f217c8c480 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.801+0000 7fc66d79b740 1 -- 172.21.15.78:0/2257285085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 msgr2=0x55f217c8ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 --2- 172.21.15.78:0/2257285085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 0x55f217c8ba80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc654019040 tx=0x7fc654004560 comp rx=0 tx=0).stop 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 -- 172.21.15.78:0/2257285085 shutdown_connections 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 --2- 172.21.15.78:0/2257285085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c98d30 0x55f217c92c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 --2- 172.21.15.78:0/2257285085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8cda0 0x55f217cb5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 --2- 172.21.15.78:0/2257285085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8c480 0x55f217c8ba80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 -- 172.21.15.78:0/2257285085 >> 172.21.15.78:0/2257285085 conn(0x55f217c8a2b0 msgr2=0x55f217c8b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 -- 172.21.15.78:0/2257285085 shutdown_connections 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 -- 172.21.15.78:0/2257285085 wait complete. 2024-09-11T01:11:26.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 -- start start 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 Processor -- start 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8c480 0x55f217cb1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8cda0 0x55f217cb2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c98d30 0x55f217caec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.802+0000 7fc66d79b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641a7290 con 0x55f217c98d30 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc66d79b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641a6bc0 con 0x55f217c8cda0 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc66d79b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc66419a350 con 0x55f217c8c480 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc66d79b740 10 client.0 _ll_register_callbacks cb 0x7fc654009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc669963640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c98d30 0x55f217caec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc669162640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8c480 0x55f217cb1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc669963640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c98d30 0x55f217caec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40622/0 (socket says 172.21.15.78:40622) 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc669963640 1 -- 172.21.15.78:0/3491399159 learned_addr learned my addr 172.21.15.78:0/3491399159 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:26.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc668961640 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8cda0 0x55f217cb2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc669963640 1 -- 172.21.15.78:0/3491399159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8c480 msgr2=0x55f217cb1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc669963640 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8c480 0x55f217cb1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc669963640 1 -- 172.21.15.78:0/3491399159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8cda0 msgr2=0x55f217cb2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc669963640 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8cda0 0x55f217cb2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc669963640 1 -- 172.21.15.78:0/3491399159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f217c90540 con 0x55f217c98d30 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc669162640 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8c480 0x55f217cb1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc669963640 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c98d30 0x55f217caec20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc65c02aa00 tx=0x7fc65c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c3be80eb12d44e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc648ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc648ff9640 1 -- 172.21.15.78:0/3491399159 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c040040 con 0x55f217c98d30 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc648ff9640 1 -- 172.21.15.78:0/3491399159 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7fc65c0318b0 con 0x55f217c98d30 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.803+0000 7fc66d79b740 1 -- 172.21.15.78:0/3491399159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f217c00850 con 0x55f217c98d30 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.804+0000 7fc648ff9640 1 -- 172.21.15.78:0/3491399159 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc65c035040 con 0x55f217c98d30 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.804+0000 7fc66d79b740 10 client.17919 fetch_fsmap learned FSMap version 48 2024-09-11T01:11:26.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.804+0000 7fc66d79b740 1 -- 172.21.15.78:0/3491399159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f217c85940 con 0x55f217c98d30 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.804+0000 7fc648ff9640 1 -- 172.21.15.78:0/3491399159 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f217c00850 con 0x55f217c98d30 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.804+0000 7fc648ff9640 1 -- 172.21.15.78:0/3491399159 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fc65c03cbb0 con 0x55f217c98d30 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc66d79b740 10 client.17919 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc66d79b740 10 client.17919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc66d79b740 20 client.17919 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc66d79b740 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f217d270b0 0x55f217d294d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc66d79b740 4 client.17919 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc66d79b740 1 -- 172.21.15.78:0/3491399159 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f217bfdc80 con 0x55f217d270b0 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc669162640 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f217d270b0 0x55f217d294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc669162640 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f217d270b0 0x55f217d294d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8251dd650ff6ff6a server_cookie=34026ae9c5695e4a in_seq=0 out_seq=0 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc648ff9640 10 client.17919 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc648ff9640 1 -- 172.21.15.78:0/3491399159 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5114 (crc 0 0 0) 0x55f217bfdc80 con 0x55f217d270b0 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc648ff9640 10 client.17919 handle_command_reply: tid=0 2024-09-11T01:11:26.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc66d79b740 1 client.17919 shutdown 2024-09-11T01:11:26.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc66d79b740 1 -- 172.21.15.78:0/3491399159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c98d30 msgr2=0x55f217caec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:26.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.805+0000 7fc66d79b740 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c98d30 0x55f217caec20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc65c02aa00 tx=0x7fc65c003d50 comp rx=0 tx=0).stop 2024-09-11T01:11:26.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.806+0000 7fc66d79b740 1 -- 172.21.15.78:0/3491399159 shutdown_connections 2024-09-11T01:11:26.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.806+0000 7fc66d79b740 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f217d270b0 0x55f217d294d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.806+0000 7fc66d79b740 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f217c98d30 0x55f217caec20 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.806+0000 7fc66d79b740 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f217c8cda0 0x55f217cb2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.806+0000 7fc648ff9640 0 client.17919 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:26.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.806+0000 7fc66d79b740 1 --2- 172.21.15.78:0/3491399159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f217c8c480 0x55f217cb1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.806+0000 7fc66d79b740 1 -- 172.21.15.78:0/3491399159 >> 172.21.15.78:0/3491399159 conn(0x55f217c8a2b0 msgr2=0x55f217c88bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:26.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.806+0000 7fc66d79b740 1 -- 172.21.15.78:0/3491399159 shutdown_connections 2024-09-11T01:11:26.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.806+0000 7fc66d79b740 1 -- 172.21.15.78:0/3491399159 wait complete. 2024-09-11T01:11:26.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.806+0000 7fc66d79b740 20 client.17919 trim_cache size 0 max 16384 2024-09-11T01:11:26.823 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:11:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:11:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:11:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:11:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "75a1f644-bccb-4806-b819-99e9253c572d", 2024-09-11T01:11:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:11:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:11:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:11:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:11:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:11:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:11:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:11:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:11:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:11:26.834 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:11:26.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.807+0000 7fc66b3c6640 1 -- 172.21.15.78:0/4137451430 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc64c0b5450 msgr2=0x7fc64c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:26.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.807+0000 7fc66b3c6640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc64c0b5450 0x7fc64c0b7910 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55f217c8b190 tx=0x7fc654005f90 comp rx=0 tx=0).stop 2024-09-11T01:11:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.807+0000 7fc66b3c6640 1 -- 172.21.15.78:0/4137451430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc664104530 msgr2=0x7fc664199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.807+0000 7fc66b3c6640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc664104530 0x7fc664199760 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc65800ebc0 tx=0x7fc65800c6a0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.807+0000 7fc66b3c6640 1 -- 172.21.15.78:0/4137451430 shutdown_connections 2024-09-11T01:11:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.807+0000 7fc66b3c6640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc64c0b5450 0x7fc64c0b7910 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.807+0000 7fc66b3c6640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc66410d0e0 0x7fc66419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.807+0000 7fc66b3c6640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc664104530 0x7fc664199760 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.807+0000 7fc66b3c6640 1 --2- 172.21.15.78:0/4137451430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc664103b30 0x7fc664199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.807+0000 7fc66b3c6640 1 -- 172.21.15.78:0/4137451430 >> 172.21.15.78:0/4137451430 conn(0x7fc6640fd700 msgr2=0x7fc6640fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.807+0000 7fc66b3c6640 1 -- 172.21.15.78:0/4137451430 shutdown_connections 2024-09-11T01:11:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:26.807+0000 7fc66b3c6640 1 -- 172.21.15.78:0/4137451430 wait complete. 2024-09-11T01:11:27.458 INFO:tasks.workunit.client.0.smithi078.stdout: 1 893711 5.99 MB/sec execute 575 sec latency 40.876 ms 2024-09-11T01:11:27.553 INFO:tasks.cephfs.filesystem:scrub status for tag:75a1f644-bccb-4806-b819-99e9253c572d - {'path': '/', 'tag': '75a1f644-bccb-4806-b819-99e9253c572d', 'options': 'recursive,force'} 2024-09-11T01:11:27.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:27 smithi110 ceph-mon[34712]: pgmap v712: 97 pgs: 97 active+clean; 747 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 294 op/s 2024-09-11T01:11:27.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:27 smithi112 ceph-mon[35384]: pgmap v712: 97 pgs: 97 active+clean; 747 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 294 op/s 2024-09-11T01:11:27.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:27 smithi078 ceph-mon[31315]: pgmap v712: 97 pgs: 97 active+clean; 747 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 294 op/s 2024-09-11T01:11:28.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 895244 6.00 MB/sec execute 576 sec latency 20.453 ms 2024-09-11T01:11:29.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 897066 6.01 MB/sec execute 577 sec latency 52.545 ms 2024-09-11T01:11:29.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:29 smithi112 ceph-mon[35384]: pgmap v713: 97 pgs: 97 active+clean; 747 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 252 op/s 2024-09-11T01:11:29.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:29 smithi078 ceph-mon[31315]: pgmap v713: 97 pgs: 97 active+clean; 747 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 252 op/s 2024-09-11T01:11:29.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:29 smithi110 ceph-mon[34712]: pgmap v713: 97 pgs: 97 active+clean; 747 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 252 op/s 2024-09-11T01:11:30.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 898594 6.02 MB/sec execute 578 sec latency 16.709 ms 2024-09-11T01:11:31.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 900234 6.01 MB/sec execute 579 sec latency 19.129 ms 2024-09-11T01:11:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:31 smithi112 ceph-mon[35384]: pgmap v714: 97 pgs: 97 active+clean; 752 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 288 op/s 2024-09-11T01:11:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:31 smithi078 ceph-mon[31315]: pgmap v714: 97 pgs: 97 active+clean; 752 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 288 op/s 2024-09-11T01:11:31.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:31 smithi110 ceph-mon[34712]: pgmap v714: 97 pgs: 97 active+clean; 752 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 288 op/s 2024-09-11T01:11:32.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 902044 6.03 MB/sec execute 580 sec latency 14.144 ms 2024-09-11T01:11:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:32 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:11:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:32 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:11:32.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:32 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:11:32.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:32 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:11:32.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:32 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:11:32.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:32 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:11:33.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 903397 6.03 MB/sec execute 581 sec latency 38.274 ms 2024-09-11T01:11:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:11:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:11:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:11:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:33 smithi112 ceph-mon[35384]: pgmap v715: 97 pgs: 97 active+clean; 757 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 289 op/s 2024-09-11T01:11:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:33 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:11:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:11:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:11:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:33 smithi078 ceph-mon[31315]: pgmap v715: 97 pgs: 97 active+clean; 757 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 289 op/s 2024-09-11T01:11:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:33 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:33.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:11:33.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:11:33.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:11:33.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:33 smithi110 ceph-mon[34712]: pgmap v715: 97 pgs: 97 active+clean; 757 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 289 op/s 2024-09-11T01:11:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:33 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:34.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 905437 6.04 MB/sec execute 582 sec latency 33.094 ms 2024-09-11T01:11:35.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 906700 6.04 MB/sec execute 583 sec latency 16.271 ms 2024-09-11T01:11:35.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:35 smithi112 ceph-mon[35384]: pgmap v716: 97 pgs: 97 active+clean; 757 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2024-09-11T01:11:35.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:35 smithi078 ceph-mon[31315]: pgmap v716: 97 pgs: 97 active+clean; 757 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2024-09-11T01:11:35.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:35 smithi110 ceph-mon[34712]: pgmap v716: 97 pgs: 97 active+clean; 757 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2024-09-11T01:11:36.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 908841 6.05 MB/sec execute 584 sec latency 17.942 ms 2024-09-11T01:11:37.455 INFO:tasks.workunit.client.0.smithi078.stdout: 1 910292 6.05 MB/sec execute 585 sec latency 13.843 ms 2024-09-11T01:11:37.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:37 smithi112 ceph-mon[35384]: pgmap v717: 97 pgs: 97 active+clean; 760 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 287 op/s 2024-09-11T01:11:37.741 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:37 smithi110 ceph-mon[34712]: pgmap v717: 97 pgs: 97 active+clean; 760 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 287 op/s 2024-09-11T01:11:37.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:37 smithi078 ceph-mon[31315]: pgmap v717: 97 pgs: 97 active+clean; 760 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 287 op/s 2024-09-11T01:11:38.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 912592 6.06 MB/sec execute 586 sec latency 17.217 ms 2024-09-11T01:11:39.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 913887 6.06 MB/sec execute 587 sec latency 42.306 ms 2024-09-11T01:11:39.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:39 smithi112 ceph-mon[35384]: pgmap v718: 97 pgs: 97 active+clean; 764 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-09-11T01:11:39.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:39 smithi078 ceph-mon[31315]: pgmap v718: 97 pgs: 97 active+clean; 764 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-09-11T01:11:39.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:39 smithi110 ceph-mon[34712]: pgmap v718: 97 pgs: 97 active+clean; 764 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-09-11T01:11:40.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 916048 6.07 MB/sec execute 588 sec latency 17.154 ms 2024-09-11T01:11:41.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 916977 6.08 MB/sec execute 589 sec latency 62.916 ms 2024-09-11T01:11:41.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:41 smithi112 ceph-mon[35384]: pgmap v719: 97 pgs: 97 active+clean; 765 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 289 op/s 2024-09-11T01:11:41.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:41 smithi078 ceph-mon[31315]: pgmap v719: 97 pgs: 97 active+clean; 765 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 289 op/s 2024-09-11T01:11:41.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:41 smithi110 ceph-mon[34712]: pgmap v719: 97 pgs: 97 active+clean; 765 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 289 op/s 2024-09-11T01:11:42.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 918322 6.10 MB/sec execute 590 sec latency 23.290 ms 2024-09-11T01:11:43.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 920443 6.11 MB/sec execute 591 sec latency 16.446 ms 2024-09-11T01:11:43.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:43 smithi112 ceph-mon[35384]: pgmap v720: 97 pgs: 97 active+clean; 770 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 293 op/s 2024-09-11T01:11:43.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:43 smithi078 ceph-mon[31315]: pgmap v720: 97 pgs: 97 active+clean; 770 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 293 op/s 2024-09-11T01:11:43.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:43 smithi110 ceph-mon[34712]: pgmap v720: 97 pgs: 97 active+clean; 770 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 293 op/s 2024-09-11T01:11:44.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 921479 6.11 MB/sec execute 592 sec latency 40.014 ms 2024-09-11T01:11:45.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 923433 6.11 MB/sec execute 593 sec latency 19.636 ms 2024-09-11T01:11:45.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:45 smithi112 ceph-mon[35384]: pgmap v721: 97 pgs: 97 active+clean; 772 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 276 op/s 2024-09-11T01:11:45.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:45 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:45.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:45 smithi078 ceph-mon[31315]: pgmap v721: 97 pgs: 97 active+clean; 772 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 276 op/s 2024-09-11T01:11:45.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:45 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:45.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:45 smithi110 ceph-mon[34712]: pgmap v721: 97 pgs: 97 active+clean; 772 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 276 op/s 2024-09-11T01:11:45.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:45 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:46.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 924789 6.12 MB/sec execute 594 sec latency 13.692 ms 2024-09-11T01:11:47.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 926963 6.13 MB/sec execute 595 sec latency 14.481 ms 2024-09-11T01:11:47.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:47 smithi112 ceph-mon[35384]: pgmap v722: 97 pgs: 97 active+clean; 776 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 299 op/s 2024-09-11T01:11:47.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:47 smithi078 ceph-mon[31315]: pgmap v722: 97 pgs: 97 active+clean; 776 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 299 op/s 2024-09-11T01:11:47.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:47 smithi110 ceph-mon[34712]: pgmap v722: 97 pgs: 97 active+clean; 776 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 299 op/s 2024-09-11T01:11:48.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 928329 6.13 MB/sec execute 596 sec latency 61.086 ms 2024-09-11T01:11:49.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 930353 6.14 MB/sec execute 597 sec latency 32.993 ms 2024-09-11T01:11:49.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:49 smithi078 ceph-mon[31315]: pgmap v723: 97 pgs: 97 active+clean; 778 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 292 op/s 2024-09-11T01:11:49.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:49 smithi110 ceph-mon[34712]: pgmap v723: 97 pgs: 97 active+clean; 778 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 292 op/s 2024-09-11T01:11:49.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:49 smithi112 ceph-mon[35384]: pgmap v723: 97 pgs: 97 active+clean; 778 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 292 op/s 2024-09-11T01:11:50.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 930956 6.14 MB/sec execute 598 sec latency 795.695 ms 2024-09-11T01:11:50.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:50 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:50.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:50 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:50.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:50 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:51.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 930956 6.12 MB/sec execute 599 sec latency 1795.780 ms 2024-09-11T01:11:51.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:51 smithi078 ceph-mon[31315]: pgmap v724: 97 pgs: 97 active+clean; 783 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 296 op/s 2024-09-11T01:11:51.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:51 smithi110 ceph-mon[34712]: pgmap v724: 97 pgs: 97 active+clean; 783 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 296 op/s 2024-09-11T01:11:51.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:51 smithi112 ceph-mon[35384]: pgmap v724: 97 pgs: 97 active+clean; 783 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 296 op/s 2024-09-11T01:11:52.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 cleanup 600 sec 2024-09-11T01:11:53.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 cleanup 601 sec 2024-09-11T01:11:53.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:53 smithi078 ceph-mon[31315]: pgmap v725: 97 pgs: 97 active+clean; 784 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 282 op/s 2024-09-11T01:11:53.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:53 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:53.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:53 smithi110 ceph-mon[34712]: pgmap v725: 97 pgs: 97 active+clean; 784 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 282 op/s 2024-09-11T01:11:53.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:53 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:53 smithi112 ceph-mon[35384]: pgmap v725: 97 pgs: 97 active+clean; 784 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 282 op/s 2024-09-11T01:11:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:53 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:54.456 INFO:tasks.workunit.client.0.smithi078.stdout: 1 cleanup 602 sec 2024-09-11T01:11:54.672 INFO:tasks.workunit.client.0.smithi078.stdout: 0 cleanup 602 sec 2024-09-11T01:11:54.672 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-09-11T01:11:54.672 INFO:tasks.workunit.client.0.smithi078.stdout: Operation Count AvgLat MaxLat 2024-09-11T01:11:54.672 INFO:tasks.workunit.client.0.smithi078.stdout: ---------------------------------------- 2024-09-11T01:11:54.672 INFO:tasks.workunit.client.0.smithi078.stdout: NTCreateX 116387 1.529 9843.965 2024-09-11T01:11:54.672 INFO:tasks.workunit.client.0.smithi078.stdout: Close 85528 0.002 0.090 2024-09-11T01:11:54.672 INFO:tasks.workunit.client.0.smithi078.stdout: Rename 4929 42.877 4775.155 2024-09-11T01:11:54.672 INFO:tasks.workunit.client.0.smithi078.stdout: Unlink 23480 1.446 56.712 2024-09-11T01:11:54.672 INFO:tasks.workunit.client.0.smithi078.stdout: Deltree 4 94.255 158.662 2024-09-11T01:11:54.673 INFO:tasks.workunit.client.0.smithi078.stdout: Mkdir 2 0.004 0.004 2024-09-11T01:11:54.673 INFO:tasks.workunit.client.0.smithi078.stdout: Qpathinfo 105521 1.269 4961.462 2024-09-11T01:11:54.673 INFO:tasks.workunit.client.0.smithi078.stdout: Qfileinfo 18539 0.001 0.025 2024-09-11T01:11:54.673 INFO:tasks.workunit.client.0.smithi078.stdout: Qfsinfo 19353 0.169 37.533 2024-09-11T01:11:54.673 INFO:tasks.workunit.client.0.smithi078.stdout: Sfileinfo 9474 0.009 3.411 2024-09-11T01:11:54.673 INFO:tasks.workunit.client.0.smithi078.stdout: Find 40823 0.342 6934.826 2024-09-11T01:11:54.673 INFO:tasks.workunit.client.0.smithi078.stdout: WriteX 58248 0.054 46.836 2024-09-11T01:11:54.673 INFO:tasks.workunit.client.0.smithi078.stdout: ReadX 182647 0.006 54.822 2024-09-11T01:11:54.673 INFO:tasks.workunit.client.0.smithi078.stdout: LockX 380 0.455 1.263 2024-09-11T01:11:54.673 INFO:tasks.workunit.client.0.smithi078.stdout: UnlockX 380 0.435 1.177 2024-09-11T01:11:54.673 INFO:tasks.workunit.client.0.smithi078.stdout: Flush 8155 2.596 86.652 2024-09-11T01:11:54.673 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-09-11T01:11:54.674 INFO:tasks.workunit.client.0.smithi078.stdout:Throughput 6.1248 MB/sec 1 clients 1 procs max_latency=9844.501 ms 2024-09-11T01:11:54.674 INFO:tasks.workunit.client.0.smithi078.stderr:+ dbench 10 2024-09-11T01:11:54.674 INFO:tasks.workunit.client.0.smithi078.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-11T01:11:54.674 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-09-11T01:11:54.674 INFO:tasks.workunit.client.0.smithi078.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-11T01:11:54.675 INFO:tasks.workunit.client.0.smithi078.stdout:9 of 10 processes prepared for launch 0 sec 2024-09-11T01:11:54.774 INFO:tasks.workunit.client.0.smithi078.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-11T01:11:54.775 INFO:tasks.workunit.client.0.smithi078.stdout:releasing clients 2024-09-11T01:11:55.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:55 smithi078 ceph-mon[31315]: pgmap v726: 97 pgs: 97 active+clean; 784 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 251 op/s 2024-09-11T01:11:55.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 374 133.82 MB/sec warmup 1 sec latency 86.549 ms 2024-09-11T01:11:55.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:55 smithi110 ceph-mon[34712]: pgmap v726: 97 pgs: 97 active+clean; 784 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 251 op/s 2024-09-11T01:11:55.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:55 smithi112 ceph-mon[35384]: pgmap v726: 97 pgs: 97 active+clean; 784 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 251 op/s 2024-09-11T01:11:56.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 829 104.29 MB/sec warmup 2 sec latency 118.597 ms 2024-09-11T01:11:57.555 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:11:57.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:57 smithi078 ceph-mon[31315]: pgmap v727: 97 pgs: 97 active+clean; 784 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-09-11T01:11:57.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 1348 84.43 MB/sec warmup 3 sec latency 136.172 ms 2024-09-11T01:11:57.832 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:11:57.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:57 smithi110 ceph-mon[34712]: pgmap v727: 97 pgs: 97 active+clean; 784 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-09-11T01:11:57.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:57 smithi112 ceph-mon[35384]: pgmap v727: 97 pgs: 97 active+clean; 784 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-09-11T01:11:58.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.323+0000 7fa2aef6b640 1 -- 172.21.15.78:0/3479026247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2a8101cc0 msgr2=0x7fa2a81020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:58.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.323+0000 7fa2aef6b640 1 --2- 172.21.15.78:0/3479026247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2a8101cc0 0x7fa2a81020a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa298009900 tx=0x7fa29802a410 comp rx=0 tx=0).stop 2024-09-11T01:11:58.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.323+0000 7fa2aef6b640 1 -- 172.21.15.78:0/3479026247 shutdown_connections 2024-09-11T01:11:58.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.323+0000 7fa2aef6b640 1 --2- 172.21.15.78:0/3479026247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2a8108050 0x7fa2a810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.323+0000 7fa2aef6b640 1 --2- 172.21.15.78:0/3479026247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a81076d0 0x7fa2a8107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.323+0000 7fa2aef6b640 1 --2- 172.21.15.78:0/3479026247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2a8101cc0 0x7fa2a81020a0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.323+0000 7fa2aef6b640 1 -- 172.21.15.78:0/3479026247 >> 172.21.15.78:0/3479026247 conn(0x7fa2a80fd750 msgr2=0x7fa2a80ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.324+0000 7fa2aef6b640 1 -- 172.21.15.78:0/3479026247 shutdown_connections 2024-09-11T01:11:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.324+0000 7fa2aef6b640 1 -- 172.21.15.78:0/3479026247 wait complete. 2024-09-11T01:11:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.324+0000 7fa2aef6b640 1 Processor -- start 2024-09-11T01:11:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.324+0000 7fa2aef6b640 1 -- start start 2024-09-11T01:11:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.324+0000 7fa2aef6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a8101cc0 0x7fa2a8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.324+0000 7fa2aef6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2a81076d0 0x7fa2a8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.324+0000 7fa2aef6b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2a8108050 0x7fa2a8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.324+0000 7fa2aef6b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8111d10 con 0x7fa2a81076d0 2024-09-11T01:11:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.324+0000 7fa2aef6b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8111b90 con 0x7fa2a8108050 2024-09-11T01:11:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.324+0000 7fa2aef6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8111e90 con 0x7fa2a8101cc0 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.324+0000 7fa2a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2a81076d0 0x7fa2a8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2a81076d0 0x7fa2a8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52286/0 (socket says 172.21.15.78:52286) 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2a7fff640 1 -- 172.21.15.78:0/4247193757 learned_addr learned my addr 172.21.15.78:0/4247193757 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2acd07640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a8101cc0 0x7fa2a8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2ad508640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2a8108050 0x7fa2a8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2acd07640 1 -- 172.21.15.78:0/4247193757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2a8108050 msgr2=0x7fa2a8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2acd07640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2a8108050 0x7fa2a8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2acd07640 1 -- 172.21.15.78:0/4247193757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2a81076d0 msgr2=0x7fa2a8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2acd07640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2a81076d0 0x7fa2a8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2acd07640 1 -- 172.21.15.78:0/4247193757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a81a76b0 con 0x7fa2a8101cc0 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2a7fff640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2a81076d0 0x7fa2a8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2acd07640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a8101cc0 0x7fa2a8197ed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa29802a930 tx=0x7fa298031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2a5ffb640 1 -- 172.21.15.78:0/4247193757 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29803f030 con 0x7fa2a8101cc0 2024-09-11T01:11:58.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2aef6b640 1 -- 172.21.15.78:0/4247193757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a81a7940 con 0x7fa2a8101cc0 2024-09-11T01:11:58.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2a5ffb640 1 -- 172.21.15.78:0/4247193757 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa298004490 con 0x7fa2a8101cc0 2024-09-11T01:11:58.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2a5ffb640 1 -- 172.21.15.78:0/4247193757 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298035070 con 0x7fa2a8101cc0 2024-09-11T01:11:58.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.325+0000 7fa2aef6b640 1 -- 172.21.15.78:0/4247193757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81a7e70 con 0x7fa2a8101cc0 2024-09-11T01:11:58.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.328+0000 7fa2b1340740 1 Processor -- start 2024-09-11T01:11:58.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.328+0000 7fa2b1340740 1 -- start start 2024-09-11T01:11:58.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.328+0000 7fa2b1340740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aee480 0x55c6e3aee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.328+0000 7fa2b1340740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aeeda0 0x55c6e3afa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.328+0000 7fa2b1340740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3afad30 0x55c6e3afd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.328+0000 7fa2b1340740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c6e3a64e50 con 0x55c6e3afad30 2024-09-11T01:11:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.328+0000 7fa2b1340740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c6e3a54280 con 0x55c6e3aee480 2024-09-11T01:11:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.328+0000 7fa2b1340740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c6e3a58090 con 0x55c6e3aeeda0 2024-09-11T01:11:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.328+0000 7fa2ad508640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aee480 0x55c6e3aee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.328+0000 7fa2ad508640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aee480 0x55c6e3aee860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:40242/0 (socket says 172.21.15.78:40242) 2024-09-11T01:11:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.328+0000 7fa2ad508640 1 -- 172.21.15.78:0/326125542 learned_addr learned my addr 172.21.15.78:0/326125542 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.328+0000 7fa2a7fff640 1 --2- 172.21.15.78:0/326125542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3afad30 0x55c6e3afd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.329+0000 7fa2ad508640 1 -- 172.21.15.78:0/326125542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aeeda0 msgr2=0x55c6e3afa7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:11:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.329+0000 7fa2ad508640 1 --2- 172.21.15.78:0/326125542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aeeda0 0x55c6e3afa7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.329+0000 7fa2ad508640 1 -- 172.21.15.78:0/326125542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3afad30 msgr2=0x55c6e3afd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.329+0000 7fa2ad508640 1 --2- 172.21.15.78:0/326125542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3afad30 0x55c6e3afd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.329+0000 7fa2ad508640 1 -- 172.21.15.78:0/326125542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e37b2650 con 0x55c6e3aee480 2024-09-11T01:11:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.329+0000 7fa2a5ffb640 1 -- 172.21.15.78:0/4247193757 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa29803ba30 con 0x7fa2a8101cc0 2024-09-11T01:11:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.329+0000 7fa2ad508640 1 --2- 172.21.15.78:0/326125542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aee480 0x55c6e3aee860 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009fd0 tx=0x7fa2a00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89877f4a17500201 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.329+0000 7fa29effd640 1 -- 172.21.15.78:0/326125542 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0004d20 con 0x55c6e3aee480 2024-09-11T01:11:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.329+0000 7fa29effd640 1 -- 172.21.15.78:0/326125542 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a0004050 con 0x55c6e3aee480 2024-09-11T01:11:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.329+0000 7fa29effd640 1 -- 172.21.15.78:0/326125542 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0005280 con 0x55c6e3aee480 2024-09-11T01:11:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.330+0000 7fa2a5ffb640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa2940b5450 0x7fa2940b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.330+0000 7fa2acd07640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa2940b5450 0x7fa2940b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.330+0000 7fa2a5ffb640 1 -- 172.21.15.78:0/4247193757 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7fa2981064f0 con 0x7fa2a8101cc0 2024-09-11T01:11:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.330+0000 7fa2b1340740 1 -- 172.21.15.78:0/326125542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aee480 msgr2=0x55c6e3aee860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.330+0000 7fa2b1340740 1 --2- 172.21.15.78:0/326125542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aee480 0x55c6e3aee860 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009fd0 tx=0x7fa2a00118d0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.330+0000 7fa2acd07640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa2940b5450 0x7fa2940b7910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55c6e3aed190 tx=0x7fa298098c40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.330+0000 7fa2b1340740 1 -- 172.21.15.78:0/326125542 shutdown_connections 2024-09-11T01:11:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.330+0000 7fa2b1340740 1 --2- 172.21.15.78:0/326125542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3afad30 0x55c6e3afd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.330+0000 7fa2b1340740 1 --2- 172.21.15.78:0/326125542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aeeda0 0x55c6e3afa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.330+0000 7fa2b1340740 1 --2- 172.21.15.78:0/326125542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aee480 0x55c6e3aee860 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.330+0000 7fa2b1340740 1 -- 172.21.15.78:0/326125542 >> 172.21.15.78:0/326125542 conn(0x55c6e3aec2b0 msgr2=0x55c6e3aed450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.330+0000 7fa2b1340740 1 -- 172.21.15.78:0/326125542 shutdown_connections 2024-09-11T01:11:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.331+0000 7fa2b1340740 1 -- 172.21.15.78:0/326125542 wait complete. 2024-09-11T01:11:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.331+0000 7fa2b1340740 1 -- start start 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.331+0000 7fa2b1340740 1 Processor -- start 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2b1340740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aee480 0x55c6e3afa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2b1340740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aeeda0 0x55c6e3af7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2b1340740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3afad30 0x55c6e3af8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2b1340740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c6e38b5700 con 0x55c6e3aee480 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2b1340740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c6e39ef110 con 0x55c6e3aeeda0 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2b1340740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c6e3afd9b0 con 0x55c6e3afad30 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2ad508640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aee480 0x55c6e3afa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2b1340740 10 client.0 _ll_register_callbacks cb 0x55c6e3a08990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2ad508640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aee480 0x55c6e3afa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52300/0 (socket says 172.21.15.78:52300) 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aeeda0 0x55c6e3af7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2ad508640 1 -- 172.21.15.78:0/3137287275 learned_addr learned my addr 172.21.15.78:0/3137287275 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2ad508640 1 --2- 172.21.15.78:0/3137287275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3afad30 0x55c6e3af8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2a7fff640 1 -- 172.21.15.78:0/3137287275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3afad30 msgr2=0x55c6e3af8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2a7fff640 1 --2- 172.21.15.78:0/3137287275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3afad30 0x55c6e3af8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2a7fff640 1 -- 172.21.15.78:0/3137287275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aee480 msgr2=0x55c6e3afa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2a7fff640 1 --2- 172.21.15.78:0/3137287275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aee480 0x55c6e3afa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.332+0000 7fa2a7fff640 1 -- 172.21.15.78:0/3137287275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e3a1cc40 con 0x55c6e3aeeda0 2024-09-11T01:11:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.333+0000 7fa2a7fff640 1 --2- 172.21.15.78:0/3137287275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aeeda0 0x55c6e3af7a50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa2a81073a0 tx=0x7fa29000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c899d454b4275ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.333+0000 7fa287fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:11:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.333+0000 7fa287fff640 1 -- 172.21.15.78:0/3137287275 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290012110 con 0x55c6e3aeeda0 2024-09-11T01:11:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.333+0000 7fa287fff640 1 -- 172.21.15.78:0/3137287275 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7fa290004050 con 0x55c6e3aeeda0 2024-09-11T01:11:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.333+0000 7fa2b1340740 1 -- 172.21.15.78:0/3137287275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a0004050 con 0x55c6e3aeeda0 2024-09-11T01:11:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.333+0000 7fa287fff640 1 -- 172.21.15.78:0/3137287275 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fa2900122b0 con 0x55c6e3aeeda0 2024-09-11T01:11:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.333+0000 7fa2b1340740 10 client.27122 fetch_fsmap learned FSMap version 53 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.333+0000 7fa2b1340740 1 -- 172.21.15.78:0/3137287275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55c6e3b093d0 con 0x55c6e3aeeda0 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.333+0000 7fa287fff640 1 -- 172.21.15.78:0/3137287275 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a0004050 con 0x55c6e3aeeda0 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.333+0000 7fa287fff640 1 -- 172.21.15.78:0/3137287275 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 53) ==== 8378+0+0 (secure 0 0 0) 0x7fa290010430 con 0x55c6e3aeeda0 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.334+0000 7fa2b1340740 10 client.27122 fetch_fsmap finished waiting for FSMap version 53 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.334+0000 7fa2b1340740 10 client.27122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.334+0000 7fa2b1340740 20 client.27122 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.334+0000 7fa2b1340740 1 --2- 172.21.15.78:0/3137287275 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c6e3b0a890 0x55c6e3b0cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.334+0000 7fa2b1340740 4 client.27122 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.334+0000 7fa2b1340740 1 -- 172.21.15.78:0/3137287275 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6e3a66da0 con 0x55c6e3b0a890 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.334+0000 7fa2ad508640 1 --2- 172.21.15.78:0/3137287275 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c6e3b0a890 0x55c6e3b0cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.334+0000 7fa287fff640 10 client.27122 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.334+0000 7fa2ad508640 1 --2- 172.21.15.78:0/3137287275 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c6e3b0a890 0x55c6e3b0cce0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56f0bffb53d690fa server_cookie=ab2bf70e7a249276 in_seq=0 out_seq=0 2024-09-11T01:11:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa287fff640 1 -- 172.21.15.78:0/3137287275 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c6e3a66da0 con 0x55c6e3b0a890 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa287fff640 10 client.27122 handle_command_reply: tid=0 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa2b1340740 1 client.27122 shutdown 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa2b1340740 1 -- 172.21.15.78:0/3137287275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aeeda0 msgr2=0x55c6e3af7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa2b1340740 1 --2- 172.21.15.78:0/3137287275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aeeda0 0x55c6e3af7a50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa2a81073a0 tx=0x7fa29000f790 comp rx=0 tx=0).stop 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa2b1340740 1 -- 172.21.15.78:0/3137287275 shutdown_connections 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa2b1340740 1 --2- 172.21.15.78:0/3137287275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3afad30 0x55c6e3af8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa2b1340740 1 --2- 172.21.15.78:0/3137287275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3aeeda0 0x55c6e3af7a50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa2b1340740 1 --2- 172.21.15.78:0/3137287275 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c6e3b0a890 0x55c6e3b0cce0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa2b1340740 1 --2- 172.21.15.78:0/3137287275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aee480 0x55c6e3afa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa287fff640 0 client.27122 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa2b1340740 1 -- 172.21.15.78:0/3137287275 >> 172.21.15.78:0/3137287275 conn(0x55c6e3aec2b0 msgr2=0x55c6e3af1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa2b1340740 1 -- 172.21.15.78:0/3137287275 shutdown_connections 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa2b1340740 1 -- 172.21.15.78:0/3137287275 wait complete. 2024-09-11T01:11:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.336+0000 7fa2b1340740 20 client.27122 trim_cache size 0 max 16384 2024-09-11T01:11:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.354+0000 7fa2b1340740 1 Processor -- start 2024-09-11T01:11:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.354+0000 7fa2b1340740 1 -- start start 2024-09-11T01:11:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2b1340740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aee480 0x55c6e3afc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2b1340740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aeeda0 0x55c6e3afc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2b1340740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3afad30 0x55c6e3af4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2b1340740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81b07c0 con 0x55c6e3aee480 2024-09-11T01:11:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2b1340740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81b2360 con 0x55c6e3afad30 2024-09-11T01:11:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2b1340740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81b0e90 con 0x55c6e3aeeda0 2024-09-11T01:11:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2ad508640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aee480 0x55c6e3afc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aeeda0 0x55c6e3afc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aeeda0 0x55c6e3afc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54636/0 (socket says 172.21.15.78:54636) 2024-09-11T01:11:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2ad508640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aee480 0x55c6e3afc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52316/0 (socket says 172.21.15.78:52316) 2024-09-11T01:11:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2a7fff640 1 -- 172.21.15.78:0/2624936963 learned_addr learned my addr 172.21.15.78:0/2624936963 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2ad508640 1 --2- 172.21.15.78:0/2624936963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3afad30 0x55c6e3af4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2a7fff640 1 -- 172.21.15.78:0/2624936963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3afad30 msgr2=0x55c6e3af4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2a7fff640 1 --2- 172.21.15.78:0/2624936963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3afad30 0x55c6e3af4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2a7fff640 1 -- 172.21.15.78:0/2624936963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aee480 msgr2=0x55c6e3afc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2a7fff640 1 --2- 172.21.15.78:0/2624936963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aee480 0x55c6e3afc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2a7fff640 1 -- 172.21.15.78:0/2624936963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e3aeebd0 con 0x55c6e3aeeda0 2024-09-11T01:11:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2ad508640 1 --2- 172.21.15.78:0/2624936963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3afad30 0x55c6e3af4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:11:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.355+0000 7fa2a7fff640 1 --2- 172.21.15.78:0/2624936963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aeeda0 0x55c6e3afc660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa290005970 tx=0x7fa2900059a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98eba8c7cb7f8a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.356+0000 7fa29e7fc640 1 -- 172.21.15.78:0/2624936963 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290029070 con 0x55c6e3aeeda0 2024-09-11T01:11:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.356+0000 7fa29e7fc640 1 -- 172.21.15.78:0/2624936963 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa290005560 con 0x55c6e3aeeda0 2024-09-11T01:11:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.356+0000 7fa29e7fc640 1 -- 172.21.15.78:0/2624936963 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290003720 con 0x55c6e3aeeda0 2024-09-11T01:11:58.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.356+0000 7fa2b1340740 1 -- 172.21.15.78:0/2624936963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aeeda0 msgr2=0x55c6e3afc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:58.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.356+0000 7fa2b1340740 1 --2- 172.21.15.78:0/2624936963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aeeda0 0x55c6e3afc660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa290005970 tx=0x7fa2900059a0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.356+0000 7fa2b1340740 1 -- 172.21.15.78:0/2624936963 shutdown_connections 2024-09-11T01:11:58.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.356+0000 7fa2b1340740 1 --2- 172.21.15.78:0/2624936963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3afad30 0x55c6e3af4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.356+0000 7fa2b1340740 1 --2- 172.21.15.78:0/2624936963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aeeda0 0x55c6e3afc660 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.356+0000 7fa2b1340740 1 --2- 172.21.15.78:0/2624936963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aee480 0x55c6e3afc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.356+0000 7fa2b1340740 1 -- 172.21.15.78:0/2624936963 >> 172.21.15.78:0/2624936963 conn(0x55c6e3aec2b0 msgr2=0x55c6e3af62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:58.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.357+0000 7fa2b1340740 1 -- 172.21.15.78:0/2624936963 shutdown_connections 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.357+0000 7fa2b1340740 1 -- 172.21.15.78:0/2624936963 wait complete. 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.357+0000 7fa2b1340740 1 -- start start 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.357+0000 7fa2b1340740 1 Processor -- start 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.357+0000 7fa2b1340740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aee480 0x55c6e3b02ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.357+0000 7fa2b1340740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aeeda0 0x55c6e3b03570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.357+0000 7fa2b1340740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3afad30 0x55c6e3afdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.357+0000 7fa2b1340740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81b1c90 con 0x55c6e3aeeda0 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.357+0000 7fa2b1340740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81b1590 con 0x55c6e3afad30 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.357+0000 7fa2b1340740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81ae7c0 con 0x55c6e3aee480 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.357+0000 7fa2b1340740 10 client.0 _ll_register_callbacks cb 0x7fa290009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aeeda0 0x55c6e3b03570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2ad508640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aee480 0x55c6e3b02ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aeeda0 0x55c6e3b03570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52330/0 (socket says 172.21.15.78:52330) 2024-09-11T01:11:58.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2ad508640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aee480 0x55c6e3b02ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54646/0 (socket says 172.21.15.78:54646) 2024-09-11T01:11:58.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2a7fff640 1 -- 172.21.15.78:0/291274122 learned_addr learned my addr 172.21.15.78:0/291274122 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:11:58.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2ad508640 1 --2- 172.21.15.78:0/291274122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3afad30 0x55c6e3afdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2ad508640 1 -- 172.21.15.78:0/291274122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3afad30 msgr2=0x55c6e3afdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:58.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2ad508640 1 --2- 172.21.15.78:0/291274122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3afad30 0x55c6e3afdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2ad508640 1 -- 172.21.15.78:0/291274122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aeeda0 msgr2=0x55c6e3b03570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:11:58.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2ad508640 1 --2- 172.21.15.78:0/291274122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aeeda0 0x55c6e3b03570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2ad508640 1 -- 172.21.15.78:0/291274122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e3afe580 con 0x55c6e3aee480 2024-09-11T01:11:58.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2ad508640 1 --2- 172.21.15.78:0/291274122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aee480 0x55c6e3b02ff0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009de0 tx=0x7fa2a0013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f137f8c760fe53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:11:58.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa287fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa287fff640 1 -- 172.21.15.78:0/291274122 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0004250 con 0x55c6e3aee480 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa287fff640 1 -- 172.21.15.78:0/291274122 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7fa2a00043f0 con 0x55c6e3aee480 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2b1340740 1 -- 172.21.15.78:0/291274122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa290005560 con 0x55c6e3aee480 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa287fff640 1 -- 172.21.15.78:0/291274122 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fa2a00045f0 con 0x55c6e3aee480 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2b1340740 10 client.27061 fetch_fsmap learned FSMap version 53 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.358+0000 7fa2b1340740 1 -- 172.21.15.78:0/291274122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55c6e3b78390 con 0x55c6e3aee480 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.359+0000 7fa287fff640 1 -- 172.21.15.78:0/291274122 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa290005560 con 0x55c6e3aee480 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.359+0000 7fa287fff640 1 -- 172.21.15.78:0/291274122 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 53) ==== 8378+0+0 (secure 0 0 0) 0x7fa2a0025040 con 0x55c6e3aee480 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.359+0000 7fa2b1340740 10 client.27061 fetch_fsmap finished waiting for FSMap version 53 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.359+0000 7fa2b1340740 10 client.27061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.359+0000 7fa2b1340740 20 client.27061 populate_metadata read hostname 'smithi078' 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.359+0000 7fa2b1340740 1 --2- 172.21.15.78:0/291274122 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c6e3b0a890 0x55c6e3b7a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:11:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.359+0000 7fa2b1340740 4 client.27061 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:11:58.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.359+0000 7fa2b1340740 1 -- 172.21.15.78:0/291274122 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c6e3a62850 con 0x55c6e3b0a890 2024-09-11T01:11:58.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.359+0000 7fa2a7fff640 1 --2- 172.21.15.78:0/291274122 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c6e3b0a890 0x55c6e3b7a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:11:58.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.360+0000 7fa287fff640 10 client.27061 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:58.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.360+0000 7fa2a7fff640 1 --2- 172.21.15.78:0/291274122 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c6e3b0a890 0x55c6e3b7a890 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e19e75334dd5ba3 server_cookie=b1d4f5f8617006ef in_seq=0 out_seq=0 2024-09-11T01:11:58.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa287fff640 1 -- 172.21.15.78:0/291274122 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x55c6e3a62850 con 0x55c6e3b0a890 2024-09-11T01:11:58.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa287fff640 10 client.27061 handle_command_reply: tid=0 2024-09-11T01:11:58.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa2b1340740 1 client.27061 shutdown 2024-09-11T01:11:58.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa2b1340740 1 -- 172.21.15.78:0/291274122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aee480 msgr2=0x55c6e3b02ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa2b1340740 1 --2- 172.21.15.78:0/291274122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aee480 0x55c6e3b02ff0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009de0 tx=0x7fa2a0013040 comp rx=0 tx=0).stop 2024-09-11T01:11:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa2b1340740 1 -- 172.21.15.78:0/291274122 shutdown_connections 2024-09-11T01:11:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa2b1340740 1 --2- 172.21.15.78:0/291274122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6e3afad30 0x55c6e3afdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa2b1340740 1 --2- 172.21.15.78:0/291274122 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c6e3b0a890 0x55c6e3b7a890 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa2b1340740 1 --2- 172.21.15.78:0/291274122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6e3aeeda0 0x55c6e3b03570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa2b1340740 1 --2- 172.21.15.78:0/291274122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6e3aee480 0x55c6e3b02ff0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa287fff640 0 client.27061 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:11:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa2b1340740 1 -- 172.21.15.78:0/291274122 >> 172.21.15.78:0/291274122 conn(0x55c6e3aec2b0 msgr2=0x55c6e3af14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa2b1340740 1 -- 172.21.15.78:0/291274122 shutdown_connections 2024-09-11T01:11:58.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa2b1340740 1 -- 172.21.15.78:0/291274122 wait complete. 2024-09-11T01:11:58.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.365+0000 7fa2b1340740 20 client.27061 trim_cache size 0 max 16384 2024-09-11T01:11:58.371 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:11:58.371 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:11:58.371 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:11:58.371 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:11:58.371 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:11:58.371 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:11:58.371 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.372 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.372 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.372 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.372 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.372 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:11:58.372 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:11:58.372 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.372 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.372 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.372 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.373 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.373 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:11:58.373 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:11:58.373 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.373 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.373 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.373 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.373 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.374 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:11:58.374 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:11:58.374 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.374 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.374 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.374 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.374 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.374 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:11:58.374 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.376 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:11:58.376 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:11:58.376 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.376 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.376 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.376 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.376 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.377 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:11:58.377 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:11:58.377 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.377 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.377 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.377 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.377 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.377 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:11:58.377 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.379 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:11:58.379 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:11:58.379 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.379 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.379 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.379 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.379 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.379 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:11:58.379 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:11:58.380 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.380 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.380 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.380 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.380 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.380 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:11:58.381 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:11:58.381 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.381 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.381 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.381 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.381 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.381 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:11:58.381 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:11:58.381 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.381 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.382 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:11:58.383 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:11:58.383 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:11:58.383 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:11:58.383 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:11:58.383 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:11:58.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:11:58.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:11:58.384 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:11:58.384 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:11:58.384 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:11:58.384 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:11:58.384 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:11:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.366+0000 7fa2aef6b640 1 -- 172.21.15.78:0/4247193757 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa2940b5450 msgr2=0x7fa2940b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.366+0000 7fa2aef6b640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa2940b5450 0x7fa2940b7910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55c6e3aed190 tx=0x7fa298098c40 comp rx=0 tx=0).stop 2024-09-11T01:11:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.366+0000 7fa2aef6b640 1 -- 172.21.15.78:0/4247193757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a8101cc0 msgr2=0x7fa2a8197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:11:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.366+0000 7fa2aef6b640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a8101cc0 0x7fa2a8197ed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa29802a930 tx=0x7fa298031a20 comp rx=0 tx=0).stop 2024-09-11T01:11:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.366+0000 7fa2aef6b640 1 -- 172.21.15.78:0/4247193757 shutdown_connections 2024-09-11T01:11:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.366+0000 7fa2aef6b640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa2940b5450 0x7fa2940b7910 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.366+0000 7fa2aef6b640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2a8108050 0x7fa2a8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.366+0000 7fa2aef6b640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2a81076d0 0x7fa2a8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.366+0000 7fa2aef6b640 1 --2- 172.21.15.78:0/4247193757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a8101cc0 0x7fa2a8197ed0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:11:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.366+0000 7fa2aef6b640 1 -- 172.21.15.78:0/4247193757 >> 172.21.15.78:0/4247193757 conn(0x7fa2a80fd750 msgr2=0x7fa2a80ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:11:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.366+0000 7fa2aef6b640 1 -- 172.21.15.78:0/4247193757 shutdown_connections 2024-09-11T01:11:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:11:58.366+0000 7fa2aef6b640 1 -- 172.21.15.78:0/4247193757 wait complete. 2024-09-11T01:11:58.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 2177 68.02 MB/sec warmup 4 sec latency 68.764 ms 2024-09-11T01:11:58.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:58 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:58.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:58 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:58.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:58 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:11:59.415 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:11:59.415 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:11:59.707 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:11:59.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:11:59 smithi078 ceph-mon[31315]: pgmap v728: 97 pgs: 97 active+clean; 789 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 195 op/s 2024-09-11T01:11:59.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 2682 62.93 MB/sec warmup 5 sec latency 104.377 ms 2024-09-11T01:11:59.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:11:59 smithi110 ceph-mon[34712]: pgmap v728: 97 pgs: 97 active+clean; 789 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 195 op/s 2024-09-11T01:11:59.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:11:59 smithi112 ceph-mon[35384]: pgmap v728: 97 pgs: 97 active+clean; 789 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 195 op/s 2024-09-11T01:12:00.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.279+0000 7fa828892640 1 -- 172.21.15.78:0/1767520992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824108020 msgr2=0x7fa82410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.279+0000 7fa828892640 1 --2- 172.21.15.78:0/1767520992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824108020 0x7fa82410e820 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009ca0 tx=0x7fa81c02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:12:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.279+0000 7fa828892640 1 -- 172.21.15.78:0/1767520992 shutdown_connections 2024-09-11T01:12:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.279+0000 7fa828892640 1 --2- 172.21.15.78:0/1767520992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824108020 0x7fa82410e820 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.279+0000 7fa828892640 1 --2- 172.21.15.78:0/1767520992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8241076a0 0x7fa824107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.279+0000 7fa828892640 1 --2- 172.21.15.78:0/1767520992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa824101c90 0x7fa824102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.279+0000 7fa828892640 1 -- 172.21.15.78:0/1767520992 >> 172.21.15.78:0/1767520992 conn(0x7fa8240fd700 msgr2=0x7fa8240ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.280+0000 7fa828892640 1 -- 172.21.15.78:0/1767520992 shutdown_connections 2024-09-11T01:12:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.280+0000 7fa828892640 1 -- 172.21.15.78:0/1767520992 wait complete. 2024-09-11T01:12:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.280+0000 7fa828892640 1 Processor -- start 2024-09-11T01:12:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.280+0000 7fa828892640 1 -- start start 2024-09-11T01:12:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.280+0000 7fa828892640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824101c90 0x7fa824199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.280+0000 7fa828892640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8241076a0 0x7fa824199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.280+0000 7fa828892640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa824108020 0x7fa82419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa828892640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa824111ce0 con 0x7fa824101c90 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa828892640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa824111b60 con 0x7fa824108020 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa828892640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa824111e60 con 0x7fa8241076a0 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa82259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824101c90 0x7fa824199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa82259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824101c90 0x7fa824199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52354/0 (socket says 172.21.15.78:52354) 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa82259c640 1 -- 172.21.15.78:0/2676808720 learned_addr learned my addr 172.21.15.78:0/2676808720 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa821d9b640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8241076a0 0x7fa824199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa822d9d640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa824108020 0x7fa82419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa82259c640 1 -- 172.21.15.78:0/2676808720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8241076a0 msgr2=0x7fa824199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa82259c640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8241076a0 0x7fa824199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa82259c640 1 -- 172.21.15.78:0/2676808720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa824108020 msgr2=0x7fa82419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa82259c640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa824108020 0x7fa82419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa82259c640 1 -- 172.21.15.78:0/2676808720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82419f2f0 con 0x7fa824101c90 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa821d9b640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8241076a0 0x7fa824199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa82259c640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824101c90 0x7fa824199240 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa81400e9f0 tx=0x7fa81400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa8137fe640 1 -- 172.21.15.78:0/2676808720 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81400cd80 con 0x7fa824101c90 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa8137fe640 1 -- 172.21.15.78:0/2676808720 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8140054d0 con 0x7fa824101c90 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa8137fe640 1 -- 172.21.15.78:0/2676808720 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814010690 con 0x7fa824101c90 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa828892640 1 -- 172.21.15.78:0/2676808720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82419f5e0 con 0x7fa824101c90 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.281+0000 7fa828892640 1 -- 172.21.15.78:0/2676808720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8241a5ca0 con 0x7fa824101c90 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.283+0000 7fa828892640 1 -- 172.21.15.78:0/2676808720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa824102070 con 0x7fa824101c90 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.284+0000 7fa8137fe640 1 -- 172.21.15.78:0/2676808720 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa814004e90 con 0x7fa824101c90 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.284+0000 7fa8137fe640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa80c0b5450 0x7fa80c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.284+0000 7fa821d9b640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa80c0b5450 0x7fa80c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.285+0000 7fa8137fe640 1 -- 172.21.15.78:0/2676808720 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7fa8140e6410 con 0x7fa824101c90 2024-09-11T01:12:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.285+0000 7fa821d9b640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa80c0b5450 0x7fa80c0b7910 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa824106e40 tx=0x7fa818005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:00.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.289+0000 7fa8137fe640 1 -- 172.21.15.78:0/2676808720 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa8140acf70 con 0x7fa824101c90 2024-09-11T01:12:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.565+0000 7fa828892640 1 -- 172.21.15.78:0/2676808720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8241a6000 con 0x7fa824101c90 2024-09-11T01:12:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.566+0000 7fa8137fe640 1 -- 172.21.15.78:0/2676808720 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11989 (secure 0 0 0) 0x7fa8140ac870 con 0x7fa824101c90 2024-09-11T01:12:00.568 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:12:00.569 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":55,"btime":"2024-09-11T01:12:00:510721+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:11:59.514604+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T01:12:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 55 2024-09-11T01:12:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.570+0000 7fa828892640 1 -- 172.21.15.78:0/2676808720 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa80c0b5450 msgr2=0x7fa80c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.570+0000 7fa828892640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa80c0b5450 0x7fa80c0b7910 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa824106e40 tx=0x7fa818005f90 comp rx=0 tx=0).stop 2024-09-11T01:12:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.570+0000 7fa828892640 1 -- 172.21.15.78:0/2676808720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824101c90 msgr2=0x7fa824199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.570+0000 7fa828892640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824101c90 0x7fa824199240 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa81400e9f0 tx=0x7fa81400edf0 comp rx=0 tx=0).stop 2024-09-11T01:12:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.570+0000 7fa828892640 1 -- 172.21.15.78:0/2676808720 shutdown_connections 2024-09-11T01:12:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.570+0000 7fa828892640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa80c0b5450 0x7fa80c0b7910 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.570+0000 7fa828892640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa824108020 0x7fa82419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.570+0000 7fa828892640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8241076a0 0x7fa824199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.570+0000 7fa828892640 1 --2- 172.21.15.78:0/2676808720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa824101c90 0x7fa824199240 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.570+0000 7fa828892640 1 -- 172.21.15.78:0/2676808720 >> 172.21.15.78:0/2676808720 conn(0x7fa8240fd700 msgr2=0x7fa82406aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.571+0000 7fa828892640 1 -- 172.21.15.78:0/2676808720 shutdown_connections 2024-09-11T01:12:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:00.571+0000 7fa828892640 1 -- 172.21.15.78:0/2676808720 wait complete. 2024-09-11T01:12:00.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 3121 56.61 MB/sec warmup 6 sec latency 96.331 ms 2024-09-11T01:12:00.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:00 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:00.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:00 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:00.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:00 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:01.285 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:12:01.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:01 smithi078 ceph-mon[31315]: pgmap v729: 97 pgs: 97 active+clean; 793 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 200 op/s 2024-09-11T01:12:01.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:01 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:01.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:01 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2676808720' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:12:01.552 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:12:01.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 3539 53.77 MB/sec warmup 7 sec latency 60.688 ms 2024-09-11T01:12:01.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:01 smithi110 ceph-mon[34712]: pgmap v729: 97 pgs: 97 active+clean; 793 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 200 op/s 2024-09-11T01:12:01.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:01 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:01.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:01 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2676808720' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:12:01.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:01 smithi112 ceph-mon[35384]: pgmap v729: 97 pgs: 97 active+clean; 793 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 200 op/s 2024-09-11T01:12:01.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:01 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:01.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:01 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2676808720' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:12:02.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.075+0000 7fd9e991a640 1 -- 172.21.15.78:0/2891113018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e4109150 msgr2=0x7fd9e410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:02.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.075+0000 7fd9e991a640 1 --2- 172.21.15.78:0/2891113018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e4109150 0x7fd9e410f950 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009ca0 tx=0x7fd9dc02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.075+0000 7fd9e991a640 1 -- 172.21.15.78:0/2891113018 shutdown_connections 2024-09-11T01:12:02.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.075+0000 7fd9e991a640 1 --2- 172.21.15.78:0/2891113018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e4109150 0x7fd9e410f950 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.075+0000 7fd9e991a640 1 --2- 172.21.15.78:0/2891113018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e41087d0 0x7fd9e4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.075+0000 7fd9e991a640 1 --2- 172.21.15.78:0/2891113018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e4102dc0 0x7fd9e41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.075+0000 7fd9e991a640 1 -- 172.21.15.78:0/2891113018 >> 172.21.15.78:0/2891113018 conn(0x7fd9e40fe830 msgr2=0x7fd9e4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.075+0000 7fd9e991a640 1 -- 172.21.15.78:0/2891113018 shutdown_connections 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.075+0000 7fd9e991a640 1 -- 172.21.15.78:0/2891113018 wait complete. 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e991a640 1 Processor -- start 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e991a640 1 -- start start 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e991a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e4102dc0 0x7fd9e41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e991a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e41087d0 0x7fd9e41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e991a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e4109150 0x7fd9e419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e991a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4112e60 con 0x7fd9e4102dc0 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e991a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4112ce0 con 0x7fd9e41087d0 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e991a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4112fe0 con 0x7fd9e4109150 2024-09-11T01:12:02.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e4102dc0 0x7fd9e41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e41087d0 0x7fd9e41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e41087d0 0x7fd9e41997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:40302/0 (socket says 172.21.15.78:40302) 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e4102dc0 0x7fd9e41992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52376/0 (socket says 172.21.15.78:52376) 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.076+0000 7fd9e27fc640 1 -- 172.21.15.78:0/2531684890 learned_addr learned my addr 172.21.15.78:0/2531684890 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e37fe640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e4109150 0x7fd9e419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e27fc640 1 -- 172.21.15.78:0/2531684890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e4109150 msgr2=0x7fd9e419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e4109150 0x7fd9e419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e27fc640 1 -- 172.21.15.78:0/2531684890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e4102dc0 msgr2=0x7fd9e41992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e4102dc0 0x7fd9e41992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e27fc640 1 -- 172.21.15.78:0/2531684890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e419f360 con 0x7fd9e41087d0 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e37fe640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e4109150 0x7fd9e419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e2ffd640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e4102dc0 0x7fd9e41992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e41087d0 0x7fd9e41997f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800e9f0 tx=0x7fd9d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:02.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e8918640 1 -- 172.21.15.78:0/2531684890 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800cd80 con 0x7fd9e41087d0 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e991a640 1 -- 172.21.15.78:0/2531684890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e419f650 con 0x7fd9e41087d0 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e8918640 1 -- 172.21.15.78:0/2531684890 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d80108e0 con 0x7fd9e41087d0 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.077+0000 7fd9e8918640 1 -- 172.21.15.78:0/2531684890 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8018980 con 0x7fd9e41087d0 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.078+0000 7fd9e991a640 1 -- 172.21.15.78:0/2531684890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e41a5d10 con 0x7fd9e41087d0 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.080+0000 7fd9ebcef740 1 Processor -- start 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.080+0000 7fd9ebcef740 1 -- start start 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.080+0000 7fd9e8918640 1 -- 172.21.15.78:0/2531684890 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd9d8010430 con 0x7fd9e41087d0 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.080+0000 7fd9ebcef740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 0x55c61d1f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.080+0000 7fd9ebcef740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d204460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.080+0000 7fd9ebcef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d2049a0 0x55c61d206d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.080+0000 7fd9ebcef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c61d16eea0 con 0x55c61d2049a0 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.080+0000 7fd9ebcef740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c61d15e2d0 con 0x55c61d1f8a10 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.080+0000 7fd9ebcef740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c61d1620e0 con 0x55c61d1f80f0 2024-09-11T01:12:02.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d204460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 0x55c61d1f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d204460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:40308/0 (socket says 172.21.15.78:40308) 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e2ffd640 1 -- 172.21.15.78:0/3044698920 learned_addr learned my addr 172.21.15.78:0/3044698920 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/3044698920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d2049a0 0x55c61d206d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e37fe640 1 -- 172.21.15.78:0/3044698920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 msgr2=0x55c61d204460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e37fe640 1 --2- 172.21.15.78:0/3044698920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d204460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e37fe640 1 -- 172.21.15.78:0/3044698920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d2049a0 msgr2=0x55c61d206d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e37fe640 1 --2- 172.21.15.78:0/3044698920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d2049a0 0x55c61d206d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e37fe640 1 -- 172.21.15.78:0/3044698920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61cebc650 con 0x55c61d1f80f0 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/3044698920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d2049a0 0x55c61d206d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e37fe640 1 --2- 172.21.15.78:0/3044698920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 0x55c61d1f84d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc006010 tx=0x7fd9dc009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b686bee5b6b1f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e8918640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd9cc0b5450 0x7fd9cc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9c9ffb640 1 -- 172.21.15.78:0/3044698920 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc031cf0 con 0x55c61d1f80f0 2024-09-11T01:12:02.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9c9ffb640 1 -- 172.21.15.78:0/3044698920 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9dc036040 con 0x55c61d1f80f0 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9c9ffb640 1 -- 172.21.15.78:0/3044698920 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc004240 con 0x55c61d1f80f0 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e2ffd640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd9cc0b5450 0x7fd9cc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.081+0000 7fd9e8918640 1 -- 172.21.15.78:0/2531684890 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7fd9d80e62f0 con 0x7fd9e41087d0 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.082+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3044698920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 msgr2=0x55c61d1f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.082+0000 7fd9e2ffd640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd9cc0b5450 0x7fd9cc0b7910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55c61d1f6e00 tx=0x7fd9d4005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.082+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3044698920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 0x55c61d1f84d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc006010 tx=0x7fd9dc009800 comp rx=0 tx=0).stop 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.082+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3044698920 shutdown_connections 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.082+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3044698920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d2049a0 0x55c61d206d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.082+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3044698920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d204460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.082+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3044698920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 0x55c61d1f84d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.082+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3044698920 >> 172.21.15.78:0/3044698920 conn(0x55c61d1f5f20 msgr2=0x55c61d1f70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.082+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3044698920 shutdown_connections 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.082+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3044698920 wait complete. 2024-09-11T01:12:02.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9ebcef740 1 -- start start 2024-09-11T01:12:02.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9ebcef740 1 Processor -- start 2024-09-11T01:12:02.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9ebcef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d1f80f0 0x55c61d204290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9ebcef740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d2016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9ebcef740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d2049a0 0x55c61d201d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9ebcef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c61cfbf700 con 0x55c61d1f80f0 2024-09-11T01:12:02.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9ebcef740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c61d0f9160 con 0x55c61d1f8a10 2024-09-11T01:12:02.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9ebcef740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c61d2075c0 con 0x55c61d2049a0 2024-09-11T01:12:02.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9ebcef740 10 client.0 _ll_register_callbacks cb 0x55c61d1129e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:02.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d2016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d2016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:40310/0 (socket says 172.21.15.78:40310) 2024-09-11T01:12:02.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9e27fc640 1 -- 172.21.15.78:0/2457220465 learned_addr learned my addr 172.21.15.78:0/2457220465 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:02.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.083+0000 7fd9e37fe640 1 --2- 172.21.15.78:0/2457220465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d2049a0 0x55c61d201d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9e27fc640 1 -- 172.21.15.78:0/2457220465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d2049a0 msgr2=0x55c61d201d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:02.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/2457220465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d2049a0 0x55c61d201d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9e27fc640 1 -- 172.21.15.78:0/2457220465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d1f80f0 msgr2=0x55c61d204290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:02.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/2457220465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d1f80f0 0x55c61d204290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9e27fc640 1 -- 172.21.15.78:0/2457220465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61d126c90 con 0x55c61d1f8a10 2024-09-11T01:12:02.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/2457220465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d2016e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800e9c0 tx=0x7fd9d8010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c37aae3648b97ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:02.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:12:02.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9c2ffd640 1 -- 172.21.15.78:0/2457220465 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80ac100 con 0x55c61d1f8a10 2024-09-11T01:12:02.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9c2ffd640 1 -- 172.21.15.78:0/2457220465 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7fd9d80abc20 con 0x55c61d1f8a10 2024-09-11T01:12:02.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9ebcef740 1 -- 172.21.15.78:0/2457220465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9dc036040 con 0x55c61d1f8a10 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9c2ffd640 1 -- 172.21.15.78:0/2457220465 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd9d80abf20 con 0x55c61d1f8a10 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9ebcef740 10 client.27143 fetch_fsmap learned FSMap version 55 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9ebcef740 1 -- 172.21.15.78:0/2457220465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c61d212f50 con 0x55c61d1f8a10 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.084+0000 7fd9c2ffd640 1 -- 172.21.15.78:0/2457220465 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9dc036040 con 0x55c61d1f8a10 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.085+0000 7fd9c2ffd640 1 -- 172.21.15.78:0/2457220465 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 55) ==== 8390+0+0 (secure 0 0 0) 0x7fd9d80a8100 con 0x55c61d1f8a10 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.085+0000 7fd9ebcef740 10 client.27143 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.085+0000 7fd9ebcef740 10 client.27143 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.085+0000 7fd9ebcef740 20 client.27143 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.085+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/2457220465 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c61d2144b0 0x55c61d216940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.085+0000 7fd9ebcef740 4 client.27143 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.085+0000 7fd9ebcef740 1 -- 172.21.15.78:0/2457220465 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c61d170df0 con 0x55c61d2144b0 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.085+0000 7fd9e2ffd640 1 --2- 172.21.15.78:0/2457220465 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c61d2144b0 0x55c61d216940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.085+0000 7fd9e2ffd640 1 --2- 172.21.15.78:0/2457220465 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c61d2144b0 0x55c61d216940 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42286f5bd0ad4d68 server_cookie=2b1830d455fb8a6e in_seq=0 out_seq=0 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.085+0000 7fd9c2ffd640 10 client.27143 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.086+0000 7fd9c2ffd640 1 -- 172.21.15.78:0/2457220465 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c61d170df0 con 0x55c61d2144b0 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.086+0000 7fd9c2ffd640 10 client.27143 handle_command_reply: tid=0 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.086+0000 7fd9ebcef740 1 client.27143 shutdown 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.087+0000 7fd9ebcef740 1 -- 172.21.15.78:0/2457220465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 msgr2=0x55c61d2016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.087+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/2457220465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d2016e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800e9c0 tx=0x7fd9d8010cc0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.087+0000 7fd9ebcef740 1 -- 172.21.15.78:0/2457220465 shutdown_connections 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.087+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/2457220465 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c61d2144b0 0x55c61d216940 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.087+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/2457220465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d2049a0 0x55c61d201d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.087+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/2457220465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d2016e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.087+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/2457220465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d1f80f0 0x55c61d204290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.087+0000 7fd9ebcef740 1 -- 172.21.15.78:0/2457220465 >> 172.21.15.78:0/2457220465 conn(0x55c61d1f5f20 msgr2=0x55c61d1fbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:02.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.087+0000 7fd9c2ffd640 0 client.27143 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:12:02.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.087+0000 7fd9ebcef740 1 -- 172.21.15.78:0/2457220465 shutdown_connections 2024-09-11T01:12:02.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.087+0000 7fd9ebcef740 1 -- 172.21.15.78:0/2457220465 wait complete. 2024-09-11T01:12:02.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.087+0000 7fd9ebcef740 20 client.27143 trim_cache size 0 max 16384 2024-09-11T01:12:02.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.111+0000 7fd9ebcef740 1 Processor -- start 2024-09-11T01:12:02.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.111+0000 7fd9ebcef740 1 -- start start 2024-09-11T01:12:02.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9ebcef740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f80f0 0x55c61d200bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9ebcef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d1f8a10 0x55c61d1f5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9ebcef740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d2049a0 0x55c61d1fff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9ebcef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e41ac530 con 0x55c61d1f8a10 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9ebcef740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e41ae0d0 con 0x55c61d1f80f0 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9ebcef740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e41acc00 con 0x55c61d2049a0 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d1f8a10 0x55c61d1f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d1f8a10 0x55c61d1f5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52398/0 (socket says 172.21.15.78:52398) 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9e27fc640 1 -- 172.21.15.78:0/3270668021 learned_addr learned my addr 172.21.15.78:0/3270668021 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9e2ffd640 1 --2- 172.21.15.78:0/3270668021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f80f0 0x55c61d200bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9e37fe640 1 --2- 172.21.15.78:0/3270668021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d2049a0 0x55c61d1fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9e27fc640 1 -- 172.21.15.78:0/3270668021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d2049a0 msgr2=0x55c61d1fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/3270668021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d2049a0 0x55c61d1fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9e27fc640 1 -- 172.21.15.78:0/3270668021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f80f0 msgr2=0x55c61d200bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:02.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/3270668021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f80f0 0x55c61d200bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9e27fc640 1 -- 172.21.15.78:0/3270668021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61d207120 con 0x55c61d1f8a10 2024-09-11T01:12:02.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9e2ffd640 1 --2- 172.21.15.78:0/3270668021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f80f0 0x55c61d200bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:02.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.112+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/3270668021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d1f8a10 0x55c61d1f5760 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80a8d80 tx=0x7fd9d800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79c33f801c2bdcf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:02.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.113+0000 7fd9c97fa640 1 -- 172.21.15.78:0/3270668021 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80ac100 con 0x55c61d1f8a10 2024-09-11T01:12:02.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.113+0000 7fd9c97fa640 1 -- 172.21.15.78:0/3270668021 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d80b1020 con 0x55c61d1f8a10 2024-09-11T01:12:02.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.113+0000 7fd9c97fa640 1 -- 172.21.15.78:0/3270668021 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80a8650 con 0x55c61d1f8a10 2024-09-11T01:12:02.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.113+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3270668021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d1f8a10 msgr2=0x55c61d1f5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:02.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.113+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3270668021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d1f8a10 0x55c61d1f5760 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80a8d80 tx=0x7fd9d800cf20 comp rx=0 tx=0).stop 2024-09-11T01:12:02.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.113+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3270668021 shutdown_connections 2024-09-11T01:12:02.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.113+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3270668021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d2049a0 0x55c61d1fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.113+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3270668021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d1f8a10 0x55c61d1f5760 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.113+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3270668021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f80f0 0x55c61d200bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.113+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3270668021 >> 172.21.15.78:0/3270668021 conn(0x55c61d1f5f20 msgr2=0x55c61d1fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:02.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.113+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3270668021 shutdown_connections 2024-09-11T01:12:02.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.114+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3270668021 wait complete. 2024-09-11T01:12:02.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.114+0000 7fd9ebcef740 1 -- start start 2024-09-11T01:12:02.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.114+0000 7fd9ebcef740 1 Processor -- start 2024-09-11T01:12:02.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.114+0000 7fd9ebcef740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 0x55c61d1ff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.114+0000 7fd9ebcef740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d20c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.114+0000 7fd9ebcef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d2049a0 0x55c61d20cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.114+0000 7fd9ebcef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e41ada00 con 0x55c61d2049a0 2024-09-11T01:12:02.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.114+0000 7fd9ebcef740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e41ad300 con 0x55c61d1f8a10 2024-09-11T01:12:02.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.114+0000 7fd9ebcef740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e41aa530 con 0x55c61d1f80f0 2024-09-11T01:12:02.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.114+0000 7fd9ebcef740 10 client.0 _ll_register_callbacks cb 0x7fd9d8010750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:02.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.114+0000 7fd9e2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 0x55c61d1ff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9e37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d2049a0 0x55c61d20cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d20c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9e2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 0x55c61d1ff0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54714/0 (socket says 172.21.15.78:54714) 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9e2ffd640 1 -- 172.21.15.78:0/3149535748 learned_addr learned my addr 172.21.15.78:0/3149535748 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9e27fc640 1 -- 172.21.15.78:0/3149535748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 msgr2=0x55c61d1ff0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 0x55c61d1ff0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9e27fc640 1 -- 172.21.15.78:0/3149535748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d2049a0 msgr2=0x55c61d20cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d2049a0 0x55c61d20cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9e27fc640 1 -- 172.21.15.78:0/3149535748 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61d20d500 con 0x55c61d1f8a10 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9e2ffd640 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 0x55c61d1ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9e37fe640 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d2049a0 0x55c61d20cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9e27fc640 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d20c850 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80ac740 tx=0x7fd9d80abe50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dab02ba55d6028f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:12:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9c2ffd640 1 -- 172.21.15.78:0/3149535748 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80a8650 con 0x55c61d1f8a10 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.115+0000 7fd9c2ffd640 1 -- 172.21.15.78:0/3149535748 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8449+0+0 (secure 0 0 0) 0x7fd9d80ab730 con 0x55c61d1f8a10 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.116+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3149535748 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9d80b1020 con 0x55c61d1f8a10 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.116+0000 7fd9c2ffd640 1 -- 172.21.15.78:0/3149535748 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd9d80ac100 con 0x55c61d1f8a10 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.116+0000 7fd9ebcef740 10 client.27149 fetch_fsmap learned FSMap version 55 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.116+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3149535748 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c61d2814b0 con 0x55c61d1f8a10 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.116+0000 7fd9c2ffd640 1 -- 172.21.15.78:0/3149535748 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d80b5d40 con 0x55c61d1f8a10 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.116+0000 7fd9c2ffd640 1 -- 172.21.15.78:0/3149535748 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 55) ==== 8390+0+0 (secure 0 0 0) 0x7fd9d80adae0 con 0x55c61d1f8a10 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.116+0000 7fd9ebcef740 10 client.27149 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.116+0000 7fd9ebcef740 10 client.27149 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.116+0000 7fd9ebcef740 20 client.27149 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.116+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c61d281a10 0x55c61d283e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.116+0000 7fd9ebcef740 4 client.27149 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.116+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3149535748 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd9d80b1020 con 0x55c61d281a10 2024-09-11T01:12:02.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.117+0000 7fd9e2ffd640 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c61d281a10 0x55c61d283e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:02.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.117+0000 7fd9c2ffd640 10 client.27149 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:12:02.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.117+0000 7fd9e2ffd640 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c61d281a10 0x55c61d283e30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8b8abfc0563f571 server_cookie=d30481e0cdc716fb in_seq=0 out_seq=0 2024-09-11T01:12:02.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.117+0000 7fd9c2ffd640 1 -- 172.21.15.78:0/3149535748 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd9d80b1020 con 0x55c61d281a10 2024-09-11T01:12:02.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.117+0000 7fd9c2ffd640 10 client.27149 handle_command_reply: tid=0 2024-09-11T01:12:02.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.117+0000 7fd9ebcef740 1 client.27149 shutdown 2024-09-11T01:12:02.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3149535748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 msgr2=0x55c61d20c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:02.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d20c850 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80ac740 tx=0x7fd9d80abe50 comp rx=0 tx=0).stop 2024-09-11T01:12:02.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3149535748 shutdown_connections 2024-09-11T01:12:02.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c61d281a10 0x55c61d283e30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c61d2049a0 0x55c61d20cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c61d1f8a10 0x55c61d20c850 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9c2ffd640 0 client.27149 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:12:02.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9ebcef740 1 --2- 172.21.15.78:0/3149535748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c61d1f80f0 0x55c61d1ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3149535748 >> 172.21.15.78:0/3149535748 conn(0x55c61d1f5f20 msgr2=0x55c61d1fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:02.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3149535748 shutdown_connections 2024-09-11T01:12:02.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9ebcef740 1 -- 172.21.15.78:0/3149535748 wait complete. 2024-09-11T01:12:02.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9ebcef740 20 client.27149 trim_cache size 0 max 16384 2024-09-11T01:12:02.125 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:12:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9e991a640 1 -- 172.21.15.78:0/2531684890 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd9cc0b5450 msgr2=0x7fd9cc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9e991a640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd9cc0b5450 0x7fd9cc0b7910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55c61d1f6e00 tx=0x7fd9d4005f90 comp rx=0 tx=0).stop 2024-09-11T01:12:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9e991a640 1 -- 172.21.15.78:0/2531684890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e41087d0 msgr2=0x7fd9e41997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.118+0000 7fd9e991a640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e41087d0 0x7fd9e41997f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800e9f0 tx=0x7fd9d800eeb0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.119+0000 7fd9e991a640 1 -- 172.21.15.78:0/2531684890 shutdown_connections 2024-09-11T01:12:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.119+0000 7fd9e991a640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd9cc0b5450 0x7fd9cc0b7910 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.119+0000 7fd9e991a640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e4109150 0x7fd9e419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.119+0000 7fd9e991a640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e41087d0 0x7fd9e41997f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.119+0000 7fd9e991a640 1 --2- 172.21.15.78:0/2531684890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e4102dc0 0x7fd9e41992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.119+0000 7fd9e991a640 1 -- 172.21.15.78:0/2531684890 >> 172.21.15.78:0/2531684890 conn(0x7fd9e40fe830 msgr2=0x7fd9e4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.119+0000 7fd9e991a640 1 -- 172.21.15.78:0/2531684890 shutdown_connections 2024-09-11T01:12:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:02.119+0000 7fd9e991a640 1 -- 172.21.15.78:0/2531684890 wait complete. 2024-09-11T01:12:02.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 3756 48.80 MB/sec warmup 8 sec latency 80.821 ms 2024-09-11T01:12:02.936 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:12:03.203 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:12:03.644 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:03 smithi078 ceph-mon[31315]: pgmap v730: 97 pgs: 97 active+clean; 801 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 196 op/s 2024-09-11T01:12:03.644 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:03 smithi078 ceph-mon[31315]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T01:12:03.644 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:03 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.729+0000 7fba0d544640 1 -- 172.21.15.78:0/963092746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba08102de0 msgr2=0x7fba081031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.729+0000 7fba0d544640 1 --2- 172.21.15.78:0/963092746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba08102de0 0x7fba081031c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009900 tx=0x7fb9f802a410 comp rx=0 tx=0).stop 2024-09-11T01:12:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.730+0000 7fba0d544640 1 -- 172.21.15.78:0/963092746 shutdown_connections 2024-09-11T01:12:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.730+0000 7fba0d544640 1 --2- 172.21.15.78:0/963092746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba08109170 0x7fba0810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.730+0000 7fba0d544640 1 --2- 172.21.15.78:0/963092746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba081087f0 0x7fba08108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.730+0000 7fba0d544640 1 --2- 172.21.15.78:0/963092746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba08102de0 0x7fba081031c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.730+0000 7fba0d544640 1 -- 172.21.15.78:0/963092746 >> 172.21.15.78:0/963092746 conn(0x7fba080fe830 msgr2=0x7fba08100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.730+0000 7fba0d544640 1 -- 172.21.15.78:0/963092746 shutdown_connections 2024-09-11T01:12:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.730+0000 7fba0d544640 1 -- 172.21.15.78:0/963092746 wait complete. 2024-09-11T01:12:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.730+0000 7fba0d544640 1 Processor -- start 2024-09-11T01:12:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.730+0000 7fba0d544640 1 -- start start 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba0d544640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba08102de0 0x7fba08199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba0d544640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba081087f0 0x7fba081997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba0d544640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba08109170 0x7fba0819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba0d544640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba08112df0 con 0x7fba08102de0 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba0d544640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fba08112c70 con 0x7fba081087f0 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba0d544640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba08112f70 con 0x7fba08109170 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba06ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba08102de0 0x7fba08199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba06ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba08102de0 0x7fba08199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38852/0 (socket says 172.21.15.78:38852) 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba06ffd640 1 -- 172.21.15.78:0/1861874099 learned_addr learned my addr 172.21.15.78:0/1861874099 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fb9fffff640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba081087f0 0x7fba081997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba077fe640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba08109170 0x7fba0819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba06ffd640 1 -- 172.21.15.78:0/1861874099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba08109170 msgr2=0x7fba0819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba06ffd640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba08109170 0x7fba0819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba06ffd640 1 -- 172.21.15.78:0/1861874099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba081087f0 msgr2=0x7fba081997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba06ffd640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba081087f0 0x7fba081997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba06ffd640 1 -- 172.21.15.78:0/1861874099 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0819f330 con 0x7fba08102de0 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba077fe640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba08109170 0x7fba0819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba06ffd640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba08102de0 0x7fba08199280 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8005940 tx=0x7fb9f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba04ff9640 1 -- 172.21.15.78:0/1861874099 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f803f030 con 0x7fba08102de0 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba0d544640 1 -- 172.21.15.78:0/1861874099 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0819f5c0 con 0x7fba08102de0 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba04ff9640 1 -- 172.21.15.78:0/1861874099 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f8003aa0 con 0x7fba08102de0 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba04ff9640 1 -- 172.21.15.78:0/1861874099 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8035070 con 0x7fba08102de0 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.731+0000 7fba0d544640 1 -- 172.21.15.78:0/1861874099 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba081a5ce0 con 0x7fba08102de0 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.734+0000 7fba04ff9640 1 -- 172.21.15.78:0/1861874099 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb9f8003c50 con 0x7fba08102de0 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.734+0000 7fba0f919740 1 Processor -- start 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.734+0000 7fba0f919740 1 -- start start 2024-09-11T01:12:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.734+0000 7fba0f919740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 0x55ee0f75b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.734+0000 7fba0f919740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f767840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba0f919740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f76a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba0f919740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f6d1ea0 con 0x55ee0f75b4d0 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba0f919740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f6c12d0 con 0x55ee0f75bdf0 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba0f919740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f6c50e0 con 0x55ee0f767d80 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 0x55ee0f75b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 0x55ee0f75b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38860/0 (socket says 172.21.15.78:38860) 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba077fe640 1 -- 172.21.15.78:0/2299462997 learned_addr learned my addr 172.21.15.78:0/2299462997 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba06ffd640 1 --2- 172.21.15.78:0/2299462997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f767840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fb9fffff640 1 --2- 172.21.15.78:0/2299462997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f76a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba077fe640 1 -- 172.21.15.78:0/2299462997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 msgr2=0x55ee0f76a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba077fe640 1 --2- 172.21.15.78:0/2299462997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f76a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba077fe640 1 -- 172.21.15.78:0/2299462997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 msgr2=0x55ee0f767840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba077fe640 1 --2- 172.21.15.78:0/2299462997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f767840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba077fe640 1 -- 172.21.15.78:0/2299462997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0f41f650 con 0x55ee0f75b4d0 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba06ffd640 1 --2- 172.21.15.78:0/2299462997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f767840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba077fe640 1 --2- 172.21.15.78:0/2299462997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 0x55ee0f75b8b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fba00009fd0 tx=0x7fba00010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56384aaa3a7c33d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fb9fd7fa640 1 -- 172.21.15.78:0/2299462997 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00011700 con 0x55ee0f75b4d0 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fba04ff9640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9f40b5450 0x7fb9f40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fb9fd7fa640 1 -- 172.21.15.78:0/2299462997 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba00011d50 con 0x55ee0f75b4d0 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.735+0000 7fb9fd7fa640 1 -- 172.21.15.78:0/2299462997 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0000e3c0 con 0x55ee0f75b4d0 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.736+0000 7fba06ffd640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9f40b5450 0x7fb9f40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.736+0000 7fba04ff9640 1 -- 172.21.15.78:0/1861874099 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8493+0+0 (secure 0 0 0) 0x7fb9f8106280 con 0x7fba08102de0 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.736+0000 7fba06ffd640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9f40b5450 0x7fb9f40b7910 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8103aa0 tx=0x7fb9f8109dd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.736+0000 7fba0f919740 1 -- 172.21.15.78:0/2299462997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 msgr2=0x55ee0f75b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.736+0000 7fba0f919740 1 --2- 172.21.15.78:0/2299462997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 0x55ee0f75b8b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fba00009fd0 tx=0x7fba00010860 comp rx=0 tx=0).stop 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.736+0000 7fba0f919740 1 -- 172.21.15.78:0/2299462997 shutdown_connections 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.736+0000 7fba0f919740 1 --2- 172.21.15.78:0/2299462997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f76a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.736+0000 7fba0f919740 1 --2- 172.21.15.78:0/2299462997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f767840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.736+0000 7fba0f919740 1 --2- 172.21.15.78:0/2299462997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 0x55ee0f75b8b0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.736+0000 7fba0f919740 1 -- 172.21.15.78:0/2299462997 >> 172.21.15.78:0/2299462997 conn(0x55ee0f759300 msgr2=0x55ee0f75a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.736+0000 7fba0f919740 1 -- 172.21.15.78:0/2299462997 shutdown_connections 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.736+0000 7fba0f919740 1 -- 172.21.15.78:0/2299462997 wait complete. 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba0f919740 1 -- start start 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba0f919740 1 Processor -- start 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba0f919740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f75b4d0 0x55ee0f767680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba0f919740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f764ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba0f919740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f767d80 0x55ee0f7650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba0f919740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f522700 con 0x55ee0f767d80 2024-09-11T01:12:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba0f919740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f65c160 con 0x55ee0f75bdf0 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba0f919740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f76aa00 con 0x55ee0f75b4d0 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba0f919740 10 client.0 _ll_register_callbacks cb 0x55ee0f6759e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f767d80 0x55ee0f7650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fb9fffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f764ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f767d80 0x55ee0f7650e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38866/0 (socket says 172.21.15.78:38866) 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba077fe640 1 -- 172.21.15.78:0/2984276195 learned_addr learned my addr 172.21.15.78:0/2984276195 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.737+0000 7fba077fe640 1 --2- 172.21.15.78:0/2984276195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f75b4d0 0x55ee0f767680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fb9fffff640 1 -- 172.21.15.78:0/2984276195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f75b4d0 msgr2=0x55ee0f767680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fb9fffff640 1 --2- 172.21.15.78:0/2984276195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f75b4d0 0x55ee0f767680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fb9fffff640 1 -- 172.21.15.78:0/2984276195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f767d80 msgr2=0x55ee0f7650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fb9fffff640 1 --2- 172.21.15.78:0/2984276195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f767d80 0x55ee0f7650e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fb9fffff640 1 -- 172.21.15.78:0/2984276195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0f689c90 con 0x55ee0f75bdf0 2024-09-11T01:12:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fb9fffff640 1 --2- 172.21.15.78:0/2984276195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f764ad0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0010780 tx=0x7fb9f0010c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea7838d2cbd7e6fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fb9ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fb9ee7fc640 1 -- 172.21.15.78:0/2984276195 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f001e060 con 0x55ee0f75bdf0 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fb9ee7fc640 1 -- 172.21.15.78:0/2984276195 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8493+0+0 (secure 0 0 0) 0x7fb9f000fb90 con 0x55ee0f75bdf0 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fba0f919740 1 -- 172.21.15.78:0/2984276195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba00011d50 con 0x55ee0f75bdf0 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fb9ee7fc640 1 -- 172.21.15.78:0/2984276195 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fb9f000fd90 con 0x55ee0f75bdf0 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fba0f919740 10 client.27161 fetch_fsmap learned FSMap version 56 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fba0f919740 1 -- 172.21.15.78:0/2984276195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55ee0f7763b0 con 0x55ee0f75bdf0 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fb9ee7fc640 1 -- 172.21.15.78:0/2984276195 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba00011d50 con 0x55ee0f75bdf0 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.738+0000 7fb9ee7fc640 1 -- 172.21.15.78:0/2984276195 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 56) ==== 8414+0+0 (secure 0 0 0) 0x7fb9f00214b0 con 0x55ee0f75bdf0 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.739+0000 7fba0f919740 10 client.27161 fetch_fsmap finished waiting for FSMap version 56 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.739+0000 7fba0f919740 10 client.27161 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.739+0000 7fba0f919740 20 client.27161 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.739+0000 7fba0f919740 1 --2- 172.21.15.78:0/2984276195 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55ee0f777910 0x55ee0f779d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.739+0000 7fba0f919740 4 client.27161 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.739+0000 7fba0f919740 1 -- 172.21.15.78:0/2984276195 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee0f6d3df0 con 0x55ee0f777910 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.739+0000 7fba077fe640 1 --2- 172.21.15.78:0/2984276195 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55ee0f777910 0x55ee0f779d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.739+0000 7fb9ee7fc640 10 client.27161 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.739+0000 7fba077fe640 1 --2- 172.21.15.78:0/2984276195 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55ee0f777910 0x55ee0f779d30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=35c05783f6629297 server_cookie=457341d28e010c2f in_seq=0 out_seq=0 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fb9ee7fc640 1 -- 172.21.15.78:0/2984276195 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ee0f6d3df0 con 0x55ee0f777910 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fb9ee7fc640 10 client.27161 handle_command_reply: tid=0 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fba0f919740 1 client.27161 shutdown 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fba0f919740 1 -- 172.21.15.78:0/2984276195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 msgr2=0x55ee0f764ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fba0f919740 1 --2- 172.21.15.78:0/2984276195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f764ad0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0010780 tx=0x7fb9f0010c40 comp rx=0 tx=0).stop 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fba0f919740 1 -- 172.21.15.78:0/2984276195 shutdown_connections 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fba0f919740 1 --2- 172.21.15.78:0/2984276195 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55ee0f777910 0x55ee0f779d30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fba0f919740 1 --2- 172.21.15.78:0/2984276195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f767d80 0x55ee0f7650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fba0f919740 1 --2- 172.21.15.78:0/2984276195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f764ad0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fba0f919740 1 --2- 172.21.15.78:0/2984276195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f75b4d0 0x55ee0f767680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fb9ee7fc640 0 client.27161 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:12:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fba0f919740 1 -- 172.21.15.78:0/2984276195 >> 172.21.15.78:0/2984276195 conn(0x55ee0f759300 msgr2=0x55ee0f75eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fba0f919740 1 -- 172.21.15.78:0/2984276195 shutdown_connections 2024-09-11T01:12:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fba0f919740 1 -- 172.21.15.78:0/2984276195 wait complete. 2024-09-11T01:12:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.740+0000 7fba0f919740 20 client.27161 trim_cache size 0 max 16384 2024-09-11T01:12:03.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.761+0000 7fba0f919740 1 Processor -- start 2024-09-11T01:12:03.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.761+0000 7fba0f919740 1 -- start start 2024-09-11T01:12:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.762+0000 7fba0f919740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75b4d0 0x55ee0f773c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.762+0000 7fba0f919740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75bdf0 0x55ee0f763fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.762+0000 7fba0f919740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f75a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.762+0000 7fba0f919740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba081ae6a0 con 0x55ee0f75bdf0 2024-09-11T01:12:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.762+0000 7fba0f919740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fba081b0360 con 0x55ee0f75b4d0 2024-09-11T01:12:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.762+0000 7fba0f919740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba081aed70 con 0x55ee0f767d80 2024-09-11T01:12:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.762+0000 7fba077fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75b4d0 0x55ee0f773c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.762+0000 7fba077fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75b4d0 0x55ee0f773c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41060/0 (socket says 172.21.15.78:41060) 2024-09-11T01:12:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.762+0000 7fba077fe640 1 -- 172.21.15.78:0/1138776798 learned_addr learned my addr 172.21.15.78:0/1138776798 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.762+0000 7fb9fffff640 1 --2- 172.21.15.78:0/1138776798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75bdf0 0x55ee0f763fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.762+0000 7fba077fe640 1 --2- 172.21.15.78:0/1138776798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f75a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.763+0000 7fba077fe640 1 -- 172.21.15.78:0/1138776798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 msgr2=0x55ee0f75a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.763+0000 7fba077fe640 1 --2- 172.21.15.78:0/1138776798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f75a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.763+0000 7fba077fe640 1 -- 172.21.15.78:0/1138776798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75bdf0 msgr2=0x55ee0f763fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.763+0000 7fba077fe640 1 --2- 172.21.15.78:0/1138776798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75bdf0 0x55ee0f763fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.763+0000 7fba077fe640 1 -- 172.21.15.78:0/1138776798 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0f765410 con 0x55ee0f75b4d0 2024-09-11T01:12:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.763+0000 7fba077fe640 1 --2- 172.21.15.78:0/1138776798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75b4d0 0x55ee0f773c30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba00009f20 tx=0x7fba00009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d360d9b3a96f17a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.763+0000 7fb9fcff9640 1 -- 172.21.15.78:0/1138776798 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00024040 con 0x55ee0f75b4d0 2024-09-11T01:12:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.763+0000 7fb9fcff9640 1 -- 172.21.15.78:0/1138776798 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba00004bc0 con 0x55ee0f75b4d0 2024-09-11T01:12:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.763+0000 7fb9fcff9640 1 -- 172.21.15.78:0/1138776798 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00011500 con 0x55ee0f75b4d0 2024-09-11T01:12:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.764+0000 7fba0f919740 1 -- 172.21.15.78:0/1138776798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75b4d0 msgr2=0x55ee0f773c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.764+0000 7fba0f919740 1 --2- 172.21.15.78:0/1138776798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75b4d0 0x55ee0f773c30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba00009f20 tx=0x7fba00009500 comp rx=0 tx=0).stop 2024-09-11T01:12:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.764+0000 7fba0f919740 1 -- 172.21.15.78:0/1138776798 shutdown_connections 2024-09-11T01:12:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.764+0000 7fba0f919740 1 --2- 172.21.15.78:0/1138776798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f75a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.764+0000 7fba0f919740 1 --2- 172.21.15.78:0/1138776798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75bdf0 0x55ee0f763fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.764+0000 7fba0f919740 1 --2- 172.21.15.78:0/1138776798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75b4d0 0x55ee0f773c30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.764+0000 7fba0f919740 1 -- 172.21.15.78:0/1138776798 >> 172.21.15.78:0/1138776798 conn(0x55ee0f759300 msgr2=0x55ee0f763350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.764+0000 7fba0f919740 1 -- 172.21.15.78:0/1138776798 shutdown_connections 2024-09-11T01:12:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.764+0000 7fba0f919740 1 -- 172.21.15.78:0/1138776798 wait complete. 2024-09-11T01:12:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.765+0000 7fba0f919740 1 -- start start 2024-09-11T01:12:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.765+0000 7fba0f919740 1 Processor -- start 2024-09-11T01:12:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.765+0000 7fba0f919740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 0x55ee0f76ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.765+0000 7fba0f919740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f770490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.765+0000 7fba0f919740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f76ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.765+0000 7fba0f919740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba081afc90 con 0x55ee0f75b4d0 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.765+0000 7fba0f919740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fba081af500 con 0x55ee0f75bdf0 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.765+0000 7fba0f919740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba081ac6a0 con 0x55ee0f767d80 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.765+0000 7fba0f919740 10 client.0 _ll_register_callbacks cb 0x7fb9f0010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fba077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 0x55ee0f76ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fb9fffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f770490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fb9fffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f770490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41068/0 (socket says 172.21.15.78:41068) 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fba077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 0x55ee0f76ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38886/0 (socket says 172.21.15.78:38886) 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fb9fffff640 1 -- 172.21.15.78:0/57849859 learned_addr learned my addr 172.21.15.78:0/57849859 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fba077fe640 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f76ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fb9fffff640 1 -- 172.21.15.78:0/57849859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 msgr2=0x55ee0f76ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fb9fffff640 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f76ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fb9fffff640 1 -- 172.21.15.78:0/57849859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 msgr2=0x55ee0f76ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fb9fffff640 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 0x55ee0f76ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fb9fffff640 1 -- 172.21.15.78:0/57849859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0f76b4a0 con 0x55ee0f75bdf0 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fba077fe640 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f76ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fb9fffff640 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f770490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55ee0f758c50 tx=0x7fb9f00215c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae926b9de367b5ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fb9ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.766+0000 7fb9ee7fc640 1 -- 172.21.15.78:0/57849859 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f000a340 con 0x55ee0f75bdf0 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.767+0000 7fba0f919740 1 -- 172.21.15.78:0/57849859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba00004bc0 con 0x55ee0f75bdf0 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.767+0000 7fb9ee7fc640 1 -- 172.21.15.78:0/57849859 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8493+0+0 (secure 0 0 0) 0x7fb9f000fb90 con 0x55ee0f75bdf0 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.767+0000 7fb9ee7fc640 1 -- 172.21.15.78:0/57849859 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fb9f0005280 con 0x55ee0f75bdf0 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.767+0000 7fba0f919740 10 client.27167 fetch_fsmap learned FSMap version 56 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.767+0000 7fba0f919740 1 -- 172.21.15.78:0/57849859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55ee0f7e5350 con 0x55ee0f75bdf0 2024-09-11T01:12:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.768+0000 7fb9ee7fc640 1 -- 172.21.15.78:0/57849859 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba00004bc0 con 0x55ee0f75bdf0 2024-09-11T01:12:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.768+0000 7fb9ee7fc640 1 -- 172.21.15.78:0/57849859 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 56) ==== 8414+0+0 (secure 0 0 0) 0x7fb9f0034430 con 0x55ee0f75bdf0 2024-09-11T01:12:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.768+0000 7fba0f919740 10 client.27167 fetch_fsmap finished waiting for FSMap version 56 2024-09-11T01:12:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.768+0000 7fba0f919740 10 client.27167 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:12:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.768+0000 7fba0f919740 20 client.27167 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.768+0000 7fba0f919740 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55ee0f7e5b30 0x55ee0f7e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.768+0000 7fba0f919740 4 client.27167 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:12:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.768+0000 7fba0f919740 1 -- 172.21.15.78:0/57849859 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee0f6cf8a0 con 0x55ee0f7e5b30 2024-09-11T01:12:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.768+0000 7fba077fe640 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55ee0f7e5b30 0x55ee0f7e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.769+0000 7fb9ee7fc640 10 client.27167 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:12:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.769+0000 7fba077fe640 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55ee0f7e5b30 0x55ee0f7e7f50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=152ed57c96c8c2de server_cookie=424f2d0eaef2bc4e in_seq=0 out_seq=0 2024-09-11T01:12:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.769+0000 7fb9ee7fc640 1 -- 172.21.15.78:0/57849859 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee0f6cf8a0 con 0x55ee0f7e5b30 2024-09-11T01:12:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.769+0000 7fb9ee7fc640 10 client.27167 handle_command_reply: tid=0 2024-09-11T01:12:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.769+0000 7fba0f919740 1 client.27167 shutdown 2024-09-11T01:12:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.769+0000 7fba0f919740 1 -- 172.21.15.78:0/57849859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 msgr2=0x55ee0f770490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.769+0000 7fba0f919740 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f770490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55ee0f758c50 tx=0x7fb9f00215c0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.769+0000 7fba0f919740 1 -- 172.21.15.78:0/57849859 shutdown_connections 2024-09-11T01:12:03.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.770+0000 7fba0f919740 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55ee0f7e5b30 0x55ee0f7e7f50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.770+0000 7fba0f919740 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee0f767d80 0x55ee0f76ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.770+0000 7fba0f919740 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ee0f75bdf0 0x55ee0f770490 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.770+0000 7fba0f919740 1 --2- 172.21.15.78:0/57849859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee0f75b4d0 0x55ee0f76ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.770+0000 7fba0f919740 1 -- 172.21.15.78:0/57849859 >> 172.21.15.78:0/57849859 conn(0x55ee0f759300 msgr2=0x55ee0f75a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:03.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.770+0000 7fb9ee7fc640 0 client.27167 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:12:03.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.770+0000 7fba0f919740 1 -- 172.21.15.78:0/57849859 shutdown_connections 2024-09-11T01:12:03.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.770+0000 7fba0f919740 1 -- 172.21.15.78:0/57849859 wait complete. 2024-09-11T01:12:03.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.770+0000 7fba0f919740 20 client.27167 trim_cache size 0 max 16384 2024-09-11T01:12:03.776 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:12:03.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.771+0000 7fba0d544640 1 -- 172.21.15.78:0/1861874099 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9f40b5450 msgr2=0x7fb9f40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:03.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.771+0000 7fba0d544640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9f40b5450 0x7fb9f40b7910 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8103aa0 tx=0x7fb9f8109dd0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.771+0000 7fba0d544640 1 -- 172.21.15.78:0/1861874099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba08102de0 msgr2=0x7fba08199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.771+0000 7fba0d544640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba08102de0 0x7fba08199280 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8005940 tx=0x7fb9f8004270 comp rx=0 tx=0).stop 2024-09-11T01:12:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.771+0000 7fba0d544640 1 -- 172.21.15.78:0/1861874099 shutdown_connections 2024-09-11T01:12:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.771+0000 7fba0d544640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb9f40b5450 0x7fb9f40b7910 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.771+0000 7fba0d544640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba08109170 0x7fba0819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.771+0000 7fba0d544640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba081087f0 0x7fba081997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.771+0000 7fba0d544640 1 --2- 172.21.15.78:0/1861874099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba08102de0 0x7fba08199280 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.771+0000 7fba0d544640 1 -- 172.21.15.78:0/1861874099 >> 172.21.15.78:0/1861874099 conn(0x7fba080fe830 msgr2=0x7fba08100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.771+0000 7fba0d544640 1 -- 172.21.15.78:0/1861874099 shutdown_connections 2024-09-11T01:12:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:03.771+0000 7fba0d544640 1 -- 172.21.15.78:0/1861874099 wait complete. 2024-09-11T01:12:03.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 3938 43.86 MB/sec warmup 9 sec latency 69.418 ms 2024-09-11T01:12:03.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:03 smithi110 ceph-mon[34712]: pgmap v730: 97 pgs: 97 active+clean; 801 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 196 op/s 2024-09-11T01:12:03.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:03 smithi110 ceph-mon[34712]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T01:12:03.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:03 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:03 smithi112 ceph-mon[35384]: pgmap v730: 97 pgs: 97 active+clean; 801 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 196 op/s 2024-09-11T01:12:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:03 smithi112 ceph-mon[35384]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T01:12:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:03 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:04.486 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:12:04.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 4137 40.02 MB/sec warmup 10 sec latency 88.725 ms 2024-09-11T01:12:04.782 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:12:05.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.304+0000 7ff829b98640 1 -- 172.21.15.78:0/2444069017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824101c90 msgr2=0x7ff824102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:05.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.304+0000 7ff829b98640 1 --2- 172.21.15.78:0/2444069017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824101c90 0x7ff824102070 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7ff814009900 tx=0x7ff81402a410 comp rx=0 tx=0).stop 2024-09-11T01:12:05.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.305+0000 7ff829b98640 1 -- 172.21.15.78:0/2444069017 shutdown_connections 2024-09-11T01:12:05.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.305+0000 7ff829b98640 1 --2- 172.21.15.78:0/2444069017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff824108020 0x7ff82410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.305+0000 7ff829b98640 1 --2- 172.21.15.78:0/2444069017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8241076a0 0x7ff824107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.305+0000 7ff829b98640 1 --2- 172.21.15.78:0/2444069017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824101c90 0x7ff824102070 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.305+0000 7ff829b98640 1 -- 172.21.15.78:0/2444069017 >> 172.21.15.78:0/2444069017 conn(0x7ff8240fd700 msgr2=0x7ff8240ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.305+0000 7ff829b98640 1 -- 172.21.15.78:0/2444069017 shutdown_connections 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.305+0000 7ff829b98640 1 -- 172.21.15.78:0/2444069017 wait complete. 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.305+0000 7ff829b98640 1 Processor -- start 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.305+0000 7ff829b98640 1 -- start start 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff829b98640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff824101c90 0x7ff8240fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff829b98640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8241076a0 0x7ff8240fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff829b98640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824108020 0x7ff8240fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff829b98640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff824111d30 con 0x7ff824108020 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff829b98640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff824111bb0 con 0x7ff824101c90 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff829b98640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff824111eb0 con 0x7ff8241076a0 2024-09-11T01:12:05.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff823fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824108020 0x7ff8240fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff824101c90 0x7ff8240fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff823fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824108020 0x7ff8240fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38902/0 (socket says 172.21.15.78:38902) 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff823fff640 1 -- 172.21.15.78:0/1172467020 learned_addr learned my addr 172.21.15.78:0/1172467020 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff8237fe640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8241076a0 0x7ff8240fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff823fff640 1 -- 172.21.15.78:0/1172467020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8241076a0 msgr2=0x7ff8240fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff823fff640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8241076a0 0x7ff8240fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff823fff640 1 -- 172.21.15.78:0/1172467020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff824101c90 msgr2=0x7ff8240fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff823fff640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff824101c90 0x7ff8240fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff823fff640 1 -- 172.21.15.78:0/1172467020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8241a76b0 con 0x7ff824108020 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff823fff640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824108020 0x7ff8240fe680 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00eb10 tx=0x7ff81c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.306+0000 7ff8217fa640 1 -- 172.21.15.78:0/1172467020 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004b20 con 0x7ff824108020 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.307+0000 7ff829b98640 1 -- 172.21.15.78:0/1172467020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8241a79a0 con 0x7ff824108020 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.307+0000 7ff829b98640 1 -- 172.21.15.78:0/1172467020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8241a7f30 con 0x7ff824108020 2024-09-11T01:12:05.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.307+0000 7ff8217fa640 1 -- 172.21.15.78:0/1172467020 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff81c004cc0 con 0x7ff824108020 2024-09-11T01:12:05.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.307+0000 7ff8217fa640 1 -- 172.21.15.78:0/1172467020 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c010690 con 0x7ff824108020 2024-09-11T01:12:05.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.310+0000 7ff82bf6d740 1 Processor -- start 2024-09-11T01:12:05.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.311+0000 7ff8217fa640 1 -- 172.21.15.78:0/1172467020 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff81c005000 con 0x7ff824108020 2024-09-11T01:12:05.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.311+0000 7ff82bf6d740 1 -- start start 2024-09-11T01:12:05.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.311+0000 7ff82bf6d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 0x5556c63600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.311+0000 7ff82bf6d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c6360600 0x5556c636c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.311+0000 7ff82bf6d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c636c590 0x5556c636e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.311+0000 7ff82bf6d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5556c62cdea0 con 0x5556c635fce0 2024-09-11T01:12:05.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.311+0000 7ff82bf6d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5556c62bd2d0 con 0x5556c6360600 2024-09-11T01:12:05.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.311+0000 7ff82bf6d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5556c62c10e0 con 0x5556c636c590 2024-09-11T01:12:05.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.311+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 0x5556c63600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.311+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 0x5556c63600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38904/0 (socket says 172.21.15.78:38904) 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.311+0000 7ff8237fe640 1 -- 172.21.15.78:0/2925124979 learned_addr learned my addr 172.21.15.78:0/2925124979 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.311+0000 7ff81bfff640 1 --2- 172.21.15.78:0/2925124979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c6360600 0x5556c636c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff8237fe640 1 --2- 172.21.15.78:0/2925124979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c636c590 0x5556c636e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff8217fa640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8100b54d0 0x7ff8100b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff8237fe640 1 -- 172.21.15.78:0/2925124979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c636c590 msgr2=0x5556c636e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff8237fe640 1 --2- 172.21.15.78:0/2925124979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c636c590 0x5556c636e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff8237fe640 1 -- 172.21.15.78:0/2925124979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c6360600 msgr2=0x5556c636c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff8237fe640 1 --2- 172.21.15.78:0/2925124979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c6360600 0x5556c636c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff8237fe640 1 -- 172.21.15.78:0/2925124979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556c601b650 con 0x5556c635fce0 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff81bfff640 1 --2- 172.21.15.78:0/2925124979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c6360600 0x5556c636c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff823fff640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8100b54d0 0x7ff8100b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff8237fe640 1 --2- 172.21.15.78:0/2925124979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 0x5556c63600c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0137e0 tx=0x7ff80c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3dfa25acedd8af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:05.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff8217fa640 1 -- 172.21.15.78:0/1172467020 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8493+0+0 (secure 0 0 0) 0x7ff81c0e61e0 con 0x7ff824108020 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff819ffb640 1 -- 172.21.15.78:0/2925124979 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c013e20 con 0x5556c635fce0 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff819ffb640 1 -- 172.21.15.78:0/2925124979 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff80c013170 con 0x5556c635fce0 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.312+0000 7ff819ffb640 1 -- 172.21.15.78:0/2925124979 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c010820 con 0x5556c635fce0 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.313+0000 7ff82bf6d740 1 -- 172.21.15.78:0/2925124979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 msgr2=0x5556c63600c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.313+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/2925124979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 0x5556c63600c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0137e0 tx=0x7ff80c005f90 comp rx=0 tx=0).stop 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.313+0000 7ff823fff640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8100b54d0 0x7ff8100b7990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0ac610 tx=0x7ff81c0a0ee0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.313+0000 7ff82bf6d740 1 -- 172.21.15.78:0/2925124979 shutdown_connections 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.313+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/2925124979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c636c590 0x5556c636e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.313+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/2925124979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c6360600 0x5556c636c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.313+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/2925124979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 0x5556c63600c0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.313+0000 7ff82bf6d740 1 -- 172.21.15.78:0/2925124979 >> 172.21.15.78:0/2925124979 conn(0x5556c635db10 msgr2=0x5556c635ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.313+0000 7ff82bf6d740 1 -- 172.21.15.78:0/2925124979 shutdown_connections 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.313+0000 7ff82bf6d740 1 -- 172.21.15.78:0/2925124979 wait complete. 2024-09-11T01:12:05.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff82bf6d740 1 -- start start 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff82bf6d740 1 Processor -- start 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff82bf6d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c635fce0 0x5556c636be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff82bf6d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c6360600 0x5556c6369270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff82bf6d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c636c590 0x5556c6369810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff82bf6d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5556c611e700 con 0x5556c636c590 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff82bf6d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5556c6258160 con 0x5556c6360600 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff82bf6d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5556c636f1e0 con 0x5556c635fce0 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff82bf6d740 10 client.0 _ll_register_callbacks cb 0x5556c62719e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c636c590 0x5556c6369810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c635fce0 0x5556c636be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c6360600 0x5556c6369270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c635fce0 0x5556c636be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35058/0 (socket says 172.21.15.78:35058) 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.314+0000 7ff8237fe640 1 -- 172.21.15.78:0/4190210545 learned_addr learned my addr 172.21.15.78:0/4190210545 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:05.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff8237fe640 1 -- 172.21.15.78:0/4190210545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c6360600 msgr2=0x5556c6369270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff8237fe640 1 --2- 172.21.15.78:0/4190210545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c6360600 0x5556c6369270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff8237fe640 1 -- 172.21.15.78:0/4190210545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c636c590 msgr2=0x5556c6369810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff8237fe640 1 --2- 172.21.15.78:0/4190210545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c636c590 0x5556c6369810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff8237fe640 1 -- 172.21.15.78:0/4190210545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556c6285c90 con 0x5556c635fce0 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff8237fe640 1 --2- 172.21.15.78:0/4190210545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c635fce0 0x5556c636be20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff80c013f70 tx=0x7ff80c00ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27f948821e2a27e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff802ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff802ffd640 1 -- 172.21.15.78:0/4190210545 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c026070 con 0x5556c635fce0 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff82bf6d740 1 -- 172.21.15.78:0/4190210545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff80c013170 con 0x5556c635fce0 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff802ffd640 1 -- 172.21.15.78:0/4190210545 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8493+0+0 (secure 0 0 0) 0x7ff80c004020 con 0x5556c635fce0 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff802ffd640 1 -- 172.21.15.78:0/4190210545 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7ff80c029040 con 0x5556c635fce0 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff82bf6d740 10 client.27112 fetch_fsmap learned FSMap version 57 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff82bf6d740 1 -- 172.21.15.78:0/4190210545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5556c637ab10 con 0x5556c635fce0 2024-09-11T01:12:05.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff802ffd640 1 -- 172.21.15.78:0/4190210545 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff80c013170 con 0x5556c635fce0 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff802ffd640 1 -- 172.21.15.78:0/4190210545 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7ff80c003c60 con 0x5556c635fce0 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff82bf6d740 10 client.27112 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff82bf6d740 10 client.27112 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff82bf6d740 20 client.27112 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.315+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/4190210545 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5556c637c3b0 0x5556c637e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.316+0000 7ff82bf6d740 4 client.27112 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.316+0000 7ff82bf6d740 1 -- 172.21.15.78:0/4190210545 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556c62cfdf0 con 0x5556c637c3b0 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.316+0000 7ff81bfff640 1 --2- 172.21.15.78:0/4190210545 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5556c637c3b0 0x5556c637e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.316+0000 7ff81bfff640 1 --2- 172.21.15.78:0/4190210545 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5556c637c3b0 0x5556c637e7d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f95111cb8d8e1932 server_cookie=eb8a5ebee47ee3f4 in_seq=0 out_seq=0 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.316+0000 7ff802ffd640 10 client.27112 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff802ffd640 1 -- 172.21.15.78:0/4190210545 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5556c62cfdf0 con 0x5556c637c3b0 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff802ffd640 10 client.27112 handle_command_reply: tid=0 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff82bf6d740 1 client.27112 shutdown 2024-09-11T01:12:05.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff82bf6d740 1 -- 172.21.15.78:0/4190210545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c635fce0 msgr2=0x5556c636be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/4190210545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c635fce0 0x5556c636be20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff80c013f70 tx=0x7ff80c00ffd0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff82bf6d740 1 -- 172.21.15.78:0/4190210545 shutdown_connections 2024-09-11T01:12:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/4190210545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c636c590 0x5556c6369810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/4190210545 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5556c637c3b0 0x5556c637e7d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/4190210545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c6360600 0x5556c6369270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/4190210545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c635fce0 0x5556c636be20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff82bf6d740 1 -- 172.21.15.78:0/4190210545 >> 172.21.15.78:0/4190210545 conn(0x5556c635db10 msgr2=0x5556c6354420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff802ffd640 0 client.27112 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:12:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff82bf6d740 1 -- 172.21.15.78:0/4190210545 shutdown_connections 2024-09-11T01:12:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff82bf6d740 1 -- 172.21.15.78:0/4190210545 wait complete. 2024-09-11T01:12:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.317+0000 7ff82bf6d740 20 client.27112 trim_cache size 0 max 16384 2024-09-11T01:12:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.333+0000 7ff82bf6d740 1 Processor -- start 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.333+0000 7ff82bf6d740 1 -- start start 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.333+0000 7ff82bf6d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c635fce0 0x5556c635eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.333+0000 7ff82bf6d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c6360600 0x5556c6354310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff82bf6d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c636c590 0x5556c6366480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff82bf6d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff80400e740 con 0x5556c6360600 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff82bf6d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff804010400 con 0x5556c635fce0 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff82bf6d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff80400ee10 con 0x5556c636c590 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c6360600 0x5556c6354310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c635fce0 0x5556c635eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c6360600 0x5556c6354310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38926/0 (socket says 172.21.15.78:38926) 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff81bfff640 1 -- 172.21.15.78:0/1402166826 learned_addr learned my addr 172.21.15.78:0/1402166826 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff8237fe640 1 --2- 172.21.15.78:0/1402166826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c636c590 0x5556c6366480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff81bfff640 1 -- 172.21.15.78:0/1402166826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c636c590 msgr2=0x5556c6366480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff81bfff640 1 --2- 172.21.15.78:0/1402166826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c636c590 0x5556c6366480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff81bfff640 1 -- 172.21.15.78:0/1402166826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c635fce0 msgr2=0x5556c635eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff81bfff640 1 --2- 172.21.15.78:0/1402166826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c635fce0 0x5556c635eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff81bfff640 1 -- 172.21.15.78:0/1402166826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556c6360430 con 0x5556c6360600 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff8237fe640 1 --2- 172.21.15.78:0/1402166826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c635fce0 0x5556c635eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff81bfff640 1 --2- 172.21.15.78:0/1402166826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c6360600 0x5556c6354310 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff814005ab0 tx=0x7ff814005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfefb1e063540ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff8197fa640 1 -- 172.21.15.78:0/1402166826 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81404a020 con 0x5556c6360600 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff8197fa640 1 -- 172.21.15.78:0/1402166826 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8140054f0 con 0x5556c6360600 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.334+0000 7ff8197fa640 1 -- 172.21.15.78:0/1402166826 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814035040 con 0x5556c6360600 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.335+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1402166826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c6360600 msgr2=0x5556c6354310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.335+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/1402166826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c6360600 0x5556c6354310 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff814005ab0 tx=0x7ff814005a20 comp rx=0 tx=0).stop 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.335+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1402166826 shutdown_connections 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.335+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/1402166826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c636c590 0x5556c6366480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.335+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/1402166826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c6360600 0x5556c6354310 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.335+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/1402166826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c635fce0 0x5556c635eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.335+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1402166826 >> 172.21.15.78:0/1402166826 conn(0x5556c635db10 msgr2=0x5556c6353c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.335+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1402166826 shutdown_connections 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.335+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1402166826 wait complete. 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.335+0000 7ff82bf6d740 1 -- start start 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff82bf6d740 1 Processor -- start 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff82bf6d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 0x5556c63706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff82bf6d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c6360600 0x5556c6370c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff82bf6d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c636c590 0x5556c63e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff82bf6d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff80400fd30 con 0x5556c635fce0 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff82bf6d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff80400f5a0 con 0x5556c636c590 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff82bf6d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff80400c740 con 0x5556c6360600 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff82bf6d740 10 client.0 _ll_register_callbacks cb 0x7ff81c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 0x5556c63706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 0x5556c63706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38936/0 (socket says 172.21.15.78:38936) 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff8237fe640 1 -- 172.21.15.78:0/1760306205 learned_addr learned my addr 172.21.15.78:0/1760306205 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff8237fe640 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c636c590 0x5556c63e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff81bfff640 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c6360600 0x5556c6370c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff8237fe640 1 -- 172.21.15.78:0/1760306205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c6360600 msgr2=0x5556c6370c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff8237fe640 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c6360600 0x5556c6370c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff8237fe640 1 -- 172.21.15.78:0/1760306205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c636c590 msgr2=0x5556c63e82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff8237fe640 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c636c590 0x5556c63e82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.336+0000 7ff8237fe640 1 -- 172.21.15.78:0/1760306205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556c6369b80 con 0x5556c635fce0 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff8237fe640 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 0x5556c63706e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff80c001eb0 tx=0x7ff80c010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd82461cb18c8f16 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff802ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff802ffd640 1 -- 172.21.15.78:0/1760306205 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c026040 con 0x5556c635fce0 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff802ffd640 1 -- 172.21.15.78:0/1760306205 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8493+0+0 (secure 0 0 0) 0x7ff80c005b90 con 0x5556c635fce0 2024-09-11T01:12:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1760306205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8140054f0 con 0x5556c635fce0 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff802ffd640 1 -- 172.21.15.78:0/1760306205 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7ff80c044050 con 0x5556c635fce0 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff82bf6d740 10 client.17994 fetch_fsmap learned FSMap version 57 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1760306205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5556c63e96d0 con 0x5556c635fce0 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff802ffd640 1 -- 172.21.15.78:0/1760306205 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8140054f0 con 0x5556c635fce0 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff802ffd640 1 -- 172.21.15.78:0/1760306205 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7ff80c003960 con 0x5556c635fce0 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff82bf6d740 10 client.17994 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff82bf6d740 10 client.17994 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff82bf6d740 20 client.17994 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5556c63e9e50 0x5556c63ec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff82bf6d740 4 client.17994 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.337+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1760306205 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556c62cb8a0 con 0x5556c63e9e50 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff81bfff640 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5556c63e9e50 0x5556c63ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff802ffd640 10 client.17994 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:12:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff81bfff640 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5556c63e9e50 0x5556c63ec270 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cd3afa00810f9a7 server_cookie=591c1646a1862787 in_seq=0 out_seq=0 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff802ffd640 1 -- 172.21.15.78:0/1760306205 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556c62cb8a0 con 0x5556c63e9e50 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff802ffd640 10 client.17994 handle_command_reply: tid=0 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff82bf6d740 1 client.17994 shutdown 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1760306205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 msgr2=0x5556c63706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 0x5556c63706e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff80c001eb0 tx=0x7ff80c010430 comp rx=0 tx=0).stop 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1760306205 shutdown_connections 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5556c63e9e50 0x5556c63ec270 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556c636c590 0x5556c63e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556c6360600 0x5556c6370c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff82bf6d740 1 --2- 172.21.15.78:0/1760306205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556c635fce0 0x5556c63706e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff802ffd640 0 client.17994 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.338+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1760306205 >> 172.21.15.78:0/1760306205 conn(0x5556c635db10 msgr2=0x5556c6354420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.339+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1760306205 shutdown_connections 2024-09-11T01:12:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.339+0000 7ff82bf6d740 1 -- 172.21.15.78:0/1760306205 wait complete. 2024-09-11T01:12:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.339+0000 7ff82bf6d740 20 client.17994 trim_cache size 0 max 16384 2024-09-11T01:12:05.573 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:12:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.340+0000 7ff829b98640 1 -- 172.21.15.78:0/1172467020 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8100b54d0 msgr2=0x7ff8100b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.340+0000 7ff829b98640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8100b54d0 0x7ff8100b7990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0ac610 tx=0x7ff81c0a0ee0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.340+0000 7ff829b98640 1 -- 172.21.15.78:0/1172467020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824108020 msgr2=0x7ff8240fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.340+0000 7ff829b98640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824108020 0x7ff8240fe680 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00eb10 tx=0x7ff81c00c2d0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.340+0000 7ff829b98640 1 -- 172.21.15.78:0/1172467020 shutdown_connections 2024-09-11T01:12:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.340+0000 7ff829b98640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8100b54d0 0x7ff8100b7990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.340+0000 7ff829b98640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824108020 0x7ff8240fe680 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.340+0000 7ff829b98640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8241076a0 0x7ff8240fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.340+0000 7ff829b98640 1 --2- 172.21.15.78:0/1172467020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff824101c90 0x7ff8240fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.340+0000 7ff829b98640 1 -- 172.21.15.78:0/1172467020 >> 172.21.15.78:0/1172467020 conn(0x7ff8240fd700 msgr2=0x7ff82406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.340+0000 7ff829b98640 1 -- 172.21.15.78:0/1172467020 shutdown_connections 2024-09-11T01:12:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:05.340+0000 7ff829b98640 1 -- 172.21.15.78:0/1172467020 wait complete. 2024-09-11T01:12:05.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 4497 37.24 MB/sec warmup 11 sec latency 149.114 ms 2024-09-11T01:12:05.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:05 smithi110 ceph-mon[34712]: pgmap v732: 97 pgs: 97 active+clean; 815 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 352 op/s 2024-09-11T01:12:05.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:05 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:05.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:05 smithi112 ceph-mon[35384]: pgmap v732: 97 pgs: 97 active+clean; 815 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 352 op/s 2024-09-11T01:12:05.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:05 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:05.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:05 smithi078 ceph-mon[31315]: pgmap v732: 97 pgs: 97 active+clean; 815 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 352 op/s 2024-09-11T01:12:05.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:05 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:06.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:12:06.632 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:12:06.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 5071 37.31 MB/sec warmup 12 sec latency 91.885 ms 2024-09-11T01:12:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.183+0000 7f147470c640 1 -- 172.21.15.78:0/1835308711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c101c90 msgr2=0x7f146c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:07.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.183+0000 7f147470c640 1 --2- 172.21.15.78:0/1835308711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c101c90 0x7f146c102070 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f1468009900 tx=0x7f146802a410 comp rx=0 tx=0).stop 2024-09-11T01:12:07.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.183+0000 7f147470c640 1 -- 172.21.15.78:0/1835308711 shutdown_connections 2024-09-11T01:12:07.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.183+0000 7f147470c640 1 --2- 172.21.15.78:0/1835308711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c108020 0x7f146c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.183+0000 7f147470c640 1 --2- 172.21.15.78:0/1835308711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f146c1076a0 0x7f146c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.183+0000 7f147470c640 1 --2- 172.21.15.78:0/1835308711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c101c90 0x7f146c102070 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.183+0000 7f147470c640 1 -- 172.21.15.78:0/1835308711 >> 172.21.15.78:0/1835308711 conn(0x7f146c0fd700 msgr2=0x7f146c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:07.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.183+0000 7f147470c640 1 -- 172.21.15.78:0/1835308711 shutdown_connections 2024-09-11T01:12:07.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.183+0000 7f147470c640 1 -- 172.21.15.78:0/1835308711 wait complete. 2024-09-11T01:12:07.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.183+0000 7f147470c640 1 Processor -- start 2024-09-11T01:12:07.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.183+0000 7f147470c640 1 -- start start 2024-09-11T01:12:07.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.183+0000 7f147470c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c101c90 0x7f146c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f147470c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f146c1076a0 0x7f146c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f147470c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c108020 0x7f146c19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f147470c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f146c111d30 con 0x7f146c108020 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f147470c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f146c111bb0 con 0x7f146c101c90 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f147470c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f146c111eb0 con 0x7f146c1076a0 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f1472ca9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c108020 0x7f146c19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f1471ca7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f146c1076a0 0x7f146c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f14724a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c101c90 0x7f146c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f1472ca9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c108020 0x7f146c19dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38954/0 (socket says 172.21.15.78:38954) 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f1472ca9640 1 -- 172.21.15.78:0/3512617864 learned_addr learned my addr 172.21.15.78:0/3512617864 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f14724a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c101c90 0x7f146c1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41150/0 (socket says 172.21.15.78:41150) 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f1471ca7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f146c1076a0 0x7f146c198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35102/0 (socket says 172.21.15.78:35102) 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f1472ca9640 1 -- 172.21.15.78:0/3512617864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f146c1076a0 msgr2=0x7f146c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f1472ca9640 1 --2- 172.21.15.78:0/3512617864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f146c1076a0 0x7f146c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f1472ca9640 1 -- 172.21.15.78:0/3512617864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c101c90 msgr2=0x7f146c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f1472ca9640 1 --2- 172.21.15.78:0/3512617864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c101c90 0x7f146c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f1472ca9640 1 -- 172.21.15.78:0/3512617864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c19e380 con 0x7f146c108020 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f1472ca9640 1 --2- 172.21.15.78:0/3512617864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c108020 0x7f146c19dbb0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1464009fd0 tx=0x7f146400def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f14637fe640 1 -- 172.21.15.78:0/3512617864 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464017070 con 0x7f146c108020 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f147470c640 1 -- 172.21.15.78:0/3512617864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c19e670 con 0x7f146c108020 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f14637fe640 1 -- 172.21.15.78:0/3512617864 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1464003e20 con 0x7f146c108020 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f14637fe640 1 -- 172.21.15.78:0/3512617864 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464010460 con 0x7f146c108020 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.184+0000 7f147470c640 1 -- 172.21.15.78:0/3512617864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c06aab0 con 0x7f146c108020 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.187+0000 7f1476ae1740 1 Processor -- start 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.187+0000 7f14637fe640 1 -- 172.21.15.78:0/3512617864 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1464003970 con 0x7f146c108020 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.187+0000 7f1476ae1740 1 -- start start 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.187+0000 7f1476ae1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb2414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.187+0000 7f1476ae1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb24d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.187+0000 7f1476ae1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.187+0000 7f1476ae1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a1cb1b7ea0 con 0x55a1cb241a10 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.187+0000 7f1476ae1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a1cb1a72d0 con 0x55a1cb2410f0 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.187+0000 7f1476ae1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a1cb1ab0e0 con 0x55a1cb24d9a0 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f14724a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb24d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f1471ca7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb2414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f14724a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb24d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38956/0 (socket says 172.21.15.78:38956) 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f14724a8640 1 -- 172.21.15.78:0/919455083 learned_addr learned my addr 172.21.15.78:0/919455083 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f1471ca7640 1 --2- 172.21.15.78:0/919455083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f14724a8640 1 -- 172.21.15.78:0/919455083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb24d9a0 msgr2=0x55a1cb24fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f14724a8640 1 --2- 172.21.15.78:0/919455083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f14724a8640 1 -- 172.21.15.78:0/919455083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 msgr2=0x55a1cb2414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f14724a8640 1 --2- 172.21.15.78:0/919455083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb2414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f14724a8640 1 -- 172.21.15.78:0/919455083 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1caf05650 con 0x55a1cb241a10 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f14637fe640 1 --2- 172.21.15.78:0/3512617864 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14580b5450 0x7f14580b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f14724a8640 1 --2- 172.21.15.78:0/919455083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb24d460 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1468009230 tx=0x7f1468005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fccedf82f18aee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f1471ca7640 1 --2- 172.21.15.78:0/3512617864 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14580b5450 0x7f14580b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f14637fe640 1 -- 172.21.15.78:0/3512617864 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8493+0+0 (secure 0 0 0) 0x7f14640e5d00 con 0x7f146c108020 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f1460ff9640 1 -- 172.21.15.78:0/919455083 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468035070 con 0x55a1cb241a10 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f1460ff9640 1 -- 172.21.15.78:0/919455083 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14680054f0 con 0x55a1cb241a10 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.188+0000 7f1460ff9640 1 -- 172.21.15.78:0/919455083 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14680035f0 con 0x55a1cb241a10 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.189+0000 7f1476ae1740 1 -- 172.21.15.78:0/919455083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 msgr2=0x55a1cb24d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.189+0000 7f1476ae1740 1 --2- 172.21.15.78:0/919455083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb24d460 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1468009230 tx=0x7f1468005a20 comp rx=0 tx=0).stop 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.189+0000 7f1471ca7640 1 --2- 172.21.15.78:0/3512617864 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14580b5450 0x7f14580b7910 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f146c107220 tx=0x7f145c00a6c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.189+0000 7f1476ae1740 1 -- 172.21.15.78:0/919455083 shutdown_connections 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.189+0000 7f1476ae1740 1 --2- 172.21.15.78:0/919455083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.189+0000 7f1476ae1740 1 --2- 172.21.15.78:0/919455083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb24d460 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.189+0000 7f1476ae1740 1 --2- 172.21.15.78:0/919455083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb2414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.189+0000 7f1476ae1740 1 -- 172.21.15.78:0/919455083 >> 172.21.15.78:0/919455083 conn(0x55a1cb23ef20 msgr2=0x55a1cb2400c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.189+0000 7f1476ae1740 1 -- 172.21.15.78:0/919455083 shutdown_connections 2024-09-11T01:12:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.189+0000 7f1476ae1740 1 -- 172.21.15.78:0/919455083 wait complete. 2024-09-11T01:12:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1476ae1740 1 -- start start 2024-09-11T01:12:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1476ae1740 1 Processor -- start 2024-09-11T01:12:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1476ae1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb24d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1476ae1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb24a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1476ae1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1476ae1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a1cb008700 con 0x55a1cb241a10 2024-09-11T01:12:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1476ae1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a1cb142160 con 0x55a1cb2410f0 2024-09-11T01:12:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1476ae1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a1cb2505c0 con 0x55a1cb24d9a0 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1476ae1740 10 client.0 _ll_register_callbacks cb 0x55a1cb15b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1471ca7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb24a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f14724a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb24d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1471ca7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb24a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38962/0 (socket says 172.21.15.78:38962) 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f14724a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb24d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41166/0 (socket says 172.21.15.78:41166) 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1472ca9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1471ca7640 1 -- 172.21.15.78:0/4128360099 learned_addr learned my addr 172.21.15.78:0/4128360099 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1472ca9640 1 -- 172.21.15.78:0/4128360099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 msgr2=0x55a1cb24d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1472ca9640 1 --2- 172.21.15.78:0/4128360099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb24d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1472ca9640 1 -- 172.21.15.78:0/4128360099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 msgr2=0x55a1cb24a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.190+0000 7f1472ca9640 1 --2- 172.21.15.78:0/4128360099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb24a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1472ca9640 1 -- 172.21.15.78:0/4128360099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1cb16fc90 con 0x55a1cb24d9a0 2024-09-11T01:12:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1471ca7640 1 --2- 172.21.15.78:0/4128360099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb24a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1472ca9640 1 --2- 172.21.15.78:0/4128360099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24ac90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14640e3410 tx=0x7f14640a8880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42a113c7afb0191c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1455ffb640 1 -- 172.21.15.78:0/4128360099 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464020070 con 0x55a1cb24d9a0 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1455ffb640 1 -- 172.21.15.78:0/4128360099 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8493+0+0 (secure 0 0 0) 0x7f14640e3b60 con 0x55a1cb24d9a0 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1476ae1740 1 -- 172.21.15.78:0/4128360099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14680054f0 con 0x55a1cb24d9a0 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1455ffb640 1 -- 172.21.15.78:0/4128360099 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f14640ab8b0 con 0x55a1cb24d9a0 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1476ae1740 10 client.27127 fetch_fsmap learned FSMap version 57 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1476ae1740 1 -- 172.21.15.78:0/4128360099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55a1cb25bf20 con 0x55a1cb24d9a0 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1455ffb640 1 -- 172.21.15.78:0/4128360099 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14680054f0 con 0x55a1cb24d9a0 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1455ffb640 1 -- 172.21.15.78:0/4128360099 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f14640cdb20 con 0x55a1cb24d9a0 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1476ae1740 10 client.27127 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1476ae1740 10 client.27127 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1476ae1740 20 client.27127 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1476ae1740 1 --2- 172.21.15.78:0/4128360099 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55a1cb25d3d0 0x55a1cb25f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1476ae1740 4 client.27127 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.191+0000 7f1476ae1740 1 -- 172.21.15.78:0/4128360099 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1cb1b9df0 con 0x55a1cb25d3d0 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.192+0000 7f14724a8640 1 --2- 172.21.15.78:0/4128360099 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55a1cb25d3d0 0x55a1cb25f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.192+0000 7f14724a8640 1 --2- 172.21.15.78:0/4128360099 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55a1cb25d3d0 0x55a1cb25f840 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98a3efe8032bc81 server_cookie=91d77161674e0bd3 in_seq=0 out_seq=0 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.192+0000 7f1455ffb640 10 client.27127 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1455ffb640 1 -- 172.21.15.78:0/4128360099 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a1cb1b9df0 con 0x55a1cb25d3d0 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1455ffb640 10 client.27127 handle_command_reply: tid=0 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1476ae1740 1 client.27127 shutdown 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1476ae1740 1 -- 172.21.15.78:0/4128360099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb24d9a0 msgr2=0x55a1cb24ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1476ae1740 1 --2- 172.21.15.78:0/4128360099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24ac90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14640e3410 tx=0x7f14640a8880 comp rx=0 tx=0).stop 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1476ae1740 1 -- 172.21.15.78:0/4128360099 shutdown_connections 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1476ae1740 1 --2- 172.21.15.78:0/4128360099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24ac90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1476ae1740 1 --2- 172.21.15.78:0/4128360099 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55a1cb25d3d0 0x55a1cb25f840 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1476ae1740 1 --2- 172.21.15.78:0/4128360099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb24a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1476ae1740 1 --2- 172.21.15.78:0/4128360099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb24d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1455ffb640 0 client.27127 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:12:07.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1476ae1740 1 -- 172.21.15.78:0/4128360099 >> 172.21.15.78:0/4128360099 conn(0x55a1cb23ef20 msgr2=0x55a1cb244ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:07.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1476ae1740 1 -- 172.21.15.78:0/4128360099 shutdown_connections 2024-09-11T01:12:07.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1476ae1740 1 -- 172.21.15.78:0/4128360099 wait complete. 2024-09-11T01:12:07.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.193+0000 7f1476ae1740 20 client.27127 trim_cache size 0 max 16384 2024-09-11T01:12:07.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.207+0000 7f1476ae1740 1 Processor -- start 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1476ae1740 1 -- start start 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1476ae1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb2597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1476ae1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb241a10 0x55a1cb24ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1476ae1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1476ae1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1ac460 con 0x55a1cb24d9a0 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1476ae1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1ae120 con 0x55a1cb2410f0 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1476ae1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1acb30 con 0x55a1cb241a10 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1472ca9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1472ca9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38976/0 (socket says 172.21.15.78:38976) 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f14724a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb2597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1472ca9640 1 -- 172.21.15.78:0/2418968141 learned_addr learned my addr 172.21.15.78:0/2418968141 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1471ca7640 1 --2- 172.21.15.78:0/2418968141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb241a10 0x55a1cb24ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1472ca9640 1 -- 172.21.15.78:0/2418968141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb241a10 msgr2=0x55a1cb24ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:07.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1472ca9640 1 --2- 172.21.15.78:0/2418968141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb241a10 0x55a1cb24ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1472ca9640 1 -- 172.21.15.78:0/2418968141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 msgr2=0x55a1cb2597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:07.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1472ca9640 1 --2- 172.21.15.78:0/2418968141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb2597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1472ca9640 1 -- 172.21.15.78:0/2418968141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1cb23fc00 con 0x55a1cb24d9a0 2024-09-11T01:12:07.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f14724a8640 1 --2- 172.21.15.78:0/2418968141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb2597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:07.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.208+0000 7f1472ca9640 1 --2- 172.21.15.78:0/2418968141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24f2d0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f14640ac0d0 tx=0x7f14640b5850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c800da293fe5687e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:07.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.209+0000 7f1457fff640 1 -- 172.21.15.78:0/2418968141 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464020070 con 0x55a1cb24d9a0 2024-09-11T01:12:07.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.209+0000 7f1457fff640 1 -- 172.21.15.78:0/2418968141 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14640abc50 con 0x55a1cb24d9a0 2024-09-11T01:12:07.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.209+0000 7f1457fff640 1 -- 172.21.15.78:0/2418968141 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14640ac140 con 0x55a1cb24d9a0 2024-09-11T01:12:07.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.209+0000 7f1476ae1740 1 -- 172.21.15.78:0/2418968141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb24d9a0 msgr2=0x55a1cb24f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:07.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.209+0000 7f1476ae1740 1 --2- 172.21.15.78:0/2418968141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24f2d0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f14640ac0d0 tx=0x7f14640b5850 comp rx=0 tx=0).stop 2024-09-11T01:12:07.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.209+0000 7f1476ae1740 1 -- 172.21.15.78:0/2418968141 shutdown_connections 2024-09-11T01:12:07.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.209+0000 7f1476ae1740 1 --2- 172.21.15.78:0/2418968141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb24d9a0 0x55a1cb24f2d0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.209+0000 7f1476ae1740 1 --2- 172.21.15.78:0/2418968141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb241a10 0x55a1cb24ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.209+0000 7f1476ae1740 1 --2- 172.21.15.78:0/2418968141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb2410f0 0x55a1cb2597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.209+0000 7f1476ae1740 1 -- 172.21.15.78:0/2418968141 >> 172.21.15.78:0/2418968141 conn(0x55a1cb23ef20 msgr2=0x55a1cb23df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:07.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.210+0000 7f1476ae1740 1 -- 172.21.15.78:0/2418968141 shutdown_connections 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.210+0000 7f1476ae1740 1 -- 172.21.15.78:0/2418968141 wait complete. 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.210+0000 7f1476ae1740 1 -- start start 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.210+0000 7f1476ae1740 1 Processor -- start 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.210+0000 7f1476ae1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb2410f0 0x55a1cb255d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.210+0000 7f1476ae1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb253860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.210+0000 7f1476ae1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb24d9a0 0x55a1cb253da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.210+0000 7f1476ae1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1ada50 con 0x55a1cb241a10 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.210+0000 7f1476ae1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1ad2c0 con 0x55a1cb24d9a0 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.210+0000 7f1476ae1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1aa460 con 0x55a1cb2410f0 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.210+0000 7f1471ca7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb253860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1471ca7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb253860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38992/0 (socket says 172.21.15.78:38992) 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1471ca7640 1 -- 172.21.15.78:0/2728567632 learned_addr learned my addr 172.21.15.78:0/2728567632 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f14724a8640 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb2410f0 0x55a1cb255d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1472ca9640 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb24d9a0 0x55a1cb253da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1476ae1740 10 client.0 _ll_register_callbacks cb 0x7f1464003c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1471ca7640 1 -- 172.21.15.78:0/2728567632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb2410f0 msgr2=0x55a1cb255d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1471ca7640 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb2410f0 0x55a1cb255d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1471ca7640 1 -- 172.21.15.78:0/2728567632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb24d9a0 msgr2=0x55a1cb253da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1471ca7640 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb24d9a0 0x55a1cb253da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1471ca7640 1 -- 172.21.15.78:0/2728567632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1cb254360 con 0x55a1cb241a10 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1471ca7640 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb253860 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f145c023790 tx=0x7f145c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8539167ebc78be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1455ffb640 1 -- 172.21.15.78:0/2728567632 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c02d420 con 0x55a1cb241a10 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1476ae1740 1 -- 172.21.15.78:0/2728567632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14640abc50 con 0x55a1cb241a10 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1455ffb640 1 -- 172.21.15.78:0/2728567632 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8493+0+0 (secure 0 0 0) 0x7f145c01a030 con 0x55a1cb241a10 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1455ffb640 1 -- 172.21.15.78:0/2728567632 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f145c02d5c0 con 0x55a1cb241a10 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1476ae1740 10 client.18012 fetch_fsmap learned FSMap version 57 2024-09-11T01:12:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.211+0000 7f1476ae1740 1 -- 172.21.15.78:0/2728567632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55a1cb2cb0d0 con 0x55a1cb241a10 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.212+0000 7f1455ffb640 1 -- 172.21.15.78:0/2728567632 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14640abc50 con 0x55a1cb241a10 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.212+0000 7f1455ffb640 1 -- 172.21.15.78:0/2728567632 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f145c034070 con 0x55a1cb241a10 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.212+0000 7f1476ae1740 10 client.18012 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.212+0000 7f1476ae1740 10 client.18012 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.212+0000 7f1476ae1740 20 client.18012 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.212+0000 7f1476ae1740 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55a1cb2cb9c0 0x55a1cb2cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.212+0000 7f1476ae1740 4 client.18012 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.212+0000 7f1476ae1740 1 -- 172.21.15.78:0/2728567632 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1cb1b58a0 con 0x55a1cb2cb9c0 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.212+0000 7f14724a8640 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55a1cb2cb9c0 0x55a1cb2cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.212+0000 7f1455ffb640 10 client.18012 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.212+0000 7f14724a8640 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55a1cb2cb9c0 0x55a1cb2cdde0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7098cd6fbd4d2c78 server_cookie=3d34d69cbb192277 in_seq=0 out_seq=0 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1455ffb640 1 -- 172.21.15.78:0/2728567632 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1cb1b58a0 con 0x55a1cb2cb9c0 2024-09-11T01:12:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1455ffb640 10 client.18012 handle_command_reply: tid=0 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1476ae1740 1 client.18012 shutdown 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1476ae1740 1 -- 172.21.15.78:0/2728567632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 msgr2=0x55a1cb253860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1476ae1740 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb253860 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f145c023790 tx=0x7f145c004730 comp rx=0 tx=0).stop 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1476ae1740 1 -- 172.21.15.78:0/2728567632 shutdown_connections 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1476ae1740 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1cb24d9a0 0x55a1cb253da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1476ae1740 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55a1cb2cb9c0 0x55a1cb2cdde0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1476ae1740 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a1cb241a10 0x55a1cb253860 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1476ae1740 1 --2- 172.21.15.78:0/2728567632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1cb2410f0 0x55a1cb255d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1476ae1740 1 -- 172.21.15.78:0/2728567632 >> 172.21.15.78:0/2728567632 conn(0x55a1cb23ef20 msgr2=0x55a1cb24bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1455ffb640 0 client.18012 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1476ae1740 1 -- 172.21.15.78:0/2728567632 shutdown_connections 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1476ae1740 1 -- 172.21.15.78:0/2728567632 wait complete. 2024-09-11T01:12:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.213+0000 7f1476ae1740 20 client.18012 trim_cache size 0 max 16384 2024-09-11T01:12:07.221 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:12:07.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.214+0000 7f147470c640 1 -- 172.21.15.78:0/3512617864 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14580b5450 msgr2=0x7f14580b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:07.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.214+0000 7f147470c640 1 --2- 172.21.15.78:0/3512617864 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14580b5450 0x7f14580b7910 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f146c107220 tx=0x7f145c00a6c0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.214+0000 7f147470c640 1 -- 172.21.15.78:0/3512617864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c108020 msgr2=0x7f146c19dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:07.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.214+0000 7f147470c640 1 --2- 172.21.15.78:0/3512617864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c108020 0x7f146c19dbb0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1464009fd0 tx=0x7f146400def0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.214+0000 7f147470c640 1 -- 172.21.15.78:0/3512617864 shutdown_connections 2024-09-11T01:12:07.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.214+0000 7f147470c640 1 --2- 172.21.15.78:0/3512617864 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14580b5450 0x7f14580b7910 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.214+0000 7f147470c640 1 --2- 172.21.15.78:0/3512617864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c108020 0x7f146c19dbb0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.214+0000 7f147470c640 1 --2- 172.21.15.78:0/3512617864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f146c1076a0 0x7f146c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.214+0000 7f147470c640 1 --2- 172.21.15.78:0/3512617864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c101c90 0x7f146c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:07.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.214+0000 7f147470c640 1 -- 172.21.15.78:0/3512617864 >> 172.21.15.78:0/3512617864 conn(0x7f146c0fd700 msgr2=0x7f146c105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:07.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.214+0000 7f147470c640 1 -- 172.21.15.78:0/3512617864 shutdown_connections 2024-09-11T01:12:07.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:07.214+0000 7f147470c640 1 -- 172.21.15.78:0/3512617864 wait complete. 2024-09-11T01:12:07.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 5784 35.83 MB/sec warmup 13 sec latency 66.799 ms 2024-09-11T01:12:07.792 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:07 smithi078 ceph-mon[31315]: pgmap v733: 97 pgs: 97 active+clean; 827 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 483 op/s 2024-09-11T01:12:07.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:07 smithi110 ceph-mon[34712]: pgmap v733: 97 pgs: 97 active+clean; 827 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 483 op/s 2024-09-11T01:12:07.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:07 smithi112 ceph-mon[35384]: pgmap v733: 97 pgs: 97 active+clean; 827 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 483 op/s 2024-09-11T01:12:08.005 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:12:08.276 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:12:08.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 6295 36.70 MB/sec warmup 14 sec latency 47.610 ms 2024-09-11T01:12:08.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.781+0000 7f24d4d41640 1 -- 172.21.15.78:0/1319237579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d0108040 msgr2=0x7f24d010e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:08.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.781+0000 7f24d4d41640 1 --2- 172.21.15.78:0/1319237579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d0108040 0x7f24d010e840 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009ca0 tx=0x7f24c802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.781+0000 7f24d4d41640 1 -- 172.21.15.78:0/1319237579 shutdown_connections 2024-09-11T01:12:08.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.781+0000 7f24d4d41640 1 --2- 172.21.15.78:0/1319237579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d0108040 0x7f24d010e840 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.781+0000 7f24d4d41640 1 --2- 172.21.15.78:0/1319237579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24d01076c0 0x7f24d0107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.781+0000 7f24d4d41640 1 --2- 172.21.15.78:0/1319237579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d0101cb0 0x7f24d0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.781+0000 7f24d4d41640 1 -- 172.21.15.78:0/1319237579 >> 172.21.15.78:0/1319237579 conn(0x7f24d00fd700 msgr2=0x7f24d00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:08.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.781+0000 7f24d4d41640 1 -- 172.21.15.78:0/1319237579 shutdown_connections 2024-09-11T01:12:08.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.781+0000 7f24d4d41640 1 -- 172.21.15.78:0/1319237579 wait complete. 2024-09-11T01:12:08.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.782+0000 7f24d4d41640 1 Processor -- start 2024-09-11T01:12:08.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24d4d41640 1 -- start start 2024-09-11T01:12:08.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24d4d41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24d0101cb0 0x7f24d0199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24d4d41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d01076c0 0x7f24d019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24d4d41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d0108040 0x7f24d01a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24d4d41640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0111d00 con 0x7f24d01076c0 2024-09-11T01:12:08.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24d4d41640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0111b80 con 0x7f24d0101cb0 2024-09-11T01:12:08.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24d4d41640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0111e80 con 0x7f24d0108040 2024-09-11T01:12:08.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24cdd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d01076c0 0x7f24d019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24ce59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24d0101cb0 0x7f24d0199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d0108040 0x7f24d01a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d0108040 0x7f24d01a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35162/0 (socket says 172.21.15.78:35162) 2024-09-11T01:12:08.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24ce59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24d0101cb0 0x7f24d0199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41198/0 (socket says 172.21.15.78:41198) 2024-09-11T01:12:08.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24ced9d640 1 -- 172.21.15.78:0/903832829 learned_addr learned my addr 172.21.15.78:0/903832829 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:08.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24ce59c640 1 -- 172.21.15.78:0/903832829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d0108040 msgr2=0x7f24d01a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:08.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24ce59c640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d0108040 0x7f24d01a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24ce59c640 1 -- 172.21.15.78:0/903832829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d01076c0 msgr2=0x7f24d019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:08.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.783+0000 7f24ce59c640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d01076c0 0x7f24d019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.784+0000 7f24ce59c640 1 -- 172.21.15.78:0/903832829 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d01a55b0 con 0x7f24d0101cb0 2024-09-11T01:12:08.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.784+0000 7f24ced9d640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d0108040 0x7f24d01a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:08.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.784+0000 7f24ce59c640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24d0101cb0 0x7f24d0199ff0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f24c000e9f0 tx=0x7f24c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:08.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.784+0000 7f24c77fe640 1 -- 172.21.15.78:0/903832829 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c000ccc0 con 0x7f24d0101cb0 2024-09-11T01:12:08.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.784+0000 7f24c77fe640 1 -- 172.21.15.78:0/903832829 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c00108e0 con 0x7f24d0101cb0 2024-09-11T01:12:08.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.784+0000 7f24d4d41640 1 -- 172.21.15.78:0/903832829 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d0197de0 con 0x7f24d0101cb0 2024-09-11T01:12:08.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.784+0000 7f24c77fe640 1 -- 172.21.15.78:0/903832829 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0018880 con 0x7f24d0101cb0 2024-09-11T01:12:08.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.784+0000 7f24d4d41640 1 -- 172.21.15.78:0/903832829 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d0198370 con 0x7f24d0101cb0 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.787+0000 7f24d7116740 1 Processor -- start 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.788+0000 7f24d7116740 1 -- start start 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.788+0000 7f24d7116740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.788+0000 7f24d7116740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb42a10 0x56316bb4e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.788+0000 7f24d7116740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4e9a0 0x56316bb50d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.788+0000 7f24d7116740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56316bab8ea0 con 0x56316bb42a10 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.788+0000 7f24d7116740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56316baa82d0 con 0x56316bb4e9a0 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.788+0000 7f24d7116740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56316baac0e0 con 0x56316bb420f0 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.788+0000 7f24ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.788+0000 7f24ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb424d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35176/0 (socket says 172.21.15.78:35176) 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.788+0000 7f24ced9d640 1 -- 172.21.15.78:0/2724250050 learned_addr learned my addr 172.21.15.78:0/2724250050 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.788+0000 7f24cdd9b640 1 --2- 172.21.15.78:0/2724250050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4e9a0 0x56316bb50d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.788+0000 7f24c77fe640 1 -- 172.21.15.78:0/903832829 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f24c0010430 con 0x7f24d0101cb0 2024-09-11T01:12:08.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.789+0000 7f24ced9d640 1 -- 172.21.15.78:0/2724250050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4e9a0 msgr2=0x56316bb50d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:08.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.789+0000 7f24ced9d640 1 --2- 172.21.15.78:0/2724250050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4e9a0 0x56316bb50d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.789+0000 7f24ced9d640 1 -- 172.21.15.78:0/2724250050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb42a10 msgr2=0x56316bb4e460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:12:08.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.789+0000 7f24ced9d640 1 --2- 172.21.15.78:0/2724250050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb42a10 0x56316bb4e460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.789+0000 7f24ced9d640 1 -- 172.21.15.78:0/2724250050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56316b806650 con 0x56316bb420f0 2024-09-11T01:12:08.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.790+0000 7f24c77fe640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f24bc0b54d0 0x7f24bc0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.791+0000 7f24c77fe640 1 -- 172.21.15.78:0/903832829 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f24c00e62b0 con 0x7f24d0101cb0 2024-09-11T01:12:08.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.791+0000 7f24ced9d640 1 --2- 172.21.15.78:0/2724250050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb424d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005f80 tx=0x7f24c802f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=662f9c5766c56735 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:08.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.791+0000 7f24ce59c640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f24bc0b54d0 0x7f24bc0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.791+0000 7f24c4ff9640 1 -- 172.21.15.78:0/2724250050 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8031450 con 0x56316bb420f0 2024-09-11T01:12:08.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.791+0000 7f24c4ff9640 1 -- 172.21.15.78:0/2724250050 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c8036040 con 0x56316bb420f0 2024-09-11T01:12:08.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.791+0000 7f24c4ff9640 1 -- 172.21.15.78:0/2724250050 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80044e0 con 0x56316bb420f0 2024-09-11T01:12:08.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.792+0000 7f24d7116740 1 -- 172.21.15.78:0/2724250050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 msgr2=0x56316bb424d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:08.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.792+0000 7f24ce59c640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f24bc0b54d0 0x7f24bc0b7990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x56316bb40e00 tx=0x7f24c0009700 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:08.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.792+0000 7f24d7116740 1 --2- 172.21.15.78:0/2724250050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb424d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005f80 tx=0x7f24c802f310 comp rx=0 tx=0).stop 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.793+0000 7f24d7116740 1 -- 172.21.15.78:0/2724250050 shutdown_connections 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.793+0000 7f24d7116740 1 --2- 172.21.15.78:0/2724250050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4e9a0 0x56316bb50d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.793+0000 7f24d7116740 1 --2- 172.21.15.78:0/2724250050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb42a10 0x56316bb4e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.793+0000 7f24d7116740 1 --2- 172.21.15.78:0/2724250050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb424d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.793+0000 7f24d7116740 1 -- 172.21.15.78:0/2724250050 >> 172.21.15.78:0/2724250050 conn(0x56316bb3ff20 msgr2=0x56316bb410c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.793+0000 7f24d7116740 1 -- 172.21.15.78:0/2724250050 shutdown_connections 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.794+0000 7f24d7116740 1 -- 172.21.15.78:0/2724250050 wait complete. 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.794+0000 7f24d7116740 1 -- start start 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.794+0000 7f24d7116740 1 Processor -- start 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24d7116740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb4e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24d7116740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb42a10 0x56316bb4b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24d7116740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4e9a0 0x56316bb4bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24d7116740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56316b909700 con 0x56316bb42a10 2024-09-11T01:12:08.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24d7116740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56316ba43160 con 0x56316bb4e9a0 2024-09-11T01:12:08.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24d7116740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56316bb51440 con 0x56316bb420f0 2024-09-11T01:12:08.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24d7116740 10 client.0 _ll_register_callbacks cb 0x56316ba5c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:08.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb4e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb4e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35178/0 (socket says 172.21.15.78:35178) 2024-09-11T01:12:08.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24ced9d640 1 -- 172.21.15.78:0/459630562 learned_addr learned my addr 172.21.15.78:0/459630562 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:08.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24cdd9b640 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb42a10 0x56316bb4b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24ced9d640 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4e9a0 0x56316bb4bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24cdd9b640 1 -- 172.21.15.78:0/459630562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 msgr2=0x56316bb4e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:08.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24cdd9b640 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb4e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24cdd9b640 1 -- 172.21.15.78:0/459630562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4e9a0 msgr2=0x56316bb4bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:08.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.795+0000 7f24cdd9b640 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4e9a0 0x56316bb4bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.796+0000 7f24cdd9b640 1 -- 172.21.15.78:0/459630562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56316ba70c90 con 0x56316bb42a10 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.796+0000 7f24cdd9b640 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb42a10 0x56316bb4b740 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009760 tx=0x7f24b8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9851182186f50a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.796+0000 7f24b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.796+0000 7f24b5ffb640 1 -- 172.21.15.78:0/459630562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8004050 con 0x56316bb42a10 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.796+0000 7f24d7116740 1 -- 172.21.15.78:0/459630562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c8036040 con 0x56316bb42a10 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.796+0000 7f24b5ffb640 1 -- 172.21.15.78:0/459630562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f24b80041f0 con 0x56316bb42a10 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.796+0000 7f24b5ffb640 1 -- 172.21.15.78:0/459630562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f24b80043f0 con 0x56316bb42a10 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.796+0000 7f24d7116740 10 client.18021 fetch_fsmap learned FSMap version 57 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.796+0000 7f24d7116740 1 -- 172.21.15.78:0/459630562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56316bb65670 con 0x56316bb42a10 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.797+0000 7f24b5ffb640 1 -- 172.21.15.78:0/459630562 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c8036040 con 0x56316bb42a10 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.797+0000 7f24b5ffb640 1 -- 172.21.15.78:0/459630562 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f24b8021030 con 0x56316bb42a10 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.797+0000 7f24d7116740 10 client.18021 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T01:12:08.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.797+0000 7f24d7116740 10 client.18021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:12:08.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.797+0000 7f24d7116740 20 client.18021 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:08.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.797+0000 7f24d7116740 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316bb66b70 0x56316bb68f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.797+0000 7f24d7116740 4 client.18021 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:12:08.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.797+0000 7f24d7116740 1 -- 172.21.15.78:0/459630562 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56316babadf0 con 0x56316bb66b70 2024-09-11T01:12:08.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.798+0000 7f24ced9d640 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316bb66b70 0x56316bb68f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.799+0000 7f24ced9d640 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316bb66b70 0x56316bb68f60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b88fb154e155515 server_cookie=e0a8901b327b3162 in_seq=0 out_seq=0 2024-09-11T01:12:08.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.799+0000 7f24b5ffb640 10 client.18021 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:08.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.803+0000 7f24b5ffb640 1 -- 172.21.15.78:0/459630562 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56316babadf0 con 0x56316bb66b70 2024-09-11T01:12:08.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.803+0000 7f24b5ffb640 10 client.18021 handle_command_reply: tid=0 2024-09-11T01:12:08.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.803+0000 7f24d7116740 1 client.18021 shutdown 2024-09-11T01:12:08.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.804+0000 7f24d7116740 1 -- 172.21.15.78:0/459630562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb42a10 msgr2=0x56316bb4b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:08.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.804+0000 7f24d7116740 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb42a10 0x56316bb4b740 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009760 tx=0x7f24b8009c20 comp rx=0 tx=0).stop 2024-09-11T01:12:08.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.804+0000 7f24d7116740 1 -- 172.21.15.78:0/459630562 shutdown_connections 2024-09-11T01:12:08.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.804+0000 7f24d7116740 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4e9a0 0x56316bb4bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.804+0000 7f24d7116740 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316bb66b70 0x56316bb68f60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.804+0000 7f24d7116740 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb42a10 0x56316bb4b740 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.804+0000 7f24d7116740 1 --2- 172.21.15.78:0/459630562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb4e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.804+0000 7f24d7116740 1 -- 172.21.15.78:0/459630562 >> 172.21.15.78:0/459630562 conn(0x56316bb3ff20 msgr2=0x56316bb45ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:08.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.804+0000 7f24b5ffb640 0 client.18021 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:08.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.804+0000 7f24d7116740 1 -- 172.21.15.78:0/459630562 shutdown_connections 2024-09-11T01:12:08.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.804+0000 7f24d7116740 1 -- 172.21.15.78:0/459630562 wait complete. 2024-09-11T01:12:08.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.804+0000 7f24d7116740 20 client.18021 trim_cache size 0 max 16384 2024-09-11T01:12:08.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.825+0000 7f24d7116740 1 Processor -- start 2024-09-11T01:12:08.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.825+0000 7f24d7116740 1 -- start start 2024-09-11T01:12:08.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.825+0000 7f24d7116740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb5a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.825+0000 7f24d7116740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb4e9a0 0x56316bb40fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb5a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24d7116740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4a6d0 0x56316bb48950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24d7116740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ac3c0 con 0x56316bb4e9a0 2024-09-11T01:12:08.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb5a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35188/0 (socket says 172.21.15.78:35188) 2024-09-11T01:12:08.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24d7116740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ae080 con 0x56316bb4a6d0 2024-09-11T01:12:08.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24ced9d640 1 -- 172.21.15.78:0/241951805 learned_addr learned my addr 172.21.15.78:0/241951805 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24d7116740 1 -- 172.21.15.78:0/241951805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01aca90 con 0x56316bb420f0 2024-09-11T01:12:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24cdd9b640 1 --2- 172.21.15.78:0/241951805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb4e9a0 0x56316bb40fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24ced9d640 1 -- 172.21.15.78:0/241951805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4a6d0 msgr2=0x56316bb48950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24ced9d640 1 --2- 172.21.15.78:0/241951805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4a6d0 0x56316bb48950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24ced9d640 1 -- 172.21.15.78:0/241951805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb4e9a0 msgr2=0x56316bb40fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24ced9d640 1 --2- 172.21.15.78:0/241951805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb4e9a0 0x56316bb40fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24ced9d640 1 -- 172.21.15.78:0/241951805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56316bb42840 con 0x56316bb420f0 2024-09-11T01:12:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24cdd9b640 1 --2- 172.21.15.78:0/241951805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb4e9a0 0x56316bb40fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.826+0000 7f24ced9d640 1 --2- 172.21.15.78:0/241951805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb5a670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24c8003f20 tx=0x7f24c802a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e6579ac9699c860 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.827+0000 7f24b7fff640 1 -- 172.21.15.78:0/241951805 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c802f380 con 0x56316bb420f0 2024-09-11T01:12:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.827+0000 7f24b7fff640 1 -- 172.21.15.78:0/241951805 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c8036030 con 0x56316bb420f0 2024-09-11T01:12:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.827+0000 7f24b7fff640 1 -- 172.21.15.78:0/241951805 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8004820 con 0x56316bb420f0 2024-09-11T01:12:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.828+0000 7f24d7116740 1 -- 172.21.15.78:0/241951805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 msgr2=0x56316bb5a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.828+0000 7f24d7116740 1 --2- 172.21.15.78:0/241951805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb5a670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24c8003f20 tx=0x7f24c802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.828+0000 7f24d7116740 1 -- 172.21.15.78:0/241951805 shutdown_connections 2024-09-11T01:12:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.828+0000 7f24d7116740 1 --2- 172.21.15.78:0/241951805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb4a6d0 0x56316bb48950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.828+0000 7f24d7116740 1 --2- 172.21.15.78:0/241951805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb4e9a0 0x56316bb40fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.828+0000 7f24d7116740 1 --2- 172.21.15.78:0/241951805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb420f0 0x56316bb5a670 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.828+0000 7f24d7116740 1 -- 172.21.15.78:0/241951805 >> 172.21.15.78:0/241951805 conn(0x56316bb3ff20 msgr2=0x56316bb49fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.828+0000 7f24d7116740 1 -- 172.21.15.78:0/241951805 shutdown_connections 2024-09-11T01:12:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.828+0000 7f24d7116740 1 -- 172.21.15.78:0/241951805 wait complete. 2024-09-11T01:12:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.828+0000 7f24d7116740 1 -- start start 2024-09-11T01:12:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.828+0000 7f24d7116740 1 Processor -- start 2024-09-11T01:12:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.829+0000 7f24d7116740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb420f0 0x56316bb56ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.829+0000 7f24d7116740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb4a6d0 0x56316bb51690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.829+0000 7f24d7116740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb4e9a0 0x56316bb51c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.829+0000 7f24d7116740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ad9b0 con 0x56316bb4a6d0 2024-09-11T01:12:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.829+0000 7f24d7116740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ad220 con 0x56316bb420f0 2024-09-11T01:12:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.829+0000 7f24d7116740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01aa3c0 con 0x56316bb4e9a0 2024-09-11T01:12:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.829+0000 7f24d7116740 10 client.0 _ll_register_callbacks cb 0x7f24b8010950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.829+0000 7f24ced9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb420f0 0x56316bb56ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.829+0000 7f24ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb4e9a0 0x56316bb51c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.829+0000 7f24ced9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb420f0 0x56316bb56ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41230/0 (socket says 172.21.15.78:41230) 2024-09-11T01:12:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.829+0000 7f24ced9d640 1 -- 172.21.15.78:0/2022570404 learned_addr learned my addr 172.21.15.78:0/2022570404 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.829+0000 7f24cdd9b640 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb4a6d0 0x56316bb51690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.830+0000 7f24cdd9b640 1 -- 172.21.15.78:0/2022570404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb4e9a0 msgr2=0x56316bb51c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.830+0000 7f24cdd9b640 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb4e9a0 0x56316bb51c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.830+0000 7f24cdd9b640 1 -- 172.21.15.78:0/2022570404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb420f0 msgr2=0x56316bb56ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.830+0000 7f24cdd9b640 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb420f0 0x56316bb56ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.830+0000 7f24cdd9b640 1 -- 172.21.15.78:0/2022570404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56316bb52440 con 0x56316bb4a6d0 2024-09-11T01:12:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.830+0000 7f24ced9d640 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb4e9a0 0x56316bb51c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:12:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.830+0000 7f24cdd9b640 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb4a6d0 0x56316bb51690 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f24b8008000 tx=0x7f24b8022490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2564a90ed19bf358 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.830+0000 7f24b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:12:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.830+0000 7f24b5ffb640 1 -- 172.21.15.78:0/2022570404 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8003f40 con 0x56316bb4a6d0 2024-09-11T01:12:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.830+0000 7f24b5ffb640 1 -- 172.21.15.78:0/2022570404 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f24b80040e0 con 0x56316bb4a6d0 2024-09-11T01:12:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.830+0000 7f24d7116740 1 -- 172.21.15.78:0/2022570404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c8036030 con 0x56316bb4a6d0 2024-09-11T01:12:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.831+0000 7f24b5ffb640 1 -- 172.21.15.78:0/2022570404 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f24b8005040 con 0x56316bb4a6d0 2024-09-11T01:12:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.831+0000 7f24d7116740 10 client.18027 fetch_fsmap learned FSMap version 57 2024-09-11T01:12:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.831+0000 7f24d7116740 1 -- 172.21.15.78:0/2022570404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56316bb63d40 con 0x56316bb4a6d0 2024-09-11T01:12:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.831+0000 7f24b5ffb640 1 -- 172.21.15.78:0/2022570404 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c8036030 con 0x56316bb4a6d0 2024-09-11T01:12:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.831+0000 7f24b5ffb640 1 -- 172.21.15.78:0/2022570404 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f24b8021070 con 0x56316bb4a6d0 2024-09-11T01:12:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.832+0000 7f24d7116740 10 client.18027 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T01:12:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.832+0000 7f24d7116740 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:12:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.832+0000 7f24d7116740 20 client.18027 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.832+0000 7f24d7116740 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316bb64510 0x56316bb64920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.832+0000 7f24d7116740 4 client.18027 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:12:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.832+0000 7f24d7116740 1 -- 172.21.15.78:0/2022570404 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56316bab68a0 con 0x56316bb64510 2024-09-11T01:12:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.833+0000 7f24ced9d640 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316bb64510 0x56316bb64920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.834+0000 7f24ced9d640 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316bb64510 0x56316bb64920 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c746d308d879b6da server_cookie=50b44e01180a68ab in_seq=0 out_seq=0 2024-09-11T01:12:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.834+0000 7f24b5ffb640 10 client.18027 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.837+0000 7f24b5ffb640 1 -- 172.21.15.78:0/2022570404 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56316bab68a0 con 0x56316bb64510 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.837+0000 7f24b5ffb640 10 client.18027 handle_command_reply: tid=0 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.838+0000 7f24d7116740 1 client.18027 shutdown 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.838+0000 7f24d7116740 1 -- 172.21.15.78:0/2022570404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb4a6d0 msgr2=0x56316bb51690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.838+0000 7f24d7116740 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb4a6d0 0x56316bb51690 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f24b8008000 tx=0x7f24b8022490 comp rx=0 tx=0).stop 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.838+0000 7f24d7116740 1 -- 172.21.15.78:0/2022570404 shutdown_connections 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.838+0000 7f24d7116740 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316bb4e9a0 0x56316bb51c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.838+0000 7f24d7116740 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316bb64510 0x56316bb64920 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.838+0000 7f24b5ffb640 0 client.18027 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.838+0000 7f24d7116740 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316bb4a6d0 0x56316bb51690 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.838+0000 7f24d7116740 1 --2- 172.21.15.78:0/2022570404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316bb420f0 0x56316bb56ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.838+0000 7f24d7116740 1 -- 172.21.15.78:0/2022570404 >> 172.21.15.78:0/2022570404 conn(0x56316bb3ff20 msgr2=0x56316bb4cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.838+0000 7f24d7116740 1 -- 172.21.15.78:0/2022570404 shutdown_connections 2024-09-11T01:12:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.839+0000 7f24d7116740 1 -- 172.21.15.78:0/2022570404 wait complete. 2024-09-11T01:12:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.839+0000 7f24d7116740 20 client.18027 trim_cache size 0 max 16384 2024-09-11T01:12:08.842 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:12:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.840+0000 7f24d4d41640 1 -- 172.21.15.78:0/903832829 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f24bc0b54d0 msgr2=0x7f24bc0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.840+0000 7f24d4d41640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f24bc0b54d0 0x7f24bc0b7990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x56316bb40e00 tx=0x7f24c0009700 comp rx=0 tx=0).stop 2024-09-11T01:12:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.840+0000 7f24d4d41640 1 -- 172.21.15.78:0/903832829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24d0101cb0 msgr2=0x7f24d0199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.840+0000 7f24d4d41640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24d0101cb0 0x7f24d0199ff0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f24c000e9f0 tx=0x7f24c000eeb0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.840+0000 7f24d4d41640 1 -- 172.21.15.78:0/903832829 shutdown_connections 2024-09-11T01:12:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.840+0000 7f24d4d41640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f24bc0b54d0 0x7f24bc0b7990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.840+0000 7f24d4d41640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d0108040 0x7f24d01a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.840+0000 7f24d4d41640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d01076c0 0x7f24d019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.840+0000 7f24d4d41640 1 --2- 172.21.15.78:0/903832829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24d0101cb0 0x7f24d0199ff0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:08.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.840+0000 7f24d4d41640 1 -- 172.21.15.78:0/903832829 >> 172.21.15.78:0/903832829 conn(0x7f24d00fd700 msgr2=0x7f24d00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:08.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.841+0000 7f24d4d41640 1 -- 172.21.15.78:0/903832829 shutdown_connections 2024-09-11T01:12:08.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:08.841+0000 7f24d4d41640 1 -- 172.21.15.78:0/903832829 wait complete. 2024-09-11T01:12:09.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:09 smithi078 ceph-mon[31315]: pgmap v734: 97 pgs: 97 active+clean; 820 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 468 op/s 2024-09-11T01:12:09.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:09 smithi078 ceph-mon[31315]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T01:12:09.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:09 smithi110 ceph-mon[34712]: pgmap v734: 97 pgs: 97 active+clean; 820 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 468 op/s 2024-09-11T01:12:09.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:09 smithi110 ceph-mon[34712]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T01:12:09.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:09 smithi112 ceph-mon[35384]: pgmap v734: 97 pgs: 97 active+clean; 820 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 468 op/s 2024-09-11T01:12:09.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:09 smithi112 ceph-mon[35384]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T01:12:09.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 6832 37.24 MB/sec warmup 15 sec latency 67.486 ms 2024-09-11T01:12:10.759 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:12:10.760 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:12:10.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 7110 35.78 MB/sec warmup 16 sec latency 75.960 ms 2024-09-11T01:12:11.027 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:12:11.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.595+0000 7ff9e207d640 1 -- 172.21.15.78:0/1558066343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9dc1045c0 msgr2=0x7ff9dc10cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:11.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.595+0000 7ff9e207d640 1 --2- 172.21.15.78:0/1558066343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9dc1045c0 0x7ff9dc10cab0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009900 tx=0x7ff9d002a410 comp rx=0 tx=0).stop 2024-09-11T01:12:11.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.595+0000 7ff9e207d640 1 -- 172.21.15.78:0/1558066343 shutdown_connections 2024-09-11T01:12:11.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.595+0000 7ff9e207d640 1 --2- 172.21.15.78:0/1558066343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9dc10d100 0x7ff9dc10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.595+0000 7ff9e207d640 1 --2- 172.21.15.78:0/1558066343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9dc1045c0 0x7ff9dc10cab0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.595+0000 7ff9e207d640 1 --2- 172.21.15.78:0/1558066343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9dc103ca0 0x7ff9dc104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.595+0000 7ff9e207d640 1 -- 172.21.15.78:0/1558066343 >> 172.21.15.78:0/1558066343 conn(0x7ff9dc0fd700 msgr2=0x7ff9dc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:11.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.595+0000 7ff9e207d640 1 -- 172.21.15.78:0/1558066343 shutdown_connections 2024-09-11T01:12:11.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.595+0000 7ff9e207d640 1 -- 172.21.15.78:0/1558066343 wait complete. 2024-09-11T01:12:11.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9e207d640 1 Processor -- start 2024-09-11T01:12:11.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9e207d640 1 -- start start 2024-09-11T01:12:11.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9e207d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9dc103ca0 0x7ff9dc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9e207d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9dc1045c0 0x7ff9dc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9e207d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9dc10d100 0x7ff9dc19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9e207d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc112dc0 con 0x7ff9dc10d100 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9e207d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc112c40 con 0x7ff9dc1045c0 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9e207d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc112f40 con 0x7ff9dc103ca0 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9dc10d100 0x7ff9dc19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9daffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9dc1045c0 0x7ff9dc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9dc10d100 0x7ff9dc19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39066/0 (socket says 172.21.15.78:39066) 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9daffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9dc1045c0 0x7ff9dc199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41246/0 (socket says 172.21.15.78:41246) 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9dbfff640 1 -- 172.21.15.78:0/3323658160 learned_addr learned my addr 172.21.15.78:0/3323658160 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.596+0000 7ff9db7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9dc103ca0 0x7ff9dc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9dbfff640 1 -- 172.21.15.78:0/3323658160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9dc103ca0 msgr2=0x7ff9dc1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9dbfff640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9dc103ca0 0x7ff9dc1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9dbfff640 1 -- 172.21.15.78:0/3323658160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9dc1045c0 msgr2=0x7ff9dc199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9dbfff640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9dc1045c0 0x7ff9dc199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9dbfff640 1 -- 172.21.15.78:0/3323658160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc19f3a0 con 0x7ff9dc10d100 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9db7fe640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9dc103ca0 0x7ff9dc1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9daffd640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9dc1045c0 0x7ff9dc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9dbfff640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9dc10d100 0x7ff9dc19ebd0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400eaa0 tx=0x7ff9d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9d8ff9640 1 -- 172.21.15.78:0/3323658160 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400cdb0 con 0x7ff9dc10d100 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9d8ff9640 1 -- 172.21.15.78:0/3323658160 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9d4005020 con 0x7ff9dc10d100 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9e207d640 1 -- 172.21.15.78:0/3323658160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc19f690 con 0x7ff9dc10d100 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9d8ff9640 1 -- 172.21.15.78:0/3323658160 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40116d0 con 0x7ff9dc10d100 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.597+0000 7ff9e207d640 1 -- 172.21.15.78:0/3323658160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc1a5d50 con 0x7ff9dc10d100 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9d8ff9640 1 -- 172.21.15.78:0/3323658160 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff9d400f040 con 0x7ff9dc10d100 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9e4452740 1 Processor -- start 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9e4452740 1 -- start start 2024-09-11T01:12:11.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9e4452740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b7ee0 0x5563ae7b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9e4452740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7b8800 0x5563ae7c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9e4452740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7c4790 0x5563ae7c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9e4452740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5563ae72ee60 con 0x5563ae7b8800 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9e4452740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5563ae71e290 con 0x5563ae7b7ee0 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9e4452740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5563ae7220a0 con 0x5563ae7c4790 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7b8800 0x5563ae7c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7b8800 0x5563ae7c4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39076/0 (socket says 172.21.15.78:39076) 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9db7fe640 1 -- 172.21.15.78:0/2747833592 learned_addr learned my addr 172.21.15.78:0/2747833592 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9daffd640 1 --2- 172.21.15.78:0/2747833592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b7ee0 0x5563ae7b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9daffd640 1 --2- 172.21.15.78:0/2747833592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7c4790 0x5563ae7c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9db7fe640 1 -- 172.21.15.78:0/2747833592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7c4790 msgr2=0x5563ae7c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9db7fe640 1 --2- 172.21.15.78:0/2747833592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7c4790 0x5563ae7c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9db7fe640 1 -- 172.21.15.78:0/2747833592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b7ee0 msgr2=0x5563ae7b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9db7fe640 1 --2- 172.21.15.78:0/2747833592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b7ee0 0x5563ae7b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9db7fe640 1 -- 172.21.15.78:0/2747833592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ae47c930 con 0x5563ae7b8800 2024-09-11T01:12:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.600+0000 7ff9daffd640 1 --2- 172.21.15.78:0/2747833592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7c4790 0x5563ae7c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:12:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.601+0000 7ff9d8ff9640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff9c40b5450 0x7ff9c40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.601+0000 7ff9daffd640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff9c40b5450 0x7ff9c40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.601+0000 7ff9db7fe640 1 --2- 172.21.15.78:0/2747833592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7b8800 0x5563ae7c4250 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc011620 tx=0x7ff9cc011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f1f708df5e6e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.601+0000 7ff9d8ff9640 1 -- 172.21.15.78:0/3323658160 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7ff9d40e5d30 con 0x7ff9dc10d100 2024-09-11T01:12:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.601+0000 7ff9c1ffb640 1 -- 172.21.15.78:0/2747833592 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc01e070 con 0x5563ae7b8800 2024-09-11T01:12:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.601+0000 7ff9c1ffb640 1 -- 172.21.15.78:0/2747833592 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9cc00fb10 con 0x5563ae7b8800 2024-09-11T01:12:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.601+0000 7ff9c1ffb640 1 -- 172.21.15.78:0/2747833592 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc019c20 con 0x5563ae7b8800 2024-09-11T01:12:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.602+0000 7ff9e4452740 1 -- 172.21.15.78:0/2747833592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7b8800 msgr2=0x5563ae7c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.602+0000 7ff9e4452740 1 --2- 172.21.15.78:0/2747833592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7b8800 0x5563ae7c4250 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc011620 tx=0x7ff9cc011ae0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.602+0000 7ff9daffd640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff9c40b5450 0x7ff9c40b7910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0031440 tx=0x7ff9d0041040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.602+0000 7ff9e4452740 1 -- 172.21.15.78:0/2747833592 shutdown_connections 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.602+0000 7ff9e4452740 1 --2- 172.21.15.78:0/2747833592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7c4790 0x5563ae7c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.602+0000 7ff9e4452740 1 --2- 172.21.15.78:0/2747833592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7b8800 0x5563ae7c4250 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.602+0000 7ff9e4452740 1 --2- 172.21.15.78:0/2747833592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b7ee0 0x5563ae7b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.602+0000 7ff9e4452740 1 -- 172.21.15.78:0/2747833592 >> 172.21.15.78:0/2747833592 conn(0x5563ae7b5d10 msgr2=0x5563ae7b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.602+0000 7ff9e4452740 1 -- 172.21.15.78:0/2747833592 shutdown_connections 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.602+0000 7ff9e4452740 1 -- 172.21.15.78:0/2747833592 wait complete. 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.602+0000 7ff9e4452740 1 -- start start 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.602+0000 7ff9e4452740 1 Processor -- start 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9e4452740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b7ee0 0x5563ae7c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9e4452740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7b8800 0x5563ae7c1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9e4452740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7c4790 0x5563ae7c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9e4452740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5563ae57f890 con 0x5563ae7c4790 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9e4452740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5563ae6b8d40 con 0x5563ae7b7ee0 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9e4452740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5563ae7c7440 con 0x5563ae7b8800 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9e4452740 10 client.0 _ll_register_callbacks cb 0x5563ae71cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9db7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b7ee0 0x5563ae7c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9db7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b7ee0 0x5563ae7c1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41268/0 (socket says 172.21.15.78:41268) 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9db7fe640 1 -- 172.21.15.78:0/2288019823 learned_addr learned my addr 172.21.15.78:0/2288019823 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9dbfff640 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7c4790 0x5563ae7c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9daffd640 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7b8800 0x5563ae7c1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9db7fe640 1 -- 172.21.15.78:0/2288019823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7b8800 msgr2=0x5563ae7c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9db7fe640 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7b8800 0x5563ae7c1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9db7fe640 1 -- 172.21.15.78:0/2288019823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7c4790 msgr2=0x5563ae7c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9db7fe640 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7c4790 0x5563ae7c21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9db7fe640 1 -- 172.21.15.78:0/2288019823 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ae7d0820 con 0x5563ae7b7ee0 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9dbfff640 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7c4790 0x5563ae7c21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9db7fe640 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b7ee0 0x5563ae7c1760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc019880 tx=0x7ff9cc019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5f44487f7c8bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.603+0000 7ff9baffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9baffd640 1 -- 172.21.15.78:0/2288019823 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc027060 con 0x5563ae7b7ee0 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9baffd640 1 -- 172.21.15.78:0/2288019823 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7ff9cc00f720 con 0x5563ae7b7ee0 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9e4452740 1 -- 172.21.15.78:0/2288019823 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9cc00fb10 con 0x5563ae7b7ee0 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9baffd640 1 -- 172.21.15.78:0/2288019823 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7ff9cc01e040 con 0x5563ae7b7ee0 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9e4452740 10 client.27230 fetch_fsmap learned FSMap version 57 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9e4452740 1 -- 172.21.15.78:0/2288019823 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5563ae7d31d0 con 0x5563ae7b7ee0 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9baffd640 1 -- 172.21.15.78:0/2288019823 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9cc00fb10 con 0x5563ae7b7ee0 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9baffd640 1 -- 172.21.15.78:0/2288019823 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7ff9cc021030 con 0x5563ae7b7ee0 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9e4452740 10 client.27230 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T01:12:11.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9e4452740 10 client.27230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9e4452740 20 client.27230 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9e4452740 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5563ae7d4630 0x5563ae7d6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9e4452740 4 client.27230 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9e4452740 1 -- 172.21.15.78:0/2288019823 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563ae730db0 con 0x5563ae7d4630 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.604+0000 7ff9daffd640 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5563ae7d4630 0x5563ae7d6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.605+0000 7ff9daffd640 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5563ae7d4630 0x5563ae7d6a20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27dabf537523b4d1 server_cookie=2e602dcf1f3eddb4 in_seq=0 out_seq=0 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.605+0000 7ff9baffd640 10 client.27230 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.612+0000 7ff9baffd640 1 -- 172.21.15.78:0/2288019823 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5563ae730db0 con 0x5563ae7d4630 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.612+0000 7ff9baffd640 10 client.27230 handle_command_reply: tid=0 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.612+0000 7ff9e4452740 1 client.27230 shutdown 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.612+0000 7ff9e4452740 1 -- 172.21.15.78:0/2288019823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b7ee0 msgr2=0x5563ae7c1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.612+0000 7ff9e4452740 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b7ee0 0x5563ae7c1760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc019880 tx=0x7ff9cc019640 comp rx=0 tx=0).stop 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.612+0000 7ff9e4452740 1 -- 172.21.15.78:0/2288019823 shutdown_connections 2024-09-11T01:12:11.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.612+0000 7ff9e4452740 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5563ae7d4630 0x5563ae7d6a20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.612+0000 7ff9e4452740 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7c4790 0x5563ae7c21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.612+0000 7ff9e4452740 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7b8800 0x5563ae7c1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.612+0000 7ff9e4452740 1 --2- 172.21.15.78:0/2288019823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b7ee0 0x5563ae7c1760 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.612+0000 7ff9e4452740 1 -- 172.21.15.78:0/2288019823 >> 172.21.15.78:0/2288019823 conn(0x5563ae7b5d10 msgr2=0x5563ae7b4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:11.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.612+0000 7ff9baffd640 0 client.27230 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:11.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.613+0000 7ff9e4452740 1 -- 172.21.15.78:0/2288019823 shutdown_connections 2024-09-11T01:12:11.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.613+0000 7ff9e4452740 1 -- 172.21.15.78:0/2288019823 wait complete. 2024-09-11T01:12:11.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.613+0000 7ff9e4452740 20 client.27230 trim_cache size 0 max 16384 2024-09-11T01:12:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.631+0000 7ff9e4452740 1 Processor -- start 2024-09-11T01:12:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.631+0000 7ff9e4452740 1 -- start start 2024-09-11T01:12:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.631+0000 7ff9e4452740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7b7ee0 0x5563ae7bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.631+0000 7ff9e4452740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 0x5563ae7dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.631+0000 7ff9e4452740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7c4790 0x5563ae7dc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.631+0000 7ff9e4452740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1ae6b0 con 0x5563ae7c4790 2024-09-11T01:12:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.631+0000 7ff9e4452740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1b0370 con 0x5563ae7b8800 2024-09-11T01:12:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.631+0000 7ff9e4452740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1aed80 con 0x5563ae7b7ee0 2024-09-11T01:12:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.631+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7c4790 0x5563ae7dc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9daffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 0x5563ae7dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9daffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 0x5563ae7dc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41274/0 (socket says 172.21.15.78:41274) 2024-09-11T01:12:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9daffd640 1 -- 172.21.15.78:0/600698274 learned_addr learned my addr 172.21.15.78:0/600698274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9db7fe640 1 --2- 172.21.15.78:0/600698274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7b7ee0 0x5563ae7bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9daffd640 1 -- 172.21.15.78:0/600698274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7b7ee0 msgr2=0x5563ae7bf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9daffd640 1 --2- 172.21.15.78:0/600698274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7b7ee0 0x5563ae7bf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9daffd640 1 -- 172.21.15.78:0/600698274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7c4790 msgr2=0x5563ae7dc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9daffd640 1 --2- 172.21.15.78:0/600698274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7c4790 0x5563ae7dc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9daffd640 1 -- 172.21.15.78:0/600698274 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ae7b6790 con 0x5563ae7b8800 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9dbfff640 1 --2- 172.21.15.78:0/600698274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7c4790 0x5563ae7dc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9daffd640 1 --2- 172.21.15.78:0/600698274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 0x5563ae7dc1b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009900 tx=0x7ff9d0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c886b3d2e4fe79c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9c17fa640 1 -- 172.21.15.78:0/600698274 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0046070 con 0x5563ae7b8800 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9c17fa640 1 -- 172.21.15.78:0/600698274 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9d0004cf0 con 0x5563ae7b8800 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.632+0000 7ff9c17fa640 1 -- 172.21.15.78:0/600698274 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d00328b0 con 0x5563ae7b8800 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.633+0000 7ff9e4452740 1 -- 172.21.15.78:0/600698274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 msgr2=0x5563ae7dc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.633+0000 7ff9e4452740 1 --2- 172.21.15.78:0/600698274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 0x5563ae7dc1b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009900 tx=0x7ff9d0009260 comp rx=0 tx=0).stop 2024-09-11T01:12:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.633+0000 7ff9e4452740 1 -- 172.21.15.78:0/600698274 shutdown_connections 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.633+0000 7ff9e4452740 1 --2- 172.21.15.78:0/600698274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7c4790 0x5563ae7dc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.633+0000 7ff9e4452740 1 --2- 172.21.15.78:0/600698274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 0x5563ae7dc1b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.633+0000 7ff9e4452740 1 --2- 172.21.15.78:0/600698274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7b7ee0 0x5563ae7bf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.633+0000 7ff9e4452740 1 -- 172.21.15.78:0/600698274 >> 172.21.15.78:0/600698274 conn(0x5563ae7b5d10 msgr2=0x5563ae7b7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.633+0000 7ff9e4452740 1 -- 172.21.15.78:0/600698274 shutdown_connections 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.633+0000 7ff9e4452740 1 -- 172.21.15.78:0/600698274 wait complete. 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.633+0000 7ff9e4452740 1 -- start start 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9e4452740 1 Processor -- start 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9e4452740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7b7ee0 0x5563ae7c8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9e4452740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 0x5563ae7c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9e4452740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7c4790 0x5563ae7c9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9e4452740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1afca0 con 0x5563ae7b7ee0 2024-09-11T01:12:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9e4452740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1af510 con 0x5563ae7b8800 2024-09-11T01:12:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9e4452740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1ac6b0 con 0x5563ae7c4790 2024-09-11T01:12:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9e4452740 10 client.0 _ll_register_callbacks cb 0x7ff9cc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9daffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 0x5563ae7c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7b7ee0 0x5563ae7c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9daffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 0x5563ae7c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41282/0 (socket says 172.21.15.78:41282) 2024-09-11T01:12:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9daffd640 1 -- 172.21.15.78:0/192723216 learned_addr learned my addr 172.21.15.78:0/192723216 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9dbfff640 1 --2- 172.21.15.78:0/192723216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7c4790 0x5563ae7c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9daffd640 1 -- 172.21.15.78:0/192723216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7c4790 msgr2=0x5563ae7c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9daffd640 1 --2- 172.21.15.78:0/192723216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7c4790 0x5563ae7c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9daffd640 1 -- 172.21.15.78:0/192723216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7b7ee0 msgr2=0x5563ae7c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9daffd640 1 --2- 172.21.15.78:0/192723216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7b7ee0 0x5563ae7c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9daffd640 1 -- 172.21.15.78:0/192723216 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ae840460 con 0x5563ae7b8800 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9daffd640 1 --2- 172.21.15.78:0/192723216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 0x5563ae7c8bd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0000c00 tx=0x7ff9d0005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bd0ed18f3b9b901 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.634+0000 7ff9baffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9baffd640 1 -- 172.21.15.78:0/192723216 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d006d060 con 0x5563ae7b8800 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9baffd640 1 -- 172.21.15.78:0/192723216 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7ff9d0032630 con 0x5563ae7b8800 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9e4452740 1 -- 172.21.15.78:0/192723216 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d0004cf0 con 0x5563ae7b8800 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9baffd640 1 -- 172.21.15.78:0/192723216 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7ff9d0046040 con 0x5563ae7b8800 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9e4452740 10 client.27236 fetch_fsmap learned FSMap version 57 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9e4452740 1 -- 172.21.15.78:0/192723216 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5563ae8417a0 con 0x5563ae7b8800 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9baffd640 1 -- 172.21.15.78:0/192723216 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9d0004cf0 con 0x5563ae7b8800 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9baffd640 1 -- 172.21.15.78:0/192723216 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7ff9d0003bc0 con 0x5563ae7b8800 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9e4452740 10 client.27236 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9e4452740 10 client.27236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9e4452740 20 client.27236 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9e4452740 1 --2- 172.21.15.78:0/192723216 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5563ae841ee0 0x5563ae844300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:11.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9e4452740 4 client.27236 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:12:11.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.635+0000 7ff9e4452740 1 -- 172.21.15.78:0/192723216 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5563ae72c860 con 0x5563ae841ee0 2024-09-11T01:12:11.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.636+0000 7ff9db7fe640 1 --2- 172.21.15.78:0/192723216 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5563ae841ee0 0x5563ae844300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:11.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.636+0000 7ff9baffd640 10 client.27236 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:11.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.636+0000 7ff9db7fe640 1 --2- 172.21.15.78:0/192723216 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5563ae841ee0 0x5563ae844300 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1489dc337b88581b server_cookie=110943cd2a5fd362 in_seq=0 out_seq=0 2024-09-11T01:12:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9baffd640 1 -- 172.21.15.78:0/192723216 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5563ae72c860 con 0x5563ae841ee0 2024-09-11T01:12:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9baffd640 10 client.27236 handle_command_reply: tid=0 2024-09-11T01:12:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9e4452740 1 client.27236 shutdown 2024-09-11T01:12:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9e4452740 1 -- 172.21.15.78:0/192723216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 msgr2=0x5563ae7c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9e4452740 1 --2- 172.21.15.78:0/192723216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 0x5563ae7c8bd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0000c00 tx=0x7ff9d0005d90 comp rx=0 tx=0).stop 2024-09-11T01:12:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9e4452740 1 -- 172.21.15.78:0/192723216 shutdown_connections 2024-09-11T01:12:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9e4452740 1 --2- 172.21.15.78:0/192723216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563ae7c4790 0x5563ae7c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9e4452740 1 --2- 172.21.15.78:0/192723216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563ae7b8800 0x5563ae7c8bd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9e4452740 1 --2- 172.21.15.78:0/192723216 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5563ae841ee0 0x5563ae844300 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9e4452740 1 --2- 172.21.15.78:0/192723216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563ae7b7ee0 0x5563ae7c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9baffd640 0 client.27236 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:11.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9e4452740 1 -- 172.21.15.78:0/192723216 >> 172.21.15.78:0/192723216 conn(0x5563ae7b5d10 msgr2=0x5563ae7b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:11.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9e4452740 1 -- 172.21.15.78:0/192723216 shutdown_connections 2024-09-11T01:12:11.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9e4452740 1 -- 172.21.15.78:0/192723216 wait complete. 2024-09-11T01:12:11.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.639+0000 7ff9e4452740 20 client.27236 trim_cache size 0 max 16384 2024-09-11T01:12:11.645 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:12:11.645 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:12:11.645 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "e5791ad9-cf54-4303-938d-6fc349b35341", 2024-09-11T01:12:11.645 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:12:11.646 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:12:11.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.640+0000 7ff9e207d640 1 -- 172.21.15.78:0/3323658160 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff9c40b5450 msgr2=0x7ff9c40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:11.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.640+0000 7ff9e207d640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff9c40b5450 0x7ff9c40b7910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0031440 tx=0x7ff9d0041040 comp rx=0 tx=0).stop 2024-09-11T01:12:11.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.640+0000 7ff9e207d640 1 -- 172.21.15.78:0/3323658160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9dc10d100 msgr2=0x7ff9dc19ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:11.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.640+0000 7ff9e207d640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9dc10d100 0x7ff9dc19ebd0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400eaa0 tx=0x7ff9d400ef60 comp rx=0 tx=0).stop 2024-09-11T01:12:11.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.640+0000 7ff9e207d640 1 -- 172.21.15.78:0/3323658160 shutdown_connections 2024-09-11T01:12:11.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.640+0000 7ff9e207d640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff9c40b5450 0x7ff9c40b7910 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.640+0000 7ff9e207d640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9dc10d100 0x7ff9dc19ebd0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.640+0000 7ff9e207d640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9dc1045c0 0x7ff9dc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.640+0000 7ff9e207d640 1 --2- 172.21.15.78:0/3323658160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9dc103ca0 0x7ff9dc1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:11.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.640+0000 7ff9e207d640 1 -- 172.21.15.78:0/3323658160 >> 172.21.15.78:0/3323658160 conn(0x7ff9dc0fd700 msgr2=0x7ff9dc0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:11.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.640+0000 7ff9e207d640 1 -- 172.21.15.78:0/3323658160 shutdown_connections 2024-09-11T01:12:11.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:11.640+0000 7ff9e207d640 1 -- 172.21.15.78:0/3323658160 wait complete. 2024-09-11T01:12:11.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:11 smithi110 ceph-mon[34712]: pgmap v736: 97 pgs: 97 active+clean; 819 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.3 MiB/s wr, 553 op/s 2024-09-11T01:12:11.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:11 smithi112 ceph-mon[35384]: pgmap v736: 97 pgs: 97 active+clean; 819 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.3 MiB/s wr, 553 op/s 2024-09-11T01:12:11.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:11 smithi078 ceph-mon[31315]: pgmap v736: 97 pgs: 97 active+clean; 819 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.3 MiB/s wr, 553 op/s 2024-09-11T01:12:11.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 7295 33.99 MB/sec warmup 17 sec latency 109.812 ms 2024-09-11T01:12:12.505 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:12:12.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:12 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:12:12.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:12 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:12.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:12 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:12.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:12 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:12:12.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:12 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:12.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:12 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:12.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:12 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:12:12.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:12 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:12.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:12 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:12.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 7327 32.10 MB/sec warmup 18 sec latency 645.127 ms 2024-09-11T01:12:12.855 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:12:13.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.377+0000 7f627b857640 1 -- 172.21.15.78:0/2359296448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62741076a0 msgr2=0x7f6274107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:13.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.377+0000 7f627b857640 1 --2- 172.21.15.78:0/2359296448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62741076a0 0x7f6274107ae0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6264009900 tx=0x7f626402a410 comp rx=0 tx=0).stop 2024-09-11T01:12:13.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.377+0000 7f627b857640 1 -- 172.21.15.78:0/2359296448 shutdown_connections 2024-09-11T01:12:13.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.377+0000 7f627b857640 1 --2- 172.21.15.78:0/2359296448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6274108020 0x7f627410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.377+0000 7f627b857640 1 --2- 172.21.15.78:0/2359296448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62741076a0 0x7f6274107ae0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.377+0000 7f627b857640 1 --2- 172.21.15.78:0/2359296448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6274101c90 0x7f6274102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.377+0000 7f627b857640 1 -- 172.21.15.78:0/2359296448 >> 172.21.15.78:0/2359296448 conn(0x7f62740fd700 msgr2=0x7f62740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:13.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.377+0000 7f627b857640 1 -- 172.21.15.78:0/2359296448 shutdown_connections 2024-09-11T01:12:13.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.377+0000 7f627b857640 1 -- 172.21.15.78:0/2359296448 wait complete. 2024-09-11T01:12:13.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f627b857640 1 Processor -- start 2024-09-11T01:12:13.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f627b857640 1 -- start start 2024-09-11T01:12:13.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f627b857640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6274101c90 0x7f62740fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f627b857640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62741076a0 0x7f62740fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f627b857640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6274108020 0x7f62740fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f627b857640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6274111ca0 con 0x7f6274101c90 2024-09-11T01:12:13.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f627b857640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6274111b20 con 0x7f6274108020 2024-09-11T01:12:13.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f627b857640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6274111e20 con 0x7f62741076a0 2024-09-11T01:12:13.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f62795f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6274101c90 0x7f62740fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f62795f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6274101c90 0x7f62740fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50376/0 (socket says 172.21.15.78:50376) 2024-09-11T01:12:13.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f62795f3640 1 -- 172.21.15.78:0/929815146 learned_addr learned my addr 172.21.15.78:0/929815146 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:13.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f6279df4640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6274108020 0x7f62740fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f6278df2640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62741076a0 0x7f62740fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f62795f3640 1 -- 172.21.15.78:0/929815146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62741076a0 msgr2=0x7f62740fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f62795f3640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62741076a0 0x7f62740fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f62795f3640 1 -- 172.21.15.78:0/929815146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6274108020 msgr2=0x7f62740fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f62795f3640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6274108020 0x7f62740fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.378+0000 7f62795f3640 1 -- 172.21.15.78:0/929815146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62741a76e0 con 0x7f6274101c90 2024-09-11T01:12:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.379+0000 7f62795f3640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6274101c90 0x7f62740fdc00 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f627000e9f0 tx=0x7f627000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.379+0000 7f626a7fc640 1 -- 172.21.15.78:0/929815146 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627000cd80 con 0x7f6274101c90 2024-09-11T01:12:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.379+0000 7f627b857640 1 -- 172.21.15.78:0/929815146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62741a79d0 con 0x7f6274101c90 2024-09-11T01:12:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.379+0000 7f627b857640 1 -- 172.21.15.78:0/929815146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62741a7f30 con 0x7f6274101c90 2024-09-11T01:12:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.379+0000 7f626a7fc640 1 -- 172.21.15.78:0/929815146 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62700108e0 con 0x7f6274101c90 2024-09-11T01:12:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.379+0000 7f626a7fc640 1 -- 172.21.15.78:0/929815146 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700188e0 con 0x7f6274101c90 2024-09-11T01:12:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.381+0000 7f626a7fc640 1 -- 172.21.15.78:0/929815146 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6270010430 con 0x7f6274101c90 2024-09-11T01:12:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f627dc2c740 1 Processor -- start 2024-09-11T01:12:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f627dc2c740 1 -- start start 2024-09-11T01:12:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f627dc2c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4ceee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f627dc2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceeeda0 0x561b4cefa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f627dc2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cefd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f627dc2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561b4ce64e50 con 0x561b4cefad30 2024-09-11T01:12:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f627dc2c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561b4ce54280 con 0x561b4ceee480 2024-09-11T01:12:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f627dc2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561b4ce58090 con 0x561b4ceeeda0 2024-09-11T01:12:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f6279df4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4ceee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f6278df2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cefd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f6279df4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4ceee860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:58266/0 (socket says 172.21.15.78:58266) 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f6279df4640 1 -- 172.21.15.78:0/757370428 learned_addr learned my addr 172.21.15.78:0/757370428 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f62795f3640 1 --2- 172.21.15.78:0/757370428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceeeda0 0x561b4cefa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f626a7fc640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62600b5450 0x7f62600b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f6279df4640 1 -- 172.21.15.78:0/757370428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceeeda0 msgr2=0x561b4cefa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f6279df4640 1 --2- 172.21.15.78:0/757370428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceeeda0 0x561b4cefa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f6279df4640 1 -- 172.21.15.78:0/757370428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 msgr2=0x561b4cefd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f6279df4640 1 --2- 172.21.15.78:0/757370428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cefd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f6279df4640 1 -- 172.21.15.78:0/757370428 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b4cbb2650 con 0x561b4ceee480 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f6279df4640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62600b5450 0x7f62600b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.382+0000 7f626a7fc640 1 -- 172.21.15.78:0/929815146 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f62700e62f0 con 0x7f6274101c90 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.383+0000 7f6279df4640 1 --2- 172.21.15.78:0/757370428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4ceee860 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f626c011590 tx=0x7f626c011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70dd654b6ad6a657 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.383+0000 7f625f7fe640 1 -- 172.21.15.78:0/757370428 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c017050 con 0x561b4ceee480 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.383+0000 7f625f7fe640 1 -- 172.21.15.78:0/757370428 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f626c01bc50 con 0x561b4ceee480 2024-09-11T01:12:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.383+0000 7f625f7fe640 1 -- 172.21.15.78:0/757370428 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c00ed80 con 0x561b4ceee480 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.383+0000 7f6279df4640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62600b5450 0x7f62600b7910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f626c022000 tx=0x7f626c023000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.383+0000 7f627dc2c740 1 -- 172.21.15.78:0/757370428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 msgr2=0x561b4ceee860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.383+0000 7f627dc2c740 1 --2- 172.21.15.78:0/757370428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4ceee860 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f626c011590 tx=0x7f626c011a50 comp rx=0 tx=0).stop 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 -- 172.21.15.78:0/757370428 shutdown_connections 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 --2- 172.21.15.78:0/757370428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cefd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 --2- 172.21.15.78:0/757370428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceeeda0 0x561b4cefa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 --2- 172.21.15.78:0/757370428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4ceee860 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 -- 172.21.15.78:0/757370428 >> 172.21.15.78:0/757370428 conn(0x561b4ceec2b0 msgr2=0x561b4ceed450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 -- 172.21.15.78:0/757370428 shutdown_connections 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 -- 172.21.15.78:0/757370428 wait complete. 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 -- start start 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 Processor -- start 2024-09-11T01:12:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4cefa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceeeda0 0x561b4cef7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cef80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561b4ccb5700 con 0x561b4cefad30 2024-09-11T01:12:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561b4cdef110 con 0x561b4ceee480 2024-09-11T01:12:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561b4cefd9b0 con 0x561b4ceeeda0 2024-09-11T01:12:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.384+0000 7f627dc2c740 10 client.0 _ll_register_callbacks cb 0x561b4ce08990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f6279df4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cef80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f62795f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4cefa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f6279df4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cef80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50394/0 (socket says 172.21.15.78:50394) 2024-09-11T01:12:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f6279df4640 1 -- 172.21.15.78:0/3776821521 learned_addr learned my addr 172.21.15.78:0/3776821521 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f62795f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4cefa620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:58278/0 (socket says 172.21.15.78:58278) 2024-09-11T01:12:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f6278df2640 1 --2- 172.21.15.78:0/3776821521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceeeda0 0x561b4cef7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f6279df4640 1 -- 172.21.15.78:0/3776821521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceeeda0 msgr2=0x561b4cef7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f6279df4640 1 --2- 172.21.15.78:0/3776821521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceeeda0 0x561b4cef7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f6279df4640 1 -- 172.21.15.78:0/3776821521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 msgr2=0x561b4cefa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f6279df4640 1 --2- 172.21.15.78:0/3776821521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4cefa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f6279df4640 1 -- 172.21.15.78:0/3776821521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b4ce1cc40 con 0x561b4cefad30 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f6279df4640 1 --2- 172.21.15.78:0/3776821521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cef80a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f626c0099c0 tx=0x7f626c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c60a5be28fe0d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f625cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f625cff9640 1 -- 172.21.15.78:0/3776821521 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c01ba50 con 0x561b4cefad30 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f625cff9640 1 -- 172.21.15.78:0/3776821521 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f626c004c10 con 0x561b4cefad30 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f627dc2c740 1 -- 172.21.15.78:0/3776821521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f626c01bc50 con 0x561b4cefad30 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f625cff9640 1 -- 172.21.15.78:0/3776821521 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f626c004e10 con 0x561b4cefad30 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f627dc2c740 10 client.18057 fetch_fsmap learned FSMap version 58 2024-09-11T01:12:13.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.385+0000 7f627dc2c740 1 -- 172.21.15.78:0/3776821521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561b4cf09330 con 0x561b4cefad30 2024-09-11T01:12:13.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.386+0000 7f625cff9640 1 -- 172.21.15.78:0/3776821521 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f626c01bc50 con 0x561b4cefad30 2024-09-11T01:12:13.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.386+0000 7f625cff9640 1 -- 172.21.15.78:0/3776821521 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 58) ==== 8430+0+0 (secure 0 0 0) 0x7f626c005320 con 0x561b4cefad30 2024-09-11T01:12:13.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.386+0000 7f627dc2c740 10 client.18057 fetch_fsmap finished waiting for FSMap version 58 2024-09-11T01:12:13.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.386+0000 7f627dc2c740 10 client.18057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:12:13.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.386+0000 7f627dc2c740 20 client.18057 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:13.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.386+0000 7f627dc2c740 1 --2- 172.21.15.78:0/3776821521 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b4cf0a6e0 0x561b4cf0cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.386+0000 7f627dc2c740 4 client.18057 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:12:13.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.386+0000 7f627dc2c740 1 -- 172.21.15.78:0/3776821521 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b4ce66da0 con 0x561b4cf0a6e0 2024-09-11T01:12:13.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.386+0000 7f62795f3640 1 --2- 172.21.15.78:0/3776821521 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b4cf0a6e0 0x561b4cf0cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.386+0000 7f625cff9640 10 client.18057 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:13.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.386+0000 7f62795f3640 1 --2- 172.21.15.78:0/3776821521 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b4cf0a6e0 0x561b4cf0cad0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=136c48590652ca4d server_cookie=ff2e704406e35931 in_seq=0 out_seq=0 2024-09-11T01:12:13.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.389+0000 7f625cff9640 1 -- 172.21.15.78:0/3776821521 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561b4ce66da0 con 0x561b4cf0a6e0 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.389+0000 7f625cff9640 10 client.18057 handle_command_reply: tid=0 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.389+0000 7f627dc2c740 1 client.18057 shutdown 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.389+0000 7f627dc2c740 1 -- 172.21.15.78:0/3776821521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 msgr2=0x561b4cef80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.389+0000 7f627dc2c740 1 --2- 172.21.15.78:0/3776821521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cef80a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f626c0099c0 tx=0x7f626c00ef40 comp rx=0 tx=0).stop 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.389+0000 7f627dc2c740 1 -- 172.21.15.78:0/3776821521 shutdown_connections 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.389+0000 7f627dc2c740 1 --2- 172.21.15.78:0/3776821521 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b4cf0a6e0 0x561b4cf0cad0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.389+0000 7f627dc2c740 1 --2- 172.21.15.78:0/3776821521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cef80a0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.389+0000 7f627dc2c740 1 --2- 172.21.15.78:0/3776821521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceeeda0 0x561b4cef7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.390+0000 7f627dc2c740 1 --2- 172.21.15.78:0/3776821521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4cefa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.390+0000 7f627dc2c740 1 -- 172.21.15.78:0/3776821521 >> 172.21.15.78:0/3776821521 conn(0x561b4ceec2b0 msgr2=0x561b4cef1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.390+0000 7f625cff9640 0 client.18057 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.390+0000 7f627dc2c740 1 -- 172.21.15.78:0/3776821521 shutdown_connections 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.390+0000 7f627dc2c740 1 -- 172.21.15.78:0/3776821521 wait complete. 2024-09-11T01:12:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.390+0000 7f627dc2c740 20 client.18057 trim_cache size 0 max 16384 2024-09-11T01:12:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.407+0000 7f627dc2c740 1 Processor -- start 2024-09-11T01:12:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.407+0000 7f627dc2c740 1 -- start start 2024-09-11T01:12:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f627dc2c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4cf06be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f627dc2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4ceeeda0 0x561b4ceeb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f627dc2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4cefad30 0x561b4cefc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f627dc2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62741b0860 con 0x561b4ceeeda0 2024-09-11T01:12:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f627dc2c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f62741b2550 con 0x561b4ceee480 2024-09-11T01:12:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f627dc2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62741b0f30 con 0x561b4cefad30 2024-09-11T01:12:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f6278df2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4ceeeda0 0x561b4ceeb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f6278df2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4ceeeda0 0x561b4ceeb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50410/0 (socket says 172.21.15.78:50410) 2024-09-11T01:12:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f6278df2640 1 -- 172.21.15.78:0/485157711 learned_addr learned my addr 172.21.15.78:0/485157711 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f62795f3640 1 --2- 172.21.15.78:0/485157711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4cf06be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f6279df4640 1 --2- 172.21.15.78:0/485157711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4cefad30 0x561b4cefc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f6278df2640 1 -- 172.21.15.78:0/485157711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4cefad30 msgr2=0x561b4cefc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f6278df2640 1 --2- 172.21.15.78:0/485157711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4cefad30 0x561b4cefc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f6278df2640 1 -- 172.21.15.78:0/485157711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 msgr2=0x561b4cf06be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f6278df2640 1 --2- 172.21.15.78:0/485157711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4cf06be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f6278df2640 1 -- 172.21.15.78:0/485157711 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b4cf0c6f0 con 0x561b4ceeeda0 2024-09-11T01:12:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f6279df4640 1 --2- 172.21.15.78:0/485157711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4cefad30 0x561b4cefc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:12:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f6278df2640 1 --2- 172.21.15.78:0/485157711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4ceeeda0 0x561b4ceeb2c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6264009900 tx=0x7f6264032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1185a9b7767c492b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.408+0000 7f625effd640 1 -- 172.21.15.78:0/485157711 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264035040 con 0x561b4ceeeda0 2024-09-11T01:12:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.409+0000 7f625effd640 1 -- 172.21.15.78:0/485157711 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62640316a0 con 0x561b4ceeeda0 2024-09-11T01:12:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.409+0000 7f625effd640 1 -- 172.21.15.78:0/485157711 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264004750 con 0x561b4ceeeda0 2024-09-11T01:12:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.409+0000 7f627dc2c740 1 -- 172.21.15.78:0/485157711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4ceeeda0 msgr2=0x561b4ceeb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.409+0000 7f627dc2c740 1 --2- 172.21.15.78:0/485157711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4ceeeda0 0x561b4ceeb2c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6264009900 tx=0x7f6264032df0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.409+0000 7f627dc2c740 1 -- 172.21.15.78:0/485157711 shutdown_connections 2024-09-11T01:12:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.409+0000 7f627dc2c740 1 --2- 172.21.15.78:0/485157711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4cefad30 0x561b4cefc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.409+0000 7f627dc2c740 1 --2- 172.21.15.78:0/485157711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4ceeeda0 0x561b4ceeb2c0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.409+0000 7f627dc2c740 1 --2- 172.21.15.78:0/485157711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceee480 0x561b4cf06be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.409+0000 7f627dc2c740 1 -- 172.21.15.78:0/485157711 >> 172.21.15.78:0/485157711 conn(0x561b4ceec2b0 msgr2=0x561b4ceeabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.409+0000 7f627dc2c740 1 -- 172.21.15.78:0/485157711 shutdown_connections 2024-09-11T01:12:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.409+0000 7f627dc2c740 1 -- 172.21.15.78:0/485157711 wait complete. 2024-09-11T01:12:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f627dc2c740 1 -- start start 2024-09-11T01:12:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f627dc2c740 1 Processor -- start 2024-09-11T01:12:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f627dc2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceee480 0x561b4cf02c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f627dc2c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceeeda0 0x561b4cf03190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f627dc2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cefdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f627dc2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62741b1e50 con 0x561b4cefad30 2024-09-11T01:12:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f627dc2c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f62741b16c0 con 0x561b4ceeeda0 2024-09-11T01:12:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f627dc2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62741ae860 con 0x561b4ceee480 2024-09-11T01:12:13.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f627dc2c740 10 client.0 _ll_register_callbacks cb 0x7f626c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:13.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f6279df4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cefdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f6279df4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cefdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50422/0 (socket says 172.21.15.78:50422) 2024-09-11T01:12:13.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f6279df4640 1 -- 172.21.15.78:0/2724606675 learned_addr learned my addr 172.21.15.78:0/2724606675 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:13.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.410+0000 7f6278df2640 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceeeda0 0x561b4cf03190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f62795f3640 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceee480 0x561b4cf02c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f6279df4640 1 -- 172.21.15.78:0/2724606675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceee480 msgr2=0x561b4cf02c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:13.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f6279df4640 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceee480 0x561b4cf02c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f6279df4640 1 -- 172.21.15.78:0/2724606675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceeeda0 msgr2=0x561b4cf03190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:13.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f6279df4640 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceeeda0 0x561b4cf03190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f6279df4640 1 -- 172.21.15.78:0/2724606675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b4cf039c0 con 0x561b4cefad30 2024-09-11T01:12:13.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f62795f3640 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceee480 0x561b4cf02c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f6279df4640 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cefdc00 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f626c00ead0 tx=0x7f626c01a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a89eceec021c4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f625cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f625cff9640 1 -- 172.21.15.78:0/2724606675 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c017070 con 0x561b4cefad30 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f625cff9640 1 -- 172.21.15.78:0/2724606675 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f626c01ba50 con 0x561b4cefad30 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f627dc2c740 1 -- 172.21.15.78:0/2724606675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62640316a0 con 0x561b4cefad30 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f625cff9640 1 -- 172.21.15.78:0/2724606675 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f626c01bc50 con 0x561b4cefad30 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f627dc2c740 10 client.18063 fetch_fsmap learned FSMap version 58 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f627dc2c740 1 -- 172.21.15.78:0/2724606675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561b4cf77990 con 0x561b4cefad30 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f625cff9640 1 -- 172.21.15.78:0/2724606675 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62640316a0 con 0x561b4cefad30 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.411+0000 7f625cff9640 1 -- 172.21.15.78:0/2724606675 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 58) ==== 8430+0+0 (secure 0 0 0) 0x7f626c004fb0 con 0x561b4cefad30 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.412+0000 7f627dc2c740 10 client.18063 fetch_fsmap finished waiting for FSMap version 58 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.412+0000 7f627dc2c740 10 client.18063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:12:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.412+0000 7f627dc2c740 20 client.18063 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:13.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.412+0000 7f627dc2c740 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b4cf78050 0x561b4cf7a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:13.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.412+0000 7f627dc2c740 4 client.18063 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:12:13.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.412+0000 7f627dc2c740 1 -- 172.21.15.78:0/2724606675 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b4ce62850 con 0x561b4cf78050 2024-09-11T01:12:13.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.412+0000 7f62795f3640 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b4cf78050 0x561b4cf7a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:13.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.412+0000 7f625cff9640 10 client.18063 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:13.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.412+0000 7f62795f3640 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b4cf78050 0x561b4cf7a470 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28e76fee4165b97 server_cookie=8785e7ea32641e32 in_seq=0 out_seq=0 2024-09-11T01:12:13.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f625cff9640 1 -- 172.21.15.78:0/2724606675 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5406 (crc 0 0 0) 0x561b4ce62850 con 0x561b4cf78050 2024-09-11T01:12:13.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f625cff9640 10 client.18063 handle_command_reply: tid=0 2024-09-11T01:12:13.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f627dc2c740 1 client.18063 shutdown 2024-09-11T01:12:13.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f627dc2c740 1 -- 172.21.15.78:0/2724606675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 msgr2=0x561b4cefdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:13.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f627dc2c740 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cefdc00 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f626c00ead0 tx=0x7f626c01a4e0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f627dc2c740 1 -- 172.21.15.78:0/2724606675 shutdown_connections 2024-09-11T01:12:13.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f627dc2c740 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x561b4cf78050 0x561b4cf7a470 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f627dc2c740 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561b4cefad30 0x561b4cefdc00 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f627dc2c740 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561b4ceeeda0 0x561b4cf03190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f627dc2c740 1 --2- 172.21.15.78:0/2724606675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b4ceee480 0x561b4cf02c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f625cff9640 0 client.18063 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:13.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f627dc2c740 1 -- 172.21.15.78:0/2724606675 >> 172.21.15.78:0/2724606675 conn(0x561b4ceec2b0 msgr2=0x561b4cef98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:13.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f627dc2c740 1 -- 172.21.15.78:0/2724606675 shutdown_connections 2024-09-11T01:12:13.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f627dc2c740 1 -- 172.21.15.78:0/2724606675 wait complete. 2024-09-11T01:12:13.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.417+0000 7f627dc2c740 20 client.18063 trim_cache size 0 max 16384 2024-09-11T01:12:13.421 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (527 inodes in the stack)", 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "e5791ad9-cf54-4303-938d-6fc349b35341", 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:12:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:12:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:12:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:12:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:12:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:12:13.433 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:12:13.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.418+0000 7f627b857640 1 -- 172.21.15.78:0/929815146 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62600b5450 msgr2=0x7f62600b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.418+0000 7f627b857640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62600b5450 0x7f62600b7910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f626c022000 tx=0x7f626c023000 comp rx=0 tx=0).stop 2024-09-11T01:12:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.418+0000 7f627b857640 1 -- 172.21.15.78:0/929815146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6274101c90 msgr2=0x7f62740fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.418+0000 7f627b857640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6274101c90 0x7f62740fdc00 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f627000e9f0 tx=0x7f627000eeb0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.418+0000 7f627b857640 1 -- 172.21.15.78:0/929815146 shutdown_connections 2024-09-11T01:12:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.418+0000 7f627b857640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f62600b5450 0x7f62600b7910 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.418+0000 7f627b857640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6274108020 0x7f62740fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.418+0000 7f627b857640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62741076a0 0x7f62740fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.418+0000 7f627b857640 1 --2- 172.21.15.78:0/929815146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6274101c90 0x7f62740fdc00 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.418+0000 7f627b857640 1 -- 172.21.15.78:0/929815146 >> 172.21.15.78:0/929815146 conn(0x7f62740fd700 msgr2=0x7f62741047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.418+0000 7f627b857640 1 -- 172.21.15.78:0/929815146 shutdown_connections 2024-09-11T01:12:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:13.418+0000 7f627b857640 1 -- 172.21.15.78:0/929815146 wait complete. 2024-09-11T01:12:13.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:13 smithi110 ceph-mon[34712]: pgmap v737: 97 pgs: 97 active+clean; 817 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 493 op/s 2024-09-11T01:12:13.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:13 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:13 smithi112 ceph-mon[35384]: pgmap v737: 97 pgs: 97 active+clean; 817 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 493 op/s 2024-09-11T01:12:13.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:13 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:13.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:13 smithi078 ceph-mon[31315]: pgmap v737: 97 pgs: 97 active+clean; 817 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 493 op/s 2024-09-11T01:12:13.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:13 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:13.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 7529 30.51 MB/sec warmup 19 sec latency 1301.675 ms 2024-09-11T01:12:14.323 INFO:tasks.cephfs.filesystem:scrub status for tag:e5791ad9-cf54-4303-938d-6fc349b35341 - {'path': '/', 'tag': 'e5791ad9-cf54-4303-938d-6fc349b35341', 'options': 'recursive,force'} 2024-09-11T01:12:14.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 7945 29.67 MB/sec warmup 20 sec latency 2301.711 ms 2024-09-11T01:12:15.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:15 smithi110 ceph-mon[34712]: pgmap v738: 97 pgs: 97 active+clean; 854 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.8 MiB/s wr, 362 op/s 2024-09-11T01:12:15.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:12:15.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:12:15.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:12:15.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:12:15.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:15 smithi112 ceph-mon[35384]: pgmap v738: 97 pgs: 97 active+clean; 854 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.8 MiB/s wr, 362 op/s 2024-09-11T01:12:15.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:12:15.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:12:15.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:12:15.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:12:15.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:15 smithi078 ceph-mon[31315]: pgmap v738: 97 pgs: 97 active+clean; 854 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.8 MiB/s wr, 362 op/s 2024-09-11T01:12:15.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:12:15.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:12:15.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:12:15.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:12:15.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 8559 29.53 MB/sec warmup 21 sec latency 3301.741 ms 2024-09-11T01:12:16.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 9199 30.35 MB/sec warmup 22 sec latency 4301.771 ms 2024-09-11T01:12:17.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:17 smithi110 ceph-mon[34712]: pgmap v739: 97 pgs: 97 active+clean; 906 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 292 op/s 2024-09-11T01:12:17.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:17 smithi112 ceph-mon[35384]: pgmap v739: 97 pgs: 97 active+clean; 906 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 292 op/s 2024-09-11T01:12:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:17 smithi078 ceph-mon[31315]: pgmap v739: 97 pgs: 97 active+clean; 906 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 292 op/s 2024-09-11T01:12:17.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 9525 29.94 MB/sec warmup 23 sec latency 5297.780 ms 2024-09-11T01:12:18.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.726 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.727 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.728 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.729 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.730 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.730 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.730 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.730 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.730 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.730 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.730 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:12:18.730 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:12:18.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.775 INFO:tasks.workunit.client.0.smithi078.stdout: 10 9654 29.39 MB/sec warmup 24 sec latency 6297.808 ms 2024-09-11T01:12:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.943 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.944 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.944 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.944 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.944 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.945 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.945 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.945 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.945 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.945 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.945 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.945 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.945 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.945 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.945 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.945 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.945 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.946 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.946 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.946 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.946 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.946 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.946 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.946 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.946 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.946 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.947 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:18.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:12:18.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:19.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:19 smithi112 ceph-mon[35384]: pgmap v740: 97 pgs: 97 active+clean; 916 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 14 MiB/s wr, 293 op/s 2024-09-11T01:12:19.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:19 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:19.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:19 smithi078 ceph-mon[31315]: pgmap v740: 97 pgs: 97 active+clean; 916 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 14 MiB/s wr, 293 op/s 2024-09-11T01:12:19.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:19 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:19.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 10026 28.96 MB/sec warmup 25 sec latency 7297.851 ms 2024-09-11T01:12:19.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:19 smithi110 ceph-mon[34712]: pgmap v740: 97 pgs: 97 active+clean; 916 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 14 MiB/s wr, 293 op/s 2024-09-11T01:12:19.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:19 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:12:20.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 10285 28.17 MB/sec warmup 26 sec latency 8297.902 ms 2024-09-11T01:12:21.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:21 smithi112 ceph-mon[35384]: pgmap v741: 97 pgs: 97 active+clean; 954 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 16 MiB/s wr, 335 op/s 2024-09-11T01:12:21.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:21 smithi078 ceph-mon[31315]: pgmap v741: 97 pgs: 97 active+clean; 954 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 16 MiB/s wr, 335 op/s 2024-09-11T01:12:21.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 10677 27.70 MB/sec warmup 27 sec latency 9297.973 ms 2024-09-11T01:12:21.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:21 smithi110 ceph-mon[34712]: pgmap v741: 97 pgs: 97 active+clean; 954 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 16 MiB/s wr, 335 op/s 2024-09-11T01:12:22.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 11377 27.94 MB/sec warmup 28 sec latency 10298.005 ms 2024-09-11T01:12:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:23 smithi112 ceph-mon[35384]: pgmap v742: 97 pgs: 97 active+clean; 969 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 17 MiB/s wr, 341 op/s 2024-09-11T01:12:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:23 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:23.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:23 smithi078 ceph-mon[31315]: pgmap v742: 97 pgs: 97 active+clean; 969 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 17 MiB/s wr, 341 op/s 2024-09-11T01:12:23.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:23 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:23.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 11992 28.27 MB/sec warmup 29 sec latency 11298.037 ms 2024-09-11T01:12:23.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:23 smithi110 ceph-mon[34712]: pgmap v742: 97 pgs: 97 active+clean; 969 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 17 MiB/s wr, 341 op/s 2024-09-11T01:12:23.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:23 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:12:24.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 12622 28.62 MB/sec warmup 30 sec latency 12298.067 ms 2024-09-11T01:12:25.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:25 smithi112 ceph-mon[35384]: pgmap v743: 97 pgs: 97 active+clean; 980 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 408 op/s 2024-09-11T01:12:25.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:25 smithi078 ceph-mon[31315]: pgmap v743: 97 pgs: 97 active+clean; 980 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 408 op/s 2024-09-11T01:12:25.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 12965 28.49 MB/sec warmup 31 sec latency 13298.097 ms 2024-09-11T01:12:25.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:25 smithi110 ceph-mon[34712]: pgmap v743: 97 pgs: 97 active+clean; 980 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 408 op/s 2024-09-11T01:12:26.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 13247 27.78 MB/sec warmup 32 sec latency 14298.130 ms 2024-09-11T01:12:27.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:27 smithi112 ceph-mon[35384]: pgmap v744: 97 pgs: 97 active+clean; 991 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 438 op/s 2024-09-11T01:12:27.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:27 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:12:27.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:27 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:12:27.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:27 smithi078 ceph-mon[31315]: pgmap v744: 97 pgs: 97 active+clean; 991 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 438 op/s 2024-09-11T01:12:27.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:27 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:12:27.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:27 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:12:27.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 13821 27.81 MB/sec warmup 33 sec latency 14888.542 ms 2024-09-11T01:12:27.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:27 smithi110 ceph-mon[34712]: pgmap v744: 97 pgs: 97 active+clean; 991 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 438 op/s 2024-09-11T01:12:27.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:27 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:12:27.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:27 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:12:28.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 14560 28.15 MB/sec warmup 34 sec latency 1504.381 ms 2024-09-11T01:12:29.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:29 smithi112 ceph-mon[35384]: pgmap v745: 97 pgs: 97 active+clean; 1000 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 415 op/s 2024-09-11T01:12:29.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:29 smithi078 ceph-mon[31315]: pgmap v745: 97 pgs: 97 active+clean; 1000 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 415 op/s 2024-09-11T01:12:29.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 15097 28.14 MB/sec warmup 35 sec latency 2504.415 ms 2024-09-11T01:12:29.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:29 smithi110 ceph-mon[34712]: pgmap v745: 97 pgs: 97 active+clean; 1000 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 415 op/s 2024-09-11T01:12:30.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 15553 27.95 MB/sec warmup 36 sec latency 3504.447 ms 2024-09-11T01:12:31.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:31 smithi112 ceph-mon[35384]: pgmap v746: 97 pgs: 97 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 14 MiB/s wr, 463 op/s 2024-09-11T01:12:31.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:31 smithi078 ceph-mon[31315]: pgmap v746: 97 pgs: 97 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 14 MiB/s wr, 463 op/s 2024-09-11T01:12:31.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 16626 28.58 MB/sec warmup 37 sec latency 3522.898 ms 2024-09-11T01:12:31.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:31 smithi110 ceph-mon[34712]: pgmap v746: 97 pgs: 97 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 14 MiB/s wr, 463 op/s 2024-09-11T01:12:32.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 17645 29.92 MB/sec warmup 38 sec latency 88.488 ms 2024-09-11T01:12:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:33 smithi112 ceph-mon[35384]: pgmap v747: 97 pgs: 97 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 431 op/s 2024-09-11T01:12:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:33 smithi078 ceph-mon[31315]: pgmap v747: 97 pgs: 97 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 431 op/s 2024-09-11T01:12:33.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 18231 29.66 MB/sec warmup 39 sec latency 47.975 ms 2024-09-11T01:12:33.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:33 smithi110 ceph-mon[34712]: pgmap v747: 97 pgs: 97 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 431 op/s 2024-09-11T01:12:34.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 19163 30.02 MB/sec warmup 40 sec latency 97.441 ms 2024-09-11T01:12:35.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:35 smithi112 ceph-mon[35384]: pgmap v748: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 463 op/s 2024-09-11T01:12:35.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:35 smithi078 ceph-mon[31315]: pgmap v748: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 463 op/s 2024-09-11T01:12:35.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 20421 30.91 MB/sec warmup 41 sec latency 69.187 ms 2024-09-11T01:12:35.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:35 smithi110 ceph-mon[34712]: pgmap v748: 97 pgs: 97 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 463 op/s 2024-09-11T01:12:36.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 21314 31.76 MB/sec warmup 42 sec latency 32.443 ms 2024-09-11T01:12:37.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:37 smithi112 ceph-mon[35384]: pgmap v749: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 548 op/s 2024-09-11T01:12:37.742 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:37 smithi110 ceph-mon[34712]: pgmap v749: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 548 op/s 2024-09-11T01:12:37.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:37 smithi078 ceph-mon[31315]: pgmap v749: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 548 op/s 2024-09-11T01:12:37.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 22069 31.69 MB/sec warmup 43 sec latency 55.527 ms 2024-09-11T01:12:38.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 23251 32.22 MB/sec warmup 44 sec latency 104.709 ms 2024-09-11T01:12:39.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:39 smithi078 ceph-mon[31315]: pgmap v750: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 547 op/s 2024-09-11T01:12:39.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 24388 33.19 MB/sec warmup 45 sec latency 67.523 ms 2024-09-11T01:12:39.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:39 smithi110 ceph-mon[34712]: pgmap v750: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 547 op/s 2024-09-11T01:12:39.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:39 smithi112 ceph-mon[35384]: pgmap v750: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 547 op/s 2024-09-11T01:12:40.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 25012 33.10 MB/sec warmup 46 sec latency 287.728 ms 2024-09-11T01:12:41.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:41 smithi078 ceph-mon[31315]: pgmap v751: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 621 op/s 2024-09-11T01:12:41.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 26015 33.39 MB/sec warmup 47 sec latency 303.537 ms 2024-09-11T01:12:41.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:41 smithi110 ceph-mon[34712]: pgmap v751: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 621 op/s 2024-09-11T01:12:41.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:41 smithi112 ceph-mon[35384]: pgmap v751: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 621 op/s 2024-09-11T01:12:42.776 INFO:tasks.workunit.client.0.smithi078.stdout: 10 27037 33.98 MB/sec warmup 48 sec latency 98.484 ms 2024-09-11T01:12:43.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:43 smithi078 ceph-mon[31315]: pgmap v752: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 664 op/s 2024-09-11T01:12:43.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 27897 34.27 MB/sec warmup 49 sec latency 231.031 ms 2024-09-11T01:12:43.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:43 smithi110 ceph-mon[34712]: pgmap v752: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 664 op/s 2024-09-11T01:12:43.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:43 smithi112 ceph-mon[35384]: pgmap v752: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 664 op/s 2024-09-11T01:12:44.324 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:12:44.673 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:12:44.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 28706 34.36 MB/sec warmup 50 sec latency 180.564 ms 2024-09-11T01:12:45.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:45 smithi078 ceph-mon[31315]: pgmap v753: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 737 op/s 2024-09-11T01:12:45.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 29676 34.74 MB/sec warmup 51 sec latency 164.433 ms 2024-09-11T01:12:45.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:45 smithi110 ceph-mon[34712]: pgmap v753: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 737 op/s 2024-09-11T01:12:45.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:45 smithi112 ceph-mon[35384]: pgmap v753: 97 pgs: 97 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 737 op/s 2024-09-11T01:12:46.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 29732 34.12 MB/sec warmup 52 sec latency 1029.657 ms 2024-09-11T01:12:47.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.333+0000 7f615cd1a640 1 -- 172.21.15.78:0/530655986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158115a10 msgr2=0x7f6158117e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:47.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.333+0000 7f615cd1a640 1 --2- 172.21.15.78:0/530655986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158115a10 0x7f6158117e00 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6150009ca0 tx=0x7f615002a8b0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.333+0000 7f615cd1a640 1 -- 172.21.15.78:0/530655986 shutdown_connections 2024-09-11T01:12:47.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.333+0000 7f615cd1a640 1 --2- 172.21.15.78:0/530655986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158115a10 0x7f6158117e00 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.333+0000 7f615cd1a640 1 --2- 172.21.15.78:0/530655986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f615810aa70 0x7f61581154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.333+0000 7f615cd1a640 1 --2- 172.21.15.78:0/530655986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f615810a150 0x7f615810a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.333+0000 7f615cd1a640 1 -- 172.21.15.78:0/530655986 >> 172.21.15.78:0/530655986 conn(0x7f615806e2e0 msgr2=0x7f615806e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:47.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.333+0000 7f615cd1a640 1 -- 172.21.15.78:0/530655986 shutdown_connections 2024-09-11T01:12:47.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.333+0000 7f615cd1a640 1 -- 172.21.15.78:0/530655986 wait complete. 2024-09-11T01:12:47.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.334+0000 7f615cd1a640 1 Processor -- start 2024-09-11T01:12:47.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.336+0000 7f615cd1a640 1 -- start start 2024-09-11T01:12:47.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.336+0000 7f615cd1a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f615810a150 0x7f61581a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.336+0000 7f615cd1a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f615810aa70 0x7f61581a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.336+0000 7f615cd1a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158115a10 0x7f61581a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.336+0000 7f615cd1a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f615811b5e0 con 0x7f6158115a10 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.336+0000 7f615cd1a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f615811b460 con 0x7f615810a150 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.336+0000 7f615cd1a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f615811b760 con 0x7f615810aa70 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.336+0000 7f615659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f615810a150 0x7f61581a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.337+0000 7f615659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f615810a150 0x7f61581a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43268/0 (socket says 172.21.15.78:43268) 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.337+0000 7f615659c640 1 -- 172.21.15.78:0/2119600271 learned_addr learned my addr 172.21.15.78:0/2119600271 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.337+0000 7f614dd9b640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f615810aa70 0x7f61581a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.337+0000 7f6156d9d640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158115a10 0x7f61581a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.337+0000 7f615659c640 1 -- 172.21.15.78:0/2119600271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f615810aa70 msgr2=0x7f61581a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.337+0000 7f615659c640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f615810aa70 0x7f61581a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.337+0000 7f615659c640 1 -- 172.21.15.78:0/2119600271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158115a10 msgr2=0x7f61581a73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:47.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.337+0000 7f615659c640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158115a10 0x7f61581a73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.337+0000 7f615659c640 1 -- 172.21.15.78:0/2119600271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61581a7b80 con 0x7f615810a150 2024-09-11T01:12:47.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.337+0000 7f6156d9d640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158115a10 0x7f61581a73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:12:47.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.341+0000 7f615659c640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f615810a150 0x7f61581a1ad0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f614800b4a0 tx=0x7f614800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:47.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.341+0000 7f614ffff640 1 -- 172.21.15.78:0/2119600271 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148014070 con 0x7f615810a150 2024-09-11T01:12:47.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.342+0000 7f614ffff640 1 -- 172.21.15.78:0/2119600271 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6148010ac0 con 0x7f615810a150 2024-09-11T01:12:47.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.342+0000 7f615cd1a640 1 -- 172.21.15.78:0/2119600271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61581a7e70 con 0x7f615810a150 2024-09-11T01:12:47.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.342+0000 7f615cd1a640 1 -- 172.21.15.78:0/2119600271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61581ae5b0 con 0x7f615810a150 2024-09-11T01:12:47.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.342+0000 7f614ffff640 1 -- 172.21.15.78:0/2119600271 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148017c30 con 0x7f615810a150 2024-09-11T01:12:47.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.346+0000 7f615f0ef740 1 Processor -- start 2024-09-11T01:12:47.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.346+0000 7f615f0ef740 1 -- start start 2024-09-11T01:12:47.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.346+0000 7f615f0ef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e436480 0x55664e436860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.346+0000 7f615f0ef740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e4427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.346+0000 7f615f0ef740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e442d30 0x55664e445120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.346+0000 7f615f0ef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55664e3ace50 con 0x55664e436480 2024-09-11T01:12:47.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.346+0000 7f615f0ef740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55664e39c280 con 0x55664e442d30 2024-09-11T01:12:47.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.346+0000 7f615f0ef740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55664e3a0090 con 0x55664e436da0 2024-09-11T01:12:47.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.346+0000 7f614ffff640 1 -- 172.21.15.78:0/2119600271 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6148010610 con 0x7f615810a150 2024-09-11T01:12:47.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.347+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e436480 0x55664e436860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.347+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e436480 0x55664e436860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58362/0 (socket says 172.21.15.78:58362) 2024-09-11T01:12:47.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.347+0000 7f6156d9d640 1 -- 172.21.15.78:0/642828840 learned_addr learned my addr 172.21.15.78:0/642828840 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:47.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.347+0000 7f615659c640 1 --2- 172.21.15.78:0/642828840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e4427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.348+0000 7f6156d9d640 1 -- 172.21.15.78:0/642828840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 msgr2=0x55664e4427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:47.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.348+0000 7f6156d9d640 1 --2- 172.21.15.78:0/642828840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e4427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.348+0000 7f6156d9d640 1 -- 172.21.15.78:0/642828840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e442d30 msgr2=0x55664e445120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.348+0000 7f6156d9d640 1 --2- 172.21.15.78:0/642828840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e442d30 0x55664e445120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.348+0000 7f6156d9d640 1 -- 172.21.15.78:0/642828840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664e0fa650 con 0x55664e436480 2024-09-11T01:12:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.348+0000 7f614ffff640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f61440b99e0 0x7f61440bbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.348+0000 7f614ffff640 1 -- 172.21.15.78:0/2119600271 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f61480e5440 con 0x7f615810a150 2024-09-11T01:12:47.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.349+0000 7f6156d9d640 1 --2- 172.21.15.78:0/642828840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e436480 0x55664e436860 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f615806c3c0 tx=0x7f6150009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a163cf8fba1320b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:47.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.349+0000 7f615659c640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f61440b99e0 0x7f61440bbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.350+0000 7f615659c640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f61440b99e0 0x7f61440bbea0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f61480e2c80 tx=0x7f6148010f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:47.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.350+0000 7f614cd99640 1 -- 172.21.15.78:0/642828840 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150030800 con 0x55664e436480 2024-09-11T01:12:47.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.350+0000 7f614cd99640 1 -- 172.21.15.78:0/642828840 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6150036040 con 0x55664e436480 2024-09-11T01:12:47.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.350+0000 7f614cd99640 1 -- 172.21.15.78:0/642828840 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150004440 con 0x55664e436480 2024-09-11T01:12:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.352+0000 7f615f0ef740 1 -- 172.21.15.78:0/642828840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e436480 msgr2=0x55664e436860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:47.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.352+0000 7f615f0ef740 1 --2- 172.21.15.78:0/642828840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e436480 0x55664e436860 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f615806c3c0 tx=0x7f6150009a00 comp rx=0 tx=0).stop 2024-09-11T01:12:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.352+0000 7f615f0ef740 1 -- 172.21.15.78:0/642828840 shutdown_connections 2024-09-11T01:12:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.352+0000 7f615f0ef740 1 --2- 172.21.15.78:0/642828840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e442d30 0x55664e445120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.352+0000 7f615f0ef740 1 --2- 172.21.15.78:0/642828840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e4427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.352+0000 7f615f0ef740 1 --2- 172.21.15.78:0/642828840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e436480 0x55664e436860 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.352+0000 7f615f0ef740 1 -- 172.21.15.78:0/642828840 >> 172.21.15.78:0/642828840 conn(0x55664e4342b0 msgr2=0x55664e435450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:47.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.352+0000 7f615f0ef740 1 -- 172.21.15.78:0/642828840 shutdown_connections 2024-09-11T01:12:47.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.353+0000 7f615f0ef740 1 -- 172.21.15.78:0/642828840 wait complete. 2024-09-11T01:12:47.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.353+0000 7f615f0ef740 1 -- start start 2024-09-11T01:12:47.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.354+0000 7f615f0ef740 1 Processor -- start 2024-09-11T01:12:47.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.354+0000 7f615f0ef740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e442580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.354+0000 7f615f0ef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e436da0 0x55664e43f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.354+0000 7f615f0ef740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e442d30 0x55664e43ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.354+0000 7f615f0ef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55664e1fd700 con 0x55664e436da0 2024-09-11T01:12:47.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.354+0000 7f615f0ef740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55664e337110 con 0x55664e436480 2024-09-11T01:12:47.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.354+0000 7f615f0ef740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55664e4459a0 con 0x55664e442d30 2024-09-11T01:12:47.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.354+0000 7f615f0ef740 10 client.0 _ll_register_callbacks cb 0x55664e350990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:47.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.355+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e442580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.355+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e442d30 0x55664e43ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.355+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e442580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43288/0 (socket says 172.21.15.78:43288) 2024-09-11T01:12:47.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.355+0000 7f6156d9d640 1 -- 172.21.15.78:0/2546883306 learned_addr learned my addr 172.21.15.78:0/2546883306 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:47.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.355+0000 7f6156d9d640 1 -- 172.21.15.78:0/2546883306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e442d30 msgr2=0x55664e43ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:47.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.355+0000 7f6156d9d640 1 --2- 172.21.15.78:0/2546883306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e442d30 0x55664e43ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.355+0000 7f6156d9d640 1 -- 172.21.15.78:0/2546883306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e436da0 msgr2=0x55664e43f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:47.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.355+0000 7f6156d9d640 1 --2- 172.21.15.78:0/2546883306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e436da0 0x55664e43f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.355+0000 7f6156d9d640 1 -- 172.21.15.78:0/2546883306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664e364c40 con 0x55664e436480 2024-09-11T01:12:47.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.356+0000 7f6156d9d640 1 --2- 172.21.15.78:0/2546883306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e442580 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6150009ca0 tx=0x7f6150003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53df96a328b850bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:47.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.356+0000 7f613dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:12:47.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.358+0000 7f613dffb640 1 -- 172.21.15.78:0/2546883306 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615004a040 con 0x55664e436480 2024-09-11T01:12:47.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.358+0000 7f613dffb640 1 -- 172.21.15.78:0/2546883306 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f61500042e0 con 0x55664e436480 2024-09-11T01:12:47.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.361+0000 7f615f0ef740 1 -- 172.21.15.78:0/2546883306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6150036040 con 0x55664e436480 2024-09-11T01:12:47.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.364+0000 7f613dffb640 1 -- 172.21.15.78:0/2546883306 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f6150003b80 con 0x55664e436480 2024-09-11T01:12:47.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.364+0000 7f615f0ef740 10 client.27260 fetch_fsmap learned FSMap version 59 2024-09-11T01:12:47.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.364+0000 7f615f0ef740 1 -- 172.21.15.78:0/2546883306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55664e451280 con 0x55664e436480 2024-09-11T01:12:47.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.364+0000 7f613dffb640 1 -- 172.21.15.78:0/2546883306 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6150036040 con 0x55664e436480 2024-09-11T01:12:47.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.365+0000 7f613dffb640 1 -- 172.21.15.78:0/2546883306 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f6150005020 con 0x55664e436480 2024-09-11T01:12:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.365+0000 7f615f0ef740 10 client.27260 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:12:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.365+0000 7f615f0ef740 10 client.27260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:12:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.365+0000 7f615f0ef740 20 client.27260 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.365+0000 7f615f0ef740 1 --2- 172.21.15.78:0/2546883306 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55664e452840 0x55664e454ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.365+0000 7f615f0ef740 4 client.27260 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:12:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.365+0000 7f615f0ef740 1 -- 172.21.15.78:0/2546883306 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55664e3aeda0 con 0x55664e452840 2024-09-11T01:12:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.365+0000 7f614dd9b640 1 --2- 172.21.15.78:0/2546883306 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55664e452840 0x55664e454ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.366+0000 7f614dd9b640 1 --2- 172.21.15.78:0/2546883306 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55664e452840 0x55664e454ca0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dad13b8f51f916ec server_cookie=d73d3df28e6aae9 in_seq=0 out_seq=0 2024-09-11T01:12:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.366+0000 7f613dffb640 10 client.27260 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.368+0000 7f613dffb640 1 -- 172.21.15.78:0/2546883306 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55664e3aeda0 con 0x55664e452840 2024-09-11T01:12:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.368+0000 7f613dffb640 10 client.27260 handle_command_reply: tid=0 2024-09-11T01:12:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.368+0000 7f615f0ef740 1 client.27260 shutdown 2024-09-11T01:12:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.368+0000 7f615f0ef740 1 -- 172.21.15.78:0/2546883306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 msgr2=0x55664e442580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.368+0000 7f615f0ef740 1 --2- 172.21.15.78:0/2546883306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e442580 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6150009ca0 tx=0x7f6150003f10 comp rx=0 tx=0).stop 2024-09-11T01:12:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.368+0000 7f615f0ef740 1 -- 172.21.15.78:0/2546883306 shutdown_connections 2024-09-11T01:12:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.368+0000 7f615f0ef740 1 --2- 172.21.15.78:0/2546883306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e442d30 0x55664e43ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.368+0000 7f615f0ef740 1 --2- 172.21.15.78:0/2546883306 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55664e452840 0x55664e454ca0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.368+0000 7f615f0ef740 1 --2- 172.21.15.78:0/2546883306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e436da0 0x55664e43f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.368+0000 7f615f0ef740 1 --2- 172.21.15.78:0/2546883306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e442580 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.368+0000 7f615f0ef740 1 -- 172.21.15.78:0/2546883306 >> 172.21.15.78:0/2546883306 conn(0x55664e4342b0 msgr2=0x55664e439e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.369+0000 7f613dffb640 0 client.27260 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.369+0000 7f615f0ef740 1 -- 172.21.15.78:0/2546883306 shutdown_connections 2024-09-11T01:12:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.370+0000 7f615f0ef740 1 -- 172.21.15.78:0/2546883306 wait complete. 2024-09-11T01:12:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.370+0000 7f615f0ef740 20 client.27260 trim_cache size 0 max 16384 2024-09-11T01:12:47.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.404+0000 7f615f0ef740 1 Processor -- start 2024-09-11T01:12:47.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.405+0000 7f615f0ef740 1 -- start start 2024-09-11T01:12:47.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.405+0000 7f615f0ef740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e433bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.405+0000 7f615f0ef740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e43eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.405+0000 7f615f0ef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e442d30 0x55664e454850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.405+0000 7f615f0ef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61581b4d10 con 0x55664e442d30 2024-09-11T01:12:47.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.405+0000 7f615f0ef740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61581b6a90 con 0x55664e436480 2024-09-11T01:12:47.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.405+0000 7f615f0ef740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61581b53e0 con 0x55664e436da0 2024-09-11T01:12:47.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.405+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e442d30 0x55664e454850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.405+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e433bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.405+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e433bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43304/0 (socket says 172.21.15.78:43304) 2024-09-11T01:12:47.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.405+0000 7f6156d9d640 1 -- 172.21.15.78:0/4226639492 learned_addr learned my addr 172.21.15.78:0/4226639492 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:47.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.406+0000 7f614dd9b640 1 --2- 172.21.15.78:0/4226639492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e43eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.406+0000 7f614dd9b640 1 -- 172.21.15.78:0/4226639492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 msgr2=0x55664e433bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:47.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.406+0000 7f614dd9b640 1 --2- 172.21.15.78:0/4226639492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e433bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.406+0000 7f614dd9b640 1 -- 172.21.15.78:0/4226639492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e442d30 msgr2=0x55664e454850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:47.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.406+0000 7f614dd9b640 1 --2- 172.21.15.78:0/4226639492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e442d30 0x55664e454850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.406+0000 7f614dd9b640 1 -- 172.21.15.78:0/4226639492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664e434d30 con 0x55664e436da0 2024-09-11T01:12:47.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.407+0000 7f614dd9b640 1 --2- 172.21.15.78:0/4226639492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e43eec0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6140004700 tx=0x7f6140009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba13f54400497db7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:47.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.409+0000 7f613ffff640 1 -- 172.21.15.78:0/4226639492 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140012e70 con 0x55664e436da0 2024-09-11T01:12:47.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.409+0000 7f613ffff640 1 -- 172.21.15.78:0/4226639492 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61400054d0 con 0x55664e436da0 2024-09-11T01:12:47.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.409+0000 7f613ffff640 1 -- 172.21.15.78:0/4226639492 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140004c70 con 0x55664e436da0 2024-09-11T01:12:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.410+0000 7f615f0ef740 1 -- 172.21.15.78:0/4226639492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 msgr2=0x55664e43eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.410+0000 7f615f0ef740 1 --2- 172.21.15.78:0/4226639492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e43eec0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6140004700 tx=0x7f6140009820 comp rx=0 tx=0).stop 2024-09-11T01:12:47.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.413+0000 7f615f0ef740 1 -- 172.21.15.78:0/4226639492 shutdown_connections 2024-09-11T01:12:47.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.413+0000 7f615f0ef740 1 --2- 172.21.15.78:0/4226639492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e442d30 0x55664e454850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.413+0000 7f615f0ef740 1 --2- 172.21.15.78:0/4226639492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e43eec0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.413+0000 7f615f0ef740 1 --2- 172.21.15.78:0/4226639492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e433bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.413+0000 7f615f0ef740 1 -- 172.21.15.78:0/4226639492 >> 172.21.15.78:0/4226639492 conn(0x55664e4342b0 msgr2=0x55664e43f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:47.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.413+0000 7f615f0ef740 1 -- 172.21.15.78:0/4226639492 shutdown_connections 2024-09-11T01:12:47.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.413+0000 7f615f0ef740 1 -- 172.21.15.78:0/4226639492 wait complete. 2024-09-11T01:12:47.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.414+0000 7f615f0ef740 1 -- start start 2024-09-11T01:12:47.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.414+0000 7f615f0ef740 1 Processor -- start 2024-09-11T01:12:47.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.414+0000 7f615f0ef740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e4464f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.414+0000 7f615f0ef740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e446a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.414+0000 7f615f0ef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e442d30 0x55664e4bff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.414+0000 7f615f0ef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61581b6300 con 0x55664e442d30 2024-09-11T01:12:47.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.414+0000 7f615f0ef740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61581b5b70 con 0x55664e436480 2024-09-11T01:12:47.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.414+0000 7f615f0ef740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61581b2da0 con 0x55664e436da0 2024-09-11T01:12:47.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.415+0000 7f615f0ef740 10 client.0 _ll_register_callbacks cb 0x7f6150009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:47.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.415+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e442d30 0x55664e4bff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.415+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e442d30 0x55664e4bff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58410/0 (socket says 172.21.15.78:58410) 2024-09-11T01:12:47.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.415+0000 7f6156d9d640 1 -- 172.21.15.78:0/4054654427 learned_addr learned my addr 172.21.15.78:0/4054654427 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:47.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.415+0000 7f614dd9b640 1 --2- 172.21.15.78:0/4054654427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e446a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.415+0000 7f6156d9d640 1 -- 172.21.15.78:0/4054654427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 msgr2=0x55664e446a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:47.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.415+0000 7f6156d9d640 1 --2- 172.21.15.78:0/4054654427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e446a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.415+0000 7f6156d9d640 1 -- 172.21.15.78:0/4054654427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 msgr2=0x55664e4464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:47.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.415+0000 7f6156d9d640 1 --2- 172.21.15.78:0/4054654427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e4464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.415+0000 7f6156d9d640 1 -- 172.21.15.78:0/4054654427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664e4c0650 con 0x55664e442d30 2024-09-11T01:12:47.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.416+0000 7f6156d9d640 1 --2- 172.21.15.78:0/4054654427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e442d30 0x55664e4bff40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6150005710 tx=0x7f6150005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622a30c5772894e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:47.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.416+0000 7f613dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:12:47.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.416+0000 7f613dffb640 1 -- 172.21.15.78:0/4054654427 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150003d20 con 0x55664e442d30 2024-09-11T01:12:47.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.416+0000 7f615f0ef740 1 -- 172.21.15.78:0/4054654427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61400054d0 con 0x55664e442d30 2024-09-11T01:12:47.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.416+0000 7f613dffb640 1 -- 172.21.15.78:0/4054654427 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f6150005790 con 0x55664e442d30 2024-09-11T01:12:47.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.417+0000 7f613dffb640 1 -- 172.21.15.78:0/4054654427 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f6150071050 con 0x55664e442d30 2024-09-11T01:12:47.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.417+0000 7f615f0ef740 10 client.18078 fetch_fsmap learned FSMap version 59 2024-09-11T01:12:47.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.417+0000 7f615f0ef740 1 -- 172.21.15.78:0/4054654427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55664e4b8140 con 0x55664e442d30 2024-09-11T01:12:47.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.418+0000 7f613dffb640 1 -- 172.21.15.78:0/4054654427 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61400054d0 con 0x55664e442d30 2024-09-11T01:12:47.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.418+0000 7f613dffb640 1 -- 172.21.15.78:0/4054654427 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f6150004f30 con 0x55664e442d30 2024-09-11T01:12:47.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.418+0000 7f615f0ef740 10 client.18078 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:12:47.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.418+0000 7f615f0ef740 10 client.18078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:12:47.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.418+0000 7f615f0ef740 20 client.18078 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:47.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.418+0000 7f615f0ef740 1 --2- 172.21.15.78:0/4054654427 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55664e4b88a0 0x55664e4bacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:47.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.418+0000 7f615f0ef740 4 client.18078 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:12:47.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.418+0000 7f615f0ef740 1 -- 172.21.15.78:0/4054654427 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55664e3aa850 con 0x55664e4b88a0 2024-09-11T01:12:47.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.418+0000 7f614dd9b640 1 --2- 172.21.15.78:0/4054654427 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55664e4b88a0 0x55664e4bacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:47.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.419+0000 7f614dd9b640 1 --2- 172.21.15.78:0/4054654427 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55664e4b88a0 0x55664e4bacc0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b2744759163a09f server_cookie=54389e2b514f5810 in_seq=0 out_seq=0 2024-09-11T01:12:47.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.419+0000 7f613dffb640 10 client.18078 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:47.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f613dffb640 1 -- 172.21.15.78:0/4054654427 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x55664e3aa850 con 0x55664e4b88a0 2024-09-11T01:12:47.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f613dffb640 10 client.18078 handle_command_reply: tid=0 2024-09-11T01:12:47.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f615f0ef740 1 client.18078 shutdown 2024-09-11T01:12:47.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f615f0ef740 1 -- 172.21.15.78:0/4054654427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e442d30 msgr2=0x55664e4bff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:47.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f615f0ef740 1 --2- 172.21.15.78:0/4054654427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e442d30 0x55664e4bff40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6150005710 tx=0x7f6150005b30 comp rx=0 tx=0).stop 2024-09-11T01:12:47.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f615f0ef740 1 -- 172.21.15.78:0/4054654427 shutdown_connections 2024-09-11T01:12:47.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f615f0ef740 1 --2- 172.21.15.78:0/4054654427 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55664e4b88a0 0x55664e4bacc0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f615f0ef740 1 --2- 172.21.15.78:0/4054654427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55664e442d30 0x55664e4bff40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f615f0ef740 1 --2- 172.21.15.78:0/4054654427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55664e436da0 0x55664e446a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f615f0ef740 1 --2- 172.21.15.78:0/4054654427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55664e436480 0x55664e4464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f615f0ef740 1 -- 172.21.15.78:0/4054654427 >> 172.21.15.78:0/4054654427 conn(0x55664e4342b0 msgr2=0x55664e454850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:47.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f613dffb640 0 client.18078 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:12:47.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f615f0ef740 1 -- 172.21.15.78:0/4054654427 shutdown_connections 2024-09-11T01:12:47.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f615f0ef740 1 -- 172.21.15.78:0/4054654427 wait complete. 2024-09-11T01:12:47.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.423+0000 7f615f0ef740 20 client.18078 trim_cache size 0 max 16384 2024-09-11T01:12:47.426 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:12:47.427 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:12:47.427 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:12:47.427 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:12:47.427 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:12:47.427 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:12:47.427 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.427 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.427 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.427 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.427 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.427 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.428 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:12:47.429 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.431 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.432 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.433 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.434 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.435 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:12:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:12:47.437 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:12:47.438 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:12:47.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.424+0000 7f615cd1a640 1 -- 172.21.15.78:0/2119600271 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f61440b99e0 msgr2=0x7f61440bbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:47.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.424+0000 7f615cd1a640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f61440b99e0 0x7f61440bbea0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f61480e2c80 tx=0x7f6148010f90 comp rx=0 tx=0).stop 2024-09-11T01:12:47.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.424+0000 7f615cd1a640 1 -- 172.21.15.78:0/2119600271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f615810a150 msgr2=0x7f61581a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:47.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.424+0000 7f615cd1a640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f615810a150 0x7f61581a1ad0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f614800b4a0 tx=0x7f614800b960 comp rx=0 tx=0).stop 2024-09-11T01:12:47.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.424+0000 7f615cd1a640 1 -- 172.21.15.78:0/2119600271 shutdown_connections 2024-09-11T01:12:47.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.424+0000 7f615cd1a640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f61440b99e0 0x7f61440bbea0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.424+0000 7f615cd1a640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158115a10 0x7f61581a73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.424+0000 7f615cd1a640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f615810aa70 0x7f61581a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.424+0000 7f615cd1a640 1 --2- 172.21.15.78:0/2119600271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f615810a150 0x7f61581a1ad0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:47.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.424+0000 7f615cd1a640 1 -- 172.21.15.78:0/2119600271 >> 172.21.15.78:0/2119600271 conn(0x7f615806e2e0 msgr2=0x7f6158071750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:47.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.424+0000 7f615cd1a640 1 -- 172.21.15.78:0/2119600271 shutdown_connections 2024-09-11T01:12:47.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:47.424+0000 7f615cd1a640 1 -- 172.21.15.78:0/2119600271 wait complete. 2024-09-11T01:12:47.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:47 smithi078 ceph-mon[31315]: pgmap v754: 97 pgs: 97 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 14 MiB/s wr, 857 op/s 2024-09-11T01:12:47.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 30185 34.07 MB/sec warmup 53 sec latency 265.143 ms 2024-09-11T01:12:47.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:47 smithi110 ceph-mon[34712]: pgmap v754: 97 pgs: 97 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 14 MiB/s wr, 857 op/s 2024-09-11T01:12:47.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:47 smithi112 ceph-mon[35384]: pgmap v754: 97 pgs: 97 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 14 MiB/s wr, 857 op/s 2024-09-11T01:12:48.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 30645 33.87 MB/sec warmup 54 sec latency 348.255 ms 2024-09-11T01:12:48.838 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:12:48.838 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:12:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:12:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.707+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3174682448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c841045c0 msgr2=0x7f0c8410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.707+0000 7f0c8c21a640 1 --2- 172.21.15.78:0/3174682448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c841045c0 0x7f0c8410cab0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009900 tx=0x7f0c7402a410 comp rx=0 tx=0).stop 2024-09-11T01:12:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.707+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3174682448 shutdown_connections 2024-09-11T01:12:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.707+0000 7f0c8c21a640 1 --2- 172.21.15.78:0/3174682448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c8410d100 0x7f0c8410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.707+0000 7f0c8c21a640 1 --2- 172.21.15.78:0/3174682448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c841045c0 0x7f0c8410cab0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.707+0000 7f0c8c21a640 1 --2- 172.21.15.78:0/3174682448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c84103ca0 0x7f0c84104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.707+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3174682448 >> 172.21.15.78:0/3174682448 conn(0x7f0c840fd700 msgr2=0x7f0c840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.707+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3174682448 shutdown_connections 2024-09-11T01:12:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.707+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3174682448 wait complete. 2024-09-11T01:12:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.708+0000 7f0c8c21a640 1 Processor -- start 2024-09-11T01:12:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.708+0000 7f0c8c21a640 1 -- start start 2024-09-11T01:12:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.708+0000 7f0c8c21a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84103ca0 0x7f0c8419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.708+0000 7f0c8c21a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c841045c0 0x7f0c8419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.708+0000 7f0c8c21a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c8410d100 0x7f0c841a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.708+0000 7f0c8c21a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84112db0 con 0x7f0c84103ca0 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.708+0000 7f0c8c21a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84112c30 con 0x7f0c8410d100 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.708+0000 7f0c8c21a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84112f30 con 0x7f0c841045c0 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.708+0000 7f0c89fb6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84103ca0 0x7f0c8419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.708+0000 7f0c89fb6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84103ca0 0x7f0c8419b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58424/0 (socket says 172.21.15.78:58424) 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.708+0000 7f0c89fb6640 1 -- 172.21.15.78:0/3696121843 learned_addr learned my addr 172.21.15.78:0/3696121843 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.708+0000 7f0c8a7b7640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c8410d100 0x7f0c841a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c897b5640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c841045c0 0x7f0c8419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c89fb6640 1 -- 172.21.15.78:0/3696121843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c841045c0 msgr2=0x7f0c8419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c89fb6640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c841045c0 0x7f0c8419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c89fb6640 1 -- 172.21.15.78:0/3696121843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c8410d100 msgr2=0x7f0c841a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c89fb6640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c8410d100 0x7f0c841a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c89fb6640 1 -- 172.21.15.78:0/3696121843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c841a15d0 con 0x7f0c84103ca0 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c897b5640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c841045c0 0x7f0c8419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c89fb6640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84103ca0 0x7f0c8419b520 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000b4a0 tx=0x7f0c8000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c7affd640 1 -- 172.21.15.78:0/3696121843 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80014070 con 0x7f0c84103ca0 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c7affd640 1 -- 172.21.15.78:0/3696121843 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c80010ac0 con 0x7f0c84103ca0 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c7affd640 1 -- 172.21.15.78:0/3696121843 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80017c30 con 0x7f0c84103ca0 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3696121843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c841a18c0 con 0x7f0c84103ca0 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.709+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3696121843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c841a8000 con 0x7f0c84103ca0 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.710+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3696121843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c84104080 con 0x7f0c84103ca0 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.712+0000 7f0c7affd640 1 -- 172.21.15.78:0/3696121843 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0c80010610 con 0x7f0c84103ca0 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.713+0000 7f0c7affd640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c700b5450 0x7f0c700b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.713+0000 7f0c897b5640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c700b5450 0x7f0c700b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.713+0000 7f0c7affd640 1 -- 172.21.15.78:0/3696121843 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f0c800e58c0 con 0x7f0c84103ca0 2024-09-11T01:12:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.713+0000 7f0c897b5640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c700b5450 0x7f0c700b7910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0c74005ab0 tx=0x7f0c74031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.716+0000 7f0c7affd640 1 -- 172.21.15.78:0/3696121843 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0c800ac480 con 0x7f0c84103ca0 2024-09-11T01:12:49.732 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:49 smithi078 ceph-mon[31315]: pgmap v755: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 802 op/s 2024-09-11T01:12:49.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 30959 33.44 MB/sec warmup 55 sec latency 815.932 ms 2024-09-11T01:12:49.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:49 smithi110 ceph-mon[34712]: pgmap v755: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 802 op/s 2024-09-11T01:12:49.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:49 smithi112 ceph-mon[35384]: pgmap v755: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 802 op/s 2024-09-11T01:12:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.976+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3696121843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c8419c650 con 0x7f0c84103ca0 2024-09-11T01:12:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.977+0000 7f0c7affd640 1 -- 172.21.15.78:0/3696121843 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+12014 (secure 0 0 0) 0x7f0c800abd80 con 0x7f0c84103ca0 2024-09-11T01:12:49.979 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:12:49.980 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":59,"btime":"2024-09-11T01:12:22:445035+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":59,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:12:22.122017+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T01:12:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 59 2024-09-11T01:12:49.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.980+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3696121843 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c700b5450 msgr2=0x7f0c700b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.980+0000 7f0c8c21a640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c700b5450 0x7f0c700b7910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0c74005ab0 tx=0x7f0c74031400 comp rx=0 tx=0).stop 2024-09-11T01:12:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.980+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3696121843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84103ca0 msgr2=0x7f0c8419b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.980+0000 7f0c8c21a640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84103ca0 0x7f0c8419b520 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000b4a0 tx=0x7f0c8000b960 comp rx=0 tx=0).stop 2024-09-11T01:12:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.980+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3696121843 shutdown_connections 2024-09-11T01:12:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.980+0000 7f0c8c21a640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c700b5450 0x7f0c700b7910 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.980+0000 7f0c8c21a640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c8410d100 0x7f0c841a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.980+0000 7f0c8c21a640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c841045c0 0x7f0c8419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.980+0000 7f0c8c21a640 1 --2- 172.21.15.78:0/3696121843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84103ca0 0x7f0c8419b520 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.980+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3696121843 >> 172.21.15.78:0/3696121843 conn(0x7f0c840fd700 msgr2=0x7f0c8410f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.981+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3696121843 shutdown_connections 2024-09-11T01:12:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:49.981+0000 7f0c8c21a640 1 -- 172.21.15.78:0/3696121843 wait complete. 2024-09-11T01:12:50.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:50 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3696121843' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:12:50.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 32083 33.85 MB/sec warmup 56 sec latency 956.490 ms 2024-09-11T01:12:50.815 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:12:50.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:50 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3696121843' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:12:50.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:50 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3696121843' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:12:51.170 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:12:51.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:51 smithi078 ceph-mon[31315]: pgmap v756: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 21 MiB/s wr, 1.06k op/s 2024-09-11T01:12:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.732+0000 7f6986a28640 1 -- 172.21.15.78:0/2118973339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108050 msgr2=0x7f698010e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.732+0000 7f6986a28640 1 --2- 172.21.15.78:0/2118973339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108050 0x7f698010e850 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6978009be0 tx=0x7f697802a6c0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.732+0000 7f6986a28640 1 -- 172.21.15.78:0/2118973339 shutdown_connections 2024-09-11T01:12:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.732+0000 7f6986a28640 1 --2- 172.21.15.78:0/2118973339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108050 0x7f698010e850 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.732+0000 7f6986a28640 1 --2- 172.21.15.78:0/2118973339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69801076d0 0x7f6980107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.732+0000 7f6986a28640 1 --2- 172.21.15.78:0/2118973339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6980101cc0 0x7f69801020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.732+0000 7f6986a28640 1 -- 172.21.15.78:0/2118973339 >> 172.21.15.78:0/2118973339 conn(0x7f69800fd750 msgr2=0x7f69800ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.733+0000 7f6986a28640 1 -- 172.21.15.78:0/2118973339 shutdown_connections 2024-09-11T01:12:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.733+0000 7f6986a28640 1 -- 172.21.15.78:0/2118973339 wait complete. 2024-09-11T01:12:51.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.734+0000 7f6986a28640 1 Processor -- start 2024-09-11T01:12:51.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.734+0000 7f6986a28640 1 -- start start 2024-09-11T01:12:51.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.735+0000 7f6986a28640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6980101cc0 0x7f6980199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.735+0000 7f6986a28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69801076d0 0x7f698019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.735+0000 7f6986a28640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108050 0x7f69801a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.735+0000 7f6986a28640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6980111cd0 con 0x7f6980101cc0 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.735+0000 7f6986a28640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6980111b50 con 0x7f6980108050 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.735+0000 7f6986a28640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6980111e50 con 0x7f69801076d0 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.735+0000 7f697ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6980101cc0 0x7f6980199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.735+0000 7f697f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69801076d0 0x7f698019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.735+0000 7f697f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69801076d0 0x7f698019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:48714/0 (socket says 172.21.15.78:48714) 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.735+0000 7f697f7fe640 1 -- 172.21.15.78:0/533742281 learned_addr learned my addr 172.21.15.78:0/533742281 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.735+0000 7f6984fc5640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108050 0x7f69801a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.736+0000 7f697f7fe640 1 -- 172.21.15.78:0/533742281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108050 msgr2=0x7f69801a5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.736+0000 7f697f7fe640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108050 0x7f69801a5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.736+0000 7f697f7fe640 1 -- 172.21.15.78:0/533742281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6980101cc0 msgr2=0x7f6980199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:51.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.736+0000 7f697f7fe640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6980101cc0 0x7f6980199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.736+0000 7f697f7fe640 1 -- 172.21.15.78:0/533742281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6980197d90 con 0x7f69801076d0 2024-09-11T01:12:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.736+0000 7f697f7fe640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69801076d0 0x7f698019a4e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6974009fd0 tx=0x7f697400edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.736+0000 7f697d7fa640 1 -- 172.21.15.78:0/533742281 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974024070 con 0x7f69801076d0 2024-09-11T01:12:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.736+0000 7f697d7fa640 1 -- 172.21.15.78:0/533742281 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f697401f440 con 0x7f69801076d0 2024-09-11T01:12:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.736+0000 7f697d7fa640 1 -- 172.21.15.78:0/533742281 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974024070 con 0x7f69801076d0 2024-09-11T01:12:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.737+0000 7f6986a28640 1 -- 172.21.15.78:0/533742281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6980198080 con 0x7f69801076d0 2024-09-11T01:12:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.737+0000 7f6986a28640 1 -- 172.21.15.78:0/533742281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6980198570 con 0x7f69801076d0 2024-09-11T01:12:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.740+0000 7f697d7fa640 1 -- 172.21.15.78:0/533742281 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f697400cab0 con 0x7f69801076d0 2024-09-11T01:12:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.741+0000 7f697d7fa640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69680b5850 0x7f69680b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f6988dfd740 1 Processor -- start 2024-09-11T01:12:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f697ffff640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69680b5850 0x7f69680b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f6988dfd740 1 -- start start 2024-09-11T01:12:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f697d7fa640 1 -- 172.21.15.78:0/533742281 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f697400c8b0 con 0x7f69801076d0 2024-09-11T01:12:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f6988dfd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 0x55c7de1718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f6988dfd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de171df0 0x55c7de17d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f6988dfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de17dd80 0x55c7de180170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f6988dfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c7de0e7ea0 con 0x55c7de17dd80 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f6988dfd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c7de0d72d0 con 0x55c7de1714d0 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f6988dfd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7de0db0e0 con 0x55c7de171df0 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f697f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de171df0 0x55c7de17d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f697ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 0x55c7de1718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f697ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 0x55c7de1718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43352/0 (socket says 172.21.15.78:43352) 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f697ffff640 1 -- 172.21.15.78:0/4236569374 learned_addr learned my addr 172.21.15.78:0/4236569374 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f6984fc5640 1 --2- 172.21.15.78:0/4236569374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de17dd80 0x55c7de180170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.742+0000 7f697ffff640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69680b5850 0x7f69680b7d10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6970005fd0 tx=0x7f69700113e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.743+0000 7f697ffff640 1 -- 172.21.15.78:0/4236569374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de171df0 msgr2=0x55c7de17d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.743+0000 7f697ffff640 1 --2- 172.21.15.78:0/4236569374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de171df0 0x55c7de17d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.743+0000 7f697ffff640 1 -- 172.21.15.78:0/4236569374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de17dd80 msgr2=0x55c7de180170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.743+0000 7f697ffff640 1 --2- 172.21.15.78:0/4236569374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de17dd80 0x55c7de180170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.743+0000 7f697ffff640 1 -- 172.21.15.78:0/4236569374 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7dde35650 con 0x55c7de1714d0 2024-09-11T01:12:51.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.743+0000 7f697ffff640 1 --2- 172.21.15.78:0/4236569374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 0x55c7de1718b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6970014470 tx=0x7f6970019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e94abc20af8026 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:51.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.743+0000 7f69667fc640 1 -- 172.21.15.78:0/4236569374 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69700142d0 con 0x55c7de1714d0 2024-09-11T01:12:51.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.743+0000 7f69667fc640 1 -- 172.21.15.78:0/4236569374 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6970012900 con 0x55c7de1714d0 2024-09-11T01:12:51.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.743+0000 7f69667fc640 1 -- 172.21.15.78:0/4236569374 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970027910 con 0x55c7de1714d0 2024-09-11T01:12:51.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.744+0000 7f6988dfd740 1 -- 172.21.15.78:0/4236569374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 msgr2=0x55c7de1718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:51.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.744+0000 7f6988dfd740 1 --2- 172.21.15.78:0/4236569374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 0x55c7de1718b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6970014470 tx=0x7f6970019000 comp rx=0 tx=0).stop 2024-09-11T01:12:51.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.744+0000 7f6988dfd740 1 -- 172.21.15.78:0/4236569374 shutdown_connections 2024-09-11T01:12:51.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.744+0000 7f6988dfd740 1 --2- 172.21.15.78:0/4236569374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de17dd80 0x55c7de180170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.744+0000 7f6988dfd740 1 --2- 172.21.15.78:0/4236569374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de171df0 0x55c7de17d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.744+0000 7f6988dfd740 1 --2- 172.21.15.78:0/4236569374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 0x55c7de1718b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.744+0000 7f6988dfd740 1 -- 172.21.15.78:0/4236569374 >> 172.21.15.78:0/4236569374 conn(0x55c7de16f300 msgr2=0x55c7de1704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.744+0000 7f6988dfd740 1 -- 172.21.15.78:0/4236569374 shutdown_connections 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.744+0000 7f6988dfd740 1 -- 172.21.15.78:0/4236569374 wait complete. 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.745+0000 7f6988dfd740 1 -- start start 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.745+0000 7f6988dfd740 1 Processor -- start 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.745+0000 7f6988dfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 0x55c7de17d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.745+0000 7f6988dfd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de171df0 0x55c7de17aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.745+0000 7f6988dfd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de17dd80 0x55c7de17b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.745+0000 7f6988dfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c7ddf38700 con 0x55c7de1714d0 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.745+0000 7f6988dfd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c7de072160 con 0x55c7de171df0 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.745+0000 7f697ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 0x55c7de17d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.745+0000 7f6988dfd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7de180a00 con 0x55c7de17dd80 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f697ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 0x55c7de17d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58482/0 (socket says 172.21.15.78:58482) 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f697ffff640 1 -- 172.21.15.78:0/3288273528 learned_addr learned my addr 172.21.15.78:0/3288273528 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f697f7fe640 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de171df0 0x55c7de17aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f6984fc5640 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de17dd80 0x55c7de17b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f6988dfd740 10 client.0 _ll_register_callbacks cb 0x55c7de08b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f697ffff640 1 -- 172.21.15.78:0/3288273528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de17dd80 msgr2=0x55c7de17b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f697ffff640 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de17dd80 0x55c7de17b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f697ffff640 1 -- 172.21.15.78:0/3288273528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de171df0 msgr2=0x55c7de17aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f697ffff640 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de171df0 0x55c7de17aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f697ffff640 1 -- 172.21.15.78:0/3288273528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7de09fc90 con 0x55c7de1714d0 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f697ffff640 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 0x55c7de17d680 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6970009970 tx=0x7f6970012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce828fde818a262 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f695f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f695f7fe640 1 -- 172.21.15.78:0/3288273528 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970027690 con 0x55c7de1714d0 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.746+0000 7f6988dfd740 1 -- 172.21.15.78:0/3288273528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6970012900 con 0x55c7de1714d0 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.747+0000 7f695f7fe640 1 -- 172.21.15.78:0/3288273528 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f6970027830 con 0x55c7de1714d0 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.747+0000 7f695f7fe640 1 -- 172.21.15.78:0/3288273528 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f6970027a30 con 0x55c7de1714d0 2024-09-11T01:12:51.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.747+0000 7f6988dfd740 10 client.18096 fetch_fsmap learned FSMap version 59 2024-09-11T01:12:51.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.747+0000 7f6988dfd740 1 -- 172.21.15.78:0/3288273528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55c7de18c350 con 0x55c7de1714d0 2024-09-11T01:12:51.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.748+0000 7f695f7fe640 1 -- 172.21.15.78:0/3288273528 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6970012900 con 0x55c7de1714d0 2024-09-11T01:12:51.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.748+0000 7f695f7fe640 1 -- 172.21.15.78:0/3288273528 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f6970030a20 con 0x55c7de1714d0 2024-09-11T01:12:51.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.748+0000 7f6988dfd740 10 client.18096 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:12:51.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.748+0000 7f6988dfd740 10 client.18096 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:12:51.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.748+0000 7f6988dfd740 20 client.18096 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:51.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.748+0000 7f6988dfd740 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c7de18d760 0x55c7de18fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.748+0000 7f6988dfd740 4 client.18096 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:12:51.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.748+0000 7f6988dfd740 1 -- 172.21.15.78:0/3288273528 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7de0e9df0 con 0x55c7de18d760 2024-09-11T01:12:51.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.749+0000 7f697f7fe640 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c7de18d760 0x55c7de18fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.749+0000 7f697f7fe640 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c7de18d760 0x55c7de18fb50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a7924c814d72d95d server_cookie=c697bc090d1851cc in_seq=0 out_seq=0 2024-09-11T01:12:51.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.749+0000 7f695f7fe640 10 client.18096 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:12:51.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.751+0000 7f695f7fe640 1 -- 172.21.15.78:0/3288273528 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c7de0e9df0 con 0x55c7de18d760 2024-09-11T01:12:51.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f695f7fe640 10 client.18096 handle_command_reply: tid=0 2024-09-11T01:12:51.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f6988dfd740 1 client.18096 shutdown 2024-09-11T01:12:51.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f6988dfd740 1 -- 172.21.15.78:0/3288273528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 msgr2=0x55c7de17d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:51.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f6988dfd740 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 0x55c7de17d680 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6970009970 tx=0x7f6970012700 comp rx=0 tx=0).stop 2024-09-11T01:12:51.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f6988dfd740 1 -- 172.21.15.78:0/3288273528 shutdown_connections 2024-09-11T01:12:51.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f6988dfd740 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c7de18d760 0x55c7de18fb50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f6988dfd740 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de17dd80 0x55c7de17b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f6988dfd740 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de171df0 0x55c7de17aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f6988dfd740 1 --2- 172.21.15.78:0/3288273528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 0x55c7de17d680 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f6988dfd740 1 -- 172.21.15.78:0/3288273528 >> 172.21.15.78:0/3288273528 conn(0x55c7de16f300 msgr2=0x55c7de174e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:51.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f695f7fe640 0 client.18096 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:12:51.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f6988dfd740 1 -- 172.21.15.78:0/3288273528 shutdown_connections 2024-09-11T01:12:51.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f6988dfd740 1 -- 172.21.15.78:0/3288273528 wait complete. 2024-09-11T01:12:51.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.752+0000 7f6988dfd740 20 client.18096 trim_cache size 0 max 16384 2024-09-11T01:12:51.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.772+0000 7f6988dfd740 1 Processor -- start 2024-09-11T01:12:51.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.772+0000 7f6988dfd740 1 -- start start 2024-09-11T01:12:51.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.772+0000 7f6988dfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 0x55c7de170ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.772+0000 7f6988dfd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de171df0 0x55c7de17f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.772+0000 7f6988dfd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de17dd80 0x55c7de17f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.772+0000 7f6988dfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69801ac3e0 con 0x55c7de1714d0 2024-09-11T01:12:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.772+0000 7f6988dfd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69801ae160 con 0x55c7de171df0 2024-09-11T01:12:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.772+0000 7f6988dfd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69801acab0 con 0x55c7de17dd80 2024-09-11T01:12:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.772+0000 7f697ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 0x55c7de170ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.772+0000 7f697ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 0x55c7de170ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58490/0 (socket says 172.21.15.78:58490) 2024-09-11T01:12:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.772+0000 7f6984fc5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de17dd80 0x55c7de17f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.772+0000 7f697ffff640 1 -- 172.21.15.78:0/1170370079 learned_addr learned my addr 172.21.15.78:0/1170370079 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:51.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.773+0000 7f697f7fe640 1 --2- 172.21.15.78:0/1170370079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de171df0 0x55c7de17f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.773+0000 7f697ffff640 1 -- 172.21.15.78:0/1170370079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de17dd80 msgr2=0x55c7de17f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:51.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.773+0000 7f697ffff640 1 --2- 172.21.15.78:0/1170370079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de17dd80 0x55c7de17f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.773+0000 7f697ffff640 1 -- 172.21.15.78:0/1170370079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de171df0 msgr2=0x55c7de17f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:51.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.773+0000 7f697ffff640 1 --2- 172.21.15.78:0/1170370079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de171df0 0x55c7de17f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.773+0000 7f697ffff640 1 -- 172.21.15.78:0/1170370079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7de17b450 con 0x55c7de1714d0 2024-09-11T01:12:51.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.773+0000 7f6984fc5640 1 --2- 172.21.15.78:0/1170370079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de17dd80 0x55c7de17f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:12:51.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.774+0000 7f697ffff640 1 --2- 172.21.15.78:0/1170370079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 0x55c7de170ad0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6970014420 tx=0x7f6970011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb4a4461a4b05d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:51.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.774+0000 7f6965ffb640 1 -- 172.21.15.78:0/1170370079 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970027400 con 0x55c7de1714d0 2024-09-11T01:12:51.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.774+0000 7f6965ffb640 1 -- 172.21.15.78:0/1170370079 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6970039de0 con 0x55c7de1714d0 2024-09-11T01:12:51.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.774+0000 7f6965ffb640 1 -- 172.21.15.78:0/1170370079 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970027bd0 con 0x55c7de1714d0 2024-09-11T01:12:51.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.775+0000 7f6988dfd740 1 -- 172.21.15.78:0/1170370079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 msgr2=0x55c7de170ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:51.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.775+0000 7f6988dfd740 1 --2- 172.21.15.78:0/1170370079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 0x55c7de170ad0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6970014420 tx=0x7f6970011f70 comp rx=0 tx=0).stop 2024-09-11T01:12:51.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 33218 34.58 MB/sec warmup 57 sec latency 43.492 ms 2024-09-11T01:12:51.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.775+0000 7f6988dfd740 1 -- 172.21.15.78:0/1170370079 shutdown_connections 2024-09-11T01:12:51.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.775+0000 7f6988dfd740 1 --2- 172.21.15.78:0/1170370079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de17dd80 0x55c7de17f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.775+0000 7f6988dfd740 1 --2- 172.21.15.78:0/1170370079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de171df0 0x55c7de17f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.775+0000 7f6988dfd740 1 --2- 172.21.15.78:0/1170370079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de1714d0 0x55c7de170ad0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.775+0000 7f6988dfd740 1 -- 172.21.15.78:0/1170370079 >> 172.21.15.78:0/1170370079 conn(0x55c7de16f300 msgr2=0x55c7de1703c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:51.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.775+0000 7f6988dfd740 1 -- 172.21.15.78:0/1170370079 shutdown_connections 2024-09-11T01:12:51.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.776+0000 7f6988dfd740 1 -- 172.21.15.78:0/1170370079 wait complete. 2024-09-11T01:12:51.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.776+0000 7f6988dfd740 1 -- start start 2024-09-11T01:12:51.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.776+0000 7f6988dfd740 1 Processor -- start 2024-09-11T01:12:51.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.777+0000 7f6988dfd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 0x55c7de185e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.777+0000 7f6988dfd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de171df0 0x55c7de1863c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.777+0000 7f6988dfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de17dd80 0x55c7de180c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.777+0000 7f697ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 0x55c7de185e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.777+0000 7f6988dfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69801ad9d0 con 0x55c7de17dd80 2024-09-11T01:12:51.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.777+0000 7f6988dfd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69801ad240 con 0x55c7de1714d0 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.777+0000 7f697ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 0x55c7de185e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43374/0 (socket says 172.21.15.78:43374) 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.777+0000 7f6988dfd740 1 -- 172.21.15.78:0/2927891079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69801aa3e0 con 0x55c7de171df0 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.777+0000 7f697ffff640 1 -- 172.21.15.78:0/2927891079 learned_addr learned my addr 172.21.15.78:0/2927891079 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.777+0000 7f6984fc5640 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de17dd80 0x55c7de180c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.777+0000 7f6988dfd740 10 client.0 _ll_register_callbacks cb 0x7f6978034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.777+0000 7f697f7fe640 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de171df0 0x55c7de1863c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.778+0000 7f6984fc5640 1 -- 172.21.15.78:0/2927891079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de171df0 msgr2=0x55c7de1863c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.778+0000 7f6984fc5640 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de171df0 0x55c7de1863c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.778+0000 7f6984fc5640 1 -- 172.21.15.78:0/2927891079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 msgr2=0x55c7de185e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.778+0000 7f6984fc5640 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 0x55c7de185e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.778+0000 7f6984fc5640 1 -- 172.21.15.78:0/2927891079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7de181290 con 0x55c7de17dd80 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.778+0000 7f6984fc5640 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de17dd80 0x55c7de180c50 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6978009b30 tx=0x7f6978030710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7255a1d2de92c5d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:51.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.778+0000 7f695f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.778+0000 7f695f7fe640 1 -- 172.21.15.78:0/2927891079 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978036040 con 0x55c7de17dd80 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.778+0000 7f695f7fe640 1 -- 172.21.15.78:0/2927891079 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f6978003b80 con 0x55c7de17dd80 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.778+0000 7f6988dfd740 1 -- 172.21.15.78:0/2927891079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6970039de0 con 0x55c7de17dd80 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.779+0000 7f695f7fe640 1 -- 172.21.15.78:0/2927891079 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f6978003d80 con 0x55c7de17dd80 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.779+0000 7f6988dfd740 10 client.18102 fetch_fsmap learned FSMap version 59 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.779+0000 7f6988dfd740 1 -- 172.21.15.78:0/2927891079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55c7de1fac30 con 0x55c7de17dd80 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.779+0000 7f695f7fe640 1 -- 172.21.15.78:0/2927891079 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6970039de0 con 0x55c7de17dd80 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.779+0000 7f695f7fe640 1 -- 172.21.15.78:0/2927891079 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f69780044e0 con 0x55c7de17dd80 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.779+0000 7f6988dfd740 10 client.18102 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.779+0000 7f6988dfd740 10 client.18102 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.779+0000 7f6988dfd740 20 client.18102 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.779+0000 7f6988dfd740 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c7de1fb430 0x55c7de1fd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:51.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.780+0000 7f6988dfd740 4 client.18102 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:12:51.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.780+0000 7f6988dfd740 1 -- 172.21.15.78:0/2927891079 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7de0e58a0 con 0x55c7de1fb430 2024-09-11T01:12:51.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.780+0000 7f697ffff640 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c7de1fb430 0x55c7de1fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:51.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.781+0000 7f697ffff640 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c7de1fb430 0x55c7de1fd850 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e982fb8b069898d7 server_cookie=d089604a93ff9d2 in_seq=0 out_seq=0 2024-09-11T01:12:51.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.781+0000 7f695f7fe640 10 client.18102 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:12:51.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f695f7fe640 1 -- 172.21.15.78:0/2927891079 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7de0e58a0 con 0x55c7de1fb430 2024-09-11T01:12:51.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f695f7fe640 10 client.18102 handle_command_reply: tid=0 2024-09-11T01:12:51.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f6988dfd740 1 client.18102 shutdown 2024-09-11T01:12:51.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f6988dfd740 1 -- 172.21.15.78:0/2927891079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de17dd80 msgr2=0x55c7de180c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:51.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f6988dfd740 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de17dd80 0x55c7de180c50 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6978009b30 tx=0x7f6978030710 comp rx=0 tx=0).stop 2024-09-11T01:12:51.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f6988dfd740 1 -- 172.21.15.78:0/2927891079 shutdown_connections 2024-09-11T01:12:51.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f6988dfd740 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c7de1fb430 0x55c7de1fd850 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f695f7fe640 0 client.18102 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:12:51.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f6988dfd740 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7de17dd80 0x55c7de180c50 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f6988dfd740 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7de171df0 0x55c7de1863c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f6988dfd740 1 --2- 172.21.15.78:0/2927891079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7de1714d0 0x55c7de185e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f6988dfd740 1 -- 172.21.15.78:0/2927891079 >> 172.21.15.78:0/2927891079 conn(0x55c7de16f300 msgr2=0x55c7de17c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:51.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f6988dfd740 1 -- 172.21.15.78:0/2927891079 shutdown_connections 2024-09-11T01:12:51.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.783+0000 7f6988dfd740 1 -- 172.21.15.78:0/2927891079 wait complete. 2024-09-11T01:12:51.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.784+0000 7f6988dfd740 20 client.18102 trim_cache size 0 max 16384 2024-09-11T01:12:51.787 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:12:51.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.784+0000 7f6986a28640 1 -- 172.21.15.78:0/533742281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69680b5850 msgr2=0x7f69680b7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:51.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.784+0000 7f6986a28640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69680b5850 0x7f69680b7d10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6970005fd0 tx=0x7f69700113e0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.784+0000 7f6986a28640 1 -- 172.21.15.78:0/533742281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69801076d0 msgr2=0x7f698019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:51.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.785+0000 7f6986a28640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69801076d0 0x7f698019a4e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6974009fd0 tx=0x7f697400edd0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.785+0000 7f6986a28640 1 -- 172.21.15.78:0/533742281 shutdown_connections 2024-09-11T01:12:51.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.785+0000 7f6986a28640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69680b5850 0x7f69680b7d10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.785+0000 7f6986a28640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108050 0x7f69801a5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.785+0000 7f6986a28640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69801076d0 0x7f698019a4e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.785+0000 7f6986a28640 1 --2- 172.21.15.78:0/533742281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6980101cc0 0x7f6980199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:51.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.785+0000 7f6986a28640 1 -- 172.21.15.78:0/533742281 >> 172.21.15.78:0/533742281 conn(0x7f69800fd750 msgr2=0x7f69800fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:51.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.785+0000 7f6986a28640 1 -- 172.21.15.78:0/533742281 shutdown_connections 2024-09-11T01:12:51.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:51.785+0000 7f6986a28640 1 -- 172.21.15.78:0/533742281 wait complete. 2024-09-11T01:12:51.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:51 smithi110 ceph-mon[34712]: pgmap v756: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 21 MiB/s wr, 1.06k op/s 2024-09-11T01:12:51.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:51 smithi112 ceph-mon[35384]: pgmap v756: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 21 MiB/s wr, 1.06k op/s 2024-09-11T01:12:52.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 33974 34.54 MB/sec warmup 58 sec latency 221.723 ms 2024-09-11T01:12:53.548 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:12:53.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 34898 34.82 MB/sec warmup 59 sec latency 73.753 ms 2024-09-11T01:12:53.885 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:12:53.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:53 smithi110 ceph-mon[34712]: pgmap v757: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 24 MiB/s wr, 1.15k op/s 2024-09-11T01:12:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:53 smithi112 ceph-mon[35384]: pgmap v757: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 24 MiB/s wr, 1.15k op/s 2024-09-11T01:12:53.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:53 smithi078 ceph-mon[31315]: pgmap v757: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 24 MiB/s wr, 1.15k op/s 2024-09-11T01:12:54.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 35285 34.52 MB/sec warmup 60 sec latency 752.355 ms 2024-09-11T01:12:55.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 36179 34.78 MB/sec warmup 61 sec latency 192.088 ms 2024-09-11T01:12:55.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:55 smithi110 ceph-mon[34712]: pgmap v758: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 23 MiB/s wr, 1.10k op/s 2024-09-11T01:12:55.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:55 smithi112 ceph-mon[35384]: pgmap v758: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 23 MiB/s wr, 1.10k op/s 2024-09-11T01:12:55.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:55 smithi078 ceph-mon[31315]: pgmap v758: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 23 MiB/s wr, 1.10k op/s 2024-09-11T01:12:56.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 37019 34.93 MB/sec warmup 62 sec latency 161.878 ms 2024-09-11T01:12:57.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.233+0000 7fd4d2ead640 1 -- 172.21.15.78:0/3084140033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4cc10d100 msgr2=0x7fd4cc10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:57.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.233+0000 7fd4d2ead640 1 --2- 172.21.15.78:0/3084140033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4cc10d100 0x7fd4cc10f560 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009be0 tx=0x7fd4c402a6c0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.233+0000 7fd4d2ead640 1 -- 172.21.15.78:0/3084140033 shutdown_connections 2024-09-11T01:12:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.233+0000 7fd4d2ead640 1 --2- 172.21.15.78:0/3084140033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4cc10d100 0x7fd4cc10f560 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.233+0000 7fd4d2ead640 1 --2- 172.21.15.78:0/3084140033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc1045c0 0x7fd4cc10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.233+0000 7fd4d2ead640 1 --2- 172.21.15.78:0/3084140033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4cc103ca0 0x7fd4cc104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.233+0000 7fd4d2ead640 1 -- 172.21.15.78:0/3084140033 >> 172.21.15.78:0/3084140033 conn(0x7fd4cc0fd700 msgr2=0x7fd4cc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.233+0000 7fd4d2ead640 1 -- 172.21.15.78:0/3084140033 shutdown_connections 2024-09-11T01:12:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.233+0000 7fd4d2ead640 1 -- 172.21.15.78:0/3084140033 wait complete. 2024-09-11T01:12:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.233+0000 7fd4d2ead640 1 Processor -- start 2024-09-11T01:12:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.233+0000 7fd4d2ead640 1 -- start start 2024-09-11T01:12:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.234+0000 7fd4d2ead640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4cc103ca0 0x7fd4cc199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.234+0000 7fd4d2ead640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4cc1045c0 0x7fd4cc199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.234+0000 7fd4d2ead640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc10d100 0x7fd4cc19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.234+0000 7fd4d2ead640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc112d70 con 0x7fd4cc1045c0 2024-09-11T01:12:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.234+0000 7fd4d2ead640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc112bf0 con 0x7fd4cc103ca0 2024-09-11T01:12:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.234+0000 7fd4d2ead640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc112ef0 con 0x7fd4cc10d100 2024-09-11T01:12:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.234+0000 7fd4cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4cc1045c0 0x7fd4cc199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.234+0000 7fd4d144a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc10d100 0x7fd4cc19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.234+0000 7fd4cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4cc1045c0 0x7fd4cc199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60768/0 (socket says 172.21.15.78:60768) 2024-09-11T01:12:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.234+0000 7fd4cbfff640 1 -- 172.21.15.78:0/2137788999 learned_addr learned my addr 172.21.15.78:0/2137788999 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.235+0000 7fd4cbfff640 1 -- 172.21.15.78:0/2137788999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc10d100 msgr2=0x7fd4cc19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.235+0000 7fd4cbfff640 1 --2- 172.21.15.78:0/2137788999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc10d100 0x7fd4cc19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.235+0000 7fd4cbfff640 1 -- 172.21.15.78:0/2137788999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4cc103ca0 msgr2=0x7fd4cc199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.235+0000 7fd4cbfff640 1 --2- 172.21.15.78:0/2137788999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4cc103ca0 0x7fd4cc199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.235+0000 7fd4cbfff640 1 -- 172.21.15.78:0/2137788999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc19f2d0 con 0x7fd4cc1045c0 2024-09-11T01:12:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.235+0000 7fd4cbfff640 1 --2- 172.21.15.78:0/2137788999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4cc1045c0 0x7fd4cc199760 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800e750 tx=0x7fd4b800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:57.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.235+0000 7fd4c9ffb640 1 -- 172.21.15.78:0/2137788999 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b800ccc0 con 0x7fd4cc1045c0 2024-09-11T01:12:57.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.236+0000 7fd4c9ffb640 1 -- 172.21.15.78:0/2137788999 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b800ee60 con 0x7fd4cc1045c0 2024-09-11T01:12:57.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.236+0000 7fd4c9ffb640 1 -- 172.21.15.78:0/2137788999 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8003e20 con 0x7fd4cc1045c0 2024-09-11T01:12:57.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.236+0000 7fd4d2ead640 1 -- 172.21.15.78:0/2137788999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc19f5c0 con 0x7fd4cc1045c0 2024-09-11T01:12:57.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.237+0000 7fd4d2ead640 1 -- 172.21.15.78:0/2137788999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc101830 con 0x7fd4cc1045c0 2024-09-11T01:12:57.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.241+0000 7fd4d5282740 1 Processor -- start 2024-09-11T01:12:57.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.241+0000 7fd4d5282740 1 -- start start 2024-09-11T01:12:57.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.241+0000 7fd4d5282740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5c0f0 0x561576f5c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.241+0000 7fd4d5282740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f5ca10 0x561576f68460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.241+0000 7fd4d5282740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f689a0 0x561576f6ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.241+0000 7fd4d5282740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561576ed2ea0 con 0x561576f5c0f0 2024-09-11T01:12:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.241+0000 7fd4d5282740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561576ec22d0 con 0x561576f689a0 2024-09-11T01:12:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.241+0000 7fd4d5282740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561576ec60e0 con 0x561576f5ca10 2024-09-11T01:12:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.241+0000 7fd4d144a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5c0f0 0x561576f5c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.241+0000 7fd4d144a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5c0f0 0x561576f5c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60776/0 (socket says 172.21.15.78:60776) 2024-09-11T01:12:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.241+0000 7fd4d144a640 1 -- 172.21.15.78:0/3894212142 learned_addr learned my addr 172.21.15.78:0/3894212142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.241+0000 7fd4d0c49640 1 --2- 172.21.15.78:0/3894212142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f5ca10 0x561576f68460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.242+0000 7fd4d144a640 1 -- 172.21.15.78:0/3894212142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f5ca10 msgr2=0x561576f68460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.242+0000 7fd4d144a640 1 --2- 172.21.15.78:0/3894212142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f5ca10 0x561576f68460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.242+0000 7fd4d144a640 1 -- 172.21.15.78:0/3894212142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f689a0 msgr2=0x561576f6ad90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:12:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.242+0000 7fd4d144a640 1 --2- 172.21.15.78:0/3894212142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f689a0 0x561576f6ad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.242+0000 7fd4d144a640 1 -- 172.21.15.78:0/3894212142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561576c20650 con 0x561576f5c0f0 2024-09-11T01:12:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.243+0000 7fd4d144a640 1 --2- 172.21.15.78:0/3894212142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5c0f0 0x561576f5c4d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009be0 tx=0x7fd4c4005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ad407fa2194dff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.243+0000 7fd4b6ffd640 1 -- 172.21.15.78:0/3894212142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4049070 con 0x561576f5c0f0 2024-09-11T01:12:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.243+0000 7fd4b6ffd640 1 -- 172.21.15.78:0/3894212142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c4003e20 con 0x561576f5c0f0 2024-09-11T01:12:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.243+0000 7fd4b6ffd640 1 -- 172.21.15.78:0/3894212142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4004280 con 0x561576f5c0f0 2024-09-11T01:12:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.244+0000 7fd4c9ffb640 1 -- 172.21.15.78:0/2137788999 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd4b8004000 con 0x7fd4cc1045c0 2024-09-11T01:12:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.244+0000 7fd4d5282740 1 -- 172.21.15.78:0/3894212142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5c0f0 msgr2=0x561576f5c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.244+0000 7fd4d5282740 1 --2- 172.21.15.78:0/3894212142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5c0f0 0x561576f5c4d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009be0 tx=0x7fd4c4005ec0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.244+0000 7fd4d5282740 1 -- 172.21.15.78:0/3894212142 shutdown_connections 2024-09-11T01:12:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.244+0000 7fd4d5282740 1 --2- 172.21.15.78:0/3894212142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f689a0 0x561576f6ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.244+0000 7fd4d5282740 1 --2- 172.21.15.78:0/3894212142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f5ca10 0x561576f68460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.244+0000 7fd4d5282740 1 --2- 172.21.15.78:0/3894212142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5c0f0 0x561576f5c4d0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.244+0000 7fd4d5282740 1 -- 172.21.15.78:0/3894212142 >> 172.21.15.78:0/3894212142 conn(0x561576f59f20 msgr2=0x561576f5b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.244+0000 7fd4d5282740 1 -- 172.21.15.78:0/3894212142 shutdown_connections 2024-09-11T01:12:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.244+0000 7fd4d5282740 1 -- 172.21.15.78:0/3894212142 wait complete. 2024-09-11T01:12:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.245+0000 7fd4d5282740 1 -- start start 2024-09-11T01:12:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.245+0000 7fd4c9ffb640 1 --2- 172.21.15.78:0/2137788999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4bc0b54b0 0x7fd4bc0b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.245+0000 7fd4d5282740 1 Processor -- start 2024-09-11T01:12:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.245+0000 7fd4c9ffb640 1 -- 172.21.15.78:0/2137788999 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7fd4b80e6120 con 0x7fd4cc1045c0 2024-09-11T01:12:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4d5282740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f5c0f0 0x561576f68300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4d5282740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5ca10 0x561576f65750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4d5282740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f689a0 0x561576f65dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4d5282740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561576d23700 con 0x561576f5ca10 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4d5282740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561576e5d160 con 0x561576f5c0f0 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4d5282740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561576f6b5c0 con 0x561576f689a0 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4d5282740 10 client.0 _ll_register_callbacks cb 0x561576e769e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4d144a640 1 --2- 172.21.15.78:0/2137788999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4bc0b54b0 0x7fd4bc0b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5ca10 0x561576f65750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5ca10 0x561576f65750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60780/0 (socket says 172.21.15.78:60780) 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4cbfff640 1 -- 172.21.15.78:0/852678128 learned_addr learned my addr 172.21.15.78:0/852678128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4d144a640 1 --2- 172.21.15.78:0/852678128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f689a0 0x561576f65dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4d144a640 1 --2- 172.21.15.78:0/2137788999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4bc0b54b0 0x7fd4bc0b7970 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009370 tx=0x7fd4c40308a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4cbfff640 1 -- 172.21.15.78:0/852678128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f689a0 msgr2=0x561576f65dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4cbfff640 1 --2- 172.21.15.78:0/852678128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f689a0 0x561576f65dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4cbfff640 1 -- 172.21.15.78:0/852678128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f5c0f0 msgr2=0x561576f68300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4cbfff640 1 --2- 172.21.15.78:0/852678128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f5c0f0 0x561576f68300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.246+0000 7fd4cbfff640 1 -- 172.21.15.78:0/852678128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561576e8ac90 con 0x561576f5ca10 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.247+0000 7fd4cbfff640 1 --2- 172.21.15.78:0/852678128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5ca10 0x561576f65750 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80a8bb0 tx=0x7fd4b8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7c0362ef9808ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.247+0000 7fd4abfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.247+0000 7fd4abfff640 1 -- 172.21.15.78:0/852678128 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80acce0 con 0x561576f5ca10 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.247+0000 7fd4d5282740 1 -- 172.21.15.78:0/852678128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c4003e20 con 0x561576f5ca10 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.247+0000 7fd4abfff640 1 -- 172.21.15.78:0/852678128 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7fd4b80ab840 con 0x561576f5ca10 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.247+0000 7fd4abfff640 1 -- 172.21.15.78:0/852678128 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fd4b80ac5e0 con 0x561576f5ca10 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.247+0000 7fd4d5282740 10 client.18114 fetch_fsmap learned FSMap version 59 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.247+0000 7fd4d5282740 1 -- 172.21.15.78:0/852678128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x561576f77020 con 0x561576f5ca10 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.247+0000 7fd4abfff640 1 -- 172.21.15.78:0/852678128 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c4003e20 con 0x561576f5ca10 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.248+0000 7fd4abfff640 1 -- 172.21.15.78:0/852678128 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7fd4b80a6ce0 con 0x561576f5ca10 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.248+0000 7fd4d5282740 10 client.18114 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.248+0000 7fd4d5282740 10 client.18114 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.248+0000 7fd4d5282740 20 client.18114 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.248+0000 7fd4d5282740 1 --2- 172.21.15.78:0/852678128 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561576f78480 0x561576f7a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.248+0000 7fd4d5282740 4 client.18114 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:12:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.248+0000 7fd4d5282740 1 -- 172.21.15.78:0/852678128 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561576ed4df0 con 0x561576f78480 2024-09-11T01:12:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.248+0000 7fd4d0c49640 1 --2- 172.21.15.78:0/852678128 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561576f78480 0x561576f7a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.248+0000 7fd4abfff640 10 client.18114 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:12:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.248+0000 7fd4d0c49640 1 --2- 172.21.15.78:0/852678128 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561576f78480 0x561576f7a870 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a795d95c1fb395b server_cookie=404a51841d3364b3 in_seq=0 out_seq=0 2024-09-11T01:12:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4abfff640 1 -- 172.21.15.78:0/852678128 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561576ed4df0 con 0x561576f78480 2024-09-11T01:12:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4abfff640 10 client.18114 handle_command_reply: tid=0 2024-09-11T01:12:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4d5282740 1 client.18114 shutdown 2024-09-11T01:12:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4d5282740 1 -- 172.21.15.78:0/852678128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5ca10 msgr2=0x561576f65750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4d5282740 1 --2- 172.21.15.78:0/852678128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5ca10 0x561576f65750 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80a8bb0 tx=0x7fd4b8003820 comp rx=0 tx=0).stop 2024-09-11T01:12:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4d5282740 1 -- 172.21.15.78:0/852678128 shutdown_connections 2024-09-11T01:12:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4d5282740 1 --2- 172.21.15.78:0/852678128 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561576f78480 0x561576f7a870 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4abfff640 0 client.18114 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:12:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4d5282740 1 --2- 172.21.15.78:0/852678128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f689a0 0x561576f65dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4d5282740 1 --2- 172.21.15.78:0/852678128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f5ca10 0x561576f65750 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4d5282740 1 --2- 172.21.15.78:0/852678128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f5c0f0 0x561576f68300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4d5282740 1 -- 172.21.15.78:0/852678128 >> 172.21.15.78:0/852678128 conn(0x561576f59f20 msgr2=0x561576f5fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.251+0000 7fd4d5282740 1 -- 172.21.15.78:0/852678128 shutdown_connections 2024-09-11T01:12:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.252+0000 7fd4d5282740 1 -- 172.21.15.78:0/852678128 wait complete. 2024-09-11T01:12:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.252+0000 7fd4d5282740 20 client.18114 trim_cache size 0 max 16384 2024-09-11T01:12:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d5282740 1 Processor -- start 2024-09-11T01:12:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d5282740 1 -- start start 2024-09-11T01:12:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d5282740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f5c0f0 0x561576f5b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d5282740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f689a0 0x561576f747f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d5282740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 0x561576f69d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d5282740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1ac380 con 0x561576f78480 2024-09-11T01:12:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d5282740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1ae100 con 0x561576f689a0 2024-09-11T01:12:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d5282740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1aca50 con 0x561576f5c0f0 2024-09-11T01:12:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d144a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 0x561576f69d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d144a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 0x561576f69d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60794/0 (socket says 172.21.15.78:60794) 2024-09-11T01:12:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d144a640 1 -- 172.21.15.78:0/2271228689 learned_addr learned my addr 172.21.15.78:0/2271228689 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d0c49640 1 --2- 172.21.15.78:0/2271228689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f5c0f0 0x561576f5b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d144a640 1 -- 172.21.15.78:0/2271228689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f5c0f0 msgr2=0x561576f5b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d144a640 1 --2- 172.21.15.78:0/2271228689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f5c0f0 0x561576f5b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d144a640 1 -- 172.21.15.78:0/2271228689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f689a0 msgr2=0x561576f747f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:12:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d144a640 1 --2- 172.21.15.78:0/2271228689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f689a0 0x561576f747f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.271+0000 7fd4d144a640 1 -- 172.21.15.78:0/2271228689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561576f7ae50 con 0x561576f78480 2024-09-11T01:12:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.272+0000 7fd4d144a640 1 --2- 172.21.15.78:0/2271228689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 0x561576f69d90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009be0 tx=0x7fd4c4054070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896e2cf29adb6e1c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.272+0000 7fd4b67fc640 1 -- 172.21.15.78:0/2271228689 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4059070 con 0x561576f78480 2024-09-11T01:12:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.272+0000 7fd4b67fc640 1 -- 172.21.15.78:0/2271228689 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c4039db0 con 0x561576f78480 2024-09-11T01:12:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.272+0000 7fd4b67fc640 1 -- 172.21.15.78:0/2271228689 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4044d00 con 0x561576f78480 2024-09-11T01:12:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.273+0000 7fd4d5282740 1 -- 172.21.15.78:0/2271228689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 msgr2=0x561576f69d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.273+0000 7fd4d5282740 1 --2- 172.21.15.78:0/2271228689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 0x561576f69d90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009be0 tx=0x7fd4c4054070 comp rx=0 tx=0).stop 2024-09-11T01:12:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.273+0000 7fd4d5282740 1 -- 172.21.15.78:0/2271228689 shutdown_connections 2024-09-11T01:12:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.273+0000 7fd4d5282740 1 --2- 172.21.15.78:0/2271228689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 0x561576f69d90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.273+0000 7fd4d5282740 1 --2- 172.21.15.78:0/2271228689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f689a0 0x561576f747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.273+0000 7fd4d5282740 1 --2- 172.21.15.78:0/2271228689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f5c0f0 0x561576f5b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.273+0000 7fd4d5282740 1 -- 172.21.15.78:0/2271228689 >> 172.21.15.78:0/2271228689 conn(0x561576f59f20 msgr2=0x561576f5af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.273+0000 7fd4d5282740 1 -- 172.21.15.78:0/2271228689 shutdown_connections 2024-09-11T01:12:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.273+0000 7fd4d5282740 1 -- 172.21.15.78:0/2271228689 wait complete. 2024-09-11T01:12:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.274+0000 7fd4d5282740 1 -- start start 2024-09-11T01:12:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.274+0000 7fd4d5282740 1 Processor -- start 2024-09-11T01:12:57.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4d5282740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f5c0f0 0x561576f6b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4d5282740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f689a0 0x561576f6bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4d5282740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 0x561576f6c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4d144a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 0x561576f6c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f689a0 0x561576f6bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4d144a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 0x561576f6c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60798/0 (socket says 172.21.15.78:60798) 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4d144a640 1 -- 172.21.15.78:0/2235982880 learned_addr learned my addr 172.21.15.78:0/2235982880 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4d5282740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1ad970 con 0x561576f78480 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4d5282740 1 -- 172.21.15.78:0/2235982880 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1ad1e0 con 0x561576f5c0f0 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4d5282740 1 -- 172.21.15.78:0/2235982880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1aa380 con 0x561576f689a0 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4d5282740 10 client.0 _ll_register_callbacks cb 0x7fd4b80a84f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4d0c49640 1 --2- 172.21.15.78:0/2235982880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f5c0f0 0x561576f6b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4cbfff640 1 -- 172.21.15.78:0/2235982880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f5c0f0 msgr2=0x561576f6b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4cbfff640 1 --2- 172.21.15.78:0/2235982880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f5c0f0 0x561576f6b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4cbfff640 1 -- 172.21.15.78:0/2235982880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 msgr2=0x561576f6c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4cbfff640 1 --2- 172.21.15.78:0/2235982880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 0x561576f6c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.275+0000 7fd4cbfff640 1 -- 172.21.15.78:0/2235982880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561576fe5360 con 0x561576f689a0 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.276+0000 7fd4cbfff640 1 --2- 172.21.15.78:0/2235982880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f689a0 0x561576f6bd50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x561576f64f60 tx=0x7fd4b80ac220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=820c9f870d4a82f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.276+0000 7fd4abfff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.276+0000 7fd4abfff640 1 -- 172.21.15.78:0/2235982880 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80c7380 con 0x561576f689a0 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.276+0000 7fd4abfff640 1 -- 172.21.15.78:0/2235982880 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7fd4b80a7f30 con 0x561576f689a0 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.276+0000 7fd4d5282740 1 -- 172.21.15.78:0/2235982880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c4039db0 con 0x561576f689a0 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.276+0000 7fd4abfff640 1 -- 172.21.15.78:0/2235982880 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fd4b80ac3b0 con 0x561576f689a0 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.276+0000 7fd4d5282740 10 client.27238 fetch_fsmap learned FSMap version 59 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.276+0000 7fd4d5282740 1 -- 172.21.15.78:0/2235982880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x561576fe6660 con 0x561576f689a0 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.276+0000 7fd4abfff640 1 -- 172.21.15.78:0/2235982880 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c4039db0 con 0x561576f689a0 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.277+0000 7fd4abfff640 1 -- 172.21.15.78:0/2235982880 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7fd4b80a6ce0 con 0x561576f689a0 2024-09-11T01:12:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.277+0000 7fd4d5282740 10 client.27238 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.277+0000 7fd4d5282740 10 client.27238 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.277+0000 7fd4d5282740 20 client.27238 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.277+0000 7fd4d5282740 1 --2- 172.21.15.78:0/2235982880 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561576fe6cd0 0x561576fe90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.277+0000 7fd4d5282740 4 client.27238 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.277+0000 7fd4d5282740 1 -- 172.21.15.78:0/2235982880 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561576ed08a0 con 0x561576fe6cd0 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.277+0000 7fd4d0c49640 1 --2- 172.21.15.78:0/2235982880 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561576fe6cd0 0x561576fe90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.277+0000 7fd4abfff640 10 client.27238 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.277+0000 7fd4d0c49640 1 --2- 172.21.15.78:0/2235982880 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561576fe6cd0 0x561576fe90f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bce2b7db9da18889 server_cookie=e52320c535f0051e in_seq=0 out_seq=0 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4abfff640 1 -- 172.21.15.78:0/2235982880 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561576ed08a0 con 0x561576fe6cd0 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4abfff640 10 client.27238 handle_command_reply: tid=0 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4d5282740 1 client.27238 shutdown 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4d5282740 1 -- 172.21.15.78:0/2235982880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f689a0 msgr2=0x561576f6bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4d5282740 1 --2- 172.21.15.78:0/2235982880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f689a0 0x561576f6bd50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x561576f64f60 tx=0x7fd4b80ac220 comp rx=0 tx=0).stop 2024-09-11T01:12:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4d5282740 1 -- 172.21.15.78:0/2235982880 shutdown_connections 2024-09-11T01:12:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4d5282740 1 --2- 172.21.15.78:0/2235982880 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x561576fe6cd0 0x561576fe90f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4d5282740 1 --2- 172.21.15.78:0/2235982880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561576f78480 0x561576f6c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4d5282740 1 --2- 172.21.15.78:0/2235982880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561576f689a0 0x561576f6bd50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4d5282740 1 --2- 172.21.15.78:0/2235982880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561576f5c0f0 0x561576f6b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4d5282740 1 -- 172.21.15.78:0/2235982880 >> 172.21.15.78:0/2235982880 conn(0x561576f59f20 msgr2=0x561576f66d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4abfff640 0 client.27238 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:12:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4d5282740 1 -- 172.21.15.78:0/2235982880 shutdown_connections 2024-09-11T01:12:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.278+0000 7fd4d5282740 1 -- 172.21.15.78:0/2235982880 wait complete. 2024-09-11T01:12:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.279+0000 7fd4d5282740 20 client.27238 trim_cache size 0 max 16384 2024-09-11T01:12:57.284 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:12:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.279+0000 7fd4d2ead640 1 -- 172.21.15.78:0/2137788999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4bc0b54b0 msgr2=0x7fd4bc0b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.279+0000 7fd4d2ead640 1 --2- 172.21.15.78:0/2137788999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4bc0b54b0 0x7fd4bc0b7970 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009370 tx=0x7fd4c40308a0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.279+0000 7fd4d2ead640 1 -- 172.21.15.78:0/2137788999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4cc1045c0 msgr2=0x7fd4cc199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.279+0000 7fd4d2ead640 1 --2- 172.21.15.78:0/2137788999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4cc1045c0 0x7fd4cc199760 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800e750 tx=0x7fd4b800ec10 comp rx=0 tx=0).stop 2024-09-11T01:12:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.280+0000 7fd4d2ead640 1 -- 172.21.15.78:0/2137788999 shutdown_connections 2024-09-11T01:12:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.280+0000 7fd4d2ead640 1 --2- 172.21.15.78:0/2137788999 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd4bc0b54b0 0x7fd4bc0b7970 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.280+0000 7fd4d2ead640 1 --2- 172.21.15.78:0/2137788999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc10d100 0x7fd4cc19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.280+0000 7fd4d2ead640 1 --2- 172.21.15.78:0/2137788999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4cc1045c0 0x7fd4cc199760 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.280+0000 7fd4d2ead640 1 --2- 172.21.15.78:0/2137788999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4cc103ca0 0x7fd4cc199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.280+0000 7fd4d2ead640 1 -- 172.21.15.78:0/2137788999 >> 172.21.15.78:0/2137788999 conn(0x7fd4cc0fd700 msgr2=0x7fd4cc0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.280+0000 7fd4d2ead640 1 -- 172.21.15.78:0/2137788999 shutdown_connections 2024-09-11T01:12:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:57.280+0000 7fd4d2ead640 1 -- 172.21.15.78:0/2137788999 wait complete. 2024-09-11T01:12:57.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 37937 35.31 MB/sec warmup 63 sec latency 66.864 ms 2024-09-11T01:12:57.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:57 smithi110 ceph-mon[34712]: pgmap v759: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 24 MiB/s wr, 1.13k op/s 2024-09-11T01:12:57.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:57 smithi112 ceph-mon[35384]: pgmap v759: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 24 MiB/s wr, 1.13k op/s 2024-09-11T01:12:57.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:57 smithi078 ceph-mon[31315]: pgmap v759: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 24 MiB/s wr, 1.13k op/s 2024-09-11T01:12:58.030 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:12:58.378 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:12:58.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 38752 35.38 MB/sec warmup 64 sec latency 53.436 ms 2024-09-11T01:12:58.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.910+0000 7f262729f640 1 -- 172.21.15.78:0/2850573775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2620102190 msgr2=0x7f262010eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:58.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.910+0000 7f262729f640 1 --2- 172.21.15.78:0/2850573775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2620102190 0x7f262010eee0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f2610009900 tx=0x7f261002a410 comp rx=0 tx=0).stop 2024-09-11T01:12:58.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.911+0000 7f262729f640 1 -- 172.21.15.78:0/2850573775 shutdown_connections 2024-09-11T01:12:58.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.911+0000 7f262729f640 1 --2- 172.21.15.78:0/2850573775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f262010f420 0x7f2620111810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.911+0000 7f262729f640 1 --2- 172.21.15.78:0/2850573775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2620102190 0x7f262010eee0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.911+0000 7f262729f640 1 --2- 172.21.15.78:0/2850573775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2620101870 0x7f2620101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.911+0000 7f262729f640 1 -- 172.21.15.78:0/2850573775 >> 172.21.15.78:0/2850573775 conn(0x7f26200fd740 msgr2=0x7f26200ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:58.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.911+0000 7f262729f640 1 -- 172.21.15.78:0/2850573775 shutdown_connections 2024-09-11T01:12:58.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.912+0000 7f262729f640 1 -- 172.21.15.78:0/2850573775 wait complete. 2024-09-11T01:12:58.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.912+0000 7f262729f640 1 Processor -- start 2024-09-11T01:12:58.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.912+0000 7f262729f640 1 -- start start 2024-09-11T01:12:58.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.913+0000 7f262729f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2620101870 0x7f262006ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.913+0000 7f262729f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2620102190 0x7f262006d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.913+0000 7f262729f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f262010f420 0x7f2620072740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.913+0000 7f262729f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2620114f70 con 0x7f262010f420 2024-09-11T01:12:58.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.913+0000 7f262729f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2620114df0 con 0x7f2620101870 2024-09-11T01:12:58.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.913+0000 7f262729f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26201150f0 con 0x7f2620102190 2024-09-11T01:12:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.913+0000 7f262483a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2620102190 0x7f262006d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.913+0000 7f262483a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2620102190 0x7f262006d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:59116/0 (socket says 172.21.15.78:59116) 2024-09-11T01:12:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.913+0000 7f262483a640 1 -- 172.21.15.78:0/3028703352 learned_addr learned my addr 172.21.15.78:0/3028703352 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.913+0000 7f262583c640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f262010f420 0x7f2620072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.913+0000 7f262503b640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2620101870 0x7f262006ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.914+0000 7f262503b640 1 -- 172.21.15.78:0/3028703352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2620102190 msgr2=0x7f262006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.914+0000 7f262503b640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2620102190 0x7f262006d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.914+0000 7f262503b640 1 -- 172.21.15.78:0/3028703352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f262010f420 msgr2=0x7f2620072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.914+0000 7f262503b640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f262010f420 0x7f2620072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.914+0000 7f262503b640 1 -- 172.21.15.78:0/3028703352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2620072ec0 con 0x7f2620101870 2024-09-11T01:12:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.914+0000 7f262503b640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2620101870 0x7f262006ce60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f261c00d6d0 tx=0x7f261c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.915+0000 7f26167fc640 1 -- 172.21.15.78:0/3028703352 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c017070 con 0x7f2620101870 2024-09-11T01:12:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.915+0000 7f26167fc640 1 -- 172.21.15.78:0/3028703352 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f261c009820 con 0x7f2620101870 2024-09-11T01:12:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.915+0000 7f26167fc640 1 -- 172.21.15.78:0/3028703352 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c004c40 con 0x7f2620101870 2024-09-11T01:12:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.915+0000 7f262729f640 1 -- 172.21.15.78:0/3028703352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26200731b0 con 0x7f2620101870 2024-09-11T01:12:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.916+0000 7f262729f640 1 -- 172.21.15.78:0/3028703352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26200738b0 con 0x7f2620101870 2024-09-11T01:12:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.919+0000 7f2629674740 1 Processor -- start 2024-09-11T01:12:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.920+0000 7f26167fc640 1 -- 172.21.15.78:0/3028703352 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f261c005220 con 0x7f2620101870 2024-09-11T01:12:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.920+0000 7f2629674740 1 -- start start 2024-09-11T01:12:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.920+0000 7f2629674740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6f4d0 0x55793ce6f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.921+0000 7f2629674740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6fdf0 0x55793ce7b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.921+0000 7f2629674740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce7bd80 0x55793ce7e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.921+0000 7f2629674740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55793cde5ea0 con 0x55793ce6fdf0 2024-09-11T01:12:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.921+0000 7f2629674740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55793cdd52d0 con 0x55793ce7bd80 2024-09-11T01:12:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.921+0000 7f2629674740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55793cdd90e0 con 0x55793ce6f4d0 2024-09-11T01:12:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.921+0000 7f262583c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6f4d0 0x55793ce6f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.921+0000 7f262583c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6f4d0 0x55793ce6f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:59118/0 (socket says 172.21.15.78:59118) 2024-09-11T01:12:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.921+0000 7f262583c640 1 -- 172.21.15.78:0/85618782 learned_addr learned my addr 172.21.15.78:0/85618782 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.921+0000 7f262503b640 1 --2- 172.21.15.78:0/85618782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6fdf0 0x55793ce7b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.922+0000 7f262583c640 1 -- 172.21.15.78:0/85618782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce7bd80 msgr2=0x55793ce7e170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:12:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.922+0000 7f262483a640 1 --2- 172.21.15.78:0/85618782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce7bd80 0x55793ce7e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.922+0000 7f262583c640 1 --2- 172.21.15.78:0/85618782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce7bd80 0x55793ce7e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.922+0000 7f262583c640 1 -- 172.21.15.78:0/85618782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6fdf0 msgr2=0x55793ce7b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.922+0000 7f262583c640 1 --2- 172.21.15.78:0/85618782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6fdf0 0x55793ce7b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.922+0000 7f262583c640 1 -- 172.21.15.78:0/85618782 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793cb33650 con 0x55793ce6f4d0 2024-09-11T01:12:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.922+0000 7f262583c640 1 --2- 172.21.15.78:0/85618782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6f4d0 0x55793ce6f8b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2618009fc0 tx=0x7f2618010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61728c5bc8f11126 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.922+0000 7f260b7fe640 1 -- 172.21.15.78:0/85618782 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618016070 con 0x55793ce6f4d0 2024-09-11T01:12:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.922+0000 7f260b7fe640 1 -- 172.21.15.78:0/85618782 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2618004d20 con 0x55793ce6f4d0 2024-09-11T01:12:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.922+0000 7f260b7fe640 1 -- 172.21.15.78:0/85618782 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26180053e0 con 0x55793ce6f4d0 2024-09-11T01:12:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.923+0000 7f26167fc640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f260c0b5550 0x7f260c0b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.923+0000 7f262503b640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f260c0b5550 0x7f260c0b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.923+0000 7f26167fc640 1 -- 172.21.15.78:0/3028703352 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f261c0e59d0 con 0x7f2620101870 2024-09-11T01:12:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.923+0000 7f2629674740 1 -- 172.21.15.78:0/85618782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6f4d0 msgr2=0x55793ce6f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.923+0000 7f2629674740 1 --2- 172.21.15.78:0/85618782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6f4d0 0x55793ce6f8b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2618009fc0 tx=0x7f2618010800 comp rx=0 tx=0).stop 2024-09-11T01:12:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.924+0000 7f262503b640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f260c0b5550 0x7f260c0b7a10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55793ce6e1e0 tx=0x7f261c093040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.924+0000 7f2629674740 1 -- 172.21.15.78:0/85618782 shutdown_connections 2024-09-11T01:12:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.924+0000 7f2629674740 1 --2- 172.21.15.78:0/85618782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce7bd80 0x55793ce7e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.924+0000 7f2629674740 1 --2- 172.21.15.78:0/85618782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6fdf0 0x55793ce7b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.924+0000 7f2629674740 1 --2- 172.21.15.78:0/85618782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6f4d0 0x55793ce6f8b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.924+0000 7f2629674740 1 -- 172.21.15.78:0/85618782 >> 172.21.15.78:0/85618782 conn(0x55793ce6d300 msgr2=0x55793ce6e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.924+0000 7f2629674740 1 -- 172.21.15.78:0/85618782 shutdown_connections 2024-09-11T01:12:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.924+0000 7f2629674740 1 -- 172.21.15.78:0/85618782 wait complete. 2024-09-11T01:12:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.924+0000 7f2629674740 1 -- start start 2024-09-11T01:12:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.925+0000 7f2629674740 1 Processor -- start 2024-09-11T01:12:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.925+0000 7f2629674740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6f4d0 0x55793ce7b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.925+0000 7f2629674740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce6fdf0 0x55793ce78aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.925+0000 7f2629674740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce7bd80 0x55793ce79080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.925+0000 7f2629674740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55793cc36700 con 0x55793ce6f4d0 2024-09-11T01:12:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.925+0000 7f2629674740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55793cd70160 con 0x55793ce6fdf0 2024-09-11T01:12:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.925+0000 7f2629674740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55793ce7ea00 con 0x55793ce7bd80 2024-09-11T01:12:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.925+0000 7f2629674740 10 client.0 _ll_register_callbacks cb 0x55793cd899e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f262583c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce7bd80 0x55793ce79080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f262583c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6f4d0 0x55793ce7b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f262483a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce6fdf0 0x55793ce78aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f262583c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6f4d0 0x55793ce7b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60828/0 (socket says 172.21.15.78:60828) 2024-09-11T01:12:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f262483a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce6fdf0 0x55793ce78aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:52388/0 (socket says 172.21.15.78:52388) 2024-09-11T01:12:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f262583c640 1 -- 172.21.15.78:0/1632369628 learned_addr learned my addr 172.21.15.78:0/1632369628 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f262483a640 1 -- 172.21.15.78:0/1632369628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce7bd80 msgr2=0x55793ce79080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f262483a640 1 --2- 172.21.15.78:0/1632369628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce7bd80 0x55793ce79080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f262483a640 1 -- 172.21.15.78:0/1632369628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6f4d0 msgr2=0x55793ce7b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f262483a640 1 --2- 172.21.15.78:0/1632369628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6f4d0 0x55793ce7b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f262483a640 1 -- 172.21.15.78:0/1632369628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793cd9dc90 con 0x55793ce6fdf0 2024-09-11T01:12:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f262483a640 1 --2- 172.21.15.78:0/1632369628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce6fdf0 0x55793ce78aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2610004770 tx=0x7f2610031e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa436598d47d411 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.926+0000 7f2608ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:12:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.927+0000 7f2608ff9640 1 -- 172.21.15.78:0/1632369628 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610040070 con 0x55793ce6fdf0 2024-09-11T01:12:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.927+0000 7f2629674740 1 -- 172.21.15.78:0/1632369628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2618004d20 con 0x55793ce6fdf0 2024-09-11T01:12:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.927+0000 7f2608ff9640 1 -- 172.21.15.78:0/1632369628 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f2610004040 con 0x55793ce6fdf0 2024-09-11T01:12:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.927+0000 7f2608ff9640 1 -- 172.21.15.78:0/1632369628 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2610035040 con 0x55793ce6fdf0 2024-09-11T01:12:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.927+0000 7f2629674740 10 client.27299 fetch_fsmap learned FSMap version 59 2024-09-11T01:12:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.927+0000 7f2629674740 1 -- 172.21.15.78:0/1632369628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55793ce8a3f0 con 0x55793ce6fdf0 2024-09-11T01:12:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.927+0000 7f2608ff9640 1 -- 172.21.15.78:0/1632369628 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26100054d0 con 0x55793ce6fdf0 2024-09-11T01:12:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.928+0000 7f2608ff9640 1 -- 172.21.15.78:0/1632369628 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f2610004930 con 0x55793ce6fdf0 2024-09-11T01:12:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.928+0000 7f2629674740 10 client.27299 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:12:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.928+0000 7f2629674740 10 client.27299 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:12:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.928+0000 7f2629674740 20 client.27299 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.928+0000 7f2629674740 1 --2- 172.21.15.78:0/1632369628 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55793ce8bc70 0x55793ce8e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.928+0000 7f2629674740 4 client.27299 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:12:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.928+0000 7f2629674740 1 -- 172.21.15.78:0/1632369628 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2618004d20 con 0x55793ce8bc70 2024-09-11T01:12:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.928+0000 7f262583c640 1 --2- 172.21.15.78:0/1632369628 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55793ce8bc70 0x55793ce8e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.929+0000 7f262583c640 1 --2- 172.21.15.78:0/1632369628 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55793ce8bc70 0x55793ce8e090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92f30ccb59d0379a server_cookie=aba408650db8d972 in_seq=0 out_seq=0 2024-09-11T01:12:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.929+0000 7f2608ff9640 10 client.27299 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:12:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2608ff9640 1 -- 172.21.15.78:0/1632369628 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f2618004d20 con 0x55793ce8bc70 2024-09-11T01:12:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2608ff9640 10 client.27299 handle_command_reply: tid=0 2024-09-11T01:12:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2629674740 1 client.27299 shutdown 2024-09-11T01:12:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2629674740 1 -- 172.21.15.78:0/1632369628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce6fdf0 msgr2=0x55793ce78aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2629674740 1 --2- 172.21.15.78:0/1632369628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce6fdf0 0x55793ce78aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2610004770 tx=0x7f2610031e80 comp rx=0 tx=0).stop 2024-09-11T01:12:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2629674740 1 -- 172.21.15.78:0/1632369628 shutdown_connections 2024-09-11T01:12:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2629674740 1 --2- 172.21.15.78:0/1632369628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce7bd80 0x55793ce79080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2629674740 1 --2- 172.21.15.78:0/1632369628 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55793ce8bc70 0x55793ce8e090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2629674740 1 --2- 172.21.15.78:0/1632369628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce6fdf0 0x55793ce78aa0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2608ff9640 0 client.27299 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:12:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2629674740 1 --2- 172.21.15.78:0/1632369628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6f4d0 0x55793ce7b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2629674740 1 -- 172.21.15.78:0/1632369628 >> 172.21.15.78:0/1632369628 conn(0x55793ce6d300 msgr2=0x55793ce72e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2629674740 1 -- 172.21.15.78:0/1632369628 shutdown_connections 2024-09-11T01:12:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.930+0000 7f2629674740 1 -- 172.21.15.78:0/1632369628 wait complete. 2024-09-11T01:12:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.931+0000 7f2629674740 20 client.27299 trim_cache size 0 max 16384 2024-09-11T01:12:58.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.947+0000 7f2629674740 1 Processor -- start 2024-09-11T01:12:58.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.947+0000 7f2629674740 1 -- start start 2024-09-11T01:12:58.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.947+0000 7f2629674740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6f4d0 0x55793ce87cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.947+0000 7f2629674740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce6fdf0 0x55793ce77320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.947+0000 7f2629674740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce7bd80 0x55793ce77860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.947+0000 7f2629674740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f26201bd4e0 con 0x55793ce7bd80 2024-09-11T01:12:58.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.947+0000 7f2629674740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26201bf260 con 0x55793ce6fdf0 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.947+0000 7f2629674740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26201bdbb0 con 0x55793ce6f4d0 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.947+0000 7f262583c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce7bd80 0x55793ce77860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.947+0000 7f262583c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce7bd80 0x55793ce77860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60834/0 (socket says 172.21.15.78:60834) 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.947+0000 7f262583c640 1 -- 172.21.15.78:0/1473923328 learned_addr learned my addr 172.21.15.78:0/1473923328 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.947+0000 7f262483a640 1 --2- 172.21.15.78:0/1473923328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce6fdf0 0x55793ce77320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.948+0000 7f262583c640 1 --2- 172.21.15.78:0/1473923328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6f4d0 0x55793ce87cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.948+0000 7f262583c640 1 -- 172.21.15.78:0/1473923328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6f4d0 msgr2=0x55793ce87cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.948+0000 7f262583c640 1 --2- 172.21.15.78:0/1473923328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6f4d0 0x55793ce87cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.948+0000 7f262583c640 1 -- 172.21.15.78:0/1473923328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce6fdf0 msgr2=0x55793ce77320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.948+0000 7f262583c640 1 --2- 172.21.15.78:0/1473923328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce6fdf0 0x55793ce77320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.948+0000 7f262583c640 1 -- 172.21.15.78:0/1473923328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793ce6fc20 con 0x55793ce7bd80 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.948+0000 7f262583c640 1 --2- 172.21.15.78:0/1473923328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce7bd80 0x55793ce77860 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f2618009f90 tx=0x7f2618009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d15110feff892f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.948+0000 7f260affd640 1 -- 172.21.15.78:0/1473923328 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618037070 con 0x55793ce7bd80 2024-09-11T01:12:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.948+0000 7f260affd640 1 -- 172.21.15.78:0/1473923328 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2618003e20 con 0x55793ce7bd80 2024-09-11T01:12:58.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.948+0000 7f260affd640 1 -- 172.21.15.78:0/1473923328 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618016040 con 0x55793ce7bd80 2024-09-11T01:12:58.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.949+0000 7f2629674740 1 -- 172.21.15.78:0/1473923328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce7bd80 msgr2=0x55793ce77860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:58.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.949+0000 7f2629674740 1 --2- 172.21.15.78:0/1473923328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce7bd80 0x55793ce77860 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f2618009f90 tx=0x7f2618009600 comp rx=0 tx=0).stop 2024-09-11T01:12:58.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.949+0000 7f2629674740 1 -- 172.21.15.78:0/1473923328 shutdown_connections 2024-09-11T01:12:58.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.949+0000 7f2629674740 1 --2- 172.21.15.78:0/1473923328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce7bd80 0x55793ce77860 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.949+0000 7f2629674740 1 --2- 172.21.15.78:0/1473923328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce6fdf0 0x55793ce77320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.949+0000 7f2629674740 1 --2- 172.21.15.78:0/1473923328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6f4d0 0x55793ce87cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.949+0000 7f2629674740 1 -- 172.21.15.78:0/1473923328 >> 172.21.15.78:0/1473923328 conn(0x55793ce6d300 msgr2=0x55793ce6e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.949+0000 7f2629674740 1 -- 172.21.15.78:0/1473923328 shutdown_connections 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.949+0000 7f2629674740 1 -- 172.21.15.78:0/1473923328 wait complete. 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.949+0000 7f2629674740 1 -- start start 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.950+0000 7f2629674740 1 Processor -- start 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.950+0000 7f2629674740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6f4d0 0x55793ce83d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.950+0000 7f2629674740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6fdf0 0x55793ce842b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.950+0000 7f2629674740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce7bd80 0x55793ce7ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.950+0000 7f2629674740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f26201bead0 con 0x55793ce6f4d0 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.950+0000 7f2629674740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26201be340 con 0x55793ce7bd80 2024-09-11T01:12:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.950+0000 7f2629674740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26201076f0 con 0x55793ce6fdf0 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.950+0000 7f2629674740 10 client.0 _ll_register_callbacks cb 0x7f2610031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.950+0000 7f262583c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6f4d0 0x55793ce83d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.950+0000 7f262483a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6fdf0 0x55793ce842b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.950+0000 7f262483a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6fdf0 0x55793ce842b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:59134/0 (socket says 172.21.15.78:59134) 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.950+0000 7f262483a640 1 -- 172.21.15.78:0/1787902753 learned_addr learned my addr 172.21.15.78:0/1787902753 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f262483a640 1 -- 172.21.15.78:0/1787902753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce7bd80 msgr2=0x55793ce7ece0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f262483a640 1 --2- 172.21.15.78:0/1787902753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce7bd80 0x55793ce7ece0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f262483a640 1 -- 172.21.15.78:0/1787902753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6f4d0 msgr2=0x55793ce83d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f262483a640 1 --2- 172.21.15.78:0/1787902753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6f4d0 0x55793ce83d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f262483a640 1 -- 172.21.15.78:0/1787902753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793ce7f2d0 con 0x55793ce6fdf0 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f262483a640 1 --2- 172.21.15.78:0/1787902753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6fdf0 0x55793ce842b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2610009a20 tx=0x7f26100049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af409b4638247530 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f2608ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f2608ff9640 1 -- 172.21.15.78:0/1787902753 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261005b060 con 0x55793ce6fdf0 2024-09-11T01:12:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f2629674740 1 -- 172.21.15.78:0/1787902753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2618003e20 con 0x55793ce6fdf0 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f2608ff9640 1 -- 172.21.15.78:0/1787902753 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f2610004fa0 con 0x55793ce6fdf0 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f2608ff9640 1 -- 172.21.15.78:0/1787902753 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2610040070 con 0x55793ce6fdf0 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f2629674740 10 client.27256 fetch_fsmap learned FSMap version 59 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.951+0000 7f2629674740 1 -- 172.21.15.78:0/1787902753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55793cef9440 con 0x55793ce6fdf0 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.952+0000 7f2608ff9640 1 -- 172.21.15.78:0/1787902753 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2618003e20 con 0x55793ce6fdf0 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.952+0000 7f2608ff9640 1 -- 172.21.15.78:0/1787902753 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f261003c560 con 0x55793ce6fdf0 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.952+0000 7f2629674740 10 client.27256 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.952+0000 7f2629674740 10 client.27256 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.952+0000 7f2629674740 20 client.27256 populate_metadata read hostname 'smithi078' 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.952+0000 7f2629674740 1 --2- 172.21.15.78:0/1787902753 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55793ce8bc70 0x55793cefbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.952+0000 7f2629674740 4 client.27256 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.952+0000 7f2629674740 1 -- 172.21.15.78:0/1787902753 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55793cde7df0 con 0x55793ce8bc70 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.952+0000 7f262583c640 1 --2- 172.21.15.78:0/1787902753 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55793ce8bc70 0x55793cefbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:12:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.952+0000 7f262583c640 1 --2- 172.21.15.78:0/1787902753 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55793ce8bc70 0x55793cefbc90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5c2d7cebec51d10 server_cookie=d871ff9566c0d039 in_seq=0 out_seq=0 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.952+0000 7f2608ff9640 10 client.27256 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2608ff9640 1 -- 172.21.15.78:0/1787902753 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55793cde7df0 con 0x55793ce8bc70 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2608ff9640 10 client.27256 handle_command_reply: tid=0 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2629674740 1 client.27256 shutdown 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2629674740 1 -- 172.21.15.78:0/1787902753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6fdf0 msgr2=0x55793ce842b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2629674740 1 --2- 172.21.15.78:0/1787902753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6fdf0 0x55793ce842b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2610009a20 tx=0x7f26100049b0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2629674740 1 -- 172.21.15.78:0/1787902753 shutdown_connections 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2629674740 1 --2- 172.21.15.78:0/1787902753 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55793ce8bc70 0x55793cefbc90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2608ff9640 0 client.27256 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2629674740 1 --2- 172.21.15.78:0/1787902753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55793ce7bd80 0x55793ce7ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2629674740 1 --2- 172.21.15.78:0/1787902753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55793ce6fdf0 0x55793ce842b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2629674740 1 --2- 172.21.15.78:0/1787902753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55793ce6f4d0 0x55793ce83d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2629674740 1 -- 172.21.15.78:0/1787902753 >> 172.21.15.78:0/1787902753 conn(0x55793ce6d300 msgr2=0x55793ce772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2629674740 1 -- 172.21.15.78:0/1787902753 shutdown_connections 2024-09-11T01:12:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2629674740 1 -- 172.21.15.78:0/1787902753 wait complete. 2024-09-11T01:12:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.953+0000 7f2629674740 20 client.27256 trim_cache size 0 max 16384 2024-09-11T01:12:58.960 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:12:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.954+0000 7f262729f640 1 -- 172.21.15.78:0/3028703352 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f260c0b5550 msgr2=0x7f260c0b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.954+0000 7f262729f640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f260c0b5550 0x7f260c0b7a10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55793ce6e1e0 tx=0x7f261c093040 comp rx=0 tx=0).stop 2024-09-11T01:12:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.954+0000 7f262729f640 1 -- 172.21.15.78:0/3028703352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2620101870 msgr2=0x7f262006ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:12:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.954+0000 7f262729f640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2620101870 0x7f262006ce60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f261c00d6d0 tx=0x7f261c00db90 comp rx=0 tx=0).stop 2024-09-11T01:12:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.954+0000 7f262729f640 1 -- 172.21.15.78:0/3028703352 shutdown_connections 2024-09-11T01:12:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.954+0000 7f262729f640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f260c0b5550 0x7f260c0b7a10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.954+0000 7f262729f640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f262010f420 0x7f2620072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.954+0000 7f262729f640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2620102190 0x7f262006d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.954+0000 7f262729f640 1 --2- 172.21.15.78:0/3028703352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2620101870 0x7f262006ce60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:12:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.954+0000 7f262729f640 1 -- 172.21.15.78:0/3028703352 >> 172.21.15.78:0/3028703352 conn(0x7f26200fd740 msgr2=0x7f2620110780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:12:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.954+0000 7f262729f640 1 -- 172.21.15.78:0/3028703352 shutdown_connections 2024-09-11T01:12:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:12:58.954+0000 7f262729f640 1 -- 172.21.15.78:0/3028703352 wait complete. 2024-09-11T01:12:59.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 39275 35.21 MB/sec warmup 65 sec latency 575.816 ms 2024-09-11T01:12:59.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:12:59 smithi110 ceph-mon[34712]: pgmap v760: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 1.00k op/s 2024-09-11T01:12:59.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:12:59 smithi112 ceph-mon[35384]: pgmap v760: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 1.00k op/s 2024-09-11T01:12:59.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:12:59 smithi078 ceph-mon[31315]: pgmap v760: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 1.00k op/s 2024-09-11T01:13:00.398 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:13:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:13:00.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 39700 35.00 MB/sec warmup 66 sec latency 598.939 ms 2024-09-11T01:13:01.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.471+0000 7fb776cac640 1 -- 172.21.15.78:0/225933729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb770101ca0 msgr2=0x7fb770102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.471+0000 7fb776cac640 1 --2- 172.21.15.78:0/225933729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb770101ca0 0x7fb770102080 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fb758009ca0 tx=0x7fb75802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.471+0000 7fb776cac640 1 -- 172.21.15.78:0/225933729 shutdown_connections 2024-09-11T01:13:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.471+0000 7fb776cac640 1 --2- 172.21.15.78:0/225933729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb770108030 0x7fb77010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.471+0000 7fb776cac640 1 --2- 172.21.15.78:0/225933729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7701076b0 0x7fb770107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.471+0000 7fb776cac640 1 --2- 172.21.15.78:0/225933729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb770101ca0 0x7fb770102080 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.471+0000 7fb776cac640 1 -- 172.21.15.78:0/225933729 >> 172.21.15.78:0/225933729 conn(0x7fb7700fd730 msgr2=0x7fb7700ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.471+0000 7fb776cac640 1 -- 172.21.15.78:0/225933729 shutdown_connections 2024-09-11T01:13:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.471+0000 7fb776cac640 1 -- 172.21.15.78:0/225933729 wait complete. 2024-09-11T01:13:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.471+0000 7fb776cac640 1 Processor -- start 2024-09-11T01:13:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.471+0000 7fb776cac640 1 -- start start 2024-09-11T01:13:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.471+0000 7fb776cac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb770101ca0 0x7fb7701992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb776cac640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7701076b0 0x7fb770199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb776cac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb770108030 0x7fb77019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb776cac640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb770111cf0 con 0x7fb770101ca0 2024-09-11T01:13:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb776cac640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb770111b70 con 0x7fb7701076b0 2024-09-11T01:13:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb776cac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb770111e70 con 0x7fb770108030 2024-09-11T01:13:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb774a48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb770101ca0 0x7fb7701992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb774a48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb770101ca0 0x7fb7701992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60850/0 (socket says 172.21.15.78:60850) 2024-09-11T01:13:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb774a48640 1 -- 172.21.15.78:0/2829833082 learned_addr learned my addr 172.21.15.78:0/2829833082 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb76ffff640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7701076b0 0x7fb770199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb775249640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb770108030 0x7fb77019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb774a48640 1 -- 172.21.15.78:0/2829833082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb770108030 msgr2=0x7fb77019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb774a48640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb770108030 0x7fb77019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb774a48640 1 -- 172.21.15.78:0/2829833082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7701076b0 msgr2=0x7fb770199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb774a48640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7701076b0 0x7fb770199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb774a48640 1 -- 172.21.15.78:0/2829833082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77019f3a0 con 0x7fb770101ca0 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb775249640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb770108030 0x7fb77019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb774a48640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb770101ca0 0x7fb7701992f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb75802ade0 tx=0x7fb758003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb76dffb640 1 -- 172.21.15.78:0/2829833082 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758003d60 con 0x7fb770101ca0 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb76dffb640 1 -- 172.21.15.78:0/2829833082 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7580044f0 con 0x7fb770101ca0 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb776cac640 1 -- 172.21.15.78:0/2829833082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77019f630 con 0x7fb770101ca0 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.472+0000 7fb76dffb640 1 -- 172.21.15.78:0/2829833082 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75802fd10 con 0x7fb770101ca0 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.473+0000 7fb776cac640 1 -- 172.21.15.78:0/2829833082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7701a5dd0 con 0x7fb770101ca0 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb779081740 1 Processor -- start 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb76dffb640 1 -- 172.21.15.78:0/2829833082 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb758004040 con 0x7fb770101ca0 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb779081740 1 -- start start 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb779081740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d484d0 0x556e52d488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb779081740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 0x556e52d54840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb779081740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d57170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb779081740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556e52cbeea0 con 0x556e52d48df0 2024-09-11T01:13:01.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb779081740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556e52cae2d0 con 0x556e52d54d80 2024-09-11T01:13:01.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb779081740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556e52cb20e0 con 0x556e52d484d0 2024-09-11T01:13:01.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb774a48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 0x556e52d54840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb774a48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 0x556e52d54840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60866/0 (socket says 172.21.15.78:60866) 2024-09-11T01:13:01.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb774a48640 1 -- 172.21.15.78:0/3257668499 learned_addr learned my addr 172.21.15.78:0/3257668499 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:01.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb775249640 1 --2- 172.21.15.78:0/3257668499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d484d0 0x556e52d488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.475+0000 7fb76ffff640 1 --2- 172.21.15.78:0/3257668499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d57170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb774a48640 1 -- 172.21.15.78:0/3257668499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d484d0 msgr2=0x556e52d488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb774a48640 1 --2- 172.21.15.78:0/3257668499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d484d0 0x556e52d488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb774a48640 1 -- 172.21.15.78:0/3257668499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 msgr2=0x556e52d57170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb774a48640 1 --2- 172.21.15.78:0/3257668499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d57170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb774a48640 1 -- 172.21.15.78:0/3257668499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e52a0c650 con 0x556e52d48df0 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb76ffff640 1 --2- 172.21.15.78:0/3257668499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d57170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb774a48640 1 --2- 172.21.15.78:0/3257668499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 0x556e52d54840 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb758104c10 tx=0x7fb75810d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c0d1ed732cf809 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb766ffd640 1 -- 172.21.15.78:0/3257668499 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75810de60 con 0x556e52d48df0 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb766ffd640 1 -- 172.21.15.78:0/3257668499 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb758114030 con 0x556e52d48df0 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb766ffd640 1 -- 172.21.15.78:0/3257668499 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758119600 con 0x556e52d48df0 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb76dffb640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb75c0b5450 0x7fb75c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb775249640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb75c0b5450 0x7fb75c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.476+0000 7fb76dffb640 1 -- 172.21.15.78:0/2829833082 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7fb758107510 con 0x7fb770101ca0 2024-09-11T01:13:01.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.477+0000 7fb779081740 1 -- 172.21.15.78:0/3257668499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 msgr2=0x556e52d54840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:01.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.477+0000 7fb775249640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb75c0b5450 0x7fb75c0b7910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb76800aac0 tx=0x7fb768008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:01.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.477+0000 7fb779081740 1 --2- 172.21.15.78:0/3257668499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 0x556e52d54840 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb758104c10 tx=0x7fb75810d620 comp rx=0 tx=0).stop 2024-09-11T01:13:01.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.477+0000 7fb779081740 1 -- 172.21.15.78:0/3257668499 shutdown_connections 2024-09-11T01:13:01.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.477+0000 7fb779081740 1 --2- 172.21.15.78:0/3257668499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d57170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.477+0000 7fb779081740 1 --2- 172.21.15.78:0/3257668499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 0x556e52d54840 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.477+0000 7fb779081740 1 --2- 172.21.15.78:0/3257668499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d484d0 0x556e52d488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.477+0000 7fb779081740 1 -- 172.21.15.78:0/3257668499 >> 172.21.15.78:0/3257668499 conn(0x556e52d46300 msgr2=0x556e52d474a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.477+0000 7fb779081740 1 -- 172.21.15.78:0/3257668499 shutdown_connections 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.477+0000 7fb779081740 1 -- 172.21.15.78:0/3257668499 wait complete. 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.477+0000 7fb779081740 1 -- start start 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.477+0000 7fb779081740 1 Processor -- start 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb779081740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 0x556e52d54610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb779081740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d48df0 0x556e52d51a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb779081740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d52000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb779081740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556e52b0f700 con 0x556e52d484d0 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb779081740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556e52c49160 con 0x556e52d54d80 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb779081740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556e52cbeea0 con 0x556e52d48df0 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb779081740 10 client.0 _ll_register_callbacks cb 0x556e52c629e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb774a48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 0x556e52d54610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb774a48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 0x556e52d54610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60880/0 (socket says 172.21.15.78:60880) 2024-09-11T01:13:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb774a48640 1 -- 172.21.15.78:0/982313665 learned_addr learned my addr 172.21.15.78:0/982313665 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb775249640 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d52000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb76ffff640 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d48df0 0x556e52d51a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb774a48640 1 -- 172.21.15.78:0/982313665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d48df0 msgr2=0x556e52d51a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb774a48640 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d48df0 0x556e52d51a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb774a48640 1 -- 172.21.15.78:0/982313665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 msgr2=0x556e52d52000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb774a48640 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d52000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb774a48640 1 -- 172.21.15.78:0/982313665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e52c76c90 con 0x556e52d484d0 2024-09-11T01:13:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb76ffff640 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d48df0 0x556e52d51a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:13:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb774a48640 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 0x556e52d54610 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb758104d50 tx=0x7fb758003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c512e24721431829 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb74ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:13:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb74ffff640 1 -- 172.21.15.78:0/982313665 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580ca020 con 0x556e52d484d0 2024-09-11T01:13:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb74ffff640 1 -- 172.21.15.78:0/982313665 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7fb7580c9c40 con 0x556e52d484d0 2024-09-11T01:13:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb779081740 1 -- 172.21.15.78:0/982313665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb758114030 con 0x556e52d484d0 2024-09-11T01:13:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb74ffff640 1 -- 172.21.15.78:0/982313665 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb7580cd140 con 0x556e52d484d0 2024-09-11T01:13:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb779081740 10 client.18150 fetch_fsmap learned FSMap version 59 2024-09-11T01:13:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.478+0000 7fb779081740 1 -- 172.21.15.78:0/982313665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x556e52d62e90 con 0x556e52d484d0 2024-09-11T01:13:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.479+0000 7fb74ffff640 1 -- 172.21.15.78:0/982313665 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb758114030 con 0x556e52d484d0 2024-09-11T01:13:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.479+0000 7fb74ffff640 1 -- 172.21.15.78:0/982313665 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7fb75810dc60 con 0x556e52d484d0 2024-09-11T01:13:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.479+0000 7fb779081740 10 client.18150 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:13:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.479+0000 7fb779081740 10 client.18150 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:13:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.479+0000 7fb779081740 20 client.18150 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.479+0000 7fb779081740 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556e52d642b0 0x556e52d66750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.479+0000 7fb779081740 4 client.18150 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:13:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.479+0000 7fb779081740 1 -- 172.21.15.78:0/982313665 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e52cc0df0 con 0x556e52d642b0 2024-09-11T01:13:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.479+0000 7fb76ffff640 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556e52d642b0 0x556e52d66750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.480+0000 7fb76ffff640 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556e52d642b0 0x556e52d66750 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40fb29c4cefae98d server_cookie=fb2d890ee8501f1e in_seq=0 out_seq=0 2024-09-11T01:13:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.480+0000 7fb74ffff640 10 client.18150 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:13:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.480+0000 7fb74ffff640 1 -- 172.21.15.78:0/982313665 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556e52cc0df0 con 0x556e52d642b0 2024-09-11T01:13:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.480+0000 7fb74ffff640 10 client.18150 handle_command_reply: tid=0 2024-09-11T01:13:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.480+0000 7fb779081740 1 client.18150 shutdown 2024-09-11T01:13:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.481+0000 7fb779081740 1 -- 172.21.15.78:0/982313665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 msgr2=0x556e52d54610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.481+0000 7fb779081740 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 0x556e52d54610 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb758104d50 tx=0x7fb758003f60 comp rx=0 tx=0).stop 2024-09-11T01:13:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.481+0000 7fb779081740 1 -- 172.21.15.78:0/982313665 shutdown_connections 2024-09-11T01:13:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.481+0000 7fb779081740 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d52000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.481+0000 7fb779081740 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d48df0 0x556e52d51a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.481+0000 7fb779081740 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556e52d642b0 0x556e52d66750 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.481+0000 7fb779081740 1 --2- 172.21.15.78:0/982313665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 0x556e52d54610 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.481+0000 7fb74ffff640 0 client.18150 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:13:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.481+0000 7fb779081740 1 -- 172.21.15.78:0/982313665 >> 172.21.15.78:0/982313665 conn(0x556e52d46300 msgr2=0x556e52d4bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.481+0000 7fb779081740 1 -- 172.21.15.78:0/982313665 shutdown_connections 2024-09-11T01:13:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.481+0000 7fb779081740 1 -- 172.21.15.78:0/982313665 wait complete. 2024-09-11T01:13:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.481+0000 7fb779081740 20 client.18150 trim_cache size 0 max 16384 2024-09-11T01:13:01.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb779081740 1 Processor -- start 2024-09-11T01:13:01.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb779081740 1 -- start start 2024-09-11T01:13:01.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb779081740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 0x556e52d47a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb779081740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d48df0 0x556e52d60830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb779081740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d56170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb779081740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7701ae700 con 0x556e52d484d0 2024-09-11T01:13:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb779081740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb7701b0480 con 0x556e52d54d80 2024-09-11T01:13:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb779081740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7701aedd0 con 0x556e52d48df0 2024-09-11T01:13:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb774a48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 0x556e52d47a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb774a48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 0x556e52d47a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60892/0 (socket says 172.21.15.78:60892) 2024-09-11T01:13:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb774a48640 1 -- 172.21.15.78:0/1252634970 learned_addr learned my addr 172.21.15.78:0/1252634970 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb76ffff640 1 --2- 172.21.15.78:0/1252634970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d48df0 0x556e52d60830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb775249640 1 --2- 172.21.15.78:0/1252634970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d56170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb774a48640 1 -- 172.21.15.78:0/1252634970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d48df0 msgr2=0x556e52d60830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb774a48640 1 --2- 172.21.15.78:0/1252634970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d48df0 0x556e52d60830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb774a48640 1 -- 172.21.15.78:0/1252634970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 msgr2=0x556e52d56170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb774a48640 1 --2- 172.21.15.78:0/1252634970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d56170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.494+0000 7fb774a48640 1 -- 172.21.15.78:0/1252634970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e52d46d80 con 0x556e52d484d0 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb774a48640 1 --2- 172.21.15.78:0/1252634970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 0x556e52d47a30 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb758009620 tx=0x7fb75810d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71b560a2af1abac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb7667fc640 1 -- 172.21.15.78:0/1252634970 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580cd320 con 0x556e52d484d0 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb7667fc640 1 -- 172.21.15.78:0/1252634970 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb758114030 con 0x556e52d484d0 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb7667fc640 1 -- 172.21.15.78:0/1252634970 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580cd7a0 con 0x556e52d484d0 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb779081740 1 -- 172.21.15.78:0/1252634970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 msgr2=0x556e52d47a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb779081740 1 --2- 172.21.15.78:0/1252634970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 0x556e52d47a30 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb758009620 tx=0x7fb75810d620 comp rx=0 tx=0).stop 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb779081740 1 -- 172.21.15.78:0/1252634970 shutdown_connections 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb779081740 1 --2- 172.21.15.78:0/1252634970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d56170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb779081740 1 --2- 172.21.15.78:0/1252634970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d48df0 0x556e52d60830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb779081740 1 --2- 172.21.15.78:0/1252634970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d484d0 0x556e52d47a30 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb779081740 1 -- 172.21.15.78:0/1252634970 >> 172.21.15.78:0/1252634970 conn(0x556e52d46300 msgr2=0x556e52d47320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:01.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb779081740 1 -- 172.21.15.78:0/1252634970 shutdown_connections 2024-09-11T01:13:01.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.495+0000 7fb779081740 1 -- 172.21.15.78:0/1252634970 wait complete. 2024-09-11T01:13:01.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb779081740 1 -- start start 2024-09-11T01:13:01.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb779081740 1 Processor -- start 2024-09-11T01:13:01.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb779081740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d484d0 0x556e52d57900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb779081740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 0x556e52d57e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb779081740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d58380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb779081740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7701afcf0 con 0x556e52d48df0 2024-09-11T01:13:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb779081740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb7701af560 con 0x556e52d54d80 2024-09-11T01:13:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb779081740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7701ac700 con 0x556e52d484d0 2024-09-11T01:13:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb779081740 10 client.0 _ll_register_callbacks cb 0x7fb76800a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb76ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 0x556e52d57e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb76ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 0x556e52d57e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60906/0 (socket says 172.21.15.78:60906) 2024-09-11T01:13:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb76ffff640 1 -- 172.21.15.78:0/3411074065 learned_addr learned my addr 172.21.15.78:0/3411074065 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb774a48640 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d484d0 0x556e52d57900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.496+0000 7fb775249640 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d58380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb76ffff640 1 -- 172.21.15.78:0/3411074065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d484d0 msgr2=0x556e52d57900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb76ffff640 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d484d0 0x556e52d57900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb76ffff640 1 -- 172.21.15.78:0/3411074065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 msgr2=0x556e52d58380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb76ffff640 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d58380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb76ffff640 1 -- 172.21.15.78:0/3411074065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e52dd1380 con 0x556e52d48df0 2024-09-11T01:13:01.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb76ffff640 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 0x556e52d57e40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb760004700 tx=0x7fb76000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ad1809ad58855a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:01.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb74ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:13:01.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb74ffff640 1 -- 172.21.15.78:0/3411074065 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760027060 con 0x556e52d48df0 2024-09-11T01:13:01.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb74ffff640 1 -- 172.21.15.78:0/3411074065 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7fb76001c790 con 0x556e52d48df0 2024-09-11T01:13:01.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb779081740 1 -- 172.21.15.78:0/3411074065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb758114030 con 0x556e52d48df0 2024-09-11T01:13:01.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb74ffff640 1 -- 172.21.15.78:0/3411074065 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb76000fe30 con 0x556e52d48df0 2024-09-11T01:13:01.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb779081740 10 client.18156 fetch_fsmap learned FSMap version 59 2024-09-11T01:13:01.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb779081740 1 -- 172.21.15.78:0/3411074065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x556e52dd2710 con 0x556e52d48df0 2024-09-11T01:13:01.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.497+0000 7fb74ffff640 1 -- 172.21.15.78:0/3411074065 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb758114030 con 0x556e52d48df0 2024-09-11T01:13:01.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.498+0000 7fb74ffff640 1 -- 172.21.15.78:0/3411074065 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7fb760021b90 con 0x556e52d48df0 2024-09-11T01:13:01.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.498+0000 7fb779081740 10 client.18156 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:13:01.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.498+0000 7fb779081740 10 client.18156 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:13:01.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.498+0000 7fb779081740 20 client.18156 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:01.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.498+0000 7fb779081740 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556e52dd2f30 0x556e52dd5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:01.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.498+0000 7fb779081740 4 client.18156 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:13:01.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.498+0000 7fb779081740 1 -- 172.21.15.78:0/3411074065 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e52cbc8a0 con 0x556e52dd2f30 2024-09-11T01:13:01.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.498+0000 7fb774a48640 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556e52dd2f30 0x556e52dd5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:01.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.498+0000 7fb74ffff640 10 client.18156 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:13:01.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.498+0000 7fb774a48640 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556e52dd2f30 0x556e52dd5350 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2cedc516c6dc server_cookie=2d250d79029b8195 in_seq=0 out_seq=0 2024-09-11T01:13:01.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb74ffff640 1 -- 172.21.15.78:0/3411074065 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e52cbc8a0 con 0x556e52dd2f30 2024-09-11T01:13:01.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb74ffff640 10 client.18156 handle_command_reply: tid=0 2024-09-11T01:13:01.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb779081740 1 client.18156 shutdown 2024-09-11T01:13:01.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb779081740 1 -- 172.21.15.78:0/3411074065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 msgr2=0x556e52d57e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:01.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb779081740 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 0x556e52d57e40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb760004700 tx=0x7fb76000f7f0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb779081740 1 -- 172.21.15.78:0/3411074065 shutdown_connections 2024-09-11T01:13:01.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb779081740 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e52d54d80 0x556e52d58380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb779081740 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x556e52dd2f30 0x556e52dd5350 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb779081740 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e52d48df0 0x556e52d57e40 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb779081740 1 --2- 172.21.15.78:0/3411074065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e52d484d0 0x556e52d57900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb779081740 1 -- 172.21.15.78:0/3411074065 >> 172.21.15.78:0/3411074065 conn(0x556e52d46300 msgr2=0x556e52d53090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:01.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb74ffff640 0 client.18156 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:13:01.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb779081740 1 -- 172.21.15.78:0/3411074065 shutdown_connections 2024-09-11T01:13:01.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb779081740 1 -- 172.21.15.78:0/3411074065 wait complete. 2024-09-11T01:13:01.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.499+0000 7fb779081740 20 client.18156 trim_cache size 0 max 16384 2024-09-11T01:13:01.520 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:13:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.500+0000 7fb776cac640 1 -- 172.21.15.78:0/2829833082 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb75c0b5450 msgr2=0x7fb75c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.500+0000 7fb776cac640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb75c0b5450 0x7fb75c0b7910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb76800aac0 tx=0x7fb768008040 comp rx=0 tx=0).stop 2024-09-11T01:13:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.500+0000 7fb776cac640 1 -- 172.21.15.78:0/2829833082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb770101ca0 msgr2=0x7fb7701992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.500+0000 7fb776cac640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb770101ca0 0x7fb7701992f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb75802ade0 tx=0x7fb758003970 comp rx=0 tx=0).stop 2024-09-11T01:13:01.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.500+0000 7fb776cac640 1 -- 172.21.15.78:0/2829833082 shutdown_connections 2024-09-11T01:13:01.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.500+0000 7fb776cac640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb75c0b5450 0x7fb75c0b7910 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.500+0000 7fb776cac640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb770108030 0x7fb77019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.500+0000 7fb776cac640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7701076b0 0x7fb770199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.500+0000 7fb776cac640 1 --2- 172.21.15.78:0/2829833082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb770101ca0 0x7fb7701992f0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:01.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.500+0000 7fb776cac640 1 -- 172.21.15.78:0/2829833082 >> 172.21.15.78:0/2829833082 conn(0x7fb7700fd730 msgr2=0x7fb77006b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:01.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.500+0000 7fb776cac640 1 -- 172.21.15.78:0/2829833082 shutdown_connections 2024-09-11T01:13:01.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:01.500+0000 7fb776cac640 1 -- 172.21.15.78:0/2829833082 wait complete. 2024-09-11T01:13:01.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 39700 34.48 MB/sec warmup 67 sec latency 1437.220 ms 2024-09-11T01:13:01.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:01 smithi110 ceph-mon[34712]: pgmap v761: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 960 op/s 2024-09-11T01:13:01.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:01 smithi112 ceph-mon[35384]: pgmap v761: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 960 op/s 2024-09-11T01:13:01.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:01 smithi078 ceph-mon[31315]: pgmap v761: 97 pgs: 97 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 960 op/s 2024-09-11T01:13:02.738 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:13:02.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 39700 33.97 MB/sec warmup 68 sec latency 2437.250 ms 2024-09-11T01:13:02.987 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:13:03.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.474+0000 7f3d069e4640 1 -- 172.21.15.78:0/994125376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d00104550 msgr2=0x7f3d0010cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.474+0000 7f3d069e4640 1 --2- 172.21.15.78:0/994125376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d00104550 0x7f3d0010cb30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009900 tx=0x7f3ce802a410 comp rx=0 tx=0).stop 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.475+0000 7f3d069e4640 1 -- 172.21.15.78:0/994125376 shutdown_connections 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.475+0000 7f3d069e4640 1 --2- 172.21.15.78:0/994125376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0010d100 0x7f3d0010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.475+0000 7f3d069e4640 1 --2- 172.21.15.78:0/994125376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d00104550 0x7f3d0010cb30 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.475+0000 7f3d069e4640 1 --2- 172.21.15.78:0/994125376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d00103b50 0x7f3d00103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.475+0000 7f3d069e4640 1 -- 172.21.15.78:0/994125376 >> 172.21.15.78:0/994125376 conn(0x7f3d000fd700 msgr2=0x7f3d000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.475+0000 7f3d069e4640 1 -- 172.21.15.78:0/994125376 shutdown_connections 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.475+0000 7f3d069e4640 1 -- 172.21.15.78:0/994125376 wait complete. 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.475+0000 7f3d069e4640 1 Processor -- start 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.475+0000 7f3d069e4640 1 -- start start 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3d069e4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d00103b50 0x7f3d00199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3d069e4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d00104550 0x7f3d00199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3d069e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d0010d100 0x7f3d0019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3d069e4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3d00112da0 con 0x7f3d00103b50 2024-09-11T01:13:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3d069e4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3d00112c20 con 0x7f3d00104550 2024-09-11T01:13:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3d069e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d00112f20 con 0x7f3d0010d100 2024-09-11T01:13:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d00103b50 0x7f3d00199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d00103b50 0x7f3d00199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51012/0 (socket says 172.21.15.78:51012) 2024-09-11T01:13:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3cfffff640 1 -- 172.21.15.78:0/3766928284 learned_addr learned my addr 172.21.15.78:0/3766928284 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3d04f81640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d0010d100 0x7f3d0019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3cff7fe640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d00104550 0x7f3d00199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3cfffff640 1 -- 172.21.15.78:0/3766928284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d0010d100 msgr2=0x7f3d0019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3cfffff640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d0010d100 0x7f3d0019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3cfffff640 1 -- 172.21.15.78:0/3766928284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d00104550 msgr2=0x7f3d00199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3cfffff640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d00104550 0x7f3d00199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.476+0000 7f3cfffff640 1 -- 172.21.15.78:0/3766928284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0019f3e0 con 0x7f3d00103b50 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.477+0000 7f3cfffff640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d00103b50 0x7f3d00199330 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000b4a0 tx=0x7f3cf000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.477+0000 7f3cfd7fa640 1 -- 172.21.15.78:0/3766928284 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0014070 con 0x7f3d00103b50 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.477+0000 7f3d069e4640 1 -- 172.21.15.78:0/3766928284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0019f6d0 con 0x7f3d00103b50 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.477+0000 7f3cfd7fa640 1 -- 172.21.15.78:0/3766928284 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf0010ac0 con 0x7f3d00103b50 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.477+0000 7f3cfd7fa640 1 -- 172.21.15.78:0/3766928284 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0017c30 con 0x7f3d00103b50 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.477+0000 7f3d069e4640 1 -- 172.21.15.78:0/3766928284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d001a5f60 con 0x7f3d00103b50 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.479+0000 7f3cfd7fa640 1 -- 172.21.15.78:0/3766928284 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3cf0010610 con 0x7f3d00103b50 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.480+0000 7f3cfd7fa640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3cec0b5450 0x7f3cec0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.480+0000 7f3cff7fe640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3cec0b5450 0x7f3cec0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.480+0000 7f3cfd7fa640 1 -- 172.21.15.78:0/3766928284 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f3cf00e5910 con 0x7f3d00103b50 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.480+0000 7f3d08db9740 1 Processor -- start 2024-09-11T01:13:03.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.480+0000 7f3cff7fe640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3cec0b5450 0x7f3cec0b7910 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8031440 tx=0x7f3ce80095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3d08db9740 1 -- start start 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3d08db9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 0x55cfb41e38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3d08db9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 0x55cfb41ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3d08db9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41efd80 0x55cfb41f2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3d08db9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cfb4159ea0 con 0x55cfb41e34d0 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3d08db9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cfb41492d0 con 0x55cfb41efd80 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3d08db9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cfb414d0e0 con 0x55cfb41e3df0 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 0x55cfb41e38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 0x55cfb41e38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51024/0 (socket says 172.21.15.78:51024) 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3cfffff640 1 -- 172.21.15.78:0/1922974994 learned_addr learned my addr 172.21.15.78:0/1922974994 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3cff7fe640 1 --2- 172.21.15.78:0/1922974994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 0x55cfb41ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3cfffff640 1 -- 172.21.15.78:0/1922974994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 msgr2=0x55cfb41ef840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3cfffff640 1 --2- 172.21.15.78:0/1922974994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 0x55cfb41ef840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3cfffff640 1 -- 172.21.15.78:0/1922974994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41efd80 msgr2=0x55cfb41f2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3cfffff640 1 --2- 172.21.15.78:0/1922974994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41efd80 0x55cfb41f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3cfffff640 1 -- 172.21.15.78:0/1922974994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb3ea7650 con 0x55cfb41e34d0 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3cfffff640 1 --2- 172.21.15.78:0/1922974994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 0x55cfb41e38b0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00a9040 tx=0x7f3cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f74b64fb13d4b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.481+0000 7f3cf67fc640 1 -- 172.21.15.78:0/1922974994 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf001e030 con 0x55cfb41e34d0 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.482+0000 7f3cf67fc640 1 -- 172.21.15.78:0/1922974994 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf00ac4d0 con 0x55cfb41e34d0 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.482+0000 7f3cf67fc640 1 -- 172.21.15.78:0/1922974994 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf00a8040 con 0x55cfb41e34d0 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.482+0000 7f3d08db9740 1 -- 172.21.15.78:0/1922974994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 msgr2=0x55cfb41e38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.482+0000 7f3d08db9740 1 --2- 172.21.15.78:0/1922974994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 0x55cfb41e38b0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00a9040 tx=0x7f3cf00046c0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 -- 172.21.15.78:0/1922974994 shutdown_connections 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 --2- 172.21.15.78:0/1922974994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41efd80 0x55cfb41f2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 --2- 172.21.15.78:0/1922974994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 0x55cfb41ef840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 --2- 172.21.15.78:0/1922974994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 0x55cfb41e38b0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 -- 172.21.15.78:0/1922974994 >> 172.21.15.78:0/1922974994 conn(0x55cfb41e1300 msgr2=0x55cfb41e24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 -- 172.21.15.78:0/1922974994 shutdown_connections 2024-09-11T01:13:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 -- 172.21.15.78:0/1922974994 wait complete. 2024-09-11T01:13:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 -- start start 2024-09-11T01:13:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 Processor -- start 2024-09-11T01:13:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41e34d0 0x55cfb41ef720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 0x55cfb41ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41efd80 0x55cfb41ed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cfb3faa700 con 0x55cfb41efd80 2024-09-11T01:13:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cfb40e4160 con 0x55cfb41e34d0 2024-09-11T01:13:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.483+0000 7f3d08db9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cfb41f2a00 con 0x55cfb41e3df0 2024-09-11T01:13:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3d08db9740 10 client.0 _ll_register_callbacks cb 0x55cfb40fd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3d04f81640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41efd80 0x55cfb41ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3d04f81640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41efd80 0x55cfb41ed220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51028/0 (socket says 172.21.15.78:51028) 2024-09-11T01:13:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3d04f81640 1 -- 172.21.15.78:0/2868796730 learned_addr learned my addr 172.21.15.78:0/2868796730 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3cff7fe640 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 0x55cfb41ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3cfffff640 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41e34d0 0x55cfb41ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3d04f81640 1 -- 172.21.15.78:0/2868796730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 msgr2=0x55cfb41ecb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3d04f81640 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 0x55cfb41ecb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3d04f81640 1 -- 172.21.15.78:0/2868796730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41e34d0 msgr2=0x55cfb41ef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3d04f81640 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41e34d0 0x55cfb41ef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3d04f81640 1 -- 172.21.15.78:0/2868796730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb4111c90 con 0x55cfb41efd80 2024-09-11T01:13:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3d04f81640 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41efd80 0x55cfb41ed220 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800e770 tx=0x7f3cf800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611e8dc9356482f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3cdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3cdf7fe640 1 -- 172.21.15.78:0/2868796730 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8004d20 con 0x55cfb41efd80 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3cdf7fe640 1 -- 172.21.15.78:0/2868796730 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f3cf800cac0 con 0x55cfb41efd80 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3d08db9740 1 -- 172.21.15.78:0/2868796730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cf00ac4d0 con 0x55cfb41efd80 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.484+0000 7f3cdf7fe640 1 -- 172.21.15.78:0/2868796730 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f3cf800ee80 con 0x55cfb41efd80 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.485+0000 7f3d08db9740 10 client.18168 fetch_fsmap learned FSMap version 59 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.485+0000 7f3d08db9740 1 -- 172.21.15.78:0/2868796730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cfb41fe440 con 0x55cfb41efd80 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.485+0000 7f3cdf7fe640 1 -- 172.21.15.78:0/2868796730 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf00ac4d0 con 0x55cfb41efd80 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.485+0000 7f3cdf7fe640 1 -- 172.21.15.78:0/2868796730 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f3cf8010430 con 0x55cfb41efd80 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.485+0000 7f3d08db9740 10 client.18168 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.485+0000 7f3d08db9740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.485+0000 7f3d08db9740 20 client.18168 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.485+0000 7f3d08db9740 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cfb41ff960 0x55cfb4201d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.485+0000 7f3d08db9740 4 client.18168 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.485+0000 7f3d08db9740 1 -- 172.21.15.78:0/2868796730 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfb415bdf0 con 0x55cfb41ff960 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.485+0000 7f3cfffff640 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cfb41ff960 0x55cfb4201d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.486+0000 7f3cdf7fe640 10 client.18168 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.486+0000 7f3cfffff640 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cfb41ff960 0x55cfb4201d70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68c259ffbc688b0 server_cookie=767240deb43df4b in_seq=0 out_seq=0 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.486+0000 7f3cdf7fe640 1 -- 172.21.15.78:0/2868796730 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cfb415bdf0 con 0x55cfb41ff960 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.486+0000 7f3cdf7fe640 10 client.18168 handle_command_reply: tid=0 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.486+0000 7f3d08db9740 1 client.18168 shutdown 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.486+0000 7f3d08db9740 1 -- 172.21.15.78:0/2868796730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41efd80 msgr2=0x55cfb41ed220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.486+0000 7f3d08db9740 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41efd80 0x55cfb41ed220 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800e770 tx=0x7f3cf800ec30 comp rx=0 tx=0).stop 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.487+0000 7f3d08db9740 1 -- 172.21.15.78:0/2868796730 shutdown_connections 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.487+0000 7f3d08db9740 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cfb41ff960 0x55cfb4201d70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.487+0000 7f3d08db9740 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41efd80 0x55cfb41ed220 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.487+0000 7f3cdf7fe640 0 client.18168 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.487+0000 7f3d08db9740 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 0x55cfb41ecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.487+0000 7f3d08db9740 1 --2- 172.21.15.78:0/2868796730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41e34d0 0x55cfb41ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.487+0000 7f3d08db9740 1 -- 172.21.15.78:0/2868796730 >> 172.21.15.78:0/2868796730 conn(0x55cfb41e1300 msgr2=0x55cfb41e6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.487+0000 7f3d08db9740 1 -- 172.21.15.78:0/2868796730 shutdown_connections 2024-09-11T01:13:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.487+0000 7f3d08db9740 1 -- 172.21.15.78:0/2868796730 wait complete. 2024-09-11T01:13:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.487+0000 7f3d08db9740 20 client.18168 trim_cache size 0 max 16384 2024-09-11T01:13:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.501+0000 7f3d08db9740 1 Processor -- start 2024-09-11T01:13:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.501+0000 7f3d08db9740 1 -- start start 2024-09-11T01:13:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.501+0000 7f3d08db9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 0x55cfb41ed1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.501+0000 7f3d08db9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 0x55cfb41fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.501+0000 7f3d08db9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41efd80 0x55cfb41e0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.501+0000 7f3d08db9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3d001ae7b0 con 0x55cfb41e34d0 2024-09-11T01:13:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.501+0000 7f3d08db9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3d001b0530 con 0x55cfb41efd80 2024-09-11T01:13:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.501+0000 7f3d08db9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d001aee80 con 0x55cfb41e3df0 2024-09-11T01:13:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.501+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 0x55cfb41ed1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 0x55cfb41ed1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51040/0 (socket says 172.21.15.78:51040) 2024-09-11T01:13:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3cfffff640 1 -- 172.21.15.78:0/2040272274 learned_addr learned my addr 172.21.15.78:0/2040272274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3cff7fe640 1 --2- 172.21.15.78:0/2040272274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 0x55cfb41fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3d04f81640 1 --2- 172.21.15.78:0/2040272274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41efd80 0x55cfb41e0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3cfffff640 1 -- 172.21.15.78:0/2040272274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 msgr2=0x55cfb41fbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3cfffff640 1 --2- 172.21.15.78:0/2040272274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 0x55cfb41fbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3cfffff640 1 -- 172.21.15.78:0/2040272274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41efd80 msgr2=0x55cfb41e0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3cfffff640 1 --2- 172.21.15.78:0/2040272274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41efd80 0x55cfb41e0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3cfffff640 1 -- 172.21.15.78:0/2040272274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb41e3c20 con 0x55cfb41e34d0 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3cfffff640 1 --2- 172.21.15.78:0/2040272274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 0x55cfb41ed1b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00e36a0 tx=0x7f3cf00b08b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5b7ef0826876e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3cf5ffb640 1 -- 172.21.15.78:0/2040272274 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf001e040 con 0x55cfb41e34d0 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3cf5ffb640 1 -- 172.21.15.78:0/2040272274 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf00ac4d0 con 0x55cfb41e34d0 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.502+0000 7f3cf5ffb640 1 -- 172.21.15.78:0/2040272274 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf00a8040 con 0x55cfb41e34d0 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.503+0000 7f3d08db9740 1 -- 172.21.15.78:0/2040272274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 msgr2=0x55cfb41ed1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.503+0000 7f3d08db9740 1 --2- 172.21.15.78:0/2040272274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 0x55cfb41ed1b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00e36a0 tx=0x7f3cf00b08b0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.503+0000 7f3d08db9740 1 -- 172.21.15.78:0/2040272274 shutdown_connections 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.503+0000 7f3d08db9740 1 --2- 172.21.15.78:0/2040272274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41efd80 0x55cfb41e0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.503+0000 7f3d08db9740 1 --2- 172.21.15.78:0/2040272274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41e3df0 0x55cfb41fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.503+0000 7f3d08db9740 1 --2- 172.21.15.78:0/2040272274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e34d0 0x55cfb41ed1b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.503+0000 7f3d08db9740 1 -- 172.21.15.78:0/2040272274 >> 172.21.15.78:0/2040272274 conn(0x55cfb41e1300 msgr2=0x55cfb41dfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:03.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.503+0000 7f3d08db9740 1 -- 172.21.15.78:0/2040272274 shutdown_connections 2024-09-11T01:13:03.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.503+0000 7f3d08db9740 1 -- 172.21.15.78:0/2040272274 wait complete. 2024-09-11T01:13:03.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.503+0000 7f3d08db9740 1 -- start start 2024-09-11T01:13:03.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3d08db9740 1 Processor -- start 2024-09-11T01:13:03.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3d08db9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41e34d0 0x55cfb41f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3d08db9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e3df0 0x55cfb41f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3d08db9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41efd80 0x55cfb41f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3d08db9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3d001afda0 con 0x55cfb41e3df0 2024-09-11T01:13:03.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3d08db9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3d001af610 con 0x55cfb41e34d0 2024-09-11T01:13:03.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3d08db9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d001ac7b0 con 0x55cfb41efd80 2024-09-11T01:13:03.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3d08db9740 10 client.0 _ll_register_callbacks cb 0x7f3cf80084a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3cff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e3df0 0x55cfb41f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3cff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e3df0 0x55cfb41f81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51048/0 (socket says 172.21.15.78:51048) 2024-09-11T01:13:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3cff7fe640 1 -- 172.21.15.78:0/918545723 learned_addr learned my addr 172.21.15.78:0/918545723 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41e34d0 0x55cfb41f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3d04f81640 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41efd80 0x55cfb41f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3cff7fe640 1 -- 172.21.15.78:0/918545723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41efd80 msgr2=0x55cfb41f2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3cff7fe640 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41efd80 0x55cfb41f2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3cff7fe640 1 -- 172.21.15.78:0/918545723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41e34d0 msgr2=0x55cfb41f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3cff7fe640 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41e34d0 0x55cfb41f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3cff7fe640 1 -- 172.21.15.78:0/918545723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb41f31e0 con 0x55cfb41e3df0 2024-09-11T01:13:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.504+0000 7f3d04f81640 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41efd80 0x55cfb41f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:13:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3cfffff640 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41e34d0 0x55cfb41f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3cff7fe640 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e3df0 0x55cfb41f81e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009900 tx=0x7f3ce8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bae2dbd81f3c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3cdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:13:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3cdf7fe640 1 -- 172.21.15.78:0/918545723 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8035050 con 0x55cfb41e3df0 2024-09-11T01:13:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3cdf7fe640 1 -- 172.21.15.78:0/918545723 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f3ce8003800 con 0x55cfb41e3df0 2024-09-11T01:13:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3d08db9740 1 -- 172.21.15.78:0/918545723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cf00ac4d0 con 0x55cfb41e3df0 2024-09-11T01:13:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3cdf7fe640 1 -- 172.21.15.78:0/918545723 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f3ce8003a00 con 0x55cfb41e3df0 2024-09-11T01:13:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3d08db9740 10 client.18174 fetch_fsmap learned FSMap version 59 2024-09-11T01:13:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3d08db9740 1 -- 172.21.15.78:0/918545723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cfb426d0f0 con 0x55cfb41e3df0 2024-09-11T01:13:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3cdf7fe640 1 -- 172.21.15.78:0/918545723 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf00ac4d0 con 0x55cfb41e3df0 2024-09-11T01:13:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3cdf7fe640 1 -- 172.21.15.78:0/918545723 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f3ce805c440 con 0x55cfb41e3df0 2024-09-11T01:13:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3d08db9740 10 client.18174 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:13:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3d08db9740 10 client.18174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:13:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.505+0000 7f3d08db9740 20 client.18174 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.506+0000 7f3d08db9740 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cfb41ff960 0x55cfb426f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.506+0000 7f3d08db9740 4 client.18174 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:13:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.506+0000 7f3d08db9740 1 -- 172.21.15.78:0/918545723 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfb41578a0 con 0x55cfb41ff960 2024-09-11T01:13:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.506+0000 7f3cfffff640 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cfb41ff960 0x55cfb426f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.506+0000 7f3cdf7fe640 10 client.18174 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:03.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.506+0000 7f3cfffff640 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cfb41ff960 0x55cfb426f980 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=836c27c6d5a0653a server_cookie=8e9cc8c57411776b in_seq=0 out_seq=0 2024-09-11T01:13:03.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.506+0000 7f3cdf7fe640 1 -- 172.21.15.78:0/918545723 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfb41578a0 con 0x55cfb41ff960 2024-09-11T01:13:03.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.506+0000 7f3cdf7fe640 10 client.18174 handle_command_reply: tid=0 2024-09-11T01:13:03.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.506+0000 7f3d08db9740 1 client.18174 shutdown 2024-09-11T01:13:03.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.507+0000 7f3d08db9740 1 -- 172.21.15.78:0/918545723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e3df0 msgr2=0x55cfb41f81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:03.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.507+0000 7f3d08db9740 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e3df0 0x55cfb41f81e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009900 tx=0x7f3ce8009260 comp rx=0 tx=0).stop 2024-09-11T01:13:03.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.507+0000 7f3d08db9740 1 -- 172.21.15.78:0/918545723 shutdown_connections 2024-09-11T01:13:03.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.507+0000 7f3d08db9740 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cfb41efd80 0x55cfb41f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.507+0000 7f3d08db9740 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55cfb41ff960 0x55cfb426f980 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.507+0000 7f3d08db9740 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfb41e3df0 0x55cfb41f81e0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.507+0000 7f3d08db9740 1 --2- 172.21.15.78:0/918545723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfb41e34d0 0x55cfb41f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.507+0000 7f3d08db9740 1 -- 172.21.15.78:0/918545723 >> 172.21.15.78:0/918545723 conn(0x55cfb41e1300 msgr2=0x55cfb41ee9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:03.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.507+0000 7f3cdf7fe640 0 client.18174 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:03.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.507+0000 7f3d08db9740 1 -- 172.21.15.78:0/918545723 shutdown_connections 2024-09-11T01:13:03.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.507+0000 7f3d08db9740 1 -- 172.21.15.78:0/918545723 wait complete. 2024-09-11T01:13:03.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.507+0000 7f3d08db9740 20 client.18174 trim_cache size 0 max 16384 2024-09-11T01:13:03.527 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:13:03.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.508+0000 7f3d069e4640 1 -- 172.21.15.78:0/3766928284 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3cec0b5450 msgr2=0x7f3cec0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:03.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.508+0000 7f3d069e4640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3cec0b5450 0x7f3cec0b7910 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8031440 tx=0x7f3ce80095f0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.508+0000 7f3d069e4640 1 -- 172.21.15.78:0/3766928284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d00103b50 msgr2=0x7f3d00199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:03.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.508+0000 7f3d069e4640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d00103b50 0x7f3d00199330 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000b4a0 tx=0x7f3cf000b960 comp rx=0 tx=0).stop 2024-09-11T01:13:03.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.508+0000 7f3d069e4640 1 -- 172.21.15.78:0/3766928284 shutdown_connections 2024-09-11T01:13:03.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.508+0000 7f3d069e4640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3cec0b5450 0x7f3cec0b7910 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.508+0000 7f3d069e4640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d0010d100 0x7f3d0019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.508+0000 7f3d069e4640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d00104550 0x7f3d00199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.508+0000 7f3d069e4640 1 --2- 172.21.15.78:0/3766928284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d00103b50 0x7f3d00199330 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:03.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.508+0000 7f3d069e4640 1 -- 172.21.15.78:0/3766928284 >> 172.21.15.78:0/3766928284 conn(0x7f3d000fd700 msgr2=0x7f3d000fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:03.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.508+0000 7f3d069e4640 1 -- 172.21.15.78:0/3766928284 shutdown_connections 2024-09-11T01:13:03.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:03.508+0000 7f3d069e4640 1 -- 172.21.15.78:0/3766928284 wait complete. 2024-09-11T01:13:03.680 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:03 smithi078 ceph-mon[31315]: pgmap v762: 97 pgs: 97 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 716 op/s 2024-09-11T01:13:03.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 39700 33.48 MB/sec warmup 69 sec latency 3437.302 ms 2024-09-11T01:13:03.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:03 smithi110 ceph-mon[34712]: pgmap v762: 97 pgs: 97 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 716 op/s 2024-09-11T01:13:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:03 smithi112 ceph-mon[35384]: pgmap v762: 97 pgs: 97 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 716 op/s 2024-09-11T01:13:04.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 39945 33.20 MB/sec warmup 70 sec latency 4191.355 ms 2024-09-11T01:13:05.370 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:13:05.370 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:13:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:13:05.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:05 smithi078 ceph-mon[31315]: pgmap v763: 97 pgs: 97 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.4 MiB/s wr, 558 op/s 2024-09-11T01:13:05.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 40448 33.12 MB/sec warmup 71 sec latency 89.774 ms 2024-09-11T01:13:05.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:05 smithi110 ceph-mon[34712]: pgmap v763: 97 pgs: 97 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.4 MiB/s wr, 558 op/s 2024-09-11T01:13:05.976 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:05 smithi112 ceph-mon[35384]: pgmap v763: 97 pgs: 97 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.4 MiB/s wr, 558 op/s 2024-09-11T01:13:06.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.297+0000 7f21b58ee640 1 -- 172.21.15.78:0/3789071410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0108040 msgr2=0x7f21b010e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:06.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.297+0000 7f21b58ee640 1 --2- 172.21.15.78:0/3789071410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0108040 0x7f21b010e840 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009ca0 tx=0x7f21a802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.297+0000 7f21b58ee640 1 -- 172.21.15.78:0/3789071410 shutdown_connections 2024-09-11T01:13:06.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.297+0000 7f21b58ee640 1 --2- 172.21.15.78:0/3789071410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0108040 0x7f21b010e840 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.297+0000 7f21b58ee640 1 --2- 172.21.15.78:0/3789071410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21b01076c0 0x7f21b0107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.297+0000 7f21b58ee640 1 --2- 172.21.15.78:0/3789071410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21b0101cb0 0x7f21b0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.297+0000 7f21b58ee640 1 -- 172.21.15.78:0/3789071410 >> 172.21.15.78:0/3789071410 conn(0x7f21b00fd700 msgr2=0x7f21b00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.297+0000 7f21b58ee640 1 -- 172.21.15.78:0/3789071410 shutdown_connections 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.297+0000 7f21b58ee640 1 -- 172.21.15.78:0/3789071410 wait complete. 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.297+0000 7f21b58ee640 1 Processor -- start 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.297+0000 7f21b58ee640 1 -- start start 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21b58ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0101cb0 0x7f21b0197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21b58ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21b01076c0 0x7f21b01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21b58ee640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21b0108040 0x7f21b0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21b58ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21b0111cc0 con 0x7f21b0101cb0 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21b58ee640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f21b0111b40 con 0x7f21b0108040 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21b58ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21b0111e40 con 0x7f21b01076c0 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21af7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21b0108040 0x7f21b0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21af7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21b0108040 0x7f21b0198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:37064/0 (socket says 172.21.15.78:37064) 2024-09-11T01:13:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21af7fe640 1 -- 172.21.15.78:0/2135810655 learned_addr learned my addr 172.21.15.78:0/2135810655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21a7fff640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21b01076c0 0x7f21b01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21aeffd640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0101cb0 0x7f21b0197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21af7fe640 1 -- 172.21.15.78:0/2135810655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21b01076c0 msgr2=0x7f21b01982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21af7fe640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21b01076c0 0x7f21b01982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21af7fe640 1 -- 172.21.15.78:0/2135810655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0101cb0 msgr2=0x7f21b0197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21af7fe640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0101cb0 0x7f21b0197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.298+0000 7f21af7fe640 1 -- 172.21.15.78:0/2135810655 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b01a75d0 con 0x7f21b0108040 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.299+0000 7f21af7fe640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21b0108040 0x7f21b0198810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f21a8005cc0 tx=0x7f21a8031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.299+0000 7f21b48ec640 1 -- 172.21.15.78:0/2135810655 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8003b70 con 0x7f21b0108040 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.299+0000 7f21b58ee640 1 -- 172.21.15.78:0/2135810655 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b01a7860 con 0x7f21b0108040 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.299+0000 7f21b58ee640 1 -- 172.21.15.78:0/2135810655 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b01a7df0 con 0x7f21b0108040 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.299+0000 7f21b48ec640 1 -- 172.21.15.78:0/2135810655 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a80318f0 con 0x7f21b0108040 2024-09-11T01:13:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.299+0000 7f21b48ec640 1 -- 172.21.15.78:0/2135810655 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a802f950 con 0x7f21b0108040 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.302+0000 7f21b48ec640 1 -- 172.21.15.78:0/2135810655 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f21a802fc10 con 0x7f21b0108040 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.303+0000 7f21b7cc3740 1 Processor -- start 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.303+0000 7f21b7cc3740 1 -- start start 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.303+0000 7f21b48ec640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f219c0b54d0 0x7f219c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21a7fff640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f219c0b54d0 0x7f219c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21b48ec640 1 -- 172.21.15.78:0/2135810655 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f21a81070e0 con 0x7f21b0108040 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21b7cc3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb56f0 0x562e09cb5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21b7cc3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb6010 0x562e09cc1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21b7cc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cc1fa0 0x562e09cc4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21b7cc3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562e09c23e60 con 0x562e09cb6010 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21b7cc3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562e09c13290 con 0x562e09cb56f0 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21b7cc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562e09c170a0 con 0x562e09cc1fa0 2024-09-11T01:13:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21aeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb56f0 0x562e09cb5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21aeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb56f0 0x562e09cb5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:37080/0 (socket says 172.21.15.78:37080) 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21aeffd640 1 -- 172.21.15.78:0/1182656483 learned_addr learned my addr 172.21.15.78:0/1182656483 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21af7fe640 1 --2- 172.21.15.78:0/1182656483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cc1fa0 0x562e09cc4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21a7fff640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f219c0b54d0 0x7f219c0b7990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f21b0100090 tx=0x7f2198009830 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21aeffd640 1 -- 172.21.15.78:0/1182656483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cc1fa0 msgr2=0x562e09cc4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.304+0000 7f21aeffd640 1 --2- 172.21.15.78:0/1182656483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cc1fa0 0x562e09cc4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.305+0000 7f21aeffd640 1 -- 172.21.15.78:0/1182656483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb6010 msgr2=0x562e09cc1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.305+0000 7f21aeffd640 1 --2- 172.21.15.78:0/1182656483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb6010 0x562e09cc1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.305+0000 7f21aeffd640 1 -- 172.21.15.78:0/1182656483 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e09971930 con 0x562e09cb56f0 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.305+0000 7f21af7fe640 1 --2- 172.21.15.78:0/1182656483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cc1fa0 0x562e09cc4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.305+0000 7f21aeffd640 1 --2- 172.21.15.78:0/1182656483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb56f0 0x562e09cb5ad0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009cf0 tx=0x7f21a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee51414877fa2f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.305+0000 7f21a5ffb640 1 -- 172.21.15.78:0/1182656483 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0004230 con 0x562e09cb56f0 2024-09-11T01:13:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.305+0000 7f21a5ffb640 1 -- 172.21.15.78:0/1182656483 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a00043d0 con 0x562e09cb56f0 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.306+0000 7f21a5ffb640 1 -- 172.21.15.78:0/1182656483 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0019780 con 0x562e09cb56f0 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.306+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1182656483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb56f0 msgr2=0x562e09cb5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.306+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1182656483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb56f0 0x562e09cb5ad0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009cf0 tx=0x7f21a0011810 comp rx=0 tx=0).stop 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.306+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1182656483 shutdown_connections 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.306+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1182656483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cc1fa0 0x562e09cc4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.306+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1182656483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb6010 0x562e09cc1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.306+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1182656483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb56f0 0x562e09cb5ad0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.306+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1182656483 >> 172.21.15.78:0/1182656483 conn(0x562e09cb3520 msgr2=0x562e09cb46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.307+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1182656483 shutdown_connections 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.307+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1182656483 wait complete. 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.307+0000 7f21b7cc3740 1 -- start start 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.307+0000 7f21b7cc3740 1 Processor -- start 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21b7cc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cb56f0 0x562e09cc19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21b7cc3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb6010 0x562e09cbee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21b7cc3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cc1fa0 0x562e09cbf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21b7cc3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562e09a74890 con 0x562e09cb6010 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21b7cc3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562e09badd40 con 0x562e09cc1fa0 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21b7cc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562e09cc4c70 con 0x562e09cb56f0 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21b7cc3740 10 client.0 _ll_register_callbacks cb 0x562e09c11c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb6010 0x562e09cbee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb6010 0x562e09cbee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51076/0 (socket says 172.21.15.78:51076) 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21a7fff640 1 -- 172.21.15.78:0/1129858610 learned_addr learned my addr 172.21.15.78:0/1129858610 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21aeffd640 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cb56f0 0x562e09cc19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21af7fe640 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cc1fa0 0x562e09cbf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21a7fff640 1 -- 172.21.15.78:0/1129858610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cb56f0 msgr2=0x562e09cc19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.308+0000 7f21a7fff640 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cb56f0 0x562e09cc19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.309+0000 7f21a7fff640 1 -- 172.21.15.78:0/1129858610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cc1fa0 msgr2=0x562e09cbf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.309+0000 7f21a7fff640 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cc1fa0 0x562e09cbf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.309+0000 7f21a7fff640 1 -- 172.21.15.78:0/1129858610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e09cce020 con 0x562e09cb6010 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.309+0000 7f21aeffd640 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cb56f0 0x562e09cc19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.309+0000 7f21a7fff640 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb6010 0x562e09cbee20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2198009fd0 tx=0x7f2198015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77230fd96140a3d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.309+0000 7f218effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.309+0000 7f218effd640 1 -- 172.21.15.78:0/1129858610 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219801e820 con 0x562e09cb6010 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.309+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1129858610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21a00043d0 con 0x562e09cb6010 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.310+0000 7f218effd640 1 -- 172.21.15.78:0/1129858610 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f219801e9c0 con 0x562e09cb6010 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.310+0000 7f218effd640 1 -- 172.21.15.78:0/1129858610 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f219801ebc0 con 0x562e09cb6010 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.310+0000 7f21b7cc3740 10 client.18183 fetch_fsmap learned FSMap version 59 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.310+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1129858610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x562e09cd09e0 con 0x562e09cb6010 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.311+0000 7f218effd640 1 -- 172.21.15.78:0/1129858610 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a00043d0 con 0x562e09cb6010 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.311+0000 7f218effd640 1 -- 172.21.15.78:0/1129858610 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f21980313f0 con 0x562e09cb6010 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.311+0000 7f21b7cc3740 10 client.18183 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:13:06.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.311+0000 7f21b7cc3740 10 client.18183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.311+0000 7f21b7cc3740 20 client.18183 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.311+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562e09cd1ea0 0x562e09cd4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.311+0000 7f21b7cc3740 4 client.18183 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.311+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1129858610 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e09c25db0 con 0x562e09cd1ea0 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.311+0000 7f21aeffd640 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562e09cd1ea0 0x562e09cd4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.312+0000 7f218effd640 10 client.18183 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.312+0000 7f21aeffd640 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562e09cd1ea0 0x562e09cd4330 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e0308bcba76da0 server_cookie=d3e79b34c5657a0 in_seq=0 out_seq=0 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.313+0000 7f218effd640 1 -- 172.21.15.78:0/1129858610 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562e09c25db0 con 0x562e09cd1ea0 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.313+0000 7f218effd640 10 client.18183 handle_command_reply: tid=0 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f21b7cc3740 1 client.18183 shutdown 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1129858610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb6010 msgr2=0x562e09cbee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb6010 0x562e09cbee20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2198009fd0 tx=0x7f2198015040 comp rx=0 tx=0).stop 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1129858610 shutdown_connections 2024-09-11T01:13:06.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cc1fa0 0x562e09cbf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562e09cd1ea0 0x562e09cd4330 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f218effd640 0 client.18183 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:06.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb6010 0x562e09cbee20 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1129858610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cb56f0 0x562e09cc19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1129858610 >> 172.21.15.78:0/1129858610 conn(0x562e09cb3520 msgr2=0x562e09cb88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:06.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1129858610 shutdown_connections 2024-09-11T01:13:06.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1129858610 wait complete. 2024-09-11T01:13:06.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.314+0000 7f21b7cc3740 20 client.18183 trim_cache size 0 max 16384 2024-09-11T01:13:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21b7cc3740 1 Processor -- start 2024-09-11T01:13:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21b7cc3740 1 -- start start 2024-09-11T01:13:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21b7cc3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb56f0 0x562e09cbc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21b7cc3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb6010 0x562e09ca9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21b7cc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cc1fa0 0x562e09ca9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21b7cc3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01ac3d0 con 0x562e09cb56f0 2024-09-11T01:13:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21b7cc3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01ae150 con 0x562e09cb6010 2024-09-11T01:13:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21b7cc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01acaa0 con 0x562e09cc1fa0 2024-09-11T01:13:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb56f0 0x562e09cbc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb56f0 0x562e09cbc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51084/0 (socket says 172.21.15.78:51084) 2024-09-11T01:13:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21aeffd640 1 -- 172.21.15.78:0/3125907055 learned_addr learned my addr 172.21.15.78:0/3125907055 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21a7fff640 1 --2- 172.21.15.78:0/3125907055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb6010 0x562e09ca9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.345+0000 7f21af7fe640 1 --2- 172.21.15.78:0/3125907055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cc1fa0 0x562e09ca9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.346+0000 7f21aeffd640 1 -- 172.21.15.78:0/3125907055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cc1fa0 msgr2=0x562e09ca9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:06.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.346+0000 7f21aeffd640 1 --2- 172.21.15.78:0/3125907055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cc1fa0 0x562e09ca9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.346+0000 7f21aeffd640 1 -- 172.21.15.78:0/3125907055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb6010 msgr2=0x562e09ca9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:06.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.346+0000 7f21aeffd640 1 --2- 172.21.15.78:0/3125907055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb6010 0x562e09ca9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.346+0000 7f21aeffd640 1 -- 172.21.15.78:0/3125907055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e09cb3fa0 con 0x562e09cb56f0 2024-09-11T01:13:06.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.346+0000 7f21aeffd640 1 --2- 172.21.15.78:0/3125907055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb56f0 0x562e09cbc990 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009930 tx=0x7f21a0019420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a5a2eef041bbe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.346+0000 7f21a57fa640 1 -- 172.21.15.78:0/3125907055 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0005250 con 0x562e09cb56f0 2024-09-11T01:13:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.346+0000 7f21a57fa640 1 -- 172.21.15.78:0/3125907055 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a00053f0 con 0x562e09cb56f0 2024-09-11T01:13:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.346+0000 7f21a57fa640 1 -- 172.21.15.78:0/3125907055 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a00196f0 con 0x562e09cb56f0 2024-09-11T01:13:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.347+0000 7f21b7cc3740 1 -- 172.21.15.78:0/3125907055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb56f0 msgr2=0x562e09cbc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.347+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/3125907055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb56f0 0x562e09cbc990 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009930 tx=0x7f21a0019420 comp rx=0 tx=0).stop 2024-09-11T01:13:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.347+0000 7f21b7cc3740 1 -- 172.21.15.78:0/3125907055 shutdown_connections 2024-09-11T01:13:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.347+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/3125907055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cc1fa0 0x562e09ca9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.347+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/3125907055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb6010 0x562e09ca9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.347+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/3125907055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cb56f0 0x562e09cbc990 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.347+0000 7f21b7cc3740 1 -- 172.21.15.78:0/3125907055 >> 172.21.15.78:0/3125907055 conn(0x562e09cb3520 msgr2=0x562e09cbc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.348+0000 7f21b7cc3740 1 -- 172.21.15.78:0/3125907055 shutdown_connections 2024-09-11T01:13:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.348+0000 7f21b7cc3740 1 -- 172.21.15.78:0/3125907055 wait complete. 2024-09-11T01:13:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.348+0000 7f21b7cc3740 1 -- start start 2024-09-11T01:13:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.349+0000 7f21b7cc3740 1 Processor -- start 2024-09-11T01:13:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.349+0000 7f21b7cc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cb56f0 0x562e09cca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.349+0000 7f21b7cc3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb6010 0x562e09cc4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.349+0000 7f21b7cc3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cc1fa0 0x562e09cc5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21b7cc3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01ad9c0 con 0x562e09cc1fa0 2024-09-11T01:13:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21b7cc3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01ad230 con 0x562e09cb6010 2024-09-11T01:13:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21b7cc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01aa3f0 con 0x562e09cb56f0 2024-09-11T01:13:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21b7cc3740 10 client.0 _ll_register_callbacks cb 0x7f2198009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb6010 0x562e09cc4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21aeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cb56f0 0x562e09cca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb6010 0x562e09cc4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:37110/0 (socket says 172.21.15.78:37110) 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21a7fff640 1 -- 172.21.15.78:0/1939083871 learned_addr learned my addr 172.21.15.78:0/1939083871 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21af7fe640 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cc1fa0 0x562e09cc5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21a7fff640 1 -- 172.21.15.78:0/1939083871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cb56f0 msgr2=0x562e09cca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21a7fff640 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cb56f0 0x562e09cca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21a7fff640 1 -- 172.21.15.78:0/1939083871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cc1fa0 msgr2=0x562e09cc5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21a7fff640 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cc1fa0 0x562e09cc5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.350+0000 7f21a7fff640 1 -- 172.21.15.78:0/1939083871 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e09d3ee00 con 0x562e09cb6010 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.351+0000 7f21aeffd640 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cb56f0 0x562e09cca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.351+0000 7f21a7fff640 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb6010 0x562e09cc4f80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f21980319b0 tx=0x7f21980056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64d871c6e8cb2690 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.351+0000 7f218effd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:13:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.351+0000 7f218effd640 1 -- 172.21.15.78:0/1939083871 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198041060 con 0x562e09cb6010 2024-09-11T01:13:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.352+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1939083871 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21a00053f0 con 0x562e09cb6010 2024-09-11T01:13:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.352+0000 7f218effd640 1 -- 172.21.15.78:0/1939083871 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8449+0+0 (secure 0 0 0) 0x7f219802d030 con 0x562e09cb6010 2024-09-11T01:13:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.352+0000 7f218effd640 1 -- 172.21.15.78:0/1939083871 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2198036070 con 0x562e09cb6010 2024-09-11T01:13:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.352+0000 7f21b7cc3740 10 client.27344 fetch_fsmap learned FSMap version 59 2024-09-11T01:13:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.352+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1939083871 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x562e09d401d0 con 0x562e09cb6010 2024-09-11T01:13:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.352+0000 7f218effd640 1 -- 172.21.15.78:0/1939083871 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a00053f0 con 0x562e09cb6010 2024-09-11T01:13:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.353+0000 7f218effd640 1 -- 172.21.15.78:0/1939083871 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f219803c4f0 con 0x562e09cb6010 2024-09-11T01:13:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.353+0000 7f21b7cc3740 10 client.27344 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:13:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.353+0000 7f21b7cc3740 10 client.27344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:13:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.353+0000 7f21b7cc3740 20 client.27344 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.353+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562e09d40a40 0x562e09d42e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.353+0000 7f21b7cc3740 4 client.27344 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:13:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.353+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1939083871 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562e09c21860 con 0x562e09d40a40 2024-09-11T01:13:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.353+0000 7f21aeffd640 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562e09d40a40 0x562e09d42e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.353+0000 7f21aeffd640 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562e09d40a40 0x562e09d42e60 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d65b53ceb26f737 server_cookie=bd67f609f2f70c5d in_seq=0 out_seq=0 2024-09-11T01:13:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.353+0000 7f218effd640 10 client.27344 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:06.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.354+0000 7f218effd640 1 -- 172.21.15.78:0/1939083871 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562e09c21860 con 0x562e09d40a40 2024-09-11T01:13:06.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.354+0000 7f218effd640 10 client.27344 handle_command_reply: tid=0 2024-09-11T01:13:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.354+0000 7f21b7cc3740 1 client.27344 shutdown 2024-09-11T01:13:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.355+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1939083871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb6010 msgr2=0x562e09cc4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.355+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb6010 0x562e09cc4f80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f21980319b0 tx=0x7f21980056c0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.355+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1939083871 shutdown_connections 2024-09-11T01:13:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.355+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x562e09d40a40 0x562e09d42e60 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.355+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e09cc1fa0 0x562e09cc5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.355+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e09cb6010 0x562e09cc4f80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.355+0000 7f21b7cc3740 1 --2- 172.21.15.78:0/1939083871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e09cb56f0 0x562e09cca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.355+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1939083871 >> 172.21.15.78:0/1939083871 conn(0x562e09cb3520 msgr2=0x562e09ca9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.355+0000 7f218effd640 0 client.27344 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.355+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1939083871 shutdown_connections 2024-09-11T01:13:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.355+0000 7f21b7cc3740 1 -- 172.21.15.78:0/1939083871 wait complete. 2024-09-11T01:13:06.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.355+0000 7f21b7cc3740 20 client.27344 trim_cache size 0 max 16384 2024-09-11T01:13:06.360 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:13:06.360 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:13:06.360 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "14920c2e-6e42-4652-a1cc-ea4a4144f749", 2024-09-11T01:13:06.360 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:13:06.360 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:13:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.356+0000 7f21b58ee640 1 -- 172.21.15.78:0/2135810655 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f219c0b54d0 msgr2=0x7f219c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.356+0000 7f21b58ee640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f219c0b54d0 0x7f219c0b7990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f21b0100090 tx=0x7f2198009830 comp rx=0 tx=0).stop 2024-09-11T01:13:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.356+0000 7f21b58ee640 1 -- 172.21.15.78:0/2135810655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21b0108040 msgr2=0x7f21b0198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.356+0000 7f21b58ee640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21b0108040 0x7f21b0198810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f21a8005cc0 tx=0x7f21a8031700 comp rx=0 tx=0).stop 2024-09-11T01:13:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.356+0000 7f21b58ee640 1 -- 172.21.15.78:0/2135810655 shutdown_connections 2024-09-11T01:13:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.356+0000 7f21b58ee640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f219c0b54d0 0x7f219c0b7990 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.356+0000 7f21b58ee640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21b0108040 0x7f21b0198810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.356+0000 7f21b58ee640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21b01076c0 0x7f21b01982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.356+0000 7f21b58ee640 1 --2- 172.21.15.78:0/2135810655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0101cb0 0x7f21b0197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.356+0000 7f21b58ee640 1 -- 172.21.15.78:0/2135810655 >> 172.21.15.78:0/2135810655 conn(0x7f21b00fd700 msgr2=0x7f21b00fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:06.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.356+0000 7f21b58ee640 1 -- 172.21.15.78:0/2135810655 shutdown_connections 2024-09-11T01:13:06.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:06.357+0000 7f21b58ee640 1 -- 172.21.15.78:0/2135810655 wait complete. 2024-09-11T01:13:06.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 40973 33.01 MB/sec warmup 72 sec latency 342.153 ms 2024-09-11T01:13:07.372 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:13:07.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:07 smithi078 ceph-mon[31315]: pgmap v764: 97 pgs: 97 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.5 MiB/s wr, 625 op/s 2024-09-11T01:13:07.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:07 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:13:07.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:07 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:07 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:13:07.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:07 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:13:07.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:07 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:13:07.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:07 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:13:07.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:07 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:13:07.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:07 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:07 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:07 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:07 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.757 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:13:07.776 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:07 smithi112 ceph-mon[35384]: pgmap v764: 97 pgs: 97 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.5 MiB/s wr, 625 op/s 2024-09-11T01:13:07.776 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:07 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:13:07.776 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:07 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.776 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:07 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:13:07.776 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:07 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:13:07.776 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:07 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:13:07.776 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:07 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:13:07.776 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:07 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:13:07.777 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:07 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.777 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:07 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.777 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:07 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.777 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:07 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.777 INFO:tasks.workunit.client.0.smithi078.stdout: 10 41865 33.38 MB/sec warmup 73 sec latency 293.245 ms 2024-09-11T01:13:07.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:07 smithi110 ceph-mon[34712]: pgmap v764: 97 pgs: 97 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.5 MiB/s wr, 625 op/s 2024-09-11T01:13:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:07 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:13:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:07 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:07 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:13:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:07 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:13:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:07 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:13:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:07 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:13:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:07 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:13:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:07 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:07 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:07 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:07.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:07 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:08.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.291+0000 7f2326cec640 1 -- 172.21.15.78:0/1827356407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f232010a150 msgr2=0x7f232010a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:08.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.291+0000 7f2326cec640 1 --2- 172.21.15.78:0/1827356407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f232010a150 0x7f232010a530 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2314009900 tx=0x7f231402a480 comp rx=0 tx=0).stop 2024-09-11T01:13:08.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.291+0000 7f2326cec640 1 -- 172.21.15.78:0/1827356407 shutdown_connections 2024-09-11T01:13:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.291+0000 7f2326cec640 1 --2- 172.21.15.78:0/1827356407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2320115a10 0x7f2320117e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.291+0000 7f2326cec640 1 --2- 172.21.15.78:0/1827356407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f232010aa70 0x7f23201154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.291+0000 7f2326cec640 1 --2- 172.21.15.78:0/1827356407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f232010a150 0x7f232010a530 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.291+0000 7f2326cec640 1 -- 172.21.15.78:0/1827356407 >> 172.21.15.78:0/1827356407 conn(0x7f232006dfc0 msgr2=0x7f232006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.292+0000 7f2326cec640 1 -- 172.21.15.78:0/1827356407 shutdown_connections 2024-09-11T01:13:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.292+0000 7f2326cec640 1 -- 172.21.15.78:0/1827356407 wait complete. 2024-09-11T01:13:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.293+0000 7f2326cec640 1 Processor -- start 2024-09-11T01:13:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.293+0000 7f2326cec640 1 -- start start 2024-09-11T01:13:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.293+0000 7f2326cec640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f232010a150 0x7f232010b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.293+0000 7f2326cec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f232010aa70 0x7f232010b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f2326cec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2320115a10 0x7f232010be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f2326cec640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f232011b580 con 0x7f2320115a10 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f2326cec640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f232011b400 con 0x7f232010a150 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f2326cec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f232011b700 con 0x7f232010aa70 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f231ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f232010aa70 0x7f232010b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f2324a88640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f232010a150 0x7f232010b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f231ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f232010aa70 0x7f232010b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:32980/0 (socket says 172.21.15.78:32980) 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f231ffff640 1 -- 172.21.15.78:0/354726992 learned_addr learned my addr 172.21.15.78:0/354726992 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f2325289640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2320115a10 0x7f232010be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f231ffff640 1 -- 172.21.15.78:0/354726992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f232010a150 msgr2=0x7f232010b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f231ffff640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f232010a150 0x7f232010b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f231ffff640 1 -- 172.21.15.78:0/354726992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2320115a10 msgr2=0x7f232010be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f231ffff640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2320115a10 0x7f232010be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f231ffff640 1 -- 172.21.15.78:0/354726992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23201afc80 con 0x7f232010aa70 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f2324a88640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f232010a150 0x7f232010b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.294+0000 7f2325289640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2320115a10 0x7f232010be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.295+0000 7f231ffff640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f232010aa70 0x7f232010b8c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f230c00e9f0 tx=0x7f230c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.295+0000 7f231dffb640 1 -- 172.21.15.78:0/354726992 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c00ccc0 con 0x7f232010aa70 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.295+0000 7f231dffb640 1 -- 172.21.15.78:0/354726992 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f230c0108e0 con 0x7f232010aa70 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.295+0000 7f231dffb640 1 -- 172.21.15.78:0/354726992 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c0187c0 con 0x7f232010aa70 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.295+0000 7f2326cec640 1 -- 172.21.15.78:0/354726992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23201aff70 con 0x7f232010aa70 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.296+0000 7f2326cec640 1 -- 172.21.15.78:0/354726992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23201b03d0 con 0x7f232010aa70 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.299+0000 7f231dffb640 1 -- 172.21.15.78:0/354726992 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f230c010430 con 0x7f232010aa70 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.300+0000 7f23290c1740 1 Processor -- start 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.300+0000 7f23290c1740 1 -- start start 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.300+0000 7f231dffb640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f23100b5500 0x7f23100b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.300+0000 7f23290c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd0a0 0x5636440dd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.300+0000 7f23290c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd9c0 0x5636440e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.300+0000 7f23290c1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440ebd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f23290c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563644053e50 con 0x5636440dd0a0 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f23290c1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563644043280 con 0x5636440e9950 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f23290c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563644047090 con 0x5636440dd9c0 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f2325289640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f23100b5500 0x7f23100b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f231dffb640 1 -- 172.21.15.78:0/354726992 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8493+0+0 (secure 0 0 0) 0x7f230c0e6310 con 0x7f232010aa70 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f2325289640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440ebd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f2325289640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440ebd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:37130/0 (socket says 172.21.15.78:37130) 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f2325289640 1 -- 172.21.15.78:0/4127191536 learned_addr learned my addr 172.21.15.78:0/4127191536 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f231ffff640 1 --2- 172.21.15.78:0/4127191536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd9c0 0x5636440e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f2324a88640 1 --2- 172.21.15.78:0/4127191536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd0a0 0x5636440dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f2325289640 1 -- 172.21.15.78:0/4127191536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd9c0 msgr2=0x5636440e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f2325289640 1 --2- 172.21.15.78:0/4127191536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd9c0 0x5636440e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f2325289640 1 -- 172.21.15.78:0/4127191536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd0a0 msgr2=0x5636440dd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f2325289640 1 --2- 172.21.15.78:0/4127191536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd0a0 0x5636440dd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f2325289640 1 -- 172.21.15.78:0/4127191536 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563643da1650 con 0x5636440e9950 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f2325289640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f23100b5500 0x7f23100b79c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f231800f4d0 tx=0x7f2318011790 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.301+0000 7f2325289640 1 --2- 172.21.15.78:0/4127191536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440ebd40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f231800fd60 tx=0x7f231800ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd849cdda4d839a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.302+0000 7f230affd640 1 -- 172.21.15.78:0/4127191536 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318024070 con 0x5636440e9950 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.302+0000 7f230affd640 1 -- 172.21.15.78:0/4127191536 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2318003e20 con 0x5636440e9950 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.302+0000 7f230affd640 1 -- 172.21.15.78:0/4127191536 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318004280 con 0x5636440e9950 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.302+0000 7f23290c1740 1 -- 172.21.15.78:0/4127191536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 msgr2=0x5636440ebd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.302+0000 7f23290c1740 1 --2- 172.21.15.78:0/4127191536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440ebd40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f231800fd60 tx=0x7f231800ff90 comp rx=0 tx=0).stop 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.303+0000 7f23290c1740 1 -- 172.21.15.78:0/4127191536 shutdown_connections 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.303+0000 7f23290c1740 1 --2- 172.21.15.78:0/4127191536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440ebd40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.303+0000 7f23290c1740 1 --2- 172.21.15.78:0/4127191536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd9c0 0x5636440e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.303+0000 7f23290c1740 1 --2- 172.21.15.78:0/4127191536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd0a0 0x5636440dd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.303+0000 7f23290c1740 1 -- 172.21.15.78:0/4127191536 >> 172.21.15.78:0/4127191536 conn(0x5636440daed0 msgr2=0x5636440dc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.303+0000 7f23290c1740 1 -- 172.21.15.78:0/4127191536 shutdown_connections 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.303+0000 7f23290c1740 1 -- 172.21.15.78:0/4127191536 wait complete. 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.304+0000 7f23290c1740 1 -- start start 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.304+0000 7f23290c1740 1 Processor -- start 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.304+0000 7f23290c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd0a0 0x5636440e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.304+0000 7f23290c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd9c0 0x5636440e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.304+0000 7f23290c1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440e6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.304+0000 7f23290c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563643ea4700 con 0x5636440dd9c0 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.304+0000 7f23290c1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563643fde110 con 0x5636440e9950 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.304+0000 7f23290c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5636440ec570 con 0x5636440dd0a0 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.304+0000 7f23290c1740 10 client.0 _ll_register_callbacks cb 0x563643ff7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.305+0000 7f2324a88640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd0a0 0x5636440e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.305+0000 7f2325289640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440e6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.305+0000 7f231ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd9c0 0x5636440e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.305+0000 7f2325289640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440e6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:37136/0 (socket says 172.21.15.78:37136) 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.305+0000 7f2325289640 1 -- 172.21.15.78:0/494369904 learned_addr learned my addr 172.21.15.78:0/494369904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.305+0000 7f2325289640 1 -- 172.21.15.78:0/494369904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd0a0 msgr2=0x5636440e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.305+0000 7f2325289640 1 --2- 172.21.15.78:0/494369904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd0a0 0x5636440e9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.305+0000 7f2325289640 1 -- 172.21.15.78:0/494369904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd9c0 msgr2=0x5636440e66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.305+0000 7f2325289640 1 --2- 172.21.15.78:0/494369904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd9c0 0x5636440e66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.305+0000 7f2325289640 1 -- 172.21.15.78:0/494369904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364400bc40 con 0x5636440e9950 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.305+0000 7f2325289640 1 --2- 172.21.15.78:0/494369904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440e6d50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f23180112c0 tx=0x7f231800fe30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c070b99ccf94129f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.305+0000 7f22fffff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.306+0000 7f22fffff640 1 -- 172.21.15.78:0/494369904 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318034060 con 0x5636440e9950 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.306+0000 7f22fffff640 1 -- 172.21.15.78:0/494369904 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8493+0+0 (secure 0 0 0) 0x7f23180050c0 con 0x5636440e9950 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.306+0000 7f23290c1740 1 -- 172.21.15.78:0/494369904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2318003e20 con 0x5636440e9950 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.306+0000 7f22fffff640 1 -- 172.21.15.78:0/494369904 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2318024040 con 0x5636440e9950 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.306+0000 7f23290c1740 10 client.27356 fetch_fsmap learned FSMap version 59 2024-09-11T01:13:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.306+0000 7f23290c1740 1 -- 172.21.15.78:0/494369904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5636440f7f30 con 0x5636440e9950 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.306+0000 7f22fffff640 1 -- 172.21.15.78:0/494369904 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2318003e20 con 0x5636440e9950 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.306+0000 7f22fffff640 1 -- 172.21.15.78:0/494369904 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f231802e030 con 0x5636440e9950 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.306+0000 7f23290c1740 10 client.27356 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.306+0000 7f23290c1740 10 client.27356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.306+0000 7f23290c1740 20 client.27356 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.306+0000 7f23290c1740 1 --2- 172.21.15.78:0/494369904 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636440f9390 0x5636440fb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.307+0000 7f23290c1740 4 client.27356 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.307+0000 7f23290c1740 1 -- 172.21.15.78:0/494369904 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563644055da0 con 0x5636440f9390 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.308+0000 7f2324a88640 1 --2- 172.21.15.78:0/494369904 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636440f9390 0x5636440fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.308+0000 7f22fffff640 10 client.27356 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.308+0000 7f2324a88640 1 --2- 172.21.15.78:0/494369904 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636440f9390 0x5636440fb780 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4666c5e470fa705d server_cookie=1990bccf74751e5b in_seq=0 out_seq=0 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.309+0000 7f22fffff640 1 -- 172.21.15.78:0/494369904 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563644055da0 con 0x5636440f9390 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.309+0000 7f22fffff640 10 client.27356 handle_command_reply: tid=0 2024-09-11T01:13:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.309+0000 7f23290c1740 1 client.27356 shutdown 2024-09-11T01:13:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.309+0000 7f23290c1740 1 -- 172.21.15.78:0/494369904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 msgr2=0x5636440e6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.309+0000 7f23290c1740 1 --2- 172.21.15.78:0/494369904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440e6d50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f23180112c0 tx=0x7f231800fe30 comp rx=0 tx=0).stop 2024-09-11T01:13:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.309+0000 7f23290c1740 1 -- 172.21.15.78:0/494369904 shutdown_connections 2024-09-11T01:13:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.309+0000 7f23290c1740 1 --2- 172.21.15.78:0/494369904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440e6d50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.309+0000 7f23290c1740 1 --2- 172.21.15.78:0/494369904 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636440f9390 0x5636440fb780 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.310+0000 7f22fffff640 0 client.27356 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.310+0000 7f23290c1740 1 --2- 172.21.15.78:0/494369904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd9c0 0x5636440e66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.310+0000 7f23290c1740 1 --2- 172.21.15.78:0/494369904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd0a0 0x5636440e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.310+0000 7f23290c1740 1 -- 172.21.15.78:0/494369904 >> 172.21.15.78:0/494369904 conn(0x5636440daed0 msgr2=0x5636440e0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.310+0000 7f23290c1740 1 -- 172.21.15.78:0/494369904 shutdown_connections 2024-09-11T01:13:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.310+0000 7f23290c1740 1 -- 172.21.15.78:0/494369904 wait complete. 2024-09-11T01:13:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.310+0000 7f23290c1740 20 client.27356 trim_cache size 0 max 16384 2024-09-11T01:13:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f23290c1740 1 Processor -- start 2024-09-11T01:13:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f23290c1740 1 -- start start 2024-09-11T01:13:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f23290c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd0a0 0x5636440f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f23290c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd9c0 0x5636440dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f23290c1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440d9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f23290c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23201a3b20 con 0x5636440dd0a0 2024-09-11T01:13:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f23290c1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f23201a58a0 con 0x5636440e9950 2024-09-11T01:13:08.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f23290c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23201a41f0 con 0x5636440dd9c0 2024-09-11T01:13:08.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f2324a88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd0a0 0x5636440f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f2325289640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440d9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f2325289640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440d9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:37146/0 (socket says 172.21.15.78:37146) 2024-09-11T01:13:08.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f2324a88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd0a0 0x5636440f57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51128/0 (socket says 172.21.15.78:51128) 2024-09-11T01:13:08.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f2325289640 1 -- 172.21.15.78:0/84598883 learned_addr learned my addr 172.21.15.78:0/84598883 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:08.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.336+0000 7f231ffff640 1 --2- 172.21.15.78:0/84598883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd9c0 0x5636440dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.337+0000 7f2325289640 1 -- 172.21.15.78:0/84598883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd9c0 msgr2=0x5636440dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.337+0000 7f2325289640 1 --2- 172.21.15.78:0/84598883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd9c0 0x5636440dbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.337+0000 7f2325289640 1 -- 172.21.15.78:0/84598883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd0a0 msgr2=0x5636440f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.337+0000 7f2325289640 1 --2- 172.21.15.78:0/84598883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd0a0 0x5636440f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.337+0000 7f2325289640 1 -- 172.21.15.78:0/84598883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636440e6480 con 0x5636440e9950 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.337+0000 7f2325289640 1 --2- 172.21.15.78:0/84598883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440d9e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f23180099a0 tx=0x7f231800fea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=841083f8a050a257 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.337+0000 7f230a7fc640 1 -- 172.21.15.78:0/84598883 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318034070 con 0x5636440e9950 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.337+0000 7f230a7fc640 1 -- 172.21.15.78:0/84598883 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f231801fe50 con 0x5636440e9950 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.337+0000 7f230a7fc640 1 -- 172.21.15.78:0/84598883 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318005580 con 0x5636440e9950 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.338+0000 7f23290c1740 1 -- 172.21.15.78:0/84598883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 msgr2=0x5636440d9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.338+0000 7f23290c1740 1 --2- 172.21.15.78:0/84598883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440d9e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f23180099a0 tx=0x7f231800fea0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.338+0000 7f23290c1740 1 -- 172.21.15.78:0/84598883 shutdown_connections 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.338+0000 7f23290c1740 1 --2- 172.21.15.78:0/84598883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440e9950 0x5636440d9e10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.338+0000 7f23290c1740 1 --2- 172.21.15.78:0/84598883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440dd9c0 0x5636440dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.338+0000 7f23290c1740 1 --2- 172.21.15.78:0/84598883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd0a0 0x5636440f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.338+0000 7f23290c1740 1 -- 172.21.15.78:0/84598883 >> 172.21.15.78:0/84598883 conn(0x5636440daed0 msgr2=0x5636440d9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:08.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.338+0000 7f23290c1740 1 -- 172.21.15.78:0/84598883 shutdown_connections 2024-09-11T01:13:08.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.338+0000 7f23290c1740 1 -- 172.21.15.78:0/84598883 wait complete. 2024-09-11T01:13:08.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.338+0000 7f23290c1740 1 -- start start 2024-09-11T01:13:08.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.339+0000 7f23290c1740 1 Processor -- start 2024-09-11T01:13:08.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.339+0000 7f23290c1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440dd0a0 0x5636440f1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.339+0000 7f23290c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd9c0 0x5636440f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.339+0000 7f23290c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440e9950 0x5636440ec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.339+0000 7f23290c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23201a5110 con 0x5636440dd9c0 2024-09-11T01:13:08.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.339+0000 7f2324a88640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440dd0a0 0x5636440f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.339+0000 7f23290c1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f23201a4980 con 0x5636440dd0a0 2024-09-11T01:13:08.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.339+0000 7f2324a88640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440dd0a0 0x5636440f1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:37148/0 (socket says 172.21.15.78:37148) 2024-09-11T01:13:08.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.339+0000 7f2324a88640 1 -- 172.21.15.78:0/1579139816 learned_addr learned my addr 172.21.15.78:0/1579139816 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:08.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.339+0000 7f231ffff640 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd9c0 0x5636440f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.339+0000 7f23290c1740 1 -- 172.21.15.78:0/1579139816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23201a1b20 con 0x5636440e9950 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.339+0000 7f23290c1740 10 client.0 _ll_register_callbacks cb 0x7f230c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.340+0000 7f2325289640 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440e9950 0x5636440ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.340+0000 7f2324a88640 1 -- 172.21.15.78:0/1579139816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440e9950 msgr2=0x5636440ec810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.340+0000 7f2324a88640 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440e9950 0x5636440ec810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.340+0000 7f2324a88640 1 -- 172.21.15.78:0/1579139816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd9c0 msgr2=0x5636440f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.340+0000 7f2324a88640 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd9c0 0x5636440f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.340+0000 7f2324a88640 1 -- 172.21.15.78:0/1579139816 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636440f24e0 con 0x5636440dd0a0 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.340+0000 7f231ffff640 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd9c0 0x5636440f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.340+0000 7f2324a88640 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440dd0a0 0x5636440f1860 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2314009900 tx=0x7f2314000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a5b3bcc2dfb066e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.340+0000 7f22fffff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.340+0000 7f22fffff640 1 -- 172.21.15.78:0/1579139816 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314035040 con 0x5636440dd0a0 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.340+0000 7f22fffff640 1 -- 172.21.15.78:0/1579139816 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8493+0+0 (secure 0 0 0) 0x7f2314005450 con 0x5636440dd0a0 2024-09-11T01:13:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.340+0000 7f23290c1740 1 -- 172.21.15.78:0/1579139816 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f231801fe50 con 0x5636440dd0a0 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.341+0000 7f22fffff640 1 -- 172.21.15.78:0/1579139816 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2314005650 con 0x5636440dd0a0 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.341+0000 7f23290c1740 10 client.27362 fetch_fsmap learned FSMap version 59 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.341+0000 7f23290c1740 1 -- 172.21.15.78:0/1579139816 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5636441665f0 con 0x5636440dd0a0 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.341+0000 7f22fffff640 1 -- 172.21.15.78:0/1579139816 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2314004700 con 0x5636440dd0a0 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.341+0000 7f22fffff640 1 -- 172.21.15.78:0/1579139816 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f23140035f0 con 0x5636440dd0a0 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.341+0000 7f23290c1740 10 client.27362 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.341+0000 7f23290c1740 10 client.27362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.341+0000 7f23290c1740 20 client.27362 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.341+0000 7f23290c1740 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636440f9390 0x563644168d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.341+0000 7f23290c1740 4 client.27362 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.341+0000 7f23290c1740 1 -- 172.21.15.78:0/1579139816 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f231801fe50 con 0x5636440f9390 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.341+0000 7f231ffff640 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636440f9390 0x563644168d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.342+0000 7f231ffff640 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636440f9390 0x563644168d70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83f9cf7afb9c668 server_cookie=a3e763f8c90dc4a5 in_seq=0 out_seq=0 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.342+0000 7f22fffff640 10 client.27362 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.342+0000 7f22fffff640 1 -- 172.21.15.78:0/1579139816 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5682 (crc 0 0 0) 0x7f231801fe50 con 0x5636440f9390 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.342+0000 7f22fffff640 10 client.27362 handle_command_reply: tid=0 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.342+0000 7f23290c1740 1 client.27362 shutdown 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.342+0000 7f23290c1740 1 -- 172.21.15.78:0/1579139816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440dd0a0 msgr2=0x5636440f1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.342+0000 7f23290c1740 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440dd0a0 0x5636440f1860 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2314009900 tx=0x7f2314000bc0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.343+0000 7f23290c1740 1 -- 172.21.15.78:0/1579139816 shutdown_connections 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.343+0000 7f23290c1740 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636440e9950 0x5636440ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.343+0000 7f23290c1740 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636440f9390 0x563644168d70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.343+0000 7f23290c1740 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636440dd9c0 0x5636440f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.343+0000 7f23290c1740 1 --2- 172.21.15.78:0/1579139816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636440dd0a0 0x5636440f1860 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.343+0000 7f22fffff640 0 client.27362 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.343+0000 7f23290c1740 1 -- 172.21.15.78:0/1579139816 >> 172.21.15.78:0/1579139816 conn(0x5636440daed0 msgr2=0x5636440d9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:08.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.343+0000 7f23290c1740 1 -- 172.21.15.78:0/1579139816 shutdown_connections 2024-09-11T01:13:08.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.343+0000 7f23290c1740 1 -- 172.21.15.78:0/1579139816 wait complete. 2024-09-11T01:13:08.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.343+0000 7f23290c1740 20 client.27362 trim_cache size 0 max 16384 2024-09-11T01:13:08.349 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:13:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:13:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "14920c2e-6e42-4652-a1cc-ea4a4144f749", 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:13:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:13:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:13:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:13:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:13:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:13:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:13:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:13:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:13:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:13:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:13:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:13:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:13:08.362 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:13:08.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.344+0000 7f2326cec640 1 -- 172.21.15.78:0/354726992 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f23100b5500 msgr2=0x7f23100b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:08.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.344+0000 7f2326cec640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f23100b5500 0x7f23100b79c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f231800f4d0 tx=0x7f2318011790 comp rx=0 tx=0).stop 2024-09-11T01:13:08.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.344+0000 7f2326cec640 1 -- 172.21.15.78:0/354726992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f232010aa70 msgr2=0x7f232010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:08.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.344+0000 7f2326cec640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f232010aa70 0x7f232010b8c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f230c00e9f0 tx=0x7f230c00edf0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.344+0000 7f2326cec640 1 -- 172.21.15.78:0/354726992 shutdown_connections 2024-09-11T01:13:08.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.344+0000 7f2326cec640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f23100b5500 0x7f23100b79c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.344+0000 7f2326cec640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2320115a10 0x7f232010be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.344+0000 7f2326cec640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f232010aa70 0x7f232010b8c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.344+0000 7f2326cec640 1 --2- 172.21.15.78:0/354726992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f232010a150 0x7f232010b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:08.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.344+0000 7f2326cec640 1 -- 172.21.15.78:0/354726992 >> 172.21.15.78:0/354726992 conn(0x7f232006dfc0 msgr2=0x7f23200716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:08.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.344+0000 7f2326cec640 1 -- 172.21.15.78:0/354726992 shutdown_connections 2024-09-11T01:13:08.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:08.344+0000 7f2326cec640 1 -- 172.21.15.78:0/354726992 wait complete. 2024-09-11T01:13:08.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 42609 33.34 MB/sec warmup 74 sec latency 198.508 ms 2024-09-11T01:13:08.926 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:08 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:08.926 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:08 smithi078 ceph-mon[31315]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T01:13:08.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:08 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:08.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:08 smithi110 ceph-mon[34712]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T01:13:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:08 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:13:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:08 smithi112 ceph-mon[35384]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T01:13:09.092 INFO:tasks.cephfs.filesystem:scrub status for tag:14920c2e-6e42-4652-a1cc-ea4a4144f749 - {'path': '/', 'tag': '14920c2e-6e42-4652-a1cc-ea4a4144f749', 'options': 'recursive,force'} 2024-09-11T01:13:09.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 42922 33.14 MB/sec warmup 75 sec latency 613.567 ms 2024-09-11T01:13:09.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:09 smithi110 ceph-mon[34712]: pgmap v766: 97 pgs: 97 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.3 MiB/s wr, 499 op/s 2024-09-11T01:13:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:09 smithi112 ceph-mon[35384]: pgmap v766: 97 pgs: 97 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.3 MiB/s wr, 499 op/s 2024-09-11T01:13:09.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:09 smithi078 ceph-mon[31315]: pgmap v766: 97 pgs: 97 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.3 MiB/s wr, 499 op/s 2024-09-11T01:13:10.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 44086 33.53 MB/sec warmup 76 sec latency 33.175 ms 2024-09-11T01:13:11.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 45218 33.91 MB/sec warmup 77 sec latency 53.640 ms 2024-09-11T01:13:11.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:11 smithi110 ceph-mon[34712]: pgmap v767: 97 pgs: 97 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 5.6 MiB/s wr, 639 op/s 2024-09-11T01:13:11.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:11 smithi112 ceph-mon[35384]: pgmap v767: 97 pgs: 97 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 5.6 MiB/s wr, 639 op/s 2024-09-11T01:13:11.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:11 smithi078 ceph-mon[31315]: pgmap v767: 97 pgs: 97 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 5.6 MiB/s wr, 639 op/s 2024-09-11T01:13:12.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 46121 34.12 MB/sec warmup 78 sec latency 72.343 ms 2024-09-11T01:13:13.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 47016 34.23 MB/sec warmup 79 sec latency 110.433 ms 2024-09-11T01:13:13.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:13 smithi110 ceph-mon[34712]: pgmap v768: 97 pgs: 97 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 708 op/s 2024-09-11T01:13:13.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:13 smithi110 ceph-mon[34712]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T01:13:13.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:13 smithi112 ceph-mon[35384]: pgmap v768: 97 pgs: 97 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 708 op/s 2024-09-11T01:13:13.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:13 smithi112 ceph-mon[35384]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T01:13:13.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:13 smithi078 ceph-mon[31315]: pgmap v768: 97 pgs: 97 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 708 op/s 2024-09-11T01:13:13.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:13 smithi078 ceph-mon[31315]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T01:13:14.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 47896 34.45 MB/sec warmup 80 sec latency 111.079 ms 2024-09-11T01:13:15.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 48916 34.72 MB/sec warmup 81 sec latency 206.244 ms 2024-09-11T01:13:15.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:15 smithi110 ceph-mon[34712]: pgmap v770: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.6 MiB/s wr, 795 op/s 2024-09-11T01:13:15.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:15 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:13:15.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:15 smithi112 ceph-mon[35384]: pgmap v770: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.6 MiB/s wr, 795 op/s 2024-09-11T01:13:15.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:15 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:13:16.000 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:15 smithi078 ceph-mon[31315]: pgmap v770: 97 pgs: 97 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.6 MiB/s wr, 795 op/s 2024-09-11T01:13:16.000 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:15 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:13:16.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 49755 34.83 MB/sec warmup 82 sec latency 43.013 ms 2024-09-11T01:13:17.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 50624 34.90 MB/sec warmup 83 sec latency 91.120 ms 2024-09-11T01:13:17.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:17 smithi110 ceph-mon[34712]: pgmap v771: 97 pgs: 97 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.7 MiB/s wr, 864 op/s 2024-09-11T01:13:17.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:17 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:13:17.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:17 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:13:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:17 smithi112 ceph-mon[35384]: pgmap v771: 97 pgs: 97 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.7 MiB/s wr, 864 op/s 2024-09-11T01:13:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:17 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:13:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:17 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:13:17.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:17 smithi078 ceph-mon[31315]: pgmap v771: 97 pgs: 97 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.7 MiB/s wr, 864 op/s 2024-09-11T01:13:17.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:17 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:13:17.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:17 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:13:18.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 51682 35.19 MB/sec warmup 84 sec latency 131.903 ms 2024-09-11T01:13:18.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:18 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:13:18.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:18 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:13:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:18 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:13:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:18 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:13:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:18 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:13:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:18 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:13:18.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:18 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:13:18.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:18 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:13:18.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:18 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:13:19.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 52639 35.49 MB/sec warmup 85 sec latency 91.662 ms 2024-09-11T01:13:19.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:19 smithi110 ceph-mon[34712]: pgmap v772: 97 pgs: 97 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.6 MiB/s wr, 795 op/s 2024-09-11T01:13:19.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:19 smithi112 ceph-mon[35384]: pgmap v772: 97 pgs: 97 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.6 MiB/s wr, 795 op/s 2024-09-11T01:13:19.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:19 smithi078 ceph-mon[31315]: pgmap v772: 97 pgs: 97 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.6 MiB/s wr, 795 op/s 2024-09-11T01:13:20.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 53445 35.57 MB/sec warmup 86 sec latency 93.026 ms 2024-09-11T01:13:21.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 54140 35.47 MB/sec warmup 87 sec latency 237.663 ms 2024-09-11T01:13:21.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:21 smithi110 ceph-mon[34712]: pgmap v773: 97 pgs: 97 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 10 MiB/s wr, 724 op/s 2024-09-11T01:13:21.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:21 smithi112 ceph-mon[35384]: pgmap v773: 97 pgs: 97 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 10 MiB/s wr, 724 op/s 2024-09-11T01:13:21.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:21 smithi078 ceph-mon[31315]: pgmap v773: 97 pgs: 97 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 10 MiB/s wr, 724 op/s 2024-09-11T01:13:22.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 54181 35.09 MB/sec warmup 88 sec latency 705.649 ms 2024-09-11T01:13:23.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 54714 35.06 MB/sec warmup 89 sec latency 230.545 ms 2024-09-11T01:13:23.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:23 smithi110 ceph-mon[34712]: pgmap v774: 97 pgs: 97 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 11 MiB/s wr, 715 op/s 2024-09-11T01:13:23.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:23 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:13:23.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:23 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:13:23.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:23 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:13:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:23 smithi112 ceph-mon[35384]: pgmap v774: 97 pgs: 97 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 11 MiB/s wr, 715 op/s 2024-09-11T01:13:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:23 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:13:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:23 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:13:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:23 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:13:24.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:23 smithi078 ceph-mon[31315]: pgmap v774: 97 pgs: 97 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 11 MiB/s wr, 715 op/s 2024-09-11T01:13:24.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:23 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:13:24.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:23 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:13:24.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:23 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:13:24.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 55480 35.12 MB/sec warmup 90 sec latency 197.206 ms 2024-09-11T01:13:25.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 56310 35.26 MB/sec warmup 91 sec latency 181.370 ms 2024-09-11T01:13:25.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:25 smithi110 ceph-mon[34712]: pgmap v775: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 17 MiB/s wr, 800 op/s 2024-09-11T01:13:25.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:25 smithi112 ceph-mon[35384]: pgmap v775: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 17 MiB/s wr, 800 op/s 2024-09-11T01:13:25.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:25 smithi078 ceph-mon[31315]: pgmap v775: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 17 MiB/s wr, 800 op/s 2024-09-11T01:13:26.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 57210 35.40 MB/sec warmup 92 sec latency 39.956 ms 2024-09-11T01:13:27.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 58126 35.53 MB/sec warmup 93 sec latency 63.583 ms 2024-09-11T01:13:27.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:27 smithi110 ceph-mon[34712]: pgmap v776: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 26 MiB/s wr, 1.13k op/s 2024-09-11T01:13:27.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:27 smithi112 ceph-mon[35384]: pgmap v776: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 26 MiB/s wr, 1.13k op/s 2024-09-11T01:13:27.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:27 smithi078 ceph-mon[31315]: pgmap v776: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 26 MiB/s wr, 1.13k op/s 2024-09-11T01:13:28.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 59266 35.85 MB/sec warmup 94 sec latency 33.755 ms 2024-09-11T01:13:29.778 INFO:tasks.workunit.client.0.smithi078.stdout: 10 60129 35.93 MB/sec warmup 95 sec latency 48.291 ms 2024-09-11T01:13:29.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:29 smithi110 ceph-mon[34712]: pgmap v777: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 26 MiB/s wr, 1.11k op/s 2024-09-11T01:13:29.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:29 smithi112 ceph-mon[35384]: pgmap v777: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 26 MiB/s wr, 1.11k op/s 2024-09-11T01:13:29.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:29 smithi078 ceph-mon[31315]: pgmap v777: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 26 MiB/s wr, 1.11k op/s 2024-09-11T01:13:30.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 60926 36.03 MB/sec warmup 96 sec latency 283.942 ms 2024-09-11T01:13:31.783 INFO:tasks.workunit.client.0.smithi078.stdout: 10 61835 36.13 MB/sec warmup 97 sec latency 300.601 ms 2024-09-11T01:13:31.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:31 smithi110 ceph-mon[34712]: pgmap v778: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 26 MiB/s wr, 1.14k op/s 2024-09-11T01:13:31.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:31 smithi112 ceph-mon[35384]: pgmap v778: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 26 MiB/s wr, 1.14k op/s 2024-09-11T01:13:31.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:31 smithi078 ceph-mon[31315]: pgmap v778: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 26 MiB/s wr, 1.14k op/s 2024-09-11T01:13:32.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 62690 36.29 MB/sec warmup 98 sec latency 54.471 ms 2024-09-11T01:13:33.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 63549 36.35 MB/sec warmup 99 sec latency 123.704 ms 2024-09-11T01:13:33.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:33 smithi110 ceph-mon[34712]: pgmap v779: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 25 MiB/s wr, 1.14k op/s 2024-09-11T01:13:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:33 smithi112 ceph-mon[35384]: pgmap v779: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 25 MiB/s wr, 1.14k op/s 2024-09-11T01:13:33.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:33 smithi078 ceph-mon[31315]: pgmap v779: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 25 MiB/s wr, 1.14k op/s 2024-09-11T01:13:34.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 64498 36.48 MB/sec warmup 100 sec latency 298.536 ms 2024-09-11T01:13:35.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 64946 36.35 MB/sec warmup 101 sec latency 652.788 ms 2024-09-11T01:13:35.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:35 smithi110 ceph-mon[34712]: pgmap v780: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 24 MiB/s wr, 1.12k op/s 2024-09-11T01:13:35.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:35 smithi112 ceph-mon[35384]: pgmap v780: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 24 MiB/s wr, 1.12k op/s 2024-09-11T01:13:35.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:35 smithi078 ceph-mon[31315]: pgmap v780: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 24 MiB/s wr, 1.12k op/s 2024-09-11T01:13:36.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 65686 36.40 MB/sec warmup 102 sec latency 770.991 ms 2024-09-11T01:13:37.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 66562 36.53 MB/sec warmup 103 sec latency 134.618 ms 2024-09-11T01:13:37.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:37 smithi110 ceph-mon[34712]: pgmap v781: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 1.07k op/s 2024-09-11T01:13:37.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:37 smithi112 ceph-mon[35384]: pgmap v781: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 1.07k op/s 2024-09-11T01:13:37.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:37 smithi078 ceph-mon[31315]: pgmap v781: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 1.07k op/s 2024-09-11T01:13:38.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 67458 36.61 MB/sec warmup 104 sec latency 92.397 ms 2024-09-11T01:13:39.093 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:13:39.467 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:13:39.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 68551 36.79 MB/sec warmup 105 sec latency 140.067 ms 2024-09-11T01:13:39.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:39 smithi110 ceph-mon[34712]: pgmap v782: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.7 MiB/s wr, 653 op/s 2024-09-11T01:13:39.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:39 smithi112 ceph-mon[35384]: pgmap v782: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.7 MiB/s wr, 653 op/s 2024-09-11T01:13:39.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:39 smithi078 ceph-mon[31315]: pgmap v782: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.7 MiB/s wr, 653 op/s 2024-09-11T01:13:40.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 69546 37.03 MB/sec warmup 106 sec latency 76.603 ms 2024-09-11T01:13:41.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 70161 37.02 MB/sec warmup 107 sec latency 314.323 ms 2024-09-11T01:13:41.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:41 smithi078 ceph-mon[31315]: pgmap v783: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 8.9 MiB/s wr, 708 op/s 2024-09-11T01:13:41.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:41 smithi110 ceph-mon[34712]: pgmap v783: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 8.9 MiB/s wr, 708 op/s 2024-09-11T01:13:41.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:41 smithi112 ceph-mon[35384]: pgmap v783: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 8.9 MiB/s wr, 708 op/s 2024-09-11T01:13:42.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.070+0000 7fc233790640 1 -- 172.21.15.78:0/3793528836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c101c90 msgr2=0x7fc22c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:42.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.070+0000 7fc233790640 1 --2- 172.21.15.78:0/3793528836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c101c90 0x7fc22c102070 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc228009900 tx=0x7fc22802a410 comp rx=0 tx=0).stop 2024-09-11T01:13:42.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.071+0000 7fc233790640 1 -- 172.21.15.78:0/3793528836 shutdown_connections 2024-09-11T01:13:42.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.071+0000 7fc233790640 1 --2- 172.21.15.78:0/3793528836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc22c108020 0x7fc22c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.071+0000 7fc233790640 1 --2- 172.21.15.78:0/3793528836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22c1076a0 0x7fc22c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.071+0000 7fc233790640 1 --2- 172.21.15.78:0/3793528836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c101c90 0x7fc22c102070 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.072+0000 7fc233790640 1 -- 172.21.15.78:0/3793528836 >> 172.21.15.78:0/3793528836 conn(0x7fc22c0fd700 msgr2=0x7fc22c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:42.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.074+0000 7fc233790640 1 -- 172.21.15.78:0/3793528836 shutdown_connections 2024-09-11T01:13:42.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.075+0000 7fc233790640 1 -- 172.21.15.78:0/3793528836 wait complete. 2024-09-11T01:13:42.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.076+0000 7fc233790640 1 Processor -- start 2024-09-11T01:13:42.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.079+0000 7fc233790640 1 -- start start 2024-09-11T01:13:42.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.080+0000 7fc233790640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c101c90 0x7fc22c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.080+0000 7fc233790640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc22c1076a0 0x7fc22c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.080+0000 7fc233790640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22c108020 0x7fc22c19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.080+0000 7fc233790640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c111cf0 con 0x7fc22c101c90 2024-09-11T01:13:42.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.081+0000 7fc233790640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c111b70 con 0x7fc22c108020 2024-09-11T01:13:42.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.081+0000 7fc233790640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c111e70 con 0x7fc22c1076a0 2024-09-11T01:13:42.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.081+0000 7fc23152c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c101c90 0x7fc22c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.081+0000 7fc23152c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c101c90 0x7fc22c1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47616/0 (socket says 172.21.15.78:47616) 2024-09-11T01:13:42.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.081+0000 7fc23152c640 1 -- 172.21.15.78:0/199054788 learned_addr learned my addr 172.21.15.78:0/199054788 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:42.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.083+0000 7fc231d2d640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22c108020 0x7fc22c19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.083+0000 7fc230d2b640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc22c1076a0 0x7fc22c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.088+0000 7fc230d2b640 1 -- 172.21.15.78:0/199054788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22c108020 msgr2=0x7fc22c19da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:42.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.088+0000 7fc230d2b640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22c108020 0x7fc22c19da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.088+0000 7fc230d2b640 1 -- 172.21.15.78:0/199054788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c101c90 msgr2=0x7fc22c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.088+0000 7fc230d2b640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c101c90 0x7fc22c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.088+0000 7fc230d2b640 1 -- 172.21.15.78:0/199054788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c19e250 con 0x7fc22c1076a0 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.089+0000 7fc23152c640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c101c90 0x7fc22c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.090+0000 7fc230d2b640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc22c1076a0 0x7fc22c1986e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc21c005fd0 tx=0x7fc21c009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.090+0000 7fc2227fc640 1 -- 172.21.15.78:0/199054788 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c004030 con 0x7fc22c1076a0 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.091+0000 7fc2227fc640 1 -- 172.21.15.78:0/199054788 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc21c0078f0 con 0x7fc22c1076a0 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.091+0000 7fc2227fc640 1 -- 172.21.15.78:0/199054788 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c003c30 con 0x7fc22c1076a0 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.091+0000 7fc233790640 1 -- 172.21.15.78:0/199054788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c19e540 con 0x7fc22c1076a0 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.091+0000 7fc233790640 1 -- 172.21.15.78:0/199054788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c06aab0 con 0x7fc22c1076a0 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.099+0000 7fc235b65740 1 Processor -- start 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.100+0000 7fc235b65740 1 -- start start 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.100+0000 7fc235b65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c54b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.100+0000 7fc235b65740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c560000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.100+0000 7fc235b65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c560540 0x55953c562930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.100+0000 7fc235b65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55953c4c1e50 con 0x55953c54b480 2024-09-11T01:13:42.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.100+0000 7fc235b65740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55953c4b1280 con 0x55953c54bda0 2024-09-11T01:13:42.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.100+0000 7fc235b65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55953c4b5090 con 0x55953c560540 2024-09-11T01:13:42.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.102+0000 7fc23152c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c560000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.102+0000 7fc23152c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c560000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:56232/0 (socket says 172.21.15.78:56232) 2024-09-11T01:13:42.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.102+0000 7fc23152c640 1 -- 172.21.15.78:0/565028313 learned_addr learned my addr 172.21.15.78:0/565028313 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:42.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.103+0000 7fc2227fc640 1 -- 172.21.15.78:0/199054788 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc21c01a410 con 0x7fc22c1076a0 2024-09-11T01:13:42.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.104+0000 7fc2227fc640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc2180b5500 0x7fc2180b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.104+0000 7fc2227fc640 1 -- 172.21.15.78:0/199054788 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7fc21c003dd0 con 0x7fc22c1076a0 2024-09-11T01:13:42.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.108+0000 7fc231d2d640 1 --2- 172.21.15.78:0/565028313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c54b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.108+0000 7fc231d2d640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc2180b5500 0x7fc2180b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.109+0000 7fc230d2b640 1 --2- 172.21.15.78:0/565028313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c560540 0x55953c562930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.113+0000 7fc23152c640 1 -- 172.21.15.78:0/565028313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c560540 msgr2=0x55953c562930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.113+0000 7fc23152c640 1 --2- 172.21.15.78:0/565028313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c560540 0x55953c562930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.113+0000 7fc23152c640 1 -- 172.21.15.78:0/565028313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 msgr2=0x55953c54b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.113+0000 7fc23152c640 1 --2- 172.21.15.78:0/565028313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c54b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.113+0000 7fc23152c640 1 -- 172.21.15.78:0/565028313 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55953c20f650 con 0x55953c54bda0 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.113+0000 7fc23152c640 1 --2- 172.21.15.78:0/565028313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c560000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc228006010 tx=0x7fc228004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4471081a9fd333f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.114+0000 7fc2177fe640 1 -- 172.21.15.78:0/565028313 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228035040 con 0x55953c54bda0 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.114+0000 7fc2177fe640 1 -- 172.21.15.78:0/565028313 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc228004780 con 0x55953c54bda0 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.115+0000 7fc235b65740 1 -- 172.21.15.78:0/565028313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 msgr2=0x55953c560000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.115+0000 7fc235b65740 1 --2- 172.21.15.78:0/565028313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c560000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc228006010 tx=0x7fc228004610 comp rx=0 tx=0).stop 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.117+0000 7fc235b65740 1 -- 172.21.15.78:0/565028313 shutdown_connections 2024-09-11T01:13:42.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.117+0000 7fc235b65740 1 --2- 172.21.15.78:0/565028313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c560540 0x55953c562930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.117+0000 7fc235b65740 1 --2- 172.21.15.78:0/565028313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c560000 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.117+0000 7fc235b65740 1 --2- 172.21.15.78:0/565028313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c54b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.117+0000 7fc235b65740 1 -- 172.21.15.78:0/565028313 >> 172.21.15.78:0/565028313 conn(0x55953c5492b0 msgr2=0x55953c54a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.117+0000 7fc231d2d640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc2180b5500 0x7fc2180b79c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc224005fd0 tx=0x7fc224009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.118+0000 7fc235b65740 1 -- 172.21.15.78:0/565028313 shutdown_connections 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.118+0000 7fc235b65740 1 -- 172.21.15.78:0/565028313 wait complete. 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.118+0000 7fc235b65740 1 -- start start 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.119+0000 7fc235b65740 1 Processor -- start 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.119+0000 7fc235b65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c555280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.119+0000 7fc235b65740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c555800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.119+0000 7fc235b65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c560540 0x55953c55fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.119+0000 7fc235b65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55953c312700 con 0x55953c54b480 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.119+0000 7fc235b65740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55953c44c110 con 0x55953c54bda0 2024-09-11T01:13:42.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.119+0000 7fc235b65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55953c551b40 con 0x55953c560540 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.119+0000 7fc235b65740 10 client.0 _ll_register_callbacks cb 0x55953c465990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.120+0000 7fc231d2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c560540 0x55953c55fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.120+0000 7fc231d2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c560540 0x55953c55fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52670/0 (socket says 172.21.15.78:52670) 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.120+0000 7fc231d2d640 1 -- 172.21.15.78:0/1536856275 learned_addr learned my addr 172.21.15.78:0/1536856275 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.121+0000 7fc230d2b640 1 --2- 172.21.15.78:0/1536856275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c555800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.121+0000 7fc231d2d640 1 -- 172.21.15.78:0/1536856275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 msgr2=0x55953c555800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.121+0000 7fc231d2d640 1 --2- 172.21.15.78:0/1536856275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c555800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.121+0000 7fc231d2d640 1 -- 172.21.15.78:0/1536856275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 msgr2=0x55953c555280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.121+0000 7fc231d2d640 1 --2- 172.21.15.78:0/1536856275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c555280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.121+0000 7fc231d2d640 1 -- 172.21.15.78:0/1536856275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55953c479c40 con 0x55953c560540 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.122+0000 7fc231d2d640 1 --2- 172.21.15.78:0/1536856275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c560540 0x55953c55fa80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc22401c8b0 tx=0x7fc224004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53d77ba78095270 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.122+0000 7fc214ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.122+0000 7fc214ff9640 1 -- 172.21.15.78:0/1536856275 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22400da90 con 0x55953c560540 2024-09-11T01:13:42.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.123+0000 7fc235b65740 1 -- 172.21.15.78:0/1536856275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc228004780 con 0x55953c560540 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.123+0000 7fc214ff9640 1 -- 172.21.15.78:0/1536856275 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7fc22400dc30 con 0x55953c560540 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.123+0000 7fc214ff9640 1 -- 172.21.15.78:0/1536856275 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fc224034050 con 0x55953c560540 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.126+0000 7fc235b65740 10 client.27325 fetch_fsmap learned FSMap version 59 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.126+0000 7fc235b65740 1 -- 172.21.15.78:0/1536856275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55953c5644c0 con 0x55953c560540 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.126+0000 7fc214ff9640 1 -- 172.21.15.78:0/1536856275 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc228004780 con 0x55953c560540 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.127+0000 7fc214ff9640 1 -- 172.21.15.78:0/1536856275 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7fc22401b3c0 con 0x55953c560540 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.130+0000 7fc235b65740 10 client.27325 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.130+0000 7fc235b65740 10 client.27325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.130+0000 7fc235b65740 20 client.27325 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.130+0000 7fc235b65740 1 --2- 172.21.15.78:0/1536856275 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55953c565980 0x55953c567e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.130+0000 7fc235b65740 4 client.27325 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.130+0000 7fc235b65740 1 -- 172.21.15.78:0/1536856275 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55953c4c3da0 con 0x55953c565980 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.131+0000 7fc23152c640 1 --2- 172.21.15.78:0/1536856275 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55953c565980 0x55953c567e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.142+0000 7fc23152c640 1 --2- 172.21.15.78:0/1536856275 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55953c565980 0x55953c567e40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a559823e9988585d server_cookie=cba9a4262e0c4661 in_seq=0 out_seq=0 2024-09-11T01:13:42.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.142+0000 7fc214ff9640 10 client.27325 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:42.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.148+0000 7fc214ff9640 1 -- 172.21.15.78:0/1536856275 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55953c4c3da0 con 0x55953c565980 2024-09-11T01:13:42.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.148+0000 7fc214ff9640 10 client.27325 handle_command_reply: tid=0 2024-09-11T01:13:42.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.149+0000 7fc235b65740 1 client.27325 shutdown 2024-09-11T01:13:42.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.153+0000 7fc235b65740 1 -- 172.21.15.78:0/1536856275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c560540 msgr2=0x55953c55fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:42.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.153+0000 7fc235b65740 1 --2- 172.21.15.78:0/1536856275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c560540 0x55953c55fa80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc22401c8b0 tx=0x7fc224004a40 comp rx=0 tx=0).stop 2024-09-11T01:13:42.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.154+0000 7fc235b65740 1 -- 172.21.15.78:0/1536856275 shutdown_connections 2024-09-11T01:13:42.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.154+0000 7fc235b65740 1 --2- 172.21.15.78:0/1536856275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c560540 0x55953c55fa80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.154+0000 7fc235b65740 1 --2- 172.21.15.78:0/1536856275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c555800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.154+0000 7fc235b65740 1 --2- 172.21.15.78:0/1536856275 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55953c565980 0x55953c567e40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.154+0000 7fc235b65740 1 --2- 172.21.15.78:0/1536856275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c555280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.154+0000 7fc235b65740 1 -- 172.21.15.78:0/1536856275 >> 172.21.15.78:0/1536856275 conn(0x55953c5492b0 msgr2=0x55953c54ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:42.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.154+0000 7fc214ff9640 0 client.27325 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:42.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.154+0000 7fc235b65740 1 -- 172.21.15.78:0/1536856275 shutdown_connections 2024-09-11T01:13:42.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.154+0000 7fc235b65740 1 -- 172.21.15.78:0/1536856275 wait complete. 2024-09-11T01:13:42.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.154+0000 7fc235b65740 20 client.27325 trim_cache size 0 max 16384 2024-09-11T01:13:42.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.193+0000 7fc235b65740 1 Processor -- start 2024-09-11T01:13:42.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.195+0000 7fc235b65740 1 -- start start 2024-09-11T01:13:42.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.195+0000 7fc235b65740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c5492b0 0x55953c575680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.195+0000 7fc235b65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c54a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.195+0000 7fc235b65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c54bda0 0x55953c54a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.195+0000 7fc235b65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c1ac580 con 0x55953c54b480 2024-09-11T01:13:42.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.195+0000 7fc235b65740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c1ae300 con 0x55953c5492b0 2024-09-11T01:13:42.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.195+0000 7fc235b65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c1acc50 con 0x55953c54bda0 2024-09-11T01:13:42.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.196+0000 7fc231d2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c54bda0 0x55953c54a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.197+0000 7fc231d2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c54bda0 0x55953c54a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52682/0 (socket says 172.21.15.78:52682) 2024-09-11T01:13:42.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.197+0000 7fc231d2d640 1 -- 172.21.15.78:0/1228398491 learned_addr learned my addr 172.21.15.78:0/1228398491 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:42.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.198+0000 7fc230d2b640 1 --2- 172.21.15.78:0/1228398491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c54a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.198+0000 7fc230d2b640 1 -- 172.21.15.78:0/1228398491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c54bda0 msgr2=0x55953c54a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:42.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.198+0000 7fc230d2b640 1 --2- 172.21.15.78:0/1228398491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c54bda0 0x55953c54a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.198+0000 7fc230d2b640 1 -- 172.21.15.78:0/1228398491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c5492b0 msgr2=0x55953c575680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:13:42.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.198+0000 7fc230d2b640 1 --2- 172.21.15.78:0/1228398491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c5492b0 0x55953c575680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.198+0000 7fc230d2b640 1 -- 172.21.15.78:0/1228398491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55953c549f90 con 0x55953c54b480 2024-09-11T01:13:42.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.203+0000 7fc230d2b640 1 --2- 172.21.15.78:0/1228398491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c54a240 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0e37d0 tx=0x7fc21c0cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40cbf47d1948cd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.206+0000 7fc216ffd640 1 -- 172.21.15.78:0/1228398491 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0a8da0 con 0x55953c54b480 2024-09-11T01:13:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.207+0000 7fc216ffd640 1 -- 172.21.15.78:0/1228398491 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc21c01bec0 con 0x55953c54b480 2024-09-11T01:13:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.207+0000 7fc216ffd640 1 -- 172.21.15.78:0/1228398491 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0a8f40 con 0x55953c54b480 2024-09-11T01:13:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.209+0000 7fc235b65740 1 -- 172.21.15.78:0/1228398491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 msgr2=0x55953c54a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.209+0000 7fc235b65740 1 --2- 172.21.15.78:0/1228398491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c54a240 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0e37d0 tx=0x7fc21c0cf040 comp rx=0 tx=0).stop 2024-09-11T01:13:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.209+0000 7fc235b65740 1 -- 172.21.15.78:0/1228398491 shutdown_connections 2024-09-11T01:13:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.209+0000 7fc235b65740 1 --2- 172.21.15.78:0/1228398491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c54bda0 0x55953c54a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.209+0000 7fc235b65740 1 --2- 172.21.15.78:0/1228398491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c54a240 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.209+0000 7fc235b65740 1 --2- 172.21.15.78:0/1228398491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c5492b0 0x55953c575680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.209+0000 7fc235b65740 1 -- 172.21.15.78:0/1228398491 >> 172.21.15.78:0/1228398491 conn(0x55953c55fcb0 msgr2=0x55953c553920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:42.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.212+0000 7fc235b65740 1 -- 172.21.15.78:0/1228398491 shutdown_connections 2024-09-11T01:13:42.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.212+0000 7fc235b65740 1 -- 172.21.15.78:0/1228398491 wait complete. 2024-09-11T01:13:42.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.213+0000 7fc235b65740 1 -- start start 2024-09-11T01:13:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.213+0000 7fc235b65740 1 Processor -- start 2024-09-11T01:13:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.213+0000 7fc235b65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c5492b0 0x55953c56db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.213+0000 7fc235b65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c56e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.214+0000 7fc235b65740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c56a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.214+0000 7fc235b65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c1adb70 con 0x55953c54b480 2024-09-11T01:13:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.214+0000 7fc235b65740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c1ad3e0 con 0x55953c54bda0 2024-09-11T01:13:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.214+0000 7fc235b65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c1aa580 con 0x55953c5492b0 2024-09-11T01:13:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.214+0000 7fc235b65740 10 client.0 _ll_register_callbacks cb 0x7fc224009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.214+0000 7fc231d2d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c56a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.214+0000 7fc231d2d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c56a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:56266/0 (socket says 172.21.15.78:56266) 2024-09-11T01:13:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.214+0000 7fc231d2d640 1 -- 172.21.15.78:0/1761671683 learned_addr learned my addr 172.21.15.78:0/1761671683 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.215+0000 7fc230d2b640 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c56e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.215+0000 7fc23152c640 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c5492b0 0x55953c56db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.217+0000 7fc231d2d640 1 -- 172.21.15.78:0/1761671683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c5492b0 msgr2=0x55953c56db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:42.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.217+0000 7fc231d2d640 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c5492b0 0x55953c56db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.217+0000 7fc231d2d640 1 -- 172.21.15.78:0/1761671683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 msgr2=0x55953c56e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:42.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.217+0000 7fc231d2d640 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c56e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.217+0000 7fc231d2d640 1 -- 172.21.15.78:0/1761671683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55953c56aa40 con 0x55953c54bda0 2024-09-11T01:13:42.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.221+0000 7fc231d2d640 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c56a210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc22401e010 tx=0x7fc22401b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a15256d1443cbe14 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:42.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.221+0000 7fc214ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:13:42.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.223+0000 7fc214ff9640 1 -- 172.21.15.78:0/1761671683 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22401c460 con 0x55953c54bda0 2024-09-11T01:13:42.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.223+0000 7fc214ff9640 1 -- 172.21.15.78:0/1761671683 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7fc22400da90 con 0x55953c54bda0 2024-09-11T01:13:42.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.224+0000 7fc235b65740 1 -- 172.21.15.78:0/1761671683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc21c01bec0 con 0x55953c54bda0 2024-09-11T01:13:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.226+0000 7fc214ff9640 1 -- 172.21.15.78:0/1761671683 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fc22400dc90 con 0x55953c54bda0 2024-09-11T01:13:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.229+0000 7fc235b65740 10 client.27377 fetch_fsmap learned FSMap version 59 2024-09-11T01:13:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.229+0000 7fc235b65740 1 -- 172.21.15.78:0/1761671683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55953c5f6e60 con 0x55953c54bda0 2024-09-11T01:13:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.234+0000 7fc214ff9640 1 -- 172.21.15.78:0/1761671683 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc22402b070 con 0x55953c54bda0 2024-09-11T01:13:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.234+0000 7fc214ff9640 1 -- 172.21.15.78:0/1761671683 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7fc224005360 con 0x55953c54bda0 2024-09-11T01:13:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.236+0000 7fc235b65740 10 client.27377 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:13:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.236+0000 7fc235b65740 10 client.27377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:13:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.236+0000 7fc235b65740 20 client.27377 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.236+0000 7fc235b65740 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55953c560540 0x55953c5f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.236+0000 7fc235b65740 4 client.27377 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:13:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.236+0000 7fc235b65740 1 -- 172.21.15.78:0/1761671683 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc21c01bec0 con 0x55953c560540 2024-09-11T01:13:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.238+0000 7fc23152c640 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55953c560540 0x55953c5f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.239+0000 7fc214ff9640 10 client.27377 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.239+0000 7fc23152c640 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55953c560540 0x55953c5f96c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9dd43e9d3e0575 server_cookie=f1b9871acfc7a8e4 in_seq=0 out_seq=0 2024-09-11T01:13:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.248+0000 7fc214ff9640 1 -- 172.21.15.78:0/1761671683 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x7fc21c01bec0 con 0x55953c560540 2024-09-11T01:13:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.248+0000 7fc214ff9640 10 client.27377 handle_command_reply: tid=0 2024-09-11T01:13:42.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.249+0000 7fc235b65740 1 client.27377 shutdown 2024-09-11T01:13:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.252+0000 7fc235b65740 1 -- 172.21.15.78:0/1761671683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 msgr2=0x55953c56a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.252+0000 7fc235b65740 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c56a210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc22401e010 tx=0x7fc22401b5d0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.254+0000 7fc235b65740 1 -- 172.21.15.78:0/1761671683 shutdown_connections 2024-09-11T01:13:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.254+0000 7fc235b65740 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55953c54bda0 0x55953c56a210 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.254+0000 7fc235b65740 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55953c560540 0x55953c5f96c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.254+0000 7fc235b65740 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55953c54b480 0x55953c56e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.254+0000 7fc235b65740 1 --2- 172.21.15.78:0/1761671683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55953c5492b0 0x55953c56db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.254+0000 7fc235b65740 1 -- 172.21.15.78:0/1761671683 >> 172.21.15.78:0/1761671683 conn(0x55953c55fcb0 msgr2=0x55953c54a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:42.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.254+0000 7fc214ff9640 0 client.27377 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:42.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.254+0000 7fc235b65740 1 -- 172.21.15.78:0/1761671683 shutdown_connections 2024-09-11T01:13:42.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.254+0000 7fc235b65740 1 -- 172.21.15.78:0/1761671683 wait complete. 2024-09-11T01:13:42.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.254+0000 7fc235b65740 20 client.27377 trim_cache size 0 max 16384 2024-09-11T01:13:42.257 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:13:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:13:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:13:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:13:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:13:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:13:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:13:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:13:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:13:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:13:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:13:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:13:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:13:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:13:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:13:42.269 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:13:42.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.255+0000 7fc233790640 1 -- 172.21.15.78:0/199054788 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc2180b5500 msgr2=0x7fc2180b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:42.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.255+0000 7fc233790640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc2180b5500 0x7fc2180b79c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc224005fd0 tx=0x7fc224009500 comp rx=0 tx=0).stop 2024-09-11T01:13:42.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.255+0000 7fc233790640 1 -- 172.21.15.78:0/199054788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc22c1076a0 msgr2=0x7fc22c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:42.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.255+0000 7fc233790640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc22c1076a0 0x7fc22c1986e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc21c005fd0 tx=0x7fc21c009f60 comp rx=0 tx=0).stop 2024-09-11T01:13:42.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.256+0000 7fc233790640 1 -- 172.21.15.78:0/199054788 shutdown_connections 2024-09-11T01:13:42.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.256+0000 7fc233790640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc2180b5500 0x7fc2180b79c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.256+0000 7fc233790640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22c108020 0x7fc22c19da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.256+0000 7fc233790640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc22c1076a0 0x7fc22c1986e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.256+0000 7fc233790640 1 --2- 172.21.15.78:0/199054788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c101c90 0x7fc22c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:42.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.256+0000 7fc233790640 1 -- 172.21.15.78:0/199054788 >> 172.21.15.78:0/199054788 conn(0x7fc22c0fd700 msgr2=0x7fc22c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:42.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.256+0000 7fc233790640 1 -- 172.21.15.78:0/199054788 shutdown_connections 2024-09-11T01:13:42.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:42.256+0000 7fc233790640 1 -- 172.21.15.78:0/199054788 wait complete. 2024-09-11T01:13:42.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 70374 36.76 MB/sec warmup 108 sec latency 756.218 ms 2024-09-11T01:13:43.396 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:13:43.396 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:13:43.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:43 smithi078 ceph-mon[31315]: pgmap v784: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.0 MiB/s wr, 682 op/s 2024-09-11T01:13:43.779 INFO:tasks.workunit.client.0.smithi078.stdout: 10 70651 36.51 MB/sec warmup 109 sec latency 831.478 ms 2024-09-11T01:13:43.858 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:13:43.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:43 smithi110 ceph-mon[34712]: pgmap v784: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.0 MiB/s wr, 682 op/s 2024-09-11T01:13:43.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:43 smithi112 ceph-mon[35384]: pgmap v784: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.0 MiB/s wr, 682 op/s 2024-09-11T01:13:44.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.451+0000 7f8b55993640 1 -- 172.21.15.78:0/2673194188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50101830 msgr2=0x7f8b50101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:44.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.451+0000 7f8b55993640 1 --2- 172.21.15.78:0/2673194188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50101830 0x7f8b50101c10 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009900 tx=0x7f8b4002a480 comp rx=0 tx=0).stop 2024-09-11T01:13:44.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.451+0000 7f8b55993640 1 -- 172.21.15.78:0/2673194188 shutdown_connections 2024-09-11T01:13:44.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.451+0000 7f8b55993640 1 --2- 172.21.15.78:0/2673194188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5010f3e0 0x7f8b501117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:44.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.451+0000 7f8b55993640 1 --2- 172.21.15.78:0/2673194188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b50102150 0x7f8b5010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:44.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.451+0000 7f8b55993640 1 --2- 172.21.15.78:0/2673194188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50101830 0x7f8b50101c10 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:44.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.451+0000 7f8b55993640 1 -- 172.21.15.78:0/2673194188 >> 172.21.15.78:0/2673194188 conn(0x7f8b500fd700 msgr2=0x7f8b500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:44.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.451+0000 7f8b55993640 1 -- 172.21.15.78:0/2673194188 shutdown_connections 2024-09-11T01:13:44.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.451+0000 7f8b55993640 1 -- 172.21.15.78:0/2673194188 wait complete. 2024-09-11T01:13:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.452+0000 7f8b55993640 1 Processor -- start 2024-09-11T01:13:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.452+0000 7f8b55993640 1 -- start start 2024-09-11T01:13:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.453+0000 7f8b55993640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b50101830 0x7f8b5019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.453+0000 7f8b55993640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50102150 0x7f8b5019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.453+0000 7f8b55993640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5010f3e0 0x7f8b50199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.453+0000 7f8b4e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50102150 0x7f8b5019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.453+0000 7f8b4effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b50101830 0x7f8b5019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.453+0000 7f8b4effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b50101830 0x7f8b5019a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36152/0 (socket says 172.21.15.78:36152) 2024-09-11T01:13:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.453+0000 7f8b4effd640 1 -- 172.21.15.78:0/3107009967 learned_addr learned my addr 172.21.15.78:0/3107009967 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.453+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50106280 con 0x7f8b50102150 2024-09-11T01:13:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.453+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50106100 con 0x7f8b50101830 2024-09-11T01:13:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.453+0000 7f8b4f7fe640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5010f3e0 0x7f8b50199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.453+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50106400 con 0x7f8b5010f3e0 2024-09-11T01:13:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.454+0000 7f8b4effd640 1 -- 172.21.15.78:0/3107009967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5010f3e0 msgr2=0x7f8b50199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.454+0000 7f8b4effd640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5010f3e0 0x7f8b50199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.454+0000 7f8b4effd640 1 -- 172.21.15.78:0/3107009967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50102150 msgr2=0x7f8b5019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.454+0000 7f8b4effd640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50102150 0x7f8b5019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.454+0000 7f8b4effd640 1 -- 172.21.15.78:0/3107009967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b50199ef0 con 0x7f8b50101830 2024-09-11T01:13:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.454+0000 7f8b4effd640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b50101830 0x7f8b5019a920 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b40005940 tx=0x7f8b40004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.455+0000 7f8b54991640 1 -- 172.21.15.78:0/3107009967 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4003f030 con 0x7f8b50101830 2024-09-11T01:13:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.455+0000 7f8b54991640 1 -- 172.21.15.78:0/3107009967 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b40003aa0 con 0x7f8b50101830 2024-09-11T01:13:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.455+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5019a180 con 0x7f8b50101830 2024-09-11T01:13:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.455+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b501a3ac0 con 0x7f8b50101830 2024-09-11T01:13:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.455+0000 7f8b54991640 1 -- 172.21.15.78:0/3107009967 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40035070 con 0x7f8b50101830 2024-09-11T01:13:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.456+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b3c0052c0 con 0x7f8b50101830 2024-09-11T01:13:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.458+0000 7f8b54991640 1 -- 172.21.15.78:0/3107009967 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8b40003c50 con 0x7f8b50101830 2024-09-11T01:13:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.458+0000 7f8b54991640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8b380b5620 0x7f8b380b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.459+0000 7f8b54991640 1 -- 172.21.15.78:0/3107009967 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f8b401068d0 con 0x7f8b50101830 2024-09-11T01:13:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.459+0000 7f8b4e7fc640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8b380b5620 0x7f8b380b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.459+0000 7f8b4e7fc640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8b380b5620 0x7f8b380b7ae0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8b500ffdc0 tx=0x7f8b4400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:44.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.465+0000 7f8b54991640 1 -- 172.21.15.78:0/3107009967 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8b40049030 con 0x7f8b50101830 2024-09-11T01:13:44.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 71462 36.55 MB/sec warmup 110 sec latency 54.838 ms 2024-09-11T01:13:44.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.828+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b3c0055b0 con 0x7f8b50101830 2024-09-11T01:13:44.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.829+0000 7f8b54991640 1 -- 172.21.15.78:0/3107009967 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+12014 (secure 0 0 0) 0x7f8b400cd4b0 con 0x7f8b50101830 2024-09-11T01:13:44.831 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:13:44.831 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":59,"btime":"2024-09-11T01:12:22:445035+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":59,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:12:22.122017+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T01:13:44.831 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 59 2024-09-11T01:13:44.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.833+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8b380b5620 msgr2=0x7f8b380b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:44.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.833+0000 7f8b55993640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8b380b5620 0x7f8b380b7ae0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8b500ffdc0 tx=0x7f8b4400b390 comp rx=0 tx=0).stop 2024-09-11T01:13:44.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.833+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b50101830 msgr2=0x7f8b5019a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:44.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.833+0000 7f8b55993640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b50101830 0x7f8b5019a920 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b40005940 tx=0x7f8b40004270 comp rx=0 tx=0).stop 2024-09-11T01:13:44.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.833+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 shutdown_connections 2024-09-11T01:13:44.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.833+0000 7f8b55993640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8b380b5620 0x7f8b380b7ae0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:44.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.833+0000 7f8b55993640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5010f3e0 0x7f8b50199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:44.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.833+0000 7f8b55993640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50102150 0x7f8b5019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:44.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.833+0000 7f8b55993640 1 --2- 172.21.15.78:0/3107009967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b50101830 0x7f8b5019a920 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b40005940 tx=0x7f8b40004270 comp rx=0 tx=0).stop 2024-09-11T01:13:44.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.833+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 >> 172.21.15.78:0/3107009967 conn(0x7f8b500fd700 msgr2=0x7f8b50110730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:44.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.834+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 shutdown_connections 2024-09-11T01:13:44.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:44.835+0000 7f8b55993640 1 -- 172.21.15.78:0/3107009967 wait complete. 2024-09-11T01:13:45.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 72489 36.76 MB/sec warmup 111 sec latency 203.265 ms 2024-09-11T01:13:45.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:45 smithi110 ceph-mon[34712]: pgmap v785: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 670 op/s 2024-09-11T01:13:45.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:45 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3107009967' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:13:45.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:45 smithi112 ceph-mon[35384]: pgmap v785: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 670 op/s 2024-09-11T01:13:45.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:45 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3107009967' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:13:45.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:45 smithi078 ceph-mon[31315]: pgmap v785: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 670 op/s 2024-09-11T01:13:45.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:45 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3107009967' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:13:46.088 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:13:46.370 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:13:46.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 73263 36.89 MB/sec warmup 112 sec latency 104.239 ms 2024-09-11T01:13:46.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.910+0000 7f0e634bd640 1 -- 172.21.15.78:0/2132067050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e5c103ca0 msgr2=0x7f0e5c104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:46.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.910+0000 7f0e634bd640 1 --2- 172.21.15.78:0/2132067050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e5c103ca0 0x7f0e5c104080 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009900 tx=0x7f0e5802a3f0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.910+0000 7f0e634bd640 1 -- 172.21.15.78:0/2132067050 shutdown_connections 2024-09-11T01:13:46.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.910+0000 7f0e634bd640 1 --2- 172.21.15.78:0/2132067050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e5c10d100 0x7f0e5c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.910+0000 7f0e634bd640 1 --2- 172.21.15.78:0/2132067050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e5c1045c0 0x7f0e5c10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.910+0000 7f0e634bd640 1 --2- 172.21.15.78:0/2132067050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e5c103ca0 0x7f0e5c104080 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.910+0000 7f0e634bd640 1 -- 172.21.15.78:0/2132067050 >> 172.21.15.78:0/2132067050 conn(0x7f0e5c0fd700 msgr2=0x7f0e5c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.910+0000 7f0e634bd640 1 -- 172.21.15.78:0/2132067050 shutdown_connections 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.910+0000 7f0e634bd640 1 -- 172.21.15.78:0/2132067050 wait complete. 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.911+0000 7f0e634bd640 1 Processor -- start 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.911+0000 7f0e634bd640 1 -- start start 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.911+0000 7f0e634bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e5c103ca0 0x7f0e5c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.911+0000 7f0e634bd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e5c1045c0 0x7f0e5c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.911+0000 7f0e634bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e5c10d100 0x7f0e5c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.911+0000 7f0e634bd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c112d70 con 0x7f0e5c103ca0 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.911+0000 7f0e634bd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c112bf0 con 0x7f0e5c1045c0 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.911+0000 7f0e634bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c112ef0 con 0x7f0e5c10d100 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.911+0000 7f0e61259640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e5c103ca0 0x7f0e5c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.911+0000 7f0e61259640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e5c103ca0 0x7f0e5c199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46094/0 (socket says 172.21.15.78:46094) 2024-09-11T01:13:46.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.911+0000 7f0e61259640 1 -- 172.21.15.78:0/2029383803 learned_addr learned my addr 172.21.15.78:0/2029383803 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:46.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.911+0000 7f0e60a58640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e5c1045c0 0x7f0e5c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.912+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e5c10d100 0x7f0e5c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.912+0000 7f0e61259640 1 -- 172.21.15.78:0/2029383803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e5c10d100 msgr2=0x7f0e5c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:46.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.912+0000 7f0e61259640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e5c10d100 0x7f0e5c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.912+0000 7f0e61259640 1 -- 172.21.15.78:0/2029383803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e5c1045c0 msgr2=0x7f0e5c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:46.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.912+0000 7f0e61259640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e5c1045c0 0x7f0e5c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.912+0000 7f0e61259640 1 -- 172.21.15.78:0/2029383803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5c19f330 con 0x7f0e5c103ca0 2024-09-11T01:13:46.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.912+0000 7f0e60a58640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e5c1045c0 0x7f0e5c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:46.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.913+0000 7f0e61259640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e5c103ca0 0x7f0e5c199280 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f0e58005940 tx=0x7f0e580040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:46.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.913+0000 7f0e527fc640 1 -- 172.21.15.78:0/2029383803 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5803f030 con 0x7f0e5c103ca0 2024-09-11T01:13:46.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.913+0000 7f0e527fc640 1 -- 172.21.15.78:0/2029383803 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e580044c0 con 0x7f0e5c103ca0 2024-09-11T01:13:46.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.913+0000 7f0e527fc640 1 -- 172.21.15.78:0/2029383803 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58035070 con 0x7f0e5c103ca0 2024-09-11T01:13:46.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.914+0000 7f0e634bd640 1 -- 172.21.15.78:0/2029383803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5c19f5c0 con 0x7f0e5c103ca0 2024-09-11T01:13:46.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.914+0000 7f0e634bd640 1 -- 172.21.15.78:0/2029383803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e5c19f960 con 0x7f0e5c103ca0 2024-09-11T01:13:46.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.918+0000 7f0e527fc640 1 -- 172.21.15.78:0/2029383803 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0e5803ba20 con 0x7f0e5c103ca0 2024-09-11T01:13:46.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.918+0000 7f0e65892740 1 Processor -- start 2024-09-11T01:13:46.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.918+0000 7f0e65892740 1 -- start start 2024-09-11T01:13:46.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.918+0000 7f0e65892740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b08538e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.918+0000 7f0e65892740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b08539a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e65892740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b08539d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e65892740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b085304ea0 con 0x55b08538edf0 2024-09-11T01:13:46.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e65892740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b0852f42d0 con 0x55b08539ad80 2024-09-11T01:13:46.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e65892740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b0852f80e0 con 0x55b08538e4d0 2024-09-11T01:13:46.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e61a5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b08538e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e61a5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b08538e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:48300/0 (socket says 172.21.15.78:48300) 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e61a5a640 1 -- 172.21.15.78:0/3508025659 learned_addr learned my addr 172.21.15.78:0/3508025659 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e60a58640 1 --2- 172.21.15.78:0/3508025659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b08539d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e61259640 1 --2- 172.21.15.78:0/3508025659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b08539a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e61a5a640 1 -- 172.21.15.78:0/3508025659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 msgr2=0x55b08539d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/3508025659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b08539d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e61a5a640 1 -- 172.21.15.78:0/3508025659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 msgr2=0x55b08539a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/3508025659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b08539a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e61a5a640 1 -- 172.21.15.78:0/3508025659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b085052650 con 0x55b08538e4d0 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e61259640 1 --2- 172.21.15.78:0/3508025659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b08539a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e527fc640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0e480b55a0 0x7f0e480b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/3508025659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b08538e8b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009fd0 tx=0x7f0e540118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=146fcaae1d07f6e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e61259640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0e480b55a0 0x7f0e480b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.919+0000 7f0e477fe640 1 -- 172.21.15.78:0/3508025659 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54004d20 con 0x55b08538e4d0 2024-09-11T01:13:46.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.920+0000 7f0e477fe640 1 -- 172.21.15.78:0/3508025659 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e54004050 con 0x55b08538e4d0 2024-09-11T01:13:46.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.920+0000 7f0e477fe640 1 -- 172.21.15.78:0/3508025659 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54005220 con 0x55b08538e4d0 2024-09-11T01:13:46.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.920+0000 7f0e527fc640 1 -- 172.21.15.78:0/2029383803 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f0e581064c0 con 0x7f0e5c103ca0 2024-09-11T01:13:46.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.921+0000 7f0e65892740 1 -- 172.21.15.78:0/3508025659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 msgr2=0x55b08538e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:46.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.921+0000 7f0e65892740 1 --2- 172.21.15.78:0/3508025659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b08538e8b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009fd0 tx=0x7f0e540118d0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.921+0000 7f0e61259640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0e480b55a0 0x7f0e480b7a60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0e58103d00 tx=0x7f0e5803b460 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:46.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.921+0000 7f0e65892740 1 -- 172.21.15.78:0/3508025659 shutdown_connections 2024-09-11T01:13:46.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.921+0000 7f0e65892740 1 --2- 172.21.15.78:0/3508025659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b08539d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.921+0000 7f0e65892740 1 --2- 172.21.15.78:0/3508025659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b08539a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.921+0000 7f0e65892740 1 --2- 172.21.15.78:0/3508025659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b08538e8b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.921+0000 7f0e65892740 1 -- 172.21.15.78:0/3508025659 >> 172.21.15.78:0/3508025659 conn(0x55b08538c300 msgr2=0x55b08538d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.921+0000 7f0e65892740 1 -- 172.21.15.78:0/3508025659 shutdown_connections 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.921+0000 7f0e65892740 1 -- 172.21.15.78:0/3508025659 wait complete. 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.921+0000 7f0e65892740 1 -- start start 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.921+0000 7f0e65892740 1 Processor -- start 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.922+0000 7f0e65892740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b08539a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.922+0000 7f0e65892740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b085397ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.922+0000 7f0e65892740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b085398110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.922+0000 7f0e61a5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b08539a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.922+0000 7f0e61a5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b08539a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:48316/0 (socket says 172.21.15.78:48316) 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.922+0000 7f0e61a5a640 1 -- 172.21.15.78:0/54718356 learned_addr learned my addr 172.21.15.78:0/54718356 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:46.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.922+0000 7f0e65892740 1 -- 172.21.15.78:0/54718356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b085155700 con 0x55b08538edf0 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.922+0000 7f0e65892740 1 -- 172.21.15.78:0/54718356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b08528f160 con 0x55b08539ad80 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.922+0000 7f0e65892740 1 -- 172.21.15.78:0/54718356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b08539da00 con 0x55b08538e4d0 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.922+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b085398110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.922+0000 7f0e60a58640 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b085397ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.923+0000 7f0e61a5a640 1 -- 172.21.15.78:0/54718356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 msgr2=0x55b085398110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.923+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b085398110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.923+0000 7f0e61a5a640 1 -- 172.21.15.78:0/54718356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 msgr2=0x55b085397ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.923+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b085397ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.923+0000 7f0e61a5a640 1 -- 172.21.15.78:0/54718356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0852bcc90 con 0x55b08538e4d0 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.923+0000 7f0e60a58640 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b085397ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.923+0000 7f0e65892740 10 client.0 _ll_register_callbacks cb 0x55b0852a89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.923+0000 7f0e65892740 1 -- 172.21.15.78:0/54718356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e54004050 con 0x55b08538e4d0 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.924+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b08539a690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009b00 tx=0x7f0e54009a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc0d5abff14fedb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:46.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.924+0000 7f0e44ff9640 10 client.27346 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.924+0000 7f0e44ff9640 1 -- 172.21.15.78:0/54718356 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5403d020 con 0x55b08538e4d0 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.924+0000 7f0e44ff9640 1 -- 172.21.15.78:0/54718356 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f0e5400f4e0 con 0x55b08538e4d0 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.924+0000 7f0e44ff9640 1 -- 172.21.15.78:0/54718356 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f0e5400f6e0 con 0x55b08538e4d0 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.924+0000 7f0e65892740 10 client.27346 fetch_fsmap learned FSMap version 59 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.924+0000 7f0e65892740 1 -- 172.21.15.78:0/54718356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=59}) -- 0x55b0853a9150 con 0x55b08538e4d0 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.925+0000 7f0e44ff9640 1 -- 172.21.15.78:0/54718356 <== mon.1 v2:172.21.15.112:3300/0 4 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f0e54025050 con 0x55b08538e4d0 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.925+0000 7f0e65892740 10 client.27346 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.925+0000 7f0e65892740 10 client.27346 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.925+0000 7f0e65892740 20 client.27346 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.925+0000 7f0e65892740 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b0853aa500 0x55b0853ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.925+0000 7f0e65892740 4 client.27346 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.925+0000 7f0e65892740 1 -- 172.21.15.78:0/54718356 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b085306df0 con 0x55b0853aa500 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.925+0000 7f0e60a58640 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b0853aa500 0x55b0853ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.926+0000 7f0e44ff9640 10 client.27346 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:13:46.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.926+0000 7f0e60a58640 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b0853aa500 0x55b0853ac8f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f30b1ae3effc3d58 server_cookie=f87eaa9925edade7 in_seq=0 out_seq=0 2024-09-11T01:13:46.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.928+0000 7f0e44ff9640 1 -- 172.21.15.78:0/54718356 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b085306df0 con 0x55b0853aa500 2024-09-11T01:13:46.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.928+0000 7f0e44ff9640 10 client.27346 handle_command_reply: tid=0 2024-09-11T01:13:46.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.928+0000 7f0e65892740 1 client.27346 shutdown 2024-09-11T01:13:46.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.928+0000 7f0e65892740 1 -- 172.21.15.78:0/54718356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 msgr2=0x55b08539a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:46.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.928+0000 7f0e65892740 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b08539a690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009b00 tx=0x7f0e54009a50 comp rx=0 tx=0).stop 2024-09-11T01:13:46.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.928+0000 7f0e65892740 1 -- 172.21.15.78:0/54718356 shutdown_connections 2024-09-11T01:13:46.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.928+0000 7f0e65892740 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b0853aa500 0x55b0853ac8f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.928+0000 7f0e65892740 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b085398110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.928+0000 7f0e65892740 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b085397ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.928+0000 7f0e65892740 1 --2- 172.21.15.78:0/54718356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b08539a690 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.928+0000 7f0e65892740 1 -- 172.21.15.78:0/54718356 >> 172.21.15.78:0/54718356 conn(0x55b08538c300 msgr2=0x55b085391ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:46.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.928+0000 7f0e44ff9640 0 client.27346 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:13:46.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.929+0000 7f0e65892740 1 -- 172.21.15.78:0/54718356 shutdown_connections 2024-09-11T01:13:46.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.929+0000 7f0e65892740 1 -- 172.21.15.78:0/54718356 wait complete. 2024-09-11T01:13:46.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.929+0000 7f0e65892740 20 client.27346 trim_cache size 0 max 16384 2024-09-11T01:13:46.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.947+0000 7f0e65892740 1 Processor -- start 2024-09-11T01:13:46.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.947+0000 7f0e65892740 1 -- start start 2024-09-11T01:13:46.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.947+0000 7f0e65892740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538e4d0 0x55b08538bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e65892740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08538edf0 0x55b08539c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e65892740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08539ad80 0x55b08539c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e65892740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1ac330 con 0x55b08538e4d0 2024-09-11T01:13:46.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e65892740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1ae0b0 con 0x55b08538edf0 2024-09-11T01:13:46.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e65892740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1aca00 con 0x55b08539ad80 2024-09-11T01:13:46.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e61a5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08539ad80 0x55b08539c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e60a58640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08538edf0 0x55b08539c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e61a5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08539ad80 0x55b08539c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:48330/0 (socket says 172.21.15.78:48330) 2024-09-11T01:13:46.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e61a5a640 1 -- 172.21.15.78:0/4217267893 learned_addr learned my addr 172.21.15.78:0/4217267893 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:46.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/4217267893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538e4d0 0x55b08538bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e60a58640 1 -- 172.21.15.78:0/4217267893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08539ad80 msgr2=0x55b08539c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e60a58640 1 --2- 172.21.15.78:0/4217267893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08539ad80 0x55b08539c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e60a58640 1 -- 172.21.15.78:0/4217267893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538e4d0 msgr2=0x55b08538bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e60a58640 1 --2- 172.21.15.78:0/4217267893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538e4d0 0x55b08538bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.948+0000 7f0e60a58640 1 -- 172.21.15.78:0/4217267893 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b08538caa0 con 0x55b08538edf0 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.949+0000 7f0e60a58640 1 --2- 172.21.15.78:0/4217267893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08538edf0 0x55b08539c170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55b0853ad0c0 tx=0x7f0e4c012db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36484f3db1745ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.949+0000 7f0e46ffd640 1 -- 172.21.15.78:0/4217267893 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c012110 con 0x55b08538edf0 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.949+0000 7f0e46ffd640 1 -- 172.21.15.78:0/4217267893 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e4c01f040 con 0x55b08538edf0 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.949+0000 7f0e46ffd640 1 -- 172.21.15.78:0/4217267893 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0109e0 con 0x55b08538edf0 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.950+0000 7f0e65892740 1 -- 172.21.15.78:0/4217267893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08538edf0 msgr2=0x55b08539c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.950+0000 7f0e65892740 1 --2- 172.21.15.78:0/4217267893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08538edf0 0x55b08539c170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55b0853ad0c0 tx=0x7f0e4c012db0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.950+0000 7f0e65892740 1 -- 172.21.15.78:0/4217267893 shutdown_connections 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.950+0000 7f0e65892740 1 --2- 172.21.15.78:0/4217267893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08539ad80 0x55b08539c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.950+0000 7f0e65892740 1 --2- 172.21.15.78:0/4217267893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08538edf0 0x55b08539c170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.950+0000 7f0e65892740 1 --2- 172.21.15.78:0/4217267893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538e4d0 0x55b08538bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.950+0000 7f0e65892740 1 -- 172.21.15.78:0/4217267893 >> 172.21.15.78:0/4217267893 conn(0x55b08538c300 msgr2=0x55b085399930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:46.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.950+0000 7f0e65892740 1 -- 172.21.15.78:0/4217267893 shutdown_connections 2024-09-11T01:13:46.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.950+0000 7f0e65892740 1 -- 172.21.15.78:0/4217267893 wait complete. 2024-09-11T01:13:46.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.950+0000 7f0e65892740 1 -- start start 2024-09-11T01:13:46.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e65892740 1 Processor -- start 2024-09-11T01:13:46.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e65892740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b0853a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e65892740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b0853a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e65892740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b08539dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e65892740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1ad920 con 0x55b08538edf0 2024-09-11T01:13:46.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e65892740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1ad190 con 0x55b08539ad80 2024-09-11T01:13:46.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e65892740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1aa330 con 0x55b08538e4d0 2024-09-11T01:13:46.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e60a58640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b0853a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e61a5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b0853a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e60a58640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b0853a33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46120/0 (socket says 172.21.15.78:46120) 2024-09-11T01:13:46.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e60a58640 1 -- 172.21.15.78:0/1635041161 learned_addr learned my addr 172.21.15.78:0/1635041161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e61a5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b0853a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:48346/0 (socket says 172.21.15.78:48346) 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b08539dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.951+0000 7f0e65892740 10 client.0 _ll_register_callbacks cb 0x7f0e4c009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e60a58640 1 -- 172.21.15.78:0/1635041161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 msgr2=0x55b0853a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e60a58640 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b0853a2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e60a58640 1 -- 172.21.15.78:0/1635041161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 msgr2=0x55b08539dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e60a58640 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b08539dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e60a58640 1 -- 172.21.15.78:0/1635041161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b08539e390 con 0x55b08538edf0 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b08539dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e60a58640 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b0853a33c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c012340 tx=0x7f0e4c012370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75648b94627f9a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e44ff9640 1 -- 172.21.15.78:0/1635041161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c005250 con 0x55b08538edf0 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e65892740 1 -- 172.21.15.78:0/1635041161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e4c01f040 con 0x55b08538edf0 2024-09-11T01:13:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e44ff9640 1 -- 172.21.15.78:0/1635041161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f0e4c0053f0 con 0x55b08538edf0 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e44ff9640 1 -- 172.21.15.78:0/1635041161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f0e4c005630 con 0x55b08538edf0 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e65892740 10 client.18240 fetch_fsmap learned FSMap version 59 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e65892740 1 -- 172.21.15.78:0/1635041161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55b085417d80 con 0x55b08538edf0 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.952+0000 7f0e44ff9640 1 -- 172.21.15.78:0/1635041161 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e4c01f040 con 0x55b08538edf0 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.953+0000 7f0e44ff9640 1 -- 172.21.15.78:0/1635041161 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 59) ==== 8442+0+0 (secure 0 0 0) 0x7f0e4c022020 con 0x55b08538edf0 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.953+0000 7f0e65892740 10 client.18240 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.953+0000 7f0e65892740 10 client.18240 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.953+0000 7f0e65892740 20 client.18240 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.953+0000 7f0e65892740 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b085418490 0x55b08541a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.953+0000 7f0e65892740 4 client.18240 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.953+0000 7f0e65892740 1 -- 172.21.15.78:0/1635041161 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0853028a0 con 0x55b085418490 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.953+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b085418490 0x55b08541a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.953+0000 7f0e44ff9640 10 client.18240 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:13:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.953+0000 7f0e61a5a640 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b085418490 0x55b08541a8b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1728390746cfac13 server_cookie=516e0964589e7ead in_seq=0 out_seq=0 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e44ff9640 1 -- 172.21.15.78:0/1635041161 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0853028a0 con 0x55b085418490 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e44ff9640 10 client.18240 handle_command_reply: tid=0 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e65892740 1 client.18240 shutdown 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e65892740 1 -- 172.21.15.78:0/1635041161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 msgr2=0x55b0853a33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e65892740 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b0853a33c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c012340 tx=0x7f0e4c012370 comp rx=0 tx=0).stop 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e65892740 1 -- 172.21.15.78:0/1635041161 shutdown_connections 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e65892740 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55b085418490 0x55b08541a8b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e65892740 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b08539ad80 0x55b08539dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e65892740 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b08538edf0 0x55b0853a33c0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e65892740 1 --2- 172.21.15.78:0/1635041161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b08538e4d0 0x55b0853a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e65892740 1 -- 172.21.15.78:0/1635041161 >> 172.21.15.78:0/1635041161 conn(0x55b08538c300 msgr2=0x55b085391580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e44ff9640 0 client.18240 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:13:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e65892740 1 -- 172.21.15.78:0/1635041161 shutdown_connections 2024-09-11T01:13:46.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e65892740 1 -- 172.21.15.78:0/1635041161 wait complete. 2024-09-11T01:13:46.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.954+0000 7f0e65892740 20 client.18240 trim_cache size 0 max 16384 2024-09-11T01:13:46.961 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:13:46.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.955+0000 7f0e634bd640 1 -- 172.21.15.78:0/2029383803 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0e480b55a0 msgr2=0x7f0e480b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.955+0000 7f0e634bd640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0e480b55a0 0x7f0e480b7a60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0e58103d00 tx=0x7f0e5803b460 comp rx=0 tx=0).stop 2024-09-11T01:13:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.955+0000 7f0e634bd640 1 -- 172.21.15.78:0/2029383803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e5c103ca0 msgr2=0x7f0e5c199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.955+0000 7f0e634bd640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e5c103ca0 0x7f0e5c199280 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f0e58005940 tx=0x7f0e580040c0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.955+0000 7f0e634bd640 1 -- 172.21.15.78:0/2029383803 shutdown_connections 2024-09-11T01:13:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.955+0000 7f0e634bd640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0e480b55a0 0x7f0e480b7a60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.955+0000 7f0e634bd640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e5c10d100 0x7f0e5c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.955+0000 7f0e634bd640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e5c1045c0 0x7f0e5c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.955+0000 7f0e634bd640 1 --2- 172.21.15.78:0/2029383803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e5c103ca0 0x7f0e5c199280 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.955+0000 7f0e634bd640 1 -- 172.21.15.78:0/2029383803 >> 172.21.15.78:0/2029383803 conn(0x7f0e5c0fd700 msgr2=0x7f0e5c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.956+0000 7f0e634bd640 1 -- 172.21.15.78:0/2029383803 shutdown_connections 2024-09-11T01:13:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:46.956+0000 7f0e634bd640 1 -- 172.21.15.78:0/2029383803 wait complete. 2024-09-11T01:13:47.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 73833 36.77 MB/sec warmup 113 sec latency 85.190 ms 2024-09-11T01:13:47.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:47 smithi110 ceph-mon[34712]: pgmap v786: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 21 MiB/s wr, 937 op/s 2024-09-11T01:13:47.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:47 smithi112 ceph-mon[35384]: pgmap v786: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 21 MiB/s wr, 937 op/s 2024-09-11T01:13:47.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:47 smithi078 ceph-mon[31315]: pgmap v786: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 21 MiB/s wr, 937 op/s 2024-09-11T01:13:48.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 74710 36.76 MB/sec warmup 114 sec latency 117.356 ms 2024-09-11T01:13:49.592 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:13:49.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 75831 37.07 MB/sec warmup 115 sec latency 57.133 ms 2024-09-11T01:13:49.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:49 smithi110 ceph-mon[34712]: pgmap v787: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 924 op/s 2024-09-11T01:13:49.969 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:13:49.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:49 smithi112 ceph-mon[35384]: pgmap v787: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 924 op/s 2024-09-11T01:13:49.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:49 smithi078 ceph-mon[31315]: pgmap v787: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 924 op/s 2024-09-11T01:13:50.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:50 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:13:50.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 76584 37.15 MB/sec warmup 116 sec latency 96.595 ms 2024-09-11T01:13:50.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:50 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:13:50.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:50 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:13:51.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 77207 37.04 MB/sec warmup 117 sec latency 116.109 ms 2024-09-11T01:13:51.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.287+0000 7f6339530640 1 -- 172.21.15.78:0/3154923941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6334101c90 msgr2=0x7f6334102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.287+0000 7f6339530640 1 --2- 172.21.15.78:0/3154923941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6334101c90 0x7f6334102070 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6324009900 tx=0x7f632402a410 comp rx=0 tx=0).stop 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.287+0000 7f6339530640 1 -- 172.21.15.78:0/3154923941 shutdown_connections 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.287+0000 7f6339530640 1 --2- 172.21.15.78:0/3154923941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334108020 0x7f633410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.287+0000 7f6339530640 1 --2- 172.21.15.78:0/3154923941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63341076a0 0x7f6334107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.287+0000 7f6339530640 1 --2- 172.21.15.78:0/3154923941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6334101c90 0x7f6334102070 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.287+0000 7f6339530640 1 -- 172.21.15.78:0/3154923941 >> 172.21.15.78:0/3154923941 conn(0x7f63340fd700 msgr2=0x7f63340ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.287+0000 7f6339530640 1 -- 172.21.15.78:0/3154923941 shutdown_connections 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.287+0000 7f6339530640 1 -- 172.21.15.78:0/3154923941 wait complete. 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.288+0000 7f6339530640 1 Processor -- start 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.288+0000 7f6339530640 1 -- start start 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f6339530640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6334101c90 0x7f633419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f6339530640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63341076a0 0x7f633419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f6339530640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6334108020 0x7f63341a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f6339530640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6334111d30 con 0x7f6334101c90 2024-09-11T01:13:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f6339530640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6334111bb0 con 0x7f6334108020 2024-09-11T01:13:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f6339530640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6334111eb0 con 0x7f63341076a0 2024-09-11T01:13:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f6332ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6334101c90 0x7f633419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f63337fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6334108020 0x7f63341a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f6332ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6334101c90 0x7f633419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46142/0 (socket says 172.21.15.78:46142) 2024-09-11T01:13:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f6332ffd640 1 -- 172.21.15.78:0/1301480686 learned_addr learned my addr 172.21.15.78:0/1301480686 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f632bfff640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63341076a0 0x7f633419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f6332ffd640 1 -- 172.21.15.78:0/1301480686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63341076a0 msgr2=0x7f633419b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.289+0000 7f6332ffd640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63341076a0 0x7f633419b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.290+0000 7f6332ffd640 1 -- 172.21.15.78:0/1301480686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6334108020 msgr2=0x7f63341a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.290+0000 7f6332ffd640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6334108020 0x7f63341a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.290+0000 7f6332ffd640 1 -- 172.21.15.78:0/1301480686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63341a13f0 con 0x7f6334101c90 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.290+0000 7f6332ffd640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6334101c90 0x7f633419b340 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f6324009850 tx=0x7f6324030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.290+0000 7f6330ff9640 1 -- 172.21.15.78:0/1301480686 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632403f070 con 0x7f6334101c90 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.290+0000 7f6339530640 1 -- 172.21.15.78:0/1301480686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63341a1680 con 0x7f6334101c90 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.290+0000 7f6330ff9640 1 -- 172.21.15.78:0/1301480686 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6324004460 con 0x7f6334101c90 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.290+0000 7f6330ff9640 1 -- 172.21.15.78:0/1301480686 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324049060 con 0x7f6334101c90 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.291+0000 7f6339530640 1 -- 172.21.15.78:0/1301480686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63341a7da0 con 0x7f6334101c90 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.295+0000 7f633b905740 1 Processor -- start 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.295+0000 7f633b905740 1 -- start start 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.295+0000 7f633b905740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4242f90f0 0x55f4242f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.295+0000 7f633b905740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f9a10 0x55f424305460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.295+0000 7f633b905740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4243059a0 0x55f424307d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.295+0000 7f633b905740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f42426fea0 con 0x55f4242f90f0 2024-09-11T01:13:51.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.295+0000 7f633b905740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f42425f2d0 con 0x55f4242f9a10 2024-09-11T01:13:51.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.295+0000 7f633b905740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4242630e0 con 0x55f4243059a0 2024-09-11T01:13:51.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.296+0000 7f632bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4243059a0 0x55f424307d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.296+0000 7f632bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4243059a0 0x55f424307d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:48386/0 (socket says 172.21.15.78:48386) 2024-09-11T01:13:51.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.296+0000 7f632bfff640 1 -- 172.21.15.78:0/2583964024 learned_addr learned my addr 172.21.15.78:0/2583964024 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:51.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.297+0000 7f63337fe640 1 --2- 172.21.15.78:0/2583964024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4242f90f0 0x55f4242f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.298+0000 7f6330ff9640 1 -- 172.21.15.78:0/1301480686 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f632404c060 con 0x7f6334101c90 2024-09-11T01:13:51.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.298+0000 7f632bfff640 1 -- 172.21.15.78:0/2583964024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f9a10 msgr2=0x55f424305460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:13:51.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.298+0000 7f632bfff640 1 --2- 172.21.15.78:0/2583964024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f9a10 0x55f424305460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.298+0000 7f632bfff640 1 -- 172.21.15.78:0/2583964024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4242f90f0 msgr2=0x55f4242f94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.298+0000 7f632bfff640 1 --2- 172.21.15.78:0/2583964024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4242f90f0 0x55f4242f94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.298+0000 7f632bfff640 1 -- 172.21.15.78:0/2583964024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f423fbd650 con 0x55f4243059a0 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.299+0000 7f632bfff640 1 --2- 172.21.15.78:0/2583964024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4243059a0 0x55f424307d90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f631c009c90 tx=0x7f631c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cee0fd5ceebc73f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.299+0000 7f6330ff9640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63200b54d0 0x7f63200b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.300+0000 7f6330ff9640 1 -- 172.21.15.78:0/1301480686 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f6324106630 con 0x7f6334101c90 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.300+0000 7f63297fa640 1 -- 172.21.15.78:0/2583964024 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c01e070 con 0x55f4243059a0 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.300+0000 7f63297fa640 1 -- 172.21.15.78:0/2583964024 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f631c019ae0 con 0x55f4243059a0 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.300+0000 7f63297fa640 1 -- 172.21.15.78:0/2583964024 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c00ea40 con 0x55f4243059a0 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.300+0000 7f63337fe640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63200b54d0 0x7f63200b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.301+0000 7f633b905740 1 -- 172.21.15.78:0/2583964024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4243059a0 msgr2=0x55f424307d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.301+0000 7f63337fe640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63200b54d0 0x7f63200b7990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f632c009fd0 tx=0x7f632c00e040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:51.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.301+0000 7f633b905740 1 --2- 172.21.15.78:0/2583964024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4243059a0 0x55f424307d90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f631c009c90 tx=0x7f631c011810 comp rx=0 tx=0).stop 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.301+0000 7f633b905740 1 -- 172.21.15.78:0/2583964024 shutdown_connections 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.301+0000 7f633b905740 1 --2- 172.21.15.78:0/2583964024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4243059a0 0x55f424307d90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.301+0000 7f633b905740 1 --2- 172.21.15.78:0/2583964024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f9a10 0x55f424305460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.301+0000 7f633b905740 1 --2- 172.21.15.78:0/2583964024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4242f90f0 0x55f4242f94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.301+0000 7f633b905740 1 -- 172.21.15.78:0/2583964024 >> 172.21.15.78:0/2583964024 conn(0x55f4242f6f20 msgr2=0x55f4242f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.302+0000 7f633b905740 1 -- 172.21.15.78:0/2583964024 shutdown_connections 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.302+0000 7f633b905740 1 -- 172.21.15.78:0/2583964024 wait complete. 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.302+0000 7f633b905740 1 -- start start 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.302+0000 7f633b905740 1 Processor -- start 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.303+0000 7f633b905740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f90f0 0x55f424305380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.303+0000 7f633b905740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f9a10 0x55f4243027d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.303+0000 7f633b905740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4243059a0 0x55f424302ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.303+0000 7f633b905740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f4240c0700 con 0x55f4243059a0 2024-09-11T01:13:51.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.303+0000 7f633b905740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f4241fa160 con 0x55f4242f90f0 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.303+0000 7f633b905740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f424308620 con 0x55f4242f9a10 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.303+0000 7f633b905740 10 client.0 _ll_register_callbacks cb 0x55f4242139e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.303+0000 7f6332ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f90f0 0x55f424305380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.303+0000 7f6332ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f90f0 0x55f424305380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36264/0 (socket says 172.21.15.78:36264) 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.303+0000 7f6332ffd640 1 -- 172.21.15.78:0/3448122838 learned_addr learned my addr 172.21.15.78:0/3448122838 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.303+0000 7f632bfff640 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f9a10 0x55f4243027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.304+0000 7f63337fe640 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4243059a0 0x55f424302ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.304+0000 7f63337fe640 1 -- 172.21.15.78:0/3448122838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f9a10 msgr2=0x55f4243027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.304+0000 7f63337fe640 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f9a10 0x55f4243027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.304+0000 7f63337fe640 1 -- 172.21.15.78:0/3448122838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f90f0 msgr2=0x55f424305380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.304+0000 7f63337fe640 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f90f0 0x55f424305380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.304+0000 7f63337fe640 1 -- 172.21.15.78:0/3448122838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f424227c90 con 0x55f4243059a0 2024-09-11T01:13:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.304+0000 7f63337fe640 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4243059a0 0x55f424302ec0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f632c009520 tx=0x7f632c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ef3da605b77ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.305+0000 7f631a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.305+0000 7f631a7fc640 1 -- 172.21.15.78:0/3448122838 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c028060 con 0x55f4243059a0 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.305+0000 7f633b905740 1 -- 172.21.15.78:0/3448122838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f631c019ae0 con 0x55f4243059a0 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.305+0000 7f631a7fc640 1 -- 172.21.15.78:0/3448122838 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f632c016c30 con 0x55f4243059a0 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.306+0000 7f631a7fc640 1 -- 172.21.15.78:0/3448122838 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f632c016e30 con 0x55f4243059a0 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.306+0000 7f633b905740 10 client.18252 fetch_fsmap learned FSMap version 60 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.306+0000 7f633b905740 1 -- 172.21.15.78:0/3448122838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55f42430b090 con 0x55f4243059a0 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.307+0000 7f631a7fc640 1 -- 172.21.15.78:0/3448122838 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f631c019ae0 con 0x55f4243059a0 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.307+0000 7f631a7fc640 1 -- 172.21.15.78:0/3448122838 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 60) ==== 8434+0+0 (secure 0 0 0) 0x7f632c016070 con 0x55f4243059a0 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.308+0000 7f633b905740 10 client.18252 fetch_fsmap finished waiting for FSMap version 60 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.308+0000 7f633b905740 10 client.18252 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.308+0000 7f633b905740 20 client.18252 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.308+0000 7f633b905740 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55f42430c5f0 0x55f42430ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.308+0000 7f633b905740 4 client.18252 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.308+0000 7f633b905740 1 -- 172.21.15.78:0/3448122838 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f424271df0 con 0x55f42430c5f0 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.308+0000 7f6332ffd640 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55f42430c5f0 0x55f42430ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.309+0000 7f6332ffd640 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55f42430c5f0 0x55f42430ea30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1babff89837b9848 server_cookie=a2b37fdf6254c78a in_seq=0 out_seq=0 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.309+0000 7f631a7fc640 10 client.18252 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.310+0000 7f631a7fc640 1 -- 172.21.15.78:0/3448122838 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f424271df0 con 0x55f42430c5f0 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.310+0000 7f631a7fc640 10 client.18252 handle_command_reply: tid=0 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.310+0000 7f633b905740 1 client.18252 shutdown 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.311+0000 7f633b905740 1 -- 172.21.15.78:0/3448122838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4243059a0 msgr2=0x55f424302ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.311+0000 7f633b905740 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4243059a0 0x55f424302ec0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f632c009520 tx=0x7f632c004a40 comp rx=0 tx=0).stop 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.311+0000 7f633b905740 1 -- 172.21.15.78:0/3448122838 shutdown_connections 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.311+0000 7f633b905740 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55f42430c5f0 0x55f42430ea30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.311+0000 7f633b905740 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4243059a0 0x55f424302ec0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.311+0000 7f633b905740 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f9a10 0x55f4243027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.311+0000 7f633b905740 1 --2- 172.21.15.78:0/3448122838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f90f0 0x55f424305380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.311+0000 7f633b905740 1 -- 172.21.15.78:0/3448122838 >> 172.21.15.78:0/3448122838 conn(0x55f4242f6f20 msgr2=0x55f4242fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.311+0000 7f631a7fc640 0 client.18252 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:13:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.311+0000 7f633b905740 1 -- 172.21.15.78:0/3448122838 shutdown_connections 2024-09-11T01:13:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.311+0000 7f633b905740 1 -- 172.21.15.78:0/3448122838 wait complete. 2024-09-11T01:13:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.311+0000 7f633b905740 20 client.18252 trim_cache size 0 max 16384 2024-09-11T01:13:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.339+0000 7f633b905740 1 Processor -- start 2024-09-11T01:13:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.339+0000 7f633b905740 1 -- start start 2024-09-11T01:13:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.339+0000 7f633b905740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4242f90f0 0x55f4242f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.339+0000 7f633b905740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f9a10 0x55f4242f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.339+0000 7f633b905740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4243059a0 0x55f4243192b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.339+0000 7f633b905740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63341ac570 con 0x55f4242f90f0 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.339+0000 7f633b905740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f63341ae2f0 con 0x55f4243059a0 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.339+0000 7f633b905740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63341acc40 con 0x55f4242f9a10 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.340+0000 7f632bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f9a10 0x55f4242f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.340+0000 7f6332ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4242f90f0 0x55f4242f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.340+0000 7f632bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f9a10 0x55f4242f83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:48404/0 (socket says 172.21.15.78:48404) 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.340+0000 7f632bfff640 1 -- 172.21.15.78:0/3572089548 learned_addr learned my addr 172.21.15.78:0/3572089548 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.340+0000 7f63337fe640 1 --2- 172.21.15.78:0/3572089548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4243059a0 0x55f4243192b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.340+0000 7f632bfff640 1 -- 172.21.15.78:0/3572089548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4243059a0 msgr2=0x55f4243192b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.340+0000 7f632bfff640 1 --2- 172.21.15.78:0/3572089548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4243059a0 0x55f4243192b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.340+0000 7f632bfff640 1 -- 172.21.15.78:0/3572089548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4242f90f0 msgr2=0x55f4242f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.340+0000 7f632bfff640 1 --2- 172.21.15.78:0/3572089548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4242f90f0 0x55f4242f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.340+0000 7f632bfff640 1 -- 172.21.15.78:0/3572089548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4242f79a0 con 0x55f4242f9a10 2024-09-11T01:13:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.340+0000 7f632bfff640 1 --2- 172.21.15.78:0/3572089548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f9a10 0x55f4242f83f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f631c009a50 tx=0x7f631c00e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbbe9824ccf6be0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.340+0000 7f6328ff9640 1 -- 172.21.15.78:0/3572089548 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c028020 con 0x55f4242f9a10 2024-09-11T01:13:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.341+0000 7f6328ff9640 1 -- 172.21.15.78:0/3572089548 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f631c00e3b0 con 0x55f4242f9a10 2024-09-11T01:13:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.341+0000 7f6328ff9640 1 -- 172.21.15.78:0/3572089548 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c01e040 con 0x55f4242f9a10 2024-09-11T01:13:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.342+0000 7f633b905740 1 -- 172.21.15.78:0/3572089548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f9a10 msgr2=0x55f4242f83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.342+0000 7f633b905740 1 --2- 172.21.15.78:0/3572089548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f9a10 0x55f4242f83f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f631c009a50 tx=0x7f631c00e5a0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.342+0000 7f633b905740 1 -- 172.21.15.78:0/3572089548 shutdown_connections 2024-09-11T01:13:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.342+0000 7f633b905740 1 --2- 172.21.15.78:0/3572089548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4243059a0 0x55f4243192b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.342+0000 7f633b905740 1 --2- 172.21.15.78:0/3572089548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f9a10 0x55f4242f83f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.342+0000 7f633b905740 1 --2- 172.21.15.78:0/3572089548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4242f90f0 0x55f4242f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.342+0000 7f633b905740 1 -- 172.21.15.78:0/3572089548 >> 172.21.15.78:0/3572089548 conn(0x55f4242f6f20 msgr2=0x55f4242f7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:51.883 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:51 smithi078 ceph-mon[31315]: pgmap v788: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-09-11T01:13:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.342+0000 7f633b905740 1 -- 172.21.15.78:0/3572089548 shutdown_connections 2024-09-11T01:13:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.342+0000 7f633b905740 1 -- 172.21.15.78:0/3572089548 wait complete. 2024-09-11T01:13:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.343+0000 7f633b905740 1 -- start start 2024-09-11T01:13:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.343+0000 7f633b905740 1 Processor -- start 2024-09-11T01:13:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.343+0000 7f633b905740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f90f0 0x55f424314320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.343+0000 7f633b905740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f9a10 0x55f424314860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.343+0000 7f6332ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f90f0 0x55f424314320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.343+0000 7f6332ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f90f0 0x55f424314320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:48418/0 (socket says 172.21.15.78:48418) 2024-09-11T01:13:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.343+0000 7f6332ffd640 1 -- 172.21.15.78:0/1457158998 learned_addr learned my addr 172.21.15.78:0/1457158998 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.344+0000 7f632bfff640 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f9a10 0x55f424314860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.344+0000 7f633b905740 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4243059a0 0x55f42430f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.344+0000 7f633b905740 1 -- 172.21.15.78:0/1457158998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63341adb60 con 0x55f4243059a0 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.344+0000 7f633b905740 1 -- 172.21.15.78:0/1457158998 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f63341ad3d0 con 0x55f4242f9a10 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.344+0000 7f633b905740 1 -- 172.21.15.78:0/1457158998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63341aa5d0 con 0x55f4242f90f0 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.344+0000 7f633b905740 10 client.0 _ll_register_callbacks cb 0x7f632c009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.344+0000 7f632bfff640 1 -- 172.21.15.78:0/1457158998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f90f0 msgr2=0x55f424314320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.344+0000 7f632bfff640 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f90f0 0x55f424314320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.344+0000 7f632bfff640 1 -- 172.21.15.78:0/1457158998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4243059a0 msgr2=0x55f42430f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.344+0000 7f632bfff640 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4243059a0 0x55f42430f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.344+0000 7f632bfff640 1 -- 172.21.15.78:0/1457158998 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f42430f810 con 0x55f4242f9a10 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.345+0000 7f632bfff640 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f9a10 0x55f424314860 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f631c009a50 tx=0x7f631c009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28993eb5a763ade8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.345+0000 7f631a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.345+0000 7f631a7fc640 1 -- 172.21.15.78:0/1457158998 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c01e040 con 0x55f4242f9a10 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.345+0000 7f631a7fc640 1 -- 172.21.15.78:0/1457158998 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f631c00ea20 con 0x55f4242f9a10 2024-09-11T01:13:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.345+0000 7f633b905740 1 -- 172.21.15.78:0/1457158998 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f631c00e3b0 con 0x55f4242f9a10 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.345+0000 7f631a7fc640 1 -- 172.21.15.78:0/1457158998 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f631c009890 con 0x55f4242f9a10 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.345+0000 7f633b905740 10 client.27416 fetch_fsmap learned FSMap version 60 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.345+0000 7f633b905740 1 -- 172.21.15.78:0/1457158998 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55f4243a49e0 con 0x55f4242f9a10 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.345+0000 7f631a7fc640 1 -- 172.21.15.78:0/1457158998 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f631c031070 con 0x55f4242f9a10 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.346+0000 7f631a7fc640 1 -- 172.21.15.78:0/1457158998 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 60) ==== 8434+0+0 (secure 0 0 0) 0x7f631c0054a0 con 0x55f4242f9a10 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.346+0000 7f633b905740 10 client.27416 fetch_fsmap finished waiting for FSMap version 60 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.346+0000 7f633b905740 10 client.27416 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.346+0000 7f633b905740 20 client.27416 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.346+0000 7f633b905740 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55f4243a4f00 0x55f4243a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.346+0000 7f633b905740 4 client.27416 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.346+0000 7f633b905740 1 -- 172.21.15.78:0/1457158998 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f631c00e3b0 con 0x55f4243a4f00 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.346+0000 7f6332ffd640 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55f4243a4f00 0x55f4243a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.347+0000 7f6332ffd640 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55f4243a4f00 0x55f4243a7320 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=10b8598d2b1d5c8f server_cookie=a4903abd5fc1e46 in_seq=0 out_seq=0 2024-09-11T01:13:51.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.347+0000 7f631a7fc640 10 client.27416 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:13:51.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.348+0000 7f631a7fc640 1 -- 172.21.15.78:0/1457158998 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f631c00e3b0 con 0x55f4243a4f00 2024-09-11T01:13:51.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.348+0000 7f631a7fc640 10 client.27416 handle_command_reply: tid=0 2024-09-11T01:13:51.887 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:13:51.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.348+0000 7f633b905740 1 client.27416 shutdown 2024-09-11T01:13:51.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.349+0000 7f633b905740 1 -- 172.21.15.78:0/1457158998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f9a10 msgr2=0x55f424314860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:51.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.349+0000 7f633b905740 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f9a10 0x55f424314860 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f631c009a50 tx=0x7f631c009fb0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.349+0000 7f633b905740 1 -- 172.21.15.78:0/1457158998 shutdown_connections 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.349+0000 7f633b905740 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55f4243a4f00 0x55f4243a7320 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.349+0000 7f633b905740 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4243059a0 0x55f42430f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.349+0000 7f633b905740 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4242f9a10 0x55f424314860 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.349+0000 7f633b905740 1 --2- 172.21.15.78:0/1457158998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4242f90f0 0x55f424314320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.349+0000 7f633b905740 1 -- 172.21.15.78:0/1457158998 >> 172.21.15.78:0/1457158998 conn(0x55f4242f6f20 msgr2=0x55f4242f7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.349+0000 7f631a7fc640 0 client.27416 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.349+0000 7f633b905740 1 -- 172.21.15.78:0/1457158998 shutdown_connections 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.349+0000 7f633b905740 1 -- 172.21.15.78:0/1457158998 wait complete. 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.349+0000 7f633b905740 20 client.27416 trim_cache size 0 max 16384 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.350+0000 7f6339530640 1 -- 172.21.15.78:0/1301480686 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63200b54d0 msgr2=0x7f63200b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.350+0000 7f6339530640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63200b54d0 0x7f63200b7990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f632c009fd0 tx=0x7f632c00e040 comp rx=0 tx=0).stop 2024-09-11T01:13:51.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.350+0000 7f6339530640 1 -- 172.21.15.78:0/1301480686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6334101c90 msgr2=0x7f633419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:51.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.350+0000 7f6339530640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6334101c90 0x7f633419b340 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f6324009850 tx=0x7f6324030650 comp rx=0 tx=0).stop 2024-09-11T01:13:51.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.350+0000 7f6339530640 1 -- 172.21.15.78:0/1301480686 shutdown_connections 2024-09-11T01:13:51.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.350+0000 7f6339530640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f63200b54d0 0x7f63200b7990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.350+0000 7f6339530640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6334108020 0x7f63341a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.350+0000 7f6339530640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63341076a0 0x7f633419b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.350+0000 7f6339530640 1 --2- 172.21.15.78:0/1301480686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6334101c90 0x7f633419b340 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:51.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.350+0000 7f6339530640 1 -- 172.21.15.78:0/1301480686 >> 172.21.15.78:0/1301480686 conn(0x7f63340fd700 msgr2=0x7f633406aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:51.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.350+0000 7f6339530640 1 -- 172.21.15.78:0/1301480686 shutdown_connections 2024-09-11T01:13:51.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:51.350+0000 7f6339530640 1 -- 172.21.15.78:0/1301480686 wait complete. 2024-09-11T01:13:51.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:51 smithi110 ceph-mon[34712]: pgmap v788: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-09-11T01:13:51.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:51 smithi112 ceph-mon[35384]: pgmap v788: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-09-11T01:13:52.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 78076 37.14 MB/sec warmup 118 sec latency 98.915 ms 2024-09-11T01:13:53.410 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:13:53.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:53 smithi078 ceph-mon[31315]: pgmap v789: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 962 op/s 2024-09-11T01:13:53.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:53 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:13:53.777 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:13:53.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 79217 37.31 MB/sec warmup 119 sec latency 33.048 ms 2024-09-11T01:13:53.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:53 smithi110 ceph-mon[34712]: pgmap v789: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 962 op/s 2024-09-11T01:13:53.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:53 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:13:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:53 smithi112 ceph-mon[35384]: pgmap v789: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 962 op/s 2024-09-11T01:13:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:53 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:13:54.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.341+0000 7f168378d640 1 -- 172.21.15.78:0/1859165183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167c102dd0 msgr2=0x7f167c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:54.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.341+0000 7f168378d640 1 --2- 172.21.15.78:0/1859165183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167c102dd0 0x7f167c1031b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1678009900 tx=0x7f167802a3f0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.341+0000 7f168378d640 1 -- 172.21.15.78:0/1859165183 shutdown_connections 2024-09-11T01:13:54.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.341+0000 7f168378d640 1 --2- 172.21.15.78:0/1859165183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f167c109160 0x7f167c10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.341+0000 7f168378d640 1 --2- 172.21.15.78:0/1859165183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f167c1087e0 0x7f167c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.341+0000 7f168378d640 1 --2- 172.21.15.78:0/1859165183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167c102dd0 0x7f167c1031b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.341+0000 7f168378d640 1 -- 172.21.15.78:0/1859165183 >> 172.21.15.78:0/1859165183 conn(0x7f167c0fe800 msgr2=0x7f167c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:54.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.342+0000 7f168378d640 1 -- 172.21.15.78:0/1859165183 shutdown_connections 2024-09-11T01:13:54.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.342+0000 7f168378d640 1 -- 172.21.15.78:0/1859165183 wait complete. 2024-09-11T01:13:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.343+0000 7f168378d640 1 Processor -- start 2024-09-11T01:13:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.343+0000 7f168378d640 1 -- start start 2024-09-11T01:13:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.343+0000 7f168378d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f167c102dd0 0x7f167c10f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.343+0000 7f168378d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167c1087e0 0x7f167c10ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.343+0000 7f168378d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f167c109160 0x7f167c10d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.343+0000 7f168378d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f167c076ef0 con 0x7f167c109160 2024-09-11T01:13:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.343+0000 7f168378d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f167c076d70 con 0x7f167c102dd0 2024-09-11T01:13:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.343+0000 7f168378d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f167c077070 con 0x7f167c1087e0 2024-09-11T01:13:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.343+0000 7f1680d28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167c1087e0 0x7f167c10ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.343+0000 7f1681d2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f167c109160 0x7f167c10d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.344+0000 7f1681d2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f167c109160 0x7f167c10d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42786/0 (socket says 172.21.15.78:42786) 2024-09-11T01:13:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.344+0000 7f1681d2a640 1 -- 172.21.15.78:0/1295560292 learned_addr learned my addr 172.21.15.78:0/1295560292 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:54.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.344+0000 7f1681529640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f167c102dd0 0x7f167c10f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.344+0000 7f1680d28640 1 -- 172.21.15.78:0/1295560292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f167c102dd0 msgr2=0x7f167c10f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:54.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.344+0000 7f1680d28640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f167c102dd0 0x7f167c10f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.344+0000 7f1680d28640 1 -- 172.21.15.78:0/1295560292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f167c109160 msgr2=0x7f167c10d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:54.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.344+0000 7f1680d28640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f167c109160 0x7f167c10d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.344+0000 7f1680d28640 1 -- 172.21.15.78:0/1295560292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c1abc40 con 0x7f167c1087e0 2024-09-11T01:13:54.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.345+0000 7f1680d28640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167c1087e0 0x7f167c10ca10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f166c00b3f0 tx=0x7f166c00b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:54.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.346+0000 7f16727fc640 1 -- 172.21.15.78:0/1295560292 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c015020 con 0x7f167c1087e0 2024-09-11T01:13:54.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.346+0000 7f16727fc640 1 -- 172.21.15.78:0/1295560292 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f166c011ce0 con 0x7f167c1087e0 2024-09-11T01:13:54.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.346+0000 7f168378d640 1 -- 172.21.15.78:0/1295560292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167c1abe70 con 0x7f167c1087e0 2024-09-11T01:13:54.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.347+0000 7f168378d640 1 -- 172.21.15.78:0/1295560292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167c1ac320 con 0x7f167c1087e0 2024-09-11T01:13:54.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.347+0000 7f16727fc640 1 -- 172.21.15.78:0/1295560292 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c010d80 con 0x7f167c1087e0 2024-09-11T01:13:54.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.351+0000 7f1685b62740 1 Processor -- start 2024-09-11T01:13:54.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.351+0000 7f1685b62740 1 -- start start 2024-09-11T01:13:54.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.352+0000 7f16727fc640 1 -- 172.21.15.78:0/1295560292 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f166c01a460 con 0x7f167c1087e0 2024-09-11T01:13:54.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.352+0000 7f1685b62740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 0x5581462004d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.352+0000 7f1685b62740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558146200a10 0x55814620c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.352+0000 7f1685b62740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55814620c9a0 0x55814620ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.352+0000 7f1685b62740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558146176ea0 con 0x5581462000f0 2024-09-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.352+0000 7f1685b62740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5581461662d0 con 0x558146200a10 2024-09-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.352+0000 7f1685b62740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55814616a0e0 con 0x55814620c9a0 2024-09-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.352+0000 7f1681d2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 0x5581462004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.352+0000 7f1681d2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 0x5581462004d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42790/0 (socket says 172.21.15.78:42790) 2024-09-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.352+0000 7f1681d2a640 1 -- 172.21.15.78:0/986448041 learned_addr learned my addr 172.21.15.78:0/986448041 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.353+0000 7f1680d28640 1 --2- 172.21.15.78:0/986448041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55814620c9a0 0x55814620ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.353+0000 7f1680d28640 1 -- 172.21.15.78:0/986448041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558146200a10 msgr2=0x55814620c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.353+0000 7f1680d28640 1 --2- 172.21.15.78:0/986448041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558146200a10 0x55814620c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.353+0000 7f1680d28640 1 -- 172.21.15.78:0/986448041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 msgr2=0x5581462004d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:54.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.353+0000 7f1680d28640 1 --2- 172.21.15.78:0/986448041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 0x5581462004d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.353+0000 7f1680d28640 1 -- 172.21.15.78:0/986448041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558145ec4650 con 0x55814620c9a0 2024-09-11T01:13:54.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.354+0000 7f16727fc640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16680b5620 0x7f16680b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.354+0000 7f1680d28640 1 --2- 172.21.15.78:0/986448041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55814620c9a0 0x55814620ed90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f166c0e2e10 tx=0x7f166c0ed2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8106b2ee3981d2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:54.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.355+0000 7f16727fc640 1 -- 172.21.15.78:0/1295560292 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f166c0e56e0 con 0x7f167c1087e0 2024-09-11T01:13:54.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.355+0000 7f16677fe640 1 -- 172.21.15.78:0/986448041 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c0a8190 con 0x55814620c9a0 2024-09-11T01:13:54.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.355+0000 7f1681d2a640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16680b5620 0x7f16680b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.355+0000 7f16677fe640 1 -- 172.21.15.78:0/986448041 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f166c0a7290 con 0x55814620c9a0 2024-09-11T01:13:54.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.355+0000 7f16677fe640 1 -- 172.21.15.78:0/986448041 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c009200 con 0x55814620c9a0 2024-09-11T01:13:54.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.357+0000 7f1685b62740 1 -- 172.21.15.78:0/986448041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55814620c9a0 msgr2=0x55814620ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:54.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.357+0000 7f1685b62740 1 --2- 172.21.15.78:0/986448041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55814620c9a0 0x55814620ed90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f166c0e2e10 tx=0x7f166c0ed2a0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.357+0000 7f1681d2a640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16680b5620 0x7f16680b7ae0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1674009fd0 tx=0x7f1674009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:54.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.357+0000 7f1685b62740 1 -- 172.21.15.78:0/986448041 shutdown_connections 2024-09-11T01:13:54.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.357+0000 7f1685b62740 1 --2- 172.21.15.78:0/986448041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55814620c9a0 0x55814620ed90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.357+0000 7f1685b62740 1 --2- 172.21.15.78:0/986448041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558146200a10 0x55814620c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.357+0000 7f1685b62740 1 --2- 172.21.15.78:0/986448041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 0x5581462004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.357+0000 7f1685b62740 1 -- 172.21.15.78:0/986448041 >> 172.21.15.78:0/986448041 conn(0x5581461fdf20 msgr2=0x5581461ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:54.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.358+0000 7f1685b62740 1 -- 172.21.15.78:0/986448041 shutdown_connections 2024-09-11T01:13:54.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.358+0000 7f1685b62740 1 -- 172.21.15.78:0/986448041 wait complete. 2024-09-11T01:13:54.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.358+0000 7f1685b62740 1 -- start start 2024-09-11T01:13:54.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.358+0000 7f1685b62740 1 Processor -- start 2024-09-11T01:13:54.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.359+0000 7f1685b62740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581462000f0 0x55814620c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.359+0000 7f1685b62740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558146200a10 0x5581462096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.359+0000 7f1685b62740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814620c9a0 0x558146209c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.359+0000 7f1685b62740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558145fc7700 con 0x55814620c9a0 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.359+0000 7f1685b62740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558146101160 con 0x558146200a10 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.359+0000 7f1685b62740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55814620f620 con 0x5581462000f0 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.359+0000 7f1685b62740 10 client.0 _ll_register_callbacks cb 0x55814611a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.359+0000 7f1681529640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581462000f0 0x55814620c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.360+0000 7f1681529640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581462000f0 0x55814620c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:50700/0 (socket says 172.21.15.78:50700) 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.360+0000 7f1681529640 1 -- 172.21.15.78:0/724246820 learned_addr learned my addr 172.21.15.78:0/724246820 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.360+0000 7f1681529640 1 -- 172.21.15.78:0/724246820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558146200a10 msgr2=0x5581462096a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.360+0000 7f1681529640 1 --2- 172.21.15.78:0/724246820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558146200a10 0x5581462096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.360+0000 7f1681529640 1 -- 172.21.15.78:0/724246820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814620c9a0 msgr2=0x558146209c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.360+0000 7f1681529640 1 --2- 172.21.15.78:0/724246820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814620c9a0 0x558146209c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.360+0000 7f1681529640 1 -- 172.21.15.78:0/724246820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55814612ec90 con 0x5581462000f0 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.360+0000 7f1681529640 1 --2- 172.21.15.78:0/724246820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581462000f0 0x55814620c250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f167802a910 tx=0x7f1678004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5aa21299229a09f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:54.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.361+0000 7f1664ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.361+0000 7f1664ff9640 1 -- 172.21.15.78:0/724246820 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678035020 con 0x5581462000f0 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.361+0000 7f1685b62740 1 -- 172.21.15.78:0/724246820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558146178df0 con 0x5581462000f0 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.361+0000 7f1664ff9640 1 -- 172.21.15.78:0/724246820 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f1678031400 con 0x5581462000f0 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.361+0000 7f1664ff9640 1 -- 172.21.15.78:0/724246820 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1678004760 con 0x5581462000f0 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.361+0000 7f1685b62740 10 client.27379 fetch_fsmap learned FSMap version 61 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.361+0000 7f1685b62740 1 -- 172.21.15.78:0/724246820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55814621afc0 con 0x5581462000f0 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.362+0000 7f1664ff9640 1 -- 172.21.15.78:0/724246820 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558146178df0 con 0x5581462000f0 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.362+0000 7f1664ff9640 1 -- 172.21.15.78:0/724246820 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 61) ==== 8422+0+0 (secure 0 0 0) 0x7f167803a040 con 0x5581462000f0 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.362+0000 7f1685b62740 10 client.27379 fetch_fsmap finished waiting for FSMap version 61 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.362+0000 7f1685b62740 10 client.27379 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.362+0000 7f1685b62740 20 client.27379 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.362+0000 7f1685b62740 1 --2- 172.21.15.78:0/724246820 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55814621c470 0x55814621e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.362+0000 7f1685b62740 4 client.27379 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:13:54.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.362+0000 7f1685b62740 1 -- 172.21.15.78:0/724246820 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581461748a0 con 0x55814621c470 2024-09-11T01:13:54.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.363+0000 7f1680d28640 1 --2- 172.21.15.78:0/724246820 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55814621c470 0x55814621e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.367+0000 7f1680d28640 1 --2- 172.21.15.78:0/724246820 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55814621c470 0x55814621e910 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa48adb789eaf068 server_cookie=1b5b0eee80f6ba27 in_seq=0 out_seq=0 2024-09-11T01:13:54.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.367+0000 7f1664ff9640 10 client.27379 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:13:54.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.368+0000 7f1664ff9640 1 -- 172.21.15.78:0/724246820 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5581461748a0 con 0x55814621c470 2024-09-11T01:13:54.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.368+0000 7f1664ff9640 10 client.27379 handle_command_reply: tid=0 2024-09-11T01:13:54.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.369+0000 7f1685b62740 1 client.27379 shutdown 2024-09-11T01:13:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.369+0000 7f1685b62740 1 -- 172.21.15.78:0/724246820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581462000f0 msgr2=0x55814620c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.369+0000 7f1685b62740 1 --2- 172.21.15.78:0/724246820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581462000f0 0x55814620c250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f167802a910 tx=0x7f1678004270 comp rx=0 tx=0).stop 2024-09-11T01:13:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.369+0000 7f1685b62740 1 -- 172.21.15.78:0/724246820 shutdown_connections 2024-09-11T01:13:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.369+0000 7f1685b62740 1 --2- 172.21.15.78:0/724246820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814620c9a0 0x558146209c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.369+0000 7f1685b62740 1 --2- 172.21.15.78:0/724246820 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55814621c470 0x55814621e910 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.369+0000 7f1685b62740 1 --2- 172.21.15.78:0/724246820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558146200a10 0x5581462096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.370+0000 7f1685b62740 1 --2- 172.21.15.78:0/724246820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581462000f0 0x55814620c250 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.369+0000 7f1664ff9640 0 client.27379 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:13:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.370+0000 7f1685b62740 1 -- 172.21.15.78:0/724246820 >> 172.21.15.78:0/724246820 conn(0x5581461fdf20 msgr2=0x558146203a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.370+0000 7f1685b62740 1 -- 172.21.15.78:0/724246820 shutdown_connections 2024-09-11T01:13:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.370+0000 7f1685b62740 1 -- 172.21.15.78:0/724246820 wait complete. 2024-09-11T01:13:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.370+0000 7f1685b62740 20 client.27379 trim_cache size 0 max 16384 2024-09-11T01:13:54.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.396+0000 7f1685b62740 1 Processor -- start 2024-09-11T01:13:54.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.396+0000 7f1685b62740 1 -- start start 2024-09-11T01:13:54.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.396+0000 7f1685b62740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 0x5581461fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.396+0000 7f1685b62740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558146200a10 0x55814620dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.396+0000 7f1685b62740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55814620c9a0 0x55814620e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.396+0000 7f1685b62740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f166000e6c0 con 0x5581462000f0 2024-09-11T01:13:54.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.396+0000 7f1685b62740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f16600103b0 con 0x55814620c9a0 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.396+0000 7f1685b62740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f166000ed90 con 0x558146200a10 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.396+0000 7f1680d28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558146200a10 0x55814620dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.396+0000 7f1681529640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 0x5581461fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.396+0000 7f1681529640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 0x5581461fd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42808/0 (socket says 172.21.15.78:42808) 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.396+0000 7f1681529640 1 -- 172.21.15.78:0/2092465849 learned_addr learned my addr 172.21.15.78:0/2092465849 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.397+0000 7f1681529640 1 -- 172.21.15.78:0/2092465849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558146200a10 msgr2=0x55814620dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.397+0000 7f1681529640 1 --2- 172.21.15.78:0/2092465849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558146200a10 0x55814620dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.397+0000 7f1681529640 1 -- 172.21.15.78:0/2092465849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55814620c9a0 msgr2=0x55814620e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.397+0000 7f1681529640 1 --2- 172.21.15.78:0/2092465849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55814620c9a0 0x55814620e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.397+0000 7f1681529640 1 -- 172.21.15.78:0/2092465849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55814620c5c0 con 0x5581462000f0 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.398+0000 7f1681529640 1 --2- 172.21.15.78:0/2092465849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 0x5581461fd760 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f1678009a20 tx=0x7f1678005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c877750916d6b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.398+0000 7f1666ffd640 1 -- 172.21.15.78:0/2092465849 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678035040 con 0x5581462000f0 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.398+0000 7f1666ffd640 1 -- 172.21.15.78:0/2092465849 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1678041070 con 0x5581462000f0 2024-09-11T01:13:54.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.398+0000 7f1666ffd640 1 -- 172.21.15.78:0/2092465849 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678004900 con 0x5581462000f0 2024-09-11T01:13:54.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.400+0000 7f1685b62740 1 -- 172.21.15.78:0/2092465849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 msgr2=0x5581461fd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:54.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.400+0000 7f1685b62740 1 --2- 172.21.15.78:0/2092465849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 0x5581461fd760 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f1678009a20 tx=0x7f1678005d50 comp rx=0 tx=0).stop 2024-09-11T01:13:54.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.400+0000 7f1685b62740 1 -- 172.21.15.78:0/2092465849 shutdown_connections 2024-09-11T01:13:54.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.400+0000 7f1685b62740 1 --2- 172.21.15.78:0/2092465849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55814620c9a0 0x55814620e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.400+0000 7f1685b62740 1 --2- 172.21.15.78:0/2092465849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558146200a10 0x55814620dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.400+0000 7f1685b62740 1 --2- 172.21.15.78:0/2092465849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581462000f0 0x5581461fd760 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.400+0000 7f1685b62740 1 -- 172.21.15.78:0/2092465849 >> 172.21.15.78:0/2092465849 conn(0x5581461fdf20 msgr2=0x558146207f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:54.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.402+0000 7f1685b62740 1 -- 172.21.15.78:0/2092465849 shutdown_connections 2024-09-11T01:13:54.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.402+0000 7f1685b62740 1 -- 172.21.15.78:0/2092465849 wait complete. 2024-09-11T01:13:54.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.403+0000 7f1685b62740 1 -- start start 2024-09-11T01:13:54.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.403+0000 7f1685b62740 1 Processor -- start 2024-09-11T01:13:54.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.404+0000 7f1685b62740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581462000f0 0x558146214920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.404+0000 7f1685b62740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558146200a10 0x558146214e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.404+0000 7f1685b62740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814620c9a0 0x55814620f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.404+0000 7f1685b62740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f166000fc20 con 0x55814620c9a0 2024-09-11T01:13:54.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.404+0000 7f1685b62740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f166000f490 con 0x5581462000f0 2024-09-11T01:13:54.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.404+0000 7f1685b62740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f166000c6c0 con 0x558146200a10 2024-09-11T01:13:54.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.404+0000 7f1681529640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581462000f0 0x558146214920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.404+0000 7f1685b62740 10 client.0 _ll_register_callbacks cb 0x7f1678009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:54.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.404+0000 7f1680d28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558146200a10 0x558146214e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.404+0000 7f1681529640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581462000f0 0x558146214920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:60620/0 (socket says 172.21.15.78:60620) 2024-09-11T01:13:54.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.404+0000 7f1681529640 1 -- 172.21.15.78:0/3086522919 learned_addr learned my addr 172.21.15.78:0/3086522919 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:54.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.405+0000 7f1681529640 1 -- 172.21.15.78:0/3086522919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558146200a10 msgr2=0x558146214e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:54.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.405+0000 7f1681529640 1 --2- 172.21.15.78:0/3086522919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558146200a10 0x558146214e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.405+0000 7f1681529640 1 -- 172.21.15.78:0/3086522919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814620c9a0 msgr2=0x55814620f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:54.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.405+0000 7f1681529640 1 --2- 172.21.15.78:0/3086522919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814620c9a0 0x55814620f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.405+0000 7f1681529640 1 -- 172.21.15.78:0/3086522919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55814620fe10 con 0x5581462000f0 2024-09-11T01:13:54.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.406+0000 7f1681529640 1 --2- 172.21.15.78:0/3086522919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581462000f0 0x558146214920 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1678031a70 tx=0x7f1678031aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aee6bf11227bae82 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:54.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.406+0000 7f1664ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:13:54.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.407+0000 7f1664ff9640 1 -- 172.21.15.78:0/3086522919 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167805c070 con 0x5581462000f0 2024-09-11T01:13:54.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.407+0000 7f1664ff9640 1 -- 172.21.15.78:0/3086522919 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f1678004900 con 0x5581462000f0 2024-09-11T01:13:54.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.407+0000 7f1685b62740 1 -- 172.21.15.78:0/3086522919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1678041070 con 0x5581462000f0 2024-09-11T01:13:54.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.408+0000 7f1664ff9640 1 -- 172.21.15.78:0/3086522919 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1678035030 con 0x5581462000f0 2024-09-11T01:13:54.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.408+0000 7f1685b62740 10 client.27425 fetch_fsmap learned FSMap version 61 2024-09-11T01:13:54.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.408+0000 7f1685b62740 1 -- 172.21.15.78:0/3086522919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x558146289b40 con 0x5581462000f0 2024-09-11T01:13:54.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.408+0000 7f1664ff9640 1 -- 172.21.15.78:0/3086522919 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1678004fb0 con 0x5581462000f0 2024-09-11T01:13:54.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.408+0000 7f1664ff9640 1 -- 172.21.15.78:0/3086522919 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 61) ==== 8422+0+0 (secure 0 0 0) 0x7f167805f030 con 0x5581462000f0 2024-09-11T01:13:54.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.408+0000 7f1685b62740 10 client.27425 fetch_fsmap finished waiting for FSMap version 61 2024-09-11T01:13:54.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.408+0000 7f1685b62740 10 client.27425 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:13:54.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.408+0000 7f1685b62740 20 client.27425 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:54.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.408+0000 7f1685b62740 1 --2- 172.21.15.78:0/3086522919 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55814621c470 0x55814628c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:54.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.408+0000 7f1685b62740 4 client.27425 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:13:54.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.408+0000 7f1685b62740 1 -- 172.21.15.78:0/3086522919 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1678041070 con 0x55814621c470 2024-09-11T01:13:54.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.409+0000 7f1680d28640 1 --2- 172.21.15.78:0/3086522919 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55814621c470 0x55814628c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:54.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.409+0000 7f1680d28640 1 --2- 172.21.15.78:0/3086522919 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55814621c470 0x55814628c160 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21f5fadbb0797c15 server_cookie=7fbcc8e68762130c in_seq=0 out_seq=0 2024-09-11T01:13:54.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.409+0000 7f1664ff9640 10 client.27425 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:13:54.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.412+0000 7f1664ff9640 1 -- 172.21.15.78:0/3086522919 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1678041070 con 0x55814621c470 2024-09-11T01:13:54.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.412+0000 7f1664ff9640 10 client.27425 handle_command_reply: tid=0 2024-09-11T01:13:54.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.412+0000 7f1685b62740 1 client.27425 shutdown 2024-09-11T01:13:54.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.413+0000 7f1685b62740 1 -- 172.21.15.78:0/3086522919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581462000f0 msgr2=0x558146214920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:54.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.413+0000 7f1685b62740 1 --2- 172.21.15.78:0/3086522919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581462000f0 0x558146214920 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1678031a70 tx=0x7f1678031aa0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.413+0000 7f1685b62740 1 -- 172.21.15.78:0/3086522919 shutdown_connections 2024-09-11T01:13:54.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.413+0000 7f1685b62740 1 --2- 172.21.15.78:0/3086522919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814620c9a0 0x55814620f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.413+0000 7f1685b62740 1 --2- 172.21.15.78:0/3086522919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558146200a10 0x558146214e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.413+0000 7f1685b62740 1 --2- 172.21.15.78:0/3086522919 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55814621c470 0x55814628c160 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.413+0000 7f1685b62740 1 --2- 172.21.15.78:0/3086522919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581462000f0 0x558146214920 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.413+0000 7f1685b62740 1 -- 172.21.15.78:0/3086522919 >> 172.21.15.78:0/3086522919 conn(0x5581461fdf20 msgr2=0x55814620acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:54.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.413+0000 7f1664ff9640 0 client.27425 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:13:54.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.413+0000 7f1685b62740 1 -- 172.21.15.78:0/3086522919 shutdown_connections 2024-09-11T01:13:54.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.414+0000 7f1685b62740 1 -- 172.21.15.78:0/3086522919 wait complete. 2024-09-11T01:13:54.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.414+0000 7f1685b62740 20 client.27425 trim_cache size 0 max 16384 2024-09-11T01:13:54.417 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:13:54.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.416+0000 7f168378d640 1 -- 172.21.15.78:0/1295560292 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16680b5620 msgr2=0x7f16680b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:54.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.416+0000 7f168378d640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16680b5620 0x7f16680b7ae0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1674009fd0 tx=0x7f1674009500 comp rx=0 tx=0).stop 2024-09-11T01:13:54.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.416+0000 7f168378d640 1 -- 172.21.15.78:0/1295560292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167c1087e0 msgr2=0x7f167c10ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:54.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.416+0000 7f168378d640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167c1087e0 0x7f167c10ca10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f166c00b3f0 tx=0x7f166c00b8b0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.417+0000 7f168378d640 1 -- 172.21.15.78:0/1295560292 shutdown_connections 2024-09-11T01:13:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.417+0000 7f168378d640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16680b5620 0x7f16680b7ae0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.417+0000 7f168378d640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f167c109160 0x7f167c10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.417+0000 7f168378d640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167c1087e0 0x7f167c10ca10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.417+0000 7f168378d640 1 --2- 172.21.15.78:0/1295560292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f167c102dd0 0x7f167c10f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.417+0000 7f168378d640 1 -- 172.21.15.78:0/1295560292 >> 172.21.15.78:0/1295560292 conn(0x7f167c0fe800 msgr2=0x7f167c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:54.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.417+0000 7f168378d640 1 -- 172.21.15.78:0/1295560292 shutdown_connections 2024-09-11T01:13:54.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:54.417+0000 7f168378d640 1 -- 172.21.15.78:0/1295560292 wait complete. 2024-09-11T01:13:55.383 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:13:55.693 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:13:55.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 80553 18.95 MB/sec execute 1 sec latency 303.686 ms 2024-09-11T01:13:55.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:55 smithi110 ceph-mon[34712]: pgmap v790: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 995 op/s 2024-09-11T01:13:55.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:55 smithi078 ceph-mon[31315]: pgmap v790: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 995 op/s 2024-09-11T01:13:55.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:55 smithi112 ceph-mon[35384]: pgmap v790: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 995 op/s 2024-09-11T01:13:56.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.243+0000 7fce06530640 1 -- 172.21.15.78:0/3696524483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0010d100 msgr2=0x7fce0010f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:56.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 --2- 172.21.15.78:0/3696524483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0010d100 0x7fce0010f560 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009ca0 tx=0x7fcdf802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 -- 172.21.15.78:0/3696524483 shutdown_connections 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 --2- 172.21.15.78:0/3696524483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0010d100 0x7fce0010f560 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 --2- 172.21.15.78:0/3696524483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce001045c0 0x7fce0010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 --2- 172.21.15.78:0/3696524483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce00103ca0 0x7fce00104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 -- 172.21.15.78:0/3696524483 >> 172.21.15.78:0/3696524483 conn(0x7fce000fd700 msgr2=0x7fce000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 -- 172.21.15.78:0/3696524483 shutdown_connections 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 -- 172.21.15.78:0/3696524483 wait complete. 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 Processor -- start 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 -- start start 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce00103ca0 0x7fce00199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce001045c0 0x7fce001997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce0010d100 0x7fce0019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce00112d70 con 0x7fce00103ca0 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fce00112bf0 con 0x7fce0010d100 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.244+0000 7fce06530640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce00112ef0 con 0x7fce001045c0 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce00103ca0 0x7fce00199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce00103ca0 0x7fce00199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42834/0 (socket says 172.21.15.78:42834) 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdfffff640 1 -- 172.21.15.78:0/2389496404 learned_addr learned my addr 172.21.15.78:0/2389496404 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fce04acd640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce0010d100 0x7fce0019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdf7fff640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce001045c0 0x7fce001997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdfffff640 1 -- 172.21.15.78:0/2389496404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce001045c0 msgr2=0x7fce001997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdfffff640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce001045c0 0x7fce001997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdfffff640 1 -- 172.21.15.78:0/2389496404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce0010d100 msgr2=0x7fce0019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdfffff640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce0010d100 0x7fce0019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdfffff640 1 -- 172.21.15.78:0/2389496404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0019f310 con 0x7fce00103ca0 2024-09-11T01:13:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdfffff640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce00103ca0 0x7fce00199260 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000e9f0 tx=0x7fcdf000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdfdffb640 1 -- 172.21.15.78:0/2389496404 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf000cd80 con 0x7fce00103ca0 2024-09-11T01:13:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fce06530640 1 -- 172.21.15.78:0/2389496404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0019f600 con 0x7fce00103ca0 2024-09-11T01:13:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdfdffb640 1 -- 172.21.15.78:0/2389496404 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf00054d0 con 0x7fce00103ca0 2024-09-11T01:13:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fcdfdffb640 1 -- 172.21.15.78:0/2389496404 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0010690 con 0x7fce00103ca0 2024-09-11T01:13:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.245+0000 7fce06530640 1 -- 172.21.15.78:0/2389496404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce001a5cc0 con 0x7fce00103ca0 2024-09-11T01:13:56.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.248+0000 7fcdfdffb640 1 -- 172.21.15.78:0/2389496404 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fcdf0004e90 con 0x7fce00103ca0 2024-09-11T01:13:56.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.248+0000 7fce08905740 1 Processor -- start 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.248+0000 7fce08905740 1 -- start start 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.248+0000 7fce08905740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cab0f0 0x564f23cab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.248+0000 7fce08905740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 0x564f23cb7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.248+0000 7fce08905740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb79a0 0x564f23cb9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.248+0000 7fce08905740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f23c21ea0 con 0x564f23caba10 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.248+0000 7fce08905740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564f23c112d0 con 0x564f23cab0f0 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.248+0000 7fce08905740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f23c150e0 con 0x564f23cb79a0 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.248+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 0x564f23cb7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.248+0000 7fce04acd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cab0f0 0x564f23cab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 0x564f23cb7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42846/0 (socket says 172.21.15.78:42846) 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fcdfffff640 1 -- 172.21.15.78:0/4098727815 learned_addr learned my addr 172.21.15.78:0/4098727815 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fcdf7fff640 1 --2- 172.21.15.78:0/4098727815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb79a0 0x564f23cb9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fcdfffff640 1 -- 172.21.15.78:0/4098727815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb79a0 msgr2=0x564f23cb9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fcdfffff640 1 --2- 172.21.15.78:0/4098727815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb79a0 0x564f23cb9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fcdfffff640 1 -- 172.21.15.78:0/4098727815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cab0f0 msgr2=0x564f23cab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fcdfffff640 1 --2- 172.21.15.78:0/4098727815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cab0f0 0x564f23cab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fcdfffff640 1 -- 172.21.15.78:0/4098727815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2396f650 con 0x564f23caba10 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fcdfffff640 1 --2- 172.21.15.78:0/4098727815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 0x564f23cb7460 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00e3ab0 tx=0x7fcdf00ee4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a309b154168df2b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fcdfdffb640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdec0b5450 0x7fcdec0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fcdf67fc640 1 -- 172.21.15.78:0/4098727815 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00fa070 con 0x564f23caba10 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fcdf67fc640 1 -- 172.21.15.78:0/4098727815 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf00ac380 con 0x564f23caba10 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.249+0000 7fce04acd640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdec0b5450 0x7fcdec0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.250+0000 7fcdfdffb640 1 -- 172.21.15.78:0/2389496404 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7fcdf00e6410 con 0x7fce00103ca0 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.250+0000 7fcdf67fc640 1 -- 172.21.15.78:0/4098727815 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00abfe0 con 0x564f23caba10 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.250+0000 7fce04acd640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdec0b5450 0x7fcdec0b7910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcdf802f500 tx=0x7fcdf802f370 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.250+0000 7fce08905740 1 -- 172.21.15.78:0/4098727815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 msgr2=0x564f23cb7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.251+0000 7fce08905740 1 --2- 172.21.15.78:0/4098727815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 0x564f23cb7460 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00e3ab0 tx=0x7fcdf00ee4b0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.251+0000 7fce08905740 1 -- 172.21.15.78:0/4098727815 shutdown_connections 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.251+0000 7fce08905740 1 --2- 172.21.15.78:0/4098727815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb79a0 0x564f23cb9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.251+0000 7fce08905740 1 --2- 172.21.15.78:0/4098727815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 0x564f23cb7460 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.251+0000 7fce08905740 1 --2- 172.21.15.78:0/4098727815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cab0f0 0x564f23cab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.251+0000 7fce08905740 1 -- 172.21.15.78:0/4098727815 >> 172.21.15.78:0/4098727815 conn(0x564f23ca8f20 msgr2=0x564f23caa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.251+0000 7fce08905740 1 -- 172.21.15.78:0/4098727815 shutdown_connections 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.251+0000 7fce08905740 1 -- 172.21.15.78:0/4098727815 wait complete. 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.251+0000 7fce08905740 1 -- start start 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.251+0000 7fce08905740 1 Processor -- start 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fce08905740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cab0f0 0x564f23cb7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fce08905740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 0x564f23cb45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fce08905740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb79a0 0x564f23cb4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fce08905740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f23a72700 con 0x564f23caba10 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fce08905740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564f23bac160 con 0x564f23cb79a0 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fce08905740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f23cba670 con 0x564f23cab0f0 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fce08905740 10 client.0 _ll_register_callbacks cb 0x564f23bc59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 0x564f23cb45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 0x564f23cb45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42856/0 (socket says 172.21.15.78:42856) 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fcdf7fff640 1 -- 172.21.15.78:0/2572635953 learned_addr learned my addr 172.21.15.78:0/2572635953 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fce04acd640 1 --2- 172.21.15.78:0/2572635953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb79a0 0x564f23cb4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fcdf7fff640 1 -- 172.21.15.78:0/2572635953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cab0f0 msgr2=0x564f23cb7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fcdf7fff640 1 --2- 172.21.15.78:0/2572635953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cab0f0 0x564f23cb7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fcdf7fff640 1 -- 172.21.15.78:0/2572635953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb79a0 msgr2=0x564f23cb4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.252+0000 7fcdf7fff640 1 --2- 172.21.15.78:0/2572635953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb79a0 0x564f23cb4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.253+0000 7fcdf7fff640 1 -- 172.21.15.78:0/2572635953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f23bd9c90 con 0x564f23caba10 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.253+0000 7fcdf7fff640 1 --2- 172.21.15.78:0/2572635953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 0x564f23cb45d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fcde80107f0 tx=0x7fcde8010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75e3251b4fe5194 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.253+0000 7fcddf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:13:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.253+0000 7fcddf7fe640 1 -- 172.21.15.78:0/2572635953 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde801e060 con 0x564f23caba10 2024-09-11T01:13:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.253+0000 7fce08905740 1 -- 172.21.15.78:0/2572635953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf00ac380 con 0x564f23caba10 2024-09-11T01:13:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.253+0000 7fcddf7fe640 1 -- 172.21.15.78:0/2572635953 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7fcde800fb90 con 0x564f23caba10 2024-09-11T01:13:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.253+0000 7fcddf7fe640 1 -- 172.21.15.78:0/2572635953 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fcde800fd90 con 0x564f23caba10 2024-09-11T01:13:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.253+0000 7fce08905740 10 client.18276 fetch_fsmap learned FSMap version 61 2024-09-11T01:13:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.253+0000 7fce08905740 1 -- 172.21.15.78:0/2572635953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x564f23cce770 con 0x564f23caba10 2024-09-11T01:13:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.253+0000 7fcddf7fe640 1 -- 172.21.15.78:0/2572635953 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf00ac380 con 0x564f23caba10 2024-09-11T01:13:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.254+0000 7fcddf7fe640 1 -- 172.21.15.78:0/2572635953 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 61) ==== 8422+0+0 (secure 0 0 0) 0x7fcde8021490 con 0x564f23caba10 2024-09-11T01:13:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.254+0000 7fce08905740 10 client.18276 fetch_fsmap finished waiting for FSMap version 61 2024-09-11T01:13:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.254+0000 7fce08905740 10 client.18276 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:13:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.254+0000 7fce08905740 20 client.18276 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.254+0000 7fce08905740 1 --2- 172.21.15.78:0/2572635953 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x564f23ccfd20 0x564f23cd21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.254+0000 7fce08905740 4 client.18276 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.254+0000 7fce08905740 1 -- 172.21.15.78:0/2572635953 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f23c23df0 con 0x564f23ccfd20 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.254+0000 7fcdfffff640 1 --2- 172.21.15.78:0/2572635953 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x564f23ccfd20 0x564f23cd21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.254+0000 7fcddf7fe640 10 client.18276 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.254+0000 7fcdfffff640 1 --2- 172.21.15.78:0/2572635953 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x564f23ccfd20 0x564f23cd21c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3777ac9e45edbd83 server_cookie=6dc8f67212354bcf in_seq=0 out_seq=0 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.255+0000 7fcddf7fe640 1 -- 172.21.15.78:0/2572635953 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564f23c23df0 con 0x564f23ccfd20 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.255+0000 7fcddf7fe640 10 client.18276 handle_command_reply: tid=0 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.255+0000 7fce08905740 1 client.18276 shutdown 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.256+0000 7fce08905740 1 -- 172.21.15.78:0/2572635953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 msgr2=0x564f23cb45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.256+0000 7fce08905740 1 --2- 172.21.15.78:0/2572635953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 0x564f23cb45d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fcde80107f0 tx=0x7fcde8010cb0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.256+0000 7fce08905740 1 -- 172.21.15.78:0/2572635953 shutdown_connections 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.256+0000 7fce08905740 1 --2- 172.21.15.78:0/2572635953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb79a0 0x564f23cb4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.256+0000 7fce08905740 1 --2- 172.21.15.78:0/2572635953 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x564f23ccfd20 0x564f23cd21c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.256+0000 7fce08905740 1 --2- 172.21.15.78:0/2572635953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23caba10 0x564f23cb45d0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.256+0000 7fce08905740 1 --2- 172.21.15.78:0/2572635953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cab0f0 0x564f23cb7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.256+0000 7fce08905740 1 -- 172.21.15.78:0/2572635953 >> 172.21.15.78:0/2572635953 conn(0x564f23ca8f20 msgr2=0x564f23caeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.256+0000 7fcddf7fe640 0 client.18276 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:13:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.256+0000 7fce08905740 1 -- 172.21.15.78:0/2572635953 shutdown_connections 2024-09-11T01:13:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.256+0000 7fce08905740 1 -- 172.21.15.78:0/2572635953 wait complete. 2024-09-11T01:13:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.256+0000 7fce08905740 20 client.18276 trim_cache size 0 max 16384 2024-09-11T01:13:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.274+0000 7fce08905740 1 Processor -- start 2024-09-11T01:13:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.274+0000 7fce08905740 1 -- start start 2024-09-11T01:13:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.274+0000 7fce08905740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 0x564f23ca8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.274+0000 7fce08905740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb79a0 0x564f23cb3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.274+0000 7fce08905740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb17e0 0x564f23cb1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.274+0000 7fce08905740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce001ac390 con 0x564f23cab0f0 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.274+0000 7fce08905740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fce001ae080 con 0x564f23cb79a0 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.274+0000 7fce08905740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce001aca60 con 0x564f23cb17e0 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb79a0 0x564f23cb3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 0x564f23ca8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb79a0 0x564f23cb3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:60668/0 (socket says 172.21.15.78:60668) 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 0x564f23ca8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42872/0 (socket says 172.21.15.78:42872) 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdf7fff640 1 -- 172.21.15.78:0/3438203220 learned_addr learned my addr 172.21.15.78:0/3438203220 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fce04acd640 1 --2- 172.21.15.78:0/3438203220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb17e0 0x564f23cb1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdf7fff640 1 -- 172.21.15.78:0/3438203220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb17e0 msgr2=0x564f23cb1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdf7fff640 1 --2- 172.21.15.78:0/3438203220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb17e0 0x564f23cb1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdf7fff640 1 -- 172.21.15.78:0/3438203220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 msgr2=0x564f23ca8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdf7fff640 1 --2- 172.21.15.78:0/3438203220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 0x564f23ca8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdf7fff640 1 -- 172.21.15.78:0/3438203220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f23cd2840 con 0x564f23cb79a0 2024-09-11T01:13:56.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdfffff640 1 --2- 172.21.15.78:0/3438203220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 0x564f23ca8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:56.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdf7fff640 1 --2- 172.21.15.78:0/3438203220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb79a0 0x564f23cb3560 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcde800b010 tx=0x7fcde800a700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eacb1955ccc8f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:56.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdf5ffb640 1 -- 172.21.15.78:0/3438203220 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8021a50 con 0x564f23cb79a0 2024-09-11T01:13:56.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdf5ffb640 1 -- 172.21.15.78:0/3438203220 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde8021bf0 con 0x564f23cb79a0 2024-09-11T01:13:56.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.275+0000 7fcdf5ffb640 1 -- 172.21.15.78:0/3438203220 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80038e0 con 0x564f23cb79a0 2024-09-11T01:13:56.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.276+0000 7fce08905740 1 -- 172.21.15.78:0/3438203220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb79a0 msgr2=0x564f23cb3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:56.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.276+0000 7fce08905740 1 --2- 172.21.15.78:0/3438203220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb79a0 0x564f23cb3560 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcde800b010 tx=0x7fcde800a700 comp rx=0 tx=0).stop 2024-09-11T01:13:56.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.276+0000 7fce08905740 1 -- 172.21.15.78:0/3438203220 shutdown_connections 2024-09-11T01:13:56.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.276+0000 7fce08905740 1 --2- 172.21.15.78:0/3438203220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb17e0 0x564f23cb1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.276+0000 7fce08905740 1 --2- 172.21.15.78:0/3438203220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb79a0 0x564f23cb3560 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.276+0000 7fce08905740 1 --2- 172.21.15.78:0/3438203220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 0x564f23ca8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.276+0000 7fce08905740 1 -- 172.21.15.78:0/3438203220 >> 172.21.15.78:0/3438203220 conn(0x564f23ca8f20 msgr2=0x564f23cb2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:56.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.276+0000 7fce08905740 1 -- 172.21.15.78:0/3438203220 shutdown_connections 2024-09-11T01:13:56.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.276+0000 7fce08905740 1 -- 172.21.15.78:0/3438203220 wait complete. 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.277+0000 7fce08905740 1 -- start start 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.277+0000 7fce08905740 1 Processor -- start 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.277+0000 7fce08905740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 0x564f23cc8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.277+0000 7fce08905740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb17e0 0x564f23cc8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.277+0000 7fce08905740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb79a0 0x564f23cc5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.277+0000 7fce08905740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce001ad8f0 con 0x564f23cab0f0 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.277+0000 7fce08905740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fce001ad160 con 0x564f23cb17e0 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.277+0000 7fce08905740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce001aa390 con 0x564f23cb79a0 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.277+0000 7fce08905740 10 client.0 _ll_register_callbacks cb 0x7fcde800a770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.277+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 0x564f23cc8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 0x564f23cc8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42886/0 (socket says 172.21.15.78:42886) 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fcdfffff640 1 -- 172.21.15.78:0/361455869 learned_addr learned my addr 172.21.15.78:0/361455869 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fcdf7fff640 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb17e0 0x564f23cc8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fce04acd640 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb79a0 0x564f23cc5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fcdfffff640 1 -- 172.21.15.78:0/361455869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb79a0 msgr2=0x564f23cc5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fcdfffff640 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb79a0 0x564f23cc5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fcdfffff640 1 -- 172.21.15.78:0/361455869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb17e0 msgr2=0x564f23cc8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fcdfffff640 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb17e0 0x564f23cc8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fcdfffff640 1 -- 172.21.15.78:0/361455869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f23cc5b80 con 0x564f23cab0f0 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fce04acd640 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb79a0 0x564f23cc5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fcdfffff640 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 0x564f23cc8610 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00a9ae0 tx=0x7fcdf00ee170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9e8e281c954946 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fcddf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fcddf7fe640 1 -- 172.21.15.78:0/361455869 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00d7f50 con 0x564f23cab0f0 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.278+0000 7fcddf7fe640 1 -- 172.21.15.78:0/361455869 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7fcdf00a8ae0 con 0x564f23cab0f0 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.279+0000 7fce08905740 1 -- 172.21.15.78:0/361455869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde8021bf0 con 0x564f23cab0f0 2024-09-11T01:13:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.279+0000 7fcddf7fe640 1 -- 172.21.15.78:0/361455869 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fcdf00d8bd0 con 0x564f23cab0f0 2024-09-11T01:13:56.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.279+0000 7fce08905740 10 client.18282 fetch_fsmap learned FSMap version 61 2024-09-11T01:13:56.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.279+0000 7fce08905740 1 -- 172.21.15.78:0/361455869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x564f23d3d980 con 0x564f23cab0f0 2024-09-11T01:13:56.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.280+0000 7fcddf7fe640 1 -- 172.21.15.78:0/361455869 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde8021bf0 con 0x564f23cab0f0 2024-09-11T01:13:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.280+0000 7fcddf7fe640 1 -- 172.21.15.78:0/361455869 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 61) ==== 8422+0+0 (secure 0 0 0) 0x7fcdf00e3dc0 con 0x564f23cab0f0 2024-09-11T01:13:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.280+0000 7fce08905740 10 client.18282 fetch_fsmap finished waiting for FSMap version 61 2024-09-11T01:13:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.280+0000 7fce08905740 10 client.18282 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:13:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.280+0000 7fce08905740 20 client.18282 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.280+0000 7fce08905740 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x564f23d3ddd0 0x564f23d401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.280+0000 7fce08905740 4 client.18282 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:13:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.280+0000 7fce08905740 1 -- 172.21.15.78:0/361455869 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f23c1f8a0 con 0x564f23d3ddd0 2024-09-11T01:13:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.280+0000 7fcdf7fff640 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x564f23d3ddd0 0x564f23d401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.281+0000 7fcddf7fe640 10 client.18282 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:13:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.283+0000 7fcdf7fff640 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x564f23d3ddd0 0x564f23d401c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=918a28cac4d20c83 server_cookie=10455ea0337d656 in_seq=0 out_seq=0 2024-09-11T01:13:56.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fcddf7fe640 1 -- 172.21.15.78:0/361455869 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f23c1f8a0 con 0x564f23d3ddd0 2024-09-11T01:13:56.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fcddf7fe640 10 client.18282 handle_command_reply: tid=0 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fce08905740 1 client.18282 shutdown 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fce08905740 1 -- 172.21.15.78:0/361455869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 msgr2=0x564f23cc8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fce08905740 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 0x564f23cc8610 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00a9ae0 tx=0x7fcdf00ee170 comp rx=0 tx=0).stop 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fce08905740 1 -- 172.21.15.78:0/361455869 shutdown_connections 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fce08905740 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f23cb79a0 0x564f23cc5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fce08905740 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564f23cb17e0 0x564f23cc8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fce08905740 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x564f23d3ddd0 0x564f23d401c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fcddf7fe640 0 client.18282 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fce08905740 1 --2- 172.21.15.78:0/361455869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f23cab0f0 0x564f23cc8610 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fce08905740 1 -- 172.21.15.78:0/361455869 >> 172.21.15.78:0/361455869 conn(0x564f23ca8f20 msgr2=0x564f23cb2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fce08905740 1 -- 172.21.15.78:0/361455869 shutdown_connections 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fce08905740 1 -- 172.21.15.78:0/361455869 wait complete. 2024-09-11T01:13:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.284+0000 7fce08905740 20 client.18282 trim_cache size 0 max 16384 2024-09-11T01:13:56.288 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:13:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.285+0000 7fce06530640 1 -- 172.21.15.78:0/2389496404 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdec0b5450 msgr2=0x7fcdec0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.285+0000 7fce06530640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdec0b5450 0x7fcdec0b7910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcdf802f500 tx=0x7fcdf802f370 comp rx=0 tx=0).stop 2024-09-11T01:13:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.285+0000 7fce06530640 1 -- 172.21.15.78:0/2389496404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce00103ca0 msgr2=0x7fce00199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.285+0000 7fce06530640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce00103ca0 0x7fce00199260 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000e9f0 tx=0x7fcdf000edf0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.285+0000 7fce06530640 1 -- 172.21.15.78:0/2389496404 shutdown_connections 2024-09-11T01:13:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.285+0000 7fce06530640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcdec0b5450 0x7fcdec0b7910 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.285+0000 7fce06530640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce0010d100 0x7fce0019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.285+0000 7fce06530640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce001045c0 0x7fce001997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.285+0000 7fce06530640 1 --2- 172.21.15.78:0/2389496404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce00103ca0 0x7fce00199260 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.285+0000 7fce06530640 1 -- 172.21.15.78:0/2389496404 >> 172.21.15.78:0/2389496404 conn(0x7fce000fd700 msgr2=0x7fce0010f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.285+0000 7fce06530640 1 -- 172.21.15.78:0/2389496404 shutdown_connections 2024-09-11T01:13:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:56.285+0000 7fce06530640 1 -- 172.21.15.78:0/2389496404 wait complete. 2024-09-11T01:13:56.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 81482 33.13 MB/sec execute 2 sec latency 374.723 ms 2024-09-11T01:13:57.081 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:13:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:13:57.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 82378 39.74 MB/sec execute 3 sec latency 356.008 ms 2024-09-11T01:13:57.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:57 smithi110 ceph-mon[34712]: pgmap v791: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-09-11T01:13:57.974 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:57 smithi078 ceph-mon[31315]: pgmap v791: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-09-11T01:13:57.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:57 smithi112 ceph-mon[35384]: pgmap v791: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-09-11T01:13:58.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 83226 41.87 MB/sec execute 4 sec latency 75.955 ms 2024-09-11T01:13:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.074+0000 7f66b544e640 1 -- 172.21.15.78:0/1845671647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b0101cb0 msgr2=0x7f66b0102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.074+0000 7f66b544e640 1 --2- 172.21.15.78:0/1845671647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b0101cb0 0x7f66b0102090 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009900 tx=0x7f66a002a410 comp rx=0 tx=0).stop 2024-09-11T01:13:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.074+0000 7f66b544e640 1 -- 172.21.15.78:0/1845671647 shutdown_connections 2024-09-11T01:13:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.074+0000 7f66b544e640 1 --2- 172.21.15.78:0/1845671647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0108040 0x7f66b010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.074+0000 7f66b544e640 1 --2- 172.21.15.78:0/1845671647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b01076c0 0x7f66b0107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.074+0000 7f66b544e640 1 --2- 172.21.15.78:0/1845671647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b0101cb0 0x7f66b0102090 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.074+0000 7f66b544e640 1 -- 172.21.15.78:0/1845671647 >> 172.21.15.78:0/1845671647 conn(0x7f66b00fd700 msgr2=0x7f66b00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.074+0000 7f66b544e640 1 -- 172.21.15.78:0/1845671647 shutdown_connections 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.074+0000 7f66b544e640 1 -- 172.21.15.78:0/1845671647 wait complete. 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66b544e640 1 Processor -- start 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66b544e640 1 -- start start 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66b544e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0101cb0 0x7f66b01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66b544e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b01076c0 0x7f66b0199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66b544e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b0108040 0x7f66b019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66b544e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0112e20 con 0x7f66b0108040 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66b544e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0112ca0 con 0x7f66b0101cb0 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66b544e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0112fa0 con 0x7f66b01076c0 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b0108040 0x7f66b019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b0108040 0x7f66b019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42894/0 (socket says 172.21.15.78:42894) 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66af7fe640 1 -- 172.21.15.78:0/4217745293 learned_addr learned my addr 172.21.15.78:0/4217745293 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66a7fff640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b01076c0 0x7f66b0199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.075+0000 7f66aeffd640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0101cb0 0x7f66b01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.076+0000 7f66a7fff640 1 -- 172.21.15.78:0/4217745293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0101cb0 msgr2=0x7f66b01992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.076+0000 7f66a7fff640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0101cb0 0x7f66b01992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.076+0000 7f66a7fff640 1 -- 172.21.15.78:0/4217745293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b0108040 msgr2=0x7f66b019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.076+0000 7f66a7fff640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b0108040 0x7f66b019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.076+0000 7f66a7fff640 1 -- 172.21.15.78:0/4217745293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b019f380 con 0x7f66b01076c0 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.076+0000 7f66af7fe640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b0108040 0x7f66b019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.076+0000 7f66a7fff640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b01076c0 0x7f66b0199810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f669800e9f0 tx=0x7f669800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.076+0000 7f66acff9640 1 -- 172.21.15.78:0/4217745293 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669800ccc0 con 0x7f66b01076c0 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.076+0000 7f66b544e640 1 -- 172.21.15.78:0/4217745293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b019f670 con 0x7f66b01076c0 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.076+0000 7f66acff9640 1 -- 172.21.15.78:0/4217745293 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6698005020 con 0x7f66b01076c0 2024-09-11T01:13:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.076+0000 7f66acff9640 1 -- 172.21.15.78:0/4217745293 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698004a00 con 0x7f66b01076c0 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.076+0000 7f66b544e640 1 -- 172.21.15.78:0/4217745293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b01a5d30 con 0x7f66b01076c0 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66b7823740 1 Processor -- start 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66b7823740 1 -- start start 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66acff9640 1 -- 172.21.15.78:0/4217745293 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6698004ba0 con 0x7f66b01076c0 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66b7823740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 0x55e3c9de78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66b7823740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de7df0 0x55e3c9df7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66b7823740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9df8190 0x55e3c9dfa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66b7823740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e3c9d5dea0 con 0x55e3c9de74d0 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66b7823740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e3c9d4d2d0 con 0x55e3c9df8190 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66b7823740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e3c9d510e0 con 0x55e3c9de7df0 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 0x55e3c9de78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 0x55e3c9de78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42896/0 (socket says 172.21.15.78:42896) 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66af7fe640 1 -- 172.21.15.78:0/2764997851 learned_addr learned my addr 172.21.15.78:0/2764997851 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66aeffd640 1 --2- 172.21.15.78:0/2764997851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de7df0 0x55e3c9df7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66a7fff640 1 --2- 172.21.15.78:0/2764997851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9df8190 0x55e3c9dfa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66af7fe640 1 -- 172.21.15.78:0/2764997851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de7df0 msgr2=0x55e3c9df7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66af7fe640 1 --2- 172.21.15.78:0/2764997851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de7df0 0x55e3c9df7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66af7fe640 1 -- 172.21.15.78:0/2764997851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9df8190 msgr2=0x55e3c9dfa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66af7fe640 1 --2- 172.21.15.78:0/2764997851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9df8190 0x55e3c9dfa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.079+0000 7f66af7fe640 1 -- 172.21.15.78:0/2764997851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3c9aab650 con 0x55e3c9de74d0 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.080+0000 7f66acff9640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f669c0b5450 0x7f669c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.080+0000 7f66af7fe640 1 --2- 172.21.15.78:0/2764997851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 0x55e3c9de78b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f66a8010890 tx=0x7f66a8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726ef4515b1a11af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.080+0000 7f66aeffd640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f669c0b5450 0x7f669c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.080+0000 7f66a57fa640 1 -- 172.21.15.78:0/2764997851 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a800ebf0 con 0x55e3c9de74d0 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.080+0000 7f66acff9640 1 -- 172.21.15.78:0/4217745293 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f66980e5fb0 con 0x7f66b01076c0 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.080+0000 7f66a57fa640 1 -- 172.21.15.78:0/2764997851 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a8015070 con 0x55e3c9de74d0 2024-09-11T01:13:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.080+0000 7f66a57fa640 1 -- 172.21.15.78:0/2764997851 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a800f620 con 0x55e3c9de74d0 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.081+0000 7f66b7823740 1 -- 172.21.15.78:0/2764997851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 msgr2=0x55e3c9de78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.081+0000 7f66aeffd640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f669c0b5450 0x7f669c0b7910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f66a0031440 tx=0x7f66a0005b90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.081+0000 7f66b7823740 1 --2- 172.21.15.78:0/2764997851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 0x55e3c9de78b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f66a8010890 tx=0x7f66a8010d50 comp rx=0 tx=0).stop 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.081+0000 7f66b7823740 1 -- 172.21.15.78:0/2764997851 shutdown_connections 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.081+0000 7f66b7823740 1 --2- 172.21.15.78:0/2764997851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9df8190 0x55e3c9dfa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.081+0000 7f66b7823740 1 --2- 172.21.15.78:0/2764997851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de7df0 0x55e3c9df7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.081+0000 7f66b7823740 1 --2- 172.21.15.78:0/2764997851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 0x55e3c9de78b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.081+0000 7f66b7823740 1 -- 172.21.15.78:0/2764997851 >> 172.21.15.78:0/2764997851 conn(0x55e3c9de5300 msgr2=0x55e3c9de64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.081+0000 7f66b7823740 1 -- 172.21.15.78:0/2764997851 shutdown_connections 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.081+0000 7f66b7823740 1 -- 172.21.15.78:0/2764997851 wait complete. 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.081+0000 7f66b7823740 1 -- start start 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.081+0000 7f66b7823740 1 Processor -- start 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66b7823740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9de74d0 0x55e3c9df7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66b7823740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de7df0 0x55e3c9df4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66b7823740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9df8190 0x55e3c9df5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66b7823740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e3c9bae700 con 0x55e3c9de7df0 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66b7823740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e3c9ce8160 con 0x55e3c9de74d0 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66b7823740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e3c9dfae40 con 0x55e3c9df8190 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66b7823740 10 client.0 _ll_register_callbacks cb 0x55e3c9d019e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de7df0 0x55e3c9df4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66aeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9de74d0 0x55e3c9df7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9df8190 0x55e3c9df5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de7df0 0x55e3c9df4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42906/0 (socket says 172.21.15.78:42906) 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66a7fff640 1 -- 172.21.15.78:0/1207355762 learned_addr learned my addr 172.21.15.78:0/1207355762 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66a7fff640 1 -- 172.21.15.78:0/1207355762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9df8190 msgr2=0x55e3c9df5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66a7fff640 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9df8190 0x55e3c9df5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66a7fff640 1 -- 172.21.15.78:0/1207355762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9de74d0 msgr2=0x55e3c9df7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66a7fff640 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9de74d0 0x55e3c9df7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66a7fff640 1 -- 172.21.15.78:0/1207355762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3c9d15c90 con 0x55e3c9de7df0 2024-09-11T01:13:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66aeffd640 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9de74d0 0x55e3c9df7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66af7fe640 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9df8190 0x55e3c9df5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66a7fff640 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de7df0 0x55e3c9df4e80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f66980a89e0 tx=0x7f6698009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce47b4e85279f23 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66967fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:13:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66967fc640 1 -- 172.21.15.78:0/1207355762 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669801c040 con 0x55e3c9de7df0 2024-09-11T01:13:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66967fc640 1 -- 172.21.15.78:0/1207355762 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f66980054f0 con 0x55e3c9de7df0 2024-09-11T01:13:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66b7823740 1 -- 172.21.15.78:0/1207355762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a8015070 con 0x55e3c9de7df0 2024-09-11T01:13:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.082+0000 7f66967fc640 1 -- 172.21.15.78:0/1207355762 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f66980abd60 con 0x55e3c9de7df0 2024-09-11T01:13:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.083+0000 7f66b7823740 10 client.18294 fetch_fsmap learned FSMap version 61 2024-09-11T01:13:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.083+0000 7f66b7823740 1 -- 172.21.15.78:0/1207355762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e3c9e06730 con 0x55e3c9de7df0 2024-09-11T01:13:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.083+0000 7f66967fc640 1 -- 172.21.15.78:0/1207355762 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a8015070 con 0x55e3c9de7df0 2024-09-11T01:13:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.083+0000 7f66967fc640 1 -- 172.21.15.78:0/1207355762 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 61) ==== 8422+0+0 (secure 0 0 0) 0x7f66980e3960 con 0x55e3c9de7df0 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.083+0000 7f66b7823740 10 client.18294 fetch_fsmap finished waiting for FSMap version 61 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.083+0000 7f66b7823740 10 client.18294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.083+0000 7f66b7823740 20 client.18294 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.083+0000 7f66b7823740 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e3c9e07be0 0x55e3c9e0a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.083+0000 7f66b7823740 4 client.18294 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.083+0000 7f66b7823740 1 -- 172.21.15.78:0/1207355762 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3c9d5fdf0 con 0x55e3c9e07be0 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.100+0000 7f66aeffd640 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e3c9e07be0 0x55e3c9e0a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.100+0000 7f66967fc640 10 client.18294 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.100+0000 7f66aeffd640 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e3c9e07be0 0x55e3c9e0a020 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3693f3f77cd3033 server_cookie=dfcfcd5688285637 in_seq=0 out_seq=0 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66967fc640 1 -- 172.21.15.78:0/1207355762 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e3c9d5fdf0 con 0x55e3c9e07be0 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66967fc640 10 client.18294 handle_command_reply: tid=0 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66b7823740 1 client.18294 shutdown 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66b7823740 1 -- 172.21.15.78:0/1207355762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de7df0 msgr2=0x55e3c9df4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66b7823740 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de7df0 0x55e3c9df4e80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f66980a89e0 tx=0x7f6698009f60 comp rx=0 tx=0).stop 2024-09-11T01:13:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66b7823740 1 -- 172.21.15.78:0/1207355762 shutdown_connections 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66b7823740 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9df8190 0x55e3c9df5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66b7823740 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e3c9e07be0 0x55e3c9e0a020 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66967fc640 0 client.18294 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66b7823740 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de7df0 0x55e3c9df4e80 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66b7823740 1 --2- 172.21.15.78:0/1207355762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9de74d0 0x55e3c9df7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66b7823740 1 -- 172.21.15.78:0/1207355762 >> 172.21.15.78:0/1207355762 conn(0x55e3c9de5300 msgr2=0x55e3c9deae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66b7823740 1 -- 172.21.15.78:0/1207355762 shutdown_connections 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66b7823740 1 -- 172.21.15.78:0/1207355762 wait complete. 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.102+0000 7f66b7823740 20 client.18294 trim_cache size 0 max 16384 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.121+0000 7f66b7823740 1 Processor -- start 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.121+0000 7f66b7823740 1 -- start start 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.121+0000 7f66b7823740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 0x55e3c9e04070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.121+0000 7f66b7823740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de7df0 0x55e3c9de6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.121+0000 7f66b7823740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9df8190 0x55e3c9e0f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.121+0000 7f66b7823740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01ac470 con 0x55e3c9de74d0 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.121+0000 7f66b7823740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01ae160 con 0x55e3c9df8190 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.121+0000 7f66b7823740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01acb40 con 0x55e3c9de7df0 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.121+0000 7f66aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 0x55e3c9e04070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.121+0000 7f66aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 0x55e3c9e04070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42908/0 (socket says 172.21.15.78:42908) 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.121+0000 7f66aeffd640 1 -- 172.21.15.78:0/463770629 learned_addr learned my addr 172.21.15.78:0/463770629 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.121+0000 7f66a7fff640 1 --2- 172.21.15.78:0/463770629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de7df0 0x55e3c9de6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.122+0000 7f66aeffd640 1 -- 172.21.15.78:0/463770629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de7df0 msgr2=0x55e3c9de6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.122+0000 7f66aeffd640 1 --2- 172.21.15.78:0/463770629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de7df0 0x55e3c9de6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.122+0000 7f66af7fe640 1 --2- 172.21.15.78:0/463770629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9df8190 0x55e3c9e0f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.122+0000 7f66aeffd640 1 -- 172.21.15.78:0/463770629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9df8190 msgr2=0x55e3c9e0f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.122+0000 7f66aeffd640 1 --2- 172.21.15.78:0/463770629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9df8190 0x55e3c9e0f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.122+0000 7f66aeffd640 1 -- 172.21.15.78:0/463770629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3c9df57b0 con 0x55e3c9de74d0 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.122+0000 7f66a7fff640 1 --2- 172.21.15.78:0/463770629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de7df0 0x55e3c9de6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.122+0000 7f66af7fe640 1 --2- 172.21.15.78:0/463770629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9df8190 0x55e3c9e0f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.122+0000 7f66aeffd640 1 --2- 172.21.15.78:0/463770629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 0x55e3c9e04070 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f66a002a930 tx=0x7f66a003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ff1aa07e5f9112 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.122+0000 7f66a4ff9640 1 -- 172.21.15.78:0/463770629 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0057070 con 0x55e3c9de74d0 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.122+0000 7f66a4ff9640 1 -- 172.21.15.78:0/463770629 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a0004300 con 0x55e3c9de74d0 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.122+0000 7f66a4ff9640 1 -- 172.21.15.78:0/463770629 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0035040 con 0x55e3c9de74d0 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.123+0000 7f66b7823740 1 -- 172.21.15.78:0/463770629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 msgr2=0x55e3c9e04070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.123+0000 7f66b7823740 1 --2- 172.21.15.78:0/463770629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 0x55e3c9e04070 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f66a002a930 tx=0x7f66a003f000 comp rx=0 tx=0).stop 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.124+0000 7f66b7823740 1 -- 172.21.15.78:0/463770629 shutdown_connections 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.124+0000 7f66b7823740 1 --2- 172.21.15.78:0/463770629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9df8190 0x55e3c9e0f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.124+0000 7f66b7823740 1 --2- 172.21.15.78:0/463770629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de7df0 0x55e3c9de6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.124+0000 7f66b7823740 1 --2- 172.21.15.78:0/463770629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9de74d0 0x55e3c9e04070 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.124+0000 7f66b7823740 1 -- 172.21.15.78:0/463770629 >> 172.21.15.78:0/463770629 conn(0x55e3c9de5300 msgr2=0x55e3c9de6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.124+0000 7f66b7823740 1 -- 172.21.15.78:0/463770629 shutdown_connections 2024-09-11T01:13:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.124+0000 7f66b7823740 1 -- 172.21.15.78:0/463770629 wait complete. 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.124+0000 7f66b7823740 1 -- start start 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.125+0000 7f66b7823740 1 Processor -- start 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.125+0000 7f66b7823740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de74d0 0x55e3c9e009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.125+0000 7f66b7823740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9de7df0 0x55e3c9e0f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.125+0000 7f66b7823740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9df8190 0x55e3c9e0fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.126+0000 7f66a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9de7df0 0x55e3c9e0f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.126+0000 7f66a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9de7df0 0x55e3c9e0f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:60740/0 (socket says 172.21.15.78:60740) 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.126+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9df8190 0x55e3c9e0fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.126+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9df8190 0x55e3c9e0fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42920/0 (socket says 172.21.15.78:42920) 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.126+0000 7f66a7fff640 1 -- 172.21.15.78:0/2891571130 learned_addr learned my addr 172.21.15.78:0/2891571130 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.126+0000 7f66b7823740 1 -- 172.21.15.78:0/2891571130 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01ad9d0 con 0x55e3c9df8190 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.126+0000 7f66b7823740 1 -- 172.21.15.78:0/2891571130 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01ad240 con 0x55e3c9de7df0 2024-09-11T01:13:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.126+0000 7f66b7823740 1 -- 172.21.15.78:0/2891571130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01aa470 con 0x55e3c9de74d0 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.126+0000 7f66b7823740 10 client.0 _ll_register_callbacks cb 0x7f669800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.127+0000 7f66af7fe640 1 -- 172.21.15.78:0/2891571130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de74d0 msgr2=0x55e3c9e009e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.127+0000 7f66af7fe640 1 --2- 172.21.15.78:0/2891571130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de74d0 0x55e3c9e009e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.127+0000 7f66af7fe640 1 -- 172.21.15.78:0/2891571130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9de7df0 msgr2=0x55e3c9e0f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.127+0000 7f66af7fe640 1 --2- 172.21.15.78:0/2891571130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9de7df0 0x55e3c9e0f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.127+0000 7f66af7fe640 1 -- 172.21.15.78:0/2891571130 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3c9e73af0 con 0x55e3c9df8190 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.127+0000 7f66af7fe640 1 --2- 172.21.15.78:0/2891571130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9df8190 0x55e3c9e0fdd0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f66a8000f80 tx=0x7f66a8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920d9ee1b03fb30a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.127+0000 7f66967fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.128+0000 7f66967fc640 1 -- 172.21.15.78:0/2891571130 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8004e10 con 0x55e3c9df8190 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.128+0000 7f66967fc640 1 -- 172.21.15.78:0/2891571130 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8449+0+0 (secure 0 0 0) 0x7f66a8011440 con 0x55e3c9df8190 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.128+0000 7f66b7823740 1 -- 172.21.15.78:0/2891571130 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a0004300 con 0x55e3c9df8190 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.128+0000 7f66967fc640 1 -- 172.21.15.78:0/2891571130 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f66a8011660 con 0x55e3c9df8190 2024-09-11T01:13:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.129+0000 7f66b7823740 10 client.18300 fetch_fsmap learned FSMap version 61 2024-09-11T01:13:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.129+0000 7f66b7823740 1 -- 172.21.15.78:0/2891571130 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e3c9e74c00 con 0x55e3c9df8190 2024-09-11T01:13:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.129+0000 7f66967fc640 1 -- 172.21.15.78:0/2891571130 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a0004300 con 0x55e3c9df8190 2024-09-11T01:13:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.129+0000 7f66967fc640 1 -- 172.21.15.78:0/2891571130 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 61) ==== 8422+0+0 (secure 0 0 0) 0x7f66a8005630 con 0x55e3c9df8190 2024-09-11T01:13:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.129+0000 7f66b7823740 10 client.18300 fetch_fsmap finished waiting for FSMap version 61 2024-09-11T01:13:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.129+0000 7f66b7823740 10 client.18300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:13:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.129+0000 7f66b7823740 20 client.18300 populate_metadata read hostname 'smithi078' 2024-09-11T01:13:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.129+0000 7f66b7823740 1 --2- 172.21.15.78:0/2891571130 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e3c9e75270 0x55e3c9e77690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:13:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.130+0000 7f66b7823740 4 client.18300 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:13:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.130+0000 7f66b7823740 1 -- 172.21.15.78:0/2891571130 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3c9d5b8a0 con 0x55e3c9e75270 2024-09-11T01:13:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.130+0000 7f66aeffd640 1 --2- 172.21.15.78:0/2891571130 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e3c9e75270 0x55e3c9e77690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:13:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.131+0000 7f66967fc640 10 client.18300 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.131+0000 7f66aeffd640 1 --2- 172.21.15.78:0/2891571130 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e3c9e75270 0x55e3c9e77690 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87cb00c9a217ad94 server_cookie=4affe65220d5dd3b in_seq=0 out_seq=0 2024-09-11T01:13:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.131+0000 7f66967fc640 1 -- 172.21.15.78:0/2891571130 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3c9d5b8a0 con 0x55e3c9e75270 2024-09-11T01:13:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.132+0000 7f66967fc640 10 client.18300 handle_command_reply: tid=0 2024-09-11T01:13:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.132+0000 7f66b7823740 1 client.18300 shutdown 2024-09-11T01:13:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.133+0000 7f66b7823740 1 -- 172.21.15.78:0/2891571130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9df8190 msgr2=0x55e3c9e0fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.133+0000 7f66b7823740 1 --2- 172.21.15.78:0/2891571130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9df8190 0x55e3c9e0fdd0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f66a8000f80 tx=0x7f66a8009520 comp rx=0 tx=0).stop 2024-09-11T01:13:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.133+0000 7f66b7823740 1 -- 172.21.15.78:0/2891571130 shutdown_connections 2024-09-11T01:13:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.133+0000 7f66b7823740 1 --2- 172.21.15.78:0/2891571130 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e3c9e75270 0x55e3c9e77690 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.133+0000 7f66b7823740 1 --2- 172.21.15.78:0/2891571130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3c9df8190 0x55e3c9e0fdd0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.133+0000 7f66b7823740 1 --2- 172.21.15.78:0/2891571130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3c9de7df0 0x55e3c9e0f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.133+0000 7f66b7823740 1 --2- 172.21.15.78:0/2891571130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3c9de74d0 0x55e3c9e009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.133+0000 7f66b7823740 1 -- 172.21.15.78:0/2891571130 >> 172.21.15.78:0/2891571130 conn(0x55e3c9de5300 msgr2=0x55e3c9df64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:58.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.133+0000 7f66967fc640 0 client.18300 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:13:58.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.133+0000 7f66b7823740 1 -- 172.21.15.78:0/2891571130 shutdown_connections 2024-09-11T01:13:58.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.133+0000 7f66b7823740 1 -- 172.21.15.78:0/2891571130 wait complete. 2024-09-11T01:13:58.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.133+0000 7f66b7823740 20 client.18300 trim_cache size 0 max 16384 2024-09-11T01:13:58.934 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:13:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.134+0000 7f66b544e640 1 -- 172.21.15.78:0/4217745293 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f669c0b5450 msgr2=0x7f669c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.134+0000 7f66b544e640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f669c0b5450 0x7f669c0b7910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f66a0031440 tx=0x7f66a0005b90 comp rx=0 tx=0).stop 2024-09-11T01:13:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.134+0000 7f66b544e640 1 -- 172.21.15.78:0/4217745293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b01076c0 msgr2=0x7f66b0199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:13:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.134+0000 7f66b544e640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b01076c0 0x7f66b0199810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f669800e9f0 tx=0x7f669800eeb0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.134+0000 7f66b544e640 1 -- 172.21.15.78:0/4217745293 shutdown_connections 2024-09-11T01:13:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.134+0000 7f66b544e640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f669c0b5450 0x7f669c0b7910 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.134+0000 7f66b544e640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66b0108040 0x7f66b019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.134+0000 7f66b544e640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66b01076c0 0x7f66b0199810 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.134+0000 7f66b544e640 1 --2- 172.21.15.78:0/4217745293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0101cb0 0x7f66b01992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:13:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.134+0000 7f66b544e640 1 -- 172.21.15.78:0/4217745293 >> 172.21.15.78:0/4217745293 conn(0x7f66b00fd700 msgr2=0x7f66b00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:13:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.135+0000 7f66b544e640 1 -- 172.21.15.78:0/4217745293 shutdown_connections 2024-09-11T01:13:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:13:58.135+0000 7f66b544e640 1 -- 172.21.15.78:0/4217745293 wait complete. 2024-09-11T01:13:59.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 83743 38.83 MB/sec execute 5 sec latency 309.454 ms 2024-09-11T01:13:59.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:13:59 smithi110 ceph-mon[34712]: pgmap v792: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.8 MiB/s wr, 707 op/s 2024-09-11T01:13:59.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:13:59 smithi112 ceph-mon[35384]: pgmap v792: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.8 MiB/s wr, 707 op/s 2024-09-11T01:13:59.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:13:59 smithi078 ceph-mon[31315]: pgmap v792: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.8 MiB/s wr, 707 op/s 2024-09-11T01:14:00.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 84366 37.33 MB/sec execute 6 sec latency 358.437 ms 2024-09-11T01:14:01.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 84369 32.16 MB/sec execute 7 sec latency 1116.733 ms 2024-09-11T01:14:01.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:01 smithi110 ceph-mon[34712]: pgmap v793: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.0 MiB/s wr, 642 op/s 2024-09-11T01:14:01.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:01 smithi112 ceph-mon[35384]: pgmap v793: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.0 MiB/s wr, 642 op/s 2024-09-11T01:14:01.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:01 smithi078 ceph-mon[31315]: pgmap v793: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.0 MiB/s wr, 642 op/s 2024-09-11T01:14:02.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 84584 29.08 MB/sec execute 8 sec latency 1543.689 ms 2024-09-11T01:14:03.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 84839 27.33 MB/sec execute 9 sec latency 773.961 ms 2024-09-11T01:14:03.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:03 smithi110 ceph-mon[34712]: pgmap v794: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.4 MiB/s wr, 642 op/s 2024-09-11T01:14:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:03 smithi112 ceph-mon[35384]: pgmap v794: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.4 MiB/s wr, 642 op/s 2024-09-11T01:14:03.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:03 smithi078 ceph-mon[31315]: pgmap v794: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.4 MiB/s wr, 642 op/s 2024-09-11T01:14:04.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 85427 27.82 MB/sec execute 10 sec latency 97.832 ms 2024-09-11T01:14:05.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:05 smithi078 ceph-mon[31315]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 683 op/s 2024-09-11T01:14:05.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 86183 29.34 MB/sec execute 11 sec latency 143.725 ms 2024-09-11T01:14:05.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:05 smithi110 ceph-mon[34712]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 683 op/s 2024-09-11T01:14:05.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:05 smithi112 ceph-mon[35384]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 683 op/s 2024-09-11T01:14:06.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 86987 30.89 MB/sec execute 12 sec latency 77.408 ms 2024-09-11T01:14:07.108 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:14:07.108 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:14:07.445 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:14:07.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 87770 31.04 MB/sec execute 13 sec latency 78.540 ms 2024-09-11T01:14:07.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:07 smithi110 ceph-mon[34712]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 866 op/s 2024-09-11T01:14:07.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:07 smithi112 ceph-mon[35384]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 866 op/s 2024-09-11T01:14:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:07 smithi078 ceph-mon[31315]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 866 op/s 2024-09-11T01:14:08.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.494+0000 7f12421ef640 1 -- 172.21.15.78:0/3767428046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f123c10dd70 msgr2=0x7f123c10e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.494+0000 7f12421ef640 1 --2- 172.21.15.78:0/3767428046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f123c10dd70 0x7f123c10e1b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1224009860 tx=0x7f122402a340 comp rx=0 tx=0).stop 2024-09-11T01:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.494+0000 7f12421ef640 1 -- 172.21.15.78:0/3767428046 shutdown_connections 2024-09-11T01:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.494+0000 7f12421ef640 1 --2- 172.21.15.78:0/3767428046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f123c10e6f0 0x7f123c118130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.494+0000 7f12421ef640 1 --2- 172.21.15.78:0/3767428046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f123c10dd70 0x7f123c10e1b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.494+0000 7f12421ef640 1 --2- 172.21.15.78:0/3767428046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f123c10b5c0 0x7f123c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.494+0000 7f12421ef640 1 -- 172.21.15.78:0/3767428046 >> 172.21.15.78:0/3767428046 conn(0x7f123c072570 msgr2=0x7f123c074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.494+0000 7f12421ef640 1 -- 172.21.15.78:0/3767428046 shutdown_connections 2024-09-11T01:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.494+0000 7f12421ef640 1 -- 172.21.15.78:0/3767428046 wait complete. 2024-09-11T01:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.495+0000 7f12421ef640 1 Processor -- start 2024-09-11T01:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.495+0000 7f12421ef640 1 -- start start 2024-09-11T01:14:08.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.495+0000 7f12421ef640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f123c10b5c0 0x7f123c117480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.495+0000 7f12421ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f123c10dd70 0x7f123c1179c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.495+0000 7f123b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f123c10b5c0 0x7f123c117480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.495+0000 7f123b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f123c10b5c0 0x7f123c117480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41058/0 (socket says 172.21.15.78:41058) 2024-09-11T01:14:08.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.495+0000 7f12421ef640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f123c10e6f0 0x7f123c1ab720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.495+0000 7f12421ef640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f123c11b540 con 0x7f123c10e6f0 2024-09-11T01:14:08.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.495+0000 7f12421ef640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f123c11b3c0 con 0x7f123c10b5c0 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.495+0000 7f12421ef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f123c11b6c0 con 0x7f123c10dd70 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.495+0000 7f123b7fe640 1 -- 172.21.15.78:0/2074383201 learned_addr learned my addr 172.21.15.78:0/2074383201 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.496+0000 7f1233fff640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f123c10dd70 0x7f123c1179c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.496+0000 7f123bfff640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f123c10e6f0 0x7f123c1ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.496+0000 7f123b7fe640 1 -- 172.21.15.78:0/2074383201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f123c10dd70 msgr2=0x7f123c1179c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.496+0000 7f123b7fe640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f123c10dd70 0x7f123c1179c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.496+0000 7f123b7fe640 1 -- 172.21.15.78:0/2074383201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f123c10e6f0 msgr2=0x7f123c1ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.496+0000 7f123b7fe640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f123c10e6f0 0x7f123c1ab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.496+0000 7f123b7fe640 1 -- 172.21.15.78:0/2074383201 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123c1abef0 con 0x7f123c10b5c0 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.496+0000 7f123b7fe640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f123c10b5c0 0x7f123c117480 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f122c00e9c0 tx=0x7f122c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.496+0000 7f12397fa640 1 -- 172.21.15.78:0/2074383201 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c00ccc0 con 0x7f123c10b5c0 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.497+0000 7f12421ef640 1 -- 172.21.15.78:0/2074383201 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123c1ac1e0 con 0x7f123c10b5c0 2024-09-11T01:14:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.497+0000 7f12421ef640 1 -- 172.21.15.78:0/2074383201 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123c1b4c60 con 0x7f123c10b5c0 2024-09-11T01:14:08.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.497+0000 7f12397fa640 1 -- 172.21.15.78:0/2074383201 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122c0054d0 con 0x7f123c10b5c0 2024-09-11T01:14:08.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.497+0000 7f12397fa640 1 -- 172.21.15.78:0/2074383201 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c010430 con 0x7f123c10b5c0 2024-09-11T01:14:08.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.500+0000 7f12445c4740 1 Processor -- start 2024-09-11T01:14:08.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.500+0000 7f12445c4740 1 -- start start 2024-09-11T01:14:08.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.500+0000 7f12445c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 0x55fe9eee52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.500+0000 7f12445c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee5800 0x55fe9eef1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.500+0000 7f12445c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9eef3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.500+0000 7f12445c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fe9ee5be60 con 0x55fe9eef1790 2024-09-11T01:14:08.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.500+0000 7f12445c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fe9ee4b290 con 0x55fe9eee5800 2024-09-11T01:14:08.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.500+0000 7f12445c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fe9ee4f0a0 con 0x55fe9eee4ee0 2024-09-11T01:14:08.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.500+0000 7f1233fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9eef3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.500+0000 7f1233fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9eef3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35432/0 (socket says 172.21.15.78:35432) 2024-09-11T01:14:08.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.500+0000 7f1233fff640 1 -- 172.21.15.78:0/4082451722 learned_addr learned my addr 172.21.15.78:0/4082451722 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:08.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.500+0000 7f123bfff640 1 --2- 172.21.15.78:0/4082451722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 0x55fe9eee52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.501+0000 7f123bfff640 1 -- 172.21.15.78:0/4082451722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee5800 msgr2=0x55fe9eef1250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:14:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.501+0000 7f123bfff640 1 --2- 172.21.15.78:0/4082451722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee5800 0x55fe9eef1250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.501+0000 7f123bfff640 1 -- 172.21.15.78:0/4082451722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 msgr2=0x55fe9eef3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.501+0000 7f123bfff640 1 --2- 172.21.15.78:0/4082451722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9eef3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.501+0000 7f123bfff640 1 -- 172.21.15.78:0/4082451722 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe9eba9930 con 0x55fe9eee4ee0 2024-09-11T01:14:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.501+0000 7f12397fa640 1 -- 172.21.15.78:0/2074383201 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f122c0046c0 con 0x7f123c10b5c0 2024-09-11T01:14:08.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.502+0000 7f123bfff640 1 --2- 172.21.15.78:0/4082451722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 0x55fe9eee52c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1234009f60 tx=0x7f1234009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b3bf4708d95ac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.502+0000 7f12397fa640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f12280b54d0 0x7f12280b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.503+0000 7f12397fa640 1 -- 172.21.15.78:0/2074383201 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8493+0+0 (secure 0 0 0) 0x7f122c0e62a0 con 0x7f123c10b5c0 2024-09-11T01:14:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.503+0000 7f1231ffb640 1 -- 172.21.15.78:0/4082451722 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234016070 con 0x55fe9eee4ee0 2024-09-11T01:14:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.503+0000 7f1231ffb640 1 -- 172.21.15.78:0/4082451722 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1234004d20 con 0x55fe9eee4ee0 2024-09-11T01:14:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.503+0000 7f123b7fe640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f12280b54d0 0x7f12280b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.503+0000 7f1231ffb640 1 -- 172.21.15.78:0/4082451722 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234012810 con 0x55fe9eee4ee0 2024-09-11T01:14:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.504+0000 7f12445c4740 1 -- 172.21.15.78:0/4082451722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 msgr2=0x55fe9eee52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.504+0000 7f12445c4740 1 --2- 172.21.15.78:0/4082451722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 0x55fe9eee52c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1234009f60 tx=0x7f1234009f90 comp rx=0 tx=0).stop 2024-09-11T01:14:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.504+0000 7f12445c4740 1 -- 172.21.15.78:0/4082451722 shutdown_connections 2024-09-11T01:14:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.504+0000 7f12445c4740 1 --2- 172.21.15.78:0/4082451722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9eef3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.504+0000 7f12445c4740 1 --2- 172.21.15.78:0/4082451722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee5800 0x55fe9eef1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.504+0000 7f12445c4740 1 --2- 172.21.15.78:0/4082451722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 0x55fe9eee52c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.504+0000 7f12445c4740 1 -- 172.21.15.78:0/4082451722 >> 172.21.15.78:0/4082451722 conn(0x55fe9eee2d10 msgr2=0x55fe9eee3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.504+0000 7f12445c4740 1 -- 172.21.15.78:0/4082451722 shutdown_connections 2024-09-11T01:14:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.504+0000 7f12445c4740 1 -- 172.21.15.78:0/4082451722 wait complete. 2024-09-11T01:14:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.505+0000 7f123b7fe640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f12280b54d0 0x7f12280b7990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55fe9eee8060 tx=0x7f122c009f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.505+0000 7f12445c4740 1 -- start start 2024-09-11T01:14:08.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.505+0000 7f12445c4740 1 Processor -- start 2024-09-11T01:14:08.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f12445c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 0x55fe9eeee7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f12445c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee5800 0x55fe9eeeed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f12445c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9eeef250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f123bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 0x55fe9eeee7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f123bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 0x55fe9eeee7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:58414/0 (socket says 172.21.15.78:58414) 2024-09-11T01:14:08.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f1233fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee5800 0x55fe9eeeed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f123bfff640 1 -- 172.21.15.78:0/3179697983 learned_addr learned my addr 172.21.15.78:0/3179697983 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:08.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f12445c4740 1 -- 172.21.15.78:0/3179697983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fe9ecac890 con 0x55fe9eef1790 2024-09-11T01:14:08.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f12445c4740 1 -- 172.21.15.78:0/3179697983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fe9ede5d40 con 0x55fe9eee5800 2024-09-11T01:14:08.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f12445c4740 1 -- 172.21.15.78:0/3179697983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fe9eef4490 con 0x55fe9eee4ee0 2024-09-11T01:14:08.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f123bfff640 1 --2- 172.21.15.78:0/3179697983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9eeef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f123bfff640 1 -- 172.21.15.78:0/3179697983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee5800 msgr2=0x55fe9eeeed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:08.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f123bfff640 1 --2- 172.21.15.78:0/3179697983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee5800 0x55fe9eeeed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f123bfff640 1 -- 172.21.15.78:0/3179697983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 msgr2=0x55fe9eeef250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f123bfff640 1 --2- 172.21.15.78:0/3179697983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9eeef250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.506+0000 7f123bfff640 1 -- 172.21.15.78:0/3179697983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe9eefd980 con 0x55fe9eee4ee0 2024-09-11T01:14:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.507+0000 7f12445c4740 10 client.0 _ll_register_callbacks cb 0x55fe9ee49c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.507+0000 7f12445c4740 1 -- 172.21.15.78:0/3179697983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1234004d20 con 0x55fe9eee4ee0 2024-09-11T01:14:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.507+0000 7f123bfff640 1 --2- 172.21.15.78:0/3179697983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 0x55fe9eeee7d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1234005b40 tx=0x7f1234005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d4e0f27ca7757f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.507+0000 7f121affd640 10 client.27427 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.507+0000 7f121affd640 1 -- 172.21.15.78:0/3179697983 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234016050 con 0x55fe9eee4ee0 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.507+0000 7f121affd640 1 -- 172.21.15.78:0/3179697983 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8493+0+0 (secure 0 0 0) 0x7f1234012550 con 0x55fe9eee4ee0 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.507+0000 7f121affd640 1 -- 172.21.15.78:0/3179697983 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1234012790 con 0x55fe9eee4ee0 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.507+0000 7f12445c4740 10 client.27427 fetch_fsmap learned FSMap version 61 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.507+0000 7f12445c4740 1 -- 172.21.15.78:0/3179697983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=61}) -- 0x55fe9ef00140 con 0x55fe9eee4ee0 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.508+0000 7f121affd640 1 -- 172.21.15.78:0/3179697983 <== mon.1 v2:172.21.15.112:3300/0 4 ==== fsmap(e 61) ==== 8422+0+0 (secure 0 0 0) 0x7f123400f040 con 0x55fe9eee4ee0 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.508+0000 7f12445c4740 10 client.27427 fetch_fsmap finished waiting for FSMap version 61 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.508+0000 7f12445c4740 10 client.27427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.508+0000 7f12445c4740 20 client.27427 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.508+0000 7f12445c4740 1 --2- 172.21.15.78:0/3179697983 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fe9ef014f0 0x55fe9ef03900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.508+0000 7f12445c4740 4 client.27427 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:14:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.509+0000 7f12445c4740 1 -- 172.21.15.78:0/3179697983 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe9ee5ddb0 con 0x55fe9ef014f0 2024-09-11T01:14:08.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.509+0000 7f1233fff640 1 --2- 172.21.15.78:0/3179697983 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fe9ef014f0 0x55fe9ef03900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.509+0000 7f121affd640 10 client.27427 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:14:08.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.509+0000 7f1233fff640 1 --2- 172.21.15.78:0/3179697983 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fe9ef014f0 0x55fe9ef03900 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=941cd2ca9034f181 server_cookie=e062390e9f9aa1fd in_seq=0 out_seq=0 2024-09-11T01:14:08.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.512+0000 7f121affd640 1 -- 172.21.15.78:0/3179697983 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fe9ee5ddb0 con 0x55fe9ef014f0 2024-09-11T01:14:08.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.512+0000 7f121affd640 10 client.27427 handle_command_reply: tid=0 2024-09-11T01:14:08.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.512+0000 7f12445c4740 1 client.27427 shutdown 2024-09-11T01:14:08.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.512+0000 7f12445c4740 1 -- 172.21.15.78:0/3179697983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 msgr2=0x55fe9eeee7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:08.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.512+0000 7f12445c4740 1 --2- 172.21.15.78:0/3179697983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 0x55fe9eeee7d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1234005b40 tx=0x7f1234005ab0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.513+0000 7f12445c4740 1 -- 172.21.15.78:0/3179697983 shutdown_connections 2024-09-11T01:14:08.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.513+0000 7f12445c4740 1 --2- 172.21.15.78:0/3179697983 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fe9ef014f0 0x55fe9ef03900 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.513+0000 7f12445c4740 1 --2- 172.21.15.78:0/3179697983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9eeef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.513+0000 7f121affd640 0 client.27427 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:14:08.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.513+0000 7f12445c4740 1 --2- 172.21.15.78:0/3179697983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee5800 0x55fe9eeeed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.513+0000 7f12445c4740 1 --2- 172.21.15.78:0/3179697983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee4ee0 0x55fe9eeee7d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.513+0000 7f12445c4740 1 -- 172.21.15.78:0/3179697983 >> 172.21.15.78:0/3179697983 conn(0x55fe9eee2d10 msgr2=0x55fe9eee1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:08.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.513+0000 7f12445c4740 1 -- 172.21.15.78:0/3179697983 shutdown_connections 2024-09-11T01:14:08.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.513+0000 7f12445c4740 1 -- 172.21.15.78:0/3179697983 wait complete. 2024-09-11T01:14:08.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.513+0000 7f12445c4740 20 client.27427 trim_cache size 0 max 16384 2024-09-11T01:14:08.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.535+0000 7f12445c4740 1 Processor -- start 2024-09-11T01:14:08.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.536+0000 7f12445c4740 1 -- start start 2024-09-11T01:14:08.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.536+0000 7f12445c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee4ee0 0x55fe9eeeb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.536+0000 7f12445c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee5800 0x55fe9eeebf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.536+0000 7f12445c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9ef09210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.536+0000 7f12445c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f121c00e760 con 0x55fe9eef1790 2024-09-11T01:14:08.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.536+0000 7f12445c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f121c010450 con 0x55fe9eee4ee0 2024-09-11T01:14:08.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.536+0000 7f12445c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f121c00ee30 con 0x55fe9eee5800 2024-09-11T01:14:08.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.536+0000 7f123bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9ef09210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.536+0000 7f123bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9ef09210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35452/0 (socket says 172.21.15.78:35452) 2024-09-11T01:14:08.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.536+0000 7f123bfff640 1 -- 172.21.15.78:0/3802591939 learned_addr learned my addr 172.21.15.78:0/3802591939 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:08.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.536+0000 7f123bfff640 1 --2- 172.21.15.78:0/3802591939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee4ee0 0x55fe9eeeb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.537+0000 7f1233fff640 1 --2- 172.21.15.78:0/3802591939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee5800 0x55fe9eeebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.537+0000 7f123bfff640 1 -- 172.21.15.78:0/3802591939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee5800 msgr2=0x55fe9eeebf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:08.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.537+0000 7f123bfff640 1 --2- 172.21.15.78:0/3802591939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee5800 0x55fe9eeebf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.537+0000 7f123bfff640 1 -- 172.21.15.78:0/3802591939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 msgr2=0x55fe9ef09210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:08.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.537+0000 7f123bfff640 1 --2- 172.21.15.78:0/3802591939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9ef09210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.537+0000 7f123bfff640 1 -- 172.21.15.78:0/3802591939 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe9eee3790 con 0x55fe9eee4ee0 2024-09-11T01:14:08.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.537+0000 7f123bfff640 1 --2- 172.21.15.78:0/3802591939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee4ee0 0x55fe9eeeb9e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f12340059a0 tx=0x7f1234021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c39547de6cf0b947 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:08.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.537+0000 7f12317fa640 1 -- 172.21.15.78:0/3802591939 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234016070 con 0x55fe9eee4ee0 2024-09-11T01:14:08.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.538+0000 7f12317fa640 1 -- 172.21.15.78:0/3802591939 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1234004d20 con 0x55fe9eee4ee0 2024-09-11T01:14:08.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.538+0000 7f12317fa640 1 -- 172.21.15.78:0/3802591939 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234003970 con 0x55fe9eee4ee0 2024-09-11T01:14:08.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.538+0000 7f12445c4740 1 -- 172.21.15.78:0/3802591939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee4ee0 msgr2=0x55fe9eeeb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:08.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.538+0000 7f12445c4740 1 --2- 172.21.15.78:0/3802591939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee4ee0 0x55fe9eeeb9e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f12340059a0 tx=0x7f1234021000 comp rx=0 tx=0).stop 2024-09-11T01:14:08.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.538+0000 7f12445c4740 1 -- 172.21.15.78:0/3802591939 shutdown_connections 2024-09-11T01:14:08.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.539+0000 7f12445c4740 1 --2- 172.21.15.78:0/3802591939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eef1790 0x55fe9ef09210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.539+0000 7f12445c4740 1 --2- 172.21.15.78:0/3802591939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee5800 0x55fe9eeebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.539+0000 7f12445c4740 1 --2- 172.21.15.78:0/3802591939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eee4ee0 0x55fe9eeeb9e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.539+0000 7f12445c4740 1 -- 172.21.15.78:0/3802591939 >> 172.21.15.78:0/3802591939 conn(0x55fe9eee2d10 msgr2=0x55fe9eef0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:08.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.539+0000 7f12445c4740 1 -- 172.21.15.78:0/3802591939 shutdown_connections 2024-09-11T01:14:08.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.539+0000 7f12445c4740 1 -- 172.21.15.78:0/3802591939 wait complete. 2024-09-11T01:14:08.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.539+0000 7f12445c4740 1 -- start start 2024-09-11T01:14:08.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.539+0000 7f12445c4740 1 Processor -- start 2024-09-11T01:14:08.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f12445c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eee4ee0 0x55fe9eef5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f12445c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee5800 0x55fe9eef6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f12445c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eef1790 0x55fe9ef6d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f12445c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f121c00fcc0 con 0x55fe9eee4ee0 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f12445c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f121c00f530 con 0x55fe9eef1790 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f12445c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f121c00c760 con 0x55fe9eee5800 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f12445c4740 10 client.0 _ll_register_callbacks cb 0x7f122c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f123bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eee4ee0 0x55fe9eef5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f1233fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee5800 0x55fe9eef6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f123bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eee4ee0 0x55fe9eef5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35464/0 (socket says 172.21.15.78:35464) 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f123bfff640 1 -- 172.21.15.78:0/3577399868 learned_addr learned my addr 172.21.15.78:0/3577399868 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f123bfff640 1 --2- 172.21.15.78:0/3577399868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eef1790 0x55fe9ef6d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f123bfff640 1 -- 172.21.15.78:0/3577399868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee5800 msgr2=0x55fe9eef6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f123bfff640 1 --2- 172.21.15.78:0/3577399868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee5800 0x55fe9eef6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f123bfff640 1 -- 172.21.15.78:0/3577399868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eef1790 msgr2=0x55fe9ef6d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f123bfff640 1 --2- 172.21.15.78:0/3577399868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eef1790 0x55fe9ef6d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.540+0000 7f123bfff640 1 -- 172.21.15.78:0/3577399868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe9ef6d980 con 0x55fe9eee4ee0 2024-09-11T01:14:08.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.541+0000 7f123bfff640 1 --2- 172.21.15.78:0/3577399868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eee4ee0 0x55fe9eef5e50 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f12340097e0 tx=0x7f12340059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627992dbad4c7334 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:08.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.541+0000 7f121affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:14:08.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.541+0000 7f121affd640 1 -- 172.21.15.78:0/3577399868 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234016040 con 0x55fe9eee4ee0 2024-09-11T01:14:08.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.541+0000 7f12445c4740 1 -- 172.21.15.78:0/3577399868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1234004d20 con 0x55fe9eee4ee0 2024-09-11T01:14:08.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.541+0000 7f121affd640 1 -- 172.21.15.78:0/3577399868 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8493+0+0 (secure 0 0 0) 0x7f1234003970 con 0x55fe9eee4ee0 2024-09-11T01:14:08.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.541+0000 7f121affd640 1 -- 172.21.15.78:0/3577399868 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1234010c90 con 0x55fe9eee4ee0 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.541+0000 7f12445c4740 10 client.18318 fetch_fsmap learned FSMap version 61 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.541+0000 7f12445c4740 1 -- 172.21.15.78:0/3577399868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55fe9ef6ec10 con 0x55fe9eee4ee0 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.541+0000 7f121affd640 1 -- 172.21.15.78:0/3577399868 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1234004d20 con 0x55fe9eee4ee0 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.542+0000 7f121affd640 1 -- 172.21.15.78:0/3577399868 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 61) ==== 8422+0+0 (secure 0 0 0) 0x7f1234004030 con 0x55fe9eee4ee0 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.542+0000 7f12445c4740 10 client.18318 fetch_fsmap finished waiting for FSMap version 61 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.542+0000 7f12445c4740 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.542+0000 7f12445c4740 20 client.18318 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.542+0000 7f12445c4740 1 --2- 172.21.15.78:0/3577399868 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fe9ef014f0 0x55fe9ef713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.542+0000 7f12445c4740 4 client.18318 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.542+0000 7f12445c4740 1 -- 172.21.15.78:0/3577399868 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fe9eeef740 con 0x55fe9ef014f0 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.543+0000 7f1233fff640 1 --2- 172.21.15.78:0/3577399868 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fe9ef014f0 0x55fe9ef713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.543+0000 7f1233fff640 1 --2- 172.21.15.78:0/3577399868 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fe9ef014f0 0x55fe9ef713a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8065c0f12f2767e9 server_cookie=6873782f47e27df8 in_seq=0 out_seq=0 2024-09-11T01:14:08.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.543+0000 7f121affd640 10 client.18318 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:14:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.546+0000 7f121affd640 1 -- 172.21.15.78:0/3577399868 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fe9eeef740 con 0x55fe9ef014f0 2024-09-11T01:14:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.546+0000 7f121affd640 10 client.18318 handle_command_reply: tid=0 2024-09-11T01:14:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.546+0000 7f12445c4740 1 client.18318 shutdown 2024-09-11T01:14:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.546+0000 7f12445c4740 1 -- 172.21.15.78:0/3577399868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eee4ee0 msgr2=0x55fe9eef5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.546+0000 7f12445c4740 1 --2- 172.21.15.78:0/3577399868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eee4ee0 0x55fe9eef5e50 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f12340097e0 tx=0x7f12340059f0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.546+0000 7f12445c4740 1 -- 172.21.15.78:0/3577399868 shutdown_connections 2024-09-11T01:14:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.546+0000 7f12445c4740 1 --2- 172.21.15.78:0/3577399868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe9eef1790 0x55fe9ef6d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.546+0000 7f12445c4740 1 --2- 172.21.15.78:0/3577399868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe9eee5800 0x55fe9eef6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.546+0000 7f12445c4740 1 --2- 172.21.15.78:0/3577399868 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fe9ef014f0 0x55fe9ef713a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.546+0000 7f12445c4740 1 --2- 172.21.15.78:0/3577399868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe9eee4ee0 0x55fe9eef5e50 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.546+0000 7f121affd640 0 client.18318 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:14:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.546+0000 7f12445c4740 1 -- 172.21.15.78:0/3577399868 >> 172.21.15.78:0/3577399868 conn(0x55fe9eee2d10 msgr2=0x55fe9eee8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:08.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.547+0000 7f12445c4740 1 -- 172.21.15.78:0/3577399868 shutdown_connections 2024-09-11T01:14:08.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.547+0000 7f12445c4740 1 -- 172.21.15.78:0/3577399868 wait complete. 2024-09-11T01:14:08.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.547+0000 7f12445c4740 20 client.18318 trim_cache size 0 max 16384 2024-09-11T01:14:08.550 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:14:08.550 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:14:08.550 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "4502de91-9ca6-47fc-b564-9b5f99d24eb1", 2024-09-11T01:14:08.550 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:14:08.550 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:14:08.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.548+0000 7f12421ef640 1 -- 172.21.15.78:0/2074383201 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f12280b54d0 msgr2=0x7f12280b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:08.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.548+0000 7f12421ef640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f12280b54d0 0x7f12280b7990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55fe9eee8060 tx=0x7f122c009f90 comp rx=0 tx=0).stop 2024-09-11T01:14:08.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.548+0000 7f12421ef640 1 -- 172.21.15.78:0/2074383201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f123c10b5c0 msgr2=0x7f123c117480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:08.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.548+0000 7f12421ef640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f123c10b5c0 0x7f123c117480 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f122c00e9c0 tx=0x7f122c00ee80 comp rx=0 tx=0).stop 2024-09-11T01:14:08.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.548+0000 7f12421ef640 1 -- 172.21.15.78:0/2074383201 shutdown_connections 2024-09-11T01:14:08.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.548+0000 7f12421ef640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f12280b54d0 0x7f12280b7990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.548+0000 7f12421ef640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f123c10e6f0 0x7f123c1ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.548+0000 7f12421ef640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f123c10dd70 0x7f123c1179c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.548+0000 7f12421ef640 1 --2- 172.21.15.78:0/2074383201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f123c10b5c0 0x7f123c117480 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:08.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.548+0000 7f12421ef640 1 -- 172.21.15.78:0/2074383201 >> 172.21.15.78:0/2074383201 conn(0x7f123c072570 msgr2=0x7f123c10f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:08.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.548+0000 7f12421ef640 1 -- 172.21.15.78:0/2074383201 shutdown_connections 2024-09-11T01:14:08.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:08.548+0000 7f12421ef640 1 -- 172.21.15.78:0/2074383201 wait complete. 2024-09-11T01:14:08.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:08 smithi078 ceph-mon[31315]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T01:14:08.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:08 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:14:08.780 INFO:tasks.workunit.client.0.smithi078.stdout: 10 88474 32.01 MB/sec execute 14 sec latency 204.965 ms 2024-09-11T01:14:08.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:08 smithi110 ceph-mon[34712]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T01:14:08.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:08 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:14:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:08 smithi112 ceph-mon[35384]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T01:14:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:08 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:14:09.512 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:14:09.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:09 smithi078 ceph-mon[31315]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 915 op/s 2024-09-11T01:14:09.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:09 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:14:09.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:09 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:09 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:14:09.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:09 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:14:09.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:09 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:14:09.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:09 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:14:09.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:09 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:09 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:09 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:09 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 89324 32.02 MB/sec execute 15 sec latency 286.237 ms 2024-09-11T01:14:09.898 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:14:09.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:09 smithi110 ceph-mon[34712]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 915 op/s 2024-09-11T01:14:09.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:09 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:14:09.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:09 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:09 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:14:09.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:09 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:14:09.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:09 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:14:09.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:09 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:14:09.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:09 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:09 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:09 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:09 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:09 smithi112 ceph-mon[35384]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 915 op/s 2024-09-11T01:14:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:09 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:14:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:09 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:09 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:14:09.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:09 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:14:09.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:09 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:14:09.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:09 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:14:09.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:09 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:09 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:09 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:09.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:09 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:10.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 90265 34.11 MB/sec execute 16 sec latency 93.559 ms 2024-09-11T01:14:10.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:10 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:10.954 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:10 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:10.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.951+0000 7fec9c105640 1 -- 172.21.15.78:0/2451371201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec94108060 msgr2=0x7fec9410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:10.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.951+0000 7fec9c105640 1 --2- 172.21.15.78:0/2451371201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec94108060 0x7fec9410e860 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00bc00 tx=0x7fec8c009260 comp rx=0 tx=0).stop 2024-09-11T01:14:10.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.952+0000 7fec9c105640 1 -- 172.21.15.78:0/2451371201 shutdown_connections 2024-09-11T01:14:10.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.952+0000 7fec9c105640 1 --2- 172.21.15.78:0/2451371201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec94108060 0x7fec9410e860 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.952+0000 7fec9c105640 1 --2- 172.21.15.78:0/2451371201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec941076e0 0x7fec94107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.952+0000 7fec9c105640 1 --2- 172.21.15.78:0/2451371201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec94101cd0 0x7fec941020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.952+0000 7fec9c105640 1 -- 172.21.15.78:0/2451371201 >> 172.21.15.78:0/2451371201 conn(0x7fec940fd740 msgr2=0x7fec940ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:10.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.952+0000 7fec9c105640 1 -- 172.21.15.78:0/2451371201 shutdown_connections 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.952+0000 7fec9c105640 1 -- 172.21.15.78:0/2451371201 wait complete. 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.953+0000 7fec9c105640 1 Processor -- start 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.953+0000 7fec9c105640 1 -- start start 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.953+0000 7fec9c105640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec94101cd0 0x7fec94196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.953+0000 7fec9c105640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec941076e0 0x7fec941973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.953+0000 7fec9c105640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec94108060 0x7fec9419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.953+0000 7fec9c105640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec94110c60 con 0x7fec941076e0 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.953+0000 7fec9c105640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec94110ae0 con 0x7fec94101cd0 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.953+0000 7fec9c105640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec94110de0 con 0x7fec94108060 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.954+0000 7fec99ea1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec94101cd0 0x7fec94196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.954+0000 7fec996a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec941076e0 0x7fec941973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.954+0000 7fec99ea1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec94101cd0 0x7fec94196e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41080/0 (socket says 172.21.15.78:41080) 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.954+0000 7fec99ea1640 1 -- 172.21.15.78:0/3473766597 learned_addr learned my addr 172.21.15.78:0/3473766597 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:10.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.954+0000 7fec9a6a2640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec94108060 0x7fec9419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:10.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.954+0000 7fec9a6a2640 1 -- 172.21.15.78:0/3473766597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec94101cd0 msgr2=0x7fec94196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:10.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.954+0000 7fec9a6a2640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec94101cd0 0x7fec94196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.954+0000 7fec9a6a2640 1 -- 172.21.15.78:0/3473766597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec941076e0 msgr2=0x7fec941973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:10.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.954+0000 7fec9a6a2640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec941076e0 0x7fec941973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.954+0000 7fec9a6a2640 1 -- 172.21.15.78:0/3473766597 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9419cf20 con 0x7fec94108060 2024-09-11T01:14:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.955+0000 7fec9a6a2640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec94108060 0x7fec9419c750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fec8c034040 tx=0x7fec8c0329c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.955+0000 7fec86ffd640 1 -- 172.21.15.78:0/3473766597 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c041020 con 0x7fec94108060 2024-09-11T01:14:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.955+0000 7fec9c105640 1 -- 172.21.15.78:0/3473766597 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9419d1b0 con 0x7fec94108060 2024-09-11T01:14:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.955+0000 7fec9c105640 1 -- 172.21.15.78:0/3473766597 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec941ac0e0 con 0x7fec94108060 2024-09-11T01:14:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.955+0000 7fec86ffd640 1 -- 172.21.15.78:0/3473766597 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec8c003c40 con 0x7fec94108060 2024-09-11T01:14:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.955+0000 7fec86ffd640 1 -- 172.21.15.78:0/3473766597 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c03c850 con 0x7fec94108060 2024-09-11T01:14:10.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.959+0000 7fec86ffd640 1 -- 172.21.15.78:0/3473766597 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fec8c03c9f0 con 0x7fec94108060 2024-09-11T01:14:10.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.961+0000 7fec9e4da740 1 Processor -- start 2024-09-11T01:14:10.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.961+0000 7fec9e4da740 1 -- start start 2024-09-11T01:14:10.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.961+0000 7fec9e4da740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4af9480 0x55dab4af9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:10.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.961+0000 7fec9e4da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9da0 0x55dab4b057f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.961+0000 7fec9e4da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4b05d30 0x55dab4b08120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.961+0000 7fec9e4da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dab4a6fe50 con 0x55dab4b05d30 2024-09-11T01:14:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.961+0000 7fec9e4da740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dab4a5f280 con 0x55dab4af9480 2024-09-11T01:14:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.961+0000 7fec9e4da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dab4a63090 con 0x55dab4af9da0 2024-09-11T01:14:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.962+0000 7fec99ea1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9da0 0x55dab4b057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.962+0000 7fec99ea1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9da0 0x55dab4b057f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:58464/0 (socket says 172.21.15.78:58464) 2024-09-11T01:14:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.962+0000 7fec99ea1640 1 -- 172.21.15.78:0/1958433143 learned_addr learned my addr 172.21.15.78:0/1958433143 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.962+0000 7fec996a0640 1 --2- 172.21.15.78:0/1958433143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4af9480 0x55dab4af9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.962+0000 7fec996a0640 1 --2- 172.21.15.78:0/1958433143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4b05d30 0x55dab4b08120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.962+0000 7fec99ea1640 1 -- 172.21.15.78:0/1958433143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4af9480 msgr2=0x55dab4af9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.962+0000 7fec99ea1640 1 --2- 172.21.15.78:0/1958433143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4af9480 0x55dab4af9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.962+0000 7fec99ea1640 1 -- 172.21.15.78:0/1958433143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4b05d30 msgr2=0x55dab4b08120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:10.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.962+0000 7fec99ea1640 1 --2- 172.21.15.78:0/1958433143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4b05d30 0x55dab4b08120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.962+0000 7fec99ea1640 1 -- 172.21.15.78:0/1958433143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab47bd650 con 0x55dab4af9da0 2024-09-11T01:14:10.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.963+0000 7fec99ea1640 1 --2- 172.21.15.78:0/1958433143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9da0 0x55dab4b057f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fec8800dfb0 tx=0x7fec88005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea6a2ae016a4c756 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:10.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.963+0000 7fec86ffd640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fec800b54d0 0x7fec800b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:10.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.963+0000 7fec86ffd640 1 -- 172.21.15.78:0/3473766597 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8493+0+0 (secure 0 0 0) 0x7fec8c106420 con 0x7fec94108060 2024-09-11T01:14:10.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.964+0000 7fec996a0640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fec800b54d0 0x7fec800b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:10.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.964+0000 7fec7ffff640 1 -- 172.21.15.78:0/1958433143 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8800ce60 con 0x55dab4af9da0 2024-09-11T01:14:10.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.964+0000 7fec7ffff640 1 -- 172.21.15.78:0/1958433143 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec880288b0 con 0x55dab4af9da0 2024-09-11T01:14:10.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.964+0000 7fec7ffff640 1 -- 172.21.15.78:0/1958433143 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec880278d0 con 0x55dab4af9da0 2024-09-11T01:14:10.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.965+0000 7fec996a0640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fec800b54d0 0x7fec800b7990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fec90009820 tx=0x7fec90013040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:10.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.965+0000 7fec9e4da740 1 -- 172.21.15.78:0/1958433143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9da0 msgr2=0x55dab4b057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:10.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.965+0000 7fec9e4da740 1 --2- 172.21.15.78:0/1958433143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9da0 0x55dab4b057f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fec8800dfb0 tx=0x7fec88005f90 comp rx=0 tx=0).stop 2024-09-11T01:14:10.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.966+0000 7fec9e4da740 1 -- 172.21.15.78:0/1958433143 shutdown_connections 2024-09-11T01:14:10.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.966+0000 7fec9e4da740 1 --2- 172.21.15.78:0/1958433143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4b05d30 0x55dab4b08120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.966+0000 7fec9e4da740 1 --2- 172.21.15.78:0/1958433143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9da0 0x55dab4b057f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.966+0000 7fec9e4da740 1 --2- 172.21.15.78:0/1958433143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4af9480 0x55dab4af9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.966+0000 7fec9e4da740 1 -- 172.21.15.78:0/1958433143 >> 172.21.15.78:0/1958433143 conn(0x55dab4af72b0 msgr2=0x55dab4af8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:10.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.966+0000 7fec9e4da740 1 -- 172.21.15.78:0/1958433143 shutdown_connections 2024-09-11T01:14:10.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.969+0000 7fec9e4da740 1 -- 172.21.15.78:0/1958433143 wait complete. 2024-09-11T01:14:10.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.970+0000 7fec9e4da740 1 -- start start 2024-09-11T01:14:10.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.970+0000 7fec9e4da740 1 Processor -- start 2024-09-11T01:14:10.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.970+0000 7fec9e4da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9480 0x55dab4b05520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.970+0000 7fec9e4da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9da0 0x55dab4b02970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.970+0000 7fec9e4da740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4b05d30 0x55dab4b02eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.970+0000 7fec9e4da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dab48c0700 con 0x55dab4af9da0 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.970+0000 7fec9e4da740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dab49fa110 con 0x55dab4b05d30 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.970+0000 7fec9e4da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dab4b08a00 con 0x55dab4af9480 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.970+0000 7fec9e4da740 10 client.0 _ll_register_callbacks cb 0x55dab4a13990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.971+0000 7fec99ea1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9480 0x55dab4b05520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.971+0000 7fec99ea1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9480 0x55dab4b05520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:58468/0 (socket says 172.21.15.78:58468) 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.971+0000 7fec99ea1640 1 -- 172.21.15.78:0/3751934442 learned_addr learned my addr 172.21.15.78:0/3751934442 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.971+0000 7fec9a6a2640 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4b05d30 0x55dab4b02eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.971+0000 7fec996a0640 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9da0 0x55dab4b02970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.971+0000 7fec99ea1640 1 -- 172.21.15.78:0/3751934442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4b05d30 msgr2=0x55dab4b02eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.971+0000 7fec99ea1640 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4b05d30 0x55dab4b02eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.971+0000 7fec99ea1640 1 -- 172.21.15.78:0/3751934442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9da0 msgr2=0x55dab4b02970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:10.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.971+0000 7fec99ea1640 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9da0 0x55dab4b02970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.971+0000 7fec99ea1640 1 -- 172.21.15.78:0/3751934442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab4a27c40 con 0x55dab4af9480 2024-09-11T01:14:10.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.971+0000 7fec99ea1640 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9480 0x55dab4b05520 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec8800f010 tx=0x7fec8800d490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a2db950f70d6783 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:10.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.971+0000 7fec7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:14:10.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.972+0000 7fec7d7fa640 1 -- 172.21.15.78:0/3751934442 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88036070 con 0x55dab4af9480 2024-09-11T01:14:10.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.972+0000 7fec9e4da740 1 -- 172.21.15.78:0/3751934442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec880288b0 con 0x55dab4af9480 2024-09-11T01:14:10.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.972+0000 7fec7d7fa640 1 -- 172.21.15.78:0/3751934442 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8493+0+0 (secure 0 0 0) 0x7fec88005450 con 0x55dab4af9480 2024-09-11T01:14:10.976 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:10 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:14:10.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.972+0000 7fec7d7fa640 1 -- 172.21.15.78:0/3751934442 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fec8800ce60 con 0x55dab4af9480 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.972+0000 7fec9e4da740 10 client.27445 fetch_fsmap learned FSMap version 61 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.972+0000 7fec9e4da740 1 -- 172.21.15.78:0/3751934442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55dab4b142c0 con 0x55dab4af9480 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.972+0000 7fec7d7fa640 1 -- 172.21.15.78:0/3751934442 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec880288b0 con 0x55dab4af9480 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.973+0000 7fec7d7fa640 1 -- 172.21.15.78:0/3751934442 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 61) ==== 8422+0+0 (secure 0 0 0) 0x7fec88016070 con 0x55dab4af9480 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.973+0000 7fec9e4da740 10 client.27445 fetch_fsmap finished waiting for FSMap version 61 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.973+0000 7fec9e4da740 10 client.27445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.973+0000 7fec9e4da740 20 client.27445 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.973+0000 7fec9e4da740 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dab4b15610 0x55dab4b17a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.973+0000 7fec9e4da740 4 client.27445 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.973+0000 7fec9e4da740 1 -- 172.21.15.78:0/3751934442 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dab4a71da0 con 0x55dab4b15610 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.973+0000 7fec996a0640 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dab4b15610 0x55dab4b17a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.974+0000 7fec7d7fa640 10 client.27445 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:14:10.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.974+0000 7fec996a0640 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dab4b15610 0x55dab4b17a00 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27562d30f809ced0 server_cookie=a8f120383932b2ab in_seq=0 out_seq=0 2024-09-11T01:14:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.975+0000 7fec7d7fa640 1 -- 172.21.15.78:0/3751934442 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dab4a71da0 con 0x55dab4b15610 2024-09-11T01:14:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.975+0000 7fec7d7fa640 10 client.27445 handle_command_reply: tid=0 2024-09-11T01:14:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.975+0000 7fec9e4da740 1 client.27445 shutdown 2024-09-11T01:14:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.975+0000 7fec9e4da740 1 -- 172.21.15.78:0/3751934442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9480 msgr2=0x55dab4b05520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.975+0000 7fec9e4da740 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9480 0x55dab4b05520 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec8800f010 tx=0x7fec8800d490 comp rx=0 tx=0).stop 2024-09-11T01:14:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.976+0000 7fec9e4da740 1 -- 172.21.15.78:0/3751934442 shutdown_connections 2024-09-11T01:14:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.976+0000 7fec9e4da740 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4b05d30 0x55dab4b02eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.976+0000 7fec9e4da740 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dab4b15610 0x55dab4b17a00 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.976+0000 7fec9e4da740 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9da0 0x55dab4b02970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.976+0000 7fec9e4da740 1 --2- 172.21.15.78:0/3751934442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9480 0x55dab4b05520 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.976+0000 7fec9e4da740 1 -- 172.21.15.78:0/3751934442 >> 172.21.15.78:0/3751934442 conn(0x55dab4af72b0 msgr2=0x55dab4afce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:10.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.976+0000 7fec7d7fa640 0 client.27445 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:14:10.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.976+0000 7fec9e4da740 1 -- 172.21.15.78:0/3751934442 shutdown_connections 2024-09-11T01:14:10.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.976+0000 7fec9e4da740 1 -- 172.21.15.78:0/3751934442 wait complete. 2024-09-11T01:14:10.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:10.976+0000 7fec9e4da740 20 client.27445 trim_cache size 0 max 16384 2024-09-11T01:14:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.001+0000 7fec9e4da740 1 Processor -- start 2024-09-11T01:14:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.001+0000 7fec9e4da740 1 -- start start 2024-09-11T01:14:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.001+0000 7fec9e4da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9480 0x55dab4af6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.001+0000 7fec9e4da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9da0 0x55dab4af8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.001+0000 7fec9e4da740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4b05d30 0x55dab4affb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.001+0000 7fec9e4da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec7800e730 con 0x55dab4af9da0 2024-09-11T01:14:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.001+0000 7fec9e4da740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec78010420 con 0x55dab4b05d30 2024-09-11T01:14:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.001+0000 7fec9e4da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec7800ee00 con 0x55dab4af9480 2024-09-11T01:14:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.001+0000 7fec996a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9da0 0x55dab4af8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:11.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.001+0000 7fec996a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9da0 0x55dab4af8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35484/0 (socket says 172.21.15.78:35484) 2024-09-11T01:14:11.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.001+0000 7fec996a0640 1 -- 172.21.15.78:0/1744544961 learned_addr learned my addr 172.21.15.78:0/1744544961 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:11.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.001+0000 7fec99ea1640 1 --2- 172.21.15.78:0/1744544961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9480 0x55dab4af6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:11.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.002+0000 7fec996a0640 1 -- 172.21.15.78:0/1744544961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9480 msgr2=0x55dab4af6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:11.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.002+0000 7fec996a0640 1 --2- 172.21.15.78:0/1744544961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9480 0x55dab4af6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.002+0000 7fec996a0640 1 -- 172.21.15.78:0/1744544961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4b05d30 msgr2=0x55dab4affb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:11.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.002+0000 7fec996a0640 1 --2- 172.21.15.78:0/1744544961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4b05d30 0x55dab4affb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.002+0000 7fec996a0640 1 -- 172.21.15.78:0/1744544961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab4af7d30 con 0x55dab4af9da0 2024-09-11T01:14:11.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.002+0000 7fec996a0640 1 --2- 172.21.15.78:0/1744544961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9da0 0x55dab4af8a10 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fec940fff10 tx=0x7fec900097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6385e335c7f9bc4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.002+0000 7fec7f7fe640 1 -- 172.21.15.78:0/1744544961 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90004020 con 0x55dab4af9da0 2024-09-11T01:14:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.002+0000 7fec7f7fe640 1 -- 172.21.15.78:0/1744544961 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec90020d40 con 0x55dab4af9da0 2024-09-11T01:14:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.002+0000 7fec7f7fe640 1 -- 172.21.15.78:0/1744544961 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90003900 con 0x55dab4af9da0 2024-09-11T01:14:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.003+0000 7fec9e4da740 1 -- 172.21.15.78:0/1744544961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9da0 msgr2=0x55dab4af8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.003+0000 7fec9e4da740 1 --2- 172.21.15.78:0/1744544961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9da0 0x55dab4af8a10 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fec940fff10 tx=0x7fec900097b0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.004+0000 7fec9e4da740 1 -- 172.21.15.78:0/1744544961 shutdown_connections 2024-09-11T01:14:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.004+0000 7fec9e4da740 1 --2- 172.21.15.78:0/1744544961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4b05d30 0x55dab4affb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.004+0000 7fec9e4da740 1 --2- 172.21.15.78:0/1744544961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9da0 0x55dab4af8a10 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.004+0000 7fec9e4da740 1 --2- 172.21.15.78:0/1744544961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4af9480 0x55dab4af6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.004+0000 7fec9e4da740 1 -- 172.21.15.78:0/1744544961 >> 172.21.15.78:0/1744544961 conn(0x55dab4af72b0 msgr2=0x55dab4af8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:11.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.004+0000 7fec9e4da740 1 -- 172.21.15.78:0/1744544961 shutdown_connections 2024-09-11T01:14:11.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.004+0000 7fec9e4da740 1 -- 172.21.15.78:0/1744544961 wait complete. 2024-09-11T01:14:11.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.004+0000 7fec9e4da740 1 -- start start 2024-09-11T01:14:11.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.005+0000 7fec9e4da740 1 Processor -- start 2024-09-11T01:14:11.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.005+0000 7fec9e4da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9480 0x55dab4b09fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:11.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.005+0000 7fec9e4da740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4af9da0 0x55dab4b0a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:11.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec9e4da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4b05d30 0x55dab4b92380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:11.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec9e4da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec7800fc90 con 0x55dab4af9480 2024-09-11T01:14:11.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec9e4da740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec7800f500 con 0x55dab4af9da0 2024-09-11T01:14:11.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec9e4da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec7800c730 con 0x55dab4b05d30 2024-09-11T01:14:11.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec9e4da740 10 client.0 _ll_register_callbacks cb 0x7fec90009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:11.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec9a6a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4b05d30 0x55dab4b92380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:11.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec996a0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4af9da0 0x55dab4b0a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:11.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec996a0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4af9da0 0x55dab4b0a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:41114/0 (socket says 172.21.15.78:41114) 2024-09-11T01:14:11.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec9a6a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4b05d30 0x55dab4b92380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:58492/0 (socket says 172.21.15.78:58492) 2024-09-11T01:14:11.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec996a0640 1 -- 172.21.15.78:0/157433550 learned_addr learned my addr 172.21.15.78:0/157433550 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:11.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec996a0640 1 -- 172.21.15.78:0/157433550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4b05d30 msgr2=0x55dab4b92380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec996a0640 1 --2- 172.21.15.78:0/157433550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4b05d30 0x55dab4b92380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec996a0640 1 -- 172.21.15.78:0/157433550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9480 msgr2=0x55dab4b09fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec996a0640 1 --2- 172.21.15.78:0/157433550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9480 0x55dab4b09fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec996a0640 1 -- 172.21.15.78:0/157433550 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab4b928c0 con 0x55dab4af9da0 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.006+0000 7fec9a6a2640 1 --2- 172.21.15.78:0/157433550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4b05d30 0x55dab4b92380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.007+0000 7fec996a0640 1 --2- 172.21.15.78:0/157433550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4af9da0 0x55dab4b0a500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec90013010 tx=0x7fec90020ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d31d1d3cec77910 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.007+0000 7fec7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.007+0000 7fec7d7fa640 1 -- 172.21.15.78:0/157433550 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9003a060 con 0x55dab4af9da0 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.007+0000 7fec7d7fa640 1 -- 172.21.15.78:0/157433550 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8493+0+0 (secure 0 0 0) 0x7fec90005240 con 0x55dab4af9da0 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.007+0000 7fec9e4da740 1 -- 172.21.15.78:0/157433550 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec90020d40 con 0x55dab4af9da0 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.007+0000 7fec7d7fa640 1 -- 172.21.15.78:0/157433550 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fec90005480 con 0x55dab4af9da0 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.007+0000 7fec9e4da740 10 client.27485 fetch_fsmap learned FSMap version 61 2024-09-11T01:14:11.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.007+0000 7fec9e4da740 1 -- 172.21.15.78:0/157433550 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55dab4b93a20 con 0x55dab4af9da0 2024-09-11T01:14:11.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.008+0000 7fec7d7fa640 1 -- 172.21.15.78:0/157433550 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec90020d40 con 0x55dab4af9da0 2024-09-11T01:14:11.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.008+0000 7fec7d7fa640 1 -- 172.21.15.78:0/157433550 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 61) ==== 8422+0+0 (secure 0 0 0) 0x7fec9002b560 con 0x55dab4af9da0 2024-09-11T01:14:11.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.009+0000 7fec9e4da740 10 client.27485 fetch_fsmap finished waiting for FSMap version 61 2024-09-11T01:14:11.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.009+0000 7fec9e4da740 10 client.27485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:14:11.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.009+0000 7fec9e4da740 20 client.27485 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:11.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.009+0000 7fec9e4da740 1 --2- 172.21.15.78:0/157433550 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dab4b94130 0x55dab4b96550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:11.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.009+0000 7fec9e4da740 4 client.27485 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:14:11.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.009+0000 7fec9e4da740 1 -- 172.21.15.78:0/157433550 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dab4a6d850 con 0x55dab4b94130 2024-09-11T01:14:11.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.009+0000 7fec99ea1640 1 --2- 172.21.15.78:0/157433550 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dab4b94130 0x55dab4b96550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:11.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.009+0000 7fec99ea1640 1 --2- 172.21.15.78:0/157433550 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dab4b94130 0x55dab4b96550 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33a4ba28391bfb0 server_cookie=1c731e15a1ac5332 in_seq=0 out_seq=0 2024-09-11T01:14:11.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.009+0000 7fec7d7fa640 10 client.27485 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:14:11.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec7d7fa640 1 -- 172.21.15.78:0/157433550 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5966 (crc 0 0 0) 0x55dab4a6d850 con 0x55dab4b94130 2024-09-11T01:14:11.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec7d7fa640 10 client.27485 handle_command_reply: tid=0 2024-09-11T01:14:11.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec9e4da740 1 client.27485 shutdown 2024-09-11T01:14:11.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec9e4da740 1 -- 172.21.15.78:0/157433550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4af9da0 msgr2=0x55dab4b0a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:11.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec9e4da740 1 --2- 172.21.15.78:0/157433550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4af9da0 0x55dab4b0a500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec90013010 tx=0x7fec90020ac0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec9e4da740 1 -- 172.21.15.78:0/157433550 shutdown_connections 2024-09-11T01:14:11.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec9e4da740 1 --2- 172.21.15.78:0/157433550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dab4b05d30 0x55dab4b92380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec9e4da740 1 --2- 172.21.15.78:0/157433550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dab4af9da0 0x55dab4b0a500 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec9e4da740 1 --2- 172.21.15.78:0/157433550 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dab4b94130 0x55dab4b96550 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec9e4da740 1 --2- 172.21.15.78:0/157433550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dab4af9480 0x55dab4b09fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec7d7fa640 0 client.27485 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:14:11.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec9e4da740 1 -- 172.21.15.78:0/157433550 >> 172.21.15.78:0/157433550 conn(0x55dab4af72b0 msgr2=0x55dab4b03fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:11.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec9e4da740 1 -- 172.21.15.78:0/157433550 shutdown_connections 2024-09-11T01:14:11.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec9e4da740 1 -- 172.21.15.78:0/157433550 wait complete. 2024-09-11T01:14:11.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.011+0000 7fec9e4da740 20 client.27485 trim_cache size 0 max 16384 2024-09-11T01:14:11.015 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:14:11.015 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:14:11.015 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:14:11.015 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:14:11.015 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.015 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "4502de91-9ca6-47fc-b564-9b5f99d24eb1", 2024-09-11T01:14:11.015 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:14:11.015 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:14:11.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.016 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:14:11.016 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:14:11.016 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:14:11.016 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.016 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.016 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.016 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.016 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.016 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:14:11.016 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:14:11.016 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.018 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.019 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:14:11.020 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.022 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.023 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.024 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.024 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.024 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.024 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:14:11.024 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:14:11.024 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.024 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.024 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.024 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.024 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.025 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.026 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:14:11.027 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:14:11.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.012+0000 7fec9c105640 1 -- 172.21.15.78:0/3473766597 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fec800b54d0 msgr2=0x7fec800b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:11.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.012+0000 7fec9c105640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fec800b54d0 0x7fec800b7990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fec90009820 tx=0x7fec90013040 comp rx=0 tx=0).stop 2024-09-11T01:14:11.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.012+0000 7fec9c105640 1 -- 172.21.15.78:0/3473766597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec94108060 msgr2=0x7fec9419c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:11.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.012+0000 7fec9c105640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec94108060 0x7fec9419c750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fec8c034040 tx=0x7fec8c0329c0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.012+0000 7fec9c105640 1 -- 172.21.15.78:0/3473766597 shutdown_connections 2024-09-11T01:14:11.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.012+0000 7fec9c105640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fec800b54d0 0x7fec800b7990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.012+0000 7fec9c105640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec94108060 0x7fec9419c750 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.012+0000 7fec9c105640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec941076e0 0x7fec941973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.012+0000 7fec9c105640 1 --2- 172.21.15.78:0/3473766597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec94101cd0 0x7fec94196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:11.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.012+0000 7fec9c105640 1 -- 172.21.15.78:0/3473766597 >> 172.21.15.78:0/3473766597 conn(0x7fec940fd740 msgr2=0x7fec940fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:11.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.012+0000 7fec9c105640 1 -- 172.21.15.78:0/3473766597 shutdown_connections 2024-09-11T01:14:11.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:11.012+0000 7fec9c105640 1 -- 172.21.15.78:0/3473766597 wait complete. 2024-09-11T01:14:11.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 91179 34.81 MB/sec execute 17 sec latency 49.154 ms 2024-09-11T01:14:11.885 INFO:tasks.cephfs.filesystem:scrub status for tag:4502de91-9ca6-47fc-b564-9b5f99d24eb1 - {'path': '/', 'tag': '4502de91-9ca6-47fc-b564-9b5f99d24eb1', 'options': 'recursive,force'} 2024-09-11T01:14:11.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:11 smithi110 ceph-mon[34712]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 940 op/s 2024-09-11T01:14:11.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:11 smithi112 ceph-mon[35384]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 940 op/s 2024-09-11T01:14:11.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:11 smithi078 ceph-mon[31315]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 940 op/s 2024-09-11T01:14:12.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 92037 35.85 MB/sec execute 18 sec latency 46.391 ms 2024-09-11T01:14:13.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 92951 35.60 MB/sec execute 19 sec latency 55.544 ms 2024-09-11T01:14:13.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:13 smithi110 ceph-mon[34712]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 956 op/s 2024-09-11T01:14:13.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:13 smithi110 ceph-mon[34712]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T01:14:13.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:13 smithi112 ceph-mon[35384]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 956 op/s 2024-09-11T01:14:13.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:13 smithi112 ceph-mon[35384]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T01:14:13.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:13 smithi078 ceph-mon[31315]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 956 op/s 2024-09-11T01:14:13.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:13 smithi078 ceph-mon[31315]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T01:14:14.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 94101 37.71 MB/sec execute 20 sec latency 56.541 ms 2024-09-11T01:14:15.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 94857 37.85 MB/sec execute 21 sec latency 47.078 ms 2024-09-11T01:14:15.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:15 smithi110 ceph-mon[34712]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.9 MiB/s wr, 735 op/s 2024-09-11T01:14:15.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:15 smithi078 ceph-mon[31315]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.9 MiB/s wr, 735 op/s 2024-09-11T01:14:15.975 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:15 smithi112 ceph-mon[35384]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.9 MiB/s wr, 735 op/s 2024-09-11T01:14:16.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 95676 38.20 MB/sec execute 22 sec latency 46.591 ms 2024-09-11T01:14:17.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 96524 37.92 MB/sec execute 23 sec latency 72.363 ms 2024-09-11T01:14:17.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:17 smithi110 ceph-mon[34712]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.9 MiB/s wr, 773 op/s 2024-09-11T01:14:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:17 smithi112 ceph-mon[35384]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.9 MiB/s wr, 773 op/s 2024-09-11T01:14:17.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:17 smithi078 ceph-mon[31315]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.9 MiB/s wr, 773 op/s 2024-09-11T01:14:18.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 97619 39.18 MB/sec execute 24 sec latency 45.834 ms 2024-09-11T01:14:19.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 98429 39.39 MB/sec execute 25 sec latency 39.679 ms 2024-09-11T01:14:19.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:19 smithi110 ceph-mon[34712]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.7 MiB/s wr, 722 op/s 2024-09-11T01:14:19.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:19 smithi112 ceph-mon[35384]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.7 MiB/s wr, 722 op/s 2024-09-11T01:14:19.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:19 smithi078 ceph-mon[31315]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.7 MiB/s wr, 722 op/s 2024-09-11T01:14:20.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 99291 39.55 MB/sec execute 26 sec latency 46.259 ms 2024-09-11T01:14:20.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:20 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:14:20.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:20 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:14:20.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:20 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:14:21.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 100306 39.97 MB/sec execute 27 sec latency 55.159 ms 2024-09-11T01:14:21.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:21 smithi110 ceph-mon[34712]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 11 MiB/s wr, 752 op/s 2024-09-11T01:14:21.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:21 smithi112 ceph-mon[35384]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 11 MiB/s wr, 752 op/s 2024-09-11T01:14:21.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:21 smithi078 ceph-mon[31315]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 11 MiB/s wr, 752 op/s 2024-09-11T01:14:22.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 101277 40.62 MB/sec execute 28 sec latency 41.047 ms 2024-09-11T01:14:22.977 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:22 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:14:22.977 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:22 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:14:22.977 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:22 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:14:22.999 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:22 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:14:23.000 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:22 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:14:23.000 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:22 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:14:23.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:22 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:14:23.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:22 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:14:23.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:22 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:14:23.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 101483 39.67 MB/sec execute 29 sec latency 778.870 ms 2024-09-11T01:14:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:23 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:14:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:23 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:14:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:23 smithi112 ceph-mon[35384]: pgmap v806: 97 pgs: 97 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 11 MiB/s wr, 724 op/s 2024-09-11T01:14:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:23 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:14:23.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:23 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:14:23.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:23 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:14:23.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:23 smithi078 ceph-mon[31315]: pgmap v806: 97 pgs: 97 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 11 MiB/s wr, 724 op/s 2024-09-11T01:14:23.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:23 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:14:24.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:23 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:14:24.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:23 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:14:24.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:23 smithi110 ceph-mon[34712]: pgmap v806: 97 pgs: 97 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 11 MiB/s wr, 724 op/s 2024-09-11T01:14:24.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:23 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:14:24.781 INFO:tasks.workunit.client.0.smithi078.stdout: 10 101905 39.10 MB/sec execute 30 sec latency 895.014 ms 2024-09-11T01:14:25.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 102543 38.68 MB/sec execute 31 sec latency 103.445 ms 2024-09-11T01:14:25.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:25 smithi112 ceph-mon[35384]: pgmap v807: 97 pgs: 97 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 730 op/s 2024-09-11T01:14:25.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:25 smithi078 ceph-mon[31315]: pgmap v807: 97 pgs: 97 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 730 op/s 2024-09-11T01:14:26.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:25 smithi110 ceph-mon[34712]: pgmap v807: 97 pgs: 97 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 730 op/s 2024-09-11T01:14:26.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 103532 39.18 MB/sec execute 32 sec latency 67.399 ms 2024-09-11T01:14:27.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 104476 39.85 MB/sec execute 33 sec latency 55.129 ms 2024-09-11T01:14:27.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:27 smithi112 ceph-mon[35384]: pgmap v808: 97 pgs: 97 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 20 MiB/s wr, 919 op/s 2024-09-11T01:14:27.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:27 smithi078 ceph-mon[31315]: pgmap v808: 97 pgs: 97 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 20 MiB/s wr, 919 op/s 2024-09-11T01:14:28.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:27 smithi110 ceph-mon[34712]: pgmap v808: 97 pgs: 97 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 20 MiB/s wr, 919 op/s 2024-09-11T01:14:28.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 105124 39.62 MB/sec execute 34 sec latency 106.426 ms 2024-09-11T01:14:28.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:28 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:14:28.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:28 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:14:29.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:28 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:14:29.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 105952 39.69 MB/sec execute 35 sec latency 43.410 ms 2024-09-11T01:14:29.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:29 smithi112 ceph-mon[35384]: pgmap v809: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 932 op/s 2024-09-11T01:14:29.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:29 smithi078 ceph-mon[31315]: pgmap v809: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 932 op/s 2024-09-11T01:14:30.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:29 smithi110 ceph-mon[34712]: pgmap v809: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 932 op/s 2024-09-11T01:14:30.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 107092 40.13 MB/sec execute 36 sec latency 49.172 ms 2024-09-11T01:14:30.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:14:30.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:14:30.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:14:30.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:14:30.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:14:30.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:14:31.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:14:31.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:14:31.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:14:31.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 108009 40.73 MB/sec execute 37 sec latency 130.913 ms 2024-09-11T01:14:32.194 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:31 smithi110 ceph-mon[34712]: pgmap v810: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-09-11T01:14:32.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:31 smithi112 ceph-mon[35384]: pgmap v810: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-09-11T01:14:32.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:31 smithi078 ceph-mon[31315]: pgmap v810: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-09-11T01:14:32.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 108641 40.48 MB/sec execute 38 sec latency 63.384 ms 2024-09-11T01:14:33.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 109278 40.25 MB/sec execute 39 sec latency 160.592 ms 2024-09-11T01:14:34.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:33 smithi110 ceph-mon[34712]: pgmap v811: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 22 MiB/s wr, 1.05k op/s 2024-09-11T01:14:34.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:33 smithi112 ceph-mon[35384]: pgmap v811: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 22 MiB/s wr, 1.05k op/s 2024-09-11T01:14:34.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:33 smithi078 ceph-mon[31315]: pgmap v811: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 22 MiB/s wr, 1.05k op/s 2024-09-11T01:14:34.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 110346 40.45 MB/sec execute 40 sec latency 73.930 ms 2024-09-11T01:14:35.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 111414 41.13 MB/sec execute 41 sec latency 137.198 ms 2024-09-11T01:14:35.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:35 smithi078 ceph-mon[31315]: pgmap v812: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-09-11T01:14:35.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:35 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:14:36.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:35 smithi110 ceph-mon[34712]: pgmap v812: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-09-11T01:14:36.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:35 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:14:36.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:35 smithi112 ceph-mon[35384]: pgmap v812: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-09-11T01:14:36.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:35 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:14:36.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 112166 41.12 MB/sec execute 42 sec latency 58.860 ms 2024-09-11T01:14:37.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 112952 41.05 MB/sec execute 43 sec latency 59.427 ms 2024-09-11T01:14:38.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:37 smithi110 ceph-mon[34712]: pgmap v813: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 967 op/s 2024-09-11T01:14:38.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:37 smithi112 ceph-mon[35384]: pgmap v813: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 967 op/s 2024-09-11T01:14:38.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:37 smithi078 ceph-mon[31315]: pgmap v813: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 967 op/s 2024-09-11T01:14:38.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 113769 41.07 MB/sec execute 44 sec latency 159.748 ms 2024-09-11T01:14:39.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 114756 41.38 MB/sec execute 45 sec latency 72.640 ms 2024-09-11T01:14:40.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:39 smithi110 ceph-mon[34712]: pgmap v814: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 742 op/s 2024-09-11T01:14:40.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:39 smithi112 ceph-mon[35384]: pgmap v814: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 742 op/s 2024-09-11T01:14:40.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:39 smithi078 ceph-mon[31315]: pgmap v814: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 742 op/s 2024-09-11T01:14:40.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 115724 41.66 MB/sec execute 46 sec latency 48.461 ms 2024-09-11T01:14:41.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 116424 41.55 MB/sec execute 47 sec latency 87.010 ms 2024-09-11T01:14:41.886 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:14:42.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:41 smithi110 ceph-mon[34712]: pgmap v815: 97 pgs: 97 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 11 MiB/s wr, 809 op/s 2024-09-11T01:14:42.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:41 smithi112 ceph-mon[35384]: pgmap v815: 97 pgs: 97 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 11 MiB/s wr, 809 op/s 2024-09-11T01:14:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:14:42.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:41 smithi078 ceph-mon[31315]: pgmap v815: 97 pgs: 97 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 11 MiB/s wr, 809 op/s 2024-09-11T01:14:42.783 INFO:tasks.workunit.client.0.smithi078.stdout: 10 117324 41.69 MB/sec execute 48 sec latency 53.941 ms 2024-09-11T01:14:42.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.856+0000 7f54ba528640 1 -- 172.21.15.78:0/1159184577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54b41076a0 msgr2=0x7f54b4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:42.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.856+0000 7f54ba528640 1 --2- 172.21.15.78:0/1159184577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54b41076a0 0x7f54b4107ae0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009900 tx=0x7f54a802a410 comp rx=0 tx=0).stop 2024-09-11T01:14:42.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.856+0000 7f54ba528640 1 -- 172.21.15.78:0/1159184577 shutdown_connections 2024-09-11T01:14:42.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.856+0000 7f54ba528640 1 --2- 172.21.15.78:0/1159184577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54b4108020 0x7f54b410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.856+0000 7f54ba528640 1 --2- 172.21.15.78:0/1159184577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54b41076a0 0x7f54b4107ae0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.856+0000 7f54ba528640 1 --2- 172.21.15.78:0/1159184577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b4101c90 0x7f54b4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.856+0000 7f54ba528640 1 -- 172.21.15.78:0/1159184577 >> 172.21.15.78:0/1159184577 conn(0x7f54b40fd700 msgr2=0x7f54b40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:42.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.856+0000 7f54ba528640 1 -- 172.21.15.78:0/1159184577 shutdown_connections 2024-09-11T01:14:42.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.856+0000 7f54ba528640 1 -- 172.21.15.78:0/1159184577 wait complete. 2024-09-11T01:14:42.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.856+0000 7f54ba528640 1 Processor -- start 2024-09-11T01:14:42.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.856+0000 7f54ba528640 1 -- start start 2024-09-11T01:14:42.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54ba528640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54b4101c90 0x7f54b4074940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54ba528640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54b41076a0 0x7f54b406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54ba528640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b4108020 0x7f54b406de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54ba528640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4111c70 con 0x7f54b4101c90 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54ba528640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4111af0 con 0x7f54b4108020 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54ba528640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4111df0 con 0x7f54b41076a0 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54b4101c90 0x7f54b4074940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54b4101c90 0x7f54b4074940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55802/0 (socket says 172.21.15.78:55802) 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54b3fff640 1 -- 172.21.15.78:0/68704417 learned_addr learned my addr 172.21.15.78:0/68704417 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54b8ac5640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b4108020 0x7f54b406de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54b37fe640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54b41076a0 0x7f54b406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54b8ac5640 1 -- 172.21.15.78:0/68704417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54b41076a0 msgr2=0x7f54b406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54b8ac5640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54b41076a0 0x7f54b406d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54b8ac5640 1 -- 172.21.15.78:0/68704417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54b4101c90 msgr2=0x7f54b4074940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:42.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54b8ac5640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54b4101c90 0x7f54b4074940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54b8ac5640 1 -- 172.21.15.78:0/68704417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b406e360 con 0x7f54b4108020 2024-09-11T01:14:42.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.857+0000 7f54b8ac5640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b4108020 0x7f54b406de20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009fd0 tx=0x7f54ac00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:42.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.858+0000 7f54b17fa640 1 -- 172.21.15.78:0/68704417 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac00ce20 con 0x7f54b4108020 2024-09-11T01:14:42.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.858+0000 7f54b17fa640 1 -- 172.21.15.78:0/68704417 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54ac005020 con 0x7f54b4108020 2024-09-11T01:14:42.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.858+0000 7f54ba528640 1 -- 172.21.15.78:0/68704417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b406e650 con 0x7f54b4108020 2024-09-11T01:14:42.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.858+0000 7f54b17fa640 1 -- 172.21.15.78:0/68704417 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac010690 con 0x7f54b4108020 2024-09-11T01:14:42.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.858+0000 7f54ba528640 1 -- 172.21.15.78:0/68704417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b41b4b10 con 0x7f54b4108020 2024-09-11T01:14:42.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.861+0000 7f54b17fa640 1 -- 172.21.15.78:0/68704417 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f54ac010830 con 0x7f54b4108020 2024-09-11T01:14:42.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.861+0000 7f54bc8fd740 1 Processor -- start 2024-09-11T01:14:42.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.862+0000 7f54bc8fd740 1 -- start start 2024-09-11T01:14:42.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.862+0000 7f54bc8fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be970e0a0 0x558be970e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.862+0000 7f54bc8fd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be971a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.863+0000 7f54b17fa640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f549c0b5450 0x7f549c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.863+0000 7f54b3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be971a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.863+0000 7f54b3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be971a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36564/0 (socket says 172.21.15.78:36564) 2024-09-11T01:14:42.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.863+0000 7f54b17fa640 1 -- 172.21.15.78:0/68704417 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f54ac0e6480 con 0x7f54b4108020 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.863+0000 7f54b37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be970e0a0 0x558be970e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.863+0000 7f54b37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be970e0a0 0x558be970e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:48414/0 (socket says 172.21.15.78:48414) 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.863+0000 7f54b8ac5640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f549c0b5450 0x7f549c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.863+0000 7f54bc8fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be971a950 0x558be971cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.863+0000 7f54b3fff640 1 -- 172.21.15.78:0/1669212213 learned_addr learned my addr 172.21.15.78:0/1669212213 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.863+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1669212213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558be9684e50 con 0x558be971a950 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.863+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1669212213 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558be9674280 con 0x558be970e9c0 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.863+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1669212213 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558be9678090 con 0x558be970e0a0 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.864+0000 7f54b3fff640 1 -- 172.21.15.78:0/1669212213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be970e0a0 msgr2=0x558be970e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.864+0000 7f54b3fff640 1 --2- 172.21.15.78:0/1669212213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be970e0a0 0x558be970e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.864+0000 7f54b3fff640 1 -- 172.21.15.78:0/1669212213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be971a950 msgr2=0x558be971cd40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.864+0000 7f54b3fff640 1 --2- 172.21.15.78:0/1669212213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be971a950 0x558be971cd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.864+0000 7f54b3fff640 1 -- 172.21.15.78:0/1669212213 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be93d2650 con 0x558be970e9c0 2024-09-11T01:14:42.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.864+0000 7f54b37fe640 1 --2- 172.21.15.78:0/1669212213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be970e0a0 0x558be970e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-11T01:14:42.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.864+0000 7f54b3fff640 1 --2- 172.21.15.78:0/1669212213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be971a410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009dc0 tx=0x7f54a4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9278713df8c5d7b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:42.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.864+0000 7f54b8ac5640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f549c0b5450 0x7f549c0b7910 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0ac8d0 tx=0x7f54ac014420 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:42.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.864+0000 7f549a7fc640 1 -- 172.21.15.78:0/1669212213 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4005220 con 0x558be970e9c0 2024-09-11T01:14:42.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.864+0000 7f549a7fc640 1 -- 172.21.15.78:0/1669212213 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a4004c50 con 0x558be970e9c0 2024-09-11T01:14:42.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.864+0000 7f549a7fc640 1 -- 172.21.15.78:0/1669212213 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4019890 con 0x558be970e9c0 2024-09-11T01:14:42.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.865+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1669212213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 msgr2=0x558be971a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:42.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.865+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/1669212213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be971a410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009dc0 tx=0x7f54a4011810 comp rx=0 tx=0).stop 2024-09-11T01:14:42.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.865+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1669212213 shutdown_connections 2024-09-11T01:14:42.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.865+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/1669212213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be971a950 0x558be971cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.865+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/1669212213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be971a410 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.865+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/1669212213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be970e0a0 0x558be970e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.865+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1669212213 >> 172.21.15.78:0/1669212213 conn(0x558be970bed0 msgr2=0x558be970d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.866+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1669212213 shutdown_connections 2024-09-11T01:14:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.866+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1669212213 wait complete. 2024-09-11T01:14:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.866+0000 7f54bc8fd740 1 -- start start 2024-09-11T01:14:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.866+0000 7f54bc8fd740 1 Processor -- start 2024-09-11T01:14:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54bc8fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e0a0 0x558be971a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54bc8fd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be9717590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54bc8fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be971a950 0x558be9717ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54bc8fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558be94d5700 con 0x558be970e0a0 2024-09-11T01:14:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54bc8fd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558be960f110 con 0x558be970e9c0 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54bc8fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558be971d5d0 con 0x558be971a950 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54bc8fd740 10 client.0 _ll_register_callbacks cb 0x558be9628990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54b37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e0a0 0x558be971a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54b37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e0a0 0x558be971a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55814/0 (socket says 172.21.15.78:55814) 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54b37fe640 1 -- 172.21.15.78:0/510505881 learned_addr learned my addr 172.21.15.78:0/510505881 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54b3fff640 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be9717590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54b37fe640 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be971a950 0x558be9717ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54b37fe640 1 -- 172.21.15.78:0/510505881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be971a950 msgr2=0x558be9717ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54b37fe640 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be971a950 0x558be9717ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54b37fe640 1 -- 172.21.15.78:0/510505881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 msgr2=0x558be9717590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54b37fe640 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be9717590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54b37fe640 1 -- 172.21.15.78:0/510505881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be963cc40 con 0x558be970e0a0 2024-09-11T01:14:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.867+0000 7f54b3fff640 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be9717590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:14:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.868+0000 7f54b37fe640 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e0a0 0x558be971a140 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f54a802a930 tx=0x7f54a8005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffa812b6e858c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.868+0000 7f54937fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:14:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.868+0000 7f54937fe640 1 -- 172.21.15.78:0/510505881 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8035040 con 0x558be970e0a0 2024-09-11T01:14:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.868+0000 7f54937fe640 1 -- 172.21.15.78:0/510505881 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f54a8031d10 con 0x558be970e0a0 2024-09-11T01:14:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.868+0000 7f54bc8fd740 1 -- 172.21.15.78:0/510505881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a4004c50 con 0x558be970e0a0 2024-09-11T01:14:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.868+0000 7f54937fe640 1 -- 172.21.15.78:0/510505881 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f54a8004260 con 0x558be970e0a0 2024-09-11T01:14:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.868+0000 7f54bc8fd740 10 client.18342 fetch_fsmap learned FSMap version 64 2024-09-11T01:14:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.868+0000 7f54bc8fd740 1 -- 172.21.15.78:0/510505881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558be9728e00 con 0x558be970e0a0 2024-09-11T01:14:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.869+0000 7f54937fe640 1 -- 172.21.15.78:0/510505881 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a4004c50 con 0x558be970e0a0 2024-09-11T01:14:42.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.869+0000 7f54937fe640 1 -- 172.21.15.78:0/510505881 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f54a8004950 con 0x558be970e0a0 2024-09-11T01:14:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.869+0000 7f54bc8fd740 10 client.18342 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:14:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.869+0000 7f54bc8fd740 10 client.18342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:14:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.869+0000 7f54bc8fd740 20 client.18342 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.869+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558be972a380 0x558be972c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.869+0000 7f54bc8fd740 4 client.18342 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:14:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.869+0000 7f54bc8fd740 1 -- 172.21.15.78:0/510505881 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558be9686da0 con 0x558be972a380 2024-09-11T01:14:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.869+0000 7f54b3fff640 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558be972a380 0x558be972c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.870+0000 7f54b3fff640 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558be972a380 0x558be972c790 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b02572ec94a4ffbc server_cookie=fd60489c0300a721 in_seq=0 out_seq=0 2024-09-11T01:14:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.870+0000 7f54937fe640 10 client.18342 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:14:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54937fe640 1 -- 172.21.15.78:0/510505881 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558be9686da0 con 0x558be972a380 2024-09-11T01:14:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54937fe640 10 client.18342 handle_command_reply: tid=0 2024-09-11T01:14:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54bc8fd740 1 client.18342 shutdown 2024-09-11T01:14:42.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54bc8fd740 1 -- 172.21.15.78:0/510505881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e0a0 msgr2=0x558be971a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:42.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e0a0 0x558be971a140 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f54a802a930 tx=0x7f54a8005900 comp rx=0 tx=0).stop 2024-09-11T01:14:42.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54bc8fd740 1 -- 172.21.15.78:0/510505881 shutdown_connections 2024-09-11T01:14:42.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be971a950 0x558be9717ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be9717590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558be972a380 0x558be972c790 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/510505881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e0a0 0x558be971a140 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54bc8fd740 1 -- 172.21.15.78:0/510505881 >> 172.21.15.78:0/510505881 conn(0x558be970bed0 msgr2=0x558be9711a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:42.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54937fe640 0 client.18342 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:14:42.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54bc8fd740 1 -- 172.21.15.78:0/510505881 shutdown_connections 2024-09-11T01:14:42.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54bc8fd740 1 -- 172.21.15.78:0/510505881 wait complete. 2024-09-11T01:14:42.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.871+0000 7f54bc8fd740 20 client.18342 trim_cache size 0 max 16384 2024-09-11T01:14:42.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.900+0000 7f54bc8fd740 1 Processor -- start 2024-09-11T01:14:42.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.900+0000 7f54bc8fd740 1 -- start start 2024-09-11T01:14:42.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.900+0000 7f54bc8fd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e0a0 0x558be970b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.900+0000 7f54bc8fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e9c0 0x558be97147a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.900+0000 7f54bc8fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be971a950 0x558be9714ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.900+0000 7f54bc8fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54a000e6b0 con 0x558be970e9c0 2024-09-11T01:14:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.900+0000 7f54bc8fd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54a0010370 con 0x558be970e0a0 2024-09-11T01:14:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.900+0000 7f54bc8fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54a000ed80 con 0x558be971a950 2024-09-11T01:14:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.900+0000 7f54b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e9c0 0x558be97147a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.900+0000 7f54b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e9c0 0x558be97147a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55830/0 (socket says 172.21.15.78:55830) 2024-09-11T01:14:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.900+0000 7f54b3fff640 1 -- 172.21.15.78:0/4021367368 learned_addr learned my addr 172.21.15.78:0/4021367368 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.900+0000 7f54b37fe640 1 --2- 172.21.15.78:0/4021367368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e0a0 0x558be970b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.901+0000 7f54b37fe640 1 --2- 172.21.15.78:0/4021367368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be971a950 0x558be9714ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.901+0000 7f54b3fff640 1 -- 172.21.15.78:0/4021367368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be971a950 msgr2=0x558be9714ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.901+0000 7f54b3fff640 1 --2- 172.21.15.78:0/4021367368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be971a950 0x558be9714ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.901+0000 7f54b3fff640 1 -- 172.21.15.78:0/4021367368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e0a0 msgr2=0x558be970b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.901+0000 7f54b3fff640 1 --2- 172.21.15.78:0/4021367368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e0a0 0x558be970b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.901+0000 7f54b3fff640 1 -- 172.21.15.78:0/4021367368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be970c950 con 0x558be970e9c0 2024-09-11T01:14:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.901+0000 7f54b3fff640 1 --2- 172.21.15.78:0/4021367368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e9c0 0x558be97147a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005cf0 tx=0x7f54a4019430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e95916060bb73e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.901+0000 7f5499ffb640 1 -- 172.21.15.78:0/4021367368 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a40055f0 con 0x558be970e9c0 2024-09-11T01:14:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.901+0000 7f5499ffb640 1 -- 172.21.15.78:0/4021367368 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a4003aa0 con 0x558be970e9c0 2024-09-11T01:14:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.901+0000 7f5499ffb640 1 -- 172.21.15.78:0/4021367368 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a40199b0 con 0x558be970e9c0 2024-09-11T01:14:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.902+0000 7f54bc8fd740 1 -- 172.21.15.78:0/4021367368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e9c0 msgr2=0x558be97147a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.902+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/4021367368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e9c0 0x558be97147a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005cf0 tx=0x7f54a4019430 comp rx=0 tx=0).stop 2024-09-11T01:14:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.902+0000 7f54bc8fd740 1 -- 172.21.15.78:0/4021367368 shutdown_connections 2024-09-11T01:14:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.902+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/4021367368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be971a950 0x558be9714ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.902+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/4021367368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be970e9c0 0x558be97147a0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.902+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/4021367368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e0a0 0x558be970b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.902+0000 7f54bc8fd740 1 -- 172.21.15.78:0/4021367368 >> 172.21.15.78:0/4021367368 conn(0x558be970bed0 msgr2=0x558be970cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.902+0000 7f54bc8fd740 1 -- 172.21.15.78:0/4021367368 shutdown_connections 2024-09-11T01:14:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.902+0000 7f54bc8fd740 1 -- 172.21.15.78:0/4021367368 wait complete. 2024-09-11T01:14:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54bc8fd740 1 -- start start 2024-09-11T01:14:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54bc8fd740 1 Processor -- start 2024-09-11T01:14:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54bc8fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be970e0a0 0x558be9723060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54bc8fd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be971e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54bc8fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be971a950 0x558be971eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54bc8fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54a000fbe0 con 0x558be971a950 2024-09-11T01:14:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54bc8fd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54a000f450 con 0x558be970e9c0 2024-09-11T01:14:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54bc8fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54a000c6b0 con 0x558be970e0a0 2024-09-11T01:14:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54bc8fd740 10 client.0 _ll_register_callbacks cb 0x7f54ac0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54b37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be971a950 0x558be971eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54b3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be971e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54b37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be971a950 0x558be971eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55844/0 (socket says 172.21.15.78:55844) 2024-09-11T01:14:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54b37fe640 1 -- 172.21.15.78:0/1015750104 learned_addr learned my addr 172.21.15.78:0/1015750104 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.903+0000 7f54b37fe640 1 --2- 172.21.15.78:0/1015750104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be970e0a0 0x558be9723060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.904+0000 7f54b3fff640 1 -- 172.21.15.78:0/1015750104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be970e0a0 msgr2=0x558be9723060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.904+0000 7f54b3fff640 1 --2- 172.21.15.78:0/1015750104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be970e0a0 0x558be9723060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.904+0000 7f54b3fff640 1 -- 172.21.15.78:0/1015750104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be971a950 msgr2=0x558be971eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.904+0000 7f54b3fff640 1 --2- 172.21.15.78:0/1015750104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be971a950 0x558be971eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.904+0000 7f54b3fff640 1 -- 172.21.15.78:0/1015750104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be971f340 con 0x558be970e9c0 2024-09-11T01:14:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.904+0000 7f54b3fff640 1 --2- 172.21.15.78:0/1015750104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be971e5d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f54a402d040 tx=0x7f54a4003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff02f9403c8dbf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.904+0000 7f54937fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:14:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.904+0000 7f54937fe640 1 -- 172.21.15.78:0/1015750104 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4039030 con 0x558be970e9c0 2024-09-11T01:14:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.904+0000 7f54937fe640 1 -- 172.21.15.78:0/1015750104 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f54a40055f0 con 0x558be970e9c0 2024-09-11T01:14:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.904+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1015750104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a4003aa0 con 0x558be970e9c0 2024-09-11T01:14:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.905+0000 7f54937fe640 1 -- 172.21.15.78:0/1015750104 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f54a4043050 con 0x558be970e9c0 2024-09-11T01:14:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.905+0000 7f54bc8fd740 10 client.27503 fetch_fsmap learned FSMap version 64 2024-09-11T01:14:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.905+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1015750104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558be97972a0 con 0x558be970e9c0 2024-09-11T01:14:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.905+0000 7f54937fe640 1 -- 172.21.15.78:0/1015750104 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a4003aa0 con 0x558be970e9c0 2024-09-11T01:14:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.905+0000 7f54937fe640 1 -- 172.21.15.78:0/1015750104 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f54a4024410 con 0x558be970e9c0 2024-09-11T01:14:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.905+0000 7f54bc8fd740 10 client.27503 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:14:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.905+0000 7f54bc8fd740 10 client.27503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:14:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.905+0000 7f54bc8fd740 20 client.27503 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.905+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/1015750104 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558be9797850 0x558be9799c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.905+0000 7f54bc8fd740 4 client.27503 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:14:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.905+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1015750104 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558be9682850 con 0x558be9797850 2024-09-11T01:14:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.905+0000 7f54b37fe640 1 --2- 172.21.15.78:0/1015750104 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558be9797850 0x558be9799c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.906+0000 7f54b37fe640 1 --2- 172.21.15.78:0/1015750104 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558be9797850 0x558be9799c70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51acc95786fb542b server_cookie=32505f2dabf0828f in_seq=0 out_seq=0 2024-09-11T01:14:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.906+0000 7f54937fe640 10 client.27503 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:14:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.906+0000 7f54937fe640 1 -- 172.21.15.78:0/1015750104 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5770 (crc 0 0 0) 0x558be9682850 con 0x558be9797850 2024-09-11T01:14:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.906+0000 7f54937fe640 10 client.27503 handle_command_reply: tid=0 2024-09-11T01:14:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.906+0000 7f54bc8fd740 1 client.27503 shutdown 2024-09-11T01:14:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.906+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1015750104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 msgr2=0x558be971e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/1015750104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be971e5d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f54a402d040 tx=0x7f54a4003e80 comp rx=0 tx=0).stop 2024-09-11T01:14:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1015750104 shutdown_connections 2024-09-11T01:14:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/1015750104 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x558be9797850 0x558be9799c70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/1015750104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558be971a950 0x558be971eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/1015750104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558be970e9c0 0x558be971e5d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54bc8fd740 1 --2- 172.21.15.78:0/1015750104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558be970e0a0 0x558be9723060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1015750104 >> 172.21.15.78:0/1015750104 conn(0x558be970bed0 msgr2=0x558be9718bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54937fe640 0 client.27503 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:14:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1015750104 shutdown_connections 2024-09-11T01:14:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54bc8fd740 1 -- 172.21.15.78:0/1015750104 wait complete. 2024-09-11T01:14:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54bc8fd740 20 client.27503 trim_cache size 0 max 16384 2024-09-11T01:14:42.915 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:14:42.915 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:14:42.915 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:14:42.915 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:14:42.915 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:14:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:14:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:14:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:14:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:14:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:14:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:14:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:14:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:14:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:14:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:14:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:14:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:14:42.929 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:14:42.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:14:42.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:14:42.929 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:14:42.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54ba528640 1 -- 172.21.15.78:0/68704417 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f549c0b5450 msgr2=0x7f549c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:42.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54ba528640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f549c0b5450 0x7f549c0b7910 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0ac8d0 tx=0x7f54ac014420 comp rx=0 tx=0).stop 2024-09-11T01:14:42.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54ba528640 1 -- 172.21.15.78:0/68704417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b4108020 msgr2=0x7f54b406de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:42.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.907+0000 7f54ba528640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b4108020 0x7f54b406de20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009fd0 tx=0x7f54ac00ef10 comp rx=0 tx=0).stop 2024-09-11T01:14:42.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.908+0000 7f54ba528640 1 -- 172.21.15.78:0/68704417 shutdown_connections 2024-09-11T01:14:42.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.908+0000 7f54ba528640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f549c0b5450 0x7f549c0b7910 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.908+0000 7f54ba528640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b4108020 0x7f54b406de20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.908+0000 7f54ba528640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54b41076a0 0x7f54b406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.908+0000 7f54ba528640 1 --2- 172.21.15.78:0/68704417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54b4101c90 0x7f54b4074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:42.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.908+0000 7f54ba528640 1 -- 172.21.15.78:0/68704417 >> 172.21.15.78:0/68704417 conn(0x7f54b40fd700 msgr2=0x7f54b40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:42.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.908+0000 7f54ba528640 1 -- 172.21.15.78:0/68704417 shutdown_connections 2024-09-11T01:14:42.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:42.908+0000 7f54ba528640 1 -- 172.21.15.78:0/68704417 wait complete. 2024-09-11T01:14:43.782 INFO:tasks.workunit.client.0.smithi078.stdout: 10 118009 41.63 MB/sec execute 49 sec latency 358.353 ms 2024-09-11T01:14:44.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:43 smithi110 ceph-mon[34712]: pgmap v816: 97 pgs: 97 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.2 MiB/s wr, 639 op/s 2024-09-11T01:14:44.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:43 smithi112 ceph-mon[35384]: pgmap v816: 97 pgs: 97 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.2 MiB/s wr, 639 op/s 2024-09-11T01:14:44.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:43 smithi078 ceph-mon[31315]: pgmap v816: 97 pgs: 97 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.2 MiB/s wr, 639 op/s 2024-09-11T01:14:44.319 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:14:44.320 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:14:44.738 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:14:44.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 118145 40.97 MB/sec execute 50 sec latency 892.655 ms 2024-09-11T01:14:45.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.339+0000 7fbfc4035640 1 -- 172.21.15.78:0/3864961421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc11c0a0 msgr2=0x7fbfbc11e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:45.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.339+0000 7fbfc4035640 1 --2- 172.21.15.78:0/3864961421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc11c0a0 0x7fbfbc11e490 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009900 tx=0x7fbfb802a4d0 comp rx=0 tx=0).stop 2024-09-11T01:14:45.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.340+0000 7fbfc4035640 1 -- 172.21.15.78:0/3864961421 shutdown_connections 2024-09-11T01:14:45.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.340+0000 7fbfc4035640 1 --2- 172.21.15.78:0/3864961421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc11c0a0 0x7fbfbc11e490 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.340+0000 7fbfc4035640 1 --2- 172.21.15.78:0/3864961421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc10a530 0x7fbfbc10a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.340+0000 7fbfc4035640 1 --2- 172.21.15.78:0/3864961421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc10a150 0x7fbfbc1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.340+0000 7fbfc4035640 1 -- 172.21.15.78:0/3864961421 >> 172.21.15.78:0/3864961421 conn(0x7fbfbc06dfc0 msgr2=0x7fbfbc06e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.341+0000 7fbfc4035640 1 -- 172.21.15.78:0/3864961421 shutdown_connections 2024-09-11T01:14:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.341+0000 7fbfc4035640 1 -- 172.21.15.78:0/3864961421 wait complete. 2024-09-11T01:14:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.342+0000 7fbfc4035640 1 Processor -- start 2024-09-11T01:14:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.342+0000 7fbfc4035640 1 -- start start 2024-09-11T01:14:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.342+0000 7fbfc4035640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc10a150 0x7fbfbc1a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.342+0000 7fbfc4035640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc10a530 0x7fbfbc1a8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.342+0000 7fbfc4035640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc11c0a0 0x7fbfbc1ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.342+0000 7fbfc4035640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc121c60 con 0x7fbfbc11c0a0 2024-09-11T01:14:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.342+0000 7fbfc4035640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc121ae0 con 0x7fbfbc10a530 2024-09-11T01:14:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.342+0000 7fbfc4035640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc121de0 con 0x7fbfbc10a150 2024-09-11T01:14:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.342+0000 7fbfc1dd1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc10a150 0x7fbfbc1a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.342+0000 7fbfc1dd1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc10a150 0x7fbfbc1a7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:50856/0 (socket says 172.21.15.78:50856) 2024-09-11T01:14:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.342+0000 7fbfc1dd1640 1 -- 172.21.15.78:0/3118143320 learned_addr learned my addr 172.21.15.78:0/3118143320 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.343+0000 7fbfc25d2640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc11c0a0 0x7fbfbc1ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.343+0000 7fbfc1dd1640 1 -- 172.21.15.78:0/3118143320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc10a530 msgr2=0x7fbfbc1a8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.343+0000 7fbfc1dd1640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc10a530 0x7fbfbc1a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.343+0000 7fbfc1dd1640 1 -- 172.21.15.78:0/3118143320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc11c0a0 msgr2=0x7fbfbc1ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.343+0000 7fbfc1dd1640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc11c0a0 0x7fbfbc1ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.343+0000 7fbfc1dd1640 1 -- 172.21.15.78:0/3118143320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc1ae080 con 0x7fbfbc10a150 2024-09-11T01:14:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.344+0000 7fbfc1dd1640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc10a150 0x7fbfbc1a7fd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400ba50 tx=0x7fbfb400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.344+0000 7fbfb2ffd640 1 -- 172.21.15.78:0/3118143320 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4019070 con 0x7fbfbc10a150 2024-09-11T01:14:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.344+0000 7fbfc4035640 1 -- 172.21.15.78:0/3118143320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfbc1ae370 con 0x7fbfbc10a150 2024-09-11T01:14:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.344+0000 7fbfc4035640 1 -- 172.21.15.78:0/3118143320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfbc1b4c00 con 0x7fbfbc10a150 2024-09-11T01:14:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.344+0000 7fbfb2ffd640 1 -- 172.21.15.78:0/3118143320 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb400cc10 con 0x7fbfbc10a150 2024-09-11T01:14:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.344+0000 7fbfb2ffd640 1 -- 172.21.15.78:0/3118143320 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4023060 con 0x7fbfbc10a150 2024-09-11T01:14:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.345+0000 7fbfc4035640 1 -- 172.21.15.78:0/3118143320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa00052c0 con 0x7fbfbc10a150 2024-09-11T01:14:45.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.349+0000 7fbfb2ffd640 1 -- 172.21.15.78:0/3118143320 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fbfb401d3b0 con 0x7fbfbc10a150 2024-09-11T01:14:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.350+0000 7fbfb2ffd640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbfa80b54d0 0x7fbfa80b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.351+0000 7fbfb2ffd640 1 -- 172.21.15.78:0/3118143320 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7fbfb40e58e0 con 0x7fbfbc10a150 2024-09-11T01:14:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.351+0000 7fbfc15d0640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbfa80b54d0 0x7fbfa80b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:45.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.353+0000 7fbfc15d0640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbfa80b54d0 0x7fbfa80b7990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc11e670 tx=0x7fbfac006c40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:45.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.355+0000 7fbfb2ffd640 1 -- 172.21.15.78:0/3118143320 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbfb40abdf0 con 0x7fbfbc10a150 2024-09-11T01:14:45.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.761+0000 7fbfc4035640 1 -- 172.21.15.78:0/3118143320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfa00055b0 con 0x7fbfbc10a150 2024-09-11T01:14:45.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.762+0000 7fbfb2ffd640 1 -- 172.21.15.78:0/3118143320 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+12000 (secure 0 0 0) 0x7fbfb40b0820 con 0x7fbfbc10a150 2024-09-11T01:14:45.766 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:14:45.766 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":64,"btime":"2024-09-11T01:14:34:843329+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":64,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:14:34.441941+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:14:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 64 2024-09-11T01:14:45.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.766+0000 7fbfc4035640 1 -- 172.21.15.78:0/3118143320 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbfa80b54d0 msgr2=0x7fbfa80b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:45.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.766+0000 7fbfc4035640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbfa80b54d0 0x7fbfa80b7990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc11e670 tx=0x7fbfac006c40 comp rx=0 tx=0).stop 2024-09-11T01:14:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.766+0000 7fbfc4035640 1 -- 172.21.15.78:0/3118143320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc10a150 msgr2=0x7fbfbc1a7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.766+0000 7fbfc4035640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc10a150 0x7fbfbc1a7fd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400ba50 tx=0x7fbfb400bf10 comp rx=0 tx=0).stop 2024-09-11T01:14:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.766+0000 7fbfc4035640 1 -- 172.21.15.78:0/3118143320 shutdown_connections 2024-09-11T01:14:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.766+0000 7fbfc4035640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbfa80b54d0 0x7fbfa80b7990 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc11e670 tx=0x7fbfac006c40 comp rx=0 tx=0).stop 2024-09-11T01:14:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.766+0000 7fbfc4035640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc11c0a0 0x7fbfbc1ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.766+0000 7fbfc4035640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc10a530 0x7fbfbc1a8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.766+0000 7fbfc4035640 1 --2- 172.21.15.78:0/3118143320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc10a150 0x7fbfbc1a7fd0 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400ba50 tx=0x7fbfb400bf10 comp rx=0 tx=0).stop 2024-09-11T01:14:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.766+0000 7fbfc4035640 1 -- 172.21.15.78:0/3118143320 >> 172.21.15.78:0/3118143320 conn(0x7fbfbc06dfc0 msgr2=0x7fbfbc11c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.767+0000 7fbfc4035640 1 -- 172.21.15.78:0/3118143320 shutdown_connections 2024-09-11T01:14:45.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:45.767+0000 7fbfc4035640 1 -- 172.21.15.78:0/3118143320 wait complete. 2024-09-11T01:14:45.783 INFO:tasks.workunit.client.0.smithi078.stdout: 10 118775 40.76 MB/sec execute 51 sec latency 83.988 ms 2024-09-11T01:14:46.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:45 smithi110 ceph-mon[34712]: pgmap v817: 97 pgs: 97 active+clean; 1.8 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 687 op/s 2024-09-11T01:14:46.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:45 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3118143320' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:14:46.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:45 smithi112 ceph-mon[35384]: pgmap v817: 97 pgs: 97 active+clean; 1.8 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 687 op/s 2024-09-11T01:14:46.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:45 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3118143320' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:14:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:45 smithi078 ceph-mon[31315]: pgmap v817: 97 pgs: 97 active+clean; 1.8 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 687 op/s 2024-09-11T01:14:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:45 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3118143320' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:14:46.783 INFO:tasks.workunit.client.0.smithi078.stdout: 10 119477 40.77 MB/sec execute 52 sec latency 92.891 ms 2024-09-11T01:14:47.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:14:47.431 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:14:47.783 INFO:tasks.workunit.client.0.smithi078.stdout: 10 120327 40.76 MB/sec execute 53 sec latency 80.417 ms 2024-09-11T01:14:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.938+0000 7ffa6c72d640 1 -- 172.21.15.78:0/3233441140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa64108020 msgr2=0x7ffa6410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.938+0000 7ffa6c72d640 1 --2- 172.21.15.78:0/3233441140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa64108020 0x7ffa6410e820 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009ca0 tx=0x7ffa5c02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.938+0000 7ffa6c72d640 1 -- 172.21.15.78:0/3233441140 shutdown_connections 2024-09-11T01:14:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.938+0000 7ffa6c72d640 1 --2- 172.21.15.78:0/3233441140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa64108020 0x7ffa6410e820 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.938+0000 7ffa6c72d640 1 --2- 172.21.15.78:0/3233441140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa641076a0 0x7ffa64107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.938+0000 7ffa6c72d640 1 --2- 172.21.15.78:0/3233441140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa64101c90 0x7ffa64102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.938+0000 7ffa6c72d640 1 -- 172.21.15.78:0/3233441140 >> 172.21.15.78:0/3233441140 conn(0x7ffa640fd700 msgr2=0x7ffa640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.938+0000 7ffa6c72d640 1 -- 172.21.15.78:0/3233441140 shutdown_connections 2024-09-11T01:14:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.938+0000 7ffa6c72d640 1 -- 172.21.15.78:0/3233441140 wait complete. 2024-09-11T01:14:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa6c72d640 1 Processor -- start 2024-09-11T01:14:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa6c72d640 1 -- start start 2024-09-11T01:14:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa6c72d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa64101c90 0x7ffa6419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa6c72d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa641076a0 0x7ffa6419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa6c72d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa64108020 0x7ffa6419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa6c72d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa64111ce0 con 0x7ffa64101c90 2024-09-11T01:14:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa6c72d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffa64111b60 con 0x7ffa64108020 2024-09-11T01:14:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa6c72d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa64111e60 con 0x7ffa641076a0 2024-09-11T01:14:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa6a4c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa64101c90 0x7ffa6419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa6a4c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa64101c90 0x7ffa6419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56770/0 (socket says 172.21.15.78:56770) 2024-09-11T01:14:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa6a4c9640 1 -- 172.21.15.78:0/2264082805 learned_addr learned my addr 172.21.15.78:0/2264082805 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa69cc8640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa641076a0 0x7ffa6419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.939+0000 7ffa6acca640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa64108020 0x7ffa6419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.940+0000 7ffa6a4c9640 1 -- 172.21.15.78:0/2264082805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa641076a0 msgr2=0x7ffa6419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.940+0000 7ffa6a4c9640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa641076a0 0x7ffa6419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.940+0000 7ffa6a4c9640 1 -- 172.21.15.78:0/2264082805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa64108020 msgr2=0x7ffa6419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:47.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.940+0000 7ffa6a4c9640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa64108020 0x7ffa6419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.940+0000 7ffa6a4c9640 1 -- 172.21.15.78:0/2264082805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa641a5470 con 0x7ffa64101c90 2024-09-11T01:14:47.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.940+0000 7ffa6a4c9640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa64101c90 0x7ffa6419a090 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000b4a0 tx=0x7ffa6000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:47.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.940+0000 7ffa5b7fe640 1 -- 172.21.15.78:0/2264082805 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60014070 con 0x7ffa64101c90 2024-09-11T01:14:47.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.940+0000 7ffa6c72d640 1 -- 172.21.15.78:0/2264082805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa641a5760 con 0x7ffa64101c90 2024-09-11T01:14:47.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.940+0000 7ffa6c72d640 1 -- 172.21.15.78:0/2264082805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa641a5d70 con 0x7ffa64101c90 2024-09-11T01:14:47.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.940+0000 7ffa5b7fe640 1 -- 172.21.15.78:0/2264082805 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa60010ac0 con 0x7ffa64101c90 2024-09-11T01:14:47.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.940+0000 7ffa5b7fe640 1 -- 172.21.15.78:0/2264082805 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6001e060 con 0x7ffa64101c90 2024-09-11T01:14:47.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.943+0000 7ffa6eb02740 1 Processor -- start 2024-09-11T01:14:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.943+0000 7ffa6eb02740 1 -- start start 2024-09-11T01:14:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.943+0000 7ffa6eb02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2e4d0 0x5618b7a2e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6eb02740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2edf0 0x5618b7a3a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6eb02740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a3ad80 0x5618b7a3d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6eb02740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5618b79a4ea0 con 0x5618b7a2edf0 2024-09-11T01:14:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6eb02740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5618b79942d0 con 0x5618b7a3ad80 2024-09-11T01:14:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6eb02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5618b79980e0 con 0x5618b7a2e4d0 2024-09-11T01:14:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6acca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2e4d0 0x5618b7a2e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6acca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2e4d0 0x5618b7a2e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:50874/0 (socket says 172.21.15.78:50874) 2024-09-11T01:14:47.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6acca640 1 -- 172.21.15.78:0/1803811689 learned_addr learned my addr 172.21.15.78:0/1803811689 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:47.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa69cc8640 1 --2- 172.21.15.78:0/1803811689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a3ad80 0x5618b7a3d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6acca640 1 -- 172.21.15.78:0/1803811689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a3ad80 msgr2=0x5618b7a3d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:47.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6acca640 1 --2- 172.21.15.78:0/1803811689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a3ad80 0x5618b7a3d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6acca640 1 -- 172.21.15.78:0/1803811689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2edf0 msgr2=0x5618b7a3a840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:14:47.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6acca640 1 --2- 172.21.15.78:0/1803811689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2edf0 0x5618b7a3a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.944+0000 7ffa6acca640 1 -- 172.21.15.78:0/1803811689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618b76f2650 con 0x5618b7a2e4d0 2024-09-11T01:14:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.945+0000 7ffa5b7fe640 1 -- 172.21.15.78:0/2264082805 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ffa60017b60 con 0x7ffa64101c90 2024-09-11T01:14:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.945+0000 7ffa5b7fe640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ffa500b54d0 0x7ffa500b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.946+0000 7ffa6a4c9640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ffa500b54d0 0x7ffa500b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.946+0000 7ffa5b7fe640 1 -- 172.21.15.78:0/2264082805 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7ffa600e53f0 con 0x7ffa64101c90 2024-09-11T01:14:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.946+0000 7ffa6acca640 1 --2- 172.21.15.78:0/1803811689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2e4d0 0x5618b7a2e8b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c035000 tx=0x7ffa5c005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5a507d9f4bb9a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.946+0000 7ffa6a4c9640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ffa500b54d0 0x7ffa500b7990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5618b7a2d1e0 tx=0x7ffa6000bf90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:47.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.946+0000 7ffa58ff9640 1 -- 172.21.15.78:0/1803811689 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c031db0 con 0x5618b7a2e4d0 2024-09-11T01:14:47.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.947+0000 7ffa58ff9640 1 -- 172.21.15.78:0/1803811689 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa5c036030 con 0x5618b7a2e4d0 2024-09-11T01:14:47.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.947+0000 7ffa58ff9640 1 -- 172.21.15.78:0/1803811689 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0042a0 con 0x5618b7a2e4d0 2024-09-11T01:14:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.948+0000 7ffa6eb02740 1 -- 172.21.15.78:0/1803811689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2e4d0 msgr2=0x5618b7a2e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.948+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/1803811689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2e4d0 0x5618b7a2e8b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c035000 tx=0x7ffa5c005fa0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.948+0000 7ffa6eb02740 1 -- 172.21.15.78:0/1803811689 shutdown_connections 2024-09-11T01:14:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.948+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/1803811689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a3ad80 0x5618b7a3d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.948+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/1803811689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2edf0 0x5618b7a3a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.948+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/1803811689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2e4d0 0x5618b7a2e8b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.948+0000 7ffa6eb02740 1 -- 172.21.15.78:0/1803811689 >> 172.21.15.78:0/1803811689 conn(0x5618b7a2c300 msgr2=0x5618b7a2d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.948+0000 7ffa6eb02740 1 -- 172.21.15.78:0/1803811689 shutdown_connections 2024-09-11T01:14:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.948+0000 7ffa6eb02740 1 -- 172.21.15.78:0/1803811689 wait complete. 2024-09-11T01:14:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6eb02740 1 -- start start 2024-09-11T01:14:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6eb02740 1 Processor -- start 2024-09-11T01:14:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6eb02740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2e4d0 0x5618b7a3a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6eb02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2edf0 0x5618b7a37b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6eb02740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a3ad80 0x5618b7a38190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6eb02740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5618b77f5700 con 0x5618b7a2e4d0 2024-09-11T01:14:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6eb02740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5618b792f160 con 0x5618b7a3ad80 2024-09-11T01:14:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6eb02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5618b7a3d820 con 0x5618b7a2edf0 2024-09-11T01:14:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6eb02740 10 client.0 _ll_register_callbacks cb 0x5618b79489e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6acca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2e4d0 0x5618b7a3a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6acca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2e4d0 0x5618b7a3a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56772/0 (socket says 172.21.15.78:56772) 2024-09-11T01:14:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6acca640 1 -- 172.21.15.78:0/2939476468 learned_addr learned my addr 172.21.15.78:0/2939476468 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa69cc8640 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2edf0 0x5618b7a37b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.949+0000 7ffa6acca640 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a3ad80 0x5618b7a38190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa6acca640 1 -- 172.21.15.78:0/2939476468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2edf0 msgr2=0x5618b7a37b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa6acca640 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2edf0 0x5618b7a37b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa6acca640 1 -- 172.21.15.78:0/2939476468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a3ad80 msgr2=0x5618b7a38190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa6acca640 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a3ad80 0x5618b7a38190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa6acca640 1 -- 172.21.15.78:0/2939476468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618b795cc90 con 0x5618b7a2e4d0 2024-09-11T01:14:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa69cc8640 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2edf0 0x5618b7a37b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:14:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa6acca640 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2e4d0 0x5618b7a3a6c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009620 tx=0x7ffa5c003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc0ccbdc282d648 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:14:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa4dffb640 1 -- 172.21.15.78:0/2939476468 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c059060 con 0x5618b7a2e4d0 2024-09-11T01:14:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa4dffb640 1 -- 172.21.15.78:0/2939476468 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7ffa5c02f660 con 0x5618b7a2e4d0 2024-09-11T01:14:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2939476468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa5c036030 con 0x5618b7a2e4d0 2024-09-11T01:14:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa4dffb640 1 -- 172.21.15.78:0/2939476468 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ffa5c02f860 con 0x5618b7a2e4d0 2024-09-11T01:14:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa6eb02740 10 client.18363 fetch_fsmap learned FSMap version 64 2024-09-11T01:14:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.950+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2939476468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5618b7a519e0 con 0x5618b7a2e4d0 2024-09-11T01:14:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.951+0000 7ffa4dffb640 1 -- 172.21.15.78:0/2939476468 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa5c036030 con 0x5618b7a2e4d0 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.951+0000 7ffa4dffb640 1 -- 172.21.15.78:0/2939476468 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7ffa5c043070 con 0x5618b7a2e4d0 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.951+0000 7ffa6eb02740 10 client.18363 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.951+0000 7ffa6eb02740 10 client.18363 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.951+0000 7ffa6eb02740 20 client.18363 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.951+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5618b7a52d30 0x5618b7a55120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.951+0000 7ffa6eb02740 4 client.18363 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.951+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2939476468 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618b79a6df0 con 0x5618b7a52d30 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.951+0000 7ffa69cc8640 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5618b7a52d30 0x5618b7a55120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.951+0000 7ffa69cc8640 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5618b7a52d30 0x5618b7a55120 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a8feefbece62831a server_cookie=9a63b6322710ec02 in_seq=0 out_seq=0 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.951+0000 7ffa4dffb640 10 client.18363 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.952+0000 7ffa4dffb640 1 -- 172.21.15.78:0/2939476468 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5618b79a6df0 con 0x5618b7a52d30 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.952+0000 7ffa4dffb640 10 client.18363 handle_command_reply: tid=0 2024-09-11T01:14:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.952+0000 7ffa6eb02740 1 client.18363 shutdown 2024-09-11T01:14:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.952+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2939476468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2e4d0 msgr2=0x5618b7a3a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.952+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2e4d0 0x5618b7a3a6c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009620 tx=0x7ffa5c003db0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.952+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2939476468 shutdown_connections 2024-09-11T01:14:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.952+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5618b7a52d30 0x5618b7a55120 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.952+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a3ad80 0x5618b7a38190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.952+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2edf0 0x5618b7a37b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.952+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2939476468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2e4d0 0x5618b7a3a6c0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.952+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2939476468 >> 172.21.15.78:0/2939476468 conn(0x5618b7a2c300 msgr2=0x5618b7a31ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.952+0000 7ffa4dffb640 0 client.18363 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:14:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.953+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2939476468 shutdown_connections 2024-09-11T01:14:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.953+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2939476468 wait complete. 2024-09-11T01:14:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.953+0000 7ffa6eb02740 20 client.18363 trim_cache size 0 max 16384 2024-09-11T01:14:47.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.970+0000 7ffa6eb02740 1 Processor -- start 2024-09-11T01:14:47.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.970+0000 7ffa6eb02740 1 -- start start 2024-09-11T01:14:47.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa6eb02740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a2e4d0 0x5618b7a46a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa6eb02740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2edf0 0x5618b7a39850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa6eb02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a3ad80 0x5618b7a3c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa6eb02740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa641ac3f0 con 0x5618b7a2edf0 2024-09-11T01:14:47.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa6eb02740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffa641ae0b0 con 0x5618b7a2e4d0 2024-09-11T01:14:47.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa6eb02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa641acac0 con 0x5618b7a3ad80 2024-09-11T01:14:47.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa69cc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2edf0 0x5618b7a39850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa69cc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2edf0 0x5618b7a39850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56782/0 (socket says 172.21.15.78:56782) 2024-09-11T01:14:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa69cc8640 1 -- 172.21.15.78:0/2907217074 learned_addr learned my addr 172.21.15.78:0/2907217074 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa6acca640 1 --2- 172.21.15.78:0/2907217074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a3ad80 0x5618b7a3c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa6acca640 1 --2- 172.21.15.78:0/2907217074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a2e4d0 0x5618b7a46a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa69cc8640 1 -- 172.21.15.78:0/2907217074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a3ad80 msgr2=0x5618b7a3c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa69cc8640 1 --2- 172.21.15.78:0/2907217074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a3ad80 0x5618b7a3c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa69cc8640 1 -- 172.21.15.78:0/2907217074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a2e4d0 msgr2=0x5618b7a46a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa69cc8640 1 --2- 172.21.15.78:0/2907217074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a2e4d0 0x5618b7a46a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa69cc8640 1 -- 172.21.15.78:0/2907217074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618b7a3a2f0 con 0x5618b7a2edf0 2024-09-11T01:14:47.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.971+0000 7ffa69cc8640 1 --2- 172.21.15.78:0/2907217074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2edf0 0x5618b7a39850 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ffa540129b0 tx=0x7ffa54005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2615e7dbb7fb0250 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:47.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.972+0000 7ffa4ffff640 1 -- 172.21.15.78:0/2907217074 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa540122d0 con 0x5618b7a2edf0 2024-09-11T01:14:47.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.972+0000 7ffa4ffff640 1 -- 172.21.15.78:0/2907217074 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa5400f8a0 con 0x5618b7a2edf0 2024-09-11T01:14:47.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.972+0000 7ffa4ffff640 1 -- 172.21.15.78:0/2907217074 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa540053a0 con 0x5618b7a2edf0 2024-09-11T01:14:47.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.972+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2907217074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2edf0 msgr2=0x5618b7a39850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:47.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.972+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2907217074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2edf0 0x5618b7a39850 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ffa540129b0 tx=0x7ffa54005f90 comp rx=0 tx=0).stop 2024-09-11T01:14:47.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2907217074 shutdown_connections 2024-09-11T01:14:47.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2907217074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a3ad80 0x5618b7a3c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2907217074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a2edf0 0x5618b7a39850 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2907217074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a2e4d0 0x5618b7a46a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2907217074 >> 172.21.15.78:0/2907217074 conn(0x5618b7a2c300 msgr2=0x5618b7a39140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2907217074 shutdown_connections 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2907217074 wait complete. 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 -- start start 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 Processor -- start 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2e4d0 0x5618b7a42af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a3ad80 0x5618b7a43030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a43570 0x5618b7a3daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa641ad920 con 0x5618b7a3ad80 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffa641ad190 con 0x5618b7a43570 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa641aa3d0 con 0x5618b7a2e4d0 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.973+0000 7ffa6eb02740 10 client.0 _ll_register_callbacks cb 0x7ffa540097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:47.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa69cc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a3ad80 0x5618b7a43030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa6acca640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a43570 0x5618b7a3daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa6acca640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a43570 0x5618b7a3daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38276/0 (socket says 172.21.15.78:38276) 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa6acca640 1 -- 172.21.15.78:0/2962370182 learned_addr learned my addr 172.21.15.78:0/2962370182 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa6acca640 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2e4d0 0x5618b7a42af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa6acca640 1 -- 172.21.15.78:0/2962370182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2e4d0 msgr2=0x5618b7a42af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa6acca640 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2e4d0 0x5618b7a42af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa6acca640 1 -- 172.21.15.78:0/2962370182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a3ad80 msgr2=0x5618b7a43030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa6acca640 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a3ad80 0x5618b7a43030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa6acca640 1 -- 172.21.15.78:0/2962370182 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618b7a3e110 con 0x5618b7a43570 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa69cc8640 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a3ad80 0x5618b7a43030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa6acca640 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a43570 0x5618b7a3daa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009ca0 tx=0x7ffa5c005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf50f2dbb9af9907 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:14:47.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.974+0000 7ffa4dffb640 1 -- 172.21.15.78:0/2962370182 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c059070 con 0x5618b7a43570 2024-09-11T01:14:47.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa4dffb640 1 -- 172.21.15.78:0/2962370182 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7ffa5c02fa00 con 0x5618b7a43570 2024-09-11T01:14:47.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2962370182 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa5400f8a0 con 0x5618b7a43570 2024-09-11T01:14:47.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa4dffb640 1 -- 172.21.15.78:0/2962370182 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ffa5c003970 con 0x5618b7a43570 2024-09-11T01:14:47.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa6eb02740 10 client.27521 fetch_fsmap learned FSMap version 64 2024-09-11T01:14:47.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2962370182 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5618b7ac0290 con 0x5618b7a43570 2024-09-11T01:14:47.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa4dffb640 1 -- 172.21.15.78:0/2962370182 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa5400f8a0 con 0x5618b7a43570 2024-09-11T01:14:47.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa4dffb640 1 -- 172.21.15.78:0/2962370182 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7ffa5c043070 con 0x5618b7a43570 2024-09-11T01:14:47.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa6eb02740 10 client.27521 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:14:47.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa6eb02740 10 client.27521 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:14:47.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa6eb02740 20 client.27521 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:47.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5618b7ac0820 0x5618b7ac2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:47.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa6eb02740 4 client.27521 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:14:47.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.975+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2962370182 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618b79a28a0 con 0x5618b7ac0820 2024-09-11T01:14:47.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.976+0000 7ffa69cc8640 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5618b7ac0820 0x5618b7ac2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:47.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.976+0000 7ffa69cc8640 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5618b7ac0820 0x5618b7ac2c40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3e10be1a821f461f server_cookie=787a11b8ca5c8e4d in_seq=0 out_seq=0 2024-09-11T01:14:47.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.976+0000 7ffa4dffb640 10 client.27521 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:14:47.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa4dffb640 1 -- 172.21.15.78:0/2962370182 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618b79a28a0 con 0x5618b7ac0820 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa4dffb640 10 client.27521 handle_command_reply: tid=0 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa6eb02740 1 client.27521 shutdown 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2962370182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a43570 msgr2=0x5618b7a3daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a43570 0x5618b7a3daa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009ca0 tx=0x7ffa5c005c50 comp rx=0 tx=0).stop 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2962370182 shutdown_connections 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5618b7ac0820 0x5618b7ac2c40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b7a43570 0x5618b7a3daa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618b7a3ad80 0x5618b7a43030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa6eb02740 1 --2- 172.21.15.78:0/2962370182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b7a2e4d0 0x5618b7a42af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2962370182 >> 172.21.15.78:0/2962370182 conn(0x5618b7a2c300 msgr2=0x5618b7a2b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa4dffb640 0 client.27521 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2962370182 shutdown_connections 2024-09-11T01:14:47.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa6eb02740 1 -- 172.21.15.78:0/2962370182 wait complete. 2024-09-11T01:14:47.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.977+0000 7ffa6eb02740 20 client.27521 trim_cache size 0 max 16384 2024-09-11T01:14:47.985 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:14:47.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.978+0000 7ffa6c72d640 1 -- 172.21.15.78:0/2264082805 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ffa500b54d0 msgr2=0x7ffa500b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:47.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.978+0000 7ffa6c72d640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ffa500b54d0 0x7ffa500b7990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5618b7a2d1e0 tx=0x7ffa6000bf90 comp rx=0 tx=0).stop 2024-09-11T01:14:47.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.978+0000 7ffa6c72d640 1 -- 172.21.15.78:0/2264082805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa64101c90 msgr2=0x7ffa6419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:47.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.978+0000 7ffa6c72d640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa64101c90 0x7ffa6419a090 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000b4a0 tx=0x7ffa6000b960 comp rx=0 tx=0).stop 2024-09-11T01:14:47.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.978+0000 7ffa6c72d640 1 -- 172.21.15.78:0/2264082805 shutdown_connections 2024-09-11T01:14:47.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.978+0000 7ffa6c72d640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ffa500b54d0 0x7ffa500b7990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.978+0000 7ffa6c72d640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa64108020 0x7ffa6419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.978+0000 7ffa6c72d640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa641076a0 0x7ffa6419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.978+0000 7ffa6c72d640 1 --2- 172.21.15.78:0/2264082805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa64101c90 0x7ffa6419a090 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:47.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.978+0000 7ffa6c72d640 1 -- 172.21.15.78:0/2264082805 >> 172.21.15.78:0/2264082805 conn(0x7ffa640fd700 msgr2=0x7ffa640ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:47.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.978+0000 7ffa6c72d640 1 -- 172.21.15.78:0/2264082805 shutdown_connections 2024-09-11T01:14:47.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:47.978+0000 7ffa6c72d640 1 -- 172.21.15.78:0/2264082805 wait complete. 2024-09-11T01:14:48.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:47 smithi110 ceph-mon[34712]: pgmap v818: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 858 op/s 2024-09-11T01:14:48.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:47 smithi112 ceph-mon[35384]: pgmap v818: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 858 op/s 2024-09-11T01:14:48.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:47 smithi078 ceph-mon[31315]: pgmap v818: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 858 op/s 2024-09-11T01:14:48.783 INFO:tasks.workunit.client.0.smithi078.stdout: 10 121266 41.00 MB/sec execute 54 sec latency 96.140 ms 2024-09-11T01:14:49.615 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:14:49.783 INFO:tasks.workunit.client.0.smithi078.stdout: 10 121996 41.06 MB/sec execute 55 sec latency 80.284 ms 2024-09-11T01:14:49.873 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:49 smithi078 ceph-mon[31315]: pgmap v819: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 868 op/s 2024-09-11T01:14:49.934 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:14:50.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:49 smithi110 ceph-mon[34712]: pgmap v819: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 868 op/s 2024-09-11T01:14:50.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:49 smithi112 ceph-mon[35384]: pgmap v819: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 868 op/s 2024-09-11T01:14:50.783 INFO:tasks.workunit.client.0.smithi078.stdout: 10 122713 40.90 MB/sec execute 56 sec latency 78.532 ms 2024-09-11T01:14:51.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.321+0000 7f2518ad3640 1 -- 172.21.15.78:0/1346954990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2514075500 msgr2=0x7f25140758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:51.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.321+0000 7f2518ad3640 1 --2- 172.21.15.78:0/1346954990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2514075500 0x7f25140758e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2504009900 tx=0x7f250402a480 comp rx=0 tx=0).stop 2024-09-11T01:14:51.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.321+0000 7f2518ad3640 1 -- 172.21.15.78:0/1346954990 shutdown_connections 2024-09-11T01:14:51.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.321+0000 7f2518ad3640 1 --2- 172.21.15.78:0/1346954990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f251410f3b0 0x7f25141117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.321+0000 7f2518ad3640 1 --2- 172.21.15.78:0/1346954990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514075e20 0x7f251410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.321+0000 7f2518ad3640 1 --2- 172.21.15.78:0/1346954990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2514075500 0x7f25140758e0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.321+0000 7f2518ad3640 1 -- 172.21.15.78:0/1346954990 >> 172.21.15.78:0/1346954990 conn(0x7f25140fe830 msgr2=0x7f2514100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:51.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.321+0000 7f2518ad3640 1 -- 172.21.15.78:0/1346954990 shutdown_connections 2024-09-11T01:14:51.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.321+0000 7f2518ad3640 1 -- 172.21.15.78:0/1346954990 wait complete. 2024-09-11T01:14:51.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.322+0000 7f2518ad3640 1 Processor -- start 2024-09-11T01:14:51.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.322+0000 7f2518ad3640 1 -- start start 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2518ad3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2514075500 0x7f251419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2518ad3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514075e20 0x7f251419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2518ad3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251410f3b0 0x7f25141a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2518ad3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2514115040 con 0x7f251410f3b0 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2518ad3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2514114ec0 con 0x7f2514075e20 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2518ad3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25141151c0 con 0x7f2514075500 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2512d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251410f3b0 0x7f25141a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f251259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2514075500 0x7f251419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2512d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251410f3b0 0x7f25141a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56816/0 (socket says 172.21.15.78:56816) 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2512d9d640 1 -- 172.21.15.78:0/735586823 learned_addr learned my addr 172.21.15.78:0/735586823 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2511d9b640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514075e20 0x7f251419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2512d9d640 1 -- 172.21.15.78:0/735586823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2514075500 msgr2=0x7f251419b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2512d9d640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2514075500 0x7f251419b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2512d9d640 1 -- 172.21.15.78:0/735586823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514075e20 msgr2=0x7f251419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2512d9d640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514075e20 0x7f251419ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2512d9d640 1 -- 172.21.15.78:0/735586823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25141a15c0 con 0x7f251410f3b0 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f2511d9b640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514075e20 0x7f251419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.323+0000 7f251259c640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2514075500 0x7f251419b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.324+0000 7f2512d9d640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251410f3b0 0x7f25141a0df0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f250c00eb10 tx=0x7f250c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.324+0000 7f25037fe640 1 -- 172.21.15.78:0/735586823 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c004b20 con 0x7f251410f3b0 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.324+0000 7f25037fe640 1 -- 172.21.15.78:0/735586823 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250c004cc0 con 0x7f251410f3b0 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.324+0000 7f25037fe640 1 -- 172.21.15.78:0/735586823 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c0107b0 con 0x7f251410f3b0 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.324+0000 7f2518ad3640 1 -- 172.21.15.78:0/735586823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25141a18b0 con 0x7f251410f3b0 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.324+0000 7f2518ad3640 1 -- 172.21.15.78:0/735586823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25141a7f70 con 0x7f251410f3b0 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.328+0000 7f251aea8740 1 Processor -- start 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.328+0000 7f25037fe640 1 -- 172.21.15.78:0/735586823 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f250c005020 con 0x7f251410f3b0 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.328+0000 7f251aea8740 1 -- start start 2024-09-11T01:14:51.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.329+0000 7f251aea8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8591f90f0 0x55e8591f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.329+0000 7f251aea8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 0x55e859205460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.329+0000 7f251aea8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8592059a0 0x55e859207d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.329+0000 7f251aea8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e85916fea0 con 0x55e8591f90f0 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.329+0000 7f251aea8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e85915f2d0 con 0x55e8592059a0 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.329+0000 7f251aea8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e8591630e0 con 0x55e8591f9a10 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.329+0000 7f251259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 0x55e859205460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.329+0000 7f251259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 0x55e859205460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:50930/0 (socket says 172.21.15.78:50930) 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.329+0000 7f251259c640 1 -- 172.21.15.78:0/2036845327 learned_addr learned my addr 172.21.15.78:0/2036845327 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.329+0000 7f2511d9b640 1 --2- 172.21.15.78:0/2036845327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8592059a0 0x55e859207d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.329+0000 7f2511d9b640 1 --2- 172.21.15.78:0/2036845327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8591f90f0 0x55e8591f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.329+0000 7f25037fe640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f24fc0b54d0 0x7f24fc0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.330+0000 7f25037fe640 1 -- 172.21.15.78:0/735586823 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f250c0e6740 con 0x7f251410f3b0 2024-09-11T01:14:51.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.330+0000 7f251259c640 1 -- 172.21.15.78:0/2036845327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8592059a0 msgr2=0x55e859207d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.330+0000 7f251259c640 1 --2- 172.21.15.78:0/2036845327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8592059a0 0x55e859207d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.330+0000 7f2512d9d640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f24fc0b54d0 0x7f24fc0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.330+0000 7f251259c640 1 -- 172.21.15.78:0/2036845327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8591f90f0 msgr2=0x55e8591f94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.330+0000 7f251259c640 1 --2- 172.21.15.78:0/2036845327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8591f90f0 0x55e8591f94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.330+0000 7f251259c640 1 -- 172.21.15.78:0/2036845327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e858ebd650 con 0x55e8591f9a10 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.330+0000 7f2512d9d640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f24fc0b54d0 0x7f24fc0b7990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f250c0e3ed0 tx=0x7f250c005f40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.330+0000 7f251259c640 1 --2- 172.21.15.78:0/2036845327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 0x55e859205460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2504009230 tx=0x7f2504005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beeae28361d19589 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.330+0000 7f2500ff9640 1 -- 172.21.15.78:0/2036845327 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504035040 con 0x55e8591f9a10 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.331+0000 7f2500ff9640 1 -- 172.21.15.78:0/2036845327 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2504004770 con 0x55e8591f9a10 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.331+0000 7f2500ff9640 1 -- 172.21.15.78:0/2036845327 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504003950 con 0x55e8591f9a10 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.331+0000 7f251aea8740 1 -- 172.21.15.78:0/2036845327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 msgr2=0x55e859205460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.331+0000 7f251aea8740 1 --2- 172.21.15.78:0/2036845327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 0x55e859205460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2504009230 tx=0x7f2504005c00 comp rx=0 tx=0).stop 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.331+0000 7f251aea8740 1 -- 172.21.15.78:0/2036845327 shutdown_connections 2024-09-11T01:14:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.331+0000 7f251aea8740 1 --2- 172.21.15.78:0/2036845327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8592059a0 0x55e859207d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.331+0000 7f251aea8740 1 --2- 172.21.15.78:0/2036845327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 0x55e859205460 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.332+0000 7f251aea8740 1 --2- 172.21.15.78:0/2036845327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8591f90f0 0x55e8591f94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.332+0000 7f251aea8740 1 -- 172.21.15.78:0/2036845327 >> 172.21.15.78:0/2036845327 conn(0x55e8591f6f20 msgr2=0x55e8591f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.332+0000 7f251aea8740 1 -- 172.21.15.78:0/2036845327 shutdown_connections 2024-09-11T01:14:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.332+0000 7f251aea8740 1 -- 172.21.15.78:0/2036845327 wait complete. 2024-09-11T01:14:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.333+0000 7f251aea8740 1 -- start start 2024-09-11T01:14:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.333+0000 7f251aea8740 1 Processor -- start 2024-09-11T01:14:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.333+0000 7f251aea8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 0x55e859205280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.333+0000 7f251aea8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8591f9a10 0x55e8592026d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.333+0000 7f251aea8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8592059a0 0x55e859202d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.333+0000 7f251aea8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e858fc0700 con 0x55e8592059a0 2024-09-11T01:14:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.333+0000 7f251aea8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e8590fa160 con 0x55e8591f9a10 2024-09-11T01:14:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.333+0000 7f251aea8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e8592085c0 con 0x55e8591f90f0 2024-09-11T01:14:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.333+0000 7f251aea8740 10 client.0 _ll_register_callbacks cb 0x55e8591139e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.333+0000 7f2511d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8592059a0 0x55e859202d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.333+0000 7f2511d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 0x55e859205280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.334+0000 7f2511d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 0x55e859205280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:50946/0 (socket says 172.21.15.78:50946) 2024-09-11T01:14:51.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.334+0000 7f2511d9b640 1 -- 172.21.15.78:0/1514132799 learned_addr learned my addr 172.21.15.78:0/1514132799 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:51.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.334+0000 7f251259c640 1 --2- 172.21.15.78:0/1514132799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8591f9a10 0x55e8592026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.334+0000 7f2511d9b640 1 -- 172.21.15.78:0/1514132799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 msgr2=0x55e859205280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:51.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.334+0000 7f2511d9b640 1 --2- 172.21.15.78:0/1514132799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 0x55e859205280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.334+0000 7f2511d9b640 1 -- 172.21.15.78:0/1514132799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8591f9a10 msgr2=0x55e8592026d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:51.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.334+0000 7f2511d9b640 1 --2- 172.21.15.78:0/1514132799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8591f9a10 0x55e8592026d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.334+0000 7f2511d9b640 1 -- 172.21.15.78:0/1514132799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e859127c90 con 0x55e8592059a0 2024-09-11T01:14:51.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.334+0000 7f2511d9b640 1 --2- 172.21.15.78:0/1514132799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8592059a0 0x55e859202d00 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f2508013d90 tx=0x7f2508004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55131c0f5f0ebf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:51.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.334+0000 7f24f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.334+0000 7f24f9ffb640 1 -- 172.21.15.78:0/1514132799 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508026060 con 0x55e8592059a0 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.335+0000 7f24f9ffb640 1 -- 172.21.15.78:0/1514132799 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f250800fc50 con 0x55e8592059a0 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.335+0000 7f251aea8740 1 -- 172.21.15.78:0/1514132799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2504004770 con 0x55e8592059a0 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.335+0000 7f24f9ffb640 1 -- 172.21.15.78:0/1514132799 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f250800fe50 con 0x55e8592059a0 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.335+0000 7f251aea8740 10 client.18381 fetch_fsmap learned FSMap version 64 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.335+0000 7f251aea8740 1 -- 172.21.15.78:0/1514132799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e859213f40 con 0x55e8592059a0 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.335+0000 7f24f9ffb640 1 -- 172.21.15.78:0/1514132799 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2504004770 con 0x55e8592059a0 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.336+0000 7f24f9ffb640 1 -- 172.21.15.78:0/1514132799 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f25080129a0 con 0x55e8592059a0 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.336+0000 7f251aea8740 10 client.18381 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.336+0000 7f251aea8740 10 client.18381 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.336+0000 7f251aea8740 20 client.18381 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.336+0000 7f251aea8740 1 --2- 172.21.15.78:0/1514132799 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55e8592154a0 0x55e8592178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.336+0000 7f251aea8740 4 client.18381 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.336+0000 7f251aea8740 1 -- 172.21.15.78:0/1514132799 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e859171df0 con 0x55e8592154a0 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.336+0000 7f251259c640 1 --2- 172.21.15.78:0/1514132799 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55e8592154a0 0x55e8592178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.337+0000 7f24f9ffb640 10 client.18381 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.337+0000 7f251259c640 1 --2- 172.21.15.78:0/1514132799 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55e8592154a0 0x55e8592178e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b762f7b92762516c server_cookie=2e1531d2aa9fd0ea in_seq=0 out_seq=0 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.339+0000 7f24f9ffb640 1 -- 172.21.15.78:0/1514132799 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e859171df0 con 0x55e8592154a0 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f24f9ffb640 10 client.18381 handle_command_reply: tid=0 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f251aea8740 1 client.18381 shutdown 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f251aea8740 1 -- 172.21.15.78:0/1514132799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8592059a0 msgr2=0x55e859202d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f251aea8740 1 --2- 172.21.15.78:0/1514132799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8592059a0 0x55e859202d00 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f2508013d90 tx=0x7f2508004730 comp rx=0 tx=0).stop 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f251aea8740 1 -- 172.21.15.78:0/1514132799 shutdown_connections 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f251aea8740 1 --2- 172.21.15.78:0/1514132799 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55e8592154a0 0x55e8592178e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f24f9ffb640 0 client.18381 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f251aea8740 1 --2- 172.21.15.78:0/1514132799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8592059a0 0x55e859202d00 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f251aea8740 1 --2- 172.21.15.78:0/1514132799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8591f9a10 0x55e8592026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f251aea8740 1 --2- 172.21.15.78:0/1514132799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 0x55e859205280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f251aea8740 1 -- 172.21.15.78:0/1514132799 >> 172.21.15.78:0/1514132799 conn(0x55e8591f6f20 msgr2=0x55e8591fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:51.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f251aea8740 1 -- 172.21.15.78:0/1514132799 shutdown_connections 2024-09-11T01:14:51.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.340+0000 7f251aea8740 1 -- 172.21.15.78:0/1514132799 wait complete. 2024-09-11T01:14:51.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.341+0000 7f251aea8740 20 client.18381 trim_cache size 0 max 16384 2024-09-11T01:14:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.357+0000 7f251aea8740 1 Processor -- start 2024-09-11T01:14:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.357+0000 7f251aea8740 1 -- start start 2024-09-11T01:14:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.357+0000 7f251aea8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8591f90f0 0x55e8592117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.357+0000 7f251aea8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 0x55e859206d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.357+0000 7f251aea8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8592059a0 0x55e8592072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.357+0000 7f251aea8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f25141b0830 con 0x55e8591f90f0 2024-09-11T01:14:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.357+0000 7f251aea8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25141b24f0 con 0x55e8592059a0 2024-09-11T01:14:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.357+0000 7f251aea8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25141b0f00 con 0x55e8591f9a10 2024-09-11T01:14:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.358+0000 7f2511d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8591f90f0 0x55e8592117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.358+0000 7f251259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 0x55e859206d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.358+0000 7f2511d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8591f90f0 0x55e8592117f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56830/0 (socket says 172.21.15.78:56830) 2024-09-11T01:14:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.358+0000 7f251259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 0x55e859206d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:50950/0 (socket says 172.21.15.78:50950) 2024-09-11T01:14:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.358+0000 7f2511d9b640 1 -- 172.21.15.78:0/311868813 learned_addr learned my addr 172.21.15.78:0/311868813 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.358+0000 7f2511d9b640 1 --2- 172.21.15.78:0/311868813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8592059a0 0x55e8592072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.358+0000 7f2511d9b640 1 -- 172.21.15.78:0/311868813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 msgr2=0x55e859206d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.358+0000 7f2511d9b640 1 --2- 172.21.15.78:0/311868813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 0x55e859206d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.358+0000 7f2511d9b640 1 -- 172.21.15.78:0/311868813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8592059a0 msgr2=0x55e8592072d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.358+0000 7f2511d9b640 1 --2- 172.21.15.78:0/311868813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8592059a0 0x55e8592072d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.358+0000 7f2511d9b640 1 -- 172.21.15.78:0/311868813 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8591f9840 con 0x55e8591f90f0 2024-09-11T01:14:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.358+0000 7f2511d9b640 1 --2- 172.21.15.78:0/311868813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8591f90f0 0x55e8592117f0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2508013b20 tx=0x7f2508009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a221c802273721c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.359+0000 7f24fbfff640 1 -- 172.21.15.78:0/311868813 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508009870 con 0x55e8591f90f0 2024-09-11T01:14:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.359+0000 7f24fbfff640 1 -- 172.21.15.78:0/311868813 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2508013390 con 0x55e8591f90f0 2024-09-11T01:14:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.359+0000 7f24fbfff640 1 -- 172.21.15.78:0/311868813 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25080124e0 con 0x55e8591f90f0 2024-09-11T01:14:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.360+0000 7f251aea8740 1 -- 172.21.15.78:0/311868813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8591f90f0 msgr2=0x55e8592117f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.360+0000 7f251aea8740 1 --2- 172.21.15.78:0/311868813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8591f90f0 0x55e8592117f0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2508013b20 tx=0x7f2508009bd0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.360+0000 7f251aea8740 1 -- 172.21.15.78:0/311868813 shutdown_connections 2024-09-11T01:14:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.360+0000 7f251aea8740 1 --2- 172.21.15.78:0/311868813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8592059a0 0x55e8592072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.360+0000 7f251aea8740 1 --2- 172.21.15.78:0/311868813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f9a10 0x55e859206d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.360+0000 7f251aea8740 1 --2- 172.21.15.78:0/311868813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8591f90f0 0x55e8592117f0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.360+0000 7f251aea8740 1 -- 172.21.15.78:0/311868813 >> 172.21.15.78:0/311868813 conn(0x55e8591f6f20 msgr2=0x55e8591f7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.360+0000 7f251aea8740 1 -- 172.21.15.78:0/311868813 shutdown_connections 2024-09-11T01:14:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.360+0000 7f251aea8740 1 -- 172.21.15.78:0/311868813 wait complete. 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.360+0000 7f251aea8740 1 -- start start 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.360+0000 7f251aea8740 1 Processor -- start 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.361+0000 7f251aea8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 0x55e85920da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.361+0000 7f251aea8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8591f9a10 0x55e85920df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.361+0000 7f2511d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 0x55e85920da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.361+0000 7f2511d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 0x55e85920da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:50962/0 (socket says 172.21.15.78:50962) 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.361+0000 7f251259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8591f9a10 0x55e85920df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.361+0000 7f2511d9b640 1 -- 172.21.15.78:0/2424508337 learned_addr learned my addr 172.21.15.78:0/2424508337 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.361+0000 7f251aea8740 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8592059a0 0x55e859208810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.361+0000 7f251aea8740 1 -- 172.21.15.78:0/2424508337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f25141b1d60 con 0x55e8592059a0 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.361+0000 7f251aea8740 1 -- 172.21.15.78:0/2424508337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25141b15d0 con 0x55e8591f9a10 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.361+0000 7f251aea8740 1 -- 172.21.15.78:0/2424508337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25141ae830 con 0x55e8591f90f0 2024-09-11T01:14:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.362+0000 7f2511d9b640 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8592059a0 0x55e859208810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.362+0000 7f251aea8740 10 client.0 _ll_register_callbacks cb 0x7f2508013590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.362+0000 7f2511d9b640 1 -- 172.21.15.78:0/2424508337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8591f9a10 msgr2=0x55e85920df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.362+0000 7f2511d9b640 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8591f9a10 0x55e85920df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.362+0000 7f2511d9b640 1 -- 172.21.15.78:0/2424508337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8592059a0 msgr2=0x55e859208810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.362+0000 7f2511d9b640 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8592059a0 0x55e859208810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.362+0000 7f2511d9b640 1 -- 172.21.15.78:0/2424508337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e859208eb0 con 0x55e8591f90f0 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.362+0000 7f2511d9b640 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 0x55e85920da40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2508009cd0 tx=0x7f2508009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf398c2c39601bb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.362+0000 7f24f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f24f9ffb640 1 -- 172.21.15.78:0/2424508337 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508004450 con 0x55e8591f90f0 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f24f9ffb640 1 -- 172.21.15.78:0/2424508337 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f25080124e0 con 0x55e8591f90f0 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f251aea8740 1 -- 172.21.15.78:0/2424508337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2508013390 con 0x55e8591f90f0 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f24f9ffb640 1 -- 172.21.15.78:0/2424508337 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f2508009870 con 0x55e8591f90f0 2024-09-11T01:14:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f251aea8740 10 client.27502 fetch_fsmap learned FSMap version 64 2024-09-11T01:14:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f251aea8740 1 -- 172.21.15.78:0/2424508337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e859282d10 con 0x55e8591f90f0 2024-09-11T01:14:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f24f9ffb640 1 -- 172.21.15.78:0/2424508337 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250803d070 con 0x55e8591f90f0 2024-09-11T01:14:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f24f9ffb640 1 -- 172.21.15.78:0/2424508337 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f2508003bc0 con 0x55e8591f90f0 2024-09-11T01:14:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f251aea8740 10 client.27502 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:14:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f251aea8740 10 client.27502 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:14:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f251aea8740 20 client.27502 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f251aea8740 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55e8592154a0 0x55e859285470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f251aea8740 4 client.27502 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:14:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.363+0000 7f251aea8740 1 -- 172.21.15.78:0/2424508337 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2508013390 con 0x55e8592154a0 2024-09-11T01:14:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.364+0000 7f251259c640 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55e8592154a0 0x55e859285470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.365+0000 7f251259c640 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55e8592154a0 0x55e859285470 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f8d6091df48cce0d server_cookie=5e73dc2b17294e8f in_seq=0 out_seq=0 2024-09-11T01:14:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.365+0000 7f24f9ffb640 10 client.27502 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:14:51.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.371+0000 7f24f9ffb640 1 -- 172.21.15.78:0/2424508337 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2508013390 con 0x55e8592154a0 2024-09-11T01:14:51.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.371+0000 7f24f9ffb640 10 client.27502 handle_command_reply: tid=0 2024-09-11T01:14:51.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.371+0000 7f251aea8740 1 client.27502 shutdown 2024-09-11T01:14:51.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.372+0000 7f251aea8740 1 -- 172.21.15.78:0/2424508337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 msgr2=0x55e85920da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:51.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.372+0000 7f251aea8740 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 0x55e85920da40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2508009cd0 tx=0x7f2508009b60 comp rx=0 tx=0).stop 2024-09-11T01:14:51.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.372+0000 7f251aea8740 1 -- 172.21.15.78:0/2424508337 shutdown_connections 2024-09-11T01:14:51.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.372+0000 7f251aea8740 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55e8592154a0 0x55e859285470 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.372+0000 7f251aea8740 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8592059a0 0x55e859208810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.372+0000 7f251aea8740 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8591f9a10 0x55e85920df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.372+0000 7f24f9ffb640 0 client.27502 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:14:51.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.372+0000 7f251aea8740 1 --2- 172.21.15.78:0/2424508337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8591f90f0 0x55e85920da40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.372+0000 7f251aea8740 1 -- 172.21.15.78:0/2424508337 >> 172.21.15.78:0/2424508337 conn(0x55e8591f6f20 msgr2=0x55e859203d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.372+0000 7f251aea8740 1 -- 172.21.15.78:0/2424508337 shutdown_connections 2024-09-11T01:14:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.372+0000 7f251aea8740 1 -- 172.21.15.78:0/2424508337 wait complete. 2024-09-11T01:14:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.372+0000 7f251aea8740 20 client.27502 trim_cache size 0 max 16384 2024-09-11T01:14:51.375 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:14:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.373+0000 7f2518ad3640 1 -- 172.21.15.78:0/735586823 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f24fc0b54d0 msgr2=0x7f24fc0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.373+0000 7f2518ad3640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f24fc0b54d0 0x7f24fc0b7990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f250c0e3ed0 tx=0x7f250c005f40 comp rx=0 tx=0).stop 2024-09-11T01:14:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.373+0000 7f2518ad3640 1 -- 172.21.15.78:0/735586823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251410f3b0 msgr2=0x7f25141a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.373+0000 7f2518ad3640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251410f3b0 0x7f25141a0df0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f250c00eb10 tx=0x7f250c00c2d0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.373+0000 7f2518ad3640 1 -- 172.21.15.78:0/735586823 shutdown_connections 2024-09-11T01:14:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.373+0000 7f2518ad3640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f24fc0b54d0 0x7f24fc0b7990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.373+0000 7f2518ad3640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f251410f3b0 0x7f25141a0df0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.373+0000 7f2518ad3640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2514075e20 0x7f251419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.373+0000 7f2518ad3640 1 --2- 172.21.15.78:0/735586823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2514075500 0x7f251419b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.373+0000 7f2518ad3640 1 -- 172.21.15.78:0/735586823 >> 172.21.15.78:0/735586823 conn(0x7f25140fe830 msgr2=0x7f25141115d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.374+0000 7f2518ad3640 1 -- 172.21.15.78:0/735586823 shutdown_connections 2024-09-11T01:14:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:51.374+0000 7f2518ad3640 1 -- 172.21.15.78:0/735586823 wait complete. 2024-09-11T01:14:51.783 INFO:tasks.workunit.client.0.smithi078.stdout: 10 123529 41.00 MB/sec execute 57 sec latency 121.495 ms 2024-09-11T01:14:52.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:51 smithi110 ceph-mon[34712]: pgmap v820: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 21 MiB/s wr, 1k op/s 2024-09-11T01:14:52.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:51 smithi112 ceph-mon[35384]: pgmap v820: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 21 MiB/s wr, 1k op/s 2024-09-11T01:14:52.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:51 smithi078 ceph-mon[31315]: pgmap v820: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 21 MiB/s wr, 1k op/s 2024-09-11T01:14:52.325 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:14:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:14:52.783 INFO:tasks.workunit.client.0.smithi078.stdout: 10 124387 41.06 MB/sec execute 58 sec latency 67.534 ms 2024-09-11T01:14:53.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.367+0000 7f500c2e9640 1 -- 172.21.15.78:0/2235630062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5004101c90 msgr2=0x7f5004102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:53.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.367+0000 7f500c2e9640 1 --2- 172.21.15.78:0/2235630062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5004101c90 0x7f5004102070 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f5000009900 tx=0x7f500002a410 comp rx=0 tx=0).stop 2024-09-11T01:14:53.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.367+0000 7f500c2e9640 1 -- 172.21.15.78:0/2235630062 shutdown_connections 2024-09-11T01:14:53.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.367+0000 7f500c2e9640 1 --2- 172.21.15.78:0/2235630062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5004108020 0x7f500410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.367+0000 7f500c2e9640 1 --2- 172.21.15.78:0/2235630062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50041076a0 0x7f5004107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.367+0000 7f500c2e9640 1 --2- 172.21.15.78:0/2235630062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5004101c90 0x7f5004102070 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.367+0000 7f500c2e9640 1 -- 172.21.15.78:0/2235630062 >> 172.21.15.78:0/2235630062 conn(0x7f50040fd700 msgr2=0x7f50040ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:53.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.367+0000 7f500c2e9640 1 -- 172.21.15.78:0/2235630062 shutdown_connections 2024-09-11T01:14:53.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.367+0000 7f500c2e9640 1 -- 172.21.15.78:0/2235630062 wait complete. 2024-09-11T01:14:53.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.368+0000 7f500c2e9640 1 Processor -- start 2024-09-11T01:14:53.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.368+0000 7f500c2e9640 1 -- start start 2024-09-11T01:14:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f500c2e9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5004101c90 0x7f5004197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f500c2e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50041076a0 0x7f50041983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f500c2e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5004108020 0x7f5004198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f500c2e9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5004111ca0 con 0x7f5004108020 2024-09-11T01:14:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f500c2e9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5004111b20 con 0x7f5004101c90 2024-09-11T01:14:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f500c2e9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5004111e20 con 0x7f50041076a0 2024-09-11T01:14:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f5009884640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50041076a0 0x7f50041983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f500a886640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5004108020 0x7f5004198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f5009884640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50041076a0 0x7f50041983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:48902/0 (socket says 172.21.15.78:48902) 2024-09-11T01:14:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f5009884640 1 -- 172.21.15.78:0/1707682150 learned_addr learned my addr 172.21.15.78:0/1707682150 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f5009884640 1 -- 172.21.15.78:0/1707682150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5004101c90 msgr2=0x7f5004197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f500a085640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5004101c90 0x7f5004197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f5009884640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5004101c90 0x7f5004197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f5009884640 1 -- 172.21.15.78:0/1707682150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5004108020 msgr2=0x7f5004198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.369+0000 7f5009884640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5004108020 0x7f5004198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.370+0000 7f5009884640 1 -- 172.21.15.78:0/1707682150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500419d260 con 0x7f50041076a0 2024-09-11T01:14:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.370+0000 7f500a886640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5004108020 0x7f5004198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:14:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.370+0000 7f500a085640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5004101c90 0x7f5004197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:14:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.370+0000 7f5009884640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50041076a0 0x7f50041983d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400e9f0 tx=0x7f4ff400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.370+0000 7f4ffb7fe640 1 -- 172.21.15.78:0/1707682150 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff400cd80 con 0x7f50041076a0 2024-09-11T01:14:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.370+0000 7f4ffb7fe640 1 -- 172.21.15.78:0/1707682150 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff40108e0 con 0x7f50041076a0 2024-09-11T01:14:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.370+0000 7f4ffb7fe640 1 -- 172.21.15.78:0/1707682150 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4018980 con 0x7f50041076a0 2024-09-11T01:14:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.370+0000 7f500c2e9640 1 -- 172.21.15.78:0/1707682150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500419d550 con 0x7f50041076a0 2024-09-11T01:14:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.370+0000 7f500c2e9640 1 -- 172.21.15.78:0/1707682150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500419dae0 con 0x7f50041076a0 2024-09-11T01:14:53.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.374+0000 7f4ffb7fe640 1 -- 172.21.15.78:0/1707682150 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4ff4010430 con 0x7f50041076a0 2024-09-11T01:14:53.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.374+0000 7f500e6be740 1 Processor -- start 2024-09-11T01:14:53.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.375+0000 7f500e6be740 1 -- start start 2024-09-11T01:14:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.375+0000 7f500e6be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9f0f0 0x55fa9cc9f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.375+0000 7f500e6be740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccab460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.375+0000 7f500e6be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9ccab9a0 0x55fa9ccadd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.375+0000 7f500e6be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa9cc15ea0 con 0x55fa9ccab9a0 2024-09-11T01:14:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.375+0000 7f500e6be740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fa9cc052d0 con 0x55fa9cc9fa10 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.375+0000 7f500e6be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fa9cc090e0 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.375+0000 7f500a085640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.375+0000 7f500a085640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccab460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:58718/0 (socket says 172.21.15.78:58718) 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.375+0000 7f500a085640 1 -- 172.21.15.78:0/1982107551 learned_addr learned my addr 172.21.15.78:0/1982107551 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.375+0000 7f5009884640 1 --2- 172.21.15.78:0/1982107551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9ccab9a0 0x55fa9ccadd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.375+0000 7f500a886640 1 --2- 172.21.15.78:0/1982107551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9f0f0 0x55fa9cc9f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.376+0000 7f4ffb7fe640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4ff00b54d0 0x7f4ff00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.376+0000 7f500a085640 1 -- 172.21.15.78:0/1982107551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9f0f0 msgr2=0x55fa9cc9f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.376+0000 7f500a085640 1 --2- 172.21.15.78:0/1982107551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9f0f0 0x55fa9cc9f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.376+0000 7f500a085640 1 -- 172.21.15.78:0/1982107551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9ccab9a0 msgr2=0x55fa9ccadd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.376+0000 7f500a085640 1 --2- 172.21.15.78:0/1982107551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9ccab9a0 0x55fa9ccadd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.376+0000 7f500a085640 1 -- 172.21.15.78:0/1982107551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa9c963650 con 0x55fa9cc9fa10 2024-09-11T01:14:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.376+0000 7f4ffb7fe640 1 -- 172.21.15.78:0/1707682150 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f4ff40e6310 con 0x7f50041076a0 2024-09-11T01:14:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.376+0000 7f500a085640 1 --2- 172.21.15.78:0/1982107551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccab460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5000000c00 tx=0x7f5000032c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c4eb7b936c6cfe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.377+0000 7f500a886640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4ff00b54d0 0x7f4ff00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.377+0000 7f4ff8de6640 1 -- 172.21.15.78:0/1982107551 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500003f070 con 0x55fa9cc9fa10 2024-09-11T01:14:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.377+0000 7f4ff8de6640 1 -- 172.21.15.78:0/1982107551 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5000032e50 con 0x55fa9cc9fa10 2024-09-11T01:14:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.377+0000 7f4ff8de6640 1 -- 172.21.15.78:0/1982107551 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000035070 con 0x55fa9cc9fa10 2024-09-11T01:14:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.378+0000 7f500a886640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4ff00b54d0 0x7f4ff00b7990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009fd0 tx=0x7f4ffc004a40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.378+0000 7f500e6be740 1 -- 172.21.15.78:0/1982107551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9cc9fa10 msgr2=0x55fa9ccab460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.378+0000 7f500e6be740 1 --2- 172.21.15.78:0/1982107551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccab460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5000000c00 tx=0x7f5000032c30 comp rx=0 tx=0).stop 2024-09-11T01:14:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.379+0000 7f500e6be740 1 -- 172.21.15.78:0/1982107551 shutdown_connections 2024-09-11T01:14:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.379+0000 7f500e6be740 1 --2- 172.21.15.78:0/1982107551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9ccab9a0 0x55fa9ccadd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.379+0000 7f500e6be740 1 --2- 172.21.15.78:0/1982107551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccab460 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.379+0000 7f500e6be740 1 --2- 172.21.15.78:0/1982107551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9f0f0 0x55fa9cc9f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.379+0000 7f500e6be740 1 -- 172.21.15.78:0/1982107551 >> 172.21.15.78:0/1982107551 conn(0x55fa9cc9cf20 msgr2=0x55fa9cc9e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.379+0000 7f500e6be740 1 -- 172.21.15.78:0/1982107551 shutdown_connections 2024-09-11T01:14:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.379+0000 7f500e6be740 1 -- 172.21.15.78:0/1982107551 wait complete. 2024-09-11T01:14:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.379+0000 7f500e6be740 1 -- start start 2024-09-11T01:14:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.379+0000 7f500e6be740 1 Processor -- start 2024-09-11T01:14:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f500e6be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccab280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f500e6be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9fa10 0x55fa9cca86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f500e6be740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9ccab9a0 0x55fa9cca8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f500e6be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa9ca66700 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f500e6be740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fa9cba0160 con 0x55fa9ccab9a0 2024-09-11T01:14:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f500e6be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fa9ccae5c0 con 0x55fa9cc9fa10 2024-09-11T01:14:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f500e6be740 10 client.0 _ll_register_callbacks cb 0x55fa9cbb99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f500a085640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f5009884640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9fa10 0x55fa9cca86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f5009884640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9fa10 0x55fa9cca86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:48928/0 (socket says 172.21.15.78:48928) 2024-09-11T01:14:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f500a085640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccab280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55762/0 (socket says 172.21.15.78:55762) 2024-09-11T01:14:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f5009884640 1 -- 172.21.15.78:0/2453521625 learned_addr learned my addr 172.21.15.78:0/2453521625 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f500a886640 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9ccab9a0 0x55fa9cca8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f5009884640 1 -- 172.21.15.78:0/2453521625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9ccab9a0 msgr2=0x55fa9cca8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f5009884640 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9ccab9a0 0x55fa9cca8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f5009884640 1 -- 172.21.15.78:0/2453521625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 msgr2=0x55fa9ccab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f5009884640 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccab280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f5009884640 1 -- 172.21.15.78:0/2453521625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9cbcdc90 con 0x55fa9cc9fa10 2024-09-11T01:14:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.380+0000 7f500a085640 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:14:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.381+0000 7f5009884640 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9fa10 0x55fa9cca86d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400e9c0 tx=0x7f4ff4018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45c08276ac69a7e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.381+0000 7f4fedffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:14:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.381+0000 7f4fedffb640 1 -- 172.21.15.78:0/2453521625 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40ac540 con 0x55fa9cc9fa10 2024-09-11T01:14:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.381+0000 7f4fedffb640 1 -- 172.21.15.78:0/2453521625 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f4ff40abbd0 con 0x55fa9cc9fa10 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.381+0000 7f500e6be740 1 -- 172.21.15.78:0/2453521625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5000032e50 con 0x55fa9cc9fa10 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.381+0000 7f4fedffb640 1 -- 172.21.15.78:0/2453521625 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f4ff40e3ed0 con 0x55fa9cc9fa10 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.381+0000 7f500e6be740 10 client.27514 fetch_fsmap learned FSMap version 64 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.381+0000 7f500e6be740 1 -- 172.21.15.78:0/2453521625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55fa9ccb9ee0 con 0x55fa9cc9fa10 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.381+0000 7f4fedffb640 1 -- 172.21.15.78:0/2453521625 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5000032e50 con 0x55fa9cc9fa10 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.382+0000 7f4fedffb640 1 -- 172.21.15.78:0/2453521625 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f4ff40a80c0 con 0x55fa9cc9fa10 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.382+0000 7f500e6be740 10 client.27514 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.382+0000 7f500e6be740 10 client.27514 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.382+0000 7f500e6be740 20 client.27514 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.382+0000 7f500e6be740 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55fa9ccbb3a0 0x55fa9ccbd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.382+0000 7f500e6be740 4 client.27514 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.382+0000 7f500e6be740 1 -- 172.21.15.78:0/2453521625 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa9cc17df0 con 0x55fa9ccbb3a0 2024-09-11T01:14:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.382+0000 7f500a085640 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55fa9ccbb3a0 0x55fa9ccbd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.382+0000 7f500a085640 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55fa9ccbb3a0 0x55fa9ccbd7f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=377d788f3ef70433 server_cookie=ad7292fdc458f4e1 in_seq=0 out_seq=0 2024-09-11T01:14:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.382+0000 7f4fedffb640 10 client.27514 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:14:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f4fedffb640 1 -- 172.21.15.78:0/2453521625 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fa9cc17df0 con 0x55fa9ccbb3a0 2024-09-11T01:14:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f4fedffb640 10 client.27514 handle_command_reply: tid=0 2024-09-11T01:14:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f500e6be740 1 client.27514 shutdown 2024-09-11T01:14:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f500e6be740 1 -- 172.21.15.78:0/2453521625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9fa10 msgr2=0x55fa9cca86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f500e6be740 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9fa10 0x55fa9cca86d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400e9c0 tx=0x7f4ff4018420 comp rx=0 tx=0).stop 2024-09-11T01:14:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f500e6be740 1 -- 172.21.15.78:0/2453521625 shutdown_connections 2024-09-11T01:14:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f500e6be740 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55fa9ccbb3a0 0x55fa9ccbd7f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f500e6be740 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9ccab9a0 0x55fa9cca8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f500e6be740 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9fa10 0x55fa9cca86d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f500e6be740 1 --2- 172.21.15.78:0/2453521625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f500e6be740 1 -- 172.21.15.78:0/2453521625 >> 172.21.15.78:0/2453521625 conn(0x55fa9cc9cf20 msgr2=0x55fa9cca2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f4fedffb640 0 client.27514 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:14:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f500e6be740 1 -- 172.21.15.78:0/2453521625 shutdown_connections 2024-09-11T01:14:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.384+0000 7f500e6be740 1 -- 172.21.15.78:0/2453521625 wait complete. 2024-09-11T01:14:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.385+0000 7f500e6be740 20 client.27514 trim_cache size 0 max 16384 2024-09-11T01:14:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.406+0000 7f500e6be740 1 Processor -- start 2024-09-11T01:14:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.406+0000 7f500e6be740 1 -- start start 2024-09-11T01:14:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.406+0000 7f500e6be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccb77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.406+0000 7f500e6be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccaa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.406+0000 7f500e6be740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9ccab9a0 0x55fa9ccacd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.406+0000 7f500e6be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50041b4c00 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.406+0000 7f500e6be740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50041b68c0 con 0x55fa9ccab9a0 2024-09-11T01:14:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.406+0000 7f500e6be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50041b52d0 con 0x55fa9cc9fa10 2024-09-11T01:14:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.406+0000 7f500a085640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccb77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.406+0000 7f500a085640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccb77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55770/0 (socket says 172.21.15.78:55770) 2024-09-11T01:14:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.406+0000 7f500a085640 1 -- 172.21.15.78:0/842097028 learned_addr learned my addr 172.21.15.78:0/842097028 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.406+0000 7f5009884640 1 --2- 172.21.15.78:0/842097028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccaa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.407+0000 7f500a886640 1 --2- 172.21.15.78:0/842097028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9ccab9a0 0x55fa9ccacd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.407+0000 7f500a085640 1 -- 172.21.15.78:0/842097028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9fa10 msgr2=0x55fa9ccaa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.407+0000 7f500a085640 1 --2- 172.21.15.78:0/842097028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccaa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.407+0000 7f500a085640 1 -- 172.21.15.78:0/842097028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9ccab9a0 msgr2=0x55fa9ccacd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.407+0000 7f500a085640 1 --2- 172.21.15.78:0/842097028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9ccab9a0 0x55fa9ccacd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.407+0000 7f500a085640 1 -- 172.21.15.78:0/842097028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa9cca6bd0 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.407+0000 7f500a085640 1 --2- 172.21.15.78:0/842097028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccb77f0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5000000c00 tx=0x7f50000050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03aa8f6d56eb2e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.407+0000 7f4feffff640 1 -- 172.21.15.78:0/842097028 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000050070 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.408+0000 7f4feffff640 1 -- 172.21.15.78:0/842097028 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5000032e50 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.408+0000 7f4feffff640 1 -- 172.21.15.78:0/842097028 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500003f040 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.408+0000 7f500e6be740 1 -- 172.21.15.78:0/842097028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 msgr2=0x55fa9ccb77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.408+0000 7f500e6be740 1 --2- 172.21.15.78:0/842097028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccb77f0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5000000c00 tx=0x7f50000050e0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.408+0000 7f500e6be740 1 -- 172.21.15.78:0/842097028 shutdown_connections 2024-09-11T01:14:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.409+0000 7f500e6be740 1 --2- 172.21.15.78:0/842097028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9ccab9a0 0x55fa9ccacd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.409+0000 7f500e6be740 1 --2- 172.21.15.78:0/842097028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccaa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.409+0000 7f500e6be740 1 --2- 172.21.15.78:0/842097028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccb77f0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.409+0000 7f500e6be740 1 -- 172.21.15.78:0/842097028 >> 172.21.15.78:0/842097028 conn(0x55fa9cc9cf20 msgr2=0x55fa9cc9e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:53.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.409+0000 7f500e6be740 1 -- 172.21.15.78:0/842097028 shutdown_connections 2024-09-11T01:14:53.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.409+0000 7f500e6be740 1 -- 172.21.15.78:0/842097028 wait complete. 2024-09-11T01:14:53.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.409+0000 7f500e6be740 1 -- start start 2024-09-11T01:14:53.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.409+0000 7f500e6be740 1 Processor -- start 2024-09-11T01:14:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.410+0000 7f500e6be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccb3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.410+0000 7f500a085640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccb3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.410+0000 7f500a085640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccb3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55772/0 (socket says 172.21.15.78:55772) 2024-09-11T01:14:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.410+0000 7f500e6be740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccb4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.410+0000 7f500a085640 1 -- 172.21.15.78:0/3399349388 learned_addr learned my addr 172.21.15.78:0/3399349388 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.410+0000 7f500e6be740 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9ccab9a0 0x55fa9ccaea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.410+0000 7f5009884640 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccb4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.410+0000 7f500e6be740 1 -- 172.21.15.78:0/3399349388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50041b6130 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.410+0000 7f500a886640 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9ccab9a0 0x55fa9ccaea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.410+0000 7f500e6be740 1 -- 172.21.15.78:0/3399349388 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50041b59a0 con 0x55fa9cc9fa10 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.410+0000 7f500e6be740 1 -- 172.21.15.78:0/3399349388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50041b2c00 con 0x55fa9ccab9a0 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.411+0000 7f500e6be740 10 client.0 _ll_register_callbacks cb 0x7f4ff40e9ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.411+0000 7f500a085640 1 -- 172.21.15.78:0/3399349388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9ccab9a0 msgr2=0x55fa9ccaea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.411+0000 7f500a085640 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9ccab9a0 0x55fa9ccaea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.411+0000 7f500a085640 1 -- 172.21.15.78:0/3399349388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9cc9fa10 msgr2=0x55fa9ccb4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.411+0000 7f500a085640 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccb4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.411+0000 7f500a085640 1 -- 172.21.15.78:0/3399349388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9ccaf290 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.411+0000 7f500a085640 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccb3d10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5000033000 tx=0x7f5000032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bb73d91277e2bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.411+0000 7f4fedffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.411+0000 7f4fedffb640 1 -- 172.21.15.78:0/3399349388 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500003f070 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.412+0000 7f4fedffb640 1 -- 172.21.15.78:0/3399349388 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f50000316b0 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.412+0000 7f500e6be740 1 -- 172.21.15.78:0/3399349388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5000032e50 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.412+0000 7f4fedffb640 1 -- 172.21.15.78:0/3399349388 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f5000050070 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.412+0000 7f500e6be740 10 client.18405 fetch_fsmap learned FSMap version 64 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.412+0000 7f500e6be740 1 -- 172.21.15.78:0/3399349388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55fa9cd28ff0 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.412+0000 7f4fedffb640 1 -- 172.21.15.78:0/3399349388 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5000032e50 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.412+0000 7f4fedffb640 1 -- 172.21.15.78:0/3399349388 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f500003b600 con 0x55fa9cc9f0f0 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.412+0000 7f500e6be740 10 client.18405 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.412+0000 7f500e6be740 10 client.18405 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.412+0000 7f500e6be740 20 client.18405 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.412+0000 7f500e6be740 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55fa9cd29290 0x55fa9cd2b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.413+0000 7f500e6be740 4 client.18405 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.413+0000 7f500e6be740 1 -- 172.21.15.78:0/3399349388 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa9ccbdb80 con 0x55fa9cd29290 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.413+0000 7f5009884640 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55fa9cd29290 0x55fa9cd2b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.413+0000 7f5009884640 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55fa9cd29290 0x55fa9cd2b680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c19a1e6bd446d70 server_cookie=6a04f08aac861c87 in_seq=0 out_seq=0 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.413+0000 7f4fedffb640 10 client.18405 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f4fedffb640 1 -- 172.21.15.78:0/3399349388 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa9ccbdb80 con 0x55fa9cd29290 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f4fedffb640 10 client.18405 handle_command_reply: tid=0 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f500e6be740 1 client.18405 shutdown 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f500e6be740 1 -- 172.21.15.78:0/3399349388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 msgr2=0x55fa9ccb3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f500e6be740 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccb3d10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5000033000 tx=0x7f5000032bb0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f500e6be740 1 -- 172.21.15.78:0/3399349388 shutdown_connections 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f500e6be740 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa9ccab9a0 0x55fa9ccaea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f500e6be740 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55fa9cd29290 0x55fa9cd2b680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f500e6be740 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa9cc9fa10 0x55fa9ccb4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f4fedffb640 0 client.18405 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:14:53.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f500e6be740 1 --2- 172.21.15.78:0/3399349388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9cc9f0f0 0x55fa9ccb3d10 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f500e6be740 1 -- 172.21.15.78:0/3399349388 >> 172.21.15.78:0/3399349388 conn(0x55fa9cc9cf20 msgr2=0x55fa9ccaa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:53.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f500e6be740 1 -- 172.21.15.78:0/3399349388 shutdown_connections 2024-09-11T01:14:53.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.414+0000 7f500e6be740 1 -- 172.21.15.78:0/3399349388 wait complete. 2024-09-11T01:14:53.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.415+0000 7f500e6be740 20 client.18405 trim_cache size 0 max 16384 2024-09-11T01:14:53.421 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:14:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.416+0000 7f500c2e9640 1 -- 172.21.15.78:0/1707682150 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4ff00b54d0 msgr2=0x7f4ff00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.416+0000 7f500c2e9640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4ff00b54d0 0x7f4ff00b7990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009fd0 tx=0x7f4ffc004a40 comp rx=0 tx=0).stop 2024-09-11T01:14:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.416+0000 7f500c2e9640 1 -- 172.21.15.78:0/1707682150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50041076a0 msgr2=0x7f50041983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.416+0000 7f500c2e9640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50041076a0 0x7f50041983d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400e9f0 tx=0x7f4ff400eeb0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.416+0000 7f500c2e9640 1 -- 172.21.15.78:0/1707682150 shutdown_connections 2024-09-11T01:14:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.416+0000 7f500c2e9640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4ff00b54d0 0x7f4ff00b7990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.416+0000 7f500c2e9640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5004108020 0x7f5004198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.416+0000 7f500c2e9640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50041076a0 0x7f50041983d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.416+0000 7f500c2e9640 1 --2- 172.21.15.78:0/1707682150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5004101c90 0x7f5004197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.416+0000 7f500c2e9640 1 -- 172.21.15.78:0/1707682150 >> 172.21.15.78:0/1707682150 conn(0x7f50040fd700 msgr2=0x7f50040ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.416+0000 7f500c2e9640 1 -- 172.21.15.78:0/1707682150 shutdown_connections 2024-09-11T01:14:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:53.416+0000 7f500c2e9640 1 -- 172.21.15.78:0/1707682150 wait complete. 2024-09-11T01:14:53.784 INFO:tasks.workunit.client.0.smithi078.stdout: 10 125278 41.19 MB/sec execute 59 sec latency 133.338 ms 2024-09-11T01:14:54.179 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:53 smithi078 ceph-mon[31315]: pgmap v821: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 20 MiB/s wr, 947 op/s 2024-09-11T01:14:54.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:53 smithi110 ceph-mon[34712]: pgmap v821: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 20 MiB/s wr, 947 op/s 2024-09-11T01:14:54.219 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:14:54.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:53 smithi112 ceph-mon[35384]: pgmap v821: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 20 MiB/s wr, 947 op/s 2024-09-11T01:14:54.573 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:14:54.784 INFO:tasks.workunit.client.0.smithi078.stdout: 10 126173 41.36 MB/sec execute 60 sec latency 49.644 ms 2024-09-11T01:14:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.144+0000 7f936710b640 1 -- 172.21.15.78:0/1835234450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93601076a0 msgr2=0x7f9360107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.144+0000 7f936710b640 1 --2- 172.21.15.78:0/1835234450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93601076a0 0x7f9360107ae0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f934c009860 tx=0x7f934c02a340 comp rx=0 tx=0).stop 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.144+0000 7f936710b640 1 -- 172.21.15.78:0/1835234450 shutdown_connections 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.144+0000 7f936710b640 1 --2- 172.21.15.78:0/1835234450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9360108020 0x7f936010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.144+0000 7f936710b640 1 --2- 172.21.15.78:0/1835234450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93601076a0 0x7f9360107ae0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.144+0000 7f936710b640 1 --2- 172.21.15.78:0/1835234450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9360101c90 0x7f9360102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.144+0000 7f936710b640 1 -- 172.21.15.78:0/1835234450 >> 172.21.15.78:0/1835234450 conn(0x7f93600fd700 msgr2=0x7f93600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.144+0000 7f936710b640 1 -- 172.21.15.78:0/1835234450 shutdown_connections 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.144+0000 7f936710b640 1 -- 172.21.15.78:0/1835234450 wait complete. 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.145+0000 7f936710b640 1 Processor -- start 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.145+0000 7f936710b640 1 -- start start 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f936710b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9360101c90 0x7f9360199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f936710b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93601076a0 0x7f9360199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f936710b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9360108020 0x7f936019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f936710b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9360111ce0 con 0x7f93601076a0 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f936710b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9360111b60 con 0x7f9360108020 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f936710b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9360111e60 con 0x7f9360101c90 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f935ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93601076a0 0x7f9360199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f935ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93601076a0 0x7f9360199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55778/0 (socket says 172.21.15.78:55778) 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f935ffff640 1 -- 172.21.15.78:0/729331364 learned_addr learned my addr 172.21.15.78:0/729331364 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f9364ea7640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9360101c90 0x7f9360199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f93656a8640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9360108020 0x7f936019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f935ffff640 1 -- 172.21.15.78:0/729331364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9360101c90 msgr2=0x7f9360199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f935ffff640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9360101c90 0x7f9360199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f935ffff640 1 -- 172.21.15.78:0/729331364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9360108020 msgr2=0x7f936019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f935ffff640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9360108020 0x7f936019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f935ffff640 1 -- 172.21.15.78:0/729331364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936019f3d0 con 0x7f93601076a0 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.146+0000 7f935ffff640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93601076a0 0x7f9360199860 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f934c035040 tx=0x7f934c032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.147+0000 7f935dffb640 1 -- 172.21.15.78:0/729331364 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c037040 con 0x7f93601076a0 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.147+0000 7f936710b640 1 -- 172.21.15.78:0/729331364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936019f660 con 0x7f93601076a0 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.147+0000 7f935dffb640 1 -- 172.21.15.78:0/729331364 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934c041070 con 0x7f93601076a0 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.147+0000 7f935dffb640 1 -- 172.21.15.78:0/729331364 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c0035f0 con 0x7f93601076a0 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.147+0000 7f936710b640 1 -- 172.21.15.78:0/729331364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93601087f0 con 0x7f93601076a0 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.150+0000 7f93694e0740 1 Processor -- start 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.150+0000 7f935dffb640 1 -- 172.21.15.78:0/729331364 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f934c004050 con 0x7f93601076a0 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.150+0000 7f93694e0740 1 -- start start 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.150+0000 7f93694e0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 0x560eb84358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.150+0000 7f93694e0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8435df0 0x560eb8441840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.150+0000 7f93694e0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8441d80 0x560eb8444170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.150+0000 7f93694e0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560eb83abea0 con 0x560eb8435df0 2024-09-11T01:14:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.150+0000 7f93694e0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560eb839b2d0 con 0x560eb84354d0 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.150+0000 7f93694e0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560eb839f0e0 con 0x560eb8441d80 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.150+0000 7f93656a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 0x560eb84358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.150+0000 7f93656a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 0x560eb84358b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:58772/0 (socket says 172.21.15.78:58772) 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.150+0000 7f93656a8640 1 -- 172.21.15.78:0/34193115 learned_addr learned my addr 172.21.15.78:0/34193115 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.151+0000 7f9364ea7640 1 --2- 172.21.15.78:0/34193115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8435df0 0x560eb8441840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.151+0000 7f935ffff640 1 --2- 172.21.15.78:0/34193115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8441d80 0x560eb8444170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.151+0000 7f9364ea7640 1 -- 172.21.15.78:0/34193115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8441d80 msgr2=0x560eb8444170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.151+0000 7f9364ea7640 1 --2- 172.21.15.78:0/34193115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8441d80 0x560eb8444170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.151+0000 7f9364ea7640 1 -- 172.21.15.78:0/34193115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 msgr2=0x560eb84358b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.151+0000 7f9364ea7640 1 --2- 172.21.15.78:0/34193115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 0x560eb84358b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.151+0000 7f9364ea7640 1 -- 172.21.15.78:0/34193115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb80f9650 con 0x560eb8435df0 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.151+0000 7f93656a8640 1 --2- 172.21.15.78:0/34193115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 0x560eb84358b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:14:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.151+0000 7f935dffb640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93500b54d0 0x7f93500b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.151+0000 7f9364ea7640 1 --2- 172.21.15.78:0/34193115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8435df0 0x560eb8441840 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9354009c10 tx=0x7f9354009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ea4b2aee3167ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.151+0000 7f93656a8640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93500b54d0 0x7f93500b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.151+0000 7f935dffb640 1 -- 172.21.15.78:0/729331364 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f934c105af0 con 0x7f93601076a0 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.152+0000 7f934affd640 1 -- 172.21.15.78:0/34193115 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935401d070 con 0x560eb8435df0 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.152+0000 7f934affd640 1 -- 172.21.15.78:0/34193115 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93540035f0 con 0x560eb8435df0 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.152+0000 7f934affd640 1 -- 172.21.15.78:0/34193115 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354018680 con 0x560eb8435df0 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.153+0000 7f93694e0740 1 -- 172.21.15.78:0/34193115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8435df0 msgr2=0x560eb8441840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.153+0000 7f93656a8640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93500b54d0 0x7f93500b7990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9358009f10 tx=0x7f9358014f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.153+0000 7f93694e0740 1 --2- 172.21.15.78:0/34193115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8435df0 0x560eb8441840 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9354009c10 tx=0x7f9354009260 comp rx=0 tx=0).stop 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.153+0000 7f93694e0740 1 -- 172.21.15.78:0/34193115 shutdown_connections 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.153+0000 7f93694e0740 1 --2- 172.21.15.78:0/34193115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8441d80 0x560eb8444170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.153+0000 7f93694e0740 1 --2- 172.21.15.78:0/34193115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8435df0 0x560eb8441840 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.153+0000 7f93694e0740 1 --2- 172.21.15.78:0/34193115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 0x560eb84358b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.153+0000 7f93694e0740 1 -- 172.21.15.78:0/34193115 >> 172.21.15.78:0/34193115 conn(0x560eb8433300 msgr2=0x560eb84344a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.153+0000 7f93694e0740 1 -- 172.21.15.78:0/34193115 shutdown_connections 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.153+0000 7f93694e0740 1 -- 172.21.15.78:0/34193115 wait complete. 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.153+0000 7f93694e0740 1 -- start start 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f93694e0740 1 Processor -- start 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f93694e0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb84354d0 0x560eb8441690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f93694e0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 0x560eb843eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f93694e0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8441d80 0x560eb843f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f93694e0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560eb81fc700 con 0x560eb8441d80 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f93694e0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560eb8336160 con 0x560eb8435df0 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f93694e0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560eb8444a00 con 0x560eb84354d0 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f93656a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8441d80 0x560eb843f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f93694e0740 10 client.0 _ll_register_callbacks cb 0x560eb834f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:55.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f935ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 0x560eb843eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f935ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 0x560eb843eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:58776/0 (socket says 172.21.15.78:58776) 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f935ffff640 1 -- 172.21.15.78:0/1833001073 learned_addr learned my addr 172.21.15.78:0/1833001073 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.154+0000 7f9364ea7640 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb84354d0 0x560eb8441690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.155+0000 7f93656a8640 1 -- 172.21.15.78:0/1833001073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb84354d0 msgr2=0x560eb8441690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.155+0000 7f93656a8640 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb84354d0 0x560eb8441690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.155+0000 7f93656a8640 1 -- 172.21.15.78:0/1833001073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 msgr2=0x560eb843eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.155+0000 7f93656a8640 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 0x560eb843eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.155+0000 7f93656a8640 1 -- 172.21.15.78:0/1833001073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb8363c90 con 0x560eb8441d80 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.155+0000 7f935ffff640 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 0x560eb843eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.155+0000 7f93656a8640 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8441d80 0x560eb843f110 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f935800ba60 tx=0x7f9358016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f9b7586d01a24d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.155+0000 7f933ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.155+0000 7f933ffff640 1 -- 172.21.15.78:0/1833001073 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358020cc0 con 0x560eb8441d80 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.155+0000 7f933ffff640 1 -- 172.21.15.78:0/1833001073 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f9358009040 con 0x560eb8441d80 2024-09-11T01:14:55.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.155+0000 7f93694e0740 1 -- 172.21.15.78:0/1833001073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93540035f0 con 0x560eb8441d80 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.156+0000 7f933ffff640 1 -- 172.21.15.78:0/1833001073 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f9358020e60 con 0x560eb8441d80 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.156+0000 7f93694e0740 10 client.18417 fetch_fsmap learned FSMap version 64 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.156+0000 7f93694e0740 1 -- 172.21.15.78:0/1833001073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x560eb84503c0 con 0x560eb8441d80 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.156+0000 7f933ffff640 1 -- 172.21.15.78:0/1833001073 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9358003e20 con 0x560eb8441d80 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.156+0000 7f933ffff640 1 -- 172.21.15.78:0/1833001073 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f9358004020 con 0x560eb8441d80 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.156+0000 7f93694e0740 10 client.18417 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.156+0000 7f93694e0740 10 client.18417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.156+0000 7f93694e0740 20 client.18417 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.156+0000 7f93694e0740 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x560eb84519a0 0x560eb8453e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.156+0000 7f93694e0740 4 client.18417 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.156+0000 7f93694e0740 1 -- 172.21.15.78:0/1833001073 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f93540035f0 con 0x560eb84519a0 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.156+0000 7f9364ea7640 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x560eb84519a0 0x560eb8453e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.157+0000 7f9364ea7640 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x560eb84519a0 0x560eb8453e20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ced396000eeec0a4 server_cookie=e1cc97c1f6cf9c2e in_seq=0 out_seq=0 2024-09-11T01:14:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.157+0000 7f933ffff640 10 client.18417 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.158+0000 7f933ffff640 1 -- 172.21.15.78:0/1833001073 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f93540035f0 con 0x560eb84519a0 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.158+0000 7f933ffff640 10 client.18417 handle_command_reply: tid=0 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.158+0000 7f93694e0740 1 client.18417 shutdown 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.159+0000 7f93694e0740 1 -- 172.21.15.78:0/1833001073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8441d80 msgr2=0x560eb843f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.159+0000 7f93694e0740 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8441d80 0x560eb843f110 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f935800ba60 tx=0x7f9358016040 comp rx=0 tx=0).stop 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.159+0000 7f93694e0740 1 -- 172.21.15.78:0/1833001073 shutdown_connections 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.159+0000 7f93694e0740 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x560eb84519a0 0x560eb8453e20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.159+0000 7f93694e0740 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8441d80 0x560eb843f110 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.159+0000 7f93694e0740 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 0x560eb843eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.159+0000 7f93694e0740 1 --2- 172.21.15.78:0/1833001073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb84354d0 0x560eb8441690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.159+0000 7f93694e0740 1 -- 172.21.15.78:0/1833001073 >> 172.21.15.78:0/1833001073 conn(0x560eb8433300 msgr2=0x560eb8438ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.159+0000 7f933ffff640 0 client.18417 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:14:55.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.159+0000 7f93694e0740 1 -- 172.21.15.78:0/1833001073 shutdown_connections 2024-09-11T01:14:55.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.159+0000 7f93694e0740 1 -- 172.21.15.78:0/1833001073 wait complete. 2024-09-11T01:14:55.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.159+0000 7f93694e0740 20 client.18417 trim_cache size 0 max 16384 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.178+0000 7f93694e0740 1 Processor -- start 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.178+0000 7f93694e0740 1 -- start start 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f93694e0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb84354d0 0x560eb844dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f93694e0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 0x560eb8440820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f93694e0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8441d80 0x560eb8443170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f93694e0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93601ac440 con 0x560eb84354d0 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f93694e0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93601ae100 con 0x560eb8435df0 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f93694e0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93601acb10 con 0x560eb8441d80 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f9364ea7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb84354d0 0x560eb844dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f9364ea7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb84354d0 0x560eb844dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55796/0 (socket says 172.21.15.78:55796) 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f9364ea7640 1 -- 172.21.15.78:0/1091092536 learned_addr learned my addr 172.21.15.78:0/1091092536 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f935ffff640 1 --2- 172.21.15.78:0/1091092536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 0x560eb8440820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f93656a8640 1 --2- 172.21.15.78:0/1091092536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8441d80 0x560eb8443170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f935ffff640 1 -- 172.21.15.78:0/1091092536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8441d80 msgr2=0x560eb8443170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f935ffff640 1 --2- 172.21.15.78:0/1091092536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8441d80 0x560eb8443170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f935ffff640 1 -- 172.21.15.78:0/1091092536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb84354d0 msgr2=0x560eb844dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f935ffff640 1 --2- 172.21.15.78:0/1091092536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb84354d0 0x560eb844dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f935ffff640 1 -- 172.21.15.78:0/1091092536 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb842ebb0 con 0x560eb8435df0 2024-09-11T01:14:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.179+0000 7f935ffff640 1 --2- 172.21.15.78:0/1091092536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 0x560eb8440820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f934c02a930 tx=0x7f934c109640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ae82c4395c4fb81 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.180+0000 7f934a7fc640 1 -- 172.21.15.78:0/1091092536 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c0c8600 con 0x560eb8435df0 2024-09-11T01:14:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.180+0000 7f934a7fc640 1 -- 172.21.15.78:0/1091092536 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934c0328f0 con 0x560eb8435df0 2024-09-11T01:14:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.180+0000 7f934a7fc640 1 -- 172.21.15.78:0/1091092536 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c0cb720 con 0x560eb8435df0 2024-09-11T01:14:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.180+0000 7f93694e0740 1 -- 172.21.15.78:0/1091092536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 msgr2=0x560eb8440820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.180+0000 7f93694e0740 1 --2- 172.21.15.78:0/1091092536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 0x560eb8440820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f934c02a930 tx=0x7f934c109640 comp rx=0 tx=0).stop 2024-09-11T01:14:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.181+0000 7f93694e0740 1 -- 172.21.15.78:0/1091092536 shutdown_connections 2024-09-11T01:14:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.181+0000 7f93694e0740 1 --2- 172.21.15.78:0/1091092536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8441d80 0x560eb8443170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.181+0000 7f93694e0740 1 --2- 172.21.15.78:0/1091092536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb8435df0 0x560eb8440820 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.181+0000 7f93694e0740 1 --2- 172.21.15.78:0/1091092536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb84354d0 0x560eb844dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.181+0000 7f93694e0740 1 -- 172.21.15.78:0/1091092536 >> 172.21.15.78:0/1091092536 conn(0x560eb8433300 msgr2=0x560eb8440110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.181+0000 7f93694e0740 1 -- 172.21.15.78:0/1091092536 shutdown_connections 2024-09-11T01:14:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.181+0000 7f93694e0740 1 -- 172.21.15.78:0/1091092536 wait complete. 2024-09-11T01:14:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.181+0000 7f93694e0740 1 -- start start 2024-09-11T01:14:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.181+0000 7f93694e0740 1 Processor -- start 2024-09-11T01:14:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f93694e0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 0x560eb8449ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f93694e0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8435df0 0x560eb844a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f93694e0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8441d80 0x560eb8444c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f93694e0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93601ad970 con 0x560eb8441d80 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f93694e0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93601ad1e0 con 0x560eb84354d0 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f93694e0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93601aa460 con 0x560eb8435df0 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f93694e0740 10 client.0 _ll_register_callbacks cb 0x7f9358014e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f9364ea7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 0x560eb8449ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f9364ea7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 0x560eb8449ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:58804/0 (socket says 172.21.15.78:58804) 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f9364ea7640 1 -- 172.21.15.78:0/1401926008 learned_addr learned my addr 172.21.15.78:0/1401926008 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f93656a8640 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8441d80 0x560eb8444c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f935ffff640 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8435df0 0x560eb844a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f9364ea7640 1 -- 172.21.15.78:0/1401926008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8435df0 msgr2=0x560eb844a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f9364ea7640 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8435df0 0x560eb844a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f9364ea7640 1 -- 172.21.15.78:0/1401926008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8441d80 msgr2=0x560eb8444c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f9364ea7640 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8441d80 0x560eb8444c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f9364ea7640 1 -- 172.21.15.78:0/1401926008 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb8445190 con 0x560eb84354d0 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.182+0000 7f93656a8640 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8441d80 0x560eb8444c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.183+0000 7f9364ea7640 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 0x560eb8449ca0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9354009c10 tx=0x7f9354009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f100a713027131 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.183+0000 7f933ffff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.183+0000 7f933ffff640 1 -- 172.21.15.78:0/1401926008 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935401d040 con 0x560eb84354d0 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.183+0000 7f933ffff640 1 -- 172.21.15.78:0/1401926008 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f9354005240 con 0x560eb84354d0 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.183+0000 7f93694e0740 1 -- 172.21.15.78:0/1401926008 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f934c0328f0 con 0x560eb84354d0 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.183+0000 7f933ffff640 1 -- 172.21.15.78:0/1401926008 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f935403a050 con 0x560eb84354d0 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.183+0000 7f93694e0740 10 client.27572 fetch_fsmap learned FSMap version 64 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.183+0000 7f93694e0740 1 -- 172.21.15.78:0/1401926008 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x560eb84bf0f0 con 0x560eb84354d0 2024-09-11T01:14:55.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.184+0000 7f933ffff640 1 -- 172.21.15.78:0/1401926008 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934c0328f0 con 0x560eb84354d0 2024-09-11T01:14:55.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.184+0000 7f933ffff640 1 -- 172.21.15.78:0/1401926008 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f9354023040 con 0x560eb84354d0 2024-09-11T01:14:55.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.184+0000 7f93694e0740 10 client.27572 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:14:55.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.184+0000 7f93694e0740 10 client.27572 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:14:55.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.184+0000 7f93694e0740 20 client.27572 populate_metadata read hostname 'smithi078' 2024-09-11T01:14:55.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.184+0000 7f93694e0740 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x560eb84bf680 0x560eb84c1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:14:55.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.184+0000 7f93694e0740 4 client.27572 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:14:55.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.184+0000 7f93694e0740 1 -- 172.21.15.78:0/1401926008 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560eb83addf0 con 0x560eb84bf680 2024-09-11T01:14:55.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.184+0000 7f935ffff640 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x560eb84bf680 0x560eb84c1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:14:55.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.184+0000 7f935ffff640 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x560eb84bf680 0x560eb84c1aa0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80a1da9e81aa9cce server_cookie=a644d0eb46cf5cab in_seq=0 out_seq=0 2024-09-11T01:14:55.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.184+0000 7f933ffff640 10 client.27572 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:14:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.189+0000 7f933ffff640 1 -- 172.21.15.78:0/1401926008 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560eb83addf0 con 0x560eb84bf680 2024-09-11T01:14:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.189+0000 7f933ffff640 10 client.27572 handle_command_reply: tid=0 2024-09-11T01:14:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.189+0000 7f93694e0740 1 client.27572 shutdown 2024-09-11T01:14:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.189+0000 7f93694e0740 1 -- 172.21.15.78:0/1401926008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 msgr2=0x560eb8449ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.189+0000 7f93694e0740 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 0x560eb8449ca0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9354009c10 tx=0x7f9354009f00 comp rx=0 tx=0).stop 2024-09-11T01:14:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.189+0000 7f93694e0740 1 -- 172.21.15.78:0/1401926008 shutdown_connections 2024-09-11T01:14:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.189+0000 7f93694e0740 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x560eb84bf680 0x560eb84c1aa0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.189+0000 7f933ffff640 0 client.27572 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:14:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.189+0000 7f93694e0740 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560eb8441d80 0x560eb8444c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.189+0000 7f93694e0740 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560eb8435df0 0x560eb844a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.189+0000 7f93694e0740 1 --2- 172.21.15.78:0/1401926008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560eb84354d0 0x560eb8449ca0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.189+0000 7f93694e0740 1 -- 172.21.15.78:0/1401926008 >> 172.21.15.78:0/1401926008 conn(0x560eb8433300 msgr2=0x560eb8440930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.190+0000 7f93694e0740 1 -- 172.21.15.78:0/1401926008 shutdown_connections 2024-09-11T01:14:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.190+0000 7f93694e0740 1 -- 172.21.15.78:0/1401926008 wait complete. 2024-09-11T01:14:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.190+0000 7f93694e0740 20 client.27572 trim_cache size 0 max 16384 2024-09-11T01:14:55.193 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:14:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.190+0000 7f936710b640 1 -- 172.21.15.78:0/729331364 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93500b54d0 msgr2=0x7f93500b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.190+0000 7f936710b640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93500b54d0 0x7f93500b7990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9358009f10 tx=0x7f9358014f90 comp rx=0 tx=0).stop 2024-09-11T01:14:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.190+0000 7f936710b640 1 -- 172.21.15.78:0/729331364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93601076a0 msgr2=0x7f9360199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:14:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.190+0000 7f936710b640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93601076a0 0x7f9360199860 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f934c035040 tx=0x7f934c032d50 comp rx=0 tx=0).stop 2024-09-11T01:14:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.190+0000 7f936710b640 1 -- 172.21.15.78:0/729331364 shutdown_connections 2024-09-11T01:14:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.191+0000 7f936710b640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93500b54d0 0x7f93500b7990 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.191+0000 7f936710b640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9360108020 0x7f936019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.191+0000 7f936710b640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93601076a0 0x7f9360199860 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.191+0000 7f936710b640 1 --2- 172.21.15.78:0/729331364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9360101c90 0x7f9360199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:14:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.191+0000 7f936710b640 1 -- 172.21.15.78:0/729331364 >> 172.21.15.78:0/729331364 conn(0x7f93600fd700 msgr2=0x7f936006a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:14:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.191+0000 7f936710b640 1 -- 172.21.15.78:0/729331364 shutdown_connections 2024-09-11T01:14:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:14:55.191+0000 7f936710b640 1 -- 172.21.15.78:0/729331364 wait complete. 2024-09-11T01:14:55.784 INFO:tasks.workunit.client.0.smithi078.stdout: 10 126961 41.27 MB/sec execute 61 sec latency 256.056 ms 2024-09-11T01:14:56.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:55 smithi110 ceph-mon[34712]: pgmap v822: 97 pgs: 97 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 976 op/s 2024-09-11T01:14:56.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:55 smithi112 ceph-mon[35384]: pgmap v822: 97 pgs: 97 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 976 op/s 2024-09-11T01:14:56.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:55 smithi078 ceph-mon[31315]: pgmap v822: 97 pgs: 97 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 976 op/s 2024-09-11T01:14:56.784 INFO:tasks.workunit.client.0.smithi078.stdout: 10 127739 41.34 MB/sec execute 62 sec latency 111.983 ms 2024-09-11T01:14:57.784 INFO:tasks.workunit.client.0.smithi078.stdout: 10 128509 41.29 MB/sec execute 63 sec latency 124.888 ms 2024-09-11T01:14:58.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:57 smithi110 ceph-mon[34712]: pgmap v823: 97 pgs: 97 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 963 op/s 2024-09-11T01:14:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:57 smithi112 ceph-mon[35384]: pgmap v823: 97 pgs: 97 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 963 op/s 2024-09-11T01:14:58.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:57 smithi078 ceph-mon[31315]: pgmap v823: 97 pgs: 97 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 963 op/s 2024-09-11T01:14:58.784 INFO:tasks.workunit.client.0.smithi078.stdout: 10 129420 41.44 MB/sec execute 64 sec latency 80.796 ms 2024-09-11T01:14:59.784 INFO:tasks.workunit.client.0.smithi078.stdout: 10 129572 40.99 MB/sec execute 65 sec latency 638.615 ms 2024-09-11T01:15:00.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:14:59 smithi110 ceph-mon[34712]: pgmap v824: 97 pgs: 97 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 752 op/s 2024-09-11T01:15:00.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:14:59 smithi112 ceph-mon[35384]: pgmap v824: 97 pgs: 97 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 752 op/s 2024-09-11T01:15:00.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:14:59 smithi078 ceph-mon[31315]: pgmap v824: 97 pgs: 97 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 752 op/s 2024-09-11T01:15:00.784 INFO:tasks.workunit.client.0.smithi078.stdout: 10 130087 40.75 MB/sec execute 66 sec latency 326.274 ms 2024-09-11T01:15:01.784 INFO:tasks.workunit.client.0.smithi078.stdout: 10 130087 40.14 MB/sec execute 67 sec latency 1133.464 ms 2024-09-11T01:15:01.886 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:15:02.136 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:15:02.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:01 smithi110 ceph-mon[34712]: pgmap v825: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 828 op/s 2024-09-11T01:15:02.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:01 smithi112 ceph-mon[35384]: pgmap v825: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 828 op/s 2024-09-11T01:15:02.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:01 smithi078 ceph-mon[31315]: pgmap v825: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 828 op/s 2024-09-11T01:15:02.784 INFO:tasks.workunit.client.0.smithi078.stdout: 10 130243 39.72 MB/sec execute 68 sec latency 1524.248 ms 2024-09-11T01:15:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.329+0000 7f93a2e62640 1 -- 172.21.15.78:0/2808912994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939c1131c0 msgr2=0x7f939c113600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.329+0000 7f93a2e62640 1 --2- 172.21.15.78:0/2808912994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939c1131c0 0x7f939c113600 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f938c009900 tx=0x7f938c02a410 comp rx=0 tx=0).stop 2024-09-11T01:15:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.329+0000 7f93a2e62640 1 -- 172.21.15.78:0/2808912994 shutdown_connections 2024-09-11T01:15:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.329+0000 7f93a2e62640 1 --2- 172.21.15.78:0/2808912994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c113b40 0x7f939c11a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.329+0000 7f93a2e62640 1 --2- 172.21.15.78:0/2808912994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939c1131c0 0x7f939c113600 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.329+0000 7f93a2e62640 1 --2- 172.21.15.78:0/2808912994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c10d800 0x7f939c10dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.329+0000 7f93a2e62640 1 -- 172.21.15.78:0/2808912994 >> 172.21.15.78:0/2808912994 conn(0x7f939c06dce0 msgr2=0x7f939c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.329+0000 7f93a2e62640 1 -- 172.21.15.78:0/2808912994 shutdown_connections 2024-09-11T01:15:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.329+0000 7f93a2e62640 1 -- 172.21.15.78:0/2808912994 wait complete. 2024-09-11T01:15:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a2e62640 1 Processor -- start 2024-09-11T01:15:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a2e62640 1 -- start start 2024-09-11T01:15:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a2e62640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939c10d800 0x7f939c1a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a2e62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c1131c0 0x7f939c1a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a2e62640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c113b40 0x7f939c1a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a2e62640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f939c11d7d0 con 0x7f939c10d800 2024-09-11T01:15:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a2e62640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f939c11d650 con 0x7f939c113b40 2024-09-11T01:15:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a2e62640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f939c11d950 con 0x7f939c1131c0 2024-09-11T01:15:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a0bfe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939c10d800 0x7f939c1a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a0bfe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939c10d800 0x7f939c1a3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40574/0 (socket says 172.21.15.78:40574) 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a0bfe640 1 -- 172.21.15.78:0/1465415266 learned_addr learned my addr 172.21.15.78:0/1465415266 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f9393fff640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c1131c0 0x7f939c1a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a13ff640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c113b40 0x7f939c1a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.330+0000 7f93a0bfe640 1 -- 172.21.15.78:0/1465415266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c1131c0 msgr2=0x7f939c1a41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.331+0000 7f93a0bfe640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c1131c0 0x7f939c1a41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.331+0000 7f93a0bfe640 1 -- 172.21.15.78:0/1465415266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c113b40 msgr2=0x7f939c1a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.331+0000 7f93a0bfe640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c113b40 0x7f939c1a9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.331+0000 7f93a0bfe640 1 -- 172.21.15.78:0/1465415266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c1a9d40 con 0x7f939c10d800 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.331+0000 7f93a13ff640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c113b40 0x7f939c1a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.331+0000 7f93a0bfe640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939c10d800 0x7f939c1a3c90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f939800b4a0 tx=0x7f939800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.331+0000 7f9391ffb640 1 -- 172.21.15.78:0/1465415266 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398014070 con 0x7f939c10d800 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.331+0000 7f93a2e62640 1 -- 172.21.15.78:0/1465415266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939c1aa030 con 0x7f939c10d800 2024-09-11T01:15:03.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.331+0000 7f9391ffb640 1 -- 172.21.15.78:0/1465415266 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9398010ac0 con 0x7f939c10d800 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.331+0000 7f9391ffb640 1 -- 172.21.15.78:0/1465415266 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398017c30 con 0x7f939c10d800 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.331+0000 7f93a2e62640 1 -- 172.21.15.78:0/1465415266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939c1b0770 con 0x7f939c10d800 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.333+0000 7f9391ffb640 1 -- 172.21.15.78:0/1465415266 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9398010610 con 0x7f939c10d800 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f93a5237740 1 Processor -- start 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f93a5237740 1 -- start start 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f93a5237740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 0x55eebb53c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f93a5237740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb53cdf0 0x55eebb548840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f93a5237740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb548d80 0x55eebb54b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f93a5237740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eebb4b2ea0 con 0x55eebb548d80 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f93a5237740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eebb4a22d0 con 0x55eebb53c4d0 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f93a5237740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eebb4a60e0 con 0x55eebb53cdf0 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f9393fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb548d80 0x55eebb54b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f9393fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb548d80 0x55eebb54b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40580/0 (socket says 172.21.15.78:40580) 2024-09-11T01:15:03.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f9393fff640 1 -- 172.21.15.78:0/273220072 learned_addr learned my addr 172.21.15.78:0/273220072 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f93a13ff640 1 --2- 172.21.15.78:0/273220072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 0x55eebb53c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f93a0bfe640 1 --2- 172.21.15.78:0/273220072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb53cdf0 0x55eebb548840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f9391ffb640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93840b5450 0x7f93840b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.334+0000 7f9393fff640 1 -- 172.21.15.78:0/273220072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb53cdf0 msgr2=0x55eebb548840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.335+0000 7f9393fff640 1 --2- 172.21.15.78:0/273220072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb53cdf0 0x55eebb548840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.335+0000 7f9393fff640 1 -- 172.21.15.78:0/273220072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 msgr2=0x55eebb53c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.335+0000 7f9393fff640 1 --2- 172.21.15.78:0/273220072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 0x55eebb53c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.335+0000 7f9393fff640 1 -- 172.21.15.78:0/273220072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eebb200650 con 0x55eebb548d80 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.335+0000 7f93a13ff640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93840b5450 0x7f93840b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.335+0000 7f9391ffb640 1 -- 172.21.15.78:0/1465415266 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f93980e55b0 con 0x7f939c10d800 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.335+0000 7f9393fff640 1 --2- 172.21.15.78:0/273220072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb548d80 0x55eebb54b170 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f938c009900 tx=0x7f938c031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ad5db0730fd14f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.335+0000 7f9382ffd640 1 -- 172.21.15.78:0/273220072 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c040070 con 0x55eebb548d80 2024-09-11T01:15:03.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.335+0000 7f9382ffd640 1 -- 172.21.15.78:0/273220072 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c0316a0 con 0x55eebb548d80 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.335+0000 7f9382ffd640 1 -- 172.21.15.78:0/273220072 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c035040 con 0x55eebb548d80 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.336+0000 7f93a5237740 1 -- 172.21.15.78:0/273220072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb548d80 msgr2=0x55eebb54b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.336+0000 7f93a13ff640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93840b5450 0x7f93840b7910 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9394012930 tx=0x7f9394004ab0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.336+0000 7f93a5237740 1 --2- 172.21.15.78:0/273220072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb548d80 0x55eebb54b170 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f938c009900 tx=0x7f938c031de0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.336+0000 7f93a5237740 1 -- 172.21.15.78:0/273220072 shutdown_connections 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.336+0000 7f93a5237740 1 --2- 172.21.15.78:0/273220072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb548d80 0x55eebb54b170 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.336+0000 7f93a5237740 1 --2- 172.21.15.78:0/273220072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb53cdf0 0x55eebb548840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.336+0000 7f93a5237740 1 --2- 172.21.15.78:0/273220072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 0x55eebb53c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.336+0000 7f93a5237740 1 -- 172.21.15.78:0/273220072 >> 172.21.15.78:0/273220072 conn(0x55eebb53a300 msgr2=0x55eebb53b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.336+0000 7f93a5237740 1 -- 172.21.15.78:0/273220072 shutdown_connections 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.336+0000 7f93a5237740 1 -- 172.21.15.78:0/273220072 wait complete. 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.336+0000 7f93a5237740 1 -- start start 2024-09-11T01:15:03.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.336+0000 7f93a5237740 1 Processor -- start 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f93a5237740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb53c4d0 0x55eebb5485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f93a5237740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb545a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f93a5237740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb548d80 0x55eebb545fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f93a5237740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eebb303700 con 0x55eebb53cdf0 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f93a5237740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eebb43d160 con 0x55eebb548d80 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f93a5237740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eebb54b9a0 con 0x55eebb53c4d0 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f93a5237740 10 client.0 _ll_register_callbacks cb 0x55eebb4569e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f9393fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb545a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f9393fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb545a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40582/0 (socket says 172.21.15.78:40582) 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f9393fff640 1 -- 172.21.15.78:0/646731413 learned_addr learned my addr 172.21.15.78:0/646731413 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f93a0bfe640 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb53c4d0 0x55eebb5485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f93a13ff640 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb548d80 0x55eebb545fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f9393fff640 1 -- 172.21.15.78:0/646731413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb53c4d0 msgr2=0x55eebb5485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f9393fff640 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb53c4d0 0x55eebb5485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f9393fff640 1 -- 172.21.15.78:0/646731413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb548d80 msgr2=0x55eebb545fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f9393fff640 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb548d80 0x55eebb545fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f9393fff640 1 -- 172.21.15.78:0/646731413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eebb46ac90 con 0x55eebb53cdf0 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f9393fff640 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb545a40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f938c0039f0 tx=0x7f938c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecd5c03bd746f78 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f937bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.337+0000 7f937bfff640 1 -- 172.21.15.78:0/646731413 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c040070 con 0x55eebb53cdf0 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f937bfff640 1 -- 172.21.15.78:0/646731413 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f938c03c440 con 0x55eebb53cdf0 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f93a5237740 1 -- 172.21.15.78:0/646731413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938c0316a0 con 0x55eebb53cdf0 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f937bfff640 1 -- 172.21.15.78:0/646731413 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f938c056050 con 0x55eebb53cdf0 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f93a5237740 10 client.18435 fetch_fsmap learned FSMap version 64 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f93a5237740 1 -- 172.21.15.78:0/646731413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55eebb557280 con 0x55eebb53cdf0 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f937bfff640 1 -- 172.21.15.78:0/646731413 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c0316a0 con 0x55eebb53cdf0 2024-09-11T01:15:03.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f937bfff640 1 -- 172.21.15.78:0/646731413 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f938c003e40 con 0x55eebb53cdf0 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f93a5237740 10 client.18435 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f93a5237740 10 client.18435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f93a5237740 20 client.18435 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f93a5237740 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55eebb5585d0 0x55eebb55a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f93a5237740 4 client.18435 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f93a5237740 1 -- 172.21.15.78:0/646731413 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eebb4b4df0 con 0x55eebb5585d0 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.338+0000 7f93a0bfe640 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55eebb5585d0 0x55eebb55a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.339+0000 7f93a0bfe640 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55eebb5585d0 0x55eebb55a9c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d5dbdb6b52e3e25 server_cookie=4655e6a4c9dc27cb in_seq=0 out_seq=0 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.339+0000 7f937bfff640 10 client.18435 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f937bfff640 1 -- 172.21.15.78:0/646731413 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eebb4b4df0 con 0x55eebb5585d0 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f937bfff640 10 client.18435 handle_command_reply: tid=0 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f93a5237740 1 client.18435 shutdown 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f93a5237740 1 -- 172.21.15.78:0/646731413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 msgr2=0x55eebb545a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:03.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f93a5237740 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb545a40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f938c0039f0 tx=0x7f938c02a410 comp rx=0 tx=0).stop 2024-09-11T01:15:03.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f93a5237740 1 -- 172.21.15.78:0/646731413 shutdown_connections 2024-09-11T01:15:03.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f93a5237740 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb548d80 0x55eebb545fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f93a5237740 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55eebb5585d0 0x55eebb55a9c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f93a5237740 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb545a40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f93a5237740 1 --2- 172.21.15.78:0/646731413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb53c4d0 0x55eebb5485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f937bfff640 0 client.18435 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:03.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f93a5237740 1 -- 172.21.15.78:0/646731413 >> 172.21.15.78:0/646731413 conn(0x55eebb53a300 msgr2=0x55eebb53fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:03.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f93a5237740 1 -- 172.21.15.78:0/646731413 shutdown_connections 2024-09-11T01:15:03.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f93a5237740 1 -- 172.21.15.78:0/646731413 wait complete. 2024-09-11T01:15:03.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.340+0000 7f93a5237740 20 client.18435 trim_cache size 0 max 16384 2024-09-11T01:15:03.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f93a5237740 1 Processor -- start 2024-09-11T01:15:03.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f93a5237740 1 -- start start 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f93a5237740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 0x55eebb554c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f93a5237740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb53b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f93a5237740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb548d80 0x55eebb53b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f93a5237740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1b8f60 con 0x55eebb53cdf0 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f93a5237740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1bac20 con 0x55eebb53c4d0 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f93a5237740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1b9630 con 0x55eebb548d80 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f9393fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb53b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f9393fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb53b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40588/0 (socket says 172.21.15.78:40588) 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f9393fff640 1 -- 172.21.15.78:0/3063810928 learned_addr learned my addr 172.21.15.78:0/3063810928 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f9393fff640 1 -- 172.21.15.78:0/3063810928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb548d80 msgr2=0x55eebb53b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f93a13ff640 1 --2- 172.21.15.78:0/3063810928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb548d80 0x55eebb53b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f9393fff640 1 --2- 172.21.15.78:0/3063810928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb548d80 0x55eebb53b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f9393fff640 1 -- 172.21.15.78:0/3063810928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 msgr2=0x55eebb554c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:03.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f9393fff640 1 --2- 172.21.15.78:0/3063810928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 0x55eebb554c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f9393fff640 1 -- 172.21.15.78:0/3063810928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eebb53ad80 con 0x55eebb53cdf0 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.353+0000 7f93a13ff640 1 --2- 172.21.15.78:0/3063810928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb548d80 0x55eebb53b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.354+0000 7f9393fff640 1 --2- 172.21.15.78:0/3063810928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb53b350 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f938c033000 tx=0x7f938c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab8249093f13263 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.354+0000 7f93827fc640 1 -- 172.21.15.78:0/3063810928 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c040040 con 0x55eebb53cdf0 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.354+0000 7f93827fc640 1 -- 172.21.15.78:0/3063810928 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c04d070 con 0x55eebb53cdf0 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.354+0000 7f93827fc640 1 -- 172.21.15.78:0/3063810928 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c035040 con 0x55eebb53cdf0 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.354+0000 7f93a5237740 1 -- 172.21.15.78:0/3063810928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 msgr2=0x55eebb53b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.354+0000 7f93a5237740 1 --2- 172.21.15.78:0/3063810928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb53b350 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f938c033000 tx=0x7f938c005e80 comp rx=0 tx=0).stop 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.354+0000 7f93a5237740 1 -- 172.21.15.78:0/3063810928 shutdown_connections 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.354+0000 7f93a5237740 1 --2- 172.21.15.78:0/3063810928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb548d80 0x55eebb53b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.354+0000 7f93a5237740 1 --2- 172.21.15.78:0/3063810928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb53b350 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.354+0000 7f93a5237740 1 --2- 172.21.15.78:0/3063810928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 0x55eebb554c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.354+0000 7f93a5237740 1 -- 172.21.15.78:0/3063810928 >> 172.21.15.78:0/3063810928 conn(0x55eebb53a300 msgr2=0x55eebb540c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:03.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f93a5237740 1 -- 172.21.15.78:0/3063810928 shutdown_connections 2024-09-11T01:15:03.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f93a5237740 1 -- 172.21.15.78:0/3063810928 wait complete. 2024-09-11T01:15:03.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f93a5237740 1 -- start start 2024-09-11T01:15:03.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f93a5237740 1 Processor -- start 2024-09-11T01:15:03.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f93a5237740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 0x55eebb5511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f93a5237740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb54d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f93a5237740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb548d80 0x55eebb54dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f93a5237740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1ba490 con 0x55eebb53cdf0 2024-09-11T01:15:03.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f93a5237740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1b9d00 con 0x55eebb53c4d0 2024-09-11T01:15:03.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f93a5237740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1b6f60 con 0x55eebb548d80 2024-09-11T01:15:03.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f93a5237740 10 client.0 _ll_register_callbacks cb 0x7f93940075c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:03.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f9393fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb54d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f9393fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb54d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40604/0 (socket says 172.21.15.78:40604) 2024-09-11T01:15:03.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.355+0000 7f9393fff640 1 -- 172.21.15.78:0/2805512577 learned_addr learned my addr 172.21.15.78:0/2805512577 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:03.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f93a13ff640 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb548d80 0x55eebb54dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f9393fff640 1 -- 172.21.15.78:0/2805512577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb548d80 msgr2=0x55eebb54dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:03.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f9393fff640 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb548d80 0x55eebb54dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f9393fff640 1 -- 172.21.15.78:0/2805512577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 msgr2=0x55eebb5511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:03.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f93a0bfe640 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 0x55eebb5511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f9393fff640 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 0x55eebb5511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f9393fff640 1 -- 172.21.15.78:0/2805512577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eebb535990 con 0x55eebb53cdf0 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f93a0bfe640 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 0x55eebb5511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f9393fff640 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb54d6a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f938c000c00 tx=0x7f938c031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc40a4e2bf2ef71 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f937bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f937bfff640 1 -- 172.21.15.78:0/2805512577 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c035070 con 0x55eebb53cdf0 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f937bfff640 1 -- 172.21.15.78:0/2805512577 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f938c005430 con 0x55eebb53cdf0 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f93a5237740 1 -- 172.21.15.78:0/2805512577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938c04d070 con 0x55eebb53cdf0 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f937bfff640 1 -- 172.21.15.78:0/2805512577 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f938c040040 con 0x55eebb53cdf0 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f93a5237740 10 client.18441 fetch_fsmap learned FSMap version 64 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f93a5237740 1 -- 172.21.15.78:0/2805512577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55eebb5c5e00 con 0x55eebb53cdf0 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.356+0000 7f937bfff640 1 -- 172.21.15.78:0/2805512577 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c04d070 con 0x55eebb53cdf0 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.357+0000 7f937bfff640 1 -- 172.21.15.78:0/2805512577 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f938c004a10 con 0x55eebb53cdf0 2024-09-11T01:15:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.357+0000 7f93a5237740 10 client.18441 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:15:03.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.357+0000 7f93a5237740 10 client.18441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:15:03.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.357+0000 7f93a5237740 20 client.18441 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:03.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.357+0000 7f93a5237740 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55eebb5c6630 0x55eebb5c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:03.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.357+0000 7f93a5237740 4 client.18441 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:15:03.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.357+0000 7f93a5237740 1 -- 172.21.15.78:0/2805512577 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eebb4b08a0 con 0x55eebb5c6630 2024-09-11T01:15:03.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.357+0000 7f93a0bfe640 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55eebb5c6630 0x55eebb5c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:03.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.357+0000 7f93a0bfe640 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55eebb5c6630 0x55eebb5c8a50 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd3ce2117b6d489 server_cookie=dbf33be38bf82c5d in_seq=0 out_seq=0 2024-09-11T01:15:03.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.357+0000 7f937bfff640 10 client.18441 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:03.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f937bfff640 1 -- 172.21.15.78:0/2805512577 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eebb4b08a0 con 0x55eebb5c6630 2024-09-11T01:15:03.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f937bfff640 10 client.18441 handle_command_reply: tid=0 2024-09-11T01:15:03.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f93a5237740 1 client.18441 shutdown 2024-09-11T01:15:03.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f93a5237740 1 -- 172.21.15.78:0/2805512577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 msgr2=0x55eebb54d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:03.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f93a5237740 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb54d6a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f938c000c00 tx=0x7f938c031520 comp rx=0 tx=0).stop 2024-09-11T01:15:03.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f93a5237740 1 -- 172.21.15.78:0/2805512577 shutdown_connections 2024-09-11T01:15:03.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f93a5237740 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eebb548d80 0x55eebb54dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f93a5237740 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55eebb5c6630 0x55eebb5c8a50 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f93a5237740 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eebb53cdf0 0x55eebb54d6a0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f93a5237740 1 --2- 172.21.15.78:0/2805512577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eebb53c4d0 0x55eebb5511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f93a5237740 1 -- 172.21.15.78:0/2805512577 >> 172.21.15.78:0/2805512577 conn(0x55eebb53a300 msgr2=0x55eebb53b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f937bfff640 0 client.18441 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f93a5237740 1 -- 172.21.15.78:0/2805512577 shutdown_connections 2024-09-11T01:15:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f93a5237740 1 -- 172.21.15.78:0/2805512577 wait complete. 2024-09-11T01:15:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.358+0000 7f93a5237740 20 client.18441 trim_cache size 0 max 16384 2024-09-11T01:15:03.377 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:15:03.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.359+0000 7f93a2e62640 1 -- 172.21.15.78:0/1465415266 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93840b5450 msgr2=0x7f93840b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:03.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.359+0000 7f93a2e62640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93840b5450 0x7f93840b7910 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9394012930 tx=0x7f9394004ab0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.359+0000 7f93a2e62640 1 -- 172.21.15.78:0/1465415266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939c10d800 msgr2=0x7f939c1a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:03.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.359+0000 7f93a2e62640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939c10d800 0x7f939c1a3c90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f939800b4a0 tx=0x7f939800b960 comp rx=0 tx=0).stop 2024-09-11T01:15:03.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.359+0000 7f93a2e62640 1 -- 172.21.15.78:0/1465415266 shutdown_connections 2024-09-11T01:15:03.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.359+0000 7f93a2e62640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f93840b5450 0x7f93840b7910 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.359+0000 7f93a2e62640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c113b40 0x7f939c1a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.359+0000 7f93a2e62640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c1131c0 0x7f939c1a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.359+0000 7f93a2e62640 1 --2- 172.21.15.78:0/1465415266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939c10d800 0x7f939c1a3c90 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:03.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.359+0000 7f93a2e62640 1 -- 172.21.15.78:0/1465415266 >> 172.21.15.78:0/1465415266 conn(0x7f939c06dce0 msgr2=0x7f939c1111b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:03.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.359+0000 7f93a2e62640 1 -- 172.21.15.78:0/1465415266 shutdown_connections 2024-09-11T01:15:03.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:03.359+0000 7f93a2e62640 1 -- 172.21.15.78:0/1465415266 wait complete. 2024-09-11T01:15:03.784 INFO:tasks.workunit.client.0.smithi078.stdout: 10 130243 39.14 MB/sec execute 69 sec latency 1462.983 ms 2024-09-11T01:15:04.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:03 smithi110 ceph-mon[34712]: pgmap v826: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 773 op/s 2024-09-11T01:15:04.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:03 smithi112 ceph-mon[35384]: pgmap v826: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 773 op/s 2024-09-11T01:15:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:03 smithi078 ceph-mon[31315]: pgmap v826: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 773 op/s 2024-09-11T01:15:04.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 130243 38.58 MB/sec execute 70 sec latency 2463.107 ms 2024-09-11T01:15:05.105 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:15:05.106 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:15:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:15:05.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 130243 38.04 MB/sec execute 71 sec latency 3463.149 ms 2024-09-11T01:15:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.854+0000 7f50f917d640 1 -- 172.21.15.78:0/4194295667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50f4104530 msgr2=0x7f50f410cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.854+0000 7f50f917d640 1 --2- 172.21.15.78:0/4194295667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50f4104530 0x7f50f410cb10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009900 tx=0x7f50e802a410 comp rx=0 tx=0).stop 2024-09-11T01:15:05.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.854+0000 7f50f917d640 1 -- 172.21.15.78:0/4194295667 shutdown_connections 2024-09-11T01:15:05.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.854+0000 7f50f917d640 1 --2- 172.21.15.78:0/4194295667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50f410d0e0 0x7f50f410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.854+0000 7f50f917d640 1 --2- 172.21.15.78:0/4194295667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50f4104530 0x7f50f410cb10 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.854+0000 7f50f917d640 1 --2- 172.21.15.78:0/4194295667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50f4103b30 0x7f50f4103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.854+0000 7f50f917d640 1 -- 172.21.15.78:0/4194295667 >> 172.21.15.78:0/4194295667 conn(0x7f50f40fd700 msgr2=0x7f50f40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:05.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.854+0000 7f50f917d640 1 -- 172.21.15.78:0/4194295667 shutdown_connections 2024-09-11T01:15:05.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.854+0000 7f50f917d640 1 -- 172.21.15.78:0/4194295667 wait complete. 2024-09-11T01:15:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.855+0000 7f50f917d640 1 Processor -- start 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.855+0000 7f50f917d640 1 -- start start 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.855+0000 7f50f917d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50f4103b30 0x7f50f41019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.855+0000 7f50f917d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50f4104530 0x7f50f4101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f917d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50f410d0e0 0x7f50f4102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f917d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4112d60 con 0x7f50f4104530 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f917d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4112be0 con 0x7f50f4103b30 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f917d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4112ee0 con 0x7f50f410d0e0 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50f4104530 0x7f50f4101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50f4104530 0x7f50f4101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40608/0 (socket says 172.21.15.78:40608) 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f259c640 1 -- 172.21.15.78:0/3817158477 learned_addr learned my addr 172.21.15.78:0/3817158477 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f2d9d640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50f4103b30 0x7f50f41019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f359e640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50f410d0e0 0x7f50f4102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f259c640 1 -- 172.21.15.78:0/3817158477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50f410d0e0 msgr2=0x7f50f4102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f259c640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50f410d0e0 0x7f50f4102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f259c640 1 -- 172.21.15.78:0/3817158477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50f4103b30 msgr2=0x7f50f41019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:05.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f259c640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50f4103b30 0x7f50f41019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f259c640 1 -- 172.21.15.78:0/3817158477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f41a7630 con 0x7f50f4104530 2024-09-11T01:15:05.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50f259c640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50f4104530 0x7f50f4101ef0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f50e8006010 tx=0x7f50e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:05.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.856+0000 7f50dbfff640 1 -- 172.21.15.78:0/3817158477 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e803f030 con 0x7f50f4104530 2024-09-11T01:15:05.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.857+0000 7f50dbfff640 1 -- 172.21.15.78:0/3817158477 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e8003800 con 0x7f50f4104530 2024-09-11T01:15:05.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.857+0000 7f50f917d640 1 -- 172.21.15.78:0/3817158477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f41a78c0 con 0x7f50f4104530 2024-09-11T01:15:05.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.857+0000 7f50dbfff640 1 -- 172.21.15.78:0/3817158477 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8035070 con 0x7f50f4104530 2024-09-11T01:15:05.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.857+0000 7f50f917d640 1 -- 172.21.15.78:0/3817158477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f41a7da0 con 0x7f50f4104530 2024-09-11T01:15:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50fb552740 1 Processor -- start 2024-09-11T01:15:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50fb552740 1 -- start start 2024-09-11T01:15:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50dbfff640 1 -- 172.21.15.78:0/3817158477 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f50e8032310 con 0x7f50f4104530 2024-09-11T01:15:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50fb552740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d5f9ee0 0x55915d5fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50fb552740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d606250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50fb552740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d606790 0x55915d608b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50fb552740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55915d570e60 con 0x55915d5fa800 2024-09-11T01:15:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50fb552740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55915d560290 con 0x55915d5f9ee0 2024-09-11T01:15:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50fb552740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55915d5640a0 con 0x55915d606790 2024-09-11T01:15:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50f2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d606250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50f2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d606250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40610/0 (socket says 172.21.15.78:40610) 2024-09-11T01:15:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50f2d9d640 1 -- 172.21.15.78:0/2295189777 learned_addr learned my addr 172.21.15.78:0/2295189777 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50f259c640 1 --2- 172.21.15.78:0/2295189777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d606790 0x55915d608b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50f359e640 1 --2- 172.21.15.78:0/2295189777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d5f9ee0 0x55915d5fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50f2d9d640 1 -- 172.21.15.78:0/2295189777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d606790 msgr2=0x55915d608b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50f2d9d640 1 --2- 172.21.15.78:0/2295189777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d606790 0x55915d608b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50f2d9d640 1 -- 172.21.15.78:0/2295189777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d5f9ee0 msgr2=0x55915d5fa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50f2d9d640 1 --2- 172.21.15.78:0/2295189777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d5f9ee0 0x55915d5fa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50f2d9d640 1 -- 172.21.15.78:0/2295189777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55915d2be930 con 0x55915d5fa800 2024-09-11T01:15:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.860+0000 7f50f259c640 1 --2- 172.21.15.78:0/2295189777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d606790 0x55915d608b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:15:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.861+0000 7f50f2d9d640 1 --2- 172.21.15.78:0/2295189777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d606250 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f50e400ad30 tx=0x7f50e400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae53bd353e24bf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.861+0000 7f50dbfff640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50dc0b5450 0x7f50dc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.861+0000 7f50d97fa640 1 -- 172.21.15.78:0/2295189777 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4010840 con 0x55915d5fa800 2024-09-11T01:15:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.861+0000 7f50f359e640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50dc0b5450 0x7f50dc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.861+0000 7f50d97fa640 1 -- 172.21.15.78:0/2295189777 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e4014070 con 0x55915d5fa800 2024-09-11T01:15:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.861+0000 7f50dbfff640 1 -- 172.21.15.78:0/3817158477 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f50e81067d0 con 0x7f50f4104530 2024-09-11T01:15:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.861+0000 7f50d97fa640 1 -- 172.21.15.78:0/2295189777 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4018ae0 con 0x55915d5fa800 2024-09-11T01:15:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.862+0000 7f50fb552740 1 -- 172.21.15.78:0/2295189777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 msgr2=0x55915d606250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.862+0000 7f50f359e640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50dc0b5450 0x7f50dc0b7910 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f50f4103a70 tx=0x7f50ec00a4d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.862+0000 7f50fb552740 1 --2- 172.21.15.78:0/2295189777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d606250 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f50e400ad30 tx=0x7f50e400f790 comp rx=0 tx=0).stop 2024-09-11T01:15:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.862+0000 7f50fb552740 1 -- 172.21.15.78:0/2295189777 shutdown_connections 2024-09-11T01:15:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.862+0000 7f50fb552740 1 --2- 172.21.15.78:0/2295189777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d606790 0x55915d608b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.862+0000 7f50fb552740 1 --2- 172.21.15.78:0/2295189777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d606250 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.862+0000 7f50fb552740 1 --2- 172.21.15.78:0/2295189777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d5f9ee0 0x55915d5fa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.862+0000 7f50fb552740 1 -- 172.21.15.78:0/2295189777 >> 172.21.15.78:0/2295189777 conn(0x55915d5f7d10 msgr2=0x55915d5f8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.862+0000 7f50fb552740 1 -- 172.21.15.78:0/2295189777 shutdown_connections 2024-09-11T01:15:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.862+0000 7f50fb552740 1 -- 172.21.15.78:0/2295189777 wait complete. 2024-09-11T01:15:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.862+0000 7f50fb552740 1 -- start start 2024-09-11T01:15:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.862+0000 7f50fb552740 1 Processor -- start 2024-09-11T01:15:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50fb552740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d5f9ee0 0x55915d6036b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50fb552740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d603bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50fb552740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d606790 0x55915d604130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50fb552740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55915d3c1890 con 0x55915d5fa800 2024-09-11T01:15:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50fb552740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55915d4fad40 con 0x55915d606790 2024-09-11T01:15:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50fb552740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55915d6094e0 con 0x55915d5f9ee0 2024-09-11T01:15:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50fb552740 10 client.0 _ll_register_callbacks cb 0x55915d55ec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d603bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d603bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40616/0 (socket says 172.21.15.78:40616) 2024-09-11T01:15:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50f259c640 1 -- 172.21.15.78:0/3786890395 learned_addr learned my addr 172.21.15.78:0/3786890395 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50f359e640 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d606790 0x55915d604130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50f2d9d640 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d5f9ee0 0x55915d6036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50f259c640 1 -- 172.21.15.78:0/3786890395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d5f9ee0 msgr2=0x55915d6036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50f259c640 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d5f9ee0 0x55915d6036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50f259c640 1 -- 172.21.15.78:0/3786890395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d606790 msgr2=0x55915d604130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50f259c640 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d606790 0x55915d604130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50f259c640 1 -- 172.21.15.78:0/3786890395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55915d612810 con 0x55915d5fa800 2024-09-11T01:15:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50f359e640 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d606790 0x55915d604130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:15:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50f259c640 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d603bf0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f50e80ccc20 tx=0x7f50e802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b8a7f299546d3d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.863+0000 7f50d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:15:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50d6ff5640 1 -- 172.21.15.78:0/3786890395 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8049070 con 0x55915d5fa800 2024-09-11T01:15:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50d6ff5640 1 -- 172.21.15.78:0/3786890395 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f50e8003cb0 con 0x55915d5fa800 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50fb552740 1 -- 172.21.15.78:0/3786890395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e4014070 con 0x55915d5fa800 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50d6ff5640 1 -- 172.21.15.78:0/3786890395 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f50e80cc5e0 con 0x55915d5fa800 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50fb552740 10 client.18453 fetch_fsmap learned FSMap version 64 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50fb552740 1 -- 172.21.15.78:0/3786890395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55915d6151c0 con 0x55915d5fa800 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50d6ff5640 1 -- 172.21.15.78:0/3786890395 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e80cc7a0 con 0x55915d5fa800 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50d6ff5640 1 -- 172.21.15.78:0/3786890395 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f50e80ccef0 con 0x55915d5fa800 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50fb552740 10 client.18453 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50fb552740 10 client.18453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50fb552740 20 client.18453 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50fb552740 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55915d616670 0x55915d618a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50fb552740 4 client.18453 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.864+0000 7f50fb552740 1 -- 172.21.15.78:0/3786890395 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f50e4014070 con 0x55915d616670 2024-09-11T01:15:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.865+0000 7f50f2d9d640 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55915d616670 0x55915d618a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.865+0000 7f50d6ff5640 10 client.18453 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.865+0000 7f50f2d9d640 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55915d616670 0x55915d618a60 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f7b5111343b6a6 server_cookie=86d389c338a6f135 in_seq=0 out_seq=0 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50d6ff5640 1 -- 172.21.15.78:0/3786890395 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f50e4014070 con 0x55915d616670 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50d6ff5640 10 client.18453 handle_command_reply: tid=0 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50fb552740 1 client.18453 shutdown 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50fb552740 1 -- 172.21.15.78:0/3786890395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 msgr2=0x55915d603bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50fb552740 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d603bf0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f50e80ccc20 tx=0x7f50e802a410 comp rx=0 tx=0).stop 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50fb552740 1 -- 172.21.15.78:0/3786890395 shutdown_connections 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50fb552740 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d606790 0x55915d604130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50fb552740 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55915d616670 0x55915d618a60 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50fb552740 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d603bf0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50d6ff5640 0 client.18453 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:05.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50fb552740 1 --2- 172.21.15.78:0/3786890395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d5f9ee0 0x55915d6036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50fb552740 1 -- 172.21.15.78:0/3786890395 >> 172.21.15.78:0/3786890395 conn(0x55915d5f7d10 msgr2=0x55915d5f6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:05.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50fb552740 1 -- 172.21.15.78:0/3786890395 shutdown_connections 2024-09-11T01:15:05.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50fb552740 1 -- 172.21.15.78:0/3786890395 wait complete. 2024-09-11T01:15:05.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.866+0000 7f50fb552740 20 client.18453 trim_cache size 0 max 16384 2024-09-11T01:15:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.883+0000 7f50fb552740 1 Processor -- start 2024-09-11T01:15:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.883+0000 7f50fb552740 1 -- start start 2024-09-11T01:15:05.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50fb552740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d5f9ee0 0x55915d6025a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50fb552740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d603bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50fb552740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d606790 0x55915d6040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50fb552740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41b0730 con 0x55915d5fa800 2024-09-11T01:15:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50fb552740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41b23f0 con 0x55915d5f9ee0 2024-09-11T01:15:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50fb552740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41b0e00 con 0x55915d606790 2024-09-11T01:15:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d603bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d603bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40630/0 (socket says 172.21.15.78:40630) 2024-09-11T01:15:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50f259c640 1 -- 172.21.15.78:0/3789096776 learned_addr learned my addr 172.21.15.78:0/3789096776 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50f359e640 1 --2- 172.21.15.78:0/3789096776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d606790 0x55915d6040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50f259c640 1 -- 172.21.15.78:0/3789096776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d606790 msgr2=0x55915d6040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50f259c640 1 --2- 172.21.15.78:0/3789096776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d606790 0x55915d6040f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50f259c640 1 -- 172.21.15.78:0/3789096776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d5f9ee0 msgr2=0x55915d6025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50f2d9d640 1 --2- 172.21.15.78:0/3789096776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d5f9ee0 0x55915d6025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50f259c640 1 --2- 172.21.15.78:0/3789096776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d5f9ee0 0x55915d6025a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50f259c640 1 -- 172.21.15.78:0/3789096776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55915d5fa630 con 0x55915d5fa800 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50f359e640 1 --2- 172.21.15.78:0/3789096776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d606790 0x55915d6040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50f259c640 1 --2- 172.21.15.78:0/3789096776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d603bb0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f50e80c9450 tx=0x7f50e80c92e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554c5cfcbba9b63a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50d8ff9640 1 -- 172.21.15.78:0/3789096776 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8049070 con 0x55915d5fa800 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50d8ff9640 1 -- 172.21.15.78:0/3789096776 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e80cc7a0 con 0x55915d5fa800 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.884+0000 7f50d8ff9640 1 -- 172.21.15.78:0/3789096776 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e80cc400 con 0x55915d5fa800 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.885+0000 7f50fb552740 1 -- 172.21.15.78:0/3789096776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 msgr2=0x55915d603bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.885+0000 7f50fb552740 1 --2- 172.21.15.78:0/3789096776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d603bb0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f50e80c9450 tx=0x7f50e80c92e0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.885+0000 7f50fb552740 1 -- 172.21.15.78:0/3789096776 shutdown_connections 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.885+0000 7f50fb552740 1 --2- 172.21.15.78:0/3789096776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d606790 0x55915d6040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.885+0000 7f50fb552740 1 --2- 172.21.15.78:0/3789096776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d603bb0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.885+0000 7f50fb552740 1 --2- 172.21.15.78:0/3789096776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d5f9ee0 0x55915d6025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.885+0000 7f50fb552740 1 -- 172.21.15.78:0/3789096776 >> 172.21.15.78:0/3789096776 conn(0x55915d5f7d10 msgr2=0x55915d605500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.885+0000 7f50fb552740 1 -- 172.21.15.78:0/3789096776 shutdown_connections 2024-09-11T01:15:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.885+0000 7f50fb552740 1 -- 172.21.15.78:0/3789096776 wait complete. 2024-09-11T01:15:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50fb552740 1 -- start start 2024-09-11T01:15:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50fb552740 1 Processor -- start 2024-09-11T01:15:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50fb552740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d5f9ee0 0x55915d60ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50fb552740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d60b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50fb552740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d606790 0x55915d682570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50f2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d5f9ee0 0x55915d60ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d60b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50f2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d5f9ee0 0x55915d60ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:33804/0 (socket says 172.21.15.78:33804) 2024-09-11T01:15:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d60b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40642/0 (socket says 172.21.15.78:40642) 2024-09-11T01:15:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50f2d9d640 1 -- 172.21.15.78:0/4178966889 learned_addr learned my addr 172.21.15.78:0/4178966889 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50fb552740 1 -- 172.21.15.78:0/4178966889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41b1c60 con 0x55915d5fa800 2024-09-11T01:15:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50fb552740 1 -- 172.21.15.78:0/4178966889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41b14d0 con 0x55915d606790 2024-09-11T01:15:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.886+0000 7f50fb552740 1 -- 172.21.15.78:0/4178966889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41ae730 con 0x55915d5f9ee0 2024-09-11T01:15:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50fb552740 10 client.0 _ll_register_callbacks cb 0x7f50e8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50f359e640 1 --2- 172.21.15.78:0/4178966889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d606790 0x55915d682570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50f259c640 1 -- 172.21.15.78:0/4178966889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d5f9ee0 msgr2=0x55915d60ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50f259c640 1 --2- 172.21.15.78:0/4178966889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d5f9ee0 0x55915d60ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50f259c640 1 -- 172.21.15.78:0/4178966889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d606790 msgr2=0x55915d682570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50f259c640 1 --2- 172.21.15.78:0/4178966889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d606790 0x55915d682570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50f259c640 1 -- 172.21.15.78:0/4178966889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55915d682bb0 con 0x55915d5fa800 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50f259c640 1 --2- 172.21.15.78:0/4178966889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d60b3d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f50e80c9f00 tx=0x7f50e8003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8112eb080017ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50d6ff5640 1 -- 172.21.15.78:0/4178966889 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8049040 con 0x55915d5fa800 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50d6ff5640 1 -- 172.21.15.78:0/4178966889 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8449+0+0 (secure 0 0 0) 0x7f50e80cbef0 con 0x55915d5fa800 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50fb552740 1 -- 172.21.15.78:0/4178966889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e80cc7a0 con 0x55915d5fa800 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50d6ff5640 1 -- 172.21.15.78:0/4178966889 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f50e80cc400 con 0x55915d5fa800 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50fb552740 10 client.18459 fetch_fsmap learned FSMap version 64 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50fb552740 1 -- 172.21.15.78:0/4178966889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55915d682bb0 con 0x55915d5fa800 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.887+0000 7f50d6ff5640 1 -- 172.21.15.78:0/4178966889 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e80cc7a0 con 0x55915d5fa800 2024-09-11T01:15:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.888+0000 7f50d6ff5640 1 -- 172.21.15.78:0/4178966889 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f50e8031ac0 con 0x55915d5fa800 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.888+0000 7f50fb552740 10 client.18459 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.888+0000 7f50fb552740 10 client.18459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.888+0000 7f50fb552740 20 client.18459 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.888+0000 7f50fb552740 1 --2- 172.21.15.78:0/4178966889 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55915d616670 0x55915d6861a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.888+0000 7f50fb552740 4 client.18459 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.888+0000 7f50fb552740 1 -- 172.21.15.78:0/4178966889 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55915d572db0 con 0x55915d616670 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.888+0000 7f50f2d9d640 1 --2- 172.21.15.78:0/4178966889 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55915d616670 0x55915d6861a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.888+0000 7f50f2d9d640 1 --2- 172.21.15.78:0/4178966889 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55915d616670 0x55915d6861a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7316934ed1e9c022 server_cookie=3ca6c7166016d184 in_seq=0 out_seq=0 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.888+0000 7f50d6ff5640 10 client.18459 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.889+0000 7f50d6ff5640 1 -- 172.21.15.78:0/4178966889 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55915d572db0 con 0x55915d616670 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.889+0000 7f50d6ff5640 10 client.18459 handle_command_reply: tid=0 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.889+0000 7f50fb552740 1 client.18459 shutdown 2024-09-11T01:15:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.889+0000 7f50fb552740 1 -- 172.21.15.78:0/4178966889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 msgr2=0x55915d60b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.889+0000 7f50fb552740 1 --2- 172.21.15.78:0/4178966889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d60b3d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f50e80c9f00 tx=0x7f50e8003ee0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.897 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:15:05.897 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:15:05.897 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "85264ca2-fdd8-466e-9591-4583174b621c", 2024-09-11T01:15:05.897 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:15:05.897 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:15:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.889+0000 7f50fb552740 1 -- 172.21.15.78:0/4178966889 shutdown_connections 2024-09-11T01:15:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.889+0000 7f50fb552740 1 --2- 172.21.15.78:0/4178966889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55915d606790 0x55915d682570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.889+0000 7f50fb552740 1 --2- 172.21.15.78:0/4178966889 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55915d616670 0x55915d6861a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50fb552740 1 --2- 172.21.15.78:0/4178966889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55915d5fa800 0x55915d60b3d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50fb552740 1 --2- 172.21.15.78:0/4178966889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55915d5f9ee0 0x55915d60ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50fb552740 1 -- 172.21.15.78:0/4178966889 >> 172.21.15.78:0/4178966889 conn(0x55915d5f7d10 msgr2=0x55915d5f6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50d6ff5640 0 client.18459 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50fb552740 1 -- 172.21.15.78:0/4178966889 shutdown_connections 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50fb552740 1 -- 172.21.15.78:0/4178966889 wait complete. 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50fb552740 20 client.18459 trim_cache size 0 max 16384 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50f917d640 1 -- 172.21.15.78:0/3817158477 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50dc0b5450 msgr2=0x7f50dc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50f917d640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50dc0b5450 0x7f50dc0b7910 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f50f4103a70 tx=0x7f50ec00a4d0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50f917d640 1 -- 172.21.15.78:0/3817158477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50f4104530 msgr2=0x7f50f4101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50f917d640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50f4104530 0x7f50f4101ef0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f50e8006010 tx=0x7f50e80042f0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50f917d640 1 -- 172.21.15.78:0/3817158477 shutdown_connections 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50f917d640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50dc0b5450 0x7f50dc0b7910 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50f917d640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50f410d0e0 0x7f50f4102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50f917d640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50f4104530 0x7f50f4101ef0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50f917d640 1 --2- 172.21.15.78:0/3817158477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50f4103b30 0x7f50f41019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.890+0000 7f50f917d640 1 -- 172.21.15.78:0/3817158477 >> 172.21.15.78:0/3817158477 conn(0x7f50f40fd700 msgr2=0x7f50f40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.891+0000 7f50f917d640 1 -- 172.21.15.78:0/3817158477 shutdown_connections 2024-09-11T01:15:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:05.891+0000 7f50f917d640 1 -- 172.21.15.78:0/3817158477 wait complete. 2024-09-11T01:15:05.904 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:05 smithi078 ceph-mon[31315]: pgmap v827: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 760 op/s 2024-09-11T01:15:06.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:05 smithi110 ceph-mon[34712]: pgmap v827: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 760 op/s 2024-09-11T01:15:06.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:05 smithi112 ceph-mon[35384]: pgmap v827: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 760 op/s 2024-09-11T01:15:06.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 130528 37.80 MB/sec execute 72 sec latency 3896.939 ms 2024-09-11T01:15:06.945 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:15:07.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:06 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:15:07.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:06 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:07.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:06 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:15:07.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:06 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:15:07.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:06 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:15:07.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:06 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:15:07.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:06 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:15:07.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:06 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:07.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:06 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:07.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:06 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:07.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:06 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:15:07.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:06 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:07.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:06 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:15:07.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:06 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:15:07.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:06 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:15:07.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:06 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:15:07.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:06 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:15:07.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:06 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:07.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:06 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:07.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:06 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:07.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:06 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:15:07.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:06 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:07.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:06 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:15:07.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:06 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:15:07.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:06 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:15:07.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:06 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:15:07.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:06 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:15:07.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:06 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:07.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:06 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:07.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:06 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:07.300 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:15:07.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 131029 37.51 MB/sec execute 73 sec latency 143.972 ms 2024-09-11T01:15:08.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:07 smithi110 ceph-mon[34712]: pgmap v828: 97 pgs: 97 active+clean; 1.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 770 op/s 2024-09-11T01:15:08.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:07 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:08.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:07 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:08.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:07 smithi112 ceph-mon[35384]: pgmap v828: 97 pgs: 97 active+clean; 1.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 770 op/s 2024-09-11T01:15:08.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:07 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:08.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:07 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:08.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:07 smithi078 ceph-mon[31315]: pgmap v828: 97 pgs: 97 active+clean; 1.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 770 op/s 2024-09-11T01:15:08.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:07 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:08.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:07 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:15:08.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 131588 37.46 MB/sec execute 74 sec latency 159.634 ms 2024-09-11T01:15:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.909+0000 7f08e84a6640 1 -- 172.21.15.78:0/1774317272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e0103d20 msgr2=0x7f08e0104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.909+0000 7f08e84a6640 1 --2- 172.21.15.78:0/1774317272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e0103d20 0x7f08e0104100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009900 tx=0x7f08dc02a3f0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.909+0000 7f08e84a6640 1 -- 172.21.15.78:0/1774317272 shutdown_connections 2024-09-11T01:15:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.909+0000 7f08e84a6640 1 --2- 172.21.15.78:0/1774317272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e010d180 0x7f08e010f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.909+0000 7f08e84a6640 1 --2- 172.21.15.78:0/1774317272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08e0104640 0x7f08e010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.909+0000 7f08e84a6640 1 --2- 172.21.15.78:0/1774317272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e0103d20 0x7f08e0104100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.909+0000 7f08e84a6640 1 -- 172.21.15.78:0/1774317272 >> 172.21.15.78:0/1774317272 conn(0x7f08e00fd7c0 msgr2=0x7f08e00ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.910+0000 7f08e84a6640 1 -- 172.21.15.78:0/1774317272 shutdown_connections 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.910+0000 7f08e84a6640 1 -- 172.21.15.78:0/1774317272 wait complete. 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e84a6640 1 Processor -- start 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e84a6640 1 -- start start 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e84a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08e0103d20 0x7f08e0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e84a6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e0104640 0x7f08e0199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e84a6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e010d180 0x7f08e019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e84a6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08e0112df0 con 0x7f08e0104640 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e84a6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f08e0112c70 con 0x7f08e010d180 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e84a6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08e0112f70 con 0x7f08e0103d20 2024-09-11T01:15:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e5a41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e0104640 0x7f08e0199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e6a43640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e010d180 0x7f08e019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e5a41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e0104640 0x7f08e0199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40670/0 (socket says 172.21.15.78:40670) 2024-09-11T01:15:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e5a41640 1 -- 172.21.15.78:0/1798292304 learned_addr learned my addr 172.21.15.78:0/1798292304 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.911+0000 7f08e6242640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08e0103d20 0x7f08e0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.912+0000 7f08e6a43640 1 -- 172.21.15.78:0/1798292304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08e0103d20 msgr2=0x7f08e0199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.912+0000 7f08e6a43640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08e0103d20 0x7f08e0199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.912+0000 7f08e6a43640 1 -- 172.21.15.78:0/1798292304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e0104640 msgr2=0x7f08e0199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:08.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.912+0000 7f08e6a43640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e0104640 0x7f08e0199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.912+0000 7f08e6a43640 1 -- 172.21.15.78:0/1798292304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e019f3b0 con 0x7f08e010d180 2024-09-11T01:15:08.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.912+0000 7f08e5a41640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e0104640 0x7f08e0199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:15:08.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.913+0000 7f08e6a43640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e010d180 0x7f08e019ebe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f08d800e940 tx=0x7f08d800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:08.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.915+0000 7f08d77fe640 1 -- 172.21.15.78:0/1798292304 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8004d20 con 0x7f08e010d180 2024-09-11T01:15:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.915+0000 7f08e84a6640 1 -- 172.21.15.78:0/1798292304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e019f6a0 con 0x7f08e010d180 2024-09-11T01:15:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.915+0000 7f08e84a6640 1 -- 172.21.15.78:0/1798292304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e006cb70 con 0x7f08e010d180 2024-09-11T01:15:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.915+0000 7f08d77fe640 1 -- 172.21.15.78:0/1798292304 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08d8005020 con 0x7f08e010d180 2024-09-11T01:15:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.915+0000 7f08d77fe640 1 -- 172.21.15.78:0/1798292304 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d80116d0 con 0x7f08e010d180 2024-09-11T01:15:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.919+0000 7f08ea87b740 1 Processor -- start 2024-09-11T01:15:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.919+0000 7f08d77fe640 1 -- 172.21.15.78:0/1798292304 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f08d800f040 con 0x7f08e010d180 2024-09-11T01:15:08.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.920+0000 7f08ea87b740 1 -- start start 2024-09-11T01:15:08.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08ea87b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cdd6f00 0x56316cdd72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08ea87b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cde6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08ea87b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde71a0 0x56316cdf0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08ea87b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56316cd4fe50 con 0x56316cde6820 2024-09-11T01:15:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08ea87b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56316cd3f280 con 0x56316cdd6f00 2024-09-11T01:15:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08ea87b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56316cd43090 con 0x56316cde71a0 2024-09-11T01:15:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08e5a41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cdd6f00 0x56316cdd72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08e6242640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cde6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08e6242640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cde6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40682/0 (socket says 172.21.15.78:40682) 2024-09-11T01:15:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08e5a41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cdd6f00 0x56316cdd72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:34254/0 (socket says 172.21.15.78:34254) 2024-09-11T01:15:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08e6242640 1 -- 172.21.15.78:0/640625397 learned_addr learned my addr 172.21.15.78:0/640625397 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08e5a41640 1 --2- 172.21.15.78:0/640625397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde71a0 0x56316cdf0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.921+0000 7f08d77fe640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08cc0b5450 0x7f08cc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.922+0000 7f08e6a43640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08cc0b5450 0x7f08cc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.922+0000 7f08e6242640 1 -- 172.21.15.78:0/640625397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde71a0 msgr2=0x56316cdf0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.922+0000 7f08e6242640 1 --2- 172.21.15.78:0/640625397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde71a0 0x56316cdf0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.922+0000 7f08e6242640 1 -- 172.21.15.78:0/640625397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cdd6f00 msgr2=0x56316cdd72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.922+0000 7f08e6242640 1 --2- 172.21.15.78:0/640625397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cdd6f00 0x56316cdd72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.922+0000 7f08e6242640 1 -- 172.21.15.78:0/640625397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56316ca9d650 con 0x56316cde6820 2024-09-11T01:15:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.922+0000 7f08d77fe640 1 -- 172.21.15.78:0/1798292304 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8493+0+0 (secure 0 0 0) 0x7f08d80e5cd0 con 0x7f08e010d180 2024-09-11T01:15:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.922+0000 7f08e6a43640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08cc0b5450 0x7f08cc0b7910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f08d80a86e0 tx=0x7f08d80cf040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.922+0000 7f08e6242640 1 --2- 172.21.15.78:0/640625397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cde6c60 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009900 tx=0x7f08dc032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fcd0c41beb6971 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.923+0000 7f08d4ff9640 1 -- 172.21.15.78:0/640625397 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc03f070 con 0x56316cde6820 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.923+0000 7f08d4ff9640 1 -- 172.21.15.78:0/640625397 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08dc004030 con 0x56316cde6820 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.923+0000 7f08d4ff9640 1 -- 172.21.15.78:0/640625397 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc03f070 con 0x56316cde6820 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.923+0000 7f08ea87b740 1 -- 172.21.15.78:0/640625397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 msgr2=0x56316cde6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.923+0000 7f08ea87b740 1 --2- 172.21.15.78:0/640625397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cde6c60 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009900 tx=0x7f08dc032b90 comp rx=0 tx=0).stop 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.924+0000 7f08ea87b740 1 -- 172.21.15.78:0/640625397 shutdown_connections 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.924+0000 7f08ea87b740 1 --2- 172.21.15.78:0/640625397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde71a0 0x56316cdf0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.924+0000 7f08ea87b740 1 --2- 172.21.15.78:0/640625397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cde6c60 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.924+0000 7f08ea87b740 1 --2- 172.21.15.78:0/640625397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cdd6f00 0x56316cdd72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.924+0000 7f08ea87b740 1 -- 172.21.15.78:0/640625397 >> 172.21.15.78:0/640625397 conn(0x56316cdd9420 msgr2=0x56316cdd9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.924+0000 7f08ea87b740 1 -- 172.21.15.78:0/640625397 shutdown_connections 2024-09-11T01:15:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.924+0000 7f08ea87b740 1 -- 172.21.15.78:0/640625397 wait complete. 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.924+0000 7f08ea87b740 1 -- start start 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.924+0000 7f08ea87b740 1 Processor -- start 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.925+0000 7f08ea87b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cdd6f00 0x56316cdf08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.925+0000 7f08ea87b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cdedcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.925+0000 7f08ea87b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cde71a0 0x56316cdee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.925+0000 7f08ea87b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56316cba0700 con 0x56316cde6820 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.925+0000 7f08ea87b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56316ccda110 con 0x56316cde71a0 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.925+0000 7f08ea87b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56316cdf1220 con 0x56316cdd6f00 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.925+0000 7f08ea87b740 10 client.0 _ll_register_callbacks cb 0x56316ccf3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.925+0000 7f08e5a41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cdd6f00 0x56316cdf08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.925+0000 7f08e5a41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cdd6f00 0x56316cdf08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:33860/0 (socket says 172.21.15.78:33860) 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.925+0000 7f08e5a41640 1 -- 172.21.15.78:0/2254407025 learned_addr learned my addr 172.21.15.78:0/2254407025 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.925+0000 7f08e5a41640 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cde71a0 0x56316cdee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.925+0000 7f08e6242640 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cdedcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.926+0000 7f08e5a41640 1 -- 172.21.15.78:0/2254407025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cdd6f00 msgr2=0x56316cdf08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:08.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.926+0000 7f08e5a41640 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cdd6f00 0x56316cdf08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.926+0000 7f08e5a41640 1 -- 172.21.15.78:0/2254407025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 msgr2=0x56316cdedcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:08.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.926+0000 7f08e5a41640 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cdedcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.926+0000 7f08e5a41640 1 -- 172.21.15.78:0/2254407025 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56316cd07c40 con 0x56316cde71a0 2024-09-11T01:15:08.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.926+0000 7f08e5a41640 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cde71a0 0x56316cdee230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f08d0012dd0 tx=0x7f08d0009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d69b2ae36fe68c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:08.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.926+0000 7f08c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:15:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.927+0000 7f08c9ffb640 1 -- 172.21.15.78:0/2254407025 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0026060 con 0x56316cde71a0 2024-09-11T01:15:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.927+0000 7f08c9ffb640 1 -- 172.21.15.78:0/2254407025 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8493+0+0 (secure 0 0 0) 0x7f08d001a040 con 0x56316cde71a0 2024-09-11T01:15:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.927+0000 7f08ea87b740 1 -- 172.21.15.78:0/2254407025 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08dc004030 con 0x56316cde71a0 2024-09-11T01:15:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.927+0000 7f08c9ffb640 1 -- 172.21.15.78:0/2254407025 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f08d0004030 con 0x56316cde71a0 2024-09-11T01:15:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.927+0000 7f08ea87b740 10 client.27599 fetch_fsmap learned FSMap version 64 2024-09-11T01:15:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.927+0000 7f08ea87b740 1 -- 172.21.15.78:0/2254407025 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56316cdfcaf0 con 0x56316cde71a0 2024-09-11T01:15:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.928+0000 7f08c9ffb640 1 -- 172.21.15.78:0/2254407025 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08d001d8e0 con 0x56316cde71a0 2024-09-11T01:15:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.928+0000 7f08c9ffb640 1 -- 172.21.15.78:0/2254407025 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f08d0003680 con 0x56316cde71a0 2024-09-11T01:15:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.928+0000 7f08ea87b740 10 client.27599 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.928+0000 7f08ea87b740 10 client.27599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.928+0000 7f08ea87b740 20 client.27599 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.928+0000 7f08ea87b740 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316cdfe080 0x56316ce004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.928+0000 7f08ea87b740 4 client.27599 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.928+0000 7f08ea87b740 1 -- 172.21.15.78:0/2254407025 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f08dc004030 con 0x56316cdfe080 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.928+0000 7f08e6242640 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316cdfe080 0x56316ce004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.929+0000 7f08e6242640 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316cdfe080 0x56316ce004a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b2e4fb16e2c3186 server_cookie=46c32a92d3349507 in_seq=0 out_seq=0 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.929+0000 7f08c9ffb640 10 client.27599 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.930+0000 7f08c9ffb640 1 -- 172.21.15.78:0/2254407025 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f08dc004030 con 0x56316cdfe080 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.930+0000 7f08c9ffb640 10 client.27599 handle_command_reply: tid=0 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.930+0000 7f08ea87b740 1 client.27599 shutdown 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.930+0000 7f08ea87b740 1 -- 172.21.15.78:0/2254407025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cde71a0 msgr2=0x56316cdee230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.930+0000 7f08ea87b740 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cde71a0 0x56316cdee230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f08d0012dd0 tx=0x7f08d0009550 comp rx=0 tx=0).stop 2024-09-11T01:15:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.930+0000 7f08ea87b740 1 -- 172.21.15.78:0/2254407025 shutdown_connections 2024-09-11T01:15:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.930+0000 7f08ea87b740 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cde71a0 0x56316cdee230 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.930+0000 7f08ea87b740 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316cdfe080 0x56316ce004a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.930+0000 7f08ea87b740 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cdedcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.931+0000 7f08ea87b740 1 --2- 172.21.15.78:0/2254407025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cdd6f00 0x56316cdf08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.931+0000 7f08ea87b740 1 -- 172.21.15.78:0/2254407025 >> 172.21.15.78:0/2254407025 conn(0x56316cdd9420 msgr2=0x56316cde9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.931+0000 7f08c9ffb640 0 client.27599 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.931+0000 7f08ea87b740 1 -- 172.21.15.78:0/2254407025 shutdown_connections 2024-09-11T01:15:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.931+0000 7f08ea87b740 1 -- 172.21.15.78:0/2254407025 wait complete. 2024-09-11T01:15:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.931+0000 7f08ea87b740 20 client.27599 trim_cache size 0 max 16384 2024-09-11T01:15:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08ea87b740 1 Processor -- start 2024-09-11T01:15:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08ea87b740 1 -- start start 2024-09-11T01:15:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08ea87b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cdd6f00 0x56316cdeb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08ea87b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde6820 0x56316ce002b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08ea87b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cde71a0 0x56316cdec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08ea87b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08c400e740 con 0x56316cdd6f00 2024-09-11T01:15:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08ea87b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4010400 con 0x56316cde71a0 2024-09-11T01:15:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08ea87b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08c400ee10 con 0x56316cde6820 2024-09-11T01:15:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08e5a41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cdd6f00 0x56316cdeb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08e5a41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cdd6f00 0x56316cdeb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40692/0 (socket says 172.21.15.78:40692) 2024-09-11T01:15:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08e5a41640 1 -- 172.21.15.78:0/1992474216 learned_addr learned my addr 172.21.15.78:0/1992474216 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08e6242640 1 --2- 172.21.15.78:0/1992474216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde6820 0x56316ce002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08e5a41640 1 --2- 172.21.15.78:0/1992474216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cde71a0 0x56316cdec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08e5a41640 1 -- 172.21.15.78:0/1992474216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde6820 msgr2=0x56316ce002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.959+0000 7f08e5a41640 1 --2- 172.21.15.78:0/1992474216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde6820 0x56316ce002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.960+0000 7f08e5a41640 1 -- 172.21.15.78:0/1992474216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cde71a0 msgr2=0x56316cdec570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.960+0000 7f08e5a41640 1 --2- 172.21.15.78:0/1992474216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cde71a0 0x56316cdec570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.960+0000 7f08e5a41640 1 -- 172.21.15.78:0/1992474216 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56316cdf0c10 con 0x56316cdd6f00 2024-09-11T01:15:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.960+0000 7f08e5a41640 1 --2- 172.21.15.78:0/1992474216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cdd6f00 0x56316cdeb8b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009380 tx=0x7f08d0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e67b8225d374f80 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.960+0000 7f08cbfff640 1 -- 172.21.15.78:0/1992474216 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d000f880 con 0x56316cdd6f00 2024-09-11T01:15:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.960+0000 7f08cbfff640 1 -- 172.21.15.78:0/1992474216 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08d0003c40 con 0x56316cdd6f00 2024-09-11T01:15:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.960+0000 7f08cbfff640 1 -- 172.21.15.78:0/1992474216 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0011020 con 0x56316cdd6f00 2024-09-11T01:15:08.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.961+0000 7f08ea87b740 1 -- 172.21.15.78:0/1992474216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cdd6f00 msgr2=0x56316cdeb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.961+0000 7f08ea87b740 1 --2- 172.21.15.78:0/1992474216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cdd6f00 0x56316cdeb8b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009380 tx=0x7f08d0009310 comp rx=0 tx=0).stop 2024-09-11T01:15:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.961+0000 7f08ea87b740 1 -- 172.21.15.78:0/1992474216 shutdown_connections 2024-09-11T01:15:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.961+0000 7f08ea87b740 1 --2- 172.21.15.78:0/1992474216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cde71a0 0x56316cdec570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.961+0000 7f08ea87b740 1 --2- 172.21.15.78:0/1992474216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde6820 0x56316ce002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.961+0000 7f08ea87b740 1 --2- 172.21.15.78:0/1992474216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cdd6f00 0x56316cdeb8b0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.961+0000 7f08ea87b740 1 -- 172.21.15.78:0/1992474216 >> 172.21.15.78:0/1992474216 conn(0x56316cdd9420 msgr2=0x56316cde7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.961+0000 7f08ea87b740 1 -- 172.21.15.78:0/1992474216 shutdown_connections 2024-09-11T01:15:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.961+0000 7f08ea87b740 1 -- 172.21.15.78:0/1992474216 wait complete. 2024-09-11T01:15:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.961+0000 7f08ea87b740 1 -- start start 2024-09-11T01:15:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.961+0000 7f08ea87b740 1 Processor -- start 2024-09-11T01:15:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08ea87b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cdd6f00 0x56316cdf6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08ea87b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cdf6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08ea87b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde71a0 0x56316ce04da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08ea87b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08c400fc70 con 0x56316cde6820 2024-09-11T01:15:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08ea87b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f08c400f4e0 con 0x56316cdd6f00 2024-09-11T01:15:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08ea87b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08c400c740 con 0x56316cde71a0 2024-09-11T01:15:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08ea87b740 10 client.0 _ll_register_callbacks cb 0x7f08d00129a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08e6242640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cdf6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08e6242640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cdf6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40702/0 (socket says 172.21.15.78:40702) 2024-09-11T01:15:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08e6242640 1 -- 172.21.15.78:0/2027067958 learned_addr learned my addr 172.21.15.78:0/2027067958 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08e5a41640 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cdd6f00 0x56316cdf6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08e5a41640 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde71a0 0x56316ce04da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08e6242640 1 -- 172.21.15.78:0/2027067958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde71a0 msgr2=0x56316ce04da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:08.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08e6242640 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde71a0 0x56316ce04da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08e6242640 1 -- 172.21.15.78:0/2027067958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cdd6f00 msgr2=0x56316cdf6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:08.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08e6242640 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cdd6f00 0x56316cdf6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08e6242640 1 -- 172.21.15.78:0/2027067958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56316ce052e0 con 0x56316cde6820 2024-09-11T01:15:08.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.962+0000 7f08e5a41640 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cdd6f00 0x56316cdf6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:15:08.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.963+0000 7f08e6242640 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cdf6a40 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009850 tx=0x7f08dc02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb84751d9c66279 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:08.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.963+0000 7f08c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:15:08.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.963+0000 7f08c9ffb640 1 -- 172.21.15.78:0/2027067958 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc03f070 con 0x56316cde6820 2024-09-11T01:15:08.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.963+0000 7f08c9ffb640 1 -- 172.21.15.78:0/2027067958 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8493+0+0 (secure 0 0 0) 0x7f08dc031b10 con 0x56316cde6820 2024-09-11T01:15:08.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.963+0000 7f08ea87b740 1 -- 172.21.15.78:0/2027067958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d0003c40 con 0x56316cde6820 2024-09-11T01:15:08.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.963+0000 7f08c9ffb640 1 -- 172.21.15.78:0/2027067958 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f08dc035070 con 0x56316cde6820 2024-09-11T01:15:08.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.963+0000 7f08ea87b740 10 client.18477 fetch_fsmap learned FSMap version 64 2024-09-11T01:15:08.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.963+0000 7f08ea87b740 1 -- 172.21.15.78:0/2027067958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56316ce06590 con 0x56316cde6820 2024-09-11T01:15:08.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.963+0000 7f08c9ffb640 1 -- 172.21.15.78:0/2027067958 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08d0003c40 con 0x56316cde6820 2024-09-11T01:15:08.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.964+0000 7f08c9ffb640 1 -- 172.21.15.78:0/2027067958 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) ==== 8410+0+0 (secure 0 0 0) 0x7f08dc0046c0 con 0x56316cde6820 2024-09-11T01:15:08.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.964+0000 7f08ea87b740 10 client.18477 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T01:15:08.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.964+0000 7f08ea87b740 10 client.18477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:15:08.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.964+0000 7f08ea87b740 20 client.18477 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:08.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.964+0000 7f08ea87b740 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316ce6bfc0 0x56316ce6e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:08.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.964+0000 7f08ea87b740 4 client.18477 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:15:08.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.964+0000 7f08ea87b740 1 -- 172.21.15.78:0/2027067958 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56316cd51da0 con 0x56316ce6bfc0 2024-09-11T01:15:08.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.964+0000 7f08e5a41640 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316ce6bfc0 0x56316ce6e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:08.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.964+0000 7f08c9ffb640 10 client.18477 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:08.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.965+0000 7f08e5a41640 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316ce6bfc0 0x56316ce6e3b0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=107fb884205734bb server_cookie=b0ab71abccbca221 in_seq=0 out_seq=0 2024-09-11T01:15:08.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.965+0000 7f08c9ffb640 1 -- 172.21.15.78:0/2027067958 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6250 (crc 0 0 0) 0x56316cd51da0 con 0x56316ce6bfc0 2024-09-11T01:15:08.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.965+0000 7f08c9ffb640 10 client.18477 handle_command_reply: tid=0 2024-09-11T01:15:08.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08ea87b740 1 client.18477 shutdown 2024-09-11T01:15:08.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08ea87b740 1 -- 172.21.15.78:0/2027067958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 msgr2=0x56316cdf6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:08.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08ea87b740 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cdf6a40 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009850 tx=0x7f08dc02a3f0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08ea87b740 1 -- 172.21.15.78:0/2027067958 shutdown_connections 2024-09-11T01:15:08.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08ea87b740 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56316cde71a0 0x56316ce04da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08ea87b740 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56316ce6bfc0 0x56316ce6e3b0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08ea87b740 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56316cde6820 0x56316cdf6a40 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08ea87b740 1 --2- 172.21.15.78:0/2027067958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56316cdd6f00 0x56316cdf6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08ea87b740 1 -- 172.21.15.78:0/2027067958 >> 172.21.15.78:0/2027067958 conn(0x56316cdd9420 msgr2=0x56316cdedaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:08.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08c9ffb640 0 client.18477 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:08.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08ea87b740 1 -- 172.21.15.78:0/2027067958 shutdown_connections 2024-09-11T01:15:08.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08ea87b740 1 -- 172.21.15.78:0/2027067958 wait complete. 2024-09-11T01:15:08.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.966+0000 7f08ea87b740 20 client.18477 trim_cache size 0 max 16384 2024-09-11T01:15:08.974 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:15:08.974 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:15:08.974 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:15:08.974 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:15:08.974 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.974 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "85264ca2-fdd8-466e-9591-4583174b621c", 2024-09-11T01:15:08.974 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:15:08.974 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:15:08.975 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.977 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:15:08.978 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.980 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:15:08.981 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.983 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.984 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.984 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:15:08.984 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:15:08.984 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.984 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.984 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.984 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.984 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.984 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:15:08.984 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.986 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.987 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:08.987 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:15:08.987 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:15:08.987 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:08.987 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:08.987 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:08.987 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:08.987 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:08.987 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:15:08.987 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:15:08.987 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:15:08.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.967+0000 7f08e84a6640 1 -- 172.21.15.78:0/1798292304 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08cc0b5450 msgr2=0x7f08cc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:08.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.967+0000 7f08e84a6640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08cc0b5450 0x7f08cc0b7910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f08d80a86e0 tx=0x7f08d80cf040 comp rx=0 tx=0).stop 2024-09-11T01:15:08.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.967+0000 7f08e84a6640 1 -- 172.21.15.78:0/1798292304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e010d180 msgr2=0x7f08e019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:08.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.967+0000 7f08e84a6640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e010d180 0x7f08e019ebe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f08d800e940 tx=0x7f08d800ee00 comp rx=0 tx=0).stop 2024-09-11T01:15:08.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.967+0000 7f08e84a6640 1 -- 172.21.15.78:0/1798292304 shutdown_connections 2024-09-11T01:15:08.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.967+0000 7f08e84a6640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08cc0b5450 0x7f08cc0b7910 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.967+0000 7f08e84a6640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e010d180 0x7f08e019ebe0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.967+0000 7f08e84a6640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e0104640 0x7f08e0199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.967+0000 7f08e84a6640 1 --2- 172.21.15.78:0/1798292304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08e0103d20 0x7f08e0199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:08.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.967+0000 7f08e84a6640 1 -- 172.21.15.78:0/1798292304 >> 172.21.15.78:0/1798292304 conn(0x7f08e00fd7c0 msgr2=0x7f08e00fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:08.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.967+0000 7f08e84a6640 1 -- 172.21.15.78:0/1798292304 shutdown_connections 2024-09-11T01:15:08.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:08.967+0000 7f08e84a6640 1 -- 172.21.15.78:0/1798292304 wait complete. 2024-09-11T01:15:09.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:08 smithi110 ceph-mon[34712]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T01:15:09.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:08 smithi112 ceph-mon[35384]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T01:15:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:08 smithi078 ceph-mon[31315]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T01:15:09.749 INFO:tasks.cephfs.filesystem:scrub status for tag:85264ca2-fdd8-466e-9591-4583174b621c - {'path': '/', 'tag': '85264ca2-fdd8-466e-9591-4583174b621c', 'options': 'recursive,force'} 2024-09-11T01:15:09.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 132299 37.49 MB/sec execute 75 sec latency 156.749 ms 2024-09-11T01:15:09.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:09 smithi078 ceph-mon[31315]: pgmap v830: 97 pgs: 97 active+clean; 1.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 15 MiB/s wr, 700 op/s 2024-09-11T01:15:10.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:09 smithi110 ceph-mon[34712]: pgmap v830: 97 pgs: 97 active+clean; 1.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 15 MiB/s wr, 700 op/s 2024-09-11T01:15:10.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:09 smithi112 ceph-mon[35384]: pgmap v830: 97 pgs: 97 active+clean; 1.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 15 MiB/s wr, 700 op/s 2024-09-11T01:15:10.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 133136 37.47 MB/sec execute 76 sec latency 66.550 ms 2024-09-11T01:15:11.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 133975 37.65 MB/sec execute 77 sec latency 530.167 ms 2024-09-11T01:15:12.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:11 smithi110 ceph-mon[34712]: pgmap v831: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 641 op/s 2024-09-11T01:15:12.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:11 smithi112 ceph-mon[35384]: pgmap v831: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 641 op/s 2024-09-11T01:15:12.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:11 smithi078 ceph-mon[31315]: pgmap v831: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 641 op/s 2024-09-11T01:15:12.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 134770 37.80 MB/sec execute 78 sec latency 1530.200 ms 2024-09-11T01:15:13.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 135506 37.75 MB/sec execute 79 sec latency 2530.234 ms 2024-09-11T01:15:14.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:13 smithi110 ceph-mon[34712]: pgmap v832: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.3 MiB/s wr, 659 op/s 2024-09-11T01:15:14.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:13 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:14.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:13 smithi110 ceph-mon[34712]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T01:15:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:13 smithi112 ceph-mon[35384]: pgmap v832: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.3 MiB/s wr, 659 op/s 2024-09-11T01:15:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:13 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:13 smithi112 ceph-mon[35384]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T01:15:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:13 smithi078 ceph-mon[31315]: pgmap v832: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.3 MiB/s wr, 659 op/s 2024-09-11T01:15:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:13 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:13 smithi078 ceph-mon[31315]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T01:15:14.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 136226 37.78 MB/sec execute 80 sec latency 3530.279 ms 2024-09-11T01:15:15.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 136700 37.69 MB/sec execute 81 sec latency 4530.323 ms 2024-09-11T01:15:16.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:15 smithi110 ceph-mon[34712]: pgmap v834: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 640 op/s 2024-09-11T01:15:16.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:15 smithi112 ceph-mon[35384]: pgmap v834: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 640 op/s 2024-09-11T01:15:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:15 smithi078 ceph-mon[31315]: pgmap v834: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 640 op/s 2024-09-11T01:15:16.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 136960 37.39 MB/sec execute 82 sec latency 5530.356 ms 2024-09-11T01:15:17.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 137256 37.10 MB/sec execute 83 sec latency 5709.831 ms 2024-09-11T01:15:18.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:17 smithi110 ceph-mon[34712]: pgmap v835: 97 pgs: 97 active+clean; 1.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 10 MiB/s wr, 786 op/s 2024-09-11T01:15:18.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:17 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:15:18.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:17 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:15:18.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:17 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:15:18.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:17 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:15:18.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:17 smithi112 ceph-mon[35384]: pgmap v835: 97 pgs: 97 active+clean; 1.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 10 MiB/s wr, 786 op/s 2024-09-11T01:15:18.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:17 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:15:18.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:17 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:15:18.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:17 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:15:18.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:17 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:15:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:17 smithi078 ceph-mon[31315]: pgmap v835: 97 pgs: 97 active+clean; 1.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 10 MiB/s wr, 786 op/s 2024-09-11T01:15:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:17 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:15:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:17 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:15:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:17 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:15:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:17 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:15:18.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 137653 36.92 MB/sec execute 84 sec latency 6709.859 ms 2024-09-11T01:15:19.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:18 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:15:19.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:18 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:18 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:15:19.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:18 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:18 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:15:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:18 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:19.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 138010 36.73 MB/sec execute 85 sec latency 7709.891 ms 2024-09-11T01:15:20.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:19 smithi112 ceph-mon[35384]: pgmap v836: 97 pgs: 97 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.5 MiB/s wr, 656 op/s 2024-09-11T01:15:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:19 smithi078 ceph-mon[31315]: pgmap v836: 97 pgs: 97 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.5 MiB/s wr, 656 op/s 2024-09-11T01:15:20.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:19 smithi110 ceph-mon[34712]: pgmap v836: 97 pgs: 97 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.5 MiB/s wr, 656 op/s 2024-09-11T01:15:20.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 138437 36.56 MB/sec execute 86 sec latency 8709.931 ms 2024-09-11T01:15:21.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 138843 36.36 MB/sec execute 87 sec latency 9709.963 ms 2024-09-11T01:15:22.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:21 smithi112 ceph-mon[35384]: pgmap v837: 97 pgs: 97 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.7 MiB/s wr, 583 op/s 2024-09-11T01:15:22.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:21 smithi078 ceph-mon[31315]: pgmap v837: 97 pgs: 97 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.7 MiB/s wr, 583 op/s 2024-09-11T01:15:22.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:21 smithi110 ceph-mon[34712]: pgmap v837: 97 pgs: 97 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.7 MiB/s wr, 583 op/s 2024-09-11T01:15:22.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 139459 36.30 MB/sec execute 88 sec latency 10268.049 ms 2024-09-11T01:15:23.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 140218 36.28 MB/sec execute 89 sec latency 10617.326 ms 2024-09-11T01:15:24.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:23 smithi112 ceph-mon[35384]: pgmap v838: 97 pgs: 97 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 480 op/s 2024-09-11T01:15:24.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:23 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:24.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:23 smithi078 ceph-mon[31315]: pgmap v838: 97 pgs: 97 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 480 op/s 2024-09-11T01:15:24.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:23 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:24.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:23 smithi110 ceph-mon[34712]: pgmap v838: 97 pgs: 97 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 480 op/s 2024-09-11T01:15:24.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:23 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:24.785 INFO:tasks.workunit.client.0.smithi078.stdout: 10 141100 36.47 MB/sec execute 90 sec latency 11442.825 ms 2024-09-11T01:15:25.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 141912 36.63 MB/sec execute 91 sec latency 74.536 ms 2024-09-11T01:15:26.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:25 smithi112 ceph-mon[35384]: pgmap v839: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.7 MiB/s wr, 432 op/s 2024-09-11T01:15:26.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:25 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:25 smithi078 ceph-mon[31315]: pgmap v839: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.7 MiB/s wr, 432 op/s 2024-09-11T01:15:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:25 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:25 smithi110 ceph-mon[34712]: pgmap v839: 97 pgs: 97 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.7 MiB/s wr, 432 op/s 2024-09-11T01:15:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:25 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:26.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 142599 36.55 MB/sec execute 92 sec latency 119.317 ms 2024-09-11T01:15:27.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 143402 36.65 MB/sec execute 93 sec latency 83.498 ms 2024-09-11T01:15:28.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:27 smithi112 ceph-mon[35384]: pgmap v840: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 535 op/s 2024-09-11T01:15:28.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:27 smithi078 ceph-mon[31315]: pgmap v840: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 535 op/s 2024-09-11T01:15:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:27 smithi110 ceph-mon[34712]: pgmap v840: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 535 op/s 2024-09-11T01:15:28.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 144264 36.71 MB/sec execute 94 sec latency 59.930 ms 2024-09-11T01:15:29.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 144929 36.73 MB/sec execute 95 sec latency 263.869 ms 2024-09-11T01:15:30.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:29 smithi112 ceph-mon[35384]: pgmap v841: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.4 MiB/s wr, 486 op/s 2024-09-11T01:15:30.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:29 smithi078 ceph-mon[31315]: pgmap v841: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.4 MiB/s wr, 486 op/s 2024-09-11T01:15:30.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:29 smithi110 ceph-mon[34712]: pgmap v841: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.4 MiB/s wr, 486 op/s 2024-09-11T01:15:30.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 145023 36.42 MB/sec execute 96 sec latency 807.087 ms 2024-09-11T01:15:31.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:15:31.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T01:15:31.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T01:15:31.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T01:15:31.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T01:15:31.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:15:31.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:15:31.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:30 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:15:31.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:15:31.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T01:15:31.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T01:15:31.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T01:15:31.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T01:15:31.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:15:31.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:15:31.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:30 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:15:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:15:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T01:15:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T01:15:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T01:15:31.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T01:15:31.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:15:31.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:15:31.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:30 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:15:31.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 145430 36.29 MB/sec execute 97 sec latency 126.299 ms 2024-09-11T01:15:32.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:31 smithi078 ceph-mon[31315]: pgmap v842: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-09-11T01:15:32.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:31 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-11T01:15:32.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:31 smithi078 ceph-mon[31315]: Unable to set osd_memory_target on smithi078 to 184555929: error parsing value: Value '184555929' is below minimum 939524096 2024-09-11T01:15:32.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:31 smithi110 ceph-mon[34712]: pgmap v842: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-09-11T01:15:32.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:31 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-11T01:15:32.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:32 smithi110 ceph-mon[34712]: Unable to set osd_memory_target on smithi078 to 184555929: error parsing value: Value '184555929' is below minimum 939524096 2024-09-11T01:15:32.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:31 smithi112 ceph-mon[35384]: pgmap v842: 97 pgs: 97 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-09-11T01:15:32.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:31 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-11T01:15:32.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:32 smithi112 ceph-mon[35384]: Unable to set osd_memory_target on smithi078 to 184555929: error parsing value: Value '184555929' is below minimum 939524096 2024-09-11T01:15:32.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 146258 36.31 MB/sec execute 98 sec latency 58.413 ms 2024-09-11T01:15:33.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 146999 36.43 MB/sec execute 99 sec latency 177.873 ms 2024-09-11T01:15:34.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:33 smithi078 ceph-mon[31315]: pgmap v843: 97 pgs: 97 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 655 op/s 2024-09-11T01:15:34.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:33 smithi110 ceph-mon[34712]: pgmap v843: 97 pgs: 97 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 655 op/s 2024-09-11T01:15:34.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:33 smithi112 ceph-mon[35384]: pgmap v843: 97 pgs: 97 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 655 op/s 2024-09-11T01:15:34.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 147644 36.32 MB/sec execute 100 sec latency 154.418 ms 2024-09-11T01:15:35.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 148289 36.35 MB/sec execute 101 sec latency 198.230 ms 2024-09-11T01:15:36.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:35 smithi078 ceph-mon[31315]: pgmap v844: 97 pgs: 97 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 752 op/s 2024-09-11T01:15:36.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:35 smithi110 ceph-mon[34712]: pgmap v844: 97 pgs: 97 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 752 op/s 2024-09-11T01:15:36.597 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:35 smithi112 ceph-mon[35384]: pgmap v844: 97 pgs: 97 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 752 op/s 2024-09-11T01:15:36.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 149101 36.34 MB/sec execute 102 sec latency 79.198 ms 2024-09-11T01:15:37.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 149942 36.52 MB/sec execute 103 sec latency 65.808 ms 2024-09-11T01:15:38.242 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:37 smithi110 ceph-mon[34712]: pgmap v845: 97 pgs: 97 active+clean; 2.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 21 MiB/s wr, 971 op/s 2024-09-11T01:15:38.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:37 smithi078 ceph-mon[31315]: pgmap v845: 97 pgs: 97 active+clean; 2.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 21 MiB/s wr, 971 op/s 2024-09-11T01:15:38.276 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:37 smithi112 ceph-mon[35384]: pgmap v845: 97 pgs: 97 active+clean; 2.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 21 MiB/s wr, 971 op/s 2024-09-11T01:15:38.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 150679 36.56 MB/sec execute 104 sec latency 90.737 ms 2024-09-11T01:15:39.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:38 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:39.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:38 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:39.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:38 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:39.750 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:15:39.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 151420 36.59 MB/sec execute 105 sec latency 57.640 ms 2024-09-11T01:15:40.129 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:15:40.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:39 smithi078 ceph-mon[31315]: pgmap v846: 97 pgs: 97 active+clean; 2.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 855 op/s 2024-09-11T01:15:40.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:39 smithi110 ceph-mon[34712]: pgmap v846: 97 pgs: 97 active+clean; 2.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 855 op/s 2024-09-11T01:15:40.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:39 smithi112 ceph-mon[35384]: pgmap v846: 97 pgs: 97 active+clean; 2.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 855 op/s 2024-09-11T01:15:40.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.679+0000 7f33b45df640 1 -- 172.21.15.78:0/177593636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac101830 msgr2=0x7f33ac101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:40.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.679+0000 7f33b45df640 1 --2- 172.21.15.78:0/177593636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac101830 0x7f33ac101c10 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009900 tx=0x7f33a802a410 comp rx=0 tx=0).stop 2024-09-11T01:15:40.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.680+0000 7f33b45df640 1 -- 172.21.15.78:0/177593636 shutdown_connections 2024-09-11T01:15:40.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.680+0000 7f33b45df640 1 --2- 172.21.15.78:0/177593636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac10d0f0 0x7f33ac10f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.680+0000 7f33b45df640 1 --2- 172.21.15.78:0/177593636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac102230 0x7f33ac10c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.680+0000 7f33b45df640 1 --2- 172.21.15.78:0/177593636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac101830 0x7f33ac101c10 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.680+0000 7f33b45df640 1 -- 172.21.15.78:0/177593636 >> 172.21.15.78:0/177593636 conn(0x7f33ac076e20 msgr2=0x7f33ac077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:40.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.680+0000 7f33b45df640 1 -- 172.21.15.78:0/177593636 shutdown_connections 2024-09-11T01:15:40.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.680+0000 7f33b45df640 1 -- 172.21.15.78:0/177593636 wait complete. 2024-09-11T01:15:40.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.681+0000 7f33b45df640 1 Processor -- start 2024-09-11T01:15:40.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.681+0000 7f33b45df640 1 -- start start 2024-09-11T01:15:40.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.681+0000 7f33b45df640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac101830 0x7f33ac1028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.681+0000 7f33b45df640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102230 0x7f33ac102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.681+0000 7f33b45df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac10d0f0 0x7f33ac103330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.681+0000 7f33b45df640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac112da0 con 0x7f33ac102230 2024-09-11T01:15:40.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.681+0000 7f33b45df640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac112c20 con 0x7f33ac101830 2024-09-11T01:15:40.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.681+0000 7f33b45df640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac112f20 con 0x7f33ac10d0f0 2024-09-11T01:15:40.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.681+0000 7f33b1b7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102230 0x7f33ac102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.682+0000 7f33b1b7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102230 0x7f33ac102df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41912/0 (socket says 172.21.15.78:41912) 2024-09-11T01:15:40.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.682+0000 7f33b1b7a640 1 -- 172.21.15.78:0/1656871354 learned_addr learned my addr 172.21.15.78:0/1656871354 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:40.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.682+0000 7f33b237b640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac101830 0x7f33ac1028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.682+0000 7f33b2b7c640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac10d0f0 0x7f33ac103330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.682+0000 7f33b1b7a640 1 -- 172.21.15.78:0/1656871354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac10d0f0 msgr2=0x7f33ac103330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:40.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.682+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac10d0f0 0x7f33ac103330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.682+0000 7f33b1b7a640 1 -- 172.21.15.78:0/1656871354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac101830 msgr2=0x7f33ac1028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:40.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.682+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac101830 0x7f33ac1028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.682+0000 7f33b1b7a640 1 -- 172.21.15.78:0/1656871354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac073c60 con 0x7f33ac102230 2024-09-11T01:15:40.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.682+0000 7f33b2b7c640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac10d0f0 0x7f33ac103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:15:40.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.682+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102230 0x7f33ac102df0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f339c00e9a0 tx=0x7f339c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:40.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.683+0000 7f33a37fe640 1 -- 172.21.15.78:0/1656871354 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c00cd80 con 0x7f33ac102230 2024-09-11T01:15:40.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.683+0000 7f33a37fe640 1 -- 172.21.15.78:0/1656871354 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f339c0035f0 con 0x7f33ac102230 2024-09-11T01:15:40.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.683+0000 7f33b45df640 1 -- 172.21.15.78:0/1656871354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ac073f50 con 0x7f33ac102230 2024-09-11T01:15:40.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.683+0000 7f33a37fe640 1 -- 172.21.15.78:0/1656871354 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c010690 con 0x7f33ac102230 2024-09-11T01:15:40.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.683+0000 7f33b45df640 1 -- 172.21.15.78:0/1656871354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ac0744e0 con 0x7f33ac102230 2024-09-11T01:15:40.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.687+0000 7f33a37fe640 1 -- 172.21.15.78:0/1656871354 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f339c004000 con 0x7f33ac102230 2024-09-11T01:15:40.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.687+0000 7f33b69b4740 1 Processor -- start 2024-09-11T01:15:40.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.688+0000 7f33a37fe640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f33980b54d0 0x7f33980b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.688+0000 7f33b69b4740 1 -- start start 2024-09-11T01:15:40.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.688+0000 7f33b69b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 0x55de96914860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.688+0000 7f33b69b4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de969207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.688+0000 7f33a37fe640 1 -- 172.21.15.78:0/1656871354 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f339c0e62f0 con 0x7f33ac102230 2024-09-11T01:15:40.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.688+0000 7f33b69b4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96920d30 0x55de96923120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.688+0000 7f33b69b4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de9688ae50 con 0x55de96920d30 2024-09-11T01:15:40.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.688+0000 7f33b69b4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55de9687a280 con 0x55de96914da0 2024-09-11T01:15:40.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.688+0000 7f33b69b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55de9687e090 con 0x55de96914480 2024-09-11T01:15:40.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.688+0000 7f33b2b7c640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f33980b54d0 0x7f33980b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.688+0000 7f33b237b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 0x55de96914860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.689+0000 7f33b237b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 0x55de96914860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:43784/0 (socket says 172.21.15.78:43784) 2024-09-11T01:15:40.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.689+0000 7f33b1b7a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de969207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.689+0000 7f33b237b640 1 -- 172.21.15.78:0/4106811554 learned_addr learned my addr 172.21.15.78:0/4106811554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:40.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.689+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/4106811554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de969207f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:33678/0 (socket says 172.21.15.78:33678) 2024-09-11T01:15:40.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.689+0000 7f33b2b7c640 1 --2- 172.21.15.78:0/4106811554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96920d30 0x55de96923120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.689+0000 7f33b237b640 1 -- 172.21.15.78:0/4106811554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 msgr2=0x55de969207f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:40.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.689+0000 7f33b237b640 1 --2- 172.21.15.78:0/4106811554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de969207f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.689+0000 7f33b237b640 1 -- 172.21.15.78:0/4106811554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96920d30 msgr2=0x55de96923120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:40.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.689+0000 7f33b237b640 1 --2- 172.21.15.78:0/4106811554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96920d30 0x55de96923120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.689+0000 7f33b237b640 1 -- 172.21.15.78:0/4106811554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de965d8650 con 0x55de96914480 2024-09-11T01:15:40.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.689+0000 7f33b237b640 1 --2- 172.21.15.78:0/4106811554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 0x55de96914860 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009900 tx=0x7f33a8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed8f4338faff3a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:40.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.690+0000 7f33a0ff9640 1 -- 172.21.15.78:0/4106811554 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8035030 con 0x55de96914480 2024-09-11T01:15:40.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.690+0000 7f33b2b7c640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f33980b54d0 0x7f33980b7990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005c40 tx=0x7f33a4009e70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.690+0000 7f33a0ff9640 1 -- 172.21.15.78:0/4106811554 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a80054f0 con 0x55de96914480 2024-09-11T01:15:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.690+0000 7f33a0ff9640 1 -- 172.21.15.78:0/4106811554 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8004c20 con 0x55de96914480 2024-09-11T01:15:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.690+0000 7f33b69b4740 1 -- 172.21.15.78:0/4106811554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 msgr2=0x55de96914860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.690+0000 7f33b69b4740 1 --2- 172.21.15.78:0/4106811554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 0x55de96914860 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009900 tx=0x7f33a8005c00 comp rx=0 tx=0).stop 2024-09-11T01:15:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.691+0000 7f33b69b4740 1 -- 172.21.15.78:0/4106811554 shutdown_connections 2024-09-11T01:15:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.691+0000 7f33b69b4740 1 --2- 172.21.15.78:0/4106811554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96920d30 0x55de96923120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.691+0000 7f33b69b4740 1 --2- 172.21.15.78:0/4106811554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de969207f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.691+0000 7f33b69b4740 1 --2- 172.21.15.78:0/4106811554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 0x55de96914860 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.691+0000 7f33b69b4740 1 -- 172.21.15.78:0/4106811554 >> 172.21.15.78:0/4106811554 conn(0x55de969122b0 msgr2=0x55de96913450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.691+0000 7f33b69b4740 1 -- 172.21.15.78:0/4106811554 shutdown_connections 2024-09-11T01:15:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.691+0000 7f33b69b4740 1 -- 172.21.15.78:0/4106811554 wait complete. 2024-09-11T01:15:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.691+0000 7f33b69b4740 1 -- start start 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.691+0000 7f33b69b4740 1 Processor -- start 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.691+0000 7f33b69b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 0x55de96920630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.691+0000 7f33b69b4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de9691da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b69b4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96920d30 0x55de9691e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b69b4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de966db700 con 0x55de96920d30 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b69b4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55de96815110 con 0x55de96914da0 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b69b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55de96923950 con 0x55de96914480 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b237b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 0x55de96920630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b237b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 0x55de96920630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:43790/0 (socket says 172.21.15.78:43790) 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b237b640 1 -- 172.21.15.78:0/2454136919 learned_addr learned my addr 172.21.15.78:0/2454136919 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de9691da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b69b4740 10 client.0 _ll_register_callbacks cb 0x55de9682e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b2b7c640 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96920d30 0x55de9691e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b1b7a640 1 -- 172.21.15.78:0/2454136919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 msgr2=0x55de96920630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 0x55de96920630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b1b7a640 1 -- 172.21.15.78:0/2454136919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96920d30 msgr2=0x55de9691e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96920d30 0x55de9691e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b1b7a640 1 -- 172.21.15.78:0/2454136919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de96842c40 con 0x55de96914da0 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.692+0000 7f33b2b7c640 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96920d30 0x55de9691e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.693+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de9691da80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f339c0e3a80 tx=0x7f339c0ac100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce079a13722330d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.693+0000 7f3395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.693+0000 7f3395ffb640 1 -- 172.21.15.78:0/2454136919 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0d3030 con 0x55de96914da0 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.693+0000 7f3395ffb640 1 -- 172.21.15.78:0/2454136919 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f339c0037a0 con 0x55de96914da0 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.693+0000 7f33b69b4740 1 -- 172.21.15.78:0/2454136919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33a80054f0 con 0x55de96914da0 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.693+0000 7f3395ffb640 1 -- 172.21.15.78:0/2454136919 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f339c0aceb0 con 0x55de96914da0 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.693+0000 7f33b69b4740 10 client.27617 fetch_fsmap learned FSMap version 69 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.693+0000 7f33b69b4740 1 -- 172.21.15.78:0/2454136919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55de9692f300 con 0x55de96914da0 2024-09-11T01:15:40.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.693+0000 7f3395ffb640 1 -- 172.21.15.78:0/2454136919 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a80054f0 con 0x55de96914da0 2024-09-11T01:15:40.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.693+0000 7f3395ffb640 1 -- 172.21.15.78:0/2454136919 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 69) ==== 8366+0+0 (secure 0 0 0) 0x7f339c0b58b0 con 0x55de96914da0 2024-09-11T01:15:40.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.694+0000 7f33b69b4740 10 client.27617 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T01:15:40.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.694+0000 7f33b69b4740 10 client.27617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:15:40.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.694+0000 7f33b69b4740 20 client.27617 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:40.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.694+0000 7f33b69b4740 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55de96930860 0x55de96932c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.694+0000 7f33b69b4740 4 client.27617 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.694+0000 7f33b69b4740 1 -- 172.21.15.78:0/2454136919 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de9688cda0 con 0x55de96930860 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.694+0000 7f33b237b640 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55de96930860 0x55de96932c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.694+0000 7f3395ffb640 10 client.27617 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.694+0000 7f33b237b640 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55de96930860 0x55de96932c90 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8a2e929fc8ca93 server_cookie=3c03b721bfca16b0 in_seq=0 out_seq=0 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.695+0000 7f3395ffb640 1 -- 172.21.15.78:0/2454136919 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de9688cda0 con 0x55de96930860 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.695+0000 7f3395ffb640 10 client.27617 handle_command_reply: tid=0 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.695+0000 7f33b69b4740 1 client.27617 shutdown 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.695+0000 7f33b69b4740 1 -- 172.21.15.78:0/2454136919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 msgr2=0x55de9691da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.695+0000 7f33b69b4740 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de9691da80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f339c0e3a80 tx=0x7f339c0ac100 comp rx=0 tx=0).stop 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.695+0000 7f33b69b4740 1 -- 172.21.15.78:0/2454136919 shutdown_connections 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.695+0000 7f33b69b4740 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55de96930860 0x55de96932c90 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.696+0000 7f33b69b4740 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96920d30 0x55de9691e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.696+0000 7f33b69b4740 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de9691da80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.696+0000 7f33b69b4740 1 --2- 172.21.15.78:0/2454136919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de96914480 0x55de96920630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.696+0000 7f33b69b4740 1 -- 172.21.15.78:0/2454136919 >> 172.21.15.78:0/2454136919 conn(0x55de969122b0 msgr2=0x55de96917e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:40.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.696+0000 7f3395ffb640 0 client.27617 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:40.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.696+0000 7f33b69b4740 1 -- 172.21.15.78:0/2454136919 shutdown_connections 2024-09-11T01:15:40.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.696+0000 7f33b69b4740 1 -- 172.21.15.78:0/2454136919 wait complete. 2024-09-11T01:15:40.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.696+0000 7f33b69b4740 20 client.27617 trim_cache size 0 max 16384 2024-09-11T01:15:40.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.723+0000 7f33b69b4740 1 Processor -- start 2024-09-11T01:15:40.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.723+0000 7f33b69b4740 1 -- start start 2024-09-11T01:15:40.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.723+0000 7f33b69b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de969122b0 0x55de9692cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.723+0000 7f33b69b4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 0x55de96913300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.723+0000 7f33b69b4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de96922120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.723+0000 7f33b69b4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1b0c10 con 0x55de96914480 2024-09-11T01:15:40.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.723+0000 7f33b69b4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1c7790 con 0x55de96914da0 2024-09-11T01:15:40.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.723+0000 7f33b69b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1b12e0 con 0x55de969122b0 2024-09-11T01:15:40.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.724+0000 7f33b1b7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 0x55de96913300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.724+0000 7f33b1b7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 0x55de96913300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41938/0 (socket says 172.21.15.78:41938) 2024-09-11T01:15:40.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.724+0000 7f33b1b7a640 1 -- 172.21.15.78:0/1842732600 learned_addr learned my addr 172.21.15.78:0/1842732600 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:40.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.724+0000 7f33b1b7a640 1 -- 172.21.15.78:0/1842732600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de969122b0 msgr2=0x55de9692cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:40.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.724+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/1842732600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de969122b0 0x55de9692cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.724+0000 7f33b1b7a640 1 -- 172.21.15.78:0/1842732600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 msgr2=0x55de96922120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:40.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.724+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/1842732600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de96922120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.724+0000 7f33b1b7a640 1 -- 172.21.15.78:0/1842732600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de9691bf80 con 0x55de96914480 2024-09-11T01:15:40.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.724+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/1842732600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 0x55de96913300 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f339c0a9a20 tx=0x7f339c003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f578d7f74b75e1f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:40.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.725+0000 7f3397fff640 1 -- 172.21.15.78:0/1842732600 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0d3040 con 0x55de96914480 2024-09-11T01:15:40.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.725+0000 7f3397fff640 1 -- 172.21.15.78:0/1842732600 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f339c0ac7b0 con 0x55de96914480 2024-09-11T01:15:40.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.725+0000 7f3397fff640 1 -- 172.21.15.78:0/1842732600 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0b58b0 con 0x55de96914480 2024-09-11T01:15:40.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.726+0000 7f33b69b4740 1 -- 172.21.15.78:0/1842732600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 msgr2=0x55de96913300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:40.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.726+0000 7f33b69b4740 1 --2- 172.21.15.78:0/1842732600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 0x55de96913300 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f339c0a9a20 tx=0x7f339c003e70 comp rx=0 tx=0).stop 2024-09-11T01:15:40.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.726+0000 7f33b69b4740 1 -- 172.21.15.78:0/1842732600 shutdown_connections 2024-09-11T01:15:40.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.726+0000 7f33b69b4740 1 --2- 172.21.15.78:0/1842732600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de96922120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.726+0000 7f33b69b4740 1 --2- 172.21.15.78:0/1842732600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 0x55de96913300 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.726+0000 7f33b69b4740 1 --2- 172.21.15.78:0/1842732600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de969122b0 0x55de9692cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.726+0000 7f33b69b4740 1 -- 172.21.15.78:0/1842732600 >> 172.21.15.78:0/1842732600 conn(0x55de96919420 msgr2=0x55de969113d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:40.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.726+0000 7f33b69b4740 1 -- 172.21.15.78:0/1842732600 shutdown_connections 2024-09-11T01:15:40.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.726+0000 7f33b69b4740 1 -- 172.21.15.78:0/1842732600 wait complete. 2024-09-11T01:15:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.727+0000 7f33b69b4740 1 -- start start 2024-09-11T01:15:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.727+0000 7f33b69b4740 1 Processor -- start 2024-09-11T01:15:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.728+0000 7f33b69b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de969122b0 0x55de96929460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.728+0000 7f33b69b4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 0x55de96923ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.728+0000 7f33b69b4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de969241b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.728+0000 7f33b69b4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1c70c0 con 0x55de96914480 2024-09-11T01:15:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.728+0000 7f33b69b4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1c6ac0 con 0x55de96914da0 2024-09-11T01:15:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.728+0000 7f33b69b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1aec10 con 0x55de969122b0 2024-09-11T01:15:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.728+0000 7f33b69b4740 10 client.0 _ll_register_callbacks cb 0x7f339c00e8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.729+0000 7f33b1b7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 0x55de96923ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.729+0000 7f33b1b7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 0x55de96923ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41952/0 (socket says 172.21.15.78:41952) 2024-09-11T01:15:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.729+0000 7f33b1b7a640 1 -- 172.21.15.78:0/3453607799 learned_addr learned my addr 172.21.15.78:0/3453607799 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.729+0000 7f33b1b7a640 1 -- 172.21.15.78:0/3453607799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de969122b0 msgr2=0x55de96929460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.729+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/3453607799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de969122b0 0x55de96929460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.729+0000 7f33b1b7a640 1 -- 172.21.15.78:0/3453607799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 msgr2=0x55de969241b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:15:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.729+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/3453607799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de969241b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.729+0000 7f33b1b7a640 1 -- 172.21.15.78:0/3453607799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de969249e0 con 0x55de96914480 2024-09-11T01:15:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.730+0000 7f33b1b7a640 1 --2- 172.21.15.78:0/3453607799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 0x55de96923ba0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f339c0a8d30 tx=0x7f339c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6773a07e128e90c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.730+0000 7f3395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:15:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.730+0000 7f3395ffb640 1 -- 172.21.15.78:0/3453607799 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c08b060 con 0x55de96914480 2024-09-11T01:15:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.730+0000 7f3395ffb640 1 -- 172.21.15.78:0/3453607799 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f339c0abc20 con 0x55de96914480 2024-09-11T01:15:40.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.731+0000 7f33b69b4740 1 -- 172.21.15.78:0/3453607799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f339c0ac7b0 con 0x55de96914480 2024-09-11T01:15:40.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.731+0000 7f3395ffb640 1 -- 172.21.15.78:0/3453607799 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f339c0d3040 con 0x55de96914480 2024-09-11T01:15:40.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.731+0000 7f33b69b4740 10 client.18495 fetch_fsmap learned FSMap version 69 2024-09-11T01:15:40.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.731+0000 7f33b69b4740 1 -- 172.21.15.78:0/3453607799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55de9699e8a0 con 0x55de96914480 2024-09-11T01:15:40.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.732+0000 7f3395ffb640 1 -- 172.21.15.78:0/3453607799 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f339c0b1290 con 0x55de96914480 2024-09-11T01:15:40.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.732+0000 7f3395ffb640 1 -- 172.21.15.78:0/3453607799 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 69) ==== 8366+0+0 (secure 0 0 0) 0x7f339c0a8100 con 0x55de96914480 2024-09-11T01:15:40.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.732+0000 7f33b69b4740 10 client.18495 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T01:15:40.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.732+0000 7f33b69b4740 10 client.18495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:15:40.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.732+0000 7f33b69b4740 20 client.18495 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:40.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.732+0000 7f33b69b4740 1 --2- 172.21.15.78:0/3453607799 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55de96920d30 0x55de969a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:40.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.732+0000 7f33b69b4740 4 client.18495 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:15:40.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.732+0000 7f33b69b4740 1 -- 172.21.15.78:0/3453607799 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f339c0ac7b0 con 0x55de96920d30 2024-09-11T01:15:40.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.732+0000 7f33b237b640 1 --2- 172.21.15.78:0/3453607799 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55de96920d30 0x55de969a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:40.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.733+0000 7f33b237b640 1 --2- 172.21.15.78:0/3453607799 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55de96920d30 0x55de969a0dc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c99b02c1193aee13 server_cookie=958fa85b3eafe1e8 in_seq=0 out_seq=0 2024-09-11T01:15:40.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.733+0000 7f3395ffb640 10 client.18495 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:40.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.734+0000 7f3395ffb640 1 -- 172.21.15.78:0/3453607799 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6054 (crc 0 0 0) 0x7f339c0ac7b0 con 0x55de96920d30 2024-09-11T01:15:40.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.734+0000 7f3395ffb640 10 client.18495 handle_command_reply: tid=0 2024-09-11T01:15:40.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.734+0000 7f33b69b4740 1 client.18495 shutdown 2024-09-11T01:15:40.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.734+0000 7f33b69b4740 1 -- 172.21.15.78:0/3453607799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 msgr2=0x55de96923ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:40.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.734+0000 7f33b69b4740 1 --2- 172.21.15.78:0/3453607799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 0x55de96923ba0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f339c0a8d30 tx=0x7f339c00c840 comp rx=0 tx=0).stop 2024-09-11T01:15:40.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.734+0000 7f33b69b4740 1 -- 172.21.15.78:0/3453607799 shutdown_connections 2024-09-11T01:15:40.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.734+0000 7f33b69b4740 1 --2- 172.21.15.78:0/3453607799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de96914da0 0x55de969241b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.735+0000 7f33b69b4740 1 --2- 172.21.15.78:0/3453607799 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55de96920d30 0x55de969a0dc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.735+0000 7f33b69b4740 1 --2- 172.21.15.78:0/3453607799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de96914480 0x55de96923ba0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.735+0000 7f3395ffb640 0 client.18495 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:40.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.735+0000 7f33b69b4740 1 --2- 172.21.15.78:0/3453607799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de969122b0 0x55de96929460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.735+0000 7f33b69b4740 1 -- 172.21.15.78:0/3453607799 >> 172.21.15.78:0/3453607799 conn(0x55de96919420 msgr2=0x55de9691f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:40.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.735+0000 7f33b69b4740 1 -- 172.21.15.78:0/3453607799 shutdown_connections 2024-09-11T01:15:40.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.735+0000 7f33b69b4740 1 -- 172.21.15.78:0/3453607799 wait complete. 2024-09-11T01:15:40.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.735+0000 7f33b69b4740 20 client.18495 trim_cache size 0 max 16384 2024-09-11T01:15:40.739 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:15:40.739 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:15:40.740 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:15:40.740 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:15:40.740 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:15:40.740 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:15:40.740 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.740 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.740 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.740 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.740 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.740 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:15:40.740 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.741 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.742 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:15:40.743 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.746 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.747 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:15:40.748 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.750 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:15:40.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:15:40.752 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:15:40.752 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:15:40.752 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:15:40.752 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:15:40.752 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:15:40.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.736+0000 7f33b45df640 1 -- 172.21.15.78:0/1656871354 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f33980b54d0 msgr2=0x7f33980b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:40.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.736+0000 7f33b45df640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f33980b54d0 0x7f33980b7990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005c40 tx=0x7f33a4009e70 comp rx=0 tx=0).stop 2024-09-11T01:15:40.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.736+0000 7f33b45df640 1 -- 172.21.15.78:0/1656871354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102230 msgr2=0x7f33ac102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:40.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.736+0000 7f33b45df640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102230 0x7f33ac102df0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f339c00e9a0 tx=0x7f339c00ee60 comp rx=0 tx=0).stop 2024-09-11T01:15:40.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.737+0000 7f33b45df640 1 -- 172.21.15.78:0/1656871354 shutdown_connections 2024-09-11T01:15:40.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.737+0000 7f33b45df640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f33980b54d0 0x7f33980b7990 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.737+0000 7f33b45df640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33ac10d0f0 0x7f33ac103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.737+0000 7f33b45df640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102230 0x7f33ac102df0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.737+0000 7f33b45df640 1 --2- 172.21.15.78:0/1656871354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac101830 0x7f33ac1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:40.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.737+0000 7f33b45df640 1 -- 172.21.15.78:0/1656871354 >> 172.21.15.78:0/1656871354 conn(0x7f33ac076e20 msgr2=0x7f33ac0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:40.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.737+0000 7f33b45df640 1 -- 172.21.15.78:0/1656871354 shutdown_connections 2024-09-11T01:15:40.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:40.737+0000 7f33b45df640 1 -- 172.21.15.78:0/1656871354 wait complete. 2024-09-11T01:15:40.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 152211 36.60 MB/sec execute 106 sec latency 80.849 ms 2024-09-11T01:15:41.673 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:15:41.673 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:15:41.786 INFO:tasks.workunit.client.0.smithi078.stdout: 10 153091 36.73 MB/sec execute 107 sec latency 64.508 ms 2024-09-11T01:15:42.088 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:15:42.267 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:42 smithi078 ceph-mon[31315]: pgmap v847: 97 pgs: 97 active+clean; 2.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 19 MiB/s wr, 866 op/s 2024-09-11T01:15:42.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:42 smithi110 ceph-mon[34712]: pgmap v847: 97 pgs: 97 active+clean; 2.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 19 MiB/s wr, 866 op/s 2024-09-11T01:15:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:42 smithi112 ceph-mon[35384]: pgmap v847: 97 pgs: 97 active+clean; 2.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 19 MiB/s wr, 866 op/s 2024-09-11T01:15:42.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.661+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3200494737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc1076a0 msgr2=0x7f7fcc107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:42.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.661+0000 7f7fd1f6d640 1 --2- 172.21.15.78:0/3200494737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc1076a0 0x7f7fcc107ae0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009900 tx=0x7f7fc002a410 comp rx=0 tx=0).stop 2024-09-11T01:15:42.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.661+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3200494737 shutdown_connections 2024-09-11T01:15:42.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.661+0000 7f7fd1f6d640 1 --2- 172.21.15.78:0/3200494737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fcc108020 0x7f7fcc10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:42.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.661+0000 7f7fd1f6d640 1 --2- 172.21.15.78:0/3200494737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc1076a0 0x7f7fcc107ae0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:42.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.661+0000 7f7fd1f6d640 1 --2- 172.21.15.78:0/3200494737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc101c90 0x7f7fcc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:42.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.661+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3200494737 >> 172.21.15.78:0/3200494737 conn(0x7f7fcc0fd700 msgr2=0x7f7fcc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:42.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.661+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3200494737 shutdown_connections 2024-09-11T01:15:42.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.661+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3200494737 wait complete. 2024-09-11T01:15:42.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.662+0000 7f7fd1f6d640 1 Processor -- start 2024-09-11T01:15:42.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.662+0000 7f7fd1f6d640 1 -- start start 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.662+0000 7f7fd1f6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc101c90 0x7f7fcc0fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.662+0000 7f7fd1f6d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fcc1076a0 0x7f7fcc0fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.662+0000 7f7fd1f6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc108020 0x7f7fcc0fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.662+0000 7f7fd1f6d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc111ce0 con 0x7f7fcc101c90 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.662+0000 7f7fd1f6d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc111b60 con 0x7f7fcc1076a0 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.662+0000 7f7fd1f6d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc111e60 con 0x7f7fcc108020 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.662+0000 7f7fcb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc101c90 0x7f7fcc0fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.662+0000 7f7fcb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc101c90 0x7f7fcc0fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41960/0 (socket says 172.21.15.78:41960) 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.662+0000 7f7fcaffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fcc1076a0 0x7f7fcc0fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.662+0000 7f7fcb7fe640 1 -- 172.21.15.78:0/3271127591 learned_addr learned my addr 172.21.15.78:0/3271127591 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.663+0000 7f7fcbfff640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc108020 0x7f7fcc0fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.663+0000 7f7fcb7fe640 1 -- 172.21.15.78:0/3271127591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc108020 msgr2=0x7f7fcc0fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.663+0000 7f7fcb7fe640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc108020 0x7f7fcc0fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:42.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.663+0000 7f7fcb7fe640 1 -- 172.21.15.78:0/3271127591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fcc1076a0 msgr2=0x7f7fcc0fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:42.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.663+0000 7f7fcb7fe640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fcc1076a0 0x7f7fcc0fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:42.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.663+0000 7f7fcb7fe640 1 -- 172.21.15.78:0/3271127591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc1a7780 con 0x7f7fcc101c90 2024-09-11T01:15:42.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.663+0000 7f7fcbfff640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc108020 0x7f7fcc0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:15:42.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.663+0000 7f7fcaffd640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fcc1076a0 0x7f7fcc0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:15:42.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.663+0000 7f7fcb7fe640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc101c90 0x7f7fcc0fdc00 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00e9f0 tx=0x7f7fbc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:42.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.663+0000 7f7fc8ff9640 1 -- 172.21.15.78:0/3271127591 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00cd80 con 0x7f7fcc101c90 2024-09-11T01:15:42.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.664+0000 7f7fc8ff9640 1 -- 172.21.15.78:0/3271127591 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fbc0054d0 con 0x7f7fcc101c90 2024-09-11T01:15:42.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.664+0000 7f7fc8ff9640 1 -- 172.21.15.78:0/3271127591 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc010690 con 0x7f7fcc101c90 2024-09-11T01:15:42.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.664+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3271127591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fcc1a7a70 con 0x7f7fcc101c90 2024-09-11T01:15:42.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.664+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3271127591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fcc1a7fb0 con 0x7f7fcc101c90 2024-09-11T01:15:42.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.666+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3271127591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fcc102070 con 0x7f7fcc101c90 2024-09-11T01:15:42.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.668+0000 7f7fc8ff9640 1 -- 172.21.15.78:0/3271127591 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7fbc004e70 con 0x7f7fcc101c90 2024-09-11T01:15:42.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.669+0000 7f7fc8ff9640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7fb40b54d0 0x7f7fb40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:42.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.669+0000 7f7fc8ff9640 1 -- 172.21.15.78:0/3271127591 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f7fbc0e63e0 con 0x7f7fcc101c90 2024-09-11T01:15:42.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.670+0000 7f7fcaffd640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7fb40b54d0 0x7f7fb40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:42.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.670+0000 7f7fcaffd640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7fb40b54d0 0x7f7fb40b7990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0031440 tx=0x7f7fc0005b50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:42.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:42.674+0000 7f7fc8ff9640 1 -- 172.21.15.78:0/3271127591 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7fbc0acf40 con 0x7f7fcc101c90 2024-09-11T01:15:42.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 153878 36.79 MB/sec execute 108 sec latency 71.331 ms 2024-09-11T01:15:43.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.022+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3271127591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fcc1a8350 con 0x7f7fcc101c90 2024-09-11T01:15:43.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.022+0000 7f7fc8ff9640 1 -- 172.21.15.78:0/3271127591 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+11980 (secure 0 0 0) 0x7f7fbc0ac840 con 0x7f7fcc101c90 2024-09-11T01:15:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:15:43.025 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":69,"btime":"2024-09-11T01:15:37:981691+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:15:37.225546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24580,24565,24563,14808]},"id":1}]} 2024-09-11T01:15:43.026 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 69 2024-09-11T01:15:43.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.025+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3271127591 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7fb40b54d0 msgr2=0x7f7fb40b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:43.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.025+0000 7f7fd1f6d640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7fb40b54d0 0x7f7fb40b7990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0031440 tx=0x7f7fc0005b50 comp rx=0 tx=0).stop 2024-09-11T01:15:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.025+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3271127591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc101c90 msgr2=0x7f7fcc0fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.025+0000 7f7fd1f6d640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc101c90 0x7f7fcc0fdc00 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00e9f0 tx=0x7f7fbc00edf0 comp rx=0 tx=0).stop 2024-09-11T01:15:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.025+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3271127591 shutdown_connections 2024-09-11T01:15:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.025+0000 7f7fd1f6d640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7fb40b54d0 0x7f7fb40b7990 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.025+0000 7f7fd1f6d640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc108020 0x7f7fcc0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.025+0000 7f7fd1f6d640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fcc1076a0 0x7f7fcc0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.025+0000 7f7fd1f6d640 1 --2- 172.21.15.78:0/3271127591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc101c90 0x7f7fcc0fdc00 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.025+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3271127591 >> 172.21.15.78:0/3271127591 conn(0x7f7fcc0fd700 msgr2=0x7f7fcc100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.025+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3271127591 shutdown_connections 2024-09-11T01:15:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:43.025+0000 7f7fd1f6d640 1 -- 172.21.15.78:0/3271127591 wait complete. 2024-09-11T01:15:43.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 154728 36.88 MB/sec execute 109 sec latency 69.188 ms 2024-09-11T01:15:44.405 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:44 smithi078 ceph-mon[31315]: pgmap v848: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 825 op/s 2024-09-11T01:15:44.405 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:44 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3271127591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:15:44.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:44 smithi110 ceph-mon[34712]: pgmap v848: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 825 op/s 2024-09-11T01:15:44.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:44 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3271127591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:15:44.451 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:15:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:44 smithi112 ceph-mon[35384]: pgmap v848: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 825 op/s 2024-09-11T01:15:44.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:44 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3271127591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:15:44.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 155482 36.92 MB/sec execute 110 sec latency 68.077 ms 2024-09-11T01:15:44.857 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:15:45.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.387+0000 7f7914905640 1 -- 172.21.15.78:0/402589661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79101045c0 msgr2=0x7f791010cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:45.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.387+0000 7f7914905640 1 --2- 172.21.15.78:0/402589661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79101045c0 0x7f791010cab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7904009900 tx=0x7f790402a3f0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.388+0000 7f7914905640 1 -- 172.21.15.78:0/402589661 shutdown_connections 2024-09-11T01:15:45.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.388+0000 7f7914905640 1 --2- 172.21.15.78:0/402589661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f791010d100 0x7f791010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.388+0000 7f7914905640 1 --2- 172.21.15.78:0/402589661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79101045c0 0x7f791010cab0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.388+0000 7f7914905640 1 --2- 172.21.15.78:0/402589661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7910103ca0 0x7f7910104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.388+0000 7f7914905640 1 -- 172.21.15.78:0/402589661 >> 172.21.15.78:0/402589661 conn(0x7f79100fd700 msgr2=0x7f79100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:45.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.388+0000 7f7914905640 1 -- 172.21.15.78:0/402589661 shutdown_connections 2024-09-11T01:15:45.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.388+0000 7f7914905640 1 -- 172.21.15.78:0/402589661 wait complete. 2024-09-11T01:15:45.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f7914905640 1 Processor -- start 2024-09-11T01:15:45.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f7914905640 1 -- start start 2024-09-11T01:15:45.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f7914905640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7910103ca0 0x7f791019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f7914905640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79101045c0 0x7f791019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f7914905640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f791010d100 0x7f79101a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f7914905640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7910112dc0 con 0x7f79101045c0 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f7914905640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7910112c40 con 0x7f791010d100 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f790dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79101045c0 0x7f791019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f790dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79101045c0 0x7f791019ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47726/0 (socket says 172.21.15.78:47726) 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f790dd9b640 1 -- 172.21.15.78:0/1377035481 learned_addr learned my addr 172.21.15.78:0/1377035481 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f7914905640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7910112f40 con 0x7f7910103ca0 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f790e59c640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7910103ca0 0x7f791019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.389+0000 7f790ed9d640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f791010d100 0x7f79101a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.390+0000 7f790dd9b640 1 -- 172.21.15.78:0/1377035481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7910103ca0 msgr2=0x7f791019b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.390+0000 7f790dd9b640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7910103ca0 0x7f791019b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.390+0000 7f790dd9b640 1 -- 172.21.15.78:0/1377035481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f791010d100 msgr2=0x7f79101a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.390+0000 7f790dd9b640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f791010d100 0x7f79101a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.390+0000 7f790dd9b640 1 -- 172.21.15.78:0/1377035481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79101a15b0 con 0x7f79101045c0 2024-09-11T01:15:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.390+0000 7f790e59c640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7910103ca0 0x7f791019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:15:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.390+0000 7f790ed9d640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f791010d100 0x7f79101a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:15:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.390+0000 7f790dd9b640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79101045c0 0x7f791019ba40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7904005ab0 tx=0x7f7904003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.390+0000 7f78ff7fe640 1 -- 172.21.15.78:0/1377035481 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790403f070 con 0x7f79101045c0 2024-09-11T01:15:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.390+0000 7f78ff7fe640 1 -- 172.21.15.78:0/1377035481 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7904031b50 con 0x7f79101045c0 2024-09-11T01:15:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.390+0000 7f78ff7fe640 1 -- 172.21.15.78:0/1377035481 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904035030 con 0x7f79101045c0 2024-09-11T01:15:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.390+0000 7f7914905640 1 -- 172.21.15.78:0/1377035481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79101a1840 con 0x7f79101045c0 2024-09-11T01:15:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.391+0000 7f7914905640 1 -- 172.21.15.78:0/1377035481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79101a7f60 con 0x7f79101045c0 2024-09-11T01:15:45.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.394+0000 7f78ff7fe640 1 -- 172.21.15.78:0/1377035481 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f79040316a0 con 0x7f79101045c0 2024-09-11T01:15:45.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.394+0000 7f7916cda740 1 Processor -- start 2024-09-11T01:15:45.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.395+0000 7f7916cda740 1 -- start start 2024-09-11T01:15:45.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.395+0000 7f7916cda740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115d0f0 0x55c77115d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.395+0000 7f7916cda740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115da10 0x55c771169460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.395+0000 7f7916cda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7711699a0 0x55c77116bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.395+0000 7f7916cda740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c7710d3ea0 con 0x55c77115d0f0 2024-09-11T01:15:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.395+0000 7f7916cda740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c7710c32d0 con 0x55c77115da10 2024-09-11T01:15:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.395+0000 7f7916cda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7710c70e0 con 0x55c7711699a0 2024-09-11T01:15:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.395+0000 7f790dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7711699a0 0x55c77116bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.395+0000 7f790dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7711699a0 0x55c77116bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:49004/0 (socket says 172.21.15.78:49004) 2024-09-11T01:15:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.395+0000 7f790dd9b640 1 -- 172.21.15.78:0/3623862782 learned_addr learned my addr 172.21.15.78:0/3623862782 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.396+0000 7f790e59c640 1 --2- 172.21.15.78:0/3623862782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115da10 0x55c771169460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.396+0000 7f790dd9b640 1 -- 172.21.15.78:0/3623862782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115da10 msgr2=0x55c771169460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.396+0000 7f790dd9b640 1 --2- 172.21.15.78:0/3623862782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115da10 0x55c771169460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.396+0000 7f790dd9b640 1 -- 172.21.15.78:0/3623862782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115d0f0 msgr2=0x55c77115d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.396+0000 7f790dd9b640 1 --2- 172.21.15.78:0/3623862782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115d0f0 0x55c77115d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.396+0000 7f790dd9b640 1 -- 172.21.15.78:0/3623862782 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c770e21650 con 0x55c7711699a0 2024-09-11T01:15:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.396+0000 7f78ff7fe640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78f80b54d0 0x7f78f80b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.396+0000 7f790dd9b640 1 --2- 172.21.15.78:0/3623862782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7711699a0 0x55c77116bd90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7904103f20 tx=0x7f790410ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=252364b1d8c29933 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:45.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.397+0000 7f78ff7fe640 1 -- 172.21.15.78:0/1377035481 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f7904106820 con 0x7f79101045c0 2024-09-11T01:15:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.397+0000 7f78fcff9640 1 -- 172.21.15.78:0/3623862782 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79040c9330 con 0x55c7711699a0 2024-09-11T01:15:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.397+0000 7f78fcff9640 1 -- 172.21.15.78:0/3623862782 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79040c8440 con 0x55c7711699a0 2024-09-11T01:15:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.397+0000 7f790ed9d640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78f80b54d0 0x7f78f80b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.397+0000 7f78fcff9640 1 -- 172.21.15.78:0/3623862782 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79040cc450 con 0x55c7711699a0 2024-09-11T01:15:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.398+0000 7f790ed9d640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78f80b54d0 0x7f78f80b7990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7908009c10 tx=0x7f7908009530 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.398+0000 7f7916cda740 1 -- 172.21.15.78:0/3623862782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7711699a0 msgr2=0x55c77116bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.398+0000 7f7916cda740 1 --2- 172.21.15.78:0/3623862782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7711699a0 0x55c77116bd90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7904103f20 tx=0x7f790410ccd0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.399+0000 7f7916cda740 1 -- 172.21.15.78:0/3623862782 shutdown_connections 2024-09-11T01:15:45.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.399+0000 7f7916cda740 1 --2- 172.21.15.78:0/3623862782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7711699a0 0x55c77116bd90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.399+0000 7f7916cda740 1 --2- 172.21.15.78:0/3623862782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115da10 0x55c771169460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.399+0000 7f7916cda740 1 --2- 172.21.15.78:0/3623862782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115d0f0 0x55c77115d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.399+0000 7f7916cda740 1 -- 172.21.15.78:0/3623862782 >> 172.21.15.78:0/3623862782 conn(0x55c77115af20 msgr2=0x55c77115c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:45.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.400+0000 7f7916cda740 1 -- 172.21.15.78:0/3623862782 shutdown_connections 2024-09-11T01:15:45.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.400+0000 7f7916cda740 1 -- 172.21.15.78:0/3623862782 wait complete. 2024-09-11T01:15:45.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.401+0000 7f7916cda740 1 -- start start 2024-09-11T01:15:45.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.401+0000 7f7916cda740 1 Processor -- start 2024-09-11T01:15:45.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.402+0000 7f7916cda740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 0x55c771169390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.402+0000 7f7916cda740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115da10 0x55c7711667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.402+0000 7f790e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 0x55c771169390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.402+0000 7f790e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 0x55c771169390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:49156/0 (socket says 172.21.15.78:49156) 2024-09-11T01:15:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.402+0000 7f790dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115da10 0x55c7711667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.402+0000 7f790dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115da10 0x55c7711667e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47740/0 (socket says 172.21.15.78:47740) 2024-09-11T01:15:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.402+0000 7f790e59c640 1 -- 172.21.15.78:0/1443196886 learned_addr learned my addr 172.21.15.78:0/1443196886 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.402+0000 7f7916cda740 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7711699a0 0x55c771166f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.403+0000 7f7916cda740 1 -- 172.21.15.78:0/1443196886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c770f24700 con 0x55c77115da10 2024-09-11T01:15:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.403+0000 7f7916cda740 1 -- 172.21.15.78:0/1443196886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c77105e160 con 0x55c77115d0f0 2024-09-11T01:15:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.403+0000 7f790ed9d640 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7711699a0 0x55c771166f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.403+0000 7f7916cda740 1 -- 172.21.15.78:0/1443196886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7710c70e0 con 0x55c7711699a0 2024-09-11T01:15:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.403+0000 7f7916cda740 10 client.0 _ll_register_callbacks cb 0x55c7710779e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.403+0000 7f790e59c640 1 -- 172.21.15.78:0/1443196886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7711699a0 msgr2=0x55c771166f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.403+0000 7f790e59c640 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7711699a0 0x55c771166f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.403+0000 7f790e59c640 1 -- 172.21.15.78:0/1443196886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115da10 msgr2=0x55c7711667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.403+0000 7f790e59c640 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115da10 0x55c7711667e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.403+0000 7f790e59c640 1 -- 172.21.15.78:0/1443196886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c77108bc90 con 0x55c77115d0f0 2024-09-11T01:15:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.403+0000 7f790e59c640 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 0x55c771169390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7900011e60 tx=0x7f7900013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b77cf533fdd6a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.404+0000 7f78f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:15:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.404+0000 7f78f5ffb640 1 -- 172.21.15.78:0/1443196886 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900003d80 con 0x55c77115d0f0 2024-09-11T01:15:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.404+0000 7f78f5ffb640 1 -- 172.21.15.78:0/1443196886 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f7900004b20 con 0x55c77115d0f0 2024-09-11T01:15:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.404+0000 7f7916cda740 1 -- 172.21.15.78:0/1443196886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79040c8440 con 0x55c77115d0f0 2024-09-11T01:15:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.404+0000 7f78f5ffb640 1 -- 172.21.15.78:0/1443196886 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f7900004d20 con 0x55c77115d0f0 2024-09-11T01:15:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.404+0000 7f7916cda740 10 client.27632 fetch_fsmap learned FSMap version 69 2024-09-11T01:15:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.404+0000 7f7916cda740 1 -- 172.21.15.78:0/1443196886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=69}) -- 0x55c771177a90 con 0x55c77115d0f0 2024-09-11T01:15:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.405+0000 7f78f5ffb640 1 -- 172.21.15.78:0/1443196886 <== mon.2 v2:172.21.15.110:3300/0 4 ==== fsmap(e 69) ==== 8366+0+0 (secure 0 0 0) 0x7f790001a070 con 0x55c77115d0f0 2024-09-11T01:15:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.405+0000 7f7916cda740 10 client.27632 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T01:15:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.405+0000 7f7916cda740 10 client.27632 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:15:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.405+0000 7f7916cda740 20 client.27632 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.405+0000 7f7916cda740 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c771178e90 0x55c77117b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.405+0000 7f7916cda740 4 client.27632 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:15:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.405+0000 7f7916cda740 1 -- 172.21.15.78:0/1443196886 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f79040c8440 con 0x55c771178e90 2024-09-11T01:15:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.405+0000 7f790dd9b640 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c771178e90 0x55c77117b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.406+0000 7f790dd9b640 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c771178e90 0x55c77117b280 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d1b767977909dfe6 server_cookie=fb73f06b038e13fe in_seq=0 out_seq=0 2024-09-11T01:15:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.406+0000 7f78f5ffb640 10 client.27632 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:15:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f78f5ffb640 1 -- 172.21.15.78:0/1443196886 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f79040c8440 con 0x55c771178e90 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f78f5ffb640 10 client.27632 handle_command_reply: tid=0 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f7916cda740 1 client.27632 shutdown 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f7916cda740 1 -- 172.21.15.78:0/1443196886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 msgr2=0x55c771169390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f7916cda740 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 0x55c771169390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7900011e60 tx=0x7f7900013830 comp rx=0 tx=0).stop 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f7916cda740 1 -- 172.21.15.78:0/1443196886 shutdown_connections 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f7916cda740 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c771178e90 0x55c77117b280 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f78f5ffb640 0 client.27632 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f7916cda740 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7711699a0 0x55c771166f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f7916cda740 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115da10 0x55c7711667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f7916cda740 1 --2- 172.21.15.78:0/1443196886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 0x55c771169390 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f7916cda740 1 -- 172.21.15.78:0/1443196886 >> 172.21.15.78:0/1443196886 conn(0x55c77115af20 msgr2=0x55c771160ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f7916cda740 1 -- 172.21.15.78:0/1443196886 shutdown_connections 2024-09-11T01:15:45.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f7916cda740 1 -- 172.21.15.78:0/1443196886 wait complete. 2024-09-11T01:15:45.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.407+0000 7f7916cda740 20 client.27632 trim_cache size 0 max 16384 2024-09-11T01:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f7916cda740 1 Processor -- start 2024-09-11T01:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f7916cda740 1 -- start start 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f7916cda740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115d0f0 0x55c771165cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f7916cda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77115da10 0x55c771165060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f7916cda740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7711699a0 0x55c7711655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f7916cda740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b07d0 con 0x55c77115d0f0 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f7916cda740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b2310 con 0x55c7711699a0 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f7916cda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b0ea0 con 0x55c77115da10 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f790e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115d0f0 0x55c771165cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f790e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115d0f0 0x55c771165cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47744/0 (socket says 172.21.15.78:47744) 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f790e59c640 1 -- 172.21.15.78:0/1509291905 learned_addr learned my addr 172.21.15.78:0/1509291905 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f790ed9d640 1 --2- 172.21.15.78:0/1509291905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7711699a0 0x55c7711655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.421+0000 7f790dd9b640 1 --2- 172.21.15.78:0/1509291905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77115da10 0x55c771165060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.422+0000 7f790e59c640 1 -- 172.21.15.78:0/1509291905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77115da10 msgr2=0x55c771165060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.422+0000 7f790e59c640 1 --2- 172.21.15.78:0/1509291905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77115da10 0x55c771165060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.422+0000 7f790e59c640 1 -- 172.21.15.78:0/1509291905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7711699a0 msgr2=0x55c7711655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.422+0000 7f790e59c640 1 --2- 172.21.15.78:0/1509291905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7711699a0 0x55c7711655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.422+0000 7f790e59c640 1 -- 172.21.15.78:0/1509291905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7711672e0 con 0x55c77115d0f0 2024-09-11T01:15:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.422+0000 7f790e59c640 1 --2- 172.21.15.78:0/1509291905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115d0f0 0x55c771165cd0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f7900013870 tx=0x7f7900016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bfc27541f1b0eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.422+0000 7f78f7fff640 1 -- 172.21.15.78:0/1509291905 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900033070 con 0x55c77115d0f0 2024-09-11T01:15:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.422+0000 7f78f7fff640 1 -- 172.21.15.78:0/1509291905 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7900004b20 con 0x55c77115d0f0 2024-09-11T01:15:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.422+0000 7f78f7fff640 1 -- 172.21.15.78:0/1509291905 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900005020 con 0x55c77115d0f0 2024-09-11T01:15:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.423+0000 7f7916cda740 1 -- 172.21.15.78:0/1509291905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115d0f0 msgr2=0x55c771165cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.423+0000 7f7916cda740 1 --2- 172.21.15.78:0/1509291905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115d0f0 0x55c771165cd0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f7900013870 tx=0x7f7900016000 comp rx=0 tx=0).stop 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.423+0000 7f7916cda740 1 -- 172.21.15.78:0/1509291905 shutdown_connections 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.423+0000 7f7916cda740 1 --2- 172.21.15.78:0/1509291905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7711699a0 0x55c7711655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.423+0000 7f7916cda740 1 --2- 172.21.15.78:0/1509291905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77115da10 0x55c771165060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.423+0000 7f7916cda740 1 --2- 172.21.15.78:0/1509291905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77115d0f0 0x55c771165cd0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.423+0000 7f7916cda740 1 -- 172.21.15.78:0/1509291905 >> 172.21.15.78:0/1509291905 conn(0x55c77115af20 msgr2=0x55c7711619f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.423+0000 7f7916cda740 1 -- 172.21.15.78:0/1509291905 shutdown_connections 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.423+0000 7f7916cda740 1 -- 172.21.15.78:0/1509291905 wait complete. 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.423+0000 7f7916cda740 1 -- start start 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.423+0000 7f7916cda740 1 Processor -- start 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f7916cda740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 0x55c771171990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f7916cda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77115da10 0x55c77116c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f7916cda740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7711699a0 0x55c77116c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f7916cda740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b1c40 con 0x55c7711699a0 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f7916cda740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b1570 con 0x55c77115d0f0 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f7916cda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79101ae7d0 con 0x55c77115da10 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f790e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 0x55c771171990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f790e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 0x55c771171990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:49172/0 (socket says 172.21.15.78:49172) 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f790e59c640 1 -- 172.21.15.78:0/465271294 learned_addr learned my addr 172.21.15.78:0/465271294 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f7916cda740 10 client.0 _ll_register_callbacks cb 0x7f79000113e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f790ed9d640 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7711699a0 0x55c77116c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f790dd9b640 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77115da10 0x55c77116c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f790e59c640 1 -- 172.21.15.78:0/465271294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77115da10 msgr2=0x55c77116c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f790e59c640 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77115da10 0x55c77116c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f790e59c640 1 -- 172.21.15.78:0/465271294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7711699a0 msgr2=0x55c77116c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f790e59c640 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7711699a0 0x55c77116c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f790e59c640 1 -- 172.21.15.78:0/465271294 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c77116cf20 con 0x55c77115d0f0 2024-09-11T01:15:45.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f790ed9d640 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7711699a0 0x55c77116c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:15:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f790e59c640 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 0x55c771171990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7900011e00 tx=0x7f7900024770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=432e632b2828f6f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.424+0000 7f78f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:15:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f78f5ffb640 1 -- 172.21.15.78:0/465271294 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900033040 con 0x55c77115d0f0 2024-09-11T01:15:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f78f5ffb640 1 -- 172.21.15.78:0/465271294 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f7900024960 con 0x55c77115d0f0 2024-09-11T01:15:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f7916cda740 1 -- 172.21.15.78:0/465271294 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7900004b20 con 0x55c77115d0f0 2024-09-11T01:15:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f78f5ffb640 1 -- 172.21.15.78:0/465271294 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f7900024b60 con 0x55c77115d0f0 2024-09-11T01:15:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f7916cda740 10 client.27638 fetch_fsmap learned FSMap version 69 2024-09-11T01:15:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f7916cda740 1 -- 172.21.15.78:0/465271294 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55c7711e6ad0 con 0x55c77115d0f0 2024-09-11T01:15:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f78f5ffb640 1 -- 172.21.15.78:0/465271294 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7900004b20 con 0x55c77115d0f0 2024-09-11T01:15:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f78f5ffb640 1 -- 172.21.15.78:0/465271294 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 69) ==== 8366+0+0 (secure 0 0 0) 0x7f7900036070 con 0x55c77115d0f0 2024-09-11T01:15:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f7916cda740 10 client.27638 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f7916cda740 10 client.27638 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f7916cda740 20 client.27638 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f7916cda740 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c7711e70d0 0x55c7711e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f7916cda740 4 client.27638 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f7916cda740 1 -- 172.21.15.78:0/465271294 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7710d5df0 con 0x55c7711e70d0 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.425+0000 7f790dd9b640 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c7711e70d0 0x55c7711e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f790dd9b640 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c7711e70d0 0x55c7711e94f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=56caac1c4ca9b55d server_cookie=993dacfd5b5c2282 in_seq=0 out_seq=0 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f78f5ffb640 10 client.27638 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f78f5ffb640 1 -- 172.21.15.78:0/465271294 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7710d5df0 con 0x55c7711e70d0 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f78f5ffb640 10 client.27638 handle_command_reply: tid=0 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f7916cda740 1 client.27638 shutdown 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f7916cda740 1 -- 172.21.15.78:0/465271294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 msgr2=0x55c771171990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f7916cda740 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 0x55c771171990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7900011e00 tx=0x7f7900024770 comp rx=0 tx=0).stop 2024-09-11T01:15:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f7916cda740 1 -- 172.21.15.78:0/465271294 shutdown_connections 2024-09-11T01:15:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f7916cda740 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x55c7711e70d0 0x55c7711e94f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f7916cda740 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7711699a0 0x55c77116c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f7916cda740 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77115da10 0x55c77116c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f78f5ffb640 0 client.27638 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:15:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f7916cda740 1 --2- 172.21.15.78:0/465271294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c77115d0f0 0x55c771171990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f7916cda740 1 -- 172.21.15.78:0/465271294 >> 172.21.15.78:0/465271294 conn(0x55c77115af20 msgr2=0x55c7711619f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f7916cda740 1 -- 172.21.15.78:0/465271294 shutdown_connections 2024-09-11T01:15:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f7916cda740 1 -- 172.21.15.78:0/465271294 wait complete. 2024-09-11T01:15:45.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.426+0000 7f7916cda740 20 client.27638 trim_cache size 0 max 16384 2024-09-11T01:15:45.435 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:15:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.427+0000 7f7914905640 1 -- 172.21.15.78:0/1377035481 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78f80b54d0 msgr2=0x7f78f80b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.427+0000 7f7914905640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78f80b54d0 0x7f78f80b7990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7908009c10 tx=0x7f7908009530 comp rx=0 tx=0).stop 2024-09-11T01:15:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.427+0000 7f7914905640 1 -- 172.21.15.78:0/1377035481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79101045c0 msgr2=0x7f791019ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.427+0000 7f7914905640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79101045c0 0x7f791019ba40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7904005ab0 tx=0x7f7904003b90 comp rx=0 tx=0).stop 2024-09-11T01:15:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.427+0000 7f7914905640 1 -- 172.21.15.78:0/1377035481 shutdown_connections 2024-09-11T01:15:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.427+0000 7f7914905640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78f80b54d0 0x7f78f80b7990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.427+0000 7f7914905640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f791010d100 0x7f79101a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.427+0000 7f7914905640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79101045c0 0x7f791019ba40 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.427+0000 7f7914905640 1 --2- 172.21.15.78:0/1377035481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7910103ca0 0x7f791019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.427+0000 7f7914905640 1 -- 172.21.15.78:0/1377035481 >> 172.21.15.78:0/1377035481 conn(0x7f79100fd700 msgr2=0x7f79100fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.427+0000 7f7914905640 1 -- 172.21.15.78:0/1377035481 shutdown_connections 2024-09-11T01:15:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:45.427+0000 7f7914905640 1 -- 172.21.15.78:0/1377035481 wait complete. 2024-09-11T01:15:45.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 156078 36.83 MB/sec execute 111 sec latency 234.062 ms 2024-09-11T01:15:46.246 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:15:46.278 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:46 smithi078 ceph-mon[31315]: pgmap v849: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 830 op/s 2024-09-11T01:15:46.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:46 smithi110 ceph-mon[34712]: pgmap v849: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 830 op/s 2024-09-11T01:15:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:46 smithi112 ceph-mon[35384]: pgmap v849: 97 pgs: 97 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 830 op/s 2024-09-11T01:15:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:15:46.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 156919 36.90 MB/sec execute 112 sec latency 74.762 ms 2024-09-11T01:15:47.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.205+0000 7f8c34cda640 1 -- 172.21.15.78:0/508603255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c301076a0 msgr2=0x7f8c30107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:47.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.205+0000 7f8c34cda640 1 --2- 172.21.15.78:0/508603255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c301076a0 0x7f8c30107ae0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009860 tx=0x7f8c1802a340 comp rx=0 tx=0).stop 2024-09-11T01:15:47.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.205+0000 7f8c34cda640 1 -- 172.21.15.78:0/508603255 shutdown_connections 2024-09-11T01:15:47.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.205+0000 7f8c34cda640 1 --2- 172.21.15.78:0/508603255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c30108020 0x7f8c3010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.205+0000 7f8c34cda640 1 --2- 172.21.15.78:0/508603255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c301076a0 0x7f8c30107ae0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.205+0000 7f8c34cda640 1 --2- 172.21.15.78:0/508603255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c30101c90 0x7f8c30102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.205+0000 7f8c34cda640 1 -- 172.21.15.78:0/508603255 >> 172.21.15.78:0/508603255 conn(0x7f8c300fd700 msgr2=0x7f8c300ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:47.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.205+0000 7f8c34cda640 1 -- 172.21.15.78:0/508603255 shutdown_connections 2024-09-11T01:15:47.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.205+0000 7f8c34cda640 1 -- 172.21.15.78:0/508603255 wait complete. 2024-09-11T01:15:47.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c34cda640 1 Processor -- start 2024-09-11T01:15:47.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c34cda640 1 -- start start 2024-09-11T01:15:47.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c34cda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c30101c90 0x7f8c3019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c34cda640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c301076a0 0x7f8c3019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c34cda640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c30108020 0x7f8c301a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c34cda640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301193f0 con 0x7f8c301076a0 2024-09-11T01:15:47.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c34cda640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8c30119270 con 0x7f8c30108020 2024-09-11T01:15:47.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c34cda640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8c30119570 con 0x7f8c30101c90 2024-09-11T01:15:47.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c27fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c301076a0 0x7f8c3019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c2e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c30101c90 0x7f8c3019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c2e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c30101c90 0x7f8c3019f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:49042/0 (socket says 172.21.15.78:49042) 2024-09-11T01:15:47.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c2e59c640 1 -- 172.21.15.78:0/1305911281 learned_addr learned my addr 172.21.15.78:0/1305911281 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.206+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c30108020 0x7f8c301a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.207+0000 7f8c2e59c640 1 -- 172.21.15.78:0/1305911281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c30108020 msgr2=0x7f8c301a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.207+0000 7f8c2e59c640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c30108020 0x7f8c301a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.207+0000 7f8c2e59c640 1 -- 172.21.15.78:0/1305911281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c301076a0 msgr2=0x7f8c3019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.207+0000 7f8c2e59c640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c301076a0 0x7f8c3019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.207+0000 7f8c2e59c640 1 -- 172.21.15.78:0/1305911281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c301a58f0 con 0x7f8c30101c90 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.207+0000 7f8c27fff640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c301076a0 0x7f8c3019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.207+0000 7f8c2e59c640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c30101c90 0x7f8c3019f840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000e9c0 tx=0x7f8c2000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.207+0000 7f8c277fe640 1 -- 172.21.15.78:0/1305911281 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2000cde0 con 0x7f8c30101c90 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.207+0000 7f8c34cda640 1 -- 172.21.15.78:0/1305911281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c301a5be0 con 0x7f8c30101c90 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.207+0000 7f8c277fe640 1 -- 172.21.15.78:0/1305911281 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c200035f0 con 0x7f8c30101c90 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.207+0000 7f8c34cda640 1 -- 172.21.15.78:0/1305911281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c301ac2a0 con 0x7f8c30101c90 2024-09-11T01:15:47.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.208+0000 7f8c277fe640 1 -- 172.21.15.78:0/1305911281 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2000cde0 con 0x7f8c30101c90 2024-09-11T01:15:47.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.211+0000 7f8c370af740 1 Processor -- start 2024-09-11T01:15:47.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.211+0000 7f8c370af740 1 -- start start 2024-09-11T01:15:47.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.212+0000 7f8c370af740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 0x5591aee458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.212+0000 7f8c370af740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee51840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.212+0000 7f8c370af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 0x5591aee54170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.212+0000 7f8c370af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5591aedbbea0 con 0x5591aee51d80 2024-09-11T01:15:47.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.212+0000 7f8c370af740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591aedab2d0 con 0x5591aee454d0 2024-09-11T01:15:47.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.212+0000 7f8c370af740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591aedaf0e0 con 0x5591aee45df0 2024-09-11T01:15:47.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.212+0000 7f8c2ed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 0x5591aee458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.212+0000 7f8c2ed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 0x5591aee458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:49204/0 (socket says 172.21.15.78:49204) 2024-09-11T01:15:47.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.212+0000 7f8c2ed9d640 1 -- 172.21.15.78:0/1770135111 learned_addr learned my addr 172.21.15.78:0/1770135111 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:47.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.212+0000 7f8c27fff640 1 --2- 172.21.15.78:0/1770135111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 0x5591aee54170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.213+0000 7f8c2ed9d640 1 -- 172.21.15.78:0/1770135111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 msgr2=0x5591aee51840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:15:47.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.213+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/1770135111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee51840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.214+0000 7f8c2ed9d640 1 -- 172.21.15.78:0/1770135111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 msgr2=0x5591aee54170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:47.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.214+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/1770135111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 0x5591aee54170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.214+0000 7f8c2ed9d640 1 -- 172.21.15.78:0/1770135111 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591aeb09650 con 0x5591aee454d0 2024-09-11T01:15:47.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.214+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/1770135111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 0x5591aee458b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009fd0 tx=0x7f8c280118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10711e5723399caa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:47.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.214+0000 7f8c277fe640 1 -- 172.21.15.78:0/1305911281 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8c200037a0 con 0x7f8c30101c90 2024-09-11T01:15:47.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.214+0000 7f8c24ff9640 1 -- 172.21.15.78:0/1770135111 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28004d20 con 0x5591aee454d0 2024-09-11T01:15:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.215+0000 7f8c24ff9640 1 -- 172.21.15.78:0/1770135111 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c28004050 con 0x5591aee454d0 2024-09-11T01:15:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.215+0000 7f8c277fe640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8c1c0b5850 0x7f8c1c0b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.215+0000 7f8c2e59c640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8c1c0b5850 0x7f8c1c0b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.215+0000 7f8c24ff9640 1 -- 172.21.15.78:0/1770135111 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28005280 con 0x5591aee454d0 2024-09-11T01:15:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.216+0000 7f8c370af740 1 -- 172.21.15.78:0/1770135111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 msgr2=0x5591aee458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.216+0000 7f8c370af740 1 --2- 172.21.15.78:0/1770135111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 0x5591aee458b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009fd0 tx=0x7f8c280118d0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.216+0000 7f8c277fe640 1 -- 172.21.15.78:0/1305911281 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f8c2000cbe0 con 0x7f8c30101c90 2024-09-11T01:15:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.216+0000 7f8c2e59c640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8c1c0b5850 0x7f8c1c0b7d10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x5591aee441e0 tx=0x7f8c200a8080 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.216+0000 7f8c370af740 1 -- 172.21.15.78:0/1770135111 shutdown_connections 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.216+0000 7f8c370af740 1 --2- 172.21.15.78:0/1770135111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 0x5591aee54170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.216+0000 7f8c370af740 1 --2- 172.21.15.78:0/1770135111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee51840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.216+0000 7f8c370af740 1 --2- 172.21.15.78:0/1770135111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 0x5591aee458b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.216+0000 7f8c370af740 1 -- 172.21.15.78:0/1770135111 >> 172.21.15.78:0/1770135111 conn(0x5591aee43300 msgr2=0x5591aee444a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.217+0000 7f8c370af740 1 -- 172.21.15.78:0/1770135111 shutdown_connections 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.217+0000 7f8c370af740 1 -- 172.21.15.78:0/1770135111 wait complete. 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.217+0000 7f8c370af740 1 -- start start 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.217+0000 7f8c370af740 1 Processor -- start 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c370af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee454d0 0x5591aee51610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c370af740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee4ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c370af740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee51d80 0x5591aee4f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c370af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5591aec0c700 con 0x5591aee454d0 2024-09-11T01:15:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c370af740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591aed46160 con 0x5591aee51d80 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c370af740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591aee54aa0 con 0x5591aee45df0 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c2ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee454d0 0x5591aee51610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c370af740 10 client.0 _ll_register_callbacks cb 0x5591aed5f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c27fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee4ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c27fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee4ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:49052/0 (socket says 172.21.15.78:49052) 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c27fff640 1 -- 172.21.15.78:0/4170939295 learned_addr learned my addr 172.21.15.78:0/4170939295 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/4170939295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee51d80 0x5591aee4f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c27fff640 1 -- 172.21.15.78:0/4170939295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee51d80 msgr2=0x5591aee4f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c27fff640 1 --2- 172.21.15.78:0/4170939295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee51d80 0x5591aee4f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c27fff640 1 -- 172.21.15.78:0/4170939295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee454d0 msgr2=0x5591aee51610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c27fff640 1 --2- 172.21.15.78:0/4170939295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee454d0 0x5591aee51610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.218+0000 7f8c27fff640 1 -- 172.21.15.78:0/4170939295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591aed73c90 con 0x5591aee45df0 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.219+0000 7f8c27fff640 1 --2- 172.21.15.78:0/4170939295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee4ea60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8c18005b60 tx=0x7f8c18004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6660aff312358c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.219+0000 7f8c15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:15:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.219+0000 7f8c15ffb640 1 -- 172.21.15.78:0/4170939295 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18037070 con 0x5591aee45df0 2024-09-11T01:15:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.219+0000 7f8c370af740 1 -- 172.21.15.78:0/4170939295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c28004050 con 0x5591aee45df0 2024-09-11T01:15:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.219+0000 7f8c15ffb640 1 -- 172.21.15.78:0/4170939295 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f8c18030650 con 0x5591aee45df0 2024-09-11T01:15:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.219+0000 7f8c15ffb640 1 -- 172.21.15.78:0/4170939295 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f8c1804b050 con 0x5591aee45df0 2024-09-11T01:15:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.219+0000 7f8c370af740 10 client.27625 fetch_fsmap learned FSMap version 69 2024-09-11T01:15:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.219+0000 7f8c370af740 1 -- 172.21.15.78:0/4170939295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5591aee603d0 con 0x5591aee45df0 2024-09-11T01:15:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.220+0000 7f8c15ffb640 1 -- 172.21.15.78:0/4170939295 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c28004050 con 0x5591aee45df0 2024-09-11T01:15:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.221+0000 7f8c15ffb640 1 -- 172.21.15.78:0/4170939295 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 69) ==== 8366+0+0 (secure 0 0 0) 0x7f8c18004ae0 con 0x5591aee45df0 2024-09-11T01:15:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.221+0000 7f8c370af740 10 client.27625 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T01:15:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.221+0000 7f8c370af740 10 client.27625 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:15:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.221+0000 7f8c370af740 20 client.27625 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.221+0000 7f8c370af740 1 --2- 172.21.15.78:0/4170939295 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5591aee618a0 0x5591aee63c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.221+0000 7f8c370af740 4 client.27625 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:15:47.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.221+0000 7f8c370af740 1 -- 172.21.15.78:0/4170939295 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591aedbddf0 con 0x5591aee618a0 2024-09-11T01:15:47.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.257+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/4170939295 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5591aee618a0 0x5591aee63c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.257+0000 7f8c15ffb640 10 client.27625 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:15:47.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.257+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/4170939295 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5591aee618a0 0x5591aee63c90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8f66be96873c79a4 server_cookie=a40922df977f288e in_seq=0 out_seq=0 2024-09-11T01:15:47.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.258+0000 7f8c15ffb640 1 -- 172.21.15.78:0/4170939295 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591aedbddf0 con 0x5591aee618a0 2024-09-11T01:15:47.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.258+0000 7f8c15ffb640 10 client.27625 handle_command_reply: tid=0 2024-09-11T01:15:47.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.258+0000 7f8c370af740 1 client.27625 shutdown 2024-09-11T01:15:47.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.259+0000 7f8c370af740 1 -- 172.21.15.78:0/4170939295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 msgr2=0x5591aee4ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:47.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.259+0000 7f8c370af740 1 --2- 172.21.15.78:0/4170939295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee4ea60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8c18005b60 tx=0x7f8c18004270 comp rx=0 tx=0).stop 2024-09-11T01:15:47.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.259+0000 7f8c370af740 1 -- 172.21.15.78:0/4170939295 shutdown_connections 2024-09-11T01:15:47.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.259+0000 7f8c370af740 1 --2- 172.21.15.78:0/4170939295 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5591aee618a0 0x5591aee63c90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.259+0000 7f8c370af740 1 --2- 172.21.15.78:0/4170939295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee51d80 0x5591aee4f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.259+0000 7f8c15ffb640 0 client.27625 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:15:47.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.259+0000 7f8c370af740 1 --2- 172.21.15.78:0/4170939295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee4ea60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.259+0000 7f8c370af740 1 --2- 172.21.15.78:0/4170939295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee454d0 0x5591aee51610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.259+0000 7f8c370af740 1 -- 172.21.15.78:0/4170939295 >> 172.21.15.78:0/4170939295 conn(0x5591aee43300 msgr2=0x5591aee48e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:47.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.259+0000 7f8c370af740 1 -- 172.21.15.78:0/4170939295 shutdown_connections 2024-09-11T01:15:47.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.260+0000 7f8c370af740 1 -- 172.21.15.78:0/4170939295 wait complete. 2024-09-11T01:15:47.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.260+0000 7f8c370af740 20 client.27625 trim_cache size 0 max 16384 2024-09-11T01:15:47.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.282+0000 7f8c370af740 1 Processor -- start 2024-09-11T01:15:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.282+0000 7f8c370af740 1 -- start start 2024-09-11T01:15:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c370af740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee454d0 0x5591aee4bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c370af740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee45df0 0x5591aee4c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c370af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 0x5591aee69560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c370af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301b4c60 con 0x5591aee51d80 2024-09-11T01:15:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c370af740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301b67a0 con 0x5591aee45df0 2024-09-11T01:15:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c370af740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301b5330 con 0x5591aee454d0 2024-09-11T01:15:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c2ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee454d0 0x5591aee4bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c2ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 0x5591aee69560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c2ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee454d0 0x5591aee4bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:49062/0 (socket says 172.21.15.78:49062) 2024-09-11T01:15:47.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c2ed9d640 1 -- 172.21.15.78:0/1642212070 learned_addr learned my addr 172.21.15.78:0/1642212070 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:47.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c27fff640 1 --2- 172.21.15.78:0/1642212070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee45df0 0x5591aee4c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c2ed9d640 1 -- 172.21.15.78:0/1642212070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee45df0 msgr2=0x5591aee4c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:47.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/1642212070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee45df0 0x5591aee4c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c2ed9d640 1 -- 172.21.15.78:0/1642212070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 msgr2=0x5591aee69560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:47.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/1642212070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 0x5591aee69560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.283+0000 7f8c2ed9d640 1 -- 172.21.15.78:0/1642212070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591aee49780 con 0x5591aee454d0 2024-09-11T01:15:47.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.284+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/1642212070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee454d0 0x5591aee4bc70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009de0 tx=0x7f8c28009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb26b7d2c94abeb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:47.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.284+0000 7f8c17fff640 1 -- 172.21.15.78:0/1642212070 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28003da0 con 0x5591aee454d0 2024-09-11T01:15:47.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.285+0000 7f8c17fff640 1 -- 172.21.15.78:0/1642212070 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c28004050 con 0x5591aee454d0 2024-09-11T01:15:47.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.285+0000 7f8c17fff640 1 -- 172.21.15.78:0/1642212070 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28005020 con 0x5591aee454d0 2024-09-11T01:15:47.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.285+0000 7f8c370af740 1 -- 172.21.15.78:0/1642212070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee454d0 msgr2=0x5591aee4bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:47.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.285+0000 7f8c370af740 1 --2- 172.21.15.78:0/1642212070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee454d0 0x5591aee4bc70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009de0 tx=0x7f8c28009be0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.286+0000 7f8c370af740 1 -- 172.21.15.78:0/1642212070 shutdown_connections 2024-09-11T01:15:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.286+0000 7f8c370af740 1 --2- 172.21.15.78:0/1642212070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 0x5591aee69560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.286+0000 7f8c370af740 1 --2- 172.21.15.78:0/1642212070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee45df0 0x5591aee4c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.286+0000 7f8c370af740 1 --2- 172.21.15.78:0/1642212070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee454d0 0x5591aee4bc70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.286+0000 7f8c370af740 1 -- 172.21.15.78:0/1642212070 >> 172.21.15.78:0/1642212070 conn(0x5591aee43300 msgr2=0x5591aee4d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.286+0000 7f8c370af740 1 -- 172.21.15.78:0/1642212070 shutdown_connections 2024-09-11T01:15:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.286+0000 7f8c370af740 1 -- 172.21.15.78:0/1642212070 wait complete. 2024-09-11T01:15:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.286+0000 7f8c370af740 1 -- start start 2024-09-11T01:15:47.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.286+0000 7f8c370af740 1 Processor -- start 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.287+0000 7f8c370af740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 0x5591aee5a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.287+0000 7f8c370af740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee47090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.287+0000 7f8c370af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 0x5591aee475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.287+0000 7f8c370af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301b60d0 con 0x5591aee51d80 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.287+0000 7f8c370af740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301b5a00 con 0x5591aee454d0 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.287+0000 7f8c370af740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301b2c60 con 0x5591aee45df0 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.287+0000 7f8c370af740 10 client.0 _ll_register_callbacks cb 0x7f8c20005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.287+0000 7f8c27fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee47090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.287+0000 7f8c2ed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 0x5591aee5a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.287+0000 7f8c27fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee47090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:49078/0 (socket says 172.21.15.78:49078) 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.287+0000 7f8c27fff640 1 -- 172.21.15.78:0/1618583805 learned_addr learned my addr 172.21.15.78:0/1618583805 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.287+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 0x5591aee475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.288+0000 7f8c27fff640 1 -- 172.21.15.78:0/1618583805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 msgr2=0x5591aee5a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.288+0000 7f8c27fff640 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 0x5591aee5a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.288+0000 7f8c27fff640 1 -- 172.21.15.78:0/1618583805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 msgr2=0x5591aee475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.288+0000 7f8c27fff640 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 0x5591aee475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.288+0000 7f8c27fff640 1 -- 172.21.15.78:0/1618583805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591aee47b90 con 0x5591aee45df0 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.288+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 0x5591aee5a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.288+0000 7f8c27fff640 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee47090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8c18004e80 tx=0x7f8c18003940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54273e27e827cc7b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.288+0000 7f8c15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.288+0000 7f8c15ffb640 1 -- 172.21.15.78:0/1618583805 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18053070 con 0x5591aee45df0 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.288+0000 7f8c370af740 1 -- 172.21.15.78:0/1618583805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c28004050 con 0x5591aee45df0 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.288+0000 7f8c15ffb640 1 -- 172.21.15.78:0/1618583805 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f8c18003db0 con 0x5591aee45df0 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.289+0000 7f8c15ffb640 1 -- 172.21.15.78:0/1618583805 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f8c18037040 con 0x5591aee45df0 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.289+0000 7f8c370af740 10 client.27631 fetch_fsmap learned FSMap version 69 2024-09-11T01:15:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.289+0000 7f8c370af740 1 -- 172.21.15.78:0/1618583805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5591aeee0570 con 0x5591aee45df0 2024-09-11T01:15:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.289+0000 7f8c15ffb640 1 -- 172.21.15.78:0/1618583805 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c28004050 con 0x5591aee45df0 2024-09-11T01:15:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.289+0000 7f8c15ffb640 1 -- 172.21.15.78:0/1618583805 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 69) ==== 8366+0+0 (secure 0 0 0) 0x7f8c18005000 con 0x5591aee45df0 2024-09-11T01:15:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.290+0000 7f8c370af740 10 client.27631 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T01:15:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.290+0000 7f8c370af740 10 client.27631 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:15:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.290+0000 7f8c370af740 20 client.27631 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.290+0000 7f8c370af740 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5591aee618a0 0x5591aeee2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.290+0000 7f8c370af740 4 client.27631 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:15:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.290+0000 7f8c370af740 1 -- 172.21.15.78:0/1618583805 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591aedb98a0 con 0x5591aee618a0 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.292+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5591aee618a0 0x5591aeee2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.292+0000 7f8c2ed9d640 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5591aee618a0 0x5591aeee2d90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=505c5b367d609a48 server_cookie=47846938ee1fe56f in_seq=0 out_seq=0 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.292+0000 7f8c15ffb640 10 client.27631 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.293+0000 7f8c15ffb640 1 -- 172.21.15.78:0/1618583805 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591aedb98a0 con 0x5591aee618a0 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.293+0000 7f8c15ffb640 10 client.27631 handle_command_reply: tid=0 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.293+0000 7f8c370af740 1 client.27631 shutdown 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.293+0000 7f8c370af740 1 -- 172.21.15.78:0/1618583805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 msgr2=0x5591aee47090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.293+0000 7f8c370af740 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee47090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8c18004e80 tx=0x7f8c18003940 comp rx=0 tx=0).stop 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.293+0000 7f8c370af740 1 -- 172.21.15.78:0/1618583805 shutdown_connections 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.293+0000 7f8c370af740 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5591aee618a0 0x5591aeee2d90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.293+0000 7f8c370af740 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591aee51d80 0x5591aee475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.293+0000 7f8c370af740 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591aee45df0 0x5591aee47090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.293+0000 7f8c370af740 1 --2- 172.21.15.78:0/1618583805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591aee454d0 0x5591aee5a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.294+0000 7f8c370af740 1 -- 172.21.15.78:0/1618583805 >> 172.21.15.78:0/1618583805 conn(0x5591aee43300 msgr2=0x5591aee41c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.294+0000 7f8c15ffb640 0 client.27631 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:15:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.294+0000 7f8c370af740 1 -- 172.21.15.78:0/1618583805 shutdown_connections 2024-09-11T01:15:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.294+0000 7f8c370af740 1 -- 172.21.15.78:0/1618583805 wait complete. 2024-09-11T01:15:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.294+0000 7f8c370af740 20 client.27631 trim_cache size 0 max 16384 2024-09-11T01:15:47.300 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:15:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.295+0000 7f8c34cda640 1 -- 172.21.15.78:0/1305911281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8c1c0b5850 msgr2=0x7f8c1c0b7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.295+0000 7f8c34cda640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8c1c0b5850 0x7f8c1c0b7d10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x5591aee441e0 tx=0x7f8c200a8080 comp rx=0 tx=0).stop 2024-09-11T01:15:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.295+0000 7f8c34cda640 1 -- 172.21.15.78:0/1305911281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c30101c90 msgr2=0x7f8c3019f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.295+0000 7f8c34cda640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c30101c90 0x7f8c3019f840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000e9c0 tx=0x7f8c2000ee80 comp rx=0 tx=0).stop 2024-09-11T01:15:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.296+0000 7f8c34cda640 1 -- 172.21.15.78:0/1305911281 shutdown_connections 2024-09-11T01:15:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.296+0000 7f8c34cda640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8c1c0b5850 0x7f8c1c0b7d10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.296+0000 7f8c34cda640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c30108020 0x7f8c301a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.296+0000 7f8c34cda640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c301076a0 0x7f8c3019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.296+0000 7f8c34cda640 1 --2- 172.21.15.78:0/1305911281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c30101c90 0x7f8c3019f840 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.296+0000 7f8c34cda640 1 -- 172.21.15.78:0/1305911281 >> 172.21.15.78:0/1305911281 conn(0x7f8c300fd700 msgr2=0x7f8c300ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.296+0000 7f8c34cda640 1 -- 172.21.15.78:0/1305911281 shutdown_connections 2024-09-11T01:15:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:47.296+0000 7f8c34cda640 1 -- 172.21.15.78:0/1305911281 wait complete. 2024-09-11T01:15:47.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 157779 36.99 MB/sec execute 113 sec latency 71.350 ms 2024-09-11T01:15:48.093 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:48 smithi078 ceph-mon[31315]: pgmap v850: 97 pgs: 97 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 12 MiB/s wr, 794 op/s 2024-09-11T01:15:48.133 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:15:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:48 smithi110 ceph-mon[34712]: pgmap v850: 97 pgs: 97 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 12 MiB/s wr, 794 op/s 2024-09-11T01:15:48.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:48 smithi112 ceph-mon[35384]: pgmap v850: 97 pgs: 97 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 12 MiB/s wr, 794 op/s 2024-09-11T01:15:48.559 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:15:48.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 158635 37.16 MB/sec execute 114 sec latency 67.003 ms 2024-09-11T01:15:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.053+0000 7f0c781cf640 1 -- 172.21.15.78:0/1629324947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c701087d0 msgr2=0x7f0c70108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.053+0000 7f0c781cf640 1 --2- 172.21.15.78:0/1629324947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c701087d0 0x7f0c70108c10 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009900 tx=0x7f0c6402a410 comp rx=0 tx=0).stop 2024-09-11T01:15:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.053+0000 7f0c781cf640 1 -- 172.21.15.78:0/1629324947 shutdown_connections 2024-09-11T01:15:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.053+0000 7f0c781cf640 1 --2- 172.21.15.78:0/1629324947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c70109150 0x7f0c7010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.053+0000 7f0c781cf640 1 --2- 172.21.15.78:0/1629324947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c701087d0 0x7f0c70108c10 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.053+0000 7f0c781cf640 1 --2- 172.21.15.78:0/1629324947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c70102dc0 0x7f0c701031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.053+0000 7f0c781cf640 1 -- 172.21.15.78:0/1629324947 >> 172.21.15.78:0/1629324947 conn(0x7f0c700fe870 msgr2=0x7f0c70100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.053+0000 7f0c781cf640 1 -- 172.21.15.78:0/1629324947 shutdown_connections 2024-09-11T01:15:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.053+0000 7f0c781cf640 1 -- 172.21.15.78:0/1629324947 wait complete. 2024-09-11T01:15:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.053+0000 7f0c781cf640 1 Processor -- start 2024-09-11T01:15:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.053+0000 7f0c781cf640 1 -- start start 2024-09-11T01:15:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.053+0000 7f0c781cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70102dc0 0x7f0c700754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c781cf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c701087d0 0x7f0c700759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c781cf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c70109150 0x7f0c70075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c781cf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c70112e10 con 0x7f0c70102dc0 2024-09-11T01:15:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c781cf640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c70112c90 con 0x7f0c701087d0 2024-09-11T01:15:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c781cf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c70112f90 con 0x7f0c70109150 2024-09-11T01:15:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c75f6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70102dc0 0x7f0c700754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c75f6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70102dc0 0x7f0c700754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47816/0 (socket says 172.21.15.78:47816) 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c75f6b640 1 -- 172.21.15.78:0/58260791 learned_addr learned my addr 172.21.15.78:0/58260791 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c7576a640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c701087d0 0x7f0c700759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c7676c640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c70109150 0x7f0c70075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c75f6b640 1 -- 172.21.15.78:0/58260791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c70109150 msgr2=0x7f0c70075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c75f6b640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c70109150 0x7f0c70075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c75f6b640 1 -- 172.21.15.78:0/58260791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c701087d0 msgr2=0x7f0c700759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c75f6b640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c701087d0 0x7f0c700759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c75f6b640 1 -- 172.21.15.78:0/58260791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c701a7680 con 0x7f0c70102dc0 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c7576a640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c701087d0 0x7f0c700759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c75f6b640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70102dc0 0x7f0c700754a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000eab0 tx=0x7f0c6000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c6effd640 1 -- 172.21.15.78:0/58260791 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6000ce40 con 0x7f0c70102dc0 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c6effd640 1 -- 172.21.15.78:0/58260791 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c60011920 con 0x7f0c70102dc0 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c6effd640 1 -- 172.21.15.78:0/58260791 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60010a30 con 0x7f0c70102dc0 2024-09-11T01:15:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.054+0000 7f0c781cf640 1 -- 172.21.15.78:0/58260791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c701a7970 con 0x7f0c70102dc0 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.055+0000 7f0c781cf640 1 -- 172.21.15.78:0/58260791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c701a7ed0 con 0x7f0c70102dc0 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.058+0000 7f0c6effd640 1 -- 172.21.15.78:0/58260791 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0c6000f040 con 0x7f0c70102dc0 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.058+0000 7f0c7a5a4740 1 Processor -- start 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.058+0000 7f0c7a5a4740 1 -- start start 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.059+0000 7f0c7a5a4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e39b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.059+0000 7f0c7a5a4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 0x56158e3b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.059+0000 7f0c7a5a4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e3b0590 0x56158e3b2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.059+0000 7f0c7a5a4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56158e311ea0 con 0x56158e39b4d0 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.059+0000 7f0c7a5a4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56158e3012d0 con 0x56158e39bdf0 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.059+0000 7f0c7a5a4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56158e3050e0 con 0x56158e3b0590 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.059+0000 7f0c7676c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e39b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.059+0000 7f0c7676c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e39b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47820/0 (socket says 172.21.15.78:47820) 2024-09-11T01:15:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.059+0000 7f0c7676c640 1 -- 172.21.15.78:0/3570971000 learned_addr learned my addr 172.21.15.78:0/3570971000 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.059+0000 7f0c7576a640 1 --2- 172.21.15.78:0/3570971000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e3b0590 0x56158e3b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.059+0000 7f0c75f6b640 1 --2- 172.21.15.78:0/3570971000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 0x56158e3b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.060+0000 7f0c7576a640 1 -- 172.21.15.78:0/3570971000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 msgr2=0x56158e3b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.060+0000 7f0c7576a640 1 --2- 172.21.15.78:0/3570971000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 0x56158e3b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.060+0000 7f0c7576a640 1 -- 172.21.15.78:0/3570971000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 msgr2=0x56158e39b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.060+0000 7f0c7576a640 1 --2- 172.21.15.78:0/3570971000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e39b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.060+0000 7f0c7576a640 1 -- 172.21.15.78:0/3570971000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56158e05f650 con 0x56158e3b0590 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.060+0000 7f0c7676c640 1 --2- 172.21.15.78:0/3570971000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e39b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.060+0000 7f0c7576a640 1 --2- 172.21.15.78:0/3570971000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e3b0590 0x56158e3b2980 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009230 tx=0x7f0c64032cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8695e4189b0ece2d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.061+0000 7f0c57fff640 1 -- 172.21.15.78:0/3570971000 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64035030 con 0x56158e3b0590 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.061+0000 7f0c6effd640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c580b5550 0x7f0c580b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.061+0000 7f0c57fff640 1 -- 172.21.15.78:0/3570971000 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c640316a0 con 0x56158e3b0590 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.061+0000 7f0c57fff640 1 -- 172.21.15.78:0/3570971000 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c640046c0 con 0x56158e3b0590 2024-09-11T01:15:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.061+0000 7f0c7676c640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c580b5550 0x7f0c580b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.061+0000 7f0c6effd640 1 -- 172.21.15.78:0/58260791 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f0c600e6320 con 0x7f0c70102dc0 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.062+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3570971000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e3b0590 msgr2=0x56158e3b2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.062+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/3570971000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e3b0590 0x56158e3b2980 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009230 tx=0x7f0c64032cd0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.062+0000 7f0c7676c640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c580b5550 0x7f0c580b7a10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005fd0 tx=0x7f0c6800b520 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.062+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3570971000 shutdown_connections 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.062+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/3570971000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e3b0590 0x56158e3b2980 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.062+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/3570971000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 0x56158e3b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.062+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/3570971000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e39b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.062+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3570971000 >> 172.21.15.78:0/3570971000 conn(0x56158e399300 msgr2=0x56158e39a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.062+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3570971000 shutdown_connections 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.062+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3570971000 wait complete. 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.063+0000 7f0c7a5a4740 1 -- start start 2024-09-11T01:15:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.063+0000 7f0c7a5a4740 1 Processor -- start 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c7a5a4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39b4d0 0x56158e3afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c7a5a4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 0x56158e3ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c7a5a4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e3b0590 0x56158e3ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c7a5a4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56158e162700 con 0x56158e3b0590 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c7a5a4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56158e29c160 con 0x56158e39bdf0 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c7576a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 0x56158e3ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c75f6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39b4d0 0x56158e3afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c75f6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39b4d0 0x56158e3afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:49118/0 (socket says 172.21.15.78:49118) 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c75f6b640 1 -- 172.21.15.78:0/4111217526 learned_addr learned my addr 172.21.15.78:0/4111217526 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c7a5a4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56158e3b31e0 con 0x56158e39b4d0 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c7676c640 1 --2- 172.21.15.78:0/4111217526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e3b0590 0x56158e3ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c75f6b640 1 -- 172.21.15.78:0/4111217526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 msgr2=0x56158e3ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c75f6b640 1 --2- 172.21.15.78:0/4111217526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 0x56158e3ad2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c75f6b640 1 -- 172.21.15.78:0/4111217526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e3b0590 msgr2=0x56158e3ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c75f6b640 1 --2- 172.21.15.78:0/4111217526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e3b0590 0x56158e3ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.064+0000 7f0c75f6b640 1 -- 172.21.15.78:0/4111217526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56158e2c9c90 con 0x56158e39b4d0 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.065+0000 7f0c75f6b640 1 --2- 172.21.15.78:0/4111217526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39b4d0 0x56158e3afe90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0c600e39f0 tx=0x7f0c60011d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff6ab148ed93723b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.065+0000 7f0c557fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.065+0000 7f0c7a5a4740 10 client.0 _ll_register_callbacks cb 0x56158e2b59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.065+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/4111217526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c640316a0 con 0x56158e39b4d0 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.065+0000 7f0c557fa640 1 -- 172.21.15.78:0/4111217526 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c600ac120 con 0x56158e39b4d0 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.065+0000 7f0c557fa640 1 -- 172.21.15.78:0/4111217526 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f0c600b5880 con 0x56158e39b4d0 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.065+0000 7f0c557fa640 1 -- 172.21.15.78:0/4111217526 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f0c600a8120 con 0x56158e39b4d0 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.065+0000 7f0c7a5a4740 10 client.27643 fetch_fsmap learned FSMap version 69 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.065+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/4111217526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=69}) -- 0x56158e3be9c0 con 0x56158e39b4d0 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.065+0000 7f0c557fa640 1 -- 172.21.15.78:0/4111217526 <== mon.1 v2:172.21.15.112:3300/0 4 ==== fsmap(e 69) ==== 8366+0+0 (secure 0 0 0) 0x7f0c600e9ba0 con 0x56158e39b4d0 2024-09-11T01:15:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.066+0000 7f0c7a5a4740 10 client.27643 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.066+0000 7f0c7a5a4740 10 client.27643 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.066+0000 7f0c7a5a4740 20 client.27643 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.066+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/4111217526 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x56158e3bfea0 0x56158e3c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.066+0000 7f0c7a5a4740 4 client.27643 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.066+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/4111217526 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56158e313df0 con 0x56158e3bfea0 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.066+0000 7f0c7576a640 1 --2- 172.21.15.78:0/4111217526 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x56158e3bfea0 0x56158e3c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.066+0000 7f0c7576a640 1 --2- 172.21.15.78:0/4111217526 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x56158e3bfea0 0x56158e3c22f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f7a0cd1edd9896b server_cookie=94b57a24d9f8a1c7 in_seq=0 out_seq=0 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.066+0000 7f0c557fa640 10 client.27643 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.067+0000 7f0c557fa640 1 -- 172.21.15.78:0/4111217526 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56158e313df0 con 0x56158e3bfea0 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.067+0000 7f0c557fa640 10 client.27643 handle_command_reply: tid=0 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.067+0000 7f0c7a5a4740 1 client.27643 shutdown 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.068+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/4111217526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39b4d0 msgr2=0x56158e3afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.068+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/4111217526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39b4d0 0x56158e3afe90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0c600e39f0 tx=0x7f0c60011d00 comp rx=0 tx=0).stop 2024-09-11T01:15:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.068+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/4111217526 shutdown_connections 2024-09-11T01:15:49.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.068+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/4111217526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e3b0590 0x56158e3ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.068+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/4111217526 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x56158e3bfea0 0x56158e3c22f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.068+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/4111217526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 0x56158e3ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.068+0000 7f0c557fa640 0 client.27643 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:15:49.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.068+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/4111217526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39b4d0 0x56158e3afe90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.068+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/4111217526 >> 172.21.15.78:0/4111217526 conn(0x56158e399300 msgr2=0x56158e39eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:49.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.068+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/4111217526 shutdown_connections 2024-09-11T01:15:49.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.068+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/4111217526 wait complete. 2024-09-11T01:15:49.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.068+0000 7f0c7a5a4740 20 client.27643 trim_cache size 0 max 16384 2024-09-11T01:15:49.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.088+0000 7f0c7a5a4740 1 Processor -- start 2024-09-11T01:15:49.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.088+0000 7f0c7a5a4740 1 -- start start 2024-09-11T01:15:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.088+0000 7f0c7a5a4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e399300 0x56158e3bc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.089+0000 7f0c7a5a4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e398c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.089+0000 7f0c7a5a4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39bdf0 0x56158e394320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.089+0000 7f0c7a5a4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c701ac1e0 con 0x56158e39b4d0 2024-09-11T01:15:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.089+0000 7f0c7a5a4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c701add20 con 0x56158e399300 2024-09-11T01:15:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.089+0000 7f0c7a5a4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c701ac8b0 con 0x56158e39bdf0 2024-09-11T01:15:49.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.089+0000 7f0c7676c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39bdf0 0x56158e394320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.089+0000 7f0c75f6b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e399300 0x56158e3bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.090+0000 7f0c75f6b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e399300 0x56158e3bc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:49296/0 (socket says 172.21.15.78:49296) 2024-09-11T01:15:49.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.090+0000 7f0c75f6b640 1 -- 172.21.15.78:0/2271737360 learned_addr learned my addr 172.21.15.78:0/2271737360 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.090+0000 7f0c7676c640 1 -- 172.21.15.78:0/2271737360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e399300 msgr2=0x56158e3bc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.090+0000 7f0c7676c640 1 --2- 172.21.15.78:0/2271737360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e399300 0x56158e3bc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.090+0000 7f0c7676c640 1 -- 172.21.15.78:0/2271737360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 msgr2=0x56158e398c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.090+0000 7f0c7676c640 1 --2- 172.21.15.78:0/2271737360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e398c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.090+0000 7f0c7676c640 1 -- 172.21.15.78:0/2271737360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56158e3ab7e0 con 0x56158e39bdf0 2024-09-11T01:15:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.091+0000 7f0c7676c640 1 --2- 172.21.15.78:0/2271737360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39bdf0 0x56158e394320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800bd20 tx=0x7f0c68015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dba237e57f85e36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:49.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.091+0000 7f0c577fe640 1 -- 172.21.15.78:0/2271737360 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6802c070 con 0x56158e39bdf0 2024-09-11T01:15:49.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.091+0000 7f0c577fe640 1 -- 172.21.15.78:0/2271737360 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c68027410 con 0x56158e39bdf0 2024-09-11T01:15:49.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.091+0000 7f0c577fe640 1 -- 172.21.15.78:0/2271737360 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6801d660 con 0x56158e39bdf0 2024-09-11T01:15:49.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.092+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/2271737360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39bdf0 msgr2=0x56158e394320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:49.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.092+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/2271737360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39bdf0 0x56158e394320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800bd20 tx=0x7f0c68015000 comp rx=0 tx=0).stop 2024-09-11T01:15:49.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.092+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/2271737360 shutdown_connections 2024-09-11T01:15:49.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.092+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/2271737360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e39bdf0 0x56158e394320 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.092+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/2271737360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e398c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.092+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/2271737360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e399300 0x56158e3bc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.092+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/2271737360 >> 172.21.15.78:0/2271737360 conn(0x56158e3b1ba0 msgr2=0x56158e39a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:49.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.092+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/2271737360 shutdown_connections 2024-09-11T01:15:49.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.093+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/2271737360 wait complete. 2024-09-11T01:15:49.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.093+0000 7f0c7a5a4740 1 -- start start 2024-09-11T01:15:49.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.093+0000 7f0c7a5a4740 1 Processor -- start 2024-09-11T01:15:49.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.094+0000 7f0c7a5a4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e399300 0x56158e3b8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.094+0000 7f0c7a5a4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e3b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.094+0000 7f0c7a5a4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 0x56158e3b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.094+0000 7f0c7a5a4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c701ad650 con 0x56158e39b4d0 2024-09-11T01:15:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.094+0000 7f0c7a5a4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c701acf80 con 0x56158e39bdf0 2024-09-11T01:15:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.094+0000 7f0c7a5a4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c701aa1e0 con 0x56158e399300 2024-09-11T01:15:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.094+0000 7f0c7a5a4740 10 client.0 _ll_register_callbacks cb 0x7f0c6000ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.094+0000 7f0c7576a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e3b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.094+0000 7f0c7576a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e3b8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47844/0 (socket says 172.21.15.78:47844) 2024-09-11T01:15:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.094+0000 7f0c7576a640 1 -- 172.21.15.78:0/3818291053 learned_addr learned my addr 172.21.15.78:0/3818291053 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.094+0000 7f0c75f6b640 1 --2- 172.21.15.78:0/3818291053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e399300 0x56158e3b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.094+0000 7f0c7576a640 1 -- 172.21.15.78:0/3818291053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e399300 msgr2=0x56158e3b8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.095+0000 7f0c7576a640 1 --2- 172.21.15.78:0/3818291053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e399300 0x56158e3b8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.095+0000 7f0c7576a640 1 -- 172.21.15.78:0/3818291053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 msgr2=0x56158e3b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.095+0000 7f0c7576a640 1 --2- 172.21.15.78:0/3818291053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 0x56158e3b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.095+0000 7f0c7576a640 1 -- 172.21.15.78:0/3818291053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56158e3b5c70 con 0x56158e39b4d0 2024-09-11T01:15:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.095+0000 7f0c7576a640 1 --2- 172.21.15.78:0/3818291053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e3b8c40 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f0c64005390 tx=0x7f0c64032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb3378ce13c8b08 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.095+0000 7f0c557fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:15:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.095+0000 7f0c557fa640 1 -- 172.21.15.78:0/3818291053 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64035040 con 0x56158e39b4d0 2024-09-11T01:15:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.095+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3818291053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c68027410 con 0x56158e39b4d0 2024-09-11T01:15:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.095+0000 7f0c557fa640 1 -- 172.21.15.78:0/3818291053 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f0c64031c10 con 0x56158e39b4d0 2024-09-11T01:15:49.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.096+0000 7f0c557fa640 1 -- 172.21.15.78:0/3818291053 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f0c6405b050 con 0x56158e39b4d0 2024-09-11T01:15:49.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.096+0000 7f0c7a5a4740 10 client.18546 fetch_fsmap learned FSMap version 69 2024-09-11T01:15:49.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.096+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3818291053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x56158e42d8e0 con 0x56158e39b4d0 2024-09-11T01:15:49.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.096+0000 7f0c557fa640 1 -- 172.21.15.78:0/3818291053 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c68027410 con 0x56158e39b4d0 2024-09-11T01:15:49.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.096+0000 7f0c557fa640 1 -- 172.21.15.78:0/3818291053 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 69) ==== 8366+0+0 (secure 0 0 0) 0x7f0c64003a60 con 0x56158e39b4d0 2024-09-11T01:15:49.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.096+0000 7f0c7a5a4740 10 client.18546 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T01:15:49.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.096+0000 7f0c7a5a4740 10 client.18546 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:15:49.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.096+0000 7f0c7a5a4740 20 client.18546 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:49.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.096+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/3818291053 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x56158e3b0590 0x56158e4300d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:49.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.096+0000 7f0c7a5a4740 4 client.18546 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:15:49.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.096+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3818291053 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56158e30f8a0 con 0x56158e3b0590 2024-09-11T01:15:49.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.097+0000 7f0c75f6b640 1 --2- 172.21.15.78:0/3818291053 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x56158e3b0590 0x56158e4300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:49.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.098+0000 7f0c75f6b640 1 --2- 172.21.15.78:0/3818291053 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x56158e3b0590 0x56158e4300d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=497e43065acd055d server_cookie=6f38c23531cafca6 in_seq=0 out_seq=0 2024-09-11T01:15:49.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.098+0000 7f0c557fa640 10 client.18546 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.098+0000 7f0c557fa640 1 -- 172.21.15.78:0/3818291053 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56158e30f8a0 con 0x56158e3b0590 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.098+0000 7f0c557fa640 10 client.18546 handle_command_reply: tid=0 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.098+0000 7f0c7a5a4740 1 client.18546 shutdown 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.100+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3818291053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 msgr2=0x56158e3b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.100+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/3818291053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e3b8c40 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f0c64005390 tx=0x7f0c64032ed0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.100+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3818291053 shutdown_connections 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.100+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/3818291053 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x56158e3b0590 0x56158e4300d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.100+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/3818291053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56158e39bdf0 0x56158e3b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.100+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/3818291053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158e39b4d0 0x56158e3b8c40 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.100+0000 7f0c7a5a4740 1 --2- 172.21.15.78:0/3818291053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158e399300 0x56158e3b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.100+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3818291053 >> 172.21.15.78:0/3818291053 conn(0x56158e3b1ba0 msgr2=0x56158e3ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.101+0000 7f0c557fa640 0 client.18546 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:15:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.101+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3818291053 shutdown_connections 2024-09-11T01:15:49.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.101+0000 7f0c7a5a4740 1 -- 172.21.15.78:0/3818291053 wait complete. 2024-09-11T01:15:49.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.101+0000 7f0c7a5a4740 20 client.18546 trim_cache size 0 max 16384 2024-09-11T01:15:49.105 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:15:49.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.102+0000 7f0c781cf640 1 -- 172.21.15.78:0/58260791 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c580b5550 msgr2=0x7f0c580b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:49.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.102+0000 7f0c781cf640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c580b5550 0x7f0c580b7a10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005fd0 tx=0x7f0c6800b520 comp rx=0 tx=0).stop 2024-09-11T01:15:49.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.102+0000 7f0c781cf640 1 -- 172.21.15.78:0/58260791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70102dc0 msgr2=0x7f0c700754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:49.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.102+0000 7f0c781cf640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70102dc0 0x7f0c700754a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000eab0 tx=0x7f0c6000ef70 comp rx=0 tx=0).stop 2024-09-11T01:15:49.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.103+0000 7f0c781cf640 1 -- 172.21.15.78:0/58260791 shutdown_connections 2024-09-11T01:15:49.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.103+0000 7f0c781cf640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0c580b5550 0x7f0c580b7a10 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.103+0000 7f0c781cf640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c70109150 0x7f0c70075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.103+0000 7f0c781cf640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c701087d0 0x7f0c700759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.103+0000 7f0c781cf640 1 --2- 172.21.15.78:0/58260791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70102dc0 0x7f0c700754a0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:49.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.103+0000 7f0c781cf640 1 -- 172.21.15.78:0/58260791 >> 172.21.15.78:0/58260791 conn(0x7f0c700fe870 msgr2=0x7f0c70100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:49.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.103+0000 7f0c781cf640 1 -- 172.21.15.78:0/58260791 shutdown_connections 2024-09-11T01:15:49.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:49.103+0000 7f0c781cf640 1 -- 172.21.15.78:0/58260791 wait complete. 2024-09-11T01:15:49.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 159116 37.01 MB/sec execute 115 sec latency 97.438 ms 2024-09-11T01:15:50.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:50 smithi110 ceph-mon[34712]: pgmap v851: 97 pgs: 97 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.8 MiB/s wr, 561 op/s 2024-09-11T01:15:50.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:50 smithi112 ceph-mon[35384]: pgmap v851: 97 pgs: 97 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.8 MiB/s wr, 561 op/s 2024-09-11T01:15:50.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:50 smithi078 ceph-mon[31315]: pgmap v851: 97 pgs: 97 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.8 MiB/s wr, 561 op/s 2024-09-11T01:15:50.535 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:15:50.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 159886 37.00 MB/sec execute 116 sec latency 63.561 ms 2024-09-11T01:15:50.903 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:15:51.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:51 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:51.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:51 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:51.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:51 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:15:51.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.579+0000 7f542ce19640 1 -- 172.21.15.78:0/4003985693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5428109170 msgr2=0x7f542810f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:51.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.579+0000 7f542ce19640 1 --2- 172.21.15.78:0/4003985693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5428109170 0x7f542810f970 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f5420009ca0 tx=0x7f542002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.579+0000 7f542ce19640 1 -- 172.21.15.78:0/4003985693 shutdown_connections 2024-09-11T01:15:51.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.579+0000 7f542ce19640 1 --2- 172.21.15.78:0/4003985693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5428109170 0x7f542810f970 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.579+0000 7f542ce19640 1 --2- 172.21.15.78:0/4003985693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54281087f0 0x7f5428108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.579+0000 7f542ce19640 1 --2- 172.21.15.78:0/4003985693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5428102de0 0x7f54281031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.579+0000 7f542ce19640 1 -- 172.21.15.78:0/4003985693 >> 172.21.15.78:0/4003985693 conn(0x7f54280fe830 msgr2=0x7f5428100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:51.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.579+0000 7f542ce19640 1 -- 172.21.15.78:0/4003985693 shutdown_connections 2024-09-11T01:15:51.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.579+0000 7f542ce19640 1 -- 172.21.15.78:0/4003985693 wait complete. 2024-09-11T01:15:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.580+0000 7f542ce19640 1 Processor -- start 2024-09-11T01:15:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.580+0000 7f542ce19640 1 -- start start 2024-09-11T01:15:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f542ce19640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5428102de0 0x7f54281991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f542ce19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54281087f0 0x7f5428199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f542ce19640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5428109170 0x7f542819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f542ce19640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5428112e30 con 0x7f5428102de0 2024-09-11T01:15:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f542ce19640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5428112cb0 con 0x7f5428109170 2024-09-11T01:15:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f542ce19640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5428112fb0 con 0x7f54281087f0 2024-09-11T01:15:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f542659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5428102de0 0x7f54281991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f5426d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5428109170 0x7f542819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f542659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5428102de0 0x7f54281991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47864/0 (socket says 172.21.15.78:47864) 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f542659c640 1 -- 172.21.15.78:0/2804509157 learned_addr learned my addr 172.21.15.78:0/2804509157 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f5425d9b640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54281087f0 0x7f5428199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f5426d9d640 1 -- 172.21.15.78:0/2804509157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54281087f0 msgr2=0x7f5428199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f5426d9d640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54281087f0 0x7f5428199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f5426d9d640 1 -- 172.21.15.78:0/2804509157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5428102de0 msgr2=0x7f54281991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f5426d9d640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5428102de0 0x7f54281991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.581+0000 7f5426d9d640 1 -- 172.21.15.78:0/2804509157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542819f270 con 0x7f5428109170 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.582+0000 7f5426d9d640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5428109170 0x7f542819eaa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f542002ade0 tx=0x7f54200316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.582+0000 7f54177fe640 1 -- 172.21.15.78:0/2804509157 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420003b70 con 0x7f5428109170 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.582+0000 7f54177fe640 1 -- 172.21.15.78:0/2804509157 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54200318e0 con 0x7f5428109170 2024-09-11T01:15:51.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.582+0000 7f542ce19640 1 -- 172.21.15.78:0/2804509157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542819f500 con 0x7f5428109170 2024-09-11T01:15:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.582+0000 7f54177fe640 1 -- 172.21.15.78:0/2804509157 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542002f8a0 con 0x7f5428109170 2024-09-11T01:15:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.582+0000 7f542ce19640 1 -- 172.21.15.78:0/2804509157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54281a5c20 con 0x7f5428109170 2024-09-11T01:15:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.585+0000 7f542f1ee740 1 Processor -- start 2024-09-11T01:15:51.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.586+0000 7f54177fe640 1 -- 172.21.15.78:0/2804509157 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f542002fac0 con 0x7f5428109170 2024-09-11T01:15:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.586+0000 7f542f1ee740 1 -- start start 2024-09-11T01:15:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.586+0000 7f542f1ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 0x55b8166764d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.586+0000 7f542f1ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 0x55b816682460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.586+0000 7f542f1ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8166829a0 0x55b816684d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.586+0000 7f542f1ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b8165ecea0 con 0x55b8166829a0 2024-09-11T01:15:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.586+0000 7f542f1ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b8165dc2d0 con 0x55b816676a10 2024-09-11T01:15:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.586+0000 7f542f1ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b8165e00e0 con 0x55b8166760f0 2024-09-11T01:15:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f5425d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 0x55b8166764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f542659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 0x55b816682460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f542659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 0x55b816682460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:49342/0 (socket says 172.21.15.78:49342) 2024-09-11T01:15:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f5425d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 0x55b8166764d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:49162/0 (socket says 172.21.15.78:49162) 2024-09-11T01:15:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f542659c640 1 -- 172.21.15.78:0/4272835866 learned_addr learned my addr 172.21.15.78:0/4272835866 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f5425d9b640 1 --2- 172.21.15.78:0/4272835866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8166829a0 0x55b816684d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f542659c640 1 -- 172.21.15.78:0/4272835866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 msgr2=0x55b8166764d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f542659c640 1 --2- 172.21.15.78:0/4272835866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 0x55b8166764d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f542659c640 1 -- 172.21.15.78:0/4272835866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8166829a0 msgr2=0x55b816684d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f542659c640 1 --2- 172.21.15.78:0/4272835866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8166829a0 0x55b816684d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f542659c640 1 -- 172.21.15.78:0/4272835866 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b81633a650 con 0x55b816676a10 2024-09-11T01:15:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f5425d9b640 1 --2- 172.21.15.78:0/4272835866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 0x55b8166764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:15:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f54177fe640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54100b5450 0x7f54100b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.587+0000 7f5425d9b640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54100b5450 0x7f54100b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.588+0000 7f542659c640 1 --2- 172.21.15.78:0/4272835866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 0x55b816682460 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5418009dc0 tx=0x7f5418011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=911614ffbd23725e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:51.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.588+0000 7f54177fe640 1 -- 172.21.15.78:0/2804509157 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f5420107140 con 0x7f5428109170 2024-09-11T01:15:51.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.588+0000 7f5414ff9640 1 -- 172.21.15.78:0/4272835866 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418005220 con 0x55b816676a10 2024-09-11T01:15:51.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.588+0000 7f5414ff9640 1 -- 172.21.15.78:0/4272835866 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5418004c50 con 0x55b816676a10 2024-09-11T01:15:51.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.588+0000 7f5414ff9640 1 -- 172.21.15.78:0/4272835866 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54180167b0 con 0x55b816676a10 2024-09-11T01:15:51.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.589+0000 7f542f1ee740 1 -- 172.21.15.78:0/4272835866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 msgr2=0x55b816682460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.589+0000 7f5425d9b640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54100b5450 0x7f54100b7910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5428108320 tx=0x7f541c012040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.589+0000 7f542f1ee740 1 --2- 172.21.15.78:0/4272835866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 0x55b816682460 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5418009dc0 tx=0x7f5418011810 comp rx=0 tx=0).stop 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.589+0000 7f542f1ee740 1 -- 172.21.15.78:0/4272835866 shutdown_connections 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.589+0000 7f542f1ee740 1 --2- 172.21.15.78:0/4272835866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8166829a0 0x55b816684d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.589+0000 7f542f1ee740 1 --2- 172.21.15.78:0/4272835866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 0x55b816682460 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.589+0000 7f542f1ee740 1 --2- 172.21.15.78:0/4272835866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 0x55b8166764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.589+0000 7f542f1ee740 1 -- 172.21.15.78:0/4272835866 >> 172.21.15.78:0/4272835866 conn(0x55b816673f20 msgr2=0x55b8166750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.589+0000 7f542f1ee740 1 -- 172.21.15.78:0/4272835866 shutdown_connections 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.589+0000 7f542f1ee740 1 -- 172.21.15.78:0/4272835866 wait complete. 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.590+0000 7f542f1ee740 1 -- start start 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.590+0000 7f542f1ee740 1 Processor -- start 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.590+0000 7f542f1ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 0x55b816682190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.590+0000 7f542f1ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b816676a10 0x55b81667f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.590+0000 7f542f1ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8166829a0 0x55b81667fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.590+0000 7f542f1ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b81643d700 con 0x55b816676a10 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.590+0000 7f542f1ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b816577160 con 0x55b8166829a0 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.590+0000 7f542f1ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b816685620 con 0x55b8166760f0 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.590+0000 7f542f1ee740 10 client.0 _ll_register_callbacks cb 0x55b8165909e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.591+0000 7f542659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 0x55b816682190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.591+0000 7f542659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 0x55b816682190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:49174/0 (socket says 172.21.15.78:49174) 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.591+0000 7f542659c640 1 -- 172.21.15.78:0/2417505385 learned_addr learned my addr 172.21.15.78:0/2417505385 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.591+0000 7f5425d9b640 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b816676a10 0x55b81667f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.591+0000 7f5426d9d640 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8166829a0 0x55b81667fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.591+0000 7f542659c640 1 -- 172.21.15.78:0/2417505385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8166829a0 msgr2=0x55b81667fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.591+0000 7f542659c640 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8166829a0 0x55b81667fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.591+0000 7f542659c640 1 -- 172.21.15.78:0/2417505385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b816676a10 msgr2=0x55b81667f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.591+0000 7f542659c640 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b816676a10 0x55b81667f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.591+0000 7f542659c640 1 -- 172.21.15.78:0/2417505385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8165a4c90 con 0x55b8166760f0 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.591+0000 7f5426d9d640 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8166829a0 0x55b81667fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.592+0000 7f542659c640 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 0x55b816682190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5418005720 tx=0x7f54180055c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d645ebe637f09069 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.592+0000 7f540dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.592+0000 7f540dffb640 1 -- 172.21.15.78:0/2417505385 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418004240 con 0x55b8166760f0 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.592+0000 7f540dffb640 1 -- 172.21.15.78:0/2417505385 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f54180043e0 con 0x55b8166760f0 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.592+0000 7f542f1ee740 1 -- 172.21.15.78:0/2417505385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5418004c50 con 0x55b8166760f0 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.592+0000 7f540dffb640 1 -- 172.21.15.78:0/2417505385 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f5418004e00 con 0x55b8166760f0 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.592+0000 7f542f1ee740 10 client.27658 fetch_fsmap learned FSMap version 70 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.592+0000 7f542f1ee740 1 -- 172.21.15.78:0/2417505385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55b816699690 con 0x55b8166760f0 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.593+0000 7f540dffb640 1 -- 172.21.15.78:0/2417505385 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5418004c50 con 0x55b8166760f0 2024-09-11T01:15:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.593+0000 7f540dffb640 1 -- 172.21.15.78:0/2417505385 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f5418003bd0 con 0x55b8166760f0 2024-09-11T01:15:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.593+0000 7f542f1ee740 10 client.27658 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:15:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.593+0000 7f542f1ee740 10 client.27658 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:15:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.593+0000 7f542f1ee740 20 client.27658 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.593+0000 7f542f1ee740 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b81669a9e0 0x55b81669cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.593+0000 7f542f1ee740 4 client.27658 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:15:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.593+0000 7f542f1ee740 1 -- 172.21.15.78:0/2417505385 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8165eedf0 con 0x55b81669a9e0 2024-09-11T01:15:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.593+0000 7f5425d9b640 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b81669a9e0 0x55b81669cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.595+0000 7f5425d9b640 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b81669a9e0 0x55b81669cdd0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=482cd8f54ffd6673 server_cookie=2530e23343e22530 in_seq=0 out_seq=0 2024-09-11T01:15:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.595+0000 7f540dffb640 10 client.27658 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:15:51.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.596+0000 7f540dffb640 1 -- 172.21.15.78:0/2417505385 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b8165eedf0 con 0x55b81669a9e0 2024-09-11T01:15:51.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.596+0000 7f540dffb640 10 client.27658 handle_command_reply: tid=0 2024-09-11T01:15:51.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.596+0000 7f542f1ee740 1 client.27658 shutdown 2024-09-11T01:15:51.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.596+0000 7f542f1ee740 1 -- 172.21.15.78:0/2417505385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 msgr2=0x55b816682190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:51.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.596+0000 7f542f1ee740 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 0x55b816682190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5418005720 tx=0x7f54180055c0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.597+0000 7f542f1ee740 1 -- 172.21.15.78:0/2417505385 shutdown_connections 2024-09-11T01:15:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.597+0000 7f542f1ee740 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8166829a0 0x55b81667fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.597+0000 7f542f1ee740 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b81669a9e0 0x55b81669cdd0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.597+0000 7f540dffb640 0 client.27658 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:15:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.597+0000 7f542f1ee740 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b816676a10 0x55b81667f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.597+0000 7f542f1ee740 1 --2- 172.21.15.78:0/2417505385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166760f0 0x55b816682190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.597+0000 7f542f1ee740 1 -- 172.21.15.78:0/2417505385 >> 172.21.15.78:0/2417505385 conn(0x55b816673f20 msgr2=0x55b816679a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.597+0000 7f542f1ee740 1 -- 172.21.15.78:0/2417505385 shutdown_connections 2024-09-11T01:15:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.597+0000 7f542f1ee740 1 -- 172.21.15.78:0/2417505385 wait complete. 2024-09-11T01:15:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.597+0000 7f542f1ee740 20 client.27658 trim_cache size 0 max 16384 2024-09-11T01:15:51.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.617+0000 7f542f1ee740 1 Processor -- start 2024-09-11T01:15:51.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.617+0000 7f542f1ee740 1 -- start start 2024-09-11T01:15:51.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.617+0000 7f542f1ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8166760f0 0x55b816673760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.617+0000 7f542f1ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b816676a10 0x55b81667c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f542f1ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166829a0 0x55b81667cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f542f1ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54281ac3d0 con 0x55b816676a10 2024-09-11T01:15:51.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f542f1ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54281adf10 con 0x55b8166760f0 2024-09-11T01:15:51.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f542f1ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54281acaa0 con 0x55b8166829a0 2024-09-11T01:15:51.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f5425d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b816676a10 0x55b81667c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f5425d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b816676a10 0x55b81667c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47878/0 (socket says 172.21.15.78:47878) 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f5425d9b640 1 -- 172.21.15.78:0/63338543 learned_addr learned my addr 172.21.15.78:0/63338543 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f5426d9d640 1 --2- 172.21.15.78:0/63338543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166829a0 0x55b81667cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f542659c640 1 --2- 172.21.15.78:0/63338543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8166760f0 0x55b816673760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f5425d9b640 1 -- 172.21.15.78:0/63338543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166829a0 msgr2=0x55b81667cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f5425d9b640 1 --2- 172.21.15.78:0/63338543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166829a0 0x55b81667cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f5425d9b640 1 -- 172.21.15.78:0/63338543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8166760f0 msgr2=0x55b816673760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f5425d9b640 1 --2- 172.21.15.78:0/63338543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8166760f0 0x55b816673760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f5425d9b640 1 -- 172.21.15.78:0/63338543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8166749a0 con 0x55b816676a10 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.618+0000 7f5425d9b640 1 --2- 172.21.15.78:0/63338543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b816676a10 0x55b81667c7f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f541c00a5c0 tx=0x7f541c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70123bb3b9d3d796 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.619+0000 7f540ffff640 1 -- 172.21.15.78:0/63338543 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c005020 con 0x55b816676a10 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.619+0000 7f540ffff640 1 -- 172.21.15.78:0/63338543 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f541c019070 con 0x55b816676a10 2024-09-11T01:15:51.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.619+0000 7f540ffff640 1 -- 172.21.15.78:0/63338543 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c01f720 con 0x55b816676a10 2024-09-11T01:15:51.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.620+0000 7f542f1ee740 1 -- 172.21.15.78:0/63338543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b816676a10 msgr2=0x55b81667c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:51.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.620+0000 7f542f1ee740 1 --2- 172.21.15.78:0/63338543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b816676a10 0x55b81667c7f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f541c00a5c0 tx=0x7f541c014000 comp rx=0 tx=0).stop 2024-09-11T01:15:51.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.620+0000 7f542f1ee740 1 -- 172.21.15.78:0/63338543 shutdown_connections 2024-09-11T01:15:51.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.620+0000 7f542f1ee740 1 --2- 172.21.15.78:0/63338543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166829a0 0x55b81667cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.620+0000 7f542f1ee740 1 --2- 172.21.15.78:0/63338543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b816676a10 0x55b81667c7f0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.620+0000 7f542f1ee740 1 --2- 172.21.15.78:0/63338543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8166760f0 0x55b816673760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.620+0000 7f542f1ee740 1 -- 172.21.15.78:0/63338543 >> 172.21.15.78:0/63338543 conn(0x55b816673f20 msgr2=0x55b816681430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.620+0000 7f542f1ee740 1 -- 172.21.15.78:0/63338543 shutdown_connections 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.620+0000 7f542f1ee740 1 -- 172.21.15.78:0/63338543 wait complete. 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.620+0000 7f542f1ee740 1 -- start start 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f542f1ee740 1 Processor -- start 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f542f1ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8166760f0 0x55b81668faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f542f1ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 0x55b816690070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f542f1ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166829a0 0x55b816700670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f542f1ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54281ad840 con 0x55b8166760f0 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f542f1ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54281ad170 con 0x55b816676a10 2024-09-11T01:15:51.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f542f1ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54281aa3d0 con 0x55b8166829a0 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f542f1ee740 10 client.0 _ll_register_callbacks cb 0x7f5418009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f5426d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166829a0 0x55b816700670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f5426d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166829a0 0x55b816700670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:49200/0 (socket says 172.21.15.78:49200) 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f5426d9d640 1 -- 172.21.15.78:0/1363277918 learned_addr learned my addr 172.21.15.78:0/1363277918 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f542659c640 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8166760f0 0x55b81668faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f5425d9b640 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 0x55b816690070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f5426d9d640 1 -- 172.21.15.78:0/1363277918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 msgr2=0x55b816690070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f5426d9d640 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 0x55b816690070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f5426d9d640 1 -- 172.21.15.78:0/1363277918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8166760f0 msgr2=0x55b81668faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.621+0000 7f5426d9d640 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8166760f0 0x55b81668faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.622+0000 7f5426d9d640 1 -- 172.21.15.78:0/1363277918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b816700d80 con 0x55b8166829a0 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.622+0000 7f5425d9b640 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 0x55b816690070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.622+0000 7f5426d9d640 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166829a0 0x55b816700670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f54200ca810 tx=0x7f54200ccd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6b249460b0ac8d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:51.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.622+0000 7f540dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.622+0000 7f540dffb640 1 -- 172.21.15.78:0/1363277918 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54200cda00 con 0x55b8166829a0 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.622+0000 7f540dffb640 1 -- 172.21.15.78:0/1363277918 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f54200d66a0 con 0x55b8166829a0 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.622+0000 7f542f1ee740 1 -- 172.21.15.78:0/1363277918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f541c019070 con 0x55b8166829a0 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.622+0000 7f540dffb640 1 -- 172.21.15.78:0/1363277918 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f54200e83c0 con 0x55b8166829a0 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.622+0000 7f542f1ee740 10 client.27664 fetch_fsmap learned FSMap version 70 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.622+0000 7f542f1ee740 1 -- 172.21.15.78:0/1363277918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55b816708180 con 0x55b8166829a0 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.623+0000 7f540dffb640 1 -- 172.21.15.78:0/1363277918 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f541c019070 con 0x55b8166829a0 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.623+0000 7f540dffb640 1 -- 172.21.15.78:0/1363277918 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f54200cc800 con 0x55b8166829a0 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.623+0000 7f542f1ee740 10 client.27664 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.623+0000 7f542f1ee740 10 client.27664 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.623+0000 7f542f1ee740 20 client.27664 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.623+0000 7f542f1ee740 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b81669a9e0 0x55b81670a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.623+0000 7f542f1ee740 4 client.27664 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:15:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.623+0000 7f542f1ee740 1 -- 172.21.15.78:0/1363277918 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8165ea8a0 con 0x55b81669a9e0 2024-09-11T01:15:51.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.623+0000 7f542659c640 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b81669a9e0 0x55b81670a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:51.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.624+0000 7f542659c640 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b81669a9e0 0x55b81670a9c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e5d35f4198e2763 server_cookie=edb0b5d02221c412 in_seq=0 out_seq=0 2024-09-11T01:15:51.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.624+0000 7f540dffb640 10 client.27664 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:15:51.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.628+0000 7f540dffb640 1 -- 172.21.15.78:0/1363277918 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8165ea8a0 con 0x55b81669a9e0 2024-09-11T01:15:51.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.628+0000 7f540dffb640 10 client.27664 handle_command_reply: tid=0 2024-09-11T01:15:51.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.628+0000 7f542f1ee740 1 client.27664 shutdown 2024-09-11T01:15:51.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.628+0000 7f542f1ee740 1 -- 172.21.15.78:0/1363277918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166829a0 msgr2=0x55b816700670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:51.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.628+0000 7f542f1ee740 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166829a0 0x55b816700670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f54200ca810 tx=0x7f54200ccd10 comp rx=0 tx=0).stop 2024-09-11T01:15:51.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.628+0000 7f542f1ee740 1 -- 172.21.15.78:0/1363277918 shutdown_connections 2024-09-11T01:15:51.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.628+0000 7f542f1ee740 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8166829a0 0x55b816700670 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.628+0000 7f542f1ee740 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b816676a10 0x55b816690070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.628+0000 7f542f1ee740 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b81669a9e0 0x55b81670a9c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.628+0000 7f542f1ee740 1 --2- 172.21.15.78:0/1363277918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8166760f0 0x55b81668faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.628+0000 7f542f1ee740 1 -- 172.21.15.78:0/1363277918 >> 172.21.15.78:0/1363277918 conn(0x55b816673f20 msgr2=0x55b81667dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:51.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.628+0000 7f540dffb640 0 client.27664 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:15:51.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.629+0000 7f542f1ee740 1 -- 172.21.15.78:0/1363277918 shutdown_connections 2024-09-11T01:15:51.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.629+0000 7f542f1ee740 1 -- 172.21.15.78:0/1363277918 wait complete. 2024-09-11T01:15:51.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.629+0000 7f542f1ee740 20 client.27664 trim_cache size 0 max 16384 2024-09-11T01:15:51.633 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:15:51.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.629+0000 7f542ce19640 1 -- 172.21.15.78:0/2804509157 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54100b5450 msgr2=0x7f54100b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:51.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.630+0000 7f542ce19640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54100b5450 0x7f54100b7910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5428108320 tx=0x7f541c012040 comp rx=0 tx=0).stop 2024-09-11T01:15:51.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.630+0000 7f542ce19640 1 -- 172.21.15.78:0/2804509157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5428109170 msgr2=0x7f542819eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:51.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.630+0000 7f542ce19640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5428109170 0x7f542819eaa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f542002ade0 tx=0x7f54200316f0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.630+0000 7f542ce19640 1 -- 172.21.15.78:0/2804509157 shutdown_connections 2024-09-11T01:15:51.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.630+0000 7f542ce19640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54100b5450 0x7f54100b7910 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.630+0000 7f542ce19640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5428109170 0x7f542819eaa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.630+0000 7f542ce19640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54281087f0 0x7f5428199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.630+0000 7f542ce19640 1 --2- 172.21.15.78:0/2804509157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5428102de0 0x7f54281991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:51.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.630+0000 7f542ce19640 1 -- 172.21.15.78:0/2804509157 >> 172.21.15.78:0/2804509157 conn(0x7f54280fe830 msgr2=0x7f5428100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:51.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.630+0000 7f542ce19640 1 -- 172.21.15.78:0/2804509157 shutdown_connections 2024-09-11T01:15:51.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:51.630+0000 7f542ce19640 1 -- 172.21.15.78:0/2804509157 wait complete. 2024-09-11T01:15:51.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 160770 37.06 MB/sec execute 117 sec latency 91.703 ms 2024-09-11T01:15:52.409 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:52 smithi078 ceph-mon[31315]: pgmap v852: 97 pgs: 97 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 641 op/s 2024-09-11T01:15:52.439 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:15:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:52 smithi110 ceph-mon[34712]: pgmap v852: 97 pgs: 97 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 641 op/s 2024-09-11T01:15:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:52 smithi112 ceph-mon[35384]: pgmap v852: 97 pgs: 97 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 641 op/s 2024-09-11T01:15:52.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 161843 37.36 MB/sec execute 118 sec latency 82.824 ms 2024-09-11T01:15:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:15:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.400+0000 7fc899c7b640 1 -- 172.21.15.78:0/3577425404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc894101c90 msgr2=0x7fc894102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.400+0000 7fc899c7b640 1 --2- 172.21.15.78:0/3577425404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc894101c90 0x7fc894102070 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc884009900 tx=0x7fc88402a410 comp rx=0 tx=0).stop 2024-09-11T01:15:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.400+0000 7fc899c7b640 1 -- 172.21.15.78:0/3577425404 shutdown_connections 2024-09-11T01:15:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.400+0000 7fc899c7b640 1 --2- 172.21.15.78:0/3577425404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc894108020 0x7fc89410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.400+0000 7fc899c7b640 1 --2- 172.21.15.78:0/3577425404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8941076a0 0x7fc894107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.400+0000 7fc899c7b640 1 --2- 172.21.15.78:0/3577425404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc894101c90 0x7fc894102070 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.400+0000 7fc899c7b640 1 -- 172.21.15.78:0/3577425404 >> 172.21.15.78:0/3577425404 conn(0x7fc8940fd700 msgr2=0x7fc8940ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.400+0000 7fc899c7b640 1 -- 172.21.15.78:0/3577425404 shutdown_connections 2024-09-11T01:15:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.400+0000 7fc899c7b640 1 -- 172.21.15.78:0/3577425404 wait complete. 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.401+0000 7fc899c7b640 1 Processor -- start 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.401+0000 7fc899c7b640 1 -- start start 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.401+0000 7fc899c7b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc894101c90 0x7fc8941992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.401+0000 7fc899c7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8941076a0 0x7fc8941997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.401+0000 7fc899c7b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc894108020 0x7fc89419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc899c7b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc894111ce0 con 0x7fc8941076a0 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc899c7b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc894111b60 con 0x7fc894101c90 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc899c7b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc894111e60 con 0x7fc894108020 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8941076a0 0x7fc8941997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8941076a0 0x7fc8941997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35528/0 (socket says 172.21.15.78:35528) 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc892ffd640 1 -- 172.21.15.78:0/1468958442 learned_addr learned my addr 172.21.15.78:0/1468958442 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc8937fe640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc894101c90 0x7fc8941992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc892ffd640 1 -- 172.21.15.78:0/1468958442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc894108020 msgr2=0x7fc89419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:53.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc892ffd640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc894108020 0x7fc89419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc892ffd640 1 -- 172.21.15.78:0/1468958442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc894101c90 msgr2=0x7fc8941992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:53.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc892ffd640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc894101c90 0x7fc8941992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc892ffd640 1 -- 172.21.15.78:0/1468958442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89419f350 con 0x7fc8941076a0 2024-09-11T01:15:53.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc892ffd640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8941076a0 0x7fc8941997e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc88800e9f0 tx=0x7fc88800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:53.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc890ff9640 1 -- 172.21.15.78:0/1468958442 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88800cd80 con 0x7fc8941076a0 2024-09-11T01:15:53.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc899c7b640 1 -- 172.21.15.78:0/1468958442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89419f640 con 0x7fc8941076a0 2024-09-11T01:15:53.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc890ff9640 1 -- 172.21.15.78:0/1468958442 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8880108e0 con 0x7fc8941076a0 2024-09-11T01:15:53.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc890ff9640 1 -- 172.21.15.78:0/1468958442 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888018980 con 0x7fc8941076a0 2024-09-11T01:15:53.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.402+0000 7fc899c7b640 1 -- 172.21.15.78:0/1468958442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8941a5ed0 con 0x7fc8941076a0 2024-09-11T01:15:53.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.406+0000 7fc890ff9640 1 -- 172.21.15.78:0/1468958442 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc888010430 con 0x7fc8941076a0 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.406+0000 7fc89c050740 1 Processor -- start 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.406+0000 7fc89c050740 1 -- start start 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.406+0000 7fc89c050740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728a6b70 0x55c0728a6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.406+0000 7fc89c050740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.406+0000 7fc89c050740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728b6e10 0x55c0728c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.406+0000 7fc89c050740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c07281fea0 con 0x55c0728a6b70 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.406+0000 7fc89c050740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c07280f2d0 con 0x55c0728b6e10 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.406+0000 7fc89c050740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c0728130e0 con 0x55c0728b6490 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.407+0000 7fc893fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728a6b70 0x55c0728a6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.407+0000 7fc893fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728a6b70 0x55c0728a6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35534/0 (socket says 172.21.15.78:35534) 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.407+0000 7fc893fff640 1 -- 172.21.15.78:0/1484440748 learned_addr learned my addr 172.21.15.78:0/1484440748 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.407+0000 7fc892ffd640 1 --2- 172.21.15.78:0/1484440748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728b6e10 0x55c0728c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.407+0000 7fc890ff9640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc87c0b54d0 0x7fc87c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.407+0000 7fc8937fe640 1 --2- 172.21.15.78:0/1484440748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.408+0000 7fc890ff9640 1 -- 172.21.15.78:0/1468958442 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7fc8880e62b0 con 0x7fc8941076a0 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.408+0000 7fc8937fe640 1 -- 172.21.15.78:0/1484440748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728b6e10 msgr2=0x55c0728c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.408+0000 7fc8937fe640 1 --2- 172.21.15.78:0/1484440748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728b6e10 0x55c0728c0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.408+0000 7fc8937fe640 1 -- 172.21.15.78:0/1484440748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728a6b70 msgr2=0x55c0728a6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.408+0000 7fc8937fe640 1 --2- 172.21.15.78:0/1484440748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728a6b70 0x55c0728a6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.408+0000 7fc8937fe640 1 -- 172.21.15.78:0/1484440748 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c07256d650 con 0x55c0728b6490 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.408+0000 7fc893fff640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc87c0b54d0 0x7fc87c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.408+0000 7fc8937fe640 1 --2- 172.21.15.78:0/1484440748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728b68d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc884009230 tx=0x7fc884032bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=353a3932ae1a5ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.409+0000 7fc893fff640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc87c0b54d0 0x7fc87c0b7990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009270 tx=0x7fc88c004a40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.409+0000 7fc879ffb640 1 -- 172.21.15.78:0/1484440748 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884040070 con 0x55c0728b6490 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.409+0000 7fc879ffb640 1 -- 172.21.15.78:0/1484440748 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc884004030 con 0x55c0728b6490 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.409+0000 7fc879ffb640 1 -- 172.21.15.78:0/1484440748 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884035040 con 0x55c0728b6490 2024-09-11T01:15:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.410+0000 7fc89c050740 1 -- 172.21.15.78:0/1484440748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 msgr2=0x55c0728b68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.410+0000 7fc89c050740 1 --2- 172.21.15.78:0/1484440748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728b68d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc884009230 tx=0x7fc884032bd0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.410+0000 7fc89c050740 1 -- 172.21.15.78:0/1484440748 shutdown_connections 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.410+0000 7fc89c050740 1 --2- 172.21.15.78:0/1484440748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728b6e10 0x55c0728c0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.410+0000 7fc89c050740 1 --2- 172.21.15.78:0/1484440748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728b68d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.410+0000 7fc89c050740 1 --2- 172.21.15.78:0/1484440748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728a6b70 0x55c0728a6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.410+0000 7fc89c050740 1 -- 172.21.15.78:0/1484440748 >> 172.21.15.78:0/1484440748 conn(0x55c0728a9090 msgr2=0x55c0728a9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.410+0000 7fc89c050740 1 -- 172.21.15.78:0/1484440748 shutdown_connections 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.410+0000 7fc89c050740 1 -- 172.21.15.78:0/1484440748 wait complete. 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.410+0000 7fc89c050740 1 -- start start 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc89c050740 1 Processor -- start 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc89c050740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728a6b70 0x55c0728c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc89c050740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc89c050740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728b6e10 0x55c0728bdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc89c050740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c072670700 con 0x55c0728a6b70 2024-09-11T01:15:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc89c050740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c0727aa160 con 0x55c0728b6e10 2024-09-11T01:15:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc89c050740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c0728c0e90 con 0x55c0728b6490 2024-09-11T01:15:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc89c050740 10 client.0 _ll_register_callbacks cb 0x55c0727c39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc8937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728a6b70 0x55c0728c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc893fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728b6e10 0x55c0728bdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc8937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728a6b70 0x55c0728c05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35542/0 (socket says 172.21.15.78:35542) 2024-09-11T01:15:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc8937fe640 1 -- 172.21.15.78:0/297091904 learned_addr learned my addr 172.21.15.78:0/297091904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc893fff640 1 -- 172.21.15.78:0/297091904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 msgr2=0x55c0728bda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.411+0000 7fc893fff640 1 --2- 172.21.15.78:0/297091904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728bda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.412+0000 7fc893fff640 1 -- 172.21.15.78:0/297091904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728a6b70 msgr2=0x55c0728c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.412+0000 7fc893fff640 1 --2- 172.21.15.78:0/297091904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728a6b70 0x55c0728c05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.412+0000 7fc893fff640 1 -- 172.21.15.78:0/297091904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0727d7c90 con 0x55c0728b6e10 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.412+0000 7fc892ffd640 1 --2- 172.21.15.78:0/297091904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.412+0000 7fc893fff640 1 --2- 172.21.15.78:0/297091904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728b6e10 0x55c0728bdfb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc88c005c40 tx=0x7fc88c0095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9b8e9d72f205f06 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.412+0000 7fc872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.412+0000 7fc872ffd640 1 -- 172.21.15.78:0/297091904 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0128c0 con 0x55c0728b6e10 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.412+0000 7fc872ffd640 1 -- 172.21.15.78:0/297091904 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7fc88c012a60 con 0x55c0728b6e10 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.412+0000 7fc89c050740 1 -- 172.21.15.78:0/297091904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc884004030 con 0x55c0728b6e10 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.413+0000 7fc872ffd640 1 -- 172.21.15.78:0/297091904 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fc88c012c60 con 0x55c0728b6e10 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.413+0000 7fc89c050740 10 client.27701 fetch_fsmap learned FSMap version 70 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.413+0000 7fc89c050740 1 -- 172.21.15.78:0/297091904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55c0728cc780 con 0x55c0728b6e10 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.413+0000 7fc872ffd640 1 -- 172.21.15.78:0/297091904 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc88c012e00 con 0x55c0728b6e10 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.413+0000 7fc872ffd640 1 -- 172.21.15.78:0/297091904 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7fc88c024070 con 0x55c0728b6e10 2024-09-11T01:15:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.413+0000 7fc89c050740 10 client.27701 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.413+0000 7fc89c050740 10 client.27701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.414+0000 7fc89c050740 20 client.27701 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.414+0000 7fc89c050740 1 --2- 172.21.15.78:0/297091904 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c0728cdce0 0x55c0728d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.414+0000 7fc89c050740 4 client.27701 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.414+0000 7fc89c050740 1 -- 172.21.15.78:0/297091904 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc884004030 con 0x55c0728cdce0 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.414+0000 7fc8937fe640 1 --2- 172.21.15.78:0/297091904 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c0728cdce0 0x55c0728d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.414+0000 7fc872ffd640 10 client.27701 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.414+0000 7fc8937fe640 1 --2- 172.21.15.78:0/297091904 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c0728cdce0 0x55c0728d0110 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d128ed6a6761118 server_cookie=b8d0e822d32fcde2 in_seq=0 out_seq=0 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.415+0000 7fc872ffd640 1 -- 172.21.15.78:0/297091904 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc884004030 con 0x55c0728cdce0 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.415+0000 7fc872ffd640 10 client.27701 handle_command_reply: tid=0 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.416+0000 7fc89c050740 1 client.27701 shutdown 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.416+0000 7fc89c050740 1 -- 172.21.15.78:0/297091904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728b6e10 msgr2=0x55c0728bdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.416+0000 7fc89c050740 1 --2- 172.21.15.78:0/297091904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728b6e10 0x55c0728bdfb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc88c005c40 tx=0x7fc88c0095f0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.416+0000 7fc89c050740 1 -- 172.21.15.78:0/297091904 shutdown_connections 2024-09-11T01:15:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.416+0000 7fc89c050740 1 --2- 172.21.15.78:0/297091904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728b6e10 0x55c0728bdfb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.416+0000 7fc89c050740 1 --2- 172.21.15.78:0/297091904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.416+0000 7fc89c050740 1 --2- 172.21.15.78:0/297091904 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c0728cdce0 0x55c0728d0110 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.416+0000 7fc89c050740 1 --2- 172.21.15.78:0/297091904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728a6b70 0x55c0728c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.416+0000 7fc872ffd640 0 client.27701 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.416+0000 7fc89c050740 1 -- 172.21.15.78:0/297091904 >> 172.21.15.78:0/297091904 conn(0x55c0728a9090 msgr2=0x55c0728b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.417+0000 7fc89c050740 1 -- 172.21.15.78:0/297091904 shutdown_connections 2024-09-11T01:15:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.417+0000 7fc89c050740 1 -- 172.21.15.78:0/297091904 wait complete. 2024-09-11T01:15:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.417+0000 7fc89c050740 20 client.27701 trim_cache size 0 max 16384 2024-09-11T01:15:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc89c050740 1 Processor -- start 2024-09-11T01:15:53.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc89c050740 1 -- start start 2024-09-11T01:15:53.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc89c050740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728a6b70 0x55c0728bb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc89c050740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc89c050740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728b6e10 0x55c0728bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc89c050740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941ac460 con 0x55c0728b6e10 2024-09-11T01:15:53.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc89c050740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941adfa0 con 0x55c0728a6b70 2024-09-11T01:15:53.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc89c050740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941acb30 con 0x55c0728b6490 2024-09-11T01:15:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc8937fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728a6b70 0x55c0728bb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc893fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728b6e10 0x55c0728bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc8937fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728a6b70 0x55c0728bb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43156/0 (socket says 172.21.15.78:43156) 2024-09-11T01:15:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc8937fe640 1 -- 172.21.15.78:0/2191693235 learned_addr learned my addr 172.21.15.78:0/2191693235 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.435+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:44664/0 (socket says 172.21.15.78:44664) 2024-09-11T01:15:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.436+0000 7fc893fff640 1 -- 172.21.15.78:0/2191693235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 msgr2=0x55c0728a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.436+0000 7fc893fff640 1 --2- 172.21.15.78:0/2191693235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.436+0000 7fc893fff640 1 -- 172.21.15.78:0/2191693235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728a6b70 msgr2=0x55c0728bb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.436+0000 7fc893fff640 1 --2- 172.21.15.78:0/2191693235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728a6b70 0x55c0728bb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.436+0000 7fc893fff640 1 -- 172.21.15.78:0/2191693235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0728be880 con 0x55c0728b6e10 2024-09-11T01:15:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.436+0000 7fc892ffd640 1 --2- 172.21.15.78:0/2191693235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:15:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.436+0000 7fc893fff640 1 --2- 172.21.15.78:0/2191693235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728b6e10 0x55c0728bcc60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009fd0 tx=0x7fc88c012e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf4708caa4421de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.436+0000 7fc8797fa640 1 -- 172.21.15.78:0/2191693235 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c033070 con 0x55c0728b6e10 2024-09-11T01:15:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.436+0000 7fc8797fa640 1 -- 172.21.15.78:0/2191693235 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc88c0264c0 con 0x55c0728b6e10 2024-09-11T01:15:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.436+0000 7fc8797fa640 1 -- 172.21.15.78:0/2191693235 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c01b450 con 0x55c0728b6e10 2024-09-11T01:15:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.437+0000 7fc89c050740 1 -- 172.21.15.78:0/2191693235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728b6e10 msgr2=0x55c0728bcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.437+0000 7fc89c050740 1 --2- 172.21.15.78:0/2191693235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728b6e10 0x55c0728bcc60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009fd0 tx=0x7fc88c012e00 comp rx=0 tx=0).stop 2024-09-11T01:15:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.437+0000 7fc89c050740 1 -- 172.21.15.78:0/2191693235 shutdown_connections 2024-09-11T01:15:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.437+0000 7fc89c050740 1 --2- 172.21.15.78:0/2191693235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728b6e10 0x55c0728bcc60 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.437+0000 7fc89c050740 1 --2- 172.21.15.78:0/2191693235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6490 0x55c0728a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.437+0000 7fc89c050740 1 --2- 172.21.15.78:0/2191693235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728a6b70 0x55c0728bb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.438+0000 7fc89c050740 1 -- 172.21.15.78:0/2191693235 >> 172.21.15.78:0/2191693235 conn(0x55c0728a9090 msgr2=0x55c0728bd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.438+0000 7fc89c050740 1 -- 172.21.15.78:0/2191693235 shutdown_connections 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.438+0000 7fc89c050740 1 -- 172.21.15.78:0/2191693235 wait complete. 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.438+0000 7fc89c050740 1 -- start start 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.438+0000 7fc89c050740 1 Processor -- start 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.439+0000 7fc89c050740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728a6b70 0x55c0728c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.439+0000 7fc89c050740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728b6490 0x55c0728c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.439+0000 7fc89c050740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6e10 0x55c0728d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.439+0000 7fc89c050740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941ad8d0 con 0x55c0728b6490 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.439+0000 7fc89c050740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941ad200 con 0x55c0728a6b70 2024-09-11T01:15:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.439+0000 7fc89c050740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941aa460 con 0x55c0728b6e10 2024-09-11T01:15:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.439+0000 7fc89c050740 10 client.0 _ll_register_callbacks cb 0x7fc88c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:15:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.439+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728b6490 0x55c0728c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.439+0000 7fc893fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6e10 0x55c0728d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.439+0000 7fc892ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728b6490 0x55c0728c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35566/0 (socket says 172.21.15.78:35566) 2024-09-11T01:15:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.439+0000 7fc892ffd640 1 -- 172.21.15.78:0/3689449301 learned_addr learned my addr 172.21.15.78:0/3689449301 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:15:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.439+0000 7fc8937fe640 1 --2- 172.21.15.78:0/3689449301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728a6b70 0x55c0728c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.440+0000 7fc893fff640 1 -- 172.21.15.78:0/3689449301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728a6b70 msgr2=0x55c0728c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.440+0000 7fc893fff640 1 --2- 172.21.15.78:0/3689449301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728a6b70 0x55c0728c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.440+0000 7fc893fff640 1 -- 172.21.15.78:0/3689449301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728b6490 msgr2=0x55c0728c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:15:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.440+0000 7fc893fff640 1 --2- 172.21.15.78:0/3689449301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728b6490 0x55c0728c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.440+0000 7fc893fff640 1 -- 172.21.15.78:0/3689449301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0728d4f10 con 0x55c0728b6e10 2024-09-11T01:15:53.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.440+0000 7fc893fff640 1 --2- 172.21.15.78:0/3689449301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6e10 0x55c0728d49d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0266c0 tx=0x7fc88c0095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6868a2f939a63a51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:15:53.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.440+0000 7fc872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:15:53.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.440+0000 7fc872ffd640 1 -- 172.21.15.78:0/3689449301 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c033040 con 0x55c0728b6e10 2024-09-11T01:15:53.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.441+0000 7fc872ffd640 1 -- 172.21.15.78:0/3689449301 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7fc88c038460 con 0x55c0728b6e10 2024-09-11T01:15:53.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.441+0000 7fc89c050740 1 -- 172.21.15.78:0/3689449301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc88c0264c0 con 0x55c0728b6e10 2024-09-11T01:15:53.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.441+0000 7fc872ffd640 1 -- 172.21.15.78:0/3689449301 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fc88c038660 con 0x55c0728b6e10 2024-09-11T01:15:53.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.441+0000 7fc89c050740 10 client.27679 fetch_fsmap learned FSMap version 70 2024-09-11T01:15:53.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.441+0000 7fc89c050740 1 -- 172.21.15.78:0/3689449301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55c0728d6210 con 0x55c0728b6e10 2024-09-11T01:15:53.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.441+0000 7fc872ffd640 1 -- 172.21.15.78:0/3689449301 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc88c026b30 con 0x55c0728b6e10 2024-09-11T01:15:53.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.442+0000 7fc872ffd640 1 -- 172.21.15.78:0/3689449301 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7fc88c024070 con 0x55c0728b6e10 2024-09-11T01:15:53.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.442+0000 7fc89c050740 10 client.27679 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:15:53.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.442+0000 7fc89c050740 10 client.27679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:15:53.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.442+0000 7fc89c050740 20 client.27679 populate_metadata read hostname 'smithi078' 2024-09-11T01:15:53.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.442+0000 7fc89c050740 1 --2- 172.21.15.78:0/3689449301 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c07293bbf0 0x55c07293dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:15:53.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.442+0000 7fc89c050740 4 client.27679 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:15:53.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.442+0000 7fc89c050740 1 -- 172.21.15.78:0/3689449301 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc88c0264c0 con 0x55c07293bbf0 2024-09-11T01:15:53.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.443+0000 7fc8937fe640 1 --2- 172.21.15.78:0/3689449301 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c07293bbf0 0x55c07293dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:15:53.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.443+0000 7fc872ffd640 10 client.27679 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:53.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.443+0000 7fc8937fe640 1 --2- 172.21.15.78:0/3689449301 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c07293bbf0 0x55c07293dfe0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8741c336bfe9a789 server_cookie=e495df6444a67b83 in_seq=0 out_seq=0 2024-09-11T01:15:53.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.443+0000 7fc872ffd640 1 -- 172.21.15.78:0/3689449301 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc88c0264c0 con 0x55c07293bbf0 2024-09-11T01:15:53.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.443+0000 7fc872ffd640 10 client.27679 handle_command_reply: tid=0 2024-09-11T01:15:53.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc89c050740 1 client.27679 shutdown 2024-09-11T01:15:53.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc89c050740 1 -- 172.21.15.78:0/3689449301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6e10 msgr2=0x55c0728d49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:53.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc89c050740 1 --2- 172.21.15.78:0/3689449301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6e10 0x55c0728d49d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0266c0 tx=0x7fc88c0095f0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc89c050740 1 -- 172.21.15.78:0/3689449301 shutdown_connections 2024-09-11T01:15:53.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc89c050740 1 --2- 172.21.15.78:0/3689449301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0728b6e10 0x55c0728d49d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc89c050740 1 --2- 172.21.15.78:0/3689449301 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c07293bbf0 0x55c07293dfe0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc89c050740 1 --2- 172.21.15.78:0/3689449301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0728b6490 0x55c0728c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc89c050740 1 --2- 172.21.15.78:0/3689449301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0728a6b70 0x55c0728c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc89c050740 1 -- 172.21.15.78:0/3689449301 >> 172.21.15.78:0/3689449301 conn(0x55c0728a9090 msgr2=0x55c0728a5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:53.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc872ffd640 0 client.27679 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:15:53.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc89c050740 1 -- 172.21.15.78:0/3689449301 shutdown_connections 2024-09-11T01:15:53.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc89c050740 1 -- 172.21.15.78:0/3689449301 wait complete. 2024-09-11T01:15:53.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.444+0000 7fc89c050740 20 client.27679 trim_cache size 0 max 16384 2024-09-11T01:15:53.452 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:15:53.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.445+0000 7fc899c7b640 1 -- 172.21.15.78:0/1468958442 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc87c0b54d0 msgr2=0x7fc87c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:53.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.445+0000 7fc899c7b640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc87c0b54d0 0x7fc87c0b7990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009270 tx=0x7fc88c004a40 comp rx=0 tx=0).stop 2024-09-11T01:15:53.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.445+0000 7fc899c7b640 1 -- 172.21.15.78:0/1468958442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8941076a0 msgr2=0x7fc8941997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:15:53.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.445+0000 7fc899c7b640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8941076a0 0x7fc8941997e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc88800e9f0 tx=0x7fc88800eeb0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.445+0000 7fc899c7b640 1 -- 172.21.15.78:0/1468958442 shutdown_connections 2024-09-11T01:15:53.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.445+0000 7fc899c7b640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc87c0b54d0 0x7fc87c0b7990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.445+0000 7fc899c7b640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc894108020 0x7fc89419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.445+0000 7fc899c7b640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8941076a0 0x7fc8941997e0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.445+0000 7fc899c7b640 1 --2- 172.21.15.78:0/1468958442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc894101c90 0x7fc8941992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:15:53.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.445+0000 7fc899c7b640 1 -- 172.21.15.78:0/1468958442 >> 172.21.15.78:0/1468958442 conn(0x7fc8940fd700 msgr2=0x7fc89406ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:15:53.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.446+0000 7fc899c7b640 1 -- 172.21.15.78:0/1468958442 shutdown_connections 2024-09-11T01:15:53.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:15:53.446+0000 7fc899c7b640 1 -- 172.21.15.78:0/1468958442 wait complete. 2024-09-11T01:15:53.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 162479 37.33 MB/sec execute 119 sec latency 62.077 ms 2024-09-11T01:15:54.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:54 smithi078 ceph-mon[31315]: pgmap v853: 97 pgs: 97 active+clean; 2.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.9 MiB/s wr, 629 op/s 2024-09-11T01:15:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:54 smithi110 ceph-mon[34712]: pgmap v853: 97 pgs: 97 active+clean; 2.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.9 MiB/s wr, 629 op/s 2024-09-11T01:15:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:54 smithi112 ceph-mon[35384]: pgmap v853: 97 pgs: 97 active+clean; 2.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.9 MiB/s wr, 629 op/s 2024-09-11T01:15:54.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 162694 37.07 MB/sec execute 120 sec latency 427.472 ms 2024-09-11T01:15:55.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 162749 36.77 MB/sec execute 121 sec latency 1103.543 ms 2024-09-11T01:15:56.150 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:15:56.150 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:15:56.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:56 smithi110 ceph-mon[34712]: pgmap v854: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.7 MiB/s wr, 626 op/s 2024-09-11T01:15:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:56 smithi112 ceph-mon[35384]: pgmap v854: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.7 MiB/s wr, 626 op/s 2024-09-11T01:15:56.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:56 smithi078 ceph-mon[31315]: pgmap v854: 97 pgs: 97 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.7 MiB/s wr, 626 op/s 2024-09-11T01:15:56.787 INFO:tasks.workunit.client.0.smithi078.stdout: 10 163207 36.67 MB/sec execute 122 sec latency 966.543 ms 2024-09-11T01:15:56.857 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:15:57.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 163654 36.56 MB/sec execute 123 sec latency 340.634 ms 2024-09-11T01:15:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:15:58 smithi110 ceph-mon[34712]: pgmap v855: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 690 op/s 2024-09-11T01:15:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:15:58 smithi112 ceph-mon[35384]: pgmap v855: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 690 op/s 2024-09-11T01:15:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:15:58 smithi078 ceph-mon[31315]: pgmap v855: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 690 op/s 2024-09-11T01:15:58.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 164247 36.47 MB/sec execute 124 sec latency 186.404 ms 2024-09-11T01:15:59.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 164728 36.42 MB/sec execute 125 sec latency 354.290 ms 2024-09-11T01:16:00.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:00 smithi078 ceph-mon[31315]: pgmap v856: 97 pgs: 97 active+clean; 2.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 670 op/s 2024-09-11T01:16:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:00 smithi110 ceph-mon[34712]: pgmap v856: 97 pgs: 97 active+clean; 2.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 670 op/s 2024-09-11T01:16:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:00 smithi112 ceph-mon[35384]: pgmap v856: 97 pgs: 97 active+clean; 2.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 670 op/s 2024-09-11T01:16:00.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 165258 36.41 MB/sec execute 126 sec latency 482.725 ms 2024-09-11T01:16:01.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 165258 36.12 MB/sec execute 127 sec latency 1482.765 ms 2024-09-11T01:16:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:02 smithi110 ceph-mon[34712]: pgmap v857: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 21 MiB/s wr, 854 op/s 2024-09-11T01:16:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:02 smithi112 ceph-mon[35384]: pgmap v857: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 21 MiB/s wr, 854 op/s 2024-09-11T01:16:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:02 smithi078 ceph-mon[31315]: pgmap v857: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 21 MiB/s wr, 854 op/s 2024-09-11T01:16:02.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 165282 35.85 MB/sec execute 128 sec latency 2126.609 ms 2024-09-11T01:16:02.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.830+0000 7f698b054640 1 -- 172.21.15.78:0/838984439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f698410d100 msgr2=0x7f698410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:02.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.830+0000 7f698b054640 1 --2- 172.21.15.78:0/838984439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f698410d100 0x7f698410f560 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f697c009ca0 tx=0x7f697c02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.830+0000 7f698b054640 1 -- 172.21.15.78:0/838984439 shutdown_connections 2024-09-11T01:16:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.830+0000 7f698b054640 1 --2- 172.21.15.78:0/838984439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f698410d100 0x7f698410f560 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.830+0000 7f698b054640 1 --2- 172.21.15.78:0/838984439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69841045c0 0x7f698410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.830+0000 7f698b054640 1 --2- 172.21.15.78:0/838984439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6984103ca0 0x7f6984104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.830+0000 7f698b054640 1 -- 172.21.15.78:0/838984439 >> 172.21.15.78:0/838984439 conn(0x7f69840fd700 msgr2=0x7f69840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.830+0000 7f698b054640 1 -- 172.21.15.78:0/838984439 shutdown_connections 2024-09-11T01:16:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.830+0000 7f698b054640 1 -- 172.21.15.78:0/838984439 wait complete. 2024-09-11T01:16:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f698b054640 1 Processor -- start 2024-09-11T01:16:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f698b054640 1 -- start start 2024-09-11T01:16:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f698b054640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6984103ca0 0x7f6984199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f698b054640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69841045c0 0x7f69841997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f698b054640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f698410d100 0x7f698419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f698b054640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6984112e20 con 0x7f69841045c0 2024-09-11T01:16:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f698b054640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6984112ca0 con 0x7f698410d100 2024-09-11T01:16:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f698b054640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6984112fa0 con 0x7f6984103ca0 2024-09-11T01:16:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f6983fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69841045c0 0x7f69841997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f6983fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69841045c0 0x7f69841997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35584/0 (socket says 172.21.15.78:35584) 2024-09-11T01:16:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f6983fff640 1 -- 172.21.15.78:0/1291121255 learned_addr learned my addr 172.21.15.78:0/1291121255 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f6988df0640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6984103ca0 0x7f6984199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f69895f1640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f698410d100 0x7f698419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f6983fff640 1 -- 172.21.15.78:0/1291121255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6984103ca0 msgr2=0x7f6984199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f6983fff640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6984103ca0 0x7f6984199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f6983fff640 1 -- 172.21.15.78:0/1291121255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f698410d100 msgr2=0x7f698419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f6983fff640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f698410d100 0x7f698419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f6983fff640 1 -- 172.21.15.78:0/1291121255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698419f340 con 0x7f69841045c0 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.831+0000 7f6988df0640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6984103ca0 0x7f6984199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.832+0000 7f6983fff640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69841045c0 0x7f69841997d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f697000b3c0 tx=0x7f697000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.832+0000 7f6981ffb640 1 -- 172.21.15.78:0/1291121255 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970011830 con 0x7f69841045c0 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.832+0000 7f6981ffb640 1 -- 172.21.15.78:0/1291121255 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6970016070 con 0x7f69841045c0 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.832+0000 7f698b054640 1 -- 172.21.15.78:0/1291121255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698419f630 con 0x7f69841045c0 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.832+0000 7f6981ffb640 1 -- 172.21.15.78:0/1291121255 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970010de0 con 0x7f69841045c0 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.832+0000 7f698b054640 1 -- 172.21.15.78:0/1291121255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69841a5d70 con 0x7f69841045c0 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.834+0000 7f6981ffb640 1 -- 172.21.15.78:0/1291121255 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f69700119d0 con 0x7f69841045c0 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f698d429740 1 Processor -- start 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f698d429740 1 -- start start 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f698d429740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45a2f0 0x5556df45a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f698d429740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df45ac10 0x5556df466660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f698d429740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df466ba0 0x5556df468f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f698d429740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5556df3cce60 con 0x5556df466ba0 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f698d429740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5556df3bc290 con 0x5556df45ac10 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f698d429740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5556df3c00a0 con 0x5556df45a2f0 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f6983fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df466ba0 0x5556df468f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f6983fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df466ba0 0x5556df468f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35586/0 (socket says 172.21.15.78:35586) 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f6983fff640 1 -- 172.21.15.78:0/1186572489 learned_addr learned my addr 172.21.15.78:0/1186572489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f6988df0640 1 --2- 172.21.15.78:0/1186572489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df45ac10 0x5556df466660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f69895f1640 1 --2- 172.21.15.78:0/1186572489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45a2f0 0x5556df45a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.835+0000 7f6981ffb640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69740b54d0 0x7f69740b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f6983fff640 1 -- 172.21.15.78:0/1186572489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45a2f0 msgr2=0x5556df45a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f6983fff640 1 --2- 172.21.15.78:0/1186572489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45a2f0 0x5556df45a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f6983fff640 1 -- 172.21.15.78:0/1186572489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df45ac10 msgr2=0x5556df466660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f6983fff640 1 --2- 172.21.15.78:0/1186572489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df45ac10 0x5556df466660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f6983fff640 1 -- 172.21.15.78:0/1186572489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556df11a930 con 0x5556df466ba0 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f69895f1640 1 --2- 172.21.15.78:0/1186572489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45a2f0 0x5556df45a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f6988df0640 1 --2- 172.21.15.78:0/1186572489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df45ac10 0x5556df466660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f69895f1640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69740b54d0 0x7f69740b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f6981ffb640 1 -- 172.21.15.78:0/1291121255 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f69700e5620 con 0x7f69841045c0 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f6983fff640 1 --2- 172.21.15.78:0/1186572489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df466ba0 0x5556df468f90 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f69700e2cc0 tx=0x7f6970064940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3a8006d86d581a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f696effd640 1 -- 172.21.15.78:0/1186572489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69700ab4f0 con 0x5556df466ba0 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f696effd640 1 -- 172.21.15.78:0/1186572489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69700ac0e0 con 0x5556df466ba0 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f69895f1640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69740b54d0 0x7f69740b7990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f697c009430 tx=0x7f697c0095c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.836+0000 7f696effd640 1 -- 172.21.15.78:0/1186572489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69700ab9e0 con 0x5556df466ba0 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.837+0000 7f698d429740 1 -- 172.21.15.78:0/1186572489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df466ba0 msgr2=0x5556df468f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.837+0000 7f698d429740 1 --2- 172.21.15.78:0/1186572489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df466ba0 0x5556df468f90 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f69700e2cc0 tx=0x7f6970064940 comp rx=0 tx=0).stop 2024-09-11T01:16:02.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.837+0000 7f698d429740 1 -- 172.21.15.78:0/1186572489 shutdown_connections 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.837+0000 7f698d429740 1 --2- 172.21.15.78:0/1186572489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df466ba0 0x5556df468f90 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.837+0000 7f698d429740 1 --2- 172.21.15.78:0/1186572489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df45ac10 0x5556df466660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.837+0000 7f698d429740 1 --2- 172.21.15.78:0/1186572489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45a2f0 0x5556df45a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.837+0000 7f698d429740 1 -- 172.21.15.78:0/1186572489 >> 172.21.15.78:0/1186572489 conn(0x5556df458120 msgr2=0x5556df4592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.837+0000 7f698d429740 1 -- 172.21.15.78:0/1186572489 shutdown_connections 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.837+0000 7f698d429740 1 -- 172.21.15.78:0/1186572489 wait complete. 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.837+0000 7f698d429740 1 -- start start 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.837+0000 7f698d429740 1 Processor -- start 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f698d429740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 0x5556df463b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f698d429740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45ac10 0x5556df4640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f698d429740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 0x5556df464600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f698d429740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5556df21d890 con 0x5556df45a2f0 2024-09-11T01:16:02.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f698d429740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5556df356d40 con 0x5556df466ba0 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f698d429740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5556df469760 con 0x5556df45ac10 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f698d429740 10 client.0 _ll_register_callbacks cb 0x5556df3bac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6988df0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 0x5556df463b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6988df0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 0x5556df463b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35598/0 (socket says 172.21.15.78:35598) 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6988df0640 1 -- 172.21.15.78:0/1126414635 learned_addr learned my addr 172.21.15.78:0/1126414635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f69895f1640 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 0x5556df464600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6983fff640 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45ac10 0x5556df4640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6988df0640 1 -- 172.21.15.78:0/1126414635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45ac10 msgr2=0x5556df4640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6988df0640 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45ac10 0x5556df4640c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6988df0640 1 -- 172.21.15.78:0/1126414635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 msgr2=0x5556df464600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6988df0640 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 0x5556df464600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6988df0640 1 -- 172.21.15.78:0/1126414635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556df472b60 con 0x5556df45a2f0 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6983fff640 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45ac10 0x5556df4640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:16:02.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6988df0640 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 0x5556df463b80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6978012450 tx=0x7f6978012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f2de50f5fa6ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6963fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6963fff640 1 -- 172.21.15.78:0/1126414635 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697801f060 con 0x5556df45a2f0 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f698d429740 1 -- 172.21.15.78:0/1126414635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69700ac0e0 con 0x5556df45a2f0 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.838+0000 7f6963fff640 1 -- 172.21.15.78:0/1126414635 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f6978010a20 con 0x5556df45a2f0 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.839+0000 7f6963fff640 1 -- 172.21.15.78:0/1126414635 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f6978010c20 con 0x5556df45a2f0 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.839+0000 7f698d429740 10 client.18594 fetch_fsmap learned FSMap version 70 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.839+0000 7f698d429740 1 -- 172.21.15.78:0/1126414635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5556df475570 con 0x5556df45a2f0 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.839+0000 7f6963fff640 1 -- 172.21.15.78:0/1126414635 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69700ac0e0 con 0x5556df45a2f0 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.839+0000 7f6963fff640 1 -- 172.21.15.78:0/1126414635 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f6978019030 con 0x5556df45a2f0 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.839+0000 7f698d429740 10 client.18594 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.839+0000 7f698d429740 10 client.18594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.839+0000 7f698d429740 20 client.18594 populate_metadata read hostname 'smithi078' 2024-09-11T01:16:02.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.839+0000 7f698d429740 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5556df476a30 0x5556df478e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.839+0000 7f698d429740 4 client.18594 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.839+0000 7f698d429740 1 -- 172.21.15.78:0/1126414635 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556df3cedb0 con 0x5556df476a30 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.839+0000 7f6983fff640 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5556df476a30 0x5556df478e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.840+0000 7f6963fff640 10 client.18594 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.840+0000 7f6983fff640 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5556df476a30 0x5556df478e40 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed633c566695876 server_cookie=a6482ceaffdbf4af in_seq=0 out_seq=0 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.840+0000 7f6963fff640 1 -- 172.21.15.78:0/1126414635 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5556df3cedb0 con 0x5556df476a30 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.840+0000 7f6963fff640 10 client.18594 handle_command_reply: tid=0 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.840+0000 7f698d429740 1 client.18594 shutdown 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.841+0000 7f698d429740 1 -- 172.21.15.78:0/1126414635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 msgr2=0x5556df463b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.841+0000 7f698d429740 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 0x5556df463b80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6978012450 tx=0x7f6978012910 comp rx=0 tx=0).stop 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.841+0000 7f698d429740 1 -- 172.21.15.78:0/1126414635 shutdown_connections 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.841+0000 7f698d429740 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 0x5556df464600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.841+0000 7f698d429740 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45ac10 0x5556df4640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.841+0000 7f698d429740 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5556df476a30 0x5556df478e40 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.841+0000 7f698d429740 1 --2- 172.21.15.78:0/1126414635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 0x5556df463b80 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.841+0000 7f6963fff640 0 client.18594 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.841+0000 7f698d429740 1 -- 172.21.15.78:0/1126414635 >> 172.21.15.78:0/1126414635 conn(0x5556df458120 msgr2=0x5556df45d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.841+0000 7f698d429740 1 -- 172.21.15.78:0/1126414635 shutdown_connections 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.841+0000 7f698d429740 1 -- 172.21.15.78:0/1126414635 wait complete. 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.841+0000 7f698d429740 20 client.18594 trim_cache size 0 max 16384 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f698d429740 1 Processor -- start 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f698d429740 1 -- start start 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f698d429740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45a2f0 0x5556df452e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f698d429740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45ac10 0x5556df464080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f698d429740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 0x5556df4645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f698d429740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69841ac4a0 con 0x5556df45ac10 2024-09-11T01:16:02.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f698d429740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69841adfe0 con 0x5556df466ba0 2024-09-11T01:16:02.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f698d429740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69841acb70 con 0x5556df45a2f0 2024-09-11T01:16:02.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f6983fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45ac10 0x5556df464080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f6983fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45ac10 0x5556df464080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35600/0 (socket says 172.21.15.78:35600) 2024-09-11T01:16:02.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f6983fff640 1 -- 172.21.15.78:0/915577818 learned_addr learned my addr 172.21.15.78:0/915577818 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:02.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f6988df0640 1 --2- 172.21.15.78:0/915577818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45a2f0 0x5556df452e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f69895f1640 1 --2- 172.21.15.78:0/915577818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 0x5556df4645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f6983fff640 1 -- 172.21.15.78:0/915577818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45a2f0 msgr2=0x5556df452e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:02.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f6983fff640 1 --2- 172.21.15.78:0/915577818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45a2f0 0x5556df452e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f6983fff640 1 -- 172.21.15.78:0/915577818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 msgr2=0x5556df4645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f6983fff640 1 --2- 172.21.15.78:0/915577818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 0x5556df4645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.859+0000 7f6983fff640 1 -- 172.21.15.78:0/915577818 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556df462080 con 0x5556df45ac10 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.860+0000 7f6983fff640 1 --2- 172.21.15.78:0/915577818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45ac10 0x5556df464080 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f69700ab970 tx=0x7f69700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e152b562f4bd8f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.860+0000 7f696e7fc640 1 -- 172.21.15.78:0/915577818 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69700ab4f0 con 0x5556df45ac10 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.860+0000 7f696e7fc640 1 -- 172.21.15.78:0/915577818 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69700ac0e0 con 0x5556df45ac10 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.860+0000 7f696e7fc640 1 -- 172.21.15.78:0/915577818 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69700a7c50 con 0x5556df45ac10 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.860+0000 7f698d429740 1 -- 172.21.15.78:0/915577818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45ac10 msgr2=0x5556df464080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 --2- 172.21.15.78:0/915577818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45ac10 0x5556df464080 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f69700ab970 tx=0x7f69700046c0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 -- 172.21.15.78:0/915577818 shutdown_connections 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 --2- 172.21.15.78:0/915577818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 0x5556df4645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 --2- 172.21.15.78:0/915577818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45ac10 0x5556df464080 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 --2- 172.21.15.78:0/915577818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45a2f0 0x5556df452e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 -- 172.21.15.78:0/915577818 >> 172.21.15.78:0/915577818 conn(0x5556df458120 msgr2=0x5556df452750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 -- 172.21.15.78:0/915577818 shutdown_connections 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 -- 172.21.15.78:0/915577818 wait complete. 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 -- start start 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 Processor -- start 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 0x5556df46bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45ac10 0x5556df46c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.861+0000 7f698d429740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 0x5556df46c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f698d429740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69841ad910 con 0x5556df45a2f0 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f698d429740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69841ad240 con 0x5556df466ba0 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f698d429740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69841aa4c0 con 0x5556df45ac10 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f698d429740 10 client.0 _ll_register_callbacks cb 0x7f697800b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:16:02.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6988df0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 0x5556df46bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6988df0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 0x5556df46bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35610/0 (socket says 172.21.15.78:35610) 2024-09-11T01:16:02.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6988df0640 1 -- 172.21.15.78:0/4018548043 learned_addr learned my addr 172.21.15.78:0/4018548043 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:02.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f69895f1640 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 0x5556df46c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6983fff640 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45ac10 0x5556df46c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6988df0640 1 -- 172.21.15.78:0/4018548043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45ac10 msgr2=0x5556df46c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:02.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6988df0640 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45ac10 0x5556df46c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6988df0640 1 -- 172.21.15.78:0/4018548043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 msgr2=0x5556df46c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:02.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6988df0640 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 0x5556df46c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6988df0640 1 -- 172.21.15.78:0/4018548043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556df4e3b90 con 0x5556df45a2f0 2024-09-11T01:16:02.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6988df0640 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 0x5556df46bb30 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f6978005870 tx=0x7f6978010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952e0a57ae9c6a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:02.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6963fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:16:02.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6963fff640 1 -- 172.21.15.78:0/4018548043 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978022930 con 0x5556df45a2f0 2024-09-11T01:16:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6963fff640 1 -- 172.21.15.78:0/4018548043 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8449+0+0 (secure 0 0 0) 0x7f6978022ad0 con 0x5556df45a2f0 2024-09-11T01:16:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f698d429740 1 -- 172.21.15.78:0/4018548043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69700ac0e0 con 0x5556df45a2f0 2024-09-11T01:16:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f6963fff640 1 -- 172.21.15.78:0/4018548043 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f6978032050 con 0x5556df45a2f0 2024-09-11T01:16:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.862+0000 7f698d429740 10 client.18600 fetch_fsmap learned FSMap version 70 2024-09-11T01:16:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.863+0000 7f698d429740 1 -- 172.21.15.78:0/4018548043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5556df4e4ed0 con 0x5556df45a2f0 2024-09-11T01:16:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.863+0000 7f6963fff640 1 -- 172.21.15.78:0/4018548043 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69700ac0e0 con 0x5556df45a2f0 2024-09-11T01:16:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.863+0000 7f6963fff640 1 -- 172.21.15.78:0/4018548043 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f6978019070 con 0x5556df45a2f0 2024-09-11T01:16:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.863+0000 7f698d429740 10 client.18600 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:16:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.863+0000 7f698d429740 10 client.18600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:16:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.863+0000 7f698d429740 20 client.18600 populate_metadata read hostname 'smithi078' 2024-09-11T01:16:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.863+0000 7f698d429740 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5556df476a30 0x5556df4e7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.863+0000 7f698d429740 4 client.18600 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:16:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.863+0000 7f698d429740 1 -- 172.21.15.78:0/4018548043 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5556df3ca860 con 0x5556df476a30 2024-09-11T01:16:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.863+0000 7f6983fff640 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5556df476a30 0x5556df4e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.863+0000 7f6963fff640 10 client.18600 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:16:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.863+0000 7f6983fff640 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5556df476a30 0x5556df4e7660 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88de71ebe49b0a28 server_cookie=b71b983d8ce4434a in_seq=0 out_seq=0 2024-09-11T01:16:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.866+0000 7f6963fff640 1 -- 172.21.15.78:0/4018548043 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5556df3ca860 con 0x5556df476a30 2024-09-11T01:16:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.866+0000 7f6963fff640 10 client.18600 handle_command_reply: tid=0 2024-09-11T01:16:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.866+0000 7f698d429740 1 client.18600 shutdown 2024-09-11T01:16:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.866+0000 7f698d429740 1 -- 172.21.15.78:0/4018548043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 msgr2=0x5556df46bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.866+0000 7f698d429740 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 0x5556df46bb30 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f6978005870 tx=0x7f6978010fd0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.866+0000 7f698d429740 1 -- 172.21.15.78:0/4018548043 shutdown_connections 2024-09-11T01:16:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.866+0000 7f698d429740 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5556df466ba0 0x5556df46c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.866+0000 7f698d429740 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556df45ac10 0x5556df46c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.866+0000 7f698d429740 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5556df476a30 0x5556df4e7660 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.866+0000 7f698d429740 1 --2- 172.21.15.78:0/4018548043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556df45a2f0 0x5556df46bb30 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.866+0000 7f6963fff640 0 client.18600 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:16:02.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.866+0000 7f698d429740 1 -- 172.21.15.78:0/4018548043 >> 172.21.15.78:0/4018548043 conn(0x5556df458120 msgr2=0x5556df4651b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:02.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698d429740 1 -- 172.21.15.78:0/4018548043 shutdown_connections 2024-09-11T01:16:02.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698d429740 1 -- 172.21.15.78:0/4018548043 wait complete. 2024-09-11T01:16:02.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698d429740 20 client.18600 trim_cache size 0 max 16384 2024-09-11T01:16:02.905 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:16:02.905 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:16:02.906 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "33c6d527-1d18-4220-8f09-6a413e212ad7", 2024-09-11T01:16:02.906 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:16:02.906 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:16:02.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698b054640 1 -- 172.21.15.78:0/1291121255 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69740b54d0 msgr2=0x7f69740b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:02.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698b054640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69740b54d0 0x7f69740b7990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f697c009430 tx=0x7f697c0095c0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698b054640 1 -- 172.21.15.78:0/1291121255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69841045c0 msgr2=0x7f69841997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:02.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698b054640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69841045c0 0x7f69841997d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f697000b3c0 tx=0x7f697000b880 comp rx=0 tx=0).stop 2024-09-11T01:16:02.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698b054640 1 -- 172.21.15.78:0/1291121255 shutdown_connections 2024-09-11T01:16:02.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698b054640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69740b54d0 0x7f69740b7990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698b054640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f698410d100 0x7f698419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698b054640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69841045c0 0x7f69841997d0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698b054640 1 --2- 172.21.15.78:0/1291121255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6984103ca0 0x7f6984199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:02.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.867+0000 7f698b054640 1 -- 172.21.15.78:0/1291121255 >> 172.21.15.78:0/1291121255 conn(0x7f69840fd700 msgr2=0x7f698410f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:02.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.868+0000 7f698b054640 1 -- 172.21.15.78:0/1291121255 shutdown_connections 2024-09-11T01:16:02.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:01.868+0000 7f698b054640 1 -- 172.21.15.78:0/1291121255 wait complete. 2024-09-11T01:16:03.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:03 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:16:03.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:03 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:03.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:03 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:16:03.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:03 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:16:03.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:03 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:16:03.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:03 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:16:03.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:03 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:16:03.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:03 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:03.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:03 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:03 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:16:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:03 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:03 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:16:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:03 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:16:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:03 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:16:03.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:03 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:16:03.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:03 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:16:03.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:03 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:03.442 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:03 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:03 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:16:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:03 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:03.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:03 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:16:03.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:03 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:16:03.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:03 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:16:03.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:03 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:16:03.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:03 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:16:03.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:03 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:03.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:03 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:03.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 165282 35.58 MB/sec execute 129 sec latency 1395.264 ms 2024-09-11T01:16:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:04 smithi110 ceph-mon[34712]: pgmap v858: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 889 op/s 2024-09-11T01:16:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:04.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:04 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:04 smithi112 ceph-mon[35384]: pgmap v858: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 889 op/s 2024-09-11T01:16:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:04 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:04 smithi078 ceph-mon[31315]: pgmap v858: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 889 op/s 2024-09-11T01:16:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:04 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:16:04.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 165446 35.40 MB/sec execute 130 sec latency 2118.566 ms 2024-09-11T01:16:05.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 166051 35.35 MB/sec execute 131 sec latency 119.073 ms 2024-09-11T01:16:06.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:06 smithi110 ceph-mon[34712]: pgmap v859: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 878 op/s 2024-09-11T01:16:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:06 smithi112 ceph-mon[35384]: pgmap v859: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 878 op/s 2024-09-11T01:16:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:06 smithi078 ceph-mon[31315]: pgmap v859: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 878 op/s 2024-09-11T01:16:06.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 166458 35.20 MB/sec execute 132 sec latency 485.717 ms 2024-09-11T01:16:07.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 167197 35.28 MB/sec execute 133 sec latency 53.566 ms 2024-09-11T01:16:08.118 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:16:08.304 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:08 smithi078 ceph-mon[31315]: pgmap v860: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 22 MiB/s wr, 849 op/s 2024-09-11T01:16:08.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:08 smithi110 ceph-mon[34712]: pgmap v860: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 22 MiB/s wr, 849 op/s 2024-09-11T01:16:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:08 smithi112 ceph-mon[35384]: pgmap v860: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 22 MiB/s wr, 849 op/s 2024-09-11T01:16:08.732 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:16:08.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 167793 35.23 MB/sec execute 134 sec latency 253.125 ms 2024-09-11T01:16:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:09 smithi078 ceph-mon[31315]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T01:16:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:09 smithi110 ceph-mon[34712]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T01:16:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:09 smithi112 ceph-mon[35384]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T01:16:09.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 168248 35.13 MB/sec execute 135 sec latency 540.036 ms 2024-09-11T01:16:10.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:10 smithi110 ceph-mon[34712]: pgmap v862: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 673 op/s 2024-09-11T01:16:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:10 smithi112 ceph-mon[35384]: pgmap v862: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 673 op/s 2024-09-11T01:16:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:10 smithi078 ceph-mon[31315]: pgmap v862: 97 pgs: 97 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 673 op/s 2024-09-11T01:16:10.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 168874 35.18 MB/sec execute 136 sec latency 287.549 ms 2024-09-11T01:16:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.578+0000 7efc09ec7640 1 -- 172.21.15.78:0/744316343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc04101a90 msgr2=0x7efc04101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.578+0000 7efc09ec7640 1 --2- 172.21.15.78:0/744316343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc04101a90 0x7efc04101e70 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009900 tx=0x7efbf402a410 comp rx=0 tx=0).stop 2024-09-11T01:16:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.579+0000 7efc09ec7640 1 -- 172.21.15.78:0/744316343 shutdown_connections 2024-09-11T01:16:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.579+0000 7efc09ec7640 1 --2- 172.21.15.78:0/744316343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc04106ed0 0x7efc04107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.579+0000 7efc09ec7640 1 --2- 172.21.15.78:0/744316343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc041023b0 0x7efc04106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.579+0000 7efc09ec7640 1 --2- 172.21.15.78:0/744316343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc04101a90 0x7efc04101e70 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.579+0000 7efc09ec7640 1 -- 172.21.15.78:0/744316343 >> 172.21.15.78:0/744316343 conn(0x7efc040fd700 msgr2=0x7efc040ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.579+0000 7efc09ec7640 1 -- 172.21.15.78:0/744316343 shutdown_connections 2024-09-11T01:16:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.579+0000 7efc09ec7640 1 -- 172.21.15.78:0/744316343 wait complete. 2024-09-11T01:16:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.580+0000 7efc09ec7640 1 Processor -- start 2024-09-11T01:16:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.580+0000 7efc09ec7640 1 -- start start 2024-09-11T01:16:11.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.580+0000 7efc09ec7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc04101a90 0x7efc04074780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.580+0000 7efc09ec7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc041023b0 0x7efc04074cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.580+0000 7efc09ec7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc04106ed0 0x7efc0406dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.580+0000 7efc09ec7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc04119390 con 0x7efc041023b0 2024-09-11T01:16:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.580+0000 7efc09ec7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc04119210 con 0x7efc04101a90 2024-09-11T01:16:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.580+0000 7efc09ec7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc04119510 con 0x7efc04106ed0 2024-09-11T01:16:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.581+0000 7efc02ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc041023b0 0x7efc04074cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.581+0000 7efc02ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc041023b0 0x7efc04074cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46792/0 (socket says 172.21.15.78:46792) 2024-09-11T01:16:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.581+0000 7efc02ffd640 1 -- 172.21.15.78:0/788332586 learned_addr learned my addr 172.21.15.78:0/788332586 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.581+0000 7efc03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc04106ed0 0x7efc0406dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.581+0000 7efc037fe640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc04101a90 0x7efc04074780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.581+0000 7efc02ffd640 1 -- 172.21.15.78:0/788332586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc04106ed0 msgr2=0x7efc0406dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.581+0000 7efc02ffd640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc04106ed0 0x7efc0406dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.581+0000 7efc02ffd640 1 -- 172.21.15.78:0/788332586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc04101a90 msgr2=0x7efc04074780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.581+0000 7efc02ffd640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc04101a90 0x7efc04074780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.581+0000 7efc02ffd640 1 -- 172.21.15.78:0/788332586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0406e3a0 con 0x7efc041023b0 2024-09-11T01:16:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.581+0000 7efc037fe640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc04101a90 0x7efc04074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:16:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.581+0000 7efc02ffd640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc041023b0 0x7efc04074cc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7efbf800e9c0 tx=0x7efbf800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.582+0000 7efc00ff9640 1 -- 172.21.15.78:0/788332586 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf800ccc0 con 0x7efc041023b0 2024-09-11T01:16:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.582+0000 7efc00ff9640 1 -- 172.21.15.78:0/788332586 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf80035f0 con 0x7efc041023b0 2024-09-11T01:16:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.582+0000 7efc00ff9640 1 -- 172.21.15.78:0/788332586 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf800ce60 con 0x7efc041023b0 2024-09-11T01:16:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.582+0000 7efc09ec7640 1 -- 172.21.15.78:0/788332586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0406e690 con 0x7efc041023b0 2024-09-11T01:16:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.582+0000 7efc09ec7640 1 -- 172.21.15.78:0/788332586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc040735c0 con 0x7efc041023b0 2024-09-11T01:16:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.586+0000 7efc0c29c740 1 Processor -- start 2024-09-11T01:16:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.587+0000 7efc0c29c740 1 -- start start 2024-09-11T01:16:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.587+0000 7efc0c29c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e73b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.587+0000 7efc0c29c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73bda0 0x56108e7477f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.587+0000 7efc0c29c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e747d30 0x56108e74a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.587+0000 7efc0c29c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56108e6b1e50 con 0x56108e73bda0 2024-09-11T01:16:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.587+0000 7efc0c29c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56108e6a1280 con 0x56108e747d30 2024-09-11T01:16:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.587+0000 7efc0c29c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56108e6a5090 con 0x56108e73b480 2024-09-11T01:16:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.588+0000 7efc03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e73b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.588+0000 7efc03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e73b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42944/0 (socket says 172.21.15.78:42944) 2024-09-11T01:16:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.588+0000 7efc03fff640 1 -- 172.21.15.78:0/3452094270 learned_addr learned my addr 172.21.15.78:0/3452094270 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.589+0000 7efc03fff640 1 -- 172.21.15.78:0/3452094270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e747d30 msgr2=0x56108e74a120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:16:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.589+0000 7efc03fff640 1 --2- 172.21.15.78:0/3452094270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e747d30 0x56108e74a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.589+0000 7efc03fff640 1 -- 172.21.15.78:0/3452094270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73bda0 msgr2=0x56108e7477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.589+0000 7efc03fff640 1 --2- 172.21.15.78:0/3452094270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73bda0 0x56108e7477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.589+0000 7efc03fff640 1 -- 172.21.15.78:0/3452094270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56108e3ff650 con 0x56108e73b480 2024-09-11T01:16:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.589+0000 7efc03fff640 1 --2- 172.21.15.78:0/3452094270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e73b860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009fb0 tx=0x7efbfc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8069c132bc70ff5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.590+0000 7efbe9ffb640 1 -- 172.21.15.78:0/3452094270 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc004d20 con 0x56108e73b480 2024-09-11T01:16:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.590+0000 7efbe9ffb640 1 -- 172.21.15.78:0/3452094270 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbfc004050 con 0x56108e73b480 2024-09-11T01:16:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.590+0000 7efbe9ffb640 1 -- 172.21.15.78:0/3452094270 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc005280 con 0x56108e73b480 2024-09-11T01:16:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.591+0000 7efc00ff9640 1 -- 172.21.15.78:0/788332586 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7efbf8004000 con 0x7efc041023b0 2024-09-11T01:16:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.592+0000 7efc00ff9640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efbec0b54d0 0x7efbec0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.592+0000 7efc00ff9640 1 -- 172.21.15.78:0/788332586 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8493+0+0 (secure 0 0 0) 0x7efbf80e6290 con 0x7efc041023b0 2024-09-11T01:16:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.593+0000 7efc037fe640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efbec0b54d0 0x7efbec0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.594+0000 7efc0c29c740 1 -- 172.21.15.78:0/3452094270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 msgr2=0x56108e73b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.594+0000 7efc037fe640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efbec0b54d0 0x7efbec0b7990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efbf4000c00 tx=0x7efbf4005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.594+0000 7efc0c29c740 1 --2- 172.21.15.78:0/3452094270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e73b860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009fb0 tx=0x7efbfc0118d0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.594+0000 7efc0c29c740 1 -- 172.21.15.78:0/3452094270 shutdown_connections 2024-09-11T01:16:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.594+0000 7efc0c29c740 1 --2- 172.21.15.78:0/3452094270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e747d30 0x56108e74a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.594+0000 7efc0c29c740 1 --2- 172.21.15.78:0/3452094270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73bda0 0x56108e7477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.594+0000 7efc0c29c740 1 --2- 172.21.15.78:0/3452094270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e73b860 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.594+0000 7efc0c29c740 1 -- 172.21.15.78:0/3452094270 >> 172.21.15.78:0/3452094270 conn(0x56108e7392b0 msgr2=0x56108e73a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.594+0000 7efc0c29c740 1 -- 172.21.15.78:0/3452094270 shutdown_connections 2024-09-11T01:16:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.594+0000 7efc0c29c740 1 -- 172.21.15.78:0/3452094270 wait complete. 2024-09-11T01:16:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.595+0000 7efc0c29c740 1 -- start start 2024-09-11T01:16:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.595+0000 7efc0c29c740 1 Processor -- start 2024-09-11T01:16:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.595+0000 7efc0c29c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e7476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.596+0000 7efc0c29c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e73bda0 0x56108e744b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.596+0000 7efc0c29c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e747d30 0x56108e7451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.596+0000 7efc0c29c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56108e502700 con 0x56108e747d30 2024-09-11T01:16:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.596+0000 7efc0c29c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56108e63c110 con 0x56108e73bda0 2024-09-11T01:16:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.596+0000 7efc0c29c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56108e74a9b0 con 0x56108e73b480 2024-09-11T01:16:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.596+0000 7efc0c29c740 10 client.0 _ll_register_callbacks cb 0x56108e655990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:16:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.596+0000 7efc037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e7476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.596+0000 7efc037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e7476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42946/0 (socket says 172.21.15.78:42946) 2024-09-11T01:16:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.596+0000 7efc037fe640 1 -- 172.21.15.78:0/2236159423 learned_addr learned my addr 172.21.15.78:0/2236159423 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.596+0000 7efc02ffd640 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e73bda0 0x56108e744b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.596+0000 7efc03fff640 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e747d30 0x56108e7451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.597+0000 7efc037fe640 1 -- 172.21.15.78:0/2236159423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e73bda0 msgr2=0x56108e744b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.597+0000 7efc037fe640 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e73bda0 0x56108e744b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.597+0000 7efc037fe640 1 -- 172.21.15.78:0/2236159423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e747d30 msgr2=0x56108e7451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.597+0000 7efc037fe640 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e747d30 0x56108e7451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.597+0000 7efc037fe640 1 -- 172.21.15.78:0/2236159423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56108e669c40 con 0x56108e73b480 2024-09-11T01:16:11.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.598+0000 7efc037fe640 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e7476d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efbf40092a0 tx=0x7efbf402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8226d1d17cccf92 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:11.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.598+0000 7efbe2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:16:11.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.598+0000 7efbe2ffd640 1 -- 172.21.15.78:0/2236159423 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf404c020 con 0x56108e73b480 2024-09-11T01:16:11.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.598+0000 7efc0c29c740 1 -- 172.21.15.78:0/2236159423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbfc004050 con 0x56108e73b480 2024-09-11T01:16:11.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.598+0000 7efbe2ffd640 1 -- 172.21.15.78:0/2236159423 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8493+0+0 (secure 0 0 0) 0x7efbf40048c0 con 0x56108e73b480 2024-09-11T01:16:11.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.598+0000 7efbe2ffd640 1 -- 172.21.15.78:0/2236159423 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7efbf4057050 con 0x56108e73b480 2024-09-11T01:16:11.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.598+0000 7efc0c29c740 10 client.27706 fetch_fsmap learned FSMap version 70 2024-09-11T01:16:11.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.598+0000 7efc0c29c740 1 -- 172.21.15.78:0/2236159423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x56108e7563e0 con 0x56108e73b480 2024-09-11T01:16:11.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.599+0000 7efbe2ffd640 1 -- 172.21.15.78:0/2236159423 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbfc004050 con 0x56108e73b480 2024-09-11T01:16:11.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.599+0000 7efbe2ffd640 1 -- 172.21.15.78:0/2236159423 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7efbf4003b80 con 0x56108e73b480 2024-09-11T01:16:11.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.599+0000 7efc0c29c740 10 client.27706 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:16:11.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.599+0000 7efc0c29c740 10 client.27706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:16:11.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.599+0000 7efc0c29c740 20 client.27706 populate_metadata read hostname 'smithi078' 2024-09-11T01:16:11.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.599+0000 7efc0c29c740 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56108e757c10 0x56108e75a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.599+0000 7efc0c29c740 4 client.27706 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:16:11.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.599+0000 7efc0c29c740 1 -- 172.21.15.78:0/2236159423 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56108e6b3da0 con 0x56108e757c10 2024-09-11T01:16:11.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.600+0000 7efc02ffd640 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56108e757c10 0x56108e75a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.600+0000 7efc02ffd640 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56108e757c10 0x56108e75a060 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bbf206b32d87ad1 server_cookie=71de046b8672069e in_seq=0 out_seq=0 2024-09-11T01:16:11.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.600+0000 7efbe2ffd640 10 client.27706 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:16:11.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efbe2ffd640 1 -- 172.21.15.78:0/2236159423 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56108e6b3da0 con 0x56108e757c10 2024-09-11T01:16:11.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efbe2ffd640 10 client.27706 handle_command_reply: tid=0 2024-09-11T01:16:11.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efc0c29c740 1 client.27706 shutdown 2024-09-11T01:16:11.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efc0c29c740 1 -- 172.21.15.78:0/2236159423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 msgr2=0x56108e7476d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efc0c29c740 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e7476d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efbf40092a0 tx=0x7efbf402a410 comp rx=0 tx=0).stop 2024-09-11T01:16:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efc0c29c740 1 -- 172.21.15.78:0/2236159423 shutdown_connections 2024-09-11T01:16:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efc0c29c740 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56108e757c10 0x56108e75a060 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efc0c29c740 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e747d30 0x56108e7451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efc0c29c740 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e73bda0 0x56108e744b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efc0c29c740 1 --2- 172.21.15.78:0/2236159423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e7476d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efc0c29c740 1 -- 172.21.15.78:0/2236159423 >> 172.21.15.78:0/2236159423 conn(0x56108e7392b0 msgr2=0x56108e73ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efbe2ffd640 0 client.27706 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:16:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.602+0000 7efc0c29c740 1 -- 172.21.15.78:0/2236159423 shutdown_connections 2024-09-11T01:16:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.603+0000 7efc0c29c740 1 -- 172.21.15.78:0/2236159423 wait complete. 2024-09-11T01:16:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.603+0000 7efc0c29c740 20 client.27706 trim_cache size 0 max 16384 2024-09-11T01:16:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc0c29c740 1 Processor -- start 2024-09-11T01:16:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc0c29c740 1 -- start start 2024-09-11T01:16:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc0c29c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e744af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc0c29c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73bda0 0x56108e744010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc0c29c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e747d30 0x56108e73fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc0c29c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc041bd490 con 0x56108e73bda0 2024-09-11T01:16:11.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc0c29c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc041befd0 con 0x56108e747d30 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc0c29c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc041bdb60 con 0x56108e73b480 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e744af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc02ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73bda0 0x56108e744010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc02ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73bda0 0x56108e744010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46816/0 (socket says 172.21.15.78:46816) 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc02ffd640 1 -- 172.21.15.78:0/4259997782 learned_addr learned my addr 172.21.15.78:0/4259997782 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.635+0000 7efc03fff640 1 --2- 172.21.15.78:0/4259997782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e747d30 0x56108e73fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.636+0000 7efc02ffd640 1 -- 172.21.15.78:0/4259997782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 msgr2=0x56108e744af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.636+0000 7efc02ffd640 1 --2- 172.21.15.78:0/4259997782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e744af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.636+0000 7efc02ffd640 1 -- 172.21.15.78:0/4259997782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e747d30 msgr2=0x56108e73fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.636+0000 7efc02ffd640 1 --2- 172.21.15.78:0/4259997782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e747d30 0x56108e73fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.636+0000 7efc02ffd640 1 -- 172.21.15.78:0/4259997782 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56108e754020 con 0x56108e73bda0 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.636+0000 7efc02ffd640 1 --2- 172.21.15.78:0/4259997782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73bda0 0x56108e744010 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7efbf800c6a0 tx=0x7efbf8009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61fba36c1797570 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.636+0000 7efbe97fa640 1 -- 172.21.15.78:0/4259997782 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf80ace30 con 0x56108e73bda0 2024-09-11T01:16:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.636+0000 7efbe97fa640 1 -- 172.21.15.78:0/4259997782 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf80ac730 con 0x56108e73bda0 2024-09-11T01:16:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.636+0000 7efbe97fa640 1 -- 172.21.15.78:0/4259997782 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf80a89a0 con 0x56108e73bda0 2024-09-11T01:16:11.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.637+0000 7efc0c29c740 1 -- 172.21.15.78:0/4259997782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73bda0 msgr2=0x56108e744010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:11.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.637+0000 7efc0c29c740 1 --2- 172.21.15.78:0/4259997782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73bda0 0x56108e744010 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7efbf800c6a0 tx=0x7efbf8009740 comp rx=0 tx=0).stop 2024-09-11T01:16:11.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.637+0000 7efc0c29c740 1 -- 172.21.15.78:0/4259997782 shutdown_connections 2024-09-11T01:16:11.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.637+0000 7efc0c29c740 1 --2- 172.21.15.78:0/4259997782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e747d30 0x56108e73fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.637+0000 7efc0c29c740 1 --2- 172.21.15.78:0/4259997782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73bda0 0x56108e744010 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.637+0000 7efc0c29c740 1 --2- 172.21.15.78:0/4259997782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e73b480 0x56108e744af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.637+0000 7efc0c29c740 1 -- 172.21.15.78:0/4259997782 >> 172.21.15.78:0/4259997782 conn(0x56108e7392b0 msgr2=0x56108e73a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:11.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.637+0000 7efc0c29c740 1 -- 172.21.15.78:0/4259997782 shutdown_connections 2024-09-11T01:16:11.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.638+0000 7efc0c29c740 1 -- 172.21.15.78:0/4259997782 wait complete. 2024-09-11T01:16:11.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.638+0000 7efc0c29c740 1 -- start start 2024-09-11T01:16:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.638+0000 7efc0c29c740 1 Processor -- start 2024-09-11T01:16:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.639+0000 7efc0c29c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73b480 0x56108e750450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.639+0000 7efc0c29c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e73bda0 0x56108e748da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.639+0000 7efc0c29c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e747d30 0x56108e749300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.639+0000 7efc0c29c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc041be900 con 0x56108e73b480 2024-09-11T01:16:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.639+0000 7efc0c29c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc041be230 con 0x56108e73bda0 2024-09-11T01:16:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.639+0000 7efc0c29c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc041bb490 con 0x56108e747d30 2024-09-11T01:16:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.639+0000 7efc0c29c740 10 client.0 _ll_register_callbacks cb 0x7efbf403f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:16:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.639+0000 7efc03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e747d30 0x56108e749300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.639+0000 7efc02ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e73bda0 0x56108e748da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.639+0000 7efc03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e747d30 0x56108e749300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42974/0 (socket says 172.21.15.78:42974) 2024-09-11T01:16:11.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.639+0000 7efc03fff640 1 -- 172.21.15.78:0/754670754 learned_addr learned my addr 172.21.15.78:0/754670754 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:11.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.639+0000 7efc037fe640 1 --2- 172.21.15.78:0/754670754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73b480 0x56108e750450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.640+0000 7efc03fff640 1 -- 172.21.15.78:0/754670754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e73bda0 msgr2=0x56108e748da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:11.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.640+0000 7efc03fff640 1 --2- 172.21.15.78:0/754670754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e73bda0 0x56108e748da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.640+0000 7efc03fff640 1 -- 172.21.15.78:0/754670754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73b480 msgr2=0x56108e750450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:11.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.640+0000 7efc03fff640 1 --2- 172.21.15.78:0/754670754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73b480 0x56108e750450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.640+0000 7efc03fff640 1 -- 172.21.15.78:0/754670754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56108e749b30 con 0x56108e747d30 2024-09-11T01:16:11.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.640+0000 7efc03fff640 1 --2- 172.21.15.78:0/754670754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e747d30 0x56108e749300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009230 tx=0x7efbfc004810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f140f980fc78ebe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:11.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.640+0000 7efbe2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:16:11.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.640+0000 7efbe2ffd640 1 -- 172.21.15.78:0/754670754 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc004250 con 0x56108e747d30 2024-09-11T01:16:11.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.640+0000 7efbe2ffd640 1 -- 172.21.15.78:0/754670754 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8493+0+0 (secure 0 0 0) 0x7efbfc0043f0 con 0x56108e747d30 2024-09-11T01:16:11.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.640+0000 7efc0c29c740 1 -- 172.21.15.78:0/754670754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf80ac730 con 0x56108e747d30 2024-09-11T01:16:11.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.640+0000 7efbe2ffd640 1 -- 172.21.15.78:0/754670754 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7efbfc00f4e0 con 0x56108e747d30 2024-09-11T01:16:11.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.641+0000 7efc0c29c740 10 client.27709 fetch_fsmap learned FSMap version 70 2024-09-11T01:16:11.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.641+0000 7efc0c29c740 1 -- 172.21.15.78:0/754670754 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x56108e7c4c30 con 0x56108e747d30 2024-09-11T01:16:11.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.641+0000 7efbe2ffd640 1 -- 172.21.15.78:0/754670754 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf80ac730 con 0x56108e747d30 2024-09-11T01:16:11.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.641+0000 7efbe2ffd640 1 -- 172.21.15.78:0/754670754 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7efbfc033070 con 0x56108e747d30 2024-09-11T01:16:11.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.641+0000 7efc0c29c740 10 client.27709 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:16:11.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.641+0000 7efc0c29c740 10 client.27709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:16:11.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.641+0000 7efc0c29c740 20 client.27709 populate_metadata read hostname 'smithi078' 2024-09-11T01:16:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.641+0000 7efc0c29c740 1 --2- 172.21.15.78:0/754670754 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56108e7c5190 0x56108e7c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.641+0000 7efc0c29c740 4 client.27709 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:16:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.641+0000 7efc0c29c740 1 -- 172.21.15.78:0/754670754 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56108e6af850 con 0x56108e7c5190 2024-09-11T01:16:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.642+0000 7efc037fe640 1 --2- 172.21.15.78:0/754670754 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56108e7c5190 0x56108e7c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.643+0000 7efc037fe640 1 --2- 172.21.15.78:0/754670754 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56108e7c5190 0x56108e7c75b0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c622598059bdbab9 server_cookie=b0d791b96de4ceb5 in_seq=0 out_seq=0 2024-09-11T01:16:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.643+0000 7efbe2ffd640 10 client.27709 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:16:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.644+0000 7efbe2ffd640 1 -- 172.21.15.78:0/754670754 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6534 (crc 0 0 0) 0x56108e6af850 con 0x56108e7c5190 2024-09-11T01:16:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.644+0000 7efbe2ffd640 10 client.27709 handle_command_reply: tid=0 2024-09-11T01:16:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.644+0000 7efc0c29c740 1 client.27709 shutdown 2024-09-11T01:16:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.644+0000 7efc0c29c740 1 -- 172.21.15.78:0/754670754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e747d30 msgr2=0x56108e749300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.644+0000 7efc0c29c740 1 --2- 172.21.15.78:0/754670754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e747d30 0x56108e749300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009230 tx=0x7efbfc004810 comp rx=0 tx=0).stop 2024-09-11T01:16:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.644+0000 7efc0c29c740 1 -- 172.21.15.78:0/754670754 shutdown_connections 2024-09-11T01:16:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.644+0000 7efc0c29c740 1 --2- 172.21.15.78:0/754670754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56108e747d30 0x56108e749300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.644+0000 7efc0c29c740 1 --2- 172.21.15.78:0/754670754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56108e73bda0 0x56108e748da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.644+0000 7efc0c29c740 1 --2- 172.21.15.78:0/754670754 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56108e7c5190 0x56108e7c75b0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.644+0000 7efc0c29c740 1 --2- 172.21.15.78:0/754670754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56108e73b480 0x56108e750450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.644+0000 7efc0c29c740 1 -- 172.21.15.78:0/754670754 >> 172.21.15.78:0/754670754 conn(0x56108e7392b0 msgr2=0x56108e73fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.645+0000 7efbe2ffd640 0 client.27709 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:16:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.645+0000 7efc0c29c740 1 -- 172.21.15.78:0/754670754 shutdown_connections 2024-09-11T01:16:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.645+0000 7efc0c29c740 1 -- 172.21.15.78:0/754670754 wait complete. 2024-09-11T01:16:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.645+0000 7efc0c29c740 20 client.27709 trim_cache size 0 max 16384 2024-09-11T01:16:11.654 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:16:11.654 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:16:11.654 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:16:11.654 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:16:11.654 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.654 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "33c6d527-1d18-4220-8f09-6a413e212ad7", 2024-09-11T01:16:11.654 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:16:11.654 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:16:11.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.654 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.655 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.656 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.656 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:16:11.656 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:16:11.656 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.656 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.656 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.656 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.656 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.656 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:16:11.656 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.658 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.661 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:16:11.662 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.663 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.664 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:11.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:11.668 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:16:11.668 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:16:11.668 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:16:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.646+0000 7efc09ec7640 1 -- 172.21.15.78:0/788332586 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efbec0b54d0 msgr2=0x7efbec0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:11.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.646+0000 7efc09ec7640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efbec0b54d0 0x7efbec0b7990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efbf4000c00 tx=0x7efbf4005a20 comp rx=0 tx=0).stop 2024-09-11T01:16:11.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.646+0000 7efc09ec7640 1 -- 172.21.15.78:0/788332586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc041023b0 msgr2=0x7efc04074cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:11.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.646+0000 7efc09ec7640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc041023b0 0x7efc04074cc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7efbf800e9c0 tx=0x7efbf800ee80 comp rx=0 tx=0).stop 2024-09-11T01:16:11.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.646+0000 7efc09ec7640 1 -- 172.21.15.78:0/788332586 shutdown_connections 2024-09-11T01:16:11.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.646+0000 7efc09ec7640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7efbec0b54d0 0x7efbec0b7990 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.646+0000 7efc09ec7640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc04106ed0 0x7efc0406dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.646+0000 7efc09ec7640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc041023b0 0x7efc04074cc0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.646+0000 7efc09ec7640 1 --2- 172.21.15.78:0/788332586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc04101a90 0x7efc04074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:11.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.646+0000 7efc09ec7640 1 -- 172.21.15.78:0/788332586 >> 172.21.15.78:0/788332586 conn(0x7efc040fd700 msgr2=0x7efc04104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:11.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.646+0000 7efc09ec7640 1 -- 172.21.15.78:0/788332586 shutdown_connections 2024-09-11T01:16:11.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:11.646+0000 7efc09ec7640 1 -- 172.21.15.78:0/788332586 wait complete. 2024-09-11T01:16:11.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 169481 35.12 MB/sec execute 137 sec latency 273.534 ms 2024-09-11T01:16:12.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:12 smithi110 ceph-mon[34712]: pgmap v863: 97 pgs: 97 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 581 op/s 2024-09-11T01:16:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:12 smithi112 ceph-mon[35384]: pgmap v863: 97 pgs: 97 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 581 op/s 2024-09-11T01:16:12.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:12 smithi078 ceph-mon[31315]: pgmap v863: 97 pgs: 97 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 581 op/s 2024-09-11T01:16:12.721 INFO:tasks.cephfs.filesystem:scrub status for tag:33c6d527-1d18-4220-8f09-6a413e212ad7 - {'path': '/', 'tag': '33c6d527-1d18-4220-8f09-6a413e212ad7', 'options': 'recursive,force'} 2024-09-11T01:16:12.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 170187 35.14 MB/sec execute 138 sec latency 80.654 ms 2024-09-11T01:16:13.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 170895 35.19 MB/sec execute 139 sec latency 140.566 ms 2024-09-11T01:16:14.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:14 smithi110 ceph-mon[34712]: pgmap v864: 97 pgs: 97 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 511 op/s 2024-09-11T01:16:14.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:14 smithi110 ceph-mon[34712]: osdmap e99: 12 total, 12 up, 12 in 2024-09-11T01:16:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:14 smithi078 ceph-mon[31315]: pgmap v864: 97 pgs: 97 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 511 op/s 2024-09-11T01:16:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:14 smithi078 ceph-mon[31315]: osdmap e99: 12 total, 12 up, 12 in 2024-09-11T01:16:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:14 smithi112 ceph-mon[35384]: pgmap v864: 97 pgs: 97 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 511 op/s 2024-09-11T01:16:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:14 smithi112 ceph-mon[35384]: osdmap e99: 12 total, 12 up, 12 in 2024-09-11T01:16:14.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 171577 35.20 MB/sec execute 140 sec latency 57.767 ms 2024-09-11T01:16:15.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 172308 35.20 MB/sec execute 141 sec latency 183.380 ms 2024-09-11T01:16:16.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:16 smithi110 ceph-mon[34712]: pgmap v866: 97 pgs: 97 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 610 op/s 2024-09-11T01:16:16.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:16 smithi112 ceph-mon[35384]: pgmap v866: 97 pgs: 97 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 610 op/s 2024-09-11T01:16:16.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:16 smithi078 ceph-mon[31315]: pgmap v866: 97 pgs: 97 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 610 op/s 2024-09-11T01:16:16.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 173106 35.27 MB/sec execute 142 sec latency 65.092 ms 2024-09-11T01:16:17.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 173822 35.31 MB/sec execute 143 sec latency 77.157 ms 2024-09-11T01:16:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:18 smithi110 ceph-mon[34712]: pgmap v867: 97 pgs: 97 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 786 op/s 2024-09-11T01:16:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:18 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:16:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:18 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:16:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:18 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:16:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:18 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:16:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:18 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:16:18.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:18 smithi112 ceph-mon[35384]: pgmap v867: 97 pgs: 97 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 786 op/s 2024-09-11T01:16:18.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:18 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:16:18.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:18 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:16:18.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:18 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:16:18.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:18 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:16:18.476 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:18 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:16:18.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:18 smithi078 ceph-mon[31315]: pgmap v867: 97 pgs: 97 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 786 op/s 2024-09-11T01:16:18.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:18 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:16:18.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:18 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:16:18.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:18 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:16:18.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:18 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:16:18.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:18 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:16:18.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 174442 35.29 MB/sec execute 144 sec latency 233.199 ms 2024-09-11T01:16:19.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 175199 35.33 MB/sec execute 145 sec latency 83.568 ms 2024-09-11T01:16:20.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:20 smithi110 ceph-mon[34712]: pgmap v868: 97 pgs: 97 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 750 op/s 2024-09-11T01:16:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:20 smithi112 ceph-mon[35384]: pgmap v868: 97 pgs: 97 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 750 op/s 2024-09-11T01:16:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:20 smithi078 ceph-mon[31315]: pgmap v868: 97 pgs: 97 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 750 op/s 2024-09-11T01:16:20.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 175922 35.35 MB/sec execute 146 sec latency 102.496 ms 2024-09-11T01:16:21.788 INFO:tasks.workunit.client.0.smithi078.stdout: 10 176691 35.39 MB/sec execute 147 sec latency 137.676 ms 2024-09-11T01:16:22.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:22 smithi110 ceph-mon[34712]: pgmap v869: 97 pgs: 97 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 19 MiB/s wr, 805 op/s 2024-09-11T01:16:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:22 smithi112 ceph-mon[35384]: pgmap v869: 97 pgs: 97 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 19 MiB/s wr, 805 op/s 2024-09-11T01:16:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:22 smithi078 ceph-mon[31315]: pgmap v869: 97 pgs: 97 active+clean; 2.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 19 MiB/s wr, 805 op/s 2024-09-11T01:16:22.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 177297 35.37 MB/sec execute 148 sec latency 223.385 ms 2024-09-11T01:16:23.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 177939 35.37 MB/sec execute 149 sec latency 191.466 ms 2024-09-11T01:16:24.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:24 smithi110 ceph-mon[34712]: pgmap v870: 97 pgs: 97 active+clean; 2.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 20 MiB/s wr, 853 op/s 2024-09-11T01:16:24.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:24 smithi112 ceph-mon[35384]: pgmap v870: 97 pgs: 97 active+clean; 2.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 20 MiB/s wr, 853 op/s 2024-09-11T01:16:24.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:24 smithi078 ceph-mon[31315]: pgmap v870: 97 pgs: 97 active+clean; 2.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 20 MiB/s wr, 853 op/s 2024-09-11T01:16:24.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 178583 35.34 MB/sec execute 150 sec latency 223.593 ms 2024-09-11T01:16:25.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 179366 35.38 MB/sec execute 151 sec latency 79.237 ms 2024-09-11T01:16:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:26 smithi110 ceph-mon[34712]: pgmap v871: 97 pgs: 97 active+clean; 2.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 789 op/s 2024-09-11T01:16:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:26 smithi112 ceph-mon[35384]: pgmap v871: 97 pgs: 97 active+clean; 2.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 789 op/s 2024-09-11T01:16:26.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:26 smithi078 ceph-mon[31315]: pgmap v871: 97 pgs: 97 active+clean; 2.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 789 op/s 2024-09-11T01:16:26.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 180212 35.50 MB/sec execute 152 sec latency 113.410 ms 2024-09-11T01:16:27.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 180920 35.50 MB/sec execute 153 sec latency 190.819 ms 2024-09-11T01:16:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:28 smithi110 ceph-mon[34712]: pgmap v872: 97 pgs: 97 active+clean; 2.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 20 MiB/s wr, 866 op/s 2024-09-11T01:16:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:28 smithi112 ceph-mon[35384]: pgmap v872: 97 pgs: 97 active+clean; 2.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 20 MiB/s wr, 866 op/s 2024-09-11T01:16:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:28 smithi078 ceph-mon[31315]: pgmap v872: 97 pgs: 97 active+clean; 2.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 20 MiB/s wr, 866 op/s 2024-09-11T01:16:28.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 181479 35.44 MB/sec execute 154 sec latency 247.291 ms 2024-09-11T01:16:29.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 182150 35.46 MB/sec execute 155 sec latency 260.625 ms 2024-09-11T01:16:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:30 smithi110 ceph-mon[34712]: pgmap v873: 97 pgs: 97 active+clean; 2.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 778 op/s 2024-09-11T01:16:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:30 smithi112 ceph-mon[35384]: pgmap v873: 97 pgs: 97 active+clean; 2.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 778 op/s 2024-09-11T01:16:30.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:30 smithi078 ceph-mon[31315]: pgmap v873: 97 pgs: 97 active+clean; 2.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 778 op/s 2024-09-11T01:16:30.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 182894 35.46 MB/sec execute 156 sec latency 102.655 ms 2024-09-11T01:16:31.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:31 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:16:31.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:31 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:16:31.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:31 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:16:31.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 183343 35.42 MB/sec execute 157 sec latency 457.160 ms 2024-09-11T01:16:32.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:32 smithi078 ceph-mon[31315]: pgmap v874: 97 pgs: 97 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 19 MiB/s wr, 825 op/s 2024-09-11T01:16:32.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:16:32.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:16:32.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:16:32.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:32 smithi110 ceph-mon[34712]: pgmap v874: 97 pgs: 97 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 19 MiB/s wr, 825 op/s 2024-09-11T01:16:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:16:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:16:32.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:16:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:32 smithi112 ceph-mon[35384]: pgmap v874: 97 pgs: 97 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 19 MiB/s wr, 825 op/s 2024-09-11T01:16:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T01:16:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:16:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:16:32.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:16:32.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 184177 35.51 MB/sec execute 158 sec latency 73.838 ms 2024-09-11T01:16:33.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:33 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi110 to 176.0M 2024-09-11T01:16:33.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:33 smithi078 ceph-mon[31315]: Unable to set osd_memory_target on smithi110 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-11T01:16:33.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:33 smithi078 ceph-mon[31315]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-11T01:16:33.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:33 smithi078 ceph-mon[31315]: Unable to set osd_memory_target on smithi112 to 184567398: error parsing value: Value '184567398' is below minimum 939524096 2024-09-11T01:16:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:33 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi110 to 176.0M 2024-09-11T01:16:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:33 smithi110 ceph-mon[34712]: Unable to set osd_memory_target on smithi110 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-11T01:16:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:33 smithi110 ceph-mon[34712]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-11T01:16:33.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:33 smithi110 ceph-mon[34712]: Unable to set osd_memory_target on smithi112 to 184567398: error parsing value: Value '184567398' is below minimum 939524096 2024-09-11T01:16:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:33 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi110 to 176.0M 2024-09-11T01:16:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:33 smithi112 ceph-mon[35384]: Unable to set osd_memory_target on smithi110 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-11T01:16:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:33 smithi112 ceph-mon[35384]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-11T01:16:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:33 smithi112 ceph-mon[35384]: Unable to set osd_memory_target on smithi112 to 184567398: error parsing value: Value '184567398' is below minimum 939524096 2024-09-11T01:16:33.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 184907 35.48 MB/sec execute 159 sec latency 54.884 ms 2024-09-11T01:16:34.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:34 smithi110 ceph-mon[34712]: pgmap v875: 97 pgs: 97 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 784 op/s 2024-09-11T01:16:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:34 smithi112 ceph-mon[35384]: pgmap v875: 97 pgs: 97 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 784 op/s 2024-09-11T01:16:34.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:34 smithi078 ceph-mon[31315]: pgmap v875: 97 pgs: 97 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 784 op/s 2024-09-11T01:16:34.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 185706 35.58 MB/sec execute 160 sec latency 50.835 ms 2024-09-11T01:16:35.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 186377 35.52 MB/sec execute 161 sec latency 169.916 ms 2024-09-11T01:16:36.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:36 smithi110 ceph-mon[34712]: pgmap v876: 97 pgs: 97 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 747 op/s 2024-09-11T01:16:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:36 smithi112 ceph-mon[35384]: pgmap v876: 97 pgs: 97 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 747 op/s 2024-09-11T01:16:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:36 smithi078 ceph-mon[31315]: pgmap v876: 97 pgs: 97 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 747 op/s 2024-09-11T01:16:36.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 187031 35.55 MB/sec execute 162 sec latency 197.318 ms 2024-09-11T01:16:37.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 187698 35.56 MB/sec execute 163 sec latency 199.817 ms 2024-09-11T01:16:38.526 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:38 smithi112 ceph-mon[35384]: pgmap v877: 97 pgs: 97 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 721 op/s 2024-09-11T01:16:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:38 smithi110 ceph-mon[34712]: pgmap v877: 97 pgs: 97 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 721 op/s 2024-09-11T01:16:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:38 smithi078 ceph-mon[31315]: pgmap v877: 97 pgs: 97 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 721 op/s 2024-09-11T01:16:38.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 188207 35.53 MB/sec execute 164 sec latency 394.465 ms 2024-09-11T01:16:39.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 188826 35.52 MB/sec execute 165 sec latency 138.387 ms 2024-09-11T01:16:40.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:40 smithi110 ceph-mon[34712]: pgmap v878: 97 pgs: 97 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 620 op/s 2024-09-11T01:16:40.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:40 smithi112 ceph-mon[35384]: pgmap v878: 97 pgs: 97 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 620 op/s 2024-09-11T01:16:40.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:40 smithi078 ceph-mon[31315]: pgmap v878: 97 pgs: 97 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 620 op/s 2024-09-11T01:16:40.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 189459 35.46 MB/sec execute 166 sec latency 187.910 ms 2024-09-11T01:16:41.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 190084 35.45 MB/sec execute 167 sec latency 219.191 ms 2024-09-11T01:16:42.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:42 smithi110 ceph-mon[34712]: pgmap v879: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 764 op/s 2024-09-11T01:16:42.722 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:16:42.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:42 smithi112 ceph-mon[35384]: pgmap v879: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 764 op/s 2024-09-11T01:16:42.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:42 smithi078 ceph-mon[31315]: pgmap v879: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 764 op/s 2024-09-11T01:16:42.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 190921 35.53 MB/sec execute 168 sec latency 147.004 ms 2024-09-11T01:16:43.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 191602 35.57 MB/sec execute 169 sec latency 234.113 ms 2024-09-11T01:16:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:44 smithi110 ceph-mon[34712]: pgmap v880: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 761 op/s 2024-09-11T01:16:44.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:44 smithi112 ceph-mon[35384]: pgmap v880: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 761 op/s 2024-09-11T01:16:44.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:44 smithi078 ceph-mon[31315]: pgmap v880: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 761 op/s 2024-09-11T01:16:44.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 192350 35.59 MB/sec execute 170 sec latency 48.956 ms 2024-09-11T01:16:45.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 192894 35.50 MB/sec execute 171 sec latency 267.091 ms 2024-09-11T01:16:46.731 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:46 smithi112 ceph-mon[35384]: pgmap v881: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 722 op/s 2024-09-11T01:16:47.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:46 smithi110 ceph-mon[34712]: pgmap v881: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 722 op/s 2024-09-11T01:16:47.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:46 smithi078 ceph-mon[31315]: pgmap v881: 97 pgs: 97 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 722 op/s 2024-09-11T01:16:47.516 INFO:tasks.workunit.client.0.smithi078.stdout: 10 193451 35.49 MB/sec execute 172 sec latency 246.251 ms 2024-09-11T01:16:47.516 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:16:47.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 194095 35.53 MB/sec execute 173 sec latency 175.196 ms 2024-09-11T01:16:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:48 smithi110 ceph-mon[34712]: pgmap v882: 97 pgs: 97 active+clean; 2.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 799 op/s 2024-09-11T01:16:48.691 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:48 smithi078 ceph-mon[31315]: pgmap v882: 97 pgs: 97 active+clean; 2.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 799 op/s 2024-09-11T01:16:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:48 smithi112 ceph-mon[35384]: pgmap v882: 97 pgs: 97 active+clean; 2.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 799 op/s 2024-09-11T01:16:48.789 INFO:tasks.workunit.client.0.smithi078.stdout: 10 194707 35.48 MB/sec execute 174 sec latency 168.829 ms 2024-09-11T01:16:49.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:49 smithi110 ceph-mon[34712]: pgmap v883: 97 pgs: 97 active+clean; 2.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 768 op/s 2024-09-11T01:16:49.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:49 smithi112 ceph-mon[35384]: pgmap v883: 97 pgs: 97 active+clean; 2.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 768 op/s 2024-09-11T01:16:49.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:49 smithi078 ceph-mon[31315]: pgmap v883: 97 pgs: 97 active+clean; 2.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 768 op/s 2024-09-11T01:16:49.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 195141 35.41 MB/sec execute 175 sec latency 349.595 ms 2024-09-11T01:16:50.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.019+0000 7f865d207640 1 -- 172.21.15.78:0/3239460795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8658108020 msgr2=0x7f865810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:50.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.019+0000 7f865d207640 1 --2- 172.21.15.78:0/3239460795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8658108020 0x7f865810e820 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8650009ca0 tx=0x7f865002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.019+0000 7f865d207640 1 -- 172.21.15.78:0/3239460795 shutdown_connections 2024-09-11T01:16:50.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.020+0000 7f865d207640 1 --2- 172.21.15.78:0/3239460795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8658108020 0x7f865810e820 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.020+0000 7f865d207640 1 --2- 172.21.15.78:0/3239460795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86581076a0 0x7f8658107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.020+0000 7f865d207640 1 --2- 172.21.15.78:0/3239460795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8658101c90 0x7f8658102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.020+0000 7f865d207640 1 -- 172.21.15.78:0/3239460795 >> 172.21.15.78:0/3239460795 conn(0x7f86580fd700 msgr2=0x7f86580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:50.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.020+0000 7f865d207640 1 -- 172.21.15.78:0/3239460795 shutdown_connections 2024-09-11T01:16:50.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.020+0000 7f865d207640 1 -- 172.21.15.78:0/3239460795 wait complete. 2024-09-11T01:16:50.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.020+0000 7f865d207640 1 Processor -- start 2024-09-11T01:16:50.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f865d207640 1 -- start start 2024-09-11T01:16:50.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f865d207640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8658101c90 0x7f8658197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f865d207640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86581076a0 0x7f8658198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f865d207640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8658108020 0x7f865819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f865d207640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8658111cf0 con 0x7f8658108020 2024-09-11T01:16:50.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f865d207640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8658111b70 con 0x7f86581076a0 2024-09-11T01:16:50.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f865d207640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8658111e70 con 0x7f8658101c90 2024-09-11T01:16:50.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f865759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8658108020 0x7f865819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f865659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86581076a0 0x7f8658198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f865659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86581076a0 0x7f8658198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:45708/0 (socket says 172.21.15.78:45708) 2024-09-11T01:16:50.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f865759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8658108020 0x7f865819cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40190/0 (socket says 172.21.15.78:40190) 2024-09-11T01:16:50.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f8656d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8658101c90 0x7f8658197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.021+0000 7f865659c640 1 -- 172.21.15.78:0/2707066394 learned_addr learned my addr 172.21.15.78:0/2707066394 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:50.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.022+0000 7f865659c640 1 -- 172.21.15.78:0/2707066394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8658101c90 msgr2=0x7f8658197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:50.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.022+0000 7f865659c640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8658101c90 0x7f8658197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.022+0000 7f865659c640 1 -- 172.21.15.78:0/2707066394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8658108020 msgr2=0x7f865819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:50.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.022+0000 7f865659c640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8658108020 0x7f865819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.022+0000 7f865659c640 1 -- 172.21.15.78:0/2707066394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865819d4a0 con 0x7f86581076a0 2024-09-11T01:16:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.022+0000 7f865759e640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8658108020 0x7f865819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:16:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.022+0000 7f865659c640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86581076a0 0x7f8658198450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f864c00e9f0 tx=0x7f864c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.023+0000 7f863ffff640 1 -- 172.21.15.78:0/2707066394 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c00cd80 con 0x7f86581076a0 2024-09-11T01:16:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.023+0000 7f863ffff640 1 -- 172.21.15.78:0/2707066394 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f864c0108e0 con 0x7f86581076a0 2024-09-11T01:16:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.023+0000 7f865d207640 1 -- 172.21.15.78:0/2707066394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865819d790 con 0x7f86581076a0 2024-09-11T01:16:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.023+0000 7f863ffff640 1 -- 172.21.15.78:0/2707066394 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c018980 con 0x7f86581076a0 2024-09-11T01:16:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.023+0000 7f865d207640 1 -- 172.21.15.78:0/2707066394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865819dd20 con 0x7f86581076a0 2024-09-11T01:16:50.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.027+0000 7f865f5dc740 1 Processor -- start 2024-09-11T01:16:50.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.027+0000 7f865f5dc740 1 -- start start 2024-09-11T01:16:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.027+0000 7f865f5dc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb1960480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.027+0000 7f865f5dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 0x55fbb196c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.027+0000 7f865f5dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb196c950 0x55fbb196ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.027+0000 7f865f5dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fbb18d6e50 con 0x55fbb19609c0 2024-09-11T01:16:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.027+0000 7f865f5dc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fbb18c6280 con 0x55fbb19600a0 2024-09-11T01:16:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.027+0000 7f865f5dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fbb18ca090 con 0x55fbb196c950 2024-09-11T01:16:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.027+0000 7f863ffff640 1 -- 172.21.15.78:0/2707066394 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f864c010430 con 0x7f86581076a0 2024-09-11T01:16:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.027+0000 7f865759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb1960480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.027+0000 7f865759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb1960480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:45710/0 (socket says 172.21.15.78:45710) 2024-09-11T01:16:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.027+0000 7f865759e640 1 -- 172.21.15.78:0/544944703 learned_addr learned my addr 172.21.15.78:0/544944703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.028+0000 7f8656d9d640 1 --2- 172.21.15.78:0/544944703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 0x55fbb196c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.028+0000 7f865759e640 1 -- 172.21.15.78:0/544944703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb196c950 msgr2=0x55fbb196ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.028+0000 7f865759e640 1 --2- 172.21.15.78:0/544944703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb196c950 0x55fbb196ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.028+0000 7f865759e640 1 -- 172.21.15.78:0/544944703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 msgr2=0x55fbb196c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.028+0000 7f865759e640 1 --2- 172.21.15.78:0/544944703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 0x55fbb196c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.028+0000 7f865759e640 1 -- 172.21.15.78:0/544944703 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb1624650 con 0x55fbb19600a0 2024-09-11T01:16:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.028+0000 7f8656d9d640 1 --2- 172.21.15.78:0/544944703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 0x55fbb196c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:16:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.028+0000 7f865759e640 1 --2- 172.21.15.78:0/544944703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb1960480 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8650009ca0 tx=0x7f8650005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95392bdf63ff024 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.029+0000 7f863d7fa640 1 -- 172.21.15.78:0/544944703 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650031450 con 0x55fbb19600a0 2024-09-11T01:16:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.029+0000 7f863d7fa640 1 -- 172.21.15.78:0/544944703 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8650036040 con 0x55fbb19600a0 2024-09-11T01:16:50.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.029+0000 7f863d7fa640 1 -- 172.21.15.78:0/544944703 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650004380 con 0x55fbb19600a0 2024-09-11T01:16:50.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.029+0000 7f863ffff640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f86400b54d0 0x7f86400b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.030+0000 7f863ffff640 1 -- 172.21.15.78:0/2707066394 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7f864c0e6180 con 0x7f86581076a0 2024-09-11T01:16:50.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.030+0000 7f865f5dc740 1 -- 172.21.15.78:0/544944703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 msgr2=0x55fbb1960480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:50.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.030+0000 7f865f5dc740 1 --2- 172.21.15.78:0/544944703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb1960480 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8650009ca0 tx=0x7f8650005f80 comp rx=0 tx=0).stop 2024-09-11T01:16:50.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.031+0000 7f865f5dc740 1 -- 172.21.15.78:0/544944703 shutdown_connections 2024-09-11T01:16:50.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.031+0000 7f865f5dc740 1 --2- 172.21.15.78:0/544944703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb196c950 0x55fbb196ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.031+0000 7f865f5dc740 1 --2- 172.21.15.78:0/544944703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 0x55fbb196c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.031+0000 7f865f5dc740 1 --2- 172.21.15.78:0/544944703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb1960480 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.031+0000 7f865f5dc740 1 -- 172.21.15.78:0/544944703 >> 172.21.15.78:0/544944703 conn(0x55fbb195ded0 msgr2=0x55fbb195f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:50.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.032+0000 7f8656d9d640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f86400b54d0 0x7f86400b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.033+0000 7f865f5dc740 1 -- 172.21.15.78:0/544944703 shutdown_connections 2024-09-11T01:16:50.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.033+0000 7f865f5dc740 1 -- 172.21.15.78:0/544944703 wait complete. 2024-09-11T01:16:50.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.033+0000 7f8656d9d640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f86400b54d0 0x7f86400b7990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55fbb195edb0 tx=0x7f8648005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:50.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.033+0000 7f865f5dc740 1 -- start start 2024-09-11T01:16:50.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.033+0000 7f865f5dc740 1 Processor -- start 2024-09-11T01:16:50.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.034+0000 7f865f5dc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb196c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.034+0000 7f865f5dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 0x55fbb1969750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.034+0000 7f865f5dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb196c950 0x55fbb1969e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.034+0000 7f865f5dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fbb1727700 con 0x55fbb196c950 2024-09-11T01:16:50.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.034+0000 7f865f5dc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fbb1861110 con 0x55fbb19600a0 2024-09-11T01:16:50.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.034+0000 7f865f5dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fbb196f3f0 con 0x55fbb19609c0 2024-09-11T01:16:50.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.034+0000 7f865659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 0x55fbb1969750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.034+0000 7f865659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 0x55fbb1969750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:34470/0 (socket says 172.21.15.78:34470) 2024-09-11T01:16:50.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.034+0000 7f865659c640 1 -- 172.21.15.78:0/2433499198 learned_addr learned my addr 172.21.15.78:0/2433499198 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:50.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.034+0000 7f865f5dc740 10 client.0 _ll_register_callbacks cb 0x55fbb187a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:16:50.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.034+0000 7f865759e640 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb196c950 0x55fbb1969e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.035+0000 7f8656d9d640 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb196c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.035+0000 7f865659c640 1 -- 172.21.15.78:0/2433499198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 msgr2=0x55fbb196c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:50.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.035+0000 7f865659c640 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb196c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.035+0000 7f865659c640 1 -- 172.21.15.78:0/2433499198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb196c950 msgr2=0x55fbb1969e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:50.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.035+0000 7f865659c640 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb196c950 0x55fbb1969e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.035+0000 7f865659c640 1 -- 172.21.15.78:0/2433499198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb188ec40 con 0x55fbb19609c0 2024-09-11T01:16:50.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.035+0000 7f865659c640 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 0x55fbb1969750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f864c0ac680 tx=0x7f864c0ac580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a4914545e943c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:50.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.035+0000 7f863aff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:16:50.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.036+0000 7f863aff5640 1 -- 172.21.15.78:0/2433499198 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c0ac6f0 con 0x55fbb19609c0 2024-09-11T01:16:50.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.036+0000 7f863aff5640 1 -- 172.21.15.78:0/2433499198 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7f864c0ab950 con 0x55fbb19609c0 2024-09-11T01:16:50.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.036+0000 7f865f5dc740 1 -- 172.21.15.78:0/2433499198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8650036040 con 0x55fbb19609c0 2024-09-11T01:16:50.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.036+0000 7f863aff5640 1 -- 172.21.15.78:0/2433499198 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f864c0b1120 con 0x55fbb19609c0 2024-09-11T01:16:50.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.036+0000 7f865f5dc740 10 client.27718 fetch_fsmap learned FSMap version 70 2024-09-11T01:16:50.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.036+0000 7f865f5dc740 1 -- 172.21.15.78:0/2433499198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55fbb197ae30 con 0x55fbb19609c0 2024-09-11T01:16:50.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.037+0000 7f863aff5640 1 -- 172.21.15.78:0/2433499198 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8650036040 con 0x55fbb19609c0 2024-09-11T01:16:50.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.037+0000 7f863aff5640 1 -- 172.21.15.78:0/2433499198 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f864c0e8110 con 0x55fbb19609c0 2024-09-11T01:16:50.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.037+0000 7f865f5dc740 10 client.27718 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:16:50.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.037+0000 7f865f5dc740 10 client.27718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:16:50.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.037+0000 7f865f5dc740 20 client.27718 populate_metadata read hostname 'smithi078' 2024-09-11T01:16:50.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.037+0000 7f865f5dc740 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fbb197c2b0 0x55fbb197e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.037+0000 7f865f5dc740 4 client.27718 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:16:50.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.037+0000 7f865f5dc740 1 -- 172.21.15.78:0/2433499198 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbb18d8da0 con 0x55fbb197c2b0 2024-09-11T01:16:50.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.038+0000 7f8656d9d640 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fbb197c2b0 0x55fbb197e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.038+0000 7f8656d9d640 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fbb197c2b0 0x55fbb197e6a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1701f784a83af4b3 server_cookie=40bbcbee4fa505cb in_seq=0 out_seq=0 2024-09-11T01:16:50.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.038+0000 7f863aff5640 10 client.27718 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:16:50.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.040+0000 7f863aff5640 1 -- 172.21.15.78:0/2433499198 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fbb18d8da0 con 0x55fbb197c2b0 2024-09-11T01:16:50.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.040+0000 7f863aff5640 10 client.27718 handle_command_reply: tid=0 2024-09-11T01:16:50.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.040+0000 7f865f5dc740 1 client.27718 shutdown 2024-09-11T01:16:50.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.040+0000 7f865f5dc740 1 -- 172.21.15.78:0/2433499198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 msgr2=0x55fbb1969750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:50.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.040+0000 7f865f5dc740 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 0x55fbb1969750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f864c0ac680 tx=0x7f864c0ac580 comp rx=0 tx=0).stop 2024-09-11T01:16:50.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.040+0000 7f865f5dc740 1 -- 172.21.15.78:0/2433499198 shutdown_connections 2024-09-11T01:16:50.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.040+0000 7f865f5dc740 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fbb197c2b0 0x55fbb197e6a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.040+0000 7f865f5dc740 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb196c950 0x55fbb1969e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.040+0000 7f865f5dc740 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 0x55fbb1969750 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.040+0000 7f865f5dc740 1 --2- 172.21.15.78:0/2433499198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb196c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.040+0000 7f865f5dc740 1 -- 172.21.15.78:0/2433499198 >> 172.21.15.78:0/2433499198 conn(0x55fbb195ded0 msgr2=0x55fbb1963a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:50.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.041+0000 7f863aff5640 0 client.27718 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:16:50.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.041+0000 7f865f5dc740 1 -- 172.21.15.78:0/2433499198 shutdown_connections 2024-09-11T01:16:50.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.042+0000 7f865f5dc740 1 -- 172.21.15.78:0/2433499198 wait complete. 2024-09-11T01:16:50.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.042+0000 7f865f5dc740 20 client.27718 trim_cache size 0 max 16384 2024-09-11T01:16:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.076+0000 7f865f5dc740 1 Processor -- start 2024-09-11T01:16:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.076+0000 7f865f5dc740 1 -- start start 2024-09-11T01:16:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.076+0000 7f865f5dc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb1968c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.076+0000 7f865f5dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 0x55fbb1966960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865f5dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb196c950 0x55fbb1966ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865f5dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86581a63c0 con 0x55fbb19609c0 2024-09-11T01:16:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865f5dc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f86581adf40 con 0x55fbb19600a0 2024-09-11T01:16:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865f5dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86581a6a90 con 0x55fbb196c950 2024-09-11T01:16:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 0x55fbb1966960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 0x55fbb1966960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40218/0 (socket says 172.21.15.78:40218) 2024-09-11T01:16:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865659c640 1 -- 172.21.15.78:0/1167637698 learned_addr learned my addr 172.21.15.78:0/1167637698 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865759e640 1 --2- 172.21.15.78:0/1167637698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb196c950 0x55fbb1966ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f8656d9d640 1 --2- 172.21.15.78:0/1167637698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb1968c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865659c640 1 -- 172.21.15.78:0/1167637698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb196c950 msgr2=0x55fbb1966ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865659c640 1 --2- 172.21.15.78:0/1167637698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb196c950 0x55fbb1966ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865659c640 1 -- 172.21.15.78:0/1167637698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 msgr2=0x55fbb1968c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865659c640 1 --2- 172.21.15.78:0/1167637698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb1968c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.077+0000 7f865659c640 1 -- 172.21.15.78:0/1167637698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb1959790 con 0x55fbb19609c0 2024-09-11T01:16:50.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.078+0000 7f865659c640 1 --2- 172.21.15.78:0/1167637698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 0x55fbb1966960 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f864c0097b0 tx=0x7f864c0ac5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5079ba440d81fccd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:50.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.078+0000 7f863cff9640 1 -- 172.21.15.78:0/1167637698 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c0abe60 con 0x55fbb19609c0 2024-09-11T01:16:50.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.078+0000 7f863cff9640 1 -- 172.21.15.78:0/1167637698 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f864c0d3020 con 0x55fbb19609c0 2024-09-11T01:16:50.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.078+0000 7f863cff9640 1 -- 172.21.15.78:0/1167637698 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c0a82a0 con 0x55fbb19609c0 2024-09-11T01:16:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.079+0000 7f865f5dc740 1 -- 172.21.15.78:0/1167637698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 msgr2=0x55fbb1966960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.079+0000 7f865f5dc740 1 --2- 172.21.15.78:0/1167637698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 0x55fbb1966960 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f864c0097b0 tx=0x7f864c0ac5f0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.079+0000 7f865f5dc740 1 -- 172.21.15.78:0/1167637698 shutdown_connections 2024-09-11T01:16:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.079+0000 7f865f5dc740 1 --2- 172.21.15.78:0/1167637698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb196c950 0x55fbb1966ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.079+0000 7f865f5dc740 1 --2- 172.21.15.78:0/1167637698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19609c0 0x55fbb1966960 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.079+0000 7f865f5dc740 1 --2- 172.21.15.78:0/1167637698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb19600a0 0x55fbb1968c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.079+0000 7f865f5dc740 1 -- 172.21.15.78:0/1167637698 >> 172.21.15.78:0/1167637698 conn(0x55fbb195ded0 msgr2=0x55fbb196b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.079+0000 7f865f5dc740 1 -- 172.21.15.78:0/1167637698 shutdown_connections 2024-09-11T01:16:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.079+0000 7f865f5dc740 1 -- 172.21.15.78:0/1167637698 wait complete. 2024-09-11T01:16:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.080+0000 7f865f5dc740 1 -- start start 2024-09-11T01:16:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.080+0000 7f865f5dc740 1 Processor -- start 2024-09-11T01:16:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.081+0000 7f865f5dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19600a0 0x55fbb19748a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.081+0000 7f865f5dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 0x55fbb1974de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.081+0000 7f865f5dc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb196c950 0x55fbb196f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.081+0000 7f865f5dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86581ad870 con 0x55fbb19600a0 2024-09-11T01:16:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.081+0000 7f865f5dc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f86581ad1a0 con 0x55fbb196c950 2024-09-11T01:16:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.081+0000 7f865f5dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86581a43c0 con 0x55fbb19609c0 2024-09-11T01:16:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.081+0000 7f865f5dc740 10 client.0 _ll_register_callbacks cb 0x7f864c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:16:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.081+0000 7f865759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb196c950 0x55fbb196f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.081+0000 7f865659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 0x55fbb1974de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.081+0000 7f8656d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19600a0 0x55fbb19748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.081+0000 7f8656d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19600a0 0x55fbb19748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40222/0 (socket says 172.21.15.78:40222) 2024-09-11T01:16:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.081+0000 7f8656d9d640 1 -- 172.21.15.78:0/1953200572 learned_addr learned my addr 172.21.15.78:0/1953200572 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.082+0000 7f865659c640 1 -- 172.21.15.78:0/1953200572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb196c950 msgr2=0x55fbb196f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.082+0000 7f865659c640 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb196c950 0x55fbb196f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.082+0000 7f865659c640 1 -- 172.21.15.78:0/1953200572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19600a0 msgr2=0x55fbb19748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.082+0000 7f865659c640 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19600a0 0x55fbb19748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.082+0000 7f865659c640 1 -- 172.21.15.78:0/1953200572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb196fd60 con 0x55fbb19609c0 2024-09-11T01:16:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.082+0000 7f865759e640 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb196c950 0x55fbb196f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:16:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.082+0000 7f8656d9d640 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19600a0 0x55fbb19748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:16:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.082+0000 7f865659c640 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 0x55fbb1974de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55fbb1958f40 tx=0x7f864c0ac580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fc8212238cfb4bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.082+0000 7f863aff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:16:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.085+0000 7f863aff5640 1 -- 172.21.15.78:0/1953200572 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c0abe60 con 0x55fbb19609c0 2024-09-11T01:16:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.085+0000 7f863aff5640 1 -- 172.21.15.78:0/1953200572 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7f864c018d30 con 0x55fbb19609c0 2024-09-11T01:16:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.085+0000 7f865f5dc740 1 -- 172.21.15.78:0/1953200572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f864c0d3020 con 0x55fbb19609c0 2024-09-11T01:16:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.085+0000 7f863aff5640 1 -- 172.21.15.78:0/1953200572 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f864c0cd050 con 0x55fbb19609c0 2024-09-11T01:16:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.086+0000 7f865f5dc740 10 client.27724 fetch_fsmap learned FSMap version 70 2024-09-11T01:16:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.086+0000 7f865f5dc740 1 -- 172.21.15.78:0/1953200572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55fbb196fd60 con 0x55fbb19609c0 2024-09-11T01:16:50.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.087+0000 7f863aff5640 1 -- 172.21.15.78:0/1953200572 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f864c0ac6f0 con 0x55fbb19609c0 2024-09-11T01:16:50.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.087+0000 7f863aff5640 1 -- 172.21.15.78:0/1953200572 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f864c0b22b0 con 0x55fbb19609c0 2024-09-11T01:16:50.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.088+0000 7f865f5dc740 10 client.27724 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:16:50.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.088+0000 7f865f5dc740 10 client.27724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:16:50.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.088+0000 7f865f5dc740 20 client.27724 populate_metadata read hostname 'smithi078' 2024-09-11T01:16:50.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.088+0000 7f865f5dc740 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fbb19e9e50 0x55fbb19ec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:50.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.088+0000 7f865f5dc740 4 client.27724 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:16:50.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.088+0000 7f865f5dc740 1 -- 172.21.15.78:0/1953200572 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f864c0d3020 con 0x55fbb19e9e50 2024-09-11T01:16:50.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.089+0000 7f8656d9d640 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fbb19e9e50 0x55fbb19ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.089+0000 7f8656d9d640 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fbb19e9e50 0x55fbb19ec270 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f547d1ad2711f0bd server_cookie=5aedb4b9ec34dce in_seq=0 out_seq=0 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.089+0000 7f863aff5640 10 client.27724 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.090+0000 7f863aff5640 1 -- 172.21.15.78:0/1953200572 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6338 (crc 0 0 0) 0x7f864c0d3020 con 0x55fbb19e9e50 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.090+0000 7f863aff5640 10 client.27724 handle_command_reply: tid=0 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.090+0000 7f865f5dc740 1 client.27724 shutdown 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.090+0000 7f865f5dc740 1 -- 172.21.15.78:0/1953200572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 msgr2=0x55fbb1974de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.090+0000 7f865f5dc740 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 0x55fbb1974de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55fbb1958f40 tx=0x7f864c0ac580 comp rx=0 tx=0).stop 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.091+0000 7f865f5dc740 1 -- 172.21.15.78:0/1953200572 shutdown_connections 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.091+0000 7f865f5dc740 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbb196c950 0x55fbb196f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.091+0000 7f865f5dc740 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbb19609c0 0x55fbb1974de0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.091+0000 7f865f5dc740 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fbb19e9e50 0x55fbb19ec270 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.091+0000 7f865f5dc740 1 --2- 172.21.15.78:0/1953200572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbb19600a0 0x55fbb19748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.091+0000 7f865f5dc740 1 -- 172.21.15.78:0/1953200572 >> 172.21.15.78:0/1953200572 conn(0x55fbb195ded0 msgr2=0x55fbb196ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:50.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.091+0000 7f863aff5640 0 client.27724 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:16:50.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.091+0000 7f865f5dc740 1 -- 172.21.15.78:0/1953200572 shutdown_connections 2024-09-11T01:16:50.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.091+0000 7f865f5dc740 1 -- 172.21.15.78:0/1953200572 wait complete. 2024-09-11T01:16:50.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.091+0000 7f865f5dc740 20 client.27724 trim_cache size 0 max 16384 2024-09-11T01:16:50.095 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:16:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:16:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:16:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:16:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:16:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:16:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:16:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:16:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:16:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:16:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:16:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:16:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:16:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:16:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:16:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:16:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:16:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:16:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:16:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:16:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:16:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:16:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:16:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:16:50.110 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:16:50.110 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:16:50.110 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:16:50.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.093+0000 7f865d207640 1 -- 172.21.15.78:0/2707066394 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f86400b54d0 msgr2=0x7f86400b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:50.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.093+0000 7f865d207640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f86400b54d0 0x7f86400b7990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55fbb195edb0 tx=0x7f8648005f90 comp rx=0 tx=0).stop 2024-09-11T01:16:50.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.093+0000 7f865d207640 1 -- 172.21.15.78:0/2707066394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86581076a0 msgr2=0x7f8658198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:50.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.093+0000 7f865d207640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86581076a0 0x7f8658198450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f864c00e9f0 tx=0x7f864c00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.093+0000 7f865d207640 1 -- 172.21.15.78:0/2707066394 shutdown_connections 2024-09-11T01:16:50.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.093+0000 7f865d207640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f86400b54d0 0x7f86400b7990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.093+0000 7f865d207640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8658108020 0x7f865819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.093+0000 7f865d207640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86581076a0 0x7f8658198450 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.093+0000 7f865d207640 1 --2- 172.21.15.78:0/2707066394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8658101c90 0x7f8658197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:50.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.093+0000 7f865d207640 1 -- 172.21.15.78:0/2707066394 >> 172.21.15.78:0/2707066394 conn(0x7f86580fd700 msgr2=0x7f86580fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:50.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.094+0000 7f865d207640 1 -- 172.21.15.78:0/2707066394 shutdown_connections 2024-09-11T01:16:50.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:50.094+0000 7f865d207640 1 -- 172.21.15.78:0/2707066394 wait complete. 2024-09-11T01:16:50.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 195946 35.45 MB/sec execute 176 sec latency 80.999 ms 2024-09-11T01:16:51.111 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:16:51.111 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:16:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:16:51.680 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:51 smithi078 ceph-mon[31315]: pgmap v884: 97 pgs: 97 active+clean; 2.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 874 op/s 2024-09-11T01:16:51.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:51 smithi110 ceph-mon[34712]: pgmap v884: 97 pgs: 97 active+clean; 2.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 874 op/s 2024-09-11T01:16:51.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:51 smithi112 ceph-mon[35384]: pgmap v884: 97 pgs: 97 active+clean; 2.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 874 op/s 2024-09-11T01:16:51.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 196665 35.49 MB/sec execute 177 sec latency 74.806 ms 2024-09-11T01:16:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.112+0000 7f1531572640 1 -- 172.21.15.78:0/236529849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f152c1076e0 msgr2=0x7f152c107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.112+0000 7f1531572640 1 --2- 172.21.15.78:0/236529849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f152c1076e0 0x7f152c107b20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1520009860 tx=0x7f152002a340 comp rx=0 tx=0).stop 2024-09-11T01:16:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.113+0000 7f1531572640 1 -- 172.21.15.78:0/236529849 shutdown_connections 2024-09-11T01:16:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.113+0000 7f1531572640 1 --2- 172.21.15.78:0/236529849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f152c108060 0x7f152c10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.113+0000 7f1531572640 1 --2- 172.21.15.78:0/236529849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f152c1076e0 0x7f152c107b20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.113+0000 7f1531572640 1 --2- 172.21.15.78:0/236529849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f152c101cd0 0x7f152c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.113+0000 7f1531572640 1 -- 172.21.15.78:0/236529849 >> 172.21.15.78:0/236529849 conn(0x7f152c0fd740 msgr2=0x7f152c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.113+0000 7f1531572640 1 -- 172.21.15.78:0/236529849 shutdown_connections 2024-09-11T01:16:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.113+0000 7f1531572640 1 -- 172.21.15.78:0/236529849 wait complete. 2024-09-11T01:16:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.113+0000 7f1531572640 1 Processor -- start 2024-09-11T01:16:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.113+0000 7f1531572640 1 -- start start 2024-09-11T01:16:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f1531572640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f152c101cd0 0x7f152c108850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f1531572640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f152c1076e0 0x7f152c108d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f1531572640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f152c108060 0x7f152c1092d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f1531572640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f152c111d20 con 0x7f152c108060 2024-09-11T01:16:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f1531572640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f152c111ba0 con 0x7f152c1076e0 2024-09-11T01:16:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f1531572640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f152c111ea0 con 0x7f152c101cd0 2024-09-11T01:16:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f152a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f152c1076e0 0x7f152c108d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f152a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f152c1076e0 0x7f152c108d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:45752/0 (socket says 172.21.15.78:45752) 2024-09-11T01:16:52.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f152affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f152c101cd0 0x7f152c108850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:52.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f152a7fc640 1 -- 172.21.15.78:0/539398327 learned_addr learned my addr 172.21.15.78:0/539398327 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:52.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f152b7fe640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f152c108060 0x7f152c1092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:52.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f152a7fc640 1 -- 172.21.15.78:0/539398327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f152c101cd0 msgr2=0x7f152c108850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:52.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f152a7fc640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f152c101cd0 0x7f152c108850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:52.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f152a7fc640 1 -- 172.21.15.78:0/539398327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f152c108060 msgr2=0x7f152c1092d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:52.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f152a7fc640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f152c108060 0x7f152c1092d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:52.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f152a7fc640 1 -- 172.21.15.78:0/539398327 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152c1abaf0 con 0x7f152c1076e0 2024-09-11T01:16:52.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f152affd640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f152c101cd0 0x7f152c108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:16:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.114+0000 7f152b7fe640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f152c108060 0x7f152c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:16:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.115+0000 7f152a7fc640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f152c1076e0 0x7f152c108d90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15200303f0 tx=0x7f1520032eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.115+0000 7f1513fff640 1 -- 172.21.15.78:0/539398327 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520040020 con 0x7f152c1076e0 2024-09-11T01:16:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.115+0000 7f1531572640 1 -- 172.21.15.78:0/539398327 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152c1abd80 con 0x7f152c1076e0 2024-09-11T01:16:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.115+0000 7f1513fff640 1 -- 172.21.15.78:0/539398327 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1520030450 con 0x7f152c1076e0 2024-09-11T01:16:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.115+0000 7f1513fff640 1 -- 172.21.15.78:0/539398327 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520037040 con 0x7f152c1076e0 2024-09-11T01:16:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.116+0000 7f1531572640 1 -- 172.21.15.78:0/539398327 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152c1ac250 con 0x7f152c1076e0 2024-09-11T01:16:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.117+0000 7f1531572640 1 -- 172.21.15.78:0/539398327 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f152c1020b0 con 0x7f152c1076e0 2024-09-11T01:16:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.118+0000 7f1513fff640 1 -- 172.21.15.78:0/539398327 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1520030600 con 0x7f152c1076e0 2024-09-11T01:16:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.119+0000 7f1513fff640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15140b54d0 0x7f15140b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.119+0000 7f1513fff640 1 -- 172.21.15.78:0/539398327 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7f1520106250 con 0x7f152c1076e0 2024-09-11T01:16:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.119+0000 7f152affd640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15140b54d0 0x7f15140b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.120+0000 7f152affd640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15140b54d0 0x7f15140b7990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f151c005fd0 tx=0x7f151c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:52.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.126+0000 7f1513fff640 1 -- 172.21.15.78:0/539398327 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f152004a020 con 0x7f152c1076e0 2024-09-11T01:16:52.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.476+0000 7f1531572640 1 -- 172.21.15.78:0/539398327 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f152c1ac710 con 0x7f152c1076e0 2024-09-11T01:16:52.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.477+0000 7f1513fff640 1 -- 172.21.15.78:0/539398327 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+11979 (secure 0 0 0) 0x7f15200ccdb0 con 0x7f152c1076e0 2024-09-11T01:16:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:16:52.480 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":70,"btime":"2024-09-11T01:15:50:008465+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":70,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:15:49.712348+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:16:52.480 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 70 2024-09-11T01:16:52.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.480+0000 7f1531572640 1 -- 172.21.15.78:0/539398327 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15140b54d0 msgr2=0x7f15140b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:52.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.480+0000 7f1531572640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15140b54d0 0x7f15140b7990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f151c005fd0 tx=0x7f151c009260 comp rx=0 tx=0).stop 2024-09-11T01:16:52.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.480+0000 7f1531572640 1 -- 172.21.15.78:0/539398327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f152c1076e0 msgr2=0x7f152c108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:52.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.480+0000 7f1531572640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f152c1076e0 0x7f152c108d90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15200303f0 tx=0x7f1520032eb0 comp rx=0 tx=0).stop 2024-09-11T01:16:52.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.480+0000 7f1531572640 1 -- 172.21.15.78:0/539398327 shutdown_connections 2024-09-11T01:16:52.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.480+0000 7f1531572640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15140b54d0 0x7f15140b7990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:52.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.480+0000 7f1531572640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f152c108060 0x7f152c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:52.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.480+0000 7f1531572640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f152c1076e0 0x7f152c108d90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:52.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.480+0000 7f1531572640 1 --2- 172.21.15.78:0/539398327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f152c101cd0 0x7f152c108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:52.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.480+0000 7f1531572640 1 -- 172.21.15.78:0/539398327 >> 172.21.15.78:0/539398327 conn(0x7f152c0fd740 msgr2=0x7f152c06bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:52.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.480+0000 7f1531572640 1 -- 172.21.15.78:0/539398327 shutdown_connections 2024-09-11T01:16:52.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:52.480+0000 7f1531572640 1 -- 172.21.15.78:0/539398327 wait complete. 2024-09-11T01:16:52.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 197360 35.51 MB/sec execute 178 sec latency 74.494 ms 2024-09-11T01:16:53.229 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:16:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:53 smithi078 ceph-mon[31315]: pgmap v885: 97 pgs: 97 active+clean; 2.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 761 op/s 2024-09-11T01:16:53.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:53 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/539398327' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:16:53.615 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:16:53.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:53 smithi110 ceph-mon[34712]: pgmap v885: 97 pgs: 97 active+clean; 2.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 761 op/s 2024-09-11T01:16:53.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:53 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/539398327' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:16:53.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:53 smithi112 ceph-mon[35384]: pgmap v885: 97 pgs: 97 active+clean; 2.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 761 op/s 2024-09-11T01:16:53.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:53 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/539398327' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:16:53.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 198078 35.49 MB/sec execute 179 sec latency 68.880 ms 2024-09-11T01:16:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.282+0000 7f7c78d41640 1 -- 172.21.15.78:0/1094626531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c74108060 msgr2=0x7f7c7410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.282+0000 7f7c78d41640 1 --2- 172.21.15.78:0/1094626531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c74108060 0x7f7c7410e860 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009ca0 tx=0x7f7c6c02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.282+0000 7f7c78d41640 1 -- 172.21.15.78:0/1094626531 shutdown_connections 2024-09-11T01:16:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.282+0000 7f7c78d41640 1 --2- 172.21.15.78:0/1094626531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c74108060 0x7f7c7410e860 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.282+0000 7f7c78d41640 1 --2- 172.21.15.78:0/1094626531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c741076e0 0x7f7c74107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.282+0000 7f7c78d41640 1 --2- 172.21.15.78:0/1094626531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c74101cd0 0x7f7c741020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.282+0000 7f7c78d41640 1 -- 172.21.15.78:0/1094626531 >> 172.21.15.78:0/1094626531 conn(0x7f7c740fd740 msgr2=0x7f7c740ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.282+0000 7f7c78d41640 1 -- 172.21.15.78:0/1094626531 shutdown_connections 2024-09-11T01:16:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.282+0000 7f7c78d41640 1 -- 172.21.15.78:0/1094626531 wait complete. 2024-09-11T01:16:54.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.283+0000 7f7c78d41640 1 Processor -- start 2024-09-11T01:16:54.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.283+0000 7f7c78d41640 1 -- start start 2024-09-11T01:16:54.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.283+0000 7f7c78d41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c74101cd0 0x7f7c74199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.283+0000 7f7c78d41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c741076e0 0x7f7c741997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.283+0000 7f7c78d41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c74108060 0x7f7c7419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c78d41640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c74112e10 con 0x7f7c74101cd0 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c78d41640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7c74112c90 con 0x7f7c741076e0 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c78d41640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c74112f90 con 0x7f7c74108060 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c74101cd0 0x7f7c74199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c72d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c74108060 0x7f7c7419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c74101cd0 0x7f7c74199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35438/0 (socket says 172.21.15.78:35438) 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c72d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c74108060 0x7f7c7419eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:55322/0 (socket says 172.21.15.78:55322) 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c7259c640 1 -- 172.21.15.78:0/2695306788 learned_addr learned my addr 172.21.15.78:0/2695306788 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c72d9d640 1 -- 172.21.15.78:0/2695306788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c741076e0 msgr2=0x7f7c741997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c72d9d640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c741076e0 0x7f7c741997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c72d9d640 1 -- 172.21.15.78:0/2695306788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c74101cd0 msgr2=0x7f7c74199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c72d9d640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c74101cd0 0x7f7c74199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c72d9d640 1 -- 172.21.15.78:0/2695306788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7419f310 con 0x7f7c74108060 2024-09-11T01:16:54.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.284+0000 7f7c7259c640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c74101cd0 0x7f7c74199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:16:54.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.285+0000 7f7c72d9d640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c74108060 0x7f7c7419eb40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c02ade0 tx=0x7f7c6c0316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:54.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.285+0000 7f7c637fe640 1 -- 172.21.15.78:0/2695306788 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c003b70 con 0x7f7c74108060 2024-09-11T01:16:54.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.285+0000 7f7c637fe640 1 -- 172.21.15.78:0/2695306788 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c6c003d10 con 0x7f7c74108060 2024-09-11T01:16:54.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.285+0000 7f7c78d41640 1 -- 172.21.15.78:0/2695306788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7419f5a0 con 0x7f7c74108060 2024-09-11T01:16:54.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.285+0000 7f7c78d41640 1 -- 172.21.15.78:0/2695306788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c741a5cc0 con 0x7f7c74108060 2024-09-11T01:16:54.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.285+0000 7f7c637fe640 1 -- 172.21.15.78:0/2695306788 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c02f9e0 con 0x7f7c74108060 2024-09-11T01:16:54.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.288+0000 7f7c637fe640 1 -- 172.21.15.78:0/2695306788 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7c6c02fb80 con 0x7f7c74108060 2024-09-11T01:16:54.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.289+0000 7f7c637fe640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7c5c0b54d0 0x7f7c5c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.289+0000 7f7c637fe640 1 -- 172.21.15.78:0/2695306788 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7f7c6c106fa0 con 0x7f7c74108060 2024-09-11T01:16:54.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.289+0000 7f7c7b116740 1 Processor -- start 2024-09-11T01:16:54.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.289+0000 7f7c7259c640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7c5c0b54d0 0x7f7c5c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.290+0000 7f7c7b116740 1 -- start start 2024-09-11T01:16:54.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.290+0000 7f7c7b116740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.290+0000 7f7c7b116740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f4012a10 0x5584f401e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.290+0000 7f7c7b116740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f401e9a0 0x5584f4020d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.290+0000 7f7c7b116740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584f3f88ea0 con 0x5584f401e9a0 2024-09-11T01:16:54.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.290+0000 7f7c7b116740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5584f3f782d0 con 0x5584f40120f0 2024-09-11T01:16:54.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.290+0000 7f7c7b116740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5584f3f7c0e0 con 0x5584f4012a10 2024-09-11T01:16:54.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.290+0000 7f7c7259c640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7c5c0b54d0 0x7f7c5c0b7990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7c640095f0 tx=0x7f7c64005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:54.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.290+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.290+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40124d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:44906/0 (socket says 172.21.15.78:44906) 2024-09-11T01:16:54.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.290+0000 7f7c7259c640 1 -- 172.21.15.78:0/2291063888 learned_addr learned my addr 172.21.15.78:0/2291063888 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:54.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.290+0000 7f7c72d9d640 1 --2- 172.21.15.78:0/2291063888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f401e9a0 0x5584f4020d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.291+0000 7f7c7259c640 1 -- 172.21.15.78:0/2291063888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f4012a10 msgr2=0x5584f401e460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:16:54.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.291+0000 7f7c7259c640 1 --2- 172.21.15.78:0/2291063888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f4012a10 0x5584f401e460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.291+0000 7f7c7259c640 1 -- 172.21.15.78:0/2291063888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f401e9a0 msgr2=0x5584f4020d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:54.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.291+0000 7f7c7259c640 1 --2- 172.21.15.78:0/2291063888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f401e9a0 0x5584f4020d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.291+0000 7f7c7259c640 1 -- 172.21.15.78:0/2291063888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f3cd6650 con 0x5584f40120f0 2024-09-11T01:16:54.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.291+0000 7f7c7259c640 1 --2- 172.21.15.78:0/2291063888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40124d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400f750 tx=0x7f7c64014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf10b866c969a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:54.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.291+0000 7f7c60ff9640 1 -- 172.21.15.78:0/2291063888 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64026640 con 0x5584f40120f0 2024-09-11T01:16:54.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.291+0000 7f7c60ff9640 1 -- 172.21.15.78:0/2291063888 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c6401c8c0 con 0x5584f40120f0 2024-09-11T01:16:54.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.291+0000 7f7c60ff9640 1 -- 172.21.15.78:0/2291063888 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6401b8d0 con 0x5584f40120f0 2024-09-11T01:16:54.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.292+0000 7f7c7b116740 1 -- 172.21.15.78:0/2291063888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 msgr2=0x5584f40124d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:54.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.292+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2291063888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40124d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400f750 tx=0x7f7c64014000 comp rx=0 tx=0).stop 2024-09-11T01:16:54.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.292+0000 7f7c7b116740 1 -- 172.21.15.78:0/2291063888 shutdown_connections 2024-09-11T01:16:54.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.292+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2291063888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f401e9a0 0x5584f4020d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.292+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2291063888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f4012a10 0x5584f401e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.292+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2291063888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40124d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.292+0000 7f7c7b116740 1 -- 172.21.15.78:0/2291063888 >> 172.21.15.78:0/2291063888 conn(0x5584f400ff20 msgr2=0x5584f40110c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:54.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.293+0000 7f7c7b116740 1 -- 172.21.15.78:0/2291063888 shutdown_connections 2024-09-11T01:16:54.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.293+0000 7f7c7b116740 1 -- 172.21.15.78:0/2291063888 wait complete. 2024-09-11T01:16:54.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.293+0000 7f7c7b116740 1 -- start start 2024-09-11T01:16:54.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.293+0000 7f7c7b116740 1 Processor -- start 2024-09-11T01:16:54.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.293+0000 7f7c7b116740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f40120f0 0x5584f401e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.293+0000 7f7c7b116740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f4012a10 0x5584f401b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.293+0000 7f7c7b116740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f401e9a0 0x5584f401bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.293+0000 7f7c7b116740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584f3dd9700 con 0x5584f4012a10 2024-09-11T01:16:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.293+0000 7f7c7b116740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5584f3f13160 con 0x5584f401e9a0 2024-09-11T01:16:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.293+0000 7f7c7b116740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5584f4021620 con 0x5584f40120f0 2024-09-11T01:16:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.294+0000 7f7c7b116740 10 client.0 _ll_register_callbacks cb 0x5584f3f2c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:16:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.294+0000 7f7c71d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f4012a10 0x5584f401b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.294+0000 7f7c71d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f4012a10 0x5584f401b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35448/0 (socket says 172.21.15.78:35448) 2024-09-11T01:16:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.294+0000 7f7c71d9b640 1 -- 172.21.15.78:0/4049729292 learned_addr learned my addr 172.21.15.78:0/4049729292 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.294+0000 7f7c7259c640 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f40120f0 0x5584f401e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.294+0000 7f7c72d9d640 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f401e9a0 0x5584f401bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.294+0000 7f7c71d9b640 1 -- 172.21.15.78:0/4049729292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f40120f0 msgr2=0x5584f401e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.294+0000 7f7c71d9b640 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f40120f0 0x5584f401e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.294+0000 7f7c71d9b640 1 -- 172.21.15.78:0/4049729292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f401e9a0 msgr2=0x5584f401bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.294+0000 7f7c71d9b640 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f401e9a0 0x5584f401bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.294+0000 7f7c71d9b640 1 -- 172.21.15.78:0/4049729292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f3f40c90 con 0x5584f4012a10 2024-09-11T01:16:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.295+0000 7f7c7259c640 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f40120f0 0x5584f401e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:16:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.295+0000 7f7c71d9b640 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f4012a10 0x5584f401b6a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800bc10 tx=0x7f7c6800d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b1462ec82a6791 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.295+0000 7f7c59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:16:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.295+0000 7f7c59ffb640 1 -- 172.21.15.78:0/4049729292 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68016070 con 0x5584f4012a10 2024-09-11T01:16:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.295+0000 7f7c59ffb640 1 -- 172.21.15.78:0/4049729292 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7f7c68010660 con 0x5584f4012a10 2024-09-11T01:16:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.295+0000 7f7c7b116740 1 -- 172.21.15.78:0/4049729292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c6401c8c0 con 0x5584f4012a10 2024-09-11T01:16:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.296+0000 7f7c59ffb640 1 -- 172.21.15.78:0/4049729292 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f7c68020050 con 0x5584f4012a10 2024-09-11T01:16:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.296+0000 7f7c7b116740 10 client.18651 fetch_fsmap learned FSMap version 70 2024-09-11T01:16:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.296+0000 7f7c7b116740 1 -- 172.21.15.78:0/4049729292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5584f4029ea0 con 0x5584f4012a10 2024-09-11T01:16:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.297+0000 7f7c59ffb640 1 -- 172.21.15.78:0/4049729292 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c68010d10 con 0x5584f4012a10 2024-09-11T01:16:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.297+0000 7f7c59ffb640 1 -- 172.21.15.78:0/4049729292 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f7c6801b9f0 con 0x5584f4012a10 2024-09-11T01:16:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.297+0000 7f7c7b116740 10 client.18651 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:16:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.297+0000 7f7c7b116740 10 client.18651 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:16:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.297+0000 7f7c7b116740 20 client.18651 populate_metadata read hostname 'smithi078' 2024-09-11T01:16:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.297+0000 7f7c7b116740 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5584f403f500 0x5584f4041920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.297+0000 7f7c7b116740 4 client.18651 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:16:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.297+0000 7f7c7b116740 1 -- 172.21.15.78:0/4049729292 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c6401c8c0 con 0x5584f403f500 2024-09-11T01:16:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.297+0000 7f7c7259c640 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5584f403f500 0x5584f4041920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.298+0000 7f7c59ffb640 10 client.18651 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:16:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.298+0000 7f7c7259c640 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5584f403f500 0x5584f4041920 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78aa5e128f646cdd server_cookie=9ad2dab426e00008 in_seq=0 out_seq=0 2024-09-11T01:16:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.300+0000 7f7c59ffb640 1 -- 172.21.15.78:0/4049729292 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7c6401c8c0 con 0x5584f403f500 2024-09-11T01:16:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.300+0000 7f7c59ffb640 10 client.18651 handle_command_reply: tid=0 2024-09-11T01:16:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c7b116740 1 client.18651 shutdown 2024-09-11T01:16:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c7b116740 1 -- 172.21.15.78:0/4049729292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f4012a10 msgr2=0x5584f401b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c7b116740 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f4012a10 0x5584f401b6a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800bc10 tx=0x7f7c6800d6b0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c7b116740 1 -- 172.21.15.78:0/4049729292 shutdown_connections 2024-09-11T01:16:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c7b116740 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5584f403f500 0x5584f4041920 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c59ffb640 0 client.18651 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:16:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c7b116740 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f401e9a0 0x5584f401bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c7b116740 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f4012a10 0x5584f401b6a0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c7b116740 1 --2- 172.21.15.78:0/4049729292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f40120f0 0x5584f401e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c7b116740 1 -- 172.21.15.78:0/4049729292 >> 172.21.15.78:0/4049729292 conn(0x5584f400ff20 msgr2=0x5584f4015a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c7b116740 1 -- 172.21.15.78:0/4049729292 shutdown_connections 2024-09-11T01:16:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c7b116740 1 -- 172.21.15.78:0/4049729292 wait complete. 2024-09-11T01:16:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.301+0000 7f7c7b116740 20 client.18651 trim_cache size 0 max 16384 2024-09-11T01:16:54.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.318+0000 7f7c7b116740 1 Processor -- start 2024-09-11T01:16:54.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.319+0000 7f7c7b116740 1 -- start start 2024-09-11T01:16:54.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.319+0000 7f7c7b116740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40116f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.319+0000 7f7c7b116740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f401e9a0 0x5584f401fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.319+0000 7f7c7b116740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f403f500 0x5584f40202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.319+0000 7f7c7b116740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5400e770 con 0x5584f403f500 2024-09-11T01:16:54.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.319+0000 7f7c7b116740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7c540102b0 con 0x5584f40120f0 2024-09-11T01:16:54.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.319+0000 7f7c7b116740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5400ee40 con 0x5584f401e9a0 2024-09-11T01:16:54.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.319+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.319+0000 7f7c72d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f403f500 0x5584f40202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.319+0000 7f7c72d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f403f500 0x5584f40202d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35464/0 (socket says 172.21.15.78:35464) 2024-09-11T01:16:54.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.319+0000 7f7c72d9d640 1 -- 172.21.15.78:0/2470130117 learned_addr learned my addr 172.21.15.78:0/2470130117 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:54.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.320+0000 7f7c71d9b640 1 --2- 172.21.15.78:0/2470130117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f401e9a0 0x5584f401fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.320+0000 7f7c72d9d640 1 -- 172.21.15.78:0/2470130117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f401e9a0 msgr2=0x5584f401fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.320+0000 7f7c72d9d640 1 --2- 172.21.15.78:0/2470130117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f401e9a0 0x5584f401fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.320+0000 7f7c72d9d640 1 -- 172.21.15.78:0/2470130117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 msgr2=0x5584f40116f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.320+0000 7f7c72d9d640 1 --2- 172.21.15.78:0/2470130117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40116f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.320+0000 7f7c72d9d640 1 -- 172.21.15.78:0/2470130117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f401bfd0 con 0x5584f403f500 2024-09-11T01:16:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.320+0000 7f7c72d9d640 1 --2- 172.21.15.78:0/2470130117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f403f500 0x5584f40202d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c1047e0 tx=0x7f7c6c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab64ebca2fde719 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.321+0000 7f7c5bfff640 1 -- 172.21.15.78:0/2470130117 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c02fea0 con 0x5584f403f500 2024-09-11T01:16:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.321+0000 7f7c5bfff640 1 -- 172.21.15.78:0/2470130117 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c6c036040 con 0x5584f403f500 2024-09-11T01:16:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.321+0000 7f7c5bfff640 1 -- 172.21.15.78:0/2470130117 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c0cd920 con 0x5584f403f500 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.322+0000 7f7c7b116740 1 -- 172.21.15.78:0/2470130117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f403f500 msgr2=0x5584f40202d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.322+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2470130117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f403f500 0x5584f40202d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c1047e0 tx=0x7f7c6c031a20 comp rx=0 tx=0).stop 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.322+0000 7f7c7b116740 1 -- 172.21.15.78:0/2470130117 shutdown_connections 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.322+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2470130117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f403f500 0x5584f40202d0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.322+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2470130117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f401e9a0 0x5584f401fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.322+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2470130117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.322+0000 7f7c7b116740 1 -- 172.21.15.78:0/2470130117 >> 172.21.15.78:0/2470130117 conn(0x5584f400ff20 msgr2=0x5584f4010fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.322+0000 7f7c7b116740 1 -- 172.21.15.78:0/2470130117 shutdown_connections 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.322+0000 7f7c7b116740 1 -- 172.21.15.78:0/2470130117 wait complete. 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.323+0000 7f7c7b116740 1 -- start start 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.323+0000 7f7c7b116740 1 Processor -- start 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.323+0000 7f7c7b116740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40284b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.323+0000 7f7c7b116740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f401e9a0 0x5584f4028a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.323+0000 7f7c7b116740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f403f500 0x5584f4022c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c7b116740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5400fbe0 con 0x5584f401e9a0 2024-09-11T01:16:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c7b116740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5400f510 con 0x5584f40120f0 2024-09-11T01:16:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c7b116740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5400c770 con 0x5584f403f500 2024-09-11T01:16:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40284b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c7b116740 10 client.0 _ll_register_callbacks cb 0x7f7c6800b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:16:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40284b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:44928/0 (socket says 172.21.15.78:44928) 2024-09-11T01:16:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c7259c640 1 -- 172.21.15.78:0/2879701725 learned_addr learned my addr 172.21.15.78:0/2879701725 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c72d9d640 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f403f500 0x5584f4022c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c71d9b640 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f401e9a0 0x5584f4028a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c72d9d640 1 -- 172.21.15.78:0/2879701725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 msgr2=0x5584f40284b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c72d9d640 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40284b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c72d9d640 1 -- 172.21.15.78:0/2879701725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f401e9a0 msgr2=0x5584f4028a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c72d9d640 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f401e9a0 0x5584f4028a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c72d9d640 1 -- 172.21.15.78:0/2879701725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f4023440 con 0x5584f403f500 2024-09-11T01:16:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.324+0000 7f7c72d9d640 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f403f500 0x5584f4022c10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5584f401c260 tx=0x7f7c6c0c9900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b9a56def538506 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.325+0000 7f7c59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:16:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.325+0000 7f7c59ffb640 1 -- 172.21.15.78:0/2879701725 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c0e6070 con 0x5584f403f500 2024-09-11T01:16:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.325+0000 7f7c7b116740 1 -- 172.21.15.78:0/2879701725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c6c036040 con 0x5584f403f500 2024-09-11T01:16:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.325+0000 7f7c59ffb640 1 -- 172.21.15.78:0/2879701725 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7f7c6c0d6560 con 0x5584f403f500 2024-09-11T01:16:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.325+0000 7f7c59ffb640 1 -- 172.21.15.78:0/2879701725 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f7c6c02fea0 con 0x5584f403f500 2024-09-11T01:16:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.325+0000 7f7c7b116740 10 client.27742 fetch_fsmap learned FSMap version 70 2024-09-11T01:16:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.325+0000 7f7c7b116740 1 -- 172.21.15.78:0/2879701725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5584f40ad160 con 0x5584f403f500 2024-09-11T01:16:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.325+0000 7f7c59ffb640 1 -- 172.21.15.78:0/2879701725 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c6c036040 con 0x5584f403f500 2024-09-11T01:16:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.326+0000 7f7c59ffb640 1 -- 172.21.15.78:0/2879701725 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f7c6c0c7b60 con 0x5584f403f500 2024-09-11T01:16:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.326+0000 7f7c7b116740 10 client.27742 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:16:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.326+0000 7f7c7b116740 10 client.27742 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:16:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.326+0000 7f7c7b116740 20 client.27742 populate_metadata read hostname 'smithi078' 2024-09-11T01:16:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.326+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5584f40ad6d0 0x5584f40afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.326+0000 7f7c7b116740 4 client.27742 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:16:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.326+0000 7f7c7b116740 1 -- 172.21.15.78:0/2879701725 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584f3f8adf0 con 0x5584f40ad6d0 2024-09-11T01:16:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.326+0000 7f7c7259c640 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5584f40ad6d0 0x5584f40afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.327+0000 7f7c7259c640 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5584f40ad6d0 0x5584f40afaf0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d618d9979a8ad864 server_cookie=f6335481ef18c241 in_seq=0 out_seq=0 2024-09-11T01:16:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.327+0000 7f7c59ffb640 10 client.27742 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:16:54.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c59ffb640 1 -- 172.21.15.78:0/2879701725 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584f3f8adf0 con 0x5584f40ad6d0 2024-09-11T01:16:54.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c59ffb640 10 client.27742 handle_command_reply: tid=0 2024-09-11T01:16:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c7b116740 1 client.27742 shutdown 2024-09-11T01:16:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c7b116740 1 -- 172.21.15.78:0/2879701725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f403f500 msgr2=0x5584f4022c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f403f500 0x5584f4022c10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5584f401c260 tx=0x7f7c6c0c9900 comp rx=0 tx=0).stop 2024-09-11T01:16:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c7b116740 1 -- 172.21.15.78:0/2879701725 shutdown_connections 2024-09-11T01:16:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5584f40ad6d0 0x5584f40afaf0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584f403f500 0x5584f4022c10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584f401e9a0 0x5584f4028a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c7b116740 1 --2- 172.21.15.78:0/2879701725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584f40120f0 0x5584f40284b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c7b116740 1 -- 172.21.15.78:0/2879701725 >> 172.21.15.78:0/2879701725 conn(0x5584f400ff20 msgr2=0x5584f401ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c59ffb640 0 client.27742 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:16:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c7b116740 1 -- 172.21.15.78:0/2879701725 shutdown_connections 2024-09-11T01:16:54.338 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:16:54.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.331+0000 7f7c7b116740 1 -- 172.21.15.78:0/2879701725 wait complete. 2024-09-11T01:16:54.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.332+0000 7f7c7b116740 20 client.27742 trim_cache size 0 max 16384 2024-09-11T01:16:54.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.333+0000 7f7c78d41640 1 -- 172.21.15.78:0/2695306788 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7c5c0b54d0 msgr2=0x7f7c5c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:54.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.333+0000 7f7c78d41640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7c5c0b54d0 0x7f7c5c0b7990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7c640095f0 tx=0x7f7c64005f90 comp rx=0 tx=0).stop 2024-09-11T01:16:54.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.333+0000 7f7c78d41640 1 -- 172.21.15.78:0/2695306788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c74108060 msgr2=0x7f7c7419eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:54.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.333+0000 7f7c78d41640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c74108060 0x7f7c7419eb40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c02ade0 tx=0x7f7c6c0316f0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.333+0000 7f7c78d41640 1 -- 172.21.15.78:0/2695306788 shutdown_connections 2024-09-11T01:16:54.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.333+0000 7f7c78d41640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7c5c0b54d0 0x7f7c5c0b7990 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.333+0000 7f7c78d41640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c74108060 0x7f7c7419eb40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.333+0000 7f7c78d41640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c741076e0 0x7f7c741997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.333+0000 7f7c78d41640 1 --2- 172.21.15.78:0/2695306788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c74101cd0 0x7f7c74199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:54.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.333+0000 7f7c78d41640 1 -- 172.21.15.78:0/2695306788 >> 172.21.15.78:0/2695306788 conn(0x7f7c740fd740 msgr2=0x7f7c740fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:54.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.333+0000 7f7c78d41640 1 -- 172.21.15.78:0/2695306788 shutdown_connections 2024-09-11T01:16:54.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:54.333+0000 7f7c78d41640 1 -- 172.21.15.78:0/2695306788 wait complete. 2024-09-11T01:16:54.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 198939 35.57 MB/sec execute 180 sec latency 82.893 ms 2024-09-11T01:16:55.262 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:16:55.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:55 smithi078 ceph-mon[31315]: pgmap v886: 97 pgs: 97 active+clean; 2.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 722 op/s 2024-09-11T01:16:55.646 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:16:55.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:55 smithi110 ceph-mon[34712]: pgmap v886: 97 pgs: 97 active+clean; 2.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 722 op/s 2024-09-11T01:16:55.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:55 smithi112 ceph-mon[35384]: pgmap v886: 97 pgs: 97 active+clean; 2.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 722 op/s 2024-09-11T01:16:55.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 199702 35.61 MB/sec execute 181 sec latency 53.813 ms 2024-09-11T01:16:56.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.204+0000 7fe97543f640 1 -- 172.21.15.78:0/1559273201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe970104570 msgr2=0x7fe97010cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:56.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.204+0000 7fe97543f640 1 --2- 172.21.15.78:0/1559273201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe970104570 0x7fe97010cb50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe964009860 tx=0x7fe96402a340 comp rx=0 tx=0).stop 2024-09-11T01:16:56.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.205+0000 7fe97543f640 1 -- 172.21.15.78:0/1559273201 shutdown_connections 2024-09-11T01:16:56.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.205+0000 7fe97543f640 1 --2- 172.21.15.78:0/1559273201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe97010d120 0x7fe97010f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.205+0000 7fe97543f640 1 --2- 172.21.15.78:0/1559273201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe970104570 0x7fe97010cb50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.205+0000 7fe97543f640 1 --2- 172.21.15.78:0/1559273201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe970103b70 0x7fe970103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.205+0000 7fe97543f640 1 -- 172.21.15.78:0/1559273201 >> 172.21.15.78:0/1559273201 conn(0x7fe9700fd720 msgr2=0x7fe9700ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.205+0000 7fe97543f640 1 -- 172.21.15.78:0/1559273201 shutdown_connections 2024-09-11T01:16:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.205+0000 7fe97543f640 1 -- 172.21.15.78:0/1559273201 wait complete. 2024-09-11T01:16:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.205+0000 7fe97543f640 1 Processor -- start 2024-09-11T01:16:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.206+0000 7fe97543f640 1 -- start start 2024-09-11T01:16:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.206+0000 7fe97543f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe970103b70 0x7fe97010bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.206+0000 7fe97543f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe970104570 0x7fe97010c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.206+0000 7fe97543f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe97010d120 0x7fe9701a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.206+0000 7fe97543f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe970112da0 con 0x7fe97010d120 2024-09-11T01:16:56.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.206+0000 7fe97543f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe970112c20 con 0x7fe970103b70 2024-09-11T01:16:56.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.206+0000 7fe97543f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe970112f20 con 0x7fe970104570 2024-09-11T01:16:56.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.206+0000 7fe96f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe97010d120 0x7fe9701a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.206+0000 7fe96f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe97010d120 0x7fe9701a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35506/0 (socket says 172.21.15.78:35506) 2024-09-11T01:16:56.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.206+0000 7fe96f7fe640 1 -- 172.21.15.78:0/2405240241 learned_addr learned my addr 172.21.15.78:0/2405240241 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:56.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe96effd640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe970103b70 0x7fe97010bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe970104570 0x7fe97010c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe96f7fe640 1 -- 172.21.15.78:0/2405240241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe970104570 msgr2=0x7fe97010c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe96f7fe640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe970104570 0x7fe97010c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe96f7fe640 1 -- 172.21.15.78:0/2405240241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe970103b70 msgr2=0x7fe97010bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe96f7fe640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe970103b70 0x7fe97010bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe96f7fe640 1 -- 172.21.15.78:0/2405240241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9701a2700 con 0x7fe97010d120 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe970104570 0x7fe97010c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe96f7fe640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe97010d120 0x7fe9701a1f30 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe96800ed80 tx=0x7fe96800c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe957fff640 1 -- 172.21.15.78:0/2405240241 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968003b70 con 0x7fe97010d120 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe97543f640 1 -- 172.21.15.78:0/2405240241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9701a29f0 con 0x7fe97010d120 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe957fff640 1 -- 172.21.15.78:0/2405240241 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe968003d10 con 0x7fe97010d120 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.207+0000 7fe957fff640 1 -- 172.21.15.78:0/2405240241 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9680178f0 con 0x7fe97010d120 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.208+0000 7fe97543f640 1 -- 172.21.15.78:0/2405240241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9701ac2f0 con 0x7fe97010d120 2024-09-11T01:16:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.211+0000 7fe957fff640 1 -- 172.21.15.78:0/2405240241 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe96800b030 con 0x7fe97010d120 2024-09-11T01:16:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.211+0000 7fe977814740 1 Processor -- start 2024-09-11T01:16:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.211+0000 7fe977814740 1 -- start start 2024-09-11T01:16:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe977814740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ebe4d0 0x55d534ebe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe977814740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534eca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe977814740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ecad80 0x55d534ecd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe977814740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d534e34ea0 con 0x55d534ebedf0 2024-09-11T01:16:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe977814740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d534e242d0 con 0x55d534ebe4d0 2024-09-11T01:16:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe977814740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d534e280e0 con 0x55d534ecad80 2024-09-11T01:16:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe957fff640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe9580b5450 0x7fe9580b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe957fff640 1 -- 172.21.15.78:0/2405240241 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7fe9680e5ec0 con 0x7fe97010d120 2024-09-11T01:16:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe96effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534eca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe96f7fe640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe9580b5450 0x7fe9580b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe96effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534eca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35522/0 (socket says 172.21.15.78:35522) 2024-09-11T01:16:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe96effd640 1 -- 172.21.15.78:0/2853056704 learned_addr learned my addr 172.21.15.78:0/2853056704 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/2853056704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ebe4d0 0x55d534ebe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.212+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/2853056704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ecad80 0x55d534ecd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.213+0000 7fe96e7fc640 1 -- 172.21.15.78:0/2853056704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ebe4d0 msgr2=0x55d534ebe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.213+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/2853056704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ebe4d0 0x55d534ebe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.213+0000 7fe96e7fc640 1 -- 172.21.15.78:0/2853056704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 msgr2=0x55d534eca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.213+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/2853056704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534eca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.213+0000 7fe96e7fc640 1 -- 172.21.15.78:0/2853056704 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d534b82650 con 0x55d534ecad80 2024-09-11T01:16:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.214+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/2853056704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ecad80 0x55d534ecd170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe97010f8d0 tx=0x7fe964000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dc80bdd59e71ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.214+0000 7fe96f7fe640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe9580b5450 0x7fe9580b7910 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe968004af0 tx=0x7fe968004a80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.215+0000 7fe9557fa640 1 -- 172.21.15.78:0/2853056704 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964046070 con 0x55d534ecad80 2024-09-11T01:16:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.216+0000 7fe9557fa640 1 -- 172.21.15.78:0/2853056704 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe964038040 con 0x55d534ecad80 2024-09-11T01:16:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.216+0000 7fe9557fa640 1 -- 172.21.15.78:0/2853056704 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964037070 con 0x55d534ecad80 2024-09-11T01:16:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.217+0000 7fe977814740 1 -- 172.21.15.78:0/2853056704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ecad80 msgr2=0x55d534ecd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.217+0000 7fe977814740 1 --2- 172.21.15.78:0/2853056704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ecad80 0x55d534ecd170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe97010f8d0 tx=0x7fe964000bc0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.217+0000 7fe977814740 1 -- 172.21.15.78:0/2853056704 shutdown_connections 2024-09-11T01:16:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.217+0000 7fe977814740 1 --2- 172.21.15.78:0/2853056704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ecad80 0x55d534ecd170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.217+0000 7fe977814740 1 --2- 172.21.15.78:0/2853056704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534eca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.217+0000 7fe977814740 1 --2- 172.21.15.78:0/2853056704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ebe4d0 0x55d534ebe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.217+0000 7fe977814740 1 -- 172.21.15.78:0/2853056704 >> 172.21.15.78:0/2853056704 conn(0x55d534ebc300 msgr2=0x55d534ebd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.217+0000 7fe977814740 1 -- 172.21.15.78:0/2853056704 shutdown_connections 2024-09-11T01:16:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.217+0000 7fe977814740 1 -- 172.21.15.78:0/2853056704 wait complete. 2024-09-11T01:16:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.218+0000 7fe977814740 1 -- start start 2024-09-11T01:16:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.218+0000 7fe977814740 1 Processor -- start 2024-09-11T01:16:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe977814740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 0x55d534eca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe977814740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534ec7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe977814740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ecad80 0x55d534ec8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe977814740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d534c85700 con 0x55d534ebedf0 2024-09-11T01:16:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe977814740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d534dbf160 con 0x55d534ecad80 2024-09-11T01:16:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe977814740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d534ecd9a0 con 0x55d534ebe4d0 2024-09-11T01:16:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe977814740 10 client.0 _ll_register_callbacks cb 0x55d534dd89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:16:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe96e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 0x55d534eca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe96e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 0x55d534eca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:55400/0 (socket says 172.21.15.78:55400) 2024-09-11T01:16:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe96e7fc640 1 -- 172.21.15.78:0/3292065832 learned_addr learned my addr 172.21.15.78:0/3292065832 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ecad80 0x55d534ec8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe96effd640 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534ec7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe96e7fc640 1 -- 172.21.15.78:0/3292065832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ecad80 msgr2=0x55d534ec8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ecad80 0x55d534ec8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe96e7fc640 1 -- 172.21.15.78:0/3292065832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 msgr2=0x55d534ec7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534ec7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.219+0000 7fe96e7fc640 1 -- 172.21.15.78:0/3292065832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d534decc90 con 0x55d534ebe4d0 2024-09-11T01:16:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.220+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 0x55d534eca710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe9640057c0 tx=0x7fe964005d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=605d1058fa45ae2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.220+0000 7fe952ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:16:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.220+0000 7fe952ff5640 1 -- 172.21.15.78:0/3292065832 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964037030 con 0x55d534ebe4d0 2024-09-11T01:16:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.220+0000 7fe977814740 1 -- 172.21.15.78:0/3292065832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe964038040 con 0x55d534ebe4d0 2024-09-11T01:16:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.220+0000 7fe952ff5640 1 -- 172.21.15.78:0/3292065832 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7fe964005460 con 0x55d534ebe4d0 2024-09-11T01:16:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.221+0000 7fe952ff5640 1 -- 172.21.15.78:0/3292065832 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fe964056050 con 0x55d534ebe4d0 2024-09-11T01:16:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.221+0000 7fe977814740 10 client.27751 fetch_fsmap learned FSMap version 70 2024-09-11T01:16:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.221+0000 7fe977814740 1 -- 172.21.15.78:0/3292065832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55d534ed9370 con 0x55d534ebe4d0 2024-09-11T01:16:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.221+0000 7fe952ff5640 1 -- 172.21.15.78:0/3292065832 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe964004b70 con 0x55d534ebe4d0 2024-09-11T01:16:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.221+0000 7fe952ff5640 1 -- 172.21.15.78:0/3292065832 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7fe964049070 con 0x55d534ebe4d0 2024-09-11T01:16:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.221+0000 7fe977814740 10 client.27751 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:16:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.221+0000 7fe977814740 10 client.27751 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:16:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.221+0000 7fe977814740 20 client.27751 populate_metadata read hostname 'smithi078' 2024-09-11T01:16:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.221+0000 7fe977814740 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d534eda6c0 0x55d534edcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.221+0000 7fe977814740 4 client.27751 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:16:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.222+0000 7fe977814740 1 -- 172.21.15.78:0/3292065832 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe964038040 con 0x55d534eda6c0 2024-09-11T01:16:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.222+0000 7fe96effd640 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d534eda6c0 0x55d534edcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.222+0000 7fe952ff5640 10 client.27751 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:16:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.222+0000 7fe96effd640 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d534eda6c0 0x55d534edcab0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ea2310659f9e70c server_cookie=2527c5d017b94789 in_seq=0 out_seq=0 2024-09-11T01:16:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe952ff5640 1 -- 172.21.15.78:0/3292065832 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe964038040 con 0x55d534eda6c0 2024-09-11T01:16:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe952ff5640 10 client.27751 handle_command_reply: tid=0 2024-09-11T01:16:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe977814740 1 client.27751 shutdown 2024-09-11T01:16:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe977814740 1 -- 172.21.15.78:0/3292065832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 msgr2=0x55d534eca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe977814740 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 0x55d534eca710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe9640057c0 tx=0x7fe964005d10 comp rx=0 tx=0).stop 2024-09-11T01:16:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe977814740 1 -- 172.21.15.78:0/3292065832 shutdown_connections 2024-09-11T01:16:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe977814740 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d534eda6c0 0x55d534edcab0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe952ff5640 0 client.27751 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:16:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe977814740 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ecad80 0x55d534ec8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe977814740 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534ec7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe977814740 1 --2- 172.21.15.78:0/3292065832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 0x55d534eca710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe977814740 1 -- 172.21.15.78:0/3292065832 >> 172.21.15.78:0/3292065832 conn(0x55d534ebc300 msgr2=0x55d534ec1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe977814740 1 -- 172.21.15.78:0/3292065832 shutdown_connections 2024-09-11T01:16:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe977814740 1 -- 172.21.15.78:0/3292065832 wait complete. 2024-09-11T01:16:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.226+0000 7fe977814740 20 client.27751 trim_cache size 0 max 16384 2024-09-11T01:16:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.246+0000 7fe977814740 1 Processor -- start 2024-09-11T01:16:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.246+0000 7fe977814740 1 -- start start 2024-09-11T01:16:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.247+0000 7fe977814740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 0x55d534ebda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.247+0000 7fe977814740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534ebbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.247+0000 7fe96effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534ebbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.247+0000 7fe96e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 0x55d534ebda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.247+0000 7fe96e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 0x55d534ebda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:55406/0 (socket says 172.21.15.78:55406) 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.247+0000 7fe96effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534ebbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35544/0 (socket says 172.21.15.78:35544) 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.247+0000 7fe977814740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ecad80 0x55d534ecc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.247+0000 7fe96e7fc640 1 -- 172.21.15.78:0/304917332 learned_addr learned my addr 172.21.15.78:0/304917332 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.247+0000 7fe977814740 1 -- 172.21.15.78:0/304917332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701b4c20 con 0x55d534ebedf0 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.247+0000 7fe977814740 1 -- 172.21.15.78:0/304917332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701b6760 con 0x55d534ecad80 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.247+0000 7fe977814740 1 -- 172.21.15.78:0/304917332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701b52f0 con 0x55d534ebe4d0 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.247+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/304917332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ecad80 0x55d534ecc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.248+0000 7fe96e7fc640 1 -- 172.21.15.78:0/304917332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ecad80 msgr2=0x55d534ecc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.248+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/304917332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ecad80 0x55d534ecc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.248+0000 7fe96e7fc640 1 -- 172.21.15.78:0/304917332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 msgr2=0x55d534ebbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:56.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.248+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/304917332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534ebbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.248+0000 7fe96e7fc640 1 -- 172.21.15.78:0/304917332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d534ebec20 con 0x55d534ebe4d0 2024-09-11T01:16:56.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.248+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/304917332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 0x55d534ebda60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe964005e00 tx=0x7fe964030bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70be612782c107cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:56.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.248+0000 7fe954ff9640 1 -- 172.21.15.78:0/304917332 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964037040 con 0x55d534ebe4d0 2024-09-11T01:16:56.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.249+0000 7fe954ff9640 1 -- 172.21.15.78:0/304917332 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe96404fa10 con 0x55d534ebe4d0 2024-09-11T01:16:56.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.249+0000 7fe954ff9640 1 -- 172.21.15.78:0/304917332 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964046040 con 0x55d534ebe4d0 2024-09-11T01:16:56.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.250+0000 7fe977814740 1 -- 172.21.15.78:0/304917332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 msgr2=0x55d534ebda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:56.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.250+0000 7fe977814740 1 --2- 172.21.15.78:0/304917332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 0x55d534ebda60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe964005e00 tx=0x7fe964030bd0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.250+0000 7fe977814740 1 -- 172.21.15.78:0/304917332 shutdown_connections 2024-09-11T01:16:56.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.250+0000 7fe977814740 1 --2- 172.21.15.78:0/304917332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ecad80 0x55d534ecc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.250+0000 7fe977814740 1 --2- 172.21.15.78:0/304917332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534ebbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.250+0000 7fe977814740 1 --2- 172.21.15.78:0/304917332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ebe4d0 0x55d534ebda60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.250+0000 7fe977814740 1 -- 172.21.15.78:0/304917332 >> 172.21.15.78:0/304917332 conn(0x55d534ebc300 msgr2=0x55d534ebd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:56.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.250+0000 7fe977814740 1 -- 172.21.15.78:0/304917332 shutdown_connections 2024-09-11T01:16:56.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.250+0000 7fe977814740 1 -- 172.21.15.78:0/304917332 wait complete. 2024-09-11T01:16:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.251+0000 7fe977814740 1 -- start start 2024-09-11T01:16:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.251+0000 7fe977814740 1 Processor -- start 2024-09-11T01:16:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.251+0000 7fe977814740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ebe4d0 0x55d534edc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.251+0000 7fe977814740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534edc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.251+0000 7fe977814740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ecad80 0x55d534f401f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.251+0000 7fe977814740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701b6090 con 0x55d534ebedf0 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.251+0000 7fe977814740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701b59c0 con 0x55d534ebe4d0 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.251+0000 7fe977814740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701b2c70 con 0x55d534ecad80 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.251+0000 7fe977814740 10 client.0 _ll_register_callbacks cb 0x7fe964044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.251+0000 7fe96effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534edc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.251+0000 7fe96effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534edc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35560/0 (socket says 172.21.15.78:35560) 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.252+0000 7fe96effd640 1 -- 172.21.15.78:0/1287357744 learned_addr learned my addr 172.21.15.78:0/1287357744 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.252+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ebe4d0 0x55d534edc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.252+0000 7fe96effd640 1 -- 172.21.15.78:0/1287357744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ecad80 msgr2=0x55d534f401f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.252+0000 7fe96effd640 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ecad80 0x55d534f401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.252+0000 7fe96effd640 1 -- 172.21.15.78:0/1287357744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ebe4d0 msgr2=0x55d534edc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.252+0000 7fe96effd640 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ebe4d0 0x55d534edc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.252+0000 7fe96effd640 1 -- 172.21.15.78:0/1287357744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d534f40930 con 0x55d534ebedf0 2024-09-11T01:16:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.252+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ebe4d0 0x55d534edc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:16:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.252+0000 7fe96effd640 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534edc930 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe960009cd0 tx=0x7fe960009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cda882a2af3a33 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.252+0000 7fe952ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:16:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.253+0000 7fe952ff5640 1 -- 172.21.15.78:0/1287357744 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960004020 con 0x55d534ebedf0 2024-09-11T01:16:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.253+0000 7fe977814740 1 -- 172.21.15.78:0/1287357744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe96404fa10 con 0x55d534ebedf0 2024-09-11T01:16:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.253+0000 7fe952ff5640 1 -- 172.21.15.78:0/1287357744 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7fe9600041c0 con 0x55d534ebedf0 2024-09-11T01:16:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.253+0000 7fe952ff5640 1 -- 172.21.15.78:0/1287357744 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fe9600043c0 con 0x55d534ebedf0 2024-09-11T01:16:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.253+0000 7fe977814740 10 client.18675 fetch_fsmap learned FSMap version 70 2024-09-11T01:16:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.253+0000 7fe977814740 1 -- 172.21.15.78:0/1287357744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55d534f47cc0 con 0x55d534ebedf0 2024-09-11T01:16:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.253+0000 7fe952ff5640 1 -- 172.21.15.78:0/1287357744 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe96404fa10 con 0x55d534ebedf0 2024-09-11T01:16:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.253+0000 7fe952ff5640 1 -- 172.21.15.78:0/1287357744 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7fe9600256a0 con 0x55d534ebedf0 2024-09-11T01:16:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.254+0000 7fe977814740 10 client.18675 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:16:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.254+0000 7fe977814740 10 client.18675 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:16:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.254+0000 7fe977814740 20 client.18675 populate_metadata read hostname 'smithi078' 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.254+0000 7fe977814740 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d534eda6c0 0x55d534f4a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.254+0000 7fe977814740 4 client.18675 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.254+0000 7fe977814740 1 -- 172.21.15.78:0/1287357744 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d534e36df0 con 0x55d534eda6c0 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.254+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d534eda6c0 0x55d534f4a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.254+0000 7fe952ff5640 10 client.18675 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.254+0000 7fe96e7fc640 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d534eda6c0 0x55d534f4a4e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c4df25e198e6b21d server_cookie=dc193f888ad2a936 in_seq=0 out_seq=0 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.255+0000 7fe952ff5640 1 -- 172.21.15.78:0/1287357744 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d534e36df0 con 0x55d534eda6c0 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.255+0000 7fe952ff5640 10 client.18675 handle_command_reply: tid=0 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.255+0000 7fe977814740 1 client.18675 shutdown 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.255+0000 7fe977814740 1 -- 172.21.15.78:0/1287357744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 msgr2=0x55d534edc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.255+0000 7fe977814740 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534edc930 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe960009cd0 tx=0x7fe960009dc0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.255+0000 7fe977814740 1 -- 172.21.15.78:0/1287357744 shutdown_connections 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.255+0000 7fe977814740 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55d534eda6c0 0x55d534f4a4e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.255+0000 7fe977814740 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d534ecad80 0x55d534f401f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.255+0000 7fe977814740 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d534ebedf0 0x55d534edc930 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.255+0000 7fe977814740 1 --2- 172.21.15.78:0/1287357744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d534ebe4d0 0x55d534edc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.255+0000 7fe977814740 1 -- 172.21.15.78:0/1287357744 >> 172.21.15.78:0/1287357744 conn(0x55d534ebc300 msgr2=0x55d534ec9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.255+0000 7fe952ff5640 0 client.18675 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:16:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.256+0000 7fe977814740 1 -- 172.21.15.78:0/1287357744 shutdown_connections 2024-09-11T01:16:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.256+0000 7fe977814740 1 -- 172.21.15.78:0/1287357744 wait complete. 2024-09-11T01:16:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.256+0000 7fe977814740 20 client.18675 trim_cache size 0 max 16384 2024-09-11T01:16:56.261 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:16:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.257+0000 7fe97543f640 1 -- 172.21.15.78:0/2405240241 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe9580b5450 msgr2=0x7fe9580b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.257+0000 7fe97543f640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe9580b5450 0x7fe9580b7910 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe968004af0 tx=0x7fe968004a80 comp rx=0 tx=0).stop 2024-09-11T01:16:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.257+0000 7fe97543f640 1 -- 172.21.15.78:0/2405240241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe97010d120 msgr2=0x7fe9701a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.257+0000 7fe97543f640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe97010d120 0x7fe9701a1f30 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe96800ed80 tx=0x7fe96800c6e0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.257+0000 7fe97543f640 1 -- 172.21.15.78:0/2405240241 shutdown_connections 2024-09-11T01:16:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.257+0000 7fe97543f640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe9580b5450 0x7fe9580b7910 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.257+0000 7fe97543f640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe97010d120 0x7fe9701a1f30 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.257+0000 7fe97543f640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe970104570 0x7fe97010c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.257+0000 7fe97543f640 1 --2- 172.21.15.78:0/2405240241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe970103b70 0x7fe97010bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.257+0000 7fe97543f640 1 -- 172.21.15.78:0/2405240241 >> 172.21.15.78:0/2405240241 conn(0x7fe9700fd720 msgr2=0x7fe9700fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.258+0000 7fe97543f640 1 -- 172.21.15.78:0/2405240241 shutdown_connections 2024-09-11T01:16:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:56.258+0000 7fe97543f640 1 -- 172.21.15.78:0/2405240241 wait complete. 2024-09-11T01:16:56.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 200307 35.58 MB/sec execute 182 sec latency 166.736 ms 2024-09-11T01:16:56.989 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:16:57.373 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:16:57.654 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:57 smithi078 ceph-mon[31315]: pgmap v887: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 775 op/s 2024-09-11T01:16:57.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:57 smithi110 ceph-mon[34712]: pgmap v887: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 775 op/s 2024-09-11T01:16:57.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:57 smithi112 ceph-mon[35384]: pgmap v887: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 775 op/s 2024-09-11T01:16:57.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 201068 35.59 MB/sec execute 183 sec latency 101.925 ms 2024-09-11T01:16:58.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 201283 35.44 MB/sec execute 184 sec latency 452.640 ms 2024-09-11T01:16:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.982+0000 7f154d51c640 1 -- 172.21.15.78:0/3722753200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15481076a0 msgr2=0x7f1548107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.982+0000 7f154d51c640 1 --2- 172.21.15.78:0/3722753200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15481076a0 0x7f1548107ae0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1530009900 tx=0x7f153002a410 comp rx=0 tx=0).stop 2024-09-11T01:16:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.982+0000 7f154d51c640 1 -- 172.21.15.78:0/3722753200 shutdown_connections 2024-09-11T01:16:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.982+0000 7f154d51c640 1 --2- 172.21.15.78:0/3722753200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1548108020 0x7f154810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.982+0000 7f154d51c640 1 --2- 172.21.15.78:0/3722753200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15481076a0 0x7f1548107ae0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.982+0000 7f154d51c640 1 --2- 172.21.15.78:0/3722753200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1548101c90 0x7f1548102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.982+0000 7f154d51c640 1 -- 172.21.15.78:0/3722753200 >> 172.21.15.78:0/3722753200 conn(0x7f15480fd700 msgr2=0x7f15480ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.982+0000 7f154d51c640 1 -- 172.21.15.78:0/3722753200 shutdown_connections 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.982+0000 7f154d51c640 1 -- 172.21.15.78:0/3722753200 wait complete. 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f154d51c640 1 Processor -- start 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f154d51c640 1 -- start start 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f154d51c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1548101c90 0x7f154819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f154d51c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15481076a0 0x7f154819d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f154d51c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1548108020 0x7f1548197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f154d51c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1548111ce0 con 0x7f1548101c90 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f154d51c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1548111b60 con 0x7f15481076a0 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f154d51c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1548111e60 con 0x7f1548108020 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f1546ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1548101c90 0x7f154819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f1546ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1548101c90 0x7f154819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35582/0 (socket says 172.21.15.78:35582) 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f1546ffd640 1 -- 172.21.15.78:0/2300657452 learned_addr learned my addr 172.21.15.78:0/2300657452 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f153ffff640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15481076a0 0x7f154819d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.983+0000 7f15477fe640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1548108020 0x7f1548197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.984+0000 7f1546ffd640 1 -- 172.21.15.78:0/2300657452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1548108020 msgr2=0x7f1548197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.984+0000 7f1546ffd640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1548108020 0x7f1548197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.984+0000 7f1546ffd640 1 -- 172.21.15.78:0/2300657452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15481076a0 msgr2=0x7f154819d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.984+0000 7f1546ffd640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15481076a0 0x7f154819d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.984+0000 7f1546ffd640 1 -- 172.21.15.78:0/2300657452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1548197780 con 0x7f1548101c90 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.984+0000 7f15477fe640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1548108020 0x7f1548197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.984+0000 7f1546ffd640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1548101c90 0x7f154819cd00 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f153800d950 tx=0x7f153800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.984+0000 7f1544ff9640 1 -- 172.21.15.78:0/2300657452 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1538017070 con 0x7f1548101c90 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.984+0000 7f1544ff9640 1 -- 172.21.15.78:0/2300657452 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1538009c30 con 0x7f1548101c90 2024-09-11T01:16:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.984+0000 7f154d51c640 1 -- 172.21.15.78:0/2300657452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1548197a70 con 0x7f1548101c90 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.984+0000 7f1544ff9640 1 -- 172.21.15.78:0/2300657452 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153800f620 con 0x7f1548101c90 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.984+0000 7f154d51c640 1 -- 172.21.15.78:0/2300657452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1548197f80 con 0x7f1548101c90 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.987+0000 7f154f8f1740 1 Processor -- start 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.987+0000 7f154f8f1740 1 -- start start 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.987+0000 7f154f8f1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5009720f0 0x55a5009724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.987+0000 7f1544ff9640 1 -- 172.21.15.78:0/2300657452 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1538010460 con 0x7f1548101c90 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.987+0000 7f154f8f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a50097e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.987+0000 7f154f8f1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a50097e9a0 0x55a500980d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.987+0000 7f154f8f1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a5008e8ea0 con 0x55a5009720f0 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f154f8f1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a5008d82d0 con 0x55a50097e9a0 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f154f8f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a5008dc0e0 con 0x55a500972a10 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f15477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5009720f0 0x55a5009724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f15477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5009720f0 0x55a5009724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35592/0 (socket says 172.21.15.78:35592) 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f15477fe640 1 -- 172.21.15.78:0/2804994134 learned_addr learned my addr 172.21.15.78:0/2804994134 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f1546ffd640 1 --2- 172.21.15.78:0/2804994134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a50097e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f153ffff640 1 --2- 172.21.15.78:0/2804994134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a50097e9a0 0x55a500980d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f15477fe640 1 -- 172.21.15.78:0/2804994134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 msgr2=0x55a50097e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f15477fe640 1 --2- 172.21.15.78:0/2804994134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a50097e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f15477fe640 1 -- 172.21.15.78:0/2804994134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a50097e9a0 msgr2=0x55a500980d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f15477fe640 1 --2- 172.21.15.78:0/2804994134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a50097e9a0 0x55a500980d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f15477fe640 1 -- 172.21.15.78:0/2804994134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a500636650 con 0x55a5009720f0 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f1546ffd640 1 --2- 172.21.15.78:0/2804994134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a50097e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f153ffff640 1 --2- 172.21.15.78:0/2804994134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a50097e9a0 0x55a500980d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f15477fe640 1 --2- 172.21.15.78:0/2804994134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5009720f0 0x55a5009724d0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f154000c450 tx=0x7f154000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6c6c95d2eae36a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f1544ff9640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15340b5450 0x7f15340b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f153d7fa640 1 -- 172.21.15.78:0/2804994134 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540007430 con 0x55a5009720f0 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.988+0000 7f1546ffd640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15340b5450 0x7f15340b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.989+0000 7f153d7fa640 1 -- 172.21.15.78:0/2804994134 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1540007a80 con 0x55a5009720f0 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.989+0000 7f153d7fa640 1 -- 172.21.15.78:0/2804994134 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154000a9f0 con 0x55a5009720f0 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.989+0000 7f1544ff9640 1 -- 172.21.15.78:0/2300657452 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7f15380e5c30 con 0x7f1548101c90 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.989+0000 7f154f8f1740 1 -- 172.21.15.78:0/2804994134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5009720f0 msgr2=0x55a5009724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.989+0000 7f154f8f1740 1 --2- 172.21.15.78:0/2804994134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5009720f0 0x55a5009724d0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f154000c450 tx=0x7f154000c910 comp rx=0 tx=0).stop 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.989+0000 7f154f8f1740 1 -- 172.21.15.78:0/2804994134 shutdown_connections 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.989+0000 7f154f8f1740 1 --2- 172.21.15.78:0/2804994134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a50097e9a0 0x55a500980d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.989+0000 7f154f8f1740 1 --2- 172.21.15.78:0/2804994134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a50097e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.989+0000 7f154f8f1740 1 --2- 172.21.15.78:0/2804994134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5009720f0 0x55a5009724d0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.989+0000 7f154f8f1740 1 -- 172.21.15.78:0/2804994134 >> 172.21.15.78:0/2804994134 conn(0x55a50096ff20 msgr2=0x55a5009710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.989+0000 7f154f8f1740 1 -- 172.21.15.78:0/2804994134 shutdown_connections 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.989+0000 7f154f8f1740 1 -- 172.21.15.78:0/2804994134 wait complete. 2024-09-11T01:16:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f1546ffd640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15340b5450 0x7f15340b7910 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f15380e3470 tx=0x7f15380b5040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f154f8f1740 1 -- start start 2024-09-11T01:16:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f154f8f1740 1 Processor -- start 2024-09-11T01:16:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f154f8f1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5009720f0 0x55a50097e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f154f8f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a50097b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f154f8f1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a50097bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f154f8f1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a500739700 con 0x55a50097e9a0 2024-09-11T01:16:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f154f8f1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a500873160 con 0x55a5009720f0 2024-09-11T01:16:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f154f8f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a500981620 con 0x55a500972a10 2024-09-11T01:16:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f154f8f1740 10 client.0 _ll_register_callbacks cb 0x55a50088c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f15477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a50097bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f15477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a50097bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35604/0 (socket says 172.21.15.78:35604) 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.990+0000 7f15477fe640 1 -- 172.21.15.78:0/3860689620 learned_addr learned my addr 172.21.15.78:0/3860689620 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f15477fe640 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5009720f0 0x55a50097e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f153ffff640 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a50097b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f15477fe640 1 -- 172.21.15.78:0/3860689620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 msgr2=0x55a50097b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f15477fe640 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a50097b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f15477fe640 1 -- 172.21.15.78:0/3860689620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5009720f0 msgr2=0x55a50097e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f15477fe640 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5009720f0 0x55a50097e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f15477fe640 1 -- 172.21.15.78:0/3860689620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5008a0c90 con 0x55a50097e9a0 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f15477fe640 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a50097bd40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1540009670 tx=0x7f154000a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5237dd30da575e19 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f152e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:16:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f152e7fc640 1 -- 172.21.15.78:0/3860689620 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154000a770 con 0x55a50097e9a0 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f152e7fc640 1 -- 172.21.15.78:0/3860689620 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7f1540012040 con 0x55a50097e9a0 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f154f8f1740 1 -- 172.21.15.78:0/3860689620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1540007a80 con 0x55a50097e9a0 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f152e7fc640 1 -- 172.21.15.78:0/3860689620 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f154000a910 con 0x55a50097e9a0 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f154f8f1740 10 client.18687 fetch_fsmap learned FSMap version 70 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.991+0000 7f154f8f1740 1 -- 172.21.15.78:0/3860689620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55a50098cef0 con 0x55a50097e9a0 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.992+0000 7f152e7fc640 1 -- 172.21.15.78:0/3860689620 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1540007a80 con 0x55a50097e9a0 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.992+0000 7f152e7fc640 1 -- 172.21.15.78:0/3860689620 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f154001e030 con 0x55a50097e9a0 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.992+0000 7f154f8f1740 10 client.18687 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.992+0000 7f154f8f1740 10 client.18687 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.992+0000 7f154f8f1740 20 client.18687 populate_metadata read hostname 'smithi078' 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.992+0000 7f154f8f1740 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a50098e350 0x55a500990740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.992+0000 7f154f8f1740 4 client.18687 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.992+0000 7f154f8f1740 1 -- 172.21.15.78:0/3860689620 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5008eadf0 con 0x55a50098e350 2024-09-11T01:16:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.992+0000 7f153ffff640 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a50098e350 0x55a500990740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.993+0000 7f152e7fc640 10 client.18687 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.993+0000 7f153ffff640 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a50098e350 0x55a500990740 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=904c762fd697196f server_cookie=e35096057213a7bf in_seq=0 out_seq=0 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.994+0000 7f152e7fc640 1 -- 172.21.15.78:0/3860689620 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5008eadf0 con 0x55a50098e350 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.994+0000 7f152e7fc640 10 client.18687 handle_command_reply: tid=0 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f154f8f1740 1 client.18687 shutdown 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f154f8f1740 1 -- 172.21.15.78:0/3860689620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 msgr2=0x55a50097bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f154f8f1740 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a50097bd40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1540009670 tx=0x7f154000a5d0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f154f8f1740 1 -- 172.21.15.78:0/3860689620 shutdown_connections 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f154f8f1740 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a50097bd40 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f154f8f1740 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a50097b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f154f8f1740 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a50098e350 0x55a500990740 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f154f8f1740 1 --2- 172.21.15.78:0/3860689620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5009720f0 0x55a50097e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f152e7fc640 0 client.18687 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f154f8f1740 1 -- 172.21.15.78:0/3860689620 >> 172.21.15.78:0/3860689620 conn(0x55a50096ff20 msgr2=0x55a500975a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f154f8f1740 1 -- 172.21.15.78:0/3860689620 shutdown_connections 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f154f8f1740 1 -- 172.21.15.78:0/3860689620 wait complete. 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:57.995+0000 7f154f8f1740 20 client.18687 trim_cache size 0 max 16384 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.010+0000 7f154f8f1740 1 Processor -- start 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.010+0000 7f154f8f1740 1 -- start start 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.010+0000 7f154f8f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5009720f0 0x55a50098a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.010+0000 7f154f8f1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a500972a10 0x55a50097fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.010+0000 7f154f8f1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a5009802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.010+0000 7f154f8f1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15481c5db0 con 0x55a50097e9a0 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.010+0000 7f154f8f1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15481c7870 con 0x55a500972a10 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.010+0000 7f154f8f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15481c6400 con 0x55a5009720f0 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.010+0000 7f15477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a5009802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.010+0000 7f15477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a5009802d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35612/0 (socket says 172.21.15.78:35612) 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.010+0000 7f15477fe640 1 -- 172.21.15.78:0/365668839 learned_addr learned my addr 172.21.15.78:0/365668839 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.011+0000 7f153ffff640 1 --2- 172.21.15.78:0/365668839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a500972a10 0x55a50097fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.011+0000 7f15477fe640 1 --2- 172.21.15.78:0/365668839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5009720f0 0x55a50098a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.011+0000 7f15477fe640 1 -- 172.21.15.78:0/365668839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5009720f0 msgr2=0x55a50098a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.011+0000 7f15477fe640 1 --2- 172.21.15.78:0/365668839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5009720f0 0x55a50098a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.011+0000 7f15477fe640 1 -- 172.21.15.78:0/365668839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a500972a10 msgr2=0x55a50097fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.011+0000 7f15477fe640 1 --2- 172.21.15.78:0/365668839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a500972a10 0x55a50097fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.011+0000 7f15477fe640 1 -- 172.21.15.78:0/365668839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50096b7e0 con 0x55a50097e9a0 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.011+0000 7f15477fe640 1 --2- 172.21.15.78:0/365668839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a5009802d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f1540009f90 tx=0x7f154002d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a3ea91246c5276 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.011+0000 7f153cff9640 1 -- 172.21.15.78:0/365668839 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540009b20 con 0x55a50097e9a0 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.011+0000 7f153cff9640 1 -- 172.21.15.78:0/365668839 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f154000a6d0 con 0x55a50097e9a0 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.011+0000 7f153cff9640 1 -- 172.21.15.78:0/365668839 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154002d670 con 0x55a50097e9a0 2024-09-11T01:16:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 -- 172.21.15.78:0/365668839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 msgr2=0x55a5009802d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 --2- 172.21.15.78:0/365668839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a5009802d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f1540009f90 tx=0x7f154002d430 comp rx=0 tx=0).stop 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 -- 172.21.15.78:0/365668839 shutdown_connections 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 --2- 172.21.15.78:0/365668839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a5009802d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 --2- 172.21.15.78:0/365668839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a500972a10 0x55a50097fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 --2- 172.21.15.78:0/365668839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5009720f0 0x55a50098a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 -- 172.21.15.78:0/365668839 >> 172.21.15.78:0/365668839 conn(0x55a50096ff20 msgr2=0x55a5009769b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 -- 172.21.15.78:0/365668839 shutdown_connections 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 -- 172.21.15.78:0/365668839 wait complete. 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 -- start start 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 Processor -- start 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5009720f0 0x55a5009868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.012+0000 7f154f8f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a500986df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f154f8f1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a500977930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f154f8f1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15481c71a0 con 0x55a50097e9a0 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f154f8f1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15481c6ad0 con 0x55a5009720f0 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f154f8f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1548073330 con 0x55a500972a10 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f154f8f1740 10 client.0 _ll_register_callbacks cb 0x7f1540009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f15477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a500977930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f15477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a500977930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35628/0 (socket says 172.21.15.78:35628) 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f15477fe640 1 -- 172.21.15.78:0/2381196093 learned_addr learned my addr 172.21.15.78:0/2381196093 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f15477fe640 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5009720f0 0x55a5009868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f153ffff640 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a500986df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f15477fe640 1 -- 172.21.15.78:0/2381196093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 msgr2=0x55a500986df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f15477fe640 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a500986df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f15477fe640 1 -- 172.21.15.78:0/2381196093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5009720f0 msgr2=0x55a5009868b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f15477fe640 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5009720f0 0x55a5009868b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f15477fe640 1 -- 172.21.15.78:0/2381196093 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a500977ef0 con 0x55a50097e9a0 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f152e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.013+0000 7f15477fe640 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a500977930 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f15400099c0 tx=0x7f1540018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf843de4080f5c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f152e7fc640 1 -- 172.21.15.78:0/2381196093 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154002d880 con 0x55a50097e9a0 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f152e7fc640 1 -- 172.21.15.78:0/2381196093 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8449+0+0 (secure 0 0 0) 0x7f1540012040 con 0x55a50097e9a0 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f154f8f1740 1 -- 172.21.15.78:0/2381196093 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f154000a6d0 con 0x55a50097e9a0 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f152e7fc640 1 -- 172.21.15.78:0/2381196093 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f154000aab0 con 0x55a50097e9a0 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f154f8f1740 10 client.18693 fetch_fsmap learned FSMap version 70 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f154f8f1740 1 -- 172.21.15.78:0/2381196093 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55a5009790e0 con 0x55a50097e9a0 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f152e7fc640 1 -- 172.21.15.78:0/2381196093 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f154000a6d0 con 0x55a50097e9a0 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f152e7fc640 1 -- 172.21.15.78:0/2381196093 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f154001e070 con 0x55a50097e9a0 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f154f8f1740 10 client.18693 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f154f8f1740 10 client.18693 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f154f8f1740 20 client.18693 populate_metadata read hostname 'smithi078' 2024-09-11T01:16:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f154f8f1740 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a5009fc0c0 0x55a5009fe4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:16:59.636 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:16:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f154f8f1740 4 client.18693 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:16:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f154f8f1740 1 -- 172.21.15.78:0/2381196093 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5008e68a0 con 0x55a5009fc0c0 2024-09-11T01:16:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.014+0000 7f153ffff640 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a5009fc0c0 0x55a5009fe4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:16:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f152e7fc640 10 client.18693 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:16:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f153ffff640 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a5009fc0c0 0x55a5009fe4b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f9e69d453f30912 server_cookie=c54b4d429050b796 in_seq=0 out_seq=0 2024-09-11T01:16:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f152e7fc640 1 -- 172.21.15.78:0/2381196093 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5008e68a0 con 0x55a5009fc0c0 2024-09-11T01:16:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f152e7fc640 10 client.18693 handle_command_reply: tid=0 2024-09-11T01:16:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f154f8f1740 1 client.18693 shutdown 2024-09-11T01:16:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f154f8f1740 1 -- 172.21.15.78:0/2381196093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 msgr2=0x55a500977930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f154f8f1740 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a500977930 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f15400099c0 tx=0x7f1540018040 comp rx=0 tx=0).stop 2024-09-11T01:16:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f154f8f1740 1 -- 172.21.15.78:0/2381196093 shutdown_connections 2024-09-11T01:16:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f154f8f1740 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a50097e9a0 0x55a500977930 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f154f8f1740 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a500972a10 0x55a500986df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f154f8f1740 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55a5009fc0c0 0x55a5009fe4b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f154f8f1740 1 --2- 172.21.15.78:0/2381196093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5009720f0 0x55a5009868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f154f8f1740 1 -- 172.21.15.78:0/2381196093 >> 172.21.15.78:0/2381196093 conn(0x55a50096ff20 msgr2=0x55a50097cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f152e7fc640 0 client.18693 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f154f8f1740 1 -- 172.21.15.78:0/2381196093 shutdown_connections 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f154f8f1740 1 -- 172.21.15.78:0/2381196093 wait complete. 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.015+0000 7f154f8f1740 20 client.18693 trim_cache size 0 max 16384 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.016+0000 7f154d51c640 1 -- 172.21.15.78:0/2300657452 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15340b5450 msgr2=0x7f15340b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.016+0000 7f154d51c640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15340b5450 0x7f15340b7910 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f15380e3470 tx=0x7f15380b5040 comp rx=0 tx=0).stop 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.016+0000 7f154d51c640 1 -- 172.21.15.78:0/2300657452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1548101c90 msgr2=0x7f154819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.016+0000 7f154d51c640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1548101c90 0x7f154819cd00 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f153800d950 tx=0x7f153800de10 comp rx=0 tx=0).stop 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.016+0000 7f154d51c640 1 -- 172.21.15.78:0/2300657452 shutdown_connections 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.016+0000 7f154d51c640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15340b5450 0x7f15340b7910 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.016+0000 7f154d51c640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1548108020 0x7f1548197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.016+0000 7f154d51c640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15481076a0 0x7f154819d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.016+0000 7f154d51c640 1 --2- 172.21.15.78:0/2300657452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1548101c90 0x7f154819cd00 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:16:59.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.016+0000 7f154d51c640 1 -- 172.21.15.78:0/2300657452 >> 172.21.15.78:0/2300657452 conn(0x7f15480fd700 msgr2=0x7f1548100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:16:59.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.016+0000 7f154d51c640 1 -- 172.21.15.78:0/2300657452 shutdown_connections 2024-09-11T01:16:59.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:16:58.016+0000 7f154d51c640 1 -- 172.21.15.78:0/2300657452 wait complete. 2024-09-11T01:16:59.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:16:59 smithi110 ceph-mon[34712]: pgmap v888: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 656 op/s 2024-09-11T01:16:59.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:16:59 smithi112 ceph-mon[35384]: pgmap v888: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 656 op/s 2024-09-11T01:16:59.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:16:59 smithi078 ceph-mon[31315]: pgmap v888: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 656 op/s 2024-09-11T01:16:59.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 201890 35.45 MB/sec execute 185 sec latency 130.423 ms 2024-09-11T01:17:00.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 202542 35.49 MB/sec execute 186 sec latency 322.230 ms 2024-09-11T01:17:01.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:01 smithi110 ceph-mon[34712]: pgmap v889: 97 pgs: 97 active+clean; 2.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 751 op/s 2024-09-11T01:17:01.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:01 smithi112 ceph-mon[35384]: pgmap v889: 97 pgs: 97 active+clean; 2.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 751 op/s 2024-09-11T01:17:01.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:01 smithi078 ceph-mon[31315]: pgmap v889: 97 pgs: 97 active+clean; 2.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 751 op/s 2024-09-11T01:17:01.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 202610 35.33 MB/sec execute 187 sec latency 906.829 ms 2024-09-11T01:17:02.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 202869 35.21 MB/sec execute 188 sec latency 1301.968 ms 2024-09-11T01:17:03.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:03 smithi110 ceph-mon[34712]: pgmap v890: 97 pgs: 97 active+clean; 2.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 711 op/s 2024-09-11T01:17:03.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:03 smithi112 ceph-mon[35384]: pgmap v890: 97 pgs: 97 active+clean; 2.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 711 op/s 2024-09-11T01:17:03.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:03 smithi078 ceph-mon[31315]: pgmap v890: 97 pgs: 97 active+clean; 2.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 711 op/s 2024-09-11T01:17:03.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 203371 35.14 MB/sec execute 189 sec latency 122.112 ms 2024-09-11T01:17:04.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 203786 35.04 MB/sec execute 190 sec latency 130.924 ms 2024-09-11T01:17:05.535 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:17:05.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:05 smithi110 ceph-mon[34712]: pgmap v891: 97 pgs: 97 active+clean; 2.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 693 op/s 2024-09-11T01:17:05.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:05 smithi112 ceph-mon[35384]: pgmap v891: 97 pgs: 97 active+clean; 2.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 693 op/s 2024-09-11T01:17:05.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:05 smithi078 ceph-mon[31315]: pgmap v891: 97 pgs: 97 active+clean; 2.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 693 op/s 2024-09-11T01:17:05.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 204594 35.12 MB/sec execute 191 sec latency 146.370 ms 2024-09-11T01:17:05.918 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:17:06.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 205057 35.02 MB/sec execute 192 sec latency 457.541 ms 2024-09-11T01:17:07.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:07 smithi110 ceph-mon[34712]: pgmap v892: 97 pgs: 97 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 764 op/s 2024-09-11T01:17:07.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:07 smithi112 ceph-mon[35384]: pgmap v892: 97 pgs: 97 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 764 op/s 2024-09-11T01:17:07.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:07 smithi078 ceph-mon[31315]: pgmap v892: 97 pgs: 97 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 764 op/s 2024-09-11T01:17:07.790 INFO:tasks.workunit.client.0.smithi078.stdout: 10 205411 34.96 MB/sec execute 193 sec latency 394.400 ms 2024-09-11T01:17:08.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 205955 34.98 MB/sec execute 194 sec latency 232.183 ms 2024-09-11T01:17:08.812 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:08 smithi078 ceph-mon[31315]: osdmap e100: 12 total, 12 up, 12 in 2024-09-11T01:17:08.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.819+0000 7f8f0f918640 1 -- 172.21.15.78:0/3389594764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f08108060 msgr2=0x7f8f0810e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:08.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.819+0000 7f8f0f918640 1 --2- 172.21.15.78:0/3389594764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f08108060 0x7f8f0810e860 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009ca0 tx=0x7f8f0002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.819+0000 7f8f0f918640 1 -- 172.21.15.78:0/3389594764 shutdown_connections 2024-09-11T01:17:08.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.819+0000 7f8f0f918640 1 --2- 172.21.15.78:0/3389594764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f08108060 0x7f8f0810e860 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.819+0000 7f8f0f918640 1 --2- 172.21.15.78:0/3389594764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f081076e0 0x7f8f08107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.819+0000 7f8f0f918640 1 --2- 172.21.15.78:0/3389594764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f08101cd0 0x7f8f081020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.819+0000 7f8f0f918640 1 -- 172.21.15.78:0/3389594764 >> 172.21.15.78:0/3389594764 conn(0x7f8f080fd740 msgr2=0x7f8f080ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:08.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.819+0000 7f8f0f918640 1 -- 172.21.15.78:0/3389594764 shutdown_connections 2024-09-11T01:17:08.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.819+0000 7f8f0f918640 1 -- 172.21.15.78:0/3389594764 wait complete. 2024-09-11T01:17:08.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.820+0000 7f8f0f918640 1 Processor -- start 2024-09-11T01:17:08.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.820+0000 7f8f0f918640 1 -- start start 2024-09-11T01:17:08.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0f918640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f08101cd0 0x7f8f08113f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0f918640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f081076e0 0x7f8f08112480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0f918640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f08108060 0x7f8f08112a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0f918640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f08111d20 con 0x7f8f081076e0 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0f918640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f08111ba0 con 0x7f8f08108060 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0f918640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f08111ea0 con 0x7f8f08101cd0 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0ceb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f081076e0 0x7f8f08112480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0d6b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f08101cd0 0x7f8f08113f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0ceb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f081076e0 0x7f8f08112480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48670/0 (socket says 172.21.15.78:48670) 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0ceb3640 1 -- 172.21.15.78:0/1065155431 learned_addr learned my addr 172.21.15.78:0/1065155431 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0ceb3640 1 -- 172.21.15.78:0/1065155431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f08101cd0 msgr2=0x7f8f08113f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/1065155431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f08101cd0 0x7f8f08113f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0ceb3640 1 -- 172.21.15.78:0/1065155431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f08108060 msgr2=0x7f8f08112a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T01:17:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/1065155431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f08108060 0x7f8f08112a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.821+0000 7f8f0ceb3640 1 -- 172.21.15.78:0/1065155431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f081132c0 con 0x7f8f081076e0 2024-09-11T01:17:08.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.822+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/1065155431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f081076e0 0x7f8f08112480 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800e9f0 tx=0x7f8ef800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:08.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.822+0000 7f8efe7fc640 1 -- 172.21.15.78:0/1065155431 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef800cd80 con 0x7f8f081076e0 2024-09-11T01:17:08.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.822+0000 7f8efe7fc640 1 -- 172.21.15.78:0/1065155431 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef80108e0 con 0x7f8f081076e0 2024-09-11T01:17:08.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.822+0000 7f8efe7fc640 1 -- 172.21.15.78:0/1065155431 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8018880 con 0x7f8f081076e0 2024-09-11T01:17:08.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.822+0000 7f8f0f918640 1 -- 172.21.15.78:0/1065155431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f081a7940 con 0x7f8f081076e0 2024-09-11T01:17:08.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.822+0000 7f8f0f918640 1 -- 172.21.15.78:0/1065155431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f081a7e30 con 0x7f8f081076e0 2024-09-11T01:17:08.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.825+0000 7f8f11ced740 1 Processor -- start 2024-09-11T01:17:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.825+0000 7f8f11ced740 1 -- start start 2024-09-11T01:17:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8f11ced740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 0x55cabe6b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8f11ced740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 0x55cabe6cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8f11ced740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6cc590 0x55cabe6ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8f11ced740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cabe62dea0 con 0x55cabe6b74d0 2024-09-11T01:17:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8f11ced740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cabe61d2d0 con 0x55cabe6b7df0 2024-09-11T01:17:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8f11ced740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cabe6210e0 con 0x55cabe6cc590 2024-09-11T01:17:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8f0deb5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 0x55cabe6b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8f0d6b4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 0x55cabe6cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8f0d6b4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 0x55cabe6cc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:49368/0 (socket says 172.21.15.78:49368) 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8f0d6b4640 1 -- 172.21.15.78:0/2410044407 learned_addr learned my addr 172.21.15.78:0/2410044407 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8f0deb5640 1 --2- 172.21.15.78:0/2410044407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 0x55cabe6b78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48674/0 (socket says 172.21.15.78:48674) 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8efe7fc640 1 -- 172.21.15.78:0/1065155431 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8ef8010430 con 0x7f8f081076e0 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.826+0000 7f8f0d6b4640 1 -- 172.21.15.78:0/2410044407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6cc590 msgr2=0x55cabe6ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.827+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/2410044407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6cc590 0x55cabe6ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.827+0000 7f8f0d6b4640 1 --2- 172.21.15.78:0/2410044407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6cc590 0x55cabe6ce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.827+0000 7f8f0d6b4640 1 -- 172.21.15.78:0/2410044407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 msgr2=0x55cabe6b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.827+0000 7f8f0d6b4640 1 --2- 172.21.15.78:0/2410044407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 0x55cabe6b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.827+0000 7f8f0d6b4640 1 -- 172.21.15.78:0/2410044407 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabe37b650 con 0x55cabe6b7df0 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.827+0000 7f8f0d6b4640 1 --2- 172.21.15.78:0/2410044407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 0x55cabe6cc050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400bd30 tx=0x7f8f040107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=372d6bb5313fbedd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.827+0000 7f8ef37fe640 1 -- 172.21.15.78:0/2410044407 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04005220 con 0x55cabe6b7df0 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.827+0000 7f8ef37fe640 1 -- 172.21.15.78:0/2410044407 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f04004c50 con 0x55cabe6b7df0 2024-09-11T01:17:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.827+0000 7f8ef37fe640 1 -- 172.21.15.78:0/2410044407 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04011750 con 0x55cabe6b7df0 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.828+0000 7f8efe7fc640 1 --2- 172.21.15.78:0/1065155431 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8ef40b5450 0x7f8ef40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.828+0000 7f8efe7fc640 1 -- 172.21.15.78:0/1065155431 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8493+0+0 (secure 0 0 0) 0x7f8ef80e64e0 con 0x7f8f081076e0 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.828+0000 7f8f11ced740 1 -- 172.21.15.78:0/2410044407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 msgr2=0x55cabe6cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.828+0000 7f8f11ced740 1 --2- 172.21.15.78:0/2410044407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 0x55cabe6cc050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400bd30 tx=0x7f8f040107a0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.829+0000 7f8f0deb5640 1 --2- 172.21.15.78:0/1065155431 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8ef40b5450 0x7f8ef40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.829+0000 7f8f11ced740 1 -- 172.21.15.78:0/2410044407 shutdown_connections 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.829+0000 7f8f11ced740 1 --2- 172.21.15.78:0/2410044407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6cc590 0x55cabe6ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.829+0000 7f8f11ced740 1 --2- 172.21.15.78:0/2410044407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 0x55cabe6cc050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.829+0000 7f8f11ced740 1 --2- 172.21.15.78:0/2410044407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 0x55cabe6b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.829+0000 7f8f11ced740 1 -- 172.21.15.78:0/2410044407 >> 172.21.15.78:0/2410044407 conn(0x55cabe6b5300 msgr2=0x55cabe6b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.829+0000 7f8f11ced740 1 -- 172.21.15.78:0/2410044407 shutdown_connections 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.829+0000 7f8f11ced740 1 -- 172.21.15.78:0/2410044407 wait complete. 2024-09-11T01:17:08.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.830+0000 7f8f0deb5640 1 --2- 172.21.15.78:0/1065155431 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8ef40b5450 0x7f8ef40b7910 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8f00006010 tx=0x7f8f00005c50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.830+0000 7f8f11ced740 1 -- start start 2024-09-11T01:17:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.830+0000 7f8f11ced740 1 Processor -- start 2024-09-11T01:17:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.830+0000 7f8f11ced740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 0x55cabe6c1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.830+0000 7f8f11ced740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6b7df0 0x55cabe6c1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.830+0000 7f8f11ced740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6cc590 0x55cabe6cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.830+0000 7f8f11ced740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cabe47e700 con 0x55cabe6b74d0 2024-09-11T01:17:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.830+0000 7f8f11ced740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cabe5b8160 con 0x55cabe6cc590 2024-09-11T01:17:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.830+0000 7f8f11ced740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cabe6bdb80 con 0x55cabe6b7df0 2024-09-11T01:17:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.830+0000 7f8f11ced740 10 client.0 _ll_register_callbacks cb 0x55cabe5d19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:17:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.831+0000 7f8f0d6b4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 0x55cabe6c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.831+0000 7f8f0d6b4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 0x55cabe6c1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48676/0 (socket says 172.21.15.78:48676) 2024-09-11T01:17:08.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.831+0000 7f8f0d6b4640 1 -- 172.21.15.78:0/3793596491 learned_addr learned my addr 172.21.15.78:0/3793596491 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.831+0000 7f8f0deb5640 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6cc590 0x55cabe6cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.831+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6b7df0 0x55cabe6c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.831+0000 7f8f0d6b4640 1 -- 172.21.15.78:0/3793596491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6b7df0 msgr2=0x55cabe6c1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.831+0000 7f8f0d6b4640 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6b7df0 0x55cabe6c1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.831+0000 7f8f0d6b4640 1 -- 172.21.15.78:0/3793596491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6cc590 msgr2=0x55cabe6cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.831+0000 7f8f0d6b4640 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6cc590 0x55cabe6cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.831+0000 7f8f0d6b4640 1 -- 172.21.15.78:0/3793596491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabe5e5c90 con 0x55cabe6b74d0 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.832+0000 7f8f0d6b4640 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 0x55cabe6c1310 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8f040057c0 tx=0x7f8f040048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a690097e5a80f076 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.832+0000 7f8ef0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.832+0000 7f8ef0ff9640 1 -- 172.21.15.78:0/3793596491 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0402a020 con 0x55cabe6b74d0 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.832+0000 7f8f11ced740 1 -- 172.21.15.78:0/3793596491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f04004c50 con 0x55cabe6b74d0 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.832+0000 7f8ef0ff9640 1 -- 172.21.15.78:0/3793596491 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8493+0+0 (secure 0 0 0) 0x7f8f04005230 con 0x55cabe6b74d0 2024-09-11T01:17:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.832+0000 7f8ef0ff9640 1 -- 172.21.15.78:0/3793596491 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f8f04005430 con 0x55cabe6b74d0 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.832+0000 7f8f11ced740 10 client.18705 fetch_fsmap learned FSMap version 70 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.832+0000 7f8f11ced740 1 -- 172.21.15.78:0/3793596491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55cabe6e3530 con 0x55cabe6b74d0 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.833+0000 7f8ef0ff9640 1 -- 172.21.15.78:0/3793596491 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f04004c50 con 0x55cabe6b74d0 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.833+0000 7f8ef0ff9640 1 -- 172.21.15.78:0/3793596491 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f8f040039d0 con 0x55cabe6b74d0 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.833+0000 7f8f11ced740 10 client.18705 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.833+0000 7f8f11ced740 10 client.18705 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.833+0000 7f8f11ced740 20 client.18705 populate_metadata read hostname 'smithi078' 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.833+0000 7f8f11ced740 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cabe6e4880 0x55cabe6e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.833+0000 7f8f11ced740 4 client.18705 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.833+0000 7f8f11ced740 1 -- 172.21.15.78:0/3793596491 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cabe62fdf0 con 0x55cabe6e4880 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.833+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cabe6e4880 0x55cabe6e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.834+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cabe6e4880 0x55cabe6e6c70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eba420911a1f60c2 server_cookie=342daff5f64077ee in_seq=0 out_seq=0 2024-09-11T01:17:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.834+0000 7f8ef0ff9640 10 client.18705 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.835+0000 7f8ef0ff9640 1 -- 172.21.15.78:0/3793596491 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cabe62fdf0 con 0x55cabe6e4880 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.835+0000 7f8ef0ff9640 10 client.18705 handle_command_reply: tid=0 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.835+0000 7f8f11ced740 1 client.18705 shutdown 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.835+0000 7f8f11ced740 1 -- 172.21.15.78:0/3793596491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 msgr2=0x55cabe6c1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.835+0000 7f8f11ced740 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 0x55cabe6c1310 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8f040057c0 tx=0x7f8f040048f0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.835+0000 7f8f11ced740 1 -- 172.21.15.78:0/3793596491 shutdown_connections 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.835+0000 7f8f11ced740 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6cc590 0x55cabe6cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.835+0000 7f8f11ced740 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6b7df0 0x55cabe6c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.835+0000 7f8f11ced740 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cabe6e4880 0x55cabe6e6c70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.835+0000 7f8f11ced740 1 --2- 172.21.15.78:0/3793596491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b74d0 0x55cabe6c1310 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.836+0000 7f8ef0ff9640 0 client.18705 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.836+0000 7f8f11ced740 1 -- 172.21.15.78:0/3793596491 >> 172.21.15.78:0/3793596491 conn(0x55cabe6b5300 msgr2=0x55cabe6baea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.836+0000 7f8f11ced740 1 -- 172.21.15.78:0/3793596491 shutdown_connections 2024-09-11T01:17:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.836+0000 7f8f11ced740 1 -- 172.21.15.78:0/3793596491 wait complete. 2024-09-11T01:17:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.836+0000 7f8f11ced740 20 client.18705 trim_cache size 0 max 16384 2024-09-11T01:17:08.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.853+0000 7f8f11ced740 1 Processor -- start 2024-09-11T01:17:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.853+0000 7f8f11ced740 1 -- start start 2024-09-11T01:17:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.853+0000 7f8f11ced740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b74d0 0x55cabe6e9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.853+0000 7f8f11ced740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b7df0 0x55cabe6ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.853+0000 7f8f11ced740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6cc590 0x55cabe6b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.853+0000 7f8f11ced740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f08070f50 con 0x55cabe6b7df0 2024-09-11T01:17:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.853+0000 7f8f11ced740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f08072a90 con 0x55cabe6b74d0 2024-09-11T01:17:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.853+0000 7f8f11ced740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f08071620 con 0x55cabe6cc590 2024-09-11T01:17:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8f0ceb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b7df0 0x55cabe6ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8f0deb5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6cc590 0x55cabe6b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8f0ceb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b7df0 0x55cabe6ea1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48686/0 (socket says 172.21.15.78:48686) 2024-09-11T01:17:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8f0deb5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6cc590 0x55cabe6b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57350/0 (socket says 172.21.15.78:57350) 2024-09-11T01:17:08.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8f0ceb3640 1 -- 172.21.15.78:0/1750542545 learned_addr learned my addr 172.21.15.78:0/1750542545 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:08.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8f0d6b4640 1 --2- 172.21.15.78:0/1750542545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b74d0 0x55cabe6e9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8f0ceb3640 1 -- 172.21.15.78:0/1750542545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6cc590 msgr2=0x55cabe6b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:08.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/1750542545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6cc590 0x55cabe6b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8f0ceb3640 1 -- 172.21.15.78:0/1750542545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b74d0 msgr2=0x55cabe6e9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:08.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/1750542545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b74d0 0x55cabe6e9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8f0ceb3640 1 -- 172.21.15.78:0/1750542545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabe6c1c00 con 0x55cabe6b7df0 2024-09-11T01:17:08.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/1750542545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b7df0 0x55cabe6ea1e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80e3c50 tx=0x7f8ef8009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9192423fec63d2c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:08.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.854+0000 7f8ef2ffd640 1 -- 172.21.15.78:0/1750542545 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80ac920 con 0x55cabe6b7df0 2024-09-11T01:17:08.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.855+0000 7f8ef2ffd640 1 -- 172.21.15.78:0/1750542545 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef80b1400 con 0x55cabe6b7df0 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.855+0000 7f8ef2ffd640 1 -- 172.21.15.78:0/1750542545 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80a8270 con 0x55cabe6b7df0 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.855+0000 7f8f11ced740 1 -- 172.21.15.78:0/1750542545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b7df0 msgr2=0x55cabe6ea1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.855+0000 7f8f11ced740 1 --2- 172.21.15.78:0/1750542545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b7df0 0x55cabe6ea1e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80e3c50 tx=0x7f8ef8009700 comp rx=0 tx=0).stop 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.855+0000 7f8f11ced740 1 -- 172.21.15.78:0/1750542545 shutdown_connections 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.855+0000 7f8f11ced740 1 --2- 172.21.15.78:0/1750542545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6cc590 0x55cabe6b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.855+0000 7f8f11ced740 1 --2- 172.21.15.78:0/1750542545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6b7df0 0x55cabe6ea1e0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.855+0000 7f8f11ced740 1 --2- 172.21.15.78:0/1750542545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b74d0 0x55cabe6e9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.855+0000 7f8f11ced740 1 -- 172.21.15.78:0/1750542545 >> 172.21.15.78:0/1750542545 conn(0x55cabe6b5300 msgr2=0x55cabe6bf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.856+0000 7f8f11ced740 1 -- 172.21.15.78:0/1750542545 shutdown_connections 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.856+0000 7f8f11ced740 1 -- 172.21.15.78:0/1750542545 wait complete. 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.856+0000 7f8f11ced740 1 -- start start 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.856+0000 7f8f11ced740 1 Processor -- start 2024-09-11T01:17:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.856+0000 7f8f11ced740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6b74d0 0x55cabe745560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f11ced740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 0x55cabe745aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f11ced740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6cc590 0x55cabe750400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f11ced740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f080723c0 con 0x55cabe6cc590 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f11ced740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f08071cf0 con 0x55cabe6b7df0 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f11ced740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0806ef70 con 0x55cabe6b74d0 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f0d6b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6b74d0 0x55cabe745560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f0d6b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6b74d0 0x55cabe745560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57360/0 (socket says 172.21.15.78:57360) 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f0d6b4640 1 -- 172.21.15.78:0/2522832072 learned_addr learned my addr 172.21.15.78:0/2522832072 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f0deb5640 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6cc590 0x55cabe750400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f11ced740 10 client.0 _ll_register_callbacks cb 0x7f8ef8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 0x55cabe745aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f0ceb3640 1 -- 172.21.15.78:0/2522832072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6b74d0 msgr2=0x55cabe745560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6b74d0 0x55cabe745560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f0ceb3640 1 -- 172.21.15.78:0/2522832072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6cc590 msgr2=0x55cabe750400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6cc590 0x55cabe750400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f0ceb3640 1 -- 172.21.15.78:0/2522832072 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabe750a40 con 0x55cabe6b7df0 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f0d6b4640 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6b74d0 0x55cabe745560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f0ceb3640 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 0x55cabe745aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55cabe6a9ea0 tx=0x7f8ef80ac570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=456d6ed5915a9807 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8ef0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8ef0ff9640 1 -- 172.21.15.78:0/2522832072 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80e4070 con 0x55cabe6b7df0 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8ef0ff9640 1 -- 172.21.15.78:0/2522832072 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8493+0+0 (secure 0 0 0) 0x7f8ef80abb80 con 0x55cabe6b7df0 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.857+0000 7f8f11ced740 1 -- 172.21.15.78:0/2522832072 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef80b1400 con 0x55cabe6b7df0 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.858+0000 7f8ef0ff9640 1 -- 172.21.15.78:0/2522832072 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f8ef80b11d0 con 0x55cabe6b7df0 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.858+0000 7f8f11ced740 10 client.27824 fetch_fsmap learned FSMap version 70 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.858+0000 7f8f11ced740 1 -- 172.21.15.78:0/2522832072 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55cabe751d40 con 0x55cabe6b7df0 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.858+0000 7f8ef0ff9640 1 -- 172.21.15.78:0/2522832072 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef80b1400 con 0x55cabe6b7df0 2024-09-11T01:17:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.858+0000 7f8ef0ff9640 1 -- 172.21.15.78:0/2522832072 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f8ef80ab7f0 con 0x55cabe6b7df0 2024-09-11T01:17:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.858+0000 7f8f11ced740 10 client.27824 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:17:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.858+0000 7f8f11ced740 10 client.27824 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:17:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.858+0000 7f8f11ced740 20 client.27824 populate_metadata read hostname 'smithi078' 2024-09-11T01:17:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.858+0000 7f8f11ced740 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cabe752460 0x55cabe7548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.858+0000 7f8f11ced740 4 client.27824 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:17:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.858+0000 7f8f11ced740 1 -- 172.21.15.78:0/2522832072 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cabe62b8a0 con 0x55cabe752460 2024-09-11T01:17:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.859+0000 7f8f0d6b4640 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cabe752460 0x55cabe7548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.859+0000 7f8f0d6b4640 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cabe752460 0x55cabe7548c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a12bbc17f7e966b server_cookie=3df2401e5ed6e728 in_seq=0 out_seq=0 2024-09-11T01:17:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.859+0000 7f8ef0ff9640 10 client.27824 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:17:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8ef0ff9640 1 -- 172.21.15.78:0/2522832072 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cabe62b8a0 con 0x55cabe752460 2024-09-11T01:17:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8ef0ff9640 10 client.27824 handle_command_reply: tid=0 2024-09-11T01:17:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8f11ced740 1 client.27824 shutdown 2024-09-11T01:17:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8f11ced740 1 -- 172.21.15.78:0/2522832072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 msgr2=0x55cabe745aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8f11ced740 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 0x55cabe745aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55cabe6a9ea0 tx=0x7f8ef80ac570 comp rx=0 tx=0).stop 2024-09-11T01:17:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8f11ced740 1 -- 172.21.15.78:0/2522832072 shutdown_connections 2024-09-11T01:17:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8f11ced740 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55cabe752460 0x55cabe7548c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8f11ced740 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabe6cc590 0x55cabe750400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8f11ced740 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cabe6b7df0 0x55cabe745aa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8ef0ff9640 0 client.27824 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:17:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8f11ced740 1 --2- 172.21.15.78:0/2522832072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cabe6b74d0 0x55cabe745560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8f11ced740 1 -- 172.21.15.78:0/2522832072 >> 172.21.15.78:0/2522832072 conn(0x55cabe6b5300 msgr2=0x55cabe6bfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8f11ced740 1 -- 172.21.15.78:0/2522832072 shutdown_connections 2024-09-11T01:17:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.860+0000 7f8f11ced740 1 -- 172.21.15.78:0/2522832072 wait complete. 2024-09-11T01:17:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.861+0000 7f8f11ced740 20 client.27824 trim_cache size 0 max 16384 2024-09-11T01:17:08.867 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:17:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.861+0000 7f8f0f918640 1 -- 172.21.15.78:0/1065155431 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8ef40b5450 msgr2=0x7f8ef40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.861+0000 7f8f0f918640 1 --2- 172.21.15.78:0/1065155431 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8ef40b5450 0x7f8ef40b7910 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8f00006010 tx=0x7f8f00005c50 comp rx=0 tx=0).stop 2024-09-11T01:17:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.861+0000 7f8f0f918640 1 -- 172.21.15.78:0/1065155431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f081076e0 msgr2=0x7f8f08112480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:08.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.861+0000 7f8f0f918640 1 --2- 172.21.15.78:0/1065155431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f081076e0 0x7f8f08112480 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800e9f0 tx=0x7f8ef800eeb0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.862+0000 7f8f0f918640 1 -- 172.21.15.78:0/1065155431 shutdown_connections 2024-09-11T01:17:08.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.862+0000 7f8f0f918640 1 --2- 172.21.15.78:0/1065155431 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8ef40b5450 0x7f8ef40b7910 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.862+0000 7f8f0f918640 1 --2- 172.21.15.78:0/1065155431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f08108060 0x7f8f08112a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.862+0000 7f8f0f918640 1 --2- 172.21.15.78:0/1065155431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f081076e0 0x7f8f08112480 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.862+0000 7f8f0f918640 1 --2- 172.21.15.78:0/1065155431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f08101cd0 0x7f8f08113f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:08.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.862+0000 7f8f0f918640 1 -- 172.21.15.78:0/1065155431 >> 172.21.15.78:0/1065155431 conn(0x7f8f080fd740 msgr2=0x7f8f0806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:08.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.862+0000 7f8f0f918640 1 -- 172.21.15.78:0/1065155431 shutdown_connections 2024-09-11T01:17:08.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:08.862+0000 7f8f0f918640 1 -- 172.21.15.78:0/1065155431 wait complete. 2024-09-11T01:17:08.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:08 smithi110 ceph-mon[34712]: osdmap e100: 12 total, 12 up, 12 in 2024-09-11T01:17:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:08 smithi112 ceph-mon[35384]: osdmap e100: 12 total, 12 up, 12 in 2024-09-11T01:17:09.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 206545 34.97 MB/sec execute 195 sec latency 177.392 ms 2024-09-11T01:17:09.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:09 smithi110 ceph-mon[34712]: pgmap v894: 97 pgs: 97 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 740 op/s 2024-09-11T01:17:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:09 smithi112 ceph-mon[35384]: pgmap v894: 97 pgs: 97 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 740 op/s 2024-09-11T01:17:09.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:09 smithi078 ceph-mon[31315]: pgmap v894: 97 pgs: 97 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 740 op/s 2024-09-11T01:17:10.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 206977 34.87 MB/sec execute 196 sec latency 194.376 ms 2024-09-11T01:17:10.884 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:17:11.244 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:17:11.683 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:11 smithi078 ceph-mon[31315]: pgmap v895: 97 pgs: 97 active+clean; 2.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 735 op/s 2024-09-11T01:17:11.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 207636 34.89 MB/sec execute 197 sec latency 61.928 ms 2024-09-11T01:17:11.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.809+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/1884950917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d78101960 msgr2=0x7f0d78101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:11.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.809+0000 7f0d7fb7e640 1 --2- 172.21.15.78:0/1884950917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d78101960 0x7f0d78101d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009ca0 tx=0x7f0d7402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.809+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/1884950917 shutdown_connections 2024-09-11T01:17:11.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.809+0000 7f0d7fb7e640 1 --2- 172.21.15.78:0/1884950917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d7810f490 0x7f0d78111880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.809+0000 7f0d7fb7e640 1 --2- 172.21.15.78:0/1884950917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d78102280 0x7f0d7810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.809+0000 7f0d7fb7e640 1 --2- 172.21.15.78:0/1884950917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d78101960 0x7f0d78101d40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.809+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/1884950917 >> 172.21.15.78:0/1884950917 conn(0x7f0d780fd700 msgr2=0x7f0d780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:11.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.809+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/1884950917 shutdown_connections 2024-09-11T01:17:11.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.809+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/1884950917 wait complete. 2024-09-11T01:17:11.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.810+0000 7f0d7fb7e640 1 Processor -- start 2024-09-11T01:17:11.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.810+0000 7f0d7fb7e640 1 -- start start 2024-09-11T01:17:11.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.811+0000 7f0d7fb7e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d78101960 0x7f0d78108110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.811+0000 7f0d7fb7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d78102280 0x7f0d78105360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.811+0000 7f0d7fb7e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d7810f490 0x7f0d781058a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.811+0000 7f0d7fb7e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78114fd0 con 0x7f0d7810f490 2024-09-11T01:17:11.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.811+0000 7f0d7fb7e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78114e50 con 0x7f0d78101960 2024-09-11T01:17:11.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.811+0000 7f0d7fb7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78115150 con 0x7f0d78102280 2024-09-11T01:17:11.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.811+0000 7f0d7e11b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d7810f490 0x7f0d781058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.811+0000 7f0d7e11b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d7810f490 0x7f0d781058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48724/0 (socket says 172.21.15.78:48724) 2024-09-11T01:17:11.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.811+0000 7f0d7e11b640 1 -- 172.21.15.78:0/822773631 learned_addr learned my addr 172.21.15.78:0/822773631 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:11.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.811+0000 7f0d7d91a640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d78101960 0x7f0d78108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.811+0000 7f0d7d119640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d78102280 0x7f0d78105360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.812+0000 7f0d7e11b640 1 -- 172.21.15.78:0/822773631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d78102280 msgr2=0x7f0d78105360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:11.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.812+0000 7f0d7e11b640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d78102280 0x7f0d78105360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.812+0000 7f0d7e11b640 1 -- 172.21.15.78:0/822773631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d78101960 msgr2=0x7f0d78108110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:11.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.812+0000 7f0d7e11b640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d78101960 0x7f0d78108110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.812+0000 7f0d7e11b640 1 -- 172.21.15.78:0/822773631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d78106160 con 0x7f0d7810f490 2024-09-11T01:17:11.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.812+0000 7f0d7e11b640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d7810f490 0x7f0d781058a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000e970 tx=0x7f0d7000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:11.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.813+0000 7f0d6effd640 1 -- 172.21.15.78:0/822773631 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7000ccc0 con 0x7f0d7810f490 2024-09-11T01:17:11.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.813+0000 7f0d6effd640 1 -- 172.21.15.78:0/822773631 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d700054d0 con 0x7f0d7810f490 2024-09-11T01:17:11.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.813+0000 7f0d6effd640 1 -- 172.21.15.78:0/822773631 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70010430 con 0x7f0d7810f490 2024-09-11T01:17:11.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.813+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/822773631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d781abdd0 con 0x7f0d7810f490 2024-09-11T01:17:11.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.813+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/822773631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d781ac1d0 con 0x7f0d7810f490 2024-09-11T01:17:11.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.817+0000 7f0d6effd640 1 -- 172.21.15.78:0/822773631 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0d70005020 con 0x7f0d7810f490 2024-09-11T01:17:11.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.819+0000 7f0d81f53740 1 Processor -- start 2024-09-11T01:17:11.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.820+0000 7f0d81f53740 1 -- start start 2024-09-11T01:17:11.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.820+0000 7f0d81f53740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06960f0 0x5636a06964d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.820+0000 7f0d81f53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 0x5636a06a2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.820+0000 7f0d81f53740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a06a29a0 0x5636a06a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.820+0000 7f0d81f53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5636a060cea0 con 0x5636a0696a10 2024-09-11T01:17:11.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.820+0000 7f0d81f53740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5636a05fc2d0 con 0x5636a06a29a0 2024-09-11T01:17:11.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.820+0000 7f0d81f53740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5636a06000e0 con 0x5636a06960f0 2024-09-11T01:17:11.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.820+0000 7f0d7d91a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 0x5636a06a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.820+0000 7f0d7d91a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 0x5636a06a2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48736/0 (socket says 172.21.15.78:48736) 2024-09-11T01:17:11.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.820+0000 7f0d7d91a640 1 -- 172.21.15.78:0/2244934745 learned_addr learned my addr 172.21.15.78:0/2244934745 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:11.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.820+0000 7f0d7d119640 1 --2- 172.21.15.78:0/2244934745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06960f0 0x5636a06964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.821+0000 7f0d7d91a640 1 -- 172.21.15.78:0/2244934745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06960f0 msgr2=0x5636a06964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:11.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.821+0000 7f0d7d91a640 1 --2- 172.21.15.78:0/2244934745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06960f0 0x5636a06964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.821+0000 7f0d7d91a640 1 -- 172.21.15.78:0/2244934745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a06a29a0 msgr2=0x5636a06a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:11.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.821+0000 7f0d7d91a640 1 --2- 172.21.15.78:0/2244934745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a06a29a0 0x5636a06a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.821+0000 7f0d7d91a640 1 -- 172.21.15.78:0/2244934745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636a035a650 con 0x5636a0696a10 2024-09-11T01:17:11.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.821+0000 7f0d7d119640 1 --2- 172.21.15.78:0/2244934745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06960f0 0x5636a06964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:17:11.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.821+0000 7f0d7d91a640 1 --2- 172.21.15.78:0/2244934745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 0x5636a06a2460 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0d7402ade0 tx=0x7f0d74005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec9abcdb9389261 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:11.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.822+0000 7f0d6effd640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0d640b55a0 0x7f0d640b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.822+0000 7f0d6effd640 1 -- 172.21.15.78:0/822773631 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8493+0+0 (secure 0 0 0) 0x7f0d700e63e0 con 0x7f0d7810f490 2024-09-11T01:17:11.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.823+0000 7f0d63fff640 1 -- 172.21.15.78:0/2244934745 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74031cf0 con 0x5636a0696a10 2024-09-11T01:17:11.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.823+0000 7f0d63fff640 1 -- 172.21.15.78:0/2244934745 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d74036030 con 0x5636a0696a10 2024-09-11T01:17:11.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.823+0000 7f0d63fff640 1 -- 172.21.15.78:0/2244934745 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d740042a0 con 0x5636a0696a10 2024-09-11T01:17:11.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.824+0000 7f0d7d119640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0d640b55a0 0x7f0d640b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.824+0000 7f0d81f53740 1 -- 172.21.15.78:0/2244934745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 msgr2=0x5636a06a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:11.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.824+0000 7f0d81f53740 1 --2- 172.21.15.78:0/2244934745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 0x5636a06a2460 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0d7402ade0 tx=0x7f0d74005fa0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.824+0000 7f0d81f53740 1 -- 172.21.15.78:0/2244934745 shutdown_connections 2024-09-11T01:17:11.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.824+0000 7f0d81f53740 1 --2- 172.21.15.78:0/2244934745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a06a29a0 0x5636a06a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.824+0000 7f0d81f53740 1 --2- 172.21.15.78:0/2244934745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 0x5636a06a2460 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.825+0000 7f0d81f53740 1 --2- 172.21.15.78:0/2244934745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06960f0 0x5636a06964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.825+0000 7f0d81f53740 1 -- 172.21.15.78:0/2244934745 >> 172.21.15.78:0/2244934745 conn(0x5636a0693f20 msgr2=0x5636a06950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:11.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.825+0000 7f0d7d119640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0d640b55a0 0x7f0d640b7a60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0d78107e60 tx=0x7f0d6800c430 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:11.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.825+0000 7f0d81f53740 1 -- 172.21.15.78:0/2244934745 shutdown_connections 2024-09-11T01:17:11.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.825+0000 7f0d81f53740 1 -- 172.21.15.78:0/2244934745 wait complete. 2024-09-11T01:17:11.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.826+0000 7f0d81f53740 1 -- start start 2024-09-11T01:17:11.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.826+0000 7f0d81f53740 1 Processor -- start 2024-09-11T01:17:11.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.827+0000 7f0d81f53740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06960f0 0x5636a06a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.827+0000 7f0d81f53740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a0696a10 0x5636a069f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.827+0000 7f0d81f53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06a29a0 0x5636a069fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.827+0000 7f0d7d91a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06960f0 0x5636a06a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.828+0000 7f0d81f53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5636a045d700 con 0x5636a06a29a0 2024-09-11T01:17:11.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.828+0000 7f0d81f53740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5636a0597160 con 0x5636a0696a10 2024-09-11T01:17:11.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.828+0000 7f0d81f53740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5636a06a5440 con 0x5636a06960f0 2024-09-11T01:17:11.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.828+0000 7f0d81f53740 10 client.0 _ll_register_callbacks cb 0x5636a05b09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:17:11.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.828+0000 7f0d7e11b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06a29a0 0x5636a069fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.828+0000 7f0d7e11b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06a29a0 0x5636a069fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48744/0 (socket says 172.21.15.78:48744) 2024-09-11T01:17:11.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.828+0000 7f0d7e11b640 1 -- 172.21.15.78:0/4158752833 learned_addr learned my addr 172.21.15.78:0/4158752833 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:11.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.828+0000 7f0d7e11b640 1 -- 172.21.15.78:0/4158752833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06960f0 msgr2=0x5636a06a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:11.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.828+0000 7f0d7e11b640 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06960f0 0x5636a06a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.828+0000 7f0d7e11b640 1 -- 172.21.15.78:0/4158752833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a0696a10 msgr2=0x5636a069f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:11.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.831+0000 7f0d7d119640 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a0696a10 0x5636a069f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.832+0000 7f0d7e11b640 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a0696a10 0x5636a069f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.832+0000 7f0d7e11b640 1 -- 172.21.15.78:0/4158752833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a05c4c90 con 0x5636a06a29a0 2024-09-11T01:17:11.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.832+0000 7f0d7e11b640 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06a29a0 0x5636a069fd40 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0d700ac810 tx=0x7f0d700ac290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e6681c23e16c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:11.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.832+0000 7f0d617fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:17:11.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.835+0000 7f0d7d119640 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a0696a10 0x5636a069f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:17:11.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.851+0000 7f0d617fa640 1 -- 172.21.15.78:0/4158752833 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700ac390 con 0x5636a06a29a0 2024-09-11T01:17:11.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.851+0000 7f0d617fa640 1 -- 172.21.15.78:0/4158752833 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8493+0+0 (secure 0 0 0) 0x7f0d700cfbd0 con 0x5636a06a29a0 2024-09-11T01:17:11.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.851+0000 7f0d81f53740 1 -- 172.21.15.78:0/4158752833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d74036030 con 0x5636a06a29a0 2024-09-11T01:17:11.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.852+0000 7f0d617fa640 1 -- 172.21.15.78:0/4158752833 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f0d700acf80 con 0x5636a06a29a0 2024-09-11T01:17:11.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.853+0000 7f0d81f53740 10 client.18723 fetch_fsmap learned FSMap version 70 2024-09-11T01:17:11.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.853+0000 7f0d81f53740 1 -- 172.21.15.78:0/4158752833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5636a06b0d40 con 0x5636a06a29a0 2024-09-11T01:17:11.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.859+0000 7f0d617fa640 1 -- 172.21.15.78:0/4158752833 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d700ac880 con 0x5636a06a29a0 2024-09-11T01:17:11.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.859+0000 7f0d617fa640 1 -- 172.21.15.78:0/4158752833 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f0d700adbb0 con 0x5636a06a29a0 2024-09-11T01:17:11.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.860+0000 7f0d81f53740 10 client.18723 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:17:11.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.860+0000 7f0d81f53740 10 client.18723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:17:11.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.860+0000 7f0d81f53740 20 client.18723 populate_metadata read hostname 'smithi078' 2024-09-11T01:17:11.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.860+0000 7f0d81f53740 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636a06b2200 0x5636a06b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.860+0000 7f0d81f53740 4 client.18723 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:17:11.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.860+0000 7f0d81f53740 1 -- 172.21.15.78:0/4158752833 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0d74036030 con 0x5636a06b2200 2024-09-11T01:17:11.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.862+0000 7f0d7d91a640 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636a06b2200 0x5636a06b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.867+0000 7f0d7d91a640 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636a06b2200 0x5636a06b45f0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f37f7cc09fc5cd3 server_cookie=142e3f8e7fe4a10b in_seq=0 out_seq=0 2024-09-11T01:17:11.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.867+0000 7f0d617fa640 10 client.18723 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:11.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.876+0000 7f0d617fa640 1 -- 172.21.15.78:0/4158752833 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0d74036030 con 0x5636a06b2200 2024-09-11T01:17:11.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.876+0000 7f0d617fa640 10 client.18723 handle_command_reply: tid=0 2024-09-11T01:17:11.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.876+0000 7f0d81f53740 1 client.18723 shutdown 2024-09-11T01:17:11.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.876+0000 7f0d81f53740 1 -- 172.21.15.78:0/4158752833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06a29a0 msgr2=0x5636a069fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:11.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.876+0000 7f0d81f53740 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06a29a0 0x5636a069fd40 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0d700ac810 tx=0x7f0d700ac290 comp rx=0 tx=0).stop 2024-09-11T01:17:11.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.877+0000 7f0d81f53740 1 -- 172.21.15.78:0/4158752833 shutdown_connections 2024-09-11T01:17:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.877+0000 7f0d81f53740 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636a06b2200 0x5636a06b45f0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.877+0000 7f0d81f53740 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06a29a0 0x5636a069fd40 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.877+0000 7f0d617fa640 0 client.18723 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.877+0000 7f0d81f53740 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a0696a10 0x5636a069f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.877+0000 7f0d81f53740 1 --2- 172.21.15.78:0/4158752833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06960f0 0x5636a06a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.877+0000 7f0d81f53740 1 -- 172.21.15.78:0/4158752833 >> 172.21.15.78:0/4158752833 conn(0x5636a0693f20 msgr2=0x5636a0699ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.877+0000 7f0d81f53740 1 -- 172.21.15.78:0/4158752833 shutdown_connections 2024-09-11T01:17:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.877+0000 7f0d81f53740 1 -- 172.21.15.78:0/4158752833 wait complete. 2024-09-11T01:17:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.878+0000 7f0d81f53740 20 client.18723 trim_cache size 0 max 16384 2024-09-11T01:17:11.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.895+0000 7f0d81f53740 1 Processor -- start 2024-09-11T01:17:11.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.895+0000 7f0d81f53740 1 -- start start 2024-09-11T01:17:11.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.895+0000 7f0d81f53740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a06960f0 0x5636a06ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.895+0000 7f0d81f53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 0x5636a06a1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.895+0000 7f0d81f53740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06a29a0 0x5636a06a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.895+0000 7f0d81f53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781b4d50 con 0x5636a0696a10 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.896+0000 7f0d7d119640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 0x5636a06a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.896+0000 7f0d81f53740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781b6890 con 0x5636a06960f0 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.896+0000 7f0d81f53740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781b5420 con 0x5636a06a29a0 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.896+0000 7f0d7d119640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 0x5636a06a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48746/0 (socket says 172.21.15.78:48746) 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.896+0000 7f0d7d119640 1 -- 172.21.15.78:0/3399541741 learned_addr learned my addr 172.21.15.78:0/3399541741 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.896+0000 7f0d7e11b640 1 --2- 172.21.15.78:0/3399541741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06a29a0 0x5636a06a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.896+0000 7f0d7d119640 1 -- 172.21.15.78:0/3399541741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06a29a0 msgr2=0x5636a06a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.896+0000 7f0d7d119640 1 --2- 172.21.15.78:0/3399541741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06a29a0 0x5636a06a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.896+0000 7f0d7d119640 1 -- 172.21.15.78:0/3399541741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a06960f0 msgr2=0x5636a06ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.896+0000 7f0d7d119640 1 --2- 172.21.15.78:0/3399541741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a06960f0 0x5636a06ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.896+0000 7f0d7d119640 1 -- 172.21.15.78:0/3399541741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636a069f4c0 con 0x5636a0696a10 2024-09-11T01:17:11.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.896+0000 7f0d7d119640 1 --2- 172.21.15.78:0/3399541741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 0x5636a06a1450 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800c730 tx=0x7f0d68013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa6d949c76c71fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:11.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.897+0000 7f0d637fe640 1 -- 172.21.15.78:0/3399541741 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680054f0 con 0x5636a0696a10 2024-09-11T01:17:11.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.897+0000 7f0d637fe640 1 -- 172.21.15.78:0/3399541741 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d68024020 con 0x5636a0696a10 2024-09-11T01:17:11.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.897+0000 7f0d637fe640 1 -- 172.21.15.78:0/3399541741 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68003b50 con 0x5636a0696a10 2024-09-11T01:17:11.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.897+0000 7f0d81f53740 1 -- 172.21.15.78:0/3399541741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 msgr2=0x5636a06a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:11.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.897+0000 7f0d81f53740 1 --2- 172.21.15.78:0/3399541741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 0x5636a06a1450 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800c730 tx=0x7f0d68013000 comp rx=0 tx=0).stop 2024-09-11T01:17:11.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.897+0000 7f0d81f53740 1 -- 172.21.15.78:0/3399541741 shutdown_connections 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.897+0000 7f0d81f53740 1 --2- 172.21.15.78:0/3399541741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06a29a0 0x5636a06a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.897+0000 7f0d81f53740 1 --2- 172.21.15.78:0/3399541741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a0696a10 0x5636a06a1450 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.897+0000 7f0d81f53740 1 --2- 172.21.15.78:0/3399541741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a06960f0 0x5636a06ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.897+0000 7f0d81f53740 1 -- 172.21.15.78:0/3399541741 >> 172.21.15.78:0/3399541741 conn(0x5636a0693f20 msgr2=0x5636a06a0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.897+0000 7f0d81f53740 1 -- 172.21.15.78:0/3399541741 shutdown_connections 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.898+0000 7f0d81f53740 1 -- 172.21.15.78:0/3399541741 wait complete. 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.898+0000 7f0d81f53740 1 -- start start 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.898+0000 7f0d81f53740 1 Processor -- start 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.898+0000 7f0d81f53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06960f0 0x5636a06aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.898+0000 7f0d81f53740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a0696a10 0x5636a06a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.898+0000 7f0d81f53740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06a29a0 0x5636a06a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d81f53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781b61c0 con 0x5636a06960f0 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d81f53740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781b5af0 con 0x5636a0696a10 2024-09-11T01:17:11.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d81f53740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781b2d50 con 0x5636a06a29a0 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d7d91a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06960f0 0x5636a06aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d81f53740 10 client.0 _ll_register_callbacks cb 0x7f0d700a8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d7d91a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06960f0 0x5636a06aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48750/0 (socket says 172.21.15.78:48750) 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d7d91a640 1 -- 172.21.15.78:0/3626793095 learned_addr learned my addr 172.21.15.78:0/3626793095 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d7d119640 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a0696a10 0x5636a06a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d7e11b640 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06a29a0 0x5636a06a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d7d91a640 1 -- 172.21.15.78:0/3626793095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06a29a0 msgr2=0x5636a06a6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d7d91a640 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06a29a0 0x5636a06a6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d7d91a640 1 -- 172.21.15.78:0/3626793095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a0696a10 msgr2=0x5636a06a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d7d91a640 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a0696a10 0x5636a06a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d7d91a640 1 -- 172.21.15.78:0/3626793095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a06a7600 con 0x5636a06960f0 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d7d91a640 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06960f0 0x5636a06aae40 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0d7402f540 tx=0x7f0d74003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74eb81284f2ccf62 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:11.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d617fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d617fa640 1 -- 172.21.15.78:0/3626793095 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74055060 con 0x5636a06960f0 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d617fa640 1 -- 172.21.15.78:0/3626793095 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8493+0+0 (secure 0 0 0) 0x7f0d74031450 con 0x5636a06960f0 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.899+0000 7f0d81f53740 1 -- 172.21.15.78:0/3626793095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d68024020 con 0x5636a06960f0 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.900+0000 7f0d617fa640 1 -- 172.21.15.78:0/3626793095 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f0d7405d050 con 0x5636a06960f0 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.900+0000 7f0d81f53740 10 client.18729 fetch_fsmap learned FSMap version 70 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.900+0000 7f0d81f53740 1 -- 172.21.15.78:0/3626793095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5636a07200b0 con 0x5636a06960f0 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.900+0000 7f0d617fa640 1 -- 172.21.15.78:0/3626793095 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d68024020 con 0x5636a06960f0 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.900+0000 7f0d617fa640 1 -- 172.21.15.78:0/3626793095 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f0d74043070 con 0x5636a06960f0 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.900+0000 7f0d81f53740 10 client.18729 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.900+0000 7f0d81f53740 10 client.18729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.900+0000 7f0d81f53740 20 client.18729 populate_metadata read hostname 'smithi078' 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.900+0000 7f0d81f53740 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636a06b2200 0x5636a07224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.900+0000 7f0d81f53740 4 client.18729 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:17:11.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.900+0000 7f0d81f53740 1 -- 172.21.15.78:0/3626793095 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636a060edf0 con 0x5636a06b2200 2024-09-11T01:17:11.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.900+0000 7f0d7d119640 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636a06b2200 0x5636a07224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:11.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.901+0000 7f0d7d119640 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636a06b2200 0x5636a07224e0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c3ae14a0a6c3d server_cookie=4c60993f0258de27 in_seq=0 out_seq=0 2024-09-11T01:17:11.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.901+0000 7f0d617fa640 10 client.18729 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:11.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.901+0000 7f0d617fa640 1 -- 172.21.15.78:0/3626793095 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636a060edf0 con 0x5636a06b2200 2024-09-11T01:17:11.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.901+0000 7f0d617fa640 10 client.18729 handle_command_reply: tid=0 2024-09-11T01:17:11.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.901+0000 7f0d81f53740 1 client.18729 shutdown 2024-09-11T01:17:11.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.902+0000 7f0d81f53740 1 -- 172.21.15.78:0/3626793095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06960f0 msgr2=0x5636a06aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:11.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.902+0000 7f0d81f53740 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06960f0 0x5636a06aae40 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0d7402f540 tx=0x7f0d74003c30 comp rx=0 tx=0).stop 2024-09-11T01:17:11.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.902+0000 7f0d81f53740 1 -- 172.21.15.78:0/3626793095 shutdown_connections 2024-09-11T01:17:11.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.902+0000 7f0d81f53740 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a06a29a0 0x5636a06a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.902+0000 7f0d81f53740 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636a0696a10 0x5636a06a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.902+0000 7f0d81f53740 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5636a06b2200 0x5636a07224e0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.902+0000 7f0d81f53740 1 --2- 172.21.15.78:0/3626793095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636a06960f0 0x5636a06aae40 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.902+0000 7f0d81f53740 1 -- 172.21.15.78:0/3626793095 >> 172.21.15.78:0/3626793095 conn(0x5636a0693f20 msgr2=0x5636a06a1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:11.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.902+0000 7f0d617fa640 0 client.18729 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:11.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.902+0000 7f0d81f53740 1 -- 172.21.15.78:0/3626793095 shutdown_connections 2024-09-11T01:17:11.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.902+0000 7f0d81f53740 1 -- 172.21.15.78:0/3626793095 wait complete. 2024-09-11T01:17:11.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.902+0000 7f0d81f53740 20 client.18729 trim_cache size 0 max 16384 2024-09-11T01:17:11.908 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:17:11.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.903+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/822773631 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0d640b55a0 msgr2=0x7f0d640b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:11.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.903+0000 7f0d7fb7e640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0d640b55a0 0x7f0d640b7a60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0d78107e60 tx=0x7f0d6800c430 comp rx=0 tx=0).stop 2024-09-11T01:17:11.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.903+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/822773631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d7810f490 msgr2=0x7f0d781058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:11.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.903+0000 7f0d7fb7e640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d7810f490 0x7f0d781058a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000e970 tx=0x7f0d7000ee30 comp rx=0 tx=0).stop 2024-09-11T01:17:11.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.903+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/822773631 shutdown_connections 2024-09-11T01:17:11.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.903+0000 7f0d7fb7e640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f0d640b55a0 0x7f0d640b7a60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.903+0000 7f0d7fb7e640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d7810f490 0x7f0d781058a0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.903+0000 7f0d7fb7e640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d78102280 0x7f0d78105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.903+0000 7f0d7fb7e640 1 --2- 172.21.15.78:0/822773631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d78101960 0x7f0d78108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:11.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.903+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/822773631 >> 172.21.15.78:0/822773631 conn(0x7f0d780fd700 msgr2=0x7f0d780ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:11.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.903+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/822773631 shutdown_connections 2024-09-11T01:17:11.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:11.903+0000 7f0d7fb7e640 1 -- 172.21.15.78:0/822773631 wait complete. 2024-09-11T01:17:11.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:11 smithi110 ceph-mon[34712]: pgmap v895: 97 pgs: 97 active+clean; 2.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 735 op/s 2024-09-11T01:17:11.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:11 smithi112 ceph-mon[35384]: pgmap v895: 97 pgs: 97 active+clean; 2.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 735 op/s 2024-09-11T01:17:12.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 208144 34.83 MB/sec execute 198 sec latency 266.117 ms 2024-09-11T01:17:13.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 208891 34.87 MB/sec execute 199 sec latency 69.924 ms 2024-09-11T01:17:13.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:13 smithi110 ceph-mon[34712]: pgmap v896: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 696 op/s 2024-09-11T01:17:13.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:13 smithi112 ceph-mon[35384]: pgmap v896: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 696 op/s 2024-09-11T01:17:13.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:13 smithi078 ceph-mon[31315]: pgmap v896: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 696 op/s 2024-09-11T01:17:14.587 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:17:14.587 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:17:14.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 209621 34.88 MB/sec execute 200 sec latency 186.566 ms 2024-09-11T01:17:14.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:14 smithi110 ceph-mon[34712]: osdmap e101: 12 total, 12 up, 12 in 2024-09-11T01:17:14.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:14 smithi112 ceph-mon[35384]: osdmap e101: 12 total, 12 up, 12 in 2024-09-11T01:17:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:17:14.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:14 smithi078 ceph-mon[31315]: osdmap e101: 12 total, 12 up, 12 in 2024-09-11T01:17:15.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 210303 34.89 MB/sec execute 201 sec latency 94.031 ms 2024-09-11T01:17:15.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:15 smithi110 ceph-mon[34712]: pgmap v898: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 582 op/s 2024-09-11T01:17:15.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:15 smithi112 ceph-mon[35384]: pgmap v898: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 582 op/s 2024-09-11T01:17:15.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:15 smithi078 ceph-mon[31315]: pgmap v898: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 582 op/s 2024-09-11T01:17:16.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 210935 34.90 MB/sec execute 202 sec latency 137.008 ms 2024-09-11T01:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.073+0000 7f965dc92640 1 -- 172.21.15.78:0/2544393755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f965810d0e0 msgr2=0x7f965810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.073+0000 7f965dc92640 1 --2- 172.21.15.78:0/2544393755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f965810d0e0 0x7f965810f550 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9650009be0 tx=0x7f965002a6c0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.073+0000 7f965dc92640 1 -- 172.21.15.78:0/2544393755 shutdown_connections 2024-09-11T01:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.073+0000 7f965dc92640 1 --2- 172.21.15.78:0/2544393755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f965810d0e0 0x7f965810f550 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.073+0000 7f965dc92640 1 --2- 172.21.15.78:0/2544393755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9658104530 0x7f965810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.073+0000 7f965dc92640 1 --2- 172.21.15.78:0/2544393755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9658103b30 0x7f9658103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.073+0000 7f965dc92640 1 -- 172.21.15.78:0/2544393755 >> 172.21.15.78:0/2544393755 conn(0x7f96580fd700 msgr2=0x7f96580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.075+0000 7f965dc92640 1 -- 172.21.15.78:0/2544393755 shutdown_connections 2024-09-11T01:17:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.075+0000 7f965dc92640 1 -- 172.21.15.78:0/2544393755 wait complete. 2024-09-11T01:17:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.076+0000 7f965dc92640 1 Processor -- start 2024-09-11T01:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.077+0000 7f965dc92640 1 -- start start 2024-09-11T01:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.077+0000 7f965dc92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9658103b30 0x7f9658103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.077+0000 7f965dc92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9658104530 0x7f96581019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.077+0000 7f965dc92640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f965810d0e0 0x7f9658101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.077+0000 7f96577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9658103b30 0x7f9658103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.077+0000 7f965dc92640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9658112d60 con 0x7f9658103b30 2024-09-11T01:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.077+0000 7f965dc92640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9658112be0 con 0x7f965810d0e0 2024-09-11T01:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.077+0000 7f965dc92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9658112ee0 con 0x7f9658104530 2024-09-11T01:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.077+0000 7f96577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9658103b30 0x7f9658103460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49454/0 (socket says 172.21.15.78:49454) 2024-09-11T01:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.077+0000 7f96577fe640 1 -- 172.21.15.78:0/3755021584 learned_addr learned my addr 172.21.15.78:0/3755021584 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.078+0000 7f96577fe640 1 -- 172.21.15.78:0/3755021584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9658104530 msgr2=0x7f96581019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.078+0000 7f96577fe640 1 --2- 172.21.15.78:0/3755021584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9658104530 0x7f96581019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.078+0000 7f96577fe640 1 -- 172.21.15.78:0/3755021584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f965810d0e0 msgr2=0x7f9658101fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.078+0000 7f96577fe640 1 --2- 172.21.15.78:0/3755021584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f965810d0e0 0x7f9658101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.078+0000 7f96577fe640 1 -- 172.21.15.78:0/3755021584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96581abb10 con 0x7f9658103b30 2024-09-11T01:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.078+0000 7f96577fe640 1 --2- 172.21.15.78:0/3755021584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9658103b30 0x7f9658103460 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f964800c890 tx=0x7f964800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.079+0000 7f9654ff9640 1 -- 172.21.15.78:0/3755021584 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648004220 con 0x7f9658103b30 2024-09-11T01:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.079+0000 7f9654ff9640 1 -- 172.21.15.78:0/3755021584 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96480043c0 con 0x7f9658103b30 2024-09-11T01:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.079+0000 7f9654ff9640 1 -- 172.21.15.78:0/3755021584 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648004a00 con 0x7f9658103b30 2024-09-11T01:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.079+0000 7f965dc92640 1 -- 172.21.15.78:0/3755021584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96581abce0 con 0x7f9658103b30 2024-09-11T01:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.079+0000 7f965dc92640 1 -- 172.21.15.78:0/3755021584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96581ac1a0 con 0x7f9658103b30 2024-09-11T01:17:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.083+0000 7f9654ff9640 1 -- 172.21.15.78:0/3755021584 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9648004c60 con 0x7f9658103b30 2024-09-11T01:17:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.084+0000 7f9654ff9640 1 --2- 172.21.15.78:0/3755021584 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f96400b54d0 0x7f96400b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.085+0000 7f9654ff9640 1 -- 172.21.15.78:0/3755021584 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7f96480e5cb0 con 0x7f9658103b30 2024-09-11T01:17:17.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.085+0000 7f9660067740 1 Processor -- start 2024-09-11T01:17:17.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.085+0000 7f9656ffd640 1 --2- 172.21.15.78:0/3755021584 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f96400b54d0 0x7f96400b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.085+0000 7f9660067740 1 -- start start 2024-09-11T01:17:17.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.085+0000 7f9660067740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7d2ee0 0x55dcac7d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.085+0000 7f9660067740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d3800 0x55dcac7df250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.085+0000 7f9660067740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 0x55dcac7e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.085+0000 7f9657fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 0x55dcac7e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.085+0000 7f96577fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7d2ee0 0x55dcac7d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.085+0000 7f96577fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7d2ee0 0x55dcac7d32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:51464/0 (socket says 172.21.15.78:51464) 2024-09-11T01:17:17.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.085+0000 7f96577fe640 1 -- 172.21.15.78:0/4293527 learned_addr learned my addr 172.21.15.78:0/4293527 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:17.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.086+0000 7f9660067740 1 -- 172.21.15.78:0/4293527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dcac749e60 con 0x55dcac7df790 2024-09-11T01:17:17.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.086+0000 7f9660067740 1 -- 172.21.15.78:0/4293527 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dcac739290 con 0x55dcac7d2ee0 2024-09-11T01:17:17.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.086+0000 7f9656ffd640 1 --2- 172.21.15.78:0/4293527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d3800 0x55dcac7df250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.086+0000 7f9660067740 1 -- 172.21.15.78:0/4293527 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dcac73d0a0 con 0x55dcac7d3800 2024-09-11T01:17:17.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.086+0000 7f9656ffd640 1 --2- 172.21.15.78:0/3755021584 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f96400b54d0 0x7f96400b7990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f964c0058c0 tx=0x7f964c005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:17.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.087+0000 7f9657fff640 1 -- 172.21.15.78:0/4293527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d3800 msgr2=0x55dcac7df250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:17.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.087+0000 7f9657fff640 1 --2- 172.21.15.78:0/4293527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d3800 0x55dcac7df250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.087+0000 7f9657fff640 1 -- 172.21.15.78:0/4293527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7d2ee0 msgr2=0x55dcac7d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:17.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.087+0000 7f9657fff640 1 --2- 172.21.15.78:0/4293527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7d2ee0 0x55dcac7d32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.087+0000 7f9657fff640 1 -- 172.21.15.78:0/4293527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcac497930 con 0x55dcac7df790 2024-09-11T01:17:17.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.087+0000 7f9657fff640 1 --2- 172.21.15.78:0/4293527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 0x55dcac7e1b80 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f965002acb0 tx=0x7f9650030410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdc1dc77adf1804 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:17.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.087+0000 7f963dffb640 1 -- 172.21.15.78:0/4293527 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650040020 con 0x55dcac7df790 2024-09-11T01:17:17.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.087+0000 7f963dffb640 1 -- 172.21.15.78:0/4293527 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9650030780 con 0x55dcac7df790 2024-09-11T01:17:17.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.087+0000 7f963dffb640 1 -- 172.21.15.78:0/4293527 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650036040 con 0x55dcac7df790 2024-09-11T01:17:17.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.088+0000 7f9660067740 1 -- 172.21.15.78:0/4293527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 msgr2=0x55dcac7e1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:17.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.088+0000 7f9660067740 1 --2- 172.21.15.78:0/4293527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 0x55dcac7e1b80 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f965002acb0 tx=0x7f9650030410 comp rx=0 tx=0).stop 2024-09-11T01:17:17.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.088+0000 7f9660067740 1 -- 172.21.15.78:0/4293527 shutdown_connections 2024-09-11T01:17:17.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.088+0000 7f9660067740 1 --2- 172.21.15.78:0/4293527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 0x55dcac7e1b80 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.088+0000 7f9660067740 1 --2- 172.21.15.78:0/4293527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d3800 0x55dcac7df250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.088+0000 7f9660067740 1 --2- 172.21.15.78:0/4293527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7d2ee0 0x55dcac7d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.088+0000 7f9660067740 1 -- 172.21.15.78:0/4293527 >> 172.21.15.78:0/4293527 conn(0x55dcac7d0d10 msgr2=0x55dcac7d1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:17.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.089+0000 7f9660067740 1 -- 172.21.15.78:0/4293527 shutdown_connections 2024-09-11T01:17:17.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.089+0000 7f9660067740 1 -- 172.21.15.78:0/4293527 wait complete. 2024-09-11T01:17:17.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.089+0000 7f9660067740 1 -- start start 2024-09-11T01:17:17.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.089+0000 7f9660067740 1 Processor -- start 2024-09-11T01:17:17.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.090+0000 7f9660067740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 0x55dcac7dc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.090+0000 7f9660067740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 0x55dcac7dccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.090+0000 7f9660067740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7df790 0x55dcac7dd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.090+0000 7f9656ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 0x55dcac7dccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.090+0000 7f9656ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 0x55dcac7dccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49474/0 (socket says 172.21.15.78:49474) 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.090+0000 7f9656ffd640 1 -- 172.21.15.78:0/111111192 learned_addr learned my addr 172.21.15.78:0/111111192 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.090+0000 7f96577fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 0x55dcac7dc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.090+0000 7f9660067740 1 -- 172.21.15.78:0/111111192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dcac59a890 con 0x55dcac7d3800 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.090+0000 7f9660067740 1 -- 172.21.15.78:0/111111192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dcac6d3d40 con 0x55dcac7df790 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.090+0000 7f9660067740 1 -- 172.21.15.78:0/111111192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dcac7e23e0 con 0x55dcac7d2ee0 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.090+0000 7f9657fff640 1 --2- 172.21.15.78:0/111111192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7df790 0x55dcac7dd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.091+0000 7f9660067740 10 client.0 _ll_register_callbacks cb 0x55dcac737c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.091+0000 7f9656ffd640 1 -- 172.21.15.78:0/111111192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 msgr2=0x55dcac7dc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.091+0000 7f9656ffd640 1 --2- 172.21.15.78:0/111111192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 0x55dcac7dc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.091+0000 7f9656ffd640 1 -- 172.21.15.78:0/111111192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7df790 msgr2=0x55dcac7dd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:17.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.091+0000 7f9656ffd640 1 --2- 172.21.15.78:0/111111192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7df790 0x55dcac7dd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.091+0000 7f9656ffd640 1 -- 172.21.15.78:0/111111192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcac7eb810 con 0x55dcac7d3800 2024-09-11T01:17:17.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.091+0000 7f9656ffd640 1 --2- 172.21.15.78:0/111111192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 0x55dcac7dccc0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f964c01d7b0 tx=0x7f964c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca0ec2f1f640708 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:17.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.091+0000 7f9636ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:17:17.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.092+0000 7f9636ffd640 1 -- 172.21.15.78:0/111111192 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c02a060 con 0x55dcac7d3800 2024-09-11T01:17:17.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.092+0000 7f9636ffd640 1 -- 172.21.15.78:0/111111192 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7f964c010dc0 con 0x55dcac7d3800 2024-09-11T01:17:17.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.092+0000 7f9660067740 1 -- 172.21.15.78:0/111111192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dcac74bdb0 con 0x55dcac7d3800 2024-09-11T01:17:17.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.092+0000 7f9636ffd640 1 -- 172.21.15.78:0/111111192 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f964c02f470 con 0x55dcac7d3800 2024-09-11T01:17:17.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.092+0000 7f9660067740 10 client.18738 fetch_fsmap learned FSMap version 70 2024-09-11T01:17:17.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.093+0000 7f9660067740 1 -- 172.21.15.78:0/111111192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55dcac7ee220 con 0x55dcac7d3800 2024-09-11T01:17:17.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.093+0000 7f9636ffd640 1 -- 172.21.15.78:0/111111192 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dcac74bdb0 con 0x55dcac7d3800 2024-09-11T01:17:17.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.093+0000 7f9636ffd640 1 -- 172.21.15.78:0/111111192 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f964c024030 con 0x55dcac7d3800 2024-09-11T01:17:17.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.093+0000 7f9660067740 10 client.18738 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:17:17.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.093+0000 7f9660067740 10 client.18738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:17:17.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.093+0000 7f9660067740 20 client.18738 populate_metadata read hostname 'smithi078' 2024-09-11T01:17:17.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.093+0000 7f9660067740 1 --2- 172.21.15.78:0/111111192 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dcac7efb20 0x55dcac7f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.094+0000 7f9660067740 4 client.18738 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:17:17.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.094+0000 7f9660067740 1 -- 172.21.15.78:0/111111192 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcac747860 con 0x55dcac7efb20 2024-09-11T01:17:17.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.094+0000 7f96577fe640 1 --2- 172.21.15.78:0/111111192 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dcac7efb20 0x55dcac7f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.094+0000 7f96577fe640 1 --2- 172.21.15.78:0/111111192 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dcac7efb20 0x55dcac7f1fe0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7bd37ad50335ed9 server_cookie=c06a047ddef7acd in_seq=0 out_seq=0 2024-09-11T01:17:17.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.094+0000 7f9636ffd640 10 client.18738 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:17.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.096+0000 7f9636ffd640 1 -- 172.21.15.78:0/111111192 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dcac747860 con 0x55dcac7efb20 2024-09-11T01:17:17.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.096+0000 7f9636ffd640 10 client.18738 handle_command_reply: tid=0 2024-09-11T01:17:17.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.096+0000 7f9660067740 1 client.18738 shutdown 2024-09-11T01:17:17.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.096+0000 7f9660067740 1 -- 172.21.15.78:0/111111192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 msgr2=0x55dcac7dccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:17.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.096+0000 7f9660067740 1 --2- 172.21.15.78:0/111111192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 0x55dcac7dccc0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f964c01d7b0 tx=0x7f964c0046c0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.097+0000 7f9660067740 1 -- 172.21.15.78:0/111111192 shutdown_connections 2024-09-11T01:17:17.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.097+0000 7f9660067740 1 --2- 172.21.15.78:0/111111192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7df790 0x55dcac7dd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.097+0000 7f9660067740 1 --2- 172.21.15.78:0/111111192 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dcac7efb20 0x55dcac7f1fe0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.097+0000 7f9636ffd640 0 client.18738 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:17.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.097+0000 7f9660067740 1 --2- 172.21.15.78:0/111111192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 0x55dcac7dccc0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.097+0000 7f9660067740 1 --2- 172.21.15.78:0/111111192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 0x55dcac7dc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.097+0000 7f9660067740 1 -- 172.21.15.78:0/111111192 >> 172.21.15.78:0/111111192 conn(0x55dcac7d0d10 msgr2=0x55dcac7cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:17.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.097+0000 7f9660067740 1 -- 172.21.15.78:0/111111192 shutdown_connections 2024-09-11T01:17:17.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.098+0000 7f9660067740 1 -- 172.21.15.78:0/111111192 wait complete. 2024-09-11T01:17:17.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.098+0000 7f9660067740 20 client.18738 trim_cache size 0 max 16384 2024-09-11T01:17:17.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.128+0000 7f9660067740 1 Processor -- start 2024-09-11T01:17:17.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.128+0000 7f9660067740 1 -- start start 2024-09-11T01:17:17.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.129+0000 7f9660067740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 0x55dcac7de4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.129+0000 7f9660067740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7d3800 0x55dcac7d9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.129+0000 7f9660067740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 0x55dcac7d9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.129+0000 7f9660067740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96581b4b40 con 0x55dcac7df790 2024-09-11T01:17:17.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.129+0000 7f9660067740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f96581b6680 con 0x55dcac7d3800 2024-09-11T01:17:17.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.129+0000 7f9660067740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96581b5210 con 0x55dcac7d2ee0 2024-09-11T01:17:17.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.129+0000 7f9657fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 0x55dcac7d9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.129+0000 7f9657fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 0x55dcac7d9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49482/0 (socket says 172.21.15.78:49482) 2024-09-11T01:17:17.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.129+0000 7f9657fff640 1 -- 172.21.15.78:0/4040359556 learned_addr learned my addr 172.21.15.78:0/4040359556 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.130+0000 7f9657fff640 1 -- 172.21.15.78:0/4040359556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 msgr2=0x55dcac7de4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:17:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.130+0000 7f9657fff640 1 --2- 172.21.15.78:0/4040359556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 0x55dcac7de4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.130+0000 7f9657fff640 1 -- 172.21.15.78:0/4040359556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7d3800 msgr2=0x55dcac7d9990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:17:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.130+0000 7f9657fff640 1 --2- 172.21.15.78:0/4040359556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7d3800 0x55dcac7d9990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.130+0000 7f9657fff640 1 -- 172.21.15.78:0/4040359556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcac7db6e0 con 0x55dcac7df790 2024-09-11T01:17:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.130+0000 7f9657fff640 1 --2- 172.21.15.78:0/4040359556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 0x55dcac7d9ed0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9650009be0 tx=0x7f9650004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c51750c3e7edf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.131+0000 7f963d7fa640 1 -- 172.21.15.78:0/4040359556 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965004e070 con 0x55dcac7df790 2024-09-11T01:17:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.131+0000 7f963d7fa640 1 -- 172.21.15.78:0/4040359556 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9650003970 con 0x55dcac7df790 2024-09-11T01:17:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.131+0000 7f963d7fa640 1 -- 172.21.15.78:0/4040359556 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650040070 con 0x55dcac7df790 2024-09-11T01:17:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.132+0000 7f9660067740 1 -- 172.21.15.78:0/4040359556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 msgr2=0x55dcac7d9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.132+0000 7f9660067740 1 --2- 172.21.15.78:0/4040359556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 0x55dcac7d9ed0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9650009be0 tx=0x7f9650004a80 comp rx=0 tx=0).stop 2024-09-11T01:17:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.132+0000 7f9660067740 1 -- 172.21.15.78:0/4040359556 shutdown_connections 2024-09-11T01:17:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.132+0000 7f9660067740 1 --2- 172.21.15.78:0/4040359556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7df790 0x55dcac7d9ed0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.132+0000 7f9660067740 1 --2- 172.21.15.78:0/4040359556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7d3800 0x55dcac7d9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.132+0000 7f9660067740 1 --2- 172.21.15.78:0/4040359556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 0x55dcac7de4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.132+0000 7f9660067740 1 -- 172.21.15.78:0/4040359556 >> 172.21.15.78:0/4040359556 conn(0x55dcac7d0d10 msgr2=0x55dcac7dddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:17.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.136+0000 7f9660067740 1 -- 172.21.15.78:0/4040359556 shutdown_connections 2024-09-11T01:17:17.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.137+0000 7f9660067740 1 -- 172.21.15.78:0/4040359556 wait complete. 2024-09-11T01:17:17.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.137+0000 7f9660067740 1 -- start start 2024-09-11T01:17:17.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.137+0000 7f9660067740 1 Processor -- start 2024-09-11T01:17:17.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.138+0000 7f9660067740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 0x55dcac7f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.138+0000 7f9660067740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 0x55dcac7f4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.138+0000 7f9660067740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7df790 0x55dcac85b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.138+0000 7f9660067740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96581b5fb0 con 0x55dcac7d3800 2024-09-11T01:17:17.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f9660067740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f96581b58e0 con 0x55dcac7df790 2024-09-11T01:17:17.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f96577fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 0x55dcac7f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f9657fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7df790 0x55dcac85b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f96577fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 0x55dcac7f3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:36526/0 (socket says 172.21.15.78:36526) 2024-09-11T01:17:17.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f96577fe640 1 -- 172.21.15.78:0/2303691850 learned_addr learned my addr 172.21.15.78:0/2303691850 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:17.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f9660067740 1 -- 172.21.15.78:0/2303691850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96581b2b40 con 0x55dcac7d2ee0 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f9660067740 10 client.0 _ll_register_callbacks cb 0x7f964c01d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f9656ffd640 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 0x55dcac7f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f9657fff640 1 -- 172.21.15.78:0/2303691850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 msgr2=0x55dcac7f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f9657fff640 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 0x55dcac7f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f9657fff640 1 -- 172.21.15.78:0/2303691850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 msgr2=0x55dcac7f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f9657fff640 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 0x55dcac7f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f9657fff640 1 -- 172.21.15.78:0/2303691850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcac85bfd0 con 0x55dcac7df790 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.139+0000 7f9656ffd640 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 0x55dcac7f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.140+0000 7f9657fff640 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7df790 0x55dcac85b890 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f965002abe0 tx=0x7f9650009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab3d45c15a6aeb46 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.140+0000 7f9636ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.140+0000 7f9636ffd640 1 -- 172.21.15.78:0/2303691850 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965004e070 con 0x55dcac7df790 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.140+0000 7f9636ffd640 1 -- 172.21.15.78:0/2303691850 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7f965003c880 con 0x55dcac7df790 2024-09-11T01:17:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.140+0000 7f9660067740 1 -- 172.21.15.78:0/2303691850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9650003970 con 0x55dcac7df790 2024-09-11T01:17:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.140+0000 7f9636ffd640 1 -- 172.21.15.78:0/2303691850 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f9650040070 con 0x55dcac7df790 2024-09-11T01:17:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.142+0000 7f9660067740 10 client.27845 fetch_fsmap learned FSMap version 70 2024-09-11T01:17:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.142+0000 7f9660067740 1 -- 172.21.15.78:0/2303691850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55dcac85d2d0 con 0x55dcac7df790 2024-09-11T01:17:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.143+0000 7f9636ffd640 1 -- 172.21.15.78:0/2303691850 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9650031440 con 0x55dcac7df790 2024-09-11T01:17:17.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.143+0000 7f9636ffd640 1 -- 172.21.15.78:0/2303691850 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f9650046430 con 0x55dcac7df790 2024-09-11T01:17:17.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.143+0000 7f9660067740 10 client.27845 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:17:17.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.143+0000 7f9660067740 10 client.27845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:17:17.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.143+0000 7f9660067740 20 client.27845 populate_metadata read hostname 'smithi078' 2024-09-11T01:17:17.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.143+0000 7f9660067740 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dcac7efb20 0x55dcac85f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:17.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.143+0000 7f9660067740 4 client.27845 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:17:17.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.143+0000 7f9660067740 1 -- 172.21.15.78:0/2303691850 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f9650003970 con 0x55dcac7efb20 2024-09-11T01:17:17.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.143+0000 7f96577fe640 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dcac7efb20 0x55dcac85f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:17.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.145+0000 7f96577fe640 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dcac7efb20 0x55dcac85f4b0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bb0d5a13e712c4e server_cookie=72384d5dd3df3494 in_seq=0 out_seq=0 2024-09-11T01:17:17.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.145+0000 7f9636ffd640 10 client.27845 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:17.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.146+0000 7f9636ffd640 1 -- 172.21.15.78:0/2303691850 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f9650003970 con 0x55dcac7efb20 2024-09-11T01:17:17.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.146+0000 7f9636ffd640 10 client.27845 handle_command_reply: tid=0 2024-09-11T01:17:17.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.146+0000 7f9660067740 1 client.27845 shutdown 2024-09-11T01:17:17.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.147+0000 7f9660067740 1 -- 172.21.15.78:0/2303691850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7df790 msgr2=0x55dcac85b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:17.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.147+0000 7f9660067740 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7df790 0x55dcac85b890 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f965002abe0 tx=0x7f9650009520 comp rx=0 tx=0).stop 2024-09-11T01:17:17.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.147+0000 7f9660067740 1 -- 172.21.15.78:0/2303691850 shutdown_connections 2024-09-11T01:17:17.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.147+0000 7f9660067740 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcac7df790 0x55dcac85b890 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.147+0000 7f9660067740 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dcac7efb20 0x55dcac85f4b0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.147+0000 7f9660067740 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dcac7d3800 0x55dcac7f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.147+0000 7f9660067740 1 --2- 172.21.15.78:0/2303691850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcac7d2ee0 0x55dcac7f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.147+0000 7f9636ffd640 0 client.27845 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:17.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.147+0000 7f9660067740 1 -- 172.21.15.78:0/2303691850 >> 172.21.15.78:0/2303691850 conn(0x55dcac7d0d10 msgr2=0x55dcac7cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:17.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.147+0000 7f9660067740 1 -- 172.21.15.78:0/2303691850 shutdown_connections 2024-09-11T01:17:17.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.148+0000 7f9660067740 1 -- 172.21.15.78:0/2303691850 wait complete. 2024-09-11T01:17:17.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.148+0000 7f9660067740 20 client.27845 trim_cache size 0 max 16384 2024-09-11T01:17:17.151 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:17:17.151 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:17:17.151 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "21c0a2c6-e49f-4cac-b354-bcd7d75438b4", 2024-09-11T01:17:17.151 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:17:17.151 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:17:17.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.150+0000 7f965dc92640 1 -- 172.21.15.78:0/3755021584 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f96400b54d0 msgr2=0x7f96400b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:17.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.150+0000 7f965dc92640 1 --2- 172.21.15.78:0/3755021584 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f96400b54d0 0x7f96400b7990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f964c0058c0 tx=0x7f964c005850 comp rx=0 tx=0).stop 2024-09-11T01:17:17.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.150+0000 7f965dc92640 1 -- 172.21.15.78:0/3755021584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9658103b30 msgr2=0x7f9658103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:17.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.150+0000 7f965dc92640 1 --2- 172.21.15.78:0/3755021584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9658103b30 0x7f9658103460 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f964800c890 tx=0x7f964800cd50 comp rx=0 tx=0).stop 2024-09-11T01:17:17.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.150+0000 7f965dc92640 1 -- 172.21.15.78:0/3755021584 shutdown_connections 2024-09-11T01:17:17.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.150+0000 7f965dc92640 1 --2- 172.21.15.78:0/3755021584 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f96400b54d0 0x7f96400b7990 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.150+0000 7f965dc92640 1 --2- 172.21.15.78:0/3755021584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f965810d0e0 0x7f9658101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.150+0000 7f965dc92640 1 --2- 172.21.15.78:0/3755021584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9658104530 0x7f96581019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.150+0000 7f965dc92640 1 --2- 172.21.15.78:0/3755021584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9658103b30 0x7f9658103460 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:17.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.150+0000 7f965dc92640 1 -- 172.21.15.78:0/3755021584 >> 172.21.15.78:0/3755021584 conn(0x7f96580fd700 msgr2=0x7f965810f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:17.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.152+0000 7f965dc92640 1 -- 172.21.15.78:0/3755021584 shutdown_connections 2024-09-11T01:17:17.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:17.152+0000 7f965dc92640 1 -- 172.21.15.78:0/3755021584 wait complete. 2024-09-11T01:17:17.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 211238 34.81 MB/sec execute 203 sec latency 629.591 ms 2024-09-11T01:17:17.929 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:17 smithi078 ceph-mon[31315]: pgmap v899: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 715 op/s 2024-09-11T01:17:17.930 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:17 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:17:17.930 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:17 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:17.930 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:17 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:17:17.930 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:17 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:17:17.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:17 smithi110 ceph-mon[34712]: pgmap v899: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 715 op/s 2024-09-11T01:17:17.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:17 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:17:17.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:17 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:17.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:17 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:17:17.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:17 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:17:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:17 smithi112 ceph-mon[35384]: pgmap v899: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 715 op/s 2024-09-11T01:17:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:17 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:17:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:17 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:17 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:17:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:17 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:17:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:17:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:17:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:17:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:18.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 211801 34.79 MB/sec execute 204 sec latency 672.090 ms 2024-09-11T01:17:18.805 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:17:18.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:17:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:17:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:17:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:17:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:17:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:17:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:19.160 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:17:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.766+0000 7f4560440640 1 -- 172.21.15.78:0/743214400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4550097e60 msgr2=0x7f4550098240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:19.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.766+0000 7f4560440640 1 --2- 172.21.15.78:0/743214400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4550097e60 0x7f4550098240 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4548009900 tx=0x7f454802a400 comp rx=0 tx=0).stop 2024-09-11T01:17:19.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.767+0000 7f4560440640 1 -- 172.21.15.78:0/743214400 shutdown_connections 2024-09-11T01:17:19.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.767+0000 7f4560440640 1 --2- 172.21.15.78:0/743214400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f455009abb0 0x7f45500a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.767+0000 7f4560440640 1 --2- 172.21.15.78:0/743214400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f455009a230 0x7f455009a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.767+0000 7f4560440640 1 --2- 172.21.15.78:0/743214400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4550097e60 0x7f4550098240 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.767+0000 7f4560440640 1 -- 172.21.15.78:0/743214400 >> 172.21.15.78:0/743214400 conn(0x7f455000e5a0 msgr2=0x7f455000fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.767+0000 7f4560440640 1 -- 172.21.15.78:0/743214400 shutdown_connections 2024-09-11T01:17:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.767+0000 7f4560440640 1 -- 172.21.15.78:0/743214400 wait complete. 2024-09-11T01:17:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.768+0000 7f4560440640 1 Processor -- start 2024-09-11T01:17:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.768+0000 7f4560440640 1 -- start start 2024-09-11T01:17:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f4560440640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4550097e60 0x7f45500a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f4560440640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f455009a230 0x7f45500a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f4560440640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f455009abb0 0x7f4550137bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f4560440640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f455000b3b0 con 0x7f4550097e60 2024-09-11T01:17:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f4560440640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f455000b230 con 0x7f455009a230 2024-09-11T01:17:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f4560440640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f455000b530 con 0x7f455009abb0 2024-09-11T01:17:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f455e1dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4550097e60 0x7f45500a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f455e1dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4550097e60 0x7f45500a3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49506/0 (socket says 172.21.15.78:49506) 2024-09-11T01:17:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f455e1dc640 1 -- 172.21.15.78:0/3566492865 learned_addr learned my addr 172.21.15.78:0/3566492865 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f455d9db640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f455009a230 0x7f45500a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f455e1dc640 1 -- 172.21.15.78:0/3566492865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f455009abb0 msgr2=0x7f4550137bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T01:17:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f455e1dc640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f455009abb0 0x7f4550137bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f455e1dc640 1 -- 172.21.15.78:0/3566492865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f455009a230 msgr2=0x7f45500a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f455e1dc640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f455009a230 0x7f45500a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.769+0000 7f455e1dc640 1 -- 172.21.15.78:0/3566492865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4550138380 con 0x7f4550097e60 2024-09-11T01:17:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.770+0000 7f455e1dc640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4550097e60 0x7f45500a3ad0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f4548005940 tx=0x7f45480040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.770+0000 7f454f7fe640 1 -- 172.21.15.78:0/3566492865 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454803f030 con 0x7f4550097e60 2024-09-11T01:17:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.770+0000 7f454f7fe640 1 -- 172.21.15.78:0/3566492865 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45480044c0 con 0x7f4550097e60 2024-09-11T01:17:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.770+0000 7f454f7fe640 1 -- 172.21.15.78:0/3566492865 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548035070 con 0x7f4550097e60 2024-09-11T01:17:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.771+0000 7f4560440640 1 -- 172.21.15.78:0/3566492865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4550138610 con 0x7f4550097e60 2024-09-11T01:17:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.771+0000 7f4560440640 1 -- 172.21.15.78:0/3566492865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455013ed50 con 0x7f4550097e60 2024-09-11T01:17:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.777+0000 7f4562815740 1 Processor -- start 2024-09-11T01:17:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.778+0000 7f4562815740 1 -- start start 2024-09-11T01:17:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.778+0000 7f454f7fe640 1 -- 172.21.15.78:0/3566492865 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4548003c80 con 0x7f4550097e60 2024-09-11T01:17:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f4562815740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e791b280 0x55b8e791b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f4562815740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791bba0 0x55b8e79275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f4562815740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e7927b30 0x55b8e7929f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f4562815740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b8e7891e50 con 0x55b8e791bba0 2024-09-11T01:17:19.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f4562815740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b8e7881280 con 0x55b8e791b280 2024-09-11T01:17:19.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f4562815740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b8e7885090 con 0x55b8e7927b30 2024-09-11T01:17:19.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f455e1dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791bba0 0x55b8e79275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f455e9dd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e791b280 0x55b8e791b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f455e1dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791bba0 0x55b8e79275f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49520/0 (socket says 172.21.15.78:49520) 2024-09-11T01:17:19.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f455e1dc640 1 -- 172.21.15.78:0/2838393930 learned_addr learned my addr 172.21.15.78:0/2838393930 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:19.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f455d9db640 1 --2- 172.21.15.78:0/2838393930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e7927b30 0x55b8e7929f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f455e1dc640 1 -- 172.21.15.78:0/2838393930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e7927b30 msgr2=0x55b8e7929f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:19.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f455e1dc640 1 --2- 172.21.15.78:0/2838393930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e7927b30 0x55b8e7929f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.779+0000 7f455e1dc640 1 -- 172.21.15.78:0/2838393930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e791b280 msgr2=0x55b8e791b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.780+0000 7f454f7fe640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f45440b5620 0x7f45440b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.780+0000 7f454f7fe640 1 -- 172.21.15.78:0/3566492865 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7f4548106240 con 0x7f4550097e60 2024-09-11T01:17:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.780+0000 7f455e1dc640 1 --2- 172.21.15.78:0/2838393930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e791b280 0x55b8e791b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.780+0000 7f455d9db640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f45440b5620 0x7f45440b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.781+0000 7f455e1dc640 1 -- 172.21.15.78:0/2838393930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8e75df650 con 0x55b8e791bba0 2024-09-11T01:17:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.781+0000 7f455e1dc640 1 --2- 172.21.15.78:0/2838393930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791bba0 0x55b8e79275f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f45481038b0 tx=0x7f4548005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23aa7b1a8c491bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.781+0000 7f455d9db640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f45440b5620 0x7f45440b7ae0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f455009d630 tx=0x7f4554005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.781+0000 7f454cde6640 1 -- 172.21.15.78:0/2838393930 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45480c8540 con 0x55b8e791bba0 2024-09-11T01:17:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.781+0000 7f454cde6640 1 -- 172.21.15.78:0/2838393930 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45480cc1b0 con 0x55b8e791bba0 2024-09-11T01:17:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.781+0000 7f454cde6640 1 -- 172.21.15.78:0/2838393930 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548109d60 con 0x55b8e791bba0 2024-09-11T01:17:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.782+0000 7f4562815740 1 -- 172.21.15.78:0/2838393930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791bba0 msgr2=0x55b8e79275f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.782+0000 7f4562815740 1 --2- 172.21.15.78:0/2838393930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791bba0 0x55b8e79275f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f45481038b0 tx=0x7f4548005a20 comp rx=0 tx=0).stop 2024-09-11T01:17:19.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.783+0000 7f4562815740 1 -- 172.21.15.78:0/2838393930 shutdown_connections 2024-09-11T01:17:19.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.783+0000 7f4562815740 1 --2- 172.21.15.78:0/2838393930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e7927b30 0x55b8e7929f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.783+0000 7f4562815740 1 --2- 172.21.15.78:0/2838393930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791bba0 0x55b8e79275f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.783+0000 7f4562815740 1 --2- 172.21.15.78:0/2838393930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e791b280 0x55b8e791b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.783+0000 7f4562815740 1 -- 172.21.15.78:0/2838393930 >> 172.21.15.78:0/2838393930 conn(0x55b8e79190b0 msgr2=0x55b8e791a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:19.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.783+0000 7f4562815740 1 -- 172.21.15.78:0/2838393930 shutdown_connections 2024-09-11T01:17:19.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.783+0000 7f4562815740 1 -- 172.21.15.78:0/2838393930 wait complete. 2024-09-11T01:17:19.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.784+0000 7f4562815740 1 -- start start 2024-09-11T01:17:19.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.784+0000 7f4562815740 1 Processor -- start 2024-09-11T01:17:19.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.784+0000 7f4562815740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e791b280 0x55b8e79274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.785+0000 7f4562815740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791bba0 0x55b8e7924910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.785+0000 7f4562815740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e7927b30 0x55b8e7924fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.785+0000 7f4562815740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b8e76e2700 con 0x55b8e791bba0 2024-09-11T01:17:19.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.785+0000 7f4562815740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b8e781c110 con 0x55b8e7927b30 2024-09-11T01:17:19.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.785+0000 7f4562815740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b8e792a570 con 0x55b8e791b280 2024-09-11T01:17:19.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.785+0000 7f4562815740 10 client.0 _ll_register_callbacks cb 0x55b8e7835990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:17:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.785+0000 7f455e9dd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e7927b30 0x55b8e7924fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.785+0000 7f455e9dd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e7927b30 0x55b8e7924fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:51506/0 (socket says 172.21.15.78:51506) 2024-09-11T01:17:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.785+0000 7f455e9dd640 1 -- 172.21.15.78:0/3184430368 learned_addr learned my addr 172.21.15.78:0/3184430368 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.785+0000 7f455d9db640 1 --2- 172.21.15.78:0/3184430368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791bba0 0x55b8e7924910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.786+0000 7f455e9dd640 1 -- 172.21.15.78:0/3184430368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e791b280 msgr2=0x55b8e79274c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:17:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.786+0000 7f455e9dd640 1 --2- 172.21.15.78:0/3184430368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e791b280 0x55b8e79274c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.786+0000 7f455e9dd640 1 -- 172.21.15.78:0/3184430368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791bba0 msgr2=0x55b8e7924910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.786+0000 7f455e9dd640 1 --2- 172.21.15.78:0/3184430368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791bba0 0x55b8e7924910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.786+0000 7f455e9dd640 1 -- 172.21.15.78:0/3184430368 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8e7849c40 con 0x55b8e7927b30 2024-09-11T01:17:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.786+0000 7f455e9dd640 1 --2- 172.21.15.78:0/3184430368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e7927b30 0x55b8e7924fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4558051c90 tx=0x7f455806c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5daee8ab95d2ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.786+0000 7f4541ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:17:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.786+0000 7f4541ffb640 1 -- 172.21.15.78:0/3184430368 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455808a060 con 0x55b8e7927b30 2024-09-11T01:17:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.787+0000 7f4541ffb640 1 -- 172.21.15.78:0/3184430368 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7f4558085440 con 0x55b8e7927b30 2024-09-11T01:17:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.787+0000 7f4562815740 1 -- 172.21.15.78:0/3184430368 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45480cc1b0 con 0x55b8e7927b30 2024-09-11T01:17:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.787+0000 7f4541ffb640 1 -- 172.21.15.78:0/3184430368 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f4558077e50 con 0x55b8e7927b30 2024-09-11T01:17:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.787+0000 7f4562815740 10 client.27857 fetch_fsmap learned FSMap version 70 2024-09-11T01:17:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.787+0000 7f4562815740 1 -- 172.21.15.78:0/3184430368 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55b8e7935ed0 con 0x55b8e7927b30 2024-09-11T01:17:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.788+0000 7f4541ffb640 1 -- 172.21.15.78:0/3184430368 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45480cc1b0 con 0x55b8e7927b30 2024-09-11T01:17:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.788+0000 7f4541ffb640 1 -- 172.21.15.78:0/3184430368 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f455808e6a0 con 0x55b8e7927b30 2024-09-11T01:17:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.788+0000 7f4562815740 10 client.27857 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:17:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.788+0000 7f4562815740 10 client.27857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:17:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.788+0000 7f4562815740 20 client.27857 populate_metadata read hostname 'smithi078' 2024-09-11T01:17:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.788+0000 7f4562815740 1 --2- 172.21.15.78:0/3184430368 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b8e7937330 0x55b8e7939740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.788+0000 7f4562815740 4 client.27857 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:17:19.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.788+0000 7f4562815740 1 -- 172.21.15.78:0/3184430368 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8e7893da0 con 0x55b8e7937330 2024-09-11T01:17:19.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.788+0000 7f455e1dc640 1 --2- 172.21.15.78:0/3184430368 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b8e7937330 0x55b8e7939740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 212446 34.77 MB/sec execute 205 sec latency 88.098 ms 2024-09-11T01:17:19.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.789+0000 7f455e1dc640 1 --2- 172.21.15.78:0/3184430368 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b8e7937330 0x55b8e7939740 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee50ee7e4d7372a8 server_cookie=7c000d8ddef55c7c in_seq=0 out_seq=0 2024-09-11T01:17:19.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.789+0000 7f4541ffb640 10 client.27857 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:19.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.791+0000 7f4541ffb640 1 -- 172.21.15.78:0/3184430368 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b8e7893da0 con 0x55b8e7937330 2024-09-11T01:17:19.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.791+0000 7f4541ffb640 10 client.27857 handle_command_reply: tid=0 2024-09-11T01:17:19.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.791+0000 7f4562815740 1 client.27857 shutdown 2024-09-11T01:17:19.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.791+0000 7f4562815740 1 -- 172.21.15.78:0/3184430368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e7927b30 msgr2=0x55b8e7924fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:19.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.791+0000 7f4562815740 1 --2- 172.21.15.78:0/3184430368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e7927b30 0x55b8e7924fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4558051c90 tx=0x7f455806c8a0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.791+0000 7f4562815740 1 -- 172.21.15.78:0/3184430368 shutdown_connections 2024-09-11T01:17:19.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.791+0000 7f4562815740 1 --2- 172.21.15.78:0/3184430368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e7927b30 0x55b8e7924fe0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.791+0000 7f4562815740 1 --2- 172.21.15.78:0/3184430368 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b8e7937330 0x55b8e7939740 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.791+0000 7f4541ffb640 0 client.27857 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:19.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.791+0000 7f4562815740 1 --2- 172.21.15.78:0/3184430368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791bba0 0x55b8e7924910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.791+0000 7f4562815740 1 --2- 172.21.15.78:0/3184430368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e791b280 0x55b8e79274c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.791+0000 7f4562815740 1 -- 172.21.15.78:0/3184430368 >> 172.21.15.78:0/3184430368 conn(0x55b8e79190b0 msgr2=0x55b8e791ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:19.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.792+0000 7f4562815740 1 -- 172.21.15.78:0/3184430368 shutdown_connections 2024-09-11T01:17:19.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.792+0000 7f4562815740 1 -- 172.21.15.78:0/3184430368 wait complete. 2024-09-11T01:17:19.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.792+0000 7f4562815740 20 client.27857 trim_cache size 0 max 16384 2024-09-11T01:17:19.821 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:19 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:19.822 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:19 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:19.822 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:19 smithi078 ceph-mon[31315]: pgmap v900: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 705 op/s 2024-09-11T01:17:19.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f4562815740 1 Processor -- start 2024-09-11T01:17:19.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f4562815740 1 -- start start 2024-09-11T01:17:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f4562815740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791b280 0x55b8e7924f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f4562815740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e791bba0 0x55b8e79336d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f4562815740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e7927b30 0x55b8e7921b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f4562815740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4550147600 con 0x55b8e791b280 2024-09-11T01:17:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f4562815740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4550149140 con 0x55b8e791bba0 2024-09-11T01:17:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f4562815740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4550147cd0 con 0x55b8e7927b30 2024-09-11T01:17:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f455e1dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791b280 0x55b8e7924f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f455d9db640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e791bba0 0x55b8e79336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f455e1dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791b280 0x55b8e7924f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49540/0 (socket says 172.21.15.78:49540) 2024-09-11T01:17:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f455e1dc640 1 -- 172.21.15.78:0/4195049747 learned_addr learned my addr 172.21.15.78:0/4195049747 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.819+0000 7f455e9dd640 1 --2- 172.21.15.78:0/4195049747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e7927b30 0x55b8e7921b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.820+0000 7f455d9db640 1 -- 172.21.15.78:0/4195049747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e7927b30 msgr2=0x55b8e7921b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.820+0000 7f455d9db640 1 --2- 172.21.15.78:0/4195049747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e7927b30 0x55b8e7921b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.820+0000 7f455d9db640 1 -- 172.21.15.78:0/4195049747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791b280 msgr2=0x55b8e7924f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.820+0000 7f455d9db640 1 --2- 172.21.15.78:0/4195049747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791b280 0x55b8e7924f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.820+0000 7f455d9db640 1 -- 172.21.15.78:0/4195049747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8e7919d90 con 0x55b8e791bba0 2024-09-11T01:17:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.820+0000 7f455d9db640 1 --2- 172.21.15.78:0/4195049747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e791bba0 0x55b8e79336d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f45540128e0 tx=0x7f4554010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd3cbc88c84457e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.821+0000 7f4543fff640 1 -- 172.21.15.78:0/4195049747 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455401c8f0 con 0x55b8e791bba0 2024-09-11T01:17:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.822+0000 7f4543fff640 1 -- 172.21.15.78:0/4195049747 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f455401b460 con 0x55b8e791bba0 2024-09-11T01:17:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.822+0000 7f4543fff640 1 -- 172.21.15.78:0/4195049747 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554026630 con 0x55b8e791bba0 2024-09-11T01:17:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.823+0000 7f4562815740 1 -- 172.21.15.78:0/4195049747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e791bba0 msgr2=0x55b8e79336d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.823+0000 7f4562815740 1 --2- 172.21.15.78:0/4195049747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e791bba0 0x55b8e79336d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f45540128e0 tx=0x7f4554010000 comp rx=0 tx=0).stop 2024-09-11T01:17:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.823+0000 7f4562815740 1 -- 172.21.15.78:0/4195049747 shutdown_connections 2024-09-11T01:17:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.823+0000 7f4562815740 1 --2- 172.21.15.78:0/4195049747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e7927b30 0x55b8e7921b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.823+0000 7f4562815740 1 --2- 172.21.15.78:0/4195049747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e791bba0 0x55b8e79336d0 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f45540128e0 tx=0x7f4554010000 comp rx=0 tx=0).stop 2024-09-11T01:17:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.823+0000 7f4562815740 1 --2- 172.21.15.78:0/4195049747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e791b280 0x55b8e7924f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.823+0000 7f4562815740 1 -- 172.21.15.78:0/4195049747 >> 172.21.15.78:0/4195049747 conn(0x55b8e79190b0 msgr2=0x55b8e7924880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:19.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.823+0000 7f4562815740 1 -- 172.21.15.78:0/4195049747 shutdown_connections 2024-09-11T01:17:19.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.823+0000 7f4562815740 1 -- 172.21.15.78:0/4195049747 wait complete. 2024-09-11T01:17:19.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.824+0000 7f4562815740 1 -- start start 2024-09-11T01:17:19.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.824+0000 7f4562815740 1 Processor -- start 2024-09-11T01:17:19.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.825+0000 7f4562815740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e791b280 0x55b8e7930120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.825+0000 7f4562815740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e7927b30 0x55b8e792a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.825+0000 7f4562815740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e792ad50 0x55b8e79b5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.825+0000 7f455e1dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e791b280 0x55b8e7930120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.825+0000 7f455e1dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e791b280 0x55b8e7930120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:36580/0 (socket says 172.21.15.78:36580) 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.825+0000 7f455e1dc640 1 -- 172.21.15.78:0/1371949475 learned_addr learned my addr 172.21.15.78:0/1371949475 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.825+0000 7f4562815740 1 -- 172.21.15.78:0/1371949475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4550148a70 con 0x55b8e7927b30 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.825+0000 7f4562815740 1 -- 172.21.15.78:0/1371949475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f45501483a0 con 0x55b8e792ad50 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.825+0000 7f455e9dd640 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e792ad50 0x55b8e79b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.825+0000 7f4562815740 1 -- 172.21.15.78:0/1371949475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4550145600 con 0x55b8e791b280 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.825+0000 7f455d9db640 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e7927b30 0x55b8e792a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.826+0000 7f4562815740 10 client.0 _ll_register_callbacks cb 0x7f4558069370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.826+0000 7f455e9dd640 1 -- 172.21.15.78:0/1371949475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e791b280 msgr2=0x55b8e7930120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.826+0000 7f455e9dd640 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e791b280 0x55b8e7930120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.826+0000 7f455e9dd640 1 -- 172.21.15.78:0/1371949475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e7927b30 msgr2=0x55b8e792a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.826+0000 7f455e9dd640 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e7927b30 0x55b8e792a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.826+0000 7f455e9dd640 1 -- 172.21.15.78:0/1371949475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8e79b5760 con 0x55b8e792ad50 2024-09-11T01:17:19.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.826+0000 7f455e9dd640 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e792ad50 0x55b8e79b5220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4558068e50 tx=0x7f455806ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7b64701c1090427 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.826+0000 7f4562815740 1 -- 172.21.15.78:0/1371949475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f455401b460 con 0x55b8e792ad50 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.826+0000 7f4541ffb640 10 client.27863 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.826+0000 7f4541ffb640 1 -- 172.21.15.78:0/1371949475 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558094080 con 0x55b8e792ad50 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.826+0000 7f4541ffb640 1 -- 172.21.15.78:0/1371949475 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7f455808e5b0 con 0x55b8e792ad50 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.827+0000 7f4541ffb640 1 -- 172.21.15.78:0/1371949475 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f455808e7b0 con 0x55b8e792ad50 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.827+0000 7f4562815740 10 client.27863 fetch_fsmap learned FSMap version 70 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.827+0000 7f4562815740 1 -- 172.21.15.78:0/1371949475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=70}) -- 0x55b8e79b67c0 con 0x55b8e792ad50 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.827+0000 7f4541ffb640 1 -- 172.21.15.78:0/1371949475 <== mon.2 v2:172.21.15.110:3300/0 4 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f4558098440 con 0x55b8e792ad50 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.827+0000 7f4562815740 10 client.27863 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.827+0000 7f4562815740 10 client.27863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.827+0000 7f4562815740 20 client.27863 populate_metadata read hostname 'smithi078' 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.828+0000 7f4562815740 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b8e79b6ce0 0x55b8e79b9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:19.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.828+0000 7f4562815740 4 client.27863 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:17:19.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.828+0000 7f4562815740 1 -- 172.21.15.78:0/1371949475 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f455401b460 con 0x55b8e79b6ce0 2024-09-11T01:17:19.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.828+0000 7f455e1dc640 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b8e79b6ce0 0x55b8e79b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:19.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.828+0000 7f455e1dc640 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b8e79b6ce0 0x55b8e79b9100 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c27ce6c82259c95 server_cookie=e5260f2836c596bf in_seq=0 out_seq=0 2024-09-11T01:17:19.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.828+0000 7f4541ffb640 10 client.27863 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:19.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.831+0000 7f4541ffb640 1 -- 172.21.15.78:0/1371949475 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6818 (crc 0 0 0) 0x7f455401b460 con 0x55b8e79b6ce0 2024-09-11T01:17:19.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.831+0000 7f4541ffb640 10 client.27863 handle_command_reply: tid=0 2024-09-11T01:17:19.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.831+0000 7f4562815740 1 client.27863 shutdown 2024-09-11T01:17:19.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.831+0000 7f4562815740 1 -- 172.21.15.78:0/1371949475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e792ad50 msgr2=0x55b8e79b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:19.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.831+0000 7f4562815740 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e792ad50 0x55b8e79b5220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4558068e50 tx=0x7f455806ccc0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.831+0000 7f4562815740 1 -- 172.21.15.78:0/1371949475 shutdown_connections 2024-09-11T01:17:19.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.831+0000 7f4562815740 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b8e792ad50 0x55b8e79b5220 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.831+0000 7f4562815740 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55b8e79b6ce0 0x55b8e79b9100 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.831+0000 7f4541ffb640 0 client.27863 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:19.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.831+0000 7f4562815740 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8e7927b30 0x55b8e792a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.831+0000 7f4562815740 1 --2- 172.21.15.78:0/1371949475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b8e791b280 0x55b8e7930120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.831+0000 7f4562815740 1 -- 172.21.15.78:0/1371949475 >> 172.21.15.78:0/1371949475 conn(0x55b8e79190b0 msgr2=0x55b8e79179b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:19.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.832+0000 7f4562815740 1 -- 172.21.15.78:0/1371949475 shutdown_connections 2024-09-11T01:17:19.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.832+0000 7f4562815740 1 -- 172.21.15.78:0/1371949475 wait complete. 2024-09-11T01:17:19.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.832+0000 7f4562815740 20 client.27863 trim_cache size 0 max 16384 2024-09-11T01:17:19.836 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:17:19.836 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:17:19.836 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:17:19.836 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:17:19.836 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.836 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "21c0a2c6-e49f-4cac-b354-bcd7d75438b4", 2024-09-11T01:17:19.836 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:17:19.836 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:17:19.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.836 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:17:19.836 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.837 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:17:19.838 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.839 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.839 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.839 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.839 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.839 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:17:19.839 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:17:19.839 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.839 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.839 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.839 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.840 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.841 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.845 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.846 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.847 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.848 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:19.849 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:19.850 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:19.850 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:19.850 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:17:19.850 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:17:19.850 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:17:19.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.833+0000 7f4560440640 1 -- 172.21.15.78:0/3566492865 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f45440b5620 msgr2=0x7f45440b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:19.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.833+0000 7f4560440640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f45440b5620 0x7f45440b7ae0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f455009d630 tx=0x7f4554005f90 comp rx=0 tx=0).stop 2024-09-11T01:17:19.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.833+0000 7f4560440640 1 -- 172.21.15.78:0/3566492865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4550097e60 msgr2=0x7f45500a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:19.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.833+0000 7f4560440640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4550097e60 0x7f45500a3ad0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f4548005940 tx=0x7f45480040c0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.833+0000 7f4560440640 1 -- 172.21.15.78:0/3566492865 shutdown_connections 2024-09-11T01:17:19.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.833+0000 7f4560440640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f45440b5620 0x7f45440b7ae0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.833+0000 7f4560440640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f455009abb0 0x7f4550137bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.833+0000 7f4560440640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f455009a230 0x7f45500a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.833+0000 7f4560440640 1 --2- 172.21.15.78:0/3566492865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4550097e60 0x7f45500a3ad0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:19.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.833+0000 7f4560440640 1 -- 172.21.15.78:0/3566492865 >> 172.21.15.78:0/3566492865 conn(0x7f455000e5a0 msgr2=0x7f455000bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:19.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.833+0000 7f4560440640 1 -- 172.21.15.78:0/3566492865 shutdown_connections 2024-09-11T01:17:19.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:19.833+0000 7f4560440640 1 -- 172.21.15.78:0/3566492865 wait complete. 2024-09-11T01:17:19.946 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:19 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:19.946 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:19 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:19.946 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:19 smithi110 ceph-mon[34712]: pgmap v900: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 705 op/s 2024-09-11T01:17:19.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:19 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:19.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:19 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:17:19.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:19 smithi112 ceph-mon[35384]: pgmap v900: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 705 op/s 2024-09-11T01:17:20.737 INFO:tasks.cephfs.filesystem:scrub status for tag:21c0a2c6-e49f-4cac-b354-bcd7d75438b4 - {'path': '/', 'tag': '21c0a2c6-e49f-4cac-b354-bcd7d75438b4', 'options': 'recursive,force'} 2024-09-11T01:17:20.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 213230 34.82 MB/sec execute 206 sec latency 56.098 ms 2024-09-11T01:17:21.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 214020 34.84 MB/sec execute 207 sec latency 95.122 ms 2024-09-11T01:17:21.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:21 smithi110 ceph-mon[34712]: pgmap v901: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 673 op/s 2024-09-11T01:17:21.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:21 smithi112 ceph-mon[35384]: pgmap v901: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 673 op/s 2024-09-11T01:17:21.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:21 smithi078 ceph-mon[31315]: pgmap v901: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 673 op/s 2024-09-11T01:17:22.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 214686 34.86 MB/sec execute 208 sec latency 111.863 ms 2024-09-11T01:17:23.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 215428 34.90 MB/sec execute 209 sec latency 77.830 ms 2024-09-11T01:17:23.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:23 smithi110 ceph-mon[34712]: pgmap v902: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 654 op/s 2024-09-11T01:17:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:23 smithi112 ceph-mon[35384]: pgmap v902: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 654 op/s 2024-09-11T01:17:23.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:23 smithi078 ceph-mon[31315]: pgmap v902: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 654 op/s 2024-09-11T01:17:24.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 216047 34.88 MB/sec execute 210 sec latency 80.434 ms 2024-09-11T01:17:25.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 216853 34.94 MB/sec execute 211 sec latency 86.291 ms 2024-09-11T01:17:25.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:25 smithi110 ceph-mon[34712]: pgmap v903: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 713 op/s 2024-09-11T01:17:25.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:25 smithi112 ceph-mon[35384]: pgmap v903: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 713 op/s 2024-09-11T01:17:25.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:25 smithi078 ceph-mon[31315]: pgmap v903: 97 pgs: 97 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 713 op/s 2024-09-11T01:17:26.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 217537 34.91 MB/sec execute 212 sec latency 113.779 ms 2024-09-11T01:17:27.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 217743 34.77 MB/sec execute 213 sec latency 793.078 ms 2024-09-11T01:17:27.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:27 smithi110 ceph-mon[34712]: pgmap v904: 97 pgs: 97 active+clean; 2.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 16 MiB/s wr, 759 op/s 2024-09-11T01:17:27.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:27 smithi112 ceph-mon[35384]: pgmap v904: 97 pgs: 97 active+clean; 2.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 16 MiB/s wr, 759 op/s 2024-09-11T01:17:27.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:27 smithi078 ceph-mon[31315]: pgmap v904: 97 pgs: 97 active+clean; 2.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 16 MiB/s wr, 759 op/s 2024-09-11T01:17:28.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 218365 34.80 MB/sec execute 214 sec latency 229.340 ms 2024-09-11T01:17:29.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 219022 34.82 MB/sec execute 215 sec latency 145.507 ms 2024-09-11T01:17:29.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:29 smithi110 ceph-mon[34712]: pgmap v905: 97 pgs: 97 active+clean; 2.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 652 op/s 2024-09-11T01:17:29.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:29 smithi112 ceph-mon[35384]: pgmap v905: 97 pgs: 97 active+clean; 2.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 652 op/s 2024-09-11T01:17:29.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:29 smithi078 ceph-mon[31315]: pgmap v905: 97 pgs: 97 active+clean; 2.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 652 op/s 2024-09-11T01:17:30.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 219679 34.82 MB/sec execute 216 sec latency 75.486 ms 2024-09-11T01:17:31.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 220161 34.77 MB/sec execute 217 sec latency 347.165 ms 2024-09-11T01:17:31.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:31 smithi110 ceph-mon[34712]: pgmap v906: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 728 op/s 2024-09-11T01:17:31.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:31 smithi112 ceph-mon[35384]: pgmap v906: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 728 op/s 2024-09-11T01:17:31.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:31 smithi078 ceph-mon[31315]: pgmap v906: 97 pgs: 97 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 728 op/s 2024-09-11T01:17:32.791 INFO:tasks.workunit.client.0.smithi078.stdout: 10 220852 34.79 MB/sec execute 218 sec latency 78.533 ms 2024-09-11T01:17:32.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:32 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:17:32.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:32 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:17:32.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:32 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:17:32.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:32 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:17:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:32 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:17:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:32 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:17:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:32 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:17:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:32 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:17:32.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:32 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:17:32.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:32 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:17:32.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:32 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:17:32.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:32 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:17:33.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 221623 34.80 MB/sec execute 219 sec latency 107.196 ms 2024-09-11T01:17:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:17:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:17:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:33 smithi110 ceph-mon[34712]: pgmap v907: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 775 op/s 2024-09-11T01:17:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:17:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:17:33.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:17:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:17:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:17:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:33 smithi112 ceph-mon[35384]: pgmap v907: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 775 op/s 2024-09-11T01:17:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:17:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:17:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:17:33.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:17:33.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:17:33.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:33 smithi078 ceph-mon[31315]: pgmap v907: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 775 op/s 2024-09-11T01:17:33.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:17:33.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:17:33.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:17:34.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 222434 34.86 MB/sec execute 220 sec latency 82.101 ms 2024-09-11T01:17:35.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 223048 34.86 MB/sec execute 221 sec latency 117.921 ms 2024-09-11T01:17:35.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:35 smithi110 ceph-mon[34712]: pgmap v908: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 779 op/s 2024-09-11T01:17:35.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:35 smithi112 ceph-mon[35384]: pgmap v908: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 779 op/s 2024-09-11T01:17:35.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:35 smithi078 ceph-mon[31315]: pgmap v908: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 779 op/s 2024-09-11T01:17:36.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 223760 34.86 MB/sec execute 222 sec latency 62.872 ms 2024-09-11T01:17:37.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:37 smithi078 ceph-mon[31315]: pgmap v909: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 788 op/s 2024-09-11T01:17:37.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 224433 34.85 MB/sec execute 223 sec latency 94.633 ms 2024-09-11T01:17:37.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:37 smithi110 ceph-mon[34712]: pgmap v909: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 788 op/s 2024-09-11T01:17:37.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:37 smithi112 ceph-mon[35384]: pgmap v909: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 788 op/s 2024-09-11T01:17:38.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 225266 34.89 MB/sec execute 224 sec latency 62.231 ms 2024-09-11T01:17:39.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 225874 34.88 MB/sec execute 225 sec latency 417.805 ms 2024-09-11T01:17:39.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:39 smithi110 ceph-mon[34712]: pgmap v910: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 748 op/s 2024-09-11T01:17:39.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:39 smithi112 ceph-mon[35384]: pgmap v910: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 748 op/s 2024-09-11T01:17:39.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:39 smithi078 ceph-mon[31315]: pgmap v910: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 748 op/s 2024-09-11T01:17:40.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 226483 34.91 MB/sec execute 226 sec latency 113.623 ms 2024-09-11T01:17:41.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 227207 34.94 MB/sec execute 227 sec latency 79.934 ms 2024-09-11T01:17:41.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:41 smithi110 ceph-mon[34712]: pgmap v911: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 19 MiB/s wr, 802 op/s 2024-09-11T01:17:41.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:41 smithi112 ceph-mon[35384]: pgmap v911: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 19 MiB/s wr, 802 op/s 2024-09-11T01:17:41.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:41 smithi078 ceph-mon[31315]: pgmap v911: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 19 MiB/s wr, 802 op/s 2024-09-11T01:17:42.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 227647 34.89 MB/sec execute 228 sec latency 216.466 ms 2024-09-11T01:17:43.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 228334 34.84 MB/sec execute 229 sec latency 89.549 ms 2024-09-11T01:17:43.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:43 smithi110 ceph-mon[34712]: pgmap v912: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 763 op/s 2024-09-11T01:17:43.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:43 smithi112 ceph-mon[35384]: pgmap v912: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 763 op/s 2024-09-11T01:17:43.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:43 smithi078 ceph-mon[31315]: pgmap v912: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 763 op/s 2024-09-11T01:17:44.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 229235 34.91 MB/sec execute 230 sec latency 88.153 ms 2024-09-11T01:17:45.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 230019 34.97 MB/sec execute 231 sec latency 50.238 ms 2024-09-11T01:17:45.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:45 smithi110 ceph-mon[34712]: pgmap v913: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 735 op/s 2024-09-11T01:17:45.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:45 smithi112 ceph-mon[35384]: pgmap v913: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 735 op/s 2024-09-11T01:17:45.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:45 smithi078 ceph-mon[31315]: pgmap v913: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 735 op/s 2024-09-11T01:17:46.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 230597 34.99 MB/sec execute 232 sec latency 147.671 ms 2024-09-11T01:17:47.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 231125 34.94 MB/sec execute 233 sec latency 137.488 ms 2024-09-11T01:17:47.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:47 smithi110 ceph-mon[34712]: pgmap v914: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 740 op/s 2024-09-11T01:17:47.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:47 smithi112 ceph-mon[35384]: pgmap v914: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 740 op/s 2024-09-11T01:17:47.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:47 smithi078 ceph-mon[31315]: pgmap v914: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 740 op/s 2024-09-11T01:17:48.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 231766 34.90 MB/sec execute 234 sec latency 163.533 ms 2024-09-11T01:17:49.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 232640 34.95 MB/sec execute 235 sec latency 87.748 ms 2024-09-11T01:17:49.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:49 smithi110 ceph-mon[34712]: pgmap v915: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 703 op/s 2024-09-11T01:17:49.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:49 smithi112 ceph-mon[35384]: pgmap v915: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 703 op/s 2024-09-11T01:17:49.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:49 smithi078 ceph-mon[31315]: pgmap v915: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 703 op/s 2024-09-11T01:17:50.738 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:17:50.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 233477 35.03 MB/sec execute 236 sec latency 141.047 ms 2024-09-11T01:17:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:17:51.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 234092 35.01 MB/sec execute 237 sec latency 97.830 ms 2024-09-11T01:17:51.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:51 smithi110 ceph-mon[34712]: pgmap v916: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 697 op/s 2024-09-11T01:17:51.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:51 smithi112 ceph-mon[35384]: pgmap v916: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 697 op/s 2024-09-11T01:17:51.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:51 smithi078 ceph-mon[31315]: pgmap v916: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 697 op/s 2024-09-11T01:17:52.792 INFO:tasks.workunit.client.0.smithi078.stdout: 10 234691 35.00 MB/sec execute 238 sec latency 157.230 ms 2024-09-11T01:17:53.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 234789 34.88 MB/sec execute 239 sec latency 796.625 ms 2024-09-11T01:17:53.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:53 smithi110 ceph-mon[34712]: pgmap v917: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 690 op/s 2024-09-11T01:17:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:53 smithi112 ceph-mon[35384]: pgmap v917: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 690 op/s 2024-09-11T01:17:53.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:53 smithi078 ceph-mon[31315]: pgmap v917: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 690 op/s 2024-09-11T01:17:54.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.758+0000 7f55b0699640 1 -- 172.21.15.78:0/1620759046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81076a0 msgr2=0x7f55a8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:54.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.758+0000 7f55b0699640 1 --2- 172.21.15.78:0/1620759046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81076a0 0x7f55a8107ae0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f5598009900 tx=0x7f559802a410 comp rx=0 tx=0).stop 2024-09-11T01:17:54.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.758+0000 7f55b0699640 1 -- 172.21.15.78:0/1620759046 shutdown_connections 2024-09-11T01:17:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.758+0000 7f55b0699640 1 --2- 172.21.15.78:0/1620759046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a8108020 0x7f55a810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.758+0000 7f55b0699640 1 --2- 172.21.15.78:0/1620759046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81076a0 0x7f55a8107ae0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.758+0000 7f55b0699640 1 --2- 172.21.15.78:0/1620759046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a8101c90 0x7f55a8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.758+0000 7f55b0699640 1 -- 172.21.15.78:0/1620759046 >> 172.21.15.78:0/1620759046 conn(0x7f55a80fd700 msgr2=0x7f55a80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.758+0000 7f55b0699640 1 -- 172.21.15.78:0/1620759046 shutdown_connections 2024-09-11T01:17:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.758+0000 7f55b0699640 1 -- 172.21.15.78:0/1620759046 wait complete. 2024-09-11T01:17:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55b0699640 1 Processor -- start 2024-09-11T01:17:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55b0699640 1 -- start start 2024-09-11T01:17:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55b0699640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a8101c90 0x7f55a8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55b0699640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81076a0 0x7f55a8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55b0699640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a8108020 0x7f55a819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55b0699640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8112e00 con 0x7f55a81076a0 2024-09-11T01:17:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55b0699640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8112c80 con 0x7f55a8108020 2024-09-11T01:17:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55b0699640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8112f80 con 0x7f55a8101c90 2024-09-11T01:17:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55adc34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81076a0 0x7f55a8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55adc34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81076a0 0x7f55a8199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58570/0 (socket says 172.21.15.78:58570) 2024-09-11T01:17:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55adc34640 1 -- 172.21.15.78:0/821774188 learned_addr learned my addr 172.21.15.78:0/821774188 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55ae435640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a8101c90 0x7f55a8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55adc34640 1 -- 172.21.15.78:0/821774188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a8101c90 msgr2=0x7f55a8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55aec36640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a8108020 0x7f55a819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55adc34640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a8101c90 0x7f55a8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55adc34640 1 -- 172.21.15.78:0/821774188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a8108020 msgr2=0x7f55a819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55adc34640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a8108020 0x7f55a819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55adc34640 1 -- 172.21.15.78:0/821774188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a819f3c0 con 0x7f55a81076a0 2024-09-11T01:17:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55ae435640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a8101c90 0x7f55a8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:17:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.759+0000 7f55aec36640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a8108020 0x7f55a819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:17:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.760+0000 7f55adc34640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81076a0 0x7f55a8199850 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5598005940 tx=0x7f55980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.760+0000 7f559f7fe640 1 -- 172.21.15.78:0/821774188 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559803f030 con 0x7f55a81076a0 2024-09-11T01:17:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.760+0000 7f55b0699640 1 -- 172.21.15.78:0/821774188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a819f650 con 0x7f55a81076a0 2024-09-11T01:17:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.760+0000 7f559f7fe640 1 -- 172.21.15.78:0/821774188 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5598003cb0 con 0x7f55a81076a0 2024-09-11T01:17:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.760+0000 7f559f7fe640 1 -- 172.21.15.78:0/821774188 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598035070 con 0x7f55a81076a0 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.760+0000 7f55b0699640 1 -- 172.21.15.78:0/821774188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a81a5d70 con 0x7f55a81076a0 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.762+0000 7f559f7fe640 1 -- 172.21.15.78:0/821774188 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f559803bbf0 con 0x7f55a81076a0 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55b2a6e740 1 Processor -- start 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55b2a6e740 1 -- start start 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55b2a6e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb0d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55b2a6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb1d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55b2a6e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb20150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55b2a6e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55662ea83e50 con 0x55662eb1dd60 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55b2a6e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55662ea73280 con 0x55662eb0d0a0 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55b2a6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55662ea77090 con 0x55662eb0d9c0 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55adc34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb20150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55adc34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb20150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58580/0 (socket says 172.21.15.78:58580) 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55adc34640 1 -- 172.21.15.78:0/890665054 learned_addr learned my addr 172.21.15.78:0/890665054 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55ae435640 1 --2- 172.21.15.78:0/890665054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb1d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.763+0000 7f55aec36640 1 --2- 172.21.15.78:0/890665054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb0d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f55adc34640 1 -- 172.21.15.78:0/890665054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 msgr2=0x55662eb1d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f55adc34640 1 --2- 172.21.15.78:0/890665054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb1d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f55adc34640 1 -- 172.21.15.78:0/890665054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 msgr2=0x55662eb0d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f55adc34640 1 --2- 172.21.15.78:0/890665054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb0d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f55adc34640 1 -- 172.21.15.78:0/890665054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662e7d1650 con 0x55662eb1dd60 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f55aec36640 1 --2- 172.21.15.78:0/890665054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb0d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f55adc34640 1 --2- 172.21.15.78:0/890665054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb20150 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f55981042a0 tx=0x7f559810af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ba986411ecf4c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f559f7fe640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55940b5450 0x7f55940b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f55aec36640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55940b5450 0x7f55940b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f559cff9640 1 -- 172.21.15.78:0/890665054 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980c96e0 con 0x55662eb1dd60 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f559f7fe640 1 -- 172.21.15.78:0/821774188 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7f5598106bd0 con 0x7f55a81076a0 2024-09-11T01:17:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f559cff9640 1 -- 172.21.15.78:0/890665054 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55980cd790 con 0x55662eb1dd60 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.764+0000 7f559cff9640 1 -- 172.21.15.78:0/890665054 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980cc800 con 0x55662eb1dd60 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.765+0000 7f55b2a6e740 1 -- 172.21.15.78:0/890665054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 msgr2=0x55662eb20150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.765+0000 7f55aec36640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55940b5450 0x7f55940b7910 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f55a0005fd0 tx=0x7f55a0009590 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.765+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/890665054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb20150 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f55981042a0 tx=0x7f559810af80 comp rx=0 tx=0).stop 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.765+0000 7f55b2a6e740 1 -- 172.21.15.78:0/890665054 shutdown_connections 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.765+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/890665054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb20150 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.765+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/890665054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb1d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.765+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/890665054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb0d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.765+0000 7f55b2a6e740 1 -- 172.21.15.78:0/890665054 >> 172.21.15.78:0/890665054 conn(0x55662eb0aed0 msgr2=0x55662eb0c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.765+0000 7f55b2a6e740 1 -- 172.21.15.78:0/890665054 shutdown_connections 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.765+0000 7f55b2a6e740 1 -- 172.21.15.78:0/890665054 wait complete. 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.765+0000 7f55b2a6e740 1 -- start start 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55b2a6e740 1 Processor -- start 2024-09-11T01:17:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55b2a6e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb0d0a0 0x55662eb1d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55b2a6e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d9c0 0x55662eb1ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55ae435640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb0d0a0 0x55662eb1d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55ae435640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb0d0a0 0x55662eb1d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58586/0 (socket says 172.21.15.78:58586) 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55ae435640 1 -- 172.21.15.78:0/1110361059 learned_addr learned my addr 172.21.15.78:0/1110361059 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1110361059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb1dd60 0x55662eb1b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1110361059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55662e8d4700 con 0x55662eb0d0a0 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1110361059 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55662ea0e110 con 0x55662eb0d9c0 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1110361059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55662eb207d0 con 0x55662eb1dd60 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55ae435640 1 -- 172.21.15.78:0/1110361059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb1dd60 msgr2=0x55662eb1b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55ae435640 1 --2- 172.21.15.78:0/1110361059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb1dd60 0x55662eb1b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55ae435640 1 -- 172.21.15.78:0/1110361059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d9c0 msgr2=0x55662eb1ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55ae435640 1 --2- 172.21.15.78:0/1110361059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d9c0 0x55662eb1ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55b2a6e740 10 client.0 _ll_register_callbacks cb 0x55662ea27990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:17:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.766+0000 7f55ae435640 1 -- 172.21.15.78:0/1110361059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662ea3bc40 con 0x55662eb0d0a0 2024-09-11T01:17:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f5591ffb640 10 client.18771 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:17:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f55ae435640 1 --2- 172.21.15.78:0/1110361059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb0d0a0 0x55662eb1d6f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f55a400f4f0 tx=0x7f55a400f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6b45c1c7b52f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1110361059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55980cd790 con 0x55662eb0d0a0 2024-09-11T01:17:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f5591ffb640 1 -- 172.21.15.78:0/1110361059 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4013850 con 0x55662eb0d0a0 2024-09-11T01:17:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f5591ffb640 1 -- 172.21.15.78:0/1110361059 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7f55a40139f0 con 0x55662eb0d0a0 2024-09-11T01:17:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f5591ffb640 1 -- 172.21.15.78:0/1110361059 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f55a4013c50 con 0x55662eb0d0a0 2024-09-11T01:17:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f55b2a6e740 10 client.18771 fetch_fsmap learned FSMap version 70 2024-09-11T01:17:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1110361059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=70}) -- 0x55662eb3d050 con 0x55662eb0d0a0 2024-09-11T01:17:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f5591ffb640 1 -- 172.21.15.78:0/1110361059 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f55a4025420 con 0x55662eb0d0a0 2024-09-11T01:17:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f55b2a6e740 10 client.18771 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:17:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f55b2a6e740 10 client.18771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:17:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f55b2a6e740 20 client.18771 populate_metadata read hostname 'smithi078' 2024-09-11T01:17:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1110361059 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55662eb3e450 0x55662eb408a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f55b2a6e740 4 client.18771 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:17:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.767+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1110361059 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55662ea85da0 con 0x55662eb3e450 2024-09-11T01:17:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.768+0000 7f55adc34640 1 --2- 172.21.15.78:0/1110361059 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55662eb3e450 0x55662eb408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.768+0000 7f5591ffb640 10 client.18771 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.768+0000 7f55adc34640 1 --2- 172.21.15.78:0/1110361059 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55662eb3e450 0x55662eb408a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2303422ec9cf9427 server_cookie=49f7f4c0c3ec939d in_seq=0 out_seq=0 2024-09-11T01:17:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.769+0000 7f5591ffb640 1 -- 172.21.15.78:0/1110361059 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55662ea85da0 con 0x55662eb3e450 2024-09-11T01:17:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.769+0000 7f5591ffb640 10 client.18771 handle_command_reply: tid=0 2024-09-11T01:17:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.769+0000 7f55b2a6e740 1 client.18771 shutdown 2024-09-11T01:17:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.770+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1110361059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb0d0a0 msgr2=0x55662eb1d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.770+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1110361059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb0d0a0 0x55662eb1d6f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f55a400f4f0 tx=0x7f55a400f9b0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.770+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1110361059 shutdown_connections 2024-09-11T01:17:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.770+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1110361059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb1dd60 0x55662eb1b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.770+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1110361059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d9c0 0x55662eb1ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.770+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1110361059 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55662eb3e450 0x55662eb408a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.770+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1110361059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb0d0a0 0x55662eb1d6f0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.770+0000 7f5591ffb640 0 client.18771 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.770+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1110361059 >> 172.21.15.78:0/1110361059 conn(0x55662eb0aed0 msgr2=0x55662eb10a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.770+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1110361059 shutdown_connections 2024-09-11T01:17:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.770+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1110361059 wait complete. 2024-09-11T01:17:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.770+0000 7f55b2a6e740 20 client.18771 trim_cache size 0 max 16384 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55b2a6e740 1 Processor -- start 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55b2a6e740 1 -- start start 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55b2a6e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb299c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55b2a6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb0a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55b2a6e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb0f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55b2a6e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8073050 con 0x55662eb1dd60 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55b2a6e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8074b90 con 0x55662eb0d0a0 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55b2a6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8073720 con 0x55662eb0d9c0 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55aec36640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb0f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55aec36640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb0f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58590/0 (socket says 172.21.15.78:58590) 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55aec36640 1 -- 172.21.15.78:0/964812967 learned_addr learned my addr 172.21.15.78:0/964812967 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55ae435640 1 --2- 172.21.15.78:0/964812967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb299c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.789+0000 7f55adc34640 1 --2- 172.21.15.78:0/964812967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb0a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.790+0000 7f55aec36640 1 -- 172.21.15.78:0/964812967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 msgr2=0x55662eb0a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.790+0000 7f55aec36640 1 --2- 172.21.15.78:0/964812967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb0a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.790+0000 7f55aec36640 1 -- 172.21.15.78:0/964812967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 msgr2=0x55662eb299c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.790+0000 7f55aec36640 1 --2- 172.21.15.78:0/964812967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb299c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.790+0000 7f55aec36640 1 -- 172.21.15.78:0/964812967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662eb05e20 con 0x55662eb1dd60 2024-09-11T01:17:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.790+0000 7f55adc34640 1 --2- 172.21.15.78:0/964812967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb0a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:17:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.790+0000 7f55aec36640 1 --2- 172.21.15.78:0/964812967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb0f100 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f55a0012830 tx=0x7f55a0014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2bd883127a5fbd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.790+0000 7f5593fff640 1 -- 172.21.15.78:0/964812967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a000f8d0 con 0x55662eb1dd60 2024-09-11T01:17:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.790+0000 7f5593fff640 1 -- 172.21.15.78:0/964812967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a001f420 con 0x55662eb1dd60 2024-09-11T01:17:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.790+0000 7f5593fff640 1 -- 172.21.15.78:0/964812967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a0027660 con 0x55662eb1dd60 2024-09-11T01:17:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.791+0000 7f55b2a6e740 1 -- 172.21.15.78:0/964812967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 msgr2=0x55662eb0f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.791+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/964812967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb0f100 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f55a0012830 tx=0x7f55a0014070 comp rx=0 tx=0).stop 2024-09-11T01:17:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.791+0000 7f55b2a6e740 1 -- 172.21.15.78:0/964812967 shutdown_connections 2024-09-11T01:17:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.791+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/964812967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb0f100 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.791+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/964812967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb0a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.791+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/964812967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb299c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.791+0000 7f55b2a6e740 1 -- 172.21.15.78:0/964812967 >> 172.21.15.78:0/964812967 conn(0x55662eb0aed0 msgr2=0x55662eb10a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.791+0000 7f55b2a6e740 1 -- 172.21.15.78:0/964812967 shutdown_connections 2024-09-11T01:17:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.791+0000 7f55b2a6e740 1 -- 172.21.15.78:0/964812967 wait complete. 2024-09-11T01:17:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.791+0000 7f55b2a6e740 1 -- start start 2024-09-11T01:17:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.791+0000 7f55b2a6e740 1 Processor -- start 2024-09-11T01:17:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55b2a6e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb25da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55b2a6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb26320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55b2a6e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb20b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55b2a6e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55a80744c0 con 0x55662eb1dd60 2024-09-11T01:17:54.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55b2a6e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8073df0 con 0x55662eb0d0a0 2024-09-11T01:17:54.237 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:17:54.237 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:17:54.237 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:17:54.237 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:17:54.237 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.238 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.239 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:17:54.240 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.242 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:17:54.242 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:17:54.242 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.242 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.242 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.242 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.242 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.242 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:17:54.242 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:17:54.242 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.242 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:17:54.243 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.245 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:17:54.245 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:17:54.245 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.245 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.245 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.245 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.245 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.245 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.246 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:17:54.247 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.249 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.250 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.250 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.250 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:17:54.250 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:17:54.250 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.250 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.251 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.251 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.251 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.251 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:17:54.251 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:17:54.251 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.251 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.251 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.251 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.251 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:17:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:17:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:17:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:17:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:17:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:17:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:17:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:17:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:17:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:17:54.252 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:17:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55b2a6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8071050 con 0x55662eb0d9c0 2024-09-11T01:17:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55b2a6e740 10 client.0 _ll_register_callbacks cb 0x7f55a400a530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:17:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55aec36640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb20b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55aec36640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb20b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58594/0 (socket says 172.21.15.78:58594) 2024-09-11T01:17:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55aec36640 1 -- 172.21.15.78:0/1022894609 learned_addr learned my addr 172.21.15.78:0/1022894609 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55ae435640 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb25da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55adc34640 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb26320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55aec36640 1 -- 172.21.15.78:0/1022894609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 msgr2=0x55662eb26320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55aec36640 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb26320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55aec36640 1 -- 172.21.15.78:0/1022894609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 msgr2=0x55662eb25da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:17:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55aec36640 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb25da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55aec36640 1 -- 172.21.15.78:0/1022894609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662eb21320 con 0x55662eb1dd60 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55aec36640 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb20b20 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f55a0012010 tx=0x7f55a0012510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cb7c8b158d7d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f5591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f5591ffb640 1 -- 172.21.15.78:0/1022894609 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a0037060 con 0x55662eb1dd60 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f5591ffb640 1 -- 172.21.15.78:0/1022894609 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7f55a001fae0 con 0x55662eb1dd60 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.792+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1022894609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55a001f420 con 0x55662eb1dd60 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.793+0000 7f5591ffb640 1 -- 172.21.15.78:0/1022894609 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f55a000f8d0 con 0x55662eb1dd60 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.793+0000 7f55b2a6e740 10 client.18777 fetch_fsmap learned FSMap version 70 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.793+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1022894609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55662ebabfc0 con 0x55662eb1dd60 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.793+0000 7f5591ffb640 1 -- 172.21.15.78:0/1022894609 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a001f420 con 0x55662eb1dd60 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.793+0000 7f5591ffb640 1 -- 172.21.15.78:0/1022894609 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f55a003cac0 con 0x55662eb1dd60 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.793+0000 7f55b2a6e740 10 client.18777 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.793+0000 7f55b2a6e740 10 client.18777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:17:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.793+0000 7f55b2a6e740 20 client.18777 populate_metadata read hostname 'smithi078' 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.793+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55662ebac630 0x55662ebaea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.793+0000 7f55b2a6e740 4 client.18777 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.793+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1022894609 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55662ea81850 con 0x55662ebac630 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.793+0000 7f55ae435640 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55662ebac630 0x55662ebaea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.794+0000 7f55ae435640 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55662ebac630 0x55662ebaea50 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80932677f73a9478 server_cookie=aa8ec2cddaf675b0 in_seq=0 out_seq=0 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.794+0000 7f5591ffb640 10 client.18777 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f5591ffb640 1 -- 172.21.15.78:0/1022894609 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6622 (crc 0 0 0) 0x55662ea81850 con 0x55662ebac630 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f5591ffb640 10 client.18777 handle_command_reply: tid=0 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f55b2a6e740 1 client.18777 shutdown 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1022894609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 msgr2=0x55662eb20b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb20b20 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f55a0012010 tx=0x7f55a0012510 comp rx=0 tx=0).stop 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1022894609 shutdown_connections 2024-09-11T01:17:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55662ebac630 0x55662ebaea50 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55662eb1dd60 0x55662eb20b20 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55662eb0d9c0 0x55662eb26320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f55b2a6e740 1 --2- 172.21.15.78:0/1022894609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662eb0d0a0 0x55662eb25da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f5591ffb640 0 client.18777 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1022894609 >> 172.21.15.78:0/1022894609 conn(0x55662eb0aed0 msgr2=0x55662eb1c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1022894609 shutdown_connections 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f55b2a6e740 1 -- 172.21.15.78:0/1022894609 wait complete. 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.795+0000 7f55b2a6e740 20 client.18777 trim_cache size 0 max 16384 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.796+0000 7f55b0699640 1 -- 172.21.15.78:0/821774188 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55940b5450 msgr2=0x7f55940b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.796+0000 7f55b0699640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55940b5450 0x7f55940b7910 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f55a0005fd0 tx=0x7f55a0009590 comp rx=0 tx=0).stop 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.796+0000 7f55b0699640 1 -- 172.21.15.78:0/821774188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81076a0 msgr2=0x7f55a8199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.796+0000 7f55b0699640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81076a0 0x7f55a8199850 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5598005940 tx=0x7f55980042f0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.796+0000 7f55b0699640 1 -- 172.21.15.78:0/821774188 shutdown_connections 2024-09-11T01:17:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.796+0000 7f55b0699640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55940b5450 0x7f55940b7910 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.796+0000 7f55b0699640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a8108020 0x7f55a819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.796+0000 7f55b0699640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81076a0 0x7f55a8199850 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.796+0000 7f55b0699640 1 --2- 172.21.15.78:0/821774188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a8101c90 0x7f55a8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.796+0000 7f55b0699640 1 -- 172.21.15.78:0/821774188 >> 172.21.15.78:0/821774188 conn(0x7f55a80fd700 msgr2=0x7f55a80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.796+0000 7f55b0699640 1 -- 172.21.15.78:0/821774188 shutdown_connections 2024-09-11T01:17:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:52.796+0000 7f55b0699640 1 -- 172.21.15.78:0/821774188 wait complete. 2024-09-11T01:17:54.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 235167 34.82 MB/sec execute 240 sec latency 140.838 ms 2024-09-11T01:17:55.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 235767 34.78 MB/sec execute 241 sec latency 222.207 ms 2024-09-11T01:17:55.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:55 smithi110 ceph-mon[34712]: pgmap v918: 97 pgs: 97 active+clean; 2.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 642 op/s 2024-09-11T01:17:55.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:55 smithi112 ceph-mon[35384]: pgmap v918: 97 pgs: 97 active+clean; 2.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 642 op/s 2024-09-11T01:17:55.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:55 smithi078 ceph-mon[31315]: pgmap v918: 97 pgs: 97 active+clean; 2.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 642 op/s 2024-09-11T01:17:56.162 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:17:56.162 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:17:56.545 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:17:56.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 236569 34.84 MB/sec execute 242 sec latency 65.268 ms 2024-09-11T01:17:57.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.084+0000 7fc44cd96640 1 -- 172.21.15.78:0/1318042105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc448104640 msgr2=0x7fc44810cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:57.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.084+0000 7fc44cd96640 1 --2- 172.21.15.78:0/1318042105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc448104640 0x7fc44810cb30 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009900 tx=0x7fc43c02a410 comp rx=0 tx=0).stop 2024-09-11T01:17:57.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.084+0000 7fc44cd96640 1 -- 172.21.15.78:0/1318042105 shutdown_connections 2024-09-11T01:17:57.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.084+0000 7fc44cd96640 1 --2- 172.21.15.78:0/1318042105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc44810d180 0x7fc44810f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.084+0000 7fc44cd96640 1 --2- 172.21.15.78:0/1318042105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc448104640 0x7fc44810cb30 secure :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009900 tx=0x7fc43c02a410 comp rx=0 tx=0).stop 2024-09-11T01:17:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.084+0000 7fc44cd96640 1 --2- 172.21.15.78:0/1318042105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc448103d20 0x7fc448104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.084+0000 7fc44cd96640 1 -- 172.21.15.78:0/1318042105 >> 172.21.15.78:0/1318042105 conn(0x7fc4480fd7c0 msgr2=0x7fc4480ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.085+0000 7fc44cd96640 1 -- 172.21.15.78:0/1318042105 shutdown_connections 2024-09-11T01:17:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.085+0000 7fc44cd96640 1 -- 172.21.15.78:0/1318042105 wait complete. 2024-09-11T01:17:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.086+0000 7fc44cd96640 1 Processor -- start 2024-09-11T01:17:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.087+0000 7fc44cd96640 1 -- start start 2024-09-11T01:17:57.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.087+0000 7fc44cd96640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc448103d20 0x7fc44810c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:57.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.087+0000 7fc44cd96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc44810d180 0x7fc44810c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:57.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.087+0000 7fc44cd96640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4481a1040 0x7fc4481a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:57.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.087+0000 7fc44cd96640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc448112e40 con 0x7fc448103d20 2024-09-11T01:17:57.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.087+0000 7fc44cd96640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc448112cc0 con 0x7fc4481a1040 2024-09-11T01:17:57.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.087+0000 7fc44cd96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc448112fc0 con 0x7fc44810d180 2024-09-11T01:17:57.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.088+0000 7fc446d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4481a1040 0x7fc4481a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:57.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.088+0000 7fc44659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc448103d20 0x7fc44810c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:57.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.088+0000 7fc44659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc448103d20 0x7fc44810c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39106/0 (socket says 172.21.15.78:39106) 2024-09-11T01:17:57.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.088+0000 7fc44659c640 1 -- 172.21.15.78:0/3515684586 learned_addr learned my addr 172.21.15.78:0/3515684586 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:17:57.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.088+0000 7fc445d9b640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc44810d180 0x7fc44810c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.088+0000 7fc44659c640 1 -- 172.21.15.78:0/3515684586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc44810d180 msgr2=0x7fc44810c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.088+0000 7fc44659c640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc44810d180 0x7fc44810c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.088+0000 7fc44659c640 1 -- 172.21.15.78:0/3515684586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4481a1040 msgr2=0x7fc4481a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.088+0000 7fc44659c640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4481a1040 0x7fc4481a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.088+0000 7fc44659c640 1 -- 172.21.15.78:0/3515684586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4481a1c20 con 0x7fc448103d20 2024-09-11T01:17:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.088+0000 7fc44659c640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc448103d20 0x7fc44810c0e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fc43800ea60 tx=0x7fc43800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.089+0000 7fc4377fe640 1 -- 172.21.15.78:0/3515684586 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43800cde0 con 0x7fc448103d20 2024-09-11T01:17:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.089+0000 7fc4377fe640 1 -- 172.21.15.78:0/3515684586 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4380035f0 con 0x7fc448103d20 2024-09-11T01:17:57.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.089+0000 7fc4377fe640 1 -- 172.21.15.78:0/3515684586 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438010690 con 0x7fc448103d20 2024-09-11T01:17:57.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.089+0000 7fc44cd96640 1 -- 172.21.15.78:0/3515684586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4481a81c0 con 0x7fc448103d20 2024-09-11T01:17:57.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.089+0000 7fc44cd96640 1 -- 172.21.15.78:0/3515684586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4481a8560 con 0x7fc448103d20 2024-09-11T01:17:57.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.091+0000 7fc4357fa640 1 -- 172.21.15.78:0/3515684586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc448104100 con 0x7fc448103d20 2024-09-11T01:17:57.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.094+0000 7fc4377fe640 1 -- 172.21.15.78:0/3515684586 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc438004000 con 0x7fc448103d20 2024-09-11T01:17:57.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.095+0000 7fc4377fe640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc4300b55a0 0x7fc4300b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:17:57.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.095+0000 7fc445d9b640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc4300b55a0 0x7fc4300b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:17:57.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.095+0000 7fc4377fe640 1 -- 172.21.15.78:0/3515684586 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7fc4380e6410 con 0x7fc448103d20 2024-09-11T01:17:57.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.096+0000 7fc445d9b640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc4300b55a0 0x7fc4300b7a60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009230 tx=0x7fc43c031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:17:57.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.099+0000 7fc4377fe640 1 -- 172.21.15.78:0/3515684586 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc4380ad0e0 con 0x7fc448103d20 2024-09-11T01:17:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.424+0000 7fc4357fa640 1 -- 172.21.15.78:0/3515684586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4481a1df0 con 0x7fc448103d20 2024-09-11T01:17:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.425+0000 7fc4377fe640 1 -- 172.21.15.78:0/3515684586 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+11979 (secure 0 0 0) 0x7fc4380ac9e0 con 0x7fc448103d20 2024-09-11T01:17:57.430 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:17:57.430 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":70,"btime":"2024-09-11T01:15:50:008465+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":70,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:15:49.712348+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:17:57.430 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 70 2024-09-11T01:17:57.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.428+0000 7fc4357fa640 1 -- 172.21.15.78:0/3515684586 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc4300b55a0 msgr2=0x7fc4300b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:57.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.428+0000 7fc4357fa640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc4300b55a0 0x7fc4300b7a60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009230 tx=0x7fc43c031400 comp rx=0 tx=0).stop 2024-09-11T01:17:57.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.428+0000 7fc4357fa640 1 -- 172.21.15.78:0/3515684586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc448103d20 msgr2=0x7fc44810c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:17:57.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.428+0000 7fc4357fa640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc448103d20 0x7fc44810c0e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fc43800ea60 tx=0x7fc43800ef20 comp rx=0 tx=0).stop 2024-09-11T01:17:57.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.429+0000 7fc4357fa640 1 -- 172.21.15.78:0/3515684586 shutdown_connections 2024-09-11T01:17:57.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.429+0000 7fc4357fa640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc4300b55a0 0x7fc4300b7a60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:57.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.429+0000 7fc4357fa640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4481a1040 0x7fc4481a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:57.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.429+0000 7fc4357fa640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc44810d180 0x7fc44810c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:57.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.429+0000 7fc4357fa640 1 --2- 172.21.15.78:0/3515684586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc448103d20 0x7fc44810c0e0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:17:57.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.429+0000 7fc4357fa640 1 -- 172.21.15.78:0/3515684586 >> 172.21.15.78:0/3515684586 conn(0x7fc4480fd7c0 msgr2=0x7fc4480ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:17:57.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.429+0000 7fc4357fa640 1 -- 172.21.15.78:0/3515684586 shutdown_connections 2024-09-11T01:17:57.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:17:57.429+0000 7fc4357fa640 1 -- 172.21.15.78:0/3515684586 wait complete. 2024-09-11T01:17:57.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:57 smithi078 ceph-mon[31315]: pgmap v919: 97 pgs: 97 active+clean; 2.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 681 op/s 2024-09-11T01:17:57.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:57 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3515684586' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:17:57.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 237230 34.86 MB/sec execute 243 sec latency 84.860 ms 2024-09-11T01:17:57.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:57 smithi110 ceph-mon[34712]: pgmap v919: 97 pgs: 97 active+clean; 2.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 681 op/s 2024-09-11T01:17:57.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:57 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3515684586' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:17:57.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:57 smithi112 ceph-mon[35384]: pgmap v919: 97 pgs: 97 active+clean; 2.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 681 op/s 2024-09-11T01:17:57.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:57 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3515684586' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:17:58.271 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:17:58.619 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:17:58.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 237915 34.83 MB/sec execute 244 sec latency 117.698 ms 2024-09-11T01:17:59.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 238693 34.89 MB/sec execute 245 sec latency 105.264 ms 2024-09-11T01:17:59.892 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:17:59 smithi078 ceph-mon[31315]: pgmap v920: 97 pgs: 97 active+clean; 2.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 661 op/s 2024-09-11T01:17:59.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:17:59 smithi110 ceph-mon[34712]: pgmap v920: 97 pgs: 97 active+clean; 2.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 661 op/s 2024-09-11T01:17:59.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:17:59 smithi112 ceph-mon[35384]: pgmap v920: 97 pgs: 97 active+clean; 2.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 661 op/s 2024-09-11T01:18:00.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.007+0000 7f08a2794640 1 -- 172.21.15.78:0/2755535701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f089c10a9c0 msgr2=0x7f089c10ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:00.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.007+0000 7f08a2794640 1 --2- 172.21.15.78:0/2755535701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f089c10a9c0 0x7f089c10ae00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0890009900 tx=0x7f089002a410 comp rx=0 tx=0).stop 2024-09-11T01:18:00.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.007+0000 7f08a2794640 1 -- 172.21.15.78:0/2755535701 shutdown_connections 2024-09-11T01:18:00.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.007+0000 7f08a2794640 1 --2- 172.21.15.78:0/2755535701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f089c10b340 0x7f089c111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.007+0000 7f08a2794640 1 --2- 172.21.15.78:0/2755535701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f089c10a9c0 0x7f089c10ae00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.007+0000 7f08a2794640 1 --2- 172.21.15.78:0/2755535701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c104fb0 0x7f089c105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.007+0000 7f08a2794640 1 -- 172.21.15.78:0/2755535701 >> 172.21.15.78:0/2755535701 conn(0x7f089c100a40 msgr2=0x7f089c102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:00.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.007+0000 7f08a2794640 1 -- 172.21.15.78:0/2755535701 shutdown_connections 2024-09-11T01:18:00.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.008+0000 7f08a2794640 1 -- 172.21.15.78:0/2755535701 wait complete. 2024-09-11T01:18:00.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.008+0000 7f08a2794640 1 Processor -- start 2024-09-11T01:18:00.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.008+0000 7f08a2794640 1 -- start start 2024-09-11T01:18:00.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f08a2794640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c104fb0 0x7f089c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f08a2794640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f089c10a9c0 0x7f089c19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f08a2794640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f089c10b340 0x7f089c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f089bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c104fb0 0x7f089c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f089bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c104fb0 0x7f089c19b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39124/0 (socket says 172.21.15.78:39124) 2024-09-11T01:18:00.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f089bfff640 1 -- 172.21.15.78:0/1327077306 learned_addr learned my addr 172.21.15.78:0/1327077306 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:00.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f08a2794640 1 -- 172.21.15.78:0/1327077306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f089c114f90 con 0x7f089c104fb0 2024-09-11T01:18:00.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f08a2794640 1 -- 172.21.15.78:0/1327077306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f089c114e10 con 0x7f089c10b340 2024-09-11T01:18:00.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f08a2794640 1 -- 172.21.15.78:0/1327077306 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f089c115110 con 0x7f089c10a9c0 2024-09-11T01:18:00.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f08a0d31640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f089c10b340 0x7f089c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f089b7fe640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f089c10a9c0 0x7f089c19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f089bfff640 1 -- 172.21.15.78:0/1327077306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f089c10a9c0 msgr2=0x7f089c19b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:00.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f089bfff640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f089c10a9c0 0x7f089c19b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f089bfff640 1 -- 172.21.15.78:0/1327077306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f089c10b340 msgr2=0x7f089c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:00.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f089bfff640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f089c10b340 0x7f089c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.009+0000 7f089bfff640 1 -- 172.21.15.78:0/1327077306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c1a1350 con 0x7f089c104fb0 2024-09-11T01:18:00.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.010+0000 7f089bfff640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c104fb0 0x7f089c19b2a0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f088c00e9f0 tx=0x7f088c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:00.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.010+0000 7f08997fa640 1 -- 172.21.15.78:0/1327077306 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c00cd80 con 0x7f089c104fb0 2024-09-11T01:18:00.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.010+0000 7f08997fa640 1 -- 172.21.15.78:0/1327077306 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f088c0108e0 con 0x7f089c104fb0 2024-09-11T01:18:00.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.010+0000 7f08997fa640 1 -- 172.21.15.78:0/1327077306 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c018980 con 0x7f089c104fb0 2024-09-11T01:18:00.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.010+0000 7f08a2794640 1 -- 172.21.15.78:0/1327077306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c1a1640 con 0x7f089c104fb0 2024-09-11T01:18:00.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.010+0000 7f08a2794640 1 -- 172.21.15.78:0/1327077306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c1a7d00 con 0x7f089c104fb0 2024-09-11T01:18:00.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.014+0000 7f08997fa640 1 -- 172.21.15.78:0/1327077306 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f088c010430 con 0x7f089c104fb0 2024-09-11T01:18:00.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.014+0000 7f08a4b69740 1 Processor -- start 2024-09-11T01:18:00.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.014+0000 7f08a4b69740 1 -- start start 2024-09-11T01:18:00.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.014+0000 7f08a4b69740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d3d4d0 0x558cd7d3d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.014+0000 7f08a4b69740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3ddf0 0x558cd7d49840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.014+0000 7f08a4b69740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d49d80 0x558cd7d4c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.014+0000 7f08a4b69740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558cd7cb3ea0 con 0x558cd7d3ddf0 2024-09-11T01:18:00.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.015+0000 7f08a4b69740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558cd7ca32d0 con 0x558cd7d3d4d0 2024-09-11T01:18:00.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.015+0000 7f08a4b69740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558cd7ca70e0 con 0x558cd7d49d80 2024-09-11T01:18:00.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.015+0000 7f089bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3ddf0 0x558cd7d49840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.015+0000 7f08a0d31640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d3d4d0 0x558cd7d3d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.015+0000 7f08a0d31640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d3d4d0 0x558cd7d3d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39106/0 (socket says 172.21.15.78:39106) 2024-09-11T01:18:00.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.015+0000 7f08a0d31640 1 -- 172.21.15.78:0/1976421522 learned_addr learned my addr 172.21.15.78:0/1976421522 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:00.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.015+0000 7f089bfff640 1 -- 172.21.15.78:0/1976421522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d49d80 msgr2=0x558cd7d4c170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:18:00.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.015+0000 7f089bfff640 1 --2- 172.21.15.78:0/1976421522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d49d80 0x558cd7d4c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.015+0000 7f089bfff640 1 -- 172.21.15.78:0/1976421522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d3d4d0 msgr2=0x558cd7d3d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:00.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.015+0000 7f089bfff640 1 --2- 172.21.15.78:0/1976421522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d3d4d0 0x558cd7d3d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.015+0000 7f089bfff640 1 -- 172.21.15.78:0/1976421522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cd7a01650 con 0x558cd7d3ddf0 2024-09-11T01:18:00.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.016+0000 7f08997fa640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08840b54d0 0x7f08840b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.016+0000 7f089bfff640 1 --2- 172.21.15.78:0/1976421522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3ddf0 0x558cd7d49840 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f088c0e3990 tx=0x7f088c0ee910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213a73b8841d8237 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.016+0000 7f08827fc640 1 -- 172.21.15.78:0/1976421522 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c0a8dc0 con 0x558cd7d3ddf0 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.016+0000 7f08a0d31640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08840b54d0 0x7f08840b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.016+0000 7f08997fa640 1 -- 172.21.15.78:0/1327077306 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7f088c0e6200 con 0x7f089c104fb0 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.016+0000 7f08827fc640 1 -- 172.21.15.78:0/1976421522 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f088c0ac280 con 0x558cd7d3ddf0 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.016+0000 7f08827fc640 1 -- 172.21.15.78:0/1976421522 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c0af5e0 con 0x558cd7d3ddf0 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.017+0000 7f08a0d31640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08840b54d0 0x7f08840b7990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f08940116b0 tx=0x7f0894009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.017+0000 7f08a4b69740 1 -- 172.21.15.78:0/1976421522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3ddf0 msgr2=0x558cd7d49840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.017+0000 7f08a4b69740 1 --2- 172.21.15.78:0/1976421522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3ddf0 0x558cd7d49840 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f088c0e3990 tx=0x7f088c0ee910 comp rx=0 tx=0).stop 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.017+0000 7f08a4b69740 1 -- 172.21.15.78:0/1976421522 shutdown_connections 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.017+0000 7f08a4b69740 1 --2- 172.21.15.78:0/1976421522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d49d80 0x558cd7d4c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.017+0000 7f08a4b69740 1 --2- 172.21.15.78:0/1976421522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3ddf0 0x558cd7d49840 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.017+0000 7f08a4b69740 1 --2- 172.21.15.78:0/1976421522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d3d4d0 0x558cd7d3d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.017+0000 7f08a4b69740 1 -- 172.21.15.78:0/1976421522 >> 172.21.15.78:0/1976421522 conn(0x558cd7d3b300 msgr2=0x558cd7d3c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:00.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.018+0000 7f08a4b69740 1 -- 172.21.15.78:0/1976421522 shutdown_connections 2024-09-11T01:18:00.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.018+0000 7f08a4b69740 1 -- 172.21.15.78:0/1976421522 wait complete. 2024-09-11T01:18:00.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.018+0000 7f08a4b69740 1 -- start start 2024-09-11T01:18:00.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.019+0000 7f08a4b69740 1 Processor -- start 2024-09-11T01:18:00.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.019+0000 7f08a4b69740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d3d4d0 0x558cd7d49660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.019+0000 7f08a4b69740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d46ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.019+0000 7f089bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d3d4d0 0x558cd7d49660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.020+0000 7f089bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d3d4d0 0x558cd7d49660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39112/0 (socket says 172.21.15.78:39112) 2024-09-11T01:18:00.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.020+0000 7f08a4b69740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d49d80 0x558cd7d470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.020+0000 7f08a0d31640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d49d80 0x558cd7d470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.020+0000 7f089bfff640 1 -- 172.21.15.78:0/217193841 learned_addr learned my addr 172.21.15.78:0/217193841 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:00.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.020+0000 7f08a4b69740 1 -- 172.21.15.78:0/217193841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558cd7b04700 con 0x558cd7d49d80 2024-09-11T01:18:00.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.020+0000 7f08a4b69740 1 -- 172.21.15.78:0/217193841 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558cd7c3e160 con 0x558cd7d3d4d0 2024-09-11T01:18:00.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.020+0000 7f08a4b69740 1 -- 172.21.15.78:0/217193841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558cd7d4ca50 con 0x558cd7d3ddf0 2024-09-11T01:18:00.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.020+0000 7f089b7fe640 1 --2- 172.21.15.78:0/217193841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d46ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.020+0000 7f08a4b69740 10 client.0 _ll_register_callbacks cb 0x558cd7c579e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:00.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.021+0000 7f089b7fe640 1 -- 172.21.15.78:0/217193841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d3d4d0 msgr2=0x558cd7d49660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:00.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.021+0000 7f089b7fe640 1 --2- 172.21.15.78:0/217193841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d3d4d0 0x558cd7d49660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.021+0000 7f089b7fe640 1 -- 172.21.15.78:0/217193841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d49d80 msgr2=0x558cd7d470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:00.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.021+0000 7f089b7fe640 1 --2- 172.21.15.78:0/217193841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d49d80 0x558cd7d470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.021+0000 7f089b7fe640 1 -- 172.21.15.78:0/217193841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cd7c6bc90 con 0x558cd7d3ddf0 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.021+0000 7f08a4b69740 1 -- 172.21.15.78:0/217193841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f088c0ac280 con 0x558cd7d3ddf0 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.021+0000 7f089b7fe640 1 --2- 172.21.15.78:0/217193841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d46ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f089002a930 tx=0x7f0890005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=823d03c7de947f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.021+0000 7f087b7fe640 10 client.27841 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.022+0000 7f087b7fe640 1 -- 172.21.15.78:0/217193841 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890035040 con 0x558cd7d3ddf0 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.022+0000 7f087b7fe640 1 -- 172.21.15.78:0/217193841 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7f0890004030 con 0x558cd7d3ddf0 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.022+0000 7f087b7fe640 1 -- 172.21.15.78:0/217193841 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f0890004250 con 0x558cd7d3ddf0 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.022+0000 7f08a4b69740 10 client.27841 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.022+0000 7f08a4b69740 1 -- 172.21.15.78:0/217193841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=70}) -- 0x558cd7d60ab0 con 0x558cd7d3ddf0 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.022+0000 7f087b7fe640 1 -- 172.21.15.78:0/217193841 <== mon.1 v2:172.21.15.112:3300/0 4 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f08900035f0 con 0x558cd7d3ddf0 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.022+0000 7f08a4b69740 10 client.27841 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.022+0000 7f08a4b69740 10 client.27841 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.022+0000 7f08a4b69740 20 client.27841 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.022+0000 7f08a4b69740 1 --2- 172.21.15.78:0/217193841 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558cd7d61e70 0x558cd7d642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.022+0000 7f08a4b69740 4 client.27841 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:18:00.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.022+0000 7f08a4b69740 1 -- 172.21.15.78:0/217193841 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cd7cb5df0 con 0x558cd7d61e70 2024-09-11T01:18:00.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.023+0000 7f089bfff640 1 --2- 172.21.15.78:0/217193841 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558cd7d61e70 0x558cd7d642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.023+0000 7f089bfff640 1 --2- 172.21.15.78:0/217193841 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558cd7d61e70 0x558cd7d642f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c3d5f0bf160cad94 server_cookie=e531fa14d36e84a8 in_seq=0 out_seq=0 2024-09-11T01:18:00.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.023+0000 7f087b7fe640 10 client.27841 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:18:00.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.024+0000 7f087b7fe640 1 -- 172.21.15.78:0/217193841 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558cd7cb5df0 con 0x558cd7d61e70 2024-09-11T01:18:00.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.024+0000 7f087b7fe640 10 client.27841 handle_command_reply: tid=0 2024-09-11T01:18:00.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.024+0000 7f08a4b69740 1 client.27841 shutdown 2024-09-11T01:18:00.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.024+0000 7f08a4b69740 1 -- 172.21.15.78:0/217193841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 msgr2=0x558cd7d46ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:00.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.024+0000 7f08a4b69740 1 --2- 172.21.15.78:0/217193841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d46ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f089002a930 tx=0x7f0890005e80 comp rx=0 tx=0).stop 2024-09-11T01:18:00.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.024+0000 7f08a4b69740 1 -- 172.21.15.78:0/217193841 shutdown_connections 2024-09-11T01:18:00.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.024+0000 7f08a4b69740 1 --2- 172.21.15.78:0/217193841 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558cd7d61e70 0x558cd7d642f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.025+0000 7f08a4b69740 1 --2- 172.21.15.78:0/217193841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d49d80 0x558cd7d470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.025+0000 7f08a4b69740 1 --2- 172.21.15.78:0/217193841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d46ab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.025+0000 7f08a4b69740 1 --2- 172.21.15.78:0/217193841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d3d4d0 0x558cd7d49660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.025+0000 7f08a4b69740 1 -- 172.21.15.78:0/217193841 >> 172.21.15.78:0/217193841 conn(0x558cd7d3b300 msgr2=0x558cd7d40ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:00.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.025+0000 7f087b7fe640 0 client.27841 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:18:00.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.025+0000 7f08a4b69740 1 -- 172.21.15.78:0/217193841 shutdown_connections 2024-09-11T01:18:00.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.025+0000 7f08a4b69740 1 -- 172.21.15.78:0/217193841 wait complete. 2024-09-11T01:18:00.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.025+0000 7f08a4b69740 20 client.27841 trim_cache size 0 max 16384 2024-09-11T01:18:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.044+0000 7f08a4b69740 1 Processor -- start 2024-09-11T01:18:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.044+0000 7f08a4b69740 1 -- start start 2024-09-11T01:18:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.044+0000 7f08a4b69740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3d4d0 0x558cd7d5e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f08a4b69740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d3c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f08a4b69740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d49d80 0x558cd7d4b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f08a4b69740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1ac3e0 con 0x558cd7d3d4d0 2024-09-11T01:18:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f08a4b69740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1adf20 con 0x558cd7d49d80 2024-09-11T01:18:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f08a4b69740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1acab0 con 0x558cd7d3ddf0 2024-09-11T01:18:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f089b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d3c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f089b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d3c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:51956/0 (socket says 172.21.15.78:51956) 2024-09-11T01:18:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f089b7fe640 1 -- 172.21.15.78:0/3118095783 learned_addr learned my addr 172.21.15.78:0/3118095783 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f08a0d31640 1 --2- 172.21.15.78:0/3118095783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d49d80 0x558cd7d4b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f089bfff640 1 --2- 172.21.15.78:0/3118095783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3d4d0 0x558cd7d5e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f089b7fe640 1 -- 172.21.15.78:0/3118095783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d49d80 msgr2=0x558cd7d4b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f089b7fe640 1 --2- 172.21.15.78:0/3118095783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d49d80 0x558cd7d4b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f089b7fe640 1 -- 172.21.15.78:0/3118095783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3d4d0 msgr2=0x558cd7d5e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f089b7fe640 1 --2- 172.21.15.78:0/3118095783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3d4d0 0x558cd7d5e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f089b7fe640 1 -- 172.21.15.78:0/3118095783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cd7d44fb0 con 0x558cd7d3ddf0 2024-09-11T01:18:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.045+0000 7f08a0d31640 1 --2- 172.21.15.78:0/3118095783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d49d80 0x558cd7d4b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:18:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.046+0000 7f089b7fe640 1 --2- 172.21.15.78:0/3118095783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d3c6f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0890009900 tx=0x7f0890038000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc2c26dd3caa06dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.046+0000 7f0881ffb640 1 -- 172.21.15.78:0/3118095783 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890035070 con 0x558cd7d3ddf0 2024-09-11T01:18:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.046+0000 7f0881ffb640 1 -- 172.21.15.78:0/3118095783 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f088c0ac280 con 0x558cd7d3ddf0 2024-09-11T01:18:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.046+0000 7f0881ffb640 1 -- 172.21.15.78:0/3118095783 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890032cf0 con 0x558cd7d3ddf0 2024-09-11T01:18:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.047+0000 7f08a4b69740 1 -- 172.21.15.78:0/3118095783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 msgr2=0x558cd7d3c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.047+0000 7f08a4b69740 1 --2- 172.21.15.78:0/3118095783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d3c6f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0890009900 tx=0x7f0890038000 comp rx=0 tx=0).stop 2024-09-11T01:18:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.047+0000 7f08a4b69740 1 -- 172.21.15.78:0/3118095783 shutdown_connections 2024-09-11T01:18:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.047+0000 7f08a4b69740 1 --2- 172.21.15.78:0/3118095783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d49d80 0x558cd7d4b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.047+0000 7f08a4b69740 1 --2- 172.21.15.78:0/3118095783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d3c6f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.047+0000 7f08a4b69740 1 --2- 172.21.15.78:0/3118095783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3d4d0 0x558cd7d5e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.047+0000 7f08a4b69740 1 -- 172.21.15.78:0/3118095783 >> 172.21.15.78:0/3118095783 conn(0x558cd7d3b300 msgr2=0x558cd7d3bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.048+0000 7f08a4b69740 1 -- 172.21.15.78:0/3118095783 shutdown_connections 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.048+0000 7f08a4b69740 1 -- 172.21.15.78:0/3118095783 wait complete. 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.048+0000 7f08a4b69740 1 -- start start 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.048+0000 7f08a4b69740 1 Processor -- start 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.049+0000 7f08a4b69740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3d4d0 0x558cd7d5a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.049+0000 7f08a4b69740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d5aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.049+0000 7f08a4b69740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d49d80 0x558cd7d55560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.049+0000 7f08a4b69740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1ad850 con 0x558cd7d3d4d0 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.049+0000 7f08a4b69740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1ad180 con 0x558cd7d49d80 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.049+0000 7f089bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3d4d0 0x558cd7d5a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.049+0000 7f08a4b69740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1aa460 con 0x558cd7d3ddf0 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.049+0000 7f089bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3d4d0 0x558cd7d5a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39176/0 (socket says 172.21.15.78:39176) 2024-09-11T01:18:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.049+0000 7f089bfff640 1 -- 172.21.15.78:0/1497103133 learned_addr learned my addr 172.21.15.78:0/1497103133 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.049+0000 7f08a4b69740 10 client.0 _ll_register_callbacks cb 0x7f08940093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.049+0000 7f089b7fe640 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d5aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.050+0000 7f08a0d31640 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d49d80 0x558cd7d55560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.050+0000 7f089bfff640 1 -- 172.21.15.78:0/1497103133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 msgr2=0x558cd7d5aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.050+0000 7f089bfff640 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d5aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.050+0000 7f089bfff640 1 -- 172.21.15.78:0/1497103133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d49d80 msgr2=0x558cd7d55560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.050+0000 7f089bfff640 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d49d80 0x558cd7d55560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.050+0000 7f089bfff640 1 -- 172.21.15.78:0/1497103133 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cd7d5b1e0 con 0x558cd7d3d4d0 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.050+0000 7f089bfff640 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3d4d0 0x558cd7d5a5b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f088c00c6a0 tx=0x7f088c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f45cd8362596ce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.050+0000 7f087b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.050+0000 7f087b7fe640 1 -- 172.21.15.78:0/1497103133 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c0af830 con 0x558cd7d3d4d0 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.050+0000 7f087b7fe640 1 -- 172.21.15.78:0/1497103133 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8449+0+0 (secure 0 0 0) 0x7f088c0ac9d0 con 0x558cd7d3d4d0 2024-09-11T01:18:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.050+0000 7f08a4b69740 1 -- 172.21.15.78:0/1497103133 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f088c0ac280 con 0x558cd7d3d4d0 2024-09-11T01:18:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.051+0000 7f087b7fe640 1 -- 172.21.15.78:0/1497103133 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f088c0afa90 con 0x558cd7d3d4d0 2024-09-11T01:18:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.051+0000 7f08a4b69740 10 client.18801 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.051+0000 7f08a4b69740 1 -- 172.21.15.78:0/1497103133 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x558cd7dcf320 con 0x558cd7d3d4d0 2024-09-11T01:18:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.051+0000 7f087b7fe640 1 -- 172.21.15.78:0/1497103133 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f088c0ac280 con 0x558cd7d3d4d0 2024-09-11T01:18:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.051+0000 7f087b7fe640 1 -- 172.21.15.78:0/1497103133 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f088c0b57c0 con 0x558cd7d3d4d0 2024-09-11T01:18:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.051+0000 7f08a4b69740 10 client.18801 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.051+0000 7f08a4b69740 10 client.18801 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:18:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.051+0000 7f08a4b69740 20 client.18801 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.051+0000 7f08a4b69740 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558cd7dcf5b0 0x558cd7dd19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.051+0000 7f08a4b69740 4 client.18801 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:18:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.051+0000 7f08a4b69740 1 -- 172.21.15.78:0/1497103133 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558cd7cb18a0 con 0x558cd7dcf5b0 2024-09-11T01:18:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.052+0000 7f089b7fe640 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558cd7dcf5b0 0x558cd7dd19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.052+0000 7f087b7fe640 10 client.18801 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:18:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.052+0000 7f089b7fe640 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558cd7dcf5b0 0x558cd7dd19a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fb1fb4d350b6535d server_cookie=d12ad68b622ecdde in_seq=0 out_seq=0 2024-09-11T01:18:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.052+0000 7f087b7fe640 1 -- 172.21.15.78:0/1497103133 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558cd7cb18a0 con 0x558cd7dcf5b0 2024-09-11T01:18:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.052+0000 7f087b7fe640 10 client.18801 handle_command_reply: tid=0 2024-09-11T01:18:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.052+0000 7f08a4b69740 1 client.18801 shutdown 2024-09-11T01:18:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.053+0000 7f08a4b69740 1 -- 172.21.15.78:0/1497103133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3d4d0 msgr2=0x558cd7d5a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.053+0000 7f08a4b69740 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3d4d0 0x558cd7d5a5b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f088c00c6a0 tx=0x7f088c009310 comp rx=0 tx=0).stop 2024-09-11T01:18:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.053+0000 7f08a4b69740 1 -- 172.21.15.78:0/1497103133 shutdown_connections 2024-09-11T01:18:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.053+0000 7f08a4b69740 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x558cd7dcf5b0 0x558cd7dd19a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.053+0000 7f08a4b69740 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558cd7d49d80 0x558cd7d55560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.053+0000 7f08a4b69740 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cd7d3ddf0 0x558cd7d5aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.053+0000 7f08a4b69740 1 --2- 172.21.15.78:0/1497103133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cd7d3d4d0 0x558cd7d5a5b0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.053+0000 7f08a4b69740 1 -- 172.21.15.78:0/1497103133 >> 172.21.15.78:0/1497103133 conn(0x558cd7d3b300 msgr2=0x558cd7d480e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.053+0000 7f087b7fe640 0 client.18801 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:18:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.053+0000 7f08a4b69740 1 -- 172.21.15.78:0/1497103133 shutdown_connections 2024-09-11T01:18:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.053+0000 7f08a4b69740 1 -- 172.21.15.78:0/1497103133 wait complete. 2024-09-11T01:18:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.053+0000 7f08a4b69740 20 client.18801 trim_cache size 0 max 16384 2024-09-11T01:18:00.061 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:18:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.054+0000 7f08a2794640 1 -- 172.21.15.78:0/1327077306 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08840b54d0 msgr2=0x7f08840b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.054+0000 7f08a2794640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08840b54d0 0x7f08840b7990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f08940116b0 tx=0x7f0894009500 comp rx=0 tx=0).stop 2024-09-11T01:18:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.054+0000 7f08a2794640 1 -- 172.21.15.78:0/1327077306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c104fb0 msgr2=0x7f089c19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.054+0000 7f08a2794640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c104fb0 0x7f089c19b2a0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f088c00e9f0 tx=0x7f088c00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.054+0000 7f08a2794640 1 -- 172.21.15.78:0/1327077306 shutdown_connections 2024-09-11T01:18:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.054+0000 7f08a2794640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f08840b54d0 0x7f08840b7990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.054+0000 7f08a2794640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f089c10b340 0x7f089c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.054+0000 7f08a2794640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f089c10a9c0 0x7f089c19b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.054+0000 7f08a2794640 1 --2- 172.21.15.78:0/1327077306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c104fb0 0x7f089c19b2a0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.054+0000 7f08a2794640 1 -- 172.21.15.78:0/1327077306 >> 172.21.15.78:0/1327077306 conn(0x7f089c100a40 msgr2=0x7f089c107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.054+0000 7f08a2794640 1 -- 172.21.15.78:0/1327077306 shutdown_connections 2024-09-11T01:18:00.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:00.054+0000 7f08a2794640 1 -- 172.21.15.78:0/1327077306 wait complete. 2024-09-11T01:18:00.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 239360 34.87 MB/sec execute 246 sec latency 136.087 ms 2024-09-11T01:18:00.903 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:18:01.298 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:18:01.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:01 smithi078 ceph-mon[31315]: pgmap v921: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 705 op/s 2024-09-11T01:18:01.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 239771 34.80 MB/sec execute 247 sec latency 231.783 ms 2024-09-11T01:18:01.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:01 smithi110 ceph-mon[34712]: pgmap v921: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 705 op/s 2024-09-11T01:18:01.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:01 smithi112 ceph-mon[35384]: pgmap v921: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 705 op/s 2024-09-11T01:18:02.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 239931 34.69 MB/sec execute 248 sec latency 855.820 ms 2024-09-11T01:18:03.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.279+0000 7fcb59840640 1 -- 172.21.15.78:0/3715417170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c098030 msgr2=0x7fcb4c098490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:03.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.279+0000 7fcb59840640 1 --2- 172.21.15.78:0/3715417170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c098030 0x7fcb4c098490 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009900 tx=0x7fcb4802a410 comp rx=0 tx=0).stop 2024-09-11T01:18:03.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.280+0000 7fcb59840640 1 -- 172.21.15.78:0/3715417170 shutdown_connections 2024-09-11T01:18:03.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.280+0000 7fcb59840640 1 --2- 172.21.15.78:0/3715417170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb4c0989d0 0x7fcb4c0a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.280+0000 7fcb59840640 1 --2- 172.21.15.78:0/3715417170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c098030 0x7fcb4c098490 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.280+0000 7fcb59840640 1 --2- 172.21.15.78:0/3715417170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c006720 0x7fcb4c006b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.280+0000 7fcb59840640 1 -- 172.21.15.78:0/3715417170 >> 172.21.15.78:0/3715417170 conn(0x7fcb4c00b150 msgr2=0x7fcb4c00c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.280+0000 7fcb59840640 1 -- 172.21.15.78:0/3715417170 shutdown_connections 2024-09-11T01:18:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.280+0000 7fcb59840640 1 -- 172.21.15.78:0/3715417170 wait complete. 2024-09-11T01:18:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.281+0000 7fcb59840640 1 Processor -- start 2024-09-11T01:18:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.282+0000 7fcb59840640 1 -- start start 2024-09-11T01:18:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.282+0000 7fcb59840640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c006720 0x7fcb4c12bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.282+0000 7fcb59840640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c098030 0x7fcb4c12c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.282+0000 7fcb5883e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c006720 0x7fcb4c12bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb5883e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c006720 0x7fcb4c12bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38888/0 (socket says 172.21.15.78:38888) 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb5883e640 1 -- 172.21.15.78:0/1202574910 learned_addr learned my addr 172.21.15.78:0/1202574910 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb53fff640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c098030 0x7fcb4c12c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb59840640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb4c0989d0 0x7fcb4c131680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb59840640 1 -- 172.21.15.78:0/1202574910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c0a58b0 con 0x7fcb4c098030 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb59840640 1 -- 172.21.15.78:0/1202574910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c0a5730 con 0x7fcb4c006720 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb59840640 1 -- 172.21.15.78:0/1202574910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c0a5a30 con 0x7fcb4c0989d0 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb53fff640 1 -- 172.21.15.78:0/1202574910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb4c0989d0 msgr2=0x7fcb4c131680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb53fff640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb4c0989d0 0x7fcb4c131680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb53fff640 1 -- 172.21.15.78:0/1202574910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c006720 msgr2=0x7fcb4c12bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb53fff640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c006720 0x7fcb4c12bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb53fff640 1 -- 172.21.15.78:0/1202574910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c131e50 con 0x7fcb4c098030 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.283+0000 7fcb53fff640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c098030 0x7fcb4c12c2e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fcb4802aa00 tx=0x7fcb48003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.284+0000 7fcb51ffb640 1 -- 172.21.15.78:0/1202574910 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4803f070 con 0x7fcb4c098030 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.284+0000 7fcb51ffb640 1 -- 172.21.15.78:0/1202574910 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb48003e40 con 0x7fcb4c098030 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.284+0000 7fcb59840640 1 -- 172.21.15.78:0/1202574910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4c132080 con 0x7fcb4c098030 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.284+0000 7fcb51ffb640 1 -- 172.21.15.78:0/1202574910 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48035030 con 0x7fcb4c098030 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.286+0000 7fcb59840640 1 -- 172.21.15.78:0/1202574910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4c094920 con 0x7fcb4c098030 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.292+0000 7fcb5ce77740 1 Processor -- start 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.292+0000 7fcb5ce77740 1 -- start start 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.293+0000 7fcb5ce77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 0x55c6277eda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.293+0000 7fcb5ce77740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277edfc0 0x55c6277f9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.293+0000 7fcb5ce77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f9f50 0x55c6277fc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.293+0000 7fcb5ce77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c627763ea0 con 0x55c6277f9f50 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.293+0000 7fcb5ce77740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c6277532d0 con 0x55c6277edfc0 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.293+0000 7fcb5ce77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c6277570e0 con 0x55c6277ed6a0 2024-09-11T01:18:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.293+0000 7fcb5903f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 0x55c6277eda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.293+0000 7fcb5903f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 0x55c6277eda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53144/0 (socket says 172.21.15.78:53144) 2024-09-11T01:18:03.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.293+0000 7fcb5903f640 1 -- 172.21.15.78:0/235236383 learned_addr learned my addr 172.21.15.78:0/235236383 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.293+0000 7fcb51ffb640 1 -- 172.21.15.78:0/1202574910 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fcb4803b460 con 0x7fcb4c098030 2024-09-11T01:18:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.293+0000 7fcb53fff640 1 --2- 172.21.15.78:0/235236383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f9f50 0x55c6277fc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.294+0000 7fcb5903f640 1 -- 172.21.15.78:0/235236383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277edfc0 msgr2=0x55c6277f9a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:18:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.294+0000 7fcb5903f640 1 --2- 172.21.15.78:0/235236383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277edfc0 0x55c6277f9a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.294+0000 7fcb5903f640 1 -- 172.21.15.78:0/235236383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f9f50 msgr2=0x55c6277fc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.294+0000 7fcb5903f640 1 --2- 172.21.15.78:0/235236383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f9f50 0x55c6277fc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.294+0000 7fcb5903f640 1 -- 172.21.15.78:0/235236383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6274b1650 con 0x55c6277ed6a0 2024-09-11T01:18:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.294+0000 7fcb5903f640 1 --2- 172.21.15.78:0/235236383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 0x55c6277eda80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009720 tx=0x7fcb40009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd5521668af0419 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.295+0000 7fcb51ffb640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcb440b5650 0x7fcb440b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.295+0000 7fcb3effd640 1 -- 172.21.15.78:0/235236383 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40015020 con 0x55c6277ed6a0 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.295+0000 7fcb3effd640 1 -- 172.21.15.78:0/235236383 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb40009e30 con 0x55c6277ed6a0 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.295+0000 7fcb3effd640 1 -- 172.21.15.78:0/235236383 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40010380 con 0x55c6277ed6a0 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.295+0000 7fcb51ffb640 1 -- 172.21.15.78:0/1202574910 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8493+0+0 (secure 0 0 0) 0x7fcb481069d0 con 0x7fcb4c098030 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.296+0000 7fcb5ce77740 1 -- 172.21.15.78:0/235236383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 msgr2=0x55c6277eda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.296+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/235236383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 0x55c6277eda80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009720 tx=0x7fcb40009be0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.296+0000 7fcb5ce77740 1 -- 172.21.15.78:0/235236383 shutdown_connections 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.296+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/235236383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f9f50 0x55c6277fc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.296+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/235236383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277edfc0 0x55c6277f9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.296+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/235236383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 0x55c6277eda80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.296+0000 7fcb5ce77740 1 -- 172.21.15.78:0/235236383 >> 172.21.15.78:0/235236383 conn(0x55c6277eb4d0 msgr2=0x55c6277ec670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.297+0000 7fcb5ce77740 1 -- 172.21.15.78:0/235236383 shutdown_connections 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.298+0000 7fcb5883e640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcb440b5650 0x7fcb440b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.298+0000 7fcb5ce77740 1 -- 172.21.15.78:0/235236383 wait complete. 2024-09-11T01:18:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.299+0000 7fcb5ce77740 1 -- start start 2024-09-11T01:18:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.299+0000 7fcb5ce77740 1 Processor -- start 2024-09-11T01:18:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.300+0000 7fcb5883e640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcb440b5650 0x7fcb440b7b10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcb540511b0 tx=0x7fcb54079040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.300+0000 7fcb5ce77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277ed6a0 0x55c6277f9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.300+0000 7fcb5ce77740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277f9f50 0x55c6277f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.300+0000 7fcb5ce77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277f73d0 0x55c6277f7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.300+0000 7fcb5ce77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c6275b4700 con 0x55c6277ed6a0 2024-09-11T01:18:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.300+0000 7fcb5ce77740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c6276ee160 con 0x55c6277f9f50 2024-09-11T01:18:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.300+0000 7fcb5ce77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c62780dbe0 con 0x55c6277f73d0 2024-09-11T01:18:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.300+0000 7fcb5ce77740 10 client.0 _ll_register_callbacks cb 0x55c6277079e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.301+0000 7fcb5883e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277ed6a0 0x55c6277f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.301+0000 7fcb5883e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277ed6a0 0x55c6277f9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45930/0 (socket says 172.21.15.78:45930) 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.301+0000 7fcb5883e640 1 -- 172.21.15.78:0/361338475 learned_addr learned my addr 172.21.15.78:0/361338475 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.301+0000 7fcb53fff640 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277f9f50 0x55c6277f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.301+0000 7fcb5903f640 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277f73d0 0x55c6277f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.303+0000 7fcb53fff640 1 -- 172.21.15.78:0/361338475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277f73d0 msgr2=0x55c6277f7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.303+0000 7fcb53fff640 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277f73d0 0x55c6277f7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.303+0000 7fcb53fff640 1 -- 172.21.15.78:0/361338475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277ed6a0 msgr2=0x55c6277f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.303+0000 7fcb53fff640 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277ed6a0 0x55c6277f9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.303+0000 7fcb53fff640 1 -- 172.21.15.78:0/361338475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62771bc90 con 0x55c6277f9f50 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.303+0000 7fcb53fff640 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277f9f50 0x55c6277f6da0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb481040e0 tx=0x7fcb48031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d70ee8e94b823d15 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.303+0000 7fcb33fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.304+0000 7fcb33fff640 1 -- 172.21.15.78:0/361338475 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48049040 con 0x55c6277f9f50 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.304+0000 7fcb33fff640 1 -- 172.21.15.78:0/361338475 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8493+0+0 (secure 0 0 0) 0x7fcb480cd280 con 0x55c6277f9f50 2024-09-11T01:18:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.305+0000 7fcb5ce77740 1 -- 172.21.15.78:0/361338475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb40009e30 con 0x55c6277f9f50 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.306+0000 7fcb33fff640 1 -- 172.21.15.78:0/361338475 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fcb480cc580 con 0x55c6277f9f50 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.306+0000 7fcb5ce77740 10 client.27908 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.306+0000 7fcb5ce77740 1 -- 172.21.15.78:0/361338475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55c627819a50 con 0x55c6277f9f50 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.307+0000 7fcb33fff640 1 -- 172.21.15.78:0/361338475 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb40009e30 con 0x55c6277f9f50 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.307+0000 7fcb33fff640 1 -- 172.21.15.78:0/361338475 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7fcb480d5f10 con 0x55c6277f9f50 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.307+0000 7fcb5ce77740 10 client.27908 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.307+0000 7fcb5ce77740 10 client.27908 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.307+0000 7fcb5ce77740 20 client.27908 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.307+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c62781af50 0x55c62781d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.307+0000 7fcb5ce77740 4 client.27908 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.307+0000 7fcb5ce77740 1 -- 172.21.15.78:0/361338475 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c627765df0 con 0x55c62781af50 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.308+0000 7fcb5883e640 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c62781af50 0x55c62781d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.309+0000 7fcb5883e640 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c62781af50 0x55c62781d370 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f5f19eb3bc26900 server_cookie=2c8f35eba9fdc994 in_seq=0 out_seq=0 2024-09-11T01:18:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.309+0000 7fcb33fff640 10 client.27908 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.310+0000 7fcb33fff640 1 -- 172.21.15.78:0/361338475 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c627765df0 con 0x55c62781af50 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.310+0000 7fcb33fff640 10 client.27908 handle_command_reply: tid=0 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.310+0000 7fcb5ce77740 1 client.27908 shutdown 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.310+0000 7fcb5ce77740 1 -- 172.21.15.78:0/361338475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277f9f50 msgr2=0x55c6277f6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.310+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277f9f50 0x55c6277f6da0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb481040e0 tx=0x7fcb48031400 comp rx=0 tx=0).stop 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.311+0000 7fcb5ce77740 1 -- 172.21.15.78:0/361338475 shutdown_connections 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.311+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c62781af50 0x55c62781d370 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.311+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277f73d0 0x55c6277f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.311+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277f9f50 0x55c6277f6da0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.311+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/361338475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277ed6a0 0x55c6277f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.311+0000 7fcb5ce77740 1 -- 172.21.15.78:0/361338475 >> 172.21.15.78:0/361338475 conn(0x55c6277eb4d0 msgr2=0x55c6277f1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.311+0000 7fcb33fff640 0 client.27908 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:18:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.312+0000 7fcb5ce77740 1 -- 172.21.15.78:0/361338475 shutdown_connections 2024-09-11T01:18:03.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.312+0000 7fcb5ce77740 1 -- 172.21.15.78:0/361338475 wait complete. 2024-09-11T01:18:03.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.312+0000 7fcb5ce77740 20 client.27908 trim_cache size 0 max 16384 2024-09-11T01:18:03.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.337+0000 7fcb5ce77740 1 Processor -- start 2024-09-11T01:18:03.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.337+0000 7fcb5ce77740 1 -- start start 2024-09-11T01:18:03.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.338+0000 7fcb5ce77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 0x55c6277f5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.338+0000 7fcb5ce77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f73d0 0x55c6277f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.338+0000 7fcb5ce77740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277f9f50 0x55c6277f4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.338+0000 7fcb53fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f73d0 0x55c6277f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.338+0000 7fcb5883e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 0x55c6277f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.338+0000 7fcb5883e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 0x55c6277f5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53166/0 (socket says 172.21.15.78:53166) 2024-09-11T01:18:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.338+0000 7fcb5883e640 1 -- 172.21.15.78:0/2245860630 learned_addr learned my addr 172.21.15.78:0/2245860630 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.338+0000 7fcb53fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f73d0 0x55c6277f5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45938/0 (socket says 172.21.15.78:45938) 2024-09-11T01:18:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.338+0000 7fcb5903f640 1 --2- 172.21.15.78:0/2245860630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277f9f50 0x55c6277f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.338+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2245860630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c13efb0 con 0x55c6277f73d0 2024-09-11T01:18:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.338+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2245860630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c140af0 con 0x55c6277f9f50 2024-09-11T01:18:03.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.338+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2245860630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c13f680 con 0x55c6277ed6a0 2024-09-11T01:18:03.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.339+0000 7fcb5883e640 1 -- 172.21.15.78:0/2245860630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277f9f50 msgr2=0x55c6277f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:03.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.339+0000 7fcb5883e640 1 --2- 172.21.15.78:0/2245860630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277f9f50 0x55c6277f4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.339+0000 7fcb5883e640 1 -- 172.21.15.78:0/2245860630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f73d0 msgr2=0x55c6277f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:03.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.339+0000 7fcb5883e640 1 --2- 172.21.15.78:0/2245860630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f73d0 0x55c6277f5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.339+0000 7fcb5883e640 1 -- 172.21.15.78:0/2245860630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6277ebc70 con 0x55c6277ed6a0 2024-09-11T01:18:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.339+0000 7fcb5883e640 1 --2- 172.21.15.78:0/2245860630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 0x55c6277f5680 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb54066520 tx=0x7fcb5407b070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8d369a8fc31c6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.340+0000 7fcb3e7fc640 1 -- 172.21.15.78:0/2245860630 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5406c990 con 0x55c6277ed6a0 2024-09-11T01:18:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.340+0000 7fcb3e7fc640 1 -- 172.21.15.78:0/2245860630 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb54094470 con 0x55c6277ed6a0 2024-09-11T01:18:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.340+0000 7fcb3e7fc640 1 -- 172.21.15.78:0/2245860630 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb540856a0 con 0x55c6277ed6a0 2024-09-11T01:18:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.341+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2245860630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 msgr2=0x55c6277f5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.341+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/2245860630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 0x55c6277f5680 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb54066520 tx=0x7fcb5407b070 comp rx=0 tx=0).stop 2024-09-11T01:18:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.342+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2245860630 shutdown_connections 2024-09-11T01:18:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.342+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/2245860630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277f9f50 0x55c6277f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.342+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/2245860630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f73d0 0x55c6277f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.342+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/2245860630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277ed6a0 0x55c6277f5680 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.342+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2245860630 >> 172.21.15.78:0/2245860630 conn(0x55c6277eb4d0 msgr2=0x55c6277ee9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:03.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.342+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2245860630 shutdown_connections 2024-09-11T01:18:03.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.342+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2245860630 wait complete. 2024-09-11T01:18:03.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.342+0000 7fcb5ce77740 1 -- start start 2024-09-11T01:18:03.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.343+0000 7fcb5ce77740 1 Processor -- start 2024-09-11T01:18:03.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.343+0000 7fcb5ce77740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277ed6a0 0x55c6278130e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.343+0000 7fcb5ce77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f73d0 0x55c627813640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.343+0000 7fcb5ce77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277f9f50 0x55c62780fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.343+0000 7fcb5ce77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c140420 con 0x55c6277f73d0 2024-09-11T01:18:03.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.343+0000 7fcb5ce77740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c13fd50 con 0x55c6277ed6a0 2024-09-11T01:18:03.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.343+0000 7fcb5ce77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c13cfb0 con 0x55c6277f9f50 2024-09-11T01:18:03.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.343+0000 7fcb5ce77740 10 client.0 _ll_register_callbacks cb 0x7fcb48104590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:03.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.343+0000 7fcb5903f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277f9f50 0x55c62780fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.343+0000 7fcb5903f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277f9f50 0x55c62780fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53174/0 (socket says 172.21.15.78:53174) 2024-09-11T01:18:03.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.343+0000 7fcb5903f640 1 -- 172.21.15.78:0/2964036583 learned_addr learned my addr 172.21.15.78:0/2964036583 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:03.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.344+0000 7fcb5903f640 1 -- 172.21.15.78:0/2964036583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277ed6a0 msgr2=0x55c6278130e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:18:03.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.344+0000 7fcb5903f640 1 --2- 172.21.15.78:0/2964036583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277ed6a0 0x55c6278130e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.344+0000 7fcb5903f640 1 -- 172.21.15.78:0/2964036583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f73d0 msgr2=0x55c627813640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:18:03.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.344+0000 7fcb5903f640 1 --2- 172.21.15.78:0/2964036583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f73d0 0x55c627813640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.344+0000 7fcb5903f640 1 -- 172.21.15.78:0/2964036583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c627810660 con 0x55c6277f9f50 2024-09-11T01:18:03.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.346+0000 7fcb5903f640 1 --2- 172.21.15.78:0/2964036583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277f9f50 0x55c62780fed0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcb400096f0 tx=0x7fcb40009440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a507063c4c3b37cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:03.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.346+0000 7fcb33fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:18:03.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.346+0000 7fcb33fff640 1 -- 172.21.15.78:0/2964036583 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40028060 con 0x55c6277f9f50 2024-09-11T01:18:03.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.346+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2964036583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb54094470 con 0x55c6277f9f50 2024-09-11T01:18:03.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.346+0000 7fcb33fff640 1 -- 172.21.15.78:0/2964036583 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8493+0+0 (secure 0 0 0) 0x7fcb40011630 con 0x55c6277f9f50 2024-09-11T01:18:03.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.347+0000 7fcb33fff640 1 -- 172.21.15.78:0/2964036583 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fcb40015040 con 0x55c6277f9f50 2024-09-11T01:18:03.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.347+0000 7fcb5ce77740 10 client.27859 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:03.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.347+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2964036583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55c6278888d0 con 0x55c6277f9f50 2024-09-11T01:18:03.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.348+0000 7fcb33fff640 1 -- 172.21.15.78:0/2964036583 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb54094470 con 0x55c6277f9f50 2024-09-11T01:18:03.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.348+0000 7fcb33fff640 1 -- 172.21.15.78:0/2964036583 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7fcb4001a8b0 con 0x55c6277f9f50 2024-09-11T01:18:03.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.349+0000 7fcb5ce77740 10 client.27859 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:03.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.349+0000 7fcb5ce77740 10 client.27859 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:18:03.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.349+0000 7fcb5ce77740 20 client.27859 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:03.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.349+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/2964036583 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c627888e70 0x55c62788b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:03.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.349+0000 7fcb5ce77740 4 client.27859 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:18:03.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.349+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2964036583 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6277618a0 con 0x55c627888e70 2024-09-11T01:18:03.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.352+0000 7fcb5883e640 1 --2- 172.21.15.78:0/2964036583 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c627888e70 0x55c62788b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:03.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.354+0000 7fcb5883e640 1 --2- 172.21.15.78:0/2964036583 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c627888e70 0x55c62788b290 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c58a6faaf1324b2 server_cookie=1b5a74d431eee0a3 in_seq=0 out_seq=0 2024-09-11T01:18:03.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.354+0000 7fcb33fff640 10 client.27859 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:18:03.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb33fff640 1 -- 172.21.15.78:0/2964036583 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6277618a0 con 0x55c627888e70 2024-09-11T01:18:03.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb33fff640 10 client.27859 handle_command_reply: tid=0 2024-09-11T01:18:03.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb5ce77740 1 client.27859 shutdown 2024-09-11T01:18:03.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2964036583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277f9f50 msgr2=0x55c62780fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:03.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/2964036583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277f9f50 0x55c62780fed0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcb400096f0 tx=0x7fcb40009440 comp rx=0 tx=0).stop 2024-09-11T01:18:03.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2964036583 shutdown_connections 2024-09-11T01:18:03.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/2964036583 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x55c627888e70 0x55c62788b290 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb33fff640 0 client.27859 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:18:03.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/2964036583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c6277f9f50 0x55c62780fed0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/2964036583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6277f73d0 0x55c627813640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb5ce77740 1 --2- 172.21.15.78:0/2964036583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6277ed6a0 0x55c6278130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2964036583 >> 172.21.15.78:0/2964036583 conn(0x55c6277eb4d0 msgr2=0x55c6277ee9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:03.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2964036583 shutdown_connections 2024-09-11T01:18:03.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.355+0000 7fcb5ce77740 1 -- 172.21.15.78:0/2964036583 wait complete. 2024-09-11T01:18:03.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.356+0000 7fcb5ce77740 20 client.27859 trim_cache size 0 max 16384 2024-09-11T01:18:03.358 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:18:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.356+0000 7fcb59840640 1 -- 172.21.15.78:0/1202574910 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcb440b5650 msgr2=0x7fcb440b7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.356+0000 7fcb59840640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcb440b5650 0x7fcb440b7b10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcb540511b0 tx=0x7fcb54079040 comp rx=0 tx=0).stop 2024-09-11T01:18:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.356+0000 7fcb59840640 1 -- 172.21.15.78:0/1202574910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c098030 msgr2=0x7fcb4c12c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.356+0000 7fcb59840640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c098030 0x7fcb4c12c2e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fcb4802aa00 tx=0x7fcb48003cd0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.357+0000 7fcb59840640 1 -- 172.21.15.78:0/1202574910 shutdown_connections 2024-09-11T01:18:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.357+0000 7fcb59840640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcb440b5650 0x7fcb440b7b10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.357+0000 7fcb59840640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb4c0989d0 0x7fcb4c131680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.357+0000 7fcb59840640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb4c098030 0x7fcb4c12c2e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.357+0000 7fcb59840640 1 --2- 172.21.15.78:0/1202574910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb4c006720 0x7fcb4c12bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.357+0000 7fcb59840640 1 -- 172.21.15.78:0/1202574910 >> 172.21.15.78:0/1202574910 conn(0x7fcb4c00b150 msgr2=0x7fcb4c09b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.357+0000 7fcb59840640 1 -- 172.21.15.78:0/1202574910 shutdown_connections 2024-09-11T01:18:03.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:03.357+0000 7fcb59840640 1 -- 172.21.15.78:0/1202574910 wait complete. 2024-09-11T01:18:03.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 240604 34.71 MB/sec execute 249 sec latency 141.948 ms 2024-09-11T01:18:03.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:03 smithi110 ceph-mon[34712]: pgmap v922: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 17 MiB/s wr, 775 op/s 2024-09-11T01:18:03.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:03 smithi110 ceph-mon[34712]: osdmap e102: 12 total, 12 up, 12 in 2024-09-11T01:18:03.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:03 smithi112 ceph-mon[35384]: pgmap v922: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 17 MiB/s wr, 775 op/s 2024-09-11T01:18:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:03 smithi112 ceph-mon[35384]: osdmap e102: 12 total, 12 up, 12 in 2024-09-11T01:18:03.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:03 smithi078 ceph-mon[31315]: pgmap v922: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 17 MiB/s wr, 775 op/s 2024-09-11T01:18:03.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:03 smithi078 ceph-mon[31315]: osdmap e102: 12 total, 12 up, 12 in 2024-09-11T01:18:04.279 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:18:04.634 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:18:04.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 241410 34.77 MB/sec execute 250 sec latency 85.282 ms 2024-09-11T01:18:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.178+0000 7f80bbf64640 1 -- 172.21.15.78:0/3254010131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b410d120 msgr2=0x7f80b410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.178+0000 7f80bbf64640 1 --2- 172.21.15.78:0/3254010131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b410d120 0x7f80b410f550 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009be0 tx=0x7f80ac02a6c0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.178+0000 7f80bbf64640 1 -- 172.21.15.78:0/3254010131 shutdown_connections 2024-09-11T01:18:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.178+0000 7f80bbf64640 1 --2- 172.21.15.78:0/3254010131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b410d120 0x7f80b410f550 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.178+0000 7f80bbf64640 1 --2- 172.21.15.78:0/3254010131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b4104570 0x7f80b410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.178+0000 7f80bbf64640 1 --2- 172.21.15.78:0/3254010131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80b4103b70 0x7f80b4103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.178+0000 7f80bbf64640 1 -- 172.21.15.78:0/3254010131 >> 172.21.15.78:0/3254010131 conn(0x7f80b40fd700 msgr2=0x7f80b40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.179+0000 7f80bbf64640 1 -- 172.21.15.78:0/3254010131 shutdown_connections 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.179+0000 7f80bbf64640 1 -- 172.21.15.78:0/3254010131 wait complete. 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80bbf64640 1 Processor -- start 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80bbf64640 1 -- start start 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80bbf64640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80b4103b70 0x7f80b419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80bbf64640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b4104570 0x7f80b419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80bbf64640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b410d120 0x7f80b41a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80b9d00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80b4103b70 0x7f80b419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80b9d00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80b4103b70 0x7f80b419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45950/0 (socket says 172.21.15.78:45950) 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80b9d00640 1 -- 172.21.15.78:0/2562680961 learned_addr learned my addr 172.21.15.78:0/2562680961 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80b94ff640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b4104570 0x7f80b419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80bbf64640 1 -- 172.21.15.78:0/2562680961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4112d60 con 0x7f80b4103b70 2024-09-11T01:18:05.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80bbf64640 1 -- 172.21.15.78:0/2562680961 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4112be0 con 0x7f80b4104570 2024-09-11T01:18:05.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80bbf64640 1 -- 172.21.15.78:0/2562680961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4112ee0 con 0x7f80b410d120 2024-09-11T01:18:05.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.180+0000 7f80ba501640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b410d120 0x7f80b41a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.181+0000 7f80ba501640 1 -- 172.21.15.78:0/2562680961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b4104570 msgr2=0x7f80b419b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:05.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.181+0000 7f80ba501640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b4104570 0x7f80b419b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.181+0000 7f80ba501640 1 -- 172.21.15.78:0/2562680961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80b4103b70 msgr2=0x7f80b419b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:05.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.181+0000 7f80ba501640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80b4103b70 0x7f80b419b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.181+0000 7f80ba501640 1 -- 172.21.15.78:0/2562680961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b41a14e0 con 0x7f80b410d120 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.181+0000 7f80b9d00640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80b4103b70 0x7f80b419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.181+0000 7f80ba501640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b410d120 0x7f80b41a0d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f80ac02abe0 tx=0x7f80ac003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.181+0000 7f80aaffd640 1 -- 172.21.15.78:0/2562680961 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac041020 con 0x7f80b410d120 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.182+0000 7f80bbf64640 1 -- 172.21.15.78:0/2562680961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b41a1770 con 0x7f80b410d120 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.182+0000 7f80bbf64640 1 -- 172.21.15.78:0/2562680961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b41a7e90 con 0x7f80b410d120 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.182+0000 7f80aaffd640 1 -- 172.21.15.78:0/2562680961 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80ac003e10 con 0x7f80b410d120 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.182+0000 7f80aaffd640 1 -- 172.21.15.78:0/2562680961 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac036040 con 0x7f80b410d120 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.184+0000 7f80be339740 1 Processor -- start 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.185+0000 7f80aaffd640 1 -- 172.21.15.78:0/2562680961 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f80ac0045c0 con 0x7f80b410d120 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.185+0000 7f80be339740 1 -- start start 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80be339740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b04d0 0x55ab622b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80be339740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b0df0 0x55ab622bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80be339740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 0x55ab622bf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80be339740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ab62226ea0 con 0x55ab622b0df0 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80be339740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ab622162d0 con 0x55ab622bcd80 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80be339740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ab6221a0e0 con 0x55ab622b04d0 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80b9d00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b0df0 0x55ab622bc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80b9d00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b0df0 0x55ab622bc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45966/0 (socket says 172.21.15.78:45966) 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80b9d00640 1 -- 172.21.15.78:0/262803713 learned_addr learned my addr 172.21.15.78:0/262803713 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80b94ff640 1 --2- 172.21.15.78:0/262803713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b04d0 0x55ab622b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80b94ff640 1 --2- 172.21.15.78:0/262803713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 0x55ab622bf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80b9d00640 1 -- 172.21.15.78:0/262803713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b04d0 msgr2=0x55ab622b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80b9d00640 1 --2- 172.21.15.78:0/262803713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b04d0 0x55ab622b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80b9d00640 1 -- 172.21.15.78:0/262803713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 msgr2=0x55ab622bf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80b9d00640 1 --2- 172.21.15.78:0/262803713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 0x55ab622bf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80b9d00640 1 -- 172.21.15.78:0/262803713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab61f74650 con 0x55ab622b0df0 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80b94ff640 1 --2- 172.21.15.78:0/262803713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 0x55ab622bf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:18:05.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.186+0000 7f80b9d00640 1 --2- 172.21.15.78:0/262803713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b0df0 0x55ab622bc840 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f80b000f540 tx=0x7f80b000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c1c4a6dba4b6ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:05.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.187+0000 7f809ffff640 1 -- 172.21.15.78:0/262803713 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0026070 con 0x55ab622b0df0 2024-09-11T01:18:05.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.187+0000 7f809ffff640 1 -- 172.21.15.78:0/262803713 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b000eae0 con 0x55ab622b0df0 2024-09-11T01:18:05.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.187+0000 7f809ffff640 1 -- 172.21.15.78:0/262803713 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0021bb0 con 0x55ab622b0df0 2024-09-11T01:18:05.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.188+0000 7f80aaffd640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80a00b5450 0x7f80a00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.188+0000 7f80b94ff640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80a00b5450 0x7f80a00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.188+0000 7f80be339740 1 -- 172.21.15.78:0/262803713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b0df0 msgr2=0x55ab622bc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:05.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.188+0000 7f80be339740 1 --2- 172.21.15.78:0/262803713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b0df0 0x55ab622bc840 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f80b000f540 tx=0x7f80b000fa00 comp rx=0 tx=0).stop 2024-09-11T01:18:05.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.188+0000 7f80be339740 1 -- 172.21.15.78:0/262803713 shutdown_connections 2024-09-11T01:18:05.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.188+0000 7f80aaffd640 1 -- 172.21.15.78:0/2562680961 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8493+0+0 (secure 0 0 0) 0x7f80ac106560 con 0x7f80b410d120 2024-09-11T01:18:05.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.188+0000 7f80be339740 1 --2- 172.21.15.78:0/262803713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 0x55ab622bf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.188+0000 7f80be339740 1 --2- 172.21.15.78:0/262803713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b0df0 0x55ab622bc840 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.188+0000 7f80be339740 1 --2- 172.21.15.78:0/262803713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b04d0 0x55ab622b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.188+0000 7f80be339740 1 -- 172.21.15.78:0/262803713 >> 172.21.15.78:0/262803713 conn(0x55ab622ae300 msgr2=0x55ab622af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.189+0000 7f80be339740 1 -- 172.21.15.78:0/262803713 shutdown_connections 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.189+0000 7f80be339740 1 -- 172.21.15.78:0/262803713 wait complete. 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.189+0000 7f80b94ff640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80a00b5450 0x7f80a00b7910 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009730 tx=0x7f80a4013040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.189+0000 7f80be339740 1 -- start start 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.189+0000 7f80be339740 1 Processor -- start 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.190+0000 7f80be339740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b04d0 0x55ab622bc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.190+0000 7f80be339740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622b0df0 0x55ab622b9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.190+0000 7f80be339740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622bcd80 0x55ab622b9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.190+0000 7f80be339740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ab62077700 con 0x55ab622b04d0 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.190+0000 7f80be339740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ab621b1160 con 0x55ab622b0df0 2024-09-11T01:18:05.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.190+0000 7f80be339740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ab622bfa50 con 0x55ab622bcd80 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.190+0000 7f80be339740 10 client.0 _ll_register_callbacks cb 0x55ab621ca9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.190+0000 7f80b9d00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b04d0 0x55ab622bc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.190+0000 7f80b9d00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b04d0 0x55ab622bc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45972/0 (socket says 172.21.15.78:45972) 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.190+0000 7f80ba501640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622bcd80 0x55ab622b9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.190+0000 7f80b9d00640 1 -- 172.21.15.78:0/1030385689 learned_addr learned my addr 172.21.15.78:0/1030385689 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.190+0000 7f80b94ff640 1 --2- 172.21.15.78:0/1030385689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622b0df0 0x55ab622b9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.191+0000 7f80b94ff640 1 -- 172.21.15.78:0/1030385689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622bcd80 msgr2=0x55ab622b9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.191+0000 7f80b94ff640 1 --2- 172.21.15.78:0/1030385689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622bcd80 0x55ab622b9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.191+0000 7f80b94ff640 1 -- 172.21.15.78:0/1030385689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b04d0 msgr2=0x55ab622bc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.191+0000 7f80b94ff640 1 --2- 172.21.15.78:0/1030385689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b04d0 0x55ab622bc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.191+0000 7f80b94ff640 1 -- 172.21.15.78:0/1030385689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab621dec90 con 0x55ab622b0df0 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.191+0000 7f80b94ff640 1 --2- 172.21.15.78:0/1030385689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622b0df0 0x55ab622b9a30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f80b41012c0 tx=0x7f80a40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c5b36e6bfa300f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.191+0000 7f809d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.191+0000 7f809d7fa640 1 -- 172.21.15.78:0/1030385689 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4021700 con 0x55ab622b0df0 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.191+0000 7f809d7fa640 1 -- 172.21.15.78:0/1030385689 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8493+0+0 (secure 0 0 0) 0x7f80a4009040 con 0x55ab622b0df0 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.191+0000 7f80be339740 1 -- 172.21.15.78:0/1030385689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80b000eae0 con 0x55ab622b0df0 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.192+0000 7f809d7fa640 1 -- 172.21.15.78:0/1030385689 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f80a40218a0 con 0x55ab622b0df0 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.192+0000 7f80be339740 10 client.27923 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.192+0000 7f80be339740 1 -- 172.21.15.78:0/1030385689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55ab622cb3f0 con 0x55ab622b0df0 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.192+0000 7f809d7fa640 1 -- 172.21.15.78:0/1030385689 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b000eae0 con 0x55ab622b0df0 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.193+0000 7f809d7fa640 1 -- 172.21.15.78:0/1030385689 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f80a402bcb0 con 0x55ab622b0df0 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.193+0000 7f80be339740 10 client.27923 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.193+0000 7f80be339740 10 client.27923 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.193+0000 7f80be339740 20 client.27923 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.193+0000 7f80be339740 1 --2- 172.21.15.78:0/1030385689 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55ab622cc950 0x55ab622ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.193+0000 7f80be339740 4 client.27923 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:18:05.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.193+0000 7f80be339740 1 -- 172.21.15.78:0/1030385689 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab62228df0 con 0x55ab622cc950 2024-09-11T01:18:05.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.193+0000 7f80b9d00640 1 --2- 172.21.15.78:0/1030385689 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55ab622cc950 0x55ab622ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.196+0000 7f80b9d00640 1 --2- 172.21.15.78:0/1030385689 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55ab622cc950 0x55ab622ceda0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff41bc2cec4f43f9 server_cookie=76477d87c5e8a2e2 in_seq=0 out_seq=0 2024-09-11T01:18:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.196+0000 7f809d7fa640 10 client.27923 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:18:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.197+0000 7f809d7fa640 1 -- 172.21.15.78:0/1030385689 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab62228df0 con 0x55ab622cc950 2024-09-11T01:18:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.197+0000 7f809d7fa640 10 client.27923 handle_command_reply: tid=0 2024-09-11T01:18:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f80be339740 1 client.27923 shutdown 2024-09-11T01:18:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f80be339740 1 -- 172.21.15.78:0/1030385689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622b0df0 msgr2=0x55ab622b9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f80be339740 1 --2- 172.21.15.78:0/1030385689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622b0df0 0x55ab622b9a30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f80b41012c0 tx=0x7f80a40046c0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f80be339740 1 -- 172.21.15.78:0/1030385689 shutdown_connections 2024-09-11T01:18:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f80be339740 1 --2- 172.21.15.78:0/1030385689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622bcd80 0x55ab622b9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f80be339740 1 --2- 172.21.15.78:0/1030385689 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55ab622cc950 0x55ab622ceda0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f80be339740 1 --2- 172.21.15.78:0/1030385689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622b0df0 0x55ab622b9a30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f80be339740 1 --2- 172.21.15.78:0/1030385689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b04d0 0x55ab622bc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f80be339740 1 -- 172.21.15.78:0/1030385689 >> 172.21.15.78:0/1030385689 conn(0x55ab622ae300 msgr2=0x55ab622b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:05.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f809d7fa640 0 client.27923 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:18:05.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f80be339740 1 -- 172.21.15.78:0/1030385689 shutdown_connections 2024-09-11T01:18:05.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f80be339740 1 -- 172.21.15.78:0/1030385689 wait complete. 2024-09-11T01:18:05.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.198+0000 7f80be339740 20 client.27923 trim_cache size 0 max 16384 2024-09-11T01:18:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.224+0000 7f80be339740 1 Processor -- start 2024-09-11T01:18:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.224+0000 7f80be339740 1 -- start start 2024-09-11T01:18:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.224+0000 7f80be339740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b04d0 0x55ab622c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.224+0000 7f80be339740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b0df0 0x55ab622ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.224+0000 7f80be339740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 0x55ab622b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.224+0000 7f80be339740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f809800e850 con 0x55ab622b0df0 2024-09-11T01:18:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.224+0000 7f80be339740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8098010390 con 0x55ab622bcd80 2024-09-11T01:18:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.224+0000 7f80be339740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f809800ef20 con 0x55ab622b04d0 2024-09-11T01:18:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.225+0000 7f80ba501640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 0x55ab622b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.225+0000 7f80ba501640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 0x55ab622b82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38978/0 (socket says 172.21.15.78:38978) 2024-09-11T01:18:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.225+0000 7f80ba501640 1 -- 172.21.15.78:0/2026197628 learned_addr learned my addr 172.21.15.78:0/2026197628 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.225+0000 7f80b9d00640 1 --2- 172.21.15.78:0/2026197628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b04d0 0x55ab622c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.225+0000 7f80b94ff640 1 --2- 172.21.15.78:0/2026197628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b0df0 0x55ab622ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.225+0000 7f80ba501640 1 -- 172.21.15.78:0/2026197628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b04d0 msgr2=0x55ab622c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.225+0000 7f80ba501640 1 --2- 172.21.15.78:0/2026197628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b04d0 0x55ab622c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.225+0000 7f80ba501640 1 -- 172.21.15.78:0/2026197628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b0df0 msgr2=0x55ab622ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.225+0000 7f80ba501640 1 --2- 172.21.15.78:0/2026197628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b0df0 0x55ab622ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.225+0000 7f80ba501640 1 -- 172.21.15.78:0/2026197628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab622b0c20 con 0x55ab622bcd80 2024-09-11T01:18:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.226+0000 7f80ba501640 1 --2- 172.21.15.78:0/2026197628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 0x55ab622b82b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f80ac006010 tx=0x7f80ac0c8920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdf1c7e3213108e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.226+0000 7f809f7fe640 1 -- 172.21.15.78:0/2026197628 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0c8170 con 0x55ab622bcd80 2024-09-11T01:18:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.226+0000 7f809f7fe640 1 -- 172.21.15.78:0/2026197628 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80ac0cc510 con 0x55ab622bcd80 2024-09-11T01:18:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.226+0000 7f809f7fe640 1 -- 172.21.15.78:0/2026197628 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0d5f90 con 0x55ab622bcd80 2024-09-11T01:18:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.227+0000 7f80be339740 1 -- 172.21.15.78:0/2026197628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 msgr2=0x55ab622b82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.227+0000 7f80be339740 1 --2- 172.21.15.78:0/2026197628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 0x55ab622b82b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f80ac006010 tx=0x7f80ac0c8920 comp rx=0 tx=0).stop 2024-09-11T01:18:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.227+0000 7f80be339740 1 -- 172.21.15.78:0/2026197628 shutdown_connections 2024-09-11T01:18:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.227+0000 7f80be339740 1 --2- 172.21.15.78:0/2026197628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622bcd80 0x55ab622b82b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.227+0000 7f80be339740 1 --2- 172.21.15.78:0/2026197628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622b0df0 0x55ab622ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.227+0000 7f80be339740 1 --2- 172.21.15.78:0/2026197628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b04d0 0x55ab622c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.228+0000 7f80be339740 1 -- 172.21.15.78:0/2026197628 >> 172.21.15.78:0/2026197628 conn(0x55ab622ae300 msgr2=0x55ab622aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.228+0000 7f80be339740 1 -- 172.21.15.78:0/2026197628 shutdown_connections 2024-09-11T01:18:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.228+0000 7f80be339740 1 -- 172.21.15.78:0/2026197628 wait complete. 2024-09-11T01:18:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.228+0000 7f80be339740 1 -- start start 2024-09-11T01:18:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.228+0000 7f80be339740 1 Processor -- start 2024-09-11T01:18:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.229+0000 7f80be339740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622b04d0 0x55ab622c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.229+0000 7f80be339740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b0df0 0x55ab622c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.229+0000 7f80be339740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622bcd80 0x55ab62332c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.229+0000 7f80be339740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f809800fcc0 con 0x55ab622bcd80 2024-09-11T01:18:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.229+0000 7f80be339740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f809800f5f0 con 0x55ab622b04d0 2024-09-11T01:18:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.229+0000 7f80be339740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f809800c850 con 0x55ab622b0df0 2024-09-11T01:18:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.229+0000 7f80be339740 10 client.0 _ll_register_callbacks cb 0x7f80a4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.229+0000 7f80b94ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b0df0 0x55ab622c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.229+0000 7f80b94ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b0df0 0x55ab622c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53212/0 (socket says 172.21.15.78:53212) 2024-09-11T01:18:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.229+0000 7f80b94ff640 1 -- 172.21.15.78:0/2152036630 learned_addr learned my addr 172.21.15.78:0/2152036630 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.229+0000 7f80ba501640 1 --2- 172.21.15.78:0/2152036630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622bcd80 0x55ab62332c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.230+0000 7f80b94ff640 1 -- 172.21.15.78:0/2152036630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622b04d0 msgr2=0x55ab622c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.230+0000 7f80b94ff640 1 --2- 172.21.15.78:0/2152036630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622b04d0 0x55ab622c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.230+0000 7f80b94ff640 1 -- 172.21.15.78:0/2152036630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622bcd80 msgr2=0x55ab62332c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.230+0000 7f80b94ff640 1 --2- 172.21.15.78:0/2152036630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622bcd80 0x55ab62332c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.230+0000 7f80b94ff640 1 -- 172.21.15.78:0/2152036630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab62333150 con 0x55ab622b0df0 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.230+0000 7f80b94ff640 1 --2- 172.21.15.78:0/2152036630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b0df0 0x55ab622c52b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f80a4003e50 tx=0x7f80a4021700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c69bde762be80c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.230+0000 7f809d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.230+0000 7f809d7fa640 1 -- 172.21.15.78:0/2152036630 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4038070 con 0x55ab622b0df0 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.230+0000 7f80be339740 1 -- 172.21.15.78:0/2152036630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80ac0cc510 con 0x55ab622b0df0 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.231+0000 7f809d7fa640 1 -- 172.21.15.78:0/2152036630 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8493+0+0 (secure 0 0 0) 0x7f80a40228d0 con 0x55ab622b0df0 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.231+0000 7f809d7fa640 1 -- 172.21.15.78:0/2152036630 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f80a4022af0 con 0x55ab622b0df0 2024-09-11T01:18:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.231+0000 7f80be339740 10 client.27877 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:05.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.231+0000 7f80be339740 1 -- 172.21.15.78:0/2152036630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55ab6234b0b0 con 0x55ab622b0df0 2024-09-11T01:18:05.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.232+0000 7f809d7fa640 1 -- 172.21.15.78:0/2152036630 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80ac0cc510 con 0x55ab622b0df0 2024-09-11T01:18:05.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.232+0000 7f809d7fa640 1 -- 172.21.15.78:0/2152036630 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f80a402b990 con 0x55ab622b0df0 2024-09-11T01:18:05.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.232+0000 7f80be339740 10 client.27877 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:05.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.232+0000 7f80be339740 10 client.27877 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:18:05.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.232+0000 7f80be339740 20 client.27877 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:05.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.232+0000 7f80be339740 1 --2- 172.21.15.78:0/2152036630 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55ab6234b860 0x55ab6234dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:05.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.232+0000 7f80be339740 4 client.27877 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:18:05.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.232+0000 7f80be339740 1 -- 172.21.15.78:0/2152036630 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab622cf130 con 0x55ab6234b860 2024-09-11T01:18:05.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.232+0000 7f80b9d00640 1 --2- 172.21.15.78:0/2152036630 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55ab6234b860 0x55ab6234dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:05.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.233+0000 7f809d7fa640 10 client.27877 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:18:05.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.233+0000 7f80b9d00640 1 --2- 172.21.15.78:0/2152036630 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55ab6234b860 0x55ab6234dc80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cb114fcc8ee9bc0 server_cookie=bb922d720585468b in_seq=0 out_seq=0 2024-09-11T01:18:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.237+0000 7f809d7fa640 1 -- 172.21.15.78:0/2152036630 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab622cf130 con 0x55ab6234b860 2024-09-11T01:18:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.237+0000 7f809d7fa640 10 client.27877 handle_command_reply: tid=0 2024-09-11T01:18:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.237+0000 7f80be339740 1 client.27877 shutdown 2024-09-11T01:18:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.237+0000 7f80be339740 1 -- 172.21.15.78:0/2152036630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b0df0 msgr2=0x55ab622c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.237+0000 7f80be339740 1 --2- 172.21.15.78:0/2152036630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b0df0 0x55ab622c52b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f80a4003e50 tx=0x7f80a4021700 comp rx=0 tx=0).stop 2024-09-11T01:18:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.237+0000 7f80be339740 1 -- 172.21.15.78:0/2152036630 shutdown_connections 2024-09-11T01:18:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.237+0000 7f80be339740 1 --2- 172.21.15.78:0/2152036630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab622bcd80 0x55ab62332c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.237+0000 7f80be339740 1 --2- 172.21.15.78:0/2152036630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab622b0df0 0x55ab622c52b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.237+0000 7f80be339740 1 --2- 172.21.15.78:0/2152036630 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55ab6234b860 0x55ab6234dc80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.237+0000 7f80be339740 1 --2- 172.21.15.78:0/2152036630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab622b04d0 0x55ab622c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.237+0000 7f809d7fa640 0 client.27877 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:18:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.237+0000 7f80be339740 1 -- 172.21.15.78:0/2152036630 >> 172.21.15.78:0/2152036630 conn(0x55ab622ae300 msgr2=0x55ab622b8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.238+0000 7f80be339740 1 -- 172.21.15.78:0/2152036630 shutdown_connections 2024-09-11T01:18:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.238+0000 7f80be339740 1 -- 172.21.15.78:0/2152036630 wait complete. 2024-09-11T01:18:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.238+0000 7f80be339740 20 client.27877 trim_cache size 0 max 16384 2024-09-11T01:18:05.241 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:18:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.239+0000 7f80bbf64640 1 -- 172.21.15.78:0/2562680961 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80a00b5450 msgr2=0x7f80a00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.239+0000 7f80bbf64640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80a00b5450 0x7f80a00b7910 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009730 tx=0x7f80a4013040 comp rx=0 tx=0).stop 2024-09-11T01:18:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.239+0000 7f80bbf64640 1 -- 172.21.15.78:0/2562680961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b410d120 msgr2=0x7f80b41a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.239+0000 7f80bbf64640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b410d120 0x7f80b41a0d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f80ac02abe0 tx=0x7f80ac003c20 comp rx=0 tx=0).stop 2024-09-11T01:18:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.239+0000 7f80bbf64640 1 -- 172.21.15.78:0/2562680961 shutdown_connections 2024-09-11T01:18:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.239+0000 7f80bbf64640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f80a00b5450 0x7f80a00b7910 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.239+0000 7f80bbf64640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b410d120 0x7f80b41a0d10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.239+0000 7f80bbf64640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b4104570 0x7f80b419b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.239+0000 7f80bbf64640 1 --2- 172.21.15.78:0/2562680961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80b4103b70 0x7f80b419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.239+0000 7f80bbf64640 1 -- 172.21.15.78:0/2562680961 >> 172.21.15.78:0/2562680961 conn(0x7f80b40fd700 msgr2=0x7f80b40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.240+0000 7f80bbf64640 1 -- 172.21.15.78:0/2562680961 shutdown_connections 2024-09-11T01:18:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:05.240+0000 7f80bbf64640 1 -- 172.21.15.78:0/2562680961 wait complete. 2024-09-11T01:18:05.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 242003 34.75 MB/sec execute 251 sec latency 72.401 ms 2024-09-11T01:18:05.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:05 smithi110 ceph-mon[34712]: pgmap v924: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 787 op/s 2024-09-11T01:18:05.949 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:05 smithi078 ceph-mon[31315]: pgmap v924: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 787 op/s 2024-09-11T01:18:05.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:05 smithi112 ceph-mon[35384]: pgmap v924: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 787 op/s 2024-09-11T01:18:05.994 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:18:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:18:06.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 242783 34.79 MB/sec execute 252 sec latency 75.092 ms 2024-09-11T01:18:06.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.883+0000 7f142f4f9640 1 -- 172.21.15.78:0/3898004867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1428113860 msgr2=0x7f1428115c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.883+0000 7f142f4f9640 1 --2- 172.21.15.78:0/3898004867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1428113860 0x7f1428115c50 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1420009ca0 tx=0x7f142002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.883+0000 7f142f4f9640 1 -- 172.21.15.78:0/3898004867 shutdown_connections 2024-09-11T01:18:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.883+0000 7f142f4f9640 1 --2- 172.21.15.78:0/3898004867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1428113860 0x7f1428115c50 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.883+0000 7f142f4f9640 1 --2- 172.21.15.78:0/3898004867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1428110f30 0x7f1428113320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.883+0000 7f142f4f9640 1 --2- 172.21.15.78:0/3898004867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1428101250 0x7f1428101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.883+0000 7f142f4f9640 1 -- 172.21.15.78:0/3898004867 >> 172.21.15.78:0/3898004867 conn(0x7f1428078140 msgr2=0x7f14280ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.883+0000 7f142f4f9640 1 -- 172.21.15.78:0/3898004867 shutdown_connections 2024-09-11T01:18:06.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.883+0000 7f142f4f9640 1 -- 172.21.15.78:0/3898004867 wait complete. 2024-09-11T01:18:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.884+0000 7f142f4f9640 1 Processor -- start 2024-09-11T01:18:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.884+0000 7f142f4f9640 1 -- start start 2024-09-11T01:18:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.884+0000 7f142f4f9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1428101250 0x7f142819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.884+0000 7f142f4f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1428110f30 0x7f142819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.884+0000 7f142d295640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1428101250 0x7f142819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.884+0000 7f142ca94640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1428110f30 0x7f142819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.884+0000 7f142ca94640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1428110f30 0x7f142819fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46008/0 (socket says 172.21.15.78:46008) 2024-09-11T01:18:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.884+0000 7f142d295640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1428101250 0x7f142819f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39004/0 (socket says 172.21.15.78:39004) 2024-09-11T01:18:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.884+0000 7f142f4f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1428113860 0x7f14281a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.884+0000 7f142ca94640 1 -- 172.21.15.78:0/243230041 learned_addr learned my addr 172.21.15.78:0/243230041 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.885+0000 7f142f4f9640 1 -- 172.21.15.78:0/243230041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14281193b0 con 0x7f1428110f30 2024-09-11T01:18:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.885+0000 7f142f4f9640 1 -- 172.21.15.78:0/243230041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1428119230 con 0x7f1428101250 2024-09-11T01:18:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.885+0000 7f142f4f9640 1 -- 172.21.15.78:0/243230041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1428119530 con 0x7f1428113860 2024-09-11T01:18:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.885+0000 7f142da96640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1428113860 0x7f14281a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.885+0000 7f142ca94640 1 -- 172.21.15.78:0/243230041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1428113860 msgr2=0x7f14281a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.885+0000 7f142ca94640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1428113860 0x7f14281a51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.885+0000 7f142ca94640 1 -- 172.21.15.78:0/243230041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1428101250 msgr2=0x7f142819f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.885+0000 7f142ca94640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1428101250 0x7f142819f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.885+0000 7f142ca94640 1 -- 172.21.15.78:0/243230041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14281a5990 con 0x7f1428110f30 2024-09-11T01:18:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.885+0000 7f142da96640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1428113860 0x7f14281a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:18:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.886+0000 7f142d295640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1428101250 0x7f142819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:18:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.886+0000 7f142ca94640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1428110f30 0x7f142819fe20 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f141800e9c0 tx=0x7f141800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.886+0000 7f141e7fc640 1 -- 172.21.15.78:0/243230041 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141800cd80 con 0x7f1428110f30 2024-09-11T01:18:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.886+0000 7f142f4f9640 1 -- 172.21.15.78:0/243230041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14281a5c80 con 0x7f1428110f30 2024-09-11T01:18:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.886+0000 7f141e7fc640 1 -- 172.21.15.78:0/243230041 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14180054d0 con 0x7f1428110f30 2024-09-11T01:18:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.886+0000 7f141e7fc640 1 -- 172.21.15.78:0/243230041 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418010430 con 0x7f1428110f30 2024-09-11T01:18:06.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.886+0000 7f142f4f9640 1 -- 172.21.15.78:0/243230041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14281ac340 con 0x7f1428110f30 2024-09-11T01:18:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.891+0000 7f141e7fc640 1 -- 172.21.15.78:0/243230041 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1418005020 con 0x7f1428110f30 2024-09-11T01:18:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.892+0000 7f14318ce740 1 Processor -- start 2024-09-11T01:18:06.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.892+0000 7f14318ce740 1 -- start start 2024-09-11T01:18:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.892+0000 7f14318ce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d60f0 0x55593e9d64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.892+0000 7f14318ce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d6a10 0x55593e9e2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.892+0000 7f14318ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593e9e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.892+0000 7f14318ce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55593e94cea0 con 0x55593e9d6a10 2024-09-11T01:18:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.892+0000 7f14318ce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55593e93c2d0 con 0x55593e9d60f0 2024-09-11T01:18:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.892+0000 7f14318ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55593e9400e0 con 0x55593e9e29a0 2024-09-11T01:18:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.893+0000 7f142da96640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d60f0 0x55593e9d64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.893+0000 7f142da96640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d60f0 0x55593e9d64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39006/0 (socket says 172.21.15.78:39006) 2024-09-11T01:18:06.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.893+0000 7f142da96640 1 -- 172.21.15.78:0/1118375388 learned_addr learned my addr 172.21.15.78:0/1118375388 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:06.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.893+0000 7f142d295640 1 --2- 172.21.15.78:0/1118375388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d6a10 0x55593e9e2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.893+0000 7f142da96640 1 -- 172.21.15.78:0/1118375388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 msgr2=0x55593e9e4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.894+0000 7f142ca94640 1 --2- 172.21.15.78:0/1118375388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593e9e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.894+0000 7f142da96640 1 --2- 172.21.15.78:0/1118375388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593e9e4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.894+0000 7f142da96640 1 -- 172.21.15.78:0/1118375388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d6a10 msgr2=0x55593e9e2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.894+0000 7f142da96640 1 --2- 172.21.15.78:0/1118375388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d6a10 0x55593e9e2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.894+0000 7f142da96640 1 -- 172.21.15.78:0/1118375388 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55593e69a650 con 0x55593e9d60f0 2024-09-11T01:18:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.895+0000 7f142da96640 1 --2- 172.21.15.78:0/1118375388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d60f0 0x55593e9d64d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1420004af0 tx=0x7f1420009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a3a1b4e90e3d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.895+0000 7f142d295640 1 --2- 172.21.15.78:0/1118375388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d6a10 0x55593e9e2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:18:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.895+0000 7f14137fe640 1 -- 172.21.15.78:0/1118375388 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420031cf0 con 0x55593e9d60f0 2024-09-11T01:18:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.895+0000 7f14137fe640 1 -- 172.21.15.78:0/1118375388 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1420036070 con 0x55593e9d60f0 2024-09-11T01:18:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.895+0000 7f14137fe640 1 -- 172.21.15.78:0/1118375388 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420004280 con 0x55593e9d60f0 2024-09-11T01:18:06.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.895+0000 7f141e7fc640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14140b5450 0x7f14140b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.896+0000 7f141e7fc640 1 -- 172.21.15.78:0/243230041 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8493+0+0 (secure 0 0 0) 0x7f14180e64f0 con 0x7f1428110f30 2024-09-11T01:18:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.896+0000 7f14318ce740 1 -- 172.21.15.78:0/1118375388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d60f0 msgr2=0x55593e9d64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.896+0000 7f14318ce740 1 --2- 172.21.15.78:0/1118375388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d60f0 0x55593e9d64d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1420004af0 tx=0x7f1420009880 comp rx=0 tx=0).stop 2024-09-11T01:18:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.896+0000 7f14318ce740 1 -- 172.21.15.78:0/1118375388 shutdown_connections 2024-09-11T01:18:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.896+0000 7f14318ce740 1 --2- 172.21.15.78:0/1118375388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593e9e4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.896+0000 7f14318ce740 1 --2- 172.21.15.78:0/1118375388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d6a10 0x55593e9e2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.896+0000 7f14318ce740 1 --2- 172.21.15.78:0/1118375388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d60f0 0x55593e9d64d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.896+0000 7f14318ce740 1 -- 172.21.15.78:0/1118375388 >> 172.21.15.78:0/1118375388 conn(0x55593e9d3f20 msgr2=0x55593e9d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.896+0000 7f142d295640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14140b5450 0x7f14140b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.897+0000 7f14318ce740 1 -- 172.21.15.78:0/1118375388 shutdown_connections 2024-09-11T01:18:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.897+0000 7f14318ce740 1 -- 172.21.15.78:0/1118375388 wait complete. 2024-09-11T01:18:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.897+0000 7f142d295640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14140b5450 0x7f14140b7910 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1424005fd0 tx=0x7f142400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.897+0000 7f14318ce740 1 -- start start 2024-09-11T01:18:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.898+0000 7f14318ce740 1 Processor -- start 2024-09-11T01:18:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.898+0000 7f14318ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9d60f0 0x55593e9e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.899+0000 7f14318ce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593e9df700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.899+0000 7f14318ce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9e29a0 0x55593e9dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.899+0000 7f14318ce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55593e79d700 con 0x55593e9e29a0 2024-09-11T01:18:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.899+0000 7f142ca94640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593e9df700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.899+0000 7f14318ce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55593e8d7160 con 0x55593e9d6a10 2024-09-11T01:18:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.899+0000 7f142ca94640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593e9df700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39016/0 (socket says 172.21.15.78:39016) 2024-09-11T01:18:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.899+0000 7f142ca94640 1 -- 172.21.15.78:0/2352564141 learned_addr learned my addr 172.21.15.78:0/2352564141 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.899+0000 7f14318ce740 1 -- 172.21.15.78:0/2352564141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55593e9e55c0 con 0x55593e9d60f0 2024-09-11T01:18:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.899+0000 7f14318ce740 10 client.0 _ll_register_callbacks cb 0x55593e8f09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.899+0000 7f142d295640 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9d60f0 0x55593e9e22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.900+0000 7f142da96640 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9e29a0 0x55593e9dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.900+0000 7f142ca94640 1 -- 172.21.15.78:0/2352564141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9d60f0 msgr2=0x55593e9e22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.900+0000 7f142ca94640 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9d60f0 0x55593e9e22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.900+0000 7f142ca94640 1 -- 172.21.15.78:0/2352564141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9e29a0 msgr2=0x55593e9dfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.900+0000 7f142ca94640 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9e29a0 0x55593e9dfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.900+0000 7f142ca94640 1 -- 172.21.15.78:0/2352564141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55593e904c90 con 0x55593e9d6a10 2024-09-11T01:18:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.900+0000 7f142d295640 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9d60f0 0x55593e9e22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:18:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.900+0000 7f142ca94640 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593e9df700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f14180058c0 tx=0x7f14180a9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79a0bdda699f75aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.900+0000 7f1410ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:18:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.900+0000 7f1410ff9640 1 -- 172.21.15.78:0/2352564141 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14180b1020 con 0x55593e9d6a10 2024-09-11T01:18:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.901+0000 7f14318ce740 1 -- 172.21.15.78:0/2352564141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1420036070 con 0x55593e9d6a10 2024-09-11T01:18:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.901+0000 7f1410ff9640 1 -- 172.21.15.78:0/2352564141 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8493+0+0 (secure 0 0 0) 0x7f14180b5c30 con 0x55593e9d6a10 2024-09-11T01:18:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.901+0000 7f1410ff9640 1 -- 172.21.15.78:0/2352564141 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f14180ac0a0 con 0x55593e9d6a10 2024-09-11T01:18:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.901+0000 7f14318ce740 10 client.27938 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.901+0000 7f14318ce740 1 -- 172.21.15.78:0/2352564141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55593e9f0f50 con 0x55593e9d6a10 2024-09-11T01:18:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.902+0000 7f1410ff9640 1 -- 172.21.15.78:0/2352564141 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1420036070 con 0x55593e9d6a10 2024-09-11T01:18:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.902+0000 7f1410ff9640 1 -- 172.21.15.78:0/2352564141 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f14180a8280 con 0x55593e9d6a10 2024-09-11T01:18:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.902+0000 7f14318ce740 10 client.27938 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.902+0000 7f14318ce740 10 client.27938 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:18:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.902+0000 7f14318ce740 20 client.27938 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.902+0000 7f14318ce740 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55593e9f2360 0x55593e9f4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.902+0000 7f14318ce740 4 client.27938 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.902+0000 7f14318ce740 1 -- 172.21.15.78:0/2352564141 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55593e94edf0 con 0x55593e9f2360 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.903+0000 7f142d295640 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55593e9f2360 0x55593e9f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.903+0000 7f142d295640 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55593e9f2360 0x55593e9f4750 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27d85a2fbab10c66 server_cookie=5ac098c6c9db6236 in_seq=0 out_seq=0 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.903+0000 7f1410ff9640 10 client.27938 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.904+0000 7f1410ff9640 1 -- 172.21.15.78:0/2352564141 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55593e94edf0 con 0x55593e9f2360 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.904+0000 7f1410ff9640 10 client.27938 handle_command_reply: tid=0 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.904+0000 7f14318ce740 1 client.27938 shutdown 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.904+0000 7f14318ce740 1 -- 172.21.15.78:0/2352564141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 msgr2=0x55593e9df700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.904+0000 7f14318ce740 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593e9df700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f14180058c0 tx=0x7f14180a9040 comp rx=0 tx=0).stop 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.905+0000 7f14318ce740 1 -- 172.21.15.78:0/2352564141 shutdown_connections 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.905+0000 7f14318ce740 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55593e9f2360 0x55593e9f4750 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.905+0000 7f14318ce740 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9e29a0 0x55593e9dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.905+0000 7f14318ce740 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593e9df700 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.905+0000 7f14318ce740 1 --2- 172.21.15.78:0/2352564141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9d60f0 0x55593e9e22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.905+0000 7f14318ce740 1 -- 172.21.15.78:0/2352564141 >> 172.21.15.78:0/2352564141 conn(0x55593e9d3f20 msgr2=0x55593e9d9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.905+0000 7f1410ff9640 0 client.27938 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:18:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.905+0000 7f14318ce740 1 -- 172.21.15.78:0/2352564141 shutdown_connections 2024-09-11T01:18:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.905+0000 7f14318ce740 1 -- 172.21.15.78:0/2352564141 wait complete. 2024-09-11T01:18:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.905+0000 7f14318ce740 20 client.27938 trim_cache size 0 max 16384 2024-09-11T01:18:06.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.931+0000 7f14318ce740 1 Processor -- start 2024-09-11T01:18:06.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.931+0000 7f14318ce740 1 -- start start 2024-09-11T01:18:06.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.931+0000 7f14318ce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d60f0 0x55593e9e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.931+0000 7f14318ce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593e9e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.931+0000 7f14318ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593e9dc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.931+0000 7f14318ce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b4e30 con 0x55593e9d60f0 2024-09-11T01:18:06.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.931+0000 7f14318ce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b6970 con 0x55593e9d6a10 2024-09-11T01:18:06.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.931+0000 7f14318ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b5500 con 0x55593e9e29a0 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.931+0000 7f142ca94640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593e9e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.931+0000 7f142da96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593e9dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.931+0000 7f142ca94640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593e9e42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39018/0 (socket says 172.21.15.78:39018) 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.931+0000 7f142ca94640 1 -- 172.21.15.78:0/930135380 learned_addr learned my addr 172.21.15.78:0/930135380 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.932+0000 7f142ca94640 1 -- 172.21.15.78:0/930135380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 msgr2=0x55593e9dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.932+0000 7f142ca94640 1 --2- 172.21.15.78:0/930135380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593e9dc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.932+0000 7f142ca94640 1 -- 172.21.15.78:0/930135380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d60f0 msgr2=0x55593e9e3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.932+0000 7f142ca94640 1 --2- 172.21.15.78:0/930135380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d60f0 0x55593e9e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.932+0000 7f142ca94640 1 -- 172.21.15.78:0/930135380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55593e9e00a0 con 0x55593e9d6a10 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.932+0000 7f142ca94640 1 --2- 172.21.15.78:0/930135380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593e9e42d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f14180ac8c0 tx=0x7f14180a90b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb6bc8e1d4f78b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.932+0000 7f1412ffd640 1 -- 172.21.15.78:0/930135380 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14180b1040 con 0x55593e9d6a10 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.932+0000 7f1412ffd640 1 -- 172.21.15.78:0/930135380 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14180ac440 con 0x55593e9d6a10 2024-09-11T01:18:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.932+0000 7f1412ffd640 1 -- 172.21.15.78:0/930135380 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14180e4090 con 0x55593e9d6a10 2024-09-11T01:18:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.933+0000 7f14318ce740 1 -- 172.21.15.78:0/930135380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 msgr2=0x55593e9e42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.933+0000 7f14318ce740 1 --2- 172.21.15.78:0/930135380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593e9e42d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f14180ac8c0 tx=0x7f14180a90b0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.934+0000 7f14318ce740 1 -- 172.21.15.78:0/930135380 shutdown_connections 2024-09-11T01:18:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.934+0000 7f14318ce740 1 --2- 172.21.15.78:0/930135380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593e9dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.934+0000 7f14318ce740 1 --2- 172.21.15.78:0/930135380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593e9e42d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.934+0000 7f14318ce740 1 --2- 172.21.15.78:0/930135380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d60f0 0x55593e9e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.934+0000 7f14318ce740 1 -- 172.21.15.78:0/930135380 >> 172.21.15.78:0/930135380 conn(0x55593e9d3f20 msgr2=0x55593e9da930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.934+0000 7f14318ce740 1 -- 172.21.15.78:0/930135380 shutdown_connections 2024-09-11T01:18:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.934+0000 7f14318ce740 1 -- 172.21.15.78:0/930135380 wait complete. 2024-09-11T01:18:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.934+0000 7f14318ce740 1 -- start start 2024-09-11T01:18:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.934+0000 7f14318ce740 1 Processor -- start 2024-09-11T01:18:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.935+0000 7f14318ce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d60f0 0x55593ea5c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.935+0000 7f14318ce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593ea5cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.935+0000 7f14318ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593ea59440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.935+0000 7f14318ce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b62a0 con 0x55593e9d60f0 2024-09-11T01:18:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.935+0000 7f14318ce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b5bd0 con 0x55593e9d6a10 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.935+0000 7f14318ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b2e30 con 0x55593e9e29a0 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.935+0000 7f14318ce740 10 client.0 _ll_register_callbacks cb 0x7f1418009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.935+0000 7f142ca94640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593ea5cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.935+0000 7f142da96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593ea59440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.935+0000 7f142ca94640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593ea5cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39020/0 (socket says 172.21.15.78:39020) 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.935+0000 7f142ca94640 1 -- 172.21.15.78:0/1480586393 learned_addr learned my addr 172.21.15.78:0/1480586393 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.935+0000 7f142d295640 1 --2- 172.21.15.78:0/1480586393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d60f0 0x55593ea5c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.936+0000 7f142da96640 1 -- 172.21.15.78:0/1480586393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 msgr2=0x55593ea5cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.936+0000 7f142da96640 1 --2- 172.21.15.78:0/1480586393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593ea5cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.936+0000 7f142da96640 1 -- 172.21.15.78:0/1480586393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d60f0 msgr2=0x55593ea5c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.936+0000 7f142da96640 1 --2- 172.21.15.78:0/1480586393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d60f0 0x55593ea5c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.936+0000 7f142da96640 1 -- 172.21.15.78:0/1480586393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55593ea59c70 con 0x55593e9e29a0 2024-09-11T01:18:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.936+0000 7f142da96640 1 --2- 172.21.15.78:0/1480586393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593ea59440 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1420005e30 tx=0x7f142002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29475bbd72932aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.936+0000 7f1410ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:18:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.936+0000 7f1410ff9640 1 -- 172.21.15.78:0/1480586393 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420049040 con 0x55593e9e29a0 2024-09-11T01:18:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.936+0000 7f1410ff9640 1 -- 172.21.15.78:0/1480586393 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8493+0+0 (secure 0 0 0) 0x7f1420031650 con 0x55593e9e29a0 2024-09-11T01:18:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.936+0000 7f14318ce740 1 -- 172.21.15.78:0/1480586393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14180ac440 con 0x55593e9e29a0 2024-09-11T01:18:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.937+0000 7f1410ff9640 1 -- 172.21.15.78:0/1480586393 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f1420031850 con 0x55593e9e29a0 2024-09-11T01:18:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.937+0000 7f14318ce740 10 client.27889 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.937+0000 7f14318ce740 1 -- 172.21.15.78:0/1480586393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55593e9ea560 con 0x55593e9e29a0 2024-09-11T01:18:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.937+0000 7f1410ff9640 1 -- 172.21.15.78:0/1480586393 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14180ac440 con 0x55593e9e29a0 2024-09-11T01:18:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.937+0000 7f1410ff9640 1 -- 172.21.15.78:0/1480586393 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f142004c050 con 0x55593e9e29a0 2024-09-11T01:18:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.937+0000 7f14318ce740 10 client.27889 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.937+0000 7f14318ce740 10 client.27889 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:18:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.937+0000 7f14318ce740 20 client.27889 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.937+0000 7f14318ce740 1 --2- 172.21.15.78:0/1480586393 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55593e9ea8a0 0x55593e9eacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.937+0000 7f14318ce740 4 client.27889 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:18:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.937+0000 7f14318ce740 1 -- 172.21.15.78:0/1480586393 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55593e94a8a0 con 0x55593e9ea8a0 2024-09-11T01:18:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.938+0000 7f142d295640 1 --2- 172.21.15.78:0/1480586393 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55593e9ea8a0 0x55593e9eacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.938+0000 7f1410ff9640 10 client.27889 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:18:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.938+0000 7f142d295640 1 --2- 172.21.15.78:0/1480586393 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55593e9ea8a0 0x55593e9eacb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ed961fc27fae9c9 server_cookie=b02a3ada085552e8 in_seq=0 out_seq=0 2024-09-11T01:18:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.939+0000 7f1410ff9640 1 -- 172.21.15.78:0/1480586393 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55593e94a8a0 con 0x55593e9ea8a0 2024-09-11T01:18:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.939+0000 7f1410ff9640 10 client.27889 handle_command_reply: tid=0 2024-09-11T01:18:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.939+0000 7f14318ce740 1 client.27889 shutdown 2024-09-11T01:18:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.939+0000 7f14318ce740 1 -- 172.21.15.78:0/1480586393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 msgr2=0x55593ea59440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.939+0000 7f14318ce740 1 --2- 172.21.15.78:0/1480586393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593ea59440 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1420005e30 tx=0x7f142002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.939+0000 7f14318ce740 1 -- 172.21.15.78:0/1480586393 shutdown_connections 2024-09-11T01:18:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.939+0000 7f14318ce740 1 --2- 172.21.15.78:0/1480586393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55593e9e29a0 0x55593ea59440 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.939+0000 7f14318ce740 1 --2- 172.21.15.78:0/1480586393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55593e9d6a10 0x55593ea5cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.939+0000 7f14318ce740 1 --2- 172.21.15.78:0/1480586393 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55593e9ea8a0 0x55593e9eacb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.939+0000 7f14318ce740 1 --2- 172.21.15.78:0/1480586393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55593e9d60f0 0x55593ea5c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.939+0000 7f14318ce740 1 -- 172.21.15.78:0/1480586393 >> 172.21.15.78:0/1480586393 conn(0x55593e9d3f20 msgr2=0x55593e9e0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:06.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.939+0000 7f1410ff9640 0 client.27889 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:18:06.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.940+0000 7f14318ce740 1 -- 172.21.15.78:0/1480586393 shutdown_connections 2024-09-11T01:18:06.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.940+0000 7f14318ce740 1 -- 172.21.15.78:0/1480586393 wait complete. 2024-09-11T01:18:06.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.940+0000 7f14318ce740 20 client.27889 trim_cache size 0 max 16384 2024-09-11T01:18:06.945 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:18:06.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.941+0000 7f142f4f9640 1 -- 172.21.15.78:0/243230041 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14140b5450 msgr2=0x7f14140b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:06.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.941+0000 7f142f4f9640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14140b5450 0x7f14140b7910 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1424005fd0 tx=0x7f142400b390 comp rx=0 tx=0).stop 2024-09-11T01:18:06.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.941+0000 7f142f4f9640 1 -- 172.21.15.78:0/243230041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1428110f30 msgr2=0x7f142819fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:06.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.941+0000 7f142f4f9640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1428110f30 0x7f142819fe20 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f141800e9c0 tx=0x7f141800ee80 comp rx=0 tx=0).stop 2024-09-11T01:18:06.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.942+0000 7f142f4f9640 1 -- 172.21.15.78:0/243230041 shutdown_connections 2024-09-11T01:18:06.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.942+0000 7f142f4f9640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f14140b5450 0x7f14140b7910 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.942+0000 7f142f4f9640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1428113860 0x7f14281a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.942+0000 7f142f4f9640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1428110f30 0x7f142819fe20 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.942+0000 7f142f4f9640 1 --2- 172.21.15.78:0/243230041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1428101250 0x7f142819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:06.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.942+0000 7f142f4f9640 1 -- 172.21.15.78:0/243230041 >> 172.21.15.78:0/243230041 conn(0x7f1428078140 msgr2=0x7f1428114060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:06.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.942+0000 7f142f4f9640 1 -- 172.21.15.78:0/243230041 shutdown_connections 2024-09-11T01:18:06.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:06.942+0000 7f142f4f9640 1 -- 172.21.15.78:0/243230041 wait complete. 2024-09-11T01:18:07.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 243229 34.73 MB/sec execute 253 sec latency 422.047 ms 2024-09-11T01:18:07.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:07 smithi110 ceph-mon[34712]: pgmap v925: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 804 op/s 2024-09-11T01:18:07.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:07 smithi112 ceph-mon[35384]: pgmap v925: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 804 op/s 2024-09-11T01:18:07.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:07 smithi078 ceph-mon[31315]: pgmap v925: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 804 op/s 2024-09-11T01:18:08.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 243779 34.72 MB/sec execute 254 sec latency 426.880 ms 2024-09-11T01:18:09.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 244252 34.70 MB/sec execute 255 sec latency 206.017 ms 2024-09-11T01:18:09.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:09 smithi110 ceph-mon[34712]: pgmap v926: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 759 op/s 2024-09-11T01:18:09.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:09 smithi112 ceph-mon[35384]: pgmap v926: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 759 op/s 2024-09-11T01:18:09.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:09 smithi078 ceph-mon[31315]: pgmap v926: 97 pgs: 97 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 759 op/s 2024-09-11T01:18:10.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 244785 34.66 MB/sec execute 256 sec latency 144.533 ms 2024-09-11T01:18:11.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 245371 34.64 MB/sec execute 257 sec latency 135.164 ms 2024-09-11T01:18:11.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:11 smithi110 ceph-mon[34712]: pgmap v927: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 848 op/s 2024-09-11T01:18:11.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:11 smithi112 ceph-mon[35384]: pgmap v927: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 848 op/s 2024-09-11T01:18:11.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:11 smithi078 ceph-mon[31315]: pgmap v927: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 848 op/s 2024-09-11T01:18:12.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 245969 34.63 MB/sec execute 258 sec latency 157.864 ms 2024-09-11T01:18:13.297 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:18:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:18:13.666 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:13 smithi078 ceph-mon[31315]: pgmap v928: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 729 op/s 2024-09-11T01:18:13.667 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:13 smithi078 ceph-mon[31315]: osdmap e103: 12 total, 12 up, 12 in 2024-09-11T01:18:13.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 246775 34.66 MB/sec execute 259 sec latency 75.738 ms 2024-09-11T01:18:13.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:13 smithi110 ceph-mon[34712]: pgmap v928: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 729 op/s 2024-09-11T01:18:13.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:13 smithi110 ceph-mon[34712]: osdmap e103: 12 total, 12 up, 12 in 2024-09-11T01:18:13.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:13 smithi112 ceph-mon[35384]: pgmap v928: 97 pgs: 97 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 729 op/s 2024-09-11T01:18:13.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:13 smithi112 ceph-mon[35384]: osdmap e103: 12 total, 12 up, 12 in 2024-09-11T01:18:14.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.587+0000 7ff682656640 1 -- 172.21.15.78:0/1864706905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff67c1076a0 msgr2=0x7ff67c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:14.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.587+0000 7ff682656640 1 --2- 172.21.15.78:0/1864706905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff67c1076a0 0x7ff67c107ae0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7ff670009900 tx=0x7ff67002a410 comp rx=0 tx=0).stop 2024-09-11T01:18:14.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.587+0000 7ff682656640 1 -- 172.21.15.78:0/1864706905 shutdown_connections 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.587+0000 7ff682656640 1 --2- 172.21.15.78:0/1864706905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff67c108020 0x7ff67c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.587+0000 7ff682656640 1 --2- 172.21.15.78:0/1864706905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff67c1076a0 0x7ff67c107ae0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.587+0000 7ff682656640 1 --2- 172.21.15.78:0/1864706905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff67c101c90 0x7ff67c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.587+0000 7ff682656640 1 -- 172.21.15.78:0/1864706905 >> 172.21.15.78:0/1864706905 conn(0x7ff67c0fd700 msgr2=0x7ff67c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.587+0000 7ff682656640 1 -- 172.21.15.78:0/1864706905 shutdown_connections 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.587+0000 7ff682656640 1 -- 172.21.15.78:0/1864706905 wait complete. 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.588+0000 7ff682656640 1 Processor -- start 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.588+0000 7ff682656640 1 -- start start 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff682656640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff67c101c90 0x7ff67c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff682656640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff67c1076a0 0x7ff67c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff682656640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff67c108020 0x7ff67c19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff682656640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c111d30 con 0x7ff67c101c90 2024-09-11T01:18:14.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff682656640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c111bb0 con 0x7ff67c108020 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff682656640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c111eb0 con 0x7ff67c1076a0 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff67bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff67c101c90 0x7ff67c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff67bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff67c101c90 0x7ff67c198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37034/0 (socket says 172.21.15.78:37034) 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff67bfff640 1 -- 172.21.15.78:0/2519757900 learned_addr learned my addr 172.21.15.78:0/2519757900 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff67b7fe640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff67c1076a0 0x7ff67c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff680bf3640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff67c108020 0x7ff67c19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff67bfff640 1 -- 172.21.15.78:0/2519757900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff67c1076a0 msgr2=0x7ff67c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff67bfff640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff67c1076a0 0x7ff67c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff67bfff640 1 -- 172.21.15.78:0/2519757900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff67c108020 msgr2=0x7ff67c19da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff67bfff640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff67c108020 0x7ff67c19da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff67bfff640 1 -- 172.21.15.78:0/2519757900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67c19e230 con 0x7ff67c101c90 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff67bfff640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff67c101c90 0x7ff67c198180 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00d950 tx=0x7ff66c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:14.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.589+0000 7ff6797fa640 1 -- 172.21.15.78:0/2519757900 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c017070 con 0x7ff67c101c90 2024-09-11T01:18:14.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.590+0000 7ff682656640 1 -- 172.21.15.78:0/2519757900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67c19e520 con 0x7ff67c101c90 2024-09-11T01:18:14.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.590+0000 7ff6797fa640 1 -- 172.21.15.78:0/2519757900 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff66c009c30 con 0x7ff67c101c90 2024-09-11T01:18:14.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.590+0000 7ff6797fa640 1 -- 172.21.15.78:0/2519757900 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c00f620 con 0x7ff67c101c90 2024-09-11T01:18:14.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.590+0000 7ff682656640 1 -- 172.21.15.78:0/2519757900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff67c06ac80 con 0x7ff67c101c90 2024-09-11T01:18:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.593+0000 7ff6797fa640 1 -- 172.21.15.78:0/2519757900 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff66c010460 con 0x7ff67c101c90 2024-09-11T01:18:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.594+0000 7ff684a2b740 1 Processor -- start 2024-09-11T01:18:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.594+0000 7ff684a2b740 1 -- start start 2024-09-11T01:18:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.594+0000 7ff684a2b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bca9fe0f0 0x563bca9fe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.594+0000 7ff684a2b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bca9fea10 0x563bcaa0e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.594+0000 7ff684a2b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bcaa0edb0 0x563bcaa111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.594+0000 7ff684a2b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563bca974ea0 con 0x563bcaa0edb0 2024-09-11T01:18:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.594+0000 7ff684a2b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563bca9642d0 con 0x563bca9fe0f0 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.594+0000 7ff684a2b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563bca9680e0 con 0x563bca9fea10 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.594+0000 7ff6797fa640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff6640b5450 0x7ff6640b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.594+0000 7ff67b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bcaa0edb0 0x563bcaa111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.594+0000 7ff67b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bcaa0edb0 0x563bcaa111a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37040/0 (socket says 172.21.15.78:37040) 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.594+0000 7ff67b7fe640 1 -- 172.21.15.78:0/1152964898 learned_addr learned my addr 172.21.15.78:0/1152964898 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.595+0000 7ff680bf3640 1 --2- 172.21.15.78:0/1152964898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bca9fe0f0 0x563bca9fe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.595+0000 7ff6797fa640 1 -- 172.21.15.78:0/2519757900 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7ff66c0e5b60 con 0x7ff67c101c90 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.595+0000 7ff67bfff640 1 --2- 172.21.15.78:0/1152964898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bca9fea10 0x563bcaa0e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.595+0000 7ff680bf3640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff6640b5450 0x7ff6640b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.595+0000 7ff67b7fe640 1 -- 172.21.15.78:0/1152964898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bca9fea10 msgr2=0x563bcaa0e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.595+0000 7ff67b7fe640 1 --2- 172.21.15.78:0/1152964898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bca9fea10 0x563bcaa0e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.595+0000 7ff67b7fe640 1 -- 172.21.15.78:0/1152964898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bca9fe0f0 msgr2=0x563bca9fe4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.595+0000 7ff67b7fe640 1 --2- 172.21.15.78:0/1152964898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bca9fe0f0 0x563bca9fe4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.595+0000 7ff67b7fe640 1 -- 172.21.15.78:0/1152964898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bca6c2650 con 0x563bcaa0edb0 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.595+0000 7ff680bf3640 1 --2- 172.21.15.78:0/1152964898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bca9fe0f0 0x563bca9fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.595+0000 7ff67b7fe640 1 --2- 172.21.15.78:0/1152964898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bcaa0edb0 0x563bcaa111a0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff670009230 tx=0x7ff670004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a8c36c96cbb5e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.595+0000 7ff680bf3640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff6640b5450 0x7ff6640b7910 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff674005c40 tx=0x7ff674009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.596+0000 7ff6627fc640 1 -- 172.21.15.78:0/1152964898 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff670040070 con 0x563bcaa0edb0 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.596+0000 7ff6627fc640 1 -- 172.21.15.78:0/1152964898 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff670005500 con 0x563bcaa0edb0 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.596+0000 7ff6627fc640 1 -- 172.21.15.78:0/1152964898 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff670035040 con 0x563bcaa0edb0 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.596+0000 7ff684a2b740 1 -- 172.21.15.78:0/1152964898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bcaa0edb0 msgr2=0x563bcaa111a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.596+0000 7ff684a2b740 1 --2- 172.21.15.78:0/1152964898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bcaa0edb0 0x563bcaa111a0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff670009230 tx=0x7ff670004170 comp rx=0 tx=0).stop 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.596+0000 7ff684a2b740 1 -- 172.21.15.78:0/1152964898 shutdown_connections 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.596+0000 7ff684a2b740 1 --2- 172.21.15.78:0/1152964898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bcaa0edb0 0x563bcaa111a0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.596+0000 7ff684a2b740 1 --2- 172.21.15.78:0/1152964898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bca9fea10 0x563bcaa0e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.596+0000 7ff684a2b740 1 --2- 172.21.15.78:0/1152964898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bca9fe0f0 0x563bca9fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.596+0000 7ff684a2b740 1 -- 172.21.15.78:0/1152964898 >> 172.21.15.78:0/1152964898 conn(0x563bca9fbf20 msgr2=0x563bca9fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.597+0000 7ff684a2b740 1 -- 172.21.15.78:0/1152964898 shutdown_connections 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.597+0000 7ff684a2b740 1 -- 172.21.15.78:0/1152964898 wait complete. 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.597+0000 7ff684a2b740 1 -- start start 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.597+0000 7ff684a2b740 1 Processor -- start 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff684a2b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fe0f0 0x563bcaa0e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff684a2b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bca9fea10 0x563bcaa0bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff684a2b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa0c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff684a2b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563bca7c5700 con 0x563bca9fe0f0 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff684a2b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563bca8ff160 con 0x563bcaa0edb0 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff684a2b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563bcaa11a00 con 0x563bca9fea10 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff684a2b740 10 client.0 _ll_register_callbacks cb 0x563bca9189e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff67bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fe0f0 0x563bcaa0e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff67b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bca9fea10 0x563bcaa0bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff67b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bca9fea10 0x563bcaa0bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:58848/0 (socket says 172.21.15.78:58848) 2024-09-11T01:18:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff67b7fe640 1 -- 172.21.15.78:0/2026400725 learned_addr learned my addr 172.21.15.78:0/2026400725 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff680bf3640 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa0c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff67bfff640 1 -- 172.21.15.78:0/2026400725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bca9fea10 msgr2=0x563bcaa0bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff67bfff640 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bca9fea10 0x563bcaa0bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff67bfff640 1 -- 172.21.15.78:0/2026400725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 msgr2=0x563bcaa0c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff67bfff640 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa0c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff67bfff640 1 -- 172.21.15.78:0/2026400725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bca92cc90 con 0x563bca9fe0f0 2024-09-11T01:18:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff680bf3640 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa0c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:18:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff67bfff640 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fe0f0 0x563bcaa0e710 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0e32f0 tx=0x7ff66c0e3820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba95394fcb794db4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.598+0000 7ff65b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.599+0000 7ff65b7fe640 1 -- 172.21.15.78:0/2026400725 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c020040 con 0x563bca9fe0f0 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.599+0000 7ff684a2b740 1 -- 172.21.15.78:0/2026400725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff670005500 con 0x563bca9fe0f0 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.599+0000 7ff65b7fe640 1 -- 172.21.15.78:0/2026400725 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7ff66c0971d0 con 0x563bca9fe0f0 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.599+0000 7ff65b7fe640 1 -- 172.21.15.78:0/2026400725 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7ff66c0abbe0 con 0x563bca9fe0f0 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.599+0000 7ff684a2b740 10 client.18858 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.599+0000 7ff684a2b740 1 -- 172.21.15.78:0/2026400725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x563bcaa1d320 con 0x563bca9fe0f0 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.599+0000 7ff65b7fe640 1 -- 172.21.15.78:0/2026400725 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff670005500 con 0x563bca9fe0f0 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.599+0000 7ff65b7fe640 1 -- 172.21.15.78:0/2026400725 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7ff66c0aae20 con 0x563bca9fe0f0 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.599+0000 7ff684a2b740 10 client.18858 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.599+0000 7ff684a2b740 10 client.18858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.599+0000 7ff684a2b740 20 client.18858 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.599+0000 7ff684a2b740 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563bcaa1e7a0 0x563bcaa20c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.600+0000 7ff684a2b740 4 client.18858 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:18:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.600+0000 7ff684a2b740 1 -- 172.21.15.78:0/2026400725 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bca976df0 con 0x563bcaa1e7a0 2024-09-11T01:18:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.600+0000 7ff67b7fe640 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563bcaa1e7a0 0x563bcaa20c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.600+0000 7ff65b7fe640 10 client.18858 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.600+0000 7ff67b7fe640 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563bcaa1e7a0 0x563bcaa20c10 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884f0fd9dbf938bb server_cookie=56d3234da41a3c8f in_seq=0 out_seq=0 2024-09-11T01:18:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.602+0000 7ff65b7fe640 1 -- 172.21.15.78:0/2026400725 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563bca976df0 con 0x563bcaa1e7a0 2024-09-11T01:18:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.602+0000 7ff65b7fe640 10 client.18858 handle_command_reply: tid=0 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.602+0000 7ff684a2b740 1 client.18858 shutdown 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.602+0000 7ff684a2b740 1 -- 172.21.15.78:0/2026400725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fe0f0 msgr2=0x563bcaa0e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.602+0000 7ff684a2b740 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fe0f0 0x563bcaa0e710 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0e32f0 tx=0x7ff66c0e3820 comp rx=0 tx=0).stop 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.603+0000 7ff684a2b740 1 -- 172.21.15.78:0/2026400725 shutdown_connections 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.603+0000 7ff684a2b740 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa0c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.603+0000 7ff684a2b740 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bca9fea10 0x563bcaa0bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.603+0000 7ff684a2b740 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563bcaa1e7a0 0x563bcaa20c10 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.603+0000 7ff684a2b740 1 --2- 172.21.15.78:0/2026400725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fe0f0 0x563bcaa0e710 secure :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0e32f0 tx=0x7ff66c0e3820 comp rx=0 tx=0).stop 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.603+0000 7ff684a2b740 1 -- 172.21.15.78:0/2026400725 >> 172.21.15.78:0/2026400725 conn(0x563bca9fbf20 msgr2=0x563bcaa01a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.603+0000 7ff65b7fe640 0 client.18858 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.603+0000 7ff684a2b740 1 -- 172.21.15.78:0/2026400725 shutdown_connections 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.603+0000 7ff684a2b740 1 -- 172.21.15.78:0/2026400725 wait complete. 2024-09-11T01:18:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.603+0000 7ff684a2b740 20 client.18858 trim_cache size 0 max 16384 2024-09-11T01:18:14.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff684a2b740 1 Processor -- start 2024-09-11T01:18:14.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff684a2b740 1 -- start start 2024-09-11T01:18:14.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff684a2b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 0x563bcaa1ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff684a2b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa0f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff684a2b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcaa11320 0x563bcaa26500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff684a2b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c1ac420 con 0x563bca9fea10 2024-09-11T01:18:14.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff684a2b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c1adf60 con 0x563bcaa0edb0 2024-09-11T01:18:14.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff684a2b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c1acaf0 con 0x563bcaa11320 2024-09-11T01:18:14.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff67bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 0x563bcaa1ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff67bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 0x563bcaa1ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37058/0 (socket says 172.21.15.78:37058) 2024-09-11T01:18:14.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff67bfff640 1 -- 172.21.15.78:0/3002644663 learned_addr learned my addr 172.21.15.78:0/3002644663 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:14.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff67b7fe640 1 --2- 172.21.15.78:0/3002644663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa0f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.618+0000 7ff680bf3640 1 --2- 172.21.15.78:0/3002644663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcaa11320 0x563bcaa26500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.619+0000 7ff67bfff640 1 -- 172.21.15.78:0/3002644663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcaa11320 msgr2=0x563bcaa26500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:14.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.619+0000 7ff67bfff640 1 --2- 172.21.15.78:0/3002644663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcaa11320 0x563bcaa26500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.619+0000 7ff67bfff640 1 -- 172.21.15.78:0/3002644663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 msgr2=0x563bcaa0f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:14.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.619+0000 7ff67bfff640 1 --2- 172.21.15.78:0/3002644663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa0f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.619+0000 7ff67bfff640 1 -- 172.21.15.78:0/3002644663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bca9fd9c0 con 0x563bca9fea10 2024-09-11T01:18:14.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.619+0000 7ff67bfff640 1 --2- 172.21.15.78:0/3002644663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 0x563bcaa1ac30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0a9340 tx=0x7ff66c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de0f563c09a81cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:14.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.619+0000 7ff661ffb640 1 -- 172.21.15.78:0/3002644663 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c020030 con 0x563bca9fea10 2024-09-11T01:18:14.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.619+0000 7ff661ffb640 1 -- 172.21.15.78:0/3002644663 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff66c009de0 con 0x563bca9fea10 2024-09-11T01:18:14.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.619+0000 7ff661ffb640 1 -- 172.21.15.78:0/3002644663 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0aafa0 con 0x563bca9fea10 2024-09-11T01:18:14.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.620+0000 7ff684a2b740 1 -- 172.21.15.78:0/3002644663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 msgr2=0x563bcaa1ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:14.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.620+0000 7ff684a2b740 1 --2- 172.21.15.78:0/3002644663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 0x563bcaa1ac30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0a9340 tx=0x7ff66c000bc0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.620+0000 7ff684a2b740 1 -- 172.21.15.78:0/3002644663 shutdown_connections 2024-09-11T01:18:14.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.620+0000 7ff684a2b740 1 --2- 172.21.15.78:0/3002644663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcaa11320 0x563bcaa26500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.620+0000 7ff684a2b740 1 --2- 172.21.15.78:0/3002644663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa0f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.620+0000 7ff684a2b740 1 --2- 172.21.15.78:0/3002644663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 0x563bcaa1ac30 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.620+0000 7ff684a2b740 1 -- 172.21.15.78:0/3002644663 >> 172.21.15.78:0/3002644663 conn(0x563bca9fbf20 msgr2=0x563bcaa10c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:14.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.620+0000 7ff684a2b740 1 -- 172.21.15.78:0/3002644663 shutdown_connections 2024-09-11T01:18:14.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.620+0000 7ff684a2b740 1 -- 172.21.15.78:0/3002644663 wait complete. 2024-09-11T01:18:14.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.620+0000 7ff684a2b740 1 -- start start 2024-09-11T01:18:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff684a2b740 1 Processor -- start 2024-09-11T01:18:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff684a2b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 0x563bcaa04420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff684a2b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa04960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff684a2b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcaa11320 0x563bcaa04ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff684a2b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c1ad890 con 0x563bca9fea10 2024-09-11T01:18:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff684a2b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c1ad1c0 con 0x563bcaa0edb0 2024-09-11T01:18:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff684a2b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c1aa420 con 0x563bcaa11320 2024-09-11T01:18:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff684a2b740 10 client.0 _ll_register_callbacks cb 0x7ff66c0aba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff67bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 0x563bcaa04420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff67bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 0x563bcaa04420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37070/0 (socket says 172.21.15.78:37070) 2024-09-11T01:18:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff67bfff640 1 -- 172.21.15.78:0/1099729226 learned_addr learned my addr 172.21.15.78:0/1099729226 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff67b7fe640 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa04960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff680bf3640 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcaa11320 0x563bcaa04ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff67bfff640 1 -- 172.21.15.78:0/1099729226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcaa11320 msgr2=0x563bcaa04ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff67bfff640 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcaa11320 0x563bcaa04ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff67bfff640 1 -- 172.21.15.78:0/1099729226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 msgr2=0x563bcaa04960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.621+0000 7ff67bfff640 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa04960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.622+0000 7ff67bfff640 1 -- 172.21.15.78:0/1099729226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bcaa16f00 con 0x563bca9fea10 2024-09-11T01:18:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.622+0000 7ff680bf3640 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcaa11320 0x563bcaa04ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:18:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.622+0000 7ff67bfff640 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 0x563bcaa04420 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0ac060 tx=0x7ff66c0aa880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cd31d94dd6212b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.622+0000 7ff65b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:18:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.622+0000 7ff65b7fe640 1 -- 172.21.15.78:0/1099729226 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c020070 con 0x563bca9fea10 2024-09-11T01:18:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.622+0000 7ff65b7fe640 1 -- 172.21.15.78:0/1099729226 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7ff66c0ab330 con 0x563bca9fea10 2024-09-11T01:18:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.622+0000 7ff684a2b740 1 -- 172.21.15.78:0/1099729226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff66c009de0 con 0x563bca9fea10 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.622+0000 7ff65b7fe640 1 -- 172.21.15.78:0/1099729226 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7ff66c0a8720 con 0x563bca9fea10 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.622+0000 7ff684a2b740 10 client.18864 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.622+0000 7ff684a2b740 1 -- 172.21.15.78:0/1099729226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x563bcaa904e0 con 0x563bca9fea10 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.622+0000 7ff65b7fe640 1 -- 172.21.15.78:0/1099729226 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff66c009de0 con 0x563bca9fea10 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.623+0000 7ff65b7fe640 1 -- 172.21.15.78:0/1099729226 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7ff66c0a7a20 con 0x563bca9fea10 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.623+0000 7ff684a2b740 10 client.18864 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.623+0000 7ff684a2b740 10 client.18864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.623+0000 7ff684a2b740 20 client.18864 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.623+0000 7ff684a2b740 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563bcaa909e0 0x563bcaa92e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.623+0000 7ff684a2b740 4 client.18864 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.623+0000 7ff684a2b740 1 -- 172.21.15.78:0/1099729226 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563bca9728a0 con 0x563bcaa909e0 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.623+0000 7ff67b7fe640 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563bcaa909e0 0x563bcaa92e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.623+0000 7ff67b7fe640 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563bcaa909e0 0x563bcaa92e00 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f9d0e9d81f8488 server_cookie=8421c9f67c1fd7d5 in_seq=0 out_seq=0 2024-09-11T01:18:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.623+0000 7ff65b7fe640 10 client.18864 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff65b7fe640 1 -- 172.21.15.78:0/1099729226 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563bca9728a0 con 0x563bcaa909e0 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff65b7fe640 10 client.18864 handle_command_reply: tid=0 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff684a2b740 1 client.18864 shutdown 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff684a2b740 1 -- 172.21.15.78:0/1099729226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 msgr2=0x563bcaa04420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff684a2b740 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 0x563bcaa04420 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0ac060 tx=0x7ff66c0aa880 comp rx=0 tx=0).stop 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff684a2b740 1 -- 172.21.15.78:0/1099729226 shutdown_connections 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff684a2b740 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563bcaa11320 0x563bcaa04ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff684a2b740 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563bcaa0edb0 0x563bcaa04960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff684a2b740 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x563bcaa909e0 0x563bcaa92e00 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff684a2b740 1 --2- 172.21.15.78:0/1099729226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bca9fea10 0x563bcaa04420 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff684a2b740 1 -- 172.21.15.78:0/1099729226 >> 172.21.15.78:0/1099729226 conn(0x563bca9fbf20 msgr2=0x563bcaa0d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff65b7fe640 0 client.18864 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff684a2b740 1 -- 172.21.15.78:0/1099729226 shutdown_connections 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff684a2b740 1 -- 172.21.15.78:0/1099729226 wait complete. 2024-09-11T01:18:14.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.624+0000 7ff684a2b740 20 client.18864 trim_cache size 0 max 16384 2024-09-11T01:18:14.637 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:18:14.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.625+0000 7ff682656640 1 -- 172.21.15.78:0/2519757900 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff6640b5450 msgr2=0x7ff6640b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.625+0000 7ff682656640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff6640b5450 0x7ff6640b7910 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff674005c40 tx=0x7ff674009500 comp rx=0 tx=0).stop 2024-09-11T01:18:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.625+0000 7ff682656640 1 -- 172.21.15.78:0/2519757900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff67c101c90 msgr2=0x7ff67c198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.625+0000 7ff682656640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff67c101c90 0x7ff67c198180 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00d950 tx=0x7ff66c00de10 comp rx=0 tx=0).stop 2024-09-11T01:18:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.625+0000 7ff682656640 1 -- 172.21.15.78:0/2519757900 shutdown_connections 2024-09-11T01:18:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.625+0000 7ff682656640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff6640b5450 0x7ff6640b7910 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.625+0000 7ff682656640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff67c108020 0x7ff67c19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.625+0000 7ff682656640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff67c1076a0 0x7ff67c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.625+0000 7ff682656640 1 --2- 172.21.15.78:0/2519757900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff67c101c90 0x7ff67c198180 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.625+0000 7ff682656640 1 -- 172.21.15.78:0/2519757900 >> 172.21.15.78:0/2519757900 conn(0x7ff67c0fd700 msgr2=0x7ff67c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.625+0000 7ff682656640 1 -- 172.21.15.78:0/2519757900 shutdown_connections 2024-09-11T01:18:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:14.625+0000 7ff682656640 1 -- 172.21.15.78:0/2519757900 wait complete. 2024-09-11T01:18:14.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 247519 34.73 MB/sec execute 260 sec latency 102.555 ms 2024-09-11T01:18:15.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 248102 34.68 MB/sec execute 261 sec latency 79.854 ms 2024-09-11T01:18:15.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:15 smithi110 ceph-mon[34712]: pgmap v930: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 722 op/s 2024-09-11T01:18:15.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:15 smithi112 ceph-mon[35384]: pgmap v930: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 722 op/s 2024-09-11T01:18:15.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:15 smithi078 ceph-mon[31315]: pgmap v930: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 722 op/s 2024-09-11T01:18:16.651 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:18:16.651 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:18:16.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 248853 34.71 MB/sec execute 262 sec latency 76.134 ms 2024-09-11T01:18:17.018 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:18:17.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.605+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1736562047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b481076a0 msgr2=0x7f3b48107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:17.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.605+0000 7f3b4f31b640 1 --2- 172.21.15.78:0/1736562047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b481076a0 0x7f3b48107ae0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009860 tx=0x7f3b4402a340 comp rx=0 tx=0).stop 2024-09-11T01:18:17.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.605+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1736562047 shutdown_connections 2024-09-11T01:18:17.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.605+0000 7f3b4f31b640 1 --2- 172.21.15.78:0/1736562047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b48108020 0x7f3b4810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.605+0000 7f3b4f31b640 1 --2- 172.21.15.78:0/1736562047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b481076a0 0x7f3b48107ae0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.605+0000 7f3b4f31b640 1 --2- 172.21.15.78:0/1736562047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b48101c90 0x7f3b48102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.605+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1736562047 >> 172.21.15.78:0/1736562047 conn(0x7f3b480fd700 msgr2=0x7f3b480ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:17.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.605+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1736562047 shutdown_connections 2024-09-11T01:18:17.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.605+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1736562047 wait complete. 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.605+0000 7f3b4f31b640 1 Processor -- start 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4f31b640 1 -- start start 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4f31b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b48101c90 0x7f3b4819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4f31b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b481076a0 0x7f3b4819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4f31b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b48108020 0x7f3b481a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4f31b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48110bb0 con 0x7f3b48101c90 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4f31b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48110a30 con 0x7f3b481076a0 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4f31b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48110d30 con 0x7f3b48108020 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4d0b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b48101c90 0x7f3b4819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4d0b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b48101c90 0x7f3b4819b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37096/0 (socket says 172.21.15.78:37096) 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4d0b7640 1 -- 172.21.15.78:0/1028748211 learned_addr learned my addr 172.21.15.78:0/1028748211 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4c8b6640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b481076a0 0x7f3b4819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4d8b8640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b48108020 0x7f3b481a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4d0b7640 1 -- 172.21.15.78:0/1028748211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b48108020 msgr2=0x7f3b481a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4d0b7640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b48108020 0x7f3b481a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4d0b7640 1 -- 172.21.15.78:0/1028748211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b481076a0 msgr2=0x7f3b4819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4d0b7640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b481076a0 0x7f3b4819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.606+0000 7f3b4d0b7640 1 -- 172.21.15.78:0/1028748211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b481a1540 con 0x7f3b48101c90 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.607+0000 7f3b4d0b7640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b48101c90 0x7f3b4819b490 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00ea60 tx=0x7f3b3c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.607+0000 7f3b3a7fc640 1 -- 172.21.15.78:0/1028748211 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c00cde0 con 0x7f3b48101c90 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.607+0000 7f3b3a7fc640 1 -- 172.21.15.78:0/1028748211 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b3c0035f0 con 0x7f3b48101c90 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.607+0000 7f3b3a7fc640 1 -- 172.21.15.78:0/1028748211 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c010690 con 0x7f3b48101c90 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.607+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1028748211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b481a1830 con 0x7f3b48101c90 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.607+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1028748211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b481a7ef0 con 0x7f3b48101c90 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.611+0000 7f3b516f0740 1 Processor -- start 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.611+0000 7f3b3a7fc640 1 -- 172.21.15.78:0/1028748211 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3b3c004000 con 0x7f3b48101c90 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.611+0000 7f3b516f0740 1 -- start start 2024-09-11T01:18:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.611+0000 7f3b516f0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 0x555894ed92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.611+0000 7f3b516f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894ee5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.611+0000 7f3b516f0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ee5790 0x555894ee7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.611+0000 7f3b516f0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555894e4fe60 con 0x555894ee5790 2024-09-11T01:18:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.611+0000 7f3b516f0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555894e3f290 con 0x555894ed8ee0 2024-09-11T01:18:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.611+0000 7f3b516f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555894e430a0 con 0x555894ed9800 2024-09-11T01:18:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.612+0000 7f3b4d8b8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 0x555894ed92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.612+0000 7f3b4d8b8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 0x555894ed92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43936/0 (socket says 172.21.15.78:43936) 2024-09-11T01:18:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.612+0000 7f3b4c8b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ee5790 0x555894ee7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.612+0000 7f3b4d8b8640 1 -- 172.21.15.78:0/3368863427 learned_addr learned my addr 172.21.15.78:0/3368863427 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.612+0000 7f3b4c8b6640 1 -- 172.21.15.78:0/3368863427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 msgr2=0x555894ee5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.613+0000 7f3b4d0b7640 1 --2- 172.21.15.78:0/3368863427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894ee5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.613+0000 7f3b4c8b6640 1 --2- 172.21.15.78:0/3368863427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894ee5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.613+0000 7f3b4c8b6640 1 -- 172.21.15.78:0/3368863427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 msgr2=0x555894ed92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.613+0000 7f3b4c8b6640 1 --2- 172.21.15.78:0/3368863427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 0x555894ed92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.613+0000 7f3b4c8b6640 1 -- 172.21.15.78:0/3368863427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555894b9d930 con 0x555894ee5790 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.613+0000 7f3b4d8b8640 1 --2- 172.21.15.78:0/3368863427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 0x555894ed92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.613+0000 7f3b4c8b6640 1 --2- 172.21.15.78:0/3368863427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ee5790 0x555894ee7b80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f3b44035040 tx=0x7f3b44005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802f6dbad2de7670 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.613+0000 7f3b337fe640 1 -- 172.21.15.78:0/3368863427 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44037040 con 0x555894ee5790 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.613+0000 7f3b337fe640 1 -- 172.21.15.78:0/3368863427 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b44030450 con 0x555894ee5790 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.613+0000 7f3b337fe640 1 -- 172.21.15.78:0/3368863427 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44003970 con 0x555894ee5790 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.613+0000 7f3b3a7fc640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3b340b5440 0x7f3b340b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.614+0000 7f3b3a7fc640 1 -- 172.21.15.78:0/1028748211 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f3b3c0e6280 con 0x7f3b48101c90 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.614+0000 7f3b4d8b8640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3b340b5440 0x7f3b340b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.614+0000 7f3b516f0740 1 -- 172.21.15.78:0/3368863427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ee5790 msgr2=0x555894ee7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.614+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3368863427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ee5790 0x555894ee7b80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f3b44035040 tx=0x7f3b44005b10 comp rx=0 tx=0).stop 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.614+0000 7f3b516f0740 1 -- 172.21.15.78:0/3368863427 shutdown_connections 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.614+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3368863427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ee5790 0x555894ee7b80 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.614+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3368863427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894ee5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.614+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3368863427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 0x555894ed92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.614+0000 7f3b516f0740 1 -- 172.21.15.78:0/3368863427 >> 172.21.15.78:0/3368863427 conn(0x555894ed6d10 msgr2=0x555894ed7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.614+0000 7f3b516f0740 1 -- 172.21.15.78:0/3368863427 shutdown_connections 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.614+0000 7f3b516f0740 1 -- 172.21.15.78:0/3368863427 wait complete. 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.615+0000 7f3b516f0740 1 -- start start 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.615+0000 7f3b516f0740 1 Processor -- start 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.615+0000 7f3b4d8b8640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3b340b5440 0x7f3b340b7900 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e840 tx=0x7f3b40010040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b516f0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ed8ee0 0x555894ee2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b516f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894ee2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b516f0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 0x555894ee3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b516f0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555894ca0890 con 0x555894ed8ee0 2024-09-11T01:18:17.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b516f0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555894dd9d40 con 0x555894ee5790 2024-09-11T01:18:17.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b516f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555894ef0bf0 con 0x555894ed9800 2024-09-11T01:18:17.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b516f0740 10 client.0 _ll_register_callbacks cb 0x555894e3dc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:17.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b4d8b8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 0x555894ee3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b4d8b8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 0x555894ee3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43946/0 (socket says 172.21.15.78:43946) 2024-09-11T01:18:17.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b4d8b8640 1 -- 172.21.15.78:0/3555732282 learned_addr learned my addr 172.21.15.78:0/3555732282 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:17.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b4d0b7640 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ed8ee0 0x555894ee2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b4c8b6640 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894ee2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b4d8b8640 1 -- 172.21.15.78:0/3555732282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 msgr2=0x555894ee2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b4d8b8640 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894ee2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b4d8b8640 1 -- 172.21.15.78:0/3555732282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ed8ee0 msgr2=0x555894ee2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b4d8b8640 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ed8ee0 0x555894ee2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.616+0000 7f3b4d8b8640 1 -- 172.21.15.78:0/3555732282 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555894efa020 con 0x555894ee5790 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.617+0000 7f3b4d8b8640 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 0x555894ee3210 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b40003c20 tx=0x7f3b40004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b31c2da6b1842a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.617+0000 7f3b30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.617+0000 7f3b30ff9640 1 -- 172.21.15.78:0/3555732282 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4002a060 con 0x555894ee5790 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.617+0000 7f3b30ff9640 1 -- 172.21.15.78:0/3555732282 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f3b40018460 con 0x555894ee5790 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.617+0000 7f3b516f0740 1 -- 172.21.15.78:0/3555732282 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b44030450 con 0x555894ee5790 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.617+0000 7f3b30ff9640 1 -- 172.21.15.78:0/3555732282 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f3b40003e60 con 0x555894ee5790 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.617+0000 7f3b516f0740 10 client.27974 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.617+0000 7f3b516f0740 1 -- 172.21.15.78:0/3555732282 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x555894efca10 con 0x555894ee5790 2024-09-11T01:18:17.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.618+0000 7f3b30ff9640 1 -- 172.21.15.78:0/3555732282 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b44030450 con 0x555894ee5790 2024-09-11T01:18:17.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.618+0000 7f3b30ff9640 1 -- 172.21.15.78:0/3555732282 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f3b4000b040 con 0x555894ee5790 2024-09-11T01:18:17.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.618+0000 7f3b516f0740 10 client.27974 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:17.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.619+0000 7f3b516f0740 10 client.27974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:18:17.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.619+0000 7f3b516f0740 20 client.27974 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:17.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.619+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555894efdf20 0x555894f00310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.619+0000 7f3b516f0740 4 client.27974 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:18:17.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.619+0000 7f3b516f0740 1 -- 172.21.15.78:0/3555732282 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555894e51db0 con 0x555894efdf20 2024-09-11T01:18:17.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.619+0000 7f3b4d0b7640 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555894efdf20 0x555894f00310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.620+0000 7f3b30ff9640 10 client.27974 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.620+0000 7f3b4d0b7640 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555894efdf20 0x555894f00310 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7528a6422ad5957f server_cookie=79e51f1efb29a13d in_seq=0 out_seq=0 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.621+0000 7f3b30ff9640 1 -- 172.21.15.78:0/3555732282 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555894e51db0 con 0x555894efdf20 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.621+0000 7f3b30ff9640 10 client.27974 handle_command_reply: tid=0 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b516f0740 1 client.27974 shutdown 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b516f0740 1 -- 172.21.15.78:0/3555732282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 msgr2=0x555894ee3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 0x555894ee3210 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b40003c20 tx=0x7f3b40004ab0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b516f0740 1 -- 172.21.15.78:0/3555732282 shutdown_connections 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 0x555894ee3210 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894ee2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555894efdf20 0x555894f00310 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3555732282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ed8ee0 0x555894ee2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b516f0740 1 -- 172.21.15.78:0/3555732282 >> 172.21.15.78:0/3555732282 conn(0x555894ed6d10 msgr2=0x555894ed5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:17.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b30ff9640 0 client.27974 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:17.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b516f0740 1 -- 172.21.15.78:0/3555732282 shutdown_connections 2024-09-11T01:18:17.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b516f0740 1 -- 172.21.15.78:0/3555732282 wait complete. 2024-09-11T01:18:17.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.622+0000 7f3b516f0740 20 client.27974 trim_cache size 0 max 16384 2024-09-11T01:18:17.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.645+0000 7f3b516f0740 1 Processor -- start 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b516f0740 1 -- start start 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b516f0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 0x555894ee44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b516f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894edf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b516f0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ee5790 0x555894edfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b516f0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481ac360 con 0x555894ee5790 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b516f0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481adea0 con 0x555894ed8ee0 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b516f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481aca30 con 0x555894ed9800 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b4d8b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ee5790 0x555894edfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b4d0b7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 0x555894ee44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b4c8b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894edf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b4d0b7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 0x555894ee44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43958/0 (socket says 172.21.15.78:43958) 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b4c8b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894edf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:58902/0 (socket says 172.21.15.78:58902) 2024-09-11T01:18:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b4d0b7640 1 -- 172.21.15.78:0/1100857847 learned_addr learned my addr 172.21.15.78:0/1100857847 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:17.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b4d8b8640 1 -- 172.21.15.78:0/1100857847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 msgr2=0x555894edf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:17.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b4d8b8640 1 --2- 172.21.15.78:0/1100857847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894edf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b4d8b8640 1 -- 172.21.15.78:0/1100857847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 msgr2=0x555894ee44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b4d8b8640 1 --2- 172.21.15.78:0/1100857847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 0x555894ee44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.646+0000 7f3b4d8b8640 1 -- 172.21.15.78:0/1100857847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555894ee16f0 con 0x555894ee5790 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.647+0000 7f3b4d8b8640 1 --2- 172.21.15.78:0/1100857847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ee5790 0x555894edfee0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f3b4001aa00 tx=0x7f3b400091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d9fa72942e95a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.647+0000 7f3b32ffd640 1 -- 172.21.15.78:0/1100857847 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40004890 con 0x555894ee5790 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.647+0000 7f3b32ffd640 1 -- 172.21.15.78:0/1100857847 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b4001ae50 con 0x555894ee5790 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.647+0000 7f3b32ffd640 1 -- 172.21.15.78:0/1100857847 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4002fb80 con 0x555894ee5790 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.648+0000 7f3b516f0740 1 -- 172.21.15.78:0/1100857847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ee5790 msgr2=0x555894edfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.648+0000 7f3b516f0740 1 --2- 172.21.15.78:0/1100857847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ee5790 0x555894edfee0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f3b4001aa00 tx=0x7f3b400091e0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.648+0000 7f3b516f0740 1 -- 172.21.15.78:0/1100857847 shutdown_connections 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.648+0000 7f3b516f0740 1 --2- 172.21.15.78:0/1100857847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ee5790 0x555894edfee0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.648+0000 7f3b516f0740 1 --2- 172.21.15.78:0/1100857847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894edf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.648+0000 7f3b516f0740 1 --2- 172.21.15.78:0/1100857847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ed8ee0 0x555894ee44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.648+0000 7f3b516f0740 1 -- 172.21.15.78:0/1100857847 >> 172.21.15.78:0/1100857847 conn(0x555894ed6d10 msgr2=0x555894ee3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.648+0000 7f3b516f0740 1 -- 172.21.15.78:0/1100857847 shutdown_connections 2024-09-11T01:18:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.648+0000 7f3b516f0740 1 -- 172.21.15.78:0/1100857847 wait complete. 2024-09-11T01:18:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.648+0000 7f3b516f0740 1 -- start start 2024-09-11T01:18:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b516f0740 1 Processor -- start 2024-09-11T01:18:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b516f0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ed8ee0 0x555894ef6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b516f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894ef29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b516f0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 0x555894ef2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b516f0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481ad7d0 con 0x555894ed8ee0 2024-09-11T01:18:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b516f0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481ad100 con 0x555894ee5790 2024-09-11T01:18:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b516f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481aa380 con 0x555894ed9800 2024-09-11T01:18:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b516f0740 10 client.0 _ll_register_callbacks cb 0x7f3b40009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b4d0b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ed8ee0 0x555894ef6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b4d0b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ed8ee0 0x555894ef6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37144/0 (socket says 172.21.15.78:37144) 2024-09-11T01:18:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b4d0b7640 1 -- 172.21.15.78:0/3521301223 learned_addr learned my addr 172.21.15.78:0/3521301223 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b4d8b8640 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 0x555894ef2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.649+0000 7f3b4c8b6640 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894ef29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b4d0b7640 1 -- 172.21.15.78:0/3521301223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 msgr2=0x555894ef29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b4d0b7640 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894ef29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b4d0b7640 1 -- 172.21.15.78:0/3521301223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 msgr2=0x555894ef2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b4d0b7640 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 0x555894ef2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b4d0b7640 1 -- 172.21.15.78:0/3521301223 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555894ef3580 con 0x555894ed8ee0 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b4d8b8640 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 0x555894ef2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b4d0b7640 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ed8ee0 0x555894ef6450 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c004610 tx=0x7f3b3c003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629a65080255f9b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b30ff9640 1 -- 172.21.15.78:0/3521301223 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0ac740 con 0x555894ed8ee0 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b30ff9640 1 -- 172.21.15.78:0/3521301223 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f3b3c0e3c90 con 0x555894ed8ee0 2024-09-11T01:18:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b516f0740 1 -- 172.21.15.78:0/3521301223 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b4001ae50 con 0x555894ed8ee0 2024-09-11T01:18:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b30ff9640 1 -- 172.21.15.78:0/3521301223 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f3b3c0b1220 con 0x555894ed8ee0 2024-09-11T01:18:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b516f0740 10 client.18882 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b516f0740 1 -- 172.21.15.78:0/3521301223 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x555894f6bd10 con 0x555894ed8ee0 2024-09-11T01:18:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.650+0000 7f3b30ff9640 1 -- 172.21.15.78:0/3521301223 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b4001ae50 con 0x555894ed8ee0 2024-09-11T01:18:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.651+0000 7f3b30ff9640 1 -- 172.21.15.78:0/3521301223 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f3b3c0a8090 con 0x555894ed8ee0 2024-09-11T01:18:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.651+0000 7f3b516f0740 10 client.18882 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.651+0000 7f3b516f0740 10 client.18882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:18:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.651+0000 7f3b516f0740 20 client.18882 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.651+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555894f6c4b0 0x555894f6e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.651+0000 7f3b516f0740 4 client.18882 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:18:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.651+0000 7f3b516f0740 1 -- 172.21.15.78:0/3521301223 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555894e4d860 con 0x555894f6c4b0 2024-09-11T01:18:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.651+0000 7f3b4c8b6640 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555894f6c4b0 0x555894f6e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.651+0000 7f3b30ff9640 10 client.18882 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.651+0000 7f3b4c8b6640 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555894f6c4b0 0x555894f6e8d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82cec1c61ceea6f4 server_cookie=712f26e8d88c36b2 in_seq=0 out_seq=0 2024-09-11T01:18:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b30ff9640 1 -- 172.21.15.78:0/3521301223 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555894e4d860 con 0x555894f6c4b0 2024-09-11T01:18:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b30ff9640 10 client.18882 handle_command_reply: tid=0 2024-09-11T01:18:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b516f0740 1 client.18882 shutdown 2024-09-11T01:18:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b516f0740 1 -- 172.21.15.78:0/3521301223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ed8ee0 msgr2=0x555894ef6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ed8ee0 0x555894ef6450 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c004610 tx=0x7f3b3c003cf0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b516f0740 1 -- 172.21.15.78:0/3521301223 shutdown_connections 2024-09-11T01:18:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555894ee5790 0x555894ef2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555894ed9800 0x555894ef29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555894f6c4b0 0x555894f6e8d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b516f0740 1 --2- 172.21.15.78:0/3521301223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555894ed8ee0 0x555894ef6450 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b516f0740 1 -- 172.21.15.78:0/3521301223 >> 172.21.15.78:0/3521301223 conn(0x555894ed6d10 msgr2=0x555894ee45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b30ff9640 0 client.18882 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b516f0740 1 -- 172.21.15.78:0/3521301223 shutdown_connections 2024-09-11T01:18:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b516f0740 1 -- 172.21.15.78:0/3521301223 wait complete. 2024-09-11T01:18:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.655+0000 7f3b516f0740 20 client.18882 trim_cache size 0 max 16384 2024-09-11T01:18:17.661 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:18:17.661 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:18:17.661 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "e66fef11-baf9-43ef-9bea-6ece866f92fa", 2024-09-11T01:18:17.662 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:18:17.662 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:18:17.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.656+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1028748211 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3b340b5440 msgr2=0x7f3b340b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:17.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.656+0000 7f3b4f31b640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3b340b5440 0x7f3b340b7900 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e840 tx=0x7f3b40010040 comp rx=0 tx=0).stop 2024-09-11T01:18:17.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.656+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1028748211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b48101c90 msgr2=0x7f3b4819b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:17.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.656+0000 7f3b4f31b640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b48101c90 0x7f3b4819b490 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00ea60 tx=0x7f3b3c00ef20 comp rx=0 tx=0).stop 2024-09-11T01:18:17.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.656+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1028748211 shutdown_connections 2024-09-11T01:18:17.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.656+0000 7f3b4f31b640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3b340b5440 0x7f3b340b7900 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.656+0000 7f3b4f31b640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b48108020 0x7f3b481a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.656+0000 7f3b4f31b640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b481076a0 0x7f3b4819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.656+0000 7f3b4f31b640 1 --2- 172.21.15.78:0/1028748211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b48101c90 0x7f3b4819b490 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.656+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1028748211 >> 172.21.15.78:0/1028748211 conn(0x7f3b480fd700 msgr2=0x7f3b480ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.656+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1028748211 shutdown_connections 2024-09-11T01:18:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:17.656+0000 7f3b4f31b640 1 -- 172.21.15.78:0/1028748211 wait complete. 2024-09-11T01:18:17.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:17 smithi078 ceph-mon[31315]: pgmap v931: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 668 op/s 2024-09-11T01:18:17.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 249418 34.68 MB/sec execute 263 sec latency 157.536 ms 2024-09-11T01:18:17.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:17 smithi110 ceph-mon[34712]: pgmap v931: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 668 op/s 2024-09-11T01:18:17.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:17 smithi112 ceph-mon[35384]: pgmap v931: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 668 op/s 2024-09-11T01:18:18.488 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:18:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:18 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:18:18.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:18:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:18:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:18:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:18:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:18.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:18.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 249730 34.59 MB/sec execute 264 sec latency 432.157 ms 2024-09-11T01:18:18.814 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:18:18.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:18 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:18:18.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:18.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:18:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:18:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:18:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:18:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:18.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:18 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:18:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:18:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:18:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:18:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:18:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:18.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:19.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 250100 34.53 MB/sec execute 265 sec latency 501.889 ms 2024-09-11T01:18:19.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:19 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:18:19.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:19 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:19.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:19 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:19.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:19 smithi110 ceph-mon[34712]: pgmap v932: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 633 op/s 2024-09-11T01:18:19.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:19 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:19.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:19 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:18:20.041 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:19 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:20.042 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:19 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:20.042 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:19 smithi112 ceph-mon[35384]: pgmap v932: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 633 op/s 2024-09-11T01:18:20.042 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:19 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:20.042 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:19 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:18:20.043 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:19 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:20.043 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:19 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:20.043 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:19 smithi078 ceph-mon[31315]: pgmap v932: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 633 op/s 2024-09-11T01:18:20.043 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:19 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:18:20.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 250329 34.45 MB/sec execute 266 sec latency 428.376 ms 2024-09-11T01:18:20.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.378+0000 7f454039e640 1 -- 172.21.15.78:0/2331164467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f453810a150 msgr2=0x7f45381111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:20.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.378+0000 7f454039e640 1 --2- 172.21.15.78:0/2331164467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f453810a150 0x7f45381111b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4530009be0 tx=0x7f453002a770 comp rx=0 tx=0).stop 2024-09-11T01:18:20.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.380+0000 7f454039e640 1 -- 172.21.15.78:0/2331164467 shutdown_connections 2024-09-11T01:18:20.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.380+0000 7f454039e640 1 --2- 172.21.15.78:0/2331164467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f453811c0a0 0x7f453811e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.380+0000 7f454039e640 1 --2- 172.21.15.78:0/2331164467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f453810a530 0x7f453810a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.380+0000 7f454039e640 1 --2- 172.21.15.78:0/2331164467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f453810a150 0x7f45381111b0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.380+0000 7f454039e640 1 -- 172.21.15.78:0/2331164467 >> 172.21.15.78:0/2331164467 conn(0x7f453806dfc0 msgr2=0x7f453806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:20.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.385+0000 7f454039e640 1 -- 172.21.15.78:0/2331164467 shutdown_connections 2024-09-11T01:18:20.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.386+0000 7f454039e640 1 -- 172.21.15.78:0/2331164467 wait complete. 2024-09-11T01:18:20.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.388+0000 7f454039e640 1 Processor -- start 2024-09-11T01:18:20.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.391+0000 7f454039e640 1 -- start start 2024-09-11T01:18:20.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.391+0000 7f454039e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f453810a150 0x7f45381a8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.392+0000 7f454039e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f453810a530 0x7f45381a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.392+0000 7f454039e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f453811c0a0 0x7f45381ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.392+0000 7f454039e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4538121c10 con 0x7f453810a150 2024-09-11T01:18:20.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.392+0000 7f454039e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4538121a90 con 0x7f453811c0a0 2024-09-11T01:18:20.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.392+0000 7f454039e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4538121d90 con 0x7f453810a530 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.394+0000 7f453d939640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f453810a530 0x7f45381a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.394+0000 7f453d939640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f453810a530 0x7f45381a86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:58920/0 (socket says 172.21.15.78:58920) 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.394+0000 7f453d939640 1 -- 172.21.15.78:0/3577573031 learned_addr learned my addr 172.21.15.78:0/3577573031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.395+0000 7f453e93b640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f453811c0a0 0x7f45381ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.396+0000 7f453e13a640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f453810a150 0x7f45381a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.399+0000 7f453e13a640 1 -- 172.21.15.78:0/3577573031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f453810a530 msgr2=0x7f45381a86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.399+0000 7f453e13a640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f453810a530 0x7f45381a86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.399+0000 7f453e13a640 1 -- 172.21.15.78:0/3577573031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f453811c0a0 msgr2=0x7f45381ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.399+0000 7f453e13a640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f453811c0a0 0x7f45381ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.399+0000 7f453e13a640 1 -- 172.21.15.78:0/3577573031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45381ae230 con 0x7f453810a150 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.399+0000 7f453d939640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f453810a530 0x7f45381a86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.410+0000 7f453e13a640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f453810a150 0x7f45381a8180 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4530034040 tx=0x7f4530003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.413+0000 7f452f7fe640 1 -- 172.21.15.78:0/3577573031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530041020 con 0x7f453810a150 2024-09-11T01:18:20.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.413+0000 7f454039e640 1 -- 172.21.15.78:0/3577573031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45381ae4c0 con 0x7f453810a150 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.413+0000 7f454039e640 1 -- 172.21.15.78:0/3577573031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4538111880 con 0x7f453810a150 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.415+0000 7f452f7fe640 1 -- 172.21.15.78:0/3577573031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45300318f0 con 0x7f453810a150 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.415+0000 7f452f7fe640 1 -- 172.21.15.78:0/3577573031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530036040 con 0x7f453810a150 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.417+0000 7f4542773740 1 Processor -- start 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.419+0000 7f4542773740 1 -- start start 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.420+0000 7f4542773740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01ce1480 0x55ce01ce1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.420+0000 7f4542773740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1da0 0x55ce01ced7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.420+0000 7f4542773740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01cedd30 0x55ce01cf0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.420+0000 7f4542773740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce01c57e50 con 0x55ce01ce1da0 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.420+0000 7f4542773740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ce01c47280 con 0x55ce01ce1480 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.420+0000 7f4542773740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ce01c4b090 con 0x55ce01cedd30 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.422+0000 7f453d939640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01cedd30 0x55ce01cf0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.422+0000 7f453e93b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01ce1480 0x55ce01ce1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.423+0000 7f453e93b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01ce1480 0x55ce01ce1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43990/0 (socket says 172.21.15.78:43990) 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.423+0000 7f453e93b640 1 -- 172.21.15.78:0/1810126504 learned_addr learned my addr 172.21.15.78:0/1810126504 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.426+0000 7f453e93b640 1 -- 172.21.15.78:0/1810126504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01cedd30 msgr2=0x55ce01cf0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.426+0000 7f453e93b640 1 --2- 172.21.15.78:0/1810126504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01cedd30 0x55ce01cf0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.426+0000 7f453e93b640 1 -- 172.21.15.78:0/1810126504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1da0 msgr2=0x55ce01ced7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.426+0000 7f453e93b640 1 --2- 172.21.15.78:0/1810126504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1da0 0x55ce01ced7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.426+0000 7f453e93b640 1 -- 172.21.15.78:0/1810126504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce019a5650 con 0x55ce01ce1480 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.428+0000 7f453e93b640 1 --2- 172.21.15.78:0/1810126504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01ce1480 0x55ce01ce1860 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f452800b7a0 tx=0x7f452800bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73712df3c93ce331 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.430+0000 7f452cde6640 1 -- 172.21.15.78:0/1810126504 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452800d950 con 0x55ce01ce1480 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.430+0000 7f452cde6640 1 -- 172.21.15.78:0/1810126504 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4528012420 con 0x55ce01ce1480 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.431+0000 7f4542773740 1 -- 172.21.15.78:0/1810126504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01ce1480 msgr2=0x55ce01ce1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.431+0000 7f4542773740 1 --2- 172.21.15.78:0/1810126504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01ce1480 0x55ce01ce1860 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f452800b7a0 tx=0x7f452800bc60 comp rx=0 tx=0).stop 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.432+0000 7f452f7fe640 1 -- 172.21.15.78:0/3577573031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4530031440 con 0x7f453810a150 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.433+0000 7f4542773740 1 -- 172.21.15.78:0/1810126504 shutdown_connections 2024-09-11T01:18:20.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.433+0000 7f4542773740 1 --2- 172.21.15.78:0/1810126504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01cedd30 0x55ce01cf0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.433+0000 7f4542773740 1 --2- 172.21.15.78:0/1810126504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1da0 0x55ce01ced7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.433+0000 7f4542773740 1 --2- 172.21.15.78:0/1810126504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01ce1480 0x55ce01ce1860 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.433+0000 7f4542773740 1 -- 172.21.15.78:0/1810126504 >> 172.21.15.78:0/1810126504 conn(0x55ce01cdf2b0 msgr2=0x55ce01ce0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.433+0000 7f452f7fe640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f45240b55d0 0x7f45240b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.433+0000 7f452f7fe640 1 -- 172.21.15.78:0/3577573031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f4530106570 con 0x7f453810a150 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.436+0000 7f453d939640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f45240b55d0 0x7f45240b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.437+0000 7f453d939640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f45240b55d0 0x7f45240b7a90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f453811b860 tx=0x7f4534009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.441+0000 7f4542773740 1 -- 172.21.15.78:0/1810126504 shutdown_connections 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.441+0000 7f4542773740 1 -- 172.21.15.78:0/1810126504 wait complete. 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.441+0000 7f4542773740 1 -- start start 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.441+0000 7f4542773740 1 Processor -- start 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.443+0000 7f4542773740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 0x55ce01ced4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.443+0000 7f4542773740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01ce1da0 0x55ce01cea930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.443+0000 7f4542773740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01cedd30 0x55ce01ceae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.443+0000 7f4542773740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce01aa8700 con 0x55ce01ce1480 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.443+0000 7f4542773740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ce01be2110 con 0x55ce01cedd30 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.443+0000 7f4542773740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ce01cf0a00 con 0x55ce01ce1da0 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.443+0000 7f4542773740 10 client.0 _ll_register_callbacks cb 0x55ce01bfb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.445+0000 7f453e93b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01cedd30 0x55ce01ceae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.445+0000 7f453d939640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01ce1da0 0x55ce01cea930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.445+0000 7f453d939640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01ce1da0 0x55ce01cea930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:58940/0 (socket says 172.21.15.78:58940) 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.445+0000 7f453d939640 1 -- 172.21.15.78:0/1482224548 learned_addr learned my addr 172.21.15.78:0/1482224548 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.445+0000 7f453e13a640 1 --2- 172.21.15.78:0/1482224548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 0x55ce01ced4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.446+0000 7f453e13a640 1 -- 172.21.15.78:0/1482224548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01ce1da0 msgr2=0x55ce01cea930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.446+0000 7f453e13a640 1 --2- 172.21.15.78:0/1482224548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01ce1da0 0x55ce01cea930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.446+0000 7f453e13a640 1 -- 172.21.15.78:0/1482224548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01cedd30 msgr2=0x55ce01ceae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.446+0000 7f453e13a640 1 --2- 172.21.15.78:0/1482224548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01cedd30 0x55ce01ceae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.446+0000 7f453e13a640 1 -- 172.21.15.78:0/1482224548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce01c0fc40 con 0x55ce01ce1480 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.446+0000 7f453e13a640 1 --2- 172.21.15.78:0/1482224548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 0x55ce01ced4e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f4530103d00 tx=0x7f45300cc8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60902a202f3c388b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.446+0000 7f4521ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.446+0000 7f4521ffb640 1 -- 172.21.15.78:0/1482224548 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45300b5070 con 0x55ce01ce1480 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.446+0000 7f4542773740 1 -- 172.21.15.78:0/1482224548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4528012420 con 0x55ce01ce1480 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.446+0000 7f4521ffb640 1 -- 172.21.15.78:0/1482224548 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f45300d1200 con 0x55ce01ce1480 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.447+0000 7f4521ffb640 1 -- 172.21.15.78:0/1482224548 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f45300cc540 con 0x55ce01ce1480 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.447+0000 7f4542773740 10 client.18891 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.447+0000 7f4542773740 1 -- 172.21.15.78:0/1482224548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55ce01cfc2d0 con 0x55ce01ce1480 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.447+0000 7f4521ffb640 1 -- 172.21.15.78:0/1482224548 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4528012420 con 0x55ce01ce1480 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.448+0000 7f4521ffb640 1 -- 172.21.15.78:0/1482224548 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f45300cbc90 con 0x55ce01ce1480 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.449+0000 7f4542773740 10 client.18891 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.449+0000 7f4542773740 10 client.18891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:18:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.449+0000 7f4542773740 20 client.18891 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.449+0000 7f4542773740 1 --2- 172.21.15.78:0/1482224548 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce01cfd620 0x55ce01cffa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.449+0000 7f4542773740 4 client.18891 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.449+0000 7f4542773740 1 -- 172.21.15.78:0/1482224548 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce01c59da0 con 0x55ce01cfd620 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.455+0000 7f453d939640 1 --2- 172.21.15.78:0/1482224548 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce01cfd620 0x55ce01cffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.464+0000 7f4521ffb640 10 client.18891 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.464+0000 7f453d939640 1 --2- 172.21.15.78:0/1482224548 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce01cfd620 0x55ce01cffa30 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38211e312c06cbaf server_cookie=16fc7f10da02c725 in_seq=0 out_seq=0 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.467+0000 7f4521ffb640 1 -- 172.21.15.78:0/1482224548 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ce01c59da0 con 0x55ce01cfd620 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.467+0000 7f4521ffb640 10 client.18891 handle_command_reply: tid=0 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.468+0000 7f4542773740 1 client.18891 shutdown 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.472+0000 7f4542773740 1 -- 172.21.15.78:0/1482224548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 msgr2=0x55ce01ced4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.472+0000 7f4542773740 1 --2- 172.21.15.78:0/1482224548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 0x55ce01ced4e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f4530103d00 tx=0x7f45300cc8c0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.475+0000 7f4542773740 1 -- 172.21.15.78:0/1482224548 shutdown_connections 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.475+0000 7f4542773740 1 --2- 172.21.15.78:0/1482224548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01cedd30 0x55ce01ceae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.475+0000 7f4542773740 1 --2- 172.21.15.78:0/1482224548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01ce1da0 0x55ce01cea930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.475+0000 7f4542773740 1 --2- 172.21.15.78:0/1482224548 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce01cfd620 0x55ce01cffa30 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.475+0000 7f4542773740 1 --2- 172.21.15.78:0/1482224548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 0x55ce01ced4e0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.475+0000 7f4521ffb640 0 client.18891 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.475+0000 7f4542773740 1 -- 172.21.15.78:0/1482224548 >> 172.21.15.78:0/1482224548 conn(0x55ce01cdf2b0 msgr2=0x55ce01ce4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.476+0000 7f4542773740 1 -- 172.21.15.78:0/1482224548 shutdown_connections 2024-09-11T01:18:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.476+0000 7f4542773740 1 -- 172.21.15.78:0/1482224548 wait complete. 2024-09-11T01:18:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.478+0000 7f4542773740 20 client.18891 trim_cache size 0 max 16384 2024-09-11T01:18:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.516+0000 7f4542773740 1 Processor -- start 2024-09-11T01:18:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.517+0000 7f4542773740 1 -- start start 2024-09-11T01:18:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.517+0000 7f4542773740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01ce1480 0x55ce01ce0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.517+0000 7f4542773740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1da0 0x55ce01cdebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.517+0000 7f4542773740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01cedd30 0x55ce01cef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.517+0000 7f4542773740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f451c00e7a0 con 0x55ce01ce1da0 2024-09-11T01:18:20.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.517+0000 7f4542773740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f451c0102e0 con 0x55ce01cedd30 2024-09-11T01:18:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.517+0000 7f4542773740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f451c00ee70 con 0x55ce01ce1480 2024-09-11T01:18:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.518+0000 7f453e93b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01cedd30 0x55ce01cef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.520+0000 7f453d939640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1da0 0x55ce01cdebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.520+0000 7f453e93b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01cedd30 0x55ce01cef120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:44014/0 (socket says 172.21.15.78:44014) 2024-09-11T01:18:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.520+0000 7f453e93b640 1 -- 172.21.15.78:0/2055798034 learned_addr learned my addr 172.21.15.78:0/2055798034 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.521+0000 7f453e93b640 1 -- 172.21.15.78:0/2055798034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01ce1480 msgr2=0x55ce01ce0370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:18:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.521+0000 7f453e93b640 1 --2- 172.21.15.78:0/2055798034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01ce1480 0x55ce01ce0370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.521+0000 7f453e93b640 1 -- 172.21.15.78:0/2055798034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1da0 msgr2=0x55ce01cdebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.521+0000 7f453e93b640 1 --2- 172.21.15.78:0/2055798034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1da0 0x55ce01cdebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.521+0000 7f453e93b640 1 -- 172.21.15.78:0/2055798034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce01ce5730 con 0x55ce01cedd30 2024-09-11T01:18:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.523+0000 7f453e93b640 1 --2- 172.21.15.78:0/2055798034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01cedd30 0x55ce01cef120 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4528000c00 tx=0x7f4528006ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfc8616ef9cac8ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.525+0000 7f4523fff640 1 -- 172.21.15.78:0/2055798034 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452800db50 con 0x55ce01cedd30 2024-09-11T01:18:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.525+0000 7f4523fff640 1 -- 172.21.15.78:0/2055798034 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45280044f0 con 0x55ce01cedd30 2024-09-11T01:18:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.526+0000 7f4542773740 1 -- 172.21.15.78:0/2055798034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01cedd30 msgr2=0x55ce01cef120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.526+0000 7f4542773740 1 --2- 172.21.15.78:0/2055798034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01cedd30 0x55ce01cef120 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4528000c00 tx=0x7f4528006ec0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.529+0000 7f4542773740 1 -- 172.21.15.78:0/2055798034 shutdown_connections 2024-09-11T01:18:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.529+0000 7f4542773740 1 --2- 172.21.15.78:0/2055798034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01cedd30 0x55ce01cef120 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.529+0000 7f4542773740 1 --2- 172.21.15.78:0/2055798034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1da0 0x55ce01cdebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.529+0000 7f4542773740 1 --2- 172.21.15.78:0/2055798034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01ce1480 0x55ce01ce0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.529+0000 7f4542773740 1 -- 172.21.15.78:0/2055798034 >> 172.21.15.78:0/2055798034 conn(0x55ce01cdf2b0 msgr2=0x55ce01ce91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.531+0000 7f4542773740 1 -- 172.21.15.78:0/2055798034 shutdown_connections 2024-09-11T01:18:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.533+0000 7f4542773740 1 -- 172.21.15.78:0/2055798034 wait complete. 2024-09-11T01:18:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.534+0000 7f4542773740 1 -- start start 2024-09-11T01:18:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.534+0000 7f4542773740 1 Processor -- start 2024-09-11T01:18:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.535+0000 7f4542773740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 0x55ce01cf6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.535+0000 7f4542773740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01ce1da0 0x55ce01cf6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.535+0000 7f4542773740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01cedd30 0x55ce01cf0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.535+0000 7f4542773740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f451c00fc10 con 0x55ce01ce1480 2024-09-11T01:18:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.535+0000 7f4542773740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f451c00f540 con 0x55ce01ce1da0 2024-09-11T01:18:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.535+0000 7f4542773740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f451c00c7a0 con 0x55ce01cedd30 2024-09-11T01:18:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.535+0000 7f4542773740 10 client.0 _ll_register_callbacks cb 0x7f4530005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.536+0000 7f453d939640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01ce1da0 0x55ce01cf6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.538+0000 7f453e13a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 0x55ce01cf6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.538+0000 7f453e13a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 0x55ce01cf6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37190/0 (socket says 172.21.15.78:37190) 2024-09-11T01:18:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.538+0000 7f453e13a640 1 -- 172.21.15.78:0/2727328643 learned_addr learned my addr 172.21.15.78:0/2727328643 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.539+0000 7f453e13a640 1 -- 172.21.15.78:0/2727328643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01cedd30 msgr2=0x55ce01cf0f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:18:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.539+0000 7f453e13a640 1 --2- 172.21.15.78:0/2727328643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01cedd30 0x55ce01cf0f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.539+0000 7f453e13a640 1 -- 172.21.15.78:0/2727328643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01ce1da0 msgr2=0x55ce01cf6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.539+0000 7f453e13a640 1 --2- 172.21.15.78:0/2727328643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01ce1da0 0x55ce01cf6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.539+0000 7f453e13a640 1 -- 172.21.15.78:0/2727328643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce01cf1790 con 0x55ce01ce1480 2024-09-11T01:18:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.540+0000 7f453e13a640 1 --2- 172.21.15.78:0/2727328643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 0x55ce01cf6210 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f4530005ee0 tx=0x7f4530000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ef2a7c81e1b265 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.540+0000 7f4521ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:18:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.545+0000 7f4521ffb640 1 -- 172.21.15.78:0/2727328643 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45300b5070 con 0x55ce01ce1480 2024-09-11T01:18:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.545+0000 7f4542773740 1 -- 172.21.15.78:0/2727328643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45280044f0 con 0x55ce01ce1480 2024-09-11T01:18:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.545+0000 7f4521ffb640 1 -- 172.21.15.78:0/2727328643 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f4530004b40 con 0x55ce01ce1480 2024-09-11T01:18:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.549+0000 7f4521ffb640 1 -- 172.21.15.78:0/2727328643 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f45300cd130 con 0x55ce01ce1480 2024-09-11T01:18:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.551+0000 7f4542773740 10 client.18897 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.551+0000 7f4542773740 1 -- 172.21.15.78:0/2727328643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55ce01d6b260 con 0x55ce01ce1480 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.558+0000 7f4521ffb640 1 -- 172.21.15.78:0/2727328643 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45280044f0 con 0x55ce01ce1480 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.558+0000 7f4521ffb640 1 -- 172.21.15.78:0/2727328643 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f45300c8380 con 0x55ce01ce1480 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.558+0000 7f4542773740 10 client.18897 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.558+0000 7f4542773740 10 client.18897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.558+0000 7f4542773740 20 client.18897 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.558+0000 7f4542773740 1 --2- 172.21.15.78:0/2727328643 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce01d6b7d0 0x55ce01d6dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.558+0000 7f4542773740 4 client.18897 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.558+0000 7f4542773740 1 -- 172.21.15.78:0/2727328643 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce01c55850 con 0x55ce01d6b7d0 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.559+0000 7f453d939640 1 --2- 172.21.15.78:0/2727328643 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce01d6b7d0 0x55ce01d6dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.564+0000 7f4521ffb640 10 client.18897 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.564+0000 7f453d939640 1 --2- 172.21.15.78:0/2727328643 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce01d6b7d0 0x55ce01d6dbf0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5f5801ff98ad535 server_cookie=d84853240776e2bf in_seq=0 out_seq=0 2024-09-11T01:18:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.568+0000 7f4521ffb640 1 -- 172.21.15.78:0/2727328643 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7102 (crc 0 0 0) 0x55ce01c55850 con 0x55ce01d6b7d0 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.568+0000 7f4521ffb640 10 client.18897 handle_command_reply: tid=0 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.568+0000 7f4542773740 1 client.18897 shutdown 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.572+0000 7f4542773740 1 -- 172.21.15.78:0/2727328643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 msgr2=0x55ce01cf6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.572+0000 7f4542773740 1 --2- 172.21.15.78:0/2727328643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 0x55ce01cf6210 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f4530005ee0 tx=0x7f4530000f40 comp rx=0 tx=0).stop 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.572+0000 7f4542773740 1 -- 172.21.15.78:0/2727328643 shutdown_connections 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.572+0000 7f4542773740 1 --2- 172.21.15.78:0/2727328643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce01cedd30 0x55ce01cf0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.572+0000 7f4542773740 1 --2- 172.21.15.78:0/2727328643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ce01ce1da0 0x55ce01cf6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.572+0000 7f4542773740 1 --2- 172.21.15.78:0/2727328643 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55ce01d6b7d0 0x55ce01d6dbf0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.572+0000 7f4542773740 1 --2- 172.21.15.78:0/2727328643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce01ce1480 0x55ce01cf6210 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.572+0000 7f4542773740 1 -- 172.21.15.78:0/2727328643 >> 172.21.15.78:0/2727328643 conn(0x55ce01cdf2b0 msgr2=0x55ce01ce9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.572+0000 7f4521ffb640 0 client.18897 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.572+0000 7f4542773740 1 -- 172.21.15.78:0/2727328643 shutdown_connections 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.572+0000 7f4542773740 1 -- 172.21.15.78:0/2727328643 wait complete. 2024-09-11T01:18:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.572+0000 7f4542773740 20 client.18897 trim_cache size 0 max 16384 2024-09-11T01:18:20.955 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:18:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:18:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:18:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:18:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "e66fef11-baf9-43ef-9bea-6ece866f92fa", 2024-09-11T01:18:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:18:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:18:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:18:17.654570+0000", 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:18:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:18:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:18:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:18:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:18:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:18:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:18:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:18:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:18:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:18:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:18:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:18:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:18:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:18:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:18:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:18:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:18:20.970 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:18:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.577+0000 7f454039e640 1 -- 172.21.15.78:0/3577573031 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f45240b55d0 msgr2=0x7f45240b7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.577+0000 7f454039e640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f45240b55d0 0x7f45240b7a90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f453811b860 tx=0x7f4534009260 comp rx=0 tx=0).stop 2024-09-11T01:18:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.577+0000 7f454039e640 1 -- 172.21.15.78:0/3577573031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f453810a150 msgr2=0x7f45381a8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.577+0000 7f454039e640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f453810a150 0x7f45381a8180 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4530034040 tx=0x7f4530003c10 comp rx=0 tx=0).stop 2024-09-11T01:18:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.577+0000 7f454039e640 1 -- 172.21.15.78:0/3577573031 shutdown_connections 2024-09-11T01:18:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.577+0000 7f454039e640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f45240b55d0 0x7f45240b7a90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.577+0000 7f454039e640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f453811c0a0 0x7f45381ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.577+0000 7f454039e640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f453810a530 0x7f45381a86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.577+0000 7f454039e640 1 --2- 172.21.15.78:0/3577573031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f453810a150 0x7f45381a8180 secure :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4530034040 tx=0x7f4530003c10 comp rx=0 tx=0).stop 2024-09-11T01:18:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.577+0000 7f454039e640 1 -- 172.21.15.78:0/3577573031 >> 172.21.15.78:0/3577573031 conn(0x7f453806dfc0 msgr2=0x7f453811c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.581+0000 7f454039e640 1 -- 172.21.15.78:0/3577573031 shutdown_connections 2024-09-11T01:18:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:19.581+0000 7f454039e640 1 -- 172.21.15.78:0/3577573031 wait complete. 2024-09-11T01:18:21.793 INFO:tasks.workunit.client.0.smithi078.stdout: 10 250983 34.50 MB/sec execute 267 sec latency 132.627 ms 2024-09-11T01:18:21.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:21 smithi110 ceph-mon[34712]: pgmap v933: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 635 op/s 2024-09-11T01:18:21.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:21 smithi112 ceph-mon[35384]: pgmap v933: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 635 op/s 2024-09-11T01:18:21.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:21 smithi078 ceph-mon[31315]: pgmap v933: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 635 op/s 2024-09-11T01:18:22.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 251560 34.48 MB/sec execute 268 sec latency 112.138 ms 2024-09-11T01:18:23.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 252155 34.47 MB/sec execute 269 sec latency 130.606 ms 2024-09-11T01:18:23.826 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:23 smithi078 ceph-mon[31315]: pgmap v934: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 701 op/s 2024-09-11T01:18:23.873 INFO:tasks.cephfs.filesystem:scrub status for tag:e66fef11-baf9-43ef-9bea-6ece866f92fa - {'path': '/', 'tag': 'e66fef11-baf9-43ef-9bea-6ece866f92fa', 'options': 'recursive,force'} 2024-09-11T01:18:23.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:23 smithi110 ceph-mon[34712]: pgmap v934: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 701 op/s 2024-09-11T01:18:23.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:23 smithi112 ceph-mon[35384]: pgmap v934: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 701 op/s 2024-09-11T01:18:24.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 252764 34.46 MB/sec execute 270 sec latency 129.368 ms 2024-09-11T01:18:25.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 253502 34.47 MB/sec execute 271 sec latency 74.345 ms 2024-09-11T01:18:25.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:25 smithi110 ceph-mon[34712]: pgmap v935: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 725 op/s 2024-09-11T01:18:25.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:25 smithi112 ceph-mon[35384]: pgmap v935: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 725 op/s 2024-09-11T01:18:25.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:25 smithi078 ceph-mon[31315]: pgmap v935: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 725 op/s 2024-09-11T01:18:26.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 254366 34.51 MB/sec execute 272 sec latency 75.262 ms 2024-09-11T01:18:27.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 255192 34.55 MB/sec execute 273 sec latency 66.099 ms 2024-09-11T01:18:27.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:27 smithi112 ceph-mon[35384]: pgmap v936: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 736 op/s 2024-09-11T01:18:27.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:27 smithi078 ceph-mon[31315]: pgmap v936: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 736 op/s 2024-09-11T01:18:28.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:27 smithi110 ceph-mon[34712]: pgmap v936: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 736 op/s 2024-09-11T01:18:28.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 255903 34.59 MB/sec execute 274 sec latency 107.722 ms 2024-09-11T01:18:29.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 256517 34.56 MB/sec execute 275 sec latency 122.484 ms 2024-09-11T01:18:29.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:29 smithi112 ceph-mon[35384]: pgmap v937: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 662 op/s 2024-09-11T01:18:29.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:29 smithi078 ceph-mon[31315]: pgmap v937: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 662 op/s 2024-09-11T01:18:30.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:29 smithi110 ceph-mon[34712]: pgmap v937: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 662 op/s 2024-09-11T01:18:30.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 257262 34.59 MB/sec execute 276 sec latency 81.332 ms 2024-09-11T01:18:31.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 258033 34.59 MB/sec execute 277 sec latency 89.755 ms 2024-09-11T01:18:31.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:31 smithi112 ceph-mon[35384]: pgmap v938: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 832 op/s 2024-09-11T01:18:31.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:31 smithi078 ceph-mon[31315]: pgmap v938: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 832 op/s 2024-09-11T01:18:32.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:31 smithi110 ceph-mon[34712]: pgmap v938: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 832 op/s 2024-09-11T01:18:32.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 258802 34.63 MB/sec execute 278 sec latency 89.058 ms 2024-09-11T01:18:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:32 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:18:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:32 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:18:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:32 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:18:32.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:32 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:18:32.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:32 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:18:32.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:32 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:18:33.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:32 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:18:33.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:32 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:18:33.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:32 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:18:33.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 259408 34.65 MB/sec execute 279 sec latency 131.101 ms 2024-09-11T01:18:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:18:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:18:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:33 smithi112 ceph-mon[35384]: pgmap v939: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 687 op/s 2024-09-11T01:18:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:33 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:18:33.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:18:33.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:18:33.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:33 smithi078 ceph-mon[31315]: pgmap v939: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 687 op/s 2024-09-11T01:18:33.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:33 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:18:34.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:18:34.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:18:34.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:33 smithi110 ceph-mon[34712]: pgmap v939: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 687 op/s 2024-09-11T01:18:34.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:33 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:18:34.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 260089 34.63 MB/sec execute 280 sec latency 71.755 ms 2024-09-11T01:18:34.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:18:34.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:18:34.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:18:34.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:18:34.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:18:34.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:18:35.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:18:35.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:18:35.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:18:35.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 260821 34.65 MB/sec execute 281 sec latency 68.827 ms 2024-09-11T01:18:35.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:35 smithi112 ceph-mon[35384]: pgmap v940: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 686 op/s 2024-09-11T01:18:35.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:35 smithi078 ceph-mon[31315]: pgmap v940: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 686 op/s 2024-09-11T01:18:36.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:35 smithi110 ceph-mon[34712]: pgmap v940: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 686 op/s 2024-09-11T01:18:36.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 260954 34.56 MB/sec execute 282 sec latency 821.926 ms 2024-09-11T01:18:37.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 261414 34.52 MB/sec execute 283 sec latency 76.074 ms 2024-09-11T01:18:37.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:37 smithi112 ceph-mon[35384]: pgmap v941: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 657 op/s 2024-09-11T01:18:37.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:37 smithi078 ceph-mon[31315]: pgmap v941: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 657 op/s 2024-09-11T01:18:38.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:37 smithi110 ceph-mon[34712]: pgmap v941: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 657 op/s 2024-09-11T01:18:38.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 262066 34.53 MB/sec execute 284 sec latency 210.852 ms 2024-09-11T01:18:39.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 262627 34.51 MB/sec execute 285 sec latency 156.350 ms 2024-09-11T01:18:39.991 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:39 smithi110 ceph-mon[34712]: pgmap v942: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 588 op/s 2024-09-11T01:18:39.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:39 smithi078 ceph-mon[31315]: pgmap v942: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 588 op/s 2024-09-11T01:18:40.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:39 smithi112 ceph-mon[35384]: pgmap v942: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 588 op/s 2024-09-11T01:18:40.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 263402 34.51 MB/sec execute 286 sec latency 189.723 ms 2024-09-11T01:18:41.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 264082 34.55 MB/sec execute 287 sec latency 234.100 ms 2024-09-11T01:18:41.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:41 smithi078 ceph-mon[31315]: pgmap v943: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 787 op/s 2024-09-11T01:18:42.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:41 smithi110 ceph-mon[34712]: pgmap v943: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 787 op/s 2024-09-11T01:18:42.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:41 smithi112 ceph-mon[35384]: pgmap v943: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 787 op/s 2024-09-11T01:18:42.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 264813 34.56 MB/sec execute 288 sec latency 69.742 ms 2024-09-11T01:18:43.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 265339 34.54 MB/sec execute 289 sec latency 163.451 ms 2024-09-11T01:18:43.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:43 smithi078 ceph-mon[31315]: pgmap v944: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 786 op/s 2024-09-11T01:18:44.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:43 smithi110 ceph-mon[34712]: pgmap v944: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 786 op/s 2024-09-11T01:18:44.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:43 smithi112 ceph-mon[35384]: pgmap v944: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 786 op/s 2024-09-11T01:18:44.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 265842 34.50 MB/sec execute 290 sec latency 246.397 ms 2024-09-11T01:18:45.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 266598 34.51 MB/sec execute 291 sec latency 177.538 ms 2024-09-11T01:18:45.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:45 smithi078 ceph-mon[31315]: pgmap v945: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 20 MiB/s wr, 841 op/s 2024-09-11T01:18:46.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:45 smithi110 ceph-mon[34712]: pgmap v945: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 20 MiB/s wr, 841 op/s 2024-09-11T01:18:46.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:45 smithi112 ceph-mon[35384]: pgmap v945: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 20 MiB/s wr, 841 op/s 2024-09-11T01:18:46.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 267459 34.57 MB/sec execute 292 sec latency 68.683 ms 2024-09-11T01:18:47.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 268199 34.60 MB/sec execute 293 sec latency 65.373 ms 2024-09-11T01:18:47.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:47 smithi078 ceph-mon[31315]: pgmap v946: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 838 op/s 2024-09-11T01:18:48.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:47 smithi110 ceph-mon[34712]: pgmap v946: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 838 op/s 2024-09-11T01:18:48.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:47 smithi112 ceph-mon[35384]: pgmap v946: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 838 op/s 2024-09-11T01:18:48.794 INFO:tasks.workunit.client.0.smithi078.stdout: 10 268609 34.56 MB/sec execute 294 sec latency 244.665 ms 2024-09-11T01:18:49.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 269111 34.53 MB/sec execute 295 sec latency 142.098 ms 2024-09-11T01:18:49.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:49 smithi078 ceph-mon[31315]: pgmap v947: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 22 MiB/s wr, 869 op/s 2024-09-11T01:18:50.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:49 smithi110 ceph-mon[34712]: pgmap v947: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 22 MiB/s wr, 869 op/s 2024-09-11T01:18:50.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:49 smithi112 ceph-mon[35384]: pgmap v947: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 22 MiB/s wr, 869 op/s 2024-09-11T01:18:50.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 269805 34.51 MB/sec execute 296 sec latency 121.058 ms 2024-09-11T01:18:51.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 270605 34.56 MB/sec execute 297 sec latency 73.451 ms 2024-09-11T01:18:51.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:51 smithi078 ceph-mon[31315]: pgmap v948: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 23 MiB/s wr, 951 op/s 2024-09-11T01:18:52.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:51 smithi110 ceph-mon[34712]: pgmap v948: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 23 MiB/s wr, 951 op/s 2024-09-11T01:18:52.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:51 smithi112 ceph-mon[35384]: pgmap v948: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 23 MiB/s wr, 951 op/s 2024-09-11T01:18:52.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 271256 34.57 MB/sec execute 298 sec latency 70.204 ms 2024-09-11T01:18:53.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 271932 34.56 MB/sec execute 299 sec latency 128.958 ms 2024-09-11T01:18:53.874 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:18:53.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:53 smithi078 ceph-mon[31315]: pgmap v949: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 19 MiB/s wr, 836 op/s 2024-09-11T01:18:54.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:53 smithi110 ceph-mon[34712]: pgmap v949: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 19 MiB/s wr, 836 op/s 2024-09-11T01:18:54.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:53 smithi112 ceph-mon[35384]: pgmap v949: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 19 MiB/s wr, 836 op/s 2024-09-11T01:18:54.291 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:18:54.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 272605 34.58 MB/sec execute 300 sec latency 500.147 ms 2024-09-11T01:18:55.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 273181 34.56 MB/sec execute 301 sec latency 109.170 ms 2024-09-11T01:18:55.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:55 smithi078 ceph-mon[31315]: pgmap v950: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 730 op/s 2024-09-11T01:18:56.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:55 smithi110 ceph-mon[34712]: pgmap v950: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 730 op/s 2024-09-11T01:18:56.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:55 smithi112 ceph-mon[35384]: pgmap v950: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 730 op/s 2024-09-11T01:18:56.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 273863 34.57 MB/sec execute 302 sec latency 90.020 ms 2024-09-11T01:18:56.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.847+0000 7f27aaf1d640 1 -- 172.21.15.78:0/3217977662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a410d800 msgr2=0x7f27a410dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:56.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.847+0000 7f27aaf1d640 1 --2- 172.21.15.78:0/3217977662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a410d800 0x7f27a410dbe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2798009900 tx=0x7f279802a3f0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.847+0000 7f27aaf1d640 1 -- 172.21.15.78:0/3217977662 shutdown_connections 2024-09-11T01:18:56.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.847+0000 7f27aaf1d640 1 --2- 172.21.15.78:0/3217977662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4113b40 0x7f27a411a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.847+0000 7f27aaf1d640 1 --2- 172.21.15.78:0/3217977662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27a41131c0 0x7f27a4113600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.847+0000 7f27aaf1d640 1 --2- 172.21.15.78:0/3217977662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a410d800 0x7f27a410dbe0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.847+0000 7f27aaf1d640 1 -- 172.21.15.78:0/3217977662 >> 172.21.15.78:0/3217977662 conn(0x7f27a406dce0 msgr2=0x7f27a406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:56.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.847+0000 7f27aaf1d640 1 -- 172.21.15.78:0/3217977662 shutdown_connections 2024-09-11T01:18:56.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.847+0000 7f27aaf1d640 1 -- 172.21.15.78:0/3217977662 wait complete. 2024-09-11T01:18:56.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.847+0000 7f27aaf1d640 1 Processor -- start 2024-09-11T01:18:56.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.848+0000 7f27aaf1d640 1 -- start start 2024-09-11T01:18:56.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.849+0000 7f27aaf1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a410d800 0x7f27a41a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.849+0000 7f27aaf1d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27a41131c0 0x7f27a41a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.849+0000 7f27aaf1d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4113b40 0x7f27a41a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.849+0000 7f27aaf1d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27a411d790 con 0x7f27a4113b40 2024-09-11T01:18:56.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.849+0000 7f27aaf1d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27a411d610 con 0x7f27a41131c0 2024-09-11T01:18:56.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.849+0000 7f27aaf1d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27a411d910 con 0x7f27a410d800 2024-09-11T01:18:56.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.849+0000 7f27a3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27a41131c0 0x7f27a41a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.849+0000 7f27a3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27a41131c0 0x7f27a41a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:33022/0 (socket says 172.21.15.78:33022) 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.849+0000 7f27a3fff640 1 -- 172.21.15.78:0/1766718656 learned_addr learned my addr 172.21.15.78:0/1766718656 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.850+0000 7f27a94ba640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4113b40 0x7f27a41a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.849+0000 7f27a8cb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a410d800 0x7f27a41a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.850+0000 7f27a3fff640 1 -- 172.21.15.78:0/1766718656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a410d800 msgr2=0x7f27a41a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.850+0000 7f27a3fff640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a410d800 0x7f27a41a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.850+0000 7f27a3fff640 1 -- 172.21.15.78:0/1766718656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4113b40 msgr2=0x7f27a41a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.850+0000 7f27a3fff640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4113b40 0x7f27a41a95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.850+0000 7f27a3fff640 1 -- 172.21.15.78:0/1766718656 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a41a9db0 con 0x7f27a41131c0 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.851+0000 7f27a3fff640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27a41131c0 0x7f27a41a4240 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f279000d950 tx=0x7f279000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.851+0000 7f27a1ffb640 1 -- 172.21.15.78:0/1766718656 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790017070 con 0x7f27a41131c0 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.851+0000 7f27a1ffb640 1 -- 172.21.15.78:0/1766718656 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2790009c30 con 0x7f27a41131c0 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.851+0000 7f27a1ffb640 1 -- 172.21.15.78:0/1766718656 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279000f620 con 0x7f27a41131c0 2024-09-11T01:18:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.851+0000 7f27aaf1d640 1 -- 172.21.15.78:0/1766718656 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a41aa0a0 con 0x7f27a41131c0 2024-09-11T01:18:56.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.851+0000 7f27aaf1d640 1 -- 172.21.15.78:0/1766718656 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a41b0760 con 0x7f27a41131c0 2024-09-11T01:18:56.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.855+0000 7f27ad2f2740 1 Processor -- start 2024-09-11T01:18:56.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.856+0000 7f27a1ffb640 1 -- 172.21.15.78:0/1766718656 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2790023060 con 0x7f27a41131c0 2024-09-11T01:18:56.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.857+0000 7f27ad2f2740 1 -- start start 2024-09-11T01:18:56.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.857+0000 7f27ad2f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e783232070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.857+0000 7f27ad2f2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 0x55e78323e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.857+0000 7f27ad2f2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e78323e540 0x55e783240930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.857+0000 7f27ad2f2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e78319fe50 con 0x55e78323e540 2024-09-11T01:18:56.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.857+0000 7f27ad2f2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e78318f280 con 0x55e7832325b0 2024-09-11T01:18:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.857+0000 7f27ad2f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e783193090 con 0x55e783231c90 2024-09-11T01:18:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.857+0000 7f27a94ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e783232070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.857+0000 7f27a94ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e783232070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42458/0 (socket says 172.21.15.78:42458) 2024-09-11T01:18:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.857+0000 7f27a94ba640 1 -- 172.21.15.78:0/3768171716 learned_addr learned my addr 172.21.15.78:0/3768171716 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.857+0000 7f27a1ffb640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27940b55a0 0x7f27940b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.858+0000 7f27a1ffb640 1 -- 172.21.15.78:0/1766718656 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f27900e5430 con 0x7f27a41131c0 2024-09-11T01:18:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.858+0000 7f27a94ba640 1 -- 172.21.15.78:0/3768171716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 msgr2=0x55e78323e000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:18:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.858+0000 7f27a94ba640 1 --2- 172.21.15.78:0/3768171716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 0x55e78323e000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.858+0000 7f27a94ba640 1 -- 172.21.15.78:0/3768171716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e78323e540 msgr2=0x55e783240930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:18:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.858+0000 7f27a94ba640 1 --2- 172.21.15.78:0/3768171716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e78323e540 0x55e783240930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.858+0000 7f27a94ba640 1 -- 172.21.15.78:0/3768171716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e782eed650 con 0x55e783231c90 2024-09-11T01:18:56.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.858+0000 7f27a94ba640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27940b55a0 0x7f27940b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.858+0000 7f27a94ba640 1 --2- 172.21.15.78:0/3768171716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e783232070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f279c005c40 tx=0x7f279c0139f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21ad7d1f832def73 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:56.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.859+0000 7f278effd640 1 -- 172.21.15.78:0/3768171716 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c00faa0 con 0x55e783231c90 2024-09-11T01:18:56.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.859+0000 7f278effd640 1 -- 172.21.15.78:0/3768171716 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f279c00fc40 con 0x55e783231c90 2024-09-11T01:18:56.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.860+0000 7f27ad2f2740 1 -- 172.21.15.78:0/3768171716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 msgr2=0x55e783232070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:56.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.860+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/3768171716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e783232070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f279c005c40 tx=0x7f279c0139f0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.860+0000 7f27a94ba640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27940b55a0 0x7f27940b7a60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f279c023000 tx=0x7f279c01beb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:56.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.861+0000 7f27ad2f2740 1 -- 172.21.15.78:0/3768171716 shutdown_connections 2024-09-11T01:18:56.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.861+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/3768171716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e78323e540 0x55e783240930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.861+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/3768171716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 0x55e78323e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.861+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/3768171716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e783232070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.861+0000 7f27ad2f2740 1 -- 172.21.15.78:0/3768171716 >> 172.21.15.78:0/3768171716 conn(0x55e78322fac0 msgr2=0x55e783230c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.861+0000 7f27ad2f2740 1 -- 172.21.15.78:0/3768171716 shutdown_connections 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.861+0000 7f27ad2f2740 1 -- 172.21.15.78:0/3768171716 wait complete. 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.861+0000 7f27ad2f2740 1 -- start start 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.862+0000 7f27ad2f2740 1 Processor -- start 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.862+0000 7f27ad2f2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e783231c90 0x55e78322c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.862+0000 7f27ad2f2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 0x55e78322cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.862+0000 7f27ad2f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e78323e540 0x55e78322d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.862+0000 7f27ad2f2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e782ff0700 con 0x55e783231c90 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.862+0000 7f27ad2f2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e78312a110 con 0x55e7832325b0 2024-09-11T01:18:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.862+0000 7f27ad2f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e783226fd0 con 0x55e78323e540 2024-09-11T01:18:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.862+0000 7f27ad2f2740 10 client.0 _ll_register_callbacks cb 0x55e783143990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.863+0000 7f27a94ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e78323e540 0x55e78322d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.863+0000 7f27a94ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e78323e540 0x55e78322d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42468/0 (socket says 172.21.15.78:42468) 2024-09-11T01:18:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.863+0000 7f27a94ba640 1 -- 172.21.15.78:0/1081971780 learned_addr learned my addr 172.21.15.78:0/1081971780 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.863+0000 7f27a3fff640 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 0x55e78322cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.863+0000 7f27a8cb9640 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e783231c90 0x55e78322c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.864+0000 7f27a94ba640 1 -- 172.21.15.78:0/1081971780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 msgr2=0x55e78322cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.864+0000 7f27a94ba640 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 0x55e78322cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.864+0000 7f27a94ba640 1 -- 172.21.15.78:0/1081971780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e783231c90 msgr2=0x55e78322c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.864+0000 7f27a94ba640 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e783231c90 0x55e78322c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.864+0000 7f27a94ba640 1 -- 172.21.15.78:0/1081971780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e783157c40 con 0x55e78323e540 2024-09-11T01:18:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.864+0000 7f27a3fff640 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 0x55e78322cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:18:56.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.864+0000 7f27a94ba640 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e78323e540 0x55e78322d130 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f279c0133a0 tx=0x7f279c018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=103a2f74771aa779 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:56.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.864+0000 7f2783fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:18:56.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.867+0000 7f2783fff640 1 -- 172.21.15.78:0/1081971780 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c036060 con 0x55e78323e540 2024-09-11T01:18:56.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.867+0000 7f2783fff640 1 -- 172.21.15.78:0/1081971780 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f279c00ebc0 con 0x55e78323e540 2024-09-11T01:18:56.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.867+0000 7f27ad2f2740 1 -- 172.21.15.78:0/1081971780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f279c00fc40 con 0x55e78323e540 2024-09-11T01:18:56.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.869+0000 7f2783fff640 1 -- 172.21.15.78:0/1081971780 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f279c02d040 con 0x55e78323e540 2024-09-11T01:18:56.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.869+0000 7f27ad2f2740 10 client.27946 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:56.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.869+0000 7f27ad2f2740 1 -- 172.21.15.78:0/1081971780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55e7832554e0 con 0x55e78323e540 2024-09-11T01:18:56.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.870+0000 7f2783fff640 1 -- 172.21.15.78:0/1081971780 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f279c00fc40 con 0x55e78323e540 2024-09-11T01:18:56.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.870+0000 7f2783fff640 1 -- 172.21.15.78:0/1081971780 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f279c030070 con 0x55e78323e540 2024-09-11T01:18:56.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.870+0000 7f27ad2f2740 10 client.27946 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:56.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.870+0000 7f27ad2f2740 10 client.27946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:18:56.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.870+0000 7f27ad2f2740 20 client.27946 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:56.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.870+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e783256830 0x55e783258c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.870+0000 7f27ad2f2740 4 client.27946 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:18:56.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.870+0000 7f27ad2f2740 1 -- 172.21.15.78:0/1081971780 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7831a1da0 con 0x55e783256830 2024-09-11T01:18:56.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.872+0000 7f27a8cb9640 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e783256830 0x55e783258c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.872+0000 7f27a8cb9640 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e783256830 0x55e783258c20 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad123dcadce77c66 server_cookie=e2e9a390a63ff429 in_seq=0 out_seq=0 2024-09-11T01:18:56.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.873+0000 7f2783fff640 10 client.27946 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:56.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.873+0000 7f2783fff640 1 -- 172.21.15.78:0/1081971780 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e7831a1da0 con 0x55e783256830 2024-09-11T01:18:56.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.873+0000 7f2783fff640 10 client.27946 handle_command_reply: tid=0 2024-09-11T01:18:56.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.873+0000 7f27ad2f2740 1 client.27946 shutdown 2024-09-11T01:18:56.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.874+0000 7f27ad2f2740 1 -- 172.21.15.78:0/1081971780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e78323e540 msgr2=0x55e78322d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:56.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.874+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e78323e540 0x55e78322d130 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f279c0133a0 tx=0x7f279c018000 comp rx=0 tx=0).stop 2024-09-11T01:18:56.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.874+0000 7f27ad2f2740 1 -- 172.21.15.78:0/1081971780 shutdown_connections 2024-09-11T01:18:56.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.874+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e78323e540 0x55e78322d130 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.874+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 0x55e78322cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.874+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e783256830 0x55e783258c20 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.874+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/1081971780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e783231c90 0x55e78322c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.874+0000 7f2783fff640 0 client.27946 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:56.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.874+0000 7f27ad2f2740 1 -- 172.21.15.78:0/1081971780 >> 172.21.15.78:0/1081971780 conn(0x55e78322fac0 msgr2=0x55e783235660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:56.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.876+0000 7f27ad2f2740 1 -- 172.21.15.78:0/1081971780 shutdown_connections 2024-09-11T01:18:56.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.876+0000 7f27ad2f2740 1 -- 172.21.15.78:0/1081971780 wait complete. 2024-09-11T01:18:56.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.877+0000 7f27ad2f2740 20 client.27946 trim_cache size 0 max 16384 2024-09-11T01:18:56.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.918+0000 7f27ad2f2740 1 Processor -- start 2024-09-11T01:18:56.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.918+0000 7f27ad2f2740 1 -- start start 2024-09-11T01:18:56.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.919+0000 7f27ad2f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e783229900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.919+0000 7f27ad2f2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7832325b0 0x55e783229e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.919+0000 7f27ad2f2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e78323e540 0x55e78323f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.919+0000 7f27ad2f2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27a41b4bf0 con 0x55e7832325b0 2024-09-11T01:18:56.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.919+0000 7f27ad2f2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27a41b6730 con 0x55e78323e540 2024-09-11T01:18:56.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.919+0000 7f27ad2f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27a41b52c0 con 0x55e783231c90 2024-09-11T01:18:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.919+0000 7f27a3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7832325b0 0x55e783229e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.919+0000 7f27a3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7832325b0 0x55e783229e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44494/0 (socket says 172.21.15.78:44494) 2024-09-11T01:18:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.920+0000 7f27a3fff640 1 -- 172.21.15.78:0/2711312450 learned_addr learned my addr 172.21.15.78:0/2711312450 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.920+0000 7f27a94ba640 1 --2- 172.21.15.78:0/2711312450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e78323e540 0x55e78323f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.920+0000 7f27a8cb9640 1 --2- 172.21.15.78:0/2711312450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e783229900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.920+0000 7f27a3fff640 1 -- 172.21.15.78:0/2711312450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 msgr2=0x55e783229900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.920+0000 7f27a3fff640 1 --2- 172.21.15.78:0/2711312450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e783229900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.920+0000 7f27a3fff640 1 -- 172.21.15.78:0/2711312450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e78323e540 msgr2=0x55e78323f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.920+0000 7f27a3fff640 1 --2- 172.21.15.78:0/2711312450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e78323e540 0x55e78323f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.920+0000 7f27a3fff640 1 -- 172.21.15.78:0/2711312450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7832591b0 con 0x55e7832325b0 2024-09-11T01:18:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.920+0000 7f27a3fff640 1 --2- 172.21.15.78:0/2711312450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7832325b0 0x55e783229e40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f279000e010 tx=0x7f27900ab0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe737614a2dab88 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.922+0000 7f278e7fc640 1 -- 172.21.15.78:0/2711312450 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790096070 con 0x55e7832325b0 2024-09-11T01:18:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.922+0000 7f278e7fc640 1 -- 172.21.15.78:0/2711312450 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27900ab450 con 0x55e7832325b0 2024-09-11T01:18:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.922+0000 7f278e7fc640 1 -- 172.21.15.78:0/2711312450 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790020040 con 0x55e7832325b0 2024-09-11T01:18:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.923+0000 7f27ad2f2740 1 -- 172.21.15.78:0/2711312450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7832325b0 msgr2=0x55e783229e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.923+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/2711312450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7832325b0 0x55e783229e40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f279000e010 tx=0x7f27900ab0b0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.924+0000 7f27ad2f2740 1 -- 172.21.15.78:0/2711312450 shutdown_connections 2024-09-11T01:18:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.924+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/2711312450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e78323e540 0x55e78323f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.924+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/2711312450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7832325b0 0x55e783229e40 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.924+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/2711312450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e783229900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.924+0000 7f27ad2f2740 1 -- 172.21.15.78:0/2711312450 >> 172.21.15.78:0/2711312450 conn(0x55e78322fac0 msgr2=0x55e78322e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.924+0000 7f27ad2f2740 1 -- 172.21.15.78:0/2711312450 shutdown_connections 2024-09-11T01:18:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.924+0000 7f27ad2f2740 1 -- 172.21.15.78:0/2711312450 wait complete. 2024-09-11T01:18:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.925+0000 7f27ad2f2740 1 -- start start 2024-09-11T01:18:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.925+0000 7f27ad2f2740 1 Processor -- start 2024-09-11T01:18:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.925+0000 7f27ad2f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e78323c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.925+0000 7f27ad2f2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 0x55e78323c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.926+0000 7f27ad2f2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e78323e540 0x55e783229150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.926+0000 7f27ad2f2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27a41b6060 con 0x55e78323e540 2024-09-11T01:18:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.926+0000 7f27ad2f2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27a41b5990 con 0x55e7832325b0 2024-09-11T01:18:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.926+0000 7f27ad2f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27a41b2bf0 con 0x55e783231c90 2024-09-11T01:18:56.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.926+0000 7f27ad2f2740 10 client.0 _ll_register_callbacks cb 0x7f279c013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:18:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.926+0000 7f27a94ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e78323e540 0x55e783229150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.926+0000 7f27a94ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e78323e540 0x55e783229150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44502/0 (socket says 172.21.15.78:44502) 2024-09-11T01:18:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.926+0000 7f27a94ba640 1 -- 172.21.15.78:0/566870752 learned_addr learned my addr 172.21.15.78:0/566870752 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:18:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.927+0000 7f27a8cb9640 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e78323c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.927+0000 7f27a3fff640 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 0x55e78323c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.927+0000 7f27a8cb9640 1 -- 172.21.15.78:0/566870752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 msgr2=0x55e78323c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.927+0000 7f27a8cb9640 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 0x55e78323c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.927+0000 7f27a8cb9640 1 -- 172.21.15.78:0/566870752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e78323e540 msgr2=0x55e783229150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:18:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.927+0000 7f27a8cb9640 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e78323e540 0x55e783229150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.927+0000 7f27a8cb9640 1 -- 172.21.15.78:0/566870752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e783229980 con 0x55e783231c90 2024-09-11T01:18:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.928+0000 7f27a8cb9640 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e78323c270 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f279802a9e0 tx=0x7f2798004520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bba1d1e48b316734 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:18:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.928+0000 7f2783fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:18:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.930+0000 7f2783fff640 1 -- 172.21.15.78:0/566870752 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798035040 con 0x55e783231c90 2024-09-11T01:18:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.930+0000 7f27ad2f2740 1 -- 172.21.15.78:0/566870752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27900ab450 con 0x55e783231c90 2024-09-11T01:18:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.930+0000 7f2783fff640 1 -- 172.21.15.78:0/566870752 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f2798043030 con 0x55e783231c90 2024-09-11T01:18:56.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.931+0000 7f2783fff640 1 -- 172.21.15.78:0/566870752 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f2798053050 con 0x55e783231c90 2024-09-11T01:18:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.932+0000 7f27ad2f2740 10 client.27949 fetch_fsmap learned FSMap version 70 2024-09-11T01:18:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.932+0000 7f27ad2f2740 1 -- 172.21.15.78:0/566870752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55e7832c4130 con 0x55e783231c90 2024-09-11T01:18:56.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.935+0000 7f2783fff640 1 -- 172.21.15.78:0/566870752 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27900ab450 con 0x55e783231c90 2024-09-11T01:18:56.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.935+0000 7f2783fff640 1 -- 172.21.15.78:0/566870752 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f27980035f0 con 0x55e783231c90 2024-09-11T01:18:56.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.939+0000 7f27ad2f2740 10 client.27949 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:18:56.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.939+0000 7f27ad2f2740 10 client.27949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:18:56.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.939+0000 7f27ad2f2740 20 client.27949 populate_metadata read hostname 'smithi078' 2024-09-11T01:18:56.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.939+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e7832c4850 0x55e7832c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:18:56.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.939+0000 7f27ad2f2740 4 client.27949 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:18:56.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.939+0000 7f27ad2f2740 1 -- 172.21.15.78:0/566870752 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e78319d850 con 0x55e7832c4850 2024-09-11T01:18:56.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.943+0000 7f27a3fff640 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e7832c4850 0x55e7832c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:18:56.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.946+0000 7f27a3fff640 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e7832c4850 0x55e7832c6c70 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3be6bce3635bc501 server_cookie=2da349493fb43c36 in_seq=0 out_seq=0 2024-09-11T01:18:56.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.947+0000 7f2783fff640 10 client.27949 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:56.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.951+0000 7f2783fff640 1 -- 172.21.15.78:0/566870752 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6906 (crc 0 0 0) 0x55e78319d850 con 0x55e7832c4850 2024-09-11T01:18:56.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.951+0000 7f2783fff640 10 client.27949 handle_command_reply: tid=0 2024-09-11T01:18:56.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.951+0000 7f27ad2f2740 1 client.27949 shutdown 2024-09-11T01:18:56.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.955+0000 7f27ad2f2740 1 -- 172.21.15.78:0/566870752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 msgr2=0x55e78323c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:56.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.955+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e78323c270 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f279802a9e0 tx=0x7f2798004520 comp rx=0 tx=0).stop 2024-09-11T01:18:56.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.955+0000 7f27ad2f2740 1 -- 172.21.15.78:0/566870752 shutdown_connections 2024-09-11T01:18:56.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.955+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e7832c4850 0x55e7832c6c70 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.955+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e78323e540 0x55e783229150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.955+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e7832325b0 0x55e78323c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.955+0000 7f27ad2f2740 1 --2- 172.21.15.78:0/566870752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e783231c90 0x55e78323c270 secure :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f279802a9e0 tx=0x7f2798004520 comp rx=0 tx=0).stop 2024-09-11T01:18:56.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.955+0000 7f27ad2f2740 1 -- 172.21.15.78:0/566870752 >> 172.21.15.78:0/566870752 conn(0x55e78322fac0 msgr2=0x55e78322c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:56.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.955+0000 7f2783fff640 0 client.27949 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:18:56.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.955+0000 7f27ad2f2740 1 -- 172.21.15.78:0/566870752 shutdown_connections 2024-09-11T01:18:56.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.955+0000 7f27ad2f2740 1 -- 172.21.15.78:0/566870752 wait complete. 2024-09-11T01:18:56.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.955+0000 7f27ad2f2740 20 client.27949 trim_cache size 0 max 16384 2024-09-11T01:18:56.958 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:18:56.959 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:18:56.959 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:18:56.959 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:18:56.959 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:18:56.959 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:18:17.654570+0000", 2024-09-11T01:18:56.959 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.960 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:18:56.960 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:18:56.960 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.960 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:18:56.960 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:18:56.960 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.961 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:18:56.961 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:18:56.961 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.962 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:18:56.962 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:18:56.962 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.964 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:18:56.964 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:18:56.964 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.964 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:18:56.964 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:18:56.965 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.965 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:18:56.965 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:18:56.965 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.966 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:18:56.966 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:18:56.966 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.967 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:18:56.967 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:18:56.967 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.967 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:18:56.967 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:18:56.967 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.968 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:18:56.968 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:18:56.968 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.968 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.969 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.969 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:18:56.969 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:18:56.969 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.969 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.969 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.969 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.969 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.969 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:18:56.969 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:18:56.970 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.970 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.970 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.970 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.970 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.970 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:18:56.970 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:18:56.970 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.970 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.970 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.970 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.971 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.971 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:18:56.971 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:18:56.971 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.971 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.971 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.971 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.971 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.971 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:18:56.973 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:18:56.974 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:18:56.975 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:18:56.975 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:18:56.975 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:18:56.975 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:18:56.975 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:18:56.975 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:18:56.975 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:18:56.975 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:18:56.975 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:18:56.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.959+0000 7f27aaf1d640 1 -- 172.21.15.78:0/1766718656 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27940b55a0 msgr2=0x7f27940b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:56.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.959+0000 7f27aaf1d640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27940b55a0 0x7f27940b7a60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f279c023000 tx=0x7f279c01beb0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.959+0000 7f27aaf1d640 1 -- 172.21.15.78:0/1766718656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27a41131c0 msgr2=0x7f27a41a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:18:56.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.959+0000 7f27aaf1d640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27a41131c0 0x7f27a41a4240 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f279000d950 tx=0x7f279000de10 comp rx=0 tx=0).stop 2024-09-11T01:18:56.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.961+0000 7f27aaf1d640 1 -- 172.21.15.78:0/1766718656 shutdown_connections 2024-09-11T01:18:56.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.961+0000 7f27aaf1d640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27940b55a0 0x7f27940b7a60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.961+0000 7f27aaf1d640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4113b40 0x7f27a41a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.961+0000 7f27aaf1d640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27a41131c0 0x7f27a41a4240 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.961+0000 7f27aaf1d640 1 --2- 172.21.15.78:0/1766718656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a410d800 0x7f27a41a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:18:56.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.961+0000 7f27aaf1d640 1 -- 172.21.15.78:0/1766718656 >> 172.21.15.78:0/1766718656 conn(0x7f27a406dce0 msgr2=0x7f27a410b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:18:56.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.972+0000 7f27aaf1d640 1 -- 172.21.15.78:0/1766718656 shutdown_connections 2024-09-11T01:18:56.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:56.972+0000 7f27aaf1d640 1 -- 172.21.15.78:0/1766718656 wait complete. 2024-09-11T01:18:57.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 274041 34.49 MB/sec execute 303 sec latency 644.946 ms 2024-09-11T01:18:57.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:57 smithi078 ceph-mon[31315]: pgmap v951: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 667 op/s 2024-09-11T01:18:58.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:57 smithi110 ceph-mon[34712]: pgmap v951: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 667 op/s 2024-09-11T01:18:58.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:57 smithi112 ceph-mon[35384]: pgmap v951: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 667 op/s 2024-09-11T01:18:58.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 274582 34.48 MB/sec execute 304 sec latency 114.780 ms 2024-09-11T01:18:58.896 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:18:58.896 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:18:59.334 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:18:59.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 275183 34.45 MB/sec execute 305 sec latency 89.607 ms 2024-09-11T01:19:00.000 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:18:59 smithi078 ceph-mon[31315]: pgmap v952: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 623 op/s 2024-09-11T01:19:00.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:18:59 smithi110 ceph-mon[34712]: pgmap v952: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 623 op/s 2024-09-11T01:19:00.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:18:59 smithi112 ceph-mon[35384]: pgmap v952: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 623 op/s 2024-09-11T01:19:00.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:59.997+0000 7f945ef5f640 1 -- 172.21.15.78:0/1938495593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f945810d100 msgr2=0x7f945810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:59.997+0000 7f945ef5f640 1 --2- 172.21.15.78:0/1938495593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f945810d100 0x7f945810f560 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f944c009900 tx=0x7f944c02a440 comp rx=0 tx=0).stop 2024-09-11T01:19:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:59.997+0000 7f945ef5f640 1 -- 172.21.15.78:0/1938495593 shutdown_connections 2024-09-11T01:19:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:59.997+0000 7f945ef5f640 1 --2- 172.21.15.78:0/1938495593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f945810d100 0x7f945810f560 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:59.997+0000 7f945ef5f640 1 --2- 172.21.15.78:0/1938495593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94581045c0 0x7f945810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:59.997+0000 7f945ef5f640 1 --2- 172.21.15.78:0/1938495593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9458103ca0 0x7f9458104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:18:59.997+0000 7f945ef5f640 1 -- 172.21.15.78:0/1938495593 >> 172.21.15.78:0/1938495593 conn(0x7f94580fd700 msgr2=0x7f94580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.002+0000 7f945ef5f640 1 -- 172.21.15.78:0/1938495593 shutdown_connections 2024-09-11T01:19:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.005+0000 7f945ef5f640 1 -- 172.21.15.78:0/1938495593 wait complete. 2024-09-11T01:19:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.006+0000 7f945ef5f640 1 Processor -- start 2024-09-11T01:19:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.009+0000 7f945ef5f640 1 -- start start 2024-09-11T01:19:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.009+0000 7f945ef5f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9458103ca0 0x7f945810bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.009+0000 7f945ef5f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94581045c0 0x7f945810c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.009+0000 7f945ef5f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f945810d100 0x7f945819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.009+0000 7f945ef5f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9458112ec0 con 0x7f9458103ca0 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.009+0000 7f945ef5f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9458112d40 con 0x7f945810d100 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.009+0000 7f945ef5f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9458113040 con 0x7f94581045c0 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.010+0000 7f945ccfb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9458103ca0 0x7f945810bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.011+0000 7f945ccfb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9458103ca0 0x7f945810bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44534/0 (socket says 172.21.15.78:44534) 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.011+0000 7f945ccfb640 1 -- 172.21.15.78:0/3460039311 learned_addr learned my addr 172.21.15.78:0/3460039311 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.013+0000 7f945ccfb640 1 -- 172.21.15.78:0/3460039311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94581045c0 msgr2=0x7f945810c400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.013+0000 7f945ccfb640 1 --2- 172.21.15.78:0/3460039311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94581045c0 0x7f945810c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.013+0000 7f945ccfb640 1 -- 172.21.15.78:0/3460039311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f945810d100 msgr2=0x7f945819fcc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.013+0000 7f945ccfb640 1 --2- 172.21.15.78:0/3460039311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f945810d100 0x7f945819fcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.013+0000 7f945ccfb640 1 -- 172.21.15.78:0/3460039311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94581a0440 con 0x7f9458103ca0 2024-09-11T01:19:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.015+0000 7f945ccfb640 1 --2- 172.21.15.78:0/3460039311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9458103ca0 0x7f945810bec0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f944800b450 tx=0x7f944800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:00.569 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:19:00.569 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":70,"btime":"2024-09-11T01:15:50:008465+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":70,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:15:49.712348+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:19:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.018+0000 7f9455ffb640 1 -- 172.21.15.78:0/3460039311 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448015020 con 0x7f9458103ca0 2024-09-11T01:19:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.018+0000 7f9455ffb640 1 -- 172.21.15.78:0/3460039311 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9448011d40 con 0x7f9458103ca0 2024-09-11T01:19:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.018+0000 7f9455ffb640 1 -- 172.21.15.78:0/3460039311 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944800f340 con 0x7f9458103ca0 2024-09-11T01:19:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.019+0000 7f945ef5f640 1 -- 172.21.15.78:0/3460039311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94581a0730 con 0x7f9458103ca0 2024-09-11T01:19:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.020+0000 7f945ef5f640 1 -- 172.21.15.78:0/3460039311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945819cd90 con 0x7f9458103ca0 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.021+0000 7f945ef5f640 1 -- 172.21.15.78:0/3460039311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9444005290 con 0x7f9458103ca0 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.024+0000 7f9455ffb640 1 -- 172.21.15.78:0/3460039311 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9448011890 con 0x7f9458103ca0 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.024+0000 7f9455ffb640 1 --2- 172.21.15.78:0/3460039311 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94400b5550 0x7f94400b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.025+0000 7f9455ffb640 1 -- 172.21.15.78:0/3460039311 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f94480e5e90 con 0x7f9458103ca0 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.025+0000 7f9457fff640 1 --2- 172.21.15.78:0/3460039311 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94400b5550 0x7f94400b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.025+0000 7f9457fff640 1 --2- 172.21.15.78:0/3460039311 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94400b5550 0x7f94400b7a10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f945819dbd0 tx=0x7f9450009230 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.028+0000 7f9455ffb640 1 -- 172.21.15.78:0/3460039311 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f944801f020 con 0x7f9458103ca0 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.307+0000 7f945ef5f640 1 -- 172.21.15.78:0/3460039311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9444005580 con 0x7f9458103ca0 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.310+0000 7f9455ffb640 1 -- 172.21.15.78:0/3460039311 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+11979 (secure 0 0 0) 0x7f944800d040 con 0x7f9458103ca0 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 70 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.317+0000 7f945ef5f640 1 -- 172.21.15.78:0/3460039311 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94400b5550 msgr2=0x7f94400b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.317+0000 7f945ef5f640 1 --2- 172.21.15.78:0/3460039311 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94400b5550 0x7f94400b7a10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f945819dbd0 tx=0x7f9450009230 comp rx=0 tx=0).stop 2024-09-11T01:19:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.317+0000 7f945ef5f640 1 -- 172.21.15.78:0/3460039311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9458103ca0 msgr2=0x7f945810bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.317+0000 7f945ef5f640 1 --2- 172.21.15.78:0/3460039311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9458103ca0 0x7f945810bec0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f944800b450 tx=0x7f944800b910 comp rx=0 tx=0).stop 2024-09-11T01:19:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.317+0000 7f945ef5f640 1 -- 172.21.15.78:0/3460039311 shutdown_connections 2024-09-11T01:19:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.317+0000 7f945ef5f640 1 --2- 172.21.15.78:0/3460039311 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94400b5550 0x7f94400b7a10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.317+0000 7f945ef5f640 1 --2- 172.21.15.78:0/3460039311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f945810d100 0x7f945819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.317+0000 7f945ef5f640 1 --2- 172.21.15.78:0/3460039311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94581045c0 0x7f945810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.317+0000 7f945ef5f640 1 --2- 172.21.15.78:0/3460039311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9458103ca0 0x7f945810bec0 secure :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f944800b450 tx=0x7f944800b910 comp rx=0 tx=0).stop 2024-09-11T01:19:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.317+0000 7f945ef5f640 1 -- 172.21.15.78:0/3460039311 >> 172.21.15.78:0/3460039311 conn(0x7f94580fd700 msgr2=0x7f94580ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.318+0000 7f945ef5f640 1 -- 172.21.15.78:0/3460039311 shutdown_connections 2024-09-11T01:19:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:00.318+0000 7f945ef5f640 1 -- 172.21.15.78:0/3460039311 wait complete. 2024-09-11T01:19:00.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 275357 34.37 MB/sec execute 306 sec latency 890.267 ms 2024-09-11T01:19:00.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:00 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3460039311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:19:01.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:00 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3460039311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:19:01.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:00 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3460039311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:19:01.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 275357 34.26 MB/sec execute 307 sec latency 1890.316 ms 2024-09-11T01:19:02.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:01 smithi110 ceph-mon[34712]: pgmap v953: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 707 op/s 2024-09-11T01:19:02.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:01 smithi112 ceph-mon[35384]: pgmap v953: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 707 op/s 2024-09-11T01:19:02.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:01 smithi078 ceph-mon[31315]: pgmap v953: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 707 op/s 2024-09-11T01:19:02.796 INFO:tasks.workunit.client.0.smithi078.stdout: 10 275357 34.15 MB/sec execute 308 sec latency 2890.406 ms 2024-09-11T01:19:02.974 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:19:03.239 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:19:03.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.749+0000 7f27df145640 1 -- 172.21.15.78:0/2689747971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d810d1a0 msgr2=0x7f27d810f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.749+0000 7f27df145640 1 --2- 172.21.15.78:0/2689747971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d810d1a0 0x7f27d810f600 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009900 tx=0x7f27d002a410 comp rx=0 tx=0).stop 2024-09-11T01:19:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.749+0000 7f27df145640 1 -- 172.21.15.78:0/2689747971 shutdown_connections 2024-09-11T01:19:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.749+0000 7f27df145640 1 --2- 172.21.15.78:0/2689747971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d810d1a0 0x7f27d810f600 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.749+0000 7f27df145640 1 --2- 172.21.15.78:0/2689747971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d8104660 0x7f27d810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.749+0000 7f27df145640 1 --2- 172.21.15.78:0/2689747971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d8103d40 0x7f27d8104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.749+0000 7f27df145640 1 -- 172.21.15.78:0/2689747971 >> 172.21.15.78:0/2689747971 conn(0x7f27d80fd7e0 msgr2=0x7f27d80ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.749+0000 7f27df145640 1 -- 172.21.15.78:0/2689747971 shutdown_connections 2024-09-11T01:19:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.749+0000 7f27df145640 1 -- 172.21.15.78:0/2689747971 wait complete. 2024-09-11T01:19:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.749+0000 7f27df145640 1 Processor -- start 2024-09-11T01:19:03.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27df145640 1 -- start start 2024-09-11T01:19:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27df145640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d8103d40 0x7f27d8199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27df145640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d8104660 0x7f27d8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27df145640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d810d1a0 0x7f27d819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27df145640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27d8112e10 con 0x7f27d8104660 2024-09-11T01:19:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27df145640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27d8112c90 con 0x7f27d810d1a0 2024-09-11T01:19:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27df145640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27d8112f90 con 0x7f27d8103d40 2024-09-11T01:19:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d8104660 0x7f27d8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d8104660 0x7f27d8199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56926/0 (socket says 172.21.15.78:56926) 2024-09-11T01:19:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27d7fff640 1 -- 172.21.15.78:0/1540304037 learned_addr learned my addr 172.21.15.78:0/1540304037 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27dcee1640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d8103d40 0x7f27d8199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27dd6e2640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d810d1a0 0x7f27d819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27d7fff640 1 -- 172.21.15.78:0/1540304037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d8103d40 msgr2=0x7f27d8199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27d7fff640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d8103d40 0x7f27d8199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27d7fff640 1 -- 172.21.15.78:0/1540304037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d810d1a0 msgr2=0x7f27d819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27d7fff640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d810d1a0 0x7f27d819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27d7fff640 1 -- 172.21.15.78:0/1540304037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d819f2b0 con 0x7f27d8104660 2024-09-11T01:19:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.750+0000 7f27d7fff640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d8104660 0x7f27d8199740 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f27c000c970 tx=0x7f27c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.751+0000 7f27d5ffb640 1 -- 172.21.15.78:0/1540304037 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0007bb0 con 0x7f27d8104660 2024-09-11T01:19:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.751+0000 7f27d5ffb640 1 -- 172.21.15.78:0/1540304037 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c0007d50 con 0x7f27d8104660 2024-09-11T01:19:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.751+0000 7f27d5ffb640 1 -- 172.21.15.78:0/1540304037 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c000f7b0 con 0x7f27d8104660 2024-09-11T01:19:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.751+0000 7f27df145640 1 -- 172.21.15.78:0/1540304037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d819f5a0 con 0x7f27d8104660 2024-09-11T01:19:03.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.751+0000 7f27df145640 1 -- 172.21.15.78:0/1540304037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d81a5e30 con 0x7f27d8104660 2024-09-11T01:19:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.753+0000 7f27d5ffb640 1 -- 172.21.15.78:0/1540304037 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f27c0018480 con 0x7f27d8104660 2024-09-11T01:19:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27e151a740 1 Processor -- start 2024-09-11T01:19:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27e151a740 1 -- start start 2024-09-11T01:19:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27e151a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9880f0 0x56030e9884d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27e151a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e994460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27e151a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9949a0 0x56030e996d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27e151a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56030e8feea0 con 0x56030e9949a0 2024-09-11T01:19:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27e151a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56030e8ee2d0 con 0x56030e988a10 2024-09-11T01:19:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27e151a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56030e8f20e0 con 0x56030e9880f0 2024-09-11T01:19:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9949a0 0x56030e996d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27d5ffb640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27c40b54d0 0x7f27c40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9949a0 0x56030e996d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56936/0 (socket says 172.21.15.78:56936) 2024-09-11T01:19:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27d7fff640 1 -- 172.21.15.78:0/2194139778 learned_addr learned my addr 172.21.15.78:0/2194139778 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27dd6e2640 1 --2- 172.21.15.78:0/2194139778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9880f0 0x56030e9884d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27dcee1640 1 --2- 172.21.15.78:0/2194139778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e994460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27dd6e2640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27c40b54d0 0x7f27c40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.754+0000 7f27d5ffb640 1 -- 172.21.15.78:0/1540304037 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f27c00e6280 con 0x7f27d8104660 2024-09-11T01:19:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.755+0000 7f27dcee1640 1 -- 172.21.15.78:0/2194139778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9880f0 msgr2=0x56030e9884d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:03.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.755+0000 7f27dcee1640 1 --2- 172.21.15.78:0/2194139778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9880f0 0x56030e9884d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.755+0000 7f27dcee1640 1 -- 172.21.15.78:0/2194139778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9949a0 msgr2=0x56030e996d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.755+0000 7f27dcee1640 1 --2- 172.21.15.78:0/2194139778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9949a0 0x56030e996d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.755+0000 7f27dcee1640 1 -- 172.21.15.78:0/2194139778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56030e64c650 con 0x56030e988a10 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.755+0000 7f27dd6e2640 1 --2- 172.21.15.78:0/2194139778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9880f0 0x56030e9884d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.755+0000 7f27d7fff640 1 --2- 172.21.15.78:0/2194139778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9949a0 0x56030e996d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.755+0000 7f27dd6e2640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27c40b54d0 0x7f27c40b7990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f27d0005940 tx=0x7f27d0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.755+0000 7f27dcee1640 1 --2- 172.21.15.78:0/2194139778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e994460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f27c800ae70 tx=0x7f27c800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95aa10b649b89d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.755+0000 7f27ceffd640 1 -- 172.21.15.78:0/2194139778 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8003b70 con 0x56030e988a10 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.755+0000 7f27ceffd640 1 -- 172.21.15.78:0/2194139778 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c8003d10 con 0x56030e988a10 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.755+0000 7f27ceffd640 1 -- 172.21.15.78:0/2194139778 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8010aa0 con 0x56030e988a10 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.756+0000 7f27e151a740 1 -- 172.21.15.78:0/2194139778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 msgr2=0x56030e994460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.756+0000 7f27e151a740 1 --2- 172.21.15.78:0/2194139778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e994460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f27c800ae70 tx=0x7f27c800f790 comp rx=0 tx=0).stop 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.756+0000 7f27e151a740 1 -- 172.21.15.78:0/2194139778 shutdown_connections 2024-09-11T01:19:03.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.756+0000 7f27e151a740 1 --2- 172.21.15.78:0/2194139778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9949a0 0x56030e996d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.756+0000 7f27e151a740 1 --2- 172.21.15.78:0/2194139778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e994460 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.756+0000 7f27e151a740 1 --2- 172.21.15.78:0/2194139778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9880f0 0x56030e9884d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.756+0000 7f27e151a740 1 -- 172.21.15.78:0/2194139778 >> 172.21.15.78:0/2194139778 conn(0x56030e985f20 msgr2=0x56030e9870c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:03.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.756+0000 7f27e151a740 1 -- 172.21.15.78:0/2194139778 shutdown_connections 2024-09-11T01:19:03.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.756+0000 7f27e151a740 1 -- 172.21.15.78:0/2194139778 wait complete. 2024-09-11T01:19:03.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.756+0000 7f27e151a740 1 -- start start 2024-09-11T01:19:03.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.756+0000 7f27e151a740 1 Processor -- start 2024-09-11T01:19:03.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27e151a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e9880f0 0x56030e994190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27e151a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e988a10 0x56030e9915e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27e151a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e991b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27e151a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56030e74f700 con 0x56030e988a10 2024-09-11T01:19:03.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27e151a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56030e889160 con 0x56030e9880f0 2024-09-11T01:19:03.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27e151a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56030e997620 con 0x56030e9949a0 2024-09-11T01:19:03.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27e151a740 10 client.0 _ll_register_callbacks cb 0x56030e8a29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:03.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e988a10 0x56030e9915e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e988a10 0x56030e9915e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56938/0 (socket says 172.21.15.78:56938) 2024-09-11T01:19:03.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27d7fff640 1 -- 172.21.15.78:0/1256213810 learned_addr learned my addr 172.21.15.78:0/1256213810 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:03.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27dcee1640 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e9880f0 0x56030e994190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27dd6e2640 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e991b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27d7fff640 1 -- 172.21.15.78:0/1256213810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 msgr2=0x56030e991b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:03.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27d7fff640 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e991b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27d7fff640 1 -- 172.21.15.78:0/1256213810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e9880f0 msgr2=0x56030e994190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27d7fff640 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e9880f0 0x56030e994190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27d7fff640 1 -- 172.21.15.78:0/1256213810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56030e8b6c90 con 0x56030e988a10 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27dd6e2640 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e991b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.757+0000 7f27d7fff640 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e988a10 0x56030e9915e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f27c00a8d00 tx=0x7f27c00abf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce2a69b54c827eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.758+0000 7f27b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.759+0000 7f27b7fff640 1 -- 172.21.15.78:0/1256213810 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c00ace30 con 0x56030e988a10 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.759+0000 7f27b7fff640 1 -- 172.21.15.78:0/1256213810 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f27c0018a30 con 0x56030e988a10 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.759+0000 7f27e151a740 1 -- 172.21.15.78:0/1256213810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27c8003d10 con 0x56030e988a10 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.759+0000 7f27b7fff640 1 -- 172.21.15.78:0/1256213810 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f27c00ac730 con 0x56030e988a10 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.759+0000 7f27e151a740 10 client.18930 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.759+0000 7f27e151a740 1 -- 172.21.15.78:0/1256213810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x56030e9a2e80 con 0x56030e988a10 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.760+0000 7f27b7fff640 1 -- 172.21.15.78:0/1256213810 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c8003d10 con 0x56030e988a10 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.760+0000 7f27b7fff640 1 -- 172.21.15.78:0/1256213810 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f27c00b57a0 con 0x56030e988a10 2024-09-11T01:19:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.760+0000 7f27e151a740 10 client.18930 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.760+0000 7f27e151a740 10 client.18930 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.760+0000 7f27e151a740 20 client.18930 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.760+0000 7f27e151a740 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56030e9a4340 0x56030e9a6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.760+0000 7f27e151a740 4 client.18930 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.760+0000 7f27e151a740 1 -- 172.21.15.78:0/1256213810 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56030e900df0 con 0x56030e9a4340 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.760+0000 7f27dcee1640 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56030e9a4340 0x56030e9a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.760+0000 7f27b7fff640 10 client.18930 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.760+0000 7f27dcee1640 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56030e9a4340 0x56030e9a6780 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=514f33b8085d7530 server_cookie=84d0bd18b51c865c in_seq=0 out_seq=0 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27b7fff640 1 -- 172.21.15.78:0/1256213810 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56030e900df0 con 0x56030e9a4340 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27b7fff640 10 client.18930 handle_command_reply: tid=0 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27e151a740 1 client.18930 shutdown 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27e151a740 1 -- 172.21.15.78:0/1256213810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e988a10 msgr2=0x56030e9915e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27e151a740 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e988a10 0x56030e9915e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f27c00a8d00 tx=0x7f27c00abf60 comp rx=0 tx=0).stop 2024-09-11T01:19:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27e151a740 1 -- 172.21.15.78:0/1256213810 shutdown_connections 2024-09-11T01:19:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27e151a740 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56030e9a4340 0x56030e9a6780 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27e151a740 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e991b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27e151a740 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e988a10 0x56030e9915e0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27e151a740 1 --2- 172.21.15.78:0/1256213810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e9880f0 0x56030e994190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27e151a740 1 -- 172.21.15.78:0/1256213810 >> 172.21.15.78:0/1256213810 conn(0x56030e985f20 msgr2=0x56030e98ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27b7fff640 0 client.18930 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:19:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27e151a740 1 -- 172.21.15.78:0/1256213810 shutdown_connections 2024-09-11T01:19:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27e151a740 1 -- 172.21.15.78:0/1256213810 wait complete. 2024-09-11T01:19:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.761+0000 7f27e151a740 20 client.18930 trim_cache size 0 max 16384 2024-09-11T01:19:03.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.774+0000 7f27e151a740 1 Processor -- start 2024-09-11T01:19:03.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.774+0000 7f27e151a740 1 -- start start 2024-09-11T01:19:03.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.774+0000 7f27e151a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 0x56030e98c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.774+0000 7f27e151a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e993320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.774+0000 7f27e151a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e98e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.774+0000 7f27e151a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27d81ae540 con 0x56030e9880f0 2024-09-11T01:19:03.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.774+0000 7f27e151a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27d81b0080 con 0x56030e988a10 2024-09-11T01:19:03.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.774+0000 7f27e151a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27d81aec10 con 0x56030e9949a0 2024-09-11T01:19:03.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27dcee1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 0x56030e98c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27dcee1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 0x56030e98c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56946/0 (socket says 172.21.15.78:56946) 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27dcee1640 1 -- 172.21.15.78:0/155533104 learned_addr learned my addr 172.21.15.78:0/155533104 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27d7fff640 1 --2- 172.21.15.78:0/155533104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e993320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27dcee1640 1 -- 172.21.15.78:0/155533104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 msgr2=0x56030e98e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27dd6e2640 1 --2- 172.21.15.78:0/155533104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e98e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27dcee1640 1 --2- 172.21.15.78:0/155533104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e98e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27dcee1640 1 -- 172.21.15.78:0/155533104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 msgr2=0x56030e993320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27dcee1640 1 --2- 172.21.15.78:0/155533104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e993320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27dcee1640 1 -- 172.21.15.78:0/155533104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56030e997100 con 0x56030e9880f0 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27d7fff640 1 --2- 172.21.15.78:0/155533104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e993320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27dcee1640 1 --2- 172.21.15.78:0/155533104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 0x56030e98c8d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f27c801a040 tx=0x7f27c800a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98167b4c779a3129 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27ce7fc640 1 -- 172.21.15.78:0/155533104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8030070 con 0x56030e9880f0 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27ce7fc640 1 -- 172.21.15.78:0/155533104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c8005e40 con 0x56030e9880f0 2024-09-11T01:19:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.775+0000 7f27ce7fc640 1 -- 172.21.15.78:0/155533104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8005040 con 0x56030e9880f0 2024-09-11T01:19:03.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.776+0000 7f27e151a740 1 -- 172.21.15.78:0/155533104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 msgr2=0x56030e98c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:03.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.776+0000 7f27e151a740 1 --2- 172.21.15.78:0/155533104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 0x56030e98c8d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f27c801a040 tx=0x7f27c800a570 comp rx=0 tx=0).stop 2024-09-11T01:19:03.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.776+0000 7f27e151a740 1 -- 172.21.15.78:0/155533104 shutdown_connections 2024-09-11T01:19:03.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.776+0000 7f27e151a740 1 --2- 172.21.15.78:0/155533104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e98e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.776+0000 7f27e151a740 1 --2- 172.21.15.78:0/155533104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e993320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.776+0000 7f27e151a740 1 --2- 172.21.15.78:0/155533104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 0x56030e98c8d0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.776+0000 7f27e151a740 1 -- 172.21.15.78:0/155533104 >> 172.21.15.78:0/155533104 conn(0x56030e985f20 msgr2=0x56030e992c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.776+0000 7f27e151a740 1 -- 172.21.15.78:0/155533104 shutdown_connections 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.776+0000 7f27e151a740 1 -- 172.21.15.78:0/155533104 wait complete. 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.776+0000 7f27e151a740 1 -- start start 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.776+0000 7f27e151a740 1 Processor -- start 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27e151a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 0x56030e998ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27e151a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e999010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27e151a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e999590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27e151a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27d81af9b0 con 0x56030e9880f0 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27e151a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27d81af2e0 con 0x56030e988a10 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27e151a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27d81ac540 con 0x56030e9949a0 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27e151a740 10 client.0 _ll_register_callbacks cb 0x7f27c000a4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27dcee1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 0x56030e998ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27dcee1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 0x56030e998ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56948/0 (socket says 172.21.15.78:56948) 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27dcee1640 1 -- 172.21.15.78:0/2034990260 learned_addr learned my addr 172.21.15.78:0/2034990260 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27d7fff640 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e999010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27dd6e2640 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e999590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27dcee1640 1 -- 172.21.15.78:0/2034990260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 msgr2=0x56030e999590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27dcee1640 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e999590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27dcee1640 1 -- 172.21.15.78:0/2034990260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 msgr2=0x56030e999010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27dcee1640 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e999010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27dcee1640 1 -- 172.21.15.78:0/2034990260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56030ea10540 con 0x56030e9880f0 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27dd6e2640 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e999590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27dcee1640 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 0x56030e998ad0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f27c8000f80 tx=0x7f27c8010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8673684debbe6601 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.777+0000 7f27b7fff640 1 -- 172.21.15.78:0/2034990260 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8030070 con 0x56030e9880f0 2024-09-11T01:19:03.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27b7fff640 1 -- 172.21.15.78:0/2034990260 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f27c8004be0 con 0x56030e9880f0 2024-09-11T01:19:03.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27e151a740 1 -- 172.21.15.78:0/2034990260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27c8005e40 con 0x56030e9880f0 2024-09-11T01:19:03.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27b7fff640 1 -- 172.21.15.78:0/2034990260 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f27c8004de0 con 0x56030e9880f0 2024-09-11T01:19:03.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27e151a740 10 client.18936 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:03.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27e151a740 1 -- 172.21.15.78:0/2034990260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x56030ea11840 con 0x56030e9880f0 2024-09-11T01:19:03.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27b7fff640 1 -- 172.21.15.78:0/2034990260 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c8005e40 con 0x56030e9880f0 2024-09-11T01:19:03.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27b7fff640 1 -- 172.21.15.78:0/2034990260 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f27c802b050 con 0x56030e9880f0 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27e151a740 10 client.18936 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27e151a740 10 client.18936 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27e151a740 20 client.18936 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27e151a740 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56030ea11d60 0x56030ea14150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27e151a740 4 client.18936 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27e151a740 1 -- 172.21.15.78:0/2034990260 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56030e8fc8a0 con 0x56030ea11d60 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27d7fff640 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56030ea11d60 0x56030ea14150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.778+0000 7f27b7fff640 10 client.18936 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27d7fff640 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56030ea11d60 0x56030ea14150 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=81098d6db0243a0e server_cookie=924d86fc767e9c1a in_seq=0 out_seq=0 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27b7fff640 1 -- 172.21.15.78:0/2034990260 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56030e8fc8a0 con 0x56030ea11d60 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27b7fff640 10 client.18936 handle_command_reply: tid=0 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27e151a740 1 client.18936 shutdown 2024-09-11T01:19:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27e151a740 1 -- 172.21.15.78:0/2034990260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 msgr2=0x56030e998ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27e151a740 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 0x56030e998ad0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f27c8000f80 tx=0x7f27c8010640 comp rx=0 tx=0).stop 2024-09-11T01:19:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27e151a740 1 -- 172.21.15.78:0/2034990260 shutdown_connections 2024-09-11T01:19:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27e151a740 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x56030ea11d60 0x56030ea14150 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27e151a740 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56030e9949a0 0x56030e999590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27e151a740 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56030e988a10 0x56030e999010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27e151a740 1 --2- 172.21.15.78:0/2034990260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56030e9880f0 0x56030e998ad0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27e151a740 1 -- 172.21.15.78:0/2034990260 >> 172.21.15.78:0/2034990260 conn(0x56030e985f20 msgr2=0x56030e993430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27b7fff640 0 client.18936 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:19:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27e151a740 1 -- 172.21.15.78:0/2034990260 shutdown_connections 2024-09-11T01:19:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27e151a740 1 -- 172.21.15.78:0/2034990260 wait complete. 2024-09-11T01:19:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.779+0000 7f27e151a740 20 client.18936 trim_cache size 0 max 16384 2024-09-11T01:19:03.789 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:19:03.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.780+0000 7f27df145640 1 -- 172.21.15.78:0/1540304037 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27c40b54d0 msgr2=0x7f27c40b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:03.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.780+0000 7f27df145640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27c40b54d0 0x7f27c40b7990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f27d0005940 tx=0x7f27d0009260 comp rx=0 tx=0).stop 2024-09-11T01:19:03.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.780+0000 7f27df145640 1 -- 172.21.15.78:0/1540304037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d8104660 msgr2=0x7f27d8199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:03.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.780+0000 7f27df145640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d8104660 0x7f27d8199740 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f27c000c970 tx=0x7f27c000ce30 comp rx=0 tx=0).stop 2024-09-11T01:19:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.780+0000 7f27df145640 1 -- 172.21.15.78:0/1540304037 shutdown_connections 2024-09-11T01:19:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.780+0000 7f27df145640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27c40b54d0 0x7f27c40b7990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.780+0000 7f27df145640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d810d1a0 0x7f27d819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.780+0000 7f27df145640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d8104660 0x7f27d8199740 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.780+0000 7f27df145640 1 --2- 172.21.15.78:0/1540304037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d8103d40 0x7f27d8199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.780+0000 7f27df145640 1 -- 172.21.15.78:0/1540304037 >> 172.21.15.78:0/1540304037 conn(0x7f27d80fd7e0 msgr2=0x7f27d810f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.780+0000 7f27df145640 1 -- 172.21.15.78:0/1540304037 shutdown_connections 2024-09-11T01:19:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:03.780+0000 7f27df145640 1 -- 172.21.15.78:0/1540304037 wait complete. 2024-09-11T01:19:03.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 275357 34.04 MB/sec execute 309 sec latency 3890.484 ms 2024-09-11T01:19:03.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:03 smithi078 ceph-mon[31315]: pgmap v954: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 696 op/s 2024-09-11T01:19:04.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:03 smithi110 ceph-mon[34712]: pgmap v954: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 696 op/s 2024-09-11T01:19:04.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:03 smithi112 ceph-mon[35384]: pgmap v954: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 696 op/s 2024-09-11T01:19:04.529 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:19:04.796 INFO:tasks.workunit.client.0.smithi078.stdout: 10 275570 33.96 MB/sec execute 310 sec latency 4654.673 ms 2024-09-11T01:19:04.912 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:19:05.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.426+0000 7fea788bc640 1 -- 172.21.15.78:0/1696199682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74108910 msgr2=0x7fea74108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:05.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.426+0000 7fea788bc640 1 --2- 172.21.15.78:0/1696199682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74108910 0x7fea74108d50 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fea68009930 tx=0x7fea6802a420 comp rx=0 tx=0).stop 2024-09-11T01:19:05.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.426+0000 7fea788bc640 1 -- 172.21.15.78:0/1696199682 shutdown_connections 2024-09-11T01:19:05.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.426+0000 7fea788bc640 1 --2- 172.21.15.78:0/1696199682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74109290 0x7fea7410f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.426+0000 7fea788bc640 1 --2- 172.21.15.78:0/1696199682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74108910 0x7fea74108d50 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.426+0000 7fea788bc640 1 --2- 172.21.15.78:0/1696199682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea74101810 0x7fea74101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.426+0000 7fea788bc640 1 -- 172.21.15.78:0/1696199682 >> 172.21.15.78:0/1696199682 conn(0x7fea74076f20 msgr2=0x7fea74077330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:05.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.426+0000 7fea788bc640 1 -- 172.21.15.78:0/1696199682 shutdown_connections 2024-09-11T01:19:05.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.426+0000 7fea788bc640 1 -- 172.21.15.78:0/1696199682 wait complete. 2024-09-11T01:19:05.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea788bc640 1 Processor -- start 2024-09-11T01:19:05.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea788bc640 1 -- start start 2024-09-11T01:19:05.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea788bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74101810 0x7fea741992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea788bc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea74108910 0x7fea74199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea788bc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74109290 0x7fea7419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea788bc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea74112dd0 con 0x7fea74109290 2024-09-11T01:19:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea788bc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea74112c50 con 0x7fea74108910 2024-09-11T01:19:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea788bc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea74112f50 con 0x7fea74101810 2024-09-11T01:19:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea72d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74109290 0x7fea7419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea72d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74109290 0x7fea7419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56966/0 (socket says 172.21.15.78:56966) 2024-09-11T01:19:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea72d9d640 1 -- 172.21.15.78:0/862974281 learned_addr learned my addr 172.21.15.78:0/862974281 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea7259c640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74101810 0x7fea741992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea71d9b640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea74108910 0x7fea74199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea72d9d640 1 -- 172.21.15.78:0/862974281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74101810 msgr2=0x7fea741992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea72d9d640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74101810 0x7fea741992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea72d9d640 1 -- 172.21.15.78:0/862974281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea74108910 msgr2=0x7fea74199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea72d9d640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea74108910 0x7fea74199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea72d9d640 1 -- 172.21.15.78:0/862974281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7419f370 con 0x7fea74109290 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.427+0000 7fea71d9b640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea74108910 0x7fea74199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.428+0000 7fea72d9d640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74109290 0x7fea7419eba0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00eaa0 tx=0x7fea6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.428+0000 7fea637fe640 1 -- 172.21.15.78:0/862974281 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c00cdb0 con 0x7fea74109290 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.428+0000 7fea637fe640 1 -- 172.21.15.78:0/862974281 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea6c005020 con 0x7fea74109290 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.428+0000 7fea788bc640 1 -- 172.21.15.78:0/862974281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7419f660 con 0x7fea74109290 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.428+0000 7fea637fe640 1 -- 172.21.15.78:0/862974281 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0116d0 con 0x7fea74109290 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.428+0000 7fea788bc640 1 -- 172.21.15.78:0/862974281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea741a5d20 con 0x7fea74109290 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.430+0000 7fea7ac91740 1 Processor -- start 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.430+0000 7fea637fe640 1 -- 172.21.15.78:0/862974281 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fea6c00f040 con 0x7fea74109290 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea7ac91740 1 -- start start 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea7ac91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a86238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea7ac91740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 0x5567a862f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea7ac91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a862fd80 0x5567a8632170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea7ac91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5567a8599ea0 con 0x5567a86234d0 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea7ac91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5567a85892d0 con 0x5567a862fd80 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea7ac91740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5567a858d0e0 con 0x5567a8623df0 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea71d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a86238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea71d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a86238b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56968/0 (socket says 172.21.15.78:56968) 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea71d9b640 1 -- 172.21.15.78:0/2840818660 learned_addr learned my addr 172.21.15.78:0/2840818660 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea7259c640 1 --2- 172.21.15.78:0/2840818660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 0x5567a862f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea71d9b640 1 --2- 172.21.15.78:0/2840818660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a862fd80 0x5567a8632170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea71d9b640 1 -- 172.21.15.78:0/2840818660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 msgr2=0x5567a862f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea71d9b640 1 --2- 172.21.15.78:0/2840818660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 0x5567a862f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea71d9b640 1 -- 172.21.15.78:0/2840818660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a862fd80 msgr2=0x5567a8632170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea71d9b640 1 --2- 172.21.15.78:0/2840818660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a862fd80 0x5567a8632170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea71d9b640 1 -- 172.21.15.78:0/2840818660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567a82e7650 con 0x5567a86234d0 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.431+0000 7fea637fe640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fea5c0b5450 0x7fea5c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.432+0000 7fea71d9b640 1 --2- 172.21.15.78:0/2840818660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a86238b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fea68031880 tx=0x7fea68031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845113393c480cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.432+0000 7fea7259c640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fea5c0b5450 0x7fea5c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.432+0000 7fea637fe640 1 -- 172.21.15.78:0/862974281 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7fea6c0e5eb0 con 0x7fea74109290 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.432+0000 7fea60ff9640 1 -- 172.21.15.78:0/2840818660 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68035070 con 0x5567a86234d0 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.432+0000 7fea60ff9640 1 -- 172.21.15.78:0/2840818660 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea680318e0 con 0x5567a86234d0 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.432+0000 7fea60ff9640 1 -- 172.21.15.78:0/2840818660 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea680046c0 con 0x5567a86234d0 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 -- 172.21.15.78:0/2840818660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 msgr2=0x5567a86238b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7259c640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fea5c0b5450 0x7fea5c0b7910 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x5567a86221e0 tx=0x7fea6400a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 --2- 172.21.15.78:0/2840818660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a86238b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fea68031880 tx=0x7fea68031400 comp rx=0 tx=0).stop 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 -- 172.21.15.78:0/2840818660 shutdown_connections 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 --2- 172.21.15.78:0/2840818660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a862fd80 0x5567a8632170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 --2- 172.21.15.78:0/2840818660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 0x5567a862f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 --2- 172.21.15.78:0/2840818660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a86238b0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 -- 172.21.15.78:0/2840818660 >> 172.21.15.78:0/2840818660 conn(0x5567a8621300 msgr2=0x5567a86224a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 -- 172.21.15.78:0/2840818660 shutdown_connections 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 -- 172.21.15.78:0/2840818660 wait complete. 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 -- start start 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 Processor -- start 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a862f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a8623df0 0x5567a862cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a862fd80 0x5567a862d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.433+0000 7fea7ac91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5567a83ea700 con 0x5567a86234d0 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7ac91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5567a8524160 con 0x5567a8623df0 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7ac91740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5567a86329a0 con 0x5567a862fd80 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7ac91740 10 client.0 _ll_register_callbacks cb 0x5567a853d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a862f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a862f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56978/0 (socket says 172.21.15.78:56978) 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7259c640 1 -- 172.21.15.78:0/1887291787 learned_addr learned my addr 172.21.15.78:0/1887291787 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea72d9d640 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a862fd80 0x5567a862d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea71d9b640 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a8623df0 0x5567a862cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7259c640 1 -- 172.21.15.78:0/1887291787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a862fd80 msgr2=0x5567a862d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7259c640 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a862fd80 0x5567a862d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7259c640 1 -- 172.21.15.78:0/1887291787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a8623df0 msgr2=0x5567a862cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7259c640 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a8623df0 0x5567a862cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7259c640 1 -- 172.21.15.78:0/1887291787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a8551c90 con 0x5567a86234d0 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7259c640 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a862f6b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fea6401cbe0 tx=0x7fea64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a890eb1ae26b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea59ffb640 1 -- 172.21.15.78:0/1887291787 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6402c060 con 0x5567a86234d0 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea59ffb640 1 -- 172.21.15.78:0/1887291787 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7fea64016030 con 0x5567a86234d0 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7ac91740 1 -- 172.21.15.78:0/1887291787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea680318e0 con 0x5567a86234d0 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea59ffb640 1 -- 172.21.15.78:0/1887291787 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fea6400adf0 con 0x5567a86234d0 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7ac91740 10 client.18948 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.434+0000 7fea7ac91740 1 -- 172.21.15.78:0/1887291787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5567a863e3a0 con 0x5567a86234d0 2024-09-11T01:19:05.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.435+0000 7fea59ffb640 1 -- 172.21.15.78:0/1887291787 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea680318e0 con 0x5567a86234d0 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.435+0000 7fea59ffb640 1 -- 172.21.15.78:0/1887291787 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7fea6401a6d0 con 0x5567a86234d0 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.435+0000 7fea7ac91740 10 client.18948 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.435+0000 7fea7ac91740 10 client.18948 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.435+0000 7fea7ac91740 20 client.18948 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.435+0000 7fea7ac91740 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5567a863f860 0x5567a8641c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.435+0000 7fea7ac91740 4 client.18948 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.435+0000 7fea7ac91740 1 -- 172.21.15.78:0/1887291787 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567a859bdf0 con 0x5567a863f860 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.435+0000 7fea71d9b640 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5567a863f860 0x5567a8641c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.435+0000 7fea59ffb640 10 client.18948 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.435+0000 7fea71d9b640 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5567a863f860 0x5567a8641c70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d81c5e687c8631df server_cookie=8ccb450eb71abda7 in_seq=0 out_seq=0 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea59ffb640 1 -- 172.21.15.78:0/1887291787 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567a859bdf0 con 0x5567a863f860 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea59ffb640 10 client.18948 handle_command_reply: tid=0 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea7ac91740 1 client.18948 shutdown 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea7ac91740 1 -- 172.21.15.78:0/1887291787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 msgr2=0x5567a862f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:05.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea7ac91740 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a862f6b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fea6401cbe0 tx=0x7fea64004730 comp rx=0 tx=0).stop 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea7ac91740 1 -- 172.21.15.78:0/1887291787 shutdown_connections 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea7ac91740 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5567a863f860 0x5567a8641c70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea7ac91740 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a862fd80 0x5567a862d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea7ac91740 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a8623df0 0x5567a862cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea59ffb640 0 client.18948 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea7ac91740 1 --2- 172.21.15.78:0/1887291787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a862f6b0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea7ac91740 1 -- 172.21.15.78:0/1887291787 >> 172.21.15.78:0/1887291787 conn(0x5567a8621300 msgr2=0x5567a8626e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea7ac91740 1 -- 172.21.15.78:0/1887291787 shutdown_connections 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea7ac91740 1 -- 172.21.15.78:0/1887291787 wait complete. 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.436+0000 7fea7ac91740 20 client.18948 trim_cache size 0 max 16384 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.449+0000 7fea7ac91740 1 Processor -- start 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.449+0000 7fea7ac91740 1 -- start start 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.449+0000 7fea7ac91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a863bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.449+0000 7fea7ac91740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 0x5567a862bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.449+0000 7fea7ac91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a862fd80 0x5567a8638040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.449+0000 7fea7ac91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea741ae650 con 0x5567a86234d0 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.449+0000 7fea7ac91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea741b0190 con 0x5567a862fd80 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.449+0000 7fea7ac91740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea741aed20 con 0x5567a8623df0 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.449+0000 7fea7259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a863bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.449+0000 7fea7259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a863bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56988/0 (socket says 172.21.15.78:56988) 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.449+0000 7fea7259c640 1 -- 172.21.15.78:0/760343816 learned_addr learned my addr 172.21.15.78:0/760343816 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.450+0000 7fea71d9b640 1 --2- 172.21.15.78:0/760343816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 0x5567a862bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.450+0000 7fea7259c640 1 -- 172.21.15.78:0/760343816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 msgr2=0x5567a862bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.450+0000 7fea7259c640 1 --2- 172.21.15.78:0/760343816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 0x5567a862bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.450+0000 7fea7259c640 1 -- 172.21.15.78:0/760343816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a862fd80 msgr2=0x5567a8638040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.450+0000 7fea7259c640 1 --2- 172.21.15.78:0/760343816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a862fd80 0x5567a8638040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.450+0000 7fea7259c640 1 -- 172.21.15.78:0/760343816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567a862fa20 con 0x5567a86234d0 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.450+0000 7fea7259c640 1 --2- 172.21.15.78:0/760343816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a863bc00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fea640058c0 tx=0x7fea6400af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a3e285b24d7a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.450+0000 7fea5bfff640 1 -- 172.21.15.78:0/760343816 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6403f070 con 0x5567a86234d0 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.450+0000 7fea5bfff640 1 -- 172.21.15.78:0/760343816 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea6401a850 con 0x5567a86234d0 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.450+0000 7fea5bfff640 1 -- 172.21.15.78:0/760343816 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6400fb70 con 0x5567a86234d0 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 -- 172.21.15.78:0/760343816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 msgr2=0x5567a863bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 --2- 172.21.15.78:0/760343816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a863bc00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fea640058c0 tx=0x7fea6400af90 comp rx=0 tx=0).stop 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 -- 172.21.15.78:0/760343816 shutdown_connections 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 --2- 172.21.15.78:0/760343816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a862fd80 0x5567a8638040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 --2- 172.21.15.78:0/760343816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 0x5567a862bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 --2- 172.21.15.78:0/760343816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a86234d0 0x5567a863bc00 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 -- 172.21.15.78:0/760343816 >> 172.21.15.78:0/760343816 conn(0x5567a8621300 msgr2=0x5567a862b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 -- 172.21.15.78:0/760343816 shutdown_connections 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 -- 172.21.15.78:0/760343816 wait complete. 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 -- start start 2024-09-11T01:19:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 Processor -- start 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a86234d0 0x5567a8646580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 0x5567a8646ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a862fd80 0x5567a8647250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea741afac0 con 0x5567a862fd80 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea741af3f0 con 0x5567a86234d0 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea741ac650 con 0x5567a8623df0 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea7ac91740 10 client.0 _ll_register_callbacks cb 0x7fea6400a660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea72d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a862fd80 0x5567a8647250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea72d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a862fd80 0x5567a8647250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56998/0 (socket says 172.21.15.78:56998) 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea72d9d640 1 -- 172.21.15.78:0/575981394 learned_addr learned my addr 172.21.15.78:0/575981394 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.451+0000 7fea71d9b640 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 0x5567a8646ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea72d9d640 1 -- 172.21.15.78:0/575981394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 msgr2=0x5567a8646ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea72d9d640 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 0x5567a8646ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea7259c640 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a86234d0 0x5567a8646580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea72d9d640 1 -- 172.21.15.78:0/575981394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a86234d0 msgr2=0x5567a8646580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:05.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea72d9d640 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a86234d0 0x5567a8646580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea72d9d640 1 -- 172.21.15.78:0/575981394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a8647a80 con 0x5567a862fd80 2024-09-11T01:19:05.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea7259c640 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a86234d0 0x5567a8646580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:19:05.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea72d9d640 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a862fd80 0x5567a8647250 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0ac2a0 tx=0x7fea6c0b51e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0ba9954926e31c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:05.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:05.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea59ffb640 1 -- 172.21.15.78:0/575981394 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c01f070 con 0x5567a862fd80 2024-09-11T01:19:05.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea59ffb640 1 -- 172.21.15.78:0/575981394 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7fea6c0cf270 con 0x5567a862fd80 2024-09-11T01:19:05.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea7ac91740 1 -- 172.21.15.78:0/575981394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea6401a850 con 0x5567a862fd80 2024-09-11T01:19:05.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea59ffb640 1 -- 172.21.15.78:0/575981394 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fea6c0b5040 con 0x5567a862fd80 2024-09-11T01:19:05.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea7ac91740 10 client.18954 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:05.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea7ac91740 1 -- 172.21.15.78:0/575981394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5567a86b1070 con 0x5567a862fd80 2024-09-11T01:19:05.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea59ffb640 1 -- 172.21.15.78:0/575981394 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea6c059020 con 0x5567a862fd80 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.452+0000 7fea59ffb640 1 -- 172.21.15.78:0/575981394 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7fea6c0d30a0 con 0x5567a862fd80 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.453+0000 7fea7ac91740 10 client.18954 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.453+0000 7fea7ac91740 10 client.18954 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.453+0000 7fea7ac91740 20 client.18954 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.453+0000 7fea7ac91740 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5567a86b1810 0x5567a86b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.453+0000 7fea7ac91740 4 client.18954 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.453+0000 7fea7ac91740 1 -- 172.21.15.78:0/575981394 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567a85978a0 con 0x5567a86b1810 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.453+0000 7fea7259c640 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5567a86b1810 0x5567a86b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.453+0000 7fea59ffb640 10 client.18954 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.453+0000 7fea7259c640 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5567a86b1810 0x5567a86b3c30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5a6dc5394fa8d27a server_cookie=34f620cb41ee56f3 in_seq=0 out_seq=0 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.453+0000 7fea59ffb640 1 -- 172.21.15.78:0/575981394 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567a85978a0 con 0x5567a86b1810 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.453+0000 7fea59ffb640 10 client.18954 handle_command_reply: tid=0 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.453+0000 7fea7ac91740 1 client.18954 shutdown 2024-09-11T01:19:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea7ac91740 1 -- 172.21.15.78:0/575981394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a862fd80 msgr2=0x5567a8647250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:05.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea7ac91740 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a862fd80 0x5567a8647250 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0ac2a0 tx=0x7fea6c0b51e0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea7ac91740 1 -- 172.21.15.78:0/575981394 shutdown_connections 2024-09-11T01:19:05.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea7ac91740 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5567a86b1810 0x5567a86b3c30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea7ac91740 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567a862fd80 0x5567a8647250 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea59ffb640 0 client.18954 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:19:05.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea7ac91740 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567a8623df0 0x5567a8646ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea7ac91740 1 --2- 172.21.15.78:0/575981394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5567a86234d0 0x5567a8646580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea7ac91740 1 -- 172.21.15.78:0/575981394 >> 172.21.15.78:0/575981394 conn(0x5567a8621300 msgr2=0x5567a8622350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:05.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea7ac91740 1 -- 172.21.15.78:0/575981394 shutdown_connections 2024-09-11T01:19:05.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea7ac91740 1 -- 172.21.15.78:0/575981394 wait complete. 2024-09-11T01:19:05.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea7ac91740 20 client.18954 trim_cache size 0 max 16384 2024-09-11T01:19:05.471 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:19:05.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea788bc640 1 -- 172.21.15.78:0/862974281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fea5c0b5450 msgr2=0x7fea5c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:05.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea788bc640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fea5c0b5450 0x7fea5c0b7910 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x5567a86221e0 tx=0x7fea6400a2d0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea788bc640 1 -- 172.21.15.78:0/862974281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74109290 msgr2=0x7fea7419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:05.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.454+0000 7fea788bc640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74109290 0x7fea7419eba0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00eaa0 tx=0x7fea6c00ef60 comp rx=0 tx=0).stop 2024-09-11T01:19:05.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.455+0000 7fea788bc640 1 -- 172.21.15.78:0/862974281 shutdown_connections 2024-09-11T01:19:05.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.455+0000 7fea788bc640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fea5c0b5450 0x7fea5c0b7910 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.455+0000 7fea788bc640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74109290 0x7fea7419eba0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.455+0000 7fea788bc640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea74108910 0x7fea74199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.455+0000 7fea788bc640 1 --2- 172.21.15.78:0/862974281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74101810 0x7fea741992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:05.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.455+0000 7fea788bc640 1 -- 172.21.15.78:0/862974281 >> 172.21.15.78:0/862974281 conn(0x7fea74076f20 msgr2=0x7fea74100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:05.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.455+0000 7fea788bc640 1 -- 172.21.15.78:0/862974281 shutdown_connections 2024-09-11T01:19:05.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:05.455+0000 7fea788bc640 1 -- 172.21.15.78:0/862974281 wait complete. 2024-09-11T01:19:05.796 INFO:tasks.workunit.client.0.smithi078.stdout: 10 275627 33.87 MB/sec execute 311 sec latency 978.127 ms 2024-09-11T01:19:06.063 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:05 smithi078 ceph-mon[31315]: pgmap v955: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 585 op/s 2024-09-11T01:19:06.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:05 smithi110 ceph-mon[34712]: pgmap v955: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 585 op/s 2024-09-11T01:19:06.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:05 smithi112 ceph-mon[35384]: pgmap v955: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 585 op/s 2024-09-11T01:19:06.390 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:19:06.639 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:19:06.795 INFO:tasks.workunit.client.0.smithi078.stdout: 10 275627 33.76 MB/sec execute 312 sec latency 1978.167 ms 2024-09-11T01:19:07.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.145+0000 7f15ea896640 1 -- 172.21.15.78:0/1146875516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15e41045e0 msgr2=0x7f15e410cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:07.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.145+0000 7f15ea896640 1 --2- 172.21.15.78:0/1146875516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15e41045e0 0x7f15e410cad0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009900 tx=0x7f15d802a410 comp rx=0 tx=0).stop 2024-09-11T01:19:07.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.145+0000 7f15ea896640 1 -- 172.21.15.78:0/1146875516 shutdown_connections 2024-09-11T01:19:07.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.145+0000 7f15ea896640 1 --2- 172.21.15.78:0/1146875516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e410d120 0x7f15e410f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.145+0000 7f15ea896640 1 --2- 172.21.15.78:0/1146875516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15e41045e0 0x7f15e410cad0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.145+0000 7f15ea896640 1 --2- 172.21.15.78:0/1146875516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15e4103cc0 0x7f15e41040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.145+0000 7f15ea896640 1 -- 172.21.15.78:0/1146875516 >> 172.21.15.78:0/1146875516 conn(0x7f15e40fd700 msgr2=0x7f15e40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.145+0000 7f15ea896640 1 -- 172.21.15.78:0/1146875516 shutdown_connections 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.145+0000 7f15ea896640 1 -- 172.21.15.78:0/1146875516 wait complete. 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15ea896640 1 Processor -- start 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15ea896640 1 -- start start 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15ea896640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15e4103cc0 0x7f15e4199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15ea896640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15e41045e0 0x7f15e41998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15ea896640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e410d120 0x7f15e419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15ea896640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4112d90 con 0x7f15e4103cc0 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15ea896640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4112c10 con 0x7f15e410d120 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15ea896640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4112f10 con 0x7f15e41045e0 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15e3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15e4103cc0 0x7f15e4199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15e3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15e4103cc0 0x7f15e4199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57010/0 (socket says 172.21.15.78:57010) 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15e3fff640 1 -- 172.21.15.78:0/2210125024 learned_addr learned my addr 172.21.15.78:0/2210125024 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:07.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15e37fe640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15e41045e0 0x7f15e41998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.146+0000 7f15e8e33640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e410d120 0x7f15e419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15e3fff640 1 -- 172.21.15.78:0/2210125024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15e41045e0 msgr2=0x7f15e41998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15e3fff640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15e41045e0 0x7f15e41998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15e3fff640 1 -- 172.21.15.78:0/2210125024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e410d120 msgr2=0x7f15e419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15e3fff640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e410d120 0x7f15e419ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15e3fff640 1 -- 172.21.15.78:0/2210125024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e419f430 con 0x7f15e4103cc0 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15e37fe640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15e41045e0 0x7f15e41998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15e8e33640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e410d120 0x7f15e419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15e3fff640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15e4103cc0 0x7f15e4199380 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f15d400d770 tx=0x7f15d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15e17fa640 1 -- 172.21.15.78:0/2210125024 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4017070 con 0x7f15e4103cc0 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15e17fa640 1 -- 172.21.15.78:0/2210125024 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d40054d0 con 0x7f15e4103cc0 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15e17fa640 1 -- 172.21.15.78:0/2210125024 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4004c30 con 0x7f15e4103cc0 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15ea896640 1 -- 172.21.15.78:0/2210125024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e419f720 con 0x7f15e4103cc0 2024-09-11T01:19:07.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.147+0000 7f15ea896640 1 -- 172.21.15.78:0/2210125024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e41a5de0 con 0x7f15e4103cc0 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15e17fa640 1 -- 172.21.15.78:0/2210125024 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f15d4004e10 con 0x7f15e4103cc0 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15ecc6b740 1 Processor -- start 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15ecc6b740 1 -- start start 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15ecc6b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eee0f0 0x563863eee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15ecc6b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eeea10 0x563863efa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15ecc6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 0x563863efcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15ecc6b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563863e64ea0 con 0x563863eeea10 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15ecc6b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563863e542d0 con 0x563863eee0f0 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15ecc6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563863e580e0 con 0x563863efa9a0 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15e3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eeea10 0x563863efa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15e3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eeea10 0x563863efa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57016/0 (socket says 172.21.15.78:57016) 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15e3fff640 1 -- 172.21.15.78:0/531924955 learned_addr learned my addr 172.21.15.78:0/531924955 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:07.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15e8e33640 1 --2- 172.21.15.78:0/531924955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eee0f0 0x563863eee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15e37fe640 1 --2- 172.21.15.78:0/531924955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 0x563863efcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15e17fa640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15cc0b5450 0x7f15cc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15e3fff640 1 -- 172.21.15.78:0/531924955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 msgr2=0x563863efcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15e3fff640 1 --2- 172.21.15.78:0/531924955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 0x563863efcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.150+0000 7f15e3fff640 1 -- 172.21.15.78:0/531924955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eee0f0 msgr2=0x563863eee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.151+0000 7f15e3fff640 1 --2- 172.21.15.78:0/531924955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eee0f0 0x563863eee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.151+0000 7f15e3fff640 1 -- 172.21.15.78:0/531924955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563863bb2650 con 0x563863eeea10 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.151+0000 7f15e8e33640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15cc0b5450 0x7f15cc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.151+0000 7f15e17fa640 1 -- 172.21.15.78:0/2210125024 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f15d40e5b50 con 0x7f15e4103cc0 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.151+0000 7f15e3fff640 1 --2- 172.21.15.78:0/531924955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eeea10 0x563863efa460 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f15d40e3250 tx=0x7f15d40dc480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f9188a6591585f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.151+0000 7f15ca7fc640 1 -- 172.21.15.78:0/531924955 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4020030 con 0x563863eeea10 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.151+0000 7f15ca7fc640 1 -- 172.21.15.78:0/531924955 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d40abb20 con 0x563863eeea10 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.151+0000 7f15ca7fc640 1 -- 172.21.15.78:0/531924955 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d40ab960 con 0x563863eeea10 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.152+0000 7f15ecc6b740 1 -- 172.21.15.78:0/531924955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eeea10 msgr2=0x563863efa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:07.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.152+0000 7f15e8e33640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15cc0b5450 0x7f15cc0b7910 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f15dc005fd0 tx=0x7f15dc004a40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:07.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.152+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/531924955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eeea10 0x563863efa460 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f15d40e3250 tx=0x7f15d40dc480 comp rx=0 tx=0).stop 2024-09-11T01:19:07.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.152+0000 7f15ecc6b740 1 -- 172.21.15.78:0/531924955 shutdown_connections 2024-09-11T01:19:07.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.152+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/531924955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 0x563863efcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.152+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/531924955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eeea10 0x563863efa460 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.152+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/531924955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eee0f0 0x563863eee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.152+0000 7f15ecc6b740 1 -- 172.21.15.78:0/531924955 >> 172.21.15.78:0/531924955 conn(0x563863eebf20 msgr2=0x563863eed0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:07.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.152+0000 7f15ecc6b740 1 -- 172.21.15.78:0/531924955 shutdown_connections 2024-09-11T01:19:07.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.152+0000 7f15ecc6b740 1 -- 172.21.15.78:0/531924955 wait complete. 2024-09-11T01:19:07.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.152+0000 7f15ecc6b740 1 -- start start 2024-09-11T01:19:07.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.152+0000 7f15ecc6b740 1 Processor -- start 2024-09-11T01:19:07.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15ecc6b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 0x563863efa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15ecc6b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eeea10 0x563863ef76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15ecc6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 0x563863ef7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15ecc6b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563863cb5700 con 0x563863eee0f0 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15ecc6b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563863def160 con 0x563863eeea10 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15ecc6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563863efd560 con 0x563863efa9a0 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15ecc6b740 10 client.0 _ll_register_callbacks cb 0x563863e089e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15e3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 0x563863efa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15e3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 0x563863efa280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57030/0 (socket says 172.21.15.78:57030) 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15e3fff640 1 -- 172.21.15.78:0/3162645598 learned_addr learned my addr 172.21.15.78:0/3162645598 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15e37fe640 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eeea10 0x563863ef76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15e8e33640 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 0x563863ef7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15e3fff640 1 -- 172.21.15.78:0/3162645598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 msgr2=0x563863ef7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15e3fff640 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 0x563863ef7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15e3fff640 1 -- 172.21.15.78:0/3162645598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eeea10 msgr2=0x563863ef76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15e3fff640 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eeea10 0x563863ef76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15e3fff640 1 -- 172.21.15.78:0/3162645598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563863e1cc90 con 0x563863eee0f0 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15e3fff640 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 0x563863efa280 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f15d40e3390 tx=0x7f15d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de110626e715308c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15c37fe640 1 -- 172.21.15.78:0/3162645598 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4020050 con 0x563863eee0f0 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15c37fe640 1 -- 172.21.15.78:0/3162645598 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f15d40e2ea0 con 0x563863eee0f0 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15ecc6b740 1 -- 172.21.15.78:0/3162645598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15d40abb20 con 0x563863eee0f0 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15c37fe640 1 -- 172.21.15.78:0/3162645598 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f15d40ab780 con 0x563863eee0f0 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15ecc6b740 10 client.18966 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.153+0000 7f15ecc6b740 1 -- 172.21.15.78:0/3162645598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x563863f08eb0 con 0x563863eee0f0 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.154+0000 7f15c37fe640 1 -- 172.21.15.78:0/3162645598 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d40abb20 con 0x563863eee0f0 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.154+0000 7f15c37fe640 1 -- 172.21.15.78:0/3162645598 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f15d40e2420 con 0x563863eee0f0 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.154+0000 7f15ecc6b740 10 client.18966 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.154+0000 7f15ecc6b740 10 client.18966 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.154+0000 7f15ecc6b740 20 client.18966 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.154+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x563863f0a200 0x563863f0c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.154+0000 7f15ecc6b740 4 client.18966 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.154+0000 7f15ecc6b740 1 -- 172.21.15.78:0/3162645598 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563863e66df0 con 0x563863f0a200 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.154+0000 7f15e37fe640 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x563863f0a200 0x563863f0c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.155+0000 7f15e37fe640 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x563863f0a200 0x563863f0c5f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f309b77739b4fa7a server_cookie=c2889b60a302a36d in_seq=0 out_seq=0 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.155+0000 7f15c37fe640 10 client.18966 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15c37fe640 1 -- 172.21.15.78:0/3162645598 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563863e66df0 con 0x563863f0a200 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15c37fe640 10 client.18966 handle_command_reply: tid=0 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15ecc6b740 1 client.18966 shutdown 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15ecc6b740 1 -- 172.21.15.78:0/3162645598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 msgr2=0x563863efa280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 0x563863efa280 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f15d40e3390 tx=0x7f15d4004730 comp rx=0 tx=0).stop 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15ecc6b740 1 -- 172.21.15.78:0/3162645598 shutdown_connections 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 0x563863ef7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x563863f0a200 0x563863f0c5f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eeea10 0x563863ef76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/3162645598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 0x563863efa280 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15ecc6b740 1 -- 172.21.15.78:0/3162645598 >> 172.21.15.78:0/3162645598 conn(0x563863eebf20 msgr2=0x563863ef1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15c37fe640 0 client.18966 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:19:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15ecc6b740 1 -- 172.21.15.78:0/3162645598 shutdown_connections 2024-09-11T01:19:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15ecc6b740 1 -- 172.21.15.78:0/3162645598 wait complete. 2024-09-11T01:19:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.156+0000 7f15ecc6b740 20 client.18966 trim_cache size 0 max 16384 2024-09-11T01:19:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15ecc6b740 1 Processor -- start 2024-09-11T01:19:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15ecc6b740 1 -- start start 2024-09-11T01:19:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15ecc6b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eee0f0 0x563863f06830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15ecc6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863eeea10 0x563863efbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15ecc6b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863efa9a0 0x563863efc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15ecc6b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41ae6c0 con 0x563863efa9a0 2024-09-11T01:19:07.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15ecc6b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41b0200 con 0x563863eee0f0 2024-09-11T01:19:07.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15ecc6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41aed90 con 0x563863eeea10 2024-09-11T01:19:07.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15e8e33640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863efa9a0 0x563863efc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15e8e33640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863efa9a0 0x563863efc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57032/0 (socket says 172.21.15.78:57032) 2024-09-11T01:19:07.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15e8e33640 1 -- 172.21.15.78:0/2156045761 learned_addr learned my addr 172.21.15.78:0/2156045761 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:07.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15e37fe640 1 --2- 172.21.15.78:0/2156045761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863eeea10 0x563863efbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.173+0000 7f15e3fff640 1 --2- 172.21.15.78:0/2156045761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eee0f0 0x563863f06830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.174+0000 7f15e8e33640 1 -- 172.21.15.78:0/2156045761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863eeea10 msgr2=0x563863efbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:07.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.174+0000 7f15e8e33640 1 --2- 172.21.15.78:0/2156045761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863eeea10 0x563863efbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.174+0000 7f15e8e33640 1 -- 172.21.15.78:0/2156045761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eee0f0 msgr2=0x563863f06830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.174+0000 7f15e8e33640 1 --2- 172.21.15.78:0/2156045761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eee0f0 0x563863f06830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.174+0000 7f15e8e33640 1 -- 172.21.15.78:0/2156045761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563863ef5bd0 con 0x563863efa9a0 2024-09-11T01:19:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.174+0000 7f15e8e33640 1 --2- 172.21.15.78:0/2156045761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863efa9a0 0x563863efc2d0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f15dc010840 tx=0x7f15dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92209edeeee73e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.174+0000 7f15c9ffb640 1 -- 172.21.15.78:0/2156045761 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc010d70 con 0x563863efa9a0 2024-09-11T01:19:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.174+0000 7f15c9ffb640 1 -- 172.21.15.78:0/2156045761 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15dc01d8c0 con 0x563863efa9a0 2024-09-11T01:19:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.174+0000 7f15c9ffb640 1 -- 172.21.15.78:0/2156045761 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc01b650 con 0x563863efa9a0 2024-09-11T01:19:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.175+0000 7f15ecc6b740 1 -- 172.21.15.78:0/2156045761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863efa9a0 msgr2=0x563863efc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.175+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/2156045761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863efa9a0 0x563863efc2d0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f15dc010840 tx=0x7f15dc016000 comp rx=0 tx=0).stop 2024-09-11T01:19:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.175+0000 7f15ecc6b740 1 -- 172.21.15.78:0/2156045761 shutdown_connections 2024-09-11T01:19:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.175+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/2156045761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863efa9a0 0x563863efc2d0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.175+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/2156045761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863eeea10 0x563863efbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.175+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/2156045761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eee0f0 0x563863f06830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.175+0000 7f15ecc6b740 1 -- 172.21.15.78:0/2156045761 >> 172.21.15.78:0/2156045761 conn(0x563863eebf20 msgr2=0x563863eed420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.175+0000 7f15ecc6b740 1 -- 172.21.15.78:0/2156045761 shutdown_connections 2024-09-11T01:19:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.175+0000 7f15ecc6b740 1 -- 172.21.15.78:0/2156045761 wait complete. 2024-09-11T01:19:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.175+0000 7f15ecc6b740 1 -- start start 2024-09-11T01:19:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15ecc6b740 1 Processor -- start 2024-09-11T01:19:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15ecc6b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 0x563863f0b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15ecc6b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eeea10 0x563863f0b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15ecc6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 0x563863f0bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15ecc6b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41afb30 con 0x563863eee0f0 2024-09-11T01:19:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15ecc6b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41af460 con 0x563863eeea10 2024-09-11T01:19:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15ecc6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41ac6c0 con 0x563863efa9a0 2024-09-11T01:19:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15ecc6b740 10 client.0 _ll_register_callbacks cb 0x7f15dc009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15e3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 0x563863f0b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15e3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 0x563863f0b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57042/0 (socket says 172.21.15.78:57042) 2024-09-11T01:19:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15e3fff640 1 -- 172.21.15.78:0/755028164 learned_addr learned my addr 172.21.15.78:0/755028164 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15e8e33640 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 0x563863f0bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15e37fe640 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eeea10 0x563863f0b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15e3fff640 1 -- 172.21.15.78:0/755028164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 msgr2=0x563863f0bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15e3fff640 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 0x563863f0bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15e3fff640 1 -- 172.21.15.78:0/755028164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eeea10 msgr2=0x563863f0b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15e3fff640 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eeea10 0x563863f0b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15e3fff640 1 -- 172.21.15.78:0/755028164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563863f75d90 con 0x563863eee0f0 2024-09-11T01:19:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15e3fff640 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 0x563863f0b180 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f15d400d740 tx=0x7f15d40a7bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793407f7291fb89d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.176+0000 7f15c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.177+0000 7f15c37fe640 1 -- 172.21.15.78:0/755028164 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4020070 con 0x563863eee0f0 2024-09-11T01:19:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.177+0000 7f15c37fe640 1 -- 172.21.15.78:0/755028164 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f15d4005130 con 0x563863eee0f0 2024-09-11T01:19:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.177+0000 7f15ecc6b740 1 -- 172.21.15.78:0/755028164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15dc01d8c0 con 0x563863eee0f0 2024-09-11T01:19:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.177+0000 7f15c37fe640 1 -- 172.21.15.78:0/755028164 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f15d4005330 con 0x563863eee0f0 2024-09-11T01:19:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.177+0000 7f15ecc6b740 10 client.18972 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.177+0000 7f15ecc6b740 1 -- 172.21.15.78:0/755028164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x563863f77090 con 0x563863eee0f0 2024-09-11T01:19:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.177+0000 7f15c37fe640 1 -- 172.21.15.78:0/755028164 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15dc01d8c0 con 0x563863eee0f0 2024-09-11T01:19:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.177+0000 7f15c37fe640 1 -- 172.21.15.78:0/755028164 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f15d40b5110 con 0x563863eee0f0 2024-09-11T01:19:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.177+0000 7f15ecc6b740 10 client.18972 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.177+0000 7f15ecc6b740 10 client.18972 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:19:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.177+0000 7f15ecc6b740 20 client.18972 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.177+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x563863f77570 0x563863f79990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15ecc6b740 4 client.18972 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:19:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15ecc6b740 1 -- 172.21.15.78:0/755028164 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563863e628a0 con 0x563863f77570 2024-09-11T01:19:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15e37fe640 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x563863f77570 0x563863f79990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15c37fe640 10 client.18972 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15e37fe640 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x563863f77570 0x563863f79990 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32fa3e2dca283ae3 server_cookie=e6c0058ef65d872a in_seq=0 out_seq=0 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15c37fe640 1 -- 172.21.15.78:0/755028164 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563863e628a0 con 0x563863f77570 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15c37fe640 10 client.18972 handle_command_reply: tid=0 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15ecc6b740 1 client.18972 shutdown 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15ecc6b740 1 -- 172.21.15.78:0/755028164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 msgr2=0x563863f0b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 0x563863f0b180 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f15d400d740 tx=0x7f15d40a7bf0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15ecc6b740 1 -- 172.21.15.78:0/755028164 shutdown_connections 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563863efa9a0 0x563863f0bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x563863f77570 0x563863f79990 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563863eeea10 0x563863f0b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.178+0000 7f15ecc6b740 1 --2- 172.21.15.78:0/755028164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563863eee0f0 0x563863f0b180 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ecc6b740 1 -- 172.21.15.78:0/755028164 >> 172.21.15.78:0/755028164 conn(0x563863eebf20 msgr2=0x563863ef8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15c37fe640 0 client.18972 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:19:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ecc6b740 1 -- 172.21.15.78:0/755028164 shutdown_connections 2024-09-11T01:19:07.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ecc6b740 1 -- 172.21.15.78:0/755028164 wait complete. 2024-09-11T01:19:07.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ecc6b740 20 client.18972 trim_cache size 0 max 16384 2024-09-11T01:19:07.187 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:19:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ea896640 1 -- 172.21.15.78:0/2210125024 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15cc0b5450 msgr2=0x7f15cc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ea896640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15cc0b5450 0x7f15cc0b7910 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f15dc005fd0 tx=0x7f15dc004a40 comp rx=0 tx=0).stop 2024-09-11T01:19:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ea896640 1 -- 172.21.15.78:0/2210125024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15e4103cc0 msgr2=0x7f15e4199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ea896640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15e4103cc0 0x7f15e4199380 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f15d400d770 tx=0x7f15d400dc30 comp rx=0 tx=0).stop 2024-09-11T01:19:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ea896640 1 -- 172.21.15.78:0/2210125024 shutdown_connections 2024-09-11T01:19:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ea896640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f15cc0b5450 0x7f15cc0b7910 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ea896640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e410d120 0x7f15e419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ea896640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15e41045e0 0x7f15e41998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ea896640 1 --2- 172.21.15.78:0/2210125024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15e4103cc0 0x7f15e4199380 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ea896640 1 -- 172.21.15.78:0/2210125024 >> 172.21.15.78:0/2210125024 conn(0x7f15e40fd700 msgr2=0x7f15e40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ea896640 1 -- 172.21.15.78:0/2210125024 shutdown_connections 2024-09-11T01:19:07.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:07.179+0000 7f15ea896640 1 -- 172.21.15.78:0/2210125024 wait complete. 2024-09-11T01:19:07.796 INFO:tasks.workunit.client.0.smithi078.stdout: 10 275627 33.65 MB/sec execute 313 sec latency 2978.258 ms 2024-09-11T01:19:07.871 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:07 smithi078 ceph-mon[31315]: pgmap v956: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 12 MiB/s wr, 536 op/s 2024-09-11T01:19:07.898 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:19:08.143 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:19:08.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:07 smithi110 ceph-mon[34712]: pgmap v956: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 12 MiB/s wr, 536 op/s 2024-09-11T01:19:08.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:07 smithi112 ceph-mon[35384]: pgmap v956: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 12 MiB/s wr, 536 op/s 2024-09-11T01:19:08.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 -- 172.21.15.78:0/2024976687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac101c90 msgr2=0x7f6dac102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:08.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 --2- 172.21.15.78:0/2024976687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac101c90 0x7f6dac102070 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009900 tx=0x7f6d9c02a410 comp rx=0 tx=0).stop 2024-09-11T01:19:08.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 -- 172.21.15.78:0/2024976687 shutdown_connections 2024-09-11T01:19:08.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 --2- 172.21.15.78:0/2024976687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac108020 0x7f6dac10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 --2- 172.21.15.78:0/2024976687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac1076a0 0x7f6dac107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 --2- 172.21.15.78:0/2024976687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac101c90 0x7f6dac102070 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 -- 172.21.15.78:0/2024976687 >> 172.21.15.78:0/2024976687 conn(0x7f6dac0fd700 msgr2=0x7f6dac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:08.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 -- 172.21.15.78:0/2024976687 shutdown_connections 2024-09-11T01:19:08.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 -- 172.21.15.78:0/2024976687 wait complete. 2024-09-11T01:19:08.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 Processor -- start 2024-09-11T01:19:08.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 -- start start 2024-09-11T01:19:08.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac101c90 0x7f6dac197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac1076a0 0x7f6dac198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac108020 0x7f6dac198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac111ce0 con 0x7f6dac108020 2024-09-11T01:19:08.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac111b60 con 0x7f6dac1076a0 2024-09-11T01:19:08.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.630+0000 7f6db23ef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac111e60 con 0x7f6dac101c90 2024-09-11T01:19:08.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6db098c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac108020 0x7f6dac198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6db098c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac108020 0x7f6dac198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57052/0 (socket says 172.21.15.78:57052) 2024-09-11T01:19:08.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6db098c640 1 -- 172.21.15.78:0/1516998371 learned_addr learned my addr 172.21.15.78:0/1516998371 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:08.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6da3fff640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac1076a0 0x7f6dac198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6dabfff640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac101c90 0x7f6dac197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6db098c640 1 -- 172.21.15.78:0/1516998371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac101c90 msgr2=0x7f6dac197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:08.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6db098c640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac101c90 0x7f6dac197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6db098c640 1 -- 172.21.15.78:0/1516998371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac1076a0 msgr2=0x7f6dac198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6db098c640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac1076a0 0x7f6dac198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6db098c640 1 -- 172.21.15.78:0/1516998371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dac1a76e0 con 0x7f6dac108020 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6da3fff640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac1076a0 0x7f6dac198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6db098c640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac108020 0x7f6dac198980 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f6da400b970 tx=0x7f6da400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6da9ffb640 1 -- 172.21.15.78:0/1516998371 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4019070 con 0x7f6dac108020 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6da9ffb640 1 -- 172.21.15.78:0/1516998371 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da4014440 con 0x7f6dac108020 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6da9ffb640 1 -- 172.21.15.78:0/1516998371 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da40075d0 con 0x7f6dac108020 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6db23ef640 1 -- 172.21.15.78:0/1516998371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dac1a79d0 con 0x7f6dac108020 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.631+0000 7f6db23ef640 1 -- 172.21.15.78:0/1516998371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dac1a7f40 con 0x7f6dac108020 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6da9ffb640 1 -- 172.21.15.78:0/1516998371 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6da400cb40 con 0x7f6dac108020 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6db47c4740 1 Processor -- start 2024-09-11T01:19:08.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6db47c4740 1 -- start start 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6db47c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 0x5621b9a074d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6db47c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a13460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6db47c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a139a0 0x5621b9a15d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6db47c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5621b997dea0 con 0x5621b9a07a10 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6db47c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5621b996d2d0 con 0x5621b9a070f0 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6db47c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5621b99710e0 con 0x5621b9a139a0 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6dabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a13460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6dabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a13460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57066/0 (socket says 172.21.15.78:57066) 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6dabfff640 1 -- 172.21.15.78:0/1171224391 learned_addr learned my addr 172.21.15.78:0/1171224391 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6da3fff640 1 --2- 172.21.15.78:0/1171224391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a139a0 0x5621b9a15d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.634+0000 7f6da3fff640 1 --2- 172.21.15.78:0/1171224391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 0x5621b9a074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.635+0000 7f6da9ffb640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d980b5450 0x7f6d980b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.635+0000 7f6dabfff640 1 -- 172.21.15.78:0/1171224391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a139a0 msgr2=0x5621b9a15d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:08.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.635+0000 7f6dabfff640 1 --2- 172.21.15.78:0/1171224391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a139a0 0x5621b9a15d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.635+0000 7f6dabfff640 1 -- 172.21.15.78:0/1171224391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 msgr2=0x5621b9a074d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.635+0000 7f6dabfff640 1 --2- 172.21.15.78:0/1171224391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 0x5621b9a074d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.635+0000 7f6dabfff640 1 -- 172.21.15.78:0/1171224391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b96cb650 con 0x5621b9a07a10 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.635+0000 7f6db098c640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d980b5450 0x7f6d980b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.635+0000 7f6dabfff640 1 --2- 172.21.15.78:0/1171224391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a13460 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c02a930 tx=0x7f6d9c031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f03e17cd3837cc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.635+0000 7f6da9ffb640 1 -- 172.21.15.78:0/1516998371 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f6da40e5810 con 0x7f6dac108020 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.635+0000 7f6da27fc640 1 -- 172.21.15.78:0/1171224391 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c03f070 con 0x5621b9a07a10 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.635+0000 7f6da27fc640 1 -- 172.21.15.78:0/1171224391 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c031e40 con 0x5621b9a07a10 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.635+0000 7f6da27fc640 1 -- 172.21.15.78:0/1171224391 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c035070 con 0x5621b9a07a10 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.636+0000 7f6db47c4740 1 -- 172.21.15.78:0/1171224391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 msgr2=0x5621b9a13460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.636+0000 7f6db098c640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d980b5450 0x7f6d980b7910 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6da40e3050 tx=0x7f6da400e040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.636+0000 7f6db47c4740 1 --2- 172.21.15.78:0/1171224391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a13460 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c02a930 tx=0x7f6d9c031cd0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.636+0000 7f6db47c4740 1 -- 172.21.15.78:0/1171224391 shutdown_connections 2024-09-11T01:19:08.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.636+0000 7f6db47c4740 1 --2- 172.21.15.78:0/1171224391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a139a0 0x5621b9a15d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.636+0000 7f6db47c4740 1 --2- 172.21.15.78:0/1171224391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a13460 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.636+0000 7f6db47c4740 1 --2- 172.21.15.78:0/1171224391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 0x5621b9a074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.636+0000 7f6db47c4740 1 -- 172.21.15.78:0/1171224391 >> 172.21.15.78:0/1171224391 conn(0x5621b9a04f20 msgr2=0x5621b9a060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.636+0000 7f6db47c4740 1 -- 172.21.15.78:0/1171224391 shutdown_connections 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.636+0000 7f6db47c4740 1 -- 172.21.15.78:0/1171224391 wait complete. 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.636+0000 7f6db47c4740 1 -- start start 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.636+0000 7f6db47c4740 1 Processor -- start 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6db47c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 0x5621b9a13270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6db47c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a07a10 0x5621b9a106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6db47c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a139a0 0x5621b9a10ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6db47c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5621b97ce700 con 0x5621b9a139a0 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6db47c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5621b9908160 con 0x5621b9a070f0 2024-09-11T01:19:08.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6db47c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5621b9a165c0 con 0x5621b9a07a10 2024-09-11T01:19:08.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6db47c4740 10 client.0 _ll_register_callbacks cb 0x5621b99219e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:08.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6da3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a139a0 0x5621b9a10ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6da3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a139a0 0x5621b9a10ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57080/0 (socket says 172.21.15.78:57080) 2024-09-11T01:19:08.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6da3fff640 1 -- 172.21.15.78:0/3797144851 learned_addr learned my addr 172.21.15.78:0/3797144851 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:08.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6dabfff640 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a07a10 0x5621b9a106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6da3fff640 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 0x5621b9a13270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6da3fff640 1 -- 172.21.15.78:0/3797144851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a07a10 msgr2=0x5621b9a106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:08.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6da3fff640 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a07a10 0x5621b9a106c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6da3fff640 1 -- 172.21.15.78:0/3797144851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 msgr2=0x5621b9a13270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:08.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6da3fff640 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 0x5621b9a13270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6da3fff640 1 -- 172.21.15.78:0/3797144851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b9935c90 con 0x5621b9a139a0 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6da3fff640 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a139a0 0x5621b9a10ca0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6d94014c90 tx=0x7f6d94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6114d737ba29485e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6d8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6d8b7fe640 1 -- 172.21.15.78:0/3797144851 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94004020 con 0x5621b9a139a0 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6d8b7fe640 1 -- 172.21.15.78:0/3797144851 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f6d940041c0 con 0x5621b9a139a0 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6db47c4740 1 -- 172.21.15.78:0/3797144851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621b997fdf0 con 0x5621b9a139a0 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.637+0000 7f6d8b7fe640 1 -- 172.21.15.78:0/3797144851 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f6d94027050 con 0x5621b9a139a0 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.638+0000 7f6db47c4740 10 client.18984 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.638+0000 7f6db47c4740 1 -- 172.21.15.78:0/3797144851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5621b9a21fb0 con 0x5621b9a139a0 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.638+0000 7f6d8b7fe640 1 -- 172.21.15.78:0/3797144851 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5621b997fdf0 con 0x5621b9a139a0 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.638+0000 7f6d8b7fe640 1 -- 172.21.15.78:0/3797144851 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f6d940035f0 con 0x5621b9a139a0 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.638+0000 7f6db47c4740 10 client.18984 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.638+0000 7f6db47c4740 10 client.18984 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.638+0000 7f6db47c4740 20 client.18984 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:08.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.638+0000 7f6db47c4740 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5621b9a23430 0x5621b9a25860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.638+0000 7f6db47c4740 4 client.18984 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.638+0000 7f6db47c4740 1 -- 172.21.15.78:0/3797144851 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621b997b8a0 con 0x5621b9a23430 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.638+0000 7f6dabfff640 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5621b9a23430 0x5621b9a25860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.639+0000 7f6d8b7fe640 10 client.18984 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.639+0000 7f6dabfff640 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5621b9a23430 0x5621b9a25860 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5be7aa751d32cb56 server_cookie=4d2784968646acbe in_seq=0 out_seq=0 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.639+0000 7f6d8b7fe640 1 -- 172.21.15.78:0/3797144851 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5621b997b8a0 con 0x5621b9a23430 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.639+0000 7f6d8b7fe640 10 client.18984 handle_command_reply: tid=0 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.639+0000 7f6db47c4740 1 client.18984 shutdown 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.639+0000 7f6db47c4740 1 -- 172.21.15.78:0/3797144851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a139a0 msgr2=0x5621b9a10ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.639+0000 7f6db47c4740 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a139a0 0x5621b9a10ca0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6d94014c90 tx=0x7f6d94004730 comp rx=0 tx=0).stop 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.640+0000 7f6db47c4740 1 -- 172.21.15.78:0/3797144851 shutdown_connections 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.640+0000 7f6db47c4740 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5621b9a23430 0x5621b9a25860 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.640+0000 7f6db47c4740 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a139a0 0x5621b9a10ca0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.640+0000 7f6db47c4740 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a07a10 0x5621b9a106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.640+0000 7f6db47c4740 1 --2- 172.21.15.78:0/3797144851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 0x5621b9a13270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.640+0000 7f6db47c4740 1 -- 172.21.15.78:0/3797144851 >> 172.21.15.78:0/3797144851 conn(0x5621b9a04f20 msgr2=0x5621b9a0aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:08.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.640+0000 7f6d8b7fe640 0 client.18984 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:19:08.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.640+0000 7f6db47c4740 1 -- 172.21.15.78:0/3797144851 shutdown_connections 2024-09-11T01:19:08.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.640+0000 7f6db47c4740 1 -- 172.21.15.78:0/3797144851 wait complete. 2024-09-11T01:19:08.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.640+0000 7f6db47c4740 20 client.18984 trim_cache size 0 max 16384 2024-09-11T01:19:08.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.653+0000 7f6db47c4740 1 Processor -- start 2024-09-11T01:19:08.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.653+0000 7f6db47c4740 1 -- start start 2024-09-11T01:19:08.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.653+0000 7f6db47c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 0x5621b9a1f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.653+0000 7f6db47c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a14d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.653+0000 7f6db47c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a139a0 0x5621b9a152d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.653+0000 7f6db47c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1b0800 con 0x5621b9a07a10 2024-09-11T01:19:08.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.653+0000 7f6db47c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1b2340 con 0x5621b9a070f0 2024-09-11T01:19:08.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.653+0000 7f6db47c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1b0ed0 con 0x5621b9a139a0 2024-09-11T01:19:08.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6dabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a14d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6dabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a14d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57084/0 (socket says 172.21.15.78:57084) 2024-09-11T01:19:08.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6dabfff640 1 -- 172.21.15.78:0/3159767246 learned_addr learned my addr 172.21.15.78:0/3159767246 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:08.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6da3fff640 1 --2- 172.21.15.78:0/3159767246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a139a0 0x5621b9a152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6dabfff640 1 -- 172.21.15.78:0/3159767246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a139a0 msgr2=0x5621b9a152d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:08.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6dabfff640 1 --2- 172.21.15.78:0/3159767246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a139a0 0x5621b9a152d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6dabfff640 1 -- 172.21.15.78:0/3159767246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 msgr2=0x5621b9a1f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:08.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6dabfff640 1 --2- 172.21.15.78:0/3159767246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 0x5621b9a1f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6dabfff640 1 -- 172.21.15.78:0/3159767246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b9a07840 con 0x5621b9a07a10 2024-09-11T01:19:08.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6da3fff640 1 --2- 172.21.15.78:0/3159767246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a139a0 0x5621b9a152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:19:08.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6dabfff640 1 --2- 172.21.15.78:0/3159767246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a14d90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009900 tx=0x7f6d9c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79138994ff0ec882 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:08.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6da1ffb640 1 -- 172.21.15.78:0/3159767246 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c035050 con 0x5621b9a07a10 2024-09-11T01:19:08.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6da1ffb640 1 -- 172.21.15.78:0/3159767246 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c004040 con 0x5621b9a07a10 2024-09-11T01:19:08.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.654+0000 7f6da1ffb640 1 -- 172.21.15.78:0/3159767246 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c03f040 con 0x5621b9a07a10 2024-09-11T01:19:08.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.655+0000 7f6db47c4740 1 -- 172.21.15.78:0/3159767246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 msgr2=0x5621b9a14d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:08.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.655+0000 7f6db47c4740 1 --2- 172.21.15.78:0/3159767246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a14d90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009900 tx=0x7f6d9c02a410 comp rx=0 tx=0).stop 2024-09-11T01:19:08.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.655+0000 7f6db47c4740 1 -- 172.21.15.78:0/3159767246 shutdown_connections 2024-09-11T01:19:08.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.655+0000 7f6db47c4740 1 --2- 172.21.15.78:0/3159767246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a139a0 0x5621b9a152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.655+0000 7f6db47c4740 1 --2- 172.21.15.78:0/3159767246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a14d90 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.655+0000 7f6db47c4740 1 --2- 172.21.15.78:0/3159767246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a070f0 0x5621b9a1f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.655+0000 7f6db47c4740 1 -- 172.21.15.78:0/3159767246 >> 172.21.15.78:0/3159767246 conn(0x5621b9a04f20 msgr2=0x5621b9a0b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.655+0000 7f6db47c4740 1 -- 172.21.15.78:0/3159767246 shutdown_connections 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.655+0000 7f6db47c4740 1 -- 172.21.15.78:0/3159767246 wait complete. 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.655+0000 7f6db47c4740 1 -- start start 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.655+0000 7f6db47c4740 1 Processor -- start 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6db47c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a070f0 0x5621b9a1b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6db47c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a1be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6db47c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a139a0 0x5621b9a16890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6db47c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1b1c70 con 0x5621b9a07a10 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6db47c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1b15a0 con 0x5621b9a139a0 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6db47c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac1ae800 con 0x5621b9a070f0 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6db47c4740 10 client.0 _ll_register_callbacks cb 0x7f6d940148d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6dabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a1be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6dabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a1be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57086/0 (socket says 172.21.15.78:57086) 2024-09-11T01:19:08.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6dabfff640 1 -- 172.21.15.78:0/948137409 learned_addr learned my addr 172.21.15.78:0/948137409 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6da3fff640 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a139a0 0x5621b9a16890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6da3fff640 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a070f0 0x5621b9a1b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6dabfff640 1 -- 172.21.15.78:0/948137409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a070f0 msgr2=0x5621b9a1b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6dabfff640 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a070f0 0x5621b9a1b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6dabfff640 1 -- 172.21.15.78:0/948137409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a139a0 msgr2=0x5621b9a16890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6dabfff640 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a139a0 0x5621b9a16890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6dabfff640 1 -- 172.21.15.78:0/948137409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b9a16dd0 con 0x5621b9a07a10 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6da3fff640 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a070f0 0x5621b9a1b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6dabfff640 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a1be20 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c000c00 tx=0x7f6d9c004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf01eaa3d1a7c1f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6d8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6d8b7fe640 1 -- 172.21.15.78:0/948137409 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c03f070 con 0x5621b9a07a10 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6d8b7fe640 1 -- 172.21.15.78:0/948137409 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f6d9c0314a0 con 0x5621b9a07a10 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.656+0000 7f6db47c4740 1 -- 172.21.15.78:0/948137409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d9c004040 con 0x5621b9a07a10 2024-09-11T01:19:08.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.657+0000 7f6d8b7fe640 1 -- 172.21.15.78:0/948137409 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f6d9c060050 con 0x5621b9a07a10 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.657+0000 7f6db47c4740 10 client.18990 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.657+0000 7f6db47c4740 1 -- 172.21.15.78:0/948137409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5621b9a90a80 con 0x5621b9a07a10 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.657+0000 7f6d8b7fe640 1 -- 172.21.15.78:0/948137409 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c004040 con 0x5621b9a07a10 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.657+0000 7f6d8b7fe640 1 -- 172.21.15.78:0/948137409 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f6d9c03b430 con 0x5621b9a07a10 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.657+0000 7f6db47c4740 10 client.18990 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.657+0000 7f6db47c4740 10 client.18990 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.657+0000 7f6db47c4740 20 client.18990 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.657+0000 7f6db47c4740 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5621b9a90d00 0x5621b9a930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.657+0000 7f6db47c4740 4 client.18990 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.657+0000 7f6db47c4740 1 -- 172.21.15.78:0/948137409 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621b9978cd0 con 0x5621b9a90d00 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.657+0000 7f6da3fff640 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5621b9a90d00 0x5621b9a930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6d8b7fe640 10 client.18990 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:19:08.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6da3fff640 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5621b9a90d00 0x5621b9a930f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53084d01bfcfa459 server_cookie=fa25cf1827eb394f in_seq=0 out_seq=0 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6d8b7fe640 1 -- 172.21.15.78:0/948137409 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621b9978cd0 con 0x5621b9a90d00 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6d8b7fe640 10 client.18990 handle_command_reply: tid=0 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6db47c4740 1 client.18990 shutdown 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6db47c4740 1 -- 172.21.15.78:0/948137409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 msgr2=0x5621b9a1be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6db47c4740 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a1be20 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c000c00 tx=0x7f6d9c004280 comp rx=0 tx=0).stop 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6db47c4740 1 -- 172.21.15.78:0/948137409 shutdown_connections 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6db47c4740 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5621b9a139a0 0x5621b9a16890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6db47c4740 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x5621b9a90d00 0x5621b9a930f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6db47c4740 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621b9a07a10 0x5621b9a1be20 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6d8b7fe640 0 client.18990 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6db47c4740 1 --2- 172.21.15.78:0/948137409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5621b9a070f0 0x5621b9a1b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6db47c4740 1 -- 172.21.15.78:0/948137409 >> 172.21.15.78:0/948137409 conn(0x5621b9a04f20 msgr2=0x5621b9a11cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6db47c4740 1 -- 172.21.15.78:0/948137409 shutdown_connections 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.658+0000 7f6db47c4740 1 -- 172.21.15.78:0/948137409 wait complete. 2024-09-11T01:19:08.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db47c4740 20 client.18990 trim_cache size 0 max 16384 2024-09-11T01:19:08.667 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:19:08.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db23ef640 1 -- 172.21.15.78:0/1516998371 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d980b5450 msgr2=0x7f6d980b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:08.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db23ef640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d980b5450 0x7f6d980b7910 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6da40e3050 tx=0x7f6da400e040 comp rx=0 tx=0).stop 2024-09-11T01:19:08.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db23ef640 1 -- 172.21.15.78:0/1516998371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac108020 msgr2=0x7f6dac198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:08.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db23ef640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac108020 0x7f6dac198980 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f6da400b970 tx=0x7f6da400be30 comp rx=0 tx=0).stop 2024-09-11T01:19:08.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db23ef640 1 -- 172.21.15.78:0/1516998371 shutdown_connections 2024-09-11T01:19:08.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db23ef640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6d980b5450 0x7f6d980b7910 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db23ef640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dac108020 0x7f6dac198980 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db23ef640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6dac1076a0 0x7f6dac198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db23ef640 1 --2- 172.21.15.78:0/1516998371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dac101c90 0x7f6dac197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:08.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db23ef640 1 -- 172.21.15.78:0/1516998371 >> 172.21.15.78:0/1516998371 conn(0x7f6dac0fd700 msgr2=0x7f6dac0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:08.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db23ef640 1 -- 172.21.15.78:0/1516998371 shutdown_connections 2024-09-11T01:19:08.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:08.659+0000 7f6db23ef640 1 -- 172.21.15.78:0/1516998371 wait complete. 2024-09-11T01:19:08.796 INFO:tasks.workunit.client.0.smithi078.stdout: 10 275627 33.54 MB/sec execute 314 sec latency 3978.307 ms 2024-09-11T01:19:09.452 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:19:09.796 INFO:tasks.workunit.client.0.smithi078.stdout: 10 275865 33.48 MB/sec execute 315 sec latency 4713.837 ms 2024-09-11T01:19:09.815 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:19:10.025 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:09 smithi112 ceph-mon[35384]: pgmap v957: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 488 op/s 2024-09-11T01:19:10.082 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:09 smithi078 ceph-mon[31315]: pgmap v957: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 488 op/s 2024-09-11T01:19:10.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:09 smithi110 ceph-mon[34712]: pgmap v957: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 488 op/s 2024-09-11T01:19:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.394+0000 7f50ae33b640 1 -- 172.21.15.78:0/3200994202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50a8109160 msgr2=0x7f50a810f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.394+0000 7f50ae33b640 1 --2- 172.21.15.78:0/3200994202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50a8109160 0x7f50a810f960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009ca0 tx=0x7f50a002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.394+0000 7f50ae33b640 1 -- 172.21.15.78:0/3200994202 shutdown_connections 2024-09-11T01:19:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.394+0000 7f50ae33b640 1 --2- 172.21.15.78:0/3200994202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50a8109160 0x7f50a810f960 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.394+0000 7f50ae33b640 1 --2- 172.21.15.78:0/3200994202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a81087e0 0x7f50a8108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.395+0000 7f50ae33b640 1 --2- 172.21.15.78:0/3200994202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50a8102dd0 0x7f50a81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.395+0000 7f50ae33b640 1 -- 172.21.15.78:0/3200994202 >> 172.21.15.78:0/3200994202 conn(0x7f50a80fe800 msgr2=0x7f50a8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.395+0000 7f50ae33b640 1 -- 172.21.15.78:0/3200994202 shutdown_connections 2024-09-11T01:19:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.395+0000 7f50ae33b640 1 -- 172.21.15.78:0/3200994202 wait complete. 2024-09-11T01:19:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.395+0000 7f50ae33b640 1 Processor -- start 2024-09-11T01:19:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.395+0000 7f50ae33b640 1 -- start start 2024-09-11T01:19:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50ae33b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50a8102dd0 0x7f50a819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50ae33b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a81087e0 0x7f50a819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50ae33b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50a8109160 0x7f50a81a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50ae33b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50a8112e70 con 0x7f50a81087e0 2024-09-11T01:19:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50ae33b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50a8112cf0 con 0x7f50a8102dd0 2024-09-11T01:19:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50ae33b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50a8112ff0 con 0x7f50a8109160 2024-09-11T01:19:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a81087e0 0x7f50a819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50a8102dd0 0x7f50a819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a81087e0 0x7f50a819b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57096/0 (socket says 172.21.15.78:57096) 2024-09-11T01:19:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50a77fe640 1 -- 172.21.15.78:0/26979789 learned_addr learned my addr 172.21.15.78:0/26979789 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50ac8d8640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50a8109160 0x7f50a81a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50a7fff640 1 -- 172.21.15.78:0/26979789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50a8109160 msgr2=0x7f50a81a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.396+0000 7f50a7fff640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50a8109160 0x7f50a81a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.397+0000 7f50a7fff640 1 -- 172.21.15.78:0/26979789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a81087e0 msgr2=0x7f50a819b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.397+0000 7f50a7fff640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a81087e0 0x7f50a819b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.397+0000 7f50a7fff640 1 -- 172.21.15.78:0/26979789 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a81a1500 con 0x7f50a8102dd0 2024-09-11T01:19:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.397+0000 7f50a77fe640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a81087e0 0x7f50a819b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:19:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.397+0000 7f50a7fff640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50a8102dd0 0x7f50a819b450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f509800e9f0 tx=0x7f509800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.397+0000 7f50a57fa640 1 -- 172.21.15.78:0/26979789 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509800ccc0 con 0x7f50a8102dd0 2024-09-11T01:19:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.397+0000 7f50a57fa640 1 -- 172.21.15.78:0/26979789 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5098005020 con 0x7f50a8102dd0 2024-09-11T01:19:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.397+0000 7f50ae33b640 1 -- 172.21.15.78:0/26979789 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a81a17f0 con 0x7f50a8102dd0 2024-09-11T01:19:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.398+0000 7f50a57fa640 1 -- 172.21.15.78:0/26979789 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098004a00 con 0x7f50a8102dd0 2024-09-11T01:19:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.398+0000 7f50ae33b640 1 -- 172.21.15.78:0/26979789 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a81a7eb0 con 0x7f50a8102dd0 2024-09-11T01:19:10.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.401+0000 7f50b0710740 1 Processor -- start 2024-09-11T01:19:10.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.401+0000 7f50a57fa640 1 -- 172.21.15.78:0/26979789 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5098004ba0 con 0x7f50a8102dd0 2024-09-11T01:19:10.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.401+0000 7f50b0710740 1 -- start start 2024-09-11T01:19:10.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50b0710740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e577ce0 0x56294e5780c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50b0710740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e578600 0x56294e584050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50b0710740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e584590 0x56294e586980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50b0710740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56294e4e5ea0 con 0x56294e578600 2024-09-11T01:19:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50b0710740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56294e4d52d0 con 0x56294e584590 2024-09-11T01:19:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50b0710740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56294e4d90e0 con 0x56294e577ce0 2024-09-11T01:19:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50a77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e584590 0x56294e586980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e578600 0x56294e584050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50a77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e584590 0x56294e586980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43828/0 (socket says 172.21.15.78:43828) 2024-09-11T01:19:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50a77fe640 1 -- 172.21.15.78:0/2621102200 learned_addr learned my addr 172.21.15.78:0/2621102200 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50ac8d8640 1 --2- 172.21.15.78:0/2621102200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e577ce0 0x56294e5780c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50a7fff640 1 -- 172.21.15.78:0/2621102200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e577ce0 msgr2=0x56294e5780c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50a7fff640 1 --2- 172.21.15.78:0/2621102200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e577ce0 0x56294e5780c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.402+0000 7f50a7fff640 1 -- 172.21.15.78:0/2621102200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e584590 msgr2=0x56294e586980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.403+0000 7f50a7fff640 1 --2- 172.21.15.78:0/2621102200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e584590 0x56294e586980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.403+0000 7f50a7fff640 1 -- 172.21.15.78:0/2621102200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56294e233650 con 0x56294e578600 2024-09-11T01:19:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.403+0000 7f50a7fff640 1 --2- 172.21.15.78:0/2621102200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e578600 0x56294e584050 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f50980e36c0 tx=0x7f50980ee220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc5b2d63baa3724 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.403+0000 7f50a57fa640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50900b5550 0x7f50900b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.404+0000 7f50ac8d8640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50900b5550 0x7f50900b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.404+0000 7f50a57fa640 1 -- 172.21.15.78:0/26979789 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f50980e5fc0 con 0x7f50a8102dd0 2024-09-11T01:19:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.404+0000 7f508e7fc640 1 -- 172.21.15.78:0/2621102200 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50980f9070 con 0x56294e578600 2024-09-11T01:19:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.405+0000 7f508e7fc640 1 -- 172.21.15.78:0/2621102200 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50980abf30 con 0x56294e578600 2024-09-11T01:19:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.405+0000 7f50ac8d8640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50900b5550 0x7f50900b7a10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f50a0005fe0 tx=0x7f50a0004a40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:10.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.406+0000 7f508e7fc640 1 -- 172.21.15.78:0/2621102200 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50980ac1f0 con 0x56294e578600 2024-09-11T01:19:10.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.406+0000 7f50b0710740 1 -- 172.21.15.78:0/2621102200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e578600 msgr2=0x56294e584050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.406+0000 7f50b0710740 1 --2- 172.21.15.78:0/2621102200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e578600 0x56294e584050 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f50980e36c0 tx=0x7f50980ee220 comp rx=0 tx=0).stop 2024-09-11T01:19:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.406+0000 7f50b0710740 1 -- 172.21.15.78:0/2621102200 shutdown_connections 2024-09-11T01:19:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.406+0000 7f50b0710740 1 --2- 172.21.15.78:0/2621102200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e584590 0x56294e586980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.406+0000 7f50b0710740 1 --2- 172.21.15.78:0/2621102200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e578600 0x56294e584050 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.406+0000 7f50b0710740 1 --2- 172.21.15.78:0/2621102200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e577ce0 0x56294e5780c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.406+0000 7f50b0710740 1 -- 172.21.15.78:0/2621102200 >> 172.21.15.78:0/2621102200 conn(0x56294e575b10 msgr2=0x56294e576cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:10.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.408+0000 7f50b0710740 1 -- 172.21.15.78:0/2621102200 shutdown_connections 2024-09-11T01:19:10.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.408+0000 7f50b0710740 1 -- 172.21.15.78:0/2621102200 wait complete. 2024-09-11T01:19:10.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.409+0000 7f50b0710740 1 -- start start 2024-09-11T01:19:10.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.409+0000 7f50b0710740 1 Processor -- start 2024-09-11T01:19:10.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.410+0000 7f50b0710740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 0x56294e583d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.410+0000 7f50b0710740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e578600 0x56294e5811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.411+0000 7f50a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 0x56294e583d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.411+0000 7f50a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 0x56294e583d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57104/0 (socket says 172.21.15.78:57104) 2024-09-11T01:19:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.411+0000 7f50b0710740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e584590 0x56294e5816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.411+0000 7f50b0710740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56294e336700 con 0x56294e577ce0 2024-09-11T01:19:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.411+0000 7f50b0710740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56294e470160 con 0x56294e584590 2024-09-11T01:19:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.411+0000 7f50b0710740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56294e5872e0 con 0x56294e578600 2024-09-11T01:19:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.411+0000 7f50a7fff640 1 -- 172.21.15.78:0/2250290489 learned_addr learned my addr 172.21.15.78:0/2250290489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.411+0000 7f50b0710740 10 client.0 _ll_register_callbacks cb 0x56294e4899e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.411+0000 7f50ac8d8640 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e584590 0x56294e5816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.412+0000 7f50a77fe640 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e578600 0x56294e5811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.412+0000 7f50a7fff640 1 -- 172.21.15.78:0/2250290489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e578600 msgr2=0x56294e5811b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.412+0000 7f50a7fff640 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e578600 0x56294e5811b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.412+0000 7f50a7fff640 1 -- 172.21.15.78:0/2250290489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e584590 msgr2=0x56294e5816f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.412+0000 7f50a7fff640 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e584590 0x56294e5816f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.412+0000 7f50a7fff640 1 -- 172.21.15.78:0/2250290489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56294e49dc90 con 0x56294e577ce0 2024-09-11T01:19:10.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.412+0000 7f50877fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:10.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.412+0000 7f50a7fff640 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 0x56294e583d60 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f509801a040 tx=0x7f50980e36c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251abb9e67bcbf07 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:10.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.412+0000 7f50877fe640 1 -- 172.21.15.78:0/2250290489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509801c030 con 0x56294e577ce0 2024-09-11T01:19:10.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.412+0000 7f50b0710740 1 -- 172.21.15.78:0/2250290489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50980abf30 con 0x56294e577ce0 2024-09-11T01:19:10.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.412+0000 7f50877fe640 1 -- 172.21.15.78:0/2250290489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f50980cde00 con 0x56294e577ce0 2024-09-11T01:19:10.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.413+0000 7f50877fe640 1 -- 172.21.15.78:0/2250290489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f509801c030 con 0x56294e577ce0 2024-09-11T01:19:10.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.413+0000 7f50b0710740 10 client.19002 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:10.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.413+0000 7f50b0710740 1 -- 172.21.15.78:0/2250290489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x56294e592b50 con 0x56294e577ce0 2024-09-11T01:19:10.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.413+0000 7f50877fe640 1 -- 172.21.15.78:0/2250290489 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50980abf30 con 0x56294e577ce0 2024-09-11T01:19:10.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.413+0000 7f50877fe640 1 -- 172.21.15.78:0/2250290489 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f50980a8690 con 0x56294e577ce0 2024-09-11T01:19:10.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.413+0000 7f50b0710740 10 client.19002 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.413+0000 7f50b0710740 10 client.19002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.413+0000 7f50b0710740 20 client.19002 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.413+0000 7f50b0710740 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56294e593fb0 0x56294e596450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.413+0000 7f50b0710740 4 client.19002 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.413+0000 7f50b0710740 1 -- 172.21.15.78:0/2250290489 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56294e4e7df0 con 0x56294e593fb0 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.414+0000 7f50a77fe640 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56294e593fb0 0x56294e596450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.414+0000 7f50877fe640 10 client.19002 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.414+0000 7f50a77fe640 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56294e593fb0 0x56294e596450 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c74b432d1f24ceec server_cookie=ab08cbbff0d0e221 in_seq=0 out_seq=0 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.415+0000 7f50877fe640 1 -- 172.21.15.78:0/2250290489 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56294e4e7df0 con 0x56294e593fb0 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.415+0000 7f50877fe640 10 client.19002 handle_command_reply: tid=0 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.415+0000 7f50b0710740 1 client.19002 shutdown 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.415+0000 7f50b0710740 1 -- 172.21.15.78:0/2250290489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 msgr2=0x56294e583d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.416+0000 7f50b0710740 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 0x56294e583d60 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f509801a040 tx=0x7f50980e36c0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.416+0000 7f50b0710740 1 -- 172.21.15.78:0/2250290489 shutdown_connections 2024-09-11T01:19:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.416+0000 7f50b0710740 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e584590 0x56294e5816f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.416+0000 7f50b0710740 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e578600 0x56294e5811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.416+0000 7f50b0710740 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56294e593fb0 0x56294e596450 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.416+0000 7f50877fe640 0 client.19002 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.416+0000 7f50b0710740 1 --2- 172.21.15.78:0/2250290489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 0x56294e583d60 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.416+0000 7f50b0710740 1 -- 172.21.15.78:0/2250290489 >> 172.21.15.78:0/2250290489 conn(0x56294e575b10 msgr2=0x56294e56c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.416+0000 7f50b0710740 1 -- 172.21.15.78:0/2250290489 shutdown_connections 2024-09-11T01:19:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.416+0000 7f50b0710740 1 -- 172.21.15.78:0/2250290489 wait complete. 2024-09-11T01:19:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.417+0000 7f50b0710740 20 client.19002 trim_cache size 0 max 16384 2024-09-11T01:19:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.433+0000 7f50b0710740 1 Processor -- start 2024-09-11T01:19:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.433+0000 7f50b0710740 1 -- start start 2024-09-11T01:19:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.434+0000 7f50b0710740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 0x56294e56c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.434+0000 7f50b0710740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e578600 0x56294e57e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.434+0000 7f50b0710740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e584590 0x56294e57e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.434+0000 7f50b0710740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81ac660 con 0x56294e577ce0 2024-09-11T01:19:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.434+0000 7f50b0710740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81ae1a0 con 0x56294e578600 2024-09-11T01:19:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.434+0000 7f50b0710740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81acd30 con 0x56294e584590 2024-09-11T01:19:10.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.434+0000 7f50ac8d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e584590 0x56294e57e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.434+0000 7f50ac8d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e584590 0x56294e57e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:45454/0 (socket says 172.21.15.78:45454) 2024-09-11T01:19:10.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.434+0000 7f50ac8d8640 1 -- 172.21.15.78:0/287494150 learned_addr learned my addr 172.21.15.78:0/287494150 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:10.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.435+0000 7f50a7fff640 1 --2- 172.21.15.78:0/287494150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 0x56294e56c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.435+0000 7f50ac8d8640 1 -- 172.21.15.78:0/287494150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e578600 msgr2=0x56294e57e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:19:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.435+0000 7f50ac8d8640 1 --2- 172.21.15.78:0/287494150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e578600 0x56294e57e3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.435+0000 7f50ac8d8640 1 -- 172.21.15.78:0/287494150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 msgr2=0x56294e56c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.435+0000 7f50ac8d8640 1 --2- 172.21.15.78:0/287494150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 0x56294e56c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.435+0000 7f50ac8d8640 1 -- 172.21.15.78:0/287494150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56294e578430 con 0x56294e584590 2024-09-11T01:19:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.435+0000 7f50a7fff640 1 --2- 172.21.15.78:0/287494150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 0x56294e56c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:19:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.435+0000 7f50ac8d8640 1 --2- 172.21.15.78:0/287494150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e584590 0x56294e57e900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f50a0035000 tx=0x7f50a0040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=933abe0b1e3cf2be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.436+0000 7f508dffb640 1 -- 172.21.15.78:0/287494150 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0032e30 con 0x56294e584590 2024-09-11T01:19:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.436+0000 7f508dffb640 1 -- 172.21.15.78:0/287494150 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a0036040 con 0x56294e584590 2024-09-11T01:19:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.436+0000 7f508dffb640 1 -- 172.21.15.78:0/287494150 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a002f610 con 0x56294e584590 2024-09-11T01:19:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.437+0000 7f50b0710740 1 -- 172.21.15.78:0/287494150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e584590 msgr2=0x56294e57e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.437+0000 7f50b0710740 1 --2- 172.21.15.78:0/287494150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e584590 0x56294e57e900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f50a0035000 tx=0x7f50a0040000 comp rx=0 tx=0).stop 2024-09-11T01:19:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.437+0000 7f50b0710740 1 -- 172.21.15.78:0/287494150 shutdown_connections 2024-09-11T01:19:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.437+0000 7f50b0710740 1 --2- 172.21.15.78:0/287494150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e584590 0x56294e57e900 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.437+0000 7f50b0710740 1 --2- 172.21.15.78:0/287494150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e578600 0x56294e57e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.437+0000 7f50b0710740 1 --2- 172.21.15.78:0/287494150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e577ce0 0x56294e56c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.437+0000 7f50b0710740 1 -- 172.21.15.78:0/287494150 >> 172.21.15.78:0/287494150 conn(0x56294e575b10 msgr2=0x56294e576f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.438+0000 7f50b0710740 1 -- 172.21.15.78:0/287494150 shutdown_connections 2024-09-11T01:19:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.438+0000 7f50b0710740 1 -- 172.21.15.78:0/287494150 wait complete. 2024-09-11T01:19:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.438+0000 7f50b0710740 1 -- start start 2024-09-11T01:19:10.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.438+0000 7f50b0710740 1 Processor -- start 2024-09-11T01:19:10.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.439+0000 7f50b0710740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e577ce0 0x56294e588e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.439+0000 7f50b0710740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e578600 0x56294e5893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.439+0000 7f50b0710740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e584590 0x56294e5fff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.439+0000 7f50a77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e578600 0x56294e5893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.439+0000 7f50b0710740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81adad0 con 0x56294e584590 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.439+0000 7f50b0710740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81ad400 con 0x56294e578600 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.439+0000 7f50a77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e578600 0x56294e5893c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43858/0 (socket says 172.21.15.78:43858) 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.439+0000 7f50a77fe640 1 -- 172.21.15.78:0/2152047736 learned_addr learned my addr 172.21.15.78:0/2152047736 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.439+0000 7f50a7fff640 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e577ce0 0x56294e588e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.439+0000 7f50b0710740 1 -- 172.21.15.78:0/2152047736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81aa620 con 0x56294e577ce0 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.439+0000 7f50b0710740 10 client.0 _ll_register_callbacks cb 0x7f50980f7070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.439+0000 7f50ac8d8640 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e584590 0x56294e5fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.440+0000 7f50a77fe640 1 -- 172.21.15.78:0/2152047736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e577ce0 msgr2=0x56294e588e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.440+0000 7f50a77fe640 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e577ce0 0x56294e588e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.440+0000 7f50a77fe640 1 -- 172.21.15.78:0/2152047736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e584590 msgr2=0x56294e5fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.440+0000 7f50a77fe640 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e584590 0x56294e5fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.440+0000 7f50a77fe640 1 -- 172.21.15.78:0/2152047736 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56294e600660 con 0x56294e578600 2024-09-11T01:19:10.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.441+0000 7f50a77fe640 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e578600 0x56294e5893c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f509c009fd0 tx=0x7f509c012b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d435ebd4f061ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:10.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.441+0000 7f50877fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:19:10.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.441+0000 7f50877fe640 1 -- 172.21.15.78:0/2152047736 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c028060 con 0x56294e578600 2024-09-11T01:19:10.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.441+0000 7f50877fe640 1 -- 172.21.15.78:0/2152047736 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8449+0+0 (secure 0 0 0) 0x7f509c0122d0 con 0x56294e578600 2024-09-11T01:19:10.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.441+0000 7f50b0710740 1 -- 172.21.15.78:0/2152047736 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50a0036040 con 0x56294e578600 2024-09-11T01:19:10.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.442+0000 7f50877fe640 1 -- 172.21.15.78:0/2152047736 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f509c004030 con 0x56294e578600 2024-09-11T01:19:10.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.442+0000 7f50b0710740 10 client.28073 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:10.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.442+0000 7f50b0710740 1 -- 172.21.15.78:0/2152047736 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x56294e56ccc0 con 0x56294e578600 2024-09-11T01:19:10.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.442+0000 7f50877fe640 1 -- 172.21.15.78:0/2152047736 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f509c00fda0 con 0x56294e578600 2024-09-11T01:19:10.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.442+0000 7f50877fe640 1 -- 172.21.15.78:0/2152047736 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f509c003c00 con 0x56294e578600 2024-09-11T01:19:10.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.442+0000 7f50b0710740 10 client.28073 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:10.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.443+0000 7f50b0710740 10 client.28073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:19:10.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.443+0000 7f50b0710740 20 client.28073 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:10.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.443+0000 7f50b0710740 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56294e593fb0 0x56294e56f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:10.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.443+0000 7f50b0710740 4 client.28073 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:19:10.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.443+0000 7f50b0710740 1 -- 172.21.15.78:0/2152047736 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f50a0036040 con 0x56294e593fb0 2024-09-11T01:19:10.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.443+0000 7f50a7fff640 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56294e593fb0 0x56294e56f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:10.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.444+0000 7f50877fe640 10 client.28073 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:10.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.444+0000 7f50a7fff640 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56294e593fb0 0x56294e56f4b0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d281cfe846ed7b20 server_cookie=cdb46f7a4b374c67 in_seq=0 out_seq=0 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.444+0000 7f50877fe640 1 -- 172.21.15.78:0/2152047736 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f50a0036040 con 0x56294e593fb0 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.444+0000 7f50877fe640 10 client.28073 handle_command_reply: tid=0 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.444+0000 7f50b0710740 1 client.28073 shutdown 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.445+0000 7f50b0710740 1 -- 172.21.15.78:0/2152047736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e578600 msgr2=0x56294e5893c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.445+0000 7f50b0710740 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e578600 0x56294e5893c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f509c009fd0 tx=0x7f509c012b70 comp rx=0 tx=0).stop 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.445+0000 7f50b0710740 1 -- 172.21.15.78:0/2152047736 shutdown_connections 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.445+0000 7f50b0710740 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56294e593fb0 0x56294e56f4b0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.445+0000 7f50b0710740 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56294e584590 0x56294e5fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.445+0000 7f50b0710740 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56294e578600 0x56294e5893c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.445+0000 7f50b0710740 1 --2- 172.21.15.78:0/2152047736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56294e577ce0 0x56294e588e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.445+0000 7f50b0710740 1 -- 172.21.15.78:0/2152047736 >> 172.21.15.78:0/2152047736 conn(0x56294e575b10 msgr2=0x56294e56c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.445+0000 7f50877fe640 0 client.28073 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:10.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.446+0000 7f50b0710740 1 -- 172.21.15.78:0/2152047736 shutdown_connections 2024-09-11T01:19:10.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.446+0000 7f50b0710740 1 -- 172.21.15.78:0/2152047736 wait complete. 2024-09-11T01:19:10.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.446+0000 7f50b0710740 20 client.28073 trim_cache size 0 max 16384 2024-09-11T01:19:10.450 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:19:10.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.447+0000 7f50ae33b640 1 -- 172.21.15.78:0/26979789 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50900b5550 msgr2=0x7f50900b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:10.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.447+0000 7f50ae33b640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50900b5550 0x7f50900b7a10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f50a0005fe0 tx=0x7f50a0004a40 comp rx=0 tx=0).stop 2024-09-11T01:19:10.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.447+0000 7f50ae33b640 1 -- 172.21.15.78:0/26979789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50a8102dd0 msgr2=0x7f50a819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:10.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.447+0000 7f50ae33b640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50a8102dd0 0x7f50a819b450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f509800e9f0 tx=0x7f509800eeb0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.447+0000 7f50ae33b640 1 -- 172.21.15.78:0/26979789 shutdown_connections 2024-09-11T01:19:10.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.447+0000 7f50ae33b640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f50900b5550 0x7f50900b7a10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.447+0000 7f50ae33b640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50a8109160 0x7f50a81a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.447+0000 7f50ae33b640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a81087e0 0x7f50a819b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.447+0000 7f50ae33b640 1 --2- 172.21.15.78:0/26979789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50a8102dd0 0x7f50a819b450 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:10.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.447+0000 7f50ae33b640 1 -- 172.21.15.78:0/26979789 >> 172.21.15.78:0/26979789 conn(0x7f50a80fe800 msgr2=0x7f50a8104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:10.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.448+0000 7f50ae33b640 1 -- 172.21.15.78:0/26979789 shutdown_connections 2024-09-11T01:19:10.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:10.448+0000 7f50ae33b640 1 -- 172.21.15.78:0/26979789 wait complete. 2024-09-11T01:19:10.796 INFO:tasks.workunit.client.0.smithi078.stdout: 10 276637 33.51 MB/sec execute 316 sec latency 84.454 ms 2024-09-11T01:19:11.796 INFO:tasks.workunit.client.0.smithi078.stdout: 10 277235 33.51 MB/sec execute 317 sec latency 112.282 ms 2024-09-11T01:19:12.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:11 smithi110 ceph-mon[34712]: pgmap v958: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 10 MiB/s wr, 444 op/s 2024-09-11T01:19:12.219 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:19:12.219 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:19:12.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:11 smithi112 ceph-mon[35384]: pgmap v958: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 10 MiB/s wr, 444 op/s 2024-09-11T01:19:12.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:11 smithi078 ceph-mon[31315]: pgmap v958: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 10 MiB/s wr, 444 op/s 2024-09-11T01:19:12.587 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:19:12.796 INFO:tasks.workunit.client.0.smithi078.stdout: 10 277892 33.52 MB/sec execute 318 sec latency 88.377 ms 2024-09-11T01:19:13.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.211+0000 7f4785d9b640 1 -- 172.21.15.78:0/3360635552 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478003a020 con 0x7f4788108030 2024-09-11T01:19:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.212+0000 7f478d076640 1 -- 172.21.15.78:0/3360635552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4788108030 msgr2=0x7f478806b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.212+0000 7f478d076640 1 --2- 172.21.15.78:0/3360635552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4788108030 0x7f478806b490 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f47800133d0 tx=0x7f4780032a40 comp rx=0 tx=0).stop 2024-09-11T01:19:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.212+0000 7f478d076640 1 -- 172.21.15.78:0/3360635552 shutdown_connections 2024-09-11T01:19:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.212+0000 7f478d076640 1 --2- 172.21.15.78:0/3360635552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4788108030 0x7f478806b490 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.212+0000 7f478d076640 1 --2- 172.21.15.78:0/3360635552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47881076b0 0x7f4788107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.212+0000 7f478d076640 1 --2- 172.21.15.78:0/3360635552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4788101ca0 0x7f4788102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.212+0000 7f478d076640 1 -- 172.21.15.78:0/3360635552 >> 172.21.15.78:0/3360635552 conn(0x7f47880fd6f0 msgr2=0x7f47880ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.212+0000 7f478d076640 1 -- 172.21.15.78:0/3360635552 shutdown_connections 2024-09-11T01:19:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.212+0000 7f478d076640 1 -- 172.21.15.78:0/3360635552 wait complete. 2024-09-11T01:19:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.214+0000 7f478d076640 1 Processor -- start 2024-09-11T01:19:13.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.214+0000 7f478d076640 1 -- start start 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.214+0000 7f478d076640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4788101ca0 0x7f4788197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.214+0000 7f478d076640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47881076b0 0x7f47881976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.214+0000 7f478d076640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4788108030 0x7f478819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.214+0000 7f478d076640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4788110c10 con 0x7f47881076b0 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.214+0000 7f478d076640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4788110a90 con 0x7f4788101ca0 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.214+0000 7f478d076640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4788110d90 con 0x7f4788108030 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.215+0000 7f478659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47881076b0 0x7f47881976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.215+0000 7f478759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4788108030 0x7f478819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.215+0000 7f478759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4788108030 0x7f478819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:45472/0 (socket says 172.21.15.78:45472) 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.215+0000 7f478759e640 1 -- 172.21.15.78:0/3206171878 learned_addr learned my addr 172.21.15.78:0/3206171878 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.215+0000 7f4786d9d640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4788101ca0 0x7f4788197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.215+0000 7f478759e640 1 -- 172.21.15.78:0/3206171878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4788101ca0 msgr2=0x7f4788197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.215+0000 7f478759e640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4788101ca0 0x7f4788197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.215+0000 7f478759e640 1 -- 172.21.15.78:0/3206171878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47881076b0 msgr2=0x7f47881976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.215+0000 7f478759e640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47881076b0 0x7f47881976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.215+0000 7f478759e640 1 -- 172.21.15.78:0/3206171878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478819d220 con 0x7f4788108030 2024-09-11T01:19:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.215+0000 7f4786d9d640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4788101ca0 0x7f4788197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:19:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.215+0000 7f478659c640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47881076b0 0x7f47881976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:19:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.216+0000 7f478759e640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4788108030 0x7f478819ca50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4780014090 tx=0x7f47800113c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.216+0000 7f476ffff640 1 -- 172.21.15.78:0/3206171878 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478004d070 con 0x7f4788108030 2024-09-11T01:19:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.216+0000 7f478d076640 1 -- 172.21.15.78:0/3206171878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478819d4b0 con 0x7f4788108030 2024-09-11T01:19:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.216+0000 7f478d076640 1 -- 172.21.15.78:0/3206171878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47881a3bd0 con 0x7f4788108030 2024-09-11T01:19:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.216+0000 7f476ffff640 1 -- 172.21.15.78:0/3206171878 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4780003e20 con 0x7f4788108030 2024-09-11T01:19:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.216+0000 7f476ffff640 1 -- 172.21.15.78:0/3206171878 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478003a020 con 0x7f4788108030 2024-09-11T01:19:13.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.220+0000 7f476ffff640 1 -- 172.21.15.78:0/3206171878 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4780047030 con 0x7f4788108030 2024-09-11T01:19:13.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.221+0000 7f478f44b740 1 Processor -- start 2024-09-11T01:19:13.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.221+0000 7f478f44b740 1 -- start start 2024-09-11T01:19:13.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.221+0000 7f478f44b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a6ee0 0x5591856a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.221+0000 7f478f44b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a7800 0x5591856b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.221+0000 7f478f44b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856b3790 0x5591856b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.221+0000 7f478f44b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55918561de60 con 0x5591856b3790 2024-09-11T01:19:13.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.221+0000 7f478f44b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55918560d290 con 0x5591856a6ee0 2024-09-11T01:19:13.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.221+0000 7f478f44b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591856110a0 con 0x5591856a7800 2024-09-11T01:19:13.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.222+0000 7f4786d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a7800 0x5591856b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.222+0000 7f478659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a6ee0 0x5591856a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.222+0000 7f478659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a6ee0 0x5591856a72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43886/0 (socket says 172.21.15.78:43886) 2024-09-11T01:19:13.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.222+0000 7f478659c640 1 -- 172.21.15.78:0/87090930 learned_addr learned my addr 172.21.15.78:0/87090930 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:13.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.222+0000 7f478659c640 1 --2- 172.21.15.78:0/87090930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856b3790 0x5591856b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.222+0000 7f478659c640 1 -- 172.21.15.78:0/87090930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a7800 msgr2=0x5591856b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.222+0000 7f478659c640 1 --2- 172.21.15.78:0/87090930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a7800 0x5591856b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.222+0000 7f478659c640 1 -- 172.21.15.78:0/87090930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856b3790 msgr2=0x5591856b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.222+0000 7f478659c640 1 --2- 172.21.15.78:0/87090930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856b3790 0x5591856b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.222+0000 7f478659c640 1 -- 172.21.15.78:0/87090930 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55918536b930 con 0x5591856a6ee0 2024-09-11T01:19:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.223+0000 7f476ffff640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f47700b54d0 0x7f47700b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.223+0000 7f476ffff640 1 -- 172.21.15.78:0/3206171878 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8493+0+0 (secure 0 0 0) 0x7f478010e4a0 con 0x7f4788108030 2024-09-11T01:19:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.223+0000 7f478659c640 1 --2- 172.21.15.78:0/87090930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a6ee0 0x5591856a72c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f477c011870 tx=0x7f477c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9b29e5cb4b3de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.223+0000 7f4786d9d640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f47700b54d0 0x7f47700b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.223+0000 7f476d7fa640 1 -- 172.21.15.78:0/87090930 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c004030 con 0x5591856a6ee0 2024-09-11T01:19:13.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.224+0000 7f476d7fa640 1 -- 172.21.15.78:0/87090930 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f477c00f840 con 0x5591856a6ee0 2024-09-11T01:19:13.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.224+0000 7f476d7fa640 1 -- 172.21.15.78:0/87090930 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c003a90 con 0x5591856a6ee0 2024-09-11T01:19:13.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.225+0000 7f478f44b740 1 -- 172.21.15.78:0/87090930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a6ee0 msgr2=0x5591856a72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:13.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.225+0000 7f478f44b740 1 --2- 172.21.15.78:0/87090930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a6ee0 0x5591856a72c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f477c011870 tx=0x7f477c0046c0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.225+0000 7f4786d9d640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f47700b54d0 0x7f47700b7990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x5591856aa060 tx=0x7f4778008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:13.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.225+0000 7f478f44b740 1 -- 172.21.15.78:0/87090930 shutdown_connections 2024-09-11T01:19:13.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.225+0000 7f478f44b740 1 --2- 172.21.15.78:0/87090930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856b3790 0x5591856b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.225+0000 7f478f44b740 1 --2- 172.21.15.78:0/87090930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a7800 0x5591856b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.225+0000 7f478f44b740 1 --2- 172.21.15.78:0/87090930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a6ee0 0x5591856a72c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.225+0000 7f478f44b740 1 -- 172.21.15.78:0/87090930 >> 172.21.15.78:0/87090930 conn(0x5591856a4d10 msgr2=0x5591856a5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:13.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.225+0000 7f478f44b740 1 -- 172.21.15.78:0/87090930 shutdown_connections 2024-09-11T01:19:13.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.225+0000 7f478f44b740 1 -- 172.21.15.78:0/87090930 wait complete. 2024-09-11T01:19:13.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.225+0000 7f478f44b740 1 -- start start 2024-09-11T01:19:13.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.226+0000 7f478f44b740 1 Processor -- start 2024-09-11T01:19:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.226+0000 7f478f44b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a6ee0 0x5591856b3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.226+0000 7f478f44b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856a7800 0x5591856b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.226+0000 7f478f44b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856b3790 0x5591856b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.226+0000 7f478f44b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55918546e890 con 0x5591856a7800 2024-09-11T01:19:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.226+0000 7f478f44b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591855a7d40 con 0x5591856b3790 2024-09-11T01:19:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.226+0000 7f478f44b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591856b64e0 con 0x5591856a6ee0 2024-09-11T01:19:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.226+0000 7f478f44b740 10 client.0 _ll_register_callbacks cb 0x55918560bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.226+0000 7f478759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856b3790 0x5591856b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.226+0000 7f478759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856b3790 0x5591856b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43892/0 (socket says 172.21.15.78:43892) 2024-09-11T01:19:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.226+0000 7f478759e640 1 -- 172.21.15.78:0/1432074960 learned_addr learned my addr 172.21.15.78:0/1432074960 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.226+0000 7f478659c640 1 --2- 172.21.15.78:0/1432074960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856a7800 0x5591856b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f478759e640 1 -- 172.21.15.78:0/1432074960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a6ee0 msgr2=0x5591856b3200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f478759e640 1 --2- 172.21.15.78:0/1432074960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a6ee0 0x5591856b3200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f478759e640 1 -- 172.21.15.78:0/1432074960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856a7800 msgr2=0x5591856b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f478759e640 1 --2- 172.21.15.78:0/1432074960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856a7800 0x5591856b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f478759e640 1 -- 172.21.15.78:0/1432074960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591856bf8f0 con 0x5591856b3790 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f478759e640 1 --2- 172.21.15.78:0/1432074960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856b3790 0x5591856b0de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f47800d4870 tx=0x7f47800d43f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f54d176f5b5d9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f476aff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f476aff5640 1 -- 172.21.15.78:0/1432074960 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478010c030 con 0x5591856b3790 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f476aff5640 1 -- 172.21.15.78:0/1432074960 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8493+0+0 (secure 0 0 0) 0x7f4780003970 con 0x5591856b3790 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f478f44b740 1 -- 172.21.15.78:0/1432074960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f477c00f840 con 0x5591856b3790 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f476aff5640 1 -- 172.21.15.78:0/1432074960 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f47800d48e0 con 0x5591856b3790 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f478f44b740 10 client.28085 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.227+0000 7f478f44b740 1 -- 172.21.15.78:0/1432074960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5591856c2250 con 0x5591856b3790 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.228+0000 7f476aff5640 1 -- 172.21.15.78:0/1432074960 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f477c00f840 con 0x5591856b3790 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.228+0000 7f476aff5640 1 -- 172.21.15.78:0/1432074960 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f47800d3b40 con 0x5591856b3790 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.228+0000 7f478f44b740 10 client.28085 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.228+0000 7f478f44b740 10 client.28085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.228+0000 7f478f44b740 20 client.28085 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.228+0000 7f478f44b740 1 --2- 172.21.15.78:0/1432074960 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5591856c36b0 0x5591856c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.228+0000 7f478f44b740 4 client.28085 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.228+0000 7f478f44b740 1 -- 172.21.15.78:0/1432074960 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55918561fdb0 con 0x5591856c36b0 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.228+0000 7f4786d9d640 1 --2- 172.21.15.78:0/1432074960 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5591856c36b0 0x5591856c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.229+0000 7f476aff5640 10 client.28085 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.229+0000 7f4786d9d640 1 --2- 172.21.15.78:0/1432074960 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5591856c36b0 0x5591856c5b20 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59252f3a4bebf873 server_cookie=786f2e6f34e83e83 in_seq=0 out_seq=0 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f476aff5640 1 -- 172.21.15.78:0/1432074960 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55918561fdb0 con 0x5591856c36b0 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f476aff5640 10 client.28085 handle_command_reply: tid=0 2024-09-11T01:19:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f478f44b740 1 client.28085 shutdown 2024-09-11T01:19:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f478f44b740 1 -- 172.21.15.78:0/1432074960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856b3790 msgr2=0x5591856b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f478f44b740 1 --2- 172.21.15.78:0/1432074960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856b3790 0x5591856b0de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f47800d4870 tx=0x7f47800d43f0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f478f44b740 1 -- 172.21.15.78:0/1432074960 shutdown_connections 2024-09-11T01:19:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f478f44b740 1 --2- 172.21.15.78:0/1432074960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856b3790 0x5591856b0de0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f478f44b740 1 --2- 172.21.15.78:0/1432074960 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5591856c36b0 0x5591856c5b20 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f478f44b740 1 --2- 172.21.15.78:0/1432074960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856a7800 0x5591856b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f478f44b740 1 --2- 172.21.15.78:0/1432074960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a6ee0 0x5591856b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f478f44b740 1 -- 172.21.15.78:0/1432074960 >> 172.21.15.78:0/1432074960 conn(0x5591856a4d10 msgr2=0x5591856a3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f476aff5640 0 client.28085 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f478f44b740 1 -- 172.21.15.78:0/1432074960 shutdown_connections 2024-09-11T01:19:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f478f44b740 1 -- 172.21.15.78:0/1432074960 wait complete. 2024-09-11T01:19:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.230+0000 7f478f44b740 20 client.28085 trim_cache size 0 max 16384 2024-09-11T01:19:13.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.250+0000 7f478f44b740 1 Processor -- start 2024-09-11T01:19:13.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.250+0000 7f478f44b740 1 -- start start 2024-09-11T01:19:13.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f478f44b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a6ee0 0x5591856ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f478f44b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a7800 0x5591856adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f478f44b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856b3790 0x5591856cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f478f44b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f477400e660 con 0x5591856b3790 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f478f44b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f47740101a0 con 0x5591856a7800 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f478f44b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f477400ed30 con 0x5591856a6ee0 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f478759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856b3790 0x5591856cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f478759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856b3790 0x5591856cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57154/0 (socket says 172.21.15.78:57154) 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f478759e640 1 -- 172.21.15.78:0/2104028919 learned_addr learned my addr 172.21.15.78:0/2104028919 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f4786d9d640 1 --2- 172.21.15.78:0/2104028919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a6ee0 0x5591856ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f478659c640 1 --2- 172.21.15.78:0/2104028919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a7800 0x5591856adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f4786d9d640 1 -- 172.21.15.78:0/2104028919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a7800 msgr2=0x5591856adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f4786d9d640 1 --2- 172.21.15.78:0/2104028919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a7800 0x5591856adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f4786d9d640 1 -- 172.21.15.78:0/2104028919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856b3790 msgr2=0x5591856cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f4786d9d640 1 --2- 172.21.15.78:0/2104028919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856b3790 0x5591856cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f4786d9d640 1 -- 172.21.15.78:0/2104028919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591856c5e90 con 0x5591856a6ee0 2024-09-11T01:19:13.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.251+0000 7f4786d9d640 1 --2- 172.21.15.78:0/2104028919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a6ee0 0x5591856ad860 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5591856c6330 tx=0x7f4778012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c3654b1750dbb96 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:13.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.252+0000 7f476cff9640 1 -- 172.21.15.78:0/2104028919 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477801b030 con 0x5591856a6ee0 2024-09-11T01:19:13.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.252+0000 7f476cff9640 1 -- 172.21.15.78:0/2104028919 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f477802d070 con 0x5591856a6ee0 2024-09-11T01:19:13.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.252+0000 7f476cff9640 1 -- 172.21.15.78:0/2104028919 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778028410 con 0x5591856a6ee0 2024-09-11T01:19:13.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.252+0000 7f478f44b740 1 -- 172.21.15.78:0/2104028919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a6ee0 msgr2=0x5591856ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:13.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.252+0000 7f478f44b740 1 --2- 172.21.15.78:0/2104028919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a6ee0 0x5591856ad860 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5591856c6330 tx=0x7f4778012000 comp rx=0 tx=0).stop 2024-09-11T01:19:13.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.252+0000 7f478f44b740 1 -- 172.21.15.78:0/2104028919 shutdown_connections 2024-09-11T01:19:13.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.252+0000 7f478f44b740 1 --2- 172.21.15.78:0/2104028919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856b3790 0x5591856cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.252+0000 7f478f44b740 1 --2- 172.21.15.78:0/2104028919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a7800 0x5591856adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.252+0000 7f478f44b740 1 --2- 172.21.15.78:0/2104028919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856a6ee0 0x5591856ad860 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.252+0000 7f478f44b740 1 -- 172.21.15.78:0/2104028919 >> 172.21.15.78:0/2104028919 conn(0x5591856a4d10 msgr2=0x5591856ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:13.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478f44b740 1 -- 172.21.15.78:0/2104028919 shutdown_connections 2024-09-11T01:19:13.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478f44b740 1 -- 172.21.15.78:0/2104028919 wait complete. 2024-09-11T01:19:13.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478f44b740 1 -- start start 2024-09-11T01:19:13.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478f44b740 1 Processor -- start 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478f44b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a6ee0 0x5591856bb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478f44b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856a7800 0x5591856bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478f44b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856b3790 0x5591856b6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478f44b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f477400fad0 con 0x5591856a7800 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478f44b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f477400f400 con 0x5591856a6ee0 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478f44b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f477400c660 con 0x5591856b3790 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856a7800 0x5591856bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856a7800 0x5591856bbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57166/0 (socket says 172.21.15.78:57166) 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.253+0000 7f478659c640 1 -- 172.21.15.78:0/4159684258 learned_addr learned my addr 172.21.15.78:0/4159684258 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.254+0000 7f478f44b740 10 client.0 _ll_register_callbacks cb 0x7f4780005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.254+0000 7f4786d9d640 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a6ee0 0x5591856bb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.254+0000 7f478759e640 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856b3790 0x5591856b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.254+0000 7f478659c640 1 -- 172.21.15.78:0/4159684258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856b3790 msgr2=0x5591856b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.254+0000 7f478659c640 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856b3790 0x5591856b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.254+0000 7f478659c640 1 -- 172.21.15.78:0/4159684258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a6ee0 msgr2=0x5591856bb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.254+0000 7f478659c640 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a6ee0 0x5591856bb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.254+0000 7f478659c640 1 -- 172.21.15.78:0/4159684258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591856b6d50 con 0x5591856a7800 2024-09-11T01:19:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.254+0000 7f478659c640 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856a7800 0x5591856bbda0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f477c00f790 tx=0x7f477c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61259bb053c6f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.254+0000 7f476aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.254+0000 7f476aff5640 1 -- 172.21.15.78:0/4159684258 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c005640 con 0x5591856a7800 2024-09-11T01:19:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.255+0000 7f476aff5640 1 -- 172.21.15.78:0/4159684258 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8493+0+0 (secure 0 0 0) 0x7f477c011240 con 0x5591856a7800 2024-09-11T01:19:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.255+0000 7f478f44b740 1 -- 172.21.15.78:0/4159684258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f477802d070 con 0x5591856a7800 2024-09-11T01:19:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.255+0000 7f476aff5640 1 -- 172.21.15.78:0/4159684258 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f477c00fae0 con 0x5591856a7800 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.255+0000 7f478f44b740 10 client.19026 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.255+0000 7f478f44b740 1 -- 172.21.15.78:0/4159684258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x559185730dd0 con 0x5591856a7800 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.256+0000 7f476aff5640 1 -- 172.21.15.78:0/4159684258 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f477802d070 con 0x5591856a7800 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.256+0000 7f476aff5640 1 -- 172.21.15.78:0/4159684258 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7f477c030020 con 0x5591856a7800 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.256+0000 7f478f44b740 10 client.19026 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.256+0000 7f478f44b740 10 client.19026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.256+0000 7f478f44b740 20 client.19026 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.256+0000 7f478f44b740 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559185731330 0x559185733750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.256+0000 7f478f44b740 4 client.19026 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.256+0000 7f478f44b740 1 -- 172.21.15.78:0/4159684258 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55918561b860 con 0x559185731330 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.256+0000 7f4786d9d640 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559185731330 0x559185733750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.256+0000 7f476aff5640 10 client.19026 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.257+0000 7f4786d9d640 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559185731330 0x559185733750 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39068f7ad7522859 server_cookie=57f12b0b54beb4a1 in_seq=0 out_seq=0 2024-09-11T01:19:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.258+0000 7f476aff5640 1 -- 172.21.15.78:0/4159684258 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55918561b860 con 0x559185731330 2024-09-11T01:19:13.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.258+0000 7f476aff5640 10 client.19026 handle_command_reply: tid=0 2024-09-11T01:19:13.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.258+0000 7f478f44b740 1 client.19026 shutdown 2024-09-11T01:19:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.258+0000 7f478f44b740 1 -- 172.21.15.78:0/4159684258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856a7800 msgr2=0x5591856bbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.258+0000 7f478f44b740 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856a7800 0x5591856bbda0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f477c00f790 tx=0x7f477c00fa70 comp rx=0 tx=0).stop 2024-09-11T01:19:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.259+0000 7f478f44b740 1 -- 172.21.15.78:0/4159684258 shutdown_connections 2024-09-11T01:19:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.259+0000 7f478f44b740 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591856b3790 0x5591856b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.259+0000 7f478f44b740 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559185731330 0x559185733750 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.259+0000 7f478f44b740 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591856a7800 0x5591856bbda0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.259+0000 7f476aff5640 0 client.19026 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.259+0000 7f478f44b740 1 --2- 172.21.15.78:0/4159684258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591856a6ee0 0x5591856bb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.259+0000 7f478f44b740 1 -- 172.21.15.78:0/4159684258 >> 172.21.15.78:0/4159684258 conn(0x5591856a4d10 msgr2=0x5591856aa090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:13.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.259+0000 7f478f44b740 1 -- 172.21.15.78:0/4159684258 shutdown_connections 2024-09-11T01:19:13.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.259+0000 7f478f44b740 1 -- 172.21.15.78:0/4159684258 wait complete. 2024-09-11T01:19:13.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.259+0000 7f478f44b740 20 client.19026 trim_cache size 0 max 16384 2024-09-11T01:19:13.265 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:19:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:19:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "b41a2b77-b2b1-4f1b-8588-4f32a78b721f", 2024-09-11T01:19:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:19:13.265 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:19:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.260+0000 7f478d076640 1 -- 172.21.15.78:0/3206171878 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f47700b54d0 msgr2=0x7f47700b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.260+0000 7f478d076640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f47700b54d0 0x7f47700b7990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x5591856aa060 tx=0x7f4778008040 comp rx=0 tx=0).stop 2024-09-11T01:19:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.260+0000 7f478d076640 1 -- 172.21.15.78:0/3206171878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4788108030 msgr2=0x7f478819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.260+0000 7f478d076640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4788108030 0x7f478819ca50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4780014090 tx=0x7f47800113c0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.260+0000 7f478d076640 1 -- 172.21.15.78:0/3206171878 shutdown_connections 2024-09-11T01:19:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.260+0000 7f478d076640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f47700b54d0 0x7f47700b7990 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.260+0000 7f478d076640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4788108030 0x7f478819ca50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.260+0000 7f478d076640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47881076b0 0x7f47881976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.260+0000 7f478d076640 1 --2- 172.21.15.78:0/3206171878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4788101ca0 0x7f4788197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.260+0000 7f478d076640 1 -- 172.21.15.78:0/3206171878 >> 172.21.15.78:0/3206171878 conn(0x7f47880fd6f0 msgr2=0x7f478810ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.261+0000 7f478d076640 1 -- 172.21.15.78:0/3206171878 shutdown_connections 2024-09-11T01:19:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:13.261+0000 7f478d076640 1 -- 172.21.15.78:0/3206171878 wait complete. 2024-09-11T01:19:13.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 278215 33.46 MB/sec execute 319 sec latency 367.676 ms 2024-09-11T01:19:14.057 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:13 smithi078 ceph-mon[31315]: pgmap v959: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 278 op/s 2024-09-11T01:19:14.057 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:13 smithi078 ceph-mon[31315]: osdmap e104: 12 total, 12 up, 12 in 2024-09-11T01:19:14.057 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:13 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:19:14.057 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:13 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:14.057 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:13 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:19:14.057 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:13 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:19:14.058 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:13 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:19:14.058 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:13 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:14.104 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:19:14.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:13 smithi110 ceph-mon[34712]: pgmap v959: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 278 op/s 2024-09-11T01:19:14.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:13 smithi110 ceph-mon[34712]: osdmap e104: 12 total, 12 up, 12 in 2024-09-11T01:19:14.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:13 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:19:14.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:13 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:14.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:13 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:19:14.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:13 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:19:14.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:13 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:19:14.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:13 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:13 smithi112 ceph-mon[35384]: pgmap v959: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 278 op/s 2024-09-11T01:19:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:13 smithi112 ceph-mon[35384]: osdmap e104: 12 total, 12 up, 12 in 2024-09-11T01:19:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:13 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:19:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:13 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:13 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:19:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:13 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:19:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:13 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:19:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:13 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:14.515 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:19:14.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 278912 33.46 MB/sec execute 320 sec latency 348.067 ms 2024-09-11T01:19:14.962 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:14 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:19:14.962 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:14 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:19:14.962 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:14.962 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:14.962 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:14.962 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:14 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:15.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.077+0000 7fb0de59e640 1 -- 172.21.15.78:0/3994391048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0d8103b10 msgr2=0x7fb0d8103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:15.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.077+0000 7fb0de59e640 1 --2- 172.21.15.78:0/3994391048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0d8103b10 0x7fb0d8103ef0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009900 tx=0x7fb0d402a3f0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.077+0000 7fb0de59e640 1 -- 172.21.15.78:0/3994391048 shutdown_connections 2024-09-11T01:19:15.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.077+0000 7fb0de59e640 1 --2- 172.21.15.78:0/3994391048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d810cf70 0x7fb0d810f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.077+0000 7fb0de59e640 1 --2- 172.21.15.78:0/3994391048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d8104430 0x7fb0d810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.077+0000 7fb0de59e640 1 --2- 172.21.15.78:0/3994391048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0d8103b10 0x7fb0d8103ef0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.077+0000 7fb0de59e640 1 -- 172.21.15.78:0/3994391048 >> 172.21.15.78:0/3994391048 conn(0x7fb0d80fd5b0 msgr2=0x7fb0d80ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:15.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.078+0000 7fb0de59e640 1 -- 172.21.15.78:0/3994391048 shutdown_connections 2024-09-11T01:19:15.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.078+0000 7fb0de59e640 1 -- 172.21.15.78:0/3994391048 wait complete. 2024-09-11T01:19:15.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0de59e640 1 Processor -- start 2024-09-11T01:19:15.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0de59e640 1 -- start start 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0de59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d8103b10 0x7fb0d81017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0de59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d8104430 0x7fb0d8101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0de59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0d810cf70 0x7fb0d8102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0de59e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8112c70 con 0x7fb0d810cf70 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0de59e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8112af0 con 0x7fb0d8103b10 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0de59e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8112df0 con 0x7fb0d8104430 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0dd59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d8103b10 0x7fb0d81017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0dd59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d8103b10 0x7fb0d81017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:51558/0 (socket says 172.21.15.78:51558) 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0dd59c640 1 -- 172.21.15.78:0/123456180 learned_addr learned my addr 172.21.15.78:0/123456180 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d8104430 0x7fb0d8101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.079+0000 7fb0ddd9d640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0d810cf70 0x7fb0d8102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.080+0000 7fb0dcd9b640 1 -- 172.21.15.78:0/123456180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d8103b10 msgr2=0x7fb0d81017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.080+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d8103b10 0x7fb0d81017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.080+0000 7fb0dcd9b640 1 -- 172.21.15.78:0/123456180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0d810cf70 msgr2=0x7fb0d8102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.080+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0d810cf70 0x7fb0d8102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.080+0000 7fb0dcd9b640 1 -- 172.21.15.78:0/123456180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d81a74a0 con 0x7fb0d8104430 2024-09-11T01:19:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.080+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d8104430 0x7fb0d8101d00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800e770 tx=0x7fb0c800ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.080+0000 7fb0ce7fc640 1 -- 172.21.15.78:0/123456180 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c800ccc0 con 0x7fb0d8104430 2024-09-11T01:19:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.080+0000 7fb0de59e640 1 -- 172.21.15.78:0/123456180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d81a7790 con 0x7fb0d8104430 2024-09-11T01:19:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.080+0000 7fb0de59e640 1 -- 172.21.15.78:0/123456180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d81a7da0 con 0x7fb0d8104430 2024-09-11T01:19:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.080+0000 7fb0ce7fc640 1 -- 172.21.15.78:0/123456180 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c800ee80 con 0x7fb0d8104430 2024-09-11T01:19:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.081+0000 7fb0ce7fc640 1 -- 172.21.15.78:0/123456180 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8003e40 con 0x7fb0d8104430 2024-09-11T01:19:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.084+0000 7fb0e1bd5740 1 Processor -- start 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.084+0000 7fb0e1bd5740 1 -- start start 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.084+0000 7fb0e1bd5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ec650 0x55596c7eca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.084+0000 7fb0e1bd5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 0x55596c7f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.084+0000 7fb0e1bd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7fb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.084+0000 7fb0e1bd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55596c762e50 con 0x55596c7f8f00 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.084+0000 7fb0e1bd5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55596c752280 con 0x55596c7ecf70 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.084+0000 7fb0e1bd5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55596c756090 con 0x55596c7ec650 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0dd59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 0x55596c7f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0dd59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 0x55596c7f89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:51572/0 (socket says 172.21.15.78:51572) 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0dd59c640 1 -- 172.21.15.78:0/4095747488 learned_addr learned my addr 172.21.15.78:0/4095747488 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0ddd9d640 1 --2- 172.21.15.78:0/4095747488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ec650 0x55596c7eca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0ce7fc640 1 -- 172.21.15.78:0/123456180 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb0c80104b0 con 0x7fb0d8104430 2024-09-11T01:19:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0dd59c640 1 -- 172.21.15.78:0/4095747488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ec650 msgr2=0x55596c7eca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0dd59c640 1 --2- 172.21.15.78:0/4095747488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ec650 0x55596c7eca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/4095747488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7fb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0dd59c640 1 -- 172.21.15.78:0/4095747488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 msgr2=0x55596c7fb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0dd59c640 1 --2- 172.21.15.78:0/4095747488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7fb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0dd59c640 1 -- 172.21.15.78:0/4095747488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55596c4b0650 con 0x55596c7ecf70 2024-09-11T01:19:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/4095747488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:19:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.085+0000 7fb0dd59c640 1 --2- 172.21.15.78:0/4095747488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 0x55596c7f89c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4006010 tx=0x7fb0d4004240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8044eec95a203037 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.086+0000 7fb0c37fe640 1 -- 172.21.15.78:0/4095747488 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4040070 con 0x55596c7ecf70 2024-09-11T01:19:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.086+0000 7fb0c37fe640 1 -- 172.21.15.78:0/4095747488 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0d40047a0 con 0x55596c7ecf70 2024-09-11T01:19:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.086+0000 7fb0c37fe640 1 -- 172.21.15.78:0/4095747488 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4035040 con 0x55596c7ecf70 2024-09-11T01:19:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.086+0000 7fb0ce7fc640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0c40b5600 0x7fb0c40b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.087+0000 7fb0ce7fc640 1 -- 172.21.15.78:0/123456180 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8493+0+0 (secure 0 0 0) 0x7fb0c80e5ff0 con 0x7fb0d8104430 2024-09-11T01:19:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.087+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/4095747488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 msgr2=0x55596c7f89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.087+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/4095747488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 0x55596c7f89c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4006010 tx=0x7fb0d4004240 comp rx=0 tx=0).stop 2024-09-11T01:19:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.087+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/4095747488 shutdown_connections 2024-09-11T01:19:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.087+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/4095747488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.087+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/4095747488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 0x55596c7f89c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.087+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/4095747488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ec650 0x55596c7eca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.087+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/4095747488 >> 172.21.15.78:0/4095747488 conn(0x55596c7ea480 msgr2=0x55596c7eb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.087+0000 7fb0ddd9d640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0c40b5600 0x7fb0c40b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.088+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/4095747488 shutdown_connections 2024-09-11T01:19:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.088+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/4095747488 wait complete. 2024-09-11T01:19:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.088+0000 7fb0ddd9d640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0c40b5600 0x7fb0c40b7ac0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8103880 tx=0x7fb0d000d040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.088+0000 7fb0e1bd5740 1 -- start start 2024-09-11T01:19:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.089+0000 7fb0e1bd5740 1 Processor -- start 2024-09-11T01:19:15.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.089+0000 7fb0e1bd5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 0x55596c7f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0e1bd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7ecf70 0x55596c7f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0e1bd5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7f8f00 0x55596c7f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0e1bd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55596c5b3700 con 0x55596c7ecf70 2024-09-11T01:19:15.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0e1bd5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55596c6ed110 con 0x55596c7ec650 2024-09-11T01:19:15.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0e1bd5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55596c7fbb20 con 0x55596c7f8f00 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0e1bd5740 10 client.0 _ll_register_callbacks cb 0x55596c706990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0dd59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 0x55596c7f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0dd59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 0x55596c7f87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:51586/0 (socket says 172.21.15.78:51586) 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0dd59c640 1 -- 172.21.15.78:0/3414973698 learned_addr learned my addr 172.21.15.78:0/3414973698 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7ecf70 0x55596c7f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0ddd9d640 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7f8f00 0x55596c7f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0dd59c640 1 -- 172.21.15.78:0/3414973698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7f8f00 msgr2=0x55596c7f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0dd59c640 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7f8f00 0x55596c7f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0dd59c640 1 -- 172.21.15.78:0/3414973698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7ecf70 msgr2=0x55596c7f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0dd59c640 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7ecf70 0x55596c7f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.090+0000 7fb0dd59c640 1 -- 172.21.15.78:0/3414973698 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55596c71ac40 con 0x55596c7ec650 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.091+0000 7fb0dd59c640 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 0x55596c7f87b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4004580 tx=0x7fb0d4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7cc7529011146b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.091+0000 7fb0c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:19:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.091+0000 7fb0c0ff9640 1 -- 172.21.15.78:0/3414973698 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4035070 con 0x55596c7ec650 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.091+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3414973698 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d40047a0 con 0x55596c7ec650 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.091+0000 7fb0c0ff9640 1 -- 172.21.15.78:0/3414973698 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8493+0+0 (secure 0 0 0) 0x7fb0d4005430 con 0x55596c7ec650 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.091+0000 7fb0c0ff9640 1 -- 172.21.15.78:0/3414973698 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fb0d4040040 con 0x55596c7ec650 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.091+0000 7fb0e1bd5740 10 client.28103 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.091+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3414973698 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55596c807410 con 0x55596c7ec650 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.092+0000 7fb0c0ff9640 1 -- 172.21.15.78:0/3414973698 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0d4009520 con 0x55596c7ec650 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.092+0000 7fb0c0ff9640 1 -- 172.21.15.78:0/3414973698 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7fb0d4054070 con 0x55596c7ec650 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.092+0000 7fb0e1bd5740 10 client.28103 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.092+0000 7fb0e1bd5740 10 client.28103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.092+0000 7fb0e1bd5740 20 client.28103 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.092+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55596c8089c0 0x55596c80ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.092+0000 7fb0e1bd5740 4 client.28103 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:19:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.092+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3414973698 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb0d40047a0 con 0x55596c8089c0 2024-09-11T01:19:15.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.092+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55596c8089c0 0x55596c80ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.093+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55596c8089c0 0x55596c80ae00 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f23e5ae76e8e8809 server_cookie=6d742245911d311b in_seq=0 out_seq=0 2024-09-11T01:19:15.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.093+0000 7fb0c0ff9640 10 client.28103 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:15.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.094+0000 7fb0c0ff9640 1 -- 172.21.15.78:0/3414973698 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fb0d40047a0 con 0x55596c8089c0 2024-09-11T01:19:15.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.094+0000 7fb0c0ff9640 10 client.28103 handle_command_reply: tid=0 2024-09-11T01:19:15.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.095+0000 7fb0e1bd5740 1 client.28103 shutdown 2024-09-11T01:19:15.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.095+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3414973698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 msgr2=0x55596c7f87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:15.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.095+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 0x55596c7f87b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4004580 tx=0x7fb0d4000bc0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.095+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3414973698 shutdown_connections 2024-09-11T01:19:15.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.095+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7f8f00 0x55596c7f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.095+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55596c8089c0 0x55596c80ae00 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.095+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7ecf70 0x55596c7f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.095+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/3414973698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 0x55596c7f87b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.096+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3414973698 >> 172.21.15.78:0/3414973698 conn(0x55596c7ea480 msgr2=0x55596c7efff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:15.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.096+0000 7fb0c0ff9640 0 client.28103 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:15.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.096+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3414973698 shutdown_connections 2024-09-11T01:19:15.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.096+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3414973698 wait complete. 2024-09-11T01:19:15.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.096+0000 7fb0e1bd5740 20 client.28103 trim_cache size 0 max 16384 2024-09-11T01:19:15.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0e1bd5740 1 Processor -- start 2024-09-11T01:19:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0e1bd5740 1 -- start start 2024-09-11T01:19:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0e1bd5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ec650 0x55596c7e9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0e1bd5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 0x55596c80a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0e1bd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7eb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0e1bd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc00e740 con 0x55596c7f8f00 2024-09-11T01:19:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0e1bd5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc010280 con 0x55596c7ecf70 2024-09-11T01:19:15.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0e1bd5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc00ee10 con 0x55596c7ec650 2024-09-11T01:19:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0dd59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ec650 0x55596c7e9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0dcd9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 0x55596c80a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0dd59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ec650 0x55596c7e9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52306/0 (socket says 172.21.15.78:52306) 2024-09-11T01:19:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0dd59c640 1 -- 172.21.15.78:0/114236582 learned_addr learned my addr 172.21.15.78:0/114236582 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.120+0000 7fb0ddd9d640 1 --2- 172.21.15.78:0/114236582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7eb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.121+0000 7fb0dcd9b640 1 -- 172.21.15.78:0/114236582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ec650 msgr2=0x55596c7e9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.121+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/114236582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ec650 0x55596c7e9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.121+0000 7fb0dcd9b640 1 -- 172.21.15.78:0/114236582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 msgr2=0x55596c7eb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.121+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/114236582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7eb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.121+0000 7fb0dcd9b640 1 -- 172.21.15.78:0/114236582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55596c7f4b60 con 0x55596c7ecf70 2024-09-11T01:19:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.121+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/114236582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 0x55596c80a9d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80e3700 tx=0x7fb0c80e8440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcbd77fe432f5dfb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.121+0000 7fb0c2ffd640 1 -- 172.21.15.78:0/114236582 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c80d3020 con 0x55596c7ecf70 2024-09-11T01:19:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.121+0000 7fb0c2ffd640 1 -- 172.21.15.78:0/114236582 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c80e3e50 con 0x55596c7ecf70 2024-09-11T01:19:15.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.121+0000 7fb0c2ffd640 1 -- 172.21.15.78:0/114236582 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c80b0f10 con 0x55596c7ecf70 2024-09-11T01:19:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.122+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/114236582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 msgr2=0x55596c80a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.122+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/114236582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 0x55596c80a9d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80e3700 tx=0x7fb0c80e8440 comp rx=0 tx=0).stop 2024-09-11T01:19:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.122+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/114236582 shutdown_connections 2024-09-11T01:19:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.122+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/114236582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7eb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.122+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/114236582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ecf70 0x55596c80a9d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.122+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/114236582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ec650 0x55596c7e9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.122+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/114236582 >> 172.21.15.78:0/114236582 conn(0x55596c7ea480 msgr2=0x55596c7eb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.122+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/114236582 shutdown_connections 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.122+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/114236582 wait complete. 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.123+0000 7fb0e1bd5740 1 -- start start 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.123+0000 7fb0e1bd5740 1 Processor -- start 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.123+0000 7fb0e1bd5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 0x55596c801190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0e1bd5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ecf70 0x55596c801710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0e1bd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7fbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0e1bd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc00fbb0 con 0x55596c7f8f00 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0dd59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 0x55596c801190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0dd59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 0x55596c801190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:51604/0 (socket says 172.21.15.78:51604) 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0dd59c640 1 -- 172.21.15.78:0/3945697741 learned_addr learned my addr 172.21.15.78:0/3945697741 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3945697741 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc00f4e0 con 0x55596c7ec650 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3945697741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc00c740 con 0x55596c7ecf70 2024-09-11T01:19:15.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ecf70 0x55596c801710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0ddd9d640 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7fbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0e1bd5740 10 client.0 _ll_register_callbacks cb 0x7fb0d0009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0dd59c640 1 -- 172.21.15.78:0/3945697741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ecf70 msgr2=0x55596c801710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0dd59c640 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ecf70 0x55596c801710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0dd59c640 1 -- 172.21.15.78:0/3945697741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 msgr2=0x55596c7fbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0dd59c640 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7fbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.124+0000 7fb0dd59c640 1 -- 172.21.15.78:0/3945697741 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55596c7fc720 con 0x55596c7ec650 2024-09-11T01:19:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.125+0000 7fb0dd59c640 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 0x55596c801190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb0d402a9e0 tx=0x7fb0d4004d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8547799619e33b9d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:15.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.125+0000 7fb0c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:19:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.125+0000 7fb0c0ff9640 1 -- 172.21.15.78:0/3945697741 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d405a020 con 0x55596c7ec650 2024-09-11T01:19:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.126+0000 7fb0c0ff9640 1 -- 172.21.15.78:0/3945697741 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8493+0+0 (secure 0 0 0) 0x7fb0d4031ba0 con 0x55596c7ec650 2024-09-11T01:19:15.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.126+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3945697741 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0c80e3e50 con 0x55596c7ec650 2024-09-11T01:19:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.128+0000 7fb0c0ff9640 1 -- 172.21.15.78:0/3945697741 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fb0d4065050 con 0x55596c7ec650 2024-09-11T01:19:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.128+0000 7fb0e1bd5740 10 client.28109 fetch_fsmap learned FSMap version 70 2024-09-11T01:19:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.128+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3945697741 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55596c87ecd0 con 0x55596c7ec650 2024-09-11T01:19:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.128+0000 7fb0c0ff9640 1 -- 172.21.15.78:0/3945697741 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0d40047a0 con 0x55596c7ec650 2024-09-11T01:19:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.128+0000 7fb0c0ff9640 1 -- 172.21.15.78:0/3945697741 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 70) ==== 8362+0+0 (secure 0 0 0) 0x7fb0d4054040 con 0x55596c7ec650 2024-09-11T01:19:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.128+0000 7fb0e1bd5740 10 client.28109 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T01:19:15.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.128+0000 7fb0e1bd5740 10 client.28109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:19:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.128+0000 7fb0e1bd5740 20 client.28109 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.128+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55596c87f480 0x55596c8818a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.128+0000 7fb0e1bd5740 4 client.28109 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:19:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.128+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3945697741 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb0c80e3e50 con 0x55596c87f480 2024-09-11T01:19:15.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.129+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55596c87f480 0x55596c8818a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.129+0000 7fb0dcd9b640 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55596c87f480 0x55596c8818a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aea30224e53a4e05 server_cookie=62a5989240f27da2 in_seq=0 out_seq=0 2024-09-11T01:19:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.129+0000 7fb0c0ff9640 10 client.28109 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:15.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0c0ff9640 1 -- 172.21.15.78:0/3945697741 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7386 (crc 0 0 0) 0x7fb0c80e3e50 con 0x55596c87f480 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0c0ff9640 10 client.28109 handle_command_reply: tid=0 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0e1bd5740 1 client.28109 shutdown 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3945697741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 msgr2=0x55596c801190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 0x55596c801190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb0d402a9e0 tx=0x7fb0d4004d90 comp rx=0 tx=0).stop 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3945697741 shutdown_connections 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55596c87f480 0x55596c8818a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55596c7f8f00 0x55596c7fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55596c7ecf70 0x55596c801710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0e1bd5740 1 --2- 172.21.15.78:0/3945697741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55596c7ec650 0x55596c801190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0c0ff9640 0 client.28109 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3945697741 >> 172.21.15.78:0/3945697741 conn(0x55596c7ea480 msgr2=0x55596c7eb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3945697741 shutdown_connections 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0e1bd5740 1 -- 172.21.15.78:0/3945697741 wait complete. 2024-09-11T01:19:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.130+0000 7fb0e1bd5740 20 client.28109 trim_cache size 0 max 16384 2024-09-11T01:19:15.134 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:19:15.134 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:19:15.134 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:19:15.134 INFO:teuthology.orchestra.run.smithi078.stdout: "b41a2b77-b2b1-4f1b-8588-4f32a78b721f": { 2024-09-11T01:19:15.134 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.134 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "b41a2b77-b2b1-4f1b-8588-4f32a78b721f", 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: "b41a2b77-b2b1-4f1b-8588-4f32a78b721f": { 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:19:13.257610+0000", 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.135 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:18:17.654570+0000", 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.136 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.137 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:19:15.138 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.140 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:19:15.141 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.142 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.142 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.142 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.142 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.142 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:19:15.142 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:19:15.142 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.142 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.142 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.142 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.142 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.143 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.144 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.145 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.146 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:19:15.147 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:15.149 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:15.150 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:19:15.150 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:19:15.150 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:19:15.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.131+0000 7fb0de59e640 1 -- 172.21.15.78:0/123456180 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0c40b5600 msgr2=0x7fb0c40b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:15.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.131+0000 7fb0de59e640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0c40b5600 0x7fb0c40b7ac0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8103880 tx=0x7fb0d000d040 comp rx=0 tx=0).stop 2024-09-11T01:19:15.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.131+0000 7fb0de59e640 1 -- 172.21.15.78:0/123456180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d8104430 msgr2=0x7fb0d8101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:15.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.131+0000 7fb0de59e640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d8104430 0x7fb0d8101d00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800e770 tx=0x7fb0c800ec30 comp rx=0 tx=0).stop 2024-09-11T01:19:15.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.131+0000 7fb0de59e640 1 -- 172.21.15.78:0/123456180 shutdown_connections 2024-09-11T01:19:15.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.131+0000 7fb0de59e640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb0c40b5600 0x7fb0c40b7ac0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.131+0000 7fb0de59e640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0d810cf70 0x7fb0d8102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.131+0000 7fb0de59e640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d8104430 0x7fb0d8101d00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.131+0000 7fb0de59e640 1 --2- 172.21.15.78:0/123456180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d8103b10 0x7fb0d81017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:15.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.131+0000 7fb0de59e640 1 -- 172.21.15.78:0/123456180 >> 172.21.15.78:0/123456180 conn(0x7fb0d80fd5b0 msgr2=0x7fb0d80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:15.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.131+0000 7fb0de59e640 1 -- 172.21.15.78:0/123456180 shutdown_connections 2024-09-11T01:19:15.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:15.131+0000 7fb0de59e640 1 -- 172.21.15.78:0/123456180 wait complete. 2024-09-11T01:19:15.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:14 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:19:15.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:14 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:19:15.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:15.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:15.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:15.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:14 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:15.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:14 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:19:15.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:14 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:19:15.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:15.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:15.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:15.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:14 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:19:15.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 279671 33.51 MB/sec execute 321 sec latency 73.389 ms 2024-09-11T01:19:15.999 INFO:tasks.cephfs.filesystem:scrub status for tag:b41a2b77-b2b1-4f1b-8588-4f32a78b721f - {'path': '/', 'tag': 'b41a2b77-b2b1-4f1b-8588-4f32a78b721f', 'options': 'recursive,force'} 2024-09-11T01:19:16.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:15 smithi110 ceph-mon[34712]: pgmap v961: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-09-11T01:19:16.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:15 smithi112 ceph-mon[35384]: pgmap v961: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-09-11T01:19:16.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:15 smithi078 ceph-mon[31315]: pgmap v961: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-09-11T01:19:16.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 280341 33.51 MB/sec execute 322 sec latency 67.383 ms 2024-09-11T01:19:17.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 281055 33.54 MB/sec execute 323 sec latency 80.995 ms 2024-09-11T01:19:18.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:17 smithi110 ceph-mon[34712]: pgmap v962: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.4 MiB/s wr, 361 op/s 2024-09-11T01:19:18.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:17 smithi112 ceph-mon[35384]: pgmap v962: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.4 MiB/s wr, 361 op/s 2024-09-11T01:19:18.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:17 smithi078 ceph-mon[31315]: pgmap v962: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.4 MiB/s wr, 361 op/s 2024-09-11T01:19:18.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 281612 33.51 MB/sec execute 324 sec latency 162.321 ms 2024-09-11T01:19:19.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 282123 33.48 MB/sec execute 325 sec latency 381.401 ms 2024-09-11T01:19:20.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:19 smithi110 ceph-mon[34712]: pgmap v963: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.8 MiB/s wr, 460 op/s 2024-09-11T01:19:20.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:19 smithi112 ceph-mon[35384]: pgmap v963: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.8 MiB/s wr, 460 op/s 2024-09-11T01:19:20.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:19 smithi078 ceph-mon[31315]: pgmap v963: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.8 MiB/s wr, 460 op/s 2024-09-11T01:19:20.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 282523 33.44 MB/sec execute 326 sec latency 522.506 ms 2024-09-11T01:19:21.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 283330 33.48 MB/sec execute 327 sec latency 103.643 ms 2024-09-11T01:19:22.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:21 smithi110 ceph-mon[34712]: pgmap v964: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 601 op/s 2024-09-11T01:19:22.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:21 smithi112 ceph-mon[35384]: pgmap v964: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 601 op/s 2024-09-11T01:19:22.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:21 smithi078 ceph-mon[31315]: pgmap v964: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 601 op/s 2024-09-11T01:19:22.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 283817 33.48 MB/sec execute 328 sec latency 163.306 ms 2024-09-11T01:19:23.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 284480 33.49 MB/sec execute 329 sec latency 68.959 ms 2024-09-11T01:19:24.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:23 smithi110 ceph-mon[34712]: pgmap v965: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 740 op/s 2024-09-11T01:19:24.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:23 smithi110 ceph-mon[34712]: osdmap e105: 12 total, 12 up, 12 in 2024-09-11T01:19:24.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:23 smithi112 ceph-mon[35384]: pgmap v965: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 740 op/s 2024-09-11T01:19:24.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:23 smithi112 ceph-mon[35384]: osdmap e105: 12 total, 12 up, 12 in 2024-09-11T01:19:24.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:23 smithi078 ceph-mon[31315]: pgmap v965: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 740 op/s 2024-09-11T01:19:24.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:23 smithi078 ceph-mon[31315]: osdmap e105: 12 total, 12 up, 12 in 2024-09-11T01:19:24.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 285105 33.48 MB/sec execute 330 sec latency 77.683 ms 2024-09-11T01:19:25.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 285778 33.49 MB/sec execute 331 sec latency 139.936 ms 2024-09-11T01:19:26.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:25 smithi110 ceph-mon[34712]: pgmap v967: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 740 op/s 2024-09-11T01:19:26.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:25 smithi112 ceph-mon[35384]: pgmap v967: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 740 op/s 2024-09-11T01:19:26.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:25 smithi078 ceph-mon[31315]: pgmap v967: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 740 op/s 2024-09-11T01:19:26.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 286602 33.49 MB/sec execute 332 sec latency 109.003 ms 2024-09-11T01:19:27.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 287336 33.55 MB/sec execute 333 sec latency 91.697 ms 2024-09-11T01:19:28.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:27 smithi110 ceph-mon[34712]: pgmap v968: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 766 op/s 2024-09-11T01:19:28.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:27 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:19:28.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:27 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:19:28.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:27 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:19:28.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:27 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:19:28.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:27 smithi112 ceph-mon[35384]: pgmap v968: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 766 op/s 2024-09-11T01:19:28.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:27 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:19:28.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:27 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:19:28.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:27 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:19:28.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:27 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:19:28.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:27 smithi078 ceph-mon[31315]: pgmap v968: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 766 op/s 2024-09-11T01:19:28.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:27 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:19:28.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:27 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:19:28.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:27 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:19:28.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:27 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:19:28.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 287993 33.56 MB/sec execute 334 sec latency 69.686 ms 2024-09-11T01:19:29.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:28 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:19:29.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:28 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:19:29.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:28 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:19:29.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:28 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:19:29.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:28 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:19:29.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:28 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:19:29.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 288525 33.53 MB/sec execute 335 sec latency 168.538 ms 2024-09-11T01:19:30.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:29 smithi110 ceph-mon[34712]: pgmap v969: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 737 op/s 2024-09-11T01:19:30.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:29 smithi112 ceph-mon[35384]: pgmap v969: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 737 op/s 2024-09-11T01:19:30.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:29 smithi078 ceph-mon[31315]: pgmap v969: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 737 op/s 2024-09-11T01:19:30.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 289183 33.52 MB/sec execute 336 sec latency 98.892 ms 2024-09-11T01:19:31.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 289950 33.54 MB/sec execute 337 sec latency 79.689 ms 2024-09-11T01:19:32.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:31 smithi110 ceph-mon[34712]: pgmap v970: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 765 op/s 2024-09-11T01:19:32.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:31 smithi112 ceph-mon[35384]: pgmap v970: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 765 op/s 2024-09-11T01:19:32.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:31 smithi078 ceph-mon[31315]: pgmap v970: 97 pgs: 97 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 765 op/s 2024-09-11T01:19:32.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 290738 33.56 MB/sec execute 338 sec latency 81.574 ms 2024-09-11T01:19:33.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 291432 33.61 MB/sec execute 339 sec latency 157.804 ms 2024-09-11T01:19:34.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:33 smithi110 ceph-mon[34712]: pgmap v971: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 703 op/s 2024-09-11T01:19:34.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:33 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:19:34.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:33 smithi112 ceph-mon[35384]: pgmap v971: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 703 op/s 2024-09-11T01:19:34.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:33 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:19:34.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:33 smithi078 ceph-mon[31315]: pgmap v971: 97 pgs: 97 active+clean; 2.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 703 op/s 2024-09-11T01:19:34.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:33 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:19:34.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 291951 33.59 MB/sec execute 340 sec latency 94.843 ms 2024-09-11T01:19:34.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:34 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:19:35.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:34 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:19:35.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:34 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:19:35.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 292327 33.54 MB/sec execute 341 sec latency 174.947 ms 2024-09-11T01:19:36.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:35 smithi110 ceph-mon[34712]: pgmap v972: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 720 op/s 2024-09-11T01:19:36.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:35 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:36.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:35 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:36.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:35 smithi112 ceph-mon[35384]: pgmap v972: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 720 op/s 2024-09-11T01:19:36.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:35 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:36.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:35 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:36.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:35 smithi078 ceph-mon[31315]: pgmap v972: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 720 op/s 2024-09-11T01:19:36.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:35 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:36.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:35 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:36.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 292536 33.46 MB/sec execute 342 sec latency 434.258 ms 2024-09-11T01:19:37.798 INFO:tasks.workunit.client.0.smithi078.stdout: 10 292677 33.39 MB/sec execute 343 sec latency 1136.513 ms 2024-09-11T01:19:37.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:37 smithi112 ceph-mon[35384]: pgmap v973: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 652 op/s 2024-09-11T01:19:38.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:37 smithi110 ceph-mon[34712]: pgmap v973: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 652 op/s 2024-09-11T01:19:38.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:37 smithi078 ceph-mon[31315]: pgmap v973: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 652 op/s 2024-09-11T01:19:38.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 292801 33.32 MB/sec execute 344 sec latency 616.381 ms 2024-09-11T01:19:39.797 INFO:tasks.workunit.client.0.smithi078.stdout: 10 293179 33.28 MB/sec execute 345 sec latency 270.771 ms 2024-09-11T01:19:40.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:39 smithi110 ceph-mon[34712]: pgmap v974: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 560 op/s 2024-09-11T01:19:40.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:39 smithi112 ceph-mon[35384]: pgmap v974: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 560 op/s 2024-09-11T01:19:40.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:39 smithi078 ceph-mon[31315]: pgmap v974: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 560 op/s 2024-09-11T01:19:40.798 INFO:tasks.workunit.client.0.smithi078.stdout: 10 293670 33.25 MB/sec execute 346 sec latency 291.935 ms 2024-09-11T01:19:41.798 INFO:tasks.workunit.client.0.smithi078.stdout: 10 294103 33.23 MB/sec execute 347 sec latency 182.979 ms 2024-09-11T01:19:42.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:41 smithi110 ceph-mon[34712]: pgmap v975: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 675 op/s 2024-09-11T01:19:42.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:41 smithi112 ceph-mon[35384]: pgmap v975: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 675 op/s 2024-09-11T01:19:42.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:41 smithi078 ceph-mon[31315]: pgmap v975: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 675 op/s 2024-09-11T01:19:42.798 INFO:tasks.workunit.client.0.smithi078.stdout: 10 294475 33.19 MB/sec execute 348 sec latency 188.185 ms 2024-09-11T01:19:43.798 INFO:tasks.workunit.client.0.smithi078.stdout: 10 295019 33.19 MB/sec execute 349 sec latency 209.241 ms 2024-09-11T01:19:44.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:43 smithi110 ceph-mon[34712]: pgmap v976: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 643 op/s 2024-09-11T01:19:44.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:44.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:44.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:44.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:43 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:44.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:43 smithi112 ceph-mon[35384]: pgmap v976: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 643 op/s 2024-09-11T01:19:44.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:44.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:44.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:44.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:43 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:44.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:43 smithi078 ceph-mon[31315]: pgmap v976: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 643 op/s 2024-09-11T01:19:44.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:44.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:44.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:44.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:43 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:44.798 INFO:tasks.workunit.client.0.smithi078.stdout: 10 295693 33.20 MB/sec execute 350 sec latency 75.565 ms 2024-09-11T01:19:45.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:19:45.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:19:45.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:45.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:19:45.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:19:45.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:45.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:19:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:19:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:19:45.798 INFO:tasks.workunit.client.0.smithi078.stdout: 10 296413 33.19 MB/sec execute 351 sec latency 80.233 ms 2024-09-11T01:19:46.000 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:19:46.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:45 smithi110 ceph-mon[34712]: pgmap v977: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 609 op/s 2024-09-11T01:19:46.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:45 smithi112 ceph-mon[35384]: pgmap v977: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 609 op/s 2024-09-11T01:19:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:45 smithi078 ceph-mon[31315]: pgmap v977: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 609 op/s 2024-09-11T01:19:46.356 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:19:46.798 INFO:tasks.workunit.client.0.smithi078.stdout: 10 297214 33.24 MB/sec execute 352 sec latency 61.544 ms 2024-09-11T01:19:46.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.958+0000 7faec810a640 1 -- 172.21.15.78:0/3156662883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faec010d140 msgr2=0x7faec010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:46.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.958+0000 7faec810a640 1 --2- 172.21.15.78:0/3156662883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faec010d140 0x7faec010f550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faeb8013f30 tx=0x7faeb8032c80 comp rx=0 tx=0).stop 2024-09-11T01:19:46.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.958+0000 7faec810a640 1 -- 172.21.15.78:0/3156662883 shutdown_connections 2024-09-11T01:19:46.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.958+0000 7faec810a640 1 --2- 172.21.15.78:0/3156662883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faec010d140 0x7faec010f550 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.958+0000 7faec810a640 1 --2- 172.21.15.78:0/3156662883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faec0104590 0x7faec010cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.958+0000 7faec810a640 1 --2- 172.21.15.78:0/3156662883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faec0103b90 0x7faec0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.958+0000 7faec810a640 1 -- 172.21.15.78:0/3156662883 >> 172.21.15.78:0/3156662883 conn(0x7faec00fd700 msgr2=0x7faec00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.959+0000 7faec810a640 1 -- 172.21.15.78:0/3156662883 shutdown_connections 2024-09-11T01:19:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.959+0000 7faec810a640 1 -- 172.21.15.78:0/3156662883 wait complete. 2024-09-11T01:19:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.960+0000 7faec810a640 1 Processor -- start 2024-09-11T01:19:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.960+0000 7faec810a640 1 -- start start 2024-09-11T01:19:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.961+0000 7faec810a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faec0103b90 0x7faec00746e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.961+0000 7faec810a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faec0104590 0x7faec0074c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.961+0000 7faec810a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faec010d140 0x7faec006d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.961+0000 7faec810a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faec0110c50 con 0x7faec0104590 2024-09-11T01:19:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.961+0000 7faec810a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faec0110ad0 con 0x7faec010d140 2024-09-11T01:19:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.961+0000 7faec810a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faec0110dd0 con 0x7faec0103b90 2024-09-11T01:19:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.961+0000 7faec56a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faec0104590 0x7faec0074c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.961+0000 7faec56a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faec0104590 0x7faec0074c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54862/0 (socket says 172.21.15.78:54862) 2024-09-11T01:19:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.961+0000 7faec56a5640 1 -- 172.21.15.78:0/3132446213 learned_addr learned my addr 172.21.15.78:0/3132446213 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.962+0000 7faec56a5640 1 -- 172.21.15.78:0/3132446213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faec0103b90 msgr2=0x7faec00746e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T01:19:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.962+0000 7faec56a5640 1 --2- 172.21.15.78:0/3132446213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faec0103b90 0x7faec00746e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.962+0000 7faec56a5640 1 -- 172.21.15.78:0/3132446213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faec010d140 msgr2=0x7faec006d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.962+0000 7faec56a5640 1 --2- 172.21.15.78:0/3132446213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faec010d140 0x7faec006d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.962+0000 7faec56a5640 1 -- 172.21.15.78:0/3132446213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec006e270 con 0x7faec0104590 2024-09-11T01:19:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.962+0000 7faec56a5640 1 --2- 172.21.15.78:0/3132446213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faec0104590 0x7faec0074c20 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7faeb00098a0 tx=0x7faeb0009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.962+0000 7faeb6ffd640 1 -- 172.21.15.78:0/3132446213 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb000ea80 con 0x7faec0104590 2024-09-11T01:19:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.963+0000 7faeb6ffd640 1 -- 172.21.15.78:0/3132446213 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb00054d0 con 0x7faec0104590 2024-09-11T01:19:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.963+0000 7faeb6ffd640 1 -- 172.21.15.78:0/3132446213 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb0004cd0 con 0x7faec0104590 2024-09-11T01:19:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.963+0000 7faec810a640 1 -- 172.21.15.78:0/3132446213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec006e560 con 0x7faec0104590 2024-09-11T01:19:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.963+0000 7faec810a640 1 -- 172.21.15.78:0/3132446213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec0073520 con 0x7faec0104590 2024-09-11T01:19:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.969+0000 7faeca4df740 1 Processor -- start 2024-09-11T01:19:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.969+0000 7faeca4df740 1 -- start start 2024-09-11T01:19:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.969+0000 7faeca4df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d0a0 0x56276ec4d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.969+0000 7faeca4df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d9c0 0x56276ec59410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faeca4df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec59950 0x56276ec5bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faeca4df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56276ebc3e50 con 0x56276ec4d0a0 2024-09-11T01:19:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faeca4df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56276ebb3280 con 0x56276ec59950 2024-09-11T01:19:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faec66a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d0a0 0x56276ec4d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:46.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faec66a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d0a0 0x56276ec4d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54878/0 (socket says 172.21.15.78:54878) 2024-09-11T01:19:46.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faec66a7640 1 -- 172.21.15.78:0/1136500413 learned_addr learned my addr 172.21.15.78:0/1136500413 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:46.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faeca4df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56276ebb7090 con 0x56276ec4d9c0 2024-09-11T01:19:46.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faec5ea6640 1 --2- 172.21.15.78:0/1136500413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d9c0 0x56276ec59410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:46.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faec5ea6640 1 -- 172.21.15.78:0/1136500413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec59950 msgr2=0x56276ec5bd40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:19:46.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faec5ea6640 1 --2- 172.21.15.78:0/1136500413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec59950 0x56276ec5bd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faec5ea6640 1 -- 172.21.15.78:0/1136500413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d0a0 msgr2=0x56276ec4d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:46.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faec5ea6640 1 --2- 172.21.15.78:0/1136500413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d0a0 0x56276ec4d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faec5ea6640 1 -- 172.21.15.78:0/1136500413 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56276e911650 con 0x56276ec4d9c0 2024-09-11T01:19:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.970+0000 7faec5ea6640 1 --2- 172.21.15.78:0/1136500413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d9c0 0x56276ec59410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faebc00b9d0 tx=0x7faebc00be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523aab3bca968e4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.971+0000 7faeabfff640 1 -- 172.21.15.78:0/1136500413 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc016070 con 0x56276ec4d9c0 2024-09-11T01:19:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.971+0000 7faeabfff640 1 -- 172.21.15.78:0/1136500413 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faebc0054d0 con 0x56276ec4d9c0 2024-09-11T01:19:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.971+0000 7faeabfff640 1 -- 172.21.15.78:0/1136500413 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc004dd0 con 0x56276ec4d9c0 2024-09-11T01:19:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.972+0000 7faeca4df740 1 -- 172.21.15.78:0/1136500413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d9c0 msgr2=0x56276ec59410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.972+0000 7faeca4df740 1 --2- 172.21.15.78:0/1136500413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d9c0 0x56276ec59410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faebc00b9d0 tx=0x7faebc00be90 comp rx=0 tx=0).stop 2024-09-11T01:19:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.972+0000 7faeca4df740 1 -- 172.21.15.78:0/1136500413 shutdown_connections 2024-09-11T01:19:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.972+0000 7faeca4df740 1 --2- 172.21.15.78:0/1136500413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec59950 0x56276ec5bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.972+0000 7faeca4df740 1 --2- 172.21.15.78:0/1136500413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d9c0 0x56276ec59410 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.972+0000 7faeca4df740 1 --2- 172.21.15.78:0/1136500413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d0a0 0x56276ec4d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.972+0000 7faeca4df740 1 -- 172.21.15.78:0/1136500413 >> 172.21.15.78:0/1136500413 conn(0x56276ec4aed0 msgr2=0x56276ec4c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.973+0000 7faeb6ffd640 1 -- 172.21.15.78:0/3132446213 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7faeb000f040 con 0x7faec0104590 2024-09-11T01:19:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.973+0000 7faeca4df740 1 -- 172.21.15.78:0/1136500413 shutdown_connections 2024-09-11T01:19:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.973+0000 7faeca4df740 1 -- 172.21.15.78:0/1136500413 wait complete. 2024-09-11T01:19:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.973+0000 7faeca4df740 1 -- start start 2024-09-11T01:19:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.973+0000 7faeca4df740 1 Processor -- start 2024-09-11T01:19:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.974+0000 7faeca4df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d0a0 0x56276ec59320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.974+0000 7faeca4df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d9c0 0x56276ec56770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.974+0000 7faeca4df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec56dc0 0x56276ec57240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.974+0000 7faeca4df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56276ea14700 con 0x56276ec56dc0 2024-09-11T01:19:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.974+0000 7faeca4df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56276eb4e110 con 0x56276ec4d9c0 2024-09-11T01:19:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.974+0000 7faeca4df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56276ec6d680 con 0x56276ec4d0a0 2024-09-11T01:19:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.974+0000 7faeca4df740 10 client.0 _ll_register_callbacks cb 0x56276eb67990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec66a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d0a0 0x56276ec59320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec5ea6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d9c0 0x56276ec56770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec66a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d0a0 0x56276ec59320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54220/0 (socket says 172.21.15.78:54220) 2024-09-11T01:19:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec66a7640 1 -- 172.21.15.78:0/3018446105 learned_addr learned my addr 172.21.15.78:0/3018446105 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faeb6ffd640 1 --2- 172.21.15.78:0/3132446213 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faeac0b5530 0x7faeac0b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec56a5640 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec56dc0 0x56276ec57240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec5ea6640 1 -- 172.21.15.78:0/3018446105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d0a0 msgr2=0x56276ec59320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec5ea6640 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d0a0 0x56276ec59320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec5ea6640 1 -- 172.21.15.78:0/3018446105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec56dc0 msgr2=0x56276ec57240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec5ea6640 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec56dc0 0x56276ec57240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec5ea6640 1 -- 172.21.15.78:0/3018446105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56276eb7bc40 con 0x56276ec4d9c0 2024-09-11T01:19:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec56a5640 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec56dc0 0x56276ec57240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:19:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec66a7640 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d0a0 0x56276ec59320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:19:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec66a7640 1 --2- 172.21.15.78:0/3132446213 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faeac0b5530 0x7faeac0b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faec5ea6640 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d9c0 0x56276ec56770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faebc00b9d0 tx=0x7faebc0048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=911dd12db4ab200c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faea97fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:19:46.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.975+0000 7faeb6ffd640 1 -- 172.21.15.78:0/3132446213 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7faeb00e58b0 con 0x7faec0104590 2024-09-11T01:19:46.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.976+0000 7faea97fa640 1 -- 172.21.15.78:0/3018446105 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc016070 con 0x56276ec4d9c0 2024-09-11T01:19:46.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.976+0000 7faea97fa640 1 -- 172.21.15.78:0/3018446105 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7faebc00dbe0 con 0x56276ec4d9c0 2024-09-11T01:19:46.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.976+0000 7faeca4df740 1 -- 172.21.15.78:0/3018446105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faebc0054d0 con 0x56276ec4d9c0 2024-09-11T01:19:46.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.976+0000 7faec66a7640 1 --2- 172.21.15.78:0/3132446213 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faeac0b5530 0x7faeac0b79f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faeb8013f30 tx=0x7faeb8009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.977+0000 7faea97fa640 1 -- 172.21.15.78:0/3018446105 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7faebc004040 con 0x56276ec4d9c0 2024-09-11T01:19:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.977+0000 7faeca4df740 10 client.28115 fetch_fsmap learned FSMap version 72 2024-09-11T01:19:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.977+0000 7faeca4df740 1 -- 172.21.15.78:0/3018446105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x56276ec794b0 con 0x56276ec4d9c0 2024-09-11T01:19:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.978+0000 7faea97fa640 1 -- 172.21.15.78:0/3018446105 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faebc0054d0 con 0x56276ec4d9c0 2024-09-11T01:19:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.978+0000 7faea97fa640 1 -- 172.21.15.78:0/3018446105 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7faebc0036c0 con 0x56276ec4d9c0 2024-09-11T01:19:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.979+0000 7faeca4df740 10 client.28115 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:19:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.979+0000 7faeca4df740 10 client.28115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:19:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.979+0000 7faeca4df740 20 client.28115 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.979+0000 7faeca4df740 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56276ec7a860 0x56276ec7cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.979+0000 7faeca4df740 4 client.28115 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:19:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.979+0000 7faeca4df740 1 -- 172.21.15.78:0/3018446105 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56276ebc5da0 con 0x56276ec7a860 2024-09-11T01:19:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.979+0000 7faec56a5640 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56276ec7a860 0x56276ec7cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.981+0000 7faec56a5640 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56276ec7a860 0x56276ec7cc50 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48079f5cb8e9764 server_cookie=2d935c4514d0acfe in_seq=0 out_seq=0 2024-09-11T01:19:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.981+0000 7faea97fa640 10 client.28115 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.983+0000 7faea97fa640 1 -- 172.21.15.78:0/3018446105 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56276ebc5da0 con 0x56276ec7a860 2024-09-11T01:19:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.983+0000 7faea97fa640 10 client.28115 handle_command_reply: tid=0 2024-09-11T01:19:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.983+0000 7faeca4df740 1 client.28115 shutdown 2024-09-11T01:19:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.984+0000 7faeca4df740 1 -- 172.21.15.78:0/3018446105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d9c0 msgr2=0x56276ec56770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.984+0000 7faeca4df740 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d9c0 0x56276ec56770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faebc00b9d0 tx=0x7faebc0048f0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.984+0000 7faeca4df740 1 -- 172.21.15.78:0/3018446105 shutdown_connections 2024-09-11T01:19:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.984+0000 7faeca4df740 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56276ec7a860 0x56276ec7cc50 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.984+0000 7faeca4df740 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec56dc0 0x56276ec57240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.984+0000 7faeca4df740 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d9c0 0x56276ec56770 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.984+0000 7faeca4df740 1 --2- 172.21.15.78:0/3018446105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec4d0a0 0x56276ec59320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.984+0000 7faeca4df740 1 -- 172.21.15.78:0/3018446105 >> 172.21.15.78:0/3018446105 conn(0x56276ec4aed0 msgr2=0x56276ec50a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.984+0000 7faea97fa640 0 client.28115 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.984+0000 7faeca4df740 1 -- 172.21.15.78:0/3018446105 shutdown_connections 2024-09-11T01:19:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.985+0000 7faeca4df740 1 -- 172.21.15.78:0/3018446105 wait complete. 2024-09-11T01:19:46.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:46.985+0000 7faeca4df740 20 client.28115 trim_cache size 0 max 16384 2024-09-11T01:19:47.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.013+0000 7faeca4df740 1 Processor -- start 2024-09-11T01:19:47.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faeca4df740 1 -- start start 2024-09-11T01:19:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faeca4df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 0x56276ec50fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faeca4df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d9c0 0x56276ec45e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faeca4df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec56dc0 0x56276ec4bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faeca4df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b4d00 con 0x56276ec4d9c0 2024-09-11T01:19:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faeca4df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b6900 con 0x56276ec4d0a0 2024-09-11T01:19:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faeca4df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b53d0 con 0x56276ec56dc0 2024-09-11T01:19:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faec5ea6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 0x56276ec50fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faec66a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec56dc0 0x56276ec4bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faec5ea6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 0x56276ec50fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38078/0 (socket says 172.21.15.78:38078) 2024-09-11T01:19:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faec66a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec56dc0 0x56276ec4bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54232/0 (socket says 172.21.15.78:54232) 2024-09-11T01:19:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faec5ea6640 1 -- 172.21.15.78:0/875380572 learned_addr learned my addr 172.21.15.78:0/875380572 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.014+0000 7faec56a5640 1 --2- 172.21.15.78:0/875380572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d9c0 0x56276ec45e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.015+0000 7faec5ea6640 1 -- 172.21.15.78:0/875380572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec56dc0 msgr2=0x56276ec4bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.015+0000 7faec5ea6640 1 --2- 172.21.15.78:0/875380572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec56dc0 0x56276ec4bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.015+0000 7faec5ea6640 1 -- 172.21.15.78:0/875380572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d9c0 msgr2=0x56276ec45e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.015+0000 7faec5ea6640 1 --2- 172.21.15.78:0/875380572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d9c0 0x56276ec45e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.015+0000 7faec5ea6640 1 -- 172.21.15.78:0/875380572 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56276ec54c70 con 0x56276ec4d0a0 2024-09-11T01:19:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.015+0000 7faec56a5640 1 --2- 172.21.15.78:0/875380572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d9c0 0x56276ec45e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:19:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.015+0000 7faec66a7640 1 --2- 172.21.15.78:0/875380572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec56dc0 0x56276ec4bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:19:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.015+0000 7faec5ea6640 1 --2- 172.21.15.78:0/875380572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 0x56276ec50fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faebc010480 tx=0x7faebc005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a65b73756286f6a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.015+0000 7faeab7fe640 1 -- 172.21.15.78:0/875380572 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc032030 con 0x56276ec4d0a0 2024-09-11T01:19:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.015+0000 7faeab7fe640 1 -- 172.21.15.78:0/875380572 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faebc028020 con 0x56276ec4d0a0 2024-09-11T01:19:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.015+0000 7faeab7fe640 1 -- 172.21.15.78:0/875380572 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc016070 con 0x56276ec4d0a0 2024-09-11T01:19:47.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.016+0000 7faeca4df740 1 -- 172.21.15.78:0/875380572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 msgr2=0x56276ec50fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.016+0000 7faeca4df740 1 --2- 172.21.15.78:0/875380572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 0x56276ec50fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faebc010480 tx=0x7faebc005c50 comp rx=0 tx=0).stop 2024-09-11T01:19:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.016+0000 7faeca4df740 1 -- 172.21.15.78:0/875380572 shutdown_connections 2024-09-11T01:19:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.016+0000 7faeca4df740 1 --2- 172.21.15.78:0/875380572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec56dc0 0x56276ec4bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.016+0000 7faeca4df740 1 --2- 172.21.15.78:0/875380572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d9c0 0x56276ec45e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.016+0000 7faeca4df740 1 --2- 172.21.15.78:0/875380572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 0x56276ec50fe0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.016+0000 7faeca4df740 1 -- 172.21.15.78:0/875380572 >> 172.21.15.78:0/875380572 conn(0x56276ec4aed0 msgr2=0x56276ec51960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.016+0000 7faeca4df740 1 -- 172.21.15.78:0/875380572 shutdown_connections 2024-09-11T01:19:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.017+0000 7faeca4df740 1 -- 172.21.15.78:0/875380572 wait complete. 2024-09-11T01:19:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.017+0000 7faeca4df740 1 -- start start 2024-09-11T01:19:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.017+0000 7faeca4df740 1 Processor -- start 2024-09-11T01:19:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faeca4df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 0x56276ec81ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faeca4df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d9c0 0x56276ec820e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faec5ea6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 0x56276ec81ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faec5ea6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 0x56276ec81ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38088/0 (socket says 172.21.15.78:38088) 2024-09-11T01:19:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faec5ea6640 1 -- 172.21.15.78:0/3421541160 learned_addr learned my addr 172.21.15.78:0/3421541160 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faeca4df740 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec56dc0 0x56276ec71940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faeca4df740 1 -- 172.21.15.78:0/3421541160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b6230 con 0x56276ec4d9c0 2024-09-11T01:19:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faec56a5640 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d9c0 0x56276ec820e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faeca4df740 1 -- 172.21.15.78:0/3421541160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b5aa0 con 0x56276ec4d0a0 2024-09-11T01:19:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faeca4df740 1 -- 172.21.15.78:0/3421541160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b2d00 con 0x56276ec56dc0 2024-09-11T01:19:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faeca4df740 10 client.0 _ll_register_callbacks cb 0x7faeb0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faec5ea6640 1 -- 172.21.15.78:0/3421541160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec56dc0 msgr2=0x56276ec71940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:19:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faec5ea6640 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec56dc0 0x56276ec71940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faec5ea6640 1 -- 172.21.15.78:0/3421541160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d9c0 msgr2=0x56276ec820e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faec5ea6640 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d9c0 0x56276ec820e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.018+0000 7faec5ea6640 1 -- 172.21.15.78:0/3421541160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56276ec71f00 con 0x56276ec4d0a0 2024-09-11T01:19:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.019+0000 7faec5ea6640 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 0x56276ec81ba0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faebc00bed0 tx=0x7faebc003910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed0a2a64b8a573d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.019+0000 7faea97fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:19:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.019+0000 7faea97fa640 1 -- 172.21.15.78:0/3421541160 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc03c060 con 0x56276ec4d0a0 2024-09-11T01:19:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.019+0000 7faea97fa640 1 -- 172.21.15.78:0/3421541160 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7faebc004910 con 0x56276ec4d0a0 2024-09-11T01:19:47.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.019+0000 7faeca4df740 1 -- 172.21.15.78:0/3421541160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faebc028020 con 0x56276ec4d0a0 2024-09-11T01:19:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.019+0000 7faea97fa640 1 -- 172.21.15.78:0/3421541160 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7faebc016040 con 0x56276ec4d0a0 2024-09-11T01:19:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.019+0000 7faeca4df740 10 client.28121 fetch_fsmap learned FSMap version 72 2024-09-11T01:19:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.019+0000 7faeca4df740 1 -- 172.21.15.78:0/3421541160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x56276ec73250 con 0x56276ec4d0a0 2024-09-11T01:19:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.020+0000 7faea97fa640 1 -- 172.21.15.78:0/3421541160 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faebc01e070 con 0x56276ec4d0a0 2024-09-11T01:19:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.020+0000 7faea97fa640 1 -- 172.21.15.78:0/3421541160 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7faebc0051a0 con 0x56276ec4d0a0 2024-09-11T01:19:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.020+0000 7faeca4df740 10 client.28121 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:19:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.020+0000 7faeca4df740 10 client.28121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:19:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.020+0000 7faeca4df740 20 client.28121 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.020+0000 7faeca4df740 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56276ece75c0 0x56276ece99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:47.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.020+0000 7faeca4df740 4 client.28121 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:19:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.020+0000 7faeca4df740 1 -- 172.21.15.78:0/3421541160 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7faebc028020 con 0x56276ece75c0 2024-09-11T01:19:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.020+0000 7faec56a5640 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56276ece75c0 0x56276ece99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.020+0000 7faea97fa640 10 client.28121 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.020+0000 7faec56a5640 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56276ece75c0 0x56276ece99b0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c981562b6e94d9b server_cookie=7c622ac03042a4fa in_seq=0 out_seq=0 2024-09-11T01:19:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.021+0000 7faea97fa640 1 -- 172.21.15.78:0/3421541160 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7190 (crc 0 0 0) 0x7faebc028020 con 0x56276ece75c0 2024-09-11T01:19:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.021+0000 7faea97fa640 10 client.28121 handle_command_reply: tid=0 2024-09-11T01:19:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.021+0000 7faeca4df740 1 client.28121 shutdown 2024-09-11T01:19:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.021+0000 7faeca4df740 1 -- 172.21.15.78:0/3421541160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 msgr2=0x56276ec81ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.021+0000 7faeca4df740 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 0x56276ec81ba0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faebc00bed0 tx=0x7faebc003910 comp rx=0 tx=0).stop 2024-09-11T01:19:47.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.021+0000 7faeca4df740 1 -- 172.21.15.78:0/3421541160 shutdown_connections 2024-09-11T01:19:47.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.021+0000 7faeca4df740 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56276ec56dc0 0x56276ec71940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.021+0000 7faeca4df740 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x56276ece75c0 0x56276ece99b0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.021+0000 7faea97fa640 0 client.28121 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:19:47.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.021+0000 7faeca4df740 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56276ec4d9c0 0x56276ec820e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.021+0000 7faeca4df740 1 --2- 172.21.15.78:0/3421541160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56276ec4d0a0 0x56276ec81ba0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.021+0000 7faeca4df740 1 -- 172.21.15.78:0/3421541160 >> 172.21.15.78:0/3421541160 conn(0x56276ec4aed0 msgr2=0x56276ec4bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:47.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.022+0000 7faeca4df740 1 -- 172.21.15.78:0/3421541160 shutdown_connections 2024-09-11T01:19:47.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.022+0000 7faeca4df740 1 -- 172.21.15.78:0/3421541160 wait complete. 2024-09-11T01:19:47.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.022+0000 7faeca4df740 20 client.28121 trim_cache size 0 max 16384 2024-09-11T01:19:47.032 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:19:47.032 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:19:47.032 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:19:47.032 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:19:47.032 INFO:teuthology.orchestra.run.smithi078.stdout: "b41a2b77-b2b1-4f1b-8588-4f32a78b721f": { 2024-09-11T01:19:47.032 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:19:13.257610+0000", 2024-09-11T01:19:47.033 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.033 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.033 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.033 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.033 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.033 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:19:47.033 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:18:17.654570+0000", 2024-09-11T01:19:47.033 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.033 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.033 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.033 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:19:47.034 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.036 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:19:47.037 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.038 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.039 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.040 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.041 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:19:47.042 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:19:47.043 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.045 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:19:47.046 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:19:47.047 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:19:47.047 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:19:47.047 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:19:47.047 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:19:47.047 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:19:47.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.022+0000 7faec810a640 1 -- 172.21.15.78:0/3132446213 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faeac0b5530 msgr2=0x7faeac0b79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:47.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.022+0000 7faec810a640 1 --2- 172.21.15.78:0/3132446213 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faeac0b5530 0x7faeac0b79f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faeb8013f30 tx=0x7faeb8009500 comp rx=0 tx=0).stop 2024-09-11T01:19:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.022+0000 7faec810a640 1 -- 172.21.15.78:0/3132446213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faec0104590 msgr2=0x7faec0074c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.022+0000 7faec810a640 1 --2- 172.21.15.78:0/3132446213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faec0104590 0x7faec0074c20 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7faeb00098a0 tx=0x7faeb0009d60 comp rx=0 tx=0).stop 2024-09-11T01:19:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.023+0000 7faec810a640 1 -- 172.21.15.78:0/3132446213 shutdown_connections 2024-09-11T01:19:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.023+0000 7faec810a640 1 --2- 172.21.15.78:0/3132446213 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faeac0b5530 0x7faeac0b79f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.023+0000 7faec810a640 1 --2- 172.21.15.78:0/3132446213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faec010d140 0x7faec006d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.023+0000 7faec810a640 1 --2- 172.21.15.78:0/3132446213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faec0104590 0x7faec0074c20 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.023+0000 7faec810a640 1 --2- 172.21.15.78:0/3132446213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faec0103b90 0x7faec00746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.023+0000 7faec810a640 1 -- 172.21.15.78:0/3132446213 >> 172.21.15.78:0/3132446213 conn(0x7faec00fd700 msgr2=0x7faec00fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.023+0000 7faec810a640 1 -- 172.21.15.78:0/3132446213 shutdown_connections 2024-09-11T01:19:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:47.023+0000 7faec810a640 1 -- 172.21.15.78:0/3132446213 wait complete. 2024-09-11T01:19:47.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 297878 33.26 MB/sec execute 353 sec latency 96.917 ms 2024-09-11T01:19:47.984 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:19:47.984 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:19:48.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:47 smithi110 ceph-mon[34712]: pgmap v978: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 641 op/s 2024-09-11T01:19:48.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:47 smithi112 ceph-mon[35384]: pgmap v978: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 641 op/s 2024-09-11T01:19:48.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:47 smithi078 ceph-mon[31315]: pgmap v978: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 641 op/s 2024-09-11T01:19:48.316 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:19:48.798 INFO:tasks.workunit.client.0.smithi078.stdout: 10 298418 33.21 MB/sec execute 354 sec latency 142.877 ms 2024-09-11T01:19:48.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.906+0000 7f99f0022640 1 -- 172.21.15.78:0/1062005165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e81076a0 msgr2=0x7f99e8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.906+0000 7f99f0022640 1 --2- 172.21.15.78:0/1062005165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e81076a0 0x7f99e8107ae0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009900 tx=0x7f99d802a410 comp rx=0 tx=0).stop 2024-09-11T01:19:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.907+0000 7f99f0022640 1 -- 172.21.15.78:0/1062005165 shutdown_connections 2024-09-11T01:19:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.907+0000 7f99f0022640 1 --2- 172.21.15.78:0/1062005165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99e8108020 0x7f99e810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.907+0000 7f99f0022640 1 --2- 172.21.15.78:0/1062005165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e81076a0 0x7f99e8107ae0 secure :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009900 tx=0x7f99d802a410 comp rx=0 tx=0).stop 2024-09-11T01:19:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.907+0000 7f99f0022640 1 --2- 172.21.15.78:0/1062005165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e8101c90 0x7f99e8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.907+0000 7f99f0022640 1 -- 172.21.15.78:0/1062005165 >> 172.21.15.78:0/1062005165 conn(0x7f99e80fd700 msgr2=0x7f99e80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.907+0000 7f99f0022640 1 -- 172.21.15.78:0/1062005165 shutdown_connections 2024-09-11T01:19:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.907+0000 7f99f0022640 1 -- 172.21.15.78:0/1062005165 wait complete. 2024-09-11T01:19:48.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.908+0000 7f99f0022640 1 Processor -- start 2024-09-11T01:19:48.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.908+0000 7f99f0022640 1 -- start start 2024-09-11T01:19:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.908+0000 7f99f0022640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99e8101c90 0x7f99e810e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.908+0000 7f99f0022640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e8108020 0x7f99e810b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.908+0000 7f99f0022640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e810bba0 0x7f99e810c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.908+0000 7f99f0022640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8110b60 con 0x7f99e8108020 2024-09-11T01:19:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.908+0000 7f99f0022640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f99e81109e0 con 0x7f99e8101c90 2024-09-11T01:19:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.908+0000 7f99f0022640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8110ce0 con 0x7f99e810bba0 2024-09-11T01:19:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.908+0000 7f99ed5bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e8108020 0x7f99e810b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.909+0000 7f99ed5bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e8108020 0x7f99e810b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54924/0 (socket says 172.21.15.78:54924) 2024-09-11T01:19:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.909+0000 7f99ed5bd640 1 -- 172.21.15.78:0/1204643591 learned_addr learned my addr 172.21.15.78:0/1204643591 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.909+0000 7f99ee5bf640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e810bba0 0x7f99e810c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.909+0000 7f99ee5bf640 1 -- 172.21.15.78:0/1204643591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99e8101c90 msgr2=0x7f99e810e410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T01:19:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.909+0000 7f99ee5bf640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99e8101c90 0x7f99e810e410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.909+0000 7f99ee5bf640 1 -- 172.21.15.78:0/1204643591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e8108020 msgr2=0x7f99e810b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.909+0000 7f99ee5bf640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e8108020 0x7f99e810b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.909+0000 7f99ee5bf640 1 -- 172.21.15.78:0/1204643591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e8073c40 con 0x7f99e810bba0 2024-09-11T01:19:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.910+0000 7f99ee5bf640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e810bba0 0x7f99e810c050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f99e000ec50 tx=0x7f99e000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.910+0000 7f99deffd640 1 -- 172.21.15.78:0/1204643591 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0003b70 con 0x7f99e810bba0 2024-09-11T01:19:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.910+0000 7f99f0022640 1 -- 172.21.15.78:0/1204643591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e8073f30 con 0x7f99e810bba0 2024-09-11T01:19:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.910+0000 7f99deffd640 1 -- 172.21.15.78:0/1204643591 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99e0003d10 con 0x7f99e810bba0 2024-09-11T01:19:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.910+0000 7f99f0022640 1 -- 172.21.15.78:0/1204643591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e8074440 con 0x7f99e810bba0 2024-09-11T01:19:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.910+0000 7f99deffd640 1 -- 172.21.15.78:0/1204643591 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0010850 con 0x7f99e810bba0 2024-09-11T01:19:48.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.912+0000 7f99f0022640 1 -- 172.21.15.78:0/1204643591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99cc005290 con 0x7f99e810bba0 2024-09-11T01:19:48.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.913+0000 7f99deffd640 1 -- 172.21.15.78:0/1204643591 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f99e0046b60 con 0x7f99e810bba0 2024-09-11T01:19:48.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.914+0000 7f99deffd640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f99d40b5550 0x7f99d40b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:48.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.914+0000 7f99deffd640 1 -- 172.21.15.78:0/1204643591 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7f99e00e6460 con 0x7f99e810bba0 2024-09-11T01:19:48.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.914+0000 7f99eddbe640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f99d40b5550 0x7f99d40b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:48.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.915+0000 7f99eddbe640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f99d40b5550 0x7f99d40b7a10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f99e400b990 tx=0x7f99e400b370 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:48.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:48.920+0000 7f99deffd640 1 -- 172.21.15.78:0/1204643591 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f99e00acfc0 con 0x7f99e810bba0 2024-09-11T01:19:49.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.284+0000 7f99f0022640 1 -- 172.21.15.78:0/1204643591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99cc005580 con 0x7f99e810bba0 2024-09-11T01:19:49.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.285+0000 7f99deffd640 1 -- 172.21.15.78:0/1204643591 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+11986 (secure 0 0 0) 0x7f99e00ac8c0 con 0x7f99e810bba0 2024-09-11T01:19:49.287 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:19:49.289 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":72,"btime":"2024-09-11T01:19:32:815193+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":72,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:19:32.242847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:19:49.289 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 72 2024-09-11T01:19:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.290+0000 7f99f0022640 1 -- 172.21.15.78:0/1204643591 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f99d40b5550 msgr2=0x7f99d40b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.290+0000 7f99f0022640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f99d40b5550 0x7f99d40b7a10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f99e400b990 tx=0x7f99e400b370 comp rx=0 tx=0).stop 2024-09-11T01:19:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.290+0000 7f99f0022640 1 -- 172.21.15.78:0/1204643591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e810bba0 msgr2=0x7f99e810c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.290+0000 7f99f0022640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e810bba0 0x7f99e810c050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f99e000ec50 tx=0x7f99e000c6a0 comp rx=0 tx=0).stop 2024-09-11T01:19:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.290+0000 7f99f0022640 1 -- 172.21.15.78:0/1204643591 shutdown_connections 2024-09-11T01:19:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.290+0000 7f99f0022640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f99d40b5550 0x7f99d40b7a10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.290+0000 7f99f0022640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e810bba0 0x7f99e810c050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.290+0000 7f99f0022640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99e8108020 0x7f99e810b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.290+0000 7f99f0022640 1 --2- 172.21.15.78:0/1204643591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99e8101c90 0x7f99e810e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.290+0000 7f99f0022640 1 -- 172.21.15.78:0/1204643591 >> 172.21.15.78:0/1204643591 conn(0x7f99e80fd700 msgr2=0x7f99e80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:49.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.290+0000 7f99f0022640 1 -- 172.21.15.78:0/1204643591 shutdown_connections 2024-09-11T01:19:49.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:49.291+0000 7f99f0022640 1 -- 172.21.15.78:0/1204643591 wait complete. 2024-09-11T01:19:49.798 INFO:tasks.workunit.client.0.smithi078.stdout: 10 299144 33.25 MB/sec execute 355 sec latency 82.141 ms 2024-09-11T01:19:49.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:49 smithi078 ceph-mon[31315]: pgmap v979: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 15 MiB/s wr, 610 op/s 2024-09-11T01:19:49.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:49 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1204643591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:19:49.974 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:19:50.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:49 smithi110 ceph-mon[34712]: pgmap v979: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 15 MiB/s wr, 610 op/s 2024-09-11T01:19:50.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:49 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1204643591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:19:50.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:49 smithi112 ceph-mon[35384]: pgmap v979: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 15 MiB/s wr, 610 op/s 2024-09-11T01:19:50.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:49 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1204643591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:19:50.313 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:19:50.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 299889 33.27 MB/sec execute 356 sec latency 85.539 ms 2024-09-11T01:19:50.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.842+0000 7fc07d0c8640 1 -- 172.21.15.78:0/2763688581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc078103b70 msgr2=0x7fc078103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:50.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.842+0000 7fc07d0c8640 1 --2- 172.21.15.78:0/2763688581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc078103b70 0x7fc078103f50 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fc068009900 tx=0x7fc06802a410 comp rx=0 tx=0).stop 2024-09-11T01:19:50.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.842+0000 7fc07d0c8640 1 -- 172.21.15.78:0/2763688581 shutdown_connections 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.842+0000 7fc07d0c8640 1 --2- 172.21.15.78:0/2763688581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07810d120 0x7fc07810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.842+0000 7fc07d0c8640 1 --2- 172.21.15.78:0/2763688581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc078104570 0x7fc07810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.842+0000 7fc07d0c8640 1 --2- 172.21.15.78:0/2763688581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc078103b70 0x7fc078103f50 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.842+0000 7fc07d0c8640 1 -- 172.21.15.78:0/2763688581 >> 172.21.15.78:0/2763688581 conn(0x7fc0780fd700 msgr2=0x7fc0780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.842+0000 7fc07d0c8640 1 -- 172.21.15.78:0/2763688581 shutdown_connections 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.842+0000 7fc07d0c8640 1 -- 172.21.15.78:0/2763688581 wait complete. 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.843+0000 7fc07d0c8640 1 Processor -- start 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.843+0000 7fc07d0c8640 1 -- start start 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07d0c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc078103b70 0x7fc0781019f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07d0c8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc078104570 0x7fc078101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07d0c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07810d120 0x7fc078102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07d0c8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc078112d60 con 0x7fc07810d120 2024-09-11T01:19:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07d0c8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc078112be0 con 0x7fc078104570 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07d0c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc078112ee0 con 0x7fc078103b70 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc078104570 0x7fc078101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc076d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc078103b70 0x7fc0781019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc078104570 0x7fc078101f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38128/0 (socket says 172.21.15.78:38128) 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07659c640 1 -- 172.21.15.78:0/246455267 learned_addr learned my addr 172.21.15.78:0/246455267 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07759e640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07810d120 0x7fc078102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07659c640 1 -- 172.21.15.78:0/246455267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc078103b70 msgr2=0x7fc0781019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07659c640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc078103b70 0x7fc0781019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07659c640 1 -- 172.21.15.78:0/246455267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07810d120 msgr2=0x7fc078102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07659c640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07810d120 0x7fc078102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07659c640 1 -- 172.21.15.78:0/246455267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0781a7650 con 0x7fc078104570 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.844+0000 7fc07759e640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07810d120 0x7fc078102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:19:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.845+0000 7fc07659c640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc078104570 0x7fc078101f30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00e9f0 tx=0x7fc06c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.845+0000 7fc05ffff640 1 -- 172.21.15.78:0/246455267 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00cd80 con 0x7fc078104570 2024-09-11T01:19:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.845+0000 7fc07d0c8640 1 -- 172.21.15.78:0/246455267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0781a7940 con 0x7fc078104570 2024-09-11T01:19:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.845+0000 7fc05ffff640 1 -- 172.21.15.78:0/246455267 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc06c0108e0 con 0x7fc078104570 2024-09-11T01:19:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.845+0000 7fc05ffff640 1 -- 172.21.15.78:0/246455267 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c018980 con 0x7fc078104570 2024-09-11T01:19:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.845+0000 7fc07d0c8640 1 -- 172.21.15.78:0/246455267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0781a7ed0 con 0x7fc078104570 2024-09-11T01:19:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.848+0000 7fc05ffff640 1 -- 172.21.15.78:0/246455267 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc06c010430 con 0x7fc078104570 2024-09-11T01:19:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc07f49d740 1 Processor -- start 2024-09-11T01:19:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc07f49d740 1 -- start start 2024-09-11T01:19:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc07f49d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559760f0 0x5627559764d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc07f49d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x562755982460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc05ffff640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc0600b54d0 0x7fc0600b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc07f49d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559829a0 0x562755984d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc07f49d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5627558ecea0 con 0x5627559829a0 2024-09-11T01:19:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc07f49d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5627558dc2d0 con 0x562755976a10 2024-09-11T01:19:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc07759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559760f0 0x5627559764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc07759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559760f0 0x5627559764d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54274/0 (socket says 172.21.15.78:54274) 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc07759e640 1 -- 172.21.15.78:0/532616069 learned_addr learned my addr 172.21.15.78:0/532616069 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc076d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x562755982460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.849+0000 7fc07f49d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5627558e00e0 con 0x5627559760f0 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.850+0000 7fc05ffff640 1 -- 172.21.15.78:0/246455267 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7fc06c0e6270 con 0x7fc078104570 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.850+0000 7fc07759e640 1 --2- 172.21.15.78:0/532616069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559829a0 0x562755984d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.850+0000 7fc07659c640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc0600b54d0 0x7fc0600b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.850+0000 7fc07659c640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc0600b54d0 0x7fc0600b7990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0ac4e0 tx=0x7fc06c009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.850+0000 7fc07759e640 1 -- 172.21.15.78:0/532616069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559760f0 msgr2=0x5627559764d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.850+0000 7fc07759e640 1 --2- 172.21.15.78:0/532616069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559760f0 0x5627559764d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.850+0000 7fc07759e640 1 -- 172.21.15.78:0/532616069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 msgr2=0x562755982460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.850+0000 7fc07759e640 1 --2- 172.21.15.78:0/532616069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x562755982460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.850+0000 7fc07759e640 1 -- 172.21.15.78:0/532616069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56275563a650 con 0x5627559829a0 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.850+0000 7fc076d9d640 1 --2- 172.21.15.78:0/532616069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x562755982460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.850+0000 7fc07759e640 1 --2- 172.21.15.78:0/532616069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559829a0 0x562755984d90 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc070009fd0 tx=0x7fc070009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cafddc4a96fb63 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.851+0000 7fc05d7fa640 1 -- 172.21.15.78:0/532616069 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07000fd10 con 0x5627559829a0 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.851+0000 7fc05d7fa640 1 -- 172.21.15.78:0/532616069 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc070003970 con 0x5627559829a0 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.851+0000 7fc05d7fa640 1 -- 172.21.15.78:0/532616069 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070004290 con 0x5627559829a0 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.852+0000 7fc07f49d740 1 -- 172.21.15.78:0/532616069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559829a0 msgr2=0x562755984d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.852+0000 7fc07f49d740 1 --2- 172.21.15.78:0/532616069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559829a0 0x562755984d90 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc070009fd0 tx=0x7fc070009500 comp rx=0 tx=0).stop 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.852+0000 7fc07f49d740 1 -- 172.21.15.78:0/532616069 shutdown_connections 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.852+0000 7fc07f49d740 1 --2- 172.21.15.78:0/532616069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559829a0 0x562755984d90 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.852+0000 7fc07f49d740 1 --2- 172.21.15.78:0/532616069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x562755982460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.852+0000 7fc07f49d740 1 --2- 172.21.15.78:0/532616069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559760f0 0x5627559764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.852+0000 7fc07f49d740 1 -- 172.21.15.78:0/532616069 >> 172.21.15.78:0/532616069 conn(0x562755973f20 msgr2=0x5627559750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.852+0000 7fc07f49d740 1 -- 172.21.15.78:0/532616069 shutdown_connections 2024-09-11T01:19:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.852+0000 7fc07f49d740 1 -- 172.21.15.78:0/532616069 wait complete. 2024-09-11T01:19:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.852+0000 7fc07f49d740 1 -- start start 2024-09-11T01:19:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.852+0000 7fc07f49d740 1 Processor -- start 2024-09-11T01:19:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07f49d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 0x562755982250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07f49d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x56275597f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07f49d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 0x56275597fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07f49d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56275573d700 con 0x5627559760f0 2024-09-11T01:19:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07f49d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562755877160 con 0x562755976a10 2024-09-11T01:19:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07f49d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562755985620 con 0x5627559829a0 2024-09-11T01:19:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07f49d740 10 client.0 _ll_register_callbacks cb 0x5627558909e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 0x562755982250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 0x562755982250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54972/0 (socket says 172.21.15.78:54972) 2024-09-11T01:19:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07759e640 1 -- 172.21.15.78:0/1909897045 learned_addr learned my addr 172.21.15.78:0/1909897045 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc076d9d640 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x56275597f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07759e640 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 0x56275597fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07759e640 1 -- 172.21.15.78:0/1909897045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 msgr2=0x56275597fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07759e640 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 0x56275597fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07759e640 1 -- 172.21.15.78:0/1909897045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 msgr2=0x56275597f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07759e640 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x56275597f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.853+0000 7fc07759e640 1 -- 172.21.15.78:0/1909897045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627558a4c90 con 0x5627559760f0 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.854+0000 7fc07759e640 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 0x562755982250 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc070005f40 tx=0x7fc0700021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c731bba4076e2a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.854+0000 7fc05aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.854+0000 7fc05aff5640 1 -- 172.21.15.78:0/1909897045 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070026040 con 0x5627559760f0 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.854+0000 7fc05aff5640 1 -- 172.21.15.78:0/1909897045 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7fc070004000 con 0x5627559760f0 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.854+0000 7fc07f49d740 1 -- 172.21.15.78:0/1909897045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc070003970 con 0x5627559760f0 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.855+0000 7fc05aff5640 1 -- 172.21.15.78:0/1909897045 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fc070034050 con 0x5627559760f0 2024-09-11T01:19:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.855+0000 7fc07f49d740 10 client.19074 fetch_fsmap learned FSMap version 72 2024-09-11T01:19:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.855+0000 7fc07f49d740 1 -- 172.21.15.78:0/1909897045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x562755990f40 con 0x5627559760f0 2024-09-11T01:19:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.855+0000 7fc05aff5640 1 -- 172.21.15.78:0/1909897045 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc070003970 con 0x5627559760f0 2024-09-11T01:19:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.855+0000 7fc05aff5640 1 -- 172.21.15.78:0/1909897045 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7fc07001d030 con 0x5627559760f0 2024-09-11T01:19:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.855+0000 7fc07f49d740 10 client.19074 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:19:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.855+0000 7fc07f49d740 10 client.19074 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:19:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.855+0000 7fc07f49d740 20 client.19074 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.855+0000 7fc07f49d740 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x562755992290 0x562755994680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.855+0000 7fc07f49d740 4 client.19074 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:19:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.855+0000 7fc07f49d740 1 -- 172.21.15.78:0/1909897045 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627558eedf0 con 0x562755992290 2024-09-11T01:19:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.855+0000 7fc076d9d640 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x562755992290 0x562755994680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.856+0000 7fc05aff5640 10 client.19074 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:19:50.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.856+0000 7fc076d9d640 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x562755992290 0x562755994680 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b799f230d97368a server_cookie=5bd19122c7da41e6 in_seq=0 out_seq=0 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc05aff5640 1 -- 172.21.15.78:0/1909897045 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5627558eedf0 con 0x562755992290 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc05aff5640 10 client.19074 handle_command_reply: tid=0 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc07f49d740 1 client.19074 shutdown 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc07f49d740 1 -- 172.21.15.78:0/1909897045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 msgr2=0x562755982250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc07f49d740 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 0x562755982250 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc070005f40 tx=0x7fc0700021d0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc07f49d740 1 -- 172.21.15.78:0/1909897045 shutdown_connections 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc07f49d740 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x562755992290 0x562755994680 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc05aff5640 0 client.19074 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc07f49d740 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 0x56275597fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc07f49d740 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x56275597f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc07f49d740 1 --2- 172.21.15.78:0/1909897045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 0x562755982250 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc07f49d740 1 -- 172.21.15.78:0/1909897045 >> 172.21.15.78:0/1909897045 conn(0x562755973f20 msgr2=0x562755979a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc07f49d740 1 -- 172.21.15.78:0/1909897045 shutdown_connections 2024-09-11T01:19:50.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.857+0000 7fc07f49d740 1 -- 172.21.15.78:0/1909897045 wait complete. 2024-09-11T01:19:50.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.858+0000 7fc07f49d740 20 client.19074 trim_cache size 0 max 16384 2024-09-11T01:19:50.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc07f49d740 1 Processor -- start 2024-09-11T01:19:50.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc07f49d740 1 -- start start 2024-09-11T01:19:50.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc07f49d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 0x562755973760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc07f49d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x5627559813e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc07f49d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 0x562755983d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc07f49d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781ac4b0 con 0x5627559760f0 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc07f49d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781ae0b0 con 0x562755976a10 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc07f49d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781acb80 con 0x5627559829a0 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc07759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 0x562755973760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc07759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 0x562755973760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54980/0 (socket says 172.21.15.78:54980) 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc07759e640 1 -- 172.21.15.78:0/3821985977 learned_addr learned my addr 172.21.15.78:0/3821985977 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc076d9d640 1 --2- 172.21.15.78:0/3821985977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x5627559813e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.875+0000 7fc07759e640 1 --2- 172.21.15.78:0/3821985977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 0x562755983d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.876+0000 7fc076d9d640 1 -- 172.21.15.78:0/3821985977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 msgr2=0x562755983d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.876+0000 7fc076d9d640 1 --2- 172.21.15.78:0/3821985977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 0x562755983d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.876+0000 7fc076d9d640 1 -- 172.21.15.78:0/3821985977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 msgr2=0x562755973760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:50.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.876+0000 7fc076d9d640 1 --2- 172.21.15.78:0/3821985977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 0x562755973760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.876+0000 7fc076d9d640 1 -- 172.21.15.78:0/3821985977 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56275597ffd0 con 0x562755976a10 2024-09-11T01:19:50.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.876+0000 7fc07759e640 1 --2- 172.21.15.78:0/3821985977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 0x562755973760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:19:50.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.876+0000 7fc076d9d640 1 --2- 172.21.15.78:0/3821985977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x5627559813e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc068009230 tx=0x7fc068041f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7e0c6d50cbc388e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:50.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.880+0000 7fc05cff9640 1 -- 172.21.15.78:0/3821985977 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06804f070 con 0x562755976a10 2024-09-11T01:19:50.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.880+0000 7fc05cff9640 1 -- 172.21.15.78:0/3821985977 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc068004400 con 0x562755976a10 2024-09-11T01:19:50.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.880+0000 7fc05cff9640 1 -- 172.21.15.78:0/3821985977 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068035040 con 0x562755976a10 2024-09-11T01:19:50.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.881+0000 7fc07f49d740 1 -- 172.21.15.78:0/3821985977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 msgr2=0x5627559813e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:50.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.881+0000 7fc07f49d740 1 --2- 172.21.15.78:0/3821985977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x5627559813e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc068009230 tx=0x7fc068041f50 comp rx=0 tx=0).stop 2024-09-11T01:19:50.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.881+0000 7fc07f49d740 1 -- 172.21.15.78:0/3821985977 shutdown_connections 2024-09-11T01:19:50.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.881+0000 7fc07f49d740 1 --2- 172.21.15.78:0/3821985977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 0x562755983d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.881+0000 7fc07f49d740 1 --2- 172.21.15.78:0/3821985977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562755976a10 0x5627559813e0 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc068009230 tx=0x7fc068041f50 comp rx=0 tx=0).stop 2024-09-11T01:19:50.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.881+0000 7fc07f49d740 1 --2- 172.21.15.78:0/3821985977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627559760f0 0x562755973760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.881+0000 7fc07f49d740 1 -- 172.21.15.78:0/3821985977 >> 172.21.15.78:0/3821985977 conn(0x562755973f20 msgr2=0x562755980cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.881+0000 7fc07f49d740 1 -- 172.21.15.78:0/3821985977 shutdown_connections 2024-09-11T01:19:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.881+0000 7fc07f49d740 1 -- 172.21.15.78:0/3821985977 wait complete. 2024-09-11T01:19:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07f49d740 1 -- start start 2024-09-11T01:19:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07f49d740 1 Processor -- start 2024-09-11T01:19:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07f49d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627559760f0 0x56275598aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07f49d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 0x56275598b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07f49d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562755992290 0x56275597b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07f49d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781ad9e0 con 0x562755992290 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07f49d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781ad250 con 0x5627559760f0 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07f49d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781aa4d0 con 0x5627559829a0 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07f49d740 10 client.0 _ll_register_callbacks cb 0x7fc0700093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562755992290 0x56275597b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562755992290 0x56275597b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54984/0 (socket says 172.21.15.78:54984) 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07759e640 1 -- 172.21.15.78:0/2763267284 learned_addr learned my addr 172.21.15.78:0/2763267284 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc07759e640 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627559760f0 0x56275598aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.882+0000 7fc076d9d640 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 0x56275598b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc07759e640 1 -- 172.21.15.78:0/2763267284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 msgr2=0x56275598b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc07759e640 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 0x56275598b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc07759e640 1 -- 172.21.15.78:0/2763267284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627559760f0 msgr2=0x56275598aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc07759e640 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627559760f0 0x56275598aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc07759e640 1 -- 172.21.15.78:0/2763267284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56275597c0f0 con 0x562755992290 2024-09-11T01:19:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc07759e640 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562755992290 0x56275597b8c0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc070006010 tx=0x7fc0700021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c4da6dd0adc6ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc05aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc05aff5640 1 -- 172.21.15.78:0/2763267284 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07000fe50 con 0x562755992290 2024-09-11T01:19:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc05aff5640 1 -- 172.21.15.78:0/2763267284 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7fc070026040 con 0x562755992290 2024-09-11T01:19:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc07f49d740 1 -- 172.21.15.78:0/2763267284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc068004400 con 0x562755992290 2024-09-11T01:19:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc05aff5640 1 -- 172.21.15.78:0/2763267284 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fc070009570 con 0x562755992290 2024-09-11T01:19:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc07f49d740 10 client.19080 fetch_fsmap learned FSMap version 72 2024-09-11T01:19:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.883+0000 7fc07f49d740 1 -- 172.21.15.78:0/2763267284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x56275597d3f0 con 0x562755992290 2024-09-11T01:19:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.884+0000 7fc05aff5640 1 -- 172.21.15.78:0/2763267284 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc068004400 con 0x562755992290 2024-09-11T01:19:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.884+0000 7fc05aff5640 1 -- 172.21.15.78:0/2763267284 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7fc07001d050 con 0x562755992290 2024-09-11T01:19:50.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.884+0000 7fc07f49d740 10 client.19080 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.884+0000 7fc07f49d740 10 client.19080 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.884+0000 7fc07f49d740 20 client.19080 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.884+0000 7fc07f49d740 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x562755a00070 0x562755a02460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.884+0000 7fc07f49d740 4 client.19080 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.884+0000 7fc07f49d740 1 -- 172.21.15.78:0/2763267284 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627558ea8a0 con 0x562755a00070 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.884+0000 7fc076d9d640 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x562755a00070 0x562755a02460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc076d9d640 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x562755a00070 0x562755a02460 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9de16b1ffcfcc4bd server_cookie=88fcd2cf78f7768a in_seq=0 out_seq=0 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc05aff5640 10 client.19080 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc05aff5640 1 -- 172.21.15.78:0/2763267284 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627558ea8a0 con 0x562755a00070 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc05aff5640 10 client.19080 handle_command_reply: tid=0 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc07f49d740 1 client.19080 shutdown 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc07f49d740 1 -- 172.21.15.78:0/2763267284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562755992290 msgr2=0x56275597b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:50.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc07f49d740 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562755992290 0x56275597b8c0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc070006010 tx=0x7fc0700021d0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc07f49d740 1 -- 172.21.15.78:0/2763267284 shutdown_connections 2024-09-11T01:19:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc07f49d740 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x562755a00070 0x562755a02460 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc07f49d740 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562755992290 0x56275597b8c0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc07f49d740 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627559829a0 0x56275598b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc07f49d740 1 --2- 172.21.15.78:0/2763267284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627559760f0 0x56275598aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc07f49d740 1 -- 172.21.15.78:0/2763267284 >> 172.21.15.78:0/2763267284 conn(0x562755973f20 msgr2=0x5627559814f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.885+0000 7fc05aff5640 0 client.19080 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:19:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07f49d740 1 -- 172.21.15.78:0/2763267284 shutdown_connections 2024-09-11T01:19:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07f49d740 1 -- 172.21.15.78:0/2763267284 wait complete. 2024-09-11T01:19:50.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07f49d740 20 client.19080 trim_cache size 0 max 16384 2024-09-11T01:19:50.891 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:19:50.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07d0c8640 1 -- 172.21.15.78:0/246455267 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc0600b54d0 msgr2=0x7fc0600b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:50.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07d0c8640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc0600b54d0 0x7fc0600b7990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0ac4e0 tx=0x7fc06c009310 comp rx=0 tx=0).stop 2024-09-11T01:19:50.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07d0c8640 1 -- 172.21.15.78:0/246455267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc078104570 msgr2=0x7fc078101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:50.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07d0c8640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc078104570 0x7fc078101f30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00e9f0 tx=0x7fc06c00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07d0c8640 1 -- 172.21.15.78:0/246455267 shutdown_connections 2024-09-11T01:19:50.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07d0c8640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc0600b54d0 0x7fc0600b7990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07d0c8640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07810d120 0x7fc078102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07d0c8640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc078104570 0x7fc078101f30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07d0c8640 1 --2- 172.21.15.78:0/246455267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc078103b70 0x7fc0781019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:50.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.886+0000 7fc07d0c8640 1 -- 172.21.15.78:0/246455267 >> 172.21.15.78:0/246455267 conn(0x7fc0780fd700 msgr2=0x7fc07810f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:50.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.887+0000 7fc07d0c8640 1 -- 172.21.15.78:0/246455267 shutdown_connections 2024-09-11T01:19:50.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:50.887+0000 7fc07d0c8640 1 -- 172.21.15.78:0/246455267 wait complete. 2024-09-11T01:19:51.798 INFO:tasks.workunit.client.0.smithi078.stdout: 10 300442 33.26 MB/sec execute 357 sec latency 238.869 ms 2024-09-11T01:19:52.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:51 smithi110 ceph-mon[34712]: pgmap v980: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 691 op/s 2024-09-11T01:19:52.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:51 smithi112 ceph-mon[35384]: pgmap v980: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 691 op/s 2024-09-11T01:19:52.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:51 smithi078 ceph-mon[31315]: pgmap v980: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 691 op/s 2024-09-11T01:19:52.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 300643 33.21 MB/sec execute 358 sec latency 623.735 ms 2024-09-11T01:19:53.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 301125 33.20 MB/sec execute 359 sec latency 194.575 ms 2024-09-11T01:19:54.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:53 smithi110 ceph-mon[34712]: pgmap v981: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 565 op/s 2024-09-11T01:19:54.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:53 smithi112 ceph-mon[35384]: pgmap v981: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 565 op/s 2024-09-11T01:19:54.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:53 smithi078 ceph-mon[31315]: pgmap v981: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 565 op/s 2024-09-11T01:19:54.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 301689 33.17 MB/sec execute 360 sec latency 115.305 ms 2024-09-11T01:19:54.840 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:19:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:19:55.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 302362 33.16 MB/sec execute 361 sec latency 98.651 ms 2024-09-11T01:19:56.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:55 smithi110 ceph-mon[34712]: pgmap v982: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 575 op/s 2024-09-11T01:19:56.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:55 smithi112 ceph-mon[35384]: pgmap v982: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 575 op/s 2024-09-11T01:19:56.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:55 smithi078 ceph-mon[31315]: pgmap v982: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 575 op/s 2024-09-11T01:19:56.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 302972 33.16 MB/sec execute 362 sec latency 149.859 ms 2024-09-11T01:19:57.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.230+0000 7fef4a65b640 1 -- 172.21.15.78:0/843734808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44113b40 msgr2=0x7fef44115f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:57.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.230+0000 7fef4a65b640 1 --2- 172.21.15.78:0/843734808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44113b40 0x7fef44115f30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009860 tx=0x7fef2c02a340 comp rx=0 tx=0).stop 2024-09-11T01:19:57.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.230+0000 7fef4a65b640 1 -- 172.21.15.78:0/843734808 shutdown_connections 2024-09-11T01:19:57.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.230+0000 7fef4a65b640 1 --2- 172.21.15.78:0/843734808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44113b40 0x7fef44115f30 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.230+0000 7fef4a65b640 1 --2- 172.21.15.78:0/843734808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44075e60 0x7fef44113600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.230+0000 7fef4a65b640 1 --2- 172.21.15.78:0/843734808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef44075540 0x7fef44075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.230+0000 7fef4a65b640 1 -- 172.21.15.78:0/843734808 >> 172.21.15.78:0/843734808 conn(0x7fef440fe830 msgr2=0x7fef44100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:57.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.231+0000 7fef4a65b640 1 -- 172.21.15.78:0/843734808 shutdown_connections 2024-09-11T01:19:57.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.231+0000 7fef4a65b640 1 -- 172.21.15.78:0/843734808 wait complete. 2024-09-11T01:19:57.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.231+0000 7fef4a65b640 1 Processor -- start 2024-09-11T01:19:57.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.232+0000 7fef4a65b640 1 -- start start 2024-09-11T01:19:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.232+0000 7fef4a65b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44075540 0x7fef44074770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.232+0000 7fef4a65b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44075e60 0x7fef44074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.232+0000 7fef4a65b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef44113b40 0x7fef4406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.232+0000 7fef4a65b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef441193a0 con 0x7fef44075e60 2024-09-11T01:19:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.232+0000 7fef4a65b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fef44119220 con 0x7fef44075540 2024-09-11T01:19:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.232+0000 7fef4a65b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef44119520 con 0x7fef44113b40 2024-09-11T01:19:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.232+0000 7fef43fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44075540 0x7fef44074770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.232+0000 7fef43fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44075540 0x7fef44074770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:57756/0 (socket says 172.21.15.78:57756) 2024-09-11T01:19:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.232+0000 7fef43fff640 1 -- 172.21.15.78:0/1959305167 learned_addr learned my addr 172.21.15.78:0/1959305167 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.233+0000 7fef437fe640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44075e60 0x7fef44074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.233+0000 7fef43fff640 1 -- 172.21.15.78:0/1959305167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef44113b40 msgr2=0x7fef4406dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.233+0000 7fef43fff640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef44113b40 0x7fef4406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.233+0000 7fef43fff640 1 -- 172.21.15.78:0/1959305167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44075e60 msgr2=0x7fef44074cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.233+0000 7fef43fff640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44075e60 0x7fef44074cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.233+0000 7fef43fff640 1 -- 172.21.15.78:0/1959305167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4406e380 con 0x7fef44075540 2024-09-11T01:19:57.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.234+0000 7fef43fff640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44075540 0x7fef44074770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef3400efb0 tx=0x7fef3400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.236+0000 7fef417fa640 1 -- 172.21.15.78:0/1959305167 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3401b070 con 0x7fef44075540 2024-09-11T01:19:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.236+0000 7fef417fa640 1 -- 172.21.15.78:0/1959305167 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef3400ba20 con 0x7fef44075540 2024-09-11T01:19:57.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.236+0000 7fef417fa640 1 -- 172.21.15.78:0/1959305167 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34016b30 con 0x7fef44075540 2024-09-11T01:19:57.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.237+0000 7fef4a65b640 1 -- 172.21.15.78:0/1959305167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4406e670 con 0x7fef44075540 2024-09-11T01:19:57.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.237+0000 7fef4a65b640 1 -- 172.21.15.78:0/1959305167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef440735b0 con 0x7fef44075540 2024-09-11T01:19:57.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.241+0000 7fef417fa640 1 -- 172.21.15.78:0/1959305167 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fef3400b040 con 0x7fef44075540 2024-09-11T01:19:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.242+0000 7fef4ca30740 1 Processor -- start 2024-09-11T01:19:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.242+0000 7fef417fa640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fef300b5620 0x7fef300b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.242+0000 7fef4ca30740 1 -- start start 2024-09-11T01:19:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef417fa640 1 -- 172.21.15.78:0/1959305167 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7fef340e6360 con 0x7fef44075540 2024-09-11T01:19:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef48bf8640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fef300b5620 0x7fef300b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef4ca30740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 0x56270f6ed4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef4ca30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6eda10 0x56270f6f9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef4ca30740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f6fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 0x56270f6ed4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef4ca30740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56270f663ea0 con 0x56270f6ed0f0 2024-09-11T01:19:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef4ca30740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56270f6532d0 con 0x56270f6f99a0 2024-09-11T01:19:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef4ca30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56270f6570e0 con 0x56270f6eda10 2024-09-11T01:19:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 0x56270f6ed4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45706/0 (socket says 172.21.15.78:45706) 2024-09-11T01:19:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef43fff640 1 -- 172.21.15.78:0/1346198646 learned_addr learned my addr 172.21.15.78:0/1346198646 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef48bf8640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fef300b5620 0x7fef300b7ae0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fef2c035040 tx=0x7fef2c030340 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef48bf8640 1 --2- 172.21.15.78:0/1346198646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f6fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.243+0000 7fef437fe640 1 --2- 172.21.15.78:0/1346198646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6eda10 0x56270f6f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.244+0000 7fef43fff640 1 -- 172.21.15.78:0/1346198646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6eda10 msgr2=0x56270f6f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.244+0000 7fef43fff640 1 --2- 172.21.15.78:0/1346198646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6eda10 0x56270f6f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.244+0000 7fef43fff640 1 -- 172.21.15.78:0/1346198646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 msgr2=0x56270f6fbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.244+0000 7fef43fff640 1 --2- 172.21.15.78:0/1346198646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f6fbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.244+0000 7fef43fff640 1 -- 172.21.15.78:0/1346198646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56270f3b1650 con 0x56270f6ed0f0 2024-09-11T01:19:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.244+0000 7fef43fff640 1 --2- 172.21.15.78:0/1346198646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 0x56270f6ed4d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fef340a8d90 tx=0x7fef340a8790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d742311e98483b9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.244+0000 7fef3a7fc640 1 -- 172.21.15.78:0/1346198646 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef340ac7c0 con 0x56270f6ed0f0 2024-09-11T01:19:57.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.244+0000 7fef3a7fc640 1 -- 172.21.15.78:0/1346198646 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef340b12a0 con 0x56270f6ed0f0 2024-09-11T01:19:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.244+0000 7fef3a7fc640 1 -- 172.21.15.78:0/1346198646 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef340aba20 con 0x56270f6ed0f0 2024-09-11T01:19:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.245+0000 7fef4ca30740 1 -- 172.21.15.78:0/1346198646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 msgr2=0x56270f6ed4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.245+0000 7fef4ca30740 1 --2- 172.21.15.78:0/1346198646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 0x56270f6ed4d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fef340a8d90 tx=0x7fef340a8790 comp rx=0 tx=0).stop 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.245+0000 7fef4ca30740 1 -- 172.21.15.78:0/1346198646 shutdown_connections 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.245+0000 7fef4ca30740 1 --2- 172.21.15.78:0/1346198646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f6fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.246+0000 7fef4ca30740 1 --2- 172.21.15.78:0/1346198646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6eda10 0x56270f6f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.246+0000 7fef4ca30740 1 --2- 172.21.15.78:0/1346198646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 0x56270f6ed4d0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.246+0000 7fef4ca30740 1 -- 172.21.15.78:0/1346198646 >> 172.21.15.78:0/1346198646 conn(0x56270f6eaf20 msgr2=0x56270f6ec0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.246+0000 7fef4ca30740 1 -- 172.21.15.78:0/1346198646 shutdown_connections 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.246+0000 7fef4ca30740 1 -- 172.21.15.78:0/1346198646 wait complete. 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.246+0000 7fef4ca30740 1 -- start start 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.247+0000 7fef4ca30740 1 Processor -- start 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.247+0000 7fef4ca30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6ed0f0 0x56270f6f9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.247+0000 7fef4ca30740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6eda10 0x56270f6f6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef4ca30740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6f99a0 0x56270f6f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef4ca30740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56270f4b4700 con 0x56270f6f99a0 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef43fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6ed0f0 0x56270f6f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef43fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6ed0f0 0x56270f6f9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:46694/0 (socket says 172.21.15.78:46694) 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef43fff640 1 -- 172.21.15.78:0/2395980894 learned_addr learned my addr 172.21.15.78:0/2395980894 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef4ca30740 1 -- 172.21.15.78:0/2395980894 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56270f5ee160 con 0x56270f6eda10 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef4ca30740 1 -- 172.21.15.78:0/2395980894 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56270f6fc620 con 0x56270f6ed0f0 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef4ca30740 10 client.0 _ll_register_callbacks cb 0x56270f6079e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef48bf8640 1 --2- 172.21.15.78:0/2395980894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6f99a0 0x56270f6f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef43fff640 1 -- 172.21.15.78:0/2395980894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6eda10 msgr2=0x56270f6f6690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef43fff640 1 --2- 172.21.15.78:0/2395980894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6eda10 0x56270f6f6690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef43fff640 1 -- 172.21.15.78:0/2395980894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6f99a0 msgr2=0x56270f6f6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef43fff640 1 --2- 172.21.15.78:0/2395980894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6f99a0 0x56270f6f6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.248+0000 7fef43fff640 1 -- 172.21.15.78:0/2395980894 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56270f61bc90 con 0x56270f6ed0f0 2024-09-11T01:19:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.249+0000 7fef43fff640 1 --2- 172.21.15.78:0/2395980894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6ed0f0 0x56270f6f9240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fef340e4090 tx=0x7fef340e3ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c185a07d51c3b220 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.249+0000 7fef237fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.249+0000 7fef237fe640 1 -- 172.21.15.78:0/2395980894 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef340ac7c0 con 0x56270f6ed0f0 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.249+0000 7fef4ca30740 1 -- 172.21.15.78:0/2395980894 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef340b12a0 con 0x56270f6ed0f0 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.249+0000 7fef237fe640 1 -- 172.21.15.78:0/2395980894 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7fef3400b570 con 0x56270f6ed0f0 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.249+0000 7fef237fe640 1 -- 172.21.15.78:0/2395980894 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fef3400b7b0 con 0x56270f6ed0f0 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.249+0000 7fef4ca30740 10 client.28090 fetch_fsmap learned FSMap version 72 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.249+0000 7fef4ca30740 1 -- 172.21.15.78:0/2395980894 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x56270f707f20 con 0x56270f6ed0f0 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.250+0000 7fef237fe640 1 -- 172.21.15.78:0/2395980894 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef340cfb10 con 0x56270f6ed0f0 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.250+0000 7fef237fe640 1 -- 172.21.15.78:0/2395980894 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7fef340a8110 con 0x56270f6ed0f0 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.250+0000 7fef4ca30740 10 client.28090 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.250+0000 7fef4ca30740 10 client.28090 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.250+0000 7fef4ca30740 20 client.28090 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.250+0000 7fef4ca30740 1 --2- 172.21.15.78:0/2395980894 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56270f709270 0x56270f70b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.250+0000 7fef4ca30740 4 client.28090 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:19:57.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.250+0000 7fef4ca30740 1 -- 172.21.15.78:0/2395980894 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fef340b12a0 con 0x56270f709270 2024-09-11T01:19:57.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.250+0000 7fef437fe640 1 --2- 172.21.15.78:0/2395980894 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56270f709270 0x56270f70b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.251+0000 7fef437fe640 1 --2- 172.21.15.78:0/2395980894 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56270f709270 0x56270f70b660 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=963cb0587bcaa8bb server_cookie=4cdf4b56fc1aa199 in_seq=0 out_seq=0 2024-09-11T01:19:57.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.251+0000 7fef237fe640 10 client.28090 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:19:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.254+0000 7fef237fe640 1 -- 172.21.15.78:0/2395980894 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fef340b12a0 con 0x56270f709270 2024-09-11T01:19:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.254+0000 7fef237fe640 10 client.28090 handle_command_reply: tid=0 2024-09-11T01:19:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.254+0000 7fef4ca30740 1 client.28090 shutdown 2024-09-11T01:19:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.254+0000 7fef4ca30740 1 -- 172.21.15.78:0/2395980894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6ed0f0 msgr2=0x56270f6f9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.254+0000 7fef4ca30740 1 --2- 172.21.15.78:0/2395980894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6ed0f0 0x56270f6f9240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fef340e4090 tx=0x7fef340e3ad0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.255+0000 7fef4ca30740 1 -- 172.21.15.78:0/2395980894 shutdown_connections 2024-09-11T01:19:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.255+0000 7fef4ca30740 1 --2- 172.21.15.78:0/2395980894 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56270f709270 0x56270f70b660 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.255+0000 7fef237fe640 0 client.28090 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:19:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.255+0000 7fef4ca30740 1 --2- 172.21.15.78:0/2395980894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6f99a0 0x56270f6f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.255+0000 7fef4ca30740 1 --2- 172.21.15.78:0/2395980894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6eda10 0x56270f6f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.255+0000 7fef4ca30740 1 --2- 172.21.15.78:0/2395980894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6ed0f0 0x56270f6f9240 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.255+0000 7fef4ca30740 1 -- 172.21.15.78:0/2395980894 >> 172.21.15.78:0/2395980894 conn(0x56270f6eaf20 msgr2=0x56270f6f0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.255+0000 7fef4ca30740 1 -- 172.21.15.78:0/2395980894 shutdown_connections 2024-09-11T01:19:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.255+0000 7fef4ca30740 1 -- 172.21.15.78:0/2395980894 wait complete. 2024-09-11T01:19:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.255+0000 7fef4ca30740 20 client.28090 trim_cache size 0 max 16384 2024-09-11T01:19:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.276+0000 7fef4ca30740 1 Processor -- start 2024-09-11T01:19:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.276+0000 7fef4ca30740 1 -- start start 2024-09-11T01:19:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef4ca30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6ed0f0 0x56270f6ec6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef4ca30740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6eda10 0x56270f6fad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef4ca30740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f6fb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef4ca30740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef441bd390 con 0x56270f6eda10 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef4ca30740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fef441bef90 con 0x56270f6f99a0 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef4ca30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef441bda60 con 0x56270f6ed0f0 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef48bf8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f6fb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef48bf8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f6fb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:57790/0 (socket says 172.21.15.78:57790) 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef48bf8640 1 -- 172.21.15.78:0/3973664705 learned_addr learned my addr 172.21.15.78:0/3973664705 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef43fff640 1 --2- 172.21.15.78:0/3973664705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6ed0f0 0x56270f6ec6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef48bf8640 1 -- 172.21.15.78:0/3973664705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6ed0f0 msgr2=0x56270f6ec6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef48bf8640 1 --2- 172.21.15.78:0/3973664705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6ed0f0 0x56270f6ec6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef48bf8640 1 -- 172.21.15.78:0/3973664705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6eda10 msgr2=0x56270f6fad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef48bf8640 1 --2- 172.21.15.78:0/3973664705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6eda10 0x56270f6fad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef48bf8640 1 -- 172.21.15.78:0/3973664705 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56270f6ed840 con 0x56270f6f99a0 2024-09-11T01:19:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.277+0000 7fef48bf8640 1 --2- 172.21.15.78:0/3973664705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f6fb2d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009860 tx=0x7fef2c03c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f194d74d04521 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.278+0000 7fef39ffb640 1 -- 172.21.15.78:0/3973664705 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c037040 con 0x56270f6f99a0 2024-09-11T01:19:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.278+0000 7fef39ffb640 1 -- 172.21.15.78:0/3973664705 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef2c04e070 con 0x56270f6f99a0 2024-09-11T01:19:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.278+0000 7fef39ffb640 1 -- 172.21.15.78:0/3973664705 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c003850 con 0x56270f6f99a0 2024-09-11T01:19:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.279+0000 7fef4ca30740 1 -- 172.21.15.78:0/3973664705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 msgr2=0x56270f6fb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.279+0000 7fef4ca30740 1 --2- 172.21.15.78:0/3973664705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f6fb2d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009860 tx=0x7fef2c03c000 comp rx=0 tx=0).stop 2024-09-11T01:19:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.279+0000 7fef4ca30740 1 -- 172.21.15.78:0/3973664705 shutdown_connections 2024-09-11T01:19:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.279+0000 7fef4ca30740 1 --2- 172.21.15.78:0/3973664705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f6fb2d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.279+0000 7fef4ca30740 1 --2- 172.21.15.78:0/3973664705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6eda10 0x56270f6fad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.279+0000 7fef4ca30740 1 --2- 172.21.15.78:0/3973664705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6ed0f0 0x56270f6ec6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.279+0000 7fef4ca30740 1 -- 172.21.15.78:0/3973664705 >> 172.21.15.78:0/3973664705 conn(0x56270f6eaf20 msgr2=0x56270f6ebfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.279+0000 7fef4ca30740 1 -- 172.21.15.78:0/3973664705 shutdown_connections 2024-09-11T01:19:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.279+0000 7fef4ca30740 1 -- 172.21.15.78:0/3973664705 wait complete. 2024-09-11T01:19:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.279+0000 7fef4ca30740 1 -- start start 2024-09-11T01:19:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.279+0000 7fef4ca30740 1 Processor -- start 2024-09-11T01:19:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef4ca30740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 0x56270f70ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef4ca30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6eda10 0x56270f70b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef4ca30740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f774dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef4ca30740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef441be8c0 con 0x56270f6ed0f0 2024-09-11T01:19:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 0x56270f70ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 0x56270f70ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45740/0 (socket says 172.21.15.78:45740) 2024-09-11T01:19:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef43fff640 1 -- 172.21.15.78:0/3638128019 learned_addr learned my addr 172.21.15.78:0/3638128019 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:19:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef4ca30740 1 -- 172.21.15.78:0/3638128019 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fef441be130 con 0x56270f6f99a0 2024-09-11T01:19:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef4ca30740 1 -- 172.21.15.78:0/3638128019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef441bb390 con 0x56270f6eda10 2024-09-11T01:19:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef437fe640 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6eda10 0x56270f70b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef48bf8640 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f774dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef4ca30740 10 client.0 _ll_register_callbacks cb 0x7fef340ac1a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:19:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef43fff640 1 -- 172.21.15.78:0/3638128019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6eda10 msgr2=0x56270f70b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef43fff640 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6eda10 0x56270f70b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef43fff640 1 -- 172.21.15.78:0/3638128019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 msgr2=0x56270f774dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:19:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef43fff640 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f774dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.280+0000 7fef43fff640 1 -- 172.21.15.78:0/3638128019 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56270f7754c0 con 0x56270f6ed0f0 2024-09-11T01:19:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.281+0000 7fef43fff640 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 0x56270f70ae60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fef34000f80 tx=0x7fef34009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e3e2fb8f091a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:19:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.281+0000 7fef237fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:19:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.281+0000 7fef237fe640 1 -- 172.21.15.78:0/3638128019 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef340adcf0 con 0x56270f6ed0f0 2024-09-11T01:19:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.281+0000 7fef237fe640 1 -- 172.21.15.78:0/3638128019 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7fef340a8110 con 0x56270f6ed0f0 2024-09-11T01:19:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.281+0000 7fef4ca30740 1 -- 172.21.15.78:0/3638128019 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef2c04e070 con 0x56270f6ed0f0 2024-09-11T01:19:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.281+0000 7fef237fe640 1 -- 172.21.15.78:0/3638128019 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fef340ac7c0 con 0x56270f6ed0f0 2024-09-11T01:19:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.281+0000 7fef4ca30740 10 client.19095 fetch_fsmap learned FSMap version 72 2024-09-11T01:19:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.281+0000 7fef4ca30740 1 -- 172.21.15.78:0/3638128019 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x56270f7754c0 con 0x56270f6ed0f0 2024-09-11T01:19:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.281+0000 7fef237fe640 1 -- 172.21.15.78:0/3638128019 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef2c04e070 con 0x56270f6ed0f0 2024-09-11T01:19:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.282+0000 7fef237fe640 1 -- 172.21.15.78:0/3638128019 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7fef340b2430 con 0x56270f6ed0f0 2024-09-11T01:19:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.282+0000 7fef4ca30740 10 client.19095 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:19:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.282+0000 7fef4ca30740 10 client.19095 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:19:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.282+0000 7fef4ca30740 20 client.19095 populate_metadata read hostname 'smithi078' 2024-09-11T01:19:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.282+0000 7fef4ca30740 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56270f709270 0x56270f778ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:19:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.282+0000 7fef4ca30740 4 client.19095 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:19:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.282+0000 7fef4ca30740 1 -- 172.21.15.78:0/3638128019 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56270f665df0 con 0x56270f709270 2024-09-11T01:19:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.282+0000 7fef437fe640 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56270f709270 0x56270f778ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:19:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.282+0000 7fef237fe640 10 client.19095 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.282+0000 7fef437fe640 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56270f709270 0x56270f778ac0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e0a5720f99349f8c server_cookie=269b71ff34451b9c in_seq=0 out_seq=0 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef237fe640 1 -- 172.21.15.78:0/3638128019 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56270f665df0 con 0x56270f709270 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef237fe640 10 client.19095 handle_command_reply: tid=0 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef4ca30740 1 client.19095 shutdown 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef4ca30740 1 -- 172.21.15.78:0/3638128019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 msgr2=0x56270f70ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef4ca30740 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 0x56270f70ae60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fef34000f80 tx=0x7fef34009b50 comp rx=0 tx=0).stop 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef4ca30740 1 -- 172.21.15.78:0/3638128019 shutdown_connections 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef4ca30740 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x56270f709270 0x56270f778ac0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef237fe640 0 client.19095 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef4ca30740 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56270f6f99a0 0x56270f774dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef4ca30740 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56270f6eda10 0x56270f70b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef4ca30740 1 --2- 172.21.15.78:0/3638128019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56270f6ed0f0 0x56270f70ae60 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef4ca30740 1 -- 172.21.15.78:0/3638128019 >> 172.21.15.78:0/3638128019 conn(0x56270f6eaf20 msgr2=0x56270f6f7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:57.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef4ca30740 1 -- 172.21.15.78:0/3638128019 shutdown_connections 2024-09-11T01:19:57.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef4ca30740 1 -- 172.21.15.78:0/3638128019 wait complete. 2024-09-11T01:19:57.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.284+0000 7fef4ca30740 20 client.19095 trim_cache size 0 max 16384 2024-09-11T01:19:57.295 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:19:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.285+0000 7fef4a65b640 1 -- 172.21.15.78:0/1959305167 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fef300b5620 msgr2=0x7fef300b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.285+0000 7fef4a65b640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fef300b5620 0x7fef300b7ae0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fef2c035040 tx=0x7fef2c030340 comp rx=0 tx=0).stop 2024-09-11T01:19:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.285+0000 7fef4a65b640 1 -- 172.21.15.78:0/1959305167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44075540 msgr2=0x7fef44074770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:19:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.285+0000 7fef4a65b640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44075540 0x7fef44074770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef3400efb0 tx=0x7fef3400c800 comp rx=0 tx=0).stop 2024-09-11T01:19:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.285+0000 7fef4a65b640 1 -- 172.21.15.78:0/1959305167 shutdown_connections 2024-09-11T01:19:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.285+0000 7fef4a65b640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fef300b5620 0x7fef300b7ae0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.285+0000 7fef4a65b640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef44113b40 0x7fef4406dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.285+0000 7fef4a65b640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44075e60 0x7fef44074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.285+0000 7fef4a65b640 1 --2- 172.21.15.78:0/1959305167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44075540 0x7fef44074770 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:19:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.285+0000 7fef4a65b640 1 -- 172.21.15.78:0/1959305167 >> 172.21.15.78:0/1959305167 conn(0x7fef440fe830 msgr2=0x7fef440ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:19:57.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.285+0000 7fef4a65b640 1 -- 172.21.15.78:0/1959305167 shutdown_connections 2024-09-11T01:19:57.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:19:57.285+0000 7fef4a65b640 1 -- 172.21.15.78:0/1959305167 wait complete. 2024-09-11T01:19:57.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 303662 33.19 MB/sec execute 363 sec latency 119.465 ms 2024-09-11T01:19:57.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:57 smithi078 ceph-mon[31315]: pgmap v983: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 652 op/s 2024-09-11T01:19:58.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:57 smithi110 ceph-mon[34712]: pgmap v983: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 652 op/s 2024-09-11T01:19:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:57 smithi112 ceph-mon[35384]: pgmap v983: 97 pgs: 97 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 652 op/s 2024-09-11T01:19:58.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 304168 33.18 MB/sec execute 364 sec latency 168.890 ms 2024-09-11T01:19:59.184 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:19:59.529 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:19:59.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 304608 33.16 MB/sec execute 365 sec latency 239.198 ms 2024-09-11T01:19:59.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:19:59 smithi078 ceph-mon[31315]: pgmap v984: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 652 op/s 2024-09-11T01:20:00.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:19:59 smithi110 ceph-mon[34712]: pgmap v984: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 652 op/s 2024-09-11T01:20:00.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:19:59 smithi112 ceph-mon[35384]: pgmap v984: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 652 op/s 2024-09-11T01:20:00.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 305023 33.12 MB/sec execute 366 sec latency 418.502 ms 2024-09-11T01:20:01.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:00 smithi110 ceph-mon[34712]: overall HEALTH_OK 2024-09-11T01:20:01.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:00 smithi112 ceph-mon[35384]: overall HEALTH_OK 2024-09-11T01:20:01.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:00 smithi078 ceph-mon[31315]: overall HEALTH_OK 2024-09-11T01:20:01.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.557+0000 7fcf007ce640 1 -- 172.21.15.78:0/2934489513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcef8101c90 msgr2=0x7fcef8102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:01.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.557+0000 7fcf007ce640 1 --2- 172.21.15.78:0/2934489513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcef8101c90 0x7fcef8102070 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009900 tx=0x7fcef402a410 comp rx=0 tx=0).stop 2024-09-11T01:20:01.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.557+0000 7fcf007ce640 1 -- 172.21.15.78:0/2934489513 shutdown_connections 2024-09-11T01:20:01.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.557+0000 7fcf007ce640 1 --2- 172.21.15.78:0/2934489513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcef8108020 0x7fcef810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.557+0000 7fcf007ce640 1 --2- 172.21.15.78:0/2934489513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcef81076a0 0x7fcef8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.557+0000 7fcf007ce640 1 --2- 172.21.15.78:0/2934489513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcef8101c90 0x7fcef8102070 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.557+0000 7fcf007ce640 1 -- 172.21.15.78:0/2934489513 >> 172.21.15.78:0/2934489513 conn(0x7fcef80fd700 msgr2=0x7fcef80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:01.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.557+0000 7fcf007ce640 1 -- 172.21.15.78:0/2934489513 shutdown_connections 2024-09-11T01:20:01.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.557+0000 7fcf007ce640 1 -- 172.21.15.78:0/2934489513 wait complete. 2024-09-11T01:20:01.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.558+0000 7fcf007ce640 1 Processor -- start 2024-09-11T01:20:01.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.558+0000 7fcf007ce640 1 -- start start 2024-09-11T01:20:01.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.558+0000 7fcf007ce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcef8101c90 0x7fcef810d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.558+0000 7fcf007ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcef81076a0 0x7fcef810de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.558+0000 7fcf007ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcef8108020 0x7fcef81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.559+0000 7fcf007ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcef806b1d0 con 0x7fcef8108020 2024-09-11T01:20:01.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.559+0000 7fcf007ce640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcef806b050 con 0x7fcef8101c90 2024-09-11T01:20:01.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.559+0000 7fcf007ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcef806b350 con 0x7fcef81076a0 2024-09-11T01:20:01.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.559+0000 7fcefed6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcef8108020 0x7fcef81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.559+0000 7fcefed6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcef8108020 0x7fcef81996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45768/0 (socket says 172.21.15.78:45768) 2024-09-11T01:20:01.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.559+0000 7fcefed6b640 1 -- 172.21.15.78:0/1423319923 learned_addr learned my addr 172.21.15.78:0/1423319923 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:01.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.559+0000 7fcefe56a640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcef8101c90 0x7fcef810d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.559+0000 7fcefed6b640 1 -- 172.21.15.78:0/1423319923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcef81076a0 msgr2=0x7fcef810de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:01.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.559+0000 7fcefed6b640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcef81076a0 0x7fcef810de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.559+0000 7fcefed6b640 1 -- 172.21.15.78:0/1423319923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcef8101c90 msgr2=0x7fcef810d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:01.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.559+0000 7fcefed6b640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcef8101c90 0x7fcef810d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.559+0000 7fcefed6b640 1 -- 172.21.15.78:0/1423319923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef8199e60 con 0x7fcef8108020 2024-09-11T01:20:01.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.560+0000 7fcefed6b640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcef8108020 0x7fcef81996e0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fcef0030b20 tx=0x7fcef002e2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:01.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.561+0000 7fceef7fe640 1 -- 172.21.15.78:0/1423319923 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef0004b20 con 0x7fcef8108020 2024-09-11T01:20:01.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.561+0000 7fceef7fe640 1 -- 172.21.15.78:0/1423319923 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcef0004cc0 con 0x7fcef8108020 2024-09-11T01:20:01.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.561+0000 7fceef7fe640 1 -- 172.21.15.78:0/1423319923 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef00327b0 con 0x7fcef8108020 2024-09-11T01:20:01.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.561+0000 7fcf007ce640 1 -- 172.21.15.78:0/1423319923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef819a150 con 0x7fcef8108020 2024-09-11T01:20:01.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.561+0000 7fcf007ce640 1 -- 172.21.15.78:0/1423319923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef819faa0 con 0x7fcef8108020 2024-09-11T01:20:01.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.565+0000 7fcf02ba3740 1 Processor -- start 2024-09-11T01:20:01.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.565+0000 7fceef7fe640 1 -- 172.21.15.78:0/1423319923 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fcef0005000 con 0x7fcef8108020 2024-09-11T01:20:01.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.566+0000 7fcf02ba3740 1 -- start start 2024-09-11T01:20:01.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.566+0000 7fcf02ba3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a90e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.566+0000 7fcf02ba3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 0x555d3a91a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.566+0000 7fcf02ba3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a91ad80 0x555d3a91d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.566+0000 7fcf02ba3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555d3a884ea0 con 0x555d3a90edf0 2024-09-11T01:20:01.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.566+0000 7fcf02ba3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555d3a8742d0 con 0x555d3a91ad80 2024-09-11T01:20:01.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.566+0000 7fcf02ba3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555d3a8780e0 con 0x555d3a90e4d0 2024-09-11T01:20:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.566+0000 7fcefe56a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 0x555d3a91a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.566+0000 7fcefe56a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 0x555d3a91a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45774/0 (socket says 172.21.15.78:45774) 2024-09-11T01:20:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.566+0000 7fcefe56a640 1 -- 172.21.15.78:0/2915639708 learned_addr learned my addr 172.21.15.78:0/2915639708 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.567+0000 7fceef7fe640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcee40b55a0 0x7fcee40b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.567+0000 7fcefe56a640 1 -- 172.21.15.78:0/2915639708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 msgr2=0x555d3a90e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.567+0000 7fcefe56a640 1 --2- 172.21.15.78:0/2915639708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a90e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.567+0000 7fcefe56a640 1 -- 172.21.15.78:0/2915639708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a91ad80 msgr2=0x555d3a91d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.567+0000 7fcefe56a640 1 --2- 172.21.15.78:0/2915639708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a91ad80 0x555d3a91d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.567+0000 7fcefe56a640 1 -- 172.21.15.78:0/2915639708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d3a5d2650 con 0x555d3a90edf0 2024-09-11T01:20:01.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.567+0000 7fceef7fe640 1 -- 172.21.15.78:0/1423319923 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7fcef0108640 con 0x7fcef8108020 2024-09-11T01:20:01.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.567+0000 7fcefe56a640 1 --2- 172.21.15.78:0/2915639708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 0x555d3a91a840 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009900 tx=0x7fcef4032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc6931378ac4dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:01.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.567+0000 7fceecff9640 1 -- 172.21.15.78:0/2915639708 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef403f070 con 0x555d3a90edf0 2024-09-11T01:20:01.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.568+0000 7fcefed6b640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcee40b55a0 0x7fcee40b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.568+0000 7fceecff9640 1 -- 172.21.15.78:0/2915639708 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcef4032e40 con 0x555d3a90edf0 2024-09-11T01:20:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.569+0000 7fceecff9640 1 -- 172.21.15.78:0/2915639708 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4035070 con 0x555d3a90edf0 2024-09-11T01:20:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.569+0000 7fcf02ba3740 1 -- 172.21.15.78:0/2915639708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 msgr2=0x555d3a91a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.569+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/2915639708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 0x555d3a91a840 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009900 tx=0x7fcef4032b90 comp rx=0 tx=0).stop 2024-09-11T01:20:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.569+0000 7fcefed6b640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcee40b55a0 0x7fcee40b7a60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcef00049d0 tx=0x7fcef002ef20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.569+0000 7fcf02ba3740 1 -- 172.21.15.78:0/2915639708 shutdown_connections 2024-09-11T01:20:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.569+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/2915639708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a91ad80 0x555d3a91d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.569+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/2915639708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 0x555d3a91a840 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.569+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/2915639708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a90e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.569+0000 7fcf02ba3740 1 -- 172.21.15.78:0/2915639708 >> 172.21.15.78:0/2915639708 conn(0x555d3a90c300 msgr2=0x555d3a90d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.570+0000 7fcf02ba3740 1 -- 172.21.15.78:0/2915639708 shutdown_connections 2024-09-11T01:20:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.570+0000 7fcf02ba3740 1 -- 172.21.15.78:0/2915639708 wait complete. 2024-09-11T01:20:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.570+0000 7fcf02ba3740 1 -- start start 2024-09-11T01:20:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.570+0000 7fcf02ba3740 1 Processor -- start 2024-09-11T01:20:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.571+0000 7fcf02ba3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a91a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.571+0000 7fcf02ba3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a90edf0 0x555d3a917a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.571+0000 7fcf02ba3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a91ad80 0x555d3a917fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.571+0000 7fcf02ba3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555d3a6d5700 con 0x555d3a91ad80 2024-09-11T01:20:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.571+0000 7fcf02ba3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555d3a80f160 con 0x555d3a90edf0 2024-09-11T01:20:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.571+0000 7fcf02ba3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555d3a91da40 con 0x555d3a90e4d0 2024-09-11T01:20:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.571+0000 7fcf02ba3740 10 client.0 _ll_register_callbacks cb 0x555d3a8289e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:01.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.571+0000 7fcefe56a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a90edf0 0x555d3a917a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.571+0000 7fcefdd69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a91a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.571+0000 7fcefdd69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a91a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:46750/0 (socket says 172.21.15.78:46750) 2024-09-11T01:20:01.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.571+0000 7fcefdd69640 1 -- 172.21.15.78:0/3462023324 learned_addr learned my addr 172.21.15.78:0/3462023324 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.571+0000 7fcefdd69640 1 --2- 172.21.15.78:0/3462023324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a91ad80 0x555d3a917fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.572+0000 7fcefdd69640 1 -- 172.21.15.78:0/3462023324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a90edf0 msgr2=0x555d3a917a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.572+0000 7fcefdd69640 1 --2- 172.21.15.78:0/3462023324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a90edf0 0x555d3a917a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.572+0000 7fcefdd69640 1 -- 172.21.15.78:0/3462023324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a91ad80 msgr2=0x555d3a917fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.572+0000 7fcefdd69640 1 --2- 172.21.15.78:0/3462023324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a91ad80 0x555d3a917fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.572+0000 7fcefdd69640 1 -- 172.21.15.78:0/3462023324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d3a83cc90 con 0x555d3a90e4d0 2024-09-11T01:20:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.572+0000 7fcefdd69640 1 --2- 172.21.15.78:0/3462023324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a91a5f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009fc0 tx=0x7fcee8009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=320b512641ccdf43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.572+0000 7fcee1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:20:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.572+0000 7fcee1ffb640 1 -- 172.21.15.78:0/3462023324 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8018030 con 0x555d3a90e4d0 2024-09-11T01:20:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.572+0000 7fcee1ffb640 1 -- 172.21.15.78:0/3462023324 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7fcee800ea60 con 0x555d3a90e4d0 2024-09-11T01:20:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.573+0000 7fcf02ba3740 1 -- 172.21.15.78:0/3462023324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555d3a886df0 con 0x555d3a90e4d0 2024-09-11T01:20:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.573+0000 7fcee1ffb640 1 -- 172.21.15.78:0/3462023324 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fcee800ec60 con 0x555d3a90e4d0 2024-09-11T01:20:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.573+0000 7fcf02ba3740 10 client.28102 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.573+0000 7fcf02ba3740 1 -- 172.21.15.78:0/3462023324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x555d3a929420 con 0x555d3a90e4d0 2024-09-11T01:20:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.574+0000 7fcee1ffb640 1 -- 172.21.15.78:0/3462023324 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555d3a886df0 con 0x555d3a90e4d0 2024-09-11T01:20:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.574+0000 7fcee1ffb640 1 -- 172.21.15.78:0/3462023324 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7fcee8004a70 con 0x555d3a90e4d0 2024-09-11T01:20:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.574+0000 7fcf02ba3740 10 client.28102 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.574+0000 7fcf02ba3740 10 client.28102 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:20:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.574+0000 7fcf02ba3740 20 client.28102 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.574+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/3462023324 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555d3a92acd0 0x555d3a92d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.574+0000 7fcf02ba3740 4 client.28102 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:20:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.574+0000 7fcf02ba3740 1 -- 172.21.15.78:0/3462023324 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d3a8828a0 con 0x555d3a92acd0 2024-09-11T01:20:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.574+0000 7fcefe56a640 1 --2- 172.21.15.78:0/3462023324 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555d3a92acd0 0x555d3a92d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.575+0000 7fcefe56a640 1 --2- 172.21.15.78:0/3462023324 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555d3a92acd0 0x555d3a92d110 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd58396280e16118 server_cookie=38072330d912fbef in_seq=0 out_seq=0 2024-09-11T01:20:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.575+0000 7fcee1ffb640 10 client.28102 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:20:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.577+0000 7fcee1ffb640 1 -- 172.21.15.78:0/3462023324 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555d3a8828a0 con 0x555d3a92acd0 2024-09-11T01:20:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.577+0000 7fcee1ffb640 10 client.28102 handle_command_reply: tid=0 2024-09-11T01:20:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.577+0000 7fcf02ba3740 1 client.28102 shutdown 2024-09-11T01:20:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.578+0000 7fcf02ba3740 1 -- 172.21.15.78:0/3462023324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 msgr2=0x555d3a91a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.578+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/3462023324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a91a5f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009fc0 tx=0x7fcee8009650 comp rx=0 tx=0).stop 2024-09-11T01:20:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.578+0000 7fcf02ba3740 1 -- 172.21.15.78:0/3462023324 shutdown_connections 2024-09-11T01:20:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.578+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/3462023324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a91ad80 0x555d3a917fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.578+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/3462023324 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555d3a92acd0 0x555d3a92d110 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.578+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/3462023324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a90edf0 0x555d3a917a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.578+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/3462023324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a91a5f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.578+0000 7fcf02ba3740 1 -- 172.21.15.78:0/3462023324 >> 172.21.15.78:0/3462023324 conn(0x555d3a90c300 msgr2=0x555d3a911ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.578+0000 7fcee1ffb640 0 client.28102 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:20:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.578+0000 7fcf02ba3740 1 -- 172.21.15.78:0/3462023324 shutdown_connections 2024-09-11T01:20:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.578+0000 7fcf02ba3740 1 -- 172.21.15.78:0/3462023324 wait complete. 2024-09-11T01:20:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.578+0000 7fcf02ba3740 20 client.28102 trim_cache size 0 max 16384 2024-09-11T01:20:01.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.599+0000 7fcf02ba3740 1 Processor -- start 2024-09-11T01:20:01.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.600+0000 7fcf02ba3740 1 -- start start 2024-09-11T01:20:01.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.600+0000 7fcf02ba3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a9177f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.600+0000 7fcf02ba3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 0x555d3a917d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.600+0000 7fcf02ba3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a91ad80 0x555d3a914c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.600+0000 7fcf02ba3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81a83d0 con 0x555d3a90edf0 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.600+0000 7fcf02ba3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81a9fd0 con 0x555d3a91ad80 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.600+0000 7fcf02ba3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81a8aa0 con 0x555d3a90e4d0 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcefdd69640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a91ad80 0x555d3a914c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcefe56a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 0x555d3a917d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcefe56a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 0x555d3a917d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45788/0 (socket says 172.21.15.78:45788) 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcefe56a640 1 -- 172.21.15.78:0/2560802705 learned_addr learned my addr 172.21.15.78:0/2560802705 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcefdd69640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a91ad80 0x555d3a914c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:57850/0 (socket says 172.21.15.78:57850) 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcefdd69640 1 --2- 172.21.15.78:0/2560802705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a9177f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcefe56a640 1 -- 172.21.15.78:0/2560802705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 msgr2=0x555d3a9177f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcefe56a640 1 --2- 172.21.15.78:0/2560802705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a9177f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcefe56a640 1 -- 172.21.15.78:0/2560802705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a91ad80 msgr2=0x555d3a914c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcefe56a640 1 --2- 172.21.15.78:0/2560802705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a91ad80 0x555d3a914c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcefe56a640 1 -- 172.21.15.78:0/2560802705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d3a90cd80 con 0x555d3a90edf0 2024-09-11T01:20:01.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcefe56a640 1 --2- 172.21.15.78:0/2560802705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 0x555d3a917d30 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fcef40095d0 tx=0x7fcef4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253247877b23ad1f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcee3fff640 1 -- 172.21.15.78:0/2560802705 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef403f040 con 0x555d3a90edf0 2024-09-11T01:20:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcee3fff640 1 -- 172.21.15.78:0/2560802705 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcef4031d10 con 0x555d3a90edf0 2024-09-11T01:20:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.601+0000 7fcee3fff640 1 -- 172.21.15.78:0/2560802705 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4035040 con 0x555d3a90edf0 2024-09-11T01:20:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.602+0000 7fcf02ba3740 1 -- 172.21.15.78:0/2560802705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 msgr2=0x555d3a917d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.602+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/2560802705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 0x555d3a917d30 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fcef40095d0 tx=0x7fcef4005fd0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.602+0000 7fcf02ba3740 1 -- 172.21.15.78:0/2560802705 shutdown_connections 2024-09-11T01:20:01.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.602+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/2560802705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a91ad80 0x555d3a914c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.602+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/2560802705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90edf0 0x555d3a917d30 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.602+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/2560802705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a90e4d0 0x555d3a9177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.602+0000 7fcf02ba3740 1 -- 172.21.15.78:0/2560802705 >> 172.21.15.78:0/2560802705 conn(0x555d3a90c300 msgr2=0x555d3a912c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:01.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.602+0000 7fcf02ba3740 1 -- 172.21.15.78:0/2560802705 shutdown_connections 2024-09-11T01:20:01.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.602+0000 7fcf02ba3740 1 -- 172.21.15.78:0/2560802705 wait complete. 2024-09-11T01:20:01.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.603+0000 7fcf02ba3740 1 -- start start 2024-09-11T01:20:01.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.603+0000 7fcf02ba3740 1 Processor -- start 2024-09-11T01:20:01.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.604+0000 7fcf02ba3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90e4d0 0x555d3a92f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.604+0000 7fcf02ba3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a90edf0 0x555d3a92f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.604+0000 7fcf02ba3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a91ad80 0x555d3a996b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.604+0000 7fcf02ba3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81a9900 con 0x555d3a90e4d0 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.604+0000 7fcf02ba3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81a9170 con 0x555d3a90edf0 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.604+0000 7fcefdd69640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90e4d0 0x555d3a92f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.604+0000 7fcf02ba3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcef8102070 con 0x555d3a91ad80 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.604+0000 7fcefdd69640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90e4d0 0x555d3a92f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45792/0 (socket says 172.21.15.78:45792) 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.604+0000 7fcefdd69640 1 -- 172.21.15.78:0/1082024060 learned_addr learned my addr 172.21.15.78:0/1082024060 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.604+0000 7fcf02ba3740 10 client.0 _ll_register_callbacks cb 0x7fcee8009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.604+0000 7fcefdd69640 1 --2- 172.21.15.78:0/1082024060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a91ad80 0x555d3a996b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.605+0000 7fcefdd69640 1 -- 172.21.15.78:0/1082024060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a91ad80 msgr2=0x555d3a996b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.605+0000 7fcefdd69640 1 --2- 172.21.15.78:0/1082024060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a91ad80 0x555d3a996b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.605+0000 7fcefdd69640 1 -- 172.21.15.78:0/1082024060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a90edf0 msgr2=0x555d3a92f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:01.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.605+0000 7fcefdd69640 1 --2- 172.21.15.78:0/1082024060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a90edf0 0x555d3a92f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.605+0000 7fcefdd69640 1 -- 172.21.15.78:0/1082024060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d3a997240 con 0x555d3a90e4d0 2024-09-11T01:20:01.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.605+0000 7fcefdd69640 1 --2- 172.21.15.78:0/1082024060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90e4d0 0x555d3a92f040 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fcee8004e40 tx=0x7fcee8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce79c82600a6486 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:01.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.605+0000 7fcee1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:20:01.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.606+0000 7fcee1ffb640 1 -- 172.21.15.78:0/1082024060 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8018030 con 0x555d3a90e4d0 2024-09-11T01:20:01.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.606+0000 7fcee1ffb640 1 -- 172.21.15.78:0/1082024060 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8449+0+0 (secure 0 0 0) 0x7fcee8004b70 con 0x555d3a90e4d0 2024-09-11T01:20:01.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.606+0000 7fcf02ba3740 1 -- 172.21.15.78:0/1082024060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef4031d10 con 0x555d3a90e4d0 2024-09-11T01:20:01.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.606+0000 7fcee1ffb640 1 -- 172.21.15.78:0/1082024060 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fcee8004020 con 0x555d3a90e4d0 2024-09-11T01:20:01.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.606+0000 7fcf02ba3740 10 client.19113 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:01.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.606+0000 7fcf02ba3740 1 -- 172.21.15.78:0/1082024060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x555d3a998550 con 0x555d3a90e4d0 2024-09-11T01:20:01.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.606+0000 7fcee1ffb640 1 -- 172.21.15.78:0/1082024060 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcef4031d10 con 0x555d3a90e4d0 2024-09-11T01:20:01.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.607+0000 7fcee1ffb640 1 -- 172.21.15.78:0/1082024060 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7fcee80035f0 con 0x555d3a90e4d0 2024-09-11T01:20:01.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.607+0000 7fcf02ba3740 10 client.19113 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:01.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.607+0000 7fcf02ba3740 10 client.19113 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:20:01.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.607+0000 7fcf02ba3740 20 client.19113 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:01.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.607+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/1082024060 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555d3a998ca0 0x555d3a99b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:01.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.607+0000 7fcf02ba3740 4 client.19113 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:20:01.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.607+0000 7fcf02ba3740 1 -- 172.21.15.78:0/1082024060 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d3a87fcd0 con 0x555d3a998ca0 2024-09-11T01:20:01.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.607+0000 7fcefe56a640 1 --2- 172.21.15.78:0/1082024060 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555d3a998ca0 0x555d3a99b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:01.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.608+0000 7fcefe56a640 1 --2- 172.21.15.78:0/1082024060 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555d3a998ca0 0x555d3a99b0c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e5cee7614e45506 server_cookie=dcc62d9dbaa47c9f in_seq=0 out_seq=0 2024-09-11T01:20:01.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.608+0000 7fcee1ffb640 10 client.19113 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:20:01.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.609+0000 7fcee1ffb640 1 -- 172.21.15.78:0/1082024060 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d3a87fcd0 con 0x555d3a998ca0 2024-09-11T01:20:01.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.609+0000 7fcee1ffb640 10 client.19113 handle_command_reply: tid=0 2024-09-11T01:20:01.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.609+0000 7fcf02ba3740 1 client.19113 shutdown 2024-09-11T01:20:01.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.610+0000 7fcf02ba3740 1 -- 172.21.15.78:0/1082024060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90e4d0 msgr2=0x555d3a92f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:01.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.610+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/1082024060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90e4d0 0x555d3a92f040 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fcee8004e40 tx=0x7fcee8009650 comp rx=0 tx=0).stop 2024-09-11T01:20:01.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.610+0000 7fcf02ba3740 1 -- 172.21.15.78:0/1082024060 shutdown_connections 2024-09-11T01:20:01.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.610+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/1082024060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d3a91ad80 0x555d3a996b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.610+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/1082024060 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x555d3a998ca0 0x555d3a99b0c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.610+0000 7fcee1ffb640 0 client.19113 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:20:01.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.610+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/1082024060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d3a90edf0 0x555d3a92f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.610+0000 7fcf02ba3740 1 --2- 172.21.15.78:0/1082024060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555d3a90e4d0 0x555d3a92f040 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.610+0000 7fcf02ba3740 1 -- 172.21.15.78:0/1082024060 >> 172.21.15.78:0/1082024060 conn(0x555d3a90c300 msgr2=0x555d3a9177f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:01.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.611+0000 7fcf02ba3740 1 -- 172.21.15.78:0/1082024060 shutdown_connections 2024-09-11T01:20:01.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.611+0000 7fcf02ba3740 1 -- 172.21.15.78:0/1082024060 wait complete. 2024-09-11T01:20:01.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.611+0000 7fcf02ba3740 20 client.19113 trim_cache size 0 max 16384 2024-09-11T01:20:01.614 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:20:01.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.612+0000 7fcf007ce640 1 -- 172.21.15.78:0/1423319923 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcee40b55a0 msgr2=0x7fcee40b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:01.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.612+0000 7fcf007ce640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcee40b55a0 0x7fcee40b7a60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcef00049d0 tx=0x7fcef002ef20 comp rx=0 tx=0).stop 2024-09-11T01:20:01.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.612+0000 7fcf007ce640 1 -- 172.21.15.78:0/1423319923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcef8108020 msgr2=0x7fcef81996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:01.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.612+0000 7fcf007ce640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcef8108020 0x7fcef81996e0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fcef0030b20 tx=0x7fcef002e2e0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.612+0000 7fcf007ce640 1 -- 172.21.15.78:0/1423319923 shutdown_connections 2024-09-11T01:20:01.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.612+0000 7fcf007ce640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fcee40b55a0 0x7fcee40b7a60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.612+0000 7fcf007ce640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcef8108020 0x7fcef81996e0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.612+0000 7fcf007ce640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcef81076a0 0x7fcef810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.612+0000 7fcf007ce640 1 --2- 172.21.15.78:0/1423319923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcef8101c90 0x7fcef810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:01.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.612+0000 7fcf007ce640 1 -- 172.21.15.78:0/1423319923 >> 172.21.15.78:0/1423319923 conn(0x7fcef80fd700 msgr2=0x7fcef80fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:01.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.613+0000 7fcf007ce640 1 -- 172.21.15.78:0/1423319923 shutdown_connections 2024-09-11T01:20:01.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:01.613+0000 7fcf007ce640 1 -- 172.21.15.78:0/1423319923 wait complete. 2024-09-11T01:20:01.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 305566 33.10 MB/sec execute 367 sec latency 101.859 ms 2024-09-11T01:20:02.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:01 smithi110 ceph-mon[34712]: pgmap v985: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 697 op/s 2024-09-11T01:20:02.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:01 smithi112 ceph-mon[35384]: pgmap v985: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 697 op/s 2024-09-11T01:20:02.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:01 smithi078 ceph-mon[31315]: pgmap v985: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 697 op/s 2024-09-11T01:20:02.466 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:20:02.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 306165 33.10 MB/sec execute 368 sec latency 111.025 ms 2024-09-11T01:20:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:20:03.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 306457 33.05 MB/sec execute 369 sec latency 292.154 ms 2024-09-11T01:20:04.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:03 smithi110 ceph-mon[34712]: pgmap v986: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 678 op/s 2024-09-11T01:20:04.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:03 smithi110 ceph-mon[34712]: osdmap e106: 12 total, 12 up, 12 in 2024-09-11T01:20:04.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:03 smithi112 ceph-mon[35384]: pgmap v986: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 678 op/s 2024-09-11T01:20:04.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:03 smithi112 ceph-mon[35384]: osdmap e106: 12 total, 12 up, 12 in 2024-09-11T01:20:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:03 smithi078 ceph-mon[31315]: pgmap v986: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 678 op/s 2024-09-11T01:20:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:03 smithi078 ceph-mon[31315]: osdmap e106: 12 total, 12 up, 12 in 2024-09-11T01:20:04.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.947+0000 7f6f1cd29640 1 -- 172.21.15.78:0/1742011337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1810f3a0 msgr2=0x7f6f18111790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:04.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.947+0000 7f6f1cd29640 1 --2- 172.21.15.78:0/1742011337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1810f3a0 0x7f6f18111790 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009ca0 tx=0x7f6f1002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.947+0000 7f6f1cd29640 1 -- 172.21.15.78:0/1742011337 shutdown_connections 2024-09-11T01:20:04.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.947+0000 7f6f1cd29640 1 --2- 172.21.15.78:0/1742011337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1810f3a0 0x7f6f18111790 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.947+0000 7f6f1cd29640 1 --2- 172.21.15.78:0/1742011337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f181020f0 0x7f6f1810edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.947+0000 7f6f1cd29640 1 --2- 172.21.15.78:0/1742011337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f181016f0 0x7f6f18101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.947+0000 7f6f1cd29640 1 -- 172.21.15.78:0/1742011337 >> 172.21.15.78:0/1742011337 conn(0x7f6f18076e50 msgr2=0x7f6f18077260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:04.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.947+0000 7f6f1cd29640 1 -- 172.21.15.78:0/1742011337 shutdown_connections 2024-09-11T01:20:04.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.947+0000 7f6f1cd29640 1 -- 172.21.15.78:0/1742011337 wait complete. 2024-09-11T01:20:04.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.948+0000 7f6f1cd29640 1 Processor -- start 2024-09-11T01:20:04.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.948+0000 7f6f1cd29640 1 -- start start 2024-09-11T01:20:04.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.948+0000 7f6f1cd29640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f181016f0 0x7f6f1819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.948+0000 7f6f1cd29640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f181020f0 0x7f6f1819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.948+0000 7f6f1cd29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f1810f3a0 0x7f6f181a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.948+0000 7f6f1cd29640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f18114f80 con 0x7f6f181020f0 2024-09-11T01:20:04.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.948+0000 7f6f1cd29640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f18114e00 con 0x7f6f181016f0 2024-09-11T01:20:04.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.948+0000 7f6f1cd29640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f18115100 con 0x7f6f1810f3a0 2024-09-11T01:20:04.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.948+0000 7f6f15d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f181020f0 0x7f6f1819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.948+0000 7f6f15d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f181020f0 0x7f6f1819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42756/0 (socket says 172.21.15.78:42756) 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f15d9b640 1 -- 172.21.15.78:0/3877668891 learned_addr learned my addr 172.21.15.78:0/3877668891 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f1659c640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f181016f0 0x7f6f1819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f16d9d640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f1810f3a0 0x7f6f181a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f15d9b640 1 -- 172.21.15.78:0/3877668891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f1810f3a0 msgr2=0x7f6f181a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f15d9b640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f1810f3a0 0x7f6f181a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f15d9b640 1 -- 172.21.15.78:0/3877668891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f181016f0 msgr2=0x7f6f1819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f15d9b640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f181016f0 0x7f6f1819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f15d9b640 1 -- 172.21.15.78:0/3877668891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f181a15a0 con 0x7f6f181020f0 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f15d9b640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f181020f0 0x7f6f1819ba30 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800d950 tx=0x7f6f0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f077fe640 1 -- 172.21.15.78:0/3877668891 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08017030 con 0x7f6f181020f0 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f1cd29640 1 -- 172.21.15.78:0/3877668891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f181a1890 con 0x7f6f181020f0 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f077fe640 1 -- 172.21.15.78:0/3877668891 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f08004020 con 0x7f6f181020f0 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f077fe640 1 -- 172.21.15.78:0/3877668891 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f080047a0 con 0x7f6f181020f0 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.949+0000 7f6f1cd29640 1 -- 172.21.15.78:0/3877668891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f181a7f50 con 0x7f6f181020f0 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.953+0000 7f6f1f0fe740 1 Processor -- start 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.953+0000 7f6f1f0fe740 1 -- start start 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f1f0fe740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 0x5625210f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f1f0fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f9a10 0x562521105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f1f0fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625211059a0 0x562521107d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f1f0fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56252106fea0 con 0x5625211059a0 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f1f0fe740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56252105f2d0 con 0x5625210f90f0 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f1f0fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5625210630e0 con 0x5625210f9a10 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f16d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 0x5625210f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f16d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 0x5625210f94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59782/0 (socket says 172.21.15.78:59782) 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f16d9d640 1 -- 172.21.15.78:0/1344659539 learned_addr learned my addr 172.21.15.78:0/1344659539 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f1659c640 1 --2- 172.21.15.78:0/1344659539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f9a10 0x562521105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f16d9d640 1 -- 172.21.15.78:0/1344659539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f9a10 msgr2=0x562521105460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f16d9d640 1 --2- 172.21.15.78:0/1344659539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f9a10 0x562521105460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f16d9d640 1 -- 172.21.15.78:0/1344659539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625211059a0 msgr2=0x562521107d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f16d9d640 1 --2- 172.21.15.78:0/1344659539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625211059a0 0x562521107d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f16d9d640 1 -- 172.21.15.78:0/1344659539 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562520dbd650 con 0x5625210f90f0 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.954+0000 7f6f077fe640 1 -- 172.21.15.78:0/3877668891 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6f08009a30 con 0x7f6f181020f0 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.955+0000 7f6f16d9d640 1 --2- 172.21.15.78:0/1344659539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 0x5625210f94d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6f10035000 tx=0x7f6f10005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ea6a26b217702b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.955+0000 7f6f04ff9640 1 -- 172.21.15.78:0/1344659539 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10031cf0 con 0x5625210f90f0 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.955+0000 7f6f04ff9640 1 -- 172.21.15.78:0/1344659539 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f10036030 con 0x5625210f90f0 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.955+0000 7f6f04ff9640 1 -- 172.21.15.78:0/1344659539 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f100042a0 con 0x5625210f90f0 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.956+0000 7f6f077fe640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6f000b54d0 0x7f6f000b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.956+0000 7f6f077fe640 1 -- 172.21.15.78:0/3877668891 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8493+0+0 (secure 0 0 0) 0x7f6f080e58d0 con 0x7f6f181020f0 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.956+0000 7f6f1659c640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6f000b54d0 0x7f6f000b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.957+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/1344659539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 msgr2=0x5625210f94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:04.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.957+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/1344659539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 0x5625210f94d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6f10035000 tx=0x7f6f10005fa0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.957+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/1344659539 shutdown_connections 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.957+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/1344659539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625211059a0 0x562521107d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.957+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/1344659539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f9a10 0x562521105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.957+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/1344659539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 0x5625210f94d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.957+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/1344659539 >> 172.21.15.78:0/1344659539 conn(0x5625210f6f20 msgr2=0x5625210f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.957+0000 7f6f1659c640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6f000b54d0 0x7f6f000b7990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x5625210f7e00 tx=0x7f6f0c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.957+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/1344659539 shutdown_connections 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.957+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/1344659539 wait complete. 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.957+0000 7f6f1f0fe740 1 -- start start 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f1f0fe740 1 Processor -- start 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f1f0fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f90f0 0x5625211052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f1f0fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 0x562521102730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f1f0fe740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625211059a0 0x562521102db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f1f0fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562520ec0700 con 0x5625210f9a10 2024-09-11T01:20:04.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f1f0fe740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562520ffa160 con 0x5625211059a0 2024-09-11T01:20:04.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f1f0fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562521108440 con 0x5625210f90f0 2024-09-11T01:20:04.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f1f0fe740 10 client.0 _ll_register_callbacks cb 0x5625210139e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:04.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f15d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 0x562521102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f1659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f90f0 0x5625211052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f16d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625211059a0 0x562521102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f15d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 0x562521102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42758/0 (socket says 172.21.15.78:42758) 2024-09-11T01:20:04.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.958+0000 7f6f15d9b640 1 -- 172.21.15.78:0/3597223918 learned_addr learned my addr 172.21.15.78:0/3597223918 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:04.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6f15d9b640 1 -- 172.21.15.78:0/3597223918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f90f0 msgr2=0x5625211052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:04.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6f15d9b640 1 --2- 172.21.15.78:0/3597223918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f90f0 0x5625211052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6f15d9b640 1 -- 172.21.15.78:0/3597223918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625211059a0 msgr2=0x562521102db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:04.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6f15d9b640 1 --2- 172.21.15.78:0/3597223918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625211059a0 0x562521102db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6f15d9b640 1 -- 172.21.15.78:0/3597223918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562521027c90 con 0x5625210f9a10 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6f15d9b640 1 --2- 172.21.15.78:0/3597223918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 0x562521102730 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f6f080a8f80 tx=0x7f6f080056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2f704b98312cbc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6efdffb640 1 -- 172.21.15.78:0/3597223918 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f080abd10 con 0x5625210f9a10 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6efdffb640 1 -- 172.21.15.78:0/3597223918 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8493+0+0 (secure 0 0 0) 0x7f6f080e3260 con 0x5625210f9a10 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3597223918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f10036030 con 0x5625210f9a10 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6efdffb640 1 -- 172.21.15.78:0/3597223918 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f6f080b07f0 con 0x5625210f9a10 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6f1f0fe740 10 client.19122 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.959+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3597223918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x562521113e80 con 0x5625210f9a10 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.960+0000 7f6efdffb640 1 -- 172.21.15.78:0/3597223918 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f10036030 con 0x5625210f9a10 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.960+0000 7f6efdffb640 1 -- 172.21.15.78:0/3597223918 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f6f08091ae0 con 0x5625210f9a10 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.960+0000 7f6f1f0fe740 10 client.19122 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.960+0000 7f6f1f0fe740 10 client.19122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:20:04.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.960+0000 7f6f1f0fe740 20 client.19122 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.960+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/3597223918 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x562521115590 0x562521117980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.960+0000 7f6f1f0fe740 4 client.19122 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.960+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3597223918 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562521071df0 con 0x562521115590 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.960+0000 7f6f1659c640 1 --2- 172.21.15.78:0/3597223918 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x562521115590 0x562521117980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.961+0000 7f6f1659c640 1 --2- 172.21.15.78:0/3597223918 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x562521115590 0x562521117980 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1837fd3526c1ec76 server_cookie=fcaa07978b0c8d99 in_seq=0 out_seq=0 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.961+0000 7f6efdffb640 10 client.19122 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.965+0000 7f6efdffb640 1 -- 172.21.15.78:0/3597223918 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562521071df0 con 0x562521115590 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6efdffb640 10 client.19122 handle_command_reply: tid=0 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6f1f0fe740 1 client.19122 shutdown 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3597223918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 msgr2=0x562521102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/3597223918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 0x562521102730 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f6f080a8f80 tx=0x7f6f080056a0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3597223918 shutdown_connections 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/3597223918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625211059a0 0x562521102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/3597223918 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x562521115590 0x562521117980 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6efdffb640 0 client.19122 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/3597223918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 0x562521102730 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/3597223918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f90f0 0x5625211052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3597223918 >> 172.21.15.78:0/3597223918 conn(0x5625210f6f20 msgr2=0x5625210fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3597223918 shutdown_connections 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3597223918 wait complete. 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.966+0000 7f6f1f0fe740 20 client.19122 trim_cache size 0 max 16384 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f1f0fe740 1 Processor -- start 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f1f0fe740 1 -- start start 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f1f0fe740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 0x5625210f6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f1f0fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 0x5625210ff980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f1f0fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625211059a0 0x5625210ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f1f0fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181b0800 con 0x5625210f9a10 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f1f0fe740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181b2400 con 0x5625210f90f0 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f1f0fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181b0ed0 con 0x5625211059a0 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f15d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 0x5625210ff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f1659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 0x5625210f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f1659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 0x5625210f6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59808/0 (socket says 172.21.15.78:59808) 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f1659c640 1 -- 172.21.15.78:0/2649973255 learned_addr learned my addr 172.21.15.78:0/2649973255 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.985+0000 7f6f15d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 0x5625210ff980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42774/0 (socket says 172.21.15.78:42774) 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.986+0000 7f6f16d9d640 1 --2- 172.21.15.78:0/2649973255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625211059a0 0x5625210ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.986+0000 7f6f1659c640 1 -- 172.21.15.78:0/2649973255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625211059a0 msgr2=0x5625210ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.986+0000 7f6f1659c640 1 --2- 172.21.15.78:0/2649973255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625211059a0 0x5625210ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.986+0000 7f6f1659c640 1 -- 172.21.15.78:0/2649973255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 msgr2=0x5625210ff980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.986+0000 7f6f1659c640 1 --2- 172.21.15.78:0/2649973255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 0x5625210ff980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.986+0000 7f6f1659c640 1 -- 172.21.15.78:0/2649973255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562521103120 con 0x5625210f90f0 2024-09-11T01:20:04.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.986+0000 7f6f15d9b640 1 --2- 172.21.15.78:0/2649973255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 0x5625210ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.986+0000 7f6f1659c640 1 --2- 172.21.15.78:0/2649973255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 0x5625210f6760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c01d7b0 tx=0x7f6f0c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ef5991920926f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.986+0000 7f6efffff640 1 -- 172.21.15.78:0/2649973255 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c01dd60 con 0x5625210f90f0 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.986+0000 7f6efffff640 1 -- 172.21.15.78:0/2649973255 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0c0054d0 con 0x5625210f90f0 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.986+0000 7f6efffff640 1 -- 172.21.15.78:0/2649973255 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c030420 con 0x5625210f90f0 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.987+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/2649973255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 msgr2=0x5625210f6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.987+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/2649973255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 0x5625210f6760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c01d7b0 tx=0x7f6f0c01b000 comp rx=0 tx=0).stop 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.987+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/2649973255 shutdown_connections 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.987+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/2649973255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625211059a0 0x5625210ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.987+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/2649973255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625210f9a10 0x5625210ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.987+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/2649973255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f90f0 0x5625210f6760 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.987+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/2649973255 >> 172.21.15.78:0/2649973255 conn(0x5625210f6f20 msgr2=0x562521104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.988+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/2649973255 shutdown_connections 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.988+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/2649973255 wait complete. 2024-09-11T01:20:04.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.988+0000 7f6f1f0fe740 1 -- start start 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.988+0000 7f6f1f0fe740 1 Processor -- start 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.988+0000 7f6f1f0fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f90f0 0x5625211000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f1f0fe740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f9a10 0x56252110d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f1f0fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625211059a0 0x56252110dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f1f0fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181b1d30 con 0x5625211059a0 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f1f0fe740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181b15a0 con 0x5625210f9a10 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f1f0fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181ae800 con 0x5625210f90f0 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f1f0fe740 10 client.0 _ll_register_callbacks cb 0x7f6f08004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f15d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f9a10 0x56252110d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f16d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625211059a0 0x56252110dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f15d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f9a10 0x56252110d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59810/0 (socket says 172.21.15.78:59810) 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f15d9b640 1 -- 172.21.15.78:0/3053816754 learned_addr learned my addr 172.21.15.78:0/3053816754 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f1659c640 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f90f0 0x5625211000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f15d9b640 1 -- 172.21.15.78:0/3053816754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f90f0 msgr2=0x5625211000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f15d9b640 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f90f0 0x5625211000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f15d9b640 1 -- 172.21.15.78:0/3053816754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625211059a0 msgr2=0x56252110dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f15d9b640 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625211059a0 0x56252110dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f15d9b640 1 -- 172.21.15.78:0/3053816754 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625210f9840 con 0x5625210f9a10 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f16d9d640 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625211059a0 0x56252110dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6f15d9b640 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f9a10 0x56252110d730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6f080aba10 tx=0x7f6f08004400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8590db59bc5455f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.989+0000 7f6efdffb640 1 -- 172.21.15.78:0/3053816754 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08014070 con 0x5625210f9a10 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.990+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3053816754 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f0c0054d0 con 0x5625210f9a10 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.990+0000 7f6efdffb640 1 -- 172.21.15.78:0/3053816754 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8493+0+0 (secure 0 0 0) 0x7f6f080aaf70 con 0x5625210f9a10 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.990+0000 7f6efdffb640 1 -- 172.21.15.78:0/3053816754 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f6f0805b040 con 0x5625210f9a10 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.990+0000 7f6f1f0fe740 10 client.28184 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:04.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.990+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3053816754 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x5625210ff780 con 0x5625210f9a10 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.990+0000 7f6efdffb640 1 -- 172.21.15.78:0/3053816754 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0c0054d0 con 0x5625210f9a10 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.991+0000 7f6efdffb640 1 -- 172.21.15.78:0/3053816754 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f6f080b1980 con 0x5625210f9a10 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.991+0000 7f6f1f0fe740 10 client.28184 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.991+0000 7f6f1f0fe740 10 client.28184 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.991+0000 7f6f1f0fe740 20 client.28184 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.991+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x562521115590 0x562521185960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.991+0000 7f6f1f0fe740 4 client.28184 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.991+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3053816754 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56252106d8a0 con 0x562521115590 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.991+0000 7f6f1659c640 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x562521115590 0x562521185960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.992+0000 7f6f1659c640 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x562521115590 0x562521185960 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2a91468aa542428 server_cookie=f963060c42bbf70f in_seq=0 out_seq=0 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.992+0000 7f6efdffb640 10 client.28184 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.993+0000 7f6efdffb640 1 -- 172.21.15.78:0/3053816754 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56252106d8a0 con 0x562521115590 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.993+0000 7f6efdffb640 10 client.28184 handle_command_reply: tid=0 2024-09-11T01:20:04.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.993+0000 7f6f1f0fe740 1 client.28184 shutdown 2024-09-11T01:20:04.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.994+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3053816754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f9a10 msgr2=0x56252110d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:04.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.994+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f9a10 0x56252110d730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6f080aba10 tx=0x7f6f08004400 comp rx=0 tx=0).stop 2024-09-11T01:20:04.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.994+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3053816754 shutdown_connections 2024-09-11T01:20:04.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.994+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x562521115590 0x562521185960 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.994+0000 7f6efdffb640 0 client.28184 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:20:04.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.994+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625211059a0 0x56252110dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.994+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5625210f9a10 0x56252110d730 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.994+0000 7f6f1f0fe740 1 --2- 172.21.15.78:0/3053816754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625210f90f0 0x5625211000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.994+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3053816754 >> 172.21.15.78:0/3053816754 conn(0x5625210f6f20 msgr2=0x562521100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:04.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.994+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3053816754 shutdown_connections 2024-09-11T01:20:04.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.994+0000 7f6f1f0fe740 1 -- 172.21.15.78:0/3053816754 wait complete. 2024-09-11T01:20:04.687 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:20:04.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.995+0000 7f6f1f0fe740 20 client.28184 trim_cache size 0 max 16384 2024-09-11T01:20:04.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.995+0000 7f6f1cd29640 1 -- 172.21.15.78:0/3877668891 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6f000b54d0 msgr2=0x7f6f000b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:04.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.995+0000 7f6f1cd29640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6f000b54d0 0x7f6f000b7990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x5625210f7e00 tx=0x7f6f0c005f90 comp rx=0 tx=0).stop 2024-09-11T01:20:04.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.995+0000 7f6f1cd29640 1 -- 172.21.15.78:0/3877668891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f181020f0 msgr2=0x7f6f1819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:04.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.995+0000 7f6f1cd29640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f181020f0 0x7f6f1819ba30 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800d950 tx=0x7f6f0800de10 comp rx=0 tx=0).stop 2024-09-11T01:20:04.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.995+0000 7f6f1cd29640 1 -- 172.21.15.78:0/3877668891 shutdown_connections 2024-09-11T01:20:04.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.995+0000 7f6f1cd29640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f6f000b54d0 0x7f6f000b7990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.995+0000 7f6f1cd29640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f1810f3a0 0x7f6f181a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.995+0000 7f6f1cd29640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f181020f0 0x7f6f1819ba30 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.995+0000 7f6f1cd29640 1 --2- 172.21.15.78:0/3877668891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f181016f0 0x7f6f1819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:04.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.995+0000 7f6f1cd29640 1 -- 172.21.15.78:0/3877668891 >> 172.21.15.78:0/3877668891 conn(0x7f6f18076e50 msgr2=0x7f6f1810f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:04.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.996+0000 7f6f1cd29640 1 -- 172.21.15.78:0/3877668891 shutdown_connections 2024-09-11T01:20:04.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:03.996+0000 7f6f1cd29640 1 -- 172.21.15.78:0/3877668891 wait complete. 2024-09-11T01:20:04.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 306914 33.04 MB/sec execute 370 sec latency 367.458 ms 2024-09-11T01:20:05.799 INFO:tasks.workunit.client.0.smithi078.stdout: 10 307263 33.01 MB/sec execute 371 sec latency 658.324 ms 2024-09-11T01:20:06.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:05 smithi110 ceph-mon[34712]: pgmap v988: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 699 op/s 2024-09-11T01:20:06.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:05 smithi112 ceph-mon[35384]: pgmap v988: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 699 op/s 2024-09-11T01:20:06.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:05 smithi078 ceph-mon[31315]: pgmap v988: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 699 op/s 2024-09-11T01:20:06.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 307766 32.97 MB/sec execute 372 sec latency 141.092 ms 2024-09-11T01:20:07.245 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:20:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:20:07.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 308480 32.99 MB/sec execute 373 sec latency 74.439 ms 2024-09-11T01:20:08.062 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:07 smithi078 ceph-mon[31315]: pgmap v989: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 750 op/s 2024-09-11T01:20:08.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:07 smithi110 ceph-mon[34712]: pgmap v989: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 750 op/s 2024-09-11T01:20:08.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:07 smithi112 ceph-mon[35384]: pgmap v989: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 750 op/s 2024-09-11T01:20:08.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 308955 32.96 MB/sec execute 374 sec latency 395.079 ms 2024-09-11T01:20:09.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.436+0000 7f73d3107640 1 -- 172.21.15.78:0/134335066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73cc0776d0 msgr2=0x7f73cc077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:09.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.436+0000 7f73d3107640 1 --2- 172.21.15.78:0/134335066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73cc0776d0 0x7f73cc077ab0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009900 tx=0x7f73c002a3f0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.436+0000 7f73d3107640 1 -- 172.21.15.78:0/134335066 shutdown_connections 2024-09-11T01:20:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.436+0000 7f73d3107640 1 --2- 172.21.15.78:0/134335066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73cc1139b0 0x7f73cc115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.436+0000 7f73d3107640 1 --2- 172.21.15.78:0/134335066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73cc077ff0 0x7f73cc113470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.436+0000 7f73d3107640 1 --2- 172.21.15.78:0/134335066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73cc0776d0 0x7f73cc077ab0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.436+0000 7f73d3107640 1 -- 172.21.15.78:0/134335066 >> 172.21.15.78:0/134335066 conn(0x7f73cc1009f0 msgr2=0x7f73cc102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.437+0000 7f73d3107640 1 -- 172.21.15.78:0/134335066 shutdown_connections 2024-09-11T01:20:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.437+0000 7f73d3107640 1 -- 172.21.15.78:0/134335066 wait complete. 2024-09-11T01:20:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73d3107640 1 Processor -- start 2024-09-11T01:20:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73d3107640 1 -- start start 2024-09-11T01:20:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73d3107640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73cc0776d0 0x7f73cc19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73d3107640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73cc077ff0 0x7f73cc19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73d3107640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73cc1139b0 0x7f73cc1a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73d3107640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1193f0 con 0x7f73cc077ff0 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73d3107640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc119270 con 0x7f73cc1139b0 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73d3107640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc119570 con 0x7f73cc0776d0 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73d0ea3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73cc0776d0 0x7f73cc19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73d0ea3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73cc0776d0 0x7f73cc19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54352/0 (socket says 172.21.15.78:54352) 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73d0ea3640 1 -- 172.21.15.78:0/153364124 learned_addr learned my addr 172.21.15.78:0/153364124 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73cbfff640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73cc077ff0 0x7f73cc19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.438+0000 7f73d16a4640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73cc1139b0 0x7f73cc1a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.439+0000 7f73d0ea3640 1 -- 172.21.15.78:0/153364124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73cc1139b0 msgr2=0x7f73cc1a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.439+0000 7f73d0ea3640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73cc1139b0 0x7f73cc1a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.439+0000 7f73d0ea3640 1 -- 172.21.15.78:0/153364124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73cc077ff0 msgr2=0x7f73cc19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.439+0000 7f73d0ea3640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73cc077ff0 0x7f73cc19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.439+0000 7f73d0ea3640 1 -- 172.21.15.78:0/153364124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc1a5750 con 0x7f73cc0776d0 2024-09-11T01:20:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.439+0000 7f73d0ea3640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73cc0776d0 0x7f73cc19f6a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009850 tx=0x7f73c0030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.439+0000 7f73c9ffb640 1 -- 172.21.15.78:0/153364124 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c003f070 con 0x7f73cc0776d0 2024-09-11T01:20:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.439+0000 7f73c9ffb640 1 -- 172.21.15.78:0/153364124 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c0004460 con 0x7f73cc0776d0 2024-09-11T01:20:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.439+0000 7f73c9ffb640 1 -- 172.21.15.78:0/153364124 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0035030 con 0x7f73cc0776d0 2024-09-11T01:20:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.440+0000 7f73d3107640 1 -- 172.21.15.78:0/153364124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73cc1a59e0 con 0x7f73cc0776d0 2024-09-11T01:20:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.440+0000 7f73d3107640 1 -- 172.21.15.78:0/153364124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73cc1ac100 con 0x7f73cc0776d0 2024-09-11T01:20:09.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.444+0000 7f73c9ffb640 1 -- 172.21.15.78:0/153364124 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f73c0030870 con 0x7f73cc0776d0 2024-09-11T01:20:09.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.445+0000 7f73c9ffb640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f73bc0b54d0 0x7f73bc0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.445+0000 7f73d54dc740 1 Processor -- start 2024-09-11T01:20:09.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.446+0000 7f73d54dc740 1 -- start start 2024-09-11T01:20:09.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.446+0000 7f73c9ffb640 1 -- 172.21.15.78:0/153364124 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8493+0+0 (secure 0 0 0) 0x7f73c0106960 con 0x7f73cc0776d0 2024-09-11T01:20:09.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.446+0000 7f73d54dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bce4d0 0x555ab5bce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.446+0000 7f73d54dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bcedf0 0x555ab5bda840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.446+0000 7f73d54dc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bdd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.446+0000 7f73d54dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555ab5b44ea0 con 0x555ab5bcedf0 2024-09-11T01:20:09.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.446+0000 7f73d54dc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555ab5b342d0 con 0x555ab5bdad80 2024-09-11T01:20:09.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.446+0000 7f73d54dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555ab5b380e0 con 0x555ab5bce4d0 2024-09-11T01:20:09.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.446+0000 7f73d16a4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bdd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.446+0000 7f73d16a4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bdd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59830/0 (socket says 172.21.15.78:59830) 2024-09-11T01:20:09.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.446+0000 7f73d16a4640 1 -- 172.21.15.78:0/1794903716 learned_addr learned my addr 172.21.15.78:0/1794903716 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:09.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.447+0000 7f73d0ea3640 1 --2- 172.21.15.78:0/1794903716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bce4d0 0x555ab5bce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.447+0000 7f73d0ea3640 1 -- 172.21.15.78:0/1794903716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 msgr2=0x555ab5bdd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:09.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.447+0000 7f73d0ea3640 1 --2- 172.21.15.78:0/1794903716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bdd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.447+0000 7f73d0ea3640 1 -- 172.21.15.78:0/1794903716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bcedf0 msgr2=0x555ab5bda840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:20:09.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.447+0000 7f73d0ea3640 1 --2- 172.21.15.78:0/1794903716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bcedf0 0x555ab5bda840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.447+0000 7f73d0ea3640 1 -- 172.21.15.78:0/1794903716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab5892650 con 0x555ab5bce4d0 2024-09-11T01:20:09.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.447+0000 7f73cbfff640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f73bc0b54d0 0x7f73bc0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.447+0000 7f73d0ea3640 1 --2- 172.21.15.78:0/1794903716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bce4d0 0x555ab5bce8b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f73c01040d0 tx=0x7f73c002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3443f83f63aef9c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:09.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.447+0000 7f73b6ffd640 1 -- 172.21.15.78:0/1794903716 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c000b0c0 con 0x555ab5bce4d0 2024-09-11T01:20:09.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.447+0000 7f73b6ffd640 1 -- 172.21.15.78:0/1794903716 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c00cc9c0 con 0x555ab5bce4d0 2024-09-11T01:20:09.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.448+0000 7f73d54dc740 1 -- 172.21.15.78:0/1794903716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bce4d0 msgr2=0x555ab5bce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:09.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.448+0000 7f73cbfff640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f73bc0b54d0 0x7f73bc0b7990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x555ab5bcd1e0 tx=0x7f73b80099c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:09.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.448+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1794903716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bce4d0 0x555ab5bce8b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f73c01040d0 tx=0x7f73c002a3f0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.449+0000 7f73d54dc740 1 -- 172.21.15.78:0/1794903716 shutdown_connections 2024-09-11T01:20:09.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.449+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1794903716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bdd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.449+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1794903716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bcedf0 0x555ab5bda840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.449+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1794903716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bce4d0 0x555ab5bce8b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.449+0000 7f73d54dc740 1 -- 172.21.15.78:0/1794903716 >> 172.21.15.78:0/1794903716 conn(0x555ab5bcc300 msgr2=0x555ab5bcd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:09.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.449+0000 7f73d54dc740 1 -- 172.21.15.78:0/1794903716 shutdown_connections 2024-09-11T01:20:09.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.449+0000 7f73d54dc740 1 -- 172.21.15.78:0/1794903716 wait complete. 2024-09-11T01:20:09.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.449+0000 7f73d54dc740 1 -- start start 2024-09-11T01:20:09.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.449+0000 7f73d54dc740 1 Processor -- start 2024-09-11T01:20:09.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.450+0000 7f73d54dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 0x555ab5bda590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.450+0000 7f73d54dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bcedf0 0x555ab5bd79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.450+0000 7f73d54dc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bd7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.450+0000 7f73d54dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555ab5995700 con 0x555ab5bce4d0 2024-09-11T01:20:09.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.450+0000 7f73d54dc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555ab5acf160 con 0x555ab5bdad80 2024-09-11T01:20:09.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.450+0000 7f73d54dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555ab5bdd7c0 con 0x555ab5bcedf0 2024-09-11T01:20:09.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.450+0000 7f73d54dc740 10 client.0 _ll_register_callbacks cb 0x555ab5ae89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:09.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.450+0000 7f73d0ea3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 0x555ab5bda590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.450+0000 7f73d0ea3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 0x555ab5bda590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42800/0 (socket says 172.21.15.78:42800) 2024-09-11T01:20:09.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.450+0000 7f73d0ea3640 1 -- 172.21.15.78:0/1579292624 learned_addr learned my addr 172.21.15.78:0/1579292624 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:09.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.450+0000 7f73d16a4640 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bd7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.451+0000 7f73d16a4640 1 -- 172.21.15.78:0/1579292624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bcedf0 msgr2=0x555ab5bd79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:09.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.451+0000 7f73d16a4640 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bcedf0 0x555ab5bd79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.451+0000 7f73d16a4640 1 -- 172.21.15.78:0/1579292624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 msgr2=0x555ab5bda590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:09.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.451+0000 7f73d16a4640 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 0x555ab5bda590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.451+0000 7f73d16a4640 1 -- 172.21.15.78:0/1579292624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ab5afcc90 con 0x555ab5bdad80 2024-09-11T01:20:09.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.451+0000 7f73d0ea3640 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 0x555ab5bda590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:20:09.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.451+0000 7f73abfff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:20:09.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.451+0000 7f73d16a4640 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bd7f20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f73c4011d10 tx=0x7f73c400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb95ba82d7bcc711 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:09.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.451+0000 7f73abfff640 1 -- 172.21.15.78:0/1579292624 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c401e060 con 0x555ab5bdad80 2024-09-11T01:20:09.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.451+0000 7f73abfff640 1 -- 172.21.15.78:0/1579292624 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8493+0+0 (secure 0 0 0) 0x7f73c4017050 con 0x555ab5bdad80 2024-09-11T01:20:09.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.451+0000 7f73d54dc740 1 -- 172.21.15.78:0/1579292624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c00c92b0 con 0x555ab5bdad80 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.452+0000 7f73abfff640 1 -- 172.21.15.78:0/1579292624 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f73c401e060 con 0x555ab5bdad80 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.452+0000 7f73d54dc740 10 client.28196 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.452+0000 7f73d54dc740 1 -- 172.21.15.78:0/1579292624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x555ab5be90d0 con 0x555ab5bdad80 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.452+0000 7f73abfff640 1 -- 172.21.15.78:0/1579292624 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c00c92b0 con 0x555ab5bdad80 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.453+0000 7f73abfff640 1 -- 172.21.15.78:0/1579292624 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f73c4019070 con 0x555ab5bdad80 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.453+0000 7f73d54dc740 10 client.28196 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.453+0000 7f73d54dc740 10 client.28196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.453+0000 7f73d54dc740 20 client.28196 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.453+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555ab5bea590 0x555ab5bec980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.453+0000 7f73d54dc740 4 client.28196 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.453+0000 7f73d54dc740 1 -- 172.21.15.78:0/1579292624 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ab5b46df0 con 0x555ab5bea590 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.453+0000 7f73d0ea3640 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555ab5bea590 0x555ab5bec980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.454+0000 7f73abfff640 10 client.28196 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:09.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.454+0000 7f73d0ea3640 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555ab5bea590 0x555ab5bec980 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6023e97e2b0f9e3 server_cookie=73525faa7cc5a552 in_seq=0 out_seq=0 2024-09-11T01:20:09.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.454+0000 7f73abfff640 1 -- 172.21.15.78:0/1579292624 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555ab5b46df0 con 0x555ab5bea590 2024-09-11T01:20:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.454+0000 7f73abfff640 10 client.28196 handle_command_reply: tid=0 2024-09-11T01:20:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.454+0000 7f73d54dc740 1 client.28196 shutdown 2024-09-11T01:20:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.455+0000 7f73d54dc740 1 -- 172.21.15.78:0/1579292624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 msgr2=0x555ab5bd7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.455+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bd7f20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f73c4011d10 tx=0x7f73c400f790 comp rx=0 tx=0).stop 2024-09-11T01:20:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.455+0000 7f73d54dc740 1 -- 172.21.15.78:0/1579292624 shutdown_connections 2024-09-11T01:20:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.455+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bd7f20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.455+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bcedf0 0x555ab5bd79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.455+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555ab5bea590 0x555ab5bec980 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.455+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1579292624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 0x555ab5bda590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.455+0000 7f73d54dc740 1 -- 172.21.15.78:0/1579292624 >> 172.21.15.78:0/1579292624 conn(0x555ab5bcc300 msgr2=0x555ab5bd1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.455+0000 7f73abfff640 0 client.28196 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.455+0000 7f73d54dc740 1 -- 172.21.15.78:0/1579292624 shutdown_connections 2024-09-11T01:20:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.455+0000 7f73d54dc740 1 -- 172.21.15.78:0/1579292624 wait complete. 2024-09-11T01:20:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.455+0000 7f73d54dc740 20 client.28196 trim_cache size 0 max 16384 2024-09-11T01:20:09.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.476+0000 7f73d54dc740 1 Processor -- start 2024-09-11T01:20:09.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.477+0000 7f73d54dc740 1 -- start start 2024-09-11T01:20:09.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.477+0000 7f73d54dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bce4d0 0x555ab5bd4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.477+0000 7f73d54dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bcedf0 0x555ab5bd5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.477+0000 7f73d54dc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bf2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.477+0000 7f73d54dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1b4fd0 con 0x555ab5bcedf0 2024-09-11T01:20:09.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.477+0000 7f73d54dc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1b6bd0 con 0x555ab5bdad80 2024-09-11T01:20:09.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.477+0000 7f73d54dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1b56a0 con 0x555ab5bce4d0 2024-09-11T01:20:09.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.477+0000 7f73d0ea3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bce4d0 0x555ab5bd4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.477+0000 7f73d0ea3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bce4d0 0x555ab5bd4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:54374/0 (socket says 172.21.15.78:54374) 2024-09-11T01:20:09.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.477+0000 7f73d0ea3640 1 -- 172.21.15.78:0/3144189947 learned_addr learned my addr 172.21.15.78:0/3144189947 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:09.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.477+0000 7f73d16a4640 1 --2- 172.21.15.78:0/3144189947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bf2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.477+0000 7f73cbfff640 1 --2- 172.21.15.78:0/3144189947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bcedf0 0x555ab5bd5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.478+0000 7f73d16a4640 1 -- 172.21.15.78:0/3144189947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bce4d0 msgr2=0x555ab5bd4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:09.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.478+0000 7f73d16a4640 1 --2- 172.21.15.78:0/3144189947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bce4d0 0x555ab5bd4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.478+0000 7f73d16a4640 1 -- 172.21.15.78:0/3144189947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bcedf0 msgr2=0x555ab5bd5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:09.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.478+0000 7f73d16a4640 1 --2- 172.21.15.78:0/3144189947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bcedf0 0x555ab5bd5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.478+0000 7f73d16a4640 1 -- 172.21.15.78:0/3144189947 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab5bcec20 con 0x555ab5bdad80 2024-09-11T01:20:09.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.478+0000 7f73d16a4640 1 --2- 172.21.15.78:0/3144189947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bf2200 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f73c401ac20 tx=0x7f73c4011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=121b317aa62e5122 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:09.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.478+0000 7f73b67fc640 1 -- 172.21.15.78:0/3144189947 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4030070 con 0x555ab5bdad80 2024-09-11T01:20:09.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.478+0000 7f73b67fc640 1 -- 172.21.15.78:0/3144189947 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c4003e20 con 0x555ab5bdad80 2024-09-11T01:20:09.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.478+0000 7f73b67fc640 1 -- 172.21.15.78:0/3144189947 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c401e060 con 0x555ab5bdad80 2024-09-11T01:20:09.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.479+0000 7f73d54dc740 1 -- 172.21.15.78:0/3144189947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 msgr2=0x555ab5bf2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:09.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.479+0000 7f73d54dc740 1 --2- 172.21.15.78:0/3144189947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bf2200 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f73c401ac20 tx=0x7f73c4011900 comp rx=0 tx=0).stop 2024-09-11T01:20:09.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.480+0000 7f73d54dc740 1 -- 172.21.15.78:0/3144189947 shutdown_connections 2024-09-11T01:20:09.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.480+0000 7f73d54dc740 1 --2- 172.21.15.78:0/3144189947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bdad80 0x555ab5bf2200 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.480+0000 7f73d54dc740 1 --2- 172.21.15.78:0/3144189947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bcedf0 0x555ab5bd5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.480+0000 7f73d54dc740 1 --2- 172.21.15.78:0/3144189947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bce4d0 0x555ab5bd4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.480+0000 7f73d54dc740 1 -- 172.21.15.78:0/3144189947 >> 172.21.15.78:0/3144189947 conn(0x555ab5bcc300 msgr2=0x555ab5bcd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:09.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.480+0000 7f73d54dc740 1 -- 172.21.15.78:0/3144189947 shutdown_connections 2024-09-11T01:20:09.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.480+0000 7f73d54dc740 1 -- 172.21.15.78:0/3144189947 wait complete. 2024-09-11T01:20:09.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.480+0000 7f73d54dc740 1 -- start start 2024-09-11T01:20:09.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.480+0000 7f73d54dc740 1 Processor -- start 2024-09-11T01:20:09.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.481+0000 7f73d54dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 0x555ab5be3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.481+0000 7f73d54dc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bcedf0 0x555ab5bde8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.481+0000 7f73d54dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bdad80 0x555ab5bdef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.481+0000 7f73d0ea3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 0x555ab5be3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.481+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bcedf0 0x555ab5bde8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.481+0000 7f73d0ea3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 0x555ab5be3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42816/0 (socket says 172.21.15.78:42816) 2024-09-11T01:20:09.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.481+0000 7f73d0ea3640 1 -- 172.21.15.78:0/1312627088 learned_addr learned my addr 172.21.15.78:0/1312627088 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:09.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.481+0000 7f73d54dc740 1 -- 172.21.15.78:0/1312627088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1b6500 con 0x555ab5bce4d0 2024-09-11T01:20:09.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.482+0000 7f73d16a4640 1 --2- 172.21.15.78:0/1312627088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bdad80 0x555ab5bdef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.482+0000 7f73d54dc740 1 -- 172.21.15.78:0/1312627088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1b5d70 con 0x555ab5bcedf0 2024-09-11T01:20:09.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.482+0000 7f73d54dc740 1 -- 172.21.15.78:0/1312627088 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1b2f50 con 0x555ab5bdad80 2024-09-11T01:20:09.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.482+0000 7f73d54dc740 10 client.0 _ll_register_callbacks cb 0x7f73c4009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:09.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.482+0000 7f73cbfff640 1 -- 172.21.15.78:0/1312627088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bdad80 msgr2=0x555ab5bdef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:09.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.482+0000 7f73cbfff640 1 --2- 172.21.15.78:0/1312627088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bdad80 0x555ab5bdef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.482+0000 7f73cbfff640 1 -- 172.21.15.78:0/1312627088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 msgr2=0x555ab5be3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:09.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.482+0000 7f73cbfff640 1 --2- 172.21.15.78:0/1312627088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 0x555ab5be3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.482+0000 7f73cbfff640 1 -- 172.21.15.78:0/1312627088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ab5c56400 con 0x555ab5bcedf0 2024-09-11T01:20:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.482+0000 7f73cbfff640 1 --2- 172.21.15.78:0/1312627088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bcedf0 0x555ab5bde8b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009460 tx=0x7f73b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32b1ff72bd2c678b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.482+0000 7f73abfff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:20:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.483+0000 7f73abfff640 1 -- 172.21.15.78:0/1312627088 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b801f900 con 0x555ab5bcedf0 2024-09-11T01:20:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.483+0000 7f73d54dc740 1 -- 172.21.15.78:0/1312627088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c4003e20 con 0x555ab5bcedf0 2024-09-11T01:20:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.483+0000 7f73abfff640 1 -- 172.21.15.78:0/1312627088 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8493+0+0 (secure 0 0 0) 0x7f73b801faa0 con 0x555ab5bcedf0 2024-09-11T01:20:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.483+0000 7f73abfff640 1 -- 172.21.15.78:0/1312627088 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f73b801fca0 con 0x555ab5bcedf0 2024-09-11T01:20:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.483+0000 7f73d54dc740 10 client.28202 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.483+0000 7f73d54dc740 1 -- 172.21.15.78:0/1312627088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x555ab5c57680 con 0x555ab5bcedf0 2024-09-11T01:20:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.483+0000 7f73abfff640 1 -- 172.21.15.78:0/1312627088 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c4003e20 con 0x555ab5bcedf0 2024-09-11T01:20:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.484+0000 7f73abfff640 1 -- 172.21.15.78:0/1312627088 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f73b802f070 con 0x555ab5bcedf0 2024-09-11T01:20:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.484+0000 7f73d54dc740 10 client.28202 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.484+0000 7f73d54dc740 10 client.28202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:20:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.484+0000 7f73d54dc740 20 client.28202 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.484+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1312627088 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555ab5c57e30 0x555ab5c5a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.484+0000 7f73d54dc740 4 client.28202 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:20:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.484+0000 7f73d54dc740 1 -- 172.21.15.78:0/1312627088 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ab5b428a0 con 0x555ab5c57e30 2024-09-11T01:20:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.485+0000 7f73d0ea3640 1 --2- 172.21.15.78:0/1312627088 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555ab5c57e30 0x555ab5c5a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:09.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.486+0000 7f73d0ea3640 1 --2- 172.21.15.78:0/1312627088 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555ab5c57e30 0x555ab5c5a250 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c18477e0b9c9014 server_cookie=74e998e2d4c41ac9 in_seq=0 out_seq=0 2024-09-11T01:20:09.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.486+0000 7f73abfff640 10 client.28202 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:09.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.486+0000 7f73abfff640 1 -- 172.21.15.78:0/1312627088 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ab5b428a0 con 0x555ab5c57e30 2024-09-11T01:20:09.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.486+0000 7f73abfff640 10 client.28202 handle_command_reply: tid=0 2024-09-11T01:20:09.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.486+0000 7f73d54dc740 1 client.28202 shutdown 2024-09-11T01:20:09.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.487+0000 7f73d54dc740 1 -- 172.21.15.78:0/1312627088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bcedf0 msgr2=0x555ab5bde8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:09.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.487+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1312627088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bcedf0 0x555ab5bde8b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009460 tx=0x7f73b80046c0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.487+0000 7f73d54dc740 1 -- 172.21.15.78:0/1312627088 shutdown_connections 2024-09-11T01:20:09.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.487+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1312627088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ab5bdad80 0x555ab5bdef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.487+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1312627088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ab5bcedf0 0x555ab5bde8b0 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009460 tx=0x7f73b80046c0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.487+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1312627088 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x555ab5c57e30 0x555ab5c5a250 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.487+0000 7f73d54dc740 1 --2- 172.21.15.78:0/1312627088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ab5bce4d0 0x555ab5be3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.487+0000 7f73d54dc740 1 -- 172.21.15.78:0/1312627088 >> 172.21.15.78:0/1312627088 conn(0x555ab5bcc300 msgr2=0x555ab5bcac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.487+0000 7f73abfff640 0 client.28202 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.487+0000 7f73d54dc740 1 -- 172.21.15.78:0/1312627088 shutdown_connections 2024-09-11T01:20:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.487+0000 7f73d54dc740 1 -- 172.21.15.78:0/1312627088 wait complete. 2024-09-11T01:20:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.487+0000 7f73d54dc740 20 client.28202 trim_cache size 0 max 16384 2024-09-11T01:20:09.491 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:20:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.489+0000 7f73d3107640 1 -- 172.21.15.78:0/153364124 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f73bc0b54d0 msgr2=0x7f73bc0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.489+0000 7f73d3107640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f73bc0b54d0 0x7f73bc0b7990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x555ab5bcd1e0 tx=0x7f73b80099c0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.489+0000 7f73d3107640 1 -- 172.21.15.78:0/153364124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73cc0776d0 msgr2=0x7f73cc19f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.489+0000 7f73d3107640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73cc0776d0 0x7f73cc19f6a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009850 tx=0x7f73c0030650 comp rx=0 tx=0).stop 2024-09-11T01:20:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.489+0000 7f73d3107640 1 -- 172.21.15.78:0/153364124 shutdown_connections 2024-09-11T01:20:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.489+0000 7f73d3107640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f73bc0b54d0 0x7f73bc0b7990 secure :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0x555ab5bcd1e0 tx=0x7f73b80099c0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.489+0000 7f73d3107640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73cc1139b0 0x7f73cc1a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.489+0000 7f73d3107640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73cc077ff0 0x7f73cc19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.489+0000 7f73d3107640 1 --2- 172.21.15.78:0/153364124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73cc0776d0 0x7f73cc19f6a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:09.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.489+0000 7f73d3107640 1 -- 172.21.15.78:0/153364124 >> 172.21.15.78:0/153364124 conn(0x7f73cc1009f0 msgr2=0x7f73cc102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:09.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.489+0000 7f73d3107640 1 -- 172.21.15.78:0/153364124 shutdown_connections 2024-09-11T01:20:09.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:09.489+0000 7f73d3107640 1 -- 172.21.15.78:0/153364124 wait complete. 2024-09-11T01:20:09.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 309575 32.98 MB/sec execute 375 sec latency 161.879 ms 2024-09-11T01:20:10.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:09 smithi110 ceph-mon[34712]: pgmap v990: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 706 op/s 2024-09-11T01:20:10.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:09 smithi112 ceph-mon[35384]: pgmap v990: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 706 op/s 2024-09-11T01:20:10.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:09 smithi078 ceph-mon[31315]: pgmap v990: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 706 op/s 2024-09-11T01:20:10.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 310214 32.98 MB/sec execute 376 sec latency 84.455 ms 2024-09-11T01:20:11.348 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:20:11.348 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:20:11.723 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:20:11.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 310781 32.96 MB/sec execute 377 sec latency 82.516 ms 2024-09-11T01:20:11.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:11 smithi112 ceph-mon[35384]: pgmap v991: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 753 op/s 2024-09-11T01:20:12.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:11 smithi078 ceph-mon[31315]: pgmap v991: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 753 op/s 2024-09-11T01:20:12.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:11 smithi110 ceph-mon[34712]: pgmap v991: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 753 op/s 2024-09-11T01:20:12.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.302+0000 7f7bf3519640 1 -- 172.21.15.78:0/4230715433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bec10d800 msgr2=0x7f7bec10dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:12.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.302+0000 7f7bf3519640 1 --2- 172.21.15.78:0/4230715433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bec10d800 0x7f7bec10dbe0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009900 tx=0x7f7be802a410 comp rx=0 tx=0).stop 2024-09-11T01:20:12.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.303+0000 7f7bf3519640 1 -- 172.21.15.78:0/4230715433 shutdown_connections 2024-09-11T01:20:12.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.303+0000 7f7bf3519640 1 --2- 172.21.15.78:0/4230715433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bec113b40 0x7f7bec11a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.303+0000 7f7bf3519640 1 --2- 172.21.15.78:0/4230715433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bec1131c0 0x7f7bec113600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.303+0000 7f7bf3519640 1 --2- 172.21.15.78:0/4230715433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bec10d800 0x7f7bec10dbe0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.303+0000 7f7bf3519640 1 -- 172.21.15.78:0/4230715433 >> 172.21.15.78:0/4230715433 conn(0x7f7bec06dce0 msgr2=0x7f7bec06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:12.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.303+0000 7f7bf3519640 1 -- 172.21.15.78:0/4230715433 shutdown_connections 2024-09-11T01:20:12.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.303+0000 7f7bf3519640 1 -- 172.21.15.78:0/4230715433 wait complete. 2024-09-11T01:20:12.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.303+0000 7f7bf3519640 1 Processor -- start 2024-09-11T01:20:12.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.304+0000 7f7bf3519640 1 -- start start 2024-09-11T01:20:12.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.304+0000 7f7bf3519640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bec10d800 0x7f7bec1a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.304+0000 7f7bf3519640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bec1131c0 0x7f7bec1a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.304+0000 7f7bf3519640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bec113b40 0x7f7bec1a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.304+0000 7f7bf3519640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec11d7b0 con 0x7f7bec1131c0 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.304+0000 7f7bf3519640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec11d630 con 0x7f7bec10d800 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.304+0000 7f7bf3519640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec11d930 con 0x7f7bec113b40 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.304+0000 7f7bf0ab4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bec1131c0 0x7f7bec1a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.304+0000 7f7bf0ab4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bec1131c0 0x7f7bec1a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42840/0 (socket says 172.21.15.78:42840) 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.304+0000 7f7bf0ab4640 1 -- 172.21.15.78:0/3914050810 learned_addr learned my addr 172.21.15.78:0/3914050810 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.305+0000 7f7bf12b5640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bec10d800 0x7f7bec1a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.305+0000 7f7bf0ab4640 1 -- 172.21.15.78:0/3914050810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bec113b40 msgr2=0x7f7bec1a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.305+0000 7f7bf0ab4640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bec113b40 0x7f7bec1a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.305+0000 7f7bf0ab4640 1 -- 172.21.15.78:0/3914050810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bec10d800 msgr2=0x7f7bec1a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.305+0000 7f7bf0ab4640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bec10d800 0x7f7bec1a3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.305+0000 7f7bf0ab4640 1 -- 172.21.15.78:0/3914050810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bec1a9d70 con 0x7f7bec1131c0 2024-09-11T01:20:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.305+0000 7f7bf0ab4640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bec1131c0 0x7f7bec1a4200 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00e9f0 tx=0x7f7bdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.306+0000 7f7be27fc640 1 -- 172.21.15.78:0/3914050810 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00cd80 con 0x7f7bec1131c0 2024-09-11T01:20:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.306+0000 7f7be27fc640 1 -- 172.21.15.78:0/3914050810 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bdc0108e0 con 0x7f7bec1131c0 2024-09-11T01:20:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.306+0000 7f7be27fc640 1 -- 172.21.15.78:0/3914050810 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc018980 con 0x7f7bec1131c0 2024-09-11T01:20:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.306+0000 7f7bf3519640 1 -- 172.21.15.78:0/3914050810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bec1aa060 con 0x7f7bec1131c0 2024-09-11T01:20:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.306+0000 7f7bf3519640 1 -- 172.21.15.78:0/3914050810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bec06ea40 con 0x7f7bec1131c0 2024-09-11T01:20:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.310+0000 7f7bf58ee740 1 Processor -- start 2024-09-11T01:20:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.310+0000 7f7bf58ee740 1 -- start start 2024-09-11T01:20:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.310+0000 7f7bf58ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465cee0 0x55f97465d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.310+0000 7f7bf58ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465d800 0x55f974669250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.310+0000 7f7bf58ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f974669790 0x55f97466bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.310+0000 7f7bf58ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9745d3e60 con 0x55f974669790 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.310+0000 7f7bf58ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f9745c3290 con 0x55f97465cee0 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.310+0000 7f7bf58ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9745c70a0 con 0x55f97465d800 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.310+0000 7f7be27fc640 1 -- 172.21.15.78:0/3914050810 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7bdc010430 con 0x7f7bec1131c0 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.310+0000 7f7bf0ab4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f974669790 0x55f97466bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.311+0000 7f7bf0ab4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f974669790 0x55f97466bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42854/0 (socket says 172.21.15.78:42854) 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.311+0000 7f7bf0ab4640 1 -- 172.21.15.78:0/2895733108 learned_addr learned my addr 172.21.15.78:0/2895733108 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.311+0000 7f7bf0ab4640 1 -- 172.21.15.78:0/2895733108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465d800 msgr2=0x55f974669250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.311+0000 7f7bf0ab4640 1 --2- 172.21.15.78:0/2895733108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465d800 0x55f974669250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.311+0000 7f7bf0ab4640 1 -- 172.21.15.78:0/2895733108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465cee0 msgr2=0x55f97465d2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.311+0000 7f7bf0ab4640 1 --2- 172.21.15.78:0/2895733108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465cee0 0x55f97465d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.311+0000 7f7bf0ab4640 1 -- 172.21.15.78:0/2895733108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f974321930 con 0x55f974669790 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.311+0000 7f7bf0ab4640 1 --2- 172.21.15.78:0/2895733108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f974669790 0x55f97466bb80 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0ee480 tx=0x7f7bdc0ee940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978eefe5f416717d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.312+0000 7f7bd77fe640 1 -- 172.21.15.78:0/2895733108 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc0e8480 con 0x55f974669790 2024-09-11T01:20:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.312+0000 7f7bd77fe640 1 -- 172.21.15.78:0/2895733108 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bdc0e6c10 con 0x55f974669790 2024-09-11T01:20:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.312+0000 7f7bd77fe640 1 -- 172.21.15.78:0/2895733108 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc0f4b10 con 0x55f974669790 2024-09-11T01:20:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.312+0000 7f7bf58ee740 1 -- 172.21.15.78:0/2895733108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f974669790 msgr2=0x55f97466bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.312+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/2895733108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f974669790 0x55f97466bb80 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0ee480 tx=0x7f7bdc0ee940 comp rx=0 tx=0).stop 2024-09-11T01:20:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.313+0000 7f7be27fc640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7bd80b5550 0x7f7bd80b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.313+0000 7f7be27fc640 1 -- 172.21.15.78:0/3914050810 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8493+0+0 (secure 0 0 0) 0x7f7bdc0e6330 con 0x7f7bec1131c0 2024-09-11T01:20:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.313+0000 7f7bf58ee740 1 -- 172.21.15.78:0/2895733108 shutdown_connections 2024-09-11T01:20:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.313+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/2895733108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f974669790 0x55f97466bb80 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.313+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/2895733108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465d800 0x55f974669250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.313+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/2895733108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465cee0 0x55f97465d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.313+0000 7f7bf58ee740 1 -- 172.21.15.78:0/2895733108 >> 172.21.15.78:0/2895733108 conn(0x55f97465ad10 msgr2=0x55f97465beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.314+0000 7f7bf1ab6640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7bd80b5550 0x7f7bd80b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.314+0000 7f7bf58ee740 1 -- 172.21.15.78:0/2895733108 shutdown_connections 2024-09-11T01:20:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.314+0000 7f7bf58ee740 1 -- 172.21.15.78:0/2895733108 wait complete. 2024-09-11T01:20:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.314+0000 7f7bf1ab6640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7bd80b5550 0x7f7bd80b7a10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7be40092f0 tx=0x7f7be4009220 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.315+0000 7f7bf58ee740 1 -- start start 2024-09-11T01:20:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.315+0000 7f7bf58ee740 1 Processor -- start 2024-09-11T01:20:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.316+0000 7f7bf58ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465cee0 0x55f974669220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.316+0000 7f7bf58ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 0x55f974666690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.316+0000 7f7bf58ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f974669790 0x55f974666e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.316+0000 7f7bf58ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f974424890 con 0x55f97465cee0 2024-09-11T01:20:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.316+0000 7f7bf0ab4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 0x55f974666690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.316+0000 7f7bf0ab4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 0x55f974666690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59904/0 (socket says 172.21.15.78:59904) 2024-09-11T01:20:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.316+0000 7f7bf0ab4640 1 -- 172.21.15.78:0/1609552751 learned_addr learned my addr 172.21.15.78:0/1609552751 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.316+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1609552751 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f97455dd40 con 0x55f97465d800 2024-09-11T01:20:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.316+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1609552751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f97466c490 con 0x55f974669790 2024-09-11T01:20:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.316+0000 7f7bf1ab6640 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f974669790 0x55f974666e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.317+0000 7f7bf12b5640 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465cee0 0x55f974669220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.317+0000 7f7bf58ee740 10 client.0 _ll_register_callbacks cb 0x55f9745c1c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.317+0000 7f7bf1ab6640 1 -- 172.21.15.78:0/1609552751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 msgr2=0x55f974666690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.317+0000 7f7bf1ab6640 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 0x55f974666690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.317+0000 7f7bf1ab6640 1 -- 172.21.15.78:0/1609552751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465cee0 msgr2=0x55f974669220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.317+0000 7f7bf1ab6640 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465cee0 0x55f974669220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.317+0000 7f7bf1ab6640 1 -- 172.21.15.78:0/1609552751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9746757c0 con 0x55f974669790 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.317+0000 7f7bf0ab4640 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 0x55f974666690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.317+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1609552751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bdc0e6c10 con 0x55f974669790 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.317+0000 7f7bf1ab6640 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f974669790 0x55f974666e20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7be4005fd0 tx=0x7f7be4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523e0eac8773de52 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.317+0000 7f7bd4ff9640 10 client.28147 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.318+0000 7f7bd4ff9640 1 -- 172.21.15.78:0/1609552751 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4022020 con 0x55f974669790 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.318+0000 7f7bd4ff9640 1 -- 172.21.15.78:0/1609552751 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8493+0+0 (secure 0 0 0) 0x7f7be40108d0 con 0x55f974669790 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.318+0000 7f7bd4ff9640 1 -- 172.21.15.78:0/1609552751 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f7be4010b30 con 0x55f974669790 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.318+0000 7f7bf58ee740 10 client.28147 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.318+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1609552751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=72}) -- 0x55f974678070 con 0x55f974669790 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.319+0000 7f7bd4ff9640 1 -- 172.21.15.78:0/1609552751 <== mon.1 v2:172.21.15.112:3300/0 4 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f7be400f040 con 0x55f974669790 2024-09-11T01:20:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.319+0000 7f7bf58ee740 10 client.28147 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:12.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.319+0000 7f7bf58ee740 10 client.28147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:20:12.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.319+0000 7f7bf58ee740 20 client.28147 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:12.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.319+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f974679490 0x55f97467b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.319+0000 7f7bf58ee740 4 client.28147 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:20:12.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.319+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1609552751 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9745d5db0 con 0x55f974679490 2024-09-11T01:20:12.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.320+0000 7f7bf12b5640 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f974679490 0x55f97467b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.321+0000 7f7bd4ff9640 10 client.28147 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:12.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.321+0000 7f7bf12b5640 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f974679490 0x55f97467b900 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dddd1106c1a0f517 server_cookie=d322b547cacfa7e0 in_seq=0 out_seq=0 2024-09-11T01:20:12.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bd4ff9640 1 -- 172.21.15.78:0/1609552751 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9745d5db0 con 0x55f974679490 2024-09-11T01:20:12.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bd4ff9640 10 client.28147 handle_command_reply: tid=0 2024-09-11T01:20:12.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bf58ee740 1 client.28147 shutdown 2024-09-11T01:20:12.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1609552751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f974669790 msgr2=0x55f974666e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:12.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f974669790 0x55f974666e20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7be4005fd0 tx=0x7f7be4004a40 comp rx=0 tx=0).stop 2024-09-11T01:20:12.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1609552751 shutdown_connections 2024-09-11T01:20:12.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f974669790 0x55f974666e20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 0x55f974666690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f974679490 0x55f97467b900 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/1609552751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465cee0 0x55f974669220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bd4ff9640 0 client.28147 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:12.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1609552751 >> 172.21.15.78:0/1609552751 conn(0x55f97465ad10 msgr2=0x55f974659f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:12.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1609552751 shutdown_connections 2024-09-11T01:20:12.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.323+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1609552751 wait complete. 2024-09-11T01:20:12.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.324+0000 7f7bf58ee740 20 client.28147 trim_cache size 0 max 16384 2024-09-11T01:20:12.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.355+0000 7f7bf58ee740 1 Processor -- start 2024-09-11T01:20:12.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.355+0000 7f7bf58ee740 1 -- start start 2024-09-11T01:20:12.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.356+0000 7f7bf58ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465ad10 0x55f974665dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.356+0000 7f7bf58ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465cee0 0x55f9746683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.356+0000 7f7bf58ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 0x55f974663880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.356+0000 7f7bf58ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec1bd4c0 con 0x55f97465ad10 2024-09-11T01:20:12.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.356+0000 7f7bf58ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec1bf0c0 con 0x55f97465d800 2024-09-11T01:20:12.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.356+0000 7f7bf58ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec1bdb90 con 0x55f97465cee0 2024-09-11T01:20:12.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.356+0000 7f7bf1ab6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 0x55f974663880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.356+0000 7f7bf1ab6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 0x55f974663880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59906/0 (socket says 172.21.15.78:59906) 2024-09-11T01:20:12.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.356+0000 7f7bf1ab6640 1 -- 172.21.15.78:0/1594380185 learned_addr learned my addr 172.21.15.78:0/1594380185 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:12.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.357+0000 7f7bf1ab6640 1 -- 172.21.15.78:0/1594380185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465cee0 msgr2=0x55f9746683b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:20:12.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.357+0000 7f7bf1ab6640 1 --2- 172.21.15.78:0/1594380185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465cee0 0x55f9746683b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.357+0000 7f7bf1ab6640 1 -- 172.21.15.78:0/1594380185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465ad10 msgr2=0x55f974665dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:12.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.357+0000 7f7bf1ab6640 1 --2- 172.21.15.78:0/1594380185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465ad10 0x55f974665dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.357+0000 7f7bf1ab6640 1 -- 172.21.15.78:0/1594380185 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f97465b790 con 0x55f97465d800 2024-09-11T01:20:12.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.358+0000 7f7bf1ab6640 1 --2- 172.21.15.78:0/1594380185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 0x55f974663880 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7be4010e00 tx=0x7f7be401e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c70e386350fa5249 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:12.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.358+0000 7f7bd6ffd640 1 -- 172.21.15.78:0/1594380185 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4035070 con 0x55f97465d800 2024-09-11T01:20:12.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.358+0000 7f7bd6ffd640 1 -- 172.21.15.78:0/1594380185 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bdc0e6c10 con 0x55f97465d800 2024-09-11T01:20:12.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.358+0000 7f7bd6ffd640 1 -- 172.21.15.78:0/1594380185 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4022070 con 0x55f97465d800 2024-09-11T01:20:12.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.359+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1594380185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 msgr2=0x55f974663880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:12.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.359+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/1594380185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 0x55f974663880 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7be4010e00 tx=0x7f7be401e000 comp rx=0 tx=0).stop 2024-09-11T01:20:12.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.363+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1594380185 shutdown_connections 2024-09-11T01:20:12.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.363+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/1594380185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465d800 0x55f974663880 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.363+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/1594380185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465cee0 0x55f9746683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.363+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/1594380185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465ad10 0x55f974665dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.363+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1594380185 >> 172.21.15.78:0/1594380185 conn(0x55f974662010 msgr2=0x55f974667ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:12.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.363+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1594380185 shutdown_connections 2024-09-11T01:20:12.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.364+0000 7f7bf58ee740 1 -- 172.21.15.78:0/1594380185 wait complete. 2024-09-11T01:20:12.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.364+0000 7f7bf58ee740 1 -- start start 2024-09-11T01:20:12.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.364+0000 7f7bf58ee740 1 Processor -- start 2024-09-11T01:20:12.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.365+0000 7f7bf58ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465ad10 0x55f974671ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.365+0000 7f7bf58ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465cee0 0x55f97466f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.365+0000 7f7bf58ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465d800 0x55f97466fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.365+0000 7f7bf58ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec1be9f0 con 0x55f97465d800 2024-09-11T01:20:12.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.365+0000 7f7bf58ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec1be260 con 0x55f97465ad10 2024-09-11T01:20:12.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.365+0000 7f7bf58ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec1bb4e0 con 0x55f97465cee0 2024-09-11T01:20:12.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.365+0000 7f7bf58ee740 10 client.0 _ll_register_callbacks cb 0x7f7be4009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:12.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.365+0000 7f7bf1ab6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465d800 0x55f97466fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.365+0000 7f7bf1ab6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465d800 0x55f97466fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42872/0 (socket says 172.21.15.78:42872) 2024-09-11T01:20:12.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.365+0000 7f7bf1ab6640 1 -- 172.21.15.78:0/3340721611 learned_addr learned my addr 172.21.15.78:0/3340721611 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.365+0000 7f7bf12b5640 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465ad10 0x55f974671ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.365+0000 7f7bf0ab4640 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465cee0 0x55f97466f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.366+0000 7f7bf1ab6640 1 -- 172.21.15.78:0/3340721611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465cee0 msgr2=0x55f97466f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.366+0000 7f7bf1ab6640 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465cee0 0x55f97466f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.366+0000 7f7bf1ab6640 1 -- 172.21.15.78:0/3340721611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465ad10 msgr2=0x55f974671ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.366+0000 7f7bf1ab6640 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465ad10 0x55f974671ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.366+0000 7f7bf1ab6640 1 -- 172.21.15.78:0/3340721611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9746706d0 con 0x55f97465d800 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.366+0000 7f7bf12b5640 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465ad10 0x55f974671ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.366+0000 7f7bf1ab6640 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465d800 0x55f97466fec0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7be4013040 tx=0x7f7be401eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2a667960b39441 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.366+0000 7f7bd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.367+0000 7f7bd4ff9640 1 -- 172.21.15.78:0/3340721611 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4041020 con 0x55f97465d800 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.367+0000 7f7bf58ee740 1 -- 172.21.15.78:0/3340721611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bdc0e6c10 con 0x55f97465d800 2024-09-11T01:20:12.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.367+0000 7f7bd4ff9640 1 -- 172.21.15.78:0/3340721611 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8493+0+0 (secure 0 0 0) 0x7f7be4031730 con 0x55f97465d800 2024-09-11T01:20:12.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.367+0000 7f7bd4ff9640 1 -- 172.21.15.78:0/3340721611 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f7be404c050 con 0x55f97465d800 2024-09-11T01:20:12.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.367+0000 7f7bf58ee740 10 client.19158 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:12.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.367+0000 7f7bf58ee740 1 -- 172.21.15.78:0/3340721611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55f9746e7480 con 0x55f97465d800 2024-09-11T01:20:12.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.367+0000 7f7bd4ff9640 1 -- 172.21.15.78:0/3340721611 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bdc0e6c10 con 0x55f97465d800 2024-09-11T01:20:12.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.368+0000 7f7bd4ff9640 1 -- 172.21.15.78:0/3340721611 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f7be403a4e0 con 0x55f97465d800 2024-09-11T01:20:12.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.368+0000 7f7bf58ee740 10 client.19158 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:12.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.368+0000 7f7bf58ee740 10 client.19158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:20:12.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.368+0000 7f7bf58ee740 20 client.19158 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:12.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.368+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f974669790 0x55f9746e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:12.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.368+0000 7f7bf58ee740 4 client.19158 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:20:12.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.368+0000 7f7bf58ee740 1 -- 172.21.15.78:0/3340721611 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f9745d1860 con 0x55f974669790 2024-09-11T01:20:12.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.368+0000 7f7bf12b5640 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f974669790 0x55f9746e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:12.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.369+0000 7f7bf12b5640 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f974669790 0x55f9746e9ae0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f0fad9aa05ff853 server_cookie=a3cf6af67db3c3f9 in_seq=0 out_seq=0 2024-09-11T01:20:12.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.369+0000 7f7bd4ff9640 10 client.19158 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:12.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bd4ff9640 1 -- 172.21.15.78:0/3340721611 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f9745d1860 con 0x55f974669790 2024-09-11T01:20:12.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bd4ff9640 10 client.19158 handle_command_reply: tid=0 2024-09-11T01:20:12.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bf58ee740 1 client.19158 shutdown 2024-09-11T01:20:12.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bf58ee740 1 -- 172.21.15.78:0/3340721611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465d800 msgr2=0x55f97466fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:12.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465d800 0x55f97466fec0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7be4013040 tx=0x7f7be401eb70 comp rx=0 tx=0).stop 2024-09-11T01:20:12.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bf58ee740 1 -- 172.21.15.78:0/3340721611 shutdown_connections 2024-09-11T01:20:12.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55f974669790 0x55f9746e9ae0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f97465d800 0x55f97466fec0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f97465cee0 0x55f97466f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bf58ee740 1 --2- 172.21.15.78:0/3340721611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f97465ad10 0x55f974671ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bf58ee740 1 -- 172.21.15.78:0/3340721611 >> 172.21.15.78:0/3340721611 conn(0x55f974662010 msgr2=0x55f974659f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:12.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bd4ff9640 0 client.19158 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:12.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bf58ee740 1 -- 172.21.15.78:0/3340721611 shutdown_connections 2024-09-11T01:20:12.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.371+0000 7f7bf58ee740 1 -- 172.21.15.78:0/3340721611 wait complete. 2024-09-11T01:20:12.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.372+0000 7f7bf58ee740 20 client.19158 trim_cache size 0 max 16384 2024-09-11T01:20:12.376 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:20:12.376 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:20:12.376 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "746add9a-5be3-4745-8000-04e5471bc813", 2024-09-11T01:20:12.377 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:20:12.377 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:20:12.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.373+0000 7f7bf3519640 1 -- 172.21.15.78:0/3914050810 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7bd80b5550 msgr2=0x7f7bd80b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:12.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.373+0000 7f7bf3519640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7bd80b5550 0x7f7bd80b7a10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7be40092f0 tx=0x7f7be4009220 comp rx=0 tx=0).stop 2024-09-11T01:20:12.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.373+0000 7f7bf3519640 1 -- 172.21.15.78:0/3914050810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bec1131c0 msgr2=0x7f7bec1a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:12.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.373+0000 7f7bf3519640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bec1131c0 0x7f7bec1a4200 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00e9f0 tx=0x7f7bdc00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.373+0000 7f7bf3519640 1 -- 172.21.15.78:0/3914050810 shutdown_connections 2024-09-11T01:20:12.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.373+0000 7f7bf3519640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f7bd80b5550 0x7f7bd80b7a10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.373+0000 7f7bf3519640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bec113b40 0x7f7bec1a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.373+0000 7f7bf3519640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bec1131c0 0x7f7bec1a4200 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.373+0000 7f7bf3519640 1 --2- 172.21.15.78:0/3914050810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bec10d800 0x7f7bec1a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:12.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.373+0000 7f7bf3519640 1 -- 172.21.15.78:0/3914050810 >> 172.21.15.78:0/3914050810 conn(0x7f7bec06dce0 msgr2=0x7f7bec10b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:12.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.374+0000 7f7bf3519640 1 -- 172.21.15.78:0/3914050810 shutdown_connections 2024-09-11T01:20:12.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:12.374+0000 7f7bf3519640 1 -- 172.21.15.78:0/3914050810 wait complete. 2024-09-11T01:20:12.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 311232 32.94 MB/sec execute 378 sec latency 291.700 ms 2024-09-11T01:20:13.165 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:12 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:20:13.165 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:12 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:13.165 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:12 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:20:13.165 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:12 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:20:13.165 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:12 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:20:13.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:12 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:20:13.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:12 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:13.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:12 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:20:13.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:12 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:20:13.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:12 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:20:13.192 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:20:13.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:12 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:20:13.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:12 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:13.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:12 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:20:13.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:12 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:20:13.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:12 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:20:13.474 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:20:13.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 311599 32.93 MB/sec execute 379 sec latency 363.138 ms 2024-09-11T01:20:14.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:13 smithi110 ceph-mon[34712]: pgmap v992: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 735 op/s 2024-09-11T01:20:14.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:13 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:20:14.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:13 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:20:14.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:13 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:13 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:13 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:13 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:13 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:13 smithi110 ceph-mon[34712]: osdmap e107: 12 total, 12 up, 12 in 2024-09-11T01:20:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:13 smithi112 ceph-mon[35384]: pgmap v992: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 735 op/s 2024-09-11T01:20:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:13 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:20:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:13 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:20:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:13 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:13 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:13 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:13 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:13 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:13 smithi112 ceph-mon[35384]: osdmap e107: 12 total, 12 up, 12 in 2024-09-11T01:20:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:13 smithi078 ceph-mon[31315]: pgmap v992: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 735 op/s 2024-09-11T01:20:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:13 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:20:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:13 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:20:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:13 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:13 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:13 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:13 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:13 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:20:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:13 smithi078 ceph-mon[31315]: osdmap e107: 12 total, 12 up, 12 in 2024-09-11T01:20:14.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.048+0000 7f551dd06640 1 -- 172.21.15.78:0/4026917068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100a8060 msgr2=0x7f55100b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:14.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.048+0000 7f551dd06640 1 --2- 172.21.15.78:0/4026917068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100a8060 0x7f55100b1a80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f551802eb50 tx=0x7f551808f200 comp rx=0 tx=0).stop 2024-09-11T01:20:14.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.049+0000 7f551dd06640 1 -- 172.21.15.78:0/4026917068 shutdown_connections 2024-09-11T01:20:14.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.049+0000 7f551dd06640 1 --2- 172.21.15.78:0/4026917068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100a8060 0x7f55100b1a80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.049+0000 7f551dd06640 1 --2- 172.21.15.78:0/4026917068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55100a76c0 0x7f55100a7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.049+0000 7f551dd06640 1 --2- 172.21.15.78:0/4026917068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55100a5290 0x7f55100a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.049+0000 7f551dd06640 1 -- 172.21.15.78:0/4026917068 >> 172.21.15.78:0/4026917068 conn(0x7f551001a050 msgr2=0x7f551001c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:14.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.049+0000 7f551dd06640 1 -- 172.21.15.78:0/4026917068 shutdown_connections 2024-09-11T01:20:14.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.049+0000 7f551dd06640 1 -- 172.21.15.78:0/4026917068 wait complete. 2024-09-11T01:20:14.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.050+0000 7f551dd06640 1 Processor -- start 2024-09-11T01:20:14.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.050+0000 7f551dd06640 1 -- start start 2024-09-11T01:20:14.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.050+0000 7f551dd06640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100a5290 0x7f55101436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.050+0000 7f551dd06640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55100a76c0 0x7f5510143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.050+0000 7f551dd06640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55100a8060 0x7f55100b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.050+0000 7f551dd06640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55100b67b0 con 0x7f55100a8060 2024-09-11T01:20:14.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.050+0000 7f551dd06640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55100b6630 con 0x7f55100a76c0 2024-09-11T01:20:14.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.050+0000 7f551dd06640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55100b6930 con 0x7f55100a5290 2024-09-11T01:20:14.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.052+0000 7f5517fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55100a76c0 0x7f5510143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.052+0000 7f5517fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55100a76c0 0x7f5510143c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:34276/0 (socket says 172.21.15.78:34276) 2024-09-11T01:20:14.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.052+0000 7f5517fff640 1 -- 172.21.15.78:0/2855254050 learned_addr learned my addr 172.21.15.78:0/2855254050 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:14.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.052+0000 7f551cd04640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100a5290 0x7f55101436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.053+0000 7f551d505640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55100a8060 0x7f55100b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.054+0000 7f551d505640 1 -- 172.21.15.78:0/2855254050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100a5290 msgr2=0x7f55101436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.054+0000 7f551d505640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100a5290 0x7f55101436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.054+0000 7f551d505640 1 -- 172.21.15.78:0/2855254050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55100a76c0 msgr2=0x7f5510143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.054+0000 7f551d505640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55100a76c0 0x7f5510143c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.054+0000 7f551d505640 1 -- 172.21.15.78:0/2855254050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55100b14f0 con 0x7f55100a8060 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.054+0000 7f551d505640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55100a8060 0x7f55100b0c30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f55180527e0 tx=0x7f5518096b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.056+0000 7f5515ffb640 1 -- 172.21.15.78:0/2855254050 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518096e10 con 0x7f55100a8060 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.056+0000 7f551dd06640 1 -- 172.21.15.78:0/2855254050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55100b1780 con 0x7f55100a8060 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.056+0000 7f5515ffb640 1 -- 172.21.15.78:0/2855254050 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f551806e8e0 con 0x7f55100a8060 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.056+0000 7f5515ffb640 1 -- 172.21.15.78:0/2855254050 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518097780 con 0x7f55100a8060 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.057+0000 7f551dd06640 1 -- 172.21.15.78:0/2855254050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551013e170 con 0x7f55100a8060 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.059+0000 7f552133d740 1 Processor -- start 2024-09-11T01:20:14.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.060+0000 7f5515ffb640 1 -- 172.21.15.78:0/2855254050 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f551806e430 con 0x7f55100a8060 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.060+0000 7f552133d740 1 -- start start 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.060+0000 7f552133d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756650 0x55a7c7756a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.060+0000 7f552133d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756f70 0x55a7c77629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.060+0000 7f552133d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7762f00 0x55a7c77652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.060+0000 7f552133d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7c76cce50 con 0x55a7c7756f70 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.060+0000 7f552133d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a7c76bc280 con 0x55a7c7756650 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.060+0000 7f552133d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a7c76c0090 con 0x55a7c7762f00 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f551cd04640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756f70 0x55a7c77629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f551cd04640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756f70 0x55a7c77629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45270/0 (socket says 172.21.15.78:45270) 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f551cd04640 1 -- 172.21.15.78:0/3534010828 learned_addr learned my addr 172.21.15.78:0/3534010828 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f5517fff640 1 --2- 172.21.15.78:0/3534010828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756650 0x55a7c7756a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f5517fff640 1 --2- 172.21.15.78:0/3534010828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7762f00 0x55a7c77652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f551cd04640 1 -- 172.21.15.78:0/3534010828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7762f00 msgr2=0x55a7c77652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:14.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f551cd04640 1 --2- 172.21.15.78:0/3534010828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7762f00 0x55a7c77652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f551cd04640 1 -- 172.21.15.78:0/3534010828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756650 msgr2=0x55a7c7756a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:14.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f551cd04640 1 --2- 172.21.15.78:0/3534010828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756650 0x55a7c7756a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f551cd04640 1 -- 172.21.15.78:0/3534010828 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7c741a650 con 0x55a7c7756f70 2024-09-11T01:20:14.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f551cd04640 1 --2- 172.21.15.78:0/3534010828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756f70 0x55a7c77629c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f550c009c20 tx=0x7f550c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a8ce4bf85f4f8c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:14.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f5502ffd640 1 -- 172.21.15.78:0/3534010828 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c012450 con 0x55a7c7756f70 2024-09-11T01:20:14.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f5502ffd640 1 -- 172.21.15.78:0/3534010828 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f550c012aa0 con 0x55a7c7756f70 2024-09-11T01:20:14.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.061+0000 7f5515ffb640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55080b55a0 0x7f55080b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f5517fff640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55080b55a0 0x7f55080b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f5515ffb640 1 -- 172.21.15.78:0/2855254050 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f5518169fa0 con 0x7f55100a8060 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f5502ffd640 1 -- 172.21.15.78:0/3534010828 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c00f9d0 con 0x55a7c7756f70 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f5517fff640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55080b55a0 0x7f55080b7a60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f55040058c0 tx=0x7f5504009800 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f552133d740 1 -- 172.21.15.78:0/3534010828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756f70 msgr2=0x55a7c77629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f552133d740 1 --2- 172.21.15.78:0/3534010828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756f70 0x55a7c77629c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f550c009c20 tx=0x7f550c0117b0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f552133d740 1 -- 172.21.15.78:0/3534010828 shutdown_connections 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f552133d740 1 --2- 172.21.15.78:0/3534010828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7762f00 0x55a7c77652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f552133d740 1 --2- 172.21.15.78:0/3534010828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756f70 0x55a7c77629c0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f552133d740 1 --2- 172.21.15.78:0/3534010828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756650 0x55a7c7756a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f552133d740 1 -- 172.21.15.78:0/3534010828 >> 172.21.15.78:0/3534010828 conn(0x55a7c7754480 msgr2=0x55a7c7755620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f552133d740 1 -- 172.21.15.78:0/3534010828 shutdown_connections 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.062+0000 7f552133d740 1 -- 172.21.15.78:0/3534010828 wait complete. 2024-09-11T01:20:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f552133d740 1 -- start start 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f552133d740 1 Processor -- start 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f552133d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7756650 0x55a7c7762630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f552133d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756f70 0x55a7c775fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f552133d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7762f00 0x55a7c775ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f552133d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7c751d700 con 0x55a7c7762f00 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f552133d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a7c7657110 con 0x55a7c7756f70 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f552133d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a7c7765bd0 con 0x55a7c7756650 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f552133d740 10 client.0 _ll_register_callbacks cb 0x55a7c7670990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f551d505640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7762f00 0x55a7c775ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f551cd04640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7756650 0x55a7c7762630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f551d505640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7762f00 0x55a7c775ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45278/0 (socket says 172.21.15.78:45278) 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f551d505640 1 -- 172.21.15.78:0/2142321797 learned_addr learned my addr 172.21.15.78:0/2142321797 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:14.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f551cd04640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7756650 0x55a7c7762630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42474/0 (socket says 172.21.15.78:42474) 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.063+0000 7f5517fff640 1 --2- 172.21.15.78:0/2142321797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756f70 0x55a7c775fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.064+0000 7f551cd04640 1 -- 172.21.15.78:0/2142321797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756f70 msgr2=0x55a7c775fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.064+0000 7f551cd04640 1 --2- 172.21.15.78:0/2142321797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756f70 0x55a7c775fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.064+0000 7f551cd04640 1 -- 172.21.15.78:0/2142321797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7762f00 msgr2=0x55a7c775ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.064+0000 7f551cd04640 1 --2- 172.21.15.78:0/2142321797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7762f00 0x55a7c775ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.064+0000 7f551cd04640 1 -- 172.21.15.78:0/2142321797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7c7684c40 con 0x55a7c7756650 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.064+0000 7f551cd04640 1 --2- 172.21.15.78:0/2142321797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7756650 0x55a7c7762630 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f550c000c00 tx=0x7f550c0056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12fb90553793ce57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.064+0000 7f54f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.064+0000 7f54f7fff640 1 -- 172.21.15.78:0/2142321797 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c028020 con 0x55a7c7756650 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.064+0000 7f552133d740 1 -- 172.21.15.78:0/2142321797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f550c012aa0 con 0x55a7c7756650 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.065+0000 7f54f7fff640 1 -- 172.21.15.78:0/2142321797 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f550c004250 con 0x55a7c7756650 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.065+0000 7f54f7fff640 1 -- 172.21.15.78:0/2142321797 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f550c004470 con 0x55a7c7756650 2024-09-11T01:20:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.065+0000 7f552133d740 10 client.28162 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.065+0000 7f552133d740 1 -- 172.21.15.78:0/2142321797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55a7c7771430 con 0x55a7c7756650 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.065+0000 7f54f7fff640 1 -- 172.21.15.78:0/2142321797 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f550c012aa0 con 0x55a7c7756650 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.065+0000 7f54f7fff640 1 -- 172.21.15.78:0/2142321797 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f550c0035f0 con 0x55a7c7756650 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.066+0000 7f552133d740 10 client.28162 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.066+0000 7f552133d740 10 client.28162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.066+0000 7f552133d740 20 client.28162 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.066+0000 7f552133d740 1 --2- 172.21.15.78:0/2142321797 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a7c7772840 0x55a7c7774c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.066+0000 7f552133d740 4 client.28162 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.066+0000 7f552133d740 1 -- 172.21.15.78:0/2142321797 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7c76ceda0 con 0x55a7c7772840 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.072+0000 7f5517fff640 1 --2- 172.21.15.78:0/2142321797 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a7c7772840 0x55a7c7774c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.072+0000 7f54f7fff640 10 client.28162 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.072+0000 7f5517fff640 1 --2- 172.21.15.78:0/2142321797 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a7c7772840 0x55a7c7774c30 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6d7a7bf3544039 server_cookie=9b8a1c4da26a83a3 in_seq=0 out_seq=0 2024-09-11T01:20:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.073+0000 7f54f7fff640 1 -- 172.21.15.78:0/2142321797 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a7c76ceda0 con 0x55a7c7772840 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.073+0000 7f54f7fff640 10 client.28162 handle_command_reply: tid=0 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.073+0000 7f552133d740 1 client.28162 shutdown 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.073+0000 7f552133d740 1 -- 172.21.15.78:0/2142321797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7756650 msgr2=0x55a7c7762630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.073+0000 7f552133d740 1 --2- 172.21.15.78:0/2142321797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7756650 0x55a7c7762630 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f550c000c00 tx=0x7f550c0056d0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.073+0000 7f552133d740 1 -- 172.21.15.78:0/2142321797 shutdown_connections 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.073+0000 7f552133d740 1 --2- 172.21.15.78:0/2142321797 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a7c7772840 0x55a7c7774c30 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.073+0000 7f552133d740 1 --2- 172.21.15.78:0/2142321797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7762f00 0x55a7c775ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.073+0000 7f552133d740 1 --2- 172.21.15.78:0/2142321797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756f70 0x55a7c775fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.073+0000 7f552133d740 1 --2- 172.21.15.78:0/2142321797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7756650 0x55a7c7762630 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.073+0000 7f552133d740 1 -- 172.21.15.78:0/2142321797 >> 172.21.15.78:0/2142321797 conn(0x55a7c7754480 msgr2=0x55a7c7759ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.073+0000 7f54f7fff640 0 client.28162 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.074+0000 7f552133d740 1 -- 172.21.15.78:0/2142321797 shutdown_connections 2024-09-11T01:20:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.074+0000 7f552133d740 1 -- 172.21.15.78:0/2142321797 wait complete. 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.074+0000 7f552133d740 20 client.28162 trim_cache size 0 max 16384 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f552133d740 1 Processor -- start 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f552133d740 1 -- start start 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f552133d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 0x55a7c775ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f552133d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756f70 0x55a7c7759e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f552133d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7762f00 0x55a7c775a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f552133d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5510160de0 con 0x55a7c7756650 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f552133d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55101629e0 con 0x55a7c7756f70 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f552133d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55101614b0 con 0x55a7c7762f00 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f551cd04640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 0x55a7c775ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f551cd04640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 0x55a7c775ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45294/0 (socket says 172.21.15.78:45294) 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f551cd04640 1 -- 172.21.15.78:0/2338938381 learned_addr learned my addr 172.21.15.78:0/2338938381 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f5517fff640 1 --2- 172.21.15.78:0/2338938381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756f70 0x55a7c7759e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f551d505640 1 --2- 172.21.15.78:0/2338938381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7762f00 0x55a7c775a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f551cd04640 1 -- 172.21.15.78:0/2338938381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7762f00 msgr2=0x55a7c775a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f551cd04640 1 --2- 172.21.15.78:0/2338938381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7762f00 0x55a7c775a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f551cd04640 1 -- 172.21.15.78:0/2338938381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756f70 msgr2=0x55a7c7759e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f551cd04640 1 --2- 172.21.15.78:0/2338938381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756f70 0x55a7c7759e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f551cd04640 1 -- 172.21.15.78:0/2338938381 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7c77542b0 con 0x55a7c7756650 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f5517fff640 1 --2- 172.21.15.78:0/2338938381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756f70 0x55a7c7759e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f551d505640 1 --2- 172.21.15.78:0/2338938381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7762f00 0x55a7c775a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f551cd04640 1 --2- 172.21.15.78:0/2338938381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 0x55a7c775ef90 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f550c00f480 tx=0x7f550c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7d37d3c5c66cca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.092+0000 7f55027fc640 1 -- 172.21.15.78:0/2338938381 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c03b070 con 0x55a7c7756650 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.093+0000 7f55027fc640 1 -- 172.21.15.78:0/2338938381 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f550c009aa0 con 0x55a7c7756650 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.093+0000 7f55027fc640 1 -- 172.21.15.78:0/2338938381 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c028070 con 0x55a7c7756650 2024-09-11T01:20:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.093+0000 7f552133d740 1 -- 172.21.15.78:0/2338938381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 msgr2=0x55a7c775ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.093+0000 7f552133d740 1 --2- 172.21.15.78:0/2338938381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 0x55a7c775ef90 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f550c00f480 tx=0x7f550c0117b0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.093+0000 7f552133d740 1 -- 172.21.15.78:0/2338938381 shutdown_connections 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.093+0000 7f552133d740 1 --2- 172.21.15.78:0/2338938381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7762f00 0x55a7c775a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.093+0000 7f552133d740 1 --2- 172.21.15.78:0/2338938381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7756f70 0x55a7c7759e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.093+0000 7f552133d740 1 --2- 172.21.15.78:0/2338938381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 0x55a7c775ef90 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.093+0000 7f552133d740 1 -- 172.21.15.78:0/2338938381 >> 172.21.15.78:0/2338938381 conn(0x55a7c7754480 msgr2=0x55a7c7759740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.093+0000 7f552133d740 1 -- 172.21.15.78:0/2338938381 shutdown_connections 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.093+0000 7f552133d740 1 -- 172.21.15.78:0/2338938381 wait complete. 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f552133d740 1 -- start start 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f552133d740 1 Processor -- start 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f552133d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 0x55a7c7766ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f552133d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7756f70 0x55a7c7767410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f552133d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7762f00 0x55a7c7767950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f552133d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5510162310 con 0x55a7c7756650 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f552133d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5510161b80 con 0x55a7c7762f00 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f552133d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f551015ede0 con 0x55a7c7756f70 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f552133d740 10 client.0 _ll_register_callbacks cb 0x7f5504009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f551cd04640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 0x55a7c7766ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f551cd04640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 0x55a7c7766ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45304/0 (socket says 172.21.15.78:45304) 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f551cd04640 1 -- 172.21.15.78:0/3132370640 learned_addr learned my addr 172.21.15.78:0/3132370640 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f551d505640 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7762f00 0x55a7c7767950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f5517fff640 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7756f70 0x55a7c7767410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f551cd04640 1 -- 172.21.15.78:0/3132370640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7756f70 msgr2=0x55a7c7767410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f551cd04640 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7756f70 0x55a7c7767410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f551cd04640 1 -- 172.21.15.78:0/3132370640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7762f00 msgr2=0x55a7c7767950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f551cd04640 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7762f00 0x55a7c7767950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.094+0000 7f551cd04640 1 -- 172.21.15.78:0/3132370640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7c7754f00 con 0x55a7c7756650 2024-09-11T01:20:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f551cd04640 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 0x55a7c7766ed0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f550c003a60 tx=0x7f550c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37eff44df59e4d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f54f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f54f7fff640 1 -- 172.21.15.78:0/3132370640 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c028070 con 0x55a7c7756650 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f54f7fff640 1 -- 172.21.15.78:0/3132370640 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f550c012450 con 0x55a7c7756650 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f552133d740 1 -- 172.21.15.78:0/3132370640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f550c009aa0 con 0x55a7c7756650 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f54f7fff640 1 -- 172.21.15.78:0/3132370640 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f550c03b040 con 0x55a7c7756650 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f552133d740 10 client.19176 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f552133d740 1 -- 172.21.15.78:0/3132370640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55a7c77df650 con 0x55a7c7756650 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f54f7fff640 1 -- 172.21.15.78:0/3132370640 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f550c009aa0 con 0x55a7c7756650 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f54f7fff640 1 -- 172.21.15.78:0/3132370640 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f550c00fca0 con 0x55a7c7756650 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f552133d740 10 client.19176 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f552133d740 10 client.19176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f552133d740 20 client.19176 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f552133d740 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a7c77dfab0 0x55a7c77e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f552133d740 4 client.19176 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.095+0000 7f552133d740 1 -- 172.21.15.78:0/3132370640 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a7c76ca850 con 0x55a7c77dfab0 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.096+0000 7f5517fff640 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a7c77dfab0 0x55a7c77e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.096+0000 7f54f7fff640 10 client.19176 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.096+0000 7f5517fff640 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a7c77dfab0 0x55a7c77e1ed0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e340caa1586eb0c7 server_cookie=66e209cde85cf153 in_seq=0 out_seq=0 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f54f7fff640 1 -- 172.21.15.78:0/3132370640 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7670 (crc 0 0 0) 0x55a7c76ca850 con 0x55a7c77dfab0 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f54f7fff640 10 client.19176 handle_command_reply: tid=0 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f552133d740 1 client.19176 shutdown 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f552133d740 1 -- 172.21.15.78:0/3132370640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 msgr2=0x55a7c7766ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f552133d740 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 0x55a7c7766ed0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f550c003a60 tx=0x7f550c005760 comp rx=0 tx=0).stop 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f552133d740 1 -- 172.21.15.78:0/3132370640 shutdown_connections 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f552133d740 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7c7762f00 0x55a7c7767950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f552133d740 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7c7756f70 0x55a7c7767410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f552133d740 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55a7c77dfab0 0x55a7c77e1ed0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f54f7fff640 0 client.19176 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:14.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f552133d740 1 --2- 172.21.15.78:0/3132370640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c7756650 0x55a7c7766ed0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f552133d740 1 -- 172.21.15.78:0/3132370640 >> 172.21.15.78:0/3132370640 conn(0x55a7c7754480 msgr2=0x55a7c7759ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:14.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f552133d740 1 -- 172.21.15.78:0/3132370640 shutdown_connections 2024-09-11T01:20:14.545 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:20:14.545 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:20:14.545 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:20:14.545 INFO:teuthology.orchestra.run.smithi078.stdout: "746add9a-5be3-4745-8000-04e5471bc813": { 2024-09-11T01:20:14.545 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.545 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "746add9a-5be3-4745-8000-04e5471bc813", 2024-09-11T01:20:14.546 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:20:14.546 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:20:14.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.546 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:20:14.546 INFO:teuthology.orchestra.run.smithi078.stdout: "746add9a-5be3-4745-8000-04e5471bc813": { 2024-09-11T01:20:14.546 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:20:12.369593+0000", 2024-09-11T01:20:14.546 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.546 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.546 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.546 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.547 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.547 INFO:teuthology.orchestra.run.smithi078.stdout: "b41a2b77-b2b1-4f1b-8588-4f32a78b721f": { 2024-09-11T01:20:14.547 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:19:13.257610+0000", 2024-09-11T01:20:14.547 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.547 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.547 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.548 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.548 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.548 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:20:14.548 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:18:17.654570+0000", 2024-09-11T01:20:14.548 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.549 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.549 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.549 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.549 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.549 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:20:14.549 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:20:14.550 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.550 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.550 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.550 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.550 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.550 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:20:14.551 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:20:14.551 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.551 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.551 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.551 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.551 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.552 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:20:14.552 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:20:14.552 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.552 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.552 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.552 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.552 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.553 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:20:14.553 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:20:14.553 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.553 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.553 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.553 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.554 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.554 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:20:14.554 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:20:14.554 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.554 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.554 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.554 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.555 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.555 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:20:14.555 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:20:14.555 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.555 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.555 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.556 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.556 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.556 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:20:14.556 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:20:14.556 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.557 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.557 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.557 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.557 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.557 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:20:14.557 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:20:14.558 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.558 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.558 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.558 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.558 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.558 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:20:14.559 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:20:14.559 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.559 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.559 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.559 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.559 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.560 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:20:14.560 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:20:14.560 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.560 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.560 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.560 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.560 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.561 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:20:14.561 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:20:14.561 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.561 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.561 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.562 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.562 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.562 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:20:14.562 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:20:14.562 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.562 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.563 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.563 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.563 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.563 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:20:14.563 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:20:14.563 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.564 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.564 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.564 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.564 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.564 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:20:14.564 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:20:14.565 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.565 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.565 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.565 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.565 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.565 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:20:14.566 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:20:14.566 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.566 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.566 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.566 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.566 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.567 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:20:14.567 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:20:14.567 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.567 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.567 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.567 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.568 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.568 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:20:14.568 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:20:14.568 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.568 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.568 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.569 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.569 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.569 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:20:14.569 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:20:14.569 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.569 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.570 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.570 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.570 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.570 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:20:14.570 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:20:14.570 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.570 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.571 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.571 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.571 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.571 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:20:14.571 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:20:14.571 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.572 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.572 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.572 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.572 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.572 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:20:14.573 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:20:14.573 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.573 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.573 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.573 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.573 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.574 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:20:14.574 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:20:14.574 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.574 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.574 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.574 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.574 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:14.575 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:20:14.575 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:20:14.575 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:14.575 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:14.576 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:14.576 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:14.576 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:14.576 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:20:14.576 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:20:14.576 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:20:14.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f552133d740 1 -- 172.21.15.78:0/3132370640 wait complete. 2024-09-11T01:20:14.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.097+0000 7f552133d740 20 client.19176 trim_cache size 0 max 16384 2024-09-11T01:20:14.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.098+0000 7f551dd06640 1 -- 172.21.15.78:0/2855254050 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55080b55a0 msgr2=0x7f55080b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:14.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.098+0000 7f551dd06640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55080b55a0 0x7f55080b7a60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f55040058c0 tx=0x7f5504009800 comp rx=0 tx=0).stop 2024-09-11T01:20:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.098+0000 7f551dd06640 1 -- 172.21.15.78:0/2855254050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55100a8060 msgr2=0x7f55100b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.098+0000 7f551dd06640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55100a8060 0x7f55100b0c30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f55180527e0 tx=0x7f5518096b10 comp rx=0 tx=0).stop 2024-09-11T01:20:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.098+0000 7f551dd06640 1 -- 172.21.15.78:0/2855254050 shutdown_connections 2024-09-11T01:20:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.098+0000 7f551dd06640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f55080b55a0 0x7f55080b7a60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.098+0000 7f551dd06640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55100a8060 0x7f55100b0c30 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.098+0000 7f551dd06640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55100a76c0 0x7f5510143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.098+0000 7f551dd06640 1 --2- 172.21.15.78:0/2855254050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100a5290 0x7f55101436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.098+0000 7f551dd06640 1 -- 172.21.15.78:0/2855254050 >> 172.21.15.78:0/2855254050 conn(0x7f551001a050 msgr2=0x7f55100a86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.098+0000 7f551dd06640 1 -- 172.21.15.78:0/2855254050 shutdown_connections 2024-09-11T01:20:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:14.098+0000 7f551dd06640 1 -- 172.21.15.78:0/2855254050 wait complete. 2024-09-11T01:20:14.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 311954 32.88 MB/sec execute 380 sec latency 431.214 ms 2024-09-11T01:20:15.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 312336 32.86 MB/sec execute 381 sec latency 354.989 ms 2024-09-11T01:20:16.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:15 smithi110 ceph-mon[34712]: pgmap v994: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 717 op/s 2024-09-11T01:20:16.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:15 smithi112 ceph-mon[35384]: pgmap v994: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 717 op/s 2024-09-11T01:20:16.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:15 smithi078 ceph-mon[31315]: pgmap v994: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 717 op/s 2024-09-11T01:20:16.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 312922 32.84 MB/sec execute 382 sec latency 126.572 ms 2024-09-11T01:20:17.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 313517 32.84 MB/sec execute 383 sec latency 85.462 ms 2024-09-11T01:20:18.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:17 smithi110 ceph-mon[34712]: pgmap v995: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 660 op/s 2024-09-11T01:20:18.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:17 smithi112 ceph-mon[35384]: pgmap v995: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 660 op/s 2024-09-11T01:20:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:17 smithi078 ceph-mon[31315]: pgmap v995: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 660 op/s 2024-09-11T01:20:18.377 INFO:tasks.cephfs.filesystem:scrub status for tag:746add9a-5be3-4745-8000-04e5471bc813 - {'path': '/', 'tag': '746add9a-5be3-4745-8000-04e5471bc813', 'options': 'recursive,force'} 2024-09-11T01:20:18.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 314098 32.83 MB/sec execute 384 sec latency 141.453 ms 2024-09-11T01:20:19.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 314786 32.84 MB/sec execute 385 sec latency 85.040 ms 2024-09-11T01:20:20.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:19 smithi110 ceph-mon[34712]: pgmap v996: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 702 op/s 2024-09-11T01:20:20.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:19 smithi112 ceph-mon[35384]: pgmap v996: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 702 op/s 2024-09-11T01:20:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:19 smithi078 ceph-mon[31315]: pgmap v996: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 702 op/s 2024-09-11T01:20:20.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 315502 32.87 MB/sec execute 386 sec latency 80.366 ms 2024-09-11T01:20:21.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 316117 32.87 MB/sec execute 387 sec latency 81.721 ms 2024-09-11T01:20:22.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:21 smithi112 ceph-mon[35384]: pgmap v997: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 684 op/s 2024-09-11T01:20:22.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:21 smithi078 ceph-mon[31315]: pgmap v997: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 684 op/s 2024-09-11T01:20:22.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:21 smithi110 ceph-mon[34712]: pgmap v997: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 684 op/s 2024-09-11T01:20:22.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 316768 32.87 MB/sec execute 388 sec latency 74.845 ms 2024-09-11T01:20:23.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 317490 32.89 MB/sec execute 389 sec latency 68.043 ms 2024-09-11T01:20:24.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:23 smithi112 ceph-mon[35384]: pgmap v998: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 673 op/s 2024-09-11T01:20:24.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:23 smithi078 ceph-mon[31315]: pgmap v998: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 673 op/s 2024-09-11T01:20:24.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:23 smithi110 ceph-mon[34712]: pgmap v998: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 673 op/s 2024-09-11T01:20:24.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 318184 32.89 MB/sec execute 390 sec latency 89.902 ms 2024-09-11T01:20:25.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 318919 32.93 MB/sec execute 391 sec latency 63.235 ms 2024-09-11T01:20:26.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:25 smithi112 ceph-mon[35384]: pgmap v999: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 755 op/s 2024-09-11T01:20:26.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:25 smithi078 ceph-mon[31315]: pgmap v999: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 755 op/s 2024-09-11T01:20:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:25 smithi110 ceph-mon[34712]: pgmap v999: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 755 op/s 2024-09-11T01:20:26.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 319428 32.89 MB/sec execute 392 sec latency 196.960 ms 2024-09-11T01:20:27.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 319641 32.84 MB/sec execute 393 sec latency 621.902 ms 2024-09-11T01:20:28.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:27 smithi112 ceph-mon[35384]: pgmap v1000: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 647 op/s 2024-09-11T01:20:28.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:27 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:20:28.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:27 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:20:28.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:27 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:20:28.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:27 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:20:28.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:27 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:20:28.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:27 smithi078 ceph-mon[31315]: pgmap v1000: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 647 op/s 2024-09-11T01:20:28.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:27 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:20:28.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:27 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:20:28.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:27 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:20:28.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:27 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:20:28.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:27 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:20:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:27 smithi110 ceph-mon[34712]: pgmap v1000: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 647 op/s 2024-09-11T01:20:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:27 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:20:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:27 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:20:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:27 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:20:28.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:27 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:20:28.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:27 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:20:28.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 320273 32.84 MB/sec execute 394 sec latency 91.784 ms 2024-09-11T01:20:29.800 INFO:tasks.workunit.client.0.smithi078.stdout: 10 321031 32.87 MB/sec execute 395 sec latency 96.885 ms 2024-09-11T01:20:30.223 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:29 smithi112 ceph-mon[35384]: pgmap v1001: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 577 op/s 2024-09-11T01:20:30.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:29 smithi078 ceph-mon[31315]: pgmap v1001: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 577 op/s 2024-09-11T01:20:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:29 smithi110 ceph-mon[34712]: pgmap v1001: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 577 op/s 2024-09-11T01:20:30.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 321516 32.86 MB/sec execute 396 sec latency 198.268 ms 2024-09-11T01:20:31.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 322158 32.86 MB/sec execute 397 sec latency 161.154 ms 2024-09-11T01:20:32.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:31 smithi112 ceph-mon[35384]: pgmap v1002: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 718 op/s 2024-09-11T01:20:32.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:31 smithi078 ceph-mon[31315]: pgmap v1002: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 718 op/s 2024-09-11T01:20:32.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:31 smithi110 ceph-mon[34712]: pgmap v1002: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 718 op/s 2024-09-11T01:20:32.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 322742 32.86 MB/sec execute 398 sec latency 205.842 ms 2024-09-11T01:20:33.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 323429 32.84 MB/sec execute 399 sec latency 100.981 ms 2024-09-11T01:20:34.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:33 smithi112 ceph-mon[35384]: pgmap v1003: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 744 op/s 2024-09-11T01:20:34.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:33 smithi078 ceph-mon[31315]: pgmap v1003: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 744 op/s 2024-09-11T01:20:34.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:33 smithi110 ceph-mon[34712]: pgmap v1003: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 744 op/s 2024-09-11T01:20:34.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 324183 32.89 MB/sec execute 400 sec latency 167.611 ms 2024-09-11T01:20:35.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 324608 32.86 MB/sec execute 401 sec latency 661.940 ms 2024-09-11T01:20:36.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:35 smithi112 ceph-mon[35384]: pgmap v1004: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 718 op/s 2024-09-11T01:20:36.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:35 smithi078 ceph-mon[31315]: pgmap v1004: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 718 op/s 2024-09-11T01:20:36.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:35 smithi110 ceph-mon[34712]: pgmap v1004: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 718 op/s 2024-09-11T01:20:36.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 325308 32.87 MB/sec execute 402 sec latency 82.403 ms 2024-09-11T01:20:37.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 325755 32.86 MB/sec execute 403 sec latency 272.181 ms 2024-09-11T01:20:38.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:37 smithi112 ceph-mon[35384]: pgmap v1005: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 722 op/s 2024-09-11T01:20:38.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:37 smithi078 ceph-mon[31315]: pgmap v1005: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 722 op/s 2024-09-11T01:20:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:37 smithi110 ceph-mon[34712]: pgmap v1005: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 722 op/s 2024-09-11T01:20:38.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 326188 32.83 MB/sec execute 404 sec latency 457.544 ms 2024-09-11T01:20:39.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 326735 32.82 MB/sec execute 405 sec latency 483.164 ms 2024-09-11T01:20:40.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:39 smithi112 ceph-mon[35384]: pgmap v1006: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 704 op/s 2024-09-11T01:20:40.241 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:39 smithi110 ceph-mon[34712]: pgmap v1006: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 704 op/s 2024-09-11T01:20:40.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:39 smithi078 ceph-mon[31315]: pgmap v1006: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 704 op/s 2024-09-11T01:20:40.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 327489 32.83 MB/sec execute 406 sec latency 83.687 ms 2024-09-11T01:20:41.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 328354 32.87 MB/sec execute 407 sec latency 67.376 ms 2024-09-11T01:20:42.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:41 smithi112 ceph-mon[35384]: pgmap v1007: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 807 op/s 2024-09-11T01:20:42.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:41 smithi078 ceph-mon[31315]: pgmap v1007: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 807 op/s 2024-09-11T01:20:42.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:41 smithi110 ceph-mon[34712]: pgmap v1007: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 807 op/s 2024-09-11T01:20:42.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 328977 32.88 MB/sec execute 408 sec latency 113.492 ms 2024-09-11T01:20:43.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 329523 32.87 MB/sec execute 409 sec latency 182.376 ms 2024-09-11T01:20:44.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:43 smithi112 ceph-mon[35384]: pgmap v1008: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 721 op/s 2024-09-11T01:20:44.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:43 smithi078 ceph-mon[31315]: pgmap v1008: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 721 op/s 2024-09-11T01:20:44.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:43 smithi110 ceph-mon[34712]: pgmap v1008: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 721 op/s 2024-09-11T01:20:44.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 330061 32.86 MB/sec execute 410 sec latency 104.404 ms 2024-09-11T01:20:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:20:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:44 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:20:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:44 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:20:45.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:44 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:45.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 330675 32.84 MB/sec execute 411 sec latency 73.832 ms 2024-09-11T01:20:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:45 smithi078 ceph-mon[31315]: pgmap v1009: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 618 op/s 2024-09-11T01:20:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:20:46.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:20:46.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:45 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:45 smithi110 ceph-mon[34712]: pgmap v1009: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 618 op/s 2024-09-11T01:20:46.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:20:46.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:20:46.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:45 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:45 smithi112 ceph-mon[35384]: pgmap v1009: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 618 op/s 2024-09-11T01:20:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:20:46.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:20:46.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:45 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:20:46.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 331309 32.85 MB/sec execute 412 sec latency 152.754 ms 2024-09-11T01:20:47.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 332135 32.89 MB/sec execute 413 sec latency 71.546 ms 2024-09-11T01:20:48.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:47 smithi078 ceph-mon[31315]: pgmap v1010: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 648 op/s 2024-09-11T01:20:48.378 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:20:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:47 smithi110 ceph-mon[34712]: pgmap v1010: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 648 op/s 2024-09-11T01:20:48.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:47 smithi112 ceph-mon[35384]: pgmap v1010: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 648 op/s 2024-09-11T01:20:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:20:48.810 INFO:tasks.workunit.client.0.smithi078.stdout: 10 332817 32.89 MB/sec execute 414 sec latency 70.745 ms 2024-09-11T01:20:49.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 333490 32.92 MB/sec execute 415 sec latency 223.419 ms 2024-09-11T01:20:50.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:49 smithi078 ceph-mon[31315]: pgmap v1011: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 609 op/s 2024-09-11T01:20:50.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:49 smithi110 ceph-mon[34712]: pgmap v1011: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 609 op/s 2024-09-11T01:20:50.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:49 smithi112 ceph-mon[35384]: pgmap v1011: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 609 op/s 2024-09-11T01:20:50.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 334052 32.90 MB/sec execute 416 sec latency 246.237 ms 2024-09-11T01:20:51.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 334593 32.88 MB/sec execute 417 sec latency 113.693 ms 2024-09-11T01:20:52.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:51 smithi078 ceph-mon[31315]: pgmap v1012: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 687 op/s 2024-09-11T01:20:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:51 smithi110 ceph-mon[34712]: pgmap v1012: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 687 op/s 2024-09-11T01:20:52.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:51 smithi112 ceph-mon[35384]: pgmap v1012: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 687 op/s 2024-09-11T01:20:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.852+0000 7f408d66d640 1 -- 172.21.15.78:0/2569076153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4088104530 msgr2=0x7f408810cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.852+0000 7f408d66d640 1 --2- 172.21.15.78:0/2569076153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4088104530 0x7f408810cb10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4070009900 tx=0x7f407002a3f0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.853+0000 7f408d66d640 1 -- 172.21.15.78:0/2569076153 shutdown_connections 2024-09-11T01:20:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.853+0000 7f408d66d640 1 --2- 172.21.15.78:0/2569076153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f408810d0e0 0x7f408810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.853+0000 7f408d66d640 1 --2- 172.21.15.78:0/2569076153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4088104530 0x7f408810cb10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.853+0000 7f408d66d640 1 --2- 172.21.15.78:0/2569076153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4088103b30 0x7f4088103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.853+0000 7f408d66d640 1 -- 172.21.15.78:0/2569076153 >> 172.21.15.78:0/2569076153 conn(0x7f40880fd700 msgr2=0x7f40880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.853+0000 7f408d66d640 1 -- 172.21.15.78:0/2569076153 shutdown_connections 2024-09-11T01:20:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.853+0000 7f408d66d640 1 -- 172.21.15.78:0/2569076153 wait complete. 2024-09-11T01:20:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f408d66d640 1 Processor -- start 2024-09-11T01:20:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f408d66d640 1 -- start start 2024-09-11T01:20:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f408d66d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4088103b30 0x7f408810c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f408d66d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4088104530 0x7f4088109c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f408d66d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f408810d0e0 0x7f408810a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f408d66d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4088103460 con 0x7f408810d0e0 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f408d66d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f40881032e0 con 0x7f4088103b30 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f408d66d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40881035e0 con 0x7f4088104530 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f4086ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4088103b30 0x7f408810c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f4086ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4088103b30 0x7f408810c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:57234/0 (socket says 172.21.15.78:57234) 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f4086ffd640 1 -- 172.21.15.78:0/2189322937 learned_addr learned my addr 172.21.15.78:0/2189322937 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f407ffff640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4088104530 0x7f4088109c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.854+0000 7f40877fe640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f408810d0e0 0x7f408810a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.855+0000 7f4086ffd640 1 -- 172.21.15.78:0/2189322937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4088104530 msgr2=0x7f4088109c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.855+0000 7f4086ffd640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4088104530 0x7f4088109c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.855+0000 7f4086ffd640 1 -- 172.21.15.78:0/2189322937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f408810d0e0 msgr2=0x7f408810a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.855+0000 7f4086ffd640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f408810d0e0 0x7f408810a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.855+0000 7f4086ffd640 1 -- 172.21.15.78:0/2189322937 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40881abb70 con 0x7f4088103b30 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.855+0000 7f4086ffd640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4088103b30 0x7f408810c9d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f407800e9f0 tx=0x7f407800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.855+0000 7f4084ff9640 1 -- 172.21.15.78:0/2189322937 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407800cd80 con 0x7f4088103b30 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.855+0000 7f4084ff9640 1 -- 172.21.15.78:0/2189322937 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40780054d0 con 0x7f4088103b30 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.855+0000 7f4084ff9640 1 -- 172.21.15.78:0/2189322937 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078010690 con 0x7f4088103b30 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.856+0000 7f408d66d640 1 -- 172.21.15.78:0/2189322937 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40881abda0 con 0x7f4088103b30 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.856+0000 7f408d66d640 1 -- 172.21.15.78:0/2189322937 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40881ac200 con 0x7f4088103b30 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.860+0000 7f408fa42740 1 Processor -- start 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.860+0000 7f408fa42740 1 -- start start 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.861+0000 7f408fa42740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fd5480 0x5584b4fd5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.861+0000 7f408fa42740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 0x5584b4fe17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.861+0000 7f408fa42740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fe1d30 0x5584b4fe4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.861+0000 7f408fa42740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584b4f4be50 con 0x5584b4fe1d30 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.861+0000 7f408fa42740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5584b4f3b280 con 0x5584b4fd5da0 2024-09-11T01:20:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.861+0000 7f408fa42740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5584b4f3f090 con 0x5584b4fd5480 2024-09-11T01:20:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.861+0000 7f40877fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fd5480 0x5584b4fd5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.861+0000 7f40877fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fd5480 0x5584b4fd5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:58186/0 (socket says 172.21.15.78:58186) 2024-09-11T01:20:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.861+0000 7f40877fe640 1 -- 172.21.15.78:0/1949497690 learned_addr learned my addr 172.21.15.78:0/1949497690 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.861+0000 7f407ffff640 1 --2- 172.21.15.78:0/1949497690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fe1d30 0x5584b4fe4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.862+0000 7f4084ff9640 1 -- 172.21.15.78:0/2189322937 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4078004e10 con 0x7f4088103b30 2024-09-11T01:20:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.862+0000 7f407ffff640 1 -- 172.21.15.78:0/1949497690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fd5480 msgr2=0x5584b4fd5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.862+0000 7f407ffff640 1 --2- 172.21.15.78:0/1949497690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fd5480 0x5584b4fd5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.862+0000 7f407ffff640 1 -- 172.21.15.78:0/1949497690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 msgr2=0x5584b4fe17f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:20:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.862+0000 7f407ffff640 1 --2- 172.21.15.78:0/1949497690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 0x5584b4fe17f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.862+0000 7f407ffff640 1 -- 172.21.15.78:0/1949497690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584b4c99650 con 0x5584b4fe1d30 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.862+0000 7f407ffff640 1 --2- 172.21.15.78:0/1949497690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fe1d30 0x5584b4fe4120 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f407002a9e0 tx=0x7f40700040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2268190536e677b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.863+0000 7f407d7fa640 1 -- 172.21.15.78:0/1949497690 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070035030 con 0x5584b4fe1d30 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.863+0000 7f407d7fa640 1 -- 172.21.15.78:0/1949497690 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40700054f0 con 0x5584b4fe1d30 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.863+0000 7f4084ff9640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f40740b5550 0x7f40740b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.863+0000 7f4084ff9640 1 -- 172.21.15.78:0/2189322937 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f40780e64c0 con 0x7f4088103b30 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.864+0000 7f407d7fa640 1 -- 172.21.15.78:0/1949497690 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40700035f0 con 0x5584b4fe1d30 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.864+0000 7f408fa42740 1 -- 172.21.15.78:0/1949497690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fe1d30 msgr2=0x5584b4fe4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.864+0000 7f408fa42740 1 --2- 172.21.15.78:0/1949497690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fe1d30 0x5584b4fe4120 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f407002a9e0 tx=0x7f40700040c0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.864+0000 7f408fa42740 1 -- 172.21.15.78:0/1949497690 shutdown_connections 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.864+0000 7f408fa42740 1 --2- 172.21.15.78:0/1949497690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fe1d30 0x5584b4fe4120 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.864+0000 7f408fa42740 1 --2- 172.21.15.78:0/1949497690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 0x5584b4fe17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.864+0000 7f408fa42740 1 --2- 172.21.15.78:0/1949497690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fd5480 0x5584b4fd5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.864+0000 7f408fa42740 1 -- 172.21.15.78:0/1949497690 >> 172.21.15.78:0/1949497690 conn(0x5584b4fd32b0 msgr2=0x5584b4fd4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.864+0000 7f40877fe640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f40740b5550 0x7f40740b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.865+0000 7f408fa42740 1 -- 172.21.15.78:0/1949497690 shutdown_connections 2024-09-11T01:20:52.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.865+0000 7f40877fe640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f40740b5550 0x7f40740b7a10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4080011650 tx=0x7f4080009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:52.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.865+0000 7f408fa42740 1 -- 172.21.15.78:0/1949497690 wait complete. 2024-09-11T01:20:52.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.865+0000 7f408fa42740 1 -- start start 2024-09-11T01:20:52.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.866+0000 7f408fa42740 1 Processor -- start 2024-09-11T01:20:52.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.866+0000 7f408fa42740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4fe1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.866+0000 7f408fa42740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 0x5584b4fdeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.866+0000 7f408fa42740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fe1d30 0x5584b4fdf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.866+0000 7f408fa42740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584b4d9c700 con 0x5584b4fd5480 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.866+0000 7f408fa42740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5584b4ed6110 con 0x5584b4fd5da0 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.866+0000 7f408fa42740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5584b4fe49a0 con 0x5584b4fe1d30 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.866+0000 7f4086ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4fe1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.866+0000 7f408fa42740 10 client.0 _ll_register_callbacks cb 0x5584b4eef990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.866+0000 7f4086ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4fe1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37996/0 (socket says 172.21.15.78:37996) 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.866+0000 7f4086ffd640 1 -- 172.21.15.78:0/1368572164 learned_addr learned my addr 172.21.15.78:0/1368572164 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.867+0000 7f40877fe640 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fe1d30 0x5584b4fdf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.867+0000 7f407ffff640 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 0x5584b4fdeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.867+0000 7f407ffff640 1 -- 172.21.15.78:0/1368572164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fe1d30 msgr2=0x5584b4fdf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.867+0000 7f407ffff640 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fe1d30 0x5584b4fdf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.867+0000 7f407ffff640 1 -- 172.21.15.78:0/1368572164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 msgr2=0x5584b4fe1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.867+0000 7f407ffff640 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4fe1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.867+0000 7f407ffff640 1 -- 172.21.15.78:0/1368572164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584b4f03c40 con 0x5584b4fd5da0 2024-09-11T01:20:52.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.867+0000 7f40877fe640 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fe1d30 0x5584b4fdf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:20:52.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.867+0000 7f4086ffd640 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4fe1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:20:52.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.867+0000 7f407ffff640 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 0x5584b4fdeaa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4070009850 tx=0x7f4070005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5045cfb3ed787ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:52.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.867+0000 7f406e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:20:52.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.868+0000 7f406e7fc640 1 -- 172.21.15.78:0/1368572164 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070035040 con 0x5584b4fd5da0 2024-09-11T01:20:52.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.868+0000 7f406e7fc640 1 -- 172.21.15.78:0/1368572164 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f40700035f0 con 0x5584b4fd5da0 2024-09-11T01:20:52.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.868+0000 7f408fa42740 1 -- 172.21.15.78:0/1368572164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40700054f0 con 0x5584b4fd5da0 2024-09-11T01:20:52.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.868+0000 7f406e7fc640 1 -- 172.21.15.78:0/1368572164 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f4070005320 con 0x5584b4fd5da0 2024-09-11T01:20:52.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.868+0000 7f408fa42740 10 client.28241 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:52.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.868+0000 7f408fa42740 1 -- 172.21.15.78:0/1368572164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x5584b4ff0370 con 0x5584b4fd5da0 2024-09-11T01:20:52.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.868+0000 7f406e7fc640 1 -- 172.21.15.78:0/1368572164 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4070003ca0 con 0x5584b4fd5da0 2024-09-11T01:20:52.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.869+0000 7f406e7fc640 1 -- 172.21.15.78:0/1368572164 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f407004b070 con 0x5584b4fd5da0 2024-09-11T01:20:52.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.869+0000 7f408fa42740 10 client.28241 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:52.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.869+0000 7f408fa42740 10 client.28241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:20:52.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.869+0000 7f408fa42740 20 client.28241 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:52.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.869+0000 7f408fa42740 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5584b4ff16c0 0x5584b4ff3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.869+0000 7f408fa42740 4 client.28241 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.869+0000 7f408fa42740 1 -- 172.21.15.78:0/1368572164 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f40700054f0 con 0x5584b4ff16c0 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.869+0000 7f4086ffd640 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5584b4ff16c0 0x5584b4ff3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.869+0000 7f406e7fc640 10 client.28241 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.870+0000 7f4086ffd640 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5584b4ff16c0 0x5584b4ff3ab0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ef3e7bc4216d364 server_cookie=95fb112cd45f6365 in_seq=0 out_seq=0 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.870+0000 7f406e7fc640 1 -- 172.21.15.78:0/1368572164 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f40700054f0 con 0x5584b4ff16c0 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.870+0000 7f406e7fc640 10 client.28241 handle_command_reply: tid=0 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.870+0000 7f408fa42740 1 client.28241 shutdown 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.871+0000 7f408fa42740 1 -- 172.21.15.78:0/1368572164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 msgr2=0x5584b4fdeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.871+0000 7f408fa42740 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 0x5584b4fdeaa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4070009850 tx=0x7f4070005230 comp rx=0 tx=0).stop 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.871+0000 7f408fa42740 1 -- 172.21.15.78:0/1368572164 shutdown_connections 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.871+0000 7f408fa42740 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fe1d30 0x5584b4fdf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.871+0000 7f408fa42740 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 0x5584b4fdeaa0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.871+0000 7f408fa42740 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5584b4ff16c0 0x5584b4ff3ab0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.871+0000 7f406e7fc640 0 client.28241 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.871+0000 7f408fa42740 1 --2- 172.21.15.78:0/1368572164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4fe1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.871+0000 7f408fa42740 1 -- 172.21.15.78:0/1368572164 >> 172.21.15.78:0/1368572164 conn(0x5584b4fd32b0 msgr2=0x5584b4fd8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.871+0000 7f408fa42740 1 -- 172.21.15.78:0/1368572164 shutdown_connections 2024-09-11T01:20:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.871+0000 7f408fa42740 1 -- 172.21.15.78:0/1368572164 wait complete. 2024-09-11T01:20:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.871+0000 7f408fa42740 20 client.28241 trim_cache size 0 max 16384 2024-09-11T01:20:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.901+0000 7f408fa42740 1 Processor -- start 2024-09-11T01:20:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.901+0000 7f408fa42740 1 -- start start 2024-09-11T01:20:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.902+0000 7f408fa42740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4fedcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.902+0000 7f408fa42740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fd5da0 0x5584b4fddf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.902+0000 7f408fa42740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fe1d30 0x5584b4fdd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.902+0000 7f408fa42740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40881bd4a0 con 0x5584b4fd5480 2024-09-11T01:20:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.902+0000 7f408fa42740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f40881bf000 con 0x5584b4fe1d30 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.902+0000 7f4086ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4fedcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.902+0000 7f4086ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4fedcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38000/0 (socket says 172.21.15.78:38000) 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.902+0000 7f4086ffd640 1 -- 172.21.15.78:0/3515724089 learned_addr learned my addr 172.21.15.78:0/3515724089 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.902+0000 7f407ffff640 1 --2- 172.21.15.78:0/3515724089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fd5da0 0x5584b4fddf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.902+0000 7f408fa42740 1 -- 172.21.15.78:0/3515724089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40881bdad0 con 0x5584b4fd5da0 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.902+0000 7f40877fe640 1 --2- 172.21.15.78:0/3515724089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fe1d30 0x5584b4fdd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.903+0000 7f407ffff640 1 -- 172.21.15.78:0/3515724089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fe1d30 msgr2=0x5584b4fdd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.903+0000 7f407ffff640 1 --2- 172.21.15.78:0/3515724089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fe1d30 0x5584b4fdd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.903+0000 7f407ffff640 1 -- 172.21.15.78:0/3515724089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 msgr2=0x5584b4fedcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.903+0000 7f407ffff640 1 --2- 172.21.15.78:0/3515724089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4fedcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.903+0000 7f407ffff640 1 -- 172.21.15.78:0/3515724089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584b4fd3d30 con 0x5584b4fd5da0 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.903+0000 7f40877fe640 1 --2- 172.21.15.78:0/3515724089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fe1d30 0x5584b4fdd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:20:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.903+0000 7f407ffff640 1 --2- 172.21.15.78:0/3515724089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fd5da0 0x5584b4fddf90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4070031a90 tx=0x7f4070005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c142d4138fda9edd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.903+0000 7f407cff9640 1 -- 172.21.15.78:0/3515724089 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070035040 con 0x5584b4fd5da0 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.904+0000 7f407cff9640 1 -- 172.21.15.78:0/3515724089 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40700314a0 con 0x5584b4fd5da0 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.904+0000 7f407cff9640 1 -- 172.21.15.78:0/3515724089 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070003ba0 con 0x5584b4fd5da0 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.904+0000 7f408fa42740 1 -- 172.21.15.78:0/3515724089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fd5da0 msgr2=0x5584b4fddf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.904+0000 7f408fa42740 1 --2- 172.21.15.78:0/3515724089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fd5da0 0x5584b4fddf90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4070031a90 tx=0x7f4070005b50 comp rx=0 tx=0).stop 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.905+0000 7f408fa42740 1 -- 172.21.15.78:0/3515724089 shutdown_connections 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.905+0000 7f408fa42740 1 --2- 172.21.15.78:0/3515724089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fe1d30 0x5584b4fdd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.905+0000 7f408fa42740 1 --2- 172.21.15.78:0/3515724089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fd5da0 0x5584b4fddf90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.905+0000 7f408fa42740 1 --2- 172.21.15.78:0/3515724089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4fedcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.905+0000 7f408fa42740 1 -- 172.21.15.78:0/3515724089 >> 172.21.15.78:0/3515724089 conn(0x5584b4fd32b0 msgr2=0x5584b4fd43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.905+0000 7f408fa42740 1 -- 172.21.15.78:0/3515724089 shutdown_connections 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.905+0000 7f408fa42740 1 -- 172.21.15.78:0/3515724089 wait complete. 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.906+0000 7f408fa42740 1 -- start start 2024-09-11T01:20:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.906+0000 7f408fa42740 1 Processor -- start 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.906+0000 7f408fa42740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4ff73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.906+0000 7f408fa42740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 0x5584b4ff28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.906+0000 7f408fa42740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fe1d30 0x5584b4ff2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.906+0000 7f408fa42740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40881be930 con 0x5584b4fd5480 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.906+0000 7f408fa42740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f40881be1a0 con 0x5584b4fd5da0 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.906+0000 7f408fa42740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f408810af00 con 0x5584b4fe1d30 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.906+0000 7f408fa42740 10 client.0 _ll_register_callbacks cb 0x7f4080009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f4086ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4ff73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f407ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 0x5584b4ff28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f4086ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4ff73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38006/0 (socket says 172.21.15.78:38006) 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f40877fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fe1d30 0x5584b4ff2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f4086ffd640 1 -- 172.21.15.78:0/1992863799 learned_addr learned my addr 172.21.15.78:0/1992863799 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f40877fe640 1 -- 172.21.15.78:0/1992863799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 msgr2=0x5584b4ff28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f40877fe640 1 --2- 172.21.15.78:0/1992863799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 0x5584b4ff28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f40877fe640 1 -- 172.21.15.78:0/1992863799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 msgr2=0x5584b4ff73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f40877fe640 1 --2- 172.21.15.78:0/1992863799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4ff73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f40877fe640 1 -- 172.21.15.78:0/1992863799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584b4fd9790 con 0x5584b4fe1d30 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f40877fe640 1 --2- 172.21.15.78:0/1992863799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fe1d30 0x5584b4ff2f40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4080005fd0 tx=0x7f4080004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbe4bbaadc2a94f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f406e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f406e7fc640 1 -- 172.21.15.78:0/1992863799 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408002e060 con 0x5584b4fe1d30 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.907+0000 7f406e7fc640 1 -- 172.21.15.78:0/1992863799 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f408000fb10 con 0x5584b4fe1d30 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.908+0000 7f408fa42740 1 -- 172.21.15.78:0/1992863799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40700314a0 con 0x5584b4fe1d30 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.908+0000 7f406e7fc640 1 -- 172.21.15.78:0/1992863799 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f408000fd10 con 0x5584b4fe1d30 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.908+0000 7f408fa42740 10 client.28183 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.908+0000 7f408fa42740 1 -- 172.21.15.78:0/1992863799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x5584b505e230 con 0x5584b4fe1d30 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.908+0000 7f406e7fc640 1 -- 172.21.15.78:0/1992863799 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4080033b30 con 0x5584b4fe1d30 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.908+0000 7f406e7fc640 1 -- 172.21.15.78:0/1992863799 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f4080029810 con 0x5584b4fe1d30 2024-09-11T01:20:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.908+0000 7f408fa42740 10 client.28183 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.908+0000 7f408fa42740 10 client.28183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.908+0000 7f408fa42740 20 client.28183 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.908+0000 7f408fa42740 1 --2- 172.21.15.78:0/1992863799 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5584b4ff16c0 0x5584b5060a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.908+0000 7f408fa42740 4 client.28183 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.908+0000 7f408fa42740 1 -- 172.21.15.78:0/1992863799 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f40700314a0 con 0x5584b4ff16c0 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.909+0000 7f4086ffd640 1 --2- 172.21.15.78:0/1992863799 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5584b4ff16c0 0x5584b5060a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.909+0000 7f406e7fc640 10 client.28183 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f4086ffd640 1 --2- 172.21.15.78:0/1992863799 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5584b4ff16c0 0x5584b5060a20 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eccf079a64c00734 server_cookie=7c5d2c7820b24c25 in_seq=0 out_seq=0 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f406e7fc640 1 -- 172.21.15.78:0/1992863799 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7474 (crc 0 0 0) 0x7f40700314a0 con 0x5584b4ff16c0 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f406e7fc640 10 client.28183 handle_command_reply: tid=0 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f408fa42740 1 client.28183 shutdown 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f408fa42740 1 -- 172.21.15.78:0/1992863799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fe1d30 msgr2=0x5584b4ff2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f408fa42740 1 --2- 172.21.15.78:0/1992863799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fe1d30 0x5584b4ff2f40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4080005fd0 tx=0x7f4080004a40 comp rx=0 tx=0).stop 2024-09-11T01:20:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f408fa42740 1 -- 172.21.15.78:0/1992863799 shutdown_connections 2024-09-11T01:20:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f408fa42740 1 --2- 172.21.15.78:0/1992863799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584b4fe1d30 0x5584b4ff2f40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f408fa42740 1 --2- 172.21.15.78:0/1992863799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5584b4fd5da0 0x5584b4ff28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f408fa42740 1 --2- 172.21.15.78:0/1992863799 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5584b4ff16c0 0x5584b5060a20 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f408fa42740 1 --2- 172.21.15.78:0/1992863799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584b4fd5480 0x5584b4ff73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f406e7fc640 0 client.28183 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:20:52.698 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:20:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:20:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:20:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:20:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "746add9a-5be3-4745-8000-04e5471bc813": { 2024-09-11T01:20:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:20:12.369593+0000", 2024-09-11T01:20:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: "b41a2b77-b2b1-4f1b-8588-4f32a78b721f": { 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:19:13.257610+0000", 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:18:17.654570+0000", 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.700 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.700 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.700 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.700 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:20:52.700 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:20:52.700 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.700 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.700 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.700 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.700 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.701 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:20:52.702 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.704 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:20:52.704 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:20:52.704 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.704 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.704 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.704 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.704 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.704 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:20:52.704 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:20:52.705 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.705 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.705 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.705 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.705 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.706 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:20:52.706 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:20:52.706 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.706 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.706 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.706 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.707 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.707 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:20:52.707 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:20:52.707 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.707 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.708 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.708 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.708 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.708 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:20:52.708 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:20:52.709 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.709 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.709 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.709 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.709 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.710 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:20:52.710 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:20:52.710 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.710 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.710 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.710 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.711 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.711 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:20:52.711 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:20:52.711 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.711 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.711 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.711 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.711 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.711 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:20:52.711 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.712 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.713 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:20:52.713 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:20:52.713 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.713 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.713 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.713 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.713 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.713 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:20:52.713 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.715 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:20:52.715 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:20:52.715 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.715 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.715 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.715 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.715 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.715 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:20:52.715 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.716 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:20:52.717 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:20:52.718 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:20:52.718 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:20:52.718 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:20:52.718 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:20:52.718 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:20:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.910+0000 7f408fa42740 1 -- 172.21.15.78:0/1992863799 >> 172.21.15.78:0/1992863799 conn(0x5584b4fd32b0 msgr2=0x5584b4fe00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.911+0000 7f408fa42740 1 -- 172.21.15.78:0/1992863799 shutdown_connections 2024-09-11T01:20:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.911+0000 7f408fa42740 1 -- 172.21.15.78:0/1992863799 wait complete. 2024-09-11T01:20:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.911+0000 7f408fa42740 20 client.28183 trim_cache size 0 max 16384 2024-09-11T01:20:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.912+0000 7f408d66d640 1 -- 172.21.15.78:0/2189322937 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f40740b5550 msgr2=0x7f40740b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.912+0000 7f408d66d640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f40740b5550 0x7f40740b7a10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4080011650 tx=0x7f4080009500 comp rx=0 tx=0).stop 2024-09-11T01:20:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.912+0000 7f408d66d640 1 -- 172.21.15.78:0/2189322937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4088103b30 msgr2=0x7f408810c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.912+0000 7f408d66d640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4088103b30 0x7f408810c9d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f407800e9f0 tx=0x7f407800edf0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.912+0000 7f408d66d640 1 -- 172.21.15.78:0/2189322937 shutdown_connections 2024-09-11T01:20:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.912+0000 7f408d66d640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f40740b5550 0x7f40740b7a10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.912+0000 7f408d66d640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f408810d0e0 0x7f408810a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.912+0000 7f408d66d640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4088104530 0x7f4088109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.912+0000 7f408d66d640 1 --2- 172.21.15.78:0/2189322937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4088103b30 0x7f408810c9d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.912+0000 7f408d66d640 1 -- 172.21.15.78:0/2189322937 >> 172.21.15.78:0/2189322937 conn(0x7f40880fd700 msgr2=0x7f40880fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.912+0000 7f408d66d640 1 -- 172.21.15.78:0/2189322937 shutdown_connections 2024-09-11T01:20:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:51.912+0000 7f408d66d640 1 -- 172.21.15.78:0/2189322937 wait complete. 2024-09-11T01:20:52.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 335211 32.88 MB/sec execute 418 sec latency 269.514 ms 2024-09-11T01:20:53.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 335356 32.83 MB/sec execute 419 sec latency 634.001 ms 2024-09-11T01:20:54.169 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:20:54.169 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:20:54.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:53 smithi078 ceph-mon[31315]: pgmap v1013: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 604 op/s 2024-09-11T01:20:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:53 smithi110 ceph-mon[34712]: pgmap v1013: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 604 op/s 2024-09-11T01:20:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:53 smithi112 ceph-mon[35384]: pgmap v1013: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 604 op/s 2024-09-11T01:20:54.560 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:20:54.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 335712 32.80 MB/sec execute 420 sec latency 931.620 ms 2024-09-11T01:20:55.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.444+0000 7fd694649640 1 -- 172.21.15.78:0/2913123736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c10edc0 msgr2=0x7fd68c10f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:55.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.444+0000 7fd694649640 1 --2- 172.21.15.78:0/2913123736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c10edc0 0x7fd68c10f200 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd688009860 tx=0x7fd68802a340 comp rx=0 tx=0).stop 2024-09-11T01:20:55.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.445+0000 7fd694649640 1 -- 172.21.15.78:0/2913123736 shutdown_connections 2024-09-11T01:20:55.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.445+0000 7fd694649640 1 --2- 172.21.15.78:0/2913123736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd68c10f740 0x7fd68c115f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:55.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.445+0000 7fd694649640 1 --2- 172.21.15.78:0/2913123736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c10edc0 0x7fd68c10f200 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:55.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.445+0000 7fd694649640 1 --2- 172.21.15.78:0/2913123736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68c1093b0 0x7fd68c109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.445+0000 7fd694649640 1 -- 172.21.15.78:0/2913123736 >> 172.21.15.78:0/2913123736 conn(0x7fd68c078a30 msgr2=0x7fd68c07ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.445+0000 7fd694649640 1 -- 172.21.15.78:0/2913123736 shutdown_connections 2024-09-11T01:20:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.445+0000 7fd694649640 1 -- 172.21.15.78:0/2913123736 wait complete. 2024-09-11T01:20:55.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.446+0000 7fd694649640 1 Processor -- start 2024-09-11T01:20:55.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.446+0000 7fd694649640 1 -- start start 2024-09-11T01:20:55.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.446+0000 7fd694649640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd68c1093b0 0x7fd68c19f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:55.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.446+0000 7fd694649640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68c10edc0 0x7fd68c19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:55.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.446+0000 7fd694649640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c10f740 0x7fd68c1a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:55.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.447+0000 7fd691be4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68c10edc0 0x7fd68c19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:55.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.447+0000 7fd691be4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68c10edc0 0x7fd68c19fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:58582/0 (socket says 172.21.15.78:58582) 2024-09-11T01:20:55.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.447+0000 7fd6923e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd68c1093b0 0x7fd68c19f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:55.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.447+0000 7fd6923e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd68c1093b0 0x7fd68c19f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42216/0 (socket says 172.21.15.78:42216) 2024-09-11T01:20:55.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.447+0000 7fd694649640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c119340 con 0x7fd68c10f740 2024-09-11T01:20:55.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.447+0000 7fd694649640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1191c0 con 0x7fd68c10edc0 2024-09-11T01:20:55.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.447+0000 7fd694649640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1194c0 con 0x7fd68c1093b0 2024-09-11T01:20:55.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.447+0000 7fd691be4640 1 -- 172.21.15.78:0/1168792830 learned_addr learned my addr 172.21.15.78:0/1168792830 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:55.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.447+0000 7fd692be6640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c10f740 0x7fd68c1a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:55.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.448+0000 7fd691be4640 1 -- 172.21.15.78:0/1168792830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd68c1093b0 msgr2=0x7fd68c19f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:55.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.448+0000 7fd691be4640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd68c1093b0 0x7fd68c19f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:55.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.448+0000 7fd691be4640 1 -- 172.21.15.78:0/1168792830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c10f740 msgr2=0x7fd68c1a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:55.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.448+0000 7fd691be4640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c10f740 0x7fd68c1a5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.448+0000 7fd691be4640 1 -- 172.21.15.78:0/1168792830 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c1a5940 con 0x7fd68c10edc0 2024-09-11T01:20:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.448+0000 7fd6923e5640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd68c1093b0 0x7fd68c19f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:20:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.448+0000 7fd691be4640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68c10edc0 0x7fd68c19fdd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd688035040 tx=0x7fd688032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.449+0000 7fd6837fe640 1 -- 172.21.15.78:0/1168792830 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688037040 con 0x7fd68c10edc0 2024-09-11T01:20:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.449+0000 7fd6837fe640 1 -- 172.21.15.78:0/1168792830 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd688041070 con 0x7fd68c10edc0 2024-09-11T01:20:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.450+0000 7fd694649640 1 -- 172.21.15.78:0/1168792830 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c1a5bd0 con 0x7fd68c10edc0 2024-09-11T01:20:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.450+0000 7fd694649640 1 -- 172.21.15.78:0/1168792830 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c1ac2f0 con 0x7fd68c10edc0 2024-09-11T01:20:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.451+0000 7fd6837fe640 1 -- 172.21.15.78:0/1168792830 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688003850 con 0x7fd68c10edc0 2024-09-11T01:20:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.452+0000 7fd694649640 1 -- 172.21.15.78:0/1168792830 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6700052c0 con 0x7fd68c10edc0 2024-09-11T01:20:55.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.454+0000 7fd6837fe640 1 -- 172.21.15.78:0/1168792830 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd688030450 con 0x7fd68c10edc0 2024-09-11T01:20:55.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.455+0000 7fd6837fe640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd6780b55a0 0x7fd6780b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:55.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.455+0000 7fd6837fe640 1 -- 172.21.15.78:0/1168792830 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7fd688106190 con 0x7fd68c10edc0 2024-09-11T01:20:55.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.455+0000 7fd6923e5640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd6780b55a0 0x7fd6780b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:55.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.456+0000 7fd6923e5640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd6780b55a0 0x7fd6780b7a60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00bfd0 tx=0x7fd67c00b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:55.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.460+0000 7fd6837fe640 1 -- 172.21.15.78:0/1168792830 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd6880ccdd0 con 0x7fd68c10edc0 2024-09-11T01:20:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.796+0000 7fd694649640 1 -- 172.21.15.78:0/1168792830 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6700055b0 con 0x7fd68c10edc0 2024-09-11T01:20:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.797+0000 7fd6837fe640 1 -- 172.21.15.78:0/1168792830 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+11986 (secure 0 0 0) 0x7fd6880cc6d0 con 0x7fd68c10edc0 2024-09-11T01:20:55.799 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:20:55.799 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":72,"btime":"2024-09-11T01:19:32:815193+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":72,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:19:32.242847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:20:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 72 2024-09-11T01:20:55.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 336308 32.78 MB/sec execute 421 sec latency 141.386 ms 2024-09-11T01:20:55.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.801+0000 7fd694649640 1 -- 172.21.15.78:0/1168792830 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd6780b55a0 msgr2=0x7fd6780b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:55.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.801+0000 7fd694649640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd6780b55a0 0x7fd6780b7a60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00bfd0 tx=0x7fd67c00b390 comp rx=0 tx=0).stop 2024-09-11T01:20:55.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.801+0000 7fd694649640 1 -- 172.21.15.78:0/1168792830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68c10edc0 msgr2=0x7fd68c19fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:55.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.801+0000 7fd694649640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68c10edc0 0x7fd68c19fdd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd688035040 tx=0x7fd688032dc0 comp rx=0 tx=0).stop 2024-09-11T01:20:55.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.801+0000 7fd694649640 1 -- 172.21.15.78:0/1168792830 shutdown_connections 2024-09-11T01:20:55.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.801+0000 7fd694649640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd6780b55a0 0x7fd6780b7a60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:55.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.801+0000 7fd694649640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c10f740 0x7fd68c1a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:55.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.801+0000 7fd694649640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68c10edc0 0x7fd68c19fdd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:55.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.801+0000 7fd694649640 1 --2- 172.21.15.78:0/1168792830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd68c1093b0 0x7fd68c19f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:55.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.801+0000 7fd694649640 1 -- 172.21.15.78:0/1168792830 >> 172.21.15.78:0/1168792830 conn(0x7fd68c078a30 msgr2=0x7fd68c07ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:55.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.801+0000 7fd694649640 1 -- 172.21.15.78:0/1168792830 shutdown_connections 2024-09-11T01:20:55.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:55.801+0000 7fd694649640 1 -- 172.21.15.78:0/1168792830 wait complete. 2024-09-11T01:20:56.187 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:55 smithi078 ceph-mon[31315]: pgmap v1014: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 532 op/s 2024-09-11T01:20:56.187 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:55 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1168792830' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:20:56.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:55 smithi110 ceph-mon[34712]: pgmap v1014: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 532 op/s 2024-09-11T01:20:56.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:55 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1168792830' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:20:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:55 smithi112 ceph-mon[35384]: pgmap v1014: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 532 op/s 2024-09-11T01:20:56.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:55 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1168792830' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:20:56.552 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:20:56.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 337045 32.83 MB/sec execute 422 sec latency 96.258 ms 2024-09-11T01:20:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:20:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.488+0000 7f4e4cb12640 1 -- 172.21.15.78:0/4266776165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e48101c90 msgr2=0x7f4e48102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.488+0000 7f4e4cb12640 1 --2- 172.21.15.78:0/4266776165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e48101c90 0x7f4e48102070 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009900 tx=0x7f4e3802a410 comp rx=0 tx=0).stop 2024-09-11T01:20:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.488+0000 7f4e4cb12640 1 -- 172.21.15.78:0/4266776165 shutdown_connections 2024-09-11T01:20:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.488+0000 7f4e4cb12640 1 --2- 172.21.15.78:0/4266776165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e48108020 0x7f4e4810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.488+0000 7f4e4cb12640 1 --2- 172.21.15.78:0/4266776165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e481076a0 0x7f4e48107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.488+0000 7f4e4cb12640 1 --2- 172.21.15.78:0/4266776165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e48101c90 0x7f4e48102070 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.488+0000 7f4e4cb12640 1 -- 172.21.15.78:0/4266776165 >> 172.21.15.78:0/4266776165 conn(0x7f4e480fd700 msgr2=0x7f4e480ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.488+0000 7f4e4cb12640 1 -- 172.21.15.78:0/4266776165 shutdown_connections 2024-09-11T01:20:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.488+0000 7f4e4cb12640 1 -- 172.21.15.78:0/4266776165 wait complete. 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.489+0000 7f4e4cb12640 1 Processor -- start 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e4cb12640 1 -- start start 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e4cb12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e48101c90 0x7f4e4810db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e4cb12640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e481076a0 0x7f4e4810e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e4cb12640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e48108020 0x7f4e4819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e4cb12640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4806b5e0 con 0x7f4e48108020 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e4cb12640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4806b460 con 0x7f4e481076a0 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e4cb12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4806b760 con 0x7f4e48101c90 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e46d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e48108020 0x7f4e4819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e46d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e48108020 0x7f4e4819da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47794/0 (socket says 172.21.15.78:47794) 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e46d9d640 1 -- 172.21.15.78:0/1764478637 learned_addr learned my addr 172.21.15.78:0/1764478637 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e4659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e48101c90 0x7f4e4810db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e45d9b640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e481076a0 0x7f4e4810e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e46d9d640 1 -- 172.21.15.78:0/1764478637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e48101c90 msgr2=0x7f4e4810db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e46d9d640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e48101c90 0x7f4e4810db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e46d9d640 1 -- 172.21.15.78:0/1764478637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e481076a0 msgr2=0x7f4e4810e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e46d9d640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e481076a0 0x7f4e4810e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.490+0000 7f4e46d9d640 1 -- 172.21.15.78:0/1764478637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4819e1f0 con 0x7f4e48108020 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.491+0000 7f4e46d9d640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e48108020 0x7f4e4819da20 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000eb10 tx=0x7f4e4000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.491+0000 7f4e377fe640 1 -- 172.21.15.78:0/1764478637 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4000ce20 con 0x7f4e48108020 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.491+0000 7f4e4cb12640 1 -- 172.21.15.78:0/1764478637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4819e4e0 con 0x7f4e48108020 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.491+0000 7f4e377fe640 1 -- 172.21.15.78:0/1764478637 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e40005020 con 0x7f4e48108020 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.491+0000 7f4e4cb12640 1 -- 172.21.15.78:0/1764478637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e481a7f90 con 0x7f4e48108020 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.491+0000 7f4e377fe640 1 -- 172.21.15.78:0/1764478637 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40010690 con 0x7f4e48108020 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.494+0000 7f4e4eee7740 1 Processor -- start 2024-09-11T01:20:57.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.496+0000 7f4e377fe640 1 -- 172.21.15.78:0/1764478637 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4e40004a40 con 0x7f4e48108020 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.496+0000 7f4e4eee7740 1 -- start start 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.496+0000 7f4e4eee7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe25494d0 0x556fe25498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.496+0000 7f4e4eee7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe2555840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.496+0000 7f4e4eee7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe2555d80 0x556fe2558170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.496+0000 7f4e4eee7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556fe24bfea0 con 0x556fe25494d0 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.496+0000 7f4e4eee7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556fe24af2d0 con 0x556fe2549df0 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.496+0000 7f4e4eee7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556fe24b30e0 con 0x556fe2555d80 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.496+0000 7f4e45d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe25494d0 0x556fe25498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.496+0000 7f4e45d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe25494d0 0x556fe25498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47804/0 (socket says 172.21.15.78:47804) 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.496+0000 7f4e45d9b640 1 -- 172.21.15.78:0/2632653822 learned_addr learned my addr 172.21.15.78:0/2632653822 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.497+0000 7f4e45d9b640 1 --2- 172.21.15.78:0/2632653822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe2555d80 0x556fe2558170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.497+0000 7f4e45d9b640 1 -- 172.21.15.78:0/2632653822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 msgr2=0x556fe2555840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.497+0000 7f4e45d9b640 1 --2- 172.21.15.78:0/2632653822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe2555840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.497+0000 7f4e45d9b640 1 -- 172.21.15.78:0/2632653822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe25494d0 msgr2=0x556fe25498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.497+0000 7f4e45d9b640 1 --2- 172.21.15.78:0/2632653822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe25494d0 0x556fe25498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.497+0000 7f4e45d9b640 1 -- 172.21.15.78:0/2632653822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe220d650 con 0x556fe2555d80 2024-09-11T01:20:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.498+0000 7f4e377fe640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4e300b54d0 0x7f4e300b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.498+0000 7f4e45d9b640 1 --2- 172.21.15.78:0/2632653822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe2555d80 0x556fe2558170 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c017700 tx=0x7f4e3c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3967ad2025906b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.498+0000 7f4e4659c640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4e300b54d0 0x7f4e300b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.498+0000 7f4e377fe640 1 -- 172.21.15.78:0/1764478637 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f4e400e6410 con 0x7f4e48108020 2024-09-11T01:20:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.498+0000 7f4e4659c640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4e300b54d0 0x7f4e300b7990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x556fe25481e0 tx=0x7f4e3800b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.498+0000 7f4e34ff9640 1 -- 172.21.15.78:0/2632653822 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c01c020 con 0x556fe2555d80 2024-09-11T01:20:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.499+0000 7f4e34ff9640 1 -- 172.21.15.78:0/2632653822 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e3c015db0 con 0x556fe2555d80 2024-09-11T01:20:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.499+0000 7f4e34ff9640 1 -- 172.21.15.78:0/2632653822 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c018aa0 con 0x556fe2555d80 2024-09-11T01:20:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.499+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2632653822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe2555d80 msgr2=0x556fe2558170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.499+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/2632653822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe2555d80 0x556fe2558170 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c017700 tx=0x7f4e3c0046c0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.499+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2632653822 shutdown_connections 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.499+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/2632653822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe2555d80 0x556fe2558170 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.499+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/2632653822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe2555840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.499+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/2632653822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe25494d0 0x556fe25498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.499+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2632653822 >> 172.21.15.78:0/2632653822 conn(0x556fe2547300 msgr2=0x556fe25484a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.500+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2632653822 shutdown_connections 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.500+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2632653822 wait complete. 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.500+0000 7f4e4eee7740 1 -- start start 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.500+0000 7f4e4eee7740 1 Processor -- start 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.501+0000 7f4e4eee7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2555600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.501+0000 7f4e4eee7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe2552a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.501+0000 7f4e4eee7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2555d80 0x556fe2552ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.501+0000 7f4e4659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2555600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e4659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2555600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42266/0 (socket says 172.21.15.78:42266) 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e4659c640 1 -- 172.21.15.78:0/286645462 learned_addr learned my addr 172.21.15.78:0/286645462 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e4eee7740 1 -- 172.21.15.78:0/286645462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556fe2310700 con 0x556fe2555d80 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e4eee7740 1 -- 172.21.15.78:0/286645462 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556fe244a160 con 0x556fe2549df0 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e4eee7740 1 -- 172.21.15.78:0/286645462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556fe2558a50 con 0x556fe25494d0 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e4eee7740 10 client.0 _ll_register_callbacks cb 0x556fe24639e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e45d9b640 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe2552a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e46d9d640 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2555d80 0x556fe2552ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e45d9b640 1 -- 172.21.15.78:0/286645462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 msgr2=0x556fe2555600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e45d9b640 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2555600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e45d9b640 1 -- 172.21.15.78:0/286645462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2555d80 msgr2=0x556fe2552ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e45d9b640 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2555d80 0x556fe2552ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e45d9b640 1 -- 172.21.15.78:0/286645462 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe2477c90 con 0x556fe2549df0 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.502+0000 7f4e4659c640 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2555600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.503+0000 7f4e45d9b640 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe2552a50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c018620 tx=0x7f4e3c009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7c889fe9e98def4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.503+0000 7f4e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.503+0000 7f4e2dffb640 1 -- 172.21.15.78:0/286645462 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c01c030 con 0x556fe2549df0 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.503+0000 7f4e4eee7740 1 -- 172.21.15.78:0/286645462 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e3c015db0 con 0x556fe2549df0 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.503+0000 7f4e2dffb640 1 -- 172.21.15.78:0/286645462 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f4e3c018820 con 0x556fe2549df0 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.504+0000 7f4e2dffb640 1 -- 172.21.15.78:0/286645462 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f4e3c0095a0 con 0x556fe2549df0 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.504+0000 7f4e4eee7740 10 client.28259 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.504+0000 7f4e4eee7740 1 -- 172.21.15.78:0/286645462 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x556fe25643b0 con 0x556fe2549df0 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.504+0000 7f4e2dffb640 1 -- 172.21.15.78:0/286645462 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e3c015db0 con 0x556fe2549df0 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.504+0000 7f4e2dffb640 1 -- 172.21.15.78:0/286645462 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f4e3c0214c0 con 0x556fe2549df0 2024-09-11T01:20:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.505+0000 7f4e4eee7740 10 client.28259 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.505+0000 7f4e4eee7740 10 client.28259 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.505+0000 7f4e4eee7740 20 client.28259 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.505+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556fe2565810 0x556fe2567c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.505+0000 7f4e4eee7740 4 client.28259 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.505+0000 7f4e4eee7740 1 -- 172.21.15.78:0/286645462 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fe24c1df0 con 0x556fe2565810 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.505+0000 7f4e4659c640 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556fe2565810 0x556fe2567c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.505+0000 7f4e4659c640 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556fe2565810 0x556fe2567c00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fb0158263e83023c server_cookie=fa76694c7e2068a9 in_seq=0 out_seq=0 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.505+0000 7f4e2dffb640 10 client.28259 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e2dffb640 1 -- 172.21.15.78:0/286645462 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556fe24c1df0 con 0x556fe2565810 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e2dffb640 10 client.28259 handle_command_reply: tid=0 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e4eee7740 1 client.28259 shutdown 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e4eee7740 1 -- 172.21.15.78:0/286645462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 msgr2=0x556fe2552a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe2552a50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c018620 tx=0x7f4e3c009b20 comp rx=0 tx=0).stop 2024-09-11T01:20:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e4eee7740 1 -- 172.21.15.78:0/286645462 shutdown_connections 2024-09-11T01:20:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556fe2565810 0x556fe2567c00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2555d80 0x556fe2552ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e2dffb640 0 client.28259 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:20:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe2552a50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/286645462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2555600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e4eee7740 1 -- 172.21.15.78:0/286645462 >> 172.21.15.78:0/286645462 conn(0x556fe2547300 msgr2=0x556fe254ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e4eee7740 1 -- 172.21.15.78:0/286645462 shutdown_connections 2024-09-11T01:20:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e4eee7740 1 -- 172.21.15.78:0/286645462 wait complete. 2024-09-11T01:20:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.509+0000 7f4e4eee7740 20 client.28259 trim_cache size 0 max 16384 2024-09-11T01:20:57.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e4eee7740 1 Processor -- start 2024-09-11T01:20:57.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e4eee7740 1 -- start start 2024-09-11T01:20:57.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e4eee7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2546c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e4eee7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2549df0 0x556fe2551f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e4659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2546c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e4659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2546c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:42268/0 (socket says 172.21.15.78:42268) 2024-09-11T01:20:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e45d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2549df0 0x556fe2551f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e45d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2549df0 0x556fe2551f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47812/0 (socket says 172.21.15.78:47812) 2024-09-11T01:20:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e4659c640 1 -- 172.21.15.78:0/2344578944 learned_addr learned my addr 172.21.15.78:0/2344578944 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/2344578944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2555d80 0x556fe254fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2344578944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2800e730 con 0x556fe2549df0 2024-09-11T01:20:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2344578944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28010330 con 0x556fe2555d80 2024-09-11T01:20:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.532+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2344578944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2800ee00 con 0x556fe25494d0 2024-09-11T01:20:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.533+0000 7f4e46d9d640 1 --2- 172.21.15.78:0/2344578944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2555d80 0x556fe254fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.533+0000 7f4e4659c640 1 -- 172.21.15.78:0/2344578944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2555d80 msgr2=0x556fe254fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.533+0000 7f4e4659c640 1 --2- 172.21.15.78:0/2344578944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2555d80 0x556fe254fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.533+0000 7f4e4659c640 1 -- 172.21.15.78:0/2344578944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2549df0 msgr2=0x556fe2551f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.533+0000 7f4e4659c640 1 --2- 172.21.15.78:0/2344578944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2549df0 0x556fe2551f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.533+0000 7f4e4659c640 1 -- 172.21.15.78:0/2344578944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe2547d80 con 0x556fe25494d0 2024-09-11T01:20:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.533+0000 7f4e4659c640 1 --2- 172.21.15.78:0/2344578944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2546c10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4e3802a930 tx=0x7f4e3803d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=162016bfd0eb2bd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.534+0000 7f4e2ffff640 1 -- 172.21.15.78:0/2344578944 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38035040 con 0x556fe25494d0 2024-09-11T01:20:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.534+0000 7f4e2ffff640 1 -- 172.21.15.78:0/2344578944 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e38004750 con 0x556fe25494d0 2024-09-11T01:20:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.534+0000 7f4e2ffff640 1 -- 172.21.15.78:0/2344578944 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e380038f0 con 0x556fe25494d0 2024-09-11T01:20:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.535+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2344578944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 msgr2=0x556fe2546c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.535+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/2344578944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2546c10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4e3802a930 tx=0x7f4e3803d000 comp rx=0 tx=0).stop 2024-09-11T01:20:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.535+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2344578944 shutdown_connections 2024-09-11T01:20:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.535+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/2344578944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2555d80 0x556fe254fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.535+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/2344578944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2549df0 0x556fe2551f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.535+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/2344578944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2546c10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.535+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2344578944 >> 172.21.15.78:0/2344578944 conn(0x556fe2547300 msgr2=0x556fe25512d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.535+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2344578944 shutdown_connections 2024-09-11T01:20:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.535+0000 7f4e4eee7740 1 -- 172.21.15.78:0/2344578944 wait complete. 2024-09-11T01:20:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.535+0000 7f4e4eee7740 1 -- start start 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.535+0000 7f4e4eee7740 1 Processor -- start 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e4eee7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2559ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e4eee7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe255a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e4eee7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2555d80 0x556fe255a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e45d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe255a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e4eee7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2800fc60 con 0x556fe2555d80 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e45d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe255a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:58624/0 (socket says 172.21.15.78:58624) 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e45d9b640 1 -- 172.21.15.78:0/1928681795 learned_addr learned my addr 172.21.15.78:0/1928681795 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e4eee7740 1 -- 172.21.15.78:0/1928681795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2800f4d0 con 0x556fe2549df0 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e4eee7740 1 -- 172.21.15.78:0/1928681795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2800c710 con 0x556fe25494d0 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e4eee7740 10 client.0 _ll_register_callbacks cb 0x7f4e3c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e4659c640 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2559ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e46d9d640 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2555d80 0x556fe255a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e45d9b640 1 -- 172.21.15.78:0/1928681795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 msgr2=0x556fe2559ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e45d9b640 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2559ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e45d9b640 1 -- 172.21.15.78:0/1928681795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2555d80 msgr2=0x556fe255a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:20:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e45d9b640 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2555d80 0x556fe255a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e45d9b640 1 -- 172.21.15.78:0/1928681795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe25d9eb0 con 0x556fe2549df0 2024-09-11T01:20:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.536+0000 7f4e46d9d640 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2555d80 0x556fe255a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:20:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.537+0000 7f4e45d9b640 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe255a430 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c000c00 tx=0x7f4e3c009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f211f40ba9163843 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:20:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.537+0000 7f4e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:20:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.537+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1928681795 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c039070 con 0x556fe2549df0 2024-09-11T01:20:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.537+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1928681795 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f4e3c0093a0 con 0x556fe2549df0 2024-09-11T01:20:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.537+0000 7f4e4eee7740 1 -- 172.21.15.78:0/1928681795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e38004750 con 0x556fe2549df0 2024-09-11T01:20:57.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.537+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1928681795 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f4e3c015910 con 0x556fe2549df0 2024-09-11T01:20:57.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.537+0000 7f4e4eee7740 10 client.28265 fetch_fsmap learned FSMap version 72 2024-09-11T01:20:57.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.537+0000 7f4e4eee7740 1 -- 172.21.15.78:0/1928681795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x556fe25db170 con 0x556fe2549df0 2024-09-11T01:20:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.538+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1928681795 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e38004750 con 0x556fe2549df0 2024-09-11T01:20:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.538+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1928681795 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f4e3c018ba0 con 0x556fe2549df0 2024-09-11T01:20:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.538+0000 7f4e4eee7740 10 client.28265 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:20:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.538+0000 7f4e4eee7740 10 client.28265 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:20:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.538+0000 7f4e4eee7740 20 client.28265 populate_metadata read hostname 'smithi078' 2024-09-11T01:20:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.538+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556fe2565810 0x556fe25dd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:20:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.538+0000 7f4e4eee7740 4 client.28265 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:20:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.538+0000 7f4e4eee7740 1 -- 172.21.15.78:0/1928681795 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556fe24bd8a0 con 0x556fe2565810 2024-09-11T01:20:57.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.538+0000 7f4e4659c640 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556fe2565810 0x556fe25dd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:20:57.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.540+0000 7f4e4659c640 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556fe2565810 0x556fe25dd8a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d4cfeff76bea6ed3 server_cookie=e40094fc346bc0da in_seq=0 out_seq=0 2024-09-11T01:20:57.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.540+0000 7f4e2dffb640 10 client.28265 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:20:57.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.540+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1928681795 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556fe24bd8a0 con 0x556fe2565810 2024-09-11T01:20:57.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.540+0000 7f4e2dffb640 10 client.28265 handle_command_reply: tid=0 2024-09-11T01:20:57.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.540+0000 7f4e4eee7740 1 client.28265 shutdown 2024-09-11T01:20:57.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.540+0000 7f4e4eee7740 1 -- 172.21.15.78:0/1928681795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 msgr2=0x556fe255a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:57.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.540+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe255a430 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c000c00 tx=0x7f4e3c009f20 comp rx=0 tx=0).stop 2024-09-11T01:20:57.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.540+0000 7f4e4eee7740 1 -- 172.21.15.78:0/1928681795 shutdown_connections 2024-09-11T01:20:57.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.540+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556fe2565810 0x556fe25dd8a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.541+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556fe2555d80 0x556fe255a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.541+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556fe2549df0 0x556fe255a430 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.541+0000 7f4e2dffb640 0 client.28265 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:20:57.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.541+0000 7f4e4eee7740 1 --2- 172.21.15.78:0/1928681795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fe25494d0 0x556fe2559ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.541+0000 7f4e4eee7740 1 -- 172.21.15.78:0/1928681795 >> 172.21.15.78:0/1928681795 conn(0x556fe2547300 msgr2=0x556fe25548a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:57.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.541+0000 7f4e4eee7740 1 -- 172.21.15.78:0/1928681795 shutdown_connections 2024-09-11T01:20:57.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.541+0000 7f4e4eee7740 1 -- 172.21.15.78:0/1928681795 wait complete. 2024-09-11T01:20:57.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.541+0000 7f4e4eee7740 20 client.28265 trim_cache size 0 max 16384 2024-09-11T01:20:57.546 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:20:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.542+0000 7f4e4cb12640 1 -- 172.21.15.78:0/1764478637 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4e300b54d0 msgr2=0x7f4e300b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.542+0000 7f4e4cb12640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4e300b54d0 0x7f4e300b7990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x556fe25481e0 tx=0x7f4e3800b040 comp rx=0 tx=0).stop 2024-09-11T01:20:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.542+0000 7f4e4cb12640 1 -- 172.21.15.78:0/1764478637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e48108020 msgr2=0x7f4e4819da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:20:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.542+0000 7f4e4cb12640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e48108020 0x7f4e4819da20 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000eb10 tx=0x7f4e4000ef10 comp rx=0 tx=0).stop 2024-09-11T01:20:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.542+0000 7f4e4cb12640 1 -- 172.21.15.78:0/1764478637 shutdown_connections 2024-09-11T01:20:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.542+0000 7f4e4cb12640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f4e300b54d0 0x7f4e300b7990 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.542+0000 7f4e4cb12640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e48108020 0x7f4e4819da20 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.542+0000 7f4e4cb12640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e481076a0 0x7f4e4810e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.542+0000 7f4e4cb12640 1 --2- 172.21.15.78:0/1764478637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e48101c90 0x7f4e4810db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:20:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.542+0000 7f4e4cb12640 1 -- 172.21.15.78:0/1764478637 >> 172.21.15.78:0/1764478637 conn(0x7f4e480fd700 msgr2=0x7f4e48104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:20:57.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.542+0000 7f4e4cb12640 1 -- 172.21.15.78:0/1764478637 shutdown_connections 2024-09-11T01:20:57.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:20:57.542+0000 7f4e4cb12640 1 -- 172.21.15.78:0/1764478637 wait complete. 2024-09-11T01:20:57.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 337604 32.82 MB/sec execute 423 sec latency 126.946 ms 2024-09-11T01:20:58.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:57 smithi078 ceph-mon[31315]: pgmap v1015: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 655 op/s 2024-09-11T01:20:58.251 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:20:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:57 smithi110 ceph-mon[34712]: pgmap v1015: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 655 op/s 2024-09-11T01:20:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:57 smithi112 ceph-mon[35384]: pgmap v1015: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 655 op/s 2024-09-11T01:20:58.623 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:20:58.801 INFO:tasks.workunit.client.0.smithi078.stdout: 10 338171 32.80 MB/sec execute 424 sec latency 70.417 ms 2024-09-11T01:20:59.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 338466 32.76 MB/sec execute 425 sec latency 389.484 ms 2024-09-11T01:21:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:20:59 smithi110 ceph-mon[34712]: pgmap v1016: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 653 op/s 2024-09-11T01:21:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:20:59 smithi112 ceph-mon[35384]: pgmap v1016: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 653 op/s 2024-09-11T01:21:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:20:59 smithi078 ceph-mon[31315]: pgmap v1016: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 653 op/s 2024-09-11T01:21:00.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 338690 32.69 MB/sec execute 426 sec latency 573.803 ms 2024-09-11T01:21:01.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 338690 32.62 MB/sec execute 427 sec latency 1573.849 ms 2024-09-11T01:21:01.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.596+0000 7f22f99ab640 1 -- 172.21.15.78:0/1932170215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f410d100 msgr2=0x7f22f410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:01.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.596+0000 7f22f99ab640 1 --2- 172.21.15.78:0/1932170215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f410d100 0x7f22f410f560 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009ca0 tx=0x7f22ec02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.596+0000 7f22f99ab640 1 -- 172.21.15.78:0/1932170215 shutdown_connections 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.596+0000 7f22f99ab640 1 --2- 172.21.15.78:0/1932170215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f410d100 0x7f22f410f560 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.597+0000 7f22f99ab640 1 --2- 172.21.15.78:0/1932170215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f41045c0 0x7f22f410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.597+0000 7f22f99ab640 1 --2- 172.21.15.78:0/1932170215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f4103ca0 0x7f22f4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.597+0000 7f22f99ab640 1 -- 172.21.15.78:0/1932170215 >> 172.21.15.78:0/1932170215 conn(0x7f22f40fd700 msgr2=0x7f22f40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.597+0000 7f22f99ab640 1 -- 172.21.15.78:0/1932170215 shutdown_connections 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.597+0000 7f22f99ab640 1 -- 172.21.15.78:0/1932170215 wait complete. 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.597+0000 7f22f99ab640 1 Processor -- start 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.597+0000 7f22f99ab640 1 -- start start 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.597+0000 7f22f99ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f4103ca0 0x7f22f4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.597+0000 7f22f99ab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f41045c0 0x7f22f4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.597+0000 7f22f99ab640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f410d100 0x7f22f419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f99ab640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4112dd0 con 0x7f22f41045c0 2024-09-11T01:21:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f99ab640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4112c50 con 0x7f22f410d100 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f99ab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4112f50 con 0x7f22f4103ca0 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f41045c0 0x7f22f4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f41045c0 0x7f22f4199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47840/0 (socket says 172.21.15.78:47840) 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f27fc640 1 -- 172.21.15.78:0/2710655034 learned_addr learned my addr 172.21.15.78:0/2710655034 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f2ffd640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f4103ca0 0x7f22f4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f37fe640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f410d100 0x7f22f419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f27fc640 1 -- 172.21.15.78:0/2710655034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f4103ca0 msgr2=0x7f22f4199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f27fc640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f4103ca0 0x7f22f4199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f27fc640 1 -- 172.21.15.78:0/2710655034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f410d100 msgr2=0x7f22f419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f27fc640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f410d100 0x7f22f419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f27fc640 1 -- 172.21.15.78:0/2710655034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f419f2d0 con 0x7f22f41045c0 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f27fc640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f41045c0 0x7f22f4199760 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00b3c0 tx=0x7f22dc00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f89a9640 1 -- 172.21.15.78:0/2710655034 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc011830 con 0x7f22f41045c0 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f89a9640 1 -- 172.21.15.78:0/2710655034 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22dc016070 con 0x7f22f41045c0 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f99ab640 1 -- 172.21.15.78:0/2710655034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f419f5c0 con 0x7f22f41045c0 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f89a9640 1 -- 172.21.15.78:0/2710655034 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc010de0 con 0x7f22f41045c0 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.598+0000 7f22f99ab640 1 -- 172.21.15.78:0/2710655034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f41a5d00 con 0x7f22f41045c0 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22f89a9640 1 -- 172.21.15.78:0/2710655034 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f22dc0119d0 con 0x7f22f41045c0 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22fbd80740 1 Processor -- start 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22fbd80740 1 -- start start 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22fbd80740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff4d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22fbd80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4da10 0x564e1ff59460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22fbd80740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff5bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22fbd80740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564e1fec3ea0 con 0x564e1ff4d0f0 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22fbd80740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564e1feb32d0 con 0x564e1ff599a0 2024-09-11T01:21:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22fbd80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e1feb70e0 con 0x564e1ff4da10 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22f37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff4d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22f37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff4d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47854/0 (socket says 172.21.15.78:47854) 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22f37fe640 1 -- 172.21.15.78:0/2433225463 learned_addr learned my addr 172.21.15.78:0/2433225463 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22f2ffd640 1 --2- 172.21.15.78:0/2433225463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4da10 0x564e1ff59460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22f37fe640 1 -- 172.21.15.78:0/2433225463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4da10 msgr2=0x564e1ff59460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22f37fe640 1 --2- 172.21.15.78:0/2433225463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4da10 0x564e1ff59460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22f37fe640 1 -- 172.21.15.78:0/2433225463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 msgr2=0x564e1ff5bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.601+0000 7f22f37fe640 1 --2- 172.21.15.78:0/2433225463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff5bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.602+0000 7f22f37fe640 1 -- 172.21.15.78:0/2433225463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e1fc11650 con 0x564e1ff4d0f0 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.602+0000 7f22f2ffd640 1 --2- 172.21.15.78:0/2433225463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4da10 0x564e1ff59460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.602+0000 7f22f89a9640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f22e00b5450 0x7f22e00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.602+0000 7f22f37fe640 1 --2- 172.21.15.78:0/2433225463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff4d4d0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f22ec005cc0 tx=0x7f22ec009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad531a12205f7e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:01.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.602+0000 7f22f2ffd640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f22e00b5450 0x7f22e00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.602+0000 7f22f89a9640 1 -- 172.21.15.78:0/2710655034 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f22dc0e5670 con 0x7f22f41045c0 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.602+0000 7f22e9ffb640 1 -- 172.21.15.78:0/2433225463 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec031df0 con 0x564e1ff4d0f0 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.602+0000 7f22e9ffb640 1 -- 172.21.15.78:0/2433225463 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22ec036040 con 0x564e1ff4d0f0 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.602+0000 7f22e9ffb640 1 -- 172.21.15.78:0/2433225463 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0042a0 con 0x564e1ff4d0f0 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.603+0000 7f22fbd80740 1 -- 172.21.15.78:0/2433225463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 msgr2=0x564e1ff4d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.603+0000 7f22f2ffd640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f22e00b5450 0x7f22e00b7910 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f22e40058c0 tx=0x7f22e4005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.603+0000 7f22fbd80740 1 --2- 172.21.15.78:0/2433225463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff4d4d0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f22ec005cc0 tx=0x7f22ec009930 comp rx=0 tx=0).stop 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.603+0000 7f22fbd80740 1 -- 172.21.15.78:0/2433225463 shutdown_connections 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.603+0000 7f22fbd80740 1 --2- 172.21.15.78:0/2433225463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff5bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.603+0000 7f22fbd80740 1 --2- 172.21.15.78:0/2433225463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4da10 0x564e1ff59460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.603+0000 7f22fbd80740 1 --2- 172.21.15.78:0/2433225463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff4d4d0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.603+0000 7f22fbd80740 1 -- 172.21.15.78:0/2433225463 >> 172.21.15.78:0/2433225463 conn(0x564e1ff4af20 msgr2=0x564e1ff4c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.603+0000 7f22fbd80740 1 -- 172.21.15.78:0/2433225463 shutdown_connections 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.603+0000 7f22fbd80740 1 -- 172.21.15.78:0/2433225463 wait complete. 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.603+0000 7f22fbd80740 1 -- start start 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.603+0000 7f22fbd80740 1 Processor -- start 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22fbd80740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22fbd80740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff4da10 0x564e1ff56700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22fbd80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff599a0 0x564e1ff56d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22fbd80740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564e1fd14700 con 0x564e1ff4d0f0 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22fbd80740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564e1fe4e160 con 0x564e1ff4da10 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22fbd80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e1ff5c5c0 con 0x564e1ff599a0 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22fbd80740 10 client.0 _ll_register_callbacks cb 0x564e1fe679e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47868/0 (socket says 172.21.15.78:47868) 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22f2ffd640 1 -- 172.21.15.78:0/1186827310 learned_addr learned my addr 172.21.15.78:0/1186827310 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:01.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22f37fe640 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff599a0 0x564e1ff56d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22f27fc640 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff4da10 0x564e1ff56700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22f2ffd640 1 -- 172.21.15.78:0/1186827310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff599a0 msgr2=0x564e1ff56d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:01.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22f2ffd640 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff599a0 0x564e1ff56d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22f2ffd640 1 -- 172.21.15.78:0/1186827310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff4da10 msgr2=0x564e1ff56700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:01.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22f2ffd640 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff4da10 0x564e1ff56700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22f2ffd640 1 -- 172.21.15.78:0/1186827310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e1fe7bc90 con 0x564e1ff4d0f0 2024-09-11T01:21:01.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22f2ffd640 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff592b0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f22e4011aa0 tx=0x7f22e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3e10530c4ce844 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:01.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:21:01.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22d2ffd640 1 -- 172.21.15.78:0/1186827310 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e400ed90 con 0x564e1ff4d0f0 2024-09-11T01:21:01.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22d2ffd640 1 -- 172.21.15.78:0/1186827310 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f22e401e420 con 0x564e1ff4d0f0 2024-09-11T01:21:01.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22fbd80740 1 -- 172.21.15.78:0/1186827310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22ec036040 con 0x564e1ff4d0f0 2024-09-11T01:21:01.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.604+0000 7f22d2ffd640 1 -- 172.21.15.78:0/1186827310 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f22e401e620 con 0x564e1ff4d0f0 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22fbd80740 10 client.19227 fetch_fsmap learned FSMap version 72 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22fbd80740 1 -- 172.21.15.78:0/1186827310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x564e1ff67fb0 con 0x564e1ff4d0f0 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22d2ffd640 1 -- 172.21.15.78:0/1186827310 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22ec036040 con 0x564e1ff4d0f0 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22d2ffd640 1 -- 172.21.15.78:0/1186827310 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f22e402f880 con 0x564e1ff4d0f0 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22fbd80740 10 client.19227 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22fbd80740 10 client.19227 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22fbd80740 20 client.19227 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22fbd80740 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x564e1ff693c0 0x564e1ff6b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22fbd80740 4 client.19227 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22fbd80740 1 -- 172.21.15.78:0/1186827310 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e1fec5df0 con 0x564e1ff693c0 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22f27fc640 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x564e1ff693c0 0x564e1ff6b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22d2ffd640 10 client.19227 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.605+0000 7f22f27fc640 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x564e1ff693c0 0x564e1ff6b800 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c4668195b9f44fa9 server_cookie=85879ddafefd06d1 in_seq=0 out_seq=0 2024-09-11T01:21:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22d2ffd640 1 -- 172.21.15.78:0/1186827310 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564e1fec5df0 con 0x564e1ff693c0 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22d2ffd640 10 client.19227 handle_command_reply: tid=0 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22fbd80740 1 client.19227 shutdown 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22fbd80740 1 -- 172.21.15.78:0/1186827310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 msgr2=0x564e1ff592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22fbd80740 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff592b0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f22e4011aa0 tx=0x7f22e40046c0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22fbd80740 1 -- 172.21.15.78:0/1186827310 shutdown_connections 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22fbd80740 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x564e1ff693c0 0x564e1ff6b800 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22fbd80740 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff599a0 0x564e1ff56d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22fbd80740 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff4da10 0x564e1ff56700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22d2ffd640 0 client.19227 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22fbd80740 1 --2- 172.21.15.78:0/1186827310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff592b0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22fbd80740 1 -- 172.21.15.78:0/1186827310 >> 172.21.15.78:0/1186827310 conn(0x564e1ff4af20 msgr2=0x564e1ff50a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22fbd80740 1 -- 172.21.15.78:0/1186827310 shutdown_connections 2024-09-11T01:21:01.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.606+0000 7f22fbd80740 1 -- 172.21.15.78:0/1186827310 wait complete. 2024-09-11T01:21:01.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.607+0000 7f22fbd80740 20 client.19227 trim_cache size 0 max 16384 2024-09-11T01:21:01.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22fbd80740 1 Processor -- start 2024-09-11T01:21:01.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22fbd80740 1 -- start start 2024-09-11T01:21:01.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22fbd80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4d0f0 0x564e1ff65820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22fbd80740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4da10 0x564e1ff55bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22fbd80740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff54f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22fbd80740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41ae680 con 0x564e1ff4da10 2024-09-11T01:21:01.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22fbd80740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41b0280 con 0x564e1ff599a0 2024-09-11T01:21:01.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22fbd80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41aed50 con 0x564e1ff4d0f0 2024-09-11T01:21:01.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4da10 0x564e1ff55bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4da10 0x564e1ff55bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47878/0 (socket says 172.21.15.78:47878) 2024-09-11T01:21:01.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22f27fc640 1 -- 172.21.15.78:0/444119506 learned_addr learned my addr 172.21.15.78:0/444119506 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:01.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22f27fc640 1 -- 172.21.15.78:0/444119506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4d0f0 msgr2=0x564e1ff65820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:01.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22f27fc640 1 --2- 172.21.15.78:0/444119506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4d0f0 0x564e1ff65820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22f27fc640 1 -- 172.21.15.78:0/444119506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 msgr2=0x564e1ff54f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:01.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22f37fe640 1 --2- 172.21.15.78:0/444119506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff54f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22f27fc640 1 --2- 172.21.15.78:0/444119506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff54f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22f27fc640 1 -- 172.21.15.78:0/444119506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e1ff570a0 con 0x564e1ff4da10 2024-09-11T01:21:01.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22f37fe640 1 --2- 172.21.15.78:0/444119506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff54f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:21:01.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22f27fc640 1 --2- 172.21.15.78:0/444119506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4da10 0x564e1ff55bf0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0a8d00 tx=0x7f22dc093040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cbd398063247e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.620+0000 7f22e97fa640 1 -- 172.21.15.78:0/444119506 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0ac190 con 0x564e1ff4da10 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.621+0000 7f22e97fa640 1 -- 172.21.15.78:0/444119506 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22dc0aba90 con 0x564e1ff4da10 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.621+0000 7f22e97fa640 1 -- 172.21.15.78:0/444119506 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0a7d00 con 0x564e1ff4da10 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.621+0000 7f22fbd80740 1 -- 172.21.15.78:0/444119506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4da10 msgr2=0x564e1ff55bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.621+0000 7f22fbd80740 1 --2- 172.21.15.78:0/444119506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4da10 0x564e1ff55bf0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0a8d00 tx=0x7f22dc093040 comp rx=0 tx=0).stop 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.621+0000 7f22fbd80740 1 -- 172.21.15.78:0/444119506 shutdown_connections 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.621+0000 7f22fbd80740 1 --2- 172.21.15.78:0/444119506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff54f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.621+0000 7f22fbd80740 1 --2- 172.21.15.78:0/444119506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4da10 0x564e1ff55bf0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.621+0000 7f22fbd80740 1 --2- 172.21.15.78:0/444119506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4d0f0 0x564e1ff65820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.621+0000 7f22fbd80740 1 -- 172.21.15.78:0/444119506 >> 172.21.15.78:0/444119506 conn(0x564e1ff4af20 msgr2=0x564e1ff51950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.621+0000 7f22fbd80740 1 -- 172.21.15.78:0/444119506 shutdown_connections 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.621+0000 7f22fbd80740 1 -- 172.21.15.78:0/444119506 wait complete. 2024-09-11T01:21:01.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.621+0000 7f22fbd80740 1 -- start start 2024-09-11T01:21:01.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22fbd80740 1 Processor -- start 2024-09-11T01:21:01.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22fbd80740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff61bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22fbd80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4da10 0x564e1ff62150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22fbd80740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff5c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22fbd80740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41afbb0 con 0x564e1ff4d0f0 2024-09-11T01:21:01.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22fbd80740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41af420 con 0x564e1ff599a0 2024-09-11T01:21:01.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22fbd80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41ac680 con 0x564e1ff4da10 2024-09-11T01:21:01.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22fbd80740 10 client.0 _ll_register_callbacks cb 0x7f22e400c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:01.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff61bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff61bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47890/0 (socket says 172.21.15.78:47890) 2024-09-11T01:21:01.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22f2ffd640 1 -- 172.21.15.78:0/205887608 learned_addr learned my addr 172.21.15.78:0/205887608 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:01.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22f37fe640 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff5c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22f2ffd640 1 -- 172.21.15.78:0/205887608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4da10 msgr2=0x564e1ff62150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22f2ffd640 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4da10 0x564e1ff62150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22f2ffd640 1 -- 172.21.15.78:0/205887608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 msgr2=0x564e1ff5c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22f2ffd640 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff5c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22f2ffd640 1 -- 172.21.15.78:0/205887608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e1ff5d160 con 0x564e1ff4d0f0 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22f37fe640 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff5c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22f2ffd640 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff61bd0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f22e40114c0 tx=0x7f22e4026690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be19f144b9f8dd22 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.622+0000 7f22d2ffd640 1 -- 172.21.15.78:0/205887608 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e403e060 con 0x564e1ff4d0f0 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22d2ffd640 1 -- 172.21.15.78:0/205887608 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f22e401e420 con 0x564e1ff4d0f0 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22fbd80740 1 -- 172.21.15.78:0/205887608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22dc0aba90 con 0x564e1ff4d0f0 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22d2ffd640 1 -- 172.21.15.78:0/205887608 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f22e4035040 con 0x564e1ff4d0f0 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22fbd80740 10 client.19233 fetch_fsmap learned FSMap version 72 2024-09-11T01:21:01.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22fbd80740 1 -- 172.21.15.78:0/205887608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x564e1ffd6f00 con 0x564e1ff4d0f0 2024-09-11T01:21:01.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22d2ffd640 1 -- 172.21.15.78:0/205887608 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22dc0aba90 con 0x564e1ff4d0f0 2024-09-11T01:21:01.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22d2ffd640 1 -- 172.21.15.78:0/205887608 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f22e401e850 con 0x564e1ff4d0f0 2024-09-11T01:21:01.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22fbd80740 10 client.19233 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:21:01.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22fbd80740 10 client.19233 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:21:01.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22fbd80740 20 client.19233 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:01.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22fbd80740 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x564e1ffd7680 0x564e1ffd9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:01.921 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:21:01.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22fbd80740 4 client.19233 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:21:01.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22fbd80740 1 -- 172.21.15.78:0/205887608 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e1fec18a0 con 0x564e1ffd7680 2024-09-11T01:21:01.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.623+0000 7f22f27fc640 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x564e1ffd7680 0x564e1ffd9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:01.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22f27fc640 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x564e1ffd7680 0x564e1ffd9aa0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f942099d580631d7 server_cookie=bcf922309c85c432 in_seq=0 out_seq=0 2024-09-11T01:21:01.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22d2ffd640 10 client.19233 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:21:01.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22d2ffd640 1 -- 172.21.15.78:0/205887608 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e1fec18a0 con 0x564e1ffd7680 2024-09-11T01:21:01.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22d2ffd640 10 client.19233 handle_command_reply: tid=0 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22fbd80740 1 client.19233 shutdown 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22fbd80740 1 -- 172.21.15.78:0/205887608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 msgr2=0x564e1ff61bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22fbd80740 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff61bd0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f22e40114c0 tx=0x7f22e4026690 comp rx=0 tx=0).stop 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22fbd80740 1 -- 172.21.15.78:0/205887608 shutdown_connections 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22fbd80740 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x564e1ffd7680 0x564e1ffd9aa0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22fbd80740 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e1ff599a0 0x564e1ff5c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22fbd80740 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e1ff4da10 0x564e1ff62150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22fbd80740 1 --2- 172.21.15.78:0/205887608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564e1ff4d0f0 0x564e1ff61bd0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22d2ffd640 0 client.19233 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22fbd80740 1 -- 172.21.15.78:0/205887608 >> 172.21.15.78:0/205887608 conn(0x564e1ff4af20 msgr2=0x564e1ff54ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22fbd80740 1 -- 172.21.15.78:0/205887608 shutdown_connections 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22fbd80740 1 -- 172.21.15.78:0/205887608 wait complete. 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.624+0000 7f22fbd80740 20 client.19233 trim_cache size 0 max 16384 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.625+0000 7f22f99ab640 1 -- 172.21.15.78:0/2710655034 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f22e00b5450 msgr2=0x7f22e00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:01.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.625+0000 7f22f99ab640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f22e00b5450 0x7f22e00b7910 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f22e40058c0 tx=0x7f22e4005850 comp rx=0 tx=0).stop 2024-09-11T01:21:01.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.625+0000 7f22f99ab640 1 -- 172.21.15.78:0/2710655034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f41045c0 msgr2=0x7f22f4199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:01.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.625+0000 7f22f99ab640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f41045c0 0x7f22f4199760 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00b3c0 tx=0x7f22dc00b880 comp rx=0 tx=0).stop 2024-09-11T01:21:01.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.625+0000 7f22f99ab640 1 -- 172.21.15.78:0/2710655034 shutdown_connections 2024-09-11T01:21:01.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.625+0000 7f22f99ab640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f22e00b5450 0x7f22e00b7910 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.625+0000 7f22f99ab640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f410d100 0x7f22f419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.625+0000 7f22f99ab640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f41045c0 0x7f22f4199760 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.625+0000 7f22f99ab640 1 --2- 172.21.15.78:0/2710655034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22f4103ca0 0x7f22f4199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:01.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.625+0000 7f22f99ab640 1 -- 172.21.15.78:0/2710655034 >> 172.21.15.78:0/2710655034 conn(0x7f22f40fd700 msgr2=0x7f22f410f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:01.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.625+0000 7f22f99ab640 1 -- 172.21.15.78:0/2710655034 shutdown_connections 2024-09-11T01:21:01.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:00.625+0000 7f22f99ab640 1 -- 172.21.15.78:0/2710655034 wait complete. 2024-09-11T01:21:02.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:01 smithi078 ceph-mon[31315]: pgmap v1017: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 702 op/s 2024-09-11T01:21:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:02 smithi110 ceph-mon[34712]: pgmap v1017: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 702 op/s 2024-09-11T01:21:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:02 smithi112 ceph-mon[35384]: pgmap v1017: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 702 op/s 2024-09-11T01:21:02.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 339045 32.58 MB/sec execute 428 sec latency 1976.851 ms 2024-09-11T01:21:03.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 339779 32.62 MB/sec execute 429 sec latency 156.584 ms 2024-09-11T01:21:03.916 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:21:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:04 smithi078 ceph-mon[31315]: pgmap v1018: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 18 MiB/s wr, 720 op/s 2024-09-11T01:21:04.281 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:21:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:04 smithi110 ceph-mon[34712]: pgmap v1018: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 18 MiB/s wr, 720 op/s 2024-09-11T01:21:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:04 smithi112 ceph-mon[35384]: pgmap v1018: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 18 MiB/s wr, 720 op/s 2024-09-11T01:21:04.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 340247 32.60 MB/sec execute 430 sec latency 174.682 ms 2024-09-11T01:21:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.964+0000 7f95c471b640 1 -- 172.21.15.78:0/929289763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95bc101c90 msgr2=0x7f95bc102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.964+0000 7f95c471b640 1 --2- 172.21.15.78:0/929289763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95bc101c90 0x7f95bc102070 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009900 tx=0x7f95b002a440 comp rx=0 tx=0).stop 2024-09-11T01:21:04.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.969+0000 7f95c471b640 1 -- 172.21.15.78:0/929289763 shutdown_connections 2024-09-11T01:21:04.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.969+0000 7f95c471b640 1 --2- 172.21.15.78:0/929289763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95bc108020 0x7f95bc10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:04.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.969+0000 7f95c471b640 1 --2- 172.21.15.78:0/929289763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95bc1076a0 0x7f95bc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:04.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.969+0000 7f95c471b640 1 --2- 172.21.15.78:0/929289763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95bc101c90 0x7f95bc102070 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:04.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.969+0000 7f95c471b640 1 -- 172.21.15.78:0/929289763 >> 172.21.15.78:0/929289763 conn(0x7f95bc0fd700 msgr2=0x7f95bc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:04.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.975+0000 7f95c471b640 1 -- 172.21.15.78:0/929289763 shutdown_connections 2024-09-11T01:21:04.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.976+0000 7f95c471b640 1 -- 172.21.15.78:0/929289763 wait complete. 2024-09-11T01:21:04.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.977+0000 7f95c471b640 1 Processor -- start 2024-09-11T01:21:04.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.979+0000 7f95c471b640 1 -- start start 2024-09-11T01:21:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.979+0000 7f95c471b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95bc101c90 0x7f95bc197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.979+0000 7f95c471b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95bc1076a0 0x7f95bc197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.980+0000 7f95c471b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95bc108020 0x7f95bc19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.980+0000 7f95c471b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc110c20 con 0x7f95bc101c90 2024-09-11T01:21:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.980+0000 7f95c471b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc110aa0 con 0x7f95bc1076a0 2024-09-11T01:21:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.980+0000 7f95c471b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc110da0 con 0x7f95bc108020 2024-09-11T01:21:04.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.981+0000 7f95c1cb6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95bc1076a0 0x7f95bc197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:04.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.981+0000 7f95c1cb6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95bc1076a0 0x7f95bc197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:54912/0 (socket says 172.21.15.78:54912) 2024-09-11T01:21:04.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.981+0000 7f95c1cb6640 1 -- 172.21.15.78:0/1110990147 learned_addr learned my addr 172.21.15.78:0/1110990147 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.982+0000 7f95c2cb8640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95bc108020 0x7f95bc19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:04.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.985+0000 7f95c24b7640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95bc101c90 0x7f95bc197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:04.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.986+0000 7f95c1cb6640 1 -- 172.21.15.78:0/1110990147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95bc108020 msgr2=0x7f95bc19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:04.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.986+0000 7f95c1cb6640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95bc108020 0x7f95bc19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:04.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.986+0000 7f95c1cb6640 1 -- 172.21.15.78:0/1110990147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95bc101c90 msgr2=0x7f95bc197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:04.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.986+0000 7f95c1cb6640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95bc101c90 0x7f95bc197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:04.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.986+0000 7f95c1cb6640 1 -- 172.21.15.78:0/1110990147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc19d1c0 con 0x7f95bc1076a0 2024-09-11T01:21:04.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.991+0000 7f95c1cb6640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95bc1076a0 0x7f95bc197650 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f95b800b450 tx=0x7f95b800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:04.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.993+0000 7f95af7fe640 1 -- 172.21.15.78:0/1110990147 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8015020 con 0x7f95bc1076a0 2024-09-11T01:21:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.993+0000 7f95af7fe640 1 -- 172.21.15.78:0/1110990147 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b8011d40 con 0x7f95bc1076a0 2024-09-11T01:21:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.993+0000 7f95c471b640 1 -- 172.21.15.78:0/1110990147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95bc19d4b0 con 0x7f95bc1076a0 2024-09-11T01:21:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.993+0000 7f95c471b640 1 -- 172.21.15.78:0/1110990147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95bc1a3bf0 con 0x7f95bc1076a0 2024-09-11T01:21:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.993+0000 7f95af7fe640 1 -- 172.21.15.78:0/1110990147 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b800f340 con 0x7f95bc1076a0 2024-09-11T01:21:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.998+0000 7f95c6af0740 1 Processor -- start 2024-09-11T01:21:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:04.999+0000 7f95c6af0740 1 -- start start 2024-09-11T01:21:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.000+0000 7f95c6af0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41e920 0x562abc41ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.000+0000 7f95c6af0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41f240 0x562abc42ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.000+0000 7f95c6af0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc42b1d0 0x562abc42d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.000+0000 7f95af7fe640 1 -- 172.21.15.78:0/1110990147 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f95b800f4e0 con 0x7f95bc1076a0 2024-09-11T01:21:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.000+0000 7f95c6af0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562abc38cea0 con 0x562abc41e920 2024-09-11T01:21:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.000+0000 7f95c6af0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562abc37c2d0 con 0x562abc41f240 2024-09-11T01:21:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.000+0000 7f95c6af0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562abc3800e0 con 0x562abc42b1d0 2024-09-11T01:21:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.000+0000 7f95c1cb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc42b1d0 0x562abc42d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.000+0000 7f95c1cb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc42b1d0 0x562abc42d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:45728/0 (socket says 172.21.15.78:45728) 2024-09-11T01:21:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.000+0000 7f95c1cb6640 1 -- 172.21.15.78:0/3570465291 learned_addr learned my addr 172.21.15.78:0/3570465291 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.001+0000 7f95af7fe640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f95a80b5650 0x7f95a80b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.002+0000 7f95c24b7640 1 --2- 172.21.15.78:0/3570465291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41f240 0x562abc42ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.002+0000 7f95c2cb8640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f95a80b5650 0x7f95a80b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.002+0000 7f95af7fe640 1 -- 172.21.15.78:0/1110990147 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f95b80e5c10 con 0x7f95bc1076a0 2024-09-11T01:21:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.002+0000 7f95c2cb8640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f95a80b5650 0x7f95a80b7b10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f95b400a780 tx=0x7f95b4005f60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.003+0000 7f95c2cb8640 1 --2- 172.21.15.78:0/3570465291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41e920 0x562abc41ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.003+0000 7f95c1cb6640 1 -- 172.21.15.78:0/3570465291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41f240 msgr2=0x562abc42ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.003+0000 7f95c1cb6640 1 --2- 172.21.15.78:0/3570465291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41f240 0x562abc42ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.003+0000 7f95c1cb6640 1 -- 172.21.15.78:0/3570465291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41e920 msgr2=0x562abc41ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.003+0000 7f95c1cb6640 1 --2- 172.21.15.78:0/3570465291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41e920 0x562abc41ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.003+0000 7f95c1cb6640 1 -- 172.21.15.78:0/3570465291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abc0da650 con 0x562abc42b1d0 2024-09-11T01:21:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.003+0000 7f95c2cb8640 1 --2- 172.21.15.78:0/3570465291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41e920 0x562abc41ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:21:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.004+0000 7f95c1cb6640 1 --2- 172.21.15.78:0/3570465291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc42b1d0 0x562abc42d5c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f95b80e32b0 tx=0x7f95b80108d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26392d8cced6ba9e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.004+0000 7f95acff9640 1 -- 172.21.15.78:0/3570465291 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b80abe40 con 0x562abc42b1d0 2024-09-11T01:21:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.004+0000 7f95acff9640 1 -- 172.21.15.78:0/3570465291 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b80ea040 con 0x562abc42b1d0 2024-09-11T01:21:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.004+0000 7f95acff9640 1 -- 172.21.15.78:0/3570465291 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b80ab7e0 con 0x562abc42b1d0 2024-09-11T01:21:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.005+0000 7f95c6af0740 1 -- 172.21.15.78:0/3570465291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc42b1d0 msgr2=0x562abc42d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.005+0000 7f95c6af0740 1 --2- 172.21.15.78:0/3570465291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc42b1d0 0x562abc42d5c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f95b80e32b0 tx=0x7f95b80108d0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.006+0000 7f95c6af0740 1 -- 172.21.15.78:0/3570465291 shutdown_connections 2024-09-11T01:21:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.006+0000 7f95c6af0740 1 --2- 172.21.15.78:0/3570465291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc42b1d0 0x562abc42d5c0 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f95b80e32b0 tx=0x7f95b80108d0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.006+0000 7f95c6af0740 1 --2- 172.21.15.78:0/3570465291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41f240 0x562abc42ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.006+0000 7f95c6af0740 1 --2- 172.21.15.78:0/3570465291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41e920 0x562abc41ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.006+0000 7f95c6af0740 1 -- 172.21.15.78:0/3570465291 >> 172.21.15.78:0/3570465291 conn(0x562abc41c750 msgr2=0x562abc41d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.008+0000 7f95c6af0740 1 -- 172.21.15.78:0/3570465291 shutdown_connections 2024-09-11T01:21:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.009+0000 7f95c6af0740 1 -- 172.21.15.78:0/3570465291 wait complete. 2024-09-11T01:21:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.010+0000 7f95c6af0740 1 -- start start 2024-09-11T01:21:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.010+0000 7f95c6af0740 1 Processor -- start 2024-09-11T01:21:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.011+0000 7f95c6af0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41e920 0x562abc419350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.011+0000 7f95c6af0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41f240 0x562abc419890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.011+0000 7f95c6af0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc419dd0 0x562abc429c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.011+0000 7f95c6af0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562abc1dd700 con 0x562abc41f240 2024-09-11T01:21:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.012+0000 7f95c6af0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562abc317160 con 0x562abc419dd0 2024-09-11T01:21:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.012+0000 7f95c6af0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562abc413a90 con 0x562abc41e920 2024-09-11T01:21:05.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.013+0000 7f95c24b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41e920 0x562abc419350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.013+0000 7f95c2cb8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc419dd0 0x562abc429c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.013+0000 7f95c24b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41e920 0x562abc419350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:45734/0 (socket says 172.21.15.78:45734) 2024-09-11T01:21:05.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.013+0000 7f95c24b7640 1 -- 172.21.15.78:0/2846322300 learned_addr learned my addr 172.21.15.78:0/2846322300 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:05.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.013+0000 7f95c6af0740 10 client.0 _ll_register_callbacks cb 0x562abc3309e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:05.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.015+0000 7f95c24b7640 1 -- 172.21.15.78:0/2846322300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc419dd0 msgr2=0x562abc429c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.015+0000 7f95c24b7640 1 --2- 172.21.15.78:0/2846322300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc419dd0 0x562abc429c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.015+0000 7f95c24b7640 1 -- 172.21.15.78:0/2846322300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41f240 msgr2=0x562abc419890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.015+0000 7f95c24b7640 1 --2- 172.21.15.78:0/2846322300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41f240 0x562abc419890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.015+0000 7f95c24b7640 1 -- 172.21.15.78:0/2846322300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abc344c90 con 0x562abc41e920 2024-09-11T01:21:05.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.024+0000 7f95c24b7640 1 --2- 172.21.15.78:0/2846322300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41e920 0x562abc419350 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f95b002a9b0 tx=0x7f95b0003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4df14b94816e2cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:05.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.024+0000 7f95a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:21:05.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.025+0000 7f95a5ffb640 1 -- 172.21.15.78:0/2846322300 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0035070 con 0x562abc41e920 2024-09-11T01:21:05.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.025+0000 7f95a5ffb640 1 -- 172.21.15.78:0/2846322300 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f95b0003b90 con 0x562abc41e920 2024-09-11T01:21:05.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.025+0000 7f95c6af0740 1 -- 172.21.15.78:0/2846322300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95b80ea040 con 0x562abc41e920 2024-09-11T01:21:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.030+0000 7f95a5ffb640 1 -- 172.21.15.78:0/2846322300 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f95b0003d90 con 0x562abc41e920 2024-09-11T01:21:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.031+0000 7f95c6af0740 10 client.28225 fetch_fsmap learned FSMap version 72 2024-09-11T01:21:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.031+0000 7f95c6af0740 1 -- 172.21.15.78:0/2846322300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x562abc442510 con 0x562abc41e920 2024-09-11T01:21:05.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.036+0000 7f95a5ffb640 1 -- 172.21.15.78:0/2846322300 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b80ea040 con 0x562abc41e920 2024-09-11T01:21:05.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.036+0000 7f95a5ffb640 1 -- 172.21.15.78:0/2846322300 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f95b0044580 con 0x562abc41e920 2024-09-11T01:21:05.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.036+0000 7f95c6af0740 10 client.28225 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:21:05.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.036+0000 7f95c6af0740 10 client.28225 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:21:05.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.036+0000 7f95c6af0740 20 client.28225 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:05.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.037+0000 7f95c6af0740 1 --2- 172.21.15.78:0/2846322300 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562abc4439d0 0x562abc445dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.037+0000 7f95c6af0740 4 client.28225 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:21:05.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.037+0000 7f95c6af0740 1 -- 172.21.15.78:0/2846322300 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562abc38edf0 con 0x562abc4439d0 2024-09-11T01:21:05.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.037+0000 7f95c1cb6640 1 --2- 172.21.15.78:0/2846322300 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562abc4439d0 0x562abc445dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.038+0000 7f95c1cb6640 1 --2- 172.21.15.78:0/2846322300 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562abc4439d0 0x562abc445dc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e1aea73d9c37ed7 server_cookie=f0fca2d4f05dd9e6 in_seq=0 out_seq=0 2024-09-11T01:21:05.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.040+0000 7f95a5ffb640 10 client.28225 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:21:05.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.040+0000 7f95a5ffb640 1 -- 172.21.15.78:0/2846322300 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562abc38edf0 con 0x562abc4439d0 2024-09-11T01:21:05.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.040+0000 7f95a5ffb640 10 client.28225 handle_command_reply: tid=0 2024-09-11T01:21:05.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.040+0000 7f95c6af0740 1 client.28225 shutdown 2024-09-11T01:21:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.042+0000 7f95c6af0740 1 -- 172.21.15.78:0/2846322300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41e920 msgr2=0x562abc419350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:05.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.042+0000 7f95c6af0740 1 --2- 172.21.15.78:0/2846322300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41e920 0x562abc419350 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f95b002a9b0 tx=0x7f95b0003820 comp rx=0 tx=0).stop 2024-09-11T01:21:05.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.045+0000 7f95c6af0740 1 -- 172.21.15.78:0/2846322300 shutdown_connections 2024-09-11T01:21:05.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.045+0000 7f95c6af0740 1 --2- 172.21.15.78:0/2846322300 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562abc4439d0 0x562abc445dc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.045+0000 7f95c6af0740 1 --2- 172.21.15.78:0/2846322300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc419dd0 0x562abc429c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.045+0000 7f95c6af0740 1 --2- 172.21.15.78:0/2846322300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41f240 0x562abc419890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.045+0000 7f95c6af0740 1 --2- 172.21.15.78:0/2846322300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41e920 0x562abc419350 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.045+0000 7f95a5ffb640 0 client.28225 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:21:05.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.045+0000 7f95c6af0740 1 -- 172.21.15.78:0/2846322300 >> 172.21.15.78:0/2846322300 conn(0x562abc41c750 msgr2=0x562abc4130c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:05.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.045+0000 7f95c6af0740 1 -- 172.21.15.78:0/2846322300 shutdown_connections 2024-09-11T01:21:05.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.045+0000 7f95c6af0740 1 -- 172.21.15.78:0/2846322300 wait complete. 2024-09-11T01:21:05.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.045+0000 7f95c6af0740 20 client.28225 trim_cache size 0 max 16384 2024-09-11T01:21:05.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.072+0000 7f95c6af0740 1 Processor -- start 2024-09-11T01:21:05.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.073+0000 7f95c6af0740 1 -- start start 2024-09-11T01:21:05.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.073+0000 7f95c6af0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc419dd0 0x562abc419810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.073+0000 7f95c6af0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41e920 0x562abc44b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.073+0000 7f95c6af0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41f240 0x562abc44bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.073+0000 7f95c6af0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1ac630 con 0x562abc41f240 2024-09-11T01:21:05.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.073+0000 7f95c6af0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1ae230 con 0x562abc41e920 2024-09-11T01:21:05.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.073+0000 7f95c6af0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1acd00 con 0x562abc419dd0 2024-09-11T01:21:05.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.077+0000 7f95c1cb6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41e920 0x562abc44b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.077+0000 7f95c2cb8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41f240 0x562abc44bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.077+0000 7f95c2cb8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41f240 0x562abc44bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49636/0 (socket says 172.21.15.78:49636) 2024-09-11T01:21:05.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.077+0000 7f95c2cb8640 1 -- 172.21.15.78:0/2734113417 learned_addr learned my addr 172.21.15.78:0/2734113417 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:05.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.077+0000 7f95c24b7640 1 --2- 172.21.15.78:0/2734113417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc419dd0 0x562abc419810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.079+0000 7f95c24b7640 1 -- 172.21.15.78:0/2734113417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41e920 msgr2=0x562abc44b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:05.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.079+0000 7f95c24b7640 1 --2- 172.21.15.78:0/2734113417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41e920 0x562abc44b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.079+0000 7f95c24b7640 1 -- 172.21.15.78:0/2734113417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41f240 msgr2=0x562abc44bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:05.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.079+0000 7f95c24b7640 1 --2- 172.21.15.78:0/2734113417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41f240 0x562abc44bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.079+0000 7f95c24b7640 1 -- 172.21.15.78:0/2734113417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abc41de30 con 0x562abc419dd0 2024-09-11T01:21:05.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.079+0000 7f95c24b7640 1 --2- 172.21.15.78:0/2734113417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc419dd0 0x562abc419810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f95b00449d0 tx=0x7f95b0001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb792b9e125c3958 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:05.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.080+0000 7f95a7fff640 1 -- 172.21.15.78:0/2734113417 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0051030 con 0x562abc419dd0 2024-09-11T01:21:05.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.080+0000 7f95a7fff640 1 -- 172.21.15.78:0/2734113417 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b0053070 con 0x562abc419dd0 2024-09-11T01:21:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.081+0000 7f95c6af0740 1 -- 172.21.15.78:0/2734113417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc419dd0 msgr2=0x562abc419810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.081+0000 7f95c6af0740 1 --2- 172.21.15.78:0/2734113417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc419dd0 0x562abc419810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f95b00449d0 tx=0x7f95b0001e50 comp rx=0 tx=0).stop 2024-09-11T01:21:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.082+0000 7f95c6af0740 1 -- 172.21.15.78:0/2734113417 shutdown_connections 2024-09-11T01:21:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.082+0000 7f95c6af0740 1 --2- 172.21.15.78:0/2734113417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc41f240 0x562abc44bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.082+0000 7f95c6af0740 1 --2- 172.21.15.78:0/2734113417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41e920 0x562abc44b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.082+0000 7f95c6af0740 1 --2- 172.21.15.78:0/2734113417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc419dd0 0x562abc419810 secure :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f95b00449d0 tx=0x7f95b0001e50 comp rx=0 tx=0).stop 2024-09-11T01:21:05.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.082+0000 7f95c6af0740 1 -- 172.21.15.78:0/2734113417 >> 172.21.15.78:0/2734113417 conn(0x562abc41c750 msgr2=0x562abc419100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:05.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.084+0000 7f95c6af0740 1 -- 172.21.15.78:0/2734113417 shutdown_connections 2024-09-11T01:21:05.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.084+0000 7f95c6af0740 1 -- 172.21.15.78:0/2734113417 wait complete. 2024-09-11T01:21:05.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.085+0000 7f95c6af0740 1 -- start start 2024-09-11T01:21:05.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.085+0000 7f95c6af0740 1 Processor -- start 2024-09-11T01:21:05.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.085+0000 7f95c6af0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41e920 0x562abc429680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.085+0000 7f95c6af0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41f240 0x562abc425b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.085+0000 7f95c6af0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc426140 0x562abc4265a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.085+0000 7f95c6af0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1adb60 con 0x562abc426140 2024-09-11T01:21:05.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.085+0000 7f95c6af0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1ad3d0 con 0x562abc41e920 2024-09-11T01:21:05.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.085+0000 7f95c6af0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1aa630 con 0x562abc41f240 2024-09-11T01:21:05.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.085+0000 7f95c6af0740 10 client.0 _ll_register_callbacks cb 0x7f95b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:05.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.086+0000 7f95c1cb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41f240 0x562abc425b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.086+0000 7f95c1cb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41f240 0x562abc425b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:45756/0 (socket says 172.21.15.78:45756) 2024-09-11T01:21:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.086+0000 7f95c1cb6640 1 -- 172.21.15.78:0/650224490 learned_addr learned my addr 172.21.15.78:0/650224490 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.087+0000 7f95c24b7640 1 --2- 172.21.15.78:0/650224490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41e920 0x562abc429680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.087+0000 7f95c1cb6640 1 -- 172.21.15.78:0/650224490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41e920 msgr2=0x562abc429680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.087+0000 7f95c1cb6640 1 --2- 172.21.15.78:0/650224490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41e920 0x562abc429680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.087+0000 7f95c1cb6640 1 -- 172.21.15.78:0/650224490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc426140 msgr2=0x562abc4265a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.087+0000 7f95c1cb6640 1 --2- 172.21.15.78:0/650224490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc426140 0x562abc4265a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.087+0000 7f95c1cb6640 1 -- 172.21.15.78:0/650224490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abc446130 con 0x562abc41f240 2024-09-11T01:21:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.088+0000 7f95c1cb6640 1 --2- 172.21.15.78:0/650224490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41f240 0x562abc425b10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f95b80b50d0 tx=0x7f95b8010710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=609b6fca772459ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.089+0000 7f95a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:21:05.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.092+0000 7f95a5ffb640 1 -- 172.21.15.78:0/650224490 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b80ab9c0 con 0x562abc41f240 2024-09-11T01:21:05.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.093+0000 7f95c6af0740 1 -- 172.21.15.78:0/650224490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95b0053070 con 0x562abc41f240 2024-09-11T01:21:05.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.093+0000 7f95a5ffb640 1 -- 172.21.15.78:0/650224490 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8449+0+0 (secure 0 0 0) 0x7f95b8011890 con 0x562abc41f240 2024-09-11T01:21:05.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.095+0000 7f95a5ffb640 1 -- 172.21.15.78:0/650224490 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f95b80ab7e0 con 0x562abc41f240 2024-09-11T01:21:05.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.095+0000 7f95c6af0740 10 client.28231 fetch_fsmap learned FSMap version 72 2024-09-11T01:21:05.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.095+0000 7f95c6af0740 1 -- 172.21.15.78:0/650224490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x562abc4b1cd0 con 0x562abc41f240 2024-09-11T01:21:05.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.097+0000 7f95a5ffb640 1 -- 172.21.15.78:0/650224490 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b0053070 con 0x562abc41f240 2024-09-11T01:21:05.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.097+0000 7f95a5ffb640 1 -- 172.21.15.78:0/650224490 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f95b80a79c0 con 0x562abc41f240 2024-09-11T01:21:05.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.097+0000 7f95c6af0740 10 client.28231 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:21:05.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.097+0000 7f95c6af0740 10 client.28231 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:21:05.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.097+0000 7f95c6af0740 20 client.28231 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:05.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.097+0000 7f95c6af0740 1 --2- 172.21.15.78:0/650224490 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562abc4b23e0 0x562abc4b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:05.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.097+0000 7f95c6af0740 4 client.28231 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:21:05.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.097+0000 7f95c6af0740 1 -- 172.21.15.78:0/650224490 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562abc38a8a0 con 0x562abc4b23e0 2024-09-11T01:21:05.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.098+0000 7f95c24b7640 1 --2- 172.21.15.78:0/650224490 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562abc4b23e0 0x562abc4b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:05.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.101+0000 7f95c24b7640 1 --2- 172.21.15.78:0/650224490 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562abc4b23e0 0x562abc4b4800 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bade4e3046483864 server_cookie=13d9a1f81acb0fb3 in_seq=0 out_seq=0 2024-09-11T01:21:05.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.101+0000 7f95a5ffb640 10 client.28231 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:21:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.108+0000 7f95a5ffb640 1 -- 172.21.15.78:0/650224490 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562abc38a8a0 con 0x562abc4b23e0 2024-09-11T01:21:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.108+0000 7f95a5ffb640 10 client.28231 handle_command_reply: tid=0 2024-09-11T01:21:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.108+0000 7f95c6af0740 1 client.28231 shutdown 2024-09-11T01:21:05.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.111+0000 7f95c6af0740 1 -- 172.21.15.78:0/650224490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41f240 msgr2=0x562abc425b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:05.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.111+0000 7f95c6af0740 1 --2- 172.21.15.78:0/650224490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41f240 0x562abc425b10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f95b80b50d0 tx=0x7f95b8010710 comp rx=0 tx=0).stop 2024-09-11T01:21:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.112+0000 7f95c6af0740 1 -- 172.21.15.78:0/650224490 shutdown_connections 2024-09-11T01:21:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.112+0000 7f95c6af0740 1 --2- 172.21.15.78:0/650224490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562abc426140 0x562abc4265a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.112+0000 7f95c6af0740 1 --2- 172.21.15.78:0/650224490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562abc41f240 0x562abc425b10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.112+0000 7f95c6af0740 1 --2- 172.21.15.78:0/650224490 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x562abc4b23e0 0x562abc4b4800 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.112+0000 7f95c6af0740 1 --2- 172.21.15.78:0/650224490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562abc41e920 0x562abc429680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.112+0000 7f95c6af0740 1 -- 172.21.15.78:0/650224490 >> 172.21.15.78:0/650224490 conn(0x562abc41c750 msgr2=0x562abc421980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.113+0000 7f95a5ffb640 0 client.28231 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:21:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.114+0000 7f95c6af0740 1 -- 172.21.15.78:0/650224490 shutdown_connections 2024-09-11T01:21:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.116+0000 7f95c6af0740 1 -- 172.21.15.78:0/650224490 wait complete. 2024-09-11T01:21:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.116+0000 7f95c6af0740 20 client.28231 trim_cache size 0 max 16384 2024-09-11T01:21:05.123 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:21:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.122+0000 7f95c471b640 1 -- 172.21.15.78:0/1110990147 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f95a80b5650 msgr2=0x7f95a80b7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.122+0000 7f95c471b640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f95a80b5650 0x7f95a80b7b10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f95b400a780 tx=0x7f95b4005f60 comp rx=0 tx=0).stop 2024-09-11T01:21:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.122+0000 7f95c471b640 1 -- 172.21.15.78:0/1110990147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95bc1076a0 msgr2=0x7f95bc197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.122+0000 7f95c471b640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95bc1076a0 0x7f95bc197650 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f95b800b450 tx=0x7f95b800b910 comp rx=0 tx=0).stop 2024-09-11T01:21:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.122+0000 7f95c471b640 1 -- 172.21.15.78:0/1110990147 shutdown_connections 2024-09-11T01:21:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.122+0000 7f95c471b640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f95a80b5650 0x7f95a80b7b10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.122+0000 7f95c471b640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95bc108020 0x7f95bc19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.122+0000 7f95c471b640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95bc1076a0 0x7f95bc197650 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.122+0000 7f95c471b640 1 --2- 172.21.15.78:0/1110990147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95bc101c90 0x7f95bc197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.122+0000 7f95c471b640 1 -- 172.21.15.78:0/1110990147 >> 172.21.15.78:0/1110990147 conn(0x7f95bc0fd700 msgr2=0x7f95bc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:05.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.122+0000 7f95c471b640 1 -- 172.21.15.78:0/1110990147 shutdown_connections 2024-09-11T01:21:05.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:05.122+0000 7f95c471b640 1 -- 172.21.15.78:0/1110990147 wait complete. 2024-09-11T01:21:05.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 340623 32.58 MB/sec execute 431 sec latency 302.009 ms 2024-09-11T01:21:06.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:06 smithi110 ceph-mon[34712]: pgmap v1019: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 680 op/s 2024-09-11T01:21:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:06 smithi112 ceph-mon[35384]: pgmap v1019: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 680 op/s 2024-09-11T01:21:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:06 smithi078 ceph-mon[31315]: pgmap v1019: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 680 op/s 2024-09-11T01:21:06.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 341166 32.59 MB/sec execute 432 sec latency 184.275 ms 2024-09-11T01:21:07.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 341686 32.55 MB/sec execute 433 sec latency 127.477 ms 2024-09-11T01:21:08.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:08 smithi110 ceph-mon[34712]: pgmap v1020: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 743 op/s 2024-09-11T01:21:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:08 smithi112 ceph-mon[35384]: pgmap v1020: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 743 op/s 2024-09-11T01:21:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:08 smithi078 ceph-mon[31315]: pgmap v1020: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 743 op/s 2024-09-11T01:21:08.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 342198 32.54 MB/sec execute 434 sec latency 182.274 ms 2024-09-11T01:21:08.954 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:21:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:09 smithi078 ceph-mon[31315]: osdmap e108: 12 total, 12 up, 12 in 2024-09-11T01:21:09.338 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:21:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:09 smithi110 ceph-mon[34712]: osdmap e108: 12 total, 12 up, 12 in 2024-09-11T01:21:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:09 smithi112 ceph-mon[35384]: osdmap e108: 12 total, 12 up, 12 in 2024-09-11T01:21:09.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 342904 32.56 MB/sec execute 435 sec latency 217.862 ms 2024-09-11T01:21:09.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.982+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1377343721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ff8109170 msgr2=0x7f9ff810f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:09.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.982+0000 7f9ffcd84640 1 --2- 172.21.15.78:0/1377343721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ff8109170 0x7f9ff810f970 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009be0 tx=0x7f9ff002a6d0 comp rx=0 tx=0).stop 2024-09-11T01:21:09.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.983+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1377343721 shutdown_connections 2024-09-11T01:21:09.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.983+0000 7f9ffcd84640 1 --2- 172.21.15.78:0/1377343721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ff8109170 0x7f9ff810f970 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:09.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.983+0000 7f9ffcd84640 1 --2- 172.21.15.78:0/1377343721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff81087f0 0x7f9ff8108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:09.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.983+0000 7f9ffcd84640 1 --2- 172.21.15.78:0/1377343721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff8102de0 0x7f9ff81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:09.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.983+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1377343721 >> 172.21.15.78:0/1377343721 conn(0x7f9ff80fe830 msgr2=0x7f9ff8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:09.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.983+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1377343721 shutdown_connections 2024-09-11T01:21:09.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.983+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1377343721 wait complete. 2024-09-11T01:21:09.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.984+0000 7f9ffcd84640 1 Processor -- start 2024-09-11T01:21:09.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.984+0000 7f9ffcd84640 1 -- start start 2024-09-11T01:21:09.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.984+0000 7f9ffcd84640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ff8102de0 0x7f9ff8199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:09.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.984+0000 7f9ffcd84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff81087f0 0x7f9ff8199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:09.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.984+0000 7f9ffcd84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff8109170 0x7f9ff819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:09.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.984+0000 7f9ff659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ff8102de0 0x7f9ff8199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:09.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.984+0000 7f9ff659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ff8102de0 0x7f9ff8199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:54968/0 (socket says 172.21.15.78:54968) 2024-09-11T01:21:09.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.985+0000 7f9ff659c640 1 -- 172.21.15.78:0/1745249195 learned_addr learned my addr 172.21.15.78:0/1745249195 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:09.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.985+0000 7f9ff5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff81087f0 0x7f9ff8199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:09.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.985+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1745249195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff8112e30 con 0x7f9ff8109170 2024-09-11T01:21:09.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.985+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff8109170 0x7f9ff819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:09.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.985+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1745249195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff8112cb0 con 0x7f9ff8102de0 2024-09-11T01:21:09.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.985+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1745249195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff8112fb0 con 0x7f9ff81087f0 2024-09-11T01:21:09.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.985+0000 7f9ff659c640 1 -- 172.21.15.78:0/1745249195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff81087f0 msgr2=0x7f9ff8199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:09.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.985+0000 7f9ff659c640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff81087f0 0x7f9ff8199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:09.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.985+0000 7f9ff659c640 1 -- 172.21.15.78:0/1745249195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff8109170 msgr2=0x7f9ff819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:09.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.985+0000 7f9ff659c640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff8109170 0x7f9ff819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:09.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.985+0000 7f9ff659c640 1 -- 172.21.15.78:0/1745249195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff819f2d0 con 0x7f9ff8102de0 2024-09-11T01:21:09.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.986+0000 7f9ff659c640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ff8102de0 0x7f9ff8199220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800e9f0 tx=0x7f9fe800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:09.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.986+0000 7f9fe77fe640 1 -- 172.21.15.78:0/1745249195 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800cd80 con 0x7f9ff8102de0 2024-09-11T01:21:09.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.986+0000 7f9fe77fe640 1 -- 172.21.15.78:0/1745249195 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe80108e0 con 0x7f9ff8102de0 2024-09-11T01:21:09.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.986+0000 7f9fe77fe640 1 -- 172.21.15.78:0/1745249195 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8018980 con 0x7f9ff8102de0 2024-09-11T01:21:09.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.986+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1745249195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff819f5c0 con 0x7f9ff8102de0 2024-09-11T01:21:09.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.986+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1745249195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff81a5c80 con 0x7f9ff8102de0 2024-09-11T01:21:09.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.990+0000 7f9fff159740 1 Processor -- start 2024-09-11T01:21:09.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.990+0000 7f9fe77fe640 1 -- 172.21.15.78:0/1745249195 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9fe8010430 con 0x7f9ff8102de0 2024-09-11T01:21:09.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.991+0000 7f9fff159740 1 -- start start 2024-09-11T01:21:09.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.991+0000 7f9fff159740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:09.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.991+0000 7f9fff159740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed198200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:09.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.991+0000 7f9fff159740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed1aa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:09.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.991+0000 7f9fff159740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b6ed109ea0 con 0x55b6ed194c80 2024-09-11T01:21:09.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.991+0000 7f9fff159740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b6ed0f92d0 con 0x55b6ed1978e0 2024-09-11T01:21:09.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.991+0000 7f9fff159740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b6ed0fd0e0 con 0x55b6ed1a8430 2024-09-11T01:21:09.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.991+0000 7f9ff6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:09.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.992+0000 7f9ff6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:54982/0 (socket says 172.21.15.78:54982) 2024-09-11T01:21:09.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.992+0000 7f9ff6d9d640 1 -- 172.21.15.78:0/3321283633 learned_addr learned my addr 172.21.15.78:0/3321283633 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:09.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.992+0000 7f9ff659c640 1 --2- 172.21.15.78:0/3321283633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:09.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.992+0000 7f9ff5d9b640 1 --2- 172.21.15.78:0/3321283633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed1aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:09.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.992+0000 7f9ff6d9d640 1 -- 172.21.15.78:0/3321283633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 msgr2=0x55b6ed1aa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:09.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.992+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/3321283633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed1aa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:09.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.992+0000 7f9ff6d9d640 1 -- 172.21.15.78:0/3321283633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 msgr2=0x55b6ed198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:09.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.992+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/3321283633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:09.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.992+0000 7f9ff6d9d640 1 -- 172.21.15.78:0/3321283633 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ece57650 con 0x55b6ed1978e0 2024-09-11T01:21:09.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.993+0000 7f9fe77fe640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9fe00b54d0 0x7f9fe00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:09.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.993+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/3321283633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed197cc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009be0 tx=0x7f9ff0005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d84a09dc68962bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:09.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.993+0000 7f9fe77fe640 1 -- 172.21.15.78:0/1745249195 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8493+0+0 (secure 0 0 0) 0x7f9fe80e6220 con 0x7f9ff8102de0 2024-09-11T01:21:09.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.994+0000 7f9ff659c640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9fe00b54d0 0x7f9fe00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:09.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.994+0000 7f9fe4ff9640 1 -- 172.21.15.78:0/3321283633 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0041030 con 0x55b6ed1978e0 2024-09-11T01:21:09.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.994+0000 7f9fe4ff9640 1 -- 172.21.15.78:0/3321283633 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ff0003e20 con 0x55b6ed1978e0 2024-09-11T01:21:09.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.995+0000 7f9fe4ff9640 1 -- 172.21.15.78:0/3321283633 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0036040 con 0x55b6ed1978e0 2024-09-11T01:21:09.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.995+0000 7f9fff159740 1 -- 172.21.15.78:0/3321283633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 msgr2=0x55b6ed197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:09.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.995+0000 7f9fff159740 1 --2- 172.21.15.78:0/3321283633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed197cc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009be0 tx=0x7f9ff0005ec0 comp rx=0 tx=0).stop 2024-09-11T01:21:09.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.995+0000 7f9fff159740 1 -- 172.21.15.78:0/3321283633 shutdown_connections 2024-09-11T01:21:09.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.995+0000 7f9fff159740 1 --2- 172.21.15.78:0/3321283633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed1aa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:09.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.995+0000 7f9fff159740 1 --2- 172.21.15.78:0/3321283633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:09.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.995+0000 7f9fff159740 1 --2- 172.21.15.78:0/3321283633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed197cc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:09.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.995+0000 7f9fff159740 1 -- 172.21.15.78:0/3321283633 >> 172.21.15.78:0/3321283633 conn(0x55b6ed195710 msgr2=0x55b6ed1968b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:09.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.997+0000 7f9ff659c640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9fe00b54d0 0x7f9fe00b7990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80e3a40 tx=0x7f9fe8000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:09.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.998+0000 7f9fff159740 1 -- 172.21.15.78:0/3321283633 shutdown_connections 2024-09-11T01:21:10.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.998+0000 7f9fff159740 1 -- 172.21.15.78:0/3321283633 wait complete. 2024-09-11T01:21:10.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.998+0000 7f9fff159740 1 -- start start 2024-09-11T01:21:10.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.999+0000 7f9fff159740 1 Processor -- start 2024-09-11T01:21:10.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.999+0000 7f9fff159740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:10.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.999+0000 7f9fff159740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed191950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:10.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.999+0000 7f9fff159740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed191e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:10.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.999+0000 7f9fff159740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b6ecf5a700 con 0x55b6ed194c80 2024-09-11T01:21:10.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.999+0000 7f9fff159740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b6ed094160 con 0x55b6ed1978e0 2024-09-11T01:21:10.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.999+0000 7f9fff159740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b6ed1ab260 con 0x55b6ed1a8430 2024-09-11T01:21:10.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.999+0000 7f9fff159740 10 client.0 _ll_register_callbacks cb 0x55b6ed0ad9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:10.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.999+0000 7f9ff5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed191950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:10.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.999+0000 7f9ff5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed191950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:54994/0 (socket says 172.21.15.78:54994) 2024-09-11T01:21:10.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:09.999+0000 7f9ff5d9b640 1 -- 172.21.15.78:0/1769857987 learned_addr learned my addr 172.21.15.78:0/1769857987 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:10.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.000+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed191e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:10.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.000+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:10.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.000+0000 7f9ff5d9b640 1 -- 172.21.15.78:0/1769857987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 msgr2=0x55b6ed191e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:10.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.000+0000 7f9ff5d9b640 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed191e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.000+0000 7f9ff5d9b640 1 -- 172.21.15.78:0/1769857987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 msgr2=0x55b6ed1944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.000+0000 7f9ff5d9b640 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.000+0000 7f9ff5d9b640 1 -- 172.21.15.78:0/1769857987 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ed0c1c90 con 0x55b6ed1978e0 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.000+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.000+0000 7f9ff5d9b640 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed191950 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b960 tx=0x7f9fec00be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f49633b68fe52346 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.000+0000 7f9fddffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.000+0000 7f9fddffb640 1 -- 172.21.15.78:0/1769857987 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec003ea0 con 0x55b6ed1978e0 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.001+0000 7f9fff159740 1 -- 172.21.15.78:0/1769857987 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ff0003e20 con 0x55b6ed1978e0 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.001+0000 7f9fddffb640 1 -- 172.21.15.78:0/1769857987 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8493+0+0 (secure 0 0 0) 0x7f9fec011440 con 0x55b6ed1978e0 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.001+0000 7f9fddffb640 1 -- 172.21.15.78:0/1769857987 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f9fec01b640 con 0x55b6ed1978e0 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.001+0000 7f9fff159740 10 client.28295 fetch_fsmap learned FSMap version 72 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.001+0000 7f9fff159740 1 -- 172.21.15.78:0/1769857987 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55b6ed1b6bd0 con 0x55b6ed1978e0 2024-09-11T01:21:10.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.001+0000 7f9fddffb640 1 -- 172.21.15.78:0/1769857987 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ff0003e20 con 0x55b6ed1978e0 2024-09-11T01:21:10.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.002+0000 7f9fddffb640 1 -- 172.21.15.78:0/1769857987 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f9fec022020 con 0x55b6ed1978e0 2024-09-11T01:21:10.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.002+0000 7f9fff159740 10 client.28295 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:21:10.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.002+0000 7f9fff159740 10 client.28295 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:21:10.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.002+0000 7f9fff159740 20 client.28295 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:10.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.002+0000 7f9fff159740 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b6ed1b8080 0x55b6ed1ba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:10.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.002+0000 7f9fff159740 4 client.28295 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:21:10.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.002+0000 7f9fff159740 1 -- 172.21.15.78:0/1769857987 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ed10bdf0 con 0x55b6ed1b8080 2024-09-11T01:21:10.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.002+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b6ed1b8080 0x55b6ed1ba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:10.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.003+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b6ed1b8080 0x55b6ed1ba500 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9545ef6beffaa49 server_cookie=525e273dfd6e41 in_seq=0 out_seq=0 2024-09-11T01:21:10.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.003+0000 7f9fddffb640 10 client.28295 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:21:10.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.003+0000 7f9fddffb640 1 -- 172.21.15.78:0/1769857987 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6ed10bdf0 con 0x55b6ed1b8080 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.003+0000 7f9fddffb640 10 client.28295 handle_command_reply: tid=0 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.003+0000 7f9fff159740 1 client.28295 shutdown 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.004+0000 7f9fff159740 1 -- 172.21.15.78:0/1769857987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 msgr2=0x55b6ed191950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.004+0000 7f9fff159740 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed191950 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b960 tx=0x7f9fec00be20 comp rx=0 tx=0).stop 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.004+0000 7f9fff159740 1 -- 172.21.15.78:0/1769857987 shutdown_connections 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.004+0000 7f9fff159740 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed191e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.004+0000 7f9fff159740 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed191950 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.004+0000 7f9fff159740 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b6ed1b8080 0x55b6ed1ba500 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.004+0000 7f9fff159740 1 --2- 172.21.15.78:0/1769857987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.004+0000 7f9fff159740 1 -- 172.21.15.78:0/1769857987 >> 172.21.15.78:0/1769857987 conn(0x55b6ed195710 msgr2=0x55b6ed19aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.004+0000 7f9fddffb640 0 client.28295 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.005+0000 7f9fff159740 1 -- 172.21.15.78:0/1769857987 shutdown_connections 2024-09-11T01:21:10.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.005+0000 7f9fff159740 1 -- 172.21.15.78:0/1769857987 wait complete. 2024-09-11T01:21:10.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.005+0000 7f9fff159740 20 client.28295 trim_cache size 0 max 16384 2024-09-11T01:21:10.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.025+0000 7f9fff159740 1 Processor -- start 2024-09-11T01:21:10.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.025+0000 7f9fff159740 1 -- start start 2024-09-11T01:21:10.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.025+0000 7f9fff159740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:10.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.025+0000 7f9fff159740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:10.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.025+0000 7f9fff159740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:10.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.025+0000 7f9fff159740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81ac460 con 0x55b6ed194c80 2024-09-11T01:21:10.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.025+0000 7f9fff159740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81ae060 con 0x55b6ed1978e0 2024-09-11T01:21:10.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.025+0000 7f9fff159740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81acb30 con 0x55b6ed1a8430 2024-09-11T01:21:10.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.025+0000 7f9ff6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:10.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.025+0000 7f9ff6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:10.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.025+0000 7f9ff6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49696/0 (socket says 172.21.15.78:49696) 2024-09-11T01:21:10.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.026+0000 7f9ff6d9d640 1 -- 172.21.15.78:0/2089407100 learned_addr learned my addr 172.21.15.78:0/2089407100 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:10.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.026+0000 7f9ff5d9b640 1 --2- 172.21.15.78:0/2089407100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:10.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.026+0000 7f9ff6d9d640 1 -- 172.21.15.78:0/2089407100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 msgr2=0x55b6ed19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:10.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.026+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/2089407100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.026+0000 7f9ff6d9d640 1 -- 172.21.15.78:0/2089407100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 msgr2=0x55b6ed1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:10.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.026+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/2089407100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.026+0000 7f9ff6d9d640 1 -- 172.21.15.78:0/2089407100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ed1ba860 con 0x55b6ed1a8430 2024-09-11T01:21:10.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.026+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/2089407100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed19aef0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0000f80 tx=0x7f9ff0033040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=649119a152107af0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:10.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.032+0000 7f9fdffff640 1 -- 172.21.15.78:0/2089407100 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0041040 con 0x55b6ed1a8430 2024-09-11T01:21:10.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.032+0000 7f9fdffff640 1 -- 172.21.15.78:0/2089407100 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ff0003e20 con 0x55b6ed1a8430 2024-09-11T01:21:10.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.032+0000 7f9fdffff640 1 -- 172.21.15.78:0/2089407100 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0036040 con 0x55b6ed1a8430 2024-09-11T01:21:10.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.033+0000 7f9fff159740 1 -- 172.21.15.78:0/2089407100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 msgr2=0x55b6ed19aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:10.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.033+0000 7f9fff159740 1 --2- 172.21.15.78:0/2089407100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed19aef0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0000f80 tx=0x7f9ff0033040 comp rx=0 tx=0).stop 2024-09-11T01:21:10.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.034+0000 7f9fff159740 1 -- 172.21.15.78:0/2089407100 shutdown_connections 2024-09-11T01:21:10.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.034+0000 7f9fff159740 1 --2- 172.21.15.78:0/2089407100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed19aef0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.034+0000 7f9fff159740 1 --2- 172.21.15.78:0/2089407100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.034+0000 7f9fff159740 1 --2- 172.21.15.78:0/2089407100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.034+0000 7f9fff159740 1 -- 172.21.15.78:0/2089407100 >> 172.21.15.78:0/2089407100 conn(0x55b6ed195710 msgr2=0x55b6ed197cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:10.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.034+0000 7f9fff159740 1 -- 172.21.15.78:0/2089407100 shutdown_connections 2024-09-11T01:21:10.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.034+0000 7f9fff159740 1 -- 172.21.15.78:0/2089407100 wait complete. 2024-09-11T01:21:10.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.034+0000 7f9fff159740 1 -- start start 2024-09-11T01:21:10.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.034+0000 7f9fff159740 1 Processor -- start 2024-09-11T01:21:10.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.035+0000 7f9fff159740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:10.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.035+0000 7f9fff159740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed1acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:10.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.035+0000 7f9fff159740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed1ad290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:10.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.035+0000 7f9fff159740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81ad990 con 0x55b6ed194c80 2024-09-11T01:21:10.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.035+0000 7f9fff159740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81ad200 con 0x55b6ed1978e0 2024-09-11T01:21:10.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.035+0000 7f9fff159740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81aa440 con 0x55b6ed1a8430 2024-09-11T01:21:10.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.035+0000 7f9fff159740 10 client.0 _ll_register_callbacks cb 0x7f9fec004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:10.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.035+0000 7f9ff6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:10.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.035+0000 7f9ff6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1ac7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49706/0 (socket says 172.21.15.78:49706) 2024-09-11T01:21:10.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.035+0000 7f9ff6d9d640 1 -- 172.21.15.78:0/1427810924 learned_addr learned my addr 172.21.15.78:0/1427810924 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:10.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.035+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed1ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.035+0000 7f9ff5d9b640 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed1acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.036+0000 7f9ff6d9d640 1 -- 172.21.15.78:0/1427810924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 msgr2=0x55b6ed1ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.036+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed1ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.036+0000 7f9ff6d9d640 1 -- 172.21.15.78:0/1427810924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 msgr2=0x55b6ed1acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.036+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed1acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.036+0000 7f9ff6d9d640 1 -- 172.21.15.78:0/1427810924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ed2242c0 con 0x55b6ed194c80 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.036+0000 7f9ff5d9b640 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed1acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.036+0000 7f9ff6d9d640 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1ac7d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009370 tx=0x7f9ff003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d31c81ba9caee2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.036+0000 7f9fddffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.036+0000 7f9fddffb640 1 -- 172.21.15.78:0/1427810924 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0036030 con 0x55b6ed194c80 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.036+0000 7f9fddffb640 1 -- 172.21.15.78:0/1427810924 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8493+0+0 (secure 0 0 0) 0x7f9ff0004b40 con 0x55b6ed194c80 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.036+0000 7f9fff159740 1 -- 172.21.15.78:0/1427810924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ff0003e20 con 0x55b6ed194c80 2024-09-11T01:21:10.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.037+0000 7f9fddffb640 1 -- 172.21.15.78:0/1427810924 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f9ff0041070 con 0x55b6ed194c80 2024-09-11T01:21:10.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.037+0000 7f9fff159740 10 client.19260 fetch_fsmap learned FSMap version 72 2024-09-11T01:21:10.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.037+0000 7f9fff159740 1 -- 172.21.15.78:0/1427810924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55b6ed2255c0 con 0x55b6ed194c80 2024-09-11T01:21:10.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.037+0000 7f9fddffb640 1 -- 172.21.15.78:0/1427810924 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ff0003fd0 con 0x55b6ed194c80 2024-09-11T01:21:10.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.038+0000 7f9fddffb640 1 -- 172.21.15.78:0/1427810924 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f9ff003d7d0 con 0x55b6ed194c80 2024-09-11T01:21:10.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.038+0000 7f9fff159740 10 client.19260 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:21:10.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.038+0000 7f9fff159740 10 client.19260 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:21:10.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.038+0000 7f9fff159740 20 client.19260 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:10.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.038+0000 7f9fff159740 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b6ed2257e0 0x55b6ed227bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:10.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.038+0000 7f9fff159740 4 client.19260 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:21:10.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.038+0000 7f9fff159740 1 -- 172.21.15.78:0/1427810924 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9ff0003e20 con 0x55b6ed2257e0 2024-09-11T01:21:10.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.038+0000 7f9ff5d9b640 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b6ed2257e0 0x55b6ed227bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:10.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.039+0000 7f9fddffb640 10 client.19260 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:21:10.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.039+0000 7f9ff5d9b640 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b6ed2257e0 0x55b6ed227bd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f247ca2945377727 server_cookie=bb8f16456fc4ce2b in_seq=0 out_seq=0 2024-09-11T01:21:10.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.040+0000 7f9fddffb640 1 -- 172.21.15.78:0/1427810924 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9ff0003e20 con 0x55b6ed2257e0 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.040+0000 7f9fddffb640 10 client.19260 handle_command_reply: tid=0 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.040+0000 7f9fff159740 1 client.19260 shutdown 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.040+0000 7f9fff159740 1 -- 172.21.15.78:0/1427810924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 msgr2=0x55b6ed1ac7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.040+0000 7f9fff159740 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1ac7d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009370 tx=0x7f9ff003f040 comp rx=0 tx=0).stop 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.040+0000 7f9fff159740 1 -- 172.21.15.78:0/1427810924 shutdown_connections 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.040+0000 7f9fff159740 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6ed1a8430 0x55b6ed1ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.040+0000 7f9fff159740 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6ed1978e0 0x55b6ed1acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.040+0000 7f9fff159740 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55b6ed2257e0 0x55b6ed227bd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.040+0000 7f9fff159740 1 --2- 172.21.15.78:0/1427810924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6ed194c80 0x55b6ed1ac7d0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.040+0000 7f9fddffb640 0 client.19260 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.040+0000 7f9fff159740 1 -- 172.21.15.78:0/1427810924 >> 172.21.15.78:0/1427810924 conn(0x55b6ed195710 msgr2=0x55b6ed19a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.041+0000 7f9fff159740 1 -- 172.21.15.78:0/1427810924 shutdown_connections 2024-09-11T01:21:10.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.041+0000 7f9fff159740 1 -- 172.21.15.78:0/1427810924 wait complete. 2024-09-11T01:21:10.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.041+0000 7f9fff159740 20 client.19260 trim_cache size 0 max 16384 2024-09-11T01:21:10.045 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:21:10.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.042+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1745249195 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9fe00b54d0 msgr2=0x7f9fe00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:10.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.042+0000 7f9ffcd84640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9fe00b54d0 0x7f9fe00b7990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80e3a40 tx=0x7f9fe8000bc0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.042+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1745249195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ff8102de0 msgr2=0x7f9ff8199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:10.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.042+0000 7f9ffcd84640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ff8102de0 0x7f9ff8199220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800e9f0 tx=0x7f9fe800eeb0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.043+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1745249195 shutdown_connections 2024-09-11T01:21:10.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.043+0000 7f9ffcd84640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f9fe00b54d0 0x7f9fe00b7990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.043+0000 7f9ffcd84640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff8109170 0x7f9ff819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.043+0000 7f9ffcd84640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff81087f0 0x7f9ff8199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.043+0000 7f9ffcd84640 1 --2- 172.21.15.78:0/1745249195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ff8102de0 0x7f9ff8199220 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:10.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.043+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1745249195 >> 172.21.15.78:0/1745249195 conn(0x7f9ff80fe830 msgr2=0x7f9ff8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:10.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.043+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1745249195 shutdown_connections 2024-09-11T01:21:10.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:10.043+0000 7f9ffcd84640 1 -- 172.21.15.78:0/1745249195 wait complete. 2024-09-11T01:21:10.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:10 smithi078 ceph-mon[31315]: pgmap v1022: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 646 op/s 2024-09-11T01:21:10.341 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:10 smithi110 ceph-mon[34712]: pgmap v1022: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 646 op/s 2024-09-11T01:21:10.374 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:10 smithi112 ceph-mon[35384]: pgmap v1022: 97 pgs: 97 active+clean; 2.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 646 op/s 2024-09-11T01:21:10.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 343473 32.55 MB/sec execute 436 sec latency 208.459 ms 2024-09-11T01:21:11.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 343803 32.51 MB/sec execute 437 sec latency 570.670 ms 2024-09-11T01:21:12.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:12 smithi110 ceph-mon[34712]: pgmap v1023: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 618 op/s 2024-09-11T01:21:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:12 smithi112 ceph-mon[35384]: pgmap v1023: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 618 op/s 2024-09-11T01:21:12.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:12 smithi078 ceph-mon[31315]: pgmap v1023: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 618 op/s 2024-09-11T01:21:12.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 344362 32.52 MB/sec execute 438 sec latency 270.137 ms 2024-09-11T01:21:13.318 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:21:13.693 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:21:13.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 344965 32.52 MB/sec execute 439 sec latency 172.818 ms 2024-09-11T01:21:14.134 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:14 smithi078 ceph-mon[31315]: pgmap v1024: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 611 op/s 2024-09-11T01:21:14.134 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:14 smithi078 ceph-mon[31315]: osdmap e109: 12 total, 12 up, 12 in 2024-09-11T01:21:14.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.250+0000 7fa2b85ce640 1 -- 172.21.15.78:0/2637663070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2b0102dc0 msgr2=0x7fa2b01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:14.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.250+0000 7fa2b85ce640 1 --2- 172.21.15.78:0/2637663070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2b0102dc0 0x7fa2b01031a0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009900 tx=0x7fa2ac02a3f0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.250+0000 7fa2b85ce640 1 -- 172.21.15.78:0/2637663070 shutdown_connections 2024-09-11T01:21:14.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.250+0000 7fa2b85ce640 1 --2- 172.21.15.78:0/2637663070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b0109150 0x7fa2b010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.250+0000 7fa2b85ce640 1 --2- 172.21.15.78:0/2637663070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b01087d0 0x7fa2b0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.250+0000 7fa2b85ce640 1 --2- 172.21.15.78:0/2637663070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2b0102dc0 0x7fa2b01031a0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.250+0000 7fa2b85ce640 1 -- 172.21.15.78:0/2637663070 >> 172.21.15.78:0/2637663070 conn(0x7fa2b00fe870 msgr2=0x7fa2b0100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:14.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.250+0000 7fa2b85ce640 1 -- 172.21.15.78:0/2637663070 shutdown_connections 2024-09-11T01:21:14.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.250+0000 7fa2b85ce640 1 -- 172.21.15.78:0/2637663070 wait complete. 2024-09-11T01:21:14.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.251+0000 7fa2b85ce640 1 Processor -- start 2024-09-11T01:21:14.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.251+0000 7fa2b85ce640 1 -- start start 2024-09-11T01:21:14.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.251+0000 7fa2b85ce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0102dc0 0x7fa2b0199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.251+0000 7fa2b85ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b01087d0 0x7fa2b0199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.251+0000 7fa2b85ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2b0109150 0x7fa2b019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.251+0000 7fa2b85ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0112e60 con 0x7fa2b0109150 2024-09-11T01:21:14.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.251+0000 7fa2b85ce640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0112ce0 con 0x7fa2b0102dc0 2024-09-11T01:21:14.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.251+0000 7fa2b85ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0112fe0 con 0x7fa2b01087d0 2024-09-11T01:21:14.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.252+0000 7fa2b636a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0102dc0 0x7fa2b0199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.252+0000 7fa2b636a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0102dc0 0x7fa2b0199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38682/0 (socket says 172.21.15.78:38682) 2024-09-11T01:21:14.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.252+0000 7fa2b6b6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2b0109150 0x7fa2b019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.252+0000 7fa2b636a640 1 -- 172.21.15.78:0/3398338145 learned_addr learned my addr 172.21.15.78:0/3398338145 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:14.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.252+0000 7fa2b5b69640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b01087d0 0x7fa2b0199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.252+0000 7fa2b6b6b640 1 -- 172.21.15.78:0/3398338145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b01087d0 msgr2=0x7fa2b0199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:14.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.252+0000 7fa2b6b6b640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b01087d0 0x7fa2b0199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.252+0000 7fa2b6b6b640 1 -- 172.21.15.78:0/3398338145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0102dc0 msgr2=0x7fa2b0199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:14.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.252+0000 7fa2b6b6b640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0102dc0 0x7fa2b0199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.252+0000 7fa2b6b6b640 1 -- 172.21.15.78:0/3398338145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b019f3c0 con 0x7fa2b0109150 2024-09-11T01:21:14.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.252+0000 7fa2b6b6b640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2b0109150 0x7fa2b019ebf0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800e940 tx=0x7fa2a800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:14.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.252+0000 7fa2a77fe640 1 -- 172.21.15.78:0/3398338145 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8004d20 con 0x7fa2b0109150 2024-09-11T01:21:14.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.253+0000 7fa2b85ce640 1 -- 172.21.15.78:0/3398338145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b019f6b0 con 0x7fa2b0109150 2024-09-11T01:21:14.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.253+0000 7fa2a77fe640 1 -- 172.21.15.78:0/3398338145 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a8005020 con 0x7fa2b0109150 2024-09-11T01:21:14.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.253+0000 7fa2b85ce640 1 -- 172.21.15.78:0/3398338145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b0075380 con 0x7fa2b0109150 2024-09-11T01:21:14.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.253+0000 7fa2a77fe640 1 -- 172.21.15.78:0/3398338145 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80116d0 con 0x7fa2b0109150 2024-09-11T01:21:14.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.255+0000 7fa2ba9a3740 1 Processor -- start 2024-09-11T01:21:14.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.256+0000 7fa2a77fe640 1 -- 172.21.15.78:0/3398338145 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa2a800f040 con 0x7fa2b0109150 2024-09-11T01:21:14.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.257+0000 7fa2ba9a3740 1 -- start start 2024-09-11T01:21:14.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.257+0000 7fa2ba9a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 0x55e9c56a80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.257+0000 7fa2ba9a3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56a8600 0x55e9c56b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.257+0000 7fa2ba9a3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56b4590 0x55e9c56b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.257+0000 7fa2ba9a3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e9c5615ea0 con 0x55e9c56b4590 2024-09-11T01:21:14.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.257+0000 7fa2ba9a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e9c56052d0 con 0x55e9c56a7ce0 2024-09-11T01:21:14.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.257+0000 7fa2ba9a3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e9c56090e0 con 0x55e9c56a8600 2024-09-11T01:21:14.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2a77fe640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa29c0b5450 0x7fa29c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2b5b69640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 0x55e9c56a80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2b5b69640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 0x55e9c56a80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38688/0 (socket says 172.21.15.78:38688) 2024-09-11T01:21:14.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2b5b69640 1 -- 172.21.15.78:0/3564498634 learned_addr learned my addr 172.21.15.78:0/3564498634 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:14.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2b5b69640 1 --2- 172.21.15.78:0/3564498634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56b4590 0x55e9c56b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2b636a640 1 --2- 172.21.15.78:0/3564498634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56a8600 0x55e9c56b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2a77fe640 1 -- 172.21.15.78:0/3398338145 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7fa2a80e5d10 con 0x7fa2b0109150 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2b6b6b640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa29c0b5450 0x7fa29c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2b636a640 1 -- 172.21.15.78:0/3564498634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 msgr2=0x55e9c56a80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2b636a640 1 --2- 172.21.15.78:0/3564498634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 0x55e9c56a80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2b636a640 1 -- 172.21.15.78:0/3564498634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56b4590 msgr2=0x55e9c56b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2b636a640 1 --2- 172.21.15.78:0/3564498634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56b4590 0x55e9c56b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2b636a640 1 -- 172.21.15.78:0/3564498634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c5363650 con 0x55e9c56a8600 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.258+0000 7fa2b5b69640 1 --2- 172.21.15.78:0/3564498634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 0x55e9c56a80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.259+0000 7fa2b636a640 1 --2- 172.21.15.78:0/3564498634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56a8600 0x55e9c56b4050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac031440 tx=0x7fa2ac005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8da269500cd8bdf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.259+0000 7fa2a4ff9640 1 -- 172.21.15.78:0/3564498634 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac035040 con 0x55e9c56a8600 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.259+0000 7fa2b6b6b640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa29c0b5450 0x7fa29c0b7910 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8005fd0 tx=0x7fa2a800ef30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:14.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.259+0000 7fa2a4ff9640 1 -- 172.21.15.78:0/3564498634 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2ac0054d0 con 0x55e9c56a8600 2024-09-11T01:21:14.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.260+0000 7fa2a4ff9640 1 -- 172.21.15.78:0/3564498634 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac003850 con 0x55e9c56a8600 2024-09-11T01:21:14.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.260+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/3564498634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56a8600 msgr2=0x55e9c56b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:14.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.260+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/3564498634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56a8600 0x55e9c56b4050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac031440 tx=0x7fa2ac005c00 comp rx=0 tx=0).stop 2024-09-11T01:21:14.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.260+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/3564498634 shutdown_connections 2024-09-11T01:21:14.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.260+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/3564498634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56b4590 0x55e9c56b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.260+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/3564498634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56a8600 0x55e9c56b4050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.260+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/3564498634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 0x55e9c56a80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.260+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/3564498634 >> 172.21.15.78:0/3564498634 conn(0x55e9c56a5b10 msgr2=0x55e9c56a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.260+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/3564498634 shutdown_connections 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.261+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/3564498634 wait complete. 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.261+0000 7fa2ba9a3740 1 -- start start 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.261+0000 7fa2ba9a3740 1 Processor -- start 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.261+0000 7fa2ba9a3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56a7ce0 0x55e9c56b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.261+0000 7fa2ba9a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 0x55e9c56b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.261+0000 7fa2ba9a3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56b4590 0x55e9c56b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.261+0000 7fa2ba9a3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e9c5466700 con 0x55e9c56b4590 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.261+0000 7fa2ba9a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e9c55a0160 con 0x55e9c56a8600 2024-09-11T01:21:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.261+0000 7fa2ba9a3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e9c56b7280 con 0x55e9c56a7ce0 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.261+0000 7fa2ba9a3740 10 client.0 _ll_register_callbacks cb 0x55e9c55b99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.262+0000 7fa2b5b69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56a7ce0 0x55e9c56b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.262+0000 7fa2b5b69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56a7ce0 0x55e9c56b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:39310/0 (socket says 172.21.15.78:39310) 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.262+0000 7fa2b636a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 0x55e9c56b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.262+0000 7fa2b5b69640 1 -- 172.21.15.78:0/1637420301 learned_addr learned my addr 172.21.15.78:0/1637420301 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.262+0000 7fa2b5b69640 1 --2- 172.21.15.78:0/1637420301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56b4590 0x55e9c56b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.262+0000 7fa2b636a640 1 -- 172.21.15.78:0/1637420301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56a7ce0 msgr2=0x55e9c56b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.262+0000 7fa2b636a640 1 --2- 172.21.15.78:0/1637420301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56a7ce0 0x55e9c56b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.262+0000 7fa2b636a640 1 -- 172.21.15.78:0/1637420301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56b4590 msgr2=0x55e9c56b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.262+0000 7fa2b636a640 1 --2- 172.21.15.78:0/1637420301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56b4590 0x55e9c56b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.262+0000 7fa2b636a640 1 -- 172.21.15.78:0/1637420301 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c55cdc90 con 0x55e9c56a8600 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.262+0000 7fa2b636a640 1 --2- 172.21.15.78:0/1637420301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 0x55e9c56b1220 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009850 tx=0x7fa2ac005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31df76e1c46eb9c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.262+0000 7fa299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:21:14.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.263+0000 7fa299ffb640 1 -- 172.21.15.78:0/1637420301 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac052060 con 0x55e9c56a8600 2024-09-11T01:21:14.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.263+0000 7fa299ffb640 1 -- 172.21.15.78:0/1637420301 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7fa2ac0047a0 con 0x55e9c56a8600 2024-09-11T01:21:14.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.263+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1637420301 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2ac0054d0 con 0x55e9c56a8600 2024-09-11T01:21:14.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.263+0000 7fa299ffb640 1 -- 172.21.15.78:0/1637420301 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fa2ac035050 con 0x55e9c56a8600 2024-09-11T01:21:14.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.263+0000 7fa2ba9a3740 10 client.28313 fetch_fsmap learned FSMap version 72 2024-09-11T01:21:14.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.263+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1637420301 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55e9c56c2b60 con 0x55e9c56a8600 2024-09-11T01:21:14.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.264+0000 7fa299ffb640 1 -- 172.21.15.78:0/1637420301 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2ac0054d0 con 0x55e9c56a8600 2024-09-11T01:21:14.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.264+0000 7fa299ffb640 1 -- 172.21.15.78:0/1637420301 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7fa2ac0039b0 con 0x55e9c56a8600 2024-09-11T01:21:14.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.264+0000 7fa2ba9a3740 10 client.28313 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:21:14.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.264+0000 7fa2ba9a3740 10 client.28313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:21:14.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.264+0000 7fa2ba9a3740 20 client.28313 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:14.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.264+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/1637420301 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e9c56c40b0 0x55e9c56c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.264+0000 7fa2ba9a3740 4 client.28313 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:21:14.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.264+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1637420301 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9c5617df0 con 0x55e9c56c40b0 2024-09-11T01:21:14.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.264+0000 7fa2b5b69640 1 --2- 172.21.15.78:0/1637420301 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e9c56c40b0 0x55e9c56c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.265+0000 7fa299ffb640 10 client.28313 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.265+0000 7fa2b5b69640 1 --2- 172.21.15.78:0/1637420301 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e9c56c40b0 0x55e9c56c64f0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78fb339c8407c514 server_cookie=8ec2aec0e3b5a4c5 in_seq=0 out_seq=0 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa299ffb640 1 -- 172.21.15.78:0/1637420301 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e9c5617df0 con 0x55e9c56c40b0 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa299ffb640 10 client.28313 handle_command_reply: tid=0 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa2ba9a3740 1 client.28313 shutdown 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1637420301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 msgr2=0x55e9c56b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/1637420301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 0x55e9c56b1220 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009850 tx=0x7fa2ac005850 comp rx=0 tx=0).stop 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1637420301 shutdown_connections 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/1637420301 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e9c56c40b0 0x55e9c56c64f0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/1637420301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56b4590 0x55e9c56b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/1637420301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 0x55e9c56b1220 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/1637420301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56a7ce0 0x55e9c56b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1637420301 >> 172.21.15.78:0/1637420301 conn(0x55e9c56a5b10 msgr2=0x55e9c56ab6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa299ffb640 0 client.28313 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1637420301 shutdown_connections 2024-09-11T01:21:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1637420301 wait complete. 2024-09-11T01:21:14.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.266+0000 7fa2ba9a3740 20 client.28313 trim_cache size 0 max 16384 2024-09-11T01:21:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2ba9a3740 1 Processor -- start 2024-09-11T01:21:14.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2ba9a3740 1 -- start start 2024-09-11T01:21:14.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2ba9a3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56a7ce0 0x55e9c56ac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2ba9a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 0x55e9c56aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2ba9a3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56b4590 0x55e9c56ae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2ba9a3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01ac400 con 0x55e9c56a7ce0 2024-09-11T01:21:14.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2ba9a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01ae000 con 0x55e9c56a8600 2024-09-11T01:21:14.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2ba9a3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01acad0 con 0x55e9c56b4590 2024-09-11T01:21:14.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2b636a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 0x55e9c56aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2b636a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 0x55e9c56aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38696/0 (socket says 172.21.15.78:38696) 2024-09-11T01:21:14.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2b636a640 1 -- 172.21.15.78:0/1110253184 learned_addr learned my addr 172.21.15.78:0/1110253184 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:14.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2b5b69640 1 --2- 172.21.15.78:0/1110253184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56a7ce0 0x55e9c56ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.290+0000 7fa2b5b69640 1 --2- 172.21.15.78:0/1110253184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56b4590 0x55e9c56ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.291+0000 7fa2b636a640 1 -- 172.21.15.78:0/1110253184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56b4590 msgr2=0x55e9c56ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.291+0000 7fa2b636a640 1 --2- 172.21.15.78:0/1110253184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56b4590 0x55e9c56ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.291+0000 7fa2b636a640 1 -- 172.21.15.78:0/1110253184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56a7ce0 msgr2=0x55e9c56ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.291+0000 7fa2b636a640 1 --2- 172.21.15.78:0/1110253184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56a7ce0 0x55e9c56ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.291+0000 7fa2b636a640 1 -- 172.21.15.78:0/1110253184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c56a8430 con 0x55e9c56a8600 2024-09-11T01:21:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.291+0000 7fa2b5b69640 1 --2- 172.21.15.78:0/1110253184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56b4590 0x55e9c56ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:21:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.291+0000 7fa2b636a640 1 --2- 172.21.15.78:0/1110253184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 0x55e9c56aca40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0057c0 tx=0x7fa2ac02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d051c3541216c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.291+0000 7fa29bfff640 1 -- 172.21.15.78:0/1110253184 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac005d50 con 0x55e9c56a8600 2024-09-11T01:21:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.291+0000 7fa29bfff640 1 -- 172.21.15.78:0/1110253184 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2ac005000 con 0x55e9c56a8600 2024-09-11T01:21:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.291+0000 7fa29bfff640 1 -- 172.21.15.78:0/1110253184 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac003850 con 0x55e9c56a8600 2024-09-11T01:21:14.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.292+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1110253184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 msgr2=0x55e9c56aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:14.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.292+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/1110253184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 0x55e9c56aca40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0057c0 tx=0x7fa2ac02a3f0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.292+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1110253184 shutdown_connections 2024-09-11T01:21:14.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.292+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/1110253184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56b4590 0x55e9c56ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.292+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/1110253184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a8600 0x55e9c56aca40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.292+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/1110253184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56a7ce0 0x55e9c56ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.292+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1110253184 >> 172.21.15.78:0/1110253184 conn(0x55e9c56a5b10 msgr2=0x55e9c56c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:14.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.293+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1110253184 shutdown_connections 2024-09-11T01:21:14.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.293+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/1110253184 wait complete. 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.293+0000 7fa2ba9a3740 1 -- start start 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.293+0000 7fa2ba9a3740 1 Processor -- start 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.293+0000 7fa2ba9a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 0x55e9c56b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.293+0000 7fa2ba9a3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56a8600 0x55e9c56b8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.293+0000 7fa2ba9a3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56b4590 0x55e9c57290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.293+0000 7fa2ba9a3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01ad930 con 0x55e9c56a8600 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.293+0000 7fa2ba9a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01ad1a0 con 0x55e9c56a7ce0 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.293+0000 7fa2ba9a3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01aa400 con 0x55e9c56b4590 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.293+0000 7fa2ba9a3740 10 client.0 _ll_register_callbacks cb 0x7fa2a0009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.293+0000 7fa2b636a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56a8600 0x55e9c56b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa2b5b69640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 0x55e9c56b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa2b5b69640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 0x55e9c56b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38702/0 (socket says 172.21.15.78:38702) 2024-09-11T01:21:14.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa2b5b69640 1 -- 172.21.15.78:0/2797567175 learned_addr learned my addr 172.21.15.78:0/2797567175 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa2b5b69640 1 --2- 172.21.15.78:0/2797567175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56b4590 0x55e9c57290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa2b636a640 1 -- 172.21.15.78:0/2797567175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56b4590 msgr2=0x55e9c57290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa2b636a640 1 --2- 172.21.15.78:0/2797567175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56b4590 0x55e9c57290a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa2b636a640 1 -- 172.21.15.78:0/2797567175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 msgr2=0x55e9c56b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa2b636a640 1 --2- 172.21.15.78:0/2797567175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 0x55e9c56b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa2b636a640 1 -- 172.21.15.78:0/2797567175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c57297e0 con 0x55e9c56a8600 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa2b636a640 1 --2- 172.21.15.78:0/2797567175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56a8600 0x55e9c56b8540 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009850 tx=0x7fa2ac0053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67788b5298252fba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa299ffb640 1 -- 172.21.15.78:0/2797567175 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac005d50 con 0x55e9c56a8600 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/2797567175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2ac005000 con 0x55e9c56a8600 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.294+0000 7fa299ffb640 1 -- 172.21.15.78:0/2797567175 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7fa2ac003800 con 0x55e9c56a8600 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.295+0000 7fa299ffb640 1 -- 172.21.15.78:0/2797567175 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fa2ac004080 con 0x55e9c56a8600 2024-09-11T01:21:14.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.295+0000 7fa2ba9a3740 10 client.19278 fetch_fsmap learned FSMap version 72 2024-09-11T01:21:14.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.295+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/2797567175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55e9c5730c60 con 0x55e9c56a8600 2024-09-11T01:21:14.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.295+0000 7fa299ffb640 1 -- 172.21.15.78:0/2797567175 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2ac005550 con 0x55e9c56a8600 2024-09-11T01:21:14.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.295+0000 7fa299ffb640 1 -- 172.21.15.78:0/2797567175 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7fa2ac060480 con 0x55e9c56a8600 2024-09-11T01:21:14.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.295+0000 7fa2ba9a3740 10 client.19278 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:21:14.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.295+0000 7fa2ba9a3740 10 client.19278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:21:14.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.295+0000 7fa2ba9a3740 20 client.19278 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:14.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.295+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/2797567175 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e9c5731260 0x55e9c5733680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:14.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.295+0000 7fa2ba9a3740 4 client.19278 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:21:14.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.295+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/2797567175 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa2ac005000 con 0x55e9c5731260 2024-09-11T01:21:14.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.296+0000 7fa2b5b69640 1 --2- 172.21.15.78:0/2797567175 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e9c5731260 0x55e9c5733680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:14.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.296+0000 7fa2b5b69640 1 --2- 172.21.15.78:0/2797567175 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e9c5731260 0x55e9c5733680 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23c389536605444 server_cookie=9b76974f1fb7d67e in_seq=0 out_seq=0 2024-09-11T01:21:14.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.296+0000 7fa299ffb640 10 client.19278 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa299ffb640 1 -- 172.21.15.78:0/2797567175 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa2ac005000 con 0x55e9c5731260 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa299ffb640 10 client.19278 handle_command_reply: tid=0 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa2ba9a3740 1 client.19278 shutdown 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/2797567175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56a8600 msgr2=0x55e9c56b8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/2797567175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56a8600 0x55e9c56b8540 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009850 tx=0x7fa2ac0053e0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/2797567175 shutdown_connections 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/2797567175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9c56b4590 0x55e9c57290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/2797567175 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55e9c5731260 0x55e9c5733680 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa299ffb640 0 client.19278 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/2797567175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e9c56a8600 0x55e9c56b8540 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa2ba9a3740 1 --2- 172.21.15.78:0/2797567175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9c56a7ce0 0x55e9c56b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/2797567175 >> 172.21.15.78:0/2797567175 conn(0x55e9c56a5b10 msgr2=0x55e9c56ac500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/2797567175 shutdown_connections 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.297+0000 7fa2ba9a3740 1 -- 172.21.15.78:0/2797567175 wait complete. 2024-09-11T01:21:14.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.298+0000 7fa2ba9a3740 20 client.19278 trim_cache size 0 max 16384 2024-09-11T01:21:14.303 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:21:14.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.298+0000 7fa2b85ce640 1 -- 172.21.15.78:0/3398338145 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa29c0b5450 msgr2=0x7fa29c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:14.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.298+0000 7fa2b85ce640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa29c0b5450 0x7fa29c0b7910 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8005fd0 tx=0x7fa2a800ef30 comp rx=0 tx=0).stop 2024-09-11T01:21:14.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.298+0000 7fa2b85ce640 1 -- 172.21.15.78:0/3398338145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2b0109150 msgr2=0x7fa2b019ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:14.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.298+0000 7fa2b85ce640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2b0109150 0x7fa2b019ebf0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800e940 tx=0x7fa2a800ee00 comp rx=0 tx=0).stop 2024-09-11T01:21:14.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.299+0000 7fa2b85ce640 1 -- 172.21.15.78:0/3398338145 shutdown_connections 2024-09-11T01:21:14.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.299+0000 7fa2b85ce640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fa29c0b5450 0x7fa29c0b7910 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.299+0000 7fa2b85ce640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2b0109150 0x7fa2b019ebf0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.299+0000 7fa2b85ce640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b01087d0 0x7fa2b0199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.299+0000 7fa2b85ce640 1 --2- 172.21.15.78:0/3398338145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b0102dc0 0x7fa2b0199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:14.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.299+0000 7fa2b85ce640 1 -- 172.21.15.78:0/3398338145 >> 172.21.15.78:0/3398338145 conn(0x7fa2b00fe870 msgr2=0x7fa2b0100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:14.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.299+0000 7fa2b85ce640 1 -- 172.21.15.78:0/3398338145 shutdown_connections 2024-09-11T01:21:14.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:14.299+0000 7fa2b85ce640 1 -- 172.21.15.78:0/3398338145 wait complete. 2024-09-11T01:21:14.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:14 smithi110 ceph-mon[34712]: pgmap v1024: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 611 op/s 2024-09-11T01:21:14.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:14 smithi110 ceph-mon[34712]: osdmap e109: 12 total, 12 up, 12 in 2024-09-11T01:21:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:14 smithi112 ceph-mon[35384]: pgmap v1024: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 611 op/s 2024-09-11T01:21:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:14 smithi112 ceph-mon[35384]: osdmap e109: 12 total, 12 up, 12 in 2024-09-11T01:21:14.802 INFO:tasks.workunit.client.0.smithi078.stdout: 10 345559 32.52 MB/sec execute 440 sec latency 83.912 ms 2024-09-11T01:21:15.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 346220 32.51 MB/sec execute 441 sec latency 59.256 ms 2024-09-11T01:21:16.159 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:21:16.160 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:21:16.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:16 smithi110 ceph-mon[34712]: pgmap v1026: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 607 op/s 2024-09-11T01:21:16.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:16 smithi112 ceph-mon[35384]: pgmap v1026: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 607 op/s 2024-09-11T01:21:16.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:16 smithi078 ceph-mon[31315]: pgmap v1026: 97 pgs: 97 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 607 op/s 2024-09-11T01:21:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:21:16.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 346893 32.54 MB/sec execute 442 sec latency 91.355 ms 2024-09-11T01:21:17.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.202+0000 7fc0f639e640 1 -- 172.21.15.78:0/3036166213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0f0107510 msgr2=0x7fc0f0107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:17.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.202+0000 7fc0f639e640 1 --2- 172.21.15.78:0/3036166213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0f0107510 0x7fc0f0107950 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009900 tx=0x7fc0e002a410 comp rx=0 tx=0).stop 2024-09-11T01:21:17.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.202+0000 7fc0f639e640 1 -- 172.21.15.78:0/3036166213 shutdown_connections 2024-09-11T01:21:17.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.202+0000 7fc0f639e640 1 --2- 172.21.15.78:0/3036166213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0f0107e90 0x7fc0f010e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.202+0000 7fc0f639e640 1 --2- 172.21.15.78:0/3036166213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0f0107510 0x7fc0f0107950 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.202+0000 7fc0f639e640 1 --2- 172.21.15.78:0/3036166213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f0101b00 0x7fc0f0101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.202+0000 7fc0f639e640 1 -- 172.21.15.78:0/3036166213 >> 172.21.15.78:0/3036166213 conn(0x7fc0f00fd5b0 msgr2=0x7fc0f00ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:17.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.202+0000 7fc0f639e640 1 -- 172.21.15.78:0/3036166213 shutdown_connections 2024-09-11T01:21:17.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.202+0000 7fc0f639e640 1 -- 172.21.15.78:0/3036166213 wait complete. 2024-09-11T01:21:17.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.203+0000 7fc0f639e640 1 Processor -- start 2024-09-11T01:21:17.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.203+0000 7fc0f639e640 1 -- start start 2024-09-11T01:21:17.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.204+0000 7fc0f639e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0f0101b00 0x7fc0f0198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.204+0000 7fc0f639e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0f0107510 0x7fc0f0199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.204+0000 7fc0f639e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f0107e90 0x7fc0f019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.204+0000 7fc0f539c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0f0101b00 0x7fc0f0198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.204+0000 7fc0f539c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0f0101b00 0x7fc0f0198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37012/0 (socket says 172.21.15.78:37012) 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.204+0000 7fc0f539c640 1 -- 172.21.15.78:0/3506130371 learned_addr learned my addr 172.21.15.78:0/3506130371 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.204+0000 7fc0f639e640 1 -- 172.21.15.78:0/3506130371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f0111ae0 con 0x7fc0f0101b00 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.204+0000 7fc0f639e640 1 -- 172.21.15.78:0/3506130371 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f0111960 con 0x7fc0f0107510 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.204+0000 7fc0f639e640 1 -- 172.21.15.78:0/3506130371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f0111c60 con 0x7fc0f0107e90 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.204+0000 7fc0f4b9b640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0f0107510 0x7fc0f0199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.204+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f0107e90 0x7fc0f019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.205+0000 7fc0f539c640 1 -- 172.21.15.78:0/3506130371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f0107e90 msgr2=0x7fc0f019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.205+0000 7fc0f539c640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f0107e90 0x7fc0f019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.205+0000 7fc0f539c640 1 -- 172.21.15.78:0/3506130371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0f0107510 msgr2=0x7fc0f0199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.205+0000 7fc0f539c640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0f0107510 0x7fc0f0199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.205+0000 7fc0f539c640 1 -- 172.21.15.78:0/3506130371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f019f070 con 0x7fc0f0101b00 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.205+0000 7fc0f539c640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0f0101b00 0x7fc0f0198fc0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00e9f0 tx=0x7fc0ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.205+0000 7fc0e67fc640 1 -- 172.21.15.78:0/3506130371 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec00cd80 con 0x7fc0f0101b00 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.205+0000 7fc0f639e640 1 -- 172.21.15.78:0/3506130371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f019f360 con 0x7fc0f0101b00 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.205+0000 7fc0e67fc640 1 -- 172.21.15.78:0/3506130371 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0ec0108e0 con 0x7fc0f0101b00 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.205+0000 7fc0f639e640 1 -- 172.21.15.78:0/3506130371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f01a5bf0 con 0x7fc0f0101b00 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.205+0000 7fc0e67fc640 1 -- 172.21.15.78:0/3506130371 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec018980 con 0x7fc0f0101b00 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.208+0000 7fc0f99d5740 1 Processor -- start 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.208+0000 7fc0f99d5740 1 -- start start 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.208+0000 7fc0f99d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 0x5566c81834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.208+0000 7fc0f99d5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c818d450 0x5566c8183a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.208+0000 7fc0f99d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c819a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.208+0000 7fc0f99d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5566c80f9e60 con 0x5566c8198050 2024-09-11T01:21:17.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.208+0000 7fc0f99d5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5566c80e9290 con 0x5566c818d450 2024-09-11T01:21:17.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.208+0000 7fc0f99d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5566c80ed0a0 con 0x5566c81830f0 2024-09-11T01:21:17.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0f4b9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0f5b9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 0x5566c81834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0f4b9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37018/0 (socket says 172.21.15.78:37018) 2024-09-11T01:21:17.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0f4b9b640 1 -- 172.21.15.78:0/1060523412 learned_addr learned my addr 172.21.15.78:0/1060523412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:17.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0e67fc640 1 -- 172.21.15.78:0/3506130371 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc0ec010430 con 0x7fc0f0101b00 2024-09-11T01:21:17.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0f5b9d640 1 -- 172.21.15.78:0/1060523412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c818d450 msgr2=0x5566c8183a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:17.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/1060523412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c818d450 0x5566c8183a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0f5b9d640 1 -- 172.21.15.78:0/1060523412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 msgr2=0x5566c819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/1060523412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0f5b9d640 1 -- 172.21.15.78:0/1060523412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c7e47930 con 0x5566c81830f0 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0f4b9b640 1 --2- 172.21.15.78:0/1060523412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/1060523412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 0x5566c81834d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009f00 tx=0x7fc0e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dedc72453a5ab97 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0db7fe640 1 -- 172.21.15.78:0/1060523412 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8004d20 con 0x5566c81830f0 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0db7fe640 1 -- 172.21.15.78:0/1060523412 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e8004030 con 0x5566c81830f0 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.209+0000 7fc0e67fc640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc0dc0b5700 0x7fc0dc0b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.210+0000 7fc0db7fe640 1 -- 172.21.15.78:0/1060523412 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8005280 con 0x5566c81830f0 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.210+0000 7fc0e67fc640 1 -- 172.21.15.78:0/3506130371 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7fc0ec018720 con 0x7fc0f0101b00 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.210+0000 7fc0f539c640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc0dc0b5700 0x7fc0dc0b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.210+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1060523412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 msgr2=0x5566c81834d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.210+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/1060523412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 0x5566c81834d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009f00 tx=0x7fc0e8011810 comp rx=0 tx=0).stop 2024-09-11T01:21:17.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.210+0000 7fc0f539c640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc0dc0b5700 0x7fc0dc0b7bc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5566c8182bd0 tx=0x7fc0ec076750 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.210+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1060523412 shutdown_connections 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.210+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/1060523412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.210+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/1060523412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c818d450 0x5566c8183a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.210+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/1060523412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 0x5566c81834d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.210+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1060523412 >> 172.21.15.78:0/1060523412 conn(0x5566c8180f20 msgr2=0x5566c81820c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.211+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1060523412 shutdown_connections 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.211+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1060523412 wait complete. 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.211+0000 7fc0f99d5740 1 -- start start 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.211+0000 7fc0f99d5740 1 Processor -- start 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f99d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c81830f0 0x5566c818a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f99d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 0x5566c818a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f99d5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c8198050 0x5566c818aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f99d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5566c7f4a890 con 0x5566c81830f0 2024-09-11T01:21:17.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f99d5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5566c8083d40 con 0x5566c8198050 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f99d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5566c819ae40 con 0x5566c818d450 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f99d5740 10 client.0 _ll_register_callbacks cb 0x5566c80e7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f5b9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c81830f0 0x5566c818a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f4b9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 0x5566c818a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f4b9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 0x5566c818a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:39344/0 (socket says 172.21.15.78:39344) 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f4b9b640 1 -- 172.21.15.78:0/2379363408 learned_addr learned my addr 172.21.15.78:0/2379363408 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f5b9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c81830f0 0x5566c818a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37032/0 (socket says 172.21.15.78:37032) 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c8198050 0x5566c818aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f4b9b640 1 -- 172.21.15.78:0/2379363408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c8198050 msgr2=0x5566c818aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f4b9b640 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c8198050 0x5566c818aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f4b9b640 1 -- 172.21.15.78:0/2379363408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c81830f0 msgr2=0x5566c818a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f4b9b640 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c81830f0 0x5566c818a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.212+0000 7fc0f4b9b640 1 -- 172.21.15.78:0/2379363408 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c81a4280 con 0x5566c818d450 2024-09-11T01:21:17.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.213+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c81830f0 0x5566c818a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.213+0000 7fc0f4b9b640 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 0x5566c818a960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00096c0 tx=0x7fc0e0032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=377878752e77aeaa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.213+0000 7fc0d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.213+0000 7fc0d8ff9640 1 -- 172.21.15.78:0/2379363408 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0035040 con 0x5566c818d450 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.213+0000 7fc0d8ff9640 1 -- 172.21.15.78:0/2379363408 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7fc0e0031720 con 0x5566c818d450 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.213+0000 7fc0f99d5740 1 -- 172.21.15.78:0/2379363408 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e8004030 con 0x5566c818d450 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.213+0000 7fc0d8ff9640 1 -- 172.21.15.78:0/2379363408 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fc0e0031920 con 0x5566c818d450 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.213+0000 7fc0f99d5740 10 client.28279 fetch_fsmap learned FSMap version 72 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.213+0000 7fc0f99d5740 1 -- 172.21.15.78:0/2379363408 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x5566c81a6c30 con 0x5566c818d450 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.214+0000 7fc0d8ff9640 1 -- 172.21.15.78:0/2379363408 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e8004030 con 0x5566c818d450 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.214+0000 7fc0d8ff9640 1 -- 172.21.15.78:0/2379363408 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7fc0e0004750 con 0x5566c818d450 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.214+0000 7fc0f99d5740 10 client.28279 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.214+0000 7fc0f99d5740 10 client.28279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:21:17.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.214+0000 7fc0f99d5740 20 client.28279 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.214+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5566c81a80f0 0x5566c81aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.214+0000 7fc0f99d5740 4 client.28279 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.214+0000 7fc0f99d5740 1 -- 172.21.15.78:0/2379363408 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566c80fbdb0 con 0x5566c81a80f0 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.215+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5566c81a80f0 0x5566c81aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.215+0000 7fc0d8ff9640 10 client.28279 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.215+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5566c81a80f0 0x5566c81aa4e0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b549a57796f8c230 server_cookie=8e1b8fcdae901736 in_seq=0 out_seq=0 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.219+0000 7fc0d8ff9640 1 -- 172.21.15.78:0/2379363408 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5566c80fbdb0 con 0x5566c81a80f0 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.219+0000 7fc0d8ff9640 10 client.28279 handle_command_reply: tid=0 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.219+0000 7fc0f99d5740 1 client.28279 shutdown 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.219+0000 7fc0f99d5740 1 -- 172.21.15.78:0/2379363408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 msgr2=0x5566c818a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.220+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 0x5566c818a960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00096c0 tx=0x7fc0e0032e90 comp rx=0 tx=0).stop 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.220+0000 7fc0f99d5740 1 -- 172.21.15.78:0/2379363408 shutdown_connections 2024-09-11T01:21:17.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.220+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c8198050 0x5566c818aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.220+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 0x5566c818a960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.220+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5566c81a80f0 0x5566c81aa4e0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.220+0000 7fc0d8ff9640 0 client.28279 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.220+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/2379363408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c81830f0 0x5566c818a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.220+0000 7fc0f99d5740 1 -- 172.21.15.78:0/2379363408 >> 172.21.15.78:0/2379363408 conn(0x5566c8180f20 msgr2=0x5566c817f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.220+0000 7fc0f99d5740 1 -- 172.21.15.78:0/2379363408 shutdown_connections 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.220+0000 7fc0f99d5740 1 -- 172.21.15.78:0/2379363408 wait complete. 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.221+0000 7fc0f99d5740 20 client.28279 trim_cache size 0 max 16384 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.244+0000 7fc0f99d5740 1 Processor -- start 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.244+0000 7fc0f99d5740 1 -- start start 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.245+0000 7fc0f99d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 0x5566c817fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.245+0000 7fc0f99d5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c818d450 0x5566c81affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.245+0000 7fc0f99d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c81b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.245+0000 7fc0f5b9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 0x5566c817fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.245+0000 7fc0f5b9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 0x5566c817fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:39350/0 (socket says 172.21.15.78:39350) 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.245+0000 7fc0f5b9d640 1 -- 172.21.15.78:0/4219502750 learned_addr learned my addr 172.21.15.78:0/4219502750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.245+0000 7fc0f99d5740 1 -- 172.21.15.78:0/4219502750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01ac150 con 0x5566c8198050 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.245+0000 7fc0f4b9b640 1 --2- 172.21.15.78:0/4219502750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c818d450 0x5566c81affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.245+0000 7fc0f99d5740 1 -- 172.21.15.78:0/4219502750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01add50 con 0x5566c818d450 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.245+0000 7fc0f99d5740 1 -- 172.21.15.78:0/4219502750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01ac820 con 0x5566c81830f0 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.245+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/4219502750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c81b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.246+0000 7fc0f5b9d640 1 -- 172.21.15.78:0/4219502750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c818d450 msgr2=0x5566c81affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.246+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/4219502750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c818d450 0x5566c81affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.246+0000 7fc0f5b9d640 1 -- 172.21.15.78:0/4219502750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 msgr2=0x5566c81b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.246+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/4219502750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c81b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.246+0000 7fc0f5b9d640 1 -- 172.21.15.78:0/4219502750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c81893a0 con 0x5566c81830f0 2024-09-11T01:21:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.246+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/4219502750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 0x5566c817fe60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009d40 tx=0x7fc0e8004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=696b7d58fa387b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.247+0000 7fc0daffd640 1 -- 172.21.15.78:0/4219502750 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8035070 con 0x5566c81830f0 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.247+0000 7fc0daffd640 1 -- 172.21.15.78:0/4219502750 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e800fcf0 con 0x5566c81830f0 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.247+0000 7fc0daffd640 1 -- 172.21.15.78:0/4219502750 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e801e040 con 0x5566c81830f0 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.248+0000 7fc0f99d5740 1 -- 172.21.15.78:0/4219502750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 msgr2=0x5566c817fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.248+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/4219502750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 0x5566c817fe60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009d40 tx=0x7fc0e8004970 comp rx=0 tx=0).stop 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.248+0000 7fc0f99d5740 1 -- 172.21.15.78:0/4219502750 shutdown_connections 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.248+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/4219502750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c81b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.248+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/4219502750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c818d450 0x5566c81affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.248+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/4219502750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c81830f0 0x5566c817fe60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.248+0000 7fc0f99d5740 1 -- 172.21.15.78:0/4219502750 >> 172.21.15.78:0/4219502750 conn(0x5566c8180f20 msgr2=0x5566c817f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.248+0000 7fc0f99d5740 1 -- 172.21.15.78:0/4219502750 shutdown_connections 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.249+0000 7fc0f99d5740 1 -- 172.21.15.78:0/4219502750 wait complete. 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.249+0000 7fc0f99d5740 1 -- start start 2024-09-11T01:21:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.249+0000 7fc0f99d5740 1 Processor -- start 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f99d5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c81830f0 0x5566c81a0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f99d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 0x5566c81a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f99d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c8187420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f99d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01ad680 con 0x5566c8198050 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f99d5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01acef0 con 0x5566c81830f0 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f99d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01aa170 con 0x5566c818d450 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f99d5740 10 client.0 _ll_register_callbacks cb 0x7fc0e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f4b9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 0x5566c81a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f5b9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c8187420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f4b9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 0x5566c81a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:39352/0 (socket says 172.21.15.78:39352) 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f5b9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c8187420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37052/0 (socket says 172.21.15.78:37052) 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f4b9b640 1 -- 172.21.15.78:0/1299056227 learned_addr learned my addr 172.21.15.78:0/1299056227 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/1299056227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c81830f0 0x5566c81a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f4b9b640 1 -- 172.21.15.78:0/1299056227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c81830f0 msgr2=0x5566c81a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f4b9b640 1 --2- 172.21.15.78:0/1299056227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c81830f0 0x5566c81a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f4b9b640 1 -- 172.21.15.78:0/1299056227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 msgr2=0x5566c8187420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f4b9b640 1 --2- 172.21.15.78:0/1299056227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c8187420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.250+0000 7fc0f4b9b640 1 -- 172.21.15.78:0/1299056227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c8187c50 con 0x5566c818d450 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.251+0000 7fc0f4b9b640 1 --2- 172.21.15.78:0/1299056227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 0x5566c81a09a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5566c81872f0 tx=0x7fc0e00048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1502cb4e80d84765 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.251+0000 7fc0d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.251+0000 7fc0d8ff9640 1 -- 172.21.15.78:0/1299056227 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0035070 con 0x5566c818d450 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.251+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1299056227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e800fcf0 con 0x5566c818d450 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.251+0000 7fc0d8ff9640 1 -- 172.21.15.78:0/1299056227 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7fc0e0004d90 con 0x5566c818d450 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.251+0000 7fc0d8ff9640 1 -- 172.21.15.78:0/1299056227 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fc0e0004fb0 con 0x5566c818d450 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.251+0000 7fc0f99d5740 10 client.28285 fetch_fsmap learned FSMap version 72 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.251+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1299056227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x5566c8215be0 con 0x5566c818d450 2024-09-11T01:21:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.251+0000 7fc0d8ff9640 1 -- 172.21.15.78:0/1299056227 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e800fcf0 con 0x5566c818d450 2024-09-11T01:21:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.252+0000 7fc0d8ff9640 1 -- 172.21.15.78:0/1299056227 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7fc0e0057030 con 0x5566c818d450 2024-09-11T01:21:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.252+0000 7fc0f99d5740 10 client.28285 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:21:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.252+0000 7fc0f99d5740 10 client.28285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:21:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.253+0000 7fc0f99d5740 20 client.28285 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.253+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/1299056227 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5566c81a80f0 0x5566c82183d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.253+0000 7fc0f99d5740 4 client.28285 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:21:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.253+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1299056227 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5566c80f7860 con 0x5566c81a80f0 2024-09-11T01:21:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.254+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/1299056227 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5566c81a80f0 0x5566c82183d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.256+0000 7fc0f5b9d640 1 --2- 172.21.15.78:0/1299056227 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5566c81a80f0 0x5566c82183d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=159c9ca8636fda07 server_cookie=93c4eb4f7b581699 in_seq=0 out_seq=0 2024-09-11T01:21:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.256+0000 7fc0d8ff9640 10 client.28285 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.259+0000 7fc0d8ff9640 1 -- 172.21.15.78:0/1299056227 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5566c80f7860 con 0x5566c81a80f0 2024-09-11T01:21:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.259+0000 7fc0d8ff9640 10 client.28285 handle_command_reply: tid=0 2024-09-11T01:21:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.259+0000 7fc0f99d5740 1 client.28285 shutdown 2024-09-11T01:21:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.260+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1299056227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 msgr2=0x5566c81a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.260+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/1299056227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 0x5566c81a09a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5566c81872f0 tx=0x7fc0e00048f0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.261+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1299056227 shutdown_connections 2024-09-11T01:21:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.261+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/1299056227 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5566c81a80f0 0x5566c82183d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.261+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/1299056227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c8198050 0x5566c8187420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.261+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/1299056227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566c818d450 0x5566c81a09a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.261+0000 7fc0f99d5740 1 --2- 172.21.15.78:0/1299056227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566c81830f0 0x5566c81a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.261+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1299056227 >> 172.21.15.78:0/1299056227 conn(0x5566c8180f20 msgr2=0x5566c81affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.261+0000 7fc0d8ff9640 0 client.28285 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.261+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1299056227 shutdown_connections 2024-09-11T01:21:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.261+0000 7fc0f99d5740 1 -- 172.21.15.78:0/1299056227 wait complete. 2024-09-11T01:21:17.297 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:21:17.297 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:21:17.297 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "291fcaa4-1805-423d-a0bc-814f1bc2759f", 2024-09-11T01:21:17.297 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:21:17.297 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:21:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.261+0000 7fc0f99d5740 20 client.28285 trim_cache size 0 max 16384 2024-09-11T01:21:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.262+0000 7fc0f639e640 1 -- 172.21.15.78:0/3506130371 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc0dc0b5700 msgr2=0x7fc0dc0b7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.262+0000 7fc0f639e640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc0dc0b5700 0x7fc0dc0b7bc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5566c8182bd0 tx=0x7fc0ec076750 comp rx=0 tx=0).stop 2024-09-11T01:21:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.262+0000 7fc0f639e640 1 -- 172.21.15.78:0/3506130371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0f0101b00 msgr2=0x7fc0f0198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.262+0000 7fc0f639e640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0f0101b00 0x7fc0f0198fc0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00e9f0 tx=0x7fc0ec00eeb0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.262+0000 7fc0f639e640 1 -- 172.21.15.78:0/3506130371 shutdown_connections 2024-09-11T01:21:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.262+0000 7fc0f639e640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc0dc0b5700 0x7fc0dc0b7bc0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.262+0000 7fc0f639e640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f0107e90 0x7fc0f019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.262+0000 7fc0f639e640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0f0107510 0x7fc0f0199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.262+0000 7fc0f639e640 1 --2- 172.21.15.78:0/3506130371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0f0101b00 0x7fc0f0198fc0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.262+0000 7fc0f639e640 1 -- 172.21.15.78:0/3506130371 >> 172.21.15.78:0/3506130371 conn(0x7fc0f00fd5b0 msgr2=0x7fc0f0104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.263+0000 7fc0f639e640 1 -- 172.21.15.78:0/3506130371 shutdown_connections 2024-09-11T01:21:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:17.263+0000 7fc0f639e640 1 -- 172.21.15.78:0/3506130371 wait complete. 2024-09-11T01:21:17.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 347195 32.50 MB/sec execute 443 sec latency 588.084 ms 2024-09-11T01:21:18.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:18 smithi078 ceph-mon[31315]: pgmap v1027: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 596 op/s 2024-09-11T01:21:18.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:18 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:21:18.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:18.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:21:18.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:21:18.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:21:18.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:18 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:21:18.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:18 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:18.200 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:21:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:18 smithi110 ceph-mon[34712]: pgmap v1027: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 596 op/s 2024-09-11T01:21:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:18 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:21:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:18.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:21:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:21:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:21:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:18 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:21:18.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:18 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:18 smithi112 ceph-mon[35384]: pgmap v1027: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 596 op/s 2024-09-11T01:21:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:18 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:21:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:21:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:21:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:21:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:18 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:21:18.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:18 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:18.606 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:21:18.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 347725 32.47 MB/sec execute 444 sec latency 648.740 ms 2024-09-11T01:21:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.200+0000 7f743bd8c640 1 -- 172.21.15.78:0/3627634696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74341055a0 msgr2=0x7f7434105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.200+0000 7f743bd8c640 1 --2- 172.21.15.78:0/3627634696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74341055a0 0x7f7434105a00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7430009860 tx=0x7f743002a340 comp rx=0 tx=0).stop 2024-09-11T01:21:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.201+0000 7f743bd8c640 1 -- 172.21.15.78:0/3627634696 shutdown_connections 2024-09-11T01:21:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.201+0000 7f743bd8c640 1 --2- 172.21.15.78:0/3627634696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434105f40 0x7f743410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.201+0000 7f743bd8c640 1 --2- 172.21.15.78:0/3627634696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74341055a0 0x7f7434105a00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.201+0000 7f743bd8c640 1 --2- 172.21.15.78:0/3627634696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7434101250 0x7f7434101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.201+0000 7f743bd8c640 1 -- 172.21.15.78:0/3627634696 >> 172.21.15.78:0/3627634696 conn(0x7f7434078140 msgr2=0x7f74340ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.201+0000 7f743bd8c640 1 -- 172.21.15.78:0/3627634696 shutdown_connections 2024-09-11T01:21:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.201+0000 7f743bd8c640 1 -- 172.21.15.78:0/3627634696 wait complete. 2024-09-11T01:21:19.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f743bd8c640 1 Processor -- start 2024-09-11T01:21:19.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f743bd8c640 1 -- start start 2024-09-11T01:21:19.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f743bd8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7434101250 0x7f743419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f743bd8c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74341055a0 0x7f743419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f743bd8c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7434105f40 0x7f74341a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f743bd8c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7434112db0 con 0x7f74341055a0 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f743bd8c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7434112c30 con 0x7f7434105f40 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f743bd8c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7434112f30 con 0x7f7434101250 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f7439b28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7434101250 0x7f743419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f7439327640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74341055a0 0x7f743419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f7439b28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7434101250 0x7f743419b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:39362/0 (socket says 172.21.15.78:39362) 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f7439b28640 1 -- 172.21.15.78:0/2121121480 learned_addr learned my addr 172.21.15.78:0/2121121480 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.202+0000 7f743a329640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7434105f40 0x7f74341a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.203+0000 7f7439b28640 1 -- 172.21.15.78:0/2121121480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7434105f40 msgr2=0x7f74341a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.203+0000 7f7439b28640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7434105f40 0x7f74341a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.203+0000 7f7439b28640 1 -- 172.21.15.78:0/2121121480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74341055a0 msgr2=0x7f743419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.203+0000 7f7439b28640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74341055a0 0x7f743419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.203+0000 7f7439b28640 1 -- 172.21.15.78:0/2121121480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74341a1550 con 0x7f7434101250 2024-09-11T01:21:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.203+0000 7f743a329640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7434105f40 0x7f74341a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:21:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.203+0000 7f7439b28640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7434101250 0x7f743419b4a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f742400e9f0 tx=0x7f742400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.203+0000 7f742affd640 1 -- 172.21.15.78:0/2121121480 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742400ccc0 con 0x7f7434101250 2024-09-11T01:21:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.203+0000 7f742affd640 1 -- 172.21.15.78:0/2121121480 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74240054d0 con 0x7f7434101250 2024-09-11T01:21:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.203+0000 7f742affd640 1 -- 172.21.15.78:0/2121121480 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424010430 con 0x7f7434101250 2024-09-11T01:21:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.203+0000 7f743bd8c640 1 -- 172.21.15.78:0/2121121480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74341a1840 con 0x7f7434101250 2024-09-11T01:21:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.203+0000 7f743bd8c640 1 -- 172.21.15.78:0/2121121480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74341a7f00 con 0x7f7434101250 2024-09-11T01:21:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.206+0000 7f742affd640 1 -- 172.21.15.78:0/2121121480 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7424005020 con 0x7f7434101250 2024-09-11T01:21:19.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.206+0000 7f743e161740 1 Processor -- start 2024-09-11T01:21:19.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.206+0000 7f743e161740 1 -- start start 2024-09-11T01:21:19.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f743e161740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7d1480 0x55becd7d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f743e161740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1da0 0x55becd7dd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f743e161740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7ddd30 0x55becd7e0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f743e161740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55becd747e50 con 0x55becd7d1480 2024-09-11T01:21:19.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f743e161740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55becd737280 con 0x55becd7d1da0 2024-09-11T01:21:19.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f743e161740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55becd73b090 con 0x55becd7ddd30 2024-09-11T01:21:19.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f7439b28640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1da0 0x55becd7dd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f7439b28640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1da0 0x55becd7dd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38764/0 (socket says 172.21.15.78:38764) 2024-09-11T01:21:19.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f7439b28640 1 -- 172.21.15.78:0/734378507 learned_addr learned my addr 172.21.15.78:0/734378507 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f7439327640 1 --2- 172.21.15.78:0/734378507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7ddd30 0x55becd7e0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f743a329640 1 --2- 172.21.15.78:0/734378507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7d1480 0x55becd7d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f7439b28640 1 -- 172.21.15.78:0/734378507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7ddd30 msgr2=0x55becd7e0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f7439b28640 1 --2- 172.21.15.78:0/734378507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7ddd30 0x55becd7e0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f7439b28640 1 -- 172.21.15.78:0/734378507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7d1480 msgr2=0x55becd7d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f7439b28640 1 --2- 172.21.15.78:0/734378507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7d1480 0x55becd7d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.207+0000 7f7439b28640 1 -- 172.21.15.78:0/734378507 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55becd495650 con 0x55becd7d1da0 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.208+0000 7f7439b28640 1 --2- 172.21.15.78:0/734378507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1da0 0x55becd7dd7f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f74240e3a70 tx=0x7f74240ee8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce78be7878621c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.208+0000 7f742affd640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f74200b55a0 0x7f74200b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.208+0000 7f741ffff640 1 -- 172.21.15.78:0/734378507 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74240a8eb0 con 0x55becd7d1da0 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.208+0000 7f743a329640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f74200b55a0 0x7f74200b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.208+0000 7f741ffff640 1 -- 172.21.15.78:0/734378507 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74240fb030 con 0x55becd7d1da0 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.208+0000 7f741ffff640 1 -- 172.21.15.78:0/734378507 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74240abfd0 con 0x55becd7d1da0 2024-09-11T01:21:19.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.208+0000 7f742affd640 1 -- 172.21.15.78:0/2121121480 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7f74240e6400 con 0x7f7434101250 2024-09-11T01:21:19.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.209+0000 7f743e161740 1 -- 172.21.15.78:0/734378507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1da0 msgr2=0x55becd7dd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:19.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.209+0000 7f743a329640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f74200b55a0 0x7f74200b7a60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f742c005fd0 tx=0x7f742c009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:19.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.209+0000 7f743e161740 1 --2- 172.21.15.78:0/734378507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1da0 0x55becd7dd7f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f74240e3a70 tx=0x7f74240ee8c0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.209+0000 7f743e161740 1 -- 172.21.15.78:0/734378507 shutdown_connections 2024-09-11T01:21:19.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.209+0000 7f743e161740 1 --2- 172.21.15.78:0/734378507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7ddd30 0x55becd7e0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.209+0000 7f743e161740 1 --2- 172.21.15.78:0/734378507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1da0 0x55becd7dd7f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.209+0000 7f743e161740 1 --2- 172.21.15.78:0/734378507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7d1480 0x55becd7d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.209+0000 7f743e161740 1 -- 172.21.15.78:0/734378507 >> 172.21.15.78:0/734378507 conn(0x55becd7cf2b0 msgr2=0x55becd7d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:19.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.209+0000 7f743e161740 1 -- 172.21.15.78:0/734378507 shutdown_connections 2024-09-11T01:21:19.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.209+0000 7f743e161740 1 -- 172.21.15.78:0/734378507 wait complete. 2024-09-11T01:21:19.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.210+0000 7f743e161740 1 -- start start 2024-09-11T01:21:19.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.210+0000 7f743e161740 1 Processor -- start 2024-09-11T01:21:19.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.210+0000 7f743e161740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7dd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.210+0000 7f743e161740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 0x55becd7daac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.210+0000 7f743e161740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7ddd30 0x55becd7db140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.210+0000 7f743e161740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55becd598700 con 0x55becd7ddd30 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.210+0000 7f743e161740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55becd6d2110 con 0x55becd7d1480 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.210+0000 7f743e161740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55becd7e09b0 con 0x55becd7d1da0 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.210+0000 7f743e161740 10 client.0 _ll_register_callbacks cb 0x55becd6eb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.211+0000 7f7439b28640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7dd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.211+0000 7f7439327640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 0x55becd7daac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.211+0000 7f7439b28640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7dd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38766/0 (socket says 172.21.15.78:38766) 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.211+0000 7f7439327640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 0x55becd7daac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:39380/0 (socket says 172.21.15.78:39380) 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.211+0000 7f7439b28640 1 -- 172.21.15.78:0/1324601971 learned_addr learned my addr 172.21.15.78:0/1324601971 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.211+0000 7f743a329640 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7ddd30 0x55becd7db140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.211+0000 7f7439b28640 1 -- 172.21.15.78:0/1324601971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 msgr2=0x55becd7daac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.211+0000 7f7439b28640 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 0x55becd7daac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.211+0000 7f7439b28640 1 -- 172.21.15.78:0/1324601971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7ddd30 msgr2=0x55becd7db140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.211+0000 7f7439b28640 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7ddd30 0x55becd7db140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.211+0000 7f7439b28640 1 -- 172.21.15.78:0/1324601971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55becd6ffc40 con 0x55becd7d1480 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.212+0000 7f7439b28640 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7dd670 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7424009230 tx=0x7f7424005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beb229434aa048cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.212+0000 7f741d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.212+0000 7f743a329640 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7ddd30 0x55becd7db140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.213+0000 7f741d7fa640 1 -- 172.21.15.78:0/1324601971 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74240ac1b0 con 0x55becd7d1480 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.213+0000 7f743e161740 1 -- 172.21.15.78:0/1324601971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74240fb030 con 0x55becd7d1480 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.213+0000 7f741d7fa640 1 -- 172.21.15.78:0/1324601971 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7f74240f9d50 con 0x55becd7d1480 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.213+0000 7f741d7fa640 1 -- 172.21.15.78:0/1324601971 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f74240ac3a0 con 0x55becd7d1480 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.213+0000 7f743e161740 10 client.28346 fetch_fsmap learned FSMap version 72 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.213+0000 7f743e161740 1 -- 172.21.15.78:0/1324601971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55becd7ec330 con 0x55becd7d1480 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.213+0000 7f741d7fa640 1 -- 172.21.15.78:0/1324601971 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7424005680 con 0x55becd7d1480 2024-09-11T01:21:19.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.214+0000 7f741d7fa640 1 -- 172.21.15.78:0/1324601971 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f74240e3db0 con 0x55becd7d1480 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.214+0000 7f743e161740 10 client.28346 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.214+0000 7f743e161740 10 client.28346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.214+0000 7f743e161740 20 client.28346 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.214+0000 7f743e161740 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55becd7ed880 0x55becd7efca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.215+0000 7f743e161740 4 client.28346 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.215+0000 7f743e161740 1 -- 172.21.15.78:0/1324601971 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f74240fb030 con 0x55becd7ed880 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.215+0000 7f7439327640 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55becd7ed880 0x55becd7efca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.216+0000 7f741d7fa640 10 client.28346 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.216+0000 7f7439327640 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55becd7ed880 0x55becd7efca0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed5626afb0c8b64 server_cookie=f3d4d75ee5bfaab5 in_seq=0 out_seq=0 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.216+0000 7f741d7fa640 1 -- 172.21.15.78:0/1324601971 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f74240fb030 con 0x55becd7ed880 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.216+0000 7f741d7fa640 10 client.28346 handle_command_reply: tid=0 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.216+0000 7f743e161740 1 client.28346 shutdown 2024-09-11T01:21:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.217+0000 7f743e161740 1 -- 172.21.15.78:0/1324601971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 msgr2=0x55becd7dd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.217+0000 7f743e161740 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7dd670 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7424009230 tx=0x7f7424005a20 comp rx=0 tx=0).stop 2024-09-11T01:21:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.218+0000 7f743e161740 1 -- 172.21.15.78:0/1324601971 shutdown_connections 2024-09-11T01:21:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.218+0000 7f743e161740 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55becd7ed880 0x55becd7efca0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.218+0000 7f743e161740 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7ddd30 0x55becd7db140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.218+0000 7f741d7fa640 0 client.28346 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.218+0000 7f743e161740 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 0x55becd7daac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.218+0000 7f743e161740 1 --2- 172.21.15.78:0/1324601971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7dd670 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.218+0000 7f743e161740 1 -- 172.21.15.78:0/1324601971 >> 172.21.15.78:0/1324601971 conn(0x55becd7cf2b0 msgr2=0x55becd7d4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.218+0000 7f743e161740 1 -- 172.21.15.78:0/1324601971 shutdown_connections 2024-09-11T01:21:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.218+0000 7f743e161740 1 -- 172.21.15.78:0/1324601971 wait complete. 2024-09-11T01:21:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.218+0000 7f743e161740 20 client.28346 trim_cache size 0 max 16384 2024-09-11T01:21:19.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.242+0000 7f743e161740 1 Processor -- start 2024-09-11T01:21:19.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.242+0000 7f743e161740 1 -- start start 2024-09-11T01:21:19.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f743e161740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7e9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f743e161740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 0x55becd7df120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f743e161740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7ddd30 0x55becd7df660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f743e161740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f74341ac540 con 0x55becd7ddd30 2024-09-11T01:21:19.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f743e161740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f74341ae140 con 0x55becd7d1480 2024-09-11T01:21:19.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f743e161740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74341acc10 con 0x55becd7d1da0 2024-09-11T01:21:19.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f7439327640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 0x55becd7df120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f7439327640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 0x55becd7df120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:39382/0 (socket says 172.21.15.78:39382) 2024-09-11T01:21:19.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f7439327640 1 -- 172.21.15.78:0/1612903829 learned_addr learned my addr 172.21.15.78:0/1612903829 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:19.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f743a329640 1 --2- 172.21.15.78:0/1612903829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7ddd30 0x55becd7df660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f7439327640 1 -- 172.21.15.78:0/1612903829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 msgr2=0x55becd7e9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:19.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f7439327640 1 --2- 172.21.15.78:0/1612903829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7e9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f7439327640 1 -- 172.21.15.78:0/1612903829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7ddd30 msgr2=0x55becd7df660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:19.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f7439327640 1 --2- 172.21.15.78:0/1612903829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7ddd30 0x55becd7df660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f7439327640 1 -- 172.21.15.78:0/1612903829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55becd7db4b0 con 0x55becd7d1da0 2024-09-11T01:21:19.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.243+0000 7f743a329640 1 --2- 172.21.15.78:0/1612903829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7ddd30 0x55becd7df660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:21:19.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.244+0000 7f7439327640 1 --2- 172.21.15.78:0/1612903829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 0x55becd7df120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f74300057c0 tx=0x7f74300058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18569d6e5b99680f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:19.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.244+0000 7f741f7fe640 1 -- 172.21.15.78:0/1612903829 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743004e070 con 0x55becd7d1da0 2024-09-11T01:21:19.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.244+0000 7f741f7fe640 1 -- 172.21.15.78:0/1612903829 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7430030e50 con 0x55becd7d1da0 2024-09-11T01:21:19.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.244+0000 7f741f7fe640 1 -- 172.21.15.78:0/1612903829 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430037040 con 0x55becd7d1da0 2024-09-11T01:21:19.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.245+0000 7f743e161740 1 -- 172.21.15.78:0/1612903829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 msgr2=0x55becd7df120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:19.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.245+0000 7f743e161740 1 --2- 172.21.15.78:0/1612903829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 0x55becd7df120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f74300057c0 tx=0x7f74300058d0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.246+0000 7f743e161740 1 -- 172.21.15.78:0/1612903829 shutdown_connections 2024-09-11T01:21:19.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.246+0000 7f743e161740 1 --2- 172.21.15.78:0/1612903829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7ddd30 0x55becd7df660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.246+0000 7f743e161740 1 --2- 172.21.15.78:0/1612903829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7d1da0 0x55becd7df120 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.246+0000 7f743e161740 1 --2- 172.21.15.78:0/1612903829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7e9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.246+0000 7f743e161740 1 -- 172.21.15.78:0/1612903829 >> 172.21.15.78:0/1612903829 conn(0x55becd7cf2b0 msgr2=0x55becd7d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:19.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.246+0000 7f743e161740 1 -- 172.21.15.78:0/1612903829 shutdown_connections 2024-09-11T01:21:19.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.246+0000 7f743e161740 1 -- 172.21.15.78:0/1612903829 wait complete. 2024-09-11T01:21:19.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.247+0000 7f743e161740 1 -- start start 2024-09-11T01:21:19.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.247+0000 7f743e161740 1 Processor -- start 2024-09-11T01:21:19.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f743e161740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7e6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f743e161740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7d1da0 0x55becd7f5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f743e161740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7ddd30 0x55becd7f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f743e161740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f74341ada70 con 0x55becd7d1da0 2024-09-11T01:21:19.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f743e161740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f74341ad2e0 con 0x55becd7d1480 2024-09-11T01:21:19.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f743e161740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74341aa580 con 0x55becd7ddd30 2024-09-11T01:21:19.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f743e161740 10 client.0 _ll_register_callbacks cb 0x7f742c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:19.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f7439b28640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7e6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f7439b28640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7e6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:38790/0 (socket says 172.21.15.78:38790) 2024-09-11T01:21:19.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f7439b28640 1 -- 172.21.15.78:0/2344859999 learned_addr learned my addr 172.21.15.78:0/2344859999 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f743a329640 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7ddd30 0x55becd7f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f7439327640 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7d1da0 0x55becd7f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f7439b28640 1 -- 172.21.15.78:0/2344859999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7ddd30 msgr2=0x55becd7f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f7439b28640 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7ddd30 0x55becd7f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f7439b28640 1 -- 172.21.15.78:0/2344859999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7d1da0 msgr2=0x55becd7f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f7439b28640 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7d1da0 0x55becd7f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.248+0000 7f7439b28640 1 -- 172.21.15.78:0/2344859999 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55becd7f5ff0 con 0x55becd7d1480 2024-09-11T01:21:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.249+0000 7f7439b28640 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7e6450 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f742400c6a0 tx=0x7f74240b1270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c7858dfb059232 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.249+0000 7f741d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:21:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.249+0000 7f741d7fa640 1 -- 172.21.15.78:0/2344859999 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74240ac630 con 0x55becd7d1480 2024-09-11T01:21:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.249+0000 7f741d7fa640 1 -- 172.21.15.78:0/2344859999 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7f74240acac0 con 0x55becd7d1480 2024-09-11T01:21:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.249+0000 7f743e161740 1 -- 172.21.15.78:0/2344859999 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7430030e50 con 0x55becd7d1480 2024-09-11T01:21:19.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.250+0000 7f741d7fa640 1 -- 172.21.15.78:0/2344859999 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f74240a81b0 con 0x55becd7d1480 2024-09-11T01:21:19.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.250+0000 7f743e161740 10 client.28349 fetch_fsmap learned FSMap version 72 2024-09-11T01:21:19.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.250+0000 7f743e161740 1 -- 172.21.15.78:0/2344859999 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55becd85a860 con 0x55becd7d1480 2024-09-11T01:21:19.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.250+0000 7f741d7fa640 1 -- 172.21.15.78:0/2344859999 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74240fb070 con 0x55becd7d1480 2024-09-11T01:21:19.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.250+0000 7f741d7fa640 1 -- 172.21.15.78:0/2344859999 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 72) ==== 8378+0+0 (secure 0 0 0) 0x7f74240adb90 con 0x55becd7d1480 2024-09-11T01:21:19.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.250+0000 7f743e161740 10 client.28349 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T01:21:19.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.250+0000 7f743e161740 10 client.28349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:21:19.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.250+0000 7f743e161740 20 client.28349 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:19.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.250+0000 7f743e161740 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55becd7ed880 0x55becd85ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:19.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.250+0000 7f743e161740 4 client.28349 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:21:19.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.250+0000 7f743e161740 1 -- 172.21.15.78:0/2344859999 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f7430030e50 con 0x55becd7ed880 2024-09-11T01:21:19.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.250+0000 7f7439327640 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55becd7ed880 0x55becd85ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:19.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.251+0000 7f741d7fa640 10 client.28349 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:19.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.251+0000 7f7439327640 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55becd7ed880 0x55becd85ce00 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfa9c4ca06c9f7f3 server_cookie=2e915641d529f8b in_seq=0 out_seq=0 2024-09-11T01:21:19.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f741d7fa640 1 -- 172.21.15.78:0/2344859999 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7954 (crc 0 0 0) 0x7f7430030e50 con 0x55becd7ed880 2024-09-11T01:21:19.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f741d7fa640 10 client.28349 handle_command_reply: tid=0 2024-09-11T01:21:19.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f743e161740 1 client.28349 shutdown 2024-09-11T01:21:19.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f743e161740 1 -- 172.21.15.78:0/2344859999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 msgr2=0x55becd7e6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:19.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f743e161740 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7e6450 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f742400c6a0 tx=0x7f74240b1270 comp rx=0 tx=0).stop 2024-09-11T01:21:19.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f743e161740 1 -- 172.21.15.78:0/2344859999 shutdown_connections 2024-09-11T01:21:19.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f743e161740 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55becd7ddd30 0x55becd7f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f743e161740 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55becd7ed880 0x55becd85ce00 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f743e161740 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becd7d1da0 0x55becd7f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f743e161740 1 --2- 172.21.15.78:0/2344859999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55becd7d1480 0x55becd7e6450 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f741d7fa640 0 client.28349 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:19.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f743e161740 1 -- 172.21.15.78:0/2344859999 >> 172.21.15.78:0/2344859999 conn(0x55becd7cf2b0 msgr2=0x55becd7dc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:19.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.252+0000 7f743e161740 1 -- 172.21.15.78:0/2344859999 shutdown_connections 2024-09-11T01:21:19.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.253+0000 7f743e161740 1 -- 172.21.15.78:0/2344859999 wait complete. 2024-09-11T01:21:19.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.253+0000 7f743e161740 20 client.28349 trim_cache size 0 max 16384 2024-09-11T01:21:19.257 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: "291fcaa4-1805-423d-a0bc-814f1bc2759f": { 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "291fcaa4-1805-423d-a0bc-814f1bc2759f", 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: "746add9a-5be3-4745-8000-04e5471bc813": { 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:20:12.369593+0000", 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: "b41a2b77-b2b1-4f1b-8588-4f32a78b721f": { 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:19:13.257610+0000", 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:21:19.259 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:18:17.654570+0000", 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.260 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.261 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.262 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.263 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.264 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.265 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.266 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.267 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:21:19.267 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: "291fcaa4-1805-423d-a0bc-814f1bc2759f": { 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:21:17.256483+0000", 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.268 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.269 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:21:19.270 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.272 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.273 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:21:19.274 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:21:19.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.255+0000 7f743bd8c640 1 -- 172.21.15.78:0/2121121480 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f74200b55a0 msgr2=0x7f74200b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:19.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.255+0000 7f743bd8c640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f74200b55a0 0x7f74200b7a60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f742c005fd0 tx=0x7f742c009500 comp rx=0 tx=0).stop 2024-09-11T01:21:19.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.255+0000 7f743bd8c640 1 -- 172.21.15.78:0/2121121480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7434101250 msgr2=0x7f743419b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:19.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.255+0000 7f743bd8c640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7434101250 0x7f743419b4a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f742400e9f0 tx=0x7f742400eeb0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.255+0000 7f743bd8c640 1 -- 172.21.15.78:0/2121121480 shutdown_connections 2024-09-11T01:21:19.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.255+0000 7f743bd8c640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f74200b55a0 0x7f74200b7a60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.255+0000 7f743bd8c640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7434105f40 0x7f74341a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.255+0000 7f743bd8c640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74341055a0 0x7f743419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.255+0000 7f743bd8c640 1 --2- 172.21.15.78:0/2121121480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7434101250 0x7f743419b4a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:19.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.255+0000 7f743bd8c640 1 -- 172.21.15.78:0/2121121480 >> 172.21.15.78:0/2121121480 conn(0x7f7434078140 msgr2=0x7f7434108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:19.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.255+0000 7f743bd8c640 1 -- 172.21.15.78:0/2121121480 shutdown_connections 2024-09-11T01:21:19.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:19.256+0000 7f743bd8c640 1 -- 172.21.15.78:0/2121121480 wait complete. 2024-09-11T01:21:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:19 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:21:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:19 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:19 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:19 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:19.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:19 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:19 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:21:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:19 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:19 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:19 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:19.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:19 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:19.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:19 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:21:19.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:19 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:19.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:19 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:19.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:19 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:19.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:19 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:21:19.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 348355 32.48 MB/sec execute 445 sec latency 163.846 ms 2024-09-11T01:21:20.035 INFO:tasks.cephfs.filesystem:scrub status for tag:291fcaa4-1805-423d-a0bc-814f1bc2759f - {'path': '/', 'tag': '291fcaa4-1805-423d-a0bc-814f1bc2759f', 'options': 'recursive,force'} 2024-09-11T01:21:20.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:20 smithi110 ceph-mon[34712]: pgmap v1028: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 639 op/s 2024-09-11T01:21:20.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:20 smithi112 ceph-mon[35384]: pgmap v1028: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 639 op/s 2024-09-11T01:21:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:20 smithi078 ceph-mon[31315]: pgmap v1028: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 639 op/s 2024-09-11T01:21:20.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 349014 32.49 MB/sec execute 446 sec latency 260.520 ms 2024-09-11T01:21:21.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 349644 32.51 MB/sec execute 447 sec latency 79.096 ms 2024-09-11T01:21:22.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:22 smithi110 ceph-mon[34712]: pgmap v1029: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 664 op/s 2024-09-11T01:21:22.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:22 smithi112 ceph-mon[35384]: pgmap v1029: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 664 op/s 2024-09-11T01:21:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:22 smithi078 ceph-mon[31315]: pgmap v1029: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 664 op/s 2024-09-11T01:21:22.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 350169 32.49 MB/sec execute 448 sec latency 73.809 ms 2024-09-11T01:21:23.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 350674 32.48 MB/sec execute 449 sec latency 180.150 ms 2024-09-11T01:21:24.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:24 smithi110 ceph-mon[34712]: pgmap v1030: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 644 op/s 2024-09-11T01:21:24.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:24 smithi112 ceph-mon[35384]: pgmap v1030: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 644 op/s 2024-09-11T01:21:24.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:24 smithi078 ceph-mon[31315]: pgmap v1030: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 644 op/s 2024-09-11T01:21:24.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 351346 32.47 MB/sec execute 450 sec latency 76.676 ms 2024-09-11T01:21:25.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 351983 32.48 MB/sec execute 451 sec latency 142.035 ms 2024-09-11T01:21:26.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:26 smithi110 ceph-mon[34712]: pgmap v1031: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 692 op/s 2024-09-11T01:21:26.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:26 smithi112 ceph-mon[35384]: pgmap v1031: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 692 op/s 2024-09-11T01:21:26.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:26 smithi078 ceph-mon[31315]: pgmap v1031: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 692 op/s 2024-09-11T01:21:26.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 352738 32.52 MB/sec execute 452 sec latency 91.886 ms 2024-09-11T01:21:27.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 353129 32.50 MB/sec execute 453 sec latency 440.317 ms 2024-09-11T01:21:28.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:28 smithi110 ceph-mon[34712]: pgmap v1032: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 623 op/s 2024-09-11T01:21:28.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:28 smithi112 ceph-mon[35384]: pgmap v1032: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 623 op/s 2024-09-11T01:21:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:28 smithi078 ceph-mon[31315]: pgmap v1032: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 623 op/s 2024-09-11T01:21:28.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 353268 32.44 MB/sec execute 454 sec latency 649.580 ms 2024-09-11T01:21:29.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 353652 32.40 MB/sec execute 455 sec latency 130.810 ms 2024-09-11T01:21:30.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:30 smithi110 ceph-mon[34712]: pgmap v1033: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 592 op/s 2024-09-11T01:21:30.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:30 smithi112 ceph-mon[35384]: pgmap v1033: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 592 op/s 2024-09-11T01:21:30.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:30 smithi078 ceph-mon[31315]: pgmap v1033: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 592 op/s 2024-09-11T01:21:30.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 354097 32.37 MB/sec execute 456 sec latency 212.434 ms 2024-09-11T01:21:31.803 INFO:tasks.workunit.client.0.smithi078.stdout: 10 354845 32.38 MB/sec execute 457 sec latency 99.641 ms 2024-09-11T01:21:32.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:32 smithi110 ceph-mon[34712]: pgmap v1034: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 692 op/s 2024-09-11T01:21:32.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:32 smithi112 ceph-mon[35384]: pgmap v1034: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 692 op/s 2024-09-11T01:21:32.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:32 smithi078 ceph-mon[31315]: pgmap v1034: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 692 op/s 2024-09-11T01:21:32.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 355635 32.41 MB/sec execute 458 sec latency 77.803 ms 2024-09-11T01:21:33.398 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:21:33.398 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:21:33.398 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:21:33.398 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:21:33.398 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:21:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:21:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:21:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:21:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:21:33.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:21:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:21:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:21:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:21:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:21:33.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:21:33.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 356454 32.46 MB/sec execute 459 sec latency 45.711 ms 2024-09-11T01:21:34.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:34 smithi078 ceph-mon[31315]: pgmap v1035: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 726 op/s 2024-09-11T01:21:34.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:34 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:21:34.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:34 smithi110 ceph-mon[34712]: pgmap v1035: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 726 op/s 2024-09-11T01:21:34.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:34 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:21:34.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:34 smithi112 ceph-mon[35384]: pgmap v1035: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 726 op/s 2024-09-11T01:21:34.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:34 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:21:34.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 356873 32.44 MB/sec execute 460 sec latency 105.652 ms 2024-09-11T01:21:35.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 357385 32.42 MB/sec execute 461 sec latency 84.039 ms 2024-09-11T01:21:36.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:36 smithi110 ceph-mon[34712]: pgmap v1036: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 19 MiB/s wr, 715 op/s 2024-09-11T01:21:36.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:36 smithi112 ceph-mon[35384]: pgmap v1036: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 19 MiB/s wr, 715 op/s 2024-09-11T01:21:36.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:36 smithi078 ceph-mon[31315]: pgmap v1036: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 19 MiB/s wr, 715 op/s 2024-09-11T01:21:36.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 357992 32.40 MB/sec execute 462 sec latency 135.278 ms 2024-09-11T01:21:37.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 358724 32.42 MB/sec execute 463 sec latency 100.893 ms 2024-09-11T01:21:38.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:38 smithi110 ceph-mon[34712]: pgmap v1037: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 709 op/s 2024-09-11T01:21:38.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:38 smithi112 ceph-mon[35384]: pgmap v1037: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 709 op/s 2024-09-11T01:21:38.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:38 smithi078 ceph-mon[31315]: pgmap v1037: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 709 op/s 2024-09-11T01:21:38.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 359491 32.47 MB/sec execute 464 sec latency 73.468 ms 2024-09-11T01:21:39.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 360122 32.46 MB/sec execute 465 sec latency 54.023 ms 2024-09-11T01:21:40.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:40 smithi110 ceph-mon[34712]: pgmap v1038: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 18 MiB/s wr, 692 op/s 2024-09-11T01:21:40.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:40 smithi112 ceph-mon[35384]: pgmap v1038: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 18 MiB/s wr, 692 op/s 2024-09-11T01:21:40.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:40 smithi078 ceph-mon[31315]: pgmap v1038: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 18 MiB/s wr, 692 op/s 2024-09-11T01:21:40.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 360723 32.45 MB/sec execute 466 sec latency 73.823 ms 2024-09-11T01:21:41.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 361280 32.45 MB/sec execute 467 sec latency 118.351 ms 2024-09-11T01:21:42.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:42 smithi110 ceph-mon[34712]: pgmap v1039: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 19 MiB/s wr, 761 op/s 2024-09-11T01:21:42.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:42 smithi112 ceph-mon[35384]: pgmap v1039: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 19 MiB/s wr, 761 op/s 2024-09-11T01:21:42.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:42 smithi078 ceph-mon[31315]: pgmap v1039: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 19 MiB/s wr, 761 op/s 2024-09-11T01:21:42.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 361905 32.45 MB/sec execute 468 sec latency 108.125 ms 2024-09-11T01:21:43.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 362559 32.45 MB/sec execute 469 sec latency 98.885 ms 2024-09-11T01:21:44.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:44 smithi110 ceph-mon[34712]: pgmap v1040: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 635 op/s 2024-09-11T01:21:44.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:44 smithi112 ceph-mon[35384]: pgmap v1040: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 635 op/s 2024-09-11T01:21:44.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:44 smithi078 ceph-mon[31315]: pgmap v1040: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 635 op/s 2024-09-11T01:21:44.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 363319 32.48 MB/sec execute 470 sec latency 138.921 ms 2024-09-11T01:21:45.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 363882 32.47 MB/sec execute 471 sec latency 150.404 ms 2024-09-11T01:21:46.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:46 smithi110 ceph-mon[34712]: pgmap v1041: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 559 op/s 2024-09-11T01:21:46.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:46 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:21:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:46 smithi112 ceph-mon[35384]: pgmap v1041: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 559 op/s 2024-09-11T01:21:46.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:46 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:21:46.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:46 smithi078 ceph-mon[31315]: pgmap v1041: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 559 op/s 2024-09-11T01:21:46.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:46 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:21:46.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 364433 32.47 MB/sec execute 472 sec latency 186.621 ms 2024-09-11T01:21:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:21:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:21:47.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:21:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:21:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:21:47.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:21:47.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:21:47.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:21:47.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:21:47.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 364996 32.46 MB/sec execute 473 sec latency 149.321 ms 2024-09-11T01:21:48.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:48 smithi110 ceph-mon[34712]: pgmap v1042: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 519 op/s 2024-09-11T01:21:48.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:48 smithi112 ceph-mon[35384]: pgmap v1042: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 519 op/s 2024-09-11T01:21:48.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:48 smithi078 ceph-mon[31315]: pgmap v1042: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 519 op/s 2024-09-11T01:21:48.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 365606 32.46 MB/sec execute 474 sec latency 157.362 ms 2024-09-11T01:21:49.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 366163 32.46 MB/sec execute 475 sec latency 122.684 ms 2024-09-11T01:21:50.036 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:21:50.363 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:21:50.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:50 smithi110 ceph-mon[34712]: pgmap v1043: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 514 op/s 2024-09-11T01:21:50.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:50 smithi112 ceph-mon[35384]: pgmap v1043: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 514 op/s 2024-09-11T01:21:50.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:50 smithi078 ceph-mon[31315]: pgmap v1043: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 514 op/s 2024-09-11T01:21:50.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 366795 32.45 MB/sec execute 476 sec latency 74.874 ms 2024-09-11T01:21:51.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.072+0000 7f3facc95640 1 -- 172.21.15.78:0/547388411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fa8102dc0 msgr2=0x7f3fa81031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:51.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.072+0000 7f3facc95640 1 --2- 172.21.15.78:0/547388411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fa8102dc0 0x7f3fa81031a0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009900 tx=0x7f3f9802a410 comp rx=0 tx=0).stop 2024-09-11T01:21:51.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.072+0000 7f3facc95640 1 -- 172.21.15.78:0/547388411 shutdown_connections 2024-09-11T01:21:51.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.072+0000 7f3facc95640 1 --2- 172.21.15.78:0/547388411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fa8109150 0x7f3fa810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.072+0000 7f3facc95640 1 --2- 172.21.15.78:0/547388411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa81087d0 0x7f3fa8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.072+0000 7f3facc95640 1 --2- 172.21.15.78:0/547388411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fa8102dc0 0x7f3fa81031a0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.072+0000 7f3facc95640 1 -- 172.21.15.78:0/547388411 >> 172.21.15.78:0/547388411 conn(0x7f3fa80fe870 msgr2=0x7f3fa8100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:51.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.072+0000 7f3facc95640 1 -- 172.21.15.78:0/547388411 shutdown_connections 2024-09-11T01:21:51.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.072+0000 7f3facc95640 1 -- 172.21.15.78:0/547388411 wait complete. 2024-09-11T01:21:51.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.073+0000 7f3facc95640 1 Processor -- start 2024-09-11T01:21:51.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.073+0000 7f3facc95640 1 -- start start 2024-09-11T01:21:51.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3facc95640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fa8102dc0 0x7f3fa819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3facc95640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fa81087d0 0x7f3fa819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3facc95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa8109150 0x7f3fa81a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3facc95640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8112e10 con 0x7f3fa81087d0 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3facc95640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8112c90 con 0x7f3fa8102dc0 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3facc95640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8112f90 con 0x7f3fa8109150 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fa81087d0 0x7f3fa819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fa8102dc0 0x7f3fa819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fa81087d0 0x7f3fa819ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47808/0 (socket says 172.21.15.78:47808) 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3f9dd9b640 1 -- 172.21.15.78:0/975757713 learned_addr learned my addr 172.21.15.78:0/975757713 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fa8102dc0 0x7f3fa819b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43614/0 (socket says 172.21.15.78:43614) 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3fa6d9d640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa8109150 0x7f3fa81a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3f9dd9b640 1 -- 172.21.15.78:0/975757713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa8109150 msgr2=0x7f3fa81a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3f9dd9b640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa8109150 0x7f3fa81a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3f9dd9b640 1 -- 172.21.15.78:0/975757713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fa8102dc0 msgr2=0x7f3fa819b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3f9dd9b640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fa8102dc0 0x7f3fa819b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.074+0000 7f3f9dd9b640 1 -- 172.21.15.78:0/975757713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa81a15c0 con 0x7f3fa81087d0 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.075+0000 7f3f9dd9b640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fa81087d0 0x7f3fa819ba50 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000e9f0 tx=0x7f3f9000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.075+0000 7f3f9ffff640 1 -- 172.21.15.78:0/975757713 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9000cd80 con 0x7f3fa81087d0 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.075+0000 7f3facc95640 1 -- 172.21.15.78:0/975757713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa81a18b0 con 0x7f3fa81087d0 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.075+0000 7f3f9ffff640 1 -- 172.21.15.78:0/975757713 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f900054d0 con 0x7f3fa81087d0 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.075+0000 7f3f9ffff640 1 -- 172.21.15.78:0/975757713 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90010690 con 0x7f3fa81087d0 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.075+0000 7f3facc95640 1 -- 172.21.15.78:0/975757713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa81a7f70 con 0x7f3fa81087d0 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.079+0000 7f3faf06a740 1 Processor -- start 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.080+0000 7f3f9ffff640 1 -- 172.21.15.78:0/975757713 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3f90004e90 con 0x7f3fa81087d0 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.080+0000 7f3faf06a740 1 -- start start 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3faf06a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bd7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3faf06a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3faf06a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6be3540 0x5589f6be5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3faf06a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5589f6b44e50 con 0x5589f6bd75b0 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3faf06a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5589f6b34280 con 0x5589f6bd6c90 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3faf06a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589f6b38090 con 0x5589f6be3540 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47820/0 (socket says 172.21.15.78:47820) 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3fa659c640 1 -- 172.21.15.78:0/3689811659 learned_addr learned my addr 172.21.15.78:0/3689811659 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3f9dd9b640 1 --2- 172.21.15.78:0/3689811659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6be3540 0x5589f6be5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3fa6d9d640 1 --2- 172.21.15.78:0/3689811659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bd7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3fa659c640 1 -- 172.21.15.78:0/3689811659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6be3540 msgr2=0x5589f6be5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3fa659c640 1 --2- 172.21.15.78:0/3689811659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6be3540 0x5589f6be5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3fa659c640 1 -- 172.21.15.78:0/3689811659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 msgr2=0x5589f6bd7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3fa659c640 1 --2- 172.21.15.78:0/3689811659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bd7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.081+0000 7f3fa659c640 1 -- 172.21.15.78:0/3689811659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589f6892650 con 0x5589f6bd75b0 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.082+0000 7f3fa659c640 1 --2- 172.21.15.78:0/3689811659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be3000 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009900 tx=0x7f3f98032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cb2ee57bbb55d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:51.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.082+0000 7f3f9cd99640 1 -- 172.21.15.78:0/3689811659 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98040070 con 0x5589f6bd75b0 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.082+0000 7f3f9cd99640 1 -- 172.21.15.78:0/3689811659 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f98004030 con 0x5589f6bd75b0 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.082+0000 7f3f9cd99640 1 -- 172.21.15.78:0/3689811659 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98035040 con 0x5589f6bd75b0 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.083+0000 7f3f9ffff640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3f940b54d0 0x7f3f940b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.083+0000 7f3faf06a740 1 -- 172.21.15.78:0/3689811659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 msgr2=0x5589f6be3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.083+0000 7f3faf06a740 1 --2- 172.21.15.78:0/3689811659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be3000 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009900 tx=0x7f3f98032bd0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.083+0000 7f3faf06a740 1 -- 172.21.15.78:0/3689811659 shutdown_connections 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.083+0000 7f3faf06a740 1 --2- 172.21.15.78:0/3689811659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6be3540 0x5589f6be5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.083+0000 7f3faf06a740 1 --2- 172.21.15.78:0/3689811659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be3000 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.083+0000 7f3faf06a740 1 --2- 172.21.15.78:0/3689811659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bd7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.083+0000 7f3faf06a740 1 -- 172.21.15.78:0/3689811659 >> 172.21.15.78:0/3689811659 conn(0x5589f6bd4ac0 msgr2=0x5589f6bd5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.083+0000 7f3fa6d9d640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3f940b54d0 0x7f3f940b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.083+0000 7f3faf06a740 1 -- 172.21.15.78:0/3689811659 shutdown_connections 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.083+0000 7f3faf06a740 1 -- 172.21.15.78:0/3689811659 wait complete. 2024-09-11T01:21:51.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.084+0000 7f3f9ffff640 1 -- 172.21.15.78:0/975757713 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7f3f900e6380 con 0x7f3fa81087d0 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.084+0000 7f3fa6d9d640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3f940b54d0 0x7f3f940b7990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0005fd0 tx=0x7f3fa00096a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.084+0000 7f3faf06a740 1 -- start start 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.084+0000 7f3faf06a740 1 Processor -- start 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.084+0000 7f3faf06a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6bd6c90 0x5589f6be2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.084+0000 7f3faf06a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.084+0000 7f3faf06a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6be3540 0x5589f6be08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3faf06a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5589f6995700 con 0x5589f6bd75b0 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3faf06a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5589f6acf110 con 0x5589f6be3540 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3faf06a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589f6be6160 con 0x5589f6bd6c90 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6bd6c90 0x5589f6be2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47832/0 (socket says 172.21.15.78:47832) 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3faf06a740 10 client.0 _ll_register_callbacks cb 0x5589f6ae8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3f9dd9b640 1 -- 172.21.15.78:0/1156079077 learned_addr learned my addr 172.21.15.78:0/1156079077 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:51.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3fa6d9d640 1 --2- 172.21.15.78:0/1156079077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6be3540 0x5589f6be08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3fa659c640 1 -- 172.21.15.78:0/1156079077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6be3540 msgr2=0x5589f6be08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3fa659c640 1 --2- 172.21.15.78:0/1156079077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6be3540 0x5589f6be08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3fa659c640 1 -- 172.21.15.78:0/1156079077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 msgr2=0x5589f6be02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3fa659c640 1 --2- 172.21.15.78:0/1156079077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3fa659c640 1 -- 172.21.15.78:0/1156079077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589f6afcc40 con 0x5589f6bd6c90 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3fa659c640 1 --2- 172.21.15.78:0/1156079077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6bd6c90 0x5589f6be2e50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009850 tx=0x7f3f9802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca9e0efc735eead0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3f8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.085+0000 7f3f8dffb640 1 -- 172.21.15.78:0/1156079077 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98035070 con 0x5589f6bd6c90 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.086+0000 7f3faf06a740 1 -- 172.21.15.78:0/1156079077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f98004030 con 0x5589f6bd6c90 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.086+0000 7f3f8dffb640 1 -- 172.21.15.78:0/1156079077 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7f3f980041e0 con 0x5589f6bd6c90 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.086+0000 7f3f8dffb640 1 -- 172.21.15.78:0/1156079077 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f3f98040040 con 0x5589f6bd6c90 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.086+0000 7f3faf06a740 10 client.28315 fetch_fsmap learned FSMap version 73 2024-09-11T01:21:51.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.086+0000 7f3faf06a740 1 -- 172.21.15.78:0/1156079077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5589f6bcbcc0 con 0x5589f6bd6c90 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.086+0000 7f3f8dffb640 1 -- 172.21.15.78:0/1156079077 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f98004030 con 0x5589f6bd6c90 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.086+0000 7f3f8dffb640 1 -- 172.21.15.78:0/1156079077 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 73) ==== 8366+0+0 (secure 0 0 0) 0x7f3f98003ab0 con 0x5589f6bd6c90 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.086+0000 7f3faf06a740 10 client.28315 fetch_fsmap finished waiting for FSMap version 73 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.086+0000 7f3faf06a740 10 client.28315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.086+0000 7f3faf06a740 20 client.28315 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.086+0000 7f3faf06a740 1 --2- 172.21.15.78:0/1156079077 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5589f6bcd260 0x5589f6bcf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.087+0000 7f3faf06a740 4 client.28315 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.087+0000 7f3faf06a740 1 -- 172.21.15.78:0/1156079077 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589f6b46da0 con 0x5589f6bcd260 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.087+0000 7f3f9dd9b640 1 --2- 172.21.15.78:0/1156079077 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5589f6bcd260 0x5589f6bcf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.087+0000 7f3f9dd9b640 1 --2- 172.21.15.78:0/1156079077 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5589f6bcd260 0x5589f6bcf6f0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=318714d870b7862f server_cookie=6471294970d83b05 in_seq=0 out_seq=0 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.087+0000 7f3f8dffb640 10 client.28315 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.089+0000 7f3f8dffb640 1 -- 172.21.15.78:0/1156079077 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589f6b46da0 con 0x5589f6bcd260 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.089+0000 7f3f8dffb640 10 client.28315 handle_command_reply: tid=0 2024-09-11T01:21:51.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.089+0000 7f3faf06a740 1 client.28315 shutdown 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.089+0000 7f3faf06a740 1 -- 172.21.15.78:0/1156079077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6bd6c90 msgr2=0x5589f6be2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.089+0000 7f3faf06a740 1 --2- 172.21.15.78:0/1156079077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6bd6c90 0x5589f6be2e50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009850 tx=0x7f3f9802a410 comp rx=0 tx=0).stop 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.089+0000 7f3faf06a740 1 -- 172.21.15.78:0/1156079077 shutdown_connections 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.089+0000 7f3faf06a740 1 --2- 172.21.15.78:0/1156079077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6be3540 0x5589f6be08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.089+0000 7f3faf06a740 1 --2- 172.21.15.78:0/1156079077 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5589f6bcd260 0x5589f6bcf6f0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.089+0000 7f3faf06a740 1 --2- 172.21.15.78:0/1156079077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.089+0000 7f3faf06a740 1 --2- 172.21.15.78:0/1156079077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6bd6c90 0x5589f6be2e50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.089+0000 7f3faf06a740 1 -- 172.21.15.78:0/1156079077 >> 172.21.15.78:0/1156079077 conn(0x5589f6bd4ac0 msgr2=0x5589f6bda660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.090+0000 7f3f8dffb640 0 client.28315 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.090+0000 7f3faf06a740 1 -- 172.21.15.78:0/1156079077 shutdown_connections 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.090+0000 7f3faf06a740 1 -- 172.21.15.78:0/1156079077 wait complete. 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.090+0000 7f3faf06a740 20 client.28315 trim_cache size 0 max 16384 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.119+0000 7f3faf06a740 1 Processor -- start 2024-09-11T01:21:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.119+0000 7f3faf06a740 1 -- start start 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3faf06a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bef460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3faf06a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3faf06a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6be3540 0x5589f6be4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3faf06a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81ac3a0 con 0x5589f6bd75b0 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3faf06a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81adf10 con 0x5589f6bd6c90 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3faf06a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81aca70 con 0x5589f6be3540 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3fa6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6be3540 0x5589f6be4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bef460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43652/0 (socket says 172.21.15.78:43652) 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3fa6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6be3540 0x5589f6be4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53342/0 (socket says 172.21.15.78:53342) 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3fa659c640 1 -- 172.21.15.78:0/2465017203 learned_addr learned my addr 172.21.15.78:0/2465017203 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3fa659c640 1 -- 172.21.15.78:0/2465017203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6be3540 msgr2=0x5589f6be4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3fa659c640 1 --2- 172.21.15.78:0/2465017203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6be3540 0x5589f6be4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3fa659c640 1 -- 172.21.15.78:0/2465017203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 msgr2=0x5589f6be4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3fa659c640 1 --2- 172.21.15.78:0/2465017203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.120+0000 7f3fa659c640 1 -- 172.21.15.78:0/2465017203 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589f6be1560 con 0x5589f6bd6c90 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.121+0000 7f3fa659c640 1 --2- 172.21.15.78:0/2465017203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bef460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3f980097a0 tx=0x7f3f980097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a96d2cabd8b1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.121+0000 7f3f8ffff640 1 -- 172.21.15.78:0/2465017203 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9805a040 con 0x5589f6bd6c90 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.121+0000 7f3f8ffff640 1 -- 172.21.15.78:0/2465017203 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f980314a0 con 0x5589f6bd6c90 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.121+0000 7f3f8ffff640 1 -- 172.21.15.78:0/2465017203 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98035040 con 0x5589f6bd6c90 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.122+0000 7f3faf06a740 1 -- 172.21.15.78:0/2465017203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 msgr2=0x5589f6bef460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.122+0000 7f3faf06a740 1 --2- 172.21.15.78:0/2465017203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bef460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3f980097a0 tx=0x7f3f980097d0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.122+0000 7f3faf06a740 1 -- 172.21.15.78:0/2465017203 shutdown_connections 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.122+0000 7f3faf06a740 1 --2- 172.21.15.78:0/2465017203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6be3540 0x5589f6be4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.122+0000 7f3faf06a740 1 --2- 172.21.15.78:0/2465017203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6bd75b0 0x5589f6be4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.122+0000 7f3faf06a740 1 --2- 172.21.15.78:0/2465017203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bef460 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.122+0000 7f3faf06a740 1 -- 172.21.15.78:0/2465017203 >> 172.21.15.78:0/2465017203 conn(0x5589f6bd4ac0 msgr2=0x5589f6bd5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.122+0000 7f3faf06a740 1 -- 172.21.15.78:0/2465017203 shutdown_connections 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.122+0000 7f3faf06a740 1 -- 172.21.15.78:0/2465017203 wait complete. 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.122+0000 7f3faf06a740 1 -- start start 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.122+0000 7f3faf06a740 1 Processor -- start 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3faf06a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bf6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3faf06a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6bd75b0 0x5589f6bf7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3faf06a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6be3540 0x5589f6bf2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3faf06a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81ad840 con 0x5589f6be3540 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3faf06a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81ad140 con 0x5589f6bd6c90 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3faf06a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81aa3a0 con 0x5589f6bd75b0 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3faf06a740 10 client.0 _ll_register_callbacks cb 0x7f3fa0009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:21:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bf6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3fa6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6be3540 0x5589f6bf2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6bd75b0 0x5589f6bf7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3fa659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bf6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:43654/0 (socket says 172.21.15.78:43654) 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3fa659c640 1 -- 172.21.15.78:0/1336376197 learned_addr learned my addr 172.21.15.78:0/1336376197 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3fa659c640 1 -- 172.21.15.78:0/1336376197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6bd75b0 msgr2=0x5589f6bf7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3fa659c640 1 --2- 172.21.15.78:0/1336376197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6bd75b0 0x5589f6bf7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3fa659c640 1 -- 172.21.15.78:0/1336376197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6be3540 msgr2=0x5589f6bf2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3fa659c640 1 --2- 172.21.15.78:0/1336376197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6be3540 0x5589f6bf2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3fa659c640 1 -- 172.21.15.78:0/1336376197 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589f6bf2e70 con 0x5589f6bd6c90 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.123+0000 7f3f9dd9b640 1 --2- 172.21.15.78:0/1336376197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6bd75b0 0x5589f6bf7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.124+0000 7f3fa659c640 1 --2- 172.21.15.78:0/1336376197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bf6e80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009850 tx=0x7f3f98003ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22f77ddb40f21978 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.124+0000 7f3f8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.124+0000 7f3f8dffb640 1 -- 172.21.15.78:0/1336376197 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98063030 con 0x5589f6bd6c90 2024-09-11T01:21:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.124+0000 7f3f8dffb640 1 -- 172.21.15.78:0/1336376197 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7f3f98004770 con 0x5589f6bd6c90 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.124+0000 7f3faf06a740 1 -- 172.21.15.78:0/1336376197 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f980314a0 con 0x5589f6bd6c90 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.124+0000 7f3f8dffb640 1 -- 172.21.15.78:0/1336376197 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f3f9806d050 con 0x5589f6bd6c90 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.124+0000 7f3faf06a740 10 client.28367 fetch_fsmap learned FSMap version 73 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.124+0000 7f3faf06a740 1 -- 172.21.15.78:0/1336376197 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5589f6c57940 con 0x5589f6bd6c90 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.124+0000 7f3f8dffb640 1 -- 172.21.15.78:0/1336376197 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f98031650 con 0x5589f6bd6c90 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.125+0000 7f3f8dffb640 1 -- 172.21.15.78:0/1336376197 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 73) ==== 8366+0+0 (secure 0 0 0) 0x7f3f98031b10 con 0x5589f6bd6c90 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.125+0000 7f3faf06a740 10 client.28367 fetch_fsmap finished waiting for FSMap version 73 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.125+0000 7f3faf06a740 10 client.28367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.125+0000 7f3faf06a740 20 client.28367 populate_metadata read hostname 'smithi078' 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.125+0000 7f3faf06a740 1 --2- 172.21.15.78:0/1336376197 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5589f6c57d70 0x5589f6c5a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.125+0000 7f3faf06a740 4 client.28367 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.125+0000 7f3faf06a740 1 -- 172.21.15.78:0/1336376197 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3f980314a0 con 0x5589f6c57d70 2024-09-11T01:21:51.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.134+0000 7f3f9dd9b640 1 --2- 172.21.15.78:0/1336376197 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5589f6c57d70 0x5589f6c5a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.135+0000 7f3f9dd9b640 1 --2- 172.21.15.78:0/1336376197 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5589f6c57d70 0x5589f6c5a190 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc4f95d528d58da server_cookie=3ce411cf4f0b163d in_seq=0 out_seq=0 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.135+0000 7f3f8dffb640 10 client.28367 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.135+0000 7f3f8dffb640 1 -- 172.21.15.78:0/1336376197 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7758 (crc 0 0 0) 0x7f3f980314a0 con 0x5589f6c57d70 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.135+0000 7f3f8dffb640 10 client.28367 handle_command_reply: tid=0 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.135+0000 7f3faf06a740 1 client.28367 shutdown 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.135+0000 7f3faf06a740 1 -- 172.21.15.78:0/1336376197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 msgr2=0x5589f6bf6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.135+0000 7f3faf06a740 1 --2- 172.21.15.78:0/1336376197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bf6e80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009850 tx=0x7f3f98003ed0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.136+0000 7f3faf06a740 1 -- 172.21.15.78:0/1336376197 shutdown_connections 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.136+0000 7f3faf06a740 1 --2- 172.21.15.78:0/1336376197 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5589f6c57d70 0x5589f6c5a190 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.136+0000 7f3faf06a740 1 --2- 172.21.15.78:0/1336376197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f6be3540 0x5589f6bf2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.136+0000 7f3f8dffb640 0 client.28367 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.136+0000 7f3faf06a740 1 --2- 172.21.15.78:0/1336376197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589f6bd75b0 0x5589f6bf7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.136+0000 7f3faf06a740 1 --2- 172.21.15.78:0/1336376197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589f6bd6c90 0x5589f6bf6e80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.136+0000 7f3faf06a740 1 -- 172.21.15.78:0/1336376197 >> 172.21.15.78:0/1336376197 conn(0x5589f6bd4ac0 msgr2=0x5589f6bcabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:51.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.136+0000 7f3faf06a740 1 -- 172.21.15.78:0/1336376197 shutdown_connections 2024-09-11T01:21:51.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.136+0000 7f3faf06a740 1 -- 172.21.15.78:0/1336376197 wait complete. 2024-09-11T01:21:51.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.136+0000 7f3faf06a740 20 client.28367 trim_cache size 0 max 16384 2024-09-11T01:21:51.373 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:21:51.373 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:21:51.373 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:21:51.373 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:21:51.373 INFO:teuthology.orchestra.run.smithi078.stdout: "746add9a-5be3-4745-8000-04e5471bc813": { 2024-09-11T01:21:51.373 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:20:12.369593+0000", 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "b41a2b77-b2b1-4f1b-8588-4f32a78b721f": { 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:19:13.257610+0000", 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.374 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:18:17.654570+0000", 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.375 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.376 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:21:51.377 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:21:51.378 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.379 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.380 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:21:51.381 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "291fcaa4-1805-423d-a0bc-814f1bc2759f": { 2024-09-11T01:21:51.382 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:21:17.256483+0000", 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:21:51.385 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:21:51.388 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:21:51.389 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:21:51.389 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:21:51.389 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:21:51.389 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:21:51.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.137+0000 7f3facc95640 1 -- 172.21.15.78:0/975757713 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3f940b54d0 msgr2=0x7f3f940b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:51.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.137+0000 7f3facc95640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3f940b54d0 0x7f3f940b7990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0005fd0 tx=0x7f3fa00096a0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.137+0000 7f3facc95640 1 -- 172.21.15.78:0/975757713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fa81087d0 msgr2=0x7f3fa819ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:51.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.137+0000 7f3facc95640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fa81087d0 0x7f3fa819ba50 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000e9f0 tx=0x7f3f9000edf0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.137+0000 7f3facc95640 1 -- 172.21.15.78:0/975757713 shutdown_connections 2024-09-11T01:21:51.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.137+0000 7f3facc95640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f3f940b54d0 0x7f3f940b7990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.137+0000 7f3facc95640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa8109150 0x7f3fa81a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.137+0000 7f3facc95640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fa81087d0 0x7f3fa819ba50 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.137+0000 7f3facc95640 1 --2- 172.21.15.78:0/975757713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fa8102dc0 0x7f3fa819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.137+0000 7f3facc95640 1 -- 172.21.15.78:0/975757713 >> 172.21.15.78:0/975757713 conn(0x7f3fa80fe870 msgr2=0x7f3fa8075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.137+0000 7f3facc95640 1 -- 172.21.15.78:0/975757713 shutdown_connections 2024-09-11T01:21:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:51.137+0000 7f3facc95640 1 -- 172.21.15.78:0/975757713 wait complete. 2024-09-11T01:21:51.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 367434 32.46 MB/sec execute 477 sec latency 90.511 ms 2024-09-11T01:21:52.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:52 smithi110 ceph-mon[34712]: pgmap v1044: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 573 op/s 2024-09-11T01:21:52.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:52 smithi112 ceph-mon[35384]: pgmap v1044: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 573 op/s 2024-09-11T01:21:52.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:52 smithi078 ceph-mon[31315]: pgmap v1044: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 573 op/s 2024-09-11T01:21:52.560 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:21:52.560 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:21:52.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 368107 32.48 MB/sec execute 478 sec latency 129.395 ms 2024-09-11T01:21:52.921 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:21:53.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 368288 32.43 MB/sec execute 479 sec latency 763.356 ms 2024-09-11T01:21:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:54 smithi110 ceph-mon[34712]: pgmap v1045: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 535 op/s 2024-09-11T01:21:54.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:54 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:21:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:54 smithi112 ceph-mon[35384]: pgmap v1045: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 535 op/s 2024-09-11T01:21:54.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:54 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:21:54.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:54 smithi078 ceph-mon[31315]: pgmap v1045: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 535 op/s 2024-09-11T01:21:54.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:54 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:21:54.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 368392 32.37 MB/sec execute 480 sec latency 1428.915 ms 2024-09-11T01:21:55.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 368625 32.33 MB/sec execute 481 sec latency 878.703 ms 2024-09-11T01:21:56.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:56 smithi110 ceph-mon[34712]: pgmap v1046: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 554 op/s 2024-09-11T01:21:56.473 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:56 smithi112 ceph-mon[35384]: pgmap v1046: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 554 op/s 2024-09-11T01:21:56.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:56 smithi078 ceph-mon[31315]: pgmap v1046: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 554 op/s 2024-09-11T01:21:56.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 368983 32.30 MB/sec execute 482 sec latency 131.289 ms 2024-09-11T01:21:57.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 369554 32.30 MB/sec execute 483 sec latency 88.308 ms 2024-09-11T01:21:58.070 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:21:58.070 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":74,"btime":"2024-09-11T01:21:53:096049+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":74,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:21:52.258249+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:21:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.266+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/4104664090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e41111b0 msgr2=0x7fb6e41135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.266+0000 7fb6ea3e2640 1 --2- 172.21.15.78:0/4104664090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e41111b0 0x7fb6e41135a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009900 tx=0x7fb6d402a410 comp rx=0 tx=0).stop 2024-09-11T01:21:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.266+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/4104664090 shutdown_connections 2024-09-11T01:21:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.266+0000 7fb6ea3e2640 1 --2- 172.21.15.78:0/4104664090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e4113ae0 0x7fb6e4115ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.266+0000 7fb6ea3e2640 1 --2- 172.21.15.78:0/4104664090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e41111b0 0x7fb6e41135a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.266+0000 7fb6ea3e2640 1 --2- 172.21.15.78:0/4104664090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e410a3b0 0x7fb6e410a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.266+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/4104664090 >> 172.21.15.78:0/4104664090 conn(0x7fb6e406cf20 msgr2=0x7fb6e406d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.266+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/4104664090 shutdown_connections 2024-09-11T01:21:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.266+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/4104664090 wait complete. 2024-09-11T01:21:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6ea3e2640 1 Processor -- start 2024-09-11T01:21:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6ea3e2640 1 -- start start 2024-09-11T01:21:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6ea3e2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e410a3b0 0x7fb6e419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6ea3e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e41111b0 0x7fb6e419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6ea3e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e4113ae0 0x7fb6e41a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6e9be1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e4113ae0 0x7fb6e41a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6e93e0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e410a3b0 0x7fb6e419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6e9be1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e4113ae0 0x7fb6e41a4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33704/0 (socket says 172.21.15.78:33704) 2024-09-11T01:21:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6e8bdf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e41111b0 0x7fb6e419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6e9be1640 1 -- 172.21.15.78:0/1917172546 learned_addr learned my addr 172.21.15.78:0/1917172546 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:21:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41191d0 con 0x7fb6e4113ae0 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4119050 con 0x7fb6e410a3b0 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.267+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4119350 con 0x7fb6e41111b0 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.268+0000 7fb6e9be1640 1 -- 172.21.15.78:0/1917172546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e41111b0 msgr2=0x7fb6e419fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.268+0000 7fb6e9be1640 1 --2- 172.21.15.78:0/1917172546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e41111b0 0x7fb6e419fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.268+0000 7fb6e9be1640 1 -- 172.21.15.78:0/1917172546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e410a3b0 msgr2=0x7fb6e419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.268+0000 7fb6e9be1640 1 --2- 172.21.15.78:0/1917172546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e410a3b0 0x7fb6e419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.268+0000 7fb6e9be1640 1 -- 172.21.15.78:0/1917172546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e41a56b0 con 0x7fb6e4113ae0 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.268+0000 7fb6e9be1640 1 --2- 172.21.15.78:0/1917172546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e4113ae0 0x7fb6e41a4ee0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00fbc0 tx=0x7fb6dc00d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.268+0000 7fb6da7fc640 1 -- 172.21.15.78:0/1917172546 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc008b60 con 0x7fb6e4113ae0 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.268+0000 7fb6da7fc640 1 -- 172.21.15.78:0/1917172546 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6dc008d00 con 0x7fb6e4113ae0 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.268+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e41a5940 con 0x7fb6e4113ae0 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.268+0000 7fb6da7fc640 1 -- 172.21.15.78:0/1917172546 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00a610 con 0x7fb6e4113ae0 2024-09-11T01:21:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.268+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e41ac060 con 0x7fb6e4113ae0 2024-09-11T01:21:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.272+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e410a790 con 0x7fb6e4113ae0 2024-09-11T01:21:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.272+0000 7fb6da7fc640 1 -- 172.21.15.78:0/1917172546 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb6dc00dbb0 con 0x7fb6e4113ae0 2024-09-11T01:21:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.274+0000 7fb6da7fc640 1 --2- 172.21.15.78:0/1917172546 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb6d00b5500 0x7fb6d00b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:21:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.274+0000 7fb6da7fc640 1 -- 172.21.15.78:0/1917172546 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7fb6dc0e6cd0 con 0x7fb6e4113ae0 2024-09-11T01:21:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.277+0000 7fb6e93e0640 1 --2- 172.21.15.78:0/1917172546 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb6d00b5500 0x7fb6d00b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:21:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.282+0000 7fb6e93e0640 1 --2- 172.21.15.78:0/1917172546 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb6d00b5500 0x7fb6d00b79c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009bd0 tx=0x7fb6e0005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:21:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.285+0000 7fb6da7fc640 1 -- 172.21.15.78:0/1917172546 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb6dc011040 con 0x7fb6e4113ae0 2024-09-11T01:21:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.659+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6e41ac350 con 0x7fb6e4113ae0 2024-09-11T01:21:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.660+0000 7fb6da7fc640 1 -- 172.21.15.78:0/1917172546 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+11979 (secure 0 0 0) 0x7fb6dc0ad890 con 0x7fb6e4113ae0 2024-09-11T01:21:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 74 2024-09-11T01:21:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.662+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb6d00b5500 msgr2=0x7fb6d00b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.662+0000 7fb6ea3e2640 1 --2- 172.21.15.78:0/1917172546 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb6d00b5500 0x7fb6d00b79c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009bd0 tx=0x7fb6e0005f90 comp rx=0 tx=0).stop 2024-09-11T01:21:58.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.662+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e4113ae0 msgr2=0x7fb6e41a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:21:58.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.662+0000 7fb6ea3e2640 1 --2- 172.21.15.78:0/1917172546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e4113ae0 0x7fb6e41a4ee0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00fbc0 tx=0x7fb6dc00d490 comp rx=0 tx=0).stop 2024-09-11T01:21:58.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.662+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 shutdown_connections 2024-09-11T01:21:58.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.662+0000 7fb6ea3e2640 1 --2- 172.21.15.78:0/1917172546 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb6d00b5500 0x7fb6d00b79c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:58.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.662+0000 7fb6ea3e2640 1 --2- 172.21.15.78:0/1917172546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e4113ae0 0x7fb6e41a4ee0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:58.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.662+0000 7fb6ea3e2640 1 --2- 172.21.15.78:0/1917172546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e41111b0 0x7fb6e419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:58.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.662+0000 7fb6ea3e2640 1 --2- 172.21.15.78:0/1917172546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e410a3b0 0x7fb6e419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:21:58.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.662+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 >> 172.21.15.78:0/1917172546 conn(0x7fb6e406cf20 msgr2=0x7fb6e4113ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:21:58.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.663+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 shutdown_connections 2024-09-11T01:21:58.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:21:57.663+0000 7fb6ea3e2640 1 -- 172.21.15.78:0/1917172546 wait complete. 2024-09-11T01:21:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:58 smithi110 ceph-mon[34712]: pgmap v1047: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 593 op/s 2024-09-11T01:21:58.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:21:58 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1917172546' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:21:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:58 smithi112 ceph-mon[35384]: pgmap v1047: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 593 op/s 2024-09-11T01:21:58.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:21:58 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1917172546' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:21:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:58 smithi078 ceph-mon[31315]: pgmap v1047: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 593 op/s 2024-09-11T01:21:58.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:21:58 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1917172546' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:21:58.804 INFO:tasks.workunit.client.0.smithi078.stdout: 10 370005 32.27 MB/sec execute 484 sec latency 268.458 ms 2024-09-11T01:21:59.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 370563 32.27 MB/sec execute 485 sec latency 115.536 ms 2024-09-11T01:22:00.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:00 smithi110 ceph-mon[34712]: pgmap v1048: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 634 op/s 2024-09-11T01:22:00.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:00 smithi112 ceph-mon[35384]: pgmap v1048: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 634 op/s 2024-09-11T01:22:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:00 smithi078 ceph-mon[31315]: pgmap v1048: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 634 op/s 2024-09-11T01:22:00.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 370881 32.25 MB/sec execute 486 sec latency 290.779 ms 2024-09-11T01:22:01.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 371026 32.19 MB/sec execute 487 sec latency 684.267 ms 2024-09-11T01:22:02.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:02 smithi110 ceph-mon[34712]: pgmap v1049: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 21 MiB/s wr, 764 op/s 2024-09-11T01:22:02.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:02 smithi112 ceph-mon[35384]: pgmap v1049: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 21 MiB/s wr, 764 op/s 2024-09-11T01:22:02.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:02 smithi078 ceph-mon[31315]: pgmap v1049: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 21 MiB/s wr, 764 op/s 2024-09-11T01:22:02.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 371026 32.12 MB/sec execute 488 sec latency 1684.366 ms 2024-09-11T01:22:03.398 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:03 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:22:03.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:03 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:22:03.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:03 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:22:03.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 371026 32.06 MB/sec execute 489 sec latency 2684.435 ms 2024-09-11T01:22:04.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:04 smithi110 ceph-mon[34712]: pgmap v1050: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 21 MiB/s wr, 774 op/s 2024-09-11T01:22:04.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:04 smithi112 ceph-mon[35384]: pgmap v1050: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 21 MiB/s wr, 774 op/s 2024-09-11T01:22:04.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:04 smithi078 ceph-mon[31315]: pgmap v1050: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 21 MiB/s wr, 774 op/s 2024-09-11T01:22:04.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 371131 32.01 MB/sec execute 490 sec latency 3511.834 ms 2024-09-11T01:22:05.530 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:22:05.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 371394 31.96 MB/sec execute 491 sec latency 213.949 ms 2024-09-11T01:22:05.968 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:22:06.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:06 smithi110 ceph-mon[34712]: pgmap v1051: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 711 op/s 2024-09-11T01:22:06.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:06 smithi112 ceph-mon[35384]: pgmap v1051: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 711 op/s 2024-09-11T01:22:06.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:06 smithi078 ceph-mon[31315]: pgmap v1051: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 711 op/s 2024-09-11T01:22:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.704+0000 7ff440050640 1 -- 172.21.15.78:0/311183810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438101a90 msgr2=0x7ff438101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.704+0000 7ff440050640 1 --2- 172.21.15.78:0/311183810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438101a90 0x7ff438101e70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff434009900 tx=0x7ff43402a420 comp rx=0 tx=0).stop 2024-09-11T01:22:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.704+0000 7ff440050640 1 -- 172.21.15.78:0/311183810 shutdown_connections 2024-09-11T01:22:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.704+0000 7ff440050640 1 --2- 172.21.15.78:0/311183810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff438106ed0 0x7ff438107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.704+0000 7ff440050640 1 --2- 172.21.15.78:0/311183810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4381023b0 0x7ff438106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.704+0000 7ff440050640 1 --2- 172.21.15.78:0/311183810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438101a90 0x7ff438101e70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.704+0000 7ff440050640 1 -- 172.21.15.78:0/311183810 >> 172.21.15.78:0/311183810 conn(0x7ff4380fd700 msgr2=0x7ff4380ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.705+0000 7ff440050640 1 -- 172.21.15.78:0/311183810 shutdown_connections 2024-09-11T01:22:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.705+0000 7ff440050640 1 -- 172.21.15.78:0/311183810 wait complete. 2024-09-11T01:22:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.705+0000 7ff440050640 1 Processor -- start 2024-09-11T01:22:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.706+0000 7ff440050640 1 -- start start 2024-09-11T01:22:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.706+0000 7ff440050640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff438101a90 0x7ff43819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.706+0000 7ff440050640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4381023b0 0x7ff43819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.706+0000 7ff440050640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438106ed0 0x7ff4381a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.706+0000 7ff440050640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff438119390 con 0x7ff4381023b0 2024-09-11T01:22:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.706+0000 7ff43ddec640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff438101a90 0x7ff43819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.706+0000 7ff43ddec640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff438101a90 0x7ff43819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:35636/0 (socket says 172.21.15.78:35636) 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.706+0000 7ff43d5eb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4381023b0 0x7ff43819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.706+0000 7ff43ddec640 1 -- 172.21.15.78:0/4068577763 learned_addr learned my addr 172.21.15.78:0/4068577763 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.706+0000 7ff440050640 1 -- 172.21.15.78:0/4068577763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff438119210 con 0x7ff438101a90 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.706+0000 7ff440050640 1 -- 172.21.15.78:0/4068577763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff438119510 con 0x7ff438106ed0 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.706+0000 7ff43e5ed640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438106ed0 0x7ff4381a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.707+0000 7ff43ddec640 1 -- 172.21.15.78:0/4068577763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438106ed0 msgr2=0x7ff4381a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.707+0000 7ff43ddec640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438106ed0 0x7ff4381a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.707+0000 7ff43ddec640 1 -- 172.21.15.78:0/4068577763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4381023b0 msgr2=0x7ff43819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.707+0000 7ff43ddec640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4381023b0 0x7ff43819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.707+0000 7ff43ddec640 1 -- 172.21.15.78:0/4068577763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4381a5770 con 0x7ff438101a90 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.707+0000 7ff43d5eb640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4381023b0 0x7ff43819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.707+0000 7ff43ddec640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff438101a90 0x7ff43819f6c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff434009850 tx=0x7ff4340305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.707+0000 7ff42effd640 1 -- 172.21.15.78:0/4068577763 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43403f070 con 0x7ff438101a90 2024-09-11T01:22:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.707+0000 7ff440050640 1 -- 172.21.15.78:0/4068577763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4381a5a00 con 0x7ff438101a90 2024-09-11T01:22:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.707+0000 7ff440050640 1 -- 172.21.15.78:0/4068577763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4381ac120 con 0x7ff438101a90 2024-09-11T01:22:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.707+0000 7ff42effd640 1 -- 172.21.15.78:0/4068577763 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff434004460 con 0x7ff438101a90 2024-09-11T01:22:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.707+0000 7ff42effd640 1 -- 172.21.15.78:0/4068577763 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434035030 con 0x7ff438101a90 2024-09-11T01:22:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.711+0000 7ff42effd640 1 -- 172.21.15.78:0/4068577763 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff434003cd0 con 0x7ff438101a90 2024-09-11T01:22:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.711+0000 7ff442425740 1 Processor -- start 2024-09-11T01:22:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff442425740 1 -- start start 2024-09-11T01:22:06.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff442425740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b5c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff442425740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b5cdf0 0x5582f1b68840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff442425740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b68d80 0x5582f1b6b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff442425740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5582f1ad2ea0 con 0x5582f1b5c4d0 2024-09-11T01:22:06.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff442425740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5582f1ac22d0 con 0x5582f1b5cdf0 2024-09-11T01:22:06.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff442425740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5582f1ac60e0 con 0x5582f1b68d80 2024-09-11T01:22:06.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff43e5ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b5c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff43e5ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b5c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35154/0 (socket says 172.21.15.78:35154) 2024-09-11T01:22:06.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff43e5ed640 1 -- 172.21.15.78:0/1001948751 learned_addr learned my addr 172.21.15.78:0/1001948751 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:06.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff43ddec640 1 --2- 172.21.15.78:0/1001948751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b5cdf0 0x5582f1b68840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff43d5eb640 1 --2- 172.21.15.78:0/1001948751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b68d80 0x5582f1b6b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff42effd640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff4240b5580 0x7ff4240b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff43e5ed640 1 -- 172.21.15.78:0/1001948751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b68d80 msgr2=0x5582f1b6b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.712+0000 7ff43e5ed640 1 --2- 172.21.15.78:0/1001948751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b68d80 0x5582f1b6b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.713+0000 7ff43e5ed640 1 -- 172.21.15.78:0/1001948751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b5cdf0 msgr2=0x5582f1b68840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.713+0000 7ff43e5ed640 1 --2- 172.21.15.78:0/1001948751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b5cdf0 0x5582f1b68840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.713+0000 7ff43e5ed640 1 -- 172.21.15.78:0/1001948751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f1820650 con 0x5582f1b5c4d0 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.713+0000 7ff43d5eb640 1 --2- 172.21.15.78:0/1001948751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b68d80 0x5582f1b6b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.713+0000 7ff42effd640 1 -- 172.21.15.78:0/4068577763 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7ff4341069a0 con 0x7ff438101a90 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.713+0000 7ff43ddec640 1 --2- 172.21.15.78:0/1001948751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b5cdf0 0x5582f1b68840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.713+0000 7ff43e5ed640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff4240b5580 0x7ff4240b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.713+0000 7ff43e5ed640 1 --2- 172.21.15.78:0/1001948751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b5c8b0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7ff430010370 tx=0x7ff430010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f3bf892178cba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.714+0000 7ff423fff640 1 -- 172.21.15.78:0/1001948751 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430022070 con 0x5582f1b5c4d0 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.714+0000 7ff43e5ed640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff4240b5580 0x7ff4240b7a40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff43001f000 tx=0x7ff430022000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:06.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.714+0000 7ff423fff640 1 -- 172.21.15.78:0/1001948751 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4300113e0 con 0x5582f1b5c4d0 2024-09-11T01:22:06.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.714+0000 7ff423fff640 1 -- 172.21.15.78:0/1001948751 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430016070 con 0x5582f1b5c4d0 2024-09-11T01:22:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.715+0000 7ff442425740 1 -- 172.21.15.78:0/1001948751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 msgr2=0x5582f1b5c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.715+0000 7ff442425740 1 --2- 172.21.15.78:0/1001948751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b5c8b0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7ff430010370 tx=0x7ff430010830 comp rx=0 tx=0).stop 2024-09-11T01:22:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.715+0000 7ff442425740 1 -- 172.21.15.78:0/1001948751 shutdown_connections 2024-09-11T01:22:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.715+0000 7ff442425740 1 --2- 172.21.15.78:0/1001948751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b68d80 0x5582f1b6b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.715+0000 7ff442425740 1 --2- 172.21.15.78:0/1001948751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b5cdf0 0x5582f1b68840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.715+0000 7ff442425740 1 --2- 172.21.15.78:0/1001948751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b5c8b0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.715+0000 7ff442425740 1 -- 172.21.15.78:0/1001948751 >> 172.21.15.78:0/1001948751 conn(0x5582f1b5a300 msgr2=0x5582f1b5b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.715+0000 7ff442425740 1 -- 172.21.15.78:0/1001948751 shutdown_connections 2024-09-11T01:22:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.715+0000 7ff442425740 1 -- 172.21.15.78:0/1001948751 wait complete. 2024-09-11T01:22:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.716+0000 7ff442425740 1 -- start start 2024-09-11T01:22:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.716+0000 7ff442425740 1 Processor -- start 2024-09-11T01:22:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.716+0000 7ff442425740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b68640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.717+0000 7ff442425740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5cdf0 0x5582f1b65a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.717+0000 7ff442425740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b68d80 0x5582f1b66070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.717+0000 7ff43ddec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b68640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.717+0000 7ff442425740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5582f1923700 con 0x5582f1b5c4d0 2024-09-11T01:22:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.717+0000 7ff43ddec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b68640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35164/0 (socket says 172.21.15.78:35164) 2024-09-11T01:22:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.717+0000 7ff43ddec640 1 -- 172.21.15.78:0/2242487272 learned_addr learned my addr 172.21.15.78:0/2242487272 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.717+0000 7ff442425740 1 -- 172.21.15.78:0/2242487272 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5582f1a5d160 con 0x5582f1b68d80 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.717+0000 7ff43d5eb640 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5cdf0 0x5582f1b65a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.717+0000 7ff43e5ed640 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b68d80 0x5582f1b66070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.717+0000 7ff442425740 1 -- 172.21.15.78:0/2242487272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5582f1b6ba00 con 0x5582f1b5cdf0 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.717+0000 7ff442425740 10 client.0 _ll_register_callbacks cb 0x5582f1a769e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff43ddec640 1 -- 172.21.15.78:0/2242487272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5cdf0 msgr2=0x5582f1b65a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff43ddec640 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5cdf0 0x5582f1b65a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff43ddec640 1 -- 172.21.15.78:0/2242487272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b68d80 msgr2=0x5582f1b66070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff43ddec640 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b68d80 0x5582f1b66070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff43ddec640 1 -- 172.21.15.78:0/2242487272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f1a8ac90 con 0x5582f1b5c4d0 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff43ddec640 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b68640 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7ff434031000 tx=0x7ff43402a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776f370a6f520b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff4217fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff4217fa640 1 -- 172.21.15.78:0/2242487272 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434049030 con 0x5582f1b5c4d0 2024-09-11T01:22:06.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff4217fa640 1 -- 172.21.15.78:0/2242487272 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7ff4340b40b0 con 0x5582f1b5c4d0 2024-09-11T01:22:06.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff442425740 1 -- 172.21.15.78:0/2242487272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4300113e0 con 0x5582f1b5c4d0 2024-09-11T01:22:06.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff4217fa640 1 -- 172.21.15.78:0/2242487272 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7ff4340cc620 con 0x5582f1b5c4d0 2024-09-11T01:22:06.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff442425740 10 client.19350 fetch_fsmap learned FSMap version 75 2024-09-11T01:22:06.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.718+0000 7ff442425740 1 -- 172.21.15.78:0/2242487272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x5582f1b772e0 con 0x5582f1b5c4d0 2024-09-11T01:22:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.719+0000 7ff4217fa640 1 -- 172.21.15.78:0/2242487272 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4300113e0 con 0x5582f1b5c4d0 2024-09-11T01:22:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.719+0000 7ff4217fa640 1 -- 172.21.15.78:0/2242487272 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 75) ==== 8370+0+0 (secure 0 0 0) 0x7ff4340d5f00 con 0x5582f1b5c4d0 2024-09-11T01:22:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.719+0000 7ff442425740 10 client.19350 fetch_fsmap finished waiting for FSMap version 75 2024-09-11T01:22:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.719+0000 7ff442425740 10 client.19350 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:22:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.719+0000 7ff442425740 20 client.19350 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.719+0000 7ff442425740 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5582f1b78ad0 0x5582f1b7af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.720+0000 7ff442425740 4 client.19350 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.720+0000 7ff442425740 1 -- 172.21.15.78:0/2242487272 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582f1ad4df0 con 0x5582f1b78ad0 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.720+0000 7ff43d5eb640 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5582f1b78ad0 0x5582f1b7af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.721+0000 7ff43d5eb640 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5582f1b78ad0 0x5582f1b7af50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e279ec19db130650 server_cookie=28f56cbeed97d986 in_seq=0 out_seq=0 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.721+0000 7ff4217fa640 10 client.19350 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.721+0000 7ff4217fa640 1 -- 172.21.15.78:0/2242487272 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582f1ad4df0 con 0x5582f1b78ad0 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.721+0000 7ff4217fa640 10 client.19350 handle_command_reply: tid=0 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.722+0000 7ff442425740 1 client.19350 shutdown 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.722+0000 7ff442425740 1 -- 172.21.15.78:0/2242487272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 msgr2=0x5582f1b68640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.722+0000 7ff442425740 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b68640 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7ff434031000 tx=0x7ff43402a920 comp rx=0 tx=0).stop 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.722+0000 7ff442425740 1 -- 172.21.15.78:0/2242487272 shutdown_connections 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.722+0000 7ff442425740 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5582f1b78ad0 0x5582f1b7af50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.722+0000 7ff442425740 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b68d80 0x5582f1b66070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.722+0000 7ff442425740 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5cdf0 0x5582f1b65a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.722+0000 7ff442425740 1 --2- 172.21.15.78:0/2242487272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b68640 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.722+0000 7ff442425740 1 -- 172.21.15.78:0/2242487272 >> 172.21.15.78:0/2242487272 conn(0x5582f1b5a300 msgr2=0x5582f1b5fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.722+0000 7ff4217fa640 0 client.19350 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:22:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.722+0000 7ff442425740 1 -- 172.21.15.78:0/2242487272 shutdown_connections 2024-09-11T01:22:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.723+0000 7ff442425740 1 -- 172.21.15.78:0/2242487272 wait complete. 2024-09-11T01:22:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.723+0000 7ff442425740 20 client.19350 trim_cache size 0 max 16384 2024-09-11T01:22:06.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.742+0000 7ff442425740 1 Processor -- start 2024-09-11T01:22:06.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.742+0000 7ff442425740 1 -- start start 2024-09-11T01:22:06.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.742+0000 7ff442425740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5c4d0 0x5582f1b55320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.742+0000 7ff442425740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b5cdf0 0x5582f1b59310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.742+0000 7ff442425740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b68d80 0x5582f1b62ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.742+0000 7ff442425740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c00e6b0 con 0x5582f1b68d80 2024-09-11T01:22:06.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.743+0000 7ff442425740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c010250 con 0x5582f1b5cdf0 2024-09-11T01:22:06.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.743+0000 7ff442425740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c00ed80 con 0x5582f1b5c4d0 2024-09-11T01:22:06.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.743+0000 7ff43ddec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5c4d0 0x5582f1b55320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.743+0000 7ff43ddec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5c4d0 0x5582f1b55320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:51558/0 (socket says 172.21.15.78:51558) 2024-09-11T01:22:06.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.743+0000 7ff43ddec640 1 -- 172.21.15.78:0/357793272 learned_addr learned my addr 172.21.15.78:0/357793272 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:06.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.743+0000 7ff43d5eb640 1 --2- 172.21.15.78:0/357793272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b5cdf0 0x5582f1b59310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.743+0000 7ff43e5ed640 1 --2- 172.21.15.78:0/357793272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b68d80 0x5582f1b62ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.743+0000 7ff43d5eb640 1 -- 172.21.15.78:0/357793272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5c4d0 msgr2=0x5582f1b55320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:06.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.743+0000 7ff43d5eb640 1 --2- 172.21.15.78:0/357793272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5c4d0 0x5582f1b55320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.743+0000 7ff43d5eb640 1 -- 172.21.15.78:0/357793272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b68d80 msgr2=0x5582f1b62ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:06.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.743+0000 7ff43d5eb640 1 --2- 172.21.15.78:0/357793272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b68d80 0x5582f1b62ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.743+0000 7ff43d5eb640 1 -- 172.21.15.78:0/357793272 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f1b663e0 con 0x5582f1b5cdf0 2024-09-11T01:22:06.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.744+0000 7ff43d5eb640 1 --2- 172.21.15.78:0/357793272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b5cdf0 0x5582f1b59310 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5582f1b667c0 tx=0x7ff428015b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60a3aaacafa746bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:06.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.744+0000 7ff4237fe640 1 -- 172.21.15.78:0/357793272 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428015c60 con 0x5582f1b5cdf0 2024-09-11T01:22:06.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.744+0000 7ff4237fe640 1 -- 172.21.15.78:0/357793272 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff428015e00 con 0x5582f1b5cdf0 2024-09-11T01:22:06.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.744+0000 7ff4237fe640 1 -- 172.21.15.78:0/357793272 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428013ac0 con 0x5582f1b5cdf0 2024-09-11T01:22:06.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.745+0000 7ff442425740 1 -- 172.21.15.78:0/357793272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b5cdf0 msgr2=0x5582f1b59310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:06.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.745+0000 7ff442425740 1 --2- 172.21.15.78:0/357793272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b5cdf0 0x5582f1b59310 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5582f1b667c0 tx=0x7ff428015b30 comp rx=0 tx=0).stop 2024-09-11T01:22:06.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.745+0000 7ff442425740 1 -- 172.21.15.78:0/357793272 shutdown_connections 2024-09-11T01:22:06.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.745+0000 7ff442425740 1 --2- 172.21.15.78:0/357793272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b68d80 0x5582f1b62ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.745+0000 7ff442425740 1 --2- 172.21.15.78:0/357793272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b5cdf0 0x5582f1b59310 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.745+0000 7ff442425740 1 --2- 172.21.15.78:0/357793272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5c4d0 0x5582f1b55320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.745+0000 7ff442425740 1 -- 172.21.15.78:0/357793272 >> 172.21.15.78:0/357793272 conn(0x5582f1b5a300 msgr2=0x5582f1b58c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.746+0000 7ff442425740 1 -- 172.21.15.78:0/357793272 shutdown_connections 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.746+0000 7ff442425740 1 -- 172.21.15.78:0/357793272 wait complete. 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.746+0000 7ff442425740 1 -- start start 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.746+0000 7ff442425740 1 Processor -- start 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff442425740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b7e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff43ddec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b7e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff442425740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5cdf0 0x5582f1b7ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff442425740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b68d80 0x5582f1b7f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff43ddec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b7e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35172/0 (socket says 172.21.15.78:35172) 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff43ddec640 1 -- 172.21.15.78:0/4274138110 learned_addr learned my addr 172.21.15.78:0/4274138110 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff442425740 1 -- 172.21.15.78:0/4274138110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c00fb80 con 0x5582f1b5c4d0 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff442425740 1 -- 172.21.15.78:0/4274138110 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c00f4b0 con 0x5582f1b68d80 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff442425740 1 -- 172.21.15.78:0/4274138110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c00c6b0 con 0x5582f1b5cdf0 2024-09-11T01:22:06.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff43e5ed640 1 --2- 172.21.15.78:0/4274138110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b68d80 0x5582f1b7f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff442425740 10 client.0 _ll_register_callbacks cb 0x7ff4340c8d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:06.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff43ddec640 1 -- 172.21.15.78:0/4274138110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5cdf0 msgr2=0x5582f1b7ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:06.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff43ddec640 1 --2- 172.21.15.78:0/4274138110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5cdf0 0x5582f1b7ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff43ddec640 1 -- 172.21.15.78:0/4274138110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b68d80 msgr2=0x5582f1b7f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:06.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff43ddec640 1 --2- 172.21.15.78:0/4274138110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b68d80 0x5582f1b7f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.747+0000 7ff43ddec640 1 -- 172.21.15.78:0/4274138110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f1be4b00 con 0x5582f1b5c4d0 2024-09-11T01:22:06.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.748+0000 7ff43ddec640 1 --2- 172.21.15.78:0/4274138110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b7e810 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7ff4340ccbb0 tx=0x7ff434004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5daf6d9d59ceaeb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:06.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.748+0000 7ff4217fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:22:06.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.748+0000 7ff4217fa640 1 -- 172.21.15.78:0/4274138110 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434049070 con 0x5582f1b5c4d0 2024-09-11T01:22:06.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.748+0000 7ff4217fa640 1 -- 172.21.15.78:0/4274138110 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8449+0+0 (secure 0 0 0) 0x7ff4340d4c60 con 0x5582f1b5c4d0 2024-09-11T01:22:06.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.748+0000 7ff442425740 1 -- 172.21.15.78:0/4274138110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff428015e00 con 0x5582f1b5c4d0 2024-09-11T01:22:06.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.748+0000 7ff4217fa640 1 -- 172.21.15.78:0/4274138110 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7ff434104560 con 0x5582f1b5c4d0 2024-09-11T01:22:06.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.749+0000 7ff442425740 10 client.19356 fetch_fsmap learned FSMap version 75 2024-09-11T01:22:06.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.749+0000 7ff442425740 1 -- 172.21.15.78:0/4274138110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x5582f1be5df0 con 0x5582f1b5c4d0 2024-09-11T01:22:06.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.749+0000 7ff4217fa640 1 -- 172.21.15.78:0/4274138110 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff428015e00 con 0x5582f1b5c4d0 2024-09-11T01:22:06.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.750+0000 7ff4217fa640 1 -- 172.21.15.78:0/4274138110 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 75) ==== 8370+0+0 (secure 0 0 0) 0x7ff4340cc110 con 0x5582f1b5c4d0 2024-09-11T01:22:06.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.750+0000 7ff442425740 10 client.19356 fetch_fsmap finished waiting for FSMap version 75 2024-09-11T01:22:06.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.750+0000 7ff442425740 10 client.19356 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:22:06.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.750+0000 7ff442425740 20 client.19356 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:06.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.750+0000 7ff442425740 1 --2- 172.21.15.78:0/4274138110 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5582f1be6430 0x5582f1be8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:06.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.750+0000 7ff442425740 4 client.19356 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:22:06.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.750+0000 7ff442425740 1 -- 172.21.15.78:0/4274138110 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582f1ad08a0 con 0x5582f1be6430 2024-09-11T01:22:06.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.750+0000 7ff43d5eb640 1 --2- 172.21.15.78:0/4274138110 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5582f1be6430 0x5582f1be8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:06.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.751+0000 7ff43d5eb640 1 --2- 172.21.15.78:0/4274138110 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5582f1be6430 0x5582f1be8890 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d1b41bd7c85aad8 server_cookie=794d067d7a97dd72 in_seq=0 out_seq=0 2024-09-11T01:22:06.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.751+0000 7ff4217fa640 10 client.19356 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:22:06.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.752+0000 7ff4217fa640 1 -- 172.21.15.78:0/4274138110 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582f1ad08a0 con 0x5582f1be6430 2024-09-11T01:22:06.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.752+0000 7ff4217fa640 10 client.19356 handle_command_reply: tid=0 2024-09-11T01:22:06.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.752+0000 7ff442425740 1 client.19356 shutdown 2024-09-11T01:22:06.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.754+0000 7ff442425740 1 -- 172.21.15.78:0/4274138110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 msgr2=0x5582f1b7e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:06.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.754+0000 7ff442425740 1 --2- 172.21.15.78:0/4274138110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b7e810 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7ff4340ccbb0 tx=0x7ff434004610 comp rx=0 tx=0).stop 2024-09-11T01:22:06.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.754+0000 7ff442425740 1 -- 172.21.15.78:0/4274138110 shutdown_connections 2024-09-11T01:22:06.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.754+0000 7ff442425740 1 --2- 172.21.15.78:0/4274138110 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x5582f1be6430 0x5582f1be8890 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.754+0000 7ff442425740 1 --2- 172.21.15.78:0/4274138110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5582f1b68d80 0x5582f1b7f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.754+0000 7ff4217fa640 0 client.19356 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:22:06.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.754+0000 7ff442425740 1 --2- 172.21.15.78:0/4274138110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582f1b5cdf0 0x5582f1b7ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.754+0000 7ff442425740 1 --2- 172.21.15.78:0/4274138110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5582f1b5c4d0 0x5582f1b7e810 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.754+0000 7ff442425740 1 -- 172.21.15.78:0/4274138110 >> 172.21.15.78:0/4274138110 conn(0x5582f1b5a300 msgr2=0x5582f1b678e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:06.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.755+0000 7ff442425740 1 -- 172.21.15.78:0/4274138110 shutdown_connections 2024-09-11T01:22:06.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.755+0000 7ff442425740 1 -- 172.21.15.78:0/4274138110 wait complete. 2024-09-11T01:22:06.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.755+0000 7ff442425740 20 client.19356 trim_cache size 0 max 16384 2024-09-11T01:22:06.758 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:22:06.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.756+0000 7ff440050640 1 -- 172.21.15.78:0/4068577763 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff4240b5580 msgr2=0x7ff4240b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:06.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.756+0000 7ff440050640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff4240b5580 0x7ff4240b7a40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff43001f000 tx=0x7ff430022000 comp rx=0 tx=0).stop 2024-09-11T01:22:06.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.756+0000 7ff440050640 1 -- 172.21.15.78:0/4068577763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff438101a90 msgr2=0x7ff43819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:06.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.756+0000 7ff440050640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff438101a90 0x7ff43819f6c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff434009850 tx=0x7ff4340305d0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.756+0000 7ff440050640 1 -- 172.21.15.78:0/4068577763 shutdown_connections 2024-09-11T01:22:06.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.756+0000 7ff440050640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff4240b5580 0x7ff4240b7a40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.756+0000 7ff440050640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff438106ed0 0x7ff4381a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.756+0000 7ff440050640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4381023b0 0x7ff43819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.756+0000 7ff440050640 1 --2- 172.21.15.78:0/4068577763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff438101a90 0x7ff43819f6c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:06.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.756+0000 7ff440050640 1 -- 172.21.15.78:0/4068577763 >> 172.21.15.78:0/4068577763 conn(0x7ff4380fd700 msgr2=0x7ff438105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:06.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.756+0000 7ff440050640 1 -- 172.21.15.78:0/4068577763 shutdown_connections 2024-09-11T01:22:06.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:06.757+0000 7ff440050640 1 -- 172.21.15.78:0/4068577763 wait complete. 2024-09-11T01:22:06.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 371526 31.91 MB/sec execute 492 sec latency 386.244 ms 2024-09-11T01:22:07.518 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:22:07.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 372117 31.93 MB/sec execute 493 sec latency 145.176 ms 2024-09-11T01:22:07.899 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:22:08.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:08 smithi110 ceph-mon[34712]: pgmap v1052: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 15 MiB/s wr, 667 op/s 2024-09-11T01:22:08.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:08 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:22:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:08 smithi112 ceph-mon[35384]: pgmap v1052: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 15 MiB/s wr, 667 op/s 2024-09-11T01:22:08.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:08 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:22:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:08 smithi078 ceph-mon[31315]: pgmap v1052: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 15 MiB/s wr, 667 op/s 2024-09-11T01:22:08.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:08 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:22:08.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 372427 31.89 MB/sec execute 494 sec latency 522.762 ms 2024-09-11T01:22:09.440 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:09 smithi110 ceph-mon[34712]: osdmap e110: 12 total, 12 up, 12 in 2024-09-11T01:22:09.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:09 smithi112 ceph-mon[35384]: osdmap e110: 12 total, 12 up, 12 in 2024-09-11T01:22:09.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:09 smithi078 ceph-mon[31315]: osdmap e110: 12 total, 12 up, 12 in 2024-09-11T01:22:09.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 372835 31.87 MB/sec execute 495 sec latency 590.500 ms 2024-09-11T01:22:09.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.210+0000 7f4125c9e640 1 -- 172.21.15.78:0/2663106697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41201023d0 msgr2=0x7f41201068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:09.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.210+0000 7f4125c9e640 1 --2- 172.21.15.78:0/2663106697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41201023d0 0x7f41201068a0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f4108009900 tx=0x7f410802a410 comp rx=0 tx=0).stop 2024-09-11T01:22:09.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.214+0000 7f4125c9e640 1 -- 172.21.15.78:0/2663106697 shutdown_connections 2024-09-11T01:22:09.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.214+0000 7f4125c9e640 1 --2- 172.21.15.78:0/2663106697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4120106ef0 0x7f4120107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.214+0000 7f4125c9e640 1 --2- 172.21.15.78:0/2663106697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41201023d0 0x7f41201068a0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.214+0000 7f4125c9e640 1 --2- 172.21.15.78:0/2663106697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4120101ab0 0x7f4120101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.214+0000 7f4125c9e640 1 -- 172.21.15.78:0/2663106697 >> 172.21.15.78:0/2663106697 conn(0x7f41200fd700 msgr2=0x7f41200ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.214+0000 7f4125c9e640 1 -- 172.21.15.78:0/2663106697 shutdown_connections 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.214+0000 7f4125c9e640 1 -- 172.21.15.78:0/2663106697 wait complete. 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.215+0000 7f4125c9e640 1 Processor -- start 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.215+0000 7f4125c9e640 1 -- start start 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.215+0000 7f4125c9e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4120101ab0 0x7f412019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.215+0000 7f4125c9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41201023d0 0x7f412019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.215+0000 7f4125c9e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4120106ef0 0x7f41201a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.215+0000 7f4125c9e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4120119360 con 0x7f4120106ef0 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.215+0000 7f4125c9e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f41201191e0 con 0x7f4120101ab0 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.215+0000 7f4125c9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41201194e0 con 0x7f41201023d0 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.216+0000 7f4117fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41201023d0 0x7f412019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.216+0000 7f4117fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41201023d0 0x7f412019fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:51584/0 (socket says 172.21.15.78:51584) 2024-09-11T01:22:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.216+0000 7f4117fff640 1 -- 172.21.15.78:0/3252959237 learned_addr learned my addr 172.21.15.78:0/3252959237 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.217+0000 7f411ffff640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4120106ef0 0x7f41201a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.217+0000 7f411f7fe640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4120101ab0 0x7f412019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.217+0000 7f4117fff640 1 -- 172.21.15.78:0/3252959237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4120101ab0 msgr2=0x7f412019f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.217+0000 7f4117fff640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4120101ab0 0x7f412019f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.217+0000 7f4117fff640 1 -- 172.21.15.78:0/3252959237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4120106ef0 msgr2=0x7f41201a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.217+0000 7f4117fff640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4120106ef0 0x7f41201a51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.217+0000 7f4117fff640 1 -- 172.21.15.78:0/3252959237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41201a59c0 con 0x7f41201023d0 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.222+0000 7f4117fff640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41201023d0 0x7f412019fe50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4108006010 tx=0x7f4108004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.223+0000 7f411d7fa640 1 -- 172.21.15.78:0/3252959237 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410803f030 con 0x7f41201023d0 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.223+0000 7f4125c9e640 1 -- 172.21.15.78:0/3252959237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41201a5c50 con 0x7f41201023d0 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.223+0000 7f4125c9e640 1 -- 172.21.15.78:0/3252959237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4120107e70 con 0x7f41201023d0 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.223+0000 7f411d7fa640 1 -- 172.21.15.78:0/3252959237 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4108003aa0 con 0x7f41201023d0 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.223+0000 7f411d7fa640 1 -- 172.21.15.78:0/3252959237 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108035070 con 0x7f41201023d0 2024-09-11T01:22:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.228+0000 7f4128073740 1 Processor -- start 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.228+0000 7f4128073740 1 -- start start 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.229+0000 7f4128073740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.229+0000 7f4128073740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0aea10 0x5595ce0ba460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.229+0000 7f4128073740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0ba9a0 0x5595ce0bcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.229+0000 7f4128073740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5595ce024ea0 con 0x5595ce0ba9a0 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.229+0000 7f4128073740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5595ce0142d0 con 0x5595ce0aea10 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.229+0000 7f4128073740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5595ce0180e0 con 0x5595ce0ae0f0 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.229+0000 7f411d7fa640 1 -- 172.21.15.78:0/3252959237 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f410803bb40 con 0x7f41201023d0 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.229+0000 7f411ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.229+0000 7f411ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:51600/0 (socket says 172.21.15.78:51600) 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.229+0000 7f411ffff640 1 -- 172.21.15.78:0/2626374631 learned_addr learned my addr 172.21.15.78:0/2626374631 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.229+0000 7f4117fff640 1 --2- 172.21.15.78:0/2626374631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0ba9a0 0x5595ce0bcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.230+0000 7f411f7fe640 1 --2- 172.21.15.78:0/2626374631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0aea10 0x5595ce0ba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.230+0000 7f4117fff640 1 -- 172.21.15.78:0/2626374631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 msgr2=0x5595ce0ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.230+0000 7f4117fff640 1 --2- 172.21.15.78:0/2626374631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.230+0000 7f4117fff640 1 -- 172.21.15.78:0/2626374631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0aea10 msgr2=0x5595ce0ba460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.230+0000 7f4117fff640 1 --2- 172.21.15.78:0/2626374631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0aea10 0x5595ce0ba460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.230+0000 7f4117fff640 1 -- 172.21.15.78:0/2626374631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595cdd72650 con 0x5595ce0ba9a0 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.230+0000 7f411ffff640 1 --2- 172.21.15.78:0/2626374631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.230+0000 7f4117fff640 1 --2- 172.21.15.78:0/2626374631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0ba9a0 0x5595ce0bcd90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4108103cd0 tx=0x7f410810c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadf34a5dc798a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.230+0000 7f411d7fa640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f410c0b55a0 0x7f410c0b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.231+0000 7f411ffff640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f410c0b55a0 0x7f410c0b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.231+0000 7f411d7fa640 1 -- 172.21.15.78:0/3252959237 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8493+0+0 (secure 0 0 0) 0x7f4108106560 con 0x7f41201023d0 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.231+0000 7f4115ffb640 1 -- 172.21.15.78:0/2626374631 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080c8fb0 con 0x5595ce0ba9a0 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.231+0000 7f4115ffb640 1 -- 172.21.15.78:0/2626374631 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4108113030 con 0x5595ce0ba9a0 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.231+0000 7f4115ffb640 1 -- 172.21.15.78:0/2626374631 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080c8fb0 con 0x5595ce0ba9a0 2024-09-11T01:22:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.232+0000 7f4128073740 1 -- 172.21.15.78:0/2626374631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0ba9a0 msgr2=0x5595ce0bcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.232+0000 7f411ffff640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f410c0b55a0 0x7f410c0b7a60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4118005f40 tx=0x7f41180096a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.232+0000 7f4128073740 1 --2- 172.21.15.78:0/2626374631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0ba9a0 0x5595ce0bcd90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4108103cd0 tx=0x7f410810c800 comp rx=0 tx=0).stop 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.232+0000 7f4128073740 1 -- 172.21.15.78:0/2626374631 shutdown_connections 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.232+0000 7f4128073740 1 --2- 172.21.15.78:0/2626374631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0ba9a0 0x5595ce0bcd90 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.232+0000 7f4128073740 1 --2- 172.21.15.78:0/2626374631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0aea10 0x5595ce0ba460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.232+0000 7f4128073740 1 --2- 172.21.15.78:0/2626374631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.232+0000 7f4128073740 1 -- 172.21.15.78:0/2626374631 >> 172.21.15.78:0/2626374631 conn(0x5595ce0abf20 msgr2=0x5595ce0ad0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.232+0000 7f4128073740 1 -- 172.21.15.78:0/2626374631 shutdown_connections 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.232+0000 7f4128073740 1 -- 172.21.15.78:0/2626374631 wait complete. 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.233+0000 7f4128073740 1 -- start start 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.233+0000 7f4128073740 1 Processor -- start 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.233+0000 7f4128073740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.233+0000 7f4128073740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.233+0000 7f4128073740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ba9a0 0x5595ce0b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.233+0000 7f4128073740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5595cde75700 con 0x5595ce0aea10 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.233+0000 7f4128073740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5595cdfaf160 con 0x5595ce0ae0f0 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.233+0000 7f4128073740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5595ce0bd3e0 con 0x5595ce0ba9a0 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.233+0000 7f4128073740 10 client.0 _ll_register_callbacks cb 0x5595cdfc89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.234+0000 7f4117fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.234+0000 7f4117fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0b7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35186/0 (socket says 172.21.15.78:35186) 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.234+0000 7f4117fff640 1 -- 172.21.15.78:0/655277660 learned_addr learned my addr 172.21.15.78:0/655277660 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.234+0000 7f411ffff640 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ba9a0 0x5595ce0b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.234+0000 7f411f7fe640 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.234+0000 7f411ffff640 1 -- 172.21.15.78:0/655277660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ae0f0 msgr2=0x5595ce0ba200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.234+0000 7f411ffff640 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ba200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.234+0000 7f411ffff640 1 -- 172.21.15.78:0/655277660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 msgr2=0x5595ce0b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.234+0000 7f411ffff640 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0b7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.234+0000 7f411ffff640 1 -- 172.21.15.78:0/655277660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595cdfdcc90 con 0x5595ce0ba9a0 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.234+0000 7f4117fff640 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.235+0000 7f411f7fe640 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.235+0000 7f411ffff640 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ba9a0 0x5595ce0b7bd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4118011fb0 tx=0x7f4118004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da016c7bbefb053 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.235+0000 7f40feffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.235+0000 7f40feffd640 1 -- 172.21.15.78:0/655277660 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118004b20 con 0x5595ce0ba9a0 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.235+0000 7f4128073740 1 -- 172.21.15.78:0/655277660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4108113030 con 0x5595ce0ba9a0 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.235+0000 7f40feffd640 1 -- 172.21.15.78:0/655277660 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8493+0+0 (secure 0 0 0) 0x7f4118017030 con 0x5595ce0ba9a0 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.235+0000 7f40feffd640 1 -- 172.21.15.78:0/655277660 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f4118004cc0 con 0x5595ce0ba9a0 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.235+0000 7f4128073740 10 client.28351 fetch_fsmap learned FSMap version 76 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.235+0000 7f4128073740 1 -- 172.21.15.78:0/655277660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5595ce0c8ca0 con 0x5595ce0ba9a0 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.236+0000 7f40feffd640 1 -- 172.21.15.78:0/655277660 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4108113030 con 0x5595ce0ba9a0 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.236+0000 7f40feffd640 1 -- 172.21.15.78:0/655277660 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f411802e020 con 0x5595ce0ba9a0 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.236+0000 7f4128073740 10 client.28351 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:22:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.236+0000 7f4128073740 10 client.28351 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:22:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.236+0000 7f4128073740 20 client.28351 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.236+0000 7f4128073740 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5595ce0ca1c0 0x5595ce0cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.236+0000 7f4128073740 4 client.28351 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:22:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.236+0000 7f4128073740 1 -- 172.21.15.78:0/655277660 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595ce026df0 con 0x5595ce0ca1c0 2024-09-11T01:22:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.236+0000 7f411f7fe640 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5595ce0ca1c0 0x5595ce0cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.237+0000 7f40feffd640 10 client.28351 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:22:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.237+0000 7f411f7fe640 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5595ce0ca1c0 0x5595ce0cc5d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1a9f8553d356e33e server_cookie=74d69ff099e40d1c in_seq=0 out_seq=0 2024-09-11T01:22:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.239+0000 7f40feffd640 1 -- 172.21.15.78:0/655277660 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5595ce026df0 con 0x5595ce0ca1c0 2024-09-11T01:22:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.239+0000 7f40feffd640 10 client.28351 handle_command_reply: tid=0 2024-09-11T01:22:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.239+0000 7f4128073740 1 client.28351 shutdown 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.239+0000 7f4128073740 1 -- 172.21.15.78:0/655277660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ba9a0 msgr2=0x5595ce0b7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.239+0000 7f4128073740 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ba9a0 0x5595ce0b7bd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4118011fb0 tx=0x7f4118004ab0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.239+0000 7f4128073740 1 -- 172.21.15.78:0/655277660 shutdown_connections 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.239+0000 7f4128073740 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5595ce0ca1c0 0x5595ce0cc5d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.239+0000 7f4128073740 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ba9a0 0x5595ce0b7bd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.239+0000 7f4128073740 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.239+0000 7f4128073740 1 --2- 172.21.15.78:0/655277660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.239+0000 7f40feffd640 0 client.28351 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.239+0000 7f4128073740 1 -- 172.21.15.78:0/655277660 >> 172.21.15.78:0/655277660 conn(0x5595ce0abf20 msgr2=0x5595ce0b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.240+0000 7f4128073740 1 -- 172.21.15.78:0/655277660 shutdown_connections 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.240+0000 7f4128073740 1 -- 172.21.15.78:0/655277660 wait complete. 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.240+0000 7f4128073740 20 client.28351 trim_cache size 0 max 16384 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.263+0000 7f4128073740 1 Processor -- start 2024-09-11T01:22:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.263+0000 7f4128073740 1 -- start start 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.263+0000 7f4128073740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.263+0000 7f4128073740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.263+0000 7f4128073740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ba9a0 0x5595ce0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f4128073740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f410000e740 con 0x5595ce0aea10 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f4128073740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4100010340 con 0x5595ce0ba9a0 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f4128073740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f410000ee10 con 0x5595ce0ae0f0 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f411f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f4117fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f411f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:51624/0 (socket says 172.21.15.78:51624) 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f411f7fe640 1 -- 172.21.15.78:0/3936408244 learned_addr learned my addr 172.21.15.78:0/3936408244 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f411ffff640 1 --2- 172.21.15.78:0/3936408244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ba9a0 0x5595ce0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f4117fff640 1 -- 172.21.15.78:0/3936408244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 msgr2=0x5595ce0ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f4117fff640 1 --2- 172.21.15.78:0/3936408244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f4117fff640 1 -- 172.21.15.78:0/3936408244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ba9a0 msgr2=0x5595ce0b6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f4117fff640 1 --2- 172.21.15.78:0/3936408244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ba9a0 0x5595ce0b6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.264+0000 7f4117fff640 1 -- 172.21.15.78:0/3936408244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595ce0ccc50 con 0x5595ce0aea10 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.265+0000 7f4117fff640 1 --2- 172.21.15.78:0/3936408244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0b5ed0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f41081042d0 tx=0x7f4108109c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc782eacfaa3da1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.266+0000 7f41157fa640 1 -- 172.21.15.78:0/3936408244 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080c8fb0 con 0x5595ce0aea10 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.266+0000 7f41157fa640 1 -- 172.21.15.78:0/3936408244 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4108113050 con 0x5595ce0aea10 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.266+0000 7f41157fa640 1 -- 172.21.15.78:0/3936408244 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410803be60 con 0x5595ce0aea10 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.267+0000 7f4128073740 1 -- 172.21.15.78:0/3936408244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 msgr2=0x5595ce0b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.267+0000 7f4128073740 1 --2- 172.21.15.78:0/3936408244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0b5ed0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f41081042d0 tx=0x7f4108109c20 comp rx=0 tx=0).stop 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.267+0000 7f4128073740 1 -- 172.21.15.78:0/3936408244 shutdown_connections 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.267+0000 7f4128073740 1 --2- 172.21.15.78:0/3936408244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ba9a0 0x5595ce0b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.267+0000 7f4128073740 1 --2- 172.21.15.78:0/3936408244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0b5ed0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.267+0000 7f4128073740 1 --2- 172.21.15.78:0/3936408244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.267+0000 7f4128073740 1 -- 172.21.15.78:0/3936408244 >> 172.21.15.78:0/3936408244 conn(0x5595ce0abf20 msgr2=0x5595ce0acf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.268+0000 7f4128073740 1 -- 172.21.15.78:0/3936408244 shutdown_connections 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.268+0000 7f4128073740 1 -- 172.21.15.78:0/3936408244 wait complete. 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.268+0000 7f4128073740 1 -- start start 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.269+0000 7f4128073740 1 Processor -- start 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.269+0000 7f4128073740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0bea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.269+0000 7f4128073740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0bef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.269+0000 7f4128073740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ba9a0 0x5595ce1360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f411f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f411f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0bea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:51634/0 (socket says 172.21.15.78:51634) 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f4128073740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f410000fc70 con 0x5595ce0aea10 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f4128073740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f410000f5a0 con 0x5595ce0ba9a0 2024-09-11T01:22:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f4128073740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f410000c740 con 0x5595ce0ae0f0 2024-09-11T01:22:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f4128073740 10 client.0 _ll_register_callbacks cb 0x7f4118009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f411f7fe640 1 -- 172.21.15.78:0/401893906 learned_addr learned my addr 172.21.15.78:0/401893906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f411ffff640 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ba9a0 0x5595ce1360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f4117fff640 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0bef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f411f7fe640 1 -- 172.21.15.78:0/401893906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ba9a0 msgr2=0x5595ce1360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f411f7fe640 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ba9a0 0x5595ce1360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f411f7fe640 1 -- 172.21.15.78:0/401893906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 msgr2=0x5595ce0bef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f411f7fe640 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0bef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.270+0000 7f411f7fe640 1 -- 172.21.15.78:0/401893906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595ce0ac9a0 con 0x5595ce0ae0f0 2024-09-11T01:22:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.271+0000 7f411f7fe640 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0bea10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4110022710 tx=0x7f4110022bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31da1a5d0489ec51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.271+0000 7f40feffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:22:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.272+0000 7f40feffd640 1 -- 172.21.15.78:0/401893906 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411002e990 con 0x5595ce0ae0f0 2024-09-11T01:22:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.272+0000 7f40feffd640 1 -- 172.21.15.78:0/401893906 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8493+0+0 (secure 0 0 0) 0x7f411002eb30 con 0x5595ce0ae0f0 2024-09-11T01:22:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.272+0000 7f4128073740 1 -- 172.21.15.78:0/401893906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4108113050 con 0x5595ce0ae0f0 2024-09-11T01:22:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.272+0000 7f40feffd640 1 -- 172.21.15.78:0/401893906 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f411002ed30 con 0x5595ce0ae0f0 2024-09-11T01:22:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.272+0000 7f4128073740 10 client.28357 fetch_fsmap learned FSMap version 76 2024-09-11T01:22:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.272+0000 7f4128073740 1 -- 172.21.15.78:0/401893906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5595ce137730 con 0x5595ce0ae0f0 2024-09-11T01:22:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.273+0000 7f40feffd640 1 -- 172.21.15.78:0/401893906 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4108113050 con 0x5595ce0ae0f0 2024-09-11T01:22:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.273+0000 7f40feffd640 1 -- 172.21.15.78:0/401893906 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f411001f040 con 0x5595ce0ae0f0 2024-09-11T01:22:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.273+0000 7f4128073740 10 client.28357 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:22:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.273+0000 7f4128073740 10 client.28357 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:22:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.273+0000 7f4128073740 20 client.28357 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:09.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.273+0000 7f4128073740 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5595ce137f60 0x5595ce13a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:09.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.273+0000 7f4128073740 4 client.28357 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:22:09.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.273+0000 7f4128073740 1 -- 172.21.15.78:0/401893906 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595ce0228a0 con 0x5595ce137f60 2024-09-11T01:22:09.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.273+0000 7f4117fff640 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5595ce137f60 0x5595ce13a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:09.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.274+0000 7f4117fff640 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5595ce137f60 0x5595ce13a380 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a0b6138ea0a644bb server_cookie=4c62798e78ac63a4 in_seq=0 out_seq=0 2024-09-11T01:22:09.844 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:22:09.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.274+0000 7f40feffd640 10 client.28357 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:22:09.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.276+0000 7f40feffd640 1 -- 172.21.15.78:0/401893906 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595ce0228a0 con 0x5595ce137f60 2024-09-11T01:22:09.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.276+0000 7f40feffd640 10 client.28357 handle_command_reply: tid=0 2024-09-11T01:22:09.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.277+0000 7f4128073740 1 client.28357 shutdown 2024-09-11T01:22:09.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.277+0000 7f4128073740 1 -- 172.21.15.78:0/401893906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 msgr2=0x5595ce0bea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:09.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.277+0000 7f4128073740 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0bea10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4110022710 tx=0x7f4110022bd0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.277+0000 7f4128073740 1 -- 172.21.15.78:0/401893906 shutdown_connections 2024-09-11T01:22:09.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.277+0000 7f4128073740 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5595ce137f60 0x5595ce13a380 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.277+0000 7f4128073740 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595ce0ba9a0 0x5595ce1360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.277+0000 7f40feffd640 0 client.28357 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.277+0000 7f4128073740 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595ce0aea10 0x5595ce0bef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.277+0000 7f4128073740 1 --2- 172.21.15.78:0/401893906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ce0ae0f0 0x5595ce0bea10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.277+0000 7f4128073740 1 -- 172.21.15.78:0/401893906 >> 172.21.15.78:0/401893906 conn(0x5595ce0abf20 msgr2=0x5595ce0acf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.277+0000 7f4128073740 1 -- 172.21.15.78:0/401893906 shutdown_connections 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.277+0000 7f4128073740 1 -- 172.21.15.78:0/401893906 wait complete. 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.278+0000 7f4128073740 20 client.28357 trim_cache size 0 max 16384 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.278+0000 7f4125c9e640 1 -- 172.21.15.78:0/3252959237 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f410c0b55a0 msgr2=0x7f410c0b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.278+0000 7f4125c9e640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f410c0b55a0 0x7f410c0b7a60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4118005f40 tx=0x7f41180096a0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.278+0000 7f4125c9e640 1 -- 172.21.15.78:0/3252959237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41201023d0 msgr2=0x7f412019fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.278+0000 7f4125c9e640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41201023d0 0x7f412019fe50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4108006010 tx=0x7f4108004060 comp rx=0 tx=0).stop 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.278+0000 7f4125c9e640 1 -- 172.21.15.78:0/3252959237 shutdown_connections 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.278+0000 7f4125c9e640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f410c0b55a0 0x7f410c0b7a60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.279+0000 7f4125c9e640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4120106ef0 0x7f41201a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.279+0000 7f4125c9e640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41201023d0 0x7f412019fe50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.279+0000 7f4125c9e640 1 --2- 172.21.15.78:0/3252959237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4120101ab0 0x7f412019f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.279+0000 7f4125c9e640 1 -- 172.21.15.78:0/3252959237 >> 172.21.15.78:0/3252959237 conn(0x7f41200fd700 msgr2=0x7f4120104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.279+0000 7f4125c9e640 1 -- 172.21.15.78:0/3252959237 shutdown_connections 2024-09-11T01:22:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:09.279+0000 7f4125c9e640 1 -- 172.21.15.78:0/3252959237 wait complete. 2024-09-11T01:22:10.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:10 smithi112 ceph-mon[35384]: pgmap v1054: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.0 MiB/s wr, 569 op/s 2024-09-11T01:22:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:10 smithi078 ceph-mon[31315]: pgmap v1054: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.0 MiB/s wr, 569 op/s 2024-09-11T01:22:10.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:10 smithi110 ceph-mon[34712]: pgmap v1054: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.0 MiB/s wr, 569 op/s 2024-09-11T01:22:10.805 INFO:tasks.workunit.client.0.smithi078.stdout: 10 373347 31.86 MB/sec execute 496 sec latency 354.407 ms 2024-09-11T01:22:11.806 INFO:tasks.workunit.client.0.smithi078.stdout: 10 373887 31.86 MB/sec execute 497 sec latency 124.463 ms 2024-09-11T01:22:12.239 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:22:12.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:12 smithi112 ceph-mon[35384]: pgmap v1055: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.5 MiB/s wr, 592 op/s 2024-09-11T01:22:12.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:12 smithi078 ceph-mon[31315]: pgmap v1055: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.5 MiB/s wr, 592 op/s 2024-09-11T01:22:12.561 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:22:12.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:12 smithi110 ceph-mon[34712]: pgmap v1055: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.5 MiB/s wr, 592 op/s 2024-09-11T01:22:12.806 INFO:tasks.workunit.client.0.smithi078.stdout: 10 374552 31.87 MB/sec execute 498 sec latency 73.955 ms 2024-09-11T01:22:13.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.111+0000 7f37dc20a640 1 -- 172.21.15.78:0/905700278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37d4101c90 msgr2=0x7f37d4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:13.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.112+0000 7f37dc20a640 1 --2- 172.21.15.78:0/905700278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37d4101c90 0x7f37d4102070 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009900 tx=0x7f37d002a410 comp rx=0 tx=0).stop 2024-09-11T01:22:13.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.112+0000 7f37dc20a640 1 -- 172.21.15.78:0/905700278 shutdown_connections 2024-09-11T01:22:13.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.112+0000 7f37dc20a640 1 --2- 172.21.15.78:0/905700278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37d4108020 0x7f37d410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.112+0000 7f37dc20a640 1 --2- 172.21.15.78:0/905700278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37d41076a0 0x7f37d4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.112+0000 7f37dc20a640 1 --2- 172.21.15.78:0/905700278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37d4101c90 0x7f37d4102070 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.112+0000 7f37dc20a640 1 -- 172.21.15.78:0/905700278 >> 172.21.15.78:0/905700278 conn(0x7f37d40fd700 msgr2=0x7f37d40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:13.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.112+0000 7f37dc20a640 1 -- 172.21.15.78:0/905700278 shutdown_connections 2024-09-11T01:22:13.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.112+0000 7f37dc20a640 1 -- 172.21.15.78:0/905700278 wait complete. 2024-09-11T01:22:13.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37dc20a640 1 Processor -- start 2024-09-11T01:22:13.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37dc20a640 1 -- start start 2024-09-11T01:22:13.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37dc20a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37d4101c90 0x7f37d4108810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37dc20a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37d41076a0 0x7f37d4108d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37dc20a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37d4108020 0x7f37d4109290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37dc20a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37d4111ce0 con 0x7f37d4108020 2024-09-11T01:22:13.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37dc20a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37d4111b60 con 0x7f37d4101c90 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37dc20a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37d4111e60 con 0x7f37d41076a0 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37da7a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37d4108020 0x7f37d4109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37da7a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37d4108020 0x7f37d4109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35236/0 (socket says 172.21.15.78:35236) 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37da7a7640 1 -- 172.21.15.78:0/2369759368 learned_addr learned my addr 172.21.15.78:0/2369759368 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37d9fa6640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37d4101c90 0x7f37d4108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.113+0000 7f37d97a5640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37d41076a0 0x7f37d4108d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.114+0000 7f37da7a7640 1 -- 172.21.15.78:0/2369759368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37d41076a0 msgr2=0x7f37d4108d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.114+0000 7f37da7a7640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37d41076a0 0x7f37d4108d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.114+0000 7f37da7a7640 1 -- 172.21.15.78:0/2369759368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37d4101c90 msgr2=0x7f37d4108810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.114+0000 7f37da7a7640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37d4101c90 0x7f37d4108810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.114+0000 7f37da7a7640 1 -- 172.21.15.78:0/2369759368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d41ab950 con 0x7f37d4108020 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.114+0000 7f37da7a7640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37d4108020 0x7f37d4109290 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00eb10 tx=0x7f37cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.114+0000 7f37caffd640 1 -- 172.21.15.78:0/2369759368 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc004b20 con 0x7f37d4108020 2024-09-11T01:22:13.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.114+0000 7f37dc20a640 1 -- 172.21.15.78:0/2369759368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d41abc40 con 0x7f37d4108020 2024-09-11T01:22:13.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.114+0000 7f37caffd640 1 -- 172.21.15.78:0/2369759368 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37cc004cc0 con 0x7f37d4108020 2024-09-11T01:22:13.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.114+0000 7f37dc20a640 1 -- 172.21.15.78:0/2369759368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d41ac120 con 0x7f37d4108020 2024-09-11T01:22:13.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.114+0000 7f37caffd640 1 -- 172.21.15.78:0/2369759368 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc010690 con 0x7f37d4108020 2024-09-11T01:22:13.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.118+0000 7f37de5df740 1 Processor -- start 2024-09-11T01:22:13.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.119+0000 7f37caffd640 1 -- 172.21.15.78:0/2369759368 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f37cc010830 con 0x7f37d4108020 2024-09-11T01:22:13.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.119+0000 7f37de5df740 1 -- start start 2024-09-11T01:22:13.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.119+0000 7f37de5df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe84d0 0x55b819fe88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.119+0000 7f37de5df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 0x55b819ff4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.119+0000 7f37de5df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819ff4d80 0x55b819ff7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.119+0000 7f37de5df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b819f5eea0 con 0x55b819fe8df0 2024-09-11T01:22:13.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.119+0000 7f37de5df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b819f4e2d0 con 0x55b819fe84d0 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.119+0000 7f37de5df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b819f520e0 con 0x55b819ff4d80 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.119+0000 7f37d9fa6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 0x55b819ff4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37d9fa6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 0x55b819ff4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35252/0 (socket says 172.21.15.78:35252) 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37d9fa6640 1 -- 172.21.15.78:0/1123350128 learned_addr learned my addr 172.21.15.78:0/1123350128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37d97a5640 1 --2- 172.21.15.78:0/1123350128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819ff4d80 0x55b819ff7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37d97a5640 1 -- 172.21.15.78:0/1123350128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe84d0 msgr2=0x55b819fe88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37d97a5640 1 --2- 172.21.15.78:0/1123350128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe84d0 0x55b819fe88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37d97a5640 1 -- 172.21.15.78:0/1123350128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 msgr2=0x55b819ff4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37caffd640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f37c00b54d0 0x7f37c00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37d97a5640 1 --2- 172.21.15.78:0/1123350128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 0x55b819ff4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37d97a5640 1 -- 172.21.15.78:0/1123350128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b819cac650 con 0x55b819ff4d80 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37d9fa6640 1 --2- 172.21.15.78:0/1123350128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 0x55b819ff4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:22:13.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37caffd640 1 -- 172.21.15.78:0/2369759368 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8493+0+0 (secure 0 0 0) 0x7f37cc0e62c0 con 0x7f37d4108020 2024-09-11T01:22:13.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37d97a5640 1 --2- 172.21.15.78:0/1123350128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819ff4d80 0x55b819ff7170 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f37d410c320 tx=0x7f37c4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d5a3feaaace27e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:13.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.120+0000 7f37d97a5640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f37c00b54d0 0x7f37c00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.121+0000 7f37bffff640 1 -- 172.21.15.78:0/1123350128 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c401c5b0 con 0x55b819ff4d80 2024-09-11T01:22:13.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.121+0000 7f37bffff640 1 -- 172.21.15.78:0/1123350128 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c400fd40 con 0x55b819ff4d80 2024-09-11T01:22:13.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.121+0000 7f37bffff640 1 -- 172.21.15.78:0/1123350128 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c401a950 con 0x55b819ff4d80 2024-09-11T01:22:13.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.122+0000 7f37de5df740 1 -- 172.21.15.78:0/1123350128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819ff4d80 msgr2=0x55b819ff7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:13.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.122+0000 7f37de5df740 1 --2- 172.21.15.78:0/1123350128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819ff4d80 0x55b819ff7170 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f37d410c320 tx=0x7f37c4011810 comp rx=0 tx=0).stop 2024-09-11T01:22:13.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.122+0000 7f37d97a5640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f37c00b54d0 0x7f37c00b7990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f37c4021000 tx=0x7f37c4022000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:13.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.122+0000 7f37de5df740 1 -- 172.21.15.78:0/1123350128 shutdown_connections 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.122+0000 7f37de5df740 1 --2- 172.21.15.78:0/1123350128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819ff4d80 0x55b819ff7170 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.122+0000 7f37de5df740 1 --2- 172.21.15.78:0/1123350128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 0x55b819ff4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.122+0000 7f37de5df740 1 --2- 172.21.15.78:0/1123350128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe84d0 0x55b819fe88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.122+0000 7f37de5df740 1 -- 172.21.15.78:0/1123350128 >> 172.21.15.78:0/1123350128 conn(0x55b819fe6300 msgr2=0x55b819fe74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.122+0000 7f37de5df740 1 -- 172.21.15.78:0/1123350128 shutdown_connections 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.122+0000 7f37de5df740 1 -- 172.21.15.78:0/1123350128 wait complete. 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.122+0000 7f37de5df740 1 -- start start 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.122+0000 7f37de5df740 1 Processor -- start 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37de5df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819fe84d0 0x55b819ff4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37de5df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe8df0 0x55b819ff1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37de5df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819ff4d80 0x55b819ff2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37de5df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b819daf700 con 0x55b819ff4d80 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37d9fa6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819fe84d0 0x55b819ff4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37da7a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819ff4d80 0x55b819ff2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37d97a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe8df0 0x55b819ff1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37d9fa6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819fe84d0 0x55b819ff4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:51666/0 (socket says 172.21.15.78:51666) 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37d9fa6640 1 -- 172.21.15.78:0/3494482472 learned_addr learned my addr 172.21.15.78:0/3494482472 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37de5df740 1 -- 172.21.15.78:0/3494482472 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b819ee9160 con 0x55b819fe8df0 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37de5df740 1 -- 172.21.15.78:0/3494482472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b819ff7a00 con 0x55b819fe84d0 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37de5df740 10 client.0 _ll_register_callbacks cb 0x55b819f029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37da7a7640 1 -- 172.21.15.78:0/3494482472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819fe84d0 msgr2=0x55b819ff4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37da7a7640 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819fe84d0 0x55b819ff4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37da7a7640 1 -- 172.21.15.78:0/3494482472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe8df0 msgr2=0x55b819ff1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37da7a7640 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe8df0 0x55b819ff1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37da7a7640 1 -- 172.21.15.78:0/3494482472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b819f16c90 con 0x55b819ff4d80 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37d97a5640 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe8df0 0x55b819ff1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:22:13.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.123+0000 7f37d9fa6640 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819fe84d0 0x55b819ff4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.124+0000 7f37da7a7640 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819ff4d80 0x55b819ff2070 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0e3a30 tx=0x7f37cc0ac530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f486dcfe3215ea07 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.124+0000 7f37bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.124+0000 7f37bd7fa640 1 -- 172.21.15.78:0/3494482472 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc0a6e60 con 0x55b819ff4d80 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.124+0000 7f37bd7fa640 1 -- 172.21.15.78:0/3494482472 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8493+0+0 (secure 0 0 0) 0x7f37cc010b50 con 0x55b819ff4d80 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.124+0000 7f37de5df740 1 -- 172.21.15.78:0/3494482472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37c400fd40 con 0x55b819ff4d80 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.124+0000 7f37bd7fa640 1 -- 172.21.15.78:0/3494482472 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f37cc0ac270 con 0x55b819ff4d80 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.124+0000 7f37de5df740 10 client.19386 fetch_fsmap learned FSMap version 76 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.124+0000 7f37de5df740 1 -- 172.21.15.78:0/3494482472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55b81a0033d0 con 0x55b819ff4d80 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.124+0000 7f37bd7fa640 1 -- 172.21.15.78:0/3494482472 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37cc0e5070 con 0x55b819ff4d80 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.125+0000 7f37bd7fa640 1 -- 172.21.15.78:0/3494482472 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f37cc0a80b0 con 0x55b819ff4d80 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.125+0000 7f37de5df740 10 client.19386 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.125+0000 7f37de5df740 10 client.19386 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.125+0000 7f37de5df740 20 client.19386 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:13.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.125+0000 7f37de5df740 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b81a004980 0x55b81a006e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.125+0000 7f37de5df740 4 client.19386 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.125+0000 7f37de5df740 1 -- 172.21.15.78:0/3494482472 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f37c400fd40 con 0x55b81a004980 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.125+0000 7f37d9fa6640 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b81a004980 0x55b81a006e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.125+0000 7f37bd7fa640 10 client.19386 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.125+0000 7f37d9fa6640 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b81a004980 0x55b81a006e00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=86d2ec37e7db4607 server_cookie=d3f0770bd070f75b in_seq=0 out_seq=0 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.126+0000 7f37bd7fa640 1 -- 172.21.15.78:0/3494482472 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f37c400fd40 con 0x55b81a004980 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.126+0000 7f37bd7fa640 10 client.19386 handle_command_reply: tid=0 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.126+0000 7f37de5df740 1 client.19386 shutdown 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.127+0000 7f37de5df740 1 -- 172.21.15.78:0/3494482472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819ff4d80 msgr2=0x55b819ff2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.127+0000 7f37de5df740 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819ff4d80 0x55b819ff2070 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0e3a30 tx=0x7f37cc0ac530 comp rx=0 tx=0).stop 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.127+0000 7f37de5df740 1 -- 172.21.15.78:0/3494482472 shutdown_connections 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.127+0000 7f37de5df740 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819ff4d80 0x55b819ff2070 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.127+0000 7f37de5df740 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b81a004980 0x55b81a006e00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.127+0000 7f37de5df740 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe8df0 0x55b819ff1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.127+0000 7f37de5df740 1 --2- 172.21.15.78:0/3494482472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819fe84d0 0x55b819ff4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.127+0000 7f37bd7fa640 0 client.19386 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:22:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.127+0000 7f37de5df740 1 -- 172.21.15.78:0/3494482472 >> 172.21.15.78:0/3494482472 conn(0x55b819fe6300 msgr2=0x55b819febe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.127+0000 7f37de5df740 1 -- 172.21.15.78:0/3494482472 shutdown_connections 2024-09-11T01:22:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.127+0000 7f37de5df740 1 -- 172.21.15.78:0/3494482472 wait complete. 2024-09-11T01:22:13.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.127+0000 7f37de5df740 20 client.19386 trim_cache size 0 max 16384 2024-09-11T01:22:13.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.146+0000 7f37de5df740 1 Processor -- start 2024-09-11T01:22:13.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.146+0000 7f37de5df740 1 -- start start 2024-09-11T01:22:13.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.146+0000 7f37de5df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819fe84d0 0x55b819fe5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.146+0000 7f37de5df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 0x55b819feeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.146+0000 7f37de5df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819ff4d80 0x55b819fef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.146+0000 7f37de5df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37b800e740 con 0x55b819fe8df0 2024-09-11T01:22:13.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.146+0000 7f37de5df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8010340 con 0x55b819ff4d80 2024-09-11T01:22:13.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.146+0000 7f37de5df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37b800ee10 con 0x55b819fe84d0 2024-09-11T01:22:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.147+0000 7f37da7a7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819ff4d80 0x55b819fef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.147+0000 7f37da7a7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819ff4d80 0x55b819fef1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:35806/0 (socket says 172.21.15.78:35806) 2024-09-11T01:22:13.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.147+0000 7f37da7a7640 1 -- 172.21.15.78:0/4066561402 learned_addr learned my addr 172.21.15.78:0/4066561402 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.147+0000 7f37d9fa6640 1 --2- 172.21.15.78:0/4066561402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819fe84d0 0x55b819fe5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.147+0000 7f37d97a5640 1 --2- 172.21.15.78:0/4066561402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 0x55b819feeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.147+0000 7f37da7a7640 1 -- 172.21.15.78:0/4066561402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819fe84d0 msgr2=0x55b819fe5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.147+0000 7f37da7a7640 1 --2- 172.21.15.78:0/4066561402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819fe84d0 0x55b819fe5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.147+0000 7f37da7a7640 1 -- 172.21.15.78:0/4066561402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 msgr2=0x55b819feeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.147+0000 7f37da7a7640 1 --2- 172.21.15.78:0/4066561402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 0x55b819feeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.147+0000 7f37da7a7640 1 -- 172.21.15.78:0/4066561402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b819fe6d80 con 0x55b819ff4d80 2024-09-11T01:22:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.147+0000 7f37d97a5640 1 --2- 172.21.15.78:0/4066561402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 0x55b819feeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:22:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.147+0000 7f37da7a7640 1 --2- 172.21.15.78:0/4066561402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819ff4d80 0x55b819fef1e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0ac460 tx=0x7f37cc0ac660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e2896d35ceb2a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.148+0000 7f37bf7fe640 1 -- 172.21.15.78:0/4066561402 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc0cb070 con 0x55b819ff4d80 2024-09-11T01:22:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.148+0000 7f37bf7fe640 1 -- 172.21.15.78:0/4066561402 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37cc0e5070 con 0x55b819ff4d80 2024-09-11T01:22:13.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.148+0000 7f37bf7fe640 1 -- 172.21.15.78:0/4066561402 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc0a6e60 con 0x55b819ff4d80 2024-09-11T01:22:13.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.149+0000 7f37de5df740 1 -- 172.21.15.78:0/4066561402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819ff4d80 msgr2=0x55b819fef1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:13.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.149+0000 7f37de5df740 1 --2- 172.21.15.78:0/4066561402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819ff4d80 0x55b819fef1e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0ac460 tx=0x7f37cc0ac660 comp rx=0 tx=0).stop 2024-09-11T01:22:13.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.149+0000 7f37de5df740 1 -- 172.21.15.78:0/4066561402 shutdown_connections 2024-09-11T01:22:13.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.149+0000 7f37de5df740 1 --2- 172.21.15.78:0/4066561402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819ff4d80 0x55b819fef1e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.149+0000 7f37de5df740 1 --2- 172.21.15.78:0/4066561402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe8df0 0x55b819feeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.149+0000 7f37de5df740 1 --2- 172.21.15.78:0/4066561402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819fe84d0 0x55b819fe5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.149+0000 7f37de5df740 1 -- 172.21.15.78:0/4066561402 >> 172.21.15.78:0/4066561402 conn(0x55b819fe6300 msgr2=0x55b819ff38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:13.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.149+0000 7f37de5df740 1 -- 172.21.15.78:0/4066561402 shutdown_connections 2024-09-11T01:22:13.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.149+0000 7f37de5df740 1 -- 172.21.15.78:0/4066561402 wait complete. 2024-09-11T01:22:13.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.149+0000 7f37de5df740 1 -- start start 2024-09-11T01:22:13.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37de5df740 1 Processor -- start 2024-09-11T01:22:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37de5df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe84d0 0x55b819ff9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37de5df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe8df0 0x55b819ff98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37de5df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819ff4d80 0x55b81a0708d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37de5df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37b800fc70 con 0x55b819fe84d0 2024-09-11T01:22:13.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37de5df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37b800f5a0 con 0x55b819fe8df0 2024-09-11T01:22:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37de5df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37b800c740 con 0x55b819ff4d80 2024-09-11T01:22:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37de5df740 10 client.0 _ll_register_callbacks cb 0x7f37cc004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37d97a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe8df0 0x55b819ff98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37d97a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe8df0 0x55b819ff98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:35820/0 (socket says 172.21.15.78:35820) 2024-09-11T01:22:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37d97a5640 1 -- 172.21.15.78:0/2254892639 learned_addr learned my addr 172.21.15.78:0/2254892639 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37d9fa6640 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe84d0 0x55b819ff9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.150+0000 7f37da7a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819ff4d80 0x55b81a0708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.151+0000 7f37da7a7640 1 -- 172.21.15.78:0/2254892639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe8df0 msgr2=0x55b819ff98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.151+0000 7f37da7a7640 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe8df0 0x55b819ff98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.151+0000 7f37da7a7640 1 -- 172.21.15.78:0/2254892639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe84d0 msgr2=0x55b819ff9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.151+0000 7f37da7a7640 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe84d0 0x55b819ff9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.151+0000 7f37da7a7640 1 -- 172.21.15.78:0/2254892639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b81a070e40 con 0x55b819ff4d80 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.151+0000 7f37d9fa6640 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe84d0 0x55b819ff9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.151+0000 7f37da7a7640 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819ff4d80 0x55b81a0708d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0049d0 tx=0x7f37cc0a82e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d49dc95db3aab9c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.151+0000 7f37bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.151+0000 7f37bd7fa640 1 -- 172.21.15.78:0/2254892639 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc0cb070 con 0x55b819ff4d80 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.151+0000 7f37bd7fa640 1 -- 172.21.15.78:0/2254892639 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8493+0+0 (secure 0 0 0) 0x7f37cc010b50 con 0x55b819ff4d80 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.152+0000 7f37de5df740 1 -- 172.21.15.78:0/2254892639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37cc0e5070 con 0x55b819ff4d80 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.152+0000 7f37bd7fa640 1 -- 172.21.15.78:0/2254892639 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f37cc0a6e60 con 0x55b819ff4d80 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.152+0000 7f37de5df740 10 client.28375 fetch_fsmap learned FSMap version 76 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.152+0000 7f37de5df740 1 -- 172.21.15.78:0/2254892639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55b81a072140 con 0x55b819ff4d80 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.152+0000 7f37bd7fa640 1 -- 172.21.15.78:0/2254892639 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37cc0ace60 con 0x55b819ff4d80 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.153+0000 7f37bd7fa640 1 -- 172.21.15.78:0/2254892639 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f37cc0ab430 con 0x55b819ff4d80 2024-09-11T01:22:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.153+0000 7f37de5df740 10 client.28375 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.153+0000 7f37de5df740 10 client.28375 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.153+0000 7f37de5df740 20 client.28375 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.153+0000 7f37de5df740 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b81a0726f0 0x55b81a074b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.153+0000 7f37de5df740 4 client.28375 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.153+0000 7f37de5df740 1 -- 172.21.15.78:0/2254892639 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f37cc0e5070 con 0x55b81a0726f0 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.153+0000 7f37d9fa6640 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b81a0726f0 0x55b81a074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.154+0000 7f37d9fa6640 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b81a0726f0 0x55b81a074b10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6dd122bb433b73f1 server_cookie=f83d5dce28c199f6 in_seq=0 out_seq=0 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.154+0000 7f37bd7fa640 10 client.28375 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.154+0000 7f37bd7fa640 1 -- 172.21.15.78:0/2254892639 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f37cc0e5070 con 0x55b81a0726f0 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.154+0000 7f37bd7fa640 10 client.28375 handle_command_reply: tid=0 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.154+0000 7f37de5df740 1 client.28375 shutdown 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.155+0000 7f37de5df740 1 -- 172.21.15.78:0/2254892639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819ff4d80 msgr2=0x55b81a0708d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.155+0000 7f37de5df740 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819ff4d80 0x55b81a0708d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0049d0 tx=0x7f37cc0a82e0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.155+0000 7f37de5df740 1 -- 172.21.15.78:0/2254892639 shutdown_connections 2024-09-11T01:22:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.155+0000 7f37de5df740 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b819ff4d80 0x55b81a0708d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.155+0000 7f37de5df740 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55b81a0726f0 0x55b81a074b10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.155+0000 7f37de5df740 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b819fe8df0 0x55b819ff98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.155+0000 7f37bd7fa640 0 client.28375 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:22:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.155+0000 7f37de5df740 1 --2- 172.21.15.78:0/2254892639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b819fe84d0 0x55b819ff9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.155+0000 7f37de5df740 1 -- 172.21.15.78:0/2254892639 >> 172.21.15.78:0/2254892639 conn(0x55b819fe6300 msgr2=0x55b819feb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.156+0000 7f37de5df740 1 -- 172.21.15.78:0/2254892639 shutdown_connections 2024-09-11T01:22:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.156+0000 7f37de5df740 1 -- 172.21.15.78:0/2254892639 wait complete. 2024-09-11T01:22:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.156+0000 7f37de5df740 20 client.28375 trim_cache size 0 max 16384 2024-09-11T01:22:13.161 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:22:13.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.156+0000 7f37dc20a640 1 -- 172.21.15.78:0/2369759368 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f37c00b54d0 msgr2=0x7f37c00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:13.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.156+0000 7f37dc20a640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f37c00b54d0 0x7f37c00b7990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f37c4021000 tx=0x7f37c4022000 comp rx=0 tx=0).stop 2024-09-11T01:22:13.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.156+0000 7f37dc20a640 1 -- 172.21.15.78:0/2369759368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37d4108020 msgr2=0x7f37d4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:13.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.156+0000 7f37dc20a640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37d4108020 0x7f37d4109290 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00eb10 tx=0x7f37cc00c2d0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.157+0000 7f37dc20a640 1 -- 172.21.15.78:0/2369759368 shutdown_connections 2024-09-11T01:22:13.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.157+0000 7f37dc20a640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f37c00b54d0 0x7f37c00b7990 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.157+0000 7f37dc20a640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37d4108020 0x7f37d4109290 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.157+0000 7f37dc20a640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37d41076a0 0x7f37d4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.157+0000 7f37dc20a640 1 --2- 172.21.15.78:0/2369759368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37d4101c90 0x7f37d4108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.157+0000 7f37dc20a640 1 -- 172.21.15.78:0/2369759368 >> 172.21.15.78:0/2369759368 conn(0x7f37d40fd700 msgr2=0x7f37d410d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.157+0000 7f37dc20a640 1 -- 172.21.15.78:0/2369759368 shutdown_connections 2024-09-11T01:22:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:13.157+0000 7f37dc20a640 1 -- 172.21.15.78:0/2369759368 wait complete. 2024-09-11T01:22:13.806 INFO:tasks.workunit.client.0.smithi078.stdout: 10 375114 31.87 MB/sec execute 499 sec latency 76.143 ms 2024-09-11T01:22:13.890 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:22:14.235 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:22:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:14 smithi112 ceph-mon[35384]: pgmap v1056: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 9.1 MiB/s wr, 582 op/s 2024-09-11T01:22:14.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:14 smithi112 ceph-mon[35384]: osdmap e111: 12 total, 12 up, 12 in 2024-09-11T01:22:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:14 smithi078 ceph-mon[31315]: pgmap v1056: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 9.1 MiB/s wr, 582 op/s 2024-09-11T01:22:14.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:14 smithi078 ceph-mon[31315]: osdmap e111: 12 total, 12 up, 12 in 2024-09-11T01:22:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:14 smithi110 ceph-mon[34712]: pgmap v1056: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 9.1 MiB/s wr, 582 op/s 2024-09-11T01:22:14.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:14 smithi110 ceph-mon[34712]: osdmap e111: 12 total, 12 up, 12 in 2024-09-11T01:22:14.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.801+0000 7fc52c304640 1 -- 172.21.15.78:0/3860956096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524107720 msgr2=0x7fc524107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:14.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.801+0000 7fc52c304640 1 --2- 172.21.15.78:0/3860956096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524107720 0x7fc524107b60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc514009860 tx=0x7fc51402a360 comp rx=0 tx=0).stop 2024-09-11T01:22:14.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.803+0000 7fc52c304640 1 -- 172.21.15.78:0/3860956096 shutdown_connections 2024-09-11T01:22:14.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.803+0000 7fc52c304640 1 --2- 172.21.15.78:0/3860956096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5241080a0 0x7fc52410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.803+0000 7fc52c304640 1 --2- 172.21.15.78:0/3860956096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc524107720 0x7fc524107b60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.803+0000 7fc52c304640 1 --2- 172.21.15.78:0/3860956096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc524101d10 0x7fc5241020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.803+0000 7fc52c304640 1 -- 172.21.15.78:0/3860956096 >> 172.21.15.78:0/3860956096 conn(0x7fc5240fd7c0 msgr2=0x7fc5240ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:14.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.804+0000 7fc52c304640 1 -- 172.21.15.78:0/3860956096 shutdown_connections 2024-09-11T01:22:14.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.804+0000 7fc52c304640 1 -- 172.21.15.78:0/3860956096 wait complete. 2024-09-11T01:22:14.806 INFO:tasks.workunit.client.0.smithi078.stdout: 10 375645 31.86 MB/sec execute 500 sec latency 99.715 ms 2024-09-11T01:22:14.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.804+0000 7fc52c304640 1 Processor -- start 2024-09-11T01:22:14.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.805+0000 7fc52c304640 1 -- start start 2024-09-11T01:22:14.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.805+0000 7fc52c304640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc524101d10 0x7fc5241a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.805+0000 7fc52c304640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc524107720 0x7fc5241a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.805+0000 7fc52c304640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5241080a0 0x7fc52419f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.805+0000 7fc52989f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc524107720 0x7fc5241a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.805+0000 7fc52a0a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc524101d10 0x7fc5241a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.806+0000 7fc52989f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc524107720 0x7fc5241a59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39186/0 (socket says 172.21.15.78:39186) 2024-09-11T01:22:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.806+0000 7fc52989f640 1 -- 172.21.15.78:0/1357513915 learned_addr learned my addr 172.21.15.78:0/1357513915 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.806+0000 7fc52c304640 1 -- 172.21.15.78:0/1357513915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc524111cf0 con 0x7fc524101d10 2024-09-11T01:22:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.806+0000 7fc52c304640 1 -- 172.21.15.78:0/1357513915 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc524111b70 con 0x7fc524107720 2024-09-11T01:22:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.806+0000 7fc52a8a1640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5241080a0 0x7fc52419f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.806+0000 7fc52c304640 1 -- 172.21.15.78:0/1357513915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc524111e70 con 0x7fc5241080a0 2024-09-11T01:22:14.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.806+0000 7fc52a0a0640 1 -- 172.21.15.78:0/1357513915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5241080a0 msgr2=0x7fc52419f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:14.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.806+0000 7fc52a0a0640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5241080a0 0x7fc52419f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.806+0000 7fc52a0a0640 1 -- 172.21.15.78:0/1357513915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc524107720 msgr2=0x7fc5241a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:14.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.806+0000 7fc52a0a0640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc524107720 0x7fc5241a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.806+0000 7fc52a0a0640 1 -- 172.21.15.78:0/1357513915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5241a0250 con 0x7fc524101d10 2024-09-11T01:22:14.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.807+0000 7fc52a0a0640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc524101d10 0x7fc5241a5490 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c970 tx=0x7fc52000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:14.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.807+0000 7fc51b7fe640 1 -- 172.21.15.78:0/1357513915 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520007bb0 con 0x7fc524101d10 2024-09-11T01:22:14.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.807+0000 7fc51b7fe640 1 -- 172.21.15.78:0/1357513915 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc520007d50 con 0x7fc524101d10 2024-09-11T01:22:14.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.807+0000 7fc51b7fe640 1 -- 172.21.15.78:0/1357513915 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520018780 con 0x7fc524101d10 2024-09-11T01:22:14.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.807+0000 7fc52c304640 1 -- 172.21.15.78:0/1357513915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5241a0480 con 0x7fc524101d10 2024-09-11T01:22:14.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.808+0000 7fc52c304640 1 -- 172.21.15.78:0/1357513915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5241ac2d0 con 0x7fc524101d10 2024-09-11T01:22:14.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.812+0000 7fc52e6d9740 1 Processor -- start 2024-09-11T01:22:14.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.812+0000 7fc51b7fe640 1 -- 172.21.15.78:0/1357513915 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc520018920 con 0x7fc524101d10 2024-09-11T01:22:14.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.812+0000 7fc52e6d9740 1 -- start start 2024-09-11T01:22:14.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.813+0000 7fc52e6d9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.813+0000 7fc52e6d9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6ac460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.813+0000 7fc52e6d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.813+0000 7fc52e6d9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ccdd616ea0 con 0x55ccdd6a00f0 2024-09-11T01:22:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.813+0000 7fc52e6d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ccdd6062d0 con 0x55ccdd6ac9a0 2024-09-11T01:22:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.813+0000 7fc52e6d9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ccdd60a0e0 con 0x55ccdd6a0a10 2024-09-11T01:22:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.813+0000 7fc52a0a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.813+0000 7fc52a0a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53514/0 (socket says 172.21.15.78:53514) 2024-09-11T01:22:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.813+0000 7fc52a0a0640 1 -- 172.21.15.78:0/2818103395 learned_addr learned my addr 172.21.15.78:0/2818103395 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.813+0000 7fc52989f640 1 --2- 172.21.15.78:0/2818103395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.814+0000 7fc52a0a0640 1 -- 172.21.15.78:0/2818103395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6ac9a0 msgr2=0x55ccdd6aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.814+0000 7fc52a0a0640 1 --2- 172.21.15.78:0/2818103395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.814+0000 7fc52a8a1640 1 --2- 172.21.15.78:0/2818103395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.814+0000 7fc52a0a0640 1 -- 172.21.15.78:0/2818103395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 msgr2=0x55ccdd6a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.814+0000 7fc52a0a0640 1 --2- 172.21.15.78:0/2818103395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.814+0000 7fc52a0a0640 1 -- 172.21.15.78:0/2818103395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccdd364650 con 0x55ccdd6a0a10 2024-09-11T01:22:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.814+0000 7fc52a8a1640 1 --2- 172.21.15.78:0/2818103395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:22:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.814+0000 7fc52a0a0640 1 --2- 172.21.15.78:0/2818103395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6ac460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc5200e35d0 tx=0x7fc5200ee220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe65e933afcedc1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.814+0000 7fc518de6640 1 -- 172.21.15.78:0/2818103395 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5200f9070 con 0x55ccdd6a0a10 2024-09-11T01:22:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.814+0000 7fc518de6640 1 -- 172.21.15.78:0/2818103395 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5200eee30 con 0x55ccdd6a0a10 2024-09-11T01:22:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.815+0000 7fc51b7fe640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc5100b55a0 0x7fc5100b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.815+0000 7fc518de6640 1 -- 172.21.15.78:0/2818103395 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5200f3730 con 0x55ccdd6a0a10 2024-09-11T01:22:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.815+0000 7fc51b7fe640 1 -- 172.21.15.78:0/1357513915 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7fc5200e5ed0 con 0x7fc524101d10 2024-09-11T01:22:14.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.815+0000 7fc52a8a1640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc5100b55a0 0x7fc5100b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.816+0000 7fc52e6d9740 1 -- 172.21.15.78:0/2818103395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a0a10 msgr2=0x55ccdd6ac460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:14.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.816+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/2818103395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6ac460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc5200e35d0 tx=0x7fc5200ee220 comp rx=0 tx=0).stop 2024-09-11T01:22:14.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.816+0000 7fc52e6d9740 1 -- 172.21.15.78:0/2818103395 shutdown_connections 2024-09-11T01:22:14.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.816+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/2818103395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.816+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/2818103395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6ac460 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.816+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/2818103395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.816+0000 7fc52e6d9740 1 -- 172.21.15.78:0/2818103395 >> 172.21.15.78:0/2818103395 conn(0x55ccdd69df20 msgr2=0x55ccdd69f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:14.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.816+0000 7fc52a8a1640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc5100b55a0 0x7fc5100b7a60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009a60 tx=0x7fc51c0095c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:14.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.817+0000 7fc52e6d9740 1 -- 172.21.15.78:0/2818103395 shutdown_connections 2024-09-11T01:22:14.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.817+0000 7fc52e6d9740 1 -- 172.21.15.78:0/2818103395 wait complete. 2024-09-11T01:22:14.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.817+0000 7fc52e6d9740 1 -- start start 2024-09-11T01:22:14.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.817+0000 7fc52e6d9740 1 Processor -- start 2024-09-11T01:22:14.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52e6d9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52a0a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52a0a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6ac2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56788/0 (socket says 172.21.15.78:56788) 2024-09-11T01:22:14.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52e6d9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52a0a0640 1 -- 172.21.15.78:0/1054577740 learned_addr learned my addr 172.21.15.78:0/1054577740 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:14.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52e6d9740 1 -- 172.21.15.78:0/1054577740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ccdd467700 con 0x55ccdd6a00f0 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52e6d9740 1 -- 172.21.15.78:0/1054577740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ccdd5a1160 con 0x55ccdd6ac9a0 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52e6d9740 1 -- 172.21.15.78:0/1054577740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ccdd6af670 con 0x55ccdd6a0a10 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52e6d9740 10 client.0 _ll_register_callbacks cb 0x55ccdd5ba9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52a0a0640 1 -- 172.21.15.78:0/1054577740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a0a10 msgr2=0x55ccdd6a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52a0a0640 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52a8a1640 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52a0a0640 1 -- 172.21.15.78:0/1054577740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6ac9a0 msgr2=0x55ccdd6a9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52a0a0640 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6a9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52a0a0640 1 -- 172.21.15.78:0/1054577740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccdd5cec90 con 0x55ccdd6a00f0 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.818+0000 7fc52a8a1640 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.819+0000 7fc52a0a0640 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6ac2b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fc5200ac090 tx=0x7fc5200057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c739b6b61f55b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.819+0000 7fc50dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.819+0000 7fc50dffb640 1 -- 172.21.15.78:0/1054577740 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5200a89e0 con 0x55ccdd6a00f0 2024-09-11T01:22:14.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.819+0000 7fc50dffb640 1 -- 172.21.15.78:0/1054577740 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7fc5200a8600 con 0x55ccdd6a00f0 2024-09-11T01:22:14.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.819+0000 7fc52e6d9740 1 -- 172.21.15.78:0/1054577740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5200eee30 con 0x55ccdd6a00f0 2024-09-11T01:22:14.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.819+0000 7fc50dffb640 1 -- 172.21.15.78:0/1054577740 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fc5200abb00 con 0x55ccdd6a00f0 2024-09-11T01:22:14.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.820+0000 7fc52e6d9740 10 client.19398 fetch_fsmap learned FSMap version 76 2024-09-11T01:22:14.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.820+0000 7fc52e6d9740 1 -- 172.21.15.78:0/1054577740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55ccdd6bafb0 con 0x55ccdd6a00f0 2024-09-11T01:22:14.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.820+0000 7fc50dffb640 1 -- 172.21.15.78:0/1054577740 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5200eee30 con 0x55ccdd6a00f0 2024-09-11T01:22:14.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.820+0000 7fc50dffb640 1 -- 172.21.15.78:0/1054577740 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7fc5200d3170 con 0x55ccdd6a00f0 2024-09-11T01:22:14.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.820+0000 7fc52e6d9740 10 client.19398 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:22:14.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.820+0000 7fc52e6d9740 10 client.19398 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:22:14.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.820+0000 7fc52e6d9740 20 client.19398 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:14.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.820+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ccdd6bc300 0x55ccdd6be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.820+0000 7fc52e6d9740 4 client.19398 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:22:14.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.820+0000 7fc52e6d9740 1 -- 172.21.15.78:0/1054577740 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccdd618df0 con 0x55ccdd6bc300 2024-09-11T01:22:14.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.820+0000 7fc52989f640 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ccdd6bc300 0x55ccdd6be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.821+0000 7fc50dffb640 10 client.19398 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:22:14.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.821+0000 7fc52989f640 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ccdd6bc300 0x55ccdd6be6f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11824add3bf95024 server_cookie=6b863adac4ec8932 in_seq=0 out_seq=0 2024-09-11T01:22:14.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.823+0000 7fc50dffb640 1 -- 172.21.15.78:0/1054577740 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ccdd618df0 con 0x55ccdd6bc300 2024-09-11T01:22:14.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.823+0000 7fc50dffb640 10 client.19398 handle_command_reply: tid=0 2024-09-11T01:22:14.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.825+0000 7fc52e6d9740 1 client.19398 shutdown 2024-09-11T01:22:14.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.827+0000 7fc52e6d9740 1 -- 172.21.15.78:0/1054577740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 msgr2=0x55ccdd6ac2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:14.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.827+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6ac2b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fc5200ac090 tx=0x7fc5200057d0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.827+0000 7fc52e6d9740 1 -- 172.21.15.78:0/1054577740 shutdown_connections 2024-09-11T01:22:14.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.827+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.827+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.827+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ccdd6bc300 0x55ccdd6be6f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.827+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/1054577740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6ac2b0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.827+0000 7fc52e6d9740 1 -- 172.21.15.78:0/1054577740 >> 172.21.15.78:0/1054577740 conn(0x55ccdd69df20 msgr2=0x55ccdd6a3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:14.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.827+0000 7fc50dffb640 0 client.19398 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:22:14.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.828+0000 7fc52e6d9740 1 -- 172.21.15.78:0/1054577740 shutdown_connections 2024-09-11T01:22:14.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.828+0000 7fc52e6d9740 1 -- 172.21.15.78:0/1054577740 wait complete. 2024-09-11T01:22:14.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.828+0000 7fc52e6d9740 20 client.19398 trim_cache size 0 max 16384 2024-09-11T01:22:14.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.854+0000 7fc52e6d9740 1 Processor -- start 2024-09-11T01:22:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.854+0000 7fc52e6d9740 1 -- start start 2024-09-11T01:22:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.854+0000 7fc52e6d9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a00f0 0x55ccdd69d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.855+0000 7fc52e6d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.855+0000 7fc52a0a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a00f0 0x55ccdd69d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.855+0000 7fc52a0a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a00f0 0x55ccdd69d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53526/0 (socket says 172.21.15.78:53526) 2024-09-11T01:22:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.855+0000 7fc52a0a0640 1 -- 172.21.15.78:0/3652335899 learned_addr learned my addr 172.21.15.78:0/3652335899 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.855+0000 7fc52989f640 1 --2- 172.21.15.78:0/3652335899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.855+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/3652335899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.855+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3652335899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc50800e770 con 0x55ccdd6ac9a0 2024-09-11T01:22:14.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.855+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3652335899 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc508010370 con 0x55ccdd6a0a10 2024-09-11T01:22:14.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.855+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3652335899 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc50800ee40 con 0x55ccdd6a00f0 2024-09-11T01:22:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.856+0000 7fc52a8a1640 1 --2- 172.21.15.78:0/3652335899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.856+0000 7fc52989f640 1 -- 172.21.15.78:0/3652335899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a00f0 msgr2=0x55ccdd69d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.856+0000 7fc52989f640 1 --2- 172.21.15.78:0/3652335899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a00f0 0x55ccdd69d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.856+0000 7fc52989f640 1 -- 172.21.15.78:0/3652335899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6ac9a0 msgr2=0x55ccdd6add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.856+0000 7fc52989f640 1 --2- 172.21.15.78:0/3652335899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.856+0000 7fc52989f640 1 -- 172.21.15.78:0/3652335899 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccdd6af500 con 0x55ccdd6a0a10 2024-09-11T01:22:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.856+0000 7fc52a8a1640 1 --2- 172.21.15.78:0/3652335899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:22:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.857+0000 7fc52989f640 1 --2- 172.21.15.78:0/3652335899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6ab440 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc524103d60 tx=0x7fc514009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f18de0d3703fec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.857+0000 7fc50ffff640 1 -- 172.21.15.78:0/3652335899 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514037040 con 0x55ccdd6a0a10 2024-09-11T01:22:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.857+0000 7fc50ffff640 1 -- 172.21.15.78:0/3652335899 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc514004440 con 0x55ccdd6a0a10 2024-09-11T01:22:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.858+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3652335899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6a0a10 msgr2=0x55ccdd6ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.858+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/3652335899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6ab440 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc524103d60 tx=0x7fc514009540 comp rx=0 tx=0).stop 2024-09-11T01:22:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.858+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3652335899 shutdown_connections 2024-09-11T01:22:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.858+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/3652335899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.858+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/3652335899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6a0a10 0x55ccdd6ab440 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc524103d60 tx=0x7fc514009540 comp rx=0 tx=0).stop 2024-09-11T01:22:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.858+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/3652335899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6a00f0 0x55ccdd69d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.858+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3652335899 >> 172.21.15.78:0/3652335899 conn(0x55ccdd69df20 msgr2=0x55ccdd6aad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.860+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3652335899 shutdown_connections 2024-09-11T01:22:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.863+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3652335899 wait complete. 2024-09-11T01:22:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.863+0000 7fc52e6d9740 1 -- start start 2024-09-11T01:22:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.863+0000 7fc52e6d9740 1 Processor -- start 2024-09-11T01:22:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.864+0000 7fc52e6d9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6bd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.864+0000 7fc52e6d9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6bd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.864+0000 7fc52e6d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6bde00 0x55ccdd727e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.864+0000 7fc52e6d9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc50800fca0 con 0x55ccdd6a00f0 2024-09-11T01:22:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.864+0000 7fc52e6d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc50800f5d0 con 0x55ccdd6bde00 2024-09-11T01:22:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.864+0000 7fc52e6d9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc50800c770 con 0x55ccdd6ac9a0 2024-09-11T01:22:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.864+0000 7fc52989f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6bd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.864+0000 7fc52989f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6bd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53530/0 (socket says 172.21.15.78:53530) 2024-09-11T01:22:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.864+0000 7fc52989f640 1 -- 172.21.15.78:0/3138687744 learned_addr learned my addr 172.21.15.78:0/3138687744 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.865+0000 7fc52989f640 1 -- 172.21.15.78:0/3138687744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6bde00 msgr2=0x55ccdd727e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:22:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.865+0000 7fc52989f640 1 --2- 172.21.15.78:0/3138687744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6bde00 0x55ccdd727e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.865+0000 7fc52989f640 1 -- 172.21.15.78:0/3138687744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 msgr2=0x55ccdd6bd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.865+0000 7fc52989f640 1 --2- 172.21.15.78:0/3138687744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6bd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.865+0000 7fc52989f640 1 -- 172.21.15.78:0/3138687744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccdd69e9a0 con 0x55ccdd6ac9a0 2024-09-11T01:22:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.865+0000 7fc52e6d9740 10 client.0 _ll_register_callbacks cb 0x7fc5200f7070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.865+0000 7fc52989f640 1 --2- 172.21.15.78:0/3138687744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6bd8c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc514030490 tx=0x7fc514004f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f153d820a9d8e492 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.865+0000 7fc50dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:22:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.865+0000 7fc50dffb640 1 -- 172.21.15.78:0/3138687744 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514053020 con 0x55ccdd6ac9a0 2024-09-11T01:22:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.865+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3138687744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ccdd6148a0 con 0x55ccdd6ac9a0 2024-09-11T01:22:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.866+0000 7fc50dffb640 1 -- 172.21.15.78:0/3138687744 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7fc514004160 con 0x55ccdd6ac9a0 2024-09-11T01:22:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.866+0000 7fc50dffb640 1 -- 172.21.15.78:0/3138687744 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fc51405e050 con 0x55ccdd6ac9a0 2024-09-11T01:22:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.866+0000 7fc52e6d9740 10 client.28390 fetch_fsmap learned FSMap version 76 2024-09-11T01:22:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.866+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3138687744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=76}) -- 0x55ccdd729440 con 0x55ccdd6ac9a0 2024-09-11T01:22:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.867+0000 7fc50dffb640 1 -- 172.21.15.78:0/3138687744 <== mon.1 v2:172.21.15.112:3300/0 4 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7fc514005030 con 0x55ccdd6ac9a0 2024-09-11T01:22:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.867+0000 7fc52e6d9740 10 client.28390 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:22:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.867+0000 7fc52e6d9740 10 client.28390 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:22:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.867+0000 7fc52e6d9740 20 client.28390 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.867+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/3138687744 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ccdd729ad0 0x55ccdd72bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.867+0000 7fc52e6d9740 4 client.28390 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:22:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.867+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3138687744 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccdd6148a0 con 0x55ccdd729ad0 2024-09-11T01:22:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.867+0000 7fc52a0a0640 1 --2- 172.21.15.78:0/3138687744 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ccdd729ad0 0x55ccdd72bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.868+0000 7fc52a0a0640 1 --2- 172.21.15.78:0/3138687744 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ccdd729ad0 0x55ccdd72bf10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7899f863a6d5df5 server_cookie=7fd3f205e81ed319 in_seq=0 out_seq=0 2024-09-11T01:22:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.868+0000 7fc50dffb640 10 client.28390 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:22:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.868+0000 7fc50dffb640 1 -- 172.21.15.78:0/3138687744 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccdd6148a0 con 0x55ccdd729ad0 2024-09-11T01:22:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.868+0000 7fc50dffb640 10 client.28390 handle_command_reply: tid=0 2024-09-11T01:22:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.868+0000 7fc52e6d9740 1 client.28390 shutdown 2024-09-11T01:22:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.869+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3138687744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6ac9a0 msgr2=0x55ccdd6bd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.869+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/3138687744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6bd8c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc514030490 tx=0x7fc514004f70 comp rx=0 tx=0).stop 2024-09-11T01:22:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.869+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3138687744 shutdown_connections 2024-09-11T01:22:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.869+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/3138687744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdd6bde00 0x55ccdd727e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.870+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/3138687744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccdd6ac9a0 0x55ccdd6bd8c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.870+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/3138687744 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55ccdd729ad0 0x55ccdd72bf10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.870+0000 7fc52e6d9740 1 --2- 172.21.15.78:0/3138687744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ccdd6a00f0 0x55ccdd6bd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.870+0000 7fc50dffb640 0 client.28390 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:22:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.870+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3138687744 >> 172.21.15.78:0/3138687744 conn(0x55ccdd69df20 msgr2=0x55ccdd6ab550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.870+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3138687744 shutdown_connections 2024-09-11T01:22:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.870+0000 7fc52e6d9740 1 -- 172.21.15.78:0/3138687744 wait complete. 2024-09-11T01:22:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.870+0000 7fc52e6d9740 20 client.28390 trim_cache size 0 max 16384 2024-09-11T01:22:14.875 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:22:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.872+0000 7fc52c304640 1 -- 172.21.15.78:0/1357513915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc5100b55a0 msgr2=0x7fc5100b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.872+0000 7fc52c304640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc5100b55a0 0x7fc5100b7a60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009a60 tx=0x7fc51c0095c0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.872+0000 7fc52c304640 1 -- 172.21.15.78:0/1357513915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc524101d10 msgr2=0x7fc5241a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.872+0000 7fc52c304640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc524101d10 0x7fc5241a5490 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c970 tx=0x7fc52000ce30 comp rx=0 tx=0).stop 2024-09-11T01:22:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.872+0000 7fc52c304640 1 -- 172.21.15.78:0/1357513915 shutdown_connections 2024-09-11T01:22:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.872+0000 7fc52c304640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc5100b55a0 0x7fc5100b7a60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.872+0000 7fc52c304640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5241080a0 0x7fc52419f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.872+0000 7fc52c304640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc524107720 0x7fc5241a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.872+0000 7fc52c304640 1 --2- 172.21.15.78:0/1357513915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc524101d10 0x7fc5241a5490 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.872+0000 7fc52c304640 1 -- 172.21.15.78:0/1357513915 >> 172.21.15.78:0/1357513915 conn(0x7fc5240fd7c0 msgr2=0x7fc5240ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.873+0000 7fc52c304640 1 -- 172.21.15.78:0/1357513915 shutdown_connections 2024-09-11T01:22:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:14.873+0000 7fc52c304640 1 -- 172.21.15.78:0/1357513915 wait complete. 2024-09-11T01:22:15.719 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:22:15.806 INFO:tasks.workunit.client.0.smithi078.stdout: 10 376202 31.85 MB/sec execute 501 sec latency 85.871 ms 2024-09-11T01:22:16.120 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:22:16.474 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:16 smithi112 ceph-mon[35384]: pgmap v1058: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 711 op/s 2024-09-11T01:22:16.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:16 smithi078 ceph-mon[31315]: pgmap v1058: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 711 op/s 2024-09-11T01:22:16.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:16 smithi110 ceph-mon[34712]: pgmap v1058: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 711 op/s 2024-09-11T01:22:16.806 INFO:tasks.workunit.client.0.smithi078.stdout: 10 376850 31.85 MB/sec execute 502 sec latency 96.006 ms 2024-09-11T01:22:17.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.602+0000 7fb5ee30a640 1 -- 172.21.15.78:0/1296428953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e8102de0 msgr2=0x7fb5e81031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:17.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.602+0000 7fb5ee30a640 1 --2- 172.21.15.78:0/1296428953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e8102de0 0x7fb5e81031c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009900 tx=0x7fb5d802a3f0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.602+0000 7fb5ee30a640 1 -- 172.21.15.78:0/1296428953 shutdown_connections 2024-09-11T01:22:17.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.602+0000 7fb5ee30a640 1 --2- 172.21.15.78:0/1296428953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e8109170 0x7fb5e810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.602+0000 7fb5ee30a640 1 --2- 172.21.15.78:0/1296428953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e81087f0 0x7fb5e8108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.602+0000 7fb5ee30a640 1 --2- 172.21.15.78:0/1296428953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e8102de0 0x7fb5e81031c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.602+0000 7fb5ee30a640 1 -- 172.21.15.78:0/1296428953 >> 172.21.15.78:0/1296428953 conn(0x7fb5e80fe830 msgr2=0x7fb5e8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:17.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.603+0000 7fb5ee30a640 1 -- 172.21.15.78:0/1296428953 shutdown_connections 2024-09-11T01:22:17.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.603+0000 7fb5ee30a640 1 -- 172.21.15.78:0/1296428953 wait complete. 2024-09-11T01:22:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.603+0000 7fb5ee30a640 1 Processor -- start 2024-09-11T01:22:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.604+0000 7fb5ee30a640 1 -- start start 2024-09-11T01:22:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.604+0000 7fb5ee30a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e8102de0 0x7fb5e8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.604+0000 7fb5ee30a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e81087f0 0x7fb5e81997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.604+0000 7fb5ee30a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e8109170 0x7fb5e819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.604+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e8102de0 0x7fb5e8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.604+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e8102de0 0x7fb5e8199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53552/0 (socket says 172.21.15.78:53552) 2024-09-11T01:22:17.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.604+0000 7fb5ee30a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8112e30 con 0x7fb5e81087f0 2024-09-11T01:22:17.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.604+0000 7fb5ee30a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8112cb0 con 0x7fb5e8109170 2024-09-11T01:22:17.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.604+0000 7fb5e7fff640 1 -- 172.21.15.78:0/2627068181 learned_addr learned my addr 172.21.15.78:0/2627068181 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:17.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.604+0000 7fb5ee30a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8112fb0 con 0x7fb5e8102de0 2024-09-11T01:22:17.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.604+0000 7fb5e77fe640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e81087f0 0x7fb5e81997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.604+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e8109170 0x7fb5e819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.605+0000 7fb5e7fff640 1 -- 172.21.15.78:0/2627068181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e8109170 msgr2=0x7fb5e819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:17.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.605+0000 7fb5e7fff640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e8109170 0x7fb5e819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.605+0000 7fb5e7fff640 1 -- 172.21.15.78:0/2627068181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e81087f0 msgr2=0x7fb5e81997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:17.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.605+0000 7fb5e7fff640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e81087f0 0x7fb5e81997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.605+0000 7fb5e7fff640 1 -- 172.21.15.78:0/2627068181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e819f340 con 0x7fb5e8102de0 2024-09-11T01:22:17.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.605+0000 7fb5e7fff640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e8102de0 0x7fb5e8199290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8005940 tx=0x7fb5d80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:17.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.606+0000 7fb5e57fa640 1 -- 172.21.15.78:0/2627068181 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d803f030 con 0x7fb5e8102de0 2024-09-11T01:22:17.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.606+0000 7fb5e57fa640 1 -- 172.21.15.78:0/2627068181 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d80044c0 con 0x7fb5e8102de0 2024-09-11T01:22:17.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.606+0000 7fb5ee30a640 1 -- 172.21.15.78:0/2627068181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e819f5d0 con 0x7fb5e8102de0 2024-09-11T01:22:17.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.606+0000 7fb5e57fa640 1 -- 172.21.15.78:0/2627068181 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8035070 con 0x7fb5e8102de0 2024-09-11T01:22:17.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.606+0000 7fb5ee30a640 1 -- 172.21.15.78:0/2627068181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e81a5cf0 con 0x7fb5e8102de0 2024-09-11T01:22:17.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.609+0000 7fb5f06df740 1 Processor -- start 2024-09-11T01:22:17.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.609+0000 7fb5f06df740 1 -- start start 2024-09-11T01:22:17.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5f06df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d70f0 0x55d8737d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5f06df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d7a10 0x55d8737e7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5f06df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737e7db0 0x55d8737ea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5f06df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d87374dea0 con 0x55d8737d70f0 2024-09-11T01:22:17.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5f06df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d87373d2d0 con 0x55d8737d7a10 2024-09-11T01:22:17.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5f06df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d8737410e0 con 0x55d8737e7db0 2024-09-11T01:22:17.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5ec8a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d70f0 0x55d8737d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5ec8a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d70f0 0x55d8737d74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56840/0 (socket says 172.21.15.78:56840) 2024-09-11T01:22:17.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5ec8a7640 1 -- 172.21.15.78:0/2118219559 learned_addr learned my addr 172.21.15.78:0/2118219559 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:17.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5e77fe640 1 --2- 172.21.15.78:0/2118219559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737e7db0 0x55d8737ea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5e57fa640 1 -- 172.21.15.78:0/2627068181 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb5d803b9f0 con 0x7fb5e8102de0 2024-09-11T01:22:17.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5ec8a7640 1 -- 172.21.15.78:0/2118219559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737e7db0 msgr2=0x55d8737ea1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:17.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/2118219559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737e7db0 0x55d8737ea1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5ec8a7640 1 -- 172.21.15.78:0/2118219559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d7a10 msgr2=0x55d8737e7870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:22:17.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/2118219559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d7a10 0x55d8737e7870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.610+0000 7fb5ec8a7640 1 -- 172.21.15.78:0/2118219559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d87349b650 con 0x55d8737d70f0 2024-09-11T01:22:17.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.611+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/2118219559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d70f0 0x55d8737d74d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0011440 tx=0x7fb5e0011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c661b3f1576d4a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:17.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.611+0000 7fb5ce7fc640 1 -- 172.21.15.78:0/2118219559 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0004d20 con 0x55d8737d70f0 2024-09-11T01:22:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.611+0000 7fb5e57fa640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5d00b54d0 0x7fb5d00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.612+0000 7fb5ce7fc640 1 -- 172.21.15.78:0/2118219559 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e0004050 con 0x55d8737d70f0 2024-09-11T01:22:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.612+0000 7fb5e57fa640 1 -- 172.21.15.78:0/2627068181 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7fb5d8106570 con 0x7fb5e8102de0 2024-09-11T01:22:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.612+0000 7fb5e7fff640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5d00b54d0 0x7fb5d00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.612+0000 7fb5ce7fc640 1 -- 172.21.15.78:0/2118219559 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000f7a0 con 0x55d8737d70f0 2024-09-11T01:22:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.612+0000 7fb5f06df740 1 -- 172.21.15.78:0/2118219559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d70f0 msgr2=0x55d8737d74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.612+0000 7fb5f06df740 1 --2- 172.21.15.78:0/2118219559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d70f0 0x55d8737d74d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0011440 tx=0x7fb5e0011900 comp rx=0 tx=0).stop 2024-09-11T01:22:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.613+0000 7fb5f06df740 1 -- 172.21.15.78:0/2118219559 shutdown_connections 2024-09-11T01:22:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.613+0000 7fb5f06df740 1 --2- 172.21.15.78:0/2118219559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737e7db0 0x55d8737ea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.613+0000 7fb5f06df740 1 --2- 172.21.15.78:0/2118219559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d7a10 0x55d8737e7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.613+0000 7fb5f06df740 1 --2- 172.21.15.78:0/2118219559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d70f0 0x55d8737d74d0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.613+0000 7fb5f06df740 1 -- 172.21.15.78:0/2118219559 >> 172.21.15.78:0/2118219559 conn(0x55d8737d4f20 msgr2=0x55d8737d60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.613+0000 7fb5e7fff640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5d00b54d0 0x7fb5d00b7990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55d8737d6bd0 tx=0x7fb5d8005be0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.613+0000 7fb5f06df740 1 -- 172.21.15.78:0/2118219559 shutdown_connections 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.613+0000 7fb5f06df740 1 -- 172.21.15.78:0/2118219559 wait complete. 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.614+0000 7fb5f06df740 1 -- start start 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.614+0000 7fb5f06df740 1 Processor -- start 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.614+0000 7fb5f06df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 0x55d8737e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.614+0000 7fb5f06df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 0x55d8737e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.614+0000 7fb5f06df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737e7db0 0x55d8737e5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.614+0000 7fb5f06df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d87359e700 con 0x55d8737d7a10 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.614+0000 7fb5f06df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d8736d8160 con 0x55d8737e7db0 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.614+0000 7fb5f06df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d8737eaab0 con 0x55d8737d70f0 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.614+0000 7fb5f06df740 10 client.0 _ll_register_callbacks cb 0x55d8736f19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:17.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.615+0000 7fb5ec8a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 0x55d8737e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.615+0000 7fb5ec8a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 0x55d8737e76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53580/0 (socket says 172.21.15.78:53580) 2024-09-11T01:22:17.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.615+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 0x55d8737e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.615+0000 7fb5ec8a7640 1 -- 172.21.15.78:0/3821536672 learned_addr learned my addr 172.21.15.78:0/3821536672 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:17.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.615+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/3821536672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737e7db0 0x55d8737e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.615+0000 7fb5e77fe640 1 -- 172.21.15.78:0/3821536672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 msgr2=0x55d8737e76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:17.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.615+0000 7fb5e77fe640 1 --2- 172.21.15.78:0/3821536672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 0x55d8737e76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.615+0000 7fb5e77fe640 1 -- 172.21.15.78:0/3821536672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737e7db0 msgr2=0x55d8737e5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.615+0000 7fb5e77fe640 1 --2- 172.21.15.78:0/3821536672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737e7db0 0x55d8737e5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.615+0000 7fb5e77fe640 1 -- 172.21.15.78:0/3821536672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d873705c90 con 0x55d8737d7a10 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.615+0000 7fb5e77fe640 1 --2- 172.21.15.78:0/3821536672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 0x55d8737e4b90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8102180 tx=0x7fb5dc012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfaedb1682ab3d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.615+0000 7fb5c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5c77fe640 1 -- 172.21.15.78:0/3821536672 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00f920 con 0x55d8737d7a10 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5c77fe640 1 -- 172.21.15.78:0/3821536672 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7fb5dc00fac0 con 0x55d8737d7a10 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5f06df740 1 -- 172.21.15.78:0/3821536672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e0004050 con 0x55d8737d7a10 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5c77fe640 1 -- 172.21.15.78:0/3821536672 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb5dc00fcc0 con 0x55d8737d7a10 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5f06df740 10 client.19413 fetch_fsmap learned FSMap version 76 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5f06df740 1 -- 172.21.15.78:0/3821536672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55d8737f6560 con 0x55d8737d7a10 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5c77fe640 1 -- 172.21.15.78:0/3821536672 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e0004050 con 0x55d8737d7a10 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5c77fe640 1 -- 172.21.15.78:0/3821536672 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7fb5dc01c420 con 0x55d8737d7a10 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5f06df740 10 client.19413 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:22:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5f06df740 10 client.19413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5f06df740 20 client.19413 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5f06df740 1 --2- 172.21.15.78:0/3821536672 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d8737f7ac0 0x55d8737f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5f06df740 4 client.19413 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.616+0000 7fb5f06df740 1 -- 172.21.15.78:0/3821536672 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d87374fdf0 con 0x55d8737f7ac0 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.617+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/3821536672 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d8737f7ac0 0x55d8737f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.617+0000 7fb5c77fe640 10 client.19413 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.617+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/3821536672 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d8737f7ac0 0x55d8737f9eb0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=478934d063b145bf server_cookie=f3591d5cf1ccdfa2 in_seq=0 out_seq=0 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.618+0000 7fb5c77fe640 1 -- 172.21.15.78:0/3821536672 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d87374fdf0 con 0x55d8737f7ac0 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.618+0000 7fb5c77fe640 10 client.19413 handle_command_reply: tid=0 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.618+0000 7fb5f06df740 1 client.19413 shutdown 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.618+0000 7fb5f06df740 1 -- 172.21.15.78:0/3821536672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 msgr2=0x55d8737e4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.618+0000 7fb5f06df740 1 --2- 172.21.15.78:0/3821536672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 0x55d8737e4b90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8102180 tx=0x7fb5dc012b60 comp rx=0 tx=0).stop 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.618+0000 7fb5f06df740 1 -- 172.21.15.78:0/3821536672 shutdown_connections 2024-09-11T01:22:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.618+0000 7fb5f06df740 1 --2- 172.21.15.78:0/3821536672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737e7db0 0x55d8737e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.618+0000 7fb5f06df740 1 --2- 172.21.15.78:0/3821536672 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d8737f7ac0 0x55d8737f9eb0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.618+0000 7fb5f06df740 1 --2- 172.21.15.78:0/3821536672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 0x55d8737e4b90 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.618+0000 7fb5f06df740 1 --2- 172.21.15.78:0/3821536672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 0x55d8737e76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.618+0000 7fb5f06df740 1 -- 172.21.15.78:0/3821536672 >> 172.21.15.78:0/3821536672 conn(0x55d8737d4f20 msgr2=0x55d8737daa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.618+0000 7fb5c77fe640 0 client.19413 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:22:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.619+0000 7fb5f06df740 1 -- 172.21.15.78:0/3821536672 shutdown_connections 2024-09-11T01:22:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.619+0000 7fb5f06df740 1 -- 172.21.15.78:0/3821536672 wait complete. 2024-09-11T01:22:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.619+0000 7fb5f06df740 20 client.19413 trim_cache size 0 max 16384 2024-09-11T01:22:17.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.639+0000 7fb5f06df740 1 Processor -- start 2024-09-11T01:22:17.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.639+0000 7fb5f06df740 1 -- start start 2024-09-11T01:22:17.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5f06df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d70f0 0x55d8737f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5f06df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 0x55d8737d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5f06df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737e7db0 0x55d8737d6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5f06df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ac4b0 con 0x55d8737d7a10 2024-09-11T01:22:17.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5f06df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ae0b0 con 0x55d8737d70f0 2024-09-11T01:22:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5f06df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81acb80 con 0x55d8737e7db0 2024-09-11T01:22:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5ec8a7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d70f0 0x55d8737f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 0x55d8737d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 0x55d8737d4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56858/0 (socket says 172.21.15.78:56858) 2024-09-11T01:22:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5ec8a7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d70f0 0x55d8737f3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39288/0 (socket says 172.21.15.78:39288) 2024-09-11T01:22:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5e77fe640 1 -- 172.21.15.78:0/80439925 learned_addr learned my addr 172.21.15.78:0/80439925 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/80439925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737e7db0 0x55d8737d6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5e77fe640 1 -- 172.21.15.78:0/80439925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737e7db0 msgr2=0x55d8737d6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5e77fe640 1 --2- 172.21.15.78:0/80439925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737e7db0 0x55d8737d6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5e77fe640 1 -- 172.21.15.78:0/80439925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d70f0 msgr2=0x55d8737f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5e77fe640 1 --2- 172.21.15.78:0/80439925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d70f0 0x55d8737f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5e77fe640 1 -- 172.21.15.78:0/80439925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8737e5580 con 0x55d8737d7a10 2024-09-11T01:22:17.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.640+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/80439925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d70f0 0x55d8737f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:22:17.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.641+0000 7fb5e77fe640 1 --2- 172.21.15.78:0/80439925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 0x55d8737d4760 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc01ca30 tx=0x7fb5dc0123a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94ef7917985cfd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:17.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.641+0000 7fb5cdffb640 1 -- 172.21.15.78:0/80439925 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc029020 con 0x55d8737d7a10 2024-09-11T01:22:17.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.641+0000 7fb5cdffb640 1 -- 172.21.15.78:0/80439925 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5dc00fe60 con 0x55d8737d7a10 2024-09-11T01:22:17.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.641+0000 7fb5cdffb640 1 -- 172.21.15.78:0/80439925 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc01c420 con 0x55d8737d7a10 2024-09-11T01:22:17.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.642+0000 7fb5f06df740 1 -- 172.21.15.78:0/80439925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 msgr2=0x55d8737d4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:17.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.642+0000 7fb5f06df740 1 --2- 172.21.15.78:0/80439925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 0x55d8737d4760 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc01ca30 tx=0x7fb5dc0123a0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.642+0000 7fb5f06df740 1 -- 172.21.15.78:0/80439925 shutdown_connections 2024-09-11T01:22:17.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.642+0000 7fb5f06df740 1 --2- 172.21.15.78:0/80439925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737e7db0 0x55d8737d6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.642+0000 7fb5f06df740 1 --2- 172.21.15.78:0/80439925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737d7a10 0x55d8737d4760 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.642+0000 7fb5f06df740 1 --2- 172.21.15.78:0/80439925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d70f0 0x55d8737f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.642+0000 7fb5f06df740 1 -- 172.21.15.78:0/80439925 >> 172.21.15.78:0/80439925 conn(0x55d8737d4f20 msgr2=0x55d8737d5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:17.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.642+0000 7fb5f06df740 1 -- 172.21.15.78:0/80439925 shutdown_connections 2024-09-11T01:22:17.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.642+0000 7fb5f06df740 1 -- 172.21.15.78:0/80439925 wait complete. 2024-09-11T01:22:17.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.643+0000 7fb5f06df740 1 -- start start 2024-09-11T01:22:17.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.643+0000 7fb5f06df740 1 Processor -- start 2024-09-11T01:22:17.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.643+0000 7fb5f06df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 0x55d8737e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5f06df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d7a10 0x55d8737efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5f06df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737e7db0 0x55d8737f0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5f06df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ad9e0 con 0x55d8737e7db0 2024-09-11T01:22:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5f06df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ad310 con 0x55d8737d7a10 2024-09-11T01:22:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5f06df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81aa4b0 con 0x55d8737d70f0 2024-09-11T01:22:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5f06df740 10 client.0 _ll_register_callbacks cb 0x7fb5dc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5ec8a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 0x55d8737e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5ec8a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 0x55d8737e2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53596/0 (socket says 172.21.15.78:53596) 2024-09-11T01:22:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5ec8a7640 1 -- 172.21.15.78:0/1484692860 learned_addr learned my addr 172.21.15.78:0/1484692860 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5e77fe640 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d7a10 0x55d8737efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737e7db0 0x55d8737f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5ec8a7640 1 -- 172.21.15.78:0/1484692860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d7a10 msgr2=0x55d8737efdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d7a10 0x55d8737efdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5ec8a7640 1 -- 172.21.15.78:0/1484692860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737e7db0 msgr2=0x55d8737f0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737e7db0 0x55d8737f0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.644+0000 7fb5ec8a7640 1 -- 172.21.15.78:0/1484692860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8737dfc50 con 0x55d8737d70f0 2024-09-11T01:22:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.645+0000 7fb5ec8a7640 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 0x55d8737e2760 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0011440 tx=0x7fb5e002ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad5d830ff71bd1a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.645+0000 7fb5c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:22:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.645+0000 7fb5c77fe640 1 -- 172.21.15.78:0/1484692860 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e001d040 con 0x55d8737d70f0 2024-09-11T01:22:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.645+0000 7fb5f06df740 1 -- 172.21.15.78:0/1484692860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5dc00fe60 con 0x55d8737d70f0 2024-09-11T01:22:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.645+0000 7fb5c77fe640 1 -- 172.21.15.78:0/1484692860 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7fb5e00043c0 con 0x55d8737d70f0 2024-09-11T01:22:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.645+0000 7fb5c77fe640 1 -- 172.21.15.78:0/1484692860 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb5e00045c0 con 0x55d8737d70f0 2024-09-11T01:22:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.645+0000 7fb5f06df740 10 client.28408 fetch_fsmap learned FSMap version 76 2024-09-11T01:22:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.645+0000 7fb5f06df740 1 -- 172.21.15.78:0/1484692860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55d873869470 con 0x55d8737d70f0 2024-09-11T01:22:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.646+0000 7fb5c77fe640 1 -- 172.21.15.78:0/1484692860 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5dc00fe60 con 0x55d8737d70f0 2024-09-11T01:22:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.646+0000 7fb5c77fe640 1 -- 172.21.15.78:0/1484692860 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7fb5e0029420 con 0x55d8737d70f0 2024-09-11T01:22:17.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.646+0000 7fb5f06df740 10 client.28408 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:22:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.646+0000 7fb5f06df740 10 client.28408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:22:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.646+0000 7fb5f06df740 20 client.28408 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.646+0000 7fb5f06df740 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d873869a40 0x55d87386be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.646+0000 7fb5f06df740 4 client.28408 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:22:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.646+0000 7fb5f06df740 1 -- 172.21.15.78:0/1484692860 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d87374b8a0 con 0x55d873869a40 2024-09-11T01:22:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.646+0000 7fb5e77fe640 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d873869a40 0x55d87386be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.647+0000 7fb5e77fe640 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d873869a40 0x55d87386be60 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12960bec27798aef server_cookie=8f919e89843a8f0e in_seq=0 out_seq=0 2024-09-11T01:22:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.647+0000 7fb5c77fe640 10 client.28408 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:22:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.648+0000 7fb5c77fe640 1 -- 172.21.15.78:0/1484692860 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d87374b8a0 con 0x55d873869a40 2024-09-11T01:22:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.648+0000 7fb5c77fe640 10 client.28408 handle_command_reply: tid=0 2024-09-11T01:22:17.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.648+0000 7fb5f06df740 1 client.28408 shutdown 2024-09-11T01:22:17.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.648+0000 7fb5f06df740 1 -- 172.21.15.78:0/1484692860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 msgr2=0x55d8737e2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:17.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.648+0000 7fb5f06df740 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 0x55d8737e2760 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0011440 tx=0x7fb5e002ff90 comp rx=0 tx=0).stop 2024-09-11T01:22:17.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.649+0000 7fb5f06df740 1 -- 172.21.15.78:0/1484692860 shutdown_connections 2024-09-11T01:22:17.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.649+0000 7fb5f06df740 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55d873869a40 0x55d87386be60 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.649+0000 7fb5c77fe640 0 client.28408 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:22:17.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.649+0000 7fb5f06df740 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8737e7db0 0x55d8737f0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.649+0000 7fb5f06df740 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d8737d7a10 0x55d8737efdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.649+0000 7fb5f06df740 1 --2- 172.21.15.78:0/1484692860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8737d70f0 0x55d8737e2760 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.649+0000 7fb5f06df740 1 -- 172.21.15.78:0/1484692860 >> 172.21.15.78:0/1484692860 conn(0x55d8737d4f20 msgr2=0x55d8737d6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:17.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.649+0000 7fb5f06df740 1 -- 172.21.15.78:0/1484692860 shutdown_connections 2024-09-11T01:22:17.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.649+0000 7fb5f06df740 1 -- 172.21.15.78:0/1484692860 wait complete. 2024-09-11T01:22:17.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.649+0000 7fb5f06df740 20 client.28408 trim_cache size 0 max 16384 2024-09-11T01:22:17.655 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:22:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.650+0000 7fb5ee30a640 1 -- 172.21.15.78:0/2627068181 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5d00b54d0 msgr2=0x7fb5d00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.650+0000 7fb5ee30a640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5d00b54d0 0x7fb5d00b7990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55d8737d6bd0 tx=0x7fb5d8005be0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.650+0000 7fb5ee30a640 1 -- 172.21.15.78:0/2627068181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e8102de0 msgr2=0x7fb5e8199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.650+0000 7fb5ee30a640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e8102de0 0x7fb5e8199290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8005940 tx=0x7fb5d80040c0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.650+0000 7fb5ee30a640 1 -- 172.21.15.78:0/2627068181 shutdown_connections 2024-09-11T01:22:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.650+0000 7fb5ee30a640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fb5d00b54d0 0x7fb5d00b7990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.650+0000 7fb5ee30a640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e8109170 0x7fb5e819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.650+0000 7fb5ee30a640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e81087f0 0x7fb5e81997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.650+0000 7fb5ee30a640 1 --2- 172.21.15.78:0/2627068181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5e8102de0 0x7fb5e8199290 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.650+0000 7fb5ee30a640 1 -- 172.21.15.78:0/2627068181 >> 172.21.15.78:0/2627068181 conn(0x7fb5e80fe830 msgr2=0x7fb5e8100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.650+0000 7fb5ee30a640 1 -- 172.21.15.78:0/2627068181 shutdown_connections 2024-09-11T01:22:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:17.650+0000 7fb5ee30a640 1 -- 172.21.15.78:0/2627068181 wait complete. 2024-09-11T01:22:17.806 INFO:tasks.workunit.client.0.smithi078.stdout: 10 377576 31.87 MB/sec execute 503 sec latency 85.106 ms 2024-09-11T01:22:18.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:18 smithi078 ceph-mon[31315]: pgmap v1059: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 699 op/s 2024-09-11T01:22:18.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:18 smithi110 ceph-mon[34712]: pgmap v1059: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 699 op/s 2024-09-11T01:22:18.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:18 smithi112 ceph-mon[35384]: pgmap v1059: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 699 op/s 2024-09-11T01:22:18.806 INFO:tasks.workunit.client.0.smithi078.stdout: 10 378244 31.89 MB/sec execute 504 sec latency 79.491 ms 2024-09-11T01:22:19.573 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:22:19.573 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:22:19.806 INFO:tasks.workunit.client.0.smithi078.stdout: 10 378790 31.88 MB/sec execute 505 sec latency 144.896 ms 2024-09-11T01:22:20.038 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:22:20.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:20 smithi078 ceph-mon[31315]: pgmap v1060: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 668 op/s 2024-09-11T01:22:20.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:20 smithi110 ceph-mon[34712]: pgmap v1060: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 668 op/s 2024-09-11T01:22:20.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:20 smithi112 ceph-mon[35384]: pgmap v1060: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 668 op/s 2024-09-11T01:22:20.806 INFO:tasks.workunit.client.0.smithi078.stdout: 10 378898 31.83 MB/sec execute 506 sec latency 678.378 ms 2024-09-11T01:22:21.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 379328 31.80 MB/sec execute 507 sec latency 166.953 ms 2024-09-11T01:22:21.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.872+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3628139938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb681045e0 msgr2=0x7fbb6810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:21.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.872+0000 7fbb6d3fc640 1 --2- 172.21.15.78:0/3628139938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb681045e0 0x7fbb6810e020 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009be0 tx=0x7fbb6002a6c0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.873+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3628139938 shutdown_connections 2024-09-11T01:22:21.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.873+0000 7fbb6d3fc640 1 --2- 172.21.15.78:0/3628139938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb681045e0 0x7fbb6810e020 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.873+0000 7fbb6d3fc640 1 --2- 172.21.15.78:0/3628139938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb68103c40 0x7fbb681040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.873+0000 7fbb6d3fc640 1 --2- 172.21.15.78:0/3628139938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb68101810 0x7fbb68101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.873+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3628139938 >> 172.21.15.78:0/3628139938 conn(0x7fbb68076f20 msgr2=0x7fbb68077330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:21.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.873+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3628139938 shutdown_connections 2024-09-11T01:22:21.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.873+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3628139938 wait complete. 2024-09-11T01:22:21.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.874+0000 7fbb6d3fc640 1 Processor -- start 2024-09-11T01:22:21.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.874+0000 7fbb6d3fc640 1 -- start start 2024-09-11T01:22:21.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.874+0000 7fbb6d3fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68101810 0x7fbb68196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.875+0000 7fbb66ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68101810 0x7fbb68196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.875+0000 7fbb66ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68101810 0x7fbb68196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:39322/0 (socket says 172.21.15.78:39322) 2024-09-11T01:22:21.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.875+0000 7fbb6d3fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb68103c40 0x7fbb681973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.875+0000 7fbb6d3fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb681045e0 0x7fbb68197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.875+0000 7fbb66ffd640 1 -- 172.21.15.78:0/3193475787 learned_addr learned my addr 172.21.15.78:0/3193475787 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.875+0000 7fbb667fc640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb68103c40 0x7fbb681973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.876+0000 7fbb677fe640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb681045e0 0x7fbb68197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.876+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3193475787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68110bd0 con 0x7fbb681045e0 2024-09-11T01:22:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.876+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3193475787 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68110a50 con 0x7fbb68101810 2024-09-11T01:22:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.876+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3193475787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68110d50 con 0x7fbb68103c40 2024-09-11T01:22:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.877+0000 7fbb66ffd640 1 -- 172.21.15.78:0/3193475787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb68103c40 msgr2=0x7fbb681973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.877+0000 7fbb66ffd640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb68103c40 0x7fbb681973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.877+0000 7fbb66ffd640 1 -- 172.21.15.78:0/3193475787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb681045e0 msgr2=0x7fbb68197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.877+0000 7fbb66ffd640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb681045e0 0x7fbb68197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.877+0000 7fbb66ffd640 1 -- 172.21.15.78:0/3193475787 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb681aba80 con 0x7fbb68101810 2024-09-11T01:22:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.877+0000 7fbb677fe640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb681045e0 0x7fbb68197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:22:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.877+0000 7fbb66ffd640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68101810 0x7fbb68196eb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800ea60 tx=0x7fbb5800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.877+0000 7fbb4ffff640 1 -- 172.21.15.78:0/3193475787 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800cc80 con 0x7fbb68101810 2024-09-11T01:22:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.878+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3193475787 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb681abd70 con 0x7fbb68101810 2024-09-11T01:22:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.878+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3193475787 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb681ac270 con 0x7fbb68101810 2024-09-11T01:22:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.878+0000 7fbb4ffff640 1 -- 172.21.15.78:0/3193475787 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb5800ce20 con 0x7fbb68101810 2024-09-11T01:22:21.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.878+0000 7fbb4ffff640 1 -- 172.21.15.78:0/3193475787 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800cc80 con 0x7fbb68101810 2024-09-11T01:22:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.881+0000 7fbb6f7d1740 1 Processor -- start 2024-09-11T01:22:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.881+0000 7fbb6f7d1740 1 -- start start 2024-09-11T01:22:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.882+0000 7fbb6f7d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 0x5616dce492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.882+0000 7fbb6f7d1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce55250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.882+0000 7fbb6f7d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce55790 0x5616dce57b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.882+0000 7fbb4ffff640 1 -- 172.21.15.78:0/3193475787 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fbb5800b040 con 0x7fbb68101810 2024-09-11T01:22:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.882+0000 7fbb6f7d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5616dcdbfe60 con 0x5616dce48ee0 2024-09-11T01:22:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.882+0000 7fbb6f7d1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5616dcdaf290 con 0x5616dce49800 2024-09-11T01:22:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.882+0000 7fbb6f7d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5616dcdb30a0 con 0x5616dce55790 2024-09-11T01:22:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.882+0000 7fbb677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 0x5616dce492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.882+0000 7fbb677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 0x5616dce492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56904/0 (socket says 172.21.15.78:56904) 2024-09-11T01:22:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.882+0000 7fbb677fe640 1 -- 172.21.15.78:0/2775018299 learned_addr learned my addr 172.21.15.78:0/2775018299 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.882+0000 7fbb66ffd640 1 --2- 172.21.15.78:0/2775018299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce55250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.883+0000 7fbb4ffff640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbb500b5830 0x7fbb500b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.883+0000 7fbb4ffff640 1 -- 172.21.15.78:0/3193475787 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7fbb58013070 con 0x7fbb68101810 2024-09-11T01:22:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.883+0000 7fbb677fe640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbb500b5830 0x7fbb500b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.884+0000 7fbb667fc640 1 --2- 172.21.15.78:0/2775018299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce55790 0x5616dce57b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.884+0000 7fbb677fe640 1 -- 172.21.15.78:0/2775018299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce55790 msgr2=0x5616dce57b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.884+0000 7fbb677fe640 1 --2- 172.21.15.78:0/2775018299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce55790 0x5616dce57b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.884+0000 7fbb677fe640 1 -- 172.21.15.78:0/2775018299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 msgr2=0x5616dce55250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.884+0000 7fbb677fe640 1 --2- 172.21.15.78:0/2775018299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce55250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.884+0000 7fbb677fe640 1 -- 172.21.15.78:0/2775018299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616dcb0d930 con 0x5616dce48ee0 2024-09-11T01:22:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.884+0000 7fbb66ffd640 1 --2- 172.21.15.78:0/2775018299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce55250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:22:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.884+0000 7fbb667fc640 1 --2- 172.21.15.78:0/2775018299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce55790 0x5616dce57b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:22:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.884+0000 7fbb677fe640 1 --2- 172.21.15.78:0/2775018299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 0x5616dce492c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009be0 tx=0x7fbb600304a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e05af7b7673496c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.884+0000 7fbb4d7fa640 1 -- 172.21.15.78:0/2775018299 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6004f070 con 0x5616dce48ee0 2024-09-11T01:22:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.885+0000 7fbb4d7fa640 1 -- 172.21.15.78:0/2775018299 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb60003d40 con 0x5616dce48ee0 2024-09-11T01:22:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.885+0000 7fbb4d7fa640 1 -- 172.21.15.78:0/2775018299 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60036070 con 0x5616dce48ee0 2024-09-11T01:22:21.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.886+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/2775018299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 msgr2=0x5616dce492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:21.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.886+0000 7fbb677fe640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbb500b5830 0x7fbb500b7cf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbb6004c000 tx=0x7fbb60047000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:21.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.886+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/2775018299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 0x5616dce492c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009be0 tx=0x7fbb600304a0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.886+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/2775018299 shutdown_connections 2024-09-11T01:22:21.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.886+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/2775018299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce55790 0x5616dce57b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.886+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/2775018299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce55250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.886+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/2775018299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 0x5616dce492c0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.886+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/2775018299 >> 172.21.15.78:0/2775018299 conn(0x5616dce46d10 msgr2=0x5616dce47eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:21.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.886+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/2775018299 shutdown_connections 2024-09-11T01:22:21.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.886+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/2775018299 wait complete. 2024-09-11T01:22:21.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.886+0000 7fbb6f7d1740 1 -- start start 2024-09-11T01:22:21.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.886+0000 7fbb6f7d1740 1 Processor -- start 2024-09-11T01:22:21.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb6f7d1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce48ee0 0x5616dce526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb6f7d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce49800 0x5616dce52c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb6f7d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce55790 0x5616dce53170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb6f7d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5616dcc10890 con 0x5616dce55790 2024-09-11T01:22:21.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb6f7d1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5616dcd49d40 con 0x5616dce48ee0 2024-09-11T01:22:21.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb6f7d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5616dce58440 con 0x5616dce49800 2024-09-11T01:22:21.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb6f7d1740 10 client.0 _ll_register_callbacks cb 0x5616dcdadc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:21.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce55790 0x5616dce53170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce55790 0x5616dce53170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56916/0 (socket says 172.21.15.78:56916) 2024-09-11T01:22:21.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb677fe640 1 -- 172.21.15.78:0/1243238891 learned_addr learned my addr 172.21.15.78:0/1243238891 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:21.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb677fe640 1 -- 172.21.15.78:0/1243238891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce49800 msgr2=0x5616dce52c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:22:21.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb677fe640 1 --2- 172.21.15.78:0/1243238891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce49800 0x5616dce52c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb677fe640 1 -- 172.21.15.78:0/1243238891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce48ee0 msgr2=0x5616dce526f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:22:21.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb677fe640 1 --2- 172.21.15.78:0/1243238891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce48ee0 0x5616dce526f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.887+0000 7fbb677fe640 1 -- 172.21.15.78:0/1243238891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616dce72830 con 0x5616dce55790 2024-09-11T01:22:21.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.888+0000 7fbb677fe640 1 --2- 172.21.15.78:0/1243238891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce55790 0x5616dce53170 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fbb60006010 tx=0x7fbb60005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d48a271e25a01b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:21.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.888+0000 7fbb4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:22:21.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.888+0000 7fbb4aff5640 1 -- 172.21.15.78:0/1243238891 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6004f070 con 0x5616dce55790 2024-09-11T01:22:21.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.888+0000 7fbb4aff5640 1 -- 172.21.15.78:0/1243238891 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7fbb60004040 con 0x5616dce55790 2024-09-11T01:22:21.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.888+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/1243238891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb60003d40 con 0x5616dce55790 2024-09-11T01:22:21.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.888+0000 7fbb4aff5640 1 -- 172.21.15.78:0/1243238891 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fbb60036050 con 0x5616dce55790 2024-09-11T01:22:21.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.888+0000 7fbb6f7d1740 10 client.19431 fetch_fsmap learned FSMap version 76 2024-09-11T01:22:21.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.888+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/1243238891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5616dce617d0 con 0x5616dce55790 2024-09-11T01:22:21.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.889+0000 7fbb4aff5640 1 -- 172.21.15.78:0/1243238891 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb60003d40 con 0x5616dce55790 2024-09-11T01:22:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.889+0000 7fbb4aff5640 1 -- 172.21.15.78:0/1243238891 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7fbb60048040 con 0x5616dce55790 2024-09-11T01:22:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.889+0000 7fbb6f7d1740 10 client.19431 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:22:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.889+0000 7fbb6f7d1740 10 client.19431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:22:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.889+0000 7fbb6f7d1740 20 client.19431 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.889+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/1243238891 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5616dce62b80 0x5616dce65040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.889+0000 7fbb6f7d1740 4 client.19431 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:22:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.889+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/1243238891 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616dcdc1db0 con 0x5616dce62b80 2024-09-11T01:22:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.889+0000 7fbb66ffd640 1 --2- 172.21.15.78:0/1243238891 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5616dce62b80 0x5616dce65040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.892+0000 7fbb66ffd640 1 --2- 172.21.15.78:0/1243238891 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5616dce62b80 0x5616dce65040 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a585b881462d30d server_cookie=6c3c5c7c2cc7e71c in_seq=0 out_seq=0 2024-09-11T01:22:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.892+0000 7fbb4aff5640 10 client.19431 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:22:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.893+0000 7fbb4aff5640 1 -- 172.21.15.78:0/1243238891 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5616dcdc1db0 con 0x5616dce62b80 2024-09-11T01:22:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.893+0000 7fbb4aff5640 10 client.19431 handle_command_reply: tid=0 2024-09-11T01:22:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.893+0000 7fbb6f7d1740 1 client.19431 shutdown 2024-09-11T01:22:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.893+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/1243238891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce55790 msgr2=0x5616dce53170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.893+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/1243238891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce55790 0x5616dce53170 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fbb60006010 tx=0x7fbb60005c60 comp rx=0 tx=0).stop 2024-09-11T01:22:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.893+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/1243238891 shutdown_connections 2024-09-11T01:22:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.893+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/1243238891 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5616dce62b80 0x5616dce65040 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.893+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/1243238891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce55790 0x5616dce53170 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.893+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/1243238891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce49800 0x5616dce52c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.893+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/1243238891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce48ee0 0x5616dce526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.893+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/1243238891 >> 172.21.15.78:0/1243238891 conn(0x5616dce46d10 msgr2=0x5616dce45f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.893+0000 7fbb4aff5640 0 client.19431 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:22:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.894+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/1243238891 shutdown_connections 2024-09-11T01:22:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.894+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/1243238891 wait complete. 2024-09-11T01:22:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.894+0000 7fbb6f7d1740 20 client.19431 trim_cache size 0 max 16384 2024-09-11T01:22:21.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb6f7d1740 1 Processor -- start 2024-09-11T01:22:21.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb6f7d1740 1 -- start start 2024-09-11T01:22:21.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb6f7d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 0x5616dce534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb6f7d1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce4cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb6f7d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce55790 0x5616dce4f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb6f7d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5400e780 con 0x5616dce48ee0 2024-09-11T01:22:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb6f7d1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54010380 con 0x5616dce49800 2024-09-11T01:22:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb6f7d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5400ee50 con 0x5616dce55790 2024-09-11T01:22:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb66ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 0x5616dce534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb66ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 0x5616dce534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56924/0 (socket says 172.21.15.78:56924) 2024-09-11T01:22:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb66ffd640 1 -- 172.21.15.78:0/3243158890 learned_addr learned my addr 172.21.15.78:0/3243158890 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb66ffd640 1 -- 172.21.15.78:0/3243158890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce55790 msgr2=0x5616dce4f940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:22:21.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb66ffd640 1 --2- 172.21.15.78:0/3243158890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce55790 0x5616dce4f940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb66ffd640 1 -- 172.21.15.78:0/3243158890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 msgr2=0x5616dce4cf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T01:22:21.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb66ffd640 1 --2- 172.21.15.78:0/3243158890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce4cf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.918+0000 7fbb66ffd640 1 -- 172.21.15.78:0/3243158890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616dce47790 con 0x5616dce48ee0 2024-09-11T01:22:21.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.919+0000 7fbb66ffd640 1 --2- 172.21.15.78:0/3243158890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 0x5616dce534e0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800cfd0 tx=0x7fbb5800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9045676b382ae5f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:21.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.919+0000 7fbb4cff9640 1 -- 172.21.15.78:0/3243158890 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb580abca0 con 0x5616dce48ee0 2024-09-11T01:22:21.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.919+0000 7fbb4cff9640 1 -- 172.21.15.78:0/3243158890 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb580ab5a0 con 0x5616dce48ee0 2024-09-11T01:22:21.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.919+0000 7fbb4cff9640 1 -- 172.21.15.78:0/3243158890 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb580e8040 con 0x5616dce48ee0 2024-09-11T01:22:21.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.921+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3243158890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 msgr2=0x5616dce534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:21.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.921+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/3243158890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 0x5616dce534e0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800cfd0 tx=0x7fbb5800ef90 comp rx=0 tx=0).stop 2024-09-11T01:22:21.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.922+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3243158890 shutdown_connections 2024-09-11T01:22:21.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.922+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/3243158890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce55790 0x5616dce4f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.922+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/3243158890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce4cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.922+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/3243158890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce48ee0 0x5616dce534e0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.922+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3243158890 >> 172.21.15.78:0/3243158890 conn(0x5616dce46d10 msgr2=0x5616dce54540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:21.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.923+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3243158890 shutdown_connections 2024-09-11T01:22:21.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.923+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3243158890 wait complete. 2024-09-11T01:22:21.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.923+0000 7fbb6f7d1740 1 -- start start 2024-09-11T01:22:21.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.923+0000 7fbb6f7d1740 1 Processor -- start 2024-09-11T01:22:21.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.924+0000 7fbb6f7d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce48ee0 0x5616dce66be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.924+0000 7fbb6f7d1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce67120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.924+0000 7fbb6f7d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce55790 0x5616dce67660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.924+0000 7fbb6f7d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5400fcb0 con 0x5616dce55790 2024-09-11T01:22:21.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.924+0000 7fbb66ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce48ee0 0x5616dce66be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.924+0000 7fbb667fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce67120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.924+0000 7fbb66ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce48ee0 0x5616dce66be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:53670/0 (socket says 172.21.15.78:53670) 2024-09-11T01:22:21.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.924+0000 7fbb66ffd640 1 -- 172.21.15.78:0/3118324729 learned_addr learned my addr 172.21.15.78:0/3118324729 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:21.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.924+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3118324729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5400f5e0 con 0x5616dce49800 2024-09-11T01:22:21.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.924+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3118324729 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5400c780 con 0x5616dce48ee0 2024-09-11T01:22:21.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.924+0000 7fbb677fe640 1 --2- 172.21.15.78:0/3118324729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce55790 0x5616dce67660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.924+0000 7fbb6f7d1740 10 client.0 _ll_register_callbacks cb 0x7fbb6004d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:21.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.925+0000 7fbb667fc640 1 -- 172.21.15.78:0/3118324729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce48ee0 msgr2=0x5616dce66be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:21.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.925+0000 7fbb667fc640 1 --2- 172.21.15.78:0/3118324729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce48ee0 0x5616dce66be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.925+0000 7fbb667fc640 1 -- 172.21.15.78:0/3118324729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce55790 msgr2=0x5616dce67660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:21.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.925+0000 7fbb667fc640 1 --2- 172.21.15.78:0/3118324729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce55790 0x5616dce67660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.925+0000 7fbb667fc640 1 -- 172.21.15.78:0/3118324729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616dcee2420 con 0x5616dce49800 2024-09-11T01:22:21.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.929+0000 7fbb667fc640 1 --2- 172.21.15.78:0/3118324729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce67120 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5616dce38ae0 tx=0x7fbb5c013b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de6ebb0af5bc26d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.929+0000 7fbb4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.929+0000 7fbb4aff5640 1 -- 172.21.15.78:0/3118324729 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c011910 con 0x5616dce49800 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.929+0000 7fbb4aff5640 1 -- 172.21.15.78:0/3118324729 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7fbb5c011ab0 con 0x5616dce49800 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.930+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3118324729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb580ab5a0 con 0x5616dce49800 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.930+0000 7fbb4aff5640 1 -- 172.21.15.78:0/3118324729 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fbb5c011cb0 con 0x5616dce49800 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.930+0000 7fbb6f7d1740 10 client.28463 fetch_fsmap learned FSMap version 76 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.930+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3118324729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5616dcee3790 con 0x5616dce49800 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.931+0000 7fbb4aff5640 1 -- 172.21.15.78:0/3118324729 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb580ab5a0 con 0x5616dce49800 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.931+0000 7fbb4aff5640 1 -- 172.21.15.78:0/3118324729 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7fbb5c01a070 con 0x5616dce49800 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.931+0000 7fbb6f7d1740 10 client.28463 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.931+0000 7fbb6f7d1740 10 client.28463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.931+0000 7fbb6f7d1740 20 client.28463 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:21.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.931+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/3118324729 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5616dcee3f40 0x5616dcee6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.931+0000 7fbb6f7d1740 4 client.28463 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.931+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3118324729 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5616dcdbd860 con 0x5616dcee3f40 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.931+0000 7fbb66ffd640 1 --2- 172.21.15.78:0/3118324729 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5616dcee3f40 0x5616dcee6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.932+0000 7fbb4aff5640 10 client.28463 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.932+0000 7fbb66ffd640 1 --2- 172.21.15.78:0/3118324729 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5616dcee3f40 0x5616dcee6360 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f91b54a900da56 server_cookie=ad857b09768e2c6d in_seq=0 out_seq=0 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.934+0000 7fbb4aff5640 1 -- 172.21.15.78:0/3118324729 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5616dcdbd860 con 0x5616dcee3f40 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.934+0000 7fbb4aff5640 10 client.28463 handle_command_reply: tid=0 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.934+0000 7fbb6f7d1740 1 client.28463 shutdown 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.934+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3118324729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 msgr2=0x5616dce67120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.934+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/3118324729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce67120 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5616dce38ae0 tx=0x7fbb5c013b70 comp rx=0 tx=0).stop 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.934+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3118324729 shutdown_connections 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.934+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/3118324729 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x5616dcee3f40 0x5616dcee6360 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.934+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/3118324729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616dce55790 0x5616dce67660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.934+0000 7fbb4aff5640 0 client.28463 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:22:21.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.934+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/3118324729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616dce49800 0x5616dce67120 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.934+0000 7fbb6f7d1740 1 --2- 172.21.15.78:0/3118324729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616dce48ee0 0x5616dce66be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.934+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3118324729 >> 172.21.15.78:0/3118324729 conn(0x5616dce46d10 msgr2=0x5616dce60730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:21.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.935+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3118324729 shutdown_connections 2024-09-11T01:22:21.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.935+0000 7fbb6f7d1740 1 -- 172.21.15.78:0/3118324729 wait complete. 2024-09-11T01:22:21.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.935+0000 7fbb6f7d1740 20 client.28463 trim_cache size 0 max 16384 2024-09-11T01:22:21.941 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:22:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:22:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "82736ae4-b55a-4ba3-911c-0aa1c18acfde", 2024-09-11T01:22:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:22:21.942 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:22:21.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.936+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3193475787 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbb500b5830 msgr2=0x7fbb500b7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:21.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.936+0000 7fbb6d3fc640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbb500b5830 0x7fbb500b7cf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbb6004c000 tx=0x7fbb60047000 comp rx=0 tx=0).stop 2024-09-11T01:22:21.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.936+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3193475787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68101810 msgr2=0x7fbb68196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:21.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.936+0000 7fbb6d3fc640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68101810 0x7fbb68196eb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800ea60 tx=0x7fbb5800ef20 comp rx=0 tx=0).stop 2024-09-11T01:22:21.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.936+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3193475787 shutdown_connections 2024-09-11T01:22:21.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.936+0000 7fbb6d3fc640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbb500b5830 0x7fbb500b7cf0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.936+0000 7fbb6d3fc640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb681045e0 0x7fbb68197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.936+0000 7fbb6d3fc640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb68103c40 0x7fbb681973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.936+0000 7fbb6d3fc640 1 --2- 172.21.15.78:0/3193475787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68101810 0x7fbb68196eb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:21.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.936+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3193475787 >> 172.21.15.78:0/3193475787 conn(0x7fbb68076f20 msgr2=0x7fbb68104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:21.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.936+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3193475787 shutdown_connections 2024-09-11T01:22:21.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:21.936+0000 7fbb6d3fc640 1 -- 172.21.15.78:0/3193475787 wait complete. 2024-09-11T01:22:22.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:22 smithi078 ceph-mon[31315]: pgmap v1061: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 580 op/s 2024-09-11T01:22:22.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:22 smithi110 ceph-mon[34712]: pgmap v1061: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 580 op/s 2024-09-11T01:22:22.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:22 smithi112 ceph-mon[35384]: pgmap v1061: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 580 op/s 2024-09-11T01:22:22.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 379652 31.77 MB/sec execute 508 sec latency 263.651 ms 2024-09-11T01:22:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:23 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:22:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:23 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:23 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:22:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:23 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:22:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:23 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:22:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:23 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:22:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:23 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:22:23.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:23 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:23.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:23 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:23.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:23 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:23.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:23 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:22:23.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:23 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:23.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:23 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:22:23.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:23 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:22:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:23 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:22:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:23 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:22:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:23 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:22:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:23 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:23 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:23.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:23 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:23 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:22:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:23 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:23 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:22:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:23 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:22:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:23 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:22:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:23 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:22:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:23 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:22:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:23 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:23 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:23.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:23 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:23.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 380255 31.79 MB/sec execute 509 sec latency 141.232 ms 2024-09-11T01:22:24.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:24 smithi078 ceph-mon[31315]: pgmap v1062: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 579 op/s 2024-09-11T01:22:24.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:24 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:24.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:24 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:24.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:24 smithi110 ceph-mon[34712]: pgmap v1062: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 579 op/s 2024-09-11T01:22:24.693 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:24 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:24.693 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:24 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:24.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:24 smithi112 ceph-mon[35384]: pgmap v1062: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 579 op/s 2024-09-11T01:22:24.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:24 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:24.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:24 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:22:24.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 380917 31.78 MB/sec execute 510 sec latency 111.902 ms 2024-09-11T01:22:25.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 381502 31.79 MB/sec execute 511 sec latency 94.666 ms 2024-09-11T01:22:26.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:26 smithi110 ceph-mon[34712]: pgmap v1063: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 687 op/s 2024-09-11T01:22:26.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:26 smithi112 ceph-mon[35384]: pgmap v1063: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 687 op/s 2024-09-11T01:22:26.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:26 smithi078 ceph-mon[31315]: pgmap v1063: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 687 op/s 2024-09-11T01:22:26.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 382083 31.81 MB/sec execute 512 sec latency 88.511 ms 2024-09-11T01:22:27.154 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:22:27.550 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:22:27.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 382563 31.77 MB/sec execute 513 sec latency 114.171 ms 2024-09-11T01:22:28.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.283+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/1119425456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dd81076a0 msgr2=0x7f1dd8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:28.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.283+0000 7f1ddf0ef640 1 --2- 172.21.15.78:0/1119425456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dd81076a0 0x7f1dd8107ae0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009900 tx=0x7f1dc402a410 comp rx=0 tx=0).stop 2024-09-11T01:22:28.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.284+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/1119425456 shutdown_connections 2024-09-11T01:22:28.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.284+0000 7f1ddf0ef640 1 --2- 172.21.15.78:0/1119425456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dd8108020 0x7f1dd810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.284+0000 7f1ddf0ef640 1 --2- 172.21.15.78:0/1119425456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dd81076a0 0x7f1dd8107ae0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.284+0000 7f1ddf0ef640 1 --2- 172.21.15.78:0/1119425456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd8101c90 0x7f1dd8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.284+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/1119425456 >> 172.21.15.78:0/1119425456 conn(0x7f1dd80fd700 msgr2=0x7f1dd80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:28.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.284+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/1119425456 shutdown_connections 2024-09-11T01:22:28.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.284+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/1119425456 wait complete. 2024-09-11T01:22:28.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.285+0000 7f1ddf0ef640 1 Processor -- start 2024-09-11T01:22:28.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.285+0000 7f1ddf0ef640 1 -- start start 2024-09-11T01:22:28.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.285+0000 7f1ddf0ef640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dd8101c90 0x7f1dd819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.285+0000 7f1ddf0ef640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dd81076a0 0x7f1dd819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.285+0000 7f1ddf0ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd8108020 0x7f1dd819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.285+0000 7f1ddf0ef640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8110b90 con 0x7f1dd81076a0 2024-09-11T01:22:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.285+0000 7f1ddf0ef640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8110a10 con 0x7f1dd8101c90 2024-09-11T01:22:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.285+0000 7f1ddf0ef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8110d10 con 0x7f1dd8108020 2024-09-11T01:22:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.285+0000 7f1ddce8b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dd8101c90 0x7f1dd819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.285+0000 7f1ddce8b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dd8101c90 0x7f1dd819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:44396/0 (socket says 172.21.15.78:44396) 2024-09-11T01:22:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.285+0000 7f1ddce8b640 1 -- 172.21.15.78:0/2238845006 learned_addr learned my addr 172.21.15.78:0/2238845006 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.285+0000 7f1ddd68c640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd8108020 0x7f1dd819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.286+0000 7f1ddce8b640 1 -- 172.21.15.78:0/2238845006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd8108020 msgr2=0x7f1dd819dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.286+0000 7f1ddce8b640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd8108020 0x7f1dd819dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.286+0000 7f1ddce8b640 1 -- 172.21.15.78:0/2238845006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dd81076a0 msgr2=0x7f1dd819bf70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T01:22:28.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.286+0000 7f1ddce8b640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dd81076a0 0x7f1dd819bf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.286+0000 7f1ddce8b640 1 -- 172.21.15.78:0/2238845006 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd819e330 con 0x7f1dd8101c90 2024-09-11T01:22:28.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.286+0000 7f1ddce8b640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dd8101c90 0x7f1dd819ba30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00e9f0 tx=0x7f1dcc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:28.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.286+0000 7f1dd5ffb640 1 -- 172.21.15.78:0/2238845006 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00cd80 con 0x7f1dd8101c90 2024-09-11T01:22:28.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.286+0000 7f1dd5ffb640 1 -- 172.21.15.78:0/2238845006 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dcc0054d0 con 0x7f1dd8101c90 2024-09-11T01:22:28.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.286+0000 7f1dd5ffb640 1 -- 172.21.15.78:0/2238845006 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc010430 con 0x7f1dd8101c90 2024-09-11T01:22:28.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.286+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/2238845006 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd819e620 con 0x7f1dd8101c90 2024-09-11T01:22:28.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.287+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/2238845006 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd8112740 con 0x7f1dd8101c90 2024-09-11T01:22:28.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.290+0000 7f1dd5ffb640 1 -- 172.21.15.78:0/2238845006 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1dcc005000 con 0x7f1dd8101c90 2024-09-11T01:22:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.290+0000 7f1de14c4740 1 Processor -- start 2024-09-11T01:22:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.290+0000 7f1de14c4740 1 -- start start 2024-09-11T01:22:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.291+0000 7f1de14c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7ce480 0x55affd7ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.291+0000 7f1de14c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ceda0 0x55affd7da7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.291+0000 7f1de14c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7dad30 0x55affd7dd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.291+0000 7f1de14c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55affd744e50 con 0x55affd7ce480 2024-09-11T01:22:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.291+0000 7f1de14c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55affd734280 con 0x55affd7ceda0 2024-09-11T01:22:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.291+0000 7f1de14c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55affd738090 con 0x55affd7dad30 2024-09-11T01:22:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.291+0000 7f1dd7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7dad30 0x55affd7dd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.291+0000 7f1dd7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7dad30 0x55affd7dd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52830/0 (socket says 172.21.15.78:52830) 2024-09-11T01:22:28.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.291+0000 7f1dd7fff640 1 -- 172.21.15.78:0/3093834744 learned_addr learned my addr 172.21.15.78:0/3093834744 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:28.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.291+0000 7f1ddd68c640 1 --2- 172.21.15.78:0/3093834744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7ce480 0x55affd7ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.291+0000 7f1ddce8b640 1 --2- 172.21.15.78:0/3093834744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ceda0 0x55affd7da7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.291+0000 7f1dd7fff640 1 -- 172.21.15.78:0/3093834744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ceda0 msgr2=0x55affd7da7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.292+0000 7f1dd7fff640 1 --2- 172.21.15.78:0/3093834744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ceda0 0x55affd7da7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.292+0000 7f1dd7fff640 1 -- 172.21.15.78:0/3093834744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7ce480 msgr2=0x55affd7ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.292+0000 7f1dd7fff640 1 --2- 172.21.15.78:0/3093834744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7ce480 0x55affd7ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.292+0000 7f1dd7fff640 1 -- 172.21.15.78:0/3093834744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55affd492650 con 0x55affd7dad30 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.292+0000 7f1ddce8b640 1 --2- 172.21.15.78:0/3093834744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ceda0 0x55affd7da7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.292+0000 7f1dd7fff640 1 --2- 172.21.15.78:0/3093834744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7dad30 0x55affd7dd120 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4005940 tx=0x7f1dc4009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9789f66f6701424 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.292+0000 7f1dd5ffb640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1dc80b54d0 0x7f1dc80b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.292+0000 7f1ddd68c640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1dc80b54d0 0x7f1dc80b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.292+0000 7f1dd5ffb640 1 -- 172.21.15.78:0/2238845006 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7f1dcc0e6410 con 0x7f1dd8101c90 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.293+0000 7f1dc2ffd640 1 -- 172.21.15.78:0/3093834744 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4040070 con 0x55affd7dad30 2024-09-11T01:22:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.293+0000 7f1ddd68c640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1dc80b54d0 0x7f1dc80b7990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0005fd0 tx=0x7f1dd0009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:28.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.293+0000 7f1dc2ffd640 1 -- 172.21.15.78:0/3093834744 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc4032da0 con 0x55affd7dad30 2024-09-11T01:22:28.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.293+0000 7f1dc2ffd640 1 -- 172.21.15.78:0/3093834744 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4035040 con 0x55affd7dad30 2024-09-11T01:22:28.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.294+0000 7f1de14c4740 1 -- 172.21.15.78:0/3093834744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7dad30 msgr2=0x55affd7dd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:28.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.294+0000 7f1de14c4740 1 --2- 172.21.15.78:0/3093834744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7dad30 0x55affd7dd120 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4005940 tx=0x7f1dc4009520 comp rx=0 tx=0).stop 2024-09-11T01:22:28.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.294+0000 7f1de14c4740 1 -- 172.21.15.78:0/3093834744 shutdown_connections 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.294+0000 7f1de14c4740 1 --2- 172.21.15.78:0/3093834744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7dad30 0x55affd7dd120 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.294+0000 7f1de14c4740 1 --2- 172.21.15.78:0/3093834744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ceda0 0x55affd7da7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.294+0000 7f1de14c4740 1 --2- 172.21.15.78:0/3093834744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7ce480 0x55affd7ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.294+0000 7f1de14c4740 1 -- 172.21.15.78:0/3093834744 >> 172.21.15.78:0/3093834744 conn(0x55affd7cc2b0 msgr2=0x55affd7cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.294+0000 7f1de14c4740 1 -- 172.21.15.78:0/3093834744 shutdown_connections 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.294+0000 7f1de14c4740 1 -- 172.21.15.78:0/3093834744 wait complete. 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.295+0000 7f1de14c4740 1 -- start start 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.295+0000 7f1de14c4740 1 Processor -- start 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.295+0000 7f1de14c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ce480 0x55affd7da620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.295+0000 7f1de14c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7ceda0 0x55affd7d7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.295+0000 7f1de14c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7dad30 0x55affd7d80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1de14c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55affd595700 con 0x55affd7ceda0 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1de14c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55affd6cf110 con 0x55affd7ce480 2024-09-11T01:22:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1de14c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55affd7dd950 con 0x55affd7dad30 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1ddce8b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ce480 0x55affd7da620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1ddce8b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ce480 0x55affd7da620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:44418/0 (socket says 172.21.15.78:44418) 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1ddce8b640 1 -- 172.21.15.78:0/529675984 learned_addr learned my addr 172.21.15.78:0/529675984 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1de14c4740 10 client.0 _ll_register_callbacks cb 0x55affd6e8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1dd7fff640 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7ceda0 0x55affd7d7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1ddd68c640 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7dad30 0x55affd7d80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1ddce8b640 1 -- 172.21.15.78:0/529675984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7dad30 msgr2=0x55affd7d80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1ddce8b640 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7dad30 0x55affd7d80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1ddce8b640 1 -- 172.21.15.78:0/529675984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7ceda0 msgr2=0x55affd7d7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1ddce8b640 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7ceda0 0x55affd7d7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1ddce8b640 1 -- 172.21.15.78:0/529675984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55affd6fcc40 con 0x55affd7ce480 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1ddce8b640 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ce480 0x55affd7da620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00e9c0 tx=0x7f1dcc0a9050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea518111f09f47b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:28.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.296+0000 7f1db7fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:22:28.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.297+0000 7f1db7fff640 1 -- 172.21.15.78:0/529675984 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0ac040 con 0x55affd7ce480 2024-09-11T01:22:28.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.297+0000 7f1db7fff640 1 -- 172.21.15.78:0/529675984 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7f1dcc0abb30 con 0x55affd7ce480 2024-09-11T01:22:28.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.297+0000 7f1de14c4740 1 -- 172.21.15.78:0/529675984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dc4032da0 con 0x55affd7ce480 2024-09-11T01:22:28.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.297+0000 7f1db7fff640 1 -- 172.21.15.78:0/529675984 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1dcc0ac3e0 con 0x55affd7ce480 2024-09-11T01:22:28.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.297+0000 7f1de14c4740 10 client.28472 fetch_fsmap learned FSMap version 76 2024-09-11T01:22:28.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.297+0000 7f1de14c4740 1 -- 172.21.15.78:0/529675984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55affd7e91f0 con 0x55affd7ce480 2024-09-11T01:22:28.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.297+0000 7f1db7fff640 1 -- 172.21.15.78:0/529675984 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc4032da0 con 0x55affd7ce480 2024-09-11T01:22:28.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.298+0000 7f1db7fff640 1 -- 172.21.15.78:0/529675984 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f1dcc0e5070 con 0x55affd7ce480 2024-09-11T01:22:28.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.298+0000 7f1de14c4740 10 client.28472 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:22:28.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.298+0000 7f1de14c4740 10 client.28472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:22:28.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.298+0000 7f1de14c4740 20 client.28472 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:28.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.298+0000 7f1de14c4740 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55affd7ea750 0x55affd7ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.298+0000 7f1de14c4740 4 client.28472 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:22:28.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.298+0000 7f1de14c4740 1 -- 172.21.15.78:0/529675984 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55affd746da0 con 0x55affd7ea750 2024-09-11T01:22:28.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.299+0000 7f1dd7fff640 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55affd7ea750 0x55affd7ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.299+0000 7f1dd7fff640 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55affd7ea750 0x55affd7ecb40 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f0f317aa0b85b32 server_cookie=8e95fbdf77408ae8 in_seq=0 out_seq=0 2024-09-11T01:22:28.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.299+0000 7f1db7fff640 10 client.28472 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:22:28.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.300+0000 7f1db7fff640 1 -- 172.21.15.78:0/529675984 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55affd746da0 con 0x55affd7ea750 2024-09-11T01:22:28.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.300+0000 7f1db7fff640 10 client.28472 handle_command_reply: tid=0 2024-09-11T01:22:28.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.300+0000 7f1de14c4740 1 client.28472 shutdown 2024-09-11T01:22:28.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.301+0000 7f1de14c4740 1 -- 172.21.15.78:0/529675984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ce480 msgr2=0x55affd7da620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:28.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.301+0000 7f1de14c4740 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ce480 0x55affd7da620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00e9c0 tx=0x7f1dcc0a9050 comp rx=0 tx=0).stop 2024-09-11T01:22:28.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.301+0000 7f1de14c4740 1 -- 172.21.15.78:0/529675984 shutdown_connections 2024-09-11T01:22:28.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.301+0000 7f1de14c4740 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7dad30 0x55affd7d80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.301+0000 7f1de14c4740 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55affd7ea750 0x55affd7ecb40 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.301+0000 7f1de14c4740 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7ceda0 0x55affd7d7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.301+0000 7f1de14c4740 1 --2- 172.21.15.78:0/529675984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ce480 0x55affd7da620 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.301+0000 7f1de14c4740 1 -- 172.21.15.78:0/529675984 >> 172.21.15.78:0/529675984 conn(0x55affd7cc2b0 msgr2=0x55affd7d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:28.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.301+0000 7f1db7fff640 0 client.28472 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:22:28.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.302+0000 7f1de14c4740 1 -- 172.21.15.78:0/529675984 shutdown_connections 2024-09-11T01:22:28.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.302+0000 7f1de14c4740 1 -- 172.21.15.78:0/529675984 wait complete. 2024-09-11T01:22:28.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.302+0000 7f1de14c4740 20 client.28472 trim_cache size 0 max 16384 2024-09-11T01:22:28.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.323+0000 7f1de14c4740 1 Processor -- start 2024-09-11T01:22:28.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.323+0000 7f1de14c4740 1 -- start start 2024-09-11T01:22:28.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.324+0000 7f1de14c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ce480 0x55affd7e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.324+0000 7f1de14c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7ceda0 0x55affd7ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.324+0000 7f1de14c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7dad30 0x55affd7dc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.324+0000 7f1de14c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81b4b80 con 0x55affd7dad30 2024-09-11T01:22:28.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.324+0000 7f1de14c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81b6780 con 0x55affd7ce480 2024-09-11T01:22:28.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.324+0000 7f1de14c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81b5250 con 0x55affd7ceda0 2024-09-11T01:22:28.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.324+0000 7f1ddd68c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7dad30 0x55affd7dc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.324+0000 7f1ddd68c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7dad30 0x55affd7dc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40672/0 (socket says 172.21.15.78:40672) 2024-09-11T01:22:28.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.324+0000 7f1ddd68c640 1 -- 172.21.15.78:0/2469793489 learned_addr learned my addr 172.21.15.78:0/2469793489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.325+0000 7f1ddd68c640 1 -- 172.21.15.78:0/2469793489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7ceda0 msgr2=0x55affd7ec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.325+0000 7f1ddd68c640 1 --2- 172.21.15.78:0/2469793489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7ceda0 0x55affd7ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.325+0000 7f1ddd68c640 1 -- 172.21.15.78:0/2469793489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ce480 msgr2=0x55affd7e6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.325+0000 7f1ddd68c640 1 --2- 172.21.15.78:0/2469793489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ce480 0x55affd7e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.325+0000 7f1ddd68c640 1 -- 172.21.15.78:0/2469793489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55affd7d7820 con 0x55affd7dad30 2024-09-11T01:22:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.326+0000 7f1ddd68c640 1 --2- 172.21.15.78:0/2469793489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7dad30 0x55affd7dc120 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0003c30 tx=0x7f1dd0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0ab175e7e97e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.326+0000 7f1dc27fc640 1 -- 172.21.15.78:0/2469793489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0003d20 con 0x55affd7dad30 2024-09-11T01:22:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.327+0000 7f1dc27fc640 1 -- 172.21.15.78:0/2469793489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd000fa10 con 0x55affd7dad30 2024-09-11T01:22:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.327+0000 7f1dc27fc640 1 -- 172.21.15.78:0/2469793489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd001d930 con 0x55affd7dad30 2024-09-11T01:22:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.328+0000 7f1de14c4740 1 -- 172.21.15.78:0/2469793489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7dad30 msgr2=0x55affd7dc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.328+0000 7f1de14c4740 1 --2- 172.21.15.78:0/2469793489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7dad30 0x55affd7dc120 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0003c30 tx=0x7f1dd0016000 comp rx=0 tx=0).stop 2024-09-11T01:22:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.328+0000 7f1de14c4740 1 -- 172.21.15.78:0/2469793489 shutdown_connections 2024-09-11T01:22:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.328+0000 7f1de14c4740 1 --2- 172.21.15.78:0/2469793489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7dad30 0x55affd7dc120 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.328+0000 7f1de14c4740 1 --2- 172.21.15.78:0/2469793489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7ceda0 0x55affd7ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.328+0000 7f1de14c4740 1 --2- 172.21.15.78:0/2469793489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ce480 0x55affd7e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.328+0000 7f1de14c4740 1 -- 172.21.15.78:0/2469793489 >> 172.21.15.78:0/2469793489 conn(0x55affd7cc2b0 msgr2=0x55affd7ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:28.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.329+0000 7f1de14c4740 1 -- 172.21.15.78:0/2469793489 shutdown_connections 2024-09-11T01:22:28.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.329+0000 7f1de14c4740 1 -- 172.21.15.78:0/2469793489 wait complete. 2024-09-11T01:22:28.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.329+0000 7f1de14c4740 1 -- start start 2024-09-11T01:22:28.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.329+0000 7f1de14c4740 1 Processor -- start 2024-09-11T01:22:28.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.330+0000 7f1de14c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7ce480 0x55affd7e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.330+0000 7f1de14c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ceda0 0x55affd7e3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.330+0000 7f1de14c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7dad30 0x55affd7ddba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.330+0000 7f1ddce8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7ce480 0x55affd7e2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.330+0000 7f1ddce8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7ce480 0x55affd7e2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52860/0 (socket says 172.21.15.78:52860) 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.330+0000 7f1ddd68c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7dad30 0x55affd7ddba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.330+0000 7f1ddce8b640 1 -- 172.21.15.78:0/1550060143 learned_addr learned my addr 172.21.15.78:0/1550060143 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.330+0000 7f1de14c4740 1 -- 172.21.15.78:0/1550060143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81b60b0 con 0x55affd7dad30 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.330+0000 7f1de14c4740 1 -- 172.21.15.78:0/1550060143 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81b59e0 con 0x55affd7ceda0 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.330+0000 7f1de14c4740 1 -- 172.21.15.78:0/1550060143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81b2bc0 con 0x55affd7ce480 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.331+0000 7f1de14c4740 10 client.0 _ll_register_callbacks cb 0x7f1dcc00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.331+0000 7f1dd7fff640 1 --2- 172.21.15.78:0/1550060143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ceda0 0x55affd7e3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.331+0000 7f1ddce8b640 1 -- 172.21.15.78:0/1550060143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ceda0 msgr2=0x55affd7e3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.331+0000 7f1ddce8b640 1 --2- 172.21.15.78:0/1550060143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ceda0 0x55affd7e3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.331+0000 7f1ddce8b640 1 -- 172.21.15.78:0/1550060143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7dad30 msgr2=0x55affd7ddba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.331+0000 7f1ddce8b640 1 --2- 172.21.15.78:0/1550060143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7dad30 0x55affd7ddba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.331+0000 7f1ddce8b640 1 -- 172.21.15.78:0/1550060143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55affd7e3720 con 0x55affd7ce480 2024-09-11T01:22:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.331+0000 7f1ddce8b640 1 --2- 172.21.15.78:0/1550060143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7ce480 0x55affd7e2bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0ac860 tx=0x7f1dcc0abd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3763de601ddef78 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.331+0000 7f1db7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.331+0000 7f1db7fff640 1 -- 172.21.15.78:0/1550060143 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0ac040 con 0x55affd7ce480 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.331+0000 7f1de14c4740 1 -- 172.21.15.78:0/1550060143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dd000fa10 con 0x55affd7ce480 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.331+0000 7f1db7fff640 1 -- 172.21.15.78:0/1550060143 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7f1dcc0b2540 con 0x55affd7ce480 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.332+0000 7f1db7fff640 1 -- 172.21.15.78:0/1550060143 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1dcc0b1180 con 0x55affd7ce480 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.332+0000 7f1de14c4740 10 client.28432 fetch_fsmap learned FSMap version 76 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.332+0000 7f1de14c4740 1 -- 172.21.15.78:0/1550060143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55affd857a50 con 0x55affd7ce480 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.332+0000 7f1db7fff640 1 -- 172.21.15.78:0/1550060143 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd000fa10 con 0x55affd7ce480 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.332+0000 7f1db7fff640 1 -- 172.21.15.78:0/1550060143 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f1dcc0e5030 con 0x55affd7ce480 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.332+0000 7f1de14c4740 10 client.28432 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.332+0000 7f1de14c4740 10 client.28432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.332+0000 7f1de14c4740 20 client.28432 populate_metadata read hostname 'smithi078' 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.332+0000 7f1de14c4740 1 --2- 172.21.15.78:0/1550060143 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55affd8581a0 0x55affd85a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:22:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.333+0000 7f1de14c4740 4 client.28432 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:22:28.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.333+0000 7f1de14c4740 1 -- 172.21.15.78:0/1550060143 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55affd742850 con 0x55affd8581a0 2024-09-11T01:22:28.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.333+0000 7f1dd7fff640 1 --2- 172.21.15.78:0/1550060143 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55affd8581a0 0x55affd85a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:22:28.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.333+0000 7f1db7fff640 10 client.28432 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:22:28.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.333+0000 7f1dd7fff640 1 --2- 172.21.15.78:0/1550060143 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55affd8581a0 0x55affd85a5c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64c49cedbd55d15 server_cookie=3d611f4df118b8ce in_seq=0 out_seq=0 2024-09-11T01:22:28.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.334+0000 7f1db7fff640 1 -- 172.21.15.78:0/1550060143 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8238 (crc 0 0 0) 0x55affd742850 con 0x55affd8581a0 2024-09-11T01:22:28.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.334+0000 7f1db7fff640 10 client.28432 handle_command_reply: tid=0 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.334+0000 7f1de14c4740 1 client.28432 shutdown 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.334+0000 7f1de14c4740 1 -- 172.21.15.78:0/1550060143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7ce480 msgr2=0x55affd7e2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.334+0000 7f1de14c4740 1 --2- 172.21.15.78:0/1550060143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7ce480 0x55affd7e2bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0ac860 tx=0x7f1dcc0abd60 comp rx=0 tx=0).stop 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.334+0000 7f1de14c4740 1 -- 172.21.15.78:0/1550060143 shutdown_connections 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.334+0000 7f1de14c4740 1 --2- 172.21.15.78:0/1550060143 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55affd8581a0 0x55affd85a5c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.334+0000 7f1de14c4740 1 --2- 172.21.15.78:0/1550060143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55affd7dad30 0x55affd7ddba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.334+0000 7f1de14c4740 1 --2- 172.21.15.78:0/1550060143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55affd7ceda0 0x55affd7e3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.334+0000 7f1de14c4740 1 --2- 172.21.15.78:0/1550060143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55affd7ce480 0x55affd7e2bf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.334+0000 7f1de14c4740 1 -- 172.21.15.78:0/1550060143 >> 172.21.15.78:0/1550060143 conn(0x55affd7cc2b0 msgr2=0x55affd7d90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.334+0000 7f1db7fff640 0 client.28432 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.335+0000 7f1de14c4740 1 -- 172.21.15.78:0/1550060143 shutdown_connections 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.335+0000 7f1de14c4740 1 -- 172.21.15.78:0/1550060143 wait complete. 2024-09-11T01:22:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.335+0000 7f1de14c4740 20 client.28432 trim_cache size 0 max 16384 2024-09-11T01:22:28.340 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:22:28.340 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:22:28.340 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:22:28.340 INFO:teuthology.orchestra.run.smithi078.stdout: "82736ae4-b55a-4ba3-911c-0aa1c18acfde": { 2024-09-11T01:22:28.340 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.340 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "82736ae4-b55a-4ba3-911c-0aa1c18acfde", 2024-09-11T01:22:28.340 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:22:28.340 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:22:28.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.340 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: "82736ae4-b55a-4ba3-911c-0aa1c18acfde": { 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:22:21.932379+0000", 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: "746add9a-5be3-4745-8000-04e5471bc813": { 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:20:12.369593+0000", 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.341 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.342 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.342 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.342 INFO:teuthology.orchestra.run.smithi078.stdout: "b41a2b77-b2b1-4f1b-8588-4f32a78b721f": { 2024-09-11T01:22:28.342 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:19:13.257610+0000", 2024-09-11T01:22:28.342 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.342 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.342 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.342 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.342 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.342 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:18:17.654570+0000", 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.343 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.344 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.345 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:22:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:22:28.349 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "291fcaa4-1805-423d-a0bc-814f1bc2759f": { 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:21:17.256483+0000", 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.351 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.352 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:22:28.353 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.354 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:22:28.356 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:22:28.357 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:22:28.357 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:22:28.357 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:22:28.357 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:22:28.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.336+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/2238845006 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1dc80b54d0 msgr2=0x7f1dc80b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:28.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.336+0000 7f1ddf0ef640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1dc80b54d0 0x7f1dc80b7990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0005fd0 tx=0x7f1dd0009500 comp rx=0 tx=0).stop 2024-09-11T01:22:28.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.336+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/2238845006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dd8101c90 msgr2=0x7f1dd819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:22:28.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.336+0000 7f1ddf0ef640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dd8101c90 0x7f1dd819ba30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00e9f0 tx=0x7f1dcc00edf0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.336+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/2238845006 shutdown_connections 2024-09-11T01:22:28.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.336+0000 7f1ddf0ef640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f1dc80b54d0 0x7f1dc80b7990 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.336+0000 7f1ddf0ef640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd8108020 0x7f1dd819dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.336+0000 7f1ddf0ef640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1dd81076a0 0x7f1dd819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.336+0000 7f1ddf0ef640 1 --2- 172.21.15.78:0/2238845006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dd8101c90 0x7f1dd819ba30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:22:28.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.336+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/2238845006 >> 172.21.15.78:0/2238845006 conn(0x7f1dd80fd700 msgr2=0x7f1dd80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:22:28.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.336+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/2238845006 shutdown_connections 2024-09-11T01:22:28.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:22:28.336+0000 7f1ddf0ef640 1 -- 172.21.15.78:0/2238845006 wait complete. 2024-09-11T01:22:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:28 smithi078 ceph-mon[31315]: pgmap v1064: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 719 op/s 2024-09-11T01:22:28.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:28 smithi110 ceph-mon[34712]: pgmap v1064: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 719 op/s 2024-09-11T01:22:28.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:28 smithi112 ceph-mon[35384]: pgmap v1064: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 719 op/s 2024-09-11T01:22:28.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 383236 31.78 MB/sec execute 514 sec latency 83.353 ms 2024-09-11T01:22:29.467 INFO:tasks.cephfs.filesystem:scrub status for tag:82736ae4-b55a-4ba3-911c-0aa1c18acfde - {'path': '/', 'tag': '82736ae4-b55a-4ba3-911c-0aa1c18acfde', 'options': 'recursive,force'} 2024-09-11T01:22:29.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 383975 31.80 MB/sec execute 515 sec latency 106.826 ms 2024-09-11T01:22:30.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:30 smithi110 ceph-mon[34712]: pgmap v1065: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 727 op/s 2024-09-11T01:22:30.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:30 smithi112 ceph-mon[35384]: pgmap v1065: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 727 op/s 2024-09-11T01:22:30.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:30 smithi078 ceph-mon[31315]: pgmap v1065: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 727 op/s 2024-09-11T01:22:30.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 384713 31.82 MB/sec execute 516 sec latency 56.261 ms 2024-09-11T01:22:31.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 385150 31.80 MB/sec execute 517 sec latency 273.377 ms 2024-09-11T01:22:32.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:32 smithi110 ceph-mon[34712]: pgmap v1066: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 741 op/s 2024-09-11T01:22:32.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:32 smithi112 ceph-mon[35384]: pgmap v1066: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 741 op/s 2024-09-11T01:22:32.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:32 smithi078 ceph-mon[31315]: pgmap v1066: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 741 op/s 2024-09-11T01:22:32.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 385643 31.80 MB/sec execute 518 sec latency 244.771 ms 2024-09-11T01:22:33.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:22:33.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:22:33.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:22:33.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:22:33.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:33 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:22:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:22:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:22:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:22:33.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:22:33.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:33 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:22:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:22:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:22:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:22:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:22:33.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:33 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:22:33.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 386255 31.78 MB/sec execute 519 sec latency 128.011 ms 2024-09-11T01:22:34.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:34 smithi110 ceph-mon[34712]: pgmap v1067: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 710 op/s 2024-09-11T01:22:34.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:34 smithi112 ceph-mon[35384]: pgmap v1067: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 710 op/s 2024-09-11T01:22:34.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:34 smithi078 ceph-mon[31315]: pgmap v1067: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 710 op/s 2024-09-11T01:22:34.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 387018 31.79 MB/sec execute 520 sec latency 89.565 ms 2024-09-11T01:22:35.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 387838 31.85 MB/sec execute 521 sec latency 63.154 ms 2024-09-11T01:22:36.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:36 smithi110 ceph-mon[34712]: pgmap v1068: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 664 op/s 2024-09-11T01:22:36.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:36 smithi112 ceph-mon[35384]: pgmap v1068: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 664 op/s 2024-09-11T01:22:36.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:36 smithi078 ceph-mon[31315]: pgmap v1068: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 664 op/s 2024-09-11T01:22:36.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 388322 31.84 MB/sec execute 522 sec latency 175.780 ms 2024-09-11T01:22:37.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 388824 31.81 MB/sec execute 523 sec latency 176.169 ms 2024-09-11T01:22:38.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:38 smithi110 ceph-mon[34712]: pgmap v1069: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 633 op/s 2024-09-11T01:22:38.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:38 smithi112 ceph-mon[35384]: pgmap v1069: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 633 op/s 2024-09-11T01:22:38.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:38 smithi078 ceph-mon[31315]: pgmap v1069: 97 pgs: 97 active+clean; 3.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 633 op/s 2024-09-11T01:22:38.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 388958 31.77 MB/sec execute 524 sec latency 786.685 ms 2024-09-11T01:22:39.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 389204 31.74 MB/sec execute 525 sec latency 1035.869 ms 2024-09-11T01:22:40.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:40 smithi110 ceph-mon[34712]: pgmap v1070: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 462 op/s 2024-09-11T01:22:40.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:40 smithi112 ceph-mon[35384]: pgmap v1070: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 462 op/s 2024-09-11T01:22:40.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:40 smithi078 ceph-mon[31315]: pgmap v1070: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 462 op/s 2024-09-11T01:22:40.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 389633 31.72 MB/sec execute 526 sec latency 200.050 ms 2024-09-11T01:22:41.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 390361 31.76 MB/sec execute 527 sec latency 183.986 ms 2024-09-11T01:22:42.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:42 smithi110 ceph-mon[34712]: pgmap v1071: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 645 op/s 2024-09-11T01:22:42.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:42 smithi112 ceph-mon[35384]: pgmap v1071: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 645 op/s 2024-09-11T01:22:42.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:42 smithi078 ceph-mon[31315]: pgmap v1071: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 645 op/s 2024-09-11T01:22:42.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 391102 31.77 MB/sec execute 528 sec latency 55.170 ms 2024-09-11T01:22:43.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 391715 31.79 MB/sec execute 529 sec latency 100.515 ms 2024-09-11T01:22:44.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:44 smithi110 ceph-mon[34712]: pgmap v1072: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 684 op/s 2024-09-11T01:22:44.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:44 smithi112 ceph-mon[35384]: pgmap v1072: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 684 op/s 2024-09-11T01:22:44.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:44 smithi078 ceph-mon[31315]: pgmap v1072: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 684 op/s 2024-09-11T01:22:44.807 INFO:tasks.workunit.client.0.smithi078.stdout: 10 392251 31.78 MB/sec execute 530 sec latency 83.183 ms 2024-09-11T01:22:45.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 392794 31.77 MB/sec execute 531 sec latency 140.581 ms 2024-09-11T01:22:46.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:46 smithi110 ceph-mon[34712]: pgmap v1073: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 654 op/s 2024-09-11T01:22:46.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:46 smithi112 ceph-mon[35384]: pgmap v1073: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 654 op/s 2024-09-11T01:22:46.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:46 smithi078 ceph-mon[31315]: pgmap v1073: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 654 op/s 2024-09-11T01:22:46.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 393392 31.76 MB/sec execute 532 sec latency 185.009 ms 2024-09-11T01:22:47.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:47 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:22:47.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:47 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:22:47.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:47 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:22:47.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 394074 31.78 MB/sec execute 533 sec latency 101.145 ms 2024-09-11T01:22:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:48 smithi110 ceph-mon[34712]: pgmap v1074: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 693 op/s 2024-09-11T01:22:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:48 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:22:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:48 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:22:48.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:48 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:22:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:48 smithi112 ceph-mon[35384]: pgmap v1074: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 693 op/s 2024-09-11T01:22:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:48 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:22:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:48 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:22:48.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:48 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:22:48.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:48 smithi078 ceph-mon[31315]: pgmap v1074: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 693 op/s 2024-09-11T01:22:48.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:48 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:22:48.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:48 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:22:48.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:48 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:22:48.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 394771 31.79 MB/sec execute 534 sec latency 79.155 ms 2024-09-11T01:22:49.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 395318 31.79 MB/sec execute 535 sec latency 140.457 ms 2024-09-11T01:22:50.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:50 smithi110 ceph-mon[34712]: pgmap v1075: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 626 op/s 2024-09-11T01:22:50.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:50 smithi112 ceph-mon[35384]: pgmap v1075: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 626 op/s 2024-09-11T01:22:50.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:50 smithi078 ceph-mon[31315]: pgmap v1075: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 626 op/s 2024-09-11T01:22:50.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 395915 31.80 MB/sec execute 536 sec latency 72.481 ms 2024-09-11T01:22:51.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 396441 31.77 MB/sec execute 537 sec latency 154.015 ms 2024-09-11T01:22:52.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:52 smithi110 ceph-mon[34712]: pgmap v1076: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 19 MiB/s wr, 748 op/s 2024-09-11T01:22:52.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:52 smithi112 ceph-mon[35384]: pgmap v1076: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 19 MiB/s wr, 748 op/s 2024-09-11T01:22:52.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:52 smithi078 ceph-mon[31315]: pgmap v1076: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 19 MiB/s wr, 748 op/s 2024-09-11T01:22:52.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 397120 31.79 MB/sec execute 538 sec latency 90.561 ms 2024-09-11T01:22:53.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 397736 31.80 MB/sec execute 539 sec latency 163.922 ms 2024-09-11T01:22:54.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:54 smithi110 ceph-mon[34712]: pgmap v1077: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 576 op/s 2024-09-11T01:22:54.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:54 smithi112 ceph-mon[35384]: pgmap v1077: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 576 op/s 2024-09-11T01:22:54.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:54 smithi078 ceph-mon[31315]: pgmap v1077: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 576 op/s 2024-09-11T01:22:54.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 398389 31.78 MB/sec execute 540 sec latency 98.920 ms 2024-09-11T01:22:55.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 399110 31.81 MB/sec execute 541 sec latency 66.608 ms 2024-09-11T01:22:56.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:56 smithi110 ceph-mon[34712]: pgmap v1078: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 560 op/s 2024-09-11T01:22:56.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:56 smithi112 ceph-mon[35384]: pgmap v1078: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 560 op/s 2024-09-11T01:22:56.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:56 smithi078 ceph-mon[31315]: pgmap v1078: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 560 op/s 2024-09-11T01:22:56.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 399791 31.83 MB/sec execute 542 sec latency 120.033 ms 2024-09-11T01:22:57.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 400346 31.82 MB/sec execute 543 sec latency 61.603 ms 2024-09-11T01:22:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:22:58 smithi110 ceph-mon[34712]: pgmap v1079: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 552 op/s 2024-09-11T01:22:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:22:58 smithi112 ceph-mon[35384]: pgmap v1079: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 552 op/s 2024-09-11T01:22:58.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:22:58 smithi078 ceph-mon[31315]: pgmap v1079: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 552 op/s 2024-09-11T01:22:58.808 INFO:tasks.workunit.client.0.smithi078.stdout: 10 400906 31.81 MB/sec execute 544 sec latency 236.520 ms 2024-09-11T01:22:59.468 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:22:59.810 INFO:tasks.workunit.client.0.smithi078.stdout: 10 400962 31.76 MB/sec execute 545 sec latency 1071.867 ms 2024-09-11T01:22:59.976 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:23:00.474 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:00 smithi078 ceph-mon[31315]: pgmap v1080: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 478 op/s 2024-09-11T01:23:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.625+0000 7f27dbc25640 1 -- 172.21.15.78:0/44536171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d411c0a0 msgr2=0x7f27d411e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.625+0000 7f27dbc25640 1 --2- 172.21.15.78:0/44536171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d411c0a0 0x7f27d411e490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009860 tx=0x7f27d002a410 comp rx=0 tx=0).stop 2024-09-11T01:23:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.626+0000 7f27dbc25640 1 -- 172.21.15.78:0/44536171 shutdown_connections 2024-09-11T01:23:00.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.626+0000 7f27dbc25640 1 --2- 172.21.15.78:0/44536171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d411c0a0 0x7f27d411e490 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.626+0000 7f27dbc25640 1 --2- 172.21.15.78:0/44536171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d410a530 0x7f27d410a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.626+0000 7f27dbc25640 1 --2- 172.21.15.78:0/44536171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d410a150 0x7f27d41111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.626+0000 7f27dbc25640 1 -- 172.21.15.78:0/44536171 >> 172.21.15.78:0/44536171 conn(0x7f27d406dfc0 msgr2=0x7f27d406e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:00.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.632+0000 7f27dbc25640 1 -- 172.21.15.78:0/44536171 shutdown_connections 2024-09-11T01:23:00.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.634+0000 7f27dbc25640 1 -- 172.21.15.78:0/44536171 wait complete. 2024-09-11T01:23:00.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.635+0000 7f27dbc25640 1 Processor -- start 2024-09-11T01:23:00.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.635+0000 7f27dbc25640 1 -- start start 2024-09-11T01:23:00.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.635+0000 7f27dbc25640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d410a150 0x7f27d4117940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.635+0000 7f27dbc25640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d410a530 0x7f27d4117e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.635+0000 7f27dbc25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d411c0a0 0x7f27d41a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.635+0000 7f27dbc25640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27d4114120 con 0x7f27d410a150 2024-09-11T01:23:00.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.635+0000 7f27dbc25640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27d4113fa0 con 0x7f27d410a530 2024-09-11T01:23:00.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.635+0000 7f27dbc25640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27d41142a0 con 0x7f27d411c0a0 2024-09-11T01:23:00.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.636+0000 7f27da1c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d411c0a0 0x7f27d41a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:00.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.636+0000 7f27da1c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d411c0a0 0x7f27d41a51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35032/0 (socket says 172.21.15.78:35032) 2024-09-11T01:23:00.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.636+0000 7f27da1c2640 1 -- 172.21.15.78:0/2704355340 learned_addr learned my addr 172.21.15.78:0/2704355340 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.638+0000 7f27da1c2640 1 -- 172.21.15.78:0/2704355340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d410a530 msgr2=0x7f27d4117e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T01:23:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.638+0000 7f27da1c2640 1 --2- 172.21.15.78:0/2704355340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d410a530 0x7f27d4117e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.638+0000 7f27da1c2640 1 -- 172.21.15.78:0/2704355340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d410a150 msgr2=0x7f27d4117940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.638+0000 7f27da1c2640 1 --2- 172.21.15.78:0/2704355340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d410a150 0x7f27d4117940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.638+0000 7f27da1c2640 1 -- 172.21.15.78:0/2704355340 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d41a5980 con 0x7f27d411c0a0 2024-09-11T01:23:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.639+0000 7f27da1c2640 1 --2- 172.21.15.78:0/2704355340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d411c0a0 0x7f27d41a51b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f27d002a980 tx=0x7f27d0030ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:00.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.643+0000 7f27caffd640 1 -- 172.21.15.78:0/2704355340 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0040070 con 0x7f27d411c0a0 2024-09-11T01:23:00.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.643+0000 7f27caffd640 1 -- 172.21.15.78:0/2704355340 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d0005570 con 0x7f27d411c0a0 2024-09-11T01:23:00.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.643+0000 7f27caffd640 1 -- 172.21.15.78:0/2704355340 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d00046c0 con 0x7f27d411c0a0 2024-09-11T01:23:00.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.643+0000 7f27dbc25640 1 -- 172.21.15.78:0/2704355340 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d41a5c10 con 0x7f27d411c0a0 2024-09-11T01:23:00.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.643+0000 7f27dbc25640 1 -- 172.21.15.78:0/2704355340 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d41ac330 con 0x7f27d411c0a0 2024-09-11T01:23:00.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.647+0000 7f27ddffa740 1 Processor -- start 2024-09-11T01:23:00.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.647+0000 7f27ddffa740 1 -- start start 2024-09-11T01:23:00.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.647+0000 7f27ddffa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3480 0x559fcd8b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.647+0000 7f27ddffa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8b3da0 0x559fcd8bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.647+0000 7f27ddffa740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8bfd30 0x559fcd8c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.647+0000 7f27ddffa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559fcd829e50 con 0x559fcd8b3480 2024-09-11T01:23:00.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.647+0000 7f27ddffa740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559fcd819280 con 0x559fcd8bfd30 2024-09-11T01:23:00.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.647+0000 7f27ddffa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559fcd81d090 con 0x559fcd8b3da0 2024-09-11T01:23:00.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.647+0000 7f27d91c0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3480 0x559fcd8b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:00.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.648+0000 7f27d91c0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3480 0x559fcd8b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57772/0 (socket says 172.21.15.78:57772) 2024-09-11T01:23:00.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.648+0000 7f27d91c0640 1 -- 172.21.15.78:0/4258818007 learned_addr learned my addr 172.21.15.78:0/4258818007 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:00.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.649+0000 7f27d91c0640 1 --2- 172.21.15.78:0/4258818007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8bfd30 0x559fcd8c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:00.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.652+0000 7f27d91c0640 1 -- 172.21.15.78:0/4258818007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8b3da0 msgr2=0x559fcd8bf7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:23:00.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.652+0000 7f27d91c0640 1 --2- 172.21.15.78:0/4258818007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8b3da0 0x559fcd8bf7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.652+0000 7f27d91c0640 1 -- 172.21.15.78:0/4258818007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3480 msgr2=0x559fcd8b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:00.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.652+0000 7f27d91c0640 1 --2- 172.21.15.78:0/4258818007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3480 0x559fcd8b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.652+0000 7f27d91c0640 1 -- 172.21.15.78:0/4258818007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcd577650 con 0x559fcd8bfd30 2024-09-11T01:23:00.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.653+0000 7f27caffd640 1 -- 172.21.15.78:0/2704355340 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f27d0031400 con 0x7f27d411c0a0 2024-09-11T01:23:00.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.655+0000 7f27d91c0640 1 --2- 172.21.15.78:0/4258818007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8bfd30 0x559fcd8c2120 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0346a0 tx=0x7f27cc004b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb294baa3baaebf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:00.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.660+0000 7f27caffd640 1 --2- 172.21.15.78:0/2704355340 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27c00b5650 0x7f27c00b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.660+0000 7f27bffff640 1 -- 172.21.15.78:0/4258818007 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc03f070 con 0x559fcd8bfd30 2024-09-11T01:23:00.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.660+0000 7f27bffff640 1 -- 172.21.15.78:0/4258818007 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27cc02d0f0 con 0x559fcd8bfd30 2024-09-11T01:23:00.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.660+0000 7f27caffd640 1 -- 172.21.15.78:0/2704355340 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7f27d0106c10 con 0x7f27d411c0a0 2024-09-11T01:23:00.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.661+0000 7f27ddffa740 1 -- 172.21.15.78:0/4258818007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8bfd30 msgr2=0x559fcd8c2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:00.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.661+0000 7f27ddffa740 1 --2- 172.21.15.78:0/4258818007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8bfd30 0x559fcd8c2120 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0346a0 tx=0x7f27cc004b60 comp rx=0 tx=0).stop 2024-09-11T01:23:00.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.664+0000 7f27d99c1640 1 --2- 172.21.15.78:0/2704355340 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27c00b5650 0x7f27c00b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:00.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.665+0000 7f27d99c1640 1 --2- 172.21.15.78:0/2704355340 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27c00b5650 0x7f27c00b7b10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f27c4005ab0 tx=0x7f27c4005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:00.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.666+0000 7f27ddffa740 1 -- 172.21.15.78:0/4258818007 shutdown_connections 2024-09-11T01:23:00.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.666+0000 7f27ddffa740 1 --2- 172.21.15.78:0/4258818007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8bfd30 0x559fcd8c2120 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.666+0000 7f27ddffa740 1 --2- 172.21.15.78:0/4258818007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8b3da0 0x559fcd8bf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.666+0000 7f27ddffa740 1 --2- 172.21.15.78:0/4258818007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3480 0x559fcd8b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.666+0000 7f27ddffa740 1 -- 172.21.15.78:0/4258818007 >> 172.21.15.78:0/4258818007 conn(0x559fcd8b12b0 msgr2=0x559fcd8b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.669+0000 7f27ddffa740 1 -- 172.21.15.78:0/4258818007 shutdown_connections 2024-09-11T01:23:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.669+0000 7f27ddffa740 1 -- 172.21.15.78:0/4258818007 wait complete. 2024-09-11T01:23:00.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.669+0000 7f27ddffa740 1 -- start start 2024-09-11T01:23:00.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.670+0000 7f27ddffa740 1 Processor -- start 2024-09-11T01:23:00.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.673+0000 7f27ddffa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3480 0x559fcd8bf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.673+0000 7f27ddffa740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8b3da0 0x559fcd8bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.673+0000 7f27ddffa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8bfd30 0x559fcd8bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.673+0000 7f27ddffa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559fcd67a700 con 0x559fcd8b3480 2024-09-11T01:23:00.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.673+0000 7f27ddffa740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559fcd7b4110 con 0x559fcd8b3da0 2024-09-11T01:23:00.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.673+0000 7f27ddffa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559fcd8c2a00 con 0x559fcd8bfd30 2024-09-11T01:23:00.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.673+0000 7f27ddffa740 10 client.0 _ll_register_callbacks cb 0x559fcd7cd990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.675+0000 7f27d91c0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8b3da0 0x559fcd8bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.675+0000 7f27da1c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8bfd30 0x559fcd8bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.675+0000 7f27d91c0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8b3da0 0x559fcd8bca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:53912/0 (socket says 172.21.15.78:53912) 2024-09-11T01:23:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.675+0000 7f27d91c0640 1 -- 172.21.15.78:0/2753031976 learned_addr learned my addr 172.21.15.78:0/2753031976 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.675+0000 7f27d91c0640 1 -- 172.21.15.78:0/2753031976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8bfd30 msgr2=0x559fcd8bcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.675+0000 7f27d91c0640 1 --2- 172.21.15.78:0/2753031976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8bfd30 0x559fcd8bcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.675+0000 7f27d91c0640 1 -- 172.21.15.78:0/2753031976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3480 msgr2=0x559fcd8bf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:00.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.675+0000 7f27d91c0640 1 --2- 172.21.15.78:0/2753031976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3480 0x559fcd8bf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.675+0000 7f27d91c0640 1 -- 172.21.15.78:0/2753031976 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fcd7e1c40 con 0x559fcd8b3da0 2024-09-11T01:23:00.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.676+0000 7f27d91c0640 1 --2- 172.21.15.78:0/2753031976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8b3da0 0x559fcd8bca20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f27cc02bd80 tx=0x7f27cc02b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27c7c7b7baf34da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:00.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.677+0000 7f27bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:23:00.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.677+0000 7f27bd7fa640 1 -- 172.21.15.78:0/2753031976 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc03f030 con 0x559fcd8b3da0 2024-09-11T01:23:00.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.677+0000 7f27ddffa740 1 -- 172.21.15.78:0/2753031976 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27cc02d0f0 con 0x559fcd8b3da0 2024-09-11T01:23:00.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.677+0000 7f27bd7fa640 1 -- 172.21.15.78:0/2753031976 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7f27cc02d2a0 con 0x559fcd8b3da0 2024-09-11T01:23:00.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.679+0000 7f27bd7fa640 1 -- 172.21.15.78:0/2753031976 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f27cc02d7a0 con 0x559fcd8b3da0 2024-09-11T01:23:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.683+0000 7f27ddffa740 10 client.28478 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.683+0000 7f27ddffa740 1 -- 172.21.15.78:0/2753031976 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x559fcd8ce3d0 con 0x559fcd8b3da0 2024-09-11T01:23:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.685+0000 7f27bd7fa640 1 -- 172.21.15.78:0/2753031976 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27cc02d0f0 con 0x559fcd8b3da0 2024-09-11T01:23:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.685+0000 7f27bd7fa640 1 -- 172.21.15.78:0/2753031976 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f27cc039030 con 0x559fcd8b3da0 2024-09-11T01:23:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.685+0000 7f27ddffa740 10 client.28478 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.685+0000 7f27ddffa740 10 client.28478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:23:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.685+0000 7f27ddffa740 20 client.28478 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.685+0000 7f27ddffa740 1 --2- 172.21.15.78:0/2753031976 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559fcd8cf780 0x559fcd8d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.685+0000 7f27ddffa740 4 client.28478 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:23:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.685+0000 7f27ddffa740 1 -- 172.21.15.78:0/2753031976 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fcd82bda0 con 0x559fcd8cf780 2024-09-11T01:23:00.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:00 smithi110 ceph-mon[34712]: pgmap v1080: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 478 op/s 2024-09-11T01:23:00.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.689+0000 7f27d99c1640 1 --2- 172.21.15.78:0/2753031976 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559fcd8cf780 0x559fcd8d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:00.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.694+0000 7f27d99c1640 1 --2- 172.21.15.78:0/2753031976 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559fcd8cf780 0x559fcd8d1b70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40c01fc107ca39ce server_cookie=f836892827f72ff3 in_seq=0 out_seq=0 2024-09-11T01:23:00.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.694+0000 7f27bd7fa640 10 client.28478 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:00.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.695+0000 7f27bd7fa640 1 -- 172.21.15.78:0/2753031976 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559fcd82bda0 con 0x559fcd8cf780 2024-09-11T01:23:00.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.695+0000 7f27bd7fa640 10 client.28478 handle_command_reply: tid=0 2024-09-11T01:23:00.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.695+0000 7f27ddffa740 1 client.28478 shutdown 2024-09-11T01:23:00.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.697+0000 7f27ddffa740 1 -- 172.21.15.78:0/2753031976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8b3da0 msgr2=0x559fcd8bca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:00.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.697+0000 7f27ddffa740 1 --2- 172.21.15.78:0/2753031976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8b3da0 0x559fcd8bca20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f27cc02bd80 tx=0x7f27cc02b990 comp rx=0 tx=0).stop 2024-09-11T01:23:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.700+0000 7f27ddffa740 1 -- 172.21.15.78:0/2753031976 shutdown_connections 2024-09-11T01:23:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.700+0000 7f27ddffa740 1 --2- 172.21.15.78:0/2753031976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8bfd30 0x559fcd8bcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.700+0000 7f27ddffa740 1 --2- 172.21.15.78:0/2753031976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8b3da0 0x559fcd8bca20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.700+0000 7f27ddffa740 1 --2- 172.21.15.78:0/2753031976 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559fcd8cf780 0x559fcd8d1b70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.700+0000 7f27ddffa740 1 --2- 172.21.15.78:0/2753031976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3480 0x559fcd8bf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.700+0000 7f27ddffa740 1 -- 172.21.15.78:0/2753031976 >> 172.21.15.78:0/2753031976 conn(0x559fcd8b12b0 msgr2=0x559fcd8b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:00.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.700+0000 7f27bd7fa640 0 client.28478 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:00.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.703+0000 7f27ddffa740 1 -- 172.21.15.78:0/2753031976 shutdown_connections 2024-09-11T01:23:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.704+0000 7f27ddffa740 1 -- 172.21.15.78:0/2753031976 wait complete. 2024-09-11T01:23:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.705+0000 7f27ddffa740 20 client.28478 trim_cache size 0 max 16384 2024-09-11T01:23:00.725 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:00 smithi112 ceph-mon[35384]: pgmap v1080: 97 pgs: 97 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 478 op/s 2024-09-11T01:23:00.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.739+0000 7f27ddffa740 1 Processor -- start 2024-09-11T01:23:00.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.739+0000 7f27ddffa740 1 -- start start 2024-09-11T01:23:00.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.740+0000 7f27ddffa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8b3480 0x559fcd8cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.740+0000 7f27ddffa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3da0 0x559fcd8b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.740+0000 7f27ddffa740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8bfd30 0x559fcd8b9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.740+0000 7f27ddffa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27b800e810 con 0x559fcd8b3da0 2024-09-11T01:23:00.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.740+0000 7f27ddffa740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27b8010410 con 0x559fcd8bfd30 2024-09-11T01:23:00.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.740+0000 7f27ddffa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27b800eee0 con 0x559fcd8b3480 2024-09-11T01:23:00.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.743+0000 7f27da1c2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8bfd30 0x559fcd8b9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:00.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.743+0000 7f27da1c2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8bfd30 0x559fcd8b9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:53918/0 (socket says 172.21.15.78:53918) 2024-09-11T01:23:00.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.743+0000 7f27da1c2640 1 -- 172.21.15.78:0/1942676315 learned_addr learned my addr 172.21.15.78:0/1942676315 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:00.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.743+0000 7f27d91c0640 1 --2- 172.21.15.78:0/1942676315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3da0 0x559fcd8b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:00.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.743+0000 7f27d91c0640 1 -- 172.21.15.78:0/1942676315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8b3480 msgr2=0x559fcd8cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:00.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.743+0000 7f27d91c0640 1 --2- 172.21.15.78:0/1942676315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8b3480 0x559fcd8cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.743+0000 7f27d91c0640 1 -- 172.21.15.78:0/1942676315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8bfd30 msgr2=0x559fcd8b9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:00.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.743+0000 7f27d91c0640 1 --2- 172.21.15.78:0/1942676315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8bfd30 0x559fcd8b9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.743+0000 7f27d91c0640 1 -- 172.21.15.78:0/1942676315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcd8bdcf0 con 0x559fcd8b3da0 2024-09-11T01:23:00.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.743+0000 7f27d91c0640 1 --2- 172.21.15.78:0/1942676315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3da0 0x559fcd8b2a10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f27cc000f80 tx=0x7f27cc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3116a978c9469da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:00.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.746+0000 7f27bf7fe640 1 -- 172.21.15.78:0/1942676315 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc03f020 con 0x559fcd8b3da0 2024-09-11T01:23:00.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.746+0000 7f27bf7fe640 1 -- 172.21.15.78:0/1942676315 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27cc041070 con 0x559fcd8b3da0 2024-09-11T01:23:00.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.746+0000 7f27bf7fe640 1 -- 172.21.15.78:0/1942676315 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc02d0e0 con 0x559fcd8b3da0 2024-09-11T01:23:00.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.747+0000 7f27ddffa740 1 -- 172.21.15.78:0/1942676315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3da0 msgr2=0x559fcd8b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:00.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.748+0000 7f27ddffa740 1 --2- 172.21.15.78:0/1942676315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3da0 0x559fcd8b2a10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f27cc000f80 tx=0x7f27cc009260 comp rx=0 tx=0).stop 2024-09-11T01:23:00.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.750+0000 7f27ddffa740 1 -- 172.21.15.78:0/1942676315 shutdown_connections 2024-09-11T01:23:00.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.750+0000 7f27ddffa740 1 --2- 172.21.15.78:0/1942676315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8bfd30 0x559fcd8b9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.750+0000 7f27ddffa740 1 --2- 172.21.15.78:0/1942676315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3da0 0x559fcd8b2a10 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.750+0000 7f27ddffa740 1 --2- 172.21.15.78:0/1942676315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8b3480 0x559fcd8cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.750+0000 7f27ddffa740 1 -- 172.21.15.78:0/1942676315 >> 172.21.15.78:0/1942676315 conn(0x559fcd8b12b0 msgr2=0x559fcd8b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:00.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.752+0000 7f27ddffa740 1 -- 172.21.15.78:0/1942676315 shutdown_connections 2024-09-11T01:23:00.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.752+0000 7f27ddffa740 1 -- 172.21.15.78:0/1942676315 wait complete. 2024-09-11T01:23:00.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.752+0000 7f27ddffa740 1 -- start start 2024-09-11T01:23:00.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.752+0000 7f27ddffa740 1 Processor -- start 2024-09-11T01:23:00.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.753+0000 7f27ddffa740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8b3480 0x559fcd8c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.753+0000 7f27ddffa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3da0 0x559fcd8c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.753+0000 7f27ddffa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8bfd30 0x559fcd8c4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.753+0000 7f27ddffa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27b800fd40 con 0x559fcd8b3da0 2024-09-11T01:23:00.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.753+0000 7f27ddffa740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27b800f670 con 0x559fcd8b3480 2024-09-11T01:23:00.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.753+0000 7f27ddffa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27b800c810 con 0x559fcd8bfd30 2024-09-11T01:23:00.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.753+0000 7f27ddffa740 10 client.0 _ll_register_callbacks cb 0x7f27cc03d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:00.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.754+0000 7f27da1c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8bfd30 0x559fcd8c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:00.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.754+0000 7f27d99c1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8b3480 0x559fcd8c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:00.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.754+0000 7f27da1c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8bfd30 0x559fcd8c4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35072/0 (socket says 172.21.15.78:35072) 2024-09-11T01:23:00.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.754+0000 7f27da1c2640 1 -- 172.21.15.78:0/3723087906 learned_addr learned my addr 172.21.15.78:0/3723087906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:00.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.756+0000 7f27da1c2640 1 -- 172.21.15.78:0/3723087906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8b3480 msgr2=0x559fcd8c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:00.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.756+0000 7f27da1c2640 1 --2- 172.21.15.78:0/3723087906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8b3480 0x559fcd8c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.756+0000 7f27da1c2640 1 -- 172.21.15.78:0/3723087906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3da0 msgr2=0x559fcd8c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:00.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.756+0000 7f27da1c2640 1 --2- 172.21.15.78:0/3723087906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3da0 0x559fcd8c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.756+0000 7f27da1c2640 1 -- 172.21.15.78:0/3723087906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fcd93b5d0 con 0x559fcd8bfd30 2024-09-11T01:23:00.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.756+0000 7f27da1c2640 1 --2- 172.21.15.78:0/3723087906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8bfd30 0x559fcd8c4910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f27d00c9640 tx=0x7f27d0104330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df04849a472cd969 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:00.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.756+0000 7f27bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:23:00.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.758+0000 7f27bd7fa640 1 -- 172.21.15.78:0/3723087906 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d00b8020 con 0x559fcd8bfd30 2024-09-11T01:23:00.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.758+0000 7f27bd7fa640 1 -- 172.21.15.78:0/3723087906 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8449+0+0 (secure 0 0 0) 0x7f27d00b81c0 con 0x559fcd8bfd30 2024-09-11T01:23:00.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.759+0000 7f27ddffa740 1 -- 172.21.15.78:0/3723087906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27cc041070 con 0x559fcd8bfd30 2024-09-11T01:23:00.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.759+0000 7f27bd7fa640 1 -- 172.21.15.78:0/3723087906 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f27d004a040 con 0x559fcd8bfd30 2024-09-11T01:23:00.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.760+0000 7f27ddffa740 10 client.28441 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:00.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.760+0000 7f27ddffa740 1 -- 172.21.15.78:0/3723087906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x559fcd93c940 con 0x559fcd8bfd30 2024-09-11T01:23:00.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.761+0000 7f27bd7fa640 1 -- 172.21.15.78:0/3723087906 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27cc041070 con 0x559fcd8bfd30 2024-09-11T01:23:00.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.761+0000 7f27bd7fa640 1 -- 172.21.15.78:0/3723087906 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f27d00cd2d0 con 0x559fcd8bfd30 2024-09-11T01:23:00.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.761+0000 7f27ddffa740 10 client.28441 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:00.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.761+0000 7f27ddffa740 10 client.28441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:23:00.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.761+0000 7f27ddffa740 20 client.28441 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:00.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.761+0000 7f27ddffa740 1 --2- 172.21.15.78:0/3723087906 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559fcd93d070 0x559fcd93f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:00.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.761+0000 7f27ddffa740 4 client.28441 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:23:00.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.761+0000 7f27ddffa740 1 -- 172.21.15.78:0/3723087906 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559fcd8d1f60 con 0x559fcd93d070 2024-09-11T01:23:00.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.761+0000 7f27d99c1640 1 --2- 172.21.15.78:0/3723087906 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559fcd93d070 0x559fcd93f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:00.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.763+0000 7f27d99c1640 1 --2- 172.21.15.78:0/3723087906 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559fcd93d070 0x559fcd93f490 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d72150913bdf2bb8 server_cookie=85ccecf38a7f6dad in_seq=0 out_seq=0 2024-09-11T01:23:00.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.764+0000 7f27bd7fa640 10 client.28441 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:00.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.764+0000 7f27bd7fa640 1 -- 172.21.15.78:0/3723087906 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8042 (crc 0 0 0) 0x559fcd8d1f60 con 0x559fcd93d070 2024-09-11T01:23:00.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.764+0000 7f27bd7fa640 10 client.28441 handle_command_reply: tid=0 2024-09-11T01:23:00.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.765+0000 7f27ddffa740 1 client.28441 shutdown 2024-09-11T01:23:00.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.766+0000 7f27ddffa740 1 -- 172.21.15.78:0/3723087906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8bfd30 msgr2=0x559fcd8c4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:00.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.766+0000 7f27ddffa740 1 --2- 172.21.15.78:0/3723087906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8bfd30 0x559fcd8c4910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f27d00c9640 tx=0x7f27d0104330 comp rx=0 tx=0).stop 2024-09-11T01:23:00.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.766+0000 7f27ddffa740 1 -- 172.21.15.78:0/3723087906 shutdown_connections 2024-09-11T01:23:00.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.766+0000 7f27ddffa740 1 --2- 172.21.15.78:0/3723087906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fcd8bfd30 0x559fcd8c4910 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.766+0000 7f27ddffa740 1 --2- 172.21.15.78:0/3723087906 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559fcd93d070 0x559fcd93f490 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.766+0000 7f27ddffa740 1 --2- 172.21.15.78:0/3723087906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcd8b3da0 0x559fcd8c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.766+0000 7f27ddffa740 1 --2- 172.21.15.78:0/3723087906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd8b3480 0x559fcd8c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.766+0000 7f27ddffa740 1 -- 172.21.15.78:0/3723087906 >> 172.21.15.78:0/3723087906 conn(0x559fcd8b12b0 msgr2=0x559fcd8be050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:00.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.766+0000 7f27bd7fa640 0 client.28441 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:00.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.766+0000 7f27ddffa740 1 -- 172.21.15.78:0/3723087906 shutdown_connections 2024-09-11T01:23:00.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.766+0000 7f27ddffa740 1 -- 172.21.15.78:0/3723087906 wait complete. 2024-09-11T01:23:00.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.767+0000 7f27ddffa740 20 client.28441 trim_cache size 0 max 16384 2024-09-11T01:23:00.771 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:23:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:23:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:23:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:23:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "82736ae4-b55a-4ba3-911c-0aa1c18acfde": { 2024-09-11T01:23:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:22:21.932379+0000", 2024-09-11T01:23:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "746add9a-5be3-4745-8000-04e5471bc813": { 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:20:12.369593+0000", 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "b41a2b77-b2b1-4f1b-8588-4f32a78b721f": { 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:19:13.257610+0000", 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:23:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:18:17.654570+0000", 2024-09-11T01:23:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:23:00.775 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.776 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.777 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:23:00.777 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:23:00.777 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.777 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.777 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:23:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "291fcaa4-1805-423d-a0bc-814f1bc2759f": { 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:21:17.256483+0000", 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:23:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:23:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:23:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:23:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:23:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:23:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:23:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:23:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:23:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:23:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:23:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:23:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:23:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:23:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:23:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:23:00.790 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:23:00.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.771+0000 7f27dbc25640 1 -- 172.21.15.78:0/2704355340 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27c00b5650 msgr2=0x7f27c00b7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:00.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.771+0000 7f27dbc25640 1 --2- 172.21.15.78:0/2704355340 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27c00b5650 0x7f27c00b7b10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f27c4005ab0 tx=0x7f27c4005a20 comp rx=0 tx=0).stop 2024-09-11T01:23:00.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.771+0000 7f27dbc25640 1 -- 172.21.15.78:0/2704355340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d411c0a0 msgr2=0x7f27d41a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:00.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.771+0000 7f27dbc25640 1 --2- 172.21.15.78:0/2704355340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d411c0a0 0x7f27d41a51b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f27d002a980 tx=0x7f27d0030ed0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.771+0000 7f27dbc25640 1 -- 172.21.15.78:0/2704355340 shutdown_connections 2024-09-11T01:23:00.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.771+0000 7f27dbc25640 1 --2- 172.21.15.78:0/2704355340 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f27c00b5650 0x7f27c00b7b10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.771+0000 7f27dbc25640 1 --2- 172.21.15.78:0/2704355340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d411c0a0 0x7f27d41a51b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.771+0000 7f27dbc25640 1 --2- 172.21.15.78:0/2704355340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d410a530 0x7f27d4117e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.771+0000 7f27dbc25640 1 --2- 172.21.15.78:0/2704355340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d410a150 0x7f27d4117940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:00.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.771+0000 7f27dbc25640 1 -- 172.21.15.78:0/2704355340 >> 172.21.15.78:0/2704355340 conn(0x7f27d406dfc0 msgr2=0x7f27d4071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:00.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.771+0000 7f27dbc25640 1 -- 172.21.15.78:0/2704355340 shutdown_connections 2024-09-11T01:23:00.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:00.771+0000 7f27dbc25640 1 -- 172.21.15.78:0/2704355340 wait complete. 2024-09-11T01:23:00.809 INFO:tasks.workunit.client.0.smithi078.stdout: 10 401167 31.72 MB/sec execute 546 sec latency 209.357 ms 2024-09-11T01:23:01.809 INFO:tasks.workunit.client.0.smithi078.stdout: 10 401296 31.68 MB/sec execute 547 sec latency 616.587 ms 2024-09-11T01:23:02.525 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:23:02.526 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:23:02.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:02 smithi110 ceph-mon[34712]: pgmap v1081: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 644 op/s 2024-09-11T01:23:02.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:02 smithi112 ceph-mon[35384]: pgmap v1081: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 644 op/s 2024-09-11T01:23:02.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:02 smithi078 ceph-mon[31315]: pgmap v1081: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 644 op/s 2024-09-11T01:23:02.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 401661 31.66 MB/sec execute 548 sec latency 897.833 ms 2024-09-11T01:23:02.915 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:23:03.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.499+0000 7f4713816640 1 -- 172.21.15.78:0/1993130487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c100d30 msgr2=0x7f470c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:03.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.499+0000 7f4713816640 1 --2- 172.21.15.78:0/1993130487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c100d30 0x7f470c101110 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4700009900 tx=0x7f470002a440 comp rx=0 tx=0).stop 2024-09-11T01:23:03.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.499+0000 7f4713816640 1 -- 172.21.15.78:0/1993130487 shutdown_connections 2024-09-11T01:23:03.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.499+0000 7f4713816640 1 --2- 172.21.15.78:0/1993130487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f470c108030 0x7f470c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:03.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.499+0000 7f4713816640 1 --2- 172.21.15.78:0/1993130487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f470c1076b0 0x7f470c107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:03.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.499+0000 7f4713816640 1 --2- 172.21.15.78:0/1993130487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c100d30 0x7f470c101110 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:03.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.499+0000 7f4713816640 1 -- 172.21.15.78:0/1993130487 >> 172.21.15.78:0/1993130487 conn(0x7f470c0fca20 msgr2=0x7f470c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:03.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.500+0000 7f4713816640 1 -- 172.21.15.78:0/1993130487 shutdown_connections 2024-09-11T01:23:03.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.500+0000 7f4713816640 1 -- 172.21.15.78:0/1993130487 wait complete. 2024-09-11T01:23:03.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.501+0000 7f4713816640 1 Processor -- start 2024-09-11T01:23:03.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.501+0000 7f4713816640 1 -- start start 2024-09-11T01:23:03.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.502+0000 7f4713816640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f470c100d30 0x7f470c10f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:03.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.502+0000 7f4713816640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c1076b0 0x7f470c10b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:03.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.502+0000 7f4710db1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c1076b0 0x7f470c10b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:03.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.502+0000 7f4710db1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c1076b0 0x7f470c10b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52274/0 (socket says 172.21.15.78:52274) 2024-09-11T01:23:03.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.502+0000 7f47115b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f470c100d30 0x7f470c10f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:03.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.502+0000 7f47115b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f470c100d30 0x7f470c10f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36818/0 (socket says 172.21.15.78:36818) 2024-09-11T01:23:03.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.502+0000 7f4713816640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f470c108030 0x7f470c10c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:03.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.502+0000 7f4710db1640 1 -- 172.21.15.78:0/360094791 learned_addr learned my addr 172.21.15.78:0/360094791 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:03.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.502+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f470c103210 con 0x7f470c1076b0 2024-09-11T01:23:03.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.502+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f470c103090 con 0x7f470c100d30 2024-09-11T01:23:03.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.502+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f470c103390 con 0x7f470c108030 2024-09-11T01:23:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.503+0000 7f4710db1640 1 -- 172.21.15.78:0/360094791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f470c108030 msgr2=0x7f470c10c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.503+0000 7f4710db1640 1 --2- 172.21.15.78:0/360094791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f470c108030 0x7f470c10c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.503+0000 7f4710db1640 1 -- 172.21.15.78:0/360094791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f470c100d30 msgr2=0x7f470c10f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.503+0000 7f4710db1640 1 --2- 172.21.15.78:0/360094791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f470c100d30 0x7f470c10f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.503+0000 7f4710db1640 1 -- 172.21.15.78:0/360094791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c1aba10 con 0x7f470c1076b0 2024-09-11T01:23:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.503+0000 7f4710db1640 1 --2- 172.21.15.78:0/360094791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c1076b0 0x7f470c10b9f0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00cce0 tx=0x7f46fc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.504+0000 7f470a7fc640 1 -- 172.21.15.78:0/360094791 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc00f650 con 0x7f470c1076b0 2024-09-11T01:23:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.504+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470c1abc40 con 0x7f470c1076b0 2024-09-11T01:23:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.504+0000 7f470a7fc640 1 -- 172.21.15.78:0/360094791 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46fc00fca0 con 0x7f470c1076b0 2024-09-11T01:23:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.504+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470c1ac140 con 0x7f470c1076b0 2024-09-11T01:23:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.504+0000 7f470a7fc640 1 -- 172.21.15.78:0/360094791 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc018b90 con 0x7f470c1076b0 2024-09-11T01:23:03.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.506+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470c10ccf0 con 0x7f470c1076b0 2024-09-11T01:23:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.508+0000 7f470a7fc640 1 -- 172.21.15.78:0/360094791 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f46fc00f7f0 con 0x7f470c1076b0 2024-09-11T01:23:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.509+0000 7f470a7fc640 1 --2- 172.21.15.78:0/360094791 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f46f40b5500 0x7f46f40b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.510+0000 7f47115b2640 1 --2- 172.21.15.78:0/360094791 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f46f40b5500 0x7f46f40b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.510+0000 7f470a7fc640 1 -- 172.21.15.78:0/360094791 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8493+0+0 (secure 0 0 0) 0x7f46fc0e64f0 con 0x7f470c1076b0 2024-09-11T01:23:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.511+0000 7f47115b2640 1 --2- 172.21.15.78:0/360094791 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f46f40b5500 0x7f46f40b79c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4700009850 tx=0x7f4700000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:03.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.517+0000 7f470a7fc640 1 -- 172.21.15.78:0/360094791 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f46fc014030 con 0x7f470c1076b0 2024-09-11T01:23:03.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:03 smithi110 ceph-mon[34712]: osdmap e112: 12 total, 12 up, 12 in 2024-09-11T01:23:03.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:03 smithi110 ceph-mon[34712]: pgmap v1083: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 674 op/s 2024-09-11T01:23:03.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:03 smithi112 ceph-mon[35384]: osdmap e112: 12 total, 12 up, 12 in 2024-09-11T01:23:03.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:03 smithi112 ceph-mon[35384]: pgmap v1083: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 674 op/s 2024-09-11T01:23:03.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:03 smithi078 ceph-mon[31315]: osdmap e112: 12 total, 12 up, 12 in 2024-09-11T01:23:03.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:03 smithi078 ceph-mon[31315]: pgmap v1083: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 674 op/s 2024-09-11T01:23:03.811 INFO:tasks.workunit.client.0.smithi078.stdout: 10 402212 31.63 MB/sec execute 549 sec latency 213.062 ms 2024-09-11T01:23:03.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.941+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f470c1ac770 con 0x7f470c1076b0 2024-09-11T01:23:03.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.942+0000 7f470a7fc640 1 -- 172.21.15.78:0/360094791 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11986 (secure 0 0 0) 0x7f46fc0ad090 con 0x7f470c1076b0 2024-09-11T01:23:03.944 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:23:03.945 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":76,"btime":"2024-09-11T01:22:07:149631+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:22:07.008352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:23:03.945 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 76 2024-09-11T01:23:03.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.946+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f46f40b5500 msgr2=0x7f46f40b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:03.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.946+0000 7f4713816640 1 --2- 172.21.15.78:0/360094791 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f46f40b5500 0x7f46f40b79c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4700009850 tx=0x7f4700000bc0 comp rx=0 tx=0).stop 2024-09-11T01:23:03.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.946+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c1076b0 msgr2=0x7f470c10b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:03.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.946+0000 7f4713816640 1 --2- 172.21.15.78:0/360094791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c1076b0 0x7f470c10b9f0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00cce0 tx=0x7f46fc00a620 comp rx=0 tx=0).stop 2024-09-11T01:23:03.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.946+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 shutdown_connections 2024-09-11T01:23:03.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.946+0000 7f4713816640 1 --2- 172.21.15.78:0/360094791 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f46f40b5500 0x7f46f40b79c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:03.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.946+0000 7f4713816640 1 --2- 172.21.15.78:0/360094791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f470c108030 0x7f470c10c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:03.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.946+0000 7f4713816640 1 --2- 172.21.15.78:0/360094791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c1076b0 0x7f470c10b9f0 secure :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00cce0 tx=0x7f46fc00a620 comp rx=0 tx=0).stop 2024-09-11T01:23:03.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.946+0000 7f4713816640 1 --2- 172.21.15.78:0/360094791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f470c100d30 0x7f470c10f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:03.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.946+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 >> 172.21.15.78:0/360094791 conn(0x7f470c0fca20 msgr2=0x7f470c105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:03.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.946+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 shutdown_connections 2024-09-11T01:23:03.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:03.946+0000 7f4713816640 1 -- 172.21.15.78:0/360094791 wait complete. 2024-09-11T01:23:04.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:04 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/360094791' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:23:04.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:04 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/360094791' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:23:04.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:04 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/360094791' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:23:04.811 INFO:tasks.workunit.client.0.smithi078.stdout: 10 402430 31.60 MB/sec execute 550 sec latency 224.188 ms 2024-09-11T01:23:04.932 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:23:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:23:05.677 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:05 smithi078 ceph-mon[31315]: pgmap v1084: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 22 MiB/s wr, 710 op/s 2024-09-11T01:23:05.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:05 smithi110 ceph-mon[34712]: pgmap v1084: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 22 MiB/s wr, 710 op/s 2024-09-11T01:23:05.723 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:05 smithi112 ceph-mon[35384]: pgmap v1084: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 22 MiB/s wr, 710 op/s 2024-09-11T01:23:05.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 402903 31.60 MB/sec execute 551 sec latency 142.319 ms 2024-09-11T01:23:05.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.982+0000 7feb1f324640 1 -- 172.21.15.78:0/2879461561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb181076a0 msgr2=0x7feb18107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:05.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.982+0000 7feb1f324640 1 --2- 172.21.15.78:0/2879461561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb181076a0 0x7feb18107ae0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7feb08009900 tx=0x7feb0802a410 comp rx=0 tx=0).stop 2024-09-11T01:23:05.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.982+0000 7feb1f324640 1 -- 172.21.15.78:0/2879461561 shutdown_connections 2024-09-11T01:23:05.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.982+0000 7feb1f324640 1 --2- 172.21.15.78:0/2879461561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb18108020 0x7feb1810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:05.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.982+0000 7feb1f324640 1 --2- 172.21.15.78:0/2879461561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb181076a0 0x7feb18107ae0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:05.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.982+0000 7feb1f324640 1 --2- 172.21.15.78:0/2879461561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb18101c90 0x7feb18102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:05.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.982+0000 7feb1f324640 1 -- 172.21.15.78:0/2879461561 >> 172.21.15.78:0/2879461561 conn(0x7feb180fd700 msgr2=0x7feb180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:05.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.982+0000 7feb1f324640 1 -- 172.21.15.78:0/2879461561 shutdown_connections 2024-09-11T01:23:05.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.982+0000 7feb1f324640 1 -- 172.21.15.78:0/2879461561 wait complete. 2024-09-11T01:23:05.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.983+0000 7feb1f324640 1 Processor -- start 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.983+0000 7feb1f324640 1 -- start start 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.983+0000 7feb1f324640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb18101c90 0x7feb18199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.983+0000 7feb1f324640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb181076a0 0x7feb181997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.983+0000 7feb1f324640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb18108020 0x7feb1819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.983+0000 7feb1f324640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb18112de0 con 0x7feb18101c90 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.983+0000 7feb1f324640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feb18112c60 con 0x7feb181076a0 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.983+0000 7feb1f324640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb18112f60 con 0x7feb18108020 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.983+0000 7feb1d0c0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb18101c90 0x7feb18199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.983+0000 7feb1d0c0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb18101c90 0x7feb18199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52296/0 (socket says 172.21.15.78:52296) 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.983+0000 7feb1d0c0640 1 -- 172.21.15.78:0/2129369854 learned_addr learned my addr 172.21.15.78:0/2129369854 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.983+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb181076a0 0x7feb181997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.984+0000 7feb1d8c1640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb18108020 0x7feb1819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.984+0000 7feb1d0c0640 1 -- 172.21.15.78:0/2129369854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb18108020 msgr2=0x7feb1819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:05.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.984+0000 7feb1d0c0640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb18108020 0x7feb1819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:05.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.984+0000 7feb1d0c0640 1 -- 172.21.15.78:0/2129369854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb181076a0 msgr2=0x7feb181997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:05.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.984+0000 7feb1d0c0640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb181076a0 0x7feb181997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:05.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.984+0000 7feb1d0c0640 1 -- 172.21.15.78:0/2129369854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1819f320 con 0x7feb18101c90 2024-09-11T01:23:05.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.984+0000 7feb1d0c0640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb18101c90 0x7feb18199270 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7feb1400e9f0 tx=0x7feb1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:05.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.984+0000 7feb0e7fc640 1 -- 172.21.15.78:0/2129369854 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1400cd80 con 0x7feb18101c90 2024-09-11T01:23:05.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.984+0000 7feb1f324640 1 -- 172.21.15.78:0/2129369854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1819f610 con 0x7feb18101c90 2024-09-11T01:23:05.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.985+0000 7feb1f324640 1 -- 172.21.15.78:0/2129369854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb181a5ea0 con 0x7feb18101c90 2024-09-11T01:23:05.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.985+0000 7feb0e7fc640 1 -- 172.21.15.78:0/2129369854 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb140108e0 con 0x7feb18101c90 2024-09-11T01:23:05.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.985+0000 7feb0e7fc640 1 -- 172.21.15.78:0/2129369854 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14018980 con 0x7feb18101c90 2024-09-11T01:23:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.989+0000 7feb0e7fc640 1 -- 172.21.15.78:0/2129369854 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7feb14010430 con 0x7feb18101c90 2024-09-11T01:23:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.989+0000 7feb216f9740 1 Processor -- start 2024-09-11T01:23:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.989+0000 7feb216f9740 1 -- start start 2024-09-11T01:23:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.989+0000 7feb216f9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3abf50 0x557c5e3ac330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.989+0000 7feb216f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bc8a0 0x557c5e3bcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.989+0000 7feb216f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3bd220 0x557c5e3c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.989+0000 7feb216f9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557c5e324ea0 con 0x557c5e3abf50 2024-09-11T01:23:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.989+0000 7feb216f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557c5e3142d0 con 0x557c5e3bd220 2024-09-11T01:23:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.989+0000 7feb216f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557c5e3180e0 con 0x557c5e3bc8a0 2024-09-11T01:23:05.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.990+0000 7feb0e7fc640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feb040b54d0 0x7feb040b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:05.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.990+0000 7feb0e7fc640 1 -- 172.21.15.78:0/2129369854 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8493+0+0 (secure 0 0 0) 0x7feb140e60d0 con 0x7feb18101c90 2024-09-11T01:23:05.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.990+0000 7feb1c8bf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3bd220 0x557c5e3c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:05.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.990+0000 7feb1c8bf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3bd220 0x557c5e3c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36848/0 (socket says 172.21.15.78:36848) 2024-09-11T01:23:05.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.990+0000 7feb1c8bf640 1 -- 172.21.15.78:0/1927308697 learned_addr learned my addr 172.21.15.78:0/1927308697 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:05.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.991+0000 7feb1c8bf640 1 -- 172.21.15.78:0/1927308697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bc8a0 msgr2=0x557c5e3bcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:05.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.991+0000 7feb1d8c1640 1 --2- 172.21.15.78:0/1927308697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3abf50 0x557c5e3ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:05.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.991+0000 7feb1d0c0640 1 --2- 172.21.15.78:0/1927308697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bc8a0 0x557c5e3bcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:05.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.991+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/1927308697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bc8a0 0x557c5e3bcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:05.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.991+0000 7feb1c8bf640 1 -- 172.21.15.78:0/1927308697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3abf50 msgr2=0x557c5e3ac330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:05.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.991+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/1927308697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3abf50 0x557c5e3ac330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:05.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.991+0000 7feb1c8bf640 1 -- 172.21.15.78:0/1927308697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c5e072650 con 0x557c5e3bd220 2024-09-11T01:23:05.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.991+0000 7feb1d8c1640 1 --2- 172.21.15.78:0/1927308697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3abf50 0x557c5e3ac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:23:05.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.991+0000 7feb1d8c1640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feb040b54d0 0x7feb040b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:05.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.991+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/1927308697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3bd220 0x557c5e3c5be0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feb08009900 tx=0x7feb08032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346c1674f6bd15e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:05.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.992+0000 7feb037fe640 1 -- 172.21.15.78:0/1927308697 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08040070 con 0x557c5e3bd220 2024-09-11T01:23:05.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.992+0000 7feb037fe640 1 -- 172.21.15.78:0/1927308697 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb08004030 con 0x557c5e3bd220 2024-09-11T01:23:05.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.992+0000 7feb037fe640 1 -- 172.21.15.78:0/1927308697 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08035040 con 0x557c5e3bd220 2024-09-11T01:23:05.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.993+0000 7feb216f9740 1 -- 172.21.15.78:0/1927308697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3bd220 msgr2=0x557c5e3c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:05.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.993+0000 7feb1d8c1640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feb040b54d0 0x7feb040b7990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7feb10005c40 tx=0x7feb10014040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:05.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.993+0000 7feb216f9740 1 --2- 172.21.15.78:0/1927308697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3bd220 0x557c5e3c5be0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feb08009900 tx=0x7feb08032b20 comp rx=0 tx=0).stop 2024-09-11T01:23:05.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.993+0000 7feb216f9740 1 -- 172.21.15.78:0/1927308697 shutdown_connections 2024-09-11T01:23:05.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.993+0000 7feb216f9740 1 --2- 172.21.15.78:0/1927308697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3bd220 0x557c5e3c5be0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:05.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.993+0000 7feb216f9740 1 --2- 172.21.15.78:0/1927308697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bc8a0 0x557c5e3bcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:05.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.993+0000 7feb216f9740 1 --2- 172.21.15.78:0/1927308697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3abf50 0x557c5e3ac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:05.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.993+0000 7feb216f9740 1 -- 172.21.15.78:0/1927308697 >> 172.21.15.78:0/1927308697 conn(0x557c5e3ae470 msgr2=0x557c5e3ae850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:05.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.993+0000 7feb216f9740 1 -- 172.21.15.78:0/1927308697 shutdown_connections 2024-09-11T01:23:05.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.993+0000 7feb216f9740 1 -- 172.21.15.78:0/1927308697 wait complete. 2024-09-11T01:23:05.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.994+0000 7feb216f9740 1 -- start start 2024-09-11T01:23:05.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.994+0000 7feb216f9740 1 Processor -- start 2024-09-11T01:23:05.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.995+0000 7feb216f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 0x557c5e3c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:05.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.995+0000 7feb216f9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 0x557c5e3c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:05.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.995+0000 7feb216f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bd220 0x557c5e3c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:05.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.996+0000 7feb1c8bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 0x557c5e3c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:05.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.996+0000 7feb1c8bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 0x557c5e3c2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52314/0 (socket says 172.21.15.78:52314) 2024-09-11T01:23:05.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.996+0000 7feb1c8bf640 1 -- 172.21.15.78:0/928448969 learned_addr learned my addr 172.21.15.78:0/928448969 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:05.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.996+0000 7feb1d0c0640 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 0x557c5e3c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:05.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.996+0000 7feb216f9740 1 -- 172.21.15.78:0/928448969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557c5e175700 con 0x557c5e3bc8a0 2024-09-11T01:23:05.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.996+0000 7feb216f9740 1 -- 172.21.15.78:0/928448969 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557c5e2af160 con 0x557c5e3abf50 2024-09-11T01:23:05.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.996+0000 7feb216f9740 1 -- 172.21.15.78:0/928448969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557c5e3c6270 con 0x557c5e3bd220 2024-09-11T01:23:05.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.996+0000 7feb216f9740 10 client.0 _ll_register_callbacks cb 0x557c5e2c89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:06.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.997+0000 7feb1c8bf640 1 -- 172.21.15.78:0/928448969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bd220 msgr2=0x557c5e3c3530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:23:06.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.997+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bd220 0x557c5e3c3530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.997+0000 7feb1c8bf640 1 -- 172.21.15.78:0/928448969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 msgr2=0x557c5e3c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:06.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.997+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 0x557c5e3c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.997+0000 7feb1c8bf640 1 -- 172.21.15.78:0/928448969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c5e2dcc90 con 0x557c5e3bc8a0 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.997+0000 7feb1d0c0640 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 0x557c5e3c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.997+0000 7feb216f9740 1 -- 172.21.15.78:0/928448969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb08004030 con 0x557c5e3bc8a0 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.997+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 0x557c5e3c2eb0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7feb080316d0 tx=0x7feb08005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d07bbf65998fa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.997+0000 7feb00ff9640 10 client.19476 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.997+0000 7feb00ff9640 1 -- 172.21.15.78:0/928448969 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08040070 con 0x557c5e3bc8a0 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.998+0000 7feb00ff9640 1 -- 172.21.15.78:0/928448969 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8493+0+0 (secure 0 0 0) 0x7feb08005150 con 0x557c5e3bc8a0 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.998+0000 7feb00ff9640 1 -- 172.21.15.78:0/928448969 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7feb08035040 con 0x557c5e3bc8a0 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.998+0000 7feb216f9740 10 client.19476 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.998+0000 7feb216f9740 1 -- 172.21.15.78:0/928448969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=76}) -- 0x557c5e3d1ad0 con 0x557c5e3bc8a0 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.999+0000 7feb00ff9640 1 -- 172.21.15.78:0/928448969 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7feb08005390 con 0x557c5e3bc8a0 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.999+0000 7feb216f9740 10 client.19476 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.999+0000 7feb216f9740 10 client.19476 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:23:06.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.999+0000 7feb216f9740 20 client.19476 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:06.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.999+0000 7feb216f9740 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x557c5e3d2e20 0x557c5e3d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:06.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.999+0000 7feb216f9740 4 client.19476 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:23:06.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:05.999+0000 7feb216f9740 1 -- 172.21.15.78:0/928448969 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c5e326df0 con 0x557c5e3d2e20 2024-09-11T01:23:06.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.000+0000 7feb1d0c0640 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x557c5e3d2e20 0x557c5e3d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:06.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.000+0000 7feb00ff9640 10 client.19476 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:23:06.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.000+0000 7feb1d0c0640 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x557c5e3d2e20 0x557c5e3d5210 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce5b98fc85c6d2a6 server_cookie=93d0574856e5b79f in_seq=0 out_seq=0 2024-09-11T01:23:06.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.003+0000 7feb00ff9640 1 -- 172.21.15.78:0/928448969 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c5e326df0 con 0x557c5e3d2e20 2024-09-11T01:23:06.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.003+0000 7feb00ff9640 10 client.19476 handle_command_reply: tid=0 2024-09-11T01:23:06.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.003+0000 7feb216f9740 1 client.19476 shutdown 2024-09-11T01:23:06.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.003+0000 7feb216f9740 1 -- 172.21.15.78:0/928448969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 msgr2=0x557c5e3c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:06.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.003+0000 7feb216f9740 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 0x557c5e3c2eb0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7feb080316d0 tx=0x7feb08005fd0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.004+0000 7feb216f9740 1 -- 172.21.15.78:0/928448969 shutdown_connections 2024-09-11T01:23:06.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.004+0000 7feb216f9740 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x557c5e3d2e20 0x557c5e3d5210 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.004+0000 7feb00ff9640 0 client.19476 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:23:06.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.004+0000 7feb216f9740 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bd220 0x557c5e3c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.004+0000 7feb216f9740 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 0x557c5e3c2eb0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.004+0000 7feb216f9740 1 --2- 172.21.15.78:0/928448969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 0x557c5e3c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.004+0000 7feb216f9740 1 -- 172.21.15.78:0/928448969 >> 172.21.15.78:0/928448969 conn(0x557c5e3ae470 msgr2=0x557c5e3be180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:06.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.004+0000 7feb216f9740 1 -- 172.21.15.78:0/928448969 shutdown_connections 2024-09-11T01:23:06.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.004+0000 7feb216f9740 1 -- 172.21.15.78:0/928448969 wait complete. 2024-09-11T01:23:06.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.004+0000 7feb216f9740 20 client.19476 trim_cache size 0 max 16384 2024-09-11T01:23:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.027+0000 7feb216f9740 1 Processor -- start 2024-09-11T01:23:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.027+0000 7feb216f9740 1 -- start start 2024-09-11T01:23:06.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.028+0000 7feb216f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3abf50 0x557c5e3c0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:06.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.028+0000 7feb216f9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 0x557c5e3c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:06.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.028+0000 7feb216f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3bd220 0x557c5e3ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:06.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.028+0000 7feb216f9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feafc00e720 con 0x557c5e3bc8a0 2024-09-11T01:23:06.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.028+0000 7feb216f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feafc010320 con 0x557c5e3bd220 2024-09-11T01:23:06.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.028+0000 7feb216f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feafc00edf0 con 0x557c5e3abf50 2024-09-11T01:23:06.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.028+0000 7feb1c8bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 0x557c5e3c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:06.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.028+0000 7feb1c8bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 0x557c5e3c4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52324/0 (socket says 172.21.15.78:52324) 2024-09-11T01:23:06.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.028+0000 7feb1c8bf640 1 -- 172.21.15.78:0/3832006128 learned_addr learned my addr 172.21.15.78:0/3832006128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:06.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.029+0000 7feb1d8c1640 1 --2- 172.21.15.78:0/3832006128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3bd220 0x557c5e3ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:06.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.029+0000 7feb1d0c0640 1 --2- 172.21.15.78:0/3832006128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3abf50 0x557c5e3c0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:06.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.030+0000 7feb1c8bf640 1 -- 172.21.15.78:0/3832006128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3abf50 msgr2=0x557c5e3c0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:06.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.030+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/3832006128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3abf50 0x557c5e3c0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.030+0000 7feb1c8bf640 1 -- 172.21.15.78:0/3832006128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3bd220 msgr2=0x557c5e3ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:06.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.030+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/3832006128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3bd220 0x557c5e3ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.030+0000 7feb1c8bf640 1 -- 172.21.15.78:0/3832006128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c5e3c38a0 con 0x557c5e3bc8a0 2024-09-11T01:23:06.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.030+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/3832006128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 0x557c5e3c4bf0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7feb08031440 tx=0x7feb08048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a589ba29541581d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:06.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.031+0000 7feb02ffd640 1 -- 172.21.15.78:0/3832006128 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08032e90 con 0x557c5e3bc8a0 2024-09-11T01:23:06.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.031+0000 7feb02ffd640 1 -- 172.21.15.78:0/3832006128 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb08004030 con 0x557c5e3bc8a0 2024-09-11T01:23:06.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.031+0000 7feb02ffd640 1 -- 172.21.15.78:0/3832006128 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08052720 con 0x557c5e3bc8a0 2024-09-11T01:23:06.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.032+0000 7feb216f9740 1 -- 172.21.15.78:0/3832006128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 msgr2=0x557c5e3c4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:06.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.032+0000 7feb216f9740 1 --2- 172.21.15.78:0/3832006128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 0x557c5e3c4bf0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7feb08031440 tx=0x7feb08048000 comp rx=0 tx=0).stop 2024-09-11T01:23:06.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.032+0000 7feb216f9740 1 -- 172.21.15.78:0/3832006128 shutdown_connections 2024-09-11T01:23:06.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.032+0000 7feb216f9740 1 --2- 172.21.15.78:0/3832006128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3bd220 0x557c5e3ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.032+0000 7feb216f9740 1 --2- 172.21.15.78:0/3832006128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bc8a0 0x557c5e3c4bf0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.032+0000 7feb216f9740 1 --2- 172.21.15.78:0/3832006128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3abf50 0x557c5e3c0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.032+0000 7feb216f9740 1 -- 172.21.15.78:0/3832006128 >> 172.21.15.78:0/3832006128 conn(0x557c5e3ae470 msgr2=0x557c5e3c44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:06.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.032+0000 7feb216f9740 1 -- 172.21.15.78:0/3832006128 shutdown_connections 2024-09-11T01:23:06.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.033+0000 7feb216f9740 1 -- 172.21.15.78:0/3832006128 wait complete. 2024-09-11T01:23:06.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.033+0000 7feb216f9740 1 -- start start 2024-09-11T01:23:06.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.033+0000 7feb216f9740 1 Processor -- start 2024-09-11T01:23:06.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.034+0000 7feb216f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 0x557c5e3d75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:06.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.034+0000 7feb216f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bc8a0 0x557c5e3d7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:06.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.034+0000 7feb216f9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bd220 0x557c5e43e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.034+0000 7feb216f9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feafc00fc50 con 0x557c5e3bd220 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.034+0000 7feb216f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feafc00f580 con 0x557c5e3abf50 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.034+0000 7feb216f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feafc00c720 con 0x557c5e3bc8a0 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.034+0000 7feb1d0c0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 0x557c5e3d75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.034+0000 7feb1d0c0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 0x557c5e3d75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36862/0 (socket says 172.21.15.78:36862) 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.034+0000 7feb1d0c0640 1 -- 172.21.15.78:0/3723663152 learned_addr learned my addr 172.21.15.78:0/3723663152 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.034+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bc8a0 0x557c5e3d7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.034+0000 7feb216f9740 10 client.0 _ll_register_callbacks cb 0x7feb08009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb1d8c1640 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bd220 0x557c5e43e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb1d0c0640 1 -- 172.21.15.78:0/3723663152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bc8a0 msgr2=0x557c5e3d7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb1d0c0640 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bc8a0 0x557c5e3d7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb1d0c0640 1 -- 172.21.15.78:0/3723663152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bd220 msgr2=0x557c5e43e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb1d0c0640 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bd220 0x557c5e43e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb1d0c0640 1 -- 172.21.15.78:0/3723663152 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c5e43eed0 con 0x557c5e3abf50 2024-09-11T01:23:06.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb1d8c1640 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bd220 0x557c5e43e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:23:06.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb1d0c0640 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 0x557c5e3d75d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7feb140e38f0 tx=0x7feb140e63d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9caa6ac3fde794e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:06.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:23:06.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb00ff9640 1 -- 172.21.15.78:0/3723663152 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140d3070 con 0x557c5e3abf50 2024-09-11T01:23:06.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb00ff9640 1 -- 172.21.15.78:0/3723663152 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8493+0+0 (secure 0 0 0) 0x7feb140d3210 con 0x557c5e3abf50 2024-09-11T01:23:06.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb216f9740 1 -- 172.21.15.78:0/3723663152 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb08004030 con 0x557c5e3abf50 2024-09-11T01:23:06.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.035+0000 7feb00ff9640 1 -- 172.21.15.78:0/3723663152 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7feb140b5440 con 0x557c5e3abf50 2024-09-11T01:23:06.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.036+0000 7feb216f9740 10 client.28505 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:06.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.036+0000 7feb216f9740 1 -- 172.21.15.78:0/3723663152 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x557c5e4400b0 con 0x557c5e3abf50 2024-09-11T01:23:06.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.036+0000 7feb00ff9640 1 -- 172.21.15.78:0/3723663152 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb08004030 con 0x557c5e3abf50 2024-09-11T01:23:06.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.036+0000 7feb00ff9640 1 -- 172.21.15.78:0/3723663152 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7feb140a7ec0 con 0x557c5e3abf50 2024-09-11T01:23:06.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.036+0000 7feb216f9740 10 client.28505 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:06.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.036+0000 7feb216f9740 10 client.28505 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:23:06.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.036+0000 7feb216f9740 20 client.28505 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:06.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.036+0000 7feb216f9740 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x557c5e4407b0 0x557c5e442bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:06.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.036+0000 7feb216f9740 4 client.28505 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:23:06.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.036+0000 7feb216f9740 1 -- 172.21.15.78:0/3723663152 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c5e3228a0 con 0x557c5e4407b0 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.037+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x557c5e4407b0 0x557c5e442bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.037+0000 7feb1c8bf640 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x557c5e4407b0 0x557c5e442bd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c970c3416e86548d server_cookie=dd456c567c8507f6 in_seq=0 out_seq=0 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.037+0000 7feb00ff9640 10 client.28505 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.038+0000 7feb00ff9640 1 -- 172.21.15.78:0/3723663152 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c5e3228a0 con 0x557c5e4407b0 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.038+0000 7feb00ff9640 10 client.28505 handle_command_reply: tid=0 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.038+0000 7feb216f9740 1 client.28505 shutdown 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.038+0000 7feb216f9740 1 -- 172.21.15.78:0/3723663152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 msgr2=0x557c5e3d75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.038+0000 7feb216f9740 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 0x557c5e3d75d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7feb140e38f0 tx=0x7feb140e63d0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.038+0000 7feb216f9740 1 -- 172.21.15.78:0/3723663152 shutdown_connections 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.038+0000 7feb216f9740 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x557c5e4407b0 0x557c5e442bd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.038+0000 7feb216f9740 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c5e3bd220 0x557c5e43e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.038+0000 7feb216f9740 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c5e3bc8a0 0x557c5e3d7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.038+0000 7feb00ff9640 0 client.28505 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:23:06.043 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:23:06.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.038+0000 7feb216f9740 1 --2- 172.21.15.78:0/3723663152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c5e3abf50 0x557c5e3d75d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.038+0000 7feb216f9740 1 -- 172.21.15.78:0/3723663152 >> 172.21.15.78:0/3723663152 conn(0x557c5e3ae470 msgr2=0x557c5e3c4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:06.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.039+0000 7feb216f9740 1 -- 172.21.15.78:0/3723663152 shutdown_connections 2024-09-11T01:23:06.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.039+0000 7feb216f9740 1 -- 172.21.15.78:0/3723663152 wait complete. 2024-09-11T01:23:06.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.039+0000 7feb216f9740 20 client.28505 trim_cache size 0 max 16384 2024-09-11T01:23:06.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.040+0000 7feb1f324640 1 -- 172.21.15.78:0/2129369854 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feb040b54d0 msgr2=0x7feb040b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:06.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.040+0000 7feb1f324640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feb040b54d0 0x7feb040b7990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7feb10005c40 tx=0x7feb10014040 comp rx=0 tx=0).stop 2024-09-11T01:23:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.040+0000 7feb1f324640 1 -- 172.21.15.78:0/2129369854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb18101c90 msgr2=0x7feb18199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.040+0000 7feb1f324640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb18101c90 0x7feb18199270 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7feb1400e9f0 tx=0x7feb1400eeb0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.040+0000 7feb1f324640 1 -- 172.21.15.78:0/2129369854 shutdown_connections 2024-09-11T01:23:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.040+0000 7feb1f324640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7feb040b54d0 0x7feb040b7990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.040+0000 7feb1f324640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb18108020 0x7feb1819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.040+0000 7feb1f324640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb181076a0 0x7feb181997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.040+0000 7feb1f324640 1 --2- 172.21.15.78:0/2129369854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb18101c90 0x7feb18199270 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.040+0000 7feb1f324640 1 -- 172.21.15.78:0/2129369854 >> 172.21.15.78:0/2129369854 conn(0x7feb180fd700 msgr2=0x7feb180ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.040+0000 7feb1f324640 1 -- 172.21.15.78:0/2129369854 shutdown_connections 2024-09-11T01:23:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:06.040+0000 7feb1f324640 1 -- 172.21.15.78:0/2129369854 wait complete. 2024-09-11T01:23:06.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 403538 31.62 MB/sec execute 552 sec latency 89.960 ms 2024-09-11T01:23:07.588 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:23:07.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:07 smithi078 ceph-mon[31315]: pgmap v1085: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 862 op/s 2024-09-11T01:23:07.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 404057 31.60 MB/sec execute 553 sec latency 110.799 ms 2024-09-11T01:23:07.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:07 smithi110 ceph-mon[34712]: pgmap v1085: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 862 op/s 2024-09-11T01:23:07.965 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:23:07.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:07 smithi112 ceph-mon[35384]: pgmap v1085: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 862 op/s 2024-09-11T01:23:08.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 404636 31.61 MB/sec execute 554 sec latency 88.338 ms 2024-09-11T01:23:09.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 404967 31.58 MB/sec execute 555 sec latency 440.340 ms 2024-09-11T01:23:09.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:09 smithi110 ceph-mon[34712]: pgmap v1086: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 868 op/s 2024-09-11T01:23:09.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:09 smithi110 ceph-mon[34712]: osdmap e113: 12 total, 12 up, 12 in 2024-09-11T01:23:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:09 smithi112 ceph-mon[35384]: pgmap v1086: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 868 op/s 2024-09-11T01:23:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:09 smithi112 ceph-mon[35384]: osdmap e113: 12 total, 12 up, 12 in 2024-09-11T01:23:09.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:09 smithi078 ceph-mon[31315]: pgmap v1086: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 868 op/s 2024-09-11T01:23:09.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:09 smithi078 ceph-mon[31315]: osdmap e113: 12 total, 12 up, 12 in 2024-09-11T01:23:10.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 405346 31.56 MB/sec execute 556 sec latency 253.105 ms 2024-09-11T01:23:11.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 405770 31.53 MB/sec execute 557 sec latency 146.350 ms 2024-09-11T01:23:11.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:11 smithi110 ceph-mon[34712]: pgmap v1088: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 866 op/s 2024-09-11T01:23:11.954 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:11 smithi078 ceph-mon[31315]: pgmap v1088: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 866 op/s 2024-09-11T01:23:11.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:11 smithi112 ceph-mon[35384]: pgmap v1088: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 866 op/s 2024-09-11T01:23:12.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.209+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/3477471677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc08104600 msgr2=0x7fdc0810caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:12.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.209+0000 7fdc0e1e7640 1 --2- 172.21.15.78:0/3477471677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc08104600 0x7fdc0810caf0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009900 tx=0x7fdbfc02a410 comp rx=0 tx=0).stop 2024-09-11T01:23:12.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.209+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/3477471677 shutdown_connections 2024-09-11T01:23:12.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.209+0000 7fdc0e1e7640 1 --2- 172.21.15.78:0/3477471677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc0810d140 0x7fdc0810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.209+0000 7fdc0e1e7640 1 --2- 172.21.15.78:0/3477471677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc08104600 0x7fdc0810caf0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.209+0000 7fdc0e1e7640 1 --2- 172.21.15.78:0/3477471677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc08103ce0 0x7fdc081040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.209+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/3477471677 >> 172.21.15.78:0/3477471677 conn(0x7fdc080fd700 msgr2=0x7fdc080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.209+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/3477471677 shutdown_connections 2024-09-11T01:23:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.209+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/3477471677 wait complete. 2024-09-11T01:23:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.210+0000 7fdc0e1e7640 1 Processor -- start 2024-09-11T01:23:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.210+0000 7fdc0e1e7640 1 -- start start 2024-09-11T01:23:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.210+0000 7fdc0e1e7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc08103ce0 0x7fdc08101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.210+0000 7fdc0e1e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc08104600 0x7fdc08101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.210+0000 7fdc0e1e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc0810d140 0x7fdc08102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.210+0000 7fdc0e1e7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08112e00 con 0x7fdc0810d140 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.210+0000 7fdc0e1e7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08112c80 con 0x7fdc08103ce0 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.210+0000 7fdc0e1e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08112f80 con 0x7fdc08104600 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc077fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc08103ce0 0x7fdc08101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc0810d140 0x7fdc08102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc077fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc08103ce0 0x7fdc08101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:36884/0 (socket says 172.21.15.78:36884) 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc077fe640 1 -- 172.21.15.78:0/1791006438 learned_addr learned my addr 172.21.15.78:0/1791006438 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc06ffd640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc08104600 0x7fdc08101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc077fe640 1 -- 172.21.15.78:0/1791006438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc08104600 msgr2=0x7fdc08101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc077fe640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc08104600 0x7fdc08101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc077fe640 1 -- 172.21.15.78:0/1791006438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc0810d140 msgr2=0x7fdc08102410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc077fe640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc0810d140 0x7fdc08102410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc077fe640 1 -- 172.21.15.78:0/1791006438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc081a7700 con 0x7fdc08103ce0 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc07fff640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc0810d140 0x7fdc08102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:23:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc077fe640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc08103ce0 0x7fdc08101990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800e9f0 tx=0x7fdbf800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc04ff9640 1 -- 172.21.15.78:0/1791006438 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf800cd80 con 0x7fdc08103ce0 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/1791006438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc081a79f0 con 0x7fdc08103ce0 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc04ff9640 1 -- 172.21.15.78:0/1791006438 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbf80108e0 con 0x7fdc08103ce0 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc04ff9640 1 -- 172.21.15.78:0/1791006438 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8018980 con 0x7fdc08103ce0 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.211+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/1791006438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc081a7f00 con 0x7fdc08103ce0 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc04ff9640 1 -- 172.21.15.78:0/1791006438 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fdbf8010430 con 0x7fdc08103ce0 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc105bc740 1 Processor -- start 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc105bc740 1 -- start start 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc105bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc03294d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc105bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc0329a10 0x557fc0335460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc105bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 0x557fc0337d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc105bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557fc029fea0 con 0x557fc03359a0 2024-09-11T01:23:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc105bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557fc028f2d0 con 0x557fc0329a10 2024-09-11T01:23:12.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc105bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557fc02930e0 con 0x557fc03290f0 2024-09-11T01:23:12.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc06ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 0x557fc0337d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc06ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 0x557fc0337d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52352/0 (socket says 172.21.15.78:52352) 2024-09-11T01:23:12.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc06ffd640 1 -- 172.21.15.78:0/3212271773 learned_addr learned my addr 172.21.15.78:0/3212271773 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:12.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc07fff640 1 --2- 172.21.15.78:0/3212271773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc03294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc077fe640 1 --2- 172.21.15.78:0/3212271773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc0329a10 0x557fc0335460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc06ffd640 1 -- 172.21.15.78:0/3212271773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 msgr2=0x557fc03294d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:12.794 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:23:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc06ffd640 1 --2- 172.21.15.78:0/3212271773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc03294d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc06ffd640 1 -- 172.21.15.78:0/3212271773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc0329a10 msgr2=0x557fc0335460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc06ffd640 1 --2- 172.21.15.78:0/3212271773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc0329a10 0x557fc0335460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.214+0000 7fdc06ffd640 1 -- 172.21.15.78:0/3212271773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fbffed650 con 0x557fc03359a0 2024-09-11T01:23:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.215+0000 7fdc04ff9640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdbf00b5450 0x7fdbf00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.215+0000 7fdc07fff640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdbf00b5450 0x7fdbf00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.215+0000 7fdc06ffd640 1 --2- 172.21.15.78:0/3212271773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 0x557fc0337d90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc006010 tx=0x7fdbfc0040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2995c5aba9729407 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.215+0000 7fdc04ff9640 1 -- 172.21.15.78:0/1791006438 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7fdbf80e6350 con 0x7fdc08103ce0 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.215+0000 7fdbedffb640 1 -- 172.21.15.78:0/3212271773 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc035040 con 0x557fc03359a0 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.215+0000 7fdbedffb640 1 -- 172.21.15.78:0/3212271773 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbfc0054f0 con 0x557fc03359a0 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.215+0000 7fdbedffb640 1 -- 172.21.15.78:0/3212271773 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0035f0 con 0x557fc03359a0 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.216+0000 7fdc105bc740 1 -- 172.21.15.78:0/3212271773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 msgr2=0x557fc0337d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.216+0000 7fdc07fff640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdbf00b5450 0x7fdbf00b7910 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdc08103a50 tx=0x7fdc00009600 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.216+0000 7fdc105bc740 1 --2- 172.21.15.78:0/3212271773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 0x557fc0337d90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc006010 tx=0x7fdbfc0040d0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.216+0000 7fdc105bc740 1 -- 172.21.15.78:0/3212271773 shutdown_connections 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.216+0000 7fdc105bc740 1 --2- 172.21.15.78:0/3212271773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 0x557fc0337d90 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.216+0000 7fdc105bc740 1 --2- 172.21.15.78:0/3212271773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc0329a10 0x557fc0335460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.216+0000 7fdc105bc740 1 --2- 172.21.15.78:0/3212271773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc03294d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.216+0000 7fdc105bc740 1 -- 172.21.15.78:0/3212271773 >> 172.21.15.78:0/3212271773 conn(0x557fc0326f20 msgr2=0x557fc03280c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.216+0000 7fdc105bc740 1 -- 172.21.15.78:0/3212271773 shutdown_connections 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.216+0000 7fdc105bc740 1 -- 172.21.15.78:0/3212271773 wait complete. 2024-09-11T01:23:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.216+0000 7fdc105bc740 1 -- start start 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.216+0000 7fdc105bc740 1 Processor -- start 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc105bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc03352b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc105bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc0329a10 0x557fc0332700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc105bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03359a0 0x557fc0332d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc105bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557fc00f0700 con 0x557fc0329a10 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc105bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557fc022a160 con 0x557fc03359a0 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc105bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557fc03385c0 con 0x557fc03290f0 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc105bc740 10 client.0 _ll_register_callbacks cb 0x557fc02439e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc06ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc0329a10 0x557fc0332700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc06ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc0329a10 0x557fc0332700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52358/0 (socket says 172.21.15.78:52358) 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc06ffd640 1 -- 172.21.15.78:0/2132997484 learned_addr learned my addr 172.21.15.78:0/2132997484 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc07fff640 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03359a0 0x557fc0332d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc077fe640 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc03352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc06ffd640 1 -- 172.21.15.78:0/2132997484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 msgr2=0x557fc03352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc06ffd640 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc03352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc06ffd640 1 -- 172.21.15.78:0/2132997484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03359a0 msgr2=0x557fc0332d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc06ffd640 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03359a0 0x557fc0332d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc06ffd640 1 -- 172.21.15.78:0/2132997484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc0257c90 con 0x557fc0329a10 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc06ffd640 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc0329a10 0x557fc0332700 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc02a450 tx=0x7fdbfc0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07d511464ba53c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdbe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdbe6ffd640 1 -- 172.21.15.78:0/2132997484 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc052060 con 0x557fc0329a10 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdbe6ffd640 1 -- 172.21.15.78:0/2132997484 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7fdbfc004960 con 0x557fc0329a10 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.217+0000 7fdc105bc740 1 -- 172.21.15.78:0/2132997484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbfc0054f0 con 0x557fc0329a10 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.218+0000 7fdbe6ffd640 1 -- 172.21.15.78:0/2132997484 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fdbfc035070 con 0x557fc0329a10 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.218+0000 7fdc105bc740 10 client.19491 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.218+0000 7fdc105bc740 1 -- 172.21.15.78:0/2132997484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x557fc0343f20 con 0x557fc0329a10 2024-09-11T01:23:12.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.218+0000 7fdbe6ffd640 1 -- 172.21.15.78:0/2132997484 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbfc0054f0 con 0x557fc0329a10 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.218+0000 7fdbe6ffd640 1 -- 172.21.15.78:0/2132997484 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7fdbfc003af0 con 0x557fc0329a10 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.218+0000 7fdc105bc740 10 client.19491 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.218+0000 7fdc105bc740 10 client.19491 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.218+0000 7fdc105bc740 20 client.19491 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.218+0000 7fdc105bc740 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557fc0345270 0x557fc0347660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.218+0000 7fdc105bc740 4 client.19491 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.218+0000 7fdc105bc740 1 -- 172.21.15.78:0/2132997484 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fc02a1df0 con 0x557fc0345270 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.218+0000 7fdc077fe640 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557fc0345270 0x557fc0347660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.219+0000 7fdbe6ffd640 10 client.19491 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.219+0000 7fdc077fe640 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557fc0345270 0x557fc0347660 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a7d757c9b62e3619 server_cookie=f92918918d776045 in_seq=0 out_seq=0 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.219+0000 7fdbe6ffd640 1 -- 172.21.15.78:0/2132997484 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557fc02a1df0 con 0x557fc0345270 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.219+0000 7fdbe6ffd640 10 client.19491 handle_command_reply: tid=0 2024-09-11T01:23:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.219+0000 7fdc105bc740 1 client.19491 shutdown 2024-09-11T01:23:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.219+0000 7fdc105bc740 1 -- 172.21.15.78:0/2132997484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc0329a10 msgr2=0x557fc0332700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.219+0000 7fdc105bc740 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc0329a10 0x557fc0332700 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc02a450 tx=0x7fdbfc0045b0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.220+0000 7fdc105bc740 1 -- 172.21.15.78:0/2132997484 shutdown_connections 2024-09-11T01:23:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.220+0000 7fdc105bc740 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557fc0345270 0x557fc0347660 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.220+0000 7fdc105bc740 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03359a0 0x557fc0332d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.220+0000 7fdbe6ffd640 0 client.19491 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:23:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.220+0000 7fdc105bc740 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc0329a10 0x557fc0332700 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.220+0000 7fdc105bc740 1 --2- 172.21.15.78:0/2132997484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc03352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.220+0000 7fdc105bc740 1 -- 172.21.15.78:0/2132997484 >> 172.21.15.78:0/2132997484 conn(0x557fc0326f20 msgr2=0x557fc032cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.220+0000 7fdc105bc740 1 -- 172.21.15.78:0/2132997484 shutdown_connections 2024-09-11T01:23:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.220+0000 7fdc105bc740 1 -- 172.21.15.78:0/2132997484 wait complete. 2024-09-11T01:23:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.220+0000 7fdc105bc740 20 client.19491 trim_cache size 0 max 16384 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc105bc740 1 Processor -- start 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc105bc740 1 -- start start 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc105bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03290f0 0x557fc0331bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc105bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc0329a10 0x557fc0326760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc105bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 0x557fc0336d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc105bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081b0810 con 0x557fc03359a0 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc105bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081b2410 con 0x557fc03290f0 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc105bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081b0ee0 con 0x557fc0329a10 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 0x557fc0336d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc077fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03290f0 0x557fc0331bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 0x557fc0336d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52366/0 (socket says 172.21.15.78:52366) 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc07fff640 1 -- 172.21.15.78:0/3876069259 learned_addr learned my addr 172.21.15.78:0/3876069259 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:12.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc06ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc0329a10 0x557fc0326760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc07fff640 1 -- 172.21.15.78:0/3876069259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc0329a10 msgr2=0x557fc0326760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc07fff640 1 --2- 172.21.15.78:0/3876069259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc0329a10 0x557fc0326760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc07fff640 1 -- 172.21.15.78:0/3876069259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03290f0 msgr2=0x557fc0331bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc07fff640 1 --2- 172.21.15.78:0/3876069259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03290f0 0x557fc0331bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.233+0000 7fdc07fff640 1 -- 172.21.15.78:0/3876069259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc0338500 con 0x557fc03359a0 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.234+0000 7fdc07fff640 1 --2- 172.21.15.78:0/3876069259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 0x557fc0336d90 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fdc00013bc0 tx=0x7fdc0000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b343a2096da1b49 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.234+0000 7fdbed7fa640 1 -- 172.21.15.78:0/3876069259 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0001dcf0 con 0x557fc03359a0 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.234+0000 7fdbed7fa640 1 -- 172.21.15.78:0/3876069259 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0001c910 con 0x557fc03359a0 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.234+0000 7fdbed7fa640 1 -- 172.21.15.78:0/3876069259 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00027820 con 0x557fc03359a0 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.234+0000 7fdc105bc740 1 -- 172.21.15.78:0/3876069259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 msgr2=0x557fc0336d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.234+0000 7fdc105bc740 1 --2- 172.21.15.78:0/3876069259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 0x557fc0336d90 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fdc00013bc0 tx=0x7fdc0000b000 comp rx=0 tx=0).stop 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.234+0000 7fdc105bc740 1 -- 172.21.15.78:0/3876069259 shutdown_connections 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.235+0000 7fdc105bc740 1 --2- 172.21.15.78:0/3876069259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc03359a0 0x557fc0336d90 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.235+0000 7fdc105bc740 1 --2- 172.21.15.78:0/3876069259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc0329a10 0x557fc0326760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.235+0000 7fdc105bc740 1 --2- 172.21.15.78:0/3876069259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03290f0 0x557fc0331bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.235+0000 7fdc105bc740 1 -- 172.21.15.78:0/3876069259 >> 172.21.15.78:0/3876069259 conn(0x557fc0326f20 msgr2=0x557fc0330f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.235+0000 7fdc105bc740 1 -- 172.21.15.78:0/3876069259 shutdown_connections 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.235+0000 7fdc105bc740 1 -- 172.21.15.78:0/3876069259 wait complete. 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.235+0000 7fdc105bc740 1 -- start start 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.235+0000 7fdc105bc740 1 Processor -- start 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.235+0000 7fdc105bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc034af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.235+0000 7fdc105bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc0329a10 0x557fc0347430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc105bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03359a0 0x557fc0347ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc105bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081b1d40 con 0x557fc0329a10 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc105bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081b1670 con 0x557fc03359a0 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc105bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081ae810 con 0x557fc03290f0 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc105bc740 10 client.0 _ll_register_callbacks cb 0x7fdbf80e9cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc034af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc06ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc0329a10 0x557fc0347430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc034af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:52388/0 (socket says 172.21.15.78:52388) 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc077fe640 1 -- 172.21.15.78:0/4023366298 learned_addr learned my addr 172.21.15.78:0/4023366298 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc07fff640 1 --2- 172.21.15.78:0/4023366298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03359a0 0x557fc0347ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc077fe640 1 -- 172.21.15.78:0/4023366298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03359a0 msgr2=0x557fc0347ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc077fe640 1 --2- 172.21.15.78:0/4023366298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03359a0 0x557fc0347ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc077fe640 1 -- 172.21.15.78:0/4023366298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc0329a10 msgr2=0x557fc0347430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc077fe640 1 --2- 172.21.15.78:0/4023366298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc0329a10 0x557fc0347430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc077fe640 1 -- 172.21.15.78:0/4023366298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc03b1de0 con 0x557fc03290f0 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc077fe640 1 --2- 172.21.15.78:0/4023366298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc034af50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80ac7f0 tx=0x7fdbf8018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898ec50cd21cb3c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdbe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdbe6ffd640 1 -- 172.21.15.78:0/4023366298 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf80ac630 con 0x557fc03290f0 2024-09-11T01:23:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdbe6ffd640 1 -- 172.21.15.78:0/4023366298 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7fdbf80e3d00 con 0x557fc03290f0 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.236+0000 7fdc105bc740 1 -- 172.21.15.78:0/4023366298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc0001c910 con 0x557fc03290f0 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.237+0000 7fdbe6ffd640 1 -- 172.21.15.78:0/4023366298 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fdbf80abfd0 con 0x557fc03290f0 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.237+0000 7fdc105bc740 10 client.28471 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.237+0000 7fdc105bc740 1 -- 172.21.15.78:0/4023366298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x557fc03b2fb0 con 0x557fc03290f0 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.237+0000 7fdbe6ffd640 1 -- 172.21.15.78:0/4023366298 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0001c910 con 0x557fc03290f0 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.237+0000 7fdbe6ffd640 1 -- 172.21.15.78:0/4023366298 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7fdbf8018b20 con 0x557fc03290f0 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.237+0000 7fdc105bc740 10 client.28471 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.237+0000 7fdc105bc740 10 client.28471 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.237+0000 7fdc105bc740 20 client.28471 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.237+0000 7fdc105bc740 1 --2- 172.21.15.78:0/4023366298 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557fc03b3690 0x557fc03b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.237+0000 7fdc105bc740 4 client.28471 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.237+0000 7fdc105bc740 1 -- 172.21.15.78:0/4023366298 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557fc029d8a0 con 0x557fc03b3690 2024-09-11T01:23:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.238+0000 7fdc06ffd640 1 --2- 172.21.15.78:0/4023366298 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557fc03b3690 0x557fc03b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.238+0000 7fdc06ffd640 1 --2- 172.21.15.78:0/4023366298 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557fc03b3690 0x557fc03b5ab0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec84b586c744111f server_cookie=7d1809acf5e81b91 in_seq=0 out_seq=0 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.238+0000 7fdbe6ffd640 10 client.28471 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.238+0000 7fdbe6ffd640 1 -- 172.21.15.78:0/4023366298 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557fc029d8a0 con 0x557fc03b3690 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.238+0000 7fdbe6ffd640 10 client.28471 handle_command_reply: tid=0 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.238+0000 7fdc105bc740 1 client.28471 shutdown 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.238+0000 7fdc105bc740 1 -- 172.21.15.78:0/4023366298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 msgr2=0x557fc034af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.238+0000 7fdc105bc740 1 --2- 172.21.15.78:0/4023366298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc034af50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80ac7f0 tx=0x7fdbf8018420 comp rx=0 tx=0).stop 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.238+0000 7fdc105bc740 1 -- 172.21.15.78:0/4023366298 shutdown_connections 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdc105bc740 1 --2- 172.21.15.78:0/4023366298 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x557fc03b3690 0x557fc03b5ab0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdc105bc740 1 --2- 172.21.15.78:0/4023366298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557fc03359a0 0x557fc0347ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdc105bc740 1 --2- 172.21.15.78:0/4023366298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557fc0329a10 0x557fc0347430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdbe6ffd640 0 client.28471 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdc105bc740 1 --2- 172.21.15.78:0/4023366298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fc03290f0 0x557fc034af50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdc105bc740 1 -- 172.21.15.78:0/4023366298 >> 172.21.15.78:0/4023366298 conn(0x557fc0326f20 msgr2=0x557fc0333d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdc105bc740 1 -- 172.21.15.78:0/4023366298 shutdown_connections 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdc105bc740 1 -- 172.21.15.78:0/4023366298 wait complete. 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdc105bc740 20 client.28471 trim_cache size 0 max 16384 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/1791006438 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdbf00b5450 msgr2=0x7fdbf00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdc0e1e7640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdbf00b5450 0x7fdbf00b7910 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdc08103a50 tx=0x7fdc00009600 comp rx=0 tx=0).stop 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/1791006438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc08103ce0 msgr2=0x7fdc08101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.239+0000 7fdc0e1e7640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc08103ce0 0x7fdc08101990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800e9f0 tx=0x7fdbf800eeb0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.240+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/1791006438 shutdown_connections 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.240+0000 7fdc0e1e7640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdbf00b5450 0x7fdbf00b7910 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.240+0000 7fdc0e1e7640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc0810d140 0x7fdc08102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.240+0000 7fdc0e1e7640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc08104600 0x7fdc08101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.240+0000 7fdc0e1e7640 1 --2- 172.21.15.78:0/1791006438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc08103ce0 0x7fdc08101990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.240+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/1791006438 >> 172.21.15.78:0/1791006438 conn(0x7fdc080fd700 msgr2=0x7fdc080fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:12.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.240+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/1791006438 shutdown_connections 2024-09-11T01:23:12.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:12.240+0000 7fdc0e1e7640 1 -- 172.21.15.78:0/1791006438 wait complete. 2024-09-11T01:23:12.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 406360 31.54 MB/sec execute 558 sec latency 155.251 ms 2024-09-11T01:23:13.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 407049 31.56 MB/sec execute 559 sec latency 97.048 ms 2024-09-11T01:23:13.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:13 smithi110 ceph-mon[34712]: pgmap v1089: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 799 op/s 2024-09-11T01:23:13.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:13 smithi112 ceph-mon[35384]: pgmap v1089: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 799 op/s 2024-09-11T01:23:13.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:13 smithi078 ceph-mon[31315]: pgmap v1089: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 799 op/s 2024-09-11T01:23:14.089 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:23:14.408 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:23:14.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 407686 31.56 MB/sec execute 560 sec latency 92.801 ms 2024-09-11T01:23:15.016 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:23:15.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.962+0000 7fabb4dac640 1 -- 172.21.15.78:0/2164123452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0108020 msgr2=0x7fabb010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:15.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.962+0000 7fabb4dac640 1 --2- 172.21.15.78:0/2164123452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0108020 0x7fabb010e820 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7faba8009ca0 tx=0x7faba802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.962+0000 7fabb4dac640 1 -- 172.21.15.78:0/2164123452 shutdown_connections 2024-09-11T01:23:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.962+0000 7fabb4dac640 1 --2- 172.21.15.78:0/2164123452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0108020 0x7fabb010e820 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.962+0000 7fabb4dac640 1 --2- 172.21.15.78:0/2164123452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb01076a0 0x7fabb0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.962+0000 7fabb4dac640 1 --2- 172.21.15.78:0/2164123452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb0101c90 0x7fabb0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.962+0000 7fabb4dac640 1 -- 172.21.15.78:0/2164123452 >> 172.21.15.78:0/2164123452 conn(0x7fabb00fd700 msgr2=0x7fabb00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.963+0000 7fabb4dac640 1 -- 172.21.15.78:0/2164123452 shutdown_connections 2024-09-11T01:23:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.963+0000 7fabb4dac640 1 -- 172.21.15.78:0/2164123452 wait complete. 2024-09-11T01:23:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.963+0000 7fabb4dac640 1 Processor -- start 2024-09-11T01:23:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.963+0000 7fabb4dac640 1 -- start start 2024-09-11T01:23:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabb4dac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0101c90 0x7fabb0108810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabb4dac640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb01076a0 0x7fabb0108d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabb4dac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb0108020 0x7fabb0109290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabb4dac640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0111ca0 con 0x7fabb0101c90 2024-09-11T01:23:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabb4dac640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0111b20 con 0x7fabb01076a0 2024-09-11T01:23:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabb4dac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0111e20 con 0x7fabb0108020 2024-09-11T01:23:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabae59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0101c90 0x7fabb0108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabae59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0101c90 0x7fabb0108810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55064/0 (socket says 172.21.15.78:55064) 2024-09-11T01:23:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabae59c640 1 -- 172.21.15.78:0/2043269626 learned_addr learned my addr 172.21.15.78:0/2043269626 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabaed9d640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb0108020 0x7fabb0109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabadd9b640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb01076a0 0x7fabb0108d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabae59c640 1 -- 172.21.15.78:0/2043269626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb0108020 msgr2=0x7fabb0109290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabae59c640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb0108020 0x7fabb0109290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabae59c640 1 -- 172.21.15.78:0/2043269626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb01076a0 msgr2=0x7fabb0108d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:15.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabae59c640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb01076a0 0x7fabb0108d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.964+0000 7fabae59c640 1 -- 172.21.15.78:0/2043269626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb01aba30 con 0x7fabb0101c90 2024-09-11T01:23:15.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.965+0000 7fabae59c640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0101c90 0x7fabb0108810 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7faba000ea60 tx=0x7faba000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:15.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.965+0000 7fab9f7fe640 1 -- 172.21.15.78:0/2043269626 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba000cde0 con 0x7fabb0101c90 2024-09-11T01:23:15.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.965+0000 7fab9f7fe640 1 -- 172.21.15.78:0/2043269626 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba00035f0 con 0x7fabb0101c90 2024-09-11T01:23:15.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.965+0000 7fabb4dac640 1 -- 172.21.15.78:0/2043269626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb01abd20 con 0x7fabb0101c90 2024-09-11T01:23:15.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.965+0000 7fab9f7fe640 1 -- 172.21.15.78:0/2043269626 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0010690 con 0x7fabb0101c90 2024-09-11T01:23:15.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.966+0000 7fabb4dac640 1 -- 172.21.15.78:0/2043269626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb01ac250 con 0x7fabb0101c90 2024-09-11T01:23:15.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.969+0000 7fab9f7fe640 1 -- 172.21.15.78:0/2043269626 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7faba0004000 con 0x7fabb0101c90 2024-09-11T01:23:15.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.969+0000 7fabb7181740 1 Processor -- start 2024-09-11T01:23:15.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fabb7181740 1 -- start start 2024-09-11T01:23:15.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fab9f7fe640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab980b54d0 0x7fab980b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fabaed9d640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab980b54d0 0x7fab980b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fab9f7fe640 1 -- 172.21.15.78:0/2043269626 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7faba00e62e0 con 0x7fabb0101c90 2024-09-11T01:23:15.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fabb7181740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 0x56268490c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fabb7181740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 0x562684918840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fabb7181740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 0x56268491b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fabb7181740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562684882ea0 con 0x562684918d80 2024-09-11T01:23:15.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fabb7181740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5626848722d0 con 0x56268490cdf0 2024-09-11T01:23:15.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fabb7181740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5626848760e0 con 0x56268490c4d0 2024-09-11T01:23:15.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fabae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 0x56268490c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fabae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 0x56268490c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57216/0 (socket says 172.21.15.78:57216) 2024-09-11T01:23:15.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.970+0000 7fabae59c640 1 -- 172.21.15.78:0/2911386251 learned_addr learned my addr 172.21.15.78:0/2911386251 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:15.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fabadd9b640 1 --2- 172.21.15.78:0/2911386251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 0x562684918840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fabaed9d640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab980b54d0 0x7fab980b7990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7faba8006010 tx=0x7faba8005ef0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:15.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fabaed9d640 1 --2- 172.21.15.78:0/2911386251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 0x56268491b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fabae59c640 1 -- 172.21.15.78:0/2911386251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 msgr2=0x562684918840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:15.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fabae59c640 1 --2- 172.21.15.78:0/2911386251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 0x562684918840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fabae59c640 1 -- 172.21.15.78:0/2911386251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 msgr2=0x56268491b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:15.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fabae59c640 1 --2- 172.21.15.78:0/2911386251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 0x56268491b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fabae59c640 1 -- 172.21.15.78:0/2911386251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626845d0650 con 0x56268490c4d0 2024-09-11T01:23:15.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fabaed9d640 1 --2- 172.21.15.78:0/2911386251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 0x56268491b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:23:15.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fabadd9b640 1 --2- 172.21.15.78:0/2911386251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 0x562684918840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:23:15.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fabae59c640 1 --2- 172.21.15.78:0/2911386251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 0x56268490c8b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faba00ac4a0 tx=0x7faba00ac630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1e99e97f9d64a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:15.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fab9cff9640 1 -- 172.21.15.78:0/2911386251 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba00acea0 con 0x56268490c4d0 2024-09-11T01:23:15.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fab9cff9640 1 -- 172.21.15.78:0/2911386251 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba00b1070 con 0x56268490c4d0 2024-09-11T01:23:15.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.971+0000 7fab9cff9640 1 -- 172.21.15.78:0/2911386251 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba00a8a10 con 0x56268490c4d0 2024-09-11T01:23:15.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.972+0000 7fabb7181740 1 -- 172.21.15.78:0/2911386251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 msgr2=0x56268490c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:15.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.972+0000 7fabb7181740 1 --2- 172.21.15.78:0/2911386251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 0x56268490c8b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faba00ac4a0 tx=0x7faba00ac630 comp rx=0 tx=0).stop 2024-09-11T01:23:15.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.972+0000 7fabb7181740 1 -- 172.21.15.78:0/2911386251 shutdown_connections 2024-09-11T01:23:15.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.972+0000 7fabb7181740 1 --2- 172.21.15.78:0/2911386251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 0x56268491b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.972+0000 7fabb7181740 1 --2- 172.21.15.78:0/2911386251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 0x562684918840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.972+0000 7fabb7181740 1 --2- 172.21.15.78:0/2911386251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 0x56268490c8b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.972+0000 7fabb7181740 1 -- 172.21.15.78:0/2911386251 >> 172.21.15.78:0/2911386251 conn(0x56268490a300 msgr2=0x56268490b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:15.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.973+0000 7fabb7181740 1 -- 172.21.15.78:0/2911386251 shutdown_connections 2024-09-11T01:23:15.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.973+0000 7fabb7181740 1 -- 172.21.15.78:0/2911386251 wait complete. 2024-09-11T01:23:15.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.973+0000 7fabb7181740 1 -- start start 2024-09-11T01:23:15.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.973+0000 7fabb7181740 1 Processor -- start 2024-09-11T01:23:15.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabb7181740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 0x5626849186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabb7181740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490cdf0 0x562684915b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabb7181740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562684918d80 0x5626849161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabb7181740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5626846d3700 con 0x56268490c4d0 2024-09-11T01:23:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabb7181740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56268480d160 con 0x562684918d80 2024-09-11T01:23:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabb7181740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56268491b9a0 con 0x56268490cdf0 2024-09-11T01:23:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabb7181740 10 client.0 _ll_register_callbacks cb 0x5626848269e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabae59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 0x5626849186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabae59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 0x5626849186f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55088/0 (socket says 172.21.15.78:55088) 2024-09-11T01:23:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabae59c640 1 -- 172.21.15.78:0/1524357420 learned_addr learned my addr 172.21.15.78:0/1524357420 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabadd9b640 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490cdf0 0x562684915b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabaed9d640 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562684918d80 0x5626849161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabae59c640 1 -- 172.21.15.78:0/1524357420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490cdf0 msgr2=0x562684915b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabae59c640 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490cdf0 0x562684915b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.974+0000 7fabae59c640 1 -- 172.21.15.78:0/1524357420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562684918d80 msgr2=0x5626849161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.975+0000 7fabae59c640 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562684918d80 0x5626849161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.975+0000 7fabae59c640 1 -- 172.21.15.78:0/1524357420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56268483ac90 con 0x56268490c4d0 2024-09-11T01:23:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.975+0000 7fabae59c640 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 0x5626849186f0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7faba00e6600 tx=0x7faba00ac370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8811360c23cb56d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.975+0000 7fab95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:23:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.975+0000 7fab95ffb640 1 -- 172.21.15.78:0/1524357420 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0097020 con 0x56268490c4d0 2024-09-11T01:23:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.975+0000 7fab95ffb640 1 -- 172.21.15.78:0/1524357420 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7faba00aba00 con 0x56268490c4d0 2024-09-11T01:23:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.975+0000 7fabb7181740 1 -- 172.21.15.78:0/1524357420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba00b1070 con 0x56268490c4d0 2024-09-11T01:23:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.975+0000 7fab95ffb640 1 -- 172.21.15.78:0/1524357420 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7faba00acea0 con 0x56268490c4d0 2024-09-11T01:23:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.976+0000 7fabb7181740 10 client.19509 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:15.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.976+0000 7fabb7181740 1 -- 172.21.15.78:0/1524357420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x562684927350 con 0x56268490c4d0 2024-09-11T01:23:15.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.976+0000 7fab95ffb640 1 -- 172.21.15.78:0/1524357420 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba00b1070 con 0x56268490c4d0 2024-09-11T01:23:15.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.976+0000 7fab95ffb640 1 -- 172.21.15.78:0/1524357420 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7faba00a8200 con 0x56268490c4d0 2024-09-11T01:23:15.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.976+0000 7fabb7181740 10 client.19509 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:15.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.976+0000 7fabb7181740 10 client.19509 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:23:15.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.976+0000 7fabb7181740 20 client.19509 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:15.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.976+0000 7fabb7181740 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5626849286a0 0x56268492aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.976+0000 7fabb7181740 4 client.19509 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:23:15.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.976+0000 7fabb7181740 1 -- 172.21.15.78:0/1524357420 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562684884df0 con 0x5626849286a0 2024-09-11T01:23:15.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.976+0000 7fabadd9b640 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5626849286a0 0x56268492aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.977+0000 7fab95ffb640 10 client.19509 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:23:15.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.977+0000 7fabadd9b640 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5626849286a0 0x56268492aa90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a40494d3417db36c server_cookie=f52d360081b7ab45 in_seq=0 out_seq=0 2024-09-11T01:23:15.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.980+0000 7fab95ffb640 1 -- 172.21.15.78:0/1524357420 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562684884df0 con 0x5626849286a0 2024-09-11T01:23:15.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.980+0000 7fab95ffb640 10 client.19509 handle_command_reply: tid=0 2024-09-11T01:23:15.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.980+0000 7fabb7181740 1 client.19509 shutdown 2024-09-11T01:23:15.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.980+0000 7fabb7181740 1 -- 172.21.15.78:0/1524357420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 msgr2=0x5626849186f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:15.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.980+0000 7fabb7181740 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 0x5626849186f0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7faba00e6600 tx=0x7faba00ac370 comp rx=0 tx=0).stop 2024-09-11T01:23:15.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.980+0000 7fabb7181740 1 -- 172.21.15.78:0/1524357420 shutdown_connections 2024-09-11T01:23:15.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.980+0000 7fabb7181740 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5626849286a0 0x56268492aa90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.980+0000 7fabb7181740 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562684918d80 0x5626849161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.980+0000 7fab95ffb640 0 client.19509 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:23:15.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.980+0000 7fabb7181740 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490cdf0 0x562684915b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.980+0000 7fabb7181740 1 --2- 172.21.15.78:0/1524357420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 0x5626849186f0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.980+0000 7fabb7181740 1 -- 172.21.15.78:0/1524357420 >> 172.21.15.78:0/1524357420 conn(0x56268490a300 msgr2=0x56268490fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.981+0000 7fabb7181740 1 -- 172.21.15.78:0/1524357420 shutdown_connections 2024-09-11T01:23:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.981+0000 7fabb7181740 1 -- 172.21.15.78:0/1524357420 wait complete. 2024-09-11T01:23:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.981+0000 7fabb7181740 20 client.19509 trim_cache size 0 max 16384 2024-09-11T01:23:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabb7181740 1 Processor -- start 2024-09-11T01:23:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabb7181740 1 -- start start 2024-09-11T01:23:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabb7181740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 0x562684924c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabb7181740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 0x56268490b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabb7181740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 0x562684915030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabb7181740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b4ca0 con 0x562684918d80 2024-09-11T01:23:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabb7181740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b68a0 con 0x56268490cdf0 2024-09-11T01:23:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabb7181740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b5370 con 0x56268490c4d0 2024-09-11T01:23:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 0x562684924c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabadd9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 0x56268490b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabaed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 0x562684915030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabadd9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 0x56268490b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:51334/0 (socket says 172.21.15.78:51334) 2024-09-11T01:23:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabaed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 0x562684915030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55104/0 (socket says 172.21.15.78:55104) 2024-09-11T01:23:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.997+0000 7fabadd9b640 1 -- 172.21.15.78:0/2507898081 learned_addr learned my addr 172.21.15.78:0/2507898081 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.998+0000 7fabaed9d640 1 -- 172.21.15.78:0/2507898081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 msgr2=0x562684924c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.998+0000 7fabaed9d640 1 --2- 172.21.15.78:0/2507898081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 0x562684924c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.998+0000 7fabaed9d640 1 -- 172.21.15.78:0/2507898081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 msgr2=0x56268490b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.998+0000 7fabaed9d640 1 --2- 172.21.15.78:0/2507898081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 0x56268490b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.998+0000 7fabaed9d640 1 -- 172.21.15.78:0/2507898081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268490cc20 con 0x562684918d80 2024-09-11T01:23:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.998+0000 7fabadd9b640 1 --2- 172.21.15.78:0/2507898081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 0x56268490b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:23:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.998+0000 7fabae59c640 1 --2- 172.21.15.78:0/2507898081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 0x562684924c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:23:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.998+0000 7fabaed9d640 1 --2- 172.21.15.78:0/2507898081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 0x562684915030 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7faba8005e30 tx=0x7faba803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5b9911f1cfca20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.998+0000 7fab97fff640 1 -- 172.21.15.78:0/2507898081 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80316d0 con 0x562684918d80 2024-09-11T01:23:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.998+0000 7fab97fff640 1 -- 172.21.15.78:0/2507898081 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba8003e20 con 0x562684918d80 2024-09-11T01:23:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.998+0000 7fab97fff640 1 -- 172.21.15.78:0/2507898081 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba802f660 con 0x562684918d80 2024-09-11T01:23:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.999+0000 7fabb7181740 1 -- 172.21.15.78:0/2507898081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 msgr2=0x562684915030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.999+0000 7fabb7181740 1 --2- 172.21.15.78:0/2507898081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 0x562684915030 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7faba8005e30 tx=0x7faba803f000 comp rx=0 tx=0).stop 2024-09-11T01:23:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.999+0000 7fabb7181740 1 -- 172.21.15.78:0/2507898081 shutdown_connections 2024-09-11T01:23:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.999+0000 7fabb7181740 1 --2- 172.21.15.78:0/2507898081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562684918d80 0x562684915030 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.999+0000 7fabb7181740 1 --2- 172.21.15.78:0/2507898081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56268490cdf0 0x56268490b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.999+0000 7fabb7181740 1 --2- 172.21.15.78:0/2507898081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490c4d0 0x562684924c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.999+0000 7fabb7181740 1 -- 172.21.15.78:0/2507898081 >> 172.21.15.78:0/2507898081 conn(0x56268490a300 msgr2=0x56268490b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.999+0000 7fabb7181740 1 -- 172.21.15.78:0/2507898081 shutdown_connections 2024-09-11T01:23:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:14.999+0000 7fabb7181740 1 -- 172.21.15.78:0/2507898081 wait complete. 2024-09-11T01:23:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabb7181740 1 -- start start 2024-09-11T01:23:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabb7181740 1 Processor -- start 2024-09-11T01:23:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabb7181740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 0x56268492a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabb7181740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490cdf0 0x56268492a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabb7181740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562684918d80 0x562684994200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabb7181740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b61d0 con 0x56268490c4d0 2024-09-11T01:23:15.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabb7181740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b5b00 con 0x562684918d80 2024-09-11T01:23:15.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabb7181740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b2ca0 con 0x56268490cdf0 2024-09-11T01:23:15.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabb7181740 10 client.0 _ll_register_callbacks cb 0x7faba4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:15.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabae59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 0x56268492a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabadd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490cdf0 0x56268492a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabae59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 0x56268492a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55112/0 (socket says 172.21.15.78:55112) 2024-09-11T01:23:15.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabae59c640 1 -- 172.21.15.78:0/2035462372 learned_addr learned my addr 172.21.15.78:0/2035462372 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:15.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.000+0000 7fabaed9d640 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562684918d80 0x562684994200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.001+0000 7fabae59c640 1 -- 172.21.15.78:0/2035462372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490cdf0 msgr2=0x56268492a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:15.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.001+0000 7fabae59c640 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490cdf0 0x56268492a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.001+0000 7fabae59c640 1 -- 172.21.15.78:0/2035462372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562684918d80 msgr2=0x562684994200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:15.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.001+0000 7fabae59c640 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562684918d80 0x562684994200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.001+0000 7fabae59c640 1 -- 172.21.15.78:0/2035462372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626849948a0 con 0x56268490c4d0 2024-09-11T01:23:15.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.001+0000 7fabaed9d640 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562684918d80 0x562684994200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:23:15.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.001+0000 7fabae59c640 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 0x56268492a240 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7faba00a9a10 tx=0x7faba0003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7b9635ad988589 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:15.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.001+0000 7fab95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:23:15.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fab95ffb640 1 -- 172.21.15.78:0/2035462372 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0097040 con 0x56268490c4d0 2024-09-11T01:23:15.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fab95ffb640 1 -- 172.21.15.78:0/2035462372 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7faba00cfaf0 con 0x56268490c4d0 2024-09-11T01:23:15.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fabb7181740 1 -- 172.21.15.78:0/2035462372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba8003e20 con 0x56268490c4d0 2024-09-11T01:23:15.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fab95ffb640 1 -- 172.21.15.78:0/2035462372 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7faba00b1040 con 0x56268490c4d0 2024-09-11T01:23:15.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fabb7181740 10 client.19515 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:15.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fabb7181740 1 -- 172.21.15.78:0/2035462372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x562684995b80 con 0x56268490c4d0 2024-09-11T01:23:15.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fab95ffb640 1 -- 172.21.15.78:0/2035462372 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba00cf030 con 0x56268490c4d0 2024-09-11T01:23:15.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fab95ffb640 1 -- 172.21.15.78:0/2035462372 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7faba00921a0 con 0x56268490c4d0 2024-09-11T01:23:15.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fabb7181740 10 client.19515 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:15.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fabb7181740 10 client.19515 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:23:15.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fabb7181740 20 client.19515 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:15.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fabb7181740 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5626849286a0 0x5626849981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:15.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fabb7181740 4 client.19515 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:23:15.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.002+0000 7fabb7181740 1 -- 172.21.15.78:0/2035462372 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7faba8003e20 con 0x5626849286a0 2024-09-11T01:23:15.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.003+0000 7fabadd9b640 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5626849286a0 0x5626849981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:15.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.003+0000 7fab95ffb640 10 client.19515 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:23:15.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.003+0000 7fabadd9b640 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5626849286a0 0x5626849981a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b78bcf41997c7f3 server_cookie=4458f88c48e8c92c in_seq=0 out_seq=0 2024-09-11T01:23:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.004+0000 7fab95ffb640 1 -- 172.21.15.78:0/2035462372 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7faba8003e20 con 0x5626849286a0 2024-09-11T01:23:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.004+0000 7fab95ffb640 10 client.19515 handle_command_reply: tid=0 2024-09-11T01:23:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.004+0000 7fabb7181740 1 client.19515 shutdown 2024-09-11T01:23:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.004+0000 7fabb7181740 1 -- 172.21.15.78:0/2035462372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 msgr2=0x56268492a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.004+0000 7fabb7181740 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 0x56268492a240 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7faba00a9a10 tx=0x7faba0003e90 comp rx=0 tx=0).stop 2024-09-11T01:23:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.004+0000 7fabb7181740 1 -- 172.21.15.78:0/2035462372 shutdown_connections 2024-09-11T01:23:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.004+0000 7fabb7181740 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x5626849286a0 0x5626849981a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.004+0000 7fabb7181740 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562684918d80 0x562684994200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.005+0000 7fab95ffb640 0 client.19515 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:23:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.005+0000 7fabb7181740 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56268490cdf0 0x56268492a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.005+0000 7fabb7181740 1 --2- 172.21.15.78:0/2035462372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268490c4d0 0x56268492a240 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.005+0000 7fabb7181740 1 -- 172.21.15.78:0/2035462372 >> 172.21.15.78:0/2035462372 conn(0x56268490a300 msgr2=0x56268490b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.005+0000 7fabb7181740 1 -- 172.21.15.78:0/2035462372 shutdown_connections 2024-09-11T01:23:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.005+0000 7fabb7181740 1 -- 172.21.15.78:0/2035462372 wait complete. 2024-09-11T01:23:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.005+0000 7fabb7181740 20 client.19515 trim_cache size 0 max 16384 2024-09-11T01:23:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.006+0000 7fabb4dac640 1 -- 172.21.15.78:0/2043269626 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab980b54d0 msgr2=0x7fab980b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.006+0000 7fabb4dac640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab980b54d0 0x7fab980b7990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7faba8006010 tx=0x7faba8005ef0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.006+0000 7fabb4dac640 1 -- 172.21.15.78:0/2043269626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0101c90 msgr2=0x7fabb0108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.006+0000 7fabb4dac640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0101c90 0x7fabb0108810 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7faba000ea60 tx=0x7faba000ef20 comp rx=0 tx=0).stop 2024-09-11T01:23:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.006+0000 7fabb4dac640 1 -- 172.21.15.78:0/2043269626 shutdown_connections 2024-09-11T01:23:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.006+0000 7fabb4dac640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fab980b54d0 0x7fab980b7990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.006+0000 7fabb4dac640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb0108020 0x7fabb0109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.006+0000 7fabb4dac640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb01076a0 0x7fabb0108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.006+0000 7fabb4dac640 1 --2- 172.21.15.78:0/2043269626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0101c90 0x7fabb0108810 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.006+0000 7fabb4dac640 1 -- 172.21.15.78:0/2043269626 >> 172.21.15.78:0/2043269626 conn(0x7fabb00fd700 msgr2=0x7fabb006ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.006+0000 7fabb4dac640 1 -- 172.21.15.78:0/2043269626 shutdown_connections 2024-09-11T01:23:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:15.006+0000 7fabb4dac640 1 -- 172.21.15.78:0/2043269626 wait complete. 2024-09-11T01:23:15.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 408267 31.56 MB/sec execute 561 sec latency 78.387 ms 2024-09-11T01:23:15.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:15 smithi110 ceph-mon[34712]: pgmap v1090: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 728 op/s 2024-09-11T01:23:15.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:15 smithi078 ceph-mon[31315]: pgmap v1090: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 728 op/s 2024-09-11T01:23:15.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:15 smithi112 ceph-mon[35384]: pgmap v1090: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 728 op/s 2024-09-11T01:23:16.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 408775 31.55 MB/sec execute 562 sec latency 245.896 ms 2024-09-11T01:23:17.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 409269 31.54 MB/sec execute 563 sec latency 209.065 ms 2024-09-11T01:23:17.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:17 smithi110 ceph-mon[34712]: pgmap v1091: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 603 op/s 2024-09-11T01:23:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:17 smithi112 ceph-mon[35384]: pgmap v1091: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 603 op/s 2024-09-11T01:23:17.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:17 smithi078 ceph-mon[31315]: pgmap v1091: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 603 op/s 2024-09-11T01:23:18.439 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:23:18.785 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:23:18.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 409792 31.53 MB/sec execute 564 sec latency 141.000 ms 2024-09-11T01:23:19.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.390+0000 7f94affc9640 1 -- 172.21.15.78:0/2527676647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a810b370 msgr2=0x7f94a8111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:19.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.390+0000 7f94affc9640 1 --2- 172.21.15.78:0/2527676647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a810b370 0x7f94a8111b70 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009900 tx=0x7f94a002a410 comp rx=0 tx=0).stop 2024-09-11T01:23:19.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.390+0000 7f94affc9640 1 -- 172.21.15.78:0/2527676647 shutdown_connections 2024-09-11T01:23:19.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.390+0000 7f94affc9640 1 --2- 172.21.15.78:0/2527676647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a810b370 0x7f94a8111b70 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.390+0000 7f94affc9640 1 --2- 172.21.15.78:0/2527676647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94a810a9f0 0x7f94a810ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.390+0000 7f94affc9640 1 --2- 172.21.15.78:0/2527676647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94a8104fe0 0x7f94a81053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.390+0000 7f94affc9640 1 -- 172.21.15.78:0/2527676647 >> 172.21.15.78:0/2527676647 conn(0x7f94a8100a10 msgr2=0x7f94a8102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:19.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.390+0000 7f94affc9640 1 -- 172.21.15.78:0/2527676647 shutdown_connections 2024-09-11T01:23:19.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.390+0000 7f94affc9640 1 -- 172.21.15.78:0/2527676647 wait complete. 2024-09-11T01:23:19.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.391+0000 7f94affc9640 1 Processor -- start 2024-09-11T01:23:19.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.391+0000 7f94affc9640 1 -- start start 2024-09-11T01:23:19.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.391+0000 7f94affc9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94a8104fe0 0x7f94a8111870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.391+0000 7f94affc9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a810a9f0 0x7f94a810eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.391+0000 7f94affc9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94a810b370 0x7f94a810f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.391+0000 7f94affc9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8114fa0 con 0x7f94a810a9f0 2024-09-11T01:23:19.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.391+0000 7f94affc9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8114e20 con 0x7f94a810b370 2024-09-11T01:23:19.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.391+0000 7f94affc9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8115120 con 0x7f94a8104fe0 2024-09-11T01:23:19.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.392+0000 7f94ae566640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94a810b370 0x7f94a810f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.392+0000 7f94ae566640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94a810b370 0x7f94a810f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:51352/0 (socket says 172.21.15.78:51352) 2024-09-11T01:23:19.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.392+0000 7f94ae566640 1 -- 172.21.15.78:0/207965385 learned_addr learned my addr 172.21.15.78:0/207965385 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:19.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.392+0000 7f94add65640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94a8104fe0 0x7f94a8111870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.392+0000 7f94ae566640 1 -- 172.21.15.78:0/207965385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94a8104fe0 msgr2=0x7f94a8111870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:19.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.392+0000 7f94ae566640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94a8104fe0 0x7f94a8111870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.392+0000 7f94ae566640 1 -- 172.21.15.78:0/207965385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a810a9f0 msgr2=0x7f94a810eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:19.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.392+0000 7f94ad564640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a810a9f0 0x7f94a810eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.392+0000 7f94ae566640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a810a9f0 0x7f94a810eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.392+0000 7f94ae566640 1 -- 172.21.15.78:0/207965385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a810f8c0 con 0x7f94a810b370 2024-09-11T01:23:19.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.392+0000 7f94ad564640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a810a9f0 0x7f94a810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:23:19.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.393+0000 7f94ae566640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94a810b370 0x7f94a810f000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94a0004770 tx=0x7f94a0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:19.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.393+0000 7f949effd640 1 -- 172.21.15.78:0/207965385 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a003f030 con 0x7f94a810b370 2024-09-11T01:23:19.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.393+0000 7f949effd640 1 -- 172.21.15.78:0/207965385 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a0004440 con 0x7f94a810b370 2024-09-11T01:23:19.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.393+0000 7f94affc9640 1 -- 172.21.15.78:0/207965385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a81abd40 con 0x7f94a810b370 2024-09-11T01:23:19.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.393+0000 7f949effd640 1 -- 172.21.15.78:0/207965385 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0035070 con 0x7f94a810b370 2024-09-11T01:23:19.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.393+0000 7f94affc9640 1 -- 172.21.15.78:0/207965385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81ac1e0 con 0x7f94a810b370 2024-09-11T01:23:19.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.396+0000 7f94b239e740 1 Processor -- start 2024-09-11T01:23:19.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.396+0000 7f949effd640 1 -- 172.21.15.78:0/207965385 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f94a004c060 con 0x7f94a810b370 2024-09-11T01:23:19.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.397+0000 7f94b239e740 1 -- start start 2024-09-11T01:23:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.397+0000 7f94b239e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a800f0 0x55faa5a804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.397+0000 7f94b239e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a80a10 0x55faa5a8c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.397+0000 7f94b239e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a8c9a0 0x55faa5a8ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.397+0000 7f94b239e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55faa59f6ea0 con 0x55faa5a80a10 2024-09-11T01:23:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.397+0000 7f94b239e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55faa59e62d0 con 0x55faa5a8c9a0 2024-09-11T01:23:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.397+0000 7f94b239e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55faa59ea0e0 con 0x55faa5a800f0 2024-09-11T01:23:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.397+0000 7f94add65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a80a10 0x55faa5a8c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.397+0000 7f94add65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a80a10 0x55faa5a8c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55126/0 (socket says 172.21.15.78:55126) 2024-09-11T01:23:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.397+0000 7f94add65640 1 -- 172.21.15.78:0/3353306714 learned_addr learned my addr 172.21.15.78:0/3353306714 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.397+0000 7f94ad564640 1 --2- 172.21.15.78:0/3353306714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a800f0 0x55faa5a804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.397+0000 7f94ad564640 1 --2- 172.21.15.78:0/3353306714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a8c9a0 0x55faa5a8ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f94add65640 1 -- 172.21.15.78:0/3353306714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a800f0 msgr2=0x55faa5a804d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f94add65640 1 --2- 172.21.15.78:0/3353306714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a800f0 0x55faa5a804d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f94add65640 1 -- 172.21.15.78:0/3353306714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a8c9a0 msgr2=0x55faa5a8ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f94add65640 1 --2- 172.21.15.78:0/3353306714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a8c9a0 0x55faa5a8ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f94add65640 1 -- 172.21.15.78:0/3353306714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa5744650 con 0x55faa5a80a10 2024-09-11T01:23:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f94ad564640 1 --2- 172.21.15.78:0/3353306714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a8c9a0 0x55faa5a8ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:23:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f94add65640 1 --2- 172.21.15.78:0/3353306714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a80a10 0x55faa5a8c460 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f94a40114f0 tx=0x7f94a40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d63a49bbc2c885 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f949effd640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94940b54d0 0x7f94940b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f94ad564640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94940b54d0 0x7f94940b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f9493fff640 1 -- 172.21.15.78:0/3353306714 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a401e070 con 0x55faa5a80a10 2024-09-11T01:23:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f9493fff640 1 -- 172.21.15.78:0/3353306714 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a400fad0 con 0x55faa5a80a10 2024-09-11T01:23:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f9493fff640 1 -- 172.21.15.78:0/3353306714 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4019b10 con 0x55faa5a80a10 2024-09-11T01:23:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.398+0000 7f949effd640 1 -- 172.21.15.78:0/207965385 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7f94a0105ee0 con 0x7f94a810b370 2024-09-11T01:23:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.399+0000 7f94b239e740 1 -- 172.21.15.78:0/3353306714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a80a10 msgr2=0x55faa5a8c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.399+0000 7f94b239e740 1 --2- 172.21.15.78:0/3353306714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a80a10 0x55faa5a8c460 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f94a40114f0 tx=0x7f94a40119b0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.400+0000 7f94ad564640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94940b54d0 0x7f94940b7990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f94980058c0 tx=0x7f9498005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.400+0000 7f94b239e740 1 -- 172.21.15.78:0/3353306714 shutdown_connections 2024-09-11T01:23:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.400+0000 7f94b239e740 1 --2- 172.21.15.78:0/3353306714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a8c9a0 0x55faa5a8ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.400+0000 7f94b239e740 1 --2- 172.21.15.78:0/3353306714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a80a10 0x55faa5a8c460 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.400+0000 7f94b239e740 1 --2- 172.21.15.78:0/3353306714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a800f0 0x55faa5a804d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.400+0000 7f94b239e740 1 -- 172.21.15.78:0/3353306714 >> 172.21.15.78:0/3353306714 conn(0x55faa5a7df20 msgr2=0x55faa5a7f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.400+0000 7f94b239e740 1 -- 172.21.15.78:0/3353306714 shutdown_connections 2024-09-11T01:23:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.400+0000 7f94b239e740 1 -- 172.21.15.78:0/3353306714 wait complete. 2024-09-11T01:23:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.400+0000 7f94b239e740 1 -- start start 2024-09-11T01:23:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94b239e740 1 Processor -- start 2024-09-11T01:23:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94b239e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a800f0 0x55faa5a8c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94b239e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a80a10 0x55faa5a896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94b239e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a8c9a0 0x55faa5a89d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94b239e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55faa5847700 con 0x55faa5a800f0 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94b239e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55faa5981160 con 0x55faa5a8c9a0 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94b239e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55faa5a8f620 con 0x55faa5a80a10 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94b239e740 10 client.0 _ll_register_callbacks cb 0x55faa599a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94add65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a800f0 0x55faa5a8c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94add65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a800f0 0x55faa5a8c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55140/0 (socket says 172.21.15.78:55140) 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94add65640 1 -- 172.21.15.78:0/2111994927 learned_addr learned my addr 172.21.15.78:0/2111994927 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94ad564640 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a80a10 0x55faa5a896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.401+0000 7f94ae566640 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a8c9a0 0x55faa5a89d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.402+0000 7f94add65640 1 -- 172.21.15.78:0/2111994927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a80a10 msgr2=0x55faa5a896e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.402+0000 7f94add65640 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a80a10 0x55faa5a896e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.402+0000 7f94add65640 1 -- 172.21.15.78:0/2111994927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a8c9a0 msgr2=0x55faa5a89d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.402+0000 7f94add65640 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a8c9a0 0x55faa5a89d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.402+0000 7f94add65640 1 -- 172.21.15.78:0/2111994927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa59aec90 con 0x55faa5a800f0 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.402+0000 7f94add65640 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a800f0 0x55faa5a8c290 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f94a400ffd0 tx=0x7f94a4009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fb641ff4e80ae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.402+0000 7f94917fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.403+0000 7f94917fa640 1 -- 172.21.15.78:0/2111994927 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a401e040 con 0x55faa5a800f0 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.403+0000 7f94917fa640 1 -- 172.21.15.78:0/2111994927 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7f94a400f6a0 con 0x55faa5a800f0 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.403+0000 7f94b239e740 1 -- 172.21.15.78:0/2111994927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a400fad0 con 0x55faa5a800f0 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.403+0000 7f94917fa640 1 -- 172.21.15.78:0/2111994927 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f94a4032050 con 0x55faa5a800f0 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.403+0000 7f94b239e740 10 client.19524 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.403+0000 7f94b239e740 1 -- 172.21.15.78:0/2111994927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55faa5a9af80 con 0x55faa5a800f0 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.404+0000 7f94917fa640 1 -- 172.21.15.78:0/2111994927 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a400fad0 con 0x55faa5a800f0 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.404+0000 7f94917fa640 1 -- 172.21.15.78:0/2111994927 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f94a4005040 con 0x55faa5a800f0 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.404+0000 7f94b239e740 10 client.19524 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.404+0000 7f94b239e740 10 client.19524 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:23:19.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.404+0000 7f94b239e740 20 client.19524 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.404+0000 7f94b239e740 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55faa5a9c2d0 0x55faa5a9e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.404+0000 7f94b239e740 4 client.19524 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:23:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.404+0000 7f94b239e740 1 -- 172.21.15.78:0/2111994927 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faa59f8df0 con 0x55faa5a9c2d0 2024-09-11T01:23:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.404+0000 7f94ad564640 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55faa5a9c2d0 0x55faa5a9e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.406+0000 7f94ad564640 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55faa5a9c2d0 0x55faa5a9e6c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=695518697e18bf49 server_cookie=15efc25c87c3dc64 in_seq=0 out_seq=0 2024-09-11T01:23:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.406+0000 7f94917fa640 10 client.19524 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:23:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.407+0000 7f94917fa640 1 -- 172.21.15.78:0/2111994927 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55faa59f8df0 con 0x55faa5a9c2d0 2024-09-11T01:23:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.407+0000 7f94917fa640 10 client.19524 handle_command_reply: tid=0 2024-09-11T01:23:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.408+0000 7f94b239e740 1 client.19524 shutdown 2024-09-11T01:23:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.408+0000 7f94b239e740 1 -- 172.21.15.78:0/2111994927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a800f0 msgr2=0x55faa5a8c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.408+0000 7f94b239e740 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a800f0 0x55faa5a8c290 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f94a400ffd0 tx=0x7f94a4009c80 comp rx=0 tx=0).stop 2024-09-11T01:23:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.408+0000 7f94b239e740 1 -- 172.21.15.78:0/2111994927 shutdown_connections 2024-09-11T01:23:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.408+0000 7f94b239e740 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a8c9a0 0x55faa5a89d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.408+0000 7f94b239e740 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a80a10 0x55faa5a896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.408+0000 7f94b239e740 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55faa5a9c2d0 0x55faa5a9e6c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.409+0000 7f94917fa640 0 client.19524 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:23:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.409+0000 7f94b239e740 1 --2- 172.21.15.78:0/2111994927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a800f0 0x55faa5a8c290 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.409+0000 7f94b239e740 1 -- 172.21.15.78:0/2111994927 >> 172.21.15.78:0/2111994927 conn(0x55faa5a7df20 msgr2=0x55faa5a83ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.409+0000 7f94b239e740 1 -- 172.21.15.78:0/2111994927 shutdown_connections 2024-09-11T01:23:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.409+0000 7f94b239e740 1 -- 172.21.15.78:0/2111994927 wait complete. 2024-09-11T01:23:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.409+0000 7f94b239e740 20 client.19524 trim_cache size 0 max 16384 2024-09-11T01:23:19.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.433+0000 7f94b239e740 1 Processor -- start 2024-09-11T01:23:19.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.433+0000 7f94b239e740 1 -- start start 2024-09-11T01:23:19.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.433+0000 7f94b239e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a800f0 0x55faa5a7ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.433+0000 7f94b239e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a80a10 0x55faa5a868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.433+0000 7f94b239e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a8c9a0 0x55faa5a86e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.433+0000 7f94b239e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b4ca0 con 0x55faa5a80a10 2024-09-11T01:23:19.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.433+0000 7f94b239e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b68a0 con 0x55faa5a800f0 2024-09-11T01:23:19.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.433+0000 7f94b239e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b5370 con 0x55faa5a8c9a0 2024-09-11T01:23:19.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.434+0000 7f94ae566640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a8c9a0 0x55faa5a86e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.434+0000 7f94add65640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a800f0 0x55faa5a7ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.434+0000 7f94ae566640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a8c9a0 0x55faa5a86e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57304/0 (socket says 172.21.15.78:57304) 2024-09-11T01:23:19.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.434+0000 7f94ae566640 1 -- 172.21.15.78:0/4248693621 learned_addr learned my addr 172.21.15.78:0/4248693621 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:19.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.434+0000 7f94ad564640 1 --2- 172.21.15.78:0/4248693621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a80a10 0x55faa5a868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.434+0000 7f94ae566640 1 -- 172.21.15.78:0/4248693621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a800f0 msgr2=0x55faa5a7ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:19.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.434+0000 7f94ae566640 1 --2- 172.21.15.78:0/4248693621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a800f0 0x55faa5a7ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.434+0000 7f94ae566640 1 -- 172.21.15.78:0/4248693621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a80a10 msgr2=0x55faa5a868f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:19.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.434+0000 7f94ae566640 1 --2- 172.21.15.78:0/4248693621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a80a10 0x55faa5a868f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.434+0000 7f94ae566640 1 -- 172.21.15.78:0/4248693621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa5a84400 con 0x55faa5a8c9a0 2024-09-11T01:23:19.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.434+0000 7f94ae566640 1 --2- 172.21.15.78:0/4248693621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a8c9a0 0x55faa5a86e30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f94a0103c70 tx=0x7f94a00cbdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d10f20f12da544 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:19.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.435+0000 7f94937fe640 1 -- 172.21.15.78:0/4248693621 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0049040 con 0x55faa5a8c9a0 2024-09-11T01:23:19.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.435+0000 7f94937fe640 1 -- 172.21.15.78:0/4248693621 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a00ccaa0 con 0x55faa5a8c9a0 2024-09-11T01:23:19.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.435+0000 7f94937fe640 1 -- 172.21.15.78:0/4248693621 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a00b3040 con 0x55faa5a8c9a0 2024-09-11T01:23:19.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.435+0000 7f94b239e740 1 -- 172.21.15.78:0/4248693621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a8c9a0 msgr2=0x55faa5a86e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:19.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.435+0000 7f94b239e740 1 --2- 172.21.15.78:0/4248693621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a8c9a0 0x55faa5a86e30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f94a0103c70 tx=0x7f94a00cbdb0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.435+0000 7f94b239e740 1 -- 172.21.15.78:0/4248693621 shutdown_connections 2024-09-11T01:23:19.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.435+0000 7f94b239e740 1 --2- 172.21.15.78:0/4248693621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a8c9a0 0x55faa5a86e30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.435+0000 7f94b239e740 1 --2- 172.21.15.78:0/4248693621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a80a10 0x55faa5a868f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.435+0000 7f94b239e740 1 --2- 172.21.15.78:0/4248693621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a800f0 0x55faa5a7ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.435+0000 7f94b239e740 1 -- 172.21.15.78:0/4248693621 >> 172.21.15.78:0/4248693621 conn(0x55faa5a7df20 msgr2=0x55faa5a7c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:19.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.436+0000 7f94b239e740 1 -- 172.21.15.78:0/4248693621 shutdown_connections 2024-09-11T01:23:19.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.436+0000 7f94b239e740 1 -- 172.21.15.78:0/4248693621 wait complete. 2024-09-11T01:23:19.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.436+0000 7f94b239e740 1 -- start start 2024-09-11T01:23:19.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.436+0000 7f94b239e740 1 Processor -- start 2024-09-11T01:23:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94b239e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a800f0 0x55faa5a94ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94b239e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a80a10 0x55faa5a91520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94b239e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a8c9a0 0x55faa5a91a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94ad564640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a80a10 0x55faa5a91520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94add65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a800f0 0x55faa5a94ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94ad564640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a80a10 0x55faa5a91520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:51376/0 (socket says 172.21.15.78:51376) 2024-09-11T01:23:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94ad564640 1 -- 172.21.15.78:0/2869947670 learned_addr learned my addr 172.21.15.78:0/2869947670 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94ae566640 1 --2- 172.21.15.78:0/2869947670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a8c9a0 0x55faa5a91a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94b239e740 1 -- 172.21.15.78:0/2869947670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b61d0 con 0x55faa5a800f0 2024-09-11T01:23:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94b239e740 1 -- 172.21.15.78:0/2869947670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b5b00 con 0x55faa5a80a10 2024-09-11T01:23:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94b239e740 1 -- 172.21.15.78:0/2869947670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b2ca0 con 0x55faa5a8c9a0 2024-09-11T01:23:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94b239e740 10 client.0 _ll_register_callbacks cb 0x7f94a401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94ad564640 1 -- 172.21.15.78:0/2869947670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a8c9a0 msgr2=0x55faa5a91a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94ad564640 1 --2- 172.21.15.78:0/2869947670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a8c9a0 0x55faa5a91a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94ad564640 1 -- 172.21.15.78:0/2869947670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a800f0 msgr2=0x55faa5a94ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:19.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94ad564640 1 --2- 172.21.15.78:0/2869947670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a800f0 0x55faa5a94ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.437+0000 7f94ad564640 1 -- 172.21.15.78:0/2869947670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa5a921f0 con 0x55faa5a80a10 2024-09-11T01:23:19.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.438+0000 7f94ad564640 1 --2- 172.21.15.78:0/2869947670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a80a10 0x55faa5a91520 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9498009960 tx=0x7f94980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fadf4db6c954ef5e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:19.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.438+0000 7f94917fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:23:19.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.438+0000 7f94917fa640 1 -- 172.21.15.78:0/2869947670 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498004220 con 0x55faa5a80a10 2024-09-11T01:23:19.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.438+0000 7f94917fa640 1 -- 172.21.15.78:0/2869947670 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7f94980043c0 con 0x55faa5a80a10 2024-09-11T01:23:19.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.438+0000 7f94b239e740 1 -- 172.21.15.78:0/2869947670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a00ccaa0 con 0x55faa5a80a10 2024-09-11T01:23:19.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.438+0000 7f94917fa640 1 -- 172.21.15.78:0/2869947670 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f9498021920 con 0x55faa5a80a10 2024-09-11T01:23:19.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.438+0000 7f94b239e740 10 client.28556 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:19.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.438+0000 7f94b239e740 1 -- 172.21.15.78:0/2869947670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55faa5b09f80 con 0x55faa5a80a10 2024-09-11T01:23:19.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.439+0000 7f94917fa640 1 -- 172.21.15.78:0/2869947670 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94980035f0 con 0x55faa5a80a10 2024-09-11T01:23:19.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.440+0000 7f94917fa640 1 -- 172.21.15.78:0/2869947670 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f9498030070 con 0x55faa5a80a10 2024-09-11T01:23:19.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.440+0000 7f94b239e740 10 client.28556 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:19.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.440+0000 7f94b239e740 10 client.28556 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:23:19.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.440+0000 7f94b239e740 20 client.28556 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:19.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.440+0000 7f94b239e740 1 --2- 172.21.15.78:0/2869947670 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55faa5b0a680 0x55faa5b0caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:19.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.440+0000 7f94b239e740 4 client.28556 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:23:19.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.440+0000 7f94b239e740 1 -- 172.21.15.78:0/2869947670 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f94a00ccaa0 con 0x55faa5b0a680 2024-09-11T01:23:19.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.440+0000 7f94add65640 1 --2- 172.21.15.78:0/2869947670 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55faa5b0a680 0x55faa5b0caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:19.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94917fa640 10 client.28556 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:23:19.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94add65640 1 --2- 172.21.15.78:0/2869947670 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55faa5b0a680 0x55faa5b0caa0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38dea249dd678506 server_cookie=fe089017d03b0df in_seq=0 out_seq=0 2024-09-11T01:23:19.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94917fa640 1 -- 172.21.15.78:0/2869947670 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f94a00ccaa0 con 0x55faa5b0a680 2024-09-11T01:23:19.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94917fa640 10 client.28556 handle_command_reply: tid=0 2024-09-11T01:23:19.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94b239e740 1 client.28556 shutdown 2024-09-11T01:23:19.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94b239e740 1 -- 172.21.15.78:0/2869947670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a80a10 msgr2=0x55faa5a91520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:19.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94b239e740 1 --2- 172.21.15.78:0/2869947670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a80a10 0x55faa5a91520 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9498009960 tx=0x7f94980046c0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.457 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:23:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94b239e740 1 -- 172.21.15.78:0/2869947670 shutdown_connections 2024-09-11T01:23:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94b239e740 1 --2- 172.21.15.78:0/2869947670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55faa5a8c9a0 0x55faa5a91a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94b239e740 1 --2- 172.21.15.78:0/2869947670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa5a80a10 0x55faa5a91520 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94b239e740 1 --2- 172.21.15.78:0/2869947670 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x55faa5b0a680 0x55faa5b0caa0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94b239e740 1 --2- 172.21.15.78:0/2869947670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55faa5a800f0 0x55faa5a94ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94917fa640 0 client.28556 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:23:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94b239e740 1 -- 172.21.15.78:0/2869947670 >> 172.21.15.78:0/2869947670 conn(0x55faa5a7df20 msgr2=0x55faa5a87b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:19.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94b239e740 1 -- 172.21.15.78:0/2869947670 shutdown_connections 2024-09-11T01:23:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.441+0000 7f94b239e740 1 -- 172.21.15.78:0/2869947670 wait complete. 2024-09-11T01:23:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.442+0000 7f94b239e740 20 client.28556 trim_cache size 0 max 16384 2024-09-11T01:23:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.442+0000 7f94affc9640 1 -- 172.21.15.78:0/207965385 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94940b54d0 msgr2=0x7f94940b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.442+0000 7f94affc9640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94940b54d0 0x7f94940b7990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f94980058c0 tx=0x7f9498005f90 comp rx=0 tx=0).stop 2024-09-11T01:23:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.442+0000 7f94affc9640 1 -- 172.21.15.78:0/207965385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94a810b370 msgr2=0x7f94a810f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:19.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.442+0000 7f94affc9640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94a810b370 0x7f94a810f000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94a0004770 tx=0x7f94a0004040 comp rx=0 tx=0).stop 2024-09-11T01:23:19.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.444+0000 7f94affc9640 1 -- 172.21.15.78:0/207965385 shutdown_connections 2024-09-11T01:23:19.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.444+0000 7f94affc9640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f94940b54d0 0x7f94940b7990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.444+0000 7f94affc9640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94a810b370 0x7f94a810f000 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.444+0000 7f94affc9640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a810a9f0 0x7f94a810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.444+0000 7f94affc9640 1 --2- 172.21.15.78:0/207965385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94a8104fe0 0x7f94a8111870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:19.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.444+0000 7f94affc9640 1 -- 172.21.15.78:0/207965385 >> 172.21.15.78:0/207965385 conn(0x7f94a8100a10 msgr2=0x7f94a8102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:19.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.444+0000 7f94affc9640 1 -- 172.21.15.78:0/207965385 shutdown_connections 2024-09-11T01:23:19.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:19.444+0000 7f94affc9640 1 -- 172.21.15.78:0/207965385 wait complete. 2024-09-11T01:23:19.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 410349 31.52 MB/sec execute 565 sec latency 109.004 ms 2024-09-11T01:23:19.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:19 smithi110 ceph-mon[34712]: pgmap v1092: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 582 op/s 2024-09-11T01:23:19.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:19 smithi112 ceph-mon[35384]: pgmap v1092: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 582 op/s 2024-09-11T01:23:19.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:19 smithi078 ceph-mon[31315]: pgmap v1092: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 582 op/s 2024-09-11T01:23:20.678 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:23:20.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 411053 31.55 MB/sec execute 566 sec latency 102.565 ms 2024-09-11T01:23:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:23:21.812 INFO:tasks.workunit.client.0.smithi078.stdout: 10 411274 31.52 MB/sec execute 567 sec latency 593.921 ms 2024-09-11T01:23:21.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:21 smithi110 ceph-mon[34712]: pgmap v1093: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 673 op/s 2024-09-11T01:23:21.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:21 smithi112 ceph-mon[35384]: pgmap v1093: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 673 op/s 2024-09-11T01:23:21.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:21 smithi078 ceph-mon[31315]: pgmap v1093: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 673 op/s 2024-09-11T01:23:22.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 411569 31.48 MB/sec execute 568 sec latency 276.383 ms 2024-09-11T01:23:23.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.142+0000 7f5a3affd640 1 -- 172.21.15.78:0/3804257114 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30030850 con 0x7f5a3c1023b0 2024-09-11T01:23:23.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.143+0000 7f5a42631640 1 -- 172.21.15.78:0/3804257114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a3c1023b0 msgr2=0x7f5a3c106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:23.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.143+0000 7f5a42631640 1 --2- 172.21.15.78:0/3804257114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a3c1023b0 0x7f5a3c106880 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009860 tx=0x7f5a3002a340 comp rx=0 tx=0).stop 2024-09-11T01:23:23.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.143+0000 7f5a42631640 1 -- 172.21.15.78:0/3804257114 shutdown_connections 2024-09-11T01:23:23.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.143+0000 7f5a42631640 1 --2- 172.21.15.78:0/3804257114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c106ed0 0x7f5a3c107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.143+0000 7f5a42631640 1 --2- 172.21.15.78:0/3804257114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a3c1023b0 0x7f5a3c106880 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.143+0000 7f5a42631640 1 --2- 172.21.15.78:0/3804257114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a3c101a90 0x7f5a3c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.143+0000 7f5a42631640 1 -- 172.21.15.78:0/3804257114 >> 172.21.15.78:0/3804257114 conn(0x7f5a3c0fd700 msgr2=0x7f5a3c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:23.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.143+0000 7f5a42631640 1 -- 172.21.15.78:0/3804257114 shutdown_connections 2024-09-11T01:23:23.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.143+0000 7f5a42631640 1 -- 172.21.15.78:0/3804257114 wait complete. 2024-09-11T01:23:23.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.145+0000 7f5a42631640 1 Processor -- start 2024-09-11T01:23:23.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.146+0000 7f5a42631640 1 -- start start 2024-09-11T01:23:23.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.146+0000 7f5a42631640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a3c101a90 0x7f5a3c0747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.146+0000 7f5a42631640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c1023b0 0x7f5a3c074d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.146+0000 7f5a42631640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a3c106ed0 0x7f5a3c06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.146+0000 7f5a42631640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c119390 con 0x7f5a3c106ed0 2024-09-11T01:23:23.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.146+0000 7f5a42631640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c119210 con 0x7f5a3c1023b0 2024-09-11T01:23:23.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.146+0000 7f5a42631640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c119510 con 0x7f5a3c101a90 2024-09-11T01:23:23.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.146+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a3c101a90 0x7f5a3c0747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.146+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a3c101a90 0x7f5a3c0747f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57328/0 (socket says 172.21.15.78:57328) 2024-09-11T01:23:23.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.146+0000 7f5a3bfff640 1 -- 172.21.15.78:0/150111865 learned_addr learned my addr 172.21.15.78:0/150111865 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:23.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.146+0000 7f5a40bce640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a3c106ed0 0x7f5a3c06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.146+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c1023b0 0x7f5a3c074d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.147+0000 7f5a40bce640 1 -- 172.21.15.78:0/150111865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a3c101a90 msgr2=0x7f5a3c0747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:23.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.147+0000 7f5a40bce640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a3c101a90 0x7f5a3c0747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.147+0000 7f5a40bce640 1 -- 172.21.15.78:0/150111865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c1023b0 msgr2=0x7f5a3c074d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:23.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.147+0000 7f5a40bce640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c1023b0 0x7f5a3c074d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.147+0000 7f5a40bce640 1 -- 172.21.15.78:0/150111865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c06e480 con 0x7f5a3c106ed0 2024-09-11T01:23:23.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.147+0000 7f5a40bce640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a3c106ed0 0x7f5a3c06dbc0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009fd0 tx=0x7f5a3400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:23.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.148+0000 7f5a397fa640 1 -- 172.21.15.78:0/150111865 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3400ce20 con 0x7f5a3c106ed0 2024-09-11T01:23:23.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.148+0000 7f5a397fa640 1 -- 172.21.15.78:0/150111865 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a34005020 con 0x7f5a3c106ed0 2024-09-11T01:23:23.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.148+0000 7f5a397fa640 1 -- 172.21.15.78:0/150111865 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34010690 con 0x7f5a3c106ed0 2024-09-11T01:23:23.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.148+0000 7f5a42631640 1 -- 172.21.15.78:0/150111865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3c06e770 con 0x7f5a3c106ed0 2024-09-11T01:23:23.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.148+0000 7f5a42631640 1 -- 172.21.15.78:0/150111865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3c073690 con 0x7f5a3c106ed0 2024-09-11T01:23:23.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.151+0000 7f5a397fa640 1 -- 172.21.15.78:0/150111865 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5a34004a40 con 0x7f5a3c106ed0 2024-09-11T01:23:23.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.151+0000 7f5a44a06740 1 Processor -- start 2024-09-11T01:23:23.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.152+0000 7f5a44a06740 1 -- start start 2024-09-11T01:23:23.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.152+0000 7f5a397fa640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a240b5500 0x7f5a240b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.152+0000 7f5a44a06740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.152+0000 7f5a44a06740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f32a10 0x55fa83f3e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.152+0000 7f5a44a06740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f3e9a0 0x55fa83f40d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.152+0000 7f5a44a06740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa83ea8ea0 con 0x55fa83f320f0 2024-09-11T01:23:23.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.152+0000 7f5a44a06740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fa83e982d0 con 0x55fa83f32a10 2024-09-11T01:23:23.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.152+0000 7f5a44a06740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fa83e9c0e0 con 0x55fa83f3e9a0 2024-09-11T01:23:23.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.152+0000 7f5a397fa640 1 -- 172.21.15.78:0/150111865 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7f5a340e6470 con 0x7f5a3c106ed0 2024-09-11T01:23:23.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.153+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.153+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f324d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55190/0 (socket says 172.21.15.78:55190) 2024-09-11T01:23:23.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.153+0000 7f5a3bfff640 1 -- 172.21.15.78:0/761459232 learned_addr learned my addr 172.21.15.78:0/761459232 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:23.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.153+0000 7f5a40bce640 1 --2- 172.21.15.78:0/761459232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f3e9a0 0x55fa83f40d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.153+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/761459232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f32a10 0x55fa83f3e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.153+0000 7f5a3bfff640 1 -- 172.21.15.78:0/761459232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f3e9a0 msgr2=0x55fa83f40d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:23.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.153+0000 7f5a3bfff640 1 --2- 172.21.15.78:0/761459232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f3e9a0 0x55fa83f40d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.153+0000 7f5a3bfff640 1 -- 172.21.15.78:0/761459232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f32a10 msgr2=0x55fa83f3e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:23.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.153+0000 7f5a3bfff640 1 --2- 172.21.15.78:0/761459232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f32a10 0x55fa83f3e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.153+0000 7f5a3bfff640 1 -- 172.21.15.78:0/761459232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa83bf6650 con 0x55fa83f320f0 2024-09-11T01:23:23.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.153+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a240b5500 0x7f5a240b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.153+0000 7f5a3bfff640 1 --2- 172.21.15.78:0/761459232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f324d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00b7e0 tx=0x7f5a2c00bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcc39705271b92 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:23.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.154+0000 7f5a227fc640 1 -- 172.21.15.78:0/761459232 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c003dd0 con 0x55fa83f320f0 2024-09-11T01:23:23.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.154+0000 7f5a227fc640 1 -- 172.21.15.78:0/761459232 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2c015070 con 0x55fa83f320f0 2024-09-11T01:23:23.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.154+0000 7f5a227fc640 1 -- 172.21.15.78:0/761459232 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c019770 con 0x55fa83f320f0 2024-09-11T01:23:23.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.155+0000 7f5a44a06740 1 -- 172.21.15.78:0/761459232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 msgr2=0x55fa83f324d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:23.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.155+0000 7f5a44a06740 1 --2- 172.21.15.78:0/761459232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f324d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00b7e0 tx=0x7f5a2c00bca0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.156+0000 7f5a44a06740 1 -- 172.21.15.78:0/761459232 shutdown_connections 2024-09-11T01:23:23.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.156+0000 7f5a44a06740 1 --2- 172.21.15.78:0/761459232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f3e9a0 0x55fa83f40d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.156+0000 7f5a44a06740 1 --2- 172.21.15.78:0/761459232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f32a10 0x55fa83f3e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.156+0000 7f5a44a06740 1 --2- 172.21.15.78:0/761459232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f324d0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.156+0000 7f5a44a06740 1 -- 172.21.15.78:0/761459232 >> 172.21.15.78:0/761459232 conn(0x55fa83f2ff20 msgr2=0x55fa83f310c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:23.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.159+0000 7f5a44a06740 1 -- 172.21.15.78:0/761459232 shutdown_connections 2024-09-11T01:23:23.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.159+0000 7f5a44a06740 1 -- 172.21.15.78:0/761459232 wait complete. 2024-09-11T01:23:23.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.159+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a240b5500 0x7f5a240b79c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5a300303f0 tx=0x7f5a3002a840 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:23.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.159+0000 7f5a44a06740 1 -- start start 2024-09-11T01:23:23.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a44a06740 1 Processor -- start 2024-09-11T01:23:23.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a44a06740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f320f0 0x55fa83f3e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a44a06740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f3b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a44a06740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f3e9a0 0x55fa83f3bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a44a06740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa83cf9700 con 0x55fa83f3e9a0 2024-09-11T01:23:23.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a44a06740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fa83e33160 con 0x55fa83f320f0 2024-09-11T01:23:23.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a44a06740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fa83f41620 con 0x55fa83f32a10 2024-09-11T01:23:23.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a44a06740 10 client.0 _ll_register_callbacks cb 0x55fa83e4c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f320f0 0x55fa83f3e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a3b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f3b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f320f0 0x55fa83f3e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:51420/0 (socket says 172.21.15.78:51420) 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a3bfff640 1 -- 172.21.15.78:0/3865708135 learned_addr learned my addr 172.21.15.78:0/3865708135 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a3b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f3b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57344/0 (socket says 172.21.15.78:57344) 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.160+0000 7f5a40bce640 1 --2- 172.21.15.78:0/3865708135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f3e9a0 0x55fa83f3bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.161+0000 7f5a3bfff640 1 -- 172.21.15.78:0/3865708135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 msgr2=0x55fa83f3b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.161+0000 7f5a3bfff640 1 --2- 172.21.15.78:0/3865708135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f3b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.161+0000 7f5a3bfff640 1 -- 172.21.15.78:0/3865708135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f3e9a0 msgr2=0x55fa83f3bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.161+0000 7f5a3bfff640 1 --2- 172.21.15.78:0/3865708135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f3e9a0 0x55fa83f3bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.161+0000 7f5a3bfff640 1 -- 172.21.15.78:0/3865708135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa83e60c90 con 0x55fa83f320f0 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.161+0000 7f5a3bfff640 1 --2- 172.21.15.78:0/3865708135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f320f0 0x55fa83f3e220 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c003e00 tx=0x7f5a2c003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ca560490814e7ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:23.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.161+0000 7f5a1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:23:23.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.161+0000 7f5a1b7fe640 1 -- 172.21.15.78:0/3865708135 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c011650 con 0x55fa83f320f0 2024-09-11T01:23:23.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.161+0000 7f5a1b7fe640 1 -- 172.21.15.78:0/3865708135 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7f5a2c0117f0 con 0x55fa83f320f0 2024-09-11T01:23:23.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.161+0000 7f5a44a06740 1 -- 172.21.15.78:0/3865708135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a2c015070 con 0x55fa83f320f0 2024-09-11T01:23:23.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.162+0000 7f5a1b7fe640 1 -- 172.21.15.78:0/3865708135 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f5a2c0119f0 con 0x55fa83f320f0 2024-09-11T01:23:23.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.162+0000 7f5a44a06740 10 client.28568 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:23.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.162+0000 7f5a44a06740 1 -- 172.21.15.78:0/3865708135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55fa83f4cf50 con 0x55fa83f320f0 2024-09-11T01:23:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.162+0000 7f5a1b7fe640 1 -- 172.21.15.78:0/3865708135 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2c015070 con 0x55fa83f320f0 2024-09-11T01:23:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.162+0000 7f5a1b7fe640 1 -- 172.21.15.78:0/3865708135 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f5a2c025430 con 0x55fa83f320f0 2024-09-11T01:23:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.162+0000 7f5a44a06740 10 client.28568 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.162+0000 7f5a44a06740 10 client.28568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:23:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.162+0000 7f5a44a06740 20 client.28568 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.162+0000 7f5a44a06740 1 --2- 172.21.15.78:0/3865708135 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fa83f4e350 0x55fa83f50740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.162+0000 7f5a44a06740 4 client.28568 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:23:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.162+0000 7f5a44a06740 1 -- 172.21.15.78:0/3865708135 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa83eaadf0 con 0x55fa83f4e350 2024-09-11T01:23:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.162+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/3865708135 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fa83f4e350 0x55fa83f50740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.163+0000 7f5a1b7fe640 10 client.28568 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:23.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.163+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/3865708135 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fa83f4e350 0x55fa83f50740 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22dc35f0c345a59 server_cookie=2e04bc9639dff7dc in_seq=0 out_seq=0 2024-09-11T01:23:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.166+0000 7f5a1b7fe640 1 -- 172.21.15.78:0/3865708135 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fa83eaadf0 con 0x55fa83f4e350 2024-09-11T01:23:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.166+0000 7f5a1b7fe640 10 client.28568 handle_command_reply: tid=0 2024-09-11T01:23:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.167+0000 7f5a44a06740 1 client.28568 shutdown 2024-09-11T01:23:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.167+0000 7f5a44a06740 1 -- 172.21.15.78:0/3865708135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f320f0 msgr2=0x55fa83f3e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.167+0000 7f5a44a06740 1 --2- 172.21.15.78:0/3865708135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f320f0 0x55fa83f3e220 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c003e00 tx=0x7f5a2c003e30 comp rx=0 tx=0).stop 2024-09-11T01:23:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.167+0000 7f5a44a06740 1 -- 172.21.15.78:0/3865708135 shutdown_connections 2024-09-11T01:23:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.167+0000 7f5a44a06740 1 --2- 172.21.15.78:0/3865708135 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fa83f4e350 0x55fa83f50740 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.167+0000 7f5a44a06740 1 --2- 172.21.15.78:0/3865708135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f3e9a0 0x55fa83f3bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.167+0000 7f5a44a06740 1 --2- 172.21.15.78:0/3865708135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f3b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.167+0000 7f5a44a06740 1 --2- 172.21.15.78:0/3865708135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f320f0 0x55fa83f3e220 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.167+0000 7f5a44a06740 1 -- 172.21.15.78:0/3865708135 >> 172.21.15.78:0/3865708135 conn(0x55fa83f2ff20 msgr2=0x55fa83f35a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.167+0000 7f5a1b7fe640 0 client.28568 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.168+0000 7f5a44a06740 1 -- 172.21.15.78:0/3865708135 shutdown_connections 2024-09-11T01:23:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.168+0000 7f5a44a06740 1 -- 172.21.15.78:0/3865708135 wait complete. 2024-09-11T01:23:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.168+0000 7f5a44a06740 20 client.28568 trim_cache size 0 max 16384 2024-09-11T01:23:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.189+0000 7f5a44a06740 1 Processor -- start 2024-09-11T01:23:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.189+0000 7f5a44a06740 1 -- start start 2024-09-11T01:23:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a44a06740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f4a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a44a06740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f30c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f4a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f4a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55200/0 (socket says 172.21.15.78:55200) 2024-09-11T01:23:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a44a06740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f3e9a0 0x55fa83f31140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a44a06740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2801f6c0 con 0x55fa83f320f0 2024-09-11T01:23:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a44a06740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280212c0 con 0x55fa83f3e9a0 2024-09-11T01:23:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a44a06740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2801fd90 con 0x55fa83f32a10 2024-09-11T01:23:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a3bfff640 1 -- 172.21.15.78:0/4005613844 learned_addr learned my addr 172.21.15.78:0/4005613844 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/4005613844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f30c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a40bce640 1 --2- 172.21.15.78:0/4005613844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f3e9a0 0x55fa83f31140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a3b7fe640 1 -- 172.21.15.78:0/4005613844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f3e9a0 msgr2=0x55fa83f31140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/4005613844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f3e9a0 0x55fa83f31140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a3b7fe640 1 -- 172.21.15.78:0/4005613844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 msgr2=0x55fa83f4a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/4005613844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f4a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a3b7fe640 1 -- 172.21.15.78:0/4005613844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa83f363a0 con 0x55fa83f32a10 2024-09-11T01:23:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.190+0000 7f5a40bce640 1 --2- 172.21.15.78:0/4005613844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f3e9a0 0x55fa83f31140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:23:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.191+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/4005613844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f30c00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5a30035040 tx=0x7f5a3002c070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=202e598f053ab4a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.191+0000 7f5a21ffb640 1 -- 172.21.15.78:0/4005613844 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30037040 con 0x55fa83f32a10 2024-09-11T01:23:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.191+0000 7f5a21ffb640 1 -- 172.21.15.78:0/4005613844 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a30052030 con 0x55fa83f32a10 2024-09-11T01:23:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.191+0000 7f5a21ffb640 1 -- 172.21.15.78:0/4005613844 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a300046c0 con 0x55fa83f32a10 2024-09-11T01:23:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.192+0000 7f5a44a06740 1 -- 172.21.15.78:0/4005613844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 msgr2=0x55fa83f30c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.192+0000 7f5a44a06740 1 --2- 172.21.15.78:0/4005613844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f30c00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5a30035040 tx=0x7f5a3002c070 comp rx=0 tx=0).stop 2024-09-11T01:23:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.192+0000 7f5a44a06740 1 -- 172.21.15.78:0/4005613844 shutdown_connections 2024-09-11T01:23:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.192+0000 7f5a44a06740 1 --2- 172.21.15.78:0/4005613844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f3e9a0 0x55fa83f31140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.192+0000 7f5a44a06740 1 --2- 172.21.15.78:0/4005613844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f30c00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.192+0000 7f5a44a06740 1 --2- 172.21.15.78:0/4005613844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f4a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.192+0000 7f5a44a06740 1 -- 172.21.15.78:0/4005613844 >> 172.21.15.78:0/4005613844 conn(0x55fa83f2ff20 msgr2=0x55fa83f368d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.192+0000 7f5a44a06740 1 -- 172.21.15.78:0/4005613844 shutdown_connections 2024-09-11T01:23:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.192+0000 7f5a44a06740 1 -- 172.21.15.78:0/4005613844 wait complete. 2024-09-11T01:23:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.192+0000 7f5a44a06740 1 -- start start 2024-09-11T01:23:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a44a06740 1 Processor -- start 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a44a06740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a44a06740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f387a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a44a06740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f3e9a0 0x55fa83f38f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a44a06740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28020bf0 con 0x55fa83f320f0 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a44a06740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28020520 con 0x55fa83f3e9a0 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a44a06740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2801d6c0 con 0x55fa83f32a10 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a3b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a3b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f387a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57352/0 (socket says 172.21.15.78:57352) 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a3b7fe640 1 -- 172.21.15.78:0/203625289 learned_addr learned my addr 172.21.15.78:0/203625289 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a40bce640 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f3e9a0 0x55fa83f38f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a3bfff640 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.193+0000 7f5a44a06740 10 client.0 _ll_register_callbacks cb 0x7f5a2c0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.194+0000 7f5a3b7fe640 1 -- 172.21.15.78:0/203625289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f3e9a0 msgr2=0x55fa83f38f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.194+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f3e9a0 0x55fa83f38f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.194+0000 7f5a3b7fe640 1 -- 172.21.15.78:0/203625289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 msgr2=0x55fa83f472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.194+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.194+0000 7f5a3b7fe640 1 -- 172.21.15.78:0/203625289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa83f45f50 con 0x55fa83f32a10 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.194+0000 7f5a3b7fe640 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f387a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009780 tx=0x7f5a300041f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fce6753d88a031d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.194+0000 7f5a1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.194+0000 7f5a1b7fe640 1 -- 172.21.15.78:0/203625289 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30037070 con 0x55fa83f32a10 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.194+0000 7f5a1b7fe640 1 -- 172.21.15.78:0/203625289 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7f5a30003dc0 con 0x55fa83f32a10 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.194+0000 7f5a44a06740 1 -- 172.21.15.78:0/203625289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a30052030 con 0x55fa83f32a10 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.194+0000 7f5a1b7fe640 1 -- 172.21.15.78:0/203625289 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f5a300324e0 con 0x55fa83f32a10 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.195+0000 7f5a44a06740 10 client.28522 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.195+0000 7f5a44a06740 1 -- 172.21.15.78:0/203625289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55fa83fbb400 con 0x55fa83f32a10 2024-09-11T01:23:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.195+0000 7f5a1b7fe640 1 -- 172.21.15.78:0/203625289 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a30052030 con 0x55fa83f32a10 2024-09-11T01:23:23.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.195+0000 7f5a1b7fe640 1 -- 172.21.15.78:0/203625289 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f5a300408a0 con 0x55fa83f32a10 2024-09-11T01:23:23.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.195+0000 7f5a44a06740 10 client.28522 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:23.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.195+0000 7f5a44a06740 10 client.28522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:23:23.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.195+0000 7f5a44a06740 20 client.28522 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:23.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.195+0000 7f5a44a06740 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fa83fbbb50 0x55fa83fbdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:23.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.195+0000 7f5a44a06740 4 client.28522 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:23:23.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.195+0000 7f5a44a06740 1 -- 172.21.15.78:0/203625289 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa83ea68a0 con 0x55fa83fbbb50 2024-09-11T01:23:23.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.196+0000 7f5a3bfff640 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fa83fbbb50 0x55fa83fbdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:23.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.196+0000 7f5a3bfff640 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fa83fbbb50 0x55fa83fbdf70 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c2847be2081fe0 server_cookie=36fe65a4138e931 in_seq=0 out_seq=0 2024-09-11T01:23:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.196+0000 7f5a1b7fe640 10 client.28522 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.197+0000 7f5a1b7fe640 1 -- 172.21.15.78:0/203625289 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa83ea68a0 con 0x55fa83fbbb50 2024-09-11T01:23:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.197+0000 7f5a1b7fe640 10 client.28522 handle_command_reply: tid=0 2024-09-11T01:23:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.197+0000 7f5a44a06740 1 client.28522 shutdown 2024-09-11T01:23:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.197+0000 7f5a44a06740 1 -- 172.21.15.78:0/203625289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 msgr2=0x55fa83f387a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.198+0000 7f5a44a06740 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f387a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009780 tx=0x7f5a300041f0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.198+0000 7f5a44a06740 1 -- 172.21.15.78:0/203625289 shutdown_connections 2024-09-11T01:23:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.198+0000 7f5a44a06740 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa83f3e9a0 0x55fa83f38f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.198+0000 7f5a44a06740 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa83f32a10 0x55fa83f387a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.198+0000 7f5a44a06740 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fa83fbbb50 0x55fa83fbdf70 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.198+0000 7f5a1b7fe640 0 client.28522 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.198+0000 7f5a44a06740 1 --2- 172.21.15.78:0/203625289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa83f320f0 0x55fa83f472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.198+0000 7f5a44a06740 1 -- 172.21.15.78:0/203625289 >> 172.21.15.78:0/203625289 conn(0x55fa83f2ff20 msgr2=0x55fa83f39e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:23.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.199+0000 7f5a44a06740 1 -- 172.21.15.78:0/203625289 shutdown_connections 2024-09-11T01:23:23.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.199+0000 7f5a44a06740 1 -- 172.21.15.78:0/203625289 wait complete. 2024-09-11T01:23:23.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.199+0000 7f5a44a06740 20 client.28522 trim_cache size 0 max 16384 2024-09-11T01:23:23.205 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:23:23.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.200+0000 7f5a42631640 1 -- 172.21.15.78:0/150111865 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a240b5500 msgr2=0x7f5a240b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:23.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.200+0000 7f5a42631640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a240b5500 0x7f5a240b79c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5a300303f0 tx=0x7f5a3002a840 comp rx=0 tx=0).stop 2024-09-11T01:23:23.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.200+0000 7f5a42631640 1 -- 172.21.15.78:0/150111865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a3c106ed0 msgr2=0x7f5a3c06dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:23.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.200+0000 7f5a42631640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a3c106ed0 0x7f5a3c06dbc0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009fd0 tx=0x7f5a3400ef10 comp rx=0 tx=0).stop 2024-09-11T01:23:23.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.200+0000 7f5a42631640 1 -- 172.21.15.78:0/150111865 shutdown_connections 2024-09-11T01:23:23.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.200+0000 7f5a42631640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f5a240b5500 0x7f5a240b79c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.200+0000 7f5a42631640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a3c106ed0 0x7f5a3c06dbc0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.200+0000 7f5a42631640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c1023b0 0x7f5a3c074d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.200+0000 7f5a42631640 1 --2- 172.21.15.78:0/150111865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a3c101a90 0x7f5a3c0747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:23.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.200+0000 7f5a42631640 1 -- 172.21.15.78:0/150111865 >> 172.21.15.78:0/150111865 conn(0x7f5a3c0fd700 msgr2=0x7f5a3c1047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:23.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.201+0000 7f5a42631640 1 -- 172.21.15.78:0/150111865 shutdown_connections 2024-09-11T01:23:23.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:23.201+0000 7f5a42631640 1 -- 172.21.15.78:0/150111865 wait complete. 2024-09-11T01:23:23.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 411946 31.46 MB/sec execute 569 sec latency 266.062 ms 2024-09-11T01:23:23.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:23 smithi110 ceph-mon[34712]: pgmap v1094: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 505 op/s 2024-09-11T01:23:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:23 smithi112 ceph-mon[35384]: pgmap v1094: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 505 op/s 2024-09-11T01:23:23.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:23 smithi078 ceph-mon[31315]: pgmap v1094: 97 pgs: 97 active+clean; 3.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 505 op/s 2024-09-11T01:23:24.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 412381 31.45 MB/sec execute 570 sec latency 504.240 ms 2024-09-11T01:23:25.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 412975 31.46 MB/sec execute 571 sec latency 96.503 ms 2024-09-11T01:23:25.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:25 smithi110 ceph-mon[34712]: pgmap v1095: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 546 op/s 2024-09-11T01:23:25.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:25 smithi112 ceph-mon[35384]: pgmap v1095: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 546 op/s 2024-09-11T01:23:25.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:25 smithi078 ceph-mon[31315]: pgmap v1095: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 546 op/s 2024-09-11T01:23:26.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 413495 31.43 MB/sec execute 572 sec latency 155.946 ms 2024-09-11T01:23:27.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:27 smithi078 ceph-mon[31315]: pgmap v1096: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 675 op/s 2024-09-11T01:23:27.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 413987 31.43 MB/sec execute 573 sec latency 167.844 ms 2024-09-11T01:23:27.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:27 smithi110 ceph-mon[34712]: pgmap v1096: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 675 op/s 2024-09-11T01:23:27.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:27 smithi112 ceph-mon[35384]: pgmap v1096: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 675 op/s 2024-09-11T01:23:28.440 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T01:23:28.440 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T01:23:28.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 414518 31.43 MB/sec execute 574 sec latency 348.560 ms 2024-09-11T01:23:28.888 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:23:29.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 415121 31.43 MB/sec execute 575 sec latency 380.074 ms 2024-09-11T01:23:29.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:29 smithi110 ceph-mon[34712]: pgmap v1097: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 570 op/s 2024-09-11T01:23:29.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:29 smithi112 ceph-mon[35384]: pgmap v1097: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 570 op/s 2024-09-11T01:23:29.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:29 smithi078 ceph-mon[31315]: pgmap v1097: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 570 op/s 2024-09-11T01:23:30.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 415613 31.43 MB/sec execute 576 sec latency 226.572 ms 2024-09-11T01:23:31.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.522+0000 7f16b6ad7640 1 -- 172.21.15.78:0/2248903366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16b0102de0 msgr2=0x7f16b01031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:31.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.522+0000 7f16b6ad7640 1 --2- 172.21.15.78:0/2248903366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16b0102de0 0x7f16b01031c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009900 tx=0x7f16a002a440 comp rx=0 tx=0).stop 2024-09-11T01:23:31.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.522+0000 7f16b6ad7640 1 -- 172.21.15.78:0/2248903366 shutdown_connections 2024-09-11T01:23:31.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.522+0000 7f16b6ad7640 1 --2- 172.21.15.78:0/2248903366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0109170 0x7f16b010f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.522+0000 7f16b6ad7640 1 --2- 172.21.15.78:0/2248903366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16b01087f0 0x7f16b0108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.522+0000 7f16b6ad7640 1 --2- 172.21.15.78:0/2248903366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16b0102de0 0x7f16b01031c0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.522+0000 7f16b6ad7640 1 -- 172.21.15.78:0/2248903366 >> 172.21.15.78:0/2248903366 conn(0x7f16b00fe830 msgr2=0x7f16b0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.523+0000 7f16b6ad7640 1 -- 172.21.15.78:0/2248903366 shutdown_connections 2024-09-11T01:23:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.523+0000 7f16b6ad7640 1 -- 172.21.15.78:0/2248903366 wait complete. 2024-09-11T01:23:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.524+0000 7f16b6ad7640 1 Processor -- start 2024-09-11T01:23:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.525+0000 7f16b6ad7640 1 -- start start 2024-09-11T01:23:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.526+0000 7f16b6ad7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16b0102de0 0x7f16b010f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.526+0000 7f16b6ad7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16b01087f0 0x7f16b010c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.526+0000 7f16b6ad7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0109170 0x7f16b010ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.526+0000 7f16b6ad7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16b0076f70 con 0x7f16b01087f0 2024-09-11T01:23:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.526+0000 7f16b6ad7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f16b0076df0 con 0x7f16b0109170 2024-09-11T01:23:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.526+0000 7f16b6ad7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16b00770f0 con 0x7f16b0102de0 2024-09-11T01:23:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.527+0000 7f16affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16b01087f0 0x7f16b010c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.527+0000 7f16affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16b01087f0 0x7f16b010c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51052/0 (socket says 172.21.15.78:51052) 2024-09-11T01:23:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.527+0000 7f16affff640 1 -- 172.21.15.78:0/3570936305 learned_addr learned my addr 172.21.15.78:0/3570936305 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.528+0000 7f16b5074640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0109170 0x7f16b010ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.528+0000 7f16b4873640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16b0102de0 0x7f16b010f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.529+0000 7f16b5074640 1 -- 172.21.15.78:0/3570936305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16b0102de0 msgr2=0x7f16b010f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:31.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.529+0000 7f16b5074640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16b0102de0 0x7f16b010f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.529+0000 7f16b5074640 1 -- 172.21.15.78:0/3570936305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16b01087f0 msgr2=0x7f16b010c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:31.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.529+0000 7f16b5074640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16b01087f0 0x7f16b010c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.529+0000 7f16b5074640 1 -- 172.21.15.78:0/3570936305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b010d6e0 con 0x7f16b0109170 2024-09-11T01:23:31.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.529+0000 7f16b5074640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0109170 0x7f16b010ce20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f16a800be20 tx=0x7f16a800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:31.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.530+0000 7f16adffb640 1 -- 172.21.15.78:0/3570936305 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8019070 con 0x7f16b0109170 2024-09-11T01:23:31.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.530+0000 7f16adffb640 1 -- 172.21.15.78:0/3570936305 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a80148f0 con 0x7f16b0109170 2024-09-11T01:23:31.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.530+0000 7f16adffb640 1 -- 172.21.15.78:0/3570936305 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8012880 con 0x7f16b0109170 2024-09-11T01:23:31.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.530+0000 7f16b6ad7640 1 -- 172.21.15.78:0/3570936305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b01abd60 con 0x7f16b0109170 2024-09-11T01:23:31.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.531+0000 7f16b6ad7640 1 -- 172.21.15.78:0/3570936305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b01ac1c0 con 0x7f16b0109170 2024-09-11T01:23:31.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.534+0000 7f16adffb640 1 -- 172.21.15.78:0/3570936305 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f16a8014440 con 0x7f16b0109170 2024-09-11T01:23:31.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.535+0000 7f16adffb640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16980b5580 0x7f16980b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.536+0000 7f16adffb640 1 -- 172.21.15.78:0/3570936305 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7f16a80e5a70 con 0x7f16b0109170 2024-09-11T01:23:31.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.536+0000 7f16b4873640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16980b5580 0x7f16980b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.539+0000 7f16b8eac740 1 Processor -- start 2024-09-11T01:23:31.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.539+0000 7f16b8eac740 1 -- start start 2024-09-11T01:23:31.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.540+0000 7f16b8eac740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 0x55fc123662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.540+0000 7f16b8eac740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12366800 0x55fc12372250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.540+0000 7f16b8eac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12372790 0x55fc12374b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.540+0000 7f16b8eac740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fc122dce60 con 0x55fc12366800 2024-09-11T01:23:31.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.540+0000 7f16b8eac740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fc122cc290 con 0x55fc12365ee0 2024-09-11T01:23:31.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.540+0000 7f16b8eac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fc122d00a0 con 0x55fc12372790 2024-09-11T01:23:31.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.540+0000 7f16b5074640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12372790 0x55fc12374b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.540+0000 7f16b5074640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12372790 0x55fc12374b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:38386/0 (socket says 172.21.15.78:38386) 2024-09-11T01:23:31.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.540+0000 7f16b5074640 1 -- 172.21.15.78:0/859309944 learned_addr learned my addr 172.21.15.78:0/859309944 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:31.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.541+0000 7f16b4873640 1 --2- 172.21.15.78:0/859309944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 0x55fc123662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.541+0000 7f16affff640 1 --2- 172.21.15.78:0/859309944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12366800 0x55fc12372250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.541+0000 7f16b4873640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16980b5580 0x7f16980b7a40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f16a00096c0 tx=0x7f16a000b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:31.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.541+0000 7f16b5074640 1 -- 172.21.15.78:0/859309944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 msgr2=0x55fc123662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:31.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.541+0000 7f16b5074640 1 --2- 172.21.15.78:0/859309944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 0x55fc123662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.541+0000 7f16b5074640 1 -- 172.21.15.78:0/859309944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12366800 msgr2=0x55fc12372250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:31.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.541+0000 7f16b5074640 1 --2- 172.21.15.78:0/859309944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12366800 0x55fc12372250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.541+0000 7f16b5074640 1 -- 172.21.15.78:0/859309944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1202a930 con 0x55fc12372790 2024-09-11T01:23:31.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.541+0000 7f16b4873640 1 --2- 172.21.15.78:0/859309944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 0x55fc123662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:23:31.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.541+0000 7f16b5074640 1 --2- 172.21.15.78:0/859309944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12372790 0x55fc12374b80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f16a80a8550 tx=0x7f16a80ab8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab9a3fd08940e0fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:31.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.542+0000 7f1696ffd640 1 -- 172.21.15.78:0/859309944 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a80aba90 con 0x55fc12372790 2024-09-11T01:23:31.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.542+0000 7f1696ffd640 1 -- 172.21.15.78:0/859309944 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a80abf80 con 0x55fc12372790 2024-09-11T01:23:31.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.542+0000 7f1696ffd640 1 -- 172.21.15.78:0/859309944 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a80b5230 con 0x55fc12372790 2024-09-11T01:23:31.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.543+0000 7f16b8eac740 1 -- 172.21.15.78:0/859309944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12372790 msgr2=0x55fc12374b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:31.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.543+0000 7f16b8eac740 1 --2- 172.21.15.78:0/859309944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12372790 0x55fc12374b80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f16a80a8550 tx=0x7f16a80ab8d0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.543+0000 7f16b8eac740 1 -- 172.21.15.78:0/859309944 shutdown_connections 2024-09-11T01:23:31.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.543+0000 7f16b8eac740 1 --2- 172.21.15.78:0/859309944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12372790 0x55fc12374b80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.543+0000 7f16b8eac740 1 --2- 172.21.15.78:0/859309944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12366800 0x55fc12372250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.543+0000 7f16b8eac740 1 --2- 172.21.15.78:0/859309944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 0x55fc123662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.543+0000 7f16b8eac740 1 -- 172.21.15.78:0/859309944 >> 172.21.15.78:0/859309944 conn(0x55fc12363d10 msgr2=0x55fc12364eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:31.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.545+0000 7f16b8eac740 1 -- 172.21.15.78:0/859309944 shutdown_connections 2024-09-11T01:23:31.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.545+0000 7f16b8eac740 1 -- 172.21.15.78:0/859309944 wait complete. 2024-09-11T01:23:31.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.546+0000 7f16b8eac740 1 -- start start 2024-09-11T01:23:31.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.546+0000 7f16b8eac740 1 Processor -- start 2024-09-11T01:23:31.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.547+0000 7f16b8eac740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12365ee0 0x55fc1236f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.547+0000 7f16b8eac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12366800 0x55fc1236fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.547+0000 7f16b8eac740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12372790 0x55fc12370270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.547+0000 7f16b8eac740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fc1212d890 con 0x55fc12365ee0 2024-09-11T01:23:31.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.547+0000 7f16b8eac740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fc12266d40 con 0x55fc12372790 2024-09-11T01:23:31.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.547+0000 7f16b8eac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fc12386450 con 0x55fc12366800 2024-09-11T01:23:31.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.547+0000 7f16b4873640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12365ee0 0x55fc1236f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.547+0000 7f16b4873640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12365ee0 0x55fc1236f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51082/0 (socket says 172.21.15.78:51082) 2024-09-11T01:23:31.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.547+0000 7f16b8eac740 10 client.0 _ll_register_callbacks cb 0x55fc122cac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:31.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.547+0000 7f16b4873640 1 -- 172.21.15.78:0/1171372315 learned_addr learned my addr 172.21.15.78:0/1171372315 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:31.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.547+0000 7f16b5074640 1 --2- 172.21.15.78:0/1171372315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12372790 0x55fc12370270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.548+0000 7f16b4873640 1 -- 172.21.15.78:0/1171372315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12366800 msgr2=0x55fc1236fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:31.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.548+0000 7f16b4873640 1 --2- 172.21.15.78:0/1171372315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12366800 0x55fc1236fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.548+0000 7f16b4873640 1 -- 172.21.15.78:0/1171372315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12372790 msgr2=0x55fc12370270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:31.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.548+0000 7f16b4873640 1 --2- 172.21.15.78:0/1171372315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12372790 0x55fc12370270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.548+0000 7f16b4873640 1 -- 172.21.15.78:0/1171372315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc1238f940 con 0x55fc12365ee0 2024-09-11T01:23:31.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.548+0000 7f16b4873640 1 --2- 172.21.15.78:0/1171372315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12365ee0 0x55fc1236f7f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f16a0032cf0 tx=0x7f16a0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ca3cf341a69d33 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:31.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.548+0000 7f168ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:23:31.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.549+0000 7f168ffff640 1 -- 172.21.15.78:0/1171372315 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0035040 con 0x55fc12365ee0 2024-09-11T01:23:31.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.549+0000 7f168ffff640 1 -- 172.21.15.78:0/1171372315 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7f16a003cdd0 con 0x55fc12365ee0 2024-09-11T01:23:31.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.549+0000 7f16b8eac740 1 -- 172.21.15.78:0/1171372315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a80abf80 con 0x55fc12365ee0 2024-09-11T01:23:31.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.549+0000 7f168ffff640 1 -- 172.21.15.78:0/1171372315 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f16a004a430 con 0x55fc12365ee0 2024-09-11T01:23:31.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.549+0000 7f16b8eac740 10 client.19560 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:31.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.549+0000 7f16b8eac740 1 -- 172.21.15.78:0/1171372315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55fc123922d0 con 0x55fc12365ee0 2024-09-11T01:23:31.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.550+0000 7f168ffff640 1 -- 172.21.15.78:0/1171372315 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a80abf80 con 0x55fc12365ee0 2024-09-11T01:23:31.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.550+0000 7f168ffff640 1 -- 172.21.15.78:0/1171372315 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f16a005c6d0 con 0x55fc12365ee0 2024-09-11T01:23:31.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.550+0000 7f16b8eac740 10 client.19560 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:31.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.550+0000 7f16b8eac740 10 client.19560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:23:31.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.550+0000 7f16b8eac740 20 client.19560 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:31.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.550+0000 7f16b8eac740 1 --2- 172.21.15.78:0/1171372315 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fc123937e0 0x55fc12395c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.550+0000 7f16b8eac740 4 client.19560 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:23:31.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.550+0000 7f16b8eac740 1 -- 172.21.15.78:0/1171372315 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc122dedb0 con 0x55fc123937e0 2024-09-11T01:23:31.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.551+0000 7f16affff640 1 --2- 172.21.15.78:0/1171372315 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fc123937e0 0x55fc12395c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.552+0000 7f168ffff640 10 client.19560 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:31.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.552+0000 7f16affff640 1 --2- 172.21.15.78:0/1171372315 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fc123937e0 0x55fc12395c60 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c1ad483da825f3b server_cookie=75e96c1687ca8332 in_seq=0 out_seq=0 2024-09-11T01:23:31.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.553+0000 7f168ffff640 1 -- 172.21.15.78:0/1171372315 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc122dedb0 con 0x55fc123937e0 2024-09-11T01:23:31.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.553+0000 7f168ffff640 10 client.19560 handle_command_reply: tid=0 2024-09-11T01:23:31.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.554+0000 7f16b8eac740 1 client.19560 shutdown 2024-09-11T01:23:31.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.554+0000 7f16b8eac740 1 -- 172.21.15.78:0/1171372315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12365ee0 msgr2=0x55fc1236f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:31.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.554+0000 7f16b8eac740 1 --2- 172.21.15.78:0/1171372315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12365ee0 0x55fc1236f7f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f16a0032cf0 tx=0x7f16a0031400 comp rx=0 tx=0).stop 2024-09-11T01:23:31.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.554+0000 7f16b8eac740 1 -- 172.21.15.78:0/1171372315 shutdown_connections 2024-09-11T01:23:31.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.554+0000 7f16b8eac740 1 --2- 172.21.15.78:0/1171372315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12372790 0x55fc12370270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.554+0000 7f16b8eac740 1 --2- 172.21.15.78:0/1171372315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12366800 0x55fc1236fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.554+0000 7f16b8eac740 1 --2- 172.21.15.78:0/1171372315 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fc123937e0 0x55fc12395c60 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.554+0000 7f16b8eac740 1 --2- 172.21.15.78:0/1171372315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12365ee0 0x55fc1236f7f0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.554+0000 7f168ffff640 0 client.19560 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:31.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.554+0000 7f16b8eac740 1 -- 172.21.15.78:0/1171372315 >> 172.21.15.78:0/1171372315 conn(0x55fc12363d10 msgr2=0x55fc12362f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:31.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.555+0000 7f16b8eac740 1 -- 172.21.15.78:0/1171372315 shutdown_connections 2024-09-11T01:23:31.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.555+0000 7f16b8eac740 1 -- 172.21.15.78:0/1171372315 wait complete. 2024-09-11T01:23:31.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.555+0000 7f16b8eac740 20 client.19560 trim_cache size 0 max 16384 2024-09-11T01:23:31.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.582+0000 7f16b8eac740 1 Processor -- start 2024-09-11T01:23:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.582+0000 7f16b8eac740 1 -- start start 2024-09-11T01:23:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.582+0000 7f16b8eac740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12363d10 0x55fc1236d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.582+0000 7f16b8eac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12365ee0 0x55fc123652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.582+0000 7f16b8eac740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12366800 0x55fc1239b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.582+0000 7f16b8eac740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01ac5a0 con 0x55fc12363d10 2024-09-11T01:23:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.582+0000 7f16b8eac740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01ae1a0 con 0x55fc12366800 2024-09-11T01:23:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.582+0000 7f16b8eac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01acc70 con 0x55fc12365ee0 2024-09-11T01:23:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16b4873640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12363d10 0x55fc1236d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16b4873640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12363d10 0x55fc1236d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51086/0 (socket says 172.21.15.78:51086) 2024-09-11T01:23:31.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16b4873640 1 -- 172.21.15.78:0/2248509549 learned_addr learned my addr 172.21.15.78:0/2248509549 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16b5074640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12366800 0x55fc1239b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16affff640 1 --2- 172.21.15.78:0/2248509549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12365ee0 0x55fc123652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16b5074640 1 -- 172.21.15.78:0/2248509549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12365ee0 msgr2=0x55fc123652c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16b5074640 1 --2- 172.21.15.78:0/2248509549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12365ee0 0x55fc123652c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16b5074640 1 -- 172.21.15.78:0/2248509549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12363d10 msgr2=0x55fc1236d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16b5074640 1 --2- 172.21.15.78:0/2248509549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12363d10 0x55fc1236d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16b5074640 1 -- 172.21.15.78:0/2248509549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc12363b40 con 0x55fc12366800 2024-09-11T01:23:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16b4873640 1 --2- 172.21.15.78:0/2248509549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12363d10 0x55fc1236d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:23:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16affff640 1 --2- 172.21.15.78:0/2248509549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12365ee0 0x55fc123652c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:23:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16b5074640 1 --2- 172.21.15.78:0/2248509549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12366800 0x55fc1239b530 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f16a80a8550 tx=0x7f16a80aba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca2dcfbbe2c5810 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:31.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16967fc640 1 -- 172.21.15.78:0/2248509549 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a80b5460 con 0x55fc12366800 2024-09-11T01:23:31.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16967fc640 1 -- 172.21.15.78:0/2248509549 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a80e3420 con 0x55fc12366800 2024-09-11T01:23:31.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.583+0000 7f16967fc640 1 -- 172.21.15.78:0/2248509549 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8093370 con 0x55fc12366800 2024-09-11T01:23:31.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.584+0000 7f16b8eac740 1 -- 172.21.15.78:0/2248509549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12366800 msgr2=0x55fc1239b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:31.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.584+0000 7f16b8eac740 1 --2- 172.21.15.78:0/2248509549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12366800 0x55fc1239b530 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f16a80a8550 tx=0x7f16a80aba00 comp rx=0 tx=0).stop 2024-09-11T01:23:31.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.584+0000 7f16b8eac740 1 -- 172.21.15.78:0/2248509549 shutdown_connections 2024-09-11T01:23:31.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.584+0000 7f16b8eac740 1 --2- 172.21.15.78:0/2248509549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12366800 0x55fc1239b530 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.584+0000 7f16b8eac740 1 --2- 172.21.15.78:0/2248509549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12365ee0 0x55fc123652c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.584+0000 7f16b8eac740 1 --2- 172.21.15.78:0/2248509549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12363d10 0x55fc1236d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.584+0000 7f16b8eac740 1 -- 172.21.15.78:0/2248509549 >> 172.21.15.78:0/2248509549 conn(0x55fc1236f3c0 msgr2=0x55fc12364bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.584+0000 7f16b8eac740 1 -- 172.21.15.78:0/2248509549 shutdown_connections 2024-09-11T01:23:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.584+0000 7f16b8eac740 1 -- 172.21.15.78:0/2248509549 wait complete. 2024-09-11T01:23:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16b8eac740 1 -- start start 2024-09-11T01:23:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16b8eac740 1 Processor -- start 2024-09-11T01:23:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16b8eac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12363d10 0x55fc1238c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16b8eac740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 0x55fc12386700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16b8eac740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12366800 0x55fc12386df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16affff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 0x55fc12386700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16b8eac740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01adad0 con 0x55fc12366800 2024-09-11T01:23:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16b8eac740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01ad400 con 0x55fc12365ee0 2024-09-11T01:23:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16affff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 0x55fc12386700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:59912/0 (socket says 172.21.15.78:59912) 2024-09-11T01:23:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16b8eac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16b011c0f0 con 0x55fc12363d10 2024-09-11T01:23:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16affff640 1 -- 172.21.15.78:0/3702485111 learned_addr learned my addr 172.21.15.78:0/3702485111 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16b4873640 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12363d10 0x55fc1238c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16b5074640 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12366800 0x55fc12386df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.585+0000 7f16b8eac740 10 client.0 _ll_register_callbacks cb 0x7f16a0040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:31.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f16affff640 1 -- 172.21.15.78:0/3702485111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12363d10 msgr2=0x55fc1238c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f16affff640 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12363d10 0x55fc1238c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f16affff640 1 -- 172.21.15.78:0/3702485111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12366800 msgr2=0x55fc12386df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f16affff640 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12366800 0x55fc12386df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f16affff640 1 -- 172.21.15.78:0/3702485111 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc12400730 con 0x55fc12365ee0 2024-09-11T01:23:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f16affff640 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 0x55fc12386700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16a4000c00 tx=0x7f16a4017810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c53ef1d6b9f60651 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f168ffff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-11T01:23:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f168ffff640 1 -- 172.21.15.78:0/3702485111 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a40170d0 con 0x55fc12365ee0 2024-09-11T01:23:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f168ffff640 1 -- 172.21.15.78:0/3702485111 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7f16a4024420 con 0x55fc12365ee0 2024-09-11T01:23:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f16b8eac740 1 -- 172.21.15.78:0/3702485111 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a80e3420 con 0x55fc12365ee0 2024-09-11T01:23:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f168ffff640 1 -- 172.21.15.78:0/3702485111 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f16a4017270 con 0x55fc12365ee0 2024-09-11T01:23:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f16b8eac740 10 client.28589 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:31.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f16b8eac740 1 -- 172.21.15.78:0/3702485111 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55fc12401960 con 0x55fc12365ee0 2024-09-11T01:23:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.586+0000 7f168ffff640 1 -- 172.21.15.78:0/3702485111 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a80e3420 con 0x55fc12365ee0 2024-09-11T01:23:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.587+0000 7f168ffff640 1 -- 172.21.15.78:0/3702485111 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f16a402d7c0 con 0x55fc12365ee0 2024-09-11T01:23:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.587+0000 7f16b8eac740 10 client.28589 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.587+0000 7f16b8eac740 10 client.28589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:23:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.587+0000 7f16b8eac740 20 client.28589 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.587+0000 7f16b8eac740 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fc12372790 0x55fc12404150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.587+0000 7f16b8eac740 4 client.28589 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T01:23:31.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.587+0000 7f16b8eac740 1 -- 172.21.15.78:0/3702485111 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fc122da860 con 0x55fc12372790 2024-09-11T01:23:31.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.652+0000 7f16b4873640 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fc12372790 0x55fc12404150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:31.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.652+0000 7f16b4873640 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fc12372790 0x55fc12404150 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2327339338f33a2 server_cookie=8674b132146a23ea in_seq=0 out_seq=0 2024-09-11T01:23:31.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.652+0000 7f168ffff640 10 client.28589 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:31.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.657+0000 7f168ffff640 1 -- 172.21.15.78:0/3702485111 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fc122da860 con 0x55fc12372790 2024-09-11T01:23:31.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.657+0000 7f168ffff640 10 client.28589 handle_command_reply: tid=0 2024-09-11T01:23:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.657+0000 7f16b8eac740 1 client.28589 shutdown 2024-09-11T01:23:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.657+0000 7f16b8eac740 1 -- 172.21.15.78:0/3702485111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 msgr2=0x55fc12386700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.657+0000 7f16b8eac740 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 0x55fc12386700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16a4000c00 tx=0x7f16a4017810 comp rx=0 tx=0).stop 2024-09-11T01:23:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.657+0000 7f16b8eac740 1 -- 172.21.15.78:0/3702485111 shutdown_connections 2024-09-11T01:23:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.657+0000 7f16b8eac740 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55fc12372790 0x55fc12404150 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.657+0000 7f168ffff640 0 client.28589 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.657+0000 7f16b8eac740 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc12366800 0x55fc12386df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.657+0000 7f16b8eac740 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fc12365ee0 0x55fc12386700 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.657+0000 7f16b8eac740 1 --2- 172.21.15.78:0/3702485111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fc12363d10 0x55fc1238c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.657+0000 7f16b8eac740 1 -- 172.21.15.78:0/3702485111 >> 172.21.15.78:0/3702485111 conn(0x55fc1236f3c0 msgr2=0x55fc12369090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.658+0000 7f16b8eac740 1 -- 172.21.15.78:0/3702485111 shutdown_connections 2024-09-11T01:23:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.658+0000 7f16b8eac740 1 -- 172.21.15.78:0/3702485111 wait complete. 2024-09-11T01:23:31.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.658+0000 7f16b8eac740 20 client.28589 trim_cache size 0 max 16384 2024-09-11T01:23:31.661 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:23:31.661 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-11T01:23:31.661 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "6eb6da3d-724b-47e6-a25c-a6dcc447ffc7", 2024-09-11T01:23:31.661 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-11T01:23:31.661 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:23:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.658+0000 7f16b6ad7640 1 -- 172.21.15.78:0/3570936305 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16980b5580 msgr2=0x7f16980b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.658+0000 7f16b6ad7640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16980b5580 0x7f16980b7a40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f16a00096c0 tx=0x7f16a000b040 comp rx=0 tx=0).stop 2024-09-11T01:23:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.658+0000 7f16b6ad7640 1 -- 172.21.15.78:0/3570936305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0109170 msgr2=0x7f16b010ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.658+0000 7f16b6ad7640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0109170 0x7f16b010ce20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f16a800be20 tx=0x7f16a800bf10 comp rx=0 tx=0).stop 2024-09-11T01:23:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.659+0000 7f16b6ad7640 1 -- 172.21.15.78:0/3570936305 shutdown_connections 2024-09-11T01:23:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.659+0000 7f16b6ad7640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f16980b5580 0x7f16980b7a40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.659+0000 7f16b6ad7640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0109170 0x7f16b010ce20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.659+0000 7f16b6ad7640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16b01087f0 0x7f16b010c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.659+0000 7f16b6ad7640 1 --2- 172.21.15.78:0/3570936305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16b0102de0 0x7f16b010f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.659+0000 7f16b6ad7640 1 -- 172.21.15.78:0/3570936305 >> 172.21.15.78:0/3570936305 conn(0x7f16b00fe830 msgr2=0x7f16b0106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.659+0000 7f16b6ad7640 1 -- 172.21.15.78:0/3570936305 shutdown_connections 2024-09-11T01:23:31.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:31.659+0000 7f16b6ad7640 1 -- 172.21.15.78:0/3570936305 wait complete. 2024-09-11T01:23:31.813 INFO:tasks.workunit.client.0.smithi078.stdout: 10 416024 31.41 MB/sec execute 577 sec latency 147.531 ms 2024-09-11T01:23:31.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:31 smithi110 ceph-mon[34712]: pgmap v1098: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 19 MiB/s wr, 692 op/s 2024-09-11T01:23:31.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:31 smithi112 ceph-mon[35384]: pgmap v1098: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 19 MiB/s wr, 692 op/s 2024-09-11T01:23:31.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:31 smithi078 ceph-mon[31315]: pgmap v1098: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 19 MiB/s wr, 692 op/s 2024-09-11T01:23:32.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:23:32.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:32 smithi078 ceph-mon[31315]: scrub queued for path: / 2024-09-11T01:23:32.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:23:32.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:23:32.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:23:32.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:23:32.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:32 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:23:32.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:32 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.815 INFO:tasks.workunit.client.0.smithi078.stdout: 10 416402 31.40 MB/sec execute 578 sec latency 331.770 ms 2024-09-11T01:23:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:23:32.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:32 smithi110 ceph-mon[34712]: scrub queued for path: / 2024-09-11T01:23:32.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:23:32.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:23:32.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:23:32.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:23:32.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:32 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:23:32.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:32 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:32 smithi112 ceph-mon[35384]: scrub queued for path: / 2024-09-11T01:23:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:23:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:23:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:23:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:23:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:32 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:23:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:32.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:32 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:33.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.498+0000 7f6870440640 1 -- 172.21.15.78:0/2488446943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6868102230 msgr2=0x7f6868106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:33.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.498+0000 7f6870440640 1 --2- 172.21.15.78:0/2488446943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6868102230 0x7f6868106700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6858009860 tx=0x7f685802a340 comp rx=0 tx=0).stop 2024-09-11T01:23:33.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.498+0000 7f6870440640 1 -- 172.21.15.78:0/2488446943 shutdown_connections 2024-09-11T01:23:33.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.498+0000 7f6870440640 1 --2- 172.21.15.78:0/2488446943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6868106d50 0x7f68681071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.498+0000 7f6870440640 1 --2- 172.21.15.78:0/2488446943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6868102230 0x7f6868106700 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.498+0000 7f6870440640 1 --2- 172.21.15.78:0/2488446943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6868101910 0x7f6868101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.498+0000 7f6870440640 1 -- 172.21.15.78:0/2488446943 >> 172.21.15.78:0/2488446943 conn(0x7f68680fd580 msgr2=0x7f68680ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:33.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.498+0000 7f6870440640 1 -- 172.21.15.78:0/2488446943 shutdown_connections 2024-09-11T01:23:33.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.498+0000 7f6870440640 1 -- 172.21.15.78:0/2488446943 wait complete. 2024-09-11T01:23:33.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.499+0000 7f6870440640 1 Processor -- start 2024-09-11T01:23:33.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.499+0000 7f6870440640 1 -- start start 2024-09-11T01:23:33.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.499+0000 7f6870440640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6868101910 0x7f686819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.499+0000 7f6870440640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6868102230 0x7f686819fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.500+0000 7f6870440640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6868106d50 0x7f68681a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.500+0000 7f686e1dc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6868101910 0x7f686819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.500+0000 7f686e1dc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6868101910 0x7f686819f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:33052/0 (socket says 172.21.15.78:33052) 2024-09-11T01:23:33.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.500+0000 7f686d9db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6868102230 0x7f686819fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.500+0000 7f686d9db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6868102230 0x7f686819fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:37654/0 (socket says 172.21.15.78:37654) 2024-09-11T01:23:33.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.500+0000 7f686e1dc640 1 -- 172.21.15.78:0/1186025102 learned_addr learned my addr 172.21.15.78:0/1186025102 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:33.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.500+0000 7f6870440640 1 -- 172.21.15.78:0/1186025102 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68681191c0 con 0x7f6868106d50 2024-09-11T01:23:33.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.500+0000 7f6870440640 1 -- 172.21.15.78:0/1186025102 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6868119040 con 0x7f6868101910 2024-09-11T01:23:33.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.500+0000 7f6870440640 1 -- 172.21.15.78:0/1186025102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6868119340 con 0x7f6868102230 2024-09-11T01:23:33.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.500+0000 7f686e9dd640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6868106d50 0x7f68681a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.501+0000 7f686e9dd640 1 -- 172.21.15.78:0/1186025102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6868102230 msgr2=0x7f686819fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:33.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.501+0000 7f686e9dd640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6868102230 0x7f686819fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.501+0000 7f686e9dd640 1 -- 172.21.15.78:0/1186025102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6868101910 msgr2=0x7f686819f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:33.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.501+0000 7f686e9dd640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6868101910 0x7f686819f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.501+0000 7f686e9dd640 1 -- 172.21.15.78:0/1186025102 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68681a55a0 con 0x7f6868106d50 2024-09-11T01:23:33.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.501+0000 7f686e9dd640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6868106d50 0x7f68681a4dd0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f686000ef00 tx=0x7f686000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:33.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.501+0000 7f685f7fe640 1 -- 172.21.15.78:0/1186025102 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860004b20 con 0x7f6868106d50 2024-09-11T01:23:33.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.501+0000 7f685f7fe640 1 -- 172.21.15.78:0/1186025102 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6860003e20 con 0x7f6868106d50 2024-09-11T01:23:33.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.502+0000 7f685f7fe640 1 -- 172.21.15.78:0/1186025102 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686000b570 con 0x7f6868106d50 2024-09-11T01:23:33.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.502+0000 7f6870440640 1 -- 172.21.15.78:0/1186025102 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68681a5890 con 0x7f6868106d50 2024-09-11T01:23:33.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.502+0000 7f6870440640 1 -- 172.21.15.78:0/1186025102 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68681abf50 con 0x7f6868106d50 2024-09-11T01:23:33.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.506+0000 7f6872815740 1 Processor -- start 2024-09-11T01:23:33.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.507+0000 7f685f7fe640 1 -- 172.21.15.78:0/1186025102 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f686000b040 con 0x7f6868106d50 2024-09-11T01:23:33.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.507+0000 7f6872815740 1 -- start start 2024-09-11T01:23:33.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.507+0000 7f6872815740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.508+0000 7f6872815740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.508+0000 7f6872815740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.508+0000 7f6872815740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c7d4020e50 con 0x55c7d40c7f10 2024-09-11T01:23:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.508+0000 7f6872815740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c7d4010280 con 0x55c7d40aaf70 2024-09-11T01:23:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.508+0000 7f6872815740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7d4014090 con 0x55c7d40aa650 2024-09-11T01:23:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.508+0000 7f686e1dc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.508+0000 7f686e1dc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40c79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:33068/0 (socket says 172.21.15.78:33068) 2024-09-11T01:23:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.508+0000 7f686e1dc640 1 -- 172.21.15.78:0/3893031625 learned_addr learned my addr 172.21.15.78:0/3893031625 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.508+0000 7f686e1dc640 1 -- 172.21.15.78:0/3893031625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 msgr2=0x55c7d40aaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.508+0000 7f686e1dc640 1 --2- 172.21.15.78:0/3893031625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.509+0000 7f686e1dc640 1 -- 172.21.15.78:0/3893031625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 msgr2=0x55c7d40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:33.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.509+0000 7f686e1dc640 1 --2- 172.21.15.78:0/3893031625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.509+0000 7f686e1dc640 1 -- 172.21.15.78:0/3893031625 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7d3d6e650 con 0x55c7d40aaf70 2024-09-11T01:23:33.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.509+0000 7f685f7fe640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68540b56d0 0x7f68540b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.509+0000 7f686e1dc640 1 --2- 172.21.15.78:0/3893031625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40c79d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f686400bd90 tx=0x7f686400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbe42eb093dde6ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:33.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.509+0000 7f685f7fe640 1 -- 172.21.15.78:0/1186025102 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7f68600e5e00 con 0x7f6868106d50 2024-09-11T01:23:33.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.509+0000 7f685cde6640 1 -- 172.21.15.78:0/3893031625 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864015070 con 0x55c7d40aaf70 2024-09-11T01:23:33.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.509+0000 7f685cde6640 1 -- 172.21.15.78:0/3893031625 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68640035f0 con 0x55c7d40aaf70 2024-09-11T01:23:33.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.510+0000 7f686d9db640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68540b56d0 0x7f68540b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.510+0000 7f6872815740 1 -- 172.21.15.78:0/3893031625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 msgr2=0x55c7d40c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:33.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.510+0000 7f6872815740 1 --2- 172.21.15.78:0/3893031625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40c79d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f686400bd90 tx=0x7f686400f730 comp rx=0 tx=0).stop 2024-09-11T01:23:33.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.511+0000 7f686d9db640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68540b56d0 0x7f68540b7b90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f685802a930 tx=0x7f685800b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:33.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.511+0000 7f6872815740 1 -- 172.21.15.78:0/3893031625 shutdown_connections 2024-09-11T01:23:33.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.511+0000 7f6872815740 1 --2- 172.21.15.78:0/3893031625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.511+0000 7f6872815740 1 --2- 172.21.15.78:0/3893031625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40c79d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.511+0000 7f6872815740 1 --2- 172.21.15.78:0/3893031625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40aaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.511+0000 7f6872815740 1 -- 172.21.15.78:0/3893031625 >> 172.21.15.78:0/3893031625 conn(0x55c7d40a8480 msgr2=0x55c7d40a9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:33.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.511+0000 7f6872815740 1 -- 172.21.15.78:0/3893031625 shutdown_connections 2024-09-11T01:23:33.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.511+0000 7f6872815740 1 -- 172.21.15.78:0/3893031625 wait complete. 2024-09-11T01:23:33.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.511+0000 7f6872815740 1 -- start start 2024-09-11T01:23:33.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.512+0000 7f6872815740 1 Processor -- start 2024-09-11T01:23:33.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.512+0000 7f6872815740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.512+0000 7f6872815740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.512+0000 7f6872815740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.512+0000 7f6872815740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c7d3e71700 con 0x55c7d40c7f10 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.512+0000 7f6872815740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c7d3fab110 con 0x55c7d40aaf70 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.512+0000 7f6872815740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7d40cac10 con 0x55c7d40aa650 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.512+0000 7f686e1dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.512+0000 7f686e1dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:37670/0 (socket says 172.21.15.78:37670) 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.512+0000 7f686e1dc640 1 -- 172.21.15.78:0/4272416452 learned_addr learned my addr 172.21.15.78:0/4272416452 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.513+0000 7f6872815740 10 client.0 _ll_register_callbacks cb 0x55c7d3fc4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.513+0000 7f686e9dd640 1 --2- 172.21.15.78:0/4272416452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.513+0000 7f686e1dc640 1 -- 172.21.15.78:0/4272416452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 msgr2=0x55c7d40c4c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.513+0000 7f686e1dc640 1 --2- 172.21.15.78:0/4272416452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40c4c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.513+0000 7f686e1dc640 1 -- 172.21.15.78:0/4272416452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 msgr2=0x55c7d40c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.513+0000 7f686e1dc640 1 --2- 172.21.15.78:0/4272416452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.513+0000 7f686e1dc640 1 -- 172.21.15.78:0/4272416452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7d3fd8c40 con 0x55c7d40aa650 2024-09-11T01:23:33.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.513+0000 7f686e1dc640 1 --2- 172.21.15.78:0/4272416452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40c77b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6864006010 tx=0x7f68640045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=754b4cd8f0190249 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:33.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.513+0000 7f6851ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-11T01:23:33.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.514+0000 7f6851ffb640 1 -- 172.21.15.78:0/4272416452 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864015070 con 0x55c7d40aa650 2024-09-11T01:23:33.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.514+0000 7f6872815740 1 -- 172.21.15.78:0/4272416452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68640035f0 con 0x55c7d40aa650 2024-09-11T01:23:33.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.514+0000 7f6851ffb640 1 -- 172.21.15.78:0/4272416452 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7f686400fe40 con 0x55c7d40aa650 2024-09-11T01:23:33.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.514+0000 7f6851ffb640 1 -- 172.21.15.78:0/4272416452 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f6864011b10 con 0x55c7d40aa650 2024-09-11T01:23:33.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.515+0000 7f6872815740 10 client.28540 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:33.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.515+0000 7f6872815740 1 -- 172.21.15.78:0/4272416452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55c7d40d6530 con 0x55c7d40aa650 2024-09-11T01:23:33.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.515+0000 7f6851ffb640 1 -- 172.21.15.78:0/4272416452 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68640054e0 con 0x55c7d40aa650 2024-09-11T01:23:33.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.515+0000 7f6851ffb640 1 -- 172.21.15.78:0/4272416452 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f6864003bb0 con 0x55c7d40aa650 2024-09-11T01:23:33.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.515+0000 7f6872815740 10 client.28540 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:33.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.515+0000 7f6872815740 10 client.28540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:23:33.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.515+0000 7f6872815740 20 client.28540 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:33.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.515+0000 7f6872815740 1 --2- 172.21.15.78:0/4272416452 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c7d40d7c10 0x55c7d40da0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.515+0000 7f6872815740 4 client.28540 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:23:33.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.515+0000 7f6872815740 1 -- 172.21.15.78:0/4272416452 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f68640035f0 con 0x55c7d40d7c10 2024-09-11T01:23:33.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.516+0000 7f686d9db640 1 --2- 172.21.15.78:0/4272416452 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c7d40d7c10 0x55c7d40da0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.516+0000 7f6851ffb640 10 client.28540 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:33.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.516+0000 7f686d9db640 1 --2- 172.21.15.78:0/4272416452 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c7d40d7c10 0x55c7d40da0b0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ece345c2c6eed7 server_cookie=42d11da6529fcf1e in_seq=0 out_seq=0 2024-09-11T01:23:33.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.518+0000 7f6851ffb640 1 -- 172.21.15.78:0/4272416452 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f68640035f0 con 0x55c7d40d7c10 2024-09-11T01:23:33.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.518+0000 7f6851ffb640 10 client.28540 handle_command_reply: tid=0 2024-09-11T01:23:33.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.518+0000 7f6872815740 1 client.28540 shutdown 2024-09-11T01:23:33.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.518+0000 7f6872815740 1 -- 172.21.15.78:0/4272416452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 msgr2=0x55c7d40c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:33.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.519+0000 7f6872815740 1 --2- 172.21.15.78:0/4272416452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40c77b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6864006010 tx=0x7f68640045a0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.519+0000 7f6872815740 1 -- 172.21.15.78:0/4272416452 shutdown_connections 2024-09-11T01:23:33.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.519+0000 7f6872815740 1 --2- 172.21.15.78:0/4272416452 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c7d40d7c10 0x55c7d40da0b0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.519+0000 7f6872815740 1 --2- 172.21.15.78:0/4272416452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.519+0000 7f6872815740 1 --2- 172.21.15.78:0/4272416452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.519+0000 7f6872815740 1 --2- 172.21.15.78:0/4272416452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40c77b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.519+0000 7f6872815740 1 -- 172.21.15.78:0/4272416452 >> 172.21.15.78:0/4272416452 conn(0x55c7d40a8480 msgr2=0x55c7d40a7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:33.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.519+0000 7f6851ffb640 0 client.28540 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:33.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.519+0000 7f6872815740 1 -- 172.21.15.78:0/4272416452 shutdown_connections 2024-09-11T01:23:33.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.519+0000 7f6872815740 1 -- 172.21.15.78:0/4272416452 wait complete. 2024-09-11T01:23:33.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.520+0000 7f6872815740 20 client.28540 trim_cache size 0 max 16384 2024-09-11T01:23:33.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.549+0000 7f6872815740 1 Processor -- start 2024-09-11T01:23:33.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.549+0000 7f6872815740 1 -- start start 2024-09-11T01:23:33.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.549+0000 7f6872815740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.549+0000 7f6872815740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.549+0000 7f6872815740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.549+0000 7f6872815740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6868108f70 con 0x55c7d40c7f10 2024-09-11T01:23:33.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.549+0000 7f6872815740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f686810ab70 con 0x55c7d40aaf70 2024-09-11T01:23:33.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.549+0000 7f6872815740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6868109640 con 0x55c7d40aa650 2024-09-11T01:23:33.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.550+0000 7f686e1dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.550+0000 7f686d9db640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.550+0000 7f686d9db640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:33098/0 (socket says 172.21.15.78:33098) 2024-09-11T01:23:33.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.550+0000 7f686d9db640 1 -- 172.21.15.78:0/1449842578 learned_addr learned my addr 172.21.15.78:0/1449842578 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:33.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.550+0000 7f686d9db640 1 -- 172.21.15.78:0/1449842578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 msgr2=0x55c7d40ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:33.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.550+0000 7f686d9db640 1 --2- 172.21.15.78:0/1449842578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.550+0000 7f686d9db640 1 -- 172.21.15.78:0/1449842578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 msgr2=0x55c7d40a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:33.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.550+0000 7f686d9db640 1 --2- 172.21.15.78:0/1449842578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.550+0000 7f686d9db640 1 -- 172.21.15.78:0/1449842578 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7d40a3f70 con 0x55c7d40aaf70 2024-09-11T01:23:33.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.551+0000 7f686d9db640 1 --2- 172.21.15.78:0/1449842578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40a94d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f685800b230 tx=0x7f685803b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd008d1c6a623564 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.551+0000 7f6853fff640 1 -- 172.21.15.78:0/1449842578 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858037040 con 0x55c7d40aaf70 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.551+0000 7f6853fff640 1 -- 172.21.15.78:0/1449842578 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6858055070 con 0x55c7d40aaf70 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.551+0000 7f6853fff640 1 -- 172.21.15.78:0/1449842578 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858003bf0 con 0x55c7d40aaf70 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.552+0000 7f6872815740 1 -- 172.21.15.78:0/1449842578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 msgr2=0x55c7d40a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.552+0000 7f6872815740 1 --2- 172.21.15.78:0/1449842578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40a94d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f685800b230 tx=0x7f685803b000 comp rx=0 tx=0).stop 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.552+0000 7f6872815740 1 -- 172.21.15.78:0/1449842578 shutdown_connections 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.552+0000 7f6872815740 1 --2- 172.21.15.78:0/1449842578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.552+0000 7f6872815740 1 --2- 172.21.15.78:0/1449842578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40a94d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.552+0000 7f6872815740 1 --2- 172.21.15.78:0/1449842578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.552+0000 7f6872815740 1 -- 172.21.15.78:0/1449842578 >> 172.21.15.78:0/1449842578 conn(0x55c7d40a8480 msgr2=0x55c7d40c3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.552+0000 7f6872815740 1 -- 172.21.15.78:0/1449842578 shutdown_connections 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.552+0000 7f6872815740 1 -- 172.21.15.78:0/1449842578 wait complete. 2024-09-11T01:23:33.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f6872815740 1 -- start start 2024-09-11T01:23:33.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f6872815740 1 Processor -- start 2024-09-11T01:23:33.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f6872815740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40d0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f6872815740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40df520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f6872815740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f6872815740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f686810a4a0 con 0x55c7d40c7f10 2024-09-11T01:23:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f6872815740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6868109dd0 con 0x55c7d40aaf70 2024-09-11T01:23:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f6872815740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6868074720 con 0x55c7d40aa650 2024-09-11T01:23:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f6872815740 10 client.0 _ll_register_callbacks cb 0x7f686400bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:23:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f686e9dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f686e9dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40dfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43598/0 (socket says 172.21.15.78:43598) 2024-09-11T01:23:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f686e9dd640 1 -- 172.21.15.78:0/362264044 learned_addr learned my addr 172.21.15.78:0/362264044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.553+0000 7f686d9db640 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40df520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f686e1dc640 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40d0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f686e9dd640 1 -- 172.21.15.78:0/362264044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 msgr2=0x55c7d40d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f686e9dd640 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40d0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f686e9dd640 1 -- 172.21.15.78:0/362264044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 msgr2=0x55c7d40df520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f686e9dd640 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40df520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f686e9dd640 1 -- 172.21.15.78:0/362264044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7d40e01f0 con 0x55c7d40c7f10 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f686e1dc640 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f686d9db640 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40df520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f686e9dd640 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40dfa60 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f68600e3590 tx=0x7f68600ac0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583c89b67bf9935c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f6851ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f6851ffb640 1 -- 172.21.15.78:0/362264044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686001a070 con 0x55c7d40c7f10 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f6851ffb640 1 -- 172.21.15.78:0/362264044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7f68600d3170 con 0x55c7d40c7f10 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.554+0000 7f6872815740 1 -- 172.21.15.78:0/362264044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6858055070 con 0x55c7d40c7f10 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.555+0000 7f6851ffb640 1 -- 172.21.15.78:0/362264044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f6860004cc0 con 0x55c7d40c7f10 2024-09-11T01:23:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.555+0000 7f6872815740 10 client.19578 fetch_fsmap learned FSMap version 76 2024-09-11T01:23:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.555+0000 7f6872815740 1 -- 172.21.15.78:0/362264044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55c7d4144c00 con 0x55c7d40c7f10 2024-09-11T01:23:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.555+0000 7f6851ffb640 1 -- 172.21.15.78:0/362264044 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6858055070 con 0x55c7d40c7f10 2024-09-11T01:23:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.555+0000 7f6851ffb640 1 -- 172.21.15.78:0/362264044 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 76) ==== 8378+0+0 (secure 0 0 0) 0x7f68600b5360 con 0x55c7d40c7f10 2024-09-11T01:23:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.555+0000 7f6872815740 10 client.19578 fetch_fsmap finished waiting for FSMap version 76 2024-09-11T01:23:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.555+0000 7f6872815740 10 client.19578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:23:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.555+0000 7f6872815740 20 client.19578 populate_metadata read hostname 'smithi078' 2024-09-11T01:23:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.555+0000 7f6872815740 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c7d4145320 0x55c7d4147710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.555+0000 7f6872815740 4 client.19578 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:23:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.555+0000 7f6872815740 1 -- 172.21.15.78:0/362264044 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c7d4022da0 con 0x55c7d4145320 2024-09-11T01:23:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.555+0000 7f686e1dc640 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c7d4145320 0x55c7d4147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.556+0000 7f686e1dc640 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c7d4145320 0x55c7d4147710 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e00e9bfa3e131ee server_cookie=88c3bad4b13116c1 in_seq=0 out_seq=0 2024-09-11T01:23:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.556+0000 7f6851ffb640 10 client.19578 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:33.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.557+0000 7f6851ffb640 1 -- 172.21.15.78:0/362264044 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8522 (crc 0 0 0) 0x55c7d4022da0 con 0x55c7d4145320 2024-09-11T01:23:33.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.557+0000 7f6851ffb640 10 client.19578 handle_command_reply: tid=0 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.557+0000 7f6872815740 1 client.19578 shutdown 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.558+0000 7f6872815740 1 -- 172.21.15.78:0/362264044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 msgr2=0x55c7d40dfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.558+0000 7f6872815740 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40dfa60 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f68600e3590 tx=0x7f68600ac0f0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.558+0000 7f6872815740 1 -- 172.21.15.78:0/362264044 shutdown_connections 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.558+0000 7f6872815740 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55c7d4145320 0x55c7d4147710 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.558+0000 7f6872815740 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7d40c7f10 0x55c7d40dfa60 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.558+0000 7f6872815740 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7d40aaf70 0x55c7d40df520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.558+0000 7f6872815740 1 --2- 172.21.15.78:0/362264044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7d40aa650 0x55c7d40d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.558+0000 7f6872815740 1 -- 172.21.15.78:0/362264044 >> 172.21.15.78:0/362264044 conn(0x55c7d40a8480 msgr2=0x55c7d40a94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.558+0000 7f6851ffb640 0 client.19578 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.558+0000 7f6872815740 1 -- 172.21.15.78:0/362264044 shutdown_connections 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.558+0000 7f6872815740 1 -- 172.21.15.78:0/362264044 wait complete. 2024-09-11T01:23:33.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.558+0000 7f6872815740 20 client.19578 trim_cache size 0 max 16384 2024-09-11T01:23:33.563 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:23:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T01:23:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-11T01:23:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "6eb6da3d-724b-47e6-a25c-a6dcc447ffc7": { 2024-09-11T01:23:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "6eb6da3d-724b-47e6-a25c-a6dcc447ffc7", 2024-09-11T01:23:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-11T01:23:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:23:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "6eb6da3d-724b-47e6-a25c-a6dcc447ffc7": { 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:23:31.655194+0000", 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "82736ae4-b55a-4ba3-911c-0aa1c18acfde": { 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:22:21.932379+0000", 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "746add9a-5be3-4745-8000-04e5471bc813": { 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:20:12.369593+0000", 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "b41a2b77-b2b1-4f1b-8588-4f32a78b721f": { 2024-09-11T01:23:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:19:13.257610+0000", 2024-09-11T01:23:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:23:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:18:17.654570+0000", 2024-09-11T01:23:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:23:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:23:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:23:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:23:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:23:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:23:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "291fcaa4-1805-423d-a0bc-814f1bc2759f": { 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:21:17.256483+0000", 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:23:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:23:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:23:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:23:33.582 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:23:33.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.559+0000 7f6870440640 1 -- 172.21.15.78:0/1186025102 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68540b56d0 msgr2=0x7f68540b7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:33.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.559+0000 7f6870440640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68540b56d0 0x7f68540b7b90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f685802a930 tx=0x7f685800b040 comp rx=0 tx=0).stop 2024-09-11T01:23:33.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.559+0000 7f6870440640 1 -- 172.21.15.78:0/1186025102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6868106d50 msgr2=0x7f68681a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:33.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.559+0000 7f6870440640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6868106d50 0x7f68681a4dd0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f686000ef00 tx=0x7f686000c740 comp rx=0 tx=0).stop 2024-09-11T01:23:33.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.559+0000 7f6870440640 1 -- 172.21.15.78:0/1186025102 shutdown_connections 2024-09-11T01:23:33.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.559+0000 7f6870440640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f68540b56d0 0x7f68540b7b90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.559+0000 7f6870440640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6868106d50 0x7f68681a4dd0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.559+0000 7f6870440640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6868102230 0x7f686819fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.559+0000 7f6870440640 1 --2- 172.21.15.78:0/1186025102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6868101910 0x7f686819f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:33.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.559+0000 7f6870440640 1 -- 172.21.15.78:0/1186025102 >> 172.21.15.78:0/1186025102 conn(0x7f68680fd580 msgr2=0x7f68681046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:33.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.559+0000 7f6870440640 1 -- 172.21.15.78:0/1186025102 shutdown_connections 2024-09-11T01:23:33.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:33.559+0000 7f6870440640 1 -- 172.21.15.78:0/1186025102 wait complete. 2024-09-11T01:23:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:33 smithi078 ceph-mon[31315]: scrub summary: active paths [/] 2024-09-11T01:23:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:33 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:33 smithi078 ceph-mon[31315]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:33.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:33 smithi078 ceph-mon[31315]: pgmap v1099: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 623 op/s 2024-09-11T01:23:33.815 INFO:tasks.workunit.client.0.smithi078.stdout: 10 416817 31.37 MB/sec execute 579 sec latency 140.845 ms 2024-09-11T01:23:33.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:33 smithi110 ceph-mon[34712]: scrub summary: active paths [/] 2024-09-11T01:23:33.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:33 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:33.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:33 smithi110 ceph-mon[34712]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:33.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:33 smithi110 ceph-mon[34712]: pgmap v1099: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 623 op/s 2024-09-11T01:23:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:33 smithi112 ceph-mon[35384]: scrub summary: active paths [/] 2024-09-11T01:23:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:33 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:33 smithi112 ceph-mon[35384]: scrub summary: idle+waiting paths [/] 2024-09-11T01:23:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:33 smithi112 ceph-mon[35384]: pgmap v1099: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 623 op/s 2024-09-11T01:23:34.438 INFO:tasks.cephfs.filesystem:scrub status for tag:6eb6da3d-724b-47e6-a25c-a6dcc447ffc7 - {'path': '/', 'tag': '6eb6da3d-724b-47e6-a25c-a6dcc447ffc7', 'options': 'recursive,force'} 2024-09-11T01:23:34.815 INFO:tasks.workunit.client.0.smithi078.stdout: 10 417458 31.36 MB/sec execute 580 sec latency 76.630 ms 2024-09-11T01:23:35.815 INFO:tasks.workunit.client.0.smithi078.stdout: 10 418175 31.40 MB/sec execute 581 sec latency 68.733 ms 2024-09-11T01:23:35.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:35 smithi110 ceph-mon[34712]: pgmap v1100: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 709 op/s 2024-09-11T01:23:35.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:35 smithi112 ceph-mon[35384]: pgmap v1100: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 709 op/s 2024-09-11T01:23:35.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:35 smithi078 ceph-mon[31315]: pgmap v1100: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 709 op/s 2024-09-11T01:23:36.815 INFO:tasks.workunit.client.0.smithi078.stdout: 10 418905 31.40 MB/sec execute 582 sec latency 72.632 ms 2024-09-11T01:23:37.815 INFO:tasks.workunit.client.0.smithi078.stdout: 10 419236 31.38 MB/sec execute 583 sec latency 420.492 ms 2024-09-11T01:23:37.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:37 smithi110 ceph-mon[34712]: pgmap v1101: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 775 op/s 2024-09-11T01:23:37.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:37 smithi112 ceph-mon[35384]: pgmap v1101: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 775 op/s 2024-09-11T01:23:37.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:37 smithi078 ceph-mon[31315]: pgmap v1101: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 775 op/s 2024-09-11T01:23:38.815 INFO:tasks.workunit.client.0.smithi078.stdout: 10 419703 31.38 MB/sec execute 584 sec latency 183.425 ms 2024-09-11T01:23:39.815 INFO:tasks.workunit.client.0.smithi078.stdout: 10 420050 31.35 MB/sec execute 585 sec latency 147.545 ms 2024-09-11T01:23:39.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:39 smithi110 ceph-mon[34712]: pgmap v1102: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 561 op/s 2024-09-11T01:23:39.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:39 smithi112 ceph-mon[35384]: pgmap v1102: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 561 op/s 2024-09-11T01:23:39.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:39 smithi078 ceph-mon[31315]: pgmap v1102: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 561 op/s 2024-09-11T01:23:40.815 INFO:tasks.workunit.client.0.smithi078.stdout: 10 420603 31.35 MB/sec execute 586 sec latency 95.243 ms 2024-09-11T01:23:41.815 INFO:tasks.workunit.client.0.smithi078.stdout: 10 421219 31.34 MB/sec execute 587 sec latency 106.499 ms 2024-09-11T01:23:41.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:41 smithi110 ceph-mon[34712]: pgmap v1103: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 17 MiB/s wr, 733 op/s 2024-09-11T01:23:41.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:41 smithi112 ceph-mon[35384]: pgmap v1103: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 17 MiB/s wr, 733 op/s 2024-09-11T01:23:41.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:41 smithi078 ceph-mon[31315]: pgmap v1103: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 17 MiB/s wr, 733 op/s 2024-09-11T01:23:42.815 INFO:tasks.workunit.client.0.smithi078.stdout: 10 421782 31.35 MB/sec execute 588 sec latency 77.135 ms 2024-09-11T01:23:42.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:42 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:23:42.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:42 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:23:42.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:42 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:23:42.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:42 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:23:42.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:42 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:23:42.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:42 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:23:42.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:42 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:23:42.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:42 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:23:42.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:42 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:23:43.816 INFO:tasks.workunit.client.0.smithi078.stdout: 10 422545 31.35 MB/sec execute 589 sec latency 88.913 ms 2024-09-11T01:23:43.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:43 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:23:43.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:43 smithi110 ceph-mon[34712]: scrub summary: idle 2024-09-11T01:23:43.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:43 smithi110 ceph-mon[34712]: pgmap v1104: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 718 op/s 2024-09-11T01:23:43.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:43 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:23:43.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:43 smithi112 ceph-mon[35384]: scrub summary: idle 2024-09-11T01:23:43.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:43 smithi112 ceph-mon[35384]: pgmap v1104: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 718 op/s 2024-09-11T01:23:43.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:43 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:23:43.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:43 smithi078 ceph-mon[31315]: scrub summary: idle 2024-09-11T01:23:43.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:43 smithi078 ceph-mon[31315]: pgmap v1104: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 718 op/s 2024-09-11T01:23:44.816 INFO:tasks.workunit.client.0.smithi078.stdout: 10 423265 31.38 MB/sec execute 590 sec latency 83.387 ms 2024-09-11T01:23:45.816 INFO:tasks.workunit.client.0.smithi078.stdout: 10 423848 31.38 MB/sec execute 591 sec latency 75.854 ms 2024-09-11T01:23:45.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:45 smithi110 ceph-mon[34712]: pgmap v1105: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 664 op/s 2024-09-11T01:23:45.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:45 smithi112 ceph-mon[35384]: pgmap v1105: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 664 op/s 2024-09-11T01:23:45.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:45 smithi078 ceph-mon[31315]: pgmap v1105: 97 pgs: 97 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 664 op/s 2024-09-11T01:23:46.816 INFO:tasks.workunit.client.0.smithi078.stdout: 10 424412 31.38 MB/sec execute 592 sec latency 72.046 ms 2024-09-11T01:23:47.816 INFO:tasks.workunit.client.0.smithi078.stdout: 10 424979 31.37 MB/sec execute 593 sec latency 112.475 ms 2024-09-11T01:23:47.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:47 smithi110 ceph-mon[34712]: pgmap v1106: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 694 op/s 2024-09-11T01:23:47.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:47 smithi112 ceph-mon[35384]: pgmap v1106: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 694 op/s 2024-09-11T01:23:47.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:47 smithi078 ceph-mon[31315]: pgmap v1106: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 694 op/s 2024-09-11T01:23:48.816 INFO:tasks.workunit.client.0.smithi078.stdout: 10 425524 31.36 MB/sec execute 594 sec latency 171.959 ms 2024-09-11T01:23:48.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:48 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:23:48.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:48 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:23:48.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:48 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:23:48.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:48 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:23:48.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:48 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:23:48.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:48 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:23:48.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:48 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:23:48.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:48 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:23:48.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:48 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:23:48.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:48 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:23:48.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:48 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:23:48.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:48 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:23:49.816 INFO:tasks.workunit.client.0.smithi078.stdout: 10 426205 31.38 MB/sec execute 595 sec latency 126.250 ms 2024-09-11T01:23:50.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:50 smithi110 ceph-mon[34712]: pgmap v1107: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 507 op/s 2024-09-11T01:23:50.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:50 smithi112 ceph-mon[35384]: pgmap v1107: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 507 op/s 2024-09-11T01:23:50.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:50 smithi078 ceph-mon[31315]: pgmap v1107: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 507 op/s 2024-09-11T01:23:50.816 INFO:tasks.workunit.client.0.smithi078.stdout: 10 426971 31.43 MB/sec execute 596 sec latency 85.743 ms 2024-09-11T01:23:51.816 INFO:tasks.workunit.client.0.smithi078.stdout: 10 427538 31.44 MB/sec execute 597 sec latency 70.505 ms 2024-09-11T01:23:52.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:52 smithi110 ceph-mon[34712]: pgmap v1108: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 645 op/s 2024-09-11T01:23:52.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:52 smithi112 ceph-mon[35384]: pgmap v1108: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 645 op/s 2024-09-11T01:23:52.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:52 smithi078 ceph-mon[31315]: pgmap v1108: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 16 MiB/s wr, 645 op/s 2024-09-11T01:23:52.816 INFO:tasks.workunit.client.0.smithi078.stdout: 10 428063 31.42 MB/sec execute 598 sec latency 75.417 ms 2024-09-11T01:23:53.816 INFO:tasks.workunit.client.0.smithi078.stdout: 10 428646 31.41 MB/sec execute 599 sec latency 92.562 ms 2024-09-11T01:23:54.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:54 smithi110 ceph-mon[34712]: pgmap v1109: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 556 op/s 2024-09-11T01:23:54.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:54 smithi112 ceph-mon[35384]: pgmap v1109: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 556 op/s 2024-09-11T01:23:54.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:54 smithi078 ceph-mon[31315]: pgmap v1109: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 556 op/s 2024-09-11T01:23:54.816 INFO:tasks.workunit.client.0.smithi078.stdout: 10 cleanup 600 sec 2024-09-11T01:23:55.343 INFO:tasks.workunit.client.0.smithi078.stdout: 0 cleanup 601 sec 2024-09-11T01:23:55.343 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-09-11T01:23:55.343 INFO:tasks.workunit.client.0.smithi078.stdout: Operation Count AvgLat MaxLat 2024-09-11T01:23:55.343 INFO:tasks.workunit.client.0.smithi078.stdout: ---------------------------------------- 2024-09-11T01:23:55.343 INFO:tasks.workunit.client.0.smithi078.stdout: NTCreateX 604504 4.131 4670.134 2024-09-11T01:23:55.343 INFO:tasks.workunit.client.0.smithi078.stdout: Close 443874 0.003 0.193 2024-09-11T01:23:55.343 INFO:tasks.workunit.client.0.smithi078.stdout: Rename 25594 18.570 4679.437 2024-09-11T01:23:55.343 INFO:tasks.workunit.client.0.smithi078.stdout: Unlink 122166 4.573 11442.819 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: Deltree 4 315.902 743.416 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: Mkdir 2 0.008 0.009 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: Qpathinfo 548133 3.156 10268.044 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: Qfileinfo 95639 0.001 7.269 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: Qfsinfo 100372 0.262 108.432 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: Sfileinfo 49311 0.202 861.510 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: Find 211681 0.383 8296.615 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: WriteX 299246 0.312 1035.865 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: ReadX 946689 0.032 606.300 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: LockX 1964 2.137 33.699 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: UnlockX 1964 1.798 48.030 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: Flush 42389 11.354 1136.509 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-09-11T01:23:55.344 INFO:tasks.workunit.client.0.smithi078.stdout:Throughput 31.4144 MB/sec 10 clients 10 procs max_latency=11442.825 ms 2024-09-11T01:23:55.345 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T01:23:55.345 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T01:23:56.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:56 smithi110 ceph-mon[34712]: pgmap v1110: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 517 op/s 2024-09-11T01:23:56.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:56 smithi112 ceph-mon[35384]: pgmap v1110: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 517 op/s 2024-09-11T01:23:56.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:56 smithi078 ceph-mon[31315]: pgmap v1110: 97 pgs: 97 active+clean; 2.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 517 op/s 2024-09-11T01:23:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:58 smithi110 ceph-mon[34712]: pgmap v1111: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 14 MiB/s wr, 581 op/s 2024-09-11T01:23:58.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:58 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:23:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:58 smithi112 ceph-mon[35384]: pgmap v1111: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 14 MiB/s wr, 581 op/s 2024-09-11T01:23:58.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:58 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:23:58.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:58 smithi078 ceph-mon[31315]: pgmap v1111: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 14 MiB/s wr, 581 op/s 2024-09-11T01:23:58.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:58 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:23:58.773 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-11T01:23:58.773 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T01:23:59.101 DEBUG:teuthology.parallel:result is None 2024-09-11T01:23:59.101 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T01:23:59.140 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T01:23:59.140 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-11T01:23:59.151 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-11T01:23:59.151 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T01:23:59.151 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T01:23:59.151 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-11T01:23:59.226 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-11T01:23:59.252 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T01:23:59.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:23:59 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:23:59.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 -- 172.21.15.78:0/2444642787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1d8107f70 msgr2=0x7fc1d810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:59.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 --2- 172.21.15.78:0/2444642787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1d8107f70 0x7fc1d810e770 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009ca0 tx=0x7fc1d002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 -- 172.21.15.78:0/2444642787 shutdown_connections 2024-09-11T01:23:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 --2- 172.21.15.78:0/2444642787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1d8107f70 0x7fc1d810e770 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 --2- 172.21.15.78:0/2444642787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1d81075f0 0x7fc1d8107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 --2- 172.21.15.78:0/2444642787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d8101bf0 0x7fc1d8101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 -- 172.21.15.78:0/2444642787 >> 172.21.15.78:0/2444642787 conn(0x7fc1d80fd630 msgr2=0x7fc1d80ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 -- 172.21.15.78:0/2444642787 shutdown_connections 2024-09-11T01:23:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 -- 172.21.15.78:0/2444642787 wait complete. 2024-09-11T01:23:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 Processor -- start 2024-09-11T01:23:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 -- start start 2024-09-11T01:23:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d8101bf0 0x7fc1d8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1d81075f0 0x7fc1d81997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1d8107f70 0x7fc1d819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8111c00 con 0x7fc1d8107f70 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8111a80 con 0x7fc1d81075f0 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.547+0000 7fc1de17d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8111d80 con 0x7fc1d8101bf0 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1d8107f70 0x7fc1d819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1d8107f70 0x7fc1d819eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49910/0 (socket says 172.21.15.78:49910) 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d7fff640 1 -- 172.21.15.78:0/2553931555 learned_addr learned my addr 172.21.15.78:0/2553931555 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d77fe640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d8101bf0 0x7fc1d8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1cffff640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1d81075f0 0x7fc1d81997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d7fff640 1 -- 172.21.15.78:0/2553931555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d8101bf0 msgr2=0x7fc1d8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d7fff640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d8101bf0 0x7fc1d8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d7fff640 1 -- 172.21.15.78:0/2553931555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1d81075f0 msgr2=0x7fc1d81997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d7fff640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1d81075f0 0x7fc1d81997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d7fff640 1 -- 172.21.15.78:0/2553931555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d819f310 con 0x7fc1d8107f70 2024-09-11T01:23:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d7fff640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1d8107f70 0x7fc1d819eb40 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0005cc0 tx=0x7fc1d002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d57fa640 1 -- 172.21.15.78:0/2553931555 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0003bf0 con 0x7fc1d8107f70 2024-09-11T01:23:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d57fa640 1 -- 172.21.15.78:0/2553931555 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d0003d90 con 0x7fc1d8107f70 2024-09-11T01:23:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1d57fa640 1 -- 172.21.15.78:0/2553931555 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d002fbc0 con 0x7fc1d8107f70 2024-09-11T01:23:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1de17d640 1 -- 172.21.15.78:0/2553931555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d819f5a0 con 0x7fc1d8107f70 2024-09-11T01:23:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.548+0000 7fc1de17d640 1 -- 172.21.15.78:0/2553931555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d81a5cc0 con 0x7fc1d8107f70 2024-09-11T01:23:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.549+0000 7fc1de17d640 1 -- 172.21.15.78:0/2553931555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d8101fd0 con 0x7fc1d8107f70 2024-09-11T01:23:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.551+0000 7fc1d57fa640 1 -- 172.21.15.78:0/2553931555 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc1d0004610 con 0x7fc1d8107f70 2024-09-11T01:23:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.552+0000 7fc1d57fa640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc1c40b5450 0x7fc1c40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.552+0000 7fc1d77fe640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc1c40b5450 0x7fc1c40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.552+0000 7fc1d57fa640 1 -- 172.21.15.78:0/2553931555 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7fc1d0107610 con 0x7fc1d8107f70 2024-09-11T01:23:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.552+0000 7fc1d77fe640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc1c40b5450 0x7fc1c40b7910 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800bbe0 tx=0x7fc1c800b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:59.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.555+0000 7fc1d57fa640 1 -- 172.21.15.78:0/2553931555 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc1d00ce1d0 con 0x7fc1d8107f70 2024-09-11T01:23:59.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:23:59 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:23:59.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:23:59 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:23:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.754+0000 7fc1de17d640 1 -- 172.21.15.78:0/2553931555 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fc1d806b370 con 0x7fc1c40b5450 2024-09-11T01:23:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.772+0000 7fc1d57fa640 1 -- 172.21.15.78:0/2553931555 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fc1d806b370 con 0x7fc1c40b5450 2024-09-11T01:23:59.775 INFO:teuthology.orchestra.run.smithi078.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T00:58:57", "first": "2024-09-11T00:59:00", "last": "2024-09-11T01:23:00", "last_pruned": "2024-09-11T01:23:01", "created_count": 25, "pruned_count": 18, "active": true} 2024-09-11T01:23:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.774+0000 7fc1de17d640 1 -- 172.21.15.78:0/2553931555 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc1c40b5450 msgr2=0x7fc1c40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.774+0000 7fc1de17d640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc1c40b5450 0x7fc1c40b7910 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800bbe0 tx=0x7fc1c800b390 comp rx=0 tx=0).stop 2024-09-11T01:23:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.774+0000 7fc1de17d640 1 -- 172.21.15.78:0/2553931555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1d8107f70 msgr2=0x7fc1d819eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.774+0000 7fc1de17d640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1d8107f70 0x7fc1d819eb40 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0005cc0 tx=0x7fc1d002f380 comp rx=0 tx=0).stop 2024-09-11T01:23:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.774+0000 7fc1de17d640 1 -- 172.21.15.78:0/2553931555 shutdown_connections 2024-09-11T01:23:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.774+0000 7fc1de17d640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fc1c40b5450 0x7fc1c40b7910 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.774+0000 7fc1de17d640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1d8107f70 0x7fc1d819eb40 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.774+0000 7fc1de17d640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1d81075f0 0x7fc1d81997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.774+0000 7fc1de17d640 1 --2- 172.21.15.78:0/2553931555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d8101bf0 0x7fc1d8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.774+0000 7fc1de17d640 1 -- 172.21.15.78:0/2553931555 >> 172.21.15.78:0/2553931555 conn(0x7fc1d80fd630 msgr2=0x7fc1d8104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.774+0000 7fc1de17d640 1 -- 172.21.15.78:0/2553931555 shutdown_connections 2024-09-11T01:23:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.774+0000 7fc1de17d640 1 -- 172.21.15.78:0/2553931555 wait complete. 2024-09-11T01:23:59.788 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T01:23:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.886+0000 7ff866dcd640 1 -- 172.21.15.78:0/327692933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8601054b0 msgr2=0x7ff8601058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.886+0000 7ff866dcd640 1 --2- 172.21.15.78:0/327692933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8601054b0 0x7ff8601058f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ff854009900 tx=0x7ff85402a410 comp rx=0 tx=0).stop 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.886+0000 7ff866dcd640 1 -- 172.21.15.78:0/327692933 shutdown_connections 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.886+0000 7ff866dcd640 1 --2- 172.21.15.78:0/327692933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff860105e30 0x7ff86010f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.886+0000 7ff866dcd640 1 --2- 172.21.15.78:0/327692933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8601054b0 0x7ff8601058f0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.886+0000 7ff866dcd640 1 --2- 172.21.15.78:0/327692933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff86006aaa0 0x7ff86006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.886+0000 7ff866dcd640 1 -- 172.21.15.78:0/327692933 >> 172.21.15.78:0/327692933 conn(0x7ff860077720 msgr2=0x7ff860077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.886+0000 7ff866dcd640 1 -- 172.21.15.78:0/327692933 shutdown_connections 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.886+0000 7ff866dcd640 1 -- 172.21.15.78:0/327692933 wait complete. 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff866dcd640 1 Processor -- start 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff866dcd640 1 -- start start 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff866dcd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86006aaa0 0x7ff8601992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff866dcd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8601054b0 0x7ff8601997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff866dcd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff860105e30 0x7ff86019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff866dcd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff860112d30 con 0x7ff8601054b0 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff866dcd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff860112bb0 con 0x7ff860105e30 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff866dcd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff860112eb0 con 0x7ff86006aaa0 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff85ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8601054b0 0x7ff8601997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff85ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8601054b0 0x7ff8601997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49934/0 (socket says 172.21.15.78:49934) 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff85ffff640 1 -- 172.21.15.78:0/950205305 learned_addr learned my addr 172.21.15.78:0/950205305 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff86536a640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff860105e30 0x7ff86019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.887+0000 7ff864b69640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86006aaa0 0x7ff8601992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.888+0000 7ff85ffff640 1 -- 172.21.15.78:0/950205305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86006aaa0 msgr2=0x7ff8601992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.888+0000 7ff85ffff640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86006aaa0 0x7ff8601992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.888+0000 7ff85ffff640 1 -- 172.21.15.78:0/950205305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff860105e30 msgr2=0x7ff86019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.888+0000 7ff85ffff640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff860105e30 0x7ff86019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.888+0000 7ff85ffff640 1 -- 172.21.15.78:0/950205305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86019f350 con 0x7ff8601054b0 2024-09-11T01:23:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.888+0000 7ff86536a640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff860105e30 0x7ff86019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:23:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.888+0000 7ff85ffff640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8601054b0 0x7ff8601997f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff854005940 tx=0x7ff8540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.888+0000 7ff85dffb640 1 -- 172.21.15.78:0/950205305 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85403f030 con 0x7ff8601054b0 2024-09-11T01:23:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.888+0000 7ff85dffb640 1 -- 172.21.15.78:0/950205305 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff854003cb0 con 0x7ff8601054b0 2024-09-11T01:23:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.888+0000 7ff866dcd640 1 -- 172.21.15.78:0/950205305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86019f5e0 con 0x7ff8601054b0 2024-09-11T01:23:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.888+0000 7ff85dffb640 1 -- 172.21.15.78:0/950205305 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854035070 con 0x7ff8601054b0 2024-09-11T01:23:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.888+0000 7ff866dcd640 1 -- 172.21.15.78:0/950205305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8601a5d00 con 0x7ff8601054b0 2024-09-11T01:23:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.889+0000 7ff866dcd640 1 -- 172.21.15.78:0/950205305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff86006ae80 con 0x7ff8601054b0 2024-09-11T01:23:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.891+0000 7ff85dffb640 1 -- 172.21.15.78:0/950205305 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff85403bbf0 con 0x7ff8601054b0 2024-09-11T01:23:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.892+0000 7ff85dffb640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8480b5450 0x7ff8480b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:23:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.892+0000 7ff864b69640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8480b5450 0x7ff8480b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:23:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.892+0000 7ff85dffb640 1 -- 172.21.15.78:0/950205305 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8449+0+0 (secure 0 0 0) 0x7ff854106e50 con 0x7ff8601054b0 2024-09-11T01:23:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.892+0000 7ff864b69640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8480b5450 0x7ff8480b7910 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff850005fd0 tx=0x7ff850005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:23:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:23:59.895+0000 7ff85dffb640 1 -- 172.21.15.78:0/950205305 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff8540cda10 con 0x7ff8601054b0 2024-09-11T01:24:00.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.090+0000 7ff866dcd640 1 -- 172.21.15.78:0/950205305 --> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ff860063590 con 0x7ff8480b5450 2024-09-11T01:24:00.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.105+0000 7ff85dffb640 1 -- 172.21.15.78:0/950205305 <== mgr.14217 v2:172.21.15.78:6800/1082762376 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ff860063590 con 0x7ff8480b5450 2024-09-11T01:24:00.108 INFO:teuthology.orchestra.run.smithi078.stdout:/ 1m 6m3h 2024-09-11T01:24:00.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.107+0000 7ff866dcd640 1 -- 172.21.15.78:0/950205305 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8480b5450 msgr2=0x7ff8480b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:00.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.107+0000 7ff866dcd640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8480b5450 0x7ff8480b7910 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff850005fd0 tx=0x7ff850005850 comp rx=0 tx=0).stop 2024-09-11T01:24:00.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.107+0000 7ff866dcd640 1 -- 172.21.15.78:0/950205305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8601054b0 msgr2=0x7ff8601997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:00.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.107+0000 7ff866dcd640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8601054b0 0x7ff8601997f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff854005940 tx=0x7ff8540042f0 comp rx=0 tx=0).stop 2024-09-11T01:24:00.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.107+0000 7ff866dcd640 1 -- 172.21.15.78:0/950205305 shutdown_connections 2024-09-11T01:24:00.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.107+0000 7ff866dcd640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7ff8480b5450 0x7ff8480b7910 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:00.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.107+0000 7ff866dcd640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff860105e30 0x7ff86019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:00.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.107+0000 7ff866dcd640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8601054b0 0x7ff8601997f0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:00.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.107+0000 7ff866dcd640 1 --2- 172.21.15.78:0/950205305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86006aaa0 0x7ff8601992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:00.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.107+0000 7ff866dcd640 1 -- 172.21.15.78:0/950205305 >> 172.21.15.78:0/950205305 conn(0x7ff860077720 msgr2=0x7ff8600698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:00.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.107+0000 7ff866dcd640 1 -- 172.21.15.78:0/950205305 shutdown_connections 2024-09-11T01:24:00.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:00.107+0000 7ff866dcd640 1 -- 172.21.15.78:0/950205305 wait complete. 2024-09-11T01:24:00.121 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-11T01:24:00.131 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-11T01:24:00.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:00 smithi078 ceph-mon[31315]: pgmap v1112: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 429 op/s 2024-09-11T01:24:00.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:00 smithi110 ceph-mon[34712]: pgmap v1112: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 429 op/s 2024-09-11T01:24:00.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:00 smithi112 ceph-mon[35384]: pgmap v1112: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 429 op/s 2024-09-11T01:24:01.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:01 smithi110 ceph-mon[34712]: from='client.19584 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T01:24:01.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:01 smithi110 ceph-mon[34712]: from='client.19590 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T01:24:01.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:01 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:24:01.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:01 smithi112 ceph-mon[35384]: from='client.19584 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T01:24:01.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:01 smithi112 ceph-mon[35384]: from='client.19590 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T01:24:01.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:01 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:24:01.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:01 smithi078 ceph-mon[31315]: from='client.19584 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T01:24:01.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:01 smithi078 ceph-mon[31315]: from='client.19590 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T01:24:01.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:01 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:24:02.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:02 smithi110 ceph-mon[34712]: pgmap v1113: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 643 op/s 2024-09-11T01:24:02.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:02 smithi112 ceph-mon[35384]: pgmap v1113: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 643 op/s 2024-09-11T01:24:02.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:02 smithi078 ceph-mon[31315]: pgmap v1113: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 643 op/s 2024-09-11T01:24:04.439 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T01:24:04.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:04 smithi110 ceph-mon[34712]: pgmap v1114: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 643 op/s 2024-09-11T01:24:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:04.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:04 smithi112 ceph-mon[35384]: pgmap v1114: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 643 op/s 2024-09-11T01:24:04.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:04 smithi078 ceph-mon[31315]: pgmap v1114: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 643 op/s 2024-09-11T01:24:05.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:05 smithi078 ceph-mon[31315]: pgmap v1115: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 566 op/s 2024-09-11T01:24:05.690 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:05 smithi110 ceph-mon[34712]: pgmap v1115: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 566 op/s 2024-09-11T01:24:05.724 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:05 smithi112 ceph-mon[35384]: pgmap v1115: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 566 op/s 2024-09-11T01:24:07.885 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:07 smithi078 ceph-mon[31315]: pgmap v1116: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 499 op/s 2024-09-11T01:24:07.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.883+0000 7f728e08d640 1 -- 172.21.15.78:0/1884979753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72881076a0 msgr2=0x7f7288107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:07.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.883+0000 7f728e08d640 1 --2- 172.21.15.78:0/1884979753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72881076a0 0x7f7288107ae0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f727c009860 tx=0x7f727c02a340 comp rx=0 tx=0).stop 2024-09-11T01:24:07.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.883+0000 7f728e08d640 1 -- 172.21.15.78:0/1884979753 shutdown_connections 2024-09-11T01:24:07.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.883+0000 7f728e08d640 1 --2- 172.21.15.78:0/1884979753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7288108020 0x7f728810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.883+0000 7f728e08d640 1 --2- 172.21.15.78:0/1884979753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72881076a0 0x7f7288107ae0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.883+0000 7f728e08d640 1 --2- 172.21.15.78:0/1884979753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7288101c90 0x7f7288102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.883+0000 7f728e08d640 1 -- 172.21.15.78:0/1884979753 >> 172.21.15.78:0/1884979753 conn(0x7f72880fd700 msgr2=0x7f72880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:07.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.883+0000 7f728e08d640 1 -- 172.21.15.78:0/1884979753 shutdown_connections 2024-09-11T01:24:07.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.883+0000 7f728e08d640 1 -- 172.21.15.78:0/1884979753 wait complete. 2024-09-11T01:24:07.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.883+0000 7f728e08d640 1 Processor -- start 2024-09-11T01:24:07.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f728e08d640 1 -- start start 2024-09-11T01:24:07.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f728e08d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7288101c90 0x7f728819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f728e08d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72881076a0 0x7f728819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f728e08d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7288108020 0x7f72881a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f728e08d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7288111c70 con 0x7f7288108020 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f728e08d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7288111af0 con 0x7f7288101c90 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f728e08d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7288111df0 con 0x7f72881076a0 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f7287fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7288108020 0x7f72881a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f7287fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7288108020 0x7f72881a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43380/0 (socket says 172.21.15.78:43380) 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f7287fff640 1 -- 172.21.15.78:0/660226447 learned_addr learned my addr 172.21.15.78:0/660226447 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f7286ffd640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72881076a0 0x7f728819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f72877fe640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7288101c90 0x7f728819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f7287fff640 1 -- 172.21.15.78:0/660226447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72881076a0 msgr2=0x7f728819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f7287fff640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72881076a0 0x7f728819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f7287fff640 1 -- 172.21.15.78:0/660226447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7288101c90 msgr2=0x7f728819b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:07.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f7287fff640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7288101c90 0x7f728819b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f7287fff640 1 -- 172.21.15.78:0/660226447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72881a1540 con 0x7f7288108020 2024-09-11T01:24:07.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.884+0000 7f72877fe640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7288101c90 0x7f728819b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:24:07.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.885+0000 7f7287fff640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7288108020 0x7f72881a0d70 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f728000efc0 tx=0x7f728000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:07.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.885+0000 7f7284ff9640 1 -- 172.21.15.78:0/660226447 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280003970 con 0x7f7288108020 2024-09-11T01:24:07.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.885+0000 7f728e08d640 1 -- 172.21.15.78:0/660226447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72881a1830 con 0x7f7288108020 2024-09-11T01:24:07.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.885+0000 7f7284ff9640 1 -- 172.21.15.78:0/660226447 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7280004020 con 0x7f7288108020 2024-09-11T01:24:07.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.885+0000 7f7284ff9640 1 -- 172.21.15.78:0/660226447 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728000b940 con 0x7f7288108020 2024-09-11T01:24:07.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.885+0000 7f728e08d640 1 -- 172.21.15.78:0/660226447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72881a7ef0 con 0x7f7288108020 2024-09-11T01:24:07.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.888+0000 7f7284ff9640 1 -- 172.21.15.78:0/660226447 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f728000b040 con 0x7f7288108020 2024-09-11T01:24:07.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.888+0000 7f7290462740 1 Processor -- start 2024-09-11T01:24:07.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.888+0000 7f7290462740 1 -- start start 2024-09-11T01:24:07.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f7290462740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f409c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f7290462740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 0x559d9f415c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f7290462740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f416160 0x559d9f418550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f7290462740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559d9f377e50 con 0x559d9f416160 2024-09-11T01:24:07.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f7290462740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559d9f367280 con 0x559d9f4098b0 2024-09-11T01:24:07.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f7290462740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559d9f36b090 con 0x559d9f40a1d0 2024-09-11T01:24:07.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f7286ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f409c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f7286ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f409c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:34202/0 (socket says 172.21.15.78:34202) 2024-09-11T01:24:07.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f7286ffd640 1 -- 172.21.15.78:0/3986948882 learned_addr learned my addr 172.21.15.78:0/3986948882 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:07.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f7286ffd640 1 --2- 172.21.15.78:0/3986948882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f416160 0x559d9f418550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f72877fe640 1 --2- 172.21.15.78:0/3986948882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 0x559d9f415c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f72877fe640 1 -- 172.21.15.78:0/3986948882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 msgr2=0x559d9f409c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f72877fe640 1 --2- 172.21.15.78:0/3986948882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f409c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f72877fe640 1 -- 172.21.15.78:0/3986948882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f416160 msgr2=0x559d9f418550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f72877fe640 1 --2- 172.21.15.78:0/3986948882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f416160 0x559d9f418550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f72877fe640 1 -- 172.21.15.78:0/3986948882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d9f0c5650 con 0x559d9f40a1d0 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.889+0000 7f7286ffd640 1 --2- 172.21.15.78:0/3986948882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f416160 0x559d9f418550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.890+0000 7f7284ff9640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f72700b5450 0x7f72700b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.890+0000 7f72877fe640 1 --2- 172.21.15.78:0/3986948882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 0x559d9f415c20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7278009c30 tx=0x7f72780117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e37a0068af7d532d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.890+0000 7f726dffb640 1 -- 172.21.15.78:0/3986948882 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278012450 con 0x559d9f40a1d0 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.890+0000 7f7284ff9640 1 -- 172.21.15.78:0/660226447 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8493+0+0 (secure 0 0 0) 0x7f72800e5f50 con 0x7f7288108020 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.890+0000 7f726dffb640 1 -- 172.21.15.78:0/3986948882 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7278012aa0 con 0x559d9f40a1d0 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.890+0000 7f726dffb640 1 -- 172.21.15.78:0/3986948882 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727800faf0 con 0x559d9f40a1d0 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.890+0000 7f7286ffd640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f72700b5450 0x7f72700b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.891+0000 7f7290462740 1 -- 172.21.15.78:0/3986948882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 msgr2=0x559d9f415c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.891+0000 7f7286ffd640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f72700b5450 0x7f72700b7910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f727c02cfd0 tx=0x7f727c02a340 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.891+0000 7f7290462740 1 --2- 172.21.15.78:0/3986948882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 0x559d9f415c20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7278009c30 tx=0x7f72780117b0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.891+0000 7f7290462740 1 -- 172.21.15.78:0/3986948882 shutdown_connections 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.891+0000 7f7290462740 1 --2- 172.21.15.78:0/3986948882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f416160 0x559d9f418550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.891+0000 7f7290462740 1 --2- 172.21.15.78:0/3986948882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 0x559d9f415c20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.891+0000 7f7290462740 1 --2- 172.21.15.78:0/3986948882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f409c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.891+0000 7f7290462740 1 -- 172.21.15.78:0/3986948882 >> 172.21.15.78:0/3986948882 conn(0x559d9f4076e0 msgr2=0x559d9f408880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.891+0000 7f7290462740 1 -- 172.21.15.78:0/3986948882 shutdown_connections 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.891+0000 7f7290462740 1 -- 172.21.15.78:0/3986948882 wait complete. 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f7290462740 1 -- start start 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f7290462740 1 Processor -- start 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f7290462740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f4098b0 0x559d9f415a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f7290462740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 0x559d9f412e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f7290462740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f416160 0x559d9f413450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f7290462740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559d9f1c8700 con 0x559d9f4098b0 2024-09-11T01:24:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f7290462740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559d9f302110 con 0x559d9f416160 2024-09-11T01:24:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f7290462740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559d9f418de0 con 0x559d9f40a1d0 2024-09-11T01:24:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f7286ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 0x559d9f412e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f72877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f4098b0 0x559d9f415a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f7286ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 0x559d9f412e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:60600/0 (socket says 172.21.15.78:60600) 2024-09-11T01:24:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f7286ffd640 1 -- 172.21.15.78:0/1007282278 learned_addr learned my addr 172.21.15.78:0/1007282278 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.892+0000 7f7290462740 10 client.0 _ll_register_callbacks cb 0x559d9f31b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:24:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.893+0000 7f7287fff640 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f416160 0x559d9f413450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.893+0000 7f72877fe640 1 -- 172.21.15.78:0/1007282278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 msgr2=0x559d9f412e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.893+0000 7f72877fe640 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 0x559d9f412e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.893+0000 7f72877fe640 1 -- 172.21.15.78:0/1007282278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f416160 msgr2=0x559d9f413450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.893+0000 7f72877fe640 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f416160 0x559d9f413450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.893+0000 7f72877fe640 1 -- 172.21.15.78:0/1007282278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d9f32fc40 con 0x559d9f4098b0 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.893+0000 7f7286ffd640 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 0x559d9f412e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.893+0000 7f72877fe640 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f4098b0 0x559d9f415a20 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7278000c00 tx=0x7f7278012e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1810e08f17ca70ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.893+0000 7f7266ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.893+0000 7f7266ffd640 1 -- 172.21.15.78:0/1007282278 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278029020 con 0x559d9f4098b0 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.893+0000 7f7290462740 1 -- 172.21.15.78:0/1007282278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7278012aa0 con 0x559d9f4098b0 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.893+0000 7f7266ffd640 1 -- 172.21.15.78:0/1007282278 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8493+0+0 (secure 0 0 0) 0x7f7278012660 con 0x559d9f4098b0 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.894+0000 7f7266ffd640 1 -- 172.21.15.78:0/1007282278 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f7278012860 con 0x559d9f4098b0 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.894+0000 7f7290462740 10 client.19602 fetch_fsmap learned FSMap version 80 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.894+0000 7f7290462740 1 -- 172.21.15.78:0/1007282278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x559d9f424720 con 0x559d9f4098b0 2024-09-11T01:24:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.894+0000 7f7266ffd640 1 -- 172.21.15.78:0/1007282278 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72780054f0 con 0x559d9f4098b0 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.894+0000 7f7266ffd640 1 -- 172.21.15.78:0/1007282278 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 80) ==== 8406+0+0 (secure 0 0 0) 0x7f72780046c0 con 0x559d9f4098b0 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.894+0000 7f7290462740 10 client.19602 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.894+0000 7f7290462740 10 client.19602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.894+0000 7f7290462740 20 client.19602 populate_metadata read hostname 'smithi078' 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.894+0000 7f7290462740 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559d9f425b30 0x559d9f427f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.894+0000 7f7290462740 4 client.19602 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.894+0000 7f7290462740 1 -- 172.21.15.78:0/1007282278 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7278012aa0 con 0x559d9f425b30 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.895+0000 7f7286ffd640 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559d9f425b30 0x559d9f427f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.895+0000 7f7286ffd640 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559d9f425b30 0x559d9f427f20 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a0eaa65496ff0b1 server_cookie=a7318db9afdb9ccf in_seq=0 out_seq=0 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.895+0000 7f7266ffd640 10 client.19602 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.896+0000 7f7266ffd640 1 -- 172.21.15.78:0/1007282278 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7278012aa0 con 0x559d9f425b30 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.896+0000 7f7266ffd640 10 client.19602 handle_command_reply: tid=0 2024-09-11T01:24:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.896+0000 7f7290462740 1 client.19602 shutdown 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.896+0000 7f7290462740 1 -- 172.21.15.78:0/1007282278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f4098b0 msgr2=0x559d9f415a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.896+0000 7f7290462740 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f4098b0 0x559d9f415a20 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7278000c00 tx=0x7f7278012e80 comp rx=0 tx=0).stop 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.897+0000 7f7290462740 1 -- 172.21.15.78:0/1007282278 shutdown_connections 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.897+0000 7f7290462740 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f416160 0x559d9f413450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.897+0000 7f7290462740 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f40a1d0 0x559d9f412e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.897+0000 7f7290462740 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559d9f425b30 0x559d9f427f20 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.897+0000 7f7290462740 1 --2- 172.21.15.78:0/1007282278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f4098b0 0x559d9f415a20 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.897+0000 7f7266ffd640 0 client.19602 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.897+0000 7f7290462740 1 -- 172.21.15.78:0/1007282278 >> 172.21.15.78:0/1007282278 conn(0x559d9f4076e0 msgr2=0x559d9f40d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.897+0000 7f7290462740 1 -- 172.21.15.78:0/1007282278 shutdown_connections 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.897+0000 7f7290462740 1 -- 172.21.15.78:0/1007282278 wait complete. 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.898+0000 7f7290462740 20 client.19602 trim_cache size 0 max 16384 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.919+0000 7f7290462740 1 Processor -- start 2024-09-11T01:24:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.919+0000 7f7290462740 1 -- start start 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f7290462740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f422070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f7290462740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f40a1d0 0x559d9f414bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f7290462740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f416160 0x559d9f417550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f7290462740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72881ac400 con 0x559d9f40a1d0 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f7290462740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f72881ae090 con 0x559d9f4098b0 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f7290462740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72881acad0 con 0x559d9f416160 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f72877fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f422070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f72877fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f422070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:34208/0 (socket says 172.21.15.78:34208) 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f72877fe640 1 -- 172.21.15.78:0/3502680093 learned_addr learned my addr 172.21.15.78:0/3502680093 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f7286ffd640 1 --2- 172.21.15.78:0/3502680093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f40a1d0 0x559d9f414bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f7287fff640 1 --2- 172.21.15.78:0/3502680093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f416160 0x559d9f417550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f72877fe640 1 -- 172.21.15.78:0/3502680093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f416160 msgr2=0x559d9f417550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f72877fe640 1 --2- 172.21.15.78:0/3502680093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f416160 0x559d9f417550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f72877fe640 1 -- 172.21.15.78:0/3502680093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f40a1d0 msgr2=0x559d9f414bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f72877fe640 1 --2- 172.21.15.78:0/3502680093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f40a1d0 0x559d9f414bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.920+0000 7f72877fe640 1 -- 172.21.15.78:0/3502680093 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d9f408160 con 0x559d9f4098b0 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.921+0000 7f72877fe640 1 --2- 172.21.15.78:0/3502680093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f422070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7278004a90 tx=0x7f7278011de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70b9e213b4a57ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.921+0000 7f726d7fa640 1 -- 172.21.15.78:0/3502680093 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278029040 con 0x559d9f4098b0 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.921+0000 7f726d7fa640 1 -- 172.21.15.78:0/3502680093 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7278012a00 con 0x559d9f4098b0 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.921+0000 7f726d7fa640 1 -- 172.21.15.78:0/3502680093 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278004ee0 con 0x559d9f4098b0 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 -- 172.21.15.78:0/3502680093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 msgr2=0x559d9f422070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 --2- 172.21.15.78:0/3502680093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f422070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7278004a90 tx=0x7f7278011de0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 -- 172.21.15.78:0/3502680093 shutdown_connections 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 --2- 172.21.15.78:0/3502680093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f416160 0x559d9f417550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 --2- 172.21.15.78:0/3502680093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f40a1d0 0x559d9f414bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 --2- 172.21.15.78:0/3502680093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f422070 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 -- 172.21.15.78:0/3502680093 >> 172.21.15.78:0/3502680093 conn(0x559d9f4076e0 msgr2=0x559d9f4144a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 -- 172.21.15.78:0/3502680093 shutdown_connections 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 -- 172.21.15.78:0/3502680093 wait complete. 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 -- start start 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 Processor -- start 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f41e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f40a1d0 0x559d9f41e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f416160 0x559d9f40f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72881ad930 con 0x559d9f40a1d0 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f72881ad230 con 0x559d9f4098b0 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.922+0000 7f7290462740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72881aa480 con 0x559d9f416160 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7290462740 10 client.0 _ll_register_callbacks cb 0x7f7278005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7286ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f40a1d0 0x559d9f41e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7286ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f40a1d0 0x559d9f41e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43416/0 (socket says 172.21.15.78:43416) 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7286ffd640 1 -- 172.21.15.78:0/4202280419 learned_addr learned my addr 172.21.15.78:0/4202280419 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f72877fe640 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f41e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7287fff640 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f416160 0x559d9f40f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7286ffd640 1 -- 172.21.15.78:0/4202280419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f416160 msgr2=0x559d9f40f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7286ffd640 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f416160 0x559d9f40f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7286ffd640 1 -- 172.21.15.78:0/4202280419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 msgr2=0x559d9f41e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7286ffd640 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f41e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7286ffd640 1 -- 172.21.15.78:0/4202280419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d9f40f700 con 0x559d9f40a1d0 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f72877fe640 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f41e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7286ffd640 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f40a1d0 0x559d9f41e7f0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f727c02a860 tx=0x7f727c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8db121289957ba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7266ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7266ffd640 1 -- 172.21.15.78:0/4202280419 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c037040 con 0x559d9f40a1d0 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7266ffd640 1 -- 172.21.15.78:0/4202280419 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8493+0+0 (secure 0 0 0) 0x7f727c0307d0 con 0x559d9f40a1d0 2024-09-11T01:24:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7290462740 1 -- 172.21.15.78:0/4202280419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7278012a00 con 0x559d9f40a1d0 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7266ffd640 1 -- 172.21.15.78:0/4202280419 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f727c0035f0 con 0x559d9f40a1d0 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7290462740 10 client.19608 fetch_fsmap learned FSMap version 80 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.923+0000 7f7290462740 1 -- 172.21.15.78:0/4202280419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x559d9f410a80 con 0x559d9f40a1d0 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.924+0000 7f7266ffd640 1 -- 172.21.15.78:0/4202280419 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7278012a00 con 0x559d9f40a1d0 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.924+0000 7f7266ffd640 1 -- 172.21.15.78:0/4202280419 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 80) ==== 8406+0+0 (secure 0 0 0) 0x7f727c062020 con 0x559d9f40a1d0 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.924+0000 7f7290462740 10 client.19608 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.924+0000 7f7290462740 10 client.19608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.924+0000 7f7290462740 20 client.19608 populate_metadata read hostname 'smithi078' 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.924+0000 7f7290462740 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559d9f425b30 0x559d9f495a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.924+0000 7f7290462740 4 client.19608 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.924+0000 7f7290462740 1 -- 172.21.15.78:0/4202280419 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d9f379da0 con 0x559d9f425b30 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.924+0000 7f72877fe640 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559d9f425b30 0x559d9f495a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:07.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.924+0000 7f7266ffd640 10 client.19608 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.924+0000 7f72877fe640 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559d9f425b30 0x559d9f495a30 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=833e411ccb52b9e7 server_cookie=25080c8e52c0f454 in_seq=0 out_seq=0 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7266ffd640 1 -- 172.21.15.78:0/4202280419 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8326 (crc 0 0 0) 0x559d9f379da0 con 0x559d9f425b30 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7266ffd640 10 client.19608 handle_command_reply: tid=0 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7290462740 1 client.19608 shutdown 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7290462740 1 -- 172.21.15.78:0/4202280419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f40a1d0 msgr2=0x559d9f41e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7290462740 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f40a1d0 0x559d9f41e7f0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f727c02a860 tx=0x7f727c0058d0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7290462740 1 -- 172.21.15.78:0/4202280419 shutdown_connections 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7290462740 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d9f416160 0x559d9f40f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7290462740 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x559d9f425b30 0x559d9f495a30 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7290462740 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d9f40a1d0 0x559d9f41e7f0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7290462740 1 --2- 172.21.15.78:0/4202280419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559d9f4098b0 0x559d9f41e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7266ffd640 0 client.19608 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7290462740 1 -- 172.21.15.78:0/4202280419 >> 172.21.15.78:0/4202280419 conn(0x559d9f4076e0 msgr2=0x559d9f414cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:07.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7290462740 1 -- 172.21.15.78:0/4202280419 shutdown_connections 2024-09-11T01:24:07.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7290462740 1 -- 172.21.15.78:0/4202280419 wait complete. 2024-09-11T01:24:07.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.925+0000 7f7290462740 20 client.19608 trim_cache size 0 max 16384 2024-09-11T01:24:07.949 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:24:07.949 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-11T01:24:07.949 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {}, 2024-09-11T01:24:07.949 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_stats": { 2024-09-11T01:24:07.949 INFO:teuthology.orchestra.run.smithi078.stdout: "6eb6da3d-724b-47e6-a25c-a6dcc447ffc7": { 2024-09-11T01:24:07.949 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:23:31.655194+0000", 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: "82736ae4-b55a-4ba3-911c-0aa1c18acfde": { 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:22:21.932379+0000", 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.950 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "746add9a-5be3-4745-8000-04e5471bc813": { 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:20:12.369593+0000", 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "b41a2b77-b2b1-4f1b-8588-4f32a78b721f": { 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:19:13.257610+0000", 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: "e66fef11-baf9-43ef-9bea-6ece866f92fa": { 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:18:17.654570+0000", 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: "21c0a2c6-e49f-4cac-b354-bcd7d75438b4": { 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:17:17.145289+0000", 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.952 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: "14920c2e-6e42-4652-a1cc-ea4a4144f749": { 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:13:06.354656+0000", 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.953 INFO:teuthology.orchestra.run.smithi078.stdout: "2f8adbde-7a68-4c03-95dd-9f3a43878c82": { 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:10:37.776291+0000", 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "4de34141-1c94-4d41-a1fc-9ca074174c31": { 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:50.197139+0000", 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.954 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: "114b76b0-8477-4d97-b164-754a49f52148": { 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:09:02.964979+0000", 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: "89389c97-2d94-44a5-9660-dc74c67b6658": { 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:58:58.377985+0000", 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: "cca472d2-e33e-45fa-840a-6bc7ba469dca": { 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:01:17.068439+0000", 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: "33c6d527-1d18-4220-8f09-6a413e212ad7": { 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:16:01.865570+0000", 2024-09-11T01:24:07.956 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.957 INFO:teuthology.orchestra.run.smithi078.stdout: "4502de91-9ca6-47fc-b564-9b5f99d24eb1": { 2024-09-11T01:24:07.957 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:14:08.544559+0000", 2024-09-11T01:24:07.957 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: "c382b80f-4a8b-44bf-875d-48c566a029c2": { 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:07:31.005618+0000", 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: "85264ca2-fdd8-466e-9591-4583174b621c": { 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:15:05.888548+0000", 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.958 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: "c3c800f7-782d-4ca3-b38b-b456abe7aba2": { 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:59.107271+0000", 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.959 INFO:teuthology.orchestra.run.smithi078.stdout: "6ba15803-6da9-4d97-adb8-4d6230a3c7a3": { 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:00:31.029284+0000", 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: "291fcaa4-1805-423d-a0bc-814f1bc2759f": { 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:21:17.256483+0000", 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.960 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.961 INFO:teuthology.orchestra.run.smithi078.stdout: "e5791ad9-cf54-4303-938d-6fc349b35341": { 2024-09-11T01:24:07.961 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:12:11.637624+0000", 2024-09-11T01:24:07.961 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.961 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: "75a1f644-bccb-4806-b819-99e9253c572d": { 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:11:25.225446+0000", 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: "e30a4af9-93f2-424c-b1fd-c56f7384a2df": { 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T00:59:45.320123+0000", 2024-09-11T01:24:07.962 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.963 INFO:teuthology.orchestra.run.smithi078.stdout: "092ff724-3d6b-4fec-a529-7ae22738a268": { 2024-09-11T01:24:07.963 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:05.276552+0000", 2024-09-11T01:24:07.963 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.963 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.964 INFO:teuthology.orchestra.run.smithi078.stdout: "371f8253-b1f4-400e-94c8-988b8dc35dba": { 2024-09-11T01:24:07.964 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:06:45.426436+0000", 2024-09-11T01:24:07.964 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.964 INFO:teuthology.orchestra.run.smithi078.stdout: "ce2fc4bd-f2bb-41b9-8ae0-a0cf2688ca2e": { 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:04:27.140941+0000", 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "3aff237f-73e0-453f-ae2b-f1146929cd2f": { 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:02:51.936689+0000", 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: "8eb1d395-7f13-4fb9-9ea8-5a80169444b5": { 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:03:39.796808+0000", 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: "6b1f53cf-607b-417b-b81b-ce9af7347e46": { 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:05:13.128104+0000", 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.966 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: "dc23cd05-9f92-4cee-b281-6ddbecb84d9f": { 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: "start_time": "2024-09-11T01:08:16.556804+0000", 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_passed": 0, 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_failed": 0, 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_skipped": 0 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:07.967 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:24:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.926+0000 7f728e08d640 1 -- 172.21.15.78:0/660226447 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f72700b5450 msgr2=0x7f72700b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.926+0000 7f728e08d640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f72700b5450 0x7f72700b7910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f727c02cfd0 tx=0x7f727c02a340 comp rx=0 tx=0).stop 2024-09-11T01:24:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.926+0000 7f728e08d640 1 -- 172.21.15.78:0/660226447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7288108020 msgr2=0x7f72881a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.926+0000 7f728e08d640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7288108020 0x7f72881a0d70 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f728000efc0 tx=0x7f728000c800 comp rx=0 tx=0).stop 2024-09-11T01:24:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.926+0000 7f728e08d640 1 -- 172.21.15.78:0/660226447 shutdown_connections 2024-09-11T01:24:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.926+0000 7f728e08d640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f72700b5450 0x7f72700b7910 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.926+0000 7f728e08d640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7288108020 0x7f72881a0d70 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.926+0000 7f728e08d640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72881076a0 0x7f728819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.926+0000 7f728e08d640 1 --2- 172.21.15.78:0/660226447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7288101c90 0x7f728819b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.926+0000 7f728e08d640 1 -- 172.21.15.78:0/660226447 >> 172.21.15.78:0/660226447 conn(0x7f72880fd700 msgr2=0x7f728806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.926+0000 7f728e08d640 1 -- 172.21.15.78:0/660226447 shutdown_connections 2024-09-11T01:24:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:07.926+0000 7f728e08d640 1 -- 172.21.15.78:0/660226447 wait complete. 2024-09-11T01:24:07.969 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:07 smithi110 ceph-mon[34712]: pgmap v1116: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 499 op/s 2024-09-11T01:24:07.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:07 smithi112 ceph-mon[35384]: pgmap v1116: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 499 op/s 2024-09-11T01:24:08.691 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:08 smithi078 ceph-mon[31315]: osdmap e114: 12 total, 12 up, 12 in 2024-09-11T01:24:08.691 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:08 smithi078 ceph-mon[31315]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:24:08.930 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T01:24:08.931 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:24:08.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:08 smithi110 ceph-mon[34712]: osdmap e114: 12 total, 12 up, 12 in 2024-09-11T01:24:08.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:08 smithi110 ceph-mon[34712]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:24:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:08 smithi112 ceph-mon[35384]: osdmap e114: 12 total, 12 up, 12 in 2024-09-11T01:24:08.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:08 smithi112 ceph-mon[35384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T01:24:09.197 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:09.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:09 smithi078 ceph-mon[31315]: pgmap v1118: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 458 op/s 2024-09-11T01:24:09.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.713+0000 7faf6a099640 1 -- 172.21.15.78:0/363691532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64104530 msgr2=0x7faf6410cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:09.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.713+0000 7faf6a099640 1 --2- 172.21.15.78:0/363691532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64104530 0x7faf6410cb10 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009900 tx=0x7faf4c02a410 comp rx=0 tx=0).stop 2024-09-11T01:24:09.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.713+0000 7faf6a099640 1 -- 172.21.15.78:0/363691532 shutdown_connections 2024-09-11T01:24:09.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.713+0000 7faf6a099640 1 --2- 172.21.15.78:0/363691532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf6410d0e0 0x7faf6410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:09.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.713+0000 7faf6a099640 1 --2- 172.21.15.78:0/363691532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64104530 0x7faf6410cb10 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.713+0000 7faf6a099640 1 --2- 172.21.15.78:0/363691532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf64103b30 0x7faf64103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.713+0000 7faf6a099640 1 -- 172.21.15.78:0/363691532 >> 172.21.15.78:0/363691532 conn(0x7faf640fd700 msgr2=0x7faf640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.713+0000 7faf6a099640 1 -- 172.21.15.78:0/363691532 shutdown_connections 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.713+0000 7faf6a099640 1 -- 172.21.15.78:0/363691532 wait complete. 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.713+0000 7faf6a099640 1 Processor -- start 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.713+0000 7faf6a099640 1 -- start start 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf6a099640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64103b30 0x7faf641992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf6a099640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf64104530 0x7faf64199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf6a099640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf6410d0e0 0x7faf6419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf6a099640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf64112d60 con 0x7faf64103b30 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf6a099640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faf64112be0 con 0x7faf64104530 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf6a099640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf64112ee0 con 0x7faf6410d0e0 2024-09-11T01:24:09.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64103b30 0x7faf641992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64103b30 0x7faf641992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43442/0 (socket says 172.21.15.78:43442) 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf637fe640 1 -- 172.21.15.78:0/3040874079 learned_addr learned my addr 172.21.15.78:0/3040874079 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf63fff640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf6410d0e0 0x7faf6419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf5bfff640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf64104530 0x7faf64199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf637fe640 1 -- 172.21.15.78:0/3040874079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf6410d0e0 msgr2=0x7faf6419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf637fe640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf6410d0e0 0x7faf6419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf637fe640 1 -- 172.21.15.78:0/3040874079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf64104530 msgr2=0x7faf64199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf637fe640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf64104530 0x7faf64199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf637fe640 1 -- 172.21.15.78:0/3040874079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6419f3a0 con 0x7faf64103b30 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf63fff640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf6410d0e0 0x7faf6419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf5bfff640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf64104530 0x7faf64199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf637fe640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64103b30 0x7faf641992f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7faf5400d950 tx=0x7faf5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf617fa640 1 -- 172.21.15.78:0/3040874079 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54017070 con 0x7faf64103b30 2024-09-11T01:24:09.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf6a099640 1 -- 172.21.15.78:0/3040874079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6419f690 con 0x7faf64103b30 2024-09-11T01:24:09.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf617fa640 1 -- 172.21.15.78:0/3040874079 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf54009c30 con 0x7faf64103b30 2024-09-11T01:24:09.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.714+0000 7faf617fa640 1 -- 172.21.15.78:0/3040874079 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5400f620 con 0x7faf64103b30 2024-09-11T01:24:09.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.715+0000 7faf6a099640 1 -- 172.21.15.78:0/3040874079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf641a5f20 con 0x7faf64103b30 2024-09-11T01:24:09.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.715+0000 7faf6a099640 1 -- 172.21.15.78:0/3040874079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6406b150 con 0x7faf64103b30 2024-09-11T01:24:09.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.717+0000 7faf617fa640 1 -- 172.21.15.78:0/3040874079 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7faf54005030 con 0x7faf64103b30 2024-09-11T01:24:09.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.718+0000 7faf617fa640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faf500b5450 0x7faf500b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:09.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.718+0000 7faf5bfff640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faf500b5450 0x7faf500b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:09.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.718+0000 7faf617fa640 1 -- 172.21.15.78:0/3040874079 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8493+0+0 (secure 0 0 0) 0x7faf540e5c90 con 0x7faf64103b30 2024-09-11T01:24:09.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.718+0000 7faf5bfff640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faf500b5450 0x7faf500b7910 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faf4c031440 tx=0x7faf4c005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:09.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.724+0000 7faf617fa640 1 -- 172.21.15.78:0/3040874079 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7faf540ac830 con 0x7faf64103b30 2024-09-11T01:24:09.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:09 smithi110 ceph-mon[34712]: pgmap v1118: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 458 op/s 2024-09-11T01:24:09.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.940+0000 7faf6a099640 1 -- 172.21.15.78:0/3040874079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf64103f10 con 0x7faf64103b30 2024-09-11T01:24:09.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.940+0000 7faf617fa640 1 -- 172.21.15.78:0/3040874079 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+11996 (secure 0 0 0) 0x7faf64103f10 con 0x7faf64103b30 2024-09-11T01:24:09.942 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:24:09.942 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":80,"btime":"2024-09-11T01:24:07:496110+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":80,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:24:07.017719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:24:09.943 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 80 2024-09-11T01:24:09.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.943+0000 7faf6a099640 1 -- 172.21.15.78:0/3040874079 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faf500b5450 msgr2=0x7faf500b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:09.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.943+0000 7faf6a099640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faf500b5450 0x7faf500b7910 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faf4c031440 tx=0x7faf4c005a20 comp rx=0 tx=0).stop 2024-09-11T01:24:09.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.943+0000 7faf6a099640 1 -- 172.21.15.78:0/3040874079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64103b30 msgr2=0x7faf641992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:09.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.943+0000 7faf6a099640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64103b30 0x7faf641992f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7faf5400d950 tx=0x7faf5400de10 comp rx=0 tx=0).stop 2024-09-11T01:24:09.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.943+0000 7faf6a099640 1 -- 172.21.15.78:0/3040874079 shutdown_connections 2024-09-11T01:24:09.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.943+0000 7faf6a099640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7faf500b5450 0x7faf500b7910 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:09.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.943+0000 7faf6a099640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf6410d0e0 0x7faf6419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:09.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.943+0000 7faf6a099640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf64104530 0x7faf64199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:09.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.943+0000 7faf6a099640 1 --2- 172.21.15.78:0/3040874079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64103b30 0x7faf641992f0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:09.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.943+0000 7faf6a099640 1 -- 172.21.15.78:0/3040874079 >> 172.21.15.78:0/3040874079 conn(0x7faf640fd700 msgr2=0x7faf640fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:09.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.943+0000 7faf6a099640 1 -- 172.21.15.78:0/3040874079 shutdown_connections 2024-09-11T01:24:09.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:09.943+0000 7faf6a099640 1 -- 172.21.15.78:0/3040874079 wait complete. 2024-09-11T01:24:09.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:09 smithi112 ceph-mon[35384]: pgmap v1118: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 458 op/s 2024-09-11T01:24:10.779 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:10 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3040874079' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:24:10.816 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T01:24:10.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:10 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3040874079' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:24:10.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:10 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3040874079' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:24:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:11.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.556+0000 7f5375613640 1 -- 172.21.15.78:0/1580270976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5370102920 msgr2=0x7f5370102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:11.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.556+0000 7f5375613640 1 --2- 172.21.15.78:0/1580270976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5370102920 0x7f5370102d00 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f5360009900 tx=0x7f536002a410 comp rx=0 tx=0).stop 2024-09-11T01:24:11.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.556+0000 7f5375613640 1 -- 172.21.15.78:0/1580270976 shutdown_connections 2024-09-11T01:24:11.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.556+0000 7f5375613640 1 --2- 172.21.15.78:0/1580270976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5370108120 0x7f537010e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.556+0000 7f5375613640 1 --2- 172.21.15.78:0/1580270976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5370107780 0x7f5370107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.556+0000 7f5375613640 1 --2- 172.21.15.78:0/1580270976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5370102920 0x7f5370102d00 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.556+0000 7f5375613640 1 -- 172.21.15.78:0/1580270976 >> 172.21.15.78:0/1580270976 conn(0x7f5370076f50 msgr2=0x7f5370077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:11.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.556+0000 7f5375613640 1 -- 172.21.15.78:0/1580270976 shutdown_connections 2024-09-11T01:24:11.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.556+0000 7f5375613640 1 -- 172.21.15.78:0/1580270976 wait complete. 2024-09-11T01:24:11.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.557+0000 7f5375613640 1 Processor -- start 2024-09-11T01:24:11.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.557+0000 7f5375613640 1 -- start start 2024-09-11T01:24:11.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.557+0000 7f5375613640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5370102920 0x7f537010f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.557+0000 7f5375613640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5370107780 0x7f537010f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f5375613640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5370108120 0x7f537010fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f5375613640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5370112d90 con 0x7f5370102920 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f5375613640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5370112c10 con 0x7f5370108120 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f5375613640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5370112f10 con 0x7f5370107780 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f536effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5370102920 0x7f537010f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f536effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5370102920 0x7f537010f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43452/0 (socket says 172.21.15.78:43452) 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f536effd640 1 -- 172.21.15.78:0/3097332180 learned_addr learned my addr 172.21.15.78:0/3097332180 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f536e7fc640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5370107780 0x7f537010f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f536f7fe640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5370108120 0x7f537010fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f536effd640 1 -- 172.21.15.78:0/3097332180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5370107780 msgr2=0x7f537010f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f536effd640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5370107780 0x7f537010f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f536effd640 1 -- 172.21.15.78:0/3097332180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5370108120 msgr2=0x7f537010fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f536effd640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5370108120 0x7f537010fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f536effd640 1 -- 172.21.15.78:0/3097332180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53701a7640 con 0x7f5370102920 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f536effd640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5370102920 0x7f537010f350 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f5360006010 tx=0x7f53600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:11.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f5357fff640 1 -- 172.21.15.78:0/3097332180 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536003f030 con 0x7f5370102920 2024-09-11T01:24:11.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f5375613640 1 -- 172.21.15.78:0/3097332180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53701a78d0 con 0x7f5370102920 2024-09-11T01:24:11.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f5357fff640 1 -- 172.21.15.78:0/3097332180 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5360003800 con 0x7f5370102920 2024-09-11T01:24:11.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.558+0000 7f5357fff640 1 -- 172.21.15.78:0/3097332180 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360035070 con 0x7f5370102920 2024-09-11T01:24:11.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.559+0000 7f5375613640 1 -- 172.21.15.78:0/3097332180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53701a7e00 con 0x7f5370102920 2024-09-11T01:24:11.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.561+0000 7f5357fff640 1 -- 172.21.15.78:0/3097332180 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5360031b10 con 0x7f5370102920 2024-09-11T01:24:11.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.561+0000 7f53779e8740 1 Processor -- start 2024-09-11T01:24:11.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.561+0000 7f53779e8740 1 -- start start 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.561+0000 7f53779e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840b4d0 0x556c6840b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.561+0000 7f53779e8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840bdf0 0x556c68417840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.561+0000 7f53779e8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 0x556c6841a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.561+0000 7f53779e8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556c68381ea0 con 0x556c68417d80 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.561+0000 7f53779e8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556c683712d0 con 0x556c6840bdf0 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.561+0000 7f53779e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556c683750e0 con 0x556c6840b4d0 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.561+0000 7f536e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 0x556c6841a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 0x556c6841a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43460/0 (socket says 172.21.15.78:43460) 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536e7fc640 1 -- 172.21.15.78:0/1610538484 learned_addr learned my addr 172.21.15.78:0/1610538484 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f5357fff640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f53580b5450 0x7f53580b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536effd640 1 --2- 172.21.15.78:0/1610538484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840bdf0 0x556c68417840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536f7fe640 1 --2- 172.21.15.78:0/1610538484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840b4d0 0x556c6840b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536e7fc640 1 -- 172.21.15.78:0/1610538484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840b4d0 msgr2=0x556c6840b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:11.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536e7fc640 1 --2- 172.21.15.78:0/1610538484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840b4d0 0x556c6840b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536e7fc640 1 -- 172.21.15.78:0/1610538484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840bdf0 msgr2=0x556c68417840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:11.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536e7fc640 1 --2- 172.21.15.78:0/1610538484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840bdf0 0x556c68417840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536e7fc640 1 -- 172.21.15.78:0/1610538484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c680cf650 con 0x556c68417d80 2024-09-11T01:24:11.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536f7fe640 1 --2- 172.21.15.78:0/1610538484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840b4d0 0x556c6840b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:11.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536effd640 1 --2- 172.21.15.78:0/1610538484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840bdf0 0x556c68417840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:11.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536f7fe640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f53580b5450 0x7f53580b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f5357fff640 1 -- 172.21.15.78:0/3097332180 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8493+0+0 (secure 0 0 0) 0x7f5360106930 con 0x7f5370102920 2024-09-11T01:24:11.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f536e7fc640 1 --2- 172.21.15.78:0/1610538484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 0x556c6841a170 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f536400aca0 tx=0x7f536400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df0fa90f7e30e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:11.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f53557fa640 1 -- 172.21.15.78:0/1610538484 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536401d070 con 0x556c68417d80 2024-09-11T01:24:11.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f53557fa640 1 -- 172.21.15.78:0/1610538484 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5364018af0 con 0x556c68417d80 2024-09-11T01:24:11.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.562+0000 7f53557fa640 1 -- 172.21.15.78:0/1610538484 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364010b90 con 0x556c68417d80 2024-09-11T01:24:11.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.563+0000 7f53779e8740 1 -- 172.21.15.78:0/1610538484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 msgr2=0x556c6841a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:11.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.563+0000 7f536f7fe640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f53580b5450 0x7f53580b7910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5368005c40 tx=0x7f5368004a40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:11.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.563+0000 7f53779e8740 1 --2- 172.21.15.78:0/1610538484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 0x556c6841a170 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f536400aca0 tx=0x7f536400f790 comp rx=0 tx=0).stop 2024-09-11T01:24:11.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.563+0000 7f53779e8740 1 -- 172.21.15.78:0/1610538484 shutdown_connections 2024-09-11T01:24:11.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.563+0000 7f53779e8740 1 --2- 172.21.15.78:0/1610538484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 0x556c6841a170 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.563+0000 7f53779e8740 1 --2- 172.21.15.78:0/1610538484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840bdf0 0x556c68417840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.563+0000 7f53779e8740 1 --2- 172.21.15.78:0/1610538484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840b4d0 0x556c6840b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.563+0000 7f53779e8740 1 -- 172.21.15.78:0/1610538484 >> 172.21.15.78:0/1610538484 conn(0x556c68409300 msgr2=0x556c6840a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:11.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.563+0000 7f53779e8740 1 -- 172.21.15.78:0/1610538484 shutdown_connections 2024-09-11T01:24:11.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.563+0000 7f53779e8740 1 -- 172.21.15.78:0/1610538484 wait complete. 2024-09-11T01:24:11.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.563+0000 7f53779e8740 1 -- start start 2024-09-11T01:24:11.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f53779e8740 1 Processor -- start 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f53779e8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840b4d0 0x556c684175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f53779e8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 0x556c68414a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f53779e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c68417d80 0x556c68414f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f53779e8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556c681d2700 con 0x556c6840bdf0 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f53779e8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556c6830c160 con 0x556c6840b4d0 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f53779e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556c6841aa00 con 0x556c68417d80 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f53779e8740 10 client.0 _ll_register_callbacks cb 0x556c683259e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f536e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 0x556c68414a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f536e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 0x556c68414a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43462/0 (socket says 172.21.15.78:43462) 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f536e7fc640 1 -- 172.21.15.78:0/3289613939 learned_addr learned my addr 172.21.15.78:0/3289613939 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f536effd640 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840b4d0 0x556c684175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f536f7fe640 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c68417d80 0x556c68414f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f536e7fc640 1 -- 172.21.15.78:0/3289613939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c68417d80 msgr2=0x556c68414f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f536e7fc640 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c68417d80 0x556c68414f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f536e7fc640 1 -- 172.21.15.78:0/3289613939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840b4d0 msgr2=0x556c684175d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f536e7fc640 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840b4d0 0x556c684175d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f536e7fc640 1 -- 172.21.15.78:0/3289613939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c68339c90 con 0x556c6840bdf0 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f536e7fc640 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 0x556c68414a20 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f536400a8a0 tx=0x7f5364019f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f6dda56ec92e73 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.564+0000 7f5352ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f5352ff5640 1 -- 172.21.15.78:0/3289613939 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536401d050 con 0x556c6840bdf0 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f5352ff5640 1 -- 172.21.15.78:0/3289613939 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8493+0+0 (secure 0 0 0) 0x7f5364010930 con 0x556c6840bdf0 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f53779e8740 1 -- 172.21.15.78:0/3289613939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5364018af0 con 0x556c6840bdf0 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f5352ff5640 1 -- 172.21.15.78:0/3289613939 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f5364032050 con 0x556c6840bdf0 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f53779e8740 10 client.19626 fetch_fsmap learned FSMap version 80 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f53779e8740 1 -- 172.21.15.78:0/3289613939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x556c68426230 con 0x556c6840bdf0 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f5352ff5640 1 -- 172.21.15.78:0/3289613939 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5364018af0 con 0x556c6840bdf0 2024-09-11T01:24:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f5352ff5640 1 -- 172.21.15.78:0/3289613939 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 80) ==== 8406+0+0 (secure 0 0 0) 0x7f53640046c0 con 0x556c6840bdf0 2024-09-11T01:24:11.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f53779e8740 10 client.19626 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T01:24:11.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f53779e8740 10 client.19626 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:24:11.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f53779e8740 20 client.19626 populate_metadata read hostname 'smithi078' 2024-09-11T01:24:11.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f53779e8740 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556c68427630 0x556c68429a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f53779e8740 4 client.19626 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:24:11.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.565+0000 7f53779e8740 1 -- 172.21.15.78:0/3289613939 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c68383df0 con 0x556c68427630 2024-09-11T01:24:11.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.566+0000 7f536effd640 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556c68427630 0x556c68429a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.566+0000 7f5352ff5640 10 client.19626 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:24:11.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.566+0000 7f536effd640 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556c68427630 0x556c68429a20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=19e06c9a5272d05b server_cookie=e14068db924a5b23 in_seq=0 out_seq=0 2024-09-11T01:24:11.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.566+0000 7f5352ff5640 1 -- 172.21.15.78:0/3289613939 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c68383df0 con 0x556c68427630 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.566+0000 7f5352ff5640 10 client.19626 handle_command_reply: tid=0 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.566+0000 7f53779e8740 1 client.19626 shutdown 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.566+0000 7f53779e8740 1 -- 172.21.15.78:0/3289613939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 msgr2=0x556c68414a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.566+0000 7f53779e8740 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 0x556c68414a20 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f536400a8a0 tx=0x7f5364019f60 comp rx=0 tx=0).stop 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.567+0000 7f53779e8740 1 -- 172.21.15.78:0/3289613939 shutdown_connections 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.567+0000 7f53779e8740 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556c68427630 0x556c68429a20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.567+0000 7f53779e8740 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c68417d80 0x556c68414f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.567+0000 7f53779e8740 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 0x556c68414a20 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.567+0000 7f53779e8740 1 --2- 172.21.15.78:0/3289613939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840b4d0 0x556c684175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.567+0000 7f53779e8740 1 -- 172.21.15.78:0/3289613939 >> 172.21.15.78:0/3289613939 conn(0x556c68409300 msgr2=0x556c6840ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.567+0000 7f5352ff5640 0 client.19626 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.567+0000 7f53779e8740 1 -- 172.21.15.78:0/3289613939 shutdown_connections 2024-09-11T01:24:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.567+0000 7f53779e8740 1 -- 172.21.15.78:0/3289613939 wait complete. 2024-09-11T01:24:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.567+0000 7f53779e8740 20 client.19626 trim_cache size 0 max 16384 2024-09-11T01:24:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f53779e8740 1 Processor -- start 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f53779e8740 1 -- start start 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f53779e8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840b4d0 0x556c68408c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f53779e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840bdf0 0x556c6840a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f53779e8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 0x556c6840a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f53779e8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53701b0800 con 0x556c68417d80 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f53779e8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53701b2490 con 0x556c6840b4d0 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f53779e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53701b0ed0 con 0x556c6840bdf0 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f536f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 0x556c6840a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f536f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 0x556c6840a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43476/0 (socket says 172.21.15.78:43476) 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f536f7fe640 1 -- 172.21.15.78:0/2021399619 learned_addr learned my addr 172.21.15.78:0/2021399619 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f536effd640 1 --2- 172.21.15.78:0/2021399619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840b4d0 0x556c68408c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f536e7fc640 1 --2- 172.21.15.78:0/2021399619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840bdf0 0x556c6840a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f536f7fe640 1 -- 172.21.15.78:0/2021399619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840bdf0 msgr2=0x556c6840a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f536f7fe640 1 --2- 172.21.15.78:0/2021399619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840bdf0 0x556c6840a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f536f7fe640 1 -- 172.21.15.78:0/2021399619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840b4d0 msgr2=0x556c68408c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f536f7fe640 1 --2- 172.21.15.78:0/2021399619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840b4d0 0x556c68408c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.580+0000 7f536f7fe640 1 -- 172.21.15.78:0/2021399619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c68404de0 con 0x556c68417d80 2024-09-11T01:24:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f536f7fe640 1 --2- 172.21.15.78:0/2021399619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 0x556c6840a890 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f5368010700 tx=0x7f5368012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228698145abad87e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f5354ff9640 1 -- 172.21.15.78:0/2021399619 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368004d20 con 0x556c68417d80 2024-09-11T01:24:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f5354ff9640 1 -- 172.21.15.78:0/2021399619 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5368010e30 con 0x556c68417d80 2024-09-11T01:24:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f5354ff9640 1 -- 172.21.15.78:0/2021399619 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536801c810 con 0x556c68417d80 2024-09-11T01:24:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f53779e8740 1 -- 172.21.15.78:0/2021399619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 msgr2=0x556c6840a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f53779e8740 1 --2- 172.21.15.78:0/2021399619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 0x556c6840a890 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f5368010700 tx=0x7f5368012000 comp rx=0 tx=0).stop 2024-09-11T01:24:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f53779e8740 1 -- 172.21.15.78:0/2021399619 shutdown_connections 2024-09-11T01:24:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f53779e8740 1 --2- 172.21.15.78:0/2021399619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c68417d80 0x556c6840a890 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f53779e8740 1 --2- 172.21.15.78:0/2021399619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840bdf0 0x556c6840a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f53779e8740 1 --2- 172.21.15.78:0/2021399619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c6840b4d0 0x556c68408c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f53779e8740 1 -- 172.21.15.78:0/2021399619 >> 172.21.15.78:0/2021399619 conn(0x556c68409300 msgr2=0x556c6840fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f53779e8740 1 -- 172.21.15.78:0/2021399619 shutdown_connections 2024-09-11T01:24:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.581+0000 7f53779e8740 1 -- 172.21.15.78:0/2021399619 wait complete. 2024-09-11T01:24:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f53779e8740 1 -- start start 2024-09-11T01:24:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f53779e8740 1 Processor -- start 2024-09-11T01:24:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f53779e8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840b4d0 0x556c6841bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f53779e8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 0x556c6841c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f53779e8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c68417d80 0x556c6841c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f53779e8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53701b1d30 con 0x556c6840bdf0 2024-09-11T01:24:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f53779e8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53701b1630 con 0x556c68417d80 2024-09-11T01:24:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f53779e8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53701ae800 con 0x556c6840b4d0 2024-09-11T01:24:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f53779e8740 10 client.0 _ll_register_callbacks cb 0x7f536401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:24:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f536e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 0x556c6841c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f536e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 0x556c6841c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43488/0 (socket says 172.21.15.78:43488) 2024-09-11T01:24:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f536e7fc640 1 -- 172.21.15.78:0/284748698 learned_addr learned my addr 172.21.15.78:0/284748698 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f536effd640 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840b4d0 0x556c6841bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f536f7fe640 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c68417d80 0x556c6841c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f536e7fc640 1 -- 172.21.15.78:0/284748698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840b4d0 msgr2=0x556c6841bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.582+0000 7f536e7fc640 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840b4d0 0x556c6841bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f536e7fc640 1 -- 172.21.15.78:0/284748698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c68417d80 msgr2=0x556c6841c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f536e7fc640 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c68417d80 0x556c6841c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f536e7fc640 1 -- 172.21.15.78:0/284748698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c684932a0 con 0x556c6840bdf0 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f536e7fc640 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 0x556c6841c1a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f536400f7d0 tx=0x7f5364004910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63361b0f5d7e2d58 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f5352ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f5352ff5640 1 -- 172.21.15.78:0/284748698 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536401d040 con 0x556c6840bdf0 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f5352ff5640 1 -- 172.21.15.78:0/284748698 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8493+0+0 (secure 0 0 0) 0x7f5364018c20 con 0x556c6840bdf0 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f53779e8740 1 -- 172.21.15.78:0/284748698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5368010e30 con 0x556c6840bdf0 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f5352ff5640 1 -- 172.21.15.78:0/284748698 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f5364005d60 con 0x556c6840bdf0 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f53779e8740 10 client.19632 fetch_fsmap learned FSMap version 80 2024-09-11T01:24:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f53779e8740 1 -- 172.21.15.78:0/284748698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x556c68494630 con 0x556c6840bdf0 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f5352ff5640 1 -- 172.21.15.78:0/284748698 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5368010e30 con 0x556c6840bdf0 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f5352ff5640 1 -- 172.21.15.78:0/284748698 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 80) ==== 8406+0+0 (secure 0 0 0) 0x7f5364024450 con 0x556c6840bdf0 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f53779e8740 10 client.19632 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f53779e8740 10 client.19632 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f53779e8740 20 client.19632 populate_metadata read hostname 'smithi078' 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f53779e8740 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556c68494e10 0x556c68497230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f53779e8740 4 client.19632 mds_command: new command op to 14808 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.583+0000 7f53779e8740 1 -- 172.21.15.78:0/284748698 --> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c68429e10 con 0x556c68494e10 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f536effd640 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556c68494e10 0x556c68497230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f536effd640 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556c68494e10 0x556c68497230 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=591d5196c33b3ca8 server_cookie=3cc0ce8fb970273e in_seq=0 out_seq=0 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f5352ff5640 10 client.19632 ms_handle_connect on v2:172.21.15.78:6840/3713123708 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f5352ff5640 1 -- 172.21.15.78:0/284748698 <== mds.4 v2:172.21.15.78:6840/3713123708 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c68429e10 con 0x556c68494e10 2024-09-11T01:24:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f5352ff5640 10 client.19632 handle_command_reply: tid=0 2024-09-11T01:24:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f53779e8740 1 client.19632 shutdown 2024-09-11T01:24:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f53779e8740 1 -- 172.21.15.78:0/284748698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 msgr2=0x556c6841c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f53779e8740 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 0x556c6841c1a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f536400f7d0 tx=0x7f5364004910 comp rx=0 tx=0).stop 2024-09-11T01:24:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f53779e8740 1 -- 172.21.15.78:0/284748698 shutdown_connections 2024-09-11T01:24:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f53779e8740 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.78:6840/3713123708,v1:172.21.15.78:6841/3713123708] conn(0x556c68494e10 0x556c68497230 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f53779e8740 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c68417d80 0x556c6841c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f53779e8740 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c6840bdf0 0x556c6841c1a0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f5352ff5640 0 client.19632 ms_handle_reset on v2:172.21.15.78:6840/3713123708 2024-09-11T01:24:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f53779e8740 1 --2- 172.21.15.78:0/284748698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c6840b4d0 0x556c6841bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f53779e8740 1 -- 172.21.15.78:0/284748698 >> 172.21.15.78:0/284748698 conn(0x556c68409300 msgr2=0x556c6840a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f53779e8740 1 -- 172.21.15.78:0/284748698 shutdown_connections 2024-09-11T01:24:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f53779e8740 1 -- 172.21.15.78:0/284748698 wait complete. 2024-09-11T01:24:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.584+0000 7f53779e8740 20 client.19632 trim_cache size 0 max 16384 2024-09-11T01:24:11.593 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:24:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.585+0000 7f5375613640 1 -- 172.21.15.78:0/3097332180 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f53580b5450 msgr2=0x7f53580b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.585+0000 7f5375613640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f53580b5450 0x7f53580b7910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5368005c40 tx=0x7f5368004a40 comp rx=0 tx=0).stop 2024-09-11T01:24:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.585+0000 7f5375613640 1 -- 172.21.15.78:0/3097332180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5370102920 msgr2=0x7f537010f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.585+0000 7f5375613640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5370102920 0x7f537010f350 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f5360006010 tx=0x7f53600042f0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.585+0000 7f5375613640 1 -- 172.21.15.78:0/3097332180 shutdown_connections 2024-09-11T01:24:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.585+0000 7f5375613640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f53580b5450 0x7f53580b7910 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.585+0000 7f5375613640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5370108120 0x7f537010fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.585+0000 7f5375613640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5370107780 0x7f537010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.585+0000 7f5375613640 1 --2- 172.21.15.78:0/3097332180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5370102920 0x7f537010f350 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.585+0000 7f5375613640 1 -- 172.21.15.78:0/3097332180 >> 172.21.15.78:0/3097332180 conn(0x7f5370076f50 msgr2=0x7f5370100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.585+0000 7f5375613640 1 -- 172.21.15.78:0/3097332180 shutdown_connections 2024-09-11T01:24:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:11.585+0000 7f5375613640 1 -- 172.21.15.78:0/3097332180 wait complete. 2024-09-11T01:24:11.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:11 smithi078 ceph-mon[31315]: pgmap v1119: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 210 op/s 2024-09-11T01:24:11.841 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:11 smithi110 ceph-mon[34712]: pgmap v1119: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 210 op/s 2024-09-11T01:24:11.875 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:11 smithi112 ceph-mon[35384]: pgmap v1119: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 210 op/s 2024-09-11T01:24:12.425 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T01:24:12.677 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:13.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.140+0000 7f6a05170640 1 -- 172.21.15.78:0/909364945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a0010d140 msgr2=0x7f6a0010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.140+0000 7f6a05170640 1 --2- 172.21.15.78:0/909364945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a0010d140 0x7f6a0010f5a0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009900 tx=0x7f69f802a410 comp rx=0 tx=0).stop 2024-09-11T01:24:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.141+0000 7f6a05170640 1 -- 172.21.15.78:0/909364945 shutdown_connections 2024-09-11T01:24:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.141+0000 7f6a05170640 1 --2- 172.21.15.78:0/909364945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a0010d140 0x7f6a0010f5a0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.141+0000 7f6a05170640 1 --2- 172.21.15.78:0/909364945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a00104600 0x7f6a0010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.141+0000 7f6a05170640 1 --2- 172.21.15.78:0/909364945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a00103ce0 0x7f6a001040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.141+0000 7f6a05170640 1 -- 172.21.15.78:0/909364945 >> 172.21.15.78:0/909364945 conn(0x7f6a000fd700 msgr2=0x7f6a000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.141+0000 7f6a05170640 1 -- 172.21.15.78:0/909364945 shutdown_connections 2024-09-11T01:24:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.141+0000 7f6a05170640 1 -- 172.21.15.78:0/909364945 wait complete. 2024-09-11T01:24:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.141+0000 7f6a05170640 1 Processor -- start 2024-09-11T01:24:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.141+0000 7f6a05170640 1 -- start start 2024-09-11T01:24:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f6a05170640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a00103ce0 0x7f6a00101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f6a05170640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a00104600 0x7f6a00101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f6a05170640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a0010d140 0x7f6a00102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f6a05170640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a00112df0 con 0x7f6a00103ce0 2024-09-11T01:24:13.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f6a05170640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6a00112c70 con 0x7f6a00104600 2024-09-11T01:24:13.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f6a05170640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a00112f70 con 0x7f6a0010d140 2024-09-11T01:24:13.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69fed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a00103ce0 0x7f6a00101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69fed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a00103ce0 0x7f6a00101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43500/0 (socket says 172.21.15.78:43500) 2024-09-11T01:24:13.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69fed9d640 1 -- 172.21.15.78:0/3658325257 learned_addr learned my addr 172.21.15.78:0/3658325257 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:13.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69fe59c640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a00104600 0x7f6a00101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69ff59e640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a0010d140 0x7f6a00102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69fed9d640 1 -- 172.21.15.78:0/3658325257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a0010d140 msgr2=0x7f6a00102410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:13.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69fed9d640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a0010d140 0x7f6a00102410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69fed9d640 1 -- 172.21.15.78:0/3658325257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a00104600 msgr2=0x7f6a00101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69fed9d640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a00104600 0x7f6a00101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69fed9d640 1 -- 172.21.15.78:0/3658325257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a001a76a0 con 0x7f6a00103ce0 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69fed9d640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a00103ce0 0x7f6a00101990 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f69f000ba50 tx=0x7f69f000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69f7fff640 1 -- 172.21.15.78:0/3658325257 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0019070 con 0x7f6a00103ce0 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f6a05170640 1 -- 172.21.15.78:0/3658325257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a001a7990 con 0x7f6a00103ce0 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69f7fff640 1 -- 172.21.15.78:0/3658325257 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f000cbe0 con 0x7f6a00103ce0 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.142+0000 7f69f7fff640 1 -- 172.21.15.78:0/3658325257 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f00075d0 con 0x7f6a00103ce0 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.143+0000 7f6a05170640 1 -- 172.21.15.78:0/3658325257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a001a7f50 con 0x7f6a00103ce0 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.145+0000 7f69f7fff640 1 -- 172.21.15.78:0/3658325257 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f69f0007830 con 0x7f6a00103ce0 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.145+0000 7f6a07545740 1 Processor -- start 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.145+0000 7f6a07545740 1 -- start start 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.145+0000 7f6a07545740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d050f0 0x5612b6d054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.145+0000 7f6a07545740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d11460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f6a07545740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d119a0 0x5612b6d13d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f6a07545740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5612b6c7bea0 con 0x5612b6d119a0 2024-09-11T01:24:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f6a07545740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5612b6c6b2d0 con 0x5612b6d050f0 2024-09-11T01:24:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f6a07545740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5612b6c6f0e0 con 0x5612b6d05a10 2024-09-11T01:24:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69fe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d119a0 0x5612b6d13d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69fe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d119a0 0x5612b6d13d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43506/0 (socket says 172.21.15.78:43506) 2024-09-11T01:24:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69fe59c640 1 -- 172.21.15.78:0/2960863422 learned_addr learned my addr 172.21.15.78:0/2960863422 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69fed9d640 1 --2- 172.21.15.78:0/2960863422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d11460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69ff59e640 1 --2- 172.21.15.78:0/2960863422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d050f0 0x5612b6d054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69f7fff640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69ec0b5450 0x7f69ec0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69fe59c640 1 -- 172.21.15.78:0/2960863422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 msgr2=0x5612b6d11460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69fe59c640 1 --2- 172.21.15.78:0/2960863422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d11460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69fe59c640 1 -- 172.21.15.78:0/2960863422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d050f0 msgr2=0x5612b6d054d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69fe59c640 1 --2- 172.21.15.78:0/2960863422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d050f0 0x5612b6d054d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69fe59c640 1 -- 172.21.15.78:0/2960863422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b69c9650 con 0x5612b6d119a0 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69ff59e640 1 --2- 172.21.15.78:0/2960863422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d050f0 0x5612b6d054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69ff59e640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69ec0b5450 0x7f69ec0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69f7fff640 1 -- 172.21.15.78:0/3658325257 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8493+0+0 (secure 0 0 0) 0x7f69f00e5770 con 0x7f6a00103ce0 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69fe59c640 1 --2- 172.21.15.78:0/2960863422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d119a0 0x5612b6d13d90 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f69e800abb0 tx=0x7f69e800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f900a4934e7ac4f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.146+0000 7f69f57fa640 1 -- 172.21.15.78:0/2960863422 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8004220 con 0x5612b6d119a0 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.147+0000 7f69f57fa640 1 -- 172.21.15.78:0/2960863422 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69e80043c0 con 0x5612b6d119a0 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.147+0000 7f69f57fa640 1 -- 172.21.15.78:0/2960863422 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e80187a0 con 0x5612b6d119a0 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.147+0000 7f6a07545740 1 -- 172.21.15.78:0/2960863422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d119a0 msgr2=0x5612b6d13d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.147+0000 7f69ff59e640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69ec0b5450 0x7f69ec0b7910 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f69f8031440 tx=0x7f69f8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:13.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.147+0000 7f6a07545740 1 --2- 172.21.15.78:0/2960863422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d119a0 0x5612b6d13d90 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f69e800abb0 tx=0x7f69e800f790 comp rx=0 tx=0).stop 2024-09-11T01:24:13.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.147+0000 7f6a07545740 1 -- 172.21.15.78:0/2960863422 shutdown_connections 2024-09-11T01:24:13.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.147+0000 7f6a07545740 1 --2- 172.21.15.78:0/2960863422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d119a0 0x5612b6d13d90 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.147+0000 7f6a07545740 1 --2- 172.21.15.78:0/2960863422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d11460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.147+0000 7f6a07545740 1 --2- 172.21.15.78:0/2960863422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d050f0 0x5612b6d054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.147+0000 7f6a07545740 1 -- 172.21.15.78:0/2960863422 >> 172.21.15.78:0/2960863422 conn(0x5612b6d02f20 msgr2=0x5612b6d040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:13.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.147+0000 7f6a07545740 1 -- 172.21.15.78:0/2960863422 shutdown_connections 2024-09-11T01:24:13.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.147+0000 7f6a07545740 1 -- 172.21.15.78:0/2960863422 wait complete. 2024-09-11T01:24:13.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f6a07545740 1 -- start start 2024-09-11T01:24:13.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f6a07545740 1 Processor -- start 2024-09-11T01:24:13.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f6a07545740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 0x5612b6d11260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f6a07545740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d0e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f6a07545740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d119a0 0x5612b6d0ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f6a07545740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5612b6acc700 con 0x5612b6d050f0 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f6a07545740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5612b6c06160 con 0x5612b6d119a0 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f6a07545740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5612b6d14620 con 0x5612b6d05a10 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f6a07545740 10 client.0 _ll_register_callbacks cb 0x5612b6c1f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f69fed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 0x5612b6d11260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f69fed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 0x5612b6d11260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43516/0 (socket says 172.21.15.78:43516) 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f69fed9d640 1 -- 172.21.15.78:0/3627062663 learned_addr learned my addr 172.21.15.78:0/3627062663 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.148+0000 7f69fe59c640 1 --2- 172.21.15.78:0/3627062663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d0e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f69fed9d640 1 -- 172.21.15.78:0/3627062663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 msgr2=0x5612b6d0e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f69fed9d640 1 --2- 172.21.15.78:0/3627062663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d0e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f69fed9d640 1 -- 172.21.15.78:0/3627062663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d119a0 msgr2=0x5612b6d0ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f69fed9d640 1 --2- 172.21.15.78:0/3627062663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d119a0 0x5612b6d0ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f69fed9d640 1 -- 172.21.15.78:0/3627062663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b6c33c90 con 0x5612b6d050f0 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f69fed9d640 1 --2- 172.21.15.78:0/3627062663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 0x5612b6d11260 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f69f00ab9c0 tx=0x7f69f00099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e266b53f80739d39 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f69e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f69e67fc640 1 -- 172.21.15.78:0/3627062663 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f00e3370 con 0x5612b6d050f0 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f69e67fc640 1 -- 172.21.15.78:0/3627062663 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8493+0+0 (secure 0 0 0) 0x7f69f00b4d10 con 0x5612b6d050f0 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f6a07545740 1 -- 172.21.15.78:0/3627062663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69e80043c0 con 0x5612b6d050f0 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f69e67fc640 1 -- 172.21.15.78:0/3627062663 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f69f00ab7d0 con 0x5612b6d050f0 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f6a07545740 10 client.19644 fetch_fsmap learned FSMap version 80 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f6a07545740 1 -- 172.21.15.78:0/3627062663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x5612b6d1ff50 con 0x5612b6d050f0 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f69e67fc640 1 -- 172.21.15.78:0/3627062663 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69e80043c0 con 0x5612b6d050f0 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f69e67fc640 1 -- 172.21.15.78:0/3627062663 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 80) ==== 8406+0+0 (secure 0 0 0) 0x7f69f00aaf20 con 0x5612b6d050f0 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f6a07545740 10 client.19644 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f6a07545740 10 client.19644 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:24:13.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f6a07545740 20 client.19644 populate_metadata read hostname 'smithi078' 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f6a07545740 1 --2- 172.21.15.78:0/3627062663 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5612b6d21400 0x5612b6d238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f6a07545740 4 client.19644 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.149+0000 7f6a07545740 1 -- 172.21.15.78:0/3627062663 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612b6c7ddf0 con 0x5612b6d21400 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.150+0000 7f69fe59c640 1 --2- 172.21.15.78:0/3627062663 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5612b6d21400 0x5612b6d238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.150+0000 7f69e67fc640 10 client.19644 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.150+0000 7f69fe59c640 1 --2- 172.21.15.78:0/3627062663 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5612b6d21400 0x5612b6d238a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=afbe50a5b04ac1cc server_cookie=4adc284fcc45a8dd in_seq=0 out_seq=0 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f69e67fc640 1 -- 172.21.15.78:0/3627062663 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5612b6c7ddf0 con 0x5612b6d21400 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f69e67fc640 10 client.19644 handle_command_reply: tid=0 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f6a07545740 1 client.19644 shutdown 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f6a07545740 1 -- 172.21.15.78:0/3627062663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 msgr2=0x5612b6d11260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f6a07545740 1 --2- 172.21.15.78:0/3627062663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 0x5612b6d11260 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f69f00ab9c0 tx=0x7f69f00099b0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f6a07545740 1 -- 172.21.15.78:0/3627062663 shutdown_connections 2024-09-11T01:24:13.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f6a07545740 1 --2- 172.21.15.78:0/3627062663 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5612b6d21400 0x5612b6d238a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f6a07545740 1 --2- 172.21.15.78:0/3627062663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d119a0 0x5612b6d0ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f69e67fc640 0 client.19644 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:24:13.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f6a07545740 1 --2- 172.21.15.78:0/3627062663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d0e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f6a07545740 1 --2- 172.21.15.78:0/3627062663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 0x5612b6d11260 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f6a07545740 1 -- 172.21.15.78:0/3627062663 >> 172.21.15.78:0/3627062663 conn(0x5612b6d02f20 msgr2=0x5612b6d08a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:13.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f6a07545740 1 -- 172.21.15.78:0/3627062663 shutdown_connections 2024-09-11T01:24:13.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f6a07545740 1 -- 172.21.15.78:0/3627062663 wait complete. 2024-09-11T01:24:13.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.151+0000 7f6a07545740 20 client.19644 trim_cache size 0 max 16384 2024-09-11T01:24:13.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.164+0000 7f6a07545740 1 Processor -- start 2024-09-11T01:24:13.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.164+0000 7f6a07545740 1 -- start start 2024-09-11T01:24:13.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f6a07545740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 0x5612b6d1d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f6a07545740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d12d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f6a07545740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d119a0 0x5612b6d132d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f6a07545740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a001b0830 con 0x5612b6d050f0 2024-09-11T01:24:13.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f6a07545740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6a001b24c0 con 0x5612b6d119a0 2024-09-11T01:24:13.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f6a07545740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a001b0f00 con 0x5612b6d05a10 2024-09-11T01:24:13.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69fed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 0x5612b6d1d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69fed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 0x5612b6d1d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43530/0 (socket says 172.21.15.78:43530) 2024-09-11T01:24:13.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69fed9d640 1 -- 172.21.15.78:0/113489466 learned_addr learned my addr 172.21.15.78:0/113489466 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:13.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69ff59e640 1 --2- 172.21.15.78:0/113489466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d119a0 0x5612b6d132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69fe59c640 1 --2- 172.21.15.78:0/113489466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d12d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69fed9d640 1 -- 172.21.15.78:0/113489466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 msgr2=0x5612b6d12d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:13.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69fed9d640 1 --2- 172.21.15.78:0/113489466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d12d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69fed9d640 1 -- 172.21.15.78:0/113489466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d119a0 msgr2=0x5612b6d132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69fed9d640 1 --2- 172.21.15.78:0/113489466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d119a0 0x5612b6d132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69fed9d640 1 -- 172.21.15.78:0/113489466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b6d0e460 con 0x5612b6d050f0 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69fe59c640 1 --2- 172.21.15.78:0/113489466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d12d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69fed9d640 1 --2- 172.21.15.78:0/113489466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 0x5612b6d1d880 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f69f0000f80 tx=0x7f69f0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93caf754153ddd6c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69f4ff9640 1 -- 172.21.15.78:0/113489466 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f00e3370 con 0x5612b6d050f0 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.165+0000 7f69f4ff9640 1 -- 172.21.15.78:0/113489466 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f00b06f0 con 0x5612b6d050f0 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.166+0000 7f69f4ff9640 1 -- 172.21.15.78:0/113489466 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f00ab610 con 0x5612b6d050f0 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.166+0000 7f6a07545740 1 -- 172.21.15.78:0/113489466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 msgr2=0x5612b6d1d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.166+0000 7f6a07545740 1 --2- 172.21.15.78:0/113489466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 0x5612b6d1d880 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f69f0000f80 tx=0x7f69f0004b50 comp rx=0 tx=0).stop 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.166+0000 7f6a07545740 1 -- 172.21.15.78:0/113489466 shutdown_connections 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.166+0000 7f6a07545740 1 --2- 172.21.15.78:0/113489466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d119a0 0x5612b6d132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.166+0000 7f6a07545740 1 --2- 172.21.15.78:0/113489466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d05a10 0x5612b6d12d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.166+0000 7f6a07545740 1 --2- 172.21.15.78:0/113489466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d050f0 0x5612b6d1d880 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.166+0000 7f6a07545740 1 -- 172.21.15.78:0/113489466 >> 172.21.15.78:0/113489466 conn(0x5612b6d02f20 msgr2=0x5612b6d0cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.166+0000 7f6a07545740 1 -- 172.21.15.78:0/113489466 shutdown_connections 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.166+0000 7f6a07545740 1 -- 172.21.15.78:0/113489466 wait complete. 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f6a07545740 1 -- start start 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f6a07545740 1 Processor -- start 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f6a07545740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d050f0 0x5612b6d19d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f6a07545740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d05a10 0x5612b6d1a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f6a07545740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d119a0 0x5612b6d14af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f6a07545740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a001b1d60 con 0x5612b6d05a10 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f6a07545740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6a001b1660 con 0x5612b6d050f0 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f6a07545740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a001ae830 con 0x5612b6d119a0 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f6a07545740 10 client.0 _ll_register_callbacks cb 0x7f69f00a7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:24:13.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f69fed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d050f0 0x5612b6d19d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f69fe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d05a10 0x5612b6d1a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f69fed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d050f0 0x5612b6d19d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:34328/0 (socket says 172.21.15.78:34328) 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f69fed9d640 1 -- 172.21.15.78:0/752109381 learned_addr learned my addr 172.21.15.78:0/752109381 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f69fe59c640 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d05a10 0x5612b6d1a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43538/0 (socket says 172.21.15.78:43538) 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.167+0000 7f69ff59e640 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d119a0 0x5612b6d14af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f69fe59c640 1 -- 172.21.15.78:0/752109381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d119a0 msgr2=0x5612b6d14af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f69fe59c640 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d119a0 0x5612b6d14af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f69fe59c640 1 -- 172.21.15.78:0/752109381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d050f0 msgr2=0x5612b6d19d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f69fe59c640 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d050f0 0x5612b6d19d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f69fe59c640 1 -- 172.21.15.78:0/752109381 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b6d15320 con 0x5612b6d05a10 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f69ff59e640 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d119a0 0x5612b6d14af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f69fe59c640 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d05a10 0x5612b6d1a290 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f69e8005b30 tx=0x7f69e8004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d3665a947ea941 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f69e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:24:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f69e67fc640 1 -- 172.21.15.78:0/752109381 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8005250 con 0x5612b6d05a10 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f6a07545740 1 -- 172.21.15.78:0/752109381 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f00b06f0 con 0x5612b6d05a10 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f69e67fc640 1 -- 172.21.15.78:0/752109381 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8493+0+0 (secure 0 0 0) 0x7f69e801e040 con 0x5612b6d05a10 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f69e67fc640 1 -- 172.21.15.78:0/752109381 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f69e80053f0 con 0x5612b6d05a10 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f6a07545740 10 client.19650 fetch_fsmap learned FSMap version 80 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.168+0000 7f6a07545740 1 -- 172.21.15.78:0/752109381 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x5612b6d97880 con 0x5612b6d05a10 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.169+0000 7f69e67fc640 1 -- 172.21.15.78:0/752109381 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f00b06f0 con 0x5612b6d05a10 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.169+0000 7f69e67fc640 1 -- 172.21.15.78:0/752109381 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 80) ==== 8406+0+0 (secure 0 0 0) 0x7f69e8034070 con 0x5612b6d05a10 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.169+0000 7f6a07545740 10 client.19650 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.169+0000 7f6a07545740 10 client.19650 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.169+0000 7f6a07545740 20 client.19650 populate_metadata read hostname 'smithi078' 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.169+0000 7f6a07545740 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5612b6d97f40 0x5612b6d9a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.169+0000 7f6a07545740 4 client.19650 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:24:13.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.169+0000 7f6a07545740 1 -- 172.21.15.78:0/752109381 --> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612b6c798a0 con 0x5612b6d97f40 2024-09-11T01:24:13.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.169+0000 7f69fed9d640 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5612b6d97f40 0x5612b6d9a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:13.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.169+0000 7f69e67fc640 10 client.19650 ms_handle_connect on v2:172.21.15.110:6836/1873031527 2024-09-11T01:24:13.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.169+0000 7f69fed9d640 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5612b6d97f40 0x5612b6d9a360 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b9b0f9733cd1434 server_cookie=fa87f923ffcf4386 in_seq=0 out_seq=0 2024-09-11T01:24:13.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f69e67fc640 1 -- 172.21.15.78:0/752109381 <== mds.3 v2:172.21.15.110:6836/1873031527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612b6c798a0 con 0x5612b6d97f40 2024-09-11T01:24:13.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f69e67fc640 10 client.19650 handle_command_reply: tid=0 2024-09-11T01:24:13.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a07545740 1 client.19650 shutdown 2024-09-11T01:24:13.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a07545740 1 -- 172.21.15.78:0/752109381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d05a10 msgr2=0x5612b6d1a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:13.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a07545740 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d05a10 0x5612b6d1a290 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f69e8005b30 tx=0x7f69e8004400 comp rx=0 tx=0).stop 2024-09-11T01:24:13.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a07545740 1 -- 172.21.15.78:0/752109381 shutdown_connections 2024-09-11T01:24:13.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a07545740 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.110:6836/1873031527,v1:172.21.15.110:6837/1873031527] conn(0x5612b6d97f40 0x5612b6d9a360 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a07545740 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612b6d119a0 0x5612b6d14af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a07545740 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612b6d05a10 0x5612b6d1a290 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a07545740 1 --2- 172.21.15.78:0/752109381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5612b6d050f0 0x5612b6d19d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a07545740 1 -- 172.21.15.78:0/752109381 >> 172.21.15.78:0/752109381 conn(0x5612b6d02f20 msgr2=0x5612b6d0fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:13.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f69e67fc640 0 client.19650 ms_handle_reset on v2:172.21.15.110:6836/1873031527 2024-09-11T01:24:13.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a07545740 1 -- 172.21.15.78:0/752109381 shutdown_connections 2024-09-11T01:24:13.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a07545740 1 -- 172.21.15.78:0/752109381 wait complete. 2024-09-11T01:24:13.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a07545740 20 client.19650 trim_cache size 0 max 16384 2024-09-11T01:24:13.188 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:24:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a05170640 1 -- 172.21.15.78:0/3658325257 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69ec0b5450 msgr2=0x7f69ec0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.170+0000 7f6a05170640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69ec0b5450 0x7f69ec0b7910 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f69f8031440 tx=0x7f69f8009260 comp rx=0 tx=0).stop 2024-09-11T01:24:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.171+0000 7f6a05170640 1 -- 172.21.15.78:0/3658325257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a00103ce0 msgr2=0x7f6a00101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.171+0000 7f6a05170640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a00103ce0 0x7f6a00101990 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f69f000ba50 tx=0x7f69f000bf10 comp rx=0 tx=0).stop 2024-09-11T01:24:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.171+0000 7f6a05170640 1 -- 172.21.15.78:0/3658325257 shutdown_connections 2024-09-11T01:24:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.171+0000 7f6a05170640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f69ec0b5450 0x7f69ec0b7910 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.171+0000 7f6a05170640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a0010d140 0x7f6a00102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.171+0000 7f6a05170640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a00104600 0x7f6a00101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.171+0000 7f6a05170640 1 --2- 172.21.15.78:0/3658325257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a00103ce0 0x7f6a00101990 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:13.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.171+0000 7f6a05170640 1 -- 172.21.15.78:0/3658325257 >> 172.21.15.78:0/3658325257 conn(0x7f6a000fd700 msgr2=0x7f6a000fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:13.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.171+0000 7f6a05170640 1 -- 172.21.15.78:0/3658325257 shutdown_connections 2024-09-11T01:24:13.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:13.171+0000 7f6a05170640 1 -- 172.21.15.78:0/3658325257 wait complete. 2024-09-11T01:24:13.831 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:13 smithi078 ceph-mon[31315]: pgmap v1120: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 99 KiB/s wr, 48 op/s 2024-09-11T01:24:13.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:13 smithi110 ceph-mon[34712]: pgmap v1120: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 99 KiB/s wr, 48 op/s 2024-09-11T01:24:13.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:13 smithi112 ceph-mon[35384]: pgmap v1120: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 99 KiB/s wr, 48 op/s 2024-09-11T01:24:14.087 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T01:24:14.331 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:14.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:14 smithi078 ceph-mon[31315]: osdmap e115: 12 total, 12 up, 12 in 2024-09-11T01:24:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.824+0000 7f781e617640 1 -- 172.21.15.78:0/3618506685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7818108060 msgr2=0x7f781810e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.824+0000 7f781e617640 1 --2- 172.21.15.78:0/3618506685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7818108060 0x7f781810e860 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f7810009ca0 tx=0x7f781002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.824+0000 7f781e617640 1 -- 172.21.15.78:0/3618506685 shutdown_connections 2024-09-11T01:24:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.824+0000 7f781e617640 1 --2- 172.21.15.78:0/3618506685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7818108060 0x7f781810e860 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.824+0000 7f781e617640 1 --2- 172.21.15.78:0/3618506685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78181076e0 0x7f7818107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.824+0000 7f781e617640 1 --2- 172.21.15.78:0/3618506685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7818101cd0 0x7f78181020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.824+0000 7f781e617640 1 -- 172.21.15.78:0/3618506685 >> 172.21.15.78:0/3618506685 conn(0x7f78180fd700 msgr2=0x7f78180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.824+0000 7f781e617640 1 -- 172.21.15.78:0/3618506685 shutdown_connections 2024-09-11T01:24:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.824+0000 7f781e617640 1 -- 172.21.15.78:0/3618506685 wait complete. 2024-09-11T01:24:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.825+0000 7f781e617640 1 Processor -- start 2024-09-11T01:24:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.825+0000 7f781e617640 1 -- start start 2024-09-11T01:24:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.825+0000 7f781e617640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7818101cd0 0x7f7818198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.825+0000 7f781e617640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78181076e0 0x7f78181986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.825+0000 7f781e617640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7818108060 0x7f781819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.825+0000 7f781e617640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7818111d20 con 0x7f7818101cd0 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.825+0000 7f781e617640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7818111ba0 con 0x7f78181076e0 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.825+0000 7f781e617640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7818111ea0 con 0x7f7818108060 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.825+0000 7f7817fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7818101cd0 0x7f7818198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.825+0000 7f7817fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7818101cd0 0x7f7818198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41842/0 (socket says 172.21.15.78:41842) 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.825+0000 7f7817fff640 1 -- 172.21.15.78:0/1062191479 learned_addr learned my addr 172.21.15.78:0/1062191479 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.825+0000 7f781cbb4640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7818108060 0x7f781819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.826+0000 7f7817fff640 1 -- 172.21.15.78:0/1062191479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7818108060 msgr2=0x7f781819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.826+0000 7f7817fff640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7818108060 0x7f781819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.826+0000 7f7817fff640 1 -- 172.21.15.78:0/1062191479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78181076e0 msgr2=0x7f78181986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.826+0000 7f7817fff640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78181076e0 0x7f78181986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.826+0000 7f7817fff640 1 -- 172.21.15.78:0/1062191479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781819e220 con 0x7f7818101cd0 2024-09-11T01:24:14.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.826+0000 7f7817fff640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7818101cd0 0x7f7818198170 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f780800d950 tx=0x7f780800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.826+0000 7f7815ffb640 1 -- 172.21.15.78:0/1062191479 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808017070 con 0x7f7818101cd0 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.826+0000 7f7815ffb640 1 -- 172.21.15.78:0/1062191479 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7808009c30 con 0x7f7818101cd0 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.826+0000 7f781e617640 1 -- 172.21.15.78:0/1062191479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781819e510 con 0x7f7818101cd0 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.826+0000 7f7815ffb640 1 -- 172.21.15.78:0/1062191479 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780800f620 con 0x7f7818101cd0 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.826+0000 7f781e617640 1 -- 172.21.15.78:0/1062191479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78181a4da0 con 0x7f7818101cd0 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.828+0000 7f78209ec740 1 Processor -- start 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.828+0000 7f78209ec740 1 -- start start 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.828+0000 7f7815ffb640 1 -- 172.21.15.78:0/1062191479 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7808010460 con 0x7f7818101cd0 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f78209ec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 0x55de064158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f78209ec740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de06415df0 0x55de06423a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f78209ec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06423f90 0x55de06426380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f78209ec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de0638bea0 con 0x55de064154d0 2024-09-11T01:24:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f78209ec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55de0637b2d0 con 0x55de06423f90 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f78209ec740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55de0637f0e0 con 0x55de06415df0 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f781cbb4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 0x55de064158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f781cbb4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 0x55de064158b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41848/0 (socket says 172.21.15.78:41848) 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f781cbb4640 1 -- 172.21.15.78:0/2115935494 learned_addr learned my addr 172.21.15.78:0/2115935494 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f7817fff640 1 --2- 172.21.15.78:0/2115935494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de06415df0 0x55de06423a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f780ffff640 1 --2- 172.21.15.78:0/2115935494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06423f90 0x55de06426380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f781cbb4640 1 -- 172.21.15.78:0/2115935494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de06415df0 msgr2=0x55de06423a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f781cbb4640 1 --2- 172.21.15.78:0/2115935494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de06415df0 0x55de06423a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f781cbb4640 1 -- 172.21.15.78:0/2115935494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06423f90 msgr2=0x55de06426380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f781cbb4640 1 --2- 172.21.15.78:0/2115935494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06423f90 0x55de06426380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f781cbb4640 1 -- 172.21.15.78:0/2115935494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de060d9650 con 0x55de064154d0 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f781cbb4640 1 --2- 172.21.15.78:0/2115935494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 0x55de064158b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f7810005cc0 tx=0x7f78100099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcba5ab86a4e7834 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f7815ffb640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78040b54d0 0x7f78040b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f7817fff640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78040b54d0 0x7f78040b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.829+0000 7f780e7fc640 1 -- 172.21.15.78:0/2115935494 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810031450 con 0x55de064154d0 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f780e7fc640 1 -- 172.21.15.78:0/2115935494 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7810036040 con 0x55de064154d0 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f780e7fc640 1 -- 172.21.15.78:0/2115935494 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810004440 con 0x55de064154d0 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f7815ffb640 1 -- 172.21.15.78:0/1062191479 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f78080e5c70 con 0x7f7818101cd0 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f78209ec740 1 -- 172.21.15.78:0/2115935494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 msgr2=0x55de064158b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f7817fff640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78040b54d0 0x7f78040b7990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f78080e34b0 tx=0x7f78080b5040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f78209ec740 1 --2- 172.21.15.78:0/2115935494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 0x55de064158b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f7810005cc0 tx=0x7f78100099c0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f78209ec740 1 -- 172.21.15.78:0/2115935494 shutdown_connections 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f78209ec740 1 --2- 172.21.15.78:0/2115935494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06423f90 0x55de06426380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f78209ec740 1 --2- 172.21.15.78:0/2115935494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de06415df0 0x55de06423a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f78209ec740 1 --2- 172.21.15.78:0/2115935494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 0x55de064158b0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f78209ec740 1 -- 172.21.15.78:0/2115935494 >> 172.21.15.78:0/2115935494 conn(0x55de06413300 msgr2=0x55de064144a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:14.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f78209ec740 1 -- 172.21.15.78:0/2115935494 shutdown_connections 2024-09-11T01:24:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.830+0000 7f78209ec740 1 -- 172.21.15.78:0/2115935494 wait complete. 2024-09-11T01:24:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.831+0000 7f78209ec740 1 -- start start 2024-09-11T01:24:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.831+0000 7f78209ec740 1 Processor -- start 2024-09-11T01:24:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.831+0000 7f78209ec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 0x55de06423860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.831+0000 7f78209ec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06415df0 0x55de06420cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.831+0000 7f78209ec740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de06423f90 0x55de06421290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.831+0000 7f78209ec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de061dc700 con 0x55de064154d0 2024-09-11T01:24:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.831+0000 7f78209ec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55de06316160 con 0x55de06415df0 2024-09-11T01:24:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.831+0000 7f78209ec740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55de06426be0 con 0x55de06423f90 2024-09-11T01:24:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.831+0000 7f78209ec740 10 client.0 _ll_register_callbacks cb 0x55de0632f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:24:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.831+0000 7f781cbb4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 0x55de06423860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.831+0000 7f781cbb4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 0x55de06423860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41856/0 (socket says 172.21.15.78:41856) 2024-09-11T01:24:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.831+0000 7f781cbb4640 1 -- 172.21.15.78:0/807707797 learned_addr learned my addr 172.21.15.78:0/807707797 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f781cbb4640 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de06423f90 0x55de06421290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f780ffff640 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06415df0 0x55de06420cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f781cbb4640 1 -- 172.21.15.78:0/807707797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de06423f90 msgr2=0x55de06421290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f781cbb4640 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de06423f90 0x55de06421290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f781cbb4640 1 -- 172.21.15.78:0/807707797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06415df0 msgr2=0x55de06420cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f781cbb4640 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06415df0 0x55de06420cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f781cbb4640 1 -- 172.21.15.78:0/807707797 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de06343c90 con 0x55de064154d0 2024-09-11T01:24:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f780ffff640 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06415df0 0x55de06420cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f781cbb4640 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 0x55de06423860 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f7810003eb0 tx=0x7f7810003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a6fcf837bb5fd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f77f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:24:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f77f77fe640 1 -- 172.21.15.78:0/807707797 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810058060 con 0x55de064154d0 2024-09-11T01:24:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f77f77fe640 1 -- 172.21.15.78:0/807707797 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f7810031860 con 0x55de064154d0 2024-09-11T01:24:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f78209ec740 1 -- 172.21.15.78:0/807707797 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7810036040 con 0x55de064154d0 2024-09-11T01:24:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f77f77fe640 1 -- 172.21.15.78:0/807707797 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f7810004140 con 0x55de064154d0 2024-09-11T01:24:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f78209ec740 10 client.19662 fetch_fsmap learned FSMap version 80 2024-09-11T01:24:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f78209ec740 1 -- 172.21.15.78:0/807707797 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55de064324f0 con 0x55de064154d0 2024-09-11T01:24:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f77f77fe640 1 -- 172.21.15.78:0/807707797 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7810036040 con 0x55de064154d0 2024-09-11T01:24:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f77f77fe640 1 -- 172.21.15.78:0/807707797 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 80) ==== 8406+0+0 (secure 0 0 0) 0x7f781002faf0 con 0x55de064154d0 2024-09-11T01:24:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.832+0000 7f78209ec740 10 client.19662 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.833+0000 7f78209ec740 10 client.19662 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.833+0000 7f78209ec740 20 client.19662 populate_metadata read hostname 'smithi078' 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.833+0000 7f78209ec740 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55de06433840 0x55de06435c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.833+0000 7f78209ec740 4 client.19662 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.833+0000 7f78209ec740 1 -- 172.21.15.78:0/807707797 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de0638ddf0 con 0x55de06433840 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.833+0000 7f780ffff640 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55de06433840 0x55de06435c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.833+0000 7f77f77fe640 10 client.19662 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.833+0000 7f780ffff640 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55de06433840 0x55de06435c30 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2dbaa4ec3a59226c server_cookie=d790e640f5b93f65 in_seq=0 out_seq=0 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f77f77fe640 1 -- 172.21.15.78:0/807707797 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de0638ddf0 con 0x55de06433840 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f77f77fe640 10 client.19662 handle_command_reply: tid=0 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f78209ec740 1 client.19662 shutdown 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f78209ec740 1 -- 172.21.15.78:0/807707797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 msgr2=0x55de06423860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f78209ec740 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 0x55de06423860 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f7810003eb0 tx=0x7f7810003f10 comp rx=0 tx=0).stop 2024-09-11T01:24:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f78209ec740 1 -- 172.21.15.78:0/807707797 shutdown_connections 2024-09-11T01:24:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f78209ec740 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de06423f90 0x55de06421290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f78209ec740 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55de06433840 0x55de06435c30 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f77f77fe640 0 client.19662 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:24:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f78209ec740 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06415df0 0x55de06420cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f78209ec740 1 --2- 172.21.15.78:0/807707797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de064154d0 0x55de06423860 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f78209ec740 1 -- 172.21.15.78:0/807707797 >> 172.21.15.78:0/807707797 conn(0x55de06413300 msgr2=0x55de06418e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f78209ec740 1 -- 172.21.15.78:0/807707797 shutdown_connections 2024-09-11T01:24:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f78209ec740 1 -- 172.21.15.78:0/807707797 wait complete. 2024-09-11T01:24:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.834+0000 7f78209ec740 20 client.19662 trim_cache size 0 max 16384 2024-09-11T01:24:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.847+0000 7f78209ec740 1 Processor -- start 2024-09-11T01:24:14.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.847+0000 7f78209ec740 1 -- start start 2024-09-11T01:24:14.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.847+0000 7f78209ec740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de064154d0 0x55de0642fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.847+0000 7f78209ec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06415df0 0x55de0641f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.847+0000 7f78209ec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06423f90 0x55de0641fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.847+0000 7f78209ec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78181ac480 con 0x55de06423f90 2024-09-11T01:24:14.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.847+0000 7f78209ec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f78181ae110 con 0x55de06415df0 2024-09-11T01:24:14.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.847+0000 7f78209ec740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f78181acb50 con 0x55de064154d0 2024-09-11T01:24:14.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f781cbb4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06423f90 0x55de0641fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f781cbb4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06423f90 0x55de0641fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41860/0 (socket says 172.21.15.78:41860) 2024-09-11T01:24:14.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f781cbb4640 1 -- 172.21.15.78:0/1461179765 learned_addr learned my addr 172.21.15.78:0/1461179765 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f780ffff640 1 --2- 172.21.15.78:0/1461179765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06415df0 0x55de0641f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f781cbb4640 1 -- 172.21.15.78:0/1461179765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de064154d0 msgr2=0x55de0642fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f781cbb4640 1 --2- 172.21.15.78:0/1461179765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de064154d0 0x55de0642fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f781cbb4640 1 -- 172.21.15.78:0/1461179765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06415df0 msgr2=0x55de0641f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f781cbb4640 1 --2- 172.21.15.78:0/1461179765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06415df0 0x55de0641f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f781cbb4640 1 -- 172.21.15.78:0/1461179765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de06413110 con 0x55de06423f90 2024-09-11T01:24:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f781cbb4640 1 --2- 172.21.15.78:0/1461179765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06423f90 0x55de0641fa70 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f781002c010 tx=0x7f7810030610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110967f2bce9b85f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f780dffb640 1 -- 172.21.15.78:0/1461179765 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810003f80 con 0x55de06423f90 2024-09-11T01:24:14.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f780dffb640 1 -- 172.21.15.78:0/1461179765 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7810036070 con 0x55de06423f90 2024-09-11T01:24:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.848+0000 7f780dffb640 1 -- 172.21.15.78:0/1461179765 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78100318d0 con 0x55de06423f90 2024-09-11T01:24:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.849+0000 7f78209ec740 1 -- 172.21.15.78:0/1461179765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06423f90 msgr2=0x55de0641fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.849+0000 7f78209ec740 1 --2- 172.21.15.78:0/1461179765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06423f90 0x55de0641fa70 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f781002c010 tx=0x7f7810030610 comp rx=0 tx=0).stop 2024-09-11T01:24:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.849+0000 7f78209ec740 1 -- 172.21.15.78:0/1461179765 shutdown_connections 2024-09-11T01:24:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.849+0000 7f78209ec740 1 --2- 172.21.15.78:0/1461179765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06423f90 0x55de0641fa70 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.849+0000 7f78209ec740 1 --2- 172.21.15.78:0/1461179765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06415df0 0x55de0641f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.849+0000 7f78209ec740 1 --2- 172.21.15.78:0/1461179765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de064154d0 0x55de0642fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.849+0000 7f78209ec740 1 -- 172.21.15.78:0/1461179765 >> 172.21.15.78:0/1461179765 conn(0x55de06413300 msgr2=0x55de06425d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.849+0000 7f78209ec740 1 -- 172.21.15.78:0/1461179765 shutdown_connections 2024-09-11T01:24:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.849+0000 7f78209ec740 1 -- 172.21.15.78:0/1461179765 wait complete. 2024-09-11T01:24:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.849+0000 7f78209ec740 1 -- start start 2024-09-11T01:24:14.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.849+0000 7f78209ec740 1 Processor -- start 2024-09-11T01:24:14.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f78209ec740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de064154d0 0x55de0642c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f78209ec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06415df0 0x55de0643b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f78209ec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06423f90 0x55de0643b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f78209ec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78181ad9b0 con 0x55de06415df0 2024-09-11T01:24:14.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f78209ec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f78181ad2b0 con 0x55de06423f90 2024-09-11T01:24:14.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f78209ec740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f78181aa480 con 0x55de064154d0 2024-09-11T01:24:14.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f78209ec740 10 client.0 _ll_register_callbacks cb 0x7f7800005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:24:14.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f780ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06415df0 0x55de0643b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f780ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06415df0 0x55de0643b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41872/0 (socket says 172.21.15.78:41872) 2024-09-11T01:24:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f780ffff640 1 -- 172.21.15.78:0/3401734371 learned_addr learned my addr 172.21.15.78:0/3401734371 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f781cbb4640 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06423f90 0x55de0643b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f781cbb4640 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de064154d0 0x55de0642c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f780ffff640 1 -- 172.21.15.78:0/3401734371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de064154d0 msgr2=0x55de0642c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f780ffff640 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de064154d0 0x55de0642c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f780ffff640 1 -- 172.21.15.78:0/3401734371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06423f90 msgr2=0x55de0643b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f780ffff640 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06423f90 0x55de0643b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f780ffff640 1 -- 172.21.15.78:0/3401734371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de0649f3d0 con 0x55de06415df0 2024-09-11T01:24:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f781cbb4640 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de064154d0 0x55de0642c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:24:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f780ffff640 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06415df0 0x55de0643b030 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f780000ae40 tx=0x7f7800012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd3f223542ceb7b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f77f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:24:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f77f77fe640 1 -- 172.21.15.78:0/3401734371 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800005460 con 0x55de06415df0 2024-09-11T01:24:14.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f77f77fe640 1 -- 172.21.15.78:0/3401734371 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f7800005600 con 0x55de06415df0 2024-09-11T01:24:14.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.850+0000 7f78209ec740 1 -- 172.21.15.78:0/3401734371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7810036070 con 0x55de06415df0 2024-09-11T01:24:14.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.851+0000 7f77f77fe640 1 -- 172.21.15.78:0/3401734371 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f78000047e0 con 0x55de06415df0 2024-09-11T01:24:14.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.851+0000 7f78209ec740 10 client.19668 fetch_fsmap learned FSMap version 80 2024-09-11T01:24:14.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.851+0000 7f78209ec740 1 -- 172.21.15.78:0/3401734371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55de064a06e0 con 0x55de06415df0 2024-09-11T01:24:14.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.851+0000 7f77f77fe640 1 -- 172.21.15.78:0/3401734371 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7810036070 con 0x55de06415df0 2024-09-11T01:24:14.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.851+0000 7f77f77fe640 1 -- 172.21.15.78:0/3401734371 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 80) ==== 8406+0+0 (secure 0 0 0) 0x7f780002cb20 con 0x55de06415df0 2024-09-11T01:24:14.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.851+0000 7f78209ec740 10 client.19668 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T01:24:14.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.851+0000 7f78209ec740 10 client.19668 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-11T01:24:14.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.851+0000 7f78209ec740 20 client.19668 populate_metadata read hostname 'smithi078' 2024-09-11T01:24:14.884 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:24:14.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.851+0000 7f78209ec740 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55de06433840 0x55de064a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.851+0000 7f78209ec740 4 client.19668 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:24:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.851+0000 7f78209ec740 1 -- 172.21.15.78:0/3401734371 --> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de063898a0 con 0x55de06433840 2024-09-11T01:24:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.851+0000 7f781cbb4640 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55de06433840 0x55de064a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f77f77fe640 10 client.19668 ms_handle_connect on v2:172.21.15.112:6836/854577265 2024-09-11T01:24:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f781cbb4640 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55de06433840 0x55de064a2f00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=515f0116a3062e11 server_cookie=2f7f46135ba6f028 in_seq=0 out_seq=0 2024-09-11T01:24:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f77f77fe640 1 -- 172.21.15.78:0/3401734371 <== mds.2 v2:172.21.15.112:6836/854577265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de063898a0 con 0x55de06433840 2024-09-11T01:24:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f77f77fe640 10 client.19668 handle_command_reply: tid=0 2024-09-11T01:24:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f78209ec740 1 client.19668 shutdown 2024-09-11T01:24:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f78209ec740 1 -- 172.21.15.78:0/3401734371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06415df0 msgr2=0x55de0643b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f78209ec740 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06415df0 0x55de0643b030 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f780000ae40 tx=0x7f7800012820 comp rx=0 tx=0).stop 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f78209ec740 1 -- 172.21.15.78:0/3401734371 shutdown_connections 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f78209ec740 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.112:6836/854577265,v1:172.21.15.112:6837/854577265] conn(0x55de06433840 0x55de064a2f00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f78209ec740 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de06423f90 0x55de0643b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f78209ec740 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de06415df0 0x55de0643b030 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f78209ec740 1 --2- 172.21.15.78:0/3401734371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de064154d0 0x55de0642c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f78209ec740 1 -- 172.21.15.78:0/3401734371 >> 172.21.15.78:0/3401734371 conn(0x55de06413300 msgr2=0x55de0641f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f77f77fe640 0 client.19668 ms_handle_reset on v2:172.21.15.112:6836/854577265 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f78209ec740 1 -- 172.21.15.78:0/3401734371 shutdown_connections 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f78209ec740 1 -- 172.21.15.78:0/3401734371 wait complete. 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.852+0000 7f78209ec740 20 client.19668 trim_cache size 0 max 16384 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.853+0000 7f781e617640 1 -- 172.21.15.78:0/1062191479 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78040b54d0 msgr2=0x7f78040b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.853+0000 7f781e617640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78040b54d0 0x7f78040b7990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f78080e34b0 tx=0x7f78080b5040 comp rx=0 tx=0).stop 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.853+0000 7f781e617640 1 -- 172.21.15.78:0/1062191479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7818101cd0 msgr2=0x7f7818198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.853+0000 7f781e617640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7818101cd0 0x7f7818198170 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f780800d950 tx=0x7f780800de10 comp rx=0 tx=0).stop 2024-09-11T01:24:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.853+0000 7f781e617640 1 -- 172.21.15.78:0/1062191479 shutdown_connections 2024-09-11T01:24:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.853+0000 7f781e617640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f78040b54d0 0x7f78040b7990 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.853+0000 7f781e617640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7818108060 0x7f781819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.853+0000 7f781e617640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78181076e0 0x7f78181986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.853+0000 7f781e617640 1 --2- 172.21.15.78:0/1062191479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7818101cd0 0x7f7818198170 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.853+0000 7f781e617640 1 -- 172.21.15.78:0/1062191479 >> 172.21.15.78:0/1062191479 conn(0x7f78180fd700 msgr2=0x7f78180ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.853+0000 7f781e617640 1 -- 172.21.15.78:0/1062191479 shutdown_connections 2024-09-11T01:24:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:14.853+0000 7f781e617640 1 -- 172.21.15.78:0/1062191479 wait complete. 2024-09-11T01:24:14.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:14 smithi110 ceph-mon[34712]: osdmap e115: 12 total, 12 up, 12 in 2024-09-11T01:24:14.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:14 smithi112 ceph-mon[35384]: osdmap e115: 12 total, 12 up, 12 in 2024-09-11T01:24:15.690 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T01:24:15.937 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:15.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:15 smithi110 ceph-mon[34712]: pgmap v1122: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-09-11T01:24:15.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:15 smithi112 ceph-mon[35384]: pgmap v1122: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-09-11T01:24:15.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:15 smithi078 ceph-mon[31315]: pgmap v1122: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-09-11T01:24:16.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.461+0000 7f54f7bc3640 1 -- 172.21.15.78:0/3173840467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f010d0f0 msgr2=0x7f54f010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:16.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.461+0000 7f54f7bc3640 1 --2- 172.21.15.78:0/3173840467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f010d0f0 0x7f54f010f550 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009ca0 tx=0x7f54e802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.461+0000 7f54f7bc3640 1 -- 172.21.15.78:0/3173840467 shutdown_connections 2024-09-11T01:24:16.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.461+0000 7f54f7bc3640 1 --2- 172.21.15.78:0/3173840467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f010d0f0 0x7f54f010f550 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.461+0000 7f54f7bc3640 1 --2- 172.21.15.78:0/3173840467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54f01045b0 0x7f54f010caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.461+0000 7f54f7bc3640 1 --2- 172.21.15.78:0/3173840467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54f0103c90 0x7f54f0104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.461+0000 7f54f7bc3640 1 -- 172.21.15.78:0/3173840467 >> 172.21.15.78:0/3173840467 conn(0x7f54f00fd6d0 msgr2=0x7f54f00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:16.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.461+0000 7f54f7bc3640 1 -- 172.21.15.78:0/3173840467 shutdown_connections 2024-09-11T01:24:16.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.461+0000 7f54f7bc3640 1 -- 172.21.15.78:0/3173840467 wait complete. 2024-09-11T01:24:16.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f7bc3640 1 Processor -- start 2024-09-11T01:24:16.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f7bc3640 1 -- start start 2024-09-11T01:24:16.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f7bc3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0103c90 0x7f54f01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f7bc3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54f01045b0 0x7f54f01996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f7bc3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54f010d0f0 0x7f54f019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f7bc3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0112d60 con 0x7f54f0103c90 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f7bc3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0112be0 con 0x7f54f01045b0 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f7bc3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0112ee0 con 0x7f54f010d0f0 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f595f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0103c90 0x7f54f01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f595f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0103c90 0x7f54f01991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41886/0 (socket says 172.21.15.78:41886) 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f595f640 1 -- 172.21.15.78:0/706715171 learned_addr learned my addr 172.21.15.78:0/706715171 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f6160640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54f010d0f0 0x7f54f019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f515e640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54f01045b0 0x7f54f01996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f595f640 1 -- 172.21.15.78:0/706715171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54f010d0f0 msgr2=0x7f54f019ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f595f640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54f010d0f0 0x7f54f019ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f595f640 1 -- 172.21.15.78:0/706715171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54f01045b0 msgr2=0x7f54f01996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f595f640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54f01045b0 0x7f54f01996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f595f640 1 -- 172.21.15.78:0/706715171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f019f260 con 0x7f54f0103c90 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f6160640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54f010d0f0 0x7f54f019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.462+0000 7f54f515e640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54f01045b0 0x7f54f01996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.463+0000 7f54f595f640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0103c90 0x7f54f01991b0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c890 tx=0x7f54e000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.463+0000 7f54e6ffd640 1 -- 172.21.15.78:0/706715171 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0004220 con 0x7f54f0103c90 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.463+0000 7f54f7bc3640 1 -- 172.21.15.78:0/706715171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f019f550 con 0x7f54f0103c90 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.463+0000 7f54e6ffd640 1 -- 172.21.15.78:0/706715171 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54e00043c0 con 0x7f54f0103c90 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.463+0000 7f54e6ffd640 1 -- 172.21.15.78:0/706715171 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0004a00 con 0x7f54f0103c90 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.463+0000 7f54f7bc3640 1 -- 172.21.15.78:0/706715171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f01a5de0 con 0x7f54f0103c90 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.465+0000 7f54f9f98740 1 Processor -- start 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.465+0000 7f54f9f98740 1 -- start start 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.465+0000 7f54f9f98740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c234d0 0x561c73c238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.465+0000 7f54f9f98740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 0x561c73c2f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.465+0000 7f54f9f98740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c2fd80 0x561c73c32170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.465+0000 7f54f9f98740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561c73b99ea0 con 0x561c73c23df0 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.465+0000 7f54f9f98740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561c73b892d0 con 0x561c73c234d0 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f9f98740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561c73b8d0e0 con 0x561c73c2fd80 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54e6ffd640 1 -- 172.21.15.78:0/706715171 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f54e00079b0 con 0x7f54f0103c90 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f6160640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c234d0 0x561c73c238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f595f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 0x561c73c2f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f6160640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c234d0 0x561c73c238b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.78:50072/0 (socket says 172.21.15.78:50072) 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f6160640 1 -- 172.21.15.78:0/3059591341 learned_addr learned my addr 172.21.15.78:0/3059591341 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f515e640 1 --2- 172.21.15.78:0/3059591341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c2fd80 0x561c73c32170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f595f640 1 -- 172.21.15.78:0/3059591341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c2fd80 msgr2=0x561c73c32170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f595f640 1 --2- 172.21.15.78:0/3059591341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c2fd80 0x561c73c32170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f595f640 1 -- 172.21.15.78:0/3059591341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c234d0 msgr2=0x561c73c238b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f595f640 1 --2- 172.21.15.78:0/3059591341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c234d0 0x561c73c238b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f595f640 1 -- 172.21.15.78:0/3059591341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c738e7650 con 0x561c73c23df0 2024-09-11T01:24:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f6160640 1 --2- 172.21.15.78:0/3059591341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c234d0 0x561c73c238b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54f595f640 1 --2- 172.21.15.78:0/3059591341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 0x561c73c2f840 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f54e00e3580 tx=0x7f54e00ee570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72e7b585c910c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54dbfff640 1 -- 172.21.15.78:0/3059591341 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e00f9070 con 0x561c73c23df0 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.466+0000 7f54e6ffd640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54dc0b5450 0x7f54dc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.467+0000 7f54dbfff640 1 -- 172.21.15.78:0/3059591341 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54e00f4ab0 con 0x561c73c23df0 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.467+0000 7f54dbfff640 1 -- 172.21.15.78:0/3059591341 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e00f3a60 con 0x561c73c23df0 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.467+0000 7f54f6160640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54dc0b5450 0x7f54dc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.467+0000 7f54e6ffd640 1 -- 172.21.15.78:0/706715171 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f54e00e5df0 con 0x7f54f0103c90 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.467+0000 7f54f9f98740 1 -- 172.21.15.78:0/3059591341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 msgr2=0x561c73c2f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.467+0000 7f54f6160640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54dc0b5450 0x7f54dc0b7910 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009a60 tx=0x7f54e8005c30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.467+0000 7f54f9f98740 1 --2- 172.21.15.78:0/3059591341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 0x561c73c2f840 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f54e00e3580 tx=0x7f54e00ee570 comp rx=0 tx=0).stop 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.467+0000 7f54f9f98740 1 -- 172.21.15.78:0/3059591341 shutdown_connections 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.467+0000 7f54f9f98740 1 --2- 172.21.15.78:0/3059591341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c2fd80 0x561c73c32170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.467+0000 7f54f9f98740 1 --2- 172.21.15.78:0/3059591341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 0x561c73c2f840 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.467+0000 7f54f9f98740 1 --2- 172.21.15.78:0/3059591341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c234d0 0x561c73c238b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.467+0000 7f54f9f98740 1 -- 172.21.15.78:0/3059591341 >> 172.21.15.78:0/3059591341 conn(0x561c73c21300 msgr2=0x561c73c224a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f9f98740 1 -- 172.21.15.78:0/3059591341 shutdown_connections 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f9f98740 1 -- 172.21.15.78:0/3059591341 wait complete. 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f9f98740 1 -- start start 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f9f98740 1 Processor -- start 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f9f98740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c234d0 0x561c73c2f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f9f98740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c23df0 0x561c73c2cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f9f98740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c2fd80 0x561c73c2d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f9f98740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561c739ea700 con 0x561c73c234d0 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f9f98740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561c73b24160 con 0x561c73c2fd80 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f9f98740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561c73c32a00 con 0x561c73c23df0 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f9f98740 10 client.0 _ll_register_callbacks cb 0x561c73b3d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:24:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f595f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c234d0 0x561c73c2f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f595f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c234d0 0x561c73c2f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41910/0 (socket says 172.21.15.78:41910) 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.468+0000 7f54f595f640 1 -- 172.21.15.78:0/2346397114 learned_addr learned my addr 172.21.15.78:0/2346397114 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54f6160640 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c2fd80 0x561c73c2d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54f515e640 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c23df0 0x561c73c2cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54f595f640 1 -- 172.21.15.78:0/2346397114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c23df0 msgr2=0x561c73c2cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54f595f640 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c23df0 0x561c73c2cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54f595f640 1 -- 172.21.15.78:0/2346397114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c2fd80 msgr2=0x561c73c2d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54f595f640 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c2fd80 0x561c73c2d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54f595f640 1 -- 172.21.15.78:0/2346397114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c73b51c90 con 0x561c73c234d0 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54f515e640 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c23df0 0x561c73c2cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54f595f640 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c234d0 0x561c73c2f6c0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f54e0005fd0 tx=0x7f54e00e9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad28683fbb095fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54d97fa640 1 -- 172.21.15.78:0/2346397114 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e001d040 con 0x561c73c234d0 2024-09-11T01:24:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54d97fa640 1 -- 172.21.15.78:0/2346397114 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f54e00f33a0 con 0x561c73c234d0 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54f9f98740 1 -- 172.21.15.78:0/2346397114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e00f4ab0 con 0x561c73c234d0 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54d97fa640 1 -- 172.21.15.78:0/2346397114 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f54e00a8900 con 0x561c73c234d0 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54f9f98740 10 client.19680 fetch_fsmap learned FSMap version 80 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54f9f98740 1 -- 172.21.15.78:0/2346397114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x561c73c3e3b0 con 0x561c73c234d0 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54d97fa640 1 -- 172.21.15.78:0/2346397114 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54e00f4ab0 con 0x561c73c234d0 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.469+0000 7f54d97fa640 1 -- 172.21.15.78:0/2346397114 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 80) ==== 8406+0+0 (secure 0 0 0) 0x7f54e00ac510 con 0x561c73c234d0 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.470+0000 7f54f9f98740 10 client.19680 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.470+0000 7f54f9f98740 10 client.19680 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.470+0000 7f54f9f98740 20 client.19680 populate_metadata read hostname 'smithi078' 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.470+0000 7f54f9f98740 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x561c73c3f700 0x561c73c41af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.470+0000 7f54f9f98740 4 client.19680 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.470+0000 7f54f9f98740 1 -- 172.21.15.78:0/2346397114 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c73b9bdf0 con 0x561c73c3f700 2024-09-11T01:24:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.470+0000 7f54f515e640 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x561c73c3f700 0x561c73c41af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.470+0000 7f54f515e640 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x561c73c3f700 0x561c73c41af0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48a9944ec7ea5b52 server_cookie=6b3bec962964db99 in_seq=0 out_seq=0 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.470+0000 7f54d97fa640 10 client.19680 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54d97fa640 1 -- 172.21.15.78:0/2346397114 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561c73b9bdf0 con 0x561c73c3f700 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54d97fa640 10 client.19680 handle_command_reply: tid=0 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54f9f98740 1 client.19680 shutdown 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54f9f98740 1 -- 172.21.15.78:0/2346397114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c234d0 msgr2=0x561c73c2f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54f9f98740 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c234d0 0x561c73c2f6c0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f54e0005fd0 tx=0x7f54e00e9500 comp rx=0 tx=0).stop 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54f9f98740 1 -- 172.21.15.78:0/2346397114 shutdown_connections 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54f9f98740 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c2fd80 0x561c73c2d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54f9f98740 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c23df0 0x561c73c2cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54f9f98740 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x561c73c3f700 0x561c73c41af0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54f9f98740 1 --2- 172.21.15.78:0/2346397114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c234d0 0x561c73c2f6c0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54d97fa640 0 client.19680 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:24:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54f9f98740 1 -- 172.21.15.78:0/2346397114 >> 172.21.15.78:0/2346397114 conn(0x561c73c21300 msgr2=0x561c73c26ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54f9f98740 1 -- 172.21.15.78:0/2346397114 shutdown_connections 2024-09-11T01:24:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54f9f98740 1 -- 172.21.15.78:0/2346397114 wait complete. 2024-09-11T01:24:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.471+0000 7f54f9f98740 20 client.19680 trim_cache size 0 max 16384 2024-09-11T01:24:16.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.484+0000 7f54f9f98740 1 Processor -- start 2024-09-11T01:24:16.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f9f98740 1 -- start start 2024-09-11T01:24:16.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f9f98740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c234d0 0x561c73c20c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f9f98740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c23df0 0x561c73c226f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f9f98740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c2fd80 0x561c73c31170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f9f98740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01ae5f0 con 0x561c73c2fd80 2024-09-11T01:24:16.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f9f98740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01b0280 con 0x561c73c23df0 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f9f98740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01aecc0 con 0x561c73c234d0 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f6160640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c2fd80 0x561c73c31170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f6160640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c2fd80 0x561c73c31170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41926/0 (socket says 172.21.15.78:41926) 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f6160640 1 -- 172.21.15.78:0/2168545248 learned_addr learned my addr 172.21.15.78:0/2168545248 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f515e640 1 --2- 172.21.15.78:0/2168545248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c23df0 0x561c73c226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f595f640 1 --2- 172.21.15.78:0/2168545248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c234d0 0x561c73c20c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f6160640 1 -- 172.21.15.78:0/2168545248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c234d0 msgr2=0x561c73c20c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f6160640 1 --2- 172.21.15.78:0/2168545248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c234d0 0x561c73c20c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f6160640 1 -- 172.21.15.78:0/2168545248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c23df0 msgr2=0x561c73c226f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f6160640 1 --2- 172.21.15.78:0/2168545248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c23df0 0x561c73c226f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f6160640 1 -- 172.21.15.78:0/2168545248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c73c2d500 con 0x561c73c2fd80 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f515e640 1 --2- 172.21.15.78:0/2168545248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c23df0 0x561c73c226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:24:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.485+0000 7f54f6160640 1 --2- 172.21.15.78:0/2168545248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c2fd80 0x561c73c31170 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f54e8005cc0 tx=0x7f54e8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501b097dfdad9130 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.486+0000 7f54db7fe640 1 -- 172.21.15.78:0/2168545248 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8003970 con 0x561c73c2fd80 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.486+0000 7f54db7fe640 1 -- 172.21.15.78:0/2168545248 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54e8036040 con 0x561c73c2fd80 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.486+0000 7f54db7fe640 1 -- 172.21.15.78:0/2168545248 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e80047d0 con 0x561c73c2fd80 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.486+0000 7f54f9f98740 1 -- 172.21.15.78:0/2168545248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c2fd80 msgr2=0x561c73c31170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.486+0000 7f54f9f98740 1 --2- 172.21.15.78:0/2168545248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c2fd80 0x561c73c31170 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f54e8005cc0 tx=0x7f54e8041000 comp rx=0 tx=0).stop 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.486+0000 7f54f9f98740 1 -- 172.21.15.78:0/2168545248 shutdown_connections 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.486+0000 7f54f9f98740 1 --2- 172.21.15.78:0/2168545248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c2fd80 0x561c73c31170 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.486+0000 7f54f9f98740 1 --2- 172.21.15.78:0/2168545248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c23df0 0x561c73c226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.486+0000 7f54f9f98740 1 --2- 172.21.15.78:0/2168545248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c234d0 0x561c73c20c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.486+0000 7f54f9f98740 1 -- 172.21.15.78:0/2168545248 >> 172.21.15.78:0/2168545248 conn(0x561c73c21300 msgr2=0x561c73c21fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.486+0000 7f54f9f98740 1 -- 172.21.15.78:0/2168545248 shutdown_connections 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.486+0000 7f54f9f98740 1 -- 172.21.15.78:0/2168545248 wait complete. 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f9f98740 1 -- start start 2024-09-11T01:24:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f9f98740 1 Processor -- start 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f9f98740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c234d0 0x561c73c44e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f9f98740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 0x561c73c453c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f9f98740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c2fd80 0x561c73c41be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f9f98740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01afb20 con 0x561c73c23df0 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f9f98740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01af420 con 0x561c73c2fd80 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f9f98740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01ac5f0 con 0x561c73c234d0 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f515e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 0x561c73c453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f515e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 0x561c73c453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41934/0 (socket says 172.21.15.78:41934) 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f515e640 1 -- 172.21.15.78:0/494433199 learned_addr learned my addr 172.21.15.78:0/494433199 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f9f98740 10 client.0 _ll_register_callbacks cb 0x7f54e00f7070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.487+0000 7f54f595f640 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c234d0 0x561c73c44e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54f6160640 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c2fd80 0x561c73c41be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54f515e640 1 -- 172.21.15.78:0/494433199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c234d0 msgr2=0x561c73c44e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54f515e640 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c234d0 0x561c73c44e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54f515e640 1 -- 172.21.15.78:0/494433199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c2fd80 msgr2=0x561c73c41be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54f515e640 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c2fd80 0x561c73c41be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54f515e640 1 -- 172.21.15.78:0/494433199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c73c42410 con 0x561c73c23df0 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54f515e640 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 0x561c73c453c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00bcd0 tx=0x7f54ec013c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bca89912cf3a3e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54d97fa640 1 -- 172.21.15.78:0/494433199 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec004230 con 0x561c73c23df0 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54f9f98740 1 -- 172.21.15.78:0/494433199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e8036040 con 0x561c73c23df0 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54d97fa640 1 -- 172.21.15.78:0/494433199 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f54ec0043d0 con 0x561c73c23df0 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54d97fa640 1 -- 172.21.15.78:0/494433199 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f54ec010960 con 0x561c73c23df0 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54f9f98740 10 client.19686 fetch_fsmap learned FSMap version 80 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.488+0000 7f54f9f98740 1 -- 172.21.15.78:0/494433199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x561c73cacee0 con 0x561c73c23df0 2024-09-11T01:24:16.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.489+0000 7f54d97fa640 1 -- 172.21.15.78:0/494433199 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54e8036040 con 0x561c73c23df0 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.489+0000 7f54d97fa640 1 -- 172.21.15.78:0/494433199 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 80) ==== 8406+0+0 (secure 0 0 0) 0x7f54ec021070 con 0x561c73c23df0 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.489+0000 7f54f9f98740 10 client.19686 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.489+0000 7f54f9f98740 10 client.19686 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.489+0000 7f54f9f98740 20 client.19686 populate_metadata read hostname 'smithi078' 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.489+0000 7f54f9f98740 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x561c73cad6c0 0x561c73cafae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.489+0000 7f54f9f98740 4 client.19686 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.489+0000 7f54f9f98740 1 -- 172.21.15.78:0/494433199 --> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c73b978a0 con 0x561c73cad6c0 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.489+0000 7f54f595f640 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x561c73cad6c0 0x561c73cafae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.489+0000 7f54d97fa640 10 client.19686 ms_handle_connect on v2:172.21.15.112:6838/2141614771 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.489+0000 7f54f595f640 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x561c73cad6c0 0x561c73cafae0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=976a4ed92b167e3b server_cookie=8acc552daea94191 in_seq=0 out_seq=0 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54d97fa640 1 -- 172.21.15.78:0/494433199 <== mds.1 v2:172.21.15.112:6838/2141614771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c73b978a0 con 0x561c73cad6c0 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54d97fa640 10 client.19686 handle_command_reply: tid=0 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f9f98740 1 client.19686 shutdown 2024-09-11T01:24:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f9f98740 1 -- 172.21.15.78:0/494433199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 msgr2=0x561c73c453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f9f98740 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 0x561c73c453c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00bcd0 tx=0x7f54ec013c60 comp rx=0 tx=0).stop 2024-09-11T01:24:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f9f98740 1 -- 172.21.15.78:0/494433199 shutdown_connections 2024-09-11T01:24:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f9f98740 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c73c2fd80 0x561c73c41be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f9f98740 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.112:6838/2141614771,v1:172.21.15.112:6839/2141614771] conn(0x561c73cad6c0 0x561c73cafae0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f9f98740 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c73c23df0 0x561c73c453c0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f9f98740 1 --2- 172.21.15.78:0/494433199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c73c234d0 0x561c73c44e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f9f98740 1 -- 172.21.15.78:0/494433199 >> 172.21.15.78:0/494433199 conn(0x561c73c21300 msgr2=0x561c73c22800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54d97fa640 0 client.19686 ms_handle_reset on v2:172.21.15.112:6838/2141614771 2024-09-11T01:24:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f9f98740 1 -- 172.21.15.78:0/494433199 shutdown_connections 2024-09-11T01:24:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f9f98740 1 -- 172.21.15.78:0/494433199 wait complete. 2024-09-11T01:24:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f9f98740 20 client.19686 trim_cache size 0 max 16384 2024-09-11T01:24:16.502 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:24:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f7bc3640 1 -- 172.21.15.78:0/706715171 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54dc0b5450 msgr2=0x7f54dc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.490+0000 7f54f7bc3640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54dc0b5450 0x7f54dc0b7910 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009a60 tx=0x7f54e8005c30 comp rx=0 tx=0).stop 2024-09-11T01:24:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.491+0000 7f54f7bc3640 1 -- 172.21.15.78:0/706715171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0103c90 msgr2=0x7f54f01991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.491+0000 7f54f7bc3640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0103c90 0x7f54f01991b0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c890 tx=0x7f54e000cd50 comp rx=0 tx=0).stop 2024-09-11T01:24:16.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.491+0000 7f54f7bc3640 1 -- 172.21.15.78:0/706715171 shutdown_connections 2024-09-11T01:24:16.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.491+0000 7f54f7bc3640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f54dc0b5450 0x7f54dc0b7910 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.491+0000 7f54f7bc3640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54f010d0f0 0x7f54f019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.491+0000 7f54f7bc3640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54f01045b0 0x7f54f01996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.491+0000 7f54f7bc3640 1 --2- 172.21.15.78:0/706715171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0103c90 0x7f54f01991b0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:16.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.491+0000 7f54f7bc3640 1 -- 172.21.15.78:0/706715171 >> 172.21.15.78:0/706715171 conn(0x7f54f00fd6d0 msgr2=0x7f54f00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:16.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.491+0000 7f54f7bc3640 1 -- 172.21.15.78:0/706715171 shutdown_connections 2024-09-11T01:24:16.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:16.491+0000 7f54f7bc3640 1 -- 172.21.15.78:0/706715171 wait complete. 2024-09-11T01:24:17.377 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T01:24:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:17.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:17 smithi078 ceph-mon[31315]: pgmap v1123: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 42 KiB/s wr, 24 op/s 2024-09-11T01:24:17.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:17 smithi110 ceph-mon[34712]: pgmap v1123: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 42 KiB/s wr, 24 op/s 2024-09-11T01:24:17.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:17 smithi112 ceph-mon[35384]: pgmap v1123: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 42 KiB/s wr, 24 op/s 2024-09-11T01:24:18.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.298+0000 7f5246155640 1 -- 172.21.15.78:0/3181848578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524010d120 msgr2=0x7f524010f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.298+0000 7f5246155640 1 --2- 172.21.15.78:0/3181848578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524010d120 0x7f524010f590 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5238009ca0 tx=0x7f523802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.298+0000 7f5246155640 1 -- 172.21.15.78:0/3181848578 shutdown_connections 2024-09-11T01:24:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.298+0000 7f5246155640 1 --2- 172.21.15.78:0/3181848578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524010d120 0x7f524010f590 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.298+0000 7f5246155640 1 --2- 172.21.15.78:0/3181848578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5240104570 0x7f524010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.298+0000 7f5246155640 1 --2- 172.21.15.78:0/3181848578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240103b70 0x7f5240103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.298+0000 7f5246155640 1 -- 172.21.15.78:0/3181848578 >> 172.21.15.78:0/3181848578 conn(0x7f52400fd700 msgr2=0x7f52400ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.298+0000 7f5246155640 1 -- 172.21.15.78:0/3181848578 shutdown_connections 2024-09-11T01:24:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.298+0000 7f5246155640 1 -- 172.21.15.78:0/3181848578 wait complete. 2024-09-11T01:24:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.299+0000 7f5246155640 1 Processor -- start 2024-09-11T01:24:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.299+0000 7f5246155640 1 -- start start 2024-09-11T01:24:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.299+0000 7f5246155640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240103b70 0x7f5240199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.299+0000 7f5246155640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5240104570 0x7f52401997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.299+0000 7f5246155640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524010d120 0x7f524019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.299+0000 7f5246155640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5240112de0 con 0x7f524010d120 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.299+0000 7f5246155640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5240112c60 con 0x7f5240104570 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.299+0000 7f5246155640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5240112f60 con 0x7f5240103b70 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.299+0000 7f523ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524010d120 0x7f524019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.299+0000 7f523ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524010d120 0x7f524019eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41944/0 (socket says 172.21.15.78:41944) 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.299+0000 7f523ffff640 1 -- 172.21.15.78:0/2957136107 learned_addr learned my addr 172.21.15.78:0/2957136107 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f523effd640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5240104570 0x7f52401997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f523f7fe640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240103b70 0x7f5240199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f523ffff640 1 -- 172.21.15.78:0/2957136107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240103b70 msgr2=0x7f5240199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f523ffff640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240103b70 0x7f5240199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f523ffff640 1 -- 172.21.15.78:0/2957136107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5240104570 msgr2=0x7f52401997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f523ffff640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5240104570 0x7f52401997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f523ffff640 1 -- 172.21.15.78:0/2957136107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524019f340 con 0x7f524010d120 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f523f7fe640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240103b70 0x7f5240199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f523ffff640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524010d120 0x7f524019eb70 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f5238005cc0 tx=0x7f523802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f523cff9640 1 -- 172.21.15.78:0/2957136107 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238003bf0 con 0x7f524010d120 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f523cff9640 1 -- 172.21.15.78:0/2957136107 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5238003d90 con 0x7f524010d120 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f5246155640 1 -- 172.21.15.78:0/2957136107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524019f5d0 con 0x7f524010d120 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f523cff9640 1 -- 172.21.15.78:0/2957136107 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523802fbc0 con 0x7f524010d120 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.300+0000 7f5246155640 1 -- 172.21.15.78:0/2957136107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52401a5d70 con 0x7f524010d120 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.302+0000 7f523cff9640 1 -- 172.21.15.78:0/2957136107 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5238004610 con 0x7f524010d120 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f524852a740 1 Processor -- start 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f524852a740 1 -- start start 2024-09-11T01:24:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f524852a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f24d0 0x55dc030f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f524852a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030f2df0 0x55dc030fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f524852a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 0x55dc03101170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f524852a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dc03068ea0 con 0x55dc030fed80 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f524852a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dc030582d0 con 0x55dc030f24d0 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f524852a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc0305c0e0 con 0x55dc030f2df0 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 0x55dc03101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523cff9640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f52280b5450 0x7f52280b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 0x55dc03101170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41946/0 (socket says 172.21.15.78:41946) 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523effd640 1 -- 172.21.15.78:0/28921893 learned_addr learned my addr 172.21.15.78:0/28921893 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523effd640 1 --2- 172.21.15.78:0/28921893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f24d0 0x55dc030f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523f7fe640 1 --2- 172.21.15.78:0/28921893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030f2df0 0x55dc030fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523cff9640 1 -- 172.21.15.78:0/2957136107 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f5238107520 con 0x7f524010d120 2024-09-11T01:24:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523effd640 1 -- 172.21.15.78:0/28921893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030f2df0 msgr2=0x55dc030fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523effd640 1 --2- 172.21.15.78:0/28921893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030f2df0 0x55dc030fe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523effd640 1 -- 172.21.15.78:0/28921893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f24d0 msgr2=0x55dc030f28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523effd640 1 --2- 172.21.15.78:0/28921893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f24d0 0x55dc030f28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523effd640 1 -- 172.21.15.78:0/28921893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc02db6650 con 0x55dc030fed80 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.303+0000 7f523ffff640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f52280b5450 0x7f52280b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.304+0000 7f523effd640 1 --2- 172.21.15.78:0/28921893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 0x55dc03101170 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f52340127c0 tx=0x7f523400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dec690e201a30a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.304+0000 7f5225ffb640 1 -- 172.21.15.78:0/28921893 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234012e00 con 0x55dc030fed80 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.304+0000 7f5225ffb640 1 -- 172.21.15.78:0/28921893 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5234010c20 con 0x55dc030fed80 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.304+0000 7f5225ffb640 1 -- 172.21.15.78:0/28921893 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523401ea90 con 0x55dc030fed80 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.304+0000 7f524852a740 1 -- 172.21.15.78:0/28921893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 msgr2=0x55dc03101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.304+0000 7f523ffff640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f52280b5450 0x7f52280b7910 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f52380cd910 tx=0x7f523809af10 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.304+0000 7f524852a740 1 --2- 172.21.15.78:0/28921893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 0x55dc03101170 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f52340127c0 tx=0x7f523400b7c0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 -- 172.21.15.78:0/28921893 shutdown_connections 2024-09-11T01:24:18.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 --2- 172.21.15.78:0/28921893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 0x55dc03101170 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 --2- 172.21.15.78:0/28921893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030f2df0 0x55dc030fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 --2- 172.21.15.78:0/28921893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f24d0 0x55dc030f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 -- 172.21.15.78:0/28921893 >> 172.21.15.78:0/28921893 conn(0x55dc030f0300 msgr2=0x55dc030f14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 -- 172.21.15.78:0/28921893 shutdown_connections 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 -- 172.21.15.78:0/28921893 wait complete. 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 -- start start 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 Processor -- start 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 0x55dc030fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f2df0 0x55dc030fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030fed80 0x55dc030fc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dc02eb9700 con 0x55dc030f24d0 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dc02ff3160 con 0x55dc030f2df0 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc03101a00 con 0x55dc030fed80 2024-09-11T01:24:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.305+0000 7f524852a740 10 client.0 _ll_register_callbacks cb 0x55dc0300c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:24:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f523effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 0x55dc030fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f523effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 0x55dc030fe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41952/0 (socket says 172.21.15.78:41952) 2024-09-11T01:24:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f523effd640 1 -- 172.21.15.78:0/979007924 learned_addr learned my addr 172.21.15.78:0/979007924 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f523effd640 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030fed80 0x55dc030fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f523f7fe640 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f2df0 0x55dc030fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f523effd640 1 -- 172.21.15.78:0/979007924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030fed80 msgr2=0x55dc030fc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f523effd640 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030fed80 0x55dc030fc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f523effd640 1 -- 172.21.15.78:0/979007924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f2df0 msgr2=0x55dc030fbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f523effd640 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f2df0 0x55dc030fbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f523effd640 1 -- 172.21.15.78:0/979007924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc03020c90 con 0x55dc030f24d0 2024-09-11T01:24:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f523effd640 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 0x55dc030fe690 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f523401e490 tx=0x7f523401e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a04ee600b6c3ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f521effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f521effd640 1 -- 172.21.15.78:0/979007924 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234024040 con 0x55dc030f24d0 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f521effd640 1 -- 172.21.15.78:0/979007924 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f5234010830 con 0x55dc030f24d0 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.306+0000 7f524852a740 1 -- 172.21.15.78:0/979007924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5234010c20 con 0x55dc030f24d0 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.307+0000 7f521effd640 1 -- 172.21.15.78:0/979007924 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f5234010a30 con 0x55dc030f24d0 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.307+0000 7f524852a740 10 client.19698 fetch_fsmap learned FSMap version 80 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.307+0000 7f524852a740 1 -- 172.21.15.78:0/979007924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55dc0310d3c0 con 0x55dc030f24d0 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.307+0000 7f521effd640 1 -- 172.21.15.78:0/979007924 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5234010c20 con 0x55dc030f24d0 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.307+0000 7f521effd640 1 -- 172.21.15.78:0/979007924 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 80) ==== 8406+0+0 (secure 0 0 0) 0x7f523402a070 con 0x55dc030f24d0 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.307+0000 7f524852a740 10 client.19698 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.307+0000 7f524852a740 10 client.19698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.307+0000 7f524852a740 20 client.19698 populate_metadata read hostname 'smithi078' 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.307+0000 7f524852a740 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dc0310e760 0x55dc03110b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.307+0000 7f524852a740 4 client.19698 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T01:24:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.307+0000 7f524852a740 1 -- 172.21.15.78:0/979007924 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc0306adf0 con 0x55dc0310e760 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.307+0000 7f523f7fe640 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dc0310e760 0x55dc03110b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.308+0000 7f521effd640 10 client.19698 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.308+0000 7f523f7fe640 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dc0310e760 0x55dc03110b50 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8813082a92b072e8 server_cookie=8276e6218e23fb8d in_seq=0 out_seq=0 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.308+0000 7f521effd640 1 -- 172.21.15.78:0/979007924 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc0306adf0 con 0x55dc0310e760 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.308+0000 7f521effd640 10 client.19698 handle_command_reply: tid=0 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.308+0000 7f524852a740 1 client.19698 shutdown 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.309+0000 7f524852a740 1 -- 172.21.15.78:0/979007924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 msgr2=0x55dc030fe690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.309+0000 7f524852a740 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 0x55dc030fe690 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f523401e490 tx=0x7f523401e670 comp rx=0 tx=0).stop 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.309+0000 7f524852a740 1 -- 172.21.15.78:0/979007924 shutdown_connections 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.309+0000 7f524852a740 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030fed80 0x55dc030fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.309+0000 7f524852a740 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f2df0 0x55dc030fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.309+0000 7f524852a740 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dc0310e760 0x55dc03110b50 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.309+0000 7f524852a740 1 --2- 172.21.15.78:0/979007924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 0x55dc030fe690 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.309+0000 7f521effd640 0 client.19698 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.309+0000 7f524852a740 1 -- 172.21.15.78:0/979007924 >> 172.21.15.78:0/979007924 conn(0x55dc030f0300 msgr2=0x55dc030f5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.309+0000 7f524852a740 1 -- 172.21.15.78:0/979007924 shutdown_connections 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.309+0000 7f524852a740 1 -- 172.21.15.78:0/979007924 wait complete. 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.309+0000 7f524852a740 20 client.19698 trim_cache size 0 max 16384 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f524852a740 1 Processor -- start 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f524852a740 1 -- start start 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f524852a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 0x55dc030f6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f524852a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f2df0 0x55dc030eb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f524852a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030fed80 0x55dc030fd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f524852a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52401ae650 con 0x55dc030f24d0 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f524852a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f52401b02e0 con 0x55dc030f2df0 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f524852a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52401aed20 con 0x55dc030fed80 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f523effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 0x55dc030f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f523effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 0x55dc030f6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41964/0 (socket says 172.21.15.78:41964) 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f523effd640 1 -- 172.21.15.78:0/669754345 learned_addr learned my addr 172.21.15.78:0/669754345 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f523effd640 1 -- 172.21.15.78:0/669754345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030fed80 msgr2=0x55dc030fd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f523effd640 1 --2- 172.21.15.78:0/669754345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030fed80 0x55dc030fd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f523f7fe640 1 --2- 172.21.15.78:0/669754345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f2df0 0x55dc030eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f523effd640 1 -- 172.21.15.78:0/669754345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f2df0 msgr2=0x55dc030eb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f523effd640 1 --2- 172.21.15.78:0/669754345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f2df0 0x55dc030eb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f523effd640 1 -- 172.21.15.78:0/669754345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc030fb890 con 0x55dc030f24d0 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f523f7fe640 1 --2- 172.21.15.78:0/669754345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f2df0 0x55dc030eb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f523effd640 1 --2- 172.21.15.78:0/669754345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 0x55dc030f6380 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f523400b6f0 tx=0x7f523401e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f122741a5562e8a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.322+0000 7f52257fa640 1 -- 172.21.15.78:0/669754345 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234039070 con 0x55dc030f24d0 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.323+0000 7f52257fa640 1 -- 172.21.15.78:0/669754345 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5234016070 con 0x55dc030f24d0 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.323+0000 7f52257fa640 1 -- 172.21.15.78:0/669754345 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523400bab0 con 0x55dc030f24d0 2024-09-11T01:24:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.323+0000 7f524852a740 1 -- 172.21.15.78:0/669754345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 msgr2=0x55dc030f6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.323+0000 7f524852a740 1 --2- 172.21.15.78:0/669754345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 0x55dc030f6380 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f523400b6f0 tx=0x7f523401e3a0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.324+0000 7f524852a740 1 -- 172.21.15.78:0/669754345 shutdown_connections 2024-09-11T01:24:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.324+0000 7f524852a740 1 --2- 172.21.15.78:0/669754345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030fed80 0x55dc030fd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.324+0000 7f524852a740 1 --2- 172.21.15.78:0/669754345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f2df0 0x55dc030eb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.324+0000 7f524852a740 1 --2- 172.21.15.78:0/669754345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030f24d0 0x55dc030f6380 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.324+0000 7f524852a740 1 -- 172.21.15.78:0/669754345 >> 172.21.15.78:0/669754345 conn(0x55dc030f0300 msgr2=0x55dc030fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.324+0000 7f524852a740 1 -- 172.21.15.78:0/669754345 shutdown_connections 2024-09-11T01:24:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.324+0000 7f524852a740 1 -- 172.21.15.78:0/669754345 wait complete. 2024-09-11T01:24:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.324+0000 7f524852a740 1 -- start start 2024-09-11T01:24:18.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.324+0000 7f524852a740 1 Processor -- start 2024-09-11T01:24:18.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f524852a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f24d0 0x55dc031161e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f524852a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030f2df0 0x55dc03116720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f524852a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 0x55dc03116c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f524852a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52401afb80 con 0x55dc030fed80 2024-09-11T01:24:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f524852a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f52401af480 con 0x55dc030f24d0 2024-09-11T01:24:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f524852a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52401ac650 con 0x55dc030f2df0 2024-09-11T01:24:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f524852a740 10 client.0 _ll_register_callbacks cb 0x7f523400be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T01:24:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f523effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 0x55dc03116c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f523effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 0x55dc03116c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41976/0 (socket says 172.21.15.78:41976) 2024-09-11T01:24:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f523effd640 1 -- 172.21.15.78:0/2435327630 learned_addr learned my addr 172.21.15.78:0/2435327630 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f523f7fe640 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030f2df0 0x55dc03116720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f523effd640 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f24d0 0x55dc031161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f523effd640 1 -- 172.21.15.78:0/2435327630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030f2df0 msgr2=0x55dc03116720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f523effd640 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030f2df0 0x55dc03116720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f523effd640 1 -- 172.21.15.78:0/2435327630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f24d0 msgr2=0x55dc031161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f523effd640 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f24d0 0x55dc031161e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f523effd640 1 -- 172.21.15.78:0/2435327630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc03103210 con 0x55dc030fed80 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f523effd640 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 0x55dc03116c60 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f5234010d80 tx=0x7f5234005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83019fa0b32997d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.325+0000 7f521effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f521effd640 1 -- 172.21.15.78:0/2435327630 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234039040 con 0x55dc030fed80 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f521effd640 1 -- 172.21.15.78:0/2435327630 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f523402c440 con 0x55dc030fed80 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f524852a740 1 -- 172.21.15.78:0/2435327630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5234016070 con 0x55dc030fed80 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f521effd640 1 -- 172.21.15.78:0/2435327630 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f5234013ed0 con 0x55dc030fed80 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f524852a740 10 client.19704 fetch_fsmap learned FSMap version 80 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f524852a740 1 -- 172.21.15.78:0/2435327630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55dc0317b7a0 con 0x55dc030fed80 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f521effd640 1 -- 172.21.15.78:0/2435327630 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5234016070 con 0x55dc030fed80 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f521effd640 1 -- 172.21.15.78:0/2435327630 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 80) ==== 8406+0+0 (secure 0 0 0) 0x7f523403f040 con 0x55dc030fed80 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f524852a740 10 client.19704 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T01:24:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f524852a740 10 client.19704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f524852a740 20 client.19704 populate_metadata read hostname 'smithi078' 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f524852a740 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dc0317beb0 0x55dc0317e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f524852a740 4 client.19704 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f524852a740 1 -- 172.21.15.78:0/2435327630 --> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc030668a0 con 0x55dc0317beb0 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.326+0000 7f523f7fe640 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dc0317beb0 0x55dc0317e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f521effd640 10 client.19704 ms_handle_connect on v2:172.21.15.110:6838/3180315245 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f523f7fe640 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dc0317beb0 0x55dc0317e2d0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab50332d8f662ca server_cookie=7154231ed07d39 in_seq=0 out_seq=0 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f521effd640 1 -- 172.21.15.78:0/2435327630 <== mds.0 v2:172.21.15.110:6838/3180315245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc030668a0 con 0x55dc0317beb0 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f521effd640 10 client.19704 handle_command_reply: tid=0 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f524852a740 1 client.19704 shutdown 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f524852a740 1 -- 172.21.15.78:0/2435327630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 msgr2=0x55dc03116c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f524852a740 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 0x55dc03116c60 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f5234010d80 tx=0x7f5234005760 comp rx=0 tx=0).stop 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f524852a740 1 -- 172.21.15.78:0/2435327630 shutdown_connections 2024-09-11T01:24:18.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f524852a740 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.110:6838/3180315245,v1:172.21.15.110:6839/3180315245] conn(0x55dc0317beb0 0x55dc0317e2d0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f524852a740 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc030fed80 0x55dc03116c60 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f524852a740 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc030f2df0 0x55dc03116720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f521effd640 0 client.19704 ms_handle_reset on v2:172.21.15.110:6838/3180315245 2024-09-11T01:24:18.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f524852a740 1 --2- 172.21.15.78:0/2435327630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc030f24d0 0x55dc031161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f524852a740 1 -- 172.21.15.78:0/2435327630 >> 172.21.15.78:0/2435327630 conn(0x55dc030f0300 msgr2=0x55dc030fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:18.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f524852a740 1 -- 172.21.15.78:0/2435327630 shutdown_connections 2024-09-11T01:24:18.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f524852a740 1 -- 172.21.15.78:0/2435327630 wait complete. 2024-09-11T01:24:18.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.327+0000 7f524852a740 20 client.19704 trim_cache size 0 max 16384 2024-09-11T01:24:18.344 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T01:24:18.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.328+0000 7f5246155640 1 -- 172.21.15.78:0/2957136107 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f52280b5450 msgr2=0x7f52280b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:18.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.328+0000 7f5246155640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f52280b5450 0x7f52280b7910 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f52380cd910 tx=0x7f523809af10 comp rx=0 tx=0).stop 2024-09-11T01:24:18.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.328+0000 7f5246155640 1 -- 172.21.15.78:0/2957136107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524010d120 msgr2=0x7f524019eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:18.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.328+0000 7f5246155640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524010d120 0x7f524019eb70 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f5238005cc0 tx=0x7f523802f380 comp rx=0 tx=0).stop 2024-09-11T01:24:18.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.328+0000 7f5246155640 1 -- 172.21.15.78:0/2957136107 shutdown_connections 2024-09-11T01:24:18.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.328+0000 7f5246155640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f52280b5450 0x7f52280b7910 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.328+0000 7f5246155640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524010d120 0x7f524019eb70 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.328+0000 7f5246155640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5240104570 0x7f52401997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.328+0000 7f5246155640 1 --2- 172.21.15.78:0/2957136107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240103b70 0x7f5240199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:18.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.328+0000 7f5246155640 1 -- 172.21.15.78:0/2957136107 >> 172.21.15.78:0/2957136107 conn(0x7f52400fd700 msgr2=0x7f52400fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:18.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.328+0000 7f5246155640 1 -- 172.21.15.78:0/2957136107 shutdown_connections 2024-09-11T01:24:18.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:18.328+0000 7f5246155640 1 -- 172.21.15.78:0/2957136107 wait complete. 2024-09-11T01:24:19.331 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-11T01:24:19.332 INFO:tasks.fwd_scrub:done joining 2024-09-11T01:24:19.333 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-11T01:24:19.342 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:24:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.443+0000 7fdfe4d52640 1 -- 172.21.15.78:0/66004620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfe0107f80 msgr2=0x7fdfe010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.443+0000 7fdfe4d52640 1 --2- 172.21.15.78:0/66004620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfe0107f80 0x7fdfe010e780 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009ca0 tx=0x7fdfd802a8c0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.443+0000 7fdfe4d52640 1 -- 172.21.15.78:0/66004620 shutdown_connections 2024-09-11T01:24:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.443+0000 7fdfe4d52640 1 --2- 172.21.15.78:0/66004620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfe0107f80 0x7fdfe010e780 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.443+0000 7fdfe4d52640 1 --2- 172.21.15.78:0/66004620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfe0107600 0x7fdfe0107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.443+0000 7fdfe4d52640 1 --2- 172.21.15.78:0/66004620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfe0101c00 0x7fdfe0101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.443+0000 7fdfe4d52640 1 -- 172.21.15.78:0/66004620 >> 172.21.15.78:0/66004620 conn(0x7fdfe00fd640 msgr2=0x7fdfe00ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.443+0000 7fdfe4d52640 1 -- 172.21.15.78:0/66004620 shutdown_connections 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.443+0000 7fdfe4d52640 1 -- 172.21.15.78:0/66004620 wait complete. 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfe4d52640 1 Processor -- start 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfe4d52640 1 -- start start 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfe4d52640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfe0101c00 0x7fdfe0199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfe4d52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfe0107600 0x7fdfe0199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfe4d52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfe0107f80 0x7fdfe019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfe4d52640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0112d60 con 0x7fdfe0107f80 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfe4d52640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0112be0 con 0x7fdfe0101c00 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfe4d52640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0112ee0 con 0x7fdfe0107600 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfded9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfe0107f80 0x7fdfe019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfded9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfe0107f80 0x7fdfe019ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41988/0 (socket says 172.21.15.78:41988) 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfded9d640 1 -- 172.21.15.78:0/871567875 learned_addr learned my addr 172.21.15.78:0/871567875 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfddd9b640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfe0107600 0x7fdfe0199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfde59c640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfe0101c00 0x7fdfe0199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfded9d640 1 -- 172.21.15.78:0/871567875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfe0107600 msgr2=0x7fdfe0199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfded9d640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfe0107600 0x7fdfe0199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfded9d640 1 -- 172.21.15.78:0/871567875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfe0101c00 msgr2=0x7fdfe0199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.444+0000 7fdfded9d640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfe0101c00 0x7fdfe0199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.445+0000 7fdfded9d640 1 -- 172.21.15.78:0/871567875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe019f1e0 con 0x7fdfe0107f80 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.445+0000 7fdfddd9b640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfe0107600 0x7fdfe0199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.445+0000 7fdfde59c640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfe0101c00 0x7fdfe0199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.445+0000 7fdfded9d640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfe0107f80 0x7fdfe019ea10 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8005cc0 tx=0x7fdfd802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.445+0000 7fdfd77fe640 1 -- 172.21.15.78:0/871567875 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8003bf0 con 0x7fdfe0107f80 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.445+0000 7fdfd77fe640 1 -- 172.21.15.78:0/871567875 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd8003d90 con 0x7fdfe0107f80 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.445+0000 7fdfd77fe640 1 -- 172.21.15.78:0/871567875 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd802fbc0 con 0x7fdfe0107f80 2024-09-11T01:24:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.445+0000 7fdfe4d52640 1 -- 172.21.15.78:0/871567875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe019f470 con 0x7fdfe0107f80 2024-09-11T01:24:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.445+0000 7fdfe4d52640 1 -- 172.21.15.78:0/871567875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe01a5b90 con 0x7fdfe0107f80 2024-09-11T01:24:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.446+0000 7fdfe4d52640 1 -- 172.21.15.78:0/871567875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe0101fe0 con 0x7fdfe0107f80 2024-09-11T01:24:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.447+0000 7fdfd77fe640 1 -- 172.21.15.78:0/871567875 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fdfd8004610 con 0x7fdfe0107f80 2024-09-11T01:24:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.448+0000 7fdfd77fe640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdfcc0b54d0 0x7fdfcc0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.448+0000 7fdfde59c640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdfcc0b54d0 0x7fdfcc0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.448+0000 7fdfd77fe640 1 -- 172.21.15.78:0/871567875 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7fdfd8107670 con 0x7fdfe0107f80 2024-09-11T01:24:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.448+0000 7fdfde59c640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdfcc0b54d0 0x7fdfcc0b7990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009b40 tx=0x7fdfd0005e50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:19.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.451+0000 7fdfd77fe640 1 -- 172.21.15.78:0/871567875 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdfd80ce230 con 0x7fdfe0107f80 2024-09-11T01:24:19.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.627+0000 7fdfe4d52640 1 -- 172.21.15.78:0/871567875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfe01a5f00 con 0x7fdfe0107f80 2024-09-11T01:24:19.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.629+0000 7fdfd77fe640 1 -- 172.21.15.78:0/871567875 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+23069 (secure 0 0 0) 0x7fdfd80cdb30 con 0x7fdfe0107f80 2024-09-11T01:24:19.633 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:24:19.633 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":115,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","created":"2024-09-11T00:48:27.652512+0000","modified":"2024-09-11T01:24:13.623597+0000","last_up_change":"2024-09-11T00:55:59.071507+0000","last_in_change":"2024-09-11T00:55:35.815150+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:52:10.229887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:56:49.939136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:56:51.703446+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beefc6d9-f74c-4200-96f2-ff697ad0d7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6803","nonce":1424639053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6805","nonce":1424639053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6809","nonce":1424639053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6807","nonce":1424639053}]},"public_addr":"172.21.15.78:6803/1424639053","cluster_addr":"172.21.15.78:6805/1424639053","heartbeat_back_addr":"172.21.15.78:6809/1424639053","heartbeat_front_addr":"172.21.15.78:6807/1424639053","state":["exists","up"]},{"osd":1,"uuid":"d742e767-d673-4bb9-b624-8bc969f5ceb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6801","nonce":580939385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6803","nonce":580939385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6807","nonce":580939385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6805","nonce":580939385}]},"public_addr":"172.21.15.110:6801/580939385","cluster_addr":"172.21.15.110:6803/580939385","heartbeat_back_addr":"172.21.15.110:6807/580939385","heartbeat_front_addr":"172.21.15.110:6805/580939385","state":["exists","up"]},{"osd":2,"uuid":"b0106a5b-1465-4a50-96dd-934dbae49a4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6801","nonce":2983155763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6803","nonce":2983155763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6807","nonce":2983155763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6805","nonce":2983155763}]},"public_addr":"172.21.15.112:6801/2983155763","cluster_addr":"172.21.15.112:6803/2983155763","heartbeat_back_addr":"172.21.15.112:6807/2983155763","heartbeat_front_addr":"172.21.15.112:6805/2983155763","state":["exists","up"]},{"osd":3,"uuid":"3270362b-3d61-414a-a3d9-f2416ff8b870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6811","nonce":2906086464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6813","nonce":2906086464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6817","nonce":2906086464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6815","nonce":2906086464}]},"public_addr":"172.21.15.78:6811/2906086464","cluster_addr":"172.21.15.78:6813/2906086464","heartbeat_back_addr":"172.21.15.78:6817/2906086464","heartbeat_front_addr":"172.21.15.78:6815/2906086464","state":["exists","up"]},{"osd":4,"uuid":"e47746ec-9443-4004-abfa-17c1ed73b982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6809","nonce":477287435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6811","nonce":477287435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6815","nonce":477287435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6813","nonce":477287435}]},"public_addr":"172.21.15.110:6809/477287435","cluster_addr":"172.21.15.110:6811/477287435","heartbeat_back_addr":"172.21.15.110:6815/477287435","heartbeat_front_addr":"172.21.15.110:6813/477287435","state":["exists","up"]},{"osd":5,"uuid":"f8cedb01-310d-442d-a588-33197d32e208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6809","nonce":3557095096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6811","nonce":3557095096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6815","nonce":3557095096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6813","nonce":3557095096}]},"public_addr":"172.21.15.112:6809/3557095096","cluster_addr":"172.21.15.112:6811/3557095096","heartbeat_back_addr":"172.21.15.112:6815/3557095096","heartbeat_front_addr":"172.21.15.112:6813/3557095096","state":["exists","up"]},{"osd":6,"uuid":"d508a037-8f56-436b-b7bf-9a1c69f6d9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6819","nonce":1267910775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6821","nonce":1267910775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6825","nonce":1267910775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6823","nonce":1267910775}]},"public_addr":"172.21.15.78:6819/1267910775","cluster_addr":"172.21.15.78:6821/1267910775","heartbeat_back_addr":"172.21.15.78:6825/1267910775","heartbeat_front_addr":"172.21.15.78:6823/1267910775","state":["exists","up"]},{"osd":7,"uuid":"e8842047-09a6-48b8-8d0a-2d54e7f89063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6817","nonce":1556494594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6819","nonce":1556494594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6823","nonce":1556494594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6821","nonce":1556494594}]},"public_addr":"172.21.15.110:6817/1556494594","cluster_addr":"172.21.15.110:6819/1556494594","heartbeat_back_addr":"172.21.15.110:6823/1556494594","heartbeat_front_addr":"172.21.15.110:6821/1556494594","state":["exists","up"]},{"osd":8,"uuid":"a3367da8-375a-4a46-a952-867d4b72ae2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6817","nonce":2481321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6819","nonce":2481321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6823","nonce":2481321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6821","nonce":2481321}]},"public_addr":"172.21.15.112:6817/2481321","cluster_addr":"172.21.15.112:6819/2481321","heartbeat_back_addr":"172.21.15.112:6823/2481321","heartbeat_front_addr":"172.21.15.112:6821/2481321","state":["exists","up"]},{"osd":9,"uuid":"d1825764-cefc-495b-99da-8ba429df0937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6827","nonce":197048350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6829","nonce":197048350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6833","nonce":197048350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6831","nonce":197048350}]},"public_addr":"172.21.15.78:6827/197048350","cluster_addr":"172.21.15.78:6829/197048350","heartbeat_back_addr":"172.21.15.78:6833/197048350","heartbeat_front_addr":"172.21.15.78:6831/197048350","state":["exists","up"]},{"osd":10,"uuid":"e1cfff7a-8916-4845-a482-d79fa275da30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6825","nonce":2101123168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6827","nonce":2101123168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6831","nonce":2101123168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6829","nonce":2101123168}]},"public_addr":"172.21.15.110:6825/2101123168","cluster_addr":"172.21.15.110:6827/2101123168","heartbeat_back_addr":"172.21.15.110:6831/2101123168","heartbeat_front_addr":"172.21.15.110:6829/2101123168","state":["exists","up"]},{"osd":11,"uuid":"fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6825","nonce":1526310654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6827","nonce":1526310654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6831","nonce":1526310654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6829","nonce":1526310654}]},"public_addr":"172.21.15.112:6825/1526310654","cluster_addr":"172.21.15.112:6827/1526310654","heartbeat_back_addr":"172.21.15.112:6831/1526310654","heartbeat_front_addr":"172.21.15.112:6829/1526310654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:17.334199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:41.400872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:08.205144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:33.550527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:58.394116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:24.699607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:50.450193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:16.223363+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:40.947727+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:06.845451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:32.410035+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:57.744364+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/2390785703":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:0/1038951931":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/391908936":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2706514734":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/2357025243":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6800/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2795778473":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2747741101":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6801/1289343426":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/3558896007":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/438261739":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/1289343426":"2024-09-12T00:50:15.928079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":33,"length":1},{"begin":46,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:24:19.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.631+0000 7fdfe4d52640 1 -- 172.21.15.78:0/871567875 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdfcc0b54d0 msgr2=0x7fdfcc0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:19.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.631+0000 7fdfe4d52640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdfcc0b54d0 0x7fdfcc0b7990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009b40 tx=0x7fdfd0005e50 comp rx=0 tx=0).stop 2024-09-11T01:24:19.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.631+0000 7fdfe4d52640 1 -- 172.21.15.78:0/871567875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfe0107f80 msgr2=0x7fdfe019ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:19.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.631+0000 7fdfe4d52640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfe0107f80 0x7fdfe019ea10 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8005cc0 tx=0x7fdfd802f380 comp rx=0 tx=0).stop 2024-09-11T01:24:19.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.631+0000 7fdfe4d52640 1 -- 172.21.15.78:0/871567875 shutdown_connections 2024-09-11T01:24:19.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.631+0000 7fdfe4d52640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fdfcc0b54d0 0x7fdfcc0b7990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.631+0000 7fdfe4d52640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfe0107f80 0x7fdfe019ea10 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.631+0000 7fdfe4d52640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfe0107600 0x7fdfe0199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.631+0000 7fdfe4d52640 1 --2- 172.21.15.78:0/871567875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfe0101c00 0x7fdfe0199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.631+0000 7fdfe4d52640 1 -- 172.21.15.78:0/871567875 >> 172.21.15.78:0/871567875 conn(0x7fdfe00fd640 msgr2=0x7fdfe00ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:19.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.631+0000 7fdfe4d52640 1 -- 172.21.15.78:0/871567875 shutdown_connections 2024-09-11T01:24:19.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.631+0000 7fdfe4d52640 1 -- 172.21.15.78:0/871567875 wait complete. 2024-09-11T01:24:19.644 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T00:52:10.229887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T00:56:49.939136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T00:56:51.703446+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T01:24:19.645 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T01:24:19.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.745+0000 7f320ab82640 1 -- 172.21.15.78:0/1987025051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32041086c0 msgr2=0x7f3204108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:19.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.745+0000 7f320ab82640 1 --2- 172.21.15.78:0/1987025051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32041086c0 0x7f3204108b00 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009900 tx=0x7f31f002a410 comp rx=0 tx=0).stop 2024-09-11T01:24:19.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.745+0000 7f320ab82640 1 -- 172.21.15.78:0/1987025051 shutdown_connections 2024-09-11T01:24:19.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.745+0000 7f320ab82640 1 --2- 172.21.15.78:0/1987025051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3204109040 0x7f320410f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.745+0000 7f320ab82640 1 --2- 172.21.15.78:0/1987025051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32041086c0 0x7f3204108b00 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.745+0000 7f320ab82640 1 --2- 172.21.15.78:0/1987025051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f320406a5f0 0x7f320406a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.745+0000 7f320ab82640 1 -- 172.21.15.78:0/1987025051 >> 172.21.15.78:0/1987025051 conn(0x7f32040776b0 msgr2=0x7f3204077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.745+0000 7f320ab82640 1 -- 172.21.15.78:0/1987025051 shutdown_connections 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.745+0000 7f320ab82640 1 -- 172.21.15.78:0/1987025051 wait complete. 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320ab82640 1 Processor -- start 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320ab82640 1 -- start start 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320ab82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f320406a5f0 0x7f3204199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320ab82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32041086c0 0x7f32041998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320ab82640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3204109040 0x7f320419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320ab82640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3204112d60 con 0x7f320406a5f0 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320ab82640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3204112be0 con 0x7f3204109040 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320ab82640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3204112ee0 con 0x7f32041086c0 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320891e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f320406a5f0 0x7f3204199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:19.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320891e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f320406a5f0 0x7f3204199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42002/0 (socket says 172.21.15.78:42002) 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320891e640 1 -- 172.21.15.78:0/1151861919 learned_addr learned my addr 172.21.15.78:0/1151861919 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f3203fff640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32041086c0 0x7f32041998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320891e640 1 -- 172.21.15.78:0/1151861919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32041086c0 msgr2=0x7f32041998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320891e640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32041086c0 0x7f32041998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320891e640 1 -- 172.21.15.78:0/1151861919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3204109040 msgr2=0x7f320419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320911f640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3204109040 0x7f320419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320891e640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3204109040 0x7f320419ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320891e640 1 -- 172.21.15.78:0/1151861919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320419f420 con 0x7f320406a5f0 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f3203fff640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32041086c0 0x7f32041998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f320891e640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f320406a5f0 0x7f3204199380 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f31f800b4a0 tx=0x7f31f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.746+0000 7f3201ffb640 1 -- 172.21.15.78:0/1151861919 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f8014070 con 0x7f320406a5f0 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.747+0000 7f3201ffb640 1 -- 172.21.15.78:0/1151861919 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31f8010ac0 con 0x7f320406a5f0 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.747+0000 7f320ab82640 1 -- 172.21.15.78:0/1151861919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320419f710 con 0x7f320406a5f0 2024-09-11T01:24:19.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.747+0000 7f3201ffb640 1 -- 172.21.15.78:0/1151861919 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f8017c30 con 0x7f320406a5f0 2024-09-11T01:24:19.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.747+0000 7f320ab82640 1 -- 172.21.15.78:0/1151861919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32041a5e50 con 0x7f320406a5f0 2024-09-11T01:24:19.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.748+0000 7f320ab82640 1 -- 172.21.15.78:0/1151861919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f320406aa50 con 0x7f320406a5f0 2024-09-11T01:24:19.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.749+0000 7f3201ffb640 1 -- 172.21.15.78:0/1151861919 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f31f8010610 con 0x7f320406a5f0 2024-09-11T01:24:19.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.750+0000 7f3201ffb640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f31f40b5450 0x7f31f40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:19.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.750+0000 7f3203fff640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f31f40b5450 0x7f31f40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:19.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.750+0000 7f3201ffb640 1 -- 172.21.15.78:0/1151861919 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f31f80e5940 con 0x7f320406a5f0 2024-09-11T01:24:19.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.750+0000 7f3203fff640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f31f40b5450 0x7f31f40b7910 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f31f0000c00 tx=0x7f31f0031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:19.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.754+0000 7f3201ffb640 1 -- 172.21.15.78:0/1151861919 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f31f80ac500 con 0x7f320406a5f0 2024-09-11T01:24:19.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.933+0000 7f320ab82640 1 -- 172.21.15.78:0/1151861919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f320402e5c0 con 0x7f320406a5f0 2024-09-11T01:24:19.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.934+0000 7f3201ffb640 1 -- 172.21.15.78:0/1151861919 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7f31f80abe00 con 0x7f320406a5f0 2024-09-11T01:24:19.935 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T01:24:19.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.937+0000 7f320ab82640 1 -- 172.21.15.78:0/1151861919 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f31f40b5450 msgr2=0x7f31f40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:19.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.937+0000 7f320ab82640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f31f40b5450 0x7f31f40b7910 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f31f0000c00 tx=0x7f31f0031400 comp rx=0 tx=0).stop 2024-09-11T01:24:19.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.937+0000 7f320ab82640 1 -- 172.21.15.78:0/1151861919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f320406a5f0 msgr2=0x7f3204199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:19.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.937+0000 7f320ab82640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f320406a5f0 0x7f3204199380 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f31f800b4a0 tx=0x7f31f800b960 comp rx=0 tx=0).stop 2024-09-11T01:24:19.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.937+0000 7f320ab82640 1 -- 172.21.15.78:0/1151861919 shutdown_connections 2024-09-11T01:24:19.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.937+0000 7f320ab82640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f31f40b5450 0x7f31f40b7910 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.937+0000 7f320ab82640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3204109040 0x7f320419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.937+0000 7f320ab82640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32041086c0 0x7f32041998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.937+0000 7f320ab82640 1 --2- 172.21.15.78:0/1151861919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f320406a5f0 0x7f3204199380 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:19.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.937+0000 7f320ab82640 1 -- 172.21.15.78:0/1151861919 >> 172.21.15.78:0/1151861919 conn(0x7f32040776b0 msgr2=0x7f3204106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:19.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.937+0000 7f320ab82640 1 -- 172.21.15.78:0/1151861919 shutdown_connections 2024-09-11T01:24:19.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:19.937+0000 7f320ab82640 1 -- 172.21.15.78:0/1151861919 wait complete. 2024-09-11T01:24:19.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:19 smithi110 ceph-mon[34712]: pgmap v1124: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 37 KiB/s wr, 21 op/s 2024-09-11T01:24:19.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:19 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/871567875' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T01:24:19.952 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:19 smithi078 ceph-mon[31315]: pgmap v1124: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 37 KiB/s wr, 21 op/s 2024-09-11T01:24:19.952 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:19 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/871567875' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T01:24:19.953 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T01:24:19.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:19 smithi112 ceph-mon[35384]: pgmap v1124: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 37 KiB/s wr, 21 op/s 2024-09-11T01:24:19.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:19 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/871567875' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T01:24:20.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.057+0000 7f18601f3640 1 -- 172.21.15.78:0/4099686575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1858102cd0 msgr2=0x7f18581030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.057+0000 7f18601f3640 1 --2- 172.21.15.78:0/4099686575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1858102cd0 0x7f18581030b0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f1854009900 tx=0x7f185402a410 comp rx=0 tx=0).stop 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.057+0000 7f18601f3640 1 -- 172.21.15.78:0/4099686575 shutdown_connections 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.057+0000 7f18601f3640 1 --2- 172.21.15.78:0/4099686575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1858109050 0x7f185810f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.057+0000 7f18601f3640 1 --2- 172.21.15.78:0/4099686575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18581086d0 0x7f1858108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.057+0000 7f18601f3640 1 --2- 172.21.15.78:0/4099686575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1858102cd0 0x7f18581030b0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.057+0000 7f18601f3640 1 -- 172.21.15.78:0/4099686575 >> 172.21.15.78:0/4099686575 conn(0x7f18580fe710 msgr2=0x7f1858100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.057+0000 7f18601f3640 1 -- 172.21.15.78:0/4099686575 shutdown_connections 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.057+0000 7f18601f3640 1 -- 172.21.15.78:0/4099686575 wait complete. 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f18601f3640 1 Processor -- start 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f18601f3640 1 -- start start 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f18601f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1858102cd0 0x7f1858199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f18601f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18581086d0 0x7f1858199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f18601f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1858109050 0x7f185819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f18601f3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1858112d10 con 0x7f1858102cd0 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f18601f3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1858112b90 con 0x7f18581086d0 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f18601f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1858112e90 con 0x7f1858109050 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f185df8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1858102cd0 0x7f1858199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f185df8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1858102cd0 0x7f1858199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42018/0 (socket says 172.21.15.78:42018) 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f185df8f640 1 -- 172.21.15.78:0/289290298 learned_addr learned my addr 172.21.15.78:0/289290298 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f185df8f640 1 -- 172.21.15.78:0/289290298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1858109050 msgr2=0x7f185819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f185df8f640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1858109050 0x7f185819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f185d78e640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18581086d0 0x7f1858199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f185df8f640 1 -- 172.21.15.78:0/289290298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18581086d0 msgr2=0x7f1858199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f185df8f640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18581086d0 0x7f1858199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f185df8f640 1 -- 172.21.15.78:0/289290298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185819f2f0 con 0x7f1858102cd0 2024-09-11T01:24:20.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f185df8f640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1858102cd0 0x7f1858199250 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1854005940 tx=0x7f18540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:20.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f184effd640 1 -- 172.21.15.78:0/289290298 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185403f030 con 0x7f1858102cd0 2024-09-11T01:24:20.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.058+0000 7f184effd640 1 -- 172.21.15.78:0/289290298 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1854003cb0 con 0x7f1858102cd0 2024-09-11T01:24:20.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.059+0000 7f18601f3640 1 -- 172.21.15.78:0/289290298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185819f580 con 0x7f1858102cd0 2024-09-11T01:24:20.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.059+0000 7f184effd640 1 -- 172.21.15.78:0/289290298 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854035070 con 0x7f1858102cd0 2024-09-11T01:24:20.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.059+0000 7f18601f3640 1 -- 172.21.15.78:0/289290298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18581a5ca0 con 0x7f1858102cd0 2024-09-11T01:24:20.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.060+0000 7f18601f3640 1 -- 172.21.15.78:0/289290298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18581030b0 con 0x7f1858102cd0 2024-09-11T01:24:20.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.061+0000 7f184effd640 1 -- 172.21.15.78:0/289290298 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f185403bbf0 con 0x7f1858102cd0 2024-09-11T01:24:20.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.062+0000 7f184effd640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18440b54d0 0x7f18440b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:20.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.062+0000 7f185d78e640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18440b54d0 0x7f18440b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:20.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.062+0000 7f184effd640 1 -- 172.21.15.78:0/289290298 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f1854106e20 con 0x7f1858102cd0 2024-09-11T01:24:20.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.062+0000 7f185d78e640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18440b54d0 0x7f18440b7990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1848005fd0 tx=0x7f1848005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:20.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.065+0000 7f184effd640 1 -- 172.21.15.78:0/289290298 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f18540cd9e0 con 0x7f1858102cd0 2024-09-11T01:24:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.242+0000 7f18601f3640 1 -- 172.21.15.78:0/289290298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f185802e5c0 con 0x7f1858102cd0 2024-09-11T01:24:20.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.242+0000 7f184effd640 1 -- 172.21.15.78:0/289290298 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) ==== 104+0+11 (secure 0 0 0) 0x7f18540d6190 con 0x7f1858102cd0 2024-09-11T01:24:20.244 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-09-11T01:24:20.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.244+0000 7f18601f3640 1 -- 172.21.15.78:0/289290298 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18440b54d0 msgr2=0x7f18440b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.244+0000 7f18601f3640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18440b54d0 0x7f18440b7990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1848005fd0 tx=0x7f1848005850 comp rx=0 tx=0).stop 2024-09-11T01:24:20.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.244+0000 7f18601f3640 1 -- 172.21.15.78:0/289290298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1858102cd0 msgr2=0x7f1858199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.244+0000 7f18601f3640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1858102cd0 0x7f1858199250 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1854005940 tx=0x7f18540042f0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.244+0000 7f18601f3640 1 -- 172.21.15.78:0/289290298 shutdown_connections 2024-09-11T01:24:20.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.244+0000 7f18601f3640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f18440b54d0 0x7f18440b7990 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.244+0000 7f18601f3640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1858109050 0x7f185819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.244+0000 7f18601f3640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18581086d0 0x7f1858199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.244+0000 7f18601f3640 1 --2- 172.21.15.78:0/289290298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1858102cd0 0x7f1858199250 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.244+0000 7f18601f3640 1 -- 172.21.15.78:0/289290298 >> 172.21.15.78:0/289290298 conn(0x7f18580fe710 msgr2=0x7f1858100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:20.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.244+0000 7f18601f3640 1 -- 172.21.15.78:0/289290298 shutdown_connections 2024-09-11T01:24:20.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.244+0000 7f18601f3640 1 -- 172.21.15.78:0/289290298 wait complete. 2024-09-11T01:24:20.257 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T01:24:20.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.357+0000 7f8f7ce39640 1 -- 172.21.15.78:0/26119360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810d020 msgr2=0x7f8f7810f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.357+0000 7f8f7ce39640 1 --2- 172.21.15.78:0/26119360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810d020 0x7f8f7810f4a0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009ca0 tx=0x7f8f7002a8c0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.357+0000 7f8f7ce39640 1 -- 172.21.15.78:0/26119360 shutdown_connections 2024-09-11T01:24:20.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.357+0000 7f8f7ce39640 1 --2- 172.21.15.78:0/26119360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810d020 0x7f8f7810f4a0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.357+0000 7f8f7ce39640 1 --2- 172.21.15.78:0/26119360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f78104470 0x7f8f7810ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.357+0000 7f8f7ce39640 1 --2- 172.21.15.78:0/26119360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f78103a70 0x7f8f78103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.357+0000 7f8f7ce39640 1 -- 172.21.15.78:0/26119360 >> 172.21.15.78:0/26119360 conn(0x7f8f780fd640 msgr2=0x7f8f780ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:20.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.357+0000 7f8f7ce39640 1 -- 172.21.15.78:0/26119360 shutdown_connections 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.357+0000 7f8f7ce39640 1 -- 172.21.15.78:0/26119360 wait complete. 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f7ce39640 1 Processor -- start 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f7ce39640 1 -- start start 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f7ce39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f78103a70 0x7f8f78199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f7ce39640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f78104470 0x7f8f78199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f7ce39640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810d020 0x7f8f7819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f7ce39640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78112cf0 con 0x7f8f7810d020 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f7ce39640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78112b70 con 0x7f8f78103a70 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f7ce39640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78112e70 con 0x7f8f78104470 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f76d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810d020 0x7f8f7819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f76d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810d020 0x7f8f7819eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42028/0 (socket says 172.21.15.78:42028) 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f76d9d640 1 -- 172.21.15.78:0/2886647155 learned_addr learned my addr 172.21.15.78:0/2886647155 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f7659c640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f78103a70 0x7f8f78199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f75d9b640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f78104470 0x7f8f78199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f76d9d640 1 -- 172.21.15.78:0/2886647155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f78104470 msgr2=0x7f8f78199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f76d9d640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f78104470 0x7f8f78199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f76d9d640 1 -- 172.21.15.78:0/2886647155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f78103a70 msgr2=0x7f8f78199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f76d9d640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f78103a70 0x7f8f78199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f76d9d640 1 -- 172.21.15.78:0/2886647155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7819f2d0 con 0x7f8f7810d020 2024-09-11T01:24:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.358+0000 7f8f75d9b640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f78104470 0x7f8f78199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.359+0000 7f8f76d9d640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810d020 0x7f8f7819eb00 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8f70005cc0 tx=0x7f8f7002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.359+0000 7f8f677fe640 1 -- 172.21.15.78:0/2886647155 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70003cf0 con 0x7f8f7810d020 2024-09-11T01:24:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.359+0000 7f8f677fe640 1 -- 172.21.15.78:0/2886647155 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f700044f0 con 0x7f8f7810d020 2024-09-11T01:24:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.359+0000 7f8f7ce39640 1 -- 172.21.15.78:0/2886647155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7819f560 con 0x7f8f7810d020 2024-09-11T01:24:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.359+0000 7f8f677fe640 1 -- 172.21.15.78:0/2886647155 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7002fd10 con 0x7f8f7810d020 2024-09-11T01:24:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.359+0000 7f8f7ce39640 1 -- 172.21.15.78:0/2886647155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f781a5d00 con 0x7f8f7810d020 2024-09-11T01:24:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.360+0000 7f8f7ce39640 1 -- 172.21.15.78:0/2886647155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f78103e50 con 0x7f8f7810d020 2024-09-11T01:24:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.361+0000 7f8f677fe640 1 -- 172.21.15.78:0/2886647155 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8f70004040 con 0x7f8f7810d020 2024-09-11T01:24:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.362+0000 7f8f677fe640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8f600b5450 0x7f8f600b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.362+0000 7f8f7659c640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8f600b5450 0x7f8f600b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.362+0000 7f8f677fe640 1 -- 172.21.15.78:0/2886647155 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f8f70107760 con 0x7f8f7810d020 2024-09-11T01:24:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.362+0000 7f8f7659c640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8f600b5450 0x7f8f600b7910 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8f680058c0 tx=0x7f8f6800a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:20.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.365+0000 7f8f677fe640 1 -- 172.21.15.78:0/2886647155 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8f700ce320 con 0x7f8f7810d020 2024-09-11T01:24:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.540+0000 7f8f7ce39640 1 -- 172.21.15.78:0/2886647155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8f781a60d0 con 0x7f8f7810d020 2024-09-11T01:24:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.540+0000 7f8f677fe640 1 -- 172.21.15.78:0/2886647155 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) ==== 100+0+11 (secure 0 0 0) 0x7f8f700cdc20 con 0x7f8f7810d020 2024-09-11T01:24:20.542 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-09-11T01:24:20.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.542+0000 7f8f7ce39640 1 -- 172.21.15.78:0/2886647155 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8f600b5450 msgr2=0x7f8f600b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.542+0000 7f8f7ce39640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8f600b5450 0x7f8f600b7910 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8f680058c0 tx=0x7f8f6800a2d0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.542+0000 7f8f7ce39640 1 -- 172.21.15.78:0/2886647155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810d020 msgr2=0x7f8f7819eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.542+0000 7f8f7ce39640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810d020 0x7f8f7819eb00 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8f70005cc0 tx=0x7f8f7002f470 comp rx=0 tx=0).stop 2024-09-11T01:24:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.542+0000 7f8f7ce39640 1 -- 172.21.15.78:0/2886647155 shutdown_connections 2024-09-11T01:24:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.542+0000 7f8f7ce39640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f8f600b5450 0x7f8f600b7910 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.542+0000 7f8f7ce39640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810d020 0x7f8f7819eb00 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.542+0000 7f8f7ce39640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f78104470 0x7f8f78199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.542+0000 7f8f7ce39640 1 --2- 172.21.15.78:0/2886647155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f78103a70 0x7f8f78199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.542+0000 7f8f7ce39640 1 -- 172.21.15.78:0/2886647155 >> 172.21.15.78:0/2886647155 conn(0x7f8f780fd640 msgr2=0x7f8f780fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.543+0000 7f8f7ce39640 1 -- 172.21.15.78:0/2886647155 shutdown_connections 2024-09-11T01:24:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.543+0000 7f8f7ce39640 1 -- 172.21.15.78:0/2886647155 wait complete. 2024-09-11T01:24:20.555 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-11T01:24:20.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.657+0000 7f519d756640 1 -- 172.21.15.78:0/1301021622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f519806a5f0 msgr2=0x7f519806a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.657+0000 7f519d756640 1 --2- 172.21.15.78:0/1301021622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f519806a5f0 0x7f519806a9d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5188009900 tx=0x7f518802a410 comp rx=0 tx=0).stop 2024-09-11T01:24:20.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.657+0000 7f519d756640 1 -- 172.21.15.78:0/1301021622 shutdown_connections 2024-09-11T01:24:20.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.657+0000 7f519d756640 1 --2- 172.21.15.78:0/1301021622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5198109120 0x7f519810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.657+0000 7f519d756640 1 --2- 172.21.15.78:0/1301021622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51981087a0 0x7f5198108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.657+0000 7f519d756640 1 --2- 172.21.15.78:0/1301021622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f519806a5f0 0x7f519806a9d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.657+0000 7f519d756640 1 -- 172.21.15.78:0/1301021622 >> 172.21.15.78:0/1301021622 conn(0x7f5198077670 msgr2=0x7f5198077a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:20.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.657+0000 7f519d756640 1 -- 172.21.15.78:0/1301021622 shutdown_connections 2024-09-11T01:24:20.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.657+0000 7f519d756640 1 -- 172.21.15.78:0/1301021622 wait complete. 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.657+0000 7f519d756640 1 Processor -- start 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f519d756640 1 -- start start 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f519d756640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f519806a5f0 0x7f5198199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f519d756640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51981087a0 0x7f5198199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f519d756640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5198109120 0x7f519819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f519d756640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5198112e30 con 0x7f51981087a0 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f519d756640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5198112cb0 con 0x7f5198109120 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f519d756640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5198112fb0 con 0x7f519806a5f0 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f518e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51981087a0 0x7f5198199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f518e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51981087a0 0x7f5198199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42044/0 (socket says 172.21.15.78:42044) 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f518e7fc640 1 -- 172.21.15.78:0/730966499 learned_addr learned my addr 172.21.15.78:0/730966499 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f51977fe640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5198109120 0x7f519819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f5196ffd640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f519806a5f0 0x7f5198199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:20.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f518e7fc640 1 -- 172.21.15.78:0/730966499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f519806a5f0 msgr2=0x7f5198199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f518e7fc640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f519806a5f0 0x7f5198199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f518e7fc640 1 -- 172.21.15.78:0/730966499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5198109120 msgr2=0x7f519819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:20.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f518e7fc640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5198109120 0x7f519819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:20.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.658+0000 7f518e7fc640 1 -- 172.21.15.78:0/730966499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519819f3b0 con 0x7f51981087a0 2024-09-11T01:24:20.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.659+0000 7f518e7fc640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51981087a0 0x7f5198199850 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f518000d950 tx=0x7f518000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:20.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.659+0000 7f5194ff9640 1 -- 172.21.15.78:0/730966499 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180017030 con 0x7f51981087a0 2024-09-11T01:24:20.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.659+0000 7f5194ff9640 1 -- 172.21.15.78:0/730966499 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5180004020 con 0x7f51981087a0 2024-09-11T01:24:20.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.659+0000 7f519d756640 1 -- 172.21.15.78:0/730966499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519819f6a0 con 0x7f51981087a0 2024-09-11T01:24:20.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.659+0000 7f5194ff9640 1 -- 172.21.15.78:0/730966499 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51800047a0 con 0x7f51981087a0 2024-09-11T01:24:20.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.659+0000 7f519d756640 1 -- 172.21.15.78:0/730966499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51981a5d60 con 0x7f51981087a0 2024-09-11T01:24:20.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.660+0000 7f519d756640 1 -- 172.21.15.78:0/730966499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519806a9d0 con 0x7f51981087a0 2024-09-11T01:24:20.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.661+0000 7f5194ff9640 1 -- 172.21.15.78:0/730966499 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5180009a30 con 0x7f51981087a0 2024-09-11T01:24:20.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.662+0000 7f5194ff9640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f51840b5450 0x7f51840b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:20.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.662+0000 7f5196ffd640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f51840b5450 0x7f51840b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:20.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.662+0000 7f5194ff9640 1 -- 172.21.15.78:0/730966499 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f51800e5930 con 0x7f51981087a0 2024-09-11T01:24:20.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.662+0000 7f5196ffd640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f51840b5450 0x7f51840b7910 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5188005ab0 tx=0x7f5188005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:20.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.665+0000 7f5194ff9640 1 -- 172.21.15.78:0/730966499 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f51800ac4f0 con 0x7f51981087a0 2024-09-11T01:24:20.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.884+0000 7f519d756640 1 -- 172.21.15.78:0/730966499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f51981a6140 con 0x7f51981087a0 2024-09-11T01:24:20.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.891+0000 7f5194ff9640 1 -- 172.21.15.78:0/730966499 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) ==== 81+0+828474 (secure 0 0 0) 0x7f51800abdf0 con 0x7f51981087a0 2024-09-11T01:24:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:24:20.905 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:24:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 24, 2024-09-11T01:24:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "active_gid": 14217, 2024-09-11T01:24:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "active_name": "x", 2024-09-11T01:24:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addrs": { 2024-09-11T01:24:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T01:24:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T01:24:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:6800", 2024-09-11T01:24:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1082762376 2024-09-11T01:24:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-09-11T01:24:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:6801", 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1082762376 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addr": "172.21.15.78:6801/1082762376", 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "active_change": "2024-09-11T00:50:15.928190+0000", 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "active_mgr_features": 4540719139924082687, 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "available": true, 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "standbys": [ 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 24148, 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "z", 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540719139924082687, 2024-09-11T01:24:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-09-11T01:24:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-09-11T01:24:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-09-11T01:24:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-09-11T01:24:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-09-11T01:24:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-09-11T01:24:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-09-11T01:24:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-09-11T01:24:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-09-11T01:24:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T01:24:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T01:24:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-09-11T01:24:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T01:24:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-09-11T01:24:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-09-11T01:24:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T01:24:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-11T01:24:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-09-11T01:24:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T01:24:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-09-11T01:24:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-09-11T01:24:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T01:24:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T01:24:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-09-11T01:24:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T01:24:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-09-11T01:24:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T01:24:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-09-11T01:24:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T01:24:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-09-11T01:24:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-09-11T01:24:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-09-11T01:24:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T01:24:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-09-11T01:24:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-09-11T01:24:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-09-11T01:24:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nginx": { 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nginx", 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nginx container image", 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_oauth2_proxy": { 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_oauth2_proxy", 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "oauth2-proxy container image", 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-09-11T01:24:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-11T01:24:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T01:24:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba_metrics": { 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba_metrics", 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T01:24:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T01:24:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-09-11T01:24:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-09-11T01:24:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T01:24:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T01:24:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-09-11T01:24:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-09-11T01:24:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T01:24:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T01:24:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T01:24:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-09-11T01:24:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-09-11T01:24:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T01:24:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T01:24:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-09-11T01:24:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T01:24:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-09-11T01:24:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T01:24:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T01:24:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T01:24:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-11T01:24:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-11T01:24:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "[]", 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "MULTICLUSTER_CONFIG": { 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-11T01:24:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "{}", 2024-09-11T01:24:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T01:24:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T01:24:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T01:24:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T01:24:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T01:24:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T01:24:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T01:24:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T01:24:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T01:24:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T01:24:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T01:24:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T01:24:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-09-11T01:24:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T01:24:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T01:24:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "UNSAFE_TLS_v1_2": { 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T01:24:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T01:24:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-09-11T01:24:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-09-11T01:24:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-11T01:24:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-11T01:24:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-11T01:24:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T01:24:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-11T01:24:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-11T01:24:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-09-11T01:24:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T01:24:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-09-11T01:24:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-09-11T01:24:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-09-11T01:24:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T01:24:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-09-11T01:24:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-09-11T01:24:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-09-11T01:24:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-09-11T01:24:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T01:24:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-09-11T01:24:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-09-11T01:24:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T01:24:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-09-11T01:24:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-09-11T01:24:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-09-11T01:24:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-09-11T01:24:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T01:24:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-09-11T01:24:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T01:24:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T01:24:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-09-11T01:24:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T01:24:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T01:24:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-11T01:24:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-11T01:24:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-09-11T01:24:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-09-11T01:24:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-09-11T01:24:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T01:24:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T01:24:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-09-11T01:24:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T01:24:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T01:24:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-11T01:24:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T01:24:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-09-11T01:24:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-09-11T01:24:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-09-11T01:24:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-09-11T01:24:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-11T01:24:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "max_requests": { 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_requests", 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "secondary_zone_period_retry_limit": { 2024-09-11T01:24:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.298 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-09-11T01:24:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-09-11T01:24:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-09-11T01:24:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-09-11T01:24:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-09-11T01:24:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-09-11T01:24:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-09-11T01:24:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smb", 2024-09-11T01:24:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "internal_store_backend": { 2024-09-11T01:24:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "internal_store_backend", 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "update_orchestration": { 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "update_orchestration", 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-09-11T01:24:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-09-11T01:24:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-09-11T01:24:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-09-11T01:24:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-09-11T01:24:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-09-11T01:24:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-09-11T01:24:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-09-11T01:24:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-09-11T01:24:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T01:24:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-09-11T01:24:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T01:24:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-09-11T01:24:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-09-11T01:24:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-09-11T01:24:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-09-11T01:24:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-09-11T01:24:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-09-11T01:24:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-09-11T01:24:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-09-11T01:24:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 24170, 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "y", 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540719139924082687, 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-09-11T01:24:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T01:24:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-09-11T01:24:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-09-11T01:24:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-09-11T01:24:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T01:24:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T01:24:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T01:24:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-09-11T01:24:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T01:24:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T01:24:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T01:24:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-09-11T01:24:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-09-11T01:24:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-09-11T01:24:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T01:24:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-09-11T01:24:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-09-11T01:24:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-09-11T01:24:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T01:24:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-09-11T01:24:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-09-11T01:24:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T01:24:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-09-11T01:24:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-11T01:24:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T01:24:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-09-11T01:24:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-09-11T01:24:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-09-11T01:24:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-09-11T01:24:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-09-11T01:24:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-09-11T01:24:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-09-11T01:24:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-09-11T01:24:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T01:24:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-09-11T01:24:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nginx": { 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nginx", 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nginx container image", 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T01:24:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_oauth2_proxy": { 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_oauth2_proxy", 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "oauth2-proxy container image", 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-09-11T01:24:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-09-11T01:24:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-09-11T01:24:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T01:24:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-09-11T01:24:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba_metrics": { 2024-09-11T01:24:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba_metrics", 2024-09-11T01:24:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T01:24:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-09-11T01:24:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-09-11T01:24:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-09-11T01:24:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-09-11T01:24:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-09-11T01:24:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-09-11T01:24:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-09-11T01:24:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-09-11T01:24:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-09-11T01:24:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T01:24:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-09-11T01:24:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T01:24:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-09-11T01:24:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-09-11T01:24:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-11T01:24:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-09-11T01:24:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-09-11T01:24:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-09-11T01:24:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T01:24:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T01:24:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T01:24:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T01:24:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-09-11T01:24:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T01:24:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T01:24:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T01:24:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T01:24:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T01:24:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T01:24:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T01:24:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T01:24:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T01:24:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T01:24:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T01:24:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T01:24:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T01:24:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T01:24:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "[]", 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "MULTICLUSTER_CONFIG": { 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "{}", 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T01:24:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T01:24:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T01:24:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T01:24:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T01:24:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "UNSAFE_TLS_v1_2": { 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-09-11T01:24:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-09-11T01:24:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-09-11T01:24:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-09-11T01:24:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-09-11T01:24:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-09-11T01:24:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-09-11T01:24:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-09-11T01:24:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-09-11T01:24:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-09-11T01:24:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T01:24:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-11T01:24:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-11T01:24:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-09-11T01:24:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-09-11T01:24:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-09-11T01:24:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T01:24:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T01:24:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-09-11T01:24:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-09-11T01:24:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-09-11T01:24:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-09-11T01:24:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-09-11T01:24:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-09-11T01:24:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-09-11T01:24:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-09-11T01:24:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-09-11T01:24:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-09-11T01:24:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T01:24:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-09-11T01:24:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-09-11T01:24:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-09-11T01:24:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-09-11T01:24:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-09-11T01:24:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-09-11T01:24:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-09-11T01:24:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T01:24:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T01:24:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-11T01:24:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-09-11T01:24:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-09-11T01:24:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-09-11T01:24:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T01:24:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-11T01:24:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T01:24:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T01:24:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T01:24:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-09-11T01:24:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-11T01:24:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-09-11T01:24:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "max_requests": { 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_requests", 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T01:24:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "secondary_zone_period_retry_limit": { 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-11T01:24:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-09-11T01:24:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-09-11T01:24:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-09-11T01:24:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-09-11T01:24:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-09-11T01:24:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-09-11T01:24:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smb", 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "internal_store_backend": { 2024-09-11T01:24:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "internal_store_backend", 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "update_orchestration": { 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "update_orchestration", 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-11T01:24:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-09-11T01:24:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-09-11T01:24:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T01:24:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-09-11T01:24:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-09-11T01:24:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-09-11T01:24:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T01:24:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T01:24:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-09-11T01:24:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-09-11T01:24:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-09-11T01:24:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "modules": [ 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "dashboard", 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "iostat", 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "nfs", 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "restful", 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "snap_schedule" 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T01:24:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-11T01:24:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-09-11T01:24:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-09-11T01:24:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T01:24:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T01:24:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T01:24:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T01:24:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-09-11T01:24:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-09-11T01:24:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T01:24:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-09-11T01:24:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-09-11T01:24:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-11T01:24:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-09-11T01:24:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T01:24:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-09-11T01:24:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-09-11T01:24:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-09-11T01:24:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T01:24:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-09-11T01:24:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-09-11T01:24:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T01:24:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T01:24:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-09-11T01:24:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T01:24:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-09-11T01:24:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-09-11T01:24:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T01:24:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-09-11T01:24:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-09-11T01:24:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-09-11T01:24:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-09-11T01:24:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-11T01:24:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T01:24:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-09-11T01:24:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-09-11T01:24:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T01:24:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-09-11T01:24:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-09-11T01:24:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-09-11T01:24:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-09-11T01:24:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-09-11T01:24:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T01:24:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nginx": { 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nginx", 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-11T01:24:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nginx container image", 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-11T01:24:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T01:24:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-09-11T01:24:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-09-11T01:24:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_oauth2_proxy": { 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_oauth2_proxy", 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "oauth2-proxy container image", 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-09-11T01:24:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-09-11T01:24:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba_metrics": { 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba_metrics", 2024-09-11T01:24:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-09-11T01:24:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-09-11T01:24:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T01:24:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-09-11T01:24:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-09-11T01:24:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T01:24:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-09-11T01:24:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T01:24:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-09-11T01:24:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T01:24:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-09-11T01:24:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-09-11T01:24:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-09-11T01:24:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T01:24:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-09-11T01:24:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-09-11T01:24:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T01:24:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-09-11T01:24:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-09-11T01:24:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-09-11T01:24:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T01:24:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-09-11T01:24:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-09-11T01:24:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-09-11T01:24:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T01:24:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-09-11T01:24:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-09-11T01:24:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-09-11T01:24:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T01:24:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-09-11T01:24:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-09-11T01:24:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T01:24:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-09-11T01:24:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-09-11T01:24:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-09-11T01:24:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-09-11T01:24:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-09-11T01:24:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-09-11T01:24:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-09-11T01:24:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-09-11T01:24:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-09-11T01:24:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T01:24:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-09-11T01:24:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T01:24:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T01:24:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T01:24:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T01:24:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T01:24:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T01:24:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T01:24:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T01:24:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T01:24:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "[]", 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "MULTICLUSTER_CONFIG": { 2024-09-11T01:24:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-11T01:24:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "{}", 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T01:24:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T01:24:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T01:24:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T01:24:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T01:24:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T01:24:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T01:24:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-09-11T01:24:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "UNSAFE_TLS_v1_2": { 2024-09-11T01:24:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-11T01:24:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T01:24:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T01:24:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T01:24:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T01:24:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-09-11T01:24:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-09-11T01:24:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-09-11T01:24:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-09-11T01:24:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-09-11T01:24:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T01:24:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-11T01:24:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-09-11T01:24:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-09-11T01:24:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-09-11T01:24:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T01:24:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-09-11T01:24:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-09-11T01:24:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-09-11T01:24:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-09-11T01:24:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.200 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.201 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.202 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.203 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-09-11T01:24:22.204 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.205 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.206 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.207 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.208 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.209 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.210 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-09-11T01:24:22.211 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.212 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.213 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.214 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.215 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.216 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.216 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.216 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.216 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.216 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.216 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.216 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.216 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.216 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.216 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.216 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.217 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.218 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.219 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.221 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-09-11T01:24:22.221 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.221 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.221 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.221 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.221 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.221 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.221 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.221 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.221 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.221 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.224 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.225 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.226 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-09-11T01:24:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.228 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.230 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.231 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.232 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.233 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.234 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.235 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.236 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.237 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.238 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:22.239 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.240 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.241 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.242 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.243 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.244 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.245 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.247 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.248 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.249 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-09-11T01:24:22.250 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T01:24:22.251 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.252 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.254 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.255 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.256 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.257 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.259 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.260 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.261 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.262 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.265 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.266 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.267 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.268 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.270 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.271 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.272 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.273 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.274 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.276 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.277 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.278 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.279 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.280 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.283 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.284 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.285 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.286 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.287 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.288 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.288 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.288 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.288 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.288 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.288 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.288 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.288 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.288 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.288 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.288 INFO:teuthology.orchestra.run.smithi078.stdout: "max_requests": { 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_requests", 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.289 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.290 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.291 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.292 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.293 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.296 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.297 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: "secondary_zone_period_retry_limit": { 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.298 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.299 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.300 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.301 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.301 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.301 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.301 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.301 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-09-11T01:24:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-09-11T01:24:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-09-11T01:24:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-09-11T01:24:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smb", 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "internal_store_backend": { 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "internal_store_backend", 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "update_orchestration": { 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "update_orchestration", 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T01:24:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-09-11T01:24:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-09-11T01:24:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-09-11T01:24:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-09-11T01:24:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T01:24:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-09-11T01:24:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-09-11T01:24:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-09-11T01:24:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-09-11T01:24:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-09-11T01:24:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T01:24:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-09-11T01:24:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-09-11T01:24:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-09-11T01:24:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-09-11T01:24:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-09-11T01:24:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-09-11T01:24:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-09-11T01:24:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.388 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.389 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.390 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.391 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.392 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-09-11T01:24:22.393 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-09-11T01:24:22.393 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.393 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.393 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.393 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T01:24:22.393 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.393 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.393 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.393 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.393 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.393 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.393 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.394 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.396 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.398 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.399 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T01:24:22.400 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.401 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.402 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.403 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.404 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.405 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.406 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.407 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.408 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.409 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.410 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T01:24:22.411 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.415 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.416 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.417 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.418 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.419 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.420 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.421 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.422 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T01:24:22.423 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: "services": { 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: "dashboard": "https://172.21.15.78:8443/" 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: "always_on_modules": { 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: "octopus": [ 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-11T01:24:22.424 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "pacific": [ 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-11T01:24:22.425 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "quincy": [ 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-11T01:24:22.426 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "reef": [ 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-11T01:24:22.427 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "squid": [ 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-11T01:24:22.428 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: "force_disabled_modules": {}, 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: "last_failure_osd_epoch": 5, 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: "active_clients": [ 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 4150137614 2024-09-11T01:24:22.429 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "libcephsqlite", 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1183783243 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.430 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2925630667 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.431 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 734597766 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 3944368351 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.432 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2743194845 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T01:24:22.433 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:24:22.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.897+0000 7f519d756640 1 -- 172.21.15.78:0/730966499 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f51840b5450 msgr2=0x7f51840b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:22.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.897+0000 7f519d756640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f51840b5450 0x7f51840b7910 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5188005ab0 tx=0x7f5188005a20 comp rx=0 tx=0).stop 2024-09-11T01:24:22.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.897+0000 7f519d756640 1 -- 172.21.15.78:0/730966499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51981087a0 msgr2=0x7f5198199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:22.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.897+0000 7f519d756640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51981087a0 0x7f5198199850 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f518000d950 tx=0x7f518000de10 comp rx=0 tx=0).stop 2024-09-11T01:24:22.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.897+0000 7f519d756640 1 -- 172.21.15.78:0/730966499 shutdown_connections 2024-09-11T01:24:22.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.897+0000 7f519d756640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f51840b5450 0x7f51840b7910 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:22.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.897+0000 7f519d756640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5198109120 0x7f519819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:22.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.897+0000 7f519d756640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51981087a0 0x7f5198199850 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:22.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.897+0000 7f519d756640 1 --2- 172.21.15.78:0/730966499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f519806a5f0 0x7f5198199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:22.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.897+0000 7f519d756640 1 -- 172.21.15.78:0/730966499 >> 172.21.15.78:0/730966499 conn(0x7f5198077670 msgr2=0x7f5198100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:22.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.897+0000 7f519d756640 1 -- 172.21.15.78:0/730966499 shutdown_connections 2024-09-11T01:24:22.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:20.897+0000 7f519d756640 1 -- 172.21.15.78:0/730966499 wait complete. 2024-09-11T01:24:22.435 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:20 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1151861919' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T01:24:22.436 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:20 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/289290298' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-11T01:24:22.436 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:20 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2886647155' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-11T01:24:22.444 DEBUG:tasks.check_counter:Getting stats from a 2024-09-11T01:24:22.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-11T01:24:22.445 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:21 smithi078 ceph-mon[31315]: pgmap v1125: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 19 op/s 2024-09-11T01:24:22.445 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:21 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/730966499' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-11T01:24:22.446 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:20 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1151861919' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T01:24:22.446 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:20 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/289290298' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-11T01:24:22.446 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:20 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2886647155' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-11T01:24:22.446 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:21 smithi112 ceph-mon[35384]: pgmap v1125: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 19 op/s 2024-09-11T01:24:22.446 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:21 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/730966499' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-11T01:24:22.446 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:20 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1151861919' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T01:24:22.446 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:20 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/289290298' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-11T01:24:22.446 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:20 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2886647155' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-11T01:24:22.446 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:21 smithi110 ceph-mon[34712]: pgmap v1125: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 19 op/s 2024-09-11T01:24:22.447 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:21 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/730966499' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-11T01:24:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:23.368 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:24:23.369 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T01:24:23.369 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 655746, 2024-09-11T01:24:23.369 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 698605, 2024-09-11T01:24:23.369 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 205995261, 2024-09-11T01:24:23.369 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 795440954, 2024-09-11T01:24:23.369 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 21, 2024-09-11T01:24:23.369 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 7, 2024-09-11T01:24:23.369 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 75.652403441, 2024-09-11T01:24:23.369 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 51.501196618, 2024-09-11T01:24:23.369 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 26.233647350, 2024-09-11T01:24:23.369 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 4.118798598, 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 698599, 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 42.818469690, 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000061291 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1241640, 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.185725282, 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000149 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T01:24:23.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.371 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T01:24:23.371 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 555936, 2024-09-11T01:24:23.371 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 461771, 2024-09-11T01:24:23.371 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 181204702, 2024-09-11T01:24:23.371 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 493674721, 2024-09-11T01:24:23.371 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 9, 2024-09-11T01:24:23.371 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 5, 2024-09-11T01:24:23.371 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 36.740113985, 2024-09-11T01:24:23.371 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 17.857849727, 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 19.869873979, 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 3.061897169, 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 461774, 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 22.653120516, 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000049056 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 947700, 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.140550247, 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000148 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-11T01:24:23.372 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 691696 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 209403, 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 198472, 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 68930527, 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 558020308, 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 62, 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 9, 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 19.084545766, 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 8.167671344, 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 7.417334664, 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 4.515902332, 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:23.373 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 198471, 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 12.232529913, 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000061633 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 330906, 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.055132578, 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000166 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1322592, 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 317424 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-09-11T01:24:23.374 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-09-11T01:24:23.375 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-09-11T01:24:23.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.375 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-09-11T01:24:23.375 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-11T01:24:23.375 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-11T01:24:23.375 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 89865, 2024-09-11T01:24:23.375 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 121.128300457, 2024-09-11T01:24:23.375 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001347891 2024-09-11T01:24:23.375 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 64813, 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 7.548660426, 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000116468 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 274805, 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 274046, 2024-09-11T01:24:23.376 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-09-11T01:24:23.377 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 274046, 2024-09-11T01:24:23.377 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1121.332904611, 2024-09-11T01:24:23.377 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.004091768 2024-09-11T01:24:23.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.377 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-09-11T01:24:23.377 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 759, 2024-09-11T01:24:23.377 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 18, 2024-09-11T01:24:23.377 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-09-11T01:24:23.377 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 112, 2024-09-11T01:24:23.377 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 4, 2024-09-11T01:24:23.378 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 2, 2024-09-11T01:24:23.378 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 2038, 2024-09-11T01:24:23.378 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 1016, 2024-09-11T01:24:23.378 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 992, 2024-09-11T01:24:23.378 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 30, 2024-09-11T01:24:23.378 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 510, 2024-09-11T01:24:23.378 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 427, 2024-09-11T01:24:23.378 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-09-11T01:24:23.378 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-09-11T01:24:23.378 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 6, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 456027, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 293395, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 752, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 122, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 6746, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 3000, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 3, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 12, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 205, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 5, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 982, 2024-09-11T01:24:23.379 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 2062, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 4, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 234, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 7, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 2062, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 220, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 220, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 1842, 2024-09-11T01:24:23.380 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 49, 2024-09-11T01:24:23.381 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 10579, 2024-09-11T01:24:23.381 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 105577, 2024-09-11T01:24:23.381 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T01:24:23.381 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T01:24:23.381 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T01:24:23.381 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 143770, 2024-09-11T01:24:23.381 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 29300, 2024-09-11T01:24:23.381 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 8981, 2024-09-11T01:24:23.381 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 234868 2024-09-11T01:24:23.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.382 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-09-11T01:24:23.382 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 662, 2024-09-11T01:24:23.382 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 5798, 2024-09-11T01:24:23.382 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 2953, 2024-09-11T01:24:23.382 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 5715, 2024-09-11T01:24:23.382 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 787, 2024-09-11T01:24:23.382 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 20, 2024-09-11T01:24:23.382 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 34199, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 34179, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 12, 2024-09-11T01:24:23.383 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-09-11T01:24:23.384 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 6, 2024-09-11T01:24:23.384 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-09-11T01:24:23.384 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0, 2024-09-11T01:24:23.384 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:23.384 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_succeeded": 0, 2024-09-11T01:24:23.384 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_write_failed": 0 2024-09-11T01:24:23.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.384 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-09-11T01:24:23.384 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-09-11T01:24:23.384 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 164382, 2024-09-11T01:24:23.384 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 115597, 2024-09-11T01:24:23.385 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 110611, 2024-09-11T01:24:23.385 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 53771, 2024-09-11T01:24:23.385 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-09-11T01:24:23.385 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 4986, 2024-09-11T01:24:23.385 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 326, 2024-09-11T01:24:23.385 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 198, 2024-09-11T01:24:23.385 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 186, 2024-09-11T01:24:23.385 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 140, 2024-09-11T01:24:23.385 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 5, 2024-09-11T01:24:23.385 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 12, 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 641098720, 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 1205154030, 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 0, 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 49556, 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 579.468555106, 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.011693206 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 0 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-09-11T01:24:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 1666, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 36153, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 34487, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 81, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 3021, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 2940, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 2052, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 56198, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 54146, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 36701, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 36701, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 93016, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 304996352 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 400267, 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 39, 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 274805, 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 4294, 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 5422, 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 34025, 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 239.018975155, 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.007024804 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 26460, 2024-09-11T01:24:23.388 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 338.522750348, 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.012793754 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.389 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-09-11T01:24:23.390 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 54524, 2024-09-11T01:24:23.390 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 133.751186149, 2024-09-11T01:24:23.390 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002453069 2024-09-11T01:24:23.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.390 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-09-11T01:24:23.390 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.390 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.390 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.390 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-09-11T01:24:23.390 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.391 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.391 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.391 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-09-11T01:24:23.391 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.391 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.391 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.391 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-09-11T01:24:23.391 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.392 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.392 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.392 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-09-11T01:24:23.393 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.393 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.393 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.393 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-09-11T01:24:23.393 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.393 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.393 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.394 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-09-11T01:24:23.394 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.394 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.394 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.394 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-09-11T01:24:23.394 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.394 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.395 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.395 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-09-11T01:24:23.395 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.395 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.395 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.396 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-09-11T01:24:23.396 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.396 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.396 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.396 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-09-11T01:24:23.397 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 116850, 2024-09-11T01:24:23.397 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 197.301312569, 2024-09-11T01:24:23.397 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001688500 2024-09-11T01:24:23.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.397 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-09-11T01:24:23.397 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 6962, 2024-09-11T01:24:23.398 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 53.123302329, 2024-09-11T01:24:23.398 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.007630465 2024-09-11T01:24:23.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.398 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-09-11T01:24:23.398 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.398 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.399 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.399 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-09-11T01:24:23.399 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.399 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.399 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.399 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-09-11T01:24:23.399 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 20, 2024-09-11T01:24:23.399 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.096359357, 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.054817967 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-09-11T01:24:23.400 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 58, 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.979514101, 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.016888174 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 968, 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.656586980, 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001711350 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.401 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-09-11T01:24:23.402 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.402 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.402 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.402 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-09-11T01:24:23.402 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.402 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.402 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.402 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-09-11T01:24:23.402 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.403 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.403 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.404 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-09-11T01:24:23.404 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 34179, 2024-09-11T01:24:23.404 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 155.882917623, 2024-09-11T01:24:23.404 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.004560780 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 1, 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 1, 2024-09-11T01:24:23.405 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 0, 2024-09-11T01:24:23.406 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 1, 2024-09-11T01:24:23.406 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-09-11T01:24:23.406 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 30869, 2024-09-11T01:24:23.406 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 30869, 2024-09-11T01:24:23.406 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 2599, 2024-09-11T01:24:23.406 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-09-11T01:24:23.406 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.407 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-09-11T01:24:23.407 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-09-11T01:24:23.407 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-09-11T01:24:23.407 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T01:24:23.407 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-09-11T01:24:23.407 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T01:24:23.407 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-09-11T01:24:23.408 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T01:24:23.408 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T01:24:23.408 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T01:24:23.408 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T01:24:23.408 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T01:24:23.408 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-09-11T01:24:23.409 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T01:24:23.409 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T01:24:23.409 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-09-11T01:24:23.409 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-09-11T01:24:23.409 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-09-11T01:24:23.409 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-09-11T01:24:23.409 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T01:24:23.409 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T01:24:23.409 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T01:24:23.409 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T01:24:23.409 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T01:24:23.410 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-09-11T01:24:23.410 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-09-11T01:24:23.410 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-09-11T01:24:23.410 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T01:24:23.410 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T01:24:23.410 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-09-11T01:24:23.410 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-09-11T01:24:23.411 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-09-11T01:24:23.411 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-09-11T01:24:23.411 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T01:24:23.411 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-09-11T01:24:23.411 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T01:24:23.411 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-09-11T01:24:23.412 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 496973, 2024-09-11T01:24:23.412 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 1154, 2024-09-11T01:24:23.412 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 616, 2024-09-11T01:24:23.412 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 7, 2024-09-11T01:24:23.412 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-09-11T01:24:23.412 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 7976, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 177, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 5883208, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 47285, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-09-11T01:24:23.413 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-09-11T01:24:23.414 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-09-11T01:24:23.414 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-09-11T01:24:23.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.414 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-09-11T01:24:23.414 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-09-11T01:24:23.414 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-09-11T01:24:23.414 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 91082, 2024-09-11T01:24:23.414 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 1209458693, 2024-09-11T01:24:23.414 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 33, 2024-09-11T01:24:23.414 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 91082, 2024-09-11T01:24:23.414 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-09-11T01:24:23.415 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 91082, 2024-09-11T01:24:23.415 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 995.015832677, 2024-09-11T01:24:23.415 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.010924395 2024-09-11T01:24:23.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.415 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-09-11T01:24:23.415 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-09-11T01:24:23.416 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 91082, 2024-09-11T01:24:23.416 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 93599 2024-09-11T01:24:23.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.416 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 91082, 2024-09-11T01:24:23.416 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 3368, 2024-09-11T01:24:23.416 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 87714, 2024-09-11T01:24:23.417 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-09-11T01:24:23.417 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-09-11T01:24:23.417 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 48, 2024-09-11T01:24:23.417 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 143, 2024-09-11T01:24:23.417 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 1270, 2024-09-11T01:24:23.417 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 50826, 2024-09-11T01:24:23.417 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 733, 2024-09-11T01:24:23.418 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-09-11T01:24:23.418 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-09-11T01:24:23.418 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-09-11T01:24:23.418 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 4, 2024-09-11T01:24:23.418 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 34632, 2024-09-11T01:24:23.418 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-09-11T01:24:23.418 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-09-11T01:24:23.418 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-09-11T01:24:23.419 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 2063, 2024-09-11T01:24:23.419 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 286, 2024-09-11T01:24:23.419 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-09-11T01:24:23.419 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-09-11T01:24:23.419 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-09-11T01:24:23.419 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-09-11T01:24:23.419 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-09-11T01:24:23.420 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-09-11T01:24:23.420 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T01:24:23.420 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-09-11T01:24:23.420 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-09-11T01:24:23.420 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-09-11T01:24:23.421 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-09-11T01:24:23.421 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-09-11T01:24:23.421 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-09-11T01:24:23.421 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-09-11T01:24:23.421 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-09-11T01:24:23.421 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-09-11T01:24:23.421 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-09-11T01:24:23.422 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-09-11T01:24:23.422 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-09-11T01:24:23.422 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-09-11T01:24:23.422 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-09-11T01:24:23.422 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-09-11T01:24:23.422 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-09-11T01:24:23.423 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-09-11T01:24:23.423 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-09-11T01:24:23.423 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-09-11T01:24:23.423 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 115, 2024-09-11T01:24:23.423 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-09-11T01:24:23.423 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 40, 2024-09-11T01:24:23.423 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 12, 2024-09-11T01:24:23.423 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 12, 2024-09-11T01:24:23.423 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-09-11T01:24:23.423 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-09-11T01:24:23.423 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 2690, 2024-09-11T01:24:23.423 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 54, 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 850 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 1, 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 6, 2024-09-11T01:24:23.424 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 34179, 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 34179, 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 1421081, 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 328198698, 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 1421081, 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-11T01:24:23.425 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 1421081, 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 328198698, 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-11T01:24:23.426 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-11T01:24:23.427 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:23.427 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:23.427 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 91082, 2024-09-11T01:24:23.427 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 1212919380, 2024-09-11T01:24:23.427 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 91082, 2024-09-11T01:24:23.427 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1212919380, 2024-09-11T01:24:23.427 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-11T01:24:23.427 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.428 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.428 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.428 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:23.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.428 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-09-11T01:24:23.428 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-11T01:24:23.429 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-09-11T01:24:23.429 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-11T01:24:23.429 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-11T01:24:23.429 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-11T01:24:23.429 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:23.429 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:23.430 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 91082, 2024-09-11T01:24:23.430 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 91082, 2024-09-11T01:24:23.430 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 91082, 2024-09-11T01:24:23.430 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 91082, 2024-09-11T01:24:23.430 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-11T01:24:23.430 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.430 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.430 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.430 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 34179, 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 3461503, 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 34179, 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 1270, 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 3461503, 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-11T01:24:23.431 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.432 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.432 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.432 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:23.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.432 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x565356112dc0": { 2024-09-11T01:24:23.432 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-11T01:24:23.432 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-11T01:24:23.432 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-11T01:24:23.432 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 164382, 2024-09-11T01:24:23.432 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 1200959726, 2024-09-11T01:24:23.432 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:23.433 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 164382, 2024-09-11T01:24:23.433 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-11T01:24:23.433 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-11T01:24:23.433 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 49556, 2024-09-11T01:24:23.433 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1200959726, 2024-09-11T01:24:23.433 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-11T01:24:23.433 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:23.433 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:23.433 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:23.434 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:23.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:23.434 INFO:teuthology.orchestra.run.smithi078.stdout: "trackedop": { 2024-09-11T01:24:23.434 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 2 2024-09-11T01:24:23.434 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:23.434 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:24:23.942 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:23 smithi110 ceph-mon[34712]: pgmap v1126: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-09-11T01:24:23.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:23 smithi112 ceph-mon[35384]: pgmap v1126: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-09-11T01:24:23.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:23 smithi078 ceph-mon[31315]: pgmap v1126: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-09-11T01:24:24.128 INFO:tasks.check_counter:Daemon mds.a mds.exported=12 2024-09-11T01:24:24.128 INFO:tasks.check_counter:Daemon mds.a mds.imported=5 2024-09-11T01:24:24.128 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=662 2024-09-11T01:24:24.128 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=5798 2024-09-11T01:24:24.128 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=7 2024-09-11T01:24:24.129 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-11T01:24:24.129 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T01:24:24.129 DEBUG:tasks.check_counter:Getting stats from d 2024-09-11T01:24:24.129 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-11T01:24:24.377 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:24.905 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:24:24.905 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T01:24:24.905 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 0, 2024-09-11T01:24:24.906 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2, 2024-09-11T01:24:24.906 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 0, 2024-09-11T01:24:24.906 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 0, 2024-09-11T01:24:24.906 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-09-11T01:24:24.906 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-09-11T01:24:24.906 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.001151979, 2024-09-11T01:24:24.906 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T01:24:24.906 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000204395, 2024-09-11T01:24:24.907 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T01:24:24.907 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:24.907 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:24.907 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:24.907 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:24.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:24.907 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:24.907 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:24.908 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:24.908 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:24.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:24.908 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T01:24:24.908 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T01:24:24.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:24.908 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T01:24:24.908 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 456, 2024-09-11T01:24:24.908 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 449, 2024-09-11T01:24:24.909 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 900864, 2024-09-11T01:24:24.909 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 277200, 2024-09-11T01:24:24.909 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-09-11T01:24:24.909 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-09-11T01:24:24.909 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.063534593, 2024-09-11T01:24:24.909 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.031861066, 2024-09-11T01:24:24.909 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.038246392, 2024-09-11T01:24:24.909 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T01:24:24.910 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:24.910 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 449, 2024-09-11T01:24:24.910 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.025285603, 2024-09-11T01:24:24.910 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000056315 2024-09-11T01:24:24.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:24.910 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:24.910 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:24.910 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:24.911 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:24.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:24.911 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 900864, 2024-09-11T01:24:24.911 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 277200 2024-09-11T01:24:24.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:24.911 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T01:24:24.911 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 2, 2024-09-11T01:24:24.911 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 342, 2024-09-11T01:24:24.911 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 320, 2024-09-11T01:24:24.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 221024, 2024-09-11T01:24:24.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-09-11T01:24:24.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-09-11T01:24:24.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.026255037, 2024-09-11T01:24:24.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.021911112, 2024-09-11T01:24:24.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001612823, 2024-09-11T01:24:24.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T01:24:24.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:24.913 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 340, 2024-09-11T01:24:24.913 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.015492333, 2024-09-11T01:24:24.913 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000045565 2024-09-11T01:24:24.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:24.913 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:24.913 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:24.913 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:24.913 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:24.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:24.914 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T01:24:24.914 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 221024 2024-09-11T01:24:24.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:24.914 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-09-11T01:24:24.914 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-09-11T01:24:24.914 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-09-11T01:24:24.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:24.915 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-09-11T01:24:24.915 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-09-11T01:24:24.915 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-09-11T01:24:24.915 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T01:24:24.915 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-09-11T01:24:24.915 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T01:24:24.915 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-09-11T01:24:24.915 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T01:24:24.915 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T01:24:24.916 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T01:24:24.916 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T01:24:24.916 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T01:24:24.916 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-09-11T01:24:24.916 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T01:24:24.916 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T01:24:24.916 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-09-11T01:24:24.916 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-09-11T01:24:24.917 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-09-11T01:24:24.917 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-09-11T01:24:24.917 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T01:24:24.917 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T01:24:24.917 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T01:24:24.917 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T01:24:24.917 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T01:24:24.917 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-09-11T01:24:24.917 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-09-11T01:24:24.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-09-11T01:24:24.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T01:24:24.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T01:24:24.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-09-11T01:24:24.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-09-11T01:24:24.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-09-11T01:24:24.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-09-11T01:24:24.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T01:24:24.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-09-11T01:24:24.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T01:24:24.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-09-11T01:24:24.919 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148721, 2024-09-11T01:24:24.919 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 30, 2024-09-11T01:24:24.919 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-09-11T01:24:24.919 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-09-11T01:24:24.919 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-09-11T01:24:24.919 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-09-11T01:24:24.920 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-09-11T01:24:24.920 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-09-11T01:24:24.920 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-09-11T01:24:24.920 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-09-11T01:24:24.920 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-09-11T01:24:24.920 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-09-11T01:24:24.920 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T01:24:24.920 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-09-11T01:24:24.921 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-09-11T01:24:24.921 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-09-11T01:24:24.921 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 968, 2024-09-11T01:24:24.921 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-09-11T01:24:24.921 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-09-11T01:24:24.921 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-09-11T01:24:24.921 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-09-11T01:24:24.921 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 454, 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 833939, 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 454, 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 454, 2024-09-11T01:24:24.922 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 833939, 2024-09-11T01:24:24.923 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-11T01:24:24.923 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:24.923 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:24.923 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:24.923 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:24.923 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:24.923 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:24:25.673 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-11T01:24:25.673 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-11T01:24:25.673 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-11T01:24:25.673 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-11T01:24:25.674 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-11T01:24:25.674 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-11T01:24:25.674 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-11T01:24:25.674 DEBUG:tasks.check_counter:Getting stats from g 2024-09-11T01:24:25.674 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-11T01:24:25.920 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:25.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:25 smithi110 ceph-mon[34712]: pgmap v1127: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-11T01:24:25.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:25 smithi112 ceph-mon[35384]: pgmap v1127: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-11T01:24:25.998 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:25 smithi078 ceph-mon[31315]: pgmap v1127: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-11T01:24:26.440 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:24:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T01:24:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 3567, 2024-09-11T01:24:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 3509, 2024-09-11T01:24:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 288803614, 2024-09-11T01:24:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 1242900, 2024-09-11T01:24:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 7, 2024-09-11T01:24:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-09-11T01:24:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.525147200, 2024-09-11T01:24:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.168223312, 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.274039691, 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.071531955, 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3508, 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.120474469, 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000034342 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1296496, 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 281904 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 2620, 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2955, 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 386569807, 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 1613725, 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 8, 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 6, 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.481323785, 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.102279354, 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.306029280, 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.056266168, 2024-09-11T01:24:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2954, 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.085286855, 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000028871 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 782416 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T01:24:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1757, 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1759, 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 528954324, 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 558738, 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 7, 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 5, 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.536230107, 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.058838443, 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.374181749, 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.091094123, 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1757, 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.055621089, 2024-09-11T01:24:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000031656 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-09-11T01:24:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-11T01:24:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-11T01:24:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 11732, 2024-09-11T01:24:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.398296138, 2024-09-11T01:24:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000033949 2024-09-11T01:24:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 0, 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 0, 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-09-11T01:24:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 0, 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 0, 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 36170, 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 16489, 2024-09-11T01:24:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 19681, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 0, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 12614, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 0, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 1, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 0, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 0, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 0, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-09-11T01:24:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-09-11T01:24:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2024-09-11T01:24:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 0, 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 0 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2024-09-11T01:24:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-09-11T01:24:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-09-11T01:24:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-09-11T01:24:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-09-11T01:24:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-09-11T01:24:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-09-11T01:24:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-09-11T01:24:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-09-11T01:24:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-09-11T01:24:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T01:24:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2024-09-11T01:24:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_started": 0, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_succeeded": 0, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "uninline_write_failed": 0 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 0, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 164382, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-09-11T01:24:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-09-11T01:24:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 0, 2024-09-11T01:24:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-09-11T01:24:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-09-11T01:24:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 140, 2024-09-11T01:24:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 15, 2024-09-11T01:24:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-09-11T01:24:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-09-11T01:24:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 641098720, 2024-09-11T01:24:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 1205154030, 2024-09-11T01:24:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 1205139318, 2024-09-11T01:24:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-09-11T01:24:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 164382 2024-09-11T01:24:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-09-11T01:24:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 1629, 2024-09-11T01:24:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 36003, 2024-09-11T01:24:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 34374, 2024-09-11T01:24:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 64, 2024-09-11T01:24:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 104, 2024-09-11T01:24:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 40, 2024-09-11T01:24:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 36170, 2024-09-11T01:24:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 36416, 2024-09-11T01:24:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 246, 2024-09-11T01:24:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 0, 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 0, 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 91160, 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 309501952 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 0, 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 0, 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 0, 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 0, 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 0, 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-09-11T01:24:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-09-11T01:24:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-09-11T01:24:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-09-11T01:24:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-09-11T01:24:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-09-11T01:24:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 1, 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 1, 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 0, 2024-09-11T01:24:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 1, 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 0, 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 0, 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 2605, 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-09-11T01:24:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T01:24:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T01:24:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 312343, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 2030, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-09-11T01:24:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 7976, 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 177, 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 28993820, 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 190081, 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-09-11T01:24:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 7393, 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 21, 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 3, 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 7393, 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 7393, 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 24.687642269, 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.003339326 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-09-11T01:24:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 7393, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 7394 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 7393, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 7393, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 0, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 4324, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 0, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 3067, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 0, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 0, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-09-11T01:24:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 1, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-09-11T01:24:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-09-11T01:24:26.479 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 115, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 40, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 11, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 12, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 1, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-09-11T01:24:26.480 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 0, 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 0 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 0, 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-09-11T01:24:26.481 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 0, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 7940, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 1203582448, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 7940, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-11T01:24:26.482 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 7940, 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1203582448, 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-11T01:24:26.483 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 7393, 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 1201007331, 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 7393, 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1201007331, 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-09-11T01:24:26.484 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 7393, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 7393, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 7393, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 7393, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.485 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.486 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:26.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.486 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-09-11T01:24:26.486 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-11T01:24:26.486 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-11T01:24:26.486 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-11T01:24:26.486 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-11T01:24:26.486 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-11T01:24:26.486 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:26.486 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x55d60a769040": { 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-11T01:24:26.487 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:26.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:26.489 INFO:teuthology.orchestra.run.smithi078.stdout: "trackedop": { 2024-09-11T01:24:26.489 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-09-11T01:24:26.489 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:26.489 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:24:27.357 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-11T01:24:27.357 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-11T01:24:27.357 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-11T01:24:27.357 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-11T01:24:27.357 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-11T01:24:27.357 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-11T01:24:27.357 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T01:24:27.357 DEBUG:tasks.check_counter:Getting stats from j 2024-09-11T01:24:27.358 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-11T01:24:27.603 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:27.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:27 smithi110 ceph-mon[34712]: pgmap v1128: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 20 op/s 2024-09-11T01:24:27.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:27 smithi112 ceph-mon[35384]: pgmap v1128: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 20 op/s 2024-09-11T01:24:27.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:27 smithi078 ceph-mon[31315]: pgmap v1128: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 20 op/s 2024-09-11T01:24:28.146 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T01:24:28.146 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T01:24:28.146 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 0, 2024-09-11T01:24:28.146 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2, 2024-09-11T01:24:28.146 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 0, 2024-09-11T01:24:28.146 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 0, 2024-09-11T01:24:28.146 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.001056076, 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000224021, 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:28.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 458, 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 451, 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 901408, 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 278480, 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.060295692, 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.031878391, 2024-09-11T01:24:28.148 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.035022022, 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 451, 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.026018774, 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000057691 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 901408, 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 278480 2024-09-11T01:24:28.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 2, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 343, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 320, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 221200, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.026849665, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.022909197, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001351429, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 341, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.014289340, 2024-09-11T01:24:28.150 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000041904 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 221200 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-09-11T01:24:28.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T01:24:28.152 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-09-11T01:24:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148721, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 30, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-09-11T01:24:28.154 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 968, 2024-09-11T01:24:28.155 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 456, 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 834199, 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 456, 2024-09-11T01:24:28.156 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-11T01:24:28.157 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-11T01:24:28.157 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 456, 2024-09-11T01:24:28.157 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 834199, 2024-09-11T01:24:28.157 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-11T01:24:28.157 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-11T01:24:28.157 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-11T01:24:28.157 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-11T01:24:28.157 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:28.157 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T01:24:28.157 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T01:24:28.959 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-11T01:24:28.959 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-11T01:24:28.959 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-11T01:24:28.959 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-11T01:24:28.959 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-11T01:24:28.960 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-11T01:24:28.960 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-11T01:24:28.960 DEBUG:tasks.check_counter:Getting stats from b 2024-09-11T01:24:28.960 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-11T01:24:29.289 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T01:24:29.917 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:29 smithi110 ceph-mon[34712]: pgmap v1129: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-11T01:24:29.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:29 smithi112 ceph-mon[35384]: pgmap v1129: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-11T01:24:29.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:29 smithi078 ceph-mon[31315]: pgmap v1129: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-11T01:24:30.057 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 619241, 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 737980, 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 171209597, 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 769438910, 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 239, 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 8, 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 40.198508749, 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 19.815309067, 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 21.142381845, 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 2.247077459, 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 737979, 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 28.758133645, 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000038968 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1246350, 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.195970050, 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000157 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 665258, 2024-09-11T01:24:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 805862, 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 213923063, 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 1098390229, 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 16, 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 8, 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 48.277333545, 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 26.158348706, 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 23.409145165, 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 2.928935796, 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 805858, 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 40.619394067, 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000050405 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1213756, 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.210369285, 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000173 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 1400896, 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 362816 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 666949, 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 764094, 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 191548065, 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 579878858, 2024-09-11T01:24:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 13, 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 6, 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 46.567595476, 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 24.156543484, 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 24.229283633, 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 2.309759093, 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 764095, 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 24.278612498, 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000031774 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1353284, 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.205255609, 2024-09-11T01:24:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000151 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 824752, 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 990912 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 169386, 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 146.739144915, 2024-09-11T01:24:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000866300 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 60554, 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 7.514540453, 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000124096 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 367022, 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 305856, 2024-09-11T01:24:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 305856, 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1540.201029123, 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.005035706 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 61166, 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 19, 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 2, 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 201, 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 15, 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 14, 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 1610, 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 746, 2024-09-11T01:24:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 834, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 30, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 517, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 5583, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 2, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 2, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 6, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 561701, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 330341, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 61094, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 248, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 2, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 3819, 2024-09-11T01:24:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 17, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 43, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 3921, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 36, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 1368, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 3437, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 4, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 234, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 7, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 3437, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-09-11T01:24:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 780, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 29, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 751, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 2657, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 42, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 100697, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 167053, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 214446, 2024-09-11T01:24:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 31318, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 11182, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 211997 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 6327, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 436, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 246, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 560, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 9840, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 23, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 32520, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 32497, 2024-09-11T01:24:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 29, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 46, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 29, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0, 2024-09-11T01:24:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_started": 0, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_succeeded": 0, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_write_failed": 0 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 237105, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 155070, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 147481, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 89624, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 7589, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 393, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 265, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 255, 2024-09-11T01:24:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 138, 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 8, 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 10, 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 879453396, 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 1419760002, 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 0, 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 88550, 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 634.803147486, 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.007168866 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 0 2024-09-11T01:24:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 879, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 36680, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 35801, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 78, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 612, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 534, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 1610, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 70651, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 69041, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 2, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 37809, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 37807, 2024-09-11T01:24:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 95952, 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 325963776 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 525082, 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 35436, 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 367022, 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 4302, 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 80896, 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 32945, 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 209.391016247, 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.006355775 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-09-11T01:24:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 42295, 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 251.276919820, 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.005941054 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-09-11T01:24:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 69596, 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 220.781435661, 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.003172329 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-09-11T01:24:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-09-11T01:24:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 26, 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.010916970, 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000419883 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 119, 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.780821570, 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.006561525 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 26, 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 16.979812358, 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.653069706 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.079 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 10, 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.023175321, 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002317532 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 121129, 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 415.316174683, 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.003428709 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 6339, 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 223.990975468, 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.035335380 2024-09-11T01:24:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 34, 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.005386072, 2024-09-11T01:24:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.029570178 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 19, 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 21.707142092, 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 1.142481162 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-09-11T01:24:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 26, 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.370736541, 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.014259097 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 788, 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.435511415, 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001821714 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 7, 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.017667718, 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002523959 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 32497, 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 177.113337187, 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.005450144 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-09-11T01:24:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 2, 2024-09-11T01:24:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 3, 2024-09-11T01:24:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 1, 2024-09-11T01:24:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 2, 2024-09-11T01:24:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-09-11T01:24:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 24074, 2024-09-11T01:24:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 12037, 2024-09-11T01:24:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 1532, 2024-09-11T01:24:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-09-11T01:24:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-11T01:24:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-11T01:24:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-11T01:24:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T01:24:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-11T01:24:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T01:24:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-11T01:24:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T01:24:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T01:24:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T01:24:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T01:24:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T01:24:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 542471, 2024-09-11T01:24:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 616, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 88, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 1, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 7976, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 177, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-11T01:24:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 4065726, 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 33143, 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 130373, 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 1425184966, 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 37, 2024-09-11T01:24:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 130373, 2024-09-11T01:24:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-09-11T01:24:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 130373, 2024-09-11T01:24:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1007.366871239, 2024-09-11T01:24:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.007726805 2024-09-11T01:24:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-09-11T01:24:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-09-11T01:24:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 130373, 2024-09-11T01:24:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 132984 2024-09-11T01:24:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 130373, 2024-09-11T01:24:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 4667, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 125706, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 28, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 184, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 1167, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 89688, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 662, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 15, 2024-09-11T01:24:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 33638, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 3473, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 368, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-09-11T01:24:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-09-11T01:24:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 115, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 40, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 12, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 12, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 2946, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 55, 2024-09-11T01:24:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 760 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 1, 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 6, 2024-09-11T01:24:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 32497, 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 32497, 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 1951444, 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 400921999, 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 1951444, 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T01:24:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1951444, 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 400921999, 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T01:24:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 130373, 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 1428476895, 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 130373, 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1428476895, 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-09-11T01:24:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 130373, 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 130373, 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 130373, 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 130373, 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 32497, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 3292621, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 32497, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1138, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 3292621, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x563360a2edc0": { 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 237105, 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 1415565698, 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 237105, 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T01:24:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 88550, 2024-09-11T01:24:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1415565698, 2024-09-11T01:24:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "trackedop": { 2024-09-11T01:24:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 2 2024-09-11T01:24:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:30.104 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-11T01:24:30.967 INFO:tasks.check_counter:Daemon mds.b mds.exported=43 2024-09-11T01:24:30.967 INFO:tasks.check_counter:Daemon mds.b mds.imported=36 2024-09-11T01:24:30.967 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=6327 2024-09-11T01:24:30.967 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=436 2024-09-11T01:24:30.967 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-09-11T01:24:30.967 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=26 2024-09-11T01:24:30.967 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=19 2024-09-11T01:24:30.967 DEBUG:tasks.check_counter:Getting stats from e 2024-09-11T01:24:30.968 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-11T01:24:31.216 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T01:24:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-11T01:24:31.743 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T01:24:31.743 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 4256, 2024-09-11T01:24:31.743 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 4255, 2024-09-11T01:24:31.743 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 564769892, 2024-09-11T01:24:31.743 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 1352273, 2024-09-11T01:24:31.743 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 8, 2024-09-11T01:24:31.743 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-09-11T01:24:31.743 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.699209706, 2024-09-11T01:24:31.743 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.156179275, 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.421394589, 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.094885453, 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 4254, 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.141257768, 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000033205 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.744 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 2169, 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 2108, 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 333088622, 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 798896, 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 6, 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.370803872, 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.086546057, 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.250724266, 2024-09-11T01:24:31.745 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.035397800, 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2107, 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.077416327, 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000036742 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 1316016, 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 285264 2024-09-11T01:24:31.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 1811, 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 2151, 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 449257815, 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 1267676, 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 6, 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.425483760, 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.075741447, 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.295557933, 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.040678424, 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:31.747 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2149, 2024-09-11T01:24:31.748 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.073754705, 2024-09-11T01:24:31.748 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000034320 2024-09-11T01:24:31.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.748 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:31.748 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.748 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.748 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.748 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T01:24:31.748 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 692096 2024-09-11T01:24:31.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.748 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 12013, 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.325177222, 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000027068 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T01:24:31.749 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T01:24:31.750 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.750 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.750 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.750 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:31.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.750 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-11T01:24:31.750 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 0, 2024-09-11T01:24:31.750 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 0, 2024-09-11T01:24:31.750 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-09-11T01:24:31.750 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.750 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 0, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 0, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 47823, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 23751, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 24072, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2024-09-11T01:24:31.751 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 13894, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 0, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 0, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 1, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 0, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 0, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 0, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2024-09-11T01:24:31.752 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T01:24:31.753 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 0, 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 0 2024-09-11T01:24:31.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 0, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 0, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 0, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 0, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-09-11T01:24:31.755 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0, 2024-09-11T01:24:31.756 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_started": 0, 2024-09-11T01:24:31.757 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_succeeded": 0, 2024-09-11T01:24:31.757 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_write_failed": 0 2024-09-11T01:24:31.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.757 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-09-11T01:24:31.757 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-09-11T01:24:31.757 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 0, 2024-09-11T01:24:31.757 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-09-11T01:24:31.757 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-09-11T01:24:31.757 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 221648, 2024-09-11T01:24:31.757 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 0, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 148, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 20, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 754042549, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 1347883440, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 1347855293, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.758 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.759 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.759 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 221648 2024-09-11T01:24:31.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.759 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-09-11T01:24:31.759 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 2067, 2024-09-11T01:24:31.759 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 47955, 2024-09-11T01:24:31.759 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 45888, 2024-09-11T01:24:31.759 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 62, 2024-09-11T01:24:31.759 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 114, 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 52, 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 47823, 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 48616, 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 793, 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 0, 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 0, 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 0, 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 101020, 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 323231744 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 0, 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2024-09-11T01:24:31.760 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 0, 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 0, 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.761 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.763 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.764 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-09-11T01:24:31.764 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.764 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.764 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.764 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-09-11T01:24:31.764 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.764 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.764 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.765 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-09-11T01:24:31.765 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.765 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.765 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.765 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-09-11T01:24:31.765 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.765 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-09-11T01:24:31.766 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.767 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.769 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.770 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.771 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.772 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 2, 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 3, 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 1, 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 2, 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 0, 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 0, 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 2844, 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-11T01:24:31.773 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T01:24:31.774 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-11T01:24:31.775 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 330228, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 2475, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-11T01:24:31.776 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 7976, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 177, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 37960202, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 246408, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-11T01:24:31.777 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 7676, 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 21, 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 3, 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 7676, 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 7676, 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 24.980574011, 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.003254373 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-09-11T01:24:31.778 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 7676, 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 7677 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 7676, 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 7676, 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 0, 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 4566, 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 0, 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 3108, 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 0, 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 0, 2024-09-11T01:24:31.779 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 1, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-09-11T01:24:31.780 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-09-11T01:24:31.781 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 115, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 40, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 11, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 11, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-09-11T01:24:31.782 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 0, 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 2, 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 0 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 0, 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-09-11T01:24:31.783 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 0, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 0, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 0, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 8232, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 1346344356, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 8232, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T01:24:31.784 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 8232, 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1346344356, 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T01:24:31.785 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 7676, 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 1343702543, 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 7676, 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1343702543, 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:31.786 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 7676, 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 7676, 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 7676, 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 7676, 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.787 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-11T01:24:31.788 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-11T01:24:31.789 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:31.789 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.789 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.789 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.789 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:31.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.789 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x55b4bed55040": { 2024-09-11T01:24:31.789 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:31.789 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T01:24:31.789 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:31.789 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:31.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:31.791 INFO:teuthology.orchestra.run.smithi110.stdout: "trackedop": { 2024-09-11T01:24:31.791 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-09-11T01:24:31.791 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:31.791 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-11T01:24:31.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:31 smithi110 ceph-mon[34712]: pgmap v1130: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-09-11T01:24:31.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:31 smithi112 ceph-mon[35384]: pgmap v1130: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-09-11T01:24:31.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:31 smithi078 ceph-mon[31315]: pgmap v1130: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-09-11T01:24:32.446 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-11T01:24:32.446 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-11T01:24:32.446 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-11T01:24:32.446 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-11T01:24:32.446 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-11T01:24:32.446 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-11T01:24:32.446 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T01:24:32.446 DEBUG:tasks.check_counter:Getting stats from h 2024-09-11T01:24:32.447 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-11T01:24:32.695 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T01:24:33.215 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-11T01:24:33.216 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T01:24:33.216 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 686400, 2024-09-11T01:24:33.216 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 850804, 2024-09-11T01:24:33.216 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 212523669, 2024-09-11T01:24:33.216 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 871927775, 2024-09-11T01:24:33.216 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 16, 2024-09-11T01:24:33.216 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2024-09-11T01:24:33.216 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 52.103550953, 2024-09-11T01:24:33.216 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 27.414023766, 2024-09-11T01:24:33.216 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 26.166140340, 2024-09-11T01:24:33.216 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 3.295861693, 2024-09-11T01:24:33.216 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:33.217 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 850803, 2024-09-11T01:24:33.217 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 56.812712265, 2024-09-11T01:24:33.217 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000066775 2024-09-11T01:24:33.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.217 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:33.217 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1356171, 2024-09-11T01:24:33.217 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.243966680, 2024-09-11T01:24:33.217 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000179 2024-09-11T01:24:33.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.217 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-11T01:24:33.217 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T01:24:33.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 252571, 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 264305, 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 66096364, 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 552000876, 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 14, 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 7, 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 20.538277947, 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 10.141895174, 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 9.267991357, 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 2.389762387, 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 264303, 2024-09-11T01:24:33.218 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 18.428876204, 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000069726 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 423265, 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.073796506, 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000174 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 695248 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 595069, 2024-09-11T01:24:33.219 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 562025, 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 180929902, 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 427095091, 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 62, 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 9, 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 40.982326308, 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 21.836816087, 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 22.301836638, 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 1.295964972, 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 562024, 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 28.157148426, 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000050099 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.220 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1083898, 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.158253623, 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000146 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 1331728, 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 316304 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T01:24:33.221 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 102367, 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 122.832468391, 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001199922 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 65875, 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 7.175009419, 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000108918 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.222 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-11T01:24:33.223 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 310245, 2024-09-11T01:24:33.223 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 255739, 2024-09-11T01:24:33.223 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-09-11T01:24:33.223 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 255739, 2024-09-11T01:24:33.223 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1118.908294984, 2024-09-11T01:24:33.223 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.004375196 2024-09-11T01:24:33.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.223 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-09-11T01:24:33.223 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 54505, 2024-09-11T01:24:33.223 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-09-11T01:24:33.223 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 56, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 2, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 1813, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 723, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 1060, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 30, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 779, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 938, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 1, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 1, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 6, 2024-09-11T01:24:33.224 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 510846, 2024-09-11T01:24:33.225 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 277875, 2024-09-11T01:24:33.225 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 54496, 2024-09-11T01:24:33.225 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 261, 2024-09-11T01:24:33.225 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-09-11T01:24:33.225 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-09-11T01:24:33.225 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 6786, 2024-09-11T01:24:33.225 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2024-09-11T01:24:33.225 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-09-11T01:24:33.225 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 9, 2024-09-11T01:24:33.225 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 549, 2024-09-11T01:24:33.225 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 16, 2024-09-11T01:24:33.225 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 1025, 2024-09-11T01:24:33.226 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-09-11T01:24:33.226 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 1819, 2024-09-11T01:24:33.226 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-09-11T01:24:33.226 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 4, 2024-09-11T01:24:33.226 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 234, 2024-09-11T01:24:33.226 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 7, 2024-09-11T01:24:33.226 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 1819, 2024-09-11T01:24:33.226 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-09-11T01:24:33.226 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T01:24:33.226 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-09-11T01:24:33.226 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 220, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 220, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 1599, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 24, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 44669, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 144678, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 149255, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 29642, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 10222, 2024-09-11T01:24:33.227 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 201191 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 1134, 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 3968, 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 2061, 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 3979, 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 3694, 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 20, 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 34997, 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 34977, 2024-09-11T01:24:33.228 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 9, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 2, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-09-11T01:24:33.229 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_started": 0, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_succeeded": 0, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_write_failed": 0 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 183242, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 118291, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 110611, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 72631, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 7680, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 298, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 170, 2024-09-11T01:24:33.230 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 155, 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 143, 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 6, 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 15, 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 474886130, 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 1051324405, 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 0, 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 55746, 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 604.401121377, 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.010842053 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 0 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.231 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 1465, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 37144, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 35679, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 69, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 2167, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 2098, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 1813, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 63810, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 61997, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 1, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 38819, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 38818, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 88304, 2024-09-11T01:24:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 297656320 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 458562, 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 2764, 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 310245, 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 5284, 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 16781, 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 34858, 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 203.764767218, 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.005845566 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 16099, 2024-09-11T01:24:33.233 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 230.127621860, 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.014294528 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-09-11T01:24:33.234 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 72478, 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 138.450119723, 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001910236 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1, 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000275155, 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000275155 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-09-11T01:24:33.235 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.236 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.236 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.236 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-09-11T01:24:33.236 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.237 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.237 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.238 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-09-11T01:24:33.238 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.238 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.238 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.238 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-09-11T01:24:33.238 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.238 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.238 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.239 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-09-11T01:24:33.239 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.239 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.239 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.239 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-09-11T01:24:33.239 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.240 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 86924, 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 388.498347453, 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.004469402 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 9331, 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 57.825867860, 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.006197177 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 20, 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.192410409, 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.009620520 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 103, 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 2.346506856, 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.022781619 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.243 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.244 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-09-11T01:24:33.244 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 948, 2024-09-11T01:24:33.244 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.189066462, 2024-09-11T01:24:33.244 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001254289 2024-09-11T01:24:33.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.244 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-09-11T01:24:33.244 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.244 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.244 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 34977, 2024-09-11T01:24:33.245 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 96.513311988, 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002759336 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 1, 2024-09-11T01:24:33.246 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 0, 2024-09-11T01:24:33.247 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2024-09-11T01:24:33.247 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-09-11T01:24:33.247 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 25746, 2024-09-11T01:24:33.247 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 25746, 2024-09-11T01:24:33.247 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 1540, 2024-09-11T01:24:33.247 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-09-11T01:24:33.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.247 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-11T01:24:33.247 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-11T01:24:33.247 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-11T01:24:33.248 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-11T01:24:33.249 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-11T01:24:33.249 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-11T01:24:33.249 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T01:24:33.249 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T01:24:33.249 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T01:24:33.249 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T01:24:33.249 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T01:24:33.249 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-11T01:24:33.249 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-11T01:24:33.250 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-11T01:24:33.250 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T01:24:33.250 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T01:24:33.250 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-11T01:24:33.250 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-11T01:24:33.250 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-11T01:24:33.251 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-11T01:24:33.251 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T01:24:33.251 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-11T01:24:33.251 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T01:24:33.251 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-11T01:24:33.251 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 525268, 2024-09-11T01:24:33.251 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 878, 2024-09-11T01:24:33.251 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 264, 2024-09-11T01:24:33.251 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 3, 2024-09-11T01:24:33.251 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-11T01:24:33.251 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-11T01:24:33.251 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-11T01:24:33.252 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-11T01:24:33.252 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-11T01:24:33.252 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-11T01:24:33.252 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 7976, 2024-09-11T01:24:33.252 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 177, 2024-09-11T01:24:33.252 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T01:24:33.252 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-11T01:24:33.252 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-11T01:24:33.252 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-11T01:24:33.252 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 5226244, 2024-09-11T01:24:33.252 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 38692, 2024-09-11T01:24:33.253 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-11T01:24:33.253 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-11T01:24:33.253 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-11T01:24:33.253 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-11T01:24:33.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.253 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-09-11T01:24:33.253 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-09-11T01:24:33.253 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-09-11T01:24:33.253 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 98418, 2024-09-11T01:24:33.253 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 1055124829, 2024-09-11T01:24:33.254 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 28, 2024-09-11T01:24:33.254 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 98418, 2024-09-11T01:24:33.254 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-09-11T01:24:33.254 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 98418, 2024-09-11T01:24:33.254 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1001.413151191, 2024-09-11T01:24:33.254 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.010175101 2024-09-11T01:24:33.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.254 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-09-11T01:24:33.254 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-09-11T01:24:33.254 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 98418, 2024-09-11T01:24:33.254 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 100856 2024-09-11T01:24:33.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.255 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 98418, 2024-09-11T01:24:33.255 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 3405, 2024-09-11T01:24:33.255 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 95013, 2024-09-11T01:24:33.255 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-09-11T01:24:33.255 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-09-11T01:24:33.255 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 8, 2024-09-11T01:24:33.255 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 133, 2024-09-11T01:24:33.255 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 1586, 2024-09-11T01:24:33.255 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 57332, 2024-09-11T01:24:33.255 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 895, 2024-09-11T01:24:33.256 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-09-11T01:24:33.256 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-09-11T01:24:33.256 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-09-11T01:24:33.256 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 2, 2024-09-11T01:24:33.256 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 35350, 2024-09-11T01:24:33.256 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-09-11T01:24:33.256 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-09-11T01:24:33.256 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-09-11T01:24:33.256 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 1819, 2024-09-11T01:24:33.256 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 266, 2024-09-11T01:24:33.256 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-09-11T01:24:33.256 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-09-11T01:24:33.257 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-09-11T01:24:33.257 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-09-11T01:24:33.257 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-09-11T01:24:33.257 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-09-11T01:24:33.257 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T01:24:33.257 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-09-11T01:24:33.258 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-09-11T01:24:33.258 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-09-11T01:24:33.258 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-09-11T01:24:33.258 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-09-11T01:24:33.258 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-09-11T01:24:33.258 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-09-11T01:24:33.258 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-09-11T01:24:33.258 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-09-11T01:24:33.258 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-09-11T01:24:33.258 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-09-11T01:24:33.258 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-09-11T01:24:33.259 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-09-11T01:24:33.259 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-09-11T01:24:33.259 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-09-11T01:24:33.259 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-09-11T01:24:33.259 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-09-11T01:24:33.259 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-09-11T01:24:33.259 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-09-11T01:24:33.259 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 115, 2024-09-11T01:24:33.260 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-09-11T01:24:33.260 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 40, 2024-09-11T01:24:33.260 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 12, 2024-09-11T01:24:33.260 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 12, 2024-09-11T01:24:33.260 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-09-11T01:24:33.260 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-09-11T01:24:33.260 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 2543, 2024-09-11T01:24:33.260 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 0, 2024-09-11T01:24:33.260 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 922 2024-09-11T01:24:33.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.261 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-09-11T01:24:33.261 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 1, 2024-09-11T01:24:33.261 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-09-11T01:24:33.261 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-09-11T01:24:33.261 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-09-11T01:24:33.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.261 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-09-11T01:24:33.261 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-09-11T01:24:33.261 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T01:24:33.261 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-09-11T01:24:33.262 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 6, 2024-09-11T01:24:33.262 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 34977, 2024-09-11T01:24:33.262 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 34977, 2024-09-11T01:24:33.262 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-09-11T01:24:33.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.262 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T01:24:33.262 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:33.262 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T01:24:33.262 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:33.262 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 1534036, 2024-09-11T01:24:33.262 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 321452123, 2024-09-11T01:24:33.263 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:33.263 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 1534036, 2024-09-11T01:24:33.263 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T01:24:33.263 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T01:24:33.263 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1534036, 2024-09-11T01:24:33.263 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 321452123, 2024-09-11T01:24:33.263 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:33.263 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.263 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.263 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.264 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:33.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.264 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-09-11T01:24:33.264 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:33.264 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T01:24:33.264 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:33.264 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T01:24:33.264 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T01:24:33.264 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:33.264 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:33.265 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 98418, 2024-09-11T01:24:33.265 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 1058666330, 2024-09-11T01:24:33.265 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 98418, 2024-09-11T01:24:33.265 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1058666330, 2024-09-11T01:24:33.265 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:33.265 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.265 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.265 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.265 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:33.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.266 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-09-11T01:24:33.266 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:33.266 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-09-11T01:24:33.266 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:33.266 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T01:24:33.266 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T01:24:33.266 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:33.266 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:33.266 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 98418, 2024-09-11T01:24:33.266 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 98418, 2024-09-11T01:24:33.266 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 98418, 2024-09-11T01:24:33.267 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 98418, 2024-09-11T01:24:33.267 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:33.267 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.267 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.267 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.267 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:33.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.267 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-09-11T01:24:33.267 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:33.267 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T01:24:33.267 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:33.267 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 34977, 2024-09-11T01:24:33.268 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 3541501, 2024-09-11T01:24:33.268 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:33.268 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 34977, 2024-09-11T01:24:33.268 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T01:24:33.268 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T01:24:33.268 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1586, 2024-09-11T01:24:33.268 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 3541501, 2024-09-11T01:24:33.268 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:33.268 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.268 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.268 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:33.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.269 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x55780831cdc0": { 2024-09-11T01:24:33.269 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:33.269 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T01:24:33.269 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:33.269 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 183242, 2024-09-11T01:24:33.269 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 1047130101, 2024-09-11T01:24:33.269 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:33.269 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 183242, 2024-09-11T01:24:33.269 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T01:24:33.269 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T01:24:33.269 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 55746, 2024-09-11T01:24:33.270 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1047130101, 2024-09-11T01:24:33.270 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:33.270 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:33.270 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:33.270 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:33.270 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:33.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:33.270 INFO:teuthology.orchestra.run.smithi110.stdout: "trackedop": { 2024-09-11T01:24:33.270 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-09-11T01:24:33.270 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:33.270 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-11T01:24:33.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:33 smithi110 ceph-mon[34712]: pgmap v1131: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-09-11T01:24:33.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:33 smithi112 ceph-mon[35384]: pgmap v1131: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-09-11T01:24:33.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:33 smithi078 ceph-mon[31315]: pgmap v1131: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-09-11T01:24:34.099 INFO:tasks.check_counter:Daemon mds.h mds.exported=9 2024-09-11T01:24:34.099 INFO:tasks.check_counter:Daemon mds.h mds.imported=16 2024-09-11T01:24:34.100 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=1134 2024-09-11T01:24:34.100 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=3968 2024-09-11T01:24:34.100 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-09-11T01:24:34.100 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-11T01:24:34.100 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T01:24:34.100 DEBUG:tasks.check_counter:Getting stats from k 2024-09-11T01:24:34.100 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-11T01:24:34.345 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T01:24:34.953 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-11T01:24:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T01:24:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 3694, 2024-09-11T01:24:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 3696, 2024-09-11T01:24:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 423270414, 2024-09-11T01:24:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 1174693, 2024-09-11T01:24:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 8, 2024-09-11T01:24:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-09-11T01:24:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.552493888, 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.143802242, 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.299270499, 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.085799581, 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3694, 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.130993515, 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000035461 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 2181, 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 2522, 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 399032028, 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 1386756, 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 7, 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 6, 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.438629329, 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.081466139, 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.292513293, 2024-09-11T01:24:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.049124597, 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2521, 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.084470880, 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000033506 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 694720 2024-09-11T01:24:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 1836, 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 1775, 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 228141252, 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 693906, 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 5, 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.292422522, 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.075881402, 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.190865649, 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.029761892, 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1774, 2024-09-11T01:24:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.063832415, 2024-09-11T01:24:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000035982 2024-09-11T01:24:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 1316832, 2024-09-11T01:24:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 287184 2024-09-11T01:24:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-11T01:24:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 11450, 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.283275259, 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000024740 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-11T01:24:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 0, 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 0, 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 0, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 0, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 36832, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 14850, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 21982, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 15617, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2024-09-11T01:24:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 1, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2024-09-11T01:24:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T01:24:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 0, 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 0, 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 0, 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 0, 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 0 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-09-11T01:24:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-09-11T01:24:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_started": 0, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_succeeded": 0, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_write_failed": 0 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 0, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 183242, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 0, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 143, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 16, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-09-11T01:24:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 474886130, 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 1051324405, 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 1051311112, 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 183242 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 1506, 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 36951, 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 35445, 2024-09-11T01:24:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 52, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 92, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 40, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 36832, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 37396, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 564, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 0, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 0, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 0, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 90812, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 307961856 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 0, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2024-09-11T01:24:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 0, 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 0, 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-09-11T01:24:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-09-11T01:24:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-09-11T01:24:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-09-11T01:24:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-09-11T01:24:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-09-11T01:24:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-09-11T01:24:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 1, 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 0, 2024-09-11T01:24:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 0, 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 0, 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 2845, 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T01:24:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T01:24:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-11T01:24:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 304713, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 1804, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 7976, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 177, 2024-09-11T01:24:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 29060530, 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 187395, 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-09-11T01:24:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 7149, 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 21, 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 3, 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 7149, 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 7149, 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 24.515592791, 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.003429233 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 7149, 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 7150 2024-09-11T01:24:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 7149, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 7149, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 0, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 4102, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 0, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 3045, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 0, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 0, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-09-11T01:24:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 0, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 1, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 0, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-09-11T01:24:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-09-11T01:24:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 115, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 40, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 11, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 11, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 0, 2024-09-11T01:24:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 2, 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 0 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 0, 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 0, 2024-09-11T01:24:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 0, 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 0, 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 7707, 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 1049718815, 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 7707, 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T01:24:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 7707, 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1049718815, 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 7149, 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 1047167082, 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 7149, 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1047167082, 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 7149, 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 7149, 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 7149, 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 7149, 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x55c14ac33040": { 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-11T01:24:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-11T01:24:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-11T01:24:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-11T01:24:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T01:24:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: "trackedop": { 2024-09-11T01:24:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-09-11T01:24:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T01:24:34.999 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-11T01:24:35.803 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-11T01:24:35.803 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-11T01:24:35.803 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-11T01:24:35.804 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-11T01:24:35.804 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-11T01:24:35.804 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-11T01:24:35.804 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T01:24:35.804 DEBUG:tasks.check_counter:Getting stats from c 2024-09-11T01:24:35.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-11T01:24:35.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:35 smithi110 ceph-mon[34712]: pgmap v1132: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-09-11T01:24:35.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:35 smithi112 ceph-mon[35384]: pgmap v1132: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-09-11T01:24:35.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:35 smithi078 ceph-mon[31315]: pgmap v1132: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-09-11T01:24:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T01:24:36.795 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-11T01:24:36.795 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T01:24:36.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 593980, 2024-09-11T01:24:36.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 524688, 2024-09-11T01:24:36.795 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 190826604, 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 368056095, 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 63, 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 6, 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 35.646431196, 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 17.057382019, 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 21.034521300, 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 1.616581264, 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 524691, 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 38.055972011, 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000072530 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.796 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1054771, 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.168088899, 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000159 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1337536, 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 322448 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 546384, 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 659648, 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 171591154, 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 1046817966, 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 11, 2024-09-11T01:24:36.797 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 8, 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 44.527340070, 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 23.508322776, 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 20.973589896, 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 3.604043433, 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 659644, 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 51.946829577, 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000078749 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1020912, 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.212112760, 2024-09-11T01:24:36.798 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000207 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 4896, 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 726128 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 397634, 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 404586, 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 111787075, 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 444342520, 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 19, 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 8, 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 30.470239356, 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 15.663549421, 2024-09-11T01:24:36.799 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 14.886792806, 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 2.166064118, 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 404583, 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 31.040524483, 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000076722 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 708897, 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.130141815, 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000183 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-11T01:24:36.800 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 109260, 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 128.325348674, 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001174495 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.801 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 66518, 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 7.463836000, 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000112207 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 316407, 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 262062, 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 262062, 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1122.891284095, 2024-09-11T01:24:36.802 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004284830 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 54345, 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 1, 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 66, 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 2, 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 1778, 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 737, 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 1011, 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 30, 2024-09-11T01:24:36.803 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 725, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 742, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 1, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 2, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 6, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 514350, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 284003, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 54338, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 250, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 6542, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 1000, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 1, 2024-09-11T01:24:36.804 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 8, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 373, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 8, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 701, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 1975, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 4, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 234, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 7, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 1975, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-09-11T01:24:36.805 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 220, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 220, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 1755, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 32, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 29972, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 135936, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 152936, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 34907, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 8822, 2024-09-11T01:24:36.806 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 216764 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 817, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 5805, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 2980, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 5759, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 995, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 20, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 36169, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 36149, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-09-11T01:24:36.807 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 8, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 2, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_succeeded": 0, 2024-09-11T01:24:36.808 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_write_failed": 0 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 186685, 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 128646, 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 122901, 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 63784, 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 5745, 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 317, 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 189, 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 177, 2024-09-11T01:24:36.809 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 140, 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 6, 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 12, 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 548533766, 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 1107885172, 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 0, 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 59785, 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 601.499755701, 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.010061048 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 0 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.810 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1414, 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 38116, 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 36702, 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 69, 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 3085, 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 3016, 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 1778, 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 60835, 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 59057, 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 2, 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 40208, 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 40206, 2024-09-11T01:24:36.811 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 92028, 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 297656320 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 456872, 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 704, 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 316407, 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 7003, 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 10871, 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 36036, 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 221.148204164, 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.006136868 2024-09-11T01:24:36.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.813 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-09-11T01:24:36.813 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 22490, 2024-09-11T01:24:36.813 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 300.350195830, 2024-09-11T01:24:36.813 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.013354833 2024-09-11T01:24:36.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.813 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-09-11T01:24:36.813 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.813 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.813 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 62979, 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 136.672757534, 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002170132 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.814 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1, 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000162372, 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000162372 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-09-11T01:24:36.815 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.816 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.816 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.816 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-09-11T01:24:36.817 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.817 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.817 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.817 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-09-11T01:24:36.817 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.817 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.817 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.817 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-09-11T01:24:36.817 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.817 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-09-11T01:24:36.818 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 94244, 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 298.615678669, 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003168537 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 8185, 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 47.566782028, 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.005811457 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.819 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 20, 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.302406248, 2024-09-11T01:24:36.820 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.015120312 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 58, 2024-09-11T01:24:36.821 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.569628855, 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.027062566 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1900, 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.571556047, 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000827134 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.822 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 36149, 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 115.093912348, 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003183875 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:36.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 2, 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 2, 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 0, 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 2, 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 24481, 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 12240, 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 1532, 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-09-11T01:24:36.825 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-09-11T01:24:36.826 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-09-11T01:24:36.827 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 498264, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 776, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 176, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 2, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-09-11T01:24:36.828 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 7976, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 177, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 5177188, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 37097, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-09-11T01:24:36.829 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 103432, 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 1112252987, 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 28, 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 103432, 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 103432, 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1013.490944611, 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.009798620 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.830 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 103432, 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 105922 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 103432, 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 3400, 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 100032, 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 17, 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 153, 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 1423, 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 61208, 2024-09-11T01:24:36.831 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 801, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 2, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 36553, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 1975, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 306, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-09-11T01:24:36.832 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-09-11T01:24:36.833 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 115, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 40, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 12, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 12, 2024-09-11T01:24:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 2572, 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 3, 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 909 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 1, 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T01:24:36.835 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 6, 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 36149, 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 36149, 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 1537994, 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 335749607, 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 1537994, 2024-09-11T01:24:36.836 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1537994, 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 335749607, 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:36.837 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 103432, 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 1115912644, 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 103432, 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1115912644, 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 103432, 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 103432, 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 103432, 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 103432, 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.839 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 36149, 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 3659705, 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 36149, 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-11T01:24:36.840 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1423, 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 3659705, 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x563d419ecdc0": { 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 186685, 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1103690868, 2024-09-11T01:24:36.841 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 186685, 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 59785, 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1103690868, 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: "trackedop": { 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:36.842 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-11T01:24:37.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:37 smithi110 ceph-mon[34712]: pgmap v1133: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 19 op/s 2024-09-11T01:24:37.973 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:37 smithi112 ceph-mon[35384]: pgmap v1133: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 19 op/s 2024-09-11T01:24:37.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:37 smithi078 ceph-mon[31315]: pgmap v1133: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 19 op/s 2024-09-11T01:24:38.033 INFO:tasks.check_counter:Daemon mds.c mds.exported=8 2024-09-11T01:24:38.033 INFO:tasks.check_counter:Daemon mds.c mds.imported=8 2024-09-11T01:24:38.033 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=817 2024-09-11T01:24:38.033 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=5805 2024-09-11T01:24:38.033 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-09-11T01:24:38.033 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-11T01:24:38.033 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T01:24:38.033 DEBUG:tasks.check_counter:Getting stats from f 2024-09-11T01:24:38.033 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-11T01:24:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T01:24:38.983 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-11T01:24:38.983 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T01:24:38.983 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 3826, 2024-09-11T01:24:38.983 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 3828, 2024-09-11T01:24:38.983 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 500250090, 2024-09-11T01:24:38.983 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 1216678, 2024-09-11T01:24:38.983 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 8, 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 4, 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.608511917, 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.142403500, 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.347486167, 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.092387128, 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3826, 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.131885158, 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000034470 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T01:24:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T01:24:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T01:24:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2442, 2024-09-11T01:24:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2379, 2024-09-11T01:24:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 456874422, 2024-09-11T01:24:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 885054, 2024-09-11T01:24:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-09-11T01:24:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.469023167, 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.098823156, 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.328481608, 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.042016026, 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2379, 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.093167567, 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000039162 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1319440, 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 286512 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1954, 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2294, 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 461878345, 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 1316289, 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.512651889, 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.087811053, 2024-09-11T01:24:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.287645895, 2024-09-11T01:24:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.122709287, 2024-09-11T01:24:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2292, 2024-09-11T01:24:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.082780637, 2024-09-11T01:24:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000036117 2024-09-11T01:24:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T01:24:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 695552 2024-09-11T01:24:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-09-11T01:24:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-09-11T01:24:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-09-11T01:24:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-09-11T01:24:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-11T01:24:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-11T01:24:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 12083, 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.310598383, 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000025705 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-11T01:24:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 0, 2024-09-11T01:24:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 0, 2024-09-11T01:24:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-09-11T01:24:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-09-11T01:24:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-09-11T01:24:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-09-11T01:24:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-09-11T01:24:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 0, 2024-09-11T01:24:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-09-11T01:24:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-09-11T01:24:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 33619, 2024-09-11T01:24:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 14002, 2024-09-11T01:24:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 19617, 2024-09-11T01:24:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-09-11T01:24:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 13616, 2024-09-11T01:24:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-09-11T01:24:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 2, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 0, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 0, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-09-11T01:24:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 4, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 234, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 7, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-09-11T01:24:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-09-11T01:24:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T01:24:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T01:24:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-09-11T01:24:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-09-11T01:24:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T01:24:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-09-11T01:24:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T01:24:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T01:24:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T01:24:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 0, 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 0 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-09-11T01:24:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-09-11T01:24:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-09-11T01:24:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-09-11T01:24:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-09-11T01:24:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-09-11T01:24:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-09-11T01:24:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-09-11T01:24:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-09-11T01:24:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-09-11T01:24:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-09-11T01:24:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0, 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_succeeded": 0, 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_write_failed": 0 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-09-11T01:24:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 0, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 237105, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 0, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 138, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 21, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 879453396, 2024-09-11T01:24:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 1419760002, 2024-09-11T01:24:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 1419737895, 2024-09-11T01:24:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-09-11T01:24:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 237105 2024-09-11T01:24:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-09-11T01:24:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 856, 2024-09-11T01:24:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 36234, 2024-09-11T01:24:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 35378, 2024-09-11T01:24:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 55, 2024-09-11T01:24:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 296, 2024-09-11T01:24:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 241, 2024-09-11T01:24:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 33619, 2024-09-11T01:24:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 37469, 2024-09-11T01:24:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 3850, 2024-09-11T01:24:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-09-11T01:24:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 0, 2024-09-11T01:24:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 0, 2024-09-11T01:24:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 90956, 2024-09-11T01:24:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 313696256 2024-09-11T01:24:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-09-11T01:24:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 0, 2024-09-11T01:24:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-09-11T01:24:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 0, 2024-09-11T01:24:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 0, 2024-09-11T01:24:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-09-11T01:24:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-09-11T01:24:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-09-11T01:24:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-09-11T01:24:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.004 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-09-11T01:24:39.004 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.004 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.004 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.004 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-09-11T01:24:39.005 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.005 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.005 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.005 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-09-11T01:24:39.005 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-09-11T01:24:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-09-11T01:24:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-09-11T01:24:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-09-11T01:24:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-09-11T01:24:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-09-11T01:24:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-09-11T01:24:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-09-11T01:24:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-09-11T01:24:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-09-11T01:24:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-09-11T01:24:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-09-11T01:24:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-09-11T01:24:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-09-11T01:24:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-09-11T01:24:39.016 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.016 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.016 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.016 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-09-11T01:24:39.016 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.017 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.017 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.017 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-09-11T01:24:39.017 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.017 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.018 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.018 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-09-11T01:24:39.018 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-09-11T01:24:39.018 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-09-11T01:24:39.018 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.019 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.019 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.019 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:39.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.019 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-09-11T01:24:39.019 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 2, 2024-09-11T01:24:39.020 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 3, 2024-09-11T01:24:39.020 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 1, 2024-09-11T01:24:39.020 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 2, 2024-09-11T01:24:39.020 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-09-11T01:24:39.020 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 0, 2024-09-11T01:24:39.020 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 0, 2024-09-11T01:24:39.021 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 2845, 2024-09-11T01:24:39.021 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-09-11T01:24:39.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.021 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-09-11T01:24:39.021 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-09-11T01:24:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-09-11T01:24:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T01:24:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-09-11T01:24:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T01:24:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-09-11T01:24:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T01:24:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T01:24:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T01:24:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T01:24:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T01:24:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-09-11T01:24:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-09-11T01:24:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 278540, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 1135, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-09-11T01:24:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 7976, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 177, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 25484126, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 160813, 2024-09-11T01:24:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 7660, 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 26, 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 4, 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 7660, 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7660, 2024-09-11T01:24:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 24.550364241, 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003205008 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7660, 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 7661 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 7660, 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 7660, 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 0, 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 4414, 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 0, 2024-09-11T01:24:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 3243, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 0, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 0, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 0, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 2, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 0, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-09-11T01:24:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-09-11T01:24:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 115, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 40, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 11, 2024-09-11T01:24:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 11, 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 0, 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 2, 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 0 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 0, 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-09-11T01:24:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 8218, 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1418232046, 2024-09-11T01:24:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 8218, 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 8218, 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1418232046, 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 7660, 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 1415589877, 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 7660, 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1415589877, 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 7660, 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 7660, 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 7660, 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 7660, 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-09-11T01:24:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-11T01:24:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-11T01:24:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-11T01:24:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x564b870e9040": { 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-11T01:24:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:39.039 INFO:teuthology.orchestra.run.smithi112.stdout: "trackedop": { 2024-09-11T01:24:39.039 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-09-11T01:24:39.039 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:39.039 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-11T01:24:39.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:39 smithi110 ceph-mon[34712]: pgmap v1134: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-11T01:24:39.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:39 smithi112 ceph-mon[35384]: pgmap v1134: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-11T01:24:39.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:39 smithi078 ceph-mon[31315]: pgmap v1134: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-11T01:24:40.029 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-11T01:24:40.029 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-11T01:24:40.030 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-11T01:24:40.030 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-11T01:24:40.030 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-09-11T01:24:40.030 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-11T01:24:40.030 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T01:24:40.030 DEBUG:tasks.check_counter:Getting stats from i 2024-09-11T01:24:40.031 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-11T01:24:40.291 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T01:24:40.835 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-11T01:24:40.835 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T01:24:40.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 779293, 2024-09-11T01:24:40.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 738177, 2024-09-11T01:24:40.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 242989607, 2024-09-11T01:24:40.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 596314719, 2024-09-11T01:24:40.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 64, 2024-09-11T01:24:40.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-09-11T01:24:40.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 48.068379160, 2024-09-11T01:24:40.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 23.667001636, 2024-09-11T01:24:40.837 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 27.524132627, 2024-09-11T01:24:40.837 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 2.318424572, 2024-09-11T01:24:40.837 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:40.837 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 738181, 2024-09-11T01:24:40.837 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 63.064197714, 2024-09-11T01:24:40.837 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000085431 2024-09-11T01:24:40.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.838 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:40.838 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1421251, 2024-09-11T01:24:40.838 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.226475649, 2024-09-11T01:24:40.838 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000159 2024-09-11T01:24:40.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.838 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T01:24:40.838 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T01:24:40.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 608339, 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 798291, 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 193945976, 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 862836745, 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 18, 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 10, 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 48.143446324, 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 26.787692701, 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 23.828420560, 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 1.766854553, 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:40.839 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 798284, 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 65.683099599, 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000082280 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1181324, 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.238722868, 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000202 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1347360, 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 351408 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T01:24:40.840 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 241351, 2024-09-11T01:24:40.841 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 245615, 2024-09-11T01:24:40.841 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 79512294, 2024-09-11T01:24:40.841 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 677836974, 2024-09-11T01:24:40.841 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 12, 2024-09-11T01:24:40.841 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 7, 2024-09-11T01:24:40.841 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 23.948265785, 2024-09-11T01:24:40.841 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 13.014619317, 2024-09-11T01:24:40.841 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 9.269688459, 2024-09-11T01:24:40.841 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 2.766443917, 2024-09-11T01:24:40.841 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:40.841 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 245614, 2024-09-11T01:24:40.841 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 19.397918499, 2024-09-11T01:24:40.842 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000078977 2024-09-11T01:24:40.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.842 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:40.842 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 376747, 2024-09-11T01:24:40.842 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.074251332, 2024-09-11T01:24:40.842 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000197 2024-09-11T01:24:40.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.843 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-11T01:24:40.843 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 699232 2024-09-11T01:24:40.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.843 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-09-11T01:24:40.843 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-09-11T01:24:40.843 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-09-11T01:24:40.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.844 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-09-11T01:24:40.844 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-11T01:24:40.844 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-11T01:24:40.844 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 116644, 2024-09-11T01:24:40.844 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 160.670540570, 2024-09-11T01:24:40.844 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001377443 2024-09-11T01:24:40.845 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:40.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.845 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-09-11T01:24:40.845 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-11T01:24:40.845 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-11T01:24:40.845 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 83489, 2024-09-11T01:24:40.845 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 10.434250417, 2024-09-11T01:24:40.846 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000124977 2024-09-11T01:24:40.846 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:40.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.846 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-11T01:24:40.846 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 355552, 2024-09-11T01:24:40.846 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 303629, 2024-09-11T01:24:40.846 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-09-11T01:24:40.846 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 303629, 2024-09-11T01:24:40.847 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1581.830356732, 2024-09-11T01:24:40.847 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.005209747 2024-09-11T01:24:40.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.847 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-09-11T01:24:40.847 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 51921, 2024-09-11T01:24:40.847 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 6, 2024-09-11T01:24:40.848 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-09-11T01:24:40.848 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 103, 2024-09-11T01:24:40.848 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 3, 2024-09-11T01:24:40.848 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 1, 2024-09-11T01:24:40.848 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 2687, 2024-09-11T01:24:40.848 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 1288, 2024-09-11T01:24:40.849 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 1344, 2024-09-11T01:24:40.849 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 55, 2024-09-11T01:24:40.849 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 836, 2024-09-11T01:24:40.849 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 1442, 2024-09-11T01:24:40.849 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-09-11T01:24:40.849 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-09-11T01:24:40.849 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 6, 2024-09-11T01:24:40.850 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 568658, 2024-09-11T01:24:40.850 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 296580, 2024-09-11T01:24:40.850 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 51906, 2024-09-11T01:24:40.850 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 346, 2024-09-11T01:24:40.850 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 2, 2024-09-11T01:24:40.850 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-09-11T01:24:40.850 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 4113, 2024-09-11T01:24:40.850 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 1000, 2024-09-11T01:24:40.850 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 1, 2024-09-11T01:24:40.850 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 15, 2024-09-11T01:24:40.850 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 753, 2024-09-11T01:24:40.850 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 22, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 1725, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 2259, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 4, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 234, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 7, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 2259, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 330, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T01:24:40.851 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 330, 2024-09-11T01:24:40.852 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 1929, 2024-09-11T01:24:40.852 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 45, 2024-09-11T01:24:40.852 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 69287, 2024-09-11T01:24:40.852 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 191026, 2024-09-11T01:24:40.852 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T01:24:40.852 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T01:24:40.852 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T01:24:40.852 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 185726, 2024-09-11T01:24:40.852 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 30587, 2024-09-11T01:24:40.852 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 8943, 2024-09-11T01:24:40.852 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 232888 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 532, 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 3615, 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 1910, 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 3574, 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 4271, 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 30, 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 45256, 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 45226, 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-09-11T01:24:40.853 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-09-11T01:24:40.854 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-09-11T01:24:40.854 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-09-11T01:24:40.854 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-09-11T01:24:40.854 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-09-11T01:24:40.854 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-09-11T01:24:40.854 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-09-11T01:24:40.854 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-09-11T01:24:40.854 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T01:24:40.854 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 15, 2024-09-11T01:24:40.855 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-09-11T01:24:40.855 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 4, 2024-09-11T01:24:40.855 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-09-11T01:24:40.855 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0, 2024-09-11T01:24:40.855 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-11T01:24:40.855 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_succeeded": 0, 2024-09-11T01:24:40.855 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_write_failed": 0 2024-09-11T01:24:40.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.856 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-09-11T01:24:40.856 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-09-11T01:24:40.856 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 221648, 2024-09-11T01:24:40.856 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 169742, 2024-09-11T01:24:40.856 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 159771, 2024-09-11T01:24:40.856 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 61877, 2024-09-11T01:24:40.856 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-09-11T01:24:40.856 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 9971, 2024-09-11T01:24:40.856 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 385, 2024-09-11T01:24:40.857 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 257, 2024-09-11T01:24:40.857 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 237, 2024-09-11T01:24:40.857 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 148, 2024-09-11T01:24:40.857 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 6, 2024-09-11T01:24:40.857 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-09-11T01:24:40.857 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 20, 2024-09-11T01:24:40.857 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 754042549, 2024-09-11T01:24:40.857 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 1347883440, 2024-09-11T01:24:40.857 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 0, 2024-09-11T01:24:40.858 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-09-11T01:24:40.858 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 66005, 2024-09-11T01:24:40.858 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 796.206431919, 2024-09-11T01:24:40.858 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.012062819 2024-09-11T01:24:40.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.858 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 0 2024-09-11T01:24:40.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.858 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-09-11T01:24:40.859 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 2061, 2024-09-11T01:24:40.859 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 48280, 2024-09-11T01:24:40.859 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 46219, 2024-09-11T01:24:40.859 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 81, 2024-09-11T01:24:40.859 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 1956, 2024-09-11T01:24:40.859 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 1875, 2024-09-11T01:24:40.859 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 2687, 2024-09-11T01:24:40.859 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 84054, 2024-09-11T01:24:40.860 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 81367, 2024-09-11T01:24:40.860 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-09-11T01:24:40.860 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 50069, 2024-09-11T01:24:40.860 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 50069, 2024-09-11T01:24:40.860 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 96608, 2024-09-11T01:24:40.860 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 301846528 2024-09-11T01:24:40.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.860 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-09-11T01:24:40.861 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 498234, 2024-09-11T01:24:40.861 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 9283, 2024-09-11T01:24:40.861 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 355552, 2024-09-11T01:24:40.861 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 5118, 2024-09-11T01:24:40.861 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 27877, 2024-09-11T01:24:40.861 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-09-11T01:24:40.861 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 45168, 2024-09-11T01:24:40.861 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 256.583442809, 2024-09-11T01:24:40.862 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.005680646 2024-09-11T01:24:40.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.862 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-09-11T01:24:40.862 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 20493, 2024-09-11T01:24:40.862 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 433.506806603, 2024-09-11T01:24:40.862 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.021153896 2024-09-11T01:24:40.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.862 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-09-11T01:24:40.862 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.863 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.863 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.863 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-09-11T01:24:40.863 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.863 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.863 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.864 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-09-11T01:24:40.864 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 80950, 2024-09-11T01:24:40.864 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 226.062784916, 2024-09-11T01:24:40.864 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002792622 2024-09-11T01:24:40.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.864 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-09-11T01:24:40.864 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.864 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.864 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.865 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-09-11T01:24:40.865 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.865 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.865 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.865 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-09-11T01:24:40.865 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.866 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.866 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.866 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-09-11T01:24:40.866 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.866 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.866 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.867 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-09-11T01:24:40.867 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.867 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.867 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.867 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-09-11T01:24:40.867 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.867 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.868 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.868 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-09-11T01:24:40.868 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.868 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.868 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.868 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-09-11T01:24:40.868 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.869 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.869 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.869 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-09-11T01:24:40.869 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.869 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.869 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.870 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-09-11T01:24:40.870 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3, 2024-09-11T01:24:40.870 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001305533, 2024-09-11T01:24:40.870 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000435177 2024-09-11T01:24:40.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.870 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-09-11T01:24:40.870 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 102936, 2024-09-11T01:24:40.870 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 448.459328245, 2024-09-11T01:24:40.870 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004356681 2024-09-11T01:24:40.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.870 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7429, 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 60.918654427, 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.008200115 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.871 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 30, 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.371930811, 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.012397693 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-09-11T01:24:40.872 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.873 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.873 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.873 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-09-11T01:24:40.873 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 126, 2024-09-11T01:24:40.873 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 3.241669448, 2024-09-11T01:24:40.873 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.025727535 2024-09-11T01:24:40.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.873 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-09-11T01:24:40.873 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.873 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.873 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1268, 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 2.887736225, 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002277394 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-09-11T01:24:40.874 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.875 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.875 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.875 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-09-11T01:24:40.875 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.875 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.875 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.875 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-09-11T01:24:40.875 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 45226, 2024-09-11T01:24:40.875 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 149.796697715, 2024-09-11T01:24:40.875 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003312180 2024-09-11T01:24:40.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.876 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-09-11T01:24:40.876 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-09-11T01:24:40.876 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-09-11T01:24:40.876 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.876 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.876 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.876 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:40.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.877 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-09-11T01:24:40.877 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 2, 2024-09-11T01:24:40.877 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 3, 2024-09-11T01:24:40.877 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 1, 2024-09-11T01:24:40.877 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 2, 2024-09-11T01:24:40.877 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-09-11T01:24:40.877 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 27815, 2024-09-11T01:24:40.877 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 13907, 2024-09-11T01:24:40.877 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 1510, 2024-09-11T01:24:40.878 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-09-11T01:24:40.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.878 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-09-11T01:24:40.878 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-09-11T01:24:40.878 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-09-11T01:24:40.878 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T01:24:40.878 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-09-11T01:24:40.878 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T01:24:40.878 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-09-11T01:24:40.879 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T01:24:40.879 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T01:24:40.879 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T01:24:40.879 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T01:24:40.879 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T01:24:40.879 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-09-11T01:24:40.879 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T01:24:40.879 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T01:24:40.880 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-09-11T01:24:40.880 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-09-11T01:24:40.880 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-09-11T01:24:40.880 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-09-11T01:24:40.880 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T01:24:40.880 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T01:24:40.880 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T01:24:40.880 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T01:24:40.881 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T01:24:40.881 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-09-11T01:24:40.881 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-09-11T01:24:40.881 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-09-11T01:24:40.881 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T01:24:40.881 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T01:24:40.881 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-09-11T01:24:40.881 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-09-11T01:24:40.881 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-09-11T01:24:40.882 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-09-11T01:24:40.882 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T01:24:40.882 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-09-11T01:24:40.882 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T01:24:40.882 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-09-11T01:24:40.882 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 571223, 2024-09-11T01:24:40.882 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 1268, 2024-09-11T01:24:40.882 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 176, 2024-09-11T01:24:40.882 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 2, 2024-09-11T01:24:40.883 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-09-11T01:24:40.883 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-09-11T01:24:40.883 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-09-11T01:24:40.883 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-09-11T01:24:40.883 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-09-11T01:24:40.883 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-09-11T01:24:40.883 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 7976, 2024-09-11T01:24:40.883 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 177, 2024-09-11T01:24:40.884 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T01:24:40.884 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-09-11T01:24:40.884 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-09-11T01:24:40.884 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-09-11T01:24:40.884 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 7307792, 2024-09-11T01:24:40.884 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 57061, 2024-09-11T01:24:40.884 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-09-11T01:24:40.884 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-09-11T01:24:40.884 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-09-11T01:24:40.885 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-09-11T01:24:40.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.885 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-09-11T01:24:40.885 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-09-11T01:24:40.885 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-09-11T01:24:40.885 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 119729, 2024-09-11T01:24:40.885 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 1354294582, 2024-09-11T01:24:40.885 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 32, 2024-09-11T01:24:40.886 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 119729, 2024-09-11T01:24:40.886 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-09-11T01:24:40.886 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 119729, 2024-09-11T01:24:40.886 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1249.016163621, 2024-09-11T01:24:40.886 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.010432027 2024-09-11T01:24:40.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.886 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-09-11T01:24:40.886 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-09-11T01:24:40.887 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 119729, 2024-09-11T01:24:40.887 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 122413 2024-09-11T01:24:40.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.887 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 119729, 2024-09-11T01:24:40.887 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 3912, 2024-09-11T01:24:40.887 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 115817, 2024-09-11T01:24:40.887 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-09-11T01:24:40.887 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-09-11T01:24:40.888 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 37, 2024-09-11T01:24:40.888 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 177, 2024-09-11T01:24:40.888 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 1639, 2024-09-11T01:24:40.888 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 67644, 2024-09-11T01:24:40.888 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 882, 2024-09-11T01:24:40.888 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-09-11T01:24:40.888 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-09-11T01:24:40.888 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-09-11T01:24:40.889 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 3, 2024-09-11T01:24:40.889 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 45746, 2024-09-11T01:24:40.889 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-09-11T01:24:40.889 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-09-11T01:24:40.889 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-09-11T01:24:40.889 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 2265, 2024-09-11T01:24:40.889 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 354, 2024-09-11T01:24:40.889 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-09-11T01:24:40.889 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-09-11T01:24:40.890 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-09-11T01:24:40.890 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-09-11T01:24:40.890 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-09-11T01:24:40.890 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-09-11T01:24:40.890 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T01:24:40.890 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-09-11T01:24:40.890 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-09-11T01:24:40.890 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-09-11T01:24:40.891 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-09-11T01:24:40.891 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-09-11T01:24:40.891 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-09-11T01:24:40.891 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-09-11T01:24:40.891 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-09-11T01:24:40.891 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-09-11T01:24:40.891 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-09-11T01:24:40.891 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-09-11T01:24:40.891 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-09-11T01:24:40.891 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 115, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 40, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 12, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 12, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-09-11T01:24:40.892 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 2673, 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 20, 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 973 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 1, 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-09-11T01:24:40.893 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 6, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 45226, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 45226, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 1628979, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 369803542, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 1628979, 2024-09-11T01:24:40.894 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1628979, 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 369803542, 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:40.895 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 119729, 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 1358871063, 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 119729, 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1358871063, 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:40.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 119729, 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 119729, 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 119729, 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 119729, 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:40.897 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 45226, 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 4576818, 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 45226, 2024-09-11T01:24:40.898 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-11T01:24:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-11T01:24:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1639, 2024-09-11T01:24:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 4576818, 2024-09-11T01:24:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x55a4277e0dc0": { 2024-09-11T01:24:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:40.899 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 221648, 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1343689136, 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 221648, 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 66005, 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1343689136, 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:40.900 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:40.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:40.901 INFO:teuthology.orchestra.run.smithi112.stdout: "trackedop": { 2024-09-11T01:24:40.901 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 2 2024-09-11T01:24:40.901 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:40.901 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-11T01:24:41.879 INFO:tasks.check_counter:Daemon mds.i mds.exported=15 2024-09-11T01:24:41.879 INFO:tasks.check_counter:Daemon mds.i mds.imported=22 2024-09-11T01:24:41.879 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=532 2024-09-11T01:24:41.879 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=3615 2024-09-11T01:24:41.880 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-09-11T01:24:41.880 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-11T01:24:41.880 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T01:24:41.880 DEBUG:tasks.check_counter:Getting stats from l 2024-09-11T01:24:41.880 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-11T01:24:41.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:41 smithi110 ceph-mon[34712]: pgmap v1135: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 17 op/s 2024-09-11T01:24:41.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:41 smithi112 ceph-mon[35384]: pgmap v1135: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 17 op/s 2024-09-11T01:24:41.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:41 smithi078 ceph-mon[31315]: pgmap v1135: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 17 op/s 2024-09-11T01:24:42.130 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T01:24:42.671 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-11T01:24:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T01:24:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1583, 2024-09-11T01:24:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 1520, 2024-09-11T01:24:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 282610015, 2024-09-11T01:24:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 613600, 2024-09-11T01:24:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 7, 2024-09-11T01:24:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-09-11T01:24:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.315468315, 2024-09-11T01:24:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.070640861, 2024-09-11T01:24:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.224038153, 2024-09-11T01:24:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.025212098, 2024-09-11T01:24:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1520, 2024-09-11T01:24:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.076100418, 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000050066 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1320640, 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 289552 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2197, 2024-09-11T01:24:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2542, 2024-09-11T01:24:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 366066615, 2024-09-11T01:24:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 1396237, 2024-09-11T01:24:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 7, 2024-09-11T01:24:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 6, 2024-09-11T01:24:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.408241221, 2024-09-11T01:24:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.090483579, 2024-09-11T01:24:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.253012768, 2024-09-11T01:24:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.049820523, 2024-09-11T01:24:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2540, 2024-09-11T01:24:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.096110610, 2024-09-11T01:24:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000037838 2024-09-11T01:24:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T01:24:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 697904 2024-09-11T01:24:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T01:24:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4027, 2024-09-11T01:24:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4029, 2024-09-11T01:24:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 458349714, 2024-09-11T01:24:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 1280644, 2024-09-11T01:24:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-09-11T01:24:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-09-11T01:24:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.635556459, 2024-09-11T01:24:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.154286944, 2024-09-11T01:24:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.352317334, 2024-09-11T01:24:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.102049520, 2024-09-11T01:24:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T01:24:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4027, 2024-09-11T01:24:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.147304512, 2024-09-11T01:24:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000036579 2024-09-11T01:24:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-11T01:24:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T01:24:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T01:24:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-09-11T01:24:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-09-11T01:24:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-09-11T01:24:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-09-11T01:24:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-11T01:24:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-11T01:24:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 11576, 2024-09-11T01:24:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.309024073, 2024-09-11T01:24:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000026695 2024-09-11T01:24:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-09-11T01:24:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-09-11T01:24:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-09-11T01:24:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-11T01:24:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 0, 2024-09-11T01:24:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 0, 2024-09-11T01:24:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-09-11T01:24:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-09-11T01:24:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-09-11T01:24:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-09-11T01:24:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-09-11T01:24:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 0, 2024-09-11T01:24:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-09-11T01:24:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-09-11T01:24:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 37895, 2024-09-11T01:24:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 16401, 2024-09-11T01:24:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 21494, 2024-09-11T01:24:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-09-11T01:24:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 14464, 2024-09-11T01:24:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-09-11T01:24:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-09-11T01:24:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-09-11T01:24:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 1, 2024-09-11T01:24:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 0, 2024-09-11T01:24:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 0, 2024-09-11T01:24:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-09-11T01:24:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-09-11T01:24:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-09-11T01:24:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-09-11T01:24:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-09-11T01:24:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-09-11T01:24:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-09-11T01:24:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-09-11T01:24:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-09-11T01:24:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 0 2024-09-11T01:24:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-09-11T01:24:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0, 2024-09-11T01:24:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_succeeded": 0, 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_write_failed": 0 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 0, 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 186685, 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 0, 2024-09-11T01:24:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-09-11T01:24:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-09-11T01:24:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 140, 2024-09-11T01:24:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 17, 2024-09-11T01:24:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-09-11T01:24:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-09-11T01:24:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 548533766, 2024-09-11T01:24:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 1107885172, 2024-09-11T01:24:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 1107856079, 2024-09-11T01:24:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-09-11T01:24:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 186685 2024-09-11T01:24:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-09-11T01:24:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1401, 2024-09-11T01:24:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 37884, 2024-09-11T01:24:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 36483, 2024-09-11T01:24:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 45, 2024-09-11T01:24:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 73, 2024-09-11T01:24:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 28, 2024-09-11T01:24:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 37895, 2024-09-11T01:24:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 38278, 2024-09-11T01:24:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 383, 2024-09-11T01:24:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-09-11T01:24:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 0, 2024-09-11T01:24:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 0, 2024-09-11T01:24:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 90204, 2024-09-11T01:24:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 311996416 2024-09-11T01:24:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-09-11T01:24:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 0, 2024-09-11T01:24:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-09-11T01:24:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 0, 2024-09-11T01:24:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 0, 2024-09-11T01:24:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-09-11T01:24:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-09-11T01:24:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-09-11T01:24:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-09-11T01:24:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-09-11T01:24:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-09-11T01:24:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-09-11T01:24:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-09-11T01:24:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-09-11T01:24:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-09-11T01:24:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-09-11T01:24:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-09-11T01:24:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-09-11T01:24:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-09-11T01:24:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-09-11T01:24:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-09-11T01:24:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-09-11T01:24:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-09-11T01:24:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-09-11T01:24:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-09-11T01:24:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-09-11T01:24:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-09-11T01:24:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-09-11T01:24:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-09-11T01:24:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-09-11T01:24:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-09-11T01:24:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-09-11T01:24:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 2, 2024-09-11T01:24:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 2, 2024-09-11T01:24:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 0, 2024-09-11T01:24:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 2, 2024-09-11T01:24:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-09-11T01:24:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 0, 2024-09-11T01:24:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 0, 2024-09-11T01:24:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 2851, 2024-09-11T01:24:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-09-11T01:24:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-09-11T01:24:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-09-11T01:24:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-09-11T01:24:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T01:24:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-09-11T01:24:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T01:24:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-09-11T01:24:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T01:24:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T01:24:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T01:24:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T01:24:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T01:24:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-09-11T01:24:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T01:24:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T01:24:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-09-11T01:24:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-09-11T01:24:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-09-11T01:24:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-09-11T01:24:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T01:24:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T01:24:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T01:24:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-09-11T01:24:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 302113, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 1715, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 7976, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 177, 2024-09-11T01:24:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 29552722, 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 189907, 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-09-11T01:24:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 7242, 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 21, 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 3, 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 7242, 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7242, 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 23.569388685, 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003254541 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7242, 2024-09-11T01:24:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 7243 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 7242, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 7242, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 0, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 4164, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 0, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 3076, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 0, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 0, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-09-11T01:24:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-09-11T01:24:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-09-11T01:24:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 0, 2024-09-11T01:24:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-09-11T01:24:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-09-11T01:24:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-09-11T01:24:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 1, 2024-09-11T01:24:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 0, 2024-09-11T01:24:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-09-11T01:24:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-09-11T01:24:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-09-11T01:24:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-09-11T01:24:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-09-11T01:24:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-09-11T01:24:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T01:24:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-09-11T01:24:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-09-11T01:24:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-09-11T01:24:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-09-11T01:24:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-09-11T01:24:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-09-11T01:24:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-09-11T01:24:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-09-11T01:24:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-09-11T01:24:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-09-11T01:24:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-09-11T01:24:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-09-11T01:24:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-09-11T01:24:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-09-11T01:24:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-09-11T01:24:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-09-11T01:24:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-09-11T01:24:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-09-11T01:24:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-09-11T01:24:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 115, 2024-09-11T01:24:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-09-11T01:24:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 40, 2024-09-11T01:24:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 11, 2024-09-11T01:24:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 11, 2024-09-11T01:24:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-09-11T01:24:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-09-11T01:24:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 0, 2024-09-11T01:24:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 2, 2024-09-11T01:24:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 0 2024-09-11T01:24:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-09-11T01:24:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 0, 2024-09-11T01:24:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-09-11T01:24:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-09-11T01:24:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-09-11T01:24:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-09-11T01:24:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-09-11T01:24:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T01:24:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-09-11T01:24:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-09-11T01:24:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-09-11T01:24:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-09-11T01:24:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-09-11T01:24:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T01:24:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-11T01:24:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 7803, 2024-09-11T01:24:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1106292669, 2024-09-11T01:24:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 7803, 2024-09-11T01:24:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-11T01:24:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-11T01:24:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 7803, 2024-09-11T01:24:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1106292669, 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 7242, 2024-09-11T01:24:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 1103723984, 2024-09-11T01:24:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 7242, 2024-09-11T01:24:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1103723984, 2024-09-11T01:24:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-09-11T01:24:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-09-11T01:24:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-11T01:24:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-11T01:24:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 7242, 2024-09-11T01:24:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 7242, 2024-09-11T01:24:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 7242, 2024-09-11T01:24:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 7242, 2024-09-11T01:24:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-09-11T01:24:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-11T01:24:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x556c9e62d040": { 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-11T01:24:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-11T01:24:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-11T01:24:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "trackedop": { 2024-09-11T01:24:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-09-11T01:24:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-11T01:24:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-11T01:24:43.512 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-11T01:24:43.512 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-11T01:24:43.512 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-11T01:24:43.512 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-11T01:24:43.513 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-11T01:24:43.513 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-11T01:24:43.513 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T01:24:43.513 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-11T01:24:43.525 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-11T01:24:43.525 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T01:24:43.525 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T01:24:43.564 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T01:24:43.564 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T01:24:43.623 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T01:24:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:43.940 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:43 smithi110 ceph-mon[34712]: pgmap v1136: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-09-11T01:24:43.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:43 smithi112 ceph-mon[35384]: pgmap v1136: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-09-11T01:24:43.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:43 smithi078 ceph-mon[31315]: pgmap v1136: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-09-11T01:24:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.401+0000 7feeb1567640 1 -- 172.21.15.78:0/1606988031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feeac1076a0 msgr2=0x7feeac107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.401+0000 7feeb1567640 1 --2- 172.21.15.78:0/1606988031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feeac1076a0 0x7feeac107ae0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fee94009900 tx=0x7fee9402a410 comp rx=0 tx=0).stop 2024-09-11T01:24:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.401+0000 7feeb1567640 1 -- 172.21.15.78:0/1606988031 shutdown_connections 2024-09-11T01:24:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.401+0000 7feeb1567640 1 --2- 172.21.15.78:0/1606988031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeac108020 0x7feeac10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.401+0000 7feeb1567640 1 --2- 172.21.15.78:0/1606988031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feeac1076a0 0x7feeac107ae0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.401+0000 7feeb1567640 1 --2- 172.21.15.78:0/1606988031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac101c90 0x7feeac102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.401+0000 7feeb1567640 1 -- 172.21.15.78:0/1606988031 >> 172.21.15.78:0/1606988031 conn(0x7feeac0fd700 msgr2=0x7feeac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.401+0000 7feeb1567640 1 -- 172.21.15.78:0/1606988031 shutdown_connections 2024-09-11T01:24:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.401+0000 7feeb1567640 1 -- 172.21.15.78:0/1606988031 wait complete. 2024-09-11T01:24:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.402+0000 7feeb1567640 1 Processor -- start 2024-09-11T01:24:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.402+0000 7feeb1567640 1 -- start start 2024-09-11T01:24:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.402+0000 7feeb1567640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac101c90 0x7feeac199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.402+0000 7feeb1567640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeac1076a0 0x7feeac1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.402+0000 7feeb1567640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feeac108020 0x7feeac19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.402+0000 7feeb1567640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feeac111ce0 con 0x7feeac108020 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.402+0000 7feeb1567640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feeac111b60 con 0x7feeac1076a0 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.402+0000 7feeb1567640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feeac111e60 con 0x7feeac101c90 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.402+0000 7feeab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feeac108020 0x7feeac19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.402+0000 7feeab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feeac108020 0x7feeac19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59562/0 (socket says 172.21.15.78:59562) 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feeab7fe640 1 -- 172.21.15.78:0/2468082424 learned_addr learned my addr 172.21.15.78:0/2468082424 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.402+0000 7feeaaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac101c90 0x7feeac199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feeaa7fc640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeac1076a0 0x7feeac1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feeab7fe640 1 -- 172.21.15.78:0/2468082424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac101c90 msgr2=0x7feeac199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feeab7fe640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac101c90 0x7feeac199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feeab7fe640 1 -- 172.21.15.78:0/2468082424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeac1076a0 msgr2=0x7feeac1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feeab7fe640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeac1076a0 0x7feeac1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feeab7fe640 1 -- 172.21.15.78:0/2468082424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeac19f320 con 0x7feeac108020 2024-09-11T01:24:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feeaaffd640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac101c90 0x7feeac199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:24:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feeab7fe640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feeac108020 0x7feeac19eb50 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00d750 tx=0x7fee9c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feea3fff640 1 -- 172.21.15.78:0/2468082424 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c017070 con 0x7feeac108020 2024-09-11T01:24:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feeb1567640 1 -- 172.21.15.78:0/2468082424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeac19f610 con 0x7feeac108020 2024-09-11T01:24:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feea3fff640 1 -- 172.21.15.78:0/2468082424 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c0054d0 con 0x7feeac108020 2024-09-11T01:24:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feea3fff640 1 -- 172.21.15.78:0/2468082424 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c004c60 con 0x7feeac108020 2024-09-11T01:24:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.403+0000 7feeb1567640 1 -- 172.21.15.78:0/2468082424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeac1a5cd0 con 0x7feeac108020 2024-09-11T01:24:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.404+0000 7feeb1567640 1 -- 172.21.15.78:0/2468082424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeac102070 con 0x7feeac108020 2024-09-11T01:24:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.406+0000 7feea3fff640 1 -- 172.21.15.78:0/2468082424 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fee9c004e00 con 0x7feeac108020 2024-09-11T01:24:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.406+0000 7feea3fff640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee980b5450 0x7fee980b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.406+0000 7feeaaffd640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee980b5450 0x7fee980b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.406+0000 7feea3fff640 1 -- 172.21.15.78:0/2468082424 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7fee9c0e5db0 con 0x7feeac108020 2024-09-11T01:24:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.407+0000 7feeaaffd640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee980b5450 0x7fee980b7910 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7feea4005c40 tx=0x7feea400a590 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:44.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.409+0000 7feea3fff640 1 -- 172.21.15.78:0/2468082424 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fee9c0ac970 con 0x7feeac108020 2024-09-11T01:24:44.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.599+0000 7feeb1567640 1 -- 172.21.15.78:0/2468082424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feeac19a3a0 con 0x7feeac108020 2024-09-11T01:24:44.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.599+0000 7feea3fff640 1 -- 172.21.15.78:0/2468082424 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v115) ==== 81+0+886 (secure 0 0 0) 0x7fee9c0ac270 con 0x7feeac108020 2024-09-11T01:24:44.601 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2390785703 2024-09-12T00:48:52.336916+0000 2024-09-11T01:24:44.601 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/2984952017 2024-09-12T00:48:52.336916+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/2984952017 2024-09-12T00:48:52.336916+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1038951931 2024-09-12T00:50:15.928079+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/391908936 2024-09-12T00:49:14.935550+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2706514734 2024-09-12T00:50:15.928079+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2357025243 2024-09-12T00:48:52.336916+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1743270552 2024-09-12T00:49:14.935550+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1743270552 2024-09-12T00:49:14.935550+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2795778473 2024-09-12T00:49:14.935550+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2747741101 2024-09-12T00:49:14.935550+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1289343426 2024-09-12T00:50:15.928079+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3558896007 2024-09-12T00:50:15.928079+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/438261739 2024-09-12T00:48:52.336916+0000 2024-09-11T01:24:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1289343426 2024-09-12T00:50:15.928079+0000 2024-09-11T01:24:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:listed 15 entries 2024-09-11T01:24:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.601+0000 7feeb1567640 1 -- 172.21.15.78:0/2468082424 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee980b5450 msgr2=0x7fee980b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.601+0000 7feeb1567640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee980b5450 0x7fee980b7910 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7feea4005c40 tx=0x7feea400a590 comp rx=0 tx=0).stop 2024-09-11T01:24:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.601+0000 7feeb1567640 1 -- 172.21.15.78:0/2468082424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feeac108020 msgr2=0x7feeac19eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.601+0000 7feeb1567640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feeac108020 0x7feeac19eb50 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00d750 tx=0x7fee9c00dc10 comp rx=0 tx=0).stop 2024-09-11T01:24:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.602+0000 7feeb1567640 1 -- 172.21.15.78:0/2468082424 shutdown_connections 2024-09-11T01:24:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.602+0000 7feeb1567640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fee980b5450 0x7fee980b7910 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.602+0000 7feeb1567640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feeac108020 0x7feeac19eb50 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.602+0000 7feeb1567640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeac1076a0 0x7feeac1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.602+0000 7feeb1567640 1 --2- 172.21.15.78:0/2468082424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac101c90 0x7feeac199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.602+0000 7feeb1567640 1 -- 172.21.15.78:0/2468082424 >> 172.21.15.78:0/2468082424 conn(0x7feeac0fd700 msgr2=0x7feeac06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.602+0000 7feeb1567640 1 -- 172.21.15.78:0/2468082424 shutdown_connections 2024-09-11T01:24:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:44.602+0000 7feeb1567640 1 -- 172.21.15.78:0/2468082424 wait complete. 2024-09-11T01:24:44.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:44 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/2468082424' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T01:24:44.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:44 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/2468082424' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T01:24:45.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:44 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/2468082424' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T01:24:45.338 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T01:24:45.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T01:24:45.338 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T01:24:45.496 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T01:24:45.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T01:24:45.497 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T01:24:45.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T01:24:45.564 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T01:24:45.604 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T01:24:45.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T01:24:45.604 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T01:24:45.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T01:24:45.715 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-09-11T01:24:45.821 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T01:24:45.833 INFO:tasks.cephadm:Teardown begin 2024-09-11T01:24:45.834 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T01:24:45.834 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:24:45.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:45 smithi112 ceph-mon[35384]: pgmap v1137: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 14 op/s 2024-09-11T01:24:45.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:45 smithi078 ceph-mon[31315]: pgmap v1137: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 14 op/s 2024-09-11T01:24:46.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:45 smithi110 ceph-mon[34712]: pgmap v1137: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 14 op/s 2024-09-11T01:24:46.384 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.933+0000 7f10e25d6640 1 -- 172.21.15.78:0/2290791381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc109150 msgr2=0x7f10dc10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.933+0000 7f10e25d6640 1 --2- 172.21.15.78:0/2290791381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc109150 0x7f10dc10f950 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009ca0 tx=0x7f10d402a8c0 comp rx=0 tx=0).stop 2024-09-11T01:24:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.933+0000 7f10e25d6640 1 -- 172.21.15.78:0/2290791381 shutdown_connections 2024-09-11T01:24:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.933+0000 7f10e25d6640 1 --2- 172.21.15.78:0/2290791381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc109150 0x7f10dc10f950 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.933+0000 7f10e25d6640 1 --2- 172.21.15.78:0/2290791381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc1087d0 0x7f10dc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.933+0000 7f10e25d6640 1 --2- 172.21.15.78:0/2290791381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc102dc0 0x7f10dc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.933+0000 7f10e25d6640 1 -- 172.21.15.78:0/2290791381 >> 172.21.15.78:0/2290791381 conn(0x7f10dc0fe830 msgr2=0x7f10dc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.933+0000 7f10e25d6640 1 -- 172.21.15.78:0/2290791381 shutdown_connections 2024-09-11T01:24:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.933+0000 7f10e25d6640 1 -- 172.21.15.78:0/2290791381 wait complete. 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.933+0000 7f10e25d6640 1 Processor -- start 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e25d6640 1 -- start start 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e25d6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc102dc0 0x7f10dc0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e25d6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc1087d0 0x7f10dc0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e25d6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc109150 0x7f10dc075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e25d6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc112e50 con 0x7f10dc109150 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e25d6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc112cd0 con 0x7f10dc102dc0 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e25d6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc112fd0 con 0x7f10dc1087d0 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e0b73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc109150 0x7f10dc075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e0b73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc109150 0x7f10dc075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59590/0 (socket says 172.21.15.78:59590) 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e0b73640 1 -- 172.21.15.78:0/23406447 learned_addr learned my addr 172.21.15.78:0/23406447 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10db7fe640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc1087d0 0x7f10dc0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10dbfff640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc102dc0 0x7f10dc0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e0b73640 1 -- 172.21.15.78:0/23406447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc1087d0 msgr2=0x7f10dc0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e0b73640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc1087d0 0x7f10dc0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e0b73640 1 -- 172.21.15.78:0/23406447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc102dc0 msgr2=0x7f10dc0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e0b73640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc102dc0 0x7f10dc0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10e0b73640 1 -- 172.21.15.78:0/23406447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc1a77e0 con 0x7f10dc109150 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.934+0000 7f10db7fe640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc1087d0 0x7f10dc0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.935+0000 7f10e0b73640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc109150 0x7f10dc075f20 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f10d4005cc0 tx=0x7f10d402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.935+0000 7f10d97fa640 1 -- 172.21.15.78:0/23406447 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4003cf0 con 0x7f10dc109150 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.935+0000 7f10d97fa640 1 -- 172.21.15.78:0/23406447 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d40044f0 con 0x7f10dc109150 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.935+0000 7f10d97fa640 1 -- 172.21.15.78:0/23406447 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d402fc30 con 0x7f10dc109150 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.935+0000 7f10e25d6640 1 -- 172.21.15.78:0/23406447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc1a7a70 con 0x7f10dc109150 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.935+0000 7f10e25d6640 1 -- 172.21.15.78:0/23406447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1a8030 con 0x7f10dc109150 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.936+0000 7f10e25d6640 1 -- 172.21.15.78:0/23406447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc103220 con 0x7f10dc109150 2024-09-11T01:24:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.937+0000 7f10d97fa640 1 -- 172.21.15.78:0/23406447 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f10d4004040 con 0x7f10dc109150 2024-09-11T01:24:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.938+0000 7f10d97fa640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f10c80b5450 0x7f10c80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.938+0000 7f10dbfff640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f10c80b5450 0x7f10c80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.938+0000 7f10d97fa640 1 -- 172.21.15.78:0/23406447 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f10d41075b0 con 0x7f10dc109150 2024-09-11T01:24:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.938+0000 7f10dbfff640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f10c80b5450 0x7f10c80b7910 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009990 tx=0x7f10cc009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:46.941+0000 7f10d97fa640 1 -- 172.21.15.78:0/23406447 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f10d403b410 con 0x7f10dc109150 2024-09-11T01:24:47.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.157+0000 7f10e25d6640 1 -- 172.21.15.78:0/23406447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10dc02e5c0 con 0x7f10dc109150 2024-09-11T01:24:47.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.158+0000 7f10d97fa640 1 -- 172.21.15.78:0/23406447 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+11996 (secure 0 0 0) 0x7f10d40ce110 con 0x7f10dc109150 2024-09-11T01:24:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:24:47.160 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":80,"btime":"2024-09-11T01:24:07:496110+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":80,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:24:07.017719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:24:47.160 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 80 2024-09-11T01:24:47.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.160+0000 7f10e25d6640 1 -- 172.21.15.78:0/23406447 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f10c80b5450 msgr2=0x7f10c80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:47.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.160+0000 7f10e25d6640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f10c80b5450 0x7f10c80b7910 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009990 tx=0x7f10cc009110 comp rx=0 tx=0).stop 2024-09-11T01:24:47.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.160+0000 7f10e25d6640 1 -- 172.21.15.78:0/23406447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc109150 msgr2=0x7f10dc075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:47.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.160+0000 7f10e25d6640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc109150 0x7f10dc075f20 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f10d4005cc0 tx=0x7f10d402f470 comp rx=0 tx=0).stop 2024-09-11T01:24:47.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.160+0000 7f10e25d6640 1 -- 172.21.15.78:0/23406447 shutdown_connections 2024-09-11T01:24:47.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.160+0000 7f10e25d6640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f10c80b5450 0x7f10c80b7910 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:47.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.160+0000 7f10e25d6640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10dc109150 0x7f10dc075f20 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:47.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.160+0000 7f10e25d6640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc1087d0 0x7f10dc0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:47.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.160+0000 7f10e25d6640 1 --2- 172.21.15.78:0/23406447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10dc102dc0 0x7f10dc0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:47.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.160+0000 7f10e25d6640 1 -- 172.21.15.78:0/23406447 >> 172.21.15.78:0/23406447 conn(0x7f10dc0fe830 msgr2=0x7f10dc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:47.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.160+0000 7f10e25d6640 1 -- 172.21.15.78:0/23406447 shutdown_connections 2024-09-11T01:24:47.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:47.160+0000 7f10e25d6640 1 -- 172.21.15.78:0/23406447 wait complete. 2024-09-11T01:24:47.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:47 smithi112 ceph-mon[35384]: pgmap v1138: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 19 op/s 2024-09-11T01:24:47.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:47 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/23406447' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:24:47.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:47 smithi078 ceph-mon[31315]: pgmap v1138: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 19 op/s 2024-09-11T01:24:47.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:47 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/23406447' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:24:48.060 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T01:24:48.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:47 smithi110 ceph-mon[34712]: pgmap v1138: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 19 op/s 2024-09-11T01:24:48.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:47 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/23406447' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:24:48.311 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:48.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:48 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:24:48.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:48 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:24:49.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.102+0000 7f2be2808640 1 -- 172.21.15.78:0/1266621747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bdc101c90 msgr2=0x7f2bdc102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:49.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.102+0000 7f2be2808640 1 --2- 172.21.15.78:0/1266621747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bdc101c90 0x7f2bdc102070 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009900 tx=0x7f2bcc02a410 comp rx=0 tx=0).stop 2024-09-11T01:24:49.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.102+0000 7f2be2808640 1 -- 172.21.15.78:0/1266621747 shutdown_connections 2024-09-11T01:24:49.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.102+0000 7f2be2808640 1 --2- 172.21.15.78:0/1266621747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bdc108020 0x7f2bdc10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:49.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.102+0000 7f2be2808640 1 --2- 172.21.15.78:0/1266621747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bdc1076a0 0x7f2bdc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:49.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.102+0000 7f2be2808640 1 --2- 172.21.15.78:0/1266621747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bdc101c90 0x7f2bdc102070 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:49.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.102+0000 7f2be2808640 1 -- 172.21.15.78:0/1266621747 >> 172.21.15.78:0/1266621747 conn(0x7f2bdc0fd700 msgr2=0x7f2bdc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:49.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.102+0000 7f2be2808640 1 -- 172.21.15.78:0/1266621747 shutdown_connections 2024-09-11T01:24:49.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.102+0000 7f2be2808640 1 -- 172.21.15.78:0/1266621747 wait complete. 2024-09-11T01:24:49.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.102+0000 7f2be2808640 1 Processor -- start 2024-09-11T01:24:49.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.102+0000 7f2be2808640 1 -- start start 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be2808640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bdc101c90 0x7f2bdc19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be2808640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bdc1076a0 0x7f2bdc19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be2808640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bdc108020 0x7f2bdc1a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be2808640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc111d20 con 0x7f2bdc108020 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be2808640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc111ba0 con 0x7f2bdc101c90 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be2808640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc111ea0 con 0x7f2bdc1076a0 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be0da5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bdc108020 0x7f2bdc1a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be0da5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bdc108020 0x7f2bdc1a5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59614/0 (socket says 172.21.15.78:59614) 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be0da5640 1 -- 172.21.15.78:0/3093929014 learned_addr learned my addr 172.21.15.78:0/3093929014 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2bdbfff640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bdc101c90 0x7f2bdc19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2bdb7fe640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bdc1076a0 0x7f2bdc19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be0da5640 1 -- 172.21.15.78:0/3093929014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bdc1076a0 msgr2=0x7f2bdc19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be0da5640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bdc1076a0 0x7f2bdc19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be0da5640 1 -- 172.21.15.78:0/3093929014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bdc101c90 msgr2=0x7f2bdc19a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be0da5640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bdc101c90 0x7f2bdc19a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.103+0000 7f2be0da5640 1 -- 172.21.15.78:0/3093929014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc1a5750 con 0x7f2bdc108020 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.104+0000 7f2be0da5640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bdc108020 0x7f2bdc1a5210 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400b970 tx=0x7f2bd400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.104+0000 7f2bd97fa640 1 -- 172.21.15.78:0/3093929014 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4019070 con 0x7f2bdc108020 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.104+0000 7f2be2808640 1 -- 172.21.15.78:0/3093929014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc1a5920 con 0x7f2bdc108020 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.104+0000 7f2bd97fa640 1 -- 172.21.15.78:0/3093929014 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bd4014440 con 0x7f2bdc108020 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.104+0000 7f2bd97fa640 1 -- 172.21.15.78:0/3093929014 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40075d0 con 0x7f2bdc108020 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.104+0000 7f2be2808640 1 -- 172.21.15.78:0/3093929014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc1a5e80 con 0x7f2bdc108020 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.105+0000 7f2be2808640 1 -- 172.21.15.78:0/3093929014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bdc1020f0 con 0x7f2bdc108020 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.106+0000 7f2bd97fa640 1 -- 172.21.15.78:0/3093929014 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2bd40145f0 con 0x7f2bdc108020 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.107+0000 7f2bd97fa640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2bc40b5450 0x7f2bc40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.107+0000 7f2bdbfff640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2bc40b5450 0x7f2bc40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.107+0000 7f2bd97fa640 1 -- 172.21.15.78:0/3093929014 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7f2bd40e5690 con 0x7f2bdc108020 2024-09-11T01:24:49.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.107+0000 7f2bdbfff640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2bc40b5450 0x7f2bc40b7910 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc031440 tx=0x7f2bcc005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:49.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.111+0000 7f2bd97fa640 1 -- 172.21.15.78:0/3093929014 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2bd40abba0 con 0x7f2bdc108020 2024-09-11T01:24:49.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:48 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T01:24:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.332+0000 7f2be2808640 1 -- 172.21.15.78:0/3093929014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bdc02e5c0 con 0x7f2bdc108020 2024-09-11T01:24:49.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.332+0000 7f2bd97fa640 1 -- 172.21.15.78:0/3093929014 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+11996 (secure 0 0 0) 0x7f2bd40b05d0 con 0x7f2bdc108020 2024-09-11T01:24:49.334 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:24:49.335 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":80,"btime":"2024-09-11T01:24:07:496110+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14730,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/494285934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":494285934},{"type":"v1","addr":"172.21.15.78:6835","nonce":494285934}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1108876555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1108876555},{"type":"v1","addr":"172.21.15.78:6837","nonce":1108876555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":80,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:56:53.792170+0000","modified":"2024-09-11T01:24:07.017719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24599,"mds_1":24580,"mds_2":24565,"mds_3":24563,"mds_4":14808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3902249887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3902249887},{"type":"v1","addr":"172.21.15.78:6839","nonce":3902249887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6841/3713123708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":3713123708},{"type":"v1","addr":"172.21.15.78:6841","nonce":3713123708}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/2579452587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2579452587},{"type":"v1","addr":"172.21.15.110:6833","nonce":2579452587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6833/1732756287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1732756287},{"type":"v1","addr":"172.21.15.112:6833","nonce":1732756287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/3549318748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3549318748},{"type":"v1","addr":"172.21.15.110:6835","nonce":3549318748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24553":{"gid":24553,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/303956922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":303956922},{"type":"v1","addr":"172.21.15.112:6835","nonce":303956922}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.110:6837/1873031527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1873031527},{"type":"v1","addr":"172.21.15.110:6837","nonce":1873031527}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.112:6837/854577265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":854577265},{"type":"v1","addr":"172.21.15.112:6837","nonce":854577265}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.112:6839/2141614771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2141614771},{"type":"v1","addr":"172.21.15.112:6839","nonce":2141614771}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6839/3180315245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":3180315245},{"type":"v1","addr":"172.21.15.110:6839","nonce":3180315245}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[14808,24563,24565,24580,24599]},"id":1}]} 2024-09-11T01:24:49.335 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 80 2024-09-11T01:24:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.335+0000 7f2be2808640 1 -- 172.21.15.78:0/3093929014 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2bc40b5450 msgr2=0x7f2bc40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.335+0000 7f2be2808640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2bc40b5450 0x7f2bc40b7910 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc031440 tx=0x7f2bcc005e80 comp rx=0 tx=0).stop 2024-09-11T01:24:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.335+0000 7f2be2808640 1 -- 172.21.15.78:0/3093929014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bdc108020 msgr2=0x7f2bdc1a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.335+0000 7f2be2808640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bdc108020 0x7f2bdc1a5210 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400b970 tx=0x7f2bd400bd70 comp rx=0 tx=0).stop 2024-09-11T01:24:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.335+0000 7f2be2808640 1 -- 172.21.15.78:0/3093929014 shutdown_connections 2024-09-11T01:24:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.335+0000 7f2be2808640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f2bc40b5450 0x7f2bc40b7910 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.335+0000 7f2be2808640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bdc108020 0x7f2bdc1a5210 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.335+0000 7f2be2808640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bdc1076a0 0x7f2bdc19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.335+0000 7f2be2808640 1 --2- 172.21.15.78:0/3093929014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bdc101c90 0x7f2bdc19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.335+0000 7f2be2808640 1 -- 172.21.15.78:0/3093929014 >> 172.21.15.78:0/3093929014 conn(0x7f2bdc0fd700 msgr2=0x7f2bdc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.335+0000 7f2be2808640 1 -- 172.21.15.78:0/3093929014 shutdown_connections 2024-09-11T01:24:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:49.335+0000 7f2be2808640 1 -- 172.21.15.78:0/3093929014 wait complete. 2024-09-11T01:24:49.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:49 smithi112 ceph-mon[35384]: pgmap v1139: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-11T01:24:49.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:49 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:24:49.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:49 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:24:49.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:49 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:24:49.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:49 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3093929014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:24:49.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:49 smithi078 ceph-mon[31315]: pgmap v1139: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-11T01:24:49.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:49 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:24:49.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:49 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:24:49.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:49 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:24:49.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:49 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3093929014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:24:50.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:49 smithi110 ceph-mon[34712]: pgmap v1139: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-11T01:24:50.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:49 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T01:24:50.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:49 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T01:24:50.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:49 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' 2024-09-11T01:24:50.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:49 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3093929014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T01:24:50.204 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T01:24:50.453 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:51.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.970+0000 7fe144270640 1 -- 172.21.15.78:0/2902036549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe13c101c90 msgr2=0x7fe13c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:51.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.970+0000 7fe144270640 1 --2- 172.21.15.78:0/2902036549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe13c101c90 0x7fe13c102070 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009ca0 tx=0x7fe12c02a8c0 comp rx=0 tx=0).stop 2024-09-11T01:24:51.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 -- 172.21.15.78:0/2902036549 shutdown_connections 2024-09-11T01:24:51.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 --2- 172.21.15.78:0/2902036549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe13c108020 0x7fe13c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:51.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 --2- 172.21.15.78:0/2902036549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe13c1076a0 0x7fe13c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:51.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 --2- 172.21.15.78:0/2902036549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe13c101c90 0x7fe13c102070 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:51.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 -- 172.21.15.78:0/2902036549 >> 172.21.15.78:0/2902036549 conn(0x7fe13c0fd700 msgr2=0x7fe13c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:51.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 -- 172.21.15.78:0/2902036549 shutdown_connections 2024-09-11T01:24:51.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 -- 172.21.15.78:0/2902036549 wait complete. 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 Processor -- start 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 -- start start 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe13c101c90 0x7fe13c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe13c1076a0 0x7fe13c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe13c108020 0x7fe13c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c111d20 con 0x7fe13c1076a0 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c111ba0 con 0x7fe13c108020 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.971+0000 7fe144270640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c111ea0 con 0x7fe13c101c90 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe14180b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe13c1076a0 0x7fe13c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe14180b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe13c1076a0 0x7fe13c1999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59624/0 (socket says 172.21.15.78:59624) 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe14180b640 1 -- 172.21.15.78:0/3449902509 learned_addr learned my addr 172.21.15.78:0/3449902509 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe14200c640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe13c101c90 0x7fe13c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe14280d640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe13c108020 0x7fe13c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe14180b640 1 -- 172.21.15.78:0/3449902509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe13c101c90 msgr2=0x7fe13c199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe14180b640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe13c101c90 0x7fe13c199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe14180b640 1 -- 172.21.15.78:0/3449902509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe13c108020 msgr2=0x7fe13c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe14180b640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe13c108020 0x7fe13c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe14180b640 1 -- 172.21.15.78:0/3449902509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13c19f510 con 0x7fe13c1076a0 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe14280d640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe13c108020 0x7fe13c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe14180b640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe13c1076a0 0x7fe13c1999a0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe13000b4a0 tx=0x7fe13000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe13b7fe640 1 -- 172.21.15.78:0/3449902509 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130014070 con 0x7fe13c1076a0 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe13b7fe640 1 -- 172.21.15.78:0/3449902509 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe130010ac0 con 0x7fe13c1076a0 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe144270640 1 -- 172.21.15.78:0/3449902509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13c19f800 con 0x7fe13c1076a0 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe13b7fe640 1 -- 172.21.15.78:0/3449902509 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130017c30 con 0x7fe13c1076a0 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.972+0000 7fe144270640 1 -- 172.21.15.78:0/3449902509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13c1a5ec0 con 0x7fe13c1076a0 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.973+0000 7fe144270640 1 -- 172.21.15.78:0/3449902509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe13c1020f0 con 0x7fe13c1076a0 2024-09-11T01:24:51.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.974+0000 7fe13b7fe640 1 -- 172.21.15.78:0/3449902509 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe130010610 con 0x7fe13c1076a0 2024-09-11T01:24:51.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.975+0000 7fe13b7fe640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe1240b54d0 0x7fe1240b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:51.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.976+0000 7fe14200c640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe1240b54d0 0x7fe1240b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:51.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.976+0000 7fe13b7fe640 1 -- 172.21.15.78:0/3449902509 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7fe1300e5930 con 0x7fe13c1076a0 2024-09-11T01:24:51.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.976+0000 7fe14200c640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe1240b54d0 0x7fe1240b7990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe12c006010 tx=0x7fe12c000f40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:51.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:50.979+0000 7fe13b7fe640 1 -- 172.21.15.78:0/3449902509 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe1300ac4f0 con 0x7fe13c1076a0 2024-09-11T01:24:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.159+0000 7fe144270640 1 -- 172.21.15.78:0/3449902509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe13c19a590 con 0x7fe13c1076a0 2024-09-11T01:24:51.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.160+0000 7fe13b7fe640 1 -- 172.21.15.78:0/3449902509 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+23069 (secure 0 0 0) 0x7fe13c19a590 con 0x7fe13c1076a0 2024-09-11T01:24:51.162 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T01:24:51.162 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":115,"fsid":"64384694-6fd7-11ef-bcea-c7b262605968","created":"2024-09-11T00:48:27.652512+0000","modified":"2024-09-11T01:24:13.623597+0000","last_up_change":"2024-09-11T00:55:59.071507+0000","last_in_change":"2024-09-11T00:55:35.815150+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:52:10.229887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:56:49.939136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:56:51.703446+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beefc6d9-f74c-4200-96f2-ff697ad0d7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6803","nonce":1424639053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6805","nonce":1424639053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6809","nonce":1424639053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1424639053},{"type":"v1","addr":"172.21.15.78:6807","nonce":1424639053}]},"public_addr":"172.21.15.78:6803/1424639053","cluster_addr":"172.21.15.78:6805/1424639053","heartbeat_back_addr":"172.21.15.78:6809/1424639053","heartbeat_front_addr":"172.21.15.78:6807/1424639053","state":["exists","up"]},{"osd":1,"uuid":"d742e767-d673-4bb9-b624-8bc969f5ceb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6801","nonce":580939385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6803","nonce":580939385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6807","nonce":580939385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":580939385},{"type":"v1","addr":"172.21.15.110:6805","nonce":580939385}]},"public_addr":"172.21.15.110:6801/580939385","cluster_addr":"172.21.15.110:6803/580939385","heartbeat_back_addr":"172.21.15.110:6807/580939385","heartbeat_front_addr":"172.21.15.110:6805/580939385","state":["exists","up"]},{"osd":2,"uuid":"b0106a5b-1465-4a50-96dd-934dbae49a4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6801","nonce":2983155763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6803","nonce":2983155763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6807","nonce":2983155763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2983155763},{"type":"v1","addr":"172.21.15.112:6805","nonce":2983155763}]},"public_addr":"172.21.15.112:6801/2983155763","cluster_addr":"172.21.15.112:6803/2983155763","heartbeat_back_addr":"172.21.15.112:6807/2983155763","heartbeat_front_addr":"172.21.15.112:6805/2983155763","state":["exists","up"]},{"osd":3,"uuid":"3270362b-3d61-414a-a3d9-f2416ff8b870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6811","nonce":2906086464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6813","nonce":2906086464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6817","nonce":2906086464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2906086464},{"type":"v1","addr":"172.21.15.78:6815","nonce":2906086464}]},"public_addr":"172.21.15.78:6811/2906086464","cluster_addr":"172.21.15.78:6813/2906086464","heartbeat_back_addr":"172.21.15.78:6817/2906086464","heartbeat_front_addr":"172.21.15.78:6815/2906086464","state":["exists","up"]},{"osd":4,"uuid":"e47746ec-9443-4004-abfa-17c1ed73b982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6809","nonce":477287435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6811","nonce":477287435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6815","nonce":477287435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":477287435},{"type":"v1","addr":"172.21.15.110:6813","nonce":477287435}]},"public_addr":"172.21.15.110:6809/477287435","cluster_addr":"172.21.15.110:6811/477287435","heartbeat_back_addr":"172.21.15.110:6815/477287435","heartbeat_front_addr":"172.21.15.110:6813/477287435","state":["exists","up"]},{"osd":5,"uuid":"f8cedb01-310d-442d-a588-33197d32e208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6809","nonce":3557095096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6811","nonce":3557095096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6815","nonce":3557095096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3557095096},{"type":"v1","addr":"172.21.15.112:6813","nonce":3557095096}]},"public_addr":"172.21.15.112:6809/3557095096","cluster_addr":"172.21.15.112:6811/3557095096","heartbeat_back_addr":"172.21.15.112:6815/3557095096","heartbeat_front_addr":"172.21.15.112:6813/3557095096","state":["exists","up"]},{"osd":6,"uuid":"d508a037-8f56-436b-b7bf-9a1c69f6d9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6819","nonce":1267910775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6821","nonce":1267910775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6825","nonce":1267910775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1267910775},{"type":"v1","addr":"172.21.15.78:6823","nonce":1267910775}]},"public_addr":"172.21.15.78:6819/1267910775","cluster_addr":"172.21.15.78:6821/1267910775","heartbeat_back_addr":"172.21.15.78:6825/1267910775","heartbeat_front_addr":"172.21.15.78:6823/1267910775","state":["exists","up"]},{"osd":7,"uuid":"e8842047-09a6-48b8-8d0a-2d54e7f89063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6817","nonce":1556494594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6819","nonce":1556494594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6823","nonce":1556494594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1556494594},{"type":"v1","addr":"172.21.15.110:6821","nonce":1556494594}]},"public_addr":"172.21.15.110:6817/1556494594","cluster_addr":"172.21.15.110:6819/1556494594","heartbeat_back_addr":"172.21.15.110:6823/1556494594","heartbeat_front_addr":"172.21.15.110:6821/1556494594","state":["exists","up"]},{"osd":8,"uuid":"a3367da8-375a-4a46-a952-867d4b72ae2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6817","nonce":2481321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6819","nonce":2481321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6823","nonce":2481321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2481321},{"type":"v1","addr":"172.21.15.112:6821","nonce":2481321}]},"public_addr":"172.21.15.112:6817/2481321","cluster_addr":"172.21.15.112:6819/2481321","heartbeat_back_addr":"172.21.15.112:6823/2481321","heartbeat_front_addr":"172.21.15.112:6821/2481321","state":["exists","up"]},{"osd":9,"uuid":"d1825764-cefc-495b-99da-8ba429df0937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6827","nonce":197048350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6829","nonce":197048350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6833","nonce":197048350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":197048350},{"type":"v1","addr":"172.21.15.78:6831","nonce":197048350}]},"public_addr":"172.21.15.78:6827/197048350","cluster_addr":"172.21.15.78:6829/197048350","heartbeat_back_addr":"172.21.15.78:6833/197048350","heartbeat_front_addr":"172.21.15.78:6831/197048350","state":["exists","up"]},{"osd":10,"uuid":"e1cfff7a-8916-4845-a482-d79fa275da30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6825","nonce":2101123168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6827","nonce":2101123168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6831","nonce":2101123168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2101123168},{"type":"v1","addr":"172.21.15.110:6829","nonce":2101123168}]},"public_addr":"172.21.15.110:6825/2101123168","cluster_addr":"172.21.15.110:6827/2101123168","heartbeat_back_addr":"172.21.15.110:6831/2101123168","heartbeat_front_addr":"172.21.15.110:6829/2101123168","state":["exists","up"]},{"osd":11,"uuid":"fe4618a3-1a4a-4d7b-b8fb-bb0a17e20d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6825","nonce":1526310654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6827","nonce":1526310654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6831","nonce":1526310654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1526310654},{"type":"v1","addr":"172.21.15.112:6829","nonce":1526310654}]},"public_addr":"172.21.15.112:6825/1526310654","cluster_addr":"172.21.15.112:6827/1526310654","heartbeat_back_addr":"172.21.15.112:6831/1526310654","heartbeat_front_addr":"172.21.15.112:6829/1526310654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:17.334199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:51:41.400872+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:08.205144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:33.550527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:52:58.394116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:24.699607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:53:50.450193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:16.223363+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:54:40.947727+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:06.845451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:32.410035+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:55:57.744364+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/2390785703":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/2984952017":"2024-09-12T00:48:52.336916+0000","172.21.15.78:0/1038951931":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/391908936":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2706514734":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/2357025243":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6801/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6800/1743270552":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2795778473":"2024-09-12T00:49:14.935550+0000","172.21.15.78:0/2747741101":"2024-09-12T00:49:14.935550+0000","172.21.15.78:6801/1289343426":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/3558896007":"2024-09-12T00:50:15.928079+0000","172.21.15.78:0/438261739":"2024-09-12T00:48:52.336916+0000","172.21.15.78:6800/1289343426":"2024-09-12T00:50:15.928079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":33,"length":1},{"begin":46,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:24:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.163+0000 7fe144270640 1 -- 172.21.15.78:0/3449902509 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe1240b54d0 msgr2=0x7fe1240b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.163+0000 7fe144270640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe1240b54d0 0x7fe1240b7990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe12c006010 tx=0x7fe12c000f40 comp rx=0 tx=0).stop 2024-09-11T01:24:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.163+0000 7fe144270640 1 -- 172.21.15.78:0/3449902509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe13c1076a0 msgr2=0x7fe13c1999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.163+0000 7fe144270640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe13c1076a0 0x7fe13c1999a0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe13000b4a0 tx=0x7fe13000b960 comp rx=0 tx=0).stop 2024-09-11T01:24:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.163+0000 7fe144270640 1 -- 172.21.15.78:0/3449902509 shutdown_connections 2024-09-11T01:24:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.163+0000 7fe144270640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe1240b54d0 0x7fe1240b7990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.163+0000 7fe144270640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe13c108020 0x7fe13c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.163+0000 7fe144270640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe13c1076a0 0x7fe13c1999a0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.163+0000 7fe144270640 1 --2- 172.21.15.78:0/3449902509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe13c101c90 0x7fe13c199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.163+0000 7fe144270640 1 -- 172.21.15.78:0/3449902509 >> 172.21.15.78:0/3449902509 conn(0x7fe13c0fd700 msgr2=0x7fe13c06ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:51.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.163+0000 7fe144270640 1 -- 172.21.15.78:0/3449902509 shutdown_connections 2024-09-11T01:24:51.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:51.163+0000 7fe144270640 1 -- 172.21.15.78:0/3449902509 wait complete. 2024-09-11T01:24:51.967 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:51 smithi078 ceph-mon[31315]: pgmap v1140: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 17 op/s 2024-09-11T01:24:51.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:51 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/3449902509' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T01:24:51.969 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-11T01:24:51.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:51 smithi112 ceph-mon[35384]: pgmap v1140: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 17 op/s 2024-09-11T01:24:51.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:51 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/3449902509' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T01:24:52.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:51 smithi110 ceph-mon[34712]: pgmap v1140: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 17 op/s 2024-09-11T01:24:52.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:51 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/3449902509' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T01:24:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.680+0000 7fe34585a640 1 -- 172.21.15.78:0/3760627872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe340102dc0 msgr2=0x7fe3401031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.680+0000 7fe34585a640 1 --2- 172.21.15.78:0/3760627872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe340102dc0 0x7fe3401031a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe330009900 tx=0x7fe33002a410 comp rx=0 tx=0).stop 2024-09-11T01:24:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.680+0000 7fe34585a640 1 -- 172.21.15.78:0/3760627872 shutdown_connections 2024-09-11T01:24:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.680+0000 7fe34585a640 1 --2- 172.21.15.78:0/3760627872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe340109150 0x7fe34010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.680+0000 7fe34585a640 1 --2- 172.21.15.78:0/3760627872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3401087d0 0x7fe340108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.680+0000 7fe34585a640 1 --2- 172.21.15.78:0/3760627872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe340102dc0 0x7fe3401031a0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.680+0000 7fe34585a640 1 -- 172.21.15.78:0/3760627872 >> 172.21.15.78:0/3760627872 conn(0x7fe3400fe830 msgr2=0x7fe340100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.680+0000 7fe34585a640 1 -- 172.21.15.78:0/3760627872 shutdown_connections 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.680+0000 7fe34585a640 1 -- 172.21.15.78:0/3760627872 wait complete. 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.681+0000 7fe34585a640 1 Processor -- start 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.681+0000 7fe34585a640 1 -- start start 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.681+0000 7fe34585a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe340102dc0 0x7fe3400754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.681+0000 7fe34585a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3401087d0 0x7fe3400759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.681+0000 7fe34585a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe340109150 0x7fe340075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.681+0000 7fe34585a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe340112e50 con 0x7fe340109150 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.681+0000 7fe34585a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe340112cd0 con 0x7fe3401087d0 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.681+0000 7fe34585a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe340112fd0 con 0x7fe340102dc0 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe33f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe340109150 0x7fe340075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe33f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe340109150 0x7fe340075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59630/0 (socket says 172.21.15.78:59630) 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe33f7fe640 1 -- 172.21.15.78:0/1264749587 learned_addr learned my addr 172.21.15.78:0/1264749587 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe33effd640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe340102dc0 0x7fe3400754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe3367fc640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3401087d0 0x7fe3400759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe33f7fe640 1 -- 172.21.15.78:0/1264749587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe340102dc0 msgr2=0x7fe3400754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe33f7fe640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe340102dc0 0x7fe3400754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe33f7fe640 1 -- 172.21.15.78:0/1264749587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3401087d0 msgr2=0x7fe3400759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe33f7fe640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3401087d0 0x7fe3400759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe33f7fe640 1 -- 172.21.15.78:0/1264749587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3401a7840 con 0x7fe340109150 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe33f7fe640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe340109150 0x7fe340075f20 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fe33800b970 tx=0x7fe33800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe344858640 1 -- 172.21.15.78:0/1264749587 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338019070 con 0x7fe340109150 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe344858640 1 -- 172.21.15.78:0/1264749587 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe338014440 con 0x7fe340109150 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe34585a640 1 -- 172.21.15.78:0/1264749587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3401a7b30 con 0x7fe340109150 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe344858640 1 -- 172.21.15.78:0/1264749587 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3380075d0 con 0x7fe340109150 2024-09-11T01:24:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.682+0000 7fe34585a640 1 -- 172.21.15.78:0/1264749587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3401a8070 con 0x7fe340109150 2024-09-11T01:24:52.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.683+0000 7fe34585a640 1 -- 172.21.15.78:0/1264749587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe340103220 con 0x7fe340109150 2024-09-11T01:24:52.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.685+0000 7fe344858640 1 -- 172.21.15.78:0/1264749587 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe33800cb40 con 0x7fe340109150 2024-09-11T01:24:52.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.686+0000 7fe344858640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe32c0b54d0 0x7fe32c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:52.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.686+0000 7fe33effd640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe32c0b54d0 0x7fe32c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:52.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.687+0000 7fe344858640 1 -- 172.21.15.78:0/1264749587 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8449+0+0 (secure 0 0 0) 0x7fe3380e5af0 con 0x7fe340109150 2024-09-11T01:24:52.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.687+0000 7fe33effd640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe32c0b54d0 0x7fe32c0b7990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe330031440 tx=0x7fe330005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:52.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.689+0000 7fe344858640 1 -- 172.21.15.78:0/1264749587 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe3380ac060 con 0x7fe340109150 2024-09-11T01:24:52.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.900+0000 7fe34585a640 1 -- 172.21.15.78:0/1264749587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe3400767a0 con 0x7fe340109150 2024-09-11T01:24:52.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.905+0000 7fe344858640 1 -- 172.21.15.78:0/1264749587 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) ==== 161+0+0 (secure 0 0 0) 0x7fe3400767a0 con 0x7fe340109150 2024-09-11T01:24:52.907 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T01:24:52.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.907+0000 7fe34585a640 1 -- 172.21.15.78:0/1264749587 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe32c0b54d0 msgr2=0x7fe32c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:52.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.907+0000 7fe34585a640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe32c0b54d0 0x7fe32c0b7990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe330031440 tx=0x7fe330005e80 comp rx=0 tx=0).stop 2024-09-11T01:24:52.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.907+0000 7fe34585a640 1 -- 172.21.15.78:0/1264749587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe340109150 msgr2=0x7fe340075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:52.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.907+0000 7fe34585a640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe340109150 0x7fe340075f20 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fe33800b970 tx=0x7fe33800be30 comp rx=0 tx=0).stop 2024-09-11T01:24:52.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.907+0000 7fe34585a640 1 -- 172.21.15.78:0/1264749587 shutdown_connections 2024-09-11T01:24:52.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.907+0000 7fe34585a640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fe32c0b54d0 0x7fe32c0b7990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:52.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.907+0000 7fe34585a640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe340109150 0x7fe340075f20 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:52.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.907+0000 7fe34585a640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3401087d0 0x7fe3400759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:52.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.907+0000 7fe34585a640 1 --2- 172.21.15.78:0/1264749587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe340102dc0 0x7fe3400754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:52.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.907+0000 7fe34585a640 1 -- 172.21.15.78:0/1264749587 >> 172.21.15.78:0/1264749587 conn(0x7fe3400fe830 msgr2=0x7fe340100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:52.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.908+0000 7fe34585a640 1 -- 172.21.15.78:0/1264749587 shutdown_connections 2024-09-11T01:24:52.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:52.908+0000 7fe34585a640 1 -- 172.21.15.78:0/1264749587 wait complete. 2024-09-11T01:24:53.243 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b[65617]: ignoring --setuser ceph since I am not root 2024-09-11T01:24:53.243 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b[65617]: ignoring --setgroup ceph since I am not root 2024-09-11T01:24:53.243 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-mds[65621]: -- 172.21.15.110:0/1660261446 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b0137d7860 con 0x55b0137b5400 2024-09-11T01:24:53.243 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b[65617]: starting mds.b at 2024-09-11T01:24:53.243 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h[65302]: ignoring --setuser ceph since I am not root 2024-09-11T01:24:53.244 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h[65302]: ignoring --setgroup ceph since I am not root 2024-09-11T01:24:53.244 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-mds[65306]: -- 172.21.15.110:0/869735653 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f32f05860 con 0x564f32ee3000 2024-09-11T01:24:53.244 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h[65302]: starting mds.h at 2024-09-11T01:24:53.244 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-k[64690]: ignoring --setuser ceph since I am not root 2024-09-11T01:24:53.244 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-k[64690]: ignoring --setgroup ceph since I am not root 2024-09-11T01:24:53.244 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-mds[64694]: -- 172.21.15.110:0/2051331134 <== mon.1 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55559007bc20 con 0x555590058800 2024-09-11T01:24:53.244 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-k[64690]: starting mds.k at 2024-09-11T01:24:53.244 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-e[65001]: ignoring --setuser ceph since I am not root 2024-09-11T01:24:53.245 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-e[65001]: ignoring --setgroup ceph since I am not root 2024-09-11T01:24:53.245 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-mds[65005]: -- 172.21.15.110:0/3927676485 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cdb45c3860 con 0x55cdb45a0800 2024-09-11T01:24:53.245 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 01:24:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-e[65001]: starting mds.e at 2024-09-11T01:24:53.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:52 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a[31311]: 2024-09-11T01:24:52.901+0000 7f4107cc7640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T01:24:53.247 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-a[74736]: ignoring --setuser ceph since I am not root 2024-09-11T01:24:53.247 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-a[74736]: ignoring --setgroup ceph since I am not root 2024-09-11T01:24:53.247 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-a[74736]: starting mds.a at 2024-09-11T01:24:53.247 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-g[74328]: ignoring --setuser ceph since I am not root 2024-09-11T01:24:53.248 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-g[74328]: ignoring --setgroup ceph since I am not root 2024-09-11T01:24:53.248 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-mds[74335]: -- 172.21.15.78:0/286463821 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562023535860 con 0x562023513000 2024-09-11T01:24:53.248 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-g[74328]: starting mds.g at 2024-09-11T01:24:53.474 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-i[66790]: ignoring --setuser ceph since I am not root 2024-09-11T01:24:53.474 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-i[66790]: ignoring --setgroup ceph since I am not root 2024-09-11T01:24:53.474 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-mds[66794]: -- 172.21.15.112:0/1849284000 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55926b5fd860 con 0x55926b5db000 2024-09-11T01:24:53.474 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-i[66790]: starting mds.i at 2024-09-11T01:24:53.475 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-f[66176]: ignoring --setuser ceph since I am not root 2024-09-11T01:24:53.475 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-f[66176]: ignoring --setgroup ceph since I am not root 2024-09-11T01:24:53.475 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-mds[66180]: -- 172.21.15.112:0/3041582627 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563192a4bc20 con 0x563192a28800 2024-09-11T01:24:53.475 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-f[66176]: starting mds.f at 2024-09-11T01:24:53.475 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c[66476]: ignoring --setuser ceph since I am not root 2024-09-11T01:24:53.475 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c[66476]: ignoring --setgroup ceph since I am not root 2024-09-11T01:24:53.475 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-mds[66480]: -- 172.21.15.112:0/1913625605 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56438c53b860 con 0x56438c519000 2024-09-11T01:24:53.475 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c[66476]: starting mds.c at 2024-09-11T01:24:53.476 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l[65875]: ignoring --setuser ceph since I am not root 2024-09-11T01:24:53.476 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l[65875]: ignoring --setgroup ceph since I am not root 2024-09-11T01:24:53.476 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-mds[65879]: -- 172.21.15.112:0/1807852532 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad6f2d3860 con 0x55ad6f2b1000 2024-09-11T01:24:53.476 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:24:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l[65875]: starting mds.l at 2024-09-11T01:24:53.609 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T01:24:53.858 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:53 smithi112 ceph-mon[35384]: pgmap v1141: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-11T01:24:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:53 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1264749587' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-11T01:24:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:53 smithi112 ceph-mon[35384]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T01:24:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:53 smithi112 ceph-mon[35384]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T01:24:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:53 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1264749587' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T01:24:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:53 smithi112 ceph-mon[35384]: osdmap e116: 12 total, 12 up, 12 in 2024-09-11T01:24:53.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:53 smithi112 ceph-mon[35384]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T01:24:53.979 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-mon[31315]: pgmap v1141: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-11T01:24:53.979 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1264749587' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-11T01:24:53.979 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-mon[31315]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T01:24:53.979 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-mon[31315]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T01:24:53.979 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1264749587' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T01:24:53.979 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-mon[31315]: osdmap e116: 12 total, 12 up, 12 in 2024-09-11T01:24:53.979 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:53 smithi078 ceph-mon[31315]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T01:24:54.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:53 smithi110 ceph-mon[34712]: pgmap v1141: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-11T01:24:54.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:53 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1264749587' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-11T01:24:54.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:53 smithi110 ceph-mon[34712]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T01:24:54.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:53 smithi110 ceph-mon[34712]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T01:24:54.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:53 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1264749587' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T01:24:54.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:53 smithi110 ceph-mon[34712]: osdmap e116: 12 total, 12 up, 12 in 2024-09-11T01:24:54.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:53 smithi110 ceph-mon[34712]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T01:24:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.426+0000 7fd2101af640 1 -- 172.21.15.78:0/2726481206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd208104600 msgr2=0x7fd20810caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.426+0000 7fd2101af640 1 --2- 172.21.15.78:0/2726481206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd208104600 0x7fd20810caf0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009900 tx=0x7fd1fc02a410 comp rx=0 tx=0).stop 2024-09-11T01:24:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.426+0000 7fd2101af640 1 -- 172.21.15.78:0/2726481206 shutdown_connections 2024-09-11T01:24:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.426+0000 7fd2101af640 1 --2- 172.21.15.78:0/2726481206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd20810d140 0x7fd20810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.426+0000 7fd2101af640 1 --2- 172.21.15.78:0/2726481206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd208104600 0x7fd20810caf0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.426+0000 7fd2101af640 1 --2- 172.21.15.78:0/2726481206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd208103ce0 0x7fd2081040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.426+0000 7fd2101af640 1 -- 172.21.15.78:0/2726481206 >> 172.21.15.78:0/2726481206 conn(0x7fd2080fd700 msgr2=0x7fd2080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.426+0000 7fd2101af640 1 -- 172.21.15.78:0/2726481206 shutdown_connections 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.427+0000 7fd2101af640 1 -- 172.21.15.78:0/2726481206 wait complete. 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.427+0000 7fd2101af640 1 Processor -- start 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.427+0000 7fd2101af640 1 -- start start 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.427+0000 7fd2101af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd208103ce0 0x7fd208101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.427+0000 7fd2101af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd208104600 0x7fd208101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.427+0000 7fd2101af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd20810d140 0x7fd208102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.427+0000 7fd2101af640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd208112db0 con 0x7fd20810d140 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.427+0000 7fd2101af640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd208112c30 con 0x7fd208104600 2024-09-11T01:24:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.427+0000 7fd2101af640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd208112f30 con 0x7fd208103ce0 2024-09-11T01:24:54.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.427+0000 7fd20e74c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd20810d140 0x7fd208102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:54.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.427+0000 7fd20e74c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd20810d140 0x7fd208102410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37918/0 (socket says 172.21.15.78:37918) 2024-09-11T01:24:54.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.427+0000 7fd20e74c640 1 -- 172.21.15.78:0/1159336773 learned_addr learned my addr 172.21.15.78:0/1159336773 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:54.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd20d74a640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd208104600 0x7fd208101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:54.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd20df4b640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd208103ce0 0x7fd208101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:54.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd20e74c640 1 -- 172.21.15.78:0/1159336773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd208103ce0 msgr2=0x7fd208101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:54.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd20e74c640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd208103ce0 0x7fd208101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:54.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd20e74c640 1 -- 172.21.15.78:0/1159336773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd208104600 msgr2=0x7fd208101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:54.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd20e74c640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd208104600 0x7fd208101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:54.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd20e74c640 1 -- 172.21.15.78:0/1159336773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2081a7720 con 0x7fd20810d140 2024-09-11T01:24:54.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd20df4b640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd208103ce0 0x7fd208101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T01:24:54.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd20e74c640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd20810d140 0x7fd208102410 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd20000eaa0 tx=0x7fd20000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd206ffd640 1 -- 172.21.15.78:0/1159336773 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20000cdb0 con 0x7fd20810d140 2024-09-11T01:24:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd206ffd640 1 -- 172.21.15.78:0/1159336773 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd200005020 con 0x7fd20810d140 2024-09-11T01:24:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd2101af640 1 -- 172.21.15.78:0/1159336773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2081a7a10 con 0x7fd20810d140 2024-09-11T01:24:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd206ffd640 1 -- 172.21.15.78:0/1159336773 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2000116d0 con 0x7fd20810d140 2024-09-11T01:24:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.428+0000 7fd2101af640 1 -- 172.21.15.78:0/1159336773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2081a7ed0 con 0x7fd20810d140 2024-09-11T01:24:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.429+0000 7fd2101af640 1 -- 172.21.15.78:0/1159336773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2081040c0 con 0x7fd20810d140 2024-09-11T01:24:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.431+0000 7fd206ffd640 1 -- 172.21.15.78:0/1159336773 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd20000f040 con 0x7fd20810d140 2024-09-11T01:24:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.431+0000 7fd206ffd640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd1f00b5450 0x7fd1f00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.431+0000 7fd20df4b640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd1f00b5450 0x7fd1f00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.431+0000 7fd206ffd640 1 -- 172.21.15.78:0/1159336773 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8835+0+0 (secure 0 0 0) 0x7fd2000e6370 con 0x7fd20810d140 2024-09-11T01:24:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.432+0000 7fd20df4b640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd1f00b5450 0x7fd1f00b7910 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80058c0 tx=0x7fd1f8005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.434+0000 7fd206ffd640 1 -- 172.21.15.78:0/1159336773 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd200016030 con 0x7fd20810d140 2024-09-11T01:24:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.644+0000 7fd2101af640 1 -- 172.21.15.78:0/1159336773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd20802e5c0 con 0x7fd20810d140 2024-09-11T01:24:54.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.913+0000 7fd206ffd640 1 -- 172.21.15.78:0/1159336773 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v83) ==== 104+0+0 (secure 0 0 0) 0x7fd2000acdb0 con 0x7fd20810d140 2024-09-11T01:24:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.916+0000 7fd2101af640 1 -- 172.21.15.78:0/1159336773 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd1f00b5450 msgr2=0x7fd1f00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.916+0000 7fd2101af640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd1f00b5450 0x7fd1f00b7910 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80058c0 tx=0x7fd1f8005850 comp rx=0 tx=0).stop 2024-09-11T01:24:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.916+0000 7fd2101af640 1 -- 172.21.15.78:0/1159336773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd20810d140 msgr2=0x7fd208102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.916+0000 7fd2101af640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd20810d140 0x7fd208102410 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd20000eaa0 tx=0x7fd20000ef60 comp rx=0 tx=0).stop 2024-09-11T01:24:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.916+0000 7fd2101af640 1 -- 172.21.15.78:0/1159336773 shutdown_connections 2024-09-11T01:24:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.916+0000 7fd2101af640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fd1f00b5450 0x7fd1f00b7910 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.916+0000 7fd2101af640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd20810d140 0x7fd208102410 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:54.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.916+0000 7fd2101af640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd208104600 0x7fd208101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:54.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.916+0000 7fd2101af640 1 --2- 172.21.15.78:0/1159336773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd208103ce0 0x7fd208101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:54.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.916+0000 7fd2101af640 1 -- 172.21.15.78:0/1159336773 >> 172.21.15.78:0/1159336773 conn(0x7fd2080fd700 msgr2=0x7fd2080fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:54.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.916+0000 7fd2101af640 1 -- 172.21.15.78:0/1159336773 shutdown_connections 2024-09-11T01:24:54.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:54.916+0000 7fd2101af640 1 -- 172.21.15.78:0/1159336773 wait complete. 2024-09-11T01:24:55.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.110:6833/4034519744,v1:172.21.15.110:6838/4034519744] up:boot 2024-09-11T01:24:55.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.110:6832/2265155344,v1:172.21.15.110:6836/2265155344] up:boot 2024-09-11T01:24:55.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.110:6834/3337119369,v1:172.21.15.110:6837/3337119369] up:boot 2024-09-11T01:24:55.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.110:6835/4088506724,v1:172.21.15.110:6839/4088506724] up:boot 2024-09-11T01:24:55.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.112:6832/629338248,v1:172.21.15.112:6834/629338248] up:boot 2024-09-11T01:24:55.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.112:6833/2983573526,v1:172.21.15.112:6835/2983573526] up:boot 2024-09-11T01:24:55.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.112:6836/1792567483,v1:172.21.15.112:6838/1792567483] up:boot 2024-09-11T01:24:55.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.112:6837/725331310,v1:172.21.15.112:6839/725331310] up:boot 2024-09-11T01:24:55.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.78:6839/440302384,v1:172.21.15.78:6841/440302384] up:boot 2024-09-11T01:24:55.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: mds.? [v2:172.21.15.78:6838/1682911574,v1:172.21.15.78:6840/1682911574] up:boot 2024-09-11T01:24:55.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T01:24:55.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T01:24:55.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T01:24:55.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T01:24:55.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T01:24:55.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T01:24:55.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T01:24:55.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T01:24:55.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T01:24:55.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T01:24:55.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T01:24:55.192 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:54 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1159336773' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-11T01:24:55.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.110:6833/4034519744,v1:172.21.15.110:6838/4034519744] up:boot 2024-09-11T01:24:55.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.110:6832/2265155344,v1:172.21.15.110:6836/2265155344] up:boot 2024-09-11T01:24:55.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.110:6834/3337119369,v1:172.21.15.110:6837/3337119369] up:boot 2024-09-11T01:24:55.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.110:6835/4088506724,v1:172.21.15.110:6839/4088506724] up:boot 2024-09-11T01:24:55.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.112:6832/629338248,v1:172.21.15.112:6834/629338248] up:boot 2024-09-11T01:24:55.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.112:6833/2983573526,v1:172.21.15.112:6835/2983573526] up:boot 2024-09-11T01:24:55.225 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.112:6836/1792567483,v1:172.21.15.112:6838/1792567483] up:boot 2024-09-11T01:24:55.226 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.112:6837/725331310,v1:172.21.15.112:6839/725331310] up:boot 2024-09-11T01:24:55.226 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.78:6839/440302384,v1:172.21.15.78:6841/440302384] up:boot 2024-09-11T01:24:55.226 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: mds.? [v2:172.21.15.78:6838/1682911574,v1:172.21.15.78:6840/1682911574] up:boot 2024-09-11T01:24:55.226 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T01:24:55.226 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T01:24:55.226 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T01:24:55.227 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T01:24:55.227 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T01:24:55.227 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T01:24:55.227 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T01:24:55.227 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T01:24:55.228 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T01:24:55.228 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T01:24:55.228 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T01:24:55.228 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:54 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1159336773' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-11T01:24:55.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.110:6833/4034519744,v1:172.21.15.110:6838/4034519744] up:boot 2024-09-11T01:24:55.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.110:6832/2265155344,v1:172.21.15.110:6836/2265155344] up:boot 2024-09-11T01:24:55.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.110:6834/3337119369,v1:172.21.15.110:6837/3337119369] up:boot 2024-09-11T01:24:55.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.110:6835/4088506724,v1:172.21.15.110:6839/4088506724] up:boot 2024-09-11T01:24:55.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.112:6832/629338248,v1:172.21.15.112:6834/629338248] up:boot 2024-09-11T01:24:55.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.112:6833/2983573526,v1:172.21.15.112:6835/2983573526] up:boot 2024-09-11T01:24:55.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.112:6836/1792567483,v1:172.21.15.112:6838/1792567483] up:boot 2024-09-11T01:24:55.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.112:6837/725331310,v1:172.21.15.112:6839/725331310] up:boot 2024-09-11T01:24:55.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.78:6839/440302384,v1:172.21.15.78:6841/440302384] up:boot 2024-09-11T01:24:55.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: mds.? [v2:172.21.15.78:6838/1682911574,v1:172.21.15.78:6840/1682911574] up:boot 2024-09-11T01:24:55.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T01:24:55.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T01:24:55.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T01:24:55.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T01:24:55.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T01:24:55.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T01:24:55.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T01:24:55.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T01:24:55.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T01:24:55.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T01:24:55.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: from='mgr.14217 172.21.15.78:0/3784616399' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T01:24:55.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:54 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1159336773' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-11T01:24:55.774 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T01:24:55.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:55 smithi078 ceph-mon[31315]: pgmap v1143: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-09-11T01:24:55.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:55 smithi078 ceph-mon[31315]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T01:24:55.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:55 smithi078 ceph-mon[31315]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T01:24:55.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:55 smithi078 ceph-mon[31315]: Cluster is now healthy 2024-09-11T01:24:55.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:55 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/1159336773' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T01:24:55.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:55 smithi078 ceph-mon[31315]: fsmap 12 up:standby 2024-09-11T01:24:56.028 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:56.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:55 smithi110 ceph-mon[34712]: pgmap v1143: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-09-11T01:24:56.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:55 smithi110 ceph-mon[34712]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T01:24:56.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:55 smithi110 ceph-mon[34712]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T01:24:56.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:55 smithi110 ceph-mon[34712]: Cluster is now healthy 2024-09-11T01:24:56.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:55 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/1159336773' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T01:24:56.191 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:55 smithi110 ceph-mon[34712]: fsmap 12 up:standby 2024-09-11T01:24:56.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:55 smithi112 ceph-mon[35384]: pgmap v1143: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-09-11T01:24:56.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:55 smithi112 ceph-mon[35384]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T01:24:56.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:55 smithi112 ceph-mon[35384]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T01:24:56.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:55 smithi112 ceph-mon[35384]: Cluster is now healthy 2024-09-11T01:24:56.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:55 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/1159336773' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T01:24:56.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:55 smithi112 ceph-mon[35384]: fsmap 12 up:standby 2024-09-11T01:24:56.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 -- 172.21.15.78:0/1641000859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0420102dc0 msgr2=0x7f04201031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:56.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 --2- 172.21.15.78:0/1641000859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0420102dc0 0x7f04201031a0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f0410009900 tx=0x7f041002a3f0 comp rx=0 tx=0).stop 2024-09-11T01:24:56.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 -- 172.21.15.78:0/1641000859 shutdown_connections 2024-09-11T01:24:56.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 --2- 172.21.15.78:0/1641000859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0420109150 0x7f042010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:56.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 --2- 172.21.15.78:0/1641000859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04201087d0 0x7f0420108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:56.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 --2- 172.21.15.78:0/1641000859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0420102dc0 0x7f04201031a0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:56.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 -- 172.21.15.78:0/1641000859 >> 172.21.15.78:0/1641000859 conn(0x7f04200fe830 msgr2=0x7f0420100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:56.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 -- 172.21.15.78:0/1641000859 shutdown_connections 2024-09-11T01:24:56.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 -- 172.21.15.78:0/1641000859 wait complete. 2024-09-11T01:24:56.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 Processor -- start 2024-09-11T01:24:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 -- start start 2024-09-11T01:24:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0420102dc0 0x7f04201991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04201087d0 0x7f0420199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0420109150 0x7f042019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0420112e10 con 0x7f0420102dc0 2024-09-11T01:24:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.478+0000 7f0425720640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0420112c90 con 0x7f0420109150 2024-09-11T01:24:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f0425720640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0420112f90 con 0x7f04201087d0 2024-09-11T01:24:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0420102dc0 0x7f04201991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0420102dc0 0x7f04201991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37942/0 (socket says 172.21.15.78:37942) 2024-09-11T01:24:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041effd640 1 -- 172.21.15.78:0/94109623 learned_addr learned my addr 172.21.15.78:0/94109623 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041f7fe640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0420109150 0x7f042019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041effd640 1 -- 172.21.15.78:0/94109623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04201087d0 msgr2=0x7f0420199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041effd640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04201087d0 0x7f0420199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041effd640 1 -- 172.21.15.78:0/94109623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0420109150 msgr2=0x7f042019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041effd640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0420109150 0x7f042019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041effd640 1 -- 172.21.15.78:0/94109623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042019f290 con 0x7f0420102dc0 2024-09-11T01:24:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041f7fe640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0420109150 0x7f042019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T01:24:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041effd640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0420102dc0 0x7f04201991e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f0410006010 tx=0x7f04100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041cff9640 1 -- 172.21.15.78:0/94109623 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041003f030 con 0x7f0420102dc0 2024-09-11T01:24:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041cff9640 1 -- 172.21.15.78:0/94109623 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0410003800 con 0x7f0420102dc0 2024-09-11T01:24:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f041cff9640 1 -- 172.21.15.78:0/94109623 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410035070 con 0x7f0420102dc0 2024-09-11T01:24:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f0425720640 1 -- 172.21.15.78:0/94109623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042019f520 con 0x7f0420102dc0 2024-09-11T01:24:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.479+0000 7f0425720640 1 -- 172.21.15.78:0/94109623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04201a5c40 con 0x7f0420102dc0 2024-09-11T01:24:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.480+0000 7f0425720640 1 -- 172.21.15.78:0/94109623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04201031a0 con 0x7f0420102dc0 2024-09-11T01:24:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.482+0000 7f041cff9640 1 -- 172.21.15.78:0/94109623 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0410031b10 con 0x7f0420102dc0 2024-09-11T01:24:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.482+0000 7f041cff9640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f040c0b5450 0x7f040c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.482+0000 7f0417fff640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f040c0b5450 0x7f040c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.482+0000 7f041cff9640 1 -- 172.21.15.78:0/94109623 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8835+0+0 (secure 0 0 0) 0x7f0410106e00 con 0x7f0420102dc0 2024-09-11T01:24:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.483+0000 7f0417fff640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f040c0b5450 0x7f040c0b7910 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0420102170 tx=0x7f0408005c70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.485+0000 7f041cff9640 1 -- 172.21.15.78:0/94109623 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f04100cd7e0 con 0x7f0420102dc0 2024-09-11T01:24:56.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.661+0000 7f0425720640 1 -- 172.21.15.78:0/94109623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f04201a5f30 con 0x7f0420102dc0 2024-09-11T01:24:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.922+0000 7f041cff9640 1 -- 172.21.15.78:0/94109623 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) ==== 181+0+0 (secure 0 0 0) 0x7f04100cd0e0 con 0x7f0420102dc0 2024-09-11T01:24:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T01:24:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.924+0000 7f0425720640 1 -- 172.21.15.78:0/94109623 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f040c0b5450 msgr2=0x7f040c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.924+0000 7f0425720640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f040c0b5450 0x7f040c0b7910 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0420102170 tx=0x7f0408005c70 comp rx=0 tx=0).stop 2024-09-11T01:24:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.924+0000 7f0425720640 1 -- 172.21.15.78:0/94109623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0420102dc0 msgr2=0x7f04201991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.924+0000 7f0425720640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0420102dc0 0x7f04201991e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f0410006010 tx=0x7f04100042f0 comp rx=0 tx=0).stop 2024-09-11T01:24:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.924+0000 7f0425720640 1 -- 172.21.15.78:0/94109623 shutdown_connections 2024-09-11T01:24:56.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.924+0000 7f0425720640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7f040c0b5450 0x7f040c0b7910 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:56.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.924+0000 7f0425720640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0420109150 0x7f042019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:56.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.924+0000 7f0425720640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04201087d0 0x7f0420199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:56.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.924+0000 7f0425720640 1 --2- 172.21.15.78:0/94109623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0420102dc0 0x7f04201991e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:56.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.924+0000 7f0425720640 1 -- 172.21.15.78:0/94109623 >> 172.21.15.78:0/94109623 conn(0x7f04200fe830 msgr2=0x7f04200fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:56.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.924+0000 7f0425720640 1 -- 172.21.15.78:0/94109623 shutdown_connections 2024-09-11T01:24:56.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:56.924+0000 7f0425720640 1 -- 172.21.15.78:0/94109623 wait complete. 2024-09-11T01:24:57.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:56 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/94109623' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T01:24:57.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:56 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/94109623' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T01:24:57.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:56 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/94109623' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T01:24:57.744 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 64384694-6fd7-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T01:24:57.990 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T01:24:57.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:57 smithi078 ceph-mon[31315]: pgmap v1144: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-11T01:24:57.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:57 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/94109623' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T01:24:57.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:57 smithi078 ceph-mon[31315]: osdmap e117: 12 total, 12 up, 12 in 2024-09-11T01:24:58.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:57 smithi110 ceph-mon[34712]: pgmap v1144: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-11T01:24:58.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:57 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/94109623' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T01:24:58.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:57 smithi110 ceph-mon[34712]: osdmap e117: 12 total, 12 up, 12 in 2024-09-11T01:24:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:57 smithi112 ceph-mon[35384]: pgmap v1144: 97 pgs: 97 active+clean; 3.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-11T01:24:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:57 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/94109623' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T01:24:58.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:57 smithi112 ceph-mon[35384]: osdmap e117: 12 total, 12 up, 12 in 2024-09-11T01:24:58.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.467+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/3275053073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb68103b70 msgr2=0x7fbb68103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:58.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.467+0000 7fbb6cdb1640 1 --2- 172.21.15.78:0/3275053073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb68103b70 0x7fbb68103f50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009900 tx=0x7fbb5802a410 comp rx=0 tx=0).stop 2024-09-11T01:24:58.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.467+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/3275053073 shutdown_connections 2024-09-11T01:24:58.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.467+0000 7fbb6cdb1640 1 --2- 172.21.15.78:0/3275053073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb6810d120 0x7fbb6810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.467+0000 7fbb6cdb1640 1 --2- 172.21.15.78:0/3275053073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68104570 0x7fbb6810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.467+0000 7fbb6cdb1640 1 --2- 172.21.15.78:0/3275053073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb68103b70 0x7fbb68103f50 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.467+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/3275053073 >> 172.21.15.78:0/3275053073 conn(0x7fbb680fd700 msgr2=0x7fbb680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.467+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/3275053073 shutdown_connections 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.467+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/3275053073 wait complete. 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.468+0000 7fbb6cdb1640 1 Processor -- start 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.468+0000 7fbb6cdb1640 1 -- start start 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.468+0000 7fbb6cdb1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb68103b70 0x7fbb681992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.468+0000 7fbb6cdb1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68104570 0x7fbb681997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.468+0000 7fbb6cdb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb6810d120 0x7fbb6819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.468+0000 7fbb6cdb1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68112da0 con 0x7fbb68103b70 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.468+0000 7fbb6cdb1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68112c20 con 0x7fbb68104570 2024-09-11T01:24:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.468+0000 7fbb6cdb1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68112f20 con 0x7fbb6810d120 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb6659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb68103b70 0x7fbb681992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb6659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb68103b70 0x7fbb681992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37948/0 (socket says 172.21.15.78:37948) 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb6659c640 1 -- 172.21.15.78:0/205984161 learned_addr learned my addr 172.21.15.78:0/205984161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb66d9d640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb6810d120 0x7fbb6819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb5ffff640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68104570 0x7fbb681997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb6659c640 1 -- 172.21.15.78:0/205984161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb6810d120 msgr2=0x7fbb6819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb6659c640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb6810d120 0x7fbb6819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb6659c640 1 -- 172.21.15.78:0/205984161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68104570 msgr2=0x7fbb681997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb6659c640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68104570 0x7fbb681997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb6659c640 1 -- 172.21.15.78:0/205984161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6819f360 con 0x7fbb68103b70 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb66d9d640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb6810d120 0x7fbb6819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb6659c640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb68103b70 0x7fbb681992b0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fbb58005940 tx=0x7fbb580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb5f7fe640 1 -- 172.21.15.78:0/205984161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5803f030 con 0x7fbb68103b70 2024-09-11T01:24:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/205984161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6819f5f0 con 0x7fbb68103b70 2024-09-11T01:24:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb5f7fe640 1 -- 172.21.15.78:0/205984161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb58003cb0 con 0x7fbb68103b70 2024-09-11T01:24:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/205984161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb681a5d10 con 0x7fbb68103b70 2024-09-11T01:24:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.469+0000 7fbb5f7fe640 1 -- 172.21.15.78:0/205984161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58035070 con 0x7fbb68103b70 2024-09-11T01:24:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.470+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/205984161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb68103f50 con 0x7fbb68103b70 2024-09-11T01:24:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.472+0000 7fbb5f7fe640 1 -- 172.21.15.78:0/205984161 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fbb58003800 con 0x7fbb68103b70 2024-09-11T01:24:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.473+0000 7fbb5f7fe640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbb540b5450 0x7fbb540b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T01:24:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.473+0000 7fbb5ffff640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbb540b5450 0x7fbb540b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T01:24:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.473+0000 7fbb5f7fe640 1 -- 172.21.15.78:0/205984161 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8310+0+0 (secure 0 0 0) 0x7fbb58106c60 con 0x7fbb68103b70 2024-09-11T01:24:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.473+0000 7fbb5ffff640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbb540b5450 0x7fbb540b7910 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbb680ffdc0 tx=0x7fbb50009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T01:24:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.475+0000 7fbb5f7fe640 1 -- 172.21.15.78:0/205984161 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbb580cd8b0 con 0x7fbb68103b70 2024-09-11T01:24:58.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.673+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/205984161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbb681a6030 con 0x7fbb68103b70 2024-09-11T01:24:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.928+0000 7fbb5f7fe640 1 -- 172.21.15.78:0/205984161 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) ==== 169+0+0 (secure 0 0 0) 0x7fbb580cd1b0 con 0x7fbb68103b70 2024-09-11T01:24:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T01:24:58.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.931+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/205984161 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbb540b5450 msgr2=0x7fbb540b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:58.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.931+0000 7fbb6cdb1640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbb540b5450 0x7fbb540b7910 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbb680ffdc0 tx=0x7fbb50009110 comp rx=0 tx=0).stop 2024-09-11T01:24:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.931+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/205984161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb68103b70 msgr2=0x7fbb681992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T01:24:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.931+0000 7fbb6cdb1640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb68103b70 0x7fbb681992b0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fbb58005940 tx=0x7fbb580042f0 comp rx=0 tx=0).stop 2024-09-11T01:24:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.931+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/205984161 shutdown_connections 2024-09-11T01:24:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.931+0000 7fbb6cdb1640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.78:6800/1082762376,v1:172.21.15.78:6801/1082762376] conn(0x7fbb540b5450 0x7fbb540b7910 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.931+0000 7fbb6cdb1640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb6810d120 0x7fbb6819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.931+0000 7fbb6cdb1640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb68104570 0x7fbb681997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T01:24:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.931+0000 7fbb6cdb1640 1 --2- 172.21.15.78:0/205984161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb68103b70 0x7fbb681992b0 secure :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fbb58005940 tx=0x7fbb580042f0 comp rx=0 tx=0).stop 2024-09-11T01:24:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.931+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/205984161 >> 172.21.15.78:0/205984161 conn(0x7fbb680fd700 msgr2=0x7fbb6810f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T01:24:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.931+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/205984161 shutdown_connections 2024-09-11T01:24:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T01:24:58.931+0000 7fbb6cdb1640 1 -- 172.21.15.78:0/205984161 wait complete. 2024-09-11T01:24:59.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:58 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/205984161' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T01:24:59.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:58 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/205984161' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T01:24:59.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:58 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/205984161' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T01:24:59.630 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T01:24:59.698 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T01:24:59.734 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T01:24:59.775 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T01:24:59.775 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T01:24:59.791 DEBUG:teuthology.orchestra.run.smithi110:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T01:24:59.821 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T01:24:59.836 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T01:24:59.836 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T01:24:59.837 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.a 2024-09-11T01:25:00.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:59 smithi110 ceph-mon[34712]: pgmap v1146: 65 pgs: 65 active+clean; 305 MiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T01:25:00.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:59 smithi110 ceph-mon[34712]: from='client.? 172.21.15.78:0/205984161' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T01:25:00.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:24:59 smithi110 ceph-mon[34712]: osdmap e118: 12 total, 12 up, 12 in 2024-09-11T01:25:00.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:59 smithi112 ceph-mon[35384]: pgmap v1146: 65 pgs: 65 active+clean; 305 MiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T01:25:00.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:59 smithi112 ceph-mon[35384]: from='client.? 172.21.15.78:0/205984161' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T01:25:00.224 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:24:59 smithi112 ceph-mon[35384]: osdmap e118: 12 total, 12 up, 12 in 2024-09-11T01:25:00.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:59 smithi078 systemd[1]: Stopping Ceph mon.a for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:25:00.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:59 smithi078 ceph-mon[31315]: pgmap v1146: 65 pgs: 65 active+clean; 305 MiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T01:25:00.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:59 smithi078 ceph-mon[31315]: from='client.? 172.21.15.78:0/205984161' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T01:25:00.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:24:59 smithi078 ceph-mon[31315]: osdmap e118: 12 total, 12 up, 12 in 2024-09-11T01:25:00.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a[31311]: 2024-09-11T01:25:00.637+0000 7f410d4d2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:25:00.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a[31311]: 2024-09-11T01:25:00.637+0000 7f410d4d2640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T01:25:00.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T01:25:00.388+0000 7ff88496a640 -1 client.14229: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-11T01:25:00.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T01:25:00.389+0000 7ff88496a640 -1 client.14229: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-11T01:25:00.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: Exception in thread Thread-31: 2024-09-11T01:25:00.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: Traceback (most recent call last): 2024-09-11T01:25:00.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-09-11T01:25:00.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self.run() 2024-09-11T01:25:00.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-09-11T01:25:00.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self.function(*self.args, **self.kwargs) 2024-09-11T01:25:00.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-09-11T01:25:00.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: self.prune_snapshots(sched) 2024-09-11T01:25:00.748 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-09-11T01:25:01.191 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:25:00 smithi078 podman[129944]: 2024-09-11 01:25:00.873498756 +0000 UTC m=+0.445443771 container died ac3d670696112fa5f07d8702c5ee804bbd03a8f059c1b9ce8258465df86d2935 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T01:25:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:25:01 smithi078 podman[129944]: 2024-09-11 01:25:01.389194939 +0000 UTC m=+0.961139955 container cleanup ac3d670696112fa5f07d8702c5ee804bbd03a8f059c1b9ce8258465df86d2935 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T01:25:01.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 11 01:25:01 smithi078 bash[129944]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-a 2024-09-11T01:25:01.803 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T01:25:01.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:25:01.853 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T01:25:01.853 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-11T01:25:01.853 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.b 2024-09-11T01:25:02.190 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:25:01 smithi110 systemd[1]: Stopping Ceph mon.b for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:25:02.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:25:02 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-b[34708]: 2024-09-11T01:25:02.521+0000 7efd7a23d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:25:02.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:25:02 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-b[34708]: 2024-09-11T01:25:02.521+0000 7efd7a23d640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T01:25:02.941 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:25:02 smithi110 podman[74735]: 2024-09-11 01:25:02.757271942 +0000 UTC m=+0.652524606 container died befced7d7ecb090ed341b56ce15d0bca2f35900c24c00980f033b7d2c2e1d5b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T01:25:03.433 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:25:03 smithi110 podman[74735]: 2024-09-11 01:25:03.146464334 +0000 UTC m=+1.041716990 container cleanup befced7d7ecb090ed341b56ce15d0bca2f35900c24c00980f033b7d2c2e1d5b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T01:25:03.433 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 11 01:25:03 smithi110 bash[74735]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-b 2024-09-11T01:25:03.578 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T01:25:03.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:25:03.623 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-11T01:25:03.623 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-11T01:25:03.624 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.c 2024-09-11T01:25:03.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:25:03 smithi112 systemd[1]: Stopping Ceph mon.c for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:25:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:25:04 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-c[35370]: 2024-09-11T01:25:04.111+0000 7fe2b55b6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:25:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:25:04 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-c[35370]: 2024-09-11T01:25:04.111+0000 7fe2b55b6640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T01:25:04.475 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:25:04 smithi112 podman[75991]: 2024-09-11 01:25:04.316300773 +0000 UTC m=+0.402501730 container died 4f047f3ebd3e1fb1587e92558429a3af316d7f3cf98cf9665530b13b04781591 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-11T01:25:04.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:25:04 smithi112 podman[75991]: 2024-09-11 01:25:04.797410013 +0000 UTC m=+0.883611015 container cleanup 4f047f3ebd3e1fb1587e92558429a3af316d7f3cf98cf9665530b13b04781591 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T01:25:04.974 INFO:journalctl@ceph.mon.c.smithi112.stdout:Sep 11 01:25:04 smithi112 bash[75991]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mon-c 2024-09-11T01:25:05.226 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T01:25:05.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:25:05.272 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-11T01:25:05.272 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T01:25:05.272 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.x 2024-09-11T01:25:05.588 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:05 smithi078 systemd[1]: Stopping Ceph mgr.x for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:25:05.859 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:05 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x[31552]: 2024-09-11T01:25:05.584+0000 7ff933269640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:25:05.859 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:05 smithi078 podman[130066]: 2024-09-11 01:25:05.627312367 +0000 UTC m=+0.162678681 container died 41c1f890f76fda0403489d1a9933d9a43da4740931e4e909e455a433eee2af39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T01:25:06.747 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:06 smithi078 podman[130066]: 2024-09-11 01:25:06.354301954 +0000 UTC m=+0.889668258 container cleanup 41c1f890f76fda0403489d1a9933d9a43da4740931e4e909e455a433eee2af39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T01:25:06.747 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:06 smithi078 bash[130066]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x 2024-09-11T01:25:06.747 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 11 01:25:06 smithi078 podman[130079]: 2024-09-11 01:25:06.564678167 +0000 UTC m=+0.939728822 container remove 41c1f890f76fda0403489d1a9933d9a43da4740931e4e909e455a433eee2af39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0) 2024-09-11T01:25:06.807 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T01:25:06.846 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:25:06.846 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T01:25:06.846 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T01:25:06.846 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.y 2024-09-11T01:25:07.190 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 01:25:06 smithi110 systemd[1]: Stopping Ceph mgr.y for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:25:07.588 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 01:25:07 smithi110 podman[74858]: 2024-09-11 01:25:07.274431037 +0000 UTC m=+0.216720431 container died c5d729218aea70209c6d0c38fa4d40c3ee5492e6326dae0f4deb1570df2f8aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T01:25:07.940 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 01:25:07 smithi110 podman[74858]: 2024-09-11 01:25:07.694382556 +0000 UTC m=+0.636671946 container cleanup c5d729218aea70209c6d0c38fa4d40c3ee5492e6326dae0f4deb1570df2f8aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T01:25:07.940 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 01:25:07 smithi110 bash[74858]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-y 2024-09-11T01:25:08.480 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 01:25:08 smithi110 podman[74870]: 2024-09-11 01:25:08.221407715 +0000 UTC m=+0.951011710 container remove c5d729218aea70209c6d0c38fa4d40c3ee5492e6326dae0f4deb1570df2f8aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T01:25:08.480 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 01:25:08 smithi110 systemd[1]: ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-11T01:25:08.484 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T01:25:08.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:25:08.519 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T01:25:08.520 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-11T01:25:08.520 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.z 2024-09-11T01:25:08.974 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 01:25:08 smithi112 systemd[1]: Stopping Ceph mgr.z for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:25:09.474 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 01:25:09 smithi112 podman[76113]: 2024-09-11 01:25:09.150799807 +0000 UTC m=+0.434426998 container died 8ece7eb0deefbe7926b6824a6ac82e0857220f424f20094882d541550c444a18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T01:25:09.956 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 01:25:09 smithi112 podman[76113]: 2024-09-11 01:25:09.737054448 +0000 UTC m=+1.020681626 container cleanup 8ece7eb0deefbe7926b6824a6ac82e0857220f424f20094882d541550c444a18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T01:25:09.956 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 01:25:09 smithi112 bash[76113]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-z 2024-09-11T01:25:10.213 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 01:25:09 smithi112 podman[76125]: 2024-09-11 01:25:09.955214586 +0000 UTC m=+0.804975640 container remove 8ece7eb0deefbe7926b6824a6ac82e0857220f424f20094882d541550c444a18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-11T01:25:10.213 INFO:journalctl@ceph.mgr.z.smithi112.stdout:Sep 11 01:25:09 smithi112 systemd[1]: ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-11T01:25:10.217 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-11T01:25:10.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:25:10.252 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-11T01:25:10.252 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T01:25:10.252 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.0 2024-09-11T01:25:10.747 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:10 smithi078 systemd[1]: Stopping Ceph osd.0 for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:25:11.247 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:10 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0[46957]: 2024-09-11T01:25:10.917+0000 7fe824236640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:25:11.247 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:10 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0[46957]: 2024-09-11T01:25:10.917+0000 7fe824236640 -1 osd.0 118 *** Got signal Terminated *** 2024-09-11T01:25:11.247 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:10 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0[46957]: 2024-09-11T01:25:10.917+0000 7fe824236640 -1 osd.0 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T01:25:16.604 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:16 smithi078 podman[130196]: 2024-09-11 01:25:16.306796972 +0000 UTC m=+5.864371470 container died 00a7ca0c9ebbb6c54d41b80b36cb3ece3346b148b6771614c16f81f0d7066027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T01:25:16.937 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:16 smithi078 podman[130196]: 2024-09-11 01:25:16.777636234 +0000 UTC m=+6.335210731 container cleanup 00a7ca0c9ebbb6c54d41b80b36cb3ece3346b148b6771614c16f81f0d7066027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T01:25:16.937 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:16 smithi078 bash[130196]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0 2024-09-11T01:25:17.247 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:17 smithi078 podman[130213]: 2024-09-11 01:25:17.096147808 +0000 UTC m=+0.770183354 container remove 00a7ca0c9ebbb6c54d41b80b36cb3ece3346b148b6771614c16f81f0d7066027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T01:25:17.727 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:17 smithi078 podman[130297]: 2024-09-11 01:25:17.353243636 +0000 UTC m=+0.015885241 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T01:25:17.727 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:17 smithi078 podman[130297]: 2024-09-11 01:25:17.480672472 +0000 UTC m=+0.143314068 container create 7dc260770b4191f6ac0cddbc5fdded5cfc7f6f44f8fa554ad6007c09f9ee5fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T01:25:17.997 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:17 smithi078 podman[130297]: 2024-09-11 01:25:17.725192071 +0000 UTC m=+0.387833671 container init 7dc260770b4191f6ac0cddbc5fdded5cfc7f6f44f8fa554ad6007c09f9ee5fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T01:25:17.997 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:17 smithi078 podman[130297]: 2024-09-11 01:25:17.729043798 +0000 UTC m=+0.391685399 container start 7dc260770b4191f6ac0cddbc5fdded5cfc7f6f44f8fa554ad6007c09f9ee5fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T01:25:17.997 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:17 smithi078 podman[130297]: 2024-09-11 01:25:17.833402473 +0000 UTC m=+0.496044070 container attach 7dc260770b4191f6ac0cddbc5fdded5cfc7f6f44f8fa554ad6007c09f9ee5fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T01:25:18.329 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 11 01:25:18 smithi078 podman[130297]: 2024-09-11 01:25:18.057121223 +0000 UTC m=+0.719762828 container died 7dc260770b4191f6ac0cddbc5fdded5cfc7f6f44f8fa554ad6007c09f9ee5fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T01:25:18.764 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T01:25:18.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:25:18.803 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T01:25:18.803 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T01:25:18.803 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.1 2024-09-11T01:25:19.110 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:18 smithi110 systemd[1]: Stopping Ceph osd.1 for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:25:19.440 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:19 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1[40292]: 2024-09-11T01:25:19.107+0000 7f1313ac9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:25:19.441 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:19 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1[40292]: 2024-09-11T01:25:19.107+0000 7f1313ac9640 -1 osd.1 118 *** Got signal Terminated *** 2024-09-11T01:25:19.441 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:19 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1[40292]: 2024-09-11T01:25:19.107+0000 7f1313ac9640 -1 osd.1 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T01:25:25.045 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:24 smithi110 podman[74999]: 2024-09-11 01:25:24.733675654 +0000 UTC m=+5.743838109 container died 4eae4a626bdf9aa331016b469b52549006a50da524ae65d5cbb150f759328281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:25:25.396 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:25 smithi110 podman[74999]: 2024-09-11 01:25:25.142455343 +0000 UTC m=+6.152617798 container cleanup 4eae4a626bdf9aa331016b469b52549006a50da524ae65d5cbb150f759328281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T01:25:25.396 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:25 smithi110 bash[74999]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1 2024-09-11T01:25:25.690 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:25 smithi110 podman[75014]: 2024-09-11 01:25:25.394260896 +0000 UTC m=+0.663693795 container remove 4eae4a626bdf9aa331016b469b52549006a50da524ae65d5cbb150f759328281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T01:25:26.082 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:25 smithi110 podman[75102]: 2024-09-11 01:25:25.707065031 +0000 UTC m=+0.016001381 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T01:25:26.082 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:25 smithi110 podman[75102]: 2024-09-11 01:25:25.828470727 +0000 UTC m=+0.137407067 container create 1b88585b1e44f03623c9cb3d04154adc2ffb9129e21ef241001e2ae6456c4fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:25:26.082 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:26 smithi110 podman[75102]: 2024-09-11 01:25:26.081333662 +0000 UTC m=+0.390270002 container init 1b88585b1e44f03623c9cb3d04154adc2ffb9129e21ef241001e2ae6456c4fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T01:25:26.393 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:26 smithi110 podman[75102]: 2024-09-11 01:25:26.084784382 +0000 UTC m=+0.393720734 container start 1b88585b1e44f03623c9cb3d04154adc2ffb9129e21ef241001e2ae6456c4fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T01:25:26.394 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:26 smithi110 podman[75102]: 2024-09-11 01:25:26.181258855 +0000 UTC m=+0.490195208 container attach 1b88585b1e44f03623c9cb3d04154adc2ffb9129e21ef241001e2ae6456c4fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T01:25:26.690 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 01:25:26 smithi110 podman[75102]: 2024-09-11 01:25:26.392746757 +0000 UTC m=+0.701683105 container died 1b88585b1e44f03623c9cb3d04154adc2ffb9129e21ef241001e2ae6456c4fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T01:25:27.053 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T01:25:27.093 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:25:27.094 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T01:25:27.094 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T01:25:27.094 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.2 2024-09-11T01:25:27.410 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:27 smithi112 systemd[1]: Stopping Ceph osd.2 for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:25:27.724 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:27 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2[41413]: 2024-09-11T01:25:27.406+0000 7f125871d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:25:27.724 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:27 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2[41413]: 2024-09-11T01:25:27.406+0000 7f125871d640 -1 osd.2 118 *** Got signal Terminated *** 2024-09-11T01:25:27.724 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:27 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2[41413]: 2024-09-11T01:25:27.406+0000 7f125871d640 -1 osd.2 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T01:25:33.224 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:32 smithi112 podman[76248]: 2024-09-11 01:25:32.966008836 +0000 UTC m=+5.680468008 container died 7feeff3817842e4d4c31c63400c5ff5854208f89e288e95a98dc66d3a45f2f2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:25:33.661 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:33 smithi112 podman[76248]: 2024-09-11 01:25:33.4003304 +0000 UTC m=+6.114789571 container cleanup 7feeff3817842e4d4c31c63400c5ff5854208f89e288e95a98dc66d3a45f2f2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:25:33.661 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:33 smithi112 bash[76248]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2 2024-09-11T01:25:33.974 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:33 smithi112 podman[76264]: 2024-09-11 01:25:33.660176792 +0000 UTC m=+0.691870268 container remove 7feeff3817842e4d4c31c63400c5ff5854208f89e288e95a98dc66d3a45f2f2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:25:34.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:34 smithi112 podman[76350]: 2024-09-11 01:25:33.955832768 +0000 UTC m=+0.015767346 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T01:25:34.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:34 smithi112 podman[76350]: 2024-09-11 01:25:34.086201281 +0000 UTC m=+0.146135849 container create 223c3477d3178012e53355923dd24d8b8720c837ff42ebc69c0494b3724a8f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T01:25:34.724 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:34 smithi112 podman[76350]: 2024-09-11 01:25:34.338858903 +0000 UTC m=+0.398793475 container init 223c3477d3178012e53355923dd24d8b8720c837ff42ebc69c0494b3724a8f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T01:25:34.724 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:34 smithi112 podman[76350]: 2024-09-11 01:25:34.342228805 +0000 UTC m=+0.402163375 container start 223c3477d3178012e53355923dd24d8b8720c837ff42ebc69c0494b3724a8f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T01:25:34.724 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:34 smithi112 podman[76350]: 2024-09-11 01:25:34.48066037 +0000 UTC m=+0.540595018 container attach 223c3477d3178012e53355923dd24d8b8720c837ff42ebc69c0494b3724a8f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T01:25:35.118 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 11 01:25:34 smithi112 podman[76350]: 2024-09-11 01:25:34.763669047 +0000 UTC m=+0.823603617 container died 223c3477d3178012e53355923dd24d8b8720c837ff42ebc69c0494b3724a8f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:25:35.460 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T01:25:35.500 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:25:35.500 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T01:25:35.500 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T01:25:35.500 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.3 2024-09-11T01:25:35.826 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:35 smithi078 systemd[1]: Stopping Ceph osd.3 for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:25:36.247 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:35 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3[53815]: 2024-09-11T01:25:35.824+0000 7f5110472640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:25:36.247 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:35 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3[53815]: 2024-09-11T01:25:35.824+0000 7f5110472640 -1 osd.3 118 *** Got signal Terminated *** 2024-09-11T01:25:36.247 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:35 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3[53815]: 2024-09-11T01:25:35.824+0000 7f5110472640 -1 osd.3 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T01:25:37.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:37 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:37.391+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:37.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:37 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:37.497+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:37.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:37 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:37.512+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:38.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:38 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:38.374+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:38.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:38 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:38.536+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:38.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:38 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:38.487+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:38.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:38 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:38.539+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:39.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:39 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:39.414+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:39.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:39 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:39.553+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:39.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:39 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:39.446+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:39.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:39 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:39.492+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:40.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:40 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:40.400+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:40.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:40 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:40.207+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:40.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:40 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:40.368+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:40.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:40 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:40.516+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:40.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:40 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:40.517+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:41.497 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:41 smithi078 podman[130411]: 2024-09-11 01:25:41.490422464 +0000 UTC m=+5.800867484 container died 4d2a6e3898ba3d4cd2c1eec732af983a71367bd5114bf6da8ca79c439362811a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T01:25:41.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:41 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:41.422+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:41.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:41 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:41.243+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:41.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:41 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:41.405+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:41.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:41 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:41.544+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:41.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:41 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:41.560+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:42.143 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:41 smithi078 podman[130411]: 2024-09-11 01:25:41.890884658 +0000 UTC m=+6.201329684 container cleanup 4d2a6e3898ba3d4cd2c1eec732af983a71367bd5114bf6da8ca79c439362811a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T01:25:42.143 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:41 smithi078 bash[130411]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3 2024-09-11T01:25:42.450 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:42 smithi078 podman[130438]: 2024-09-11 01:25:42.142470722 +0000 UTC m=+0.647099507 container remove 4d2a6e3898ba3d4cd2c1eec732af983a71367bd5114bf6da8ca79c439362811a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T01:25:42.450 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:42 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:42.246+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:42.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:42 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:42.445+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:42.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:42 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:42.503+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:42.739 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:42 smithi078 podman[130529]: 2024-09-11 01:25:42.35438237 +0000 UTC m=+0.015858407 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T01:25:42.739 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:42 smithi078 podman[130529]: 2024-09-11 01:25:42.45140401 +0000 UTC m=+0.112880047 container create cf4c4e896fa5bf5356bd1db5761d9d6837ec94f0e39a54d82bb18bdb8c95acb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD) 2024-09-11T01:25:42.739 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:42 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:42.449+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:42.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:42 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:42.538+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:42.997 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:42 smithi078 podman[130529]: 2024-09-11 01:25:42.737724676 +0000 UTC m=+0.399200710 container init cf4c4e896fa5bf5356bd1db5761d9d6837ec94f0e39a54d82bb18bdb8c95acb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:25:42.997 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:42 smithi078 podman[130529]: 2024-09-11 01:25:42.741106232 +0000 UTC m=+0.402582274 container start cf4c4e896fa5bf5356bd1db5761d9d6837ec94f0e39a54d82bb18bdb8c95acb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T01:25:42.997 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:42 smithi078 podman[130529]: 2024-09-11 01:25:42.846060497 +0000 UTC m=+0.507536532 container attach cf4c4e896fa5bf5356bd1db5761d9d6837ec94f0e39a54d82bb18bdb8c95acb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:25:42.998 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 11 01:25:42 smithi078 podman[130529]: 2024-09-11 01:25:42.942281697 +0000 UTC m=+0.603757735 container died cf4c4e896fa5bf5356bd1db5761d9d6837ec94f0e39a54d82bb18bdb8c95acb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T01:25:43.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:43 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:43.294+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:43.498 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:43 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:43.461+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:43.601 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T01:25:43.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:25:43.643 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T01:25:43.643 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T01:25:43.643 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.4 2024-09-11T01:25:43.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:43 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:43.459+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:43.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:43 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:43.499+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:43.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:43 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:43.543+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:43.941 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:43 smithi110 systemd[1]: Stopping Ceph osd.4 for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:25:44.440 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:43 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4[46754]: 2024-09-11T01:25:43.954+0000 7f43b12be640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:25:44.441 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:43 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4[46754]: 2024-09-11T01:25:43.954+0000 7f43b12be640 -1 osd.4 118 *** Got signal Terminated *** 2024-09-11T01:25:44.441 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:43 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4[46754]: 2024-09-11T01:25:43.954+0000 7f43b12be640 -1 osd.4 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T01:25:44.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:44 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:44.453+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:44.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:44 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:44.493+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:44.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:44 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:44.429+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:44.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:44 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:44.283+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:44.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:44 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:44.550+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:45.690 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:45 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4[46754]: 2024-09-11T01:25:45.438+0000 7f43ad8d3640 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:20.919453+0000 front 2024-09-11T01:25:20.919486+0000 (oldest deadline 2024-09-11T01:25:44.920240+0000) 2024-09-11T01:25:45.691 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:45 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:45.566+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:45.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:45 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:45.445+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:45.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:45 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:45.494+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:45.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:45 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:45.494+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:45.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:45 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:45.405+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:45.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:45 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:45.263+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:45.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:45 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:45.263+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:46.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:46 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:46.452+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:46.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:46 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:46.219+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:46.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:46 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:46.219+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:46.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:46 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:46.461+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:46.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:46 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:46.461+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.011827+0000 front 2024-09-11T01:25:21.011797+0000 (oldest deadline 2024-09-11T01:25:45.711900+0000) 2024-09-11T01:25:46.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:46 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:46.473+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:46.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:46 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:46.473+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:46.863 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:46 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4[46754]: 2024-09-11T01:25:46.443+0000 7f43ad8d3640 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:20.919453+0000 front 2024-09-11T01:25:20.919486+0000 (oldest deadline 2024-09-11T01:25:44.920240+0000) 2024-09-11T01:25:46.863 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:46 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:46.611+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:47.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:47 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:47.412+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:47.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:47 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:47.221+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:47.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:47 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:47.221+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:47.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:47 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:47.447+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:47.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:47 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:47.447+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.011827+0000 front 2024-09-11T01:25:21.011797+0000 (oldest deadline 2024-09-11T01:25:45.711900+0000) 2024-09-11T01:25:47.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:47 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:47.448+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:47.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:47 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:47.448+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:47.812 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:47 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:47.585+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:47.813 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:47 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4[46754]: 2024-09-11T01:25:47.458+0000 7f43ad8d3640 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:20.919453+0000 front 2024-09-11T01:25:20.919486+0000 (oldest deadline 2024-09-11T01:25:44.920240+0000) 2024-09-11T01:25:48.458 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:48 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:48.172+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:48.458 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:48 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:48.172+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:48.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:48 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:48.403+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:48.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:48 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:48.403+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.011827+0000 front 2024-09-11T01:25:21.011797+0000 (oldest deadline 2024-09-11T01:25:45.711900+0000) 2024-09-11T01:25:48.724 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:48 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:48.602+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:25:48.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:48 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:48.472+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:48.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:48 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:48.472+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:48.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:48 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:48.456+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:48.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:48 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:48.456+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:25:48.940 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:48 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4[46754]: 2024-09-11T01:25:48.453+0000 7f43ad8d3640 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:20.919453+0000 front 2024-09-11T01:25:20.919486+0000 (oldest deadline 2024-09-11T01:25:44.920240+0000) 2024-09-11T01:25:48.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:48 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:48.563+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:48.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:48 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:48.563+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:25:49.432 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:49 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:49.141+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:49.432 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:49 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:49.141+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:49.652 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:49 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:49.449+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:49.652 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:49 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:49.449+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:49.652 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:49 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:49.369+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:49.652 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:49 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:49.369+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.011827+0000 front 2024-09-11T01:25:21.011797+0000 (oldest deadline 2024-09-11T01:25:45.711900+0000) 2024-09-11T01:25:49.666 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:49 smithi110 podman[75203]: 2024-09-11 01:25:49.388690969 +0000 UTC m=+5.551966647 container died 17d77cbbf603d9367c72afafdaa7a97d1225a8cb7927ac1bbee31d58a20c05fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T01:25:49.666 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:49 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:49.604+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:49.666 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:49 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:49.604+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:25:49.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:49 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:49.430+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:49.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:49 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:49.430+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:25:49.940 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:49 smithi110 podman[75203]: 2024-09-11 01:25:49.768841512 +0000 UTC m=+5.932117190 container cleanup 17d77cbbf603d9367c72afafdaa7a97d1225a8cb7927ac1bbee31d58a20c05fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T01:25:49.940 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:49 smithi110 bash[75203]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4 2024-09-11T01:25:49.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:49 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:49.743+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:25:49.976 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:49 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:49.650+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:25:50.375 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:50 smithi110 podman[75230]: 2024-09-11 01:25:50.058205091 +0000 UTC m=+0.672266023 container remove 17d77cbbf603d9367c72afafdaa7a97d1225a8cb7927ac1bbee31d58a20c05fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T01:25:50.454 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:50 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:50.181+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:50.454 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:50 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:50.181+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:50.638 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:50 smithi110 podman[75313]: 2024-09-11 01:25:50.277350739 +0000 UTC m=+0.016048326 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T01:25:50.638 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:50 smithi110 podman[75313]: 2024-09-11 01:25:50.392592559 +0000 UTC m=+0.131290175 container create 8b16750cdf4c5cad50117c00f7f14e589b3f102e60f77a8a8d2c96e39e074232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T01:25:50.638 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:50 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:50.621+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:50.638 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:50 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:50.621+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:25:50.647 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:50 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:50.359+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:50.647 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:50 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:50.359+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.011827+0000 front 2024-09-11T01:25:21.011797+0000 (oldest deadline 2024-09-11T01:25:45.711900+0000) 2024-09-11T01:25:50.647 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:50 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:50.474+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:50.647 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:50 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:50.474+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:50.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:50 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:50.453+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:50.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:50 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:50.453+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:25:50.941 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:50 smithi110 podman[75313]: 2024-09-11 01:25:50.636808658 +0000 UTC m=+0.375506235 container init 8b16750cdf4c5cad50117c00f7f14e589b3f102e60f77a8a8d2c96e39e074232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T01:25:50.941 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:50 smithi110 podman[75313]: 2024-09-11 01:25:50.640279486 +0000 UTC m=+0.378977073 container start 8b16750cdf4c5cad50117c00f7f14e589b3f102e60f77a8a8d2c96e39e074232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T01:25:50.941 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:50 smithi110 podman[75313]: 2024-09-11 01:25:50.728556169 +0000 UTC m=+0.467253744 container attach 8b16750cdf4c5cad50117c00f7f14e589b3f102e60f77a8a8d2c96e39e074232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T01:25:50.941 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 11 01:25:50 smithi110 podman[75313]: 2024-09-11 01:25:50.835013579 +0000 UTC m=+0.573711170 container died 8b16750cdf4c5cad50117c00f7f14e589b3f102e60f77a8a8d2c96e39e074232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T01:25:50.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:50 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:50.721+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:25:50.974 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:50 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:50.645+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:25:51.447 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:51 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:51.138+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:51.448 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:51 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:51.138+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:51.543 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T01:25:51.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:25:51.585 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T01:25:51.586 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T01:25:51.586 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.5 2024-09-11T01:25:51.637 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:51 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:51.370+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:51.637 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:51 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:51.370+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.011827+0000 front 2024-09-11T01:25:21.011797+0000 (oldest deadline 2024-09-11T01:25:45.711900+0000) 2024-09-11T01:25:51.637 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:51 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:51.518+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:51.637 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:51 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:51.518+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:51.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:51 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:51.446+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:51.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:51 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:51.446+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:25:51.897 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:51 smithi112 systemd[1]: Stopping Ceph osd.5 for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:25:51.898 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:51 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:51.657+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:25:51.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:51 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:51.717+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:25:51.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:51 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:51.664+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:51.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:51 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:51.664+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:25:52.224 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:51 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:51.895+0000 7f20f2f22640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:25:52.224 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:51 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:51.895+0000 7f20f2f22640 -1 osd.5 118 *** Got signal Terminated *** 2024-09-11T01:25:52.224 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:51 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:51.895+0000 7f20f2f22640 -1 osd.5 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T01:25:52.420 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:52 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:52.151+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:52.420 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:52 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:52.151+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:52.680 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:52.325+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:52.680 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:52.325+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.011827+0000 front 2024-09-11T01:25:21.011797+0000 (oldest deadline 2024-09-11T01:25:45.711900+0000) 2024-09-11T01:25:52.680 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:52.507+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:52.681 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:52.507+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:52.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:52 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:52.418+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:52.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:52 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:52.418+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:25:52.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:52.705+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:25:52.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:52.627+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:52.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:52 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:52.627+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:25:52.974 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:52 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:52.678+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:25:53.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:53 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:53.176+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:53.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:53 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:53.176+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:53.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:53 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:53.403+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:53.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:53 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:53.403+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:25:53.648 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:53 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:53.354+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:53.648 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:53 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:53.354+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.011827+0000 front 2024-09-11T01:25:21.011797+0000 (oldest deadline 2024-09-11T01:25:45.711900+0000) 2024-09-11T01:25:53.648 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:53 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:53.471+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:53.649 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:53 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:53.471+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:53.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:53 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:53.738+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:25:53.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:53 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:53.606+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:53.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:53 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:53.606+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:25:53.974 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:53 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:53.646+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:25:54.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:54.359+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:54.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:54.359+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:25:54.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:54.223+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:54.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:54 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:54.223+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:54.694 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:54 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:54.328+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:54.695 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:54 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:54.328+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.011827+0000 front 2024-09-11T01:25:21.011797+0000 (oldest deadline 2024-09-11T01:25:45.711900+0000) 2024-09-11T01:25:54.695 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:54 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:54.473+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:54.695 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:54 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:54.473+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:54.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:54 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:54.763+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:25:54.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:54 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:54.763+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:25:54.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:54 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:54.617+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:54.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:54 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:54.617+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:25:54.974 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:54 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:54.693+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:25:55.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:55 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:55.206+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:55.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:55 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:55.206+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:55.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:55 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:55.206+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.742230+0000 front 2024-09-11T01:25:30.742341+0000 (oldest deadline 2024-09-11T01:25:54.242373+0000) 2024-09-11T01:25:55.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:55 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:55.360+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:55.498 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:55 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:55.360+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:25:55.498 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:55 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:55.360+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:25:55.681 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:55 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:55.310+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:55.681 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:55 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:55.310+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.011827+0000 front 2024-09-11T01:25:21.011797+0000 (oldest deadline 2024-09-11T01:25:45.711900+0000) 2024-09-11T01:25:55.681 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:55 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:55.436+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:55.681 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:55 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:55.436+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:55.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:55 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:55.713+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:25:55.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:55 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:55.713+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:25:55.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:55 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:55.636+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:55.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:55 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:55.636+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:25:55.974 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:55 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:55.732+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:25:55.974 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:55 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:55.732+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:25:56.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:56 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:56.300+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.610619+0000 front 2024-09-11T01:25:11.610597+0000 (oldest deadline 2024-09-11T01:25:36.910627+0000) 2024-09-11T01:25:56.724 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:56 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5[48146]: 2024-09-11T01:25:56.300+0000 7f20eb52f640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.011827+0000 front 2024-09-11T01:25:21.011797+0000 (oldest deadline 2024-09-11T01:25:45.711900+0000) 2024-09-11T01:25:56.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:56 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:56.477+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:56.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:56 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:56.477+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:56.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:56.246+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:56.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:56.246+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:56.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:56.246+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.742230+0000 front 2024-09-11T01:25:30.742341+0000 (oldest deadline 2024-09-11T01:25:54.242373+0000) 2024-09-11T01:25:56.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:56.310+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:56.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:56.310+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:25:56.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:56.310+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:25:56.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:56 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:56.677+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:25:56.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:56 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:56.677+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:25:56.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:56 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:56.656+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:56.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:56 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:56.656+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:25:56.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:56 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:56.656+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:25:57.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:56 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:56.735+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:25:57.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:56 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:56.735+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:25:57.692 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:57 smithi112 podman[76467]: 2024-09-11 01:25:57.532920447 +0000 UTC m=+5.748914013 container died 58ebd9a95a3d363d1cd3eddb675285621eac9dd5160120f3cbad70aca32e653a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T01:25:57.696 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:57 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:57.435+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:57.696 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:57 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:57.435+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:57.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:57 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:57.261+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:57.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:57 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:57.261+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:57.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:57 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:57.261+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.742230+0000 front 2024-09-11T01:25:30.742341+0000 (oldest deadline 2024-09-11T01:25:54.242373+0000) 2024-09-11T01:25:57.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:57 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:57.338+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:57.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:57 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:57.338+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:25:57.748 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:57 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:57.338+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:25:57.885 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:57 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:57.704+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:25:57.885 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:57 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:57.704+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:25:57.885 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:57 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:57.628+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:57.885 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:57 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:57.628+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:25:57.885 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:57 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:57.628+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:25:57.974 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:57 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:57.691+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:25:57.974 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:57 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:57.691+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:25:57.974 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:57 smithi112 podman[76467]: 2024-09-11 01:25:57.92117854 +0000 UTC m=+6.137172109 container cleanup 58ebd9a95a3d363d1cd3eddb675285621eac9dd5160120f3cbad70aca32e653a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T01:25:57.975 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:57 smithi112 bash[76467]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5 2024-09-11T01:25:58.474 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:58 smithi112 podman[76482]: 2024-09-11 01:25:58.197862883 +0000 UTC m=+0.665081249 container remove 58ebd9a95a3d363d1cd3eddb675285621eac9dd5160120f3cbad70aca32e653a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:25:58.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:58 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:58.424+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:58.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:58 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:58.424+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:58.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:58 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:58.424+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:25:58.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:58 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:58.237+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:58.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:58 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:58.237+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:58.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:58 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:58.237+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.742230+0000 front 2024-09-11T01:25:30.742341+0000 (oldest deadline 2024-09-11T01:25:54.242373+0000) 2024-09-11T01:25:58.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:58 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:58.361+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:58.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:58 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:58.361+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:25:58.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:58 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:58.361+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:25:58.818 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:58 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:58.739+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:25:58.818 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:58 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:58.739+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:25:58.819 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:58 smithi112 podman[76570]: 2024-09-11 01:25:58.431474218 +0000 UTC m=+0.016101346 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T01:25:58.819 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:58 smithi112 podman[76570]: 2024-09-11 01:25:58.54019391 +0000 UTC m=+0.124821026 container create 4bb3f43d43dde80bd75973003d0ed4d3643a5be3a9febf87507f2722e3877c01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T01:25:58.819 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:58 smithi112 podman[76570]: 2024-09-11 01:25:58.817640152 +0000 UTC m=+0.402267267 container init 4bb3f43d43dde80bd75973003d0ed4d3643a5be3a9febf87507f2722e3877c01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T01:25:58.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:58 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:58.677+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:25:58.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:58 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:58.677+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:25:58.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:58 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:58.585+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:58.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:58 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:58.585+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:25:58.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:58 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:58.585+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:25:59.224 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:58 smithi112 podman[76570]: 2024-09-11 01:25:58.82093785 +0000 UTC m=+0.405564966 container start 4bb3f43d43dde80bd75973003d0ed4d3643a5be3a9febf87507f2722e3877c01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:25:59.224 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:58 smithi112 podman[76570]: 2024-09-11 01:25:58.917720778 +0000 UTC m=+0.502347898 container attach 4bb3f43d43dde80bd75973003d0ed4d3643a5be3a9febf87507f2722e3877c01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T01:25:59.224 INFO:journalctl@ceph.osd.5.smithi112.stdout:Sep 11 01:25:59 smithi112 podman[76570]: 2024-09-11 01:25:59.01053933 +0000 UTC m=+0.595166459 container died 4bb3f43d43dde80bd75973003d0ed4d3643a5be3a9febf87507f2722e3877c01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T01:25:59.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:59 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:59.217+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:25:59.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:59 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:59.217+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:25:59.497 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:59 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:25:59.217+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.742230+0000 front 2024-09-11T01:25:30.742341+0000 (oldest deadline 2024-09-11T01:25:54.242373+0000) 2024-09-11T01:25:59.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:59 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:59.370+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:25:59.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:59 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:59.370+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:25:59.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:25:59 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:25:59.370+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:25:59.698 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:59 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:59.381+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:25:59.698 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:59 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:59.381+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:25:59.698 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:25:59 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:25:59.381+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:25:59.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T01:25:59.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:25:59.763 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T01:25:59.763 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T01:25:59.763 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.6 2024-09-11T01:25:59.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:59 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:59.644+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:25:59.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:25:59 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:25:59.644+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:25:59.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:59 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:59.625+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:25:59.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:59 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:59.625+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:25:59.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:25:59 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:25:59.625+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:25:59.974 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:59 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:59.770+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:25:59.974 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:25:59 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:25:59.770+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:00.062 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:25:59 smithi078 systemd[1]: Stopping Ceph osd.6 for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:26:00.405 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:00.060+0000 7fce32703640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:26:00.405 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:00.060+0000 7fce32703640 -1 osd.6 118 *** Got signal Terminated *** 2024-09-11T01:26:00.405 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:00.060+0000 7fce32703640 -1 osd.6 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T01:26:00.405 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:00.253+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:26:00.406 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:00.253+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:26:00.406 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:00.253+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.742230+0000 front 2024-09-11T01:25:30.742341+0000 (oldest deadline 2024-09-11T01:25:54.242373+0000) 2024-09-11T01:26:00.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:00 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:00.380+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:00.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:00 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:00.380+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:00.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:00 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:00.380+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:00.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:00.403+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:00.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:00.403+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:00.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:00 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:00.403+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:00.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:00 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:00.637+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:00.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:00 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:00.637+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:00.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:00 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:00.670+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:00.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:00 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:00.670+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:00.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:00 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:00.670+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:01.172 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:00 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:00.809+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:01.172 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:00 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:00.809+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:01.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:01 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:01.346+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:01.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:01 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:01.346+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:01.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:01 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:01.346+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:01.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:01 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:01.292+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:26:01.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:01 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:01.292+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:26:01.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:01 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:01.292+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.742230+0000 front 2024-09-11T01:25:30.742341+0000 (oldest deadline 2024-09-11T01:25:54.242373+0000) 2024-09-11T01:26:01.748 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:01 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:01.391+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:01.748 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:01 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:01.391+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:01.748 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:01 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:01.391+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:01.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:01 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:01.639+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:01.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:01 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:01.639+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:01.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:01 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:01.712+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:01.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:01 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:01.712+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:01.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:01 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:01.712+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:02.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:01 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:01.790+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:02.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:01 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:01.790+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:02.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:02 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:02.305+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:02.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:02 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:02.305+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:02.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:02 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:02.305+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:02.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:02 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:02.329+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:26:02.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:02 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:02.329+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:26:02.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:02 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:02.329+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.742230+0000 front 2024-09-11T01:25:30.742341+0000 (oldest deadline 2024-09-11T01:25:54.242373+0000) 2024-09-11T01:26:02.748 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:02 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:02.418+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:02.748 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:02 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:02.418+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:02.748 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:02 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:02.418+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:02.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:02 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:02.605+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:02.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:02 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:02.605+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:02.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:02 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:02.692+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:02.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:02 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:02.692+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:02.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:02 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:02.692+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:02.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:02 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:02.692+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:03.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:02 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:02.750+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:03.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:02 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:02.750+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:03.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:03 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:03.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:03.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:03 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:03.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:03.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:03 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:03.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:03.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:03 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:03.284+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:26:03.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:03 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:03.284+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:26:03.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:03 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:03.284+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.742230+0000 front 2024-09-11T01:25:30.742341+0000 (oldest deadline 2024-09-11T01:25:54.242373+0000) 2024-09-11T01:26:03.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:03 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:03.439+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:03.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:03 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:03.439+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:03.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:03 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:03.439+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:03.748 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:03 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:03.439+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:03.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:03 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:03.646+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:03.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:03 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:03.646+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:03.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:03 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:03.646+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.084136+0000 front 2024-09-11T01:25:40.083928+0000 (oldest deadline 2024-09-11T01:26:03.484618+0000) 2024-09-11T01:26:03.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:03 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:03.670+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:03.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:03 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:03.670+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:03.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:03 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:03.670+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:03.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:03 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:03.670+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:04.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:03 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:03.744+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:04.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:03 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:03.744+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:04.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:04 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:04.306+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:04.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:04 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:04.306+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:04.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:04 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:04.306+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:04.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:04 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:04.296+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:15.639124+0000 front 2024-09-11T01:25:15.639094+0000 (oldest deadline 2024-09-11T01:25:39.640011+0000) 2024-09-11T01:26:04.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:04 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:04.296+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:19.640668+0000 front 2024-09-11T01:25:19.640721+0000 (oldest deadline 2024-09-11T01:25:44.940665+0000) 2024-09-11T01:26:04.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:04 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:04.296+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.742230+0000 front 2024-09-11T01:25:30.742341+0000 (oldest deadline 2024-09-11T01:25:54.242373+0000) 2024-09-11T01:26:04.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:04 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6[60608]: 2024-09-11T01:26:04.296+0000 7fce2ed18640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.444288+0000 front 2024-09-11T01:25:39.444312+0000 (oldest deadline 2024-09-11T01:26:03.544452+0000) 2024-09-11T01:26:04.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:04 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:04.478+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:04.748 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:04 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:04.478+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:04.748 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:04 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:04.478+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:04.748 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:04 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:04.478+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:04.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:04 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:04.632+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:04.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:04 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:04.632+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:04.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:04 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:04.632+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.084136+0000 front 2024-09-11T01:25:40.083928+0000 (oldest deadline 2024-09-11T01:26:03.484618+0000) 2024-09-11T01:26:04.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:04 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:04.675+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:04.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:04 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:04.675+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:04.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:04 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:04.675+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:04.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:04 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:04.675+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:05.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:04 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:04.783+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:05.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:04 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:04.783+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:05.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:05 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:05.262+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:05.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:05 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:05.262+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:05.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:05 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:05.262+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:05.747 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:05 smithi078 podman[130631]: 2024-09-11 01:26:05.568044381 +0000 UTC m=+5.608619329 container died 5e8aebc0c442999b33f1774c1918c424f9a1559cc4857b7785bca0910db484c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True) 2024-09-11T01:26:05.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:05 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:05.477+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:05.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:05 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:05.477+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:05.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:05 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:05.477+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:05.747 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:05 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:05.477+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:05.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:05 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:05.601+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:05.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:05 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:05.601+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:05.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:05 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:05.601+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.084136+0000 front 2024-09-11T01:25:40.083928+0000 (oldest deadline 2024-09-11T01:26:03.484618+0000) 2024-09-11T01:26:05.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:05 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:05.637+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:05.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:05 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:05.637+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:05.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:05 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:05.637+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:05.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:05 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:05.637+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:06.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:05 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:05.822+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:06.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:05 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:05.822+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:06.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:05 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:05.822+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:06.347 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:06 smithi078 podman[130631]: 2024-09-11 01:26:06.069403951 +0000 UTC m=+6.109978850 container cleanup 5e8aebc0c442999b33f1774c1918c424f9a1559cc4857b7785bca0910db484c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-11T01:26:06.347 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:06 smithi078 bash[130631]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6 2024-09-11T01:26:06.695 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:06 smithi078 podman[130648]: 2024-09-11 01:26:06.34597446 +0000 UTC m=+0.770835921 container remove 5e8aebc0c442999b33f1774c1918c424f9a1559cc4857b7785bca0910db484c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T01:26:06.696 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:06 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:06.525+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:06.696 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:06 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:06.525+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:06.696 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:06 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:06.525+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:06.696 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:06 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:06.525+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:06.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:06 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:06.284+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:06.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:06 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:06.284+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:06.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:06 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:06.284+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:06.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:06 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:06.603+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:06.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:06 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:06.603+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:06.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:06 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:06.603+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.084136+0000 front 2024-09-11T01:25:40.083928+0000 (oldest deadline 2024-09-11T01:26:03.484618+0000) 2024-09-11T01:26:06.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:06 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:06.610+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:06.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:06 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:06.610+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:06.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:06 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:06.610+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:06.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:06 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:06.610+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:06.992 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:06 smithi078 podman[130747]: 2024-09-11 01:26:06.596996009 +0000 UTC m=+0.015988446 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T01:26:06.992 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:06 smithi078 podman[130747]: 2024-09-11 01:26:06.738543624 +0000 UTC m=+0.157536046 container create 2d8c3115d57602d37c8761d1c1c6d57b148ff425f16bf830dacc8884624ec4b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T01:26:06.992 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:06 smithi078 podman[130747]: 2024-09-11 01:26:06.990755173 +0000 UTC m=+0.409747597 container init 2d8c3115d57602d37c8761d1c1c6d57b148ff425f16bf830dacc8884624ec4b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T01:26:07.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:06 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:06.860+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:07.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:06 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:06.860+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:07.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:06 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:06.860+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:07.247 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:06 smithi078 podman[130747]: 2024-09-11 01:26:06.994329044 +0000 UTC m=+0.413321473 container start 2d8c3115d57602d37c8761d1c1c6d57b148ff425f16bf830dacc8884624ec4b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T01:26:07.247 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:07 smithi078 podman[130747]: 2024-09-11 01:26:07.082946887 +0000 UTC m=+0.501939325 container attach 2d8c3115d57602d37c8761d1c1c6d57b148ff425f16bf830dacc8884624ec4b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:26:07.247 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 11 01:26:07 smithi078 podman[130747]: 2024-09-11 01:26:07.183451203 +0000 UTC m=+0.602443635 container died 2d8c3115d57602d37c8761d1c1c6d57b148ff425f16bf830dacc8884624ec4b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-11T01:26:07.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:07 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:07.322+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:07.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:07 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:07.322+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:07.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:07 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:07.322+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:07.837 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:07 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:07.568+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:07.837 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:07 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:07.568+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:07.837 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:07 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:07.568+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:07.837 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:07 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:07.568+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:07.856 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T01:26:07.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:26:07.898 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T01:26:07.898 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T01:26:07.898 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.7 2024-09-11T01:26:07.925 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:07 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:07.623+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:07.925 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:07 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:07.623+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:07.925 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:07 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:07.623+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.084136+0000 front 2024-09-11T01:25:40.083928+0000 (oldest deadline 2024-09-11T01:26:03.484618+0000) 2024-09-11T01:26:07.925 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:07 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:07.567+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:07.926 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:07 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:07.567+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:07.926 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:07 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:07.567+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:07.926 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:07 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:07.567+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:08.190 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:07 smithi110 systemd[1]: Stopping Ceph osd.7 for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:26:08.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:07 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:07.820+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:08.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:07 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:07.820+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:08.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:07 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:07.820+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:08.602 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:08 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:08.218+0000 7f796c68c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:26:08.602 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:08 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:08.218+0000 7f796c68c640 -1 osd.7 118 *** Got signal Terminated *** 2024-09-11T01:26:08.602 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:08 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:08.218+0000 7f796c68c640 -1 osd.7 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T01:26:08.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:08 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:08.347+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:08.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:08 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:08.347+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:08.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:08 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:08.347+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:08.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:08 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:08.602+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:08.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:08 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:08.602+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:08.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:08 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:08.602+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.084136+0000 front 2024-09-11T01:25:40.083928+0000 (oldest deadline 2024-09-11T01:26:03.484618+0000) 2024-09-11T01:26:08.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:08 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:08.600+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:08.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:08 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:08.600+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:08.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:08 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:08.600+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:08.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:08 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:08.600+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:08.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:08 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:08.571+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:08.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:08 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:08.571+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:08.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:08 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:08.571+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:08.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:08 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:08.571+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:09.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:08 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:08.834+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:09.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:08 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:08.834+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:09.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:08 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:08.834+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:09.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:09 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:09.381+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:09.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:09 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:09.381+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:09.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:09 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:09.381+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:09.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:09 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:09.582+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:09.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:09 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:09.582+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:09.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:09 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:09.582+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.084136+0000 front 2024-09-11T01:25:40.083928+0000 (oldest deadline 2024-09-11T01:26:03.484618+0000) 2024-09-11T01:26:09.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:09 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:09.595+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:09.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:09 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:09.595+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:09.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:09 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:09.595+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:09.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:09 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:09.595+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:09.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:09 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:09.562+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:09.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:09 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:09.562+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:09.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:09 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:09.562+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:09.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:09 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:09.562+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:10.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:09 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:09.804+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:10.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:09 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:09.804+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:10.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:09 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:09.804+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:10.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:10 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:10.379+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:10.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:10 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:10.379+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:10.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:10 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:10.379+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:10.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:10 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:10.379+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:10.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:10 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:10.630+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:10.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:10 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:10.630+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:10.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:10 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:10.630+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.084136+0000 front 2024-09-11T01:25:40.083928+0000 (oldest deadline 2024-09-11T01:26:03.484618+0000) 2024-09-11T01:26:10.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:10 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:10.604+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:10.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:10 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:10.604+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:10.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:10 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:10.604+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:10.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:10 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:10.604+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:10.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:10 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:10.597+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:10.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:10 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:10.597+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:10.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:10 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:10.597+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:10.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:10 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:10.597+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:11.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:10 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:10.773+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:11.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:10 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:10.773+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:11.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:10 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:10.773+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:11.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:11 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:11.369+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:11.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:11 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:11.369+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:11.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:11 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:11.369+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:11.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:11 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:11.369+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:11.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:11 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:11.621+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:11.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:11 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:11.621+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:11.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:11 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:11.621+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.084136+0000 front 2024-09-11T01:25:40.083928+0000 (oldest deadline 2024-09-11T01:26:03.484618+0000) 2024-09-11T01:26:11.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:11 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:11.641+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:11.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:11 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:11.641+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:11.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:11 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:11.641+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:11.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:11 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:11.641+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:11.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:11 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:11.608+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:11.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:11 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:11.608+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:11.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:11 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:11.608+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:11.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:11 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:11.608+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:12.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:11 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:11.807+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:12.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:11 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:11.807+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:12.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:11 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:11.807+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:12.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:11 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:11.807+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.540323+0000 front 2024-09-11T01:25:48.540284+0000 (oldest deadline 2024-09-11T01:26:10.840455+0000) 2024-09-11T01:26:12.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:12 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:12.374+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:12.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:12 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:12.374+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:12.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:12 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:12.374+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:12.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:12 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:12.374+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:12.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:12 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:12.614+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.681585+0000 front 2024-09-11T01:25:23.681641+0000 (oldest deadline 2024-09-11T01:25:49.581792+0000) 2024-09-11T01:26:12.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:12 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:12.614+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:29.582283+0000 front 2024-09-11T01:25:29.582191+0000 (oldest deadline 2024-09-11T01:25:54.282303+0000) 2024-09-11T01:26:12.941 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:12 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7[53415]: 2024-09-11T01:26:12.614+0000 7f7964c99640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.084136+0000 front 2024-09-11T01:25:40.083928+0000 (oldest deadline 2024-09-11T01:26:03.484618+0000) 2024-09-11T01:26:12.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:12 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:12.636+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:12.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:12 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:12.636+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:12.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:12 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:12.636+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:12.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:12 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:12.636+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:12.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:12 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:12.585+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:12.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:12 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:12.585+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:12.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:12 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:12.585+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:12.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:12 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:12.585+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:13.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:12 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:12.817+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:13.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:12 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:12.817+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:13.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:12 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:12.817+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:13.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:12 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:12.817+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.540323+0000 front 2024-09-11T01:25:48.540284+0000 (oldest deadline 2024-09-11T01:26:10.840455+0000) 2024-09-11T01:26:13.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:13 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:13.377+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:13.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:13 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:13.377+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:13.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:13 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:13.377+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:13.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:13 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:13.377+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:13.940 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:13 smithi110 podman[75415]: 2024-09-11 01:26:13.750051804 +0000 UTC m=+5.656036834 container died 477616b7556c323d9f142e09361f1059ff303da7d7b0d4aa1762258b93d2decf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T01:26:13.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:13 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:13.685+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:13.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:13 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:13.685+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:13.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:13 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:13.685+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:13.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:13 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:13.685+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:13.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:13 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:13.685+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:13.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:13 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:13.587+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:13.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:13 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:13.587+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:13.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:13 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:13.587+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:13.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:13 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:13.587+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:14.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:13 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:13.831+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:14.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:13 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:13.831+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:14.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:13 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:13.831+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:14.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:13 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:13.831+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.540323+0000 front 2024-09-11T01:25:48.540284+0000 (oldest deadline 2024-09-11T01:26:10.840455+0000) 2024-09-11T01:26:14.440 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:14 smithi110 podman[75415]: 2024-09-11 01:26:14.211266686 +0000 UTC m=+6.117251718 container cleanup 477616b7556c323d9f142e09361f1059ff303da7d7b0d4aa1762258b93d2decf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T01:26:14.440 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:14 smithi110 bash[75415]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7 2024-09-11T01:26:14.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:14 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:14.352+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:14.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:14 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:14.352+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:14.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:14 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:14.352+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:14.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:14 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:14.352+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:14.866 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:14 smithi110 podman[75430]: 2024-09-11 01:26:14.520872079 +0000 UTC m=+0.770174537 container remove 477616b7556c323d9f142e09361f1059ff303da7d7b0d4aa1762258b93d2decf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T01:26:14.867 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:14 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:14.697+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:14.867 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:14 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:14.697+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:14.867 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:14 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:14.697+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:14.867 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:14 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:14.697+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:14.867 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:14 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:14.697+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:14.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:14 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:14.614+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:14.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:14 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:14.614+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:14.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:14 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:14.614+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:14.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:14 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:14.614+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:15.133 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:14 smithi110 podman[75519]: 2024-09-11 01:26:14.768227025 +0000 UTC m=+0.015529857 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T01:26:15.134 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:14 smithi110 podman[75519]: 2024-09-11 01:26:14.89042357 +0000 UTC m=+0.137726429 container create f216667e2bdf18c8891266dfdaa0d10c83215420d3919ffcd4ab98a84faae811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-11T01:26:15.134 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:15 smithi110 podman[75519]: 2024-09-11 01:26:15.132803695 +0000 UTC m=+0.380106512 container init f216667e2bdf18c8891266dfdaa0d10c83215420d3919ffcd4ab98a84faae811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T01:26:15.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:14 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:14.873+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:15.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:14 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:14.873+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:15.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:14 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:14.873+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:15.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:14 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:14.873+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.540323+0000 front 2024-09-11T01:25:48.540284+0000 (oldest deadline 2024-09-11T01:26:10.840455+0000) 2024-09-11T01:26:15.409 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:15 smithi110 podman[75519]: 2024-09-11 01:26:15.136153935 +0000 UTC m=+0.383456755 container start f216667e2bdf18c8891266dfdaa0d10c83215420d3919ffcd4ab98a84faae811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T01:26:15.410 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:15 smithi110 podman[75519]: 2024-09-11 01:26:15.216019787 +0000 UTC m=+0.463322616 container attach f216667e2bdf18c8891266dfdaa0d10c83215420d3919ffcd4ab98a84faae811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-11T01:26:15.410 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 11 01:26:15 smithi110 podman[75519]: 2024-09-11 01:26:15.343552489 +0000 UTC m=+0.590855303 container died f216667e2bdf18c8891266dfdaa0d10c83215420d3919ffcd4ab98a84faae811 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:26:15.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:15 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:15.308+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:15.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:15 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:15.308+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:15.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:15 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:15.308+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:15.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:15 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:15.308+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:15.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:15 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:15.659+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:15.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:15 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:15.659+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:15.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:15 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:15.659+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:15.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:15 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:15.659+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:15.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:15 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:15.659+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:15.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:15 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:15.663+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:15.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:15 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:15.663+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:15.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:15 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:15.663+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:15.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:15 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:15.663+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:16.010 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T01:26:16.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:26:16.089 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T01:26:16.090 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-11T01:26:16.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.8 2024-09-11T01:26:16.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:15 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:15.905+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:16.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:15 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:15.905+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:16.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:15 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:15.905+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:16.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:15 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:15.905+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.540323+0000 front 2024-09-11T01:25:48.540284+0000 (oldest deadline 2024-09-11T01:26:10.840455+0000) 2024-09-11T01:26:16.225 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:16 smithi112 systemd[1]: Stopping Ceph osd.8 for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:26:16.593 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:16 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:16.407+0000 7f2a0d542640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:26:16.593 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:16 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:16.407+0000 7f2a0d542640 -1 osd.8 118 *** Got signal Terminated *** 2024-09-11T01:26:16.593 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:16 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:16.407+0000 7f2a0d542640 -1 osd.8 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T01:26:16.593 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:16 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:16.333+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:16.593 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:16 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:16.333+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:16.593 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:16 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:16.333+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:16.593 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:16 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:16.333+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:16.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:16 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:16.679+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:16.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:16 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:16.679+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:16.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:16 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:16.679+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:16.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:16 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:16.679+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:16.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:16 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:16.679+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:16.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:16.664+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:16.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:16.664+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:16.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:16.664+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:16.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:16 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:16.664+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:17.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:16 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:16.880+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:17.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:16 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:16.880+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:17.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:16 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:16.880+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:17.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:16 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:16.880+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.540323+0000 front 2024-09-11T01:25:48.540284+0000 (oldest deadline 2024-09-11T01:26:10.840455+0000) 2024-09-11T01:26:17.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:17 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:17.364+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:17.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:17 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:17.364+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:17.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:17 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:17.364+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:17.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:17 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:17.364+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:17.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:17 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:17.668+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:17.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:17 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:17.668+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:17.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:17 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:17.668+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:17.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:17 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:17.668+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:17.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:17 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:17.668+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:17.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:17.659+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:17.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:17.659+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:17.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:17.659+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:17.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:17 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:17.659+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:18.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:17 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:17.876+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:18.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:17 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:17.876+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:18.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:17 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:17.876+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:18.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:17 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:17.876+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.540323+0000 front 2024-09-11T01:25:48.540284+0000 (oldest deadline 2024-09-11T01:26:10.840455+0000) 2024-09-11T01:26:18.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:18 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:18.357+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:18.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:18 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:18.357+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:18.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:18 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:18.357+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:18.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:18 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:18.357+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:18.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:18 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:18.657+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:18.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:18 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:18.657+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:18.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:18 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:18.657+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:18.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:18 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:18.657+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:18.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:18 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:18.657+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:18.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:18 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:18.657+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:18.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:18 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:18.700+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:18.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:18 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:18.700+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:18.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:18 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:18.700+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:18.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:18 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:18.700+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:19.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:18 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:18.861+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:19.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:18 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:18.861+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:19.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:18 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:18.861+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:19.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:18 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:18.861+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.540323+0000 front 2024-09-11T01:25:48.540284+0000 (oldest deadline 2024-09-11T01:26:10.840455+0000) 2024-09-11T01:26:19.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:19 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:19.313+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:19.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:19 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:19.313+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:19.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:19 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:19.313+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:19.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:19 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:19.313+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:19.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:19 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:19.676+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:19.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:19 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:19.676+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:19.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:19 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:19.676+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:19.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:19 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:19.676+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:19.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:19 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:19.676+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:55.292572+0000 front 2024-09-11T01:25:55.292616+0000 (oldest deadline 2024-09-11T01:26:19.392647+0000) 2024-09-11T01:26:20.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:19 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:19.692+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:20.192 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:19 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:19.692+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:20.192 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:19 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:19.692+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:20.192 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:19 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:19.692+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:20.192 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:19 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:19.692+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:20.192 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:19 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:19.692+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:20.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:19 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:19.850+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:20.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:19 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:19.850+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:20.225 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:19 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:19.850+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:20.225 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:19 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:19.850+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.540323+0000 front 2024-09-11T01:25:48.540284+0000 (oldest deadline 2024-09-11T01:26:10.840455+0000) 2024-09-11T01:26:20.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:20 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:20.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:20.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:20 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:20.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:20.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:20 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:20.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:20.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:20 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:20.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:20.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:20 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:20.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:20.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:20 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:20.664+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:20.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:20 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:20.664+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:20.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:20 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:20.664+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:20.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:20 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:20.664+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:20.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:20 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:20.664+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:55.292572+0000 front 2024-09-11T01:25:55.292616+0000 (oldest deadline 2024-09-11T01:26:19.392647+0000) 2024-09-11T01:26:21.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:20 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:20.723+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:21.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:20 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:20.723+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:21.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:20 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:20.723+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:21.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:20 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:20.723+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:21.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:20 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:20.723+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:21.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:20 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:20.723+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:21.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:20 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:20.860+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:21.235282+0000 front 2024-09-11T01:25:21.235208+0000 (oldest deadline 2024-09-11T01:25:47.635856+0000) 2024-09-11T01:26:21.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:20 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:20.860+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.637361+0000 front 2024-09-11T01:25:31.637389+0000 (oldest deadline 2024-09-11T01:25:55.137483+0000) 2024-09-11T01:26:21.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:20 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:20.860+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:40.438946+0000 front 2024-09-11T01:25:40.438926+0000 (oldest deadline 2024-09-11T01:26:05.039286+0000) 2024-09-11T01:26:21.224 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:20 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8[54645]: 2024-09-11T01:26:20.860+0000 7f2a05b4f640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.540323+0000 front 2024-09-11T01:25:48.540284+0000 (oldest deadline 2024-09-11T01:26:10.840455+0000) 2024-09-11T01:26:21.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:21 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:21.334+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:21.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:21 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:21.334+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:21.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:21 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:21.334+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:21.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:21 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:21.334+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:21.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:21 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:21.334+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:21.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:21 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:21.670+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:21.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:21 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:21.670+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:21.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:21 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:21.670+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:21.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:21 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:21.670+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:21.998 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:21 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:21.670+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:55.292572+0000 front 2024-09-11T01:25:55.292616+0000 (oldest deadline 2024-09-11T01:26:19.392647+0000) 2024-09-11T01:26:22.144 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:21 smithi112 podman[76674]: 2024-09-11 01:26:21.854511756 +0000 UTC m=+5.568714743 container died c75d07ecd2d25955764ae67b7156d8b8869762f59696a74384bf4529dd502422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0) 2024-09-11T01:26:22.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:21 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:21.713+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:22.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:21 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:21.713+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:22.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:21 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:21.713+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:22.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:21 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:21.713+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:22.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:21 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:21.713+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:22.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:21 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:21.713+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:22.474 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:22 smithi112 podman[76674]: 2024-09-11 01:26:22.249530783 +0000 UTC m=+5.963733772 container cleanup c75d07ecd2d25955764ae67b7156d8b8869762f59696a74384bf4529dd502422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T01:26:22.474 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:22 smithi112 bash[76674]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8 2024-09-11T01:26:22.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:22 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:22.338+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:22.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:22 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:22.338+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:22.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:22 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:22.338+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:22.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:22 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:22.338+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:22.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:22 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:22.338+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:22.826 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:22 smithi112 podman[76700]: 2024-09-11 01:26:22.517408942 +0000 UTC m=+0.663531685 container remove c75d07ecd2d25955764ae67b7156d8b8869762f59696a74384bf4529dd502422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T01:26:22.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:22 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:22.714+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:22.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:22 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:22.714+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:22.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:22 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:22.714+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:22.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:22 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:22.714+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:22.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:22 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:22.714+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:55.292572+0000 front 2024-09-11T01:25:55.292616+0000 (oldest deadline 2024-09-11T01:26:19.392647+0000) 2024-09-11T01:26:23.105 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:22 smithi112 podman[76791]: 2024-09-11 01:26:22.728192664 +0000 UTC m=+0.015537272 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T01:26:23.105 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:22 smithi112 podman[76791]: 2024-09-11 01:26:22.843207308 +0000 UTC m=+0.130551910 container create 4dbd979237309400ace1d5e1efd3c2afd5faaf7d3267458b5df5995de2cf1911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T01:26:23.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:22 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:22.708+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:23.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:22 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:22.708+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:23.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:22 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:22.708+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:23.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:22 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:22.708+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:23.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:22 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:22.708+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:23.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:22 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:22.708+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:23.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:23 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:23.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:23.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:23 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:23.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:23.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:23 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:23.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:23.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:23 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:23.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:23.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:23 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:23.295+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:23.475 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:23 smithi112 podman[76791]: 2024-09-11 01:26:23.104178895 +0000 UTC m=+0.391523492 container init 4dbd979237309400ace1d5e1efd3c2afd5faaf7d3267458b5df5995de2cf1911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T01:26:23.475 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:23 smithi112 podman[76791]: 2024-09-11 01:26:23.10750542 +0000 UTC m=+0.394850020 container start 4dbd979237309400ace1d5e1efd3c2afd5faaf7d3267458b5df5995de2cf1911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T01:26:23.475 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:23 smithi112 podman[76791]: 2024-09-11 01:26:23.19591885 +0000 UTC m=+0.483263463 container attach 4dbd979237309400ace1d5e1efd3c2afd5faaf7d3267458b5df5995de2cf1911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T01:26:23.475 INFO:journalctl@ceph.osd.8.smithi112.stdout:Sep 11 01:26:23 smithi112 podman[76791]: 2024-09-11 01:26:23.291233401 +0000 UTC m=+0.578578002 container died 4dbd979237309400ace1d5e1efd3c2afd5faaf7d3267458b5df5995de2cf1911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-09-11T01:26:23.980 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.8.service' 2024-09-11T01:26:23.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:23 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:23.724+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:23.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:23 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:23.724+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:23.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:23 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:23.724+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:23.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:23 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:23.724+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:23.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:23 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:23.724+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:55.292572+0000 front 2024-09-11T01:25:55.292616+0000 (oldest deadline 2024-09-11T01:26:19.392647+0000) 2024-09-11T01:26:24.021 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:26:24.022 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-11T01:26:24.022 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-11T01:26:24.022 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.9 2024-09-11T01:26:24.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:23 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:23.726+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:24.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:23 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:23.726+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:24.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:23 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:23.726+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:24.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:23 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:23.726+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:24.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:23 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:23.726+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:24.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:23 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:23.726+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:24.332 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:24 smithi078 systemd[1]: Stopping Ceph osd.9 for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:26:24.332 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:24 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:24.330+0000 7f68756e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:26:24.332 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:24 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:24.330+0000 7f68756e0640 -1 osd.9 118 *** Got signal Terminated *** 2024-09-11T01:26:24.332 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:24 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:24.330+0000 7f68756e0640 -1 osd.9 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T01:26:24.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:24 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:24.300+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:24.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:24 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:24.300+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:24.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:24 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:24.300+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:24.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:24 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:24.300+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:24.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:24 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:24.300+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:25.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:24 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:24.712+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:25.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:24 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:24.712+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:25.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:24 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:24.712+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:25.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:24 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:24.712+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:25.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:24 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:24.712+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:25.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:24 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:24.712+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:25.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:24 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:24.760+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:25.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:24 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:24.760+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:25.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:24 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:24.760+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:25.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:24 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:24.760+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:25.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:24 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:24.760+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:55.292572+0000 front 2024-09-11T01:25:55.292616+0000 (oldest deadline 2024-09-11T01:26:19.392647+0000) 2024-09-11T01:26:25.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:25 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:25.315+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:25.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:25 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:25.315+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:25.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:25 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:25.315+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:25.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:25 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:25.315+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:25.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:25 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:25.315+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:25.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:25 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:25.666+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:25.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:25 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:25.666+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:25.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:25 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:25.666+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:25.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:25 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:25.666+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:25.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:25 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:25.666+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:25.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:25 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:25.666+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:26.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:25 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:25.792+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:26.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:25 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:25.792+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:26.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:25 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:25.792+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:26.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:25 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:25.792+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:26.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:25 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:25.792+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:55.292572+0000 front 2024-09-11T01:25:55.292616+0000 (oldest deadline 2024-09-11T01:26:19.392647+0000) 2024-09-11T01:26:26.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:26 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:26.336+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:26.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:26 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:26.336+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:26.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:26 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:26.336+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:26.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:26 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:26.336+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:26.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:26 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:26.336+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:26.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:26 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:26.669+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:26.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:26 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:26.669+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:26.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:26 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:26.669+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:26.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:26 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:26.669+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:26.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:26 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:26.669+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:26.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:26 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:26.669+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:27.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:26 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:26.772+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:27.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:26 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:26.772+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:27.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:26 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:26.772+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:27.248 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:26 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:26.772+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:27.248 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:26 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:26.772+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:55.292572+0000 front 2024-09-11T01:25:55.292616+0000 (oldest deadline 2024-09-11T01:26:19.392647+0000) 2024-09-11T01:26:27.691 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:27 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:27.676+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:27.691 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:27 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:27.676+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:27.691 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:27 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:27.676+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:27.691 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:27 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:27.676+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:27.691 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:27 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:27.676+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:27.691 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:27 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:27.676+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:27.691 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:27 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:27.676+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.933460+0000 front 2024-09-11T01:26:03.933490+0000 (oldest deadline 2024-09-11T01:26:27.433379+0000) 2024-09-11T01:26:27.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:27 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:27.313+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:27.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:27 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:27.313+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:27.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:27 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:27.313+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:27.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:27 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:27.313+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:27.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:27 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:27.313+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:28.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:27 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:27.787+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:28.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:27 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:27.787+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:28.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:27 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:27.787+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:28.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:27 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:27.787+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:28.247 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:27 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:27.787+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:55.292572+0000 front 2024-09-11T01:25:55.292616+0000 (oldest deadline 2024-09-11T01:26:19.392647+0000) 2024-09-11T01:26:28.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:28 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:28.359+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:28.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:28 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:28.359+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:28.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:28 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:28.359+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:28.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:28 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:28.359+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:28.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:28 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:28.359+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:28.940 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:28 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:28.684+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:28.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:28 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:28.684+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:28.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:28 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:28.684+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:28.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:28 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:28.684+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:28.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:28 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:28.684+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:28.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:28 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:28.684+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:28.941 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:28 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:28.684+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.933460+0000 front 2024-09-11T01:26:03.933490+0000 (oldest deadline 2024-09-11T01:26:27.433379+0000) 2024-09-11T01:26:28.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:28 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:28.745+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:11.685145+0000 front 2024-09-11T01:25:11.685169+0000 (oldest deadline 2024-09-11T01:25:37.585251+0000) 2024-09-11T01:26:28.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:28 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:28.745+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.486464+0000 front 2024-09-11T01:25:23.486428+0000 (oldest deadline 2024-09-11T01:25:47.586396+0000) 2024-09-11T01:26:28.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:28 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:28.745+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:31.687659+0000 front 2024-09-11T01:25:31.687786+0000 (oldest deadline 2024-09-11T01:25:54.488115+0000) 2024-09-11T01:26:28.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:28 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:28.745+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:39.589967+0000 front 2024-09-11T01:25:39.590189+0000 (oldest deadline 2024-09-11T01:26:02.490087+0000) 2024-09-11T01:26:28.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:28 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:28.745+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:55.292572+0000 front 2024-09-11T01:25:55.292616+0000 (oldest deadline 2024-09-11T01:26:19.392647+0000) 2024-09-11T01:26:28.997 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:28 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9[67563]: 2024-09-11T01:26:28.745+0000 7f686dced640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:25:59.393077+0000 front 2024-09-11T01:25:59.393112+0000 (oldest deadline 2024-09-11T01:26:28.193846+0000) 2024-09-11T01:26:29.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:29 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:29.392+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:29.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:29 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:29.392+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:29.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:29 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:29.392+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:29.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:29 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:29.392+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:29.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:29 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:29.392+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:29.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:29 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:29.392+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:30.108 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:29 smithi078 podman[130850]: 2024-09-11 01:26:29.807095711 +0000 UTC m=+5.596371209 container died 6181ab3466d80151cbdd10c313275c8c88195ff4d943e71e1715bf0371b51179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T01:26:30.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:29 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:29.724+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:30.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:29 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:29.724+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:30.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:29 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:29.724+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:30.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:29 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:29.724+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:30.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:29 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:29.724+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:30.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:29 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:29.724+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:30.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:29 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:29.724+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.933460+0000 front 2024-09-11T01:26:03.933490+0000 (oldest deadline 2024-09-11T01:26:27.433379+0000) 2024-09-11T01:26:30.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:30 smithi078 podman[130850]: 2024-09-11 01:26:30.272104583 +0000 UTC m=+6.061380076 container cleanup 6181ab3466d80151cbdd10c313275c8c88195ff4d943e71e1715bf0371b51179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:26:30.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:30 smithi078 bash[130850]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9 2024-09-11T01:26:30.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:30 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:30.415+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:30.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:30 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:30.415+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:30.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:30 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:30.415+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:30.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:30 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:30.415+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:30.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:30 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:30.415+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:30.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:30 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:30.415+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:30.946 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:30 smithi078 podman[130865]: 2024-09-11 01:26:30.590035486 +0000 UTC m=+0.777967533 container remove 6181ab3466d80151cbdd10c313275c8c88195ff4d943e71e1715bf0371b51179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True) 2024-09-11T01:26:31.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:30 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:30.770+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:31.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:30 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:30.770+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:31.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:30 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:30.770+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:31.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:30 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:30.770+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:31.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:30 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:30.770+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:31.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:30 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:30.770+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:31.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:30 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:30.770+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.933460+0000 front 2024-09-11T01:26:03.933490+0000 (oldest deadline 2024-09-11T01:26:27.433379+0000) 2024-09-11T01:26:31.197 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:30 smithi078 podman[130954]: 2024-09-11 01:26:30.847647334 +0000 UTC m=+0.015527102 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T01:26:31.198 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:30 smithi078 podman[130954]: 2024-09-11 01:26:30.990770319 +0000 UTC m=+0.158650081 container create a064479472f2694193c7ea77e1611a8dafd43e1b12b874ab11e8c0d49fdbeedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:26:31.198 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:31 smithi078 podman[130954]: 2024-09-11 01:26:31.193315468 +0000 UTC m=+0.361195228 container init a064479472f2694193c7ea77e1611a8dafd43e1b12b874ab11e8c0d49fdbeedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-11T01:26:31.198 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:31 smithi078 podman[130954]: 2024-09-11 01:26:31.196810073 +0000 UTC m=+0.364689843 container start a064479472f2694193c7ea77e1611a8dafd43e1b12b874ab11e8c0d49fdbeedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T01:26:31.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:31 smithi078 podman[130954]: 2024-09-11 01:26:31.276448057 +0000 UTC m=+0.444327827 container attach a064479472f2694193c7ea77e1611a8dafd43e1b12b874ab11e8c0d49fdbeedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T01:26:31.497 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 11 01:26:31 smithi078 podman[130954]: 2024-09-11 01:26:31.38207275 +0000 UTC m=+0.549952519 container died a064479472f2694193c7ea77e1611a8dafd43e1b12b874ab11e8c0d49fdbeedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS) 2024-09-11T01:26:31.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:31 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:31.384+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:31.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:31 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:31.384+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:31.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:31 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:31.384+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:31.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:31 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:31.384+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:31.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:31 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:31.384+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:31.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:31 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:31.384+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:32.018 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.9.service' 2024-09-11T01:26:32.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:26:32.060 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-11T01:26:32.060 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-11T01:26:32.060 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.10 2024-09-11T01:26:32.088 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:31 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:31.759+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:32.088 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:31 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:31.759+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:32.088 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:31 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:31.759+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:32.088 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:31 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:31.759+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:32.088 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:31 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:31.759+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:32.088 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:31 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:31.759+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:32.089 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:31 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:31.759+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.933460+0000 front 2024-09-11T01:26:03.933490+0000 (oldest deadline 2024-09-11T01:26:27.433379+0000) 2024-09-11T01:26:32.374 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:32 smithi110 systemd[1]: Stopping Ceph osd.10 for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:26:32.690 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:32 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:32.372+0000 7fc6ef23d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:26:32.691 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:32 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:32.372+0000 7fc6ef23d640 -1 osd.10 118 *** Got signal Terminated *** 2024-09-11T01:26:32.691 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:32 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:32.372+0000 7fc6ef23d640 -1 osd.10 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T01:26:32.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:32 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:32.378+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:32.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:32 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:32.378+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:32.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:32 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:32.378+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:32.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:32 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:32.378+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:32.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:32 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:32.378+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:32.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:32 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:32.378+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:33.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:32 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:32.754+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:33.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:32 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:32.754+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:33.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:32 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:32.754+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:33.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:32 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:32.754+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:33.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:32 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:32.754+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:33.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:32 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:32.754+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:33.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:32 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:32.754+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.933460+0000 front 2024-09-11T01:26:03.933490+0000 (oldest deadline 2024-09-11T01:26:27.433379+0000) 2024-09-11T01:26:33.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:33 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:33.424+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:33.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:33 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:33.424+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:33.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:33 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:33.424+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:33.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:33 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:33.424+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:33.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:33 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:33.424+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:33.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:33 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:33.424+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:34.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:33 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:33.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:34.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:33 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:33.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:34.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:33 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:33.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:34.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:33 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:33.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:34.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:33 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:33.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:34.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:33 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:33.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:34.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:33 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:33.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.933460+0000 front 2024-09-11T01:26:03.933490+0000 (oldest deadline 2024-09-11T01:26:27.433379+0000) 2024-09-11T01:26:34.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:34 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:34.387+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:34.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:34 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:34.387+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:34.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:34 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:34.387+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:34.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:34 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:34.387+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:34.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:34 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:34.387+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:34.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:34 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:34.387+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:35.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:34 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:34.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:35.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:34 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:34.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:35.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:34 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:34.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:35.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:34 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:34.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:35.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:34 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:34.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:35.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:34 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:34.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:35.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:34 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:34.802+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.933460+0000 front 2024-09-11T01:26:03.933490+0000 (oldest deadline 2024-09-11T01:26:27.433379+0000) 2024-09-11T01:26:35.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:35 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:35.356+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:35.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:35 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:35.356+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:35.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:35 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:35.356+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:35.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:35 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:35.356+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:35.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:35 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:35.356+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:35.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:35 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:35.356+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:36.190 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:35 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:35.843+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:36.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:35 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:35.843+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:36.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:35 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:35.843+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:36.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:35 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:35.843+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:36.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:35 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:35.843+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:36.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:35 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:35.843+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:36.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:35 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:35.843+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.933460+0000 front 2024-09-11T01:26:03.933490+0000 (oldest deadline 2024-09-11T01:26:27.433379+0000) 2024-09-11T01:26:36.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:36 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:36.319+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:36.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:36 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:36.319+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:36.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:36 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:36.319+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:36.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:36 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:36.319+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:36.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:36 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:36.319+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:36.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:36 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:36.319+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:36.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:36 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:36.319+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T01:26:12.745937+0000 front 2024-09-11T01:26:12.745894+0000 (oldest deadline 2024-09-11T01:26:36.245679+0000) 2024-09-11T01:26:37.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:36 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:36.834+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:13.324696+0000 front 2024-09-11T01:25:13.324686+0000 (oldest deadline 2024-09-11T01:25:36.824409+0000) 2024-09-11T01:26:37.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:36 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:36.834+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:23.825934+0000 front 2024-09-11T01:25:23.825957+0000 (oldest deadline 2024-09-11T01:25:47.926115+0000) 2024-09-11T01:26:37.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:36 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:36.834+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:30.727617+0000 front 2024-09-11T01:25:30.727655+0000 (oldest deadline 2024-09-11T01:25:56.027660+0000) 2024-09-11T01:26:37.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:36 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:36.834+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-11T01:25:37.029301+0000 front 2024-09-11T01:25:37.029412+0000 (oldest deadline 2024-09-11T01:26:02.329530+0000) 2024-09-11T01:26:37.191 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:36 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:36.834+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:44.030456+0000 front 2024-09-11T01:25:44.030473+0000 (oldest deadline 2024-09-11T01:26:13.431184+0000) 2024-09-11T01:26:37.192 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:36 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:36.834+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:53.431834+0000 front 2024-09-11T01:25:53.431712+0000 (oldest deadline 2024-09-11T01:26:18.131583+0000) 2024-09-11T01:26:37.192 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:36 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-11T01:26:36.834+0000 7fc6eb852640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.933460+0000 front 2024-09-11T01:26:03.933490+0000 (oldest deadline 2024-09-11T01:26:27.433379+0000) 2024-09-11T01:26:37.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:37 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:37.289+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:37.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:37 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:37.289+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:37.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:37 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:37.289+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:37.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:37 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:37.289+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:37.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:37 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:37.289+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:37.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:37 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:37.289+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:37.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:37 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:37.289+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T01:26:12.745937+0000 front 2024-09-11T01:26:12.745894+0000 (oldest deadline 2024-09-11T01:26:36.245679+0000) 2024-09-11T01:26:38.166 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:37 smithi110 podman[75635]: 2024-09-11 01:26:37.857339393 +0000 UTC m=+5.600104860 container died f199885118602d2acff7c07f5ead7ad362666c580718b8871f4d9dfda02787bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T01:26:38.440 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:38 smithi110 podman[75635]: 2024-09-11 01:26:38.289591089 +0000 UTC m=+6.032356543 container cleanup f199885118602d2acff7c07f5ead7ad362666c580718b8871f4d9dfda02787bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T01:26:38.440 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:38 smithi110 bash[75635]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10 2024-09-11T01:26:38.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:38 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:38.248+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:38.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:38 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:38.248+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:38.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:38 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:38.248+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:38.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:38 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:38.248+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:38.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:38 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:38.248+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:38.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:38 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:38.248+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:38.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:38 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:38.248+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T01:26:12.745937+0000 front 2024-09-11T01:26:12.745894+0000 (oldest deadline 2024-09-11T01:26:36.245679+0000) 2024-09-11T01:26:38.893 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:38 smithi110 podman[75650]: 2024-09-11 01:26:38.484958714 +0000 UTC m=+0.628876728 container remove f199885118602d2acff7c07f5ead7ad362666c580718b8871f4d9dfda02787bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T01:26:39.170 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:38 smithi110 podman[75741]: 2024-09-11 01:26:38.891821231 +0000 UTC m=+0.110273913 container create e0f221d2c927f8058d629f22bf8c349e877facbc0a7e258c934b031ab51945c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T01:26:39.171 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:38 smithi110 podman[75741]: 2024-09-11 01:26:38.797546821 +0000 UTC m=+0.015999468 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T01:26:39.171 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:39 smithi110 podman[75741]: 2024-09-11 01:26:39.085867402 +0000 UTC m=+0.304320045 container init e0f221d2c927f8058d629f22bf8c349e877facbc0a7e258c934b031ab51945c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T01:26:39.171 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:39 smithi110 podman[75741]: 2024-09-11 01:26:39.089004809 +0000 UTC m=+0.307457450 container start e0f221d2c927f8058d629f22bf8c349e877facbc0a7e258c934b031ab51945c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T01:26:39.440 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:39 smithi110 podman[75741]: 2024-09-11 01:26:39.169023538 +0000 UTC m=+0.387476181 container attach e0f221d2c927f8058d629f22bf8c349e877facbc0a7e258c934b031ab51945c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0) 2024-09-11T01:26:39.440 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:39 smithi110 podman[75741]: 2024-09-11 01:26:39.276938484 +0000 UTC m=+0.495391127 container died e0f221d2c927f8058d629f22bf8c349e877facbc0a7e258c934b031ab51945c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T01:26:39.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:39 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:39.276+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:39.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:39 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:39.276+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:39.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:39 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:39.276+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:39.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:39 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:39.276+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:39.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:39 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:39.276+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:39.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:39 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:39.276+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:39.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:39 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:39.276+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T01:26:12.745937+0000 front 2024-09-11T01:26:12.745894+0000 (oldest deadline 2024-09-11T01:26:36.245679+0000) 2024-09-11T01:26:39.841 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 11 01:26:39 smithi110 podman[75764]: 2024-09-11 01:26:39.813981842 +0000 UTC m=+0.530295381 container remove e0f221d2c927f8058d629f22bf8c349e877facbc0a7e258c934b031ab51945c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-09-11T01:26:39.846 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.10.service' 2024-09-11T01:26:39.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:26:39.926 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-11T01:26:39.926 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-11T01:26:39.926 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.11 2024-09-11T01:26:40.224 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:39 smithi112 systemd[1]: Stopping Ceph osd.11 for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:26:40.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:40 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:40.242+0000 7f246a561640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:26:40.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:40 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:40.242+0000 7f246a561640 -1 osd.11 118 *** Got signal Terminated *** 2024-09-11T01:26:40.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:40 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:40.242+0000 7f246a561640 -1 osd.11 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T01:26:40.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:40 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:40.263+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:40.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:40 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:40.263+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:40.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:40 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:40.263+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:40.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:40 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:40.263+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:40.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:40 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:40.263+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:40.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:40 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:40.263+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:40.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:40 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:40.263+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T01:26:12.745937+0000 front 2024-09-11T01:26:12.745894+0000 (oldest deadline 2024-09-11T01:26:36.245679+0000) 2024-09-11T01:26:41.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:41 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:41.246+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:41.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:41 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:41.246+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:41.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:41 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:41.246+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:41.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:41 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:41.246+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:41.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:41 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:41.246+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:41.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:41 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:41.246+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:41.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:41 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:41.246+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T01:26:12.745937+0000 front 2024-09-11T01:26:12.745894+0000 (oldest deadline 2024-09-11T01:26:36.245679+0000) 2024-09-11T01:26:42.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:42 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:42.257+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:42.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:42 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:42.257+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:42.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:42 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:42.257+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:42.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:42 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:42.257+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:42.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:42 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:42.257+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:42.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:42 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:42.257+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:42.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:42 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:42.257+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T01:26:12.745937+0000 front 2024-09-11T01:26:12.745894+0000 (oldest deadline 2024-09-11T01:26:36.245679+0000) 2024-09-11T01:26:43.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:43 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:43.259+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:43.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:43 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:43.259+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:43.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:43 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:43.259+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:43.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:43 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:43.259+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:43.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:43 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:43.259+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:43.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:43 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:43.259+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:43.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:43 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:43.259+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T01:26:12.745937+0000 front 2024-09-11T01:26:12.745894+0000 (oldest deadline 2024-09-11T01:26:36.245679+0000) 2024-09-11T01:26:44.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:44 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:44.231+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:44.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:44 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:44.231+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:44.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:44 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:44.231+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:44.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:44 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:44.231+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:44.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:44 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:44.231+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:44.724 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:44 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:44.231+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:44.725 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:44 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:44.231+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T01:26:12.745937+0000 front 2024-09-11T01:26:12.745894+0000 (oldest deadline 2024-09-11T01:26:36.245679+0000) 2024-09-11T01:26:45.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:45 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:45.197+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-11T01:25:12.534000+0000 front 2024-09-11T01:25:12.534120+0000 (oldest deadline 2024-09-11T01:25:37.234151+0000) 2024-09-11T01:26:45.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:45 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:45.197+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-11T01:25:22.535432+0000 front 2024-09-11T01:25:22.535454+0000 (oldest deadline 2024-09-11T01:25:45.435385+0000) 2024-09-11T01:26:45.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:45 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:45.197+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6804 osd.2 since back 2024-09-11T01:25:32.238170+0000 front 2024-09-11T01:25:32.238223+0000 (oldest deadline 2024-09-11T01:25:57.538373+0000) 2024-09-11T01:26:45.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:45 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:45.197+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-11T01:25:48.441589+0000 front 2024-09-11T01:25:48.441675+0000 (oldest deadline 2024-09-11T01:26:09.541617+0000) 2024-09-11T01:26:45.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:45 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:45.197+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6812 osd.5 since back 2024-09-11T01:25:56.443517+0000 front 2024-09-11T01:25:56.443546+0000 (oldest deadline 2024-09-11T01:26:19.943510+0000) 2024-09-11T01:26:45.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:45 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:45.197+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-11T01:26:03.944917+0000 front 2024-09-11T01:26:03.944963+0000 (oldest deadline 2024-09-11T01:26:29.244775+0000) 2024-09-11T01:26:45.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:45 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:45.197+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-11T01:26:12.745937+0000 front 2024-09-11T01:26:12.745894+0000 (oldest deadline 2024-09-11T01:26:36.245679+0000) 2024-09-11T01:26:45.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:45 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11[61230]: 2024-09-11T01:26:45.197+0000 7f2466b76640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.112:6820 osd.8 since back 2024-09-11T01:26:20.946642+0000 front 2024-09-11T01:26:20.946621+0000 (oldest deadline 2024-09-11T01:26:44.347192+0000) 2024-09-11T01:26:46.107 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:45 smithi112 podman[76896]: 2024-09-11 01:26:45.835307082 +0000 UTC m=+5.717669150 container died 766ddd613566176f12ff91aa4e81f57e931fa88521dc3bfadda789dc51221dd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:26:46.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:46 smithi112 podman[76896]: 2024-09-11 01:26:46.210261884 +0000 UTC m=+6.092623955 container cleanup 766ddd613566176f12ff91aa4e81f57e931fa88521dc3bfadda789dc51221dd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T01:26:46.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:46 smithi112 bash[76896]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11 2024-09-11T01:26:46.789 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:46 smithi112 podman[76912]: 2024-09-11 01:26:46.477970569 +0000 UTC m=+0.646150056 container remove 766ddd613566176f12ff91aa4e81f57e931fa88521dc3bfadda789dc51221dd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T01:26:47.040 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:46 smithi112 podman[77001]: 2024-09-11 01:26:46.690022962 +0000 UTC m=+0.016051847 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T01:26:47.041 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:46 smithi112 podman[77001]: 2024-09-11 01:26:46.803860764 +0000 UTC m=+0.129889635 container create 92ffb1ab3849acee46932dc55ed93fb100e13b5f6eebedb9a87fe662bec755ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T01:26:47.041 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:47 smithi112 podman[77001]: 2024-09-11 01:26:47.039559968 +0000 UTC m=+0.365588845 container init 92ffb1ab3849acee46932dc55ed93fb100e13b5f6eebedb9a87fe662bec755ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:26:47.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:47 smithi112 podman[77001]: 2024-09-11 01:26:47.043028229 +0000 UTC m=+0.369057108 container start 92ffb1ab3849acee46932dc55ed93fb100e13b5f6eebedb9a87fe662bec755ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T01:26:47.474 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:47 smithi112 podman[77001]: 2024-09-11 01:26:47.122641735 +0000 UTC m=+0.448670616 container attach 92ffb1ab3849acee46932dc55ed93fb100e13b5f6eebedb9a87fe662bec755ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:26:47.475 INFO:journalctl@ceph.osd.11.smithi112.stdout:Sep 11 01:26:47 smithi112 podman[77001]: 2024-09-11 01:26:47.22794772 +0000 UTC m=+0.553976600 container died 92ffb1ab3849acee46932dc55ed93fb100e13b5f6eebedb9a87fe662bec755ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T01:26:47.863 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@osd.11.service' 2024-09-11T01:26:47.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:26:47.902 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-11T01:26:47.902 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-11T01:26:47.902 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.a 2024-09-11T01:26:48.234 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 01:26:47 smithi078 systemd[1]: Stopping Ceph mds.a for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:26:48.491 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 01:26:48 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-a[74736]: 2024-09-11T01:26:48.232+0000 7fa3e516a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:26:48.492 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 01:26:48 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-a[74736]: 2024-09-11T01:26:48.232+0000 7fa3e516a640 -1 mds.a *** got signal Terminated *** 2024-09-11T01:26:52.566 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 01:26:52 smithi078 podman[131058]: 2024-09-11 01:26:52.256256022 +0000 UTC m=+4.156916399 container died 40e21add21e03d1029c7203203ddcade61efcc2663f3307e242cef47f6e5bb59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:26:52.900 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 01:26:52 smithi078 podman[131058]: 2024-09-11 01:26:52.723145265 +0000 UTC m=+4.623805682 container cleanup 40e21add21e03d1029c7203203ddcade61efcc2663f3307e242cef47f6e5bb59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T01:26:52.901 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 11 01:26:52 smithi078 bash[131058]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-a 2024-09-11T01:26:53.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.a.service' 2024-09-11T01:26:53.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:26:53.146 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-11T01:26:53.147 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-11T01:26:53.147 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.d 2024-09-11T01:26:53.462 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 11 01:26:53 smithi078 systemd[1]: Stopping Ceph mds.d for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:26:53.747 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 11 01:26:53 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-d[73561]: 2024-09-11T01:26:53.460+0000 7efd96101640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:26:53.747 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 11 01:26:53 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-d[73561]: 2024-09-11T01:26:53.460+0000 7efd96101640 -1 mds.d *** got signal Terminated *** 2024-09-11T01:26:55.247 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 11 01:26:54 smithi078 podman[131191]: 2024-09-11 01:26:54.987106229 +0000 UTC m=+1.626401638 container died 4e4330e32fad0635110511014e257022f60a72eec2a2e2cf6b67b92f069307ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T01:26:55.629 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 11 01:26:55 smithi078 podman[131191]: 2024-09-11 01:26:55.402240299 +0000 UTC m=+2.041535705 container cleanup 4e4330e32fad0635110511014e257022f60a72eec2a2e2cf6b67b92f069307ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T01:26:55.629 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 11 01:26:55 smithi078 bash[131191]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-d 2024-09-11T01:26:55.761 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.d.service' 2024-09-11T01:26:55.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:26:55.794 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-11T01:26:55.794 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-11T01:26:55.795 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.g 2024-09-11T01:26:55.997 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 01:26:55 smithi078 systemd[1]: Stopping Ceph mds.g for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:26:56.497 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 01:26:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-g[74328]: 2024-09-11T01:26:56.105+0000 7f6e9a936640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:26:56.497 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 01:26:56 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-g[74328]: 2024-09-11T01:26:56.105+0000 7f6e9a936640 -1 mds.g *** got signal Terminated *** 2024-09-11T01:27:00.383 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 01:27:00 smithi078 podman[131312]: 2024-09-11 01:27:00.11870017 +0000 UTC m=+4.112711181 container died 33a1b47b8cfbe6b2a2ac9211a684bf8ee316e11e1e021ca40a7c75f03500307f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-g, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T01:27:00.747 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 01:27:00 smithi078 podman[131312]: 2024-09-11 01:27:00.581083522 +0000 UTC m=+4.575094530 container cleanup 33a1b47b8cfbe6b2a2ac9211a684bf8ee316e11e1e021ca40a7c75f03500307f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-g, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:27:00.747 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 11 01:27:00 smithi078 bash[131312]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-g 2024-09-11T01:27:01.010 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.g.service' 2024-09-11T01:27:01.042 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:27:01.043 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-11T01:27:01.043 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-11T01:27:01.043 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.j 2024-09-11T01:27:01.387 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 01:27:01 smithi078 systemd[1]: Stopping Ceph mds.j for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:27:01.747 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 01:27:01 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-j[73149]: 2024-09-11T01:27:01.384+0000 7fe6bb2ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:27:01.747 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 01:27:01 smithi078 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-j[73149]: 2024-09-11T01:27:01.384+0000 7fe6bb2ad640 -1 mds.j *** got signal Terminated *** 2024-09-11T01:27:04.211 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 01:27:03 smithi078 podman[131430]: 2024-09-11 01:27:03.888144536 +0000 UTC m=+2.629819676 container died a0258905a31c9195b4f3ec31502597ec8a7677f1260fd84963517494e7f0f541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:27:04.211 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 01:27:04 smithi078 podman[131430]: 2024-09-11 01:27:04.210053755 +0000 UTC m=+2.951728895 container cleanup a0258905a31c9195b4f3ec31502597ec8a7677f1260fd84963517494e7f0f541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T01:27:04.497 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 01:27:04 smithi078 bash[131430]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-j 2024-09-11T01:27:04.497 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 11 01:27:04 smithi078 podman[131443]: 2024-09-11 01:27:04.444583703 +0000 UTC m=+0.544897400 container remove a0258905a31c9195b4f3ec31502597ec8a7677f1260fd84963517494e7f0f541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T01:27:04.678 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.j.service' 2024-09-11T01:27:04.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:27:04.710 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-11T01:27:04.710 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-11T01:27:04.710 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.b 2024-09-11T01:27:04.991 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 01:27:04 smithi110 systemd[1]: Stopping Ceph mds.b for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:27:05.248 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 01:27:04 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b[65617]: 2024-09-11T01:27:04.989+0000 7f0177fd4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:27:05.248 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 01:27:04 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b[65617]: 2024-09-11T01:27:04.989+0000 7f0177fd4640 -1 mds.b *** got signal Terminated *** 2024-09-11T01:27:09.292 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 01:27:09 smithi110 podman[75844]: 2024-09-11 01:27:09.005843627 +0000 UTC m=+4.102423646 container died 7019b796cf94467945c0bc98a0eec9567e0a29528c2a00aa4fd99d3503d9181e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T01:27:09.659 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 01:27:09 smithi110 podman[75844]: 2024-09-11 01:27:09.373129924 +0000 UTC m=+4.469709898 container cleanup 7019b796cf94467945c0bc98a0eec9567e0a29528c2a00aa4fd99d3503d9181e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T01:27:09.659 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 11 01:27:09 smithi110 bash[75844]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-b 2024-09-11T01:27:09.807 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.b.service' 2024-09-11T01:27:09.843 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:27:09.844 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-11T01:27:09.844 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-11T01:27:09.844 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.e 2024-09-11T01:27:10.190 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 01:27:09 smithi110 systemd[1]: Stopping Ceph mds.e for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:27:10.690 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 01:27:10 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-e[65001]: 2024-09-11T01:27:10.202+0000 7f3b5edec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:27:10.690 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 01:27:10 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-e[65001]: 2024-09-11T01:27:10.202+0000 7f3b5edec640 -1 mds.e *** got signal Terminated *** 2024-09-11T01:27:13.279 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 01:27:12 smithi110 podman[75976]: 2024-09-11 01:27:12.999569792 +0000 UTC m=+2.944513169 container died da3ad5bad4b1c62903215706bf4a5dfd56dbe758408eca0d1915bf277c0ac79c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T01:27:13.638 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 01:27:13 smithi110 podman[75976]: 2024-09-11 01:27:13.37731682 +0000 UTC m=+3.322260196 container cleanup da3ad5bad4b1c62903215706bf4a5dfd56dbe758408eca0d1915bf277c0ac79c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:27:13.638 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 11 01:27:13 smithi110 bash[75976]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-e 2024-09-11T01:27:13.802 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.e.service' 2024-09-11T01:27:13.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:27:13.835 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-11T01:27:13.835 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-11T01:27:13.835 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.h 2024-09-11T01:27:14.190 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 01:27:13 smithi110 systemd[1]: Stopping Ceph mds.h for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:27:14.190 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 01:27:14 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h[65302]: 2024-09-11T01:27:14.139+0000 7fe0f7d0b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:27:14.191 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 01:27:14 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h[65302]: 2024-09-11T01:27:14.139+0000 7fe0f7d0b640 -1 mds.h *** got signal Terminated *** 2024-09-11T01:27:17.315 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 01:27:16 smithi110 podman[76099]: 2024-09-11 01:27:16.998304394 +0000 UTC m=+2.952040850 container died 7242af13f12682cb1a3cbd4e503eeb3ecbd6774f90460f19f038badaca9242d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:27:17.316 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 01:27:17 smithi110 podman[76099]: 2024-09-11 01:27:17.314728228 +0000 UTC m=+3.268464683 container cleanup 7242af13f12682cb1a3cbd4e503eeb3ecbd6774f90460f19f038badaca9242d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:27:17.692 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 01:27:17 smithi110 bash[76099]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h 2024-09-11T01:27:17.692 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 11 01:27:17 smithi110 podman[76111]: 2024-09-11 01:27:17.524311744 +0000 UTC m=+0.522692691 container remove 7242af13f12682cb1a3cbd4e503eeb3ecbd6774f90460f19f038badaca9242d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T01:27:17.741 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.h.service' 2024-09-11T01:27:17.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:27:17.774 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-11T01:27:17.774 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-11T01:27:17.774 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.k 2024-09-11T01:27:18.095 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 01:27:17 smithi110 systemd[1]: Stopping Ceph mds.k for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:27:18.440 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 01:27:18 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-k[64690]: 2024-09-11T01:27:18.092+0000 7f2b40b32640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:27:18.440 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 01:27:18 smithi110 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-k[64690]: 2024-09-11T01:27:18.093+0000 7f2b40b32640 -1 mds.k *** got signal Terminated *** 2024-09-11T01:27:21.287 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 01:27:21 smithi110 podman[76217]: 2024-09-11 01:27:21.001804433 +0000 UTC m=+3.008831519 container died 10ae6b85e67a07f8c5393077d4afa2d8d8595e817bf94484d4dc07955621e950 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T01:27:21.662 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 01:27:21 smithi110 podman[76217]: 2024-09-11 01:27:21.360124636 +0000 UTC m=+3.367151721 container cleanup 10ae6b85e67a07f8c5393077d4afa2d8d8595e817bf94484d4dc07955621e950 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-11T01:27:21.663 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 11 01:27:21 smithi110 bash[76217]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-k 2024-09-11T01:27:21.788 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.k.service' 2024-09-11T01:27:21.821 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:27:21.821 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-11T01:27:21.821 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-11T01:27:21.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.c 2024-09-11T01:27:22.119 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 01:27:21 smithi112 systemd[1]: Stopping Ceph mds.c for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:27:22.383 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 01:27:22 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c[66476]: 2024-09-11T01:27:22.116+0000 7f04d6ad6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:27:22.384 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 01:27:22 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c[66476]: 2024-09-11T01:27:22.116+0000 7f04d6ad6640 -1 mds.c *** got signal Terminated *** 2024-09-11T01:27:25.469 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 01:27:25 smithi112 podman[77105]: 2024-09-11 01:27:25.163728272 +0000 UTC m=+3.147636709 container stop 3af06f9e3ebf3423504e94d40d33cb46410215f91b66fe15319086565d3f4d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:27:25.469 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 01:27:25 smithi112 podman[77105]: 2024-09-11 01:27:25.189751947 +0000 UTC m=+3.173660371 container died 3af06f9e3ebf3423504e94d40d33cb46410215f91b66fe15319086565d3f4d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-11T01:27:25.724 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 01:27:25 smithi112 podman[77105]: 2024-09-11 01:27:25.567131902 +0000 UTC m=+3.551040326 container cleanup 3af06f9e3ebf3423504e94d40d33cb46410215f91b66fe15319086565d3f4d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T01:27:25.724 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 01:27:25 smithi112 bash[77105]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c 2024-09-11T01:27:25.989 INFO:journalctl@ceph.mds.c.smithi112.stdout:Sep 11 01:27:25 smithi112 podman[77132]: 2024-09-11 01:27:25.734888596 +0000 UTC m=+0.558839942 container remove 3af06f9e3ebf3423504e94d40d33cb46410215f91b66fe15319086565d3f4d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T01:27:25.993 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.c.service' 2024-09-11T01:27:26.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:27:26.028 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-11T01:27:26.029 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-11T01:27:26.029 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.f 2024-09-11T01:27:26.356 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 01:27:26 smithi112 systemd[1]: Stopping Ceph mds.f for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:27:26.724 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 01:27:26 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-f[66176]: 2024-09-11T01:27:26.353+0000 7fcc5649c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:27:26.724 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 01:27:26 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-f[66176]: 2024-09-11T01:27:26.353+0000 7fcc5649c640 -1 mds.f *** got signal Terminated *** 2024-09-11T01:27:29.281 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 01:27:29 smithi112 podman[77240]: 2024-09-11 01:27:29.016693532 +0000 UTC m=+2.773104118 container died 8aa434e7ed0f390dcd56f3f9e6d9de2480f111fee57af7a24fc7ae68ab0ec6b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T01:27:29.648 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 01:27:29 smithi112 podman[77240]: 2024-09-11 01:27:29.379177276 +0000 UTC m=+3.135587858 container cleanup 8aa434e7ed0f390dcd56f3f9e6d9de2480f111fee57af7a24fc7ae68ab0ec6b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux ) 2024-09-11T01:27:29.648 INFO:journalctl@ceph.mds.f.smithi112.stdout:Sep 11 01:27:29 smithi112 bash[77240]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-f 2024-09-11T01:27:29.811 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.f.service' 2024-09-11T01:27:29.845 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:27:29.846 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-11T01:27:29.846 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-11T01:27:29.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.i 2024-09-11T01:27:30.167 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 01:27:29 smithi112 systemd[1]: Stopping Ceph mds.i for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:27:30.474 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 01:27:30 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-i[66790]: 2024-09-11T01:27:30.166+0000 7f1c4fabb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:27:30.474 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 01:27:30 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-i[66790]: 2024-09-11T01:27:30.166+0000 7f1c4fabb640 -1 mds.i *** got signal Terminated *** 2024-09-11T01:27:33.224 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 01:27:33 smithi112 podman[77362]: 2024-09-11 01:27:33.016482858 +0000 UTC m=+2.956512704 container died be7bba0306b443a94351301c676f99124e0617d9c6412b5d7d80e870f8495a79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-i, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T01:27:33.724 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 01:27:33 smithi112 podman[77362]: 2024-09-11 01:27:33.516218053 +0000 UTC m=+3.456247898 container cleanup be7bba0306b443a94351301c676f99124e0617d9c6412b5d7d80e870f8495a79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-i, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T01:27:33.724 INFO:journalctl@ceph.mds.i.smithi112.stdout:Sep 11 01:27:33 smithi112 bash[77362]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-i 2024-09-11T01:27:33.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.i.service' 2024-09-11T01:27:33.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:27:33.977 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-11T01:27:33.977 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-11T01:27:33.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.l 2024-09-11T01:27:34.296 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:27:34 smithi112 systemd[1]: Stopping Ceph mds.l for 64384694-6fd7-11ef-bcea-c7b262605968... 2024-09-11T01:27:34.724 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:27:34 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l[65875]: 2024-09-11T01:27:34.294+0000 7f5785f77640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T01:27:34.724 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:27:34 smithi112 ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l[65875]: 2024-09-11T01:27:34.294+0000 7f5785f77640 -1 mds.l *** got signal Terminated *** 2024-09-11T01:27:37.283 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:27:37 smithi112 podman[77484]: 2024-09-11 01:27:37.020895673 +0000 UTC m=+2.828914587 container died 04aca3bc67a0e69d361b6fc7ce38d810234368f354c7220399bf97ade30eebfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T01:27:37.564 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:27:37 smithi112 podman[77484]: 2024-09-11 01:27:37.387107907 +0000 UTC m=+3.195126807 container cleanup 04aca3bc67a0e69d361b6fc7ce38d810234368f354c7220399bf97ade30eebfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-11T01:27:37.564 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:27:37 smithi112 bash[77484]: ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l 2024-09-11T01:27:37.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.l.service' 2024-09-11T01:27:37.825 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:27:37 smithi112 podman[77496]: 2024-09-11 01:27:37.563001774 +0000 UTC m=+0.532532575 container remove 04aca3bc67a0e69d361b6fc7ce38d810234368f354c7220399bf97ade30eebfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-64384694-6fd7-11ef-bcea-c7b262605968-mds-l, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T01:27:37.825 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:27:37 smithi112 systemd[1]: ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.l.service: Deactivated successfully. 2024-09-11T01:27:37.825 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:27:37 smithi112 systemd[1]: Stopped Ceph mds.l for 64384694-6fd7-11ef-bcea-c7b262605968. 2024-09-11T01:27:37.825 INFO:journalctl@ceph.mds.l.smithi112.stdout:Sep 11 01:27:37 smithi112 systemd[1]: ceph-64384694-6fd7-11ef-bcea-c7b262605968@mds.l.service: Consumed 38.541s CPU time. 2024-09-11T01:27:37.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T01:27:37.836 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-11T01:27:37.836 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 64384694-6fd7-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T01:27:38.038 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T01:27:40.597 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 64384694-6fd7-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T01:27:40.799 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T01:27:43.346 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 64384694-6fd7-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T01:27:43.550 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T01:27:46.010 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T01:27:46.044 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T01:27:46.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T01:27:46.105 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T01:27:46.107 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863/remote/smithi078/crash 2024-09-11T01:27:46.108 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/crash -- . 2024-09-11T01:27:46.149 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T01:27:46.150 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2024-09-11T01:27:46.151 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863/remote/smithi110/crash 2024-09-11T01:27:46.152 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/crash -- . 2024-09-11T01:27:46.190 INFO:teuthology.orchestra.run.smithi110.stderr:tar: /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T01:27:46.190 INFO:teuthology.orchestra.run.smithi110.stderr:tar: Error is not recoverable: exiting now 2024-09-11T01:27:46.192 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863/remote/smithi112/crash 2024-09-11T01:27:46.193 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/crash -- . 2024-09-11T01:27:46.225 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/64384694-6fd7-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T01:27:46.226 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-09-11T01:27:46.227 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T01:27:46.227 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-11T01:27:46.315 INFO:tasks.cephadm:Compressing logs... 2024-09-11T01:27:46.316 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T01:27:46.318 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T01:27:46.321 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T01:27:46.348 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:: No such file or directory 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.4.log: --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.log 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-11T01:27:46.349 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T01:27:46.350 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-11T01:27:46.351 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T01:27:46.351 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T01:27:46.353 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T01:27:46.353 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T01:27:46.353 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-11T01:27:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T01:27:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T01:27:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-11T01:27:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T01:27:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T01:27:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.log 2024-09-11T01:27:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-11T01:27:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-11T01:27:46.358 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-11T01:27:46.358 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-11T01:27:46.358 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-11T01:27:46.358 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T01:27:46.359 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T01:27:46.359 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T01:27:46.359 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-11T01:27:46.359 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T01:27:46.359 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T01:27:46.363 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-11T01:27:46.363 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T01:27:46.363 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T01:27:46.363 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-11T01:27:46.363 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T01:27:46.363 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.log 2024-09-11T01:27:46.363 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-11T01:27:46.363 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T01:27:46.363 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-11T01:27:46.363 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T01:27:46.364 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T01:27:46.365 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 87.6% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T01:27:46.411 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 82.3% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T01:27:46.463 INFO:teuthology.orchestra.run.smithi078.stderr: 95.1% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-11T01:27:46.511 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 93.2% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T01:27:46.528 INFO:teuthology.orchestra.run.smithi112.stderr: 90.4% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T01:27:46.557 INFO:teuthology.orchestra.run.smithi078.stderr: 90.3% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T01:27:46.567 INFO:teuthology.orchestra.run.smithi078.stderr: 84.6% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T01:27:46.606 INFO:teuthology.orchestra.run.smithi112.stderr: 93.3% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-11T01:27:46.610 INFO:teuthology.orchestra.run.smithi112.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T01:27:46.617 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T01:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr: 90.9% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T01:27:46.733 INFO:teuthology.orchestra.run.smithi110.stderr: 90.9% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T01:27:46.762 INFO:teuthology.orchestra.run.smithi078.stderr: 95.1% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-11T01:27:46.777 INFO:teuthology.orchestra.run.smithi078.stderr: 94.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T01:27:46.801 INFO:teuthology.orchestra.run.smithi078.stderr: 90.9% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T01:27:46.824 INFO:teuthology.orchestra.run.smithi110.stderr: 82.4% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T01:27:46.857 INFO:teuthology.orchestra.run.smithi110.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T01:27:46.860 INFO:teuthology.orchestra.run.smithi110.stderr: 90.4% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T01:27:48.007 INFO:teuthology.orchestra.run.smithi110.stderr: 87.6% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T01:27:51.973 INFO:teuthology.orchestra.run.smithi078.stderr: 90.7% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T01:27:59.590 INFO:teuthology.orchestra.run.smithi112.stderr: 91.6% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T01:28:07.813 INFO:teuthology.orchestra.run.smithi078.stderr: 90.3% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T01:28:34.901 INFO:teuthology.orchestra.run.smithi078.stderr: 90.9% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-11T01:30:18.221 INFO:teuthology.orchestra.run.smithi078.stderr: 95.1% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-11T01:30:19.082 INFO:teuthology.orchestra.run.smithi110.stderr: 93.8% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T01:30:46.822 INFO:teuthology.orchestra.run.smithi078.stderr: 91.0% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T01:31:55.462 INFO:teuthology.orchestra.run.smithi078.stderr: 91.2% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T01:33:03.818 INFO:teuthology.orchestra.run.smithi078.stderr: 90.8% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T01:33:26.509 INFO:teuthology.orchestra.run.smithi110.stderr: 95.1% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-11T01:33:58.800 INFO:teuthology.orchestra.run.smithi112.stderr: 95.2% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-11T01:34:16.301 INFO:teuthology.orchestra.run.smithi112.stderr: 95.3% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-11T01:34:24.135 INFO:teuthology.orchestra.run.smithi110.stderr: 95.0% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-11T01:36:04.543 INFO:teuthology.orchestra.run.smithi110.stderr: 91.2% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T01:36:32.270 INFO:teuthology.orchestra.run.smithi078.stderr: 93.0% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-11T01:36:32.272 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-09-11T01:36:32.273 INFO:teuthology.orchestra.run.smithi078.stderr:real 8m45.940s 2024-09-11T01:36:32.273 INFO:teuthology.orchestra.run.smithi078.stderr:user 7m31.988s 2024-09-11T01:36:32.273 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m18.833s 2024-09-11T01:37:02.445 INFO:teuthology.orchestra.run.smithi110.stderr: 91.0% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T01:43:18.248 INFO:teuthology.orchestra.run.smithi110.stderr: 90.9% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-11T01:43:18.869 INFO:teuthology.orchestra.run.smithi112.stderr: 91.0% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-11T01:44:13.809 INFO:teuthology.orchestra.run.smithi112.stderr: 90.8% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-11T01:44:21.184 INFO:teuthology.orchestra.run.smithi110.stderr: 90.9% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T01:44:26.139 INFO:teuthology.orchestra.run.smithi112.stderr: 90.9% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T01:47:49.701 INFO:teuthology.orchestra.run.smithi112.stderr: 90.7% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T02:05:33.447 INFO:teuthology.orchestra.run.smithi112.stderr: 93.0% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-11T02:05:35.200 INFO:teuthology.orchestra.run.smithi110.stderr: 93.0% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-11T02:06:11.355 INFO:teuthology.orchestra.run.smithi112.stderr: 92.9% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-11T02:06:11.374 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-11T02:06:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:real 38m25.039s 2024-09-11T02:06:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:user 13m15.225s 2024-09-11T02:06:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m37.090s 2024-09-11T02:06:40.027 INFO:teuthology.orchestra.run.smithi110.stderr: 93.1% -- replaced with /var/log/ceph/64384694-6fd7-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-11T02:06:40.029 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-09-11T02:06:40.029 INFO:teuthology.orchestra.run.smithi110.stderr:real 38m53.694s 2024-09-11T02:06:40.029 INFO:teuthology.orchestra.run.smithi110.stderr:user 12m47.776s 2024-09-11T02:06:40.029 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m37.426s 2024-09-11T02:06:40.030 INFO:tasks.cephadm:Archiving logs... 2024-09-11T02:06:40.032 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863/remote/smithi078/log 2024-09-11T02:06:40.033 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T02:07:55.636 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863/remote/smithi110/log 2024-09-11T02:07:55.638 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T02:09:57.098 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863/remote/smithi112/log 2024-09-11T02:09:57.099 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T02:12:02.720 INFO:tasks.cephadm:Removing cluster... 2024-09-11T02:12:02.721 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 64384694-6fd7-11ef-bcea-c7b262605968 --force 2024-09-11T02:12:03.050 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T02:12:03.943 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 64384694-6fd7-11ef-bcea-c7b262605968 --force 2024-09-11T02:12:04.246 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T02:12:05.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 64384694-6fd7-11ef-bcea-c7b262605968 --force 2024-09-11T02:12:05.762 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 64384694-6fd7-11ef-bcea-c7b262605968 2024-09-11T02:12:06.666 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T02:12:06.667 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T02:12:06.688 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T02:12:06.790 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T02:12:06.807 INFO:tasks.cephadm:Teardown complete 2024-09-11T02:12:06.807 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T02:12:06.821 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T02:12:06.822 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T02:12:06.825 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T02:12:06.827 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T02:12:07.034 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T02:12:07.045 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T02:12:07.045 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T02:12:07.048 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T02:12:07.050 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T02:12:07.062 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-09-11T02:12:07.065 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-09-11T02:12:07.065 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-11T02:12:07.081 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T02:12:07.081 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-11T02:12:07.081 INFO:teuthology.orchestra.run.smithi110.stdout:^* hv01.front.sepia.ceph.com 3 7 377 17 +175us[ +128us] +/- 56ms 2024-09-11T02:12:07.081 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 16 +728us[ +728us] +/- 57ms 2024-09-11T02:12:07.081 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 -1334us[-1334us] +/- 78ms 2024-09-11T02:12:07.082 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T02:12:07.094 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T02:12:07.094 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-09-11T02:12:07.095 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv01.front.sepia.ceph.com 3 7 377 15 +217us[ +167us] +/- 56ms 2024-09-11T02:12:07.095 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 +700us[ +700us] +/- 57ms 2024-09-11T02:12:07.095 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 -1322us[-1373us] +/- 78ms 2024-09-11T02:12:07.095 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T02:12:07.155 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T02:12:07.155 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-09-11T02:12:07.155 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 +166us[ +148us] +/- 56ms 2024-09-11T02:12:07.155 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 +715us[ +697us] +/- 57ms 2024-09-11T02:12:07.155 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 -1300us[-1318us] +/- 78ms 2024-09-11T02:12:07.155 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T02:12:07.167 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T02:12:07.177 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T02:12:07.177 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T02:12:07.187 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T02:12:07.392 DEBUG:teuthology.orchestra.run.smithi110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T02:12:07.525 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T02:12:07.689 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T02:12:07.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T02:12:07.721 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T02:12:07.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T02:12:07.753 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T02:12:07.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T02:12:07.781 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T02:12:07.792 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T02:12:07.802 INFO:teuthology.task.internal:Duration was 5439.272444 seconds 2024-09-11T02:12:07.803 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T02:12:07.814 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T02:12:07.814 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T02:12:07.817 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T02:12:07.819 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T02:12:07.876 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T02:12:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T02:12:07.903 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T02:12:08.313 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T02:12:08.314 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T02:12:08.314 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T02:12:08.341 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-09-11T02:12:08.341 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T02:12:08.374 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-09-11T02:12:08.374 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T02:12:08.407 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T02:12:08.407 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T02:12:08.410 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T02:12:08.417 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T02:12:08.477 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T02:12:08.477 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T02:12:08.480 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T02:12:08.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T02:12:10.379 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T02:12:10.391 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T02:12:10.391 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T02:12:10.417 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T02:12:10.448 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T02:12:10.553 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T02:12:10.564 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T02:12:10.567 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T02:12:10.570 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T02:12:10.607 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-09-11T02:12:10.610 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-09-11T02:12:10.735 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-09-11T02:12:10.768 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T02:12:10.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T02:12:10.783 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T02:12:10.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T02:12:10.800 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T02:12:10.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T02:12:10.817 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T02:12:10.828 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T02:12:10.830 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863/remote/smithi078 2024-09-11T02:12:10.830 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T02:12:10.904 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863/remote/smithi110 2024-09-11T02:12:10.904 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T02:12:10.946 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899863/remote/smithi112 2024-09-11T02:12:10.946 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T02:12:10.992 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T02:12:10.992 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T02:12:10.995 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T02:12:10.998 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T02:12:11.045 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T02:12:11.060 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T02:12:11.060 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T02:12:11.089 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T02:12:11.089 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T02:12:11.091 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T02:12:11.094 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T02:12:11.104 INFO:teuthology.orchestra.run.smithi078.stdout: 265997 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 02:12 /home/ubuntu/cephtest 2024-09-11T02:12:11.109 INFO:teuthology.orchestra.run.smithi110.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 02:12 /home/ubuntu/cephtest 2024-09-11T02:12:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 02:12 /home/ubuntu/cephtest 2024-09-11T02:12:11.112 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T02:12:11.124 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T02:12:11.151 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} duration: 5439.272444486618 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-11T02:12:11.151 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T02:12:11.248 INFO:teuthology.run:pass